From 0a923be2f6427fdf2a5769b4b8a08e26c758dfab Mon Sep 17 00:00:00 2001
From: "Jason A. Donenfeld" <Jason@zx2c4.com>
Date: Thu, 28 Apr 2022 12:35:16 +0200
Subject: [PATCH 001/935] hw/openrisc: page-align FDT address

The QEMU-provided FDT was only being recognized by the kernel when it
was used in conjunction with -initrd. Without it, the magic bytes
wouldn't be there and the kernel couldn't load it. This patch fixes the
issue by page aligning the provided FDT.

Cc: Stafford Horne <shorne@gmail.com>
Cc: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Stafford Horne <shorne@gmail.com>
---
 hw/openrisc/openrisc_sim.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/openrisc/openrisc_sim.c b/hw/openrisc/openrisc_sim.c
index 8184caa60b..99b14940f4 100644
--- a/hw/openrisc/openrisc_sim.c
+++ b/hw/openrisc/openrisc_sim.c
@@ -356,7 +356,7 @@ static uint32_t openrisc_load_fdt(Or1ksimState *state, hwaddr load_start,
     }
 
     /* We put fdt right after the kernel and/or initrd. */
-    fdt_addr = ROUND_UP(load_start, 4);
+    fdt_addr = TARGET_PAGE_ALIGN(load_start);
 
     ret = fdt_pack(fdt);
     /* Should only fail if we've built a corrupted tree */

From 8e28b65f52637917d3d02ade4e0ae85442de9f86 Mon Sep 17 00:00:00 2001
From: "Denis V. Lunev" <den@openvz.org>
Date: Thu, 7 Apr 2022 11:39:32 +0300
Subject: [PATCH 002/935] qemu-img: properly list formats which have
 consistency check implemented

Simple grep for the .bdrv_co_check callback presence gives the following
list of block drivers
* QED
* VDI
* VHDX
* VMDK
* Parallels
which have this callback. The presense of the callback means that
consistency check is supported.

The patch updates documentation accordingly.

Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Kevin Wolf <kwolf@redhat.com>
CC: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220407083932.531965-1-den@openvz.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 docs/tools/qemu-img.rst | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst
index 8885ea11cf..85a6e05b35 100644
--- a/docs/tools/qemu-img.rst
+++ b/docs/tools/qemu-img.rst
@@ -332,8 +332,8 @@ Command description:
   ``-r all`` fixes all kinds of errors, with a higher risk of choosing the
   wrong fix or hiding corruption that has already occurred.
 
-  Only the formats ``qcow2``, ``qed`` and ``vdi`` support
-  consistency checks.
+  Only the formats ``qcow2``, ``qed``, ``parallels``, ``vhdx``, ``vmdk`` and
+  ``vdi`` support consistency checks.
 
   In case the image does not have any inconsistencies, check exits with ``0``.
   Other exit codes indicate the kind of inconsistency found or if another error

From 31009d13cc505b9569971fbbe50266c1727ba5db Mon Sep 17 00:00:00 2001
From: Kevin Wolf <kwolf@redhat.com>
Date: Thu, 7 Apr 2022 15:36:55 +0200
Subject: [PATCH 003/935] docs/vhost-user: Clarifications for
 VHOST_USER_ADD/REM_MEM_REG

The specification for VHOST_USER_ADD/REM_MEM_REG messages is unclear
in several points, which has led to clients having incompatible
implementations. This changes the specification to be more explicit
about them:

* VHOST_USER_ADD_MEM_REG is not specified as receiving a file
  descriptor, though it obviously does need to do so. All
  implementations agree on this one, fix the specification.

* VHOST_USER_REM_MEM_REG is not specified as receiving a file
  descriptor either, and it also has no reason to do so. rust-vmm does
  not send file descriptors for removing a memory region (in agreement
  with the specification), libvhost-user and QEMU do (which is a bug),
  though libvhost-user doesn't actually make any use of it.

  Change the specification so that for compatibility QEMU's behaviour
  becomes legal, even if discouraged, but rust-vmm's behaviour becomes
  the explicitly recommended mode of operation.

* VHOST_USER_ADD_MEM_REG doesn't have a documented return value, which
  is the desired behaviour in the non-postcopy case. It also implemented
  like this in QEMU and rust-vmm, though libvhost-user is buggy and
  sometimes sends an unexpected reply. This will be fixed in a separate
  patch.

  However, in postcopy mode it does reply like VHOST_USER_SET_MEM_TABLE.
  This behaviour is shared between libvhost-user and QEMU; rust-vmm
  doesn't implement postcopy mode yet. Mention it explicitly in the
  spec.

* The specification doesn't mention how VHOST_USER_REM_MEM_REG
  identifies the memory region to be removed. Change it to describe the
  existing behaviour of libvhost-user (guest address, user address and
  size must match).

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20220407133657.155281-2-kwolf@redhat.com>
Reviewed-by: Raphael Norwitz <raphael.norwitz@nutanix.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 docs/interop/vhost-user.rst | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
index 4dbc84fd00..f9e721ba5f 100644
--- a/docs/interop/vhost-user.rst
+++ b/docs/interop/vhost-user.rst
@@ -308,6 +308,7 @@ replies. Here is a list of the ones that do:
 There are several messages that the master sends with file descriptors passed
 in the ancillary data:
 
+* ``VHOST_USER_ADD_MEM_REG``
 * ``VHOST_USER_SET_MEM_TABLE``
 * ``VHOST_USER_SET_LOG_BASE`` (if ``VHOST_USER_PROTOCOL_F_LOG_SHMFD``)
 * ``VHOST_USER_SET_LOG_FD``
@@ -1334,6 +1335,14 @@ Master message types
   ``VHOST_USER_REM_MEM_REG`` message, this message is used to set and
   update the memory tables of the slave device.
 
+  Exactly one file descriptor from which the memory is mapped is
+  passed in the ancillary data.
+
+  In postcopy mode (see ``VHOST_USER_POSTCOPY_LISTEN``), the slave
+  replies with the bases of the memory mapped region to the master.
+  For further details on postcopy, see ``VHOST_USER_SET_MEM_TABLE``.
+  They apply to ``VHOST_USER_ADD_MEM_REG`` accordingly.
+
 ``VHOST_USER_REM_MEM_REG``
   :id: 38
   :equivalent ioctl: N/A
@@ -1349,6 +1358,14 @@ Master message types
   ``VHOST_USER_ADD_MEM_REG`` message, this message is used to set and
   update the memory tables of the slave device.
 
+  The memory region to be removed is identified by its guest address,
+  user address and size. The mmap offset is ignored.
+
+  No file descriptors SHOULD be passed in the ancillary data. For
+  compatibility with existing incorrect implementations, the slave MAY
+  accept messages with one file descriptor. If a file descriptor is
+  passed, the slave MUST close it without using it otherwise.
+
 ``VHOST_USER_SET_STATUS``
   :id: 39
   :equivalent ioctl: VHOST_VDPA_SET_STATUS

From 5ebfdeb21223e0406a711decbb2baceb5ec8c416 Mon Sep 17 00:00:00 2001
From: Kevin Wolf <kwolf@redhat.com>
Date: Thu, 7 Apr 2022 15:36:56 +0200
Subject: [PATCH 004/935] libvhost-user: Fix extra vu_add/rem_mem_reg reply

Outside of postcopy mode, neither VHOST_USER_ADD_MEM_REG nor
VHOST_USER_REM_MEM_REG are supposed to send a reply unless explicitly
requested with the need_reply flag. Their current implementation always
sends a reply, even if it isn't requested. This confuses the master
because it will interpret the reply as a reply for the next message for
which it actually expects a reply.

need_reply is already handled correctly by vu_dispatch(), so just don't
send a reply in the non-postcopy part of the message handler for these
two commands.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20220407133657.155281-3-kwolf@redhat.com>
Reviewed-by: Raphael Norwitz <raphael.norwitz@nutanix.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 subprojects/libvhost-user/libvhost-user.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/subprojects/libvhost-user/libvhost-user.c b/subprojects/libvhost-user/libvhost-user.c
index 47d2efc60f..eccaff5168 100644
--- a/subprojects/libvhost-user/libvhost-user.c
+++ b/subprojects/libvhost-user/libvhost-user.c
@@ -800,8 +800,7 @@ vu_add_mem_reg(VuDev *dev, VhostUserMsg *vmsg) {
 
         DPRINT("Successfully added new region\n");
         dev->nregions++;
-        vmsg_set_reply_u64(vmsg, 0);
-        return true;
+        return false;
     }
 }
 
@@ -874,15 +873,13 @@ vu_rem_mem_reg(VuDev *dev, VhostUserMsg *vmsg) {
         }
     }
 
-    if (found) {
-        vmsg_set_reply_u64(vmsg, 0);
-    } else {
+    if (!found) {
         vu_panic(dev, "Specified region not found\n");
     }
 
     close(vmsg->fds[0]);
 
-    return true;
+    return false;
 }
 
 static bool

From a81d8d4a7203e4be392f9ad37af104759b7cbda5 Mon Sep 17 00:00:00 2001
From: Kevin Wolf <kwolf@redhat.com>
Date: Thu, 7 Apr 2022 15:36:57 +0200
Subject: [PATCH 005/935] vhost-user: Don't pass file descriptor for
 VHOST_USER_REM_MEM_REG

The spec clarifies now that QEMU should not send a file descriptor in a
request to remove a memory region. Change it accordingly.

For libvhost-user, this is a bug fix that makes it compatible with
rust-vmm's implementation that doesn't send a file descriptor. Keep
accepting, but ignoring a file descriptor for compatibility with older
QEMU versions.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20220407133657.155281-4-kwolf@redhat.com>
Reviewed-by: Raphael Norwitz <raphael.norwitz@nutanix.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 hw/virtio/vhost-user.c                    | 2 +-
 subprojects/libvhost-user/libvhost-user.c | 8 ++++----
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index a80315ecfc..2d434ff0bc 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -751,7 +751,7 @@ static int send_remove_regions(struct vhost_dev *dev,
             vhost_user_fill_msg_region(&region_buffer, shadow_reg, 0);
             msg->payload.mem_reg.region = region_buffer;
 
-            ret = vhost_user_write(dev, msg, &fd, 1);
+            ret = vhost_user_write(dev, msg, NULL, 0);
             if (ret < 0) {
                 return ret;
             }
diff --git a/subprojects/libvhost-user/libvhost-user.c b/subprojects/libvhost-user/libvhost-user.c
index eccaff5168..d0041c864b 100644
--- a/subprojects/libvhost-user/libvhost-user.c
+++ b/subprojects/libvhost-user/libvhost-user.c
@@ -822,15 +822,15 @@ vu_rem_mem_reg(VuDev *dev, VhostUserMsg *vmsg) {
     int i;
     bool found = false;
 
-    if (vmsg->fd_num != 1) {
+    if (vmsg->fd_num > 1) {
         vmsg_close_fds(vmsg);
-        vu_panic(dev, "VHOST_USER_REM_MEM_REG received %d fds - only 1 fd "
+        vu_panic(dev, "VHOST_USER_REM_MEM_REG received %d fds - at most 1 fd "
                       "should be sent for this message type", vmsg->fd_num);
         return false;
     }
 
     if (vmsg->size < VHOST_USER_MEM_REG_SIZE) {
-        close(vmsg->fds[0]);
+        vmsg_close_fds(vmsg);
         vu_panic(dev, "VHOST_USER_REM_MEM_REG requires a message size of at "
                       "least %d bytes and only %d bytes were received",
                       VHOST_USER_MEM_REG_SIZE, vmsg->size);
@@ -877,7 +877,7 @@ vu_rem_mem_reg(VuDev *dev, VhostUserMsg *vmsg) {
         vu_panic(dev, "Specified region not found\n");
     }
 
-    close(vmsg->fds[0]);
+    vmsg_close_fds(vmsg);
 
     return false;
 }

From 15aee7ac9587562b1d021959dedc2bbb7e9a9fb3 Mon Sep 17 00:00:00 2001
From: Hanna Reitz <hreitz@redhat.com>
Date: Wed, 27 Apr 2022 13:40:54 +0200
Subject: [PATCH 006/935] block: Classify bdrv_get_flags() as I/O function

This function is safe to call in an I/O context, and qcow2_do_open()
does so (invoked in an I/O context by qcow2_co_invalidate_cache()).

Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220427114057.36651-2-hreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 block.c                            | 2 +-
 include/block/block-global-state.h | 1 -
 include/block/block-io.h           | 1 +
 3 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/block.c b/block.c
index 8cd16e757e..2c00dddd80 100644
--- a/block.c
+++ b/block.c
@@ -6298,7 +6298,7 @@ const char *bdrv_get_device_or_node_name(const BlockDriverState *bs)
 
 int bdrv_get_flags(BlockDriverState *bs)
 {
-    GLOBAL_STATE_CODE();
+    IO_CODE();
     return bs->open_flags;
 }
 
diff --git a/include/block/block-global-state.h b/include/block/block-global-state.h
index 25bb69bbef..21265e3966 100644
--- a/include/block/block-global-state.h
+++ b/include/block/block-global-state.h
@@ -172,7 +172,6 @@ void bdrv_next_cleanup(BdrvNextIterator *it);
 BlockDriverState *bdrv_next_monitor_owned(BlockDriverState *bs);
 void bdrv_iterate_format(void (*it)(void *opaque, const char *name),
                          void *opaque, bool read_only);
-int bdrv_get_flags(BlockDriverState *bs);
 char *bdrv_get_full_backing_filename(BlockDriverState *bs, Error **errp);
 char *bdrv_dirname(BlockDriverState *bs, Error **errp);
 
diff --git a/include/block/block-io.h b/include/block/block-io.h
index 5e3f346806..62c84f0519 100644
--- a/include/block/block-io.h
+++ b/include/block/block-io.h
@@ -103,6 +103,7 @@ int bdrv_apply_auto_read_only(BlockDriverState *bs, const char *errmsg,
 bool bdrv_is_read_only(BlockDriverState *bs);
 bool bdrv_is_writable(BlockDriverState *bs);
 bool bdrv_is_sg(BlockDriverState *bs);
+int bdrv_get_flags(BlockDriverState *bs);
 bool bdrv_is_inserted(BlockDriverState *bs);
 void bdrv_lock_medium(BlockDriverState *bs, bool locked);
 void bdrv_eject(BlockDriverState *bs, bool eject_flag);

From 06e9cd19a4e9acf8d24327361a999943d846f607 Mon Sep 17 00:00:00 2001
From: Hanna Reitz <hreitz@redhat.com>
Date: Wed, 27 Apr 2022 13:40:55 +0200
Subject: [PATCH 007/935] qcow2: Do not reopen data_file in invalidate_cache

qcow2_co_invalidate_cache() closes and opens the qcow2 file, by calling
qcow2_close() and qcow2_do_open().  These two functions must thus be
usable from both a global-state and an I/O context.

As they are, they are not safe to call in an I/O context, because they
use bdrv_unref_child() and bdrv_open_child() to close/open the data_file
child, respectively, both of which are global-state functions.  When
used from qcow2_co_invalidate_cache(), we do not need to close/open the
data_file child, though (we do not do this for bs->file or bs->backing
either), and so we should skip it in the qcow2_co_invalidate_cache()
path.

To do so, add a parameter to qcow2_do_open() and qcow2_close() to make
them skip handling s->data_file, and have qcow2_co_invalidate_cache()
exempt it from the memset() on the BDRVQcow2State.

(Note that the QED driver similarly closes/opens the QED image by
invoking bdrv_qed_close()+bdrv_qed_do_open(), but both functions seem
safe to use in an I/O context.)

Fixes: https://gitlab.com/qemu-project/qemu/-/issues/945
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220427114057.36651-3-hreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 block/qcow2.c | 104 ++++++++++++++++++++++++++++++--------------------
 1 file changed, 62 insertions(+), 42 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index b5c47931ef..4f5e6440fb 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1296,7 +1296,8 @@ static int validate_compression_type(BDRVQcow2State *s, Error **errp)
 
 /* Called with s->lock held.  */
 static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options,
-                                      int flags, Error **errp)
+                                      int flags, bool open_data_file,
+                                      Error **errp)
 {
     ERRP_GUARD();
     BDRVQcow2State *s = bs->opaque;
@@ -1614,50 +1615,52 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options,
         goto fail;
     }
 
-    /* Open external data file */
-    s->data_file = bdrv_open_child(NULL, options, "data-file", bs,
-                                   &child_of_bds, BDRV_CHILD_DATA,
-                                   true, errp);
-    if (*errp) {
-        ret = -EINVAL;
-        goto fail;
-    }
+    if (open_data_file) {
+        /* Open external data file */
+        s->data_file = bdrv_open_child(NULL, options, "data-file", bs,
+                                       &child_of_bds, BDRV_CHILD_DATA,
+                                       true, errp);
+        if (*errp) {
+            ret = -EINVAL;
+            goto fail;
+        }
 
-    if (s->incompatible_features & QCOW2_INCOMPAT_DATA_FILE) {
-        if (!s->data_file && s->image_data_file) {
-            s->data_file = bdrv_open_child(s->image_data_file, options,
-                                           "data-file", bs, &child_of_bds,
-                                           BDRV_CHILD_DATA, false, errp);
+        if (s->incompatible_features & QCOW2_INCOMPAT_DATA_FILE) {
+            if (!s->data_file && s->image_data_file) {
+                s->data_file = bdrv_open_child(s->image_data_file, options,
+                                               "data-file", bs, &child_of_bds,
+                                               BDRV_CHILD_DATA, false, errp);
+                if (!s->data_file) {
+                    ret = -EINVAL;
+                    goto fail;
+                }
+            }
             if (!s->data_file) {
+                error_setg(errp, "'data-file' is required for this image");
                 ret = -EINVAL;
                 goto fail;
             }
-        }
-        if (!s->data_file) {
-            error_setg(errp, "'data-file' is required for this image");
-            ret = -EINVAL;
-            goto fail;
-        }
 
-        /* No data here */
-        bs->file->role &= ~BDRV_CHILD_DATA;
+            /* No data here */
+            bs->file->role &= ~BDRV_CHILD_DATA;
 
-        /* Must succeed because we have given up permissions if anything */
-        bdrv_child_refresh_perms(bs, bs->file, &error_abort);
-    } else {
-        if (s->data_file) {
-            error_setg(errp, "'data-file' can only be set for images with an "
-                             "external data file");
-            ret = -EINVAL;
-            goto fail;
-        }
+            /* Must succeed because we have given up permissions if anything */
+            bdrv_child_refresh_perms(bs, bs->file, &error_abort);
+        } else {
+            if (s->data_file) {
+                error_setg(errp, "'data-file' can only be set for images with "
+                                 "an external data file");
+                ret = -EINVAL;
+                goto fail;
+            }
 
-        s->data_file = bs->file;
+            s->data_file = bs->file;
 
-        if (data_file_is_raw(bs)) {
-            error_setg(errp, "data-file-raw requires a data file");
-            ret = -EINVAL;
-            goto fail;
+            if (data_file_is_raw(bs)) {
+                error_setg(errp, "data-file-raw requires a data file");
+                ret = -EINVAL;
+                goto fail;
+            }
         }
     }
 
@@ -1839,7 +1842,7 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options,
 
  fail:
     g_free(s->image_data_file);
-    if (has_data_file(bs)) {
+    if (open_data_file && has_data_file(bs)) {
         bdrv_unref_child(bs, s->data_file);
         s->data_file = NULL;
     }
@@ -1876,7 +1879,8 @@ static void coroutine_fn qcow2_open_entry(void *opaque)
     BDRVQcow2State *s = qoc->bs->opaque;
 
     qemu_co_mutex_lock(&s->lock);
-    qoc->ret = qcow2_do_open(qoc->bs, qoc->options, qoc->flags, qoc->errp);
+    qoc->ret = qcow2_do_open(qoc->bs, qoc->options, qoc->flags, true,
+                             qoc->errp);
     qemu_co_mutex_unlock(&s->lock);
 }
 
@@ -2714,7 +2718,7 @@ static int qcow2_inactivate(BlockDriverState *bs)
     return result;
 }
 
-static void qcow2_close(BlockDriverState *bs)
+static void qcow2_do_close(BlockDriverState *bs, bool close_data_file)
 {
     BDRVQcow2State *s = bs->opaque;
     qemu_vfree(s->l1_table);
@@ -2740,7 +2744,7 @@ static void qcow2_close(BlockDriverState *bs)
     g_free(s->image_backing_file);
     g_free(s->image_backing_format);
 
-    if (has_data_file(bs)) {
+    if (close_data_file && has_data_file(bs)) {
         bdrv_unref_child(bs, s->data_file);
         s->data_file = NULL;
     }
@@ -2749,11 +2753,17 @@ static void qcow2_close(BlockDriverState *bs)
     qcow2_free_snapshots(bs);
 }
 
+static void qcow2_close(BlockDriverState *bs)
+{
+    qcow2_do_close(bs, true);
+}
+
 static void coroutine_fn qcow2_co_invalidate_cache(BlockDriverState *bs,
                                                    Error **errp)
 {
     ERRP_GUARD();
     BDRVQcow2State *s = bs->opaque;
+    BdrvChild *data_file;
     int flags = s->flags;
     QCryptoBlock *crypto = NULL;
     QDict *options;
@@ -2767,14 +2777,24 @@ static void coroutine_fn qcow2_co_invalidate_cache(BlockDriverState *bs,
     crypto = s->crypto;
     s->crypto = NULL;
 
-    qcow2_close(bs);
+    /*
+     * Do not reopen s->data_file (i.e., have qcow2_do_close() not close it,
+     * and then prevent qcow2_do_open() from opening it), because this function
+     * runs in the I/O path and as such we must not invoke global-state
+     * functions like bdrv_unref_child() and bdrv_open_child().
+     */
 
+    qcow2_do_close(bs, false);
+
+    data_file = s->data_file;
     memset(s, 0, sizeof(BDRVQcow2State));
+    s->data_file = data_file;
+
     options = qdict_clone_shallow(bs->options);
 
     flags &= ~BDRV_O_INACTIVE;
     qemu_co_mutex_lock(&s->lock);
-    ret = qcow2_do_open(bs, options, flags, errp);
+    ret = qcow2_do_open(bs, options, flags, false, errp);
     qemu_co_mutex_unlock(&s->lock);
     qobject_unref(options);
     if (ret < 0) {

From ccfaf783c8eecae99b671e4e892954c9226ef5dc Mon Sep 17 00:00:00 2001
From: Hanna Reitz <hreitz@redhat.com>
Date: Wed, 27 Apr 2022 13:40:56 +0200
Subject: [PATCH 008/935] Revert "main-loop: Disable GLOBAL_STATE_CODE()
 assertions"

This reverts commit b1c073490553f80594b903ceedfc7c1aef6b1b19.  (We
wanted to do so once the 7.1 tree opens, which has happened.  The issue
reported in https://gitlab.com/qemu-project/qemu/-/issues/945 should be
fixed by the preceding patches.)

Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220427114057.36651-4-hreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 include/qemu/main-loop.h | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/include/qemu/main-loop.h b/include/qemu/main-loop.h
index d3750c8e76..89bd9edefb 100644
--- a/include/qemu/main-loop.h
+++ b/include/qemu/main-loop.h
@@ -284,8 +284,7 @@ bool qemu_in_main_thread(void);
 #else
 #define GLOBAL_STATE_CODE()                                         \
     do {                                                            \
-        /* FIXME: Re-enable after 7.0 release */                    \
-        /* assert(qemu_in_main_thread()); */                        \
+        assert(qemu_in_main_thread());                              \
     } while (0)
 #endif /* CONFIG_COCOA */
 

From 96edda8f142928c627f93b3ad74c64edf86cc22f Mon Sep 17 00:00:00 2001
From: Hanna Reitz <hreitz@redhat.com>
Date: Wed, 27 Apr 2022 13:40:57 +0200
Subject: [PATCH 009/935] iotests: Add regression test for issue 945

Create a VM with a BDS in an iothread, add -incoming defer to the
command line, and then export this BDS via NBD.  Doing so should not
fail an assertion.

Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220427114057.36651-5-hreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 .../tests/export-incoming-iothread            | 81 +++++++++++++++++++
 .../tests/export-incoming-iothread.out        |  5 ++
 2 files changed, 86 insertions(+)
 create mode 100755 tests/qemu-iotests/tests/export-incoming-iothread
 create mode 100644 tests/qemu-iotests/tests/export-incoming-iothread.out

diff --git a/tests/qemu-iotests/tests/export-incoming-iothread b/tests/qemu-iotests/tests/export-incoming-iothread
new file mode 100755
index 0000000000..7679e49103
--- /dev/null
+++ b/tests/qemu-iotests/tests/export-incoming-iothread
@@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# group: rw quick migration
+#
+# Regression test for issue 945:
+# https://gitlab.com/qemu-project/qemu/-/issues/945
+# Test adding an export on top of an iothread-ed block device while in
+# -incoming defer.
+#
+# Copyright (C) 2022 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import iotests
+from iotests import qemu_img_create
+
+
+image_size = 1 * 1024 * 1024
+test_img = os.path.join(iotests.test_dir, 'test.img')
+node_name = 'node0'
+iothread_id = 'iothr0'
+
+nbd_sock = os.path.join(iotests.sock_dir, 'nbd.sock')
+
+
+class TestExportIncomingIothread(iotests.QMPTestCase):
+    def setUp(self) -> None:
+        qemu_img_create('-f', iotests.imgfmt, test_img, str(image_size))
+
+        self.vm = iotests.VM()
+        self.vm.add_object(f'iothread,id={iothread_id}')
+        self.vm.add_blockdev((
+            f'driver={iotests.imgfmt}',
+            f'node-name={node_name}',
+            'file.driver=file',
+            f'file.filename={test_img}'
+        ))
+        self.vm.add_incoming('defer')
+        self.vm.launch()
+
+    def tearDown(self):
+        self.vm.shutdown()
+        os.remove(test_img)
+
+    def test_export_add(self):
+        result = self.vm.qmp('nbd-server-start', {
+            'addr': {
+                'type': 'unix',
+                'data': {
+                    'path': nbd_sock
+                }
+            }
+        })
+        self.assert_qmp(result, 'return', {})
+
+        # Regression test for issue 945: This should not fail an assertion
+        result = self.vm.qmp('block-export-add', {
+            'type': 'nbd',
+            'id': 'exp0',
+            'node-name': node_name,
+            'iothread': iothread_id
+        })
+        self.assert_qmp(result, 'return', {})
+
+
+if __name__ == '__main__':
+    iotests.main(supported_fmts=['generic'],
+                 unsupported_fmts=['luks'], # Would need a secret
+                 supported_protocols=['file'])
diff --git a/tests/qemu-iotests/tests/export-incoming-iothread.out b/tests/qemu-iotests/tests/export-incoming-iothread.out
new file mode 100644
index 0000000000..ae1213e6f8
--- /dev/null
+++ b/tests/qemu-iotests/tests/export-incoming-iothread.out
@@ -0,0 +1,5 @@
+.
+----------------------------------------------------------------------
+Ran 1 tests
+
+OK

From 6d17e2879854d7d0e623c06a9286085e97bf3545 Mon Sep 17 00:00:00 2001
From: Hanna Reitz <hreitz@redhat.com>
Date: Mon, 14 Mar 2022 17:27:18 +0100
Subject: [PATCH 010/935] block/vmdk: Fix reopening bs->file

VMDK disk data is stored in extents, which may or may not be separate
from bs->file.  VmdkExtent.file points to where they are stored.  Each
that is stored in bs->file will simply reuse the exact pointer value of
bs->file.

(That is why vmdk_free_extents() will unref VmdkExtent.file (e->file)
only if e->file != bs->file.)

Reopen operations can change bs->file (they will replace the whole
BdrvChild object, not just the BDS stored in that BdrvChild), and then
we will need to change all .file pointers of all such VmdkExtents to
point to the new BdrvChild.

In vmdk_reopen_prepare(), we have to check which VmdkExtents are
affected, and in vmdk_reopen_commit(), we can modify them.  We have to
split this because:
- The new BdrvChild is created only after prepare, so we can change
  VmdkExtent.file only in commit
- In commit, there no longer is any (valid) reference to the old
  BdrvChild object, so there would be nothing to compare VmdkExtent.file
  against to see whether it was equal to bs->file before reopening
  (There is BDRVReopenState.old_file_bs, but the old bs->file
  BdrvChild's .bs pointer will be NULL-ed when the new BdrvChild is
  created, and so we cannot compare VmdkExtent.file->bs against
  BDRVReopenState.old_file_bs)

Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220314162719.65384-2-hreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 block/vmdk.c | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 55 insertions(+), 1 deletion(-)

diff --git a/block/vmdk.c b/block/vmdk.c
index 37c0946066..38e5ab3806 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -178,6 +178,10 @@ typedef struct BDRVVmdkState {
     char *create_type;
 } BDRVVmdkState;
 
+typedef struct BDRVVmdkReopenState {
+    bool *extents_using_bs_file;
+} BDRVVmdkReopenState;
+
 typedef struct VmdkMetaData {
     unsigned int l1_index;
     unsigned int l2_index;
@@ -400,15 +404,63 @@ static int vmdk_is_cid_valid(BlockDriverState *bs)
     return 1;
 }
 
-/* We have nothing to do for VMDK reopen, stubs just return success */
 static int vmdk_reopen_prepare(BDRVReopenState *state,
                                BlockReopenQueue *queue, Error **errp)
 {
+    BDRVVmdkState *s;
+    BDRVVmdkReopenState *rs;
+    int i;
+
     assert(state != NULL);
     assert(state->bs != NULL);
+    assert(state->opaque == NULL);
+
+    s = state->bs->opaque;
+
+    rs = g_new0(BDRVVmdkReopenState, 1);
+    state->opaque = rs;
+
+    /*
+     * Check whether there are any extents stored in bs->file; if bs->file
+     * changes, we will need to update their .file pointers to follow suit
+     */
+    rs->extents_using_bs_file = g_new(bool, s->num_extents);
+    for (i = 0; i < s->num_extents; i++) {
+        rs->extents_using_bs_file[i] = s->extents[i].file == state->bs->file;
+    }
+
     return 0;
 }
 
+static void vmdk_reopen_clean(BDRVReopenState *state)
+{
+    BDRVVmdkReopenState *rs = state->opaque;
+
+    g_free(rs->extents_using_bs_file);
+    g_free(rs);
+    state->opaque = NULL;
+}
+
+static void vmdk_reopen_commit(BDRVReopenState *state)
+{
+    BDRVVmdkState *s = state->bs->opaque;
+    BDRVVmdkReopenState *rs = state->opaque;
+    int i;
+
+    for (i = 0; i < s->num_extents; i++) {
+        if (rs->extents_using_bs_file[i]) {
+            s->extents[i].file = state->bs->file;
+        }
+    }
+
+    vmdk_reopen_clean(state);
+}
+
+static void vmdk_reopen_abort(BDRVReopenState *state)
+{
+    vmdk_reopen_clean(state);
+}
+
 static int vmdk_parent_open(BlockDriverState *bs)
 {
     char *p_name;
@@ -3072,6 +3124,8 @@ static BlockDriver bdrv_vmdk = {
     .bdrv_open                    = vmdk_open,
     .bdrv_co_check                = vmdk_co_check,
     .bdrv_reopen_prepare          = vmdk_reopen_prepare,
+    .bdrv_reopen_commit           = vmdk_reopen_commit,
+    .bdrv_reopen_abort            = vmdk_reopen_abort,
     .bdrv_child_perm              = bdrv_default_perms,
     .bdrv_co_preadv               = vmdk_co_preadv,
     .bdrv_co_pwritev              = vmdk_co_pwritev,

From ecf3200703df77f9f5ac95e3cc8e1379e40cb36a Mon Sep 17 00:00:00 2001
From: Hanna Reitz <hreitz@redhat.com>
Date: Mon, 14 Mar 2022 17:27:19 +0100
Subject: [PATCH 011/935] iotests/reopen-file: Test reopening file child

This should work for all format drivers that support reopening, so test
it.

(This serves as a regression test for HEAD^: This test used to fail for
VMDK before HEAD^.)

Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220314162719.65384-3-hreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 tests/qemu-iotests/tests/reopen-file     | 89 ++++++++++++++++++++++++
 tests/qemu-iotests/tests/reopen-file.out |  5 ++
 2 files changed, 94 insertions(+)
 create mode 100755 tests/qemu-iotests/tests/reopen-file
 create mode 100644 tests/qemu-iotests/tests/reopen-file.out

diff --git a/tests/qemu-iotests/tests/reopen-file b/tests/qemu-iotests/tests/reopen-file
new file mode 100755
index 0000000000..8590a94d53
--- /dev/null
+++ b/tests/qemu-iotests/tests/reopen-file
@@ -0,0 +1,89 @@
+#!/usr/bin/env python3
+# group: rw quick
+#
+# Test reopening a format driver's file child
+#
+# Copyright (C) 2022 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import iotests
+from iotests import imgfmt, qemu_img_create, QMPTestCase
+
+
+image_size = 1 * 1024 * 1024
+test_img = os.path.join(iotests.test_dir, 'test.img')
+
+
+class TestReopenFile(QMPTestCase):
+    def setUp(self) -> None:
+        res = qemu_img_create('-f', imgfmt, test_img, str(image_size))
+        assert res.returncode == 0
+
+        # Add format driver node ('format') on top of the file ('file'), then
+        # add another raw node ('raw') on top of 'file' so for the reopen we
+        # can just switch from 'file' to 'raw'
+        self.vm = iotests.VM()
+        self.vm.add_blockdev(self.vm.qmp_to_opts({
+            'driver': imgfmt,
+            'node-name': 'format',
+            'file': {
+                'driver': 'file',
+                'node-name': 'file',
+                'filename': test_img
+            }
+        }))
+        self.vm.add_blockdev(self.vm.qmp_to_opts({
+            'driver': 'raw',
+            'node-name': 'raw',
+            'file': 'file'
+        }))
+        self.vm.launch()
+
+    def tearDown(self) -> None:
+        self.vm.shutdown()
+        os.remove(test_img)
+
+        # Check if there was any qemu-io run that failed
+        if 'Pattern verification failed' in self.vm.get_log():
+            print('ERROR: Pattern verification failed:')
+            print(self.vm.get_log())
+            self.fail('qemu-io pattern verification failed')
+
+    def test_reopen_file(self) -> None:
+        result = self.vm.qmp('blockdev-reopen', options=[{
+            'driver': imgfmt,
+            'node-name': 'format',
+            'file': 'raw'
+        }])
+        self.assert_qmp(result, 'return', {})
+
+        # Do some I/O to the image to see whether it still works
+        # (Pattern verification will be checked by tearDown())
+        result = self.vm.qmp('human-monitor-command',
+                             command_line='qemu-io format "write -P 42 0 64k"')
+        self.assert_qmp(result, 'return', '')
+
+        result = self.vm.qmp('human-monitor-command',
+                             command_line='qemu-io format "read -P 42 0 64k"')
+        self.assert_qmp(result, 'return', '')
+
+
+if __name__ == '__main__':
+    # Must support creating images and reopen
+    iotests.main(supported_fmts=['qcow', 'qcow2', 'qed', 'raw', 'vdi', 'vhdx',
+                                 'vmdk', 'vpc'],
+                 supported_protocols=['file'])
diff --git a/tests/qemu-iotests/tests/reopen-file.out b/tests/qemu-iotests/tests/reopen-file.out
new file mode 100644
index 0000000000..ae1213e6f8
--- /dev/null
+++ b/tests/qemu-iotests/tests/reopen-file.out
@@ -0,0 +1,5 @@
+.
+----------------------------------------------------------------------
+Ran 1 tests
+
+OK

From 34145a307d849d0b6734d0222a7aa0bb9eef7407 Mon Sep 17 00:00:00 2001
From: Stefan Hajnoczi <stefanha@redhat.com>
Date: Mon, 7 Mar 2022 15:38:51 +0000
Subject: [PATCH 012/935] coroutine-ucontext: use QEMU_DEFINE_STATIC_CO_TLS()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Thread-Local Storage variables cannot be used directly from coroutine
code because the compiler may optimize TLS variable accesses across
qemu_coroutine_yield() calls. When the coroutine is re-entered from
another thread the TLS variables from the old thread must no longer be
used.

Use QEMU_DEFINE_STATIC_CO_TLS() for the current and leader variables.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220307153853.602859-2-stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 util/coroutine-ucontext.c | 38 ++++++++++++++++++++++++--------------
 1 file changed, 24 insertions(+), 14 deletions(-)

diff --git a/util/coroutine-ucontext.c b/util/coroutine-ucontext.c
index ed368e1a3e..ddc98fb4f8 100644
--- a/util/coroutine-ucontext.c
+++ b/util/coroutine-ucontext.c
@@ -25,6 +25,7 @@
 #include "qemu/osdep.h"
 #include <ucontext.h>
 #include "qemu/coroutine_int.h"
+#include "qemu/coroutine-tls.h"
 
 #ifdef CONFIG_VALGRIND_H
 #include <valgrind/valgrind.h>
@@ -66,8 +67,8 @@ typedef struct {
 /**
  * Per-thread coroutine bookkeeping
  */
-static __thread CoroutineUContext leader;
-static __thread Coroutine *current;
+QEMU_DEFINE_STATIC_CO_TLS(Coroutine *, current);
+QEMU_DEFINE_STATIC_CO_TLS(CoroutineUContext, leader);
 
 /*
  * va_args to makecontext() must be type 'int', so passing
@@ -97,14 +98,15 @@ static inline __attribute__((always_inline))
 void finish_switch_fiber(void *fake_stack_save)
 {
 #ifdef CONFIG_ASAN
+    CoroutineUContext *leaderp = get_ptr_leader();
     const void *bottom_old;
     size_t size_old;
 
     __sanitizer_finish_switch_fiber(fake_stack_save, &bottom_old, &size_old);
 
-    if (!leader.stack) {
-        leader.stack = (void *)bottom_old;
-        leader.stack_size = size_old;
+    if (!leaderp->stack) {
+        leaderp->stack = (void *)bottom_old;
+        leaderp->stack_size = size_old;
     }
 #endif
 #ifdef CONFIG_TSAN
@@ -161,8 +163,10 @@ static void coroutine_trampoline(int i0, int i1)
 
     /* Initialize longjmp environment and switch back the caller */
     if (!sigsetjmp(self->env, 0)) {
-        start_switch_fiber_asan(COROUTINE_YIELD, &fake_stack_save, leader.stack,
-                                leader.stack_size);
+        CoroutineUContext *leaderp = get_ptr_leader();
+
+        start_switch_fiber_asan(COROUTINE_YIELD, &fake_stack_save,
+                                leaderp->stack, leaderp->stack_size);
         start_switch_fiber_tsan(&fake_stack_save, self, true); /* true=caller */
         siglongjmp(*(sigjmp_buf *)co->entry_arg, 1);
     }
@@ -297,7 +301,7 @@ qemu_coroutine_switch(Coroutine *from_, Coroutine *to_,
     int ret;
     void *fake_stack_save = NULL;
 
-    current = to_;
+    set_current(to_);
 
     ret = sigsetjmp(from->env, 0);
     if (ret == 0) {
@@ -315,18 +319,24 @@ qemu_coroutine_switch(Coroutine *from_, Coroutine *to_,
 
 Coroutine *qemu_coroutine_self(void)
 {
-    if (!current) {
-        current = &leader.base;
+    Coroutine *self = get_current();
+    CoroutineUContext *leaderp = get_ptr_leader();
+
+    if (!self) {
+        self = &leaderp->base;
+        set_current(self);
     }
 #ifdef CONFIG_TSAN
-    if (!leader.tsan_co_fiber) {
-        leader.tsan_co_fiber = __tsan_get_current_fiber();
+    if (!leaderp->tsan_co_fiber) {
+        leaderp->tsan_co_fiber = __tsan_get_current_fiber();
     }
 #endif
-    return current;
+    return self;
 }
 
 bool qemu_in_coroutine(void)
 {
-    return current && current->caller;
+    Coroutine *self = get_current();
+
+    return self && self->caller;
 }

From ac387a08a9c9f6b36757da912f0339c25f421f90 Mon Sep 17 00:00:00 2001
From: Stefan Hajnoczi <stefanha@redhat.com>
Date: Mon, 7 Mar 2022 15:38:52 +0000
Subject: [PATCH 013/935] coroutine: use QEMU_DEFINE_STATIC_CO_TLS()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Thread-Local Storage variables cannot be used directly from coroutine
code because the compiler may optimize TLS variable accesses across
qemu_coroutine_yield() calls. When the coroutine is re-entered from
another thread the TLS variables from the old thread must no longer be
used.

Use QEMU_DEFINE_STATIC_CO_TLS() for the current and leader variables.
The alloc_pool QSLIST needs a typedef so the return value of
get_ptr_alloc_pool() can be stored in a local variable.

One example of why this code is necessary: a coroutine that yields
before calling qemu_coroutine_create() to create another coroutine is
affected by the TLS issue.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220307153853.602859-3-stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 util/qemu-coroutine.c | 41 ++++++++++++++++++++++++-----------------
 1 file changed, 24 insertions(+), 17 deletions(-)

diff --git a/util/qemu-coroutine.c b/util/qemu-coroutine.c
index c03b2422ff..f3e8300c8d 100644
--- a/util/qemu-coroutine.c
+++ b/util/qemu-coroutine.c
@@ -18,6 +18,7 @@
 #include "qemu/atomic.h"
 #include "qemu/coroutine.h"
 #include "qemu/coroutine_int.h"
+#include "qemu/coroutine-tls.h"
 #include "block/aio.h"
 
 /** Initial batch size is 64, and is increased on demand */
@@ -29,17 +30,20 @@ enum {
 static QSLIST_HEAD(, Coroutine) release_pool = QSLIST_HEAD_INITIALIZER(pool);
 static unsigned int pool_batch_size = POOL_INITIAL_BATCH_SIZE;
 static unsigned int release_pool_size;
-static __thread QSLIST_HEAD(, Coroutine) alloc_pool = QSLIST_HEAD_INITIALIZER(pool);
-static __thread unsigned int alloc_pool_size;
-static __thread Notifier coroutine_pool_cleanup_notifier;
+
+typedef QSLIST_HEAD(, Coroutine) CoroutineQSList;
+QEMU_DEFINE_STATIC_CO_TLS(CoroutineQSList, alloc_pool);
+QEMU_DEFINE_STATIC_CO_TLS(unsigned int, alloc_pool_size);
+QEMU_DEFINE_STATIC_CO_TLS(Notifier, coroutine_pool_cleanup_notifier);
 
 static void coroutine_pool_cleanup(Notifier *n, void *value)
 {
     Coroutine *co;
     Coroutine *tmp;
+    CoroutineQSList *alloc_pool = get_ptr_alloc_pool();
 
-    QSLIST_FOREACH_SAFE(co, &alloc_pool, pool_next, tmp) {
-        QSLIST_REMOVE_HEAD(&alloc_pool, pool_next);
+    QSLIST_FOREACH_SAFE(co, alloc_pool, pool_next, tmp) {
+        QSLIST_REMOVE_HEAD(alloc_pool, pool_next);
         qemu_coroutine_delete(co);
     }
 }
@@ -49,27 +53,30 @@ Coroutine *qemu_coroutine_create(CoroutineEntry *entry, void *opaque)
     Coroutine *co = NULL;
 
     if (CONFIG_COROUTINE_POOL) {
-        co = QSLIST_FIRST(&alloc_pool);
+        CoroutineQSList *alloc_pool = get_ptr_alloc_pool();
+
+        co = QSLIST_FIRST(alloc_pool);
         if (!co) {
             if (release_pool_size > qatomic_read(&pool_batch_size)) {
                 /* Slow path; a good place to register the destructor, too.  */
-                if (!coroutine_pool_cleanup_notifier.notify) {
-                    coroutine_pool_cleanup_notifier.notify = coroutine_pool_cleanup;
-                    qemu_thread_atexit_add(&coroutine_pool_cleanup_notifier);
+                Notifier *notifier = get_ptr_coroutine_pool_cleanup_notifier();
+                if (!notifier->notify) {
+                    notifier->notify = coroutine_pool_cleanup;
+                    qemu_thread_atexit_add(notifier);
                 }
 
                 /* This is not exact; there could be a little skew between
                  * release_pool_size and the actual size of release_pool.  But
                  * it is just a heuristic, it does not need to be perfect.
                  */
-                alloc_pool_size = qatomic_xchg(&release_pool_size, 0);
-                QSLIST_MOVE_ATOMIC(&alloc_pool, &release_pool);
-                co = QSLIST_FIRST(&alloc_pool);
+                set_alloc_pool_size(qatomic_xchg(&release_pool_size, 0));
+                QSLIST_MOVE_ATOMIC(alloc_pool, &release_pool);
+                co = QSLIST_FIRST(alloc_pool);
             }
         }
         if (co) {
-            QSLIST_REMOVE_HEAD(&alloc_pool, pool_next);
-            alloc_pool_size--;
+            QSLIST_REMOVE_HEAD(alloc_pool, pool_next);
+            set_alloc_pool_size(get_alloc_pool_size() - 1);
         }
     }
 
@@ -93,9 +100,9 @@ static void coroutine_delete(Coroutine *co)
             qatomic_inc(&release_pool_size);
             return;
         }
-        if (alloc_pool_size < qatomic_read(&pool_batch_size)) {
-            QSLIST_INSERT_HEAD(&alloc_pool, co, pool_next);
-            alloc_pool_size++;
+        if (get_alloc_pool_size() < qatomic_read(&pool_batch_size)) {
+            QSLIST_INSERT_HEAD(get_ptr_alloc_pool(), co, pool_next);
+            set_alloc_pool_size(get_alloc_pool_size() + 1);
             return;
         }
     }

From c1fe694357a328c807ae3cc6961c19e923448fcc Mon Sep 17 00:00:00 2001
From: Stefan Hajnoczi <stefanha@redhat.com>
Date: Mon, 7 Mar 2022 15:38:53 +0000
Subject: [PATCH 014/935] coroutine-win32: use QEMU_DEFINE_STATIC_CO_TLS()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Thread-Local Storage variables cannot be used directly from coroutine
code because the compiler may optimize TLS variable accesses across
qemu_coroutine_yield() calls. When the coroutine is re-entered from
another thread the TLS variables from the old thread must no longer be
used.

Use QEMU_DEFINE_STATIC_CO_TLS() for the current and leader variables.

I think coroutine-win32.c could get away with __thread because the
variables are only used in situations where either the stale value is
correct (current) or outside coroutine context (loading leader when
current is NULL). Due to the difficulty of being sure that this is
really safe in all scenarios it seems worth converting it anyway.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220307153853.602859-4-stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 util/coroutine-win32.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/util/coroutine-win32.c b/util/coroutine-win32.c
index c196f956d2..7db2e8f8c8 100644
--- a/util/coroutine-win32.c
+++ b/util/coroutine-win32.c
@@ -24,6 +24,7 @@
 
 #include "qemu/osdep.h"
 #include "qemu/coroutine_int.h"
+#include "qemu/coroutine-tls.h"
 
 typedef struct
 {
@@ -33,8 +34,8 @@ typedef struct
     CoroutineAction action;
 } CoroutineWin32;
 
-static __thread CoroutineWin32 leader;
-static __thread Coroutine *current;
+QEMU_DEFINE_STATIC_CO_TLS(CoroutineWin32, leader);
+QEMU_DEFINE_STATIC_CO_TLS(Coroutine *, current);
 
 /* This function is marked noinline to prevent GCC from inlining it
  * into coroutine_trampoline(). If we allow it to do that then it
@@ -51,7 +52,7 @@ qemu_coroutine_switch(Coroutine *from_, Coroutine *to_,
     CoroutineWin32 *from = DO_UPCAST(CoroutineWin32, base, from_);
     CoroutineWin32 *to = DO_UPCAST(CoroutineWin32, base, to_);
 
-    current = to_;
+    set_current(to_);
 
     to->action = action;
     SwitchToFiber(to->fiber);
@@ -88,14 +89,21 @@ void qemu_coroutine_delete(Coroutine *co_)
 
 Coroutine *qemu_coroutine_self(void)
 {
+    Coroutine *current = get_current();
+
     if (!current) {
-        current = &leader.base;
-        leader.fiber = ConvertThreadToFiber(NULL);
+        CoroutineWin32 *leader = get_ptr_leader();
+
+        current = &leader->base;
+        set_current(current);
+        leader->fiber = ConvertThreadToFiber(NULL);
     }
     return current;
 }
 
 bool qemu_in_coroutine(void)
 {
+    Coroutine *current = get_current();
+
     return current && current->caller;
 }

From cda86e2b46de857e8b6e16ecd13bb85d81e07899 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Tue, 26 Apr 2022 21:23:12 -0700
Subject: [PATCH 015/935] target/arm: Enable SCTLR_EL1.BT0 for
 aarch64-linux-user

This controls whether the PACI{A,B}SP instructions trap with BTYPE=3
(indirect branch from register other than x16/x17).  The linux kernel
sets this in bti_enable().

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/998
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20220427042312.294300-1-richard.henderson@linaro.org
[PMM: remove stray change to makefile comment]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu.c                  |  2 ++
 tests/tcg/aarch64/Makefile.target |  6 ++---
 tests/tcg/aarch64/bti-3.c         | 42 +++++++++++++++++++++++++++++++
 3 files changed, 47 insertions(+), 3 deletions(-)
 create mode 100644 tests/tcg/aarch64/bti-3.c

diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index e46a766d77..2b81b18351 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -197,6 +197,8 @@ static void arm_cpu_reset(DeviceState *dev)
         /* Enable all PAC keys.  */
         env->cp15.sctlr_el[1] |= (SCTLR_EnIA | SCTLR_EnIB |
                                   SCTLR_EnDA | SCTLR_EnDB);
+        /* Trap on btype=3 for PACIxSP. */
+        env->cp15.sctlr_el[1] |= SCTLR_BT0;
         /* and to the FP/Neon instructions */
         env->cp15.cpacr_el1 = deposit64(env->cp15.cpacr_el1, 20, 2, 3);
         /* and to the SVE instructions */
diff --git a/tests/tcg/aarch64/Makefile.target b/tests/tcg/aarch64/Makefile.target
index 6ad0ad49f9..d6a74d24dc 100644
--- a/tests/tcg/aarch64/Makefile.target
+++ b/tests/tcg/aarch64/Makefile.target
@@ -28,9 +28,9 @@ endif
 # BTI Tests
 # bti-1 tests the elf notes, so we require special compiler support.
 ifneq ($(CROSS_CC_HAS_ARMV8_BTI),)
-AARCH64_TESTS += bti-1
-bti-1: CFLAGS += -mbranch-protection=standard
-bti-1: LDFLAGS += -nostdlib
+AARCH64_TESTS += bti-1 bti-3
+bti-1 bti-3: CFLAGS += -mbranch-protection=standard
+bti-1 bti-3: LDFLAGS += -nostdlib
 endif
 # bti-2 tests PROT_BTI, so no special compiler support required.
 AARCH64_TESTS += bti-2
diff --git a/tests/tcg/aarch64/bti-3.c b/tests/tcg/aarch64/bti-3.c
new file mode 100644
index 0000000000..a852856d9a
--- /dev/null
+++ b/tests/tcg/aarch64/bti-3.c
@@ -0,0 +1,42 @@
+/*
+ * BTI vs PACIASP
+ */
+
+#include "bti-crt.inc.c"
+
+static void skip2_sigill(int sig, siginfo_t *info, ucontext_t *uc)
+{
+    uc->uc_mcontext.pc += 8;
+    uc->uc_mcontext.pstate = 1;
+}
+
+#define BTYPE_1() \
+    asm("mov %0,#1; adr x16, 1f; br x16; 1: hint #25; mov %0,#0" \
+        : "=r"(skipped) : : "x16", "x30")
+
+#define BTYPE_2() \
+    asm("mov %0,#1; adr x16, 1f; blr x16; 1: hint #25; mov %0,#0" \
+        : "=r"(skipped) : : "x16", "x30")
+
+#define BTYPE_3() \
+    asm("mov %0,#1; adr x15, 1f; br x15; 1: hint #25; mov %0,#0" \
+        : "=r"(skipped) : : "x15", "x30")
+
+#define TEST(WHICH, EXPECT) \
+    do { WHICH(); fail += skipped ^ EXPECT; } while (0)
+
+int main()
+{
+    int fail = 0;
+    int skipped;
+
+    /* Signal-like with SA_SIGINFO.  */
+    signal_info(SIGILL, skip2_sigill);
+
+    /* With SCTLR_EL1.BT0 set, PACIASP is not compatible with type=3. */
+    TEST(BTYPE_1, 0);
+    TEST(BTYPE_2, 0);
+    TEST(BTYPE_3, 1);
+
+    return fail;
+}

From cf7c6d1004eaaae85fd6156556e2f38ff493ef48 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:49:43 -0700
Subject: [PATCH 016/935] target/arm: Split out cpregs.h
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Move ARMCPRegInfo and all related declarations to a new
internal header, out of the public cpu.h.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220501055028.646596-2-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/arm/pxa2xx.c            |   1 +
 hw/arm/pxa2xx_pic.c        |   1 +
 hw/intc/arm_gicv3_cpuif.c  |   1 +
 hw/intc/arm_gicv3_kvm.c    |   2 +
 target/arm/cpregs.h        | 413 +++++++++++++++++++++++++++++++++++++
 target/arm/cpu.c           |   1 +
 target/arm/cpu.h           | 368 ---------------------------------
 target/arm/cpu64.c         |   1 +
 target/arm/cpu_tcg.c       |   1 +
 target/arm/gdbstub.c       |   3 +-
 target/arm/helper.c        |   1 +
 target/arm/op_helper.c     |   1 +
 target/arm/translate-a64.c |   4 +-
 target/arm/translate.c     |   3 +-
 14 files changed, 427 insertions(+), 374 deletions(-)
 create mode 100644 target/arm/cpregs.h

diff --git a/hw/arm/pxa2xx.c b/hw/arm/pxa2xx.c
index a6f938f115..0683714733 100644
--- a/hw/arm/pxa2xx.c
+++ b/hw/arm/pxa2xx.c
@@ -30,6 +30,7 @@
 #include "qemu/cutils.h"
 #include "qemu/log.h"
 #include "qom/object.h"
+#include "target/arm/cpregs.h"
 
 static struct {
     hwaddr io_base;
diff --git a/hw/arm/pxa2xx_pic.c b/hw/arm/pxa2xx_pic.c
index ed032fed54..b80d75d839 100644
--- a/hw/arm/pxa2xx_pic.c
+++ b/hw/arm/pxa2xx_pic.c
@@ -17,6 +17,7 @@
 #include "hw/sysbus.h"
 #include "migration/vmstate.h"
 #include "qom/object.h"
+#include "target/arm/cpregs.h"
 
 #define ICIP	0x00	/* Interrupt Controller IRQ Pending register */
 #define ICMR	0x04	/* Interrupt Controller Mask register */
diff --git a/hw/intc/arm_gicv3_cpuif.c b/hw/intc/arm_gicv3_cpuif.c
index 8404f46ee0..2d5959db94 100644
--- a/hw/intc/arm_gicv3_cpuif.c
+++ b/hw/intc/arm_gicv3_cpuif.c
@@ -20,6 +20,7 @@
 #include "gicv3_internal.h"
 #include "hw/irq.h"
 #include "cpu.h"
+#include "target/arm/cpregs.h"
 
 /*
  * Special case return value from hppvi_index(); must be larger than
diff --git a/hw/intc/arm_gicv3_kvm.c b/hw/intc/arm_gicv3_kvm.c
index 06f5aceee5..611085e98d 100644
--- a/hw/intc/arm_gicv3_kvm.c
+++ b/hw/intc/arm_gicv3_kvm.c
@@ -31,6 +31,8 @@
 #include "vgic_common.h"
 #include "migration/blocker.h"
 #include "qom/object.h"
+#include "target/arm/cpregs.h"
+
 
 #ifdef DEBUG_GICV3_KVM
 #define DPRINTF(fmt, ...) \
diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h
new file mode 100644
index 0000000000..8064c0763e
--- /dev/null
+++ b/target/arm/cpregs.h
@@ -0,0 +1,413 @@
+/*
+ * QEMU ARM CP Register access and descriptions
+ *
+ * Copyright (c) 2022 Linaro Ltd
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see
+ * <http://www.gnu.org/licenses/gpl-2.0.html>
+ */
+
+#ifndef TARGET_ARM_CPREGS_H
+#define TARGET_ARM_CPREGS_H
+
+/*
+ * ARMCPRegInfo type field bits. If the SPECIAL bit is set this is a
+ * special-behaviour cp reg and bits [11..8] indicate what behaviour
+ * it has. Otherwise it is a simple cp reg, where CONST indicates that
+ * TCG can assume the value to be constant (ie load at translate time)
+ * and 64BIT indicates a 64 bit wide coprocessor register. SUPPRESS_TB_END
+ * indicates that the TB should not be ended after a write to this register
+ * (the default is that the TB ends after cp writes). OVERRIDE permits
+ * a register definition to override a previous definition for the
+ * same (cp, is64, crn, crm, opc1, opc2) tuple: either the new or the
+ * old must have the OVERRIDE bit set.
+ * ALIAS indicates that this register is an alias view of some underlying
+ * state which is also visible via another register, and that the other
+ * register is handling migration and reset; registers marked ALIAS will not be
+ * migrated but may have their state set by syncing of register state from KVM.
+ * NO_RAW indicates that this register has no underlying state and does not
+ * support raw access for state saving/loading; it will not be used for either
+ * migration or KVM state synchronization. (Typically this is for "registers"
+ * which are actually used as instructions for cache maintenance and so on.)
+ * IO indicates that this register does I/O and therefore its accesses
+ * need to be marked with gen_io_start() and also end the TB. In particular,
+ * registers which implement clocks or timers require this.
+ * RAISES_EXC is for when the read or write hook might raise an exception;
+ * the generated code will synchronize the CPU state before calling the hook
+ * so that it is safe for the hook to call raise_exception().
+ * NEWEL is for writes to registers that might change the exception
+ * level - typically on older ARM chips. For those cases we need to
+ * re-read the new el when recomputing the translation flags.
+ */
+#define ARM_CP_SPECIAL           0x0001
+#define ARM_CP_CONST             0x0002
+#define ARM_CP_64BIT             0x0004
+#define ARM_CP_SUPPRESS_TB_END   0x0008
+#define ARM_CP_OVERRIDE          0x0010
+#define ARM_CP_ALIAS             0x0020
+#define ARM_CP_IO                0x0040
+#define ARM_CP_NO_RAW            0x0080
+#define ARM_CP_NOP               (ARM_CP_SPECIAL | 0x0100)
+#define ARM_CP_WFI               (ARM_CP_SPECIAL | 0x0200)
+#define ARM_CP_NZCV              (ARM_CP_SPECIAL | 0x0300)
+#define ARM_CP_CURRENTEL         (ARM_CP_SPECIAL | 0x0400)
+#define ARM_CP_DC_ZVA            (ARM_CP_SPECIAL | 0x0500)
+#define ARM_CP_DC_GVA            (ARM_CP_SPECIAL | 0x0600)
+#define ARM_CP_DC_GZVA           (ARM_CP_SPECIAL | 0x0700)
+#define ARM_LAST_SPECIAL         ARM_CP_DC_GZVA
+#define ARM_CP_FPU               0x1000
+#define ARM_CP_SVE               0x2000
+#define ARM_CP_NO_GDB            0x4000
+#define ARM_CP_RAISES_EXC        0x8000
+#define ARM_CP_NEWEL             0x10000
+/* Used only as a terminator for ARMCPRegInfo lists */
+#define ARM_CP_SENTINEL          0xfffff
+/* Mask of only the flag bits in a type field */
+#define ARM_CP_FLAG_MASK         0x1f0ff
+
+/*
+ * Valid values for ARMCPRegInfo state field, indicating which of
+ * the AArch32 and AArch64 execution states this register is visible in.
+ * If the reginfo doesn't explicitly specify then it is AArch32 only.
+ * If the reginfo is declared to be visible in both states then a second
+ * reginfo is synthesised for the AArch32 view of the AArch64 register,
+ * such that the AArch32 view is the lower 32 bits of the AArch64 one.
+ * Note that we rely on the values of these enums as we iterate through
+ * the various states in some places.
+ */
+enum {
+    ARM_CP_STATE_AA32 = 0,
+    ARM_CP_STATE_AA64 = 1,
+    ARM_CP_STATE_BOTH = 2,
+};
+
+/*
+ * ARM CP register secure state flags.  These flags identify security state
+ * attributes for a given CP register entry.
+ * The existence of both or neither secure and non-secure flags indicates that
+ * the register has both a secure and non-secure hash entry.  A single one of
+ * these flags causes the register to only be hashed for the specified
+ * security state.
+ * Although definitions may have any combination of the S/NS bits, each
+ * registered entry will only have one to identify whether the entry is secure
+ * or non-secure.
+ */
+enum {
+    ARM_CP_SECSTATE_S =   (1 << 0), /* bit[0]: Secure state register */
+    ARM_CP_SECSTATE_NS =  (1 << 1), /* bit[1]: Non-secure state register */
+};
+
+/*
+ * Return true if cptype is a valid type field. This is used to try to
+ * catch errors where the sentinel has been accidentally left off the end
+ * of a list of registers.
+ */
+static inline bool cptype_valid(int cptype)
+{
+    return ((cptype & ~ARM_CP_FLAG_MASK) == 0)
+        || ((cptype & ARM_CP_SPECIAL) &&
+            ((cptype & ~ARM_CP_FLAG_MASK) <= ARM_LAST_SPECIAL));
+}
+
+/*
+ * Access rights:
+ * We define bits for Read and Write access for what rev C of the v7-AR ARM ARM
+ * defines as PL0 (user), PL1 (fiq/irq/svc/abt/und/sys, ie privileged), and
+ * PL2 (hyp). The other level which has Read and Write bits is Secure PL1
+ * (ie any of the privileged modes in Secure state, or Monitor mode).
+ * If a register is accessible in one privilege level it's always accessible
+ * in higher privilege levels too. Since "Secure PL1" also follows this rule
+ * (ie anything visible in PL2 is visible in S-PL1, some things are only
+ * visible in S-PL1) but "Secure PL1" is a bit of a mouthful, we bend the
+ * terminology a little and call this PL3.
+ * In AArch64 things are somewhat simpler as the PLx bits line up exactly
+ * with the ELx exception levels.
+ *
+ * If access permissions for a register are more complex than can be
+ * described with these bits, then use a laxer set of restrictions, and
+ * do the more restrictive/complex check inside a helper function.
+ */
+#define PL3_R 0x80
+#define PL3_W 0x40
+#define PL2_R (0x20 | PL3_R)
+#define PL2_W (0x10 | PL3_W)
+#define PL1_R (0x08 | PL2_R)
+#define PL1_W (0x04 | PL2_W)
+#define PL0_R (0x02 | PL1_R)
+#define PL0_W (0x01 | PL1_W)
+
+/*
+ * For user-mode some registers are accessible to EL0 via a kernel
+ * trap-and-emulate ABI. In this case we define the read permissions
+ * as actually being PL0_R. However some bits of any given register
+ * may still be masked.
+ */
+#ifdef CONFIG_USER_ONLY
+#define PL0U_R PL0_R
+#else
+#define PL0U_R PL1_R
+#endif
+
+#define PL3_RW (PL3_R | PL3_W)
+#define PL2_RW (PL2_R | PL2_W)
+#define PL1_RW (PL1_R | PL1_W)
+#define PL0_RW (PL0_R | PL0_W)
+
+typedef enum CPAccessResult {
+    /* Access is permitted */
+    CP_ACCESS_OK = 0,
+    /*
+     * Access fails due to a configurable trap or enable which would
+     * result in a categorized exception syndrome giving information about
+     * the failing instruction (ie syndrome category 0x3, 0x4, 0x5, 0x6,
+     * 0xc or 0x18). The exception is taken to the usual target EL (EL1 or
+     * PL1 if in EL0, otherwise to the current EL).
+     */
+    CP_ACCESS_TRAP = 1,
+    /*
+     * Access fails and results in an exception syndrome 0x0 ("uncategorized").
+     * Note that this is not a catch-all case -- the set of cases which may
+     * result in this failure is specifically defined by the architecture.
+     */
+    CP_ACCESS_TRAP_UNCATEGORIZED = 2,
+    /* As CP_ACCESS_TRAP, but for traps directly to EL2 or EL3 */
+    CP_ACCESS_TRAP_EL2 = 3,
+    CP_ACCESS_TRAP_EL3 = 4,
+    /* As CP_ACCESS_UNCATEGORIZED, but for traps directly to EL2 or EL3 */
+    CP_ACCESS_TRAP_UNCATEGORIZED_EL2 = 5,
+    CP_ACCESS_TRAP_UNCATEGORIZED_EL3 = 6,
+} CPAccessResult;
+
+typedef struct ARMCPRegInfo ARMCPRegInfo;
+
+/*
+ * Access functions for coprocessor registers. These cannot fail and
+ * may not raise exceptions.
+ */
+typedef uint64_t CPReadFn(CPUARMState *env, const ARMCPRegInfo *opaque);
+typedef void CPWriteFn(CPUARMState *env, const ARMCPRegInfo *opaque,
+                       uint64_t value);
+/* Access permission check functions for coprocessor registers. */
+typedef CPAccessResult CPAccessFn(CPUARMState *env,
+                                  const ARMCPRegInfo *opaque,
+                                  bool isread);
+/* Hook function for register reset */
+typedef void CPResetFn(CPUARMState *env, const ARMCPRegInfo *opaque);
+
+#define CP_ANY 0xff
+
+/* Definition of an ARM coprocessor register */
+struct ARMCPRegInfo {
+    /* Name of register (useful mainly for debugging, need not be unique) */
+    const char *name;
+    /*
+     * Location of register: coprocessor number and (crn,crm,opc1,opc2)
+     * tuple. Any of crm, opc1 and opc2 may be CP_ANY to indicate a
+     * 'wildcard' field -- any value of that field in the MRC/MCR insn
+     * will be decoded to this register. The register read and write
+     * callbacks will be passed an ARMCPRegInfo with the crn/crm/opc1/opc2
+     * used by the program, so it is possible to register a wildcard and
+     * then behave differently on read/write if necessary.
+     * For 64 bit registers, only crm and opc1 are relevant; crn and opc2
+     * must both be zero.
+     * For AArch64-visible registers, opc0 is also used.
+     * Since there are no "coprocessors" in AArch64, cp is purely used as a
+     * way to distinguish (for KVM's benefit) guest-visible system registers
+     * from demuxed ones provided to preserve the "no side effects on
+     * KVM register read/write from QEMU" semantics. cp==0x13 is guest
+     * visible (to match KVM's encoding); cp==0 will be converted to
+     * cp==0x13 when the ARMCPRegInfo is registered, for convenience.
+     */
+    uint8_t cp;
+    uint8_t crn;
+    uint8_t crm;
+    uint8_t opc0;
+    uint8_t opc1;
+    uint8_t opc2;
+    /* Execution state in which this register is visible: ARM_CP_STATE_* */
+    int state;
+    /* Register type: ARM_CP_* bits/values */
+    int type;
+    /* Access rights: PL*_[RW] */
+    int access;
+    /* Security state: ARM_CP_SECSTATE_* bits/values */
+    int secure;
+    /*
+     * The opaque pointer passed to define_arm_cp_regs_with_opaque() when
+     * this register was defined: can be used to hand data through to the
+     * register read/write functions, since they are passed the ARMCPRegInfo*.
+     */
+    void *opaque;
+    /*
+     * Value of this register, if it is ARM_CP_CONST. Otherwise, if
+     * fieldoffset is non-zero, the reset value of the register.
+     */
+    uint64_t resetvalue;
+    /*
+     * Offset of the field in CPUARMState for this register.
+     * This is not needed if either:
+     *  1. type is ARM_CP_CONST or one of the ARM_CP_SPECIALs
+     *  2. both readfn and writefn are specified
+     */
+    ptrdiff_t fieldoffset; /* offsetof(CPUARMState, field) */
+
+    /*
+     * Offsets of the secure and non-secure fields in CPUARMState for the
+     * register if it is banked.  These fields are only used during the static
+     * registration of a register.  During hashing the bank associated
+     * with a given security state is copied to fieldoffset which is used from
+     * there on out.
+     *
+     * It is expected that register definitions use either fieldoffset or
+     * bank_fieldoffsets in the definition but not both.  It is also expected
+     * that both bank offsets are set when defining a banked register.  This
+     * use indicates that a register is banked.
+     */
+    ptrdiff_t bank_fieldoffsets[2];
+
+    /*
+     * Function for making any access checks for this register in addition to
+     * those specified by the 'access' permissions bits. If NULL, no extra
+     * checks required. The access check is performed at runtime, not at
+     * translate time.
+     */
+    CPAccessFn *accessfn;
+    /*
+     * Function for handling reads of this register. If NULL, then reads
+     * will be done by loading from the offset into CPUARMState specified
+     * by fieldoffset.
+     */
+    CPReadFn *readfn;
+    /*
+     * Function for handling writes of this register. If NULL, then writes
+     * will be done by writing to the offset into CPUARMState specified
+     * by fieldoffset.
+     */
+    CPWriteFn *writefn;
+    /*
+     * Function for doing a "raw" read; used when we need to copy
+     * coprocessor state to the kernel for KVM or out for
+     * migration. This only needs to be provided if there is also a
+     * readfn and it has side effects (for instance clear-on-read bits).
+     */
+    CPReadFn *raw_readfn;
+    /*
+     * Function for doing a "raw" write; used when we need to copy KVM
+     * kernel coprocessor state into userspace, or for inbound
+     * migration. This only needs to be provided if there is also a
+     * writefn and it masks out "unwritable" bits or has write-one-to-clear
+     * or similar behaviour.
+     */
+    CPWriteFn *raw_writefn;
+    /*
+     * Function for resetting the register. If NULL, then reset will be done
+     * by writing resetvalue to the field specified in fieldoffset. If
+     * fieldoffset is 0 then no reset will be done.
+     */
+    CPResetFn *resetfn;
+
+    /*
+     * "Original" writefn and readfn.
+     * For ARMv8.1-VHE register aliases, we overwrite the read/write
+     * accessor functions of various EL1/EL0 to perform the runtime
+     * check for which sysreg should actually be modified, and then
+     * forwards the operation.  Before overwriting the accessors,
+     * the original function is copied here, so that accesses that
+     * really do go to the EL1/EL0 version proceed normally.
+     * (The corresponding EL2 register is linked via opaque.)
+     */
+    CPReadFn *orig_readfn;
+    CPWriteFn *orig_writefn;
+};
+
+/*
+ * Macros which are lvalues for the field in CPUARMState for the
+ * ARMCPRegInfo *ri.
+ */
+#define CPREG_FIELD32(env, ri) \
+    (*(uint32_t *)((char *)(env) + (ri)->fieldoffset))
+#define CPREG_FIELD64(env, ri) \
+    (*(uint64_t *)((char *)(env) + (ri)->fieldoffset))
+
+#define REGINFO_SENTINEL { .type = ARM_CP_SENTINEL }
+
+void define_arm_cp_regs_with_opaque(ARMCPU *cpu,
+                                    const ARMCPRegInfo *regs, void *opaque);
+void define_one_arm_cp_reg_with_opaque(ARMCPU *cpu,
+                                       const ARMCPRegInfo *regs, void *opaque);
+static inline void define_arm_cp_regs(ARMCPU *cpu, const ARMCPRegInfo *regs)
+{
+    define_arm_cp_regs_with_opaque(cpu, regs, 0);
+}
+static inline void define_one_arm_cp_reg(ARMCPU *cpu, const ARMCPRegInfo *regs)
+{
+    define_one_arm_cp_reg_with_opaque(cpu, regs, 0);
+}
+const ARMCPRegInfo *get_arm_cp_reginfo(GHashTable *cpregs, uint32_t encoded_cp);
+
+/*
+ * Definition of an ARM co-processor register as viewed from
+ * userspace. This is used for presenting sanitised versions of
+ * registers to userspace when emulating the Linux AArch64 CPU
+ * ID/feature ABI (advertised as HWCAP_CPUID).
+ */
+typedef struct ARMCPRegUserSpaceInfo {
+    /* Name of register */
+    const char *name;
+
+    /* Is the name actually a glob pattern */
+    bool is_glob;
+
+    /* Only some bits are exported to user space */
+    uint64_t exported_bits;
+
+    /* Fixed bits are applied after the mask */
+    uint64_t fixed_bits;
+} ARMCPRegUserSpaceInfo;
+
+#define REGUSERINFO_SENTINEL { .name = NULL }
+
+void modify_arm_cp_regs(ARMCPRegInfo *regs, const ARMCPRegUserSpaceInfo *mods);
+
+/* CPWriteFn that can be used to implement writes-ignored behaviour */
+void arm_cp_write_ignore(CPUARMState *env, const ARMCPRegInfo *ri,
+                         uint64_t value);
+/* CPReadFn that can be used for read-as-zero behaviour */
+uint64_t arm_cp_read_zero(CPUARMState *env, const ARMCPRegInfo *ri);
+
+/*
+ * CPResetFn that does nothing, for use if no reset is required even
+ * if fieldoffset is non zero.
+ */
+void arm_cp_reset_ignore(CPUARMState *env, const ARMCPRegInfo *opaque);
+
+/*
+ * Return true if this reginfo struct's field in the cpu state struct
+ * is 64 bits wide.
+ */
+static inline bool cpreg_field_is_64bit(const ARMCPRegInfo *ri)
+{
+    return (ri->state == ARM_CP_STATE_AA64) || (ri->type & ARM_CP_64BIT);
+}
+
+static inline bool cp_access_ok(int current_el,
+                                const ARMCPRegInfo *ri, int isread)
+{
+    return (ri->access >> ((current_el * 2) + isread)) & 1;
+}
+
+/* Raw read of a coprocessor register (as needed for migration, etc) */
+uint64_t read_raw_cp_reg(CPUARMState *env, const ARMCPRegInfo *ri);
+
+#endif /* TARGET_ARM_CPREGS_H */
diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index 2b81b18351..18212eb6ee 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -43,6 +43,7 @@
 #include "kvm_arm.h"
 #include "disas/capstone.h"
 #include "fpu/softfloat.h"
+#include "cpregs.h"
 
 static void arm_cpu_set_pc(CPUState *cs, vaddr value)
 {
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index db8ff04449..d1b558385c 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -2595,144 +2595,6 @@ static inline uint64_t cpreg_to_kvm_id(uint32_t cpregid)
     return kvmid;
 }
 
-/* ARMCPRegInfo type field bits. If the SPECIAL bit is set this is a
- * special-behaviour cp reg and bits [11..8] indicate what behaviour
- * it has. Otherwise it is a simple cp reg, where CONST indicates that
- * TCG can assume the value to be constant (ie load at translate time)
- * and 64BIT indicates a 64 bit wide coprocessor register. SUPPRESS_TB_END
- * indicates that the TB should not be ended after a write to this register
- * (the default is that the TB ends after cp writes). OVERRIDE permits
- * a register definition to override a previous definition for the
- * same (cp, is64, crn, crm, opc1, opc2) tuple: either the new or the
- * old must have the OVERRIDE bit set.
- * ALIAS indicates that this register is an alias view of some underlying
- * state which is also visible via another register, and that the other
- * register is handling migration and reset; registers marked ALIAS will not be
- * migrated but may have their state set by syncing of register state from KVM.
- * NO_RAW indicates that this register has no underlying state and does not
- * support raw access for state saving/loading; it will not be used for either
- * migration or KVM state synchronization. (Typically this is for "registers"
- * which are actually used as instructions for cache maintenance and so on.)
- * IO indicates that this register does I/O and therefore its accesses
- * need to be marked with gen_io_start() and also end the TB. In particular,
- * registers which implement clocks or timers require this.
- * RAISES_EXC is for when the read or write hook might raise an exception;
- * the generated code will synchronize the CPU state before calling the hook
- * so that it is safe for the hook to call raise_exception().
- * NEWEL is for writes to registers that might change the exception
- * level - typically on older ARM chips. For those cases we need to
- * re-read the new el when recomputing the translation flags.
- */
-#define ARM_CP_SPECIAL           0x0001
-#define ARM_CP_CONST             0x0002
-#define ARM_CP_64BIT             0x0004
-#define ARM_CP_SUPPRESS_TB_END   0x0008
-#define ARM_CP_OVERRIDE          0x0010
-#define ARM_CP_ALIAS             0x0020
-#define ARM_CP_IO                0x0040
-#define ARM_CP_NO_RAW            0x0080
-#define ARM_CP_NOP               (ARM_CP_SPECIAL | 0x0100)
-#define ARM_CP_WFI               (ARM_CP_SPECIAL | 0x0200)
-#define ARM_CP_NZCV              (ARM_CP_SPECIAL | 0x0300)
-#define ARM_CP_CURRENTEL         (ARM_CP_SPECIAL | 0x0400)
-#define ARM_CP_DC_ZVA            (ARM_CP_SPECIAL | 0x0500)
-#define ARM_CP_DC_GVA            (ARM_CP_SPECIAL | 0x0600)
-#define ARM_CP_DC_GZVA           (ARM_CP_SPECIAL | 0x0700)
-#define ARM_LAST_SPECIAL         ARM_CP_DC_GZVA
-#define ARM_CP_FPU               0x1000
-#define ARM_CP_SVE               0x2000
-#define ARM_CP_NO_GDB            0x4000
-#define ARM_CP_RAISES_EXC        0x8000
-#define ARM_CP_NEWEL             0x10000
-/* Used only as a terminator for ARMCPRegInfo lists */
-#define ARM_CP_SENTINEL          0xfffff
-/* Mask of only the flag bits in a type field */
-#define ARM_CP_FLAG_MASK         0x1f0ff
-
-/* Valid values for ARMCPRegInfo state field, indicating which of
- * the AArch32 and AArch64 execution states this register is visible in.
- * If the reginfo doesn't explicitly specify then it is AArch32 only.
- * If the reginfo is declared to be visible in both states then a second
- * reginfo is synthesised for the AArch32 view of the AArch64 register,
- * such that the AArch32 view is the lower 32 bits of the AArch64 one.
- * Note that we rely on the values of these enums as we iterate through
- * the various states in some places.
- */
-enum {
-    ARM_CP_STATE_AA32 = 0,
-    ARM_CP_STATE_AA64 = 1,
-    ARM_CP_STATE_BOTH = 2,
-};
-
-/* ARM CP register secure state flags.  These flags identify security state
- * attributes for a given CP register entry.
- * The existence of both or neither secure and non-secure flags indicates that
- * the register has both a secure and non-secure hash entry.  A single one of
- * these flags causes the register to only be hashed for the specified
- * security state.
- * Although definitions may have any combination of the S/NS bits, each
- * registered entry will only have one to identify whether the entry is secure
- * or non-secure.
- */
-enum {
-    ARM_CP_SECSTATE_S =   (1 << 0), /* bit[0]: Secure state register */
-    ARM_CP_SECSTATE_NS =  (1 << 1), /* bit[1]: Non-secure state register */
-};
-
-/* Return true if cptype is a valid type field. This is used to try to
- * catch errors where the sentinel has been accidentally left off the end
- * of a list of registers.
- */
-static inline bool cptype_valid(int cptype)
-{
-    return ((cptype & ~ARM_CP_FLAG_MASK) == 0)
-        || ((cptype & ARM_CP_SPECIAL) &&
-            ((cptype & ~ARM_CP_FLAG_MASK) <= ARM_LAST_SPECIAL));
-}
-
-/* Access rights:
- * We define bits for Read and Write access for what rev C of the v7-AR ARM ARM
- * defines as PL0 (user), PL1 (fiq/irq/svc/abt/und/sys, ie privileged), and
- * PL2 (hyp). The other level which has Read and Write bits is Secure PL1
- * (ie any of the privileged modes in Secure state, or Monitor mode).
- * If a register is accessible in one privilege level it's always accessible
- * in higher privilege levels too. Since "Secure PL1" also follows this rule
- * (ie anything visible in PL2 is visible in S-PL1, some things are only
- * visible in S-PL1) but "Secure PL1" is a bit of a mouthful, we bend the
- * terminology a little and call this PL3.
- * In AArch64 things are somewhat simpler as the PLx bits line up exactly
- * with the ELx exception levels.
- *
- * If access permissions for a register are more complex than can be
- * described with these bits, then use a laxer set of restrictions, and
- * do the more restrictive/complex check inside a helper function.
- */
-#define PL3_R 0x80
-#define PL3_W 0x40
-#define PL2_R (0x20 | PL3_R)
-#define PL2_W (0x10 | PL3_W)
-#define PL1_R (0x08 | PL2_R)
-#define PL1_W (0x04 | PL2_W)
-#define PL0_R (0x02 | PL1_R)
-#define PL0_W (0x01 | PL1_W)
-
-/*
- * For user-mode some registers are accessible to EL0 via a kernel
- * trap-and-emulate ABI. In this case we define the read permissions
- * as actually being PL0_R. However some bits of any given register
- * may still be masked.
- */
-#ifdef CONFIG_USER_ONLY
-#define PL0U_R PL0_R
-#else
-#define PL0U_R PL1_R
-#endif
-
-#define PL3_RW (PL3_R | PL3_W)
-#define PL2_RW (PL2_R | PL2_W)
-#define PL1_RW (PL1_R | PL1_W)
-#define PL0_RW (PL0_R | PL0_W)
-
 /* Return the highest implemented Exception Level */
 static inline int arm_highest_el(CPUARMState *env)
 {
@@ -2784,236 +2646,6 @@ static inline int arm_current_el(CPUARMState *env)
     }
 }
 
-typedef struct ARMCPRegInfo ARMCPRegInfo;
-
-typedef enum CPAccessResult {
-    /* Access is permitted */
-    CP_ACCESS_OK = 0,
-    /* Access fails due to a configurable trap or enable which would
-     * result in a categorized exception syndrome giving information about
-     * the failing instruction (ie syndrome category 0x3, 0x4, 0x5, 0x6,
-     * 0xc or 0x18). The exception is taken to the usual target EL (EL1 or
-     * PL1 if in EL0, otherwise to the current EL).
-     */
-    CP_ACCESS_TRAP = 1,
-    /* Access fails and results in an exception syndrome 0x0 ("uncategorized").
-     * Note that this is not a catch-all case -- the set of cases which may
-     * result in this failure is specifically defined by the architecture.
-     */
-    CP_ACCESS_TRAP_UNCATEGORIZED = 2,
-    /* As CP_ACCESS_TRAP, but for traps directly to EL2 or EL3 */
-    CP_ACCESS_TRAP_EL2 = 3,
-    CP_ACCESS_TRAP_EL3 = 4,
-    /* As CP_ACCESS_UNCATEGORIZED, but for traps directly to EL2 or EL3 */
-    CP_ACCESS_TRAP_UNCATEGORIZED_EL2 = 5,
-    CP_ACCESS_TRAP_UNCATEGORIZED_EL3 = 6,
-} CPAccessResult;
-
-/* Access functions for coprocessor registers. These cannot fail and
- * may not raise exceptions.
- */
-typedef uint64_t CPReadFn(CPUARMState *env, const ARMCPRegInfo *opaque);
-typedef void CPWriteFn(CPUARMState *env, const ARMCPRegInfo *opaque,
-                       uint64_t value);
-/* Access permission check functions for coprocessor registers. */
-typedef CPAccessResult CPAccessFn(CPUARMState *env,
-                                  const ARMCPRegInfo *opaque,
-                                  bool isread);
-/* Hook function for register reset */
-typedef void CPResetFn(CPUARMState *env, const ARMCPRegInfo *opaque);
-
-#define CP_ANY 0xff
-
-/* Definition of an ARM coprocessor register */
-struct ARMCPRegInfo {
-    /* Name of register (useful mainly for debugging, need not be unique) */
-    const char *name;
-    /* Location of register: coprocessor number and (crn,crm,opc1,opc2)
-     * tuple. Any of crm, opc1 and opc2 may be CP_ANY to indicate a
-     * 'wildcard' field -- any value of that field in the MRC/MCR insn
-     * will be decoded to this register. The register read and write
-     * callbacks will be passed an ARMCPRegInfo with the crn/crm/opc1/opc2
-     * used by the program, so it is possible to register a wildcard and
-     * then behave differently on read/write if necessary.
-     * For 64 bit registers, only crm and opc1 are relevant; crn and opc2
-     * must both be zero.
-     * For AArch64-visible registers, opc0 is also used.
-     * Since there are no "coprocessors" in AArch64, cp is purely used as a
-     * way to distinguish (for KVM's benefit) guest-visible system registers
-     * from demuxed ones provided to preserve the "no side effects on
-     * KVM register read/write from QEMU" semantics. cp==0x13 is guest
-     * visible (to match KVM's encoding); cp==0 will be converted to
-     * cp==0x13 when the ARMCPRegInfo is registered, for convenience.
-     */
-    uint8_t cp;
-    uint8_t crn;
-    uint8_t crm;
-    uint8_t opc0;
-    uint8_t opc1;
-    uint8_t opc2;
-    /* Execution state in which this register is visible: ARM_CP_STATE_* */
-    int state;
-    /* Register type: ARM_CP_* bits/values */
-    int type;
-    /* Access rights: PL*_[RW] */
-    int access;
-    /* Security state: ARM_CP_SECSTATE_* bits/values */
-    int secure;
-    /* The opaque pointer passed to define_arm_cp_regs_with_opaque() when
-     * this register was defined: can be used to hand data through to the
-     * register read/write functions, since they are passed the ARMCPRegInfo*.
-     */
-    void *opaque;
-    /* Value of this register, if it is ARM_CP_CONST. Otherwise, if
-     * fieldoffset is non-zero, the reset value of the register.
-     */
-    uint64_t resetvalue;
-    /* Offset of the field in CPUARMState for this register.
-     *
-     * This is not needed if either:
-     *  1. type is ARM_CP_CONST or one of the ARM_CP_SPECIALs
-     *  2. both readfn and writefn are specified
-     */
-    ptrdiff_t fieldoffset; /* offsetof(CPUARMState, field) */
-
-    /* Offsets of the secure and non-secure fields in CPUARMState for the
-     * register if it is banked.  These fields are only used during the static
-     * registration of a register.  During hashing the bank associated
-     * with a given security state is copied to fieldoffset which is used from
-     * there on out.
-     *
-     * It is expected that register definitions use either fieldoffset or
-     * bank_fieldoffsets in the definition but not both.  It is also expected
-     * that both bank offsets are set when defining a banked register.  This
-     * use indicates that a register is banked.
-     */
-    ptrdiff_t bank_fieldoffsets[2];
-
-    /* Function for making any access checks for this register in addition to
-     * those specified by the 'access' permissions bits. If NULL, no extra
-     * checks required. The access check is performed at runtime, not at
-     * translate time.
-     */
-    CPAccessFn *accessfn;
-    /* Function for handling reads of this register. If NULL, then reads
-     * will be done by loading from the offset into CPUARMState specified
-     * by fieldoffset.
-     */
-    CPReadFn *readfn;
-    /* Function for handling writes of this register. If NULL, then writes
-     * will be done by writing to the offset into CPUARMState specified
-     * by fieldoffset.
-     */
-    CPWriteFn *writefn;
-    /* Function for doing a "raw" read; used when we need to copy
-     * coprocessor state to the kernel for KVM or out for
-     * migration. This only needs to be provided if there is also a
-     * readfn and it has side effects (for instance clear-on-read bits).
-     */
-    CPReadFn *raw_readfn;
-    /* Function for doing a "raw" write; used when we need to copy KVM
-     * kernel coprocessor state into userspace, or for inbound
-     * migration. This only needs to be provided if there is also a
-     * writefn and it masks out "unwritable" bits or has write-one-to-clear
-     * or similar behaviour.
-     */
-    CPWriteFn *raw_writefn;
-    /* Function for resetting the register. If NULL, then reset will be done
-     * by writing resetvalue to the field specified in fieldoffset. If
-     * fieldoffset is 0 then no reset will be done.
-     */
-    CPResetFn *resetfn;
-
-    /*
-     * "Original" writefn and readfn.
-     * For ARMv8.1-VHE register aliases, we overwrite the read/write
-     * accessor functions of various EL1/EL0 to perform the runtime
-     * check for which sysreg should actually be modified, and then
-     * forwards the operation.  Before overwriting the accessors,
-     * the original function is copied here, so that accesses that
-     * really do go to the EL1/EL0 version proceed normally.
-     * (The corresponding EL2 register is linked via opaque.)
-     */
-    CPReadFn *orig_readfn;
-    CPWriteFn *orig_writefn;
-};
-
-/* Macros which are lvalues for the field in CPUARMState for the
- * ARMCPRegInfo *ri.
- */
-#define CPREG_FIELD32(env, ri) \
-    (*(uint32_t *)((char *)(env) + (ri)->fieldoffset))
-#define CPREG_FIELD64(env, ri) \
-    (*(uint64_t *)((char *)(env) + (ri)->fieldoffset))
-
-#define REGINFO_SENTINEL { .type = ARM_CP_SENTINEL }
-
-void define_arm_cp_regs_with_opaque(ARMCPU *cpu,
-                                    const ARMCPRegInfo *regs, void *opaque);
-void define_one_arm_cp_reg_with_opaque(ARMCPU *cpu,
-                                       const ARMCPRegInfo *regs, void *opaque);
-static inline void define_arm_cp_regs(ARMCPU *cpu, const ARMCPRegInfo *regs)
-{
-    define_arm_cp_regs_with_opaque(cpu, regs, 0);
-}
-static inline void define_one_arm_cp_reg(ARMCPU *cpu, const ARMCPRegInfo *regs)
-{
-    define_one_arm_cp_reg_with_opaque(cpu, regs, 0);
-}
-const ARMCPRegInfo *get_arm_cp_reginfo(GHashTable *cpregs, uint32_t encoded_cp);
-
-/*
- * Definition of an ARM co-processor register as viewed from
- * userspace. This is used for presenting sanitised versions of
- * registers to userspace when emulating the Linux AArch64 CPU
- * ID/feature ABI (advertised as HWCAP_CPUID).
- */
-typedef struct ARMCPRegUserSpaceInfo {
-    /* Name of register */
-    const char *name;
-
-    /* Is the name actually a glob pattern */
-    bool is_glob;
-
-    /* Only some bits are exported to user space */
-    uint64_t exported_bits;
-
-    /* Fixed bits are applied after the mask */
-    uint64_t fixed_bits;
-} ARMCPRegUserSpaceInfo;
-
-#define REGUSERINFO_SENTINEL { .name = NULL }
-
-void modify_arm_cp_regs(ARMCPRegInfo *regs, const ARMCPRegUserSpaceInfo *mods);
-
-/* CPWriteFn that can be used to implement writes-ignored behaviour */
-void arm_cp_write_ignore(CPUARMState *env, const ARMCPRegInfo *ri,
-                         uint64_t value);
-/* CPReadFn that can be used for read-as-zero behaviour */
-uint64_t arm_cp_read_zero(CPUARMState *env, const ARMCPRegInfo *ri);
-
-/* CPResetFn that does nothing, for use if no reset is required even
- * if fieldoffset is non zero.
- */
-void arm_cp_reset_ignore(CPUARMState *env, const ARMCPRegInfo *opaque);
-
-/* Return true if this reginfo struct's field in the cpu state struct
- * is 64 bits wide.
- */
-static inline bool cpreg_field_is_64bit(const ARMCPRegInfo *ri)
-{
-    return (ri->state == ARM_CP_STATE_AA64) || (ri->type & ARM_CP_64BIT);
-}
-
-static inline bool cp_access_ok(int current_el,
-                                const ARMCPRegInfo *ri, int isread)
-{
-    return (ri->access >> ((current_el * 2) + isread)) & 1;
-}
-
-/* Raw read of a coprocessor register (as needed for migration, etc) */
-uint64_t read_raw_cp_reg(CPUARMState *env, const ARMCPRegInfo *ri);
-
 /**
  * write_list_to_cpustate
  * @cpu: ARMCPU
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 2974cbc0d3..af5ba1d0b3 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -34,6 +34,7 @@
 #include "hvf_arm.h"
 #include "qapi/visitor.h"
 #include "hw/qdev-properties.h"
+#include "cpregs.h"
 
 
 #ifndef CONFIG_USER_ONLY
diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
index 13d0e9b195..0e693b182e 100644
--- a/target/arm/cpu_tcg.c
+++ b/target/arm/cpu_tcg.c
@@ -18,6 +18,7 @@
 #if !defined(CONFIG_USER_ONLY)
 #include "hw/boards.h"
 #endif
+#include "cpregs.h"
 
 /* CPU models. These are not needed for the AArch64 linux-user build. */
 #if !defined(CONFIG_USER_ONLY) || !defined(TARGET_AARCH64)
diff --git a/target/arm/gdbstub.c b/target/arm/gdbstub.c
index ca1de47511..f01a126108 100644
--- a/target/arm/gdbstub.c
+++ b/target/arm/gdbstub.c
@@ -19,8 +19,9 @@
  */
 #include "qemu/osdep.h"
 #include "cpu.h"
-#include "internals.h"
 #include "exec/gdbstub.h"
+#include "internals.h"
+#include "cpregs.h"
 
 typedef struct RegisterSysregXmlParam {
     CPUState *cs;
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 5a244c3ed9..3f2e555d6f 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -36,6 +36,7 @@
 #include "exec/cpu_ldst.h"
 #include "semihosting/common-semi.h"
 #endif
+#include "cpregs.h"
 
 #define ARM_CPU_FREQ 1000000000 /* FIXME: 1 GHz, should be configurable */
 #define PMCR_NUM_COUNTERS 4 /* QEMU IMPDEF choice */
diff --git a/target/arm/op_helper.c b/target/arm/op_helper.c
index 2b87e8808b..67be91c732 100644
--- a/target/arm/op_helper.c
+++ b/target/arm/op_helper.c
@@ -23,6 +23,7 @@
 #include "internals.h"
 #include "exec/exec-all.h"
 #include "exec/cpu_ldst.h"
+#include "cpregs.h"
 
 #define SIGNBIT (uint32_t)0x80000000
 #define SIGNBIT64 ((uint64_t)1 << 63)
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index a869d57309..348a638c5c 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -27,14 +27,12 @@
 #include "translate.h"
 #include "internals.h"
 #include "qemu/host-utils.h"
-
 #include "semihosting/semihost.h"
 #include "exec/gen-icount.h"
-
 #include "exec/helper-proto.h"
 #include "exec/helper-gen.h"
 #include "exec/log.h"
-
+#include "cpregs.h"
 #include "translate-a64.h"
 #include "qemu/atomic128.h"
 
diff --git a/target/arm/translate.c b/target/arm/translate.c
index 37fb17cdaa..fc7917cdf4 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -30,11 +30,10 @@
 #include "qemu/bitops.h"
 #include "arm_ldst.h"
 #include "semihosting/semihost.h"
-
 #include "exec/helper-proto.h"
 #include "exec/helper-gen.h"
-
 #include "exec/log.h"
+#include "cpregs.h"
 
 
 #define ENABLE_ARCH_4T    arm_dc_feature(s, ARM_FEATURE_V4T)

From 330477eae9416828c098513f36bd2f33f5f270fe Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:49:44 -0700
Subject: [PATCH 017/935] target/arm: Reorg CPAccessResult and
 access_check_cp_reg
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Rearrange the values of the enumerators of CPAccessResult
so that we may directly extract the target el. For the two
special cases in access_check_cp_reg, use CPAccessResult.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220501055028.646596-3-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpregs.h    | 26 ++++++++++++--------
 target/arm/op_helper.c | 56 +++++++++++++++++++++---------------------
 2 files changed, 44 insertions(+), 38 deletions(-)

diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h
index 8064c0763e..7f2c30eab1 100644
--- a/target/arm/cpregs.h
+++ b/target/arm/cpregs.h
@@ -167,26 +167,32 @@ static inline bool cptype_valid(int cptype)
 typedef enum CPAccessResult {
     /* Access is permitted */
     CP_ACCESS_OK = 0,
+
+    /*
+     * Combined with one of the following, the low 2 bits indicate the
+     * target exception level.  If 0, the exception is taken to the usual
+     * target EL (EL1 or PL1 if in EL0, otherwise to the current EL).
+     */
+    CP_ACCESS_EL_MASK = 3,
+
     /*
      * Access fails due to a configurable trap or enable which would
      * result in a categorized exception syndrome giving information about
      * the failing instruction (ie syndrome category 0x3, 0x4, 0x5, 0x6,
-     * 0xc or 0x18). The exception is taken to the usual target EL (EL1 or
-     * PL1 if in EL0, otherwise to the current EL).
+     * 0xc or 0x18).
      */
-    CP_ACCESS_TRAP = 1,
+    CP_ACCESS_TRAP = (1 << 2),
+    CP_ACCESS_TRAP_EL2 = CP_ACCESS_TRAP | 2,
+    CP_ACCESS_TRAP_EL3 = CP_ACCESS_TRAP | 3,
+
     /*
      * Access fails and results in an exception syndrome 0x0 ("uncategorized").
      * Note that this is not a catch-all case -- the set of cases which may
      * result in this failure is specifically defined by the architecture.
      */
-    CP_ACCESS_TRAP_UNCATEGORIZED = 2,
-    /* As CP_ACCESS_TRAP, but for traps directly to EL2 or EL3 */
-    CP_ACCESS_TRAP_EL2 = 3,
-    CP_ACCESS_TRAP_EL3 = 4,
-    /* As CP_ACCESS_UNCATEGORIZED, but for traps directly to EL2 or EL3 */
-    CP_ACCESS_TRAP_UNCATEGORIZED_EL2 = 5,
-    CP_ACCESS_TRAP_UNCATEGORIZED_EL3 = 6,
+    CP_ACCESS_TRAP_UNCATEGORIZED = (2 << 2),
+    CP_ACCESS_TRAP_UNCATEGORIZED_EL2 = CP_ACCESS_TRAP_UNCATEGORIZED | 2,
+    CP_ACCESS_TRAP_UNCATEGORIZED_EL3 = CP_ACCESS_TRAP_UNCATEGORIZED | 3,
 } CPAccessResult;
 
 typedef struct ARMCPRegInfo ARMCPRegInfo;
diff --git a/target/arm/op_helper.c b/target/arm/op_helper.c
index 67be91c732..76499ffa14 100644
--- a/target/arm/op_helper.c
+++ b/target/arm/op_helper.c
@@ -632,11 +632,13 @@ void HELPER(access_check_cp_reg)(CPUARMState *env, void *rip, uint32_t syndrome,
                                  uint32_t isread)
 {
     const ARMCPRegInfo *ri = rip;
+    CPAccessResult res = CP_ACCESS_OK;
     int target_el;
 
     if (arm_feature(env, ARM_FEATURE_XSCALE) && ri->cp < 14
         && extract32(env->cp15.c15_cpar, ri->cp, 1) == 0) {
-        raise_exception(env, EXCP_UDEF, syndrome, exception_target_el(env));
+        res = CP_ACCESS_TRAP;
+        goto fail;
     }
 
     /*
@@ -655,48 +657,46 @@ void HELPER(access_check_cp_reg)(CPUARMState *env, void *rip, uint32_t syndrome,
         mask &= ~((1 << 4) | (1 << 14));
 
         if (env->cp15.hstr_el2 & mask) {
-            target_el = 2;
-            goto exept;
+            res = CP_ACCESS_TRAP_EL2;
+            goto fail;
         }
     }
 
-    if (!ri->accessfn) {
+    if (ri->accessfn) {
+        res = ri->accessfn(env, ri, isread);
+    }
+    if (likely(res == CP_ACCESS_OK)) {
         return;
     }
 
-    switch (ri->accessfn(env, ri, isread)) {
-    case CP_ACCESS_OK:
-        return;
+ fail:
+    switch (res & ~CP_ACCESS_EL_MASK) {
     case CP_ACCESS_TRAP:
-        target_el = exception_target_el(env);
-        break;
-    case CP_ACCESS_TRAP_EL2:
-        /* Requesting a trap to EL2 when we're in EL3 is
-         * a bug in the access function.
-         */
-        assert(arm_current_el(env) != 3);
-        target_el = 2;
-        break;
-    case CP_ACCESS_TRAP_EL3:
-        target_el = 3;
         break;
     case CP_ACCESS_TRAP_UNCATEGORIZED:
-        target_el = exception_target_el(env);
-        syndrome = syn_uncategorized();
-        break;
-    case CP_ACCESS_TRAP_UNCATEGORIZED_EL2:
-        target_el = 2;
-        syndrome = syn_uncategorized();
-        break;
-    case CP_ACCESS_TRAP_UNCATEGORIZED_EL3:
-        target_el = 3;
         syndrome = syn_uncategorized();
         break;
     default:
         g_assert_not_reached();
     }
 
-exept:
+    target_el = res & CP_ACCESS_EL_MASK;
+    switch (target_el) {
+    case 0:
+        target_el = exception_target_el(env);
+        break;
+    case 2:
+        assert(arm_current_el(env) != 3);
+        assert(arm_is_el2_enabled(env));
+        break;
+    case 3:
+        assert(arm_feature(env, ARM_FEATURE_EL3));
+        break;
+    default:
+        /* No "direct" traps to EL1 */
+        g_assert_not_reached();
+    }
+
     raise_exception(env, EXCP_UDEF, syndrome, target_el);
 }
 

From 5809ac5709645b341eaca979715a32ced2e4d432 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:49:45 -0700
Subject: [PATCH 018/935] target/arm: Replace sentinels with ARRAY_SIZE in
 cpregs.h
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Remove a possible source of error by removing REGINFO_SENTINEL
and using ARRAY_SIZE (convinently hidden inside a macro) to
find the end of the set of regs being registered or modified.

The space saved by not having the extra array element reduces
the executable's .data.rel.ro section by about 9k.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220501055028.646596-4-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/arm/pxa2xx.c           |   1 -
 hw/arm/pxa2xx_pic.c       |   1 -
 hw/intc/arm_gicv3_cpuif.c |   5 --
 hw/intc/arm_gicv3_kvm.c   |   1 -
 target/arm/cpregs.h       |  53 +++++++++---------
 target/arm/cpu64.c        |   1 -
 target/arm/cpu_tcg.c      |   4 --
 target/arm/helper.c       | 111 ++++++++------------------------------
 8 files changed, 48 insertions(+), 129 deletions(-)

diff --git a/hw/arm/pxa2xx.c b/hw/arm/pxa2xx.c
index 0683714733..f4f687df68 100644
--- a/hw/arm/pxa2xx.c
+++ b/hw/arm/pxa2xx.c
@@ -384,7 +384,6 @@ static const ARMCPRegInfo pxa_cp_reginfo[] = {
     { .name = "PWRMODE", .cp = 14, .crn = 7, .crm = 0, .opc1 = 0, .opc2 = 0,
       .access = PL1_RW, .type = ARM_CP_IO,
       .readfn = arm_cp_read_zero, .writefn = pxa2xx_pwrmode_write },
-    REGINFO_SENTINEL
 };
 
 static void pxa2xx_setup_cp14(PXA2xxState *s)
diff --git a/hw/arm/pxa2xx_pic.c b/hw/arm/pxa2xx_pic.c
index b80d75d839..47132ab982 100644
--- a/hw/arm/pxa2xx_pic.c
+++ b/hw/arm/pxa2xx_pic.c
@@ -257,7 +257,6 @@ static const ARMCPRegInfo pxa_pic_cp_reginfo[] = {
     REGINFO_FOR_PIC_CP("ICLR2", 8),
     REGINFO_FOR_PIC_CP("ICFP2", 9),
     REGINFO_FOR_PIC_CP("ICPR2", 0xa),
-    REGINFO_SENTINEL
 };
 
 static const MemoryRegionOps pxa2xx_pic_ops = {
diff --git a/hw/intc/arm_gicv3_cpuif.c b/hw/intc/arm_gicv3_cpuif.c
index 2d5959db94..9efba798f8 100644
--- a/hw/intc/arm_gicv3_cpuif.c
+++ b/hw/intc/arm_gicv3_cpuif.c
@@ -2428,7 +2428,6 @@ static const ARMCPRegInfo gicv3_cpuif_reginfo[] = {
       .readfn = icc_igrpen1_el3_read,
       .writefn = icc_igrpen1_el3_write,
     },
-    REGINFO_SENTINEL
 };
 
 static uint64_t ich_ap_read(CPUARMState *env, const ARMCPRegInfo *ri)
@@ -2682,7 +2681,6 @@ static const ARMCPRegInfo gicv3_cpuif_hcr_reginfo[] = {
       .readfn = ich_vmcr_read,
       .writefn = ich_vmcr_write,
     },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo gicv3_cpuif_ich_apxr1_reginfo[] = {
@@ -2700,7 +2698,6 @@ static const ARMCPRegInfo gicv3_cpuif_ich_apxr1_reginfo[] = {
       .readfn = ich_ap_read,
       .writefn = ich_ap_write,
     },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo gicv3_cpuif_ich_apxr23_reginfo[] = {
@@ -2732,7 +2729,6 @@ static const ARMCPRegInfo gicv3_cpuif_ich_apxr23_reginfo[] = {
       .readfn = ich_ap_read,
       .writefn = ich_ap_write,
     },
-    REGINFO_SENTINEL
 };
 
 static void gicv3_cpuif_el_change_hook(ARMCPU *cpu, void *opaque)
@@ -2807,7 +2803,6 @@ void gicv3_init_cpuif(GICv3State *s)
                       .readfn = ich_lr_read,
                       .writefn = ich_lr_write,
                     },
-                    REGINFO_SENTINEL
                 };
                 define_arm_cp_regs(cpu, lr_regset);
             }
diff --git a/hw/intc/arm_gicv3_kvm.c b/hw/intc/arm_gicv3_kvm.c
index 611085e98d..2922c516e5 100644
--- a/hw/intc/arm_gicv3_kvm.c
+++ b/hw/intc/arm_gicv3_kvm.c
@@ -735,7 +735,6 @@ static const ARMCPRegInfo gicv3_cpuif_reginfo[] = {
        */
       .resetfn = arm_gicv3_icc_reset,
     },
-    REGINFO_SENTINEL
 };
 
 /**
diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h
index 7f2c30eab1..a5231504d5 100644
--- a/target/arm/cpregs.h
+++ b/target/arm/cpregs.h
@@ -71,8 +71,6 @@
 #define ARM_CP_NO_GDB            0x4000
 #define ARM_CP_RAISES_EXC        0x8000
 #define ARM_CP_NEWEL             0x10000
-/* Used only as a terminator for ARMCPRegInfo lists */
-#define ARM_CP_SENTINEL          0xfffff
 /* Mask of only the flag bits in a type field */
 #define ARM_CP_FLAG_MASK         0x1f0ff
 
@@ -108,18 +106,6 @@ enum {
     ARM_CP_SECSTATE_NS =  (1 << 1), /* bit[1]: Non-secure state register */
 };
 
-/*
- * Return true if cptype is a valid type field. This is used to try to
- * catch errors where the sentinel has been accidentally left off the end
- * of a list of registers.
- */
-static inline bool cptype_valid(int cptype)
-{
-    return ((cptype & ~ARM_CP_FLAG_MASK) == 0)
-        || ((cptype & ARM_CP_SPECIAL) &&
-            ((cptype & ~ARM_CP_FLAG_MASK) <= ARM_LAST_SPECIAL));
-}
-
 /*
  * Access rights:
  * We define bits for Read and Write access for what rev C of the v7-AR ARM ARM
@@ -346,20 +332,27 @@ struct ARMCPRegInfo {
 #define CPREG_FIELD64(env, ri) \
     (*(uint64_t *)((char *)(env) + (ri)->fieldoffset))
 
-#define REGINFO_SENTINEL { .type = ARM_CP_SENTINEL }
+void define_one_arm_cp_reg_with_opaque(ARMCPU *cpu, const ARMCPRegInfo *reg,
+                                       void *opaque);
 
-void define_arm_cp_regs_with_opaque(ARMCPU *cpu,
-                                    const ARMCPRegInfo *regs, void *opaque);
-void define_one_arm_cp_reg_with_opaque(ARMCPU *cpu,
-                                       const ARMCPRegInfo *regs, void *opaque);
-static inline void define_arm_cp_regs(ARMCPU *cpu, const ARMCPRegInfo *regs)
-{
-    define_arm_cp_regs_with_opaque(cpu, regs, 0);
-}
 static inline void define_one_arm_cp_reg(ARMCPU *cpu, const ARMCPRegInfo *regs)
 {
-    define_one_arm_cp_reg_with_opaque(cpu, regs, 0);
+    define_one_arm_cp_reg_with_opaque(cpu, regs, NULL);
 }
+
+void define_arm_cp_regs_with_opaque_len(ARMCPU *cpu, const ARMCPRegInfo *regs,
+                                        void *opaque, size_t len);
+
+#define define_arm_cp_regs_with_opaque(CPU, REGS, OPAQUE)               \
+    do {                                                                \
+        QEMU_BUILD_BUG_ON(ARRAY_SIZE(REGS) == 0);                       \
+        define_arm_cp_regs_with_opaque_len(CPU, REGS, OPAQUE,           \
+                                           ARRAY_SIZE(REGS));           \
+    } while (0)
+
+#define define_arm_cp_regs(CPU, REGS) \
+    define_arm_cp_regs_with_opaque(CPU, REGS, NULL)
+
 const ARMCPRegInfo *get_arm_cp_reginfo(GHashTable *cpregs, uint32_t encoded_cp);
 
 /*
@@ -382,9 +375,17 @@ typedef struct ARMCPRegUserSpaceInfo {
     uint64_t fixed_bits;
 } ARMCPRegUserSpaceInfo;
 
-#define REGUSERINFO_SENTINEL { .name = NULL }
+void modify_arm_cp_regs_with_len(ARMCPRegInfo *regs, size_t regs_len,
+                                 const ARMCPRegUserSpaceInfo *mods,
+                                 size_t mods_len);
 
-void modify_arm_cp_regs(ARMCPRegInfo *regs, const ARMCPRegUserSpaceInfo *mods);
+#define modify_arm_cp_regs(REGS, MODS)                                  \
+    do {                                                                \
+        QEMU_BUILD_BUG_ON(ARRAY_SIZE(REGS) == 0);                       \
+        QEMU_BUILD_BUG_ON(ARRAY_SIZE(MODS) == 0);                       \
+        modify_arm_cp_regs_with_len(REGS, ARRAY_SIZE(REGS),             \
+                                    MODS, ARRAY_SIZE(MODS));            \
+    } while (0)
 
 /* CPWriteFn that can be used to implement writes-ignored behaviour */
 void arm_cp_write_ignore(CPUARMState *env, const ARMCPRegInfo *ri,
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index af5ba1d0b3..c841d55d0e 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -91,7 +91,6 @@ static const ARMCPRegInfo cortex_a72_a57_a53_cp_reginfo[] = {
     { .name = "L2MERRSR",
       .cp = 15, .opc1 = 3, .crm = 15,
       .access = PL1_RW, .type = ARM_CP_CONST | ARM_CP_64BIT, .resetvalue = 0 },
-    REGINFO_SENTINEL
 };
 
 static void aarch64_a57_initfn(Object *obj)
diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
index 0e693b182e..9338088b22 100644
--- a/target/arm/cpu_tcg.c
+++ b/target/arm/cpu_tcg.c
@@ -264,7 +264,6 @@ static const ARMCPRegInfo cortexa8_cp_reginfo[] = {
       .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
     { .name = "L2AUXCR", .cp = 15, .crn = 9, .crm = 0, .opc1 = 1, .opc2 = 2,
       .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    REGINFO_SENTINEL
 };
 
 static void cortex_a8_initfn(Object *obj)
@@ -332,7 +331,6 @@ static const ARMCPRegInfo cortexa9_cp_reginfo[] = {
       .access = PL1_RW, .resetvalue = 0, .type = ARM_CP_CONST },
     { .name = "TLB_ATTR", .cp = 15, .crn = 15, .crm = 7, .opc1 = 5, .opc2 = 2,
       .access = PL1_RW, .resetvalue = 0, .type = ARM_CP_CONST },
-    REGINFO_SENTINEL
 };
 
 static void cortex_a9_initfn(Object *obj)
@@ -398,7 +396,6 @@ static const ARMCPRegInfo cortexa15_cp_reginfo[] = {
 #endif
     { .name = "L2ECTLR", .cp = 15, .crn = 9, .crm = 0, .opc1 = 1, .opc2 = 3,
       .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    REGINFO_SENTINEL
 };
 
 static void cortex_a7_initfn(Object *obj)
@@ -686,7 +683,6 @@ static const ARMCPRegInfo cortexr5_cp_reginfo[] = {
       .access = PL1_RW, .type = ARM_CP_CONST },
     { .name = "DCACHE_INVAL", .cp = 15, .opc1 = 0, .crn = 15, .crm = 5,
       .opc2 = 0, .access = PL1_W, .type = ARM_CP_NOP },
-    REGINFO_SENTINEL
 };
 
 static void cortex_r5_initfn(Object *obj)
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 3f2e555d6f..a68f14fe8e 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -673,7 +673,6 @@ static const ARMCPRegInfo cp_reginfo[] = {
       .secure = ARM_CP_SECSTATE_S,
       .fieldoffset = offsetof(CPUARMState, cp15.contextidr_s),
       .resetvalue = 0, .writefn = contextidr_write, .raw_writefn = raw_write, },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo not_v8_cp_reginfo[] = {
@@ -702,7 +701,6 @@ static const ARMCPRegInfo not_v8_cp_reginfo[] = {
     { .name = "CACHEMAINT", .cp = 15, .crn = 7, .crm = CP_ANY,
       .opc1 = 0, .opc2 = CP_ANY, .access = PL1_W,
       .type = ARM_CP_NOP | ARM_CP_OVERRIDE },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo not_v6_cp_reginfo[] = {
@@ -711,7 +709,6 @@ static const ARMCPRegInfo not_v6_cp_reginfo[] = {
      */
     { .name = "WFI_v5", .cp = 15, .crn = 7, .crm = 8, .opc1 = 0, .opc2 = 2,
       .access = PL1_W, .type = ARM_CP_WFI },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo not_v7_cp_reginfo[] = {
@@ -760,7 +757,6 @@ static const ARMCPRegInfo not_v7_cp_reginfo[] = {
       .opc1 = 0, .opc2 = 0, .access = PL1_RW, .type = ARM_CP_NOP },
     { .name = "NMRR", .cp = 15, .crn = 10, .crm = 2,
       .opc1 = 0, .opc2 = 1, .access = PL1_RW, .type = ARM_CP_NOP },
-    REGINFO_SENTINEL
 };
 
 static void cpacr_write(CPUARMState *env, const ARMCPRegInfo *ri,
@@ -889,7 +885,6 @@ static const ARMCPRegInfo v6_cp_reginfo[] = {
       .crn = 1, .crm = 0, .opc1 = 0, .opc2 = 2, .accessfn = cpacr_access,
       .access = PL1_RW, .fieldoffset = offsetof(CPUARMState, cp15.cpacr_el1),
       .resetfn = cpacr_reset, .writefn = cpacr_write, .readfn = cpacr_read },
-    REGINFO_SENTINEL
 };
 
 typedef struct pm_event {
@@ -2135,7 +2130,6 @@ static const ARMCPRegInfo v7_cp_reginfo[] = {
     { .name = "TLBIMVAA", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 3,
       .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb,
       .writefn = tlbimvaa_write },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo v7mp_cp_reginfo[] = {
@@ -2152,7 +2146,6 @@ static const ARMCPRegInfo v7mp_cp_reginfo[] = {
     { .name = "TLBIMVAAIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 3,
       .type = ARM_CP_NO_RAW, .access = PL1_W, .accessfn = access_ttlb,
       .writefn = tlbimvaa_is_write },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo pmovsset_cp_reginfo[] = {
@@ -2170,7 +2163,6 @@ static const ARMCPRegInfo pmovsset_cp_reginfo[] = {
       .fieldoffset = offsetof(CPUARMState, cp15.c9_pmovsr),
       .writefn = pmovsset_write,
       .raw_writefn = raw_write },
-    REGINFO_SENTINEL
 };
 
 static void teecr_write(CPUARMState *env, const ARMCPRegInfo *ri,
@@ -2211,7 +2203,6 @@ static const ARMCPRegInfo t2ee_cp_reginfo[] = {
     { .name = "TEEHBR", .cp = 14, .crn = 1, .crm = 0, .opc1 = 6, .opc2 = 0,
       .access = PL0_RW, .fieldoffset = offsetof(CPUARMState, teehbr),
       .accessfn = teehbr_access, .resetvalue = 0 },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo v6k_cp_reginfo[] = {
@@ -2243,7 +2234,6 @@ static const ARMCPRegInfo v6k_cp_reginfo[] = {
       .bank_fieldoffsets = { offsetoflow32(CPUARMState, cp15.tpidrprw_s),
                              offsetoflow32(CPUARMState, cp15.tpidrprw_ns) },
       .resetvalue = 0 },
-    REGINFO_SENTINEL
 };
 
 #ifndef CONFIG_USER_ONLY
@@ -3091,7 +3081,6 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
       .fieldoffset = offsetof(CPUARMState, cp15.c14_timer[GTIMER_SEC].cval),
       .writefn = gt_sec_cval_write, .raw_writefn = raw_write,
     },
-    REGINFO_SENTINEL
 };
 
 static CPAccessResult e2h_access(CPUARMState *env, const ARMCPRegInfo *ri,
@@ -3132,7 +3121,6 @@ static const ARMCPRegInfo generic_timer_cp_reginfo[] = {
       .access = PL0_R, .type = ARM_CP_NO_RAW | ARM_CP_IO,
       .readfn = gt_virt_cnt_read,
     },
-    REGINFO_SENTINEL
 };
 
 #endif
@@ -3496,7 +3484,6 @@ static const ARMCPRegInfo vapa_cp_reginfo[] = {
       .access = PL1_W, .accessfn = ats_access,
       .writefn = ats_write, .type = ARM_CP_NO_RAW | ARM_CP_RAISES_EXC },
 #endif
-    REGINFO_SENTINEL
 };
 
 /* Return basic MPU access permission bits.  */
@@ -3619,7 +3606,6 @@ static const ARMCPRegInfo pmsav7_cp_reginfo[] = {
       .fieldoffset = offsetof(CPUARMState, pmsav7.rnr[M_REG_NS]),
       .writefn = pmsav7_rgnr_write,
       .resetfn = arm_cp_reset_ignore },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo pmsav5_cp_reginfo[] = {
@@ -3670,7 +3656,6 @@ static const ARMCPRegInfo pmsav5_cp_reginfo[] = {
     { .name = "946_PRBS7", .cp = 15, .crn = 6, .crm = 7, .opc1 = 0,
       .opc2 = CP_ANY, .access = PL1_RW, .resetvalue = 0,
       .fieldoffset = offsetof(CPUARMState, cp15.c6_region[7]) },
-    REGINFO_SENTINEL
 };
 
 static void vmsa_ttbcr_raw_write(CPUARMState *env, const ARMCPRegInfo *ri,
@@ -3824,7 +3809,6 @@ static const ARMCPRegInfo vmsa_pmsa_cp_reginfo[] = {
       .access = PL1_RW, .accessfn = access_tvm_trvm,
       .fieldoffset = offsetof(CPUARMState, cp15.far_el[1]),
       .resetvalue = 0, },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo vmsa_cp_reginfo[] = {
@@ -3857,7 +3841,6 @@ static const ARMCPRegInfo vmsa_cp_reginfo[] = {
       /* No offsetoflow32 -- pass the entire TCR to writefn/raw_writefn. */
       .bank_fieldoffsets = { offsetof(CPUARMState, cp15.tcr_el[3]),
                              offsetof(CPUARMState, cp15.tcr_el[1])} },
-    REGINFO_SENTINEL
 };
 
 /* Note that unlike TTBCR, writing to TTBCR2 does not require flushing
@@ -3942,7 +3925,6 @@ static const ARMCPRegInfo omap_cp_reginfo[] = {
     { .name = "C9", .cp = 15, .crn = 9,
       .crm = CP_ANY, .opc1 = CP_ANY, .opc2 = CP_ANY, .access = PL1_RW,
       .type = ARM_CP_CONST | ARM_CP_OVERRIDE, .resetvalue = 0 },
-    REGINFO_SENTINEL
 };
 
 static void xscale_cpar_write(CPUARMState *env, const ARMCPRegInfo *ri,
@@ -3975,7 +3957,6 @@ static const ARMCPRegInfo xscale_cp_reginfo[] = {
     { .name = "XSCALE_UNLOCK_DCACHE",
       .cp = 15, .opc1 = 0, .crn = 9, .crm = 2, .opc2 = 1,
       .access = PL1_W, .type = ARM_CP_NOP },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo dummy_c15_cp_reginfo[] = {
@@ -3989,7 +3970,6 @@ static const ARMCPRegInfo dummy_c15_cp_reginfo[] = {
       .access = PL1_RW,
       .type = ARM_CP_CONST | ARM_CP_NO_RAW | ARM_CP_OVERRIDE,
       .resetvalue = 0 },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo cache_dirty_status_cp_reginfo[] = {
@@ -3997,7 +3977,6 @@ static const ARMCPRegInfo cache_dirty_status_cp_reginfo[] = {
     { .name = "CDSR", .cp = 15, .crn = 7, .crm = 10, .opc1 = 0, .opc2 = 6,
       .access = PL1_R, .type = ARM_CP_CONST | ARM_CP_NO_RAW,
       .resetvalue = 0 },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo cache_block_ops_cp_reginfo[] = {
@@ -4018,7 +3997,6 @@ static const ARMCPRegInfo cache_block_ops_cp_reginfo[] = {
       .access = PL0_W, .type = ARM_CP_NOP|ARM_CP_64BIT },
     { .name = "CIDCR", .cp = 15, .crm = 14, .opc1 = 0,
       .access = PL1_W, .type = ARM_CP_NOP|ARM_CP_64BIT },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo cache_test_clean_cp_reginfo[] = {
@@ -4031,7 +4009,6 @@ static const ARMCPRegInfo cache_test_clean_cp_reginfo[] = {
     { .name = "TCI_DCACHE", .cp = 15, .crn = 7, .crm = 14, .opc1 = 0, .opc2 = 3,
       .access = PL0_R, .type = ARM_CP_CONST | ARM_CP_NO_RAW,
       .resetvalue = (1 << 30) },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo strongarm_cp_reginfo[] = {
@@ -4040,7 +4017,6 @@ static const ARMCPRegInfo strongarm_cp_reginfo[] = {
       .crm = CP_ANY, .opc1 = CP_ANY, .opc2 = CP_ANY,
       .access = PL1_RW, .resetvalue = 0,
       .type = ARM_CP_CONST | ARM_CP_OVERRIDE | ARM_CP_NO_RAW },
-    REGINFO_SENTINEL
 };
 
 static uint64_t midr_read(CPUARMState *env, const ARMCPRegInfo *ri)
@@ -4107,7 +4083,6 @@ static const ARMCPRegInfo lpae_cp_reginfo[] = {
       .bank_fieldoffsets = { offsetof(CPUARMState, cp15.ttbr1_s),
                              offsetof(CPUARMState, cp15.ttbr1_ns) },
       .writefn = vmsa_ttbr_write, },
-    REGINFO_SENTINEL
 };
 
 static uint64_t aa64_fpcr_read(CPUARMState *env, const ARMCPRegInfo *ri)
@@ -5126,7 +5101,6 @@ static const ARMCPRegInfo v8_cp_reginfo[] = {
       .access = PL1_RW, .accessfn = access_trap_aa32s_el1,
       .writefn = sdcr_write,
       .fieldoffset = offsetoflow32(CPUARMState, cp15.mdcr_el3) },
-    REGINFO_SENTINEL
 };
 
 /* Used to describe the behaviour of EL2 regs when EL2 does not exist.  */
@@ -5237,7 +5211,6 @@ static const ARMCPRegInfo el3_no_el2_cp_reginfo[] = {
       .type = ARM_CP_CONST,
       .cp = 15, .opc1 = 4, .crn = 6, .crm = 0, .opc2 = 2,
       .access = PL2_RW, .resetvalue = 0 },
-    REGINFO_SENTINEL
 };
 
 /* Ditto, but for registers which exist in ARMv8 but not v7 */
@@ -5246,7 +5219,6 @@ static const ARMCPRegInfo el3_no_el2_v8_cp_reginfo[] = {
       .cp = 15, .opc1 = 4, .crn = 1, .crm = 1, .opc2 = 4,
       .access = PL2_RW,
       .type = ARM_CP_CONST, .resetvalue = 0 },
-    REGINFO_SENTINEL
 };
 
 static void do_hcr_write(CPUARMState *env, uint64_t value, uint64_t valid_mask)
@@ -5679,7 +5651,6 @@ static const ARMCPRegInfo el2_cp_reginfo[] = {
       .cp = 15, .opc0 = 3, .opc1 = 4, .crn = 1, .crm = 1, .opc2 = 3,
       .access = PL2_RW,
       .fieldoffset = offsetof(CPUARMState, cp15.hstr_el2) },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo el2_v8_cp_reginfo[] = {
@@ -5689,7 +5660,6 @@ static const ARMCPRegInfo el2_v8_cp_reginfo[] = {
       .access = PL2_RW,
       .fieldoffset = offsetofhigh32(CPUARMState, cp15.hcr_el2),
       .writefn = hcr_writehigh },
-    REGINFO_SENTINEL
 };
 
 static CPAccessResult sel2_access(CPUARMState *env, const ARMCPRegInfo *ri,
@@ -5710,7 +5680,6 @@ static const ARMCPRegInfo el2_sec_cp_reginfo[] = {
       .opc0 = 3, .opc1 = 4, .crn = 2, .crm = 6, .opc2 = 2,
       .access = PL2_RW, .accessfn = sel2_access,
       .fieldoffset = offsetof(CPUARMState, cp15.vstcr_el2) },
-    REGINFO_SENTINEL
 };
 
 static CPAccessResult nsacr_access(CPUARMState *env, const ARMCPRegInfo *ri,
@@ -5836,7 +5805,6 @@ static const ARMCPRegInfo el3_cp_reginfo[] = {
       .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 5,
       .access = PL3_W, .type = ARM_CP_NO_RAW,
       .writefn = tlbi_aa64_vae3_write },
-    REGINFO_SENTINEL
 };
 
 #ifndef CONFIG_USER_ONLY
@@ -6122,7 +6090,6 @@ static const ARMCPRegInfo debug_cp_reginfo[] = {
       .cp = 14, .opc0 = 2, .opc1 = 0, .crn = 0, .crm = 2, .opc2 = 0,
       .access = PL1_RW, .accessfn = access_tda,
       .type = ARM_CP_NOP },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo debug_lpae_cp_reginfo[] = {
@@ -6131,7 +6098,6 @@ static const ARMCPRegInfo debug_lpae_cp_reginfo[] = {
       .access = PL0_R, .type = ARM_CP_CONST|ARM_CP_64BIT, .resetvalue = 0 },
     { .name = "DBGDSAR", .cp = 14, .crm = 2, .opc1 = 0,
       .access = PL0_R, .type = ARM_CP_CONST|ARM_CP_64BIT, .resetvalue = 0 },
-    REGINFO_SENTINEL
 };
 
 /* Return the exception level to which exceptions should be taken
@@ -6617,7 +6583,6 @@ static void define_debug_regs(ARMCPU *cpu)
               .fieldoffset = offsetof(CPUARMState, cp15.dbgbcr[i]),
               .writefn = dbgbcr_write, .raw_writefn = raw_write
             },
-            REGINFO_SENTINEL
         };
         define_arm_cp_regs(cpu, dbgregs);
     }
@@ -6636,7 +6601,6 @@ static void define_debug_regs(ARMCPU *cpu)
               .fieldoffset = offsetof(CPUARMState, cp15.dbgwcr[i]),
               .writefn = dbgwcr_write, .raw_writefn = raw_write
             },
-            REGINFO_SENTINEL
         };
         define_arm_cp_regs(cpu, dbgregs);
     }
@@ -6699,7 +6663,6 @@ static void define_pmu_regs(ARMCPU *cpu)
               .type = ARM_CP_IO,
               .readfn = pmevtyper_readfn, .writefn = pmevtyper_writefn,
               .raw_writefn = pmevtyper_rawwrite },
-            REGINFO_SENTINEL
         };
         define_arm_cp_regs(cpu, pmev_regs);
         g_free(pmevcntr_name);
@@ -6717,7 +6680,6 @@ static void define_pmu_regs(ARMCPU *cpu)
               .cp = 15, .opc1 = 0, .crn = 9, .crm = 14, .opc2 = 5,
               .access = PL0_R, .accessfn = pmreg_access, .type = ARM_CP_CONST,
               .resetvalue = extract64(cpu->pmceid1, 32, 32) },
-            REGINFO_SENTINEL
         };
         define_arm_cp_regs(cpu, v81_pmu_regs);
     }
@@ -6814,7 +6776,6 @@ static const ARMCPRegInfo lor_reginfo[] = {
       .opc0 = 3, .opc1 = 0, .crn = 10, .crm = 4, .opc2 = 7,
       .access = PL1_R, .accessfn = access_lor_ns,
       .type = ARM_CP_CONST, .resetvalue = 0 },
-    REGINFO_SENTINEL
 };
 
 #ifdef TARGET_AARCH64
@@ -6877,7 +6838,6 @@ static const ARMCPRegInfo pauth_reginfo[] = {
       .opc0 = 3, .opc1 = 0, .crn = 2, .crm = 1, .opc2 = 3,
       .access = PL1_RW, .accessfn = access_pauth,
       .fieldoffset = offsetof(CPUARMState, keys.apib.hi) },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo tlbirange_reginfo[] = {
@@ -6989,7 +6949,6 @@ static const ARMCPRegInfo tlbirange_reginfo[] = {
       .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 6, .opc2 = 5,
       .access = PL3_W, .type = ARM_CP_NO_RAW,
       .writefn = tlbi_aa64_rvae3_write },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo tlbios_reginfo[] = {
@@ -7061,7 +7020,6 @@ static const ARMCPRegInfo tlbios_reginfo[] = {
       .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 1, .opc2 = 5,
       .access = PL3_W, .type = ARM_CP_NO_RAW,
       .writefn = tlbi_aa64_vae3is_write },
-    REGINFO_SENTINEL
 };
 
 static uint64_t rndr_readfn(CPUARMState *env, const ARMCPRegInfo *ri)
@@ -7100,7 +7058,6 @@ static const ARMCPRegInfo rndr_reginfo[] = {
       .type = ARM_CP_NO_RAW | ARM_CP_SUPPRESS_TB_END | ARM_CP_IO,
       .opc0 = 3, .opc1 = 3, .crn = 2, .crm = 4, .opc2 = 1,
       .access = PL0_R, .readfn = rndr_readfn },
-    REGINFO_SENTINEL
 };
 
 #ifndef CONFIG_USER_ONLY
@@ -7136,7 +7093,6 @@ static const ARMCPRegInfo dcpop_reg[] = {
       .opc0 = 1, .opc1 = 3, .crn = 7, .crm = 12, .opc2 = 1,
       .access = PL0_W, .type = ARM_CP_NO_RAW | ARM_CP_SUPPRESS_TB_END,
       .accessfn = aa64_cacheop_poc_access, .writefn = dccvap_writefn },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo dcpodp_reg[] = {
@@ -7144,7 +7100,6 @@ static const ARMCPRegInfo dcpodp_reg[] = {
       .opc0 = 1, .opc1 = 3, .crn = 7, .crm = 13, .opc2 = 1,
       .access = PL0_W, .type = ARM_CP_NO_RAW | ARM_CP_SUPPRESS_TB_END,
       .accessfn = aa64_cacheop_poc_access, .writefn = dccvap_writefn },
-    REGINFO_SENTINEL
 };
 #endif /*CONFIG_USER_ONLY*/
 
@@ -7246,14 +7201,12 @@ static const ARMCPRegInfo mte_reginfo[] = {
     { .name = "DC_CIGDSW", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 0, .crn = 7, .crm = 14, .opc2 = 6,
       .type = ARM_CP_NOP, .access = PL1_W, .accessfn = access_tsw },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo mte_tco_ro_reginfo[] = {
     { .name = "TCO", .state = ARM_CP_STATE_AA64,
       .opc0 = 3, .opc1 = 3, .crn = 4, .crm = 2, .opc2 = 7,
       .type = ARM_CP_CONST, .access = PL0_RW, },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo mte_el0_cacheop_reginfo[] = {
@@ -7305,7 +7258,6 @@ static const ARMCPRegInfo mte_el0_cacheop_reginfo[] = {
       .accessfn = aa64_zva_access,
 #endif
     },
-    REGINFO_SENTINEL
 };
 
 #endif
@@ -7351,7 +7303,6 @@ static const ARMCPRegInfo predinv_reginfo[] = {
     { .name = "CPPRCTX", .state = ARM_CP_STATE_AA32,
       .cp = 15, .opc1 = 0, .crn = 7, .crm = 3, .opc2 = 7,
       .type = ARM_CP_NOP, .access = PL0_W, .accessfn = access_predinv },
-    REGINFO_SENTINEL
 };
 
 static uint64_t ccsidr2_read(CPUARMState *env, const ARMCPRegInfo *ri)
@@ -7366,7 +7317,6 @@ static const ARMCPRegInfo ccsidr2_reginfo[] = {
       .access = PL1_R,
       .accessfn = access_aa64_tid2,
       .readfn = ccsidr2_read, .type = ARM_CP_NO_RAW },
-    REGINFO_SENTINEL
 };
 
 static CPAccessResult access_aa64_tid3(CPUARMState *env, const ARMCPRegInfo *ri,
@@ -7427,7 +7377,6 @@ static const ARMCPRegInfo jazelle_regs[] = {
       .cp = 14, .crn = 2, .crm = 0, .opc1 = 7, .opc2 = 0,
       .accessfn = access_joscr_jmcr,
       .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo vhe_reginfo[] = {
@@ -7492,7 +7441,6 @@ static const ARMCPRegInfo vhe_reginfo[] = {
       .access = PL2_RW, .accessfn = e2h_access,
       .writefn = gt_virt_cval_write, .raw_writefn = raw_write },
 #endif
-    REGINFO_SENTINEL
 };
 
 #ifndef CONFIG_USER_ONLY
@@ -7505,7 +7453,6 @@ static const ARMCPRegInfo ats1e1_reginfo[] = {
       .opc0 = 1, .opc1 = 0, .crn = 7, .crm = 9, .opc2 = 1,
       .access = PL1_W, .type = ARM_CP_NO_RAW | ARM_CP_RAISES_EXC,
       .writefn = ats_write64 },
-    REGINFO_SENTINEL
 };
 
 static const ARMCPRegInfo ats1cp_reginfo[] = {
@@ -7517,7 +7464,6 @@ static const ARMCPRegInfo ats1cp_reginfo[] = {
       .cp = 15, .opc1 = 0, .crn = 7, .crm = 9, .opc2 = 1,
       .access = PL1_W, .type = ARM_CP_NO_RAW | ARM_CP_RAISES_EXC,
       .writefn = ats_write },
-    REGINFO_SENTINEL
 };
 #endif
 
@@ -7539,7 +7485,6 @@ static const ARMCPRegInfo actlr2_hactlr2_reginfo[] = {
       .cp = 15, .opc1 = 4, .crn = 1, .crm = 0, .opc2 = 3,
       .access = PL2_RW, .type = ARM_CP_CONST,
       .resetvalue = 0 },
-    REGINFO_SENTINEL
 };
 
 void register_cp_regs_for_features(ARMCPU *cpu)
@@ -7646,7 +7591,6 @@ void register_cp_regs_for_features(ARMCPU *cpu)
               .access = PL1_R, .type = ARM_CP_CONST,
               .accessfn = access_aa32_tid3,
               .resetvalue = cpu->isar.id_isar6 },
-            REGINFO_SENTINEL
         };
         define_arm_cp_regs(cpu, v6_idregs);
         define_arm_cp_regs(cpu, v6_cp_reginfo);
@@ -7914,7 +7858,6 @@ void register_cp_regs_for_features(ARMCPU *cpu)
               .opc0 = 3, .opc1 = 3, .crn = 9, .crm = 12, .opc2 = 7,
               .access = PL0_R, .accessfn = pmreg_access, .type = ARM_CP_CONST,
               .resetvalue = cpu->pmceid1 },
-            REGINFO_SENTINEL
         };
 #ifdef CONFIG_USER_ONLY
         ARMCPRegUserSpaceInfo v8_user_idregs[] = {
@@ -7944,7 +7887,6 @@ void register_cp_regs_for_features(ARMCPU *cpu)
               .exported_bits = 0x000000f0ffffffff },
             { .name = "ID_AA64ISAR*_EL1_RESERVED",
               .is_glob = true                     },
-            REGUSERINFO_SENTINEL
         };
         modify_arm_cp_regs(v8_idregs, v8_user_idregs);
 #endif
@@ -7984,7 +7926,6 @@ void register_cp_regs_for_features(ARMCPU *cpu)
               .access = PL2_RW,
               .resetvalue = vmpidr_def,
               .fieldoffset = offsetof(CPUARMState, cp15.vmpidr_el2) },
-            REGINFO_SENTINEL
         };
         define_arm_cp_regs(cpu, vpidr_regs);
         define_arm_cp_regs(cpu, el2_cp_reginfo);
@@ -8023,7 +7964,6 @@ void register_cp_regs_for_features(ARMCPU *cpu)
                   .access = PL2_RW, .accessfn = access_el3_aa32ns,
                   .type = ARM_CP_NO_RAW,
                   .writefn = arm_cp_write_ignore, .readfn = mpidr_read },
-                REGINFO_SENTINEL
             };
             define_arm_cp_regs(cpu, vpidr_regs);
             define_arm_cp_regs(cpu, el3_no_el2_cp_reginfo);
@@ -8046,7 +7986,6 @@ void register_cp_regs_for_features(ARMCPU *cpu)
               .raw_writefn = raw_write, .writefn = sctlr_write,
               .fieldoffset = offsetof(CPUARMState, cp15.sctlr_el[3]),
               .resetvalue = cpu->reset_sctlr },
-            REGINFO_SENTINEL
         };
 
         define_arm_cp_regs(cpu, el3_regs);
@@ -8181,7 +8120,6 @@ void register_cp_regs_for_features(ARMCPU *cpu)
             { .name = "DUMMY",
               .cp = 15, .crn = 0, .crm = 7, .opc1 = 0, .opc2 = CP_ANY,
               .access = PL1_R, .type = ARM_CP_CONST, .resetvalue = 0 },
-            REGINFO_SENTINEL
         };
         ARMCPRegInfo id_v8_midr_cp_reginfo[] = {
             { .name = "MIDR_EL1", .state = ARM_CP_STATE_BOTH,
@@ -8201,7 +8139,6 @@ void register_cp_regs_for_features(ARMCPU *cpu)
               .access = PL1_R,
               .accessfn = access_aa64_tid1,
               .type = ARM_CP_CONST, .resetvalue = cpu->revidr },
-            REGINFO_SENTINEL
         };
         ARMCPRegInfo id_cp_reginfo[] = {
             /* These are common to v8 and pre-v8 */
@@ -8219,7 +8156,6 @@ void register_cp_regs_for_features(ARMCPU *cpu)
               .access = PL1_R,
               .accessfn = access_aa32_tid1,
               .type = ARM_CP_CONST, .resetvalue = 0 },
-            REGINFO_SENTINEL
         };
         /* TLBTR is specific to VMSA */
         ARMCPRegInfo id_tlbtr_reginfo = {
@@ -8246,25 +8182,23 @@ void register_cp_regs_for_features(ARMCPU *cpu)
             { .name = "MIDR_EL1",
               .exported_bits = 0x00000000ffffffff },
             { .name = "REVIDR_EL1"                },
-            REGUSERINFO_SENTINEL
         };
         modify_arm_cp_regs(id_v8_midr_cp_reginfo, id_v8_user_midr_cp_reginfo);
 #endif
         if (arm_feature(env, ARM_FEATURE_OMAPCP) ||
             arm_feature(env, ARM_FEATURE_STRONGARM)) {
-            ARMCPRegInfo *r;
+            size_t i;
             /* Register the blanket "writes ignored" value first to cover the
              * whole space. Then update the specific ID registers to allow write
              * access, so that they ignore writes rather than causing them to
              * UNDEF.
              */
             define_one_arm_cp_reg(cpu, &crn0_wi_reginfo);
-            for (r = id_pre_v8_midr_cp_reginfo;
-                 r->type != ARM_CP_SENTINEL; r++) {
-                r->access = PL1_RW;
+            for (i = 0; i < ARRAY_SIZE(id_pre_v8_midr_cp_reginfo); ++i) {
+                id_pre_v8_midr_cp_reginfo[i].access = PL1_RW;
             }
-            for (r = id_cp_reginfo; r->type != ARM_CP_SENTINEL; r++) {
-                r->access = PL1_RW;
+            for (i = 0; i < ARRAY_SIZE(id_cp_reginfo); ++i) {
+                id_cp_reginfo[i].access = PL1_RW;
             }
             id_mpuir_reginfo.access = PL1_RW;
             id_tlbtr_reginfo.access = PL1_RW;
@@ -8287,13 +8221,11 @@ void register_cp_regs_for_features(ARMCPU *cpu)
             { .name = "MPIDR_EL1", .state = ARM_CP_STATE_BOTH,
               .opc0 = 3, .crn = 0, .crm = 0, .opc1 = 0, .opc2 = 5,
               .access = PL1_R, .readfn = mpidr_read, .type = ARM_CP_NO_RAW },
-            REGINFO_SENTINEL
         };
 #ifdef CONFIG_USER_ONLY
         ARMCPRegUserSpaceInfo mpidr_user_cp_reginfo[] = {
             { .name = "MPIDR_EL1",
               .fixed_bits = 0x0000000080000000 },
-            REGUSERINFO_SENTINEL
         };
         modify_arm_cp_regs(mpidr_cp_reginfo, mpidr_user_cp_reginfo);
 #endif
@@ -8314,7 +8246,6 @@ void register_cp_regs_for_features(ARMCPU *cpu)
               .opc0 = 3, .opc1 = 6, .crn = 1, .crm = 0, .opc2 = 1,
               .access = PL3_RW, .type = ARM_CP_CONST,
               .resetvalue = 0 },
-            REGINFO_SENTINEL
         };
         define_arm_cp_regs(cpu, auxcr_reginfo);
         if (cpu_isar_feature(aa32_ac2, cpu)) {
@@ -8349,7 +8280,6 @@ void register_cp_regs_for_features(ARMCPU *cpu)
                   .type = ARM_CP_CONST,
                   .opc0 = 3, .opc1 = 1, .crn = 15, .crm = 3, .opc2 = 0,
                   .access = PL1_R, .resetvalue = cpu->reset_cbar },
-                REGINFO_SENTINEL
             };
             /* We don't implement a r/w 64 bit CBAR currently */
             assert(arm_feature(env, ARM_FEATURE_CBAR_RO));
@@ -8379,7 +8309,6 @@ void register_cp_regs_for_features(ARMCPU *cpu)
               .bank_fieldoffsets = { offsetof(CPUARMState, cp15.vbar_s),
                                      offsetof(CPUARMState, cp15.vbar_ns) },
               .resetvalue = 0 },
-            REGINFO_SENTINEL
         };
         define_arm_cp_regs(cpu, vbar_cp_reginfo);
     }
@@ -8833,8 +8762,7 @@ void define_one_arm_cp_reg_with_opaque(ARMCPU *cpu,
                    r->writefn);
         }
     }
-    /* Bad type field probably means missing sentinel at end of reg list */
-    assert(cptype_valid(r->type));
+
     for (crm = crmmin; crm <= crmmax; crm++) {
         for (opc1 = opc1min; opc1 <= opc1max; opc1++) {
             for (opc2 = opc2min; opc2 <= opc2max; opc2++) {
@@ -8880,13 +8808,13 @@ void define_one_arm_cp_reg_with_opaque(ARMCPU *cpu,
     }
 }
 
-void define_arm_cp_regs_with_opaque(ARMCPU *cpu,
-                                    const ARMCPRegInfo *regs, void *opaque)
+/* Define a whole list of registers */
+void define_arm_cp_regs_with_opaque_len(ARMCPU *cpu, const ARMCPRegInfo *regs,
+                                        void *opaque, size_t len)
 {
-    /* Define a whole list of registers */
-    const ARMCPRegInfo *r;
-    for (r = regs; r->type != ARM_CP_SENTINEL; r++) {
-        define_one_arm_cp_reg_with_opaque(cpu, r, opaque);
+    size_t i;
+    for (i = 0; i < len; ++i) {
+        define_one_arm_cp_reg_with_opaque(cpu, regs + i, opaque);
     }
 }
 
@@ -8898,17 +8826,20 @@ void define_arm_cp_regs_with_opaque(ARMCPU *cpu,
  * user-space cannot alter any values and dynamic values pertaining to
  * execution state are hidden from user space view anyway.
  */
-void modify_arm_cp_regs(ARMCPRegInfo *regs, const ARMCPRegUserSpaceInfo *mods)
+void modify_arm_cp_regs_with_len(ARMCPRegInfo *regs, size_t regs_len,
+                                 const ARMCPRegUserSpaceInfo *mods,
+                                 size_t mods_len)
 {
-    const ARMCPRegUserSpaceInfo *m;
-    ARMCPRegInfo *r;
-
-    for (m = mods; m->name; m++) {
+    for (size_t mi = 0; mi < mods_len; ++mi) {
+        const ARMCPRegUserSpaceInfo *m = mods + mi;
         GPatternSpec *pat = NULL;
+
         if (m->is_glob) {
             pat = g_pattern_spec_new(m->name);
         }
-        for (r = regs; r->type != ARM_CP_SENTINEL; r++) {
+        for (size_t ri = 0; ri < regs_len; ++ri) {
+            ARMCPRegInfo *r = regs + ri;
+
             if (pat && g_pattern_match_string(pat, r->name)) {
                 r->type = ARM_CP_CONST;
                 r->access = PL0U_R;

From 10b0220e45a93410206c270b0642870e20e69d09 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:49:46 -0700
Subject: [PATCH 019/935] target/arm: Make some more cpreg data static const
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

These particular data structures are not modified at runtime.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220501055028.646596-5-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index a68f14fe8e..ca6ba9bd82 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -7860,7 +7860,7 @@ void register_cp_regs_for_features(ARMCPU *cpu)
               .resetvalue = cpu->pmceid1 },
         };
 #ifdef CONFIG_USER_ONLY
-        ARMCPRegUserSpaceInfo v8_user_idregs[] = {
+        static const ARMCPRegUserSpaceInfo v8_user_idregs[] = {
             { .name = "ID_AA64PFR0_EL1",
               .exported_bits = 0x000f000f00ff0000,
               .fixed_bits    = 0x0000000000000011 },
@@ -8000,7 +8000,7 @@ void register_cp_regs_for_features(ARMCPU *cpu)
      */
     if (arm_feature(env, ARM_FEATURE_EL3)) {
         if (arm_feature(env, ARM_FEATURE_AARCH64)) {
-            ARMCPRegInfo nsacr = {
+            static const ARMCPRegInfo nsacr = {
                 .name = "NSACR", .type = ARM_CP_CONST,
                 .cp = 15, .opc1 = 0, .crn = 1, .crm = 1, .opc2 = 2,
                 .access = PL1_RW, .accessfn = nsacr_access,
@@ -8008,7 +8008,7 @@ void register_cp_regs_for_features(ARMCPU *cpu)
             };
             define_one_arm_cp_reg(cpu, &nsacr);
         } else {
-            ARMCPRegInfo nsacr = {
+            static const ARMCPRegInfo nsacr = {
                 .name = "NSACR",
                 .cp = 15, .opc1 = 0, .crn = 1, .crm = 1, .opc2 = 2,
                 .access = PL3_RW | PL1_R,
@@ -8019,7 +8019,7 @@ void register_cp_regs_for_features(ARMCPU *cpu)
         }
     } else {
         if (arm_feature(env, ARM_FEATURE_V8)) {
-            ARMCPRegInfo nsacr = {
+            static const ARMCPRegInfo nsacr = {
                 .name = "NSACR", .type = ARM_CP_CONST,
                 .cp = 15, .opc1 = 0, .crn = 1, .crm = 1, .opc2 = 2,
                 .access = PL1_R,
@@ -8172,13 +8172,13 @@ void register_cp_regs_for_features(ARMCPU *cpu)
               .access = PL1_R, .type = ARM_CP_CONST,
               .resetvalue = cpu->pmsav7_dregion << 8
         };
-        ARMCPRegInfo crn0_wi_reginfo = {
+        static const ARMCPRegInfo crn0_wi_reginfo = {
             .name = "CRN0_WI", .cp = 15, .crn = 0, .crm = CP_ANY,
             .opc1 = CP_ANY, .opc2 = CP_ANY, .access = PL1_W,
             .type = ARM_CP_NOP | ARM_CP_OVERRIDE
         };
 #ifdef CONFIG_USER_ONLY
-        ARMCPRegUserSpaceInfo id_v8_user_midr_cp_reginfo[] = {
+        static const ARMCPRegUserSpaceInfo id_v8_user_midr_cp_reginfo[] = {
             { .name = "MIDR_EL1",
               .exported_bits = 0x00000000ffffffff },
             { .name = "REVIDR_EL1"                },
@@ -8223,7 +8223,7 @@ void register_cp_regs_for_features(ARMCPU *cpu)
               .access = PL1_R, .readfn = mpidr_read, .type = ARM_CP_NO_RAW },
         };
 #ifdef CONFIG_USER_ONLY
-        ARMCPRegUserSpaceInfo mpidr_user_cp_reginfo[] = {
+        static const ARMCPRegUserSpaceInfo mpidr_user_cp_reginfo[] = {
             { .name = "MPIDR_EL1",
               .fixed_bits = 0x0000000080000000 },
         };
@@ -8302,7 +8302,7 @@ void register_cp_regs_for_features(ARMCPU *cpu)
     }
 
     if (arm_feature(env, ARM_FEATURE_VBAR)) {
-        ARMCPRegInfo vbar_cp_reginfo[] = {
+        static const ARMCPRegInfo vbar_cp_reginfo[] = {
             { .name = "VBAR", .state = ARM_CP_STATE_BOTH,
               .opc0 = 3, .crn = 12, .crm = 0, .opc1 = 0, .opc2 = 0,
               .access = PL1_RW, .writefn = vbar_write,

From 87c3f0f2f791094adce2b8643d9103dac1b59702 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:49:47 -0700
Subject: [PATCH 020/935] target/arm: Reorg ARMCPRegInfo type field bits

Instead of defining ARM_CP_FLAG_MASK to remove flags,
define ARM_CP_SPECIAL_MASK to isolate special cases.
Sort the specials to the low bits. Use an enum.

Split the large comment block so as to document each
value separately.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20220501055028.646596-6-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpregs.h        | 130 +++++++++++++++++++++++--------------
 target/arm/cpu.c           |   4 +-
 target/arm/helper.c        |   4 +-
 target/arm/translate-a64.c |   6 +-
 target/arm/translate.c     |   6 +-
 5 files changed, 92 insertions(+), 58 deletions(-)

diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h
index a5231504d5..ff3817decb 100644
--- a/target/arm/cpregs.h
+++ b/target/arm/cpregs.h
@@ -22,57 +22,87 @@
 #define TARGET_ARM_CPREGS_H
 
 /*
- * ARMCPRegInfo type field bits. If the SPECIAL bit is set this is a
- * special-behaviour cp reg and bits [11..8] indicate what behaviour
- * it has. Otherwise it is a simple cp reg, where CONST indicates that
- * TCG can assume the value to be constant (ie load at translate time)
- * and 64BIT indicates a 64 bit wide coprocessor register. SUPPRESS_TB_END
- * indicates that the TB should not be ended after a write to this register
- * (the default is that the TB ends after cp writes). OVERRIDE permits
- * a register definition to override a previous definition for the
- * same (cp, is64, crn, crm, opc1, opc2) tuple: either the new or the
- * old must have the OVERRIDE bit set.
- * ALIAS indicates that this register is an alias view of some underlying
- * state which is also visible via another register, and that the other
- * register is handling migration and reset; registers marked ALIAS will not be
- * migrated but may have their state set by syncing of register state from KVM.
- * NO_RAW indicates that this register has no underlying state and does not
- * support raw access for state saving/loading; it will not be used for either
- * migration or KVM state synchronization. (Typically this is for "registers"
- * which are actually used as instructions for cache maintenance and so on.)
- * IO indicates that this register does I/O and therefore its accesses
- * need to be marked with gen_io_start() and also end the TB. In particular,
- * registers which implement clocks or timers require this.
- * RAISES_EXC is for when the read or write hook might raise an exception;
- * the generated code will synchronize the CPU state before calling the hook
- * so that it is safe for the hook to call raise_exception().
- * NEWEL is for writes to registers that might change the exception
- * level - typically on older ARM chips. For those cases we need to
- * re-read the new el when recomputing the translation flags.
+ * ARMCPRegInfo type field bits:
  */
-#define ARM_CP_SPECIAL           0x0001
-#define ARM_CP_CONST             0x0002
-#define ARM_CP_64BIT             0x0004
-#define ARM_CP_SUPPRESS_TB_END   0x0008
-#define ARM_CP_OVERRIDE          0x0010
-#define ARM_CP_ALIAS             0x0020
-#define ARM_CP_IO                0x0040
-#define ARM_CP_NO_RAW            0x0080
-#define ARM_CP_NOP               (ARM_CP_SPECIAL | 0x0100)
-#define ARM_CP_WFI               (ARM_CP_SPECIAL | 0x0200)
-#define ARM_CP_NZCV              (ARM_CP_SPECIAL | 0x0300)
-#define ARM_CP_CURRENTEL         (ARM_CP_SPECIAL | 0x0400)
-#define ARM_CP_DC_ZVA            (ARM_CP_SPECIAL | 0x0500)
-#define ARM_CP_DC_GVA            (ARM_CP_SPECIAL | 0x0600)
-#define ARM_CP_DC_GZVA           (ARM_CP_SPECIAL | 0x0700)
-#define ARM_LAST_SPECIAL         ARM_CP_DC_GZVA
-#define ARM_CP_FPU               0x1000
-#define ARM_CP_SVE               0x2000
-#define ARM_CP_NO_GDB            0x4000
-#define ARM_CP_RAISES_EXC        0x8000
-#define ARM_CP_NEWEL             0x10000
-/* Mask of only the flag bits in a type field */
-#define ARM_CP_FLAG_MASK         0x1f0ff
+enum {
+    /*
+     * Register must be handled specially during translation.
+     * The method is one of the values below:
+     */
+    ARM_CP_SPECIAL_MASK          = 0x000f,
+    /* Special: no change to PE state: writes ignored, reads ignored. */
+    ARM_CP_NOP                   = 0x0001,
+    /* Special: sysreg is WFI, for v5 and v6. */
+    ARM_CP_WFI                   = 0x0002,
+    /* Special: sysreg is NZCV. */
+    ARM_CP_NZCV                  = 0x0003,
+    /* Special: sysreg is CURRENTEL. */
+    ARM_CP_CURRENTEL             = 0x0004,
+    /* Special: sysreg is DC ZVA or similar. */
+    ARM_CP_DC_ZVA                = 0x0005,
+    ARM_CP_DC_GVA                = 0x0006,
+    ARM_CP_DC_GZVA               = 0x0007,
+
+    /* Flag: reads produce resetvalue; writes ignored. */
+    ARM_CP_CONST                 = 1 << 4,
+    /* Flag: For ARM_CP_STATE_AA32, sysreg is 64-bit. */
+    ARM_CP_64BIT                 = 1 << 5,
+    /*
+     * Flag: TB should not be ended after a write to this register
+     * (the default is that the TB ends after cp writes).
+     */
+    ARM_CP_SUPPRESS_TB_END       = 1 << 6,
+    /*
+     * Flag: Permit a register definition to override a previous definition
+     * for the same (cp, is64, crn, crm, opc1, opc2) tuple: either the new
+     * or the old must have the ARM_CP_OVERRIDE bit set.
+     */
+    ARM_CP_OVERRIDE              = 1 << 7,
+    /*
+     * Flag: Register is an alias view of some underlying state which is also
+     * visible via another register, and that the other register is handling
+     * migration and reset; registers marked ARM_CP_ALIAS will not be migrated
+     * but may have their state set by syncing of register state from KVM.
+     */
+    ARM_CP_ALIAS                 = 1 << 8,
+    /*
+     * Flag: Register does I/O and therefore its accesses need to be marked
+     * with gen_io_start() and also end the TB. In particular, registers which
+     * implement clocks or timers require this.
+     */
+    ARM_CP_IO                    = 1 << 9,
+    /*
+     * Flag: Register has no underlying state and does not support raw access
+     * for state saving/loading; it will not be used for either migration or
+     * KVM state synchronization. Typically this is for "registers" which are
+     * actually used as instructions for cache maintenance and so on.
+     */
+    ARM_CP_NO_RAW                = 1 << 10,
+    /*
+     * Flag: The read or write hook might raise an exception; the generated
+     * code will synchronize the CPU state before calling the hook so that it
+     * is safe for the hook to call raise_exception().
+     */
+    ARM_CP_RAISES_EXC            = 1 << 11,
+    /*
+     * Flag: Writes to the sysreg might change the exception level - typically
+     * on older ARM chips. For those cases we need to re-read the new el when
+     * recomputing the translation flags.
+     */
+    ARM_CP_NEWEL                 = 1 << 12,
+    /*
+     * Flag: Access check for this sysreg is identical to accessing FPU state
+     * from an instruction: use translation fp_access_check().
+     */
+    ARM_CP_FPU                   = 1 << 13,
+    /*
+     * Flag: Access check for this sysreg is identical to accessing SVE state
+     * from an instruction: use translation sve_access_check().
+     */
+    ARM_CP_SVE                   = 1 << 14,
+    /* Flag: Do not expose in gdb sysreg xml. */
+    ARM_CP_NO_GDB                = 1 << 15,
+};
 
 /*
  * Valid values for ARMCPRegInfo state field, indicating which of
diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index 18212eb6ee..a7cd692010 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -117,7 +117,7 @@ static void cp_reg_reset(gpointer key, gpointer value, gpointer opaque)
     ARMCPRegInfo *ri = value;
     ARMCPU *cpu = opaque;
 
-    if (ri->type & (ARM_CP_SPECIAL | ARM_CP_ALIAS)) {
+    if (ri->type & (ARM_CP_SPECIAL_MASK | ARM_CP_ALIAS)) {
         return;
     }
 
@@ -153,7 +153,7 @@ static void cp_reg_check_reset(gpointer key, gpointer value,  gpointer opaque)
     ARMCPU *cpu = opaque;
     uint64_t oldvalue, newvalue;
 
-    if (ri->type & (ARM_CP_SPECIAL | ARM_CP_ALIAS | ARM_CP_NO_RAW)) {
+    if (ri->type & (ARM_CP_SPECIAL_MASK | ARM_CP_ALIAS | ARM_CP_NO_RAW)) {
         return;
     }
 
diff --git a/target/arm/helper.c b/target/arm/helper.c
index ca6ba9bd82..f84377babe 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -8600,7 +8600,7 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
      * multiple times. Special registers (ie NOP/WFI) are
      * never migratable and not even raw-accessible.
      */
-    if ((r->type & ARM_CP_SPECIAL)) {
+    if (r->type & ARM_CP_SPECIAL_MASK) {
         r2->type |= ARM_CP_NO_RAW;
     }
     if (((r->crm == CP_ANY) && crm != 0) ||
@@ -8750,7 +8750,7 @@ void define_one_arm_cp_reg_with_opaque(ARMCPU *cpu,
     /* Check that the register definition has enough info to handle
      * reads and writes if they are permitted.
      */
-    if (!(r->type & (ARM_CP_SPECIAL|ARM_CP_CONST))) {
+    if (!(r->type & (ARM_CP_SPECIAL_MASK | ARM_CP_CONST))) {
         if (r->access & PL3_R) {
             assert((r->fieldoffset ||
                    (r->bank_fieldoffsets[0] && r->bank_fieldoffsets[1])) ||
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 348a638c5c..a82f5d5984 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -1833,7 +1833,9 @@ static void handle_sys(DisasContext *s, uint32_t insn, bool isread,
     }
 
     /* Handle special cases first */
-    switch (ri->type & ~(ARM_CP_FLAG_MASK & ~ARM_CP_SPECIAL)) {
+    switch (ri->type & ARM_CP_SPECIAL_MASK) {
+    case 0:
+        break;
     case ARM_CP_NOP:
         return;
     case ARM_CP_NZCV:
@@ -1908,7 +1910,7 @@ static void handle_sys(DisasContext *s, uint32_t insn, bool isread,
         }
         return;
     default:
-        break;
+        g_assert_not_reached();
     }
     if ((ri->type & ARM_CP_FPU) && !fp_access_check(s)) {
         return;
diff --git a/target/arm/translate.c b/target/arm/translate.c
index fc7917cdf4..050c237b07 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -4744,7 +4744,9 @@ static void do_coproc_insn(DisasContext *s, int cpnum, int is64,
         }
 
         /* Handle special cases first */
-        switch (ri->type & ~(ARM_CP_FLAG_MASK & ~ARM_CP_SPECIAL)) {
+        switch (ri->type & ARM_CP_SPECIAL_MASK) {
+        case 0:
+            break;
         case ARM_CP_NOP:
             return;
         case ARM_CP_WFI:
@@ -4756,7 +4758,7 @@ static void do_coproc_insn(DisasContext *s, int cpnum, int is64,
             s->base.is_jmp = DISAS_WFI;
             return;
         default:
-            break;
+            g_assert_not_reached();
         }
 
         if ((tb_cflags(s->base.tb) & CF_USE_ICOUNT) && (ri->type & ARM_CP_IO)) {

From d385a60571487d9409f247d07aac94d1ee40f9fe Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:49:48 -0700
Subject: [PATCH 021/935] target/arm: Avoid bare abort() or assert(0)

Standardize on g_assert_not_reached() for "should not happen".
Retain abort() when preceeded by fprintf or error_report.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20220501055028.646596-7-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c         | 7 +++----
 target/arm/hvf/hvf.c        | 2 +-
 target/arm/kvm-stub.c       | 4 ++--
 target/arm/kvm.c            | 4 ++--
 target/arm/machine.c        | 4 ++--
 target/arm/translate-a64.c  | 4 ++--
 target/arm/translate-neon.c | 2 +-
 target/arm/translate.c      | 4 ++--
 8 files changed, 15 insertions(+), 16 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index f84377babe..06f8864c77 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -8740,8 +8740,7 @@ void define_one_arm_cp_reg_with_opaque(ARMCPU *cpu,
             break;
         default:
             /* broken reginfo with out-of-range opc1 */
-            assert(false);
-            break;
+            g_assert_not_reached();
         }
         /* assert our permissions are not too lax (stricter is fine) */
         assert((r->access & ~mask) == 0);
@@ -10823,7 +10822,7 @@ static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
             break;
         default:
             /* Never happens, but compiler isn't smart enough to tell.  */
-            abort();
+            g_assert_not_reached();
         }
     }
     *prot = ap_to_rw_prot(env, mmu_idx, ap, domain_prot);
@@ -10944,7 +10943,7 @@ static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
             break;
         default:
             /* Never happens, but compiler isn't smart enough to tell.  */
-            abort();
+            g_assert_not_reached();
         }
     }
     if (domain_prot == 3) {
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index b11a8b9a18..86710509d2 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -1200,7 +1200,7 @@ int hvf_vcpu_exec(CPUState *cpu)
         /* we got kicked, no exit to process */
         return 0;
     default:
-        assert(0);
+        g_assert_not_reached();
     }
 
     hvf_sync_vtimer(cpu);
diff --git a/target/arm/kvm-stub.c b/target/arm/kvm-stub.c
index 56a7099e6b..965a486b32 100644
--- a/target/arm/kvm-stub.c
+++ b/target/arm/kvm-stub.c
@@ -15,10 +15,10 @@
 
 bool write_kvmstate_to_list(ARMCPU *cpu)
 {
-    abort();
+    g_assert_not_reached();
 }
 
 bool write_list_to_kvmstate(ARMCPU *cpu, int level)
 {
-    abort();
+    g_assert_not_reached();
 }
diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index 5fc37ac10a..4339e1cd6e 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -540,7 +540,7 @@ bool write_kvmstate_to_list(ARMCPU *cpu)
             ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, &r);
             break;
         default:
-            abort();
+            g_assert_not_reached();
         }
         if (ret) {
             ok = false;
@@ -575,7 +575,7 @@ bool write_list_to_kvmstate(ARMCPU *cpu, int level)
             r.addr = (uintptr_t)(cpu->cpreg_values + i);
             break;
         default:
-            abort();
+            g_assert_not_reached();
         }
         ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &r);
         if (ret) {
diff --git a/target/arm/machine.c b/target/arm/machine.c
index 135d2420b5..285e387d2c 100644
--- a/target/arm/machine.c
+++ b/target/arm/machine.c
@@ -661,7 +661,7 @@ static int cpu_pre_save(void *opaque)
     if (kvm_enabled()) {
         if (!write_kvmstate_to_list(cpu)) {
             /* This should never fail */
-            abort();
+            g_assert_not_reached();
         }
 
         /*
@@ -672,7 +672,7 @@ static int cpu_pre_save(void *opaque)
     } else {
         if (!write_cpustate_to_list(cpu, false)) {
             /* This should never fail. */
-            abort();
+            g_assert_not_reached();
         }
     }
 
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index a82f5d5984..b80313670f 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -6151,7 +6151,7 @@ static void handle_fp_1src_half(DisasContext *s, int opcode, int rd, int rn)
         gen_helper_advsimd_rinth(tcg_res, tcg_op, fpst);
         break;
     default:
-        abort();
+        g_assert_not_reached();
     }
 
     write_fp_sreg(s, rd, tcg_res);
@@ -6392,7 +6392,7 @@ static void handle_fp_fcvt(DisasContext *s, int opcode,
         break;
     }
     default:
-        abort();
+        g_assert_not_reached();
     }
 }
 
diff --git a/target/arm/translate-neon.c b/target/arm/translate-neon.c
index 2e4d1ec87d..321c17e2c7 100644
--- a/target/arm/translate-neon.c
+++ b/target/arm/translate-neon.c
@@ -679,7 +679,7 @@ static bool trans_VLDST_single(DisasContext *s, arg_VLDST_single *a)
         }
         break;
     default:
-        abort();
+        g_assert_not_reached();
     }
     if ((vd + a->stride * (nregs - 1)) > 31) {
         /*
diff --git a/target/arm/translate.c b/target/arm/translate.c
index 050c237b07..4e19191ed5 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -5156,7 +5156,7 @@ static void gen_srs(DisasContext *s,
         offset = 4;
         break;
     default:
-        abort();
+        g_assert_not_reached();
     }
     tcg_gen_addi_i32(addr, addr, offset);
     tmp = load_reg(s, 14);
@@ -5181,7 +5181,7 @@ static void gen_srs(DisasContext *s,
             offset = 0;
             break;
         default:
-            abort();
+            g_assert_not_reached();
         }
         tcg_gen_addi_i32(addr, addr, offset);
         gen_helper_set_r13_banked(cpu_env, tcg_constant_i32(mode), addr);

From 39107337181e0cfcbd92e67d10dce3acd5a13f0a Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:49:49 -0700
Subject: [PATCH 022/935] target/arm: Change cpreg access permissions to enum

Create a typedef as well, and use it in ARMCPRegInfo.
This won't be perfect for debugging, but it'll nicely
display the most common cases.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220501055028.646596-8-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpregs.h | 44 +++++++++++++++++++++++---------------------
 target/arm/helper.c |  2 +-
 2 files changed, 24 insertions(+), 22 deletions(-)

diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h
index ff3817decb..858c5da57d 100644
--- a/target/arm/cpregs.h
+++ b/target/arm/cpregs.h
@@ -154,31 +154,33 @@ enum {
  * described with these bits, then use a laxer set of restrictions, and
  * do the more restrictive/complex check inside a helper function.
  */
-#define PL3_R 0x80
-#define PL3_W 0x40
-#define PL2_R (0x20 | PL3_R)
-#define PL2_W (0x10 | PL3_W)
-#define PL1_R (0x08 | PL2_R)
-#define PL1_W (0x04 | PL2_W)
-#define PL0_R (0x02 | PL1_R)
-#define PL0_W (0x01 | PL1_W)
+typedef enum {
+    PL3_R = 0x80,
+    PL3_W = 0x40,
+    PL2_R = 0x20 | PL3_R,
+    PL2_W = 0x10 | PL3_W,
+    PL1_R = 0x08 | PL2_R,
+    PL1_W = 0x04 | PL2_W,
+    PL0_R = 0x02 | PL1_R,
+    PL0_W = 0x01 | PL1_W,
 
-/*
- * For user-mode some registers are accessible to EL0 via a kernel
- * trap-and-emulate ABI. In this case we define the read permissions
- * as actually being PL0_R. However some bits of any given register
- * may still be masked.
- */
+    /*
+     * For user-mode some registers are accessible to EL0 via a kernel
+     * trap-and-emulate ABI. In this case we define the read permissions
+     * as actually being PL0_R. However some bits of any given register
+     * may still be masked.
+     */
 #ifdef CONFIG_USER_ONLY
-#define PL0U_R PL0_R
+    PL0U_R = PL0_R,
 #else
-#define PL0U_R PL1_R
+    PL0U_R = PL1_R,
 #endif
 
-#define PL3_RW (PL3_R | PL3_W)
-#define PL2_RW (PL2_R | PL2_W)
-#define PL1_RW (PL1_R | PL1_W)
-#define PL0_RW (PL0_R | PL0_W)
+    PL3_RW = PL3_R | PL3_W,
+    PL2_RW = PL2_R | PL2_W,
+    PL1_RW = PL1_R | PL1_W,
+    PL0_RW = PL0_R | PL0_W,
+} CPAccessRights;
 
 typedef enum CPAccessResult {
     /* Access is permitted */
@@ -262,7 +264,7 @@ struct ARMCPRegInfo {
     /* Register type: ARM_CP_* bits/values */
     int type;
     /* Access rights: PL*_[RW] */
-    int access;
+    CPAccessRights access;
     /* Security state: ARM_CP_SECSTATE_* bits/values */
     int secure;
     /*
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 06f8864c77..a19e04bb0b 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -8711,7 +8711,7 @@ void define_one_arm_cp_reg_with_opaque(ARMCPU *cpu,
      * to encompass the generic architectural permission check.
      */
     if (r->state != ARM_CP_STATE_AA32) {
-        int mask = 0;
+        CPAccessRights mask;
         switch (r->opc1) {
         case 0:
             /* min_EL EL1, but some accessible to EL0 via kernel ABI */

From d95101d6026641e9116225b5a22bbe6c4621828d Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:49:50 -0700
Subject: [PATCH 023/935] target/arm: Name CPState type
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Give this enum a name and use in ARMCPRegInfo,
add_cpreg_to_hashtable and define_one_arm_cp_reg_with_opaque.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220501055028.646596-9-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpregs.h | 6 +++---
 target/arm/helper.c | 6 ++++--
 2 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h
index 858c5da57d..4179a8cdd5 100644
--- a/target/arm/cpregs.h
+++ b/target/arm/cpregs.h
@@ -114,11 +114,11 @@ enum {
  * Note that we rely on the values of these enums as we iterate through
  * the various states in some places.
  */
-enum {
+typedef enum {
     ARM_CP_STATE_AA32 = 0,
     ARM_CP_STATE_AA64 = 1,
     ARM_CP_STATE_BOTH = 2,
-};
+} CPState;
 
 /*
  * ARM CP register secure state flags.  These flags identify security state
@@ -260,7 +260,7 @@ struct ARMCPRegInfo {
     uint8_t opc1;
     uint8_t opc2;
     /* Execution state in which this register is visible: ARM_CP_STATE_* */
-    int state;
+    CPState state;
     /* Register type: ARM_CP_* bits/values */
     int type;
     /* Access rights: PL*_[RW] */
diff --git a/target/arm/helper.c b/target/arm/helper.c
index a19e04bb0b..d560a6a6a9 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -8502,7 +8502,7 @@ CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
 }
 
 static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
-                                   void *opaque, int state, int secstate,
+                                   void *opaque, CPState state, int secstate,
                                    int crm, int opc1, int opc2,
                                    const char *name)
 {
@@ -8662,13 +8662,15 @@ void define_one_arm_cp_reg_with_opaque(ARMCPU *cpu,
      * bits; the ARM_CP_64BIT* flag applies only to the AArch32 view of
      * the register, if any.
      */
-    int crm, opc1, opc2, state;
+    int crm, opc1, opc2;
     int crmmin = (r->crm == CP_ANY) ? 0 : r->crm;
     int crmmax = (r->crm == CP_ANY) ? 15 : r->crm;
     int opc1min = (r->opc1 == CP_ANY) ? 0 : r->opc1;
     int opc1max = (r->opc1 == CP_ANY) ? 7 : r->opc1;
     int opc2min = (r->opc2 == CP_ANY) ? 0 : r->opc2;
     int opc2max = (r->opc2 == CP_ANY) ? 7 : r->opc2;
+    CPState state;
+
     /* 64 bit registers have only CRm and Opc1 fields */
     assert(!((r->type & ARM_CP_64BIT) && (r->opc2 || r->crn)));
     /* op0 only exists in the AArch64 encodings */

From cbe645856fb17ca0f2395fa4a75d80bdd0421614 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:49:51 -0700
Subject: [PATCH 024/935] target/arm: Name CPSecureState type

Give this enum a name and use in ARMCPRegInfo and add_cpreg_to_hashtable.
Add the enumerator ARM_CP_SECSTATE_BOTH to clarify how 0
is handled in define_one_arm_cp_reg_with_opaque.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220501055028.646596-10-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpregs.h | 7 ++++---
 target/arm/helper.c | 7 +++++--
 2 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h
index 4179a8cdd5..73984549d2 100644
--- a/target/arm/cpregs.h
+++ b/target/arm/cpregs.h
@@ -131,10 +131,11 @@ typedef enum {
  * registered entry will only have one to identify whether the entry is secure
  * or non-secure.
  */
-enum {
+typedef enum {
+    ARM_CP_SECSTATE_BOTH = 0,       /* define one cpreg for each secstate */
     ARM_CP_SECSTATE_S =   (1 << 0), /* bit[0]: Secure state register */
     ARM_CP_SECSTATE_NS =  (1 << 1), /* bit[1]: Non-secure state register */
-};
+} CPSecureState;
 
 /*
  * Access rights:
@@ -266,7 +267,7 @@ struct ARMCPRegInfo {
     /* Access rights: PL*_[RW] */
     CPAccessRights access;
     /* Security state: ARM_CP_SECSTATE_* bits/values */
-    int secure;
+    CPSecureState secure;
     /*
      * The opaque pointer passed to define_arm_cp_regs_with_opaque() when
      * this register was defined: can be used to hand data through to the
diff --git a/target/arm/helper.c b/target/arm/helper.c
index d560a6a6a9..50ad2e3e37 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -8502,7 +8502,8 @@ CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
 }
 
 static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
-                                   void *opaque, CPState state, int secstate,
+                                   void *opaque, CPState state,
+                                   CPSecureState secstate,
                                    int crm, int opc1, int opc2,
                                    const char *name)
 {
@@ -8785,7 +8786,7 @@ void define_one_arm_cp_reg_with_opaque(ARMCPU *cpu,
                                                    r->secure, crm, opc1, opc2,
                                                    r->name);
                             break;
-                        default:
+                        case ARM_CP_SECSTATE_BOTH:
                             name = g_strdup_printf("%s_S", r->name);
                             add_cpreg_to_hashtable(cpu, r, opaque, state,
                                                    ARM_CP_SECSTATE_S,
@@ -8795,6 +8796,8 @@ void define_one_arm_cp_reg_with_opaque(ARMCPU *cpu,
                                                    ARM_CP_SECSTATE_NS,
                                                    crm, opc1, opc2, r->name);
                             break;
+                        default:
+                            g_assert_not_reached();
                         }
                     } else {
                         /* AArch64 registers get mapped to non-secure instance

From 9da35a40fda3b577e7251d39f1b4aad42348cc08 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:49:52 -0700
Subject: [PATCH 025/935] target/arm: Drop always-true test in
 define_arm_vh_e2h_redirects_aliases

The new_key field is always non-zero -- drop the if.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20220501055028.646596-11-richard.henderson@linaro.org
[PMM: reinstated dropped PL3_RW mask]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 23 +++++++++++------------
 1 file changed, 11 insertions(+), 12 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 50ad2e3e37..70dc1482dd 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -5914,7 +5914,9 @@ static void define_arm_vh_e2h_redirects_aliases(ARMCPU *cpu)
 
     for (i = 0; i < ARRAY_SIZE(aliases); i++) {
         const struct E2HAlias *a = &aliases[i];
-        ARMCPRegInfo *src_reg, *dst_reg;
+        ARMCPRegInfo *src_reg, *dst_reg, *new_reg;
+        uint32_t *new_key;
+        bool ok;
 
         if (a->feature && !a->feature(&cpu->isar)) {
             continue;
@@ -5933,19 +5935,16 @@ static void define_arm_vh_e2h_redirects_aliases(ARMCPU *cpu)
         g_assert(src_reg->opaque == NULL);
 
         /* Create alias before redirection so we dup the right data. */
-        if (a->new_key) {
-            ARMCPRegInfo *new_reg = g_memdup(src_reg, sizeof(ARMCPRegInfo));
-            uint32_t *new_key = g_memdup(&a->new_key, sizeof(uint32_t));
-            bool ok;
+        new_reg = g_memdup(src_reg, sizeof(ARMCPRegInfo));
+        new_key = g_memdup(&a->new_key, sizeof(uint32_t));
 
-            new_reg->name = a->new_name;
-            new_reg->type |= ARM_CP_ALIAS;
-            /* Remove PL1/PL0 access, leaving PL2/PL3 R/W in place.  */
-            new_reg->access &= PL2_RW | PL3_RW;
+        new_reg->name = a->new_name;
+        new_reg->type |= ARM_CP_ALIAS;
+        /* Remove PL1/PL0 access, leaving PL2/PL3 R/W in place.  */
+        new_reg->access &= PL2_RW | PL3_RW;
 
-            ok = g_hash_table_insert(cpu->cp_regs, new_key, new_reg);
-            g_assert(ok);
-        }
+        ok = g_hash_table_insert(cpu->cp_regs, new_key, new_reg);
+        g_assert(ok);
 
         src_reg->opaque = dst_reg;
         src_reg->orig_readfn = src_reg->readfn ?: raw_read;

From 5860362d25f3afa99364d60295c09147a07055f1 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:49:53 -0700
Subject: [PATCH 026/935] target/arm: Store cpregs key in the hash table
 directly

Cast the uint32_t key into a gpointer directly, which
allows us to avoid allocating storage for each key.

Use g_hash_table_lookup when we already have a gpointer
(e.g. for callbacks like count_cpreg), or when using
get_arm_cp_reginfo would require casting away const.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20220501055028.646596-12-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu.c     |  4 ++--
 target/arm/gdbstub.c |  2 +-
 target/arm/helper.c  | 41 ++++++++++++++++++-----------------------
 3 files changed, 21 insertions(+), 26 deletions(-)

diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index a7cd692010..602c060fff 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -1090,8 +1090,8 @@ static void arm_cpu_initfn(Object *obj)
     ARMCPU *cpu = ARM_CPU(obj);
 
     cpu_set_cpustate_pointers(cpu);
-    cpu->cp_regs = g_hash_table_new_full(g_int_hash, g_int_equal,
-                                         g_free, cpreg_hashtable_data_destroy);
+    cpu->cp_regs = g_hash_table_new_full(g_direct_hash, g_direct_equal,
+                                         NULL, cpreg_hashtable_data_destroy);
 
     QLIST_INIT(&cpu->pre_el_change_hooks);
     QLIST_INIT(&cpu->el_change_hooks);
diff --git a/target/arm/gdbstub.c b/target/arm/gdbstub.c
index f01a126108..f5b35cd55f 100644
--- a/target/arm/gdbstub.c
+++ b/target/arm/gdbstub.c
@@ -273,7 +273,7 @@ static void arm_gen_one_xml_sysreg_tag(GString *s, DynamicGDBXMLInfo *dyn_xml,
 static void arm_register_sysreg_for_xml(gpointer key, gpointer value,
                                         gpointer p)
 {
-    uint32_t ri_key = *(uint32_t *)key;
+    uint32_t ri_key = (uintptr_t)key;
     ARMCPRegInfo *ri = value;
     RegisterSysregXmlParam *param = (RegisterSysregXmlParam *)p;
     GString *s = param->s;
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 70dc1482dd..2bc81dbc5e 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -214,11 +214,8 @@ bool write_list_to_cpustate(ARMCPU *cpu)
 static void add_cpreg_to_list(gpointer key, gpointer opaque)
 {
     ARMCPU *cpu = opaque;
-    uint64_t regidx;
-    const ARMCPRegInfo *ri;
-
-    regidx = *(uint32_t *)key;
-    ri = get_arm_cp_reginfo(cpu->cp_regs, regidx);
+    uint32_t regidx = (uintptr_t)key;
+    const ARMCPRegInfo *ri = get_arm_cp_reginfo(cpu->cp_regs, regidx);
 
     if (!(ri->type & (ARM_CP_NO_RAW|ARM_CP_ALIAS))) {
         cpu->cpreg_indexes[cpu->cpreg_array_len] = cpreg_to_kvm_id(regidx);
@@ -230,11 +227,9 @@ static void add_cpreg_to_list(gpointer key, gpointer opaque)
 static void count_cpreg(gpointer key, gpointer opaque)
 {
     ARMCPU *cpu = opaque;
-    uint64_t regidx;
     const ARMCPRegInfo *ri;
 
-    regidx = *(uint32_t *)key;
-    ri = get_arm_cp_reginfo(cpu->cp_regs, regidx);
+    ri = g_hash_table_lookup(cpu->cp_regs, key);
 
     if (!(ri->type & (ARM_CP_NO_RAW|ARM_CP_ALIAS))) {
         cpu->cpreg_array_len++;
@@ -243,8 +238,8 @@ static void count_cpreg(gpointer key, gpointer opaque)
 
 static gint cpreg_key_compare(gconstpointer a, gconstpointer b)
 {
-    uint64_t aidx = cpreg_to_kvm_id(*(uint32_t *)a);
-    uint64_t bidx = cpreg_to_kvm_id(*(uint32_t *)b);
+    uint64_t aidx = cpreg_to_kvm_id((uintptr_t)a);
+    uint64_t bidx = cpreg_to_kvm_id((uintptr_t)b);
 
     if (aidx > bidx) {
         return 1;
@@ -5915,15 +5910,16 @@ static void define_arm_vh_e2h_redirects_aliases(ARMCPU *cpu)
     for (i = 0; i < ARRAY_SIZE(aliases); i++) {
         const struct E2HAlias *a = &aliases[i];
         ARMCPRegInfo *src_reg, *dst_reg, *new_reg;
-        uint32_t *new_key;
         bool ok;
 
         if (a->feature && !a->feature(&cpu->isar)) {
             continue;
         }
 
-        src_reg = g_hash_table_lookup(cpu->cp_regs, &a->src_key);
-        dst_reg = g_hash_table_lookup(cpu->cp_regs, &a->dst_key);
+        src_reg = g_hash_table_lookup(cpu->cp_regs,
+                                      (gpointer)(uintptr_t)a->src_key);
+        dst_reg = g_hash_table_lookup(cpu->cp_regs,
+                                      (gpointer)(uintptr_t)a->dst_key);
         g_assert(src_reg != NULL);
         g_assert(dst_reg != NULL);
 
@@ -5936,14 +5932,14 @@ static void define_arm_vh_e2h_redirects_aliases(ARMCPU *cpu)
 
         /* Create alias before redirection so we dup the right data. */
         new_reg = g_memdup(src_reg, sizeof(ARMCPRegInfo));
-        new_key = g_memdup(&a->new_key, sizeof(uint32_t));
 
         new_reg->name = a->new_name;
         new_reg->type |= ARM_CP_ALIAS;
         /* Remove PL1/PL0 access, leaving PL2/PL3 R/W in place.  */
         new_reg->access &= PL2_RW | PL3_RW;
 
-        ok = g_hash_table_insert(cpu->cp_regs, new_key, new_reg);
+        ok = g_hash_table_insert(cpu->cp_regs,
+                                 (gpointer)(uintptr_t)a->new_key, new_reg);
         g_assert(ok);
 
         src_reg->opaque = dst_reg;
@@ -8509,7 +8505,7 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
     /* Private utility function for define_one_arm_cp_reg_with_opaque():
      * add a single reginfo struct to the hash table.
      */
-    uint32_t *key = g_new(uint32_t, 1);
+    uint32_t key;
     ARMCPRegInfo *r2 = g_memdup(r, sizeof(ARMCPRegInfo));
     int is64 = (r->type & ARM_CP_64BIT) ? 1 : 0;
     int ns = (secstate & ARM_CP_SECSTATE_NS) ? 1 : 0;
@@ -8576,10 +8572,10 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
         if (r->cp == 0 || r->state == ARM_CP_STATE_BOTH) {
             r2->cp = CP_REG_ARM64_SYSREG_CP;
         }
-        *key = ENCODE_AA64_CP_REG(r2->cp, r2->crn, crm,
-                                  r2->opc0, opc1, opc2);
+        key = ENCODE_AA64_CP_REG(r2->cp, r2->crn, crm,
+                                 r2->opc0, opc1, opc2);
     } else {
-        *key = ENCODE_CP_REG(r2->cp, is64, ns, r2->crn, crm, opc1, opc2);
+        key = ENCODE_CP_REG(r2->cp, is64, ns, r2->crn, crm, opc1, opc2);
     }
     if (opaque) {
         r2->opaque = opaque;
@@ -8621,8 +8617,7 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
      * requested.
      */
     if (!(r->type & ARM_CP_OVERRIDE)) {
-        ARMCPRegInfo *oldreg;
-        oldreg = g_hash_table_lookup(cpu->cp_regs, key);
+        const ARMCPRegInfo *oldreg = get_arm_cp_reginfo(cpu->cp_regs, key);
         if (oldreg && !(oldreg->type & ARM_CP_OVERRIDE)) {
             fprintf(stderr, "Register redefined: cp=%d %d bit "
                     "crn=%d crm=%d opc1=%d opc2=%d, "
@@ -8632,7 +8627,7 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
             g_assert_not_reached();
         }
     }
-    g_hash_table_insert(cpu->cp_regs, key, r2);
+    g_hash_table_insert(cpu->cp_regs, (gpointer)(uintptr_t)key, r2);
 }
 
 
@@ -8864,7 +8859,7 @@ void modify_arm_cp_regs_with_len(ARMCPRegInfo *regs, size_t regs_len,
 
 const ARMCPRegInfo *get_arm_cp_reginfo(GHashTable *cpregs, uint32_t encoded_cp)
 {
-    return g_hash_table_lookup(cpregs, &encoded_cp);
+    return g_hash_table_lookup(cpregs, (gpointer)(uintptr_t)encoded_cp);
 }
 
 void arm_cp_write_ignore(CPUARMState *env, const ARMCPRegInfo *ri,

From c27f5d3a83ed2959a6a1947708b588a6774a2aec Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:49:54 -0700
Subject: [PATCH 027/935] target/arm: Merge allocation of the cpreg and its
 name

Simplify freeing cp_regs hash table entries by using a single
allocation for the entire value.

This fixes a theoretical bug if we were to ever free the entire
hash table, because we've been installing string literal constants
into the cpreg structure in define_arm_vh_e2h_redirects_aliases.
However, at present we only free entries created for AArch32
wildcard cpregs which get overwritten by more specific cpregs,
so this bug is never exposed.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20220501055028.646596-13-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu.c    | 16 +---------------
 target/arm/helper.c | 10 ++++++++--
 2 files changed, 9 insertions(+), 17 deletions(-)

diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index 602c060fff..01176b2569 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -1071,27 +1071,13 @@ uint64_t arm_cpu_mp_affinity(int idx, uint8_t clustersz)
     return (Aff1 << ARM_AFF1_SHIFT) | Aff0;
 }
 
-static void cpreg_hashtable_data_destroy(gpointer data)
-{
-    /*
-     * Destroy function for cpu->cp_regs hashtable data entries.
-     * We must free the name string because it was g_strdup()ed in
-     * add_cpreg_to_hashtable(). It's OK to cast away the 'const'
-     * from r->name because we know we definitely allocated it.
-     */
-    ARMCPRegInfo *r = data;
-
-    g_free((void *)r->name);
-    g_free(r);
-}
-
 static void arm_cpu_initfn(Object *obj)
 {
     ARMCPU *cpu = ARM_CPU(obj);
 
     cpu_set_cpustate_pointers(cpu);
     cpu->cp_regs = g_hash_table_new_full(g_direct_hash, g_direct_equal,
-                                         NULL, cpreg_hashtable_data_destroy);
+                                         NULL, g_free);
 
     QLIST_INIT(&cpu->pre_el_change_hooks);
     QLIST_INIT(&cpu->el_change_hooks);
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 2bc81dbc5e..d92fd23445 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -8506,11 +8506,17 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
      * add a single reginfo struct to the hash table.
      */
     uint32_t key;
-    ARMCPRegInfo *r2 = g_memdup(r, sizeof(ARMCPRegInfo));
+    ARMCPRegInfo *r2;
     int is64 = (r->type & ARM_CP_64BIT) ? 1 : 0;
     int ns = (secstate & ARM_CP_SECSTATE_NS) ? 1 : 0;
+    size_t name_len;
+
+    /* Combine cpreg and name into one allocation. */
+    name_len = strlen(name) + 1;
+    r2 = g_malloc(sizeof(*r2) + name_len);
+    *r2 = *r;
+    r2->name = memcpy(r2 + 1, name, name_len);
 
-    r2->name = g_strdup(name);
     /* Reset the secure state to the specific incoming state.  This is
      * necessary as the register may have been defined with both states.
      */

From cac65299a403b34e57a2230c2b695bdb6d84f396 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:49:55 -0700
Subject: [PATCH 028/935] target/arm: Hoist computation of key in
 add_cpreg_to_hashtable

Move the computation of key to the top of the function.
Hoist the resolution of cp as well, as an input to the
computation of key.

This will be required by a subsequent patch.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20220501055028.646596-14-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 49 +++++++++++++++++++++++++--------------------
 1 file changed, 27 insertions(+), 22 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index d92fd23445..cbc873e3e6 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -8509,8 +8509,34 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
     ARMCPRegInfo *r2;
     int is64 = (r->type & ARM_CP_64BIT) ? 1 : 0;
     int ns = (secstate & ARM_CP_SECSTATE_NS) ? 1 : 0;
+    int cp = r->cp;
     size_t name_len;
 
+    switch (state) {
+    case ARM_CP_STATE_AA32:
+        /* We assume it is a cp15 register if the .cp field is left unset. */
+        if (cp == 0 && r->state == ARM_CP_STATE_BOTH) {
+            cp = 15;
+        }
+        key = ENCODE_CP_REG(cp, is64, ns, r->crn, crm, opc1, opc2);
+        break;
+    case ARM_CP_STATE_AA64:
+        /*
+         * To allow abbreviation of ARMCPRegInfo definitions, we treat
+         * cp == 0 as equivalent to the value for "standard guest-visible
+         * sysreg".  STATE_BOTH definitions are also always "standard sysreg"
+         * in their AArch64 view (the .cp value may be non-zero for the
+         * benefit of the AArch32 view).
+         */
+        if (cp == 0 || r->state == ARM_CP_STATE_BOTH) {
+            cp = CP_REG_ARM64_SYSREG_CP;
+        }
+        key = ENCODE_AA64_CP_REG(cp, r->crn, crm, r->opc0, opc1, opc2);
+        break;
+    default:
+        g_assert_not_reached();
+    }
+
     /* Combine cpreg and name into one allocation. */
     name_len = strlen(name) + 1;
     r2 = g_malloc(sizeof(*r2) + name_len);
@@ -8554,12 +8580,6 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
         }
 
         if (r->state == ARM_CP_STATE_BOTH) {
-            /* We assume it is a cp15 register if the .cp field is left unset.
-             */
-            if (r2->cp == 0) {
-                r2->cp = 15;
-            }
-
 #if HOST_BIG_ENDIAN
             if (r2->fieldoffset) {
                 r2->fieldoffset += sizeof(uint32_t);
@@ -8567,22 +8587,6 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
 #endif
         }
     }
-    if (state == ARM_CP_STATE_AA64) {
-        /* To allow abbreviation of ARMCPRegInfo
-         * definitions, we treat cp == 0 as equivalent to
-         * the value for "standard guest-visible sysreg".
-         * STATE_BOTH definitions are also always "standard
-         * sysreg" in their AArch64 view (the .cp value may
-         * be non-zero for the benefit of the AArch32 view).
-         */
-        if (r->cp == 0 || r->state == ARM_CP_STATE_BOTH) {
-            r2->cp = CP_REG_ARM64_SYSREG_CP;
-        }
-        key = ENCODE_AA64_CP_REG(r2->cp, r2->crn, crm,
-                                 r2->opc0, opc1, opc2);
-    } else {
-        key = ENCODE_CP_REG(r2->cp, is64, ns, r2->crn, crm, opc1, opc2);
-    }
     if (opaque) {
         r2->opaque = opaque;
     }
@@ -8593,6 +8597,7 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
     /* Make sure reginfo passed to helpers for wildcarded regs
      * has the correct crm/opc1/opc2 for this reg, not CP_ANY:
      */
+    r2->cp = cp;
     r2->crm = crm;
     r2->opc1 = opc1;
     r2->opc2 = opc2;

From cc946d96694e47d30c59dbf20f79f53522cc8265 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:49:56 -0700
Subject: [PATCH 029/935] target/arm: Consolidate cpreg updates in
 add_cpreg_to_hashtable

Put most of the value writeback to the same place,
and improve the comment that goes with them.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20220501055028.646596-15-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 28 ++++++++++++----------------
 1 file changed, 12 insertions(+), 16 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index cbc873e3e6..8ee96d5c04 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -8543,10 +8543,19 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
     *r2 = *r;
     r2->name = memcpy(r2 + 1, name, name_len);
 
-    /* Reset the secure state to the specific incoming state.  This is
-     * necessary as the register may have been defined with both states.
+    /*
+     * Update fields to match the instantiation, overwiting wildcards
+     * such as CP_ANY, ARM_CP_STATE_BOTH, or ARM_CP_SECSTATE_BOTH.
      */
+    r2->cp = cp;
+    r2->crm = crm;
+    r2->opc1 = opc1;
+    r2->opc2 = opc2;
+    r2->state = state;
     r2->secure = secstate;
+    if (opaque) {
+        r2->opaque = opaque;
+    }
 
     if (r->bank_fieldoffsets[0] && r->bank_fieldoffsets[1]) {
         /* Register is banked (using both entries in array).
@@ -8587,20 +8596,7 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
 #endif
         }
     }
-    if (opaque) {
-        r2->opaque = opaque;
-    }
-    /* reginfo passed to helpers is correct for the actual access,
-     * and is never ARM_CP_STATE_BOTH:
-     */
-    r2->state = state;
-    /* Make sure reginfo passed to helpers for wildcarded regs
-     * has the correct crm/opc1/opc2 for this reg, not CP_ANY:
-     */
-    r2->cp = cp;
-    r2->crm = crm;
-    r2->opc1 = opc1;
-    r2->opc2 = opc2;
+
     /* By convention, for wildcarded registers only the first
      * entry is used for migration; the others are marked as
      * ALIAS so we don't try to transfer the register

From 4c8c4541156c957498183f1ce1d5f3d97e3576c2 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:49:57 -0700
Subject: [PATCH 030/935] target/arm: Use bool for is64 and ns in
 add_cpreg_to_hashtable

Bool is a more appropriate type for these variables.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20220501055028.646596-16-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 8ee96d5c04..bba010d7cf 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -8507,8 +8507,8 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
      */
     uint32_t key;
     ARMCPRegInfo *r2;
-    int is64 = (r->type & ARM_CP_64BIT) ? 1 : 0;
-    int ns = (secstate & ARM_CP_SECSTATE_NS) ? 1 : 0;
+    bool is64 = r->type & ARM_CP_64BIT;
+    bool ns = secstate & ARM_CP_SECSTATE_NS;
     int cp = r->cp;
     size_t name_len;
 

From 10748a965279aaaccf98ddeb253653421735e916 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:49:58 -0700
Subject: [PATCH 031/935] target/arm: Hoist isbanked computation in
 add_cpreg_to_hashtable

Computing isbanked only once makes the code
a bit easier to read.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20220501055028.646596-17-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index bba010d7cf..941b777dea 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -8510,6 +8510,7 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
     bool is64 = r->type & ARM_CP_64BIT;
     bool ns = secstate & ARM_CP_SECSTATE_NS;
     int cp = r->cp;
+    bool isbanked;
     size_t name_len;
 
     switch (state) {
@@ -8557,7 +8558,8 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
         r2->opaque = opaque;
     }
 
-    if (r->bank_fieldoffsets[0] && r->bank_fieldoffsets[1]) {
+    isbanked = r->bank_fieldoffsets[0] && r->bank_fieldoffsets[1];
+    if (isbanked) {
         /* Register is banked (using both entries in array).
          * Overwriting fieldoffset as the array is only used to define
          * banked registers but later only fieldoffset is used.
@@ -8566,7 +8568,7 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
     }
 
     if (state == ARM_CP_STATE_AA32) {
-        if (r->bank_fieldoffsets[0] && r->bank_fieldoffsets[1]) {
+        if (isbanked) {
             /* If the register is banked then we don't need to migrate or
              * reset the 32-bit instance in certain cases:
              *

From dc44545b0df696fd89e7a9634094cbf33ae9006c Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:49:59 -0700
Subject: [PATCH 032/935] target/arm: Perform override check early in
 add_cpreg_to_hashtable

Perform the override check early, so that it is still done
even when we decide to discard an unreachable cpreg.

Use assert not printf+abort.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20220501055028.646596-18-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 22 ++++++++--------------
 1 file changed, 8 insertions(+), 14 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 941b777dea..fa1e7bd462 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -8538,6 +8538,14 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
         g_assert_not_reached();
     }
 
+    /* Overriding of an existing definition must be explicitly requested. */
+    if (!(r->type & ARM_CP_OVERRIDE)) {
+        const ARMCPRegInfo *oldreg = get_arm_cp_reginfo(cpu->cp_regs, key);
+        if (oldreg) {
+            assert(oldreg->type & ARM_CP_OVERRIDE);
+        }
+    }
+
     /* Combine cpreg and name into one allocation. */
     name_len = strlen(name) + 1;
     r2 = g_malloc(sizeof(*r2) + name_len);
@@ -8622,20 +8630,6 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
         assert(!raw_accessors_invalid(r2));
     }
 
-    /* Overriding of an existing definition must be explicitly
-     * requested.
-     */
-    if (!(r->type & ARM_CP_OVERRIDE)) {
-        const ARMCPRegInfo *oldreg = get_arm_cp_reginfo(cpu->cp_regs, key);
-        if (oldreg && !(oldreg->type & ARM_CP_OVERRIDE)) {
-            fprintf(stderr, "Register redefined: cp=%d %d bit "
-                    "crn=%d crm=%d opc1=%d opc2=%d, "
-                    "was %s, now %s\n", r2->cp, 32 + 32 * is64,
-                    r2->crn, r2->crm, r2->opc1, r2->opc2,
-                    oldreg->name, r2->name);
-            g_assert_not_reached();
-        }
-    }
     g_hash_table_insert(cpu->cp_regs, (gpointer)(uintptr_t)key, r2);
 }
 

From 1859f8c35abd06de15e58f0996ae11ae366f50d8 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:50:00 -0700
Subject: [PATCH 033/935] target/arm: Reformat comments in
 add_cpreg_to_hashtable

Put the block comments into the current coding style.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20220501055028.646596-19-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 24 +++++++++++++++---------
 1 file changed, 15 insertions(+), 9 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index fa1e7bd462..81612952f3 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -8496,15 +8496,16 @@ CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
     return cpu_list;
 }
 
+/*
+ * Private utility function for define_one_arm_cp_reg_with_opaque():
+ * add a single reginfo struct to the hash table.
+ */
 static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
                                    void *opaque, CPState state,
                                    CPSecureState secstate,
                                    int crm, int opc1, int opc2,
                                    const char *name)
 {
-    /* Private utility function for define_one_arm_cp_reg_with_opaque():
-     * add a single reginfo struct to the hash table.
-     */
     uint32_t key;
     ARMCPRegInfo *r2;
     bool is64 = r->type & ARM_CP_64BIT;
@@ -8568,7 +8569,8 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
 
     isbanked = r->bank_fieldoffsets[0] && r->bank_fieldoffsets[1];
     if (isbanked) {
-        /* Register is banked (using both entries in array).
+        /*
+         * Register is banked (using both entries in array).
          * Overwriting fieldoffset as the array is only used to define
          * banked registers but later only fieldoffset is used.
          */
@@ -8577,7 +8579,8 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
 
     if (state == ARM_CP_STATE_AA32) {
         if (isbanked) {
-            /* If the register is banked then we don't need to migrate or
+            /*
+             * If the register is banked then we don't need to migrate or
              * reset the 32-bit instance in certain cases:
              *
              * 1) If the register has both 32-bit and 64-bit instances then we
@@ -8592,8 +8595,9 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
                 r2->type |= ARM_CP_ALIAS;
             }
         } else if ((secstate != r->secure) && !ns) {
-            /* The register is not banked so we only want to allow migration of
-             * the non-secure instance.
+            /*
+             * The register is not banked so we only want to allow migration
+             * of the non-secure instance.
              */
             r2->type |= ARM_CP_ALIAS;
         }
@@ -8607,7 +8611,8 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
         }
     }
 
-    /* By convention, for wildcarded registers only the first
+    /*
+     * By convention, for wildcarded registers only the first
      * entry is used for migration; the others are marked as
      * ALIAS so we don't try to transfer the register
      * multiple times. Special registers (ie NOP/WFI) are
@@ -8622,7 +8627,8 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
         r2->type |= ARM_CP_ALIAS | ARM_CP_NO_GDB;
     }
 
-    /* Check that raw accesses are either forbidden or handled. Note that
+    /*
+     * Check that raw accesses are either forbidden or handled. Note that
      * we can't assert this earlier because the setup of fieldoffset for
      * banked registers has to be done first.
      */

From 7d00b4d8d382558f61a168180be6230cda0905d6 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:50:01 -0700
Subject: [PATCH 034/935] target/arm: Remove HOST_BIG_ENDIAN ifdef in
 add_cpreg_to_hashtable

Since e03b56863d2bc, our host endian indicator is unconditionally
set, which means that we can use a normal C condition.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20220501055028.646596-20-richard.henderson@linaro.org
[PMM: quote correct git hash in commit message]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 81612952f3..14ea5caad9 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -8602,12 +8602,9 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
             r2->type |= ARM_CP_ALIAS;
         }
 
-        if (r->state == ARM_CP_STATE_BOTH) {
-#if HOST_BIG_ENDIAN
-            if (r2->fieldoffset) {
-                r2->fieldoffset += sizeof(uint32_t);
-            }
-#endif
+        if (HOST_BIG_ENDIAN &&
+            r->state == ARM_CP_STATE_BOTH && r2->fieldoffset) {
+            r2->fieldoffset += sizeof(uint32_t);
         }
     }
 

From ca56aac59f6ca936b60f7f6bea5ab54e3f70e379 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:50:05 -0700
Subject: [PATCH 035/935] target/arm: Add isar predicates for FEAT_Debugv8p2

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220501055028.646596-24-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu.h | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index d1b558385c..7303103016 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -3704,6 +3704,11 @@ static inline bool isar_feature_aa32_ssbs(const ARMISARegisters *id)
     return FIELD_EX32(id->id_pfr2, ID_PFR2, SSBS) != 0;
 }
 
+static inline bool isar_feature_aa32_debugv8p2(const ARMISARegisters *id)
+{
+    return FIELD_EX32(id->id_dfr0, ID_DFR0, COPDBG) >= 8;
+}
+
 /*
  * 64-bit feature tests via id registers.
  */
@@ -4010,6 +4015,11 @@ static inline bool isar_feature_aa64_ssbs(const ARMISARegisters *id)
     return FIELD_EX64(id->id_aa64pfr1, ID_AA64PFR1, SSBS) != 0;
 }
 
+static inline bool isar_feature_aa64_debugv8p2(const ARMISARegisters *id)
+{
+    return FIELD_EX64(id->id_aa64dfr0, ID_AA64DFR0, DEBUGVER) >= 8;
+}
+
 static inline bool isar_feature_aa64_sve2(const ARMISARegisters *id)
 {
     return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, SVEVER) != 0;
@@ -4093,6 +4103,11 @@ static inline bool isar_feature_any_tts2uxn(const ARMISARegisters *id)
     return isar_feature_aa64_tts2uxn(id) || isar_feature_aa32_tts2uxn(id);
 }
 
+static inline bool isar_feature_any_debugv8p2(const ARMISARegisters *id)
+{
+    return isar_feature_aa64_debugv8p2(id) || isar_feature_aa32_debugv8p2(id);
+}
+
 /*
  * Forward to the above feature tests given an ARMCPU pointer.
  */

From 25e168ab70627bf3368944cf5b1d97490c853007 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 22:50:15 -0700
Subject: [PATCH 036/935] target/arm: Add isar_feature_{aa64,any}_ras

Add the aa64 predicate for detecting RAS support from id registers.
We already have the aa32 version from the M-profile work.
Add the 'any' predicate for testing both aa64 and aa32.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220501055028.646596-34-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu.h | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 7303103016..ca01f909a8 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -3886,6 +3886,11 @@ static inline bool isar_feature_aa64_aa32_el1(const ARMISARegisters *id)
     return FIELD_EX64(id->id_aa64pfr0, ID_AA64PFR0, EL1) >= 2;
 }
 
+static inline bool isar_feature_aa64_ras(const ARMISARegisters *id)
+{
+    return FIELD_EX64(id->id_aa64pfr0, ID_AA64PFR0, RAS) != 0;
+}
+
 static inline bool isar_feature_aa64_sve(const ARMISARegisters *id)
 {
     return FIELD_EX64(id->id_aa64pfr0, ID_AA64PFR0, SVE) != 0;
@@ -4108,6 +4113,11 @@ static inline bool isar_feature_any_debugv8p2(const ARMISARegisters *id)
     return isar_feature_aa64_debugv8p2(id) || isar_feature_aa32_debugv8p2(id);
 }
 
+static inline bool isar_feature_any_ras(const ARMISARegisters *id)
+{
+    return isar_feature_aa64_ras(id) || isar_feature_aa32_ras(id);
+}
+
 /*
  * Forward to the above feature tests given an ARMCPU pointer.
  */

From 99a50d1a67c602126fc2b3a4812d3000eba9bf34 Mon Sep 17 00:00:00 2001
From: Alex Zuepke <alex.zuepke@tum.de>
Date: Thu, 28 Apr 2022 15:27:17 +0200
Subject: [PATCH 037/935] target/arm: read access to performance counters from
 EL0

The ARMv8 manual defines that PMUSERENR_EL0.ER enables read-access
to both PMXEVCNTR_EL0 and PMEVCNTR<n>_EL0 registers, however,
we only use it for PMXEVCNTR_EL0. Extend to PMEVCNTR<n>_EL0 as well.

Signed-off-by: Alex Zuepke <alex.zuepke@tum.de>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220428132717.84190-1-alex.zuepke@tum.de
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 14ea5caad9..b4daf4f076 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -6639,10 +6639,10 @@ static void define_pmu_regs(ARMCPU *cpu)
               .crm = 8 | (3 & (i >> 3)), .opc1 = 0, .opc2 = i & 7,
               .access = PL0_RW, .type = ARM_CP_IO | ARM_CP_ALIAS,
               .readfn = pmevcntr_readfn, .writefn = pmevcntr_writefn,
-              .accessfn = pmreg_access },
+              .accessfn = pmreg_access_xevcntr },
             { .name = pmevcntr_el0_name, .state = ARM_CP_STATE_AA64,
               .opc0 = 3, .opc1 = 3, .crn = 14, .crm = 8 | (3 & (i >> 3)),
-              .opc2 = i & 7, .access = PL0_RW, .accessfn = pmreg_access,
+              .opc2 = i & 7, .access = PL0_RW, .accessfn = pmreg_access_xevcntr,
               .type = ARM_CP_IO,
               .readfn = pmevcntr_readfn, .writefn = pmevcntr_writefn,
               .raw_readfn = pmevcntr_rawread,

From 942069e0d2164e8c88a55f36a117a196a3920dc5 Mon Sep 17 00:00:00 2001
From: Daniel Henrique Barboza <danielhb413@gmail.com>
Date: Wed, 30 Mar 2022 21:17:14 -0300
Subject: [PATCH 038/935] target/ppc: initialize 'val' union in
 kvm_get_one_spr()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Valgrind isn't convinced that we are initializing the values we assign
to env->spr[spr] because it doesn't understand that the 'val' union is
being written by the kvm_vcpu_ioctl() that follows (via struct
kvm_one_reg).

This results in Valgrind complaining about uninitialized values every
time we use env->spr in a conditional, like this instance:

==707578== Thread 1:
==707578== Conditional jump or move depends on uninitialised value(s)
==707578==    at 0xA10A40: hreg_compute_hflags_value (helper_regs.c:106)
==707578==    by 0xA10C9F: hreg_compute_hflags (helper_regs.c:173)
==707578==    by 0xA110F7: hreg_store_msr (helper_regs.c:262)
==707578==    by 0xA051A3: ppc_cpu_reset (cpu_init.c:7168)
==707578==    by 0xD4730F: device_transitional_reset (qdev.c:799)
==707578==    by 0xD4A11B: resettable_phase_hold (resettable.c:182)
==707578==    by 0xD49A77: resettable_assert_reset (resettable.c:60)
==707578==    by 0xD4994B: resettable_reset (resettable.c:45)
==707578==    by 0xD458BB: device_cold_reset (qdev.c:296)
==707578==    by 0x48FBC7: cpu_reset (cpu-common.c:114)
==707578==    by 0x97B5EB: spapr_reset_vcpu (spapr_cpu_core.c:38)
==707578==    by 0x97BABB: spapr_cpu_core_reset (spapr_cpu_core.c:209)
==707578==  Uninitialised value was created by a stack allocation
==707578==    at 0xB11F08: kvm_get_one_spr (kvm.c:543)

Initializing 'val' has no impact in the logic and makes Valgrind output
more bearable.

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Message-Id: <20220331001717.616938-2-danielhb413@gmail.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/kvm.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
index a3c31b4e48..01baea467f 100644
--- a/target/ppc/kvm.c
+++ b/target/ppc/kvm.c
@@ -542,10 +542,11 @@ static void kvm_get_one_spr(CPUState *cs, uint64_t id, int spr)
 {
     PowerPCCPU *cpu = POWERPC_CPU(cs);
     CPUPPCState *env = &cpu->env;
+    /* Init 'val' to avoid "uninitialised value" Valgrind warnings */
     union {
         uint32_t u32;
         uint64_t u64;
-    } val;
+    } val = { };
     struct kvm_one_reg reg = {
         .id = id,
         .addr = (uintptr_t) &val,

From 59411579b2ce0010442154f604d7062566c10040 Mon Sep 17 00:00:00 2001
From: Daniel Henrique Barboza <danielhb413@gmail.com>
Date: Wed, 30 Mar 2022 21:17:15 -0300
Subject: [PATCH 039/935] target/ppc: init 'lpcr' in
 kvmppc_enable_cap_large_decr()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

'lpcr' is used as an input of kvm_get_one_reg(). Valgrind doesn't
understand that and it returns warnings as such for this function:

==55240== Thread 1:
==55240== Conditional jump or move depends on uninitialised value(s)
==55240==    at 0xB011E4: kvmppc_enable_cap_large_decr (kvm.c:2546)
==55240==    by 0x92F28F: cap_large_decr_cpu_apply (spapr_caps.c:523)
==55240==    by 0x930C37: spapr_caps_cpu_apply (spapr_caps.c:921)
==55240==    by 0x955D3B: spapr_reset_vcpu (spapr_cpu_core.c:73)
==55240==    by 0x95612B: spapr_cpu_core_reset (spapr_cpu_core.c:209)
==55240==    by 0x95619B: spapr_cpu_core_reset_handler (spapr_cpu_core.c:218)
==55240==    by 0xD3605F: qemu_devices_reset (reset.c:69)
==55240==    by 0x92112B: spapr_machine_reset (spapr.c:1641)
==55240==    by 0x4FBD63: qemu_system_reset (runstate.c:444)
==55240==    by 0x62812B: qdev_machine_creation_done (machine.c:1247)
==55240==    by 0x5064C3: qemu_machine_creation_done (vl.c:2725)
==55240==    by 0x5065DF: qmp_x_exit_preconfig (vl.c:2748)
==55240==  Uninitialised value was created by a stack allocation
==55240==    at 0xB01158: kvmppc_enable_cap_large_decr (kvm.c:2540)

Init 'lpcr' to avoid this warning.

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Message-Id: <20220331001717.616938-3-danielhb413@gmail.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/kvm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
index 01baea467f..69094ddb7a 100644
--- a/target/ppc/kvm.c
+++ b/target/ppc/kvm.c
@@ -2537,7 +2537,7 @@ int kvmppc_get_cap_large_decr(void)
 int kvmppc_enable_cap_large_decr(PowerPCCPU *cpu, int enable)
 {
     CPUState *cs = CPU(cpu);
-    uint64_t lpcr;
+    uint64_t lpcr = 0;
 
     kvm_get_one_reg(cs, KVM_REG_PPC_LPCR_64, &lpcr);
     /* Do we need to modify the LPCR? */

From b339427cfc90f7cd158f36eb0cb301c15f582d50 Mon Sep 17 00:00:00 2001
From: Daniel Henrique Barboza <danielhb413@gmail.com>
Date: Wed, 30 Mar 2022 21:17:16 -0300
Subject: [PATCH 040/935] target/ppc: init 'sregs' in kvmppc_put_books_sregs()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Init 'sregs' to avoid Valgrind complaints about uninitialized bytes
from kvmppc_put_books_sregs():

==54059== Thread 3:
==54059== Syscall param ioctl(generic) points to uninitialised byte(s)
==54059==    at 0x55864E4: ioctl (in /usr/lib64/libc.so.6)
==54059==    by 0xD1FA23: kvm_vcpu_ioctl (kvm-all.c:3053)
==54059==    by 0xAFB18B: kvmppc_put_books_sregs (kvm.c:891)
==54059==    by 0xAFB47B: kvm_arch_put_registers (kvm.c:949)
==54059==    by 0xD1EDA7: do_kvm_cpu_synchronize_post_init (kvm-all.c:2766)
==54059==    by 0x481AF3: process_queued_cpu_work (cpus-common.c:343)
==54059==    by 0x4EF247: qemu_wait_io_event_common (cpus.c:412)
==54059==    by 0x4EF343: qemu_wait_io_event (cpus.c:436)
==54059==    by 0xD21E83: kvm_vcpu_thread_fn (kvm-accel-ops.c:54)
==54059==    by 0xFFEBF3: qemu_thread_start (qemu-thread-posix.c:556)
==54059==    by 0x54E6DC3: start_thread (in /usr/lib64/libc.so.6)
==54059==    by 0x5596C9F: clone (in /usr/lib64/libc.so.6)
==54059==  Address 0x799d1cc is on thread 3's stack
==54059==  in frame #2, created by kvmppc_put_books_sregs (kvm.c:851)
==54059==  Uninitialised value was created by a stack allocation
==54059==    at 0xAFAEB0: kvmppc_put_books_sregs (kvm.c:851)

This happens because Valgrind does not consider the 'sregs'
initialization done by kvm_vcpu_ioctl() at the end of the function.

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Message-Id: <20220331001717.616938-4-danielhb413@gmail.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/kvm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
index 69094ddb7a..c4180b7270 100644
--- a/target/ppc/kvm.c
+++ b/target/ppc/kvm.c
@@ -850,7 +850,7 @@ static int kvm_put_vpa(CPUState *cs)
 int kvmppc_put_books_sregs(PowerPCCPU *cpu)
 {
     CPUPPCState *env = &cpu->env;
-    struct kvm_sregs sregs;
+    struct kvm_sregs sregs = { };
     int i;
 
     sregs.pvr = env->spr[SPR_PVR];

From 55baf4b584709fb4d675741a4c3bd15cb0b49c91 Mon Sep 17 00:00:00 2001
From: Daniel Henrique Barboza <danielhb413@gmail.com>
Date: Wed, 30 Mar 2022 21:17:17 -0300
Subject: [PATCH 041/935] target/ppc: init 'rmmu_info' in
 kvm_get_radix_page_info()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Init the struct to avoid Valgrind complaints about unitialized bytes,
such as this one:

==39549== Syscall param ioctl(generic) points to uninitialised byte(s)
==39549==    at 0x55864E4: ioctl (in /usr/lib64/libc.so.6)
==39549==    by 0xD1F7EF: kvm_vm_ioctl (kvm-all.c:3035)
==39549==    by 0xAF8F5B: kvm_get_radix_page_info (kvm.c:276)
==39549==    by 0xB00533: kvmppc_host_cpu_class_init (kvm.c:2369)
==39549==    by 0xD3DCE7: type_initialize (object.c:366)
==39549==    by 0xD3FACF: object_class_foreach_tramp (object.c:1071)
==39549==    by 0x502757B: g_hash_table_foreach (in /usr/lib64/libglib-2.0.so.0.7000.5)
==39549==    by 0xD3FC1B: object_class_foreach (object.c:1093)
==39549==    by 0xB0141F: kvm_ppc_register_host_cpu_type (kvm.c:2613)
==39549==    by 0xAF87E7: kvm_arch_init (kvm.c:157)
==39549==    by 0xD1E2A7: kvm_init (kvm-all.c:2595)
==39549==    by 0x8E6E93: accel_init_machine (accel-softmmu.c:39)
==39549==  Address 0x1fff00e208 is on thread 1's stack
==39549==  in frame #2, created by kvm_get_radix_page_info (kvm.c:267)
==39549==  Uninitialised value was created by a stack allocation
==39549==    at 0xAF8EE8: kvm_get_radix_page_info (kvm.c:267)

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Message-Id: <20220331001717.616938-5-danielhb413@gmail.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/kvm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
index c4180b7270..7a777a4d0c 100644
--- a/target/ppc/kvm.c
+++ b/target/ppc/kvm.c
@@ -266,7 +266,7 @@ struct ppc_radix_page_info *kvm_get_radix_page_info(void)
 {
     KVMState *s = KVM_STATE(current_accel());
     struct ppc_radix_page_info *radix_page_info;
-    struct kvm_ppc_rmmu_info rmmu_info;
+    struct kvm_ppc_rmmu_info rmmu_info = { };
     int i;
 
     if (!kvm_check_extension(s, KVM_CAP_PPC_MMU_RADIX)) {

From 1220ab3ee2bff4a3932cd40e09553ee6bbfaa8a4 Mon Sep 17 00:00:00 2001
From: Bin Meng <bin.meng@windriver.com>
Date: Thu, 21 Apr 2022 09:17:29 +0800
Subject: [PATCH 042/935] target/ppc: Fix BookE debug interrupt generation
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Per E500 core reference manual [1], chapter 8.4.4 "Branch Taken Debug
Event" and chapter 8.4.5 "Instruction Complete Debug Event":

  "A branch taken debug event occurs if both MSR[DE] and DBCR0[BRT]
  are set ... Branch taken debug events are not recognized if MSR[DE]
  is cleared when the branch instruction executes."

  "An instruction complete debug event occurs when any instruction
  completes execution so long as MSR[DE] and DBCR0[ICMP] are both
  set ... Instruction complete debug events are not recognized if
  MSR[DE] is cleared at the time of the instruction execution."

Current codes do not check MSR.DE bit before setting HFLAGS_SE and
HFLAGS_BE flag, which would cause the immediate debug interrupt to
be generated, e.g.: when DBCR0.ICMP bit is set by guest software
and MSR.DE is not set.

[1] https://www.nxp.com/docs/en/reference-manual/E500CORERM.pdf

Signed-off-by: Bin Meng <bin.meng@windriver.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Fabiano Rosas <farosas@linux.ibm.com>
Reviewed-by: Lucas Mateus Castro <lucas.araujo@eldorado.org.br>
Message-Id: <20220421011729.1148727-1-bmeng.cn@gmail.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/helper_regs.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/ppc/helper_regs.c b/target/ppc/helper_regs.c
index 9a691d6833..77bc57415c 100644
--- a/target/ppc/helper_regs.c
+++ b/target/ppc/helper_regs.c
@@ -63,10 +63,10 @@ static uint32_t hreg_compute_hflags_value(CPUPPCState *env)
 
     if (ppc_flags & POWERPC_FLAG_DE) {
         target_ulong dbcr0 = env->spr[SPR_BOOKE_DBCR0];
-        if (dbcr0 & DBCR0_ICMP) {
+        if ((dbcr0 & DBCR0_ICMP) && msr_de) {
             hflags |= 1 << HFLAGS_SE;
         }
-        if (dbcr0 & DBCR0_BRT) {
+        if ((dbcr0 & DBCR0_BRT) && msr_de) {
             hflags |= 1 << HFLAGS_BE;
         }
     } else {

From 97252353c1f6ecbb54385c9272378b5788749a16 Mon Sep 17 00:00:00 2001
From: Murilo Opsfelder Araujo <muriloo@linux.ibm.com>
Date: Tue, 3 May 2022 15:01:08 -0300
Subject: [PATCH 043/935] vhost-user: Use correct macro name TARGET_PPC64

The correct name of the macro is TARGET_PPC64.

Fixes: 27598393a232 ("Lift max memory slots limit imposed by vhost-user")
Reported-by: Fabiano Rosas <farosas@linux.ibm.com>
Signed-off-by: Murilo Opsfelder Araujo <muriloo@linux.ibm.com>
Cc: Raphael Norwitz <raphael.norwitz@nutanix.com>
Cc: Peter Turschmid <peter.turschm@nutanix.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Raphael Norwitz <raphael.norwitz@nutanix.com>
Message-Id: <20220503180108.34506-1-muriloo@linux.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/virtio/vhost-user.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index 2d434ff0bc..afd51f79b3 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -51,7 +51,7 @@
 #include "hw/acpi/acpi.h"
 #define VHOST_USER_MAX_RAM_SLOTS ACPI_MAX_RAM_SLOTS
 
-#elif defined(TARGET_PPC) || defined(TARGET_PPC_64)
+#elif defined(TARGET_PPC) || defined(TARGET_PPC64)
 #include "hw/ppc/spapr.h"
 #define VHOST_USER_MAX_RAM_SLOTS SPAPR_MAX_RAM_SLOTS
 

From a66257a287f6e2832fb6ecd7587da4933b39cfc4 Mon Sep 17 00:00:00 2001
From: Frederic Barrat <fbarrat@linux.ibm.com>
Date: Fri, 29 Apr 2022 09:16:19 +0200
Subject: [PATCH 044/935] ppc/xive: Always recompute the PIPR when pushing an
 OS context
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The Post Interrupt Priority Register (PIPR) is not restored like the
other OS-context related fields of the TIMA when pushing an OS context
on the CPU. It's not needed because it can be calculated from the
Interrupt Pending Buffer (IPB), which is saved and restored. The PIPR
must therefore always be recomputed when pushing an OS context.

This patch fixes a path on P9 and P10 where it was not done. If there
was a pending interrupt when the OS context was pulled, the IPB was
saved correctly. When pushing back the context, the code in
xive_tctx_need_resend() was checking for a interrupt raised while the
context was not on the CPU, saved in the NVT. If one was found, then
it was merged with the saved IPB and the PIPR updated and everything
was fine. However, if there was no interrupt found in the NVT, then
xive_tctx_ipb_update() was not being called and the PIPR was not
updated. This patch fixes it by always calling xive_tctx_ipb_update().

Note that on P10 (xive2.c) and because of the above, there's no longer
any need to check the CPPR value so it can go away.

Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Frederic Barrat <fbarrat@linux.ibm.com>
Message-Id: <20220429071620.177142-2-fbarrat@linux.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/intc/xive.c  | 11 ++++++++---
 hw/intc/xive2.c | 16 +++++++++-------
 2 files changed, 17 insertions(+), 10 deletions(-)

diff --git a/hw/intc/xive.c b/hw/intc/xive.c
index b8e4c7294d..c729f6a478 100644
--- a/hw/intc/xive.c
+++ b/hw/intc/xive.c
@@ -413,10 +413,15 @@ static void xive_tctx_need_resend(XiveRouter *xrtr, XiveTCTX *tctx,
         /* Reset the NVT value */
         nvt.w4 = xive_set_field32(NVT_W4_IPB, nvt.w4, 0);
         xive_router_write_nvt(xrtr, nvt_blk, nvt_idx, &nvt, 4);
-
-        /* Merge in current context */
-        xive_tctx_ipb_update(tctx, TM_QW1_OS, ipb);
     }
+    /*
+     * Always call xive_tctx_ipb_update(). Even if there were no
+     * escalation triggered, there could be a pending interrupt which
+     * was saved when the context was pulled and that we need to take
+     * into account by recalculating the PIPR (which is not
+     * saved/restored).
+     */
+    xive_tctx_ipb_update(tctx, TM_QW1_OS, ipb);
 }
 
 /*
diff --git a/hw/intc/xive2.c b/hw/intc/xive2.c
index 3aff42a69e..400fd70aa8 100644
--- a/hw/intc/xive2.c
+++ b/hw/intc/xive2.c
@@ -316,7 +316,6 @@ static void xive2_tctx_need_resend(Xive2Router *xrtr, XiveTCTX *tctx,
 {
     Xive2Nvp nvp;
     uint8_t ipb;
-    uint8_t cppr = 0;
 
     /*
      * Grab the associated thread interrupt context registers in the
@@ -337,7 +336,7 @@ static void xive2_tctx_need_resend(Xive2Router *xrtr, XiveTCTX *tctx,
     /* Automatically restore thread context registers */
     if (xive2_router_get_config(xrtr) & XIVE2_VP_SAVE_RESTORE &&
         do_restore) {
-        cppr = xive2_tctx_restore_os_ctx(xrtr, tctx, nvp_blk, nvp_idx, &nvp);
+        xive2_tctx_restore_os_ctx(xrtr, tctx, nvp_blk, nvp_idx, &nvp);
     }
 
     ipb = xive_get_field32(NVP2_W2_IPB, nvp.w2);
@@ -345,11 +344,14 @@ static void xive2_tctx_need_resend(Xive2Router *xrtr, XiveTCTX *tctx,
         nvp.w2 = xive_set_field32(NVP2_W2_IPB, nvp.w2, 0);
         xive2_router_write_nvp(xrtr, nvp_blk, nvp_idx, &nvp, 2);
     }
-
-    /* An IPB or CPPR change can trigger a resend */
-    if (ipb || cppr) {
-        xive_tctx_ipb_update(tctx, TM_QW1_OS, ipb);
-    }
+    /*
+     * Always call xive_tctx_ipb_update(). Even if there were no
+     * escalation triggered, there could be a pending interrupt which
+     * was saved when the context was pulled and that we need to take
+     * into account by recalculating the PIPR (which is not
+     * saved/restored).
+     */
+    xive_tctx_ipb_update(tctx, TM_QW1_OS, ipb);
 }
 
 /*

From f65772118718bd8c04dde2618c2c272ae2fe8939 Mon Sep 17 00:00:00 2001
From: Frederic Barrat <fbarrat@linux.ibm.com>
Date: Fri, 29 Apr 2022 09:16:20 +0200
Subject: [PATCH 045/935] ppc/xive: Update the state of the External interrupt
 signal
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

When pulling or pushing an OS context from/to a CPU, we should
re-evaluate the state of the External interrupt signal. Otherwise, we
can end up catching the External interrupt exception in hypervisor
mode, which is unexpected.

The problem is best illustrated with the following scenario:

1. an External interrupt is raised while the guest is on the CPU.

2. before the guest can ack the External interrupt, an hypervisor
interrupt is raised, for example the Hypervisor Decrementer or
Hypervisor Virtualization interrupt. The hypervisor interrupt forces
the guest to exit while the External interrupt is still pending.

3. the hypervisor handles the hypervisor interrupt. At this point, the
External interrupt is still pending. So it's very likely to be
delivered while the hypervisor is running. That's unexpected and can
result in an infinite loop where the hypervisor catches the External
interrupt, looks for an interrupt in its hypervisor queue, doesn't
find any, exits the interrupt handler with the External interrupt
still raised, repeat...

The fix is simply to always lower the External interrupt signal when
pulling an OS context. It means it needs to be raised again when
re-pushing the OS context. Fortunately, it's already the case, as we
now always call xive_tctx_ipb_update(), which will raise the signal if
needed.

Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Frederic Barrat <fbarrat@linux.ibm.com>
Message-Id: <20220429071620.177142-3-fbarrat@linux.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/intc/xive.c        | 14 ++++++++++++++
 hw/intc/xive2.c       |  2 ++
 include/hw/ppc/xive.h |  1 +
 3 files changed, 17 insertions(+)

diff --git a/hw/intc/xive.c b/hw/intc/xive.c
index c729f6a478..ae221fed73 100644
--- a/hw/intc/xive.c
+++ b/hw/intc/xive.c
@@ -114,6 +114,17 @@ static void xive_tctx_notify(XiveTCTX *tctx, uint8_t ring)
     }
 }
 
+void xive_tctx_reset_os_signal(XiveTCTX *tctx)
+{
+    /*
+     * Lower the External interrupt. Used when pulling an OS
+     * context. It is necessary to avoid catching it in the hypervisor
+     * context. It should be raised again when re-pushing the OS
+     * context.
+     */
+    qemu_irq_lower(xive_tctx_output(tctx, TM_QW1_OS));
+}
+
 static void xive_tctx_set_cppr(XiveTCTX *tctx, uint8_t ring, uint8_t cppr)
 {
     uint8_t *regs = &tctx->regs[ring];
@@ -388,6 +399,8 @@ static uint64_t xive_tm_pull_os_ctx(XivePresenter *xptr, XiveTCTX *tctx,
     /* Invalidate CAM line */
     qw1w2_new = xive_set_field32(TM_QW1W2_VO, qw1w2, 0);
     xive_tctx_set_os_cam(tctx, qw1w2_new);
+
+    xive_tctx_reset_os_signal(tctx);
     return qw1w2;
 }
 
@@ -420,6 +433,7 @@ static void xive_tctx_need_resend(XiveRouter *xrtr, XiveTCTX *tctx,
      * was saved when the context was pulled and that we need to take
      * into account by recalculating the PIPR (which is not
      * saved/restored).
+     * It will also raise the External interrupt signal if needed.
      */
     xive_tctx_ipb_update(tctx, TM_QW1_OS, ipb);
 }
diff --git a/hw/intc/xive2.c b/hw/intc/xive2.c
index 400fd70aa8..4d9ff41956 100644
--- a/hw/intc/xive2.c
+++ b/hw/intc/xive2.c
@@ -269,6 +269,7 @@ uint64_t xive2_tm_pull_os_ctx(XivePresenter *xptr, XiveTCTX *tctx,
         xive2_tctx_save_os_ctx(xrtr, tctx, nvp_blk, nvp_idx);
     }
 
+    xive_tctx_reset_os_signal(tctx);
     return qw1w2;
 }
 
@@ -350,6 +351,7 @@ static void xive2_tctx_need_resend(Xive2Router *xrtr, XiveTCTX *tctx,
      * was saved when the context was pulled and that we need to take
      * into account by recalculating the PIPR (which is not
      * saved/restored).
+     * It will also raise the External interrupt signal if needed.
      */
     xive_tctx_ipb_update(tctx, TM_QW1_OS, ipb);
 }
diff --git a/include/hw/ppc/xive.h b/include/hw/ppc/xive.h
index 126e4e2c3a..f7eea4ca81 100644
--- a/include/hw/ppc/xive.h
+++ b/include/hw/ppc/xive.h
@@ -527,6 +527,7 @@ Object *xive_tctx_create(Object *cpu, XivePresenter *xptr, Error **errp);
 void xive_tctx_reset(XiveTCTX *tctx);
 void xive_tctx_destroy(XiveTCTX *tctx);
 void xive_tctx_ipb_update(XiveTCTX *tctx, uint8_t ring, uint8_t ipb);
+void xive_tctx_reset_os_signal(XiveTCTX *tctx);
 
 /*
  * KVM XIVE device helpers

From 208d803326416b43e51e3476826813869050a350 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:20 -0300
Subject: [PATCH 046/935] target/ppc: Remove fpscr_* macros from cpu.h
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

fpscr_* defined macros are hiding the usage of *env behind them.
Substitute the usage of these macros with `env->fpscr & FP_*` to make
the code cleaner.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Message-Id: <20220504210541.115256-2-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.c        |  2 +-
 target/ppc/cpu.h        | 29 -----------------------------
 target/ppc/fpu_helper.c | 28 ++++++++++++++--------------
 3 files changed, 15 insertions(+), 44 deletions(-)

diff --git a/target/ppc/cpu.c b/target/ppc/cpu.c
index d7b42bae52..401b6f9e63 100644
--- a/target/ppc/cpu.c
+++ b/target/ppc/cpu.c
@@ -88,7 +88,7 @@ static inline void fpscr_set_rounding_mode(CPUPPCState *env)
     int rnd_type;
 
     /* Set rounding mode */
-    switch (fpscr_rn) {
+    switch (env->fpscr & FP_RN) {
     case 0:
         /* Best approximation (round to nearest) */
         rnd_type = float_round_nearest_even;
diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index c2b6c987c0..ad31e51d69 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -713,41 +713,12 @@ enum {
 #define FPSCR_NI     2  /* Floating-point non-IEEE mode                      */
 #define FPSCR_RN1    1
 #define FPSCR_RN0    0  /* Floating-point rounding control                   */
-#define fpscr_drn    (((env->fpscr) & FP_DRN) >> FPSCR_DRN0)
-#define fpscr_fex    (((env->fpscr) >> FPSCR_FEX)    & 0x1)
-#define fpscr_vx     (((env->fpscr) >> FPSCR_VX)     & 0x1)
-#define fpscr_ox     (((env->fpscr) >> FPSCR_OX)     & 0x1)
-#define fpscr_ux     (((env->fpscr) >> FPSCR_UX)     & 0x1)
-#define fpscr_zx     (((env->fpscr) >> FPSCR_ZX)     & 0x1)
-#define fpscr_xx     (((env->fpscr) >> FPSCR_XX)     & 0x1)
-#define fpscr_vxsnan (((env->fpscr) >> FPSCR_VXSNAN) & 0x1)
-#define fpscr_vxisi  (((env->fpscr) >> FPSCR_VXISI)  & 0x1)
-#define fpscr_vxidi  (((env->fpscr) >> FPSCR_VXIDI)  & 0x1)
-#define fpscr_vxzdz  (((env->fpscr) >> FPSCR_VXZDZ)  & 0x1)
-#define fpscr_vximz  (((env->fpscr) >> FPSCR_VXIMZ)  & 0x1)
-#define fpscr_vxvc   (((env->fpscr) >> FPSCR_VXVC)   & 0x1)
-#define fpscr_fpcc   (((env->fpscr) >> FPSCR_FPCC)   & 0xF)
-#define fpscr_vxsoft (((env->fpscr) >> FPSCR_VXSOFT) & 0x1)
-#define fpscr_vxsqrt (((env->fpscr) >> FPSCR_VXSQRT) & 0x1)
-#define fpscr_vxcvi  (((env->fpscr) >> FPSCR_VXCVI)  & 0x1)
-#define fpscr_ve     (((env->fpscr) >> FPSCR_VE)     & 0x1)
-#define fpscr_oe     (((env->fpscr) >> FPSCR_OE)     & 0x1)
-#define fpscr_ue     (((env->fpscr) >> FPSCR_UE)     & 0x1)
-#define fpscr_ze     (((env->fpscr) >> FPSCR_ZE)     & 0x1)
-#define fpscr_xe     (((env->fpscr) >> FPSCR_XE)     & 0x1)
-#define fpscr_ni     (((env->fpscr) >> FPSCR_NI)     & 0x1)
-#define fpscr_rn     (((env->fpscr) >> FPSCR_RN0)    & 0x3)
 /* Invalid operation exception summary */
 #define FPSCR_IX     ((1 << FPSCR_VXSNAN) | (1 << FPSCR_VXISI)  | \
                       (1 << FPSCR_VXIDI)  | (1 << FPSCR_VXZDZ)  | \
                       (1 << FPSCR_VXIMZ)  | (1 << FPSCR_VXVC)   | \
                       (1 << FPSCR_VXSOFT) | (1 << FPSCR_VXSQRT) | \
                       (1 << FPSCR_VXCVI))
-/* exception summary */
-#define fpscr_ex  (((env->fpscr) >> FPSCR_XX) & 0x1F)
-/* enabled exception summary */
-#define fpscr_eex (((env->fpscr) >> FPSCR_XX) & ((env->fpscr) >> FPSCR_XE) &  \
-                   0x1F)
 
 #define FP_DRN2         (1ull << FPSCR_DRN2)
 #define FP_DRN1         (1ull << FPSCR_DRN1)
diff --git a/target/ppc/fpu_helper.c b/target/ppc/fpu_helper.c
index 99281cc37a..f6c8318a71 100644
--- a/target/ppc/fpu_helper.c
+++ b/target/ppc/fpu_helper.c
@@ -202,7 +202,7 @@ static void finish_invalid_op_excp(CPUPPCState *env, int op, uintptr_t retaddr)
     env->fpscr |= FP_VX;
     /* Update the floating-point exception summary */
     env->fpscr |= FP_FX;
-    if (fpscr_ve != 0) {
+    if (env->fpscr & FP_VE) {
         /* Update the floating-point enabled exception summary */
         env->fpscr |= FP_FEX;
         if (fp_exceptions_enabled(env)) {
@@ -216,7 +216,7 @@ static void finish_invalid_op_arith(CPUPPCState *env, int op,
                                     bool set_fpcc, uintptr_t retaddr)
 {
     env->fpscr &= ~(FP_FR | FP_FI);
-    if (fpscr_ve == 0) {
+    if (!(env->fpscr & FP_VE)) {
         if (set_fpcc) {
             env->fpscr &= ~FP_FPCC;
             env->fpscr |= (FP_C | FP_FU);
@@ -286,7 +286,7 @@ static void float_invalid_op_vxvc(CPUPPCState *env, bool set_fpcc,
     /* Update the floating-point exception summary */
     env->fpscr |= FP_FX;
     /* We must update the target FPR before raising the exception */
-    if (fpscr_ve != 0) {
+    if (env->fpscr & FP_VE) {
         CPUState *cs = env_cpu(env);
 
         cs->exception_index = POWERPC_EXCP_PROGRAM;
@@ -303,7 +303,7 @@ static void float_invalid_op_vxcvi(CPUPPCState *env, bool set_fpcc,
 {
     env->fpscr |= FP_VXCVI;
     env->fpscr &= ~(FP_FR | FP_FI);
-    if (fpscr_ve == 0) {
+    if (!(env->fpscr & FP_VE)) {
         if (set_fpcc) {
             env->fpscr &= ~FP_FPCC;
             env->fpscr |= (FP_C | FP_FU);
@@ -318,7 +318,7 @@ static inline void float_zero_divide_excp(CPUPPCState *env, uintptr_t raddr)
     env->fpscr &= ~(FP_FR | FP_FI);
     /* Update the floating-point exception summary */
     env->fpscr |= FP_FX;
-    if (fpscr_ze != 0) {
+    if (env->fpscr & FP_ZE) {
         /* Update the floating-point enabled exception summary */
         env->fpscr |= FP_FEX;
         if (fp_exceptions_enabled(env)) {
@@ -336,7 +336,7 @@ static inline void float_overflow_excp(CPUPPCState *env)
     env->fpscr |= FP_OX;
     /* Update the floating-point exception summary */
     env->fpscr |= FP_FX;
-    if (fpscr_oe != 0) {
+    if (env->fpscr & FP_OE) {
         /* XXX: should adjust the result */
         /* Update the floating-point enabled exception summary */
         env->fpscr |= FP_FEX;
@@ -356,7 +356,7 @@ static inline void float_underflow_excp(CPUPPCState *env)
     env->fpscr |= FP_UX;
     /* Update the floating-point exception summary */
     env->fpscr |= FP_FX;
-    if (fpscr_ue != 0) {
+    if (env->fpscr & FP_UE) {
         /* XXX: should adjust the result */
         /* Update the floating-point enabled exception summary */
         env->fpscr |= FP_FEX;
@@ -374,7 +374,7 @@ static inline void float_inexact_excp(CPUPPCState *env)
     env->fpscr |= FP_XX;
     /* Update the floating-point exception summary */
     env->fpscr |= FP_FX;
-    if (fpscr_xe != 0) {
+    if (env->fpscr & FP_XE) {
         /* Update the floating-point enabled exception summary */
         env->fpscr |= FP_FEX;
         /* We must update the target FPR before raising the exception */
@@ -2274,7 +2274,7 @@ VSX_MADDQ(XSNMSUBQPO, NMSUB_FLGS, 0)
         vxvc = svxvc;                                                         \
         if (flags & float_flag_invalid_snan) {                                \
             float_invalid_op_vxsnan(env, GETPC());                            \
-            vxvc &= fpscr_ve == 0;                                            \
+            vxvc &= !(env->fpscr & FP_VE);                                    \
         }                                                                     \
         if (vxvc) {                                                           \
             float_invalid_op_vxvc(env, 0, GETPC());                           \
@@ -2375,7 +2375,7 @@ static inline void do_scalar_cmp(CPUPPCState *env, ppc_vsr_t *xa, ppc_vsr_t *xb,
         if (float64_is_signaling_nan(xa->VsrD(0), &env->fp_status) ||
             float64_is_signaling_nan(xb->VsrD(0), &env->fp_status)) {
             vxsnan_flag = true;
-            if (fpscr_ve == 0 && ordered) {
+            if (!(env->fpscr & FP_VE) && ordered) {
                 vxvc_flag = true;
             }
         } else if (float64_is_quiet_nan(xa->VsrD(0), &env->fp_status) ||
@@ -2440,7 +2440,7 @@ static inline void do_scalar_cmpq(CPUPPCState *env, ppc_vsr_t *xa,
         if (float128_is_signaling_nan(xa->f128, &env->fp_status) ||
             float128_is_signaling_nan(xb->f128, &env->fp_status)) {
             vxsnan_flag = true;
-            if (fpscr_ve == 0 && ordered) {
+            if (!(env->fpscr & FP_VE) && ordered) {
                 vxvc_flag = true;
             }
         } else if (float128_is_quiet_nan(xa->f128, &env->fp_status) ||
@@ -2590,7 +2590,7 @@ void helper_##name(CPUPPCState *env,                                          \
         t.VsrD(0) = xb->VsrD(0);                                              \
     }                                                                         \
                                                                               \
-    vex_flag = fpscr_ve & vxsnan_flag;                                        \
+    vex_flag = (env->fpscr & FP_VE) && vxsnan_flag;                           \
     if (vxsnan_flag) {                                                        \
         float_invalid_op_vxsnan(env, GETPC());                                \
     }                                                                         \
@@ -3320,7 +3320,7 @@ void helper_xsrqpi(CPUPPCState *env, uint32_t opcode,
     if (r == 0 && rmc == 0) {
         rmode = float_round_ties_away;
     } else if (r == 0 && rmc == 0x3) {
-        rmode = fpscr_rn;
+        rmode = env->fpscr & FP_RN;
     } else if (r == 1) {
         switch (rmc) {
         case 0:
@@ -3374,7 +3374,7 @@ void helper_xsrqpxp(CPUPPCState *env, uint32_t opcode,
     if (r == 0 && rmc == 0) {
         rmode = float_round_ties_away;
     } else if (r == 0 && rmc == 0x3) {
-        rmode = fpscr_rn;
+        rmode = env->fpscr & FP_RN;
     } else if (r == 1) {
         switch (rmc) {
         case 0:

From 92984c96df27d9f5bb783846c0cc2ccc8068a7a7 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:21 -0300
Subject: [PATCH 047/935] target/ppc: Remove unused msr_* macros
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Some msr_* macros are not used anywhere. Remove them as part of
the work to remove all hidden usage of *env.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Message-Id: <20220504210541.115256-3-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h | 20 --------------------
 1 file changed, 20 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index ad31e51d69..112b456220 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -463,23 +463,14 @@ typedef enum {
 #define HFSCR_MSGP     PPC_BIT(53) /* Privileged Message Send Facilities */
 #define HFSCR_IC_MSGP  0xA
 
-#define msr_sf   ((env->msr >> MSR_SF)   & 1)
-#define msr_isf  ((env->msr >> MSR_ISF)  & 1)
 #if defined(TARGET_PPC64)
 #define msr_hv   ((env->msr >> MSR_HV)   & 1)
 #else
 #define msr_hv   (0)
 #endif
 #define msr_cm   ((env->msr >> MSR_CM)   & 1)
-#define msr_icm  ((env->msr >> MSR_ICM)  & 1)
 #define msr_gs   ((env->msr >> MSR_GS)   & 1)
-#define msr_ucle ((env->msr >> MSR_UCLE) & 1)
-#define msr_vr   ((env->msr >> MSR_VR)   & 1)
-#define msr_spe  ((env->msr >> MSR_SPE)  & 1)
-#define msr_vsx  ((env->msr >> MSR_VSX)  & 1)
-#define msr_key  ((env->msr >> MSR_KEY)  & 1)
 #define msr_pow  ((env->msr >> MSR_POW)  & 1)
-#define msr_tgpr ((env->msr >> MSR_TGPR) & 1)
 #define msr_ce   ((env->msr >> MSR_CE)   & 1)
 #define msr_ile  ((env->msr >> MSR_ILE)  & 1)
 #define msr_ee   ((env->msr >> MSR_EE)   & 1)
@@ -487,25 +478,14 @@ typedef enum {
 #define msr_fp   ((env->msr >> MSR_FP)   & 1)
 #define msr_me   ((env->msr >> MSR_ME)   & 1)
 #define msr_fe0  ((env->msr >> MSR_FE0)  & 1)
-#define msr_se   ((env->msr >> MSR_SE)   & 1)
-#define msr_dwe  ((env->msr >> MSR_DWE)  & 1)
-#define msr_uble ((env->msr >> MSR_UBLE) & 1)
-#define msr_be   ((env->msr >> MSR_BE)   & 1)
 #define msr_de   ((env->msr >> MSR_DE)   & 1)
 #define msr_fe1  ((env->msr >> MSR_FE1)  & 1)
-#define msr_al   ((env->msr >> MSR_AL)   & 1)
 #define msr_ep   ((env->msr >> MSR_EP)   & 1)
 #define msr_ir   ((env->msr >> MSR_IR)   & 1)
 #define msr_dr   ((env->msr >> MSR_DR)   & 1)
-#define msr_is   ((env->msr >> MSR_IS)   & 1)
 #define msr_ds   ((env->msr >> MSR_DS)   & 1)
-#define msr_pe   ((env->msr >> MSR_PE)   & 1)
-#define msr_px   ((env->msr >> MSR_PX)   & 1)
-#define msr_pmm  ((env->msr >> MSR_PMM)  & 1)
-#define msr_ri   ((env->msr >> MSR_RI)   & 1)
 #define msr_le   ((env->msr >> MSR_LE)   & 1)
 #define msr_ts   ((env->msr >> MSR_TS1)  & 3)
-#define msr_tm   ((env->msr >> MSR_TM)   & 1)
 
 #define DBCR0_ICMP (1 << 27)
 #define DBCR0_BRT (1 << 26)

From d41ccf6eea918ec121cd38eda6e2526b446013f4 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:22 -0300
Subject: [PATCH 048/935] target/ppc: Remove msr_pr macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_pr macro hides the usage of env->msr, which is a bad behavior
Substitute it with FIELD_EX64 calls that explicitly use env->msr
as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-4-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/ppc/pegasos2.c        |  2 +-
 hw/ppc/spapr.c           |  2 +-
 target/ppc/cpu.h         |  4 +++-
 target/ppc/cpu_init.c    |  4 ++--
 target/ppc/excp_helper.c |  8 +++++---
 target/ppc/mem_helper.c  |  5 +++--
 target/ppc/mmu-radix64.c |  5 +++--
 target/ppc/mmu_common.c  | 23 ++++++++++++-----------
 8 files changed, 30 insertions(+), 23 deletions(-)

diff --git a/hw/ppc/pegasos2.c b/hw/ppc/pegasos2.c
index 56bf203dfd..9411ca6b16 100644
--- a/hw/ppc/pegasos2.c
+++ b/hw/ppc/pegasos2.c
@@ -461,7 +461,7 @@ static void pegasos2_hypercall(PPCVirtualHypervisor *vhyp, PowerPCCPU *cpu)
     /* The TCG path should also be holding the BQL at this point */
     g_assert(qemu_mutex_iothread_locked());
 
-    if (msr_pr) {
+    if (FIELD_EX64(env->msr, MSR, PR)) {
         qemu_log_mask(LOG_GUEST_ERROR, "Hypercall made with MSR[PR]=1\n");
         env->gpr[3] = H_PRIVILEGE;
     } else if (env->gpr[3] == KVMPPC_H_RTAS) {
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 22569305d2..fe9937e811 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1269,7 +1269,7 @@ static void emulate_spapr_hypercall(PPCVirtualHypervisor *vhyp,
 
     g_assert(!vhyp_cpu_in_nested(cpu));
 
-    if (msr_pr) {
+    if (FIELD_EX64(env->msr, MSR, PR)) {
         hcall_dprintf("Hypercall made with MSR[PR]=1\n");
         env->gpr[3] = H_PRIVILEGE;
     } else {
diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 112b456220..8f1dc4cb15 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -25,6 +25,7 @@
 #include "exec/cpu-defs.h"
 #include "cpu-qom.h"
 #include "qom/object.h"
+#include "hw/registerfields.h"
 
 #define TCG_GUEST_DEFAULT_MO 0
 
@@ -353,6 +354,8 @@ typedef enum {
 #define MSR_RI   1  /* Recoverable interrupt                        1        */
 #define MSR_LE   0  /* Little-endian mode                           1 hflags */
 
+FIELD(MSR, PR, MSR_PR, 1)
+
 /* PMU bits */
 #define MMCR0_FC     PPC_BIT(32)         /* Freeze Counters  */
 #define MMCR0_PMAO   PPC_BIT(56)         /* Perf Monitor Alert Ocurred */
@@ -474,7 +477,6 @@ typedef enum {
 #define msr_ce   ((env->msr >> MSR_CE)   & 1)
 #define msr_ile  ((env->msr >> MSR_ILE)  & 1)
 #define msr_ee   ((env->msr >> MSR_EE)   & 1)
-#define msr_pr   ((env->msr >> MSR_PR)   & 1)
 #define msr_fp   ((env->msr >> MSR_FP)   & 1)
 #define msr_me   ((env->msr >> MSR_ME)   & 1)
 #define msr_fe0  ((env->msr >> MSR_FE0)  & 1)
diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
index d42e2ba8e0..ac16a64846 100644
--- a/target/ppc/cpu_init.c
+++ b/target/ppc/cpu_init.c
@@ -6303,7 +6303,7 @@ static bool cpu_has_work_POWER9(CPUState *cs)
         if ((env->pending_interrupts & (1u << PPC_INTERRUPT_EXT)) &&
             (env->spr[SPR_LPCR] & LPCR_EEE)) {
             bool heic = !!(env->spr[SPR_LPCR] & LPCR_HEIC);
-            if (heic == 0 || !msr_hv || msr_pr) {
+            if (!heic || !msr_hv || FIELD_EX64(env->msr, MSR, PR)) {
                 return true;
             }
         }
@@ -6517,7 +6517,7 @@ static bool cpu_has_work_POWER10(CPUState *cs)
         if ((env->pending_interrupts & (1u << PPC_INTERRUPT_EXT)) &&
             (env->spr[SPR_LPCR] & LPCR_EEE)) {
             bool heic = !!(env->spr[SPR_LPCR] & LPCR_HEIC);
-            if (heic == 0 || !msr_hv || msr_pr) {
+            if (!heic || !msr_hv || FIELD_EX64(env->msr, MSR, PR)) {
                 return true;
             }
         }
diff --git a/target/ppc/excp_helper.c b/target/ppc/excp_helper.c
index d3e2cfcd71..7e8e34ef06 100644
--- a/target/ppc/excp_helper.c
+++ b/target/ppc/excp_helper.c
@@ -1738,7 +1738,8 @@ static void ppc_hw_interrupt(CPUPPCState *env)
         bool lpes0 = !!(env->spr[SPR_LPCR] & LPCR_LPES0);
         bool heic = !!(env->spr[SPR_LPCR] & LPCR_HEIC);
         /* HEIC blocks delivery to the hypervisor */
-        if ((async_deliver && !(heic && msr_hv && !msr_pr)) ||
+        if ((async_deliver && !(heic && msr_hv &&
+            !FIELD_EX64(env->msr, MSR, PR))) ||
             (env->has_hv_mode && msr_hv == 0 && !lpes0)) {
             if (books_vhyp_promotes_external_to_hvirt(cpu)) {
                 powerpc_excp(cpu, POWERPC_EXCP_HVIRT);
@@ -1818,7 +1819,8 @@ static void ppc_hw_interrupt(CPUPPCState *env)
              * EBB exception must be taken in problem state and
              * with BESCR_GE set.
              */
-            if (msr_pr == 1 && env->spr[SPR_BESCR] & BESCR_GE) {
+            if (FIELD_EX64(env->msr, MSR, PR) &&
+                (env->spr[SPR_BESCR] & BESCR_GE)) {
                 env->pending_interrupts &= ~(1 << PPC_INTERRUPT_EBB);
 
                 if (env->spr[SPR_BESCR] & BESCR_PMEO) {
@@ -2094,7 +2096,7 @@ static void do_ebb(CPUPPCState *env, int ebb_excp)
         env->spr[SPR_BESCR] |= BESCR_EEO;
     }
 
-    if (msr_pr == 1) {
+    if (FIELD_EX64(env->msr, MSR, PR)) {
         powerpc_excp(cpu, ebb_excp);
     } else {
         env->pending_interrupts |= 1 << PPC_INTERRUPT_EBB;
diff --git a/target/ppc/mem_helper.c b/target/ppc/mem_helper.c
index c4ff8fd632..fba7f84b7a 100644
--- a/target/ppc/mem_helper.c
+++ b/target/ppc/mem_helper.c
@@ -613,10 +613,11 @@ void helper_tbegin(CPUPPCState *env)
         (1ULL << TEXASR_FAILURE_PERSISTENT) |
         (1ULL << TEXASR_NESTING_OVERFLOW) |
         (msr_hv << TEXASR_PRIVILEGE_HV) |
-        (msr_pr << TEXASR_PRIVILEGE_PR) |
+        (FIELD_EX64(env->msr, MSR, PR) << TEXASR_PRIVILEGE_PR) |
         (1ULL << TEXASR_FAILURE_SUMMARY) |
         (1ULL << TEXASR_TFIAR_EXACT);
-    env->spr[SPR_TFIAR] = env->nip | (msr_hv << 1) | msr_pr;
+    env->spr[SPR_TFIAR] = env->nip | (msr_hv << 1) |
+                          FIELD_EX64(env->msr, MSR, PR);
     env->spr[SPR_TFHAR] = env->nip + 4;
     env->crf[0] = 0xB; /* 0b1010 = transaction failure */
 }
diff --git a/target/ppc/mmu-radix64.c b/target/ppc/mmu-radix64.c
index 5414fd63c1..e88f51fd34 100644
--- a/target/ppc/mmu-radix64.c
+++ b/target/ppc/mmu-radix64.c
@@ -191,12 +191,13 @@ static bool ppc_radix64_check_prot(PowerPCCPU *cpu, MMUAccessType access_type,
     }
 
     /* Determine permissions allowed by Encoded Access Authority */
-    if (!partition_scoped && (pte & R_PTE_EAA_PRIV) && msr_pr) {
+    if (!partition_scoped && (pte & R_PTE_EAA_PRIV) &&
+        FIELD_EX64(env->msr, MSR, PR)) {
         *prot = 0;
     } else if (mmuidx_pr(mmu_idx) || (pte & R_PTE_EAA_PRIV) ||
                partition_scoped) {
         *prot = ppc_radix64_get_prot_eaa(pte);
-    } else { /* !msr_pr && !(pte & R_PTE_EAA_PRIV) && !partition_scoped */
+    } else { /* !MSR_PR && !(pte & R_PTE_EAA_PRIV) && !partition_scoped */
         *prot = ppc_radix64_get_prot_eaa(pte);
         *prot &= ppc_radix64_get_prot_amr(cpu); /* Least combined permissions */
     }
diff --git a/target/ppc/mmu_common.c b/target/ppc/mmu_common.c
index e9c5b14c0f..6ef8b1c00d 100644
--- a/target/ppc/mmu_common.c
+++ b/target/ppc/mmu_common.c
@@ -273,8 +273,8 @@ static inline void bat_size_prot(CPUPPCState *env, target_ulong *blp,
     bl = (*BATu & 0x00001FFC) << 15;
     valid = 0;
     prot = 0;
-    if (((msr_pr == 0) && (*BATu & 0x00000002)) ||
-        ((msr_pr != 0) && (*BATu & 0x00000001))) {
+    if ((!FIELD_EX64(env->msr, MSR, PR) && (*BATu & 0x00000002)) ||
+        (FIELD_EX64(env->msr, MSR, PR) && (*BATu & 0x00000001))) {
         valid = 1;
         pp = *BATl & 0x00000003;
         if (pp != 0) {
@@ -368,16 +368,17 @@ static int get_segment_6xx_tlb(CPUPPCState *env, mmu_ctx_t *ctx,
     PowerPCCPU *cpu = env_archcpu(env);
     hwaddr hash;
     target_ulong vsid;
-    int ds, pr, target_page_bits;
+    int ds, target_page_bits;
+    bool pr;
     int ret;
     target_ulong sr, pgidx;
 
-    pr = msr_pr;
+    pr = FIELD_EX64(env->msr, MSR, PR);
     ctx->eaddr = eaddr;
 
     sr = env->sr[eaddr >> 28];
-    ctx->key = (((sr & 0x20000000) && (pr != 0)) ||
-                ((sr & 0x40000000) && (pr == 0))) ? 1 : 0;
+    ctx->key = (((sr & 0x20000000) && pr) ||
+                ((sr & 0x40000000) && !pr)) ? 1 : 0;
     ds = sr & 0x80000000 ? 1 : 0;
     ctx->nx = sr & 0x10000000 ? 1 : 0;
     vsid = sr & 0x00FFFFFF;
@@ -386,8 +387,8 @@ static int get_segment_6xx_tlb(CPUPPCState *env, mmu_ctx_t *ctx,
                   "Check segment v=" TARGET_FMT_lx " %d " TARGET_FMT_lx
                   " nip=" TARGET_FMT_lx " lr=" TARGET_FMT_lx
                   " ir=%d dr=%d pr=%d %d t=%d\n",
-                  eaddr, (int)(eaddr >> 28), sr, env->nip, env->lr, (int)msr_ir,
-                  (int)msr_dr, pr != 0 ? 1 : 0,
+                  eaddr, (int)(eaddr >> 28), sr, env->nip, env->lr,
+                  (int)msr_ir, (int)msr_dr, pr ? 1 : 0,
                   access_type == MMU_DATA_STORE, type);
     pgidx = (eaddr & ~SEGMENT_MASK_256M) >> target_page_bits;
     hash = vsid ^ pgidx;
@@ -530,7 +531,7 @@ static int mmu40x_get_physical_address(CPUPPCState *env, mmu_ctx_t *ctx,
 
     ret = -1;
     raddr = (hwaddr)-1ULL;
-    pr = msr_pr;
+    pr = FIELD_EX64(env->msr, MSR, PR);
     for (i = 0; i < env->nb_tlb; i++) {
         tlb = &env->tlb.tlbe[i];
         if (ppcemb_tlb_check(env, tlb, &raddr, address,
@@ -618,7 +619,7 @@ static int mmubooke_check_tlb(CPUPPCState *env, ppcemb_tlb_t *tlb,
 
 found_tlb:
 
-    if (msr_pr != 0) {
+    if (FIELD_EX64(env->msr, MSR, PR)) {
         prot2 = tlb->prot & 0xF;
     } else {
         prot2 = (tlb->prot >> 4) & 0xF;
@@ -768,7 +769,7 @@ static bool mmubooke206_get_as(CPUPPCState *env,
         return true;
     } else {
         *as_out = msr_ds;
-        *pr_out = msr_pr;
+        *pr_out = FIELD_EX64(env->msr, MSR, PR);
         return false;
     }
 }

From 1922322ce43091cfb004a4da8d5851b3630718f1 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:23 -0300
Subject: [PATCH 049/935] target/ppc: Remove msr_le macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_le macro hides the usage of env->msr, which is a bad behavior
Substitute it with FIELD_EX64 calls that explicitly use env->msr
as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-5-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h        |  2 +-
 target/ppc/cpu_init.c   |  2 +-
 target/ppc/gdbstub.c    |  2 +-
 target/ppc/mem_helper.c | 16 ++++++++--------
 4 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 8f1dc4cb15..c561d664de 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -355,6 +355,7 @@ typedef enum {
 #define MSR_LE   0  /* Little-endian mode                           1 hflags */
 
 FIELD(MSR, PR, MSR_PR, 1)
+FIELD(MSR, LE, MSR_LE, 1)
 
 /* PMU bits */
 #define MMCR0_FC     PPC_BIT(32)         /* Freeze Counters  */
@@ -486,7 +487,6 @@ FIELD(MSR, PR, MSR_PR, 1)
 #define msr_ir   ((env->msr >> MSR_IR)   & 1)
 #define msr_dr   ((env->msr >> MSR_DR)   & 1)
 #define msr_ds   ((env->msr >> MSR_DS)   & 1)
-#define msr_le   ((env->msr >> MSR_LE)   & 1)
 #define msr_ts   ((env->msr >> MSR_TS1)  & 3)
 
 #define DBCR0_ICMP (1 << 27)
diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
index ac16a64846..0c6b83406e 100644
--- a/target/ppc/cpu_init.c
+++ b/target/ppc/cpu_init.c
@@ -7210,7 +7210,7 @@ static bool ppc_cpu_is_big_endian(CPUState *cs)
 
     cpu_synchronize_state(cs);
 
-    return !msr_le;
+    return !FIELD_EX64(env->msr, MSR, LE);
 }
 
 #ifdef CONFIG_TCG
diff --git a/target/ppc/gdbstub.c b/target/ppc/gdbstub.c
index 1252429a2a..1a0b9ca82c 100644
--- a/target/ppc/gdbstub.c
+++ b/target/ppc/gdbstub.c
@@ -95,7 +95,7 @@ static int ppc_gdb_register_len(int n)
 void ppc_maybe_bswap_register(CPUPPCState *env, uint8_t *mem_buf, int len)
 {
 #ifndef CONFIG_USER_ONLY
-    if (!msr_le) {
+    if (!FIELD_EX64(env->msr, MSR, LE)) {
         /* do nothing */
     } else if (len == 4) {
         bswap32s((uint32_t *)mem_buf);
diff --git a/target/ppc/mem_helper.c b/target/ppc/mem_helper.c
index fba7f84b7a..9af135e88e 100644
--- a/target/ppc/mem_helper.c
+++ b/target/ppc/mem_helper.c
@@ -33,9 +33,9 @@
 static inline bool needs_byteswap(const CPUPPCState *env)
 {
 #if TARGET_BIG_ENDIAN
-  return msr_le;
+  return FIELD_EX64(env->msr, MSR, LE);
 #else
-  return !msr_le;
+  return !FIELD_EX64(env->msr, MSR, LE);
 #endif
 }
 
@@ -470,8 +470,8 @@ uint32_t helper_stqcx_be_parallel(CPUPPCState *env, target_ulong addr,
 #endif
 
 /*
- * We use msr_le to determine index ordering in a vector.  However,
- * byteswapping is not simply controlled by msr_le.  We also need to
+ * We use MSR_LE to determine index ordering in a vector.  However,
+ * byteswapping is not simply controlled by MSR_LE.  We also need to
  * take into account endianness of the target.  This is done for the
  * little-endian PPC64 user-mode target.
  */
@@ -484,7 +484,7 @@ uint32_t helper_stqcx_be_parallel(CPUPPCState *env, target_ulong addr,
         int adjust = HI_IDX * (n_elems - 1);                    \
         int sh = sizeof(r->element[0]) >> 1;                    \
         int index = (addr & 0xf) >> sh;                         \
-        if (msr_le) {                                           \
+        if (FIELD_EX64(env->msr, MSR, LE)) {                    \
             index = n_elems - index - 1;                        \
         }                                                       \
                                                                 \
@@ -511,7 +511,7 @@ LVE(lvewx, cpu_ldl_data_ra, bswap32, u32)
         int adjust = HI_IDX * (n_elems - 1);                            \
         int sh = sizeof(r->element[0]) >> 1;                            \
         int index = (addr & 0xf) >> sh;                                 \
-        if (msr_le) {                                                   \
+        if (FIELD_EX64(env->msr, MSR, LE)) {                            \
             index = n_elems - index - 1;                                \
         }                                                               \
                                                                         \
@@ -545,7 +545,7 @@ void helper_##name(CPUPPCState *env, target_ulong addr,                 \
     t.s128 = int128_zero();                                             \
     if (nb) {                                                           \
         nb = (nb >= 16) ? 16 : nb;                                      \
-        if (msr_le && !lj) {                                            \
+        if (FIELD_EX64(env->msr, MSR, LE) && !lj) {                     \
             for (i = 16; i > 16 - nb; i--) {                            \
                 t.VsrB(i - 1) = cpu_ldub_data_ra(env, addr, GETPC());   \
                 addr = addr_add(env, addr, 1);                          \
@@ -576,7 +576,7 @@ void helper_##name(CPUPPCState *env, target_ulong addr,           \
     }                                                             \
                                                                   \
     nb = (nb >= 16) ? 16 : nb;                                    \
-    if (msr_le && !lj) {                                          \
+    if (FIELD_EX64(env->msr, MSR, LE) && !lj) {                   \
         for (i = 16; i > 16 - nb; i--) {                          \
             cpu_stb_data_ra(env, addr, xt->VsrB(i - 1), GETPC()); \
             addr = addr_add(env, addr, 1);                        \

From 26363616c67633aef7f53db24050b8adfd2da3b6 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:24 -0300
Subject: [PATCH 050/935] target/ppc: Remove msr_ds macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_ds macro hides the usage of env->msr, which is a bad behavior
Substitute it with FIELD_EX64 calls that explicitly use env->msr
as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-6-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h        | 2 +-
 target/ppc/mmu_common.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index c561d664de..5abc612fe0 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -355,6 +355,7 @@ typedef enum {
 #define MSR_LE   0  /* Little-endian mode                           1 hflags */
 
 FIELD(MSR, PR, MSR_PR, 1)
+FIELD(MSR, DS, MSR_DS, 1)
 FIELD(MSR, LE, MSR_LE, 1)
 
 /* PMU bits */
@@ -486,7 +487,6 @@ FIELD(MSR, LE, MSR_LE, 1)
 #define msr_ep   ((env->msr >> MSR_EP)   & 1)
 #define msr_ir   ((env->msr >> MSR_IR)   & 1)
 #define msr_dr   ((env->msr >> MSR_DR)   & 1)
-#define msr_ds   ((env->msr >> MSR_DS)   & 1)
 #define msr_ts   ((env->msr >> MSR_TS1)  & 3)
 
 #define DBCR0_ICMP (1 << 27)
diff --git a/target/ppc/mmu_common.c b/target/ppc/mmu_common.c
index 6ef8b1c00d..7e77b9b84a 100644
--- a/target/ppc/mmu_common.c
+++ b/target/ppc/mmu_common.c
@@ -768,7 +768,7 @@ static bool mmubooke206_get_as(CPUPPCState *env,
         *pr_out = !!(epidr & EPID_EPR);
         return true;
     } else {
-        *as_out = msr_ds;
+        *as_out = FIELD_EX64(env->msr, MSR, DS);
         *pr_out = FIELD_EX64(env->msr, MSR, PR);
         return false;
     }

From 3868540f05c5e678d9cd889700dbcd66330c43cc Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:25 -0300
Subject: [PATCH 051/935] target/ppc: Remove msr_ile macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_ile macro hides the usage of env->msr, which is a bad behavior
Substitute it with FIELD_EX64 calls that explicitly use env->msr
as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-7-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 5abc612fe0..0d5a850794 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -354,6 +354,7 @@ typedef enum {
 #define MSR_RI   1  /* Recoverable interrupt                        1        */
 #define MSR_LE   0  /* Little-endian mode                           1 hflags */
 
+FIELD(MSR, ILE, MSR_ILE, 1)
 FIELD(MSR, PR, MSR_PR, 1)
 FIELD(MSR, DS, MSR_DS, 1)
 FIELD(MSR, LE, MSR_LE, 1)
@@ -477,7 +478,6 @@ FIELD(MSR, LE, MSR_LE, 1)
 #define msr_gs   ((env->msr >> MSR_GS)   & 1)
 #define msr_pow  ((env->msr >> MSR_POW)  & 1)
 #define msr_ce   ((env->msr >> MSR_CE)   & 1)
-#define msr_ile  ((env->msr >> MSR_ILE)  & 1)
 #define msr_ee   ((env->msr >> MSR_EE)   & 1)
 #define msr_fp   ((env->msr >> MSR_FP)   & 1)
 #define msr_me   ((env->msr >> MSR_ME)   & 1)
@@ -2679,7 +2679,7 @@ static inline bool ppc_interrupts_little_endian(PowerPCCPU *cpu, bool hv)
     } else if (pcc->lpcr_mask & LPCR_ILE) {
         ile = !!(env->spr[SPR_LPCR] & LPCR_ILE);
     } else {
-        ile = !!(msr_ile);
+        ile = FIELD_EX64(env->msr, MSR, ILE);
     }
 
     return ile;

From 0939b8f8df43ad163a602d36f989dc9fd6e378f4 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:26 -0300
Subject: [PATCH 052/935] target/ppc: Remove msr_ee macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_ee macro hides the usage of env->msr, which is a bad behavior
Substitute it with FIELD_EX64 calls that explicitly use env->msr
as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-8-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h         |  2 +-
 target/ppc/cpu_init.c    | 15 ++++++++++-----
 target/ppc/excp_helper.c |  2 +-
 target/ppc/kvm.c         |  3 ++-
 4 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 0d5a850794..06667c2c60 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -355,6 +355,7 @@ typedef enum {
 #define MSR_LE   0  /* Little-endian mode                           1 hflags */
 
 FIELD(MSR, ILE, MSR_ILE, 1)
+FIELD(MSR, EE, MSR_EE, 1)
 FIELD(MSR, PR, MSR_PR, 1)
 FIELD(MSR, DS, MSR_DS, 1)
 FIELD(MSR, LE, MSR_LE, 1)
@@ -478,7 +479,6 @@ FIELD(MSR, LE, MSR_LE, 1)
 #define msr_gs   ((env->msr >> MSR_GS)   & 1)
 #define msr_pow  ((env->msr >> MSR_POW)  & 1)
 #define msr_ce   ((env->msr >> MSR_CE)   & 1)
-#define msr_ee   ((env->msr >> MSR_EE)   & 1)
 #define msr_fp   ((env->msr >> MSR_FP)   & 1)
 #define msr_me   ((env->msr >> MSR_ME)   & 1)
 #define msr_fe0  ((env->msr >> MSR_FE0)  & 1)
diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
index 0c6b83406e..10e7c41bc9 100644
--- a/target/ppc/cpu_init.c
+++ b/target/ppc/cpu_init.c
@@ -5949,7 +5949,8 @@ static bool cpu_has_work_POWER7(CPUState *cs)
         }
         return false;
     } else {
-        return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
+        return FIELD_EX64(env->msr, MSR, EE) &&
+               (cs->interrupt_request & CPU_INTERRUPT_HARD);
     }
 }
 
@@ -6120,7 +6121,8 @@ static bool cpu_has_work_POWER8(CPUState *cs)
         }
         return false;
     } else {
-        return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
+        return FIELD_EX64(env->msr, MSR, EE) &&
+               (cs->interrupt_request & CPU_INTERRUPT_HARD);
     }
 }
 
@@ -6337,7 +6339,8 @@ static bool cpu_has_work_POWER9(CPUState *cs)
         }
         return false;
     } else {
-        return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
+        return FIELD_EX64(env->msr, MSR, EE) &&
+               (cs->interrupt_request & CPU_INTERRUPT_HARD);
     }
 }
 
@@ -6551,7 +6554,8 @@ static bool cpu_has_work_POWER10(CPUState *cs)
         }
         return false;
     } else {
-        return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
+        return FIELD_EX64(env->msr, MSR, EE) &&
+               (cs->interrupt_request & CPU_INTERRUPT_HARD);
     }
 }
 
@@ -7119,7 +7123,8 @@ static bool ppc_cpu_has_work(CPUState *cs)
     PowerPCCPU *cpu = POWERPC_CPU(cs);
     CPUPPCState *env = &cpu->env;
 
-    return msr_ee && (cs->interrupt_request & CPU_INTERRUPT_HARD);
+    return FIELD_EX64(env->msr, MSR, EE) &&
+           (cs->interrupt_request & CPU_INTERRUPT_HARD);
 }
 
 static void ppc_cpu_reset(DeviceState *dev)
diff --git a/target/ppc/excp_helper.c b/target/ppc/excp_helper.c
index 7e8e34ef06..4c206ba209 100644
--- a/target/ppc/excp_helper.c
+++ b/target/ppc/excp_helper.c
@@ -1709,7 +1709,7 @@ static void ppc_hw_interrupt(CPUPPCState *env)
      * clear when coming out of some power management states (in order
      * for them to become a 0x100).
      */
-    async_deliver = (msr_ee != 0) || env->resume_as_sreset;
+    async_deliver = FIELD_EX64(env->msr, MSR, EE) || env->resume_as_sreset;
 
     /* Hypervisor decrementer exception */
     if (env->pending_interrupts & (1 << PPC_INTERRUPT_HDECR)) {
diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
index 7a777a4d0c..db3a92869c 100644
--- a/target/ppc/kvm.c
+++ b/target/ppc/kvm.c
@@ -1352,7 +1352,8 @@ static int kvmppc_handle_halt(PowerPCCPU *cpu)
     CPUState *cs = CPU(cpu);
     CPUPPCState *env = &cpu->env;
 
-    if (!(cs->interrupt_request & CPU_INTERRUPT_HARD) && (msr_ee)) {
+    if (!(cs->interrupt_request & CPU_INTERRUPT_HARD) &&
+        FIELD_EX64(env->msr, MSR, EE)) {
         cs->halted = 1;
         cs->exception_index = EXCP_HLT;
     }

From acc861c2e95dec0795581016a6a8b044d28b1d61 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:27 -0300
Subject: [PATCH 053/935] target/ppc: Remove msr_ce macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_ce macro hides the usage of env->msr, which is a bad behavior
Substitute it with FIELD_EX64 calls that explicitly use env->msr
as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-9-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h         | 2 +-
 target/ppc/excp_helper.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 06667c2c60..b1883b23e2 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -354,6 +354,7 @@ typedef enum {
 #define MSR_RI   1  /* Recoverable interrupt                        1        */
 #define MSR_LE   0  /* Little-endian mode                           1 hflags */
 
+FIELD(MSR, CE, MSR_CE, 1)
 FIELD(MSR, ILE, MSR_ILE, 1)
 FIELD(MSR, EE, MSR_EE, 1)
 FIELD(MSR, PR, MSR_PR, 1)
@@ -478,7 +479,6 @@ FIELD(MSR, LE, MSR_LE, 1)
 #define msr_cm   ((env->msr >> MSR_CM)   & 1)
 #define msr_gs   ((env->msr >> MSR_GS)   & 1)
 #define msr_pow  ((env->msr >> MSR_POW)  & 1)
-#define msr_ce   ((env->msr >> MSR_CE)   & 1)
 #define msr_fp   ((env->msr >> MSR_FP)   & 1)
 #define msr_me   ((env->msr >> MSR_ME)   & 1)
 #define msr_fe0  ((env->msr >> MSR_FE0)  & 1)
diff --git a/target/ppc/excp_helper.c b/target/ppc/excp_helper.c
index 4c206ba209..ca80c1ed63 100644
--- a/target/ppc/excp_helper.c
+++ b/target/ppc/excp_helper.c
@@ -1749,7 +1749,7 @@ static void ppc_hw_interrupt(CPUPPCState *env)
             return;
         }
     }
-    if (msr_ce != 0) {
+    if (FIELD_EX64(env->msr, MSR, CE)) {
         /* External critical interrupt */
         if (env->pending_interrupts & (1 << PPC_INTERRUPT_CEXT)) {
             powerpc_excp(cpu, POWERPC_EXCP_CRITICAL);

From 8e54ad65c218a547111c9dfadec792c43b933600 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:28 -0300
Subject: [PATCH 054/935] target/ppc: Remove msr_pow macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_pow macro hides the usage of env->msr, which is a bad behavior
Substitute it with FIELD_EX64 calls that explicitly use env->msr
as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-10-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h         |  2 +-
 target/ppc/excp_helper.c | 12 ++++++------
 target/ppc/helper_regs.c |  2 +-
 3 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index b1883b23e2..5a83c4b028 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -354,6 +354,7 @@ typedef enum {
 #define MSR_RI   1  /* Recoverable interrupt                        1        */
 #define MSR_LE   0  /* Little-endian mode                           1 hflags */
 
+FIELD(MSR, POW, MSR_POW, 1)
 FIELD(MSR, CE, MSR_CE, 1)
 FIELD(MSR, ILE, MSR_ILE, 1)
 FIELD(MSR, EE, MSR_EE, 1)
@@ -478,7 +479,6 @@ FIELD(MSR, LE, MSR_LE, 1)
 #endif
 #define msr_cm   ((env->msr >> MSR_CM)   & 1)
 #define msr_gs   ((env->msr >> MSR_GS)   & 1)
-#define msr_pow  ((env->msr >> MSR_POW)  & 1)
 #define msr_fp   ((env->msr >> MSR_FP)   & 1)
 #define msr_me   ((env->msr >> MSR_ME)   & 1)
 #define msr_fe0  ((env->msr >> MSR_FE0)  & 1)
diff --git a/target/ppc/excp_helper.c b/target/ppc/excp_helper.c
index ca80c1ed63..ee63641dd0 100644
--- a/target/ppc/excp_helper.c
+++ b/target/ppc/excp_helper.c
@@ -661,7 +661,7 @@ static void powerpc_excp_6xx(PowerPCCPU *cpu, int excp)
     case POWERPC_EXCP_ITLB:      /* Instruction TLB error                    */
         break;
     case POWERPC_EXCP_RESET:     /* System reset exception                   */
-        if (msr_pow) {
+        if (FIELD_EX64(env->msr, MSR, POW)) {
             cpu_abort(cs, "Trying to deliver power-saving system reset "
                       "exception %d with no HV support\n", excp);
         }
@@ -853,7 +853,7 @@ static void powerpc_excp_7xx(PowerPCCPU *cpu, int excp)
     case POWERPC_EXCP_DECR:      /* Decrementer exception                    */
         break;
     case POWERPC_EXCP_RESET:     /* System reset exception                   */
-        if (msr_pow) {
+        if (FIELD_EX64(env->msr, MSR, POW)) {
             cpu_abort(cs, "Trying to deliver power-saving system reset "
                       "exception %d with no HV support\n", excp);
         }
@@ -1038,7 +1038,7 @@ static void powerpc_excp_74xx(PowerPCCPU *cpu, int excp)
     case POWERPC_EXCP_DECR:      /* Decrementer exception                    */
         break;
     case POWERPC_EXCP_RESET:     /* System reset exception                   */
-        if (msr_pow) {
+        if (FIELD_EX64(env->msr, MSR, POW)) {
             cpu_abort(cs, "Trying to deliver power-saving system reset "
                       "exception %d with no HV support\n", excp);
         }
@@ -1248,7 +1248,7 @@ static void powerpc_excp_booke(PowerPCCPU *cpu, int excp)
         env->spr[SPR_BOOKE_ESR] = ESR_SPV;
         break;
     case POWERPC_EXCP_RESET:     /* System reset exception                   */
-        if (msr_pow) {
+        if (FIELD_EX64(env->msr, MSR, POW)) {
             cpu_abort(cs, "Trying to deliver power-saving system reset "
                       "exception %d with no HV support\n", excp);
         }
@@ -1507,7 +1507,7 @@ static void powerpc_excp_books(PowerPCCPU *cpu, int excp)
         break;
     case POWERPC_EXCP_RESET:     /* System reset exception                   */
         /* A power-saving exception sets ME, otherwise it is unchanged */
-        if (msr_pow) {
+        if (FIELD_EX64(env->msr, MSR, POW)) {
             /* indicate that we resumed from power save mode */
             msr |= 0x10000;
             new_msr |= ((target_ulong)1 << MSR_ME);
@@ -1519,7 +1519,7 @@ static void powerpc_excp_books(PowerPCCPU *cpu, int excp)
              */
             new_msr |= (target_ulong)MSR_HVB;
         } else {
-            if (msr_pow) {
+            if (FIELD_EX64(env->msr, MSR, POW)) {
                 cpu_abort(cs, "Trying to deliver power-saving system reset "
                           "exception %d with no HV support\n", excp);
             }
diff --git a/target/ppc/helper_regs.c b/target/ppc/helper_regs.c
index 77bc57415c..79c0143a7a 100644
--- a/target/ppc/helper_regs.c
+++ b/target/ppc/helper_regs.c
@@ -261,7 +261,7 @@ int hreg_store_msr(CPUPPCState *env, target_ulong value, int alter_hv)
     env->msr = value;
     hreg_compute_hflags(env);
 #if !defined(CONFIG_USER_ONLY)
-    if (unlikely(msr_pow == 1)) {
+    if (unlikely(FIELD_EX64(env->msr, MSR, POW))) {
         if (!env->pending_interrupts && (*env->check_pow)(env)) {
             cs->halted = 1;
             excp = EXCP_HALTED;

From c354d85828a44e374ccc68b418989dcd26f650db Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:29 -0300
Subject: [PATCH 055/935] target/ppc: Remove msr_me macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_me macro hides the usage of env->msr, which is a bad behavior
Substitute it with FIELD_EX64 calls that explicitly use env->msr
as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-11-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h         |  2 +-
 target/ppc/excp_helper.c | 12 ++++++------
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 5a83c4b028..deb861f5f3 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -359,6 +359,7 @@ FIELD(MSR, CE, MSR_CE, 1)
 FIELD(MSR, ILE, MSR_ILE, 1)
 FIELD(MSR, EE, MSR_EE, 1)
 FIELD(MSR, PR, MSR_PR, 1)
+FIELD(MSR, ME, MSR_ME, 1)
 FIELD(MSR, DS, MSR_DS, 1)
 FIELD(MSR, LE, MSR_LE, 1)
 
@@ -480,7 +481,6 @@ FIELD(MSR, LE, MSR_LE, 1)
 #define msr_cm   ((env->msr >> MSR_CM)   & 1)
 #define msr_gs   ((env->msr >> MSR_GS)   & 1)
 #define msr_fp   ((env->msr >> MSR_FP)   & 1)
-#define msr_me   ((env->msr >> MSR_ME)   & 1)
 #define msr_fe0  ((env->msr >> MSR_FE0)  & 1)
 #define msr_de   ((env->msr >> MSR_DE)   & 1)
 #define msr_fe1  ((env->msr >> MSR_FE1)  & 1)
diff --git a/target/ppc/excp_helper.c b/target/ppc/excp_helper.c
index ee63641dd0..e254ae806c 100644
--- a/target/ppc/excp_helper.c
+++ b/target/ppc/excp_helper.c
@@ -444,7 +444,7 @@ static void powerpc_excp_40x(PowerPCCPU *cpu, int excp)
         srr1 = SPR_40x_SRR3;
         break;
     case POWERPC_EXCP_MCHECK:    /* Machine check exception                  */
-        if (msr_me == 0) {
+        if (!FIELD_EX64(env->msr, MSR, ME)) {
             /*
              * Machine check exception is not enabled.  Enter
              * checkstop state.
@@ -575,7 +575,7 @@ static void powerpc_excp_6xx(PowerPCCPU *cpu, int excp)
     case POWERPC_EXCP_CRITICAL:    /* Critical input                         */
         break;
     case POWERPC_EXCP_MCHECK:    /* Machine check exception                  */
-        if (msr_me == 0) {
+        if (!FIELD_EX64(env->msr, MSR, ME)) {
             /*
              * Machine check exception is not enabled.  Enter
              * checkstop state.
@@ -748,7 +748,7 @@ static void powerpc_excp_7xx(PowerPCCPU *cpu, int excp)
 
     switch (excp) {
     case POWERPC_EXCP_MCHECK:    /* Machine check exception                  */
-        if (msr_me == 0) {
+        if (!FIELD_EX64(env->msr, MSR, ME)) {
             /*
              * Machine check exception is not enabled.  Enter
              * checkstop state.
@@ -933,7 +933,7 @@ static void powerpc_excp_74xx(PowerPCCPU *cpu, int excp)
 
     switch (excp) {
     case POWERPC_EXCP_MCHECK:    /* Machine check exception                  */
-        if (msr_me == 0) {
+        if (!FIELD_EX64(env->msr, MSR, ME)) {
             /*
              * Machine check exception is not enabled.  Enter
              * checkstop state.
@@ -1128,7 +1128,7 @@ static void powerpc_excp_booke(PowerPCCPU *cpu, int excp)
         srr1 = SPR_BOOKE_CSRR1;
         break;
     case POWERPC_EXCP_MCHECK:    /* Machine check exception                  */
-        if (msr_me == 0) {
+        if (!FIELD_EX64(env->msr, MSR, ME)) {
             /*
              * Machine check exception is not enabled.  Enter
              * checkstop state.
@@ -1366,7 +1366,7 @@ static void powerpc_excp_books(PowerPCCPU *cpu, int excp)
 
     switch (excp) {
     case POWERPC_EXCP_MCHECK:    /* Machine check exception                  */
-        if (msr_me == 0) {
+        if (!FIELD_EX64(env->msr, MSR, ME)) {
             /*
              * Machine check exception is not enabled.  Enter
              * checkstop state.

From 10b2b373919ff24c8fabec035f40a4fdb1c40da3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:30 -0300
Subject: [PATCH 056/935] target/ppc: Remove msr_gs macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_gs macro hides the usage of env->msr, which is a bad behavior
Substitute it with FIELD_EX64 calls that explicitly use env->msr
as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-12-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h         | 2 +-
 target/ppc/helper_regs.c | 2 +-
 target/ppc/mmu_helper.c  | 4 ++--
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index deb861f5f3..bd5dffc9b1 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -354,6 +354,7 @@ typedef enum {
 #define MSR_RI   1  /* Recoverable interrupt                        1        */
 #define MSR_LE   0  /* Little-endian mode                           1 hflags */
 
+FIELD(MSR, GS, MSR_GS, 1)
 FIELD(MSR, POW, MSR_POW, 1)
 FIELD(MSR, CE, MSR_CE, 1)
 FIELD(MSR, ILE, MSR_ILE, 1)
@@ -479,7 +480,6 @@ FIELD(MSR, LE, MSR_LE, 1)
 #define msr_hv   (0)
 #endif
 #define msr_cm   ((env->msr >> MSR_CM)   & 1)
-#define msr_gs   ((env->msr >> MSR_GS)   & 1)
 #define msr_fp   ((env->msr >> MSR_FP)   & 1)
 #define msr_fe0  ((env->msr >> MSR_FE0)  & 1)
 #define msr_de   ((env->msr >> MSR_DE)   & 1)
diff --git a/target/ppc/helper_regs.c b/target/ppc/helper_regs.c
index 79c0143a7a..4e649d8b0e 100644
--- a/target/ppc/helper_regs.c
+++ b/target/ppc/helper_regs.c
@@ -233,7 +233,7 @@ int hreg_store_msr(CPUPPCState *env, target_ulong value, int alter_hv)
     }
     if ((env->mmu_model == POWERPC_MMU_BOOKE ||
          env->mmu_model == POWERPC_MMU_BOOKE206) &&
-        ((value >> MSR_GS) & 1) != msr_gs) {
+        ((value ^ env->msr) & R_MSR_GS_MASK)) {
         cpu_interrupt_exittb(cs);
     }
     if (unlikely((env->flags & POWERPC_FLAG_TGPR) &&
diff --git a/target/ppc/mmu_helper.c b/target/ppc/mmu_helper.c
index 142a717255..5bb5c71038 100644
--- a/target/ppc/mmu_helper.c
+++ b/target/ppc/mmu_helper.c
@@ -935,7 +935,7 @@ void helper_booke206_tlbwe(CPUPPCState *env)
     }
 
     if (((env->spr[SPR_BOOKE_MAS0] & MAS0_ATSEL) == MAS0_ATSEL_LRAT) &&
-        !msr_gs) {
+        !FIELD_EX64(env->msr, MSR, GS)) {
         /* XXX we don't support direct LRAT setting yet */
         fprintf(stderr, "cpu: don't support LRAT setting yet\n");
         return;
@@ -962,7 +962,7 @@ void helper_booke206_tlbwe(CPUPPCState *env)
                                POWERPC_EXCP_INVAL_INVAL, GETPC());
     }
 
-    if (msr_gs) {
+    if (FIELD_EX64(env->msr, MSR, GS)) {
         cpu_abort(env_cpu(env), "missing HV implementation\n");
     }
 

From 39695e156ff519a466dda8ad3dd6b37b5ef85e78 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:31 -0300
Subject: [PATCH 057/935] target/ppc: Remove msr_fp macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_fp macro hides the usage of env->msr, which is a bad behavior
Substitute it with FIELD_EX64 calls that explicitly use env->msr
as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-13-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h         |  2 +-
 target/ppc/excp_helper.c | 18 ++++++++++++------
 2 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index bd5dffc9b1..17b7f5f6d9 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -360,6 +360,7 @@ FIELD(MSR, CE, MSR_CE, 1)
 FIELD(MSR, ILE, MSR_ILE, 1)
 FIELD(MSR, EE, MSR_EE, 1)
 FIELD(MSR, PR, MSR_PR, 1)
+FIELD(MSR, FP, MSR_FP, 1)
 FIELD(MSR, ME, MSR_ME, 1)
 FIELD(MSR, DS, MSR_DS, 1)
 FIELD(MSR, LE, MSR_LE, 1)
@@ -480,7 +481,6 @@ FIELD(MSR, LE, MSR_LE, 1)
 #define msr_hv   (0)
 #endif
 #define msr_cm   ((env->msr >> MSR_CM)   & 1)
-#define msr_fp   ((env->msr >> MSR_FP)   & 1)
 #define msr_fe0  ((env->msr >> MSR_FE0)  & 1)
 #define msr_de   ((env->msr >> MSR_DE)   & 1)
 #define msr_fe1  ((env->msr >> MSR_FE1)  & 1)
diff --git a/target/ppc/excp_helper.c b/target/ppc/excp_helper.c
index e254ae806c..30baad0489 100644
--- a/target/ppc/excp_helper.c
+++ b/target/ppc/excp_helper.c
@@ -478,7 +478,8 @@ static void powerpc_excp_40x(PowerPCCPU *cpu, int excp)
     case POWERPC_EXCP_PROGRAM:   /* Program exception                        */
         switch (env->error_code & ~0xF) {
         case POWERPC_EXCP_FP:
-            if ((msr_fe0 == 0 && msr_fe1 == 0) || msr_fp == 0) {
+            if ((msr_fe0 == 0 && msr_fe1 == 0) ||
+                !FIELD_EX64(env->msr, MSR, FP)) {
                 trace_ppc_excp_fp_ignore();
                 powerpc_reset_excp_state(cpu);
                 return;
@@ -615,7 +616,8 @@ static void powerpc_excp_6xx(PowerPCCPU *cpu, int excp)
     case POWERPC_EXCP_PROGRAM:   /* Program exception                        */
         switch (env->error_code & ~0xF) {
         case POWERPC_EXCP_FP:
-            if ((msr_fe0 == 0 && msr_fe1 == 0) || msr_fp == 0) {
+            if ((msr_fe0 == 0 && msr_fe1 == 0) ||
+                !FIELD_EX64(env->msr, MSR, FP)) {
                 trace_ppc_excp_fp_ignore();
                 powerpc_reset_excp_state(cpu);
                 return;
@@ -788,7 +790,8 @@ static void powerpc_excp_7xx(PowerPCCPU *cpu, int excp)
     case POWERPC_EXCP_PROGRAM:   /* Program exception                        */
         switch (env->error_code & ~0xF) {
         case POWERPC_EXCP_FP:
-            if ((msr_fe0 == 0 && msr_fe1 == 0) || msr_fp == 0) {
+            if ((msr_fe0 == 0 && msr_fe1 == 0) ||
+                !FIELD_EX64(env->msr, MSR, FP)) {
                 trace_ppc_excp_fp_ignore();
                 powerpc_reset_excp_state(cpu);
                 return;
@@ -973,7 +976,8 @@ static void powerpc_excp_74xx(PowerPCCPU *cpu, int excp)
     case POWERPC_EXCP_PROGRAM:   /* Program exception                        */
         switch (env->error_code & ~0xF) {
         case POWERPC_EXCP_FP:
-            if ((msr_fe0 == 0 && msr_fe1 == 0) || msr_fp == 0) {
+            if ((msr_fe0 == 0 && msr_fe1 == 0) ||
+                !FIELD_EX64(env->msr, MSR, FP)) {
                 trace_ppc_excp_fp_ignore();
                 powerpc_reset_excp_state(cpu);
                 return;
@@ -1171,7 +1175,8 @@ static void powerpc_excp_booke(PowerPCCPU *cpu, int excp)
     case POWERPC_EXCP_PROGRAM:   /* Program exception                        */
         switch (env->error_code & ~0xF) {
         case POWERPC_EXCP_FP:
-            if ((msr_fe0 == 0 && msr_fe1 == 0) || msr_fp == 0) {
+            if ((msr_fe0 == 0 && msr_fe1 == 0) ||
+                !FIELD_EX64(env->msr, MSR, FP)) {
                 trace_ppc_excp_fp_ignore();
                 powerpc_reset_excp_state(cpu);
                 return;
@@ -1434,7 +1439,8 @@ static void powerpc_excp_books(PowerPCCPU *cpu, int excp)
     case POWERPC_EXCP_PROGRAM:   /* Program exception                        */
         switch (env->error_code & ~0xF) {
         case POWERPC_EXCP_FP:
-            if ((msr_fe0 == 0 && msr_fe1 == 0) || msr_fp == 0) {
+            if ((msr_fe0 == 0 && msr_fe1 == 0) ||
+                !FIELD_EX64(env->msr, MSR, FP)) {
                 trace_ppc_excp_fp_ignore();
                 powerpc_reset_excp_state(cpu);
                 return;

From cda2336027d2ef744d04271810bb64085ab2cd29 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:32 -0300
Subject: [PATCH 058/935] target/ppc: Remove msr_cm macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_cm macro hides the usage of env->msr, which is a bad behavior
Substitute it with FIELD_EX64 calls that explicitly use env->msr
as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-14-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h        | 2 +-
 target/ppc/mmu_common.c | 2 +-
 target/ppc/mmu_helper.c | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 17b7f5f6d9..75a81d4304 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -354,6 +354,7 @@ typedef enum {
 #define MSR_RI   1  /* Recoverable interrupt                        1        */
 #define MSR_LE   0  /* Little-endian mode                           1 hflags */
 
+FIELD(MSR, CM, MSR_CM, 1)
 FIELD(MSR, GS, MSR_GS, 1)
 FIELD(MSR, POW, MSR_POW, 1)
 FIELD(MSR, CE, MSR_CE, 1)
@@ -480,7 +481,6 @@ FIELD(MSR, LE, MSR_LE, 1)
 #else
 #define msr_hv   (0)
 #endif
-#define msr_cm   ((env->msr >> MSR_CM)   & 1)
 #define msr_fe0  ((env->msr >> MSR_FE0)  & 1)
 #define msr_de   ((env->msr >> MSR_DE)   & 1)
 #define msr_fe1  ((env->msr >> MSR_FE1)  & 1)
diff --git a/target/ppc/mmu_common.c b/target/ppc/mmu_common.c
index 7e77b9b84a..031bb4493b 100644
--- a/target/ppc/mmu_common.c
+++ b/target/ppc/mmu_common.c
@@ -692,7 +692,7 @@ int ppcmas_tlb_check(CPUPPCState *env, ppcmas_tlb_t *tlb,
     hwaddr mask;
     uint32_t tlb_pid;
 
-    if (!msr_cm) {
+    if (!FIELD_EX64(env->msr, MSR, CM)) {
         /* In 32bit mode we can only address 32bit EAs */
         address = (uint32_t)address;
     }
diff --git a/target/ppc/mmu_helper.c b/target/ppc/mmu_helper.c
index 5bb5c71038..15239dc95b 100644
--- a/target/ppc/mmu_helper.c
+++ b/target/ppc/mmu_helper.c
@@ -1003,7 +1003,7 @@ void helper_booke206_tlbwe(CPUPPCState *env)
     /* Add a mask for page attributes */
     mask |= MAS2_ACM | MAS2_VLE | MAS2_W | MAS2_I | MAS2_M | MAS2_G | MAS2_E;
 
-    if (!msr_cm) {
+    if (!FIELD_EX64(env->msr, MSR, CM)) {
         /*
          * Executing a tlbwe instruction in 32-bit mode will set bits
          * 0:31 of the TLB EPN field to zero.

From 4d979c9ffbe19931bd8b44810eb3c1c396798911 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:33 -0300
Subject: [PATCH 059/935] target/ppc: Remove msr_ir macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_ir macro hides the usage of env->msr, which is a bad behavior
Substitute it with FIELD_EX64 calls that explicitly use env->msr
as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-15-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h         |  2 +-
 target/ppc/helper_regs.c |  2 +-
 target/ppc/mmu_common.c  | 11 ++++++-----
 3 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 75a81d4304..6cfbec26a1 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -363,6 +363,7 @@ FIELD(MSR, EE, MSR_EE, 1)
 FIELD(MSR, PR, MSR_PR, 1)
 FIELD(MSR, FP, MSR_FP, 1)
 FIELD(MSR, ME, MSR_ME, 1)
+FIELD(MSR, IR, MSR_IR, 1)
 FIELD(MSR, DS, MSR_DS, 1)
 FIELD(MSR, LE, MSR_LE, 1)
 
@@ -485,7 +486,6 @@ FIELD(MSR, LE, MSR_LE, 1)
 #define msr_de   ((env->msr >> MSR_DE)   & 1)
 #define msr_fe1  ((env->msr >> MSR_FE1)  & 1)
 #define msr_ep   ((env->msr >> MSR_EP)   & 1)
-#define msr_ir   ((env->msr >> MSR_IR)   & 1)
 #define msr_dr   ((env->msr >> MSR_DR)   & 1)
 #define msr_ts   ((env->msr >> MSR_TS1)  & 3)
 
diff --git a/target/ppc/helper_regs.c b/target/ppc/helper_regs.c
index 4e649d8b0e..e40078c001 100644
--- a/target/ppc/helper_regs.c
+++ b/target/ppc/helper_regs.c
@@ -227,7 +227,7 @@ int hreg_store_msr(CPUPPCState *env, target_ulong value, int alter_hv)
         value &= ~MSR_HVB;
         value |= env->msr & MSR_HVB;
     }
-    if (((value >> MSR_IR) & 1) != msr_ir ||
+    if (((value ^ env->msr) & R_MSR_IR_MASK) ||
         ((value >> MSR_DR) & 1) != msr_dr) {
         cpu_interrupt_exittb(cs);
     }
diff --git a/target/ppc/mmu_common.c b/target/ppc/mmu_common.c
index 031bb4493b..30deca0425 100644
--- a/target/ppc/mmu_common.c
+++ b/target/ppc/mmu_common.c
@@ -388,7 +388,7 @@ static int get_segment_6xx_tlb(CPUPPCState *env, mmu_ctx_t *ctx,
                   " nip=" TARGET_FMT_lx " lr=" TARGET_FMT_lx
                   " ir=%d dr=%d pr=%d %d t=%d\n",
                   eaddr, (int)(eaddr >> 28), sr, env->nip, env->lr,
-                  (int)msr_ir, (int)msr_dr, pr ? 1 : 0,
+                  (int)FIELD_EX64(env->msr, MSR, IR), (int)msr_dr, pr ? 1 : 0,
                   access_type == MMU_DATA_STORE, type);
     pgidx = (eaddr & ~SEGMENT_MASK_256M) >> target_page_bits;
     hash = vsid ^ pgidx;
@@ -626,7 +626,8 @@ found_tlb:
     }
 
     /* Check the address space */
-    if ((access_type == MMU_INST_FETCH ? msr_ir : msr_dr) != (tlb->attr & 1)) {
+    if ((access_type == MMU_INST_FETCH ?
+        FIELD_EX64(env->msr, MSR, IR) : msr_dr) != (tlb->attr & 1)) {
         qemu_log_mask(CPU_LOG_MMU, "%s: AS doesn't match\n", __func__);
         return -1;
     }
@@ -839,7 +840,7 @@ found_tlb:
     if (access_type == MMU_INST_FETCH) {
         /* There is no way to fetch code using epid load */
         assert(!use_epid);
-        as = msr_ir;
+        as = FIELD_EX64(env->msr, MSR, IR);
     }
 
     if (as != ((tlb->mas1 & MAS1_TS) >> MAS1_TS_SHIFT)) {
@@ -1169,7 +1170,7 @@ int get_physical_address_wtlb(CPUPPCState *env, mmu_ctx_t *ctx,
                                      int mmu_idx)
 {
     int ret = -1;
-    bool real_mode = (type == ACCESS_CODE && msr_ir == 0)
+    bool real_mode = (type == ACCESS_CODE && !FIELD_EX64(env->msr, MSR, IR))
         || (type != ACCESS_CODE && msr_dr == 0);
 
     switch (env->mmu_model) {
@@ -1231,7 +1232,7 @@ static void booke206_update_mas_tlb_miss(CPUPPCState *env, target_ulong address,
     bool use_epid = mmubooke206_get_as(env, mmu_idx, &epid, &as, &pr);
 
     if (access_type == MMU_INST_FETCH) {
-        as = msr_ir;
+        as = FIELD_EX64(env->msr, MSR, IR);
     }
     env->spr[SPR_BOOKE_MAS0] = env->spr[SPR_BOOKE_MAS4] & MAS4_TLBSELD_MASK;
     env->spr[SPR_BOOKE_MAS1] = env->spr[SPR_BOOKE_MAS4] & MAS4_TSIZED_MASK;

From e4eea6ef667a9bf75114ad860d0e6b239b74b537 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:34 -0300
Subject: [PATCH 060/935] target/ppc: Remove msr_dr macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_dr macro hides the usage of env->msr, which is a bad behavior
Substitute it with FIELD_EX64 calls that explicitly use env->msr
as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-16-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h         |  2 +-
 target/ppc/helper_regs.c |  3 +--
 target/ppc/mmu_common.c  | 10 ++++++----
 3 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 6cfbec26a1..cd672dec93 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -364,6 +364,7 @@ FIELD(MSR, PR, MSR_PR, 1)
 FIELD(MSR, FP, MSR_FP, 1)
 FIELD(MSR, ME, MSR_ME, 1)
 FIELD(MSR, IR, MSR_IR, 1)
+FIELD(MSR, DR, MSR_DR, 1)
 FIELD(MSR, DS, MSR_DS, 1)
 FIELD(MSR, LE, MSR_LE, 1)
 
@@ -486,7 +487,6 @@ FIELD(MSR, LE, MSR_LE, 1)
 #define msr_de   ((env->msr >> MSR_DE)   & 1)
 #define msr_fe1  ((env->msr >> MSR_FE1)  & 1)
 #define msr_ep   ((env->msr >> MSR_EP)   & 1)
-#define msr_dr   ((env->msr >> MSR_DR)   & 1)
 #define msr_ts   ((env->msr >> MSR_TS1)  & 3)
 
 #define DBCR0_ICMP (1 << 27)
diff --git a/target/ppc/helper_regs.c b/target/ppc/helper_regs.c
index e40078c001..b150b78182 100644
--- a/target/ppc/helper_regs.c
+++ b/target/ppc/helper_regs.c
@@ -227,8 +227,7 @@ int hreg_store_msr(CPUPPCState *env, target_ulong value, int alter_hv)
         value &= ~MSR_HVB;
         value |= env->msr & MSR_HVB;
     }
-    if (((value ^ env->msr) & R_MSR_IR_MASK) ||
-        ((value >> MSR_DR) & 1) != msr_dr) {
+    if ((value ^ env->msr) & (R_MSR_IR_MASK | R_MSR_DR_MASK)) {
         cpu_interrupt_exittb(cs);
     }
     if ((env->mmu_model == POWERPC_MMU_BOOKE ||
diff --git a/target/ppc/mmu_common.c b/target/ppc/mmu_common.c
index 30deca0425..89107a6af2 100644
--- a/target/ppc/mmu_common.c
+++ b/target/ppc/mmu_common.c
@@ -388,7 +388,8 @@ static int get_segment_6xx_tlb(CPUPPCState *env, mmu_ctx_t *ctx,
                   " nip=" TARGET_FMT_lx " lr=" TARGET_FMT_lx
                   " ir=%d dr=%d pr=%d %d t=%d\n",
                   eaddr, (int)(eaddr >> 28), sr, env->nip, env->lr,
-                  (int)FIELD_EX64(env->msr, MSR, IR), (int)msr_dr, pr ? 1 : 0,
+                  (int)FIELD_EX64(env->msr, MSR, IR),
+                  (int)FIELD_EX64(env->msr, MSR, DR), pr ? 1 : 0,
                   access_type == MMU_DATA_STORE, type);
     pgidx = (eaddr & ~SEGMENT_MASK_256M) >> target_page_bits;
     hash = vsid ^ pgidx;
@@ -627,7 +628,8 @@ found_tlb:
 
     /* Check the address space */
     if ((access_type == MMU_INST_FETCH ?
-        FIELD_EX64(env->msr, MSR, IR) : msr_dr) != (tlb->attr & 1)) {
+        FIELD_EX64(env->msr, MSR, IR) :
+        FIELD_EX64(env->msr, MSR, DR)) != (tlb->attr & 1)) {
         qemu_log_mask(CPU_LOG_MMU, "%s: AS doesn't match\n", __func__);
         return -1;
     }
@@ -1170,8 +1172,8 @@ int get_physical_address_wtlb(CPUPPCState *env, mmu_ctx_t *ctx,
                                      int mmu_idx)
 {
     int ret = -1;
-    bool real_mode = (type == ACCESS_CODE && !FIELD_EX64(env->msr, MSR, IR))
-        || (type != ACCESS_CODE && msr_dr == 0);
+    bool real_mode = (type == ACCESS_CODE && !FIELD_EX64(env->msr, MSR, IR)) ||
+                     (type != ACCESS_CODE && !FIELD_EX64(env->msr, MSR, DR));
 
     switch (env->mmu_model) {
     case POWERPC_MMU_SOFT_6xx:

From 502423309159a4305f315e55ff6ec75aa61a91b0 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:35 -0300
Subject: [PATCH 061/935] target/ppc: Remove msr_ep macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_ep macro hides the usage of env->msr, which is a bad behavior
Substitute it with FIELD_EX64 calls that explicitly use env->msr
as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-17-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h         | 2 +-
 target/ppc/helper_regs.c | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index cd672dec93..5e804f0373 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -363,6 +363,7 @@ FIELD(MSR, EE, MSR_EE, 1)
 FIELD(MSR, PR, MSR_PR, 1)
 FIELD(MSR, FP, MSR_FP, 1)
 FIELD(MSR, ME, MSR_ME, 1)
+FIELD(MSR, EP, MSR_EP, 1)
 FIELD(MSR, IR, MSR_IR, 1)
 FIELD(MSR, DR, MSR_DR, 1)
 FIELD(MSR, DS, MSR_DS, 1)
@@ -486,7 +487,6 @@ FIELD(MSR, LE, MSR_LE, 1)
 #define msr_fe0  ((env->msr >> MSR_FE0)  & 1)
 #define msr_de   ((env->msr >> MSR_DE)   & 1)
 #define msr_fe1  ((env->msr >> MSR_FE1)  & 1)
-#define msr_ep   ((env->msr >> MSR_EP)   & 1)
 #define msr_ts   ((env->msr >> MSR_TS1)  & 3)
 
 #define DBCR0_ICMP (1 << 27)
diff --git a/target/ppc/helper_regs.c b/target/ppc/helper_regs.c
index b150b78182..97cd263131 100644
--- a/target/ppc/helper_regs.c
+++ b/target/ppc/helper_regs.c
@@ -240,8 +240,8 @@ int hreg_store_msr(CPUPPCState *env, target_ulong value, int alter_hv)
         /* Swap temporary saved registers with GPRs */
         hreg_swap_gpr_tgpr(env);
     }
-    if (unlikely((value >> MSR_EP) & 1) != msr_ep) {
-        env->excp_prefix = ((value >> MSR_EP) & 1) * 0xFFF00000;
+    if (unlikely((value ^ env->msr) & R_MSR_EP_MASK)) {
+        env->excp_prefix = FIELD_EX64(value, MSR, EP) * 0xFFF00000;
     }
     /*
      * If PR=1 then EE, IR and DR must be 1

From da806a6c63c8a62700fcb7bc09ba908f0f5e5648 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:36 -0300
Subject: [PATCH 062/935] target/ppc: Remove msr_fe0 and msr_fe1 macros
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_fe0 and msr_fe1 macros hide the usage of env->msr, which is a bad
behavior. Substitute it with FIELD_EX64 calls that explicitly use
env->msr as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-18-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h         | 11 +++++++++--
 target/ppc/excp_helper.c | 18 ++++++------------
 2 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 5e804f0373..74a3c01f99 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -363,12 +363,21 @@ FIELD(MSR, EE, MSR_EE, 1)
 FIELD(MSR, PR, MSR_PR, 1)
 FIELD(MSR, FP, MSR_FP, 1)
 FIELD(MSR, ME, MSR_ME, 1)
+FIELD(MSR, FE0, MSR_FE0, 1)
+FIELD(MSR, FE1, MSR_FE1, 1)
 FIELD(MSR, EP, MSR_EP, 1)
 FIELD(MSR, IR, MSR_IR, 1)
 FIELD(MSR, DR, MSR_DR, 1)
 FIELD(MSR, DS, MSR_DS, 1)
 FIELD(MSR, LE, MSR_LE, 1)
 
+/*
+ * FE0 and FE1 bits are not side-by-side
+ * so we can't combine them using FIELD()
+ */
+#define FIELD_EX64_FE(msr) \
+    ((FIELD_EX64(msr, MSR, FE0) << 1) | FIELD_EX64(msr, MSR, FE1))
+
 /* PMU bits */
 #define MMCR0_FC     PPC_BIT(32)         /* Freeze Counters  */
 #define MMCR0_PMAO   PPC_BIT(56)         /* Perf Monitor Alert Ocurred */
@@ -484,9 +493,7 @@ FIELD(MSR, LE, MSR_LE, 1)
 #else
 #define msr_hv   (0)
 #endif
-#define msr_fe0  ((env->msr >> MSR_FE0)  & 1)
 #define msr_de   ((env->msr >> MSR_DE)   & 1)
-#define msr_fe1  ((env->msr >> MSR_FE1)  & 1)
 #define msr_ts   ((env->msr >> MSR_TS1)  & 3)
 
 #define DBCR0_ICMP (1 << 27)
diff --git a/target/ppc/excp_helper.c b/target/ppc/excp_helper.c
index 30baad0489..aa201c63c6 100644
--- a/target/ppc/excp_helper.c
+++ b/target/ppc/excp_helper.c
@@ -478,8 +478,7 @@ static void powerpc_excp_40x(PowerPCCPU *cpu, int excp)
     case POWERPC_EXCP_PROGRAM:   /* Program exception                        */
         switch (env->error_code & ~0xF) {
         case POWERPC_EXCP_FP:
-            if ((msr_fe0 == 0 && msr_fe1 == 0) ||
-                !FIELD_EX64(env->msr, MSR, FP)) {
+            if (!FIELD_EX64_FE(env->msr) || !FIELD_EX64(env->msr, MSR, FP)) {
                 trace_ppc_excp_fp_ignore();
                 powerpc_reset_excp_state(cpu);
                 return;
@@ -616,8 +615,7 @@ static void powerpc_excp_6xx(PowerPCCPU *cpu, int excp)
     case POWERPC_EXCP_PROGRAM:   /* Program exception                        */
         switch (env->error_code & ~0xF) {
         case POWERPC_EXCP_FP:
-            if ((msr_fe0 == 0 && msr_fe1 == 0) ||
-                !FIELD_EX64(env->msr, MSR, FP)) {
+            if (!FIELD_EX64_FE(env->msr) || !FIELD_EX64(env->msr, MSR, FP)) {
                 trace_ppc_excp_fp_ignore();
                 powerpc_reset_excp_state(cpu);
                 return;
@@ -790,8 +788,7 @@ static void powerpc_excp_7xx(PowerPCCPU *cpu, int excp)
     case POWERPC_EXCP_PROGRAM:   /* Program exception                        */
         switch (env->error_code & ~0xF) {
         case POWERPC_EXCP_FP:
-            if ((msr_fe0 == 0 && msr_fe1 == 0) ||
-                !FIELD_EX64(env->msr, MSR, FP)) {
+            if (!FIELD_EX64_FE(env->msr) || !FIELD_EX64(env->msr, MSR, FP)) {
                 trace_ppc_excp_fp_ignore();
                 powerpc_reset_excp_state(cpu);
                 return;
@@ -976,8 +973,7 @@ static void powerpc_excp_74xx(PowerPCCPU *cpu, int excp)
     case POWERPC_EXCP_PROGRAM:   /* Program exception                        */
         switch (env->error_code & ~0xF) {
         case POWERPC_EXCP_FP:
-            if ((msr_fe0 == 0 && msr_fe1 == 0) ||
-                !FIELD_EX64(env->msr, MSR, FP)) {
+            if (!FIELD_EX64_FE(env->msr) || !FIELD_EX64(env->msr, MSR, FP)) {
                 trace_ppc_excp_fp_ignore();
                 powerpc_reset_excp_state(cpu);
                 return;
@@ -1175,8 +1171,7 @@ static void powerpc_excp_booke(PowerPCCPU *cpu, int excp)
     case POWERPC_EXCP_PROGRAM:   /* Program exception                        */
         switch (env->error_code & ~0xF) {
         case POWERPC_EXCP_FP:
-            if ((msr_fe0 == 0 && msr_fe1 == 0) ||
-                !FIELD_EX64(env->msr, MSR, FP)) {
+            if (!FIELD_EX64_FE(env->msr) || !FIELD_EX64(env->msr, MSR, FP)) {
                 trace_ppc_excp_fp_ignore();
                 powerpc_reset_excp_state(cpu);
                 return;
@@ -1439,8 +1434,7 @@ static void powerpc_excp_books(PowerPCCPU *cpu, int excp)
     case POWERPC_EXCP_PROGRAM:   /* Program exception                        */
         switch (env->error_code & ~0xF) {
         case POWERPC_EXCP_FP:
-            if ((msr_fe0 == 0 && msr_fe1 == 0) ||
-                !FIELD_EX64(env->msr, MSR, FP)) {
+            if (!FIELD_EX64_FE(env->msr) || !FIELD_EX64(env->msr, MSR, FP)) {
                 trace_ppc_excp_fp_ignore();
                 powerpc_reset_excp_state(cpu);
                 return;

From ca241959cd824e1feeabed4948e2c754d1e2b635 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:37 -0300
Subject: [PATCH 063/935] target/ppc: Remove msr_ts macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_ts macro hides the usage of env->msr, which is a bad
behavior. Substitute it with FIELD_EX64 calls that explicitly use
env->msr as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-19-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h     | 2 +-
 target/ppc/kvm.c     | 4 ++--
 target/ppc/machine.c | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 74a3c01f99..5ac7d7d68f 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -354,6 +354,7 @@ typedef enum {
 #define MSR_RI   1  /* Recoverable interrupt                        1        */
 #define MSR_LE   0  /* Little-endian mode                           1 hflags */
 
+FIELD(MSR, TS, MSR_TS0, 2)
 FIELD(MSR, CM, MSR_CM, 1)
 FIELD(MSR, GS, MSR_GS, 1)
 FIELD(MSR, POW, MSR_POW, 1)
@@ -494,7 +495,6 @@ FIELD(MSR, LE, MSR_LE, 1)
 #define msr_hv   (0)
 #endif
 #define msr_de   ((env->msr >> MSR_DE)   & 1)
-#define msr_ts   ((env->msr >> MSR_TS1)  & 3)
 
 #define DBCR0_ICMP (1 << 27)
 #define DBCR0_BRT (1 << 26)
diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
index db3a92869c..6eed466f80 100644
--- a/target/ppc/kvm.c
+++ b/target/ppc/kvm.c
@@ -974,7 +974,7 @@ int kvm_arch_put_registers(CPUState *cs, int level)
         }
 
 #ifdef TARGET_PPC64
-        if (msr_ts) {
+        if (FIELD_EX64(env->msr, MSR, TS)) {
             for (i = 0; i < ARRAY_SIZE(env->tm_gpr); i++) {
                 kvm_set_one_reg(cs, KVM_REG_PPC_TM_GPR(i), &env->tm_gpr[i]);
             }
@@ -1282,7 +1282,7 @@ int kvm_arch_get_registers(CPUState *cs)
         }
 
 #ifdef TARGET_PPC64
-        if (msr_ts) {
+        if (FIELD_EX64(env->msr, MSR, TS)) {
             for (i = 0; i < ARRAY_SIZE(env->tm_gpr); i++) {
                 kvm_get_one_reg(cs, KVM_REG_PPC_TM_GPR(i), &env->tm_gpr[i]);
             }
diff --git a/target/ppc/machine.c b/target/ppc/machine.c
index e673944597..7104a5c67e 100644
--- a/target/ppc/machine.c
+++ b/target/ppc/machine.c
@@ -417,7 +417,7 @@ static bool tm_needed(void *opaque)
 {
     PowerPCCPU *cpu = opaque;
     CPUPPCState *env = &cpu->env;
-    return msr_ts;
+    return FIELD_EX64(env->msr, MSR, TS);
 }
 
 static const VMStateDescription vmstate_tm = {

From 9de754d30d54b9d5d2ba0756bcbd63625c82e63c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:38 -0300
Subject: [PATCH 064/935] target/ppc: Remove msr_hv macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_hv macro hides the usage of env->msr, which is a bad
behavior. Substitute it with FIELD_EX64 calls that explicitly use
env->msr as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-20-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h         | 11 ++++++-----
 target/ppc/cpu_init.c    |  6 ++++--
 target/ppc/excp_helper.c |  8 ++++----
 target/ppc/mem_helper.c  |  4 ++--
 target/ppc/misc_helper.c |  2 +-
 target/ppc/mmu-radix64.c |  6 +++---
 6 files changed, 20 insertions(+), 17 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 5ac7d7d68f..9f19b3c0a8 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -354,6 +354,12 @@ typedef enum {
 #define MSR_RI   1  /* Recoverable interrupt                        1        */
 #define MSR_LE   0  /* Little-endian mode                           1 hflags */
 
+#if defined(TARGET_PPC64)
+FIELD(MSR, HV, MSR_HV, 1)
+#define FIELD_EX64_HV(storage) FIELD_EX64(storage, MSR, HV)
+#else
+#define FIELD_EX64_HV(storage) 0
+#endif
 FIELD(MSR, TS, MSR_TS0, 2)
 FIELD(MSR, CM, MSR_CM, 1)
 FIELD(MSR, GS, MSR_GS, 1)
@@ -489,11 +495,6 @@ FIELD(MSR, LE, MSR_LE, 1)
 #define HFSCR_MSGP     PPC_BIT(53) /* Privileged Message Send Facilities */
 #define HFSCR_IC_MSGP  0xA
 
-#if defined(TARGET_PPC64)
-#define msr_hv   ((env->msr >> MSR_HV)   & 1)
-#else
-#define msr_hv   (0)
-#endif
 #define msr_de   ((env->msr >> MSR_DE)   & 1)
 
 #define DBCR0_ICMP (1 << 27)
diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
index 10e7c41bc9..d4c7813de5 100644
--- a/target/ppc/cpu_init.c
+++ b/target/ppc/cpu_init.c
@@ -6305,7 +6305,8 @@ static bool cpu_has_work_POWER9(CPUState *cs)
         if ((env->pending_interrupts & (1u << PPC_INTERRUPT_EXT)) &&
             (env->spr[SPR_LPCR] & LPCR_EEE)) {
             bool heic = !!(env->spr[SPR_LPCR] & LPCR_HEIC);
-            if (!heic || !msr_hv || FIELD_EX64(env->msr, MSR, PR)) {
+            if (!heic || !FIELD_EX64_HV(env->msr) ||
+                FIELD_EX64(env->msr, MSR, PR)) {
                 return true;
             }
         }
@@ -6520,7 +6521,8 @@ static bool cpu_has_work_POWER10(CPUState *cs)
         if ((env->pending_interrupts & (1u << PPC_INTERRUPT_EXT)) &&
             (env->spr[SPR_LPCR] & LPCR_EEE)) {
             bool heic = !!(env->spr[SPR_LPCR] & LPCR_HEIC);
-            if (!heic || !msr_hv || FIELD_EX64(env->msr, MSR, PR)) {
+            if (!heic || !FIELD_EX64_HV(env->msr) ||
+                FIELD_EX64(env->msr, MSR, PR)) {
                 return true;
             }
         }
diff --git a/target/ppc/excp_helper.c b/target/ppc/excp_helper.c
index aa201c63c6..cb752b184a 100644
--- a/target/ppc/excp_helper.c
+++ b/target/ppc/excp_helper.c
@@ -1715,7 +1715,7 @@ static void ppc_hw_interrupt(CPUPPCState *env)
     if (env->pending_interrupts & (1 << PPC_INTERRUPT_HDECR)) {
         /* LPCR will be clear when not supported so this will work */
         bool hdice = !!(env->spr[SPR_LPCR] & LPCR_HDICE);
-        if ((async_deliver || msr_hv == 0) && hdice) {
+        if ((async_deliver || !FIELD_EX64_HV(env->msr)) && hdice) {
             /* HDEC clears on delivery */
             env->pending_interrupts &= ~(1 << PPC_INTERRUPT_HDECR);
             powerpc_excp(cpu, POWERPC_EXCP_HDECR);
@@ -1727,7 +1727,7 @@ static void ppc_hw_interrupt(CPUPPCState *env)
     if (env->pending_interrupts & (1 << PPC_INTERRUPT_HVIRT)) {
         /* LPCR will be clear when not supported so this will work */
         bool hvice = !!(env->spr[SPR_LPCR] & LPCR_HVICE);
-        if ((async_deliver || msr_hv == 0) && hvice) {
+        if ((async_deliver || !FIELD_EX64_HV(env->msr)) && hvice) {
             powerpc_excp(cpu, POWERPC_EXCP_HVIRT);
             return;
         }
@@ -1738,9 +1738,9 @@ static void ppc_hw_interrupt(CPUPPCState *env)
         bool lpes0 = !!(env->spr[SPR_LPCR] & LPCR_LPES0);
         bool heic = !!(env->spr[SPR_LPCR] & LPCR_HEIC);
         /* HEIC blocks delivery to the hypervisor */
-        if ((async_deliver && !(heic && msr_hv &&
+        if ((async_deliver && !(heic && FIELD_EX64_HV(env->msr) &&
             !FIELD_EX64(env->msr, MSR, PR))) ||
-            (env->has_hv_mode && msr_hv == 0 && !lpes0)) {
+            (env->has_hv_mode && !FIELD_EX64_HV(env->msr) && !lpes0)) {
             if (books_vhyp_promotes_external_to_hvirt(cpu)) {
                 powerpc_excp(cpu, POWERPC_EXCP_HVIRT);
             } else {
diff --git a/target/ppc/mem_helper.c b/target/ppc/mem_helper.c
index 9af135e88e..d1163f316c 100644
--- a/target/ppc/mem_helper.c
+++ b/target/ppc/mem_helper.c
@@ -612,11 +612,11 @@ void helper_tbegin(CPUPPCState *env)
     env->spr[SPR_TEXASR] =
         (1ULL << TEXASR_FAILURE_PERSISTENT) |
         (1ULL << TEXASR_NESTING_OVERFLOW) |
-        (msr_hv << TEXASR_PRIVILEGE_HV) |
+        (FIELD_EX64_HV(env->msr) << TEXASR_PRIVILEGE_HV) |
         (FIELD_EX64(env->msr, MSR, PR) << TEXASR_PRIVILEGE_PR) |
         (1ULL << TEXASR_FAILURE_SUMMARY) |
         (1ULL << TEXASR_TFIAR_EXACT);
-    env->spr[SPR_TFIAR] = env->nip | (msr_hv << 1) |
+    env->spr[SPR_TFIAR] = env->nip | (FIELD_EX64_HV(env->msr) << 1) |
                           FIELD_EX64(env->msr, MSR, PR);
     env->spr[SPR_TFHAR] = env->nip + 4;
     env->crf[0] = 0xB; /* 0b1010 = transaction failure */
diff --git a/target/ppc/misc_helper.c b/target/ppc/misc_helper.c
index 06aa716cab..b0a5e7ce76 100644
--- a/target/ppc/misc_helper.c
+++ b/target/ppc/misc_helper.c
@@ -73,7 +73,7 @@ void helper_hfscr_facility_check(CPUPPCState *env, uint32_t bit,
                                  const char *caller, uint32_t cause)
 {
 #ifdef TARGET_PPC64
-    if ((env->msr_mask & MSR_HVB) && !msr_hv &&
+    if ((env->msr_mask & MSR_HVB) && !FIELD_EX64(env->msr, MSR, HV) &&
                                      !(env->spr[SPR_HFSCR] & (1UL << bit))) {
         raise_hv_fu_exception(env, bit, caller, cause, GETPC());
     }
diff --git a/target/ppc/mmu-radix64.c b/target/ppc/mmu-radix64.c
index e88f51fd34..21ac958e48 100644
--- a/target/ppc/mmu-radix64.c
+++ b/target/ppc/mmu-radix64.c
@@ -37,7 +37,7 @@ static bool ppc_radix64_get_fully_qualified_addr(const CPUPPCState *env,
         return false;
     }
 
-    if (msr_hv) { /* MSR[HV] -> Hypervisor/bare metal */
+    if (FIELD_EX64(env->msr, MSR, HV)) { /* MSR[HV] -> Hypervisor/bare metal */
         switch (eaddr & R_EADDR_QUADRANT) {
         case R_EADDR_QUADRANT0:
             *lpid = 0;
@@ -306,7 +306,7 @@ static bool validate_pate(PowerPCCPU *cpu, uint64_t lpid, ppc_v3_pate_t *pate)
     if (!(pate->dw0 & PATE0_HR)) {
         return false;
     }
-    if (lpid == 0 && !msr_hv) {
+    if (lpid == 0 && !FIELD_EX64(env->msr, MSR, HV)) {
         return false;
     }
     if ((pate->dw0 & PATE1_R_PRTS) < 5) {
@@ -431,7 +431,7 @@ static int ppc_radix64_process_scoped_xlate(PowerPCCPU *cpu,
     *g_page_size = PRTBE_R_GET_RTS(prtbe0);
     base_addr = prtbe0 & PRTBE_R_RPDB;
     nls = prtbe0 & PRTBE_R_RPDS;
-    if (msr_hv || vhyp_flat_addressing(cpu)) {
+    if (FIELD_EX64(env->msr, MSR, HV) || vhyp_flat_addressing(cpu)) {
         /*
          * Can treat process table addresses as real addresses
          */

From 67935ecdd9a5d5dfed83cb0a7e7f6be925a7d761 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:39 -0300
Subject: [PATCH 065/935] target/ppc: Remove msr_de macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

msr_de macro hides the usage of env->msr, which is a bad
behavior. Substitute it with FIELD_EX64 calls that explicitly use
env->msr as a parameter.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-21-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h         | 3 +--
 target/ppc/helper_regs.c | 4 ++--
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 9f19b3c0a8..af249239d5 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -371,6 +371,7 @@ FIELD(MSR, PR, MSR_PR, 1)
 FIELD(MSR, FP, MSR_FP, 1)
 FIELD(MSR, ME, MSR_ME, 1)
 FIELD(MSR, FE0, MSR_FE0, 1)
+FIELD(MSR, DE, MSR_DE, 1)
 FIELD(MSR, FE1, MSR_FE1, 1)
 FIELD(MSR, EP, MSR_EP, 1)
 FIELD(MSR, IR, MSR_IR, 1)
@@ -495,8 +496,6 @@ FIELD(MSR, LE, MSR_LE, 1)
 #define HFSCR_MSGP     PPC_BIT(53) /* Privileged Message Send Facilities */
 #define HFSCR_IC_MSGP  0xA
 
-#define msr_de   ((env->msr >> MSR_DE)   & 1)
-
 #define DBCR0_ICMP (1 << 27)
 #define DBCR0_BRT (1 << 26)
 #define DBSR_ICMP (1 << 27)
diff --git a/target/ppc/helper_regs.c b/target/ppc/helper_regs.c
index 97cd263131..6159a15b7b 100644
--- a/target/ppc/helper_regs.c
+++ b/target/ppc/helper_regs.c
@@ -63,10 +63,10 @@ static uint32_t hreg_compute_hflags_value(CPUPPCState *env)
 
     if (ppc_flags & POWERPC_FLAG_DE) {
         target_ulong dbcr0 = env->spr[SPR_BOOKE_DBCR0];
-        if ((dbcr0 & DBCR0_ICMP) && msr_de) {
+        if ((dbcr0 & DBCR0_ICMP) && FIELD_EX64(env->msr, MSR, DE)) {
             hflags |= 1 << HFLAGS_SE;
         }
-        if ((dbcr0 & DBCR0_BRT) && msr_de) {
+        if ((dbcr0 & DBCR0_BRT) && FIELD_EX64(env->msr, MSR, DE)) {
             hflags |= 1 << HFLAGS_BE;
         }
     } else {

From 39af1384faec88e49e14dc066f7653f219d79850 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:40 -0300
Subject: [PATCH 066/935] target/ppc: Add unused msr bits FIELDs
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add FIELDs macros for msr bits that had an unused msr_* before.

Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-22-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index af249239d5..4577cfcc23 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -354,16 +354,31 @@ typedef enum {
 #define MSR_RI   1  /* Recoverable interrupt                        1        */
 #define MSR_LE   0  /* Little-endian mode                           1 hflags */
 
+FIELD(MSR, SF, MSR_SF, 1)
+FIELD(MSR, TAG, MSR_TAG, 1)
+FIELD(MSR, ISF, MSR_ISF, 1)
 #if defined(TARGET_PPC64)
 FIELD(MSR, HV, MSR_HV, 1)
 #define FIELD_EX64_HV(storage) FIELD_EX64(storage, MSR, HV)
 #else
 #define FIELD_EX64_HV(storage) 0
 #endif
+FIELD(MSR, TS0, MSR_TS0, 1)
+FIELD(MSR, TS1, MSR_TS1, 1)
 FIELD(MSR, TS, MSR_TS0, 2)
+FIELD(MSR, TM, MSR_TM, 1)
 FIELD(MSR, CM, MSR_CM, 1)
+FIELD(MSR, ICM, MSR_ICM, 1)
 FIELD(MSR, GS, MSR_GS, 1)
+FIELD(MSR, UCLE, MSR_UCLE, 1)
+FIELD(MSR, VR, MSR_VR, 1)
+FIELD(MSR, SPE, MSR_SPE, 1)
+FIELD(MSR, VSX, MSR_VSX, 1)
+FIELD(MSR, S, MSR_S, 1)
+FIELD(MSR, KEY, MSR_KEY, 1)
 FIELD(MSR, POW, MSR_POW, 1)
+FIELD(MSR, WE, MSR_WE, 1)
+FIELD(MSR, TGPR, MSR_TGPR, 1)
 FIELD(MSR, CE, MSR_CE, 1)
 FIELD(MSR, ILE, MSR_ILE, 1)
 FIELD(MSR, EE, MSR_EE, 1)
@@ -371,12 +386,22 @@ FIELD(MSR, PR, MSR_PR, 1)
 FIELD(MSR, FP, MSR_FP, 1)
 FIELD(MSR, ME, MSR_ME, 1)
 FIELD(MSR, FE0, MSR_FE0, 1)
+FIELD(MSR, SE, MSR_SE, 1)
+FIELD(MSR, DWE, MSR_DWE, 1)
+FIELD(MSR, UBLE, MSR_UBLE, 1)
+FIELD(MSR, BE, MSR_BE, 1)
 FIELD(MSR, DE, MSR_DE, 1)
 FIELD(MSR, FE1, MSR_FE1, 1)
+FIELD(MSR, AL, MSR_AL, 1)
 FIELD(MSR, EP, MSR_EP, 1)
 FIELD(MSR, IR, MSR_IR, 1)
 FIELD(MSR, DR, MSR_DR, 1)
+FIELD(MSR, IS, MSR_IS, 1)
 FIELD(MSR, DS, MSR_DS, 1)
+FIELD(MSR, PE, MSR_PE, 1)
+FIELD(MSR, PX, MSR_PX, 1)
+FIELD(MSR, PMM, MSR_PMM, 1)
+FIELD(MSR, RI, MSR_RI, 1)
 FIELD(MSR, LE, MSR_LE, 1)
 
 /*

From bf3dd1e6d0d7c5c4906f89776e15dddc22af784b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Wed, 4 May 2022 18:05:41 -0300
Subject: [PATCH 067/935] target/ppc: Change MSR_* to follow POWER ISA
 numbering convention
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Today we have the issue where MSR_* values are the 'inverted order'
bit numbers from what the ISA specifies. e.g. MSR_LE is bit 63 but
is defined as 0 in QEMU.

Add a macro to be used to convert from QEMU order to ISA order.

This solution requires less changes than to use the already defined
PPC_BIT macro, which would turn MSR_* in masks instead of the numbers
itself.

Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504210541.115256-23-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h | 87 ++++++++++++++++++++++++------------------------
 1 file changed, 44 insertions(+), 43 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 4577cfcc23..48596cfb25 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -38,6 +38,7 @@
 #define PPC_ELF_MACHINE     EM_PPC
 #endif
 
+#define PPC_BIT_NR(bit)         (63 - (bit))
 #define PPC_BIT(bit)            (0x8000000000000000ULL >> (bit))
 #define PPC_BIT32(bit)          (0x80000000 >> (bit))
 #define PPC_BIT8(bit)           (0x80 >> (bit))
@@ -310,49 +311,49 @@ typedef enum {
 
 /*****************************************************************************/
 /* Machine state register bits definition                                    */
-#define MSR_SF   63 /* Sixty-four-bit mode                            hflags */
-#define MSR_TAG  62 /* Tag-active mode (POWERx ?)                            */
-#define MSR_ISF  61 /* Sixty-four-bit interrupt mode on 630                  */
-#define MSR_HV   60 /* hypervisor state                               hflags */
-#define MSR_TS0  34 /* Transactional state, 2 bits (Book3s)                  */
-#define MSR_TS1  33
-#define MSR_TM   32 /* Transactional Memory Available (Book3s)               */
-#define MSR_CM   31 /* Computation mode for BookE                     hflags */
-#define MSR_ICM  30 /* Interrupt computation mode for BookE                  */
-#define MSR_GS   28 /* guest state for BookE                                 */
-#define MSR_UCLE 26 /* User-mode cache lock enable for BookE                 */
-#define MSR_VR   25 /* altivec available                            x hflags */
-#define MSR_SPE  25 /* SPE enable for BookE                         x hflags */
-#define MSR_VSX  23 /* Vector Scalar Extension (ISA 2.06 and later) x hflags */
-#define MSR_S    22 /* Secure state                                          */
-#define MSR_KEY  19 /* key bit on 603e                                       */
-#define MSR_POW  18 /* Power management                                      */
-#define MSR_WE   18 /* Wait State Enable on 405                              */
-#define MSR_TGPR 17 /* TGPR usage on 602/603                        x        */
-#define MSR_CE   17 /* Critical interrupt enable on embedded PowerPC x       */
-#define MSR_ILE  16 /* Interrupt little-endian mode                          */
-#define MSR_EE   15 /* External interrupt enable                             */
-#define MSR_PR   14 /* Problem state                                  hflags */
-#define MSR_FP   13 /* Floating point available                       hflags */
-#define MSR_ME   12 /* Machine check interrupt enable                        */
-#define MSR_FE0  11 /* Floating point exception mode 0                       */
-#define MSR_SE   10 /* Single-step trace enable                     x hflags */
-#define MSR_DWE  10 /* Debug wait enable on 405                     x        */
-#define MSR_UBLE 10 /* User BTB lock enable on e500                 x        */
-#define MSR_BE   9  /* Branch trace enable                          x hflags */
-#define MSR_DE   9  /* Debug interrupts enable on embedded PowerPC  x        */
-#define MSR_FE1  8  /* Floating point exception mode 1                       */
-#define MSR_AL   7  /* AL bit on POWER                                       */
-#define MSR_EP   6  /* Exception prefix on 601                               */
-#define MSR_IR   5  /* Instruction relocate                                  */
-#define MSR_DR   4  /* Data relocate                                         */
-#define MSR_IS   5  /* Instruction address space (BookE)                     */
-#define MSR_DS   4  /* Data address space (BookE)                            */
-#define MSR_PE   3  /* Protection enable on 403                              */
-#define MSR_PX   2  /* Protection exclusive on 403                  x        */
-#define MSR_PMM  2  /* Performance monitor mark on POWER            x        */
-#define MSR_RI   1  /* Recoverable interrupt                        1        */
-#define MSR_LE   0  /* Little-endian mode                           1 hflags */
+#define MSR_SF   PPC_BIT_NR(0)  /* Sixty-four-bit mode                hflags */
+#define MSR_TAG  PPC_BIT_NR(1)  /* Tag-active mode (POWERx ?)                */
+#define MSR_ISF  PPC_BIT_NR(2)  /* Sixty-four-bit interrupt mode on 630      */
+#define MSR_HV   PPC_BIT_NR(3)  /* hypervisor state                   hflags */
+#define MSR_TS0  PPC_BIT_NR(29) /* Transactional state, 2 bits (Book3s)      */
+#define MSR_TS1  PPC_BIT_NR(30)
+#define MSR_TM   PPC_BIT_NR(31) /* Transactional Memory Available (Book3s)   */
+#define MSR_CM   PPC_BIT_NR(32) /* Computation mode for BookE         hflags */
+#define MSR_ICM  PPC_BIT_NR(33) /* Interrupt computation mode for BookE      */
+#define MSR_GS   PPC_BIT_NR(35) /* guest state for BookE                     */
+#define MSR_UCLE PPC_BIT_NR(37) /* User-mode cache lock enable for BookE     */
+#define MSR_VR   PPC_BIT_NR(38) /* altivec available                x hflags */
+#define MSR_SPE  PPC_BIT_NR(38) /* SPE enable for BookE             x hflags */
+#define MSR_VSX  PPC_BIT_NR(40) /* Vector Scalar Extension (>= 2.06)x hflags */
+#define MSR_S    PPC_BIT_NR(41) /* Secure state                              */
+#define MSR_KEY  PPC_BIT_NR(44) /* key bit on 603e                           */
+#define MSR_POW  PPC_BIT_NR(45) /* Power management                          */
+#define MSR_WE   PPC_BIT_NR(45) /* Wait State Enable on 405                  */
+#define MSR_TGPR PPC_BIT_NR(46) /* TGPR usage on 602/603            x        */
+#define MSR_CE   PPC_BIT_NR(46) /* Critical int. enable on embedded PPC x    */
+#define MSR_ILE  PPC_BIT_NR(47) /* Interrupt little-endian mode              */
+#define MSR_EE   PPC_BIT_NR(48) /* External interrupt enable                 */
+#define MSR_PR   PPC_BIT_NR(49) /* Problem state                      hflags */
+#define MSR_FP   PPC_BIT_NR(50) /* Floating point available           hflags */
+#define MSR_ME   PPC_BIT_NR(51) /* Machine check interrupt enable            */
+#define MSR_FE0  PPC_BIT_NR(52) /* Floating point exception mode 0           */
+#define MSR_SE   PPC_BIT_NR(53) /* Single-step trace enable         x hflags */
+#define MSR_DWE  PPC_BIT_NR(53) /* Debug wait enable on 405         x        */
+#define MSR_UBLE PPC_BIT_NR(53) /* User BTB lock enable on e500     x        */
+#define MSR_BE   PPC_BIT_NR(54) /* Branch trace enable              x hflags */
+#define MSR_DE   PPC_BIT_NR(54) /* Debug int. enable on embedded PPC   x     */
+#define MSR_FE1  PPC_BIT_NR(55) /* Floating point exception mode 1           */
+#define MSR_AL   PPC_BIT_NR(56) /* AL bit on POWER                           */
+#define MSR_EP   PPC_BIT_NR(57) /* Exception prefix on 601                   */
+#define MSR_IR   PPC_BIT_NR(58) /* Instruction relocate                      */
+#define MSR_IS   PPC_BIT_NR(58) /* Instruction address space (BookE)         */
+#define MSR_DR   PPC_BIT_NR(59) /* Data relocate                             */
+#define MSR_DS   PPC_BIT_NR(59) /* Data address space (BookE)                */
+#define MSR_PE   PPC_BIT_NR(60) /* Protection enable on 403                  */
+#define MSR_PX   PPC_BIT_NR(61) /* Protection exclusive on 403        x      */
+#define MSR_PMM  PPC_BIT_NR(61) /* Performance monitor mark on POWER  x      */
+#define MSR_RI   PPC_BIT_NR(62) /* Recoverable interrupt            1        */
+#define MSR_LE   PPC_BIT_NR(63) /* Little-endian mode               1 hflags */
 
 FIELD(MSR, SF, MSR_SF, 1)
 FIELD(MSR, TAG, MSR_TAG, 1)

From a6f5770fb2b22cb219a684469c8224ebec74ca2e Mon Sep 17 00:00:00 2001
From: "Longpeng(Mike)" <longpeng2@huawei.com>
Date: Sat, 26 Mar 2022 14:02:22 +0800
Subject: [PATCH 068/935] vfio: simplify the conditional statements in
 vfio_msi_enable

It's unnecessary to test against the specific return value of
VFIO_DEVICE_SET_IRQS, since any positive return is an error
indicating the number of vectors we should retry with.

Signed-off-by: Longpeng(Mike) <longpeng2@huawei.com>
Link: https://lore.kernel.org/r/20220326060226.1892-2-longpeng2@huawei.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/pci.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index 9fd9faee1d..cab1a6ef57 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -653,7 +653,7 @@ retry:
     if (ret) {
         if (ret < 0) {
             error_report("vfio: Error: Failed to setup MSI fds: %m");
-        } else if (ret != vdev->nr_vectors) {
+        } else {
             error_report("vfio: Error: Failed to enable %d "
                          "MSI vectors, retry with %d", vdev->nr_vectors, ret);
         }
@@ -671,7 +671,7 @@ retry:
         g_free(vdev->msi_vectors);
         vdev->msi_vectors = NULL;
 
-        if (ret > 0 && ret != vdev->nr_vectors) {
+        if (ret > 0) {
             vdev->nr_vectors = ret;
             goto retry;
         }

From be4a46eccfe97069fd62f967f6660407c9aa2a03 Mon Sep 17 00:00:00 2001
From: "Longpeng(Mike)" <longpeng2@huawei.com>
Date: Sat, 26 Mar 2022 14:02:23 +0800
Subject: [PATCH 069/935] vfio: move re-enabling INTX out of the common helper

Move re-enabling INTX out, and the callers should decide to
re-enable it or not.

Signed-off-by: Longpeng(Mike) <longpeng2@huawei.com>
Link: https://lore.kernel.org/r/20220326060226.1892-3-longpeng2@huawei.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/pci.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index cab1a6ef57..b3c27c22aa 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -693,7 +693,6 @@ retry:
 
 static void vfio_msi_disable_common(VFIOPCIDevice *vdev)
 {
-    Error *err = NULL;
     int i;
 
     for (i = 0; i < vdev->nr_vectors; i++) {
@@ -712,15 +711,11 @@ static void vfio_msi_disable_common(VFIOPCIDevice *vdev)
     vdev->msi_vectors = NULL;
     vdev->nr_vectors = 0;
     vdev->interrupt = VFIO_INT_NONE;
-
-    vfio_intx_enable(vdev, &err);
-    if (err) {
-        error_reportf_err(err, VFIO_MSG_PREFIX, vdev->vbasedev.name);
-    }
 }
 
 static void vfio_msix_disable(VFIOPCIDevice *vdev)
 {
+    Error *err = NULL;
     int i;
 
     msix_unset_vector_notifiers(&vdev->pdev);
@@ -741,6 +736,10 @@ static void vfio_msix_disable(VFIOPCIDevice *vdev)
     }
 
     vfio_msi_disable_common(vdev);
+    vfio_intx_enable(vdev, &err);
+    if (err) {
+        error_reportf_err(err, VFIO_MSG_PREFIX, vdev->vbasedev.name);
+    }
 
     memset(vdev->msix->pending, 0,
            BITS_TO_LONGS(vdev->msix->entries) * sizeof(unsigned long));
@@ -750,8 +749,14 @@ static void vfio_msix_disable(VFIOPCIDevice *vdev)
 
 static void vfio_msi_disable(VFIOPCIDevice *vdev)
 {
+    Error *err = NULL;
+
     vfio_disable_irqindex(&vdev->vbasedev, VFIO_PCI_MSI_IRQ_INDEX);
     vfio_msi_disable_common(vdev);
+    vfio_intx_enable(vdev, &err);
+    if (err) {
+        error_reportf_err(err, VFIO_MSG_PREFIX, vdev->vbasedev.name);
+    }
 
     trace_vfio_msi_disable(vdev->vbasedev.name);
 }

From 8ab217d5d34275ee471a3b085ec90728b8f06d80 Mon Sep 17 00:00:00 2001
From: "Longpeng(Mike)" <longpeng2@huawei.com>
Date: Sat, 26 Mar 2022 14:02:24 +0800
Subject: [PATCH 070/935] vfio: simplify the failure path in vfio_msi_enable

Use vfio_msi_disable_common to simplify the error handling
in vfio_msi_enable.

Signed-off-by: Longpeng(Mike) <longpeng2@huawei.com>
Link: https://lore.kernel.org/r/20220326060226.1892-4-longpeng2@huawei.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/pci.c | 16 ++--------------
 1 file changed, 2 insertions(+), 14 deletions(-)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index b3c27c22aa..50562629ea 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -47,6 +47,7 @@
 
 static void vfio_disable_interrupts(VFIOPCIDevice *vdev);
 static void vfio_mmap_set_enabled(VFIOPCIDevice *vdev, bool enabled);
+static void vfio_msi_disable_common(VFIOPCIDevice *vdev);
 
 /*
  * Disabling BAR mmaping can be slow, but toggling it around INTx can
@@ -658,24 +659,12 @@ retry:
                          "MSI vectors, retry with %d", vdev->nr_vectors, ret);
         }
 
-        for (i = 0; i < vdev->nr_vectors; i++) {
-            VFIOMSIVector *vector = &vdev->msi_vectors[i];
-            if (vector->virq >= 0) {
-                vfio_remove_kvm_msi_virq(vector);
-            }
-            qemu_set_fd_handler(event_notifier_get_fd(&vector->interrupt),
-                                NULL, NULL, NULL);
-            event_notifier_cleanup(&vector->interrupt);
-        }
-
-        g_free(vdev->msi_vectors);
-        vdev->msi_vectors = NULL;
+        vfio_msi_disable_common(vdev);
 
         if (ret > 0) {
             vdev->nr_vectors = ret;
             goto retry;
         }
-        vdev->nr_vectors = 0;
 
         /*
          * Failing to setup MSI doesn't really fall within any specification.
@@ -683,7 +672,6 @@ retry:
          * out to fall back to INTx for this device.
          */
         error_report("vfio: Error: Failed to enable MSI");
-        vdev->interrupt = VFIO_INT_NONE;
 
         return;
     }

From 75d546fc18023d36779c687b948128c1a4666a96 Mon Sep 17 00:00:00 2001
From: "Longpeng(Mike)" <longpeng2@huawei.com>
Date: Sat, 26 Mar 2022 14:02:25 +0800
Subject: [PATCH 071/935] Revert "vfio: Avoid disabling and enabling vectors
 repeatedly in VFIO migration"

Commit ecebe53fe993 ("vfio: Avoid disabling and enabling vectors
repeatedly in VFIO migration") avoids inefficiently disabling and
enabling vectors repeatedly and lets the unmasked vectors be enabled
one by one.

But we want to batch multiple routes and defer the commit, and only
commit once outside the loop of setting vector notifiers, so we
cannot enable the vectors one by one in the loop now.

Revert that commit and we will take another way in the next patch,
it can not only avoid disabling/enabling vectors repeatedly, but
also satisfy our requirement of defer to commit.

Signed-off-by: Longpeng(Mike) <longpeng2@huawei.com>
Link: https://lore.kernel.org/r/20220326060226.1892-5-longpeng2@huawei.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/pci.c | 20 +++-----------------
 1 file changed, 3 insertions(+), 17 deletions(-)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index 50562629ea..8bc36f081a 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -572,9 +572,6 @@ static void vfio_msix_vector_release(PCIDevice *pdev, unsigned int nr)
 
 static void vfio_msix_enable(VFIOPCIDevice *vdev)
 {
-    PCIDevice *pdev = &vdev->pdev;
-    unsigned int nr, max_vec = 0;
-
     vfio_disable_interrupts(vdev);
 
     vdev->msi_vectors = g_new0(VFIOMSIVector, vdev->msix->entries);
@@ -593,22 +590,11 @@ static void vfio_msix_enable(VFIOPCIDevice *vdev)
      * triggering to userspace, then immediately release the vector, leaving
      * the physical device with no vectors enabled, but MSI-X enabled, just
      * like the guest view.
-     * If there are already unmasked vectors (in migration resume phase and
-     * some guest startups) which will be enabled soon, we can allocate all
-     * of them here to avoid inefficiently disabling and enabling vectors
-     * repeatedly later.
      */
-    if (!pdev->msix_function_masked) {
-        for (nr = 0; nr < msix_nr_vectors_allocated(pdev); nr++) {
-            if (!msix_is_masked(pdev, nr)) {
-                max_vec = nr;
-            }
-        }
-    }
-    vfio_msix_vector_do_use(pdev, max_vec, NULL, NULL);
-    vfio_msix_vector_release(pdev, max_vec);
+    vfio_msix_vector_do_use(&vdev->pdev, 0, NULL, NULL);
+    vfio_msix_vector_release(&vdev->pdev, 0);
 
-    if (msix_set_vector_notifiers(pdev, vfio_msix_vector_use,
+    if (msix_set_vector_notifiers(&vdev->pdev, vfio_msix_vector_use,
                                   vfio_msix_vector_release, NULL)) {
         error_report("vfio: msix_set_vector_notifiers failed");
     }

From dc580d51f7dd3e02134957be2b771636ac59e868 Mon Sep 17 00:00:00 2001
From: "Longpeng(Mike)" <longpeng2@huawei.com>
Date: Sat, 26 Mar 2022 14:02:26 +0800
Subject: [PATCH 072/935] vfio: defer to commit kvm irq routing when enable
 msi/msix

In migration resume phase, all unmasked msix vectors need to be
setup when loading the VF state. However, the setup operation would
take longer if the VM has more VFs and each VF has more unmasked
vectors.

The hot spot is kvm_irqchip_commit_routes, it'll scan and update
all irqfds that are already assigned each invocation, so more
vectors means need more time to process them.

vfio_pci_load_config
  vfio_msix_enable
    msix_set_vector_notifiers
      for (vector = 0; vector < dev->msix_entries_nr; vector++) {
        vfio_msix_vector_do_use
          vfio_add_kvm_msi_virq
            kvm_irqchip_commit_routes <-- expensive
      }

We can reduce the cost by only committing once outside the loop.
The routes are cached in kvm_state, we commit them first and then
bind irqfd for each vector.

The test VM has 128 vcpus and 8 VF (each one has 65 vectors),
we measure the cost of the vfio_msix_enable for each VF, and
we can see 90+% costs can be reduce.

VF      Count of irqfds[*]  Original        With this patch

1st           65            8               2
2nd           130           15              2
3rd           195           22              2
4th           260           24              3
5th           325           36              2
6th           390           44              3
7th           455           51              3
8th           520           58              4
Total                       258ms           21ms

[*] Count of irqfds
How many irqfds that already assigned and need to process in this
round.

The optimization can be applied to msi type too.

Signed-off-by: Longpeng(Mike) <longpeng2@huawei.com>
Link: https://lore.kernel.org/r/20220326060226.1892-6-longpeng2@huawei.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/pci.c | 130 +++++++++++++++++++++++++++++++++++++-------------
 hw/vfio/pci.h |   2 +
 2 files changed, 99 insertions(+), 33 deletions(-)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index 8bc36f081a..ef9d7bf326 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -45,6 +45,9 @@
 
 #define TYPE_VFIO_PCI_NOHOTPLUG "vfio-pci-nohotplug"
 
+/* Protected by BQL */
+static KVMRouteChange vfio_route_change;
+
 static void vfio_disable_interrupts(VFIOPCIDevice *vdev);
 static void vfio_mmap_set_enabled(VFIOPCIDevice *vdev, bool enabled);
 static void vfio_msi_disable_common(VFIOPCIDevice *vdev);
@@ -413,33 +416,36 @@ static int vfio_enable_vectors(VFIOPCIDevice *vdev, bool msix)
 static void vfio_add_kvm_msi_virq(VFIOPCIDevice *vdev, VFIOMSIVector *vector,
                                   int vector_n, bool msix)
 {
-    KVMRouteChange c;
-    int virq;
-
     if ((msix && vdev->no_kvm_msix) || (!msix && vdev->no_kvm_msi)) {
         return;
     }
 
-    if (event_notifier_init(&vector->kvm_interrupt, 0)) {
+    vector->virq = kvm_irqchip_add_msi_route(&vfio_route_change,
+                                             vector_n, &vdev->pdev);
+}
+
+static void vfio_connect_kvm_msi_virq(VFIOMSIVector *vector)
+{
+    if (vector->virq < 0) {
         return;
     }
 
-    c = kvm_irqchip_begin_route_changes(kvm_state);
-    virq = kvm_irqchip_add_msi_route(&c, vector_n, &vdev->pdev);
-    if (virq < 0) {
-        event_notifier_cleanup(&vector->kvm_interrupt);
-        return;
+    if (event_notifier_init(&vector->kvm_interrupt, 0)) {
+        goto fail_notifier;
     }
-    kvm_irqchip_commit_route_changes(&c);
 
     if (kvm_irqchip_add_irqfd_notifier_gsi(kvm_state, &vector->kvm_interrupt,
-                                       NULL, virq) < 0) {
-        kvm_irqchip_release_virq(kvm_state, virq);
-        event_notifier_cleanup(&vector->kvm_interrupt);
-        return;
+                                           NULL, vector->virq) < 0) {
+        goto fail_kvm;
     }
 
-    vector->virq = virq;
+    return;
+
+fail_kvm:
+    event_notifier_cleanup(&vector->kvm_interrupt);
+fail_notifier:
+    kvm_irqchip_release_virq(kvm_state, vector->virq);
+    vector->virq = -1;
 }
 
 static void vfio_remove_kvm_msi_virq(VFIOMSIVector *vector)
@@ -494,7 +500,14 @@ static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr,
         }
     } else {
         if (msg) {
-            vfio_add_kvm_msi_virq(vdev, vector, nr, true);
+            if (vdev->defer_kvm_irq_routing) {
+                vfio_add_kvm_msi_virq(vdev, vector, nr, true);
+            } else {
+                vfio_route_change = kvm_irqchip_begin_route_changes(kvm_state);
+                vfio_add_kvm_msi_virq(vdev, vector, nr, true);
+                kvm_irqchip_commit_route_changes(&vfio_route_change);
+                vfio_connect_kvm_msi_virq(vector);
+            }
         }
     }
 
@@ -504,11 +517,13 @@ static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr,
      * increase them as needed.
      */
     if (vdev->nr_vectors < nr + 1) {
-        vfio_disable_irqindex(&vdev->vbasedev, VFIO_PCI_MSIX_IRQ_INDEX);
         vdev->nr_vectors = nr + 1;
-        ret = vfio_enable_vectors(vdev, true);
-        if (ret) {
-            error_report("vfio: failed to enable vectors, %d", ret);
+        if (!vdev->defer_kvm_irq_routing) {
+            vfio_disable_irqindex(&vdev->vbasedev, VFIO_PCI_MSIX_IRQ_INDEX);
+            ret = vfio_enable_vectors(vdev, true);
+            if (ret) {
+                error_report("vfio: failed to enable vectors, %d", ret);
+            }
         }
     } else {
         Error *err = NULL;
@@ -570,6 +585,27 @@ static void vfio_msix_vector_release(PCIDevice *pdev, unsigned int nr)
     }
 }
 
+static void vfio_prepare_kvm_msi_virq_batch(VFIOPCIDevice *vdev)
+{
+    assert(!vdev->defer_kvm_irq_routing);
+    vdev->defer_kvm_irq_routing = true;
+    vfio_route_change = kvm_irqchip_begin_route_changes(kvm_state);
+}
+
+static void vfio_commit_kvm_msi_virq_batch(VFIOPCIDevice *vdev)
+{
+    int i;
+
+    assert(vdev->defer_kvm_irq_routing);
+    vdev->defer_kvm_irq_routing = false;
+
+    kvm_irqchip_commit_route_changes(&vfio_route_change);
+
+    for (i = 0; i < vdev->nr_vectors; i++) {
+        vfio_connect_kvm_msi_virq(&vdev->msi_vectors[i]);
+    }
+}
+
 static void vfio_msix_enable(VFIOPCIDevice *vdev)
 {
     vfio_disable_interrupts(vdev);
@@ -579,26 +615,45 @@ static void vfio_msix_enable(VFIOPCIDevice *vdev)
     vdev->interrupt = VFIO_INT_MSIX;
 
     /*
-     * Some communication channels between VF & PF or PF & fw rely on the
-     * physical state of the device and expect that enabling MSI-X from the
-     * guest enables the same on the host.  When our guest is Linux, the
-     * guest driver call to pci_enable_msix() sets the enabling bit in the
-     * MSI-X capability, but leaves the vector table masked.  We therefore
-     * can't rely on a vector_use callback (from request_irq() in the guest)
-     * to switch the physical device into MSI-X mode because that may come a
-     * long time after pci_enable_msix().  This code enables vector 0 with
-     * triggering to userspace, then immediately release the vector, leaving
-     * the physical device with no vectors enabled, but MSI-X enabled, just
-     * like the guest view.
+     * Setting vector notifiers triggers synchronous vector-use
+     * callbacks for each active vector.  Deferring to commit the KVM
+     * routes once rather than per vector provides a substantial
+     * performance improvement.
      */
-    vfio_msix_vector_do_use(&vdev->pdev, 0, NULL, NULL);
-    vfio_msix_vector_release(&vdev->pdev, 0);
+    vfio_prepare_kvm_msi_virq_batch(vdev);
 
     if (msix_set_vector_notifiers(&vdev->pdev, vfio_msix_vector_use,
                                   vfio_msix_vector_release, NULL)) {
         error_report("vfio: msix_set_vector_notifiers failed");
     }
 
+    vfio_commit_kvm_msi_virq_batch(vdev);
+
+    if (vdev->nr_vectors) {
+        int ret;
+
+        ret = vfio_enable_vectors(vdev, true);
+        if (ret) {
+            error_report("vfio: failed to enable vectors, %d", ret);
+        }
+    } else {
+        /*
+         * Some communication channels between VF & PF or PF & fw rely on the
+         * physical state of the device and expect that enabling MSI-X from the
+         * guest enables the same on the host.  When our guest is Linux, the
+         * guest driver call to pci_enable_msix() sets the enabling bit in the
+         * MSI-X capability, but leaves the vector table masked.  We therefore
+         * can't rely on a vector_use callback (from request_irq() in the guest)
+         * to switch the physical device into MSI-X mode because that may come a
+         * long time after pci_enable_msix().  This code enables vector 0 with
+         * triggering to userspace, then immediately release the vector, leaving
+         * the physical device with no vectors enabled, but MSI-X enabled, just
+         * like the guest view.
+         */
+        vfio_msix_vector_do_use(&vdev->pdev, 0, NULL, NULL);
+        vfio_msix_vector_release(&vdev->pdev, 0);
+    }
+
     trace_vfio_msix_enable(vdev->vbasedev.name);
 }
 
@@ -608,6 +663,13 @@ static void vfio_msi_enable(VFIOPCIDevice *vdev)
 
     vfio_disable_interrupts(vdev);
 
+    /*
+     * Setting vector notifiers needs to enable route for each vector.
+     * Deferring to commit the KVM routes once rather than per vector
+     * provides a substantial performance improvement.
+     */
+    vfio_prepare_kvm_msi_virq_batch(vdev);
+
     vdev->nr_vectors = msi_nr_vectors_allocated(&vdev->pdev);
 retry:
     vdev->msi_vectors = g_new0(VFIOMSIVector, vdev->nr_vectors);
@@ -633,6 +695,8 @@ retry:
         vfio_add_kvm_msi_virq(vdev, vector, i, false);
     }
 
+    vfio_commit_kvm_msi_virq_batch(vdev);
+
     /* Set interrupt type prior to possible interrupts */
     vdev->interrupt = VFIO_INT_MSI;
 
diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h
index 64777516d1..7c236a52f4 100644
--- a/hw/vfio/pci.h
+++ b/hw/vfio/pci.h
@@ -19,6 +19,7 @@
 #include "qemu/queue.h"
 #include "qemu/timer.h"
 #include "qom/object.h"
+#include "sysemu/kvm.h"
 
 #define PCI_ANY_ID (~0)
 
@@ -171,6 +172,7 @@ struct VFIOPCIDevice {
     bool no_kvm_ioeventfd;
     bool no_vfio_ioeventfd;
     bool enable_ramfb;
+    bool defer_kvm_irq_routing;
     VFIODisplay *dpy;
     Notifier irqchip_change_notifier;
 };

From 99510d271b173f21a7a1e8a238b5f35d9e88a147 Mon Sep 17 00:00:00 2001
From: Xiang Chen <chenxiang66@hisilicon.com>
Date: Sat, 16 Apr 2022 17:08:24 +0800
Subject: [PATCH 073/935] vfio/common: Fix a small boundary issue of a trace

It uses [offset, offset + size - 1] to indicate that the length of range is
size in most places in vfio trace code (such as
trace_vfio_region_region_mmap()) execpt trace_vfio_region_sparse_mmap_entry().
So change it for trace_vfio_region_sparse_mmap_entry(), but if size is zero,
the trace will be weird with an underflow, so move the trace and trace it
only if size is not zero.

Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Link: https://lore.kernel.org/r/1650100104-130737-1-git-send-email-chenxiang66@hisilicon.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/common.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index 2b1f78fdfa..6065834717 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -1544,11 +1544,10 @@ static int vfio_setup_region_sparse_mmaps(VFIORegion *region,
     region->mmaps = g_new0(VFIOMmap, sparse->nr_areas);
 
     for (i = 0, j = 0; i < sparse->nr_areas; i++) {
-        trace_vfio_region_sparse_mmap_entry(i, sparse->areas[i].offset,
-                                            sparse->areas[i].offset +
-                                            sparse->areas[i].size);
-
         if (sparse->areas[i].size) {
+            trace_vfio_region_sparse_mmap_entry(i, sparse->areas[i].offset,
+                                            sparse->areas[i].offset +
+                                            sparse->areas[i].size - 1);
             region->mmaps[j].offset = sparse->areas[i].offset;
             region->mmaps[j].size = sparse->areas[i].size;
             j++;

From 4168cdad398843ed53d650a27651868b4d3e21c9 Mon Sep 17 00:00:00 2001
From: Eric Auger <eric.auger@redhat.com>
Date: Fri, 6 May 2022 15:25:09 +0200
Subject: [PATCH 074/935] sysemu: tpm: Add a stub function for TPM_IS_CRB

In a subsequent patch, VFIO will need to recognize if
a memory region owner is a TPM CRB device. Hence VFIO
needs to use TPM_IS_CRB() even if CONFIG_TPM is unset. So
let's add a stub function.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Suggested-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Stefan Berger <stefanb@linnux.ibm.com>
Link: https://lore.kernel.org/r/20220506132510.1847942-2-eric.auger@redhat.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 include/sysemu/tpm.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/include/sysemu/tpm.h b/include/sysemu/tpm.h
index 68b2206463..fb40e30ff6 100644
--- a/include/sysemu/tpm.h
+++ b/include/sysemu/tpm.h
@@ -80,6 +80,12 @@ static inline TPMVersion tpm_get_version(TPMIf *ti)
 #define tpm_init()  (0)
 #define tpm_cleanup()
 
+/* needed for an alignment check in non-tpm code */
+static inline Object *TPM_IS_CRB(Object *obj)
+{
+     return NULL;
+}
+
 #endif /* CONFIG_TPM */
 
 #endif /* QEMU_TPM_H */

From 851d6d1a0ff29a87ec588205842edf6b86d99b5c Mon Sep 17 00:00:00 2001
From: Eric Auger <eric.auger@redhat.com>
Date: Fri, 6 May 2022 15:25:10 +0200
Subject: [PATCH 075/935] vfio/common: remove spurious tpm-crb-cmd misalignment
 warning
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The CRB command buffer currently is a RAM MemoryRegion and given
its base address alignment, it causes an error report on
vfio_listener_region_add(). This region could have been a RAM device
region, easing the detection of such safe situation but this option
was not well received. So let's add a helper function that uses the
memory region owner type to detect the situation is safe wrt
the assignment. Other device types can be checked here if such kind
of problem occurs again.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Acked-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Link: https://lore.kernel.org/r/20220506132510.1847942-3-eric.auger@redhat.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/common.c     | 27 ++++++++++++++++++++++++++-
 hw/vfio/trace-events |  1 +
 2 files changed, 27 insertions(+), 1 deletion(-)

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index 6065834717..cfcb71974a 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -40,6 +40,7 @@
 #include "trace.h"
 #include "qapi/error.h"
 #include "migration/migration.h"
+#include "sysemu/tpm.h"
 
 VFIOGroupList vfio_group_list =
     QLIST_HEAD_INITIALIZER(vfio_group_list);
@@ -861,6 +862,22 @@ static void vfio_unregister_ram_discard_listener(VFIOContainer *container,
     g_free(vrdl);
 }
 
+static bool vfio_known_safe_misalignment(MemoryRegionSection *section)
+{
+    MemoryRegion *mr = section->mr;
+
+    if (!TPM_IS_CRB(mr->owner)) {
+        return false;
+    }
+
+    /* this is a known safe misaligned region, just trace for debug purpose */
+    trace_vfio_known_safe_misalignment(memory_region_name(mr),
+                                       section->offset_within_address_space,
+                                       section->offset_within_region,
+                                       qemu_real_host_page_size());
+    return true;
+}
+
 static void vfio_listener_region_add(MemoryListener *listener,
                                      MemoryRegionSection *section)
 {
@@ -884,7 +901,15 @@ static void vfio_listener_region_add(MemoryListener *listener,
     if (unlikely((section->offset_within_address_space &
                   ~qemu_real_host_page_mask()) !=
                  (section->offset_within_region & ~qemu_real_host_page_mask()))) {
-        error_report("%s received unaligned region", __func__);
+        if (!vfio_known_safe_misalignment(section)) {
+            error_report("%s received unaligned region %s iova=0x%"PRIx64
+                         " offset_within_region=0x%"PRIx64
+                         " qemu_real_host_page_size=0x%"PRIxPTR,
+                         __func__, memory_region_name(section->mr),
+                         section->offset_within_address_space,
+                         section->offset_within_region,
+                         qemu_real_host_page_size());
+        }
         return;
     }
 
diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events
index 0ef1b5f4a6..582882db91 100644
--- a/hw/vfio/trace-events
+++ b/hw/vfio/trace-events
@@ -100,6 +100,7 @@ vfio_listener_region_add_skip(uint64_t start, uint64_t end) "SKIPPING region_add
 vfio_spapr_group_attach(int groupfd, int tablefd) "Attached groupfd %d to liobn fd %d"
 vfio_listener_region_add_iommu(uint64_t start, uint64_t end) "region_add [iommu] 0x%"PRIx64" - 0x%"PRIx64
 vfio_listener_region_add_ram(uint64_t iova_start, uint64_t iova_end, void *vaddr) "region_add [ram] 0x%"PRIx64" - 0x%"PRIx64" [%p]"
+vfio_known_safe_misalignment(const char *name, uint64_t iova, uint64_t offset_within_region, uintptr_t page_size) "Region \"%s\" iova=0x%"PRIx64" offset_within_region=0x%"PRIx64" qemu_real_host_page_size=0x%"PRIxPTR ": cannot be mapped for DMA"
 vfio_listener_region_add_no_dma_map(const char *name, uint64_t iova, uint64_t size, uint64_t page_size) "Region \"%s\" 0x%"PRIx64" size=0x%"PRIx64" is not aligned to 0x%"PRIx64" and cannot be mapped for DMA"
 vfio_listener_region_del_skip(uint64_t start, uint64_t end) "SKIPPING region_del 0x%"PRIx64" - 0x%"PRIx64
 vfio_listener_region_del(uint64_t start, uint64_t end) "region_del 0x%"PRIx64" - 0x%"PRIx64

From 9d38ffc5d82839cdc9effb2e9ee84a6941a25e1a Mon Sep 17 00:00:00 2001
From: Eric Auger <eric.auger@redhat.com>
Date: Mon, 2 May 2022 02:42:21 -0700
Subject: [PATCH 076/935] hw/vfio/pci: fix vfio_pci_hot_reset_result trace
 point

"%m" format specifier is not interpreted by the trace infrastructure
and thus "%m" is output instead of the actual errno string. Fix it by
outputting strerror(errno).

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Yi Liu <yi.l.liu@intel.com>
Link: https://lore.kernel.org/r/20220502094223.36384-2-yi.l.liu@intel.com
[aw: replace commit log as provided by Eric]
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/pci.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index ef9d7bf326..cb912bd3f4 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -2380,7 +2380,7 @@ static int vfio_pci_hot_reset(VFIOPCIDevice *vdev, bool single)
     g_free(reset);
 
     trace_vfio_pci_hot_reset_result(vdev->vbasedev.name,
-                                    ret ? "%m" : "Success");
+                                    ret ? strerror(errno) : "Success");
 
 out:
     /* Re-enable INTx on affected devices */

From 0d570a2572395cca79b83002e589aeb24fcd1732 Mon Sep 17 00:00:00 2001
From: Eric Auger <eric.auger@redhat.com>
Date: Mon, 2 May 2022 02:42:22 -0700
Subject: [PATCH 077/935] vfio/pci: Use vbasedev local variable in
 vfio_realize()

Using a VFIODevice handle local variable to improve the code readability.

no functional change intended

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Yi Liu <yi.l.liu@intel.com>
Link: https://lore.kernel.org/r/20220502094223.36384-3-yi.l.liu@intel.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/pci.c | 49 +++++++++++++++++++++++++------------------------
 1 file changed, 25 insertions(+), 24 deletions(-)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index cb912bd3f4..939dcc3d4a 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -2846,6 +2846,7 @@ static void vfio_unregister_req_notifier(VFIOPCIDevice *vdev)
 static void vfio_realize(PCIDevice *pdev, Error **errp)
 {
     VFIOPCIDevice *vdev = VFIO_PCI(pdev);
+    VFIODevice *vbasedev = &vdev->vbasedev;
     VFIODevice *vbasedev_iter;
     VFIOGroup *group;
     char *tmp, *subsys, group_path[PATH_MAX], *group_name;
@@ -2856,7 +2857,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp)
     int i, ret;
     bool is_mdev;
 
-    if (!vdev->vbasedev.sysfsdev) {
+    if (!vbasedev->sysfsdev) {
         if (!(~vdev->host.domain || ~vdev->host.bus ||
               ~vdev->host.slot || ~vdev->host.function)) {
             error_setg(errp, "No provided host device");
@@ -2864,24 +2865,24 @@ static void vfio_realize(PCIDevice *pdev, Error **errp)
                               "or -device vfio-pci,sysfsdev=PATH_TO_DEVICE\n");
             return;
         }
-        vdev->vbasedev.sysfsdev =
+        vbasedev->sysfsdev =
             g_strdup_printf("/sys/bus/pci/devices/%04x:%02x:%02x.%01x",
                             vdev->host.domain, vdev->host.bus,
                             vdev->host.slot, vdev->host.function);
     }
 
-    if (stat(vdev->vbasedev.sysfsdev, &st) < 0) {
+    if (stat(vbasedev->sysfsdev, &st) < 0) {
         error_setg_errno(errp, errno, "no such host device");
-        error_prepend(errp, VFIO_MSG_PREFIX, vdev->vbasedev.sysfsdev);
+        error_prepend(errp, VFIO_MSG_PREFIX, vbasedev->sysfsdev);
         return;
     }
 
-    vdev->vbasedev.name = g_path_get_basename(vdev->vbasedev.sysfsdev);
-    vdev->vbasedev.ops = &vfio_pci_ops;
-    vdev->vbasedev.type = VFIO_DEVICE_TYPE_PCI;
-    vdev->vbasedev.dev = DEVICE(vdev);
+    vbasedev->name = g_path_get_basename(vbasedev->sysfsdev);
+    vbasedev->ops = &vfio_pci_ops;
+    vbasedev->type = VFIO_DEVICE_TYPE_PCI;
+    vbasedev->dev = DEVICE(vdev);
 
-    tmp = g_strdup_printf("%s/iommu_group", vdev->vbasedev.sysfsdev);
+    tmp = g_strdup_printf("%s/iommu_group", vbasedev->sysfsdev);
     len = readlink(tmp, group_path, sizeof(group_path));
     g_free(tmp);
 
@@ -2899,7 +2900,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp)
         goto error;
     }
 
-    trace_vfio_realize(vdev->vbasedev.name, groupid);
+    trace_vfio_realize(vbasedev->name, groupid);
 
     group = vfio_get_group(groupid, pci_device_iommu_address_space(pdev), errp);
     if (!group) {
@@ -2907,7 +2908,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp)
     }
 
     QLIST_FOREACH(vbasedev_iter, &group->device_list, next) {
-        if (strcmp(vbasedev_iter->name, vdev->vbasedev.name) == 0) {
+        if (strcmp(vbasedev_iter->name, vbasedev->name) == 0) {
             error_setg(errp, "device is already attached");
             vfio_put_group(group);
             goto error;
@@ -2920,22 +2921,22 @@ static void vfio_realize(PCIDevice *pdev, Error **errp)
      * stays in sync with the active working set of the guest driver.  Prevent
      * the x-balloon-allowed option unless this is minimally an mdev device.
      */
-    tmp = g_strdup_printf("%s/subsystem", vdev->vbasedev.sysfsdev);
+    tmp = g_strdup_printf("%s/subsystem", vbasedev->sysfsdev);
     subsys = realpath(tmp, NULL);
     g_free(tmp);
     is_mdev = subsys && (strcmp(subsys, "/sys/bus/mdev") == 0);
     free(subsys);
 
-    trace_vfio_mdev(vdev->vbasedev.name, is_mdev);
+    trace_vfio_mdev(vbasedev->name, is_mdev);
 
-    if (vdev->vbasedev.ram_block_discard_allowed && !is_mdev) {
+    if (vbasedev->ram_block_discard_allowed && !is_mdev) {
         error_setg(errp, "x-balloon-allowed only potentially compatible "
                    "with mdev devices");
         vfio_put_group(group);
         goto error;
     }
 
-    ret = vfio_get_device(group, vdev->vbasedev.name, &vdev->vbasedev, errp);
+    ret = vfio_get_device(group, vbasedev->name, vbasedev, errp);
     if (ret) {
         vfio_put_group(group);
         goto error;
@@ -2948,7 +2949,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp)
     }
 
     /* Get a copy of config space */
-    ret = pread(vdev->vbasedev.fd, vdev->pdev.config,
+    ret = pread(vbasedev->fd, vdev->pdev.config,
                 MIN(pci_config_size(&vdev->pdev), vdev->config_size),
                 vdev->config_offset);
     if (ret < (int)MIN(pci_config_size(&vdev->pdev), vdev->config_size)) {
@@ -2976,7 +2977,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp)
             goto error;
         }
         vfio_add_emulated_word(vdev, PCI_VENDOR_ID, vdev->vendor_id, ~0);
-        trace_vfio_pci_emulated_vendor_id(vdev->vbasedev.name, vdev->vendor_id);
+        trace_vfio_pci_emulated_vendor_id(vbasedev->name, vdev->vendor_id);
     } else {
         vdev->vendor_id = pci_get_word(pdev->config + PCI_VENDOR_ID);
     }
@@ -2987,7 +2988,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp)
             goto error;
         }
         vfio_add_emulated_word(vdev, PCI_DEVICE_ID, vdev->device_id, ~0);
-        trace_vfio_pci_emulated_device_id(vdev->vbasedev.name, vdev->device_id);
+        trace_vfio_pci_emulated_device_id(vbasedev->name, vdev->device_id);
     } else {
         vdev->device_id = pci_get_word(pdev->config + PCI_DEVICE_ID);
     }
@@ -2999,7 +3000,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp)
         }
         vfio_add_emulated_word(vdev, PCI_SUBSYSTEM_VENDOR_ID,
                                vdev->sub_vendor_id, ~0);
-        trace_vfio_pci_emulated_sub_vendor_id(vdev->vbasedev.name,
+        trace_vfio_pci_emulated_sub_vendor_id(vbasedev->name,
                                               vdev->sub_vendor_id);
     }
 
@@ -3009,7 +3010,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp)
             goto error;
         }
         vfio_add_emulated_word(vdev, PCI_SUBSYSTEM_ID, vdev->sub_device_id, ~0);
-        trace_vfio_pci_emulated_sub_device_id(vdev->vbasedev.name,
+        trace_vfio_pci_emulated_sub_device_id(vbasedev->name,
                                               vdev->sub_device_id);
     }
 
@@ -3068,7 +3069,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp)
             goto out_teardown;
         }
 
-        ret = vfio_get_dev_region_info(&vdev->vbasedev,
+        ret = vfio_get_dev_region_info(vbasedev,
                         VFIO_REGION_TYPE_PCI_VENDOR_TYPE | PCI_VENDOR_ID_INTEL,
                         VFIO_REGION_SUBTYPE_INTEL_IGD_OPREGION, &opregion);
         if (ret) {
@@ -3144,9 +3145,9 @@ static void vfio_realize(PCIDevice *pdev, Error **errp)
     }
 
     if (!pdev->failover_pair_id) {
-        ret = vfio_migration_probe(&vdev->vbasedev, errp);
+        ret = vfio_migration_probe(vbasedev, errp);
         if (ret) {
-            error_report("%s: Migration disabled", vdev->vbasedev.name);
+            error_report("%s: Migration disabled", vbasedev->name);
         }
     }
 
@@ -3163,7 +3164,7 @@ out_teardown:
     vfio_teardown_msi(vdev);
     vfio_bars_exit(vdev);
 error:
-    error_prepend(errp, VFIO_MSG_PREFIX, vdev->vbasedev.name);
+    error_prepend(errp, VFIO_MSG_PREFIX, vbasedev->name);
 }
 
 static void vfio_instance_finalize(Object *obj)

From 44ee6aaae0c937abb631e57a9853c2cdef2bc9bb Mon Sep 17 00:00:00 2001
From: Yi Liu <yi.l.liu@intel.com>
Date: Mon, 2 May 2022 02:42:23 -0700
Subject: [PATCH 078/935] vfio/common: Rename VFIOGuestIOMMU::iommu into
 ::iommu_mr

Rename VFIOGuestIOMMU iommu field into iommu_mr. Then it becomes clearer
it is an IOMMU memory region.

no functional change intended

Signed-off-by: Yi Liu <yi.l.liu@intel.com>
Link: https://lore.kernel.org/r/20220502094223.36384-4-yi.l.liu@intel.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/common.c              | 16 ++++++++--------
 include/hw/vfio/vfio-common.h |  2 +-
 2 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index cfcb71974a..159f910421 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -1017,7 +1017,7 @@ static void vfio_listener_region_add(MemoryListener *listener,
          * device emulation the VFIO iommu handles to use).
          */
         giommu = g_malloc0(sizeof(*giommu));
-        giommu->iommu = iommu_mr;
+        giommu->iommu_mr = iommu_mr;
         giommu->iommu_offset = section->offset_within_address_space -
                                section->offset_within_region;
         giommu->container = container;
@@ -1032,7 +1032,7 @@ static void vfio_listener_region_add(MemoryListener *listener,
                             int128_get64(llend),
                             iommu_idx);
 
-        ret = memory_region_iommu_set_page_size_mask(giommu->iommu,
+        ret = memory_region_iommu_set_page_size_mask(giommu->iommu_mr,
                                                      container->pgsizes,
                                                      &err);
         if (ret) {
@@ -1047,7 +1047,7 @@ static void vfio_listener_region_add(MemoryListener *listener,
             goto fail;
         }
         QLIST_INSERT_HEAD(&container->giommu_list, giommu, giommu_next);
-        memory_region_iommu_replay(giommu->iommu, &giommu->n);
+        memory_region_iommu_replay(giommu->iommu_mr, &giommu->n);
 
         return;
     }
@@ -1153,7 +1153,7 @@ static void vfio_listener_region_del(MemoryListener *listener,
         VFIOGuestIOMMU *giommu;
 
         QLIST_FOREACH(giommu, &container->giommu_list, giommu_next) {
-            if (MEMORY_REGION(giommu->iommu) == section->mr &&
+            if (MEMORY_REGION(giommu->iommu_mr) == section->mr &&
                 giommu->n.start == section->offset_within_region) {
                 memory_region_unregister_iommu_notifier(section->mr,
                                                         &giommu->n);
@@ -1418,11 +1418,11 @@ static int vfio_sync_dirty_bitmap(VFIOContainer *container,
         VFIOGuestIOMMU *giommu;
 
         QLIST_FOREACH(giommu, &container->giommu_list, giommu_next) {
-            if (MEMORY_REGION(giommu->iommu) == section->mr &&
+            if (MEMORY_REGION(giommu->iommu_mr) == section->mr &&
                 giommu->n.start == section->offset_within_region) {
                 Int128 llend;
                 vfio_giommu_dirty_notifier gdn = { .giommu = giommu };
-                int idx = memory_region_iommu_attrs_to_index(giommu->iommu,
+                int idx = memory_region_iommu_attrs_to_index(giommu->iommu_mr,
                                                        MEMTXATTRS_UNSPECIFIED);
 
                 llend = int128_add(int128_make64(section->offset_within_region),
@@ -1435,7 +1435,7 @@ static int vfio_sync_dirty_bitmap(VFIOContainer *container,
                                     section->offset_within_region,
                                     int128_get64(llend),
                                     idx);
-                memory_region_iommu_replay(giommu->iommu, &gdn.n);
+                memory_region_iommu_replay(giommu->iommu_mr, &gdn.n);
                 break;
             }
         }
@@ -2270,7 +2270,7 @@ static void vfio_disconnect_container(VFIOGroup *group)
 
         QLIST_FOREACH_SAFE(giommu, &container->giommu_list, giommu_next, tmp) {
             memory_region_unregister_iommu_notifier(
-                    MEMORY_REGION(giommu->iommu), &giommu->n);
+                    MEMORY_REGION(giommu->iommu_mr), &giommu->n);
             QLIST_REMOVE(giommu, giommu_next);
             g_free(giommu);
         }
diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
index 8af11b0a76..e573f5a9f1 100644
--- a/include/hw/vfio/vfio-common.h
+++ b/include/hw/vfio/vfio-common.h
@@ -98,7 +98,7 @@ typedef struct VFIOContainer {
 
 typedef struct VFIOGuestIOMMU {
     VFIOContainer *container;
-    IOMMUMemoryRegion *iommu;
+    IOMMUMemoryRegion *iommu_mr;
     hwaddr iommu_offset;
     IOMMUNotifier n;
     QLIST_ENTRY(VFIOGuestIOMMU) giommu_next;

From 0269a6cf4ebff2eb6d8f893a5179be818efa90c2 Mon Sep 17 00:00:00 2001
From: Max Filippov <jcmvbkbc@gmail.com>
Date: Thu, 21 Apr 2022 13:08:23 -0700
Subject: [PATCH 079/935] target/xtensa: fix missing tcg_temp_free in
 gen_window_check

pc and w are allocated with tcg_const_i32 but not freed in
gen_window_check. Use tcg_constant_i32 for them both.

Fixes: 2db59a76c421 ("target-xtensa: record available window in TB flags")
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 target/xtensa/translate.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/xtensa/translate.c b/target/xtensa/translate.c
index 1485df2f22..591d87b4ef 100644
--- a/target/xtensa/translate.c
+++ b/target/xtensa/translate.c
@@ -571,8 +571,8 @@ static bool gen_window_check(DisasContext *dc, uint32_t mask)
     unsigned r = 31 - clz32(mask);
 
     if (r / 4 > dc->window) {
-        TCGv_i32 pc = tcg_const_i32(dc->pc);
-        TCGv_i32 w = tcg_const_i32(r / 4);
+        TCGv_i32 pc = tcg_constant_i32(dc->pc);
+        TCGv_i32 w = tcg_constant_i32(r / 4);
 
         gen_helper_window_check(cpu_env, pc, w);
         dc->base.is_jmp = DISAS_NORETURN;

From 2b570a178a59338ad7512478c7dda06f07658506 Mon Sep 17 00:00:00 2001
From: Max Filippov <jcmvbkbc@gmail.com>
Date: Thu, 21 Apr 2022 13:27:27 -0700
Subject: [PATCH 080/935] target/xtensa: use tcg_contatnt_* for numeric
 literals

Replace tcg_const_* for numeric literals with tcg_constant_*.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 target/xtensa/translate.c | 28 +++++++++-------------------
 1 file changed, 9 insertions(+), 19 deletions(-)

diff --git a/target/xtensa/translate.c b/target/xtensa/translate.c
index 591d87b4ef..58b3e32dcb 100644
--- a/target/xtensa/translate.c
+++ b/target/xtensa/translate.c
@@ -306,16 +306,14 @@ static void gen_right_shift_sar(DisasContext *dc, TCGv_i32 sa)
 
 static void gen_left_shift_sar(DisasContext *dc, TCGv_i32 sa)
 {
-    TCGv_i32 tmp = tcg_const_i32(32);
     if (!dc->sar_m32_allocated) {
         dc->sar_m32 = tcg_temp_local_new_i32();
         dc->sar_m32_allocated = true;
     }
     tcg_gen_andi_i32(dc->sar_m32, sa, 0x1f);
-    tcg_gen_sub_i32(cpu_SR[SAR], tmp, dc->sar_m32);
+    tcg_gen_sub_i32(cpu_SR[SAR], tcg_constant_i32(32), dc->sar_m32);
     dc->sar_5bit = false;
     dc->sar_m32_5bit = true;
-    tcg_temp_free(tmp);
 }
 
 static void gen_exception(DisasContext *dc, int excp)
@@ -1957,11 +1955,10 @@ static void translate_mov(DisasContext *dc, const OpcodeArg arg[],
 static void translate_movcond(DisasContext *dc, const OpcodeArg arg[],
                               const uint32_t par[])
 {
-    TCGv_i32 zero = tcg_const_i32(0);
+    TCGv_i32 zero = tcg_constant_i32(0);
 
     tcg_gen_movcond_i32(par[0], arg[0].out,
                         arg[2].in, zero, arg[1].in, arg[0].in);
-    tcg_temp_free(zero);
 }
 
 static void translate_movi(DisasContext *dc, const OpcodeArg arg[],
@@ -1973,7 +1970,7 @@ static void translate_movi(DisasContext *dc, const OpcodeArg arg[],
 static void translate_movp(DisasContext *dc, const OpcodeArg arg[],
                            const uint32_t par[])
 {
-    TCGv_i32 zero = tcg_const_i32(0);
+    TCGv_i32 zero = tcg_constant_i32(0);
     TCGv_i32 tmp = tcg_temp_new_i32();
 
     tcg_gen_andi_i32(tmp, arg[2].in, 1 << arg[2].imm);
@@ -1981,7 +1978,6 @@ static void translate_movp(DisasContext *dc, const OpcodeArg arg[],
                         arg[0].out, tmp, zero,
                         arg[1].in, arg[0].in);
     tcg_temp_free(tmp);
-    tcg_temp_free(zero);
 }
 
 static void translate_movsp(DisasContext *dc, const OpcodeArg arg[],
@@ -6444,7 +6440,7 @@ static void translate_compare_d(DisasContext *dc, const OpcodeArg arg[],
         [COMPARE_OLE] = gen_helper_ole_d,
         [COMPARE_ULE] = gen_helper_ule_d,
     };
-    TCGv_i32 zero = tcg_const_i32(0);
+    TCGv_i32 zero = tcg_constant_i32(0);
     TCGv_i32 res = tcg_temp_new_i32();
     TCGv_i32 set_br = tcg_temp_new_i32();
     TCGv_i32 clr_br = tcg_temp_new_i32();
@@ -6456,7 +6452,6 @@ static void translate_compare_d(DisasContext *dc, const OpcodeArg arg[],
     tcg_gen_movcond_i32(TCG_COND_NE,
                         arg[0].out, res, zero,
                         set_br, clr_br);
-    tcg_temp_free(zero);
     tcg_temp_free(res);
     tcg_temp_free(set_br);
     tcg_temp_free(clr_br);
@@ -6476,7 +6471,7 @@ static void translate_compare_s(DisasContext *dc, const OpcodeArg arg[],
         [COMPARE_ULE] = gen_helper_ule_s,
     };
     OpcodeArg arg32[3];
-    TCGv_i32 zero = tcg_const_i32(0);
+    TCGv_i32 zero = tcg_constant_i32(0);
     TCGv_i32 res = tcg_temp_new_i32();
     TCGv_i32 set_br = tcg_temp_new_i32();
     TCGv_i32 clr_br = tcg_temp_new_i32();
@@ -6490,7 +6485,6 @@ static void translate_compare_s(DisasContext *dc, const OpcodeArg arg[],
                         arg[0].out, res, zero,
                         set_br, clr_br);
     put_f32_i2(arg, arg32, 1, 2);
-    tcg_temp_free(zero);
     tcg_temp_free(res);
     tcg_temp_free(set_br);
     tcg_temp_free(clr_br);
@@ -6666,14 +6660,13 @@ static void translate_mov_s(DisasContext *dc, const OpcodeArg arg[],
 static void translate_movcond_d(DisasContext *dc, const OpcodeArg arg[],
                                 const uint32_t par[])
 {
-    TCGv_i64 zero = tcg_const_i64(0);
+    TCGv_i64 zero = tcg_constant_i64(0);
     TCGv_i64 arg2 = tcg_temp_new_i64();
 
     tcg_gen_ext_i32_i64(arg2, arg[2].in);
     tcg_gen_movcond_i64(par[0], arg[0].out,
                         arg2, zero,
                         arg[1].in, arg[0].in);
-    tcg_temp_free_i64(zero);
     tcg_temp_free_i64(arg2);
 }
 
@@ -6681,12 +6674,11 @@ static void translate_movcond_s(DisasContext *dc, const OpcodeArg arg[],
                                 const uint32_t par[])
 {
     if (arg[0].num_bits == 32) {
-        TCGv_i32 zero = tcg_const_i32(0);
+        TCGv_i32 zero = tcg_constant_i32(0);
 
         tcg_gen_movcond_i32(par[0], arg[0].out,
                             arg[2].in, zero,
                             arg[1].in, arg[0].in);
-        tcg_temp_free(zero);
     } else {
         translate_movcond_d(dc, arg, par);
     }
@@ -6695,7 +6687,7 @@ static void translate_movcond_s(DisasContext *dc, const OpcodeArg arg[],
 static void translate_movp_d(DisasContext *dc, const OpcodeArg arg[],
                              const uint32_t par[])
 {
-    TCGv_i64 zero = tcg_const_i64(0);
+    TCGv_i64 zero = tcg_constant_i64(0);
     TCGv_i32 tmp1 = tcg_temp_new_i32();
     TCGv_i64 tmp2 = tcg_temp_new_i64();
 
@@ -6704,7 +6696,6 @@ static void translate_movp_d(DisasContext *dc, const OpcodeArg arg[],
     tcg_gen_movcond_i64(par[0],
                         arg[0].out, tmp2, zero,
                         arg[1].in, arg[0].in);
-    tcg_temp_free_i64(zero);
     tcg_temp_free_i32(tmp1);
     tcg_temp_free_i64(tmp2);
 }
@@ -6713,7 +6704,7 @@ static void translate_movp_s(DisasContext *dc, const OpcodeArg arg[],
                              const uint32_t par[])
 {
     if (arg[0].num_bits == 32) {
-        TCGv_i32 zero = tcg_const_i32(0);
+        TCGv_i32 zero = tcg_constant_i32(0);
         TCGv_i32 tmp = tcg_temp_new_i32();
 
         tcg_gen_andi_i32(tmp, arg[2].in, 1 << arg[2].imm);
@@ -6721,7 +6712,6 @@ static void translate_movp_s(DisasContext *dc, const OpcodeArg arg[],
                             arg[0].out, tmp, zero,
                             arg[1].in, arg[0].in);
         tcg_temp_free(tmp);
-        tcg_temp_free(zero);
     } else {
         translate_movp_d(dc, arg, par);
     }

From f99fbd125b916b484d2d92428e3cd8746cbef14c Mon Sep 17 00:00:00 2001
From: Max Filippov <jcmvbkbc@gmail.com>
Date: Thu, 21 Apr 2022 13:38:58 -0700
Subject: [PATCH 081/935] target/xtensa: use tcg_constant_* for exceptions

Use tcg_contant_* for exception number, exception cause, debug cause
code and exception PC.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 target/xtensa/translate.c | 18 +++++-------------
 1 file changed, 5 insertions(+), 13 deletions(-)

diff --git a/target/xtensa/translate.c b/target/xtensa/translate.c
index 58b3e32dcb..a21cfa91ed 100644
--- a/target/xtensa/translate.c
+++ b/target/xtensa/translate.c
@@ -318,18 +318,13 @@ static void gen_left_shift_sar(DisasContext *dc, TCGv_i32 sa)
 
 static void gen_exception(DisasContext *dc, int excp)
 {
-    TCGv_i32 tmp = tcg_const_i32(excp);
-    gen_helper_exception(cpu_env, tmp);
-    tcg_temp_free(tmp);
+    gen_helper_exception(cpu_env, tcg_constant_i32(excp));
 }
 
 static void gen_exception_cause(DisasContext *dc, uint32_t cause)
 {
-    TCGv_i32 tpc = tcg_const_i32(dc->pc);
-    TCGv_i32 tcause = tcg_const_i32(cause);
-    gen_helper_exception_cause(cpu_env, tpc, tcause);
-    tcg_temp_free(tpc);
-    tcg_temp_free(tcause);
+    TCGv_i32 pc = tcg_constant_i32(dc->pc);
+    gen_helper_exception_cause(cpu_env, pc, tcg_constant_i32(cause));
     if (cause == ILLEGAL_INSTRUCTION_CAUSE ||
             cause == SYSCALL_CAUSE) {
         dc->base.is_jmp = DISAS_NORETURN;
@@ -338,11 +333,8 @@ static void gen_exception_cause(DisasContext *dc, uint32_t cause)
 
 static void gen_debug_exception(DisasContext *dc, uint32_t cause)
 {
-    TCGv_i32 tpc = tcg_const_i32(dc->pc);
-    TCGv_i32 tcause = tcg_const_i32(cause);
-    gen_helper_debug_exception(cpu_env, tpc, tcause);
-    tcg_temp_free(tpc);
-    tcg_temp_free(tcause);
+    TCGv_i32 pc = tcg_constant_i32(dc->pc);
+    gen_helper_debug_exception(cpu_env, pc, tcg_constant_i32(cause));
     if (cause & (DEBUGCAUSE_IB | DEBUGCAUSE_BI | DEBUGCAUSE_BN)) {
         dc->base.is_jmp = DISAS_NORETURN;
     }

From 4ee412df69907b5fc91489732b4013f23dc91083 Mon Sep 17 00:00:00 2001
From: Max Filippov <jcmvbkbc@gmail.com>
Date: Thu, 21 Apr 2022 13:38:58 -0700
Subject: [PATCH 082/935] target/xtensa: use tcg_constant_* for TLB opcodes

dtlb is a boolean flag, use tcg_constant_* for it.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 target/xtensa/translate.c | 12 ++++--------
 1 file changed, 4 insertions(+), 8 deletions(-)

diff --git a/target/xtensa/translate.c b/target/xtensa/translate.c
index a21cfa91ed..3cfcbf93a7 100644
--- a/target/xtensa/translate.c
+++ b/target/xtensa/translate.c
@@ -1708,10 +1708,9 @@ static void translate_itlb(DisasContext *dc, const OpcodeArg arg[],
                            const uint32_t par[])
 {
 #ifndef CONFIG_USER_ONLY
-    TCGv_i32 dtlb = tcg_const_i32(par[0]);
+    TCGv_i32 dtlb = tcg_constant_i32(par[0]);
 
     gen_helper_itlb(cpu_env, arg[0].in, dtlb);
-    tcg_temp_free(dtlb);
 #endif
 }
 
@@ -2048,11 +2047,10 @@ static void translate_ptlb(DisasContext *dc, const OpcodeArg arg[],
                            const uint32_t par[])
 {
 #ifndef CONFIG_USER_ONLY
-    TCGv_i32 dtlb = tcg_const_i32(par[0]);
+    TCGv_i32 dtlb = tcg_constant_i32(par[0]);
 
     tcg_gen_movi_i32(cpu_pc, dc->pc);
     gen_helper_ptlb(arg[0].out, cpu_env, arg[1].in, dtlb);
-    tcg_temp_free(dtlb);
 #endif
 }
 
@@ -2251,10 +2249,9 @@ static void translate_rtlb(DisasContext *dc, const OpcodeArg arg[],
         gen_helper_rtlb0,
         gen_helper_rtlb1,
     };
-    TCGv_i32 dtlb = tcg_const_i32(par[0]);
+    TCGv_i32 dtlb = tcg_constant_i32(par[0]);
 
     helper[par[1]](arg[0].out, cpu_env, arg[1].in, dtlb);
-    tcg_temp_free(dtlb);
 #endif
 }
 
@@ -2562,10 +2559,9 @@ static void translate_wtlb(DisasContext *dc, const OpcodeArg arg[],
                            const uint32_t par[])
 {
 #ifndef CONFIG_USER_ONLY
-    TCGv_i32 dtlb = tcg_const_i32(par[0]);
+    TCGv_i32 dtlb = tcg_constant_i32(par[0]);
 
     gen_helper_wtlb(cpu_env, arg[0].in, arg[1].in, dtlb);
-    tcg_temp_free(dtlb);
 #endif
 }
 

From dad266a1e35f0b24f07e6e18c67a02edf347c0b3 Mon Sep 17 00:00:00 2001
From: Max Filippov <jcmvbkbc@gmail.com>
Date: Thu, 21 Apr 2022 13:38:58 -0700
Subject: [PATCH 083/935] target/xtensa: use tcg_constant_* for numbered
 special registers

Numbered special registers are small arrays of consecutive SRs. Use
tcg_constant_* for the SR index.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 target/xtensa/translate.c | 16 ++++------------
 1 file changed, 4 insertions(+), 12 deletions(-)

diff --git a/target/xtensa/translate.c b/target/xtensa/translate.c
index 3cfcbf93a7..557e4e173f 100644
--- a/target/xtensa/translate.c
+++ b/target/xtensa/translate.c
@@ -2613,15 +2613,13 @@ static void translate_wsr_ccompare(DisasContext *dc, const OpcodeArg arg[],
 {
 #ifndef CONFIG_USER_ONLY
     uint32_t id = par[0] - CCOMPARE;
-    TCGv_i32 tmp = tcg_const_i32(id);
 
     assert(id < dc->config->nccompare);
     if (tb_cflags(dc->base.tb) & CF_USE_ICOUNT) {
         gen_io_start();
     }
     tcg_gen_mov_i32(cpu_SR[par[0]], arg[0].in);
-    gen_helper_update_ccompare(cpu_env, tmp);
-    tcg_temp_free(tmp);
+    gen_helper_update_ccompare(cpu_env, tcg_constant_i32(id));
 #endif
 }
 
@@ -2641,11 +2639,9 @@ static void translate_wsr_dbreaka(DisasContext *dc, const OpcodeArg arg[],
 {
 #ifndef CONFIG_USER_ONLY
     unsigned id = par[0] - DBREAKA;
-    TCGv_i32 tmp = tcg_const_i32(id);
 
     assert(id < dc->config->ndbreak);
-    gen_helper_wsr_dbreaka(cpu_env, tmp, arg[0].in);
-    tcg_temp_free(tmp);
+    gen_helper_wsr_dbreaka(cpu_env, tcg_constant_i32(id), arg[0].in);
 #endif
 }
 
@@ -2654,11 +2650,9 @@ static void translate_wsr_dbreakc(DisasContext *dc, const OpcodeArg arg[],
 {
 #ifndef CONFIG_USER_ONLY
     unsigned id = par[0] - DBREAKC;
-    TCGv_i32 tmp = tcg_const_i32(id);
 
     assert(id < dc->config->ndbreak);
-    gen_helper_wsr_dbreakc(cpu_env, tmp, arg[0].in);
-    tcg_temp_free(tmp);
+    gen_helper_wsr_dbreakc(cpu_env, tcg_constant_i32(id), arg[0].in);
 #endif
 }
 
@@ -2667,11 +2661,9 @@ static void translate_wsr_ibreaka(DisasContext *dc, const OpcodeArg arg[],
 {
 #ifndef CONFIG_USER_ONLY
     unsigned id = par[0] - IBREAKA;
-    TCGv_i32 tmp = tcg_const_i32(id);
 
     assert(id < dc->config->nibreak);
-    gen_helper_wsr_ibreaka(cpu_env, tmp, arg[0].in);
-    tcg_temp_free(tmp);
+    gen_helper_wsr_ibreaka(cpu_env, tcg_constant_i32(id), arg[0].in);
 #endif
 }
 

From 867e354cbda0cac2d20a478ef39d01aa3bf67118 Mon Sep 17 00:00:00 2001
From: Max Filippov <jcmvbkbc@gmail.com>
Date: Thu, 21 Apr 2022 13:38:58 -0700
Subject: [PATCH 084/935] target/xtensa: use tcg_constant_* for FPU conversion
 opcodes

FPU conversion opcodes pass scale (range 0..15) and rounding mode to
their helpers. Use tcg_constant_* for them.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 target/xtensa/translate.c | 18 ++++++------------
 1 file changed, 6 insertions(+), 12 deletions(-)

diff --git a/target/xtensa/translate.c b/target/xtensa/translate.c
index 557e4e173f..74272c0770 100644
--- a/target/xtensa/translate.c
+++ b/target/xtensa/translate.c
@@ -6513,20 +6513,19 @@ static void translate_const_s(DisasContext *dc, const OpcodeArg arg[],
 static void translate_float_d(DisasContext *dc, const OpcodeArg arg[],
                               const uint32_t par[])
 {
-    TCGv_i32 scale = tcg_const_i32(-arg[2].imm);
+    TCGv_i32 scale = tcg_constant_i32(-arg[2].imm);
 
     if (par[0]) {
         gen_helper_uitof_d(arg[0].out, cpu_env, arg[1].in, scale);
     } else {
         gen_helper_itof_d(arg[0].out, cpu_env, arg[1].in, scale);
     }
-    tcg_temp_free(scale);
 }
 
 static void translate_float_s(DisasContext *dc, const OpcodeArg arg[],
                               const uint32_t par[])
 {
-    TCGv_i32 scale = tcg_const_i32(-arg[2].imm);
+    TCGv_i32 scale = tcg_constant_i32(-arg[2].imm);
     OpcodeArg arg32[1];
 
     get_f32_o1(arg, arg32, 0);
@@ -6536,14 +6535,13 @@ static void translate_float_s(DisasContext *dc, const OpcodeArg arg[],
         gen_helper_itof_s(arg32[0].out, cpu_env, arg[1].in, scale);
     }
     put_f32_o1(arg, arg32, 0);
-    tcg_temp_free(scale);
 }
 
 static void translate_ftoi_d(DisasContext *dc, const OpcodeArg arg[],
                              const uint32_t par[])
 {
-    TCGv_i32 rounding_mode = tcg_const_i32(par[0]);
-    TCGv_i32 scale = tcg_const_i32(arg[2].imm);
+    TCGv_i32 rounding_mode = tcg_constant_i32(par[0]);
+    TCGv_i32 scale = tcg_constant_i32(arg[2].imm);
 
     if (par[1]) {
         gen_helper_ftoui_d(arg[0].out, cpu_env, arg[1].in,
@@ -6552,15 +6550,13 @@ static void translate_ftoi_d(DisasContext *dc, const OpcodeArg arg[],
         gen_helper_ftoi_d(arg[0].out, cpu_env, arg[1].in,
                           rounding_mode, scale);
     }
-    tcg_temp_free(rounding_mode);
-    tcg_temp_free(scale);
 }
 
 static void translate_ftoi_s(DisasContext *dc, const OpcodeArg arg[],
                              const uint32_t par[])
 {
-    TCGv_i32 rounding_mode = tcg_const_i32(par[0]);
-    TCGv_i32 scale = tcg_const_i32(arg[2].imm);
+    TCGv_i32 rounding_mode = tcg_constant_i32(par[0]);
+    TCGv_i32 scale = tcg_constant_i32(arg[2].imm);
     OpcodeArg arg32[2];
 
     get_f32_i1(arg, arg32, 1);
@@ -6572,8 +6568,6 @@ static void translate_ftoi_s(DisasContext *dc, const OpcodeArg arg[],
                           rounding_mode, scale);
     }
     put_f32_i1(arg, arg32, 1);
-    tcg_temp_free(rounding_mode);
-    tcg_temp_free(scale);
 }
 
 static void translate_ldsti(DisasContext *dc, const OpcodeArg arg[],

From 6ade0ce972042a8e6d6a62373644cd1bd0f5ba4f Mon Sep 17 00:00:00 2001
From: Max Filippov <jcmvbkbc@gmail.com>
Date: Thu, 21 Apr 2022 13:46:20 -0700
Subject: [PATCH 085/935] target/xtensa: use tcg_constant_* for remaining
 opcodes

- gen_jumpi passes target PC to the helper;
- gen_callw_slot uses callinc (1..3);
- gen_brcondi passes immediate field (less than 32 different possible
  values) to the helper;
- disas_xtensa_insn passes PC to the helpers;
- translate_entry passes PC, stack register number (0..15) and stack
  frame size to the helper;
- gen_check_exclusive passes PC and boolean flag to the helper;
- test_exceptions_retw passes PC to the helper;
- gen_check_atomctl passes PC to the helper;
- translate_ssai passes immediate shift amount (0..31) to the helper;
- gen_waiti passes next PC and an immediate (0..15) to the helper;

use tcg_constant_* for the constants listed above. Fold gen_waiti body
into the translate_waiti as it's the only user.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 target/xtensa/translate.c | 77 +++++++++++++--------------------------
 1 file changed, 25 insertions(+), 52 deletions(-)

diff --git a/target/xtensa/translate.c b/target/xtensa/translate.c
index 74272c0770..b0576d90fa 100644
--- a/target/xtensa/translate.c
+++ b/target/xtensa/translate.c
@@ -396,19 +396,15 @@ static int adjust_jump_slot(DisasContext *dc, uint32_t dest, int slot)
 
 static void gen_jumpi(DisasContext *dc, uint32_t dest, int slot)
 {
-    TCGv_i32 tmp = tcg_const_i32(dest);
-    gen_jump_slot(dc, tmp, adjust_jump_slot(dc, dest, slot));
-    tcg_temp_free(tmp);
+    gen_jump_slot(dc, tcg_constant_i32(dest),
+                  adjust_jump_slot(dc, dest, slot));
 }
 
 static void gen_callw_slot(DisasContext *dc, int callinc, TCGv_i32 dest,
         int slot)
 {
-    TCGv_i32 tcallinc = tcg_const_i32(callinc);
-
     tcg_gen_deposit_i32(cpu_SR[PS], cpu_SR[PS],
-            tcallinc, PS_CALLINC_SHIFT, PS_CALLINC_LEN);
-    tcg_temp_free(tcallinc);
+            tcg_constant_i32(callinc), PS_CALLINC_SHIFT, PS_CALLINC_LEN);
     tcg_gen_movi_i32(cpu_R[callinc << 2],
             (callinc << 30) | (dc->base.pc_next & 0x3fffffff));
     gen_jump_slot(dc, dest, slot);
@@ -454,9 +450,7 @@ static void gen_brcond(DisasContext *dc, TCGCond cond,
 static void gen_brcondi(DisasContext *dc, TCGCond cond,
                         TCGv_i32 t0, uint32_t t1, uint32_t addr)
 {
-    TCGv_i32 tmp = tcg_const_i32(t1);
-    gen_brcond(dc, cond, t0, tmp, addr);
-    tcg_temp_free(tmp);
+    gen_brcond(dc, cond, t0, tcg_constant_i32(t1), addr);
 }
 
 static uint32_t test_exceptions_sr(DisasContext *dc, const OpcodeArg arg[],
@@ -541,21 +535,6 @@ static MemOp gen_load_store_alignment(DisasContext *dc, MemOp mop,
     return mop;
 }
 
-#ifndef CONFIG_USER_ONLY
-static void gen_waiti(DisasContext *dc, uint32_t imm4)
-{
-    TCGv_i32 pc = tcg_const_i32(dc->base.pc_next);
-    TCGv_i32 intlevel = tcg_const_i32(imm4);
-
-    if (tb_cflags(dc->base.tb) & CF_USE_ICOUNT) {
-        gen_io_start();
-    }
-    gen_helper_waiti(cpu_env, pc, intlevel);
-    tcg_temp_free(pc);
-    tcg_temp_free(intlevel);
-}
-#endif
-
 static bool gen_window_check(DisasContext *dc, uint32_t mask)
 {
     unsigned r = 31 - clz32(mask);
@@ -1070,17 +1049,15 @@ static void disas_xtensa_insn(CPUXtensaState *env, DisasContext *dc)
     }
 
     if (op_flags & XTENSA_OP_UNDERFLOW) {
-        TCGv_i32 tmp = tcg_const_i32(dc->pc);
+        TCGv_i32 pc = tcg_constant_i32(dc->pc);
 
-        gen_helper_test_underflow_retw(cpu_env, tmp);
-        tcg_temp_free(tmp);
+        gen_helper_test_underflow_retw(cpu_env, pc);
     }
 
     if (op_flags & XTENSA_OP_ALLOCA) {
-        TCGv_i32 tmp = tcg_const_i32(dc->pc);
+        TCGv_i32 pc = tcg_constant_i32(dc->pc);
 
-        gen_helper_movsp(cpu_env, tmp);
-        tcg_temp_free(tmp);
+        gen_helper_movsp(cpu_env, pc);
     }
 
     if (coprocessor && !gen_check_cpenable(dc, coprocessor)) {
@@ -1660,13 +1637,10 @@ static uint32_t test_overflow_entry(DisasContext *dc, const OpcodeArg arg[],
 static void translate_entry(DisasContext *dc, const OpcodeArg arg[],
                             const uint32_t par[])
 {
-    TCGv_i32 pc = tcg_const_i32(dc->pc);
-    TCGv_i32 s = tcg_const_i32(arg[0].imm);
-    TCGv_i32 imm = tcg_const_i32(arg[1].imm);
+    TCGv_i32 pc = tcg_constant_i32(dc->pc);
+    TCGv_i32 s = tcg_constant_i32(arg[0].imm);
+    TCGv_i32 imm = tcg_constant_i32(arg[1].imm);
     gen_helper_entry(cpu_env, pc, s, imm);
-    tcg_temp_free(imm);
-    tcg_temp_free(s);
-    tcg_temp_free(pc);
 }
 
 static void translate_extui(DisasContext *dc, const OpcodeArg arg[],
@@ -1746,12 +1720,10 @@ static void gen_check_exclusive(DisasContext *dc, TCGv_i32 addr, bool is_write)
 static void gen_check_exclusive(DisasContext *dc, TCGv_i32 addr, bool is_write)
 {
     if (!option_enabled(dc, XTENSA_OPTION_MPU)) {
-        TCGv_i32 tpc = tcg_const_i32(dc->pc);
-        TCGv_i32 write = tcg_const_i32(is_write);
+        TCGv_i32 pc = tcg_constant_i32(dc->pc);
 
-        gen_helper_check_exclusive(cpu_env, tpc, addr, write);
-        tcg_temp_free(tpc);
-        tcg_temp_free(write);
+        gen_helper_check_exclusive(cpu_env, pc, addr,
+                                   tcg_constant_i32(is_write));
     }
 }
 #endif
@@ -2128,10 +2100,9 @@ static uint32_t test_exceptions_retw(DisasContext *dc, const OpcodeArg arg[],
                       "Illegal retw instruction(pc = %08x)\n", dc->pc);
         return XTENSA_OP_ILL;
     } else {
-        TCGv_i32 tmp = tcg_const_i32(dc->pc);
+        TCGv_i32 pc = tcg_constant_i32(dc->pc);
 
-        gen_helper_test_ill_retw(cpu_env, tmp);
-        tcg_temp_free(tmp);
+        gen_helper_test_ill_retw(cpu_env, pc);
         return 0;
     }
 }
@@ -2291,10 +2262,9 @@ static void gen_check_atomctl(DisasContext *dc, TCGv_i32 addr)
 #else
 static void gen_check_atomctl(DisasContext *dc, TCGv_i32 addr)
 {
-    TCGv_i32 tpc = tcg_const_i32(dc->pc);
+    TCGv_i32 pc = tcg_constant_i32(dc->pc);
 
-    gen_helper_check_atomctl(cpu_env, tpc, addr);
-    tcg_temp_free(tpc);
+    gen_helper_check_atomctl(cpu_env, pc, addr);
 }
 #endif
 
@@ -2515,9 +2485,7 @@ static void translate_ssa8l(DisasContext *dc, const OpcodeArg arg[],
 static void translate_ssai(DisasContext *dc, const OpcodeArg arg[],
                            const uint32_t par[])
 {
-    TCGv_i32 tmp = tcg_const_i32(arg[0].imm);
-    gen_right_shift_sar(dc, tmp);
-    tcg_temp_free(tmp);
+    gen_right_shift_sar(dc, tcg_constant_i32(arg[0].imm));
 }
 
 static void translate_ssl(DisasContext *dc, const OpcodeArg arg[],
@@ -2551,7 +2519,12 @@ static void translate_waiti(DisasContext *dc, const OpcodeArg arg[],
                             const uint32_t par[])
 {
 #ifndef CONFIG_USER_ONLY
-    gen_waiti(dc, arg[0].imm);
+    TCGv_i32 pc = tcg_constant_i32(dc->base.pc_next);
+
+    if (tb_cflags(dc->base.tb) & CF_USE_ICOUNT) {
+        gen_io_start();
+    }
+    gen_helper_waiti(cpu_env, pc, tcg_constant_i32(arg[0].imm));
 #endif
 }
 

From 8c48e36548a124dff8ad25b28f9cd2210dcdc407 Mon Sep 17 00:00:00 2001
From: Simon Safar <simon@simonsafar.com>
Date: Fri, 22 Apr 2022 21:08:36 -0700
Subject: [PATCH 086/935] target/xtensa: import core lx106

This is the core used in e.g. ESP8266 chips. Importing them
using import_core.sh, with the required files sourced from

https://github.com/espressif/xtensa-overlays

core-lx106.c was generated by the script; the only change is removing
the reference to core-matmap.h which doesn't seem to be available.

Signed-off-by: Simon Safar <simon@simonsafar.com>
Reviewed-by: Max Filippov <jcmvbkbc@gmail.com>
Message-Id: <20220423040835.29254-1-simon@simonsafar.com>
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 target/xtensa/core-lx106.c                    |   51 +
 target/xtensa/core-lx106/core-isa.h           |  470 +
 target/xtensa/core-lx106/gdb-config.c.inc     |   83 +
 target/xtensa/core-lx106/xtensa-modules.c.inc | 7668 +++++++++++++++++
 target/xtensa/cores.list                      |    1 +
 5 files changed, 8273 insertions(+)
 create mode 100644 target/xtensa/core-lx106.c
 create mode 100644 target/xtensa/core-lx106/core-isa.h
 create mode 100644 target/xtensa/core-lx106/gdb-config.c.inc
 create mode 100644 target/xtensa/core-lx106/xtensa-modules.c.inc

diff --git a/target/xtensa/core-lx106.c b/target/xtensa/core-lx106.c
new file mode 100644
index 0000000000..7a771d09a6
--- /dev/null
+++ b/target/xtensa/core-lx106.c
@@ -0,0 +1,51 @@
+/*
+ * Copyright (c) 2022, Simon Safar, Max Filippov, Open Source and Linux Lab.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in the
+ *       documentation and/or other materials provided with the distribution.
+ *     * Neither the name of the Open Source and Linux Lab nor the
+ *       names of its contributors may be used to endorse or promote products
+ *       derived from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "exec/gdbstub.h"
+#include "qemu/host-utils.h"
+
+#include "core-lx106/core-isa.h"
+#include "overlay_tool.h"
+
+#define xtensa_modules xtensa_modules_lx106
+#include "core-lx106/xtensa-modules.c.inc"
+
+static XtensaConfig lx106 __attribute__((unused)) = {
+    .name = "lx106",
+    .gdb_regmap = {
+        .reg = {
+#include "core-lx106/gdb-config.c.inc"
+        }
+    },
+    .isa_internal = &xtensa_modules,
+    .clock_freq_khz = 40000,
+    DEFAULT_SECTIONS
+};
+
+REGISTER_CORE(lx106)
diff --git a/target/xtensa/core-lx106/core-isa.h b/target/xtensa/core-lx106/core-isa.h
new file mode 100644
index 0000000000..86bcf1a5d6
--- /dev/null
+++ b/target/xtensa/core-lx106/core-isa.h
@@ -0,0 +1,470 @@
+/*
+ * xtensa/config/core-isa.h -- HAL definitions that are dependent on Xtensa
+ *				processor CORE configuration
+ *
+ *  See <xtensa/config/core.h>, which includes this file, for more details.
+ */
+
+/* Xtensa processor core configuration information.
+
+   Copyright (c) 1999-2010 Tensilica Inc.
+
+   Permission is hereby granted, free of charge, to any person obtaining
+   a copy of this software and associated documentation files (the
+   "Software"), to deal in the Software without restriction, including
+   without limitation the rights to use, copy, modify, merge, publish,
+   distribute, sublicense, and/or sell copies of the Software, and to
+   permit persons to whom the Software is furnished to do so, subject to
+   the following conditions:
+
+   The above copyright notice and this permission notice shall be included
+   in all copies or substantial portions of the Software.
+
+   THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+   EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+   MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+   IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+   CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+   TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+   SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.  */
+
+#ifndef _XTENSA_CORE_CONFIGURATION_H
+#define _XTENSA_CORE_CONFIGURATION_H
+
+
+/****************************************************************************
+	    Parameters Useful for Any Code, USER or PRIVILEGED
+ ****************************************************************************/
+
+/*
+ *  Note:  Macros of the form XCHAL_HAVE_*** have a value of 1 if the option is
+ *  configured, and a value of 0 otherwise.  These macros are always defined.
+ */
+
+
+/*----------------------------------------------------------------------
+				ISA
+  ----------------------------------------------------------------------*/
+
+#define XCHAL_HAVE_BE			0	/* big-endian byte ordering */
+#define XCHAL_HAVE_WINDOWED		0	/* windowed registers option */
+#define XCHAL_NUM_AREGS			16	/* num of physical addr regs */
+#define XCHAL_NUM_AREGS_LOG2		4	/* log2(XCHAL_NUM_AREGS) */
+#define XCHAL_MAX_INSTRUCTION_SIZE	3	/* max instr bytes (3..8) */
+#define XCHAL_HAVE_DEBUG		1	/* debug option */
+#define XCHAL_HAVE_DENSITY		1	/* 16-bit instructions */
+#define XCHAL_HAVE_LOOPS		0	/* zero-overhead loops */
+#define XCHAL_HAVE_NSA			1	/* NSA/NSAU instructions */
+#define XCHAL_HAVE_MINMAX		0	/* MIN/MAX instructions */
+#define XCHAL_HAVE_SEXT			0	/* SEXT instruction */
+#define XCHAL_HAVE_CLAMPS		0	/* CLAMPS instruction */
+#define XCHAL_HAVE_MUL16		1	/* MUL16S/MUL16U instructions */
+#define XCHAL_HAVE_MUL32		1	/* MULL instruction */
+#define XCHAL_HAVE_MUL32_HIGH		0	/* MULUH/MULSH instructions */
+#define XCHAL_HAVE_DIV32		0	/* QUOS/QUOU/REMS/REMU instructions */
+#define XCHAL_HAVE_L32R			1	/* L32R instruction */
+#define XCHAL_HAVE_ABSOLUTE_LITERALS	1	/* non-PC-rel (extended) L32R */
+#define XCHAL_HAVE_CONST16		0	/* CONST16 instruction */
+#define XCHAL_HAVE_ADDX			1	/* ADDX#/SUBX# instructions */
+#define XCHAL_HAVE_WIDE_BRANCHES	0	/* B*.W18 or B*.W15 instr's */
+#define XCHAL_HAVE_PREDICTED_BRANCHES	0	/* B[EQ/EQZ/NE/NEZ]T instr's */
+#define XCHAL_HAVE_CALL4AND12		0	/* (obsolete option) */
+#define XCHAL_HAVE_ABS			1	/* ABS instruction */
+/*#define XCHAL_HAVE_POPC		0*/	/* POPC instruction */
+/*#define XCHAL_HAVE_CRC		0*/	/* CRC instruction */
+#define XCHAL_HAVE_RELEASE_SYNC		0	/* L32AI/S32RI instructions */
+#define XCHAL_HAVE_S32C1I		0	/* S32C1I instruction */
+#define XCHAL_HAVE_SPECULATION		0	/* speculation */
+#define XCHAL_HAVE_FULL_RESET		1	/* all regs/state reset */
+#define XCHAL_NUM_CONTEXTS		1	/* */
+#define XCHAL_NUM_MISC_REGS		0	/* num of scratch regs (0..4) */
+#define XCHAL_HAVE_TAP_MASTER		0	/* JTAG TAP control instr's */
+#define XCHAL_HAVE_PRID			1	/* processor ID register */
+#define XCHAL_HAVE_EXTERN_REGS		1	/* WER/RER instructions */
+#define XCHAL_HAVE_MP_INTERRUPTS	0	/* interrupt distributor port */
+#define XCHAL_HAVE_MP_RUNSTALL		0	/* core RunStall control port */
+#define XCHAL_HAVE_THREADPTR		0	/* THREADPTR register */
+#define XCHAL_HAVE_BOOLEANS		0	/* boolean registers */
+#define XCHAL_HAVE_CP			0	/* CPENABLE reg (coprocessor) */
+#define XCHAL_CP_MAXCFG			0	/* max allowed cp id plus one */
+#define XCHAL_HAVE_MAC16		0	/* MAC16 package */
+#define XCHAL_HAVE_VECTORFPU2005	0	/* vector floating-point pkg */
+#define XCHAL_HAVE_FP			0	/* floating point pkg */
+#define XCHAL_HAVE_DFP			0	/* double precision FP pkg */
+#define XCHAL_HAVE_DFP_accel		0	/* double precision FP acceleration pkg */
+#define XCHAL_HAVE_VECTRA1		0	/* Vectra I  pkg */
+#define XCHAL_HAVE_VECTRALX		0	/* Vectra LX pkg */
+#define XCHAL_HAVE_HIFIPRO		0	/* HiFiPro Audio Engine pkg */
+#define XCHAL_HAVE_HIFI2		0	/* HiFi2 Audio Engine pkg */
+#define XCHAL_HAVE_CONNXD2		0	/* ConnX D2 pkg */
+
+
+/*----------------------------------------------------------------------
+				MISC
+  ----------------------------------------------------------------------*/
+
+#define XCHAL_NUM_WRITEBUFFER_ENTRIES	1	/* size of write buffer */
+#define XCHAL_INST_FETCH_WIDTH		4	/* instr-fetch width in bytes */
+#define XCHAL_DATA_WIDTH		4	/* data width in bytes */
+/*  In T1050, applies to selected core load and store instructions (see ISA): */
+#define XCHAL_UNALIGNED_LOAD_EXCEPTION	1	/* unaligned loads cause exc. */
+#define XCHAL_UNALIGNED_STORE_EXCEPTION	1	/* unaligned stores cause exc.*/
+#define XCHAL_UNALIGNED_LOAD_HW		0	/* unaligned loads work in hw */
+#define XCHAL_UNALIGNED_STORE_HW	0	/* unaligned stores work in hw*/
+
+#define XCHAL_SW_VERSION		800001	/* sw version of this header */
+
+#define XCHAL_CORE_ID			"lx106"	/* alphanum core name
+						   (CoreID) set in the Xtensa
+						   Processor Generator */
+
+#define XCHAL_BUILD_UNIQUE_ID		0x0002B6F6	/* 22-bit sw build ID */
+
+/*
+ *  These definitions describe the hardware targeted by this software.
+ */
+#define XCHAL_HW_CONFIGID0		0xC28CDAFA	/* ConfigID hi 32 bits*/
+#define XCHAL_HW_CONFIGID1		0x1082B6F6	/* ConfigID lo 32 bits*/
+#define XCHAL_HW_VERSION_NAME		"LX3.0.1"	/* full version name */
+#define XCHAL_HW_VERSION_MAJOR		2300	/* major ver# of targeted hw */
+#define XCHAL_HW_VERSION_MINOR		1	/* minor ver# of targeted hw */
+#define XCHAL_HW_VERSION		230001	/* major*100+minor */
+#define XCHAL_HW_REL_LX3		1
+#define XCHAL_HW_REL_LX3_0		1
+#define XCHAL_HW_REL_LX3_0_1		1
+#define XCHAL_HW_CONFIGID_RELIABLE	1
+/*  If software targets a *range* of hardware versions, these are the bounds: */
+#define XCHAL_HW_MIN_VERSION_MAJOR	2300	/* major v of earliest tgt hw */
+#define XCHAL_HW_MIN_VERSION_MINOR	1	/* minor v of earliest tgt hw */
+#define XCHAL_HW_MIN_VERSION		230001	/* earliest targeted hw */
+#define XCHAL_HW_MAX_VERSION_MAJOR	2300	/* major v of latest tgt hw */
+#define XCHAL_HW_MAX_VERSION_MINOR	1	/* minor v of latest tgt hw */
+#define XCHAL_HW_MAX_VERSION		230001	/* latest targeted hw */
+
+
+/*----------------------------------------------------------------------
+				CACHE
+  ----------------------------------------------------------------------*/
+
+#define XCHAL_ICACHE_LINESIZE		4	/* I-cache line size in bytes */
+#define XCHAL_DCACHE_LINESIZE		4	/* D-cache line size in bytes */
+#define XCHAL_ICACHE_LINEWIDTH		2	/* log2(I line size in bytes) */
+#define XCHAL_DCACHE_LINEWIDTH		2	/* log2(D line size in bytes) */
+
+#define XCHAL_ICACHE_SIZE		0	/* I-cache size in bytes or 0 */
+#define XCHAL_DCACHE_SIZE		0	/* D-cache size in bytes or 0 */
+
+#define XCHAL_DCACHE_IS_WRITEBACK	0	/* writeback feature */
+#define XCHAL_DCACHE_IS_COHERENT	0	/* MP coherence feature */
+
+#define XCHAL_HAVE_PREFETCH		0	/* PREFCTL register */
+
+
+
+
+/****************************************************************************
+    Parameters Useful for PRIVILEGED (Supervisory or Non-Virtualized) Code
+ ****************************************************************************/
+
+
+#ifndef XTENSA_HAL_NON_PRIVILEGED_ONLY
+
+/*----------------------------------------------------------------------
+				CACHE
+  ----------------------------------------------------------------------*/
+
+#define XCHAL_HAVE_PIF			1	/* any outbound PIF present */
+
+/*  If present, cache size in bytes == (ways * 2^(linewidth + setwidth)).  */
+
+/*  Number of cache sets in log2(lines per way):  */
+#define XCHAL_ICACHE_SETWIDTH		0
+#define XCHAL_DCACHE_SETWIDTH		0
+
+/*  Cache set associativity (number of ways):  */
+#define XCHAL_ICACHE_WAYS		1
+#define XCHAL_DCACHE_WAYS		1
+
+/*  Cache features:  */
+#define XCHAL_ICACHE_LINE_LOCKABLE	0
+#define XCHAL_DCACHE_LINE_LOCKABLE	0
+#define XCHAL_ICACHE_ECC_PARITY		0
+#define XCHAL_DCACHE_ECC_PARITY		0
+
+/*  Cache access size in bytes (affects operation of SICW instruction):  */
+#define XCHAL_ICACHE_ACCESS_SIZE	1
+#define XCHAL_DCACHE_ACCESS_SIZE	1
+
+/*  Number of encoded cache attr bits (see <xtensa/hal.h> for decoded bits):  */
+#define XCHAL_CA_BITS			4
+
+
+/*----------------------------------------------------------------------
+			INTERNAL I/D RAM/ROMs and XLMI
+  ----------------------------------------------------------------------*/
+
+#define XCHAL_NUM_INSTROM		1	/* number of core instr. ROMs */
+#define XCHAL_NUM_INSTRAM		2	/* number of core instr. RAMs */
+#define XCHAL_NUM_DATAROM		1	/* number of core data ROMs */
+#define XCHAL_NUM_DATARAM		2	/* number of core data RAMs */
+#define XCHAL_NUM_URAM			0	/* number of core unified RAMs*/
+#define XCHAL_NUM_XLMI			1	/* number of core XLMI ports */
+
+/*  Instruction ROM 0:  */
+#define XCHAL_INSTROM0_VADDR		0x40200000
+#define XCHAL_INSTROM0_PADDR		0x40200000
+// #define XCHAL_INSTROM0_VADDR		0x400000
+// #define XCHAL_INSTROM0_PADDR		0x400000
+#define XCHAL_INSTROM0_SIZE		1048576
+#define XCHAL_INSTROM0_ECC_PARITY	0
+
+/*  Instruction RAM 0:  */
+#define XCHAL_INSTRAM0_VADDR		0x40000000
+#define XCHAL_INSTRAM0_PADDR		0x40000000
+#define XCHAL_INSTRAM0_SIZE		1048576
+#define XCHAL_INSTRAM0_ECC_PARITY	0
+
+/*  Instruction RAM 1:  */
+#define XCHAL_INSTRAM1_VADDR		0x40100000
+#define XCHAL_INSTRAM1_PADDR		0x40100000
+#define XCHAL_INSTRAM1_SIZE		1048576
+#define XCHAL_INSTRAM1_ECC_PARITY	0
+
+/*  Data ROM 0:  */
+#define XCHAL_DATAROM0_VADDR		0x3FF40000
+#define XCHAL_DATAROM0_PADDR		0x3FF40000
+#define XCHAL_DATAROM0_SIZE		262144
+#define XCHAL_DATAROM0_ECC_PARITY	0
+
+/*  Data RAM 0:  */
+#define XCHAL_DATARAM0_VADDR		0x3FFC0000
+#define XCHAL_DATARAM0_PADDR		0x3FFC0000
+#define XCHAL_DATARAM0_SIZE		262144
+#define XCHAL_DATARAM0_ECC_PARITY	0
+
+/*  Data RAM 1:  */
+#define XCHAL_DATARAM1_VADDR		0x3FF80000
+#define XCHAL_DATARAM1_PADDR		0x3FF80000
+#define XCHAL_DATARAM1_SIZE		262144
+#define XCHAL_DATARAM1_ECC_PARITY	0
+
+/*  XLMI Port 0:  */
+#define XCHAL_XLMI0_VADDR		0x3FF00000
+#define XCHAL_XLMI0_PADDR		0x3FF00000
+#define XCHAL_XLMI0_SIZE		262144
+#define XCHAL_XLMI0_ECC_PARITY	0
+
+
+/*----------------------------------------------------------------------
+			INTERRUPTS and TIMERS
+  ----------------------------------------------------------------------*/
+
+#define XCHAL_HAVE_INTERRUPTS		1	/* interrupt option */
+#define XCHAL_HAVE_HIGHPRI_INTERRUPTS	1	/* med/high-pri. interrupts */
+#define XCHAL_HAVE_NMI			1	/* non-maskable interrupt */
+#define XCHAL_HAVE_CCOUNT		1	/* CCOUNT reg. (timer option) */
+#define XCHAL_NUM_TIMERS		1	/* number of CCOMPAREn regs */
+#define XCHAL_NUM_INTERRUPTS		15	/* number of interrupts */
+#define XCHAL_NUM_INTERRUPTS_LOG2	4	/* ceil(log2(NUM_INTERRUPTS)) */
+#define XCHAL_NUM_EXTINTERRUPTS		13	/* num of external interrupts */
+#define XCHAL_NUM_INTLEVELS		2	/* number of interrupt levels
+						   (not including level zero) */
+#define XCHAL_EXCM_LEVEL		1	/* level masked by PS.EXCM */
+	/* (always 1 in XEA1; levels 2 .. EXCM_LEVEL are "medium priority") */
+
+/*  Masks of interrupts at each interrupt level:  */
+#define XCHAL_INTLEVEL1_MASK		0x00003FFF
+#define XCHAL_INTLEVEL2_MASK		0x00000000
+#define XCHAL_INTLEVEL3_MASK		0x00004000
+#define XCHAL_INTLEVEL4_MASK		0x00000000
+#define XCHAL_INTLEVEL5_MASK		0x00000000
+#define XCHAL_INTLEVEL6_MASK		0x00000000
+#define XCHAL_INTLEVEL7_MASK		0x00000000
+
+/*  Masks of interrupts at each range 1..n of interrupt levels:  */
+#define XCHAL_INTLEVEL1_ANDBELOW_MASK	0x00003FFF
+#define XCHAL_INTLEVEL2_ANDBELOW_MASK	0x00003FFF
+#define XCHAL_INTLEVEL3_ANDBELOW_MASK	0x00007FFF
+#define XCHAL_INTLEVEL4_ANDBELOW_MASK	0x00007FFF
+#define XCHAL_INTLEVEL5_ANDBELOW_MASK	0x00007FFF
+#define XCHAL_INTLEVEL6_ANDBELOW_MASK	0x00007FFF
+#define XCHAL_INTLEVEL7_ANDBELOW_MASK	0x00007FFF
+
+/*  Level of each interrupt:  */
+#define XCHAL_INT0_LEVEL		1
+#define XCHAL_INT1_LEVEL		1
+#define XCHAL_INT2_LEVEL		1
+#define XCHAL_INT3_LEVEL		1
+#define XCHAL_INT4_LEVEL		1
+#define XCHAL_INT5_LEVEL		1
+#define XCHAL_INT6_LEVEL		1
+#define XCHAL_INT7_LEVEL		1
+#define XCHAL_INT8_LEVEL		1
+#define XCHAL_INT9_LEVEL		1
+#define XCHAL_INT10_LEVEL		1
+#define XCHAL_INT11_LEVEL		1
+#define XCHAL_INT12_LEVEL		1
+#define XCHAL_INT13_LEVEL		1
+#define XCHAL_INT14_LEVEL		3
+#define XCHAL_DEBUGLEVEL		2	/* debug interrupt level */
+#define XCHAL_HAVE_DEBUG_EXTERN_INT	1	/* OCD external db interrupt */
+#define XCHAL_NMILEVEL			3	/* NMI "level" (for use with
+						   EXCSAVE/EPS/EPC_n, RFI n) */
+
+/*  Type of each interrupt:  */
+#define XCHAL_INT0_TYPE 	XTHAL_INTTYPE_EXTERN_LEVEL
+#define XCHAL_INT1_TYPE 	XTHAL_INTTYPE_EXTERN_LEVEL
+#define XCHAL_INT2_TYPE 	XTHAL_INTTYPE_EXTERN_LEVEL
+#define XCHAL_INT3_TYPE 	XTHAL_INTTYPE_EXTERN_LEVEL
+#define XCHAL_INT4_TYPE 	XTHAL_INTTYPE_EXTERN_LEVEL
+#define XCHAL_INT5_TYPE 	XTHAL_INTTYPE_EXTERN_LEVEL
+#define XCHAL_INT6_TYPE 	XTHAL_INTTYPE_TIMER
+#define XCHAL_INT7_TYPE 	XTHAL_INTTYPE_SOFTWARE
+#define XCHAL_INT8_TYPE 	XTHAL_INTTYPE_EXTERN_EDGE
+#define XCHAL_INT9_TYPE 	XTHAL_INTTYPE_EXTERN_EDGE
+#define XCHAL_INT10_TYPE 	XTHAL_INTTYPE_EXTERN_EDGE
+#define XCHAL_INT11_TYPE 	XTHAL_INTTYPE_EXTERN_EDGE
+#define XCHAL_INT12_TYPE 	XTHAL_INTTYPE_EXTERN_EDGE
+#define XCHAL_INT13_TYPE 	XTHAL_INTTYPE_EXTERN_EDGE
+#define XCHAL_INT14_TYPE 	XTHAL_INTTYPE_NMI
+
+/*  Masks of interrupts for each type of interrupt:  */
+#define XCHAL_INTTYPE_MASK_UNCONFIGURED	0xFFFF8000
+#define XCHAL_INTTYPE_MASK_SOFTWARE	0x00000080
+#define XCHAL_INTTYPE_MASK_EXTERN_EDGE	0x00003F00
+#define XCHAL_INTTYPE_MASK_EXTERN_LEVEL	0x0000003F
+#define XCHAL_INTTYPE_MASK_TIMER	0x00000040
+#define XCHAL_INTTYPE_MASK_NMI		0x00004000
+#define XCHAL_INTTYPE_MASK_WRITE_ERROR	0x00000000
+
+/*  Interrupt numbers assigned to specific interrupt sources:  */
+#define XCHAL_TIMER0_INTERRUPT		6	/* CCOMPARE0 */
+#define XCHAL_TIMER1_INTERRUPT		XTHAL_TIMER_UNCONFIGURED
+#define XCHAL_TIMER2_INTERRUPT		XTHAL_TIMER_UNCONFIGURED
+#define XCHAL_TIMER3_INTERRUPT		XTHAL_TIMER_UNCONFIGURED
+#define XCHAL_NMI_INTERRUPT		14	/* non-maskable interrupt */
+
+/*  Interrupt numbers for levels at which only one interrupt is configured:  */
+#define XCHAL_INTLEVEL3_NUM		14
+/*  (There are many interrupts each at level(s) 1.)  */
+
+
+/*
+ *  External interrupt vectors/levels.
+ *  These macros describe how Xtensa processor interrupt numbers
+ *  (as numbered internally, eg. in INTERRUPT and INTENABLE registers)
+ *  map to external BInterrupt<n> pins, for those interrupts
+ *  configured as external (level-triggered, edge-triggered, or NMI).
+ *  See the Xtensa processor databook for more details.
+ */
+
+/*  Core interrupt numbers mapped to each EXTERNAL interrupt number:  */
+#define XCHAL_EXTINT0_NUM		0	/* (intlevel 1) */
+#define XCHAL_EXTINT1_NUM		1	/* (intlevel 1) */
+#define XCHAL_EXTINT2_NUM		2	/* (intlevel 1) */
+#define XCHAL_EXTINT3_NUM		3	/* (intlevel 1) */
+#define XCHAL_EXTINT4_NUM		4	/* (intlevel 1) */
+#define XCHAL_EXTINT5_NUM		5	/* (intlevel 1) */
+#define XCHAL_EXTINT6_NUM		8	/* (intlevel 1) */
+#define XCHAL_EXTINT7_NUM		9	/* (intlevel 1) */
+#define XCHAL_EXTINT8_NUM		10	/* (intlevel 1) */
+#define XCHAL_EXTINT9_NUM		11	/* (intlevel 1) */
+#define XCHAL_EXTINT10_NUM		12	/* (intlevel 1) */
+#define XCHAL_EXTINT11_NUM		13	/* (intlevel 1) */
+#define XCHAL_EXTINT12_NUM		14	/* (intlevel 3) */
+
+
+/*----------------------------------------------------------------------
+			EXCEPTIONS and VECTORS
+  ----------------------------------------------------------------------*/
+
+#define XCHAL_XEA_VERSION		2	/* Xtensa Exception Architecture
+						   number: 1 == XEA1 (old)
+							   2 == XEA2 (new)
+							   0 == XEAX (extern) */
+#define XCHAL_HAVE_XEA1			0	/* Exception Architecture 1 */
+#define XCHAL_HAVE_XEA2			1	/* Exception Architecture 2 */
+#define XCHAL_HAVE_XEAX			0	/* External Exception Arch. */
+#define XCHAL_HAVE_EXCEPTIONS		1	/* exception option */
+#define XCHAL_HAVE_MEM_ECC_PARITY	0	/* local memory ECC/parity */
+#define XCHAL_HAVE_VECTOR_SELECT	1	/* relocatable vectors */
+#define XCHAL_HAVE_VECBASE		1	/* relocatable vectors */
+#define XCHAL_VECBASE_RESET_VADDR	0x40000000  /* VECBASE reset value */
+#define XCHAL_VECBASE_RESET_PADDR	0x40000000
+#define XCHAL_RESET_VECBASE_OVERLAP	0
+
+#define XCHAL_RESET_VECTOR0_VADDR	0x50000000
+#define XCHAL_RESET_VECTOR0_PADDR	0x50000000
+#define XCHAL_RESET_VECTOR1_VADDR	0x40000080
+#define XCHAL_RESET_VECTOR1_PADDR	0x40000080
+#define XCHAL_RESET_VECTOR_VADDR	0x50000000
+#define XCHAL_RESET_VECTOR_PADDR	0x50000000
+
+// #define XCHAL_RESET_VECTOR0_VADDR	0x4000f8
+// #define XCHAL_RESET_VECTOR0_PADDR	0x4000f8
+// #define XCHAL_RESET_VECTOR1_VADDR	0x40000080
+// #define XCHAL_RESET_VECTOR1_PADDR	0x40000080
+// #define XCHAL_RESET_VECTOR_VADDR	0x4000f8
+// #define XCHAL_RESET_VECTOR_PADDR	0x4000f8
+
+
+#define XCHAL_USER_VECOFS		0x00000050
+#define XCHAL_USER_VECTOR_VADDR		0x40000050
+#define XCHAL_USER_VECTOR_PADDR		0x40000050
+#define XCHAL_KERNEL_VECOFS		0x00000030
+#define XCHAL_KERNEL_VECTOR_VADDR	0x40000030
+#define XCHAL_KERNEL_VECTOR_PADDR	0x40000030
+#define XCHAL_DOUBLEEXC_VECOFS		0x00000070
+#define XCHAL_DOUBLEEXC_VECTOR_VADDR	0x40000070
+#define XCHAL_DOUBLEEXC_VECTOR_PADDR	0x40000070
+#define XCHAL_INTLEVEL2_VECOFS		0x00000010
+#define XCHAL_INTLEVEL2_VECTOR_VADDR	0x40000010
+#define XCHAL_INTLEVEL2_VECTOR_PADDR	0x40000010
+#define XCHAL_DEBUG_VECOFS		XCHAL_INTLEVEL2_VECOFS
+#define XCHAL_DEBUG_VECTOR_VADDR	XCHAL_INTLEVEL2_VECTOR_VADDR
+#define XCHAL_DEBUG_VECTOR_PADDR	XCHAL_INTLEVEL2_VECTOR_PADDR
+#define XCHAL_NMI_VECOFS		0x00000020
+#define XCHAL_NMI_VECTOR_VADDR		0x40000020
+#define XCHAL_NMI_VECTOR_PADDR		0x40000020
+#define XCHAL_INTLEVEL3_VECOFS		XCHAL_NMI_VECOFS
+#define XCHAL_INTLEVEL3_VECTOR_VADDR	XCHAL_NMI_VECTOR_VADDR
+#define XCHAL_INTLEVEL3_VECTOR_PADDR	XCHAL_NMI_VECTOR_PADDR
+
+
+/*----------------------------------------------------------------------
+				DEBUG
+  ----------------------------------------------------------------------*/
+
+#define XCHAL_HAVE_OCD			1	/* OnChipDebug option */
+#define XCHAL_NUM_IBREAK		1	/* number of IBREAKn regs */
+#define XCHAL_NUM_DBREAK		1	/* number of DBREAKn regs */
+#define XCHAL_HAVE_OCD_DIR_ARRAY	0	/* faster OCD option */
+
+
+/*----------------------------------------------------------------------
+				MMU
+  ----------------------------------------------------------------------*/
+
+/*  See core-matmap.h header file for more details.  */
+
+#define XCHAL_HAVE_TLBS			1	/* inverse of HAVE_CACHEATTR */
+#define XCHAL_HAVE_SPANNING_WAY		1	/* one way maps I+D 4GB vaddr */
+#define XCHAL_SPANNING_WAY		0	/* TLB spanning way number */
+#define XCHAL_HAVE_IDENTITY_MAP		1	/* vaddr == paddr always */
+#define XCHAL_HAVE_CACHEATTR		0	/* CACHEATTR register present */
+#define XCHAL_HAVE_MIMIC_CACHEATTR	1	/* region protection */
+#define XCHAL_HAVE_XLT_CACHEATTR	0	/* region prot. w/translation */
+#define XCHAL_HAVE_PTP_MMU		0	/* full MMU (with page table
+						   [autorefill] and protection)
+						   usable for an MMU-based OS */
+/*  If none of the above last 4 are set, it's a custom TLB configuration.  */
+
+#define XCHAL_MMU_ASID_BITS		0	/* number of bits in ASIDs */
+#define XCHAL_MMU_RINGS			1	/* number of rings (1..4) */
+#define XCHAL_MMU_RING_BITS		0	/* num of bits in RING field */
+
+#endif /* !XTENSA_HAL_NON_PRIVILEGED_ONLY */
+
+
+#endif /* _XTENSA_CORE_CONFIGURATION_H */
+
diff --git a/target/xtensa/core-lx106/gdb-config.c.inc b/target/xtensa/core-lx106/gdb-config.c.inc
new file mode 100644
index 0000000000..365d6fe609
--- /dev/null
+++ b/target/xtensa/core-lx106/gdb-config.c.inc
@@ -0,0 +1,83 @@
+/* Configuration for the Xtensa architecture for GDB, the GNU debugger.
+
+   Copyright (c) 2003-2010 Tensilica Inc.
+
+   Permission is hereby granted, free of charge, to any person obtaining
+   a copy of this software and associated documentation files (the
+   "Software"), to deal in the Software without restriction, including
+   without limitation the rights to use, copy, modify, merge, publish,
+   distribute, sublicense, and/or sell copies of the Software, and to
+   permit persons to whom the Software is furnished to do so, subject to
+   the following conditions:
+
+   The above copyright notice and this permission notice shall be included
+   in all copies or substantial portions of the Software.
+
+   THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+   EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+   MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+   IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+   CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+   TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+   SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.  */
+  XTREG(  0,  0,32, 4, 4,0x0000,0x0006,-2, 8,0x0100,a0,          0,0,0,0,0,0)
+  XTREG(  1,  4,32, 4, 4,0x0001,0x0006,-2, 8,0x0100,a1,          0,0,0,0,0,0)
+  XTREG(  2,  8,32, 4, 4,0x0002,0x0006,-2, 8,0x0100,a2,          0,0,0,0,0,0)
+  XTREG(  3, 12,32, 4, 4,0x0003,0x0006,-2, 8,0x0100,a3,          0,0,0,0,0,0)
+  XTREG(  4, 16,32, 4, 4,0x0004,0x0006,-2, 8,0x0100,a4,          0,0,0,0,0,0)
+  XTREG(  5, 20,32, 4, 4,0x0005,0x0006,-2, 8,0x0100,a5,          0,0,0,0,0,0)
+  XTREG(  6, 24,32, 4, 4,0x0006,0x0006,-2, 8,0x0100,a6,          0,0,0,0,0,0)
+  XTREG(  7, 28,32, 4, 4,0x0007,0x0006,-2, 8,0x0100,a7,          0,0,0,0,0,0)
+  XTREG(  8, 32,32, 4, 4,0x0008,0x0006,-2, 8,0x0100,a8,          0,0,0,0,0,0)
+  XTREG(  9, 36,32, 4, 4,0x0009,0x0006,-2, 8,0x0100,a9,          0,0,0,0,0,0)
+  XTREG( 10, 40,32, 4, 4,0x000a,0x0006,-2, 8,0x0100,a10,         0,0,0,0,0,0)
+  XTREG( 11, 44,32, 4, 4,0x000b,0x0006,-2, 8,0x0100,a11,         0,0,0,0,0,0)
+  XTREG( 12, 48,32, 4, 4,0x000c,0x0006,-2, 8,0x0100,a12,         0,0,0,0,0,0)
+  XTREG( 13, 52,32, 4, 4,0x000d,0x0006,-2, 8,0x0100,a13,         0,0,0,0,0,0)
+  XTREG( 14, 56,32, 4, 4,0x000e,0x0006,-2, 8,0x0100,a14,         0,0,0,0,0,0)
+  XTREG( 15, 60,32, 4, 4,0x000f,0x0006,-2, 8,0x0100,a15,         0,0,0,0,0,0)
+  XTREG( 16, 64,32, 4, 4,0x0020,0x0006,-2, 9,0x0100,pc,          0,0,0,0,0,0)
+  XTREG( 17, 68, 6, 4, 4,0x0203,0x0006,-2, 2,0x1100,sar,         0,0,0,0,0,0)
+  XTREG( 18, 72,32, 4, 4,0x0205,0x0006,-2, 2,0x1100,litbase,     0,0,0,0,0,0)
+  XTREG( 19, 76,32, 4, 4,0x02b0,0x0002,-2, 2,0x1000,sr176,       0,0,0,0,0,0)
+  XTREG( 20, 80,32, 4, 4,0x02d0,0x0002,-2, 2,0x1000,sr208,       0,0,0,0,0,0)
+  XTREG( 21, 84, 6, 4, 4,0x02e6,0x0006,-2, 2,0x1100,ps,          0,0,0,0,0,0)
+  XTREG( 22, 88,32, 4, 4,0x0259,0x000d,-2, 2,0x1000,mmid,        0,0,0,0,0,0)
+  XTREG( 23, 92, 1, 4, 4,0x0260,0x0007,-2, 2,0x1000,ibreakenable,0,0,0,0,0,0)
+  XTREG( 24, 96,32, 4, 4,0x0268,0x0007,-2, 2,0x1000,ddr,         0,0,0,0,0,0)
+  XTREG( 25,100,32, 4, 4,0x0280,0x0007,-2, 2,0x1000,ibreaka0,    0,0,0,0,0,0)
+  XTREG( 26,104,32, 4, 4,0x0290,0x0007,-2, 2,0x1000,dbreaka0,    0,0,0,0,0,0)
+  XTREG( 27,108,32, 4, 4,0x02a0,0x0007,-2, 2,0x1000,dbreakc0,    0,0,0,0,0,0)
+  XTREG( 28,112,32, 4, 4,0x02b1,0x0007,-2, 2,0x1000,epc1,        0,0,0,0,0,0)
+  XTREG( 29,116,32, 4, 4,0x02b2,0x0007,-2, 2,0x1000,epc2,        0,0,0,0,0,0)
+  XTREG( 30,120,32, 4, 4,0x02b3,0x0007,-2, 2,0x1000,epc3,        0,0,0,0,0,0)
+  XTREG( 31,124,32, 4, 4,0x02c0,0x0007,-2, 2,0x1000,depc,        0,0,0,0,0,0)
+  XTREG( 32,128, 6, 4, 4,0x02c2,0x0007,-2, 2,0x1000,eps2,        0,0,0,0,0,0)
+  XTREG( 33,132, 6, 4, 4,0x02c3,0x0007,-2, 2,0x1000,eps3,        0,0,0,0,0,0)
+  XTREG( 34,136,32, 4, 4,0x02d1,0x0007,-2, 2,0x1000,excsave1,    0,0,0,0,0,0)
+  XTREG( 35,140,32, 4, 4,0x02d2,0x0007,-2, 2,0x1000,excsave2,    0,0,0,0,0,0)
+  XTREG( 36,144,32, 4, 4,0x02d3,0x0007,-2, 2,0x1000,excsave3,    0,0,0,0,0,0)
+  XTREG( 37,148,15, 4, 4,0x02e2,0x000b,-2, 2,0x1000,interrupt,   0,0,0,0,0,0)
+  XTREG( 38,152,15, 4, 4,0x02e2,0x000d,-2, 2,0x1000,intset,      0,0,0,0,0,0)
+  XTREG( 39,156,15, 4, 4,0x02e3,0x000d,-2, 2,0x1000,intclear,    0,0,0,0,0,0)
+  XTREG( 40,160,15, 4, 4,0x02e4,0x0007,-2, 2,0x1000,intenable,   0,0,0,0,0,0)
+  XTREG( 41,164,32, 4, 4,0x02e7,0x0007,-2, 2,0x1000,vecbase,     0,0,0,0,0,0)
+  XTREG( 42,168, 6, 4, 4,0x02e8,0x0007,-2, 2,0x1000,exccause,    0,0,0,0,0,0)
+  XTREG( 43,172,12, 4, 4,0x02e9,0x0003,-2, 2,0x1000,debugcause,  0,0,0,0,0,0)
+  XTREG( 44,176,32, 4, 4,0x02ea,0x000f,-2, 2,0x1000,ccount,      0,0,0,0,0,0)
+  XTREG( 45,180,32, 4, 4,0x02eb,0x0003,-2, 2,0x1000,prid,        0,0,0,0,0,0)
+  XTREG( 46,184,32, 4, 4,0x02ec,0x000f,-2, 2,0x1000,icount,      0,0,0,0,0,0)
+  XTREG( 47,188, 4, 4, 4,0x02ed,0x0007,-2, 2,0x1000,icountlevel, 0,0,0,0,0,0)
+  XTREG( 48,192,32, 4, 4,0x02ee,0x0007,-2, 2,0x1000,excvaddr,    0,0,0,0,0,0)
+  XTREG( 49,196,32, 4, 4,0x02f0,0x000f,-2, 2,0x1000,ccompare0,   0,0,0,0,0,0)
+  XTREG( 50,200, 4, 4, 4,0x2002,0x0006,-2, 6,0x1010,psintlevel,
+            0,0,&xtensa_mask0,0,0,0)
+  XTREG( 51,204, 1, 4, 4,0x2003,0x0006,-2, 6,0x1010,psum,
+            0,0,&xtensa_mask1,0,0,0)
+  XTREG( 52,208, 1, 4, 4,0x2004,0x0006,-2, 6,0x1010,psexcm,
+            0,0,&xtensa_mask2,0,0,0)
+  XTREG( 53,212,20, 4, 4,0x2005,0x0006,-2, 6,0x1010,litbaddr,
+            0,0,&xtensa_mask3,0,0,0)
+  XTREG( 54,216, 1, 4, 4,0x2006,0x0006,-2, 6,0x1010,litben,
+            0,0,&xtensa_mask4,0,0,0)
+  XTREG_END
diff --git a/target/xtensa/core-lx106/xtensa-modules.c.inc b/target/xtensa/core-lx106/xtensa-modules.c.inc
new file mode 100644
index 0000000000..f2b5efc6ec
--- /dev/null
+++ b/target/xtensa/core-lx106/xtensa-modules.c.inc
@@ -0,0 +1,7668 @@
+/* Xtensa configuration-specific ISA information.
+
+   Copyright (c) 2003-2010 Tensilica Inc.
+
+   Permission is hereby granted, free of charge, to any person obtaining
+   a copy of this software and associated documentation files (the
+   "Software"), to deal in the Software without restriction, including
+   without limitation the rights to use, copy, modify, merge, publish,
+   distribute, sublicense, and/or sell copies of the Software, and to
+   permit persons to whom the Software is furnished to do so, subject to
+   the following conditions:
+
+   The above copyright notice and this permission notice shall be included
+   in all copies or substantial portions of the Software.
+
+   THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+   EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+   MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+   IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+   CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+   TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+   SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.  */
+
+#include "xtensa-isa.h"
+#include "xtensa-isa-internal.h"
+
+
+/* Sysregs.  */
+
+static xtensa_sysreg_internal sysregs[] = {
+  { "MMID", 89, 0 },
+  { "DDR", 104, 0 },
+  { "176", 176, 0 },
+  { "208", 208, 0 },
+  { "INTERRUPT", 226, 0 },
+  { "INTCLEAR", 227, 0 },
+  { "CCOUNT", 234, 0 },
+  { "PRID", 235, 0 },
+  { "ICOUNT", 236, 0 },
+  { "CCOMPARE0", 240, 0 },
+  { "VECBASE", 231, 0 },
+  { "EPC1", 177, 0 },
+  { "EPC2", 178, 0 },
+  { "EPC3", 179, 0 },
+  { "EXCSAVE1", 209, 0 },
+  { "EXCSAVE2", 210, 0 },
+  { "EXCSAVE3", 211, 0 },
+  { "EPS2", 194, 0 },
+  { "EPS3", 195, 0 },
+  { "EXCCAUSE", 232, 0 },
+  { "DEPC", 192, 0 },
+  { "EXCVADDR", 238, 0 },
+  { "SAR", 3, 0 },
+  { "LITBASE", 5, 0 },
+  { "PS", 230, 0 },
+  { "INTENABLE", 228, 0 },
+  { "DBREAKA0", 144, 0 },
+  { "DBREAKC0", 160, 0 },
+  { "IBREAKA0", 128, 0 },
+  { "IBREAKENABLE", 96, 0 },
+  { "ICOUNTLEVEL", 237, 0 },
+  { "DEBUGCAUSE", 233, 0 }
+};
+
+#define NUM_SYSREGS 32
+#define MAX_SPECIAL_REG 240
+#define MAX_USER_REG 0
+
+
+/* Processor states.  */
+
+static xtensa_state_internal states[] = {
+  { "PC", 32, 0 },
+  { "ICOUNT", 32, 0 },
+  { "DDR", 32, 0 },
+  { "INTERRUPT", 15, 0 },
+  { "CCOUNT", 32, 0 },
+  { "XTSYNC", 1, 0 },
+  { "VECBASE", 25, 0 },
+  { "EPC1", 32, 0 },
+  { "EPC2", 32, 0 },
+  { "EPC3", 32, 0 },
+  { "EXCSAVE1", 32, 0 },
+  { "EXCSAVE2", 32, 0 },
+  { "EXCSAVE3", 32, 0 },
+  { "EPS2", 6, 0 },
+  { "EPS3", 6, 0 },
+  { "EXCCAUSE", 6, 0 },
+  { "PSINTLEVEL", 4, 0 },
+  { "PSUM", 1, 0 },
+  { "PSEXCM", 1, 0 },
+  { "DEPC", 32, 0 },
+  { "EXCVADDR", 32, 0 },
+  { "SAR", 6, 0 },
+  { "LITBADDR", 20, 0 },
+  { "LITBEN", 1, 0 },
+  { "InOCDMode", 1, 0 },
+  { "INTENABLE", 15, 0 },
+  { "DBREAKA0", 32, 0 },
+  { "DBREAKC0", 8, 0 },
+  { "IBREAKA0", 32, 0 },
+  { "IBREAKENABLE", 1, 0 },
+  { "ICOUNTLEVEL", 4, 0 },
+  { "DEBUGCAUSE", 6, 0 },
+  { "DBNUM", 4, 0 },
+  { "CCOMPARE0", 32, 0 }
+};
+
+#define NUM_STATES 34
+
+enum xtensa_state_id {
+  STATE_PC,
+  STATE_ICOUNT,
+  STATE_DDR,
+  STATE_INTERRUPT,
+  STATE_CCOUNT,
+  STATE_XTSYNC,
+  STATE_VECBASE,
+  STATE_EPC1,
+  STATE_EPC2,
+  STATE_EPC3,
+  STATE_EXCSAVE1,
+  STATE_EXCSAVE2,
+  STATE_EXCSAVE3,
+  STATE_EPS2,
+  STATE_EPS3,
+  STATE_EXCCAUSE,
+  STATE_PSINTLEVEL,
+  STATE_PSUM,
+  STATE_PSEXCM,
+  STATE_DEPC,
+  STATE_EXCVADDR,
+  STATE_SAR,
+  STATE_LITBADDR,
+  STATE_LITBEN,
+  STATE_InOCDMode,
+  STATE_INTENABLE,
+  STATE_DBREAKA0,
+  STATE_DBREAKC0,
+  STATE_IBREAKA0,
+  STATE_IBREAKENABLE,
+  STATE_ICOUNTLEVEL,
+  STATE_DEBUGCAUSE,
+  STATE_DBNUM,
+  STATE_CCOMPARE0
+};
+
+
+/* Field definitions.  */
+
+static unsigned
+Field_t_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 24) >> 28);
+  return tie_t;
+}
+
+static void
+Field_t_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf0) | (tie_t << 4);
+}
+
+static unsigned
+Field_s_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 20) >> 28);
+  return tie_t;
+}
+
+static void
+Field_s_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf00) | (tie_t << 8);
+}
+
+static unsigned
+Field_r_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 16) >> 28);
+  return tie_t;
+}
+
+static void
+Field_r_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf000) | (tie_t << 12);
+}
+
+static unsigned
+Field_op2_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 8) >> 28);
+  return tie_t;
+}
+
+static void
+Field_op2_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf00000) | (tie_t << 20);
+}
+
+static unsigned
+Field_op1_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 12) >> 28);
+  return tie_t;
+}
+
+static void
+Field_op1_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf0000) | (tie_t << 16);
+}
+
+static unsigned
+Field_op0_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 28) >> 28);
+  return tie_t;
+}
+
+static void
+Field_op0_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf) | (tie_t << 0);
+}
+
+static unsigned
+Field_m_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 2) | ((insn[0] << 24) >> 30);
+  return tie_t;
+}
+
+static void
+Field_m_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 30) >> 30;
+  insn[0] = (insn[0] & ~0xc0) | (tie_t << 6);
+}
+
+static unsigned
+Field_n_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 2) | ((insn[0] << 26) >> 30);
+  return tie_t;
+}
+
+static void
+Field_n_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 30) >> 30;
+  insn[0] = (insn[0] & ~0x30) | (tie_t << 4);
+}
+
+static unsigned
+Field_thi3_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 3) | ((insn[0] << 24) >> 29);
+  return tie_t;
+}
+
+static void
+Field_thi3_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 29) >> 29;
+  insn[0] = (insn[0] & ~0xe0) | (tie_t << 5);
+}
+
+static unsigned
+Field_sr_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 16) >> 28);
+  tie_t = (tie_t << 4) | ((insn[0] << 20) >> 28);
+  return tie_t;
+}
+
+static void
+Field_sr_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf00) | (tie_t << 8);
+  tie_t = (val << 24) >> 28;
+  insn[0] = (insn[0] & ~0xf000) | (tie_t << 12);
+}
+
+static unsigned
+Field_op0_Slot_inst16a_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 28) >> 28);
+  return tie_t;
+}
+
+static void
+Field_op0_Slot_inst16a_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf) | (tie_t << 0);
+}
+
+static unsigned
+Field_z_Slot_inst16b_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 1) | ((insn[0] << 25) >> 31);
+  return tie_t;
+}
+
+static void
+Field_z_Slot_inst16b_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 31) >> 31;
+  insn[0] = (insn[0] & ~0x40) | (tie_t << 6);
+}
+
+static unsigned
+Field_i_Slot_inst16b_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 1) | ((insn[0] << 24) >> 31);
+  return tie_t;
+}
+
+static void
+Field_i_Slot_inst16b_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 31) >> 31;
+  insn[0] = (insn[0] & ~0x80) | (tie_t << 7);
+}
+
+static unsigned
+Field_op0_Slot_inst16b_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 28) >> 28);
+  return tie_t;
+}
+
+static void
+Field_op0_Slot_inst16b_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf) | (tie_t << 0);
+}
+
+static unsigned
+Field_t_Slot_inst16b_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 24) >> 28);
+  return tie_t;
+}
+
+static void
+Field_t_Slot_inst16b_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf0) | (tie_t << 4);
+}
+
+static unsigned
+Field_r_Slot_inst16b_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 16) >> 28);
+  return tie_t;
+}
+
+static void
+Field_r_Slot_inst16b_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf000) | (tie_t << 12);
+}
+
+static unsigned
+Field_s_Slot_inst16b_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 20) >> 28);
+  return tie_t;
+}
+
+static void
+Field_s_Slot_inst16b_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf00) | (tie_t << 8);
+}
+
+static unsigned
+Field_t_Slot_inst16a_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 24) >> 28);
+  return tie_t;
+}
+
+static void
+Field_t_Slot_inst16a_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf0) | (tie_t << 4);
+}
+
+static unsigned
+Field_bbi4_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 1) | ((insn[0] << 19) >> 31);
+  return tie_t;
+}
+
+static void
+Field_bbi4_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 31) >> 31;
+  insn[0] = (insn[0] & ~0x1000) | (tie_t << 12);
+}
+
+static unsigned
+Field_bbi_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 1) | ((insn[0] << 19) >> 31);
+  tie_t = (tie_t << 4) | ((insn[0] << 24) >> 28);
+  return tie_t;
+}
+
+static void
+Field_bbi_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf0) | (tie_t << 4);
+  tie_t = (val << 27) >> 31;
+  insn[0] = (insn[0] & ~0x1000) | (tie_t << 12);
+}
+
+static unsigned
+Field_imm12_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 12) | ((insn[0] << 8) >> 20);
+  return tie_t;
+}
+
+static void
+Field_imm12_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 20) >> 20;
+  insn[0] = (insn[0] & ~0xfff000) | (tie_t << 12);
+}
+
+static unsigned
+Field_imm8_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 8) | ((insn[0] << 8) >> 24);
+  return tie_t;
+}
+
+static void
+Field_imm8_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 24) >> 24;
+  insn[0] = (insn[0] & ~0xff0000) | (tie_t << 16);
+}
+
+static unsigned
+Field_s_Slot_inst16a_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 20) >> 28);
+  return tie_t;
+}
+
+static void
+Field_s_Slot_inst16a_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf00) | (tie_t << 8);
+}
+
+static unsigned
+Field_imm12b_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 20) >> 28);
+  tie_t = (tie_t << 8) | ((insn[0] << 8) >> 24);
+  return tie_t;
+}
+
+static void
+Field_imm12b_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 24) >> 24;
+  insn[0] = (insn[0] & ~0xff0000) | (tie_t << 16);
+  tie_t = (val << 20) >> 28;
+  insn[0] = (insn[0] & ~0xf00) | (tie_t << 8);
+}
+
+static unsigned
+Field_imm16_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 16) | ((insn[0] << 8) >> 16);
+  return tie_t;
+}
+
+static void
+Field_imm16_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 16) >> 16;
+  insn[0] = (insn[0] & ~0xffff00) | (tie_t << 8);
+}
+
+static unsigned
+Field_offset_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 18) | ((insn[0] << 8) >> 14);
+  return tie_t;
+}
+
+static void
+Field_offset_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 14) >> 14;
+  insn[0] = (insn[0] & ~0xffffc0) | (tie_t << 6);
+}
+
+static unsigned
+Field_r_Slot_inst16a_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 16) >> 28);
+  return tie_t;
+}
+
+static void
+Field_r_Slot_inst16a_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf000) | (tie_t << 12);
+}
+
+static unsigned
+Field_sa4_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 1) | ((insn[0] << 11) >> 31);
+  return tie_t;
+}
+
+static void
+Field_sa4_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 31) >> 31;
+  insn[0] = (insn[0] & ~0x100000) | (tie_t << 20);
+}
+
+static unsigned
+Field_sae4_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 1) | ((insn[0] << 15) >> 31);
+  return tie_t;
+}
+
+static void
+Field_sae4_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 31) >> 31;
+  insn[0] = (insn[0] & ~0x10000) | (tie_t << 16);
+}
+
+static unsigned
+Field_sae_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 1) | ((insn[0] << 15) >> 31);
+  tie_t = (tie_t << 4) | ((insn[0] << 20) >> 28);
+  return tie_t;
+}
+
+static void
+Field_sae_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf00) | (tie_t << 8);
+  tie_t = (val << 27) >> 31;
+  insn[0] = (insn[0] & ~0x10000) | (tie_t << 16);
+}
+
+static unsigned
+Field_sal_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 1) | ((insn[0] << 11) >> 31);
+  tie_t = (tie_t << 4) | ((insn[0] << 24) >> 28);
+  return tie_t;
+}
+
+static void
+Field_sal_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf0) | (tie_t << 4);
+  tie_t = (val << 27) >> 31;
+  insn[0] = (insn[0] & ~0x100000) | (tie_t << 20);
+}
+
+static unsigned
+Field_sargt_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 1) | ((insn[0] << 11) >> 31);
+  tie_t = (tie_t << 4) | ((insn[0] << 20) >> 28);
+  return tie_t;
+}
+
+static void
+Field_sargt_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf00) | (tie_t << 8);
+  tie_t = (val << 27) >> 31;
+  insn[0] = (insn[0] & ~0x100000) | (tie_t << 20);
+}
+
+static unsigned
+Field_sas4_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 1) | ((insn[0] << 27) >> 31);
+  return tie_t;
+}
+
+static void
+Field_sas4_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 31) >> 31;
+  insn[0] = (insn[0] & ~0x10) | (tie_t << 4);
+}
+
+static unsigned
+Field_sas_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 1) | ((insn[0] << 27) >> 31);
+  tie_t = (tie_t << 4) | ((insn[0] << 20) >> 28);
+  return tie_t;
+}
+
+static void
+Field_sas_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf00) | (tie_t << 8);
+  tie_t = (val << 27) >> 31;
+  insn[0] = (insn[0] & ~0x10) | (tie_t << 4);
+}
+
+static unsigned
+Field_sr_Slot_inst16a_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 16) >> 28);
+  tie_t = (tie_t << 4) | ((insn[0] << 20) >> 28);
+  return tie_t;
+}
+
+static void
+Field_sr_Slot_inst16a_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf00) | (tie_t << 8);
+  tie_t = (val << 24) >> 28;
+  insn[0] = (insn[0] & ~0xf000) | (tie_t << 12);
+}
+
+static unsigned
+Field_sr_Slot_inst16b_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 16) >> 28);
+  tie_t = (tie_t << 4) | ((insn[0] << 20) >> 28);
+  return tie_t;
+}
+
+static void
+Field_sr_Slot_inst16b_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf00) | (tie_t << 8);
+  tie_t = (val << 24) >> 28;
+  insn[0] = (insn[0] & ~0xf000) | (tie_t << 12);
+}
+
+static unsigned
+Field_st_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 20) >> 28);
+  tie_t = (tie_t << 4) | ((insn[0] << 24) >> 28);
+  return tie_t;
+}
+
+static void
+Field_st_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf0) | (tie_t << 4);
+  tie_t = (val << 24) >> 28;
+  insn[0] = (insn[0] & ~0xf00) | (tie_t << 8);
+}
+
+static unsigned
+Field_st_Slot_inst16a_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 20) >> 28);
+  tie_t = (tie_t << 4) | ((insn[0] << 24) >> 28);
+  return tie_t;
+}
+
+static void
+Field_st_Slot_inst16a_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf0) | (tie_t << 4);
+  tie_t = (val << 24) >> 28;
+  insn[0] = (insn[0] & ~0xf00) | (tie_t << 8);
+}
+
+static unsigned
+Field_st_Slot_inst16b_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 20) >> 28);
+  tie_t = (tie_t << 4) | ((insn[0] << 24) >> 28);
+  return tie_t;
+}
+
+static void
+Field_st_Slot_inst16b_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf0) | (tie_t << 4);
+  tie_t = (val << 24) >> 28;
+  insn[0] = (insn[0] & ~0xf00) | (tie_t << 8);
+}
+
+static unsigned
+Field_imm4_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 16) >> 28);
+  return tie_t;
+}
+
+static void
+Field_imm4_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf000) | (tie_t << 12);
+}
+
+static unsigned
+Field_imm4_Slot_inst16a_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 16) >> 28);
+  return tie_t;
+}
+
+static void
+Field_imm4_Slot_inst16a_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf000) | (tie_t << 12);
+}
+
+static unsigned
+Field_imm4_Slot_inst16b_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 16) >> 28);
+  return tie_t;
+}
+
+static void
+Field_imm4_Slot_inst16b_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf000) | (tie_t << 12);
+}
+
+static unsigned
+Field_i_Slot_inst16a_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 1) | ((insn[0] << 24) >> 31);
+  return tie_t;
+}
+
+static void
+Field_i_Slot_inst16a_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 31) >> 31;
+  insn[0] = (insn[0] & ~0x80) | (tie_t << 7);
+}
+
+static unsigned
+Field_imm6lo_Slot_inst16a_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 16) >> 28);
+  return tie_t;
+}
+
+static void
+Field_imm6lo_Slot_inst16a_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf000) | (tie_t << 12);
+}
+
+static unsigned
+Field_imm6lo_Slot_inst16b_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 16) >> 28);
+  return tie_t;
+}
+
+static void
+Field_imm6lo_Slot_inst16b_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf000) | (tie_t << 12);
+}
+
+static unsigned
+Field_imm6hi_Slot_inst16a_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 2) | ((insn[0] << 26) >> 30);
+  return tie_t;
+}
+
+static void
+Field_imm6hi_Slot_inst16a_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 30) >> 30;
+  insn[0] = (insn[0] & ~0x30) | (tie_t << 4);
+}
+
+static unsigned
+Field_imm6hi_Slot_inst16b_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 2) | ((insn[0] << 26) >> 30);
+  return tie_t;
+}
+
+static void
+Field_imm6hi_Slot_inst16b_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 30) >> 30;
+  insn[0] = (insn[0] & ~0x30) | (tie_t << 4);
+}
+
+static unsigned
+Field_imm7lo_Slot_inst16a_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 16) >> 28);
+  return tie_t;
+}
+
+static void
+Field_imm7lo_Slot_inst16a_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf000) | (tie_t << 12);
+}
+
+static unsigned
+Field_imm7lo_Slot_inst16b_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 4) | ((insn[0] << 16) >> 28);
+  return tie_t;
+}
+
+static void
+Field_imm7lo_Slot_inst16b_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf000) | (tie_t << 12);
+}
+
+static unsigned
+Field_imm7hi_Slot_inst16a_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 3) | ((insn[0] << 25) >> 29);
+  return tie_t;
+}
+
+static void
+Field_imm7hi_Slot_inst16a_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 29) >> 29;
+  insn[0] = (insn[0] & ~0x70) | (tie_t << 4);
+}
+
+static unsigned
+Field_imm7hi_Slot_inst16b_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 3) | ((insn[0] << 25) >> 29);
+  return tie_t;
+}
+
+static void
+Field_imm7hi_Slot_inst16b_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 29) >> 29;
+  insn[0] = (insn[0] & ~0x70) | (tie_t << 4);
+}
+
+static unsigned
+Field_z_Slot_inst16a_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 1) | ((insn[0] << 25) >> 31);
+  return tie_t;
+}
+
+static void
+Field_z_Slot_inst16a_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 31) >> 31;
+  insn[0] = (insn[0] & ~0x40) | (tie_t << 6);
+}
+
+static unsigned
+Field_imm6_Slot_inst16a_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 2) | ((insn[0] << 26) >> 30);
+  tie_t = (tie_t << 4) | ((insn[0] << 16) >> 28);
+  return tie_t;
+}
+
+static void
+Field_imm6_Slot_inst16a_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf000) | (tie_t << 12);
+  tie_t = (val << 26) >> 30;
+  insn[0] = (insn[0] & ~0x30) | (tie_t << 4);
+}
+
+static unsigned
+Field_imm6_Slot_inst16b_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 2) | ((insn[0] << 26) >> 30);
+  tie_t = (tie_t << 4) | ((insn[0] << 16) >> 28);
+  return tie_t;
+}
+
+static void
+Field_imm6_Slot_inst16b_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf000) | (tie_t << 12);
+  tie_t = (val << 26) >> 30;
+  insn[0] = (insn[0] & ~0x30) | (tie_t << 4);
+}
+
+static unsigned
+Field_imm7_Slot_inst16a_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 3) | ((insn[0] << 25) >> 29);
+  tie_t = (tie_t << 4) | ((insn[0] << 16) >> 28);
+  return tie_t;
+}
+
+static void
+Field_imm7_Slot_inst16a_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf000) | (tie_t << 12);
+  tie_t = (val << 25) >> 29;
+  insn[0] = (insn[0] & ~0x70) | (tie_t << 4);
+}
+
+static unsigned
+Field_imm7_Slot_inst16b_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 3) | ((insn[0] << 25) >> 29);
+  tie_t = (tie_t << 4) | ((insn[0] << 16) >> 28);
+  return tie_t;
+}
+
+static void
+Field_imm7_Slot_inst16b_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 28) >> 28;
+  insn[0] = (insn[0] & ~0xf000) | (tie_t << 12);
+  tie_t = (val << 25) >> 29;
+  insn[0] = (insn[0] & ~0x70) | (tie_t << 4);
+}
+
+static unsigned
+Field_xt_wbr15_imm_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 15) | ((insn[0] << 8) >> 17);
+  return tie_t;
+}
+
+static void
+Field_xt_wbr15_imm_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 17) >> 17;
+  insn[0] = (insn[0] & ~0xfffe00) | (tie_t << 9);
+}
+
+static unsigned
+Field_xt_wbr18_imm_Slot_inst_get (const xtensa_insnbuf insn)
+{
+  unsigned tie_t = 0;
+  tie_t = (tie_t << 18) | ((insn[0] << 8) >> 14);
+  return tie_t;
+}
+
+static void
+Field_xt_wbr18_imm_Slot_inst_set (xtensa_insnbuf insn, uint32 val)
+{
+  uint32 tie_t;
+  tie_t = (val << 14) >> 14;
+  insn[0] = (insn[0] & ~0xffffc0) | (tie_t << 6);
+}
+
+static void
+Implicit_Field_set (xtensa_insnbuf insn ATTRIBUTE_UNUSED,
+		    uint32 val ATTRIBUTE_UNUSED)
+{
+  /* Do nothing.  */
+}
+
+static unsigned
+Implicit_Field_ar0_get (const xtensa_insnbuf insn ATTRIBUTE_UNUSED)
+{
+  return 0;
+}
+
+enum xtensa_field_id {
+  FIELD_t,
+  FIELD_bbi4,
+  FIELD_bbi,
+  FIELD_imm12,
+  FIELD_imm8,
+  FIELD_s,
+  FIELD_imm12b,
+  FIELD_imm16,
+  FIELD_m,
+  FIELD_n,
+  FIELD_offset,
+  FIELD_op0,
+  FIELD_op1,
+  FIELD_op2,
+  FIELD_r,
+  FIELD_sa4,
+  FIELD_sae4,
+  FIELD_sae,
+  FIELD_sal,
+  FIELD_sargt,
+  FIELD_sas4,
+  FIELD_sas,
+  FIELD_sr,
+  FIELD_st,
+  FIELD_thi3,
+  FIELD_imm4,
+  FIELD_i,
+  FIELD_imm6lo,
+  FIELD_imm6hi,
+  FIELD_imm7lo,
+  FIELD_imm7hi,
+  FIELD_z,
+  FIELD_imm6,
+  FIELD_imm7,
+  FIELD_xt_wbr15_imm,
+  FIELD_xt_wbr18_imm,
+  FIELD__ar0
+};
+
+
+/* Functional units.  */
+
+static xtensa_funcUnit_internal funcUnits[] = {
+
+};
+
+
+/* Register files.  */
+
+enum xtensa_regfile_id {
+  REGFILE_AR
+};
+
+static xtensa_regfile_internal regfiles[] = {
+  { "AR", "a", REGFILE_AR, 32, 16 }
+};
+
+
+/* Interfaces.  */
+
+static xtensa_interface_internal interfaces[] = {
+
+};
+
+
+/* Constant tables.  */
+
+/* constant table ai4c */
+static const unsigned CONST_TBL_ai4c_0[] = {
+  0xffffffff,
+  0x1,
+  0x2,
+  0x3,
+  0x4,
+  0x5,
+  0x6,
+  0x7,
+  0x8,
+  0x9,
+  0xa,
+  0xb,
+  0xc,
+  0xd,
+  0xe,
+  0xf,
+  0
+};
+
+/* constant table b4c */
+static const unsigned CONST_TBL_b4c_0[] = {
+  0xffffffff,
+  0x1,
+  0x2,
+  0x3,
+  0x4,
+  0x5,
+  0x6,
+  0x7,
+  0x8,
+  0xa,
+  0xc,
+  0x10,
+  0x20,
+  0x40,
+  0x80,
+  0x100,
+  0
+};
+
+/* constant table b4cu */
+static const unsigned CONST_TBL_b4cu_0[] = {
+  0x8000,
+  0x10000,
+  0x2,
+  0x3,
+  0x4,
+  0x5,
+  0x6,
+  0x7,
+  0x8,
+  0xa,
+  0xc,
+  0x10,
+  0x20,
+  0x40,
+  0x80,
+  0x100,
+  0
+};
+
+
+/* Instruction operands.  */
+
+static int
+Operand_art_decode (uint32 *valp ATTRIBUTE_UNUSED)
+{
+  return 0;
+}
+
+static int
+Operand_art_encode (uint32 *valp)
+{
+  int error;
+  error = (*valp & ~0xf) != 0;
+  return error;
+}
+
+static int
+Operand_ars_decode (uint32 *valp ATTRIBUTE_UNUSED)
+{
+  return 0;
+}
+
+static int
+Operand_ars_encode (uint32 *valp)
+{
+  int error;
+  error = (*valp & ~0xf) != 0;
+  return error;
+}
+
+static int
+Operand_arr_decode (uint32 *valp ATTRIBUTE_UNUSED)
+{
+  return 0;
+}
+
+static int
+Operand_arr_encode (uint32 *valp)
+{
+  int error;
+  error = (*valp & ~0xf) != 0;
+  return error;
+}
+
+static int
+Operand_ar0_decode (uint32 *valp ATTRIBUTE_UNUSED)
+{
+  return 0;
+}
+
+static int
+Operand_ar0_encode (uint32 *valp)
+{
+  int error;
+  error = (*valp & ~0xf) != 0;
+  return error;
+}
+
+static int
+Operand_soffsetx4_decode (uint32 *valp)
+{
+  unsigned soffsetx4_0, offset_0;
+  offset_0 = *valp & 0x3ffff;
+  soffsetx4_0 = 0x4 + ((((int) offset_0 << 14) >> 14) << 2);
+  *valp = soffsetx4_0;
+  return 0;
+}
+
+static int
+Operand_soffsetx4_encode (uint32 *valp)
+{
+  unsigned offset_0, soffsetx4_0;
+  soffsetx4_0 = *valp;
+  offset_0 = ((soffsetx4_0 - 0x4) >> 2) & 0x3ffff;
+  *valp = offset_0;
+  return 0;
+}
+
+static int
+Operand_soffsetx4_ator (uint32 *valp, uint32 pc)
+{
+  *valp -= (pc & ~0x3);
+  return 0;
+}
+
+static int
+Operand_soffsetx4_rtoa (uint32 *valp, uint32 pc)
+{
+  *valp += (pc & ~0x3);
+  return 0;
+}
+
+static int
+Operand_lsi4x4_decode (uint32 *valp)
+{
+  unsigned lsi4x4_0, r_0;
+  r_0 = *valp & 0xf;
+  lsi4x4_0 = r_0 << 2;
+  *valp = lsi4x4_0;
+  return 0;
+}
+
+static int
+Operand_lsi4x4_encode (uint32 *valp)
+{
+  unsigned r_0, lsi4x4_0;
+  lsi4x4_0 = *valp;
+  r_0 = ((lsi4x4_0 >> 2) & 0xf);
+  *valp = r_0;
+  return 0;
+}
+
+static int
+Operand_simm7_decode (uint32 *valp)
+{
+  unsigned simm7_0, imm7_0;
+  imm7_0 = *valp & 0x7f;
+  simm7_0 = ((((-((((imm7_0 >> 6) & 1)) & (((imm7_0 >> 5) & 1)))) & 0x1ffffff)) << 7) | imm7_0;
+  *valp = simm7_0;
+  return 0;
+}
+
+static int
+Operand_simm7_encode (uint32 *valp)
+{
+  unsigned imm7_0, simm7_0;
+  simm7_0 = *valp;
+  imm7_0 = (simm7_0 & 0x7f);
+  *valp = imm7_0;
+  return 0;
+}
+
+static int
+Operand_uimm6_decode (uint32 *valp)
+{
+  unsigned uimm6_0, imm6_0;
+  imm6_0 = *valp & 0x3f;
+  uimm6_0 = 0x4 + (((0) << 6) | imm6_0);
+  *valp = uimm6_0;
+  return 0;
+}
+
+static int
+Operand_uimm6_encode (uint32 *valp)
+{
+  unsigned imm6_0, uimm6_0;
+  uimm6_0 = *valp;
+  imm6_0 = (uimm6_0 - 0x4) & 0x3f;
+  *valp = imm6_0;
+  return 0;
+}
+
+static int
+Operand_uimm6_ator (uint32 *valp, uint32 pc)
+{
+  *valp -= pc;
+  return 0;
+}
+
+static int
+Operand_uimm6_rtoa (uint32 *valp, uint32 pc)
+{
+  *valp += pc;
+  return 0;
+}
+
+static int
+Operand_ai4const_decode (uint32 *valp)
+{
+  unsigned ai4const_0, t_0;
+  t_0 = *valp & 0xf;
+  ai4const_0 = CONST_TBL_ai4c_0[t_0 & 0xf];
+  *valp = ai4const_0;
+  return 0;
+}
+
+static int
+Operand_ai4const_encode (uint32 *valp)
+{
+  unsigned t_0, ai4const_0;
+  ai4const_0 = *valp;
+  switch (ai4const_0)
+    {
+    case 0xffffffff: t_0 = 0; break;
+    case 0x1: t_0 = 0x1; break;
+    case 0x2: t_0 = 0x2; break;
+    case 0x3: t_0 = 0x3; break;
+    case 0x4: t_0 = 0x4; break;
+    case 0x5: t_0 = 0x5; break;
+    case 0x6: t_0 = 0x6; break;
+    case 0x7: t_0 = 0x7; break;
+    case 0x8: t_0 = 0x8; break;
+    case 0x9: t_0 = 0x9; break;
+    case 0xa: t_0 = 0xa; break;
+    case 0xb: t_0 = 0xb; break;
+    case 0xc: t_0 = 0xc; break;
+    case 0xd: t_0 = 0xd; break;
+    case 0xe: t_0 = 0xe; break;
+    default: t_0 = 0xf; break;
+    }
+  *valp = t_0;
+  return 0;
+}
+
+static int
+Operand_b4const_decode (uint32 *valp)
+{
+  unsigned b4const_0, r_0;
+  r_0 = *valp & 0xf;
+  b4const_0 = CONST_TBL_b4c_0[r_0 & 0xf];
+  *valp = b4const_0;
+  return 0;
+}
+
+static int
+Operand_b4const_encode (uint32 *valp)
+{
+  unsigned r_0, b4const_0;
+  b4const_0 = *valp;
+  switch (b4const_0)
+    {
+    case 0xffffffff: r_0 = 0; break;
+    case 0x1: r_0 = 0x1; break;
+    case 0x2: r_0 = 0x2; break;
+    case 0x3: r_0 = 0x3; break;
+    case 0x4: r_0 = 0x4; break;
+    case 0x5: r_0 = 0x5; break;
+    case 0x6: r_0 = 0x6; break;
+    case 0x7: r_0 = 0x7; break;
+    case 0x8: r_0 = 0x8; break;
+    case 0xa: r_0 = 0x9; break;
+    case 0xc: r_0 = 0xa; break;
+    case 0x10: r_0 = 0xb; break;
+    case 0x20: r_0 = 0xc; break;
+    case 0x40: r_0 = 0xd; break;
+    case 0x80: r_0 = 0xe; break;
+    default: r_0 = 0xf; break;
+    }
+  *valp = r_0;
+  return 0;
+}
+
+static int
+Operand_b4constu_decode (uint32 *valp)
+{
+  unsigned b4constu_0, r_0;
+  r_0 = *valp & 0xf;
+  b4constu_0 = CONST_TBL_b4cu_0[r_0 & 0xf];
+  *valp = b4constu_0;
+  return 0;
+}
+
+static int
+Operand_b4constu_encode (uint32 *valp)
+{
+  unsigned r_0, b4constu_0;
+  b4constu_0 = *valp;
+  switch (b4constu_0)
+    {
+    case 0x8000: r_0 = 0; break;
+    case 0x10000: r_0 = 0x1; break;
+    case 0x2: r_0 = 0x2; break;
+    case 0x3: r_0 = 0x3; break;
+    case 0x4: r_0 = 0x4; break;
+    case 0x5: r_0 = 0x5; break;
+    case 0x6: r_0 = 0x6; break;
+    case 0x7: r_0 = 0x7; break;
+    case 0x8: r_0 = 0x8; break;
+    case 0xa: r_0 = 0x9; break;
+    case 0xc: r_0 = 0xa; break;
+    case 0x10: r_0 = 0xb; break;
+    case 0x20: r_0 = 0xc; break;
+    case 0x40: r_0 = 0xd; break;
+    case 0x80: r_0 = 0xe; break;
+    default: r_0 = 0xf; break;
+    }
+  *valp = r_0;
+  return 0;
+}
+
+static int
+Operand_uimm8_decode (uint32 *valp)
+{
+  unsigned uimm8_0, imm8_0;
+  imm8_0 = *valp & 0xff;
+  uimm8_0 = imm8_0;
+  *valp = uimm8_0;
+  return 0;
+}
+
+static int
+Operand_uimm8_encode (uint32 *valp)
+{
+  unsigned imm8_0, uimm8_0;
+  uimm8_0 = *valp;
+  imm8_0 = (uimm8_0 & 0xff);
+  *valp = imm8_0;
+  return 0;
+}
+
+static int
+Operand_uimm8x2_decode (uint32 *valp)
+{
+  unsigned uimm8x2_0, imm8_0;
+  imm8_0 = *valp & 0xff;
+  uimm8x2_0 = imm8_0 << 1;
+  *valp = uimm8x2_0;
+  return 0;
+}
+
+static int
+Operand_uimm8x2_encode (uint32 *valp)
+{
+  unsigned imm8_0, uimm8x2_0;
+  uimm8x2_0 = *valp;
+  imm8_0 = ((uimm8x2_0 >> 1) & 0xff);
+  *valp = imm8_0;
+  return 0;
+}
+
+static int
+Operand_uimm8x4_decode (uint32 *valp)
+{
+  unsigned uimm8x4_0, imm8_0;
+  imm8_0 = *valp & 0xff;
+  uimm8x4_0 = imm8_0 << 2;
+  *valp = uimm8x4_0;
+  return 0;
+}
+
+static int
+Operand_uimm8x4_encode (uint32 *valp)
+{
+  unsigned imm8_0, uimm8x4_0;
+  uimm8x4_0 = *valp;
+  imm8_0 = ((uimm8x4_0 >> 2) & 0xff);
+  *valp = imm8_0;
+  return 0;
+}
+
+static int
+Operand_uimm4x16_decode (uint32 *valp)
+{
+  unsigned uimm4x16_0, op2_0;
+  op2_0 = *valp & 0xf;
+  uimm4x16_0 = op2_0 << 4;
+  *valp = uimm4x16_0;
+  return 0;
+}
+
+static int
+Operand_uimm4x16_encode (uint32 *valp)
+{
+  unsigned op2_0, uimm4x16_0;
+  uimm4x16_0 = *valp;
+  op2_0 = ((uimm4x16_0 >> 4) & 0xf);
+  *valp = op2_0;
+  return 0;
+}
+
+static int
+Operand_simm8_decode (uint32 *valp)
+{
+  unsigned simm8_0, imm8_0;
+  imm8_0 = *valp & 0xff;
+  simm8_0 = ((int) imm8_0 << 24) >> 24;
+  *valp = simm8_0;
+  return 0;
+}
+
+static int
+Operand_simm8_encode (uint32 *valp)
+{
+  unsigned imm8_0, simm8_0;
+  simm8_0 = *valp;
+  imm8_0 = (simm8_0 & 0xff);
+  *valp = imm8_0;
+  return 0;
+}
+
+static int
+Operand_simm8x256_decode (uint32 *valp)
+{
+  unsigned simm8x256_0, imm8_0;
+  imm8_0 = *valp & 0xff;
+  simm8x256_0 = (((int) imm8_0 << 24) >> 24) << 8;
+  *valp = simm8x256_0;
+  return 0;
+}
+
+static int
+Operand_simm8x256_encode (uint32 *valp)
+{
+  unsigned imm8_0, simm8x256_0;
+  simm8x256_0 = *valp;
+  imm8_0 = ((simm8x256_0 >> 8) & 0xff);
+  *valp = imm8_0;
+  return 0;
+}
+
+static int
+Operand_simm12b_decode (uint32 *valp)
+{
+  unsigned simm12b_0, imm12b_0;
+  imm12b_0 = *valp & 0xfff;
+  simm12b_0 = ((int) imm12b_0 << 20) >> 20;
+  *valp = simm12b_0;
+  return 0;
+}
+
+static int
+Operand_simm12b_encode (uint32 *valp)
+{
+  unsigned imm12b_0, simm12b_0;
+  simm12b_0 = *valp;
+  imm12b_0 = (simm12b_0 & 0xfff);
+  *valp = imm12b_0;
+  return 0;
+}
+
+static int
+Operand_msalp32_decode (uint32 *valp)
+{
+  unsigned msalp32_0, sal_0;
+  sal_0 = *valp & 0x1f;
+  msalp32_0 = 0x20 - sal_0;
+  *valp = msalp32_0;
+  return 0;
+}
+
+static int
+Operand_msalp32_encode (uint32 *valp)
+{
+  unsigned sal_0, msalp32_0;
+  msalp32_0 = *valp;
+  sal_0 = (0x20 - msalp32_0) & 0x1f;
+  *valp = sal_0;
+  return 0;
+}
+
+static int
+Operand_op2p1_decode (uint32 *valp)
+{
+  unsigned op2p1_0, op2_0;
+  op2_0 = *valp & 0xf;
+  op2p1_0 = op2_0 + 0x1;
+  *valp = op2p1_0;
+  return 0;
+}
+
+static int
+Operand_op2p1_encode (uint32 *valp)
+{
+  unsigned op2_0, op2p1_0;
+  op2p1_0 = *valp;
+  op2_0 = (op2p1_0 - 0x1) & 0xf;
+  *valp = op2_0;
+  return 0;
+}
+
+static int
+Operand_label8_decode (uint32 *valp)
+{
+  unsigned label8_0, imm8_0;
+  imm8_0 = *valp & 0xff;
+  label8_0 = 0x4 + (((int) imm8_0 << 24) >> 24);
+  *valp = label8_0;
+  return 0;
+}
+
+static int
+Operand_label8_encode (uint32 *valp)
+{
+  unsigned imm8_0, label8_0;
+  label8_0 = *valp;
+  imm8_0 = (label8_0 - 0x4) & 0xff;
+  *valp = imm8_0;
+  return 0;
+}
+
+static int
+Operand_label8_ator (uint32 *valp, uint32 pc)
+{
+  *valp -= pc;
+  return 0;
+}
+
+static int
+Operand_label8_rtoa (uint32 *valp, uint32 pc)
+{
+  *valp += pc;
+  return 0;
+}
+
+static int
+Operand_label12_decode (uint32 *valp)
+{
+  unsigned label12_0, imm12_0;
+  imm12_0 = *valp & 0xfff;
+  label12_0 = 0x4 + (((int) imm12_0 << 20) >> 20);
+  *valp = label12_0;
+  return 0;
+}
+
+static int
+Operand_label12_encode (uint32 *valp)
+{
+  unsigned imm12_0, label12_0;
+  label12_0 = *valp;
+  imm12_0 = (label12_0 - 0x4) & 0xfff;
+  *valp = imm12_0;
+  return 0;
+}
+
+static int
+Operand_label12_ator (uint32 *valp, uint32 pc)
+{
+  *valp -= pc;
+  return 0;
+}
+
+static int
+Operand_label12_rtoa (uint32 *valp, uint32 pc)
+{
+  *valp += pc;
+  return 0;
+}
+
+static int
+Operand_soffset_decode (uint32 *valp)
+{
+  unsigned soffset_0, offset_0;
+  offset_0 = *valp & 0x3ffff;
+  soffset_0 = 0x4 + (((int) offset_0 << 14) >> 14);
+  *valp = soffset_0;
+  return 0;
+}
+
+static int
+Operand_soffset_encode (uint32 *valp)
+{
+  unsigned offset_0, soffset_0;
+  soffset_0 = *valp;
+  offset_0 = (soffset_0 - 0x4) & 0x3ffff;
+  *valp = offset_0;
+  return 0;
+}
+
+static int
+Operand_soffset_ator (uint32 *valp, uint32 pc)
+{
+  *valp -= pc;
+  return 0;
+}
+
+static int
+Operand_soffset_rtoa (uint32 *valp, uint32 pc)
+{
+  *valp += pc;
+  return 0;
+}
+
+static int
+Operand_uimm16x4_decode (uint32 *valp)
+{
+  unsigned uimm16x4_0, imm16_0;
+  imm16_0 = *valp & 0xffff;
+  uimm16x4_0 = (((0xffff) << 16) | imm16_0) << 2;
+  *valp = uimm16x4_0;
+  return 0;
+}
+
+static int
+Operand_uimm16x4_encode (uint32 *valp)
+{
+  unsigned imm16_0, uimm16x4_0;
+  uimm16x4_0 = *valp;
+  imm16_0 = (uimm16x4_0 >> 2) & 0xffff;
+  *valp = imm16_0;
+  return 0;
+}
+
+static int
+Operand_uimm16x4_ator (uint32 *valp, uint32 pc)
+{
+  *valp -= ((pc + 3) & ~0x3);
+  return 0;
+}
+
+static int
+Operand_uimm16x4_rtoa (uint32 *valp, uint32 pc)
+{
+  *valp += ((pc + 3) & ~0x3);
+  return 0;
+}
+
+static int
+Operand_immt_decode (uint32 *valp)
+{
+  unsigned immt_0, t_0;
+  t_0 = *valp & 0xf;
+  immt_0 = t_0;
+  *valp = immt_0;
+  return 0;
+}
+
+static int
+Operand_immt_encode (uint32 *valp)
+{
+  unsigned t_0, immt_0;
+  immt_0 = *valp;
+  t_0 = immt_0 & 0xf;
+  *valp = t_0;
+  return 0;
+}
+
+static int
+Operand_imms_decode (uint32 *valp)
+{
+  unsigned imms_0, s_0;
+  s_0 = *valp & 0xf;
+  imms_0 = s_0;
+  *valp = imms_0;
+  return 0;
+}
+
+static int
+Operand_imms_encode (uint32 *valp)
+{
+  unsigned s_0, imms_0;
+  imms_0 = *valp;
+  s_0 = imms_0 & 0xf;
+  *valp = s_0;
+  return 0;
+}
+
+static int
+Operand_xt_wbr15_label_decode (uint32 *valp)
+{
+  unsigned xt_wbr15_label_0, xt_wbr15_imm_0;
+  xt_wbr15_imm_0 = *valp & 0x7fff;
+  xt_wbr15_label_0 = 0x4 + (((int) xt_wbr15_imm_0 << 17) >> 17);
+  *valp = xt_wbr15_label_0;
+  return 0;
+}
+
+static int
+Operand_xt_wbr15_label_encode (uint32 *valp)
+{
+  unsigned xt_wbr15_imm_0, xt_wbr15_label_0;
+  xt_wbr15_label_0 = *valp;
+  xt_wbr15_imm_0 = (xt_wbr15_label_0 - 0x4) & 0x7fff;
+  *valp = xt_wbr15_imm_0;
+  return 0;
+}
+
+static int
+Operand_xt_wbr15_label_ator (uint32 *valp, uint32 pc)
+{
+  *valp -= pc;
+  return 0;
+}
+
+static int
+Operand_xt_wbr15_label_rtoa (uint32 *valp, uint32 pc)
+{
+  *valp += pc;
+  return 0;
+}
+
+static int
+Operand_xt_wbr18_label_decode (uint32 *valp)
+{
+  unsigned xt_wbr18_label_0, xt_wbr18_imm_0;
+  xt_wbr18_imm_0 = *valp & 0x3ffff;
+  xt_wbr18_label_0 = 0x4 + (((int) xt_wbr18_imm_0 << 14) >> 14);
+  *valp = xt_wbr18_label_0;
+  return 0;
+}
+
+static int
+Operand_xt_wbr18_label_encode (uint32 *valp)
+{
+  unsigned xt_wbr18_imm_0, xt_wbr18_label_0;
+  xt_wbr18_label_0 = *valp;
+  xt_wbr18_imm_0 = (xt_wbr18_label_0 - 0x4) & 0x3ffff;
+  *valp = xt_wbr18_imm_0;
+  return 0;
+}
+
+static int
+Operand_xt_wbr18_label_ator (uint32 *valp, uint32 pc)
+{
+  *valp -= pc;
+  return 0;
+}
+
+static int
+Operand_xt_wbr18_label_rtoa (uint32 *valp, uint32 pc)
+{
+  *valp += pc;
+  return 0;
+}
+
+static xtensa_operand_internal operands[] = {
+  { "art", FIELD_t, REGFILE_AR, 1,
+    XTENSA_OPERAND_IS_REGISTER,
+    Operand_art_encode, Operand_art_decode,
+    0, 0 },
+  { "ars", FIELD_s, REGFILE_AR, 1,
+    XTENSA_OPERAND_IS_REGISTER,
+    Operand_ars_encode, Operand_ars_decode,
+    0, 0 },
+  { "*ars_invisible", FIELD_s, REGFILE_AR, 1,
+    XTENSA_OPERAND_IS_REGISTER | XTENSA_OPERAND_IS_INVISIBLE,
+    Operand_ars_encode, Operand_ars_decode,
+    0, 0 },
+  { "arr", FIELD_r, REGFILE_AR, 1,
+    XTENSA_OPERAND_IS_REGISTER,
+    Operand_arr_encode, Operand_arr_decode,
+    0, 0 },
+  { "ar0", FIELD__ar0, REGFILE_AR, 1,
+    XTENSA_OPERAND_IS_REGISTER | XTENSA_OPERAND_IS_INVISIBLE,
+    Operand_ar0_encode, Operand_ar0_decode,
+    0, 0 },
+  { "soffsetx4", FIELD_offset, -1, 0,
+    XTENSA_OPERAND_IS_PCRELATIVE,
+    Operand_soffsetx4_encode, Operand_soffsetx4_decode,
+    Operand_soffsetx4_ator, Operand_soffsetx4_rtoa },
+  { "lsi4x4", FIELD_r, -1, 0,
+    0,
+    Operand_lsi4x4_encode, Operand_lsi4x4_decode,
+    0, 0 },
+  { "simm7", FIELD_imm7, -1, 0,
+    0,
+    Operand_simm7_encode, Operand_simm7_decode,
+    0, 0 },
+  { "uimm6", FIELD_imm6, -1, 0,
+    XTENSA_OPERAND_IS_PCRELATIVE,
+    Operand_uimm6_encode, Operand_uimm6_decode,
+    Operand_uimm6_ator, Operand_uimm6_rtoa },
+  { "ai4const", FIELD_t, -1, 0,
+    0,
+    Operand_ai4const_encode, Operand_ai4const_decode,
+    0, 0 },
+  { "b4const", FIELD_r, -1, 0,
+    0,
+    Operand_b4const_encode, Operand_b4const_decode,
+    0, 0 },
+  { "b4constu", FIELD_r, -1, 0,
+    0,
+    Operand_b4constu_encode, Operand_b4constu_decode,
+    0, 0 },
+  { "uimm8", FIELD_imm8, -1, 0,
+    0,
+    Operand_uimm8_encode, Operand_uimm8_decode,
+    0, 0 },
+  { "uimm8x2", FIELD_imm8, -1, 0,
+    0,
+    Operand_uimm8x2_encode, Operand_uimm8x2_decode,
+    0, 0 },
+  { "uimm8x4", FIELD_imm8, -1, 0,
+    0,
+    Operand_uimm8x4_encode, Operand_uimm8x4_decode,
+    0, 0 },
+  { "uimm4x16", FIELD_op2, -1, 0,
+    0,
+    Operand_uimm4x16_encode, Operand_uimm4x16_decode,
+    0, 0 },
+  { "simm8", FIELD_imm8, -1, 0,
+    0,
+    Operand_simm8_encode, Operand_simm8_decode,
+    0, 0 },
+  { "simm8x256", FIELD_imm8, -1, 0,
+    0,
+    Operand_simm8x256_encode, Operand_simm8x256_decode,
+    0, 0 },
+  { "simm12b", FIELD_imm12b, -1, 0,
+    0,
+    Operand_simm12b_encode, Operand_simm12b_decode,
+    0, 0 },
+  { "msalp32", FIELD_sal, -1, 0,
+    0,
+    Operand_msalp32_encode, Operand_msalp32_decode,
+    0, 0 },
+  { "op2p1", FIELD_op2, -1, 0,
+    0,
+    Operand_op2p1_encode, Operand_op2p1_decode,
+    0, 0 },
+  { "label8", FIELD_imm8, -1, 0,
+    XTENSA_OPERAND_IS_PCRELATIVE,
+    Operand_label8_encode, Operand_label8_decode,
+    Operand_label8_ator, Operand_label8_rtoa },
+  { "label12", FIELD_imm12, -1, 0,
+    XTENSA_OPERAND_IS_PCRELATIVE,
+    Operand_label12_encode, Operand_label12_decode,
+    Operand_label12_ator, Operand_label12_rtoa },
+  { "soffset", FIELD_offset, -1, 0,
+    XTENSA_OPERAND_IS_PCRELATIVE,
+    Operand_soffset_encode, Operand_soffset_decode,
+    Operand_soffset_ator, Operand_soffset_rtoa },
+  { "uimm16x4", FIELD_imm16, -1, 0,
+    XTENSA_OPERAND_IS_PCRELATIVE,
+    Operand_uimm16x4_encode, Operand_uimm16x4_decode,
+    Operand_uimm16x4_ator, Operand_uimm16x4_rtoa },
+  { "immt", FIELD_t, -1, 0,
+    0,
+    Operand_immt_encode, Operand_immt_decode,
+    0, 0 },
+  { "imms", FIELD_s, -1, 0,
+    0,
+    Operand_imms_encode, Operand_imms_decode,
+    0, 0 },
+  { "xt_wbr15_label", FIELD_xt_wbr15_imm, -1, 0,
+    XTENSA_OPERAND_IS_PCRELATIVE,
+    Operand_xt_wbr15_label_encode, Operand_xt_wbr15_label_decode,
+    Operand_xt_wbr15_label_ator, Operand_xt_wbr15_label_rtoa },
+  { "xt_wbr18_label", FIELD_xt_wbr18_imm, -1, 0,
+    XTENSA_OPERAND_IS_PCRELATIVE,
+    Operand_xt_wbr18_label_encode, Operand_xt_wbr18_label_decode,
+    Operand_xt_wbr18_label_ator, Operand_xt_wbr18_label_rtoa },
+  { "t", FIELD_t, -1, 0, 0, 0, 0, 0, 0 },
+  { "bbi4", FIELD_bbi4, -1, 0, 0, 0, 0, 0, 0 },
+  { "bbi", FIELD_bbi, -1, 0, 0, 0, 0, 0, 0 },
+  { "imm12", FIELD_imm12, -1, 0, 0, 0, 0, 0, 0 },
+  { "imm8", FIELD_imm8, -1, 0, 0, 0, 0, 0, 0 },
+  { "s", FIELD_s, -1, 0, 0, 0, 0, 0, 0 },
+  { "imm12b", FIELD_imm12b, -1, 0, 0, 0, 0, 0, 0 },
+  { "imm16", FIELD_imm16, -1, 0, 0, 0, 0, 0, 0 },
+  { "m", FIELD_m, -1, 0, 0, 0, 0, 0, 0 },
+  { "n", FIELD_n, -1, 0, 0, 0, 0, 0, 0 },
+  { "offset", FIELD_offset, -1, 0, 0, 0, 0, 0, 0 },
+  { "op0", FIELD_op0, -1, 0, 0, 0, 0, 0, 0 },
+  { "op1", FIELD_op1, -1, 0, 0, 0, 0, 0, 0 },
+  { "op2", FIELD_op2, -1, 0, 0, 0, 0, 0, 0 },
+  { "r", FIELD_r, -1, 0, 0, 0, 0, 0, 0 },
+  { "sa4", FIELD_sa4, -1, 0, 0, 0, 0, 0, 0 },
+  { "sae4", FIELD_sae4, -1, 0, 0, 0, 0, 0, 0 },
+  { "sae", FIELD_sae, -1, 0, 0, 0, 0, 0, 0 },
+  { "sal", FIELD_sal, -1, 0, 0, 0, 0, 0, 0 },
+  { "sargt", FIELD_sargt, -1, 0, 0, 0, 0, 0, 0 },
+  { "sas4", FIELD_sas4, -1, 0, 0, 0, 0, 0, 0 },
+  { "sas", FIELD_sas, -1, 0, 0, 0, 0, 0, 0 },
+  { "sr", FIELD_sr, -1, 0, 0, 0, 0, 0, 0 },
+  { "st", FIELD_st, -1, 0, 0, 0, 0, 0, 0 },
+  { "thi3", FIELD_thi3, -1, 0, 0, 0, 0, 0, 0 },
+  { "imm4", FIELD_imm4, -1, 0, 0, 0, 0, 0, 0 },
+  { "i", FIELD_i, -1, 0, 0, 0, 0, 0, 0 },
+  { "imm6lo", FIELD_imm6lo, -1, 0, 0, 0, 0, 0, 0 },
+  { "imm6hi", FIELD_imm6hi, -1, 0, 0, 0, 0, 0, 0 },
+  { "imm7lo", FIELD_imm7lo, -1, 0, 0, 0, 0, 0, 0 },
+  { "imm7hi", FIELD_imm7hi, -1, 0, 0, 0, 0, 0, 0 },
+  { "z", FIELD_z, -1, 0, 0, 0, 0, 0, 0 },
+  { "imm6", FIELD_imm6, -1, 0, 0, 0, 0, 0, 0 },
+  { "imm7", FIELD_imm7, -1, 0, 0, 0, 0, 0, 0 },
+  { "xt_wbr15_imm", FIELD_xt_wbr15_imm, -1, 0, 0, 0, 0, 0, 0 },
+  { "xt_wbr18_imm", FIELD_xt_wbr18_imm, -1, 0, 0, 0, 0, 0, 0 }
+};
+
+enum xtensa_operand_id {
+  OPERAND_art,
+  OPERAND_ars,
+  OPERAND__ars_invisible,
+  OPERAND_arr,
+  OPERAND_ar0,
+  OPERAND_soffsetx4,
+  OPERAND_lsi4x4,
+  OPERAND_simm7,
+  OPERAND_uimm6,
+  OPERAND_ai4const,
+  OPERAND_b4const,
+  OPERAND_b4constu,
+  OPERAND_uimm8,
+  OPERAND_uimm8x2,
+  OPERAND_uimm8x4,
+  OPERAND_uimm4x16,
+  OPERAND_simm8,
+  OPERAND_simm8x256,
+  OPERAND_simm12b,
+  OPERAND_msalp32,
+  OPERAND_op2p1,
+  OPERAND_label8,
+  OPERAND_label12,
+  OPERAND_soffset,
+  OPERAND_uimm16x4,
+  OPERAND_immt,
+  OPERAND_imms,
+  OPERAND_xt_wbr15_label,
+  OPERAND_xt_wbr18_label,
+  OPERAND_t,
+  OPERAND_bbi4,
+  OPERAND_bbi,
+  OPERAND_imm12,
+  OPERAND_imm8,
+  OPERAND_s,
+  OPERAND_imm12b,
+  OPERAND_imm16,
+  OPERAND_m,
+  OPERAND_n,
+  OPERAND_offset,
+  OPERAND_op0,
+  OPERAND_op1,
+  OPERAND_op2,
+  OPERAND_r,
+  OPERAND_sa4,
+  OPERAND_sae4,
+  OPERAND_sae,
+  OPERAND_sal,
+  OPERAND_sargt,
+  OPERAND_sas4,
+  OPERAND_sas,
+  OPERAND_sr,
+  OPERAND_st,
+  OPERAND_thi3,
+  OPERAND_imm4,
+  OPERAND_i,
+  OPERAND_imm6lo,
+  OPERAND_imm6hi,
+  OPERAND_imm7lo,
+  OPERAND_imm7hi,
+  OPERAND_z,
+  OPERAND_imm6,
+  OPERAND_imm7,
+  OPERAND_xt_wbr15_imm,
+  OPERAND_xt_wbr18_imm
+};
+
+
+/* Iclass table.  */
+
+static xtensa_arg_internal Iclass_xt_iclass_rfe_stateArgs[] = {
+  { { STATE_PSEXCM }, 'o' },
+  { { STATE_EPC1 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rfde_stateArgs[] = {
+  { { STATE_DEPC }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_add_n_args[] = {
+  { { OPERAND_arr }, 'o' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_addi_n_args[] = {
+  { { OPERAND_arr }, 'o' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_ai4const }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_bz6_args[] = {
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_uimm6 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_loadi4_args[] = {
+  { { OPERAND_art }, 'o' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_lsi4x4 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_mov_n_args[] = {
+  { { OPERAND_art }, 'o' },
+  { { OPERAND_ars }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_movi_n_args[] = {
+  { { OPERAND_ars }, 'o' },
+  { { OPERAND_simm7 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_retn_args[] = {
+  { { OPERAND__ars_invisible }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_storei4_args[] = {
+  { { OPERAND_art }, 'i' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_lsi4x4 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_addi_args[] = {
+  { { OPERAND_art }, 'o' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_simm8 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_addmi_args[] = {
+  { { OPERAND_art }, 'o' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_simm8x256 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_addsub_args[] = {
+  { { OPERAND_arr }, 'o' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_bit_args[] = {
+  { { OPERAND_arr }, 'o' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_bsi8_args[] = {
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_b4const }, 'i' },
+  { { OPERAND_label8 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_bsi8b_args[] = {
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_bbi }, 'i' },
+  { { OPERAND_label8 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_bsi8u_args[] = {
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_b4constu }, 'i' },
+  { { OPERAND_label8 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_bst8_args[] = {
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_art }, 'i' },
+  { { OPERAND_label8 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_bsz12_args[] = {
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_label12 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_call0_args[] = {
+  { { OPERAND_soffsetx4 }, 'i' },
+  { { OPERAND_ar0 }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_callx0_args[] = {
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_ar0 }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_exti_args[] = {
+  { { OPERAND_arr }, 'o' },
+  { { OPERAND_art }, 'i' },
+  { { OPERAND_sae }, 'i' },
+  { { OPERAND_op2p1 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_jump_args[] = {
+  { { OPERAND_soffset }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_jumpx_args[] = {
+  { { OPERAND_ars }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_l16ui_args[] = {
+  { { OPERAND_art }, 'o' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_uimm8x2 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_l16si_args[] = {
+  { { OPERAND_art }, 'o' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_uimm8x2 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_l32i_args[] = {
+  { { OPERAND_art }, 'o' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_uimm8x4 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_l32r_args[] = {
+  { { OPERAND_art }, 'o' },
+  { { OPERAND_uimm16x4 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_l32r_stateArgs[] = {
+  { { STATE_LITBADDR }, 'i' },
+  { { STATE_LITBEN }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_l8i_args[] = {
+  { { OPERAND_art }, 'o' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_uimm8 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_movi_args[] = {
+  { { OPERAND_art }, 'o' },
+  { { OPERAND_simm12b }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_movz_args[] = {
+  { { OPERAND_arr }, 'm' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_neg_args[] = {
+  { { OPERAND_arr }, 'o' },
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_return_args[] = {
+  { { OPERAND__ars_invisible }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_s16i_args[] = {
+  { { OPERAND_art }, 'i' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_uimm8x2 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_s32i_args[] = {
+  { { OPERAND_art }, 'i' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_uimm8x4 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_s8i_args[] = {
+  { { OPERAND_art }, 'i' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_uimm8 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_sar_args[] = {
+  { { OPERAND_ars }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_sar_stateArgs[] = {
+  { { STATE_SAR }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_sari_args[] = {
+  { { OPERAND_sas }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_sari_stateArgs[] = {
+  { { STATE_SAR }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_shifts_args[] = {
+  { { OPERAND_arr }, 'o' },
+  { { OPERAND_ars }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_shifts_stateArgs[] = {
+  { { STATE_SAR }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_shiftst_args[] = {
+  { { OPERAND_arr }, 'o' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_shiftst_stateArgs[] = {
+  { { STATE_SAR }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_shiftt_args[] = {
+  { { OPERAND_arr }, 'o' },
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_shiftt_stateArgs[] = {
+  { { STATE_SAR }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_slli_args[] = {
+  { { OPERAND_arr }, 'o' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_msalp32 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_srai_args[] = {
+  { { OPERAND_arr }, 'o' },
+  { { OPERAND_art }, 'i' },
+  { { OPERAND_sargt }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_srli_args[] = {
+  { { OPERAND_arr }, 'o' },
+  { { OPERAND_art }, 'i' },
+  { { OPERAND_s }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_sync_stateArgs[] = {
+  { { STATE_XTSYNC }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsil_args[] = {
+  { { OPERAND_art }, 'o' },
+  { { OPERAND_s }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsil_stateArgs[] = {
+  { { STATE_PSUM }, 'i' },
+  { { STATE_PSEXCM }, 'i' },
+  { { STATE_PSINTLEVEL }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_sar_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_sar_stateArgs[] = {
+  { { STATE_SAR }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_sar_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_sar_stateArgs[] = {
+  { { STATE_SAR }, 'o' },
+  { { STATE_XTSYNC }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_sar_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_sar_stateArgs[] = {
+  { { STATE_SAR }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_litbase_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_litbase_stateArgs[] = {
+  { { STATE_LITBADDR }, 'i' },
+  { { STATE_LITBEN }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_litbase_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_litbase_stateArgs[] = {
+  { { STATE_LITBADDR }, 'o' },
+  { { STATE_LITBEN }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_litbase_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_litbase_stateArgs[] = {
+  { { STATE_LITBADDR }, 'm' },
+  { { STATE_LITBEN }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_176_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_176_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_208_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_ps_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_ps_stateArgs[] = {
+  { { STATE_PSUM }, 'i' },
+  { { STATE_PSEXCM }, 'i' },
+  { { STATE_PSINTLEVEL }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_ps_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_ps_stateArgs[] = {
+  { { STATE_PSUM }, 'o' },
+  { { STATE_PSEXCM }, 'o' },
+  { { STATE_PSINTLEVEL }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_ps_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_ps_stateArgs[] = {
+  { { STATE_PSUM }, 'm' },
+  { { STATE_PSEXCM }, 'm' },
+  { { STATE_PSINTLEVEL }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_epc1_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_epc1_stateArgs[] = {
+  { { STATE_EPC1 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_epc1_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_epc1_stateArgs[] = {
+  { { STATE_EPC1 }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_epc1_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_epc1_stateArgs[] = {
+  { { STATE_EPC1 }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_excsave1_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_excsave1_stateArgs[] = {
+  { { STATE_EXCSAVE1 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_excsave1_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_excsave1_stateArgs[] = {
+  { { STATE_EXCSAVE1 }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_excsave1_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_excsave1_stateArgs[] = {
+  { { STATE_EXCSAVE1 }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_epc2_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_epc2_stateArgs[] = {
+  { { STATE_EPC2 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_epc2_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_epc2_stateArgs[] = {
+  { { STATE_EPC2 }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_epc2_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_epc2_stateArgs[] = {
+  { { STATE_EPC2 }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_excsave2_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_excsave2_stateArgs[] = {
+  { { STATE_EXCSAVE2 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_excsave2_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_excsave2_stateArgs[] = {
+  { { STATE_EXCSAVE2 }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_excsave2_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_excsave2_stateArgs[] = {
+  { { STATE_EXCSAVE2 }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_epc3_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_epc3_stateArgs[] = {
+  { { STATE_EPC3 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_epc3_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_epc3_stateArgs[] = {
+  { { STATE_EPC3 }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_epc3_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_epc3_stateArgs[] = {
+  { { STATE_EPC3 }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_excsave3_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_excsave3_stateArgs[] = {
+  { { STATE_EXCSAVE3 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_excsave3_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_excsave3_stateArgs[] = {
+  { { STATE_EXCSAVE3 }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_excsave3_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_excsave3_stateArgs[] = {
+  { { STATE_EXCSAVE3 }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_eps2_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_eps2_stateArgs[] = {
+  { { STATE_EPS2 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_eps2_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_eps2_stateArgs[] = {
+  { { STATE_EPS2 }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_eps2_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_eps2_stateArgs[] = {
+  { { STATE_EPS2 }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_eps3_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_eps3_stateArgs[] = {
+  { { STATE_EPS3 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_eps3_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_eps3_stateArgs[] = {
+  { { STATE_EPS3 }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_eps3_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_eps3_stateArgs[] = {
+  { { STATE_EPS3 }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_excvaddr_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_excvaddr_stateArgs[] = {
+  { { STATE_EXCVADDR }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_excvaddr_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_excvaddr_stateArgs[] = {
+  { { STATE_EXCVADDR }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_excvaddr_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_excvaddr_stateArgs[] = {
+  { { STATE_EXCVADDR }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_depc_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_depc_stateArgs[] = {
+  { { STATE_DEPC }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_depc_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_depc_stateArgs[] = {
+  { { STATE_DEPC }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_depc_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_depc_stateArgs[] = {
+  { { STATE_DEPC }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_exccause_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_exccause_stateArgs[] = {
+  { { STATE_EXCCAUSE }, 'i' },
+  { { STATE_XTSYNC }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_exccause_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_exccause_stateArgs[] = {
+  { { STATE_EXCCAUSE }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_exccause_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_exccause_stateArgs[] = {
+  { { STATE_EXCCAUSE }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_prid_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_vecbase_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_vecbase_stateArgs[] = {
+  { { STATE_VECBASE }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_vecbase_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_vecbase_stateArgs[] = {
+  { { STATE_VECBASE }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_vecbase_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_vecbase_stateArgs[] = {
+  { { STATE_VECBASE }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_mul16_args[] = {
+  { { OPERAND_arr }, 'o' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_mul32_args[] = {
+  { { OPERAND_arr }, 'o' },
+  { { OPERAND_ars }, 'i' },
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rfi_args[] = {
+  { { OPERAND_s }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rfi_stateArgs[] = {
+  { { STATE_PSUM }, 'o' },
+  { { STATE_PSEXCM }, 'o' },
+  { { STATE_PSINTLEVEL }, 'o' },
+  { { STATE_EPC1 }, 'i' },
+  { { STATE_EPC2 }, 'i' },
+  { { STATE_EPC3 }, 'i' },
+  { { STATE_EPS2 }, 'i' },
+  { { STATE_EPS3 }, 'i' },
+  { { STATE_InOCDMode }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wait_args[] = {
+  { { OPERAND_s }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wait_stateArgs[] = {
+  { { STATE_PSINTLEVEL }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_interrupt_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_interrupt_stateArgs[] = {
+  { { STATE_INTERRUPT }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_intset_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_intset_stateArgs[] = {
+  { { STATE_XTSYNC }, 'o' },
+  { { STATE_INTERRUPT }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_intclear_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_intclear_stateArgs[] = {
+  { { STATE_XTSYNC }, 'o' },
+  { { STATE_INTERRUPT }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_intenable_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_intenable_stateArgs[] = {
+  { { STATE_INTENABLE }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_intenable_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_intenable_stateArgs[] = {
+  { { STATE_INTENABLE }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_intenable_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_intenable_stateArgs[] = {
+  { { STATE_INTENABLE }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_break_args[] = {
+  { { OPERAND_imms }, 'i' },
+  { { OPERAND_immt }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_break_stateArgs[] = {
+  { { STATE_PSEXCM }, 'i' },
+  { { STATE_PSINTLEVEL }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_break_n_args[] = {
+  { { OPERAND_imms }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_break_n_stateArgs[] = {
+  { { STATE_PSEXCM }, 'i' },
+  { { STATE_PSINTLEVEL }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_dbreaka0_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_dbreaka0_stateArgs[] = {
+  { { STATE_DBREAKA0 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_dbreaka0_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_dbreaka0_stateArgs[] = {
+  { { STATE_DBREAKA0 }, 'o' },
+  { { STATE_XTSYNC }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_dbreaka0_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_dbreaka0_stateArgs[] = {
+  { { STATE_DBREAKA0 }, 'm' },
+  { { STATE_XTSYNC }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_dbreakc0_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_dbreakc0_stateArgs[] = {
+  { { STATE_DBREAKC0 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_dbreakc0_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_dbreakc0_stateArgs[] = {
+  { { STATE_DBREAKC0 }, 'o' },
+  { { STATE_XTSYNC }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_dbreakc0_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_dbreakc0_stateArgs[] = {
+  { { STATE_DBREAKC0 }, 'm' },
+  { { STATE_XTSYNC }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_ibreaka0_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_ibreaka0_stateArgs[] = {
+  { { STATE_IBREAKA0 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_ibreaka0_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_ibreaka0_stateArgs[] = {
+  { { STATE_IBREAKA0 }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_ibreaka0_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_ibreaka0_stateArgs[] = {
+  { { STATE_IBREAKA0 }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_ibreakenable_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_ibreakenable_stateArgs[] = {
+  { { STATE_IBREAKENABLE }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_ibreakenable_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_ibreakenable_stateArgs[] = {
+  { { STATE_IBREAKENABLE }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_ibreakenable_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_ibreakenable_stateArgs[] = {
+  { { STATE_IBREAKENABLE }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_debugcause_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_debugcause_stateArgs[] = {
+  { { STATE_DEBUGCAUSE }, 'i' },
+  { { STATE_DBNUM }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_debugcause_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_debugcause_stateArgs[] = {
+  { { STATE_DEBUGCAUSE }, 'o' },
+  { { STATE_DBNUM }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_debugcause_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_debugcause_stateArgs[] = {
+  { { STATE_DEBUGCAUSE }, 'm' },
+  { { STATE_DBNUM }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_icount_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_icount_stateArgs[] = {
+  { { STATE_ICOUNT }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_icount_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_icount_stateArgs[] = {
+  { { STATE_XTSYNC }, 'o' },
+  { { STATE_ICOUNT }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_icount_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_icount_stateArgs[] = {
+  { { STATE_XTSYNC }, 'o' },
+  { { STATE_ICOUNT }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_icountlevel_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_icountlevel_stateArgs[] = {
+  { { STATE_ICOUNTLEVEL }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_icountlevel_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_icountlevel_stateArgs[] = {
+  { { STATE_ICOUNTLEVEL }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_icountlevel_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_icountlevel_stateArgs[] = {
+  { { STATE_ICOUNTLEVEL }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_ddr_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_ddr_stateArgs[] = {
+  { { STATE_DDR }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_ddr_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_ddr_stateArgs[] = {
+  { { STATE_XTSYNC }, 'o' },
+  { { STATE_DDR }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_ddr_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_ddr_stateArgs[] = {
+  { { STATE_XTSYNC }, 'o' },
+  { { STATE_DDR }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rfdo_args[] = {
+  { { OPERAND_imms }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rfdo_stateArgs[] = {
+  { { STATE_InOCDMode }, 'm' },
+  { { STATE_EPC2 }, 'i' },
+  { { STATE_PSUM }, 'o' },
+  { { STATE_PSEXCM }, 'o' },
+  { { STATE_PSINTLEVEL }, 'o' },
+  { { STATE_EPS2 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rfdd_stateArgs[] = {
+  { { STATE_InOCDMode }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_mmid_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_mmid_stateArgs[] = {
+  { { STATE_XTSYNC }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_ccount_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_ccount_stateArgs[] = {
+  { { STATE_CCOUNT }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_ccount_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_ccount_stateArgs[] = {
+  { { STATE_XTSYNC }, 'o' },
+  { { STATE_CCOUNT }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_ccount_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_ccount_stateArgs[] = {
+  { { STATE_XTSYNC }, 'o' },
+  { { STATE_CCOUNT }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_ccompare0_args[] = {
+  { { OPERAND_art }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rsr_ccompare0_stateArgs[] = {
+  { { STATE_CCOMPARE0 }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_ccompare0_args[] = {
+  { { OPERAND_art }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wsr_ccompare0_stateArgs[] = {
+  { { STATE_CCOMPARE0 }, 'o' },
+  { { STATE_INTERRUPT }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_ccompare0_args[] = {
+  { { OPERAND_art }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_xsr_ccompare0_stateArgs[] = {
+  { { STATE_CCOMPARE0 }, 'm' },
+  { { STATE_INTERRUPT }, 'm' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_idtlb_args[] = {
+  { { OPERAND_ars }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_idtlb_stateArgs[] = {
+  { { STATE_XTSYNC }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_rdtlb_args[] = {
+  { { OPERAND_art }, 'o' },
+  { { OPERAND_ars }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wdtlb_args[] = {
+  { { OPERAND_art }, 'i' },
+  { { OPERAND_ars }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_wdtlb_stateArgs[] = {
+  { { STATE_XTSYNC }, 'o' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_iitlb_args[] = {
+  { { OPERAND_ars }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_ritlb_args[] = {
+  { { OPERAND_art }, 'o' },
+  { { OPERAND_ars }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_witlb_args[] = {
+  { { OPERAND_art }, 'i' },
+  { { OPERAND_ars }, 'i' }
+};
+
+static xtensa_arg_internal Iclass_xt_iclass_nsa_args[] = {
+  { { OPERAND_art }, 'o' },
+  { { OPERAND_ars }, 'i' }
+};
+
+static xtensa_iclass_internal iclasses[] = {
+  { 0, 0 /* xt_iclass_excw */,
+    0, 0, 0, 0 },
+  { 0, 0 /* xt_iclass_rfe */,
+    2, Iclass_xt_iclass_rfe_stateArgs, 0, 0 },
+  { 0, 0 /* xt_iclass_rfde */,
+    1, Iclass_xt_iclass_rfde_stateArgs, 0, 0 },
+  { 0, 0 /* xt_iclass_syscall */,
+    0, 0, 0, 0 },
+  { 0, 0 /* xt_iclass_simcall */,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_add_n_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_addi_n_args,
+    0, 0, 0, 0 },
+  { 2, Iclass_xt_iclass_bz6_args,
+    0, 0, 0, 0 },
+  { 0, 0 /* xt_iclass_ill_n */,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_loadi4_args,
+    0, 0, 0, 0 },
+  { 2, Iclass_xt_iclass_mov_n_args,
+    0, 0, 0, 0 },
+  { 2, Iclass_xt_iclass_movi_n_args,
+    0, 0, 0, 0 },
+  { 0, 0 /* xt_iclass_nopn */,
+    0, 0, 0, 0 },
+  { 1, Iclass_xt_iclass_retn_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_storei4_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_addi_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_addmi_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_addsub_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_bit_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_bsi8_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_bsi8b_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_bsi8u_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_bst8_args,
+    0, 0, 0, 0 },
+  { 2, Iclass_xt_iclass_bsz12_args,
+    0, 0, 0, 0 },
+  { 2, Iclass_xt_iclass_call0_args,
+    0, 0, 0, 0 },
+  { 2, Iclass_xt_iclass_callx0_args,
+    0, 0, 0, 0 },
+  { 4, Iclass_xt_iclass_exti_args,
+    0, 0, 0, 0 },
+  { 0, 0 /* xt_iclass_ill */,
+    0, 0, 0, 0 },
+  { 1, Iclass_xt_iclass_jump_args,
+    0, 0, 0, 0 },
+  { 1, Iclass_xt_iclass_jumpx_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_l16ui_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_l16si_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_l32i_args,
+    0, 0, 0, 0 },
+  { 2, Iclass_xt_iclass_l32r_args,
+    2, Iclass_xt_iclass_l32r_stateArgs, 0, 0 },
+  { 3, Iclass_xt_iclass_l8i_args,
+    0, 0, 0, 0 },
+  { 2, Iclass_xt_iclass_movi_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_movz_args,
+    0, 0, 0, 0 },
+  { 2, Iclass_xt_iclass_neg_args,
+    0, 0, 0, 0 },
+  { 0, 0 /* xt_iclass_nop */,
+    0, 0, 0, 0 },
+  { 1, Iclass_xt_iclass_return_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_s16i_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_s32i_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_s8i_args,
+    0, 0, 0, 0 },
+  { 1, Iclass_xt_iclass_sar_args,
+    1, Iclass_xt_iclass_sar_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_sari_args,
+    1, Iclass_xt_iclass_sari_stateArgs, 0, 0 },
+  { 2, Iclass_xt_iclass_shifts_args,
+    1, Iclass_xt_iclass_shifts_stateArgs, 0, 0 },
+  { 3, Iclass_xt_iclass_shiftst_args,
+    1, Iclass_xt_iclass_shiftst_stateArgs, 0, 0 },
+  { 2, Iclass_xt_iclass_shiftt_args,
+    1, Iclass_xt_iclass_shiftt_stateArgs, 0, 0 },
+  { 3, Iclass_xt_iclass_slli_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_srai_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_iclass_srli_args,
+    0, 0, 0, 0 },
+  { 0, 0 /* xt_iclass_memw */,
+    0, 0, 0, 0 },
+  { 0, 0 /* xt_iclass_extw */,
+    0, 0, 0, 0 },
+  { 0, 0 /* xt_iclass_isync */,
+    0, 0, 0, 0 },
+  { 0, 0 /* xt_iclass_sync */,
+    1, Iclass_xt_iclass_sync_stateArgs, 0, 0 },
+  { 2, Iclass_xt_iclass_rsil_args,
+    3, Iclass_xt_iclass_rsil_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_sar_args,
+    1, Iclass_xt_iclass_rsr_sar_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_sar_args,
+    2, Iclass_xt_iclass_wsr_sar_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_sar_args,
+    1, Iclass_xt_iclass_xsr_sar_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_litbase_args,
+    2, Iclass_xt_iclass_rsr_litbase_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_litbase_args,
+    2, Iclass_xt_iclass_wsr_litbase_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_litbase_args,
+    2, Iclass_xt_iclass_xsr_litbase_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_176_args,
+    0, 0, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_176_args,
+    0, 0, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_208_args,
+    0, 0, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_ps_args,
+    3, Iclass_xt_iclass_rsr_ps_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_ps_args,
+    3, Iclass_xt_iclass_wsr_ps_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_ps_args,
+    3, Iclass_xt_iclass_xsr_ps_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_epc1_args,
+    1, Iclass_xt_iclass_rsr_epc1_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_epc1_args,
+    1, Iclass_xt_iclass_wsr_epc1_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_epc1_args,
+    1, Iclass_xt_iclass_xsr_epc1_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_excsave1_args,
+    1, Iclass_xt_iclass_rsr_excsave1_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_excsave1_args,
+    1, Iclass_xt_iclass_wsr_excsave1_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_excsave1_args,
+    1, Iclass_xt_iclass_xsr_excsave1_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_epc2_args,
+    1, Iclass_xt_iclass_rsr_epc2_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_epc2_args,
+    1, Iclass_xt_iclass_wsr_epc2_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_epc2_args,
+    1, Iclass_xt_iclass_xsr_epc2_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_excsave2_args,
+    1, Iclass_xt_iclass_rsr_excsave2_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_excsave2_args,
+    1, Iclass_xt_iclass_wsr_excsave2_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_excsave2_args,
+    1, Iclass_xt_iclass_xsr_excsave2_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_epc3_args,
+    1, Iclass_xt_iclass_rsr_epc3_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_epc3_args,
+    1, Iclass_xt_iclass_wsr_epc3_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_epc3_args,
+    1, Iclass_xt_iclass_xsr_epc3_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_excsave3_args,
+    1, Iclass_xt_iclass_rsr_excsave3_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_excsave3_args,
+    1, Iclass_xt_iclass_wsr_excsave3_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_excsave3_args,
+    1, Iclass_xt_iclass_xsr_excsave3_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_eps2_args,
+    1, Iclass_xt_iclass_rsr_eps2_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_eps2_args,
+    1, Iclass_xt_iclass_wsr_eps2_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_eps2_args,
+    1, Iclass_xt_iclass_xsr_eps2_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_eps3_args,
+    1, Iclass_xt_iclass_rsr_eps3_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_eps3_args,
+    1, Iclass_xt_iclass_wsr_eps3_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_eps3_args,
+    1, Iclass_xt_iclass_xsr_eps3_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_excvaddr_args,
+    1, Iclass_xt_iclass_rsr_excvaddr_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_excvaddr_args,
+    1, Iclass_xt_iclass_wsr_excvaddr_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_excvaddr_args,
+    1, Iclass_xt_iclass_xsr_excvaddr_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_depc_args,
+    1, Iclass_xt_iclass_rsr_depc_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_depc_args,
+    1, Iclass_xt_iclass_wsr_depc_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_depc_args,
+    1, Iclass_xt_iclass_xsr_depc_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_exccause_args,
+    2, Iclass_xt_iclass_rsr_exccause_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_exccause_args,
+    1, Iclass_xt_iclass_wsr_exccause_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_exccause_args,
+    1, Iclass_xt_iclass_xsr_exccause_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_prid_args,
+    0, 0, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_vecbase_args,
+    1, Iclass_xt_iclass_rsr_vecbase_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_vecbase_args,
+    1, Iclass_xt_iclass_wsr_vecbase_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_vecbase_args,
+    1, Iclass_xt_iclass_xsr_vecbase_stateArgs, 0, 0 },
+  { 3, Iclass_xt_mul16_args,
+    0, 0, 0, 0 },
+  { 3, Iclass_xt_mul32_args,
+    0, 0, 0, 0 },
+  { 1, Iclass_xt_iclass_rfi_args,
+    9, Iclass_xt_iclass_rfi_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wait_args,
+    1, Iclass_xt_iclass_wait_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_interrupt_args,
+    1, Iclass_xt_iclass_rsr_interrupt_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_intset_args,
+    2, Iclass_xt_iclass_wsr_intset_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_intclear_args,
+    2, Iclass_xt_iclass_wsr_intclear_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_intenable_args,
+    1, Iclass_xt_iclass_rsr_intenable_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_intenable_args,
+    1, Iclass_xt_iclass_wsr_intenable_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_intenable_args,
+    1, Iclass_xt_iclass_xsr_intenable_stateArgs, 0, 0 },
+  { 2, Iclass_xt_iclass_break_args,
+    2, Iclass_xt_iclass_break_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_break_n_args,
+    2, Iclass_xt_iclass_break_n_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_dbreaka0_args,
+    1, Iclass_xt_iclass_rsr_dbreaka0_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_dbreaka0_args,
+    2, Iclass_xt_iclass_wsr_dbreaka0_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_dbreaka0_args,
+    2, Iclass_xt_iclass_xsr_dbreaka0_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_dbreakc0_args,
+    1, Iclass_xt_iclass_rsr_dbreakc0_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_dbreakc0_args,
+    2, Iclass_xt_iclass_wsr_dbreakc0_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_dbreakc0_args,
+    2, Iclass_xt_iclass_xsr_dbreakc0_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_ibreaka0_args,
+    1, Iclass_xt_iclass_rsr_ibreaka0_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_ibreaka0_args,
+    1, Iclass_xt_iclass_wsr_ibreaka0_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_ibreaka0_args,
+    1, Iclass_xt_iclass_xsr_ibreaka0_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_ibreakenable_args,
+    1, Iclass_xt_iclass_rsr_ibreakenable_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_ibreakenable_args,
+    1, Iclass_xt_iclass_wsr_ibreakenable_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_ibreakenable_args,
+    1, Iclass_xt_iclass_xsr_ibreakenable_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_debugcause_args,
+    2, Iclass_xt_iclass_rsr_debugcause_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_debugcause_args,
+    2, Iclass_xt_iclass_wsr_debugcause_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_debugcause_args,
+    2, Iclass_xt_iclass_xsr_debugcause_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_icount_args,
+    1, Iclass_xt_iclass_rsr_icount_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_icount_args,
+    2, Iclass_xt_iclass_wsr_icount_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_icount_args,
+    2, Iclass_xt_iclass_xsr_icount_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_icountlevel_args,
+    1, Iclass_xt_iclass_rsr_icountlevel_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_icountlevel_args,
+    1, Iclass_xt_iclass_wsr_icountlevel_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_icountlevel_args,
+    1, Iclass_xt_iclass_xsr_icountlevel_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_ddr_args,
+    1, Iclass_xt_iclass_rsr_ddr_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_ddr_args,
+    2, Iclass_xt_iclass_wsr_ddr_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_ddr_args,
+    2, Iclass_xt_iclass_xsr_ddr_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rfdo_args,
+    6, Iclass_xt_iclass_rfdo_stateArgs, 0, 0 },
+  { 0, 0 /* xt_iclass_rfdd */,
+    1, Iclass_xt_iclass_rfdd_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_mmid_args,
+    1, Iclass_xt_iclass_wsr_mmid_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_ccount_args,
+    1, Iclass_xt_iclass_rsr_ccount_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_ccount_args,
+    2, Iclass_xt_iclass_wsr_ccount_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_ccount_args,
+    2, Iclass_xt_iclass_xsr_ccount_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_rsr_ccompare0_args,
+    1, Iclass_xt_iclass_rsr_ccompare0_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_wsr_ccompare0_args,
+    2, Iclass_xt_iclass_wsr_ccompare0_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_xsr_ccompare0_args,
+    2, Iclass_xt_iclass_xsr_ccompare0_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_idtlb_args,
+    1, Iclass_xt_iclass_idtlb_stateArgs, 0, 0 },
+  { 2, Iclass_xt_iclass_rdtlb_args,
+    0, 0, 0, 0 },
+  { 2, Iclass_xt_iclass_wdtlb_args,
+    1, Iclass_xt_iclass_wdtlb_stateArgs, 0, 0 },
+  { 1, Iclass_xt_iclass_iitlb_args,
+    0, 0, 0, 0 },
+  { 2, Iclass_xt_iclass_ritlb_args,
+    0, 0, 0, 0 },
+  { 2, Iclass_xt_iclass_witlb_args,
+    0, 0, 0, 0 },
+  { 2, Iclass_xt_iclass_nsa_args,
+    0, 0, 0, 0 },
+  { 0, 0 /* xt_iclass_rer */,
+    0, 0, 0, 0 },
+  { 0, 0 /* xt_iclass_wer */,
+    0, 0, 0, 0 }
+};
+
+enum xtensa_iclass_id {
+  ICLASS_xt_iclass_excw,
+  ICLASS_xt_iclass_rfe,
+  ICLASS_xt_iclass_rfde,
+  ICLASS_xt_iclass_syscall,
+  ICLASS_xt_iclass_simcall,
+  ICLASS_xt_iclass_add_n,
+  ICLASS_xt_iclass_addi_n,
+  ICLASS_xt_iclass_bz6,
+  ICLASS_xt_iclass_ill_n,
+  ICLASS_xt_iclass_loadi4,
+  ICLASS_xt_iclass_mov_n,
+  ICLASS_xt_iclass_movi_n,
+  ICLASS_xt_iclass_nopn,
+  ICLASS_xt_iclass_retn,
+  ICLASS_xt_iclass_storei4,
+  ICLASS_xt_iclass_addi,
+  ICLASS_xt_iclass_addmi,
+  ICLASS_xt_iclass_addsub,
+  ICLASS_xt_iclass_bit,
+  ICLASS_xt_iclass_bsi8,
+  ICLASS_xt_iclass_bsi8b,
+  ICLASS_xt_iclass_bsi8u,
+  ICLASS_xt_iclass_bst8,
+  ICLASS_xt_iclass_bsz12,
+  ICLASS_xt_iclass_call0,
+  ICLASS_xt_iclass_callx0,
+  ICLASS_xt_iclass_exti,
+  ICLASS_xt_iclass_ill,
+  ICLASS_xt_iclass_jump,
+  ICLASS_xt_iclass_jumpx,
+  ICLASS_xt_iclass_l16ui,
+  ICLASS_xt_iclass_l16si,
+  ICLASS_xt_iclass_l32i,
+  ICLASS_xt_iclass_l32r,
+  ICLASS_xt_iclass_l8i,
+  ICLASS_xt_iclass_movi,
+  ICLASS_xt_iclass_movz,
+  ICLASS_xt_iclass_neg,
+  ICLASS_xt_iclass_nop,
+  ICLASS_xt_iclass_return,
+  ICLASS_xt_iclass_s16i,
+  ICLASS_xt_iclass_s32i,
+  ICLASS_xt_iclass_s8i,
+  ICLASS_xt_iclass_sar,
+  ICLASS_xt_iclass_sari,
+  ICLASS_xt_iclass_shifts,
+  ICLASS_xt_iclass_shiftst,
+  ICLASS_xt_iclass_shiftt,
+  ICLASS_xt_iclass_slli,
+  ICLASS_xt_iclass_srai,
+  ICLASS_xt_iclass_srli,
+  ICLASS_xt_iclass_memw,
+  ICLASS_xt_iclass_extw,
+  ICLASS_xt_iclass_isync,
+  ICLASS_xt_iclass_sync,
+  ICLASS_xt_iclass_rsil,
+  ICLASS_xt_iclass_rsr_sar,
+  ICLASS_xt_iclass_wsr_sar,
+  ICLASS_xt_iclass_xsr_sar,
+  ICLASS_xt_iclass_rsr_litbase,
+  ICLASS_xt_iclass_wsr_litbase,
+  ICLASS_xt_iclass_xsr_litbase,
+  ICLASS_xt_iclass_rsr_176,
+  ICLASS_xt_iclass_wsr_176,
+  ICLASS_xt_iclass_rsr_208,
+  ICLASS_xt_iclass_rsr_ps,
+  ICLASS_xt_iclass_wsr_ps,
+  ICLASS_xt_iclass_xsr_ps,
+  ICLASS_xt_iclass_rsr_epc1,
+  ICLASS_xt_iclass_wsr_epc1,
+  ICLASS_xt_iclass_xsr_epc1,
+  ICLASS_xt_iclass_rsr_excsave1,
+  ICLASS_xt_iclass_wsr_excsave1,
+  ICLASS_xt_iclass_xsr_excsave1,
+  ICLASS_xt_iclass_rsr_epc2,
+  ICLASS_xt_iclass_wsr_epc2,
+  ICLASS_xt_iclass_xsr_epc2,
+  ICLASS_xt_iclass_rsr_excsave2,
+  ICLASS_xt_iclass_wsr_excsave2,
+  ICLASS_xt_iclass_xsr_excsave2,
+  ICLASS_xt_iclass_rsr_epc3,
+  ICLASS_xt_iclass_wsr_epc3,
+  ICLASS_xt_iclass_xsr_epc3,
+  ICLASS_xt_iclass_rsr_excsave3,
+  ICLASS_xt_iclass_wsr_excsave3,
+  ICLASS_xt_iclass_xsr_excsave3,
+  ICLASS_xt_iclass_rsr_eps2,
+  ICLASS_xt_iclass_wsr_eps2,
+  ICLASS_xt_iclass_xsr_eps2,
+  ICLASS_xt_iclass_rsr_eps3,
+  ICLASS_xt_iclass_wsr_eps3,
+  ICLASS_xt_iclass_xsr_eps3,
+  ICLASS_xt_iclass_rsr_excvaddr,
+  ICLASS_xt_iclass_wsr_excvaddr,
+  ICLASS_xt_iclass_xsr_excvaddr,
+  ICLASS_xt_iclass_rsr_depc,
+  ICLASS_xt_iclass_wsr_depc,
+  ICLASS_xt_iclass_xsr_depc,
+  ICLASS_xt_iclass_rsr_exccause,
+  ICLASS_xt_iclass_wsr_exccause,
+  ICLASS_xt_iclass_xsr_exccause,
+  ICLASS_xt_iclass_rsr_prid,
+  ICLASS_xt_iclass_rsr_vecbase,
+  ICLASS_xt_iclass_wsr_vecbase,
+  ICLASS_xt_iclass_xsr_vecbase,
+  ICLASS_xt_mul16,
+  ICLASS_xt_mul32,
+  ICLASS_xt_iclass_rfi,
+  ICLASS_xt_iclass_wait,
+  ICLASS_xt_iclass_rsr_interrupt,
+  ICLASS_xt_iclass_wsr_intset,
+  ICLASS_xt_iclass_wsr_intclear,
+  ICLASS_xt_iclass_rsr_intenable,
+  ICLASS_xt_iclass_wsr_intenable,
+  ICLASS_xt_iclass_xsr_intenable,
+  ICLASS_xt_iclass_break,
+  ICLASS_xt_iclass_break_n,
+  ICLASS_xt_iclass_rsr_dbreaka0,
+  ICLASS_xt_iclass_wsr_dbreaka0,
+  ICLASS_xt_iclass_xsr_dbreaka0,
+  ICLASS_xt_iclass_rsr_dbreakc0,
+  ICLASS_xt_iclass_wsr_dbreakc0,
+  ICLASS_xt_iclass_xsr_dbreakc0,
+  ICLASS_xt_iclass_rsr_ibreaka0,
+  ICLASS_xt_iclass_wsr_ibreaka0,
+  ICLASS_xt_iclass_xsr_ibreaka0,
+  ICLASS_xt_iclass_rsr_ibreakenable,
+  ICLASS_xt_iclass_wsr_ibreakenable,
+  ICLASS_xt_iclass_xsr_ibreakenable,
+  ICLASS_xt_iclass_rsr_debugcause,
+  ICLASS_xt_iclass_wsr_debugcause,
+  ICLASS_xt_iclass_xsr_debugcause,
+  ICLASS_xt_iclass_rsr_icount,
+  ICLASS_xt_iclass_wsr_icount,
+  ICLASS_xt_iclass_xsr_icount,
+  ICLASS_xt_iclass_rsr_icountlevel,
+  ICLASS_xt_iclass_wsr_icountlevel,
+  ICLASS_xt_iclass_xsr_icountlevel,
+  ICLASS_xt_iclass_rsr_ddr,
+  ICLASS_xt_iclass_wsr_ddr,
+  ICLASS_xt_iclass_xsr_ddr,
+  ICLASS_xt_iclass_rfdo,
+  ICLASS_xt_iclass_rfdd,
+  ICLASS_xt_iclass_wsr_mmid,
+  ICLASS_xt_iclass_rsr_ccount,
+  ICLASS_xt_iclass_wsr_ccount,
+  ICLASS_xt_iclass_xsr_ccount,
+  ICLASS_xt_iclass_rsr_ccompare0,
+  ICLASS_xt_iclass_wsr_ccompare0,
+  ICLASS_xt_iclass_xsr_ccompare0,
+  ICLASS_xt_iclass_idtlb,
+  ICLASS_xt_iclass_rdtlb,
+  ICLASS_xt_iclass_wdtlb,
+  ICLASS_xt_iclass_iitlb,
+  ICLASS_xt_iclass_ritlb,
+  ICLASS_xt_iclass_witlb,
+  ICLASS_xt_iclass_nsa,
+  ICLASS_xt_iclass_rer,
+  ICLASS_xt_iclass_wer
+};
+
+
+/*  Opcode encodings.  */
+
+static void
+Opcode_excw_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x2080;
+}
+
+static void
+Opcode_rfe_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3000;
+}
+
+static void
+Opcode_rfde_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3200;
+}
+
+static void
+Opcode_syscall_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x5000;
+}
+
+static void
+Opcode_simcall_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x5100;
+}
+
+static void
+Opcode_add_n_Slot_inst16a_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xa;
+}
+
+static void
+Opcode_addi_n_Slot_inst16a_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xb;
+}
+
+static void
+Opcode_beqz_n_Slot_inst16b_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x8c;
+}
+
+static void
+Opcode_bnez_n_Slot_inst16b_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xcc;
+}
+
+static void
+Opcode_ill_n_Slot_inst16b_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xf06d;
+}
+
+static void
+Opcode_l32i_n_Slot_inst16a_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x8;
+}
+
+static void
+Opcode_mov_n_Slot_inst16b_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xd;
+}
+
+static void
+Opcode_movi_n_Slot_inst16b_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xc;
+}
+
+static void
+Opcode_nop_n_Slot_inst16b_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xf03d;
+}
+
+static void
+Opcode_ret_n_Slot_inst16b_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xf00d;
+}
+
+static void
+Opcode_s32i_n_Slot_inst16a_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x9;
+}
+
+static void
+Opcode_addi_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xc002;
+}
+
+static void
+Opcode_addmi_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xd002;
+}
+
+static void
+Opcode_add_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x800000;
+}
+
+static void
+Opcode_sub_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xc00000;
+}
+
+static void
+Opcode_addx2_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x900000;
+}
+
+static void
+Opcode_addx4_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xa00000;
+}
+
+static void
+Opcode_addx8_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xb00000;
+}
+
+static void
+Opcode_subx2_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xd00000;
+}
+
+static void
+Opcode_subx4_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xe00000;
+}
+
+static void
+Opcode_subx8_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xf00000;
+}
+
+static void
+Opcode_and_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x100000;
+}
+
+static void
+Opcode_or_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x200000;
+}
+
+static void
+Opcode_xor_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x300000;
+}
+
+static void
+Opcode_beqi_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x26;
+}
+
+static void
+Opcode_bnei_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x66;
+}
+
+static void
+Opcode_bgei_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xe6;
+}
+
+static void
+Opcode_blti_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xa6;
+}
+
+static void
+Opcode_bbci_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x6007;
+}
+
+static void
+Opcode_bbsi_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xe007;
+}
+
+static void
+Opcode_bgeui_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xf6;
+}
+
+static void
+Opcode_bltui_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xb6;
+}
+
+static void
+Opcode_beq_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x1007;
+}
+
+static void
+Opcode_bne_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x9007;
+}
+
+static void
+Opcode_bge_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xa007;
+}
+
+static void
+Opcode_blt_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x2007;
+}
+
+static void
+Opcode_bgeu_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xb007;
+}
+
+static void
+Opcode_bltu_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3007;
+}
+
+static void
+Opcode_bany_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x8007;
+}
+
+static void
+Opcode_bnone_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x7;
+}
+
+static void
+Opcode_ball_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x4007;
+}
+
+static void
+Opcode_bnall_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xc007;
+}
+
+static void
+Opcode_bbc_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x5007;
+}
+
+static void
+Opcode_bbs_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xd007;
+}
+
+static void
+Opcode_beqz_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x16;
+}
+
+static void
+Opcode_bnez_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x56;
+}
+
+static void
+Opcode_bgez_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xd6;
+}
+
+static void
+Opcode_bltz_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x96;
+}
+
+static void
+Opcode_call0_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x5;
+}
+
+static void
+Opcode_callx0_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xc0;
+}
+
+static void
+Opcode_extui_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x40000;
+}
+
+static void
+Opcode_ill_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0;
+}
+
+static void
+Opcode_j_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x6;
+}
+
+static void
+Opcode_jx_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xa0;
+}
+
+static void
+Opcode_l16ui_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x1002;
+}
+
+static void
+Opcode_l16si_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x9002;
+}
+
+static void
+Opcode_l32i_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x2002;
+}
+
+static void
+Opcode_l32r_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x1;
+}
+
+static void
+Opcode_l8ui_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x2;
+}
+
+static void
+Opcode_movi_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xa002;
+}
+
+static void
+Opcode_moveqz_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x830000;
+}
+
+static void
+Opcode_movnez_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x930000;
+}
+
+static void
+Opcode_movltz_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xa30000;
+}
+
+static void
+Opcode_movgez_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xb30000;
+}
+
+static void
+Opcode_neg_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x600000;
+}
+
+static void
+Opcode_abs_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x600100;
+}
+
+static void
+Opcode_nop_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x20f0;
+}
+
+static void
+Opcode_ret_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x80;
+}
+
+static void
+Opcode_s16i_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x5002;
+}
+
+static void
+Opcode_s32i_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x6002;
+}
+
+static void
+Opcode_s8i_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x4002;
+}
+
+static void
+Opcode_ssr_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x400000;
+}
+
+static void
+Opcode_ssl_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x401000;
+}
+
+static void
+Opcode_ssa8l_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x402000;
+}
+
+static void
+Opcode_ssa8b_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x403000;
+}
+
+static void
+Opcode_ssai_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x404000;
+}
+
+static void
+Opcode_sll_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xa10000;
+}
+
+static void
+Opcode_src_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x810000;
+}
+
+static void
+Opcode_srl_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x910000;
+}
+
+static void
+Opcode_sra_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xb10000;
+}
+
+static void
+Opcode_slli_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x10000;
+}
+
+static void
+Opcode_srai_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x210000;
+}
+
+static void
+Opcode_srli_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x410000;
+}
+
+static void
+Opcode_memw_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x20c0;
+}
+
+static void
+Opcode_extw_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x20d0;
+}
+
+static void
+Opcode_isync_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x2000;
+}
+
+static void
+Opcode_rsync_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x2010;
+}
+
+static void
+Opcode_esync_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x2020;
+}
+
+static void
+Opcode_dsync_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x2030;
+}
+
+static void
+Opcode_rsil_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x6000;
+}
+
+static void
+Opcode_rsr_sar_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x30300;
+}
+
+static void
+Opcode_wsr_sar_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x130300;
+}
+
+static void
+Opcode_xsr_sar_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x610300;
+}
+
+static void
+Opcode_rsr_litbase_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x30500;
+}
+
+static void
+Opcode_wsr_litbase_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x130500;
+}
+
+static void
+Opcode_xsr_litbase_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x610500;
+}
+
+static void
+Opcode_rsr_176_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3b000;
+}
+
+static void
+Opcode_wsr_176_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13b000;
+}
+
+static void
+Opcode_rsr_208_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3d000;
+}
+
+static void
+Opcode_rsr_ps_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3e600;
+}
+
+static void
+Opcode_wsr_ps_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13e600;
+}
+
+static void
+Opcode_xsr_ps_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61e600;
+}
+
+static void
+Opcode_rsr_epc1_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3b100;
+}
+
+static void
+Opcode_wsr_epc1_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13b100;
+}
+
+static void
+Opcode_xsr_epc1_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61b100;
+}
+
+static void
+Opcode_rsr_excsave1_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3d100;
+}
+
+static void
+Opcode_wsr_excsave1_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13d100;
+}
+
+static void
+Opcode_xsr_excsave1_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61d100;
+}
+
+static void
+Opcode_rsr_epc2_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3b200;
+}
+
+static void
+Opcode_wsr_epc2_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13b200;
+}
+
+static void
+Opcode_xsr_epc2_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61b200;
+}
+
+static void
+Opcode_rsr_excsave2_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3d200;
+}
+
+static void
+Opcode_wsr_excsave2_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13d200;
+}
+
+static void
+Opcode_xsr_excsave2_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61d200;
+}
+
+static void
+Opcode_rsr_epc3_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3b300;
+}
+
+static void
+Opcode_wsr_epc3_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13b300;
+}
+
+static void
+Opcode_xsr_epc3_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61b300;
+}
+
+static void
+Opcode_rsr_excsave3_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3d300;
+}
+
+static void
+Opcode_wsr_excsave3_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13d300;
+}
+
+static void
+Opcode_xsr_excsave3_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61d300;
+}
+
+static void
+Opcode_rsr_eps2_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3c200;
+}
+
+static void
+Opcode_wsr_eps2_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13c200;
+}
+
+static void
+Opcode_xsr_eps2_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61c200;
+}
+
+static void
+Opcode_rsr_eps3_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3c300;
+}
+
+static void
+Opcode_wsr_eps3_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13c300;
+}
+
+static void
+Opcode_xsr_eps3_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61c300;
+}
+
+static void
+Opcode_rsr_excvaddr_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3ee00;
+}
+
+static void
+Opcode_wsr_excvaddr_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13ee00;
+}
+
+static void
+Opcode_xsr_excvaddr_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61ee00;
+}
+
+static void
+Opcode_rsr_depc_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3c000;
+}
+
+static void
+Opcode_wsr_depc_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13c000;
+}
+
+static void
+Opcode_xsr_depc_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61c000;
+}
+
+static void
+Opcode_rsr_exccause_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3e800;
+}
+
+static void
+Opcode_wsr_exccause_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13e800;
+}
+
+static void
+Opcode_xsr_exccause_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61e800;
+}
+
+static void
+Opcode_rsr_prid_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3eb00;
+}
+
+static void
+Opcode_rsr_vecbase_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3e700;
+}
+
+static void
+Opcode_wsr_vecbase_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13e700;
+}
+
+static void
+Opcode_xsr_vecbase_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61e700;
+}
+
+static void
+Opcode_mul16u_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xc10000;
+}
+
+static void
+Opcode_mul16s_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xd10000;
+}
+
+static void
+Opcode_mull_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x820000;
+}
+
+static void
+Opcode_rfi_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3010;
+}
+
+static void
+Opcode_waiti_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x7000;
+}
+
+static void
+Opcode_rsr_interrupt_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3e200;
+}
+
+static void
+Opcode_wsr_intset_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13e200;
+}
+
+static void
+Opcode_wsr_intclear_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13e300;
+}
+
+static void
+Opcode_rsr_intenable_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3e400;
+}
+
+static void
+Opcode_wsr_intenable_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13e400;
+}
+
+static void
+Opcode_xsr_intenable_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61e400;
+}
+
+static void
+Opcode_break_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x4000;
+}
+
+static void
+Opcode_break_n_Slot_inst16b_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xf02d;
+}
+
+static void
+Opcode_rsr_dbreaka0_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x39000;
+}
+
+static void
+Opcode_wsr_dbreaka0_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x139000;
+}
+
+static void
+Opcode_xsr_dbreaka0_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x619000;
+}
+
+static void
+Opcode_rsr_dbreakc0_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3a000;
+}
+
+static void
+Opcode_wsr_dbreakc0_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13a000;
+}
+
+static void
+Opcode_xsr_dbreakc0_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61a000;
+}
+
+static void
+Opcode_rsr_ibreaka0_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x38000;
+}
+
+static void
+Opcode_wsr_ibreaka0_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x138000;
+}
+
+static void
+Opcode_xsr_ibreaka0_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x618000;
+}
+
+static void
+Opcode_rsr_ibreakenable_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x36000;
+}
+
+static void
+Opcode_wsr_ibreakenable_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x136000;
+}
+
+static void
+Opcode_xsr_ibreakenable_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x616000;
+}
+
+static void
+Opcode_rsr_debugcause_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3e900;
+}
+
+static void
+Opcode_wsr_debugcause_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13e900;
+}
+
+static void
+Opcode_xsr_debugcause_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61e900;
+}
+
+static void
+Opcode_rsr_icount_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3ec00;
+}
+
+static void
+Opcode_wsr_icount_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13ec00;
+}
+
+static void
+Opcode_xsr_icount_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61ec00;
+}
+
+static void
+Opcode_rsr_icountlevel_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3ed00;
+}
+
+static void
+Opcode_wsr_icountlevel_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13ed00;
+}
+
+static void
+Opcode_xsr_icountlevel_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61ed00;
+}
+
+static void
+Opcode_rsr_ddr_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x36800;
+}
+
+static void
+Opcode_wsr_ddr_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x136800;
+}
+
+static void
+Opcode_xsr_ddr_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x616800;
+}
+
+static void
+Opcode_rfdo_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xf1e000;
+}
+
+static void
+Opcode_rfdd_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0xf1e010;
+}
+
+static void
+Opcode_wsr_mmid_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x135900;
+}
+
+static void
+Opcode_rsr_ccount_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3ea00;
+}
+
+static void
+Opcode_wsr_ccount_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13ea00;
+}
+
+static void
+Opcode_xsr_ccount_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61ea00;
+}
+
+static void
+Opcode_rsr_ccompare0_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x3f000;
+}
+
+static void
+Opcode_wsr_ccompare0_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x13f000;
+}
+
+static void
+Opcode_xsr_ccompare0_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x61f000;
+}
+
+static void
+Opcode_idtlb_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x50c000;
+}
+
+static void
+Opcode_pdtlb_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x50d000;
+}
+
+static void
+Opcode_rdtlb0_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x50b000;
+}
+
+static void
+Opcode_rdtlb1_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x50f000;
+}
+
+static void
+Opcode_wdtlb_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x50e000;
+}
+
+static void
+Opcode_iitlb_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x504000;
+}
+
+static void
+Opcode_pitlb_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x505000;
+}
+
+static void
+Opcode_ritlb0_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x503000;
+}
+
+static void
+Opcode_ritlb1_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x507000;
+}
+
+static void
+Opcode_witlb_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x506000;
+}
+
+static void
+Opcode_nsa_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x40e000;
+}
+
+static void
+Opcode_nsau_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x40f000;
+}
+
+static void
+Opcode_rer_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x406000;
+}
+
+static void
+Opcode_wer_Slot_inst_encode (xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = 0x407000;
+}
+
+static xtensa_opcode_encode_fn Opcode_excw_encode_fns[] = {
+  Opcode_excw_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rfe_encode_fns[] = {
+  Opcode_rfe_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rfde_encode_fns[] = {
+  Opcode_rfde_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_syscall_encode_fns[] = {
+  Opcode_syscall_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_simcall_encode_fns[] = {
+  Opcode_simcall_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_add_n_encode_fns[] = {
+  0, Opcode_add_n_Slot_inst16a_encode, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_addi_n_encode_fns[] = {
+  0, Opcode_addi_n_Slot_inst16a_encode, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_beqz_n_encode_fns[] = {
+  0, 0, Opcode_beqz_n_Slot_inst16b_encode
+};
+
+static xtensa_opcode_encode_fn Opcode_bnez_n_encode_fns[] = {
+  0, 0, Opcode_bnez_n_Slot_inst16b_encode
+};
+
+static xtensa_opcode_encode_fn Opcode_ill_n_encode_fns[] = {
+  0, 0, Opcode_ill_n_Slot_inst16b_encode
+};
+
+static xtensa_opcode_encode_fn Opcode_l32i_n_encode_fns[] = {
+  0, Opcode_l32i_n_Slot_inst16a_encode, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_mov_n_encode_fns[] = {
+  0, 0, Opcode_mov_n_Slot_inst16b_encode
+};
+
+static xtensa_opcode_encode_fn Opcode_movi_n_encode_fns[] = {
+  0, 0, Opcode_movi_n_Slot_inst16b_encode
+};
+
+static xtensa_opcode_encode_fn Opcode_nop_n_encode_fns[] = {
+  0, 0, Opcode_nop_n_Slot_inst16b_encode
+};
+
+static xtensa_opcode_encode_fn Opcode_ret_n_encode_fns[] = {
+  0, 0, Opcode_ret_n_Slot_inst16b_encode
+};
+
+static xtensa_opcode_encode_fn Opcode_s32i_n_encode_fns[] = {
+  0, Opcode_s32i_n_Slot_inst16a_encode, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_addi_encode_fns[] = {
+  Opcode_addi_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_addmi_encode_fns[] = {
+  Opcode_addmi_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_add_encode_fns[] = {
+  Opcode_add_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_sub_encode_fns[] = {
+  Opcode_sub_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_addx2_encode_fns[] = {
+  Opcode_addx2_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_addx4_encode_fns[] = {
+  Opcode_addx4_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_addx8_encode_fns[] = {
+  Opcode_addx8_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_subx2_encode_fns[] = {
+  Opcode_subx2_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_subx4_encode_fns[] = {
+  Opcode_subx4_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_subx8_encode_fns[] = {
+  Opcode_subx8_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_and_encode_fns[] = {
+  Opcode_and_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_or_encode_fns[] = {
+  Opcode_or_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xor_encode_fns[] = {
+  Opcode_xor_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_beqi_encode_fns[] = {
+  Opcode_beqi_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bnei_encode_fns[] = {
+  Opcode_bnei_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bgei_encode_fns[] = {
+  Opcode_bgei_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_blti_encode_fns[] = {
+  Opcode_blti_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bbci_encode_fns[] = {
+  Opcode_bbci_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bbsi_encode_fns[] = {
+  Opcode_bbsi_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bgeui_encode_fns[] = {
+  Opcode_bgeui_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bltui_encode_fns[] = {
+  Opcode_bltui_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_beq_encode_fns[] = {
+  Opcode_beq_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bne_encode_fns[] = {
+  Opcode_bne_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bge_encode_fns[] = {
+  Opcode_bge_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_blt_encode_fns[] = {
+  Opcode_blt_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bgeu_encode_fns[] = {
+  Opcode_bgeu_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bltu_encode_fns[] = {
+  Opcode_bltu_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bany_encode_fns[] = {
+  Opcode_bany_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bnone_encode_fns[] = {
+  Opcode_bnone_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_ball_encode_fns[] = {
+  Opcode_ball_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bnall_encode_fns[] = {
+  Opcode_bnall_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bbc_encode_fns[] = {
+  Opcode_bbc_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bbs_encode_fns[] = {
+  Opcode_bbs_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_beqz_encode_fns[] = {
+  Opcode_beqz_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bnez_encode_fns[] = {
+  Opcode_bnez_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bgez_encode_fns[] = {
+  Opcode_bgez_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_bltz_encode_fns[] = {
+  Opcode_bltz_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_call0_encode_fns[] = {
+  Opcode_call0_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_callx0_encode_fns[] = {
+  Opcode_callx0_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_extui_encode_fns[] = {
+  Opcode_extui_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_ill_encode_fns[] = {
+  Opcode_ill_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_j_encode_fns[] = {
+  Opcode_j_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_jx_encode_fns[] = {
+  Opcode_jx_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_l16ui_encode_fns[] = {
+  Opcode_l16ui_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_l16si_encode_fns[] = {
+  Opcode_l16si_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_l32i_encode_fns[] = {
+  Opcode_l32i_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_l32r_encode_fns[] = {
+  Opcode_l32r_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_l8ui_encode_fns[] = {
+  Opcode_l8ui_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_movi_encode_fns[] = {
+  Opcode_movi_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_moveqz_encode_fns[] = {
+  Opcode_moveqz_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_movnez_encode_fns[] = {
+  Opcode_movnez_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_movltz_encode_fns[] = {
+  Opcode_movltz_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_movgez_encode_fns[] = {
+  Opcode_movgez_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_neg_encode_fns[] = {
+  Opcode_neg_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_abs_encode_fns[] = {
+  Opcode_abs_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_nop_encode_fns[] = {
+  Opcode_nop_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_ret_encode_fns[] = {
+  Opcode_ret_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_s16i_encode_fns[] = {
+  Opcode_s16i_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_s32i_encode_fns[] = {
+  Opcode_s32i_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_s8i_encode_fns[] = {
+  Opcode_s8i_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_ssr_encode_fns[] = {
+  Opcode_ssr_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_ssl_encode_fns[] = {
+  Opcode_ssl_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_ssa8l_encode_fns[] = {
+  Opcode_ssa8l_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_ssa8b_encode_fns[] = {
+  Opcode_ssa8b_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_ssai_encode_fns[] = {
+  Opcode_ssai_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_sll_encode_fns[] = {
+  Opcode_sll_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_src_encode_fns[] = {
+  Opcode_src_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_srl_encode_fns[] = {
+  Opcode_srl_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_sra_encode_fns[] = {
+  Opcode_sra_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_slli_encode_fns[] = {
+  Opcode_slli_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_srai_encode_fns[] = {
+  Opcode_srai_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_srli_encode_fns[] = {
+  Opcode_srli_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_memw_encode_fns[] = {
+  Opcode_memw_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_extw_encode_fns[] = {
+  Opcode_extw_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_isync_encode_fns[] = {
+  Opcode_isync_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsync_encode_fns[] = {
+  Opcode_rsync_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_esync_encode_fns[] = {
+  Opcode_esync_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_dsync_encode_fns[] = {
+  Opcode_dsync_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsil_encode_fns[] = {
+  Opcode_rsil_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_sar_encode_fns[] = {
+  Opcode_rsr_sar_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_sar_encode_fns[] = {
+  Opcode_wsr_sar_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_sar_encode_fns[] = {
+  Opcode_xsr_sar_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_litbase_encode_fns[] = {
+  Opcode_rsr_litbase_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_litbase_encode_fns[] = {
+  Opcode_wsr_litbase_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_litbase_encode_fns[] = {
+  Opcode_xsr_litbase_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_176_encode_fns[] = {
+  Opcode_rsr_176_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_176_encode_fns[] = {
+  Opcode_wsr_176_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_208_encode_fns[] = {
+  Opcode_rsr_208_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_ps_encode_fns[] = {
+  Opcode_rsr_ps_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_ps_encode_fns[] = {
+  Opcode_wsr_ps_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_ps_encode_fns[] = {
+  Opcode_xsr_ps_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_epc1_encode_fns[] = {
+  Opcode_rsr_epc1_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_epc1_encode_fns[] = {
+  Opcode_wsr_epc1_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_epc1_encode_fns[] = {
+  Opcode_xsr_epc1_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_excsave1_encode_fns[] = {
+  Opcode_rsr_excsave1_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_excsave1_encode_fns[] = {
+  Opcode_wsr_excsave1_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_excsave1_encode_fns[] = {
+  Opcode_xsr_excsave1_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_epc2_encode_fns[] = {
+  Opcode_rsr_epc2_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_epc2_encode_fns[] = {
+  Opcode_wsr_epc2_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_epc2_encode_fns[] = {
+  Opcode_xsr_epc2_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_excsave2_encode_fns[] = {
+  Opcode_rsr_excsave2_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_excsave2_encode_fns[] = {
+  Opcode_wsr_excsave2_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_excsave2_encode_fns[] = {
+  Opcode_xsr_excsave2_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_epc3_encode_fns[] = {
+  Opcode_rsr_epc3_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_epc3_encode_fns[] = {
+  Opcode_wsr_epc3_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_epc3_encode_fns[] = {
+  Opcode_xsr_epc3_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_excsave3_encode_fns[] = {
+  Opcode_rsr_excsave3_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_excsave3_encode_fns[] = {
+  Opcode_wsr_excsave3_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_excsave3_encode_fns[] = {
+  Opcode_xsr_excsave3_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_eps2_encode_fns[] = {
+  Opcode_rsr_eps2_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_eps2_encode_fns[] = {
+  Opcode_wsr_eps2_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_eps2_encode_fns[] = {
+  Opcode_xsr_eps2_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_eps3_encode_fns[] = {
+  Opcode_rsr_eps3_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_eps3_encode_fns[] = {
+  Opcode_wsr_eps3_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_eps3_encode_fns[] = {
+  Opcode_xsr_eps3_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_excvaddr_encode_fns[] = {
+  Opcode_rsr_excvaddr_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_excvaddr_encode_fns[] = {
+  Opcode_wsr_excvaddr_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_excvaddr_encode_fns[] = {
+  Opcode_xsr_excvaddr_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_depc_encode_fns[] = {
+  Opcode_rsr_depc_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_depc_encode_fns[] = {
+  Opcode_wsr_depc_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_depc_encode_fns[] = {
+  Opcode_xsr_depc_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_exccause_encode_fns[] = {
+  Opcode_rsr_exccause_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_exccause_encode_fns[] = {
+  Opcode_wsr_exccause_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_exccause_encode_fns[] = {
+  Opcode_xsr_exccause_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_prid_encode_fns[] = {
+  Opcode_rsr_prid_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_vecbase_encode_fns[] = {
+  Opcode_rsr_vecbase_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_vecbase_encode_fns[] = {
+  Opcode_wsr_vecbase_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_vecbase_encode_fns[] = {
+  Opcode_xsr_vecbase_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_mul16u_encode_fns[] = {
+  Opcode_mul16u_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_mul16s_encode_fns[] = {
+  Opcode_mul16s_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_mull_encode_fns[] = {
+  Opcode_mull_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rfi_encode_fns[] = {
+  Opcode_rfi_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_waiti_encode_fns[] = {
+  Opcode_waiti_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_interrupt_encode_fns[] = {
+  Opcode_rsr_interrupt_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_intset_encode_fns[] = {
+  Opcode_wsr_intset_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_intclear_encode_fns[] = {
+  Opcode_wsr_intclear_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_intenable_encode_fns[] = {
+  Opcode_rsr_intenable_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_intenable_encode_fns[] = {
+  Opcode_wsr_intenable_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_intenable_encode_fns[] = {
+  Opcode_xsr_intenable_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_break_encode_fns[] = {
+  Opcode_break_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_break_n_encode_fns[] = {
+  0, 0, Opcode_break_n_Slot_inst16b_encode
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_dbreaka0_encode_fns[] = {
+  Opcode_rsr_dbreaka0_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_dbreaka0_encode_fns[] = {
+  Opcode_wsr_dbreaka0_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_dbreaka0_encode_fns[] = {
+  Opcode_xsr_dbreaka0_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_dbreakc0_encode_fns[] = {
+  Opcode_rsr_dbreakc0_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_dbreakc0_encode_fns[] = {
+  Opcode_wsr_dbreakc0_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_dbreakc0_encode_fns[] = {
+  Opcode_xsr_dbreakc0_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_ibreaka0_encode_fns[] = {
+  Opcode_rsr_ibreaka0_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_ibreaka0_encode_fns[] = {
+  Opcode_wsr_ibreaka0_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_ibreaka0_encode_fns[] = {
+  Opcode_xsr_ibreaka0_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_ibreakenable_encode_fns[] = {
+  Opcode_rsr_ibreakenable_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_ibreakenable_encode_fns[] = {
+  Opcode_wsr_ibreakenable_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_ibreakenable_encode_fns[] = {
+  Opcode_xsr_ibreakenable_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_debugcause_encode_fns[] = {
+  Opcode_rsr_debugcause_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_debugcause_encode_fns[] = {
+  Opcode_wsr_debugcause_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_debugcause_encode_fns[] = {
+  Opcode_xsr_debugcause_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_icount_encode_fns[] = {
+  Opcode_rsr_icount_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_icount_encode_fns[] = {
+  Opcode_wsr_icount_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_icount_encode_fns[] = {
+  Opcode_xsr_icount_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_icountlevel_encode_fns[] = {
+  Opcode_rsr_icountlevel_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_icountlevel_encode_fns[] = {
+  Opcode_wsr_icountlevel_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_icountlevel_encode_fns[] = {
+  Opcode_xsr_icountlevel_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_ddr_encode_fns[] = {
+  Opcode_rsr_ddr_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_ddr_encode_fns[] = {
+  Opcode_wsr_ddr_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_ddr_encode_fns[] = {
+  Opcode_xsr_ddr_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rfdo_encode_fns[] = {
+  Opcode_rfdo_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rfdd_encode_fns[] = {
+  Opcode_rfdd_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_mmid_encode_fns[] = {
+  Opcode_wsr_mmid_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_ccount_encode_fns[] = {
+  Opcode_rsr_ccount_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_ccount_encode_fns[] = {
+  Opcode_wsr_ccount_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_ccount_encode_fns[] = {
+  Opcode_xsr_ccount_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rsr_ccompare0_encode_fns[] = {
+  Opcode_rsr_ccompare0_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wsr_ccompare0_encode_fns[] = {
+  Opcode_wsr_ccompare0_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_xsr_ccompare0_encode_fns[] = {
+  Opcode_xsr_ccompare0_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_idtlb_encode_fns[] = {
+  Opcode_idtlb_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_pdtlb_encode_fns[] = {
+  Opcode_pdtlb_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rdtlb0_encode_fns[] = {
+  Opcode_rdtlb0_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rdtlb1_encode_fns[] = {
+  Opcode_rdtlb1_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wdtlb_encode_fns[] = {
+  Opcode_wdtlb_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_iitlb_encode_fns[] = {
+  Opcode_iitlb_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_pitlb_encode_fns[] = {
+  Opcode_pitlb_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_ritlb0_encode_fns[] = {
+  Opcode_ritlb0_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_ritlb1_encode_fns[] = {
+  Opcode_ritlb1_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_witlb_encode_fns[] = {
+  Opcode_witlb_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_nsa_encode_fns[] = {
+  Opcode_nsa_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_nsau_encode_fns[] = {
+  Opcode_nsau_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_rer_encode_fns[] = {
+  Opcode_rer_Slot_inst_encode, 0, 0
+};
+
+static xtensa_opcode_encode_fn Opcode_wer_encode_fns[] = {
+  Opcode_wer_Slot_inst_encode, 0, 0
+};
+
+
+/* Opcode table.  */
+
+static xtensa_opcode_internal opcodes[] = {
+  { "excw", ICLASS_xt_iclass_excw,
+    0,
+    Opcode_excw_encode_fns, 0, 0 },
+  { "rfe", ICLASS_xt_iclass_rfe,
+    XTENSA_OPCODE_IS_JUMP,
+    Opcode_rfe_encode_fns, 0, 0 },
+  { "rfde", ICLASS_xt_iclass_rfde,
+    XTENSA_OPCODE_IS_JUMP,
+    Opcode_rfde_encode_fns, 0, 0 },
+  { "syscall", ICLASS_xt_iclass_syscall,
+    0,
+    Opcode_syscall_encode_fns, 0, 0 },
+  { "simcall", ICLASS_xt_iclass_simcall,
+    0,
+    Opcode_simcall_encode_fns, 0, 0 },
+  { "add.n", ICLASS_xt_iclass_add_n,
+    0,
+    Opcode_add_n_encode_fns, 0, 0 },
+  { "addi.n", ICLASS_xt_iclass_addi_n,
+    0,
+    Opcode_addi_n_encode_fns, 0, 0 },
+  { "beqz.n", ICLASS_xt_iclass_bz6,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_beqz_n_encode_fns, 0, 0 },
+  { "bnez.n", ICLASS_xt_iclass_bz6,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bnez_n_encode_fns, 0, 0 },
+  { "ill.n", ICLASS_xt_iclass_ill_n,
+    0,
+    Opcode_ill_n_encode_fns, 0, 0 },
+  { "l32i.n", ICLASS_xt_iclass_loadi4,
+    0,
+    Opcode_l32i_n_encode_fns, 0, 0 },
+  { "mov.n", ICLASS_xt_iclass_mov_n,
+    0,
+    Opcode_mov_n_encode_fns, 0, 0 },
+  { "movi.n", ICLASS_xt_iclass_movi_n,
+    0,
+    Opcode_movi_n_encode_fns, 0, 0 },
+  { "nop.n", ICLASS_xt_iclass_nopn,
+    0,
+    Opcode_nop_n_encode_fns, 0, 0 },
+  { "ret.n", ICLASS_xt_iclass_retn,
+    XTENSA_OPCODE_IS_JUMP,
+    Opcode_ret_n_encode_fns, 0, 0 },
+  { "s32i.n", ICLASS_xt_iclass_storei4,
+    0,
+    Opcode_s32i_n_encode_fns, 0, 0 },
+  { "addi", ICLASS_xt_iclass_addi,
+    0,
+    Opcode_addi_encode_fns, 0, 0 },
+  { "addmi", ICLASS_xt_iclass_addmi,
+    0,
+    Opcode_addmi_encode_fns, 0, 0 },
+  { "add", ICLASS_xt_iclass_addsub,
+    0,
+    Opcode_add_encode_fns, 0, 0 },
+  { "sub", ICLASS_xt_iclass_addsub,
+    0,
+    Opcode_sub_encode_fns, 0, 0 },
+  { "addx2", ICLASS_xt_iclass_addsub,
+    0,
+    Opcode_addx2_encode_fns, 0, 0 },
+  { "addx4", ICLASS_xt_iclass_addsub,
+    0,
+    Opcode_addx4_encode_fns, 0, 0 },
+  { "addx8", ICLASS_xt_iclass_addsub,
+    0,
+    Opcode_addx8_encode_fns, 0, 0 },
+  { "subx2", ICLASS_xt_iclass_addsub,
+    0,
+    Opcode_subx2_encode_fns, 0, 0 },
+  { "subx4", ICLASS_xt_iclass_addsub,
+    0,
+    Opcode_subx4_encode_fns, 0, 0 },
+  { "subx8", ICLASS_xt_iclass_addsub,
+    0,
+    Opcode_subx8_encode_fns, 0, 0 },
+  { "and", ICLASS_xt_iclass_bit,
+    0,
+    Opcode_and_encode_fns, 0, 0 },
+  { "or", ICLASS_xt_iclass_bit,
+    0,
+    Opcode_or_encode_fns, 0, 0 },
+  { "xor", ICLASS_xt_iclass_bit,
+    0,
+    Opcode_xor_encode_fns, 0, 0 },
+  { "beqi", ICLASS_xt_iclass_bsi8,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_beqi_encode_fns, 0, 0 },
+  { "bnei", ICLASS_xt_iclass_bsi8,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bnei_encode_fns, 0, 0 },
+  { "bgei", ICLASS_xt_iclass_bsi8,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bgei_encode_fns, 0, 0 },
+  { "blti", ICLASS_xt_iclass_bsi8,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_blti_encode_fns, 0, 0 },
+  { "bbci", ICLASS_xt_iclass_bsi8b,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bbci_encode_fns, 0, 0 },
+  { "bbsi", ICLASS_xt_iclass_bsi8b,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bbsi_encode_fns, 0, 0 },
+  { "bgeui", ICLASS_xt_iclass_bsi8u,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bgeui_encode_fns, 0, 0 },
+  { "bltui", ICLASS_xt_iclass_bsi8u,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bltui_encode_fns, 0, 0 },
+  { "beq", ICLASS_xt_iclass_bst8,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_beq_encode_fns, 0, 0 },
+  { "bne", ICLASS_xt_iclass_bst8,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bne_encode_fns, 0, 0 },
+  { "bge", ICLASS_xt_iclass_bst8,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bge_encode_fns, 0, 0 },
+  { "blt", ICLASS_xt_iclass_bst8,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_blt_encode_fns, 0, 0 },
+  { "bgeu", ICLASS_xt_iclass_bst8,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bgeu_encode_fns, 0, 0 },
+  { "bltu", ICLASS_xt_iclass_bst8,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bltu_encode_fns, 0, 0 },
+  { "bany", ICLASS_xt_iclass_bst8,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bany_encode_fns, 0, 0 },
+  { "bnone", ICLASS_xt_iclass_bst8,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bnone_encode_fns, 0, 0 },
+  { "ball", ICLASS_xt_iclass_bst8,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_ball_encode_fns, 0, 0 },
+  { "bnall", ICLASS_xt_iclass_bst8,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bnall_encode_fns, 0, 0 },
+  { "bbc", ICLASS_xt_iclass_bst8,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bbc_encode_fns, 0, 0 },
+  { "bbs", ICLASS_xt_iclass_bst8,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bbs_encode_fns, 0, 0 },
+  { "beqz", ICLASS_xt_iclass_bsz12,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_beqz_encode_fns, 0, 0 },
+  { "bnez", ICLASS_xt_iclass_bsz12,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bnez_encode_fns, 0, 0 },
+  { "bgez", ICLASS_xt_iclass_bsz12,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bgez_encode_fns, 0, 0 },
+  { "bltz", ICLASS_xt_iclass_bsz12,
+    XTENSA_OPCODE_IS_BRANCH,
+    Opcode_bltz_encode_fns, 0, 0 },
+  { "call0", ICLASS_xt_iclass_call0,
+    XTENSA_OPCODE_IS_CALL,
+    Opcode_call0_encode_fns, 0, 0 },
+  { "callx0", ICLASS_xt_iclass_callx0,
+    XTENSA_OPCODE_IS_CALL,
+    Opcode_callx0_encode_fns, 0, 0 },
+  { "extui", ICLASS_xt_iclass_exti,
+    0,
+    Opcode_extui_encode_fns, 0, 0 },
+  { "ill", ICLASS_xt_iclass_ill,
+    0,
+    Opcode_ill_encode_fns, 0, 0 },
+  { "j", ICLASS_xt_iclass_jump,
+    XTENSA_OPCODE_IS_JUMP,
+    Opcode_j_encode_fns, 0, 0 },
+  { "jx", ICLASS_xt_iclass_jumpx,
+    XTENSA_OPCODE_IS_JUMP,
+    Opcode_jx_encode_fns, 0, 0 },
+  { "l16ui", ICLASS_xt_iclass_l16ui,
+    0,
+    Opcode_l16ui_encode_fns, 0, 0 },
+  { "l16si", ICLASS_xt_iclass_l16si,
+    0,
+    Opcode_l16si_encode_fns, 0, 0 },
+  { "l32i", ICLASS_xt_iclass_l32i,
+    0,
+    Opcode_l32i_encode_fns, 0, 0 },
+  { "l32r", ICLASS_xt_iclass_l32r,
+    0,
+    Opcode_l32r_encode_fns, 0, 0 },
+  { "l8ui", ICLASS_xt_iclass_l8i,
+    0,
+    Opcode_l8ui_encode_fns, 0, 0 },
+  { "movi", ICLASS_xt_iclass_movi,
+    0,
+    Opcode_movi_encode_fns, 0, 0 },
+  { "moveqz", ICLASS_xt_iclass_movz,
+    0,
+    Opcode_moveqz_encode_fns, 0, 0 },
+  { "movnez", ICLASS_xt_iclass_movz,
+    0,
+    Opcode_movnez_encode_fns, 0, 0 },
+  { "movltz", ICLASS_xt_iclass_movz,
+    0,
+    Opcode_movltz_encode_fns, 0, 0 },
+  { "movgez", ICLASS_xt_iclass_movz,
+    0,
+    Opcode_movgez_encode_fns, 0, 0 },
+  { "neg", ICLASS_xt_iclass_neg,
+    0,
+    Opcode_neg_encode_fns, 0, 0 },
+  { "abs", ICLASS_xt_iclass_neg,
+    0,
+    Opcode_abs_encode_fns, 0, 0 },
+  { "nop", ICLASS_xt_iclass_nop,
+    0,
+    Opcode_nop_encode_fns, 0, 0 },
+  { "ret", ICLASS_xt_iclass_return,
+    XTENSA_OPCODE_IS_JUMP,
+    Opcode_ret_encode_fns, 0, 0 },
+  { "s16i", ICLASS_xt_iclass_s16i,
+    0,
+    Opcode_s16i_encode_fns, 0, 0 },
+  { "s32i", ICLASS_xt_iclass_s32i,
+    0,
+    Opcode_s32i_encode_fns, 0, 0 },
+  { "s8i", ICLASS_xt_iclass_s8i,
+    0,
+    Opcode_s8i_encode_fns, 0, 0 },
+  { "ssr", ICLASS_xt_iclass_sar,
+    0,
+    Opcode_ssr_encode_fns, 0, 0 },
+  { "ssl", ICLASS_xt_iclass_sar,
+    0,
+    Opcode_ssl_encode_fns, 0, 0 },
+  { "ssa8l", ICLASS_xt_iclass_sar,
+    0,
+    Opcode_ssa8l_encode_fns, 0, 0 },
+  { "ssa8b", ICLASS_xt_iclass_sar,
+    0,
+    Opcode_ssa8b_encode_fns, 0, 0 },
+  { "ssai", ICLASS_xt_iclass_sari,
+    0,
+    Opcode_ssai_encode_fns, 0, 0 },
+  { "sll", ICLASS_xt_iclass_shifts,
+    0,
+    Opcode_sll_encode_fns, 0, 0 },
+  { "src", ICLASS_xt_iclass_shiftst,
+    0,
+    Opcode_src_encode_fns, 0, 0 },
+  { "srl", ICLASS_xt_iclass_shiftt,
+    0,
+    Opcode_srl_encode_fns, 0, 0 },
+  { "sra", ICLASS_xt_iclass_shiftt,
+    0,
+    Opcode_sra_encode_fns, 0, 0 },
+  { "slli", ICLASS_xt_iclass_slli,
+    0,
+    Opcode_slli_encode_fns, 0, 0 },
+  { "srai", ICLASS_xt_iclass_srai,
+    0,
+    Opcode_srai_encode_fns, 0, 0 },
+  { "srli", ICLASS_xt_iclass_srli,
+    0,
+    Opcode_srli_encode_fns, 0, 0 },
+  { "memw", ICLASS_xt_iclass_memw,
+    0,
+    Opcode_memw_encode_fns, 0, 0 },
+  { "extw", ICLASS_xt_iclass_extw,
+    0,
+    Opcode_extw_encode_fns, 0, 0 },
+  { "isync", ICLASS_xt_iclass_isync,
+    0,
+    Opcode_isync_encode_fns, 0, 0 },
+  { "rsync", ICLASS_xt_iclass_sync,
+    0,
+    Opcode_rsync_encode_fns, 0, 0 },
+  { "esync", ICLASS_xt_iclass_sync,
+    0,
+    Opcode_esync_encode_fns, 0, 0 },
+  { "dsync", ICLASS_xt_iclass_sync,
+    0,
+    Opcode_dsync_encode_fns, 0, 0 },
+  { "rsil", ICLASS_xt_iclass_rsil,
+    0,
+    Opcode_rsil_encode_fns, 0, 0 },
+  { "rsr.sar", ICLASS_xt_iclass_rsr_sar,
+    0,
+    Opcode_rsr_sar_encode_fns, 0, 0 },
+  { "wsr.sar", ICLASS_xt_iclass_wsr_sar,
+    0,
+    Opcode_wsr_sar_encode_fns, 0, 0 },
+  { "xsr.sar", ICLASS_xt_iclass_xsr_sar,
+    0,
+    Opcode_xsr_sar_encode_fns, 0, 0 },
+  { "rsr.litbase", ICLASS_xt_iclass_rsr_litbase,
+    0,
+    Opcode_rsr_litbase_encode_fns, 0, 0 },
+  { "wsr.litbase", ICLASS_xt_iclass_wsr_litbase,
+    0,
+    Opcode_wsr_litbase_encode_fns, 0, 0 },
+  { "xsr.litbase", ICLASS_xt_iclass_xsr_litbase,
+    0,
+    Opcode_xsr_litbase_encode_fns, 0, 0 },
+  { "rsr.176", ICLASS_xt_iclass_rsr_176,
+    0,
+    Opcode_rsr_176_encode_fns, 0, 0 },
+  { "wsr.176", ICLASS_xt_iclass_wsr_176,
+    0,
+    Opcode_wsr_176_encode_fns, 0, 0 },
+  { "rsr.208", ICLASS_xt_iclass_rsr_208,
+    0,
+    Opcode_rsr_208_encode_fns, 0, 0 },
+  { "rsr.ps", ICLASS_xt_iclass_rsr_ps,
+    0,
+    Opcode_rsr_ps_encode_fns, 0, 0 },
+  { "wsr.ps", ICLASS_xt_iclass_wsr_ps,
+    0,
+    Opcode_wsr_ps_encode_fns, 0, 0 },
+  { "xsr.ps", ICLASS_xt_iclass_xsr_ps,
+    0,
+    Opcode_xsr_ps_encode_fns, 0, 0 },
+  { "rsr.epc1", ICLASS_xt_iclass_rsr_epc1,
+    0,
+    Opcode_rsr_epc1_encode_fns, 0, 0 },
+  { "wsr.epc1", ICLASS_xt_iclass_wsr_epc1,
+    0,
+    Opcode_wsr_epc1_encode_fns, 0, 0 },
+  { "xsr.epc1", ICLASS_xt_iclass_xsr_epc1,
+    0,
+    Opcode_xsr_epc1_encode_fns, 0, 0 },
+  { "rsr.excsave1", ICLASS_xt_iclass_rsr_excsave1,
+    0,
+    Opcode_rsr_excsave1_encode_fns, 0, 0 },
+  { "wsr.excsave1", ICLASS_xt_iclass_wsr_excsave1,
+    0,
+    Opcode_wsr_excsave1_encode_fns, 0, 0 },
+  { "xsr.excsave1", ICLASS_xt_iclass_xsr_excsave1,
+    0,
+    Opcode_xsr_excsave1_encode_fns, 0, 0 },
+  { "rsr.epc2", ICLASS_xt_iclass_rsr_epc2,
+    0,
+    Opcode_rsr_epc2_encode_fns, 0, 0 },
+  { "wsr.epc2", ICLASS_xt_iclass_wsr_epc2,
+    0,
+    Opcode_wsr_epc2_encode_fns, 0, 0 },
+  { "xsr.epc2", ICLASS_xt_iclass_xsr_epc2,
+    0,
+    Opcode_xsr_epc2_encode_fns, 0, 0 },
+  { "rsr.excsave2", ICLASS_xt_iclass_rsr_excsave2,
+    0,
+    Opcode_rsr_excsave2_encode_fns, 0, 0 },
+  { "wsr.excsave2", ICLASS_xt_iclass_wsr_excsave2,
+    0,
+    Opcode_wsr_excsave2_encode_fns, 0, 0 },
+  { "xsr.excsave2", ICLASS_xt_iclass_xsr_excsave2,
+    0,
+    Opcode_xsr_excsave2_encode_fns, 0, 0 },
+  { "rsr.epc3", ICLASS_xt_iclass_rsr_epc3,
+    0,
+    Opcode_rsr_epc3_encode_fns, 0, 0 },
+  { "wsr.epc3", ICLASS_xt_iclass_wsr_epc3,
+    0,
+    Opcode_wsr_epc3_encode_fns, 0, 0 },
+  { "xsr.epc3", ICLASS_xt_iclass_xsr_epc3,
+    0,
+    Opcode_xsr_epc3_encode_fns, 0, 0 },
+  { "rsr.excsave3", ICLASS_xt_iclass_rsr_excsave3,
+    0,
+    Opcode_rsr_excsave3_encode_fns, 0, 0 },
+  { "wsr.excsave3", ICLASS_xt_iclass_wsr_excsave3,
+    0,
+    Opcode_wsr_excsave3_encode_fns, 0, 0 },
+  { "xsr.excsave3", ICLASS_xt_iclass_xsr_excsave3,
+    0,
+    Opcode_xsr_excsave3_encode_fns, 0, 0 },
+  { "rsr.eps2", ICLASS_xt_iclass_rsr_eps2,
+    0,
+    Opcode_rsr_eps2_encode_fns, 0, 0 },
+  { "wsr.eps2", ICLASS_xt_iclass_wsr_eps2,
+    0,
+    Opcode_wsr_eps2_encode_fns, 0, 0 },
+  { "xsr.eps2", ICLASS_xt_iclass_xsr_eps2,
+    0,
+    Opcode_xsr_eps2_encode_fns, 0, 0 },
+  { "rsr.eps3", ICLASS_xt_iclass_rsr_eps3,
+    0,
+    Opcode_rsr_eps3_encode_fns, 0, 0 },
+  { "wsr.eps3", ICLASS_xt_iclass_wsr_eps3,
+    0,
+    Opcode_wsr_eps3_encode_fns, 0, 0 },
+  { "xsr.eps3", ICLASS_xt_iclass_xsr_eps3,
+    0,
+    Opcode_xsr_eps3_encode_fns, 0, 0 },
+  { "rsr.excvaddr", ICLASS_xt_iclass_rsr_excvaddr,
+    0,
+    Opcode_rsr_excvaddr_encode_fns, 0, 0 },
+  { "wsr.excvaddr", ICLASS_xt_iclass_wsr_excvaddr,
+    0,
+    Opcode_wsr_excvaddr_encode_fns, 0, 0 },
+  { "xsr.excvaddr", ICLASS_xt_iclass_xsr_excvaddr,
+    0,
+    Opcode_xsr_excvaddr_encode_fns, 0, 0 },
+  { "rsr.depc", ICLASS_xt_iclass_rsr_depc,
+    0,
+    Opcode_rsr_depc_encode_fns, 0, 0 },
+  { "wsr.depc", ICLASS_xt_iclass_wsr_depc,
+    0,
+    Opcode_wsr_depc_encode_fns, 0, 0 },
+  { "xsr.depc", ICLASS_xt_iclass_xsr_depc,
+    0,
+    Opcode_xsr_depc_encode_fns, 0, 0 },
+  { "rsr.exccause", ICLASS_xt_iclass_rsr_exccause,
+    0,
+    Opcode_rsr_exccause_encode_fns, 0, 0 },
+  { "wsr.exccause", ICLASS_xt_iclass_wsr_exccause,
+    0,
+    Opcode_wsr_exccause_encode_fns, 0, 0 },
+  { "xsr.exccause", ICLASS_xt_iclass_xsr_exccause,
+    0,
+    Opcode_xsr_exccause_encode_fns, 0, 0 },
+  { "rsr.prid", ICLASS_xt_iclass_rsr_prid,
+    0,
+    Opcode_rsr_prid_encode_fns, 0, 0 },
+  { "rsr.vecbase", ICLASS_xt_iclass_rsr_vecbase,
+    0,
+    Opcode_rsr_vecbase_encode_fns, 0, 0 },
+  { "wsr.vecbase", ICLASS_xt_iclass_wsr_vecbase,
+    0,
+    Opcode_wsr_vecbase_encode_fns, 0, 0 },
+  { "xsr.vecbase", ICLASS_xt_iclass_xsr_vecbase,
+    0,
+    Opcode_xsr_vecbase_encode_fns, 0, 0 },
+  { "mul16u", ICLASS_xt_mul16,
+    0,
+    Opcode_mul16u_encode_fns, 0, 0 },
+  { "mul16s", ICLASS_xt_mul16,
+    0,
+    Opcode_mul16s_encode_fns, 0, 0 },
+  { "mull", ICLASS_xt_mul32,
+    0,
+    Opcode_mull_encode_fns, 0, 0 },
+  { "rfi", ICLASS_xt_iclass_rfi,
+    XTENSA_OPCODE_IS_JUMP,
+    Opcode_rfi_encode_fns, 0, 0 },
+  { "waiti", ICLASS_xt_iclass_wait,
+    0,
+    Opcode_waiti_encode_fns, 0, 0 },
+  { "rsr.interrupt", ICLASS_xt_iclass_rsr_interrupt,
+    0,
+    Opcode_rsr_interrupt_encode_fns, 0, 0 },
+  { "wsr.intset", ICLASS_xt_iclass_wsr_intset,
+    0,
+    Opcode_wsr_intset_encode_fns, 0, 0 },
+  { "wsr.intclear", ICLASS_xt_iclass_wsr_intclear,
+    0,
+    Opcode_wsr_intclear_encode_fns, 0, 0 },
+  { "rsr.intenable", ICLASS_xt_iclass_rsr_intenable,
+    0,
+    Opcode_rsr_intenable_encode_fns, 0, 0 },
+  { "wsr.intenable", ICLASS_xt_iclass_wsr_intenable,
+    0,
+    Opcode_wsr_intenable_encode_fns, 0, 0 },
+  { "xsr.intenable", ICLASS_xt_iclass_xsr_intenable,
+    0,
+    Opcode_xsr_intenable_encode_fns, 0, 0 },
+  { "break", ICLASS_xt_iclass_break,
+    0,
+    Opcode_break_encode_fns, 0, 0 },
+  { "break.n", ICLASS_xt_iclass_break_n,
+    0,
+    Opcode_break_n_encode_fns, 0, 0 },
+  { "rsr.dbreaka0", ICLASS_xt_iclass_rsr_dbreaka0,
+    0,
+    Opcode_rsr_dbreaka0_encode_fns, 0, 0 },
+  { "wsr.dbreaka0", ICLASS_xt_iclass_wsr_dbreaka0,
+    0,
+    Opcode_wsr_dbreaka0_encode_fns, 0, 0 },
+  { "xsr.dbreaka0", ICLASS_xt_iclass_xsr_dbreaka0,
+    0,
+    Opcode_xsr_dbreaka0_encode_fns, 0, 0 },
+  { "rsr.dbreakc0", ICLASS_xt_iclass_rsr_dbreakc0,
+    0,
+    Opcode_rsr_dbreakc0_encode_fns, 0, 0 },
+  { "wsr.dbreakc0", ICLASS_xt_iclass_wsr_dbreakc0,
+    0,
+    Opcode_wsr_dbreakc0_encode_fns, 0, 0 },
+  { "xsr.dbreakc0", ICLASS_xt_iclass_xsr_dbreakc0,
+    0,
+    Opcode_xsr_dbreakc0_encode_fns, 0, 0 },
+  { "rsr.ibreaka0", ICLASS_xt_iclass_rsr_ibreaka0,
+    0,
+    Opcode_rsr_ibreaka0_encode_fns, 0, 0 },
+  { "wsr.ibreaka0", ICLASS_xt_iclass_wsr_ibreaka0,
+    0,
+    Opcode_wsr_ibreaka0_encode_fns, 0, 0 },
+  { "xsr.ibreaka0", ICLASS_xt_iclass_xsr_ibreaka0,
+    0,
+    Opcode_xsr_ibreaka0_encode_fns, 0, 0 },
+  { "rsr.ibreakenable", ICLASS_xt_iclass_rsr_ibreakenable,
+    0,
+    Opcode_rsr_ibreakenable_encode_fns, 0, 0 },
+  { "wsr.ibreakenable", ICLASS_xt_iclass_wsr_ibreakenable,
+    0,
+    Opcode_wsr_ibreakenable_encode_fns, 0, 0 },
+  { "xsr.ibreakenable", ICLASS_xt_iclass_xsr_ibreakenable,
+    0,
+    Opcode_xsr_ibreakenable_encode_fns, 0, 0 },
+  { "rsr.debugcause", ICLASS_xt_iclass_rsr_debugcause,
+    0,
+    Opcode_rsr_debugcause_encode_fns, 0, 0 },
+  { "wsr.debugcause", ICLASS_xt_iclass_wsr_debugcause,
+    0,
+    Opcode_wsr_debugcause_encode_fns, 0, 0 },
+  { "xsr.debugcause", ICLASS_xt_iclass_xsr_debugcause,
+    0,
+    Opcode_xsr_debugcause_encode_fns, 0, 0 },
+  { "rsr.icount", ICLASS_xt_iclass_rsr_icount,
+    0,
+    Opcode_rsr_icount_encode_fns, 0, 0 },
+  { "wsr.icount", ICLASS_xt_iclass_wsr_icount,
+    0,
+    Opcode_wsr_icount_encode_fns, 0, 0 },
+  { "xsr.icount", ICLASS_xt_iclass_xsr_icount,
+    0,
+    Opcode_xsr_icount_encode_fns, 0, 0 },
+  { "rsr.icountlevel", ICLASS_xt_iclass_rsr_icountlevel,
+    0,
+    Opcode_rsr_icountlevel_encode_fns, 0, 0 },
+  { "wsr.icountlevel", ICLASS_xt_iclass_wsr_icountlevel,
+    0,
+    Opcode_wsr_icountlevel_encode_fns, 0, 0 },
+  { "xsr.icountlevel", ICLASS_xt_iclass_xsr_icountlevel,
+    0,
+    Opcode_xsr_icountlevel_encode_fns, 0, 0 },
+  { "rsr.ddr", ICLASS_xt_iclass_rsr_ddr,
+    0,
+    Opcode_rsr_ddr_encode_fns, 0, 0 },
+  { "wsr.ddr", ICLASS_xt_iclass_wsr_ddr,
+    0,
+    Opcode_wsr_ddr_encode_fns, 0, 0 },
+  { "xsr.ddr", ICLASS_xt_iclass_xsr_ddr,
+    0,
+    Opcode_xsr_ddr_encode_fns, 0, 0 },
+  { "rfdo", ICLASS_xt_iclass_rfdo,
+    XTENSA_OPCODE_IS_JUMP,
+    Opcode_rfdo_encode_fns, 0, 0 },
+  { "rfdd", ICLASS_xt_iclass_rfdd,
+    XTENSA_OPCODE_IS_JUMP,
+    Opcode_rfdd_encode_fns, 0, 0 },
+  { "wsr.mmid", ICLASS_xt_iclass_wsr_mmid,
+    0,
+    Opcode_wsr_mmid_encode_fns, 0, 0 },
+  { "rsr.ccount", ICLASS_xt_iclass_rsr_ccount,
+    0,
+    Opcode_rsr_ccount_encode_fns, 0, 0 },
+  { "wsr.ccount", ICLASS_xt_iclass_wsr_ccount,
+    0,
+    Opcode_wsr_ccount_encode_fns, 0, 0 },
+  { "xsr.ccount", ICLASS_xt_iclass_xsr_ccount,
+    0,
+    Opcode_xsr_ccount_encode_fns, 0, 0 },
+  { "rsr.ccompare0", ICLASS_xt_iclass_rsr_ccompare0,
+    0,
+    Opcode_rsr_ccompare0_encode_fns, 0, 0 },
+  { "wsr.ccompare0", ICLASS_xt_iclass_wsr_ccompare0,
+    0,
+    Opcode_wsr_ccompare0_encode_fns, 0, 0 },
+  { "xsr.ccompare0", ICLASS_xt_iclass_xsr_ccompare0,
+    0,
+    Opcode_xsr_ccompare0_encode_fns, 0, 0 },
+  { "idtlb", ICLASS_xt_iclass_idtlb,
+    0,
+    Opcode_idtlb_encode_fns, 0, 0 },
+  { "pdtlb", ICLASS_xt_iclass_rdtlb,
+    0,
+    Opcode_pdtlb_encode_fns, 0, 0 },
+  { "rdtlb0", ICLASS_xt_iclass_rdtlb,
+    0,
+    Opcode_rdtlb0_encode_fns, 0, 0 },
+  { "rdtlb1", ICLASS_xt_iclass_rdtlb,
+    0,
+    Opcode_rdtlb1_encode_fns, 0, 0 },
+  { "wdtlb", ICLASS_xt_iclass_wdtlb,
+    0,
+    Opcode_wdtlb_encode_fns, 0, 0 },
+  { "iitlb", ICLASS_xt_iclass_iitlb,
+    0,
+    Opcode_iitlb_encode_fns, 0, 0 },
+  { "pitlb", ICLASS_xt_iclass_ritlb,
+    0,
+    Opcode_pitlb_encode_fns, 0, 0 },
+  { "ritlb0", ICLASS_xt_iclass_ritlb,
+    0,
+    Opcode_ritlb0_encode_fns, 0, 0 },
+  { "ritlb1", ICLASS_xt_iclass_ritlb,
+    0,
+    Opcode_ritlb1_encode_fns, 0, 0 },
+  { "witlb", ICLASS_xt_iclass_witlb,
+    0,
+    Opcode_witlb_encode_fns, 0, 0 },
+  { "nsa", ICLASS_xt_iclass_nsa,
+    0,
+    Opcode_nsa_encode_fns, 0, 0 },
+  { "nsau", ICLASS_xt_iclass_nsa,
+    0,
+    Opcode_nsau_encode_fns, 0, 0 },
+  { "rer", ICLASS_xt_iclass_rer,
+    0,
+    Opcode_rer_encode_fns, 0, 0 },
+  { "wer", ICLASS_xt_iclass_wer,
+    0,
+    Opcode_wer_encode_fns, 0, 0 }
+};
+
+enum xtensa_opcode_id {
+  OPCODE_EXCW,
+  OPCODE_RFE,
+  OPCODE_RFDE,
+  OPCODE_SYSCALL,
+  OPCODE_SIMCALL,
+  OPCODE_ADD_N,
+  OPCODE_ADDI_N,
+  OPCODE_BEQZ_N,
+  OPCODE_BNEZ_N,
+  OPCODE_ILL_N,
+  OPCODE_L32I_N,
+  OPCODE_MOV_N,
+  OPCODE_MOVI_N,
+  OPCODE_NOP_N,
+  OPCODE_RET_N,
+  OPCODE_S32I_N,
+  OPCODE_ADDI,
+  OPCODE_ADDMI,
+  OPCODE_ADD,
+  OPCODE_SUB,
+  OPCODE_ADDX2,
+  OPCODE_ADDX4,
+  OPCODE_ADDX8,
+  OPCODE_SUBX2,
+  OPCODE_SUBX4,
+  OPCODE_SUBX8,
+  OPCODE_AND,
+  OPCODE_OR,
+  OPCODE_XOR,
+  OPCODE_BEQI,
+  OPCODE_BNEI,
+  OPCODE_BGEI,
+  OPCODE_BLTI,
+  OPCODE_BBCI,
+  OPCODE_BBSI,
+  OPCODE_BGEUI,
+  OPCODE_BLTUI,
+  OPCODE_BEQ,
+  OPCODE_BNE,
+  OPCODE_BGE,
+  OPCODE_BLT,
+  OPCODE_BGEU,
+  OPCODE_BLTU,
+  OPCODE_BANY,
+  OPCODE_BNONE,
+  OPCODE_BALL,
+  OPCODE_BNALL,
+  OPCODE_BBC,
+  OPCODE_BBS,
+  OPCODE_BEQZ,
+  OPCODE_BNEZ,
+  OPCODE_BGEZ,
+  OPCODE_BLTZ,
+  OPCODE_CALL0,
+  OPCODE_CALLX0,
+  OPCODE_EXTUI,
+  OPCODE_ILL,
+  OPCODE_J,
+  OPCODE_JX,
+  OPCODE_L16UI,
+  OPCODE_L16SI,
+  OPCODE_L32I,
+  OPCODE_L32R,
+  OPCODE_L8UI,
+  OPCODE_MOVI,
+  OPCODE_MOVEQZ,
+  OPCODE_MOVNEZ,
+  OPCODE_MOVLTZ,
+  OPCODE_MOVGEZ,
+  OPCODE_NEG,
+  OPCODE_ABS,
+  OPCODE_NOP,
+  OPCODE_RET,
+  OPCODE_S16I,
+  OPCODE_S32I,
+  OPCODE_S8I,
+  OPCODE_SSR,
+  OPCODE_SSL,
+  OPCODE_SSA8L,
+  OPCODE_SSA8B,
+  OPCODE_SSAI,
+  OPCODE_SLL,
+  OPCODE_SRC,
+  OPCODE_SRL,
+  OPCODE_SRA,
+  OPCODE_SLLI,
+  OPCODE_SRAI,
+  OPCODE_SRLI,
+  OPCODE_MEMW,
+  OPCODE_EXTW,
+  OPCODE_ISYNC,
+  OPCODE_RSYNC,
+  OPCODE_ESYNC,
+  OPCODE_DSYNC,
+  OPCODE_RSIL,
+  OPCODE_RSR_SAR,
+  OPCODE_WSR_SAR,
+  OPCODE_XSR_SAR,
+  OPCODE_RSR_LITBASE,
+  OPCODE_WSR_LITBASE,
+  OPCODE_XSR_LITBASE,
+  OPCODE_RSR_176,
+  OPCODE_WSR_176,
+  OPCODE_RSR_208,
+  OPCODE_RSR_PS,
+  OPCODE_WSR_PS,
+  OPCODE_XSR_PS,
+  OPCODE_RSR_EPC1,
+  OPCODE_WSR_EPC1,
+  OPCODE_XSR_EPC1,
+  OPCODE_RSR_EXCSAVE1,
+  OPCODE_WSR_EXCSAVE1,
+  OPCODE_XSR_EXCSAVE1,
+  OPCODE_RSR_EPC2,
+  OPCODE_WSR_EPC2,
+  OPCODE_XSR_EPC2,
+  OPCODE_RSR_EXCSAVE2,
+  OPCODE_WSR_EXCSAVE2,
+  OPCODE_XSR_EXCSAVE2,
+  OPCODE_RSR_EPC3,
+  OPCODE_WSR_EPC3,
+  OPCODE_XSR_EPC3,
+  OPCODE_RSR_EXCSAVE3,
+  OPCODE_WSR_EXCSAVE3,
+  OPCODE_XSR_EXCSAVE3,
+  OPCODE_RSR_EPS2,
+  OPCODE_WSR_EPS2,
+  OPCODE_XSR_EPS2,
+  OPCODE_RSR_EPS3,
+  OPCODE_WSR_EPS3,
+  OPCODE_XSR_EPS3,
+  OPCODE_RSR_EXCVADDR,
+  OPCODE_WSR_EXCVADDR,
+  OPCODE_XSR_EXCVADDR,
+  OPCODE_RSR_DEPC,
+  OPCODE_WSR_DEPC,
+  OPCODE_XSR_DEPC,
+  OPCODE_RSR_EXCCAUSE,
+  OPCODE_WSR_EXCCAUSE,
+  OPCODE_XSR_EXCCAUSE,
+  OPCODE_RSR_PRID,
+  OPCODE_RSR_VECBASE,
+  OPCODE_WSR_VECBASE,
+  OPCODE_XSR_VECBASE,
+  OPCODE_MUL16U,
+  OPCODE_MUL16S,
+  OPCODE_MULL,
+  OPCODE_RFI,
+  OPCODE_WAITI,
+  OPCODE_RSR_INTERRUPT,
+  OPCODE_WSR_INTSET,
+  OPCODE_WSR_INTCLEAR,
+  OPCODE_RSR_INTENABLE,
+  OPCODE_WSR_INTENABLE,
+  OPCODE_XSR_INTENABLE,
+  OPCODE_BREAK,
+  OPCODE_BREAK_N,
+  OPCODE_RSR_DBREAKA0,
+  OPCODE_WSR_DBREAKA0,
+  OPCODE_XSR_DBREAKA0,
+  OPCODE_RSR_DBREAKC0,
+  OPCODE_WSR_DBREAKC0,
+  OPCODE_XSR_DBREAKC0,
+  OPCODE_RSR_IBREAKA0,
+  OPCODE_WSR_IBREAKA0,
+  OPCODE_XSR_IBREAKA0,
+  OPCODE_RSR_IBREAKENABLE,
+  OPCODE_WSR_IBREAKENABLE,
+  OPCODE_XSR_IBREAKENABLE,
+  OPCODE_RSR_DEBUGCAUSE,
+  OPCODE_WSR_DEBUGCAUSE,
+  OPCODE_XSR_DEBUGCAUSE,
+  OPCODE_RSR_ICOUNT,
+  OPCODE_WSR_ICOUNT,
+  OPCODE_XSR_ICOUNT,
+  OPCODE_RSR_ICOUNTLEVEL,
+  OPCODE_WSR_ICOUNTLEVEL,
+  OPCODE_XSR_ICOUNTLEVEL,
+  OPCODE_RSR_DDR,
+  OPCODE_WSR_DDR,
+  OPCODE_XSR_DDR,
+  OPCODE_RFDO,
+  OPCODE_RFDD,
+  OPCODE_WSR_MMID,
+  OPCODE_RSR_CCOUNT,
+  OPCODE_WSR_CCOUNT,
+  OPCODE_XSR_CCOUNT,
+  OPCODE_RSR_CCOMPARE0,
+  OPCODE_WSR_CCOMPARE0,
+  OPCODE_XSR_CCOMPARE0,
+  OPCODE_IDTLB,
+  OPCODE_PDTLB,
+  OPCODE_RDTLB0,
+  OPCODE_RDTLB1,
+  OPCODE_WDTLB,
+  OPCODE_IITLB,
+  OPCODE_PITLB,
+  OPCODE_RITLB0,
+  OPCODE_RITLB1,
+  OPCODE_WITLB,
+  OPCODE_NSA,
+  OPCODE_NSAU,
+  OPCODE_RER,
+  OPCODE_WER
+};
+
+
+/* Slot-specific opcode decode functions.  */
+
+static int
+Slot_inst_decode (const xtensa_insnbuf insn)
+{
+  switch (Field_op0_Slot_inst_get (insn))
+    {
+    case 0:
+      switch (Field_op1_Slot_inst_get (insn))
+	{
+	case 0:
+	  switch (Field_op2_Slot_inst_get (insn))
+	    {
+	    case 0:
+	      switch (Field_r_Slot_inst_get (insn))
+		{
+		case 0:
+		  switch (Field_m_Slot_inst_get (insn))
+		    {
+		    case 0:
+		      if (Field_s_Slot_inst_get (insn) == 0 &&
+			  Field_n_Slot_inst_get (insn) == 0)
+			return OPCODE_ILL;
+		      break;
+		    case 2:
+		      switch (Field_n_Slot_inst_get (insn))
+			{
+			case 0:
+			  return OPCODE_RET;
+			case 2:
+			  return OPCODE_JX;
+			}
+		      break;
+		    case 3:
+		      if (Field_n_Slot_inst_get (insn) == 0)
+			return OPCODE_CALLX0;
+		      break;
+		    }
+		  break;
+		case 2:
+		  if (Field_s_Slot_inst_get (insn) == 0)
+		    {
+		      switch (Field_t_Slot_inst_get (insn))
+			{
+			case 0:
+			  return OPCODE_ISYNC;
+			case 1:
+			  return OPCODE_RSYNC;
+			case 2:
+			  return OPCODE_ESYNC;
+			case 3:
+			  return OPCODE_DSYNC;
+			case 8:
+			  return OPCODE_EXCW;
+			case 12:
+			  return OPCODE_MEMW;
+			case 13:
+			  return OPCODE_EXTW;
+			case 15:
+			  return OPCODE_NOP;
+			}
+		    }
+		  break;
+		case 3:
+		  switch (Field_t_Slot_inst_get (insn))
+		    {
+		    case 0:
+		      switch (Field_s_Slot_inst_get (insn))
+			{
+			case 0:
+			  return OPCODE_RFE;
+			case 2:
+			  return OPCODE_RFDE;
+			}
+		      break;
+		    case 1:
+		      return OPCODE_RFI;
+		    }
+		  break;
+		case 4:
+		  return OPCODE_BREAK;
+		case 5:
+		  switch (Field_s_Slot_inst_get (insn))
+		    {
+		    case 0:
+		      if (Field_t_Slot_inst_get (insn) == 0)
+			return OPCODE_SYSCALL;
+		      break;
+		    case 1:
+		      if (Field_t_Slot_inst_get (insn) == 0)
+			return OPCODE_SIMCALL;
+		      break;
+		    }
+		  break;
+		case 6:
+		  return OPCODE_RSIL;
+		case 7:
+		  if (Field_t_Slot_inst_get (insn) == 0)
+		    return OPCODE_WAITI;
+		  break;
+		}
+	      break;
+	    case 1:
+	      return OPCODE_AND;
+	    case 2:
+	      return OPCODE_OR;
+	    case 3:
+	      return OPCODE_XOR;
+	    case 4:
+	      switch (Field_r_Slot_inst_get (insn))
+		{
+		case 0:
+		  if (Field_t_Slot_inst_get (insn) == 0)
+		    return OPCODE_SSR;
+		  break;
+		case 1:
+		  if (Field_t_Slot_inst_get (insn) == 0)
+		    return OPCODE_SSL;
+		  break;
+		case 2:
+		  if (Field_t_Slot_inst_get (insn) == 0)
+		    return OPCODE_SSA8L;
+		  break;
+		case 3:
+		  if (Field_t_Slot_inst_get (insn) == 0)
+		    return OPCODE_SSA8B;
+		  break;
+		case 4:
+		  if (Field_thi3_Slot_inst_get (insn) == 0)
+		    return OPCODE_SSAI;
+		  break;
+		case 6:
+		  return OPCODE_RER;
+		case 7:
+		  return OPCODE_WER;
+		case 14:
+		  return OPCODE_NSA;
+		case 15:
+		  return OPCODE_NSAU;
+		}
+	      break;
+	    case 5:
+	      switch (Field_r_Slot_inst_get (insn))
+		{
+		case 3:
+		  return OPCODE_RITLB0;
+		case 4:
+		  if (Field_t_Slot_inst_get (insn) == 0)
+		    return OPCODE_IITLB;
+		  break;
+		case 5:
+		  return OPCODE_PITLB;
+		case 6:
+		  return OPCODE_WITLB;
+		case 7:
+		  return OPCODE_RITLB1;
+		case 11:
+		  return OPCODE_RDTLB0;
+		case 12:
+		  if (Field_t_Slot_inst_get (insn) == 0)
+		    return OPCODE_IDTLB;
+		  break;
+		case 13:
+		  return OPCODE_PDTLB;
+		case 14:
+		  return OPCODE_WDTLB;
+		case 15:
+		  return OPCODE_RDTLB1;
+		}
+	      break;
+	    case 6:
+	      switch (Field_s_Slot_inst_get (insn))
+		{
+		case 0:
+		  return OPCODE_NEG;
+		case 1:
+		  return OPCODE_ABS;
+		}
+	      break;
+	    case 8:
+	      return OPCODE_ADD;
+	    case 9:
+	      return OPCODE_ADDX2;
+	    case 10:
+	      return OPCODE_ADDX4;
+	    case 11:
+	      return OPCODE_ADDX8;
+	    case 12:
+	      return OPCODE_SUB;
+	    case 13:
+	      return OPCODE_SUBX2;
+	    case 14:
+	      return OPCODE_SUBX4;
+	    case 15:
+	      return OPCODE_SUBX8;
+	    }
+	  break;
+	case 1:
+	  switch (Field_op2_Slot_inst_get (insn))
+	    {
+	    case 0:
+	    case 1:
+	      return OPCODE_SLLI;
+	    case 2:
+	    case 3:
+	      return OPCODE_SRAI;
+	    case 4:
+	      return OPCODE_SRLI;
+	    case 6:
+	      switch (Field_sr_Slot_inst_get (insn))
+		{
+		case 3:
+		  return OPCODE_XSR_SAR;
+		case 5:
+		  return OPCODE_XSR_LITBASE;
+		case 96:
+		  return OPCODE_XSR_IBREAKENABLE;
+		case 104:
+		  return OPCODE_XSR_DDR;
+		case 128:
+		  return OPCODE_XSR_IBREAKA0;
+		case 144:
+		  return OPCODE_XSR_DBREAKA0;
+		case 160:
+		  return OPCODE_XSR_DBREAKC0;
+		case 177:
+		  return OPCODE_XSR_EPC1;
+		case 178:
+		  return OPCODE_XSR_EPC2;
+		case 179:
+		  return OPCODE_XSR_EPC3;
+		case 192:
+		  return OPCODE_XSR_DEPC;
+		case 194:
+		  return OPCODE_XSR_EPS2;
+		case 195:
+		  return OPCODE_XSR_EPS3;
+		case 209:
+		  return OPCODE_XSR_EXCSAVE1;
+		case 210:
+		  return OPCODE_XSR_EXCSAVE2;
+		case 211:
+		  return OPCODE_XSR_EXCSAVE3;
+		case 228:
+		  return OPCODE_XSR_INTENABLE;
+		case 230:
+		  return OPCODE_XSR_PS;
+		case 231:
+		  return OPCODE_XSR_VECBASE;
+		case 232:
+		  return OPCODE_XSR_EXCCAUSE;
+		case 233:
+		  return OPCODE_XSR_DEBUGCAUSE;
+		case 234:
+		  return OPCODE_XSR_CCOUNT;
+		case 236:
+		  return OPCODE_XSR_ICOUNT;
+		case 237:
+		  return OPCODE_XSR_ICOUNTLEVEL;
+		case 238:
+		  return OPCODE_XSR_EXCVADDR;
+		case 240:
+		  return OPCODE_XSR_CCOMPARE0;
+		}
+	      break;
+	    case 8:
+	      return OPCODE_SRC;
+	    case 9:
+	      if (Field_s_Slot_inst_get (insn) == 0)
+		return OPCODE_SRL;
+	      break;
+	    case 10:
+	      if (Field_t_Slot_inst_get (insn) == 0)
+		return OPCODE_SLL;
+	      break;
+	    case 11:
+	      if (Field_s_Slot_inst_get (insn) == 0)
+		return OPCODE_SRA;
+	      break;
+	    case 12:
+	      return OPCODE_MUL16U;
+	    case 13:
+	      return OPCODE_MUL16S;
+	    case 15:
+	      switch (Field_r_Slot_inst_get (insn))
+		{
+		case 14:
+		  if (Field_t_Slot_inst_get (insn) == 0)
+		    return OPCODE_RFDO;
+		  if (Field_t_Slot_inst_get (insn) == 1)
+		    return OPCODE_RFDD;
+		  break;
+		}
+	      break;
+	    }
+	  break;
+	case 2:
+	  if (Field_op2_Slot_inst_get (insn) == 8)
+	    return OPCODE_MULL;
+	  break;
+	case 3:
+	  switch (Field_op2_Slot_inst_get (insn))
+	    {
+	    case 0:
+	      switch (Field_sr_Slot_inst_get (insn))
+		{
+		case 3:
+		  return OPCODE_RSR_SAR;
+		case 5:
+		  return OPCODE_RSR_LITBASE;
+		case 96:
+		  return OPCODE_RSR_IBREAKENABLE;
+		case 104:
+		  return OPCODE_RSR_DDR;
+		case 128:
+		  return OPCODE_RSR_IBREAKA0;
+		case 144:
+		  return OPCODE_RSR_DBREAKA0;
+		case 160:
+		  return OPCODE_RSR_DBREAKC0;
+		case 176:
+		  return OPCODE_RSR_176;
+		case 177:
+		  return OPCODE_RSR_EPC1;
+		case 178:
+		  return OPCODE_RSR_EPC2;
+		case 179:
+		  return OPCODE_RSR_EPC3;
+		case 192:
+		  return OPCODE_RSR_DEPC;
+		case 194:
+		  return OPCODE_RSR_EPS2;
+		case 195:
+		  return OPCODE_RSR_EPS3;
+		case 208:
+		  return OPCODE_RSR_208;
+		case 209:
+		  return OPCODE_RSR_EXCSAVE1;
+		case 210:
+		  return OPCODE_RSR_EXCSAVE2;
+		case 211:
+		  return OPCODE_RSR_EXCSAVE3;
+		case 226:
+		  return OPCODE_RSR_INTERRUPT;
+		case 228:
+		  return OPCODE_RSR_INTENABLE;
+		case 230:
+		  return OPCODE_RSR_PS;
+		case 231:
+		  return OPCODE_RSR_VECBASE;
+		case 232:
+		  return OPCODE_RSR_EXCCAUSE;
+		case 233:
+		  return OPCODE_RSR_DEBUGCAUSE;
+		case 234:
+		  return OPCODE_RSR_CCOUNT;
+		case 235:
+		  return OPCODE_RSR_PRID;
+		case 236:
+		  return OPCODE_RSR_ICOUNT;
+		case 237:
+		  return OPCODE_RSR_ICOUNTLEVEL;
+		case 238:
+		  return OPCODE_RSR_EXCVADDR;
+		case 240:
+		  return OPCODE_RSR_CCOMPARE0;
+		}
+	      break;
+	    case 1:
+	      switch (Field_sr_Slot_inst_get (insn))
+		{
+		case 3:
+		  return OPCODE_WSR_SAR;
+		case 5:
+		  return OPCODE_WSR_LITBASE;
+		case 89:
+		  return OPCODE_WSR_MMID;
+		case 96:
+		  return OPCODE_WSR_IBREAKENABLE;
+		case 104:
+		  return OPCODE_WSR_DDR;
+		case 128:
+		  return OPCODE_WSR_IBREAKA0;
+		case 144:
+		  return OPCODE_WSR_DBREAKA0;
+		case 160:
+		  return OPCODE_WSR_DBREAKC0;
+		case 176:
+		  return OPCODE_WSR_176;
+		case 177:
+		  return OPCODE_WSR_EPC1;
+		case 178:
+		  return OPCODE_WSR_EPC2;
+		case 179:
+		  return OPCODE_WSR_EPC3;
+		case 192:
+		  return OPCODE_WSR_DEPC;
+		case 194:
+		  return OPCODE_WSR_EPS2;
+		case 195:
+		  return OPCODE_WSR_EPS3;
+		case 209:
+		  return OPCODE_WSR_EXCSAVE1;
+		case 210:
+		  return OPCODE_WSR_EXCSAVE2;
+		case 211:
+		  return OPCODE_WSR_EXCSAVE3;
+		case 226:
+		  return OPCODE_WSR_INTSET;
+		case 227:
+		  return OPCODE_WSR_INTCLEAR;
+		case 228:
+		  return OPCODE_WSR_INTENABLE;
+		case 230:
+		  return OPCODE_WSR_PS;
+		case 231:
+		  return OPCODE_WSR_VECBASE;
+		case 232:
+		  return OPCODE_WSR_EXCCAUSE;
+		case 233:
+		  return OPCODE_WSR_DEBUGCAUSE;
+		case 234:
+		  return OPCODE_WSR_CCOUNT;
+		case 236:
+		  return OPCODE_WSR_ICOUNT;
+		case 237:
+		  return OPCODE_WSR_ICOUNTLEVEL;
+		case 238:
+		  return OPCODE_WSR_EXCVADDR;
+		case 240:
+		  return OPCODE_WSR_CCOMPARE0;
+		}
+	      break;
+	    case 8:
+	      return OPCODE_MOVEQZ;
+	    case 9:
+	      return OPCODE_MOVNEZ;
+	    case 10:
+	      return OPCODE_MOVLTZ;
+	    case 11:
+	      return OPCODE_MOVGEZ;
+	    }
+	  break;
+	case 4:
+	case 5:
+	  return OPCODE_EXTUI;
+	}
+      break;
+    case 1:
+      return OPCODE_L32R;
+    case 2:
+      switch (Field_r_Slot_inst_get (insn))
+	{
+	case 0:
+	  return OPCODE_L8UI;
+	case 1:
+	  return OPCODE_L16UI;
+	case 2:
+	  return OPCODE_L32I;
+	case 4:
+	  return OPCODE_S8I;
+	case 5:
+	  return OPCODE_S16I;
+	case 6:
+	  return OPCODE_S32I;
+	case 9:
+	  return OPCODE_L16SI;
+	case 10:
+	  return OPCODE_MOVI;
+	case 12:
+	  return OPCODE_ADDI;
+	case 13:
+	  return OPCODE_ADDMI;
+	}
+      break;
+    case 5:
+      if (Field_n_Slot_inst_get (insn) == 0)
+	return OPCODE_CALL0;
+      break;
+    case 6:
+      switch (Field_n_Slot_inst_get (insn))
+	{
+	case 0:
+	  return OPCODE_J;
+	case 1:
+	  switch (Field_m_Slot_inst_get (insn))
+	    {
+	    case 0:
+	      return OPCODE_BEQZ;
+	    case 1:
+	      return OPCODE_BNEZ;
+	    case 2:
+	      return OPCODE_BLTZ;
+	    case 3:
+	      return OPCODE_BGEZ;
+	    }
+	  break;
+	case 2:
+	  switch (Field_m_Slot_inst_get (insn))
+	    {
+	    case 0:
+	      return OPCODE_BEQI;
+	    case 1:
+	      return OPCODE_BNEI;
+	    case 2:
+	      return OPCODE_BLTI;
+	    case 3:
+	      return OPCODE_BGEI;
+	    }
+	  break;
+	case 3:
+	  switch (Field_m_Slot_inst_get (insn))
+	    {
+	    case 2:
+	      return OPCODE_BLTUI;
+	    case 3:
+	      return OPCODE_BGEUI;
+	    }
+	  break;
+	}
+      break;
+    case 7:
+      switch (Field_r_Slot_inst_get (insn))
+	{
+	case 0:
+	  return OPCODE_BNONE;
+	case 1:
+	  return OPCODE_BEQ;
+	case 2:
+	  return OPCODE_BLT;
+	case 3:
+	  return OPCODE_BLTU;
+	case 4:
+	  return OPCODE_BALL;
+	case 5:
+	  return OPCODE_BBC;
+	case 6:
+	case 7:
+	  return OPCODE_BBCI;
+	case 8:
+	  return OPCODE_BANY;
+	case 9:
+	  return OPCODE_BNE;
+	case 10:
+	  return OPCODE_BGE;
+	case 11:
+	  return OPCODE_BGEU;
+	case 12:
+	  return OPCODE_BNALL;
+	case 13:
+	  return OPCODE_BBS;
+	case 14:
+	case 15:
+	  return OPCODE_BBSI;
+	}
+      break;
+    }
+  return XTENSA_UNDEFINED;
+}
+
+static int
+Slot_inst16a_decode (const xtensa_insnbuf insn)
+{
+  switch (Field_op0_Slot_inst16a_get (insn))
+    {
+    case 8:
+      return OPCODE_L32I_N;
+    case 9:
+      return OPCODE_S32I_N;
+    case 10:
+      return OPCODE_ADD_N;
+    case 11:
+      return OPCODE_ADDI_N;
+    }
+  return XTENSA_UNDEFINED;
+}
+
+static int
+Slot_inst16b_decode (const xtensa_insnbuf insn)
+{
+  switch (Field_op0_Slot_inst16b_get (insn))
+    {
+    case 12:
+      switch (Field_i_Slot_inst16b_get (insn))
+	{
+	case 0:
+	  return OPCODE_MOVI_N;
+	case 1:
+	  switch (Field_z_Slot_inst16b_get (insn))
+	    {
+	    case 0:
+	      return OPCODE_BEQZ_N;
+	    case 1:
+	      return OPCODE_BNEZ_N;
+	    }
+	  break;
+	}
+      break;
+    case 13:
+      switch (Field_r_Slot_inst16b_get (insn))
+	{
+	case 0:
+	  return OPCODE_MOV_N;
+	case 15:
+	  switch (Field_t_Slot_inst16b_get (insn))
+	    {
+	    case 0:
+	      return OPCODE_RET_N;
+	    case 2:
+	      return OPCODE_BREAK_N;
+	    case 3:
+	      if (Field_s_Slot_inst16b_get (insn) == 0)
+		return OPCODE_NOP_N;
+	      break;
+	    case 6:
+	      if (Field_s_Slot_inst16b_get (insn) == 0)
+		return OPCODE_ILL_N;
+	      break;
+	    }
+	  break;
+	}
+      break;
+    }
+  return XTENSA_UNDEFINED;
+}
+
+
+/* Instruction slots.  */
+
+static void
+Slot_x24_Format_inst_0_get (const xtensa_insnbuf insn,
+			    xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = (insn[0] & 0xffffff);
+}
+
+static void
+Slot_x24_Format_inst_0_set (xtensa_insnbuf insn,
+			    const xtensa_insnbuf slotbuf)
+{
+  insn[0] = (insn[0] & ~0xffffff) | (slotbuf[0] & 0xffffff);
+}
+
+static void
+Slot_x16a_Format_inst16a_0_get (const xtensa_insnbuf insn,
+				xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = (insn[0] & 0xffff);
+}
+
+static void
+Slot_x16a_Format_inst16a_0_set (xtensa_insnbuf insn,
+				const xtensa_insnbuf slotbuf)
+{
+  insn[0] = (insn[0] & ~0xffff) | (slotbuf[0] & 0xffff);
+}
+
+static void
+Slot_x16b_Format_inst16b_0_get (const xtensa_insnbuf insn,
+				xtensa_insnbuf slotbuf)
+{
+  slotbuf[0] = (insn[0] & 0xffff);
+}
+
+static void
+Slot_x16b_Format_inst16b_0_set (xtensa_insnbuf insn,
+				const xtensa_insnbuf slotbuf)
+{
+  insn[0] = (insn[0] & ~0xffff) | (slotbuf[0] & 0xffff);
+}
+
+static xtensa_get_field_fn
+Slot_inst_get_field_fns[] = {
+  Field_t_Slot_inst_get,
+  Field_bbi4_Slot_inst_get,
+  Field_bbi_Slot_inst_get,
+  Field_imm12_Slot_inst_get,
+  Field_imm8_Slot_inst_get,
+  Field_s_Slot_inst_get,
+  Field_imm12b_Slot_inst_get,
+  Field_imm16_Slot_inst_get,
+  Field_m_Slot_inst_get,
+  Field_n_Slot_inst_get,
+  Field_offset_Slot_inst_get,
+  Field_op0_Slot_inst_get,
+  Field_op1_Slot_inst_get,
+  Field_op2_Slot_inst_get,
+  Field_r_Slot_inst_get,
+  Field_sa4_Slot_inst_get,
+  Field_sae4_Slot_inst_get,
+  Field_sae_Slot_inst_get,
+  Field_sal_Slot_inst_get,
+  Field_sargt_Slot_inst_get,
+  Field_sas4_Slot_inst_get,
+  Field_sas_Slot_inst_get,
+  Field_sr_Slot_inst_get,
+  Field_st_Slot_inst_get,
+  Field_thi3_Slot_inst_get,
+  Field_imm4_Slot_inst_get,
+  0,
+  0,
+  0,
+  0,
+  0,
+  0,
+  0,
+  0,
+  Field_xt_wbr15_imm_Slot_inst_get,
+  Field_xt_wbr18_imm_Slot_inst_get,
+  Implicit_Field_ar0_get
+};
+
+static xtensa_set_field_fn
+Slot_inst_set_field_fns[] = {
+  Field_t_Slot_inst_set,
+  Field_bbi4_Slot_inst_set,
+  Field_bbi_Slot_inst_set,
+  Field_imm12_Slot_inst_set,
+  Field_imm8_Slot_inst_set,
+  Field_s_Slot_inst_set,
+  Field_imm12b_Slot_inst_set,
+  Field_imm16_Slot_inst_set,
+  Field_m_Slot_inst_set,
+  Field_n_Slot_inst_set,
+  Field_offset_Slot_inst_set,
+  Field_op0_Slot_inst_set,
+  Field_op1_Slot_inst_set,
+  Field_op2_Slot_inst_set,
+  Field_r_Slot_inst_set,
+  Field_sa4_Slot_inst_set,
+  Field_sae4_Slot_inst_set,
+  Field_sae_Slot_inst_set,
+  Field_sal_Slot_inst_set,
+  Field_sargt_Slot_inst_set,
+  Field_sas4_Slot_inst_set,
+  Field_sas_Slot_inst_set,
+  Field_sr_Slot_inst_set,
+  Field_st_Slot_inst_set,
+  Field_thi3_Slot_inst_set,
+  Field_imm4_Slot_inst_set,
+  0,
+  0,
+  0,
+  0,
+  0,
+  0,
+  0,
+  0,
+  Field_xt_wbr15_imm_Slot_inst_set,
+  Field_xt_wbr18_imm_Slot_inst_set,
+  Implicit_Field_set
+};
+
+static xtensa_get_field_fn
+Slot_inst16a_get_field_fns[] = {
+  Field_t_Slot_inst16a_get,
+  0,
+  0,
+  0,
+  0,
+  Field_s_Slot_inst16a_get,
+  0,
+  0,
+  0,
+  0,
+  0,
+  Field_op0_Slot_inst16a_get,
+  0,
+  0,
+  Field_r_Slot_inst16a_get,
+  0,
+  0,
+  0,
+  0,
+  0,
+  0,
+  0,
+  Field_sr_Slot_inst16a_get,
+  Field_st_Slot_inst16a_get,
+  0,
+  Field_imm4_Slot_inst16a_get,
+  Field_i_Slot_inst16a_get,
+  Field_imm6lo_Slot_inst16a_get,
+  Field_imm6hi_Slot_inst16a_get,
+  Field_imm7lo_Slot_inst16a_get,
+  Field_imm7hi_Slot_inst16a_get,
+  Field_z_Slot_inst16a_get,
+  Field_imm6_Slot_inst16a_get,
+  Field_imm7_Slot_inst16a_get,
+  0,
+  0,
+  Implicit_Field_ar0_get
+};
+
+static xtensa_set_field_fn
+Slot_inst16a_set_field_fns[] = {
+  Field_t_Slot_inst16a_set,
+  0,
+  0,
+  0,
+  0,
+  Field_s_Slot_inst16a_set,
+  0,
+  0,
+  0,
+  0,
+  0,
+  Field_op0_Slot_inst16a_set,
+  0,
+  0,
+  Field_r_Slot_inst16a_set,
+  0,
+  0,
+  0,
+  0,
+  0,
+  0,
+  0,
+  Field_sr_Slot_inst16a_set,
+  Field_st_Slot_inst16a_set,
+  0,
+  Field_imm4_Slot_inst16a_set,
+  Field_i_Slot_inst16a_set,
+  Field_imm6lo_Slot_inst16a_set,
+  Field_imm6hi_Slot_inst16a_set,
+  Field_imm7lo_Slot_inst16a_set,
+  Field_imm7hi_Slot_inst16a_set,
+  Field_z_Slot_inst16a_set,
+  Field_imm6_Slot_inst16a_set,
+  Field_imm7_Slot_inst16a_set,
+  0,
+  0,
+  Implicit_Field_set
+};
+
+static xtensa_get_field_fn
+Slot_inst16b_get_field_fns[] = {
+  Field_t_Slot_inst16b_get,
+  0,
+  0,
+  0,
+  0,
+  Field_s_Slot_inst16b_get,
+  0,
+  0,
+  0,
+  0,
+  0,
+  Field_op0_Slot_inst16b_get,
+  0,
+  0,
+  Field_r_Slot_inst16b_get,
+  0,
+  0,
+  0,
+  0,
+  0,
+  0,
+  0,
+  Field_sr_Slot_inst16b_get,
+  Field_st_Slot_inst16b_get,
+  0,
+  Field_imm4_Slot_inst16b_get,
+  Field_i_Slot_inst16b_get,
+  Field_imm6lo_Slot_inst16b_get,
+  Field_imm6hi_Slot_inst16b_get,
+  Field_imm7lo_Slot_inst16b_get,
+  Field_imm7hi_Slot_inst16b_get,
+  Field_z_Slot_inst16b_get,
+  Field_imm6_Slot_inst16b_get,
+  Field_imm7_Slot_inst16b_get,
+  0,
+  0,
+  Implicit_Field_ar0_get
+};
+
+static xtensa_set_field_fn
+Slot_inst16b_set_field_fns[] = {
+  Field_t_Slot_inst16b_set,
+  0,
+  0,
+  0,
+  0,
+  Field_s_Slot_inst16b_set,
+  0,
+  0,
+  0,
+  0,
+  0,
+  Field_op0_Slot_inst16b_set,
+  0,
+  0,
+  Field_r_Slot_inst16b_set,
+  0,
+  0,
+  0,
+  0,
+  0,
+  0,
+  0,
+  Field_sr_Slot_inst16b_set,
+  Field_st_Slot_inst16b_set,
+  0,
+  Field_imm4_Slot_inst16b_set,
+  Field_i_Slot_inst16b_set,
+  Field_imm6lo_Slot_inst16b_set,
+  Field_imm6hi_Slot_inst16b_set,
+  Field_imm7lo_Slot_inst16b_set,
+  Field_imm7hi_Slot_inst16b_set,
+  Field_z_Slot_inst16b_set,
+  Field_imm6_Slot_inst16b_set,
+  Field_imm7_Slot_inst16b_set,
+  0,
+  0,
+  Implicit_Field_set
+};
+
+static xtensa_slot_internal slots[] = {
+  { "Inst", "x24", 0,
+    Slot_x24_Format_inst_0_get, Slot_x24_Format_inst_0_set,
+    Slot_inst_get_field_fns, Slot_inst_set_field_fns,
+    Slot_inst_decode, "nop" },
+  { "Inst16a", "x16a", 0,
+    Slot_x16a_Format_inst16a_0_get, Slot_x16a_Format_inst16a_0_set,
+    Slot_inst16a_get_field_fns, Slot_inst16a_set_field_fns,
+    Slot_inst16a_decode, "" },
+  { "Inst16b", "x16b", 0,
+    Slot_x16b_Format_inst16b_0_get, Slot_x16b_Format_inst16b_0_set,
+    Slot_inst16b_get_field_fns, Slot_inst16b_set_field_fns,
+    Slot_inst16b_decode, "nop.n" }
+};
+
+
+/* Instruction formats.  */
+
+static void
+Format_x24_encode (xtensa_insnbuf insn)
+{
+  insn[0] = 0;
+}
+
+static void
+Format_x16a_encode (xtensa_insnbuf insn)
+{
+  insn[0] = 0x8;
+}
+
+static void
+Format_x16b_encode (xtensa_insnbuf insn)
+{
+  insn[0] = 0xc;
+}
+
+static int Format_x24_slots[] = { 0 };
+
+static int Format_x16a_slots[] = { 1 };
+
+static int Format_x16b_slots[] = { 2 };
+
+static xtensa_format_internal formats[] = {
+  { "x24", 3, Format_x24_encode, 1, Format_x24_slots },
+  { "x16a", 2, Format_x16a_encode, 1, Format_x16a_slots },
+  { "x16b", 2, Format_x16b_encode, 1, Format_x16b_slots }
+};
+
+
+static int
+format_decoder (const xtensa_insnbuf insn)
+{
+  if ((insn[0] & 0x8) == 0)
+    return 0; /* x24 */
+  if ((insn[0] & 0xc) == 0x8)
+    return 1; /* x16a */
+  if ((insn[0] & 0xe) == 0xc)
+    return 2; /* x16b */
+  return -1;
+}
+
+static int length_table[16] = {
+  3,
+  3,
+  3,
+  3,
+  3,
+  3,
+  3,
+  3,
+  2,
+  2,
+  2,
+  2,
+  2,
+  2,
+  -1,
+  -1
+};
+
+static int
+length_decoder (const unsigned char *insn)
+{
+  int op0 = insn[0] & 0xf;
+  return length_table[op0];
+}
+
+
+/* Top-level ISA structure.  */
+
+static xtensa_isa_internal xtensa_modules = {
+  0 /* little-endian */,
+  3 /* insn_size */, 0,
+  3, formats, format_decoder, length_decoder,
+  3, slots,
+  37 /* num_fields */,
+  65, operands,
+  159, iclasses,
+  204, opcodes, 0,
+  1, regfiles,
+  NUM_STATES, states, 0,
+  NUM_SYSREGS, sysregs, 0,
+  { MAX_SPECIAL_REG, MAX_USER_REG }, { 0, 0 },
+  0, interfaces, 0,
+  0, funcUnits, 0
+};
diff --git a/target/xtensa/cores.list b/target/xtensa/cores.list
index 5772a00ab2..a526a71cfd 100644
--- a/target/xtensa/cores.list
+++ b/target/xtensa/cores.list
@@ -4,6 +4,7 @@ core-de212.c
 core-de233_fpu.c
 core-dsp3400.c
 core-fsf.c
+core-lx106.c
 core-sample_controller.c
 core-test_kc705_be.c
 core-test_mmuhifi_c3.c

From 9e377be1f042e8618c54ee786d1022caa0e2409d Mon Sep 17 00:00:00 2001
From: Max Filippov <jcmvbkbc@gmail.com>
Date: Sun, 3 Oct 2021 14:31:47 -0700
Subject: [PATCH 087/935] target/xtensa: add clock input to xtensa CPU

Create clock input for the xtensa CPU device and initialize its
frequency to the default core frequency specified in the config.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 target/xtensa/cpu.c       | 15 +++++++++++++++
 target/xtensa/cpu.h       |  5 +++++
 target/xtensa/op_helper.c |  7 ++++---
 3 files changed, 24 insertions(+), 3 deletions(-)

diff --git a/target/xtensa/cpu.c b/target/xtensa/cpu.c
index 224f723236..fd553fdfb5 100644
--- a/target/xtensa/cpu.c
+++ b/target/xtensa/cpu.c
@@ -34,6 +34,7 @@
 #include "fpu/softfloat.h"
 #include "qemu/module.h"
 #include "migration/vmstate.h"
+#include "hw/qdev-clock.h"
 
 
 static void xtensa_cpu_set_pc(CPUState *cs, vaddr value)
@@ -172,9 +173,23 @@ static void xtensa_cpu_initfn(Object *obj)
     memory_region_init_io(env->system_er, obj, NULL, env, "er",
                           UINT64_C(0x100000000));
     address_space_init(env->address_space_er, env->system_er, "ER");
+
+    cpu->clock = qdev_init_clock_in(DEVICE(obj), "clk-in", NULL, cpu, 0);
+    clock_set_hz(cpu->clock, env->config->clock_freq_khz * 1000);
 #endif
 }
 
+XtensaCPU *xtensa_cpu_create_with_clock(const char *cpu_type, Clock *cpu_refclk)
+{
+    DeviceState *cpu;
+
+    cpu = DEVICE(object_new(cpu_type));
+    qdev_connect_clock_in(cpu, "clk-in", cpu_refclk);
+    qdev_realize(cpu, NULL, &error_abort);
+
+    return XTENSA_CPU(cpu);
+}
+
 #ifndef CONFIG_USER_ONLY
 static const VMStateDescription vmstate_xtensa_cpu = {
     .name = "cpu",
diff --git a/target/xtensa/cpu.h b/target/xtensa/cpu.h
index d4b8268146..579adcb769 100644
--- a/target/xtensa/cpu.h
+++ b/target/xtensa/cpu.h
@@ -31,6 +31,7 @@
 #include "cpu-qom.h"
 #include "qemu/cpu-float.h"
 #include "exec/cpu-defs.h"
+#include "hw/clock.h"
 #include "xtensa-isa.h"
 
 /* Xtensa processors have a weak memory model */
@@ -559,6 +560,7 @@ struct ArchCPU {
     CPUState parent_obj;
     /*< public >*/
 
+    Clock *clock;
     CPUNegativeOffsetState neg;
     CPUXtensaState env;
 };
@@ -793,4 +795,7 @@ static inline void cpu_get_tb_cpu_state(CPUXtensaState *env, target_ulong *pc,
     }
 }
 
+XtensaCPU *xtensa_cpu_create_with_clock(const char *cpu_type,
+                                        Clock *cpu_refclk);
+
 #endif
diff --git a/target/xtensa/op_helper.c b/target/xtensa/op_helper.c
index d85d3516d6..1af7becc54 100644
--- a/target/xtensa/op_helper.c
+++ b/target/xtensa/op_helper.c
@@ -38,12 +38,12 @@
 
 void HELPER(update_ccount)(CPUXtensaState *env)
 {
+    XtensaCPU *cpu = XTENSA_CPU(env_cpu(env));
     uint64_t now = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
 
     env->ccount_time = now;
     env->sregs[CCOUNT] = env->ccount_base +
-        (uint32_t)((now - env->time_base) *
-                   env->config->clock_freq_khz / 1000000);
+        (uint32_t)clock_ns_to_ticks(cpu->clock, now - env->time_base);
 }
 
 void HELPER(wsr_ccount)(CPUXtensaState *env, uint32_t v)
@@ -59,6 +59,7 @@ void HELPER(wsr_ccount)(CPUXtensaState *env, uint32_t v)
 
 void HELPER(update_ccompare)(CPUXtensaState *env, uint32_t i)
 {
+    XtensaCPU *cpu = XTENSA_CPU(env_cpu(env));
     uint64_t dcc;
 
     qatomic_and(&env->sregs[INTSET],
@@ -66,7 +67,7 @@ void HELPER(update_ccompare)(CPUXtensaState *env, uint32_t i)
     HELPER(update_ccount)(env);
     dcc = (uint64_t)(env->sregs[CCOMPARE + i] - env->sregs[CCOUNT] - 1) + 1;
     timer_mod(env->ccompare[i].timer,
-              env->ccount_time + (dcc * 1000000) / env->config->clock_freq_khz);
+              env->ccount_time + clock_ticks_to_ns(cpu->clock, dcc));
     env->yield_needed = 1;
 }
 

From c6f3f334d157ff6b9bdc4e1b9d9874234138836a Mon Sep 17 00:00:00 2001
From: Max Filippov <jcmvbkbc@gmail.com>
Date: Tue, 26 Apr 2022 09:24:01 -0700
Subject: [PATCH 088/935] hw/xtensa: fix reset value of MIROUT register of MX
 PIC

MX PIC comes out of reset with IRQ routing registers set to 0, thus
not delivering any external IRQ to any connected CPU by default.
Fix the model to match the hardware.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 hw/xtensa/mx_pic.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/xtensa/mx_pic.c b/hw/xtensa/mx_pic.c
index d889f953d1..8211c993eb 100644
--- a/hw/xtensa/mx_pic.c
+++ b/hw/xtensa/mx_pic.c
@@ -334,7 +334,7 @@ void xtensa_mx_pic_reset(void *opaque)
     mx->miasg = 0;
     mx->mipipart = 0;
     for (i = 0; i < mx->n_irq; ++i) {
-        mx->mirout[i] = 1;
+        mx->mirout[i] = 0;
     }
     for (i = 0; i < mx->n_cpu; ++i) {
         mx->cpu[i].mipicause = 0;

From b9400b1fbaeb69af3e3052721fad79b2e46efc65 Mon Sep 17 00:00:00 2001
From: Max Filippov <jcmvbkbc@gmail.com>
Date: Sun, 24 Apr 2022 08:31:29 -0700
Subject: [PATCH 089/935] tests/tcg/xtensa: fix build for cores without
 windowed registers

Don't try to initialize windowbase/windowstart in crt.S if they don't
exist.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 tests/tcg/xtensa/crt.S | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tests/tcg/xtensa/crt.S b/tests/tcg/xtensa/crt.S
index d9846acace..909872cd38 100644
--- a/tests/tcg/xtensa/crt.S
+++ b/tests/tcg/xtensa/crt.S
@@ -8,10 +8,12 @@
 .text
 .global _start
 _start:
+#if XCHAL_HAVE_WINDOWED
     movi    a2, 1
     wsr     a2, windowstart
     movi    a2, 0
     wsr     a2, windowbase
+#endif
     movi    a1, _fstack
     movi    a2, 0x4000f
     wsr     a2, ps

From 8164f14bb98dce986b755af4b3dfee3eb99c95a1 Mon Sep 17 00:00:00 2001
From: Max Filippov <jcmvbkbc@gmail.com>
Date: Sun, 24 Apr 2022 08:33:16 -0700
Subject: [PATCH 090/935] tests/tcg/xtensa: restore vecbase SR after test

Writing garbage into the vecbase SR results in hang in the subsequent
tests that expect to raise an exception. Restore vecbase SR to its
reset value after the test.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 tests/tcg/xtensa/test_sr.S | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tests/tcg/xtensa/test_sr.S b/tests/tcg/xtensa/test_sr.S
index b1a91a0637..34441c7aff 100644
--- a/tests/tcg/xtensa/test_sr.S
+++ b/tests/tcg/xtensa/test_sr.S
@@ -221,6 +221,8 @@ test_sr_mask /*scompare1*/12, 0, 0
 
 #if XCHAL_HAVE_VECBASE
 test_sr vecbase, 1
+movi	a2, XCHAL_VECBASE_RESET_VADDR
+wsr	a2, vecbase
 #else
 test_sr_mask /*vecbase*/231, 0, 0
 #endif

From 64407f6a9e0731c11a65119b7372dbe5b3a42eb9 Mon Sep 17 00:00:00 2001
From: Max Filippov <jcmvbkbc@gmail.com>
Date: Mon, 25 Apr 2022 17:16:01 -0700
Subject: [PATCH 091/935] tests/tcg/xtensa: fix watchpoint test

xtensa core may have only one set of DBREAKA/DBREAKC registers. Don't
hardcode register numbers in the test as 0 and 1, use macros that only
index valid DBREAK* registers.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 tests/tcg/xtensa/test_break.S | 86 +++++++++++++++++++----------------
 1 file changed, 46 insertions(+), 40 deletions(-)

diff --git a/tests/tcg/xtensa/test_break.S b/tests/tcg/xtensa/test_break.S
index 3379a3f9f0..3aa18b5cec 100644
--- a/tests/tcg/xtensa/test_break.S
+++ b/tests/tcg/xtensa/test_break.S
@@ -200,64 +200,70 @@ test_end
 .endm
 
 #if XCHAL_NUM_DBREAK
+#define DB0 0
+#if XCHAL_NUM_DBREAK > 1
+#define DB1 1
+#else
+#define DB1 0
+#endif
 test dbreak_exact
-    dbreak_test 0, 0x4000003f, 0xd000007f, 0xd000007f, l8ui
-    dbreak_test 1, 0x4000003e, 0xd000007e, 0xd000007e, l16ui
-    dbreak_test 0, 0x4000003c, 0xd000007c, 0xd000007c, l32i
+    dbreak_test DB0, 0x4000003f, 0xd000007f, 0xd000007f, l8ui
+    dbreak_test DB1, 0x4000003e, 0xd000007e, 0xd000007e, l16ui
+    dbreak_test DB0, 0x4000003c, 0xd000007c, 0xd000007c, l32i
 
-    dbreak_test 1, 0x8000003f, 0xd000007f, 0xd000007f, s8i
-    dbreak_test 0, 0x8000003e, 0xd000007e, 0xd000007e, s16i
-    dbreak_test 1, 0x8000003c, 0xd000007c, 0xd000007c, s32i
+    dbreak_test DB1, 0x8000003f, 0xd000007f, 0xd000007f, s8i
+    dbreak_test DB0, 0x8000003e, 0xd000007e, 0xd000007e, s16i
+    dbreak_test DB1, 0x8000003c, 0xd000007c, 0xd000007c, s32i
 test_end
 
-test dbreak_overlap
-    dbreak_test 0, 0x4000003f, 0xd000007d, 0xd000007c, l16ui
-    dbreak_test 1, 0x4000003f, 0xd000007d, 0xd000007c, l32i
+test DBdbreak_overlap
+    dbreak_test DB0, 0x4000003f, 0xd000007d, 0xd000007c, l16ui
+    dbreak_test DB1, 0x4000003f, 0xd000007d, 0xd000007c, l32i
 
-    dbreak_test 0, 0x4000003e, 0xd000007e, 0xd000007f, l8ui
-    dbreak_test 1, 0x4000003e, 0xd000007e, 0xd000007c, l32i
+    dbreak_test DB0, 0x4000003e, 0xd000007e, 0xd000007f, l8ui
+    dbreak_test DB1, 0x4000003e, 0xd000007e, 0xd000007c, l32i
 
-    dbreak_test 0, 0x4000003c, 0xd000007c, 0xd000007d, l8ui
-    dbreak_test 1, 0x4000003c, 0xd000007c, 0xd000007c, l16ui
+    dbreak_test DB0, 0x4000003c, 0xd000007c, 0xd000007d, l8ui
+    dbreak_test DB1, 0x4000003c, 0xd000007c, 0xd000007c, l16ui
 
-    dbreak_test 0, 0x40000038, 0xd0000078, 0xd000007b, l8ui
-    dbreak_test 1, 0x40000038, 0xd0000078, 0xd000007a, l16ui
-    dbreak_test 0, 0x40000038, 0xd0000078, 0xd000007c, l32i
+    dbreak_test DB0, 0x40000038, 0xd0000078, 0xd000007b, l8ui
+    dbreak_test DB1, 0x40000038, 0xd0000078, 0xd000007a, l16ui
+    dbreak_test DB0, 0x40000038, 0xd0000078, 0xd000007c, l32i
 
-    dbreak_test 1, 0x40000030, 0xd0000070, 0xd0000075, l8ui
-    dbreak_test 0, 0x40000030, 0xd0000070, 0xd0000076, l16ui
-    dbreak_test 1, 0x40000030, 0xd0000070, 0xd0000078, l32i
+    dbreak_test DB1, 0x40000030, 0xd0000070, 0xd0000075, l8ui
+    dbreak_test DB0, 0x40000030, 0xd0000070, 0xd0000076, l16ui
+    dbreak_test DB1, 0x40000030, 0xd0000070, 0xd0000078, l32i
 
-    dbreak_test 0, 0x40000020, 0xd0000060, 0xd000006f, l8ui
-    dbreak_test 1, 0x40000020, 0xd0000060, 0xd0000070, l16ui
-    dbreak_test 0, 0x40000020, 0xd0000060, 0xd0000074, l32i
+    dbreak_test DB0, 0x40000020, 0xd0000060, 0xd000006f, l8ui
+    dbreak_test DB1, 0x40000020, 0xd0000060, 0xd0000070, l16ui
+    dbreak_test DB0, 0x40000020, 0xd0000060, 0xd0000074, l32i
 
 
-    dbreak_test 0, 0x8000003f, 0xd000007d, 0xd000007c, s16i
-    dbreak_test 1, 0x8000003f, 0xd000007d, 0xd000007c, s32i
+    dbreak_test DB0, 0x8000003f, 0xd000007d, 0xd000007c, s16i
+    dbreak_test DB1, 0x8000003f, 0xd000007d, 0xd000007c, s32i
 
-    dbreak_test 0, 0x8000003e, 0xd000007e, 0xd000007f, s8i
-    dbreak_test 1, 0x8000003e, 0xd000007e, 0xd000007c, s32i
+    dbreak_test DB0, 0x8000003e, 0xd000007e, 0xd000007f, s8i
+    dbreak_test DB1, 0x8000003e, 0xd000007e, 0xd000007c, s32i
 
-    dbreak_test 0, 0x8000003c, 0xd000007c, 0xd000007d, s8i
-    dbreak_test 1, 0x8000003c, 0xd000007c, 0xd000007c, s16i
+    dbreak_test DB0, 0x8000003c, 0xd000007c, 0xd000007d, s8i
+    dbreak_test DB1, 0x8000003c, 0xd000007c, 0xd000007c, s16i
 
-    dbreak_test 0, 0x80000038, 0xd0000078, 0xd000007b, s8i
-    dbreak_test 1, 0x80000038, 0xd0000078, 0xd000007a, s16i
-    dbreak_test 0, 0x80000038, 0xd0000078, 0xd000007c, s32i
+    dbreak_test DB0, 0x80000038, 0xd0000078, 0xd000007b, s8i
+    dbreak_test DB1, 0x80000038, 0xd0000078, 0xd000007a, s16i
+    dbreak_test DB0, 0x80000038, 0xd0000078, 0xd000007c, s32i
 
-    dbreak_test 1, 0x80000030, 0xd0000070, 0xd0000075, s8i
-    dbreak_test 0, 0x80000030, 0xd0000070, 0xd0000076, s16i
-    dbreak_test 1, 0x80000030, 0xd0000070, 0xd0000078, s32i
+    dbreak_test DB1, 0x80000030, 0xd0000070, 0xd0000075, s8i
+    dbreak_test DB0, 0x80000030, 0xd0000070, 0xd0000076, s16i
+    dbreak_test DB1, 0x80000030, 0xd0000070, 0xd0000078, s32i
 
-    dbreak_test 0, 0x80000020, 0xd0000060, 0xd000006f, s8i
-    dbreak_test 1, 0x80000020, 0xd0000060, 0xd0000070, s16i
-    dbreak_test 0, 0x80000020, 0xd0000060, 0xd0000074, s32i
+    dbreak_test DB0, 0x80000020, 0xd0000060, 0xd000006f, s8i
+    dbreak_test DB1, 0x80000020, 0xd0000060, 0xd0000070, s16i
+    dbreak_test DB0, 0x80000020, 0xd0000060, 0xd0000074, s32i
 test_end
 
-test dbreak_invalid
-    dbreak_test 0, 0x40000030, 0xd0000071, 0xd0000070, l16ui
-    dbreak_test 1, 0x40000035, 0xd0000072, 0xd0000070, l32i
+test DBdbreak_invalid
+    dbreak_test DB0, 0x40000030, 0xd0000071, 0xd0000070, l16ui
+    dbreak_test DB1, 0x40000035, 0xd0000072, 0xd0000070, l32i
 test_end
 #endif
 

From e120c8335d1dfa9d194e3db8cc5195a6b47fb20c Mon Sep 17 00:00:00 2001
From: Max Filippov <jcmvbkbc@gmail.com>
Date: Mon, 25 Apr 2022 18:12:53 -0700
Subject: [PATCH 092/935] tests/tcg/xtensa: remove dependency on the loop
 option

xtensa core may not have the loop option, but still have timers. Don't
use loop opcode in the timer test.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 tests/tcg/xtensa/test_timer.S | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/tests/tcg/xtensa/test_timer.S b/tests/tcg/xtensa/test_timer.S
index 1ec8e20883..2a383e7719 100644
--- a/tests/tcg/xtensa/test_timer.S
+++ b/tests/tcg/xtensa/test_timer.S
@@ -115,9 +115,9 @@ test ccompare0_interrupt
     movi    a2, 1 << XCHAL_TIMER0_INTERRUPT
     wsr     a2, intenable
     rsil    a2, 0
-    loop    a3, 1f
-    nop
 1:
+    addi    a3, a3, -1
+    bnez    a3, 1b
     test_fail
 2:
     rsr     a2, exccause
@@ -148,9 +148,9 @@ test ccompare1_interrupt
     movi    a2, 1 << XCHAL_TIMER1_INTERRUPT
     wsr     a2, intenable
     rsil    a2, INTERRUPT_LEVEL(XCHAL_TIMER1_INTERRUPT) - 1
-    loop    a3, 1f
-    nop
 1:
+    addi    a3, a3, -1
+    bnez    a3, 1b
     test_fail
 2:
 test_end
@@ -177,9 +177,9 @@ test ccompare2_interrupt
     movi    a2, 1 << XCHAL_TIMER2_INTERRUPT
     wsr     a2, intenable
     rsil    a2, INTERRUPT_LEVEL(XCHAL_TIMER2_INTERRUPT) - 1
-    loop    a3, 1f
-    nop
 1:
+    addi    a3, a3, -1
+    bnez    a3, 1b
     test_fail
 2:
 test_end
@@ -197,7 +197,7 @@ test ccompare_interrupt_masked
     wsr     a2, ccompare2
 #endif
 
-    movi    a3, 2 * WAIT_LOOPS
+    movi    a3, WAIT_LOOPS
     make_ccount_delta a2, a15
 #if XCHAL_NUM_TIMERS > 1
     wsr     a2, ccompare1
@@ -211,9 +211,10 @@ test ccompare_interrupt_masked
     movi    a2, 1 << XCHAL_TIMER0_INTERRUPT
     wsr     a2, intenable
     rsil    a2, 0
-    loop    a3, 1f
-    nop
 1:
+    addi    a3, a3, -1
+    bnez    a3, 1b
+
     test_fail
 2:
     rsr     a2, exccause
@@ -231,7 +232,6 @@ test ccompare_interrupt_masked_waiti
     wsr     a2, ccompare2
 #endif
 
-    movi    a3, 2 * WAIT_LOOPS
     make_ccount_delta a2, a15
 #if XCHAL_NUM_TIMERS > 1
     wsr     a2, ccompare1

From 703cebcfac65356aa2b19c0a5e4aa1b4be23a328 Mon Sep 17 00:00:00 2001
From: Max Filippov <jcmvbkbc@gmail.com>
Date: Mon, 25 Apr 2022 20:05:18 -0700
Subject: [PATCH 093/935] tests/tcg/xtensa: enable autorefill phys_mem tests
 for MMUv3

Autorefill tests in the phys_mem test suite are disabled for cores that
have spanning TLB way, i.e. for all MMUv3 cores. Instead of disabling it
invalidate TLB mappings for entries that conflict with the test.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 tests/tcg/xtensa/test_phys_mem.S | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/tests/tcg/xtensa/test_phys_mem.S b/tests/tcg/xtensa/test_phys_mem.S
index 9bb3ee3866..f935a70294 100644
--- a/tests/tcg/xtensa/test_phys_mem.S
+++ b/tests/tcg/xtensa/test_phys_mem.S
@@ -2,7 +2,7 @@
 
 test_suite phys_mem
 
-#if XCHAL_HAVE_PTP_MMU && !XCHAL_HAVE_SPANNING_WAY
+#if XCHAL_HAVE_PTP_MMU
 
 .purgem test_init
 
@@ -13,6 +13,14 @@ test_suite phys_mem
     witlb   a2, a3
     movi    a2, 0xc0000000
     wsr     a2, ptevaddr
+#if XCHAL_HAVE_SPANNING_WAY
+    movi    a2, 0xc0000000 | XCHAL_SPANNING_WAY
+    idtlb   a2
+    iitlb   a2
+    movi    a2, 0x20000000 | XCHAL_SPANNING_WAY
+    idtlb   a2
+    iitlb   a2
+#endif
 .endm
 
 test inst_fetch_get_pte_no_phys

From da60ecd6d8bf6551a0211714a5abb11711e0222c Mon Sep 17 00:00:00 2001
From: Max Filippov <jcmvbkbc@gmail.com>
Date: Mon, 25 Apr 2022 20:05:18 -0700
Subject: [PATCH 094/935] tests/tcg/xtensa: enable mmu tests for MMUv3

MMU test suite is disabled for cores that have spanning TLB way, i.e.
for all MMUv3 cores. Instead of disabling it make testing region virtual
addresses explicit and invalidate TLB mappings for entries that conflict
with the test.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 tests/tcg/xtensa/test_mmu.S | 182 ++++++++++++++++++++----------------
 1 file changed, 103 insertions(+), 79 deletions(-)

diff --git a/tests/tcg/xtensa/test_mmu.S b/tests/tcg/xtensa/test_mmu.S
index 4cbd6ef4f9..1006c8cf77 100644
--- a/tests/tcg/xtensa/test_mmu.S
+++ b/tests/tcg/xtensa/test_mmu.S
@@ -2,7 +2,9 @@
 
 test_suite mmu
 
-#if XCHAL_HAVE_PTP_MMU && !XCHAL_HAVE_SPANNING_WAY
+#if XCHAL_HAVE_PTP_MMU
+#define BASE 0x20000000
+#define TLB_BASE 0x80000000
 
 .purgem test_init
 
@@ -29,17 +31,27 @@ test_suite mmu
     idtlb   a2
     movi    a2, 0x00000009
     idtlb   a2
+#if XCHAL_HAVE_SPANNING_WAY
+    movi    a2, BASE | XCHAL_SPANNING_WAY
+    idtlb   a2
+    iitlb   a2
+    movi    a2, TLB_BASE | XCHAL_SPANNING_WAY
+    idtlb   a2
+    iitlb   a2
+    movi    a2, TLB_BASE
+    wsr     a2, ptevaddr
+#endif
 .endm
 
 test tlb_group
     movi    a2, 0x04000002 /* PPN */
-    movi    a3, 0x01200004 /* VPN */
+    movi    a3, BASE + 0x01200004 /* VPN */
     wdtlb   a2, a3
     witlb   a2, a3
     movi    a3, 0x00200004
     rdtlb0  a1, a3
     ritlb0  a2, a3
-    movi    a3, 0x01000001
+    movi    a3, BASE + 0x01000001
     assert  eq, a1, a3
     assert  eq, a2, a3
     movi    a3, 0x00200004
@@ -48,17 +60,17 @@ test tlb_group
     movi    a3, 0x04000002
     assert  eq, a1, a3
     assert  eq, a2, a3
-    movi    a3, 0x01234567
+    movi    a3, BASE + 0x01234567
     pdtlb   a1, a3
     pitlb   a2, a3
-    movi    a3, 0x01234014
+    movi    a3, BASE + 0x01234014
     assert  eq, a1, a3
-    movi    a3, 0x0123400c
+    movi    a3, BASE + 0x0123400c
     assert  eq, a2, a3
     movi    a3, 0x00200004
     idtlb   a3
     iitlb   a3
-    movi    a3, 0x01234567
+    movi    a3, BASE + 0x01234567
     pdtlb   a1, a3
     pitlb   a2, a3
     movi    a3, 0x00000010
@@ -72,7 +84,7 @@ test_end
 test itlb_miss
     set_vector kernel, 1f
 
-    movi    a3, 0x00100000
+    movi    a3, BASE + 0x00100000
     jx      a3
     test_fail
 1:
@@ -86,7 +98,7 @@ test_end
 test dtlb_miss
     set_vector kernel, 1f
 
-    movi    a3, 0x00100000
+    movi    a3, BASE + 0x00100000
     l8ui    a2, a3, 0
     test_fail
 1:
@@ -116,11 +128,11 @@ test dtlb_multi_hit
     set_vector kernel, 1f
 
     movi    a2, 0x04000002 /* PPN */
-    movi    a3, 0x01200004 /* VPN */
+    movi    a3, BASE + 0x01200004 /* VPN */
     wdtlb   a2, a3
-    movi    a3, 0x01200007 /* VPN */
+    movi    a3, BASE + 0x01200007 /* VPN */
     wdtlb   a2, a3
-    movi    a3, 0x01200000
+    movi    a3, BASE + 0x01200000
     pdtlb   a2, a3
     test_fail
 1:
@@ -168,15 +180,18 @@ test load_store_privilege
     and     a3, a3, a1
     movi    a1, 4
     or      a3, a3, a1
+    movi    a5, BASE
+    add     a3, a3, a5
     witlb   a2, a3
     movi    a3, 10f
     movi    a1, 0x000fffff
     and     a1, a3, a1
+    add     a1, a1, a5
 
     movi    a2, 0x04000003 /* PPN */
-    movi    a3, 0x01200004 /* VPN */
+    movi    a3, BASE + 0x01200004 /* VPN */
     wdtlb   a2, a3
-    movi    a3, 0x01200001
+    movi    a3, BASE + 0x01200001
     movi    a2, 0x4004f
     jx      a1
 10:
@@ -192,6 +207,7 @@ test load_store_privilege
     movi    a3, 1b
     movi    a1, 0x000fffff
     and     a3, a3, a1
+    add     a3, a3, a5
     assert  eq, a2, a3
     rsr     a2, exccause
     movi    a3, 26
@@ -206,9 +222,9 @@ test cring_load_store_privilege
     set_vector double, 2f
 
     movi    a2, 0x04000003 /* PPN */
-    movi    a3, 0x01200004 /* VPN */
+    movi    a3, BASE + 0x01200004 /* VPN */
     wdtlb   a2, a3
-    movi    a3, 0x01200004
+    movi    a3, BASE + 0x01200004
     movi    a2, 0x4005f    /* ring 1 + excm => cring == 0 */
     wsr     a2, ps
     isync
@@ -245,10 +261,13 @@ test inst_fetch_prohibited
     and     a3, a3, a1
     movi    a1, 4
     or      a3, a3, a1
+    movi    a5, BASE
+    add     a3, a3, a5
     witlb   a2, a3
     movi    a3, 10f
     movi    a1, 0x000fffff
     and     a1, a3, a1
+    add     a1, a1, a5
     jx      a1
     .align  4
 10:
@@ -268,9 +287,9 @@ test load_prohibited
     set_vector kernel, 2f
 
     movi    a2, 0x0400000c /* PPN */
-    movi    a3, 0x01200004 /* VPN */
+    movi    a3, BASE + 0x01200004 /* VPN */
     wdtlb   a2, a3
-    movi    a3, 0x01200002
+    movi    a3, BASE + 0x01200002
 1:
     l8ui    a2, a3, 0
     test_fail
@@ -289,9 +308,9 @@ test store_prohibited
     set_vector kernel, 2f
 
     movi    a2, 0x04000001 /* PPN */
-    movi    a3, 0x01200004 /* VPN */
+    movi    a3, BASE + 0x01200004 /* VPN */
     wdtlb   a2, a3
-    movi    a3, 0x01200003
+    movi    a3, BASE + 0x01200003
     l8ui    a2, a3, 0
 1:
     s8i     a2, a3, 0
@@ -311,10 +330,10 @@ test_end
  * and DTLB way 7 to cover this PTE, ring=pt_ring, attr=pt_attr
  */
 .macro pt_setup pt_ring, pt_attr, pte_ring, vaddr, paddr, pte_attr
-    movi    a2, 0x80000000
+    movi    a2, TLB_BASE
     wsr     a2, ptevaddr
 
-    movi    a3, 0x80000007 | (((\vaddr) >> 10) & 0xfffff000) /* way 7 */
+    movi    a3, TLB_BASE | 7 | (((\vaddr) >> 10) & 0xfffff000) /* way 7 */
     movi    a4, 0x04000003 | ((\pt_ring) << 4) /* PADDR 64M */
     wdtlb   a4, a3
     isync
@@ -324,7 +343,7 @@ test_end
     add     a2, a1, a2
     s32i    a3, a2, 0
 
-    movi    a3, 0x80000007 | (((\vaddr) >> 10) & 0xfffff000) /* way 7 */
+    movi    a3, TLB_BASE | 7 | (((\vaddr) >> 10) & 0xfffff000) /* way 7 */
     movi    a4, 0x04000000 | ((\pt_ring) << 4) | (\pt_attr) /* PADDR 64M */
     wdtlb   a4, a3
     isync
@@ -343,10 +362,13 @@ test_end
     and     a3, a3, a1
     movi    a1, 4
     or      a3, a3, a1
+    movi    a5, BASE
+    add     a3, a3, a5
     witlb   a2, a3
     movi    a3, 10f
     movi    a1, 0x000fffff
     and     a1, a3, a1
+    add     a1, a1, a5
 
     movi    a2, 0
     wsr     a2, excvaddr
@@ -396,6 +418,8 @@ test_end
     movi    a2, (\vaddr)
     movi    a1, 0xfffff
     and     a1, a1, a2
+    movi    a5, BASE
+    add     a1, a1, a5
     rsr     a2, epc1
     assert  eq, a1, a2
 .endm
@@ -403,7 +427,7 @@ test_end
 test dtlb_autoload
     set_vector kernel, 0
 
-    pt_setup    0, 3, 1, 0x1000, 0x1000, 3
+    pt_setup    0, 3, 1, BASE + 0x1000, 0x1000, 3
     assert_no_auto_tlb
 
     l8ui    a1, a3, 0
@@ -418,8 +442,8 @@ test autoload_load_store_privilege
     set_vector kernel, 0
     set_vector double, 2f
 
-    pt_setup    0, 3, 0, 0x2000, 0x2000, 3
-    movi    a3, 0x2004
+    pt_setup    0, 3, 0, BASE + 0x2000, 0x2000, 3
+    movi    a3, BASE + 0x2004
     assert_no_auto_tlb
 
     movi    a2, 0x4005f    /* ring 1 + excm => cring == 0 */
@@ -441,7 +465,7 @@ test_end
 test autoload_pte_load_prohibited
     set_vector kernel, 2f
 
-    pt_setup    0, 3, 0, 0x3000, 0, 0xc
+    pt_setup    0, 3, 0, BASE + 0x3000, 0, 0xc
     assert_no_auto_tlb
 1:
     l32i    a2, a3, 0
@@ -458,7 +482,7 @@ test_end
 test autoload_pt_load_prohibited
     set_vector kernel, 2f
 
-    pt_setup    0, 0xc, 0, 0x4000, 0x4000, 3
+    pt_setup    0, 0xc, 0, BASE + 0x4000, 0x4000, 3
     assert_no_auto_tlb
 1:
     l32i    a2, a3, 0
@@ -474,8 +498,8 @@ test_end
 
 test autoload_pt_privilege
     set_vector  kernel, 2f
-    pt_setup    0, 3, 1, 0x5000, 0, 3
-    go_ring     1, 0, 0x5001
+    pt_setup    0, 3, 1, BASE + 0x5000, 0, 3
+    go_ring     1, 0, BASE + 0x5001
 
     l8ui    a2, a3, 0
 1:
@@ -491,8 +515,8 @@ test_end
 
 test autoload_pte_privilege
     set_vector  kernel, 2f
-    pt_setup    0, 3, 0, 0x6000, 0, 3
-    go_ring     1, 0, 0x6001
+    pt_setup    0, 3, 0, BASE + 0x6000, 0, 3
+    go_ring     1, 0, BASE + 0x6001
 1:
     l8ui    a2, a3, 0
     syscall
@@ -507,9 +531,9 @@ test_end
 
 test autoload_3_level_pt
     set_vector  kernel, 2f
-    pt_setup    1, 3, 1, 0x00400000, 0, 3
-    pt_setup    1, 3, 1, 0x80001000, 0x2000000, 3
-    go_ring     1, 0, 0x00400001
+    pt_setup    1, 3, 1, BASE + 0x00400000, 0, 3
+    pt_setup    1, 3, 1, TLB_BASE + ((BASE + 0x00400000) >> 10), 0x2000000, 3
+    go_ring     1, 0, BASE + 0x00400001
 1:
     l8ui    a2, a3, 0
     syscall
@@ -526,14 +550,14 @@ test cross_page_insn
     set_vector kernel, 2f
 
     movi    a2, 0x04000003 /* PPN */
-    movi    a3, 0x00007000 /* VPN */
+    movi    a3, BASE + 0x00007000 /* VPN */
     witlb   a2, a3
     wdtlb   a2, a3
-    movi    a3, 0x00008000 /* VPN */
+    movi    a3, BASE + 0x00008000 /* VPN */
     witlb   a2, a3
     wdtlb   a2, a3
 
-    movi    a2, 0x00007fff
+    movi    a2, BASE + 0x00007fff
     movi    a3, 20f
     movi    a4, 21f
     sub     a4, a4, a3
@@ -543,8 +567,8 @@ test cross_page_insn
     addi    a2, a2, 1
     addi    a3, a3, 1
 1:
-    movi    a2, 0x00007fff
-    movi    a3, 0x00008000
+    movi    a2, BASE + 0x00007fff
+    movi    a3, BASE + 0x00008000
     /* DTLB: OK, ITLB: OK */
     jx      a2
 
@@ -560,20 +584,20 @@ test cross_page_insn
     movi    a3, 1
     assert  eq, a2, a3
     rsr     a2, epc1
-    movi    a3, 0x8002
+    movi    a3, BASE + 0x8002
     assert  eq, a2, a3
     rsr     a2, excsave1
-    movi    a3, 0x00007fff
+    movi    a3, BASE + 0x00007fff
     assert  ne, a2, a3
 
     reset_ps
     set_vector kernel, 3f
 
     movi    a2, 0x0400000c /* PPN */
-    movi    a3, 0x00008000 /* VPN */
+    movi    a3, BASE + 0x00008000 /* VPN */
     wdtlb   a2, a3
-    movi    a2, 0x00007fff
-    movi    a3, 0x00008000
+    movi    a2, BASE + 0x00007fff
+    movi    a3, BASE + 0x00008000
     /* DTLB: FAIL, ITLB: OK */
     jx      a2
 3:
@@ -581,22 +605,22 @@ test cross_page_insn
     movi    a3, 28
     assert  eq, a2, a3
     rsr     a2, epc1
-    movi    a3, 0x7fff
+    movi    a3, BASE + 0x7fff
     assert  eq, a2, a3
     rsr     a2, excsave1
-    movi    a3, 0x00007fff
+    movi    a3, BASE + 0x00007fff
     assert  eq, a2, a3
 
     reset_ps
     set_vector kernel, 4f
 
     movi    a2, 0x0400000c /* PPN */
-    movi    a3, 0x00008000 /* VPN */
+    movi    a3, BASE + 0x00008000 /* VPN */
     witlb   a2, a3
     movi    a2, 0x04000003 /* PPN */
     wdtlb   a2, a3
-    movi    a2, 0x00007fff
-    movi    a3, 0x00008000
+    movi    a2, BASE + 0x00007fff
+    movi    a3, BASE + 0x00008000
     /* DTLB: OK, ITLB: FAIL */
     jx      a2
 4:
@@ -604,20 +628,20 @@ test cross_page_insn
     movi    a3, 20
     assert  eq, a2, a3
     rsr     a2, epc1
-    movi    a3, 0x7fff
+    movi    a3, BASE + 0x7fff
     assert  eq, a2, a3
     rsr     a2, excsave1
-    movi    a3, 0x00007fff
+    movi    a3, BASE + 0x00007fff
     assert  eq, a2, a3
 
     reset_ps
     set_vector kernel, 5f
 
     movi    a2, 0x0400000c /* PPN */
-    movi    a3, 0x00008000 /* VPN */
+    movi    a3, BASE + 0x00008000 /* VPN */
     wdtlb   a2, a3
-    movi    a2, 0x00007fff
-    movi    a3, 0x00008000
+    movi    a2, BASE + 0x00007fff
+    movi    a3, BASE + 0x00008000
     /* DTLB: FAIL, ITLB: FAIL */
     jx      a2
 5:
@@ -625,10 +649,10 @@ test cross_page_insn
     movi    a3, 20
     assert  eq, a2, a3
     rsr     a2, epc1
-    movi    a3, 0x7fff
+    movi    a3, BASE + 0x7fff
     assert  eq, a2, a3
     rsr     a2, excsave1
-    movi    a3, 0x00007fff
+    movi    a3, BASE + 0x00007fff
     assert  eq, a2, a3
 test_end
 
@@ -636,14 +660,14 @@ test cross_page_tb
     set_vector kernel, 2f
 
     movi    a2, 0x04000003 /* PPN */
-    movi    a3, 0x00007000 /* VPN */
+    movi    a3, BASE + 0x00007000 /* VPN */
     witlb   a2, a3
     wdtlb   a2, a3
-    movi    a3, 0x00008000 /* VPN */
+    movi    a3, BASE + 0x00008000 /* VPN */
     witlb   a2, a3
     wdtlb   a2, a3
 
-    movi    a2, 0x00007ffc
+    movi    a2, BASE + 0x00007ffc
     movi    a3, 20f
     movi    a4, 21f
     sub     a4, a4, a3
@@ -653,8 +677,8 @@ test cross_page_tb
     addi    a2, a2, 1
     addi    a3, a3, 1
 1:
-    movi    a2, 0x00007ffc
-    movi    a3, 0x00008000
+    movi    a2, BASE + 0x00007ffc
+    movi    a3, BASE + 0x00008000
     /* DTLB: OK, ITLB: OK */
     jx      a2
 
@@ -670,20 +694,20 @@ test cross_page_tb
     movi    a3, 1
     assert  eq, a2, a3
     rsr     a2, epc1
-    movi    a3, 0x7fff
+    movi    a3, BASE + 0x7fff
     assert  eq, a2, a3
     rsr     a2, excsave1
-    movi    a3, 0x00007ffc
+    movi    a3, BASE + 0x00007ffc
     assert  ne, a2, a3
 
     reset_ps
     set_vector kernel, 3f
 
     movi    a2, 0x0400000c /* PPN */
-    movi    a3, 0x00008000 /* VPN */
+    movi    a3, BASE + 0x00008000 /* VPN */
     wdtlb   a2, a3
-    movi    a2, 0x00007ffc
-    movi    a3, 0x00008000
+    movi    a2, BASE + 0x00007ffc
+    movi    a3, BASE + 0x00008000
     /* DTLB: FAIL, ITLB: OK */
     jx      a2
 3:
@@ -691,22 +715,22 @@ test cross_page_tb
     movi    a3, 28
     assert  eq, a2, a3
     rsr     a2, epc1
-    movi    a3, 0x7ffc
+    movi    a3, BASE + 0x7ffc
     assert  eq, a2, a3
     rsr     a2, excsave1
-    movi    a3, 0x00007ffc
+    movi    a3, BASE + 0x00007ffc
     assert  eq, a2, a3
 
     reset_ps
     set_vector kernel, 4f
 
     movi    a2, 0x0400000c /* PPN */
-    movi    a3, 0x00008000 /* VPN */
+    movi    a3, BASE + 0x00008000 /* VPN */
     witlb   a2, a3
     movi    a2, 0x04000003 /* PPN */
     wdtlb   a2, a3
-    movi    a2, 0x00007ffc
-    movi    a3, 0x00008000
+    movi    a2, BASE + 0x00007ffc
+    movi    a3, BASE + 0x00008000
     /* DTLB: OK, ITLB: FAIL */
     jx      a2
 4:
@@ -714,20 +738,20 @@ test cross_page_tb
     movi    a3, 20
     assert  eq, a2, a3
     rsr     a2, epc1
-    movi    a3, 0x7fff
+    movi    a3, BASE + 0x7fff
     assert  eq, a2, a3
     rsr     a2, excsave1
-    movi    a3, 0x00007ffc
+    movi    a3, BASE + 0x00007ffc
     assert  ne, a2, a3
 
     reset_ps
     set_vector kernel, 5f
 
     movi    a2, 0x0400000c /* PPN */
-    movi    a3, 0x00008000 /* VPN */
+    movi    a3, BASE + 0x00008000 /* VPN */
     wdtlb   a2, a3
-    movi    a2, 0x00007ffc
-    movi    a3, 0x00008000
+    movi    a2, BASE + 0x00007ffc
+    movi    a3, BASE + 0x00008000
     /* DTLB: FAIL, ITLB: FAIL */
     jx      a2
 5:
@@ -735,10 +759,10 @@ test cross_page_tb
     movi    a3, 28
     assert  eq, a2, a3
     rsr     a2, epc1
-    movi    a3, 0x7ffc
+    movi    a3, BASE + 0x7ffc
     assert  eq, a2, a3
     rsr     a2, excsave1
-    movi    a3, 0x00007ffc
+    movi    a3, BASE + 0x00007ffc
     assert  eq, a2, a3
 test_end
 

From 4be4c5b826989bed67a16f6e5b931d8374589c08 Mon Sep 17 00:00:00 2001
From: Max Filippov <jcmvbkbc@gmail.com>
Date: Wed, 27 Apr 2022 10:06:00 -0700
Subject: [PATCH 095/935] tests/tcg/xtensa: fix vectors and checks in timer
 test

Timer test assumes that timer 0 IRQ has level 1 and other timers have
higher level IRQs. This assumption is not correct and the levels may be
arbitrary. Fix that assumption by providing TIMER*_VECTOR macro and
using it for vector selection and by making the check for the timer
exception cause conditional.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 tests/tcg/xtensa/test_timer.S | 48 ++++++++++++++++++++++++++++++-----
 1 file changed, 41 insertions(+), 7 deletions(-)

diff --git a/tests/tcg/xtensa/test_timer.S b/tests/tcg/xtensa/test_timer.S
index 2a383e7719..2a06eebad8 100644
--- a/tests/tcg/xtensa/test_timer.S
+++ b/tests/tcg/xtensa/test_timer.S
@@ -38,6 +38,28 @@ test_end
 
 #if XCHAL_NUM_TIMERS
 
+#if INTERRUPT_LEVEL(XCHAL_TIMER0_INTERRUPT) == 1
+#define TIMER0_VECTOR kernel
+#else
+#define TIMER0_VECTOR glue(level, INTERRUPT_LEVEL(XCHAL_TIMER0_INTERRUPT))
+#endif
+
+#if XCHAL_NUM_TIMERS > 1
+#if INTERRUPT_LEVEL(XCHAL_TIMER1_INTERRUPT) == 1
+#define TIMER1_VECTOR kernel
+#else
+#define TIMER1_VECTOR glue(level, INTERRUPT_LEVEL(XCHAL_TIMER1_INTERRUPT))
+#endif
+#endif
+
+#if XCHAL_NUM_TIMERS > 2
+#if INTERRUPT_LEVEL(XCHAL_TIMER2_INTERRUPT) == 1
+#define TIMER2_VECTOR kernel
+#else
+#define TIMER2_VECTOR glue(level, INTERRUPT_LEVEL(XCHAL_TIMER2_INTERRUPT))
+#endif
+#endif
+
 test ccount_update_deadline
     movi    a2, 0
     wsr     a2, intenable
@@ -90,9 +112,8 @@ test ccompare
     assert  nei, a5, 0
 test_end
 
-#if INTERRUPT_LEVEL(XCHAL_TIMER0_INTERRUPT) == 1
 test ccompare0_interrupt
-    set_vector kernel, 2f
+    set_vector TIMER0_VECTOR, 2f
     movi    a2, 0
     wsr     a2, intenable
     rsr     a2, interrupt
@@ -120,15 +141,16 @@ test ccompare0_interrupt
     bnez    a3, 1b
     test_fail
 2:
+#if INTERRUPT_LEVEL(XCHAL_TIMER0_INTERRUPT) == 1
     rsr     a2, exccause
     assert  eqi, a2, 4 /* LEVEL1_INTERRUPT_CAUSE */
-test_end
 #endif
+test_end
 
 #if XCHAL_NUM_TIMERS > 1
 
 test ccompare1_interrupt
-    set_vector glue(level, INTERRUPT_LEVEL(XCHAL_TIMER1_INTERRUPT)), 2f
+    set_vector TIMER1_VECTOR, 2f
     movi    a2, 0
     wsr     a2, intenable
     rsr     a2, interrupt
@@ -153,13 +175,17 @@ test ccompare1_interrupt
     bnez    a3, 1b
     test_fail
 2:
+#if INTERRUPT_LEVEL(XCHAL_TIMER1_INTERRUPT) == 1
+    rsr     a2, exccause
+    assert  eqi, a2, 4 /* LEVEL1_INTERRUPT_CAUSE */
+#endif
 test_end
 
 #endif
 #if XCHAL_NUM_TIMERS > 2
 
 test ccompare2_interrupt
-    set_vector glue(level, INTERRUPT_LEVEL(XCHAL_TIMER2_INTERRUPT)), 2f
+    set_vector TIMER2_VECTOR, 2f
     movi    a2, 0
     wsr     a2, intenable
     rsr     a2, interrupt
@@ -182,12 +208,16 @@ test ccompare2_interrupt
     bnez    a3, 1b
     test_fail
 2:
+#if INTERRUPT_LEVEL(XCHAL_TIMER2_INTERRUPT) == 1
+    rsr     a2, exccause
+    assert  eqi, a2, 4 /* LEVEL1_INTERRUPT_CAUSE */
+#endif
 test_end
 
 #endif
 
 test ccompare_interrupt_masked
-    set_vector kernel, 2f
+    set_vector TIMER0_VECTOR, 2f
     movi    a2, 0
     wsr     a2, intenable
     rsr     a2, interrupt
@@ -217,12 +247,14 @@ test ccompare_interrupt_masked
 
     test_fail
 2:
+#if INTERRUPT_LEVEL(XCHAL_TIMER0_INTERRUPT) == 1
     rsr     a2, exccause
     assert  eqi, a2, 4 /* LEVEL1_INTERRUPT_CAUSE */
+#endif
 test_end
 
 test ccompare_interrupt_masked_waiti
-    set_vector kernel, 2f
+    set_vector TIMER0_VECTOR, 2f
     movi    a2, 0
     wsr     a2, intenable
     rsr     a2, interrupt
@@ -247,8 +279,10 @@ test ccompare_interrupt_masked_waiti
     waiti   0
     test_fail
 2:
+#if INTERRUPT_LEVEL(XCHAL_TIMER0_INTERRUPT) == 1
     rsr     a2, exccause
     assert  eqi, a2, 4 /* LEVEL1_INTERRUPT_CAUSE */
+#endif
 test_end
 
 #endif

From 59491e97f89eaeee275f57fb6bb40f0152429fb3 Mon Sep 17 00:00:00 2001
From: Max Filippov <jcmvbkbc@gmail.com>
Date: Sun, 1 May 2022 22:57:49 -0700
Subject: [PATCH 096/935] target/xtensa: implement cache test option opcodes

We don't model caches, so for l*ct opcodes return tags with all bits
(including Valid) set to 0. For all other opcodes don't do anything.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
 target/xtensa/translate.c | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/target/xtensa/translate.c b/target/xtensa/translate.c
index b0576d90fa..70e11eeb45 100644
--- a/target/xtensa/translate.c
+++ b/target/xtensa/translate.c
@@ -1766,6 +1766,12 @@ static void translate_ldst(DisasContext *dc, const OpcodeArg arg[],
     tcg_temp_free(addr);
 }
 
+static void translate_lct(DisasContext *dc, const OpcodeArg arg[],
+                          const uint32_t par[])
+{
+    tcg_gen_movi_i32(arg[0].out, 0);
+}
+
 static void translate_l32r(DisasContext *dc, const OpcodeArg arg[],
                            const uint32_t par[])
 {
@@ -3319,6 +3325,14 @@ static const XtensaOpcodeOps core_ops[] = {
         .translate = translate_ldst,
         .par = (const uint32_t[]){MO_UB, false, false},
         .op_flags = XTENSA_OP_LOAD,
+    }, {
+        .name = "ldct",
+        .translate = translate_lct,
+        .op_flags = XTENSA_OP_PRIVILEGED,
+    }, {
+        .name = "ldcw",
+        .translate = translate_nop,
+        .op_flags = XTENSA_OP_PRIVILEGED,
     }, {
         .name = "lddec",
         .translate = translate_mac16,
@@ -3332,6 +3346,14 @@ static const XtensaOpcodeOps core_ops[] = {
     }, {
         .name = "ldpte",
         .op_flags = XTENSA_OP_ILL,
+    }, {
+        .name = "lict",
+        .translate = translate_lct,
+        .op_flags = XTENSA_OP_PRIVILEGED,
+    }, {
+        .name = "licw",
+        .translate = translate_nop,
+        .op_flags = XTENSA_OP_PRIVILEGED,
     }, {
         .name = (const char * const[]) {
             "loop", "loop.w15", NULL,
@@ -4635,12 +4657,28 @@ static const XtensaOpcodeOps core_ops[] = {
         .name = "saltu",
         .translate = translate_salt,
         .par = (const uint32_t[]){TCG_COND_LTU},
+    }, {
+        .name = "sdct",
+        .translate = translate_nop,
+        .op_flags = XTENSA_OP_PRIVILEGED,
+    }, {
+        .name = "sdcw",
+        .translate = translate_nop,
+        .op_flags = XTENSA_OP_PRIVILEGED,
     }, {
         .name = "setb_expstate",
         .translate = translate_setb_expstate,
     }, {
         .name = "sext",
         .translate = translate_sext,
+    }, {
+        .name = "sict",
+        .translate = translate_nop,
+        .op_flags = XTENSA_OP_PRIVILEGED,
+    }, {
+        .name = "sicw",
+        .translate = translate_nop,
+        .op_flags = XTENSA_OP_PRIVILEGED,
     }, {
         .name = "simcall",
         .translate = translate_simcall,

From b6b3da999844b710d8d8b97a93c8420e6d46cd44 Mon Sep 17 00:00:00 2001
From: Sunil Muthuswamy <sunilmut@microsoft.com>
Date: Thu, 7 Nov 2019 19:48:32 +0000
Subject: [PATCH 097/935] WHPX: support for xcr0

Support for xcr0 to be able to enable xsave/xrstor. This by itself
is not sufficient to enable xsave/xrstor. WHPX XSAVE API's also
needs to be hooked up.

Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Message-Id: <MW2PR2101MB1116F07C07A26FD7A7ED8DCFC0780@MW2PR2101MB1116.namprd21.prod.outlook.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/whpx/whpx-all.c      | 87 ++++++++++++++++++++++++++++++++
 target/i386/whpx/whpx-internal.h |  3 ++
 2 files changed, 90 insertions(+)

diff --git a/target/i386/whpx/whpx-all.c b/target/i386/whpx/whpx-all.c
index b625ad5bbb..23ae639b23 100644
--- a/target/i386/whpx/whpx-all.c
+++ b/target/i386/whpx/whpx-all.c
@@ -246,9 +246,15 @@ static bool whpx_allowed;
 static bool whp_dispatch_initialized;
 static HMODULE hWinHvPlatform, hWinHvEmulation;
 static uint32_t max_vcpu_index;
+static WHV_PROCESSOR_XSAVE_FEATURES whpx_xsave_cap;
+
 struct whpx_state whpx_global;
 struct WHPDispatch whp_dispatch;
 
+static bool whpx_has_xsave(void)
+{
+    return whpx_xsave_cap.XsaveSupport;
+}
 
 /*
  * VP support
@@ -300,6 +306,28 @@ static SegmentCache whpx_seg_h2q(const WHV_X64_SEGMENT_REGISTER *hs)
     return qs;
 }
 
+/* X64 Extended Control Registers */
+static void whpx_set_xcrs(CPUState *cpu)
+{
+    CPUX86State *env = cpu->env_ptr;
+    HRESULT hr;
+    struct whpx_state *whpx = &whpx_global;
+    WHV_REGISTER_VALUE xcr0;
+    WHV_REGISTER_NAME xcr0_name = WHvX64RegisterXCr0;
+
+    if (!whpx_has_xsave()) {
+        return;
+    }
+
+    /* Only xcr0 is supported by the hypervisor currently */
+    xcr0.Reg64 = env->xcr0;
+    hr = whp_dispatch.WHvSetVirtualProcessorRegisters(
+        whpx->partition, cpu->cpu_index, &xcr0_name, 1, &xcr0);
+    if (FAILED(hr)) {
+        error_report("WHPX: Failed to set register xcr0, hr=%08lx", hr);
+    }
+}
+
 static int whpx_set_tsc(CPUState *cpu)
 {
     CPUX86State *env = cpu->env_ptr;
@@ -435,6 +463,12 @@ static void whpx_set_registers(CPUState *cpu, int level)
 
     /* 8 Debug Registers - Skipped */
 
+    /*
+     * Extended control registers needs to be handled separately depending
+     * on whether xsave is supported/enabled or not.
+     */
+    whpx_set_xcrs(cpu);
+
     /* 16 XMM registers */
     assert(whpx_register_names[idx] == WHvX64RegisterXmm0);
     idx_next = idx + 16;
@@ -541,6 +575,30 @@ static int whpx_get_tsc(CPUState *cpu)
     return 0;
 }
 
+/* X64 Extended Control Registers */
+static void whpx_get_xcrs(CPUState *cpu)
+{
+    CPUX86State *env = cpu->env_ptr;
+    HRESULT hr;
+    struct whpx_state *whpx = &whpx_global;
+    WHV_REGISTER_VALUE xcr0;
+    WHV_REGISTER_NAME xcr0_name = WHvX64RegisterXCr0;
+
+    if (!whpx_has_xsave()) {
+        return;
+    }
+
+    /* Only xcr0 is supported by the hypervisor currently */
+    hr = whp_dispatch.WHvGetVirtualProcessorRegisters(
+        whpx->partition, cpu->cpu_index, &xcr0_name, 1, &xcr0);
+    if (FAILED(hr)) {
+        error_report("WHPX: Failed to get register xcr0, hr=%08lx", hr);
+        return;
+    }
+
+    env->xcr0 = xcr0.Reg64;
+}
+
 static void whpx_get_registers(CPUState *cpu)
 {
     struct whpx_state *whpx = &whpx_global;
@@ -634,6 +692,12 @@ static void whpx_get_registers(CPUState *cpu)
 
     /* 8 Debug Registers - Skipped */
 
+    /*
+     * Extended control registers needs to be handled separately depending
+     * on whether xsave is supported/enabled or not.
+     */
+    whpx_get_xcrs(cpu);
+
     /* 16 XMM registers */
     assert(whpx_register_names[idx] == WHvX64RegisterXmm0);
     idx_next = idx + 16;
@@ -2513,6 +2577,29 @@ static int whpx_accel_init(MachineState *ms)
         goto error;
     }
 
+    /*
+     * Query the XSAVE capability of the partition. Any error here is not
+     * considered fatal.
+     */
+    hr = whp_dispatch.WHvGetPartitionProperty(
+        whpx->partition,
+        WHvPartitionPropertyCodeProcessorXsaveFeatures,
+        &whpx_xsave_cap,
+        sizeof(whpx_xsave_cap),
+        &whpx_cap_size);
+
+    /*
+     * Windows version which don't support this property will return with the
+     * specific error code.
+     */
+    if (FAILED(hr) && hr != WHV_E_UNKNOWN_PROPERTY) {
+        error_report("WHPX: Failed to query XSAVE capability, hr=%08lx", hr);
+    }
+
+    if (!whpx_has_xsave()) {
+        printf("WHPX: Partition is not XSAVE capable\n");
+    }
+
     memset(&prop, 0, sizeof(WHV_PARTITION_PROPERTY));
     prop.ProcessorCount = ms->smp.cpus;
     hr = whp_dispatch.WHvSetPartitionProperty(
diff --git a/target/i386/whpx/whpx-internal.h b/target/i386/whpx/whpx-internal.h
index 2416ec7922..dbb7e7ba82 100644
--- a/target/i386/whpx/whpx-internal.h
+++ b/target/i386/whpx/whpx-internal.h
@@ -48,6 +48,9 @@ void whpx_apic_get(DeviceState *s);
 
 #define WHV_E_UNKNOWN_CAPABILITY 0x80370300L
 
+/* This should eventually come from the Windows SDK */
+#define WHV_E_UNKNOWN_PROPERTY 0x80370302
+
 #define LIST_WINHVPLATFORM_FUNCTIONS(X) \
   X(HRESULT, WHvGetCapability, (WHV_CAPABILITY_CODE CapabilityCode, VOID* CapabilityBuffer, UINT32 CapabilityBufferSizeInBytes, UINT32* WrittenSizeInBytes)) \
   X(HRESULT, WHvCreatePartition, (WHV_PARTITION_HANDLE* Partition)) \

From 158bb2249ed9b1ed0a376f30a89ecedb2f448552 Mon Sep 17 00:00:00 2001
From: Konstantin Kostiuk <kkostiuk@redhat.com>
Date: Thu, 28 Apr 2022 21:15:25 +0300
Subject: [PATCH 098/935] configure: Add cross prefix for widl tool
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The mingw-w64-tool package in Fedora provides widl tool with a
cross prefix, so adds it automatically for cross builds.

WIDL env can be used to redefine the path to tool.
The same behavior as with windres.

Signed-off-by: Konstantin Kostiuk <kkostiuk@redhat.com>
Message-Id: <20220428181525.300521-1-kkostiuk@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/configure b/configure
index 59c43bea05..8b775492d0 100755
--- a/configure
+++ b/configure
@@ -415,6 +415,7 @@ ranlib="${RANLIB-${cross_prefix}ranlib}"
 nm="${NM-${cross_prefix}nm}"
 smbd="$SMBD"
 strip="${STRIP-${cross_prefix}strip}"
+widl="${WIDL-${cross_prefix}widl}"
 windres="${WINDRES-${cross_prefix}windres}"
 pkg_config_exe="${PKG_CONFIG-${cross_prefix}pkg-config}"
 query_pkg_config() {
@@ -2786,6 +2787,7 @@ if test "$skip_meson" = no; then
     echo "sdl2-config = [$(meson_quote $sdl2_config)]" >> $cross
   fi
   echo "strip = [$(meson_quote $strip)]" >> $cross
+  echo "widl = [$(meson_quote $widl)]" >> $cross
   echo "windres = [$(meson_quote $windres)]" >> $cross
   if test "$cross_compile" = "yes"; then
     cross_arg="--cross-file config-meson.cross"
@@ -2907,6 +2909,7 @@ preserve_env PYTHON
 preserve_env SDL2_CONFIG
 preserve_env SMBD
 preserve_env STRIP
+preserve_env WIDL
 preserve_env WINDRES
 
 printf "exec" >>config.status

From f780a3f4808fa5ba9b822d299ae40a12d23eae6c Mon Sep 17 00:00:00 2001
From: Konstantin Kostiuk <kkostiuk@redhat.com>
Date: Thu, 28 Apr 2022 21:15:41 +0300
Subject: [PATCH 099/935] qga-vss: always build qga-vss.tlb when qga-vss.dll is
 built
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Konstantin Kostiuk <kkostiuk@redhat.com>
Message-Id: <20220428181541.300619-1-kkostiuk@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 qga/vss-win32/meson.build | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/qga/vss-win32/meson.build b/qga/vss-win32/meson.build
index 71c50d0866..26c5dd6e0e 100644
--- a/qga/vss-win32/meson.build
+++ b/qga/vss-win32/meson.build
@@ -23,8 +23,6 @@ qga_vss = shared_module(
   ]
 )
 
-all_qga += qga_vss
-
 if midl.found()
   gen_tlb = custom_target('gen-tlb',
                           input: 'qga-vss.idl',
@@ -36,3 +34,5 @@ else
                           output: 'qga-vss.tlb',
                           command: [widl, '-t', '@INPUT@', '-o', '@OUTPUT@'])
 endif
+
+all_qga += [ qga_vss, gen_tlb ]

From 119fc611eac96d5ad5c407860b29c0fd5d072f6d Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:33:48 +0200
Subject: [PATCH 100/935] meson-buildoptions: add support for string options
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Allow using the buildoptions.json file for more options, namely anything
that is not a boolean or multiple-choice.

The mapping between configure and meson is messy for string options,
so allow configure to use to something other than the name in
meson_options.txt.  This will come in handy anyway for builtin
Meson options such as b_lto or b_coverage.

Tested-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 meson_options.txt             |  2 +-
 scripts/meson-buildoptions.py | 65 ++++++++++++++++++++++++++++++-----
 scripts/meson-buildoptions.sh |  6 ++--
 3 files changed, 60 insertions(+), 13 deletions(-)

diff --git a/meson_options.txt b/meson_options.txt
index af432a4ee6..390701517c 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -11,7 +11,7 @@ option('qemu_firmwarepath', type : 'string', value : '',
 option('smbd', type : 'string', value : '',
        description: 'Path to smbd for slirp networking')
 option('sphinx_build', type : 'string', value : '',
-       description: 'Use specified sphinx-build [$sphinx_build] for building document (default to be empty)')
+       description: 'Use specified sphinx-build for building document')
 option('iasl', type : 'string', value : '',
        description: 'Path to ACPI disassembler')
 option('default_devices', type : 'boolean', value : true,
diff --git a/scripts/meson-buildoptions.py b/scripts/meson-buildoptions.py
index 693be7b966..4af8d6e732 100755
--- a/scripts/meson-buildoptions.py
+++ b/scripts/meson-buildoptions.py
@@ -38,6 +38,11 @@ SKIP_OPTIONS = {
     "trace_file",
 }
 
+OPTION_NAMES = {
+    "malloc": "enable-malloc",
+    "trace_backends": "enable-trace-backends",
+}
+
 BUILTIN_OPTIONS = {
     "strip",
 }
@@ -75,7 +80,7 @@ def help_line(left, opt, indent, long):
     right = f'{opt["description"]}'
     if long:
         value = value_to_help(opt["value"])
-        if value != "auto":
+        if value != "auto" and value != "":
             right += f" [{value}]"
     if "choices" in opt and long:
         choices = "/".join(sorted(opt["choices"]))
@@ -96,6 +101,18 @@ def allow_arg(opt):
     return not (set(opt["choices"]) <= {"auto", "disabled", "enabled"})
 
 
+# Return whether the option (a dictionary) can be used without
+# arguments.  Booleans can only be used without arguments;
+# combos require an argument if they accept neither "enabled"
+# nor "disabled"
+def require_arg(opt):
+    if opt["type"] == "boolean":
+        return False
+    if opt["type"] != "combo":
+        return True
+    return not ({"enabled", "disabled"}.intersection(opt["choices"]))
+
+
 def filter_options(json):
     if ":" in json["name"]:
         return False
@@ -110,20 +127,48 @@ def load_options(json):
     return sorted(json, key=lambda x: x["name"])
 
 
+def cli_option(opt):
+    name = opt["name"]
+    if name in OPTION_NAMES:
+        return OPTION_NAMES[name]
+    return name.replace("_", "-")
+
+
+def cli_help_key(opt):
+    key = cli_option(opt)
+    if require_arg(opt):
+        return key
+    if opt["type"] == "boolean" and opt["value"]:
+        return f"disable-{key}"
+    return f"enable-{key}"
+
+
+def cli_metavar(opt):
+    if opt["type"] == "string":
+        return "VALUE"
+    if opt["type"] == "array":
+        return "CHOICES"
+    return "CHOICE"
+
+
 def print_help(options):
     print("meson_options_help() {")
-    for opt in options:
-        key = opt["name"].replace("_", "-")
+    for opt in sorted(options, key=cli_help_key):
+        key = cli_help_key(opt)
         # The first section includes options that have an arguments,
         # and booleans (i.e., only one of enable/disable makes sense)
-        if opt["type"] == "boolean":
-            left = f"--disable-{key}" if opt["value"] else f"--enable-{key}"
+        if require_arg(opt):
+            metavar = cli_metavar(opt)
+            left = f"--{key}={metavar}"
+            help_line(left, opt, 27, True)
+        elif opt["type"] == "boolean":
+            left = f"--{key}"
             help_line(left, opt, 27, False)
         elif allow_arg(opt):
             if opt["type"] == "combo" and "enabled" in opt["choices"]:
-                left = f"--enable-{key}[=CHOICE]"
+                left = f"--{key}[=CHOICE]"
             else:
-                left = f"--enable-{key}=CHOICE"
+                left = f"--{key}=CHOICE"
             help_line(left, opt, 27, True)
 
     sh_print()
@@ -142,9 +187,11 @@ def print_parse(options):
     print("_meson_option_parse() {")
     print("  case $1 in")
     for opt in options:
-        key = opt["name"].replace("_", "-")
+        key = cli_option(opt)
         name = opt["name"]
-        if opt["type"] == "boolean":
+        if require_arg(opt):
+            print(f'    --{key}=*) quote_sh "-D{name}=$2" ;;')
+        elif opt["type"] == "boolean":
             print(f'    --enable-{key}) printf "%s" -D{name}=true ;;')
             print(f'    --disable-{key}) printf "%s" -D{name}=false ;;')
         else:
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index 21366b2102..2123317654 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -1,5 +1,7 @@
 # This file is generated by meson-buildoptions.py, do not edit!
 meson_options_help() {
+  printf "%s\n" '  --disable-coroutine-pool coroutine freelist (better performance)'
+  printf "%s\n" '  --disable-install-blobs  install provided firmware blobs'
   printf "%s\n" '  --enable-block-drv-whitelist-in-tools'
   printf "%s\n" '                           use block whitelist also in tools instead of only'
   printf "%s\n" '                           QEMU'
@@ -8,7 +10,6 @@ meson_options_help() {
   printf "%s\n" '                           (choices: auto/disabled/enabled/internal/system)'
   printf "%s\n" '  --enable-cfi             Control-Flow Integrity (CFI)'
   printf "%s\n" '  --enable-cfi-debug       Verbose errors in case of CFI violation'
-  printf "%s\n" '  --disable-coroutine-pool coroutine freelist (better performance)'
   printf "%s\n" '  --enable-debug-mutex     mutex debugging support'
   printf "%s\n" '  --enable-debug-stack-usage'
   printf "%s\n" '                           measure coroutine stack usage'
@@ -16,7 +17,6 @@ meson_options_help() {
   printf "%s\n" '                           (choices: auto/disabled/enabled/internal/system)'
   printf "%s\n" '  --enable-fuzzing         build fuzzing targets'
   printf "%s\n" '  --enable-gprof           QEMU profiling with gprof'
-  printf "%s\n" '  --disable-install-blobs  install provided firmware blobs'
   printf "%s\n" '  --enable-malloc=CHOICE   choose memory allocator to use [system] (choices:'
   printf "%s\n" '                           jemalloc/system/tcmalloc)'
   printf "%s\n" '  --enable-module-upgrades try to load modules from alternate paths for'
@@ -29,7 +29,7 @@ meson_options_help() {
   printf "%s\n" '                           (choices: auto/disabled/enabled/internal/system)'
   printf "%s\n" '  --enable-strip           Strip targets on install'
   printf "%s\n" '  --enable-tcg-interpreter TCG with bytecode interpreter (slow)'
-  printf "%s\n" '  --enable-trace-backends=CHOICE'
+  printf "%s\n" '  --enable-trace-backends=CHOICES'
   printf "%s\n" '                           Set available tracing backends [log] (choices:'
   printf "%s\n" '                           dtrace/ftrace/log/nop/simple/syslog/ust)'
   printf "%s\n" ''

From 14efd8d3b5fee7027f0ee072aea6620e0655774f Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:33:47 +0200
Subject: [PATCH 101/935] meson, configure: move Xen detection to meson

This is quite a complicated check.  I moved all the test programs to
a single file in scripts/, picking the right program with #if and a -D
flag in meson.build's cc.links() invocation.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure            | 322 +------------------------------------------
 meson.build          | 112 ++++++++++++---
 scripts/xen-detect.c | 203 +++++++++++++++++++++++++++
 3 files changed, 296 insertions(+), 341 deletions(-)
 create mode 100644 scripts/xen-detect.c

diff --git a/configure b/configure
index 8b775492d0..519982800e 100755
--- a/configure
+++ b/configure
@@ -292,7 +292,6 @@ EXTRA_CXXFLAGS=""
 EXTRA_OBJCFLAGS=""
 EXTRA_LDFLAGS=""
 
-xen_ctrl_version="$default_feature"
 vhost_kernel="$default_feature"
 vhost_net="$default_feature"
 vhost_crypto="$default_feature"
@@ -346,9 +345,6 @@ default_devices="true"
 tcg="enabled"
 cfi="false"
 
-# 4. Detection partly done in configure
-xen=${default_feature:+disabled}
-
 # parse CC options second
 for opt do
   optarg=$(expr "x$opt" : 'x[^=]*=\(.*\)')
@@ -867,10 +863,6 @@ for opt do
   ;;
   --enable-slirp=*) slirp="$optarg"
   ;;
-  --disable-xen) xen="disabled"
-  ;;
-  --enable-xen) xen="enabled"
-  ;;
   --disable-tcg) tcg="disabled"
                  plugins="no"
   ;;
@@ -1698,312 +1690,6 @@ if ! has "$pkg_config_exe"; then
   error_exit "pkg-config binary '$pkg_config_exe' not found"
 fi
 
-##########################################
-# xen probe
-
-if test "$xen" != "disabled" ; then
-  # Check whether Xen library path is specified via --extra-ldflags to avoid
-  # overriding this setting with pkg-config output. If not, try pkg-config
-  # to obtain all needed flags.
-
-  if ! echo $EXTRA_LDFLAGS | grep tools/libxc > /dev/null && \
-     $pkg_config --exists xencontrol ; then
-    xen_ctrl_version="$(printf '%d%02d%02d' \
-      $($pkg_config --modversion xencontrol | sed 's/\./ /g') )"
-    xen=enabled
-    xen_pc="xencontrol xenstore xenforeignmemory xengnttab"
-    xen_pc="$xen_pc xenevtchn xendevicemodel"
-    if $pkg_config --exists xentoolcore; then
-      xen_pc="$xen_pc xentoolcore"
-    fi
-    xen_cflags="$($pkg_config --cflags $xen_pc)"
-    xen_libs="$($pkg_config --libs $xen_pc)"
-  else
-
-    xen_libs="-lxenstore -lxenctrl"
-    xen_stable_libs="-lxenforeignmemory -lxengnttab -lxenevtchn"
-
-    # First we test whether Xen headers and libraries are available.
-    # If no, we are done and there is no Xen support.
-    # If yes, more tests are run to detect the Xen version.
-
-    # Xen (any)
-    cat > $TMPC <<EOF
-#include <xenctrl.h>
-int main(void) {
-  return 0;
-}
-EOF
-    if ! compile_prog "" "$xen_libs" ; then
-      # Xen not found
-      if test "$xen" = "enabled" ; then
-        feature_not_found "xen" "Install xen devel"
-      fi
-      xen=disabled
-
-    # Xen unstable
-    elif
-        cat > $TMPC <<EOF &&
-#undef XC_WANT_COMPAT_DEVICEMODEL_API
-#define __XEN_TOOLS__
-#include <xendevicemodel.h>
-#include <xenforeignmemory.h>
-int main(void) {
-  xendevicemodel_handle *xd;
-  xenforeignmemory_handle *xfmem;
-
-  xd = xendevicemodel_open(0, 0);
-  xendevicemodel_pin_memory_cacheattr(xd, 0, 0, 0, 0);
-
-  xfmem = xenforeignmemory_open(0, 0);
-  xenforeignmemory_map_resource(xfmem, 0, 0, 0, 0, 0, NULL, 0, 0);
-
-  return 0;
-}
-EOF
-        compile_prog "" "$xen_libs -lxendevicemodel $xen_stable_libs -lxentoolcore"
-      then
-      xen_stable_libs="-lxendevicemodel $xen_stable_libs -lxentoolcore"
-      xen_ctrl_version=41100
-      xen=enabled
-    elif
-        cat > $TMPC <<EOF &&
-#undef XC_WANT_COMPAT_MAP_FOREIGN_API
-#include <xenforeignmemory.h>
-#include <xentoolcore.h>
-int main(void) {
-  xenforeignmemory_handle *xfmem;
-
-  xfmem = xenforeignmemory_open(0, 0);
-  xenforeignmemory_map2(xfmem, 0, 0, 0, 0, 0, 0, 0);
-  xentoolcore_restrict_all(0);
-
-  return 0;
-}
-EOF
-        compile_prog "" "$xen_libs -lxendevicemodel $xen_stable_libs -lxentoolcore"
-      then
-      xen_stable_libs="-lxendevicemodel $xen_stable_libs -lxentoolcore"
-      xen_ctrl_version=41000
-      xen=enabled
-    elif
-        cat > $TMPC <<EOF &&
-#undef XC_WANT_COMPAT_DEVICEMODEL_API
-#define __XEN_TOOLS__
-#include <xendevicemodel.h>
-int main(void) {
-  xendevicemodel_handle *xd;
-
-  xd = xendevicemodel_open(0, 0);
-  xendevicemodel_close(xd);
-
-  return 0;
-}
-EOF
-        compile_prog "" "$xen_libs -lxendevicemodel $xen_stable_libs"
-      then
-      xen_stable_libs="-lxendevicemodel $xen_stable_libs"
-      xen_ctrl_version=40900
-      xen=enabled
-    elif
-        cat > $TMPC <<EOF &&
-/*
- * If we have stable libs the we don't want the libxc compat
- * layers, regardless of what CFLAGS we may have been given.
- *
- * Also, check if xengnttab_grant_copy_segment_t is defined and
- * grant copy operation is implemented.
- */
-#undef XC_WANT_COMPAT_EVTCHN_API
-#undef XC_WANT_COMPAT_GNTTAB_API
-#undef XC_WANT_COMPAT_MAP_FOREIGN_API
-#include <xenctrl.h>
-#include <xenstore.h>
-#include <xenevtchn.h>
-#include <xengnttab.h>
-#include <xenforeignmemory.h>
-#include <stdint.h>
-#include <xen/hvm/hvm_info_table.h>
-#if !defined(HVM_MAX_VCPUS)
-# error HVM_MAX_VCPUS not defined
-#endif
-int main(void) {
-  xc_interface *xc = NULL;
-  xenforeignmemory_handle *xfmem;
-  xenevtchn_handle *xe;
-  xengnttab_handle *xg;
-  xengnttab_grant_copy_segment_t* seg = NULL;
-
-  xs_daemon_open();
-
-  xc = xc_interface_open(0, 0, 0);
-  xc_hvm_set_mem_type(0, 0, HVMMEM_ram_ro, 0, 0);
-  xc_domain_add_to_physmap(0, 0, XENMAPSPACE_gmfn, 0, 0);
-  xc_hvm_inject_msi(xc, 0, 0xf0000000, 0x00000000);
-  xc_hvm_create_ioreq_server(xc, 0, HVM_IOREQSRV_BUFIOREQ_ATOMIC, NULL);
-
-  xfmem = xenforeignmemory_open(0, 0);
-  xenforeignmemory_map(xfmem, 0, 0, 0, 0, 0);
-
-  xe = xenevtchn_open(0, 0);
-  xenevtchn_fd(xe);
-
-  xg = xengnttab_open(0, 0);
-  xengnttab_grant_copy(xg, 0, seg);
-
-  return 0;
-}
-EOF
-        compile_prog "" "$xen_libs $xen_stable_libs"
-      then
-      xen_ctrl_version=40800
-      xen=enabled
-    elif
-        cat > $TMPC <<EOF &&
-/*
- * If we have stable libs the we don't want the libxc compat
- * layers, regardless of what CFLAGS we may have been given.
- */
-#undef XC_WANT_COMPAT_EVTCHN_API
-#undef XC_WANT_COMPAT_GNTTAB_API
-#undef XC_WANT_COMPAT_MAP_FOREIGN_API
-#include <xenctrl.h>
-#include <xenstore.h>
-#include <xenevtchn.h>
-#include <xengnttab.h>
-#include <xenforeignmemory.h>
-#include <stdint.h>
-#include <xen/hvm/hvm_info_table.h>
-#if !defined(HVM_MAX_VCPUS)
-# error HVM_MAX_VCPUS not defined
-#endif
-int main(void) {
-  xc_interface *xc = NULL;
-  xenforeignmemory_handle *xfmem;
-  xenevtchn_handle *xe;
-  xengnttab_handle *xg;
-
-  xs_daemon_open();
-
-  xc = xc_interface_open(0, 0, 0);
-  xc_hvm_set_mem_type(0, 0, HVMMEM_ram_ro, 0, 0);
-  xc_domain_add_to_physmap(0, 0, XENMAPSPACE_gmfn, 0, 0);
-  xc_hvm_inject_msi(xc, 0, 0xf0000000, 0x00000000);
-  xc_hvm_create_ioreq_server(xc, 0, HVM_IOREQSRV_BUFIOREQ_ATOMIC, NULL);
-
-  xfmem = xenforeignmemory_open(0, 0);
-  xenforeignmemory_map(xfmem, 0, 0, 0, 0, 0);
-
-  xe = xenevtchn_open(0, 0);
-  xenevtchn_fd(xe);
-
-  xg = xengnttab_open(0, 0);
-  xengnttab_map_grant_ref(xg, 0, 0, 0);
-
-  return 0;
-}
-EOF
-        compile_prog "" "$xen_libs $xen_stable_libs"
-      then
-      xen_ctrl_version=40701
-      xen=enabled
-
-    # Xen 4.6
-    elif
-        cat > $TMPC <<EOF &&
-#include <xenctrl.h>
-#include <xenstore.h>
-#include <stdint.h>
-#include <xen/hvm/hvm_info_table.h>
-#if !defined(HVM_MAX_VCPUS)
-# error HVM_MAX_VCPUS not defined
-#endif
-int main(void) {
-  xc_interface *xc;
-  xs_daemon_open();
-  xc = xc_interface_open(0, 0, 0);
-  xc_hvm_set_mem_type(0, 0, HVMMEM_ram_ro, 0, 0);
-  xc_gnttab_open(NULL, 0);
-  xc_domain_add_to_physmap(0, 0, XENMAPSPACE_gmfn, 0, 0);
-  xc_hvm_inject_msi(xc, 0, 0xf0000000, 0x00000000);
-  xc_hvm_create_ioreq_server(xc, 0, HVM_IOREQSRV_BUFIOREQ_ATOMIC, NULL);
-  xc_reserved_device_memory_map(xc, 0, 0, 0, 0, NULL, 0);
-  return 0;
-}
-EOF
-        compile_prog "" "$xen_libs"
-      then
-      xen_ctrl_version=40600
-      xen=enabled
-
-    # Xen 4.5
-    elif
-        cat > $TMPC <<EOF &&
-#include <xenctrl.h>
-#include <xenstore.h>
-#include <stdint.h>
-#include <xen/hvm/hvm_info_table.h>
-#if !defined(HVM_MAX_VCPUS)
-# error HVM_MAX_VCPUS not defined
-#endif
-int main(void) {
-  xc_interface *xc;
-  xs_daemon_open();
-  xc = xc_interface_open(0, 0, 0);
-  xc_hvm_set_mem_type(0, 0, HVMMEM_ram_ro, 0, 0);
-  xc_gnttab_open(NULL, 0);
-  xc_domain_add_to_physmap(0, 0, XENMAPSPACE_gmfn, 0, 0);
-  xc_hvm_inject_msi(xc, 0, 0xf0000000, 0x00000000);
-  xc_hvm_create_ioreq_server(xc, 0, 0, NULL);
-  return 0;
-}
-EOF
-        compile_prog "" "$xen_libs"
-      then
-      xen_ctrl_version=40500
-      xen=enabled
-
-    elif
-        cat > $TMPC <<EOF &&
-#include <xenctrl.h>
-#include <xenstore.h>
-#include <stdint.h>
-#include <xen/hvm/hvm_info_table.h>
-#if !defined(HVM_MAX_VCPUS)
-# error HVM_MAX_VCPUS not defined
-#endif
-int main(void) {
-  xc_interface *xc;
-  xs_daemon_open();
-  xc = xc_interface_open(0, 0, 0);
-  xc_hvm_set_mem_type(0, 0, HVMMEM_ram_ro, 0, 0);
-  xc_gnttab_open(NULL, 0);
-  xc_domain_add_to_physmap(0, 0, XENMAPSPACE_gmfn, 0, 0);
-  xc_hvm_inject_msi(xc, 0, 0xf0000000, 0x00000000);
-  return 0;
-}
-EOF
-        compile_prog "" "$xen_libs"
-      then
-      xen_ctrl_version=40200
-      xen=enabled
-
-    else
-      if test "$xen" = "enabled" ; then
-        feature_not_found "xen (unsupported version)" \
-                          "Install a supported xen (xen 4.2 or newer)"
-      fi
-      xen=disabled
-    fi
-
-    if test "$xen" = enabled; then
-      if test $xen_ctrl_version -ge 40701  ; then
-        xen_libs="$xen_libs $xen_stable_libs "
-      fi
-    fi
-  fi
-fi
-
 ##########################################
 # glib support probe
 
@@ -2555,12 +2241,6 @@ if test "$modules" = "yes"; then
 fi
 echo "CONFIG_TLS_PRIORITY=\"$tls_priority\"" >> $config_host_mak
 
-if test "$xen" = "enabled" ; then
-  echo "CONFIG_XEN_BACKEND=y" >> $config_host_mak
-  echo "CONFIG_XEN_CTRL_INTERFACE_VERSION=$xen_ctrl_version" >> $config_host_mak
-  echo "XEN_CFLAGS=$xen_cflags" >> $config_host_mak
-  echo "XEN_LIBS=$xen_libs" >> $config_host_mak
-fi
 if test "$vhost_scsi" = "yes" ; then
   echo "CONFIG_VHOST_SCSI=y" >> $config_host_mak
 fi
@@ -2839,7 +2519,7 @@ if test "$skip_meson" = no; then
         -Dwerror=$(if test "$werror" = yes; then echo true; else echo false; fi) \
         -Db_pie=$(if test "$pie" = yes; then echo true; else echo false; fi) \
         -Db_coverage=$(if test "$gcov" = yes; then echo true; else echo false; fi) \
-        -Db_lto=$lto -Dcfi=$cfi -Dtcg=$tcg -Dxen=$xen \
+        -Db_lto=$lto -Dcfi=$cfi -Dtcg=$tcg \
         -Dcapstone=$capstone -Dfdt=$fdt -Dslirp=$slirp \
         $(test -n "${LIB_FUZZING_ENGINE+xxx}" && echo "-Dfuzzing_engine=$LIB_FUZZING_ENGINE") \
         $(if test "$default_feature" = no; then echo "-Dauto_features=disabled"; fi) \
diff --git a/meson.build b/meson.build
index c26aa442d4..2d54513b9f 100644
--- a/meson.build
+++ b/meson.build
@@ -348,12 +348,6 @@ accelerators = []
 if get_option('kvm').allowed() and targetos == 'linux'
   accelerators += 'CONFIG_KVM'
 endif
-if get_option('xen').allowed() and 'CONFIG_XEN_BACKEND' in config_host
-  accelerators += 'CONFIG_XEN'
-  have_xen_pci_passthrough = get_option('xen_pci_passthrough').allowed() and targetos == 'linux'
-else
-  have_xen_pci_passthrough = false
-endif
 if get_option('whpx').allowed() and targetos == 'windows'
   if get_option('whpx').enabled() and host_machine.cpu() != 'x86_64'
     error('WHPX requires 64-bit host')
@@ -425,13 +419,6 @@ endif
 if 'CONFIG_WHPX' not in accelerators and get_option('whpx').enabled()
   error('WHPX not available on this platform')
 endif
-if not have_xen_pci_passthrough and get_option('xen_pci_passthrough').enabled()
-  if 'CONFIG_XEN' in accelerators
-    error('Xen PCI passthrough not available on this platform')
-  else
-    error('Xen PCI passthrough requested but Xen not enabled')
-  endif
-endif
 
 ################
 # Dependencies #
@@ -1257,10 +1244,86 @@ if not get_option('rdma').auto() or have_system
 endif
 
 xen = not_found
-if 'CONFIG_XEN_BACKEND' in config_host
-  xen = declare_dependency(compile_args: config_host['XEN_CFLAGS'].split(),
-                           link_args: config_host['XEN_LIBS'].split())
+if get_option('xen').enabled() or (get_option('xen').auto() and have_system)
+  xencontrol = dependency('xencontrol', required: false,
+                          method: 'pkg-config', kwargs: static_kwargs)
+  if xencontrol.found()
+    xen_pc = declare_dependency(version: xencontrol.version(),
+      dependencies: [
+        xencontrol,
+        # disabler: true makes xen_pc.found() return false if any is not found
+        dependency('xenstore', required: false,
+                   method: 'pkg-config', kwargs: static_kwargs,
+                   disabler: true),
+        dependency('xenforeignmemory', required: false,
+                   method: 'pkg-config', kwargs: static_kwargs,
+                   disabler: true),
+        dependency('xengnttab', required: false,
+                   method: 'pkg-config', kwargs: static_kwargs,
+                   disabler: true),
+        dependency('xenevtchn', required: false,
+                   method: 'pkg-config', kwargs: static_kwargs,
+                   disabler: true),
+        dependency('xendevicemodel', required: false,
+                   method: 'pkg-config', kwargs: static_kwargs,
+                   disabler: true),
+        # optional, no "disabler: true"
+        dependency('xentoolcore', required: false,
+                   method: 'pkg-config', kwargs: static_kwargs)])
+    if xen_pc.found()
+      xen = xen_pc
+    endif
+  endif
+  if not xen.found()
+    xen_tests = [ '4.11.0', '4.10.0', '4.9.0', '4.8.0', '4.7.1', '4.6.0', '4.5.0', '4.2.0' ]
+    xen_libs = {
+      '4.11.0': [ 'xenstore', 'xenctrl', 'xendevicemodel', 'xenforeignmemory', 'xengnttab', 'xenevtchn', 'xentoolcore' ],
+      '4.10.0': [ 'xenstore', 'xenctrl', 'xendevicemodel', 'xenforeignmemory', 'xengnttab', 'xenevtchn', 'xentoolcore' ],
+      '4.9.0': [ 'xenstore', 'xenctrl', 'xendevicemodel', 'xenforeignmemory', 'xengnttab', 'xenevtchn' ],
+      '4.8.0': [ 'xenstore', 'xenctrl', 'xenforeignmemory', 'xengnttab', 'xenevtchn' ],
+      '4.7.1': [ 'xenstore', 'xenctrl', 'xenforeignmemory', 'xengnttab', 'xenevtchn' ],
+      '4.6.0': [ 'xenstore', 'xenctrl' ],
+      '4.5.0': [ 'xenstore', 'xenctrl' ],
+      '4.2.0': [ 'xenstore', 'xenctrl' ],
+    }
+    xen_deps = {}
+    foreach ver: xen_tests
+      # cache the various library tests to avoid polluting the logs
+      xen_test_deps = []
+      foreach l: xen_libs[ver]
+        if l not in xen_deps
+          xen_deps += { l: cc.find_library(l, required: false) }
+        endif
+        xen_test_deps += xen_deps[l]
+      endforeach
+
+      # Use -D to pick just one of the test programs in scripts/xen-detect.c
+      xen_version = ver.split('.')
+      xen_ctrl_version = xen_version[0] + \
+        ('0' + xen_version[1]).substring(-2) + \
+        ('0' + xen_version[2]).substring(-2)
+      if cc.links(files('scripts/xen-detect.c'),
+                  args: '-DCONFIG_XEN_CTRL_INTERFACE_VERSION=' + xen_ctrl_version,
+                  dependencies: xen_test_deps)
+        xen = declare_dependency(version: ver, dependencies: xen_test_deps)
+        break
+      endif
+    endforeach
+  endif
+  if xen.found()
+    accelerators += 'CONFIG_XEN'
+  elif get_option('xen').enabled()
+    error('could not compile and link Xen test program')
+  endif
 endif
+have_xen_pci_passthrough = get_option('xen_pci_passthrough') \
+  .require(xen.found(),
+           error_message: 'Xen PCI passthrough requested but Xen not enabled') \
+  .require(targetos == 'linux',
+           error_message: 'Xen PCI passthrough not available on this platform') \
+  .allowed()
+
+
 cacard = not_found
 if not get_option('smartcard').auto() or have_system
   cacard = dependency('libcacard', required: get_option('smartcard'),
@@ -1634,6 +1697,15 @@ config_host_data.set('CONFIG_X11', x11.found())
 config_host_data.set('CONFIG_DBUS_DISPLAY', dbus_display)
 config_host_data.set('CONFIG_CFI', get_option('cfi'))
 config_host_data.set('CONFIG_SELINUX', selinux.found())
+config_host_data.set('CONFIG_XEN_BACKEND', xen.found())
+if xen.found()
+  # protect from xen.version() having less than three components
+  xen_version = xen.version().split('.') + ['0', '0']
+  xen_ctrl_version = xen_version[0] + \
+    ('0' + xen_version[1]).substring(-2) + \
+    ('0' + xen_version[2]).substring(-2)
+  config_host_data.set('CONFIG_XEN_CTRL_INTERFACE_VERSION', xen_ctrl_version)
+endif
 config_host_data.set('QEMU_VERSION', '"@0@"'.format(meson.project_version()))
 config_host_data.set('QEMU_VERSION_MAJOR', meson.project_version().split('.')[0])
 config_host_data.set('QEMU_VERSION_MINOR', meson.project_version().split('.')[1])
@@ -2359,7 +2431,7 @@ config_all += config_all_devices
 config_all += config_host
 config_all += config_all_disas
 config_all += {
-  'CONFIG_XEN': config_host.has_key('CONFIG_XEN_BACKEND'),
+  'CONFIG_XEN': xen.found(),
   'CONFIG_SOFTMMU': have_system,
   'CONFIG_USER_ONLY': have_user,
   'CONFIG_ALL': true,
@@ -3690,9 +3762,9 @@ if have_system
   summary_info += {'HVF support':       config_all.has_key('CONFIG_HVF')}
   summary_info += {'WHPX support':      config_all.has_key('CONFIG_WHPX')}
   summary_info += {'NVMM support':      config_all.has_key('CONFIG_NVMM')}
-  summary_info += {'Xen support':       config_host.has_key('CONFIG_XEN_BACKEND')}
-  if config_host.has_key('CONFIG_XEN_BACKEND')
-    summary_info += {'xen ctrl version':  config_host['CONFIG_XEN_CTRL_INTERFACE_VERSION']}
+  summary_info += {'Xen support':       xen.found()}
+  if xen.found()
+    summary_info += {'xen ctrl version':  xen.version()}
   endif
 endif
 summary_info += {'TCG support':       config_all.has_key('CONFIG_TCG')}
diff --git a/scripts/xen-detect.c b/scripts/xen-detect.c
new file mode 100644
index 0000000000..85e8206490
--- /dev/null
+++ b/scripts/xen-detect.c
@@ -0,0 +1,203 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+/* Test programs for various Xen versions that QEMU supports.  */
+#if CONFIG_XEN_CTRL_INTERFACE_VERSION == 41100
+  #undef XC_WANT_COMPAT_DEVICEMODEL_API
+  #define __XEN_TOOLS__
+  #include <xendevicemodel.h>
+  #include <xenforeignmemory.h>
+  int main(void) {
+    xendevicemodel_handle *xd;
+    xenforeignmemory_handle *xfmem;
+
+    xd = xendevicemodel_open(0, 0);
+    xendevicemodel_pin_memory_cacheattr(xd, 0, 0, 0, 0);
+
+    xfmem = xenforeignmemory_open(0, 0);
+    xenforeignmemory_map_resource(xfmem, 0, 0, 0, 0, 0, NULL, 0, 0);
+
+    return 0;
+  }
+
+#elif CONFIG_XEN_CTRL_INTERFACE_VERSION == 41000
+  #undef XC_WANT_COMPAT_MAP_FOREIGN_API
+  #include <xenforeignmemory.h>
+  #include <xentoolcore.h>
+  int main(void) {
+    xenforeignmemory_handle *xfmem;
+
+    xfmem = xenforeignmemory_open(0, 0);
+    xenforeignmemory_map2(xfmem, 0, 0, 0, 0, 0, 0, 0);
+    xentoolcore_restrict_all(0);
+
+    return 0;
+  }
+
+#elif CONFIG_XEN_CTRL_INTERFACE_VERSION == 40900
+  #undef XC_WANT_COMPAT_DEVICEMODEL_API
+  #define __XEN_TOOLS__
+  #include <xendevicemodel.h>
+  int main(void) {
+    xendevicemodel_handle *xd;
+
+    xd = xendevicemodel_open(0, 0);
+    xendevicemodel_close(xd);
+
+    return 0;
+  }
+
+#elif CONFIG_XEN_CTRL_INTERFACE_VERSION == 40800
+  /*
+   * If we have stable libs the we don't want the libxc compat
+   * layers, regardless of what CFLAGS we may have been given.
+   *
+   * Also, check if xengnttab_grant_copy_segment_t is defined and
+   * grant copy operation is implemented.
+   */
+  #undef XC_WANT_COMPAT_EVTCHN_API
+  #undef XC_WANT_COMPAT_GNTTAB_API
+  #undef XC_WANT_COMPAT_MAP_FOREIGN_API
+  #include <xenctrl.h>
+  #include <xenstore.h>
+  #include <xenevtchn.h>
+  #include <xengnttab.h>
+  #include <xenforeignmemory.h>
+  #include <stdint.h>
+  #include <xen/hvm/hvm_info_table.h>
+  #if !defined(HVM_MAX_VCPUS)
+  # error HVM_MAX_VCPUS not defined
+  #endif
+  int main(void) {
+    xc_interface *xc = NULL;
+    xenforeignmemory_handle *xfmem;
+    xenevtchn_handle *xe;
+    xengnttab_handle *xg;
+    xengnttab_grant_copy_segment_t* seg = NULL;
+
+    xs_daemon_open();
+
+    xc = xc_interface_open(0, 0, 0);
+    xc_hvm_set_mem_type(0, 0, HVMMEM_ram_ro, 0, 0);
+    xc_domain_add_to_physmap(0, 0, XENMAPSPACE_gmfn, 0, 0);
+    xc_hvm_inject_msi(xc, 0, 0xf0000000, 0x00000000);
+    xc_hvm_create_ioreq_server(xc, 0, HVM_IOREQSRV_BUFIOREQ_ATOMIC, NULL);
+
+    xfmem = xenforeignmemory_open(0, 0);
+    xenforeignmemory_map(xfmem, 0, 0, 0, 0, 0);
+
+    xe = xenevtchn_open(0, 0);
+    xenevtchn_fd(xe);
+
+    xg = xengnttab_open(0, 0);
+    xengnttab_grant_copy(xg, 0, seg);
+
+    return 0;
+  }
+
+#elif CONFIG_XEN_CTRL_INTERFACE_VERSION == 40701
+  /*
+   * If we have stable libs the we don't want the libxc compat
+   * layers, regardless of what CFLAGS we may have been given.
+   */
+  #undef XC_WANT_COMPAT_EVTCHN_API
+  #undef XC_WANT_COMPAT_GNTTAB_API
+  #undef XC_WANT_COMPAT_MAP_FOREIGN_API
+  #include <xenctrl.h>
+  #include <xenstore.h>
+  #include <xenevtchn.h>
+  #include <xengnttab.h>
+  #include <xenforeignmemory.h>
+  #include <stdint.h>
+  #include <xen/hvm/hvm_info_table.h>
+  #if !defined(HVM_MAX_VCPUS)
+  # error HVM_MAX_VCPUS not defined
+  #endif
+  int main(void) {
+    xc_interface *xc = NULL;
+    xenforeignmemory_handle *xfmem;
+    xenevtchn_handle *xe;
+    xengnttab_handle *xg;
+
+    xs_daemon_open();
+
+    xc = xc_interface_open(0, 0, 0);
+    xc_hvm_set_mem_type(0, 0, HVMMEM_ram_ro, 0, 0);
+    xc_domain_add_to_physmap(0, 0, XENMAPSPACE_gmfn, 0, 0);
+    xc_hvm_inject_msi(xc, 0, 0xf0000000, 0x00000000);
+    xc_hvm_create_ioreq_server(xc, 0, HVM_IOREQSRV_BUFIOREQ_ATOMIC, NULL);
+
+    xfmem = xenforeignmemory_open(0, 0);
+    xenforeignmemory_map(xfmem, 0, 0, 0, 0, 0);
+
+    xe = xenevtchn_open(0, 0);
+    xenevtchn_fd(xe);
+
+    xg = xengnttab_open(0, 0);
+    xengnttab_map_grant_ref(xg, 0, 0, 0);
+
+    return 0;
+  }
+
+#elif CONFIG_XEN_CTRL_INTERFACE_VERSION == 40600
+  #include <xenctrl.h>
+  #include <xenstore.h>
+  #include <stdint.h>
+  #include <xen/hvm/hvm_info_table.h>
+  #if !defined(HVM_MAX_VCPUS)
+  # error HVM_MAX_VCPUS not defined
+  #endif
+  int main(void) {
+    xc_interface *xc;
+    xs_daemon_open();
+    xc = xc_interface_open(0, 0, 0);
+    xc_hvm_set_mem_type(0, 0, HVMMEM_ram_ro, 0, 0);
+    xc_gnttab_open(NULL, 0);
+    xc_domain_add_to_physmap(0, 0, XENMAPSPACE_gmfn, 0, 0);
+    xc_hvm_inject_msi(xc, 0, 0xf0000000, 0x00000000);
+    xc_hvm_create_ioreq_server(xc, 0, HVM_IOREQSRV_BUFIOREQ_ATOMIC, NULL);
+    xc_reserved_device_memory_map(xc, 0, 0, 0, 0, NULL, 0);
+    return 0;
+  }
+
+#elif CONFIG_XEN_CTRL_INTERFACE_VERSION == 40500
+  #include <xenctrl.h>
+  #include <xenstore.h>
+  #include <stdint.h>
+  #include <xen/hvm/hvm_info_table.h>
+  #if !defined(HVM_MAX_VCPUS)
+  # error HVM_MAX_VCPUS not defined
+  #endif
+  int main(void) {
+    xc_interface *xc;
+    xs_daemon_open();
+    xc = xc_interface_open(0, 0, 0);
+    xc_hvm_set_mem_type(0, 0, HVMMEM_ram_ro, 0, 0);
+    xc_gnttab_open(NULL, 0);
+    xc_domain_add_to_physmap(0, 0, XENMAPSPACE_gmfn, 0, 0);
+    xc_hvm_inject_msi(xc, 0, 0xf0000000, 0x00000000);
+    xc_hvm_create_ioreq_server(xc, 0, 0, NULL);
+    return 0;
+  }
+
+#elif CONFIG_XEN_CTRL_INTERFACE_VERSION == 40200
+  #include <xenctrl.h>
+  #include <xenstore.h>
+  #include <stdint.h>
+  #include <xen/hvm/hvm_info_table.h>
+  #if !defined(HVM_MAX_VCPUS)
+  # error HVM_MAX_VCPUS not defined
+  #endif
+  int main(void) {
+    xc_interface *xc;
+    xs_daemon_open();
+    xc = xc_interface_open(0, 0, 0);
+    xc_hvm_set_mem_type(0, 0, HVMMEM_ram_ro, 0, 0);
+    xc_gnttab_open(NULL, 0);
+    xc_domain_add_to_physmap(0, 0, XENMAPSPACE_gmfn, 0, 0);
+    xc_hvm_inject_msi(xc, 0, 0xf0000000, 0x00000000);
+    return 0;
+  }
+
+#else
+#error invalid CONFIG_XEN_CTRL_INTERFACE_VERSION
+#endif

From 40c909f534e3f3cd2bcb797b95f96d52e6dd5cdf Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:33:49 +0200
Subject: [PATCH 102/935] configure, meson: move iasl detection to meson
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure   |  2 +-
 meson.build | 16 +++++++++-------
 2 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/configure b/configure
index 519982800e..b146763fbd 100755
--- a/configure
+++ b/configure
@@ -2508,7 +2508,7 @@ if test "$skip_meson" = no; then
         -Daudio_drv_list=$audio_drv_list \
         -Ddefault_devices=$default_devices \
         -Ddocdir="$docdir" \
-        -Diasl="$($iasl -h >/dev/null 2>&1 && printf %s "$iasl")" \
+        -Diasl="$iasl" \
         -Dqemu_firmwarepath="$firmwarepath" \
         -Dqemu_suffix="$qemu_suffix" \
         -Dsmbd="$smbd" \
diff --git a/meson.build b/meson.build
index 2d54513b9f..448a8dd2a4 100644
--- a/meson.build
+++ b/meson.build
@@ -167,6 +167,12 @@ if 'dtrace' in get_option('trace_backends')
   endif
 endif
 
+if get_option('iasl') == ''
+  iasl = find_program('iasl', required: false)
+else
+  iasl = find_program(get_option('iasl'), required: true)
+endif
+
 ##################
 # Compiler flags #
 ##################
@@ -1584,8 +1590,8 @@ foreach k : get_option('trace_backends')
   config_host_data.set('CONFIG_TRACE_' + k.to_upper(), true)
 endforeach
 config_host_data.set_quoted('CONFIG_TRACE_FILE', get_option('trace_file'))
-if get_option('iasl') != ''
-  config_host_data.set_quoted('CONFIG_IASL', get_option('iasl'))
+if iasl.found()
+  config_host_data.set_quoted('CONFIG_IASL', iasl.full_path())
 endif
 config_host_data.set_quoted('CONFIG_BINDIR', get_option('prefix') / get_option('bindir'))
 config_host_data.set_quoted('CONFIG_PREFIX', get_option('prefix'))
@@ -3631,11 +3637,7 @@ summary_info += {'sphinx-build':      sphinx_build}
 if config_host.has_key('HAVE_GDB_BIN')
   summary_info += {'gdb':             config_host['HAVE_GDB_BIN']}
 endif
-if get_option('iasl') != ''
-  summary_info += {'iasl':            get_option('iasl')}
-else
-  summary_info += {'iasl':            false}
-endif
+summary_info += {'iasl':              iasl}
 summary_info += {'genisoimage':       config_host['GENISOIMAGE']}
 if targetos == 'windows' and have_ga
   summary_info += {'wixl':            wixl}

From d2147e04f95f24d0535bba600b95be2676558367 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:33:50 +0200
Subject: [PATCH 103/935] configure: move Windows flags detection to meson

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure   | 20 --------------------
 meson.build |  8 ++++++++
 2 files changed, 8 insertions(+), 20 deletions(-)

diff --git a/configure b/configure
index b146763fbd..c54e10ea26 100755
--- a/configure
+++ b/configure
@@ -224,10 +224,6 @@ glob() {
     eval test -z '"${1#'"$2"'}"'
 }
 
-ld_has() {
-    $ld --help 2>/dev/null | grep ".$1" >/dev/null 2>&1
-}
-
 if printf %s\\n "$source_path" "$PWD" | grep -q "[[:space:]:]";
 then
   error_exit "main directory cannot contain spaces nor colons"
@@ -2089,22 +2085,6 @@ if test "$solaris" = "no" && test "$tsan" = "no"; then
     fi
 fi
 
-# Use ASLR, no-SEH and DEP if available
-if test "$mingw32" = "yes" ; then
-    flags="--no-seh --nxcompat"
-
-    # Disable ASLR for debug builds to allow debugging with gdb
-    if test "$debug" = "no" ; then
-        flags="--dynamicbase $flags"
-    fi
-
-    for flag in $flags; do
-        if ld_has $flag ; then
-            QEMU_LDFLAGS="-Wl,$flag $QEMU_LDFLAGS"
-        fi
-    done
-fi
-
 # Guest agent Windows MSI package
 
 if test "$QEMU_GA_MANUFACTURER" = ""; then
diff --git a/meson.build b/meson.build
index 448a8dd2a4..6606c9790f 100644
--- a/meson.build
+++ b/meson.build
@@ -182,6 +182,14 @@ qemu_cxxflags = config_host['QEMU_CXXFLAGS'].split()
 qemu_objcflags = config_host['QEMU_OBJCFLAGS'].split()
 qemu_ldflags = config_host['QEMU_LDFLAGS'].split()
 
+if targetos == 'windows'
+  qemu_ldflags += cc.get_supported_link_arguments('-Wl,--no-seh', '-Wl,--nxcompat')
+  # Disable ASLR for debug builds to allow debugging with gdb
+  if get_option('optimization') == '0'
+    qemu_ldflags += cc.get_supported_link_arguments('-Wl,--dynamicbase')
+  endif
+endif
+
 if get_option('gprof')
   qemu_cflags += ['-p']
   qemu_cxxflags += ['-p']

From 4fda6011d00619e9a07019d40c92d0de31d47939 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:33:51 +0200
Subject: [PATCH 104/935] configure: switch string options to automatic parsing
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure                     | 24 +-----------------------
 scripts/meson-buildoptions.py |  5 +----
 scripts/meson-buildoptions.sh |  9 +++++++++
 3 files changed, 11 insertions(+), 27 deletions(-)

diff --git a/configure b/configure
index c54e10ea26..c1eb4923dd 100755
--- a/configure
+++ b/configure
@@ -231,12 +231,10 @@ fi
 
 # default parameters
 cpu=""
-iasl="iasl"
 interp_prefix="/usr/gnemul/qemu-%M"
 static="no"
 cross_compile="no"
 cross_prefix=""
-audio_drv_list="default"
 block_drv_rw_whitelist=""
 block_drv_ro_whitelist=""
 host_cc="cc"
@@ -312,7 +310,6 @@ linux_user=""
 bsd_user=""
 pkgversion=""
 pie=""
-trace_file="trace"
 coroutine=""
 tls_priority="NORMAL"
 plugins="$default_feature"
@@ -714,8 +711,6 @@ for opt do
   ;;
   --cxx=*)
   ;;
-  --iasl=*) iasl="$optarg"
-  ;;
   --objcc=*) objcc="$optarg"
   ;;
   --make=*) make="$optarg"
@@ -724,8 +719,6 @@ for opt do
   ;;
   --python=*) python="$optarg" ; explicit_python=yes
   ;;
-  --sphinx-build=*) sphinx_build="$optarg"
-  ;;
   --skip-meson) skip_meson=yes
   ;;
   --meson=*) meson="$optarg"
@@ -766,8 +759,6 @@ for opt do
                        error_exit "Can't mix --target-list-exclude with --target-list"
                    fi
   ;;
-  --with-trace-file=*) trace_file="$optarg"
-  ;;
   --with-default-devices) default_devices="true"
   ;;
   --without-default-devices) default_devices="false"
@@ -826,8 +817,6 @@ for opt do
     # configure to be used by RPM and similar macros that set
     # lots of directory switches by default.
   ;;
-  --audio-drv-list=*) audio_drv_list="$optarg"
-  ;;
   --block-drv-rw-whitelist=*|--block-drv-whitelist=*) block_drv_rw_whitelist=$(echo "$optarg" | sed -e 's/,/ /g')
   ;;
   --block-drv-ro-whitelist=*) block_drv_ro_whitelist=$(echo "$optarg" | sed -e 's/,/ /g')
@@ -1001,12 +990,7 @@ for opt do
   --enable-jemalloc) meson_option_parse --enable-malloc=jemalloc jemalloc
   ;;
   # everything else has the same name in configure and meson
-  --enable-* | --disable-*) meson_option_parse "$opt" "$optarg"
-  ;;
-  *)
-      echo "ERROR: unknown option $opt"
-      echo "Try '$0 --help' for more information"
-      exit 1
+  --*) meson_option_parse "$opt" "$optarg"
   ;;
   esac
 done
@@ -1129,7 +1113,6 @@ $(echo Available targets: $default_target_list | \
 Advanced options (experts only):
   --cross-prefix=PREFIX    use PREFIX for compile tools, PREFIX can be blank [$cross_prefix]
   --cc=CC                  use C compiler CC [$cc]
-  --iasl=IASL              use ACPI compiler IASL [$iasl]
   --host-cc=CC             use C compiler CC [$host_cc] for code run at
                            build time
   --cxx=CXX                use C++ compiler CXX [$cxx]
@@ -1142,7 +1125,6 @@ Advanced options (experts only):
   --cross-cc-cflags-ARCH=  use compiler flags when building ARCH guest tests
   --make=MAKE              use specified make [$make]
   --python=PYTHON          use specified python [$python]
-  --sphinx-build=SPHINX    use specified sphinx-build [$sphinx_build]
   --meson=MESON            use specified meson [$meson]
   --ninja=NINJA            use specified ninja [$ninja]
   --smbd=SMBD              use specified smbd [$smbd]
@@ -2485,15 +2467,11 @@ if test "$skip_meson" = no; then
         --sysconfdir "$sysconfdir" \
         --localedir "$localedir" \
         --localstatedir "$local_statedir" \
-        -Daudio_drv_list=$audio_drv_list \
         -Ddefault_devices=$default_devices \
         -Ddocdir="$docdir" \
-        -Diasl="$iasl" \
         -Dqemu_firmwarepath="$firmwarepath" \
         -Dqemu_suffix="$qemu_suffix" \
         -Dsmbd="$smbd" \
-        -Dsphinx_build="$sphinx_build" \
-        -Dtrace_file="$trace_file" \
         -Doptimization=$(if test "$debug" = yes; then echo 0; else echo 2; fi) \
         -Ddebug=$(if test "$debug_info" = yes; then echo true; else echo false; fi) \
         -Dwerror=$(if test "$werror" = yes; then echo true; else echo false; fi) \
diff --git a/scripts/meson-buildoptions.py b/scripts/meson-buildoptions.py
index 4af8d6e732..3e540e8bb3 100755
--- a/scripts/meson-buildoptions.py
+++ b/scripts/meson-buildoptions.py
@@ -26,21 +26,18 @@ import shlex
 import sys
 
 SKIP_OPTIONS = {
-    "audio_drv_list",
     "default_devices",
     "docdir",
     "fuzzing_engine",
-    "iasl",
     "qemu_firmwarepath",
     "qemu_suffix",
     "smbd",
-    "sphinx_build",
-    "trace_file",
 }
 
 OPTION_NAMES = {
     "malloc": "enable-malloc",
     "trace_backends": "enable-trace-backends",
+    "trace_file": "with-trace-file",
 }
 
 BUILTIN_OPTIONS = {
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index 2123317654..bdaa128ee4 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -1,5 +1,7 @@
 # This file is generated by meson-buildoptions.py, do not edit!
 meson_options_help() {
+  printf "%s\n" '  --audio-drv-list=CHOICES Set audio driver list [default] (choices:'
+  printf "%s\n" '                           alsa/coreaudio/default/dsound/jack/oss/pa/sdl)'
   printf "%s\n" '  --disable-coroutine-pool coroutine freelist (better performance)'
   printf "%s\n" '  --disable-install-blobs  install provided firmware blobs'
   printf "%s\n" '  --enable-block-drv-whitelist-in-tools'
@@ -32,6 +34,9 @@ meson_options_help() {
   printf "%s\n" '  --enable-trace-backends=CHOICES'
   printf "%s\n" '                           Set available tracing backends [log] (choices:'
   printf "%s\n" '                           dtrace/ftrace/log/nop/simple/syslog/ust)'
+  printf "%s\n" '  --iasl=VALUE             Path to ACPI disassembler'
+  printf "%s\n" '  --sphinx-build=VALUE     Use specified sphinx-build for building document'
+  printf "%s\n" '  --with-trace-file=VALUE  Trace file prefix for simple backend [trace]'
   printf "%s\n" ''
   printf "%s\n" 'Optional features, enabled with --enable-FEATURE and'
   printf "%s\n" 'disabled with --disable-FEATURE, default is enabled if available'
@@ -147,6 +152,7 @@ _meson_option_parse() {
     --disable-alsa) printf "%s" -Dalsa=disabled ;;
     --enable-attr) printf "%s" -Dattr=enabled ;;
     --disable-attr) printf "%s" -Dattr=disabled ;;
+    --audio-drv-list=*) quote_sh "-Daudio_drv_list=$2" ;;
     --enable-auth-pam) printf "%s" -Dauth_pam=enabled ;;
     --disable-auth-pam) printf "%s" -Dauth_pam=disabled ;;
     --enable-avx2) printf "%s" -Davx2=enabled ;;
@@ -229,6 +235,7 @@ _meson_option_parse() {
     --disable-hax) printf "%s" -Dhax=disabled ;;
     --enable-hvf) printf "%s" -Dhvf=enabled ;;
     --disable-hvf) printf "%s" -Dhvf=disabled ;;
+    --iasl=*) quote_sh "-Diasl=$2" ;;
     --enable-iconv) printf "%s" -Diconv=enabled ;;
     --disable-iconv) printf "%s" -Diconv=disabled ;;
     --enable-install-blobs) printf "%s" -Dinstall_blobs=true ;;
@@ -333,6 +340,7 @@ _meson_option_parse() {
     --disable-snappy) printf "%s" -Dsnappy=disabled ;;
     --enable-sparse) printf "%s" -Dsparse=enabled ;;
     --disable-sparse) printf "%s" -Dsparse=disabled ;;
+    --sphinx-build=*) quote_sh "-Dsphinx_build=$2" ;;
     --enable-spice) printf "%s" -Dspice=enabled ;;
     --disable-spice) printf "%s" -Dspice=disabled ;;
     --enable-spice-protocol) printf "%s" -Dspice_protocol=enabled ;;
@@ -348,6 +356,7 @@ _meson_option_parse() {
     --enable-tpm) printf "%s" -Dtpm=enabled ;;
     --disable-tpm) printf "%s" -Dtpm=disabled ;;
     --enable-trace-backends=*) quote_sh "-Dtrace_backends=$2" ;;
+    --with-trace-file=*) quote_sh "-Dtrace_file=$2" ;;
     --enable-u2f) printf "%s" -Du2f=enabled ;;
     --disable-u2f) printf "%s" -Du2f=disabled ;;
     --enable-usb-redir) printf "%s" -Dusb_redir=enabled ;;

From 41f2ae281953c7f7cfc28c5b93f9e0fb65cbd301 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:33:52 +0200
Subject: [PATCH 105/935] meson, configure: move --tls-priority to meson
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Use the new support for string option parsing.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure                     | 5 -----
 meson.build                   | 3 ++-
 meson_options.txt             | 2 ++
 scripts/meson-buildoptions.sh | 3 +++
 4 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/configure b/configure
index c1eb4923dd..9062f9ccd6 100755
--- a/configure
+++ b/configure
@@ -311,7 +311,6 @@ bsd_user=""
 pkgversion=""
 pie=""
 coroutine=""
-tls_priority="NORMAL"
 plugins="$default_feature"
 meson=""
 meson_args=""
@@ -939,8 +938,6 @@ for opt do
   --enable-uuid|--disable-uuid)
       echo "$0: $opt is obsolete, UUID support is always built" >&2
   ;;
-  --tls-priority=*) tls_priority="$optarg"
-  ;;
   --disable-vhost-user) vhost_user="no"
   ;;
   --enable-vhost-user) vhost_user="yes"
@@ -1170,7 +1167,6 @@ Advanced options (experts only):
   --with-coroutine=BACKEND coroutine backend. Supported options:
                            ucontext, sigaltstack, windows
   --enable-gcov            enable test coverage analysis with gcov
-  --tls-priority           default TLS protocol/cipher priority string
   --enable-plugins
                            enable plugins via shared library loading
   --disable-containers     don't use containers for cross-building
@@ -2201,7 +2197,6 @@ if test "$modules" = "yes"; then
   echo "CONFIG_STAMP=_$( (echo $qemu_version; echo $pkgversion; cat $0) | $shacmd - | cut -f1 -d\ )" >> $config_host_mak
   echo "CONFIG_MODULES=y" >> $config_host_mak
 fi
-echo "CONFIG_TLS_PRIORITY=\"$tls_priority\"" >> $config_host_mak
 
 if test "$vhost_scsi" = "yes" ; then
   echo "CONFIG_VHOST_SCSI=y" >> $config_host_mak
diff --git a/meson.build b/meson.build
index 6606c9790f..ae3b3a45e7 100644
--- a/meson.build
+++ b/meson.build
@@ -1598,6 +1598,7 @@ foreach k : get_option('trace_backends')
   config_host_data.set('CONFIG_TRACE_' + k.to_upper(), true)
 endforeach
 config_host_data.set_quoted('CONFIG_TRACE_FILE', get_option('trace_file'))
+config_host_data.set_quoted('CONFIG_TLS_PRIORITY', get_option('tls_priority'))
 if iasl.found()
   config_host_data.set_quoted('CONFIG_IASL', iasl.full_path())
 endif
@@ -3820,7 +3821,7 @@ summary(summary_info, bool_yn: true, section: 'Block layer support')
 
 # Crypto
 summary_info = {}
-summary_info += {'TLS priority':      config_host['CONFIG_TLS_PRIORITY']}
+summary_info += {'TLS priority':      get_option('tls_priority')}
 summary_info += {'GNUTLS support':    gnutls}
 if gnutls.found()
   summary_info += {'  GNUTLS crypto':   gnutls_crypto.found()}
diff --git a/meson_options.txt b/meson_options.txt
index 390701517c..c8e0a10d91 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -14,6 +14,8 @@ option('sphinx_build', type : 'string', value : '',
        description: 'Use specified sphinx-build for building document')
 option('iasl', type : 'string', value : '',
        description: 'Path to ACPI disassembler')
+option('tls_priority', type : 'string', value : 'NORMAL',
+       description: 'Default TLS protocol/cipher priority string')
 option('default_devices', type : 'boolean', value : true,
        description: 'Include a default selection of devices in emulators')
 option('audio_drv_list', type: 'array', value: ['default'],
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index bdaa128ee4..3919eeec0b 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -36,6 +36,8 @@ meson_options_help() {
   printf "%s\n" '                           dtrace/ftrace/log/nop/simple/syslog/ust)'
   printf "%s\n" '  --iasl=VALUE             Path to ACPI disassembler'
   printf "%s\n" '  --sphinx-build=VALUE     Use specified sphinx-build for building document'
+  printf "%s\n" '  --tls-priority=VALUE     Default TLS protocol/cipher priority string'
+  printf "%s\n" '                           [NORMAL]'
   printf "%s\n" '  --with-trace-file=VALUE  Trace file prefix for simple backend [trace]'
   printf "%s\n" ''
   printf "%s\n" 'Optional features, enabled with --enable-FEATURE and'
@@ -351,6 +353,7 @@ _meson_option_parse() {
     --disable-tcg) printf "%s" -Dtcg=disabled ;;
     --enable-tcg-interpreter) printf "%s" -Dtcg_interpreter=true ;;
     --disable-tcg-interpreter) printf "%s" -Dtcg_interpreter=false ;;
+    --tls-priority=*) quote_sh "-Dtls_priority=$2" ;;
     --enable-tools) printf "%s" -Dtools=enabled ;;
     --disable-tools) printf "%s" -Dtools=disabled ;;
     --enable-tpm) printf "%s" -Dtpm=enabled ;;

From 622d64fffe4f287af4f0d5e438d4aa42da377811 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:33:53 +0200
Subject: [PATCH 106/935] meson, configure: move bdrv whitelists to meson
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Use the new support for string option parsing.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure                     | 14 --------------
 meson.build                   | 27 ++++++++++++++++-----------
 meson_options.txt             |  4 ++++
 scripts/meson-buildoptions.sh |  8 ++++++++
 4 files changed, 28 insertions(+), 25 deletions(-)

diff --git a/configure b/configure
index 9062f9ccd6..e81ea490da 100755
--- a/configure
+++ b/configure
@@ -235,8 +235,6 @@ interp_prefix="/usr/gnemul/qemu-%M"
 static="no"
 cross_compile="no"
 cross_prefix=""
-block_drv_rw_whitelist=""
-block_drv_ro_whitelist=""
 host_cc="cc"
 lto="false"
 stack_protector=""
@@ -816,10 +814,6 @@ for opt do
     # configure to be used by RPM and similar macros that set
     # lots of directory switches by default.
   ;;
-  --block-drv-rw-whitelist=*|--block-drv-whitelist=*) block_drv_rw_whitelist=$(echo "$optarg" | sed -e 's/,/ /g')
-  ;;
-  --block-drv-ro-whitelist=*) block_drv_ro_whitelist=$(echo "$optarg" | sed -e 's/,/ /g')
-  ;;
   --enable-debug-tcg) debug_tcg="yes"
   ;;
   --disable-debug-tcg) debug_tcg="no"
@@ -1155,12 +1149,6 @@ Advanced options (experts only):
   --disable-stack-protector disable compiler-provided stack protection
   --audio-drv-list=LIST    set audio drivers to try if -audiodev is not used
   --block-drv-whitelist=L  Same as --block-drv-rw-whitelist=L
-  --block-drv-rw-whitelist=L
-                           set block driver read-write whitelist
-                           (by default affects only QEMU, not tools like qemu-img)
-  --block-drv-ro-whitelist=L
-                           set block driver read-only whitelist
-                           (by default affects only QEMU, not tools like qemu-img)
   --with-trace-file=NAME   Full PATH,NAME of file to store traces
                            Default:trace-<pid>
   --cpu=CPU                Build for host CPU [$cpu]
@@ -2185,8 +2173,6 @@ fi
 if test "$static" = "yes" ; then
   echo "CONFIG_STATIC=y" >> $config_host_mak
 fi
-echo "CONFIG_BDRV_RW_WHITELIST=$block_drv_rw_whitelist" >> $config_host_mak
-echo "CONFIG_BDRV_RO_WHITELIST=$block_drv_ro_whitelist" >> $config_host_mak
 qemu_version=$(head $source_path/VERSION)
 echo "PKGVERSION=$pkgversion" >>$config_host_mak
 echo "SRC_PATH=$source_path" >> $config_host_mak
diff --git a/meson.build b/meson.build
index ae3b3a45e7..3604a0d264 100644
--- a/meson.build
+++ b/meson.build
@@ -1594,6 +1594,19 @@ have_virtfs = get_option('virtfs') \
 
 have_virtfs_proxy_helper = targetos != 'darwin' and have_virtfs and have_tools
 
+if get_option('block_drv_ro_whitelist') == ''
+  config_host_data.set('CONFIG_BDRV_RO_WHITELIST', '')
+else
+  config_host_data.set('CONFIG_BDRV_RO_WHITELIST',
+        '"' + get_option('block_drv_ro_whitelist').replace(',', '", "') + '", ')
+endif
+if get_option('block_drv_rw_whitelist') == ''
+  config_host_data.set('CONFIG_BDRV_RW_WHITELIST', '')
+else
+  config_host_data.set('CONFIG_BDRV_RW_WHITELIST',
+        '"' + get_option('block_drv_rw_whitelist').replace(',', '", "') + '", ')
+endif
+
 foreach k : get_option('trace_backends')
   config_host_data.set('CONFIG_TRACE_' + k.to_upper(), true)
 endforeach
@@ -2209,16 +2222,8 @@ config_host_data.set('HAVE_VSS_SDK', have_vss_sdk)
 
 ignored = ['CONFIG_QEMU_INTERP_PREFIX', # actually per-target
     'HAVE_GDB_BIN']
-arrays = ['CONFIG_BDRV_RW_WHITELIST', 'CONFIG_BDRV_RO_WHITELIST']
 foreach k, v: config_host
-  if ignored.contains(k)
-    # do nothing
-  elif arrays.contains(k)
-    if v != ''
-      v = '"' + '", "'.join(v.split()) + '", '
-    endif
-    config_host_data.set(k, v)
-  elif k.startswith('CONFIG_')
+  if k.startswith('CONFIG_') and not ignored.contains(k)
     config_host_data.set(k, v == 'y' ? 1 : v)
   endif
 endforeach
@@ -3800,8 +3805,8 @@ summary_info = {}
 summary_info += {'coroutine backend': config_host['CONFIG_COROUTINE_BACKEND']}
 summary_info += {'coroutine pool':    have_coroutine_pool}
 if have_block
-  summary_info += {'Block whitelist (rw)': config_host['CONFIG_BDRV_RW_WHITELIST']}
-  summary_info += {'Block whitelist (ro)': config_host['CONFIG_BDRV_RO_WHITELIST']}
+  summary_info += {'Block whitelist (rw)': get_option('block_drv_rw_whitelist')}
+  summary_info += {'Block whitelist (ro)': get_option('block_drv_ro_whitelist')}
   summary_info += {'Use block whitelist in tools': get_option('block_drv_whitelist_in_tools')}
   summary_info += {'VirtFS support':    have_virtfs}
   summary_info += {'build virtiofs daemon': have_virtiofsd}
diff --git a/meson_options.txt b/meson_options.txt
index c8e0a10d91..430674522f 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -21,6 +21,10 @@ option('default_devices', type : 'boolean', value : true,
 option('audio_drv_list', type: 'array', value: ['default'],
        choices: ['alsa', 'coreaudio', 'default', 'dsound', 'jack', 'oss', 'pa', 'sdl'],
        description: 'Set audio driver list')
+option('block_drv_rw_whitelist', type : 'string', value : '',
+       description: 'set block driver read-write whitelist (by default affects only QEMU, not tools like qemu-img)')
+option('block_drv_ro_whitelist', type : 'string', value : '',
+       description: 'set block driver read-only whitelist (by default affects only QEMU, not tools like qemu-img)')
 option('fuzzing_engine', type : 'string', value : '',
        description: 'fuzzing engine library for OSS-Fuzz')
 option('trace_file', type: 'string', value: 'trace',
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index 3919eeec0b..d5cba2638a 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -2,6 +2,12 @@
 meson_options_help() {
   printf "%s\n" '  --audio-drv-list=CHOICES Set audio driver list [default] (choices:'
   printf "%s\n" '                           alsa/coreaudio/default/dsound/jack/oss/pa/sdl)'
+  printf "%s\n" '  --block-drv-ro-whitelist=VALUE'
+  printf "%s\n" '                           set block driver read-only whitelist (by default'
+  printf "%s\n" '                           affects only QEMU, not tools like qemu-img)'
+  printf "%s\n" '  --block-drv-rw-whitelist=VALUE'
+  printf "%s\n" '                           set block driver read-write whitelist (by default'
+  printf "%s\n" '                           affects only QEMU, not tools like qemu-img)'
   printf "%s\n" '  --disable-coroutine-pool coroutine freelist (better performance)'
   printf "%s\n" '  --disable-install-blobs  install provided firmware blobs'
   printf "%s\n" '  --enable-block-drv-whitelist-in-tools'
@@ -161,6 +167,8 @@ _meson_option_parse() {
     --disable-avx2) printf "%s" -Davx2=disabled ;;
     --enable-avx512f) printf "%s" -Davx512f=enabled ;;
     --disable-avx512f) printf "%s" -Davx512f=disabled ;;
+    --block-drv-ro-whitelist=*) quote_sh "-Dblock_drv_ro_whitelist=$2" ;;
+    --block-drv-rw-whitelist=*) quote_sh "-Dblock_drv_rw_whitelist=$2" ;;
     --enable-block-drv-whitelist-in-tools) printf "%s" -Dblock_drv_whitelist_in_tools=true ;;
     --disable-block-drv-whitelist-in-tools) printf "%s" -Dblock_drv_whitelist_in_tools=false ;;
     --enable-bochs) printf "%s" -Dbochs=enabled ;;

From b0b4323e3219810d14f36c159e67a8332a86f9fd Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:33:54 +0200
Subject: [PATCH 107/935] meson, configure: move --with-pkgversion,
 CONFIG_STAMP to meson
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The hash is now generated with a Python script.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure                     | 23 -----------------------
 docs/meson.build              |  2 +-
 meson.build                   | 10 +++++++++-
 meson_options.txt             |  2 ++
 scripts/meson-buildoptions.py |  1 +
 scripts/meson-buildoptions.sh |  3 +++
 scripts/qemu-stamp.py         | 24 ++++++++++++++++++++++++
 7 files changed, 40 insertions(+), 25 deletions(-)
 create mode 100644 scripts/qemu-stamp.py

diff --git a/configure b/configure
index e81ea490da..b8d3d04072 100755
--- a/configure
+++ b/configure
@@ -306,7 +306,6 @@ qemu_suffix="qemu"
 softmmu="yes"
 linux_user=""
 bsd_user=""
-pkgversion=""
 pie=""
 coroutine=""
 plugins="$default_feature"
@@ -897,8 +896,6 @@ for opt do
   ;;
   --enable-fdt=*) fdt="$optarg"
   ;;
-  --with-pkgversion=*) pkgversion="$optarg"
-  ;;
   --with-coroutine=*) coroutine="$optarg"
   ;;
   --disable-vhost-net) vhost_net="no"
@@ -1136,7 +1133,6 @@ Advanced options (experts only):
   --firmwarepath=PATH      search PATH for firmware files
   --efi-aarch64=PATH       PATH of efi file to use for aarch64 VMs.
   --with-suffix=SUFFIX     suffix for QEMU data inside datadir/libdir/sysconfdir/docdir [$qemu_suffix]
-  --with-pkgversion=VERS   use specified string as sub-version of the package
   --without-default-features default all --enable-* options to "disabled"
   --without-default-devices  do not include any device that is not needed to
                            start the emulator (only use if you are including
@@ -1723,21 +1719,6 @@ if ! compile_prog "$glib_cflags -Werror" "$glib_libs" ; then
     fi
 fi
 
-##########################################
-# SHA command probe for modules
-if test "$modules" = yes; then
-    shacmd_probe="sha1sum sha1 shasum"
-    for c in $shacmd_probe; do
-        if has $c; then
-            shacmd="$c"
-            break
-        fi
-    done
-    if test "$shacmd" = ""; then
-        error_exit "one of the checksum commands is required to enable modules: $shacmd_probe"
-    fi
-fi
-
 ##########################################
 # fdt probe
 
@@ -2174,13 +2155,9 @@ if test "$static" = "yes" ; then
   echo "CONFIG_STATIC=y" >> $config_host_mak
 fi
 qemu_version=$(head $source_path/VERSION)
-echo "PKGVERSION=$pkgversion" >>$config_host_mak
 echo "SRC_PATH=$source_path" >> $config_host_mak
 echo "TARGET_DIRS=$target_list" >> $config_host_mak
 if test "$modules" = "yes"; then
-  # $shacmd can generate a hash started with digit, which the compiler doesn't
-  # like as an symbol. So prefix it with an underscore
-  echo "CONFIG_STAMP=_$( (echo $qemu_version; echo $pkgversion; cat $0) | $shacmd - | cut -f1 -d\ )" >> $config_host_mak
   echo "CONFIG_MODULES=y" >> $config_host_mak
 fi
 
diff --git a/docs/meson.build b/docs/meson.build
index 831d4aea2b..9136fed3b7 100644
--- a/docs/meson.build
+++ b/docs/meson.build
@@ -35,7 +35,7 @@ if sphinx_build.found()
 endif
 
 if build_docs
-  SPHINX_ARGS += ['-Dversion=' + meson.project_version(), '-Drelease=' + config_host['PKGVERSION']]
+  SPHINX_ARGS += ['-Dversion=' + meson.project_version(), '-Drelease=' + get_option('pkgversion')]
 
   man_pages = {
         'qemu-ga.8': (have_ga ? 'man8' : ''),
diff --git a/meson.build b/meson.build
index 3604a0d264..4ddc72f070 100644
--- a/meson.build
+++ b/meson.build
@@ -1628,6 +1628,14 @@ config_host_data.set_quoted('CONFIG_QEMU_LOCALSTATEDIR', get_option('prefix') /
 config_host_data.set_quoted('CONFIG_QEMU_MODDIR', get_option('prefix') / qemu_moddir)
 config_host_data.set_quoted('CONFIG_SYSCONFDIR', get_option('prefix') / get_option('sysconfdir'))
 
+if config_host.has_key('CONFIG_MODULES')
+  config_host_data.set('CONFIG_STAMP', run_command(
+      meson.current_source_dir() / 'scripts/qemu-stamp.py',
+      meson.project_version(), get_option('pkgversion'), '--',
+      meson.current_source_dir() / 'configure',
+      capture: true, check: true).stdout().strip())
+endif
+
 have_slirp_smbd = get_option('slirp_smbd') \
   .require(targetos != 'windows', error_message: 'Host smbd not supported on this platform.') \
   .allowed()
@@ -2799,7 +2807,7 @@ tracetool_depends = files(
 
 qemu_version_cmd = [find_program('scripts/qemu-version.sh'),
                     meson.current_source_dir(),
-                    config_host['PKGVERSION'], meson.project_version()]
+                    get_option('pkgversion'), meson.project_version()]
 qemu_version = custom_target('qemu-version.h',
                              output: 'qemu-version.h',
                              command: qemu_version_cmd,
diff --git a/meson_options.txt b/meson_options.txt
index 430674522f..bdb45092ca 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -8,6 +8,8 @@ option('docdir', type : 'string', value : 'doc',
        description: 'Base directory for documentation installation (can be empty)')
 option('qemu_firmwarepath', type : 'string', value : '',
        description: 'search PATH for firmware files')
+option('pkgversion', type : 'string', value : '',
+       description: 'use specified string as sub-version of the package')
 option('smbd', type : 'string', value : '',
        description: 'Path to smbd for slirp networking')
 option('sphinx_build', type : 'string', value : '',
diff --git a/scripts/meson-buildoptions.py b/scripts/meson-buildoptions.py
index 3e540e8bb3..0f9603a7f6 100755
--- a/scripts/meson-buildoptions.py
+++ b/scripts/meson-buildoptions.py
@@ -36,6 +36,7 @@ SKIP_OPTIONS = {
 
 OPTION_NAMES = {
     "malloc": "enable-malloc",
+    "pkgversion": "with-pkgversion",
     "trace_backends": "enable-trace-backends",
     "trace_file": "with-trace-file",
 }
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index d5cba2638a..3a86dd7271 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -44,6 +44,8 @@ meson_options_help() {
   printf "%s\n" '  --sphinx-build=VALUE     Use specified sphinx-build for building document'
   printf "%s\n" '  --tls-priority=VALUE     Default TLS protocol/cipher priority string'
   printf "%s\n" '                           [NORMAL]'
+  printf "%s\n" '  --with-pkgversion=VALUE  use specified string as sub-version of the'
+  printf "%s\n" '                           package'
   printf "%s\n" '  --with-trace-file=VALUE  Trace file prefix for simple backend [trace]'
   printf "%s\n" ''
   printf "%s\n" 'Optional features, enabled with --enable-FEATURE and'
@@ -309,6 +311,7 @@ _meson_option_parse() {
     --disable-pa) printf "%s" -Dpa=disabled ;;
     --enable-parallels) printf "%s" -Dparallels=enabled ;;
     --disable-parallels) printf "%s" -Dparallels=disabled ;;
+    --with-pkgversion=*) quote_sh "-Dpkgversion=$2" ;;
     --enable-png) printf "%s" -Dpng=enabled ;;
     --disable-png) printf "%s" -Dpng=disabled ;;
     --enable-profiler) printf "%s" -Dprofiler=true ;;
diff --git a/scripts/qemu-stamp.py b/scripts/qemu-stamp.py
new file mode 100644
index 0000000000..7beeeb07ed
--- /dev/null
+++ b/scripts/qemu-stamp.py
@@ -0,0 +1,24 @@
+#! /usr/bin/env python3
+
+# Usage: scripts/qemu-stamp.py STRING1 STRING2... -- FILE1 FILE2...
+import hashlib
+import os
+import sys
+
+sha = hashlib.sha1()
+is_file = False
+for arg in sys.argv[1:]:
+    if arg == '--':
+        is_file = True
+        continue
+    if is_file:
+        with open(arg, 'rb') as f:
+            for chunk in iter(lambda: f.read(65536), b''):
+                sha.update(chunk)
+    else:
+        sha.update(os.fsencode(arg))
+        sha.update(b'\n')
+
+# The hash can start with a digit, which the compiler doesn't
+# like as an symbol. So prefix it with an underscore
+print("_" + sha.hexdigest())

From ce6119dc8e025f17c38926aa793f96ed74cb3cc0 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:33:55 +0200
Subject: [PATCH 108/935] meson, configure: move --interp-prefix to meson
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This is the last CONFIG_* entry in config-host.mak that had to be
special cased.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure                     | 6 ------
 meson.build                   | 6 ++----
 meson_options.txt             | 2 ++
 scripts/meson-buildoptions.sh | 3 +++
 4 files changed, 7 insertions(+), 10 deletions(-)

diff --git a/configure b/configure
index b8d3d04072..4102683d75 100755
--- a/configure
+++ b/configure
@@ -231,7 +231,6 @@ fi
 
 # default parameters
 cpu=""
-interp_prefix="/usr/gnemul/qemu-%M"
 static="no"
 cross_compile="no"
 cross_prefix=""
@@ -697,8 +696,6 @@ for opt do
   ;;
   --prefix=*) prefix="$optarg"
   ;;
-  --interp-prefix=*) interp_prefix="$optarg"
-  ;;
   --cross-prefix=*)
   ;;
   --cc=*)
@@ -1091,8 +1088,6 @@ Options: [defaults in brackets after descriptions]
 Standard options:
   --help                   print this message
   --prefix=PREFIX          install in PREFIX [$prefix]
-  --interp-prefix=PREFIX   where to find shared libraries, etc.
-                           use %M for cpu name [$interp_prefix]
   --target-list=LIST       set target list (default: build all)
 $(echo Available targets: $default_target_list | \
   fold -s -w 53 | sed -e 's/^/                           /')
@@ -2289,7 +2284,6 @@ for target in $target_list; do
     esac
 done
 
-echo "CONFIG_QEMU_INTERP_PREFIX=$interp_prefix" | sed 's/%M/@0@/' >> $config_host_mak
 if test "$default_targets" = "yes"; then
   echo "CONFIG_DEFAULT_TARGETS=y" >> $config_host_mak
 fi
diff --git a/meson.build b/meson.build
index 4ddc72f070..20d9074c4f 100644
--- a/meson.build
+++ b/meson.build
@@ -2228,10 +2228,8 @@ if targetos == 'windows' and link_language == 'cpp'
 endif
 config_host_data.set('HAVE_VSS_SDK', have_vss_sdk)
 
-ignored = ['CONFIG_QEMU_INTERP_PREFIX', # actually per-target
-    'HAVE_GDB_BIN']
 foreach k, v: config_host
-  if k.startswith('CONFIG_') and not ignored.contains(k)
+  if k.startswith('CONFIG_')
     config_host_data.set(k, v == 'y' ? 1 : v)
   endif
 endforeach
@@ -2337,7 +2335,7 @@ foreach target : target_dirs
     config_target += {
       'CONFIG_USER_ONLY': 'y',
       'CONFIG_QEMU_INTERP_PREFIX':
-        config_host['CONFIG_QEMU_INTERP_PREFIX'].format(config_target['TARGET_NAME'])
+        get_option('interp_prefix').replace('%M', config_target['TARGET_NAME'])
     }
   endif
 
diff --git a/meson_options.txt b/meson_options.txt
index bdb45092ca..ae6c4b2475 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -27,6 +27,8 @@ option('block_drv_rw_whitelist', type : 'string', value : '',
        description: 'set block driver read-write whitelist (by default affects only QEMU, not tools like qemu-img)')
 option('block_drv_ro_whitelist', type : 'string', value : '',
        description: 'set block driver read-only whitelist (by default affects only QEMU, not tools like qemu-img)')
+option('interp_prefix', type : 'string', value : '/usr/gnemul/qemu-%M',
+       description: 'where to find shared libraries etc., use %M for cpu name')
 option('fuzzing_engine', type : 'string', value : '',
        description: 'fuzzing engine library for OSS-Fuzz')
 option('trace_file', type: 'string', value: 'trace',
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index 3a86dd7271..59c4a27c5a 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -41,6 +41,8 @@ meson_options_help() {
   printf "%s\n" '                           Set available tracing backends [log] (choices:'
   printf "%s\n" '                           dtrace/ftrace/log/nop/simple/syslog/ust)'
   printf "%s\n" '  --iasl=VALUE             Path to ACPI disassembler'
+  printf "%s\n" '  --interp-prefix=VALUE    where to find shared libraries etc., use %M for'
+  printf "%s\n" '                           cpu name [/usr/gnemul/qemu-%M]'
   printf "%s\n" '  --sphinx-build=VALUE     Use specified sphinx-build for building document'
   printf "%s\n" '  --tls-priority=VALUE     Default TLS protocol/cipher priority string'
   printf "%s\n" '                           [NORMAL]'
@@ -252,6 +254,7 @@ _meson_option_parse() {
     --disable-iconv) printf "%s" -Diconv=disabled ;;
     --enable-install-blobs) printf "%s" -Dinstall_blobs=true ;;
     --disable-install-blobs) printf "%s" -Dinstall_blobs=false ;;
+    --interp-prefix=*) quote_sh "-Dinterp_prefix=$2" ;;
     --enable-jack) printf "%s" -Djack=enabled ;;
     --disable-jack) printf "%s" -Djack=disabled ;;
     --enable-keyring) printf "%s" -Dkeyring=enabled ;;

From f7fb6c6ad4f6c5dd2e37af9ec60c51d6de24a24e Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:33:56 +0200
Subject: [PATCH 109/935] meson: always combine directories with prefix
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Meson allows directories such as "bindir" to be relative to the prefix.  Right
now configure is forcing an absolute path, but that is not really necessary:
just make sure all uses of the directory variables are prefixed appropriately.
Do the same also for the options that are custom for QEMU, i.e. docdir and
qemu_firmwarepath.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 meson.build | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/meson.build b/meson.build
index 20d9074c4f..65c44ea010 100644
--- a/meson.build
+++ b/meson.build
@@ -1620,7 +1620,7 @@ config_host_data.set_quoted('CONFIG_PREFIX', get_option('prefix'))
 config_host_data.set_quoted('CONFIG_QEMU_CONFDIR', get_option('prefix') / qemu_confdir)
 config_host_data.set_quoted('CONFIG_QEMU_DATADIR', get_option('prefix') / qemu_datadir)
 config_host_data.set_quoted('CONFIG_QEMU_DESKTOPDIR', get_option('prefix') / qemu_desktopdir)
-config_host_data.set_quoted('CONFIG_QEMU_FIRMWAREPATH', get_option('qemu_firmwarepath'))
+config_host_data.set_quoted('CONFIG_QEMU_FIRMWAREPATH', get_option('prefix') / get_option('qemu_firmwarepath'))
 config_host_data.set_quoted('CONFIG_QEMU_HELPERDIR', get_option('prefix') / get_option('libexecdir'))
 config_host_data.set_quoted('CONFIG_QEMU_ICONDIR', get_option('prefix') / qemu_icondir)
 config_host_data.set_quoted('CONFIG_QEMU_LOCALEDIR', get_option('prefix') / get_option('localedir'))
@@ -3629,20 +3629,20 @@ endif
 summary_info = {}
 summary_info += {'Install prefix':    get_option('prefix')}
 summary_info += {'BIOS directory':    qemu_datadir}
-summary_info += {'firmware path':     get_option('qemu_firmwarepath')}
-summary_info += {'binary directory':  get_option('bindir')}
-summary_info += {'library directory': get_option('libdir')}
+summary_info += {'firmware path':     get_option('prefix') / get_option('qemu_firmwarepath')}
+summary_info += {'binary directory':  get_option('prefix') / get_option('bindir')}
+summary_info += {'library directory': get_option('prefix') / get_option('libdir')}
 summary_info += {'module directory':  qemu_moddir}
-summary_info += {'libexec directory': get_option('libexecdir')}
-summary_info += {'include directory': get_option('includedir')}
-summary_info += {'config directory':  get_option('sysconfdir')}
+summary_info += {'libexec directory': get_option('prefix') / get_option('libexecdir')}
+summary_info += {'include directory': get_option('prefix') / get_option('includedir')}
+summary_info += {'config directory':  get_option('prefix') / get_option('sysconfdir')}
 if targetos != 'windows'
-  summary_info += {'local state directory': get_option('localstatedir')}
-  summary_info += {'Manual directory':      get_option('mandir')}
+  summary_info += {'local state directory': get_option('prefix') / get_option('localstatedir')}
+  summary_info += {'Manual directory':      get_option('prefix') / get_option('mandir')}
 else
   summary_info += {'local state directory': 'queried at runtime'}
 endif
-summary_info += {'Doc directory':     get_option('docdir')}
+summary_info += {'Doc directory':     get_option('prefix') / get_option('docdir')}
 summary_info += {'Build directory':   meson.current_build_dir()}
 summary_info += {'Source path':       meson.current_source_dir()}
 summary_info += {'GIT submodules':    config_host['GIT_SUBMODULES']}

From c09c1ce7e9507f8421b01076e1ed94b4c51d8f53 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:33:57 +0200
Subject: [PATCH 110/935] configure: switch directory options to automatic
 parsing
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

While prefix, bindir and qemu_suffix needs special treatment due to
differences between Windows and POSIX systems, everything else
needs no extra code in configure.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure                     | 58 ++---------------------------------
 meson_options.txt             |  4 +--
 scripts/meson-buildoptions.py | 11 +++++--
 scripts/meson-buildoptions.sh | 21 +++++++++++++
 4 files changed, 34 insertions(+), 60 deletions(-)

diff --git a/configure b/configure
index 4102683d75..b8e3176808 100755
--- a/configure
+++ b/configure
@@ -311,6 +311,7 @@ plugins="$default_feature"
 meson=""
 meson_args=""
 ninja=""
+bindir="bin"
 skip_meson=no
 
 # The following Meson options are handled manually (still they
@@ -670,6 +671,7 @@ if test "$mingw32" = "yes" ; then
   CONFIGURE_CFLAGS="-mthreads $CONFIGURE_CFLAGS"
   write_c_skeleton;
   prefix="/qemu"
+  bindir=""
   qemu_suffix=""
 fi
 
@@ -776,30 +778,10 @@ for opt do
     static="yes"
     QEMU_PKG_CONFIG_FLAGS="--static $QEMU_PKG_CONFIG_FLAGS"
   ;;
-  --mandir=*) mandir="$optarg"
-  ;;
   --bindir=*) bindir="$optarg"
   ;;
-  --libdir=*) libdir="$optarg"
-  ;;
-  --libexecdir=*) libexecdir="$optarg"
-  ;;
-  --includedir=*) includedir="$optarg"
-  ;;
-  --datadir=*) datadir="$optarg"
-  ;;
   --with-suffix=*) qemu_suffix="$optarg"
   ;;
-  --docdir=*) docdir="$optarg"
-  ;;
-  --localedir=*) localedir="$optarg"
-  ;;
-  --sysconfdir=*) sysconfdir="$optarg"
-  ;;
-  --localstatedir=*) local_statedir="$optarg"
-  ;;
-  --firmwarepath=*) firmwarepath="$optarg"
-  ;;
   --host=*|--build=*|\
   --disable-dependency-tracking|\
   --sbindir=*|--sharedstatedir=*|\
@@ -1022,23 +1004,6 @@ case $git_submodules_action in
     ;;
 esac
 
-libdir="${libdir:-$prefix/lib}"
-libexecdir="${libexecdir:-$prefix/libexec}"
-includedir="${includedir:-$prefix/include}"
-
-if test "$mingw32" = "yes" ; then
-    bindir="${bindir:-$prefix}"
-else
-    bindir="${bindir:-$prefix/bin}"
-fi
-mandir="${mandir:-$prefix/share/man}"
-datadir="${datadir:-$prefix/share}"
-docdir="${docdir:-$prefix/share/doc}"
-sysconfdir="${sysconfdir:-$prefix/etc}"
-local_statedir="${local_statedir:-$prefix/var}"
-firmwarepath="${firmwarepath:-$datadir/qemu-firmware}"
-localedir="${localedir:-$datadir/locale}"
-
 if eval test -z "\${cross_cc_$cpu}"; then
     eval "cross_cc_${cpu}=\$cc"
     cross_cc_vars="$cross_cc_vars cross_cc_${cpu}"
@@ -1116,16 +1081,7 @@ Advanced options (experts only):
   --with-git-submodules=validate fail if git submodules are not up to date
   --with-git-submodules=ignore   do not update or check git submodules (default if no .git dir)
   --static                 enable static build [$static]
-  --mandir=PATH            install man pages in PATH
-  --datadir=PATH           install firmware in PATH/$qemu_suffix
-  --localedir=PATH         install translation in PATH/$qemu_suffix
-  --docdir=PATH            install documentation in PATH/$qemu_suffix
   --bindir=PATH            install binaries in PATH
-  --libdir=PATH            install libraries in PATH
-  --libexecdir=PATH        install helper binaries in PATH
-  --sysconfdir=PATH        install config in PATH/$qemu_suffix
-  --localstatedir=PATH     install local state in PATH (set at runtime on win32)
-  --firmwarepath=PATH      search PATH for firmware files
   --efi-aarch64=PATH       PATH of efi file to use for aarch64 VMs.
   --with-suffix=SUFFIX     suffix for QEMU data inside datadir/libdir/sysconfdir/docdir [$qemu_suffix]
   --without-default-features default all --enable-* options to "disabled"
@@ -2410,18 +2366,8 @@ if test "$skip_meson" = no; then
   run_meson() {
     NINJA=$ninja $meson setup \
         --prefix "$prefix" \
-        --libdir "$libdir" \
-        --libexecdir "$libexecdir" \
         --bindir "$bindir" \
-        --includedir "$includedir" \
-        --datadir "$datadir" \
-        --mandir "$mandir" \
-        --sysconfdir "$sysconfdir" \
-        --localedir "$localedir" \
-        --localstatedir "$local_statedir" \
         -Ddefault_devices=$default_devices \
-        -Ddocdir="$docdir" \
-        -Dqemu_firmwarepath="$firmwarepath" \
         -Dqemu_suffix="$qemu_suffix" \
         -Dsmbd="$smbd" \
         -Doptimization=$(if test "$debug" = yes; then echo 0; else echo 2; fi) \
diff --git a/meson_options.txt b/meson_options.txt
index ae6c4b2475..15fe40c700 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -4,9 +4,9 @@
 
 option('qemu_suffix', type : 'string', value: 'qemu',
        description: 'Suffix for QEMU data/modules/config directories (can be empty)')
-option('docdir', type : 'string', value : 'doc',
+option('docdir', type : 'string', value : 'share/doc',
        description: 'Base directory for documentation installation (can be empty)')
-option('qemu_firmwarepath', type : 'string', value : '',
+option('qemu_firmwarepath', type : 'string', value : 'qemu-firmware',
        description: 'search PATH for firmware files')
 option('pkgversion', type : 'string', value : '',
        description: 'use specified string as sub-version of the package')
diff --git a/scripts/meson-buildoptions.py b/scripts/meson-buildoptions.py
index 0f9603a7f6..45cda8cd84 100755
--- a/scripts/meson-buildoptions.py
+++ b/scripts/meson-buildoptions.py
@@ -27,9 +27,7 @@ import sys
 
 SKIP_OPTIONS = {
     "default_devices",
-    "docdir",
     "fuzzing_engine",
-    "qemu_firmwarepath",
     "qemu_suffix",
     "smbd",
 }
@@ -37,12 +35,21 @@ SKIP_OPTIONS = {
 OPTION_NAMES = {
     "malloc": "enable-malloc",
     "pkgversion": "with-pkgversion",
+    "qemu_firmwarepath": "firmwarepath",
     "trace_backends": "enable-trace-backends",
     "trace_file": "with-trace-file",
 }
 
 BUILTIN_OPTIONS = {
+    "datadir",
+    "includedir",
+    "libdir",
+    "libexecdir",
+    "localedir",
+    "localstatedir",
+    "mandir",
     "strip",
+    "sysconfdir",
 }
 
 LINE_WIDTH = 76
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index 59c4a27c5a..f6a9b05c40 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -8,8 +8,11 @@ meson_options_help() {
   printf "%s\n" '  --block-drv-rw-whitelist=VALUE'
   printf "%s\n" '                           set block driver read-write whitelist (by default'
   printf "%s\n" '                           affects only QEMU, not tools like qemu-img)'
+  printf "%s\n" '  --datadir=VALUE          Data file directory [share]'
   printf "%s\n" '  --disable-coroutine-pool coroutine freelist (better performance)'
   printf "%s\n" '  --disable-install-blobs  install provided firmware blobs'
+  printf "%s\n" '  --docdir=VALUE           Base directory for documentation installation'
+  printf "%s\n" '                           (can be empty) [share/doc]'
   printf "%s\n" '  --enable-block-drv-whitelist-in-tools'
   printf "%s\n" '                           use block whitelist also in tools instead of only'
   printf "%s\n" '                           QEMU'
@@ -40,10 +43,18 @@ meson_options_help() {
   printf "%s\n" '  --enable-trace-backends=CHOICES'
   printf "%s\n" '                           Set available tracing backends [log] (choices:'
   printf "%s\n" '                           dtrace/ftrace/log/nop/simple/syslog/ust)'
+  printf "%s\n" '  --firmwarepath=VALUE     search PATH for firmware files [qemu-firmware]'
   printf "%s\n" '  --iasl=VALUE             Path to ACPI disassembler'
+  printf "%s\n" '  --includedir=VALUE       Header file directory [include]'
   printf "%s\n" '  --interp-prefix=VALUE    where to find shared libraries etc., use %M for'
   printf "%s\n" '                           cpu name [/usr/gnemul/qemu-%M]'
+  printf "%s\n" '  --libdir=VALUE           Library directory [lib64]'
+  printf "%s\n" '  --libexecdir=VALUE       Library executable directory [libexec]'
+  printf "%s\n" '  --localedir=VALUE        Locale data directory [share/locale]'
+  printf "%s\n" '  --localstatedir=VALUE    Localstate data directory [/var/local]'
+  printf "%s\n" '  --mandir=VALUE           Manual page directory [share/man]'
   printf "%s\n" '  --sphinx-build=VALUE     Use specified sphinx-build for building document'
+  printf "%s\n" '  --sysconfdir=VALUE       Sysconf data directory [etc]'
   printf "%s\n" '  --tls-priority=VALUE     Default TLS protocol/cipher priority string'
   printf "%s\n" '                           [NORMAL]'
   printf "%s\n" '  --with-pkgversion=VALUE  use specified string as sub-version of the'
@@ -206,6 +217,7 @@ _meson_option_parse() {
     --disable-curl) printf "%s" -Dcurl=disabled ;;
     --enable-curses) printf "%s" -Dcurses=enabled ;;
     --disable-curses) printf "%s" -Dcurses=disabled ;;
+    --datadir=*) quote_sh "-Ddatadir=$2" ;;
     --enable-dbus-display) printf "%s" -Ddbus_display=enabled ;;
     --disable-dbus-display) printf "%s" -Ddbus_display=disabled ;;
     --enable-debug-mutex) printf "%s" -Ddebug_mutex=true ;;
@@ -214,6 +226,7 @@ _meson_option_parse() {
     --disable-debug-stack-usage) printf "%s" -Ddebug_stack_usage=false ;;
     --enable-dmg) printf "%s" -Ddmg=enabled ;;
     --disable-dmg) printf "%s" -Ddmg=disabled ;;
+    --docdir=*) quote_sh "-Ddocdir=$2" ;;
     --enable-docs) printf "%s" -Ddocs=enabled ;;
     --disable-docs) printf "%s" -Ddocs=disabled ;;
     --enable-dsound) printf "%s" -Ddsound=enabled ;;
@@ -252,6 +265,7 @@ _meson_option_parse() {
     --iasl=*) quote_sh "-Diasl=$2" ;;
     --enable-iconv) printf "%s" -Diconv=enabled ;;
     --disable-iconv) printf "%s" -Diconv=disabled ;;
+    --includedir=*) quote_sh "-Dincludedir=$2" ;;
     --enable-install-blobs) printf "%s" -Dinstall_blobs=true ;;
     --disable-install-blobs) printf "%s" -Dinstall_blobs=false ;;
     --interp-prefix=*) quote_sh "-Dinterp_prefix=$2" ;;
@@ -265,6 +279,8 @@ _meson_option_parse() {
     --disable-l2tpv3) printf "%s" -Dl2tpv3=disabled ;;
     --enable-libdaxctl) printf "%s" -Dlibdaxctl=enabled ;;
     --disable-libdaxctl) printf "%s" -Dlibdaxctl=disabled ;;
+    --libdir=*) quote_sh "-Dlibdir=$2" ;;
+    --libexecdir=*) quote_sh "-Dlibexecdir=$2" ;;
     --enable-libiscsi) printf "%s" -Dlibiscsi=enabled ;;
     --disable-libiscsi) printf "%s" -Dlibiscsi=disabled ;;
     --enable-libnfs) printf "%s" -Dlibnfs=enabled ;;
@@ -283,6 +299,8 @@ _meson_option_parse() {
     --disable-linux-io-uring) printf "%s" -Dlinux_io_uring=disabled ;;
     --enable-live-block-migration) printf "%s" -Dlive_block_migration=enabled ;;
     --disable-live-block-migration) printf "%s" -Dlive_block_migration=disabled ;;
+    --localedir=*) quote_sh "-Dlocaledir=$2" ;;
+    --localstatedir=*) quote_sh "-Dlocalstatedir=$2" ;;
     --enable-lzfse) printf "%s" -Dlzfse=enabled ;;
     --disable-lzfse) printf "%s" -Dlzfse=disabled ;;
     --enable-lzo) printf "%s" -Dlzo=enabled ;;
@@ -290,6 +308,7 @@ _meson_option_parse() {
     --enable-malloc=*) quote_sh "-Dmalloc=$2" ;;
     --enable-malloc-trim) printf "%s" -Dmalloc_trim=enabled ;;
     --disable-malloc-trim) printf "%s" -Dmalloc_trim=disabled ;;
+    --mandir=*) quote_sh "-Dmandir=$2" ;;
     --enable-membarrier) printf "%s" -Dmembarrier=enabled ;;
     --disable-membarrier) printf "%s" -Dmembarrier=disabled ;;
     --enable-module-upgrades) printf "%s" -Dmodule_upgrades=true ;;
@@ -325,6 +344,7 @@ _meson_option_parse() {
     --disable-qcow1) printf "%s" -Dqcow1=disabled ;;
     --enable-qed) printf "%s" -Dqed=enabled ;;
     --disable-qed) printf "%s" -Dqed=disabled ;;
+    --firmwarepath=*) quote_sh "-Dqemu_firmwarepath=$2" ;;
     --enable-qga-vss) printf "%s" -Dqga_vss=enabled ;;
     --disable-qga-vss) printf "%s" -Dqga_vss=disabled ;;
     --enable-qom-cast-debug) printf "%s" -Dqom_cast_debug=true ;;
@@ -363,6 +383,7 @@ _meson_option_parse() {
     --disable-spice-protocol) printf "%s" -Dspice_protocol=disabled ;;
     --enable-strip) printf "%s" -Dstrip=true ;;
     --disable-strip) printf "%s" -Dstrip=false ;;
+    --sysconfdir=*) quote_sh "-Dsysconfdir=$2" ;;
     --enable-tcg) printf "%s" -Dtcg=enabled ;;
     --disable-tcg) printf "%s" -Dtcg=disabled ;;
     --enable-tcg-interpreter) printf "%s" -Dtcg_interpreter=true ;;

From c54b59ee0da90dfb169c05e43ebf0c01a7dd5215 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:33:58 +0200
Subject: [PATCH 111/935] meson: pass more options directly as -D
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

If an option is not used anywhere by the configure script, it can be just
added to $meson_options even if it is not parsed by the automatically
generated bits in scripts/meson-buildoptions.sh.

The only slightly tricky case is $debug, where the

  if test "$fortify_source" = "yes" ; then
    QEMU_CFLAGS="-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 $QEMU_CFLAGS"
    debug=no
  fi

assignment is dead; configure sets fortify_source=no whenever debug=yes.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure                     | 39 ++++++++++-------------------------
 meson.build                   |  2 +-
 scripts/meson-buildoptions.py |  4 ++++
 scripts/meson-buildoptions.sh |  6 ++++++
 4 files changed, 22 insertions(+), 29 deletions(-)

diff --git a/configure b/configure
index b8e3176808..fc7868948b 100755
--- a/configure
+++ b/configure
@@ -235,7 +235,6 @@ static="no"
 cross_compile="no"
 cross_prefix=""
 host_cc="cc"
-lto="false"
 stack_protector=""
 safe_stack=""
 use_containers="yes"
@@ -291,13 +290,10 @@ vhost_vsock="$default_feature"
 vhost_user="no"
 vhost_user_fs="$default_feature"
 vhost_vdpa="$default_feature"
-debug_info="yes"
 debug_tcg="no"
-debug="no"
 sanitizers="no"
 tsan="no"
 fortify_source="yes"
-gcov="no"
 EXESUF=""
 modules="no"
 prefix="/usr/local"
@@ -327,10 +323,7 @@ else
 fi
 fdt="auto"
 
-# 2. Support --with/--without option
-default_devices="true"
-
-# 3. Automatically enable/disable other options
+# 2. Automatically enable/disable other options
 tcg="enabled"
 cfi="false"
 
@@ -680,6 +673,9 @@ werror=""
 . $source_path/scripts/meson-buildoptions.sh
 
 meson_options=
+meson_option_add() {
+  meson_options="$meson_options $(quote_sh "$1")"
+}
 meson_option_parse() {
   meson_options="$meson_options $(_meson_option_parse "$@")"
   if test $? -eq 1; then
@@ -732,9 +728,9 @@ for opt do
   ;;
   --cross-cc-*)
   ;;
-  --enable-debug-info) debug_info="yes"
+  --enable-debug-info) meson_option_add -Ddebug=true
   ;;
-  --disable-debug-info) debug_info="no"
+  --disable-debug-info) meson_option_add -Ddebug=false
   ;;
   --enable-modules)
       modules="yes"
@@ -754,9 +750,9 @@ for opt do
                        error_exit "Can't mix --target-list-exclude with --target-list"
                    fi
   ;;
-  --with-default-devices) default_devices="true"
+  --with-default-devices) meson_option_add -Ddefault_devices=true
   ;;
-  --without-default-devices) default_devices="false"
+  --without-default-devices) meson_option_add -Ddefault_devices=false
   ;;
   --with-devices-*[!a-zA-Z0-9_-]*=*) error_exit "Passed bad --with-devices-FOO option"
   ;;
@@ -772,8 +768,6 @@ for opt do
   ;;
   --without-default-features) # processed above
   ;;
-  --enable-gcov) gcov="yes"
-  ;;
   --static)
     static="yes"
     QEMU_PKG_CONFIG_FLAGS="--static $QEMU_PKG_CONFIG_FLAGS"
@@ -800,7 +794,7 @@ for opt do
       # Enable debugging options that aren't excessively noisy
       debug_tcg="yes"
       meson_option_parse --enable-debug-mutex ""
-      debug="yes"
+      meson_option_add -Doptimization=0
       fortify_source="no"
   ;;
   --enable-sanitizers) sanitizers="yes"
@@ -849,10 +843,6 @@ for opt do
   ;;
   --disable-werror) werror="no"
   ;;
-  --enable-lto) lto="true"
-  ;;
-  --disable-lto) lto="false"
-  ;;
   --enable-stack-protector) stack_protector="yes"
   ;;
   --disable-stack-protector) stack_protector="no"
@@ -863,7 +853,7 @@ for opt do
   ;;
   --enable-cfi)
       cfi="true";
-      lto="true";
+      meson_option_add -Db_lto=true
   ;;
   --disable-cfi) cfi="false"
   ;;
@@ -1101,7 +1091,6 @@ Advanced options (experts only):
   --cpu=CPU                Build for host CPU [$cpu]
   --with-coroutine=BACKEND coroutine backend. Supported options:
                            ucontext, sigaltstack, windows
-  --enable-gcov            enable test coverage analysis with gcov
   --enable-plugins
                            enable plugins via shared library loading
   --disable-containers     don't use containers for cross-building
@@ -1117,7 +1106,6 @@ cat << EOF
   modules         modules support (non-Windows)
   debug-tcg       TCG debugging (default is disabled)
   debug-info      debugging information
-  lto             Enable Link-Time Optimization.
   safe-stack      SafeStack Stack Smash Protection. Depends on
                   clang/llvm >= 3.7 and requires coroutine backend ucontext.
   vhost-net       vhost-net kernel acceleration support
@@ -1938,7 +1926,6 @@ write_c_skeleton
 
 if test "$fortify_source" = "yes" ; then
   QEMU_CFLAGS="-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 $QEMU_CFLAGS"
-  debug=no
 fi
 
 case "$ARCH" in
@@ -2367,15 +2354,11 @@ if test "$skip_meson" = no; then
     NINJA=$ninja $meson setup \
         --prefix "$prefix" \
         --bindir "$bindir" \
-        -Ddefault_devices=$default_devices \
         -Dqemu_suffix="$qemu_suffix" \
         -Dsmbd="$smbd" \
-        -Doptimization=$(if test "$debug" = yes; then echo 0; else echo 2; fi) \
-        -Ddebug=$(if test "$debug_info" = yes; then echo true; else echo false; fi) \
         -Dwerror=$(if test "$werror" = yes; then echo true; else echo false; fi) \
         -Db_pie=$(if test "$pie" = yes; then echo true; else echo false; fi) \
-        -Db_coverage=$(if test "$gcov" = yes; then echo true; else echo false; fi) \
-        -Db_lto=$lto -Dcfi=$cfi -Dtcg=$tcg \
+        -Dcfi=$cfi -Dtcg=$tcg \
         -Dcapstone=$capstone -Dfdt=$fdt -Dslirp=$slirp \
         $(test -n "${LIB_FUZZING_ENGINE+xxx}" && echo "-Dfuzzing_engine=$LIB_FUZZING_ENGINE") \
         $(if test "$default_feature" = no; then echo "-Dauto_features=disabled"; fi) \
diff --git a/meson.build b/meson.build
index 65c44ea010..44f3ca3218 100644
--- a/meson.build
+++ b/meson.build
@@ -1,6 +1,6 @@
 project('qemu', ['c'], meson_version: '>=0.59.3',
         default_options: ['warning_level=1', 'c_std=gnu11', 'cpp_std=gnu++11', 'b_colorout=auto',
-                          'b_staticpic=false', 'stdsplit=false'],
+                          'b_staticpic=false', 'stdsplit=false', 'optimization=2'],
         version: files('VERSION'))
 
 add_test_setup('quick', exclude_suites: ['slow', 'thorough'], is_default: true)
diff --git a/scripts/meson-buildoptions.py b/scripts/meson-buildoptions.py
index 45cda8cd84..e624c16b01 100755
--- a/scripts/meson-buildoptions.py
+++ b/scripts/meson-buildoptions.py
@@ -33,6 +33,8 @@ SKIP_OPTIONS = {
 }
 
 OPTION_NAMES = {
+    "b_coverage": "gcov",
+    "b_lto": "lto",
     "malloc": "enable-malloc",
     "pkgversion": "with-pkgversion",
     "qemu_firmwarepath": "firmwarepath",
@@ -41,6 +43,8 @@ OPTION_NAMES = {
 }
 
 BUILTIN_OPTIONS = {
+    "b_coverage",
+    "b_lto",
     "datadir",
     "includedir",
     "libdir",
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index f6a9b05c40..4b0770a9e0 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -27,7 +27,9 @@ meson_options_help() {
   printf "%s\n" '  --enable-fdt[=CHOICE]    Whether and how to find the libfdt library'
   printf "%s\n" '                           (choices: auto/disabled/enabled/internal/system)'
   printf "%s\n" '  --enable-fuzzing         build fuzzing targets'
+  printf "%s\n" '  --enable-gcov            Enable coverage tracking.'
   printf "%s\n" '  --enable-gprof           QEMU profiling with gprof'
+  printf "%s\n" '  --enable-lto             Use link time optimization'
   printf "%s\n" '  --enable-malloc=CHOICE   choose memory allocator to use [system] (choices:'
   printf "%s\n" '                           jemalloc/system/tcmalloc)'
   printf "%s\n" '  --enable-module-upgrades try to load modules from alternate paths for'
@@ -182,6 +184,10 @@ _meson_option_parse() {
     --disable-avx2) printf "%s" -Davx2=disabled ;;
     --enable-avx512f) printf "%s" -Davx512f=enabled ;;
     --disable-avx512f) printf "%s" -Davx512f=disabled ;;
+    --enable-gcov) printf "%s" -Db_coverage=true ;;
+    --disable-gcov) printf "%s" -Db_coverage=false ;;
+    --enable-lto) printf "%s" -Db_lto=true ;;
+    --disable-lto) printf "%s" -Db_lto=false ;;
     --block-drv-ro-whitelist=*) quote_sh "-Dblock_drv_ro_whitelist=$2" ;;
     --block-drv-rw-whitelist=*) quote_sh "-Dblock_drv_rw_whitelist=$2" ;;
     --enable-block-drv-whitelist-in-tools) printf "%s" -Dblock_drv_whitelist_in_tools=true ;;

From 0a31e3a0a00f7cd3d1d7e8885a9b1f979fed3c20 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:33:59 +0200
Subject: [PATCH 112/935] configure: omit options with default values from
 meson command line
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This has no functional change, it only makes the command line shorter
and nicer.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure         | 29 +++++++++++++++++------------
 meson.build       |  2 +-
 meson_options.txt |  2 +-
 3 files changed, 19 insertions(+), 14 deletions(-)

diff --git a/configure b/configure
index fc7868948b..1c88e697c5 100755
--- a/configure
+++ b/configure
@@ -2350,19 +2350,24 @@ if test "$skip_meson" = no; then
   mv $cross config-meson.cross
 
   rm -rf meson-private meson-info meson-logs
+
+  # Built-in options
+  test "$bindir" != "bin" && meson_option_add "-Dbindir=$bindir"
+  test "$default_feature" = no && meson_option_add -Dauto_features=disabled
+  test "$pie" = no && meson_option_add -Db_pie=false
+  test "$werror" = yes && meson_option_add -Dwerror=true
+
+  # QEMU options
+  test "$capstone" != auto && meson_option_add "-Dcapstone=$capstone"
+  test "$cfi" != false && meson_option_add "-Dcfi=$cfi"
+  test "$fdt" != auto && meson_option_add "-Dfdt=$fdt"
+  test -n "${LIB_FUZZING_ENGINE+xxx}" && meson_option_add "-Dfuzzing_engine=$LIB_FUZZING_ENGINE"
+  test "$qemu_suffix" != qemu && meson_option_add "-Dqemu_suffix=$qemu_suffix"
+  test "$slirp" != auto && meson_option_add "-Dslirp=$slirp"
+  test "$smbd" != '' && meson_option_add "-Dsmbd=$smbd"
+  test "$tcg" != enabled && meson_option_add "-Dtcg=$tcg"
   run_meson() {
-    NINJA=$ninja $meson setup \
-        --prefix "$prefix" \
-        --bindir "$bindir" \
-        -Dqemu_suffix="$qemu_suffix" \
-        -Dsmbd="$smbd" \
-        -Dwerror=$(if test "$werror" = yes; then echo true; else echo false; fi) \
-        -Db_pie=$(if test "$pie" = yes; then echo true; else echo false; fi) \
-        -Dcfi=$cfi -Dtcg=$tcg \
-        -Dcapstone=$capstone -Dfdt=$fdt -Dslirp=$slirp \
-        $(test -n "${LIB_FUZZING_ENGINE+xxx}" && echo "-Dfuzzing_engine=$LIB_FUZZING_ENGINE") \
-        $(if test "$default_feature" = no; then echo "-Dauto_features=disabled"; fi) \
-        "$@" $cross_arg "$PWD" "$source_path"
+    NINJA=$ninja $meson setup --prefix "$prefix" "$@" $cross_arg "$PWD" "$source_path"
   }
   eval run_meson $meson_options
   if test "$?" -ne 0 ; then
diff --git a/meson.build b/meson.build
index 44f3ca3218..215be030d1 100644
--- a/meson.build
+++ b/meson.build
@@ -1,6 +1,6 @@
 project('qemu', ['c'], meson_version: '>=0.59.3',
         default_options: ['warning_level=1', 'c_std=gnu11', 'cpp_std=gnu++11', 'b_colorout=auto',
-                          'b_staticpic=false', 'stdsplit=false', 'optimization=2'],
+                          'b_staticpic=false', 'stdsplit=false', 'optimization=2', 'b_pie=true'],
         version: files('VERSION'))
 
 add_test_setup('quick', exclude_suites: ['slow', 'thorough'], is_default: true)
diff --git a/meson_options.txt b/meson_options.txt
index 15fe40c700..35f5a72507 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -78,7 +78,7 @@ option('xen', type: 'feature', value: 'auto',
        description: 'Xen backend support')
 option('xen_pci_passthrough', type: 'feature', value: 'auto',
        description: 'Xen PCI passthrough support')
-option('tcg', type: 'feature', value: 'auto',
+option('tcg', type: 'feature', value: 'enabled',
        description: 'TCG support')
 option('tcg_interpreter', type: 'boolean', value: false,
        description: 'TCG with bytecode interpreter (slow)')

From 10925d06173e076ed80f3108a8a9ed15a8b5b4ab Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:34:00 +0200
Subject: [PATCH 113/935] meson, virtio: place all virtio-pci devices under
 virtio_pci_ss
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Since a sourceset already exists for this, avoid unnecessary repeat
of CONFIG_VIRTIO_PCI.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/virtio/meson.build | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/hw/virtio/meson.build b/hw/virtio/meson.build
index 67dc77e00f..f371404b04 100644
--- a/hw/virtio/meson.build
+++ b/hw/virtio/meson.build
@@ -16,9 +16,7 @@ virtio_ss.add(when: 'CONFIG_VHOST_USER', if_true: files('vhost-user.c'))
 virtio_ss.add(when: 'CONFIG_VHOST_VDPA', if_true: files('vhost-shadow-virtqueue.c', 'vhost-vdpa.c'))
 virtio_ss.add(when: 'CONFIG_VIRTIO_BALLOON', if_true: files('virtio-balloon.c'))
 virtio_ss.add(when: 'CONFIG_VIRTIO_CRYPTO', if_true: files('virtio-crypto.c'))
-virtio_ss.add(when: ['CONFIG_VIRTIO_CRYPTO', 'CONFIG_VIRTIO_PCI'], if_true: files('virtio-crypto-pci.c'))
 virtio_ss.add(when: 'CONFIG_VHOST_USER_FS', if_true: files('vhost-user-fs.c'))
-virtio_ss.add(when: ['CONFIG_VHOST_USER_FS', 'CONFIG_VIRTIO_PCI'], if_true: files('vhost-user-fs-pci.c'))
 virtio_ss.add(when: 'CONFIG_VIRTIO_PMEM', if_true: files('virtio-pmem.c'))
 virtio_ss.add(when: 'CONFIG_VHOST_VSOCK', if_true: files('vhost-vsock.c', 'vhost-vsock-common.c'))
 virtio_ss.add(when: 'CONFIG_VHOST_USER_VSOCK', if_true: files('vhost-user-vsock.c', 'vhost-vsock-common.c'))
@@ -26,17 +24,20 @@ virtio_ss.add(when: 'CONFIG_VIRTIO_RNG', if_true: files('virtio-rng.c'))
 virtio_ss.add(when: 'CONFIG_VIRTIO_IOMMU', if_true: files('virtio-iommu.c'))
 virtio_ss.add(when: 'CONFIG_VIRTIO_MEM', if_true: files('virtio-mem.c'))
 virtio_ss.add(when: 'CONFIG_VHOST_USER_I2C', if_true: files('vhost-user-i2c.c'))
-virtio_ss.add(when: ['CONFIG_VIRTIO_PCI', 'CONFIG_VHOST_USER_I2C'], if_true: files('vhost-user-i2c-pci.c'))
 virtio_ss.add(when: 'CONFIG_VHOST_USER_RNG', if_true: files('vhost-user-rng.c'))
-virtio_ss.add(when: ['CONFIG_VHOST_USER_RNG', 'CONFIG_VIRTIO_PCI'], if_true: files('vhost-user-rng-pci.c'))
 
 virtio_pci_ss = ss.source_set()
 virtio_pci_ss.add(when: 'CONFIG_VHOST_VSOCK', if_true: files('vhost-vsock-pci.c'))
 virtio_pci_ss.add(when: 'CONFIG_VHOST_USER_VSOCK', if_true: files('vhost-user-vsock-pci.c'))
 virtio_pci_ss.add(when: 'CONFIG_VHOST_USER_BLK', if_true: files('vhost-user-blk-pci.c'))
+virtio_pci_ss.add(when: 'CONFIG_VHOST_USER_I2C', if_true: files('vhost-user-i2c-pci.c'))
 virtio_pci_ss.add(when: 'CONFIG_VHOST_USER_INPUT', if_true: files('vhost-user-input-pci.c'))
+virtio_pci_ss.add(when: 'CONFIG_VHOST_USER_RNG', if_true: files('vhost-user-rng-pci.c'))
 virtio_pci_ss.add(when: 'CONFIG_VHOST_USER_SCSI', if_true: files('vhost-user-scsi-pci.c'))
 virtio_pci_ss.add(when: 'CONFIG_VHOST_SCSI', if_true: files('vhost-scsi-pci.c'))
+virtio_pci_ss.add(when: 'CONFIG_VHOST_USER_FS', if_true: files('vhost-user-fs-pci.c'))
+
+virtio_pci_ss.add(when: 'CONFIG_VIRTIO_CRYPTO', if_true: files('virtio-crypto-pci.c'))
 virtio_pci_ss.add(when: 'CONFIG_VIRTIO_INPUT_HOST', if_true: files('virtio-input-host-pci.c'))
 virtio_pci_ss.add(when: 'CONFIG_VIRTIO_INPUT', if_true: files('virtio-input-pci.c'))
 virtio_pci_ss.add(when: 'CONFIG_VIRTIO_RNG', if_true: files('virtio-rng-pci.c'))

From 5e7b5ebaec4c2bcc2754ab1e8ac5a05b88fef073 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:34:01 +0200
Subject: [PATCH 114/935] configure: simplify vhost-net-{user, vdpa}
 configuration
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Cleanup to ease review of the conversion to meson.  vhost_net_user and
vhost_net_vdpa are never assigned anything in the command line parsing
loop, so they are always equal to $vhost_user and $vhost_vdpa.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure | 17 ++++-------------
 1 file changed, 4 insertions(+), 13 deletions(-)

diff --git a/configure b/configure
index 1c88e697c5..d6ddd4f05f 100755
--- a/configure
+++ b/configure
@@ -1555,10 +1555,6 @@ if test "$vhost_vsock" = "yes" && test "$vhost_kernel" != "yes"; then
 fi
 
 # vhost-user backends
-test "$vhost_net_user" = "" && vhost_net_user=$vhost_user
-if test "$vhost_net_user" = "yes" && test "$vhost_user" = "no"; then
-  error_exit "--enable-vhost-net-user requires --enable-vhost-user"
-fi
 test "$vhost_crypto" = "" && vhost_crypto=$vhost_user
 if test "$vhost_crypto" = "yes" && test "$vhost_user" = "no"; then
   error_exit "--enable-vhost-crypto requires --enable-vhost-user"
@@ -1567,16 +1563,11 @@ test "$vhost_user_fs" = "" && vhost_user_fs=$vhost_user
 if test "$vhost_user_fs" = "yes" && test "$vhost_user" = "no"; then
   error_exit "--enable-vhost-user-fs requires --enable-vhost-user"
 fi
-#vhost-vdpa backends
-test "$vhost_net_vdpa" = "" && vhost_net_vdpa=$vhost_vdpa
-if test "$vhost_net_vdpa" = "yes" && test "$vhost_vdpa" = "no"; then
-  error_exit "--enable-vhost-net-vdpa requires --enable-vhost-vdpa"
-fi
 
 # OR the vhost-kernel, vhost-vdpa and vhost-user values for simplicity
 if test "$vhost_net" = ""; then
-  test "$vhost_net_user" = "yes" && vhost_net=yes
-  test "$vhost_net_vdpa" = "yes" && vhost_net=yes
+  test "$vhost_user" = "yes" && vhost_net=yes
+  test "$vhost_vdpa" = "yes" && vhost_net=yes
   test "$vhost_kernel" = "yes" && vhost_net=yes
 fi
 
@@ -2105,10 +2096,10 @@ fi
 if test "$vhost_net" = "yes" ; then
   echo "CONFIG_VHOST_NET=y" >> $config_host_mak
 fi
-if test "$vhost_net_user" = "yes" ; then
+if test "$vhost_user" = "yes" ; then
   echo "CONFIG_VHOST_NET_USER=y" >> $config_host_mak
 fi
-if test "$vhost_net_vdpa" = "yes" ; then
+if test "$vhost_vdpa" = "yes" ; then
   echo "CONFIG_VHOST_NET_VDPA=y" >> $config_host_mak
 fi
 if test "$vhost_crypto" = "yes" ; then

From 9972ae314fc38b7e47ee7ccdbc8556b7cad8a930 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:34:02 +0200
Subject: [PATCH 115/935] build: move vhost-vsock configuration to Kconfig
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

vhost-vsock and vhost-user-vsock are two devices of their own; it should
be possible to enable/disable them with --without-default-devices, not
--without-default-features.  Compute their default value in Kconfig to
obtain the more intuitive behavior.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure                                       | 16 ----------------
 hw/virtio/Kconfig                               | 10 ++++++++++
 meson.build                                     |  1 -
 scripts/ci/org.centos/stream/8/x86_64/configure |  2 --
 4 files changed, 10 insertions(+), 19 deletions(-)

diff --git a/configure b/configure
index d6ddd4f05f..603434fc7f 100755
--- a/configure
+++ b/configure
@@ -286,7 +286,6 @@ vhost_kernel="$default_feature"
 vhost_net="$default_feature"
 vhost_crypto="$default_feature"
 vhost_scsi="$default_feature"
-vhost_vsock="$default_feature"
 vhost_user="no"
 vhost_user_fs="$default_feature"
 vhost_vdpa="$default_feature"
@@ -879,10 +878,6 @@ for opt do
   ;;
   --enable-vhost-scsi) vhost_scsi="yes"
   ;;
-  --disable-vhost-vsock) vhost_vsock="no"
-  ;;
-  --enable-vhost-vsock) vhost_vsock="yes"
-  ;;
   --disable-vhost-user-fs) vhost_user_fs="no"
   ;;
   --enable-vhost-user-fs) vhost_user_fs="yes"
@@ -1109,7 +1104,6 @@ cat << EOF
   safe-stack      SafeStack Stack Smash Protection. Depends on
                   clang/llvm >= 3.7 and requires coroutine backend ucontext.
   vhost-net       vhost-net kernel acceleration support
-  vhost-vsock     virtio sockets device support
   vhost-scsi      vhost-scsi kernel target support
   vhost-crypto    vhost-user-crypto backend support
   vhost-kernel    vhost kernel backend support
@@ -1549,10 +1543,6 @@ test "$vhost_scsi" = "" && vhost_scsi=$vhost_kernel
 if test "$vhost_scsi" = "yes" && test "$vhost_kernel" != "yes"; then
   error_exit "--enable-vhost-scsi requires --enable-vhost-kernel"
 fi
-test "$vhost_vsock" = "" && vhost_vsock=$vhost_kernel
-if test "$vhost_vsock" = "yes" && test "$vhost_kernel" != "yes"; then
-  error_exit "--enable-vhost-vsock requires --enable-vhost-kernel"
-fi
 
 # vhost-user backends
 test "$vhost_crypto" = "" && vhost_crypto=$vhost_user
@@ -2105,12 +2095,6 @@ fi
 if test "$vhost_crypto" = "yes" ; then
   echo "CONFIG_VHOST_CRYPTO=y" >> $config_host_mak
 fi
-if test "$vhost_vsock" = "yes" ; then
-  echo "CONFIG_VHOST_VSOCK=y" >> $config_host_mak
-  if test "$vhost_user" = "yes" ; then
-    echo "CONFIG_VHOST_USER_VSOCK=y" >> $config_host_mak
-  fi
-fi
 if test "$vhost_kernel" = "yes" ; then
   echo "CONFIG_VHOST_KERNEL=y" >> $config_host_mak
 fi
diff --git a/hw/virtio/Kconfig b/hw/virtio/Kconfig
index c144d42f9b..b642ae1081 100644
--- a/hw/virtio/Kconfig
+++ b/hw/virtio/Kconfig
@@ -59,6 +59,16 @@ config VIRTIO_MEM
     depends on VIRTIO_MEM_SUPPORTED
     select MEM_DEVICE
 
+config VHOST_VSOCK
+    bool
+    default y
+    depends on VIRTIO && VHOST_KERNEL
+
+config VHOST_USER_VSOCK
+    bool
+    default y
+    depends on VIRTIO && VHOST_USER
+
 config VHOST_USER_I2C
     bool
     default y
diff --git a/meson.build b/meson.build
index 215be030d1..fd7d529a8e 100644
--- a/meson.build
+++ b/meson.build
@@ -3692,7 +3692,6 @@ summary_info += {'vhost-kernel support': config_host.has_key('CONFIG_VHOST_KERNE
 summary_info += {'vhost-net support': config_host.has_key('CONFIG_VHOST_NET')}
 summary_info += {'vhost-crypto support': config_host.has_key('CONFIG_VHOST_CRYPTO')}
 summary_info += {'vhost-scsi support': config_host.has_key('CONFIG_VHOST_SCSI')}
-summary_info += {'vhost-vsock support': config_host.has_key('CONFIG_VHOST_VSOCK')}
 summary_info += {'vhost-user support': config_host.has_key('CONFIG_VHOST_USER')}
 summary_info += {'vhost-user-blk server support': have_vhost_user_blk_server}
 summary_info += {'vhost-user-fs support': config_host.has_key('CONFIG_VHOST_USER_FS')}
diff --git a/scripts/ci/org.centos/stream/8/x86_64/configure b/scripts/ci/org.centos/stream/8/x86_64/configure
index 08225ee514..e6ac2e5877 100755
--- a/scripts/ci/org.centos/stream/8/x86_64/configure
+++ b/scripts/ci/org.centos/stream/8/x86_64/configure
@@ -136,7 +136,6 @@
 --disable-vhost-user \
 --disable-vhost-user-blk-server \
 --disable-vhost-vdpa \
---disable-vhost-vsock \
 --disable-virglrenderer \
 --disable-virtfs \
 --disable-virtiofsd \
@@ -198,7 +197,6 @@
 --enable-vhost-user \
 --enable-vhost-user-blk-server \
 --enable-vhost-vdpa \
---enable-vhost-vsock \
 --enable-vnc \
 --enable-png \
 --enable-vnc-sasl \

From d13b200253cbd9cb933a1cfead9106abd564ae4a Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:34:03 +0200
Subject: [PATCH 116/935] build: move vhost-scsi configuration to Kconfig
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

vhost-scsi and vhost-user-scsi are two devices of their own; it should
be possible to enable/disable them with --without-default-devices, not
--without-default-features.  Compute their default value in Kconfig to
obtain the more intuitive behavior.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure                                       | 15 ---------------
 hw/scsi/Kconfig                                 |  5 +++++
 include/hw/virtio/virtio-scsi.h                 |  2 --
 meson.build                                     |  1 -
 scripts/ci/org.centos/stream/8/x86_64/configure |  1 -
 5 files changed, 5 insertions(+), 19 deletions(-)

diff --git a/configure b/configure
index 603434fc7f..d268a9cf49 100755
--- a/configure
+++ b/configure
@@ -285,7 +285,6 @@ EXTRA_LDFLAGS=""
 vhost_kernel="$default_feature"
 vhost_net="$default_feature"
 vhost_crypto="$default_feature"
-vhost_scsi="$default_feature"
 vhost_user="no"
 vhost_user_fs="$default_feature"
 vhost_vdpa="$default_feature"
@@ -874,10 +873,6 @@ for opt do
   ;;
   --enable-vhost-crypto) vhost_crypto="yes"
   ;;
-  --disable-vhost-scsi) vhost_scsi="no"
-  ;;
-  --enable-vhost-scsi) vhost_scsi="yes"
-  ;;
   --disable-vhost-user-fs) vhost_user_fs="no"
   ;;
   --enable-vhost-user-fs) vhost_user_fs="yes"
@@ -1104,7 +1099,6 @@ cat << EOF
   safe-stack      SafeStack Stack Smash Protection. Depends on
                   clang/llvm >= 3.7 and requires coroutine backend ucontext.
   vhost-net       vhost-net kernel acceleration support
-  vhost-scsi      vhost-scsi kernel target support
   vhost-crypto    vhost-user-crypto backend support
   vhost-kernel    vhost kernel backend support
   vhost-user      vhost-user backend support
@@ -1538,12 +1532,6 @@ if test "$vhost_kernel" = "yes" && test "$linux" != "yes"; then
   error_exit "vhost-kernel is only available on Linux"
 fi
 
-# vhost-kernel devices
-test "$vhost_scsi" = "" && vhost_scsi=$vhost_kernel
-if test "$vhost_scsi" = "yes" && test "$vhost_kernel" != "yes"; then
-  error_exit "--enable-vhost-scsi requires --enable-vhost-kernel"
-fi
-
 # vhost-user backends
 test "$vhost_crypto" = "" && vhost_crypto=$vhost_user
 if test "$vhost_crypto" = "yes" && test "$vhost_user" = "no"; then
@@ -2080,9 +2068,6 @@ if test "$modules" = "yes"; then
   echo "CONFIG_MODULES=y" >> $config_host_mak
 fi
 
-if test "$vhost_scsi" = "yes" ; then
-  echo "CONFIG_VHOST_SCSI=y" >> $config_host_mak
-fi
 if test "$vhost_net" = "yes" ; then
   echo "CONFIG_VHOST_NET=y" >> $config_host_mak
 fi
diff --git a/hw/scsi/Kconfig b/hw/scsi/Kconfig
index 77d397c949..e7b34dc8e2 100644
--- a/hw/scsi/Kconfig
+++ b/hw/scsi/Kconfig
@@ -48,6 +48,11 @@ config VIRTIO_SCSI
     depends on VIRTIO
     select SCSI
 
+config VHOST_SCSI
+    bool
+    default y
+    depends on VIRTIO && VHOST_KERNEL
+
 config VHOST_USER_SCSI
     bool
     # Only PCI devices are provided for now
diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scsi.h
index 543681bc18..0997313f0a 100644
--- a/include/hw/virtio/virtio-scsi.h
+++ b/include/hw/virtio/virtio-scsi.h
@@ -55,10 +55,8 @@ struct VirtIOSCSIConf {
     bool seg_max_adjust;
     uint32_t max_sectors;
     uint32_t cmd_per_lun;
-#ifdef CONFIG_VHOST_SCSI
     char *vhostfd;
     char *wwpn;
-#endif
     CharBackend chardev;
     uint32_t boot_tpgt;
     IOThread *iothread;
diff --git a/meson.build b/meson.build
index fd7d529a8e..d56dd70cb7 100644
--- a/meson.build
+++ b/meson.build
@@ -3691,7 +3691,6 @@ summary_info += {'QOM debugging':     get_option('qom_cast_debug')}
 summary_info += {'vhost-kernel support': config_host.has_key('CONFIG_VHOST_KERNEL')}
 summary_info += {'vhost-net support': config_host.has_key('CONFIG_VHOST_NET')}
 summary_info += {'vhost-crypto support': config_host.has_key('CONFIG_VHOST_CRYPTO')}
-summary_info += {'vhost-scsi support': config_host.has_key('CONFIG_VHOST_SCSI')}
 summary_info += {'vhost-user support': config_host.has_key('CONFIG_VHOST_USER')}
 summary_info += {'vhost-user-blk server support': have_vhost_user_blk_server}
 summary_info += {'vhost-user-fs support': config_host.has_key('CONFIG_VHOST_USER_FS')}
diff --git a/scripts/ci/org.centos/stream/8/x86_64/configure b/scripts/ci/org.centos/stream/8/x86_64/configure
index e6ac2e5877..a7f92aff90 100755
--- a/scripts/ci/org.centos/stream/8/x86_64/configure
+++ b/scripts/ci/org.centos/stream/8/x86_64/configure
@@ -132,7 +132,6 @@
 --disable-vhost-crypto \
 --disable-vhost-kernel \
 --disable-vhost-net \
---disable-vhost-scsi \
 --disable-vhost-user \
 --disable-vhost-user-blk-server \
 --disable-vhost-vdpa \

From 5166dabf4fe4c33e9595b0aaf2c03ccf411199e2 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:34:04 +0200
Subject: [PATCH 117/935] build: move vhost-user-fs configuration to Kconfig
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

vhost-user-fs is a device and it should be possible to enable/disable
it with --without-default-devices, not --without-default-features.
Compute its default value in Kconfig to obtain the more intuitive
behavior.

In this case the configure options were undocumented, too.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure         | 12 ------------
 hw/virtio/Kconfig |  5 +++++
 meson.build       |  1 -
 3 files changed, 5 insertions(+), 13 deletions(-)

diff --git a/configure b/configure
index d268a9cf49..9a83d64c9d 100755
--- a/configure
+++ b/configure
@@ -286,7 +286,6 @@ vhost_kernel="$default_feature"
 vhost_net="$default_feature"
 vhost_crypto="$default_feature"
 vhost_user="no"
-vhost_user_fs="$default_feature"
 vhost_vdpa="$default_feature"
 debug_tcg="no"
 sanitizers="no"
@@ -873,10 +872,6 @@ for opt do
   ;;
   --enable-vhost-crypto) vhost_crypto="yes"
   ;;
-  --disable-vhost-user-fs) vhost_user_fs="no"
-  ;;
-  --enable-vhost-user-fs) vhost_user_fs="yes"
-  ;;
   --disable-zlib-test)
   ;;
   --disable-virtio-blk-data-plane|--enable-virtio-blk-data-plane)
@@ -1537,10 +1532,6 @@ test "$vhost_crypto" = "" && vhost_crypto=$vhost_user
 if test "$vhost_crypto" = "yes" && test "$vhost_user" = "no"; then
   error_exit "--enable-vhost-crypto requires --enable-vhost-user"
 fi
-test "$vhost_user_fs" = "" && vhost_user_fs=$vhost_user
-if test "$vhost_user_fs" = "yes" && test "$vhost_user" = "no"; then
-  error_exit "--enable-vhost-user-fs requires --enable-vhost-user"
-fi
 
 # OR the vhost-kernel, vhost-vdpa and vhost-user values for simplicity
 if test "$vhost_net" = ""; then
@@ -2089,9 +2080,6 @@ fi
 if test "$vhost_vdpa" = "yes" ; then
   echo "CONFIG_VHOST_VDPA=y" >> $config_host_mak
 fi
-if test "$vhost_user_fs" = "yes" ; then
-  echo "CONFIG_VHOST_USER_FS=y" >> $config_host_mak
-fi
 
 # XXX: suppress that
 if [ "$bsd" = "yes" ] ; then
diff --git a/hw/virtio/Kconfig b/hw/virtio/Kconfig
index b642ae1081..f8e235f814 100644
--- a/hw/virtio/Kconfig
+++ b/hw/virtio/Kconfig
@@ -78,3 +78,8 @@ config VHOST_USER_RNG
     bool
     default y
     depends on VIRTIO && VHOST_USER
+
+config VHOST_USER_FS
+    bool
+    default y
+    depends on VIRTIO && VHOST_USER
diff --git a/meson.build b/meson.build
index d56dd70cb7..aba7167544 100644
--- a/meson.build
+++ b/meson.build
@@ -3693,7 +3693,6 @@ summary_info += {'vhost-net support': config_host.has_key('CONFIG_VHOST_NET')}
 summary_info += {'vhost-crypto support': config_host.has_key('CONFIG_VHOST_CRYPTO')}
 summary_info += {'vhost-user support': config_host.has_key('CONFIG_VHOST_USER')}
 summary_info += {'vhost-user-blk server support': have_vhost_user_blk_server}
-summary_info += {'vhost-user-fs support': config_host.has_key('CONFIG_VHOST_USER_FS')}
 summary_info += {'vhost-vdpa support': config_host.has_key('CONFIG_VHOST_VDPA')}
 summary_info += {'build guest agent': have_ga}
 summary(summary_info, bool_yn: true, section: 'Configurable features')

From 2a3129a37652e5e81d12f6e16dd3c447f09831f9 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:34:05 +0200
Subject: [PATCH 118/935] meson: create have_vhost_* variables
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

When using Meson options rather than config-host.h, the "when" clauses
have to be changed to if statements (which is not necessarily great,
though at least it highlights which parts of the build are per-target
and which are not).

Do that before moving vhost logic to meson.build, though for now
the variables are just based on config-host.mak data.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 meson.build       | 31 ++++++++++++++++++++-----------
 tests/meson.build |  2 +-
 tools/meson.build |  2 +-
 3 files changed, 22 insertions(+), 13 deletions(-)

diff --git a/meson.build b/meson.build
index aba7167544..108292e50f 100644
--- a/meson.build
+++ b/meson.build
@@ -312,6 +312,15 @@ have_tpm = get_option('tpm') \
   .require(targetos != 'windows', error_message: 'TPM emulation only available on POSIX systems') \
   .allowed()
 
+# vhost
+have_vhost_user = 'CONFIG_VHOST_USER' in config_host
+have_vhost_vdpa = 'CONFIG_VHOST_VDPA' in config_host
+have_vhost_kernel = 'CONFIG_VHOST_KERNEL' in config_host
+have_vhost_net_user = 'CONFIG_VHOST_NET_USER' in config_host
+have_vhost_net_vdpa = 'CONFIG_VHOST_NET_VDPA' in config_host
+have_vhost_net = 'CONFIG_VHOST_NET' in config_host
+have_vhost_user_crypto = 'CONFIG_VHOST_CRYPTO' in config_host
+
 # Target-specific libraries and flags
 libm = cc.find_library('m', required: false)
 threads = dependency('threads')
@@ -1442,7 +1451,7 @@ has_statx_mnt_id = cc.links(statx_mnt_id_test)
 have_vhost_user_blk_server = get_option('vhost_user_blk_server') \
   .require(targetos == 'linux',
            error_message: 'vhost_user_blk_server requires linux') \
-  .require('CONFIG_VHOST_USER' in config_host,
+  .require(have_vhost_user,
            error_message: 'vhost_user_blk_server requires vhost-user support') \
   .disable_auto_if(not have_system) \
   .allowed()
@@ -2297,9 +2306,9 @@ host_kconfig = \
   (have_ivshmem ? ['CONFIG_IVSHMEM=y'] : []) + \
   (opengl.found() ? ['CONFIG_OPENGL=y'] : []) + \
   (x11.found() ? ['CONFIG_X11=y'] : []) + \
-  ('CONFIG_VHOST_USER' in config_host ? ['CONFIG_VHOST_USER=y'] : []) + \
-  ('CONFIG_VHOST_VDPA' in config_host ? ['CONFIG_VHOST_VDPA=y'] : []) + \
-  ('CONFIG_VHOST_KERNEL' in config_host ? ['CONFIG_VHOST_KERNEL=y'] : []) + \
+  (have_vhost_user ? ['CONFIG_VHOST_USER=y'] : []) + \
+  (have_vhost_vdpa ? ['CONFIG_VHOST_VDPA=y'] : []) + \
+  (have_vhost_kernel ? ['CONFIG_VHOST_KERNEL=y'] : []) + \
   (have_virtfs ? ['CONFIG_VIRTFS=y'] : []) + \
   ('CONFIG_LINUX' in config_host ? ['CONFIG_LINUX=y'] : []) + \
   (have_pvrdma ? ['CONFIG_PVRDMA=y'] : []) + \
@@ -2981,7 +2990,7 @@ if have_system or have_user
 endif
 
 vhost_user = not_found
-if targetos == 'linux' and 'CONFIG_VHOST_USER' in config_host
+if targetos == 'linux' and have_vhost_user
   libvhost_user = subproject('libvhost-user')
   vhost_user = libvhost_user.get_variable('vhost_user_dep')
 endif
@@ -3562,7 +3571,7 @@ if have_tools
              dependencies: qemuutil,
              install: true)
 
-  if 'CONFIG_VHOST_USER' in config_host
+  if have_vhost_user
     subdir('contrib/vhost-user-blk')
     subdir('contrib/vhost-user-gpu')
     subdir('contrib/vhost-user-input')
@@ -3688,12 +3697,12 @@ if 'simple' in get_option('trace_backends')
 endif
 summary_info += {'D-Bus display':     dbus_display}
 summary_info += {'QOM debugging':     get_option('qom_cast_debug')}
-summary_info += {'vhost-kernel support': config_host.has_key('CONFIG_VHOST_KERNEL')}
-summary_info += {'vhost-net support': config_host.has_key('CONFIG_VHOST_NET')}
-summary_info += {'vhost-crypto support': config_host.has_key('CONFIG_VHOST_CRYPTO')}
-summary_info += {'vhost-user support': config_host.has_key('CONFIG_VHOST_USER')}
+summary_info += {'vhost-kernel support': have_vhost_kernel}
+summary_info += {'vhost-net support': have_vhost_net}
+summary_info += {'vhost-user support': have_vhost_user}
+summary_info += {'vhost-user-crypto support': have_vhost_user_crypto}
 summary_info += {'vhost-user-blk server support': have_vhost_user_blk_server}
-summary_info += {'vhost-vdpa support': config_host.has_key('CONFIG_VHOST_VDPA')}
+summary_info += {'vhost-vdpa support': have_vhost_vdpa}
 summary_info += {'build guest agent': have_ga}
 summary(summary_info, bool_yn: true, section: 'Configurable features')
 
diff --git a/tests/meson.build b/tests/meson.build
index 4f691e8465..8e318ec513 100644
--- a/tests/meson.build
+++ b/tests/meson.build
@@ -68,7 +68,7 @@ test_deps = {
   'test-qht-par': qht_bench,
 }
 
-if have_tools and 'CONFIG_VHOST_USER' in config_host and 'CONFIG_LINUX' in config_host
+if have_tools and have_vhost_user and 'CONFIG_LINUX' in config_host
   executable('vhost-user-bridge',
              sources: files('vhost-user-bridge.c'),
              dependencies: [qemuutil, vhost_user])
diff --git a/tools/meson.build b/tools/meson.build
index 46977af84f..10eb3a043f 100644
--- a/tools/meson.build
+++ b/tools/meson.build
@@ -3,7 +3,7 @@ have_virtiofsd = get_option('virtiofsd') \
              error_message: 'virtiofsd requires Linux') \
     .require(seccomp.found() and libcap_ng.found(),
              error_message: 'virtiofsd requires libcap-ng-devel and seccomp-devel') \
-    .require('CONFIG_VHOST_USER' in config_host,
+    .require(have_vhost_user,
              error_message: 'virtiofsd needs vhost-user-support') \
     .disable_auto_if(not have_tools and not have_system) \
     .allowed()

From 43b6d7ee1fbc5b5fb7c85d8131fdac1863214ad6 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:34:06 +0200
Subject: [PATCH 119/935] meson: use have_vhost_* variables to pick sources
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 Kconfig.host            |  3 ---
 backends/meson.build    |  8 ++++++--
 hw/net/meson.build      |  8 ++++++--
 hw/virtio/Kconfig       |  3 ---
 hw/virtio/meson.build   | 25 ++++++++++++++++---------
 meson.build             |  1 +
 net/meson.build         | 12 +++++++-----
 tests/qtest/meson.build |  4 +++-
 8 files changed, 39 insertions(+), 25 deletions(-)

diff --git a/Kconfig.host b/Kconfig.host
index 60b9c07b5e..1165c4eacd 100644
--- a/Kconfig.host
+++ b/Kconfig.host
@@ -22,15 +22,12 @@ config TPM
 
 config VHOST_USER
     bool
-    select VHOST
 
 config VHOST_VDPA
     bool
-    select VHOST
 
 config VHOST_KERNEL
     bool
-    select VHOST
 
 config VIRTFS
     bool
diff --git a/backends/meson.build b/backends/meson.build
index 535c3ca7dd..b1884a88ec 100644
--- a/backends/meson.build
+++ b/backends/meson.build
@@ -12,9 +12,13 @@ softmmu_ss.add([files(
 softmmu_ss.add(when: 'CONFIG_POSIX', if_true: files('rng-random.c'))
 softmmu_ss.add(when: 'CONFIG_POSIX', if_true: files('hostmem-file.c'))
 softmmu_ss.add(when: 'CONFIG_LINUX', if_true: files('hostmem-memfd.c'))
-softmmu_ss.add(when: ['CONFIG_VHOST_USER', 'CONFIG_VIRTIO'], if_true: files('vhost-user.c'))
+if have_vhost_user
+  softmmu_ss.add(when: 'CONFIG_VIRTIO', if_true: files('vhost-user.c'))
+endif
 softmmu_ss.add(when: 'CONFIG_VIRTIO_CRYPTO', if_true: files('cryptodev-vhost.c'))
-softmmu_ss.add(when: ['CONFIG_VIRTIO_CRYPTO', 'CONFIG_VHOST_CRYPTO'], if_true: files('cryptodev-vhost-user.c'))
+if have_vhost_user_crypto
+  softmmu_ss.add(when: 'CONFIG_VIRTIO_CRYPTO', if_true: files('cryptodev-vhost-user.c'))
+endif
 softmmu_ss.add(when: gio, if_true: files('dbus-vmstate.c'))
 softmmu_ss.add(when: 'CONFIG_SGX', if_true: files('hostmem-epc.c'))
 
diff --git a/hw/net/meson.build b/hw/net/meson.build
index 685b75badb..ebac261542 100644
--- a/hw/net/meson.build
+++ b/hw/net/meson.build
@@ -46,8 +46,12 @@ specific_ss.add(when: 'CONFIG_XILINX_ETHLITE', if_true: files('xilinx_ethlite.c'
 softmmu_ss.add(when: 'CONFIG_VIRTIO_NET', if_true: files('net_rx_pkt.c'))
 specific_ss.add(when: 'CONFIG_VIRTIO_NET', if_true: files('virtio-net.c'))
 
-softmmu_ss.add(when: ['CONFIG_VIRTIO_NET', 'CONFIG_VHOST_NET'], if_true: files('vhost_net.c'), if_false: files('vhost_net-stub.c'))
-softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('vhost_net-stub.c'))
+if have_vhost_net
+  softmmu_ss.add(when: 'CONFIG_VIRTIO_NET', if_true: files('vhost_net.c'), if_false: files('vhost_net-stub.c'))
+  softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('vhost_net-stub.c'))
+else
+  softmmu_ss.add(files('vhost_net-stub.c'))
+endif
 
 softmmu_ss.add(when: 'CONFIG_ETSEC', if_true: files(
   'fsl_etsec/etsec.c',
diff --git a/hw/virtio/Kconfig b/hw/virtio/Kconfig
index f8e235f814..e9ecae1f50 100644
--- a/hw/virtio/Kconfig
+++ b/hw/virtio/Kconfig
@@ -1,6 +1,3 @@
-config VHOST
-    bool
-
 config VIRTIO
     bool
 
diff --git a/hw/virtio/meson.build b/hw/virtio/meson.build
index f371404b04..7e8877fd64 100644
--- a/hw/virtio/meson.build
+++ b/hw/virtio/meson.build
@@ -2,18 +2,22 @@ softmmu_virtio_ss = ss.source_set()
 softmmu_virtio_ss.add(files('virtio-bus.c'))
 softmmu_virtio_ss.add(when: 'CONFIG_VIRTIO_PCI', if_true: files('virtio-pci.c'))
 softmmu_virtio_ss.add(when: 'CONFIG_VIRTIO_MMIO', if_true: files('virtio-mmio.c'))
-softmmu_virtio_ss.add(when: 'CONFIG_VHOST', if_false: files('vhost-stub.c'))
-
-softmmu_ss.add_all(when: 'CONFIG_VIRTIO', if_true: softmmu_virtio_ss)
-softmmu_ss.add(when: 'CONFIG_VIRTIO', if_false: files('vhost-stub.c'))
-
-softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('vhost-stub.c'))
 
 virtio_ss = ss.source_set()
 virtio_ss.add(files('virtio.c'))
-virtio_ss.add(when: 'CONFIG_VHOST', if_true: files('vhost.c', 'vhost-backend.c', 'vhost-iova-tree.c'))
-virtio_ss.add(when: 'CONFIG_VHOST_USER', if_true: files('vhost-user.c'))
-virtio_ss.add(when: 'CONFIG_VHOST_VDPA', if_true: files('vhost-shadow-virtqueue.c', 'vhost-vdpa.c'))
+
+if have_vhost
+  virtio_ss.add(files('vhost.c', 'vhost-backend.c', 'vhost-iova-tree.c'))
+  if have_vhost_user
+    virtio_ss.add(files('vhost-user.c'))
+  endif
+  if have_vhost_vdpa
+    virtio_ss.add(files('vhost-vdpa.c', 'vhost-shadow-virtqueue.c'))
+  endif
+else
+  softmmu_virtio_ss.add(files('vhost-stub.c'))
+endif
+
 virtio_ss.add(when: 'CONFIG_VIRTIO_BALLOON', if_true: files('virtio-balloon.c'))
 virtio_ss.add(when: 'CONFIG_VIRTIO_CRYPTO', if_true: files('virtio-crypto.c'))
 virtio_ss.add(when: 'CONFIG_VHOST_USER_FS', if_true: files('vhost-user-fs.c'))
@@ -54,3 +58,6 @@ virtio_pci_ss.add(when: 'CONFIG_VIRTIO_MEM', if_true: files('virtio-mem-pci.c'))
 virtio_ss.add_all(when: 'CONFIG_VIRTIO_PCI', if_true: virtio_pci_ss)
 
 specific_ss.add_all(when: 'CONFIG_VIRTIO', if_true: virtio_ss)
+softmmu_ss.add_all(when: 'CONFIG_VIRTIO', if_true: softmmu_virtio_ss)
+softmmu_ss.add(when: 'CONFIG_VIRTIO', if_false: files('vhost-stub.c'))
+softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('vhost-stub.c'))
diff --git a/meson.build b/meson.build
index 108292e50f..6627bf02ee 100644
--- a/meson.build
+++ b/meson.build
@@ -319,6 +319,7 @@ have_vhost_kernel = 'CONFIG_VHOST_KERNEL' in config_host
 have_vhost_net_user = 'CONFIG_VHOST_NET_USER' in config_host
 have_vhost_net_vdpa = 'CONFIG_VHOST_NET_VDPA' in config_host
 have_vhost_net = 'CONFIG_VHOST_NET' in config_host
+have_vhost = have_vhost_user or have_vhost_vdpa or have_vhost_kernel
 have_vhost_user_crypto = 'CONFIG_VHOST_CRYPTO' in config_host
 
 # Target-specific libraries and flags
diff --git a/net/meson.build b/net/meson.build
index 847bc2ac85..c965e83b26 100644
--- a/net/meson.build
+++ b/net/meson.build
@@ -26,10 +26,10 @@ softmmu_ss.add(when: vde, if_true: files('vde.c'))
 if have_netmap
   softmmu_ss.add(files('netmap.c'))
 endif
-vhost_user_ss = ss.source_set()
-vhost_user_ss.add(when: 'CONFIG_VIRTIO_NET', if_true: files('vhost-user.c'), if_false: files('vhost-user-stub.c'))
-softmmu_ss.add_all(when: 'CONFIG_VHOST_NET_USER', if_true: vhost_user_ss)
-softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('vhost-user-stub.c'))
+if have_vhost_net_user
+  softmmu_ss.add(when: 'CONFIG_VIRTIO_NET', if_true: files('vhost-user.c'), if_false: files('vhost-user-stub.c'))
+  softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('vhost-user-stub.c'))
+endif
 
 softmmu_ss.add(when: 'CONFIG_LINUX', if_true: files('tap-linux.c'))
 softmmu_ss.add(when: 'CONFIG_BSD', if_true: files('tap-bsd.c'))
@@ -40,6 +40,8 @@ if not config_host.has_key('CONFIG_LINUX') and not config_host.has_key('CONFIG_B
 endif
 softmmu_ss.add(when: 'CONFIG_POSIX', if_true: files(tap_posix))
 softmmu_ss.add(when: 'CONFIG_WIN32', if_true: files('tap-win32.c'))
-softmmu_ss.add(when: 'CONFIG_VHOST_NET_VDPA', if_true: files('vhost-vdpa.c'))
+if have_vhost_net_vdpa
+  softmmu_ss.add(files('vhost-vdpa.c'))
+endif
 
 subdir('can')
diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
index 32fb8cf755..3551b9c946 100644
--- a/tests/qtest/meson.build
+++ b/tests/qtest/meson.build
@@ -264,7 +264,9 @@ qos_test_ss.add(
 if have_virtfs
   qos_test_ss.add(files('virtio-9p-test.c'))
 endif
-qos_test_ss.add(when: 'CONFIG_VHOST_USER', if_true: files('vhost-user-test.c'))
+if have_vhost_user
+  qos_test_ss.add(files('vhost-user-test.c'))
+endif
 if have_tools and have_vhost_user_blk_server
   qos_test_ss.add(files('vhost-user-blk-test.c'))
 endif

From 2df89d542000819f0dd45134e24174997aca214e Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 20 Apr 2022 17:34:07 +0200
Subject: [PATCH 120/935] configure, meson: move vhost options to Meson
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Finish the conversion by moving all the definitions and the constraint
checks to meson_options.txt and meson.build respectively.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure                     | 82 -----------------------------------
 meson.build                   | 33 +++++++++++---
 meson_options.txt             | 10 +++++
 scripts/meson-buildoptions.sh | 15 +++++++
 4 files changed, 51 insertions(+), 89 deletions(-)

diff --git a/configure b/configure
index 9a83d64c9d..e77b29b093 100755
--- a/configure
+++ b/configure
@@ -282,11 +282,6 @@ EXTRA_CXXFLAGS=""
 EXTRA_OBJCFLAGS=""
 EXTRA_LDFLAGS=""
 
-vhost_kernel="$default_feature"
-vhost_net="$default_feature"
-vhost_crypto="$default_feature"
-vhost_user="no"
-vhost_vdpa="$default_feature"
 debug_tcg="no"
 sanitizers="no"
 tsan="no"
@@ -527,7 +522,6 @@ haiku)
 ;;
 linux)
   linux="yes"
-  vhost_user=${default_feature:-yes}
 ;;
 esac
 
@@ -864,14 +858,6 @@ for opt do
   ;;
   --with-coroutine=*) coroutine="$optarg"
   ;;
-  --disable-vhost-net) vhost_net="no"
-  ;;
-  --enable-vhost-net) vhost_net="yes"
-  ;;
-  --disable-vhost-crypto) vhost_crypto="no"
-  ;;
-  --enable-vhost-crypto) vhost_crypto="yes"
-  ;;
   --disable-zlib-test)
   ;;
   --disable-virtio-blk-data-plane|--enable-virtio-blk-data-plane)
@@ -883,18 +869,6 @@ for opt do
   --enable-uuid|--disable-uuid)
       echo "$0: $opt is obsolete, UUID support is always built" >&2
   ;;
-  --disable-vhost-user) vhost_user="no"
-  ;;
-  --enable-vhost-user) vhost_user="yes"
-  ;;
-  --disable-vhost-vdpa) vhost_vdpa="no"
-  ;;
-  --enable-vhost-vdpa) vhost_vdpa="yes"
-  ;;
-  --disable-vhost-kernel) vhost_kernel="no"
-  ;;
-  --enable-vhost-kernel) vhost_kernel="yes"
-  ;;
   --disable-capstone) capstone="disabled"
   ;;
   --enable-capstone) capstone="enabled"
@@ -1093,11 +1067,6 @@ cat << EOF
   debug-info      debugging information
   safe-stack      SafeStack Stack Smash Protection. Depends on
                   clang/llvm >= 3.7 and requires coroutine backend ucontext.
-  vhost-net       vhost-net kernel acceleration support
-  vhost-crypto    vhost-user-crypto backend support
-  vhost-kernel    vhost kernel backend support
-  vhost-user      vhost-user backend support
-  vhost-vdpa      vhost-vdpa kernel backend support
 
 NOTE: The object files are built at the place where configure is launched
 EOF
@@ -1511,35 +1480,6 @@ else
     exit 1
 fi
 
-#########################################
-# vhost interdependencies and host support
-
-# vhost backends
-if test "$vhost_user" = "yes" && test "$mingw32" = "yes"; then
-  error_exit "vhost-user is not available on Windows"
-fi
-test "$vhost_vdpa" = "" && vhost_vdpa=$linux
-if test "$vhost_vdpa" = "yes" && test "$linux" != "yes"; then
-  error_exit "vhost-vdpa is only available on Linux"
-fi
-test "$vhost_kernel" = "" && vhost_kernel=$linux
-if test "$vhost_kernel" = "yes" && test "$linux" != "yes"; then
-  error_exit "vhost-kernel is only available on Linux"
-fi
-
-# vhost-user backends
-test "$vhost_crypto" = "" && vhost_crypto=$vhost_user
-if test "$vhost_crypto" = "yes" && test "$vhost_user" = "no"; then
-  error_exit "--enable-vhost-crypto requires --enable-vhost-user"
-fi
-
-# OR the vhost-kernel, vhost-vdpa and vhost-user values for simplicity
-if test "$vhost_net" = ""; then
-  test "$vhost_user" = "yes" && vhost_net=yes
-  test "$vhost_vdpa" = "yes" && vhost_net=yes
-  test "$vhost_kernel" = "yes" && vhost_net=yes
-fi
-
 ##########################################
 # pkg-config probe
 
@@ -2059,28 +1999,6 @@ if test "$modules" = "yes"; then
   echo "CONFIG_MODULES=y" >> $config_host_mak
 fi
 
-if test "$vhost_net" = "yes" ; then
-  echo "CONFIG_VHOST_NET=y" >> $config_host_mak
-fi
-if test "$vhost_user" = "yes" ; then
-  echo "CONFIG_VHOST_NET_USER=y" >> $config_host_mak
-fi
-if test "$vhost_vdpa" = "yes" ; then
-  echo "CONFIG_VHOST_NET_VDPA=y" >> $config_host_mak
-fi
-if test "$vhost_crypto" = "yes" ; then
-  echo "CONFIG_VHOST_CRYPTO=y" >> $config_host_mak
-fi
-if test "$vhost_kernel" = "yes" ; then
-  echo "CONFIG_VHOST_KERNEL=y" >> $config_host_mak
-fi
-if test "$vhost_user" = "yes" ; then
-  echo "CONFIG_VHOST_USER=y" >> $config_host_mak
-fi
-if test "$vhost_vdpa" = "yes" ; then
-  echo "CONFIG_VHOST_VDPA=y" >> $config_host_mak
-fi
-
 # XXX: suppress that
 if [ "$bsd" = "yes" ] ; then
   echo "CONFIG_BSD=y" >> $config_host_mak
diff --git a/meson.build b/meson.build
index 6627bf02ee..40132b4804 100644
--- a/meson.build
+++ b/meson.build
@@ -313,14 +313,26 @@ have_tpm = get_option('tpm') \
   .allowed()
 
 # vhost
-have_vhost_user = 'CONFIG_VHOST_USER' in config_host
-have_vhost_vdpa = 'CONFIG_VHOST_VDPA' in config_host
-have_vhost_kernel = 'CONFIG_VHOST_KERNEL' in config_host
-have_vhost_net_user = 'CONFIG_VHOST_NET_USER' in config_host
-have_vhost_net_vdpa = 'CONFIG_VHOST_NET_VDPA' in config_host
-have_vhost_net = 'CONFIG_VHOST_NET' in config_host
+have_vhost_user = get_option('vhost_user') \
+  .disable_auto_if(targetos != 'linux') \
+  .require(targetos != 'windows',
+           error_message: 'vhost-user is not available on Windows').allowed()
+have_vhost_vdpa = get_option('vhost_vdpa') \
+  .require(targetos == 'linux',
+           error_message: 'vhost-vdpa is only available on Linux').allowed()
+have_vhost_kernel = get_option('vhost_kernel') \
+  .require(targetos == 'linux',
+           error_message: 'vhost-kernel is only available on Linux').allowed()
+have_vhost_user_crypto = get_option('vhost_crypto') \
+  .require(have_vhost_user,
+           error_message: 'vhost-crypto requires vhost-user to be enabled').allowed()
+
 have_vhost = have_vhost_user or have_vhost_vdpa or have_vhost_kernel
-have_vhost_user_crypto = 'CONFIG_VHOST_CRYPTO' in config_host
+
+have_vhost_net_user = have_vhost_user and get_option('vhost_net').allowed()
+have_vhost_net_vdpa = have_vhost_vdpa and get_option('vhost_net').allowed()
+have_vhost_net_kernel = have_vhost_kernel and get_option('vhost_net').allowed()
+have_vhost_net = have_vhost_net_kernel or have_vhost_net_user or have_vhost_net_vdpa
 
 # Target-specific libraries and flags
 libm = cc.find_library('m', required: false)
@@ -1711,6 +1723,13 @@ config_host_data.set('CONFIG_SNAPPY', snappy.found())
 config_host_data.set('CONFIG_TPM', have_tpm)
 config_host_data.set('CONFIG_USB_LIBUSB', libusb.found())
 config_host_data.set('CONFIG_VDE', vde.found())
+config_host_data.set('CONFIG_VHOST_NET', have_vhost_net)
+config_host_data.set('CONFIG_VHOST_NET_USER', have_vhost_net_user)
+config_host_data.set('CONFIG_VHOST_NET_VDPA', have_vhost_net_vdpa)
+config_host_data.set('CONFIG_VHOST_KERNEL', have_vhost_kernel)
+config_host_data.set('CONFIG_VHOST_USER', have_vhost_user)
+config_host_data.set('CONFIG_VHOST_CRYPTO', have_vhost_user_crypto)
+config_host_data.set('CONFIG_VHOST_VDPA', have_vhost_vdpa)
 config_host_data.set('CONFIG_VHOST_USER_BLK_SERVER', have_vhost_user_blk_server)
 config_host_data.set('CONFIG_PNG', png.found())
 config_host_data.set('CONFIG_VNC', vnc.found())
diff --git a/meson_options.txt b/meson_options.txt
index 35f5a72507..29c6b90cec 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -235,6 +235,16 @@ option('oss', type: 'feature', value: 'auto',
 option('pa', type: 'feature', value: 'auto',
        description: 'PulseAudio sound support')
 
+option('vhost_kernel', type: 'feature', value: 'auto',
+       description: 'vhost kernel backend support')
+option('vhost_net', type: 'feature', value: 'auto',
+       description: 'vhost-net kernel acceleration support')
+option('vhost_user', type: 'feature', value: 'auto',
+       description: 'vhost-user backend support')
+option('vhost_crypto', type: 'feature', value: 'auto',
+       description: 'vhost-user crypto backend support')
+option('vhost_vdpa', type: 'feature', value: 'auto',
+       description: 'vhost-vdpa kernel backend support')
 option('vhost_user_blk_server', type: 'feature', value: 'auto',
        description: 'build vhost-user-blk server')
 option('virtfs', type: 'feature', value: 'auto',
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index 4b0770a9e0..5d2172bfb4 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -154,8 +154,13 @@ meson_options_help() {
   printf "%s\n" '  usb-redir       libusbredir support'
   printf "%s\n" '  vde             vde network backend support'
   printf "%s\n" '  vdi             vdi image format support'
+  printf "%s\n" '  vhost-crypto    vhost-user crypto backend support'
+  printf "%s\n" '  vhost-kernel    vhost kernel backend support'
+  printf "%s\n" '  vhost-net       vhost-net kernel acceleration support'
+  printf "%s\n" '  vhost-user      vhost-user backend support'
   printf "%s\n" '  vhost-user-blk-server'
   printf "%s\n" '                  build vhost-user-blk server'
+  printf "%s\n" '  vhost-vdpa      vhost-vdpa kernel backend support'
   printf "%s\n" '  virglrenderer   virgl rendering support'
   printf "%s\n" '  virtfs          virtio-9p support'
   printf "%s\n" '  virtiofsd       build virtiofs daemon (virtiofsd)'
@@ -409,8 +414,18 @@ _meson_option_parse() {
     --disable-vde) printf "%s" -Dvde=disabled ;;
     --enable-vdi) printf "%s" -Dvdi=enabled ;;
     --disable-vdi) printf "%s" -Dvdi=disabled ;;
+    --enable-vhost-crypto) printf "%s" -Dvhost_crypto=enabled ;;
+    --disable-vhost-crypto) printf "%s" -Dvhost_crypto=disabled ;;
+    --enable-vhost-kernel) printf "%s" -Dvhost_kernel=enabled ;;
+    --disable-vhost-kernel) printf "%s" -Dvhost_kernel=disabled ;;
+    --enable-vhost-net) printf "%s" -Dvhost_net=enabled ;;
+    --disable-vhost-net) printf "%s" -Dvhost_net=disabled ;;
+    --enable-vhost-user) printf "%s" -Dvhost_user=enabled ;;
+    --disable-vhost-user) printf "%s" -Dvhost_user=disabled ;;
     --enable-vhost-user-blk-server) printf "%s" -Dvhost_user_blk_server=enabled ;;
     --disable-vhost-user-blk-server) printf "%s" -Dvhost_user_blk_server=disabled ;;
+    --enable-vhost-vdpa) printf "%s" -Dvhost_vdpa=enabled ;;
+    --disable-vhost-vdpa) printf "%s" -Dvhost_vdpa=disabled ;;
     --enable-virglrenderer) printf "%s" -Dvirglrenderer=enabled ;;
     --disable-virglrenderer) printf "%s" -Dvirglrenderer=disabled ;;
     --enable-virtfs) printf "%s" -Dvirtfs=enabled ;;

From 6033b9ecd4f6a26b78f44a94813e1e464f5b0549 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 27 Apr 2022 12:01:48 +0200
Subject: [PATCH 121/935] pc: remove -soundhw pcspk

The pcspk device is the only user of the init_isa function, and the only
-soundhw option which does not create a new device (it hacks into the
PCSpkState by hand).  Remove it, since it was deprecated.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/audio/pcspk.c           | 10 ----------
 hw/audio/soundhw.c         | 27 ++++-----------------------
 include/hw/audio/soundhw.h |  3 ---
 3 files changed, 4 insertions(+), 36 deletions(-)

diff --git a/hw/audio/pcspk.c b/hw/audio/pcspk.c
index dfc7ebca4e..daf92a4ce1 100644
--- a/hw/audio/pcspk.c
+++ b/hw/audio/pcspk.c
@@ -245,18 +245,8 @@ static const TypeInfo pcspk_info = {
     .class_init     = pcspk_class_initfn,
 };
 
-static int pcspk_audio_init_soundhw(ISABus *bus)
-{
-    PCSpkState *s = pcspk_state;
-
-    warn_report("'-soundhw pcspk' is deprecated, "
-                "please set a backend using '-machine pcspk-audiodev=<name>' instead");
-    return pcspk_audio_init(s);
-}
-
 static void pcspk_register(void)
 {
     type_register_static(&pcspk_info);
-    isa_register_soundhw("pcspk", "PC speaker", pcspk_audio_init_soundhw);
 }
 type_init(pcspk_register)
diff --git a/hw/audio/soundhw.c b/hw/audio/soundhw.c
index 173b674ff5..f7d94d7dfa 100644
--- a/hw/audio/soundhw.c
+++ b/hw/audio/soundhw.c
@@ -36,26 +36,12 @@ struct soundhw {
     const char *typename;
     int enabled;
     int isa;
-    union {
-        int (*init_isa) (ISABus *bus);
-        int (*init_pci) (PCIBus *bus);
-    } init;
+    int (*init_pci) (PCIBus *bus);
 };
 
 static struct soundhw soundhw[9];
 static int soundhw_count;
 
-void isa_register_soundhw(const char *name, const char *descr,
-                          int (*init_isa)(ISABus *bus))
-{
-    assert(soundhw_count < ARRAY_SIZE(soundhw) - 1);
-    soundhw[soundhw_count].name = name;
-    soundhw[soundhw_count].descr = descr;
-    soundhw[soundhw_count].isa = 1;
-    soundhw[soundhw_count].init.init_isa = init_isa;
-    soundhw_count++;
-}
-
 void pci_register_soundhw(const char *name, const char *descr,
                           int (*init_pci)(PCIBus *bus))
 {
@@ -63,7 +49,7 @@ void pci_register_soundhw(const char *name, const char *descr,
     soundhw[soundhw_count].name = name;
     soundhw[soundhw_count].descr = descr;
     soundhw[soundhw_count].isa = 0;
-    soundhw[soundhw_count].init.init_pci = init_pci;
+    soundhw[soundhw_count].init_pci = init_pci;
     soundhw_count++;
 }
 
@@ -158,18 +144,13 @@ void soundhw_init(void)
                 } else {
                     pci_create_simple(pci_bus, -1, c->typename);
                 }
-            } else if (c->isa) {
-                if (!isa_bus) {
-                    error_report("ISA bus not available for %s", c->name);
-                    exit(1);
-                }
-                c->init.init_isa(isa_bus);
             } else {
+                assert(!c->isa);
                 if (!pci_bus) {
                     error_report("PCI bus not available for %s", c->name);
                     exit(1);
                 }
-                c->init.init_pci(pci_bus);
+                c->init_pci(pci_bus);
             }
         }
     }
diff --git a/include/hw/audio/soundhw.h b/include/hw/audio/soundhw.h
index f09a297854..e68685fcda 100644
--- a/include/hw/audio/soundhw.h
+++ b/include/hw/audio/soundhw.h
@@ -1,9 +1,6 @@
 #ifndef HW_SOUNDHW_H
 #define HW_SOUNDHW_H
 
-void isa_register_soundhw(const char *name, const char *descr,
-                          int (*init_isa)(ISABus *bus));
-
 void pci_register_soundhw(const char *name, const char *descr,
                           int (*init_pci)(PCIBus *bus));
 void deprecated_register_soundhw(const char *name, const char *descr,

From 17bd42896d31bf57cb3cc16abefb3221391eb288 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:11 +0100
Subject: [PATCH 122/935] dino: checkpatch fixes

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-2-mark.cave-ayland@ilande.co.uk>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index eab96dd84e..4031cfb7f0 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -101,8 +101,10 @@ static const uint32_t reg800_keep_bits[DINO800_REGS] = {
 struct DinoState {
     PCIHostState parent_obj;
 
-    /* PCI_CONFIG_ADDR is parent_obj.config_reg, via pci_host_conf_be_ops,
-       so that we can map PCI_CONFIG_DATA to pci_host_data_be_ops.  */
+    /*
+     * PCI_CONFIG_ADDR is parent_obj.config_reg, via pci_host_conf_be_ops,
+     * so that we can map PCI_CONFIG_DATA to pci_host_data_be_ops.
+     */
     uint32_t config_reg_dino; /* keep original copy, including 2 lowest bits */
 
     uint32_t iar0;

From 7cdfa94166fb5aa25681152675c85c709e927602 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:12 +0100
Subject: [PATCH 123/935] dino: move registers from dino_init() to
 DINO_PCI_BRIDGE init function

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-3-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c | 44 +++++++++++++++++++++++++++++---------------
 1 file changed, 29 insertions(+), 15 deletions(-)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index 4031cfb7f0..9401b68ff7 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -526,21 +526,6 @@ PCIBus *dino_init(MemoryRegion *addr_space,
     s->iar0 = s->iar1 = CPU_HPA + 3;
     s->toc_addr = 0xFFFA0030; /* IO_COMMAND of CPU */
 
-    /* Dino PCI access from main memory.  */
-    memory_region_init_io(&s->this_mem, OBJECT(s), &dino_chip_ops,
-                          s, "dino", 4096);
-    memory_region_add_subregion(addr_space, DINO_HPA, &s->this_mem);
-
-    /* Dino PCI config. */
-    memory_region_init_io(&s->parent_obj.conf_mem, OBJECT(&s->parent_obj),
-                          &dino_config_addr_ops, dev, "pci-conf-idx", 4);
-    memory_region_init_io(&s->parent_obj.data_mem, OBJECT(&s->parent_obj),
-                          &dino_config_data_ops, dev, "pci-conf-data", 4);
-    memory_region_add_subregion(&s->this_mem, DINO_PCI_CONFIG_ADDR,
-                                &s->parent_obj.conf_mem);
-    memory_region_add_subregion(&s->this_mem, DINO_CONFIG_DATA,
-                                &s->parent_obj.data_mem);
-
     /* Dino PCI bus memory.  */
     memory_region_init(&s->pci_mem, OBJECT(s), "pci-memory", 4 * GiB);
 
@@ -550,6 +535,9 @@ PCIBus *dino_init(MemoryRegion *addr_space,
     s->parent_obj.bus = b;
     sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
 
+    memory_region_add_subregion(addr_space, DINO_HPA,
+                                sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 0));
+
     /* Set up windows into PCI bus memory.  */
     for (i = 1; i < 31; i++) {
         uint32_t addr = 0xf0000000 + i * DINO_MEM_CHUNK_SIZE;
@@ -588,6 +576,31 @@ PCIBus *dino_init(MemoryRegion *addr_space,
     return b;
 }
 
+static void dino_pcihost_init(Object *obj)
+{
+    DinoState *s = DINO_PCI_HOST_BRIDGE(obj);
+    PCIHostState *phb = PCI_HOST_BRIDGE(obj);
+    SysBusDevice *sbd = SYS_BUS_DEVICE(obj);
+
+    /* Dino PCI access from main memory.  */
+    memory_region_init_io(&s->this_mem, OBJECT(s), &dino_chip_ops,
+                          s, "dino", 4096);
+
+    /* Dino PCI config. */
+    memory_region_init_io(&phb->conf_mem, OBJECT(phb),
+                          &dino_config_addr_ops, DEVICE(s),
+                          "pci-conf-idx", 4);
+    memory_region_init_io(&phb->data_mem, OBJECT(phb),
+                          &dino_config_data_ops, DEVICE(s),
+                          "pci-conf-data", 4);
+    memory_region_add_subregion(&s->this_mem, DINO_PCI_CONFIG_ADDR,
+                                &phb->conf_mem);
+    memory_region_add_subregion(&s->this_mem, DINO_CONFIG_DATA,
+                                &phb->data_mem);
+
+    sysbus_init_mmio(sbd, &s->this_mem);
+}
+
 static void dino_pcihost_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -598,6 +611,7 @@ static void dino_pcihost_class_init(ObjectClass *klass, void *data)
 static const TypeInfo dino_pcihost_info = {
     .name          = TYPE_DINO_PCI_HOST_BRIDGE,
     .parent        = TYPE_PCI_HOST_BRIDGE,
+    .instance_init = dino_pcihost_init,
     .instance_size = sizeof(DinoState),
     .class_init    = dino_pcihost_class_init,
 };

From 63901b6cc4d8b471ac4b92826220db914b908368 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:13 +0100
Subject: [PATCH 124/935] dino: move PCI bus initialisation to
 dino_pcihost_init()

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-4-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index 9401b68ff7..c6c01ea14c 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -526,13 +526,7 @@ PCIBus *dino_init(MemoryRegion *addr_space,
     s->iar0 = s->iar1 = CPU_HPA + 3;
     s->toc_addr = 0xFFFA0030; /* IO_COMMAND of CPU */
 
-    /* Dino PCI bus memory.  */
-    memory_region_init(&s->pci_mem, OBJECT(s), "pci-memory", 4 * GiB);
-
-    b = pci_register_root_bus(dev, "pci", dino_set_irq, dino_pci_map_irq, s,
-                              &s->pci_mem, get_system_io(),
-                              PCI_DEVFN(0, 0), 32, TYPE_PCI_BUS);
-    s->parent_obj.bus = b;
+    b = s->parent_obj.bus;
     sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
 
     memory_region_add_subregion(addr_space, DINO_HPA,
@@ -598,6 +592,14 @@ static void dino_pcihost_init(Object *obj)
     memory_region_add_subregion(&s->this_mem, DINO_CONFIG_DATA,
                                 &phb->data_mem);
 
+    /* Dino PCI bus memory.  */
+    memory_region_init(&s->pci_mem, OBJECT(s), "pci-memory", 4 * GiB);
+
+    phb->bus = pci_register_root_bus(DEVICE(s), "pci",
+                                     dino_set_irq, dino_pci_map_irq, s,
+                                     &s->pci_mem, get_system_io(),
+                                     PCI_DEVFN(0, 0), 32, TYPE_PCI_BUS);
+
     sysbus_init_mmio(sbd, &s->this_mem);
 }
 

From cc363c4a10b99709ea4e9e185c7b71de72f88a17 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:14 +0100
Subject: [PATCH 125/935] dino: move PCI windows initialisation to
 dino_pcihost_init()

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-5-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index c6c01ea14c..bc27aad5dc 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -519,7 +519,6 @@ PCIBus *dino_init(MemoryRegion *addr_space,
     DeviceState *dev;
     DinoState *s;
     PCIBus *b;
-    int i;
 
     dev = qdev_new(TYPE_DINO_PCI_HOST_BRIDGE);
     s = DINO_PCI_HOST_BRIDGE(dev);
@@ -532,16 +531,6 @@ PCIBus *dino_init(MemoryRegion *addr_space,
     memory_region_add_subregion(addr_space, DINO_HPA,
                                 sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 0));
 
-    /* Set up windows into PCI bus memory.  */
-    for (i = 1; i < 31; i++) {
-        uint32_t addr = 0xf0000000 + i * DINO_MEM_CHUNK_SIZE;
-        char *name = g_strdup_printf("PCI Outbound Window %d", i);
-        memory_region_init_alias(&s->pci_mem_alias[i], OBJECT(s),
-                                 name, &s->pci_mem, addr,
-                                 DINO_MEM_CHUNK_SIZE);
-        g_free(name);
-    }
-
     /* Set up PCI view of memory: Bus master address space.  */
     memory_region_init(&s->bm, OBJECT(s), "bm-dino", 4 * GiB);
     memory_region_init_alias(&s->bm_ram_alias, OBJECT(s),
@@ -575,6 +564,7 @@ static void dino_pcihost_init(Object *obj)
     DinoState *s = DINO_PCI_HOST_BRIDGE(obj);
     PCIHostState *phb = PCI_HOST_BRIDGE(obj);
     SysBusDevice *sbd = SYS_BUS_DEVICE(obj);
+    int i;
 
     /* Dino PCI access from main memory.  */
     memory_region_init_io(&s->this_mem, OBJECT(s), &dino_chip_ops,
@@ -600,6 +590,16 @@ static void dino_pcihost_init(Object *obj)
                                      &s->pci_mem, get_system_io(),
                                      PCI_DEVFN(0, 0), 32, TYPE_PCI_BUS);
 
+    /* Set up windows into PCI bus memory.  */
+    for (i = 1; i < 31; i++) {
+        uint32_t addr = 0xf0000000 + i * DINO_MEM_CHUNK_SIZE;
+        char *name = g_strdup_printf("PCI Outbound Window %d", i);
+        memory_region_init_alias(&s->pci_mem_alias[i], OBJECT(s),
+                                 name, &s->pci_mem, addr,
+                                 DINO_MEM_CHUNK_SIZE);
+        g_free(name);
+    }
+
     sysbus_init_mmio(sbd, &s->this_mem);
 }
 

From 270b29587f4705aa8d6b46ba2c0ff5bf712cc637 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:15 +0100
Subject: [PATCH 126/935] dino: add memory-as property containing a link to the
 memory address space

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-6-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index bc27aad5dc..13b2210462 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -17,6 +17,7 @@
 #include "hw/irq.h"
 #include "hw/pci/pci.h"
 #include "hw/pci/pci_bus.h"
+#include "hw/qdev-properties.h"
 #include "migration/vmstate.h"
 #include "hppa_sys.h"
 #include "trace.h"
@@ -124,6 +125,8 @@ struct DinoState {
     MemoryRegion pci_mem;
     MemoryRegion pci_mem_alias[32];
 
+    MemoryRegion *memory_as;
+
     AddressSpace bm_as;
     MemoryRegion bm;
     MemoryRegion bm_ram_alias;
@@ -521,6 +524,8 @@ PCIBus *dino_init(MemoryRegion *addr_space,
     PCIBus *b;
 
     dev = qdev_new(TYPE_DINO_PCI_HOST_BRIDGE);
+    object_property_set_link(OBJECT(dev), "memory-as", OBJECT(addr_space),
+                             &error_fatal);
     s = DINO_PCI_HOST_BRIDGE(dev);
     s->iar0 = s->iar1 = CPU_HPA + 3;
     s->toc_addr = 0xFFFA0030; /* IO_COMMAND of CPU */
@@ -603,10 +608,17 @@ static void dino_pcihost_init(Object *obj)
     sysbus_init_mmio(sbd, &s->this_mem);
 }
 
+static Property dino_pcihost_properties[] = {
+    DEFINE_PROP_LINK("memory-as", DinoState, memory_as, TYPE_MEMORY_REGION,
+                     MemoryRegion *),
+    DEFINE_PROP_END_OF_LIST(),
+};
+
 static void dino_pcihost_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
 
+    device_class_set_props(dc, dino_pcihost_properties);
     dc->vmsd = &vmstate_dino;
 }
 

From 5ac6c43c856a81e0ca1a1bab8f2fbd7b3a66274d Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:16 +0100
Subject: [PATCH 127/935] dino: move pci_setup_iommu() to dino_pcihost_init()

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-7-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index 13b2210462..dcebb9f98f 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -556,7 +556,6 @@ PCIBus *dino_init(MemoryRegion *addr_space,
     memory_region_add_subregion(&s->bm, 0xfff00000,
                                 &s->bm_cpu_alias);
     address_space_init(&s->bm_as, &s->bm, "pci-bm");
-    pci_setup_iommu(b, dino_pcihost_set_iommu, s);
 
     *p_rtc_irq = qemu_allocate_irq(dino_set_timer_irq, s, 0);
     *p_ser_irq = qemu_allocate_irq(dino_set_serial_irq, s, 0);
@@ -605,6 +604,8 @@ static void dino_pcihost_init(Object *obj)
         g_free(name);
     }
 
+    pci_setup_iommu(phb->bus, dino_pcihost_set_iommu, s);
+
     sysbus_init_mmio(sbd, &s->this_mem);
 }
 

From 9cf69f444c4034e5278ec52ae9869f13ae9a2bd2 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:17 +0100
Subject: [PATCH 128/935] dino: move PCI bus master address space setup to
 dino_pcihost_realize()

Add a new dino_pcihost_unrealize() function to remove the address space when the
device is unrealized.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-8-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c | 25 ++++++++++++++++++++-----
 1 file changed, 20 insertions(+), 5 deletions(-)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index dcebb9f98f..ae79ddbf5c 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -536,17 +536,27 @@ PCIBus *dino_init(MemoryRegion *addr_space,
     memory_region_add_subregion(addr_space, DINO_HPA,
                                 sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 0));
 
+    *p_rtc_irq = qemu_allocate_irq(dino_set_timer_irq, s, 0);
+    *p_ser_irq = qemu_allocate_irq(dino_set_serial_irq, s, 0);
+
+    return b;
+}
+
+static void dino_pcihost_realize(DeviceState *dev, Error **errp)
+{
+    DinoState *s = DINO_PCI_HOST_BRIDGE(dev);
+
     /* Set up PCI view of memory: Bus master address space.  */
     memory_region_init(&s->bm, OBJECT(s), "bm-dino", 4 * GiB);
     memory_region_init_alias(&s->bm_ram_alias, OBJECT(s),
-                             "bm-system", addr_space, 0,
+                             "bm-system", s->memory_as, 0,
                              0xf0000000 + DINO_MEM_CHUNK_SIZE);
     memory_region_init_alias(&s->bm_pci_alias, OBJECT(s),
                              "bm-pci", &s->pci_mem,
                              0xf0000000 + DINO_MEM_CHUNK_SIZE,
                              30 * DINO_MEM_CHUNK_SIZE);
     memory_region_init_alias(&s->bm_cpu_alias, OBJECT(s),
-                             "bm-cpu", addr_space, 0xfff00000,
+                             "bm-cpu", s->memory_as, 0xfff00000,
                              0xfffff);
     memory_region_add_subregion(&s->bm, 0,
                                 &s->bm_ram_alias);
@@ -555,12 +565,15 @@ PCIBus *dino_init(MemoryRegion *addr_space,
                                 &s->bm_pci_alias);
     memory_region_add_subregion(&s->bm, 0xfff00000,
                                 &s->bm_cpu_alias);
+
     address_space_init(&s->bm_as, &s->bm, "pci-bm");
+}
 
-    *p_rtc_irq = qemu_allocate_irq(dino_set_timer_irq, s, 0);
-    *p_ser_irq = qemu_allocate_irq(dino_set_serial_irq, s, 0);
+static void dino_pcihost_unrealize(DeviceState *dev)
+{
+    DinoState *s = DINO_PCI_HOST_BRIDGE(dev);
 
-    return b;
+    address_space_destroy(&s->bm_as);
 }
 
 static void dino_pcihost_init(Object *obj)
@@ -619,6 +632,8 @@ static void dino_pcihost_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
 
+    dc->realize = dino_pcihost_realize;
+    dc->unrealize = dino_pcihost_unrealize;
     device_class_set_props(dc, dino_pcihost_properties);
     dc->vmsd = &vmstate_dino;
 }

From 98d168f359319ddd20b21058a081ec77837f7347 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:18 +0100
Subject: [PATCH 129/935] dino: move initial register configuration to new
 dino_pcihost_reset() function

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-9-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index ae79ddbf5c..d3cd8578a1 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -527,8 +527,6 @@ PCIBus *dino_init(MemoryRegion *addr_space,
     object_property_set_link(OBJECT(dev), "memory-as", OBJECT(addr_space),
                              &error_fatal);
     s = DINO_PCI_HOST_BRIDGE(dev);
-    s->iar0 = s->iar1 = CPU_HPA + 3;
-    s->toc_addr = 0xFFFA0030; /* IO_COMMAND of CPU */
 
     b = s->parent_obj.bus;
     sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
@@ -542,6 +540,14 @@ PCIBus *dino_init(MemoryRegion *addr_space,
     return b;
 }
 
+static void dino_pcihost_reset(DeviceState *dev)
+{
+    DinoState *s = DINO_PCI_HOST_BRIDGE(dev);
+
+    s->iar0 = s->iar1 = CPU_HPA + 3;
+    s->toc_addr = 0xFFFA0030; /* IO_COMMAND of CPU */
+}
+
 static void dino_pcihost_realize(DeviceState *dev, Error **errp)
 {
     DinoState *s = DINO_PCI_HOST_BRIDGE(dev);
@@ -632,6 +638,7 @@ static void dino_pcihost_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
 
+    dc->reset = dino_pcihost_reset;
     dc->realize = dino_pcihost_realize;
     dc->unrealize = dino_pcihost_unrealize;
     device_class_set_props(dc, dino_pcihost_properties);

From ee313d5abb1f911c426f8fbcb3abb4b1d8e281ce Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:19 +0100
Subject: [PATCH 130/935] dino: use QOM cast instead of directly referencing
 parent_obj

Use a QOM cast in both dino_chip_read_with_attrs() and dino_chip_write_with_attrs()
instead of directly referencing parent_obj.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-10-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index d3cd8578a1..77b8089664 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -205,6 +205,7 @@ static MemTxResult dino_chip_read_with_attrs(void *opaque, hwaddr addr,
                                              MemTxAttrs attrs)
 {
     DinoState *s = opaque;
+    PCIHostState *phb = PCI_HOST_BRIDGE(s);
     MemTxResult ret = MEMTX_OK;
     AddressSpace *io;
     uint16_t ioaddr;
@@ -214,7 +215,7 @@ static MemTxResult dino_chip_read_with_attrs(void *opaque, hwaddr addr,
     case DINO_PCI_IO_DATA ... DINO_PCI_IO_DATA + 3:
         /* Read from PCI IO space. */
         io = &address_space_io;
-        ioaddr = s->parent_obj.config_reg + (addr & 3);
+        ioaddr = phb->config_reg + (addr & 3);
         switch (size) {
         case 1:
             val = address_space_ldub(io, ioaddr, attrs, &ret);
@@ -297,6 +298,7 @@ static MemTxResult dino_chip_write_with_attrs(void *opaque, hwaddr addr,
                                               MemTxAttrs attrs)
 {
     DinoState *s = opaque;
+    PCIHostState *phb = PCI_HOST_BRIDGE(s);
     AddressSpace *io;
     MemTxResult ret;
     uint16_t ioaddr;
@@ -308,7 +310,7 @@ static MemTxResult dino_chip_write_with_attrs(void *opaque, hwaddr addr,
     case DINO_IO_DATA ... DINO_PCI_IO_DATA + 3:
         /* Write into PCI IO space.  */
         io = &address_space_io;
-        ioaddr = s->parent_obj.config_reg + (addr & 3);
+        ioaddr = phb->config_reg + (addr & 3);
         switch (size) {
         case 1:
             address_space_stb(io, ioaddr, val, attrs, &ret);

From 079e7ad989e3902ac7ecb70086d3b380faeeaf63 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:20 +0100
Subject: [PATCH 131/935] dino: return PCIBus from dino_init() using
 qdev_get_child_bus()

This allows access to the PCI bus without having to reference parent_obj directly.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-11-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index 77b8089664..d89b48493f 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -523,14 +523,12 @@ PCIBus *dino_init(MemoryRegion *addr_space,
 {
     DeviceState *dev;
     DinoState *s;
-    PCIBus *b;
+    PCIBus *pci_bus;
 
     dev = qdev_new(TYPE_DINO_PCI_HOST_BRIDGE);
     object_property_set_link(OBJECT(dev), "memory-as", OBJECT(addr_space),
                              &error_fatal);
     s = DINO_PCI_HOST_BRIDGE(dev);
-
-    b = s->parent_obj.bus;
     sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
 
     memory_region_add_subregion(addr_space, DINO_HPA,
@@ -539,7 +537,8 @@ PCIBus *dino_init(MemoryRegion *addr_space,
     *p_rtc_irq = qemu_allocate_irq(dino_set_timer_irq, s, 0);
     *p_ser_irq = qemu_allocate_irq(dino_set_serial_irq, s, 0);
 
-    return b;
+    pci_bus = PCI_BUS(qdev_get_child_bus(dev, "pci"));
+    return pci_bus;
 }
 
 static void dino_pcihost_reset(DeviceState *dev)

From 14e275efcbe9cc8653a90f2085e2cdf0f7f7e24f Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:21 +0100
Subject: [PATCH 132/935] dino: split declarations from dino.c into dino.h

This is to allow access to DinoState from outside dino.c. With the changes to
the headers it is now possible to remove the duplicate definition for
TYPE_DINO_PCI_HOST_BRIDGE from hppa_sys.h.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-12-mark.cave-ayland@ilande.co.uk>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c     | 111 +--------------------------------------
 hw/hppa/dino.h     | 127 +++++++++++++++++++++++++++++++++++++++++++++
 hw/hppa/hppa_sys.h |   2 -
 3 files changed, 128 insertions(+), 112 deletions(-)
 create mode 100644 hw/hppa/dino.h

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index d89b48493f..01546ff6fc 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -18,122 +18,13 @@
 #include "hw/pci/pci.h"
 #include "hw/pci/pci_bus.h"
 #include "hw/qdev-properties.h"
+#include "dino.h"
 #include "migration/vmstate.h"
 #include "hppa_sys.h"
 #include "trace.h"
 #include "qom/object.h"
 
 
-#define TYPE_DINO_PCI_HOST_BRIDGE "dino-pcihost"
-
-#define DINO_IAR0               0x004
-#define DINO_IODC               0x008
-#define DINO_IRR0               0x00C  /* RO */
-#define DINO_IAR1               0x010
-#define DINO_IRR1               0x014  /* RO */
-#define DINO_IMR                0x018
-#define DINO_IPR                0x01C
-#define DINO_TOC_ADDR           0x020
-#define DINO_ICR                0x024
-#define DINO_ILR                0x028  /* RO */
-#define DINO_IO_COMMAND         0x030  /* WO */
-#define DINO_IO_STATUS          0x034  /* RO */
-#define DINO_IO_CONTROL         0x038
-#define DINO_IO_GSC_ERR_RESP    0x040  /* RO */
-#define DINO_IO_ERR_INFO        0x044  /* RO */
-#define DINO_IO_PCI_ERR_RESP    0x048  /* RO */
-#define DINO_IO_FBB_EN          0x05c
-#define DINO_IO_ADDR_EN         0x060
-#define DINO_PCI_CONFIG_ADDR    0x064
-#define DINO_PCI_CONFIG_DATA    0x068
-#define DINO_PCI_IO_DATA        0x06c
-#define DINO_PCI_MEM_DATA       0x070  /* Dino 3.x only */
-#define DINO_GSC2X_CONFIG       0x7b4  /* RO */
-#define DINO_GMASK              0x800
-#define DINO_PAMR               0x804
-#define DINO_PAPR               0x808
-#define DINO_DAMODE             0x80c
-#define DINO_PCICMD             0x810
-#define DINO_PCISTS             0x814  /* R/WC */
-#define DINO_MLTIM              0x81c
-#define DINO_BRDG_FEAT          0x820
-#define DINO_PCIROR             0x824
-#define DINO_PCIWOR             0x828
-#define DINO_TLTIM              0x830
-
-#define DINO_IRQS         11      /* bits 0-10 are architected */
-#define DINO_IRR_MASK     0x5ff   /* only 10 bits are implemented */
-#define DINO_LOCAL_IRQS   (DINO_IRQS + 1)
-#define DINO_MASK_IRQ(x)  (1 << (x))
-
-#define PCIINTA   0x001
-#define PCIINTB   0x002
-#define PCIINTC   0x004
-#define PCIINTD   0x008
-#define PCIINTE   0x010
-#define PCIINTF   0x020
-#define GSCEXTINT 0x040
-/* #define xxx       0x080 - bit 7 is "default" */
-/* #define xxx    0x100 - bit 8 not used */
-/* #define xxx    0x200 - bit 9 not used */
-#define RS232INT  0x400
-
-#define DINO_MEM_CHUNK_SIZE (8 * MiB)
-
-OBJECT_DECLARE_SIMPLE_TYPE(DinoState, DINO_PCI_HOST_BRIDGE)
-
-#define DINO800_REGS (1 + (DINO_TLTIM - DINO_GMASK) / 4)
-static const uint32_t reg800_keep_bits[DINO800_REGS] = {
-    MAKE_64BIT_MASK(0, 1),  /* GMASK */
-    MAKE_64BIT_MASK(0, 7),  /* PAMR */
-    MAKE_64BIT_MASK(0, 7),  /* PAPR */
-    MAKE_64BIT_MASK(0, 8),  /* DAMODE */
-    MAKE_64BIT_MASK(0, 7),  /* PCICMD */
-    MAKE_64BIT_MASK(0, 9),  /* PCISTS */
-    MAKE_64BIT_MASK(0, 32), /* Undefined */
-    MAKE_64BIT_MASK(0, 8),  /* MLTIM */
-    MAKE_64BIT_MASK(0, 30), /* BRDG_FEAT */
-    MAKE_64BIT_MASK(0, 24), /* PCIROR */
-    MAKE_64BIT_MASK(0, 22), /* PCIWOR */
-    MAKE_64BIT_MASK(0, 32), /* Undocumented */
-    MAKE_64BIT_MASK(0, 9),  /* TLTIM */
-};
-
-struct DinoState {
-    PCIHostState parent_obj;
-
-    /*
-     * PCI_CONFIG_ADDR is parent_obj.config_reg, via pci_host_conf_be_ops,
-     * so that we can map PCI_CONFIG_DATA to pci_host_data_be_ops.
-     */
-    uint32_t config_reg_dino; /* keep original copy, including 2 lowest bits */
-
-    uint32_t iar0;
-    uint32_t iar1;
-    uint32_t imr;
-    uint32_t ipr;
-    uint32_t icr;
-    uint32_t ilr;
-    uint32_t io_fbb_en;
-    uint32_t io_addr_en;
-    uint32_t io_control;
-    uint32_t toc_addr;
-
-    uint32_t reg800[DINO800_REGS];
-
-    MemoryRegion this_mem;
-    MemoryRegion pci_mem;
-    MemoryRegion pci_mem_alias[32];
-
-    MemoryRegion *memory_as;
-
-    AddressSpace bm_as;
-    MemoryRegion bm;
-    MemoryRegion bm_ram_alias;
-    MemoryRegion bm_pci_alias;
-    MemoryRegion bm_cpu_alias;
-};
-
 /*
  * Dino can forward memory accesses from the CPU in the range between
  * 0xf0800000 and 0xff000000 to the PCI bus.
diff --git a/hw/hppa/dino.h b/hw/hppa/dino.h
new file mode 100644
index 0000000000..1a26667377
--- /dev/null
+++ b/hw/hppa/dino.h
@@ -0,0 +1,127 @@
+/*
+ * HP-PARISC Dino PCI chipset emulation, as in B160L and similiar machines
+ *
+ * (C) 2017-2019 by Helge Deller <deller@gmx.de>
+ *
+ * This work is licensed under the GNU GPL license version 2 or later.
+ *
+ * Documentation available at:
+ * https://parisc.wiki.kernel.org/images-parisc/9/91/Dino_ers.pdf
+ * https://parisc.wiki.kernel.org/images-parisc/7/70/Dino_3_1_Errata.pdf
+ */
+
+#ifndef DINO_H
+#define DINO_H
+
+#include "hw/pci/pci_host.h"
+
+#define TYPE_DINO_PCI_HOST_BRIDGE "dino-pcihost"
+OBJECT_DECLARE_SIMPLE_TYPE(DinoState, DINO_PCI_HOST_BRIDGE)
+
+#define DINO_IAR0               0x004
+#define DINO_IODC               0x008
+#define DINO_IRR0               0x00C  /* RO */
+#define DINO_IAR1               0x010
+#define DINO_IRR1               0x014  /* RO */
+#define DINO_IMR                0x018
+#define DINO_IPR                0x01C
+#define DINO_TOC_ADDR           0x020
+#define DINO_ICR                0x024
+#define DINO_ILR                0x028  /* RO */
+#define DINO_IO_COMMAND         0x030  /* WO */
+#define DINO_IO_STATUS          0x034  /* RO */
+#define DINO_IO_CONTROL         0x038
+#define DINO_IO_GSC_ERR_RESP    0x040  /* RO */
+#define DINO_IO_ERR_INFO        0x044  /* RO */
+#define DINO_IO_PCI_ERR_RESP    0x048  /* RO */
+#define DINO_IO_FBB_EN          0x05c
+#define DINO_IO_ADDR_EN         0x060
+#define DINO_PCI_CONFIG_ADDR    0x064
+#define DINO_PCI_CONFIG_DATA    0x068
+#define DINO_PCI_IO_DATA        0x06c
+#define DINO_PCI_MEM_DATA       0x070  /* Dino 3.x only */
+#define DINO_GSC2X_CONFIG       0x7b4  /* RO */
+#define DINO_GMASK              0x800
+#define DINO_PAMR               0x804
+#define DINO_PAPR               0x808
+#define DINO_DAMODE             0x80c
+#define DINO_PCICMD             0x810
+#define DINO_PCISTS             0x814  /* R/WC */
+#define DINO_MLTIM              0x81c
+#define DINO_BRDG_FEAT          0x820
+#define DINO_PCIROR             0x824
+#define DINO_PCIWOR             0x828
+#define DINO_TLTIM              0x830
+
+#define DINO_IRQS         11      /* bits 0-10 are architected */
+#define DINO_IRR_MASK     0x5ff   /* only 10 bits are implemented */
+#define DINO_LOCAL_IRQS   (DINO_IRQS + 1)
+#define DINO_MASK_IRQ(x)  (1 << (x))
+
+#define PCIINTA   0x001
+#define PCIINTB   0x002
+#define PCIINTC   0x004
+#define PCIINTD   0x008
+#define PCIINTE   0x010
+#define PCIINTF   0x020
+#define GSCEXTINT 0x040
+/* #define xxx       0x080 - bit 7 is "default" */
+/* #define xxx    0x100 - bit 8 not used */
+/* #define xxx    0x200 - bit 9 not used */
+#define RS232INT  0x400
+
+#define DINO_MEM_CHUNK_SIZE (8 * MiB)
+
+#define DINO800_REGS (1 + (DINO_TLTIM - DINO_GMASK) / 4)
+static const uint32_t reg800_keep_bits[DINO800_REGS] = {
+    MAKE_64BIT_MASK(0, 1),  /* GMASK */
+    MAKE_64BIT_MASK(0, 7),  /* PAMR */
+    MAKE_64BIT_MASK(0, 7),  /* PAPR */
+    MAKE_64BIT_MASK(0, 8),  /* DAMODE */
+    MAKE_64BIT_MASK(0, 7),  /* PCICMD */
+    MAKE_64BIT_MASK(0, 9),  /* PCISTS */
+    MAKE_64BIT_MASK(0, 32), /* Undefined */
+    MAKE_64BIT_MASK(0, 8),  /* MLTIM */
+    MAKE_64BIT_MASK(0, 30), /* BRDG_FEAT */
+    MAKE_64BIT_MASK(0, 24), /* PCIROR */
+    MAKE_64BIT_MASK(0, 22), /* PCIWOR */
+    MAKE_64BIT_MASK(0, 32), /* Undocumented */
+    MAKE_64BIT_MASK(0, 9),  /* TLTIM */
+};
+
+struct DinoState {
+    PCIHostState parent_obj;
+
+    /*
+     * PCI_CONFIG_ADDR is parent_obj.config_reg, via pci_host_conf_be_ops,
+     * so that we can map PCI_CONFIG_DATA to pci_host_data_be_ops.
+     */
+    uint32_t config_reg_dino; /* keep original copy, including 2 lowest bits */
+
+    uint32_t iar0;
+    uint32_t iar1;
+    uint32_t imr;
+    uint32_t ipr;
+    uint32_t icr;
+    uint32_t ilr;
+    uint32_t io_fbb_en;
+    uint32_t io_addr_en;
+    uint32_t io_control;
+    uint32_t toc_addr;
+
+    uint32_t reg800[DINO800_REGS];
+
+    MemoryRegion this_mem;
+    MemoryRegion pci_mem;
+    MemoryRegion pci_mem_alias[32];
+
+    MemoryRegion *memory_as;
+
+    AddressSpace bm_as;
+    MemoryRegion bm;
+    MemoryRegion bm_ram_alias;
+    MemoryRegion bm_pci_alias;
+    MemoryRegion bm_cpu_alias;
+};
+
+#endif
diff --git a/hw/hppa/hppa_sys.h b/hw/hppa/hppa_sys.h
index 0b18271cc9..9d8b28ec01 100644
--- a/hw/hppa/hppa_sys.h
+++ b/hw/hppa/hppa_sys.h
@@ -14,8 +14,6 @@ PCIBus *dino_init(MemoryRegion *, qemu_irq *, qemu_irq *);
 DeviceState *lasi_init(MemoryRegion *);
 #define enable_lasi_lan()       0
 
-#define TYPE_DINO_PCI_HOST_BRIDGE "dino-pcihost"
-
 /* hppa_pci.c.  */
 extern const MemoryRegionOps hppa_pci_ignore_ops;
 extern const MemoryRegionOps hppa_pci_conf1_ops;

From 0d98fbb57d4facff538a8155d1f3fa1eaf0111f0 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:22 +0100
Subject: [PATCH 133/935] hppa: use new CONFIG_HPPA_B160L option instead of
 CONFIG_DINO to build hppa machine

DINO refers to the GSC-PCI bridge device which will soon be handled separately,
however the QEMU HPPA machine is actually based upon the HPPA B160L as indicated
by the Linux kernel dmesg output when booted in qemu-system-hppa and the QEMU
MAINTAINERS file.

Update the machine configuration to use CONFIG_HPPA_B160L instead of CONFIG_DINO
and also update the machine description accordingly.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-13-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 configs/devices/hppa-softmmu/default.mak | 2 +-
 hw/hppa/Kconfig                          | 2 +-
 hw/hppa/machine.c                        | 2 +-
 hw/hppa/meson.build                      | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/configs/devices/hppa-softmmu/default.mak b/configs/devices/hppa-softmmu/default.mak
index b64c5eb3ff..b0364bb88f 100644
--- a/configs/devices/hppa-softmmu/default.mak
+++ b/configs/devices/hppa-softmmu/default.mak
@@ -6,4 +6,4 @@
 
 # Boards:
 #
-CONFIG_DINO=y
+CONFIG_HPPA_B160L=y
diff --git a/hw/hppa/Kconfig b/hw/hppa/Kconfig
index 22948db025..8d64ead217 100644
--- a/hw/hppa/Kconfig
+++ b/hw/hppa/Kconfig
@@ -1,4 +1,4 @@
-config DINO
+config HPPA_B160L
     bool
     imply PCI_DEVICES
     imply E1000_PCI
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index f7595c0857..971d7ffcfe 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -366,7 +366,7 @@ static void hppa_nmi(NMIState *n, int cpu_index, Error **errp)
 
 static void machine_hppa_machine_init(MachineClass *mc)
 {
-    mc->desc = "HPPA generic machine";
+    mc->desc = "HPPA B160L machine";
     mc->default_cpu_type = TYPE_HPPA_CPU;
     mc->init = machine_hppa_init;
     mc->reset = hppa_machine_reset;
diff --git a/hw/hppa/meson.build b/hw/hppa/meson.build
index 1deae83aee..32072bf204 100644
--- a/hw/hppa/meson.build
+++ b/hw/hppa/meson.build
@@ -1,4 +1,4 @@
 hppa_ss = ss.source_set()
-hppa_ss.add(when: 'CONFIG_DINO', if_true: files('pci.c', 'machine.c', 'dino.c', 'lasi.c'))
+hppa_ss.add(when: 'CONFIG_HPPA_B160L', if_true: files('pci.c', 'machine.c', 'dino.c', 'lasi.c'))
 
 hw_arch += {'hppa': hppa_ss}

From 05245daf47b7c645d9952b476d63531f6d8aa7f8 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:23 +0100
Subject: [PATCH 134/935] dino: change dino_init() to return the DINO device
 instead of PCIBus

This is in preparation for using more qdev APIs during the configuration of the
HPPA generic machine.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-14-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c     | 8 +++-----
 hw/hppa/hppa_sys.h | 3 ++-
 hw/hppa/machine.c  | 6 ++++--
 3 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index 01546ff6fc..6c488c908e 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -409,12 +409,11 @@ static void dino_set_serial_irq(void *opaque, int irq, int level)
     dino_set_irq(opaque, 10, level);
 }
 
-PCIBus *dino_init(MemoryRegion *addr_space,
-                  qemu_irq *p_rtc_irq, qemu_irq *p_ser_irq)
+DinoState *dino_init(MemoryRegion *addr_space,
+                     qemu_irq *p_rtc_irq, qemu_irq *p_ser_irq)
 {
     DeviceState *dev;
     DinoState *s;
-    PCIBus *pci_bus;
 
     dev = qdev_new(TYPE_DINO_PCI_HOST_BRIDGE);
     object_property_set_link(OBJECT(dev), "memory-as", OBJECT(addr_space),
@@ -428,8 +427,7 @@ PCIBus *dino_init(MemoryRegion *addr_space,
     *p_rtc_irq = qemu_allocate_irq(dino_set_timer_irq, s, 0);
     *p_ser_irq = qemu_allocate_irq(dino_set_serial_irq, s, 0);
 
-    pci_bus = PCI_BUS(qdev_get_child_bus(dev, "pci"));
-    return pci_bus;
+    return s;
 }
 
 static void dino_pcihost_reset(DeviceState *dev)
diff --git a/hw/hppa/hppa_sys.h b/hw/hppa/hppa_sys.h
index 9d8b28ec01..88773d2c35 100644
--- a/hw/hppa/hppa_sys.h
+++ b/hw/hppa/hppa_sys.h
@@ -9,8 +9,9 @@
 #include "hw/intc/i8259.h"
 
 #include "hppa_hardware.h"
+#include "dino.h"
 
-PCIBus *dino_init(MemoryRegion *, qemu_irq *, qemu_irq *);
+DinoState *dino_init(MemoryRegion *, qemu_irq *, qemu_irq *);
 DeviceState *lasi_init(MemoryRegion *);
 #define enable_lasi_lan()       0
 
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 971d7ffcfe..94bc1b202a 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -17,6 +17,7 @@
 #include "hw/char/serial.h"
 #include "hw/net/lasi_82596.h"
 #include "hw/nmi.h"
+#include "dino.h"
 #include "hppa_sys.h"
 #include "qemu/units.h"
 #include "qapi/error.h"
@@ -126,7 +127,7 @@ static void machine_hppa_init(MachineState *machine)
     const char *kernel_filename = machine->kernel_filename;
     const char *kernel_cmdline = machine->kernel_cmdline;
     const char *initrd_filename = machine->initrd_filename;
-    DeviceState *dev;
+    DeviceState *dev, *dino_dev;
     PCIBus *pci_bus;
     ISABus *isa_bus;
     qemu_irq rtc_irq, serial_irq;
@@ -166,7 +167,8 @@ static void machine_hppa_init(MachineState *machine)
     lasi_init(addr_space);
 
     /* Init Dino (PCI host bus chip).  */
-    pci_bus = dino_init(addr_space, &rtc_irq, &serial_irq);
+    dino_dev = DEVICE(dino_init(addr_space, &rtc_irq, &serial_irq));
+    pci_bus = PCI_BUS(qdev_get_child_bus(dino_dev, "pci"));
     assert(pci_bus);
 
     /* Create ISA bus. */

From efdb3ce2de8e5d3e27fbb7d403616be71b950130 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:24 +0100
Subject: [PATCH 135/935] machine.c: map DINO device during board configuration

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-15-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c    | 3 ---
 hw/hppa/machine.c | 3 +++
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index 6c488c908e..82f301653b 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -421,9 +421,6 @@ DinoState *dino_init(MemoryRegion *addr_space,
     s = DINO_PCI_HOST_BRIDGE(dev);
     sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
 
-    memory_region_add_subregion(addr_space, DINO_HPA,
-                                sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 0));
-
     *p_rtc_irq = qemu_allocate_irq(dino_set_timer_irq, s, 0);
     *p_ser_irq = qemu_allocate_irq(dino_set_serial_irq, s, 0);
 
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 94bc1b202a..5a490a9c37 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -168,6 +168,9 @@ static void machine_hppa_init(MachineState *machine)
 
     /* Init Dino (PCI host bus chip).  */
     dino_dev = DEVICE(dino_init(addr_space, &rtc_irq, &serial_irq));
+    memory_region_add_subregion(addr_space, DINO_HPA,
+                                sysbus_mmio_get_region(
+                                    SYS_BUS_DEVICE(dino_dev), 0));
     pci_bus = PCI_BUS(qdev_get_child_bus(dino_dev, "pci"));
     assert(pci_bus);
 

From 10c5264172bea2d7f55e013095a80000f00d05c0 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:25 +0100
Subject: [PATCH 136/935] dino.h: add defines for DINO IRQ numbers

This is to allow the DINO IRQs to be defined as qdev GPIOs.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-16-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/hw/hppa/dino.h b/hw/hppa/dino.h
index 1a26667377..70fb8c52c8 100644
--- a/hw/hppa/dino.h
+++ b/hw/hppa/dino.h
@@ -58,6 +58,18 @@ OBJECT_DECLARE_SIMPLE_TYPE(DinoState, DINO_PCI_HOST_BRIDGE)
 #define DINO_LOCAL_IRQS   (DINO_IRQS + 1)
 #define DINO_MASK_IRQ(x)  (1 << (x))
 
+#define DINO_IRQ_PCIINTA   0
+#define DINO_IRQ_PCIINTB   1
+#define DINO_IRQ_PCIINTC   2
+#define DINO_IRQ_PCIINTD   3
+#define DINO_IRQ_PCIINTE   4
+#define DINO_IRQ_PCIINTF   5
+#define DINO_IRQ_GSCEXTINT 6
+#define DINO_IRQ_BUSERRINT 7
+#define DINO_IRQ_PS2INT    8
+#define DINO_IRQ_UNUSED    9
+#define DINO_IRQ_RS232INT  10
+
 #define PCIINTA   0x001
 #define PCIINTB   0x002
 #define PCIINTC   0x004

From 4b5faaf94ff3cf47a6c63393ed1332ef4928dadf Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:26 +0100
Subject: [PATCH 137/935] dino: define IRQ inputs as qdev GPIOs

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-17-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c | 2 ++
 hw/hppa/dino.h | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index 82f301653b..f58aebf94d 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -513,6 +513,8 @@ static void dino_pcihost_init(Object *obj)
     pci_setup_iommu(phb->bus, dino_pcihost_set_iommu, s);
 
     sysbus_init_mmio(sbd, &s->this_mem);
+
+    qdev_init_gpio_in(DEVICE(obj), dino_set_irq, DINO_IRQS);
 }
 
 static Property dino_pcihost_properties[] = {
diff --git a/hw/hppa/dino.h b/hw/hppa/dino.h
index 70fb8c52c8..ca380515f2 100644
--- a/hw/hppa/dino.h
+++ b/hw/hppa/dino.h
@@ -134,6 +134,8 @@ struct DinoState {
     MemoryRegion bm_ram_alias;
     MemoryRegion bm_pci_alias;
     MemoryRegion bm_cpu_alias;
+
+    qemu_irq irqs[DINO_IRQS];
 };
 
 #endif

From a4b74c1924b79b91ca6aeb20cdd39156b5c7b4f7 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:27 +0100
Subject: [PATCH 138/935] dino: wire up serial IRQ using a qdev GPIO in
 machine.c

This makes it unnecessary to allocate a separate IRQ for the serial port.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-18-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c     | 8 +-------
 hw/hppa/hppa_sys.h | 2 +-
 hw/hppa/machine.c  | 7 ++++---
 3 files changed, 6 insertions(+), 11 deletions(-)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index f58aebf94d..80ffe27188 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -404,13 +404,8 @@ static void dino_set_timer_irq(void *opaque, int irq, int level)
     /* ??? Not connected.  */
 }
 
-static void dino_set_serial_irq(void *opaque, int irq, int level)
-{
-    dino_set_irq(opaque, 10, level);
-}
-
 DinoState *dino_init(MemoryRegion *addr_space,
-                     qemu_irq *p_rtc_irq, qemu_irq *p_ser_irq)
+                     qemu_irq *p_rtc_irq)
 {
     DeviceState *dev;
     DinoState *s;
@@ -422,7 +417,6 @@ DinoState *dino_init(MemoryRegion *addr_space,
     sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
 
     *p_rtc_irq = qemu_allocate_irq(dino_set_timer_irq, s, 0);
-    *p_ser_irq = qemu_allocate_irq(dino_set_serial_irq, s, 0);
 
     return s;
 }
diff --git a/hw/hppa/hppa_sys.h b/hw/hppa/hppa_sys.h
index 88773d2c35..c238e7ebc8 100644
--- a/hw/hppa/hppa_sys.h
+++ b/hw/hppa/hppa_sys.h
@@ -11,7 +11,7 @@
 #include "hppa_hardware.h"
 #include "dino.h"
 
-DinoState *dino_init(MemoryRegion *, qemu_irq *, qemu_irq *);
+DinoState *dino_init(MemoryRegion *, qemu_irq *);
 DeviceState *lasi_init(MemoryRegion *);
 #define enable_lasi_lan()       0
 
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 5a490a9c37..f757aecda0 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -130,7 +130,7 @@ static void machine_hppa_init(MachineState *machine)
     DeviceState *dev, *dino_dev;
     PCIBus *pci_bus;
     ISABus *isa_bus;
-    qemu_irq rtc_irq, serial_irq;
+    qemu_irq rtc_irq;
     char *firmware_filename;
     uint64_t firmware_low, firmware_high;
     long size;
@@ -167,7 +167,7 @@ static void machine_hppa_init(MachineState *machine)
     lasi_init(addr_space);
 
     /* Init Dino (PCI host bus chip).  */
-    dino_dev = DEVICE(dino_init(addr_space, &rtc_irq, &serial_irq));
+    dino_dev = DEVICE(dino_init(addr_space, &rtc_irq));
     memory_region_add_subregion(addr_space, DINO_HPA,
                                 sysbus_mmio_get_region(
                                     SYS_BUS_DEVICE(dino_dev), 0));
@@ -184,7 +184,8 @@ static void machine_hppa_init(MachineState *machine)
     /* Serial code setup.  */
     if (serial_hd(0)) {
         uint32_t addr = DINO_UART_HPA + 0x800;
-        serial_mm_init(addr_space, addr, 0, serial_irq,
+        serial_mm_init(addr_space, addr, 0,
+                       qdev_get_gpio_in(dino_dev, DINO_IRQ_RS232INT),
                        115200, serial_hd(0), DEVICE_BIG_ENDIAN);
     }
 

From 36f9bbdb1717eeb9b10cbfa3135c942c43d13d25 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:28 +0100
Subject: [PATCH 139/935] dino: remove unused dino_set_timer_irq() IRQ handler

According to the comments in dino.c the timer IRQ is unused, so remove the empty
dino_set_timer_irq() handler function and simply pass NULL to mc146818_rtc_init()
in machine.c instead.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-19-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c     | 10 +---------
 hw/hppa/hppa_sys.h |  2 +-
 hw/hppa/machine.c  |  5 ++---
 3 files changed, 4 insertions(+), 13 deletions(-)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index 80ffe27188..d8baf139d6 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -399,13 +399,7 @@ static int dino_pci_map_irq(PCIDevice *d, int irq_num)
     return slot & 0x03;
 }
 
-static void dino_set_timer_irq(void *opaque, int irq, int level)
-{
-    /* ??? Not connected.  */
-}
-
-DinoState *dino_init(MemoryRegion *addr_space,
-                     qemu_irq *p_rtc_irq)
+DinoState *dino_init(MemoryRegion *addr_space)
 {
     DeviceState *dev;
     DinoState *s;
@@ -416,8 +410,6 @@ DinoState *dino_init(MemoryRegion *addr_space,
     s = DINO_PCI_HOST_BRIDGE(dev);
     sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
 
-    *p_rtc_irq = qemu_allocate_irq(dino_set_timer_irq, s, 0);
-
     return s;
 }
 
diff --git a/hw/hppa/hppa_sys.h b/hw/hppa/hppa_sys.h
index c238e7ebc8..ebedecf1c8 100644
--- a/hw/hppa/hppa_sys.h
+++ b/hw/hppa/hppa_sys.h
@@ -11,7 +11,7 @@
 #include "hppa_hardware.h"
 #include "dino.h"
 
-DinoState *dino_init(MemoryRegion *, qemu_irq *);
+DinoState *dino_init(MemoryRegion *);
 DeviceState *lasi_init(MemoryRegion *);
 #define enable_lasi_lan()       0
 
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index f757aecda0..49ecb971be 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -130,7 +130,6 @@ static void machine_hppa_init(MachineState *machine)
     DeviceState *dev, *dino_dev;
     PCIBus *pci_bus;
     ISABus *isa_bus;
-    qemu_irq rtc_irq;
     char *firmware_filename;
     uint64_t firmware_low, firmware_high;
     long size;
@@ -167,7 +166,7 @@ static void machine_hppa_init(MachineState *machine)
     lasi_init(addr_space);
 
     /* Init Dino (PCI host bus chip).  */
-    dino_dev = DEVICE(dino_init(addr_space, &rtc_irq));
+    dino_dev = DEVICE(dino_init(addr_space));
     memory_region_add_subregion(addr_space, DINO_HPA,
                                 sysbus_mmio_get_region(
                                     SYS_BUS_DEVICE(dino_dev), 0));
@@ -179,7 +178,7 @@ static void machine_hppa_init(MachineState *machine)
     assert(isa_bus);
 
     /* Realtime clock, used by firmware for PDC_TOD call. */
-    mc146818_rtc_init(isa_bus, 2000, rtc_irq);
+    mc146818_rtc_init(isa_bus, 2000, NULL);
 
     /* Serial code setup.  */
     if (serial_hd(0)) {

From 0d068996310c33c2f40dd347746304054fdfa16d Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:29 +0100
Subject: [PATCH 140/935] hppa: move dino_init() from dino.c to machine.c

Now that dino_init() is completely decoupled from dino.c it can be moved to
machine.c with the rest of the board configuration.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-20-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c     | 14 --------------
 hw/hppa/hppa_sys.h |  2 --
 hw/hppa/machine.c  | 12 ++++++++++++
 3 files changed, 12 insertions(+), 16 deletions(-)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index d8baf139d6..6d12c385aa 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -399,20 +399,6 @@ static int dino_pci_map_irq(PCIDevice *d, int irq_num)
     return slot & 0x03;
 }
 
-DinoState *dino_init(MemoryRegion *addr_space)
-{
-    DeviceState *dev;
-    DinoState *s;
-
-    dev = qdev_new(TYPE_DINO_PCI_HOST_BRIDGE);
-    object_property_set_link(OBJECT(dev), "memory-as", OBJECT(addr_space),
-                             &error_fatal);
-    s = DINO_PCI_HOST_BRIDGE(dev);
-    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
-
-    return s;
-}
-
 static void dino_pcihost_reset(DeviceState *dev)
 {
     DinoState *s = DINO_PCI_HOST_BRIDGE(dev);
diff --git a/hw/hppa/hppa_sys.h b/hw/hppa/hppa_sys.h
index ebedecf1c8..7c31dd8396 100644
--- a/hw/hppa/hppa_sys.h
+++ b/hw/hppa/hppa_sys.h
@@ -9,9 +9,7 @@
 #include "hw/intc/i8259.h"
 
 #include "hppa_hardware.h"
-#include "dino.h"
 
-DinoState *dino_init(MemoryRegion *);
 DeviceState *lasi_init(MemoryRegion *);
 #define enable_lasi_lan()       0
 
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 49ecb971be..4b753fa346 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -122,6 +122,18 @@ static FWCfgState *create_fw_cfg(MachineState *ms)
     return fw_cfg;
 }
 
+static DinoState *dino_init(MemoryRegion *addr_space)
+{
+    DeviceState *dev;
+
+    dev = qdev_new(TYPE_DINO_PCI_HOST_BRIDGE);
+    object_property_set_link(OBJECT(dev), "memory-as", OBJECT(addr_space),
+                             &error_fatal);
+    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
+
+    return DINO_PCI_HOST_BRIDGE(dev);
+}
+
 static void machine_hppa_init(MachineState *machine)
 {
     const char *kernel_filename = machine->kernel_filename;

From 2fb11c7cac7f6074f42d821cfbff4d7b80b3ab6f Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:30 +0100
Subject: [PATCH 141/935] dino: use numerical constant for iar0 and iar1 reset
 values

This is to allow us to decouple the DINO device from the board logic. The choice
of using a hard-coded constant (along with a comment) is to match how this is
already done for toc_addr. If it is decided later that these values need to be
configurable then they can easily be converted to qdev properties.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-21-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index 6d12c385aa..aa7f812e22 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -403,7 +403,7 @@ static void dino_pcihost_reset(DeviceState *dev)
 {
     DinoState *s = DINO_PCI_HOST_BRIDGE(dev);
 
-    s->iar0 = s->iar1 = CPU_HPA + 3;
+    s->iar0 = s->iar1 = 0xFFFB0000 + 3; /* CPU_HPA + 3 */
     s->toc_addr = 0xFFFA0030; /* IO_COMMAND of CPU */
 }
 

From e111f288d9264dae070bd144a60f43162e34a7be Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:31 +0100
Subject: [PATCH 142/935] dino: move DINO HPA constants from hppa_hardware.h to
 dino.h

This is to allow us to decouple the DINO device from the board logic.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-22-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/dino.h          | 5 +++++
 hw/hppa/hppa_hardware.h | 5 -----
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/hw/hppa/dino.h b/hw/hppa/dino.h
index ca380515f2..a1b0184940 100644
--- a/hw/hppa/dino.h
+++ b/hw/hppa/dino.h
@@ -101,6 +101,11 @@ static const uint32_t reg800_keep_bits[DINO800_REGS] = {
     MAKE_64BIT_MASK(0, 9),  /* TLTIM */
 };
 
+/* offsets to DINO HPA: */
+#define DINO_PCI_ADDR           0x064
+#define DINO_CONFIG_DATA        0x068
+#define DINO_IO_DATA            0x06c
+
 struct DinoState {
     PCIHostState parent_obj;
 
diff --git a/hw/hppa/hppa_hardware.h b/hw/hppa/hppa_hardware.h
index 5edf577563..8b6b9222cb 100644
--- a/hw/hppa/hppa_hardware.h
+++ b/hw/hppa/hppa_hardware.h
@@ -30,11 +30,6 @@
 #define PCI_HPA         DINO_HPA        /* PCI bus */
 #define IDE_HPA         0xf9000000      /* Boot disc controller */
 
-/* offsets to DINO HPA: */
-#define DINO_PCI_ADDR           0x064
-#define DINO_CONFIG_DATA        0x068
-#define DINO_IO_DATA            0x06c
-
 #define PORT_PCI_CMD    (PCI_HPA + DINO_PCI_ADDR)
 #define PORT_PCI_DATA   (PCI_HPA + DINO_CONFIG_DATA)
 

From 0db9350e6e5aba3dc864d36756b01fb0922ad7cb Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:32 +0100
Subject: [PATCH 143/935] dino: move from hw/hppa to hw/pci-host

Move the DINO device implementation from hw/hppa to hw/pci-host so that it is
located with all the other PCI host bridges.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-23-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 MAINTAINERS                             | 2 ++
 hw/hppa/Kconfig                         | 2 +-
 hw/hppa/machine.c                       | 2 +-
 hw/hppa/meson.build                     | 2 +-
 hw/hppa/trace-events                    | 5 -----
 hw/pci-host/Kconfig                     | 4 ++++
 hw/{hppa => pci-host}/dino.c            | 3 +--
 hw/pci-host/meson.build                 | 3 +++
 hw/pci-host/trace-events                | 5 +++++
 {hw/hppa => include/hw/pci-host}/dino.h | 0
 10 files changed, 18 insertions(+), 10 deletions(-)
 rename hw/{hppa => pci-host}/dino.c (99%)
 rename {hw/hppa => include/hw/pci-host}/dino.h (100%)

diff --git a/MAINTAINERS b/MAINTAINERS
index 218c9459b6..59210b093a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1113,7 +1113,9 @@ S: Odd Fixes
 F: configs/devices/hppa-softmmu/default.mak
 F: hw/hppa/
 F: hw/net/*i82596*
+F: hw/pci-host/dino.c
 F: include/hw/net/lasi_82596.h
+F: include/hw/pci-host/dino.h
 F: pc-bios/hppa-firmware.img
 
 M68K Machines
diff --git a/hw/hppa/Kconfig b/hw/hppa/Kconfig
index 8d64ead217..724380ecec 100644
--- a/hw/hppa/Kconfig
+++ b/hw/hppa/Kconfig
@@ -3,7 +3,7 @@ config HPPA_B160L
     imply PCI_DEVICES
     imply E1000_PCI
     imply VIRTIO_VGA
-    select PCI
+    select DINO
     select SERIAL
     select ISA_BUS
     select I8259
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 4b753fa346..c847febe5d 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -17,7 +17,7 @@
 #include "hw/char/serial.h"
 #include "hw/net/lasi_82596.h"
 #include "hw/nmi.h"
-#include "dino.h"
+#include "hw/pci-host/dino.h"
 #include "hppa_sys.h"
 #include "qemu/units.h"
 #include "qapi/error.h"
diff --git a/hw/hppa/meson.build b/hw/hppa/meson.build
index 32072bf204..a6f9db455c 100644
--- a/hw/hppa/meson.build
+++ b/hw/hppa/meson.build
@@ -1,4 +1,4 @@
 hppa_ss = ss.source_set()
-hppa_ss.add(when: 'CONFIG_HPPA_B160L', if_true: files('pci.c', 'machine.c', 'dino.c', 'lasi.c'))
+hppa_ss.add(when: 'CONFIG_HPPA_B160L', if_true: files('pci.c', 'machine.c', 'lasi.c'))
 
 hw_arch += {'hppa': hppa_ss}
diff --git a/hw/hppa/trace-events b/hw/hppa/trace-events
index 3f42be9056..871a473771 100644
--- a/hw/hppa/trace-events
+++ b/hw/hppa/trace-events
@@ -3,11 +3,6 @@
 # pci.c
 hppa_pci_iack_write(void) ""
 
-# dino.c
-dino_chip_mem_valid(uint64_t addr, uint32_t val) "access to addr 0x%"PRIx64" is %d"
-dino_chip_read(uint64_t addr, uint32_t val) "addr 0x%"PRIx64" val 0x%08x"
-dino_chip_write(uint64_t addr, uint32_t val) "addr 0x%"PRIx64" val 0x%08x"
-
 # lasi.c
 lasi_chip_mem_valid(uint64_t addr, uint32_t val) "access to addr 0x%"PRIx64" is %d"
 lasi_chip_read(uint64_t addr, uint32_t val) "addr 0x%"PRIx64" val 0x%08x"
diff --git a/hw/pci-host/Kconfig b/hw/pci-host/Kconfig
index 2b5f7d58cc..38fd2ee8f3 100644
--- a/hw/pci-host/Kconfig
+++ b/hw/pci-host/Kconfig
@@ -77,3 +77,7 @@ config MV64361
     bool
     select PCI
     select I8259
+
+config DINO
+    bool
+    select PCI
diff --git a/hw/hppa/dino.c b/hw/pci-host/dino.c
similarity index 99%
rename from hw/hppa/dino.c
rename to hw/pci-host/dino.c
index aa7f812e22..f257c24e64 100644
--- a/hw/hppa/dino.c
+++ b/hw/pci-host/dino.c
@@ -18,9 +18,8 @@
 #include "hw/pci/pci.h"
 #include "hw/pci/pci_bus.h"
 #include "hw/qdev-properties.h"
-#include "dino.h"
+#include "hw/pci-host/dino.h"
 #include "migration/vmstate.h"
-#include "hppa_sys.h"
 #include "trace.h"
 #include "qom/object.h"
 
diff --git a/hw/pci-host/meson.build b/hw/pci-host/meson.build
index 4c4f39c15c..c07596d0d1 100644
--- a/hw/pci-host/meson.build
+++ b/hw/pci-host/meson.build
@@ -25,6 +25,9 @@ pci_ss.add(when: 'CONFIG_MV64361', if_true: files('mv64361.c'))
 # ARM devices
 pci_ss.add(when: 'CONFIG_VERSATILE_PCI', if_true: files('versatile.c'))
 
+# HPPA devices
+pci_ss.add(when: 'CONFIG_DINO', if_true: files('dino.c'))
+
 softmmu_ss.add_all(when: 'CONFIG_PCI', if_true: pci_ss)
 
 specific_ss.add(when: 'CONFIG_PCI_POWERNV', if_true: files(
diff --git a/hw/pci-host/trace-events b/hw/pci-host/trace-events
index 6e5d8d3355..437e66ff50 100644
--- a/hw/pci-host/trace-events
+++ b/hw/pci-host/trace-events
@@ -34,3 +34,8 @@ unin_read(uint64_t addr, uint64_t value) "addr=0x%" PRIx64 " val=0x%"PRIx64
 pnv_phb4_xive_notify(uint64_t notif_port, uint64_t data) "notif=@0x%"PRIx64" data=0x%"PRIx64
 pnv_phb4_xive_notify_ic(uint64_t addr, uint64_t data) "addr=@0x%"PRIx64" data=0x%"PRIx64
 pnv_phb4_xive_notify_abt(uint64_t notif_port, uint64_t data) "notif=@0x%"PRIx64" data=0x%"PRIx64
+
+# dino.c
+dino_chip_mem_valid(uint64_t addr, uint32_t val) "access to addr 0x%"PRIx64" is %d"
+dino_chip_read(uint64_t addr, uint32_t val) "addr 0x%"PRIx64" val 0x%08x"
+dino_chip_write(uint64_t addr, uint32_t val) "addr 0x%"PRIx64" val 0x%08x"
diff --git a/hw/hppa/dino.h b/include/hw/pci-host/dino.h
similarity index 100%
rename from hw/hppa/dino.h
rename to include/hw/pci-host/dino.h

From 63588da809704c8e674d8ffb01f66597cf5e6150 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:33 +0100
Subject: [PATCH 144/935] lasi: checkpatch fixes

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-24-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/lasi.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index 88c3791eb6..3b77fba1c6 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -169,10 +169,11 @@ static MemTxResult lasi_chip_write_with_attrs(void *opaque, hwaddr addr,
         break;
     case LASI_IMR:
         s->imr = val;
-        if (((val & LASI_IRQ_BITS) != val) && (val != 0xffffffff))
+        if (((val & LASI_IRQ_BITS) != val) && (val != 0xffffffff)) {
             qemu_log_mask(LOG_GUEST_ERROR,
                 "LASI: tried to set invalid %lx IMR value.\n",
                 (unsigned long) val);
+        }
         break;
     case LASI_IPR:
         /* Any write to IPR clears the register. */
@@ -200,8 +201,9 @@ static MemTxResult lasi_chip_write_with_attrs(void *opaque, hwaddr addr,
         break;
 
     case LASI_PCR:
-        if (val == 0x02) /* immediately power off */
+        if (val == 0x02) { /* immediately power off */
             qemu_system_shutdown_request(SHUTDOWN_CAUSE_GUEST_SHUTDOWN);
+        }
         break;
     case LASI_ERRLOG:
         s->errlog = val;

From 46f2594cfca9e888531a5d45a1e33d5446f75578 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:34 +0100
Subject: [PATCH 145/935] lasi: move memory region initialisation to new
 lasi_init() function

Create a new lasi_init() instance initialisation function and move the LASI
memory region initialisation into it. Rename the existing lasi_init() function
to lasi_initfn() for now.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-25-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/hppa_sys.h |  2 +-
 hw/hppa/lasi.c     | 13 ++++++++++---
 hw/hppa/machine.c  |  2 +-
 3 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/hw/hppa/hppa_sys.h b/hw/hppa/hppa_sys.h
index 7c31dd8396..0929dedded 100644
--- a/hw/hppa/hppa_sys.h
+++ b/hw/hppa/hppa_sys.h
@@ -10,7 +10,7 @@
 
 #include "hppa_hardware.h"
 
-DeviceState *lasi_init(MemoryRegion *);
+DeviceState *lasi_initfn(MemoryRegion *);
 #define enable_lasi_lan()       0
 
 /* hppa_pci.c.  */
diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index 3b77fba1c6..c028b7786e 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -296,7 +296,7 @@ static int lasi_get_irq(unsigned long hpa)
     }
 }
 
-DeviceState *lasi_init(MemoryRegion *address_space)
+DeviceState *lasi_initfn(MemoryRegion *address_space)
 {
     DeviceState *dev;
     LasiState *s;
@@ -306,8 +306,6 @@ DeviceState *lasi_init(MemoryRegion *address_space)
     s->iar = CPU_HPA + 3;
 
     /* Lasi access from main memory.  */
-    memory_region_init_io(&s->this_mem, OBJECT(s), &lasi_chip_ops,
-                          s, "lasi", 0x100000);
     memory_region_add_subregion(address_space, LASI_HPA, &s->this_mem);
 
     sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
@@ -347,6 +345,14 @@ DeviceState *lasi_init(MemoryRegion *address_space)
     return dev;
 }
 
+static void lasi_init(Object *obj)
+{
+    LasiState *s = LASI_CHIP(obj);
+
+    memory_region_init_io(&s->this_mem, OBJECT(s), &lasi_chip_ops,
+                          s, "lasi", 0x100000);
+}
+
 static void lasi_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -357,6 +363,7 @@ static void lasi_class_init(ObjectClass *klass, void *data)
 static const TypeInfo lasi_pcihost_info = {
     .name          = TYPE_LASI_CHIP,
     .parent        = TYPE_SYS_BUS_DEVICE,
+    .instance_init = lasi_init,
     .instance_size = sizeof(LasiState),
     .class_init    = lasi_class_init,
 };
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index c847febe5d..2fa55de549 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -175,7 +175,7 @@ static void machine_hppa_init(MachineState *machine)
 
 
     /* Init Lasi chip */
-    lasi_init(addr_space);
+    lasi_initfn(addr_space);
 
     /* Init Dino (PCI host bus chip).  */
     dino_dev = DEVICE(dino_init(addr_space));

From 2683758c7c48718d032ed8bffdeb2c37f3a357c8 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:35 +0100
Subject: [PATCH 146/935] lasi: move register memory mapping from lasi.c to
 machine.c

The device register should be mapped directly by the board code.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-26-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/lasi.c    | 7 +++----
 hw/hppa/machine.c | 7 +++++--
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index c028b7786e..08ebe7c6d9 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -304,10 +304,6 @@ DeviceState *lasi_initfn(MemoryRegion *address_space)
     dev = qdev_new(TYPE_LASI_CHIP);
     s = LASI_CHIP(dev);
     s->iar = CPU_HPA + 3;
-
-    /* Lasi access from main memory.  */
-    memory_region_add_subregion(address_space, LASI_HPA, &s->this_mem);
-
     sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
 
     /* LAN */
@@ -348,9 +344,12 @@ DeviceState *lasi_initfn(MemoryRegion *address_space)
 static void lasi_init(Object *obj)
 {
     LasiState *s = LASI_CHIP(obj);
+    SysBusDevice *sbd = SYS_BUS_DEVICE(obj);
 
     memory_region_init_io(&s->this_mem, OBJECT(s), &lasi_chip_ops,
                           s, "lasi", 0x100000);
+
+    sysbus_init_mmio(sbd, &s->this_mem);
 }
 
 static void lasi_class_init(ObjectClass *klass, void *data)
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 2fa55de549..6618bce435 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -139,7 +139,7 @@ static void machine_hppa_init(MachineState *machine)
     const char *kernel_filename = machine->kernel_filename;
     const char *kernel_cmdline = machine->kernel_cmdline;
     const char *initrd_filename = machine->initrd_filename;
-    DeviceState *dev, *dino_dev;
+    DeviceState *dev, *dino_dev, *lasi_dev;
     PCIBus *pci_bus;
     ISABus *isa_bus;
     char *firmware_filename;
@@ -175,7 +175,10 @@ static void machine_hppa_init(MachineState *machine)
 
 
     /* Init Lasi chip */
-    lasi_initfn(addr_space);
+    lasi_dev = lasi_initfn(addr_space);
+    memory_region_add_subregion(addr_space, LASI_HPA,
+                                sysbus_mmio_get_region(
+                                    SYS_BUS_DEVICE(lasi_dev), 0));
 
     /* Init Dino (PCI host bus chip).  */
     dino_dev = DEVICE(dino_init(addr_space));

From b3cdb7e4b29a4f1824ff6c1c12ddcf65857bfb9e Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:36 +0100
Subject: [PATCH 147/935] lasi: move initialisation of iar and rtc to new
 lasi_reset() function

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-27-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/lasi.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index 08ebe7c6d9..242713c7bd 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -303,7 +303,6 @@ DeviceState *lasi_initfn(MemoryRegion *address_space)
 
     dev = qdev_new(TYPE_LASI_CHIP);
     s = LASI_CHIP(dev);
-    s->iar = CPU_HPA + 3;
     sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
 
     /* LAN */
@@ -319,11 +318,6 @@ DeviceState *lasi_initfn(MemoryRegion *address_space)
     parallel_mm_init(address_space, LASI_LPT_HPA + 0x800, 0,
                      lpt_irq, parallel_hds[0]);
 
-    /* Real time clock (RTC), it's only one 32-bit counter @9000 */
-
-    s->rtc = time(NULL);
-    s->rtc_ref = 0;
-
     if (serial_hd(1)) {
         /* Serial port */
         qemu_irq serial_irq = qemu_allocate_irq(lasi_set_irq, s,
@@ -341,6 +335,17 @@ DeviceState *lasi_initfn(MemoryRegion *address_space)
     return dev;
 }
 
+static void lasi_reset(DeviceState *dev)
+{
+    LasiState *s = LASI_CHIP(dev);
+
+    s->iar = CPU_HPA + 3;
+
+    /* Real time clock (RTC), it's only one 32-bit counter @9000 */
+    s->rtc = time(NULL);
+    s->rtc_ref = 0;
+}
+
 static void lasi_init(Object *obj)
 {
     LasiState *s = LASI_CHIP(obj);
@@ -356,6 +361,7 @@ static void lasi_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
 
+    dc->reset = lasi_reset;
     dc->vmsd = &vmstate_lasi;
 }
 

From b514f432445608c516cd82a93ae7c761454f17ea Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:37 +0100
Subject: [PATCH 148/935] lasi: move LASIState and associated QOM structures to
 lasi.h

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-28-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/lasi.c | 44 +-------------------------------------
 hw/hppa/lasi.h | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 59 insertions(+), 43 deletions(-)
 create mode 100644 hw/hppa/lasi.h

diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index 242713c7bd..c1dfd5310f 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -24,50 +24,8 @@
 #include "hw/input/lasips2.h"
 #include "migration/vmstate.h"
 #include "qom/object.h"
+#include "lasi.h"
 
-#define TYPE_LASI_CHIP "lasi-chip"
-
-#define LASI_IRR        0x00    /* RO */
-#define LASI_IMR        0x04
-#define LASI_IPR        0x08
-#define LASI_ICR        0x0c
-#define LASI_IAR        0x10
-
-#define LASI_PCR        0x0C000 /* LASI Power Control register */
-#define LASI_ERRLOG     0x0C004 /* LASI Error Logging register */
-#define LASI_VER        0x0C008 /* LASI Version Control register */
-#define LASI_IORESET    0x0C00C /* LASI I/O Reset register */
-#define LASI_AMR        0x0C010 /* LASI Arbitration Mask register */
-#define LASI_IO_CONF    0x7FFFE /* LASI primary configuration register */
-#define LASI_IO_CONF2   0x7FFFF /* LASI secondary configuration register */
-
-#define LASI_BIT(x)     (1ul << (x))
-#define LASI_IRQ_BITS   (LASI_BIT(5) | LASI_BIT(7) | LASI_BIT(8) | LASI_BIT(9) \
-            | LASI_BIT(13) | LASI_BIT(14) | LASI_BIT(16) | LASI_BIT(17) \
-            | LASI_BIT(18) | LASI_BIT(19) | LASI_BIT(20) | LASI_BIT(21) \
-            | LASI_BIT(26))
-
-#define ICR_BUS_ERROR_BIT  LASI_BIT(8)  /* bit 8 in ICR */
-#define ICR_TOC_BIT        LASI_BIT(1)  /* bit 1 in ICR */
-
-OBJECT_DECLARE_SIMPLE_TYPE(LasiState, LASI_CHIP)
-
-struct LasiState {
-    PCIHostState parent_obj;
-
-    uint32_t irr;
-    uint32_t imr;
-    uint32_t ipr;
-    uint32_t icr;
-    uint32_t iar;
-
-    uint32_t errlog;
-    uint32_t amr;
-    uint32_t rtc;
-    time_t rtc_ref;
-
-    MemoryRegion this_mem;
-};
 
 static bool lasi_chip_mem_valid(void *opaque, hwaddr addr,
                                 unsigned size, bool is_write,
diff --git a/hw/hppa/lasi.h b/hw/hppa/lasi.h
new file mode 100644
index 0000000000..559aae2560
--- /dev/null
+++ b/hw/hppa/lasi.h
@@ -0,0 +1,58 @@
+/*
+ * HP-PARISC Lasi chipset emulation.
+ *
+ * (C) 2019 by Helge Deller <deller@gmx.de>
+ *
+ * This work is licensed under the GNU GPL license version 2 or later.
+ *
+ * Documentation available at:
+ * https://parisc.wiki.kernel.org/images-parisc/7/79/Lasi_ers.pdf
+ */
+
+#ifndef LASI_H
+#define LASI_H
+
+#define TYPE_LASI_CHIP "lasi-chip"
+OBJECT_DECLARE_SIMPLE_TYPE(LasiState, LASI_CHIP)
+
+#define LASI_IRR        0x00    /* RO */
+#define LASI_IMR        0x04
+#define LASI_IPR        0x08
+#define LASI_ICR        0x0c
+#define LASI_IAR        0x10
+
+#define LASI_PCR        0x0C000 /* LASI Power Control register */
+#define LASI_ERRLOG     0x0C004 /* LASI Error Logging register */
+#define LASI_VER        0x0C008 /* LASI Version Control register */
+#define LASI_IORESET    0x0C00C /* LASI I/O Reset register */
+#define LASI_AMR        0x0C010 /* LASI Arbitration Mask register */
+#define LASI_IO_CONF    0x7FFFE /* LASI primary configuration register */
+#define LASI_IO_CONF2   0x7FFFF /* LASI secondary configuration register */
+
+#define LASI_BIT(x)     (1ul << (x))
+#define LASI_IRQ_BITS   (LASI_BIT(5) | LASI_BIT(7) | LASI_BIT(8) | LASI_BIT(9) \
+            | LASI_BIT(13) | LASI_BIT(14) | LASI_BIT(16) | LASI_BIT(17) \
+            | LASI_BIT(18) | LASI_BIT(19) | LASI_BIT(20) | LASI_BIT(21) \
+            | LASI_BIT(26))
+
+#define ICR_BUS_ERROR_BIT  LASI_BIT(8)  /* bit 8 in ICR */
+#define ICR_TOC_BIT        LASI_BIT(1)  /* bit 1 in ICR */
+
+struct LasiState {
+    PCIHostState parent_obj;
+
+    uint32_t irr;
+    uint32_t imr;
+    uint32_t ipr;
+    uint32_t icr;
+    uint32_t iar;
+
+    uint32_t errlog;
+    uint32_t amr;
+    uint32_t rtc;
+    time_t rtc_ref;
+
+    MemoryRegion this_mem;
+};
+
+#endif

From 0f04d5777b7c3b1fe260838f3c1f05a6c6452a8d Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:38 +0100
Subject: [PATCH 149/935] lasi: replace lasi_get_irq() with defined constants

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-29-mark.cave-ayland@ilande.co.uk>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/lasi.c | 31 ++++---------------------------
 hw/hppa/lasi.h |  9 +++++++++
 2 files changed, 13 insertions(+), 27 deletions(-)

diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index c1dfd5310f..ec079601a1 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -231,29 +231,6 @@ static void lasi_set_irq(void *opaque, int irq, int level)
     }
 }
 
-static int lasi_get_irq(unsigned long hpa)
-{
-    switch (hpa) {
-    case LASI_HPA:
-        return 14;
-    case LASI_UART_HPA:
-        return 5;
-    case LASI_LPT_HPA:
-        return 7;
-    case LASI_LAN_HPA:
-        return 8;
-    case LASI_SCSI_HPA:
-        return 9;
-    case LASI_AUDIO_HPA:
-        return 13;
-    case LASI_PS2KBD_HPA:
-    case LASI_PS2MOU_HPA:
-        return 26;
-    default:
-        g_assert_not_reached();
-    }
-}
-
 DeviceState *lasi_initfn(MemoryRegion *address_space)
 {
     DeviceState *dev;
@@ -266,20 +243,20 @@ DeviceState *lasi_initfn(MemoryRegion *address_space)
     /* LAN */
     if (enable_lasi_lan()) {
         qemu_irq lan_irq = qemu_allocate_irq(lasi_set_irq, s,
-                lasi_get_irq(LASI_LAN_HPA));
+                                             LASI_IRQ_LAN_HPA);
         lasi_82596_init(address_space, LASI_LAN_HPA, lan_irq);
     }
 
     /* Parallel port */
     qemu_irq lpt_irq = qemu_allocate_irq(lasi_set_irq, s,
-            lasi_get_irq(LASI_LPT_HPA));
+                                         LASI_IRQ_LPT_HPA);
     parallel_mm_init(address_space, LASI_LPT_HPA + 0x800, 0,
                      lpt_irq, parallel_hds[0]);
 
     if (serial_hd(1)) {
         /* Serial port */
         qemu_irq serial_irq = qemu_allocate_irq(lasi_set_irq, s,
-                lasi_get_irq(LASI_UART_HPA));
+                                                LASI_IRQ_UART_HPA);
         serial_mm_init(address_space, LASI_UART_HPA + 0x800, 0,
                 serial_irq, 8000000 / 16,
                 serial_hd(0), DEVICE_NATIVE_ENDIAN);
@@ -287,7 +264,7 @@ DeviceState *lasi_initfn(MemoryRegion *address_space)
 
     /* PS/2 Keyboard/Mouse */
     qemu_irq ps2kbd_irq = qemu_allocate_irq(lasi_set_irq, s,
-            lasi_get_irq(LASI_PS2KBD_HPA));
+                                            LASI_IRQ_PS2KBD_HPA);
     lasips2_init(address_space, LASI_PS2KBD_HPA,  ps2kbd_irq);
 
     return dev;
diff --git a/hw/hppa/lasi.h b/hw/hppa/lasi.h
index 559aae2560..f40546da6e 100644
--- a/hw/hppa/lasi.h
+++ b/hw/hppa/lasi.h
@@ -38,6 +38,15 @@ OBJECT_DECLARE_SIMPLE_TYPE(LasiState, LASI_CHIP)
 #define ICR_BUS_ERROR_BIT  LASI_BIT(8)  /* bit 8 in ICR */
 #define ICR_TOC_BIT        LASI_BIT(1)  /* bit 1 in ICR */
 
+#define LASI_IRQ_HPA        14
+#define LASI_IRQ_UART_HPA   5
+#define LASI_IRQ_LPT_HPA    7
+#define LASI_IRQ_LAN_HPA    8
+#define LASI_IRQ_SCSI_HPA   9
+#define LASI_IRQ_AUDIO_HPA  13
+#define LASI_IRQ_PS2KBD_HPA 26
+#define LASI_IRQ_PS2MOU_HPA 26
+
 struct LasiState {
     PCIHostState parent_obj;
 

From cb9f6c4b65d48717be251dcb2a171152d00885e9 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:39 +0100
Subject: [PATCH 150/935] lasi: define IRQ inputs as qdev GPIOs

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-30-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/lasi.c | 2 ++
 hw/hppa/lasi.h | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index ec079601a1..9489a80bad 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -290,6 +290,8 @@ static void lasi_init(Object *obj)
                           s, "lasi", 0x100000);
 
     sysbus_init_mmio(sbd, &s->this_mem);
+
+    qdev_init_gpio_in(DEVICE(obj), lasi_set_irq, LASI_IRQS);
 }
 
 static void lasi_class_init(ObjectClass *klass, void *data)
diff --git a/hw/hppa/lasi.h b/hw/hppa/lasi.h
index f40546da6e..63a2be3740 100644
--- a/hw/hppa/lasi.h
+++ b/hw/hppa/lasi.h
@@ -38,6 +38,8 @@ OBJECT_DECLARE_SIMPLE_TYPE(LasiState, LASI_CHIP)
 #define ICR_BUS_ERROR_BIT  LASI_BIT(8)  /* bit 8 in ICR */
 #define ICR_TOC_BIT        LASI_BIT(1)  /* bit 1 in ICR */
 
+#define LASI_IRQS           27
+
 #define LASI_IRQ_HPA        14
 #define LASI_IRQ_UART_HPA   5
 #define LASI_IRQ_LPT_HPA    7

From e004499fd52f0bb10e1e77bedc3be0dbdd2e0f52 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:40 +0100
Subject: [PATCH 151/935] lasi: use qdev GPIOs to wire up IRQs in lasi_initfn()

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-31-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/lasi.c | 21 +++++++--------------
 1 file changed, 7 insertions(+), 14 deletions(-)

diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index 9489a80bad..32c7514d3a 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -234,38 +234,31 @@ static void lasi_set_irq(void *opaque, int irq, int level)
 DeviceState *lasi_initfn(MemoryRegion *address_space)
 {
     DeviceState *dev;
-    LasiState *s;
 
     dev = qdev_new(TYPE_LASI_CHIP);
-    s = LASI_CHIP(dev);
     sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
 
     /* LAN */
     if (enable_lasi_lan()) {
-        qemu_irq lan_irq = qemu_allocate_irq(lasi_set_irq, s,
-                                             LASI_IRQ_LAN_HPA);
-        lasi_82596_init(address_space, LASI_LAN_HPA, lan_irq);
+        lasi_82596_init(address_space, LASI_LAN_HPA,
+                        qdev_get_gpio_in(dev, LASI_IRQ_LAN_HPA));
     }
 
     /* Parallel port */
-    qemu_irq lpt_irq = qemu_allocate_irq(lasi_set_irq, s,
-                                         LASI_IRQ_LPT_HPA);
     parallel_mm_init(address_space, LASI_LPT_HPA + 0x800, 0,
-                     lpt_irq, parallel_hds[0]);
+                     qdev_get_gpio_in(dev, LASI_IRQ_LAN_HPA),
+                     parallel_hds[0]);
 
     if (serial_hd(1)) {
         /* Serial port */
-        qemu_irq serial_irq = qemu_allocate_irq(lasi_set_irq, s,
-                                                LASI_IRQ_UART_HPA);
         serial_mm_init(address_space, LASI_UART_HPA + 0x800, 0,
-                serial_irq, 8000000 / 16,
+                qdev_get_gpio_in(dev, LASI_IRQ_UART_HPA), 8000000 / 16,
                 serial_hd(0), DEVICE_NATIVE_ENDIAN);
     }
 
     /* PS/2 Keyboard/Mouse */
-    qemu_irq ps2kbd_irq = qemu_allocate_irq(lasi_set_irq, s,
-                                            LASI_IRQ_PS2KBD_HPA);
-    lasips2_init(address_space, LASI_PS2KBD_HPA,  ps2kbd_irq);
+    lasips2_init(address_space, LASI_PS2KBD_HPA,
+                 qdev_get_gpio_in(dev, LASI_IRQ_PS2KBD_HPA));
 
     return dev;
 }

From fe744ca3eee65e5d666a06e72c1e6b671eb2c74a Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:41 +0100
Subject: [PATCH 152/935] lasi: fix serial port initialisation

The existing code checks for serial_hd(1) but sets the LASI serial port chardev
to serial_hd(0). Use serial_hd(1) for the LASI serial port and also set the
serial port endian to DEVICE_BIG_ENDIAN (which also matches the endian of the
existing serial port).

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-32-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/lasi.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index 32c7514d3a..219eb779f3 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -253,7 +253,7 @@ DeviceState *lasi_initfn(MemoryRegion *address_space)
         /* Serial port */
         serial_mm_init(address_space, LASI_UART_HPA + 0x800, 0,
                 qdev_get_gpio_in(dev, LASI_IRQ_UART_HPA), 8000000 / 16,
-                serial_hd(0), DEVICE_NATIVE_ENDIAN);
+                serial_hd(1), DEVICE_BIG_ENDIAN);
     }
 
     /* PS/2 Keyboard/Mouse */

From 954f6f7564ef48688d2146e498cb012605d461c6 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:42 +0100
Subject: [PATCH 153/935] lasi: update lasi_initfn() to return LASIState

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-33-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/hppa_sys.h | 3 ++-
 hw/hppa/lasi.c     | 4 ++--
 hw/hppa/machine.c  | 3 ++-
 3 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/hw/hppa/hppa_sys.h b/hw/hppa/hppa_sys.h
index 0929dedded..3832b787d1 100644
--- a/hw/hppa/hppa_sys.h
+++ b/hw/hppa/hppa_sys.h
@@ -7,10 +7,11 @@
 #include "hw/pci/pci_host.h"
 #include "hw/boards.h"
 #include "hw/intc/i8259.h"
+#include "lasi.h"
 
 #include "hppa_hardware.h"
 
-DeviceState *lasi_initfn(MemoryRegion *);
+LasiState *lasi_initfn(MemoryRegion *);
 #define enable_lasi_lan()       0
 
 /* hppa_pci.c.  */
diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index 219eb779f3..65139bb29b 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -231,7 +231,7 @@ static void lasi_set_irq(void *opaque, int irq, int level)
     }
 }
 
-DeviceState *lasi_initfn(MemoryRegion *address_space)
+LasiState *lasi_initfn(MemoryRegion *address_space)
 {
     DeviceState *dev;
 
@@ -260,7 +260,7 @@ DeviceState *lasi_initfn(MemoryRegion *address_space)
     lasips2_init(address_space, LASI_PS2KBD_HPA,
                  qdev_get_gpio_in(dev, LASI_IRQ_PS2KBD_HPA));
 
-    return dev;
+    return LASI_CHIP(dev);
 }
 
 static void lasi_reset(DeviceState *dev)
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 6618bce435..c815dde305 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -18,6 +18,7 @@
 #include "hw/net/lasi_82596.h"
 #include "hw/nmi.h"
 #include "hw/pci-host/dino.h"
+#include "lasi.h"
 #include "hppa_sys.h"
 #include "qemu/units.h"
 #include "qapi/error.h"
@@ -175,7 +176,7 @@ static void machine_hppa_init(MachineState *machine)
 
 
     /* Init Lasi chip */
-    lasi_dev = lasi_initfn(addr_space);
+    lasi_dev = DEVICE(lasi_initfn(addr_space));
     memory_region_add_subregion(addr_space, LASI_HPA,
                                 sysbus_mmio_get_region(
                                     SYS_BUS_DEVICE(lasi_dev), 0));

From c3c3fe47086221de21cc387ef4c876f316463876 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:43 +0100
Subject: [PATCH 154/935] lasi: move LAN initialisation to machine.c

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-34-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/lasi.c    | 7 -------
 hw/hppa/machine.c | 5 +++++
 2 files changed, 5 insertions(+), 7 deletions(-)

diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index 65139bb29b..88ff9141e4 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -18,7 +18,6 @@
 #include "sysemu/sysemu.h"
 #include "sysemu/runstate.h"
 #include "hppa_sys.h"
-#include "hw/net/lasi_82596.h"
 #include "hw/char/parallel.h"
 #include "hw/char/serial.h"
 #include "hw/input/lasips2.h"
@@ -238,12 +237,6 @@ LasiState *lasi_initfn(MemoryRegion *address_space)
     dev = qdev_new(TYPE_LASI_CHIP);
     sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
 
-    /* LAN */
-    if (enable_lasi_lan()) {
-        lasi_82596_init(address_space, LASI_LAN_HPA,
-                        qdev_get_gpio_in(dev, LASI_IRQ_LAN_HPA));
-    }
-
     /* Parallel port */
     parallel_mm_init(address_space, LASI_LPT_HPA + 0x800, 0,
                      qdev_get_gpio_in(dev, LASI_IRQ_LAN_HPA),
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index c815dde305..b3b4a00ad6 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -221,6 +221,11 @@ static void machine_hppa_init(MachineState *machine)
     }
 
     /* Network setup. */
+    if (enable_lasi_lan()) {
+        lasi_82596_init(addr_space, LASI_LAN_HPA,
+                        qdev_get_gpio_in(lasi_dev, LASI_IRQ_LAN_HPA));
+    }
+
     for (i = 0; i < nb_nics; i++) {
         if (!enable_lasi_lan()) {
             pci_nic_init_nofail(&nd_table[i], pci_bus, "tulip", NULL);

From 9701e56943282ea94876c1e4202ed30ce3a3c706 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:44 +0100
Subject: [PATCH 155/935] lasi: move parallel port initialisation to machine.c

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-35-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/lasi.c    | 6 ------
 hw/hppa/machine.c | 6 ++++++
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index 88ff9141e4..6faa98dca5 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -18,7 +18,6 @@
 #include "sysemu/sysemu.h"
 #include "sysemu/runstate.h"
 #include "hppa_sys.h"
-#include "hw/char/parallel.h"
 #include "hw/char/serial.h"
 #include "hw/input/lasips2.h"
 #include "migration/vmstate.h"
@@ -237,11 +236,6 @@ LasiState *lasi_initfn(MemoryRegion *address_space)
     dev = qdev_new(TYPE_LASI_CHIP);
     sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
 
-    /* Parallel port */
-    parallel_mm_init(address_space, LASI_LPT_HPA + 0x800, 0,
-                     qdev_get_gpio_in(dev, LASI_IRQ_LAN_HPA),
-                     parallel_hds[0]);
-
     if (serial_hd(1)) {
         /* Serial port */
         serial_mm_init(address_space, LASI_UART_HPA + 0x800, 0,
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index b3b4a00ad6..5244178aaf 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -15,6 +15,7 @@
 #include "hw/rtc/mc146818rtc.h"
 #include "hw/timer/i8254.h"
 #include "hw/char/serial.h"
+#include "hw/char/parallel.h"
 #include "hw/net/lasi_82596.h"
 #include "hw/nmi.h"
 #include "hw/pci-host/dino.h"
@@ -204,6 +205,11 @@ static void machine_hppa_init(MachineState *machine)
                        115200, serial_hd(0), DEVICE_BIG_ENDIAN);
     }
 
+    /* Parallel port */
+    parallel_mm_init(addr_space, LASI_LPT_HPA + 0x800, 0,
+                     qdev_get_gpio_in(lasi_dev, LASI_IRQ_LAN_HPA),
+                     parallel_hds[0]);
+
     /* fw_cfg configuration interface */
     create_fw_cfg(machine);
 

From 2da547b81968f601ca870ace0a1e1528c4402d09 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:45 +0100
Subject: [PATCH 156/935] lasi: move second serial port initialisation to
 machine.c

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-36-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/lasi.c    | 8 --------
 hw/hppa/machine.c | 7 +++++++
 2 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index 6faa98dca5..753a08d454 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -18,7 +18,6 @@
 #include "sysemu/sysemu.h"
 #include "sysemu/runstate.h"
 #include "hppa_sys.h"
-#include "hw/char/serial.h"
 #include "hw/input/lasips2.h"
 #include "migration/vmstate.h"
 #include "qom/object.h"
@@ -236,13 +235,6 @@ LasiState *lasi_initfn(MemoryRegion *address_space)
     dev = qdev_new(TYPE_LASI_CHIP);
     sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
 
-    if (serial_hd(1)) {
-        /* Serial port */
-        serial_mm_init(address_space, LASI_UART_HPA + 0x800, 0,
-                qdev_get_gpio_in(dev, LASI_IRQ_UART_HPA), 8000000 / 16,
-                serial_hd(1), DEVICE_BIG_ENDIAN);
-    }
-
     /* PS/2 Keyboard/Mouse */
     lasips2_init(address_space, LASI_PS2KBD_HPA,
                  qdev_get_gpio_in(dev, LASI_IRQ_PS2KBD_HPA));
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 5244178aaf..5b43ae2960 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -205,6 +205,13 @@ static void machine_hppa_init(MachineState *machine)
                        115200, serial_hd(0), DEVICE_BIG_ENDIAN);
     }
 
+    if (serial_hd(1)) {
+        /* Serial port */
+        serial_mm_init(addr_space, LASI_UART_HPA + 0x800, 0,
+                qdev_get_gpio_in(lasi_dev, LASI_IRQ_UART_HPA), 8000000 / 16,
+                serial_hd(1), DEVICE_BIG_ENDIAN);
+    }
+
     /* Parallel port */
     parallel_mm_init(addr_space, LASI_LPT_HPA + 0x800, 0,
                      qdev_get_gpio_in(lasi_dev, LASI_IRQ_LAN_HPA),

From d26c575c44379d9df7843677c7d59729ffe18764 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:46 +0100
Subject: [PATCH 157/935] lasi: move PS2 initialisation to machine.c

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-37-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/lasi.c    | 5 -----
 hw/hppa/machine.c | 5 +++++
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index 753a08d454..9d8c9e3936 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -18,7 +18,6 @@
 #include "sysemu/sysemu.h"
 #include "sysemu/runstate.h"
 #include "hppa_sys.h"
-#include "hw/input/lasips2.h"
 #include "migration/vmstate.h"
 #include "qom/object.h"
 #include "lasi.h"
@@ -235,10 +234,6 @@ LasiState *lasi_initfn(MemoryRegion *address_space)
     dev = qdev_new(TYPE_LASI_CHIP);
     sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
 
-    /* PS/2 Keyboard/Mouse */
-    lasips2_init(address_space, LASI_PS2KBD_HPA,
-                 qdev_get_gpio_in(dev, LASI_IRQ_PS2KBD_HPA));
-
     return LASI_CHIP(dev);
 }
 
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 5b43ae2960..1d48ec341d 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -16,6 +16,7 @@
 #include "hw/timer/i8254.h"
 #include "hw/char/serial.h"
 #include "hw/char/parallel.h"
+#include "hw/input/lasips2.h"
 #include "hw/net/lasi_82596.h"
 #include "hw/nmi.h"
 #include "hw/pci-host/dino.h"
@@ -245,6 +246,10 @@ static void machine_hppa_init(MachineState *machine)
         }
     }
 
+    /* PS/2 Keyboard/Mouse */
+    lasips2_init(addr_space, LASI_PS2KBD_HPA,
+                 qdev_get_gpio_in(lasi_dev, LASI_IRQ_PS2KBD_HPA));
+
     /* register power switch emulation */
     qemu_register_powerdown_notifier(&hppa_system_powerdown_notifier);
 

From deaa5d3b4e8eccf4010c9099b99c267ff1f2223e Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:47 +0100
Subject: [PATCH 158/935] lasi: remove address space parameter from
 lasi_initfn()

Now that all of the LASI devices are mapped by the board, this parameter is no
longer required.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-38-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/hppa_sys.h | 2 +-
 hw/hppa/lasi.c     | 2 +-
 hw/hppa/machine.c  | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/hw/hppa/hppa_sys.h b/hw/hppa/hppa_sys.h
index 3832b787d1..31e3856059 100644
--- a/hw/hppa/hppa_sys.h
+++ b/hw/hppa/hppa_sys.h
@@ -11,7 +11,7 @@
 
 #include "hppa_hardware.h"
 
-LasiState *lasi_initfn(MemoryRegion *);
+LasiState *lasi_initfn(void);
 #define enable_lasi_lan()       0
 
 /* hppa_pci.c.  */
diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index 9d8c9e3936..81c8e4d2d9 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -227,7 +227,7 @@ static void lasi_set_irq(void *opaque, int irq, int level)
     }
 }
 
-LasiState *lasi_initfn(MemoryRegion *address_space)
+LasiState *lasi_initfn(void)
 {
     DeviceState *dev;
 
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 1d48ec341d..52f12f6604 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -178,7 +178,7 @@ static void machine_hppa_init(MachineState *machine)
 
 
     /* Init Lasi chip */
-    lasi_dev = DEVICE(lasi_initfn(addr_space));
+    lasi_dev = DEVICE(lasi_initfn());
     memory_region_add_subregion(addr_space, LASI_HPA,
                                 sysbus_mmio_get_region(
                                     SYS_BUS_DEVICE(lasi_dev), 0));

From e881e3c83a47214d5d5d4141a1bc4ad99486f80a Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:48 +0100
Subject: [PATCH 159/935] lasi: move lasi_initfn() to machine.c

Move the simplified lasi_initfn() back to machine.c whilst also renaming it
back to its original lasi_init() name.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-39-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/hppa_sys.h |  2 --
 hw/hppa/lasi.c     | 10 ----------
 hw/hppa/machine.c  | 12 +++++++++++-
 3 files changed, 11 insertions(+), 13 deletions(-)

diff --git a/hw/hppa/hppa_sys.h b/hw/hppa/hppa_sys.h
index 31e3856059..f7a127be19 100644
--- a/hw/hppa/hppa_sys.h
+++ b/hw/hppa/hppa_sys.h
@@ -7,11 +7,9 @@
 #include "hw/pci/pci_host.h"
 #include "hw/boards.h"
 #include "hw/intc/i8259.h"
-#include "lasi.h"
 
 #include "hppa_hardware.h"
 
-LasiState *lasi_initfn(void);
 #define enable_lasi_lan()       0
 
 /* hppa_pci.c.  */
diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index 81c8e4d2d9..ad50880a13 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -227,16 +227,6 @@ static void lasi_set_irq(void *opaque, int irq, int level)
     }
 }
 
-LasiState *lasi_initfn(void)
-{
-    DeviceState *dev;
-
-    dev = qdev_new(TYPE_LASI_CHIP);
-    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
-
-    return LASI_CHIP(dev);
-}
-
 static void lasi_reset(DeviceState *dev)
 {
     LasiState *s = LASI_CHIP(dev);
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 52f12f6604..58b25ca5bc 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -125,6 +125,16 @@ static FWCfgState *create_fw_cfg(MachineState *ms)
     return fw_cfg;
 }
 
+static LasiState *lasi_init(void)
+{
+    DeviceState *dev;
+
+    dev = qdev_new(TYPE_LASI_CHIP);
+    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
+
+    return LASI_CHIP(dev);
+}
+
 static DinoState *dino_init(MemoryRegion *addr_space)
 {
     DeviceState *dev;
@@ -178,7 +188,7 @@ static void machine_hppa_init(MachineState *machine)
 
 
     /* Init Lasi chip */
-    lasi_dev = DEVICE(lasi_initfn());
+    lasi_dev = DEVICE(lasi_init());
     memory_region_add_subregion(addr_space, LASI_HPA,
                                 sysbus_mmio_get_region(
                                     SYS_BUS_DEVICE(lasi_dev), 0));

From ca7b468be8944d2c166f6b7cf25cd5e8e006bc03 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:49 +0100
Subject: [PATCH 160/935] lasi: use constants for device register offsets

Instead of generating the offset based upon the physical address of the
register, add constants for each of the device registers to lasi.h and
update lasi.c to use them.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-40-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/lasi.c | 28 ++++++++++++++--------------
 hw/hppa/lasi.h |  5 +++++
 2 files changed, 19 insertions(+), 14 deletions(-)

diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index ad50880a13..11ca33fba3 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -36,10 +36,10 @@ static bool lasi_chip_mem_valid(void *opaque, hwaddr addr,
     case LASI_ICR:
     case LASI_IAR:
 
-    case (LASI_LAN_HPA - LASI_HPA):
-    case (LASI_LPT_HPA - LASI_HPA):
-    case (LASI_UART_HPA - LASI_HPA):
-    case (LASI_RTC_HPA - LASI_HPA):
+    case LASI_LPT:
+    case LASI_UART:
+    case LASI_LAN:
+    case LASI_RTC:
 
     case LASI_PCR ... LASI_AMR:
         ret = true;
@@ -76,12 +76,12 @@ static MemTxResult lasi_chip_read_with_attrs(void *opaque, hwaddr addr,
         val = s->iar;
         break;
 
-    case (LASI_LAN_HPA - LASI_HPA):
-    case (LASI_LPT_HPA - LASI_HPA):
-    case (LASI_UART_HPA - LASI_HPA):
+    case LASI_LPT:
+    case LASI_UART:
+    case LASI_LAN:
         val = 0;
         break;
-    case (LASI_RTC_HPA - LASI_HPA):
+    case LASI_RTC:
         val = time(NULL);
         val += s->rtc_ref;
         break;
@@ -141,16 +141,16 @@ static MemTxResult lasi_chip_write_with_attrs(void *opaque, hwaddr addr,
         s->iar = val;
         break;
 
-    case (LASI_LAN_HPA - LASI_HPA):
-        /* XXX: reset LAN card */
-        break;
-    case (LASI_LPT_HPA - LASI_HPA):
+    case LASI_LPT:
         /* XXX: reset parallel port */
         break;
-    case (LASI_UART_HPA - LASI_HPA):
+    case LASI_UART:
         /* XXX: reset serial port */
         break;
-    case (LASI_RTC_HPA - LASI_HPA):
+    case LASI_LAN:
+        /* XXX: reset LAN card */
+        break;
+    case LASI_RTC:
         s->rtc_ref = val - time(NULL);
         break;
 
diff --git a/hw/hppa/lasi.h b/hw/hppa/lasi.h
index 63a2be3740..11cf7d6b0b 100644
--- a/hw/hppa/lasi.h
+++ b/hw/hppa/lasi.h
@@ -21,6 +21,11 @@ OBJECT_DECLARE_SIMPLE_TYPE(LasiState, LASI_CHIP)
 #define LASI_ICR        0x0c
 #define LASI_IAR        0x10
 
+#define LASI_LPT        0x02000
+#define LASI_UART       0x05000
+#define LASI_LAN        0x07000
+#define LASI_RTC        0x09000
+
 #define LASI_PCR        0x0C000 /* LASI Power Control register */
 #define LASI_ERRLOG     0x0C004 /* LASI Error Logging register */
 #define LASI_VER        0x0C008 /* LASI Version Control register */

From 8e81ffe32070490b27d8e9ccac48a3f470e19c96 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:50 +0100
Subject: [PATCH 161/935] lasi: use numerical constant for iar reset value

This is to allow us to decouple the LASI device from the board logic. If it is
decided later that this value needs to be configurable then it can easily be
converted to a qdev property.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-41-mark.cave-ayland@ilande.co.uk>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/lasi.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/hppa/lasi.c b/hw/hppa/lasi.c
index 11ca33fba3..5ef36f3f58 100644
--- a/hw/hppa/lasi.c
+++ b/hw/hppa/lasi.c
@@ -231,7 +231,7 @@ static void lasi_reset(DeviceState *dev)
 {
     LasiState *s = LASI_CHIP(dev);
 
-    s->iar = CPU_HPA + 3;
+    s->iar = 0xFFFB0000 + 3; /* CPU_HPA + 3 */
 
     /* Real time clock (RTC), it's only one 32-bit counter @9000 */
     s->rtc = time(NULL);

From 134ba73f32036256e0a89bcd166ddbe1fd19b824 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:51 +0100
Subject: [PATCH 162/935] hppa: move device headers from hppa_sys.h into
 individual .c files

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-42-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/hppa_sys.h | 3 ---
 hw/hppa/lasi.h     | 4 ++++
 hw/hppa/machine.c  | 2 ++
 hw/hppa/pci.c      | 3 +++
 4 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/hw/hppa/hppa_sys.h b/hw/hppa/hppa_sys.h
index f7a127be19..9964d4a7a7 100644
--- a/hw/hppa/hppa_sys.h
+++ b/hw/hppa/hppa_sys.h
@@ -3,10 +3,7 @@
 #ifndef HW_HPPA_SYS_H
 #define HW_HPPA_SYS_H
 
-#include "hw/pci/pci.h"
-#include "hw/pci/pci_host.h"
 #include "hw/boards.h"
-#include "hw/intc/i8259.h"
 
 #include "hppa_hardware.h"
 
diff --git a/hw/hppa/lasi.h b/hw/hppa/lasi.h
index 11cf7d6b0b..ecc7065ce8 100644
--- a/hw/hppa/lasi.h
+++ b/hw/hppa/lasi.h
@@ -12,6 +12,10 @@
 #ifndef LASI_H
 #define LASI_H
 
+#include "exec/address-spaces.h"
+#include "hw/pci/pci_host.h"
+#include "hw/boards.h"
+
 #define TYPE_LASI_CHIP "lasi-chip"
 OBJECT_DECLARE_SIMPLE_TYPE(LasiState, LASI_CHIP)
 
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 58b25ca5bc..9d904b7a60 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -16,9 +16,11 @@
 #include "hw/timer/i8254.h"
 #include "hw/char/serial.h"
 #include "hw/char/parallel.h"
+#include "hw/intc/i8259.h"
 #include "hw/input/lasips2.h"
 #include "hw/net/lasi_82596.h"
 #include "hw/nmi.h"
+#include "hw/pci/pci.h"
 #include "hw/pci-host/dino.h"
 #include "lasi.h"
 #include "hppa_sys.h"
diff --git a/hw/hppa/pci.c b/hw/hppa/pci.c
index 32609aba63..a92dcb6b9e 100644
--- a/hw/hppa/pci.c
+++ b/hw/hppa/pci.c
@@ -6,6 +6,9 @@
 #include "qemu/osdep.h"
 #include "hppa_sys.h"
 #include "qemu/log.h"
+#include "hw/pci/pci.h"
+#include "hw/pci/pci_host.h"
+#include "hw/intc/i8259.h"
 #include "trace.h"
 
 

From 45f569a11666051f98edc678616d135c9feaf506 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:52 +0100
Subject: [PATCH 163/935] lasi: move from hw/hppa to hw/misc

Move the LASI device implementation from hw/hppa to hw/misc so that it is
located with all the other miscellaneous devices.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-43-mark.cave-ayland@ilande.co.uk>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 MAINTAINERS                         | 2 ++
 hw/hppa/Kconfig                     | 1 +
 hw/hppa/machine.c                   | 2 +-
 hw/hppa/meson.build                 | 2 +-
 hw/hppa/trace-events                | 5 -----
 hw/misc/Kconfig                     | 3 +++
 hw/{hppa => misc}/lasi.c            | 3 +--
 hw/misc/meson.build                 | 3 +++
 hw/misc/trace-events                | 5 +++++
 {hw/hppa => include/hw/misc}/lasi.h | 0
 10 files changed, 17 insertions(+), 9 deletions(-)
 rename hw/{hppa => misc}/lasi.c (99%)
 rename {hw/hppa => include/hw/misc}/lasi.h (100%)

diff --git a/MAINTAINERS b/MAINTAINERS
index 59210b093a..662ec47246 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1113,7 +1113,9 @@ S: Odd Fixes
 F: configs/devices/hppa-softmmu/default.mak
 F: hw/hppa/
 F: hw/net/*i82596*
+F: hw/misc/lasi.c
 F: hw/pci-host/dino.c
+F: include/hw/misc/lasi.h
 F: include/hw/net/lasi_82596.h
 F: include/hw/pci-host/dino.h
 F: pc-bios/hppa-firmware.img
diff --git a/hw/hppa/Kconfig b/hw/hppa/Kconfig
index 724380ecec..5dd8b5b21e 100644
--- a/hw/hppa/Kconfig
+++ b/hw/hppa/Kconfig
@@ -4,6 +4,7 @@ config HPPA_B160L
     imply E1000_PCI
     imply VIRTIO_VGA
     select DINO
+    select LASI
     select SERIAL
     select ISA_BUS
     select I8259
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 9d904b7a60..f3d72b4c35 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -22,7 +22,7 @@
 #include "hw/nmi.h"
 #include "hw/pci/pci.h"
 #include "hw/pci-host/dino.h"
-#include "lasi.h"
+#include "hw/misc/lasi.h"
 #include "hppa_sys.h"
 #include "qemu/units.h"
 #include "qapi/error.h"
diff --git a/hw/hppa/meson.build b/hw/hppa/meson.build
index a6f9db455c..bd5a1fb1d2 100644
--- a/hw/hppa/meson.build
+++ b/hw/hppa/meson.build
@@ -1,4 +1,4 @@
 hppa_ss = ss.source_set()
-hppa_ss.add(when: 'CONFIG_HPPA_B160L', if_true: files('pci.c', 'machine.c', 'lasi.c'))
+hppa_ss.add(when: 'CONFIG_HPPA_B160L', if_true: files('pci.c', 'machine.c'))
 
 hw_arch += {'hppa': hppa_ss}
diff --git a/hw/hppa/trace-events b/hw/hppa/trace-events
index 871a473771..1a4fbe2fa8 100644
--- a/hw/hppa/trace-events
+++ b/hw/hppa/trace-events
@@ -2,8 +2,3 @@
 
 # pci.c
 hppa_pci_iack_write(void) ""
-
-# lasi.c
-lasi_chip_mem_valid(uint64_t addr, uint32_t val) "access to addr 0x%"PRIx64" is %d"
-lasi_chip_read(uint64_t addr, uint32_t val) "addr 0x%"PRIx64" val 0x%08x"
-lasi_chip_write(uint64_t addr, uint32_t val) "addr 0x%"PRIx64" val 0x%08x"
diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
index 507058d8bf..cbabe9f78c 100644
--- a/hw/misc/Kconfig
+++ b/hw/misc/Kconfig
@@ -171,4 +171,7 @@ config SIFIVE_U_PRCI
 config VIRT_CTRL
     bool
 
+config LASI
+    bool
+
 source macio/Kconfig
diff --git a/hw/hppa/lasi.c b/hw/misc/lasi.c
similarity index 99%
rename from hw/hppa/lasi.c
rename to hw/misc/lasi.c
index 5ef36f3f58..23a7634a8c 100644
--- a/hw/hppa/lasi.c
+++ b/hw/misc/lasi.c
@@ -17,10 +17,9 @@
 #include "hw/irq.h"
 #include "sysemu/sysemu.h"
 #include "sysemu/runstate.h"
-#include "hppa_sys.h"
 #include "migration/vmstate.h"
 #include "qom/object.h"
-#include "lasi.h"
+#include "hw/misc/lasi.h"
 
 
 static bool lasi_chip_mem_valid(void *opaque, hwaddr addr,
diff --git a/hw/misc/meson.build b/hw/misc/meson.build
index 2ff05c7afa..132b7b7344 100644
--- a/hw/misc/meson.build
+++ b/hw/misc/meson.build
@@ -134,3 +134,6 @@ specific_ss.add(when: 'CONFIG_MIPS_CPS', if_true: files('mips_cmgcr.c', 'mips_cp
 specific_ss.add(when: 'CONFIG_MIPS_ITU', if_true: files('mips_itu.c'))
 
 specific_ss.add(when: 'CONFIG_SBSA_REF', if_true: files('sbsa_ec.c'))
+
+# HPPA devices
+softmmu_ss.add(when: 'CONFIG_LASI', if_true: files('lasi.c'))
diff --git a/hw/misc/trace-events b/hw/misc/trace-events
index 4e0c7973a4..c5e37b0154 100644
--- a/hw/misc/trace-events
+++ b/hw/misc/trace-events
@@ -263,3 +263,8 @@ virt_ctrl_write(void *dev, unsigned int addr, unsigned int size, uint64_t value)
 virt_ctrl_reset(void *dev) "ctrl: %p"
 virt_ctrl_realize(void *dev) "ctrl: %p"
 virt_ctrl_instance_init(void *dev) "ctrl: %p"
+
+# lasi.c
+lasi_chip_mem_valid(uint64_t addr, uint32_t val) "access to addr 0x%"PRIx64" is %d"
+lasi_chip_read(uint64_t addr, uint32_t val) "addr 0x%"PRIx64" val 0x%08x"
+lasi_chip_write(uint64_t addr, uint32_t val) "addr 0x%"PRIx64" val 0x%08x"
diff --git a/hw/hppa/lasi.h b/include/hw/misc/lasi.h
similarity index 100%
rename from hw/hppa/lasi.h
rename to include/hw/misc/lasi.h

From 28f5332ae9283fa76f736e9011ae647799b9cdc0 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:53 +0100
Subject: [PATCH 164/935] hppa: move hppa_pci_ignore_ops from pci.c to
 machine.c

The memory region only has one user which is for ensuring accesses to the ISA
bus memory do not fault.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-44-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/hppa_sys.h |  1 -
 hw/hppa/machine.c  | 23 +++++++++++++++++++++++
 hw/hppa/pci.c      | 26 --------------------------
 3 files changed, 23 insertions(+), 27 deletions(-)

diff --git a/hw/hppa/hppa_sys.h b/hw/hppa/hppa_sys.h
index 9964d4a7a7..d984b2895d 100644
--- a/hw/hppa/hppa_sys.h
+++ b/hw/hppa/hppa_sys.h
@@ -10,7 +10,6 @@
 #define enable_lasi_lan()       0
 
 /* hppa_pci.c.  */
-extern const MemoryRegionOps hppa_pci_ignore_ops;
 extern const MemoryRegionOps hppa_pci_conf1_ops;
 extern const MemoryRegionOps hppa_pci_iack_ops;
 
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index f3d72b4c35..ca595d343e 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -57,6 +57,29 @@ static Notifier hppa_system_powerdown_notifier = {
     .notify = hppa_powerdown_req
 };
 
+/* Fallback for unassigned PCI I/O operations.  Avoids MCHK.  */
+static uint64_t ignore_read(void *opaque, hwaddr addr, unsigned size)
+{
+    return 0;
+}
+
+static void ignore_write(void *opaque, hwaddr addr, uint64_t v, unsigned size)
+{
+}
+
+static const MemoryRegionOps hppa_pci_ignore_ops = {
+    .read = ignore_read,
+    .write = ignore_write,
+    .endianness = DEVICE_BIG_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+    },
+    .impl = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+    },
+};
 
 static ISABus *hppa_isa_bus(void)
 {
diff --git a/hw/hppa/pci.c b/hw/hppa/pci.c
index a92dcb6b9e..4d62d54c22 100644
--- a/hw/hppa/pci.c
+++ b/hw/hppa/pci.c
@@ -12,32 +12,6 @@
 #include "trace.h"
 
 
-/* Fallback for unassigned PCI I/O operations.  Avoids MCHK.  */
-
-static uint64_t ignore_read(void *opaque, hwaddr addr, unsigned size)
-{
-    return 0;
-}
-
-static void ignore_write(void *opaque, hwaddr addr, uint64_t v, unsigned size)
-{
-}
-
-const MemoryRegionOps hppa_pci_ignore_ops = {
-    .read = ignore_read,
-    .write = ignore_write,
-    .endianness = DEVICE_BIG_ENDIAN,
-    .valid = {
-        .min_access_size = 1,
-        .max_access_size = 8,
-    },
-    .impl = {
-        .min_access_size = 1,
-        .max_access_size = 8,
-    },
-};
-
-
 /* PCI config space reads/writes, to byte-word addressable memory.  */
 static uint64_t bw_conf1_read(void *opaque, hwaddr addr,
                               unsigned size)

From 3c2ce5bba379aa87a3098bffc06101c8a9544940 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:54 +0100
Subject: [PATCH 165/935] hppa: remove hw/hppa/pci.c

The functions and definitions in this file are not used anywhere within the
generic hppa machine.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-45-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/hppa_sys.h   |  6 ----
 hw/hppa/meson.build  |  2 +-
 hw/hppa/pci.c        | 65 --------------------------------------------
 hw/hppa/trace-events |  3 --
 4 files changed, 1 insertion(+), 75 deletions(-)
 delete mode 100644 hw/hppa/pci.c

diff --git a/hw/hppa/hppa_sys.h b/hw/hppa/hppa_sys.h
index d984b2895d..17e2c6dec1 100644
--- a/hw/hppa/hppa_sys.h
+++ b/hw/hppa/hppa_sys.h
@@ -3,14 +3,8 @@
 #ifndef HW_HPPA_SYS_H
 #define HW_HPPA_SYS_H
 
-#include "hw/boards.h"
-
 #include "hppa_hardware.h"
 
 #define enable_lasi_lan()       0
 
-/* hppa_pci.c.  */
-extern const MemoryRegionOps hppa_pci_conf1_ops;
-extern const MemoryRegionOps hppa_pci_iack_ops;
-
 #endif
diff --git a/hw/hppa/meson.build b/hw/hppa/meson.build
index bd5a1fb1d2..3d0c586c30 100644
--- a/hw/hppa/meson.build
+++ b/hw/hppa/meson.build
@@ -1,4 +1,4 @@
 hppa_ss = ss.source_set()
-hppa_ss.add(when: 'CONFIG_HPPA_B160L', if_true: files('pci.c', 'machine.c'))
+hppa_ss.add(when: 'CONFIG_HPPA_B160L', if_true: files('machine.c'))
 
 hw_arch += {'hppa': hppa_ss}
diff --git a/hw/hppa/pci.c b/hw/hppa/pci.c
deleted file mode 100644
index 4d62d54c22..0000000000
--- a/hw/hppa/pci.c
+++ /dev/null
@@ -1,65 +0,0 @@
-/*
- * QEMU HP-PARISC PCI support functions.
- *
- */
-
-#include "qemu/osdep.h"
-#include "hppa_sys.h"
-#include "qemu/log.h"
-#include "hw/pci/pci.h"
-#include "hw/pci/pci_host.h"
-#include "hw/intc/i8259.h"
-#include "trace.h"
-
-
-/* PCI config space reads/writes, to byte-word addressable memory.  */
-static uint64_t bw_conf1_read(void *opaque, hwaddr addr,
-                              unsigned size)
-{
-    PCIBus *b = opaque;
-    return pci_data_read(b, addr, size);
-}
-
-static void bw_conf1_write(void *opaque, hwaddr addr,
-                           uint64_t val, unsigned size)
-{
-    PCIBus *b = opaque;
-    pci_data_write(b, addr, val, size);
-}
-
-const MemoryRegionOps hppa_pci_conf1_ops = {
-    .read = bw_conf1_read,
-    .write = bw_conf1_write,
-    .endianness = DEVICE_BIG_ENDIAN,
-    .impl = {
-        .min_access_size = 1,
-        .max_access_size = 4,
-    },
-};
-
-/* PCI/EISA Interrupt Acknowledge Cycle.  */
-
-static uint64_t iack_read(void *opaque, hwaddr addr, unsigned size)
-{
-    return pic_read_irq(isa_pic);
-}
-
-static void special_write(void *opaque, hwaddr addr,
-                          uint64_t val, unsigned size)
-{
-    trace_hppa_pci_iack_write();
-}
-
-const MemoryRegionOps hppa_pci_iack_ops = {
-    .read = iack_read,
-    .write = special_write,
-    .endianness = DEVICE_BIG_ENDIAN,
-    .valid = {
-        .min_access_size = 4,
-        .max_access_size = 4,
-    },
-    .impl = {
-        .min_access_size = 4,
-        .max_access_size = 4,
-    },
-};
diff --git a/hw/hppa/trace-events b/hw/hppa/trace-events
index 1a4fbe2fa8..23bc9b19b9 100644
--- a/hw/hppa/trace-events
+++ b/hw/hppa/trace-events
@@ -1,4 +1 @@
 # See docs/devel/tracing.rst for syntax documentation.
-
-# pci.c
-hppa_pci_iack_write(void) ""

From 192e32284b15827e056ff504e0eeb19c2e164759 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:55 +0100
Subject: [PATCH 166/935] hppa: remove unused trace-events from from hw/hppa

Now that there are no longer any devices in hw/hppa the trace-events file is
empty and can be removed.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-46-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/trace-events | 1 -
 meson.build          | 1 -
 2 files changed, 2 deletions(-)
 delete mode 100644 hw/hppa/trace-events

diff --git a/hw/hppa/trace-events b/hw/hppa/trace-events
deleted file mode 100644
index 23bc9b19b9..0000000000
--- a/hw/hppa/trace-events
+++ /dev/null
@@ -1 +0,0 @@
-# See docs/devel/tracing.rst for syntax documentation.
diff --git a/meson.build b/meson.build
index 40132b4804..fa672e57bc 100644
--- a/meson.build
+++ b/meson.build
@@ -2944,7 +2944,6 @@ if have_system
     'hw/char',
     'hw/display',
     'hw/dma',
-    'hw/hppa',
     'hw/hyperv',
     'hw/i2c',
     'hw/i386',

From 932befaa557d9f4ab3d830107ada66dfe3cffba2 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:56 +0100
Subject: [PATCH 167/935] hppa: move enable_lan() define from hppa_sys.h to
 machine.c

Now that the board configuration is in one place, the define is only needed when
wiring up the board in machine.c.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-47-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/hppa_sys.h | 2 --
 hw/hppa/machine.c  | 3 +++
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/hw/hppa/hppa_sys.h b/hw/hppa/hppa_sys.h
index 17e2c6dec1..e7f65cad75 100644
--- a/hw/hppa/hppa_sys.h
+++ b/hw/hppa/hppa_sys.h
@@ -5,6 +5,4 @@
 
 #include "hppa_hardware.h"
 
-#define enable_lasi_lan()       0
-
 #endif
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index ca595d343e..e6431aa541 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -36,6 +36,9 @@
 
 #define HPA_POWER_BUTTON (FIRMWARE_END - 0x10)
 
+#define enable_lasi_lan()       0
+
+
 static void hppa_powerdown_req(Notifier *n, void *opaque)
 {
     hwaddr soft_power_reg = HPA_POWER_BUTTON;

From 148da6708580c46c87f8e917d6bfd53b8e6a59ab Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:57 +0100
Subject: [PATCH 168/935] hppa: remove the empty hppa_sys.h file

This file is now just a simple wrapper that includes hppa_hardware.h so remove
the file completely, and update its single user in machine.c to include
hppa_hardware.h directly.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-48-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/hppa_sys.h | 8 --------
 hw/hppa/machine.c  | 2 +-
 2 files changed, 1 insertion(+), 9 deletions(-)
 delete mode 100644 hw/hppa/hppa_sys.h

diff --git a/hw/hppa/hppa_sys.h b/hw/hppa/hppa_sys.h
deleted file mode 100644
index e7f65cad75..0000000000
--- a/hw/hppa/hppa_sys.h
+++ /dev/null
@@ -1,8 +0,0 @@
-/* HPPA cores and system support chips.  */
-
-#ifndef HW_HPPA_SYS_H
-#define HW_HPPA_SYS_H
-
-#include "hppa_hardware.h"
-
-#endif
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index e6431aa541..75820838bf 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -23,7 +23,7 @@
 #include "hw/pci/pci.h"
 #include "hw/pci-host/dino.h"
 #include "hw/misc/lasi.h"
-#include "hppa_sys.h"
+#include "hppa_hardware.h"
 #include "qemu/units.h"
 #include "qapi/error.h"
 #include "net/net.h"

From c165905c1764bebf90e31527fafbe0f1fa7c888e Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:58 +0100
Subject: [PATCH 169/935] hppa: use MACHINE QOM macros for defining the hppa
 machine

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-49-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/machine.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 75820838bf..2393335a64 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -471,8 +471,8 @@ static void machine_hppa_machine_init_class_init(ObjectClass *oc, void *data)
 }
 
 static const TypeInfo machine_hppa_machine_init_typeinfo = {
-    .name = ("hppa" "-machine"),
-    .parent = "machine",
+    .name = MACHINE_TYPE_NAME("hppa"),
+    .parent = TYPE_MACHINE,
     .class_init = machine_hppa_machine_init_class_init,
     .interfaces = (InterfaceInfo[]) {
         { TYPE_NMI },

From 42cc2bf63c2b61b43300e70a4965aa9d48eda790 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:25:59 +0100
Subject: [PATCH 170/935] hppa: fold machine_hppa_machine_init() into
 machine_hppa_machine_init_class_init()

There is no need for a separate function to set the machine class properties
separately from the others.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-50-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/machine.c | 12 ++++--------
 1 file changed, 4 insertions(+), 8 deletions(-)

diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 2393335a64..724ee24795 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -446,8 +446,11 @@ static void hppa_nmi(NMIState *n, int cpu_index, Error **errp)
     }
 }
 
-static void machine_hppa_machine_init(MachineClass *mc)
+static void machine_hppa_machine_init_class_init(ObjectClass *oc, void *data)
 {
+    MachineClass *mc = MACHINE_CLASS(oc);
+    NMIClass *nc = NMI_CLASS(oc);
+
     mc->desc = "HPPA B160L machine";
     mc->default_cpu_type = TYPE_HPPA_CPU;
     mc->init = machine_hppa_init;
@@ -459,14 +462,7 @@ static void machine_hppa_machine_init(MachineClass *mc)
     mc->default_ram_size = 512 * MiB;
     mc->default_boot_order = "cd";
     mc->default_ram_id = "ram";
-}
 
-static void machine_hppa_machine_init_class_init(ObjectClass *oc, void *data)
-{
-    MachineClass *mc = MACHINE_CLASS(oc);
-    machine_hppa_machine_init(mc);
-
-    NMIClass *nc = NMI_CLASS(oc);
     nc->nmi_monitor_handler = hppa_nmi;
 }
 

From 297d410387cb5a93bb32469aef34b02843d99b71 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 10:26:00 +0100
Subject: [PATCH 171/935] hppa: simplify machine function names in machine.c

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20220504092600.10048-51-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/machine.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 724ee24795..ae0bc07e75 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -446,7 +446,7 @@ static void hppa_nmi(NMIState *n, int cpu_index, Error **errp)
     }
 }
 
-static void machine_hppa_machine_init_class_init(ObjectClass *oc, void *data)
+static void hppa_machine_init_class_init(ObjectClass *oc, void *data)
 {
     MachineClass *mc = MACHINE_CLASS(oc);
     NMIClass *nc = NMI_CLASS(oc);
@@ -466,19 +466,19 @@ static void machine_hppa_machine_init_class_init(ObjectClass *oc, void *data)
     nc->nmi_monitor_handler = hppa_nmi;
 }
 
-static const TypeInfo machine_hppa_machine_init_typeinfo = {
+static const TypeInfo hppa_machine_init_typeinfo = {
     .name = MACHINE_TYPE_NAME("hppa"),
     .parent = TYPE_MACHINE,
-    .class_init = machine_hppa_machine_init_class_init,
+    .class_init = hppa_machine_init_class_init,
     .interfaces = (InterfaceInfo[]) {
         { TYPE_NMI },
         { }
     },
 };
 
-static void machine_hppa_machine_init_register_types(void)
+static void hppa_machine_init_register_types(void)
 {
-    type_register_static(&machine_hppa_machine_init_typeinfo);
+    type_register_static(&hppa_machine_init_typeinfo);
 }
 
-type_init(machine_hppa_machine_init_register_types)
+type_init(hppa_machine_init_register_types)

From 5d61789e04bf3313d012eb5c04d9de234013ff39 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 16:37:06 +0100
Subject: [PATCH 172/935] artist: checkpatch and newline style fixes

Ensure that subsequent patches do not cause checkpatch to fail and also tidy up
extra/missing newlines.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Message-Id: <20220504153708.10352-2-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Helge Deller <deller@gmx.de>
Reviewed-by: Sven Schnelle <svens@stackframe.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/display/artist.c | 23 +++++++++++------------
 1 file changed, 11 insertions(+), 12 deletions(-)

diff --git a/hw/display/artist.c b/hw/display/artist.c
index 8e121bb0b4..be92113494 100644
--- a/hw/display/artist.c
+++ b/hw/display/artist.c
@@ -211,8 +211,9 @@ static void artist_invalidate_lines(struct vram_buffer *buf,
     int start = starty * buf->width;
     int size;
 
-    if (starty + height > buf->height)
+    if (starty + height > buf->height) {
         height = buf->height - starty;
+    }
 
     size = height * buf->width;
 
@@ -321,8 +322,9 @@ static void artist_get_cursor_pos(ARTISTState *s, int *x, int *y)
     }
 
     lx = artist_get_x(s->cursor_pos);
-    if (lx < offset)
+    if (lx < offset) {
         offset = lx;
+    }
     *x = (lx - offset) / 2;
 
     *y = 1146 - artist_get_y(s->cursor_pos);
@@ -343,6 +345,7 @@ static void artist_get_cursor_pos(ARTISTState *s, int *x, int *y)
 static void artist_invalidate_cursor(ARTISTState *s)
 {
     int x, y;
+
     artist_get_cursor_pos(s, &x, &y);
     artist_invalidate_lines(&s->vram_buffer[ARTIST_BUFFER_AP],
                             y, s->cursor_height);
@@ -470,10 +473,9 @@ static void draw_line(ARTISTState *s,
 
     if ((x1 >= buf->width && x2 >= buf->width) ||
         (y1 >= buf->height && y2 >= buf->height)) {
-	return;
+        return;
     }
 
-
     if (update_start) {
         s->vram_start = (x2 << 16) | y2;
     }
@@ -553,15 +555,15 @@ static void draw_line(ARTISTState *s,
         x++;
     } while (x <= x2 && (max_pix == -1 || --max_pix > 0));
 
-    if (c1)
+    if (c1) {
         artist_invalidate_lines(buf, x1, x2 - x1);
-    else
+    } else {
         artist_invalidate_lines(buf, y1 > y2 ? y2 : y1, x2 - x1);
+    }
 }
 
 static void draw_line_pattern_start(ARTISTState *s)
 {
-
     int startx = artist_get_x(s->vram_start);
     int starty = artist_get_y(s->vram_start);
     int endx = artist_get_x(s->blockmove_size);
@@ -574,7 +576,6 @@ static void draw_line_pattern_start(ARTISTState *s)
 
 static void draw_line_pattern_next(ARTISTState *s)
 {
-
     int startx = artist_get_x(s->vram_start);
     int starty = artist_get_y(s->vram_start);
     int endx = artist_get_x(s->blockmove_size);
@@ -589,7 +590,6 @@ static void draw_line_pattern_next(ARTISTState *s)
 
 static void draw_line_size(ARTISTState *s, bool update_start)
 {
-
     int startx = artist_get_x(s->vram_start);
     int starty = artist_get_y(s->vram_start);
     int endx = artist_get_x(s->line_size);
@@ -600,7 +600,6 @@ static void draw_line_size(ARTISTState *s, bool update_start)
 
 static void draw_line_xy(ARTISTState *s, bool update_start)
 {
-
     int startx = artist_get_x(s->vram_start);
     int starty = artist_get_y(s->vram_start);
     int sizex = artist_get_x(s->blockmove_size);
@@ -650,7 +649,6 @@ static void draw_line_xy(ARTISTState *s, bool update_start)
 
 static void draw_line_end(ARTISTState *s, bool update_start)
 {
-
     int startx = artist_get_x(s->vram_start);
     int starty = artist_get_y(s->vram_start);
     int endx = artist_get_x(s->line_end);
@@ -835,6 +833,7 @@ static void artist_vram_write(void *opaque, hwaddr addr, uint64_t val,
                               unsigned size)
 {
     ARTISTState *s = opaque;
+
     s->vram_char_y = 0;
     trace_artist_vram_write(size, addr, val);
     vram_bit_write(opaque, addr, 0, val, size);
@@ -1244,7 +1243,6 @@ static void artist_update_display(void *opaque)
     DisplaySurface *surface = qemu_console_surface(s->con);
     int first = 0, last;
 
-
     framebuffer_update_display(surface, &s->fbsection, s->width, s->height,
                                s->width, s->width * 4, 0, 0, artist_draw_line,
                                s, &first, &last);
@@ -1258,6 +1256,7 @@ static void artist_invalidate(void *opaque)
 {
     ARTISTState *s = ARTIST(opaque);
     struct vram_buffer *buf = &s->vram_buffer[ARTIST_BUFFER_AP];
+
     memory_region_set_dirty(&buf->mr, 0, buf->size);
 }
 

From ada804eb8972c42418a8052c0a1e0493714c75f5 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 16:37:07 +0100
Subject: [PATCH 173/935] artist: remove unused ROP8OFF() macro

This macro is unused and so can simply be removed.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Message-Id: <20220504153708.10352-3-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Helge Deller <deller@gmx.de>
Reviewed-by: Sven Schnelle <svens@stackframe.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/display/artist.c | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/hw/display/artist.c b/hw/display/artist.c
index be92113494..4a9f7b5e59 100644
--- a/hw/display/artist.c
+++ b/hw/display/artist.c
@@ -25,12 +25,6 @@
 #define TYPE_ARTIST "artist"
 OBJECT_DECLARE_SIMPLE_TYPE(ARTISTState, ARTIST)
 
-#if HOST_BIG_ENDIAN
-#define ROP8OFF(_i) (3 - (_i))
-#else
-#define ROP8OFF
-#endif
-
 struct vram_buffer {
     MemoryRegion mr;
     uint8_t *data;

From 1f110516190f146df40f49aef2f1ef5903b8f540 Mon Sep 17 00:00:00 2001
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Date: Wed, 4 May 2022 16:37:08 +0100
Subject: [PATCH 174/935] artist: only render dirty scanlines on the display
 surface

The framebuffer_update_display() function returns the dirty scanlines that were
touched since the last display update, however artist_update_display() always calls
dpy_gfx_update() with start and end scanlines of 0 and s->height causing the
entire display surface to be rendered on every update.

Update artist_update_display() so that dpy_gfx_update() only renders the dirty
scanlines on the display surface, bypassing the display surface rendering
completely if framebuffer_update_display() indicates no changes occurred.

This noticeably improves boot performance when the framebuffer is enabled on my
rather modest laptop here, including making the GTK UI usable.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Message-Id: <20220504153708.10352-4-mark.cave-ayland@ilande.co.uk>
Reviewed-by: Helge Deller <deller@gmx.de>
Reviewed-by: Sven Schnelle <svens@stackframe.org>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/display/artist.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/hw/display/artist.c b/hw/display/artist.c
index 4a9f7b5e59..39fc0c4ca5 100644
--- a/hw/display/artist.c
+++ b/hw/display/artist.c
@@ -1243,7 +1243,9 @@ static void artist_update_display(void *opaque)
 
     artist_draw_cursor(s);
 
-    dpy_gfx_update(s->con, 0, 0, s->width, s->height);
+    if (first >= 0) {
+        dpy_gfx_update(s->con, 0, first, s->width, last - first + 1);
+    }
 }
 
 static void artist_invalidate(void *opaque)

From 82f96346e1fc8eb81eb8e80fe0b95405a8cf42ee Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Tue, 12 Apr 2022 18:58:35 +0200
Subject: [PATCH 175/935] disas: Remove old libopcode arm disassembler

Capstone should be superior to the old libopcode disassembler, so
we can drop the old file nowadays.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220412165836.355850-3-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 MAINTAINERS             |    2 -
 disas.c                 |    1 -
 disas/arm.c             | 4012 ---------------------------------------
 disas/meson.build       |    1 -
 include/disas/dis-asm.h |    1 -
 target/arm/cpu.c        |    8 -
 6 files changed, 4025 deletions(-)
 delete mode 100644 disas/arm.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 662ec47246..7f3788b694 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -165,7 +165,6 @@ F: tests/qtest/arm-cpu-features.c
 F: hw/arm/
 F: hw/cpu/a*mpcore.c
 F: include/hw/cpu/a*mpcore.h
-F: disas/arm.c
 F: disas/arm-a64.cc
 F: disas/libvixl/
 F: docs/system/target-arm.rst
@@ -3271,7 +3270,6 @@ M: Richard Henderson <richard.henderson@linaro.org>
 S: Maintained
 L: qemu-arm@nongnu.org
 F: tcg/arm/
-F: disas/arm.c
 
 i386 TCG target
 M: Richard Henderson <richard.henderson@linaro.org>
diff --git a/disas.c b/disas.c
index d41f34915d..4e443b4d49 100644
--- a/disas.c
+++ b/disas.c
@@ -192,7 +192,6 @@ static void initialize_debug_host(CPUDebug *s)
     s->info.mach = bfd_mach_sparc_v9b;
 #elif defined(__arm__)
     /* TCG only generates code for arm mode.  */
-    s->info.print_insn = print_insn_arm;
     s->info.cap_arch = CS_ARCH_ARM;
 #elif defined(__MIPSEB__)
     s->info.print_insn = print_insn_big_mips;
diff --git a/disas/arm.c b/disas/arm.c
deleted file mode 100644
index 7d940f2396..0000000000
--- a/disas/arm.c
+++ /dev/null
@@ -1,4012 +0,0 @@
-/* Instruction printing code for the ARM
-   Copyright 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004
-   2007, Free Software Foundation, Inc.
-   Contributed by Richard Earnshaw (rwe@pegasus.esprit.ec.org)
-   Modification by James G. Smith (jsmith@cygnus.co.uk)
-
-   This file is part of libopcodes.
-
-   This program is free software; you can redistribute it and/or modify it under
-   the terms of the GNU General Public License as published by the Free
-   Software Foundation; either version 2 of the License, or (at your option)
-   any later version.
-
-   This program is distributed in the hope that it will be useful, but WITHOUT
-   ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
-   FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
-   more details.
-
-   You should have received a copy of the GNU General Public License
-   along with this program; if not, see <http://www.gnu.org/licenses/>.  */
-
-/* Start of qemu specific additions.  Mostly this is stub definitions
-   for things we don't care about.  */
-
-#include "qemu/osdep.h"
-#include "disas/dis-asm.h"
-
-#define ARM_EXT_V1	 0
-#define ARM_EXT_V2	 0
-#define ARM_EXT_V2S	 0
-#define ARM_EXT_V3	 0
-#define ARM_EXT_V3M	 0
-#define ARM_EXT_V4	 0
-#define ARM_EXT_V4T	 0
-#define ARM_EXT_V5	 0
-#define ARM_EXT_V5T	 0
-#define ARM_EXT_V5ExP	 0
-#define ARM_EXT_V5E	 0
-#define ARM_EXT_V5J	 0
-#define ARM_EXT_V6       0
-#define ARM_EXT_V6K      0
-#define ARM_EXT_V6Z      0
-#define ARM_EXT_V6T2	 0
-#define ARM_EXT_V7	 0
-#define ARM_EXT_DIV	 0
-
-/* Co-processor space extensions.  */
-#define ARM_CEXT_XSCALE   0
-#define ARM_CEXT_MAVERICK 0
-#define ARM_CEXT_IWMMXT   0
-
-#define FPU_FPA_EXT_V1	 0
-#define FPU_FPA_EXT_V2	 0
-#define FPU_VFP_EXT_NONE 0
-#define FPU_VFP_EXT_V1xD 0
-#define FPU_VFP_EXT_V1	 0
-#define FPU_VFP_EXT_V2	 0
-#define FPU_MAVERICK	 0
-#define FPU_VFP_EXT_V3	 0
-#define FPU_NEON_EXT_V1	 0
-
-/* Assume host uses ieee float.  */
-static void floatformat_to_double (unsigned char *data, double *dest)
-{
-    union {
-        uint32_t i;
-        float f;
-    } u;
-    u.i = data[0] | (data[1] << 8) | (data[2] << 16) | (data[3] << 24);
-    *dest = u.f;
-}
-
-static int arm_read_memory(bfd_vma memaddr, bfd_byte *b, int length,
-                           struct disassemble_info *info)
-{
-    assert((info->flags & INSN_ARM_BE32) == 0 || length == 2 || length == 4);
-
-    if ((info->flags & INSN_ARM_BE32) != 0 && length == 2) {
-        memaddr ^= 2;
-    }
-    return info->read_memory_func(memaddr, b, length, info);
-}
-
-/* End of qemu specific additions.  */
-
-struct opcode32
-{
-  unsigned long arch;		/* Architecture defining this insn.  */
-  unsigned long value, mask;	/* Recognise insn if (op&mask)==value.  */
-  const char *assembler;	/* How to disassemble this insn.  */
-};
-
-struct opcode16
-{
-  unsigned long arch;		/* Architecture defining this insn.  */
-  unsigned short value, mask;	/* Recognise insn if (op&mask)==value.  */
-  const char *assembler;	/* How to disassemble this insn.  */
-};
-
-/* print_insn_coprocessor recognizes the following format control codes:
-
-   %%			%
-
-   %c			print condition code (always bits 28-31 in ARM mode)
-   %q			print shifter argument
-   %u			print condition code (unconditional in ARM mode)
-   %A			print address for ldc/stc/ldf/stf instruction
-   %B			print vstm/vldm register list
-   %C			print vstr/vldr address operand
-   %I                   print cirrus signed shift immediate: bits 0..3|4..6
-   %F			print the COUNT field of a LFM/SFM instruction.
-   %P			print floating point precision in arithmetic insn
-   %Q			print floating point precision in ldf/stf insn
-   %R			print floating point rounding mode
-
-   %<bitfield>r		print as an ARM register
-   %<bitfield>d		print the bitfield in decimal
-   %<bitfield>k		print immediate for VFPv3 conversion instruction
-   %<bitfield>x		print the bitfield in hex
-   %<bitfield>X		print the bitfield as 1 hex digit without leading "0x"
-   %<bitfield>f		print a floating point constant if >7 else a
-			floating point register
-   %<bitfield>w         print as an iWMMXt width field - [bhwd]ss/us
-   %<bitfield>g         print as an iWMMXt 64-bit register
-   %<bitfield>G         print as an iWMMXt general purpose or control register
-   %<bitfield>D		print as a NEON D register
-   %<bitfield>Q		print as a NEON Q register
-
-   %y<code>		print a single precision VFP reg.
-			  Codes: 0=>Sm, 1=>Sd, 2=>Sn, 3=>multi-list, 4=>Sm pair
-   %z<code>		print a double precision VFP reg
-			  Codes: 0=>Dm, 1=>Dd, 2=>Dn, 3=>multi-list
-
-   %<bitfield>'c	print specified char iff bitfield is all ones
-   %<bitfield>`c	print specified char iff bitfield is all zeroes
-   %<bitfield>?ab...    select from array of values in big endian order
-
-   %L			print as an iWMMXt N/M width field.
-   %Z			print the Immediate of a WSHUFH instruction.
-   %l			like 'A' except use byte offsets for 'B' & 'H'
-			versions.
-   %i			print 5-bit immediate in bits 8,3..0
-			(print "32" when 0)
-   %r			print register offset address for wldt/wstr instruction
-*/
-
-/* Common coprocessor opcodes shared between Arm and Thumb-2.  */
-
-static const struct opcode32 coprocessor_opcodes[] =
-{
-  /* XScale instructions.  */
-  {ARM_CEXT_XSCALE, 0x0e200010, 0x0fff0ff0, "mia%c\tacc0, %0-3r, %12-15r"},
-  {ARM_CEXT_XSCALE, 0x0e280010, 0x0fff0ff0, "miaph%c\tacc0, %0-3r, %12-15r"},
-  {ARM_CEXT_XSCALE, 0x0e2c0010, 0x0ffc0ff0, "mia%17'T%17`B%16'T%16`B%c\tacc0, %0-3r, %12-15r"},
-  {ARM_CEXT_XSCALE, 0x0c400000, 0x0ff00fff, "mar%c\tacc0, %12-15r, %16-19r"},
-  {ARM_CEXT_XSCALE, 0x0c500000, 0x0ff00fff, "mra%c\t%12-15r, %16-19r, acc0"},
-
-  /* Intel Wireless MMX technology instructions.  */
-#define FIRST_IWMMXT_INSN 0x0e130130
-#define IWMMXT_INSN_COUNT 73
-  {ARM_CEXT_IWMMXT, 0x0e130130, 0x0f3f0fff, "tandc%22-23w%c\t%12-15r"},
-  {ARM_CEXT_XSCALE, 0x0e400010, 0x0ff00f3f, "tbcst%6-7w%c\t%16-19g, %12-15r"},
-  {ARM_CEXT_XSCALE, 0x0e130170, 0x0f3f0ff8, "textrc%22-23w%c\t%12-15r, #%0-2d"},
-  {ARM_CEXT_XSCALE, 0x0e100070, 0x0f300ff0, "textrm%3?su%22-23w%c\t%12-15r, %16-19g, #%0-2d"},
-  {ARM_CEXT_XSCALE, 0x0e600010, 0x0ff00f38, "tinsr%6-7w%c\t%16-19g, %12-15r, #%0-2d"},
-  {ARM_CEXT_XSCALE, 0x0e000110, 0x0ff00fff, "tmcr%c\t%16-19G, %12-15r"},
-  {ARM_CEXT_XSCALE, 0x0c400000, 0x0ff00ff0, "tmcrr%c\t%0-3g, %12-15r, %16-19r"},
-  {ARM_CEXT_XSCALE, 0x0e2c0010, 0x0ffc0e10, "tmia%17?tb%16?tb%c\t%5-8g, %0-3r, %12-15r"},
-  {ARM_CEXT_XSCALE, 0x0e200010, 0x0fff0e10, "tmia%c\t%5-8g, %0-3r, %12-15r"},
-  {ARM_CEXT_XSCALE, 0x0e280010, 0x0fff0e10, "tmiaph%c\t%5-8g, %0-3r, %12-15r"},
-  {ARM_CEXT_XSCALE, 0x0e100030, 0x0f300fff, "tmovmsk%22-23w%c\t%12-15r, %16-19g"},
-  {ARM_CEXT_XSCALE, 0x0e100110, 0x0ff00ff0, "tmrc%c\t%12-15r, %16-19G"},
-  {ARM_CEXT_XSCALE, 0x0c500000, 0x0ff00ff0, "tmrrc%c\t%12-15r, %16-19r, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e130150, 0x0f3f0fff, "torc%22-23w%c\t%12-15r"},
-  {ARM_CEXT_XSCALE, 0x0e130190, 0x0f3f0fff, "torvsc%22-23w%c\t%12-15r"},
-  {ARM_CEXT_XSCALE, 0x0e2001c0, 0x0f300fff, "wabs%22-23w%c\t%12-15g, %16-19g"},
-  {ARM_CEXT_XSCALE, 0x0e0001c0, 0x0f300fff, "wacc%22-23w%c\t%12-15g, %16-19g"},
-  {ARM_CEXT_XSCALE, 0x0e000180, 0x0f000ff0, "wadd%20-23w%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e2001a0, 0x0f300ff0, "waddbhus%22?ml%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0ea001a0, 0x0ff00ff0, "waddsubhx%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e000020, 0x0f800ff0, "waligni%c\t%12-15g, %16-19g, %0-3g, #%20-22d"},
-  {ARM_CEXT_XSCALE, 0x0e800020, 0x0fc00ff0, "walignr%20-21d%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e200000, 0x0fe00ff0, "wand%20'n%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e800000, 0x0fa00ff0, "wavg2%22?hb%20'r%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e400000, 0x0fe00ff0, "wavg4%20'r%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e000060, 0x0f300ff0, "wcmpeq%22-23w%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e100060, 0x0f100ff0, "wcmpgt%21?su%22-23w%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0xfc500100, 0xfe500f00, "wldrd\t%12-15g, %r"},
-  {ARM_CEXT_XSCALE, 0xfc100100, 0xfe500f00, "wldrw\t%12-15G, %A"},
-  {ARM_CEXT_XSCALE, 0x0c100000, 0x0e100e00, "wldr%L%c\t%12-15g, %l"},
-  {ARM_CEXT_XSCALE, 0x0e400100, 0x0fc00ff0, "wmac%21?su%20'z%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e800100, 0x0fc00ff0, "wmadd%21?su%20'x%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0ec00100, 0x0fd00ff0, "wmadd%21?sun%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e000160, 0x0f100ff0, "wmax%21?su%22-23w%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e000080, 0x0f100fe0, "wmerge%c\t%12-15g, %16-19g, %0-3g, #%21-23d"},
-  {ARM_CEXT_XSCALE, 0x0e0000a0, 0x0f800ff0, "wmia%21?tb%20?tb%22'n%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e800120, 0x0f800ff0, "wmiaw%21?tb%20?tb%22'n%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e100160, 0x0f100ff0, "wmin%21?su%22-23w%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e000100, 0x0fc00ff0, "wmul%21?su%20?ml%23'r%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0ed00100, 0x0fd00ff0, "wmul%21?sumr%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0ee000c0, 0x0fe00ff0, "wmulwsm%20`r%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0ec000c0, 0x0fe00ff0, "wmulwum%20`r%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0eb000c0, 0x0ff00ff0, "wmulwl%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e8000a0, 0x0f800ff0, "wqmia%21?tb%20?tb%22'n%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e100080, 0x0fd00ff0, "wqmulm%21'r%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0ec000e0, 0x0fd00ff0, "wqmulwm%21'r%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e000000, 0x0ff00ff0, "wor%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e000080, 0x0f000ff0, "wpack%20-23w%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0xfe300040, 0xff300ef0, "wror%22-23w\t%12-15g, %16-19g, #%i"},
-  {ARM_CEXT_XSCALE, 0x0e300040, 0x0f300ff0, "wror%22-23w%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e300140, 0x0f300ff0, "wror%22-23wg%c\t%12-15g, %16-19g, %0-3G"},
-  {ARM_CEXT_XSCALE, 0x0e000120, 0x0fa00ff0, "wsad%22?hb%20'z%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e0001e0, 0x0f000ff0, "wshufh%c\t%12-15g, %16-19g, #%Z"},
-  {ARM_CEXT_XSCALE, 0xfe100040, 0xff300ef0, "wsll%22-23w\t%12-15g, %16-19g, #%i"},
-  {ARM_CEXT_XSCALE, 0x0e100040, 0x0f300ff0, "wsll%22-23w%8'g%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e100148, 0x0f300ffc, "wsll%22-23w%8'g%c\t%12-15g, %16-19g, %0-3G"},
-  {ARM_CEXT_XSCALE, 0xfe000040, 0xff300ef0, "wsra%22-23w\t%12-15g, %16-19g, #%i"},
-  {ARM_CEXT_XSCALE, 0x0e000040, 0x0f300ff0, "wsra%22-23w%8'g%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e000148, 0x0f300ffc, "wsra%22-23w%8'g%c\t%12-15g, %16-19g, %0-3G"},
-  {ARM_CEXT_XSCALE, 0xfe200040, 0xff300ef0, "wsrl%22-23w\t%12-15g, %16-19g, #%i"},
-  {ARM_CEXT_XSCALE, 0x0e200040, 0x0f300ff0, "wsrl%22-23w%8'g%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e200148, 0x0f300ffc, "wsrl%22-23w%8'g%c\t%12-15g, %16-19g, %0-3G"},
-  {ARM_CEXT_XSCALE, 0xfc400100, 0xfe500f00, "wstrd\t%12-15g, %r"},
-  {ARM_CEXT_XSCALE, 0xfc000100, 0xfe500f00, "wstrw\t%12-15G, %A"},
-  {ARM_CEXT_XSCALE, 0x0c000000, 0x0e100e00, "wstr%L%c\t%12-15g, %l"},
-  {ARM_CEXT_XSCALE, 0x0e0001a0, 0x0f000ff0, "wsub%20-23w%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0ed001c0, 0x0ff00ff0, "wsubaddhx%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e1001c0, 0x0f300ff0, "wabsdiff%22-23w%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e0000c0, 0x0fd00fff, "wunpckeh%21?sub%c\t%12-15g, %16-19g"},
-  {ARM_CEXT_XSCALE, 0x0e4000c0, 0x0fd00fff, "wunpckeh%21?suh%c\t%12-15g, %16-19g"},
-  {ARM_CEXT_XSCALE, 0x0e8000c0, 0x0fd00fff, "wunpckeh%21?suw%c\t%12-15g, %16-19g"},
-  {ARM_CEXT_XSCALE, 0x0e0000e0, 0x0f100fff, "wunpckel%21?su%22-23w%c\t%12-15g, %16-19g"},
-  {ARM_CEXT_XSCALE, 0x0e1000c0, 0x0f300ff0, "wunpckih%22-23w%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e1000e0, 0x0f300ff0, "wunpckil%22-23w%c\t%12-15g, %16-19g, %0-3g"},
-  {ARM_CEXT_XSCALE, 0x0e100000, 0x0ff00ff0, "wxor%c\t%12-15g, %16-19g, %0-3g"},
-
-  /* Floating point coprocessor (FPA) instructions */
-  {FPU_FPA_EXT_V1, 0x0e000100, 0x0ff08f10, "adf%c%P%R\t%12-14f, %16-18f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e100100, 0x0ff08f10, "muf%c%P%R\t%12-14f, %16-18f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e200100, 0x0ff08f10, "suf%c%P%R\t%12-14f, %16-18f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e300100, 0x0ff08f10, "rsf%c%P%R\t%12-14f, %16-18f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e400100, 0x0ff08f10, "dvf%c%P%R\t%12-14f, %16-18f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e500100, 0x0ff08f10, "rdf%c%P%R\t%12-14f, %16-18f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e600100, 0x0ff08f10, "pow%c%P%R\t%12-14f, %16-18f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e700100, 0x0ff08f10, "rpw%c%P%R\t%12-14f, %16-18f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e800100, 0x0ff08f10, "rmf%c%P%R\t%12-14f, %16-18f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e900100, 0x0ff08f10, "fml%c%P%R\t%12-14f, %16-18f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0ea00100, 0x0ff08f10, "fdv%c%P%R\t%12-14f, %16-18f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0eb00100, 0x0ff08f10, "frd%c%P%R\t%12-14f, %16-18f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0ec00100, 0x0ff08f10, "pol%c%P%R\t%12-14f, %16-18f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e008100, 0x0ff08f10, "mvf%c%P%R\t%12-14f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e108100, 0x0ff08f10, "mnf%c%P%R\t%12-14f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e208100, 0x0ff08f10, "abs%c%P%R\t%12-14f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e308100, 0x0ff08f10, "rnd%c%P%R\t%12-14f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e408100, 0x0ff08f10, "sqt%c%P%R\t%12-14f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e508100, 0x0ff08f10, "log%c%P%R\t%12-14f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e608100, 0x0ff08f10, "lgn%c%P%R\t%12-14f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e708100, 0x0ff08f10, "exp%c%P%R\t%12-14f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e808100, 0x0ff08f10, "sin%c%P%R\t%12-14f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e908100, 0x0ff08f10, "cos%c%P%R\t%12-14f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0ea08100, 0x0ff08f10, "tan%c%P%R\t%12-14f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0eb08100, 0x0ff08f10, "asn%c%P%R\t%12-14f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0ec08100, 0x0ff08f10, "acs%c%P%R\t%12-14f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0ed08100, 0x0ff08f10, "atn%c%P%R\t%12-14f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0ee08100, 0x0ff08f10, "urd%c%P%R\t%12-14f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0ef08100, 0x0ff08f10, "nrm%c%P%R\t%12-14f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0e000110, 0x0ff00f1f, "flt%c%P%R\t%16-18f, %12-15r"},
-  {FPU_FPA_EXT_V1, 0x0e100110, 0x0fff0f98, "fix%c%R\t%12-15r, %0-2f"},
-  {FPU_FPA_EXT_V1, 0x0e200110, 0x0fff0fff, "wfs%c\t%12-15r"},
-  {FPU_FPA_EXT_V1, 0x0e300110, 0x0fff0fff, "rfs%c\t%12-15r"},
-  {FPU_FPA_EXT_V1, 0x0e400110, 0x0fff0fff, "wfc%c\t%12-15r"},
-  {FPU_FPA_EXT_V1, 0x0e500110, 0x0fff0fff, "rfc%c\t%12-15r"},
-  {FPU_FPA_EXT_V1, 0x0e90f110, 0x0ff8fff0, "cmf%c\t%16-18f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0eb0f110, 0x0ff8fff0, "cnf%c\t%16-18f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0ed0f110, 0x0ff8fff0, "cmfe%c\t%16-18f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0ef0f110, 0x0ff8fff0, "cnfe%c\t%16-18f, %0-3f"},
-  {FPU_FPA_EXT_V1, 0x0c000100, 0x0e100f00, "stf%c%Q\t%12-14f, %A"},
-  {FPU_FPA_EXT_V1, 0x0c100100, 0x0e100f00, "ldf%c%Q\t%12-14f, %A"},
-  {FPU_FPA_EXT_V2, 0x0c000200, 0x0e100f00, "sfm%c\t%12-14f, %F, %A"},
-  {FPU_FPA_EXT_V2, 0x0c100200, 0x0e100f00, "lfm%c\t%12-14f, %F, %A"},
-
-  /* Register load/store */
-  {FPU_NEON_EXT_V1, 0x0d200b00, 0x0fb00f01, "vstmdb%c\t%16-19r%21'!, %B"},
-  {FPU_NEON_EXT_V1, 0x0d300b00, 0x0fb00f01, "vldmdb%c\t%16-19r%21'!, %B"},
-  {FPU_NEON_EXT_V1, 0x0c800b00, 0x0f900f01, "vstmia%c\t%16-19r%21'!, %B"},
-  {FPU_NEON_EXT_V1, 0x0c900b00, 0x0f900f01, "vldmia%c\t%16-19r%21'!, %B"},
-  {FPU_NEON_EXT_V1, 0x0d000b00, 0x0f300f00, "vstr%c\t%12-15,22D, %C"},
-  {FPU_NEON_EXT_V1, 0x0d100b00, 0x0f300f00, "vldr%c\t%12-15,22D, %C"},
-
-  /* Data transfer between ARM and NEON registers */
-  {FPU_NEON_EXT_V1, 0x0e800b10, 0x0ff00f70, "vdup%c.32\t%16-19,7D, %12-15r"},
-  {FPU_NEON_EXT_V1, 0x0e800b30, 0x0ff00f70, "vdup%c.16\t%16-19,7D, %12-15r"},
-  {FPU_NEON_EXT_V1, 0x0ea00b10, 0x0ff00f70, "vdup%c.32\t%16-19,7Q, %12-15r"},
-  {FPU_NEON_EXT_V1, 0x0ea00b30, 0x0ff00f70, "vdup%c.16\t%16-19,7Q, %12-15r"},
-  {FPU_NEON_EXT_V1, 0x0ec00b10, 0x0ff00f70, "vdup%c.8\t%16-19,7D, %12-15r"},
-  {FPU_NEON_EXT_V1, 0x0ee00b10, 0x0ff00f70, "vdup%c.8\t%16-19,7Q, %12-15r"},
-  {FPU_NEON_EXT_V1, 0x0c400b10, 0x0ff00fd0, "vmov%c\t%0-3,5D, %12-15r, %16-19r"},
-  {FPU_NEON_EXT_V1, 0x0c500b10, 0x0ff00fd0, "vmov%c\t%12-15r, %16-19r, %0-3,5D"},
-  {FPU_NEON_EXT_V1, 0x0e000b10, 0x0fd00f70, "vmov%c.32\t%16-19,7D[%21d], %12-15r"},
-  {FPU_NEON_EXT_V1, 0x0e100b10, 0x0f500f70, "vmov%c.32\t%12-15r, %16-19,7D[%21d]"},
-  {FPU_NEON_EXT_V1, 0x0e000b30, 0x0fd00f30, "vmov%c.16\t%16-19,7D[%6,21d], %12-15r"},
-  {FPU_NEON_EXT_V1, 0x0e100b30, 0x0f500f30, "vmov%c.%23?us16\t%12-15r, %16-19,7D[%6,21d]"},
-  {FPU_NEON_EXT_V1, 0x0e400b10, 0x0fd00f10, "vmov%c.8\t%16-19,7D[%5,6,21d], %12-15r"},
-  {FPU_NEON_EXT_V1, 0x0e500b10, 0x0f500f10, "vmov%c.%23?us8\t%12-15r, %16-19,7D[%5,6,21d]"},
-
-  /* Floating point coprocessor (VFP) instructions */
-  {FPU_VFP_EXT_V1xD, 0x0ef1fa10, 0x0fffffff, "fmstat%c"},
-  {FPU_VFP_EXT_V1xD, 0x0ee00a10, 0x0fff0fff, "fmxr%c\tfpsid, %12-15r"},
-  {FPU_VFP_EXT_V1xD, 0x0ee10a10, 0x0fff0fff, "fmxr%c\tfpscr, %12-15r"},
-  {FPU_VFP_EXT_V1xD, 0x0ee60a10, 0x0fff0fff, "fmxr%c\tmvfr1, %12-15r"},
-  {FPU_VFP_EXT_V1xD, 0x0ee70a10, 0x0fff0fff, "fmxr%c\tmvfr0, %12-15r"},
-  {FPU_VFP_EXT_V1xD, 0x0ee80a10, 0x0fff0fff, "fmxr%c\tfpexc, %12-15r"},
-  {FPU_VFP_EXT_V1xD, 0x0ee90a10, 0x0fff0fff, "fmxr%c\tfpinst, %12-15r\t@ Impl def"},
-  {FPU_VFP_EXT_V1xD, 0x0eea0a10, 0x0fff0fff, "fmxr%c\tfpinst2, %12-15r\t@ Impl def"},
-  {FPU_VFP_EXT_V1xD, 0x0ef00a10, 0x0fff0fff, "fmrx%c\t%12-15r, fpsid"},
-  {FPU_VFP_EXT_V1xD, 0x0ef10a10, 0x0fff0fff, "fmrx%c\t%12-15r, fpscr"},
-  {FPU_VFP_EXT_V1xD, 0x0ef60a10, 0x0fff0fff, "fmrx%c\t%12-15r, mvfr1"},
-  {FPU_VFP_EXT_V1xD, 0x0ef70a10, 0x0fff0fff, "fmrx%c\t%12-15r, mvfr0"},
-  {FPU_VFP_EXT_V1xD, 0x0ef80a10, 0x0fff0fff, "fmrx%c\t%12-15r, fpexc"},
-  {FPU_VFP_EXT_V1xD, 0x0ef90a10, 0x0fff0fff, "fmrx%c\t%12-15r, fpinst\t@ Impl def"},
-  {FPU_VFP_EXT_V1xD, 0x0efa0a10, 0x0fff0fff, "fmrx%c\t%12-15r, fpinst2\t@ Impl def"},
-  {FPU_VFP_EXT_V1, 0x0e000b10, 0x0ff00fff, "fmdlr%c\t%z2, %12-15r"},
-  {FPU_VFP_EXT_V1, 0x0e100b10, 0x0ff00fff, "fmrdl%c\t%12-15r, %z2"},
-  {FPU_VFP_EXT_V1, 0x0e200b10, 0x0ff00fff, "fmdhr%c\t%z2, %12-15r"},
-  {FPU_VFP_EXT_V1, 0x0e300b10, 0x0ff00fff, "fmrdh%c\t%12-15r, %z2"},
-  {FPU_VFP_EXT_V1xD, 0x0ee00a10, 0x0ff00fff, "fmxr%c\t<impl def %16-19x>, %12-15r"},
-  {FPU_VFP_EXT_V1xD, 0x0ef00a10, 0x0ff00fff, "fmrx%c\t%12-15r, <impl def %16-19x>"},
-  {FPU_VFP_EXT_V1xD, 0x0e000a10, 0x0ff00f7f, "fmsr%c\t%y2, %12-15r"},
-  {FPU_VFP_EXT_V1xD, 0x0e100a10, 0x0ff00f7f, "fmrs%c\t%12-15r, %y2"},
-  {FPU_VFP_EXT_V1xD, 0x0eb50a40, 0x0fbf0f70, "fcmp%7'ezs%c\t%y1"},
-  {FPU_VFP_EXT_V1, 0x0eb50b40, 0x0fbf0f70, "fcmp%7'ezd%c\t%z1"},
-  {FPU_VFP_EXT_V1xD, 0x0eb00a40, 0x0fbf0fd0, "fcpys%c\t%y1, %y0"},
-  {FPU_VFP_EXT_V1xD, 0x0eb00ac0, 0x0fbf0fd0, "fabss%c\t%y1, %y0"},
-  {FPU_VFP_EXT_V1, 0x0eb00b40, 0x0fbf0fd0, "fcpyd%c\t%z1, %z0"},
-  {FPU_VFP_EXT_V1, 0x0eb00bc0, 0x0fbf0fd0, "fabsd%c\t%z1, %z0"},
-  {FPU_VFP_EXT_V1xD, 0x0eb10a40, 0x0fbf0fd0, "fnegs%c\t%y1, %y0"},
-  {FPU_VFP_EXT_V1xD, 0x0eb10ac0, 0x0fbf0fd0, "fsqrts%c\t%y1, %y0"},
-  {FPU_VFP_EXT_V1, 0x0eb10b40, 0x0fbf0fd0, "fnegd%c\t%z1, %z0"},
-  {FPU_VFP_EXT_V1, 0x0eb10bc0, 0x0fbf0fd0, "fsqrtd%c\t%z1, %z0"},
-  {FPU_VFP_EXT_V1, 0x0eb70ac0, 0x0fbf0fd0, "fcvtds%c\t%z1, %y0"},
-  {FPU_VFP_EXT_V1, 0x0eb70bc0, 0x0fbf0fd0, "fcvtsd%c\t%y1, %z0"},
-  {FPU_VFP_EXT_V1xD, 0x0eb80a40, 0x0fbf0fd0, "fuitos%c\t%y1, %y0"},
-  {FPU_VFP_EXT_V1xD, 0x0eb80ac0, 0x0fbf0fd0, "fsitos%c\t%y1, %y0"},
-  {FPU_VFP_EXT_V1, 0x0eb80b40, 0x0fbf0fd0, "fuitod%c\t%z1, %y0"},
-  {FPU_VFP_EXT_V1, 0x0eb80bc0, 0x0fbf0fd0, "fsitod%c\t%z1, %y0"},
-  {FPU_VFP_EXT_V1xD, 0x0eb40a40, 0x0fbf0f50, "fcmp%7'es%c\t%y1, %y0"},
-  {FPU_VFP_EXT_V1, 0x0eb40b40, 0x0fbf0f50, "fcmp%7'ed%c\t%z1, %z0"},
-  {FPU_VFP_EXT_V3, 0x0eba0a40, 0x0fbe0f50, "f%16?us%7?lhtos%c\t%y1, #%5,0-3k"},
-  {FPU_VFP_EXT_V3, 0x0eba0b40, 0x0fbe0f50, "f%16?us%7?lhtod%c\t%z1, #%5,0-3k"},
-  {FPU_VFP_EXT_V1xD, 0x0ebc0a40, 0x0fbe0f50, "fto%16?sui%7'zs%c\t%y1, %y0"},
-  {FPU_VFP_EXT_V1, 0x0ebc0b40, 0x0fbe0f50, "fto%16?sui%7'zd%c\t%y1, %z0"},
-  {FPU_VFP_EXT_V3, 0x0ebe0a40, 0x0fbe0f50, "fto%16?us%7?lhs%c\t%y1, #%5,0-3k"},
-  {FPU_VFP_EXT_V3, 0x0ebe0b40, 0x0fbe0f50, "fto%16?us%7?lhd%c\t%z1, #%5,0-3k"},
-  {FPU_VFP_EXT_V1, 0x0c500b10, 0x0fb00ff0, "fmrrd%c\t%12-15r, %16-19r, %z0"},
-  {FPU_VFP_EXT_V3, 0x0eb00a00, 0x0fb00ff0, "fconsts%c\t%y1, #%0-3,16-19d"},
-  {FPU_VFP_EXT_V3, 0x0eb00b00, 0x0fb00ff0, "fconstd%c\t%z1, #%0-3,16-19d"},
-  {FPU_VFP_EXT_V2, 0x0c400a10, 0x0ff00fd0, "fmsrr%c\t%y4, %12-15r, %16-19r"},
-  {FPU_VFP_EXT_V2, 0x0c400b10, 0x0ff00fd0, "fmdrr%c\t%z0, %12-15r, %16-19r"},
-  {FPU_VFP_EXT_V2, 0x0c500a10, 0x0ff00fd0, "fmrrs%c\t%12-15r, %16-19r, %y4"},
-  {FPU_VFP_EXT_V1xD, 0x0e000a00, 0x0fb00f50, "fmacs%c\t%y1, %y2, %y0"},
-  {FPU_VFP_EXT_V1xD, 0x0e000a40, 0x0fb00f50, "fnmacs%c\t%y1, %y2, %y0"},
-  {FPU_VFP_EXT_V1, 0x0e000b00, 0x0fb00f50, "fmacd%c\t%z1, %z2, %z0"},
-  {FPU_VFP_EXT_V1, 0x0e000b40, 0x0fb00f50, "fnmacd%c\t%z1, %z2, %z0"},
-  {FPU_VFP_EXT_V1xD, 0x0e100a00, 0x0fb00f50, "fmscs%c\t%y1, %y2, %y0"},
-  {FPU_VFP_EXT_V1xD, 0x0e100a40, 0x0fb00f50, "fnmscs%c\t%y1, %y2, %y0"},
-  {FPU_VFP_EXT_V1, 0x0e100b00, 0x0fb00f50, "fmscd%c\t%z1, %z2, %z0"},
-  {FPU_VFP_EXT_V1, 0x0e100b40, 0x0fb00f50, "fnmscd%c\t%z1, %z2, %z0"},
-  {FPU_VFP_EXT_V1xD, 0x0e200a00, 0x0fb00f50, "fmuls%c\t%y1, %y2, %y0"},
-  {FPU_VFP_EXT_V1xD, 0x0e200a40, 0x0fb00f50, "fnmuls%c\t%y1, %y2, %y0"},
-  {FPU_VFP_EXT_V1, 0x0e200b00, 0x0fb00f50, "fmuld%c\t%z1, %z2, %z0"},
-  {FPU_VFP_EXT_V1, 0x0e200b40, 0x0fb00f50, "fnmuld%c\t%z1, %z2, %z0"},
-  {FPU_VFP_EXT_V1xD, 0x0e300a00, 0x0fb00f50, "fadds%c\t%y1, %y2, %y0"},
-  {FPU_VFP_EXT_V1xD, 0x0e300a40, 0x0fb00f50, "fsubs%c\t%y1, %y2, %y0"},
-  {FPU_VFP_EXT_V1, 0x0e300b00, 0x0fb00f50, "faddd%c\t%z1, %z2, %z0"},
-  {FPU_VFP_EXT_V1, 0x0e300b40, 0x0fb00f50, "fsubd%c\t%z1, %z2, %z0"},
-  {FPU_VFP_EXT_V1xD, 0x0e800a00, 0x0fb00f50, "fdivs%c\t%y1, %y2, %y0"},
-  {FPU_VFP_EXT_V1, 0x0e800b00, 0x0fb00f50, "fdivd%c\t%z1, %z2, %z0"},
-  {FPU_VFP_EXT_V1xD, 0x0d200a00, 0x0fb00f00, "fstmdbs%c\t%16-19r!, %y3"},
-  {FPU_VFP_EXT_V1xD, 0x0d200b00, 0x0fb00f00, "fstmdb%0?xd%c\t%16-19r!, %z3"},
-  {FPU_VFP_EXT_V1xD, 0x0d300a00, 0x0fb00f00, "fldmdbs%c\t%16-19r!, %y3"},
-  {FPU_VFP_EXT_V1xD, 0x0d300b00, 0x0fb00f00, "fldmdb%0?xd%c\t%16-19r!, %z3"},
-  {FPU_VFP_EXT_V1xD, 0x0d000a00, 0x0f300f00, "fsts%c\t%y1, %A"},
-  {FPU_VFP_EXT_V1, 0x0d000b00, 0x0f300f00, "fstd%c\t%z1, %A"},
-  {FPU_VFP_EXT_V1xD, 0x0d100a00, 0x0f300f00, "flds%c\t%y1, %A"},
-  {FPU_VFP_EXT_V1, 0x0d100b00, 0x0f300f00, "fldd%c\t%z1, %A"},
-  {FPU_VFP_EXT_V1xD, 0x0c800a00, 0x0f900f00, "fstmias%c\t%16-19r%21'!, %y3"},
-  {FPU_VFP_EXT_V1xD, 0x0c800b00, 0x0f900f00, "fstmia%0?xd%c\t%16-19r%21'!, %z3"},
-  {FPU_VFP_EXT_V1xD, 0x0c900a00, 0x0f900f00, "fldmias%c\t%16-19r%21'!, %y3"},
-  {FPU_VFP_EXT_V1xD, 0x0c900b00, 0x0f900f00, "fldmia%0?xd%c\t%16-19r%21'!, %z3"},
-
-  /* Cirrus coprocessor instructions.  */
-  {ARM_CEXT_MAVERICK, 0x0d100400, 0x0f500f00, "cfldrs%c\tmvf%12-15d, %A"},
-  {ARM_CEXT_MAVERICK, 0x0c100400, 0x0f500f00, "cfldrs%c\tmvf%12-15d, %A"},
-  {ARM_CEXT_MAVERICK, 0x0d500400, 0x0f500f00, "cfldrd%c\tmvd%12-15d, %A"},
-  {ARM_CEXT_MAVERICK, 0x0c500400, 0x0f500f00, "cfldrd%c\tmvd%12-15d, %A"},
-  {ARM_CEXT_MAVERICK, 0x0d100500, 0x0f500f00, "cfldr32%c\tmvfx%12-15d, %A"},
-  {ARM_CEXT_MAVERICK, 0x0c100500, 0x0f500f00, "cfldr32%c\tmvfx%12-15d, %A"},
-  {ARM_CEXT_MAVERICK, 0x0d500500, 0x0f500f00, "cfldr64%c\tmvdx%12-15d, %A"},
-  {ARM_CEXT_MAVERICK, 0x0c500500, 0x0f500f00, "cfldr64%c\tmvdx%12-15d, %A"},
-  {ARM_CEXT_MAVERICK, 0x0d000400, 0x0f500f00, "cfstrs%c\tmvf%12-15d, %A"},
-  {ARM_CEXT_MAVERICK, 0x0c000400, 0x0f500f00, "cfstrs%c\tmvf%12-15d, %A"},
-  {ARM_CEXT_MAVERICK, 0x0d400400, 0x0f500f00, "cfstrd%c\tmvd%12-15d, %A"},
-  {ARM_CEXT_MAVERICK, 0x0c400400, 0x0f500f00, "cfstrd%c\tmvd%12-15d, %A"},
-  {ARM_CEXT_MAVERICK, 0x0d000500, 0x0f500f00, "cfstr32%c\tmvfx%12-15d, %A"},
-  {ARM_CEXT_MAVERICK, 0x0c000500, 0x0f500f00, "cfstr32%c\tmvfx%12-15d, %A"},
-  {ARM_CEXT_MAVERICK, 0x0d400500, 0x0f500f00, "cfstr64%c\tmvdx%12-15d, %A"},
-  {ARM_CEXT_MAVERICK, 0x0c400500, 0x0f500f00, "cfstr64%c\tmvdx%12-15d, %A"},
-  {ARM_CEXT_MAVERICK, 0x0e000450, 0x0ff00ff0, "cfmvsr%c\tmvf%16-19d, %12-15r"},
-  {ARM_CEXT_MAVERICK, 0x0e100450, 0x0ff00ff0, "cfmvrs%c\t%12-15r, mvf%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e000410, 0x0ff00ff0, "cfmvdlr%c\tmvd%16-19d, %12-15r"},
-  {ARM_CEXT_MAVERICK, 0x0e100410, 0x0ff00ff0, "cfmvrdl%c\t%12-15r, mvd%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e000430, 0x0ff00ff0, "cfmvdhr%c\tmvd%16-19d, %12-15r"},
-  {ARM_CEXT_MAVERICK, 0x0e100430, 0x0ff00fff, "cfmvrdh%c\t%12-15r, mvd%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e000510, 0x0ff00fff, "cfmv64lr%c\tmvdx%16-19d, %12-15r"},
-  {ARM_CEXT_MAVERICK, 0x0e100510, 0x0ff00fff, "cfmvr64l%c\t%12-15r, mvdx%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e000530, 0x0ff00fff, "cfmv64hr%c\tmvdx%16-19d, %12-15r"},
-  {ARM_CEXT_MAVERICK, 0x0e100530, 0x0ff00fff, "cfmvr64h%c\t%12-15r, mvdx%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e200440, 0x0ff00fff, "cfmval32%c\tmvax%12-15d, mvfx%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e100440, 0x0ff00fff, "cfmv32al%c\tmvfx%12-15d, mvax%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e200460, 0x0ff00fff, "cfmvam32%c\tmvax%12-15d, mvfx%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e100460, 0x0ff00fff, "cfmv32am%c\tmvfx%12-15d, mvax%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e200480, 0x0ff00fff, "cfmvah32%c\tmvax%12-15d, mvfx%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e100480, 0x0ff00fff, "cfmv32ah%c\tmvfx%12-15d, mvax%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e2004a0, 0x0ff00fff, "cfmva32%c\tmvax%12-15d, mvfx%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e1004a0, 0x0ff00fff, "cfmv32a%c\tmvfx%12-15d, mvax%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e2004c0, 0x0ff00fff, "cfmva64%c\tmvax%12-15d, mvdx%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e1004c0, 0x0ff00fff, "cfmv64a%c\tmvdx%12-15d, mvax%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e2004e0, 0x0fff0fff, "cfmvsc32%c\tdspsc, mvdx%12-15d"},
-  {ARM_CEXT_MAVERICK, 0x0e1004e0, 0x0fff0fff, "cfmv32sc%c\tmvdx%12-15d, dspsc"},
-  {ARM_CEXT_MAVERICK, 0x0e000400, 0x0ff00fff, "cfcpys%c\tmvf%12-15d, mvf%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e000420, 0x0ff00fff, "cfcpyd%c\tmvd%12-15d, mvd%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e000460, 0x0ff00fff, "cfcvtsd%c\tmvd%12-15d, mvf%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e000440, 0x0ff00fff, "cfcvtds%c\tmvf%12-15d, mvd%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e000480, 0x0ff00fff, "cfcvt32s%c\tmvf%12-15d, mvfx%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e0004a0, 0x0ff00fff, "cfcvt32d%c\tmvd%12-15d, mvfx%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e0004c0, 0x0ff00fff, "cfcvt64s%c\tmvf%12-15d, mvdx%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e0004e0, 0x0ff00fff, "cfcvt64d%c\tmvd%12-15d, mvdx%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e100580, 0x0ff00fff, "cfcvts32%c\tmvfx%12-15d, mvf%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e1005a0, 0x0ff00fff, "cfcvtd32%c\tmvfx%12-15d, mvd%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e1005c0, 0x0ff00fff, "cftruncs32%c\tmvfx%12-15d, mvf%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e1005e0, 0x0ff00fff, "cftruncd32%c\tmvfx%12-15d, mvd%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e000550, 0x0ff00ff0, "cfrshl32%c\tmvfx%16-19d, mvfx%0-3d, %12-15r"},
-  {ARM_CEXT_MAVERICK, 0x0e000570, 0x0ff00ff0, "cfrshl64%c\tmvdx%16-19d, mvdx%0-3d, %12-15r"},
-  {ARM_CEXT_MAVERICK, 0x0e000500, 0x0ff00f10, "cfsh32%c\tmvfx%12-15d, mvfx%16-19d, #%I"},
-  {ARM_CEXT_MAVERICK, 0x0e200500, 0x0ff00f10, "cfsh64%c\tmvdx%12-15d, mvdx%16-19d, #%I"},
-  {ARM_CEXT_MAVERICK, 0x0e100490, 0x0ff00ff0, "cfcmps%c\t%12-15r, mvf%16-19d, mvf%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e1004b0, 0x0ff00ff0, "cfcmpd%c\t%12-15r, mvd%16-19d, mvd%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e100590, 0x0ff00ff0, "cfcmp32%c\t%12-15r, mvfx%16-19d, mvfx%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e1005b0, 0x0ff00ff0, "cfcmp64%c\t%12-15r, mvdx%16-19d, mvdx%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e300400, 0x0ff00fff, "cfabss%c\tmvf%12-15d, mvf%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e300420, 0x0ff00fff, "cfabsd%c\tmvd%12-15d, mvd%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e300440, 0x0ff00fff, "cfnegs%c\tmvf%12-15d, mvf%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e300460, 0x0ff00fff, "cfnegd%c\tmvd%12-15d, mvd%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e300480, 0x0ff00ff0, "cfadds%c\tmvf%12-15d, mvf%16-19d, mvf%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e3004a0, 0x0ff00ff0, "cfaddd%c\tmvd%12-15d, mvd%16-19d, mvd%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e3004c0, 0x0ff00ff0, "cfsubs%c\tmvf%12-15d, mvf%16-19d, mvf%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e3004e0, 0x0ff00ff0, "cfsubd%c\tmvd%12-15d, mvd%16-19d, mvd%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e100400, 0x0ff00ff0, "cfmuls%c\tmvf%12-15d, mvf%16-19d, mvf%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e100420, 0x0ff00ff0, "cfmuld%c\tmvd%12-15d, mvd%16-19d, mvd%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e300500, 0x0ff00fff, "cfabs32%c\tmvfx%12-15d, mvfx%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e300520, 0x0ff00fff, "cfabs64%c\tmvdx%12-15d, mvdx%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e300540, 0x0ff00fff, "cfneg32%c\tmvfx%12-15d, mvfx%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e300560, 0x0ff00fff, "cfneg64%c\tmvdx%12-15d, mvdx%16-19d"},
-  {ARM_CEXT_MAVERICK, 0x0e300580, 0x0ff00ff0, "cfadd32%c\tmvfx%12-15d, mvfx%16-19d, mvfx%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e3005a0, 0x0ff00ff0, "cfadd64%c\tmvdx%12-15d, mvdx%16-19d, mvdx%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e3005c0, 0x0ff00ff0, "cfsub32%c\tmvfx%12-15d, mvfx%16-19d, mvfx%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e3005e0, 0x0ff00ff0, "cfsub64%c\tmvdx%12-15d, mvdx%16-19d, mvdx%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e100500, 0x0ff00ff0, "cfmul32%c\tmvfx%12-15d, mvfx%16-19d, mvfx%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e100520, 0x0ff00ff0, "cfmul64%c\tmvdx%12-15d, mvdx%16-19d, mvdx%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e100540, 0x0ff00ff0, "cfmac32%c\tmvfx%12-15d, mvfx%16-19d, mvfx%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e100560, 0x0ff00ff0, "cfmsc32%c\tmvfx%12-15d, mvfx%16-19d, mvfx%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e000600, 0x0ff00f10, "cfmadd32%c\tmvax%5-7d, mvfx%12-15d, mvfx%16-19d, mvfx%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e100600, 0x0ff00f10, "cfmsub32%c\tmvax%5-7d, mvfx%12-15d, mvfx%16-19d, mvfx%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e200600, 0x0ff00f10, "cfmadda32%c\tmvax%5-7d, mvax%12-15d, mvfx%16-19d, mvfx%0-3d"},
-  {ARM_CEXT_MAVERICK, 0x0e300600, 0x0ff00f10, "cfmsuba32%c\tmvax%5-7d, mvax%12-15d, mvfx%16-19d, mvfx%0-3d"},
-
-  /* Generic coprocessor instructions */
-  {ARM_EXT_V2, 0x0c400000, 0x0ff00000, "mcrr%c\t%8-11d, %4-7d, %12-15r, %16-19r, cr%0-3d"},
-  {ARM_EXT_V2, 0x0c500000, 0x0ff00000, "mrrc%c\t%8-11d, %4-7d, %12-15r, %16-19r, cr%0-3d"},
-  {ARM_EXT_V2, 0x0e000000, 0x0f000010, "cdp%c\t%8-11d, %20-23d, cr%12-15d, cr%16-19d, cr%0-3d, {%5-7d}"},
-  {ARM_EXT_V2, 0x0e100010, 0x0f100010, "mrc%c\t%8-11d, %21-23d, %12-15r, cr%16-19d, cr%0-3d, {%5-7d}"},
-  {ARM_EXT_V2, 0x0e000010, 0x0f100010, "mcr%c\t%8-11d, %21-23d, %12-15r, cr%16-19d, cr%0-3d, {%5-7d}"},
-  {ARM_EXT_V2, 0x0c000000, 0x0e100000, "stc%22'l%c\t%8-11d, cr%12-15d, %A"},
-  {ARM_EXT_V2, 0x0c100000, 0x0e100000, "ldc%22'l%c\t%8-11d, cr%12-15d, %A"},
-
-  /* V6 coprocessor instructions */
-  {ARM_EXT_V6, 0xfc500000, 0xfff00000, "mrrc2%c\t%8-11d, %4-7d, %12-15r, %16-19r, cr%0-3d"},
-  {ARM_EXT_V6, 0xfc400000, 0xfff00000, "mcrr2%c\t%8-11d, %4-7d, %12-15r, %16-19r, cr%0-3d"},
-
-  /* V5 coprocessor instructions */
-  {ARM_EXT_V5, 0xfc100000, 0xfe100000, "ldc2%22'l%c\t%8-11d, cr%12-15d, %A"},
-  {ARM_EXT_V5, 0xfc000000, 0xfe100000, "stc2%22'l%c\t%8-11d, cr%12-15d, %A"},
-  {ARM_EXT_V5, 0xfe000000, 0xff000010, "cdp2%c\t%8-11d, %20-23d, cr%12-15d, cr%16-19d, cr%0-3d, {%5-7d}"},
-  {ARM_EXT_V5, 0xfe000010, 0xff100010, "mcr2%c\t%8-11d, %21-23d, %12-15r, cr%16-19d, cr%0-3d, {%5-7d}"},
-  {ARM_EXT_V5, 0xfe100010, 0xff100010, "mrc2%c\t%8-11d, %21-23d, %12-15r, cr%16-19d, cr%0-3d, {%5-7d}"},
-
-  {0, 0, 0, 0}
-};
-
-/* Neon opcode table:  This does not encode the top byte -- that is
-   checked by the print_insn_neon routine, as it depends on whether we are
-   doing thumb32 or arm32 disassembly.  */
-
-/* print_insn_neon recognizes the following format control codes:
-
-   %%			%
-
-   %c			print condition code
-   %A			print v{st,ld}[1234] operands
-   %B			print v{st,ld}[1234] any one operands
-   %C			print v{st,ld}[1234] single->all operands
-   %D			print scalar
-   %E			print vmov, vmvn, vorr, vbic encoded constant
-   %F			print vtbl,vtbx register list
-
-   %<bitfield>r		print as an ARM register
-   %<bitfield>d		print the bitfield in decimal
-   %<bitfield>e         print the 2^N - bitfield in decimal
-   %<bitfield>D		print as a NEON D register
-   %<bitfield>Q		print as a NEON Q register
-   %<bitfield>R		print as a NEON D or Q register
-   %<bitfield>Sn	print byte scaled width limited by n
-   %<bitfield>Tn	print short scaled width limited by n
-   %<bitfield>Un	print long scaled width limited by n
-
-   %<bitfield>'c	print specified char iff bitfield is all ones
-   %<bitfield>`c	print specified char iff bitfield is all zeroes
-   %<bitfield>?ab...    select from array of values in big endian order  */
-
-static const struct opcode32 neon_opcodes[] =
-{
-  /* Extract */
-  {FPU_NEON_EXT_V1, 0xf2b00840, 0xffb00850, "vext%c.8\t%12-15,22R, %16-19,7R, %0-3,5R, #%8-11d"},
-  {FPU_NEON_EXT_V1, 0xf2b00000, 0xffb00810, "vext%c.8\t%12-15,22R, %16-19,7R, %0-3,5R, #%8-11d"},
-
-  /* Move data element to all lanes */
-  {FPU_NEON_EXT_V1, 0xf3b40c00, 0xffb70f90, "vdup%c.32\t%12-15,22R, %0-3,5D[%19d]"},
-  {FPU_NEON_EXT_V1, 0xf3b20c00, 0xffb30f90, "vdup%c.16\t%12-15,22R, %0-3,5D[%18-19d]"},
-  {FPU_NEON_EXT_V1, 0xf3b10c00, 0xffb10f90, "vdup%c.8\t%12-15,22R, %0-3,5D[%17-19d]"},
-
-  /* Table lookup */
-  {FPU_NEON_EXT_V1, 0xf3b00800, 0xffb00c50, "vtbl%c.8\t%12-15,22D, %F, %0-3,5D"},
-  {FPU_NEON_EXT_V1, 0xf3b00840, 0xffb00c50, "vtbx%c.8\t%12-15,22D, %F, %0-3,5D"},
-
-  /* Two registers, miscellaneous */
-  {FPU_NEON_EXT_V1, 0xf2880a10, 0xfebf0fd0, "vmovl%c.%24?us8\t%12-15,22Q, %0-3,5D"},
-  {FPU_NEON_EXT_V1, 0xf2900a10, 0xfebf0fd0, "vmovl%c.%24?us16\t%12-15,22Q, %0-3,5D"},
-  {FPU_NEON_EXT_V1, 0xf2a00a10, 0xfebf0fd0, "vmovl%c.%24?us32\t%12-15,22Q, %0-3,5D"},
-  {FPU_NEON_EXT_V1, 0xf3b00500, 0xffbf0f90, "vcnt%c.8\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b00580, 0xffbf0f90, "vmvn%c\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b20000, 0xffbf0f90, "vswp%c\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b20200, 0xffb30fd0, "vmovn%c.i%18-19T2\t%12-15,22D, %0-3,5Q"},
-  {FPU_NEON_EXT_V1, 0xf3b20240, 0xffb30fd0, "vqmovun%c.s%18-19T2\t%12-15,22D, %0-3,5Q"},
-  {FPU_NEON_EXT_V1, 0xf3b20280, 0xffb30fd0, "vqmovn%c.s%18-19T2\t%12-15,22D, %0-3,5Q"},
-  {FPU_NEON_EXT_V1, 0xf3b202c0, 0xffb30fd0, "vqmovn%c.u%18-19T2\t%12-15,22D, %0-3,5Q"},
-  {FPU_NEON_EXT_V1, 0xf3b20300, 0xffb30fd0, "vshll%c.i%18-19S2\t%12-15,22Q, %0-3,5D, #%18-19S2"},
-  {FPU_NEON_EXT_V1, 0xf3bb0400, 0xffbf0e90, "vrecpe%c.%8?fu%18-19S2\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3bb0480, 0xffbf0e90, "vrsqrte%c.%8?fu%18-19S2\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b00000, 0xffb30f90, "vrev64%c.%18-19S2\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b00080, 0xffb30f90, "vrev32%c.%18-19S2\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b00100, 0xffb30f90, "vrev16%c.%18-19S2\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b00400, 0xffb30f90, "vcls%c.s%18-19S2\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b00480, 0xffb30f90, "vclz%c.i%18-19S2\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b00700, 0xffb30f90, "vqabs%c.s%18-19S2\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b00780, 0xffb30f90, "vqneg%c.s%18-19S2\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b20080, 0xffb30f90, "vtrn%c.%18-19S2\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b20100, 0xffb30f90, "vuzp%c.%18-19S2\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b20180, 0xffb30f90, "vzip%c.%18-19S2\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b10000, 0xffb30b90, "vcgt%c.%10?fs%18-19S2\t%12-15,22R, %0-3,5R, #0"},
-  {FPU_NEON_EXT_V1, 0xf3b10080, 0xffb30b90, "vcge%c.%10?fs%18-19S2\t%12-15,22R, %0-3,5R, #0"},
-  {FPU_NEON_EXT_V1, 0xf3b10100, 0xffb30b90, "vceq%c.%10?fi%18-19S2\t%12-15,22R, %0-3,5R, #0"},
-  {FPU_NEON_EXT_V1, 0xf3b10180, 0xffb30b90, "vcle%c.%10?fs%18-19S2\t%12-15,22R, %0-3,5R, #0"},
-  {FPU_NEON_EXT_V1, 0xf3b10200, 0xffb30b90, "vclt%c.%10?fs%18-19S2\t%12-15,22R, %0-3,5R, #0"},
-  {FPU_NEON_EXT_V1, 0xf3b10300, 0xffb30b90, "vabs%c.%10?fs%18-19S2\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b10380, 0xffb30b90, "vneg%c.%10?fs%18-19S2\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b00200, 0xffb30f10, "vpaddl%c.%7?us%18-19S2\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b00600, 0xffb30f10, "vpadal%c.%7?us%18-19S2\t%12-15,22R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3b30600, 0xffb30e10, "vcvt%c.%7-8?usff%18-19Sa.%7-8?ffus%18-19Sa\t%12-15,22R, %0-3,5R"},
-
-  /* Three registers of the same length */
-  {FPU_NEON_EXT_V1, 0xf2000110, 0xffb00f10, "vand%c\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2100110, 0xffb00f10, "vbic%c\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2200110, 0xffb00f10, "vorr%c\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2300110, 0xffb00f10, "vorn%c\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3000110, 0xffb00f10, "veor%c\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3100110, 0xffb00f10, "vbsl%c\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3200110, 0xffb00f10, "vbit%c\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3300110, 0xffb00f10, "vbif%c\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000d00, 0xffa00f10, "vadd%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000d10, 0xffa00f10, "vmla%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000e00, 0xffa00f10, "vceq%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000f00, 0xffa00f10, "vmax%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000f10, 0xffa00f10, "vrecps%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2200d00, 0xffa00f10, "vsub%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2200d10, 0xffa00f10, "vmls%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2200f00, 0xffa00f10, "vmin%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2200f10, 0xffa00f10, "vrsqrts%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3000d00, 0xffa00f10, "vpadd%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3000d10, 0xffa00f10, "vmul%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3000e00, 0xffa00f10, "vcge%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3000e10, 0xffa00f10, "vacge%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3000f00, 0xffa00f10, "vpmax%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3200d00, 0xffa00f10, "vabd%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3200e00, 0xffa00f10, "vcgt%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3200e10, 0xffa00f10, "vacgt%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3200f00, 0xffa00f10, "vpmin%c.f%20U0\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000800, 0xff800f10, "vadd%c.i%20-21S3\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000810, 0xff800f10, "vtst%c.%20-21S2\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000900, 0xff800f10, "vmla%c.i%20-21S2\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000b00, 0xff800f10, "vqdmulh%c.s%20-21S6\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000b10, 0xff800f10, "vpadd%c.i%20-21S2\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3000800, 0xff800f10, "vsub%c.i%20-21S3\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3000810, 0xff800f10, "vceq%c.i%20-21S2\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3000900, 0xff800f10, "vmls%c.i%20-21S2\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf3000b00, 0xff800f10, "vqrdmulh%c.s%20-21S6\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000000, 0xfe800f10, "vhadd%c.%24?us%20-21S2\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000010, 0xfe800f10, "vqadd%c.%24?us%20-21S3\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000100, 0xfe800f10, "vrhadd%c.%24?us%20-21S2\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000200, 0xfe800f10, "vhsub%c.%24?us%20-21S2\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000210, 0xfe800f10, "vqsub%c.%24?us%20-21S3\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000300, 0xfe800f10, "vcgt%c.%24?us%20-21S2\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000310, 0xfe800f10, "vcge%c.%24?us%20-21S2\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000400, 0xfe800f10, "vshl%c.%24?us%20-21S3\t%12-15,22R, %0-3,5R, %16-19,7R"},
-  {FPU_NEON_EXT_V1, 0xf2000410, 0xfe800f10, "vqshl%c.%24?us%20-21S3\t%12-15,22R, %0-3,5R, %16-19,7R"},
-  {FPU_NEON_EXT_V1, 0xf2000500, 0xfe800f10, "vrshl%c.%24?us%20-21S3\t%12-15,22R, %0-3,5R, %16-19,7R"},
-  {FPU_NEON_EXT_V1, 0xf2000510, 0xfe800f10, "vqrshl%c.%24?us%20-21S3\t%12-15,22R, %0-3,5R, %16-19,7R"},
-  {FPU_NEON_EXT_V1, 0xf2000600, 0xfe800f10, "vmax%c.%24?us%20-21S2\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000610, 0xfe800f10, "vmin%c.%24?us%20-21S2\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000700, 0xfe800f10, "vabd%c.%24?us%20-21S2\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000710, 0xfe800f10, "vaba%c.%24?us%20-21S2\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000910, 0xfe800f10, "vmul%c.%24?pi%20-21S2\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000a00, 0xfe800f10, "vpmax%c.%24?us%20-21S2\t%12-15,22R, %16-19,7R, %0-3,5R"},
-  {FPU_NEON_EXT_V1, 0xf2000a10, 0xfe800f10, "vpmin%c.%24?us%20-21S2\t%12-15,22R, %16-19,7R, %0-3,5R"},
-
-  /* One register and an immediate value */
-  {FPU_NEON_EXT_V1, 0xf2800e10, 0xfeb80fb0, "vmov%c.i8\t%12-15,22R, %E"},
-  {FPU_NEON_EXT_V1, 0xf2800e30, 0xfeb80fb0, "vmov%c.i64\t%12-15,22R, %E"},
-  {FPU_NEON_EXT_V1, 0xf2800f10, 0xfeb80fb0, "vmov%c.f32\t%12-15,22R, %E"},
-  {FPU_NEON_EXT_V1, 0xf2800810, 0xfeb80db0, "vmov%c.i16\t%12-15,22R, %E"},
-  {FPU_NEON_EXT_V1, 0xf2800830, 0xfeb80db0, "vmvn%c.i16\t%12-15,22R, %E"},
-  {FPU_NEON_EXT_V1, 0xf2800910, 0xfeb80db0, "vorr%c.i16\t%12-15,22R, %E"},
-  {FPU_NEON_EXT_V1, 0xf2800930, 0xfeb80db0, "vbic%c.i16\t%12-15,22R, %E"},
-  {FPU_NEON_EXT_V1, 0xf2800c10, 0xfeb80eb0, "vmov%c.i32\t%12-15,22R, %E"},
-  {FPU_NEON_EXT_V1, 0xf2800c30, 0xfeb80eb0, "vmvn%c.i32\t%12-15,22R, %E"},
-  {FPU_NEON_EXT_V1, 0xf2800110, 0xfeb809b0, "vorr%c.i32\t%12-15,22R, %E"},
-  {FPU_NEON_EXT_V1, 0xf2800130, 0xfeb809b0, "vbic%c.i32\t%12-15,22R, %E"},
-  {FPU_NEON_EXT_V1, 0xf2800010, 0xfeb808b0, "vmov%c.i32\t%12-15,22R, %E"},
-  {FPU_NEON_EXT_V1, 0xf2800030, 0xfeb808b0, "vmvn%c.i32\t%12-15,22R, %E"},
-
-  /* Two registers and a shift amount */
-  {FPU_NEON_EXT_V1, 0xf2880810, 0xffb80fd0, "vshrn%c.i16\t%12-15,22D, %0-3,5Q, #%16-18e"},
-  {FPU_NEON_EXT_V1, 0xf2880850, 0xffb80fd0, "vrshrn%c.i16\t%12-15,22D, %0-3,5Q, #%16-18e"},
-  {FPU_NEON_EXT_V1, 0xf2880810, 0xfeb80fd0, "vqshrun%c.s16\t%12-15,22D, %0-3,5Q, #%16-18e"},
-  {FPU_NEON_EXT_V1, 0xf2880850, 0xfeb80fd0, "vqrshrun%c.s16\t%12-15,22D, %0-3,5Q, #%16-18e"},
-  {FPU_NEON_EXT_V1, 0xf2880910, 0xfeb80fd0, "vqshrn%c.%24?us16\t%12-15,22D, %0-3,5Q, #%16-18e"},
-  {FPU_NEON_EXT_V1, 0xf2880950, 0xfeb80fd0, "vqrshrn%c.%24?us16\t%12-15,22D, %0-3,5Q, #%16-18e"},
-  {FPU_NEON_EXT_V1, 0xf2880a10, 0xfeb80fd0, "vshll%c.%24?us8\t%12-15,22D, %0-3,5Q, #%16-18d"},
-  {FPU_NEON_EXT_V1, 0xf2900810, 0xffb00fd0, "vshrn%c.i32\t%12-15,22D, %0-3,5Q, #%16-19e"},
-  {FPU_NEON_EXT_V1, 0xf2900850, 0xffb00fd0, "vrshrn%c.i32\t%12-15,22D, %0-3,5Q, #%16-19e"},
-  {FPU_NEON_EXT_V1, 0xf2880510, 0xffb80f90, "vshl%c.%24?us8\t%12-15,22R, %0-3,5R, #%16-18d"},
-  {FPU_NEON_EXT_V1, 0xf3880410, 0xffb80f90, "vsri%c.8\t%12-15,22R, %0-3,5R, #%16-18e"},
-  {FPU_NEON_EXT_V1, 0xf3880510, 0xffb80f90, "vsli%c.8\t%12-15,22R, %0-3,5R, #%16-18d"},
-  {FPU_NEON_EXT_V1, 0xf3880610, 0xffb80f90, "vqshlu%c.s8\t%12-15,22R, %0-3,5R, #%16-18d"},
-  {FPU_NEON_EXT_V1, 0xf2900810, 0xfeb00fd0, "vqshrun%c.s32\t%12-15,22D, %0-3,5Q, #%16-19e"},
-  {FPU_NEON_EXT_V1, 0xf2900850, 0xfeb00fd0, "vqrshrun%c.s32\t%12-15,22D, %0-3,5Q, #%16-19e"},
-  {FPU_NEON_EXT_V1, 0xf2900910, 0xfeb00fd0, "vqshrn%c.%24?us32\t%12-15,22D, %0-3,5Q, #%16-19e"},
-  {FPU_NEON_EXT_V1, 0xf2900950, 0xfeb00fd0, "vqrshrn%c.%24?us32\t%12-15,22D, %0-3,5Q, #%16-19e"},
-  {FPU_NEON_EXT_V1, 0xf2900a10, 0xfeb00fd0, "vshll%c.%24?us16\t%12-15,22D, %0-3,5Q, #%16-19d"},
-  {FPU_NEON_EXT_V1, 0xf2880010, 0xfeb80f90, "vshr%c.%24?us8\t%12-15,22R, %0-3,5R, #%16-18e"},
-  {FPU_NEON_EXT_V1, 0xf2880110, 0xfeb80f90, "vsra%c.%24?us8\t%12-15,22R, %0-3,5R, #%16-18e"},
-  {FPU_NEON_EXT_V1, 0xf2880210, 0xfeb80f90, "vrshr%c.%24?us8\t%12-15,22R, %0-3,5R, #%16-18e"},
-  {FPU_NEON_EXT_V1, 0xf2880310, 0xfeb80f90, "vrsra%c.%24?us8\t%12-15,22R, %0-3,5R, #%16-18e"},
-  {FPU_NEON_EXT_V1, 0xf2880710, 0xfeb80f90, "vqshl%c.%24?us8\t%12-15,22R, %0-3,5R, #%16-18d"},
-  {FPU_NEON_EXT_V1, 0xf2a00810, 0xffa00fd0, "vshrn%c.i64\t%12-15,22D, %0-3,5Q, #%16-20e"},
-  {FPU_NEON_EXT_V1, 0xf2a00850, 0xffa00fd0, "vrshrn%c.i64\t%12-15,22D, %0-3,5Q, #%16-20e"},
-  {FPU_NEON_EXT_V1, 0xf2900510, 0xffb00f90, "vshl%c.%24?us16\t%12-15,22R, %0-3,5R, #%16-19d"},
-  {FPU_NEON_EXT_V1, 0xf3900410, 0xffb00f90, "vsri%c.16\t%12-15,22R, %0-3,5R, #%16-19e"},
-  {FPU_NEON_EXT_V1, 0xf3900510, 0xffb00f90, "vsli%c.16\t%12-15,22R, %0-3,5R, #%16-19d"},
-  {FPU_NEON_EXT_V1, 0xf3900610, 0xffb00f90, "vqshlu%c.s16\t%12-15,22R, %0-3,5R, #%16-19d"},
-  {FPU_NEON_EXT_V1, 0xf2a00a10, 0xfea00fd0, "vshll%c.%24?us32\t%12-15,22D, %0-3,5Q, #%16-20d"},
-  {FPU_NEON_EXT_V1, 0xf2900010, 0xfeb00f90, "vshr%c.%24?us16\t%12-15,22R, %0-3,5R, #%16-19e"},
-  {FPU_NEON_EXT_V1, 0xf2900110, 0xfeb00f90, "vsra%c.%24?us16\t%12-15,22R, %0-3,5R, #%16-19e"},
-  {FPU_NEON_EXT_V1, 0xf2900210, 0xfeb00f90, "vrshr%c.%24?us16\t%12-15,22R, %0-3,5R, #%16-19e"},
-  {FPU_NEON_EXT_V1, 0xf2900310, 0xfeb00f90, "vrsra%c.%24?us16\t%12-15,22R, %0-3,5R, #%16-19e"},
-  {FPU_NEON_EXT_V1, 0xf2900710, 0xfeb00f90, "vqshl%c.%24?us16\t%12-15,22R, %0-3,5R, #%16-19d"},
-  {FPU_NEON_EXT_V1, 0xf2800810, 0xfec00fd0, "vqshrun%c.s64\t%12-15,22D, %0-3,5Q, #%16-20e"},
-  {FPU_NEON_EXT_V1, 0xf2800850, 0xfec00fd0, "vqrshrun%c.s64\t%12-15,22D, %0-3,5Q, #%16-20e"},
-  {FPU_NEON_EXT_V1, 0xf2800910, 0xfec00fd0, "vqshrn%c.%24?us64\t%12-15,22D, %0-3,5Q, #%16-20e"},
-  {FPU_NEON_EXT_V1, 0xf2800950, 0xfec00fd0, "vqrshrn%c.%24?us64\t%12-15,22D, %0-3,5Q, #%16-20e"},
-  {FPU_NEON_EXT_V1, 0xf2a00510, 0xffa00f90, "vshl%c.%24?us32\t%12-15,22R, %0-3,5R, #%16-20d"},
-  {FPU_NEON_EXT_V1, 0xf3a00410, 0xffa00f90, "vsri%c.32\t%12-15,22R, %0-3,5R, #%16-20e"},
-  {FPU_NEON_EXT_V1, 0xf3a00510, 0xffa00f90, "vsli%c.32\t%12-15,22R, %0-3,5R, #%16-20d"},
-  {FPU_NEON_EXT_V1, 0xf3a00610, 0xffa00f90, "vqshlu%c.s32\t%12-15,22R, %0-3,5R, #%16-20d"},
-  {FPU_NEON_EXT_V1, 0xf2a00010, 0xfea00f90, "vshr%c.%24?us32\t%12-15,22R, %0-3,5R, #%16-20e"},
-  {FPU_NEON_EXT_V1, 0xf2a00110, 0xfea00f90, "vsra%c.%24?us32\t%12-15,22R, %0-3,5R, #%16-20e"},
-  {FPU_NEON_EXT_V1, 0xf2a00210, 0xfea00f90, "vrshr%c.%24?us32\t%12-15,22R, %0-3,5R, #%16-20e"},
-  {FPU_NEON_EXT_V1, 0xf2a00310, 0xfea00f90, "vrsra%c.%24?us32\t%12-15,22R, %0-3,5R, #%16-20e"},
-  {FPU_NEON_EXT_V1, 0xf2a00710, 0xfea00f90, "vqshl%c.%24?us32\t%12-15,22R, %0-3,5R, #%16-20d"},
-  {FPU_NEON_EXT_V1, 0xf2800590, 0xff800f90, "vshl%c.%24?us64\t%12-15,22R, %0-3,5R, #%16-21d"},
-  {FPU_NEON_EXT_V1, 0xf3800490, 0xff800f90, "vsri%c.64\t%12-15,22R, %0-3,5R, #%16-21e"},
-  {FPU_NEON_EXT_V1, 0xf3800590, 0xff800f90, "vsli%c.64\t%12-15,22R, %0-3,5R, #%16-21d"},
-  {FPU_NEON_EXT_V1, 0xf3800690, 0xff800f90, "vqshlu%c.s64\t%12-15,22R, %0-3,5R, #%16-21d"},
-  {FPU_NEON_EXT_V1, 0xf2800090, 0xfe800f90, "vshr%c.%24?us64\t%12-15,22R, %0-3,5R, #%16-21e"},
-  {FPU_NEON_EXT_V1, 0xf2800190, 0xfe800f90, "vsra%c.%24?us64\t%12-15,22R, %0-3,5R, #%16-21e"},
-  {FPU_NEON_EXT_V1, 0xf2800290, 0xfe800f90, "vrshr%c.%24?us64\t%12-15,22R, %0-3,5R, #%16-21e"},
-  {FPU_NEON_EXT_V1, 0xf2800390, 0xfe800f90, "vrsra%c.%24?us64\t%12-15,22R, %0-3,5R, #%16-21e"},
-  {FPU_NEON_EXT_V1, 0xf2800790, 0xfe800f90, "vqshl%c.%24?us64\t%12-15,22R, %0-3,5R, #%16-21d"},
-  {FPU_NEON_EXT_V1, 0xf2a00e10, 0xfea00e90, "vcvt%c.%24,8?usff32.%24,8?ffus32\t%12-15,22R, %0-3,5R, #%16-20e"},
-
-  /* Three registers of different lengths */
-  {FPU_NEON_EXT_V1, 0xf2800e00, 0xfea00f50, "vmull%c.p%20S0\t%12-15,22Q, %16-19,7D, %0-3,5D"},
-  {FPU_NEON_EXT_V1, 0xf2800400, 0xff800f50, "vaddhn%c.i%20-21T2\t%12-15,22D, %16-19,7Q, %0-3,5Q"},
-  {FPU_NEON_EXT_V1, 0xf2800600, 0xff800f50, "vsubhn%c.i%20-21T2\t%12-15,22D, %16-19,7Q, %0-3,5Q"},
-  {FPU_NEON_EXT_V1, 0xf2800900, 0xff800f50, "vqdmlal%c.s%20-21S6\t%12-15,22Q, %16-19,7D, %0-3,5D"},
-  {FPU_NEON_EXT_V1, 0xf2800b00, 0xff800f50, "vqdmlsl%c.s%20-21S6\t%12-15,22Q, %16-19,7D, %0-3,5D"},
-  {FPU_NEON_EXT_V1, 0xf2800d00, 0xff800f50, "vqdmull%c.s%20-21S6\t%12-15,22Q, %16-19,7D, %0-3,5D"},
-  {FPU_NEON_EXT_V1, 0xf3800400, 0xff800f50, "vraddhn%c.i%20-21T2\t%12-15,22D, %16-19,7Q, %0-3,5Q"},
-  {FPU_NEON_EXT_V1, 0xf3800600, 0xff800f50, "vrsubhn%c.i%20-21T2\t%12-15,22D, %16-19,7Q, %0-3,5Q"},
-  {FPU_NEON_EXT_V1, 0xf2800000, 0xfe800f50, "vaddl%c.%24?us%20-21S2\t%12-15,22Q, %16-19,7D, %0-3,5D"},
-  {FPU_NEON_EXT_V1, 0xf2800100, 0xfe800f50, "vaddw%c.%24?us%20-21S2\t%12-15,22Q, %16-19,7Q, %0-3,5D"},
-  {FPU_NEON_EXT_V1, 0xf2800200, 0xfe800f50, "vsubl%c.%24?us%20-21S2\t%12-15,22Q, %16-19,7D, %0-3,5D"},
-  {FPU_NEON_EXT_V1, 0xf2800300, 0xfe800f50, "vsubw%c.%24?us%20-21S2\t%12-15,22Q, %16-19,7Q, %0-3,5D"},
-  {FPU_NEON_EXT_V1, 0xf2800500, 0xfe800f50, "vabal%c.%24?us%20-21S2\t%12-15,22Q, %16-19,7D, %0-3,5D"},
-  {FPU_NEON_EXT_V1, 0xf2800700, 0xfe800f50, "vabdl%c.%24?us%20-21S2\t%12-15,22Q, %16-19,7D, %0-3,5D"},
-  {FPU_NEON_EXT_V1, 0xf2800800, 0xfe800f50, "vmlal%c.%24?us%20-21S2\t%12-15,22Q, %16-19,7D, %0-3,5D"},
-  {FPU_NEON_EXT_V1, 0xf2800a00, 0xfe800f50, "vmlsl%c.%24?us%20-21S2\t%12-15,22Q, %16-19,7D, %0-3,5D"},
-  {FPU_NEON_EXT_V1, 0xf2800c00, 0xfe800f50, "vmull%c.%24?us%20-21S2\t%12-15,22Q, %16-19,7D, %0-3,5D"},
-
-  /* Two registers and a scalar */
-  {FPU_NEON_EXT_V1, 0xf2800040, 0xff800f50, "vmla%c.i%20-21S6\t%12-15,22D, %16-19,7D, %D"},
-  {FPU_NEON_EXT_V1, 0xf2800140, 0xff800f50, "vmla%c.f%20-21Sa\t%12-15,22D, %16-19,7D, %D"},
-  {FPU_NEON_EXT_V1, 0xf2800340, 0xff800f50, "vqdmlal%c.s%20-21S6\t%12-15,22Q, %16-19,7D, %D"},
-  {FPU_NEON_EXT_V1, 0xf2800440, 0xff800f50, "vmls%c.i%20-21S6\t%12-15,22D, %16-19,7D, %D"},
-  {FPU_NEON_EXT_V1, 0xf2800540, 0xff800f50, "vmls%c.f%20-21S6\t%12-15,22D, %16-19,7D, %D"},
-  {FPU_NEON_EXT_V1, 0xf2800740, 0xff800f50, "vqdmlsl%c.s%20-21S6\t%12-15,22Q, %16-19,7D, %D"},
-  {FPU_NEON_EXT_V1, 0xf2800840, 0xff800f50, "vmul%c.i%20-21S6\t%12-15,22D, %16-19,7D, %D"},
-  {FPU_NEON_EXT_V1, 0xf2800940, 0xff800f50, "vmul%c.f%20-21Sa\t%12-15,22D, %16-19,7D, %D"},
-  {FPU_NEON_EXT_V1, 0xf2800b40, 0xff800f50, "vqdmull%c.s%20-21S6\t%12-15,22Q, %16-19,7D, %D"},
-  {FPU_NEON_EXT_V1, 0xf2800c40, 0xff800f50, "vqdmulh%c.s%20-21S6\t%12-15,22D, %16-19,7D, %D"},
-  {FPU_NEON_EXT_V1, 0xf2800d40, 0xff800f50, "vqrdmulh%c.s%20-21S6\t%12-15,22D, %16-19,7D, %D"},
-  {FPU_NEON_EXT_V1, 0xf3800040, 0xff800f50, "vmla%c.i%20-21S6\t%12-15,22Q, %16-19,7Q, %D"},
-  {FPU_NEON_EXT_V1, 0xf3800140, 0xff800f50, "vmla%c.f%20-21Sa\t%12-15,22Q, %16-19,7Q, %D"},
-  {FPU_NEON_EXT_V1, 0xf3800440, 0xff800f50, "vmls%c.i%20-21S6\t%12-15,22Q, %16-19,7Q, %D"},
-  {FPU_NEON_EXT_V1, 0xf3800540, 0xff800f50, "vmls%c.f%20-21Sa\t%12-15,22Q, %16-19,7Q, %D"},
-  {FPU_NEON_EXT_V1, 0xf3800840, 0xff800f50, "vmul%c.i%20-21S6\t%12-15,22Q, %16-19,7Q, %D"},
-  {FPU_NEON_EXT_V1, 0xf3800940, 0xff800f50, "vmul%c.f%20-21Sa\t%12-15,22Q, %16-19,7Q, %D"},
-  {FPU_NEON_EXT_V1, 0xf3800c40, 0xff800f50, "vqdmulh%c.s%20-21S6\t%12-15,22Q, %16-19,7Q, %D"},
-  {FPU_NEON_EXT_V1, 0xf3800d40, 0xff800f50, "vqrdmulh%c.s%20-21S6\t%12-15,22Q, %16-19,7Q, %D"},
-  {FPU_NEON_EXT_V1, 0xf2800240, 0xfe800f50, "vmlal%c.%24?us%20-21S6\t%12-15,22Q, %16-19,7D, %D"},
-  {FPU_NEON_EXT_V1, 0xf2800640, 0xfe800f50, "vmlsl%c.%24?us%20-21S6\t%12-15,22Q, %16-19,7D, %D"},
-  {FPU_NEON_EXT_V1, 0xf2800a40, 0xfe800f50, "vmull%c.%24?us%20-21S6\t%12-15,22Q, %16-19,7D, %D"},
-
-  /* Element and structure load/store */
-  {FPU_NEON_EXT_V1, 0xf4a00fc0, 0xffb00fc0, "vld4%c.32\t%C"},
-  {FPU_NEON_EXT_V1, 0xf4a00c00, 0xffb00f00, "vld1%c.%6-7S2\t%C"},
-  {FPU_NEON_EXT_V1, 0xf4a00d00, 0xffb00f00, "vld2%c.%6-7S2\t%C"},
-  {FPU_NEON_EXT_V1, 0xf4a00e00, 0xffb00f00, "vld3%c.%6-7S2\t%C"},
-  {FPU_NEON_EXT_V1, 0xf4a00f00, 0xffb00f00, "vld4%c.%6-7S2\t%C"},
-  {FPU_NEON_EXT_V1, 0xf4000200, 0xff900f00, "v%21?ls%21?dt1%c.%6-7S3\t%A"},
-  {FPU_NEON_EXT_V1, 0xf4000300, 0xff900f00, "v%21?ls%21?dt2%c.%6-7S2\t%A"},
-  {FPU_NEON_EXT_V1, 0xf4000400, 0xff900f00, "v%21?ls%21?dt3%c.%6-7S2\t%A"},
-  {FPU_NEON_EXT_V1, 0xf4000500, 0xff900f00, "v%21?ls%21?dt3%c.%6-7S2\t%A"},
-  {FPU_NEON_EXT_V1, 0xf4000600, 0xff900f00, "v%21?ls%21?dt1%c.%6-7S3\t%A"},
-  {FPU_NEON_EXT_V1, 0xf4000700, 0xff900f00, "v%21?ls%21?dt1%c.%6-7S3\t%A"},
-  {FPU_NEON_EXT_V1, 0xf4000800, 0xff900f00, "v%21?ls%21?dt2%c.%6-7S2\t%A"},
-  {FPU_NEON_EXT_V1, 0xf4000900, 0xff900f00, "v%21?ls%21?dt2%c.%6-7S2\t%A"},
-  {FPU_NEON_EXT_V1, 0xf4000a00, 0xff900f00, "v%21?ls%21?dt1%c.%6-7S3\t%A"},
-  {FPU_NEON_EXT_V1, 0xf4000000, 0xff900e00, "v%21?ls%21?dt4%c.%6-7S2\t%A"},
-  {FPU_NEON_EXT_V1, 0xf4800000, 0xff900300, "v%21?ls%21?dt1%c.%10-11S2\t%B"},
-  {FPU_NEON_EXT_V1, 0xf4800100, 0xff900300, "v%21?ls%21?dt2%c.%10-11S2\t%B"},
-  {FPU_NEON_EXT_V1, 0xf4800200, 0xff900300, "v%21?ls%21?dt3%c.%10-11S2\t%B"},
-  {FPU_NEON_EXT_V1, 0xf4800300, 0xff900300, "v%21?ls%21?dt4%c.%10-11S2\t%B"},
-
-  {0,0 ,0, 0}
-};
-
-/* Opcode tables: ARM, 16-bit Thumb, 32-bit Thumb.  All three are partially
-   ordered: they must be searched linearly from the top to obtain a correct
-   match.  */
-
-/* print_insn_arm recognizes the following format control codes:
-
-   %%			%
-
-   %a			print address for ldr/str instruction
-   %s                   print address for ldr/str halfword/signextend instruction
-   %b			print branch destination
-   %c			print condition code (always bits 28-31)
-   %m			print register mask for ldm/stm instruction
-   %o			print operand2 (immediate or register + shift)
-   %p			print 'p' iff bits 12-15 are 15
-   %t			print 't' iff bit 21 set and bit 24 clear
-   %B			print arm BLX(1) destination
-   %C			print the PSR sub type.
-   %U			print barrier type.
-   %P			print address for pli instruction.
-
-   %<bitfield>r		print as an ARM register
-   %<bitfield>d		print the bitfield in decimal
-   %<bitfield>W         print the bitfield plus one in decimal
-   %<bitfield>x		print the bitfield in hex
-   %<bitfield>X		print the bitfield as 1 hex digit without leading "0x"
-
-   %<bitfield>'c	print specified char iff bitfield is all ones
-   %<bitfield>`c	print specified char iff bitfield is all zeroes
-   %<bitfield>?ab...    select from array of values in big endian order
-
-   %e                   print arm SMI operand (bits 0..7,8..19).
-   %E			print the LSB and WIDTH fields of a BFI or BFC instruction.
-   %V                   print the 16-bit immediate field of a MOVT or MOVW instruction.  */
-
-static const struct opcode32 arm_opcodes[] =
-{
-  /* ARM instructions.  */
-  {ARM_EXT_V1, 0xe1a00000, 0xffffffff, "nop\t\t\t(mov r0,r0)"},
-  {ARM_EXT_V4T | ARM_EXT_V5, 0x012FFF10, 0x0ffffff0, "bx%c\t%0-3r"},
-  {ARM_EXT_V2, 0x00000090, 0x0fe000f0, "mul%20's%c\t%16-19r, %0-3r, %8-11r"},
-  {ARM_EXT_V2, 0x00200090, 0x0fe000f0, "mla%20's%c\t%16-19r, %0-3r, %8-11r, %12-15r"},
-  {ARM_EXT_V2S, 0x01000090, 0x0fb00ff0, "swp%22'b%c\t%12-15r, %0-3r, [%16-19r]"},
-  {ARM_EXT_V3M, 0x00800090, 0x0fa000f0, "%22?sumull%20's%c\t%12-15r, %16-19r, %0-3r, %8-11r"},
-  {ARM_EXT_V3M, 0x00a00090, 0x0fa000f0, "%22?sumlal%20's%c\t%12-15r, %16-19r, %0-3r, %8-11r"},
-
-  /* IDIV instructions.  */
-  {ARM_EXT_DIV, 0x0710f010, 0x0ff0f0f0, "sdiv%c\t%16-19r, %0-3r, %8-11r"},
-  {ARM_EXT_DIV, 0x0730f010, 0x0ff0f0f0, "udiv%c\t%16-19r, %0-3r, %8-11r"},
-
-  /* V7 instructions.  */
-  {ARM_EXT_V7, 0xf450f000, 0xfd70f000, "pli\t%P"},
-  {ARM_EXT_V7, 0x0320f0f0, 0x0ffffff0, "dbg%c\t#%0-3d"},
-  {ARM_EXT_V7, 0xf57ff050, 0xfffffff0, "dmb\t%U"},
-  {ARM_EXT_V7, 0xf57ff040, 0xfffffff0, "dsb\t%U"},
-  {ARM_EXT_V7, 0xf57ff060, 0xfffffff0, "isb\t%U"},
-
-  /* ARM V6T2 instructions.  */
-  {ARM_EXT_V6T2, 0x07c0001f, 0x0fe0007f, "bfc%c\t%12-15r, %E"},
-  {ARM_EXT_V6T2, 0x07c00010, 0x0fe00070, "bfi%c\t%12-15r, %0-3r, %E"},
-  {ARM_EXT_V6T2, 0x00600090, 0x0ff000f0, "mls%c\t%16-19r, %0-3r, %8-11r, %12-15r"},
-  {ARM_EXT_V6T2, 0x006000b0, 0x0f7000f0, "strht%c\t%12-15r, %s"},
-  {ARM_EXT_V6T2, 0x00300090, 0x0f300090, "ldr%6's%5?hbt%c\t%12-15r, %s"},
-  {ARM_EXT_V6T2, 0x03000000, 0x0ff00000, "movw%c\t%12-15r, %V"},
-  {ARM_EXT_V6T2, 0x03400000, 0x0ff00000, "movt%c\t%12-15r, %V"},
-  {ARM_EXT_V6T2, 0x06ff0f30, 0x0fff0ff0, "rbit%c\t%12-15r, %0-3r"},
-  {ARM_EXT_V6T2, 0x07a00050, 0x0fa00070, "%22?usbfx%c\t%12-15r, %0-3r, #%7-11d, #%16-20W"},
-
-  /* ARM V6Z instructions.  */
-  {ARM_EXT_V6Z, 0x01600070, 0x0ff000f0, "smc%c\t%e"},
-
-  /* ARM V6K instructions.  */
-  {ARM_EXT_V6K, 0xf57ff01f, 0xffffffff, "clrex"},
-  {ARM_EXT_V6K, 0x01d00f9f, 0x0ff00fff, "ldrexb%c\t%12-15r, [%16-19r]"},
-  {ARM_EXT_V6K, 0x01b00f9f, 0x0ff00fff, "ldrexd%c\t%12-15r, [%16-19r]"},
-  {ARM_EXT_V6K, 0x01f00f9f, 0x0ff00fff, "ldrexh%c\t%12-15r, [%16-19r]"},
-  {ARM_EXT_V6K, 0x01c00f90, 0x0ff00ff0, "strexb%c\t%12-15r, %0-3r, [%16-19r]"},
-  {ARM_EXT_V6K, 0x01a00f90, 0x0ff00ff0, "strexd%c\t%12-15r, %0-3r, [%16-19r]"},
-  {ARM_EXT_V6K, 0x01e00f90, 0x0ff00ff0, "strexh%c\t%12-15r, %0-3r, [%16-19r]"},
-
-  /* ARM V6K NOP hints.  */
-  {ARM_EXT_V6K, 0x0320f001, 0x0fffffff, "yield%c"},
-  {ARM_EXT_V6K, 0x0320f002, 0x0fffffff, "wfe%c"},
-  {ARM_EXT_V6K, 0x0320f003, 0x0fffffff, "wfi%c"},
-  {ARM_EXT_V6K, 0x0320f004, 0x0fffffff, "sev%c"},
-  {ARM_EXT_V6K, 0x0320f000, 0x0fffff00, "nop%c\t{%0-7d}"},
-
-  /* ARM V6 instructions. */
-  {ARM_EXT_V6, 0xf1080000, 0xfffffe3f, "cpsie\t%8'a%7'i%6'f"},
-  {ARM_EXT_V6, 0xf10a0000, 0xfffffe20, "cpsie\t%8'a%7'i%6'f,#%0-4d"},
-  {ARM_EXT_V6, 0xf10C0000, 0xfffffe3f, "cpsid\t%8'a%7'i%6'f"},
-  {ARM_EXT_V6, 0xf10e0000, 0xfffffe20, "cpsid\t%8'a%7'i%6'f,#%0-4d"},
-  {ARM_EXT_V6, 0xf1000000, 0xfff1fe20, "cps\t#%0-4d"},
-  {ARM_EXT_V6, 0x06800010, 0x0ff00ff0, "pkhbt%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06800010, 0x0ff00070, "pkhbt%c\t%12-15r, %16-19r, %0-3r, lsl #%7-11d"},
-  {ARM_EXT_V6, 0x06800050, 0x0ff00ff0, "pkhtb%c\t%12-15r, %16-19r, %0-3r, asr #32"},
-  {ARM_EXT_V6, 0x06800050, 0x0ff00070, "pkhtb%c\t%12-15r, %16-19r, %0-3r, asr #%7-11d"},
-  {ARM_EXT_V6, 0x01900f9f, 0x0ff00fff, "ldrex%c\tr%12-15d, [%16-19r]"},
-  {ARM_EXT_V6, 0x06200f10, 0x0ff00ff0, "qadd16%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06200f90, 0x0ff00ff0, "qadd8%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06200f30, 0x0ff00ff0, "qaddsubx%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06200f70, 0x0ff00ff0, "qsub16%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06200ff0, 0x0ff00ff0, "qsub8%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06200f50, 0x0ff00ff0, "qsubaddx%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06100f10, 0x0ff00ff0, "sadd16%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06100f90, 0x0ff00ff0, "sadd8%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06100f30, 0x0ff00ff0, "saddaddx%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06300f10, 0x0ff00ff0, "shadd16%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06300f90, 0x0ff00ff0, "shadd8%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06300f30, 0x0ff00ff0, "shaddsubx%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06300f70, 0x0ff00ff0, "shsub16%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06300ff0, 0x0ff00ff0, "shsub8%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06300f50, 0x0ff00ff0, "shsubaddx%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06100f70, 0x0ff00ff0, "ssub16%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06100ff0, 0x0ff00ff0, "ssub8%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06100f50, 0x0ff00ff0, "ssubaddx%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06500f10, 0x0ff00ff0, "uadd16%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06500f90, 0x0ff00ff0, "uadd8%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06500f30, 0x0ff00ff0, "uaddsubx%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06700f10, 0x0ff00ff0, "uhadd16%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06700f90, 0x0ff00ff0, "uhadd8%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06700f30, 0x0ff00ff0, "uhaddsubx%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06700f70, 0x0ff00ff0, "uhsub16%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06700ff0, 0x0ff00ff0, "uhsub8%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06700f50, 0x0ff00ff0, "uhsubaddx%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06600f10, 0x0ff00ff0, "uqadd16%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06600f90, 0x0ff00ff0, "uqadd8%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06600f30, 0x0ff00ff0, "uqaddsubx%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06600f70, 0x0ff00ff0, "uqsub16%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06600ff0, 0x0ff00ff0, "uqsub8%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06600f50, 0x0ff00ff0, "uqsubaddx%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06500f70, 0x0ff00ff0, "usub16%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06500ff0, 0x0ff00ff0, "usub8%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06500f50, 0x0ff00ff0, "usubaddx%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06bf0f30, 0x0fff0ff0, "rev%c\t\%12-15r, %0-3r"},
-  {ARM_EXT_V6, 0x06bf0fb0, 0x0fff0ff0, "rev16%c\t\%12-15r, %0-3r"},
-  {ARM_EXT_V6, 0x06ff0fb0, 0x0fff0ff0, "revsh%c\t\%12-15r, %0-3r"},
-  {ARM_EXT_V6, 0xf8100a00, 0xfe50ffff, "rfe%23?id%24?ba\t\%16-19r%21'!"},
-  {ARM_EXT_V6, 0x06bf0070, 0x0fff0ff0, "sxth%c\t%12-15r, %0-3r"},
-  {ARM_EXT_V6, 0x06bf0470, 0x0fff0ff0, "sxth%c\t%12-15r, %0-3r, ror #8"},
-  {ARM_EXT_V6, 0x06bf0870, 0x0fff0ff0, "sxth%c\t%12-15r, %0-3r, ror #16"},
-  {ARM_EXT_V6, 0x06bf0c70, 0x0fff0ff0, "sxth%c\t%12-15r, %0-3r, ror #24"},
-  {ARM_EXT_V6, 0x068f0070, 0x0fff0ff0, "sxtb16%c\t%12-15r, %0-3r"},
-  {ARM_EXT_V6, 0x068f0470, 0x0fff0ff0, "sxtb16%c\t%12-15r, %0-3r, ror #8"},
-  {ARM_EXT_V6, 0x068f0870, 0x0fff0ff0, "sxtb16%c\t%12-15r, %0-3r, ror #16"},
-  {ARM_EXT_V6, 0x068f0c70, 0x0fff0ff0, "sxtb16%c\t%12-15r, %0-3r, ror #24"},
-  {ARM_EXT_V6, 0x06af0070, 0x0fff0ff0, "sxtb%c\t%12-15r, %0-3r"},
-  {ARM_EXT_V6, 0x06af0470, 0x0fff0ff0, "sxtb%c\t%12-15r, %0-3r, ror #8"},
-  {ARM_EXT_V6, 0x06af0870, 0x0fff0ff0, "sxtb%c\t%12-15r, %0-3r, ror #16"},
-  {ARM_EXT_V6, 0x06af0c70, 0x0fff0ff0, "sxtb%c\t%12-15r, %0-3r, ror #24"},
-  {ARM_EXT_V6, 0x06ff0070, 0x0fff0ff0, "uxth%c\t%12-15r, %0-3r"},
-  {ARM_EXT_V6, 0x06ff0470, 0x0fff0ff0, "uxth%c\t%12-15r, %0-3r, ror #8"},
-  {ARM_EXT_V6, 0x06ff0870, 0x0fff0ff0, "uxth%c\t%12-15r, %0-3r, ror #16"},
-  {ARM_EXT_V6, 0x06ff0c70, 0x0fff0ff0, "uxth%c\t%12-15r, %0-3r, ror #24"},
-  {ARM_EXT_V6, 0x06cf0070, 0x0fff0ff0, "uxtb16%c\t%12-15r, %0-3r"},
-  {ARM_EXT_V6, 0x06cf0470, 0x0fff0ff0, "uxtb16%c\t%12-15r, %0-3r, ror #8"},
-  {ARM_EXT_V6, 0x06cf0870, 0x0fff0ff0, "uxtb16%c\t%12-15r, %0-3r, ror #16"},
-  {ARM_EXT_V6, 0x06cf0c70, 0x0fff0ff0, "uxtb16%c\t%12-15r, %0-3r, ror #24"},
-  {ARM_EXT_V6, 0x06ef0070, 0x0fff0ff0, "uxtb%c\t%12-15r, %0-3r"},
-  {ARM_EXT_V6, 0x06ef0470, 0x0fff0ff0, "uxtb%c\t%12-15r, %0-3r, ror #8"},
-  {ARM_EXT_V6, 0x06ef0870, 0x0fff0ff0, "uxtb%c\t%12-15r, %0-3r, ror #16"},
-  {ARM_EXT_V6, 0x06ef0c70, 0x0fff0ff0, "uxtb%c\t%12-15r, %0-3r, ror #24"},
-  {ARM_EXT_V6, 0x06b00070, 0x0ff00ff0, "sxtah%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06b00470, 0x0ff00ff0, "sxtah%c\t%12-15r, %16-19r, %0-3r, ror #8"},
-  {ARM_EXT_V6, 0x06b00870, 0x0ff00ff0, "sxtah%c\t%12-15r, %16-19r, %0-3r, ror #16"},
-  {ARM_EXT_V6, 0x06b00c70, 0x0ff00ff0, "sxtah%c\t%12-15r, %16-19r, %0-3r, ror #24"},
-  {ARM_EXT_V6, 0x06800070, 0x0ff00ff0, "sxtab16%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06800470, 0x0ff00ff0, "sxtab16%c\t%12-15r, %16-19r, %0-3r, ror #8"},
-  {ARM_EXT_V6, 0x06800870, 0x0ff00ff0, "sxtab16%c\t%12-15r, %16-19r, %0-3r, ror #16"},
-  {ARM_EXT_V6, 0x06800c70, 0x0ff00ff0, "sxtab16%c\t%12-15r, %16-19r, %0-3r, ror #24"},
-  {ARM_EXT_V6, 0x06a00070, 0x0ff00ff0, "sxtab%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06a00470, 0x0ff00ff0, "sxtab%c\t%12-15r, %16-19r, %0-3r, ror #8"},
-  {ARM_EXT_V6, 0x06a00870, 0x0ff00ff0, "sxtab%c\t%12-15r, %16-19r, %0-3r, ror #16"},
-  {ARM_EXT_V6, 0x06a00c70, 0x0ff00ff0, "sxtab%c\t%12-15r, %16-19r, %0-3r, ror #24"},
-  {ARM_EXT_V6, 0x06f00070, 0x0ff00ff0, "uxtah%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06f00470, 0x0ff00ff0, "uxtah%c\t%12-15r, %16-19r, %0-3r, ror #8"},
-  {ARM_EXT_V6, 0x06f00870, 0x0ff00ff0, "uxtah%c\t%12-15r, %16-19r, %0-3r, ror #16"},
-  {ARM_EXT_V6, 0x06f00c70, 0x0ff00ff0, "uxtah%c\t%12-15r, %16-19r, %0-3r, ror #24"},
-  {ARM_EXT_V6, 0x06c00070, 0x0ff00ff0, "uxtab16%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06c00470, 0x0ff00ff0, "uxtab16%c\t%12-15r, %16-19r, %0-3r, ror #8"},
-  {ARM_EXT_V6, 0x06c00870, 0x0ff00ff0, "uxtab16%c\t%12-15r, %16-19r, %0-3r, ror #16"},
-  {ARM_EXT_V6, 0x06c00c70, 0x0ff00ff0, "uxtab16%c\t%12-15r, %16-19r, %0-3r, ROR #24"},
-  {ARM_EXT_V6, 0x06e00070, 0x0ff00ff0, "uxtab%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0x06e00470, 0x0ff00ff0, "uxtab%c\t%12-15r, %16-19r, %0-3r, ror #8"},
-  {ARM_EXT_V6, 0x06e00870, 0x0ff00ff0, "uxtab%c\t%12-15r, %16-19r, %0-3r, ror #16"},
-  {ARM_EXT_V6, 0x06e00c70, 0x0ff00ff0, "uxtab%c\t%12-15r, %16-19r, %0-3r, ror #24"},
-  {ARM_EXT_V6, 0x06800fb0, 0x0ff00ff0, "sel%c\t%12-15r, %16-19r, %0-3r"},
-  {ARM_EXT_V6, 0xf1010000, 0xfffffc00, "setend\t%9?ble"},
-  {ARM_EXT_V6, 0x0700f010, 0x0ff0f0d0, "smuad%5'x%c\t%16-19r, %0-3r, %8-11r"},
-  {ARM_EXT_V6, 0x0700f050, 0x0ff0f0d0, "smusd%5'x%c\t%16-19r, %0-3r, %8-11r"},
-  {ARM_EXT_V6, 0x07000010, 0x0ff000d0, "smlad%5'x%c\t%16-19r, %0-3r, %8-11r, %12-15r"},
-  {ARM_EXT_V6, 0x07400010, 0x0ff000d0, "smlald%5'x%c\t%12-15r, %16-19r, %0-3r, %8-11r"},
-  {ARM_EXT_V6, 0x07000050, 0x0ff000d0, "smlsd%5'x%c\t%16-19r, %0-3r, %8-11r, %12-15r"},
-  {ARM_EXT_V6, 0x07400050, 0x0ff000d0, "smlsld%5'x%c\t%12-15r, %16-19r, %0-3r, %8-11r"},
-  {ARM_EXT_V6, 0x0750f010, 0x0ff0f0d0, "smmul%5'r%c\t%16-19r, %0-3r, %8-11r"},
-  {ARM_EXT_V6, 0x07500010, 0x0ff000d0, "smmla%5'r%c\t%16-19r, %0-3r, %8-11r, %12-15r"},
-  {ARM_EXT_V6, 0x075000d0, 0x0ff000d0, "smmls%5'r%c\t%16-19r, %0-3r, %8-11r, %12-15r"},
-  {ARM_EXT_V6, 0xf84d0500, 0xfe5fffe0, "srs%23?id%24?ba\t%16-19r%21'!, #%0-4d"},
-  {ARM_EXT_V6, 0x06a00010, 0x0fe00ff0, "ssat%c\t%12-15r, #%16-20W, %0-3r"},
-  {ARM_EXT_V6, 0x06a00010, 0x0fe00070, "ssat%c\t%12-15r, #%16-20W, %0-3r, lsl #%7-11d"},
-  {ARM_EXT_V6, 0x06a00050, 0x0fe00070, "ssat%c\t%12-15r, #%16-20W, %0-3r, asr #%7-11d"},
-  {ARM_EXT_V6, 0x06a00f30, 0x0ff00ff0, "ssat16%c\t%12-15r, #%16-19W, %0-3r"},
-  {ARM_EXT_V6, 0x01800f90, 0x0ff00ff0, "strex%c\t%12-15r, %0-3r, [%16-19r]"},
-  {ARM_EXT_V6, 0x00400090, 0x0ff000f0, "umaal%c\t%12-15r, %16-19r, %0-3r, %8-11r"},
-  {ARM_EXT_V6, 0x0780f010, 0x0ff0f0f0, "usad8%c\t%16-19r, %0-3r, %8-11r"},
-  {ARM_EXT_V6, 0x07800010, 0x0ff000f0, "usada8%c\t%16-19r, %0-3r, %8-11r, %12-15r"},
-  {ARM_EXT_V6, 0x06e00010, 0x0fe00ff0, "usat%c\t%12-15r, #%16-20d, %0-3r"},
-  {ARM_EXT_V6, 0x06e00010, 0x0fe00070, "usat%c\t%12-15r, #%16-20d, %0-3r, lsl #%7-11d"},
-  {ARM_EXT_V6, 0x06e00050, 0x0fe00070, "usat%c\t%12-15r, #%16-20d, %0-3r, asr #%7-11d"},
-  {ARM_EXT_V6, 0x06e00f30, 0x0ff00ff0, "usat16%c\t%12-15r, #%16-19d, %0-3r"},
-
-  /* V5J instruction.  */
-  {ARM_EXT_V5J, 0x012fff20, 0x0ffffff0, "bxj%c\t%0-3r"},
-
-  /* V5 Instructions.  */
-  {ARM_EXT_V5, 0xe1200070, 0xfff000f0, "bkpt\t0x%16-19X%12-15X%8-11X%0-3X"},
-  {ARM_EXT_V5, 0xfa000000, 0xfe000000, "blx\t%B"},
-  {ARM_EXT_V5, 0x012fff30, 0x0ffffff0, "blx%c\t%0-3r"},
-  {ARM_EXT_V5, 0x016f0f10, 0x0fff0ff0, "clz%c\t%12-15r, %0-3r"},
-
-  /* V5E "El Segundo" Instructions.  */
-  {ARM_EXT_V5E, 0x000000d0, 0x0e1000f0, "ldrd%c\t%12-15r, %s"},
-  {ARM_EXT_V5E, 0x000000f0, 0x0e1000f0, "strd%c\t%12-15r, %s"},
-  {ARM_EXT_V5E, 0xf450f000, 0xfc70f000, "pld\t%a"},
-  {ARM_EXT_V5ExP, 0x01000080, 0x0ff000f0, "smlabb%c\t%16-19r, %0-3r, %8-11r, %12-15r"},
-  {ARM_EXT_V5ExP, 0x010000a0, 0x0ff000f0, "smlatb%c\t%16-19r, %0-3r, %8-11r, %12-15r"},
-  {ARM_EXT_V5ExP, 0x010000c0, 0x0ff000f0, "smlabt%c\t%16-19r, %0-3r, %8-11r, %12-15r"},
-  {ARM_EXT_V5ExP, 0x010000e0, 0x0ff000f0, "smlatt%c\t%16-19r, %0-3r, %8-11r, %12-15r"},
-
-  {ARM_EXT_V5ExP, 0x01200080, 0x0ff000f0, "smlawb%c\t%16-19r, %0-3r, %8-11r, %12-15r"},
-  {ARM_EXT_V5ExP, 0x012000c0, 0x0ff000f0, "smlawt%c\t%16-19r, %0-3r, %8-11r, %12-15r"},
-
-  {ARM_EXT_V5ExP, 0x01400080, 0x0ff000f0, "smlalbb%c\t%12-15r, %16-19r, %0-3r, %8-11r"},
-  {ARM_EXT_V5ExP, 0x014000a0, 0x0ff000f0, "smlaltb%c\t%12-15r, %16-19r, %0-3r, %8-11r"},
-  {ARM_EXT_V5ExP, 0x014000c0, 0x0ff000f0, "smlalbt%c\t%12-15r, %16-19r, %0-3r, %8-11r"},
-  {ARM_EXT_V5ExP, 0x014000e0, 0x0ff000f0, "smlaltt%c\t%12-15r, %16-19r, %0-3r, %8-11r"},
-
-  {ARM_EXT_V5ExP, 0x01600080, 0x0ff0f0f0, "smulbb%c\t%16-19r, %0-3r, %8-11r"},
-  {ARM_EXT_V5ExP, 0x016000a0, 0x0ff0f0f0, "smultb%c\t%16-19r, %0-3r, %8-11r"},
-  {ARM_EXT_V5ExP, 0x016000c0, 0x0ff0f0f0, "smulbt%c\t%16-19r, %0-3r, %8-11r"},
-  {ARM_EXT_V5ExP, 0x016000e0, 0x0ff0f0f0, "smultt%c\t%16-19r, %0-3r, %8-11r"},
-
-  {ARM_EXT_V5ExP, 0x012000a0, 0x0ff0f0f0, "smulwb%c\t%16-19r, %0-3r, %8-11r"},
-  {ARM_EXT_V5ExP, 0x012000e0, 0x0ff0f0f0, "smulwt%c\t%16-19r, %0-3r, %8-11r"},
-
-  {ARM_EXT_V5ExP, 0x01000050, 0x0ff00ff0,  "qadd%c\t%12-15r, %0-3r, %16-19r"},
-  {ARM_EXT_V5ExP, 0x01400050, 0x0ff00ff0, "qdadd%c\t%12-15r, %0-3r, %16-19r"},
-  {ARM_EXT_V5ExP, 0x01200050, 0x0ff00ff0,  "qsub%c\t%12-15r, %0-3r, %16-19r"},
-  {ARM_EXT_V5ExP, 0x01600050, 0x0ff00ff0, "qdsub%c\t%12-15r, %0-3r, %16-19r"},
-
-  /* ARM Instructions.  */
-  {ARM_EXT_V1, 0x00000090, 0x0e100090, "str%6's%5?hb%c\t%12-15r, %s"},
-  {ARM_EXT_V1, 0x00100090, 0x0e100090, "ldr%6's%5?hb%c\t%12-15r, %s"},
-  {ARM_EXT_V1, 0x00000000, 0x0de00000, "and%20's%c\t%12-15r, %16-19r, %o"},
-  {ARM_EXT_V1, 0x00200000, 0x0de00000, "eor%20's%c\t%12-15r, %16-19r, %o"},
-  {ARM_EXT_V1, 0x00400000, 0x0de00000, "sub%20's%c\t%12-15r, %16-19r, %o"},
-  {ARM_EXT_V1, 0x00600000, 0x0de00000, "rsb%20's%c\t%12-15r, %16-19r, %o"},
-  {ARM_EXT_V1, 0x00800000, 0x0de00000, "add%20's%c\t%12-15r, %16-19r, %o"},
-  {ARM_EXT_V1, 0x00a00000, 0x0de00000, "adc%20's%c\t%12-15r, %16-19r, %o"},
-  {ARM_EXT_V1, 0x00c00000, 0x0de00000, "sbc%20's%c\t%12-15r, %16-19r, %o"},
-  {ARM_EXT_V1, 0x00e00000, 0x0de00000, "rsc%20's%c\t%12-15r, %16-19r, %o"},
-  {ARM_EXT_V3, 0x0120f000, 0x0db0f000, "msr%c\t%22?SCPSR%C, %o"},
-  {ARM_EXT_V3, 0x010f0000, 0x0fbf0fff, "mrs%c\t%12-15r, %22?SCPSR"},
-  {ARM_EXT_V1, 0x01000000, 0x0de00000, "tst%p%c\t%16-19r, %o"},
-  {ARM_EXT_V1, 0x01200000, 0x0de00000, "teq%p%c\t%16-19r, %o"},
-  {ARM_EXT_V1, 0x01400000, 0x0de00000, "cmp%p%c\t%16-19r, %o"},
-  {ARM_EXT_V1, 0x01600000, 0x0de00000, "cmn%p%c\t%16-19r, %o"},
-  {ARM_EXT_V1, 0x01800000, 0x0de00000, "orr%20's%c\t%12-15r, %16-19r, %o"},
-  {ARM_EXT_V1, 0x03a00000, 0x0fef0000, "mov%20's%c\t%12-15r, %o"},
-  {ARM_EXT_V1, 0x01a00000, 0x0def0ff0, "mov%20's%c\t%12-15r, %0-3r"},
-  {ARM_EXT_V1, 0x01a00000, 0x0def0060, "lsl%20's%c\t%12-15r, %q"},
-  {ARM_EXT_V1, 0x01a00020, 0x0def0060, "lsr%20's%c\t%12-15r, %q"},
-  {ARM_EXT_V1, 0x01a00040, 0x0def0060, "asr%20's%c\t%12-15r, %q"},
-  {ARM_EXT_V1, 0x01a00060, 0x0def0ff0, "rrx%20's%c\t%12-15r, %0-3r"},
-  {ARM_EXT_V1, 0x01a00060, 0x0def0060, "ror%20's%c\t%12-15r, %q"},
-  {ARM_EXT_V1, 0x01c00000, 0x0de00000, "bic%20's%c\t%12-15r, %16-19r, %o"},
-  {ARM_EXT_V1, 0x01e00000, 0x0de00000, "mvn%20's%c\t%12-15r, %o"},
-  {ARM_EXT_V1, 0x052d0004, 0x0fff0fff, "push%c\t{%12-15r}\t\t; (str%c %12-15r, %a)"},
-  {ARM_EXT_V1, 0x04000000, 0x0e100000, "str%22'b%t%c\t%12-15r, %a"},
-  {ARM_EXT_V1, 0x06000000, 0x0e100ff0, "str%22'b%t%c\t%12-15r, %a"},
-  {ARM_EXT_V1, 0x04000000, 0x0c100010, "str%22'b%t%c\t%12-15r, %a"},
-  {ARM_EXT_V1, 0x06000010, 0x0e000010, "undefined"},
-  {ARM_EXT_V1, 0x049d0004, 0x0fff0fff, "pop%c\t{%12-15r}\t\t; (ldr%c %12-15r, %a)"},
-  {ARM_EXT_V1, 0x04100000, 0x0c100000, "ldr%22'b%t%c\t%12-15r, %a"},
-  {ARM_EXT_V1, 0x092d0000, 0x0fff0000, "push%c\t%m"},
-  {ARM_EXT_V1, 0x08800000, 0x0ff00000, "stm%c\t%16-19r%21'!, %m%22'^"},
-  {ARM_EXT_V1, 0x08000000, 0x0e100000, "stm%23?id%24?ba%c\t%16-19r%21'!, %m%22'^"},
-  {ARM_EXT_V1, 0x08bd0000, 0x0fff0000, "pop%c\t%m"},
-  {ARM_EXT_V1, 0x08900000, 0x0f900000, "ldm%c\t%16-19r%21'!, %m%22'^"},
-  {ARM_EXT_V1, 0x08100000, 0x0e100000, "ldm%23?id%24?ba%c\t%16-19r%21'!, %m%22'^"},
-  {ARM_EXT_V1, 0x0a000000, 0x0e000000, "b%24'l%c\t%b"},
-  {ARM_EXT_V1, 0x0f000000, 0x0f000000, "svc%c\t%0-23x"},
-
-  /* The rest.  */
-  {ARM_EXT_V1, 0x00000000, 0x00000000, "undefined instruction %0-31x"},
-  {0, 0x00000000, 0x00000000, 0}
-};
-
-/* print_insn_thumb16 recognizes the following format control codes:
-
-   %S                   print Thumb register (bits 3..5 as high number if bit 6 set)
-   %D                   print Thumb register (bits 0..2 as high number if bit 7 set)
-   %<bitfield>I         print bitfield as a signed decimal
-				(top bit of range being the sign bit)
-   %N                   print Thumb register mask (with LR)
-   %O                   print Thumb register mask (with PC)
-   %M                   print Thumb register mask
-   %b			print CZB's 6-bit unsigned branch destination
-   %s			print Thumb right-shift immediate (6..10; 0 == 32).
-   %c			print the condition code
-   %C			print the condition code, or "s" if not conditional
-   %x			print warning if conditional an not at end of IT block"
-   %X			print "\t; unpredictable <IT:code>" if conditional
-   %I			print IT instruction suffix and operands
-   %<bitfield>r		print bitfield as an ARM register
-   %<bitfield>d		print bitfield as a decimal
-   %<bitfield>H         print (bitfield * 2) as a decimal
-   %<bitfield>W         print (bitfield * 4) as a decimal
-   %<bitfield>a         print (bitfield * 4) as a pc-rel offset + decoded symbol
-   %<bitfield>B         print Thumb branch destination (signed displacement)
-   %<bitfield>c         print bitfield as a condition code
-   %<bitnum>'c		print specified char iff bit is one
-   %<bitnum>?ab		print a if bit is one else print b.  */
-
-static const struct opcode16 thumb_opcodes[] =
-{
-  /* Thumb instructions.  */
-
-  /* ARM V6K no-argument instructions.  */
-  {ARM_EXT_V6K, 0xbf00, 0xffff, "nop%c"},
-  {ARM_EXT_V6K, 0xbf10, 0xffff, "yield%c"},
-  {ARM_EXT_V6K, 0xbf20, 0xffff, "wfe%c"},
-  {ARM_EXT_V6K, 0xbf30, 0xffff, "wfi%c"},
-  {ARM_EXT_V6K, 0xbf40, 0xffff, "sev%c"},
-  {ARM_EXT_V6K, 0xbf00, 0xff0f, "nop%c\t{%4-7d}"},
-
-  /* ARM V6T2 instructions.  */
-  {ARM_EXT_V6T2, 0xb900, 0xfd00, "cbnz\t%0-2r, %b%X"},
-  {ARM_EXT_V6T2, 0xb100, 0xfd00, "cbz\t%0-2r, %b%X"},
-  {ARM_EXT_V6T2, 0xbf00, 0xff00, "it%I%X"},
-
-  /* ARM V6.  */
-  {ARM_EXT_V6, 0xb660, 0xfff8, "cpsie\t%2'a%1'i%0'f%X"},
-  {ARM_EXT_V6, 0xb670, 0xfff8, "cpsid\t%2'a%1'i%0'f%X"},
-  {ARM_EXT_V6, 0x4600, 0xffc0, "mov%c\t%0-2r, %3-5r"},
-  {ARM_EXT_V6, 0xba00, 0xffc0, "rev%c\t%0-2r, %3-5r"},
-  {ARM_EXT_V6, 0xba40, 0xffc0, "rev16%c\t%0-2r, %3-5r"},
-  {ARM_EXT_V6, 0xbac0, 0xffc0, "revsh%c\t%0-2r, %3-5r"},
-  {ARM_EXT_V6, 0xb650, 0xfff7, "setend\t%3?ble%X"},
-  {ARM_EXT_V6, 0xb200, 0xffc0, "sxth%c\t%0-2r, %3-5r"},
-  {ARM_EXT_V6, 0xb240, 0xffc0, "sxtb%c\t%0-2r, %3-5r"},
-  {ARM_EXT_V6, 0xb280, 0xffc0, "uxth%c\t%0-2r, %3-5r"},
-  {ARM_EXT_V6, 0xb2c0, 0xffc0, "uxtb%c\t%0-2r, %3-5r"},
-
-  /* ARM V5 ISA extends Thumb.  */
-  {ARM_EXT_V5T, 0xbe00, 0xff00, "bkpt\t%0-7x"}, /* Is always unconditional.  */
-  /* This is BLX(2).  BLX(1) is a 32-bit instruction.  */
-  {ARM_EXT_V5T, 0x4780, 0xff87, "blx%c\t%3-6r%x"},	/* note: 4 bit register number.  */
-  /* ARM V4T ISA (Thumb v1).  */
-  {ARM_EXT_V4T, 0x46C0, 0xFFFF, "nop%c\t\t\t(mov r8, r8)"},
-  /* Format 4.  */
-  {ARM_EXT_V4T, 0x4000, 0xFFC0, "and%C\t%0-2r, %3-5r"},
-  {ARM_EXT_V4T, 0x4040, 0xFFC0, "eor%C\t%0-2r, %3-5r"},
-  {ARM_EXT_V4T, 0x4080, 0xFFC0, "lsl%C\t%0-2r, %3-5r"},
-  {ARM_EXT_V4T, 0x40C0, 0xFFC0, "lsr%C\t%0-2r, %3-5r"},
-  {ARM_EXT_V4T, 0x4100, 0xFFC0, "asr%C\t%0-2r, %3-5r"},
-  {ARM_EXT_V4T, 0x4140, 0xFFC0, "adc%C\t%0-2r, %3-5r"},
-  {ARM_EXT_V4T, 0x4180, 0xFFC0, "sbc%C\t%0-2r, %3-5r"},
-  {ARM_EXT_V4T, 0x41C0, 0xFFC0, "ror%C\t%0-2r, %3-5r"},
-  {ARM_EXT_V4T, 0x4200, 0xFFC0, "tst%c\t%0-2r, %3-5r"},
-  {ARM_EXT_V4T, 0x4240, 0xFFC0, "neg%C\t%0-2r, %3-5r"},
-  {ARM_EXT_V4T, 0x4280, 0xFFC0, "cmp%c\t%0-2r, %3-5r"},
-  {ARM_EXT_V4T, 0x42C0, 0xFFC0, "cmn%c\t%0-2r, %3-5r"},
-  {ARM_EXT_V4T, 0x4300, 0xFFC0, "orr%C\t%0-2r, %3-5r"},
-  {ARM_EXT_V4T, 0x4340, 0xFFC0, "mul%C\t%0-2r, %3-5r"},
-  {ARM_EXT_V4T, 0x4380, 0xFFC0, "bic%C\t%0-2r, %3-5r"},
-  {ARM_EXT_V4T, 0x43C0, 0xFFC0, "mvn%C\t%0-2r, %3-5r"},
-  /* format 13 */
-  {ARM_EXT_V4T, 0xB000, 0xFF80, "add%c\tsp, #%0-6W"},
-  {ARM_EXT_V4T, 0xB080, 0xFF80, "sub%c\tsp, #%0-6W"},
-  /* format 5 */
-  {ARM_EXT_V4T, 0x4700, 0xFF80, "bx%c\t%S%x"},
-  {ARM_EXT_V4T, 0x4400, 0xFF00, "add%c\t%D, %S"},
-  {ARM_EXT_V4T, 0x4500, 0xFF00, "cmp%c\t%D, %S"},
-  {ARM_EXT_V4T, 0x4600, 0xFF00, "mov%c\t%D, %S"},
-  /* format 14 */
-  {ARM_EXT_V4T, 0xB400, 0xFE00, "push%c\t%N"},
-  {ARM_EXT_V4T, 0xBC00, 0xFE00, "pop%c\t%O"},
-  /* format 2 */
-  {ARM_EXT_V4T, 0x1800, 0xFE00, "add%C\t%0-2r, %3-5r, %6-8r"},
-  {ARM_EXT_V4T, 0x1A00, 0xFE00, "sub%C\t%0-2r, %3-5r, %6-8r"},
-  {ARM_EXT_V4T, 0x1C00, 0xFE00, "add%C\t%0-2r, %3-5r, #%6-8d"},
-  {ARM_EXT_V4T, 0x1E00, 0xFE00, "sub%C\t%0-2r, %3-5r, #%6-8d"},
-  /* format 8 */
-  {ARM_EXT_V4T, 0x5200, 0xFE00, "strh%c\t%0-2r, [%3-5r, %6-8r]"},
-  {ARM_EXT_V4T, 0x5A00, 0xFE00, "ldrh%c\t%0-2r, [%3-5r, %6-8r]"},
-  {ARM_EXT_V4T, 0x5600, 0xF600, "ldrs%11?hb%c\t%0-2r, [%3-5r, %6-8r]"},
-  /* format 7 */
-  {ARM_EXT_V4T, 0x5000, 0xFA00, "str%10'b%c\t%0-2r, [%3-5r, %6-8r]"},
-  {ARM_EXT_V4T, 0x5800, 0xFA00, "ldr%10'b%c\t%0-2r, [%3-5r, %6-8r]"},
-  /* format 1 */
-  {ARM_EXT_V4T, 0x0000, 0xF800, "lsl%C\t%0-2r, %3-5r, #%6-10d"},
-  {ARM_EXT_V4T, 0x0800, 0xF800, "lsr%C\t%0-2r, %3-5r, %s"},
-  {ARM_EXT_V4T, 0x1000, 0xF800, "asr%C\t%0-2r, %3-5r, %s"},
-  /* format 3 */
-  {ARM_EXT_V4T, 0x2000, 0xF800, "mov%C\t%8-10r, #%0-7d"},
-  {ARM_EXT_V4T, 0x2800, 0xF800, "cmp%c\t%8-10r, #%0-7d"},
-  {ARM_EXT_V4T, 0x3000, 0xF800, "add%C\t%8-10r, #%0-7d"},
-  {ARM_EXT_V4T, 0x3800, 0xF800, "sub%C\t%8-10r, #%0-7d"},
-  /* format 6 */
-  {ARM_EXT_V4T, 0x4800, 0xF800, "ldr%c\t%8-10r, [pc, #%0-7W]\t(%0-7a)"},  /* TODO: Disassemble PC relative "LDR rD,=<symbolic>" */
-  /* format 9 */
-  {ARM_EXT_V4T, 0x6000, 0xF800, "str%c\t%0-2r, [%3-5r, #%6-10W]"},
-  {ARM_EXT_V4T, 0x6800, 0xF800, "ldr%c\t%0-2r, [%3-5r, #%6-10W]"},
-  {ARM_EXT_V4T, 0x7000, 0xF800, "strb%c\t%0-2r, [%3-5r, #%6-10d]"},
-  {ARM_EXT_V4T, 0x7800, 0xF800, "ldrb%c\t%0-2r, [%3-5r, #%6-10d]"},
-  /* format 10 */
-  {ARM_EXT_V4T, 0x8000, 0xF800, "strh%c\t%0-2r, [%3-5r, #%6-10H]"},
-  {ARM_EXT_V4T, 0x8800, 0xF800, "ldrh%c\t%0-2r, [%3-5r, #%6-10H]"},
-  /* format 11 */
-  {ARM_EXT_V4T, 0x9000, 0xF800, "str%c\t%8-10r, [sp, #%0-7W]"},
-  {ARM_EXT_V4T, 0x9800, 0xF800, "ldr%c\t%8-10r, [sp, #%0-7W]"},
-  /* format 12 */
-  {ARM_EXT_V4T, 0xA000, 0xF800, "add%c\t%8-10r, pc, #%0-7W\t(adr %8-10r, %0-7a)"},
-  {ARM_EXT_V4T, 0xA800, 0xF800, "add%c\t%8-10r, sp, #%0-7W"},
-  /* format 15 */
-  {ARM_EXT_V4T, 0xC000, 0xF800, "stmia%c\t%8-10r!, %M"},
-  {ARM_EXT_V4T, 0xC800, 0xF800, "ldmia%c\t%8-10r!, %M"},
-  /* format 17 */
-  {ARM_EXT_V4T, 0xDF00, 0xFF00, "svc%c\t%0-7d"},
-  /* format 16 */
-  {ARM_EXT_V4T, 0xDE00, 0xFE00, "undefined"},
-  {ARM_EXT_V4T, 0xD000, 0xF000, "b%8-11c.n\t%0-7B%X"},
-  /* format 18 */
-  {ARM_EXT_V4T, 0xE000, 0xF800, "b%c.n\t%0-10B%x"},
-
-  /* The E800 .. FFFF range is unconditionally redirected to the
-     32-bit table, because even in pre-V6T2 ISAs, BL and BLX(1) pairs
-     are processed via that table.  Thus, we can never encounter a
-     bare "second half of BL/BLX(1)" instruction here.  */
-  {ARM_EXT_V1,  0x0000, 0x0000, "undefined"},
-  {0, 0, 0, 0}
-};
-
-/* Thumb32 opcodes use the same table structure as the ARM opcodes.
-   We adopt the convention that hw1 is the high 16 bits of .value and
-   .mask, hw2 the low 16 bits.
-
-   print_insn_thumb32 recognizes the following format control codes:
-
-       %%		%
-
-       %I		print a 12-bit immediate from hw1[10],hw2[14:12,7:0]
-       %M		print a modified 12-bit immediate (same location)
-       %J		print a 16-bit immediate from hw1[3:0,10],hw2[14:12,7:0]
-       %K		print a 16-bit immediate from hw2[3:0],hw1[3:0],hw2[11:4]
-       %S		print a possibly-shifted Rm
-
-       %a		print the address of a plain load/store
-       %w		print the width and signedness of a core load/store
-       %m		print register mask for ldm/stm
-
-       %E		print the lsb and width fields of a bfc/bfi instruction
-       %F		print the lsb and width fields of a sbfx/ubfx instruction
-       %b		print a conditional branch offset
-       %B		print an unconditional branch offset
-       %s		print the shift field of an SSAT instruction
-       %R		print the rotation field of an SXT instruction
-       %U		print barrier type.
-       %P		print address for pli instruction.
-       %c		print the condition code
-       %x		print warning if conditional an not at end of IT block"
-       %X		print "\t; unpredictable <IT:code>" if conditional
-
-       %<bitfield>d	print bitfield in decimal
-       %<bitfield>W	print bitfield*4 in decimal
-       %<bitfield>r	print bitfield as an ARM register
-       %<bitfield>c	print bitfield as a condition code
-
-       %<bitfield>'c	print specified char iff bitfield is all ones
-       %<bitfield>`c	print specified char iff bitfield is all zeroes
-       %<bitfield>?ab... select from array of values in big endian order
-
-   With one exception at the bottom (done because BL and BLX(1) need
-   to come dead last), this table was machine-sorted first in
-   decreasing order of number of bits set in the mask, then in
-   increasing numeric order of mask, then in increasing numeric order
-   of opcode.  This order is not the clearest for a human reader, but
-   is guaranteed never to catch a special-case bit pattern with a more
-   general mask, which is important, because this instruction encoding
-   makes heavy use of special-case bit patterns.  */
-static const struct opcode32 thumb32_opcodes[] =
-{
-  /* V7 instructions.  */
-  {ARM_EXT_V7, 0xf910f000, 0xff70f000, "pli%c\t%a"},
-  {ARM_EXT_V7, 0xf3af80f0, 0xfffffff0, "dbg%c\t#%0-3d"},
-  {ARM_EXT_V7, 0xf3bf8f50, 0xfffffff0, "dmb%c\t%U"},
-  {ARM_EXT_V7, 0xf3bf8f40, 0xfffffff0, "dsb%c\t%U"},
-  {ARM_EXT_V7, 0xf3bf8f60, 0xfffffff0, "isb%c\t%U"},
-  {ARM_EXT_DIV, 0xfb90f0f0, 0xfff0f0f0, "sdiv%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_DIV, 0xfbb0f0f0, 0xfff0f0f0, "udiv%c\t%8-11r, %16-19r, %0-3r"},
-
-  /* Instructions defined in the basic V6T2 set.  */
-  {ARM_EXT_V6T2, 0xf3af8000, 0xffffffff, "nop%c.w"},
-  {ARM_EXT_V6T2, 0xf3af8001, 0xffffffff, "yield%c.w"},
-  {ARM_EXT_V6T2, 0xf3af8002, 0xffffffff, "wfe%c.w"},
-  {ARM_EXT_V6T2, 0xf3af8003, 0xffffffff, "wfi%c.w"},
-  {ARM_EXT_V6T2, 0xf3af9004, 0xffffffff, "sev%c.w"},
-  {ARM_EXT_V6T2, 0xf3af8000, 0xffffff00, "nop%c.w\t{%0-7d}"},
-
-  {ARM_EXT_V6T2, 0xf3bf8f2f, 0xffffffff, "clrex%c"},
-  {ARM_EXT_V6T2, 0xf3af8400, 0xffffff1f, "cpsie.w\t%7'a%6'i%5'f%X"},
-  {ARM_EXT_V6T2, 0xf3af8600, 0xffffff1f, "cpsid.w\t%7'a%6'i%5'f%X"},
-  {ARM_EXT_V6T2, 0xf3c08f00, 0xfff0ffff, "bxj%c\t%16-19r%x"},
-  {ARM_EXT_V6T2, 0xe810c000, 0xffd0ffff, "rfedb%c\t%16-19r%21'!"},
-  {ARM_EXT_V6T2, 0xe990c000, 0xffd0ffff, "rfeia%c\t%16-19r%21'!"},
-  {ARM_EXT_V6T2, 0xf3ef8000, 0xffeff000, "mrs%c\t%8-11r, %D"},
-  {ARM_EXT_V6T2, 0xf3af8100, 0xffffffe0, "cps\t#%0-4d%X"},
-  {ARM_EXT_V6T2, 0xe8d0f000, 0xfff0fff0, "tbb%c\t[%16-19r, %0-3r]%x"},
-  {ARM_EXT_V6T2, 0xe8d0f010, 0xfff0fff0, "tbh%c\t[%16-19r, %0-3r, lsl #1]%x"},
-  {ARM_EXT_V6T2, 0xf3af8500, 0xffffff00, "cpsie\t%7'a%6'i%5'f, #%0-4d%X"},
-  {ARM_EXT_V6T2, 0xf3af8700, 0xffffff00, "cpsid\t%7'a%6'i%5'f, #%0-4d%X"},
-  {ARM_EXT_V6T2, 0xf3de8f00, 0xffffff00, "subs%c\tpc, lr, #%0-7d"},
-  {ARM_EXT_V6T2, 0xf3808000, 0xffe0f000, "msr%c\t%C, %16-19r"},
-  {ARM_EXT_V6T2, 0xe8500f00, 0xfff00fff, "ldrex%c\t%12-15r, [%16-19r]"},
-  {ARM_EXT_V6T2, 0xe8d00f4f, 0xfff00fef, "ldrex%4?hb%c\t%12-15r, [%16-19r]"},
-  {ARM_EXT_V6T2, 0xe800c000, 0xffd0ffe0, "srsdb%c\t%16-19r%21'!, #%0-4d"},
-  {ARM_EXT_V6T2, 0xe980c000, 0xffd0ffe0, "srsia%c\t%16-19r%21'!, #%0-4d"},
-  {ARM_EXT_V6T2, 0xfa0ff080, 0xfffff0c0, "sxth%c.w\t%8-11r, %0-3r%R"},
-  {ARM_EXT_V6T2, 0xfa1ff080, 0xfffff0c0, "uxth%c.w\t%8-11r, %0-3r%R"},
-  {ARM_EXT_V6T2, 0xfa2ff080, 0xfffff0c0, "sxtb16%c\t%8-11r, %0-3r%R"},
-  {ARM_EXT_V6T2, 0xfa3ff080, 0xfffff0c0, "uxtb16%c\t%8-11r, %0-3r%R"},
-  {ARM_EXT_V6T2, 0xfa4ff080, 0xfffff0c0, "sxtb%c.w\t%8-11r, %0-3r%R"},
-  {ARM_EXT_V6T2, 0xfa5ff080, 0xfffff0c0, "uxtb%c.w\t%8-11r, %0-3r%R"},
-  {ARM_EXT_V6T2, 0xe8400000, 0xfff000ff, "strex%c\t%8-11r, %12-15r, [%16-19r]"},
-  {ARM_EXT_V6T2, 0xe8d0007f, 0xfff000ff, "ldrexd%c\t%12-15r, %8-11r, [%16-19r]"},
-  {ARM_EXT_V6T2, 0xfa80f000, 0xfff0f0f0, "sadd8%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfa80f010, 0xfff0f0f0, "qadd8%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfa80f020, 0xfff0f0f0, "shadd8%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfa80f040, 0xfff0f0f0, "uadd8%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfa80f050, 0xfff0f0f0, "uqadd8%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfa80f060, 0xfff0f0f0, "uhadd8%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfa80f080, 0xfff0f0f0, "qadd%c\t%8-11r, %0-3r, %16-19r"},
-  {ARM_EXT_V6T2, 0xfa80f090, 0xfff0f0f0, "qdadd%c\t%8-11r, %0-3r, %16-19r"},
-  {ARM_EXT_V6T2, 0xfa80f0a0, 0xfff0f0f0, "qsub%c\t%8-11r, %0-3r, %16-19r"},
-  {ARM_EXT_V6T2, 0xfa80f0b0, 0xfff0f0f0, "qdsub%c\t%8-11r, %0-3r, %16-19r"},
-  {ARM_EXT_V6T2, 0xfa90f000, 0xfff0f0f0, "sadd16%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfa90f010, 0xfff0f0f0, "qadd16%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfa90f020, 0xfff0f0f0, "shadd16%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfa90f040, 0xfff0f0f0, "uadd16%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfa90f050, 0xfff0f0f0, "uqadd16%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfa90f060, 0xfff0f0f0, "uhadd16%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfa90f080, 0xfff0f0f0, "rev%c.w\t%8-11r, %16-19r"},
-  {ARM_EXT_V6T2, 0xfa90f090, 0xfff0f0f0, "rev16%c.w\t%8-11r, %16-19r"},
-  {ARM_EXT_V6T2, 0xfa90f0a0, 0xfff0f0f0, "rbit%c\t%8-11r, %16-19r"},
-  {ARM_EXT_V6T2, 0xfa90f0b0, 0xfff0f0f0, "revsh%c.w\t%8-11r, %16-19r"},
-  {ARM_EXT_V6T2, 0xfaa0f000, 0xfff0f0f0, "saddsubx%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfaa0f010, 0xfff0f0f0, "qaddsubx%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfaa0f020, 0xfff0f0f0, "shaddsubx%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfaa0f040, 0xfff0f0f0, "uaddsubx%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfaa0f050, 0xfff0f0f0, "uqaddsubx%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfaa0f060, 0xfff0f0f0, "uhaddsubx%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfaa0f080, 0xfff0f0f0, "sel%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfab0f080, 0xfff0f0f0, "clz%c\t%8-11r, %16-19r"},
-  {ARM_EXT_V6T2, 0xfac0f000, 0xfff0f0f0, "ssub8%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfac0f010, 0xfff0f0f0, "qsub8%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfac0f020, 0xfff0f0f0, "shsub8%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfac0f040, 0xfff0f0f0, "usub8%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfac0f050, 0xfff0f0f0, "uqsub8%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfac0f060, 0xfff0f0f0, "uhsub8%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfad0f000, 0xfff0f0f0, "ssub16%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfad0f010, 0xfff0f0f0, "qsub16%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfad0f020, 0xfff0f0f0, "shsub16%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfad0f040, 0xfff0f0f0, "usub16%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfad0f050, 0xfff0f0f0, "uqsub16%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfad0f060, 0xfff0f0f0, "uhsub16%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfae0f000, 0xfff0f0f0, "ssubaddx%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfae0f010, 0xfff0f0f0, "qsubaddx%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfae0f020, 0xfff0f0f0, "shsubaddx%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfae0f040, 0xfff0f0f0, "usubaddx%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfae0f050, 0xfff0f0f0, "uqsubaddx%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfae0f060, 0xfff0f0f0, "uhsubaddx%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfb00f000, 0xfff0f0f0, "mul%c.w\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfb70f000, 0xfff0f0f0, "usad8%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfa00f000, 0xffe0f0f0, "lsl%20's%c.w\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfa20f000, 0xffe0f0f0, "lsr%20's%c.w\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfa40f000, 0xffe0f0f0, "asr%20's%c.w\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfa60f000, 0xffe0f0f0, "ror%20's%c.w\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xe8c00f40, 0xfff00fe0, "strex%4?hb%c\t%0-3r, %12-15r, [%16-19r]"},
-  {ARM_EXT_V6T2, 0xf3200000, 0xfff0f0e0, "ssat16%c\t%8-11r, #%0-4d, %16-19r"},
-  {ARM_EXT_V6T2, 0xf3a00000, 0xfff0f0e0, "usat16%c\t%8-11r, #%0-4d, %16-19r"},
-  {ARM_EXT_V6T2, 0xfb20f000, 0xfff0f0e0, "smuad%4'x%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfb30f000, 0xfff0f0e0, "smulw%4?tb%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfb40f000, 0xfff0f0e0, "smusd%4'x%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfb50f000, 0xfff0f0e0, "smmul%4'r%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfa00f080, 0xfff0f0c0, "sxtah%c\t%8-11r, %16-19r, %0-3r%R"},
-  {ARM_EXT_V6T2, 0xfa10f080, 0xfff0f0c0, "uxtah%c\t%8-11r, %16-19r, %0-3r%R"},
-  {ARM_EXT_V6T2, 0xfa20f080, 0xfff0f0c0, "sxtab16%c\t%8-11r, %16-19r, %0-3r%R"},
-  {ARM_EXT_V6T2, 0xfa30f080, 0xfff0f0c0, "uxtab16%c\t%8-11r, %16-19r, %0-3r%R"},
-  {ARM_EXT_V6T2, 0xfa40f080, 0xfff0f0c0, "sxtab%c\t%8-11r, %16-19r, %0-3r%R"},
-  {ARM_EXT_V6T2, 0xfa50f080, 0xfff0f0c0, "uxtab%c\t%8-11r, %16-19r, %0-3r%R"},
-  {ARM_EXT_V6T2, 0xfb10f000, 0xfff0f0c0, "smul%5?tb%4?tb%c\t%8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xf36f0000, 0xffff8020, "bfc%c\t%8-11r, %E"},
-  {ARM_EXT_V6T2, 0xea100f00, 0xfff08f00, "tst%c.w\t%16-19r, %S"},
-  {ARM_EXT_V6T2, 0xea900f00, 0xfff08f00, "teq%c\t%16-19r, %S"},
-  {ARM_EXT_V6T2, 0xeb100f00, 0xfff08f00, "cmn%c.w\t%16-19r, %S"},
-  {ARM_EXT_V6T2, 0xebb00f00, 0xfff08f00, "cmp%c.w\t%16-19r, %S"},
-  {ARM_EXT_V6T2, 0xf0100f00, 0xfbf08f00, "tst%c.w\t%16-19r, %M"},
-  {ARM_EXT_V6T2, 0xf0900f00, 0xfbf08f00, "teq%c\t%16-19r, %M"},
-  {ARM_EXT_V6T2, 0xf1100f00, 0xfbf08f00, "cmn%c.w\t%16-19r, %M"},
-  {ARM_EXT_V6T2, 0xf1b00f00, 0xfbf08f00, "cmp%c.w\t%16-19r, %M"},
-  {ARM_EXT_V6T2, 0xea4f0000, 0xffef8000, "mov%20's%c.w\t%8-11r, %S"},
-  {ARM_EXT_V6T2, 0xea6f0000, 0xffef8000, "mvn%20's%c.w\t%8-11r, %S"},
-  {ARM_EXT_V6T2, 0xe8c00070, 0xfff000f0, "strexd%c\t%0-3r, %12-15r, %8-11r, [%16-19r]"},
-  {ARM_EXT_V6T2, 0xfb000000, 0xfff000f0, "mla%c\t%8-11r, %16-19r, %0-3r, %12-15r"},
-  {ARM_EXT_V6T2, 0xfb000010, 0xfff000f0, "mls%c\t%8-11r, %16-19r, %0-3r, %12-15r"},
-  {ARM_EXT_V6T2, 0xfb700000, 0xfff000f0, "usada8%c\t%8-11r, %16-19r, %0-3r, %12-15r"},
-  {ARM_EXT_V6T2, 0xfb800000, 0xfff000f0, "smull%c\t%12-15r, %8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfba00000, 0xfff000f0, "umull%c\t%12-15r, %8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfbc00000, 0xfff000f0, "smlal%c\t%12-15r, %8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfbe00000, 0xfff000f0, "umlal%c\t%12-15r, %8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfbe00060, 0xfff000f0, "umaal%c\t%12-15r, %8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xe8500f00, 0xfff00f00, "ldrex%c\t%12-15r, [%16-19r, #%0-7W]"},
-  {ARM_EXT_V6T2, 0xf7f08000, 0xfff0f000, "smc%c\t%K"},
-  {ARM_EXT_V6T2, 0xf04f0000, 0xfbef8000, "mov%20's%c.w\t%8-11r, %M"},
-  {ARM_EXT_V6T2, 0xf06f0000, 0xfbef8000, "mvn%20's%c.w\t%8-11r, %M"},
-  {ARM_EXT_V6T2, 0xf810f000, 0xff70f000, "pld%c\t%a"},
-  {ARM_EXT_V6T2, 0xfb200000, 0xfff000e0, "smlad%4'x%c\t%8-11r, %16-19r, %0-3r, %12-15r"},
-  {ARM_EXT_V6T2, 0xfb300000, 0xfff000e0, "smlaw%4?tb%c\t%8-11r, %16-19r, %0-3r, %12-15r"},
-  {ARM_EXT_V6T2, 0xfb400000, 0xfff000e0, "smlsd%4'x%c\t%8-11r, %16-19r, %0-3r, %12-15r"},
-  {ARM_EXT_V6T2, 0xfb500000, 0xfff000e0, "smmla%4'r%c\t%8-11r, %16-19r, %0-3r, %12-15r"},
-  {ARM_EXT_V6T2, 0xfb600000, 0xfff000e0, "smmls%4'r%c\t%8-11r, %16-19r, %0-3r, %12-15r"},
-  {ARM_EXT_V6T2, 0xfbc000c0, 0xfff000e0, "smlald%4'x%c\t%12-15r, %8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xfbd000c0, 0xfff000e0, "smlsld%4'x%c\t%12-15r, %8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xeac00000, 0xfff08030, "pkhbt%c\t%8-11r, %16-19r, %S"},
-  {ARM_EXT_V6T2, 0xeac00020, 0xfff08030, "pkhtb%c\t%8-11r, %16-19r, %S"},
-  {ARM_EXT_V6T2, 0xf3400000, 0xfff08020, "sbfx%c\t%8-11r, %16-19r, %F"},
-  {ARM_EXT_V6T2, 0xf3c00000, 0xfff08020, "ubfx%c\t%8-11r, %16-19r, %F"},
-  {ARM_EXT_V6T2, 0xf8000e00, 0xff900f00, "str%wt%c\t%12-15r, %a"},
-  {ARM_EXT_V6T2, 0xfb100000, 0xfff000c0, "smla%5?tb%4?tb%c\t%8-11r, %16-19r, %0-3r, %12-15r"},
-  {ARM_EXT_V6T2, 0xfbc00080, 0xfff000c0, "smlal%5?tb%4?tb%c\t%12-15r, %8-11r, %16-19r, %0-3r"},
-  {ARM_EXT_V6T2, 0xf3600000, 0xfff08020, "bfi%c\t%8-11r, %16-19r, %E"},
-  {ARM_EXT_V6T2, 0xf8100e00, 0xfe900f00, "ldr%wt%c\t%12-15r, %a"},
-  {ARM_EXT_V6T2, 0xf3000000, 0xffd08020, "ssat%c\t%8-11r, #%0-4d, %16-19r%s"},
-  {ARM_EXT_V6T2, 0xf3800000, 0xffd08020, "usat%c\t%8-11r, #%0-4d, %16-19r%s"},
-  {ARM_EXT_V6T2, 0xf2000000, 0xfbf08000, "addw%c\t%8-11r, %16-19r, %I"},
-  {ARM_EXT_V6T2, 0xf2400000, 0xfbf08000, "movw%c\t%8-11r, %J"},
-  {ARM_EXT_V6T2, 0xf2a00000, 0xfbf08000, "subw%c\t%8-11r, %16-19r, %I"},
-  {ARM_EXT_V6T2, 0xf2c00000, 0xfbf08000, "movt%c\t%8-11r, %J"},
-  {ARM_EXT_V6T2, 0xea000000, 0xffe08000, "and%20's%c.w\t%8-11r, %16-19r, %S"},
-  {ARM_EXT_V6T2, 0xea200000, 0xffe08000, "bic%20's%c.w\t%8-11r, %16-19r, %S"},
-  {ARM_EXT_V6T2, 0xea400000, 0xffe08000, "orr%20's%c.w\t%8-11r, %16-19r, %S"},
-  {ARM_EXT_V6T2, 0xea600000, 0xffe08000, "orn%20's%c\t%8-11r, %16-19r, %S"},
-  {ARM_EXT_V6T2, 0xea800000, 0xffe08000, "eor%20's%c.w\t%8-11r, %16-19r, %S"},
-  {ARM_EXT_V6T2, 0xeb000000, 0xffe08000, "add%20's%c.w\t%8-11r, %16-19r, %S"},
-  {ARM_EXT_V6T2, 0xeb400000, 0xffe08000, "adc%20's%c.w\t%8-11r, %16-19r, %S"},
-  {ARM_EXT_V6T2, 0xeb600000, 0xffe08000, "sbc%20's%c.w\t%8-11r, %16-19r, %S"},
-  {ARM_EXT_V6T2, 0xeba00000, 0xffe08000, "sub%20's%c.w\t%8-11r, %16-19r, %S"},
-  {ARM_EXT_V6T2, 0xebc00000, 0xffe08000, "rsb%20's%c\t%8-11r, %16-19r, %S"},
-  {ARM_EXT_V6T2, 0xe8400000, 0xfff00000, "strex%c\t%8-11r, %12-15r, [%16-19r, #%0-7W]"},
-  {ARM_EXT_V6T2, 0xf0000000, 0xfbe08000, "and%20's%c.w\t%8-11r, %16-19r, %M"},
-  {ARM_EXT_V6T2, 0xf0200000, 0xfbe08000, "bic%20's%c.w\t%8-11r, %16-19r, %M"},
-  {ARM_EXT_V6T2, 0xf0400000, 0xfbe08000, "orr%20's%c.w\t%8-11r, %16-19r, %M"},
-  {ARM_EXT_V6T2, 0xf0600000, 0xfbe08000, "orn%20's%c\t%8-11r, %16-19r, %M"},
-  {ARM_EXT_V6T2, 0xf0800000, 0xfbe08000, "eor%20's%c.w\t%8-11r, %16-19r, %M"},
-  {ARM_EXT_V6T2, 0xf1000000, 0xfbe08000, "add%20's%c.w\t%8-11r, %16-19r, %M"},
-  {ARM_EXT_V6T2, 0xf1400000, 0xfbe08000, "adc%20's%c.w\t%8-11r, %16-19r, %M"},
-  {ARM_EXT_V6T2, 0xf1600000, 0xfbe08000, "sbc%20's%c.w\t%8-11r, %16-19r, %M"},
-  {ARM_EXT_V6T2, 0xf1a00000, 0xfbe08000, "sub%20's%c.w\t%8-11r, %16-19r, %M"},
-  {ARM_EXT_V6T2, 0xf1c00000, 0xfbe08000, "rsb%20's%c\t%8-11r, %16-19r, %M"},
-  {ARM_EXT_V6T2, 0xe8800000, 0xffd00000, "stmia%c.w\t%16-19r%21'!, %m"},
-  {ARM_EXT_V6T2, 0xe8900000, 0xffd00000, "ldmia%c.w\t%16-19r%21'!, %m"},
-  {ARM_EXT_V6T2, 0xe9000000, 0xffd00000, "stmdb%c\t%16-19r%21'!, %m"},
-  {ARM_EXT_V6T2, 0xe9100000, 0xffd00000, "ldmdb%c\t%16-19r%21'!, %m"},
-  {ARM_EXT_V6T2, 0xe9c00000, 0xffd000ff, "strd%c\t%12-15r, %8-11r, [%16-19r]"},
-  {ARM_EXT_V6T2, 0xe9d00000, 0xffd000ff, "ldrd%c\t%12-15r, %8-11r, [%16-19r]"},
-  {ARM_EXT_V6T2, 0xe9400000, 0xff500000, "strd%c\t%12-15r, %8-11r, [%16-19r, #%23`-%0-7W]%21'!"},
-  {ARM_EXT_V6T2, 0xe9500000, 0xff500000, "ldrd%c\t%12-15r, %8-11r, [%16-19r, #%23`-%0-7W]%21'!"},
-  {ARM_EXT_V6T2, 0xe8600000, 0xff700000, "strd%c\t%12-15r, %8-11r, [%16-19r], #%23`-%0-7W"},
-  {ARM_EXT_V6T2, 0xe8700000, 0xff700000, "ldrd%c\t%12-15r, %8-11r, [%16-19r], #%23`-%0-7W"},
-  {ARM_EXT_V6T2, 0xf8000000, 0xff100000, "str%w%c.w\t%12-15r, %a"},
-  {ARM_EXT_V6T2, 0xf8100000, 0xfe100000, "ldr%w%c.w\t%12-15r, %a"},
-
-  /* Filter out Bcc with cond=E or F, which are used for other instructions.  */
-  {ARM_EXT_V6T2, 0xf3c08000, 0xfbc0d000, "undefined (bcc, cond=0xF)"},
-  {ARM_EXT_V6T2, 0xf3808000, 0xfbc0d000, "undefined (bcc, cond=0xE)"},
-  {ARM_EXT_V6T2, 0xf0008000, 0xf800d000, "b%22-25c.w\t%b%X"},
-  {ARM_EXT_V6T2, 0xf0009000, 0xf800d000, "b%c.w\t%B%x"},
-
-  /* These have been 32-bit since the invention of Thumb.  */
-  {ARM_EXT_V4T,  0xf000c000, 0xf800d000, "blx%c\t%B%x"},
-  {ARM_EXT_V4T,  0xf000d000, 0xf800d000, "bl%c\t%B%x"},
-
-  /* Fallback.  */
-  {ARM_EXT_V1,   0x00000000, 0x00000000, "undefined"},
-  {0, 0, 0, 0}
-};
-
-static const char *const arm_conditional[] =
-{"eq", "ne", "cs", "cc", "mi", "pl", "vs", "vc",
- "hi", "ls", "ge", "lt", "gt", "le", "al", "<und>", ""};
-
-static const char *const arm_fp_const[] =
-{"0.0", "1.0", "2.0", "3.0", "4.0", "5.0", "0.5", "10.0"};
-
-static const char *const arm_shift[] =
-{"lsl", "lsr", "asr", "ror"};
-
-typedef struct
-{
-  const char *name;
-  const char *description;
-  const char *reg_names[16];
-}
-arm_regname;
-
-static const arm_regname regnames[] =
-{
-  { "raw" , "Select raw register names",
-    { "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7", "r8", "r9", "r10", "r11", "r12", "r13", "r14", "r15"}},
-  { "gcc",  "Select register names used by GCC",
-    { "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7", "r8", "r9", "sl",  "fp",  "ip",  "sp",  "lr",  "pc" }},
-  { "std",  "Select register names used in ARM's ISA documentation",
-    { "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7", "r8", "r9", "r10", "r11", "r12", "sp",  "lr",  "pc" }},
-  { "apcs", "Select register names used in the APCS",
-    { "a1", "a2", "a3", "a4", "v1", "v2", "v3", "v4", "v5", "v6", "sl",  "fp",  "ip",  "sp",  "lr",  "pc" }},
-  { "atpcs", "Select register names used in the ATPCS",
-    { "a1", "a2", "a3", "a4", "v1", "v2", "v3", "v4", "v5", "v6", "v7",  "v8",  "IP",  "SP",  "LR",  "PC" }},
-  { "special-atpcs", "Select special register names used in the ATPCS",
-    { "a1", "a2", "a3", "a4", "v1", "v2", "v3", "WR", "v5", "SB", "SL",  "FP",  "IP",  "SP",  "LR",  "PC" }},
-};
-
-static const char *const iwmmxt_wwnames[] =
-{"b", "h", "w", "d"};
-
-static const char *const iwmmxt_wwssnames[] =
-{"b", "bus", "bc", "bss",
- "h", "hus", "hc", "hss",
- "w", "wus", "wc", "wss",
- "d", "dus", "dc", "dss"
-};
-
-static const char *const iwmmxt_regnames[] =
-{ "wr0", "wr1", "wr2", "wr3", "wr4", "wr5", "wr6", "wr7",
-  "wr8", "wr9", "wr10", "wr11", "wr12", "wr13", "wr14", "wr15"
-};
-
-static const char *const iwmmxt_cregnames[] =
-{ "wcid", "wcon", "wcssf", "wcasf", "reserved", "reserved", "reserved", "reserved",
-  "wcgr0", "wcgr1", "wcgr2", "wcgr3", "reserved", "reserved", "reserved", "reserved"
-};
-
-/* Default to GCC register name set.  */
-static unsigned int regname_selected = 1;
-
-#define arm_regnames      regnames[regname_selected].reg_names
-
-static bfd_boolean force_thumb = false;
-
-/* Current IT instruction state.  This contains the same state as the IT
-   bits in the CPSR.  */
-static unsigned int ifthen_state;
-/* IT state for the next instruction.  */
-static unsigned int ifthen_next_state;
-/* The address of the insn for which the IT state is valid.  */
-static bfd_vma ifthen_address;
-#define IFTHEN_COND ((ifthen_state >> 4) & 0xf)
-
-/* Cached mapping symbol state.  */
-enum map_type {
-  MAP_ARM,
-  MAP_THUMB,
-  MAP_DATA
-};
-
-/* Decode a bitfield of the form matching regexp (N(-N)?,)*N(-N)?.
-   Returns pointer to following character of the format string and
-   fills in *VALUEP and *WIDTHP with the extracted value and number of
-   bits extracted.  WIDTHP can be NULL. */
-
-static const char *
-arm_decode_bitfield (const char *ptr, unsigned long insn,
-		     unsigned long *valuep, int *widthp)
-{
-  unsigned long value = 0;
-  int width = 0;
-
-  do
-    {
-      int start, end;
-      int bits;
-
-      for (start = 0; *ptr >= '0' && *ptr <= '9'; ptr++)
-	start = start * 10 + *ptr - '0';
-      if (*ptr == '-')
-	for (end = 0, ptr++; *ptr >= '0' && *ptr <= '9'; ptr++)
-	  end = end * 10 + *ptr - '0';
-      else
-	end = start;
-      bits = end - start;
-      if (bits < 0)
-	abort ();
-      value |= ((insn >> start) & ((2ul << bits) - 1)) << width;
-      width += bits + 1;
-    }
-  while (*ptr++ == ',');
-  *valuep = value;
-  if (widthp)
-    *widthp = width;
-  return ptr - 1;
-}
-
-static void
-arm_decode_shift (long given, fprintf_function func, void *stream,
-		  int print_shift)
-{
-  func (stream, "%s", arm_regnames[given & 0xf]);
-
-  if ((given & 0xff0) != 0)
-    {
-      if ((given & 0x10) == 0)
-	{
-	  int amount = (given & 0xf80) >> 7;
-	  int shift = (given & 0x60) >> 5;
-
-	  if (amount == 0)
-	    {
-	      if (shift == 3)
-		{
-		  func (stream, ", rrx");
-		  return;
-		}
-
-	      amount = 32;
-	    }
-
-	  if (print_shift)
-	    func (stream, ", %s #%d", arm_shift[shift], amount);
-	  else
-	    func (stream, ", #%d", amount);
-	}
-      else if (print_shift)
-	func (stream, ", %s %s", arm_shift[(given & 0x60) >> 5],
-	      arm_regnames[(given & 0xf00) >> 8]);
-      else
-	func (stream, ", %s", arm_regnames[(given & 0xf00) >> 8]);
-    }
-}
-
-/* Print one coprocessor instruction on INFO->STREAM.
-   Return true if the instruction matched, false if this is not a
-   recognised coprocessor instruction.  */
-
-static bfd_boolean
-print_insn_coprocessor (bfd_vma pc, struct disassemble_info *info, long given,
-			bfd_boolean thumb)
-{
-  const struct opcode32 *insn;
-  void *stream = info->stream;
-  fprintf_function func = info->fprintf_func;
-  unsigned long mask;
-  unsigned long value;
-  int cond;
-
-  for (insn = coprocessor_opcodes; insn->assembler; insn++)
-    {
-      if (insn->value == FIRST_IWMMXT_INSN
-	  && info->mach != bfd_mach_arm_XScale
-	  && info->mach != bfd_mach_arm_iWMMXt
-	  && info->mach != bfd_mach_arm_iWMMXt2)
-	insn = insn + IWMMXT_INSN_COUNT;
-
-      mask = insn->mask;
-      value = insn->value;
-      if (thumb)
-	{
-	  /* The high 4 bits are 0xe for Arm conditional instructions, and
-	     0xe for arm unconditional instructions.  The rest of the
-	     encoding is the same.  */
-	  mask |= 0xf0000000;
-	  value |= 0xe0000000;
-	  if (ifthen_state)
-	    cond = IFTHEN_COND;
-	  else
-	    cond = 16;
-	}
-      else
-	{
-          /* Only match unconditional instructions against unconditional
-	     patterns.  */
-	  if ((given & 0xf0000000) == 0xf0000000)
-	    {
-	      mask |= 0xf0000000;
-	      cond = 16;
-	    }
-	  else
-	    {
-	      cond = (given >> 28) & 0xf;
-	      if (cond == 0xe)
-		cond = 16;
-	    }
-	}
-      if ((given & mask) == value)
-	{
-	  const char *c;
-
-	  for (c = insn->assembler; *c; c++)
-	    {
-	      if (*c == '%')
-		{
-		  switch (*++c)
-		    {
-		    case '%':
-		      func (stream, "%%");
-		      break;
-
-		    case 'A':
-		      func (stream, "[%s", arm_regnames [(given >> 16) & 0xf]);
-
-		      if ((given & (1 << 24)) != 0)
-			{
-			  int offset = given & 0xff;
-
-			  if (offset)
-			    func (stream, ", #%s%d]%s",
-				  ((given & 0x00800000) == 0 ? "-" : ""),
-				  offset * 4,
-				  ((given & 0x00200000) != 0 ? "!" : ""));
-			  else
-			    func (stream, "]");
-			}
-		      else
-			{
-			  int offset = given & 0xff;
-
-			  func (stream, "]");
-
-			  if (given & (1 << 21))
-			    {
-			      if (offset)
-				func (stream, ", #%s%d",
-				      ((given & 0x00800000) == 0 ? "-" : ""),
-				      offset * 4);
-			    }
-			  else
-			    func (stream, ", {%d}", offset);
-			}
-		      break;
-
-		    case 'B':
-		      {
-			int regno = ((given >> 12) & 0xf) | ((given >> (22 - 4)) & 0x10);
-			int offset = (given >> 1) & 0x3f;
-
-			if (offset == 1)
-			  func (stream, "{d%d}", regno);
-			else if (regno + offset > 32)
-			  func (stream, "{d%d-<overflow reg d%d>}", regno, regno + offset - 1);
-			else
-			  func (stream, "{d%d-d%d}", regno, regno + offset - 1);
-		      }
-		      break;
-
-		    case 'C':
-		      {
-			int rn = (given >> 16) & 0xf;
-			int offset = (given & 0xff) * 4;
-			int add = (given >> 23) & 1;
-
-			func (stream, "[%s", arm_regnames[rn]);
-
-			if (offset)
-			  {
-			    if (!add)
-			      offset = -offset;
-			    func (stream, ", #%d", offset);
-			  }
-			func (stream, "]");
-			if (rn == 15)
-			  {
-			    func (stream, "\t; ");
-                            /* FIXME: Unsure if info->bytes_per_chunk is the
-                               right thing to use here.  */
-			    info->print_address_func (offset + pc
-                              + info->bytes_per_chunk * 2, info);
-			  }
-		      }
-		      break;
-
-		    case 'c':
-		      func (stream, "%s", arm_conditional[cond]);
-		      break;
-
-		    case 'I':
-		      /* Print a Cirrus/DSP shift immediate.  */
-		      /* Immediates are 7bit signed ints with bits 0..3 in
-			 bits 0..3 of opcode and bits 4..6 in bits 5..7
-			 of opcode.  */
-		      {
-			int imm;
-
-			imm = (given & 0xf) | ((given & 0xe0) >> 1);
-
-			/* Is ``imm'' a negative number?  */
-			if (imm & 0x40)
-			  imm |= (~0u << 7);
-
-			func (stream, "%d", imm);
-		      }
-
-		      break;
-
-		    case 'F':
-		      switch (given & 0x00408000)
-			{
-			case 0:
-			  func (stream, "4");
-			  break;
-			case 0x8000:
-			  func (stream, "1");
-			  break;
-			case 0x00400000:
-			  func (stream, "2");
-			  break;
-			default:
-			  func (stream, "3");
-			}
-		      break;
-
-		    case 'P':
-		      switch (given & 0x00080080)
-			{
-			case 0:
-			  func (stream, "s");
-			  break;
-			case 0x80:
-			  func (stream, "d");
-			  break;
-			case 0x00080000:
-			  func (stream, "e");
-			  break;
-			default:
-			  func (stream, "<illegal precision>");
-			  break;
-			}
-		      break;
-		    case 'Q':
-		      switch (given & 0x00408000)
-			{
-			case 0:
-			  func (stream, "s");
-			  break;
-			case 0x8000:
-			  func (stream, "d");
-			  break;
-			case 0x00400000:
-			  func (stream, "e");
-			  break;
-			default:
-			  func (stream, "p");
-			  break;
-			}
-		      break;
-		    case 'R':
-		      switch (given & 0x60)
-			{
-			case 0:
-			  break;
-			case 0x20:
-			  func (stream, "p");
-			  break;
-			case 0x40:
-			  func (stream, "m");
-			  break;
-			default:
-			  func (stream, "z");
-			  break;
-			}
-		      break;
-
-		    case '0': case '1': case '2': case '3': case '4':
-		    case '5': case '6': case '7': case '8': case '9':
-		      {
-			int width;
-			unsigned long value;
-
-			c = arm_decode_bitfield (c, given, &value, &width);
-
-			switch (*c)
-			  {
-			  case 'r':
-			    func (stream, "%s", arm_regnames[value]);
-			    break;
-			  case 'D':
-			    func (stream, "d%ld", value);
-			    break;
-			  case 'Q':
-			    if (value & 1)
-			      func (stream, "<illegal reg q%ld.5>", value >> 1);
-			    else
-			      func (stream, "q%ld", value >> 1);
-			    break;
-			  case 'd':
-			    func (stream, "%ld", value);
-			    break;
-                          case 'k':
-                            {
-                              int from = (given & (1 << 7)) ? 32 : 16;
-                              func (stream, "%ld", from - value);
-                            }
-                            break;
-
-			  case 'f':
-			    if (value > 7)
-			      func (stream, "#%s", arm_fp_const[value & 7]);
-			    else
-			      func (stream, "f%ld", value);
-			    break;
-
-			  case 'w':
-			    if (width == 2)
-			      func (stream, "%s", iwmmxt_wwnames[value]);
-			    else
-			      func (stream, "%s", iwmmxt_wwssnames[value]);
-			    break;
-
-			  case 'g':
-			    func (stream, "%s", iwmmxt_regnames[value]);
-			    break;
-			  case 'G':
-			    func (stream, "%s", iwmmxt_cregnames[value]);
-			    break;
-
-			  case 'x':
-			    func (stream, "0x%lx", value);
-			    break;
-
-			  case '`':
-			    c++;
-			    if (value == 0)
-			      func (stream, "%c", *c);
-			    break;
-			  case '\'':
-			    c++;
-			    if (value == ((1ul << width) - 1))
-			      func (stream, "%c", *c);
-			    break;
-			  case '?':
-			    func (stream, "%c", c[(1 << width) - (int)value]);
-			    c += 1 << width;
-			    break;
-			  default:
-			    abort ();
-			  }
-			break;
-
-		      case 'y':
-		      case 'z':
-			{
-			  int single = *c++ == 'y';
-			  int regno;
-
-			  switch (*c)
-			    {
-			    case '4': /* Sm pair */
-			      func (stream, "{");
-			      /* Fall through.  */
-			    case '0': /* Sm, Dm */
-			      regno = given & 0x0000000f;
-			      if (single)
-				{
-				  regno <<= 1;
-				  regno += (given >> 5) & 1;
-				}
-                              else
-                                regno += ((given >> 5) & 1) << 4;
-			      break;
-
-			    case '1': /* Sd, Dd */
-			      regno = (given >> 12) & 0x0000000f;
-			      if (single)
-				{
-				  regno <<= 1;
-				  regno += (given >> 22) & 1;
-				}
-                              else
-                                regno += ((given >> 22) & 1) << 4;
-			      break;
-
-			    case '2': /* Sn, Dn */
-			      regno = (given >> 16) & 0x0000000f;
-			      if (single)
-				{
-				  regno <<= 1;
-				  regno += (given >> 7) & 1;
-				}
-                              else
-                                regno += ((given >> 7) & 1) << 4;
-			      break;
-
-			    case '3': /* List */
-			      func (stream, "{");
-			      regno = (given >> 12) & 0x0000000f;
-			      if (single)
-				{
-				  regno <<= 1;
-				  regno += (given >> 22) & 1;
-				}
-                              else
-                                regno += ((given >> 22) & 1) << 4;
-			      break;
-
-			    default:
-			      abort ();
-			    }
-
-			  func (stream, "%c%d", single ? 's' : 'd', regno);
-
-			  if (*c == '3')
-			    {
-			      int count = given & 0xff;
-
-			      if (single == 0)
-				count >>= 1;
-
-			      if (--count)
-				{
-				  func (stream, "-%c%d",
-					single ? 's' : 'd',
-					regno + count);
-				}
-
-			      func (stream, "}");
-			    }
-			  else if (*c == '4')
-			    func (stream, ", %c%d}", single ? 's' : 'd',
-				  regno + 1);
-			}
-			break;
-
-		      case 'L':
-			switch (given & 0x00400100)
-			  {
-			  case 0x00000000: func (stream, "b"); break;
-			  case 0x00400000: func (stream, "h"); break;
-			  case 0x00000100: func (stream, "w"); break;
-			  case 0x00400100: func (stream, "d"); break;
-			  default:
-			    break;
-			  }
-			break;
-
-		      case 'Z':
-			{
-			  int value;
-			  /* given (20, 23) | given (0, 3) */
-			  value = ((given >> 16) & 0xf0) | (given & 0xf);
-			  func (stream, "%d", value);
-			}
-			break;
-
-		      case 'l':
-			/* This is like the 'A' operator, except that if
-			   the width field "M" is zero, then the offset is
-			   *not* multiplied by four.  */
-			{
-			  int offset = given & 0xff;
-			  int multiplier = (given & 0x00000100) ? 4 : 1;
-
-			  func (stream, "[%s", arm_regnames [(given >> 16) & 0xf]);
-
-			  if (offset)
-			    {
-			      if ((given & 0x01000000) != 0)
-				func (stream, ", #%s%d]%s",
-				      ((given & 0x00800000) == 0 ? "-" : ""),
-				      offset * multiplier,
-				      ((given & 0x00200000) != 0 ? "!" : ""));
-			      else
-				func (stream, "], #%s%d",
-				      ((given & 0x00800000) == 0 ? "-" : ""),
-				      offset * multiplier);
-			    }
-			  else
-			    func (stream, "]");
-			}
-			break;
-
-		      case 'r':
-			{
-			  int imm4 = (given >> 4) & 0xf;
-			  int puw_bits = ((given >> 22) & 6) | ((given >> 21) & 1);
-			  int ubit = (given >> 23) & 1;
-			  const char *rm = arm_regnames [given & 0xf];
-			  const char *rn = arm_regnames [(given >> 16) & 0xf];
-
-			  switch (puw_bits)
-			    {
-			    case 1:
-			      /* fall through */
-			    case 3:
-			      func (stream, "[%s], %c%s", rn, ubit ? '+' : '-', rm);
-			      if (imm4)
-				func (stream, ", lsl #%d", imm4);
-			      break;
-
-			    case 4:
-			      /* fall through */
-			    case 5:
-			      /* fall through */
-			    case 6:
-			      /* fall through */
-			    case 7:
-			      func (stream, "[%s, %c%s", rn, ubit ? '+' : '-', rm);
-			      if (imm4 > 0)
-				func (stream, ", lsl #%d", imm4);
-			      func (stream, "]");
-			      if (puw_bits == 5 || puw_bits == 7)
-				func (stream, "!");
-			      break;
-
-			    default:
-			      func (stream, "INVALID");
-			    }
-			}
-			break;
-
-		      case 'i':
-			{
-			  long imm5;
-			  imm5 = ((given & 0x100) >> 4) | (given & 0xf);
-			  func (stream, "%ld", (imm5 == 0) ? 32 : imm5);
-			}
-			break;
-
-		      default:
-			abort ();
-		      }
-		    }
-		}
-	      else
-		func (stream, "%c", *c);
-	    }
-	  return true;
-	}
-    }
-  return false;
-}
-
-static void
-print_arm_address (bfd_vma pc, struct disassemble_info *info, long given)
-{
-  void *stream = info->stream;
-  fprintf_function func = info->fprintf_func;
-
-  if (((given & 0x000f0000) == 0x000f0000)
-      && ((given & 0x02000000) == 0))
-    {
-      int offset = given & 0xfff;
-
-      func (stream, "[pc");
-
-      if (given & 0x01000000)
-	{
-	  if ((given & 0x00800000) == 0)
-	    offset = - offset;
-
-	  /* Pre-indexed.  */
-	  func (stream, ", #%d]", offset);
-
-	  offset += pc + 8;
-
-	  /* Cope with the possibility of write-back
-	     being used.  Probably a very dangerous thing
-	     for the programmer to do, but who are we to
-	     argue ?  */
-	  if (given & 0x00200000)
-	    func (stream, "!");
-	}
-      else
-	{
-	  /* Post indexed.  */
-	  func (stream, "], #%d", offset);
-
-	  /* ie ignore the offset.  */
-	  offset = pc + 8;
-	}
-
-      func (stream, "\t; ");
-      info->print_address_func (offset, info);
-    }
-  else
-    {
-      func (stream, "[%s",
-	    arm_regnames[(given >> 16) & 0xf]);
-      if ((given & 0x01000000) != 0)
-	{
-	  if ((given & 0x02000000) == 0)
-	    {
-	      int offset = given & 0xfff;
-	      if (offset)
-		func (stream, ", #%s%d",
-		      (((given & 0x00800000) == 0)
-		       ? "-" : ""), offset);
-	    }
-	  else
-	    {
-	      func (stream, ", %s",
-		    (((given & 0x00800000) == 0)
-		     ? "-" : ""));
-	      arm_decode_shift (given, func, stream, 1);
-	    }
-
-	  func (stream, "]%s",
-		((given & 0x00200000) != 0) ? "!" : "");
-	}
-      else
-	{
-	  if ((given & 0x02000000) == 0)
-	    {
-	      int offset = given & 0xfff;
-	      if (offset)
-		func (stream, "], #%s%d",
-		      (((given & 0x00800000) == 0)
-		       ? "-" : ""), offset);
-	      else
-		func (stream, "]");
-	    }
-	  else
-	    {
-	      func (stream, "], %s",
-		    (((given & 0x00800000) == 0)
-		     ? "-" : ""));
-	      arm_decode_shift (given, func, stream, 1);
-	    }
-	}
-    }
-}
-
-/* Print one neon instruction on INFO->STREAM.
-   Return true if the instruction matched, false if this is not a
-   recognised neon instruction.  */
-
-static bfd_boolean
-print_insn_neon (struct disassemble_info *info, long given, bfd_boolean thumb)
-{
-  const struct opcode32 *insn;
-  void *stream = info->stream;
-  fprintf_function func = info->fprintf_func;
-
-  if (thumb)
-    {
-      if ((given & 0xef000000) == 0xef000000)
-	{
-	  /* move bit 28 to bit 24 to translate Thumb2 to ARM encoding.  */
-	  unsigned long bit28 = given & (1 << 28);
-
-	  given &= 0x00ffffff;
-	  if (bit28)
-            given |= 0xf3000000;
-          else
-	    given |= 0xf2000000;
-	}
-      else if ((given & 0xff000000) == 0xf9000000)
-	given ^= 0xf9000000 ^ 0xf4000000;
-      else
-	return false;
-    }
-
-  for (insn = neon_opcodes; insn->assembler; insn++)
-    {
-      if ((given & insn->mask) == insn->value)
-	{
-	  const char *c;
-
-	  for (c = insn->assembler; *c; c++)
-	    {
-	      if (*c == '%')
-		{
-		  switch (*++c)
-		    {
-		    case '%':
-		      func (stream, "%%");
-		      break;
-
-		    case 'c':
-		      if (thumb && ifthen_state)
-			func (stream, "%s", arm_conditional[IFTHEN_COND]);
-		      break;
-
-		    case 'A':
-		      {
-			static const unsigned char enc[16] =
-			{
-			  0x4, 0x14, /* st4 0,1 */
-			  0x4, /* st1 2 */
-			  0x4, /* st2 3 */
-			  0x3, /* st3 4 */
-			  0x13, /* st3 5 */
-			  0x3, /* st1 6 */
-			  0x1, /* st1 7 */
-			  0x2, /* st2 8 */
-			  0x12, /* st2 9 */
-			  0x2, /* st1 10 */
-			  0, 0, 0, 0, 0
-			};
-			int rd = ((given >> 12) & 0xf) | (((given >> 22) & 1) << 4);
-			int rn = ((given >> 16) & 0xf);
-			int rm = ((given >> 0) & 0xf);
-			int align = ((given >> 4) & 0x3);
-			int type = ((given >> 8) & 0xf);
-			int n = enc[type] & 0xf;
-			int stride = (enc[type] >> 4) + 1;
-			int ix;
-
-			func (stream, "{");
-			if (stride > 1)
-			  for (ix = 0; ix != n; ix++)
-			    func (stream, "%sd%d", ix ? "," : "", rd + ix * stride);
-			else if (n == 1)
-			  func (stream, "d%d", rd);
-			else
-			  func (stream, "d%d-d%d", rd, rd + n - 1);
-			func (stream, "}, [%s", arm_regnames[rn]);
-			if (align)
-			  func (stream, ", :%d", 32 << align);
-			func (stream, "]");
-			if (rm == 0xd)
-			  func (stream, "!");
-			else if (rm != 0xf)
-			  func (stream, ", %s", arm_regnames[rm]);
-		      }
-		      break;
-
-		    case 'B':
-		      {
-			int rd = ((given >> 12) & 0xf) | (((given >> 22) & 1) << 4);
-			int rn = ((given >> 16) & 0xf);
-			int rm = ((given >> 0) & 0xf);
-			int idx_align = ((given >> 4) & 0xf);
-                        int align = 0;
-			int size = ((given >> 10) & 0x3);
-			int idx = idx_align >> (size + 1);
-                        int length = ((given >> 8) & 3) + 1;
-                        int stride = 1;
-                        int i;
-
-                        if (length > 1 && size > 0)
-                          stride = (idx_align & (1 << size)) ? 2 : 1;
-
-                        switch (length)
-                          {
-                          case 1:
-                            {
-                              int amask = (1 << size) - 1;
-                              if ((idx_align & (1 << size)) != 0)
-                                return false;
-                              if (size > 0)
-                                {
-                                  if ((idx_align & amask) == amask)
-                                    align = 8 << size;
-                                  else if ((idx_align & amask) != 0)
-                                    return false;
-                                }
-                              }
-                            break;
-
-                          case 2:
-                            if (size == 2 && (idx_align & 2) != 0)
-                              return false;
-                            align = (idx_align & 1) ? 16 << size : 0;
-                            break;
-
-                          case 3:
-                            if ((size == 2 && (idx_align & 3) != 0)
-                                || (idx_align & 1) != 0)
-                              return false;
-                            break;
-
-                          case 4:
-                            if (size == 2)
-                              {
-                                if ((idx_align & 3) == 3)
-                                  return false;
-                                align = (idx_align & 3) * 64;
-                              }
-                            else
-                              align = (idx_align & 1) ? 32 << size : 0;
-                            break;
-
-                          default:
-                            abort ();
-                          }
-
-			func (stream, "{");
-                        for (i = 0; i < length; i++)
-                          func (stream, "%sd%d[%d]", (i == 0) ? "" : ",",
-                            rd + i * stride, idx);
-                        func (stream, "}, [%s", arm_regnames[rn]);
-			if (align)
-			  func (stream, ", :%d", align);
-			func (stream, "]");
-			if (rm == 0xd)
-			  func (stream, "!");
-			else if (rm != 0xf)
-			  func (stream, ", %s", arm_regnames[rm]);
-		      }
-		      break;
-
-		    case 'C':
-		      {
-			int rd = ((given >> 12) & 0xf) | (((given >> 22) & 1) << 4);
-			int rn = ((given >> 16) & 0xf);
-			int rm = ((given >> 0) & 0xf);
-			int align = ((given >> 4) & 0x1);
-			int size = ((given >> 6) & 0x3);
-			int type = ((given >> 8) & 0x3);
-			int n = type + 1;
-			int stride = ((given >> 5) & 0x1);
-			int ix;
-
-			if (stride && (n == 1))
-			  n++;
-			else
-			  stride++;
-
-			func (stream, "{");
-			if (stride > 1)
-			  for (ix = 0; ix != n; ix++)
-			    func (stream, "%sd%d[]", ix ? "," : "", rd + ix * stride);
-			else if (n == 1)
-			  func (stream, "d%d[]", rd);
-			else
-			  func (stream, "d%d[]-d%d[]", rd, rd + n - 1);
-			func (stream, "}, [%s", arm_regnames[rn]);
-			if (align)
-			  {
-                            int align = (8 * (type + 1)) << size;
-                            if (type == 3)
-                              align = (size > 1) ? align >> 1 : align;
-			    if (type == 2 || (type == 0 && !size))
-			      func (stream, ", :<bad align %d>", align);
-			    else
-			      func (stream, ", :%d", align);
-			  }
-			func (stream, "]");
-			if (rm == 0xd)
-			  func (stream, "!");
-			else if (rm != 0xf)
-			  func (stream, ", %s", arm_regnames[rm]);
-		      }
-		      break;
-
-		    case 'D':
-		      {
-			int raw_reg = (given & 0xf) | ((given >> 1) & 0x10);
-			int size = (given >> 20) & 3;
-			int reg = raw_reg & ((4 << size) - 1);
-			int ix = raw_reg >> size >> 2;
-
-			func (stream, "d%d[%d]", reg, ix);
-		      }
-		      break;
-
-		    case 'E':
-		      /* Neon encoded constant for mov, mvn, vorr, vbic */
-		      {
-			int bits = 0;
-			int cmode = (given >> 8) & 0xf;
-			int op = (given >> 5) & 0x1;
-			unsigned long value = 0, hival = 0;
-			unsigned shift;
-                        int size = 0;
-                        int isfloat = 0;
-
-			bits |= ((given >> 24) & 1) << 7;
-			bits |= ((given >> 16) & 7) << 4;
-			bits |= ((given >> 0) & 15) << 0;
-
-			if (cmode < 8)
-			  {
-			    shift = (cmode >> 1) & 3;
-			    value = (unsigned long)bits << (8 * shift);
-                            size = 32;
-			  }
-			else if (cmode < 12)
-			  {
-			    shift = (cmode >> 1) & 1;
-			    value = (unsigned long)bits << (8 * shift);
-                            size = 16;
-			  }
-			else if (cmode < 14)
-			  {
-			    shift = (cmode & 1) + 1;
-			    value = (unsigned long)bits << (8 * shift);
-			    value |= (1ul << (8 * shift)) - 1;
-                            size = 32;
-			  }
-			else if (cmode == 14)
-			  {
-			    if (op)
-			      {
-				/* bit replication into bytes */
-				int ix;
-				unsigned long mask;
-
-				value = 0;
-                                hival = 0;
-				for (ix = 7; ix >= 0; ix--)
-				  {
-				    mask = ((bits >> ix) & 1) ? 0xff : 0;
-                                    if (ix <= 3)
-				      value = (value << 8) | mask;
-                                    else
-                                      hival = (hival << 8) | mask;
-				  }
-                                size = 64;
-			      }
-                            else
-                              {
-                                /* byte replication */
-                                value = (unsigned long)bits;
-                                size = 8;
-                              }
-			  }
-			else if (!op)
-			  {
-			    /* floating point encoding */
-			    int tmp;
-
-			    value = (unsigned long)(bits & 0x7f) << 19;
-			    value |= (unsigned long)(bits & 0x80) << 24;
-			    tmp = bits & 0x40 ? 0x3c : 0x40;
-			    value |= (unsigned long)tmp << 24;
-                            size = 32;
-                            isfloat = 1;
-			  }
-			else
-			  {
-			    func (stream, "<illegal constant %.8x:%x:%x>",
-                                  bits, cmode, op);
-			    break;
-			  }
-                        switch (size)
-                          {
-                          case 8:
-			    func (stream, "#%ld\t; 0x%.2lx", value, value);
-                            break;
-
-                          case 16:
-                            func (stream, "#%ld\t; 0x%.4lx", value, value);
-                            break;
-
-                          case 32:
-                            if (isfloat)
-                              {
-                                unsigned char valbytes[4];
-                                double fvalue;
-
-                                /* Do this a byte at a time so we don't have to
-                                   worry about the host's endianness.  */
-                                valbytes[0] = value & 0xff;
-                                valbytes[1] = (value >> 8) & 0xff;
-                                valbytes[2] = (value >> 16) & 0xff;
-                                valbytes[3] = (value >> 24) & 0xff;
-
-                                floatformat_to_double (valbytes, &fvalue);
-
-                                func (stream, "#%.7g\t; 0x%.8lx", fvalue,
-                                      value);
-                              }
-                            else
-                              func (stream, "#%ld\t; 0x%.8lx",
-				(long) ((value & 0x80000000)
-					? value | ~0xffffffffl : value), value);
-                            break;
-
-                          case 64:
-                            func (stream, "#0x%.8lx%.8lx", hival, value);
-                            break;
-
-                          default:
-                            abort ();
-                          }
-		      }
-		      break;
-
-		    case 'F':
-		      {
-			int regno = ((given >> 16) & 0xf) | ((given >> (7 - 4)) & 0x10);
-			int num = (given >> 8) & 0x3;
-
-			if (!num)
-			  func (stream, "{d%d}", regno);
-			else if (num + regno >= 32)
-			  func (stream, "{d%d-<overflow reg d%d}", regno, regno + num);
-			else
-			  func (stream, "{d%d-d%d}", regno, regno + num);
-		      }
-		      break;
-
-
-		    case '0': case '1': case '2': case '3': case '4':
-		    case '5': case '6': case '7': case '8': case '9':
-		      {
-			int width;
-			unsigned long value;
-
-			c = arm_decode_bitfield (c, given, &value, &width);
-
-			switch (*c)
-			  {
-			  case 'r':
-			    func (stream, "%s", arm_regnames[value]);
-			    break;
-			  case 'd':
-			    func (stream, "%ld", value);
-			    break;
-			  case 'e':
-			    func (stream, "%ld", (1ul << width) - value);
-			    break;
-
-			  case 'S':
-			  case 'T':
-			  case 'U':
-			    /* various width encodings */
-			    {
-			      int base = 8 << (*c - 'S'); /* 8,16 or 32 */
-			      int limit;
-			      unsigned low, high;
-
-			      c++;
-			      if (*c >= '0' && *c <= '9')
-				limit = *c - '0';
-			      else if (*c >= 'a' && *c <= 'f')
-				limit = *c - 'a' + 10;
-			      else
-				abort ();
-			      low = limit >> 2;
-			      high = limit & 3;
-
-			      if (value < low || value > high)
-				func (stream, "<illegal width %d>", base << value);
-			      else
-				func (stream, "%d", base << value);
-			    }
-			    break;
-			  case 'R':
-			    if (given & (1 << 6))
-			      goto Q;
-			    /* FALLTHROUGH */
-			  case 'D':
-			    func (stream, "d%ld", value);
-			    break;
-			  case 'Q':
-			  Q:
-			    if (value & 1)
-			      func (stream, "<illegal reg q%ld.5>", value >> 1);
-			    else
-			      func (stream, "q%ld", value >> 1);
-			    break;
-
-			  case '`':
-			    c++;
-			    if (value == 0)
-			      func (stream, "%c", *c);
-			    break;
-			  case '\'':
-			    c++;
-			    if (value == ((1ul << width) - 1))
-			      func (stream, "%c", *c);
-			    break;
-			  case '?':
-			    func (stream, "%c", c[(1 << width) - (int)value]);
-			    c += 1 << width;
-			    break;
-			  default:
-			    abort ();
-			  }
-			break;
-
-		      default:
-			abort ();
-		      }
-		    }
-		}
-	      else
-		func (stream, "%c", *c);
-	    }
-	  return true;
-	}
-    }
-  return false;
-}
-
-/* Print one ARM instruction from PC on INFO->STREAM.  */
-
-static void
-print_insn_arm_internal (bfd_vma pc, struct disassemble_info *info, long given)
-{
-  const struct opcode32 *insn;
-  void *stream = info->stream;
-  fprintf_function func = info->fprintf_func;
-
-  if (print_insn_coprocessor (pc, info, given, false))
-    return;
-
-  if (print_insn_neon (info, given, false))
-    return;
-
-  for (insn = arm_opcodes; insn->assembler; insn++)
-    {
-      if (insn->value == FIRST_IWMMXT_INSN
-	  && info->mach != bfd_mach_arm_XScale
-	  && info->mach != bfd_mach_arm_iWMMXt)
-	insn = insn + IWMMXT_INSN_COUNT;
-
-      if ((given & insn->mask) == insn->value
-	  /* Special case: an instruction with all bits set in the condition field
-	     (0xFnnn_nnnn) is only matched if all those bits are set in insn->mask,
-	     or by the catchall at the end of the table.  */
-	  && ((given & 0xF0000000) != 0xF0000000
-	      || (insn->mask & 0xF0000000) == 0xF0000000
-	      || (insn->mask == 0 && insn->value == 0)))
-	{
-	  const char *c;
-
-	  for (c = insn->assembler; *c; c++)
-	    {
-	      if (*c == '%')
-		{
-		  switch (*++c)
-		    {
-		    case '%':
-		      func (stream, "%%");
-		      break;
-
-		    case 'a':
-		      print_arm_address (pc, info, given);
-		      break;
-
-		    case 'P':
-		      /* Set P address bit and use normal address
-			 printing routine.  */
-		      print_arm_address (pc, info, given | (1 << 24));
-		      break;
-
-		    case 's':
-                      if ((given & 0x004f0000) == 0x004f0000)
-			{
-                          /* PC relative with immediate offset.  */
-			  int offset = ((given & 0xf00) >> 4) | (given & 0xf);
-
-			  if ((given & 0x00800000) == 0)
-			    offset = -offset;
-
-			  func (stream, "[pc, #%d]\t; ", offset);
-			  info->print_address_func (offset + pc + 8, info);
-			}
-		      else
-			{
-			  func (stream, "[%s",
-				arm_regnames[(given >> 16) & 0xf]);
-			  if ((given & 0x01000000) != 0)
-			    {
-                              /* Pre-indexed.  */
-			      if ((given & 0x00400000) == 0x00400000)
-				{
-                                  /* Immediate.  */
-                                  int offset = ((given & 0xf00) >> 4) | (given & 0xf);
-				  if (offset)
-				    func (stream, ", #%s%d",
-					  (((given & 0x00800000) == 0)
-					   ? "-" : ""), offset);
-				}
-			      else
-				{
-                                  /* Register.  */
-				  func (stream, ", %s%s",
-					(((given & 0x00800000) == 0)
-					 ? "-" : ""),
-                                        arm_regnames[given & 0xf]);
-				}
-
-			      func (stream, "]%s",
-				    ((given & 0x00200000) != 0) ? "!" : "");
-			    }
-			  else
-			    {
-                              /* Post-indexed.  */
-			      if ((given & 0x00400000) == 0x00400000)
-				{
-                                  /* Immediate.  */
-                                  int offset = ((given & 0xf00) >> 4) | (given & 0xf);
-				  if (offset)
-				    func (stream, "], #%s%d",
-					  (((given & 0x00800000) == 0)
-					   ? "-" : ""), offset);
-				  else
-				    func (stream, "]");
-				}
-			      else
-				{
-                                  /* Register.  */
-				  func (stream, "], %s%s",
-					(((given & 0x00800000) == 0)
-					 ? "-" : ""),
-                                        arm_regnames[given & 0xf]);
-				}
-			    }
-			}
-		      break;
-
-		    case 'b':
-		      {
-			int disp = (((given & 0xffffff) ^ 0x800000) - 0x800000);
-			info->print_address_func (disp*4 + pc + 8, info);
-		      }
-		      break;
-
-		    case 'c':
-		      if (((given >> 28) & 0xf) != 0xe)
-			func (stream, "%s",
-			      arm_conditional [(given >> 28) & 0xf]);
-		      break;
-
-		    case 'm':
-		      {
-			int started = 0;
-			int reg;
-
-			func (stream, "{");
-			for (reg = 0; reg < 16; reg++)
-			  if ((given & (1 << reg)) != 0)
-			    {
-			      if (started)
-				func (stream, ", ");
-			      started = 1;
-			      func (stream, "%s", arm_regnames[reg]);
-			    }
-			func (stream, "}");
-		      }
-		      break;
-
-		    case 'q':
-		      arm_decode_shift (given, func, stream, 0);
-		      break;
-
-		    case 'o':
-		      if ((given & 0x02000000) != 0)
-			{
-			  int rotate = (given & 0xf00) >> 7;
-			  int immed = (given & 0xff);
-			  immed = (((immed << (32 - rotate))
-				    | (immed >> rotate)) & 0xffffffff);
-			  func (stream, "#%d\t; 0x%x", immed, immed);
-			}
-		      else
-			arm_decode_shift (given, func, stream, 1);
-		      break;
-
-		    case 'p':
-		      if ((given & 0x0000f000) == 0x0000f000)
-			func (stream, "p");
-		      break;
-
-		    case 't':
-		      if ((given & 0x01200000) == 0x00200000)
-			func (stream, "t");
-		      break;
-
-		    case 'A':
-		      func (stream, "[%s", arm_regnames [(given >> 16) & 0xf]);
-
-		      if ((given & (1 << 24)) != 0)
-			{
-			  int offset = given & 0xff;
-
-			  if (offset)
-			    func (stream, ", #%s%d]%s",
-				  ((given & 0x00800000) == 0 ? "-" : ""),
-				  offset * 4,
-				  ((given & 0x00200000) != 0 ? "!" : ""));
-			  else
-			    func (stream, "]");
-			}
-		      else
-			{
-			  int offset = given & 0xff;
-
-			  func (stream, "]");
-
-			  if (given & (1 << 21))
-			    {
-			      if (offset)
-				func (stream, ", #%s%d",
-				      ((given & 0x00800000) == 0 ? "-" : ""),
-				      offset * 4);
-			    }
-			  else
-			    func (stream, ", {%d}", offset);
-			}
-		      break;
-
-		    case 'B':
-		      /* Print ARM V5 BLX(1) address: pc+25 bits.  */
-		      {
-			bfd_vma address;
-			bfd_vma offset = 0;
-
-			if (given & 0x00800000)
-			  /* Is signed, hi bits should be ones.  */
-			  offset = (-1) ^ 0x00ffffff;
-
-			/* Offset is (SignExtend(offset field)<<2).  */
-			offset += given & 0x00ffffff;
-			offset <<= 2;
-			address = offset + pc + 8;
-
-			if (given & 0x01000000)
-			  /* H bit allows addressing to 2-byte boundaries.  */
-			  address += 2;
-
-		        info->print_address_func (address, info);
-		      }
-		      break;
-
-		    case 'C':
-		      func (stream, "_");
-		      if (given & 0x80000)
-			func (stream, "f");
-		      if (given & 0x40000)
-			func (stream, "s");
-		      if (given & 0x20000)
-			func (stream, "x");
-		      if (given & 0x10000)
-			func (stream, "c");
-		      break;
-
-		    case 'U':
-		      switch (given & 0xf)
-			{
-			case 0xf: func(stream, "sy"); break;
-			case 0x7: func(stream, "un"); break;
-			case 0xe: func(stream, "st"); break;
-			case 0x6: func(stream, "unst"); break;
-			default:
-			  func(stream, "#%d", (int)given & 0xf);
-			  break;
-			}
-		      break;
-
-		    case '0': case '1': case '2': case '3': case '4':
-		    case '5': case '6': case '7': case '8': case '9':
-		      {
-			int width;
-			unsigned long value;
-
-			c = arm_decode_bitfield (c, given, &value, &width);
-
-			switch (*c)
-			  {
-			  case 'r':
-			    func (stream, "%s", arm_regnames[value]);
-			    break;
-			  case 'd':
-			    func (stream, "%ld", value);
-			    break;
-			  case 'b':
-			    func (stream, "%ld", value * 8);
-			    break;
-			  case 'W':
-			    func (stream, "%ld", value + 1);
-			    break;
-			  case 'x':
-			    func (stream, "0x%08lx", value);
-
-			    /* Some SWI instructions have special
-			       meanings.  */
-			    if ((given & 0x0fffffff) == 0x0FF00000)
-			      func (stream, "\t; IMB");
-			    else if ((given & 0x0fffffff) == 0x0FF00001)
-			      func (stream, "\t; IMBRange");
-			    break;
-			  case 'X':
-			    func (stream, "%01lx", value & 0xf);
-			    break;
-			  case '`':
-			    c++;
-			    if (value == 0)
-			      func (stream, "%c", *c);
-			    break;
-			  case '\'':
-			    c++;
-			    if (value == ((1ul << width) - 1))
-			      func (stream, "%c", *c);
-			    break;
-			  case '?':
-			    func (stream, "%c", c[(1 << width) - (int)value]);
-			    c += 1 << width;
-			    break;
-			  default:
-			    abort ();
-			  }
-			break;
-
-		      case 'e':
-			{
-			  int imm;
-
-			  imm = (given & 0xf) | ((given & 0xfff00) >> 4);
-			  func (stream, "%d", imm);
-			}
-			break;
-
-		      case 'E':
-			/* LSB and WIDTH fields of BFI or BFC.  The machine-
-			   language instruction encodes LSB and MSB.  */
-			{
-			  long msb = (given & 0x001f0000) >> 16;
-			  long lsb = (given & 0x00000f80) >> 7;
-
-			  long width = msb - lsb + 1;
-			  if (width > 0)
-			    func (stream, "#%lu, #%lu", lsb, width);
-			  else
-			    func (stream, "(invalid: %lu:%lu)", lsb, msb);
-			}
-			break;
-
-		      case 'V':
-			/* 16-bit unsigned immediate from a MOVT or MOVW
-			   instruction, encoded in bits 0:11 and 15:19.  */
-			{
-			  long hi = (given & 0x000f0000) >> 4;
-			  long lo = (given & 0x00000fff);
-			  long imm16 = hi | lo;
-			  func (stream, "#%lu\t; 0x%lx", imm16, imm16);
-			}
-			break;
-
-		      default:
-			abort ();
-		      }
-		    }
-		}
-	      else
-		func (stream, "%c", *c);
-	    }
-	  return;
-	}
-    }
-  abort ();
-}
-
-/* Print one 16-bit Thumb instruction from PC on INFO->STREAM.  */
-
-static void
-print_insn_thumb16 (bfd_vma pc, struct disassemble_info *info, long given)
-{
-  const struct opcode16 *insn;
-  void *stream = info->stream;
-  fprintf_function func = info->fprintf_func;
-
-  for (insn = thumb_opcodes; insn->assembler; insn++)
-    if ((given & insn->mask) == insn->value)
-      {
-	const char *c = insn->assembler;
-	for (; *c; c++)
-	  {
-	    int domaskpc = 0;
-	    int domasklr = 0;
-
-	    if (*c != '%')
-	      {
-		func (stream, "%c", *c);
-		continue;
-	      }
-
-	    switch (*++c)
-	      {
-	      case '%':
-		func (stream, "%%");
-		break;
-
-	      case 'c':
-		if (ifthen_state)
-		  func (stream, "%s", arm_conditional[IFTHEN_COND]);
-		break;
-
-	      case 'C':
-		if (ifthen_state)
-		  func (stream, "%s", arm_conditional[IFTHEN_COND]);
-		else
-		  func (stream, "s");
-		break;
-
-	      case 'I':
-		{
-		  unsigned int tmp;
-
-		  ifthen_next_state = given & 0xff;
-		  for (tmp = given << 1; tmp & 0xf; tmp <<= 1)
-		    func (stream, ((given ^ tmp) & 0x10) ? "e" : "t");
-		  func (stream, "\t%s", arm_conditional[(given >> 4) & 0xf]);
-		}
-		break;
-
-	      case 'x':
-		if (ifthen_next_state)
-		  func (stream, "\t; unpredictable branch in IT block\n");
-		break;
-
-	      case 'X':
-		if (ifthen_state)
-		  func (stream, "\t; unpredictable <IT:%s>",
-			arm_conditional[IFTHEN_COND]);
-		break;
-
-	      case 'S':
-		{
-		  long reg;
-
-		  reg = (given >> 3) & 0x7;
-		  if (given & (1 << 6))
-		    reg += 8;
-
-		  func (stream, "%s", arm_regnames[reg]);
-		}
-		break;
-
-	      case 'D':
-		{
-		  long reg;
-
-		  reg = given & 0x7;
-		  if (given & (1 << 7))
-		    reg += 8;
-
-		  func (stream, "%s", arm_regnames[reg]);
-		}
-		break;
-
-	      case 'N':
-		if (given & (1 << 8))
-		  domasklr = 1;
-		/* Fall through.  */
-	      case 'O':
-		if (*c == 'O' && (given & (1 << 8)))
-		  domaskpc = 1;
-		/* Fall through.  */
-	      case 'M':
-		{
-		  int started = 0;
-		  int reg;
-
-		  func (stream, "{");
-
-		  /* It would be nice if we could spot
-		     ranges, and generate the rS-rE format: */
-		  for (reg = 0; (reg < 8); reg++)
-		    if ((given & (1 << reg)) != 0)
-		      {
-			if (started)
-			  func (stream, ", ");
-			started = 1;
-			func (stream, "%s", arm_regnames[reg]);
-		      }
-
-		  if (domasklr)
-		    {
-		      if (started)
-			func (stream, ", ");
-		      started = 1;
-		      func (stream, "%s", arm_regnames[14] /* "lr" */);
-		    }
-
-		  if (domaskpc)
-		    {
-		      if (started)
-			func (stream, ", ");
-		      func (stream, "%s", arm_regnames[15] /* "pc" */);
-		    }
-
-		  func (stream, "}");
-		}
-		break;
-
-	      case 'b':
-		/* Print ARM V6T2 CZB address: pc+4+6 bits.  */
-		{
-		  bfd_vma address = (pc + 4
-				     + ((given & 0x00f8) >> 2)
-				     + ((given & 0x0200) >> 3));
-		  info->print_address_func (address, info);
-		}
-		break;
-
-	      case 's':
-		/* Right shift immediate -- bits 6..10; 1-31 print
-		   as themselves, 0 prints as 32.  */
-		{
-		  long imm = (given & 0x07c0) >> 6;
-		  if (imm == 0)
-		    imm = 32;
-		  func (stream, "#%ld", imm);
-		}
-		break;
-
-	      case '0': case '1': case '2': case '3': case '4':
-	      case '5': case '6': case '7': case '8': case '9':
-		{
-		  int bitstart = *c++ - '0';
-		  int bitend = 0;
-
-		  while (*c >= '0' && *c <= '9')
-		    bitstart = (bitstart * 10) + *c++ - '0';
-
-		  switch (*c)
-		    {
-		    case '-':
-		      {
-			long reg;
-
-			c++;
-			while (*c >= '0' && *c <= '9')
-			  bitend = (bitend * 10) + *c++ - '0';
-			if (!bitend)
-			  abort ();
-			reg = given >> bitstart;
-			reg &= (2 << (bitend - bitstart)) - 1;
-			switch (*c)
-			  {
-			  case 'r':
-			    func (stream, "%s", arm_regnames[reg]);
-			    break;
-
-			  case 'd':
-			    func (stream, "%ld", reg);
-			    break;
-
-			  case 'H':
-			    func (stream, "%ld", reg << 1);
-			    break;
-
-			  case 'W':
-			    func (stream, "%ld", reg << 2);
-			    break;
-
-			  case 'a':
-			    /* PC-relative address -- the bottom two
-			       bits of the address are dropped
-			       before the calculation.  */
-			    info->print_address_func
-			      (((pc + 4) & ~3) + (reg << 2), info);
-			    break;
-
-			  case 'x':
-			    func (stream, "0x%04lx", reg);
-			    break;
-
-			  case 'B':
-			    reg = ((reg ^ (1 << bitend)) - (1 << bitend));
-			    info->print_address_func (reg * 2 + pc + 4, info);
-			    break;
-
-			  case 'c':
-			    func (stream, "%s", arm_conditional [reg]);
-			    break;
-
-			  default:
-			    abort ();
-			  }
-		      }
-		      break;
-
-		    case '\'':
-		      c++;
-		      if ((given & (1 << bitstart)) != 0)
-			func (stream, "%c", *c);
-		      break;
-
-		    case '?':
-		      ++c;
-		      if ((given & (1 << bitstart)) != 0)
-			func (stream, "%c", *c++);
-		      else
-			func (stream, "%c", *++c);
-		      break;
-
-		    default:
-		      abort ();
-		    }
-		}
-		break;
-
-	      default:
-		abort ();
-	      }
-	  }
-	return;
-      }
-
-  /* No match.  */
-  abort ();
-}
-
-/* Return the name of an V7M special register.  */
-static const char *
-psr_name (int regno)
-{
-  switch (regno)
-    {
-    case 0: return "APSR";
-    case 1: return "IAPSR";
-    case 2: return "EAPSR";
-    case 3: return "PSR";
-    case 5: return "IPSR";
-    case 6: return "EPSR";
-    case 7: return "IEPSR";
-    case 8: return "MSP";
-    case 9: return "PSP";
-    case 16: return "PRIMASK";
-    case 17: return "BASEPRI";
-    case 18: return "BASEPRI_MASK";
-    case 19: return "FAULTMASK";
-    case 20: return "CONTROL";
-    default: return "<unknown>";
-    }
-}
-
-/* Print one 32-bit Thumb instruction from PC on INFO->STREAM.  */
-
-static void
-print_insn_thumb32 (bfd_vma pc, struct disassemble_info *info, long given)
-{
-  const struct opcode32 *insn;
-  void *stream = info->stream;
-  fprintf_function func = info->fprintf_func;
-
-  if (print_insn_coprocessor (pc, info, given, true))
-    return;
-
-  if (print_insn_neon (info, given, true))
-    return;
-
-  for (insn = thumb32_opcodes; insn->assembler; insn++)
-    if ((given & insn->mask) == insn->value)
-      {
-	const char *c = insn->assembler;
-	for (; *c; c++)
-	  {
-	    if (*c != '%')
-	      {
-		func (stream, "%c", *c);
-		continue;
-	      }
-
-	    switch (*++c)
-	      {
-	      case '%':
-		func (stream, "%%");
-		break;
-
-	      case 'c':
-		if (ifthen_state)
-		  func (stream, "%s", arm_conditional[IFTHEN_COND]);
-		break;
-
-	      case 'x':
-		if (ifthen_next_state)
-		  func (stream, "\t; unpredictable branch in IT block\n");
-		break;
-
-	      case 'X':
-		if (ifthen_state)
-		  func (stream, "\t; unpredictable <IT:%s>",
-			arm_conditional[IFTHEN_COND]);
-		break;
-
-	      case 'I':
-		{
-		  unsigned int imm12 = 0;
-		  imm12 |= (given & 0x000000ffu);
-		  imm12 |= (given & 0x00007000u) >> 4;
-		  imm12 |= (given & 0x04000000u) >> 15;
-		  func (stream, "#%u\t; 0x%x", imm12, imm12);
-		}
-		break;
-
-	      case 'M':
-		{
-		  unsigned int bits = 0, imm, imm8, mod;
-		  bits |= (given & 0x000000ffu);
-		  bits |= (given & 0x00007000u) >> 4;
-		  bits |= (given & 0x04000000u) >> 15;
-		  imm8 = (bits & 0x0ff);
-		  mod = (bits & 0xf00) >> 8;
-		  switch (mod)
-		    {
-		    case 0: imm = imm8; break;
-		    case 1: imm = ((imm8<<16) | imm8); break;
-		    case 2: imm = ((imm8<<24) | (imm8 << 8)); break;
-		    case 3: imm = ((imm8<<24) | (imm8 << 16) | (imm8 << 8) | imm8); break;
-		    default:
-		      mod  = (bits & 0xf80) >> 7;
-		      imm8 = (bits & 0x07f) | 0x80;
-		      imm  = (((imm8 << (32 - mod)) | (imm8 >> mod)) & 0xffffffff);
-		    }
-		  func (stream, "#%u\t; 0x%x", imm, imm);
-		}
-		break;
-
-	      case 'J':
-		{
-		  unsigned int imm = 0;
-		  imm |= (given & 0x000000ffu);
-		  imm |= (given & 0x00007000u) >> 4;
-		  imm |= (given & 0x04000000u) >> 15;
-		  imm |= (given & 0x000f0000u) >> 4;
-		  func (stream, "#%u\t; 0x%x", imm, imm);
-		}
-		break;
-
-	      case 'K':
-		{
-		  unsigned int imm = 0;
-		  imm |= (given & 0x000f0000u) >> 16;
-		  imm |= (given & 0x00000ff0u) >> 0;
-		  imm |= (given & 0x0000000fu) << 12;
-		  func (stream, "#%u\t; 0x%x", imm, imm);
-		}
-		break;
-
-	      case 'S':
-		{
-		  unsigned int reg = (given & 0x0000000fu);
-		  unsigned int stp = (given & 0x00000030u) >> 4;
-		  unsigned int imm = 0;
-		  imm |= (given & 0x000000c0u) >> 6;
-		  imm |= (given & 0x00007000u) >> 10;
-
-		  func (stream, "%s", arm_regnames[reg]);
-		  switch (stp)
-		    {
-		    case 0:
-		      if (imm > 0)
-			func (stream, ", lsl #%u", imm);
-		      break;
-
-		    case 1:
-		      if (imm == 0)
-			imm = 32;
-		      func (stream, ", lsr #%u", imm);
-		      break;
-
-		    case 2:
-		      if (imm == 0)
-			imm = 32;
-		      func (stream, ", asr #%u", imm);
-		      break;
-
-		    case 3:
-		      if (imm == 0)
-			func (stream, ", rrx");
-		      else
-			func (stream, ", ror #%u", imm);
-		    }
-		}
-		break;
-
-	      case 'a':
-		{
-		  unsigned int Rn  = (given & 0x000f0000) >> 16;
-		  unsigned int U   = (given & 0x00800000) >> 23;
-		  unsigned int op  = (given & 0x00000f00) >> 8;
-		  unsigned int i12 = (given & 0x00000fff);
-		  unsigned int i8  = (given & 0x000000ff);
-		  bfd_boolean writeback = false, postind = false;
-		  int offset = 0;
-
-		  func (stream, "[%s", arm_regnames[Rn]);
-		  if (U) /* 12-bit positive immediate offset */
-		    offset = i12;
-		  else if (Rn == 15) /* 12-bit negative immediate offset */
-		    offset = -(int)i12;
-		  else if (op == 0x0) /* shifted register offset */
-		    {
-		      unsigned int Rm = (i8 & 0x0f);
-		      unsigned int sh = (i8 & 0x30) >> 4;
-		      func (stream, ", %s", arm_regnames[Rm]);
-		      if (sh)
-			func (stream, ", lsl #%u", sh);
-		      func (stream, "]");
-		      break;
-		    }
-		  else switch (op)
-		    {
-		    case 0xE:  /* 8-bit positive immediate offset */
-		      offset = i8;
-		      break;
-
-		    case 0xC:  /* 8-bit negative immediate offset */
-		      offset = -i8;
-		      break;
-
-		    case 0xF:  /* 8-bit + preindex with wb */
-		      offset = i8;
-		      writeback = true;
-		      break;
-
-		    case 0xD:  /* 8-bit - preindex with wb */
-		      offset = -i8;
-		      writeback = true;
-		      break;
-
-		    case 0xB:  /* 8-bit + postindex */
-		      offset = i8;
-		      postind = true;
-		      break;
-
-		    case 0x9:  /* 8-bit - postindex */
-		      offset = -i8;
-		      postind = true;
-		      break;
-
-		    default:
-		      func (stream, ", <undefined>]");
-		      goto skip;
-		    }
-
-		  if (postind)
-		    func (stream, "], #%d", offset);
-		  else
-		    {
-		      if (offset)
-			func (stream, ", #%d", offset);
-		      func (stream, writeback ? "]!" : "]");
-		    }
-
-		  if (Rn == 15)
-		    {
-		      func (stream, "\t; ");
-		      info->print_address_func (((pc + 4) & ~3) + offset, info);
-		    }
-		}
-	      skip:
-		break;
-
-	      case 'A':
-		{
-		  unsigned int P   = (given & 0x01000000) >> 24;
-		  unsigned int U   = (given & 0x00800000) >> 23;
-		  unsigned int W   = (given & 0x00400000) >> 21;
-		  unsigned int Rn  = (given & 0x000f0000) >> 16;
-		  unsigned int off = (given & 0x000000ff);
-
-		  func (stream, "[%s", arm_regnames[Rn]);
-		  if (P)
-		    {
-		      if (off || !U)
-			func (stream, ", #%c%u", U ? '+' : '-', off * 4);
-		      func (stream, "]");
-		      if (W)
-			func (stream, "!");
-		    }
-		  else
-		    {
-		      func (stream, "], ");
-		      if (W)
-			func (stream, "#%c%u", U ? '+' : '-', off * 4);
-		      else
-			func (stream, "{%u}", off);
-		    }
-		}
-		break;
-
-	      case 'w':
-		{
-		  unsigned int Sbit = (given & 0x01000000) >> 24;
-		  unsigned int type = (given & 0x00600000) >> 21;
-		  switch (type)
-		    {
-		    case 0: func (stream, Sbit ? "sb" : "b"); break;
-		    case 1: func (stream, Sbit ? "sh" : "h"); break;
-		    case 2:
-		      if (Sbit)
-			func (stream, "??");
-		      break;
-		    case 3:
-		      func (stream, "??");
-		      break;
-		    }
-		}
-		break;
-
-	      case 'm':
-		{
-		  int started = 0;
-		  int reg;
-
-		  func (stream, "{");
-		  for (reg = 0; reg < 16; reg++)
-		    if ((given & (1 << reg)) != 0)
-		      {
-			if (started)
-			  func (stream, ", ");
-			started = 1;
-			func (stream, "%s", arm_regnames[reg]);
-		      }
-		  func (stream, "}");
-		}
-		break;
-
-	      case 'E':
-		{
-		  unsigned int msb = (given & 0x0000001f);
-		  unsigned int lsb = 0;
-		  lsb |= (given & 0x000000c0u) >> 6;
-		  lsb |= (given & 0x00007000u) >> 10;
-		  func (stream, "#%u, #%u", lsb, msb - lsb + 1);
-		}
-		break;
-
-	      case 'F':
-		{
-		  unsigned int width = (given & 0x0000001f) + 1;
-		  unsigned int lsb = 0;
-		  lsb |= (given & 0x000000c0u) >> 6;
-		  lsb |= (given & 0x00007000u) >> 10;
-		  func (stream, "#%u, #%u", lsb, width);
-		}
-		break;
-
-	      case 'b':
-		{
-		  unsigned int S = (given & 0x04000000u) >> 26;
-		  unsigned int J1 = (given & 0x00002000u) >> 13;
-		  unsigned int J2 = (given & 0x00000800u) >> 11;
-		  int offset = 0;
-
-		  offset |= !S << 20;
-		  offset |= J2 << 19;
-		  offset |= J1 << 18;
-		  offset |= (given & 0x003f0000) >> 4;
-		  offset |= (given & 0x000007ff) << 1;
-		  offset -= (1 << 20);
-
-		  info->print_address_func (pc + 4 + offset, info);
-		}
-		break;
-
-	      case 'B':
-		{
-		  unsigned int S = (given & 0x04000000u) >> 26;
-		  unsigned int I1 = (given & 0x00002000u) >> 13;
-		  unsigned int I2 = (given & 0x00000800u) >> 11;
-		  int offset = 0;
-
-		  offset |= !S << 24;
-		  offset |= !(I1 ^ S) << 23;
-		  offset |= !(I2 ^ S) << 22;
-		  offset |= (given & 0x03ff0000u) >> 4;
-		  offset |= (given & 0x000007ffu) << 1;
-		  offset -= (1 << 24);
-		  offset += pc + 4;
-
-		  /* BLX target addresses are always word aligned.  */
-		  if ((given & 0x00001000u) == 0)
-		      offset &= ~2u;
-
-		  info->print_address_func (offset, info);
-		}
-		break;
-
-	      case 's':
-		{
-		  unsigned int shift = 0;
-		  shift |= (given & 0x000000c0u) >> 6;
-		  shift |= (given & 0x00007000u) >> 10;
-		  if (given & 0x00200000u)
-		    func (stream, ", asr #%u", shift);
-		  else if (shift)
-		    func (stream, ", lsl #%u", shift);
-		  /* else print nothing - lsl #0 */
-		}
-		break;
-
-	      case 'R':
-		{
-		  unsigned int rot = (given & 0x00000030) >> 4;
-		  if (rot)
-		    func (stream, ", ror #%u", rot * 8);
-		}
-		break;
-
-	      case 'U':
-		switch (given & 0xf)
-		  {
-		  case 0xf: func(stream, "sy"); break;
-		  case 0x7: func(stream, "un"); break;
-		  case 0xe: func(stream, "st"); break;
-		  case 0x6: func(stream, "unst"); break;
-		  default:
-		    func(stream, "#%d", (int)given & 0xf);
-		    break;
-		  }
-		break;
-
-	      case 'C':
-		if ((given & 0xff) == 0)
-		  {
-		    func (stream, "%cPSR_", (given & 0x100000) ? 'S' : 'C');
-		    if (given & 0x800)
-		      func (stream, "f");
-		    if (given & 0x400)
-		      func (stream, "s");
-		    if (given & 0x200)
-		      func (stream, "x");
-		    if (given & 0x100)
-		      func (stream, "c");
-		  }
-		else
-		  {
-		    func (stream, "%s", psr_name (given & 0xff));
-		  }
-		break;
-
-	      case 'D':
-		if ((given & 0xff) == 0)
-		  func (stream, "%cPSR", (given & 0x100000) ? 'S' : 'C');
-		else
-		  func (stream, "%s", psr_name (given & 0xff));
-		break;
-
-	      case '0': case '1': case '2': case '3': case '4':
-	      case '5': case '6': case '7': case '8': case '9':
-		{
-		  int width;
-		  unsigned long val;
-
-		  c = arm_decode_bitfield (c, given, &val, &width);
-
-		  switch (*c)
-		    {
-		    case 'd': func (stream, "%lu", val); break;
-		    case 'W': func (stream, "%lu", val * 4); break;
-		    case 'r': func (stream, "%s", arm_regnames[val]); break;
-
-		    case 'c':
-		      func (stream, "%s", arm_conditional[val]);
-		      break;
-
-		    case '\'':
-		      c++;
-		      if (val == ((1ul << width) - 1))
-			func (stream, "%c", *c);
-		      break;
-
-		    case '`':
-		      c++;
-		      if (val == 0)
-			func (stream, "%c", *c);
-		      break;
-
-		    case '?':
-		      func (stream, "%c", c[(1 << width) - (int)val]);
-		      c += 1 << width;
-		      break;
-
-		    default:
-		      abort ();
-		    }
-		}
-		break;
-
-	      default:
-		abort ();
-	      }
-	  }
-	return;
-      }
-
-  /* No match.  */
-  abort ();
-}
-
-/* Print data bytes on INFO->STREAM.  */
-
-static void
-print_insn_data (bfd_vma pc ATTRIBUTE_UNUSED, struct disassemble_info *info,
-		 long given)
-{
-  switch (info->bytes_per_chunk)
-    {
-    case 1:
-      info->fprintf_func (info->stream, ".byte\t0x%02lx", given);
-      break;
-    case 2:
-      info->fprintf_func (info->stream, ".short\t0x%04lx", given);
-      break;
-    case 4:
-      info->fprintf_func (info->stream, ".word\t0x%08lx", given);
-      break;
-    default:
-      abort ();
-    }
-}
-
-/* Search back through the insn stream to determine if this instruction is
-   conditionally executed.  */
-static void
-find_ifthen_state (bfd_vma pc, struct disassemble_info *info,
-		   bfd_boolean little)
-{
-  unsigned char b[2];
-  unsigned int insn;
-  int status;
-  /* COUNT is twice the number of instructions seen.  It will be odd if we
-     just crossed an instruction boundary.  */
-  int count;
-  int it_count;
-  unsigned int seen_it;
-  bfd_vma addr;
-
-  ifthen_address = pc;
-  ifthen_state = 0;
-
-  addr = pc;
-  count = 1;
-  it_count = 0;
-  seen_it = 0;
-  /* Scan backwards looking for IT instructions, keeping track of where
-     instruction boundaries are.  We don't know if something is actually an
-     IT instruction until we find a definite instruction boundary.  */
-  for (;;)
-    {
-      if (addr == 0 || info->symbol_at_address_func(addr, info))
-	{
-	  /* A symbol must be on an instruction boundary, and will not
-	     be within an IT block.  */
-	  if (seen_it && (count & 1))
-	    break;
-
-	  return;
-	}
-      addr -= 2;
-      status = arm_read_memory (addr, (bfd_byte *)b, 2, info);
-      if (status)
-	return;
-
-      if (little)
-	insn = (b[0]) | (b[1] << 8);
-      else
-	insn = (b[1]) | (b[0] << 8);
-      if (seen_it)
-	{
-	  if ((insn & 0xf800) < 0xe800)
-	    {
-	      /* Addr + 2 is an instruction boundary.  See if this matches
-	         the expected boundary based on the position of the last
-		 IT candidate.  */
-	      if (count & 1)
-		break;
-	      seen_it = 0;
-	    }
-	}
-      if ((insn & 0xff00) == 0xbf00 && (insn & 0xf) != 0)
-	{
-	  /* This could be an IT instruction.  */
-	  seen_it = insn;
-	  it_count = count >> 1;
-	}
-      if ((insn & 0xf800) >= 0xe800)
-	count++;
-      else
-	count = (count + 2) | 1;
-      /* IT blocks contain at most 4 instructions.  */
-      if (count >= 8 && !seen_it)
-	return;
-    }
-  /* We found an IT instruction.  */
-  ifthen_state = (seen_it & 0xe0) | ((seen_it << it_count) & 0x1f);
-  if ((ifthen_state & 0xf) == 0)
-    ifthen_state = 0;
-}
-
-/* NOTE: There are no checks in these routines that
-   the relevant number of data bytes exist.  */
-
-int
-print_insn_arm (bfd_vma pc, struct disassemble_info *info)
-{
-  unsigned char b[4];
-  long		given;
-  int           status;
-  int           is_thumb = false;
-  int           is_data = false;
-  unsigned int	size = 4;
-  void	 	(*printer) (bfd_vma, struct disassemble_info *, long);
-  int little;
-
-  little = (info->endian == BFD_ENDIAN_LITTLE);
-  is_thumb |= (pc & 1);
-  pc &= ~(bfd_vma)1;
-
-  if (force_thumb)
-    is_thumb = true;
-
-  info->bytes_per_line = 4;
-
-  if (is_data)
-    {
-      int i;
-
-      /* size was already set above.  */
-      info->bytes_per_chunk = size;
-      printer = print_insn_data;
-
-      status = arm_read_memory (pc, (bfd_byte *)b, size, info);
-      given = 0;
-      if (little)
-	for (i = size - 1; i >= 0; i--)
-	  given = b[i] | (given << 8);
-      else
-	for (i = 0; i < (int) size; i++)
-	  given = b[i] | (given << 8);
-    }
-  else if (!is_thumb)
-    {
-      /* In ARM mode endianness is a straightforward issue: the instruction
-	 is four bytes long and is either ordered 0123 or 3210.  */
-      printer = print_insn_arm_internal;
-      info->bytes_per_chunk = 4;
-      size = 4;
-
-      status = arm_read_memory (pc, (bfd_byte *)b, 4, info);
-      if (little)
-	given = (b[0]) | (b[1] << 8) | (b[2] << 16) | ((unsigned)b[3] << 24);
-      else
-	given = (b[3]) | (b[2] << 8) | (b[1] << 16) | ((unsigned)b[0] << 24);
-    }
-  else
-    {
-      /* In Thumb mode we have the additional wrinkle of two
-	 instruction lengths.  Fortunately, the bits that determine
-	 the length of the current instruction are always to be found
-	 in the first two bytes.  */
-      printer = print_insn_thumb16;
-      info->bytes_per_chunk = 2;
-      size = 2;
-
-      status = arm_read_memory (pc, (bfd_byte *)b, 2, info);
-      if (little)
-	given = (b[0]) | (b[1] << 8);
-      else
-	given = (b[1]) | (b[0] << 8);
-
-      if (!status)
-	{
-	  /* These bit patterns signal a four-byte Thumb
-	     instruction.  */
-	  if ((given & 0xF800) == 0xF800
-	      || (given & 0xF800) == 0xF000
-	      || (given & 0xF800) == 0xE800)
-	    {
-	      status = arm_read_memory (pc + 2, (bfd_byte *)b, 2, info);
-	      if (little)
-		given = (b[0]) | (b[1] << 8) | (given << 16);
-	      else
-		given = (b[1]) | (b[0] << 8) | (given << 16);
-
-	      printer = print_insn_thumb32;
-	      size = 4;
-	    }
-	}
-
-      if (ifthen_address != pc)
-	find_ifthen_state(pc, info, little);
-
-      if (ifthen_state)
-	{
-	  if ((ifthen_state & 0xf) == 0x8)
-	    ifthen_next_state = 0;
-	  else
-	    ifthen_next_state = (ifthen_state & 0xe0)
-				| ((ifthen_state & 0xf) << 1);
-	}
-    }
-
-  if (status)
-    {
-      info->memory_error_func (status, pc, info);
-      return -1;
-    }
-  if (info->flags & INSN_HAS_RELOC)
-    /* If the instruction has a reloc associated with it, then
-       the offset field in the instruction will actually be the
-       addend for the reloc.  (We are using REL type relocs).
-       In such cases, we can ignore the pc when computing
-       addresses, since the addend is not currently pc-relative.  */
-    pc = 0;
-
-  /* We include the hexdump of the instruction. The format here
-     matches that used by objdump and the ARM ARM (in particular,
-     32 bit Thumb instructions are displayed as pairs of halfwords,
-     not as a single word.)  */
-  if (is_thumb)
-    {
-      if (size == 2)
-	{
-	  info->fprintf_func(info->stream, "%04lx       ",
-			     ((unsigned long)given) & 0xffff);
-	}
-      else
-	{
-	  info->fprintf_func(info->stream, "%04lx %04lx  ",
-			     (((unsigned long)given) >> 16) & 0xffff,
-			     ((unsigned long)given) & 0xffff);
-	}
-    }
-  else
-    {
-      info->fprintf_func(info->stream, "%08lx      ",
-			 ((unsigned long)given) & 0xffffffff);
-    }
-
-  printer (pc, info, given);
-
-  if (is_thumb)
-    {
-      ifthen_state = ifthen_next_state;
-      ifthen_address += size;
-    }
-  return size;
-}
diff --git a/disas/meson.build b/disas/meson.build
index e3e1b67923..a6402e7e1c 100644
--- a/disas/meson.build
+++ b/disas/meson.build
@@ -4,7 +4,6 @@ subdir('libvixl')
 common_ss.add(when: 'CONFIG_ALPHA_DIS', if_true: files('alpha.c'))
 common_ss.add(when: 'CONFIG_ARM_A64_DIS', if_true: files('arm-a64.cc'))
 common_ss.add_all(when: 'CONFIG_ARM_A64_DIS', if_true: libvixl_ss)
-common_ss.add(when: 'CONFIG_ARM_DIS', if_true: files('arm.c'))
 common_ss.add(when: 'CONFIG_CRIS_DIS', if_true: files('cris.c'))
 common_ss.add(when: 'CONFIG_HEXAGON_DIS', if_true: files('hexagon.c'))
 common_ss.add(when: 'CONFIG_HPPA_DIS', if_true: files('hppa.c'))
diff --git a/include/disas/dis-asm.h b/include/disas/dis-asm.h
index aebd23dc20..e857f0ae8b 100644
--- a/include/disas/dis-asm.h
+++ b/include/disas/dis-asm.h
@@ -429,7 +429,6 @@ int print_insn_h8500            (bfd_vma, disassemble_info*);
 int print_insn_arm_a64          (bfd_vma, disassemble_info*);
 int print_insn_alpha            (bfd_vma, disassemble_info*);
 disassembler_ftype arc_get_disassembler (int, int);
-int print_insn_arm              (bfd_vma, disassemble_info*);
 int print_insn_sparc            (bfd_vma, disassemble_info*);
 int print_insn_big_a29k         (bfd_vma, disassemble_info*);
 int print_insn_little_a29k      (bfd_vma, disassemble_info*);
diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index 01176b2569..16fb5f9c12 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -775,12 +775,6 @@ static bool arm_cpu_virtio_is_big_endian(CPUState *cs)
 
 #endif
 
-static int
-print_insn_thumb1(bfd_vma pc, disassemble_info *info)
-{
-  return print_insn_arm(pc | 1, info);
-}
-
 static void arm_disas_set_info(CPUState *cpu, disassemble_info *info)
 {
     ARMCPU *ac = ARM_CPU(cpu);
@@ -801,12 +795,10 @@ static void arm_disas_set_info(CPUState *cpu, disassemble_info *info)
     } else {
         int cap_mode;
         if (env->thumb) {
-            info->print_insn = print_insn_thumb1;
             info->cap_insn_unit = 2;
             info->cap_insn_split = 4;
             cap_mode = CS_MODE_THUMB;
         } else {
-            info->print_insn = print_insn_arm;
             info->cap_insn_unit = 4;
             info->cap_insn_split = 4;
             cap_mode = CS_MODE_ARM;

From 457248a54c9d3dbc3c7f4dc8e184f07ca858640c Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Tue, 12 Apr 2022 18:58:36 +0200
Subject: [PATCH 176/935] disas: Remove old libopcode i386 disassembler

Capstone should be superior to the old libopcode disassembler,
so we can drop the old file nowadays.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220412165836.355850-4-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 MAINTAINERS             |    2 -
 disas.c                 |    2 -
 disas/i386.c            | 6771 ---------------------------------------
 disas/meson.build       |    1 -
 include/disas/dis-asm.h |    1 -
 target/i386/cpu.c       |    1 -
 6 files changed, 6778 deletions(-)
 delete mode 100644 disas/i386.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 7f3788b694..85d593a41f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -334,7 +334,6 @@ F: target/i386/tcg/
 F: tests/tcg/i386/
 F: tests/tcg/x86_64/
 F: hw/i386/
-F: disas/i386.c
 F: docs/system/cpu-models-x86*
 T: git https://gitlab.com/ehabkost/qemu.git x86-next
 
@@ -3275,7 +3274,6 @@ i386 TCG target
 M: Richard Henderson <richard.henderson@linaro.org>
 S: Maintained
 F: tcg/i386/
-F: disas/i386.c
 
 LoongArch64 TCG target
 M: WANG Xuerui <git@xen0n.name>
diff --git a/disas.c b/disas.c
index 4e443b4d49..b6f4dc73b2 100644
--- a/disas.c
+++ b/disas.c
@@ -153,14 +153,12 @@ static void initialize_debug_host(CPUDebug *s)
     s->info.print_insn = print_insn_tci;
 #elif defined(__i386__)
     s->info.mach = bfd_mach_i386_i386;
-    s->info.print_insn = print_insn_i386;
     s->info.cap_arch = CS_ARCH_X86;
     s->info.cap_mode = CS_MODE_32;
     s->info.cap_insn_unit = 1;
     s->info.cap_insn_split = 8;
 #elif defined(__x86_64__)
     s->info.mach = bfd_mach_x86_64;
-    s->info.print_insn = print_insn_i386;
     s->info.cap_arch = CS_ARCH_X86;
     s->info.cap_mode = CS_MODE_64;
     s->info.cap_insn_unit = 1;
diff --git a/disas/i386.c b/disas/i386.c
deleted file mode 100644
index 06c835236e..0000000000
--- a/disas/i386.c
+++ /dev/null
@@ -1,6771 +0,0 @@
-/* opcodes/i386-dis.c r1.126 */
-/* Print i386 instructions for GDB, the GNU debugger.
-   Copyright 1988, 1989, 1991, 1993, 1994, 1995, 1996, 1997, 1998, 1999,
-   2001, 2002, 2003, 2004, 2005, 2006 Free Software Foundation, Inc.
-
-   This file is part of GDB.
-
-   This program is free software; you can redistribute it and/or modify
-   it under the terms of the GNU General Public License as published by
-   the Free Software Foundation; either version 2 of the License, or
-   (at your option) any later version.
-
-   This program is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-   GNU General Public License for more details.
-
-   You should have received a copy of the GNU General Public License
-   along with this program; if not, see <http://www.gnu.org/licenses/>. */
-
-/* 80386 instruction printer by Pace Willisson (pace@prep.ai.mit.edu)
-   July 1988
-    modified by John Hassey (hassey@dg-rtp.dg.com)
-    x86-64 support added by Jan Hubicka (jh@suse.cz)
-    VIA PadLock support by Michal Ludvig (mludvig@suse.cz).  */
-
-/* The main tables describing the instructions is essentially a copy
-   of the "Opcode Map" chapter (Appendix A) of the Intel 80386
-   Programmers Manual.  Usually, there is a capital letter, followed
-   by a small letter.  The capital letter tell the addressing mode,
-   and the small letter tells about the operand size.  Refer to
-   the Intel manual for details.  */
-
-#include "qemu/osdep.h"
-#include "disas/dis-asm.h"
-#include "qemu/cutils.h"
-
-/* include/opcode/i386.h r1.78 */
-
-/* opcode/i386.h -- Intel 80386 opcode macros
-   Copyright 1989, 1991, 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999,
-   2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007
-   Free Software Foundation, Inc.
-
-   This file is part of GAS, the GNU Assembler, and GDB, the GNU Debugger.
-
-   This program is free software; you can redistribute it and/or modify
-   it under the terms of the GNU General Public License as published by
-   the Free Software Foundation; either version 2 of the License, or
-   (at your option) any later version.
-
-   This program is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-   GNU General Public License for more details.
-
-   You should have received a copy of the GNU General Public License
-   along with this program; if not, see <http://www.gnu.org/licenses/>.  */
-
-/* The SystemV/386 SVR3.2 assembler, and probably all AT&T derived
-   ix86 Unix assemblers, generate floating point instructions with
-   reversed source and destination registers in certain cases.
-   Unfortunately, gcc and possibly many other programs use this
-   reversed syntax, so we're stuck with it.
-
-   eg. `fsub %st(3),%st' results in st = st - st(3) as expected, but
-   `fsub %st,%st(3)' results in st(3) = st - st(3), rather than
-   the expected st(3) = st(3) - st
-
-   This happens with all the non-commutative arithmetic floating point
-   operations with two register operands, where the source register is
-   %st, and destination register is %st(i).
-
-   The affected opcode map is dceX, dcfX, deeX, defX.  */
-
-#ifndef SYSV386_COMPAT
-/* Set non-zero for broken, compatible instructions.  Set to zero for
-   non-broken opcodes at your peril.  gcc generates SystemV/386
-   compatible instructions.  */
-#define SYSV386_COMPAT 1
-#endif
-#ifndef OLDGCC_COMPAT
-/* Set non-zero to cater for old (<= 2.8.1) versions of gcc that could
-   generate nonsense fsubp, fsubrp, fdivp and fdivrp with operands
-   reversed.  */
-#define OLDGCC_COMPAT SYSV386_COMPAT
-#endif
-
-#define MOV_AX_DISP32 0xa0
-#define POP_SEG_SHORT 0x07
-#define JUMP_PC_RELATIVE 0xeb
-#define INT_OPCODE  0xcd
-#define INT3_OPCODE 0xcc
-/* The opcode for the fwait instruction, which disassembler treats as a
-   prefix when it can.  */
-#define FWAIT_OPCODE 0x9b
-#define ADDR_PREFIX_OPCODE 0x67
-#define DATA_PREFIX_OPCODE 0x66
-#define LOCK_PREFIX_OPCODE 0xf0
-#define CS_PREFIX_OPCODE 0x2e
-#define DS_PREFIX_OPCODE 0x3e
-#define ES_PREFIX_OPCODE 0x26
-#define FS_PREFIX_OPCODE 0x64
-#define GS_PREFIX_OPCODE 0x65
-#define SS_PREFIX_OPCODE 0x36
-#define REPNE_PREFIX_OPCODE 0xf2
-#define REPE_PREFIX_OPCODE  0xf3
-
-#define TWO_BYTE_OPCODE_ESCAPE 0x0f
-#define NOP_OPCODE (char) 0x90
-
-/* register numbers */
-#define EBP_REG_NUM 5
-#define ESP_REG_NUM 4
-
-/* modrm_byte.regmem for twobyte escape */
-#define ESCAPE_TO_TWO_BYTE_ADDRESSING ESP_REG_NUM
-/* index_base_byte.index for no index register addressing */
-#define NO_INDEX_REGISTER ESP_REG_NUM
-/* index_base_byte.base for no base register addressing */
-#define NO_BASE_REGISTER EBP_REG_NUM
-#define NO_BASE_REGISTER_16 6
-
-/* modrm.mode = REGMEM_FIELD_HAS_REG when a register is in there */
-#define REGMEM_FIELD_HAS_REG 0x3/* always = 0x3 */
-#define REGMEM_FIELD_HAS_MEM (~REGMEM_FIELD_HAS_REG)
-
-/* x86-64 extension prefix.  */
-#define REX_OPCODE	0x40
-
-/* Indicates 64 bit operand size.  */
-#define REX_W	8
-/* High extension to reg field of modrm byte.  */
-#define REX_R	4
-/* High extension to SIB index field.  */
-#define REX_X	2
-/* High extension to base field of modrm or SIB, or reg field of opcode.  */
-#define REX_B	1
-
-/* max operands per insn */
-#define MAX_OPERANDS 4
-
-/* max immediates per insn (lcall, ljmp, insertq, extrq) */
-#define MAX_IMMEDIATE_OPERANDS 2
-
-/* max memory refs per insn (string ops) */
-#define MAX_MEMORY_OPERANDS 2
-
-/* max size of insn mnemonics.  */
-#define MAX_MNEM_SIZE 16
-
-/* max size of register name in insn mnemonics.  */
-#define MAX_REG_NAME_SIZE 8
-
-/* opcodes/i386-dis.c r1.126 */
-
-static int fetch_data2(struct disassemble_info *, bfd_byte *);
-static int fetch_data(struct disassemble_info *, bfd_byte *);
-static void ckprefix (void);
-static const char *prefix_name (int, int);
-static int print_insn (bfd_vma, disassemble_info *);
-static void dofloat (int);
-static void OP_ST (int, int);
-static void OP_STi (int, int);
-static int putop (const char *, int);
-static void oappend (const char *);
-static void append_seg (void);
-static void OP_indirE (int, int);
-static void print_operand_value (char *buf, size_t bufsize, int hex, bfd_vma disp);
-static void print_displacement (char *, bfd_vma);
-static void OP_E (int, int);
-static void OP_G (int, int);
-static void OP_vvvv (int, int);
-static bfd_vma get64 (void);
-static bfd_signed_vma get32 (void);
-static bfd_signed_vma get32s (void);
-static int get16 (void);
-static void set_op (bfd_vma, int);
-static void OP_REG (int, int);
-static void OP_IMREG (int, int);
-static void OP_I (int, int);
-static void OP_I64 (int, int);
-static void OP_sI (int, int);
-static void OP_J (int, int);
-static void OP_SEG (int, int);
-static void OP_DIR (int, int);
-static void OP_OFF (int, int);
-static void OP_OFF64 (int, int);
-static void ptr_reg (int, int);
-static void OP_ESreg (int, int);
-static void OP_DSreg (int, int);
-static void OP_C (int, int);
-static void OP_D (int, int);
-static void OP_T (int, int);
-static void OP_R (int, int);
-static void OP_MMX (int, int);
-static void OP_XMM (int, int);
-static void OP_EM (int, int);
-static void OP_EX (int, int);
-static void OP_EMC (int,int);
-static void OP_MXC (int,int);
-static void OP_MS (int, int);
-static void OP_XS (int, int);
-static void OP_M (int, int);
-static void OP_VMX (int, int);
-static void OP_0fae (int, int);
-static void OP_0f07 (int, int);
-static void NOP_Fixup1 (int, int);
-static void NOP_Fixup2 (int, int);
-static void OP_3DNowSuffix (int, int);
-static void OP_SIMD_Suffix (int, int);
-static void SIMD_Fixup (int, int);
-static void PNI_Fixup (int, int);
-static void SVME_Fixup (int, int);
-static void INVLPG_Fixup (int, int);
-static void BadOp (void);
-static void VMX_Fixup (int, int);
-static void REP_Fixup (int, int);
-static void CMPXCHG8B_Fixup (int, int);
-static void XMM_Fixup (int, int);
-static void CRC32_Fixup (int, int);
-
-struct dis_private {
-  /* Points to first byte not fetched.  */
-  bfd_byte *max_fetched;
-  bfd_byte the_buffer[MAX_MNEM_SIZE];
-  bfd_vma insn_start;
-  int orig_sizeflag;
-  sigjmp_buf bailout;
-};
-
-enum address_mode
-{
-  mode_16bit,
-  mode_32bit,
-  mode_64bit
-};
-
-static enum address_mode address_mode;
-
-/* Flags for the prefixes for the current instruction.  See below.  */
-static int prefixes;
-
-/* REX prefix the current instruction.  See below.  */
-static int rex;
-/* Bits of REX we've already used.  */
-static int rex_used;
-/* Mark parts used in the REX prefix.  When we are testing for
-   empty prefix (for 8bit register REX extension), just mask it
-   out.  Otherwise test for REX bit is excuse for existence of REX
-   only in case value is nonzero.  */
-#define USED_REX(value)					\
-  {							\
-    if (value)						\
-      {							\
-	if ((rex & value))				\
-	  rex_used |= (value) | REX_OPCODE;		\
-      }							\
-    else						\
-      rex_used |= REX_OPCODE;				\
-  }
-
-/* Flags for prefixes which we somehow handled when printing the
-   current instruction.  */
-static int used_prefixes;
-
-/* The VEX.vvvv register, unencoded.  */
-static int vex_reg;
-
-/* Flags stored in PREFIXES.  */
-#define PREFIX_REPZ 1
-#define PREFIX_REPNZ 2
-#define PREFIX_LOCK 4
-#define PREFIX_CS 8
-#define PREFIX_SS 0x10
-#define PREFIX_DS 0x20
-#define PREFIX_ES 0x40
-#define PREFIX_FS 0x80
-#define PREFIX_GS 0x100
-#define PREFIX_DATA 0x200
-#define PREFIX_ADDR 0x400
-#define PREFIX_FWAIT 0x800
-
-#define PREFIX_VEX_0F    0x1000
-#define PREFIX_VEX_0F38  0x2000
-#define PREFIX_VEX_0F3A  0x4000
-
-/* Make sure that bytes from INFO->PRIVATE_DATA->BUFFER (inclusive)
-   to ADDR (exclusive) are valid.  Returns 1 for success, longjmps
-   on error.  */
-static int
-fetch_data2(struct disassemble_info *info, bfd_byte *addr)
-{
-  int status;
-  struct dis_private *priv = (struct dis_private *) info->private_data;
-  bfd_vma start = priv->insn_start + (priv->max_fetched - priv->the_buffer);
-
-  if (addr <= priv->the_buffer + MAX_MNEM_SIZE)
-    status = (*info->read_memory_func) (start,
-					priv->max_fetched,
-					addr - priv->max_fetched,
-					info);
-  else
-    status = -1;
-  if (status != 0)
-    {
-      /* If we did manage to read at least one byte, then
-	 print_insn_i386 will do something sensible.  Otherwise, print
-	 an error.  We do that here because this is where we know
-	 STATUS.  */
-      if (priv->max_fetched == priv->the_buffer)
-	(*info->memory_error_func) (status, start, info);
-      siglongjmp(priv->bailout, 1);
-    }
-  else
-    priv->max_fetched = addr;
-  return 1;
-}
-
-static int
-fetch_data(struct disassemble_info *info, bfd_byte *addr)
-{
-    if (addr <= ((struct dis_private *) (info->private_data))->max_fetched) {
-        return 1;
-    } else {
-        return fetch_data2(info, addr);
-    }
-}
-
-
-#define XX { NULL, 0 }
-
-#define Bv { OP_vvvv, v_mode }
-#define Eb { OP_E, b_mode }
-#define Ev { OP_E, v_mode }
-#define Ed { OP_E, d_mode }
-#define Edq { OP_E, dq_mode }
-#define Edqw { OP_E, dqw_mode }
-#define Edqb { OP_E, dqb_mode }
-#define Edqd { OP_E, dqd_mode }
-#define indirEv { OP_indirE, stack_v_mode }
-#define indirEp { OP_indirE, f_mode }
-#define stackEv { OP_E, stack_v_mode }
-#define Em { OP_E, m_mode }
-#define Ew { OP_E, w_mode }
-#define M { OP_M, 0 }		/* lea, lgdt, etc. */
-#define Ma { OP_M, v_mode }
-#define Mp { OP_M, f_mode }		/* 32 or 48 bit memory operand for LDS, LES etc */
-#define Mq { OP_M, q_mode }
-#define Gb { OP_G, b_mode }
-#define Gv { OP_G, v_mode }
-#define Gd { OP_G, d_mode }
-#define Gdq { OP_G, dq_mode }
-#define Gm { OP_G, m_mode }
-#define Gw { OP_G, w_mode }
-#define Rd { OP_R, d_mode }
-#define Rm { OP_R, m_mode }
-#define Ib { OP_I, b_mode }
-#define sIb { OP_sI, b_mode }	/* sign extended byte */
-#define Iv { OP_I, v_mode }
-#define Iq { OP_I, q_mode }
-#define Iv64 { OP_I64, v_mode }
-#define Iw { OP_I, w_mode }
-#define I1 { OP_I, const_1_mode }
-#define Jb { OP_J, b_mode }
-#define Jv { OP_J, v_mode }
-#define Cm { OP_C, m_mode }
-#define Dm { OP_D, m_mode }
-#define Td { OP_T, d_mode }
-
-#define RMeAX { OP_REG, eAX_reg }
-#define RMeBX { OP_REG, eBX_reg }
-#define RMeCX { OP_REG, eCX_reg }
-#define RMeDX { OP_REG, eDX_reg }
-#define RMeSP { OP_REG, eSP_reg }
-#define RMeBP { OP_REG, eBP_reg }
-#define RMeSI { OP_REG, eSI_reg }
-#define RMeDI { OP_REG, eDI_reg }
-#define RMrAX { OP_REG, rAX_reg }
-#define RMrBX { OP_REG, rBX_reg }
-#define RMrCX { OP_REG, rCX_reg }
-#define RMrDX { OP_REG, rDX_reg }
-#define RMrSP { OP_REG, rSP_reg }
-#define RMrBP { OP_REG, rBP_reg }
-#define RMrSI { OP_REG, rSI_reg }
-#define RMrDI { OP_REG, rDI_reg }
-#define RMAL { OP_REG, al_reg }
-#define RMAL { OP_REG, al_reg }
-#define RMCL { OP_REG, cl_reg }
-#define RMDL { OP_REG, dl_reg }
-#define RMBL { OP_REG, bl_reg }
-#define RMAH { OP_REG, ah_reg }
-#define RMCH { OP_REG, ch_reg }
-#define RMDH { OP_REG, dh_reg }
-#define RMBH { OP_REG, bh_reg }
-#define RMAX { OP_REG, ax_reg }
-#define RMDX { OP_REG, dx_reg }
-
-#define eAX { OP_IMREG, eAX_reg }
-#define eBX { OP_IMREG, eBX_reg }
-#define eCX { OP_IMREG, eCX_reg }
-#define eDX { OP_IMREG, eDX_reg }
-#define eSP { OP_IMREG, eSP_reg }
-#define eBP { OP_IMREG, eBP_reg }
-#define eSI { OP_IMREG, eSI_reg }
-#define eDI { OP_IMREG, eDI_reg }
-#define AL { OP_IMREG, al_reg }
-#define CL { OP_IMREG, cl_reg }
-#define DL { OP_IMREG, dl_reg }
-#define BL { OP_IMREG, bl_reg }
-#define AH { OP_IMREG, ah_reg }
-#define CH { OP_IMREG, ch_reg }
-#define DH { OP_IMREG, dh_reg }
-#define BH { OP_IMREG, bh_reg }
-#define AX { OP_IMREG, ax_reg }
-#define DX { OP_IMREG, dx_reg }
-#define zAX { OP_IMREG, z_mode_ax_reg }
-#define indirDX { OP_IMREG, indir_dx_reg }
-
-#define Sw { OP_SEG, w_mode }
-#define Sv { OP_SEG, v_mode }
-#define Ap { OP_DIR, 0 }
-#define Ob { OP_OFF64, b_mode }
-#define Ov { OP_OFF64, v_mode }
-#define Xb { OP_DSreg, eSI_reg }
-#define Xv { OP_DSreg, eSI_reg }
-#define Xz { OP_DSreg, eSI_reg }
-#define Yb { OP_ESreg, eDI_reg }
-#define Yv { OP_ESreg, eDI_reg }
-#define DSBX { OP_DSreg, eBX_reg }
-
-#define es { OP_REG, es_reg }
-#define ss { OP_REG, ss_reg }
-#define cs { OP_REG, cs_reg }
-#define ds { OP_REG, ds_reg }
-#define fs { OP_REG, fs_reg }
-#define gs { OP_REG, gs_reg }
-
-#define MX { OP_MMX, 0 }
-#define XM { OP_XMM, 0 }
-#define EM { OP_EM, v_mode }
-#define EMd { OP_EM, d_mode }
-#define EMq { OP_EM, q_mode }
-#define EXd { OP_EX, d_mode }
-#define EXq { OP_EX, q_mode }
-#define EXx { OP_EX, x_mode }
-#define MS { OP_MS, v_mode }
-#define XS { OP_XS, v_mode }
-#define EMC { OP_EMC, v_mode }
-#define MXC { OP_MXC, 0 }
-#define VM { OP_VMX, q_mode }
-#define OPSUF { OP_3DNowSuffix, 0 }
-#define OPSIMD { OP_SIMD_Suffix, 0 }
-#define XMM0 { XMM_Fixup, 0 }
-
-/* Used handle "rep" prefix for string instructions.  */
-#define Xbr { REP_Fixup, eSI_reg }
-#define Xvr { REP_Fixup, eSI_reg }
-#define Ybr { REP_Fixup, eDI_reg }
-#define Yvr { REP_Fixup, eDI_reg }
-#define Yzr { REP_Fixup, eDI_reg }
-#define indirDXr { REP_Fixup, indir_dx_reg }
-#define ALr { REP_Fixup, al_reg }
-#define eAXr { REP_Fixup, eAX_reg }
-
-#define cond_jump_flag { NULL, cond_jump_mode }
-#define loop_jcxz_flag { NULL, loop_jcxz_mode }
-
-/* bits in sizeflag */
-#define SUFFIX_ALWAYS 4
-#define AFLAG 2
-#define DFLAG 1
-
-#define b_mode 1  /* byte operand */
-#define v_mode 2  /* operand size depends on prefixes */
-#define w_mode 3  /* word operand */
-#define d_mode 4  /* double word operand  */
-#define q_mode 5  /* quad word operand */
-#define t_mode 6  /* ten-byte operand */
-#define x_mode 7  /* 16-byte XMM operand */
-#define m_mode 8  /* d_mode in 32bit, q_mode in 64bit mode.  */
-#define cond_jump_mode 9
-#define loop_jcxz_mode 10
-#define dq_mode 11 /* operand size depends on REX prefixes.  */
-#define dqw_mode 12 /* registers like dq_mode, memory like w_mode.  */
-#define f_mode 13 /* 4- or 6-byte pointer operand */
-#define const_1_mode 14
-#define stack_v_mode 15 /* v_mode for stack-related opcodes.  */
-#define z_mode 16 /* non-quad operand size depends on prefixes */
-#define o_mode 17  /* 16-byte operand */
-#define dqb_mode 18 /* registers like dq_mode, memory like b_mode.  */
-#define dqd_mode 19 /* registers like dq_mode, memory like d_mode.  */
-
-#define es_reg 100
-#define cs_reg 101
-#define ss_reg 102
-#define ds_reg 103
-#define fs_reg 104
-#define gs_reg 105
-
-#define eAX_reg 108
-#define eCX_reg 109
-#define eDX_reg 110
-#define eBX_reg 111
-#define eSP_reg 112
-#define eBP_reg 113
-#define eSI_reg 114
-#define eDI_reg 115
-
-#define al_reg 116
-#define cl_reg 117
-#define dl_reg 118
-#define bl_reg 119
-#define ah_reg 120
-#define ch_reg 121
-#define dh_reg 122
-#define bh_reg 123
-
-#define ax_reg 124
-#define cx_reg 125
-#define dx_reg 126
-#define bx_reg 127
-#define sp_reg 128
-#define bp_reg 129
-#define si_reg 130
-#define di_reg 131
-
-#define rAX_reg 132
-#define rCX_reg 133
-#define rDX_reg 134
-#define rBX_reg 135
-#define rSP_reg 136
-#define rBP_reg 137
-#define rSI_reg 138
-#define rDI_reg 139
-
-#define z_mode_ax_reg 149
-#define indir_dx_reg 150
-
-#define FLOATCODE 1
-#define USE_GROUPS 2
-#define USE_PREFIX_USER_TABLE 3
-#define X86_64_SPECIAL 4
-#define IS_3BYTE_OPCODE 5
-
-#define FLOAT	  NULL, { { NULL, FLOATCODE } }
-
-#define GRP1a	  NULL, { { NULL, USE_GROUPS }, { NULL,  0 } }
-#define GRP1b	  NULL, { { NULL, USE_GROUPS }, { NULL,  1 } }
-#define GRP1S	  NULL, { { NULL, USE_GROUPS }, { NULL,  2 } }
-#define GRP1Ss	  NULL, { { NULL, USE_GROUPS }, { NULL,  3 } }
-#define GRP2b	  NULL, { { NULL, USE_GROUPS }, { NULL,  4 } }
-#define GRP2S	  NULL, { { NULL, USE_GROUPS }, { NULL,  5 } }
-#define GRP2b_one NULL, { { NULL, USE_GROUPS }, { NULL,  6 } }
-#define GRP2S_one NULL, { { NULL, USE_GROUPS }, { NULL,  7 } }
-#define GRP2b_cl  NULL, { { NULL, USE_GROUPS }, { NULL,  8 } }
-#define GRP2S_cl  NULL, { { NULL, USE_GROUPS }, { NULL,  9 } }
-#define GRP3b	  NULL, { { NULL, USE_GROUPS }, { NULL, 10 } }
-#define GRP3S	  NULL, { { NULL, USE_GROUPS }, { NULL, 11 } }
-#define GRP4	  NULL, { { NULL, USE_GROUPS }, { NULL, 12 } }
-#define GRP5	  NULL, { { NULL, USE_GROUPS }, { NULL, 13 } }
-#define GRP6	  NULL, { { NULL, USE_GROUPS }, { NULL, 14 } }
-#define GRP7	  NULL, { { NULL, USE_GROUPS }, { NULL, 15 } }
-#define GRP8	  NULL, { { NULL, USE_GROUPS }, { NULL, 16 } }
-#define GRP9	  NULL, { { NULL, USE_GROUPS }, { NULL, 17 } }
-#define GRP11_C6  NULL, { { NULL, USE_GROUPS }, { NULL, 18 } }
-#define GRP11_C7  NULL, { { NULL, USE_GROUPS }, { NULL, 19 } }
-#define GRP12	  NULL, { { NULL, USE_GROUPS }, { NULL, 20 } }
-#define GRP13	  NULL, { { NULL, USE_GROUPS }, { NULL, 21 } }
-#define GRP14	  NULL, { { NULL, USE_GROUPS }, { NULL, 22 } }
-#define GRP15	  NULL, { { NULL, USE_GROUPS }, { NULL, 23 } }
-#define GRP16	  NULL, { { NULL, USE_GROUPS }, { NULL, 24 } }
-#define GRPAMD	  NULL, { { NULL, USE_GROUPS }, { NULL, 25 } }
-#define GRPPADLCK1 NULL, { { NULL, USE_GROUPS }, { NULL, 26 } }
-#define GRPPADLCK2 NULL, { { NULL, USE_GROUPS }, { NULL, 27 } }
-
-#define PREGRP0   NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL,  0 } }
-#define PREGRP1   NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL,  1 } }
-#define PREGRP2   NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL,  2 } }
-#define PREGRP3   NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL,  3 } }
-#define PREGRP4   NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL,  4 } }
-#define PREGRP5   NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL,  5 } }
-#define PREGRP6   NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL,  6 } }
-#define PREGRP7   NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL,  7 } }
-#define PREGRP8   NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL,  8 } }
-#define PREGRP9   NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL,  9 } }
-#define PREGRP10  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 10 } }
-#define PREGRP11  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 11 } }
-#define PREGRP12  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 12 } }
-#define PREGRP13  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 13 } }
-#define PREGRP14  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 14 } }
-#define PREGRP15  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 15 } }
-#define PREGRP16  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 16 } }
-#define PREGRP17  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 17 } }
-#define PREGRP18  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 18 } }
-#define PREGRP19  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 19 } }
-#define PREGRP20  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 20 } }
-#define PREGRP21  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 21 } }
-#define PREGRP22  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 22 } }
-#define PREGRP23  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 23 } }
-#define PREGRP24  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 24 } }
-#define PREGRP25  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 25 } }
-#define PREGRP26  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 26 } }
-#define PREGRP27  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 27 } }
-#define PREGRP28  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 28 } }
-#define PREGRP29  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 29 } }
-#define PREGRP30  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 30 } }
-#define PREGRP31  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 31 } }
-#define PREGRP32  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 32 } }
-#define PREGRP33  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 33 } }
-#define PREGRP34  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 34 } }
-#define PREGRP35  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 35 } }
-#define PREGRP36  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 36 } }
-#define PREGRP37  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 37 } }
-#define PREGRP38  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 38 } }
-#define PREGRP39  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 39 } }
-#define PREGRP40  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 40 } }
-#define PREGRP41  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 41 } }
-#define PREGRP42  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 42 } }
-#define PREGRP43  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 43 } }
-#define PREGRP44  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 44 } }
-#define PREGRP45  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 45 } }
-#define PREGRP46  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 46 } }
-#define PREGRP47  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 47 } }
-#define PREGRP48  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 48 } }
-#define PREGRP49  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 49 } }
-#define PREGRP50  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 50 } }
-#define PREGRP51  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 51 } }
-#define PREGRP52  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 52 } }
-#define PREGRP53  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 53 } }
-#define PREGRP54  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 54 } }
-#define PREGRP55  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 55 } }
-#define PREGRP56  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 56 } }
-#define PREGRP57  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 57 } }
-#define PREGRP58  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 58 } }
-#define PREGRP59  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 59 } }
-#define PREGRP60  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 60 } }
-#define PREGRP61  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 61 } }
-#define PREGRP62  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 62 } }
-#define PREGRP63  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 63 } }
-#define PREGRP64  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 64 } }
-#define PREGRP65  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 65 } }
-#define PREGRP66  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 66 } }
-#define PREGRP67  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 67 } }
-#define PREGRP68  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 68 } }
-#define PREGRP69  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 69 } }
-#define PREGRP70  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 70 } }
-#define PREGRP71  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 71 } }
-#define PREGRP72  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 72 } }
-#define PREGRP73  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 73 } }
-#define PREGRP74  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 74 } }
-#define PREGRP75  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 75 } }
-#define PREGRP76  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 76 } }
-#define PREGRP77  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 77 } }
-#define PREGRP78  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 78 } }
-#define PREGRP79  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 79 } }
-#define PREGRP80  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 80 } }
-#define PREGRP81  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 81 } }
-#define PREGRP82  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 82 } }
-#define PREGRP83  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 83 } }
-#define PREGRP84  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 84 } }
-#define PREGRP85  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 85 } }
-#define PREGRP86  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 86 } }
-#define PREGRP87  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 87 } }
-#define PREGRP88  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 88 } }
-#define PREGRP89  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 89 } }
-#define PREGRP90  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 90 } }
-#define PREGRP91  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 91 } }
-#define PREGRP92  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 92 } }
-#define PREGRP93  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 93 } }
-#define PREGRP94  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 94 } }
-#define PREGRP95  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 95 } }
-#define PREGRP96  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 96 } }
-#define PREGRP97  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 97 } }
-#define PREGRP98  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 98 } }
-#define PREGRP99  NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 99 } }
-#define PREGRP100 NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 100 } }
-#define PREGRP101 NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 101 } }
-#define PREGRP102 NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 102 } }
-#define PREGRP103 NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 103 } }
-#define PREGRP104 NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 104 } }
-#define PREGRP105 NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 105 } }
-#define PREGRP106 NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 106 } }
-#define PREGRP107 NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 107 } }
-#define PREGRP108 NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 108 } }
-#define PREGRP109 NULL, { { NULL, USE_PREFIX_USER_TABLE }, { NULL, 109 } }
-
-#define X86_64_0  NULL, { { NULL, X86_64_SPECIAL }, { NULL, 0 } }
-#define X86_64_1  NULL, { { NULL, X86_64_SPECIAL }, { NULL, 1 } }
-#define X86_64_2  NULL, { { NULL, X86_64_SPECIAL }, { NULL, 2 } }
-#define X86_64_3  NULL, { { NULL, X86_64_SPECIAL }, { NULL, 3 } }
-
-#define THREE_BYTE_0 NULL, { { NULL, IS_3BYTE_OPCODE }, { NULL, 0 } }
-#define THREE_BYTE_1 NULL, { { NULL, IS_3BYTE_OPCODE }, { NULL, 1 } }
-
-typedef void (*op_rtn) (int bytemode, int sizeflag);
-
-struct dis386 {
-  const char *name;
-  struct
-    {
-      op_rtn rtn;
-      int bytemode;
-    } op[MAX_OPERANDS];
-};
-
-/* Upper case letters in the instruction names here are macros.
-   'A' => print 'b' if no register operands or suffix_always is true
-   'B' => print 'b' if suffix_always is true
-   'C' => print 's' or 'l' ('w' or 'd' in Intel mode) depending on operand
-   .      size prefix
-   'D' => print 'w' if no register operands or 'w', 'l' or 'q', if
-   .      suffix_always is true
-   'E' => print 'e' if 32-bit form of jcxz
-   'F' => print 'w' or 'l' depending on address size prefix (loop insns)
-   'G' => print 'w' or 'l' depending on operand size prefix (i/o insns)
-   'H' => print ",pt" or ",pn" branch hint
-   'I' => honor following macro letter even in Intel mode (implemented only
-   .      for some of the macro letters)
-   'J' => print 'l'
-   'K' => print 'd' or 'q' if rex prefix is present.
-   'L' => print 'l' if suffix_always is true
-   'N' => print 'n' if instruction has no wait "prefix"
-   'O' => print 'd' or 'o' (or 'q' in Intel mode)
-   'P' => print 'w', 'l' or 'q' if instruction has an operand size prefix,
-   .      or suffix_always is true.  print 'q' if rex prefix is present.
-   'Q' => print 'w', 'l' or 'q' if no register operands or suffix_always
-   .      is true
-   'R' => print 'w', 'l' or 'q' ('d' for 'l' and 'e' in Intel mode)
-   'S' => print 'w', 'l' or 'q' if suffix_always is true
-   'T' => print 'q' in 64bit mode and behave as 'P' otherwise
-   'U' => print 'q' in 64bit mode and behave as 'Q' otherwise
-   'V' => print 'q' in 64bit mode and behave as 'S' otherwise
-   'W' => print 'b', 'w' or 'l' ('d' in Intel mode)
-   'X' => print 's', 'd' depending on data16 prefix (for XMM)
-   'Y' => 'q' if instruction has an REX 64bit overwrite prefix
-   'Z' => print 'q' in 64bit mode and behave as 'L' otherwise
-
-   Many of the above letters print nothing in Intel mode.  See "putop"
-   for the details.
-
-   Braces '{' and '}', and vertical bars '|', indicate alternative
-   mnemonic strings for AT&T, Intel, X86_64 AT&T, and X86_64 Intel
-   modes.  In cases where there are only two alternatives, the X86_64
-   instruction is reserved, and "(bad)" is printed.
-*/
-
-static const struct dis386 dis386[] = {
-  /* 00 */
-  { "addB",		{ Eb, Gb } },
-  { "addS",		{ Ev, Gv } },
-  { "addB",		{ Gb, Eb } },
-  { "addS",		{ Gv, Ev } },
-  { "addB",		{ AL, Ib } },
-  { "addS",		{ eAX, Iv } },
-  { "push{T|}",		{ es } },
-  { "pop{T|}",		{ es } },
-  /* 08 */
-  { "orB",		{ Eb, Gb } },
-  { "orS",		{ Ev, Gv } },
-  { "orB",		{ Gb, Eb } },
-  { "orS",		{ Gv, Ev } },
-  { "orB",		{ AL, Ib } },
-  { "orS",		{ eAX, Iv } },
-  { "push{T|}",		{ cs } },
-  { "(bad)",		{ XX } },	/* 0x0f extended opcode escape */
-  /* 10 */
-  { "adcB",		{ Eb, Gb } },
-  { "adcS",		{ Ev, Gv } },
-  { "adcB",		{ Gb, Eb } },
-  { "adcS",		{ Gv, Ev } },
-  { "adcB",		{ AL, Ib } },
-  { "adcS",		{ eAX, Iv } },
-  { "push{T|}",		{ ss } },
-  { "pop{T|}",		{ ss } },
-  /* 18 */
-  { "sbbB",		{ Eb, Gb } },
-  { "sbbS",		{ Ev, Gv } },
-  { "sbbB",		{ Gb, Eb } },
-  { "sbbS",		{ Gv, Ev } },
-  { "sbbB",		{ AL, Ib } },
-  { "sbbS",		{ eAX, Iv } },
-  { "push{T|}",		{ ds } },
-  { "pop{T|}",		{ ds } },
-  /* 20 */
-  { "andB",		{ Eb, Gb } },
-  { "andS",		{ Ev, Gv } },
-  { "andB",		{ Gb, Eb } },
-  { "andS",		{ Gv, Ev } },
-  { "andB",		{ AL, Ib } },
-  { "andS",		{ eAX, Iv } },
-  { "(bad)",		{ XX } },	/* SEG ES prefix */
-  { "daa{|}",		{ XX } },
-  /* 28 */
-  { "subB",		{ Eb, Gb } },
-  { "subS",		{ Ev, Gv } },
-  { "subB",		{ Gb, Eb } },
-  { "subS",		{ Gv, Ev } },
-  { "subB",		{ AL, Ib } },
-  { "subS",		{ eAX, Iv } },
-  { "(bad)",		{ XX } },	/* SEG CS prefix */
-  { "das{|}",		{ XX } },
-  /* 30 */
-  { "xorB",		{ Eb, Gb } },
-  { "xorS",		{ Ev, Gv } },
-  { "xorB",		{ Gb, Eb } },
-  { "xorS",		{ Gv, Ev } },
-  { "xorB",		{ AL, Ib } },
-  { "xorS",		{ eAX, Iv } },
-  { "(bad)",		{ XX } },	/* SEG SS prefix */
-  { "aaa{|}",		{ XX } },
-  /* 38 */
-  { "cmpB",		{ Eb, Gb } },
-  { "cmpS",		{ Ev, Gv } },
-  { "cmpB",		{ Gb, Eb } },
-  { "cmpS",		{ Gv, Ev } },
-  { "cmpB",		{ AL, Ib } },
-  { "cmpS",		{ eAX, Iv } },
-  { "(bad)",		{ XX } },	/* SEG DS prefix */
-  { "aas{|}",		{ XX } },
-  /* 40 */
-  { "inc{S|}",		{ RMeAX } },
-  { "inc{S|}",		{ RMeCX } },
-  { "inc{S|}",		{ RMeDX } },
-  { "inc{S|}",		{ RMeBX } },
-  { "inc{S|}",		{ RMeSP } },
-  { "inc{S|}",		{ RMeBP } },
-  { "inc{S|}",		{ RMeSI } },
-  { "inc{S|}",		{ RMeDI } },
-  /* 48 */
-  { "dec{S|}",		{ RMeAX } },
-  { "dec{S|}",		{ RMeCX } },
-  { "dec{S|}",		{ RMeDX } },
-  { "dec{S|}",		{ RMeBX } },
-  { "dec{S|}",		{ RMeSP } },
-  { "dec{S|}",		{ RMeBP } },
-  { "dec{S|}",		{ RMeSI } },
-  { "dec{S|}",		{ RMeDI } },
-  /* 50 */
-  { "pushV",		{ RMrAX } },
-  { "pushV",		{ RMrCX } },
-  { "pushV",		{ RMrDX } },
-  { "pushV",		{ RMrBX } },
-  { "pushV",		{ RMrSP } },
-  { "pushV",		{ RMrBP } },
-  { "pushV",		{ RMrSI } },
-  { "pushV",		{ RMrDI } },
-  /* 58 */
-  { "popV",		{ RMrAX } },
-  { "popV",		{ RMrCX } },
-  { "popV",		{ RMrDX } },
-  { "popV",		{ RMrBX } },
-  { "popV",		{ RMrSP } },
-  { "popV",		{ RMrBP } },
-  { "popV",		{ RMrSI } },
-  { "popV",		{ RMrDI } },
-  /* 60 */
-  { X86_64_0 },
-  { X86_64_1 },
-  { X86_64_2 },
-  { X86_64_3 },
-  { "(bad)",		{ XX } },	/* seg fs */
-  { "(bad)",		{ XX } },	/* seg gs */
-  { "(bad)",		{ XX } },	/* op size prefix */
-  { "(bad)",		{ XX } },	/* adr size prefix */
-  /* 68 */
-  { "pushT",		{ Iq } },
-  { "imulS",		{ Gv, Ev, Iv } },
-  { "pushT",		{ sIb } },
-  { "imulS",		{ Gv, Ev, sIb } },
-  { "ins{b||b|}",	{ Ybr, indirDX } },
-  { "ins{R||G|}",	{ Yzr, indirDX } },
-  { "outs{b||b|}",	{ indirDXr, Xb } },
-  { "outs{R||G|}",	{ indirDXr, Xz } },
-  /* 70 */
-  { "joH",		{ Jb, XX, cond_jump_flag } },
-  { "jnoH",		{ Jb, XX, cond_jump_flag } },
-  { "jbH",		{ Jb, XX, cond_jump_flag } },
-  { "jaeH",		{ Jb, XX, cond_jump_flag } },
-  { "jeH",		{ Jb, XX, cond_jump_flag } },
-  { "jneH",		{ Jb, XX, cond_jump_flag } },
-  { "jbeH",		{ Jb, XX, cond_jump_flag } },
-  { "jaH",		{ Jb, XX, cond_jump_flag } },
-  /* 78 */
-  { "jsH",		{ Jb, XX, cond_jump_flag } },
-  { "jnsH",		{ Jb, XX, cond_jump_flag } },
-  { "jpH",		{ Jb, XX, cond_jump_flag } },
-  { "jnpH",		{ Jb, XX, cond_jump_flag } },
-  { "jlH",		{ Jb, XX, cond_jump_flag } },
-  { "jgeH",		{ Jb, XX, cond_jump_flag } },
-  { "jleH",		{ Jb, XX, cond_jump_flag } },
-  { "jgH",		{ Jb, XX, cond_jump_flag } },
-  /* 80 */
-  { GRP1b },
-  { GRP1S },
-  { "(bad)",		{ XX } },
-  { GRP1Ss },
-  { "testB",		{ Eb, Gb } },
-  { "testS",		{ Ev, Gv } },
-  { "xchgB",		{ Eb, Gb } },
-  { "xchgS",		{ Ev, Gv } },
-  /* 88 */
-  { "movB",		{ Eb, Gb } },
-  { "movS",		{ Ev, Gv } },
-  { "movB",		{ Gb, Eb } },
-  { "movS",		{ Gv, Ev } },
-  { "movD",		{ Sv, Sw } },
-  { "leaS",		{ Gv, M } },
-  { "movD",		{ Sw, Sv } },
-  { GRP1a },
-  /* 90 */
-  { PREGRP38 },
-  { "xchgS",		{ RMeCX, eAX } },
-  { "xchgS",		{ RMeDX, eAX } },
-  { "xchgS",		{ RMeBX, eAX } },
-  { "xchgS",		{ RMeSP, eAX } },
-  { "xchgS",		{ RMeBP, eAX } },
-  { "xchgS",		{ RMeSI, eAX } },
-  { "xchgS",		{ RMeDI, eAX } },
-  /* 98 */
-  { "cW{t||t|}R",	{ XX } },
-  { "cR{t||t|}O",	{ XX } },
-  { "Jcall{T|}",	{ Ap } },
-  { "(bad)",		{ XX } },	/* fwait */
-  { "pushfT",		{ XX } },
-  { "popfT",		{ XX } },
-  { "sahf{|}",		{ XX } },
-  { "lahf{|}",		{ XX } },
-  /* a0 */
-  { "movB",		{ AL, Ob } },
-  { "movS",		{ eAX, Ov } },
-  { "movB",		{ Ob, AL } },
-  { "movS",		{ Ov, eAX } },
-  { "movs{b||b|}",	{ Ybr, Xb } },
-  { "movs{R||R|}",	{ Yvr, Xv } },
-  { "cmps{b||b|}",	{ Xb, Yb } },
-  { "cmps{R||R|}",	{ Xv, Yv } },
-  /* a8 */
-  { "testB",		{ AL, Ib } },
-  { "testS",		{ eAX, Iv } },
-  { "stosB",		{ Ybr, AL } },
-  { "stosS",		{ Yvr, eAX } },
-  { "lodsB",		{ ALr, Xb } },
-  { "lodsS",		{ eAXr, Xv } },
-  { "scasB",		{ AL, Yb } },
-  { "scasS",		{ eAX, Yv } },
-  /* b0 */
-  { "movB",		{ RMAL, Ib } },
-  { "movB",		{ RMCL, Ib } },
-  { "movB",		{ RMDL, Ib } },
-  { "movB",		{ RMBL, Ib } },
-  { "movB",		{ RMAH, Ib } },
-  { "movB",		{ RMCH, Ib } },
-  { "movB",		{ RMDH, Ib } },
-  { "movB",		{ RMBH, Ib } },
-  /* b8 */
-  { "movS",		{ RMeAX, Iv64 } },
-  { "movS",		{ RMeCX, Iv64 } },
-  { "movS",		{ RMeDX, Iv64 } },
-  { "movS",		{ RMeBX, Iv64 } },
-  { "movS",		{ RMeSP, Iv64 } },
-  { "movS",		{ RMeBP, Iv64 } },
-  { "movS",		{ RMeSI, Iv64 } },
-  { "movS",		{ RMeDI, Iv64 } },
-  /* c0 */
-  { GRP2b },
-  { GRP2S },
-  { "retT",		{ Iw } },
-  { "retT",		{ XX } },
-  { "les{S|}",		{ Gv, Mp } },
-  { "ldsS",		{ Gv, Mp } },
-  { GRP11_C6 },
-  { GRP11_C7 },
-  /* c8 */
-  { "enterT",		{ Iw, Ib } },
-  { "leaveT",		{ XX } },
-  { "lretP",		{ Iw } },
-  { "lretP",		{ XX } },
-  { "int3",		{ XX } },
-  { "int",		{ Ib } },
-  { "into{|}",		{ XX } },
-  { "iretP",		{ XX } },
-  /* d0 */
-  { GRP2b_one },
-  { GRP2S_one },
-  { GRP2b_cl },
-  { GRP2S_cl },
-  { "aam{|}",		{ sIb } },
-  { "aad{|}",		{ sIb } },
-  { "(bad)",		{ XX } },
-  { "xlat",		{ DSBX } },
-  /* d8 */
-  { FLOAT },
-  { FLOAT },
-  { FLOAT },
-  { FLOAT },
-  { FLOAT },
-  { FLOAT },
-  { FLOAT },
-  { FLOAT },
-  /* e0 */
-  { "loopneFH",		{ Jb, XX, loop_jcxz_flag } },
-  { "loopeFH",		{ Jb, XX, loop_jcxz_flag } },
-  { "loopFH",		{ Jb, XX, loop_jcxz_flag } },
-  { "jEcxzH",		{ Jb, XX, loop_jcxz_flag } },
-  { "inB",		{ AL, Ib } },
-  { "inG",		{ zAX, Ib } },
-  { "outB",		{ Ib, AL } },
-  { "outG",		{ Ib, zAX } },
-  /* e8 */
-  { "callT",		{ Jv } },
-  { "jmpT",		{ Jv } },
-  { "Jjmp{T|}",		{ Ap } },
-  { "jmp",		{ Jb } },
-  { "inB",		{ AL, indirDX } },
-  { "inG",		{ zAX, indirDX } },
-  { "outB",		{ indirDX, AL } },
-  { "outG",		{ indirDX, zAX } },
-  /* f0 */
-  { "(bad)",		{ XX } },	/* lock prefix */
-  { "icebp",		{ XX } },
-  { "(bad)",		{ XX } },	/* repne */
-  { "(bad)",		{ XX } },	/* repz */
-  { "hlt",		{ XX } },
-  { "cmc",		{ XX } },
-  { GRP3b },
-  { GRP3S },
-  /* f8 */
-  { "clc",		{ XX } },
-  { "stc",		{ XX } },
-  { "cli",		{ XX } },
-  { "sti",		{ XX } },
-  { "cld",		{ XX } },
-  { "std",		{ XX } },
-  { GRP4 },
-  { GRP5 },
-};
-
-static const struct dis386 dis386_twobyte[] = {
-  /* 00 */
-  { GRP6 },
-  { GRP7 },
-  { "larS",		{ Gv, Ew } },
-  { "lslS",		{ Gv, Ew } },
-  { "(bad)",		{ XX } },
-  { "syscall",		{ XX } },
-  { "clts",		{ XX } },
-  { "sysretP",		{ XX } },
-  /* 08 */
-  { "invd",		{ XX } },
-  { "wbinvd",		{ XX } },
-  { "(bad)",		{ XX } },
-  { "ud2a",		{ XX } },
-  { "(bad)",		{ XX } },
-  { GRPAMD },
-  { "femms",		{ XX } },
-  { "",			{ MX, EM, OPSUF } }, /* See OP_3DNowSuffix.  */
-  /* 10 */
-  { PREGRP8 },
-  { PREGRP9 },
-  { PREGRP30 },
-  { "movlpX",		{ EXq, XM, { SIMD_Fixup, 'h' } } },
-  { "unpcklpX",		{ XM, EXq } },
-  { "unpckhpX",		{ XM, EXq } },
-  { PREGRP31 },
-  { "movhpX",		{ EXq, XM, { SIMD_Fixup, 'l' } } },
-  /* 18 */
-  { GRP16 },
-  { "(bad)",		{ XX } },
-  { "(bad)",		{ XX } },
-  { "(bad)",		{ XX } },
-  { "(bad)",		{ XX } },
-  { "(bad)",		{ XX } },
-  { "(bad)",		{ XX } },
-  { "nopQ",		{ Ev } },
-  /* 20 */
-  { "movZ",		{ Rm, Cm } },
-  { "movZ",		{ Rm, Dm } },
-  { "movZ",		{ Cm, Rm } },
-  { "movZ",		{ Dm, Rm } },
-  { "movL",		{ Rd, Td } },
-  { "(bad)",		{ XX } },
-  { "movL",		{ Td, Rd } },
-  { "(bad)",		{ XX } },
-  /* 28 */
-  { "movapX",		{ XM, EXx } },
-  { "movapX",		{ EXx,  XM } },
-  { PREGRP2 },
-  { PREGRP33 },
-  { PREGRP4 },
-  { PREGRP3 },
-  { PREGRP93 },
-  { PREGRP94 },
-  /* 30 */
-  { "wrmsr",		{ XX } },
-  { "rdtsc",		{ XX } },
-  { "rdmsr",		{ XX } },
-  { "rdpmc",		{ XX } },
-  { "sysenter",		{ XX } },
-  { "sysexit",		{ XX } },
-  { "(bad)",		{ XX } },
-  { "(bad)",		{ XX } },
-  /* 38 */
-  { THREE_BYTE_0 },
-  { "(bad)",		{ XX } },
-  { THREE_BYTE_1 },
-  { "(bad)",		{ XX } },
-  { "(bad)",		{ XX } },
-  { "(bad)",		{ XX } },
-  { "(bad)",		{ XX } },
-  { "(bad)",		{ XX } },
-  /* 40 */
-  { "cmovo",		{ Gv, Ev } },
-  { "cmovno",		{ Gv, Ev } },
-  { "cmovb",		{ Gv, Ev } },
-  { "cmovae",		{ Gv, Ev } },
-  { "cmove",		{ Gv, Ev } },
-  { "cmovne",		{ Gv, Ev } },
-  { "cmovbe",		{ Gv, Ev } },
-  { "cmova",		{ Gv, Ev } },
-  /* 48 */
-  { "cmovs",		{ Gv, Ev } },
-  { "cmovns",		{ Gv, Ev } },
-  { "cmovp",		{ Gv, Ev } },
-  { "cmovnp",		{ Gv, Ev } },
-  { "cmovl",		{ Gv, Ev } },
-  { "cmovge",		{ Gv, Ev } },
-  { "cmovle",		{ Gv, Ev } },
-  { "cmovg",		{ Gv, Ev } },
-  /* 50 */
-  { "movmskpX",		{ Gdq, XS } },
-  { PREGRP13 },
-  { PREGRP12 },
-  { PREGRP11 },
-  { "andpX",		{ XM, EXx } },
-  { "andnpX",		{ XM, EXx } },
-  { "orpX",		{ XM, EXx } },
-  { "xorpX",		{ XM, EXx } },
-  /* 58 */
-  { PREGRP0 },
-  { PREGRP10 },
-  { PREGRP17 },
-  { PREGRP16 },
-  { PREGRP14 },
-  { PREGRP7 },
-  { PREGRP5 },
-  { PREGRP6 },
-  /* 60 */
-  { PREGRP95 },
-  { PREGRP96 },
-  { PREGRP97 },
-  { "packsswb",		{ MX, EM } },
-  { "pcmpgtb",		{ MX, EM } },
-  { "pcmpgtw",		{ MX, EM } },
-  { "pcmpgtd",		{ MX, EM } },
-  { "packuswb",		{ MX, EM } },
-  /* 68 */
-  { "punpckhbw",	{ MX, EM } },
-  { "punpckhwd",	{ MX, EM } },
-  { "punpckhdq",	{ MX, EM } },
-  { "packssdw",		{ MX, EM } },
-  { PREGRP26 },
-  { PREGRP24 },
-  { "movd",		{ MX, Edq } },
-  { PREGRP19 },
-  /* 70 */
-  { PREGRP22 },
-  { GRP12 },
-  { GRP13 },
-  { GRP14 },
-  { "pcmpeqb",		{ MX, EM } },
-  { "pcmpeqw",		{ MX, EM } },
-  { "pcmpeqd",		{ MX, EM } },
-  { "emms",		{ XX } },
-  /* 78 */
-  { PREGRP34 },
-  { PREGRP35 },
-  { "(bad)",		{ XX } },
-  { "(bad)",		{ XX } },
-  { PREGRP28 },
-  { PREGRP29 },
-  { PREGRP23 },
-  { PREGRP20 },
-  /* 80 */
-  { "joH",		{ Jv, XX, cond_jump_flag } },
-  { "jnoH",		{ Jv, XX, cond_jump_flag } },
-  { "jbH",		{ Jv, XX, cond_jump_flag } },
-  { "jaeH",		{ Jv, XX, cond_jump_flag } },
-  { "jeH",		{ Jv, XX, cond_jump_flag } },
-  { "jneH",		{ Jv, XX, cond_jump_flag } },
-  { "jbeH",		{ Jv, XX, cond_jump_flag } },
-  { "jaH",		{ Jv, XX, cond_jump_flag } },
-  /* 88 */
-  { "jsH",		{ Jv, XX, cond_jump_flag } },
-  { "jnsH",		{ Jv, XX, cond_jump_flag } },
-  { "jpH",		{ Jv, XX, cond_jump_flag } },
-  { "jnpH",		{ Jv, XX, cond_jump_flag } },
-  { "jlH",		{ Jv, XX, cond_jump_flag } },
-  { "jgeH",		{ Jv, XX, cond_jump_flag } },
-  { "jleH",		{ Jv, XX, cond_jump_flag } },
-  { "jgH",		{ Jv, XX, cond_jump_flag } },
-  /* 90 */
-  { "seto",		{ Eb } },
-  { "setno",		{ Eb } },
-  { "setb",		{ Eb } },
-  { "setae",		{ Eb } },
-  { "sete",		{ Eb } },
-  { "setne",		{ Eb } },
-  { "setbe",		{ Eb } },
-  { "seta",		{ Eb } },
-  /* 98 */
-  { "sets",		{ Eb } },
-  { "setns",		{ Eb } },
-  { "setp",		{ Eb } },
-  { "setnp",		{ Eb } },
-  { "setl",		{ Eb } },
-  { "setge",		{ Eb } },
-  { "setle",		{ Eb } },
-  { "setg",		{ Eb } },
-  /* a0 */
-  { "pushT",		{ fs } },
-  { "popT",		{ fs } },
-  { "cpuid",		{ XX } },
-  { "btS",		{ Ev, Gv } },
-  { "shldS",		{ Ev, Gv, Ib } },
-  { "shldS",		{ Ev, Gv, CL } },
-  { GRPPADLCK2 },
-  { GRPPADLCK1 },
-  /* a8 */
-  { "pushT",		{ gs } },
-  { "popT",		{ gs } },
-  { "rsm",		{ XX } },
-  { "btsS",		{ Ev, Gv } },
-  { "shrdS",		{ Ev, Gv, Ib } },
-  { "shrdS",		{ Ev, Gv, CL } },
-  { GRP15 },
-  { "imulS",		{ Gv, Ev } },
-  /* b0 */
-  { "cmpxchgB",		{ Eb, Gb } },
-  { "cmpxchgS",		{ Ev, Gv } },
-  { "lssS",		{ Gv, Mp } },
-  { "btrS",		{ Ev, Gv } },
-  { "lfsS",		{ Gv, Mp } },
-  { "lgsS",		{ Gv, Mp } },
-  { "movz{bR|x|bR|x}",	{ Gv, Eb } },
-  { "movz{wR|x|wR|x}",	{ Gv, Ew } }, /* yes, there really is movzww ! */
-  /* b8 */
-  { PREGRP37 },
-  { "ud2b",		{ XX } },
-  { GRP8 },
-  { "btcS",		{ Ev, Gv } },
-  { PREGRP107 },
-  { PREGRP36 },
-  { "movs{bR|x|bR|x}",	{ Gv, Eb } },
-  { "movs{wR|x|wR|x}",	{ Gv, Ew } }, /* yes, there really is movsww ! */
-  /* c0 */
-  { "xaddB",		{ Eb, Gb } },
-  { "xaddS",		{ Ev, Gv } },
-  { PREGRP1 },
-  { "movntiS",		{ Ev, Gv } },
-  { "pinsrw",		{ MX, Edqw, Ib } },
-  { "pextrw",		{ Gdq, MS, Ib } },
-  { "shufpX",		{ XM, EXx, Ib } },
-  { GRP9 },
-  /* c8 */
-  { "bswap",		{ RMeAX } },
-  { "bswap",		{ RMeCX } },
-  { "bswap",		{ RMeDX } },
-  { "bswap",		{ RMeBX } },
-  { "bswap",		{ RMeSP } },
-  { "bswap",		{ RMeBP } },
-  { "bswap",		{ RMeSI } },
-  { "bswap",		{ RMeDI } },
-  /* d0 */
-  { PREGRP27 },
-  { "psrlw",		{ MX, EM } },
-  { "psrld",		{ MX, EM } },
-  { "psrlq",		{ MX, EM } },
-  { "paddq",		{ MX, EM } },
-  { "pmullw",		{ MX, EM } },
-  { PREGRP21 },
-  { "pmovmskb",		{ Gdq, MS } },
-  /* d8 */
-  { "psubusb",		{ MX, EM } },
-  { "psubusw",		{ MX, EM } },
-  { "pminub",		{ MX, EM } },
-  { "pand",		{ MX, EM } },
-  { "paddusb",		{ MX, EM } },
-  { "paddusw",		{ MX, EM } },
-  { "pmaxub",		{ MX, EM } },
-  { "pandn",		{ MX, EM } },
-  /* e0 */
-  { "pavgb",		{ MX, EM } },
-  { "psraw",		{ MX, EM } },
-  { "psrad",		{ MX, EM } },
-  { "pavgw",		{ MX, EM } },
-  { "pmulhuw",		{ MX, EM } },
-  { "pmulhw",		{ MX, EM } },
-  { PREGRP15 },
-  { PREGRP25 },
-  /* e8 */
-  { "psubsb",		{ MX, EM } },
-  { "psubsw",		{ MX, EM } },
-  { "pminsw",		{ MX, EM } },
-  { "por",		{ MX, EM } },
-  { "paddsb",		{ MX, EM } },
-  { "paddsw",		{ MX, EM } },
-  { "pmaxsw",		{ MX, EM } },
-  { "pxor",		{ MX, EM } },
-  /* f0 */
-  { PREGRP32 },
-  { "psllw",		{ MX, EM } },
-  { "pslld",		{ MX, EM } },
-  { "psllq",		{ MX, EM } },
-  { "pmuludq",		{ MX, EM } },
-  { "pmaddwd",		{ MX, EM } },
-  { "psadbw",		{ MX, EM } },
-  { PREGRP18 },
-  /* f8 */
-  { "psubb",		{ MX, EM } },
-  { "psubw",		{ MX, EM } },
-  { "psubd",		{ MX, EM } },
-  { "psubq",		{ MX, EM } },
-  { "paddb",		{ MX, EM } },
-  { "paddw",		{ MX, EM } },
-  { "paddd",		{ MX, EM } },
-  { "(bad)",		{ XX } },
-};
-
-static const unsigned char onebyte_has_modrm[256] = {
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-  /*       -------------------------------        */
-  /* 00 */ 1,1,1,1,0,0,0,0,1,1,1,1,0,0,0,0, /* 00 */
-  /* 10 */ 1,1,1,1,0,0,0,0,1,1,1,1,0,0,0,0, /* 10 */
-  /* 20 */ 1,1,1,1,0,0,0,0,1,1,1,1,0,0,0,0, /* 20 */
-  /* 30 */ 1,1,1,1,0,0,0,0,1,1,1,1,0,0,0,0, /* 30 */
-  /* 40 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 40 */
-  /* 50 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 50 */
-  /* 60 */ 0,0,1,1,0,0,0,0,0,1,0,1,0,0,0,0, /* 60 */
-  /* 70 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 70 */
-  /* 80 */ 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, /* 80 */
-  /* 90 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 90 */
-  /* a0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* a0 */
-  /* b0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* b0 */
-  /* c0 */ 1,1,0,0,1,1,1,1,0,0,0,0,0,0,0,0, /* c0 */
-  /* d0 */ 1,1,1,1,0,0,0,0,1,1,1,1,1,1,1,1, /* d0 */
-  /* e0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* e0 */
-  /* f0 */ 0,0,0,0,0,0,1,1,0,0,0,0,0,0,1,1  /* f0 */
-  /*       -------------------------------        */
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-};
-
-static const unsigned char twobyte_has_modrm[256] = {
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-  /*       -------------------------------        */
-  /* 00 */ 1,1,1,1,0,0,0,0,0,0,0,0,0,1,0,1, /* 0f */
-  /* 10 */ 1,1,1,1,1,1,1,1,1,0,0,0,0,0,0,1, /* 1f */
-  /* 20 */ 1,1,1,1,1,0,1,0,1,1,1,1,1,1,1,1, /* 2f */
-  /* 30 */ 0,0,0,0,0,0,0,0,1,0,1,0,0,0,0,0, /* 3f */
-  /* 40 */ 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, /* 4f */
-  /* 50 */ 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, /* 5f */
-  /* 60 */ 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, /* 6f */
-  /* 70 */ 1,1,1,1,1,1,1,0,1,1,0,0,1,1,1,1, /* 7f */
-  /* 80 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 8f */
-  /* 90 */ 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, /* 9f */
-  /* a0 */ 0,0,0,1,1,1,1,1,0,0,0,1,1,1,1,1, /* af */
-  /* b0 */ 1,1,1,1,1,1,1,1,1,0,1,1,1,1,1,1, /* bf */
-  /* c0 */ 1,1,1,1,1,1,1,1,0,0,0,0,0,0,0,0, /* cf */
-  /* d0 */ 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, /* df */
-  /* e0 */ 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, /* ef */
-  /* f0 */ 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0  /* ff */
-  /*       -------------------------------        */
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-};
-
-static const unsigned char twobyte_uses_DATA_prefix[256] = {
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-  /*       -------------------------------        */
-  /* 00 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 0f */
-  /* 10 */ 1,1,1,0,0,0,1,0,0,0,0,0,0,0,0,0, /* 1f */
-  /* 20 */ 0,0,0,0,0,0,0,0,0,0,1,1,1,1,0,0, /* 2f */
-  /* 30 */ 0,0,0,0,0,0,0,0,1,0,1,0,0,0,0,0, /* 3f */
-  /* 40 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 4f */
-  /* 50 */ 0,1,1,1,0,0,0,0,1,1,1,1,1,1,1,1, /* 5f */
-  /* 60 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1, /* 6f */
-  /* 70 */ 1,0,0,0,0,0,0,0,1,1,0,0,1,1,1,1, /* 7f */
-  /* 80 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 8f */
-  /* 90 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 9f */
-  /* a0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* af */
-  /* b0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* bf */
-  /* c0 */ 0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0, /* cf */
-  /* d0 */ 1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0, /* df */
-  /* e0 */ 0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0, /* ef */
-  /* f0 */ 1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0  /* ff */
-  /*       -------------------------------        */
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-};
-
-static const unsigned char twobyte_uses_REPNZ_prefix[256] = {
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-  /*       -------------------------------        */
-  /* 00 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 0f */
-  /* 10 */ 1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 1f */
-  /* 20 */ 0,0,0,0,0,0,0,0,0,0,1,1,1,1,0,0, /* 2f */
-  /* 30 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 3f */
-  /* 40 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 4f */
-  /* 50 */ 0,1,0,0,0,0,0,0,1,1,1,0,1,1,1,1, /* 5f */
-  /* 60 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 6f */
-  /* 70 */ 1,0,0,0,0,0,0,0,1,1,0,0,1,1,0,0, /* 7f */
-  /* 80 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 8f */
-  /* 90 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 9f */
-  /* a0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* af */
-  /* b0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* bf */
-  /* c0 */ 0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0, /* cf */
-  /* d0 */ 1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0, /* df */
-  /* e0 */ 0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0, /* ef */
-  /* f0 */ 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* ff */
-  /*       -------------------------------        */
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-};
-
-static const unsigned char twobyte_uses_REPZ_prefix[256] = {
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-  /*       -------------------------------        */
-  /* 00 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 0f */
-  /* 10 */ 1,1,1,0,0,0,1,0,0,0,0,0,0,0,0,0, /* 1f */
-  /* 20 */ 0,0,0,0,0,0,0,0,0,0,1,1,1,1,0,0, /* 2f */
-  /* 30 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 3f */
-  /* 40 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 4f */
-  /* 50 */ 0,1,1,1,0,0,0,0,1,1,1,1,1,1,1,1, /* 5f */
-  /* 60 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1, /* 6f */
-  /* 70 */ 1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1, /* 7f */
-  /* 80 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 8f */
-  /* 90 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 9f */
-  /* a0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* af */
-  /* b0 */ 0,0,0,0,0,0,0,0,1,0,0,0,1,1,0,0, /* bf */
-  /* c0 */ 0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0, /* cf */
-  /* d0 */ 0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0, /* df */
-  /* e0 */ 0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0, /* ef */
-  /* f0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* ff */
-  /*       -------------------------------        */
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-};
-
-/* This is used to determine if opcode 0f 38 XX uses DATA prefix.  */
-static const unsigned char threebyte_0x38_uses_DATA_prefix[256] = {
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-  /*       -------------------------------        */
-  /* 00 */ 1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,0, /* 0f */
-  /* 10 */ 1,0,0,0,1,1,0,1,0,0,0,0,1,1,1,0, /* 1f */
-  /* 20 */ 1,1,1,1,1,1,0,0,1,1,1,1,0,0,0,0, /* 2f */
-  /* 30 */ 1,1,1,1,1,1,0,1,1,1,1,1,1,1,1,1, /* 3f */
-  /* 40 */ 1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 4f */
-  /* 50 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 5f */
-  /* 60 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 6f */
-  /* 70 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 7f */
-  /* 80 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 8f */
-  /* 90 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 9f */
-  /* a0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* af */
-  /* b0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* bf */
-  /* c0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* cf */
-  /* d0 */ 0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1, /* df */
-  /* e0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* ef */
-  /* f0 */ 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0, /* ff */
-  /*       -------------------------------        */
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-};
-
-/* This is used to determine if opcode 0f 38 XX uses REPNZ prefix.  */
-static const unsigned char threebyte_0x38_uses_REPNZ_prefix[256] = {
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-  /*       -------------------------------        */
-  /* 00 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 0f */
-  /* 10 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 1f */
-  /* 20 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 2f */
-  /* 30 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 3f */
-  /* 40 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 4f */
-  /* 50 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 5f */
-  /* 60 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 6f */
-  /* 70 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 7f */
-  /* 80 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 8f */
-  /* 90 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 9f */
-  /* a0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* af */
-  /* b0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* bf */
-  /* c0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* cf */
-  /* d0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* df */
-  /* e0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* ef */
-  /* f0 */ 1,1,0,0,0,1,0,1,0,0,0,0,0,0,0,0, /* ff */
-  /*       -------------------------------        */
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-};
-
-/* This is used to determine if opcode 0f 38 XX uses REPZ prefix.  */
-static const unsigned char threebyte_0x38_uses_REPZ_prefix[256] = {
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-  /*       -------------------------------        */
-  /* 00 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 0f */
-  /* 10 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 1f */
-  /* 20 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 2f */
-  /* 30 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 3f */
-  /* 40 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 4f */
-  /* 50 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 5f */
-  /* 60 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 6f */
-  /* 70 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 7f */
-  /* 80 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 8f */
-  /* 90 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 9f */
-  /* a0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* af */
-  /* b0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* bf */
-  /* c0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* cf */
-  /* d0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* df */
-  /* e0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* ef */
-  /* f0 */ 0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0, /* ff */
-  /*       -------------------------------        */
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-};
-
-/* This is used to determine if opcode 0f 3a XX uses DATA prefix.  */
-static const unsigned char threebyte_0x3a_uses_DATA_prefix[256] = {
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-  /*       -------------------------------        */
-  /* 00 */ 0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1, /* 0f */
-  /* 10 */ 0,0,0,0,1,1,1,1,0,0,0,0,0,0,0,0, /* 1f */
-  /* 20 */ 1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 2f */
-  /* 30 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 3f */
-  /* 40 */ 1,1,1,0,1,0,0,0,0,0,0,0,0,0,0,0, /* 4f */
-  /* 50 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 5f */
-  /* 60 */ 1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0, /* 6f */
-  /* 70 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 7f */
-  /* 80 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 8f */
-  /* 90 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 9f */
-  /* a0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* af */
-  /* b0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* bf */
-  /* c0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* cf */
-  /* d0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1, /* df */
-  /* e0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* ef */
-  /* f0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* ff */
-  /*       -------------------------------        */
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-};
-
-/* This is used to determine if opcode 0f 3a XX uses REPNZ prefix.  */
-static const unsigned char threebyte_0x3a_uses_REPNZ_prefix[256] = {
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-  /*       -------------------------------        */
-  /* 00 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 0f */
-  /* 10 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 1f */
-  /* 20 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 2f */
-  /* 30 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 3f */
-  /* 40 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 4f */
-  /* 50 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 5f */
-  /* 60 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 6f */
-  /* 70 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 7f */
-  /* 80 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 8f */
-  /* 90 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 9f */
-  /* a0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* af */
-  /* b0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* bf */
-  /* c0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* cf */
-  /* d0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* df */
-  /* e0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* ef */
-  /* f0 */ 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* ff */
-  /*       -------------------------------        */
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-};
-
-/* This is used to determine if opcode 0f 3a XX uses REPZ prefix.  */
-static const unsigned char threebyte_0x3a_uses_REPZ_prefix[256] = {
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-  /*       -------------------------------        */
-  /* 00 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 0f */
-  /* 10 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 1f */
-  /* 20 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 2f */
-  /* 30 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 3f */
-  /* 40 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 4f */
-  /* 50 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 5f */
-  /* 60 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 6f */
-  /* 70 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 7f */
-  /* 80 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 8f */
-  /* 90 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* 9f */
-  /* a0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* af */
-  /* b0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* bf */
-  /* c0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* cf */
-  /* d0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* df */
-  /* e0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* ef */
-  /* f0 */ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, /* ff */
-  /*       -------------------------------        */
-  /*       0 1 2 3 4 5 6 7 8 9 a b c d e f        */
-};
-
-static char obuf[100];
-static char *obufp;
-static char scratchbuf[100];
-static unsigned char *start_codep;
-static unsigned char *insn_codep;
-static unsigned char *codep;
-static disassemble_info *the_info;
-static struct
-  {
-    int mod;
-    int reg;
-    int rm;
-  }
-modrm;
-static unsigned char need_modrm;
-
-/* If we are accessing mod/rm/reg without need_modrm set, then the
-   values are stale.  Hitting this abort likely indicates that you
-   need to update onebyte_has_modrm or twobyte_has_modrm.  */
-#define MODRM_CHECK  if (!need_modrm) abort ()
-
-static const char * const *names64;
-static const char * const *names32;
-static const char * const *names16;
-static const char * const *names8;
-static const char * const *names8rex;
-static const char * const *names_seg;
-static const char * const *index16;
-
-static const char * const intel_names64[] = {
-  "rax", "rcx", "rdx", "rbx", "rsp", "rbp", "rsi", "rdi",
-  "r8", "r9", "r10", "r11", "r12", "r13", "r14", "r15"
-};
-static const char * const intel_names32[] = {
-  "eax", "ecx", "edx", "ebx", "esp", "ebp", "esi", "edi",
-  "r8d", "r9d", "r10d", "r11d", "r12d", "r13d", "r14d", "r15d"
-};
-static const char * const intel_names16[] = {
-  "ax", "cx", "dx", "bx", "sp", "bp", "si", "di",
-  "r8w", "r9w", "r10w", "r11w", "r12w", "r13w", "r14w", "r15w"
-};
-static const char * const intel_names8[] = {
-  "al", "cl", "dl", "bl", "ah", "ch", "dh", "bh",
-};
-static const char * const intel_names8rex[] = {
-  "al", "cl", "dl", "bl", "spl", "bpl", "sil", "dil",
-  "r8b", "r9b", "r10b", "r11b", "r12b", "r13b", "r14b", "r15b"
-};
-static const char * const intel_names_seg[] = {
-  "es", "cs", "ss", "ds", "fs", "gs", "?", "?",
-};
-static const char * const intel_index16[] = {
-  "bx+si", "bx+di", "bp+si", "bp+di", "si", "di", "bp", "bx"
-};
-
-static const char * const att_names64[] = {
-  "%rax", "%rcx", "%rdx", "%rbx", "%rsp", "%rbp", "%rsi", "%rdi",
-  "%r8", "%r9", "%r10", "%r11", "%r12", "%r13", "%r14", "%r15"
-};
-static const char * const att_names32[] = {
-  "%eax", "%ecx", "%edx", "%ebx", "%esp", "%ebp", "%esi", "%edi",
-  "%r8d", "%r9d", "%r10d", "%r11d", "%r12d", "%r13d", "%r14d", "%r15d"
-};
-static const char * const att_names16[] = {
-  "%ax", "%cx", "%dx", "%bx", "%sp", "%bp", "%si", "%di",
-  "%r8w", "%r9w", "%r10w", "%r11w", "%r12w", "%r13w", "%r14w", "%r15w"
-};
-static const char * const att_names8[] = {
-  "%al", "%cl", "%dl", "%bl", "%ah", "%ch", "%dh", "%bh",
-};
-static const char * const att_names8rex[] = {
-  "%al", "%cl", "%dl", "%bl", "%spl", "%bpl", "%sil", "%dil",
-  "%r8b", "%r9b", "%r10b", "%r11b", "%r12b", "%r13b", "%r14b", "%r15b"
-};
-static const char * const att_names_seg[] = {
-  "%es", "%cs", "%ss", "%ds", "%fs", "%gs", "%?", "%?",
-};
-static const char * const att_index16[] = {
-  "%bx,%si", "%bx,%di", "%bp,%si", "%bp,%di", "%si", "%di", "%bp", "%bx"
-};
-
-static const struct dis386 grps[][8] = {
-  /* GRP1a */
-  {
-    { "popU",	{ stackEv } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-  },
-  /* GRP1b */
-  {
-    { "addA",	{ Eb, Ib } },
-    { "orA",	{ Eb, Ib } },
-    { "adcA",	{ Eb, Ib } },
-    { "sbbA",	{ Eb, Ib } },
-    { "andA",	{ Eb, Ib } },
-    { "subA",	{ Eb, Ib } },
-    { "xorA",	{ Eb, Ib } },
-    { "cmpA",	{ Eb, Ib } },
-  },
-  /* GRP1S */
-  {
-    { "addQ",	{ Ev, Iv } },
-    { "orQ",	{ Ev, Iv } },
-    { "adcQ",	{ Ev, Iv } },
-    { "sbbQ",	{ Ev, Iv } },
-    { "andQ",	{ Ev, Iv } },
-    { "subQ",	{ Ev, Iv } },
-    { "xorQ",	{ Ev, Iv } },
-    { "cmpQ",	{ Ev, Iv } },
-  },
-  /* GRP1Ss */
-  {
-    { "addQ",	{ Ev, sIb } },
-    { "orQ",	{ Ev, sIb } },
-    { "adcQ",	{ Ev, sIb } },
-    { "sbbQ",	{ Ev, sIb } },
-    { "andQ",	{ Ev, sIb } },
-    { "subQ",	{ Ev, sIb } },
-    { "xorQ",	{ Ev, sIb } },
-    { "cmpQ",	{ Ev, sIb } },
-  },
-  /* GRP2b */
-  {
-    { "rolA",	{ Eb, Ib } },
-    { "rorA",	{ Eb, Ib } },
-    { "rclA",	{ Eb, Ib } },
-    { "rcrA",	{ Eb, Ib } },
-    { "shlA",	{ Eb, Ib } },
-    { "shrA",	{ Eb, Ib } },
-    { "(bad)",	{ XX } },
-    { "sarA",	{ Eb, Ib } },
-  },
-  /* GRP2S */
-  {
-    { "rolQ",	{ Ev, Ib } },
-    { "rorQ",	{ Ev, Ib } },
-    { "rclQ",	{ Ev, Ib } },
-    { "rcrQ",	{ Ev, Ib } },
-    { "shlQ",	{ Ev, Ib } },
-    { "shrQ",	{ Ev, Ib } },
-    { "(bad)",	{ XX } },
-    { "sarQ",	{ Ev, Ib } },
-  },
-  /* GRP2b_one */
-  {
-    { "rolA",	{ Eb, I1 } },
-    { "rorA",	{ Eb, I1 } },
-    { "rclA",	{ Eb, I1 } },
-    { "rcrA",	{ Eb, I1 } },
-    { "shlA",	{ Eb, I1 } },
-    { "shrA",	{ Eb, I1 } },
-    { "(bad)",	{ XX } },
-    { "sarA",	{ Eb, I1 } },
-  },
-  /* GRP2S_one */
-  {
-    { "rolQ",	{ Ev, I1 } },
-    { "rorQ",	{ Ev, I1 } },
-    { "rclQ",	{ Ev, I1 } },
-    { "rcrQ",	{ Ev, I1 } },
-    { "shlQ",	{ Ev, I1 } },
-    { "shrQ",	{ Ev, I1 } },
-    { "(bad)",	{ XX } },
-    { "sarQ",	{ Ev, I1 } },
-  },
-  /* GRP2b_cl */
-  {
-    { "rolA",	{ Eb, CL } },
-    { "rorA",	{ Eb, CL } },
-    { "rclA",	{ Eb, CL } },
-    { "rcrA",	{ Eb, CL } },
-    { "shlA",	{ Eb, CL } },
-    { "shrA",	{ Eb, CL } },
-    { "(bad)",	{ XX } },
-    { "sarA",	{ Eb, CL } },
-  },
-  /* GRP2S_cl */
-  {
-    { "rolQ",	{ Ev, CL } },
-    { "rorQ",	{ Ev, CL } },
-    { "rclQ",	{ Ev, CL } },
-    { "rcrQ",	{ Ev, CL } },
-    { "shlQ",	{ Ev, CL } },
-    { "shrQ",	{ Ev, CL } },
-    { "(bad)",	{ XX } },
-    { "sarQ",	{ Ev, CL } },
-  },
-  /* GRP3b */
-  {
-    { "testA",	{ Eb, Ib } },
-    { "(bad)",	{ Eb } },
-    { "notA",	{ Eb } },
-    { "negA",	{ Eb } },
-    { "mulA",	{ Eb } },	/* Don't print the implicit %al register,  */
-    { "imulA",	{ Eb } },	/* to distinguish these opcodes from other */
-    { "divA",	{ Eb } },	/* mul/imul opcodes.  Do the same for div  */
-    { "idivA",	{ Eb } },	/* and idiv for consistency.		   */
-  },
-  /* GRP3S */
-  {
-    { "testQ",	{ Ev, Iv } },
-    { "(bad)",	{ XX } },
-    { "notQ",	{ Ev } },
-    { "negQ",	{ Ev } },
-    { "mulQ",	{ Ev } },	/* Don't print the implicit register.  */
-    { "imulQ",	{ Ev } },
-    { "divQ",	{ Ev } },
-    { "idivQ",	{ Ev } },
-  },
-  /* GRP4 */
-  {
-    { "incA",	{ Eb } },
-    { "decA",	{ Eb } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-  },
-  /* GRP5 */
-  {
-    { "incQ",	{ Ev } },
-    { "decQ",	{ Ev } },
-    { "callT",	{ indirEv } },
-    { "JcallT",	{ indirEp } },
-    { "jmpT",	{ indirEv } },
-    { "JjmpT",	{ indirEp } },
-    { "pushU",	{ stackEv } },
-    { "(bad)",	{ XX } },
-  },
-  /* GRP6 */
-  {
-    { "sldtD",	{ Sv } },
-    { "strD",	{ Sv } },
-    { "lldt",	{ Ew } },
-    { "ltr",	{ Ew } },
-    { "verr",	{ Ew } },
-    { "verw",	{ Ew } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-  },
-  /* GRP7 */
-  {
-    { "sgdt{Q|IQ||}", { { VMX_Fixup, 0 } } },
-    { "sidt{Q|IQ||}", { { PNI_Fixup, 0 } } },
-    { "lgdt{Q|Q||}",	 { M } },
-    { "lidt{Q|Q||}",	 { { SVME_Fixup, 0 } } },
-    { "smswD",	{ Sv } },
-    { "(bad)",	{ XX } },
-    { "lmsw",	{ Ew } },
-    { "invlpg",	{ { INVLPG_Fixup, w_mode } } },
-  },
-  /* GRP8 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "btQ",	{ Ev, Ib } },
-    { "btsQ",	{ Ev, Ib } },
-    { "btrQ",	{ Ev, Ib } },
-    { "btcQ",	{ Ev, Ib } },
-  },
-  /* GRP9 */
-  {
-    { "(bad)",	{ XX } },
-    { "cmpxchg8b", { { CMPXCHG8B_Fixup, q_mode } } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "",	{ VM } },		/* See OP_VMX.  */
-    { "vmptrst", { Mq } },
-  },
-  /* GRP11_C6 */
-  {
-    { "movA",	{ Eb, Ib } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-  },
-  /* GRP11_C7 */
-  {
-    { "movQ",	{ Ev, Iv } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",  { XX } },
-  },
-  /* GRP12 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "psrlw",	{ MS, Ib } },
-    { "(bad)",	{ XX } },
-    { "psraw",	{ MS, Ib } },
-    { "(bad)",	{ XX } },
-    { "psllw",	{ MS, Ib } },
-    { "(bad)",	{ XX } },
-  },
-  /* GRP13 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "psrld",	{ MS, Ib } },
-    { "(bad)",	{ XX } },
-    { "psrad",	{ MS, Ib } },
-    { "(bad)",	{ XX } },
-    { "pslld",	{ MS, Ib } },
-    { "(bad)",	{ XX } },
-  },
-  /* GRP14 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "psrlq",	{ MS, Ib } },
-    { "psrldq",	{ MS, Ib } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "psllq",	{ MS, Ib } },
-    { "pslldq",	{ MS, Ib } },
-  },
-  /* GRP15 */
-  {
-    { "fxsave",		{ Ev } },
-    { "fxrstor",	{ Ev } },
-    { "ldmxcsr",	{ Ev } },
-    { "stmxcsr",	{ Ev } },
-    { "(bad)",		{ XX } },
-    { "lfence",		{ { OP_0fae, 0 } } },
-    { "mfence",		{ { OP_0fae, 0 } } },
-    { "clflush",	{ { OP_0fae, 0 } } },
-  },
-  /* GRP16 */
-  {
-    { "prefetchnta",	{ Ev } },
-    { "prefetcht0",	{ Ev } },
-    { "prefetcht1",	{ Ev } },
-    { "prefetcht2",	{ Ev } },
-    { "(bad)",		{ XX } },
-    { "(bad)",		{ XX } },
-    { "(bad)",		{ XX } },
-    { "(bad)",		{ XX } },
-  },
-  /* GRPAMD */
-  {
-    { "prefetch",	{ Eb } },
-    { "prefetchw",	{ Eb } },
-    { "(bad)",		{ XX } },
-    { "(bad)",		{ XX } },
-    { "(bad)",		{ XX } },
-    { "(bad)",		{ XX } },
-    { "(bad)",		{ XX } },
-    { "(bad)",		{ XX } },
-  },
-  /* GRPPADLCK1 */
-  {
-    { "xstore-rng",	{ { OP_0f07, 0 } } },
-    { "xcrypt-ecb",	{ { OP_0f07, 0 } } },
-    { "xcrypt-cbc",	{ { OP_0f07, 0 } } },
-    { "xcrypt-ctr",	{ { OP_0f07, 0 } } },
-    { "xcrypt-cfb",	{ { OP_0f07, 0 } } },
-    { "xcrypt-ofb",	{ { OP_0f07, 0 } } },
-    { "(bad)",		{ { OP_0f07, 0 } } },
-    { "(bad)",		{ { OP_0f07, 0 } } },
-  },
-  /* GRPPADLCK2 */
-  {
-    { "montmul",	{ { OP_0f07, 0 } } },
-    { "xsha1",		{ { OP_0f07, 0 } } },
-    { "xsha256",	{ { OP_0f07, 0 } } },
-    { "(bad)",		{ { OP_0f07, 0 } } },
-    { "(bad)",		{ { OP_0f07, 0 } } },
-    { "(bad)",		{ { OP_0f07, 0 } } },
-    { "(bad)",		{ { OP_0f07, 0 } } },
-    { "(bad)",		{ { OP_0f07, 0 } } },
-  }
-};
-
-static const struct dis386 prefix_user_table[][4] = {
-  /* PREGRP0 */
-  {
-    { "addps", { XM, EXx } },
-    { "addss", { XM, EXd } },
-    { "addpd", { XM, EXx } },
-    { "addsd", { XM, EXq } },
-  },
-  /* PREGRP1 */
-  {
-    { "", { XM, EXx, OPSIMD } },	/* See OP_SIMD_SUFFIX.  */
-    { "", { XM, EXx, OPSIMD } },
-    { "", { XM, EXx, OPSIMD } },
-    { "", { XM, EXx, OPSIMD } },
-  },
-  /* PREGRP2 */
-  {
-    { "cvtpi2ps", { XM, EMC } },
-    { "cvtsi2ssY", { XM, Ev } },
-    { "cvtpi2pd", { XM, EMC } },
-    { "cvtsi2sdY", { XM, Ev } },
-  },
-  /* PREGRP3 */
-  {
-    { "cvtps2pi", { MXC, EXx } },
-    { "cvtss2siY", { Gv, EXx } },
-    { "cvtpd2pi", { MXC, EXx } },
-    { "cvtsd2siY", { Gv, EXx } },
-  },
-  /* PREGRP4 */
-  {
-    { "cvttps2pi", { MXC, EXx } },
-    { "cvttss2siY", { Gv, EXx } },
-    { "cvttpd2pi", { MXC, EXx } },
-    { "cvttsd2siY", { Gv, EXx } },
-  },
-  /* PREGRP5 */
-  {
-    { "divps",	{ XM, EXx } },
-    { "divss",	{ XM, EXx } },
-    { "divpd",	{ XM, EXx } },
-    { "divsd",	{ XM, EXx } },
-  },
-  /* PREGRP6 */
-  {
-    { "maxps",	{ XM, EXx } },
-    { "maxss",	{ XM, EXx } },
-    { "maxpd",	{ XM, EXx } },
-    { "maxsd",	{ XM, EXx } },
-  },
-  /* PREGRP7 */
-  {
-    { "minps",	{ XM, EXx } },
-    { "minss",	{ XM, EXx } },
-    { "minpd",	{ XM, EXx } },
-    { "minsd",	{ XM, EXx } },
-  },
-  /* PREGRP8 */
-  {
-    { "movups",	{ XM, EXx } },
-    { "movss",	{ XM, EXx } },
-    { "movupd",	{ XM, EXx } },
-    { "movsd",	{ XM, EXx } },
-  },
-  /* PREGRP9 */
-  {
-    { "movups",	{ EXx,  XM } },
-    { "movss",	{ EXx,  XM } },
-    { "movupd",	{ EXx,  XM } },
-    { "movsd",	{ EXx,  XM } },
-  },
-  /* PREGRP10 */
-  {
-    { "mulps",	{ XM, EXx } },
-    { "mulss",	{ XM, EXx } },
-    { "mulpd",	{ XM, EXx } },
-    { "mulsd",	{ XM, EXx } },
-  },
-  /* PREGRP11 */
-  {
-    { "rcpps",	{ XM, EXx } },
-    { "rcpss",	{ XM, EXx } },
-    { "(bad)",	{ XM, EXx } },
-    { "(bad)",	{ XM, EXx } },
-  },
-  /* PREGRP12 */
-  {
-    { "rsqrtps",{ XM, EXx } },
-    { "rsqrtss",{ XM, EXx } },
-    { "(bad)",	{ XM, EXx } },
-    { "(bad)",	{ XM, EXx } },
-  },
-  /* PREGRP13 */
-  {
-    { "sqrtps", { XM, EXx } },
-    { "sqrtss", { XM, EXx } },
-    { "sqrtpd", { XM, EXx } },
-    { "sqrtsd",	{ XM, EXx } },
-  },
-  /* PREGRP14 */
-  {
-    { "subps",	{ XM, EXx } },
-    { "subss",	{ XM, EXx } },
-    { "subpd",	{ XM, EXx } },
-    { "subsd",	{ XM, EXx } },
-  },
-  /* PREGRP15 */
-  {
-    { "(bad)",	{ XM, EXx } },
-    { "cvtdq2pd", { XM, EXq } },
-    { "cvttpd2dq", { XM, EXx } },
-    { "cvtpd2dq", { XM, EXx } },
-  },
-  /* PREGRP16 */
-  {
-    { "cvtdq2ps", { XM, EXx } },
-    { "cvttps2dq", { XM, EXx } },
-    { "cvtps2dq", { XM, EXx } },
-    { "(bad)",	{ XM, EXx } },
-  },
-  /* PREGRP17 */
-  {
-    { "cvtps2pd", { XM, EXq } },
-    { "cvtss2sd", { XM, EXx } },
-    { "cvtpd2ps", { XM, EXx } },
-    { "cvtsd2ss", { XM, EXx } },
-  },
-  /* PREGRP18 */
-  {
-    { "maskmovq", { MX, MS } },
-    { "(bad)",	{ XM, EXx } },
-    { "maskmovdqu", { XM, XS } },
-    { "(bad)",	{ XM, EXx } },
-  },
-  /* PREGRP19 */
-  {
-    { "movq",	{ MX, EM } },
-    { "movdqu",	{ XM, EXx } },
-    { "movdqa",	{ XM, EXx } },
-    { "(bad)",	{ XM, EXx } },
-  },
-  /* PREGRP20 */
-  {
-    { "movq",	{ EM, MX } },
-    { "movdqu",	{ EXx,  XM } },
-    { "movdqa",	{ EXx,  XM } },
-    { "(bad)",	{ EXx,  XM } },
-  },
-  /* PREGRP21 */
-  {
-    { "(bad)",	{ EXx,  XM } },
-    { "movq2dq",{ XM, MS } },
-    { "movq",	{ EXx,  XM } },
-    { "movdq2q",{ MX, XS } },
-  },
-  /* PREGRP22 */
-  {
-    { "pshufw",	{ MX, EM, Ib } },
-    { "pshufhw",{ XM, EXx, Ib } },
-    { "pshufd",	{ XM, EXx, Ib } },
-    { "pshuflw",{ XM, EXx, Ib } },
-  },
-  /* PREGRP23 */
-  {
-    { "movd",	{ Edq, MX } },
-    { "movq",	{ XM, EXx } },
-    { "movd",	{ Edq, XM } },
-    { "(bad)",	{ Ed, XM } },
-  },
-  /* PREGRP24 */
-  {
-    { "(bad)",	{ MX, EXx } },
-    { "(bad)",	{ XM, EXx } },
-    { "punpckhqdq", { XM, EXx } },
-    { "(bad)",	{ XM, EXx } },
-  },
-  /* PREGRP25 */
-  {
-    { "movntq",	{ EM, MX } },
-    { "(bad)",	{ EM, XM } },
-    { "movntdq",{ EM, XM } },
-    { "(bad)",	{ EM, XM } },
-  },
-  /* PREGRP26 */
-  {
-    { "(bad)",	{ MX, EXx } },
-    { "(bad)",	{ XM, EXx } },
-    { "punpcklqdq", { XM, EXx } },
-    { "(bad)",	{ XM, EXx } },
-  },
-  /* PREGRP27 */
-  {
-    { "(bad)",	{ MX, EXx } },
-    { "(bad)",	{ XM, EXx } },
-    { "addsubpd", { XM, EXx } },
-    { "addsubps", { XM, EXx } },
-  },
-  /* PREGRP28 */
-  {
-    { "(bad)",	{ MX, EXx } },
-    { "(bad)",	{ XM, EXx } },
-    { "haddpd",	{ XM, EXx } },
-    { "haddps",	{ XM, EXx } },
-  },
-  /* PREGRP29 */
-  {
-    { "(bad)",	{ MX, EXx } },
-    { "(bad)",	{ XM, EXx } },
-    { "hsubpd",	{ XM, EXx } },
-    { "hsubps",	{ XM, EXx } },
-  },
-  /* PREGRP30 */
-  {
-    { "movlpX",	{ XM, EXq, { SIMD_Fixup, 'h' } } }, /* really only 2 operands */
-    { "movsldup", { XM, EXx } },
-    { "movlpd",	{ XM, EXq } },
-    { "movddup", { XM, EXq } },
-  },
-  /* PREGRP31 */
-  {
-    { "movhpX",	{ XM, EXq, { SIMD_Fixup, 'l' } } },
-    { "movshdup", { XM, EXx } },
-    { "movhpd",	{ XM, EXq } },
-    { "(bad)",	{ XM, EXq } },
-  },
-  /* PREGRP32 */
-  {
-    { "(bad)",	{ XM, EXx } },
-    { "(bad)",	{ XM, EXx } },
-    { "(bad)",	{ XM, EXx } },
-    { "lddqu",	{ XM, M } },
-  },
-  /* PREGRP33 */
-  {
-    {"movntps", { Ev, XM } },
-    {"movntss", { Ev, XM } },
-    {"movntpd", { Ev, XM } },
-    {"movntsd", { Ev, XM } },
-  },
-
-  /* PREGRP34 */
-  {
-    {"vmread",	{ Em, Gm } },
-    {"(bad)",	{ XX } },
-    {"extrq",	{ XS, Ib, Ib } },
-    {"insertq",	{ XM, XS, Ib, Ib } },
-  },
-
- /* PREGRP35 */
-  {
-    {"vmwrite",	{ Gm, Em } },
-    {"(bad)",	{ XX } },
-    {"extrq",	{ XM, XS } },
-    {"insertq",	{ XM, XS } },
-  },
-
-  /* PREGRP36 */
-  {
-    { "bsrS",	{ Gv, Ev } },
-    { "lzcntS",	{ Gv, Ev } },
-    { "bsrS",	{ Gv, Ev } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP37 */
-  {
-    { "(bad)", { XX } },
-    { "popcntS", { Gv, Ev } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-  },
-
-  /* PREGRP38 */
-  {
-    { "xchgS", { { NOP_Fixup1, eAX_reg }, { NOP_Fixup2, eAX_reg } } },
-    { "pause", { XX } },
-    { "xchgS", { { NOP_Fixup1, eAX_reg }, { NOP_Fixup2, eAX_reg } } },
-    { "(bad)", { XX } },
-  },
-
-  /* PREGRP39 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pblendvb", {XM, EXx, XMM0 } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP40 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "blendvps", {XM, EXx, XMM0 } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP41 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "blendvpd", { XM, EXx, XMM0 } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP42 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "ptest",  { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP43 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmovsxbw", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP44 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmovsxbd", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP45 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmovsxbq", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP46 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmovsxwd", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP47 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmovsxwq", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP48 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmovsxdq", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP49 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmuldq", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP50 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pcmpeqq", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP51 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "movntdqa", { XM, EM } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP52 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "packusdw", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP53 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmovzxbw", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP54 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmovzxbd", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP55 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmovzxbq", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP56 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmovzxwd", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP57 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmovzxwq", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP58 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmovzxdq", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP59 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pminsb",	{ XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP60 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pminsd",	{ XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP61 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pminuw",	{ XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP62 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pminud",	{ XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP63 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmaxsb",	{ XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP64 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmaxsd",	{ XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP65 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmaxuw", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP66 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmaxud", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP67 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pmulld", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP68 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "phminposuw", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP69 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "roundps", { XM, EXx, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP70 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "roundpd", { XM, EXx, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP71 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "roundss", { XM, EXx, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP72 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "roundsd", { XM, EXx, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP73 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "blendps", { XM, EXx, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP74 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "blendpd", { XM, EXx, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP75 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pblendw", { XM, EXx, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP76 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pextrb",	{ Edqb, XM, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP77 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pextrw",	{ Edqw, XM, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP78 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pextrK",	{ Edq, XM, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP79 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "extractps", { Edqd, XM, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP80 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pinsrb",	{ XM, Edqb, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP81 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "insertps", { XM, EXx, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP82 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pinsrK",	{ XM, Edq, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP83 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "dpps",	{ XM, EXx, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP84 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "dppd",	{ XM, EXx, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP85 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "mpsadbw", { XM, EXx, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP86 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pcmpgtq", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP87 */
-  {
-    { "movbe",	{ Gv, Ev } },
-    { "(bad)",	{ XX } },
-    { "movbe",	{ Gv, Ev } },
-    { "crc32",	{ Gdq, { CRC32_Fixup, b_mode } } },
-  },
-
-  /* PREGRP88 */
-  {
-    { "movbe",	{ Ev, Gv } },
-    { "(bad)",	{ XX } },
-    { "movbe",	{ Ev, Gv } },
-    { "crc32",	{ Gdq, { CRC32_Fixup, v_mode } } },
-  },
-
-  /* PREGRP89 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pcmpestrm", { XM, EXx, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP90 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pcmpestri", { XM, EXx, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP91 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pcmpistrm", { XM, EXx, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP92 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pcmpistri", { XM, EXx, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP93 */
-  {
-    { "ucomiss",{ XM, EXd } },
-    { "(bad)",	{ XX } },
-    { "ucomisd",{ XM, EXq } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP94 */
-  {
-    { "comiss",	{ XM, EXd } },
-    { "(bad)",	{ XX } },
-    { "comisd",	{ XM, EXq } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP95 */
-  {
-    { "punpcklbw",{ MX, EMd } },
-    { "(bad)",	{ XX } },
-    { "punpcklbw",{ MX, EMq } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP96 */
-  {
-    { "punpcklwd",{ MX, EMd } },
-    { "(bad)",	{ XX } },
-    { "punpcklwd",{ MX, EMq } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP97 */
-  {
-    { "punpckldq",{ MX, EMd } },
-    { "(bad)",	{ XX } },
-    { "punpckldq",{ MX, EMq } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP98 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "pclmulqdq", { XM, EXx, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP99 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "aesimc", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP100 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "aesenc", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP101 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "aesenclast", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP102 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "aesdec", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP103 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "aesdeclast", { XM, EXx } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP104 */
-  {
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "aeskeygenassist", { XM, EXx, Ib } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP105 */
-  {
-    { "andnS",	{ Gv, Bv, Ev } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP106 */
-  {
-    { "bextrS",	{ Gv, Ev, Bv } },
-    { "sarxS",	{ Gv, Ev, Bv } },
-    { "shlxS",	{ Gv, Ev, Bv } },
-    { "shrxS",	{ Gv, Ev, Bv } },
-  },
-
-  /* PREGRP107 */
-  {
-    { "bsfS",	{ Gv, Ev } },
-    { "tzcntS",	{ Gv, Ev } },
-    { "bsfS",	{ Gv, Ev } },
-    { "(bad)",	{ XX } },
-  },
-
-  /* PREGRP108 */
-  {
-    { "bzhi",   { Gv, Ev, Bv } },
-    { "pext",   { Gv, Bv, Ev } },
-    { "(bad)",  { XX } },
-    { "pdep",   { Gv, Bv, Ev } },
-  },
-
-  /* PREGRP109 */
-  {
-    { "(bad)",  { XX } },
-    { "(bad)",  { XX } },
-    { "(bad)",  { XX } },
-    { "rorx",   { Gv, Ev, Ib } },
-  },
-};
-
-static const struct dis386 x86_64_table[][2] = {
-  {
-    { "pusha{P|}", { XX } },
-    { "(bad)", { XX } },
-  },
-  {
-    { "popa{P|}", { XX } },
-    { "(bad)", { XX } },
-  },
-  {
-    { "bound{S|}", { Gv, Ma } },
-    { "(bad)", { XX } },
-  },
-  {
-    { "arpl", { Ew, Gw } },
-    { "movs{||lq|xd}", { Gv, Ed } },
-  },
-};
-
-static const struct dis386 three_byte_table[][256] = {
-  /* THREE_BYTE_0 */
-  {
-    /* 00 */
-    { "pshufb", { MX, EM } },
-    { "phaddw", { MX, EM } },
-    { "phaddd",	{ MX, EM } },
-    { "phaddsw", { MX, EM } },
-    { "pmaddubsw", { MX, EM } },
-    { "phsubw", { MX, EM } },
-    { "phsubd", { MX, EM } },
-    { "phsubsw", { MX, EM } },
-    /* 08 */
-    { "psignb", { MX, EM } },
-    { "psignw", { MX, EM } },
-    { "psignd", { MX, EM } },
-    { "pmulhrsw", { MX, EM } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 10 */
-    { PREGRP39 },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { PREGRP40 },
-    { PREGRP41 },
-    { "(bad)", { XX } },
-    { PREGRP42 },
-    /* 18 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "pabsb", { MX, EM } },
-    { "pabsw", { MX, EM } },
-    { "pabsd", { MX, EM } },
-    { "(bad)", { XX } },
-    /* 20 */
-    { PREGRP43 },
-    { PREGRP44 },
-    { PREGRP45 },
-    { PREGRP46 },
-    { PREGRP47 },
-    { PREGRP48 },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 28 */
-    { PREGRP49 },
-    { PREGRP50 },
-    { PREGRP51 },
-    { PREGRP52 },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 30 */
-    { PREGRP53 },
-    { PREGRP54 },
-    { PREGRP55 },
-    { PREGRP56 },
-    { PREGRP57 },
-    { PREGRP58 },
-    { "(bad)", { XX } },
-    { PREGRP86 },
-    /* 38 */
-    { PREGRP59 },
-    { PREGRP60 },
-    { PREGRP61 },
-    { PREGRP62 },
-    { PREGRP63 },
-    { PREGRP64 },
-    { PREGRP65 },
-    { PREGRP66 },
-    /* 40 */
-    { PREGRP67 },
-    { PREGRP68 },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 48 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 50 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 58 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 60 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 68 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 70 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 78 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 80 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 88 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 90 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 98 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* a0 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* a8 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* b0 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* b8 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* c0 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* c8 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* d0 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* d8 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { PREGRP99 },
-    { PREGRP100 },
-    { PREGRP101 },
-    { PREGRP102 },
-    { PREGRP103 },
-    /* e0 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* e8 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* f0 */
-    { PREGRP87 },
-    { PREGRP88 },
-    { PREGRP105 },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { PREGRP108 },
-    { "(bad)", { XX } },
-    { PREGRP106 },
-    /* f8 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-  },
-  /* THREE_BYTE_1 */
-  {
-    /* 00 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 08 */
-    { PREGRP69 },
-    { PREGRP70 },
-    { PREGRP71 },
-    { PREGRP72 },
-    { PREGRP73 },
-    { PREGRP74 },
-    { PREGRP75 },
-    { "palignr", { MX, EM, Ib } },
-    /* 10 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { PREGRP76 },
-    { PREGRP77 },
-    { PREGRP78 },
-    { PREGRP79 },
-    /* 18 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 20 */
-    { PREGRP80 },
-    { PREGRP81 },
-    { PREGRP82 },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 28 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 30 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 38 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 40 */
-    { PREGRP83 },
-    { PREGRP84 },
-    { PREGRP85 },
-    { "(bad)", { XX } },
-    { PREGRP98 },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 48 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 50 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 58 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 60 */
-    { PREGRP89 },
-    { PREGRP90 },
-    { PREGRP91 },
-    { PREGRP92 },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 68 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 70 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 78 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 80 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 88 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 90 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* 98 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* a0 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* a8 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* b0 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* b8 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* c0 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* c8 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* d0 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* d8 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { PREGRP104 },
-    /* e0 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* e8 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* f0 */
-    { PREGRP109 },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    /* f8 */
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-    { "(bad)", { XX } },
-  }
-};
-
-#define INTERNAL_DISASSEMBLER_ERROR "<internal disassembler error>"
-
-static void
-ckprefix (void)
-{
-  int newrex;
-  rex = 0;
-  prefixes = 0;
-  used_prefixes = 0;
-  rex_used = 0;
-  while (1)
-    {
-      fetch_data(the_info, codep + 1);
-      newrex = 0;
-      switch (*codep)
-	{
-	/* REX prefixes family.  */
-	case 0x40:
-	case 0x41:
-	case 0x42:
-	case 0x43:
-	case 0x44:
-	case 0x45:
-	case 0x46:
-	case 0x47:
-	case 0x48:
-	case 0x49:
-	case 0x4a:
-	case 0x4b:
-	case 0x4c:
-	case 0x4d:
-	case 0x4e:
-	case 0x4f:
-	    if (address_mode == mode_64bit)
-	      newrex = *codep;
-	    else
-	      return;
-	  break;
-	case 0xf3:
-	  prefixes |= PREFIX_REPZ;
-	  break;
-	case 0xf2:
-	  prefixes |= PREFIX_REPNZ;
-	  break;
-	case 0xf0:
-	  prefixes |= PREFIX_LOCK;
-	  break;
-	case 0x2e:
-	  prefixes |= PREFIX_CS;
-	  break;
-	case 0x36:
-	  prefixes |= PREFIX_SS;
-	  break;
-	case 0x3e:
-	  prefixes |= PREFIX_DS;
-	  break;
-	case 0x26:
-	  prefixes |= PREFIX_ES;
-	  break;
-	case 0x64:
-	  prefixes |= PREFIX_FS;
-	  break;
-	case 0x65:
-	  prefixes |= PREFIX_GS;
-	  break;
-	case 0x66:
-	  prefixes |= PREFIX_DATA;
-	  break;
-	case 0x67:
-	  prefixes |= PREFIX_ADDR;
-	  break;
-	case FWAIT_OPCODE:
-	  /* fwait is really an instruction.  If there are prefixes
-	     before the fwait, they belong to the fwait, *not* to the
-	     following instruction.  */
-	  if (prefixes || rex)
-	    {
-	      prefixes |= PREFIX_FWAIT;
-	      codep++;
-	      return;
-	    }
-	  prefixes = PREFIX_FWAIT;
-	  break;
-	default:
-	  return;
-	}
-      /* Rex is ignored when followed by another prefix.  */
-      if (rex)
-	{
-	  rex_used = rex;
-	  return;
-	}
-      rex = newrex;
-      codep++;
-    }
-}
-
-static void
-ckvexprefix (void)
-{
-    int op, vex2, vex3, newrex = 0, newpfx = prefixes;
-
-    if (address_mode == mode_16bit) {
-        return;
-    }
-
-    fetch_data(the_info, codep + 1);
-    op = *codep;
-
-    if (op != 0xc4 && op != 0xc5) {
-        return;
-    }
-
-    fetch_data(the_info, codep + 2);
-    vex2 = codep[1];
-
-    if (address_mode == mode_32bit && (vex2 & 0xc0) != 0xc0) {
-        return;
-    }
-
-    if (op == 0xc4) {
-        /* Three byte VEX prefix.  */
-        fetch_data(the_info, codep + 3);
-        vex3 = codep[2];
-
-        newrex |= (vex2 & 0x80 ? 0 : REX_R);
-        newrex |= (vex2 & 0x40 ? 0 : REX_X);
-        newrex |= (vex2 & 0x20 ? 0 : REX_B);
-        newrex |= (vex3 & 0x80 ? REX_W : 0);
-        switch (vex2 & 0x1f) {      /* VEX.m-mmmm */
-        case 1:
-            newpfx |= PREFIX_VEX_0F;
-            break;
-        case 2:
-            newpfx |= PREFIX_VEX_0F | PREFIX_VEX_0F38;
-            break;
-        case 3:
-            newpfx |= PREFIX_VEX_0F | PREFIX_VEX_0F3A;
-            break;
-        }
-        vex2 = vex3;
-        codep += 3;
-    } else {
-        /* Two byte VEX prefix.  */
-        newrex |= (vex2 & 0x80 ? 0 : REX_R);
-        newpfx |= PREFIX_VEX_0F;
-        codep += 2;
-    }
-
-    vex_reg = (~vex2 >> 3) & 15;     /* VEX.vvvv */
-    switch (vex2 & 3) {              /* VEX.pp */
-    case 1:
-        newpfx |= PREFIX_DATA;     /* 0x66 */
-        break;
-    case 2:
-        newpfx |= PREFIX_REPZ;     /* 0xf3 */
-        break;
-    case 3:
-        newpfx |= PREFIX_REPNZ;    /* 0xf2 */
-        break;
-    }
-
-    rex = newrex;
-    prefixes = newpfx;
-}
-
-/* Return the name of the prefix byte PREF, or NULL if PREF is not a
-   prefix byte.  */
-
-static const char *
-prefix_name (int pref, int sizeflag)
-{
-  static const char * const rexes [16] =
-    {
-      "rex",		/* 0x40 */
-      "rex.B",		/* 0x41 */
-      "rex.X",		/* 0x42 */
-      "rex.XB",		/* 0x43 */
-      "rex.R",		/* 0x44 */
-      "rex.RB",		/* 0x45 */
-      "rex.RX",		/* 0x46 */
-      "rex.RXB",	/* 0x47 */
-      "rex.W",		/* 0x48 */
-      "rex.WB",		/* 0x49 */
-      "rex.WX",		/* 0x4a */
-      "rex.WXB",	/* 0x4b */
-      "rex.WR",		/* 0x4c */
-      "rex.WRB",	/* 0x4d */
-      "rex.WRX",	/* 0x4e */
-      "rex.WRXB",	/* 0x4f */
-    };
-
-  switch (pref)
-    {
-    /* REX prefixes family.  */
-    case 0x40:
-    case 0x41:
-    case 0x42:
-    case 0x43:
-    case 0x44:
-    case 0x45:
-    case 0x46:
-    case 0x47:
-    case 0x48:
-    case 0x49:
-    case 0x4a:
-    case 0x4b:
-    case 0x4c:
-    case 0x4d:
-    case 0x4e:
-    case 0x4f:
-      return rexes [pref - 0x40];
-    case 0xf3:
-      return "repz";
-    case 0xf2:
-      return "repnz";
-    case 0xf0:
-      return "lock";
-    case 0x2e:
-      return "cs";
-    case 0x36:
-      return "ss";
-    case 0x3e:
-      return "ds";
-    case 0x26:
-      return "es";
-    case 0x64:
-      return "fs";
-    case 0x65:
-      return "gs";
-    case 0x66:
-      return (sizeflag & DFLAG) ? "data16" : "data32";
-    case 0x67:
-      if (address_mode == mode_64bit)
-	return (sizeflag & AFLAG) ? "addr32" : "addr64";
-      else
-	return (sizeflag & AFLAG) ? "addr16" : "addr32";
-    case FWAIT_OPCODE:
-      return "fwait";
-    default:
-      return NULL;
-    }
-}
-
-static char op_out[MAX_OPERANDS][100];
-static int op_ad, op_index[MAX_OPERANDS];
-static int two_source_ops;
-static bfd_vma op_address[MAX_OPERANDS];
-static bfd_vma op_riprel[MAX_OPERANDS];
-static bfd_vma start_pc;
-
-/*
- *   On the 386's of 1988, the maximum length of an instruction is 15 bytes.
- *   (see topic "Redundant prefixes" in the "Differences from 8086"
- *   section of the "Virtual 8086 Mode" chapter.)
- * 'pc' should be the address of this instruction, it will
- *   be used to print the target address if this is a relative jump or call
- * The function returns the length of this instruction in bytes.
- */
-
-static char intel_syntax;
-static char open_char;
-static char close_char;
-static char separator_char;
-static char scale_char;
-
-int
-print_insn_i386 (bfd_vma pc, disassemble_info *info)
-{
-  intel_syntax = -1;
-
-  return print_insn (pc, info);
-}
-
-static int
-print_insn (bfd_vma pc, disassemble_info *info)
-{
-  const struct dis386 *dp;
-  int i;
-  char *op_txt[MAX_OPERANDS];
-  int needcomma;
-  unsigned char uses_DATA_prefix, uses_LOCK_prefix;
-  unsigned char uses_REPNZ_prefix, uses_REPZ_prefix;
-  int sizeflag;
-  const char *p;
-  struct dis_private priv;
-  unsigned char op;
-  unsigned char threebyte;
-
-  if (info->mach == bfd_mach_x86_64_intel_syntax
-      || info->mach == bfd_mach_x86_64)
-    address_mode = mode_64bit;
-  else
-    address_mode = mode_32bit;
-
-  if (intel_syntax == (char) -1)
-    intel_syntax = (info->mach == bfd_mach_i386_i386_intel_syntax
-		    || info->mach == bfd_mach_x86_64_intel_syntax);
-
-  if (info->mach == bfd_mach_i386_i386
-      || info->mach == bfd_mach_x86_64
-      || info->mach == bfd_mach_i386_i386_intel_syntax
-      || info->mach == bfd_mach_x86_64_intel_syntax)
-    priv.orig_sizeflag = AFLAG | DFLAG;
-  else if (info->mach == bfd_mach_i386_i8086)
-    priv.orig_sizeflag = 0;
-  else
-    abort ();
-
-  for (p = info->disassembler_options; p != NULL; )
-    {
-      if (strncmp (p, "x86-64", 6) == 0)
-	{
-	  address_mode = mode_64bit;
-	  priv.orig_sizeflag = AFLAG | DFLAG;
-	}
-      else if (strncmp (p, "i386", 4) == 0)
-	{
-	  address_mode = mode_32bit;
-	  priv.orig_sizeflag = AFLAG | DFLAG;
-	}
-      else if (strncmp (p, "i8086", 5) == 0)
-	{
-	  address_mode = mode_16bit;
-	  priv.orig_sizeflag = 0;
-	}
-      else if (strncmp (p, "intel", 5) == 0)
-	{
-	  intel_syntax = 1;
-	}
-      else if (strncmp (p, "att", 3) == 0)
-	{
-	  intel_syntax = 0;
-	}
-      else if (strncmp (p, "addr", 4) == 0)
-	{
-	  if (address_mode == mode_64bit)
-	    {
-	      if (p[4] == '3' && p[5] == '2')
-		priv.orig_sizeflag &= ~AFLAG;
-	      else if (p[4] == '6' && p[5] == '4')
-		priv.orig_sizeflag |= AFLAG;
-	    }
-	  else
-	    {
-	      if (p[4] == '1' && p[5] == '6')
-		priv.orig_sizeflag &= ~AFLAG;
-	      else if (p[4] == '3' && p[5] == '2')
-		priv.orig_sizeflag |= AFLAG;
-	    }
-	}
-      else if (strncmp (p, "data", 4) == 0)
-	{
-	  if (p[4] == '1' && p[5] == '6')
-	    priv.orig_sizeflag &= ~DFLAG;
-	  else if (p[4] == '3' && p[5] == '2')
-	    priv.orig_sizeflag |= DFLAG;
-	}
-      else if (strncmp (p, "suffix", 6) == 0)
-	priv.orig_sizeflag |= SUFFIX_ALWAYS;
-
-      p = strchr (p, ',');
-      if (p != NULL)
-	p++;
-    }
-
-  if (intel_syntax)
-    {
-      names64 = intel_names64;
-      names32 = intel_names32;
-      names16 = intel_names16;
-      names8 = intel_names8;
-      names8rex = intel_names8rex;
-      names_seg = intel_names_seg;
-      index16 = intel_index16;
-      open_char = '[';
-      close_char = ']';
-      separator_char = '+';
-      scale_char = '*';
-    }
-  else
-    {
-      names64 = att_names64;
-      names32 = att_names32;
-      names16 = att_names16;
-      names8 = att_names8;
-      names8rex = att_names8rex;
-      names_seg = att_names_seg;
-      index16 = att_index16;
-      open_char = '(';
-      close_char =  ')';
-      separator_char = ',';
-      scale_char = ',';
-    }
-
-  /* The output looks better if we put 7 bytes on a line, since that
-     puts most long word instructions on a single line.  */
-  info->bytes_per_line = 7;
-
-  info->private_data = &priv;
-  priv.max_fetched = priv.the_buffer;
-  priv.insn_start = pc;
-
-  obuf[0] = 0;
-  for (i = 0; i < MAX_OPERANDS; ++i)
-    {
-      op_out[i][0] = 0;
-      op_index[i] = -1;
-    }
-
-  the_info = info;
-  start_pc = pc;
-  start_codep = priv.the_buffer;
-  codep = priv.the_buffer;
-
-  if (sigsetjmp(priv.bailout, 0) != 0)
-    {
-      const char *name;
-
-      /* Getting here means we tried for data but didn't get it.  That
-	 means we have an incomplete instruction of some sort.  Just
-	 print the first byte as a prefix or a .byte pseudo-op.  */
-      if (codep > priv.the_buffer)
-	{
-	  name = prefix_name (priv.the_buffer[0], priv.orig_sizeflag);
-	  if (name != NULL)
-	    (*info->fprintf_func) (info->stream, "%s", name);
-	  else
-	    {
-	      /* Just print the first byte as a .byte instruction.  */
-	      (*info->fprintf_func) (info->stream, ".byte 0x%x",
-				     (unsigned int) priv.the_buffer[0]);
-	    }
-
-	  return 1;
-	}
-
-      return -1;
-    }
-
-  obufp = obuf;
-  ckprefix ();
-  ckvexprefix ();
-
-  insn_codep = codep;
-  sizeflag = priv.orig_sizeflag;
-
-  fetch_data(info, codep + 1);
-  two_source_ops = (*codep == 0x62) || (*codep == 0xc8);
-
-  if (((prefixes & PREFIX_FWAIT)
-       && ((*codep < 0xd8) || (*codep > 0xdf)))
-      || (rex && rex_used))
-    {
-      const char *name;
-
-      /* fwait not followed by floating point instruction, or rex followed
-	 by other prefixes.  Print the first prefix.  */
-      name = prefix_name (priv.the_buffer[0], priv.orig_sizeflag);
-      if (name == NULL)
-	name = INTERNAL_DISASSEMBLER_ERROR;
-      (*info->fprintf_func) (info->stream, "%s", name);
-      return 1;
-    }
-
-  op = 0;
-  if (prefixes & PREFIX_VEX_0F)
-    {
-      used_prefixes |= PREFIX_VEX_0F | PREFIX_VEX_0F38 | PREFIX_VEX_0F3A;
-      if (prefixes & PREFIX_VEX_0F38)
-        threebyte = 0x38;
-      else if (prefixes & PREFIX_VEX_0F3A)
-        threebyte = 0x3a;
-      else
-        threebyte = *codep++;
-      goto vex_opcode;
-    }
-  if (*codep == 0x0f)
-    {
-      fetch_data(info, codep + 2);
-      threebyte = codep[1];
-      codep += 2;
-    vex_opcode:
-      dp = &dis386_twobyte[threebyte];
-      need_modrm = twobyte_has_modrm[threebyte];
-      uses_DATA_prefix = twobyte_uses_DATA_prefix[threebyte];
-      uses_REPNZ_prefix = twobyte_uses_REPNZ_prefix[threebyte];
-      uses_REPZ_prefix = twobyte_uses_REPZ_prefix[threebyte];
-      uses_LOCK_prefix = (threebyte & ~0x02) == 0x20;
-      if (dp->name == NULL && dp->op[0].bytemode == IS_3BYTE_OPCODE)
-	{
-          fetch_data(info, codep + 2);
-	  op = *codep++;
-	  switch (threebyte)
-	    {
-	    case 0x38:
-	      uses_DATA_prefix = threebyte_0x38_uses_DATA_prefix[op];
-	      uses_REPNZ_prefix = threebyte_0x38_uses_REPNZ_prefix[op];
-	      uses_REPZ_prefix = threebyte_0x38_uses_REPZ_prefix[op];
-	      break;
-	    case 0x3a:
-	      uses_DATA_prefix = threebyte_0x3a_uses_DATA_prefix[op];
-	      uses_REPNZ_prefix = threebyte_0x3a_uses_REPNZ_prefix[op];
-	      uses_REPZ_prefix = threebyte_0x3a_uses_REPZ_prefix[op];
-	      break;
-	    default:
-	      break;
-	    }
-	}
-    }
-  else
-    {
-      dp = &dis386[*codep];
-      need_modrm = onebyte_has_modrm[*codep];
-      uses_DATA_prefix = 0;
-      uses_REPNZ_prefix = 0;
-      /* pause is 0xf3 0x90.  */
-      uses_REPZ_prefix = *codep == 0x90;
-      uses_LOCK_prefix = 0;
-      codep++;
-    }
-
-  if (!uses_REPZ_prefix && (prefixes & PREFIX_REPZ))
-    {
-      oappend ("repz ");
-      used_prefixes |= PREFIX_REPZ;
-    }
-  if (!uses_REPNZ_prefix && (prefixes & PREFIX_REPNZ))
-    {
-      oappend ("repnz ");
-      used_prefixes |= PREFIX_REPNZ;
-    }
-
-  if (!uses_LOCK_prefix && (prefixes & PREFIX_LOCK))
-    {
-      oappend ("lock ");
-      used_prefixes |= PREFIX_LOCK;
-    }
-
-  if (prefixes & PREFIX_ADDR)
-    {
-      sizeflag ^= AFLAG;
-      if (dp->op[2].bytemode != loop_jcxz_mode || intel_syntax)
-	{
-	  if ((sizeflag & AFLAG) || address_mode == mode_64bit)
-	    oappend ("addr32 ");
-	  else
-	    oappend ("addr16 ");
-	  used_prefixes |= PREFIX_ADDR;
-	}
-    }
-
-  if (!uses_DATA_prefix && (prefixes & PREFIX_DATA))
-    {
-      sizeflag ^= DFLAG;
-      if (dp->op[2].bytemode == cond_jump_mode
-	  && dp->op[0].bytemode == v_mode
-	  && !intel_syntax)
-	{
-	  if (sizeflag & DFLAG)
-	    oappend ("data32 ");
-	  else
-	    oappend ("data16 ");
-	  used_prefixes |= PREFIX_DATA;
-	}
-    }
-
-  if (dp->name == NULL && dp->op[0].bytemode == IS_3BYTE_OPCODE)
-    {
-      dp = &three_byte_table[dp->op[1].bytemode][op];
-      modrm.mod = (*codep >> 6) & 3;
-      modrm.reg = (*codep >> 3) & 7;
-      modrm.rm = *codep & 7;
-    }
-  else if (need_modrm)
-    {
-      fetch_data(info, codep + 1);
-      modrm.mod = (*codep >> 6) & 3;
-      modrm.reg = (*codep >> 3) & 7;
-      modrm.rm = *codep & 7;
-    }
-
-  if (dp->name == NULL && dp->op[0].bytemode == FLOATCODE)
-    {
-      dofloat (sizeflag);
-    }
-  else
-    {
-      int index;
-      if (dp->name == NULL)
-	{
-	  switch (dp->op[0].bytemode)
-	    {
-	    case USE_GROUPS:
-	      dp = &grps[dp->op[1].bytemode][modrm.reg];
-	      break;
-
-	    case USE_PREFIX_USER_TABLE:
-	      index = 0;
-	      used_prefixes |= (prefixes & PREFIX_REPZ);
-	      if (prefixes & PREFIX_REPZ)
-		index = 1;
-	      else
-		{
-		  /* We should check PREFIX_REPNZ and PREFIX_REPZ
-		     before PREFIX_DATA.  */
-		  used_prefixes |= (prefixes & PREFIX_REPNZ);
-		  if (prefixes & PREFIX_REPNZ)
-		    index = 3;
-		  else
-		    {
-		      used_prefixes |= (prefixes & PREFIX_DATA);
-		      if (prefixes & PREFIX_DATA)
-			index = 2;
-		    }
-		}
-	      dp = &prefix_user_table[dp->op[1].bytemode][index];
-	      break;
-
-	    case X86_64_SPECIAL:
-	      index = address_mode == mode_64bit ? 1 : 0;
-	      dp = &x86_64_table[dp->op[1].bytemode][index];
-	      break;
-
-	    default:
-	      oappend (INTERNAL_DISASSEMBLER_ERROR);
-	      break;
-	    }
-	}
-
-      if (dp->name != NULL && putop (dp->name, sizeflag) == 0)
-        {
-	  for (i = 0; i < MAX_OPERANDS; ++i)
-	    {
-	      obufp = op_out[i];
-	      op_ad = MAX_OPERANDS - 1 - i;
-	      if (dp->op[i].rtn)
-		(*dp->op[i].rtn) (dp->op[i].bytemode, sizeflag);
-	    }
-	}
-    }
-
-  /* See if any prefixes were not used.  If so, print the first one
-     separately.  If we don't do this, we'll wind up printing an
-     instruction stream which does not precisely correspond to the
-     bytes we are disassembling.  */
-  if ((prefixes & ~used_prefixes) != 0)
-    {
-      const char *name;
-
-      name = prefix_name (priv.the_buffer[0], priv.orig_sizeflag);
-      if (name == NULL)
-	name = INTERNAL_DISASSEMBLER_ERROR;
-      (*info->fprintf_func) (info->stream, "%s", name);
-      return 1;
-    }
-  if (rex & ~rex_used)
-    {
-      const char *name;
-      name = prefix_name (rex | 0x40, priv.orig_sizeflag);
-      if (name == NULL)
-	name = INTERNAL_DISASSEMBLER_ERROR;
-      (*info->fprintf_func) (info->stream, "%s ", name);
-    }
-
-  obufp = obuf + strlen (obuf);
-  for (i = strlen (obuf); i < 6; i++)
-    oappend (" ");
-  oappend (" ");
-  (*info->fprintf_func) (info->stream, "%s", obuf);
-
-  /* The enter and bound instructions are printed with operands in the same
-     order as the intel book; everything else is printed in reverse order.  */
-  if (intel_syntax || two_source_ops)
-    {
-      bfd_vma riprel;
-
-      for (i = 0; i < MAX_OPERANDS; ++i)
-        op_txt[i] = op_out[i];
-
-      for (i = 0; i < (MAX_OPERANDS >> 1); ++i)
-	{
-          op_ad = op_index[i];
-          op_index[i] = op_index[MAX_OPERANDS - 1 - i];
-          op_index[MAX_OPERANDS - 1 - i] = op_ad;
-	  riprel = op_riprel[i];
-	  op_riprel[i] = op_riprel [MAX_OPERANDS - 1 - i];
-	  op_riprel[MAX_OPERANDS - 1 - i] = riprel;
-	}
-    }
-  else
-    {
-      for (i = 0; i < MAX_OPERANDS; ++i)
-        op_txt[MAX_OPERANDS - 1 - i] = op_out[i];
-    }
-
-  needcomma = 0;
-  for (i = 0; i < MAX_OPERANDS; ++i)
-    if (*op_txt[i])
-      {
-	if (needcomma)
-	  (*info->fprintf_func) (info->stream, ",");
-	if (op_index[i] != -1 && !op_riprel[i])
-	  (*info->print_address_func) ((bfd_vma) op_address[op_index[i]], info);
-	else
-	  (*info->fprintf_func) (info->stream, "%s", op_txt[i]);
-	needcomma = 1;
-      }
-
-  for (i = 0; i < MAX_OPERANDS; i++)
-    if (op_index[i] != -1 && op_riprel[i])
-      {
-	(*info->fprintf_func) (info->stream, "        # ");
-	(*info->print_address_func) ((bfd_vma) (start_pc + codep - start_codep
-						+ op_address[op_index[i]]), info);
-	break;
-      }
-  return codep - priv.the_buffer;
-}
-
-static const char *float_mem[] = {
-  /* d8 */
-  "fadd{s||s|}",
-  "fmul{s||s|}",
-  "fcom{s||s|}",
-  "fcomp{s||s|}",
-  "fsub{s||s|}",
-  "fsubr{s||s|}",
-  "fdiv{s||s|}",
-  "fdivr{s||s|}",
-  /* d9 */
-  "fld{s||s|}",
-  "(bad)",
-  "fst{s||s|}",
-  "fstp{s||s|}",
-  "fldenvIC",
-  "fldcw",
-  "fNstenvIC",
-  "fNstcw",
-  /* da */
-  "fiadd{l||l|}",
-  "fimul{l||l|}",
-  "ficom{l||l|}",
-  "ficomp{l||l|}",
-  "fisub{l||l|}",
-  "fisubr{l||l|}",
-  "fidiv{l||l|}",
-  "fidivr{l||l|}",
-  /* db */
-  "fild{l||l|}",
-  "fisttp{l||l|}",
-  "fist{l||l|}",
-  "fistp{l||l|}",
-  "(bad)",
-  "fld{t||t|}",
-  "(bad)",
-  "fstp{t||t|}",
-  /* dc */
-  "fadd{l||l|}",
-  "fmul{l||l|}",
-  "fcom{l||l|}",
-  "fcomp{l||l|}",
-  "fsub{l||l|}",
-  "fsubr{l||l|}",
-  "fdiv{l||l|}",
-  "fdivr{l||l|}",
-  /* dd */
-  "fld{l||l|}",
-  "fisttp{ll||ll|}",
-  "fst{l||l|}",
-  "fstp{l||l|}",
-  "frstorIC",
-  "(bad)",
-  "fNsaveIC",
-  "fNstsw",
-  /* de */
-  "fiadd",
-  "fimul",
-  "ficom",
-  "ficomp",
-  "fisub",
-  "fisubr",
-  "fidiv",
-  "fidivr",
-  /* df */
-  "fild",
-  "fisttp",
-  "fist",
-  "fistp",
-  "fbld",
-  "fild{ll||ll|}",
-  "fbstp",
-  "fistp{ll||ll|}",
-};
-
-static const unsigned char float_mem_mode[] = {
-  /* d8 */
-  d_mode,
-  d_mode,
-  d_mode,
-  d_mode,
-  d_mode,
-  d_mode,
-  d_mode,
-  d_mode,
-  /* d9 */
-  d_mode,
-  0,
-  d_mode,
-  d_mode,
-  0,
-  w_mode,
-  0,
-  w_mode,
-  /* da */
-  d_mode,
-  d_mode,
-  d_mode,
-  d_mode,
-  d_mode,
-  d_mode,
-  d_mode,
-  d_mode,
-  /* db */
-  d_mode,
-  d_mode,
-  d_mode,
-  d_mode,
-  0,
-  t_mode,
-  0,
-  t_mode,
-  /* dc */
-  q_mode,
-  q_mode,
-  q_mode,
-  q_mode,
-  q_mode,
-  q_mode,
-  q_mode,
-  q_mode,
-  /* dd */
-  q_mode,
-  q_mode,
-  q_mode,
-  q_mode,
-  0,
-  0,
-  0,
-  w_mode,
-  /* de */
-  w_mode,
-  w_mode,
-  w_mode,
-  w_mode,
-  w_mode,
-  w_mode,
-  w_mode,
-  w_mode,
-  /* df */
-  w_mode,
-  w_mode,
-  w_mode,
-  w_mode,
-  t_mode,
-  q_mode,
-  t_mode,
-  q_mode
-};
-
-#define ST { OP_ST, 0 }
-#define STi { OP_STi, 0 }
-
-#define FGRPd9_2 NULL, { { NULL, 0 } }
-#define FGRPd9_4 NULL, { { NULL, 1 } }
-#define FGRPd9_5 NULL, { { NULL, 2 } }
-#define FGRPd9_6 NULL, { { NULL, 3 } }
-#define FGRPd9_7 NULL, { { NULL, 4 } }
-#define FGRPda_5 NULL, { { NULL, 5 } }
-#define FGRPdb_4 NULL, { { NULL, 6 } }
-#define FGRPde_3 NULL, { { NULL, 7 } }
-#define FGRPdf_4 NULL, { { NULL, 8 } }
-
-static const struct dis386 float_reg[][8] = {
-  /* d8 */
-  {
-    { "fadd",	{ ST, STi } },
-    { "fmul",	{ ST, STi } },
-    { "fcom",	{ STi } },
-    { "fcomp",	{ STi } },
-    { "fsub",	{ ST, STi } },
-    { "fsubr",	{ ST, STi } },
-    { "fdiv",	{ ST, STi } },
-    { "fdivr",	{ ST, STi } },
-  },
-  /* d9 */
-  {
-    { "fld",	{ STi } },
-    { "fxch",	{ STi } },
-    { FGRPd9_2 },
-    { "(bad)",	{ XX } },
-    { FGRPd9_4 },
-    { FGRPd9_5 },
-    { FGRPd9_6 },
-    { FGRPd9_7 },
-  },
-  /* da */
-  {
-    { "fcmovb",	{ ST, STi } },
-    { "fcmove",	{ ST, STi } },
-    { "fcmovbe",{ ST, STi } },
-    { "fcmovu",	{ ST, STi } },
-    { "(bad)",	{ XX } },
-    { FGRPda_5 },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-  },
-  /* db */
-  {
-    { "fcmovnb",{ ST, STi } },
-    { "fcmovne",{ ST, STi } },
-    { "fcmovnbe",{ ST, STi } },
-    { "fcmovnu",{ ST, STi } },
-    { FGRPdb_4 },
-    { "fucomi",	{ ST, STi } },
-    { "fcomi",	{ ST, STi } },
-    { "(bad)",	{ XX } },
-  },
-  /* dc */
-  {
-    { "fadd",	{ STi, ST } },
-    { "fmul",	{ STi, ST } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-#if SYSV386_COMPAT
-    { "fsub",	{ STi, ST } },
-    { "fsubr",	{ STi, ST } },
-    { "fdiv",	{ STi, ST } },
-    { "fdivr",	{ STi, ST } },
-#else
-    { "fsubr",	{ STi, ST } },
-    { "fsub",	{ STi, ST } },
-    { "fdivr",	{ STi, ST } },
-    { "fdiv",	{ STi, ST } },
-#endif
-  },
-  /* dd */
-  {
-    { "ffree",	{ STi } },
-    { "(bad)",	{ XX } },
-    { "fst",	{ STi } },
-    { "fstp",	{ STi } },
-    { "fucom",	{ STi } },
-    { "fucomp",	{ STi } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-  },
-  /* de */
-  {
-    { "faddp",	{ STi, ST } },
-    { "fmulp",	{ STi, ST } },
-    { "(bad)",	{ XX } },
-    { FGRPde_3 },
-#if SYSV386_COMPAT
-    { "fsubp",	{ STi, ST } },
-    { "fsubrp",	{ STi, ST } },
-    { "fdivp",	{ STi, ST } },
-    { "fdivrp",	{ STi, ST } },
-#else
-    { "fsubrp",	{ STi, ST } },
-    { "fsubp",	{ STi, ST } },
-    { "fdivrp",	{ STi, ST } },
-    { "fdivp",	{ STi, ST } },
-#endif
-  },
-  /* df */
-  {
-    { "ffreep",	{ STi } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { "(bad)",	{ XX } },
-    { FGRPdf_4 },
-    { "fucomip", { ST, STi } },
-    { "fcomip", { ST, STi } },
-    { "(bad)",	{ XX } },
-  },
-};
-
-static const char *fgrps[][8] = {
-  /* d9_2  0 */
-  {
-    "fnop","(bad)","(bad)","(bad)","(bad)","(bad)","(bad)","(bad)",
-  },
-
-  /* d9_4  1 */
-  {
-    "fchs","fabs","(bad)","(bad)","ftst","fxam","(bad)","(bad)",
-  },
-
-  /* d9_5  2 */
-  {
-    "fld1","fldl2t","fldl2e","fldpi","fldlg2","fldln2","fldz","(bad)",
-  },
-
-  /* d9_6  3 */
-  {
-    "f2xm1","fyl2x","fptan","fpatan","fxtract","fprem1","fdecstp","fincstp",
-  },
-
-  /* d9_7  4 */
-  {
-    "fprem","fyl2xp1","fsqrt","fsincos","frndint","fscale","fsin","fcos",
-  },
-
-  /* da_5  5 */
-  {
-    "(bad)","fucompp","(bad)","(bad)","(bad)","(bad)","(bad)","(bad)",
-  },
-
-  /* db_4  6 */
-  {
-    "feni(287 only)","fdisi(287 only)","fNclex","fNinit",
-    "fNsetpm(287 only)","(bad)","(bad)","(bad)",
-  },
-
-  /* de_3  7 */
-  {
-    "(bad)","fcompp","(bad)","(bad)","(bad)","(bad)","(bad)","(bad)",
-  },
-
-  /* df_4  8 */
-  {
-    "fNstsw","(bad)","(bad)","(bad)","(bad)","(bad)","(bad)","(bad)",
-  },
-};
-
-static void
-dofloat (int sizeflag)
-{
-  const struct dis386 *dp;
-  unsigned char floatop;
-
-  floatop = codep[-1];
-
-  if (modrm.mod != 3)
-    {
-      int fp_indx = (floatop - 0xd8) * 8 + modrm.reg;
-
-      putop (float_mem[fp_indx], sizeflag);
-      obufp = op_out[0];
-      op_ad = 2;
-      OP_E (float_mem_mode[fp_indx], sizeflag);
-      return;
-    }
-  /* Skip mod/rm byte.  */
-  MODRM_CHECK;
-  codep++;
-
-  dp = &float_reg[floatop - 0xd8][modrm.reg];
-  if (dp->name == NULL)
-    {
-      putop (fgrps[dp->op[0].bytemode][modrm.rm], sizeflag);
-
-      /* Instruction fnstsw is only one with strange arg.  */
-      if (floatop == 0xdf && codep[-1] == 0xe0)
-        pstrcpy (op_out[0], sizeof(op_out[0]), names16[0]);
-    }
-  else
-    {
-      putop (dp->name, sizeflag);
-
-      obufp = op_out[0];
-      op_ad = 2;
-      if (dp->op[0].rtn)
-	(*dp->op[0].rtn) (dp->op[0].bytemode, sizeflag);
-
-      obufp = op_out[1];
-      op_ad = 1;
-      if (dp->op[1].rtn)
-	(*dp->op[1].rtn) (dp->op[1].bytemode, sizeflag);
-    }
-}
-
-static void
-OP_ST (int bytemode ATTRIBUTE_UNUSED, int sizeflag ATTRIBUTE_UNUSED)
-{
-  oappend ("%st" + intel_syntax);
-}
-
-static void
-OP_STi (int bytemode ATTRIBUTE_UNUSED, int sizeflag ATTRIBUTE_UNUSED)
-{
-  snprintf (scratchbuf, sizeof(scratchbuf), "%%st(%d)", modrm.rm);
-  oappend (scratchbuf + intel_syntax);
-}
-
-/* Capital letters in template are macros.  */
-static int
-putop (const char *template, int sizeflag)
-{
-  const char *p;
-  int alt = 0;
-
-  for (p = template; *p; p++)
-    {
-      switch (*p)
-	{
-	default:
-	  *obufp++ = *p;
-	  break;
-	case '{':
-	  alt = 0;
-	  if (intel_syntax)
-	    alt += 1;
-	  if (address_mode == mode_64bit)
-	    alt += 2;
-	  while (alt != 0)
-	    {
-	      while (*++p != '|')
-		{
-		  if (*p == '}')
-		    {
-		      /* Alternative not valid.  */
-                      pstrcpy (obuf, sizeof(obuf), "(bad)");
-		      obufp = obuf + 5;
-		      return 1;
-		    }
-		  else if (*p == '\0')
-		    abort ();
-		}
-	      alt--;
-	    }
-	  /* Fall through.  */
-	case 'I':
-	  alt = 1;
-	  continue;
-	case '|':
-	  while (*++p != '}')
-	    {
-	      if (*p == '\0')
-		abort ();
-	    }
-	  break;
-	case '}':
-	  break;
-	case 'A':
-	  if (intel_syntax)
-	    break;
-	  if (modrm.mod != 3 || (sizeflag & SUFFIX_ALWAYS))
-	    *obufp++ = 'b';
-	  break;
-	case 'B':
-	  if (intel_syntax)
-	    break;
-	  if (sizeflag & SUFFIX_ALWAYS)
-	    *obufp++ = 'b';
-	  break;
-	case 'C':
-	  if (intel_syntax && !alt)
-	    break;
-	  if ((prefixes & PREFIX_DATA) || (sizeflag & SUFFIX_ALWAYS))
-	    {
-	      if (sizeflag & DFLAG)
-		*obufp++ = intel_syntax ? 'd' : 'l';
-	      else
-		*obufp++ = intel_syntax ? 'w' : 's';
-	      used_prefixes |= (prefixes & PREFIX_DATA);
-	    }
-	  break;
-	case 'D':
-	  if (intel_syntax || !(sizeflag & SUFFIX_ALWAYS))
-	    break;
-	  USED_REX (REX_W);
-	  if (modrm.mod == 3)
-	    {
-	      if (rex & REX_W)
-		*obufp++ = 'q';
-	      else if (sizeflag & DFLAG)
-		*obufp++ = intel_syntax ? 'd' : 'l';
-	      else
-		*obufp++ = 'w';
-	      used_prefixes |= (prefixes & PREFIX_DATA);
-	    }
-	  else
-	    *obufp++ = 'w';
-	  break;
-	case 'E':		/* For jcxz/jecxz */
-	  if (address_mode == mode_64bit)
-	    {
-	      if (sizeflag & AFLAG)
-		*obufp++ = 'r';
-	      else
-		*obufp++ = 'e';
-	    }
-	  else
-	    if (sizeflag & AFLAG)
-	      *obufp++ = 'e';
-	  used_prefixes |= (prefixes & PREFIX_ADDR);
-	  break;
-	case 'F':
-	  if (intel_syntax)
-	    break;
-	  if ((prefixes & PREFIX_ADDR) || (sizeflag & SUFFIX_ALWAYS))
-	    {
-	      if (sizeflag & AFLAG)
-		*obufp++ = address_mode == mode_64bit ? 'q' : 'l';
-	      else
-		*obufp++ = address_mode == mode_64bit ? 'l' : 'w';
-	      used_prefixes |= (prefixes & PREFIX_ADDR);
-	    }
-	  break;
-	case 'G':
-	  if (intel_syntax || (obufp[-1] != 's' && !(sizeflag & SUFFIX_ALWAYS)))
-	    break;
-	  if ((rex & REX_W) || (sizeflag & DFLAG))
-	    *obufp++ = 'l';
-	  else
-	    *obufp++ = 'w';
-	  if (!(rex & REX_W))
-	    used_prefixes |= (prefixes & PREFIX_DATA);
-	  break;
-	case 'H':
-	  if (intel_syntax)
-	    break;
-	  if ((prefixes & (PREFIX_CS | PREFIX_DS)) == PREFIX_CS
-	      || (prefixes & (PREFIX_CS | PREFIX_DS)) == PREFIX_DS)
-	    {
-	      used_prefixes |= prefixes & (PREFIX_CS | PREFIX_DS);
-	      *obufp++ = ',';
-	      *obufp++ = 'p';
-	      if (prefixes & PREFIX_DS)
-		*obufp++ = 't';
-	      else
-		*obufp++ = 'n';
-	    }
-	  break;
-	case 'J':
-	  if (intel_syntax)
-	    break;
-	  *obufp++ = 'l';
-	  break;
-	case 'K':
-	  USED_REX (REX_W);
-	  if (rex & REX_W)
-	    *obufp++ = 'q';
-	  else
-	    *obufp++ = 'd';
-	  break;
-	case 'Z':
-	  if (intel_syntax)
-	    break;
-	  if (address_mode == mode_64bit && (sizeflag & SUFFIX_ALWAYS))
-	    {
-	      *obufp++ = 'q';
-	      break;
-	    }
-	  /* Fall through.  */
-	case 'L':
-	  if (intel_syntax)
-	    break;
-	  if (sizeflag & SUFFIX_ALWAYS)
-	    *obufp++ = 'l';
-	  break;
-	case 'N':
-	  if ((prefixes & PREFIX_FWAIT) == 0)
-	    *obufp++ = 'n';
-	  else
-	    used_prefixes |= PREFIX_FWAIT;
-	  break;
-	case 'O':
-	  USED_REX (REX_W);
-	  if (rex & REX_W)
-	    *obufp++ = 'o';
-	  else if (intel_syntax && (sizeflag & DFLAG))
-	    *obufp++ = 'q';
-	  else
-	    *obufp++ = 'd';
-	  if (!(rex & REX_W))
-	    used_prefixes |= (prefixes & PREFIX_DATA);
-	  break;
-	case 'T':
-	  if (intel_syntax)
-	    break;
-	  if (address_mode == mode_64bit && (sizeflag & DFLAG))
-	    {
-	      *obufp++ = 'q';
-	      break;
-	    }
-	  /* Fall through.  */
-	case 'P':
-	  if (intel_syntax)
-	    break;
-	  if ((prefixes & PREFIX_DATA)
-	      || (rex & REX_W)
-	      || (sizeflag & SUFFIX_ALWAYS))
-	    {
-	      USED_REX (REX_W);
-	      if (rex & REX_W)
-		*obufp++ = 'q';
-	      else
-		{
-		   if (sizeflag & DFLAG)
-		      *obufp++ = 'l';
-		   else
-		     *obufp++ = 'w';
-		}
-	      used_prefixes |= (prefixes & PREFIX_DATA);
-	    }
-	  break;
-	case 'U':
-	  if (intel_syntax)
-	    break;
-	  if (address_mode == mode_64bit && (sizeflag & DFLAG))
-	    {
-	      if (modrm.mod != 3 || (sizeflag & SUFFIX_ALWAYS))
-		*obufp++ = 'q';
-	      break;
-	    }
-	  /* Fall through.  */
-	case 'Q':
-	  if (intel_syntax && !alt)
-	    break;
-	  USED_REX (REX_W);
-	  if (modrm.mod != 3 || (sizeflag & SUFFIX_ALWAYS))
-	    {
-	      if (rex & REX_W)
-		*obufp++ = 'q';
-	      else
-		{
-		  if (sizeflag & DFLAG)
-		    *obufp++ = intel_syntax ? 'd' : 'l';
-		  else
-		    *obufp++ = 'w';
-		}
-	      used_prefixes |= (prefixes & PREFIX_DATA);
-	    }
-	  break;
-	case 'R':
-	  USED_REX (REX_W);
-	  if (rex & REX_W)
-	    *obufp++ = 'q';
-	  else if (sizeflag & DFLAG)
-	    {
-	      if (intel_syntax)
-		  *obufp++ = 'd';
-	      else
-		  *obufp++ = 'l';
-	    }
-	  else
-	    *obufp++ = 'w';
-	  if (intel_syntax && !p[1]
-	      && ((rex & REX_W) || (sizeflag & DFLAG)))
-	    *obufp++ = 'e';
-	  if (!(rex & REX_W))
-	    used_prefixes |= (prefixes & PREFIX_DATA);
-	  break;
-	case 'V':
-	  if (intel_syntax)
-	    break;
-	  if (address_mode == mode_64bit && (sizeflag & DFLAG))
-	    {
-	      if (sizeflag & SUFFIX_ALWAYS)
-		*obufp++ = 'q';
-	      break;
-	    }
-	  /* Fall through.  */
-	case 'S':
-	  if (intel_syntax)
-	    break;
-	  if (sizeflag & SUFFIX_ALWAYS)
-	    {
-	      if (rex & REX_W)
-		*obufp++ = 'q';
-	      else
-		{
-		  if (sizeflag & DFLAG)
-		    *obufp++ = 'l';
-		  else
-		    *obufp++ = 'w';
-		  used_prefixes |= (prefixes & PREFIX_DATA);
-		}
-	    }
-	  break;
-	case 'X':
-	  if (prefixes & PREFIX_DATA)
-	    *obufp++ = 'd';
-	  else
-	    *obufp++ = 's';
-	  used_prefixes |= (prefixes & PREFIX_DATA);
-	  break;
-	case 'Y':
-	  if (intel_syntax)
-	    break;
-	  if (rex & REX_W)
-	    {
-	      USED_REX (REX_W);
-	      *obufp++ = 'q';
-	    }
-	  break;
-	  /* implicit operand size 'l' for i386 or 'q' for x86-64 */
-	case 'W':
-	  /* operand size flag for cwtl, cbtw */
-	  USED_REX (REX_W);
-	  if (rex & REX_W)
-	    {
-	      if (intel_syntax)
-		*obufp++ = 'd';
-	      else
-		*obufp++ = 'l';
-	    }
-	  else if (sizeflag & DFLAG)
-	    *obufp++ = 'w';
-	  else
-	    *obufp++ = 'b';
-	  if (!(rex & REX_W))
-	    used_prefixes |= (prefixes & PREFIX_DATA);
-	  break;
-	}
-      alt = 0;
-    }
-  *obufp = 0;
-  return 0;
-}
-
-static void
-oappend (const char *s)
-{
-  strcpy (obufp, s);
-  obufp += strlen (s);
-}
-
-static void
-append_seg (void)
-{
-  if (prefixes & PREFIX_CS)
-    {
-      used_prefixes |= PREFIX_CS;
-      oappend ("%cs:" + intel_syntax);
-    }
-  if (prefixes & PREFIX_DS)
-    {
-      used_prefixes |= PREFIX_DS;
-      oappend ("%ds:" + intel_syntax);
-    }
-  if (prefixes & PREFIX_SS)
-    {
-      used_prefixes |= PREFIX_SS;
-      oappend ("%ss:" + intel_syntax);
-    }
-  if (prefixes & PREFIX_ES)
-    {
-      used_prefixes |= PREFIX_ES;
-      oappend ("%es:" + intel_syntax);
-    }
-  if (prefixes & PREFIX_FS)
-    {
-      used_prefixes |= PREFIX_FS;
-      oappend ("%fs:" + intel_syntax);
-    }
-  if (prefixes & PREFIX_GS)
-    {
-      used_prefixes |= PREFIX_GS;
-      oappend ("%gs:" + intel_syntax);
-    }
-}
-
-static void
-OP_indirE (int bytemode, int sizeflag)
-{
-  if (!intel_syntax)
-    oappend ("*");
-  OP_E (bytemode, sizeflag);
-}
-
-static void
-print_operand_value (char *buf, size_t bufsize, int hex, bfd_vma disp)
-{
-  if (address_mode == mode_64bit)
-    {
-      if (hex)
-	{
-	  char tmp[30];
-	  int i;
-	  buf[0] = '0';
-	  buf[1] = 'x';
-          snprintf_vma (tmp, sizeof(tmp), disp);
-          for (i = 0; tmp[i] == '0' && tmp[i + 1]; i++) {
-          }
-          pstrcpy (buf + 2, bufsize - 2, tmp + i);
-	}
-      else
-	{
-	  bfd_signed_vma v = disp;
-	  char tmp[30];
-	  int i;
-	  if (v < 0)
-	    {
-	      *(buf++) = '-';
-	      v = -disp;
-	      /* Check for possible overflow on 0x8000000000000000.  */
-	      if (v < 0)
-		{
-                  pstrcpy (buf, bufsize, "9223372036854775808");
-		  return;
-		}
-	    }
-	  if (!v)
-	    {
-                pstrcpy (buf, bufsize, "0");
-	      return;
-	    }
-
-	  i = 0;
-	  tmp[29] = 0;
-	  while (v)
-	    {
-	      tmp[28 - i] = (v % 10) + '0';
-	      v /= 10;
-	      i++;
-	    }
-          pstrcpy (buf, bufsize, tmp + 29 - i);
-	}
-    }
-  else
-    {
-      if (hex)
-        snprintf (buf, bufsize, "0x%x", (unsigned int) disp);
-      else
-        snprintf (buf, bufsize, "%d", (int) disp);
-    }
-}
-
-/* Put DISP in BUF as signed hex number.  */
-
-static void
-print_displacement (char *buf, bfd_vma disp)
-{
-  bfd_signed_vma val = disp;
-  char tmp[30];
-  int i, j = 0;
-
-  if (val < 0)
-    {
-      buf[j++] = '-';
-      val = -disp;
-
-      /* Check for possible overflow.  */
-      if (val < 0)
-	{
-	  switch (address_mode)
-	    {
-	    case mode_64bit:
-	      strcpy (buf + j, "0x8000000000000000");
-	      break;
-	    case mode_32bit:
-	      strcpy (buf + j, "0x80000000");
-	      break;
-	    case mode_16bit:
-	      strcpy (buf + j, "0x8000");
-	      break;
-	    }
-	  return;
-	}
-    }
-
-  buf[j++] = '0';
-  buf[j++] = 'x';
-
-  snprintf_vma (tmp, sizeof(tmp), val);
-  for (i = 0; tmp[i] == '0'; i++)
-    continue;
-  if (tmp[i] == '\0')
-    i--;
-  strcpy (buf + j, tmp + i);
-}
-
-static void
-intel_operand_size (int bytemode, int sizeflag)
-{
-  switch (bytemode)
-    {
-    case b_mode:
-    case dqb_mode:
-      oappend ("BYTE PTR ");
-      break;
-    case w_mode:
-    case dqw_mode:
-      oappend ("WORD PTR ");
-      break;
-    case stack_v_mode:
-      if (address_mode == mode_64bit && (sizeflag & DFLAG))
-	{
-	  oappend ("QWORD PTR ");
-	  used_prefixes |= (prefixes & PREFIX_DATA);
-	  break;
-	}
-      /* FALLTHRU */
-    case v_mode:
-    case dq_mode:
-      USED_REX (REX_W);
-      if (rex & REX_W)
-	oappend ("QWORD PTR ");
-      else if ((sizeflag & DFLAG) || bytemode == dq_mode)
-	oappend ("DWORD PTR ");
-      else
-	oappend ("WORD PTR ");
-      used_prefixes |= (prefixes & PREFIX_DATA);
-      break;
-    case z_mode:
-      if ((rex & REX_W) || (sizeflag & DFLAG))
-	*obufp++ = 'D';
-      oappend ("WORD PTR ");
-      if (!(rex & REX_W))
-	used_prefixes |= (prefixes & PREFIX_DATA);
-      break;
-    case d_mode:
-    case dqd_mode:
-      oappend ("DWORD PTR ");
-      break;
-    case q_mode:
-      oappend ("QWORD PTR ");
-      break;
-    case m_mode:
-      if (address_mode == mode_64bit)
-	oappend ("QWORD PTR ");
-      else
-	oappend ("DWORD PTR ");
-      break;
-    case f_mode:
-      if (sizeflag & DFLAG)
-	oappend ("FWORD PTR ");
-      else
-	oappend ("DWORD PTR ");
-      used_prefixes |= (prefixes & PREFIX_DATA);
-      break;
-    case t_mode:
-      oappend ("TBYTE PTR ");
-      break;
-    case x_mode:
-      oappend ("XMMWORD PTR ");
-      break;
-    case o_mode:
-      oappend ("OWORD PTR ");
-      break;
-    default:
-      break;
-    }
-}
-
-static void
-OP_E (int bytemode, int sizeflag)
-{
-  bfd_vma disp;
-  int add = 0;
-  int riprel = 0;
-  USED_REX (REX_B);
-  if (rex & REX_B)
-    add += 8;
-
-  /* Skip mod/rm byte.  */
-  MODRM_CHECK;
-  codep++;
-
-  if (modrm.mod == 3)
-    {
-      switch (bytemode)
-	{
-	case b_mode:
-	  USED_REX (0);
-	  if (rex)
-	    oappend (names8rex[modrm.rm + add]);
-	  else
-	    oappend (names8[modrm.rm + add]);
-	  break;
-	case w_mode:
-	  oappend (names16[modrm.rm + add]);
-	  break;
-	case d_mode:
-	  oappend (names32[modrm.rm + add]);
-	  break;
-	case q_mode:
-	  oappend (names64[modrm.rm + add]);
-	  break;
-	case m_mode:
-	  if (address_mode == mode_64bit)
-	    oappend (names64[modrm.rm + add]);
-	  else
-	    oappend (names32[modrm.rm + add]);
-	  break;
-	case stack_v_mode:
-	  if (address_mode == mode_64bit && (sizeflag & DFLAG))
-	    {
-	      oappend (names64[modrm.rm + add]);
-	      used_prefixes |= (prefixes & PREFIX_DATA);
-	      break;
-	    }
-	  bytemode = v_mode;
-	  /* FALLTHRU */
-	case v_mode:
-	case dq_mode:
-	case dqb_mode:
-	case dqd_mode:
-	case dqw_mode:
-	  USED_REX (REX_W);
-	  if (rex & REX_W)
-	    oappend (names64[modrm.rm + add]);
-	  else if ((sizeflag & DFLAG) || bytemode != v_mode)
-	    oappend (names32[modrm.rm + add]);
-	  else
-	    oappend (names16[modrm.rm + add]);
-	  used_prefixes |= (prefixes & PREFIX_DATA);
-	  break;
-	case 0:
-	  break;
-	default:
-	  oappend (INTERNAL_DISASSEMBLER_ERROR);
-	  break;
-	}
-      return;
-    }
-
-  disp = 0;
-  if (intel_syntax)
-    intel_operand_size (bytemode, sizeflag);
-  append_seg ();
-
-  if ((sizeflag & AFLAG) || address_mode == mode_64bit)
-    {
-      /* 32/64 bit address mode */
-      int havedisp;
-      int havesib;
-      int havebase;
-      int base;
-      int index = 0;
-      int scale = 0;
-
-      havesib = 0;
-      havebase = 1;
-      base = modrm.rm;
-
-      if (base == 4)
-	{
-	  havesib = 1;
-          fetch_data(the_info, codep + 1);
-	  index = (*codep >> 3) & 7;
-	  if (address_mode == mode_64bit || index != 0x4)
-	    /* When INDEX == 0x4 in 32 bit mode, SCALE is ignored.  */
-	    scale = (*codep >> 6) & 3;
-	  base = *codep & 7;
-	  USED_REX (REX_X);
-	  if (rex & REX_X)
-	    index += 8;
-	  codep++;
-	}
-      base += add;
-
-      switch (modrm.mod)
-	{
-	case 0:
-	  if ((base & 7) == 5)
-	    {
-	      havebase = 0;
-	      if (address_mode == mode_64bit && !havesib)
-		riprel = 1;
-	      disp = get32s ();
-	    }
-	  break;
-	case 1:
-          fetch_data (the_info, codep + 1);
-	  disp = *codep++;
-	  if ((disp & 0x80) != 0)
-	    disp -= 0x100;
-	  break;
-	case 2:
-	  disp = get32s ();
-	  break;
-	}
-
-      havedisp = havebase || (havesib && (index != 4 || scale != 0));
-
-      if (!intel_syntax)
-	if (modrm.mod != 0 || (base & 7) == 5)
-	  {
-	    if (havedisp || riprel)
-	      print_displacement (scratchbuf, disp);
-	    else
-              print_operand_value (scratchbuf, sizeof(scratchbuf), 1, disp);
-	    oappend (scratchbuf);
-	    if (riprel)
-	      {
-		set_op (disp, 1);
-		oappend ("(%rip)");
-	      }
-	  }
-
-      if (havedisp || (intel_syntax && riprel))
-	{
-	  *obufp++ = open_char;
-	  if (intel_syntax && riprel)
-	    {
-	      set_op (disp, 1);
-	      oappend ("rip");
-	    }
-	  *obufp = '\0';
-	  if (havebase)
-	    oappend (address_mode == mode_64bit && (sizeflag & AFLAG)
-		     ? names64[base] : names32[base]);
-	  if (havesib)
-	    {
-	      if (index != 4)
-		{
-		  if (!intel_syntax || havebase)
-		    {
-		      *obufp++ = separator_char;
-		      *obufp = '\0';
-		    }
-		  oappend (address_mode == mode_64bit && (sizeflag & AFLAG)
-			   ? names64[index] : names32[index]);
-		}
-	      if (scale != 0 || (!intel_syntax && index != 4))
-		{
-		  *obufp++ = scale_char;
-		  *obufp = '\0';
-		  snprintf (scratchbuf, sizeof(scratchbuf), "%d", 1 << scale);
-		  oappend (scratchbuf);
-		}
-	    }
-	  if (intel_syntax
-	      && (disp || modrm.mod != 0 || (base & 7) == 5))
-	    {
-	      if ((bfd_signed_vma) disp >= 0)
-		{
-		  *obufp++ = '+';
-		  *obufp = '\0';
-		}
-	      else if (modrm.mod != 1)
-		{
-		  *obufp++ = '-';
-		  *obufp = '\0';
-		  disp = - (bfd_signed_vma) disp;
-		}
-
-	      print_displacement (scratchbuf, disp);
-	      oappend (scratchbuf);
-	    }
-
-	  *obufp++ = close_char;
-	  *obufp = '\0';
-	}
-      else if (intel_syntax)
-	{
-	  if (modrm.mod != 0 || (base & 7) == 5)
-	    {
-	      if (prefixes & (PREFIX_CS | PREFIX_SS | PREFIX_DS
-			      | PREFIX_ES | PREFIX_FS | PREFIX_GS))
-		;
-	      else
-		{
-		  oappend (names_seg[ds_reg - es_reg]);
-		  oappend (":");
-		}
-	      print_operand_value (scratchbuf, sizeof(scratchbuf), 1, disp);
-	      oappend (scratchbuf);
-	    }
-	}
-    }
-  else
-    { /* 16 bit address mode */
-      switch (modrm.mod)
-	{
-	case 0:
-	  if (modrm.rm == 6)
-	    {
-	      disp = get16 ();
-	      if ((disp & 0x8000) != 0)
-		disp -= 0x10000;
-	    }
-	  break;
-	case 1:
-          fetch_data(the_info, codep + 1);
-	  disp = *codep++;
-	  if ((disp & 0x80) != 0)
-	    disp -= 0x100;
-	  break;
-	case 2:
-	  disp = get16 ();
-	  if ((disp & 0x8000) != 0)
-	    disp -= 0x10000;
-	  break;
-	}
-
-      if (!intel_syntax)
-	if (modrm.mod != 0 || modrm.rm == 6)
-	  {
-	    print_displacement (scratchbuf, disp);
-	    oappend (scratchbuf);
-	  }
-
-      if (modrm.mod != 0 || modrm.rm != 6)
-	{
-	  *obufp++ = open_char;
-	  *obufp = '\0';
-	  oappend (index16[modrm.rm]);
-	  if (intel_syntax
-	      && (disp || modrm.mod != 0 || modrm.rm == 6))
-	    {
-	      if ((bfd_signed_vma) disp >= 0)
-		{
-		  *obufp++ = '+';
-		  *obufp = '\0';
-		}
-	      else if (modrm.mod != 1)
-		{
-		  *obufp++ = '-';
-		  *obufp = '\0';
-		  disp = - (bfd_signed_vma) disp;
-		}
-
-	      print_displacement (scratchbuf, disp);
-	      oappend (scratchbuf);
-	    }
-
-	  *obufp++ = close_char;
-	  *obufp = '\0';
-	}
-      else if (intel_syntax)
-	{
-	  if (prefixes & (PREFIX_CS | PREFIX_SS | PREFIX_DS
-			  | PREFIX_ES | PREFIX_FS | PREFIX_GS))
-	    ;
-	  else
-	    {
-	      oappend (names_seg[ds_reg - es_reg]);
-	      oappend (":");
-	    }
-	  print_operand_value (scratchbuf, sizeof(scratchbuf), 1,
-                               disp & 0xffff);
-	  oappend (scratchbuf);
-	}
-    }
-}
-
-static void
-OP_G (int bytemode, int sizeflag)
-{
-  int add = 0;
-  USED_REX (REX_R);
-  if (rex & REX_R)
-    add += 8;
-  switch (bytemode)
-    {
-    case b_mode:
-      USED_REX (0);
-      if (rex)
-	oappend (names8rex[modrm.reg + add]);
-      else
-	oappend (names8[modrm.reg + add]);
-      break;
-    case w_mode:
-      oappend (names16[modrm.reg + add]);
-      break;
-    case d_mode:
-      oappend (names32[modrm.reg + add]);
-      break;
-    case q_mode:
-      oappend (names64[modrm.reg + add]);
-      break;
-    case v_mode:
-    case dq_mode:
-    case dqb_mode:
-    case dqd_mode:
-    case dqw_mode:
-      USED_REX (REX_W);
-      if (rex & REX_W)
-	oappend (names64[modrm.reg + add]);
-      else if ((sizeflag & DFLAG) || bytemode != v_mode)
-	oappend (names32[modrm.reg + add]);
-      else
-	oappend (names16[modrm.reg + add]);
-      used_prefixes |= (prefixes & PREFIX_DATA);
-      break;
-    case m_mode:
-      if (address_mode == mode_64bit)
-	oappend (names64[modrm.reg + add]);
-      else
-	oappend (names32[modrm.reg + add]);
-      break;
-    default:
-      oappend (INTERNAL_DISASSEMBLER_ERROR);
-      break;
-    }
-}
-
-static void
-OP_vvvv (int bytemode, int sizeflags)
-{
-    USED_REX (REX_W);
-    if (rex & REX_W) {
-        oappend(names64[vex_reg]);
-    } else {
-        oappend(names32[vex_reg]);
-    }
-}
-
-static bfd_vma
-get64 (void)
-{
-  bfd_vma x;
-#ifdef BFD64
-  unsigned int a;
-  unsigned int b;
-
-  fetch_data(the_info, codep + 8);
-  a = *codep++ & 0xff;
-  a |= (*codep++ & 0xff) << 8;
-  a |= (*codep++ & 0xff) << 16;
-  a |= (*codep++ & 0xff) << 24;
-  b = *codep++ & 0xff;
-  b |= (*codep++ & 0xff) << 8;
-  b |= (*codep++ & 0xff) << 16;
-  b |= (*codep++ & 0xff) << 24;
-  x = a + ((bfd_vma) b << 32);
-#else
-  abort ();
-  x = 0;
-#endif
-  return x;
-}
-
-static bfd_signed_vma
-get32 (void)
-{
-  bfd_signed_vma x = 0;
-
-  fetch_data(the_info, codep + 4);
-  x = *codep++ & (bfd_signed_vma) 0xff;
-  x |= (*codep++ & (bfd_signed_vma) 0xff) << 8;
-  x |= (*codep++ & (bfd_signed_vma) 0xff) << 16;
-  x |= (*codep++ & (bfd_signed_vma) 0xff) << 24;
-  return x;
-}
-
-static bfd_signed_vma
-get32s (void)
-{
-  bfd_signed_vma x = 0;
-
-  fetch_data(the_info, codep + 4);
-  x = *codep++ & (bfd_signed_vma) 0xff;
-  x |= (*codep++ & (bfd_signed_vma) 0xff) << 8;
-  x |= (*codep++ & (bfd_signed_vma) 0xff) << 16;
-  x |= (*codep++ & (bfd_signed_vma) 0xff) << 24;
-
-  x = (x ^ ((bfd_signed_vma) 1 << 31)) - ((bfd_signed_vma) 1 << 31);
-
-  return x;
-}
-
-static int
-get16 (void)
-{
-  int x = 0;
-
-  fetch_data(the_info, codep + 2);
-  x = *codep++ & 0xff;
-  x |= (*codep++ & 0xff) << 8;
-  return x;
-}
-
-static void
-set_op (bfd_vma op, int riprel)
-{
-  op_index[op_ad] = op_ad;
-  if (address_mode == mode_64bit)
-    {
-      op_address[op_ad] = op;
-      op_riprel[op_ad] = riprel;
-    }
-  else
-    {
-      /* Mask to get a 32-bit address.  */
-      op_address[op_ad] = op & 0xffffffff;
-      op_riprel[op_ad] = riprel & 0xffffffff;
-    }
-}
-
-static void
-OP_REG (int code, int sizeflag)
-{
-  const char *s;
-  int add = 0;
-  USED_REX (REX_B);
-  if (rex & REX_B)
-    add = 8;
-
-  switch (code)
-    {
-    case ax_reg: case cx_reg: case dx_reg: case bx_reg:
-    case sp_reg: case bp_reg: case si_reg: case di_reg:
-      s = names16[code - ax_reg + add];
-      break;
-    case es_reg: case ss_reg: case cs_reg:
-    case ds_reg: case fs_reg: case gs_reg:
-      s = names_seg[code - es_reg + add];
-      break;
-    case al_reg: case ah_reg: case cl_reg: case ch_reg:
-    case dl_reg: case dh_reg: case bl_reg: case bh_reg:
-      USED_REX (0);
-      if (rex)
-	s = names8rex[code - al_reg + add];
-      else
-	s = names8[code - al_reg];
-      break;
-    case rAX_reg: case rCX_reg: case rDX_reg: case rBX_reg:
-    case rSP_reg: case rBP_reg: case rSI_reg: case rDI_reg:
-      if (address_mode == mode_64bit && (sizeflag & DFLAG))
-	{
-	  s = names64[code - rAX_reg + add];
-	  break;
-	}
-      code += eAX_reg - rAX_reg;
-      /* Fall through.  */
-    case eAX_reg: case eCX_reg: case eDX_reg: case eBX_reg:
-    case eSP_reg: case eBP_reg: case eSI_reg: case eDI_reg:
-      USED_REX (REX_W);
-      if (rex & REX_W)
-	s = names64[code - eAX_reg + add];
-      else if (sizeflag & DFLAG)
-	s = names32[code - eAX_reg + add];
-      else
-	s = names16[code - eAX_reg + add];
-      used_prefixes |= (prefixes & PREFIX_DATA);
-      break;
-    default:
-      s = INTERNAL_DISASSEMBLER_ERROR;
-      break;
-    }
-  oappend (s);
-}
-
-static void
-OP_IMREG (int code, int sizeflag)
-{
-  const char *s;
-
-  switch (code)
-    {
-    case indir_dx_reg:
-      if (intel_syntax)
-	s = "dx";
-      else
-	s = "(%dx)";
-      break;
-    case ax_reg: case cx_reg: case dx_reg: case bx_reg:
-    case sp_reg: case bp_reg: case si_reg: case di_reg:
-      s = names16[code - ax_reg];
-      break;
-    case es_reg: case ss_reg: case cs_reg:
-    case ds_reg: case fs_reg: case gs_reg:
-      s = names_seg[code - es_reg];
-      break;
-    case al_reg: case ah_reg: case cl_reg: case ch_reg:
-    case dl_reg: case dh_reg: case bl_reg: case bh_reg:
-      USED_REX (0);
-      if (rex)
-	s = names8rex[code - al_reg];
-      else
-	s = names8[code - al_reg];
-      break;
-    case eAX_reg: case eCX_reg: case eDX_reg: case eBX_reg:
-    case eSP_reg: case eBP_reg: case eSI_reg: case eDI_reg:
-      USED_REX (REX_W);
-      if (rex & REX_W)
-	s = names64[code - eAX_reg];
-      else if (sizeflag & DFLAG)
-	s = names32[code - eAX_reg];
-      else
-	s = names16[code - eAX_reg];
-      used_prefixes |= (prefixes & PREFIX_DATA);
-      break;
-    case z_mode_ax_reg:
-      if ((rex & REX_W) || (sizeflag & DFLAG))
-	s = *names32;
-      else
-	s = *names16;
-      if (!(rex & REX_W))
-	used_prefixes |= (prefixes & PREFIX_DATA);
-      break;
-    default:
-      s = INTERNAL_DISASSEMBLER_ERROR;
-      break;
-    }
-  oappend (s);
-}
-
-static void
-OP_I (int bytemode, int sizeflag)
-{
-  bfd_signed_vma op;
-  bfd_signed_vma mask = -1;
-
-  switch (bytemode)
-    {
-    case b_mode:
-      fetch_data(the_info, codep + 1);
-      op = *codep++;
-      mask = 0xff;
-      break;
-    case q_mode:
-      if (address_mode == mode_64bit)
-	{
-	  op = get32s ();
-	  break;
-	}
-      /* Fall through.  */
-    case v_mode:
-      USED_REX (REX_W);
-      if (rex & REX_W)
-	op = get32s ();
-      else if (sizeflag & DFLAG)
-	{
-	  op = get32 ();
-	  mask = 0xffffffff;
-	}
-      else
-	{
-	  op = get16 ();
-	  mask = 0xfffff;
-	}
-      used_prefixes |= (prefixes & PREFIX_DATA);
-      break;
-    case w_mode:
-      mask = 0xfffff;
-      op = get16 ();
-      break;
-    case const_1_mode:
-      if (intel_syntax)
-        oappend ("1");
-      return;
-    default:
-      oappend (INTERNAL_DISASSEMBLER_ERROR);
-      return;
-    }
-
-  op &= mask;
-  scratchbuf[0] = '$';
-  print_operand_value (scratchbuf + 1, sizeof(scratchbuf) - 1, 1, op);
-  oappend (scratchbuf + intel_syntax);
-  scratchbuf[0] = '\0';
-}
-
-static void
-OP_I64 (int bytemode, int sizeflag)
-{
-  bfd_signed_vma op;
-  bfd_signed_vma mask = -1;
-
-  if (address_mode != mode_64bit)
-    {
-      OP_I (bytemode, sizeflag);
-      return;
-    }
-
-  switch (bytemode)
-    {
-    case b_mode:
-      fetch_data(the_info, codep + 1);
-      op = *codep++;
-      mask = 0xff;
-      break;
-    case v_mode:
-      USED_REX (REX_W);
-      if (rex & REX_W)
-	op = get64 ();
-      else if (sizeflag & DFLAG)
-	{
-	  op = get32 ();
-	  mask = 0xffffffff;
-	}
-      else
-	{
-	  op = get16 ();
-	  mask = 0xfffff;
-	}
-      used_prefixes |= (prefixes & PREFIX_DATA);
-      break;
-    case w_mode:
-      mask = 0xfffff;
-      op = get16 ();
-      break;
-    default:
-      oappend (INTERNAL_DISASSEMBLER_ERROR);
-      return;
-    }
-
-  op &= mask;
-  scratchbuf[0] = '$';
-  print_operand_value (scratchbuf + 1, sizeof(scratchbuf) - 1, 1, op);
-  oappend (scratchbuf + intel_syntax);
-  scratchbuf[0] = '\0';
-}
-
-static void
-OP_sI (int bytemode, int sizeflag)
-{
-  bfd_signed_vma op;
-
-  switch (bytemode)
-    {
-    case b_mode:
-      fetch_data(the_info, codep + 1);
-      op = *codep++;
-      if ((op & 0x80) != 0)
-	op -= 0x100;
-      break;
-    case v_mode:
-      USED_REX (REX_W);
-      if (rex & REX_W)
-	op = get32s ();
-      else if (sizeflag & DFLAG)
-	{
-	  op = get32s ();
-	}
-      else
-	{
-	  op = get16 ();
-	  if ((op & 0x8000) != 0)
-	    op -= 0x10000;
-	}
-      used_prefixes |= (prefixes & PREFIX_DATA);
-      break;
-    case w_mode:
-      op = get16 ();
-      if ((op & 0x8000) != 0)
-	op -= 0x10000;
-      break;
-    default:
-      oappend (INTERNAL_DISASSEMBLER_ERROR);
-      return;
-    }
-
-  scratchbuf[0] = '$';
-  print_operand_value (scratchbuf + 1, sizeof(scratchbuf) - 1, 1, op);
-  oappend (scratchbuf + intel_syntax);
-}
-
-static void
-OP_J (int bytemode, int sizeflag)
-{
-  bfd_vma disp;
-  bfd_vma mask = -1;
-  bfd_vma segment = 0;
-
-  switch (bytemode)
-    {
-    case b_mode:
-      fetch_data(the_info, codep + 1);
-      disp = *codep++;
-      if ((disp & 0x80) != 0)
-	disp -= 0x100;
-      break;
-    case v_mode:
-      if ((sizeflag & DFLAG) || (rex & REX_W))
-	disp = get32s ();
-      else
-	{
-	  disp = get16 ();
-	  if ((disp & 0x8000) != 0)
-	    disp -= 0x10000;
-	  /* In 16bit mode, address is wrapped around at 64k within
-	     the same segment.  Otherwise, a data16 prefix on a jump
-	     instruction means that the pc is masked to 16 bits after
-	     the displacement is added!  */
-	  mask = 0xffff;
-	  if ((prefixes & PREFIX_DATA) == 0)
-	    segment = ((start_pc + codep - start_codep)
-		       & ~((bfd_vma) 0xffff));
-	}
-      used_prefixes |= (prefixes & PREFIX_DATA);
-      break;
-    default:
-      oappend (INTERNAL_DISASSEMBLER_ERROR);
-      return;
-    }
-  disp = ((start_pc + codep - start_codep + disp) & mask) | segment;
-  set_op (disp, 0);
-  print_operand_value (scratchbuf, sizeof(scratchbuf), 1, disp);
-  oappend (scratchbuf);
-}
-
-static void
-OP_SEG (int bytemode, int sizeflag)
-{
-  if (bytemode == w_mode)
-    oappend (names_seg[modrm.reg]);
-  else
-    OP_E (modrm.mod == 3 ? bytemode : w_mode, sizeflag);
-}
-
-static void
-OP_DIR (int dummy ATTRIBUTE_UNUSED, int sizeflag)
-{
-  int seg, offset;
-
-  if (sizeflag & DFLAG)
-    {
-      offset = get32 ();
-      seg = get16 ();
-    }
-  else
-    {
-      offset = get16 ();
-      seg = get16 ();
-    }
-  used_prefixes |= (prefixes & PREFIX_DATA);
-  if (intel_syntax)
-    snprintf (scratchbuf, sizeof(scratchbuf), "0x%x:0x%x", seg, offset);
-  else
-    snprintf (scratchbuf, sizeof(scratchbuf), "$0x%x,$0x%x", seg, offset);
-  oappend (scratchbuf);
-}
-
-static void
-OP_OFF (int bytemode, int sizeflag)
-{
-  bfd_vma off;
-
-  if (intel_syntax && (sizeflag & SUFFIX_ALWAYS))
-    intel_operand_size (bytemode, sizeflag);
-  append_seg ();
-
-  if ((sizeflag & AFLAG) || address_mode == mode_64bit)
-    off = get32 ();
-  else
-    off = get16 ();
-
-  if (intel_syntax)
-    {
-      if (!(prefixes & (PREFIX_CS | PREFIX_SS | PREFIX_DS
-			| PREFIX_ES | PREFIX_FS | PREFIX_GS)))
-	{
-	  oappend (names_seg[ds_reg - es_reg]);
-	  oappend (":");
-	}
-    }
-  print_operand_value (scratchbuf, sizeof(scratchbuf), 1, off);
-  oappend (scratchbuf);
-}
-
-static void
-OP_OFF64 (int bytemode, int sizeflag)
-{
-  bfd_vma off;
-
-  if (address_mode != mode_64bit
-      || (prefixes & PREFIX_ADDR))
-    {
-      OP_OFF (bytemode, sizeflag);
-      return;
-    }
-
-  if (intel_syntax && (sizeflag & SUFFIX_ALWAYS))
-    intel_operand_size (bytemode, sizeflag);
-  append_seg ();
-
-  off = get64 ();
-
-  if (intel_syntax)
-    {
-      if (!(prefixes & (PREFIX_CS | PREFIX_SS | PREFIX_DS
-			| PREFIX_ES | PREFIX_FS | PREFIX_GS)))
-	{
-	  oappend (names_seg[ds_reg - es_reg]);
-	  oappend (":");
-	}
-    }
-  print_operand_value (scratchbuf, sizeof(scratchbuf), 1, off);
-  oappend (scratchbuf);
-}
-
-static void
-ptr_reg (int code, int sizeflag)
-{
-  const char *s;
-
-  *obufp++ = open_char;
-  used_prefixes |= (prefixes & PREFIX_ADDR);
-  if (address_mode == mode_64bit)
-    {
-      if (!(sizeflag & AFLAG))
-	s = names32[code - eAX_reg];
-      else
-	s = names64[code - eAX_reg];
-    }
-  else if (sizeflag & AFLAG)
-    s = names32[code - eAX_reg];
-  else
-    s = names16[code - eAX_reg];
-  oappend (s);
-  *obufp++ = close_char;
-  *obufp = 0;
-}
-
-static void
-OP_ESreg (int code, int sizeflag)
-{
-  if (intel_syntax)
-    {
-      switch (codep[-1])
-	{
-	case 0x6d:	/* insw/insl */
-	  intel_operand_size (z_mode, sizeflag);
-	  break;
-	case 0xa5:	/* movsw/movsl/movsq */
-	case 0xa7:	/* cmpsw/cmpsl/cmpsq */
-	case 0xab:	/* stosw/stosl */
-	case 0xaf:	/* scasw/scasl */
-	  intel_operand_size (v_mode, sizeflag);
-	  break;
-	default:
-	  intel_operand_size (b_mode, sizeflag);
-	}
-    }
-  oappend ("%es:" + intel_syntax);
-  ptr_reg (code, sizeflag);
-}
-
-static void
-OP_DSreg (int code, int sizeflag)
-{
-  if (intel_syntax)
-    {
-      switch (codep[-1])
-	{
-	case 0x6f:	/* outsw/outsl */
-	  intel_operand_size (z_mode, sizeflag);
-	  break;
-	case 0xa5:	/* movsw/movsl/movsq */
-	case 0xa7:	/* cmpsw/cmpsl/cmpsq */
-	case 0xad:	/* lodsw/lodsl/lodsq */
-	  intel_operand_size (v_mode, sizeflag);
-	  break;
-	default:
-	  intel_operand_size (b_mode, sizeflag);
-	}
-    }
-  if ((prefixes
-       & (PREFIX_CS
-	  | PREFIX_DS
-	  | PREFIX_SS
-	  | PREFIX_ES
-	  | PREFIX_FS
-	  | PREFIX_GS)) == 0)
-    prefixes |= PREFIX_DS;
-  append_seg ();
-  ptr_reg (code, sizeflag);
-}
-
-static void
-OP_C (int dummy ATTRIBUTE_UNUSED, int sizeflag ATTRIBUTE_UNUSED)
-{
-  int add = 0;
-  if (rex & REX_R)
-    {
-      USED_REX (REX_R);
-      add = 8;
-    }
-  else if (address_mode != mode_64bit && (prefixes & PREFIX_LOCK))
-    {
-      used_prefixes |= PREFIX_LOCK;
-      add = 8;
-    }
-  snprintf (scratchbuf, sizeof(scratchbuf), "%%cr%d", modrm.reg + add);
-  oappend (scratchbuf + intel_syntax);
-}
-
-static void
-OP_D (int dummy ATTRIBUTE_UNUSED, int sizeflag ATTRIBUTE_UNUSED)
-{
-  int add = 0;
-  USED_REX (REX_R);
-  if (rex & REX_R)
-    add = 8;
-  if (intel_syntax)
-    snprintf (scratchbuf, sizeof(scratchbuf), "db%d", modrm.reg + add);
-  else
-    snprintf (scratchbuf, sizeof(scratchbuf), "%%db%d", modrm.reg + add);
-  oappend (scratchbuf);
-}
-
-static void
-OP_T (int dummy ATTRIBUTE_UNUSED, int sizeflag ATTRIBUTE_UNUSED)
-{
-  snprintf (scratchbuf, sizeof(scratchbuf), "%%tr%d", modrm.reg);
-  oappend (scratchbuf + intel_syntax);
-}
-
-static void
-OP_R (int bytemode, int sizeflag)
-{
-  if (modrm.mod == 3)
-    OP_E (bytemode, sizeflag);
-  else
-    BadOp ();
-}
-
-static void
-OP_MMX (int bytemode ATTRIBUTE_UNUSED, int sizeflag ATTRIBUTE_UNUSED)
-{
-  used_prefixes |= (prefixes & PREFIX_DATA);
-  if (prefixes & PREFIX_DATA)
-    {
-      int add = 0;
-      USED_REX (REX_R);
-      if (rex & REX_R)
-	add = 8;
-      snprintf (scratchbuf, sizeof(scratchbuf), "%%xmm%d", modrm.reg + add);
-    }
-  else
-    snprintf (scratchbuf, sizeof(scratchbuf), "%%mm%d", modrm.reg);
-  oappend (scratchbuf + intel_syntax);
-}
-
-static void
-OP_XMM (int bytemode ATTRIBUTE_UNUSED, int sizeflag ATTRIBUTE_UNUSED)
-{
-  int add = 0;
-  USED_REX (REX_R);
-  if (rex & REX_R)
-    add = 8;
-  snprintf (scratchbuf, sizeof(scratchbuf), "%%xmm%d", modrm.reg + add);
-  oappend (scratchbuf + intel_syntax);
-}
-
-static void
-OP_EM (int bytemode, int sizeflag)
-{
-  if (modrm.mod != 3)
-    {
-      if (intel_syntax && bytemode == v_mode)
-	{
-	  bytemode = (prefixes & PREFIX_DATA) ? x_mode : q_mode;
-	  used_prefixes |= (prefixes & PREFIX_DATA);
-	}
-      OP_E (bytemode, sizeflag);
-      return;
-    }
-
-  /* Skip mod/rm byte.  */
-  MODRM_CHECK;
-  codep++;
-  used_prefixes |= (prefixes & PREFIX_DATA);
-  if (prefixes & PREFIX_DATA)
-    {
-      int add = 0;
-
-      USED_REX (REX_B);
-      if (rex & REX_B)
-	add = 8;
-      snprintf (scratchbuf, sizeof(scratchbuf), "%%xmm%d", modrm.rm + add);
-    }
-  else
-    snprintf (scratchbuf, sizeof(scratchbuf), "%%mm%d", modrm.rm);
-  oappend (scratchbuf + intel_syntax);
-}
-
-/* cvt* are the only instructions in sse2 which have
-   both SSE and MMX operands and also have 0x66 prefix
-   in their opcode. 0x66 was originally used to differentiate
-   between SSE and MMX instruction(operands). So we have to handle the
-   cvt* separately using OP_EMC and OP_MXC */
-static void
-OP_EMC (int bytemode, int sizeflag)
-{
-  if (modrm.mod != 3)
-    {
-      if (intel_syntax && bytemode == v_mode)
-	{
-	  bytemode = (prefixes & PREFIX_DATA) ? x_mode : q_mode;
-	  used_prefixes |= (prefixes & PREFIX_DATA);
-	}
-      OP_E (bytemode, sizeflag);
-      return;
-    }
-
-  /* Skip mod/rm byte.  */
-  MODRM_CHECK;
-  codep++;
-  used_prefixes |= (prefixes & PREFIX_DATA);
-  snprintf (scratchbuf, sizeof(scratchbuf), "%%mm%d", modrm.rm);
-  oappend (scratchbuf + intel_syntax);
-}
-
-static void
-OP_MXC (int bytemode ATTRIBUTE_UNUSED, int sizeflag ATTRIBUTE_UNUSED)
-{
-  used_prefixes |= (prefixes & PREFIX_DATA);
-  snprintf (scratchbuf, sizeof(scratchbuf), "%%mm%d", modrm.reg);
-  oappend (scratchbuf + intel_syntax);
-}
-
-static void
-OP_EX (int bytemode, int sizeflag)
-{
-  int add = 0;
-  if (modrm.mod != 3)
-    {
-      OP_E (bytemode, sizeflag);
-      return;
-    }
-  USED_REX (REX_B);
-  if (rex & REX_B)
-    add = 8;
-
-  /* Skip mod/rm byte.  */
-  MODRM_CHECK;
-  codep++;
-  snprintf (scratchbuf, sizeof(scratchbuf), "%%xmm%d", modrm.rm + add);
-  oappend (scratchbuf + intel_syntax);
-}
-
-static void
-OP_MS (int bytemode, int sizeflag)
-{
-  if (modrm.mod == 3)
-    OP_EM (bytemode, sizeflag);
-  else
-    BadOp ();
-}
-
-static void
-OP_XS (int bytemode, int sizeflag)
-{
-  if (modrm.mod == 3)
-    OP_EX (bytemode, sizeflag);
-  else
-    BadOp ();
-}
-
-static void
-OP_M (int bytemode, int sizeflag)
-{
-  if (modrm.mod == 3)
-    /* bad bound,lea,lds,les,lfs,lgs,lss,cmpxchg8b,vmptrst modrm */
-    BadOp ();
-  else
-    OP_E (bytemode, sizeflag);
-}
-
-static void
-OP_0f07 (int bytemode, int sizeflag)
-{
-  if (modrm.mod != 3 || modrm.rm != 0)
-    BadOp ();
-  else
-    OP_E (bytemode, sizeflag);
-}
-
-static void
-OP_0fae (int bytemode, int sizeflag)
-{
-  if (modrm.mod == 3)
-    {
-      if (modrm.reg == 7)
-	strcpy (obuf + strlen (obuf) - sizeof ("clflush") + 1, "sfence");
-
-      if (modrm.reg < 5 || modrm.rm != 0)
-	{
-	  BadOp ();	/* bad sfence, mfence, or lfence */
-	  return;
-	}
-    }
-  else if (modrm.reg != 7)
-    {
-      BadOp ();		/* bad clflush */
-      return;
-    }
-
-  OP_E (bytemode, sizeflag);
-}
-
-/* NOP is an alias of "xchg %ax,%ax" in 16bit mode, "xchg %eax,%eax" in
-   32bit mode and "xchg %rax,%rax" in 64bit mode.  */
-
-static void
-NOP_Fixup1 (int bytemode, int sizeflag)
-{
-  if ((prefixes & PREFIX_DATA) != 0
-      || (rex != 0
-	  && rex != 0x48
-	  && address_mode == mode_64bit))
-    OP_REG (bytemode, sizeflag);
-  else
-    strcpy (obuf, "nop");
-}
-
-static void
-NOP_Fixup2 (int bytemode, int sizeflag)
-{
-  if ((prefixes & PREFIX_DATA) != 0
-      || (rex != 0
-	  && rex != 0x48
-	  && address_mode == mode_64bit))
-    OP_IMREG (bytemode, sizeflag);
-}
-
-static const char *Suffix3DNow[] = {
-/* 00 */	NULL,		NULL,		NULL,		NULL,
-/* 04 */	NULL,		NULL,		NULL,		NULL,
-/* 08 */	NULL,		NULL,		NULL,		NULL,
-/* 0C */	"pi2fw",	"pi2fd",	NULL,		NULL,
-/* 10 */	NULL,		NULL,		NULL,		NULL,
-/* 14 */	NULL,		NULL,		NULL,		NULL,
-/* 18 */	NULL,		NULL,		NULL,		NULL,
-/* 1C */	"pf2iw",	"pf2id",	NULL,		NULL,
-/* 20 */	NULL,		NULL,		NULL,		NULL,
-/* 24 */	NULL,		NULL,		NULL,		NULL,
-/* 28 */	NULL,		NULL,		NULL,		NULL,
-/* 2C */	NULL,		NULL,		NULL,		NULL,
-/* 30 */	NULL,		NULL,		NULL,		NULL,
-/* 34 */	NULL,		NULL,		NULL,		NULL,
-/* 38 */	NULL,		NULL,		NULL,		NULL,
-/* 3C */	NULL,		NULL,		NULL,		NULL,
-/* 40 */	NULL,		NULL,		NULL,		NULL,
-/* 44 */	NULL,		NULL,		NULL,		NULL,
-/* 48 */	NULL,		NULL,		NULL,		NULL,
-/* 4C */	NULL,		NULL,		NULL,		NULL,
-/* 50 */	NULL,		NULL,		NULL,		NULL,
-/* 54 */	NULL,		NULL,		NULL,		NULL,
-/* 58 */	NULL,		NULL,		NULL,		NULL,
-/* 5C */	NULL,		NULL,		NULL,		NULL,
-/* 60 */	NULL,		NULL,		NULL,		NULL,
-/* 64 */	NULL,		NULL,		NULL,		NULL,
-/* 68 */	NULL,		NULL,		NULL,		NULL,
-/* 6C */	NULL,		NULL,		NULL,		NULL,
-/* 70 */	NULL,		NULL,		NULL,		NULL,
-/* 74 */	NULL,		NULL,		NULL,		NULL,
-/* 78 */	NULL,		NULL,		NULL,		NULL,
-/* 7C */	NULL,		NULL,		NULL,		NULL,
-/* 80 */	NULL,		NULL,		NULL,		NULL,
-/* 84 */	NULL,		NULL,		NULL,		NULL,
-/* 88 */	NULL,		NULL,		"pfnacc",	NULL,
-/* 8C */	NULL,		NULL,		"pfpnacc",	NULL,
-/* 90 */	"pfcmpge",	NULL,		NULL,		NULL,
-/* 94 */	"pfmin",	NULL,		"pfrcp",	"pfrsqrt",
-/* 98 */	NULL,		NULL,		"pfsub",	NULL,
-/* 9C */	NULL,		NULL,		"pfadd",	NULL,
-/* A0 */	"pfcmpgt",	NULL,		NULL,		NULL,
-/* A4 */	"pfmax",	NULL,		"pfrcpit1",	"pfrsqit1",
-/* A8 */	NULL,		NULL,		"pfsubr",	NULL,
-/* AC */	NULL,		NULL,		"pfacc",	NULL,
-/* B0 */	"pfcmpeq",	NULL,		NULL,		NULL,
-/* B4 */	"pfmul",	NULL,		"pfrcpit2",	"pmulhrw",
-/* B8 */	NULL,		NULL,		NULL,		"pswapd",
-/* BC */	NULL,		NULL,		NULL,		"pavgusb",
-/* C0 */	NULL,		NULL,		NULL,		NULL,
-/* C4 */	NULL,		NULL,		NULL,		NULL,
-/* C8 */	NULL,		NULL,		NULL,		NULL,
-/* CC */	NULL,		NULL,		NULL,		NULL,
-/* D0 */	NULL,		NULL,		NULL,		NULL,
-/* D4 */	NULL,		NULL,		NULL,		NULL,
-/* D8 */	NULL,		NULL,		NULL,		NULL,
-/* DC */	NULL,		NULL,		NULL,		NULL,
-/* E0 */	NULL,		NULL,		NULL,		NULL,
-/* E4 */	NULL,		NULL,		NULL,		NULL,
-/* E8 */	NULL,		NULL,		NULL,		NULL,
-/* EC */	NULL,		NULL,		NULL,		NULL,
-/* F0 */	NULL,		NULL,		NULL,		NULL,
-/* F4 */	NULL,		NULL,		NULL,		NULL,
-/* F8 */	NULL,		NULL,		NULL,		NULL,
-/* FC */	NULL,		NULL,		NULL,		NULL,
-};
-
-static void
-OP_3DNowSuffix (int bytemode ATTRIBUTE_UNUSED, int sizeflag ATTRIBUTE_UNUSED)
-{
-  const char *mnemonic;
-
-  fetch_data(the_info, codep + 1);
-  /* AMD 3DNow! instructions are specified by an opcode suffix in the
-     place where an 8-bit immediate would normally go.  ie. the last
-     byte of the instruction.  */
-  obufp = obuf + strlen (obuf);
-  mnemonic = Suffix3DNow[*codep++ & 0xff];
-  if (mnemonic)
-    oappend (mnemonic);
-  else
-    {
-      /* Since a variable sized modrm/sib chunk is between the start
-	 of the opcode (0x0f0f) and the opcode suffix, we need to do
-	 all the modrm processing first, and don't know until now that
-	 we have a bad opcode.  This necessitates some cleaning up.  */
-      op_out[0][0] = '\0';
-      op_out[1][0] = '\0';
-      BadOp ();
-    }
-}
-
-static const char *simd_cmp_op[] = {
-  "eq",
-  "lt",
-  "le",
-  "unord",
-  "neq",
-  "nlt",
-  "nle",
-  "ord"
-};
-
-static void
-OP_SIMD_Suffix (int bytemode ATTRIBUTE_UNUSED, int sizeflag ATTRIBUTE_UNUSED)
-{
-  unsigned int cmp_type;
-
-  fetch_data(the_info, codep + 1);
-  obufp = obuf + strlen (obuf);
-  cmp_type = *codep++ & 0xff;
-  if (cmp_type < 8)
-    {
-      char suffix1 = 'p', suffix2 = 's';
-      used_prefixes |= (prefixes & PREFIX_REPZ);
-      if (prefixes & PREFIX_REPZ)
-	suffix1 = 's';
-      else
-	{
-	  used_prefixes |= (prefixes & PREFIX_DATA);
-	  if (prefixes & PREFIX_DATA)
-	    suffix2 = 'd';
-	  else
-	    {
-	      used_prefixes |= (prefixes & PREFIX_REPNZ);
-	      if (prefixes & PREFIX_REPNZ)
-		suffix1 = 's', suffix2 = 'd';
-	    }
-	}
-      snprintf (scratchbuf, sizeof(scratchbuf), "cmp%s%c%c",
-                simd_cmp_op[cmp_type], suffix1, suffix2);
-      used_prefixes |= (prefixes & PREFIX_REPZ);
-      oappend (scratchbuf);
-    }
-  else
-    {
-      /* We have a bad extension byte.  Clean up.  */
-      op_out[0][0] = '\0';
-      op_out[1][0] = '\0';
-      BadOp ();
-    }
-}
-
-static void
-SIMD_Fixup (int extrachar, int sizeflag ATTRIBUTE_UNUSED)
-{
-  /* Change movlps/movhps to movhlps/movlhps for 2 register operand
-     forms of these instructions.  */
-  if (modrm.mod == 3)
-    {
-      char *p = obuf + strlen (obuf);
-      *(p + 1) = '\0';
-      *p       = *(p - 1);
-      *(p - 1) = *(p - 2);
-      *(p - 2) = *(p - 3);
-      *(p - 3) = extrachar;
-    }
-}
-
-static void
-PNI_Fixup (int extrachar ATTRIBUTE_UNUSED, int sizeflag)
-{
-  if (modrm.mod == 3 && modrm.reg == 1 && modrm.rm <= 1)
-    {
-      /* Override "sidt".  */
-      size_t olen = strlen (obuf);
-      char *p = obuf + olen - 4;
-      const char * const *names = (address_mode == mode_64bit
-			    ? names64 : names32);
-
-      /* We might have a suffix when disassembling with -Msuffix.  */
-      if (*p == 'i')
-	--p;
-
-      /* Remove "addr16/addr32" if we aren't in Intel mode.  */
-      if (!intel_syntax
-	  && (prefixes & PREFIX_ADDR)
-	  && olen >= (4 + 7)
-	  && *(p - 1) == ' '
-	  && strncmp (p - 7, "addr", 4) == 0
-	  && (strncmp (p - 3, "16", 2) == 0
-	      || strncmp (p - 3, "32", 2) == 0))
-	p -= 7;
-
-      if (modrm.rm)
-	{
-	  /* mwait %eax,%ecx  */
-	  strcpy (p, "mwait");
-	  if (!intel_syntax)
-	    strcpy (op_out[0], names[0]);
-	}
-      else
-	{
-	  /* monitor %eax,%ecx,%edx"  */
-	  strcpy (p, "monitor");
-	  if (!intel_syntax)
-	    {
-	      const char * const *op1_names;
-	      if (!(prefixes & PREFIX_ADDR))
-		op1_names = (address_mode == mode_16bit
-			     ? names16 : names);
-	      else
-		{
-		  op1_names = (address_mode != mode_32bit
-			       ? names32 : names16);
-		  used_prefixes |= PREFIX_ADDR;
-		}
-	      strcpy (op_out[0], op1_names[0]);
-	      strcpy (op_out[2], names[2]);
-	    }
-	}
-      if (!intel_syntax)
-	{
-	  strcpy (op_out[1], names[1]);
-	  two_source_ops = 1;
-	}
-
-      codep++;
-    }
-  else
-    OP_M (0, sizeflag);
-}
-
-static void
-SVME_Fixup (int bytemode, int sizeflag)
-{
-  const char *alt;
-  char *p;
-
-  switch (*codep)
-    {
-    case 0xd8:
-      alt = "vmrun";
-      break;
-    case 0xd9:
-      alt = "vmmcall";
-      break;
-    case 0xda:
-      alt = "vmload";
-      break;
-    case 0xdb:
-      alt = "vmsave";
-      break;
-    case 0xdc:
-      alt = "stgi";
-      break;
-    case 0xdd:
-      alt = "clgi";
-      break;
-    case 0xde:
-      alt = "skinit";
-      break;
-    case 0xdf:
-      alt = "invlpga";
-      break;
-    default:
-      OP_M (bytemode, sizeflag);
-      return;
-    }
-  /* Override "lidt".  */
-  p = obuf + strlen (obuf) - 4;
-  /* We might have a suffix.  */
-  if (*p == 'i')
-    --p;
-  strcpy (p, alt);
-  if (!(prefixes & PREFIX_ADDR))
-    {
-      ++codep;
-      return;
-    }
-  used_prefixes |= PREFIX_ADDR;
-  switch (*codep++)
-    {
-    case 0xdf:
-      strcpy (op_out[1], names32[1]);
-      two_source_ops = 1;
-	  /* Fall through.  */
-    case 0xd8:
-    case 0xda:
-    case 0xdb:
-      *obufp++ = open_char;
-      if (address_mode == mode_64bit || (sizeflag & AFLAG))
-        alt = names32[0];
-      else
-        alt = names16[0];
-      strcpy (obufp, alt);
-      obufp += strlen (alt);
-      *obufp++ = close_char;
-      *obufp = '\0';
-      break;
-    }
-}
-
-static void
-INVLPG_Fixup (int bytemode, int sizeflag)
-{
-  const char *alt;
-
-  switch (*codep)
-    {
-    case 0xf8:
-      alt = "swapgs";
-      break;
-    case 0xf9:
-      alt = "rdtscp";
-      break;
-    default:
-      OP_M (bytemode, sizeflag);
-      return;
-    }
-  /* Override "invlpg".  */
-  strcpy (obuf + strlen (obuf) - 6, alt);
-  codep++;
-}
-
-static void
-BadOp (void)
-{
-  /* Throw away prefixes and 1st. opcode byte.  */
-  codep = insn_codep + 1;
-  oappend ("(bad)");
-}
-
-static void
-VMX_Fixup (int extrachar ATTRIBUTE_UNUSED, int sizeflag)
-{
-  if (modrm.mod == 3
-      && modrm.reg == 0
-      && modrm.rm >=1
-      && modrm.rm <= 4)
-    {
-      /* Override "sgdt".  */
-      char *p = obuf + strlen (obuf) - 4;
-
-      /* We might have a suffix when disassembling with -Msuffix.  */
-      if (*p == 'g')
-	--p;
-
-      switch (modrm.rm)
-	{
-	case 1:
-	  strcpy (p, "vmcall");
-	  break;
-	case 2:
-	  strcpy (p, "vmlaunch");
-	  break;
-	case 3:
-	  strcpy (p, "vmresume");
-	  break;
-	case 4:
-	  strcpy (p, "vmxoff");
-	  break;
-	}
-
-      codep++;
-    }
-  else
-    OP_E (0, sizeflag);
-}
-
-static void
-OP_VMX (int bytemode, int sizeflag)
-{
-  used_prefixes |= (prefixes & (PREFIX_DATA | PREFIX_REPZ));
-  if (prefixes & PREFIX_DATA)
-    strcpy (obuf, "vmclear");
-  else if (prefixes & PREFIX_REPZ)
-    strcpy (obuf, "vmxon");
-  else
-    strcpy (obuf, "vmptrld");
-  OP_E (bytemode, sizeflag);
-}
-
-static void
-REP_Fixup (int bytemode, int sizeflag)
-{
-  /* The 0xf3 prefix should be displayed as "rep" for ins, outs, movs,
-     lods and stos.  */
-  size_t ilen = 0;
-
-  if (prefixes & PREFIX_REPZ)
-    switch (*insn_codep)
-      {
-      case 0x6e:	/* outsb */
-      case 0x6f:	/* outsw/outsl */
-      case 0xa4:	/* movsb */
-      case 0xa5:	/* movsw/movsl/movsq */
-	if (!intel_syntax)
-	  ilen = 5;
-	else
-	  ilen = 4;
-	break;
-      case 0xaa:	/* stosb */
-      case 0xab:	/* stosw/stosl/stosq */
-      case 0xac:	/* lodsb */
-      case 0xad:	/* lodsw/lodsl/lodsq */
-	if (!intel_syntax && (sizeflag & SUFFIX_ALWAYS))
-	  ilen = 5;
-	else
-	  ilen = 4;
-	break;
-      case 0x6c:	/* insb */
-      case 0x6d:	/* insl/insw */
-	if (!intel_syntax)
-	  ilen = 4;
-	else
-	  ilen = 3;
-	break;
-      default:
-	abort ();
-	break;
-      }
-
-  if (ilen != 0)
-    {
-      size_t olen;
-      char *p;
-
-      olen = strlen (obuf);
-      p = obuf + olen - ilen - 1 - 4;
-      /* Handle "repz [addr16|addr32]".  */
-      if ((prefixes & PREFIX_ADDR))
-	p -= 1 + 6;
-
-      memmove (p + 3, p + 4, olen - (p + 3 - obuf));
-    }
-
-  switch (bytemode)
-    {
-    case al_reg:
-    case eAX_reg:
-    case indir_dx_reg:
-      OP_IMREG (bytemode, sizeflag);
-      break;
-    case eDI_reg:
-      OP_ESreg (bytemode, sizeflag);
-      break;
-    case eSI_reg:
-      OP_DSreg (bytemode, sizeflag);
-      break;
-    default:
-      abort ();
-      break;
-    }
-}
-
-static void
-CMPXCHG8B_Fixup (int bytemode, int sizeflag)
-{
-  USED_REX (REX_W);
-  if (rex & REX_W)
-    {
-      /* Change cmpxchg8b to cmpxchg16b.  */
-      char *p = obuf + strlen (obuf) - 2;
-      strcpy (p, "16b");
-      bytemode = o_mode;
-    }
-  OP_M (bytemode, sizeflag);
-}
-
-static void
-XMM_Fixup (int reg, int sizeflag ATTRIBUTE_UNUSED)
-{
-  snprintf (scratchbuf, sizeof(scratchbuf), "%%xmm%d", reg);
-  oappend (scratchbuf + intel_syntax);
-}
-
-static void
-CRC32_Fixup (int bytemode, int sizeflag)
-{
-  /* Add proper suffix to "crc32".  */
-  char *p = obuf + strlen (obuf);
-
-  switch (bytemode)
-    {
-    case b_mode:
-      if (intel_syntax)
-	break;
-
-      *p++ = 'b';
-      break;
-    case v_mode:
-      if (intel_syntax)
-	break;
-
-      USED_REX (REX_W);
-      if (rex & REX_W)
-	*p++ = 'q';
-      else if (sizeflag & DFLAG)
-	*p++ = 'l';
-      else
-	*p++ = 'w';
-      used_prefixes |= (prefixes & PREFIX_DATA);
-      break;
-    default:
-      oappend (INTERNAL_DISASSEMBLER_ERROR);
-      break;
-    }
-  *p = '\0';
-
-  if (modrm.mod == 3)
-    {
-      int add;
-
-      /* Skip mod/rm byte.  */
-      MODRM_CHECK;
-      codep++;
-
-      USED_REX (REX_B);
-      add = (rex & REX_B) ? 8 : 0;
-      if (bytemode == b_mode)
-	{
-	  USED_REX (0);
-	  if (rex)
-	    oappend (names8rex[modrm.rm + add]);
-	  else
-	    oappend (names8[modrm.rm + add]);
-	}
-      else
-	{
-	  USED_REX (REX_W);
-	  if (rex & REX_W)
-	    oappend (names64[modrm.rm + add]);
-	  else if ((prefixes & PREFIX_DATA))
-	    oappend (names16[modrm.rm + add]);
-	  else
-	    oappend (names32[modrm.rm + add]);
-	}
-    }
-  else
-    OP_E (bytemode, sizeflag);
-}
diff --git a/disas/meson.build b/disas/meson.build
index a6402e7e1c..b0d18b9d47 100644
--- a/disas/meson.build
+++ b/disas/meson.build
@@ -7,7 +7,6 @@ common_ss.add_all(when: 'CONFIG_ARM_A64_DIS', if_true: libvixl_ss)
 common_ss.add(when: 'CONFIG_CRIS_DIS', if_true: files('cris.c'))
 common_ss.add(when: 'CONFIG_HEXAGON_DIS', if_true: files('hexagon.c'))
 common_ss.add(when: 'CONFIG_HPPA_DIS', if_true: files('hppa.c'))
-common_ss.add(when: 'CONFIG_I386_DIS', if_true: files('i386.c'))
 common_ss.add(when: 'CONFIG_M68K_DIS', if_true: files('m68k.c'))
 common_ss.add(when: 'CONFIG_MICROBLAZE_DIS', if_true: files('microblaze.c'))
 common_ss.add(when: 'CONFIG_MIPS_DIS', if_true: files('mips.c'))
diff --git a/include/disas/dis-asm.h b/include/disas/dis-asm.h
index e857f0ae8b..6484caca9e 100644
--- a/include/disas/dis-asm.h
+++ b/include/disas/dis-asm.h
@@ -418,7 +418,6 @@ int print_insn_tci(bfd_vma, disassemble_info*);
 int print_insn_big_mips         (bfd_vma, disassemble_info*);
 int print_insn_little_mips      (bfd_vma, disassemble_info*);
 int print_insn_nanomips         (bfd_vma, disassemble_info*);
-int print_insn_i386             (bfd_vma, disassemble_info*);
 int print_insn_m68k             (bfd_vma, disassemble_info*);
 int print_insn_z8001            (bfd_vma, disassemble_info*);
 int print_insn_z8002            (bfd_vma, disassemble_info*);
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 99343be926..62c240fa91 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6821,7 +6821,6 @@ static void x86_disas_set_info(CPUState *cs, disassemble_info *info)
     info->mach = (env->hflags & HF_CS64_MASK ? bfd_mach_x86_64
                   : env->hflags & HF_CS32_MASK ? bfd_mach_i386_i386
                   : bfd_mach_i386_i8086);
-    info->print_insn = print_insn_i386;
 
     info->cap_arch = CS_ARCH_X86;
     info->cap_mode = (env->hflags & HF_CS64_MASK ? CS_MODE_64

From 333f944c15e7a6f5503f92d80529a368519d6638 Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Thu, 5 May 2022 19:36:19 +0200
Subject: [PATCH 177/935] disas: Remove old libopcode ppc disassembler
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Capstone should be superior to the old libopcode disassembler,
so we can drop the old file nowadays.

Message-Id: <20220505173619.488350-1-thuth@redhat.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 MAINTAINERS             |    2 -
 disas.c                 |    2 -
 disas/meson.build       |    1 -
 disas/ppc.c             | 5435 ---------------------------------------
 include/disas/dis-asm.h |    1 -
 target/ppc/cpu_init.c   |    2 -
 6 files changed, 5443 deletions(-)
 delete mode 100644 disas/ppc.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 85d593a41f..aee6d623b7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -271,7 +271,6 @@ F: target/ppc/
 F: hw/ppc/ppc.c
 F: hw/ppc/ppc_booke.c
 F: include/hw/ppc/ppc.h
-F: disas/ppc.c
 
 RISC-V TCG CPUs
 M: Palmer Dabbelt <palmer@dabbelt.com>
@@ -3293,7 +3292,6 @@ PPC TCG target
 M: Richard Henderson <richard.henderson@linaro.org>
 S: Odd Fixes
 F: tcg/ppc/
-F: disas/ppc.c
 
 RISC-V TCG target
 M: Palmer Dabbelt <palmer@dabbelt.com>
diff --git a/disas.c b/disas.c
index b6f4dc73b2..b2753e1902 100644
--- a/disas.c
+++ b/disas.c
@@ -164,8 +164,6 @@ static void initialize_debug_host(CPUDebug *s)
     s->info.cap_insn_unit = 1;
     s->info.cap_insn_split = 8;
 #elif defined(_ARCH_PPC)
-    s->info.disassembler_options = (char *)"any";
-    s->info.print_insn = print_insn_ppc;
     s->info.cap_arch = CS_ARCH_PPC;
 # ifdef _ARCH_PPC64
     s->info.cap_mode = CS_MODE_64;
diff --git a/disas/meson.build b/disas/meson.build
index b0d18b9d47..7da48ea74a 100644
--- a/disas/meson.build
+++ b/disas/meson.build
@@ -12,7 +12,6 @@ common_ss.add(when: 'CONFIG_MICROBLAZE_DIS', if_true: files('microblaze.c'))
 common_ss.add(when: 'CONFIG_MIPS_DIS', if_true: files('mips.c'))
 common_ss.add(when: 'CONFIG_NANOMIPS_DIS', if_true: files('nanomips.cpp'))
 common_ss.add(when: 'CONFIG_NIOS2_DIS', if_true: files('nios2.c'))
-common_ss.add(when: 'CONFIG_PPC_DIS', if_true: files('ppc.c'))
 common_ss.add(when: 'CONFIG_RISCV_DIS', if_true: files('riscv.c'))
 common_ss.add(when: 'CONFIG_SH4_DIS', if_true: files('sh4.c'))
 common_ss.add(when: 'CONFIG_SPARC_DIS', if_true: files('sparc.c'))
diff --git a/disas/ppc.c b/disas/ppc.c
deleted file mode 100644
index 02be878198..0000000000
--- a/disas/ppc.c
+++ /dev/null
@@ -1,5435 +0,0 @@
-/* ppc-dis.c -- Disassemble PowerPC instructions
-   Copyright 1994, 1995, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007
-   Free Software Foundation, Inc.
-   Written by Ian Lance Taylor, Cygnus Support
-
-This file is part of GDB, GAS, and the GNU binutils.
-
-GDB, GAS, and the GNU binutils are free software; you can redistribute
-them and/or modify them under the terms of the GNU General Public
-License as published by the Free Software Foundation; either version
-2, or (at your option) any later version.
-
-GDB, GAS, and the GNU binutils are distributed in the hope that they
-will be useful, but WITHOUT ANY WARRANTY; without even the implied
-warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
-the GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this file; see the file COPYING.  If not,
-see <http://www.gnu.org/licenses/>.  */
-#include "qemu/osdep.h"
-#include "disas/dis-asm.h"
-#define BFD_DEFAULT_TARGET_SIZE 64
-
-/* ppc.h -- Header file for PowerPC opcode table
-   Copyright 1994, 1995, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006,
-   2007 Free Software Foundation, Inc.
-   Written by Ian Lance Taylor, Cygnus Support
-
-This file is part of GDB, GAS, and the GNU binutils.
-
-GDB, GAS, and the GNU binutils are free software; you can redistribute
-them and/or modify them under the terms of the GNU General Public
-License as published by the Free Software Foundation; either version
-1, or (at your option) any later version.
-
-GDB, GAS, and the GNU binutils are distributed in the hope that they
-will be useful, but WITHOUT ANY WARRANTY; without even the implied
-warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
-the GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this file; see the file COPYING.  If not,
-see <http://www.gnu.org/licenses/>.  */
-
-/* The opcode table is an array of struct powerpc_opcode.  */
-
-struct powerpc_opcode
-{
-  /* The opcode name.  */
-  const char *name;
-
-  /* The opcode itself.  Those bits which will be filled in with
-     operands are zeroes.  */
-  unsigned long opcode;
-
-  /* The opcode mask.  This is used by the disassembler.  This is a
-     mask containing ones indicating those bits which must match the
-     opcode field, and zeroes indicating those bits which need not
-     match (and are presumably filled in by operands).  */
-  unsigned long mask;
-
-  /* One bit flags for the opcode.  These are used to indicate which
-     specific processors support the instructions.  The defined values
-     are listed below.  */
-  unsigned long flags;
-
-  /* An array of operand codes.  Each code is an index into the
-     operand table.  They appear in the order which the operands must
-     appear in assembly code, and are terminated by a zero.  */
-  unsigned char operands[8];
-};
-
-/* The table itself is sorted by major opcode number, and is otherwise
-   in the order in which the disassembler should consider
-   instructions.  */
-extern const struct powerpc_opcode powerpc_opcodes[];
-extern const int powerpc_num_opcodes;
-
-/* Values defined for the flags field of a struct powerpc_opcode.  */
-
-/* Opcode is defined for the PowerPC architecture.  */
-#define PPC_OPCODE_PPC			 1
-
-/* Opcode is defined for the POWER (RS/6000) architecture.  */
-#define PPC_OPCODE_POWER		 2
-
-/* Opcode is defined for the POWER2 (Rios 2) architecture.  */
-#define PPC_OPCODE_POWER2		 4
-
-/* Opcode is only defined on 32 bit architectures.  */
-#define PPC_OPCODE_32			 8
-
-/* Opcode is only defined on 64 bit architectures.  */
-#define PPC_OPCODE_64		      0x10
-
-/* Opcode is supported by the Motorola PowerPC 601 processor.  The 601
-   is assumed to support all PowerPC (PPC_OPCODE_PPC) instructions,
-   but it also supports many additional POWER instructions.  */
-#define PPC_OPCODE_601		      0x20
-
-/* Opcode is supported in both the Power and PowerPC architectures
-   (ie, compiler's -mcpu=common or assembler's -mcom).  */
-#define PPC_OPCODE_COMMON	      0x40
-
-/* Opcode is supported for any Power or PowerPC platform (this is
-   for the assembler's -many option, and it eliminates duplicates).  */
-#define PPC_OPCODE_ANY		      0x80
-
-/* Opcode is supported as part of the 64-bit bridge.  */
-#define PPC_OPCODE_64_BRIDGE	     0x100
-
-/* Opcode is supported by Altivec Vector Unit */
-#define PPC_OPCODE_ALTIVEC	     0x200
-
-/* Opcode is supported by PowerPC 403 processor.  */
-#define PPC_OPCODE_403		     0x400
-
-/* Opcode is supported by PowerPC BookE processor.  */
-#define PPC_OPCODE_BOOKE	     0x800
-
-/* Opcode is only supported by 64-bit PowerPC BookE processor.  */
-#define PPC_OPCODE_BOOKE64	    0x1000
-
-/* Opcode is supported by PowerPC 440 processor.  */
-#define PPC_OPCODE_440		    0x2000
-
-/* Opcode is only supported by Power4 architecture.  */
-#define PPC_OPCODE_POWER4	    0x4000
-
-/* Opcode isn't supported by Power4 architecture.  */
-#define PPC_OPCODE_NOPOWER4	    0x8000
-
-/* Opcode is only supported by POWERPC Classic architecture.  */
-#define PPC_OPCODE_CLASSIC	   0x10000
-
-/* Opcode is only supported by e500x2 Core.  */
-#define PPC_OPCODE_SPE		   0x20000
-
-/* Opcode is supported by e500x2 Integer select APU.  */
-#define PPC_OPCODE_ISEL		   0x40000
-
-/* Opcode is an e500 SPE floating point instruction.  */
-#define PPC_OPCODE_EFS		   0x80000
-
-/* Opcode is supported by branch locking APU.  */
-#define PPC_OPCODE_BRLOCK	  0x100000
-
-/* Opcode is supported by performance monitor APU.  */
-#define PPC_OPCODE_PMR		  0x200000
-
-/* Opcode is supported by cache locking APU.  */
-#define PPC_OPCODE_CACHELCK	  0x400000
-
-/* Opcode is supported by machine check APU.  */
-#define PPC_OPCODE_RFMCI	  0x800000
-
-/* Opcode is only supported by Power5 architecture.  */
-#define PPC_OPCODE_POWER5	 0x1000000
-
-/* Opcode is supported by PowerPC e300 family.  */
-#define PPC_OPCODE_E300          0x2000000
-
-/* Opcode is only supported by Power6 architecture.  */
-#define PPC_OPCODE_POWER6	 0x4000000
-
-/* Opcode is only supported by PowerPC Cell family.  */
-#define PPC_OPCODE_CELL		 0x8000000
-
-/* A macro to extract the major opcode from an instruction.  */
-#define PPC_OP(i) (((i) >> 26) & 0x3f)
-
-/* The operands table is an array of struct powerpc_operand.  */
-
-struct powerpc_operand
-{
-  /* A bitmask of bits in the operand.  */
-  unsigned int bitm;
-
-  /* How far the operand is left shifted in the instruction.
-     -1 to indicate that BITM and SHIFT cannot be used to determine
-     where the operand goes in the insn.  */
-  int shift;
-
-  /* Insertion function.  This is used by the assembler.  To insert an
-     operand value into an instruction, check this field.
-
-     If it is NULL, execute
-	 i |= (op & o->bitm) << o->shift;
-     (i is the instruction which we are filling in, o is a pointer to
-     this structure, and op is the operand value).
-
-     If this field is not NULL, then simply call it with the
-     instruction and the operand value.  It will return the new value
-     of the instruction.  If the ERRMSG argument is not NULL, then if
-     the operand value is illegal, *ERRMSG will be set to a warning
-     string (the operand will be inserted in any case).  If the
-     operand value is legal, *ERRMSG will be unchanged (most operands
-     can accept any value).  */
-  unsigned long (*insert)
-    (unsigned long instruction, long op, int dialect, const char **errmsg);
-
-  /* Extraction function.  This is used by the disassembler.  To
-     extract this operand type from an instruction, check this field.
-
-     If it is NULL, compute
-	 op = (i >> o->shift) & o->bitm;
-	 if ((o->flags & PPC_OPERAND_SIGNED) != 0)
-	   sign_extend (op);
-     (i is the instruction, o is a pointer to this structure, and op
-     is the result).
-
-     If this field is not NULL, then simply call it with the
-     instruction value.  It will return the value of the operand.  If
-     the INVALID argument is not NULL, *INVALID will be set to
-     non-zero if this operand type can not actually be extracted from
-     this operand (i.e., the instruction does not match).  If the
-     operand is valid, *INVALID will not be changed.  */
-  long (*extract) (unsigned long instruction, int dialect, int *invalid);
-
-  /* One bit syntax flags.  */
-  unsigned long flags;
-};
-
-/* Elements in the table are retrieved by indexing with values from
-   the operands field of the powerpc_opcodes table.  */
-
-extern const struct powerpc_operand powerpc_operands[];
-extern const unsigned int num_powerpc_operands;
-
-/* Values defined for the flags field of a struct powerpc_operand.  */
-
-/* This operand takes signed values.  */
-#define PPC_OPERAND_SIGNED (0x1)
-
-/* This operand takes signed values, but also accepts a full positive
-   range of values when running in 32 bit mode.  That is, if bits is
-   16, it takes any value from -0x8000 to 0xffff.  In 64 bit mode,
-   this flag is ignored.  */
-#define PPC_OPERAND_SIGNOPT (0x2)
-
-/* This operand does not actually exist in the assembler input.  This
-   is used to support extended mnemonics such as mr, for which two
-   operands fields are identical.  The assembler should call the
-   insert function with any op value.  The disassembler should call
-   the extract function, ignore the return value, and check the value
-   placed in the valid argument.  */
-#define PPC_OPERAND_FAKE (0x4)
-
-/* The next operand should be wrapped in parentheses rather than
-   separated from this one by a comma.  This is used for the load and
-   store instructions which want their operands to look like
-       reg,displacement(reg)
-   */
-#define PPC_OPERAND_PARENS (0x8)
-
-/* This operand may use the symbolic names for the CR fields, which
-   are
-       lt  0	gt  1	eq  2	so  3	un  3
-       cr0 0	cr1 1	cr2 2	cr3 3
-       cr4 4	cr5 5	cr6 6	cr7 7
-   These may be combined arithmetically, as in cr2*4+gt.  These are
-   only supported on the PowerPC, not the POWER.  */
-#define PPC_OPERAND_CR (0x10)
-
-/* This operand names a register.  The disassembler uses this to print
-   register names with a leading 'r'.  */
-#define PPC_OPERAND_GPR (0x20)
-
-/* Like PPC_OPERAND_GPR, but don't print a leading 'r' for r0.  */
-#define PPC_OPERAND_GPR_0 (0x40)
-
-/* This operand names a floating point register.  The disassembler
-   prints these with a leading 'f'.  */
-#define PPC_OPERAND_FPR (0x80)
-
-/* This operand is a relative branch displacement.  The disassembler
-   prints these symbolically if possible.  */
-#define PPC_OPERAND_RELATIVE (0x100)
-
-/* This operand is an absolute branch address.  The disassembler
-   prints these symbolically if possible.  */
-#define PPC_OPERAND_ABSOLUTE (0x200)
-
-/* This operand is optional, and is zero if omitted.  This is used for
-   example, in the optional BF field in the comparison instructions.  The
-   assembler must count the number of operands remaining on the line,
-   and the number of operands remaining for the opcode, and decide
-   whether this operand is present or not.  The disassembler should
-   print this operand out only if it is not zero.  */
-#define PPC_OPERAND_OPTIONAL (0x400)
-
-/* This flag is only used with PPC_OPERAND_OPTIONAL.  If this operand
-   is omitted, then for the next operand use this operand value plus
-   1, ignoring the next operand field for the opcode.  This wretched
-   hack is needed because the Power rotate instructions can take
-   either 4 or 5 operands.  The disassembler should print this operand
-   out regardless of the PPC_OPERAND_OPTIONAL field.  */
-#define PPC_OPERAND_NEXT (0x800)
-
-/* This operand should be regarded as a negative number for the
-   purposes of overflow checking (i.e., the normal most negative
-   number is disallowed and one more than the normal most positive
-   number is allowed).  This flag will only be set for a signed
-   operand.  */
-#define PPC_OPERAND_NEGATIVE (0x1000)
-
-/* This operand names a vector unit register.  The disassembler
-   prints these with a leading 'v'.  */
-#define PPC_OPERAND_VR (0x2000)
-
-/* This operand is for the DS field in a DS form instruction.  */
-#define PPC_OPERAND_DS (0x4000)
-
-/* This operand is for the DQ field in a DQ form instruction.  */
-#define PPC_OPERAND_DQ (0x8000)
-
-/* Valid range of operand is 0..n rather than 0..n-1.  */
-#define PPC_OPERAND_PLUS1 (0x10000)
-
-/* The POWER and PowerPC assemblers use a few macros.  We keep them
-   with the operands table for simplicity.  The macro table is an
-   array of struct powerpc_macro.  */
-
-struct powerpc_macro
-{
-  /* The macro name.  */
-  const char *name;
-
-  /* The number of operands the macro takes.  */
-  unsigned int operands;
-
-  /* One bit flags for the opcode.  These are used to indicate which
-     specific processors support the instructions.  The values are the
-     same as those for the struct powerpc_opcode flags field.  */
-  unsigned long flags;
-
-  /* A format string to turn the macro into a normal instruction.
-     Each %N in the string is replaced with operand number N (zero
-     based).  */
-  const char *format;
-};
-
-extern const struct powerpc_macro powerpc_macros[];
-extern const int powerpc_num_macros;
-
-/* ppc-opc.c -- PowerPC opcode list
-   Copyright 1994, 1995, 1996, 1997, 1998, 2000, 2001, 2002, 2003, 2004,
-   2005, 2006, 2007 Free Software Foundation, Inc.
-   Written by Ian Lance Taylor, Cygnus Support
-
-   This file is part of GDB, GAS, and the GNU binutils.
-
-   GDB, GAS, and the GNU binutils are free software; you can redistribute
-   them and/or modify them under the terms of the GNU General Public
-   License as published by the Free Software Foundation; either version
-   2, or (at your option) any later version.
-
-   GDB, GAS, and the GNU binutils are distributed in the hope that they
-   will be useful, but WITHOUT ANY WARRANTY; without even the implied
-   warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
-   the GNU General Public License for more details.
-
-   You should have received a copy of the GNU General Public License
-   along with this file; see the file COPYING.
-   If not, see <http://www.gnu.org/licenses/>.  */
-
-/* This file holds the PowerPC opcode table.  The opcode table
-   includes almost all of the extended instruction mnemonics.  This
-   permits the disassembler to use them, and simplifies the assembler
-   logic, at the cost of increasing the table size.  The table is
-   strictly constant data, so the compiler should be able to put it in
-   the .text section.
-
-   This file also holds the operand table.  All knowledge about
-   inserting operands into instructions and vice-versa is kept in this
-   file.  */
-
-/* Local insertion and extraction functions.  */
-
-static unsigned long insert_bat (unsigned long, long, int, const char **);
-static long extract_bat (unsigned long, int, int *);
-static unsigned long insert_bba (unsigned long, long, int, const char **);
-static long extract_bba (unsigned long, int, int *);
-static unsigned long insert_bdm (unsigned long, long, int, const char **);
-static long extract_bdm (unsigned long, int, int *);
-static unsigned long insert_bdp (unsigned long, long, int, const char **);
-static long extract_bdp (unsigned long, int, int *);
-static unsigned long insert_bo (unsigned long, long, int, const char **);
-static long extract_bo (unsigned long, int, int *);
-static unsigned long insert_boe (unsigned long, long, int, const char **);
-static long extract_boe (unsigned long, int, int *);
-static unsigned long insert_fxm (unsigned long, long, int, const char **);
-static long extract_fxm (unsigned long, int, int *);
-static unsigned long insert_mbe (unsigned long, long, int, const char **);
-static long extract_mbe (unsigned long, int, int *);
-static unsigned long insert_mb6 (unsigned long, long, int, const char **);
-static long extract_mb6 (unsigned long, int, int *);
-static long extract_nb (unsigned long, int, int *);
-static unsigned long insert_nsi (unsigned long, long, int, const char **);
-static long extract_nsi (unsigned long, int, int *);
-static unsigned long insert_ral (unsigned long, long, int, const char **);
-static unsigned long insert_ram (unsigned long, long, int, const char **);
-static unsigned long insert_raq (unsigned long, long, int, const char **);
-static unsigned long insert_ras (unsigned long, long, int, const char **);
-static unsigned long insert_rbs (unsigned long, long, int, const char **);
-static long extract_rbs (unsigned long, int, int *);
-static unsigned long insert_sh6 (unsigned long, long, int, const char **);
-static long extract_sh6 (unsigned long, int, int *);
-static unsigned long insert_spr (unsigned long, long, int, const char **);
-static long extract_spr (unsigned long, int, int *);
-static unsigned long insert_sprg (unsigned long, long, int, const char **);
-static long extract_sprg (unsigned long, int, int *);
-static unsigned long insert_tbr (unsigned long, long, int, const char **);
-static long extract_tbr (unsigned long, int, int *);
-
-/* The operands table.
-
-   The fields are bitm, shift, insert, extract, flags.
-
-   We used to put parens around the various additions, like the one
-   for BA just below.  However, that caused trouble with feeble
-   compilers with a limit on depth of a parenthesized expression, like
-   (reportedly) the compiler in Microsoft Developer Studio 5.  So we
-   omit the parens, since the macros are never used in a context where
-   the addition will be ambiguous.  */
-
-const struct powerpc_operand powerpc_operands[] =
-{
-  /* The zero index is used to indicate the end of the list of
-     operands.  */
-#define UNUSED 0
-  { 0, 0, NULL, NULL, 0 },
-
-  /* The BA field in an XL form instruction.  */
-#define BA UNUSED + 1
-  /* The BI field in a B form or XL form instruction.  */
-#define BI BA
-#define BI_MASK (0x1f << 16)
-  { 0x1f, 16, NULL, NULL, PPC_OPERAND_CR },
-
-  /* The BA field in an XL form instruction when it must be the same
-     as the BT field in the same instruction.  */
-#define BAT BA + 1
-  { 0x1f, 16, insert_bat, extract_bat, PPC_OPERAND_FAKE },
-
-  /* The BB field in an XL form instruction.  */
-#define BB BAT + 1
-#define BB_MASK (0x1f << 11)
-  { 0x1f, 11, NULL, NULL, PPC_OPERAND_CR },
-
-  /* The BB field in an XL form instruction when it must be the same
-     as the BA field in the same instruction.  */
-#define BBA BB + 1
-  { 0x1f, 11, insert_bba, extract_bba, PPC_OPERAND_FAKE },
-
-  /* The BD field in a B form instruction.  The lower two bits are
-     forced to zero.  */
-#define BD BBA + 1
-  { 0xfffc, 0, NULL, NULL, PPC_OPERAND_RELATIVE | PPC_OPERAND_SIGNED },
-
-  /* The BD field in a B form instruction when absolute addressing is
-     used.  */
-#define BDA BD + 1
-  { 0xfffc, 0, NULL, NULL, PPC_OPERAND_ABSOLUTE | PPC_OPERAND_SIGNED },
-
-  /* The BD field in a B form instruction when the - modifier is used.
-     This sets the y bit of the BO field appropriately.  */
-#define BDM BDA + 1
-  { 0xfffc, 0, insert_bdm, extract_bdm,
-      PPC_OPERAND_RELATIVE | PPC_OPERAND_SIGNED },
-
-  /* The BD field in a B form instruction when the - modifier is used
-     and absolute address is used.  */
-#define BDMA BDM + 1
-  { 0xfffc, 0, insert_bdm, extract_bdm,
-      PPC_OPERAND_ABSOLUTE | PPC_OPERAND_SIGNED },
-
-  /* The BD field in a B form instruction when the + modifier is used.
-     This sets the y bit of the BO field appropriately.  */
-#define BDP BDMA + 1
-  { 0xfffc, 0, insert_bdp, extract_bdp,
-      PPC_OPERAND_RELATIVE | PPC_OPERAND_SIGNED },
-
-  /* The BD field in a B form instruction when the + modifier is used
-     and absolute addressing is used.  */
-#define BDPA BDP + 1
-  { 0xfffc, 0, insert_bdp, extract_bdp,
-      PPC_OPERAND_ABSOLUTE | PPC_OPERAND_SIGNED },
-
-  /* The BF field in an X or XL form instruction.  */
-#define BF BDPA + 1
-  /* The CRFD field in an X form instruction.  */
-#define CRFD BF
-  { 0x7, 23, NULL, NULL, PPC_OPERAND_CR },
-
-  /* The BF field in an X or XL form instruction.  */
-#define BFF BF + 1
-  { 0x7, 23, NULL, NULL, 0 },
-
-  /* An optional BF field.  This is used for comparison instructions,
-     in which an omitted BF field is taken as zero.  */
-#define OBF BFF + 1
-  { 0x7, 23, NULL, NULL, PPC_OPERAND_CR | PPC_OPERAND_OPTIONAL },
-
-  /* The BFA field in an X or XL form instruction.  */
-#define BFA OBF + 1
-  { 0x7, 18, NULL, NULL, PPC_OPERAND_CR },
-
-  /* The BO field in a B form instruction.  Certain values are
-     illegal.  */
-#define BO BFA + 1
-#define BO_MASK (0x1f << 21)
-  { 0x1f, 21, insert_bo, extract_bo, 0 },
-
-  /* The BO field in a B form instruction when the + or - modifier is
-     used.  This is like the BO field, but it must be even.  */
-#define BOE BO + 1
-  { 0x1e, 21, insert_boe, extract_boe, 0 },
-
-#define BH BOE + 1
-  { 0x3, 11, NULL, NULL, PPC_OPERAND_OPTIONAL },
-
-  /* The BT field in an X or XL form instruction.  */
-#define BT BH + 1
-  { 0x1f, 21, NULL, NULL, PPC_OPERAND_CR },
-
-  /* The condition register number portion of the BI field in a B form
-     or XL form instruction.  This is used for the extended
-     conditional branch mnemonics, which set the lower two bits of the
-     BI field.  This field is optional.  */
-#define CR BT + 1
-  { 0x7, 18, NULL, NULL, PPC_OPERAND_CR | PPC_OPERAND_OPTIONAL },
-
-  /* The CRB field in an X form instruction.  */
-#define CRB CR + 1
-  /* The MB field in an M form instruction.  */
-#define MB CRB
-#define MB_MASK (0x1f << 6)
-  { 0x1f, 6, NULL, NULL, 0 },
-
-  /* The CRFS field in an X form instruction.  */
-#define CRFS CRB + 1
-  { 0x7, 0, NULL, NULL, PPC_OPERAND_CR },
-
-  /* The CT field in an X form instruction.  */
-#define CT CRFS + 1
-  /* The MO field in an mbar instruction.  */
-#define MO CT
-  { 0x1f, 21, NULL, NULL, PPC_OPERAND_OPTIONAL },
-
-  /* The D field in a D form instruction.  This is a displacement off
-     a register, and implies that the next operand is a register in
-     parentheses.  */
-#define D CT + 1
-  { 0xffff, 0, NULL, NULL, PPC_OPERAND_PARENS | PPC_OPERAND_SIGNED },
-
-  /* The DE field in a DE form instruction.  This is like D, but is 12
-     bits only.  */
-#define DE D + 1
-  { 0xfff, 4, NULL, NULL, PPC_OPERAND_PARENS | PPC_OPERAND_SIGNED },
-
-  /* The DES field in a DES form instruction.  This is like DS, but is 14
-     bits only (12 stored.)  */
-#define DES DE + 1
-  { 0x3ffc, 2, NULL, NULL, PPC_OPERAND_PARENS | PPC_OPERAND_SIGNED },
-
-  /* The DQ field in a DQ form instruction.  This is like D, but the
-     lower four bits are forced to zero. */
-#define DQ DES + 1
-  { 0xfff0, 0, NULL, NULL,
-    PPC_OPERAND_PARENS | PPC_OPERAND_SIGNED | PPC_OPERAND_DQ },
-
-  /* The DS field in a DS form instruction.  This is like D, but the
-     lower two bits are forced to zero.  */
-#undef DS
-#define DS DQ + 1
-  { 0xfffc, 0, NULL, NULL,
-    PPC_OPERAND_PARENS | PPC_OPERAND_SIGNED | PPC_OPERAND_DS },
-
-  /* The E field in a wrteei instruction.  */
-#define E DS + 1
-  { 0x1, 15, NULL, NULL, 0 },
-
-  /* The FL1 field in a POWER SC form instruction.  */
-#define FL1 E + 1
-  /* The U field in an X form instruction.  */
-#define U FL1
-  { 0xf, 12, NULL, NULL, 0 },
-
-  /* The FL2 field in a POWER SC form instruction.  */
-#define FL2 FL1 + 1
-  { 0x7, 2, NULL, NULL, 0 },
-
-  /* The FLM field in an XFL form instruction.  */
-#define FLM FL2 + 1
-  { 0xff, 17, NULL, NULL, 0 },
-
-  /* The FRA field in an X or A form instruction.  */
-#define FRA FLM + 1
-#define FRA_MASK (0x1f << 16)
-  { 0x1f, 16, NULL, NULL, PPC_OPERAND_FPR },
-
-  /* The FRB field in an X or A form instruction.  */
-#define FRB FRA + 1
-#define FRB_MASK (0x1f << 11)
-  { 0x1f, 11, NULL, NULL, PPC_OPERAND_FPR },
-
-  /* The FRC field in an A form instruction.  */
-#define FRC FRB + 1
-#define FRC_MASK (0x1f << 6)
-  { 0x1f, 6, NULL, NULL, PPC_OPERAND_FPR },
-
-  /* The FRS field in an X form instruction or the FRT field in a D, X
-     or A form instruction.  */
-#define FRS FRC + 1
-#define FRT FRS
-  { 0x1f, 21, NULL, NULL, PPC_OPERAND_FPR },
-
-  /* The FXM field in an XFX instruction.  */
-#define FXM FRS + 1
-  { 0xff, 12, insert_fxm, extract_fxm, 0 },
-
-  /* Power4 version for mfcr.  */
-#define FXM4 FXM + 1
-  { 0xff, 12, insert_fxm, extract_fxm, PPC_OPERAND_OPTIONAL },
-
-  /* The L field in a D or X form instruction.  */
-#define L FXM4 + 1
-  { 0x1, 21, NULL, NULL, PPC_OPERAND_OPTIONAL },
-
-  /* The LEV field in a POWER SVC form instruction.  */
-#define SVC_LEV L + 1
-  { 0x7f, 5, NULL, NULL, 0 },
-
-  /* The LEV field in an SC form instruction.  */
-#define LEV SVC_LEV + 1
-  { 0x7f, 5, NULL, NULL, PPC_OPERAND_OPTIONAL },
-
-  /* The LI field in an I form instruction.  The lower two bits are
-     forced to zero.  */
-#define LI LEV + 1
-  { 0x3fffffc, 0, NULL, NULL, PPC_OPERAND_RELATIVE | PPC_OPERAND_SIGNED },
-
-  /* The LI field in an I form instruction when used as an absolute
-     address.  */
-#define LIA LI + 1
-  { 0x3fffffc, 0, NULL, NULL, PPC_OPERAND_ABSOLUTE | PPC_OPERAND_SIGNED },
-
-  /* The LS field in an X (sync) form instruction.  */
-#define LS LIA + 1
-  { 0x3, 21, NULL, NULL, PPC_OPERAND_OPTIONAL },
-
-  /* The ME field in an M form instruction.  */
-#define ME LS + 1
-#define ME_MASK (0x1f << 1)
-  { 0x1f, 1, NULL, NULL, 0 },
-
-  /* The MB and ME fields in an M form instruction expressed a single
-     operand which is a bitmask indicating which bits to select.  This
-     is a two operand form using PPC_OPERAND_NEXT.  See the
-     description in opcode/ppc.h for what this means.  */
-#define MBE ME + 1
-  { 0x1f, 6, NULL, NULL, PPC_OPERAND_OPTIONAL | PPC_OPERAND_NEXT },
-  { -1, 0, insert_mbe, extract_mbe, 0 },
-
-  /* The MB or ME field in an MD or MDS form instruction.  The high
-     bit is wrapped to the low end.  */
-#define MB6 MBE + 2
-#define ME6 MB6
-#define MB6_MASK (0x3f << 5)
-  { 0x3f, 5, insert_mb6, extract_mb6, 0 },
-
-  /* The NB field in an X form instruction.  The value 32 is stored as
-     0.  */
-#define NB MB6 + 1
-  { 0x1f, 11, NULL, extract_nb, PPC_OPERAND_PLUS1 },
-
-  /* The NSI field in a D form instruction.  This is the same as the
-     SI field, only negated.  */
-#define NSI NB + 1
-  { 0xffff, 0, insert_nsi, extract_nsi,
-      PPC_OPERAND_NEGATIVE | PPC_OPERAND_SIGNED },
-
-  /* The RA field in an D, DS, DQ, X, XO, M, or MDS form instruction.  */
-#define RA NSI + 1
-#define RA_MASK (0x1f << 16)
-  { 0x1f, 16, NULL, NULL, PPC_OPERAND_GPR },
-
-  /* As above, but 0 in the RA field means zero, not r0.  */
-#define RA0 RA + 1
-  { 0x1f, 16, NULL, NULL, PPC_OPERAND_GPR_0 },
-
-  /* The RA field in the DQ form lq instruction, which has special
-     value restrictions.  */
-#define RAQ RA0 + 1
-  { 0x1f, 16, insert_raq, NULL, PPC_OPERAND_GPR_0 },
-
-  /* The RA field in a D or X form instruction which is an updating
-     load, which means that the RA field may not be zero and may not
-     equal the RT field.  */
-#define RAL RAQ + 1
-  { 0x1f, 16, insert_ral, NULL, PPC_OPERAND_GPR_0 },
-
-  /* The RA field in an lmw instruction, which has special value
-     restrictions.  */
-#define RAM RAL + 1
-  { 0x1f, 16, insert_ram, NULL, PPC_OPERAND_GPR_0 },
-
-  /* The RA field in a D or X form instruction which is an updating
-     store or an updating floating point load, which means that the RA
-     field may not be zero.  */
-#define RAS RAM + 1
-  { 0x1f, 16, insert_ras, NULL, PPC_OPERAND_GPR_0 },
-
-  /* The RA field of the tlbwe instruction, which is optional.  */
-#define RAOPT RAS + 1
-  { 0x1f, 16, NULL, NULL, PPC_OPERAND_GPR | PPC_OPERAND_OPTIONAL },
-
-  /* The RB field in an X, XO, M, or MDS form instruction.  */
-#define RB RAOPT + 1
-#define RB_MASK (0x1f << 11)
-  { 0x1f, 11, NULL, NULL, PPC_OPERAND_GPR },
-
-  /* The RB field in an X form instruction when it must be the same as
-     the RS field in the instruction.  This is used for extended
-     mnemonics like mr.  */
-#define RBS RB + 1
-  { 0x1f, 11, insert_rbs, extract_rbs, PPC_OPERAND_FAKE },
-
-  /* The RS field in a D, DS, X, XFX, XS, M, MD or MDS form
-     instruction or the RT field in a D, DS, X, XFX or XO form
-     instruction.  */
-#define RS RBS + 1
-#define RT RS
-#define RT_MASK (0x1f << 21)
-  { 0x1f, 21, NULL, NULL, PPC_OPERAND_GPR },
-
-  /* The RS and RT fields of the DS form stq instruction, which have
-     special value restrictions.  */
-#define RSQ RS + 1
-#define RTQ RSQ
-  { 0x1e, 21, NULL, NULL, PPC_OPERAND_GPR_0 },
-
-  /* The RS field of the tlbwe instruction, which is optional.  */
-#define RSO RSQ + 1
-#define RTO RSO
-  { 0x1f, 21, NULL, NULL, PPC_OPERAND_GPR | PPC_OPERAND_OPTIONAL },
-
-  /* The SH field in an X or M form instruction.  */
-#define SH RSO + 1
-#define SH_MASK (0x1f << 11)
-  /* The other UIMM field in a EVX form instruction.  */
-#define EVUIMM SH
-  { 0x1f, 11, NULL, NULL, 0 },
-
-  /* The SH field in an MD form instruction.  This is split.  */
-#define SH6 SH + 1
-#define SH6_MASK ((0x1f << 11) | (1 << 1))
-  { 0x3f, -1, insert_sh6, extract_sh6, 0 },
-
-  /* The SH field of the tlbwe instruction, which is optional.  */
-#define SHO SH6 + 1
-  { 0x1f, 11, NULL, NULL, PPC_OPERAND_OPTIONAL },
-
-  /* The SI field in a D form instruction.  */
-#define SI SHO + 1
-  { 0xffff, 0, NULL, NULL, PPC_OPERAND_SIGNED },
-
-  /* The SI field in a D form instruction when we accept a wide range
-     of positive values.  */
-#define SISIGNOPT SI + 1
-  { 0xffff, 0, NULL, NULL, PPC_OPERAND_SIGNED | PPC_OPERAND_SIGNOPT },
-
-  /* The SPR field in an XFX form instruction.  This is flipped--the
-     lower 5 bits are stored in the upper 5 and vice- versa.  */
-#define SPR SISIGNOPT + 1
-#define PMR SPR
-#define SPR_MASK (0x3ff << 11)
-  { 0x3ff, 11, insert_spr, extract_spr, 0 },
-
-  /* The BAT index number in an XFX form m[ft]ibat[lu] instruction.  */
-#define SPRBAT SPR + 1
-#define SPRBAT_MASK (0x3 << 17)
-  { 0x3, 17, NULL, NULL, 0 },
-
-  /* The SPRG register number in an XFX form m[ft]sprg instruction.  */
-#define SPRG SPRBAT + 1
-  { 0x1f, 16, insert_sprg, extract_sprg, 0 },
-
-  /* The SR field in an X form instruction.  */
-#define SR SPRG + 1
-  { 0xf, 16, NULL, NULL, 0 },
-
-  /* The STRM field in an X AltiVec form instruction.  */
-#define STRM SR + 1
-  { 0x3, 21, NULL, NULL, 0 },
-
-  /* The SV field in a POWER SC form instruction.  */
-#define SV STRM + 1
-  { 0x3fff, 2, NULL, NULL, 0 },
-
-  /* The TBR field in an XFX form instruction.  This is like the SPR
-     field, but it is optional.  */
-#define TBR SV + 1
-  { 0x3ff, 11, insert_tbr, extract_tbr, PPC_OPERAND_OPTIONAL },
-
-  /* The TO field in a D or X form instruction.  */
-#define TO TBR + 1
-#define TO_MASK (0x1f << 21)
-  { 0x1f, 21, NULL, NULL, 0 },
-
-  /* The UI field in a D form instruction.  */
-#define UI TO + 1
-  { 0xffff, 0, NULL, NULL, 0 },
-
-  /* The VA field in a VA, VX or VXR form instruction.  */
-#define VA UI + 1
-  { 0x1f, 16, NULL, NULL, PPC_OPERAND_VR },
-
-  /* The VB field in a VA, VX or VXR form instruction.  */
-#define VB VA + 1
-  { 0x1f, 11, NULL, NULL, PPC_OPERAND_VR },
-
-  /* The VC field in a VA form instruction.  */
-#define VC VB + 1
-  { 0x1f, 6, NULL, NULL, PPC_OPERAND_VR },
-
-  /* The VD or VS field in a VA, VX, VXR or X form instruction.  */
-#define VD VC + 1
-#define VS VD
-  { 0x1f, 21, NULL, NULL, PPC_OPERAND_VR },
-
-  /* The SIMM field in a VX form instruction.  */
-#define SIMM VD + 1
-  { 0x1f, 16, NULL, NULL, PPC_OPERAND_SIGNED},
-
-  /* The UIMM field in a VX form instruction, and TE in Z form.  */
-#define UIMM SIMM + 1
-#define TE UIMM
-  { 0x1f, 16, NULL, NULL, 0 },
-
-  /* The SHB field in a VA form instruction.  */
-#define SHB UIMM + 1
-  { 0xf, 6, NULL, NULL, 0 },
-
-  /* The other UIMM field in a half word EVX form instruction.  */
-#define EVUIMM_2 SHB + 1
-  { 0x3e, 10, NULL, NULL, PPC_OPERAND_PARENS },
-
-  /* The other UIMM field in a word EVX form instruction.  */
-#define EVUIMM_4 EVUIMM_2 + 1
-  { 0x7c, 9, NULL, NULL, PPC_OPERAND_PARENS },
-
-  /* The other UIMM field in a double EVX form instruction.  */
-#define EVUIMM_8 EVUIMM_4 + 1
-  { 0xf8, 8, NULL, NULL, PPC_OPERAND_PARENS },
-
-  /* The WS field.  */
-#define WS EVUIMM_8 + 1
-  { 0x7, 11, NULL, NULL, 0 },
-
-  /* The L field in an mtmsrd or A form instruction or W in an X form.  */
-#define A_L WS + 1
-#define W A_L
-  { 0x1, 16, NULL, NULL, PPC_OPERAND_OPTIONAL },
-
-#define RMC A_L + 1
-  { 0x3, 9, NULL, NULL, 0 },
-
-#define R RMC + 1
-  { 0x1, 16, NULL, NULL, 0 },
-
-#define SP R + 1
-  { 0x3, 19, NULL, NULL, 0 },
-
-#define S SP + 1
-  { 0x1, 20, NULL, NULL, 0 },
-
-  /* SH field starting at bit position 16.  */
-#define SH16 S + 1
-  /* The DCM and DGM fields in a Z form instruction.  */
-#define DCM SH16
-#define DGM DCM
-  { 0x3f, 10, NULL, NULL, 0 },
-
-  /* The EH field in larx instruction.  */
-#define EH SH16 + 1
-  { 0x1, 0, NULL, NULL, PPC_OPERAND_OPTIONAL },
-
-  /* The L field in an mtfsf or XFL form instruction.  */
-#define XFL_L EH + 1
-  { 0x1, 25, NULL, NULL, PPC_OPERAND_OPTIONAL},
-};
-
-const unsigned int num_powerpc_operands = (sizeof (powerpc_operands)
-					   / sizeof (powerpc_operands[0]));
-
-/* The functions used to insert and extract complicated operands.  */
-
-/* The BA field in an XL form instruction when it must be the same as
-   the BT field in the same instruction.  This operand is marked FAKE.
-   The insertion function just copies the BT field into the BA field,
-   and the extraction function just checks that the fields are the
-   same.  */
-
-static unsigned long
-insert_bat (unsigned long insn,
-	    long value ATTRIBUTE_UNUSED,
-	    int dialect ATTRIBUTE_UNUSED,
-	    const char **errmsg ATTRIBUTE_UNUSED)
-{
-  return insn | (((insn >> 21) & 0x1f) << 16);
-}
-
-static long
-extract_bat (unsigned long insn,
-	     int dialect ATTRIBUTE_UNUSED,
-	     int *invalid)
-{
-  if (((insn >> 21) & 0x1f) != ((insn >> 16) & 0x1f))
-    *invalid = 1;
-  return 0;
-}
-
-/* The BB field in an XL form instruction when it must be the same as
-   the BA field in the same instruction.  This operand is marked FAKE.
-   The insertion function just copies the BA field into the BB field,
-   and the extraction function just checks that the fields are the
-   same.  */
-
-static unsigned long
-insert_bba (unsigned long insn,
-	    long value ATTRIBUTE_UNUSED,
-	    int dialect ATTRIBUTE_UNUSED,
-	    const char **errmsg ATTRIBUTE_UNUSED)
-{
-  return insn | (((insn >> 16) & 0x1f) << 11);
-}
-
-static long
-extract_bba (unsigned long insn,
-	     int dialect ATTRIBUTE_UNUSED,
-	     int *invalid)
-{
-  if (((insn >> 16) & 0x1f) != ((insn >> 11) & 0x1f))
-    *invalid = 1;
-  return 0;
-}
-
-/* The BD field in a B form instruction when the - modifier is used.
-   This modifier means that the branch is not expected to be taken.
-   For chips built to versions of the architecture prior to version 2
-   (ie. not Power4 compatible), we set the y bit of the BO field to 1
-   if the offset is negative.  When extracting, we require that the y
-   bit be 1 and that the offset be positive, since if the y bit is 0
-   we just want to print the normal form of the instruction.
-   Power4 compatible targets use two bits, "a", and "t", instead of
-   the "y" bit.  "at" == 00 => no hint, "at" == 01 => unpredictable,
-   "at" == 10 => not taken, "at" == 11 => taken.  The "t" bit is 00001
-   in BO field, the "a" bit is 00010 for branch on CR(BI) and 01000
-   for branch on CTR.  We only handle the taken/not-taken hint here.
-   Note that we don't relax the conditions tested here when
-   disassembling with -Many because insns using extract_bdm and
-   extract_bdp always occur in pairs.  One or the other will always
-   be valid.  */
-
-static unsigned long
-insert_bdm (unsigned long insn,
-	    long value,
-	    int dialect,
-	    const char **errmsg ATTRIBUTE_UNUSED)
-{
-  if ((dialect & PPC_OPCODE_POWER4) == 0)
-    {
-      if ((value & 0x8000) != 0)
-	insn |= 1 << 21;
-    }
-  else
-    {
-      if ((insn & (0x14 << 21)) == (0x04 << 21))
-	insn |= 0x02 << 21;
-      else if ((insn & (0x14 << 21)) == (0x10 << 21))
-	insn |= 0x08 << 21;
-    }
-  return insn | (value & 0xfffc);
-}
-
-static long
-extract_bdm (unsigned long insn,
-	     int dialect,
-	     int *invalid)
-{
-  if ((dialect & PPC_OPCODE_POWER4) == 0)
-    {
-      if (((insn & (1 << 21)) == 0) != ((insn & (1 << 15)) == 0))
-	*invalid = 1;
-    }
-  else
-    {
-      if ((insn & (0x17 << 21)) != (0x06 << 21)
-	  && (insn & (0x1d << 21)) != (0x18 << 21))
-	*invalid = 1;
-    }
-
-  return ((insn & 0xfffc) ^ 0x8000) - 0x8000;
-}
-
-/* The BD field in a B form instruction when the + modifier is used.
-   This is like BDM, above, except that the branch is expected to be
-   taken.  */
-
-static unsigned long
-insert_bdp (unsigned long insn,
-	    long value,
-	    int dialect,
-	    const char **errmsg ATTRIBUTE_UNUSED)
-{
-  if ((dialect & PPC_OPCODE_POWER4) == 0)
-    {
-      if ((value & 0x8000) == 0)
-	insn |= 1 << 21;
-    }
-  else
-    {
-      if ((insn & (0x14 << 21)) == (0x04 << 21))
-	insn |= 0x03 << 21;
-      else if ((insn & (0x14 << 21)) == (0x10 << 21))
-	insn |= 0x09 << 21;
-    }
-  return insn | (value & 0xfffc);
-}
-
-static long
-extract_bdp (unsigned long insn,
-	     int dialect,
-	     int *invalid)
-{
-  if ((dialect & PPC_OPCODE_POWER4) == 0)
-    {
-      if (((insn & (1 << 21)) == 0) == ((insn & (1 << 15)) == 0))
-	*invalid = 1;
-    }
-  else
-    {
-      if ((insn & (0x17 << 21)) != (0x07 << 21)
-	  && (insn & (0x1d << 21)) != (0x19 << 21))
-	*invalid = 1;
-    }
-
-  return ((insn & 0xfffc) ^ 0x8000) - 0x8000;
-}
-
-/* Check for legal values of a BO field.  */
-
-static int
-valid_bo (long value, int dialect, int extract)
-{
-  if ((dialect & PPC_OPCODE_POWER4) == 0)
-    {
-      int valid;
-      /* Certain encodings have bits that are required to be zero.
-	 These are (z must be zero, y may be anything):
-	     001zy
-	     011zy
-	     1z00y
-	     1z01y
-	     1z1zz
-      */
-      switch (value & 0x14)
-	{
-	default:
-	case 0:
-	  valid = 1;
-	  break;
-	case 0x4:
-	  valid = (value & 0x2) == 0;
-	  break;
-	case 0x10:
-	  valid = (value & 0x8) == 0;
-	  break;
-	case 0x14:
-	  valid = value == 0x14;
-	  break;
-	}
-      /* When disassembling with -Many, accept power4 encodings too.  */
-      if (valid
-	  || (dialect & PPC_OPCODE_ANY) == 0
-	  || !extract)
-	return valid;
-    }
-
-  /* Certain encodings have bits that are required to be zero.
-     These are (z must be zero, a & t may be anything):
-	 0000z
-	 0001z
-	 0100z
-	 0101z
-	 001at
-	 011at
-	 1a00t
-	 1a01t
-	 1z1zz
-  */
-  if ((value & 0x14) == 0)
-    return (value & 0x1) == 0;
-  else if ((value & 0x14) == 0x14)
-    return value == 0x14;
-  else
-    return 1;
-}
-
-/* The BO field in a B form instruction.  Warn about attempts to set
-   the field to an illegal value.  */
-
-static unsigned long
-insert_bo (unsigned long insn,
-	   long value,
-	   int dialect,
-	   const char **errmsg)
-{
-  if (!valid_bo (value, dialect, 0))
-    *errmsg = "invalid conditional option";
-  return insn | ((value & 0x1f) << 21);
-}
-
-static long
-extract_bo (unsigned long insn,
-	    int dialect,
-	    int *invalid)
-{
-  long value;
-
-  value = (insn >> 21) & 0x1f;
-  if (!valid_bo (value, dialect, 1))
-    *invalid = 1;
-  return value;
-}
-
-/* The BO field in a B form instruction when the + or - modifier is
-   used.  This is like the BO field, but it must be even.  When
-   extracting it, we force it to be even.  */
-
-static unsigned long
-insert_boe (unsigned long insn,
-	    long value,
-	    int dialect,
-	    const char **errmsg)
-{
-  if (!valid_bo (value, dialect, 0))
-    *errmsg = "invalid conditional option";
-  else if ((value & 1) != 0)
-    *errmsg = "attempt to set y bit when using + or - modifier";
-
-  return insn | ((value & 0x1f) << 21);
-}
-
-static long
-extract_boe (unsigned long insn,
-	     int dialect,
-	     int *invalid)
-{
-  long value;
-
-  value = (insn >> 21) & 0x1f;
-  if (!valid_bo (value, dialect, 1))
-    *invalid = 1;
-  return value & 0x1e;
-}
-
-/* FXM mask in mfcr and mtcrf instructions.  */
-
-static unsigned long
-insert_fxm (unsigned long insn,
-	    long value,
-	    int dialect,
-	    const char **errmsg)
-{
-  /* If we're handling the mfocrf and mtocrf insns ensure that exactly
-     one bit of the mask field is set.  */
-  if ((insn & (1 << 20)) != 0)
-    {
-      if (value == 0 || (value & -value) != value)
-	{
-	  *errmsg = "invalid mask field";
-	  value = 0;
-	}
-    }
-
-  /* If the optional field on mfcr is missing that means we want to use
-     the old form of the instruction that moves the whole cr.  In that
-     case we'll have VALUE zero.  There doesn't seem to be a way to
-     distinguish this from the case where someone writes mfcr %r3,0.  */
-  else if (value == 0)
-    ;
-
-  /* If only one bit of the FXM field is set, we can use the new form
-     of the instruction, which is faster.  Unlike the Power4 branch hint
-     encoding, this is not backward compatible.  Do not generate the
-     new form unless -mpower4 has been given, or -many and the two
-     operand form of mfcr was used.  */
-  else if ((value & -value) == value
-	   && ((dialect & PPC_OPCODE_POWER4) != 0
-	       || ((dialect & PPC_OPCODE_ANY) != 0
-		   && (insn & (0x3ff << 1)) == 19 << 1)))
-    insn |= 1 << 20;
-
-  /* Any other value on mfcr is an error.  */
-  else if ((insn & (0x3ff << 1)) == 19 << 1)
-    {
-      *errmsg = "ignoring invalid mfcr mask";
-      value = 0;
-    }
-
-  return insn | ((value & 0xff) << 12);
-}
-
-static long
-extract_fxm (unsigned long insn,
-	     int dialect ATTRIBUTE_UNUSED,
-	     int *invalid)
-{
-  long mask = (insn >> 12) & 0xff;
-
-  /* Is this a Power4 insn?  */
-  if ((insn & (1 << 20)) != 0)
-    {
-      /* Exactly one bit of MASK should be set.  */
-      if (mask == 0 || (mask & -mask) != mask)
-	*invalid = 1;
-    }
-
-  /* Check that non-power4 form of mfcr has a zero MASK.  */
-  else if ((insn & (0x3ff << 1)) == 19 << 1)
-    {
-      if (mask != 0)
-	*invalid = 1;
-    }
-
-  return mask;
-}
-
-/* The MB and ME fields in an M form instruction expressed as a single
-   operand which is itself a bitmask.  The extraction function always
-   marks it as invalid, since we never want to recognize an
-   instruction which uses a field of this type.  */
-
-static unsigned long
-insert_mbe (unsigned long insn,
-	    long value,
-	    int dialect ATTRIBUTE_UNUSED,
-	    const char **errmsg)
-{
-  unsigned long uval, mask;
-  int mb, me, mx, count, last;
-
-  uval = value;
-
-  if (uval == 0)
-    {
-      *errmsg = "illegal bitmask";
-      return insn;
-    }
-
-  mb = 0;
-  me = 32;
-  if ((uval & 1) != 0)
-    last = 1;
-  else
-    last = 0;
-  count = 0;
-
-  /* mb: location of last 0->1 transition */
-  /* me: location of last 1->0 transition */
-  /* count: # transitions */
-
-  for (mx = 0, mask = 1L << 31; mx < 32; ++mx, mask >>= 1)
-    {
-      if ((uval & mask) && !last)
-	{
-	  ++count;
-	  mb = mx;
-	  last = 1;
-	}
-      else if (!(uval & mask) && last)
-	{
-	  ++count;
-	  me = mx;
-	  last = 0;
-	}
-    }
-  if (me == 0)
-    me = 32;
-
-  if (count != 2 && (count != 0 || ! last))
-    *errmsg = "illegal bitmask";
-
-  return insn | (mb << 6) | ((me - 1) << 1);
-}
-
-static long
-extract_mbe (unsigned long insn,
-	     int dialect ATTRIBUTE_UNUSED,
-	     int *invalid)
-{
-  long ret;
-  int mb, me;
-  int i;
-
-  *invalid = 1;
-
-  mb = (insn >> 6) & 0x1f;
-  me = (insn >> 1) & 0x1f;
-  if (mb < me + 1)
-    {
-      ret = 0;
-      for (i = mb; i <= me; i++)
-	ret |= 1L << (31 - i);
-    }
-  else if (mb == me + 1)
-    ret = ~0;
-  else /* (mb > me + 1) */
-    {
-      ret = ~0;
-      for (i = me + 1; i < mb; i++)
-	ret &= ~(1L << (31 - i));
-    }
-  return ret;
-}
-
-/* The MB or ME field in an MD or MDS form instruction.  The high bit
-   is wrapped to the low end.  */
-
-static unsigned long
-insert_mb6 (unsigned long insn,
-	    long value,
-	    int dialect ATTRIBUTE_UNUSED,
-	    const char **errmsg ATTRIBUTE_UNUSED)
-{
-  return insn | ((value & 0x1f) << 6) | (value & 0x20);
-}
-
-static long
-extract_mb6 (unsigned long insn,
-	     int dialect ATTRIBUTE_UNUSED,
-	     int *invalid ATTRIBUTE_UNUSED)
-{
-  return ((insn >> 6) & 0x1f) | (insn & 0x20);
-}
-
-/* The NB field in an X form instruction.  The value 32 is stored as
-   0.  */
-
-static long
-extract_nb (unsigned long insn,
-	    int dialect ATTRIBUTE_UNUSED,
-	    int *invalid ATTRIBUTE_UNUSED)
-{
-  long ret;
-
-  ret = (insn >> 11) & 0x1f;
-  if (ret == 0)
-    ret = 32;
-  return ret;
-}
-
-/* The NSI field in a D form instruction.  This is the same as the SI
-   field, only negated.  The extraction function always marks it as
-   invalid, since we never want to recognize an instruction which uses
-   a field of this type.  */
-
-static unsigned long
-insert_nsi (unsigned long insn,
-	    long value,
-	    int dialect ATTRIBUTE_UNUSED,
-	    const char **errmsg ATTRIBUTE_UNUSED)
-{
-  return insn | (-value & 0xffff);
-}
-
-static long
-extract_nsi (unsigned long insn,
-	     int dialect ATTRIBUTE_UNUSED,
-	     int *invalid)
-{
-  *invalid = 1;
-  return -(((insn & 0xffff) ^ 0x8000) - 0x8000);
-}
-
-/* The RA field in a D or X form instruction which is an updating
-   load, which means that the RA field may not be zero and may not
-   equal the RT field.  */
-
-static unsigned long
-insert_ral (unsigned long insn,
-	    long value,
-	    int dialect ATTRIBUTE_UNUSED,
-	    const char **errmsg)
-{
-  if (value == 0
-      || (unsigned long) value == ((insn >> 21) & 0x1f))
-    *errmsg = "invalid register operand when updating";
-  return insn | ((value & 0x1f) << 16);
-}
-
-/* The RA field in an lmw instruction, which has special value
-   restrictions.  */
-
-static unsigned long
-insert_ram (unsigned long insn,
-	    long value,
-	    int dialect ATTRIBUTE_UNUSED,
-	    const char **errmsg)
-{
-  if ((unsigned long) value >= ((insn >> 21) & 0x1f))
-    *errmsg = "index register in load range";
-  return insn | ((value & 0x1f) << 16);
-}
-
-/* The RA field in the DQ form lq instruction, which has special
-   value restrictions.  */
-
-static unsigned long
-insert_raq (unsigned long insn,
-	    long value,
-	    int dialect ATTRIBUTE_UNUSED,
-	    const char **errmsg)
-{
-  long rtvalue = (insn & RT_MASK) >> 21;
-
-  if (value == rtvalue)
-    *errmsg = "source and target register operands must be different";
-  return insn | ((value & 0x1f) << 16);
-}
-
-/* The RA field in a D or X form instruction which is an updating
-   store or an updating floating point load, which means that the RA
-   field may not be zero.  */
-
-static unsigned long
-insert_ras (unsigned long insn,
-	    long value,
-	    int dialect ATTRIBUTE_UNUSED,
-	    const char **errmsg)
-{
-  if (value == 0)
-    *errmsg = "invalid register operand when updating";
-  return insn | ((value & 0x1f) << 16);
-}
-
-/* The RB field in an X form instruction when it must be the same as
-   the RS field in the instruction.  This is used for extended
-   mnemonics like mr.  This operand is marked FAKE.  The insertion
-   function just copies the BT field into the BA field, and the
-   extraction function just checks that the fields are the same.  */
-
-static unsigned long
-insert_rbs (unsigned long insn,
-	    long value ATTRIBUTE_UNUSED,
-	    int dialect ATTRIBUTE_UNUSED,
-	    const char **errmsg ATTRIBUTE_UNUSED)
-{
-  return insn | (((insn >> 21) & 0x1f) << 11);
-}
-
-static long
-extract_rbs (unsigned long insn,
-	     int dialect ATTRIBUTE_UNUSED,
-	     int *invalid)
-{
-  if (((insn >> 21) & 0x1f) != ((insn >> 11) & 0x1f))
-    *invalid = 1;
-  return 0;
-}
-
-/* The SH field in an MD form instruction.  This is split.  */
-
-static unsigned long
-insert_sh6 (unsigned long insn,
-	    long value,
-	    int dialect ATTRIBUTE_UNUSED,
-	    const char **errmsg ATTRIBUTE_UNUSED)
-{
-  return insn | ((value & 0x1f) << 11) | ((value & 0x20) >> 4);
-}
-
-static long
-extract_sh6 (unsigned long insn,
-	     int dialect ATTRIBUTE_UNUSED,
-	     int *invalid ATTRIBUTE_UNUSED)
-{
-  return ((insn >> 11) & 0x1f) | ((insn << 4) & 0x20);
-}
-
-/* The SPR field in an XFX form instruction.  This is flipped--the
-   lower 5 bits are stored in the upper 5 and vice- versa.  */
-
-static unsigned long
-insert_spr (unsigned long insn,
-	    long value,
-	    int dialect ATTRIBUTE_UNUSED,
-	    const char **errmsg ATTRIBUTE_UNUSED)
-{
-  return insn | ((value & 0x1f) << 16) | ((value & 0x3e0) << 6);
-}
-
-static long
-extract_spr (unsigned long insn,
-	     int dialect ATTRIBUTE_UNUSED,
-	     int *invalid ATTRIBUTE_UNUSED)
-{
-  return ((insn >> 16) & 0x1f) | ((insn >> 6) & 0x3e0);
-}
-
-/* Some dialects have 8 SPRG registers instead of the standard 4.  */
-
-static unsigned long
-insert_sprg (unsigned long insn,
-	     long value,
-	     int dialect,
-	     const char **errmsg)
-{
-  /* This check uses PPC_OPCODE_403 because PPC405 is later defined
-     as a synonym.  If ever a 405 specific dialect is added this
-     check should use that instead.  */
-  if (value > 7
-      || (value > 3
-	  && (dialect & (PPC_OPCODE_BOOKE | PPC_OPCODE_403)) == 0))
-    *errmsg = "invalid sprg number";
-
-  /* If this is mfsprg4..7 then use spr 260..263 which can be read in
-     user mode.  Anything else must use spr 272..279.  */
-  if (value <= 3 || (insn & 0x100) != 0)
-    value |= 0x10;
-
-  return insn | ((value & 0x17) << 16);
-}
-
-static long
-extract_sprg (unsigned long insn,
-	      int dialect,
-	      int *invalid)
-{
-  unsigned long val = (insn >> 16) & 0x1f;
-
-  /* mfsprg can use 260..263 and 272..279.  mtsprg only uses spr 272..279
-     If not BOOKE or 405, then both use only 272..275.  */
-  if (val <= 3
-      || (val < 0x10 && (insn & 0x100) != 0)
-      || (val - 0x10 > 3
-	  && (dialect & (PPC_OPCODE_BOOKE | PPC_OPCODE_403)) == 0))
-    *invalid = 1;
-  return val & 7;
-}
-
-/* The TBR field in an XFX instruction.  This is just like SPR, but it
-   is optional.  When TBR is omitted, it must be inserted as 268 (the
-   magic number of the TB register).  These functions treat 0
-   (indicating an omitted optional operand) as 268.  This means that
-   ``mftb 4,0'' is not handled correctly.  This does not matter very
-   much, since the architecture manual does not define mftb as
-   accepting any values other than 268 or 269.  */
-
-#define TB (268)
-
-static unsigned long
-insert_tbr (unsigned long insn,
-	    long value,
-	    int dialect ATTRIBUTE_UNUSED,
-	    const char **errmsg ATTRIBUTE_UNUSED)
-{
-  if (value == 0)
-    value = TB;
-  return insn | ((value & 0x1f) << 16) | ((value & 0x3e0) << 6);
-}
-
-static long
-extract_tbr (unsigned long insn,
-	     int dialect ATTRIBUTE_UNUSED,
-	     int *invalid ATTRIBUTE_UNUSED)
-{
-  long ret;
-
-  ret = ((insn >> 16) & 0x1f) | ((insn >> 6) & 0x3e0);
-  if (ret == TB)
-    ret = 0;
-  return ret;
-}
-
-/* Macros used to form opcodes.  */
-
-/* The main opcode.  */
-#define OP(x) ((((unsigned long)(x)) & 0x3f) << 26)
-#define OP_MASK OP (0x3f)
-
-/* The main opcode combined with a trap code in the TO field of a D
-   form instruction.  Used for extended mnemonics for the trap
-   instructions.  */
-#define OPTO(x,to) (OP (x) | ((((unsigned long)(to)) & 0x1f) << 21))
-#define OPTO_MASK (OP_MASK | TO_MASK)
-
-/* The main opcode combined with a comparison size bit in the L field
-   of a D form or X form instruction.  Used for extended mnemonics for
-   the comparison instructions.  */
-#define OPL(x,l) (OP (x) | ((((unsigned long)(l)) & 1) << 21))
-#define OPL_MASK OPL (0x3f,1)
-
-/* An A form instruction.  */
-#define A(op, xop, rc) (OP (op) | ((((unsigned long)(xop)) & 0x1f) << 1) | (((unsigned long)(rc)) & 1))
-#define A_MASK A (0x3f, 0x1f, 1)
-
-/* An A_MASK with the FRB field fixed.  */
-#define AFRB_MASK (A_MASK | FRB_MASK)
-
-/* An A_MASK with the FRC field fixed.  */
-#define AFRC_MASK (A_MASK | FRC_MASK)
-
-/* An A_MASK with the FRA and FRC fields fixed.  */
-#define AFRAFRC_MASK (A_MASK | FRA_MASK | FRC_MASK)
-
-/* An AFRAFRC_MASK, but with L bit clear.  */
-#define AFRALFRC_MASK (AFRAFRC_MASK & ~((unsigned long) 1 << 16))
-
-/* A B form instruction.  */
-#define B(op, aa, lk) (OP (op) | ((((unsigned long)(aa)) & 1) << 1) | ((lk) & 1))
-#define B_MASK B (0x3f, 1, 1)
-
-/* A B form instruction setting the BO field.  */
-#define BBO(op, bo, aa, lk) (B ((op), (aa), (lk)) | ((((unsigned long)(bo)) & 0x1f) << 21))
-#define BBO_MASK BBO (0x3f, 0x1f, 1, 1)
-
-/* A BBO_MASK with the y bit of the BO field removed.  This permits
-   matching a conditional branch regardless of the setting of the y
-   bit.  Similarly for the 'at' bits used for power4 branch hints.  */
-#define Y_MASK   (((unsigned long) 1) << 21)
-#define AT1_MASK (((unsigned long) 3) << 21)
-#define AT2_MASK (((unsigned long) 9) << 21)
-#define BBOY_MASK  (BBO_MASK &~ Y_MASK)
-#define BBOAT_MASK (BBO_MASK &~ AT1_MASK)
-
-/* A B form instruction setting the BO field and the condition bits of
-   the BI field.  */
-#define BBOCB(op, bo, cb, aa, lk) \
-  (BBO ((op), (bo), (aa), (lk)) | ((((unsigned long)(cb)) & 0x3) << 16))
-#define BBOCB_MASK BBOCB (0x3f, 0x1f, 0x3, 1, 1)
-
-/* A BBOCB_MASK with the y bit of the BO field removed.  */
-#define BBOYCB_MASK (BBOCB_MASK &~ Y_MASK)
-#define BBOATCB_MASK (BBOCB_MASK &~ AT1_MASK)
-#define BBOAT2CB_MASK (BBOCB_MASK &~ AT2_MASK)
-
-/* A BBOYCB_MASK in which the BI field is fixed.  */
-#define BBOYBI_MASK (BBOYCB_MASK | BI_MASK)
-#define BBOATBI_MASK (BBOAT2CB_MASK | BI_MASK)
-
-/* A Context form instruction.  */
-#define CTX(op, xop)   (OP (op) | (((unsigned long)(xop)) & 0x7))
-#define CTX_MASK CTX(0x3f, 0x7)
-
-/* A User Context form instruction.  */
-#define UCTX(op, xop)  (OP (op) | (((unsigned long)(xop)) & 0x1f))
-#define UCTX_MASK UCTX(0x3f, 0x1f)
-
-/* The main opcode mask with the RA field clear.  */
-#define DRA_MASK (OP_MASK | RA_MASK)
-
-/* A DS form instruction.  */
-#define DSO(op, xop) (OP (op) | ((xop) & 0x3))
-#define DS_MASK DSO (0x3f, 3)
-
-/* A DE form instruction.  */
-#define DEO(op, xop) (OP (op) | ((xop) & 0xf))
-#define DE_MASK DEO (0x3e, 0xf)
-
-/* An EVSEL form instruction.  */
-#define EVSEL(op, xop) (OP (op) | (((unsigned long)(xop)) & 0xff) << 3)
-#define EVSEL_MASK EVSEL(0x3f, 0xff)
-
-/* An M form instruction.  */
-#define M(op, rc) (OP (op) | ((rc) & 1))
-#define M_MASK M (0x3f, 1)
-
-/* An M form instruction with the ME field specified.  */
-#define MME(op, me, rc) (M ((op), (rc)) | ((((unsigned long)(me)) & 0x1f) << 1))
-
-/* An M_MASK with the MB and ME fields fixed.  */
-#define MMBME_MASK (M_MASK | MB_MASK | ME_MASK)
-
-/* An M_MASK with the SH and ME fields fixed.  */
-#define MSHME_MASK (M_MASK | SH_MASK | ME_MASK)
-
-/* An MD form instruction.  */
-#define MD(op, xop, rc) (OP (op) | ((((unsigned long)(xop)) & 0x7) << 2) | ((rc) & 1))
-#define MD_MASK MD (0x3f, 0x7, 1)
-
-/* An MD_MASK with the MB field fixed.  */
-#define MDMB_MASK (MD_MASK | MB6_MASK)
-
-/* An MD_MASK with the SH field fixed.  */
-#define MDSH_MASK (MD_MASK | SH6_MASK)
-
-/* An MDS form instruction.  */
-#define MDS(op, xop, rc) (OP (op) | ((((unsigned long)(xop)) & 0xf) << 1) | ((rc) & 1))
-#define MDS_MASK MDS (0x3f, 0xf, 1)
-
-/* An MDS_MASK with the MB field fixed.  */
-#define MDSMB_MASK (MDS_MASK | MB6_MASK)
-
-/* An SC form instruction.  */
-#define SC(op, sa, lk) (OP (op) | ((((unsigned long)(sa)) & 1) << 1) | ((lk) & 1))
-#define SC_MASK (OP_MASK | (((unsigned long)0x3ff) << 16) | (((unsigned long)1) << 1) | 1)
-
-/* A VX form instruction.  */
-#define VX(op, xop) (OP (op) | (((unsigned long)(xop)) & 0x7ff))
-
-/* The mask for an VX form instruction.  */
-#define VX_MASK	VX(0x3f, 0x7ff)
-
-/* A VA form instruction.  */
-#define VXA(op, xop) (OP (op) | (((unsigned long)(xop)) & 0x03f))
-
-/* The mask for a VA form instruction.  */
-#define VXA_MASK VXA(0x3f, 0x3f)
-
-/* A VXR form instruction.  */
-#define VXR(op, xop, rc) (OP (op) | (((rc) & 1) << 10) | (((unsigned long)(xop)) & 0x3ff))
-
-/* The mask for a VXR form instruction.  */
-#define VXR_MASK VXR(0x3f, 0x3ff, 1)
-
-/* An X form instruction.  */
-#define X(op, xop) (OP (op) | ((((unsigned long)(xop)) & 0x3ff) << 1))
-
-/* A Z form instruction.  */
-#define Z(op, xop) (OP (op) | ((((unsigned long)(xop)) & 0x1ff) << 1))
-
-/* An X form instruction with the RC bit specified.  */
-#define XRC(op, xop, rc) (X ((op), (xop)) | ((rc) & 1))
-
-/* A Z form instruction with the RC bit specified.  */
-#define ZRC(op, xop, rc) (Z ((op), (xop)) | ((rc) & 1))
-
-/* The mask for an X form instruction.  */
-#define X_MASK XRC (0x3f, 0x3ff, 1)
-
-/* The mask for a Z form instruction.  */
-#define Z_MASK ZRC (0x3f, 0x1ff, 1)
-#define Z2_MASK ZRC (0x3f, 0xff, 1)
-
-/* An X_MASK with the RA field fixed.  */
-#define XRA_MASK (X_MASK | RA_MASK)
-
-/* An XRA_MASK with the W field clear.  */
-#define XWRA_MASK (XRA_MASK & ~((unsigned long) 1 << 16))
-
-/* An X_MASK with the RB field fixed.  */
-#define XRB_MASK (X_MASK | RB_MASK)
-
-/* An X_MASK with the RT field fixed.  */
-#define XRT_MASK (X_MASK | RT_MASK)
-
-/* An XRT_MASK mask with the L bits clear.  */
-#define XLRT_MASK (XRT_MASK & ~((unsigned long) 0x3 << 21))
-
-/* An X_MASK with the RA and RB fields fixed.  */
-#define XRARB_MASK (X_MASK | RA_MASK | RB_MASK)
-
-/* An X form instruction with the RA field fixed.  */
-#define XRA(op, xop, ra) (X((op), (xop)) | (((ra) << 16) & XRA_MASK))
-
-/* An XRARB_MASK, but with the L bit clear.  */
-#define XRLARB_MASK (XRARB_MASK & ~((unsigned long) 1 << 16))
-
-/* An X_MASK with the RT and RA fields fixed.  */
-#define XRTRA_MASK (X_MASK | RT_MASK | RA_MASK)
-
-/* An XRTRA_MASK, but with L bit clear.  */
-#define XRTLRA_MASK (XRTRA_MASK & ~((unsigned long) 1 << 21))
-
-/* An X form instruction with the L bit specified.  */
-#define XOPL(op, xop, l) (X ((op), (xop)) | ((((unsigned long)(l)) & 1) << 21))
-
-/* The mask for an X form comparison instruction.  */
-#define XCMP_MASK (X_MASK | (((unsigned long)1) << 22))
-
-/* The mask for an X form comparison instruction with the L field
-   fixed.  */
-#define XCMPL_MASK (XCMP_MASK | (((unsigned long)1) << 21))
-
-/* An X form trap instruction with the TO field specified.  */
-#define XTO(op, xop, to) (X ((op), (xop)) | ((((unsigned long)(to)) & 0x1f) << 21))
-#define XTO_MASK (X_MASK | TO_MASK)
-
-/* An X form tlb instruction with the SH field specified.  */
-#define XTLB(op, xop, sh) (X ((op), (xop)) | ((((unsigned long)(sh)) & 0x1f) << 11))
-#define XTLB_MASK (X_MASK | SH_MASK)
-
-/* An X form sync instruction.  */
-#define XSYNC(op, xop, l) (X ((op), (xop)) | ((((unsigned long)(l)) & 3) << 21))
-
-/* An X form sync instruction with everything filled in except the LS field.  */
-#define XSYNC_MASK (0xff9fffff)
-
-/* An X_MASK, but with the EH bit clear.  */
-#define XEH_MASK (X_MASK & ~((unsigned long )1))
-
-/* An X form AltiVec dss instruction.  */
-#define XDSS(op, xop, a) (X ((op), (xop)) | ((((unsigned long)(a)) & 1) << 25))
-#define XDSS_MASK XDSS(0x3f, 0x3ff, 1)
-
-/* An XFL form instruction.  */
-#define XFL(op, xop, rc) (OP (op) | ((((unsigned long)(xop)) & 0x3ff) << 1) | (((unsigned long)(rc)) & 1))
-#define XFL_MASK XFL (0x3f, 0x3ff, 1)
-
-/* An X form isel instruction.  */
-#define XISEL(op, xop)  (OP (op) | ((((unsigned long)(xop)) & 0x1f) << 1))
-#define XISEL_MASK      XISEL(0x3f, 0x1f)
-
-/* An XL form instruction with the LK field set to 0.  */
-#define XL(op, xop) (OP (op) | ((((unsigned long)(xop)) & 0x3ff) << 1))
-
-/* An XL form instruction which uses the LK field.  */
-#define XLLK(op, xop, lk) (XL ((op), (xop)) | ((lk) & 1))
-
-/* The mask for an XL form instruction.  */
-#define XL_MASK XLLK (0x3f, 0x3ff, 1)
-
-/* An XL form instruction which explicitly sets the BO field.  */
-#define XLO(op, bo, xop, lk) \
-  (XLLK ((op), (xop), (lk)) | ((((unsigned long)(bo)) & 0x1f) << 21))
-#define XLO_MASK (XL_MASK | BO_MASK)
-
-/* An XL form instruction which explicitly sets the y bit of the BO
-   field.  */
-#define XLYLK(op, xop, y, lk) (XLLK ((op), (xop), (lk)) | ((((unsigned long)(y)) & 1) << 21))
-#define XLYLK_MASK (XL_MASK | Y_MASK)
-
-/* An XL form instruction which sets the BO field and the condition
-   bits of the BI field.  */
-#define XLOCB(op, bo, cb, xop, lk) \
-  (XLO ((op), (bo), (xop), (lk)) | ((((unsigned long)(cb)) & 3) << 16))
-#define XLOCB_MASK XLOCB (0x3f, 0x1f, 0x3, 0x3ff, 1)
-
-/* An XL_MASK or XLYLK_MASK or XLOCB_MASK with the BB field fixed.  */
-#define XLBB_MASK (XL_MASK | BB_MASK)
-#define XLYBB_MASK (XLYLK_MASK | BB_MASK)
-#define XLBOCBBB_MASK (XLOCB_MASK | BB_MASK)
-
-/* A mask for branch instructions using the BH field.  */
-#define XLBH_MASK (XL_MASK | (0x1c << 11))
-
-/* An XL_MASK with the BO and BB fields fixed.  */
-#define XLBOBB_MASK (XL_MASK | BO_MASK | BB_MASK)
-
-/* An XL_MASK with the BO, BI and BB fields fixed.  */
-#define XLBOBIBB_MASK (XL_MASK | BO_MASK | BI_MASK | BB_MASK)
-
-/* An XO form instruction.  */
-#define XO(op, xop, oe, rc) \
-  (OP (op) | ((((unsigned long)(xop)) & 0x1ff) << 1) | ((((unsigned long)(oe)) & 1) << 10) | (((unsigned long)(rc)) & 1))
-#define XO_MASK XO (0x3f, 0x1ff, 1, 1)
-
-/* An XO_MASK with the RB field fixed.  */
-#define XORB_MASK (XO_MASK | RB_MASK)
-
-/* An XS form instruction.  */
-#define XS(op, xop, rc) (OP (op) | ((((unsigned long)(xop)) & 0x1ff) << 2) | (((unsigned long)(rc)) & 1))
-#define XS_MASK XS (0x3f, 0x1ff, 1)
-
-/* A mask for the FXM version of an XFX form instruction.  */
-#define XFXFXM_MASK (X_MASK | (1 << 11) | (1 << 20))
-
-/* An XFX form instruction with the FXM field filled in.  */
-#define XFXM(op, xop, fxm, p4) \
-  (X ((op), (xop)) | ((((unsigned long)(fxm)) & 0xff) << 12) \
-   | ((unsigned long)(p4) << 20))
-
-/* An XFX form instruction with the SPR field filled in.  */
-#define XSPR(op, xop, spr) \
-  (X ((op), (xop)) | ((((unsigned long)(spr)) & 0x1f) << 16) | ((((unsigned long)(spr)) & 0x3e0) << 6))
-#define XSPR_MASK (X_MASK | SPR_MASK)
-
-/* An XFX form instruction with the SPR field filled in except for the
-   SPRBAT field.  */
-#define XSPRBAT_MASK (XSPR_MASK &~ SPRBAT_MASK)
-
-/* An XFX form instruction with the SPR field filled in except for the
-   SPRG field.  */
-#define XSPRG_MASK (XSPR_MASK & ~(0x1f << 16))
-
-/* An X form instruction with everything filled in except the E field.  */
-#define XE_MASK (0xffff7fff)
-
-/* An X form user context instruction.  */
-#define XUC(op, xop)  (OP (op) | (((unsigned long)(xop)) & 0x1f))
-#define XUC_MASK      XUC(0x3f, 0x1f)
-
-/* The BO encodings used in extended conditional branch mnemonics.  */
-#define BODNZF	(0x0)
-#define BODNZFP	(0x1)
-#define BODZF	(0x2)
-#define BODZFP	(0x3)
-#define BODNZT	(0x8)
-#define BODNZTP	(0x9)
-#define BODZT	(0xa)
-#define BODZTP	(0xb)
-
-#define BOF	(0x4)
-#define BOFP	(0x5)
-#define BOFM4	(0x6)
-#define BOFP4	(0x7)
-#define BOT	(0xc)
-#define BOTP	(0xd)
-#define BOTM4	(0xe)
-#define BOTP4	(0xf)
-
-#define BODNZ	(0x10)
-#define BODNZP	(0x11)
-#define BODZ	(0x12)
-#define BODZP	(0x13)
-#define BODNZM4 (0x18)
-#define BODNZP4 (0x19)
-#define BODZM4	(0x1a)
-#define BODZP4	(0x1b)
-
-#define BOU	(0x14)
-
-/* The BI condition bit encodings used in extended conditional branch
-   mnemonics.  */
-#define CBLT	(0)
-#define CBGT	(1)
-#define CBEQ	(2)
-#define CBSO	(3)
-
-/* The TO encodings used in extended trap mnemonics.  */
-#define TOLGT	(0x1)
-#define TOLLT	(0x2)
-#define TOEQ	(0x4)
-#define TOLGE	(0x5)
-#define TOLNL	(0x5)
-#define TOLLE	(0x6)
-#define TOLNG	(0x6)
-#define TOGT	(0x8)
-#define TOGE	(0xc)
-#define TONL	(0xc)
-#define TOLT	(0x10)
-#define TOLE	(0x14)
-#define TONG	(0x14)
-#define TONE	(0x18)
-#define TOU	(0x1f)
-
-/* Smaller names for the flags so each entry in the opcodes table will
-   fit on a single line.  */
-#undef	PPC
-#define PPC     PPC_OPCODE_PPC
-#define PPCCOM	PPC_OPCODE_PPC | PPC_OPCODE_COMMON
-#define NOPOWER4 PPC_OPCODE_NOPOWER4 | PPCCOM
-#define POWER4	PPC_OPCODE_POWER4
-#define POWER5	PPC_OPCODE_POWER5
-#define POWER6	PPC_OPCODE_POWER6
-/* Documentation purposes only; we don't actually check the isa for disas.  */
-#define POWER7  PPC_OPCODE_POWER6
-#define POWER9  PPC_OPCODE_POWER6
-#define CELL	PPC_OPCODE_CELL
-#define PPC32   PPC_OPCODE_32 | PPC_OPCODE_PPC
-#define PPC64   PPC_OPCODE_64 | PPC_OPCODE_PPC
-#define PPC403	PPC_OPCODE_403
-#define PPC405	PPC403
-#define PPC440	PPC_OPCODE_440
-#define PPC750	PPC
-#define PPC860	PPC
-#define PPCVEC	PPC_OPCODE_ALTIVEC
-#define	POWER   PPC_OPCODE_POWER
-#define	POWER2	PPC_OPCODE_POWER | PPC_OPCODE_POWER2
-#define PPCPWR2	PPC_OPCODE_PPC | PPC_OPCODE_POWER | PPC_OPCODE_POWER2
-#define	POWER32	PPC_OPCODE_POWER | PPC_OPCODE_32
-#define	COM     PPC_OPCODE_POWER | PPC_OPCODE_PPC | PPC_OPCODE_COMMON
-#define	COM32   PPC_OPCODE_POWER | PPC_OPCODE_PPC | PPC_OPCODE_COMMON | PPC_OPCODE_32
-#define	M601    PPC_OPCODE_POWER | PPC_OPCODE_601
-#define PWRCOM	PPC_OPCODE_POWER | PPC_OPCODE_601 | PPC_OPCODE_COMMON
-#define	MFDEC1	PPC_OPCODE_POWER
-#define	MFDEC2	PPC_OPCODE_PPC | PPC_OPCODE_601 | PPC_OPCODE_BOOKE
-#define BOOKE	PPC_OPCODE_BOOKE
-#define BOOKE64	PPC_OPCODE_BOOKE64
-#define CLASSIC	PPC_OPCODE_CLASSIC
-#define PPCE300 PPC_OPCODE_E300
-#define PPCSPE	PPC_OPCODE_SPE
-#define PPCISEL	PPC_OPCODE_ISEL
-#define PPCEFS	PPC_OPCODE_EFS
-#define PPCBRLK	PPC_OPCODE_BRLOCK
-#define PPCPMR	PPC_OPCODE_PMR
-#define PPCCHLK	PPC_OPCODE_CACHELCK
-#define PPCCHLK64	PPC_OPCODE_CACHELCK | PPC_OPCODE_BOOKE64
-#define PPCRFMCI	PPC_OPCODE_RFMCI
-
-/* The opcode table.
-
-   The format of the opcode table is:
-
-   NAME	     OPCODE	MASK		FLAGS		{ OPERANDS }
-
-   NAME is the name of the instruction.
-   OPCODE is the instruction opcode.
-   MASK is the opcode mask; this is used to tell the disassembler
-     which bits in the actual opcode must match OPCODE.
-   FLAGS are flags indicated what processors support the instruction.
-   OPERANDS is the list of operands.
-
-   The disassembler reads the table in order and prints the first
-   instruction which matches, so this table is sorted to put more
-   specific instructions before more general instructions.  It is also
-   sorted by major opcode.  */
-
-const struct powerpc_opcode powerpc_opcodes[] = {
-{ "attn",    X(0,256), X_MASK,		POWER4,		{ 0 } },
-{ "tdlgti",  OPTO(2,TOLGT), OPTO_MASK,	PPC64,		{ RA, SI } },
-{ "tdllti",  OPTO(2,TOLLT), OPTO_MASK,	PPC64,		{ RA, SI } },
-{ "tdeqi",   OPTO(2,TOEQ), OPTO_MASK,	PPC64,		{ RA, SI } },
-{ "tdlgei",  OPTO(2,TOLGE), OPTO_MASK,	PPC64,		{ RA, SI } },
-{ "tdlnli",  OPTO(2,TOLNL), OPTO_MASK,	PPC64,		{ RA, SI } },
-{ "tdllei",  OPTO(2,TOLLE), OPTO_MASK,	PPC64,		{ RA, SI } },
-{ "tdlngi",  OPTO(2,TOLNG), OPTO_MASK,	PPC64,		{ RA, SI } },
-{ "tdgti",   OPTO(2,TOGT), OPTO_MASK,	PPC64,		{ RA, SI } },
-{ "tdgei",   OPTO(2,TOGE), OPTO_MASK,	PPC64,		{ RA, SI } },
-{ "tdnli",   OPTO(2,TONL), OPTO_MASK,	PPC64,		{ RA, SI } },
-{ "tdlti",   OPTO(2,TOLT), OPTO_MASK,	PPC64,		{ RA, SI } },
-{ "tdlei",   OPTO(2,TOLE), OPTO_MASK,	PPC64,		{ RA, SI } },
-{ "tdngi",   OPTO(2,TONG), OPTO_MASK,	PPC64,		{ RA, SI } },
-{ "tdnei",   OPTO(2,TONE), OPTO_MASK,	PPC64,		{ RA, SI } },
-{ "tdi",     OP(2),	OP_MASK,	PPC64,		{ TO, RA, SI } },
-
-{ "twlgti",  OPTO(3,TOLGT), OPTO_MASK,	PPCCOM,		{ RA, SI } },
-{ "tlgti",   OPTO(3,TOLGT), OPTO_MASK,	PWRCOM,		{ RA, SI } },
-{ "twllti",  OPTO(3,TOLLT), OPTO_MASK,	PPCCOM,		{ RA, SI } },
-{ "tllti",   OPTO(3,TOLLT), OPTO_MASK,	PWRCOM,		{ RA, SI } },
-{ "tweqi",   OPTO(3,TOEQ), OPTO_MASK,	PPCCOM,		{ RA, SI } },
-{ "teqi",    OPTO(3,TOEQ), OPTO_MASK,	PWRCOM,		{ RA, SI } },
-{ "twlgei",  OPTO(3,TOLGE), OPTO_MASK,	PPCCOM,		{ RA, SI } },
-{ "tlgei",   OPTO(3,TOLGE), OPTO_MASK,	PWRCOM,		{ RA, SI } },
-{ "twlnli",  OPTO(3,TOLNL), OPTO_MASK,	PPCCOM,		{ RA, SI } },
-{ "tlnli",   OPTO(3,TOLNL), OPTO_MASK,	PWRCOM,		{ RA, SI } },
-{ "twllei",  OPTO(3,TOLLE), OPTO_MASK,	PPCCOM,		{ RA, SI } },
-{ "tllei",   OPTO(3,TOLLE), OPTO_MASK,	PWRCOM,		{ RA, SI } },
-{ "twlngi",  OPTO(3,TOLNG), OPTO_MASK,	PPCCOM,		{ RA, SI } },
-{ "tlngi",   OPTO(3,TOLNG), OPTO_MASK,	PWRCOM,		{ RA, SI } },
-{ "twgti",   OPTO(3,TOGT), OPTO_MASK,	PPCCOM,		{ RA, SI } },
-{ "tgti",    OPTO(3,TOGT), OPTO_MASK,	PWRCOM,		{ RA, SI } },
-{ "twgei",   OPTO(3,TOGE), OPTO_MASK,	PPCCOM,		{ RA, SI } },
-{ "tgei",    OPTO(3,TOGE), OPTO_MASK,	PWRCOM,		{ RA, SI } },
-{ "twnli",   OPTO(3,TONL), OPTO_MASK,	PPCCOM,		{ RA, SI } },
-{ "tnli",    OPTO(3,TONL), OPTO_MASK,	PWRCOM,		{ RA, SI } },
-{ "twlti",   OPTO(3,TOLT), OPTO_MASK,	PPCCOM,		{ RA, SI } },
-{ "tlti",    OPTO(3,TOLT), OPTO_MASK,	PWRCOM,		{ RA, SI } },
-{ "twlei",   OPTO(3,TOLE), OPTO_MASK,	PPCCOM,		{ RA, SI } },
-{ "tlei",    OPTO(3,TOLE), OPTO_MASK,	PWRCOM,		{ RA, SI } },
-{ "twngi",   OPTO(3,TONG), OPTO_MASK,	PPCCOM,		{ RA, SI } },
-{ "tngi",    OPTO(3,TONG), OPTO_MASK,	PWRCOM,		{ RA, SI } },
-{ "twnei",   OPTO(3,TONE), OPTO_MASK,	PPCCOM,		{ RA, SI } },
-{ "tnei",    OPTO(3,TONE), OPTO_MASK,	PWRCOM,		{ RA, SI } },
-{ "twi",     OP(3),	OP_MASK,	PPCCOM,		{ TO, RA, SI } },
-{ "ti",      OP(3),	OP_MASK,	PWRCOM,		{ TO, RA, SI } },
-
-{ "macchw",	XO(4,172,0,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "macchw.",	XO(4,172,0,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "macchwo",	XO(4,172,1,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "macchwo.",	XO(4,172,1,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "macchws",	XO(4,236,0,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "macchws.",	XO(4,236,0,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "macchwso",	XO(4,236,1,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "macchwso.",	XO(4,236,1,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "macchwsu",	XO(4,204,0,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "macchwsu.",	XO(4,204,0,1), XO_MASK, PPC405|PPC440,	{ RT, RA, RB } },
-{ "macchwsuo",	XO(4,204,1,0), XO_MASK, PPC405|PPC440,	{ RT, RA, RB } },
-{ "macchwsuo.",	XO(4,204,1,1), XO_MASK, PPC405|PPC440,	{ RT, RA, RB } },
-{ "macchwu",	XO(4,140,0,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "macchwu.",	XO(4,140,0,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "macchwuo",	XO(4,140,1,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "macchwuo.",	XO(4,140,1,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "machhw",	XO(4,44,0,0),  XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "machhw.",	XO(4,44,0,1),  XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "machhwo",	XO(4,44,1,0),  XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "machhwo.",	XO(4,44,1,1),  XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "machhws",	XO(4,108,0,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "machhws.",	XO(4,108,0,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "machhwso",	XO(4,108,1,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "machhwso.",	XO(4,108,1,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "machhwsu",	XO(4,76,0,0),  XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "machhwsu.",	XO(4,76,0,1),  XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "machhwsuo",	XO(4,76,1,0),  XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "machhwsuo.",	XO(4,76,1,1),  XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "machhwu",	XO(4,12,0,0),  XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "machhwu.",	XO(4,12,0,1),  XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "machhwuo",	XO(4,12,1,0),  XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "machhwuo.",	XO(4,12,1,1),  XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "maclhw",	XO(4,428,0,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "maclhw.",	XO(4,428,0,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "maclhwo",	XO(4,428,1,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "maclhwo.",	XO(4,428,1,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "maclhws",	XO(4,492,0,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "maclhws.",	XO(4,492,0,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "maclhwso",	XO(4,492,1,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "maclhwso.",	XO(4,492,1,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "maclhwsu",	XO(4,460,0,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "maclhwsu.",	XO(4,460,0,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "maclhwsuo",	XO(4,460,1,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "maclhwsuo.",	XO(4,460,1,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "maclhwu",	XO(4,396,0,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "maclhwu.",	XO(4,396,0,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "maclhwuo",	XO(4,396,1,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "maclhwuo.",	XO(4,396,1,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "mulchw",	XRC(4,168,0),  X_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "mulchw.",	XRC(4,168,1),  X_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "mulchwu",	XRC(4,136,0),  X_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "mulchwu.",	XRC(4,136,1),  X_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "mulhhw",	XRC(4,40,0),   X_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "mulhhw.",	XRC(4,40,1),   X_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "mulhhwu",	XRC(4,8,0),    X_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "mulhhwu.",	XRC(4,8,1),    X_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "mullhw",	XRC(4,424,0),  X_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "mullhw.",	XRC(4,424,1),  X_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "mullhwu",	XRC(4,392,0),  X_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "mullhwu.",	XRC(4,392,1),  X_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmacchw",	XO(4,174,0,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmacchw.",	XO(4,174,0,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmacchwo",	XO(4,174,1,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmacchwo.",	XO(4,174,1,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmacchws",	XO(4,238,0,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmacchws.",	XO(4,238,0,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmacchwso",	XO(4,238,1,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmacchwso.",	XO(4,238,1,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmachhw",	XO(4,46,0,0),  XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmachhw.",	XO(4,46,0,1),  XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmachhwo",	XO(4,46,1,0),  XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmachhwo.",	XO(4,46,1,1),  XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmachhws",	XO(4,110,0,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmachhws.",	XO(4,110,0,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmachhwso",	XO(4,110,1,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmachhwso.",	XO(4,110,1,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmaclhw",	XO(4,430,0,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmaclhw.",	XO(4,430,0,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmaclhwo",	XO(4,430,1,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmaclhwo.",	XO(4,430,1,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmaclhws",	XO(4,494,0,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmaclhws.",	XO(4,494,0,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmaclhwso",	XO(4,494,1,0), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "nmaclhwso.",	XO(4,494,1,1), XO_MASK,	PPC405|PPC440,	{ RT, RA, RB } },
-{ "mfvscr",  VX(4, 1540), VX_MASK,	PPCVEC,		{ VD } },
-{ "mtvscr",  VX(4, 1604), VX_MASK,	PPCVEC,		{ VB } },
-
-  /* Double-precision opcodes.  */
-  /* Some of these conflict with AltiVec, so move them before, since
-     PPCVEC includes the PPC_OPCODE_PPC set.  */
-{ "efscfd",   VX(4, 719), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efdabs",   VX(4, 740), VX_MASK,	PPCEFS,		{ RS, RA } },
-{ "efdnabs",  VX(4, 741), VX_MASK,	PPCEFS,		{ RS, RA } },
-{ "efdneg",   VX(4, 742), VX_MASK,	PPCEFS,		{ RS, RA } },
-{ "efdadd",   VX(4, 736), VX_MASK,	PPCEFS,		{ RS, RA, RB } },
-{ "efdsub",   VX(4, 737), VX_MASK,	PPCEFS,		{ RS, RA, RB } },
-{ "efdmul",   VX(4, 744), VX_MASK,	PPCEFS,		{ RS, RA, RB } },
-{ "efddiv",   VX(4, 745), VX_MASK,	PPCEFS,		{ RS, RA, RB } },
-{ "efdcmpgt", VX(4, 748), VX_MASK,	PPCEFS,		{ CRFD, RA, RB } },
-{ "efdcmplt", VX(4, 749), VX_MASK,	PPCEFS,		{ CRFD, RA, RB } },
-{ "efdcmpeq", VX(4, 750), VX_MASK,	PPCEFS,		{ CRFD, RA, RB } },
-{ "efdtstgt", VX(4, 764), VX_MASK,	PPCEFS,		{ CRFD, RA, RB } },
-{ "efdtstlt", VX(4, 765), VX_MASK,	PPCEFS,		{ CRFD, RA, RB } },
-{ "efdtsteq", VX(4, 766), VX_MASK,	PPCEFS,		{ CRFD, RA, RB } },
-{ "efdcfsi",  VX(4, 753), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efdcfsid", VX(4, 739), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efdcfui",  VX(4, 752), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efdcfuid", VX(4, 738), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efdcfsf",  VX(4, 755), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efdcfuf",  VX(4, 754), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efdctsi",  VX(4, 757), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efdctsidz",VX(4, 747), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efdctsiz", VX(4, 762), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efdctui",  VX(4, 756), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efdctuidz",VX(4, 746), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efdctuiz", VX(4, 760), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efdctsf",  VX(4, 759), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efdctuf",  VX(4, 758), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efdcfs",   VX(4, 751), VX_MASK,	PPCEFS,		{ RS, RB } },
-  /* End of double-precision opcodes.  */
-
-{ "vaddcuw", VX(4,  384), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vaddfp",  VX(4,   10), VX_MASK, 	PPCVEC,		{ VD, VA, VB } },
-{ "vaddsbs", VX(4,  768), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vaddshs", VX(4,  832), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vaddsws", VX(4,  896), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vaddubm", VX(4,    0), VX_MASK, 	PPCVEC,		{ VD, VA, VB } },
-{ "vaddubs", VX(4,  512), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vadduhm", VX(4,   64), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vadduhs", VX(4,  576), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vadduwm", VX(4,  128), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vadduws", VX(4,  640), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vand",    VX(4, 1028), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vandc",   VX(4, 1092), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vavgsb",  VX(4, 1282), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vavgsh",  VX(4, 1346), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vavgsw",  VX(4, 1410), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vavgub",  VX(4, 1026), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vavguh",  VX(4, 1090), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vavguw",  VX(4, 1154), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vcfsx",   VX(4,  842), VX_MASK,	PPCVEC,		{ VD, VB, UIMM } },
-{ "vcfux",   VX(4,  778), VX_MASK,	PPCVEC,		{ VD, VB, UIMM } },
-{ "vcmpbfp",   VXR(4, 966, 0), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpbfp.",  VXR(4, 966, 1), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpeqfp",  VXR(4, 198, 0), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpeqfp.", VXR(4, 198, 1), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpequb",  VXR(4,   6, 0), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpequb.", VXR(4,   6, 1), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpequh",  VXR(4,  70, 0), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpequh.", VXR(4,  70, 1), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpequw",  VXR(4, 134, 0), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpequw.", VXR(4, 134, 1), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpgefp",  VXR(4, 454, 0), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpgefp.", VXR(4, 454, 1), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpgtfp",  VXR(4, 710, 0), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpgtfp.", VXR(4, 710, 1), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpgtsb",  VXR(4, 774, 0), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpgtsb.", VXR(4, 774, 1), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpgtsh",  VXR(4, 838, 0), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpgtsh.", VXR(4, 838, 1), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpgtsw",  VXR(4, 902, 0), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpgtsw.", VXR(4, 902, 1), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpgtub",  VXR(4, 518, 0), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpgtub.", VXR(4, 518, 1), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpgtuh",  VXR(4, 582, 0), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpgtuh.", VXR(4, 582, 1), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpgtuw",  VXR(4, 646, 0), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vcmpgtuw.", VXR(4, 646, 1), VXR_MASK, PPCVEC,	{ VD, VA, VB } },
-{ "vctsxs",    VX(4,  970), VX_MASK,	PPCVEC,		{ VD, VB, UIMM } },
-{ "vctuxs",    VX(4,  906), VX_MASK,	PPCVEC,		{ VD, VB, UIMM } },
-{ "vexptefp",  VX(4,  394), VX_MASK,	PPCVEC,		{ VD, VB } },
-{ "vlogefp",   VX(4,  458), VX_MASK,	PPCVEC,		{ VD, VB } },
-{ "vmaddfp",   VXA(4,  46), VXA_MASK,	PPCVEC,		{ VD, VA, VC, VB } },
-{ "vmaxfp",    VX(4, 1034), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmaxsb",    VX(4,  258), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmaxsh",    VX(4,  322), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmaxsw",    VX(4,  386), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmaxub",    VX(4,    2), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmaxuh",    VX(4,   66), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmaxuw",    VX(4,  130), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmhaddshs", VXA(4,  32), VXA_MASK,	PPCVEC,		{ VD, VA, VB, VC } },
-{ "vmhraddshs", VXA(4, 33), VXA_MASK,	PPCVEC,		{ VD, VA, VB, VC } },
-{ "vminfp",    VX(4, 1098), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vminsb",    VX(4,  770), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vminsh",    VX(4,  834), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vminsw",    VX(4,  898), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vminub",    VX(4,  514), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vminuh",    VX(4,  578), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vminuw",    VX(4,  642), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmladduhm", VXA(4,  34), VXA_MASK,	PPCVEC,		{ VD, VA, VB, VC } },
-{ "vmrghb",    VX(4,   12), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmrghh",    VX(4,   76), VX_MASK,    PPCVEC,		{ VD, VA, VB } },
-{ "vmrghw",    VX(4,  140), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmrglb",    VX(4,  268), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmrglh",    VX(4,  332), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmrglw",    VX(4,  396), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmsummbm",  VXA(4,  37), VXA_MASK,	PPCVEC,		{ VD, VA, VB, VC } },
-{ "vmsumshm",  VXA(4,  40), VXA_MASK,	PPCVEC,		{ VD, VA, VB, VC } },
-{ "vmsumshs",  VXA(4,  41), VXA_MASK,	PPCVEC,		{ VD, VA, VB, VC } },
-{ "vmsumubm",  VXA(4,  36), VXA_MASK,   PPCVEC,		{ VD, VA, VB, VC } },
-{ "vmsumuhm",  VXA(4,  38), VXA_MASK,   PPCVEC,		{ VD, VA, VB, VC } },
-{ "vmsumuhs",  VXA(4,  39), VXA_MASK,   PPCVEC,		{ VD, VA, VB, VC } },
-{ "vmulesb",   VX(4,  776), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmulesh",   VX(4,  840), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmuleub",   VX(4,  520), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmuleuh",   VX(4,  584), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmulosb",   VX(4,  264), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmulosh",   VX(4,  328), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmuloub",   VX(4,    8), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vmulouh",   VX(4,   72), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vnmsubfp",  VXA(4,  47), VXA_MASK,	PPCVEC,		{ VD, VA, VC, VB } },
-{ "vnor",      VX(4, 1284), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vor",       VX(4, 1156), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vperm",     VXA(4,  43), VXA_MASK,	PPCVEC,		{ VD, VA, VB, VC } },
-{ "vpkpx",     VX(4,  782), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vpkshss",   VX(4,  398), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vpkshus",   VX(4,  270), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vpkswss",   VX(4,  462), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vpkswus",   VX(4,  334), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vpkuhum",   VX(4,   14), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vpkuhus",   VX(4,  142), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vpkuwum",   VX(4,   78), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vpkuwus",   VX(4,  206), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vrefp",     VX(4,  266), VX_MASK,	PPCVEC,		{ VD, VB } },
-{ "vrfim",     VX(4,  714), VX_MASK,	PPCVEC,		{ VD, VB } },
-{ "vrfin",     VX(4,  522), VX_MASK,	PPCVEC,		{ VD, VB } },
-{ "vrfip",     VX(4,  650), VX_MASK,	PPCVEC,		{ VD, VB } },
-{ "vrfiz",     VX(4,  586), VX_MASK,	PPCVEC,		{ VD, VB } },
-{ "vrlb",      VX(4,    4), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vrlh",      VX(4,   68), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vrlw",      VX(4,  132), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vrsqrtefp", VX(4,  330), VX_MASK,	PPCVEC,		{ VD, VB } },
-{ "vrldmi",    VX(4,  197), VX_MASK,    PPCVEC,         { VD, VA, VB } },
-{ "vrldnm",    VX(4,  453), VX_MASK,    PPCVEC,         { VD, VA, VB } },
-{ "vrlwmi",    VX(4,  133), VX_MASK,    PPCVEC,         { VD, VA, VB} },
-{ "vrlwnm",    VX(4,  389), VX_MASK,    PPCVEC,         { VD, VA, VB } },
-{ "vsel",      VXA(4,  42), VXA_MASK,	PPCVEC,		{ VD, VA, VB, VC } },
-{ "vsl",       VX(4,  452), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vslb",      VX(4,  260), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsldoi",    VXA(4,  44), VXA_MASK,	PPCVEC,		{ VD, VA, VB, SHB } },
-{ "vslh",      VX(4,  324), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vslo",      VX(4, 1036), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vslw",      VX(4,  388), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vspltb",    VX(4,  524), VX_MASK,	PPCVEC,		{ VD, VB, UIMM } },
-{ "vsplth",    VX(4,  588), VX_MASK,	PPCVEC,		{ VD, VB, UIMM } },
-{ "vspltisb",  VX(4,  780), VX_MASK,	PPCVEC,		{ VD, SIMM } },
-{ "vspltish",  VX(4,  844), VX_MASK,	PPCVEC,		{ VD, SIMM } },
-{ "vspltisw",  VX(4,  908), VX_MASK,	PPCVEC,		{ VD, SIMM } },
-{ "vspltw",    VX(4,  652), VX_MASK,	PPCVEC,		{ VD, VB, UIMM } },
-{ "vsr",       VX(4,  708), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsrab",     VX(4,  772), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsrah",     VX(4,  836), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsraw",     VX(4,  900), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsrb",      VX(4,  516), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsrh",      VX(4,  580), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsro",      VX(4, 1100), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsrw",      VX(4,  644), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsubcuw",   VX(4, 1408), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsubfp",    VX(4,   74), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsubsbs",   VX(4, 1792), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsubshs",   VX(4, 1856), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsubsws",   VX(4, 1920), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsububm",   VX(4, 1024), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsububs",   VX(4, 1536), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsubuhm",   VX(4, 1088), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsubuhs",   VX(4, 1600), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsubuwm",   VX(4, 1152), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsubuws",   VX(4, 1664), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsumsws",   VX(4, 1928), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsum2sws",  VX(4, 1672), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsum4sbs",  VX(4, 1800), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsum4shs",  VX(4, 1608), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vsum4ubs",  VX(4, 1544), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-{ "vupkhpx",   VX(4,  846), VX_MASK,	PPCVEC,		{ VD, VB } },
-{ "vupkhsb",   VX(4,  526), VX_MASK,	PPCVEC,		{ VD, VB } },
-{ "vupkhsh",   VX(4,  590), VX_MASK,	PPCVEC,		{ VD, VB } },
-{ "vupklpx",   VX(4,  974), VX_MASK,	PPCVEC,		{ VD, VB } },
-{ "vupklsb",   VX(4,  654), VX_MASK,	PPCVEC,		{ VD, VB } },
-{ "vupklsh",   VX(4,  718), VX_MASK,	PPCVEC,		{ VD, VB } },
-{ "vxor",      VX(4, 1220), VX_MASK,	PPCVEC,		{ VD, VA, VB } },
-
-{ "evaddw",    VX(4, 512), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evaddiw",   VX(4, 514), VX_MASK,	PPCSPE,		{ RS, RB, UIMM } },
-{ "evsubfw",   VX(4, 516), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evsubw",    VX(4, 516), VX_MASK,	PPCSPE,		{ RS, RB, RA } },
-{ "evsubifw",  VX(4, 518), VX_MASK,	PPCSPE,		{ RS, UIMM, RB } },
-{ "evsubiw",   VX(4, 518), VX_MASK,	PPCSPE,		{ RS, RB, UIMM } },
-{ "evabs",     VX(4, 520), VX_MASK,	PPCSPE,		{ RS, RA } },
-{ "evneg",     VX(4, 521), VX_MASK,	PPCSPE,		{ RS, RA } },
-{ "evextsb",   VX(4, 522), VX_MASK,	PPCSPE,		{ RS, RA } },
-{ "evextsh",   VX(4, 523), VX_MASK,	PPCSPE,		{ RS, RA } },
-{ "evrndw",    VX(4, 524), VX_MASK,	PPCSPE,		{ RS, RA } },
-{ "evcntlzw",  VX(4, 525), VX_MASK,	PPCSPE,		{ RS, RA } },
-{ "evcntlsw",  VX(4, 526), VX_MASK,	PPCSPE,		{ RS, RA } },
-
-{ "brinc",     VX(4, 527), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "evand",     VX(4, 529), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evandc",    VX(4, 530), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmr",      VX(4, 535), VX_MASK,	PPCSPE,		{ RS, RA, BBA } },
-{ "evor",      VX(4, 535), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evorc",     VX(4, 539), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evxor",     VX(4, 534), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "eveqv",     VX(4, 537), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evnand",    VX(4, 542), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evnot",     VX(4, 536), VX_MASK,	PPCSPE,		{ RS, RA, BBA } },
-{ "evnor",     VX(4, 536), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "evrlw",     VX(4, 552), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evrlwi",    VX(4, 554), VX_MASK,	PPCSPE,		{ RS, RA, EVUIMM } },
-{ "evslw",     VX(4, 548), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evslwi",    VX(4, 550), VX_MASK,	PPCSPE,		{ RS, RA, EVUIMM } },
-{ "evsrws",    VX(4, 545), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evsrwu",    VX(4, 544), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evsrwis",   VX(4, 547), VX_MASK,	PPCSPE,		{ RS, RA, EVUIMM } },
-{ "evsrwiu",   VX(4, 546), VX_MASK,	PPCSPE,		{ RS, RA, EVUIMM } },
-{ "evsplati",  VX(4, 553), VX_MASK,	PPCSPE,		{ RS, SIMM } },
-{ "evsplatfi", VX(4, 555), VX_MASK,	PPCSPE,		{ RS, SIMM } },
-{ "evmergehi", VX(4, 556), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmergelo", VX(4, 557), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmergehilo",VX(4,558), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmergelohi",VX(4,559), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "evcmpgts",  VX(4, 561), VX_MASK,	PPCSPE,		{ CRFD, RA, RB } },
-{ "evcmpgtu",  VX(4, 560), VX_MASK,	PPCSPE,		{ CRFD, RA, RB } },
-{ "evcmplts",  VX(4, 563), VX_MASK,	PPCSPE,		{ CRFD, RA, RB } },
-{ "evcmpltu",  VX(4, 562), VX_MASK,	PPCSPE,		{ CRFD, RA, RB } },
-{ "evcmpeq",   VX(4, 564), VX_MASK,	PPCSPE,		{ CRFD, RA, RB } },
-{ "evsel",     EVSEL(4,79),EVSEL_MASK,	PPCSPE,		{ RS, RA, RB, CRFS } },
-
-{ "evldd",     VX(4, 769), VX_MASK,	PPCSPE,		{ RS, EVUIMM_8, RA } },
-{ "evlddx",    VX(4, 768), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evldw",     VX(4, 771), VX_MASK,	PPCSPE,		{ RS, EVUIMM_8, RA } },
-{ "evldwx",    VX(4, 770), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evldh",     VX(4, 773), VX_MASK,	PPCSPE,		{ RS, EVUIMM_8, RA } },
-{ "evldhx",    VX(4, 772), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evlwhe",    VX(4, 785), VX_MASK,	PPCSPE,		{ RS, EVUIMM_4, RA } },
-{ "evlwhex",   VX(4, 784), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evlwhou",   VX(4, 789), VX_MASK,	PPCSPE,		{ RS, EVUIMM_4, RA } },
-{ "evlwhoux",  VX(4, 788), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evlwhos",   VX(4, 791), VX_MASK,	PPCSPE,		{ RS, EVUIMM_4, RA } },
-{ "evlwhosx",  VX(4, 790), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evlwwsplat",VX(4, 793), VX_MASK,	PPCSPE,		{ RS, EVUIMM_4, RA } },
-{ "evlwwsplatx",VX(4, 792), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evlwhsplat",VX(4, 797), VX_MASK,	PPCSPE,		{ RS, EVUIMM_4, RA } },
-{ "evlwhsplatx",VX(4, 796), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evlhhesplat",VX(4, 777), VX_MASK,	PPCSPE,		{ RS, EVUIMM_2, RA } },
-{ "evlhhesplatx",VX(4, 776), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evlhhousplat",VX(4, 781), VX_MASK,	PPCSPE,		{ RS, EVUIMM_2, RA } },
-{ "evlhhousplatx",VX(4, 780), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evlhhossplat",VX(4, 783), VX_MASK,	PPCSPE,		{ RS, EVUIMM_2, RA } },
-{ "evlhhossplatx",VX(4, 782), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "evstdd",    VX(4, 801), VX_MASK,	PPCSPE,		{ RS, EVUIMM_8, RA } },
-{ "evstddx",   VX(4, 800), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evstdw",    VX(4, 803), VX_MASK,	PPCSPE,		{ RS, EVUIMM_8, RA } },
-{ "evstdwx",   VX(4, 802), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evstdh",    VX(4, 805), VX_MASK,	PPCSPE,		{ RS, EVUIMM_8, RA } },
-{ "evstdhx",   VX(4, 804), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evstwwe",   VX(4, 825), VX_MASK,	PPCSPE,		{ RS, EVUIMM_4, RA } },
-{ "evstwwex",  VX(4, 824), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evstwwo",   VX(4, 829), VX_MASK,	PPCSPE,		{ RS, EVUIMM_4, RA } },
-{ "evstwwox",  VX(4, 828), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evstwhe",   VX(4, 817), VX_MASK,	PPCSPE,		{ RS, EVUIMM_4, RA } },
-{ "evstwhex",  VX(4, 816), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evstwho",   VX(4, 821), VX_MASK,	PPCSPE,		{ RS, EVUIMM_4, RA } },
-{ "evstwhox",  VX(4, 820), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "evfsabs",   VX(4, 644), VX_MASK,	PPCSPE,		{ RS, RA } },
-{ "evfsnabs",  VX(4, 645), VX_MASK,	PPCSPE,		{ RS, RA } },
-{ "evfsneg",   VX(4, 646), VX_MASK,	PPCSPE,		{ RS, RA } },
-{ "evfsadd",   VX(4, 640), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evfssub",   VX(4, 641), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evfsmul",   VX(4, 648), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evfsdiv",   VX(4, 649), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evfscmpgt", VX(4, 652), VX_MASK,	PPCSPE,		{ CRFD, RA, RB } },
-{ "evfscmplt", VX(4, 653), VX_MASK,	PPCSPE,		{ CRFD, RA, RB } },
-{ "evfscmpeq", VX(4, 654), VX_MASK,	PPCSPE,		{ CRFD, RA, RB } },
-{ "evfststgt", VX(4, 668), VX_MASK,	PPCSPE,		{ CRFD, RA, RB } },
-{ "evfststlt", VX(4, 669), VX_MASK,	PPCSPE,		{ CRFD, RA, RB } },
-{ "evfststeq", VX(4, 670), VX_MASK,	PPCSPE,		{ CRFD, RA, RB } },
-{ "evfscfui",  VX(4, 656), VX_MASK,	PPCSPE,		{ RS, RB } },
-{ "evfsctuiz", VX(4, 664), VX_MASK,	PPCSPE,		{ RS, RB } },
-{ "evfscfsi",  VX(4, 657), VX_MASK,	PPCSPE,		{ RS, RB } },
-{ "evfscfuf",  VX(4, 658), VX_MASK,	PPCSPE,		{ RS, RB } },
-{ "evfscfsf",  VX(4, 659), VX_MASK,	PPCSPE,		{ RS, RB } },
-{ "evfsctui",  VX(4, 660), VX_MASK,	PPCSPE,		{ RS, RB } },
-{ "evfsctsi",  VX(4, 661), VX_MASK,	PPCSPE,		{ RS, RB } },
-{ "evfsctsiz", VX(4, 666), VX_MASK,	PPCSPE,		{ RS, RB } },
-{ "evfsctuf",  VX(4, 662), VX_MASK,	PPCSPE,		{ RS, RB } },
-{ "evfsctsf",  VX(4, 663), VX_MASK,	PPCSPE,		{ RS, RB } },
-
-{ "efsabs",   VX(4, 708), VX_MASK,	PPCEFS,		{ RS, RA } },
-{ "efsnabs",  VX(4, 709), VX_MASK,	PPCEFS,		{ RS, RA } },
-{ "efsneg",   VX(4, 710), VX_MASK,	PPCEFS,		{ RS, RA } },
-{ "efsadd",   VX(4, 704), VX_MASK,	PPCEFS,		{ RS, RA, RB } },
-{ "efssub",   VX(4, 705), VX_MASK,	PPCEFS,		{ RS, RA, RB } },
-{ "efsmul",   VX(4, 712), VX_MASK,	PPCEFS,		{ RS, RA, RB } },
-{ "efsdiv",   VX(4, 713), VX_MASK,	PPCEFS,		{ RS, RA, RB } },
-{ "efscmpgt", VX(4, 716), VX_MASK,	PPCEFS,		{ CRFD, RA, RB } },
-{ "efscmplt", VX(4, 717), VX_MASK,	PPCEFS,		{ CRFD, RA, RB } },
-{ "efscmpeq", VX(4, 718), VX_MASK,	PPCEFS,		{ CRFD, RA, RB } },
-{ "efststgt", VX(4, 732), VX_MASK,	PPCEFS,		{ CRFD, RA, RB } },
-{ "efststlt", VX(4, 733), VX_MASK,	PPCEFS,		{ CRFD, RA, RB } },
-{ "efststeq", VX(4, 734), VX_MASK,	PPCEFS,		{ CRFD, RA, RB } },
-{ "efscfui",  VX(4, 720), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efsctuiz", VX(4, 728), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efscfsi",  VX(4, 721), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efscfuf",  VX(4, 722), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efscfsf",  VX(4, 723), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efsctui",  VX(4, 724), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efsctsi",  VX(4, 725), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efsctsiz", VX(4, 730), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efsctuf",  VX(4, 726), VX_MASK,	PPCEFS,		{ RS, RB } },
-{ "efsctsf",  VX(4, 727), VX_MASK,	PPCEFS,		{ RS, RB } },
-
-{ "evmhossf",  VX(4, 1031), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhossfa", VX(4, 1063), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhosmf",  VX(4, 1039), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhosmfa", VX(4, 1071), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhosmi",  VX(4, 1037), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhosmia", VX(4, 1069), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhoumi",  VX(4, 1036), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhoumia", VX(4, 1068), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhessf",  VX(4, 1027), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhessfa", VX(4, 1059), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhesmf",  VX(4, 1035), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhesmfa", VX(4, 1067), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhesmi",  VX(4, 1033), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhesmia", VX(4, 1065), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmheumi",  VX(4, 1032), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmheumia", VX(4, 1064), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "evmhossfaaw",VX(4, 1287), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhossiaaw",VX(4, 1285), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhosmfaaw",VX(4, 1295), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhosmiaaw",VX(4, 1293), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhousiaaw",VX(4, 1284), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhoumiaaw",VX(4, 1292), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhessfaaw",VX(4, 1283), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhessiaaw",VX(4, 1281), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhesmfaaw",VX(4, 1291), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhesmiaaw",VX(4, 1289), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmheusiaaw",VX(4, 1280), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmheumiaaw",VX(4, 1288), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "evmhossfanw",VX(4, 1415), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhossianw",VX(4, 1413), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhosmfanw",VX(4, 1423), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhosmianw",VX(4, 1421), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhousianw",VX(4, 1412), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhoumianw",VX(4, 1420), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhessfanw",VX(4, 1411), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhessianw",VX(4, 1409), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhesmfanw",VX(4, 1419), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhesmianw",VX(4, 1417), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmheusianw",VX(4, 1408), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmheumianw",VX(4, 1416), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "evmhogsmfaa",VX(4, 1327), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhogsmiaa",VX(4, 1325), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhogumiaa",VX(4, 1324), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhegsmfaa",VX(4, 1323), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhegsmiaa",VX(4, 1321), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhegumiaa",VX(4, 1320), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "evmhogsmfan",VX(4, 1455), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhogsmian",VX(4, 1453), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhogumian",VX(4, 1452), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhegsmfan",VX(4, 1451), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhegsmian",VX(4, 1449), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmhegumian",VX(4, 1448), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "evmwhssf",  VX(4, 1095), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwhssfa", VX(4, 1127), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwhsmf",  VX(4, 1103), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwhsmfa", VX(4, 1135), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwhsmi",  VX(4, 1101), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwhsmia", VX(4, 1133), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwhumi",  VX(4, 1100), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwhumia", VX(4, 1132), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "evmwlumi",  VX(4, 1096), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwlumia", VX(4, 1128), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "evmwlssiaaw",VX(4, 1345), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwlsmiaaw",VX(4, 1353), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwlusiaaw",VX(4, 1344), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwlumiaaw",VX(4, 1352), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "evmwlssianw",VX(4, 1473), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwlsmianw",VX(4, 1481), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwlusianw",VX(4, 1472), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwlumianw",VX(4, 1480), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "evmwssf",   VX(4, 1107), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwssfa",  VX(4, 1139), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwsmf",   VX(4, 1115), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwsmfa",  VX(4, 1147), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwsmi",   VX(4, 1113), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwsmia",  VX(4, 1145), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwumi",   VX(4, 1112), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwumia",  VX(4, 1144), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "evmwssfaa", VX(4, 1363), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwsmfaa", VX(4, 1371), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwsmiaa", VX(4, 1369), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwumiaa", VX(4, 1368), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "evmwssfan", VX(4, 1491), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwsmfan", VX(4, 1499), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwsmian", VX(4, 1497), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evmwumian", VX(4, 1496), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "evaddssiaaw",VX(4, 1217), VX_MASK,	PPCSPE,		{ RS, RA } },
-{ "evaddsmiaaw",VX(4, 1225), VX_MASK,	PPCSPE,		{ RS, RA } },
-{ "evaddusiaaw",VX(4, 1216), VX_MASK,	PPCSPE,		{ RS, RA } },
-{ "evaddumiaaw",VX(4, 1224), VX_MASK,	PPCSPE,		{ RS, RA } },
-
-{ "evsubfssiaaw",VX(4, 1219), VX_MASK,	PPCSPE,		{ RS, RA } },
-{ "evsubfsmiaaw",VX(4, 1227), VX_MASK,	PPCSPE,		{ RS, RA } },
-{ "evsubfusiaaw",VX(4, 1218), VX_MASK,	PPCSPE,		{ RS, RA } },
-{ "evsubfumiaaw",VX(4, 1226), VX_MASK,	PPCSPE,		{ RS, RA } },
-
-{ "evmra",    VX(4, 1220), VX_MASK,	PPCSPE,		{ RS, RA } },
-
-{ "evdivws",  VX(4, 1222), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-{ "evdivwu",  VX(4, 1223), VX_MASK,	PPCSPE,		{ RS, RA, RB } },
-
-{ "mulli",   OP(7),	OP_MASK,	PPCCOM,		{ RT, RA, SI } },
-{ "muli",    OP(7),	OP_MASK,	PWRCOM,		{ RT, RA, SI } },
-
-{ "subfic",  OP(8),	OP_MASK,	PPCCOM,		{ RT, RA, SI } },
-{ "sfi",     OP(8),	OP_MASK,	PWRCOM,		{ RT, RA, SI } },
-
-{ "dozi",    OP(9),	OP_MASK,	M601,		{ RT, RA, SI } },
-
-{ "bce",     B(9,0,0),	B_MASK,		BOOKE64,	{ BO, BI, BD } },
-{ "bcel",    B(9,0,1),	B_MASK,		BOOKE64,	{ BO, BI, BD } },
-{ "bcea",    B(9,1,0),	B_MASK,		BOOKE64,	{ BO, BI, BDA } },
-{ "bcela",   B(9,1,1),	B_MASK,		BOOKE64,	{ BO, BI, BDA } },
-
-{ "cmplwi",  OPL(10,0),	OPL_MASK,	PPCCOM,		{ OBF, RA, UI } },
-{ "cmpldi",  OPL(10,1), OPL_MASK,	PPC64,		{ OBF, RA, UI } },
-{ "cmpli",   OP(10),	OP_MASK,	PPC,		{ BF, L, RA, UI } },
-{ "cmpli",   OP(10),	OP_MASK,	PWRCOM,		{ BF, RA, UI } },
-
-{ "cmpwi",   OPL(11,0),	OPL_MASK,	PPCCOM,		{ OBF, RA, SI } },
-{ "cmpdi",   OPL(11,1),	OPL_MASK,	PPC64,		{ OBF, RA, SI } },
-{ "cmpi",    OP(11),	OP_MASK,	PPC,		{ BF, L, RA, SI } },
-{ "cmpi",    OP(11),	OP_MASK,	PWRCOM,		{ BF, RA, SI } },
-
-{ "addic",   OP(12),	OP_MASK,	PPCCOM,		{ RT, RA, SI } },
-{ "ai",	     OP(12),	OP_MASK,	PWRCOM,		{ RT, RA, SI } },
-{ "subic",   OP(12),	OP_MASK,	PPCCOM,		{ RT, RA, NSI } },
-
-{ "addic.",  OP(13),	OP_MASK,	PPCCOM,		{ RT, RA, SI } },
-{ "ai.",     OP(13),	OP_MASK,	PWRCOM,		{ RT, RA, SI } },
-{ "subic.",  OP(13),	OP_MASK,	PPCCOM,		{ RT, RA, NSI } },
-
-{ "li",	     OP(14),	DRA_MASK,	PPCCOM,		{ RT, SI } },
-{ "lil",     OP(14),	DRA_MASK,	PWRCOM,		{ RT, SI } },
-{ "addi",    OP(14),	OP_MASK,	PPCCOM,		{ RT, RA0, SI } },
-{ "cal",     OP(14),	OP_MASK,	PWRCOM,		{ RT, D, RA0 } },
-{ "subi",    OP(14),	OP_MASK,	PPCCOM,		{ RT, RA0, NSI } },
-{ "la",	     OP(14),	OP_MASK,	PPCCOM,		{ RT, D, RA0 } },
-
-{ "lis",     OP(15),	DRA_MASK,	PPCCOM,		{ RT, SISIGNOPT } },
-{ "liu",     OP(15),	DRA_MASK,	PWRCOM,		{ RT, SISIGNOPT } },
-{ "addis",   OP(15),	OP_MASK,	PPCCOM,		{ RT,RA0,SISIGNOPT } },
-{ "cau",     OP(15),	OP_MASK,	PWRCOM,		{ RT,RA0,SISIGNOPT } },
-{ "subis",   OP(15),	OP_MASK,	PPCCOM,		{ RT, RA0, NSI } },
-
-{ "bdnz-",   BBO(16,BODNZ,0,0),      BBOATBI_MASK, PPCCOM,	{ BDM } },
-{ "bdnz+",   BBO(16,BODNZ,0,0),      BBOATBI_MASK, PPCCOM,	{ BDP } },
-{ "bdnz",    BBO(16,BODNZ,0,0),      BBOATBI_MASK, PPCCOM,	{ BD } },
-{ "bdn",     BBO(16,BODNZ,0,0),      BBOATBI_MASK, PWRCOM,	{ BD } },
-{ "bdnzl-",  BBO(16,BODNZ,0,1),      BBOATBI_MASK, PPCCOM,	{ BDM } },
-{ "bdnzl+",  BBO(16,BODNZ,0,1),      BBOATBI_MASK, PPCCOM,	{ BDP } },
-{ "bdnzl",   BBO(16,BODNZ,0,1),      BBOATBI_MASK, PPCCOM,	{ BD } },
-{ "bdnl",    BBO(16,BODNZ,0,1),      BBOATBI_MASK, PWRCOM,	{ BD } },
-{ "bdnza-",  BBO(16,BODNZ,1,0),      BBOATBI_MASK, PPCCOM,	{ BDMA } },
-{ "bdnza+",  BBO(16,BODNZ,1,0),      BBOATBI_MASK, PPCCOM,	{ BDPA } },
-{ "bdnza",   BBO(16,BODNZ,1,0),      BBOATBI_MASK, PPCCOM,	{ BDA } },
-{ "bdna",    BBO(16,BODNZ,1,0),      BBOATBI_MASK, PWRCOM,	{ BDA } },
-{ "bdnzla-", BBO(16,BODNZ,1,1),      BBOATBI_MASK, PPCCOM,	{ BDMA } },
-{ "bdnzla+", BBO(16,BODNZ,1,1),      BBOATBI_MASK, PPCCOM,	{ BDPA } },
-{ "bdnzla",  BBO(16,BODNZ,1,1),      BBOATBI_MASK, PPCCOM,	{ BDA } },
-{ "bdnla",   BBO(16,BODNZ,1,1),      BBOATBI_MASK, PWRCOM,	{ BDA } },
-{ "bdz-",    BBO(16,BODZ,0,0),       BBOATBI_MASK, PPCCOM,	{ BDM } },
-{ "bdz+",    BBO(16,BODZ,0,0),       BBOATBI_MASK, PPCCOM,	{ BDP } },
-{ "bdz",     BBO(16,BODZ,0,0),       BBOATBI_MASK, COM,		{ BD } },
-{ "bdzl-",   BBO(16,BODZ,0,1),       BBOATBI_MASK, PPCCOM,	{ BDM } },
-{ "bdzl+",   BBO(16,BODZ,0,1),       BBOATBI_MASK, PPCCOM,	{ BDP } },
-{ "bdzl",    BBO(16,BODZ,0,1),       BBOATBI_MASK, COM,		{ BD } },
-{ "bdza-",   BBO(16,BODZ,1,0),       BBOATBI_MASK, PPCCOM,	{ BDMA } },
-{ "bdza+",   BBO(16,BODZ,1,0),       BBOATBI_MASK, PPCCOM,	{ BDPA } },
-{ "bdza",    BBO(16,BODZ,1,0),       BBOATBI_MASK, COM,		{ BDA } },
-{ "bdzla-",  BBO(16,BODZ,1,1),       BBOATBI_MASK, PPCCOM,	{ BDMA } },
-{ "bdzla+",  BBO(16,BODZ,1,1),       BBOATBI_MASK, PPCCOM,	{ BDPA } },
-{ "bdzla",   BBO(16,BODZ,1,1),       BBOATBI_MASK, COM,		{ BDA } },
-{ "blt-",    BBOCB(16,BOT,CBLT,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "blt+",    BBOCB(16,BOT,CBLT,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "blt",     BBOCB(16,BOT,CBLT,0,0), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "bltl-",   BBOCB(16,BOT,CBLT,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bltl+",   BBOCB(16,BOT,CBLT,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bltl",    BBOCB(16,BOT,CBLT,0,1), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "blta-",   BBOCB(16,BOT,CBLT,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "blta+",   BBOCB(16,BOT,CBLT,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "blta",    BBOCB(16,BOT,CBLT,1,0), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "bltla-",  BBOCB(16,BOT,CBLT,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bltla+",  BBOCB(16,BOT,CBLT,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bltla",   BBOCB(16,BOT,CBLT,1,1), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "bgt-",    BBOCB(16,BOT,CBGT,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bgt+",    BBOCB(16,BOT,CBGT,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bgt",     BBOCB(16,BOT,CBGT,0,0), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "bgtl-",   BBOCB(16,BOT,CBGT,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bgtl+",   BBOCB(16,BOT,CBGT,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bgtl",    BBOCB(16,BOT,CBGT,0,1), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "bgta-",   BBOCB(16,BOT,CBGT,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bgta+",   BBOCB(16,BOT,CBGT,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bgta",    BBOCB(16,BOT,CBGT,1,0), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "bgtla-",  BBOCB(16,BOT,CBGT,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bgtla+",  BBOCB(16,BOT,CBGT,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bgtla",   BBOCB(16,BOT,CBGT,1,1), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "beq-",    BBOCB(16,BOT,CBEQ,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "beq+",    BBOCB(16,BOT,CBEQ,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "beq",     BBOCB(16,BOT,CBEQ,0,0), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "beql-",   BBOCB(16,BOT,CBEQ,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "beql+",   BBOCB(16,BOT,CBEQ,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "beql",    BBOCB(16,BOT,CBEQ,0,1), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "beqa-",   BBOCB(16,BOT,CBEQ,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "beqa+",   BBOCB(16,BOT,CBEQ,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "beqa",    BBOCB(16,BOT,CBEQ,1,0), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "beqla-",  BBOCB(16,BOT,CBEQ,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "beqla+",  BBOCB(16,BOT,CBEQ,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "beqla",   BBOCB(16,BOT,CBEQ,1,1), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "bso-",    BBOCB(16,BOT,CBSO,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bso+",    BBOCB(16,BOT,CBSO,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bso",     BBOCB(16,BOT,CBSO,0,0), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "bsol-",   BBOCB(16,BOT,CBSO,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bsol+",   BBOCB(16,BOT,CBSO,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bsol",    BBOCB(16,BOT,CBSO,0,1), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "bsoa-",   BBOCB(16,BOT,CBSO,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bsoa+",   BBOCB(16,BOT,CBSO,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bsoa",    BBOCB(16,BOT,CBSO,1,0), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "bsola-",  BBOCB(16,BOT,CBSO,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bsola+",  BBOCB(16,BOT,CBSO,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bsola",   BBOCB(16,BOT,CBSO,1,1), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "bun-",    BBOCB(16,BOT,CBSO,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bun+",    BBOCB(16,BOT,CBSO,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bun",     BBOCB(16,BOT,CBSO,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BD } },
-{ "bunl-",   BBOCB(16,BOT,CBSO,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bunl+",   BBOCB(16,BOT,CBSO,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bunl",    BBOCB(16,BOT,CBSO,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BD } },
-{ "buna-",   BBOCB(16,BOT,CBSO,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "buna+",   BBOCB(16,BOT,CBSO,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "buna",    BBOCB(16,BOT,CBSO,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDA } },
-{ "bunla-",  BBOCB(16,BOT,CBSO,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bunla+",  BBOCB(16,BOT,CBSO,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bunla",   BBOCB(16,BOT,CBSO,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDA } },
-{ "bge-",    BBOCB(16,BOF,CBLT,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bge+",    BBOCB(16,BOF,CBLT,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bge",     BBOCB(16,BOF,CBLT,0,0), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "bgel-",   BBOCB(16,BOF,CBLT,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bgel+",   BBOCB(16,BOF,CBLT,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bgel",    BBOCB(16,BOF,CBLT,0,1), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "bgea-",   BBOCB(16,BOF,CBLT,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bgea+",   BBOCB(16,BOF,CBLT,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bgea",    BBOCB(16,BOF,CBLT,1,0), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "bgela-",  BBOCB(16,BOF,CBLT,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bgela+",  BBOCB(16,BOF,CBLT,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bgela",   BBOCB(16,BOF,CBLT,1,1), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "bnl-",    BBOCB(16,BOF,CBLT,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bnl+",    BBOCB(16,BOF,CBLT,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bnl",     BBOCB(16,BOF,CBLT,0,0), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "bnll-",   BBOCB(16,BOF,CBLT,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bnll+",   BBOCB(16,BOF,CBLT,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bnll",    BBOCB(16,BOF,CBLT,0,1), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "bnla-",   BBOCB(16,BOF,CBLT,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bnla+",   BBOCB(16,BOF,CBLT,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bnla",    BBOCB(16,BOF,CBLT,1,0), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "bnlla-",  BBOCB(16,BOF,CBLT,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bnlla+",  BBOCB(16,BOF,CBLT,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bnlla",   BBOCB(16,BOF,CBLT,1,1), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "ble-",    BBOCB(16,BOF,CBGT,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "ble+",    BBOCB(16,BOF,CBGT,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "ble",     BBOCB(16,BOF,CBGT,0,0), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "blel-",   BBOCB(16,BOF,CBGT,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "blel+",   BBOCB(16,BOF,CBGT,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "blel",    BBOCB(16,BOF,CBGT,0,1), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "blea-",   BBOCB(16,BOF,CBGT,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "blea+",   BBOCB(16,BOF,CBGT,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "blea",    BBOCB(16,BOF,CBGT,1,0), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "blela-",  BBOCB(16,BOF,CBGT,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "blela+",  BBOCB(16,BOF,CBGT,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "blela",   BBOCB(16,BOF,CBGT,1,1), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "bng-",    BBOCB(16,BOF,CBGT,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bng+",    BBOCB(16,BOF,CBGT,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bng",     BBOCB(16,BOF,CBGT,0,0), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "bngl-",   BBOCB(16,BOF,CBGT,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bngl+",   BBOCB(16,BOF,CBGT,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bngl",    BBOCB(16,BOF,CBGT,0,1), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "bnga-",   BBOCB(16,BOF,CBGT,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bnga+",   BBOCB(16,BOF,CBGT,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bnga",    BBOCB(16,BOF,CBGT,1,0), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "bngla-",  BBOCB(16,BOF,CBGT,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bngla+",  BBOCB(16,BOF,CBGT,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bngla",   BBOCB(16,BOF,CBGT,1,1), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "bne-",    BBOCB(16,BOF,CBEQ,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bne+",    BBOCB(16,BOF,CBEQ,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bne",     BBOCB(16,BOF,CBEQ,0,0), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "bnel-",   BBOCB(16,BOF,CBEQ,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bnel+",   BBOCB(16,BOF,CBEQ,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bnel",    BBOCB(16,BOF,CBEQ,0,1), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "bnea-",   BBOCB(16,BOF,CBEQ,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bnea+",   BBOCB(16,BOF,CBEQ,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bnea",    BBOCB(16,BOF,CBEQ,1,0), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "bnela-",  BBOCB(16,BOF,CBEQ,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bnela+",  BBOCB(16,BOF,CBEQ,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bnela",   BBOCB(16,BOF,CBEQ,1,1), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "bns-",    BBOCB(16,BOF,CBSO,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bns+",    BBOCB(16,BOF,CBSO,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bns",     BBOCB(16,BOF,CBSO,0,0), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "bnsl-",   BBOCB(16,BOF,CBSO,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bnsl+",   BBOCB(16,BOF,CBSO,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bnsl",    BBOCB(16,BOF,CBSO,0,1), BBOATCB_MASK, COM,		{ CR, BD } },
-{ "bnsa-",   BBOCB(16,BOF,CBSO,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bnsa+",   BBOCB(16,BOF,CBSO,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bnsa",    BBOCB(16,BOF,CBSO,1,0), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "bnsla-",  BBOCB(16,BOF,CBSO,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bnsla+",  BBOCB(16,BOF,CBSO,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bnsla",   BBOCB(16,BOF,CBSO,1,1), BBOATCB_MASK, COM,		{ CR, BDA } },
-{ "bnu-",    BBOCB(16,BOF,CBSO,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bnu+",    BBOCB(16,BOF,CBSO,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bnu",     BBOCB(16,BOF,CBSO,0,0), BBOATCB_MASK, PPCCOM,	{ CR, BD } },
-{ "bnul-",   BBOCB(16,BOF,CBSO,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDM } },
-{ "bnul+",   BBOCB(16,BOF,CBSO,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BDP } },
-{ "bnul",    BBOCB(16,BOF,CBSO,0,1), BBOATCB_MASK, PPCCOM,	{ CR, BD } },
-{ "bnua-",   BBOCB(16,BOF,CBSO,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bnua+",   BBOCB(16,BOF,CBSO,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bnua",    BBOCB(16,BOF,CBSO,1,0), BBOATCB_MASK, PPCCOM,	{ CR, BDA } },
-{ "bnula-",  BBOCB(16,BOF,CBSO,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDMA } },
-{ "bnula+",  BBOCB(16,BOF,CBSO,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDPA } },
-{ "bnula",   BBOCB(16,BOF,CBSO,1,1), BBOATCB_MASK, PPCCOM,	{ CR, BDA } },
-{ "bdnzt-",  BBO(16,BODNZT,0,0), BBOY_MASK, NOPOWER4,	{ BI, BDM } },
-{ "bdnzt+",  BBO(16,BODNZT,0,0), BBOY_MASK, NOPOWER4,	{ BI, BDP } },
-{ "bdnzt",   BBO(16,BODNZT,0,0), BBOY_MASK, PPCCOM,	{ BI, BD } },
-{ "bdnztl-", BBO(16,BODNZT,0,1), BBOY_MASK, NOPOWER4,	{ BI, BDM } },
-{ "bdnztl+", BBO(16,BODNZT,0,1), BBOY_MASK, NOPOWER4,	{ BI, BDP } },
-{ "bdnztl",  BBO(16,BODNZT,0,1), BBOY_MASK, PPCCOM,	{ BI, BD } },
-{ "bdnzta-", BBO(16,BODNZT,1,0), BBOY_MASK, NOPOWER4,	{ BI, BDMA } },
-{ "bdnzta+", BBO(16,BODNZT,1,0), BBOY_MASK, NOPOWER4,	{ BI, BDPA } },
-{ "bdnzta",  BBO(16,BODNZT,1,0), BBOY_MASK, PPCCOM,	{ BI, BDA } },
-{ "bdnztla-",BBO(16,BODNZT,1,1), BBOY_MASK, NOPOWER4,	{ BI, BDMA } },
-{ "bdnztla+",BBO(16,BODNZT,1,1), BBOY_MASK, NOPOWER4,	{ BI, BDPA } },
-{ "bdnztla", BBO(16,BODNZT,1,1), BBOY_MASK, PPCCOM,	{ BI, BDA } },
-{ "bdnzf-",  BBO(16,BODNZF,0,0), BBOY_MASK, NOPOWER4,	{ BI, BDM } },
-{ "bdnzf+",  BBO(16,BODNZF,0,0), BBOY_MASK, NOPOWER4,	{ BI, BDP } },
-{ "bdnzf",   BBO(16,BODNZF,0,0), BBOY_MASK, PPCCOM,	{ BI, BD } },
-{ "bdnzfl-", BBO(16,BODNZF,0,1), BBOY_MASK, NOPOWER4,	{ BI, BDM } },
-{ "bdnzfl+", BBO(16,BODNZF,0,1), BBOY_MASK, NOPOWER4,	{ BI, BDP } },
-{ "bdnzfl",  BBO(16,BODNZF,0,1), BBOY_MASK, PPCCOM,	{ BI, BD } },
-{ "bdnzfa-", BBO(16,BODNZF,1,0), BBOY_MASK, NOPOWER4,	{ BI, BDMA } },
-{ "bdnzfa+", BBO(16,BODNZF,1,0), BBOY_MASK, NOPOWER4,	{ BI, BDPA } },
-{ "bdnzfa",  BBO(16,BODNZF,1,0), BBOY_MASK, PPCCOM,	{ BI, BDA } },
-{ "bdnzfla-",BBO(16,BODNZF,1,1), BBOY_MASK, NOPOWER4,	{ BI, BDMA } },
-{ "bdnzfla+",BBO(16,BODNZF,1,1), BBOY_MASK, NOPOWER4,	{ BI, BDPA } },
-{ "bdnzfla", BBO(16,BODNZF,1,1), BBOY_MASK, PPCCOM,	{ BI, BDA } },
-{ "bt-",     BBO(16,BOT,0,0), BBOAT_MASK, PPCCOM,	{ BI, BDM } },
-{ "bt+",     BBO(16,BOT,0,0), BBOAT_MASK, PPCCOM,	{ BI, BDP } },
-{ "bt",	     BBO(16,BOT,0,0), BBOAT_MASK, PPCCOM,	{ BI, BD } },
-{ "bbt",     BBO(16,BOT,0,0), BBOAT_MASK, PWRCOM,	{ BI, BD } },
-{ "btl-",    BBO(16,BOT,0,1), BBOAT_MASK, PPCCOM,	{ BI, BDM } },
-{ "btl+",    BBO(16,BOT,0,1), BBOAT_MASK, PPCCOM,	{ BI, BDP } },
-{ "btl",     BBO(16,BOT,0,1), BBOAT_MASK, PPCCOM,	{ BI, BD } },
-{ "bbtl",    BBO(16,BOT,0,1), BBOAT_MASK, PWRCOM,	{ BI, BD } },
-{ "bta-",    BBO(16,BOT,1,0), BBOAT_MASK, PPCCOM,	{ BI, BDMA } },
-{ "bta+",    BBO(16,BOT,1,0), BBOAT_MASK, PPCCOM,	{ BI, BDPA } },
-{ "bta",     BBO(16,BOT,1,0), BBOAT_MASK, PPCCOM,	{ BI, BDA } },
-{ "bbta",    BBO(16,BOT,1,0), BBOAT_MASK, PWRCOM,	{ BI, BDA } },
-{ "btla-",   BBO(16,BOT,1,1), BBOAT_MASK, PPCCOM,	{ BI, BDMA } },
-{ "btla+",   BBO(16,BOT,1,1), BBOAT_MASK, PPCCOM,	{ BI, BDPA } },
-{ "btla",    BBO(16,BOT,1,1), BBOAT_MASK, PPCCOM,	{ BI, BDA } },
-{ "bbtla",   BBO(16,BOT,1,1), BBOAT_MASK, PWRCOM,	{ BI, BDA } },
-{ "bf-",     BBO(16,BOF,0,0), BBOAT_MASK, PPCCOM,	{ BI, BDM } },
-{ "bf+",     BBO(16,BOF,0,0), BBOAT_MASK, PPCCOM,	{ BI, BDP } },
-{ "bf",	     BBO(16,BOF,0,0), BBOAT_MASK, PPCCOM,	{ BI, BD } },
-{ "bbf",     BBO(16,BOF,0,0), BBOAT_MASK, PWRCOM,	{ BI, BD } },
-{ "bfl-",    BBO(16,BOF,0,1), BBOAT_MASK, PPCCOM,	{ BI, BDM } },
-{ "bfl+",    BBO(16,BOF,0,1), BBOAT_MASK, PPCCOM,	{ BI, BDP } },
-{ "bfl",     BBO(16,BOF,0,1), BBOAT_MASK, PPCCOM,	{ BI, BD } },
-{ "bbfl",    BBO(16,BOF,0,1), BBOAT_MASK, PWRCOM,	{ BI, BD } },
-{ "bfa-",    BBO(16,BOF,1,0), BBOAT_MASK, PPCCOM,	{ BI, BDMA } },
-{ "bfa+",    BBO(16,BOF,1,0), BBOAT_MASK, PPCCOM,	{ BI, BDPA } },
-{ "bfa",     BBO(16,BOF,1,0), BBOAT_MASK, PPCCOM,	{ BI, BDA } },
-{ "bbfa",    BBO(16,BOF,1,0), BBOAT_MASK, PWRCOM,	{ BI, BDA } },
-{ "bfla-",   BBO(16,BOF,1,1), BBOAT_MASK, PPCCOM,	{ BI, BDMA } },
-{ "bfla+",   BBO(16,BOF,1,1), BBOAT_MASK, PPCCOM,	{ BI, BDPA } },
-{ "bfla",    BBO(16,BOF,1,1), BBOAT_MASK, PPCCOM,	{ BI, BDA } },
-{ "bbfla",   BBO(16,BOF,1,1), BBOAT_MASK, PWRCOM,	{ BI, BDA } },
-{ "bdzt-",   BBO(16,BODZT,0,0), BBOY_MASK, NOPOWER4,	{ BI, BDM } },
-{ "bdzt+",   BBO(16,BODZT,0,0), BBOY_MASK, NOPOWER4,	{ BI, BDP } },
-{ "bdzt",    BBO(16,BODZT,0,0), BBOY_MASK, PPCCOM,	{ BI, BD } },
-{ "bdztl-",  BBO(16,BODZT,0,1), BBOY_MASK, NOPOWER4,	{ BI, BDM } },
-{ "bdztl+",  BBO(16,BODZT,0,1), BBOY_MASK, NOPOWER4,	{ BI, BDP } },
-{ "bdztl",   BBO(16,BODZT,0,1), BBOY_MASK, PPCCOM,	{ BI, BD } },
-{ "bdzta-",  BBO(16,BODZT,1,0), BBOY_MASK, NOPOWER4,	{ BI, BDMA } },
-{ "bdzta+",  BBO(16,BODZT,1,0), BBOY_MASK, NOPOWER4,	{ BI, BDPA } },
-{ "bdzta",   BBO(16,BODZT,1,0), BBOY_MASK, PPCCOM,	{ BI, BDA } },
-{ "bdztla-", BBO(16,BODZT,1,1), BBOY_MASK, NOPOWER4,	{ BI, BDMA } },
-{ "bdztla+", BBO(16,BODZT,1,1), BBOY_MASK, NOPOWER4,	{ BI, BDPA } },
-{ "bdztla",  BBO(16,BODZT,1,1), BBOY_MASK, PPCCOM,	{ BI, BDA } },
-{ "bdzf-",   BBO(16,BODZF,0,0), BBOY_MASK, NOPOWER4,	{ BI, BDM } },
-{ "bdzf+",   BBO(16,BODZF,0,0), BBOY_MASK, NOPOWER4,	{ BI, BDP } },
-{ "bdzf",    BBO(16,BODZF,0,0), BBOY_MASK, PPCCOM,	{ BI, BD } },
-{ "bdzfl-",  BBO(16,BODZF,0,1), BBOY_MASK, NOPOWER4,	{ BI, BDM } },
-{ "bdzfl+",  BBO(16,BODZF,0,1), BBOY_MASK, NOPOWER4,	{ BI, BDP } },
-{ "bdzfl",   BBO(16,BODZF,0,1), BBOY_MASK, PPCCOM,	{ BI, BD } },
-{ "bdzfa-",  BBO(16,BODZF,1,0), BBOY_MASK, NOPOWER4,	{ BI, BDMA } },
-{ "bdzfa+",  BBO(16,BODZF,1,0), BBOY_MASK, NOPOWER4,	{ BI, BDPA } },
-{ "bdzfa",   BBO(16,BODZF,1,0), BBOY_MASK, PPCCOM,	{ BI, BDA } },
-{ "bdzfla-", BBO(16,BODZF,1,1), BBOY_MASK, NOPOWER4,	{ BI, BDMA } },
-{ "bdzfla+", BBO(16,BODZF,1,1), BBOY_MASK, NOPOWER4,	{ BI, BDPA } },
-{ "bdzfla",  BBO(16,BODZF,1,1), BBOY_MASK, PPCCOM,	{ BI, BDA } },
-{ "bc-",     B(16,0,0),	B_MASK,		PPCCOM,		{ BOE, BI, BDM } },
-{ "bc+",     B(16,0,0),	B_MASK,		PPCCOM,		{ BOE, BI, BDP } },
-{ "bc",	     B(16,0,0),	B_MASK,		COM,		{ BO, BI, BD } },
-{ "bcl-",    B(16,0,1),	B_MASK,		PPCCOM,		{ BOE, BI, BDM } },
-{ "bcl+",    B(16,0,1),	B_MASK,		PPCCOM,		{ BOE, BI, BDP } },
-{ "bcl",     B(16,0,1),	B_MASK,		COM,		{ BO, BI, BD } },
-{ "bca-",    B(16,1,0),	B_MASK,		PPCCOM,		{ BOE, BI, BDMA } },
-{ "bca+",    B(16,1,0),	B_MASK,		PPCCOM,		{ BOE, BI, BDPA } },
-{ "bca",     B(16,1,0),	B_MASK,		COM,		{ BO, BI, BDA } },
-{ "bcla-",   B(16,1,1),	B_MASK,		PPCCOM,		{ BOE, BI, BDMA } },
-{ "bcla+",   B(16,1,1),	B_MASK,		PPCCOM,		{ BOE, BI, BDPA } },
-{ "bcla",    B(16,1,1),	B_MASK,		COM,		{ BO, BI, BDA } },
-
-{ "sc",      SC(17,1,0), SC_MASK,	PPC,		{ LEV } },
-{ "svc",     SC(17,0,0), SC_MASK,	POWER,		{ SVC_LEV, FL1, FL2 } },
-{ "svcl",    SC(17,0,1), SC_MASK,	POWER,		{ SVC_LEV, FL1, FL2 } },
-{ "svca",    SC(17,1,0), SC_MASK,	PWRCOM,		{ SV } },
-{ "svcla",   SC(17,1,1), SC_MASK,	POWER,		{ SV } },
-
-{ "b",	     B(18,0,0),	B_MASK,		COM,		{ LI } },
-{ "bl",      B(18,0,1),	B_MASK,		COM,		{ LI } },
-{ "ba",      B(18,1,0),	B_MASK,		COM,		{ LIA } },
-{ "bla",     B(18,1,1),	B_MASK,		COM,		{ LIA } },
-
-{ "mcrf",    XL(19,0),	XLBB_MASK|(3 << 21)|(3 << 16), COM,	{ BF, BFA } },
-
-{ "blr",     XLO(19,BOU,16,0), XLBOBIBB_MASK, PPCCOM,	{ 0 } },
-{ "br",      XLO(19,BOU,16,0), XLBOBIBB_MASK, PWRCOM,	{ 0 } },
-{ "blrl",    XLO(19,BOU,16,1), XLBOBIBB_MASK, PPCCOM,	{ 0 } },
-{ "brl",     XLO(19,BOU,16,1), XLBOBIBB_MASK, PWRCOM,	{ 0 } },
-{ "bdnzlr",  XLO(19,BODNZ,16,0), XLBOBIBB_MASK, PPCCOM,	{ 0 } },
-{ "bdnzlr-", XLO(19,BODNZ,16,0), XLBOBIBB_MASK, NOPOWER4,	{ 0 } },
-{ "bdnzlr-", XLO(19,BODNZM4,16,0), XLBOBIBB_MASK, POWER4,	{ 0 } },
-{ "bdnzlr+", XLO(19,BODNZP,16,0), XLBOBIBB_MASK, NOPOWER4,	{ 0 } },
-{ "bdnzlr+", XLO(19,BODNZP4,16,0), XLBOBIBB_MASK, POWER4,	{ 0 } },
-{ "bdnzlrl", XLO(19,BODNZ,16,1), XLBOBIBB_MASK, PPCCOM,	{ 0 } },
-{ "bdnzlrl-",XLO(19,BODNZ,16,1), XLBOBIBB_MASK, NOPOWER4,	{ 0 } },
-{ "bdnzlrl-",XLO(19,BODNZM4,16,1), XLBOBIBB_MASK, POWER4,	{ 0 } },
-{ "bdnzlrl+",XLO(19,BODNZP,16,1), XLBOBIBB_MASK, NOPOWER4,	{ 0 } },
-{ "bdnzlrl+",XLO(19,BODNZP4,16,1), XLBOBIBB_MASK, POWER4,	{ 0 } },
-{ "bdzlr",   XLO(19,BODZ,16,0), XLBOBIBB_MASK, PPCCOM,	{ 0 } },
-{ "bdzlr-",  XLO(19,BODZ,16,0), XLBOBIBB_MASK, NOPOWER4,	{ 0 } },
-{ "bdzlr-",  XLO(19,BODZM4,16,0), XLBOBIBB_MASK, POWER4,	{ 0 } },
-{ "bdzlr+",  XLO(19,BODZP,16,0), XLBOBIBB_MASK, NOPOWER4,	{ 0 } },
-{ "bdzlr+",  XLO(19,BODZP4,16,0), XLBOBIBB_MASK, POWER4,	{ 0 } },
-{ "bdzlrl",  XLO(19,BODZ,16,1), XLBOBIBB_MASK, PPCCOM,	{ 0 } },
-{ "bdzlrl-", XLO(19,BODZ,16,1), XLBOBIBB_MASK, NOPOWER4,	{ 0 } },
-{ "bdzlrl-", XLO(19,BODZM4,16,1), XLBOBIBB_MASK, POWER4,	{ 0 } },
-{ "bdzlrl+", XLO(19,BODZP,16,1), XLBOBIBB_MASK, NOPOWER4,	{ 0 } },
-{ "bdzlrl+", XLO(19,BODZP4,16,1), XLBOBIBB_MASK, POWER4,	{ 0 } },
-{ "bltlr",   XLOCB(19,BOT,CBLT,16,0), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bltlr-",  XLOCB(19,BOT,CBLT,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bltlr-",  XLOCB(19,BOTM4,CBLT,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bltlr+",  XLOCB(19,BOTP,CBLT,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bltlr+",  XLOCB(19,BOTP4,CBLT,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bltr",    XLOCB(19,BOT,CBLT,16,0), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "bltlrl",  XLOCB(19,BOT,CBLT,16,1), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bltlrl-", XLOCB(19,BOT,CBLT,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bltlrl-", XLOCB(19,BOTM4,CBLT,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bltlrl+", XLOCB(19,BOTP,CBLT,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bltlrl+", XLOCB(19,BOTP4,CBLT,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bltrl",   XLOCB(19,BOT,CBLT,16,1), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "bgtlr",   XLOCB(19,BOT,CBGT,16,0), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bgtlr-",  XLOCB(19,BOT,CBGT,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bgtlr-",  XLOCB(19,BOTM4,CBGT,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bgtlr+",  XLOCB(19,BOTP,CBGT,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bgtlr+",  XLOCB(19,BOTP4,CBGT,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bgtr",    XLOCB(19,BOT,CBGT,16,0), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "bgtlrl",  XLOCB(19,BOT,CBGT,16,1), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bgtlrl-", XLOCB(19,BOT,CBGT,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bgtlrl-", XLOCB(19,BOTM4,CBGT,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bgtlrl+", XLOCB(19,BOTP,CBGT,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bgtlrl+", XLOCB(19,BOTP4,CBGT,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bgtrl",   XLOCB(19,BOT,CBGT,16,1), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "beqlr",   XLOCB(19,BOT,CBEQ,16,0), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "beqlr-",  XLOCB(19,BOT,CBEQ,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "beqlr-",  XLOCB(19,BOTM4,CBEQ,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "beqlr+",  XLOCB(19,BOTP,CBEQ,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "beqlr+",  XLOCB(19,BOTP4,CBEQ,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "beqr",    XLOCB(19,BOT,CBEQ,16,0), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "beqlrl",  XLOCB(19,BOT,CBEQ,16,1), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "beqlrl-", XLOCB(19,BOT,CBEQ,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "beqlrl-", XLOCB(19,BOTM4,CBEQ,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "beqlrl+", XLOCB(19,BOTP,CBEQ,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "beqlrl+", XLOCB(19,BOTP4,CBEQ,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "beqrl",   XLOCB(19,BOT,CBEQ,16,1), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "bsolr",   XLOCB(19,BOT,CBSO,16,0), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bsolr-",  XLOCB(19,BOT,CBSO,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bsolr-",  XLOCB(19,BOTM4,CBSO,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bsolr+",  XLOCB(19,BOTP,CBSO,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bsolr+",  XLOCB(19,BOTP4,CBSO,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bsor",    XLOCB(19,BOT,CBSO,16,0), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "bsolrl",  XLOCB(19,BOT,CBSO,16,1), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bsolrl-", XLOCB(19,BOT,CBSO,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bsolrl-", XLOCB(19,BOTM4,CBSO,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bsolrl+", XLOCB(19,BOTP,CBSO,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bsolrl+", XLOCB(19,BOTP4,CBSO,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bsorl",   XLOCB(19,BOT,CBSO,16,1), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "bunlr",   XLOCB(19,BOT,CBSO,16,0), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bunlr-",  XLOCB(19,BOT,CBSO,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bunlr-",  XLOCB(19,BOTM4,CBSO,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bunlr+",  XLOCB(19,BOTP,CBSO,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bunlr+",  XLOCB(19,BOTP4,CBSO,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bunlrl",  XLOCB(19,BOT,CBSO,16,1), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bunlrl-", XLOCB(19,BOT,CBSO,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bunlrl-", XLOCB(19,BOTM4,CBSO,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bunlrl+", XLOCB(19,BOTP,CBSO,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bunlrl+", XLOCB(19,BOTP4,CBSO,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bgelr",   XLOCB(19,BOF,CBLT,16,0), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bgelr-",  XLOCB(19,BOF,CBLT,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bgelr-",  XLOCB(19,BOFM4,CBLT,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bgelr+",  XLOCB(19,BOFP,CBLT,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bgelr+",  XLOCB(19,BOFP4,CBLT,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bger",    XLOCB(19,BOF,CBLT,16,0), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "bgelrl",  XLOCB(19,BOF,CBLT,16,1), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bgelrl-", XLOCB(19,BOF,CBLT,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bgelrl-", XLOCB(19,BOFM4,CBLT,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bgelrl+", XLOCB(19,BOFP,CBLT,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bgelrl+", XLOCB(19,BOFP4,CBLT,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bgerl",   XLOCB(19,BOF,CBLT,16,1), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "bnllr",   XLOCB(19,BOF,CBLT,16,0), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bnllr-",  XLOCB(19,BOF,CBLT,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnllr-",  XLOCB(19,BOFM4,CBLT,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnllr+",  XLOCB(19,BOFP,CBLT,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnllr+",  XLOCB(19,BOFP4,CBLT,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnlr",    XLOCB(19,BOF,CBLT,16,0), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "bnllrl",  XLOCB(19,BOF,CBLT,16,1), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bnllrl-", XLOCB(19,BOF,CBLT,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnllrl-", XLOCB(19,BOFM4,CBLT,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnllrl+", XLOCB(19,BOFP,CBLT,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnllrl+", XLOCB(19,BOFP4,CBLT,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnlrl",   XLOCB(19,BOF,CBLT,16,1), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "blelr",   XLOCB(19,BOF,CBGT,16,0), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "blelr-",  XLOCB(19,BOF,CBGT,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "blelr-",  XLOCB(19,BOFM4,CBGT,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "blelr+",  XLOCB(19,BOFP,CBGT,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "blelr+",  XLOCB(19,BOFP4,CBGT,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bler",    XLOCB(19,BOF,CBGT,16,0), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "blelrl",  XLOCB(19,BOF,CBGT,16,1), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "blelrl-", XLOCB(19,BOF,CBGT,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "blelrl-", XLOCB(19,BOFM4,CBGT,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "blelrl+", XLOCB(19,BOFP,CBGT,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "blelrl+", XLOCB(19,BOFP4,CBGT,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "blerl",   XLOCB(19,BOF,CBGT,16,1), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "bnglr",   XLOCB(19,BOF,CBGT,16,0), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bnglr-",  XLOCB(19,BOF,CBGT,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnglr-",  XLOCB(19,BOFM4,CBGT,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnglr+",  XLOCB(19,BOFP,CBGT,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnglr+",  XLOCB(19,BOFP4,CBGT,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bngr",    XLOCB(19,BOF,CBGT,16,0), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "bnglrl",  XLOCB(19,BOF,CBGT,16,1), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bnglrl-", XLOCB(19,BOF,CBGT,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnglrl-", XLOCB(19,BOFM4,CBGT,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnglrl+", XLOCB(19,BOFP,CBGT,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnglrl+", XLOCB(19,BOFP4,CBGT,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bngrl",   XLOCB(19,BOF,CBGT,16,1), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "bnelr",   XLOCB(19,BOF,CBEQ,16,0), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bnelr-",  XLOCB(19,BOF,CBEQ,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnelr-",  XLOCB(19,BOFM4,CBEQ,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnelr+",  XLOCB(19,BOFP,CBEQ,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnelr+",  XLOCB(19,BOFP4,CBEQ,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bner",    XLOCB(19,BOF,CBEQ,16,0), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "bnelrl",  XLOCB(19,BOF,CBEQ,16,1), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bnelrl-", XLOCB(19,BOF,CBEQ,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnelrl-", XLOCB(19,BOFM4,CBEQ,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnelrl+", XLOCB(19,BOFP,CBEQ,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnelrl+", XLOCB(19,BOFP4,CBEQ,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnerl",   XLOCB(19,BOF,CBEQ,16,1), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "bnslr",   XLOCB(19,BOF,CBSO,16,0), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bnslr-",  XLOCB(19,BOF,CBSO,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnslr-",  XLOCB(19,BOFM4,CBSO,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnslr+",  XLOCB(19,BOFP,CBSO,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnslr+",  XLOCB(19,BOFP4,CBSO,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnsr",    XLOCB(19,BOF,CBSO,16,0), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "bnslrl",  XLOCB(19,BOF,CBSO,16,1), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bnslrl-", XLOCB(19,BOF,CBSO,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnslrl-", XLOCB(19,BOFM4,CBSO,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnslrl+", XLOCB(19,BOFP,CBSO,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnslrl+", XLOCB(19,BOFP4,CBSO,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnsrl",   XLOCB(19,BOF,CBSO,16,1), XLBOCBBB_MASK, PWRCOM, { CR } },
-{ "bnulr",   XLOCB(19,BOF,CBSO,16,0), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bnulr-",  XLOCB(19,BOF,CBSO,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnulr-",  XLOCB(19,BOFM4,CBSO,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnulr+",  XLOCB(19,BOFP,CBSO,16,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnulr+",  XLOCB(19,BOFP4,CBSO,16,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnulrl",  XLOCB(19,BOF,CBSO,16,1), XLBOCBBB_MASK, PPCCOM, { CR } },
-{ "bnulrl-", XLOCB(19,BOF,CBSO,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnulrl-", XLOCB(19,BOFM4,CBSO,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnulrl+", XLOCB(19,BOFP,CBSO,16,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnulrl+", XLOCB(19,BOFP4,CBSO,16,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "btlr",    XLO(19,BOT,16,0), XLBOBB_MASK, PPCCOM,	{ BI } },
-{ "btlr-",   XLO(19,BOT,16,0), XLBOBB_MASK, NOPOWER4,	{ BI } },
-{ "btlr-",   XLO(19,BOTM4,16,0), XLBOBB_MASK, POWER4,	{ BI } },
-{ "btlr+",   XLO(19,BOTP,16,0), XLBOBB_MASK, NOPOWER4,	{ BI } },
-{ "btlr+",   XLO(19,BOTP4,16,0), XLBOBB_MASK, POWER4,	{ BI } },
-{ "bbtr",    XLO(19,BOT,16,0), XLBOBB_MASK, PWRCOM,	{ BI } },
-{ "btlrl",   XLO(19,BOT,16,1), XLBOBB_MASK, PPCCOM,	{ BI } },
-{ "btlrl-",  XLO(19,BOT,16,1), XLBOBB_MASK, NOPOWER4,	{ BI } },
-{ "btlrl-",  XLO(19,BOTM4,16,1), XLBOBB_MASK, POWER4,	{ BI } },
-{ "btlrl+",  XLO(19,BOTP,16,1), XLBOBB_MASK, NOPOWER4,	{ BI } },
-{ "btlrl+",  XLO(19,BOTP4,16,1), XLBOBB_MASK, POWER4,	{ BI } },
-{ "bbtrl",   XLO(19,BOT,16,1), XLBOBB_MASK, PWRCOM,	{ BI } },
-{ "bflr",    XLO(19,BOF,16,0), XLBOBB_MASK, PPCCOM,	{ BI } },
-{ "bflr-",   XLO(19,BOF,16,0), XLBOBB_MASK, NOPOWER4,	{ BI } },
-{ "bflr-",   XLO(19,BOFM4,16,0), XLBOBB_MASK, POWER4,	{ BI } },
-{ "bflr+",   XLO(19,BOFP,16,0), XLBOBB_MASK, NOPOWER4,	{ BI } },
-{ "bflr+",   XLO(19,BOFP4,16,0), XLBOBB_MASK, POWER4,	{ BI } },
-{ "bbfr",    XLO(19,BOF,16,0), XLBOBB_MASK, PWRCOM,	{ BI } },
-{ "bflrl",   XLO(19,BOF,16,1), XLBOBB_MASK, PPCCOM,	{ BI } },
-{ "bflrl-",  XLO(19,BOF,16,1), XLBOBB_MASK, NOPOWER4,	{ BI } },
-{ "bflrl-",  XLO(19,BOFM4,16,1), XLBOBB_MASK, POWER4,	{ BI } },
-{ "bflrl+",  XLO(19,BOFP,16,1), XLBOBB_MASK, NOPOWER4,	{ BI } },
-{ "bflrl+",  XLO(19,BOFP4,16,1), XLBOBB_MASK, POWER4,	{ BI } },
-{ "bbfrl",   XLO(19,BOF,16,1), XLBOBB_MASK, PWRCOM,	{ BI } },
-{ "bdnztlr", XLO(19,BODNZT,16,0), XLBOBB_MASK, PPCCOM,	{ BI } },
-{ "bdnztlr-",XLO(19,BODNZT,16,0), XLBOBB_MASK, NOPOWER4, { BI } },
-{ "bdnztlr+",XLO(19,BODNZTP,16,0), XLBOBB_MASK, NOPOWER4, { BI } },
-{ "bdnztlrl",XLO(19,BODNZT,16,1), XLBOBB_MASK, PPCCOM,	{ BI } },
-{ "bdnztlrl-",XLO(19,BODNZT,16,1), XLBOBB_MASK, NOPOWER4, { BI } },
-{ "bdnztlrl+",XLO(19,BODNZTP,16,1), XLBOBB_MASK, NOPOWER4, { BI } },
-{ "bdnzflr", XLO(19,BODNZF,16,0), XLBOBB_MASK, PPCCOM,	{ BI } },
-{ "bdnzflr-",XLO(19,BODNZF,16,0), XLBOBB_MASK, NOPOWER4, { BI } },
-{ "bdnzflr+",XLO(19,BODNZFP,16,0), XLBOBB_MASK, NOPOWER4, { BI } },
-{ "bdnzflrl",XLO(19,BODNZF,16,1), XLBOBB_MASK, PPCCOM,	{ BI } },
-{ "bdnzflrl-",XLO(19,BODNZF,16,1), XLBOBB_MASK, NOPOWER4, { BI } },
-{ "bdnzflrl+",XLO(19,BODNZFP,16,1), XLBOBB_MASK, NOPOWER4, { BI } },
-{ "bdztlr",  XLO(19,BODZT,16,0), XLBOBB_MASK, PPCCOM,	{ BI } },
-{ "bdztlr-", XLO(19,BODZT,16,0), XLBOBB_MASK, NOPOWER4,	{ BI } },
-{ "bdztlr+", XLO(19,BODZTP,16,0), XLBOBB_MASK, NOPOWER4, { BI } },
-{ "bdztlrl", XLO(19,BODZT,16,1), XLBOBB_MASK, PPCCOM,	{ BI } },
-{ "bdztlrl-",XLO(19,BODZT,16,1), XLBOBB_MASK, NOPOWER4,	{ BI } },
-{ "bdztlrl+",XLO(19,BODZTP,16,1), XLBOBB_MASK, NOPOWER4, { BI } },
-{ "bdzflr",  XLO(19,BODZF,16,0), XLBOBB_MASK, PPCCOM,	{ BI } },
-{ "bdzflr-", XLO(19,BODZF,16,0), XLBOBB_MASK, NOPOWER4,	{ BI } },
-{ "bdzflr+", XLO(19,BODZFP,16,0), XLBOBB_MASK, NOPOWER4, { BI } },
-{ "bdzflrl", XLO(19,BODZF,16,1), XLBOBB_MASK, PPCCOM,	{ BI } },
-{ "bdzflrl-",XLO(19,BODZF,16,1), XLBOBB_MASK, NOPOWER4,	{ BI } },
-{ "bdzflrl+",XLO(19,BODZFP,16,1), XLBOBB_MASK, NOPOWER4, { BI } },
-{ "bclr+",   XLYLK(19,16,1,0), XLYBB_MASK, PPCCOM,	{ BOE, BI } },
-{ "bclrl+",  XLYLK(19,16,1,1), XLYBB_MASK, PPCCOM,	{ BOE, BI } },
-{ "bclr-",   XLYLK(19,16,0,0), XLYBB_MASK, PPCCOM,	{ BOE, BI } },
-{ "bclrl-",  XLYLK(19,16,0,1), XLYBB_MASK, PPCCOM,	{ BOE, BI } },
-{ "bclr",    XLLK(19,16,0), XLBH_MASK,	PPCCOM,		{ BO, BI, BH } },
-{ "bclrl",   XLLK(19,16,1), XLBH_MASK,	PPCCOM,		{ BO, BI, BH } },
-{ "bcr",     XLLK(19,16,0), XLBB_MASK,	PWRCOM,		{ BO, BI } },
-{ "bcrl",    XLLK(19,16,1), XLBB_MASK,	PWRCOM,		{ BO, BI } },
-{ "bclre",   XLLK(19,17,0), XLBB_MASK,	BOOKE64,	{ BO, BI } },
-{ "bclrel",  XLLK(19,17,1), XLBB_MASK,	BOOKE64,	{ BO, BI } },
-
-{ "rfid",    XL(19,18),	0xffffffff,	PPC64,		{ 0 } },
-
-{ "crnot",   XL(19,33), XL_MASK,	PPCCOM,		{ BT, BA, BBA } },
-{ "crnor",   XL(19,33),	XL_MASK,	COM,		{ BT, BA, BB } },
-{ "rfmci",    X(19,38), 0xffffffff,	PPCRFMCI,	{ 0 } },
-
-{ "rfi",     XL(19,50),	0xffffffff,	COM,		{ 0 } },
-{ "rfci",    XL(19,51),	0xffffffff,	PPC403 | BOOKE,	{ 0 } },
-
-{ "rfsvc",   XL(19,82),	0xffffffff,	POWER,		{ 0 } },
-
-{ "crandc",  XL(19,129), XL_MASK,	COM,		{ BT, BA, BB } },
-
-{ "isync",   XL(19,150), 0xffffffff,	PPCCOM,		{ 0 } },
-{ "ics",     XL(19,150), 0xffffffff,	PWRCOM,		{ 0 } },
-
-{ "crclr",   XL(19,193), XL_MASK,	PPCCOM,		{ BT, BAT, BBA } },
-{ "crxor",   XL(19,193), XL_MASK,	COM,		{ BT, BA, BB } },
-
-{ "crnand",  XL(19,225), XL_MASK,	COM,		{ BT, BA, BB } },
-
-{ "crand",   XL(19,257), XL_MASK,	COM,		{ BT, BA, BB } },
-
-{ "hrfid",   XL(19,274), 0xffffffff,	POWER5 | CELL,	{ 0 } },
-
-{ "crset",   XL(19,289), XL_MASK,	PPCCOM,		{ BT, BAT, BBA } },
-{ "creqv",   XL(19,289), XL_MASK,	COM,		{ BT, BA, BB } },
-
-{ "doze",    XL(19,402), 0xffffffff,	POWER6,		{ 0 } },
-
-{ "crorc",   XL(19,417), XL_MASK,	COM,		{ BT, BA, BB } },
-
-{ "nap",     XL(19,434), 0xffffffff,	POWER6,		{ 0 } },
-
-{ "crmove",  XL(19,449), XL_MASK,	PPCCOM,		{ BT, BA, BBA } },
-{ "cror",    XL(19,449), XL_MASK,	COM,		{ BT, BA, BB } },
-
-{ "sleep",   XL(19,466), 0xffffffff,	POWER6,		{ 0 } },
-{ "rvwinkle", XL(19,498), 0xffffffff,	POWER6,		{ 0 } },
-
-{ "bctr",    XLO(19,BOU,528,0), XLBOBIBB_MASK, COM,	{ 0 } },
-{ "bctrl",   XLO(19,BOU,528,1), XLBOBIBB_MASK, COM,	{ 0 } },
-{ "bltctr",  XLOCB(19,BOT,CBLT,528,0),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bltctr-", XLOCB(19,BOT,CBLT,528,0),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bltctr-", XLOCB(19,BOTM4,CBLT,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bltctr+", XLOCB(19,BOTP,CBLT,528,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bltctr+", XLOCB(19,BOTP4,CBLT,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bltctrl", XLOCB(19,BOT,CBLT,528,1),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bltctrl-",XLOCB(19,BOT,CBLT,528,1),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bltctrl-",XLOCB(19,BOTM4,CBLT,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bltctrl+",XLOCB(19,BOTP,CBLT,528,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bltctrl+",XLOCB(19,BOTP4,CBLT,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bgtctr",  XLOCB(19,BOT,CBGT,528,0),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bgtctr-", XLOCB(19,BOT,CBGT,528,0),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bgtctr-", XLOCB(19,BOTM4,CBGT,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bgtctr+", XLOCB(19,BOTP,CBGT,528,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bgtctr+", XLOCB(19,BOTP4,CBGT,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bgtctrl", XLOCB(19,BOT,CBGT,528,1),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bgtctrl-",XLOCB(19,BOT,CBGT,528,1),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bgtctrl-",XLOCB(19,BOTM4,CBGT,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bgtctrl+",XLOCB(19,BOTP,CBGT,528,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bgtctrl+",XLOCB(19,BOTP4,CBGT,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "beqctr",  XLOCB(19,BOT,CBEQ,528,0),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "beqctr-", XLOCB(19,BOT,CBEQ,528,0),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "beqctr-", XLOCB(19,BOTM4,CBEQ,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "beqctr+", XLOCB(19,BOTP,CBEQ,528,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "beqctr+", XLOCB(19,BOTP4,CBEQ,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "beqctrl", XLOCB(19,BOT,CBEQ,528,1),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "beqctrl-",XLOCB(19,BOT,CBEQ,528,1),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "beqctrl-",XLOCB(19,BOTM4,CBEQ,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "beqctrl+",XLOCB(19,BOTP,CBEQ,528,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "beqctrl+",XLOCB(19,BOTP4,CBEQ,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bsoctr",  XLOCB(19,BOT,CBSO,528,0),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bsoctr-", XLOCB(19,BOT,CBSO,528,0),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bsoctr-", XLOCB(19,BOTM4,CBSO,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bsoctr+", XLOCB(19,BOTP,CBSO,528,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bsoctr+", XLOCB(19,BOTP4,CBSO,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bsoctrl", XLOCB(19,BOT,CBSO,528,1),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bsoctrl-",XLOCB(19,BOT,CBSO,528,1),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bsoctrl-",XLOCB(19,BOTM4,CBSO,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bsoctrl+",XLOCB(19,BOTP,CBSO,528,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bsoctrl+",XLOCB(19,BOTP4,CBSO,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bunctr",  XLOCB(19,BOT,CBSO,528,0),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bunctr-", XLOCB(19,BOT,CBSO,528,0),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bunctr-", XLOCB(19,BOTM4,CBSO,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bunctr+", XLOCB(19,BOTP,CBSO,528,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bunctr+", XLOCB(19,BOTP4,CBSO,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bunctrl", XLOCB(19,BOT,CBSO,528,1),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bunctrl-",XLOCB(19,BOT,CBSO,528,1),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bunctrl-",XLOCB(19,BOTM4,CBSO,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bunctrl+",XLOCB(19,BOTP,CBSO,528,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bunctrl+",XLOCB(19,BOTP4,CBSO,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bgectr",  XLOCB(19,BOF,CBLT,528,0),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bgectr-", XLOCB(19,BOF,CBLT,528,0),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bgectr-", XLOCB(19,BOFM4,CBLT,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bgectr+", XLOCB(19,BOFP,CBLT,528,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bgectr+", XLOCB(19,BOFP4,CBLT,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bgectrl", XLOCB(19,BOF,CBLT,528,1),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bgectrl-",XLOCB(19,BOF,CBLT,528,1),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bgectrl-",XLOCB(19,BOFM4,CBLT,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bgectrl+",XLOCB(19,BOFP,CBLT,528,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bgectrl+",XLOCB(19,BOFP4,CBLT,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnlctr",  XLOCB(19,BOF,CBLT,528,0),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bnlctr-", XLOCB(19,BOF,CBLT,528,0),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnlctr-", XLOCB(19,BOFM4,CBLT,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnlctr+", XLOCB(19,BOFP,CBLT,528,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnlctr+", XLOCB(19,BOFP4,CBLT,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnlctrl", XLOCB(19,BOF,CBLT,528,1),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bnlctrl-",XLOCB(19,BOF,CBLT,528,1),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnlctrl-",XLOCB(19,BOFM4,CBLT,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnlctrl+",XLOCB(19,BOFP,CBLT,528,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnlctrl+",XLOCB(19,BOFP4,CBLT,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "blectr",  XLOCB(19,BOF,CBGT,528,0),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "blectr-", XLOCB(19,BOF,CBGT,528,0),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "blectr-", XLOCB(19,BOFM4,CBGT,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "blectr+", XLOCB(19,BOFP,CBGT,528,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "blectr+", XLOCB(19,BOFP4,CBGT,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "blectrl", XLOCB(19,BOF,CBGT,528,1),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "blectrl-",XLOCB(19,BOF,CBGT,528,1),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "blectrl-",XLOCB(19,BOFM4,CBGT,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "blectrl+",XLOCB(19,BOFP,CBGT,528,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "blectrl+",XLOCB(19,BOFP4,CBGT,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bngctr",  XLOCB(19,BOF,CBGT,528,0),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bngctr-", XLOCB(19,BOF,CBGT,528,0),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bngctr-", XLOCB(19,BOFM4,CBGT,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bngctr+", XLOCB(19,BOFP,CBGT,528,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bngctr+", XLOCB(19,BOFP4,CBGT,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bngctrl", XLOCB(19,BOF,CBGT,528,1),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bngctrl-",XLOCB(19,BOF,CBGT,528,1),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bngctrl-",XLOCB(19,BOFM4,CBGT,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bngctrl+",XLOCB(19,BOFP,CBGT,528,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bngctrl+",XLOCB(19,BOFP4,CBGT,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnectr",  XLOCB(19,BOF,CBEQ,528,0),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bnectr-", XLOCB(19,BOF,CBEQ,528,0),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnectr-", XLOCB(19,BOFM4,CBEQ,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnectr+", XLOCB(19,BOFP,CBEQ,528,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnectr+", XLOCB(19,BOFP4,CBEQ,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnectrl", XLOCB(19,BOF,CBEQ,528,1),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bnectrl-",XLOCB(19,BOF,CBEQ,528,1),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnectrl-",XLOCB(19,BOFM4,CBEQ,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnectrl+",XLOCB(19,BOFP,CBEQ,528,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnectrl+",XLOCB(19,BOFP4,CBEQ,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnsctr",  XLOCB(19,BOF,CBSO,528,0),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bnsctr-", XLOCB(19,BOF,CBSO,528,0),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnsctr-", XLOCB(19,BOFM4,CBSO,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnsctr+", XLOCB(19,BOFP,CBSO,528,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnsctr+", XLOCB(19,BOFP4,CBSO,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnsctrl", XLOCB(19,BOF,CBSO,528,1),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bnsctrl-",XLOCB(19,BOF,CBSO,528,1),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnsctrl-",XLOCB(19,BOFM4,CBSO,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnsctrl+",XLOCB(19,BOFP,CBSO,528,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnsctrl+",XLOCB(19,BOFP4,CBSO,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnuctr",  XLOCB(19,BOF,CBSO,528,0),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bnuctr-", XLOCB(19,BOF,CBSO,528,0),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnuctr-", XLOCB(19,BOFM4,CBSO,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnuctr+", XLOCB(19,BOFP,CBSO,528,0), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnuctr+", XLOCB(19,BOFP4,CBSO,528,0), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnuctrl", XLOCB(19,BOF,CBSO,528,1),  XLBOCBBB_MASK, PPCCOM,	{ CR } },
-{ "bnuctrl-",XLOCB(19,BOF,CBSO,528,1),  XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnuctrl-",XLOCB(19,BOFM4,CBSO,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "bnuctrl+",XLOCB(19,BOFP,CBSO,528,1), XLBOCBBB_MASK, NOPOWER4, { CR } },
-{ "bnuctrl+",XLOCB(19,BOFP4,CBSO,528,1), XLBOCBBB_MASK, POWER4, { CR } },
-{ "btctr",   XLO(19,BOT,528,0),  XLBOBB_MASK, PPCCOM,	{ BI } },
-{ "btctr-",  XLO(19,BOT,528,0),  XLBOBB_MASK, NOPOWER4,	{ BI } },
-{ "btctr-",  XLO(19,BOTM4,528,0), XLBOBB_MASK, POWER4, { BI } },
-{ "btctr+",  XLO(19,BOTP,528,0), XLBOBB_MASK, NOPOWER4,	{ BI } },
-{ "btctr+",  XLO(19,BOTP4,528,0), XLBOBB_MASK, POWER4, { BI } },
-{ "btctrl",  XLO(19,BOT,528,1),  XLBOBB_MASK, PPCCOM,	{ BI } },
-{ "btctrl-", XLO(19,BOT,528,1),  XLBOBB_MASK, NOPOWER4,	{ BI } },
-{ "btctrl-", XLO(19,BOTM4,528,1), XLBOBB_MASK, POWER4, { BI } },
-{ "btctrl+", XLO(19,BOTP,528,1), XLBOBB_MASK, NOPOWER4,	{ BI } },
-{ "btctrl+", XLO(19,BOTP4,528,1), XLBOBB_MASK, POWER4, { BI } },
-{ "bfctr",   XLO(19,BOF,528,0),  XLBOBB_MASK, PPCCOM,	{ BI } },
-{ "bfctr-",  XLO(19,BOF,528,0),  XLBOBB_MASK, NOPOWER4, { BI } },
-{ "bfctr-",  XLO(19,BOFM4,528,0), XLBOBB_MASK, POWER4, { BI } },
-{ "bfctr+",  XLO(19,BOFP,528,0), XLBOBB_MASK, NOPOWER4, { BI } },
-{ "bfctr+",  XLO(19,BOFP4,528,0), XLBOBB_MASK, POWER4, { BI } },
-{ "bfctrl",  XLO(19,BOF,528,1),  XLBOBB_MASK, PPCCOM,	{ BI } },
-{ "bfctrl-", XLO(19,BOF,528,1),  XLBOBB_MASK, NOPOWER4, { BI } },
-{ "bfctrl-", XLO(19,BOFM4,528,1), XLBOBB_MASK, POWER4, { BI } },
-{ "bfctrl+", XLO(19,BOFP,528,1), XLBOBB_MASK, NOPOWER4, { BI } },
-{ "bfctrl+", XLO(19,BOFP4,528,1), XLBOBB_MASK, POWER4, { BI } },
-{ "bcctr-",  XLYLK(19,528,0,0),  XLYBB_MASK,  PPCCOM,	{ BOE, BI } },
-{ "bcctr+",  XLYLK(19,528,1,0),  XLYBB_MASK,  PPCCOM,	{ BOE, BI } },
-{ "bcctrl-", XLYLK(19,528,0,1),  XLYBB_MASK,  PPCCOM,	{ BOE, BI } },
-{ "bcctrl+", XLYLK(19,528,1,1),  XLYBB_MASK,  PPCCOM,	{ BOE, BI } },
-{ "bcctr",   XLLK(19,528,0),     XLBH_MASK,   PPCCOM,	{ BO, BI, BH } },
-{ "bcctrl",  XLLK(19,528,1),     XLBH_MASK,   PPCCOM,	{ BO, BI, BH } },
-{ "bcc",     XLLK(19,528,0),     XLBB_MASK,   PWRCOM,	{ BO, BI } },
-{ "bccl",    XLLK(19,528,1),     XLBB_MASK,   PWRCOM,	{ BO, BI } },
-{ "bcctre",  XLLK(19,529,0),     XLBB_MASK,   BOOKE64,	{ BO, BI } },
-{ "bcctrel", XLLK(19,529,1),     XLBB_MASK,   BOOKE64,	{ BO, BI } },
-
-{ "rlwimi",  M(20,0),	M_MASK,		PPCCOM,		{ RA,RS,SH,MBE,ME } },
-{ "rlimi",   M(20,0),	M_MASK,		PWRCOM,		{ RA,RS,SH,MBE,ME } },
-
-{ "rlwimi.", M(20,1),	M_MASK,		PPCCOM,		{ RA,RS,SH,MBE,ME } },
-{ "rlimi.",  M(20,1),	M_MASK,		PWRCOM,		{ RA,RS,SH,MBE,ME } },
-
-{ "rotlwi",  MME(21,31,0), MMBME_MASK,	PPCCOM,		{ RA, RS, SH } },
-{ "clrlwi",  MME(21,31,0), MSHME_MASK,	PPCCOM,		{ RA, RS, MB } },
-{ "rlwinm",  M(21,0),	M_MASK,		PPCCOM,		{ RA,RS,SH,MBE,ME } },
-{ "rlinm",   M(21,0),	M_MASK,		PWRCOM,		{ RA,RS,SH,MBE,ME } },
-{ "rotlwi.", MME(21,31,1), MMBME_MASK,	PPCCOM,		{ RA,RS,SH } },
-{ "clrlwi.", MME(21,31,1), MSHME_MASK,	PPCCOM,		{ RA, RS, MB } },
-{ "rlwinm.", M(21,1),	M_MASK,		PPCCOM,		{ RA,RS,SH,MBE,ME } },
-{ "rlinm.",  M(21,1),	M_MASK,		PWRCOM,		{ RA,RS,SH,MBE,ME } },
-
-{ "rlmi",    M(22,0),	M_MASK,		M601,		{ RA,RS,RB,MBE,ME } },
-{ "rlmi.",   M(22,1),	M_MASK,		M601,		{ RA,RS,RB,MBE,ME } },
-
-{ "be",	     B(22,0,0),	B_MASK,		BOOKE64,	{ LI } },
-{ "bel",     B(22,0,1),	B_MASK,		BOOKE64,	{ LI } },
-{ "bea",     B(22,1,0),	B_MASK,		BOOKE64,	{ LIA } },
-{ "bela",    B(22,1,1),	B_MASK,		BOOKE64,	{ LIA } },
-
-{ "rotlw",   MME(23,31,0), MMBME_MASK,	PPCCOM,		{ RA, RS, RB } },
-{ "rlwnm",   M(23,0),	M_MASK,		PPCCOM,		{ RA,RS,RB,MBE,ME } },
-{ "rlnm",    M(23,0),	M_MASK,		PWRCOM,		{ RA,RS,RB,MBE,ME } },
-{ "rotlw.",  MME(23,31,1), MMBME_MASK,	PPCCOM,		{ RA, RS, RB } },
-{ "rlwnm.",  M(23,1),	M_MASK,		PPCCOM,		{ RA,RS,RB,MBE,ME } },
-{ "rlnm.",   M(23,1),	M_MASK,		PWRCOM,		{ RA,RS,RB,MBE,ME } },
-
-{ "nop",     OP(24),	0xffffffff,	PPCCOM,		{ 0 } },
-{ "ori",     OP(24),	OP_MASK,	PPCCOM,		{ RA, RS, UI } },
-{ "oril",    OP(24),	OP_MASK,	PWRCOM,		{ RA, RS, UI } },
-
-{ "oris",    OP(25),	OP_MASK,	PPCCOM,		{ RA, RS, UI } },
-{ "oriu",    OP(25),	OP_MASK,	PWRCOM,		{ RA, RS, UI } },
-
-{ "xori",    OP(26),	OP_MASK,	PPCCOM,		{ RA, RS, UI } },
-{ "xoril",   OP(26),	OP_MASK,	PWRCOM,		{ RA, RS, UI } },
-
-{ "xoris",   OP(27),	OP_MASK,	PPCCOM,		{ RA, RS, UI } },
-{ "xoriu",   OP(27),	OP_MASK,	PWRCOM,		{ RA, RS, UI } },
-
-{ "andi.",   OP(28),	OP_MASK,	PPCCOM,		{ RA, RS, UI } },
-{ "andil.",  OP(28),	OP_MASK,	PWRCOM,		{ RA, RS, UI } },
-
-{ "andis.",  OP(29),	OP_MASK,	PPCCOM,		{ RA, RS, UI } },
-{ "andiu.",  OP(29),	OP_MASK,	PWRCOM,		{ RA, RS, UI } },
-
-{ "rotldi",  MD(30,0,0), MDMB_MASK,	PPC64,		{ RA, RS, SH6 } },
-{ "clrldi",  MD(30,0,0), MDSH_MASK,	PPC64,		{ RA, RS, MB6 } },
-{ "rldicl",  MD(30,0,0), MD_MASK,	PPC64,		{ RA, RS, SH6, MB6 } },
-{ "rotldi.", MD(30,0,1), MDMB_MASK,	PPC64,		{ RA, RS, SH6 } },
-{ "clrldi.", MD(30,0,1), MDSH_MASK,	PPC64,		{ RA, RS, MB6 } },
-{ "rldicl.", MD(30,0,1), MD_MASK,	PPC64,		{ RA, RS, SH6, MB6 } },
-
-{ "rldicr",  MD(30,1,0), MD_MASK,	PPC64,		{ RA, RS, SH6, ME6 } },
-{ "rldicr.", MD(30,1,1), MD_MASK,	PPC64,		{ RA, RS, SH6, ME6 } },
-
-{ "rldic",   MD(30,2,0), MD_MASK,	PPC64,		{ RA, RS, SH6, MB6 } },
-{ "rldic.",  MD(30,2,1), MD_MASK,	PPC64,		{ RA, RS, SH6, MB6 } },
-
-{ "rldimi",  MD(30,3,0), MD_MASK,	PPC64,		{ RA, RS, SH6, MB6 } },
-{ "rldimi.", MD(30,3,1), MD_MASK,	PPC64,		{ RA, RS, SH6, MB6 } },
-
-{ "rotld",   MDS(30,8,0), MDSMB_MASK,	PPC64,		{ RA, RS, RB } },
-{ "rldcl",   MDS(30,8,0), MDS_MASK,	PPC64,		{ RA, RS, RB, MB6 } },
-{ "rotld.",  MDS(30,8,1), MDSMB_MASK,	PPC64,		{ RA, RS, RB } },
-{ "rldcl.",  MDS(30,8,1), MDS_MASK,	PPC64,		{ RA, RS, RB, MB6 } },
-
-{ "rldcr",   MDS(30,9,0), MDS_MASK,	PPC64,		{ RA, RS, RB, ME6 } },
-{ "rldcr.",  MDS(30,9,1), MDS_MASK,	PPC64,		{ RA, RS, RB, ME6 } },
-
-{ "cmpw",    XOPL(31,0,0), XCMPL_MASK, PPCCOM,		{ OBF, RA, RB } },
-{ "cmpd",    XOPL(31,0,1), XCMPL_MASK, PPC64,		{ OBF, RA, RB } },
-{ "cmp",     X(31,0),	XCMP_MASK,	PPC,		{ BF, L, RA, RB } },
-{ "cmp",     X(31,0),	XCMPL_MASK,	PWRCOM,		{ BF, RA, RB } },
-
-{ "twlgt",   XTO(31,4,TOLGT), XTO_MASK, PPCCOM,		{ RA, RB } },
-{ "tlgt",    XTO(31,4,TOLGT), XTO_MASK, PWRCOM,		{ RA, RB } },
-{ "twllt",   XTO(31,4,TOLLT), XTO_MASK, PPCCOM,		{ RA, RB } },
-{ "tllt",    XTO(31,4,TOLLT), XTO_MASK, PWRCOM,		{ RA, RB } },
-{ "tweq",    XTO(31,4,TOEQ), XTO_MASK,	PPCCOM,		{ RA, RB } },
-{ "teq",     XTO(31,4,TOEQ), XTO_MASK,	PWRCOM,		{ RA, RB } },
-{ "twlge",   XTO(31,4,TOLGE), XTO_MASK, PPCCOM,		{ RA, RB } },
-{ "tlge",    XTO(31,4,TOLGE), XTO_MASK, PWRCOM,		{ RA, RB } },
-{ "twlnl",   XTO(31,4,TOLNL), XTO_MASK, PPCCOM,		{ RA, RB } },
-{ "tlnl",    XTO(31,4,TOLNL), XTO_MASK, PWRCOM,		{ RA, RB } },
-{ "twlle",   XTO(31,4,TOLLE), XTO_MASK, PPCCOM,		{ RA, RB } },
-{ "tlle",    XTO(31,4,TOLLE), XTO_MASK, PWRCOM,		{ RA, RB } },
-{ "twlng",   XTO(31,4,TOLNG), XTO_MASK, PPCCOM,		{ RA, RB } },
-{ "tlng",    XTO(31,4,TOLNG), XTO_MASK, PWRCOM,		{ RA, RB } },
-{ "twgt",    XTO(31,4,TOGT), XTO_MASK,	PPCCOM,		{ RA, RB } },
-{ "tgt",     XTO(31,4,TOGT), XTO_MASK,	PWRCOM,		{ RA, RB } },
-{ "twge",    XTO(31,4,TOGE), XTO_MASK,	PPCCOM,		{ RA, RB } },
-{ "tge",     XTO(31,4,TOGE), XTO_MASK,	PWRCOM,		{ RA, RB } },
-{ "twnl",    XTO(31,4,TONL), XTO_MASK,	PPCCOM,		{ RA, RB } },
-{ "tnl",     XTO(31,4,TONL), XTO_MASK,	PWRCOM,		{ RA, RB } },
-{ "twlt",    XTO(31,4,TOLT), XTO_MASK,	PPCCOM,		{ RA, RB } },
-{ "tlt",     XTO(31,4,TOLT), XTO_MASK,	PWRCOM,		{ RA, RB } },
-{ "twle",    XTO(31,4,TOLE), XTO_MASK,	PPCCOM,		{ RA, RB } },
-{ "tle",     XTO(31,4,TOLE), XTO_MASK,	PWRCOM,		{ RA, RB } },
-{ "twng",    XTO(31,4,TONG), XTO_MASK,	PPCCOM,		{ RA, RB } },
-{ "tng",     XTO(31,4,TONG), XTO_MASK,	PWRCOM,		{ RA, RB } },
-{ "twne",    XTO(31,4,TONE), XTO_MASK,	PPCCOM,		{ RA, RB } },
-{ "tne",     XTO(31,4,TONE), XTO_MASK,	PWRCOM,		{ RA, RB } },
-{ "trap",    XTO(31,4,TOU), 0xffffffff,	PPCCOM,		{ 0 } },
-{ "tw",      X(31,4),	X_MASK,		PPCCOM,		{ TO, RA, RB } },
-{ "t",       X(31,4),	X_MASK,		PWRCOM,		{ TO, RA, RB } },
-
-{ "subfc",   XO(31,8,0,0), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "sf",      XO(31,8,0,0), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "subc",    XO(31,8,0,0), XO_MASK,	PPC,		{ RT, RB, RA } },
-{ "subfc.",  XO(31,8,0,1), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "sf.",     XO(31,8,0,1), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "subc.",   XO(31,8,0,1), XO_MASK,	PPCCOM,		{ RT, RB, RA } },
-{ "subfco",  XO(31,8,1,0), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "sfo",     XO(31,8,1,0), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "subco",   XO(31,8,1,0), XO_MASK,	PPC,		{ RT, RB, RA } },
-{ "subfco.", XO(31,8,1,1), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "sfo.",    XO(31,8,1,1), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "subco.",  XO(31,8,1,1), XO_MASK,	PPC,		{ RT, RB, RA } },
-
-{ "mulhdu",  XO(31,9,0,0), XO_MASK,	PPC64,		{ RT, RA, RB } },
-{ "mulhdu.", XO(31,9,0,1), XO_MASK,	PPC64,		{ RT, RA, RB } },
-
-{ "addc",    XO(31,10,0,0), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "a",       XO(31,10,0,0), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "addc.",   XO(31,10,0,1), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "a.",      XO(31,10,0,1), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "addco",   XO(31,10,1,0), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "ao",      XO(31,10,1,0), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "addco.",  XO(31,10,1,1), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "ao.",     XO(31,10,1,1), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-
-{ "mulhwu",  XO(31,11,0,0), XO_MASK,	PPC,		{ RT, RA, RB } },
-{ "mulhwu.", XO(31,11,0,1), XO_MASK,	PPC,		{ RT, RA, RB } },
-
-{ "isellt",  X(31,15),      X_MASK,	PPCISEL,	{ RT, RA, RB } },
-{ "iselgt",  X(31,47),      X_MASK,	PPCISEL,	{ RT, RA, RB } },
-{ "iseleq",  X(31,79),      X_MASK,	PPCISEL,	{ RT, RA, RB } },
-{ "isel",    XISEL(31,15),  XISEL_MASK,	PPCISEL,	{ RT, RA, RB, CRB } },
-
-{ "mfocrf",  XFXM(31,19,0,1), XFXFXM_MASK, COM,		{ RT, FXM } },
-{ "mfcr",    X(31,19),	XRARB_MASK,	NOPOWER4 | COM,	{ RT } },
-{ "mfcr",    X(31,19),	XFXFXM_MASK,	POWER4,		{ RT, FXM4 } },
-
-{ "lwarx",   X(31,20),	XEH_MASK,	PPC,		{ RT, RA0, RB, EH } },
-
-{ "ldx",     X(31,21),	X_MASK,		PPC64,		{ RT, RA0, RB } },
-
-{ "icbt",    X(31,22),	X_MASK,		BOOKE|PPCE300,	{ CT, RA, RB } },
-{ "icbt",    X(31,262),	XRT_MASK,	PPC403,		{ RA, RB } },
-
-{ "lwzx",    X(31,23),	X_MASK,		PPCCOM,		{ RT, RA0, RB } },
-{ "lx",      X(31,23),	X_MASK,		PWRCOM,		{ RT, RA, RB } },
-
-{ "slw",     XRC(31,24,0), X_MASK,	PPCCOM,		{ RA, RS, RB } },
-{ "sl",      XRC(31,24,0), X_MASK,	PWRCOM,		{ RA, RS, RB } },
-{ "slw.",    XRC(31,24,1), X_MASK,	PPCCOM,		{ RA, RS, RB } },
-{ "sl.",     XRC(31,24,1), X_MASK,	PWRCOM,		{ RA, RS, RB } },
-
-{ "cntlzw",  XRC(31,26,0), XRB_MASK,	PPCCOM,		{ RA, RS } },
-{ "cntlz",   XRC(31,26,0), XRB_MASK,	PWRCOM,		{ RA, RS } },
-{ "cntlzw.", XRC(31,26,1), XRB_MASK,	PPCCOM,		{ RA, RS } },
-{ "cntlz.",  XRC(31,26,1), XRB_MASK, 	PWRCOM,		{ RA, RS } },
-
-{ "sld",     XRC(31,27,0), X_MASK,	PPC64,		{ RA, RS, RB } },
-{ "sld.",    XRC(31,27,1), X_MASK,	PPC64,		{ RA, RS, RB } },
-
-{ "and",     XRC(31,28,0), X_MASK,	COM,		{ RA, RS, RB } },
-{ "and.",    XRC(31,28,1), X_MASK,	COM,		{ RA, RS, RB } },
-
-{ "maskg",   XRC(31,29,0), X_MASK,	M601,		{ RA, RS, RB } },
-{ "maskg.",  XRC(31,29,1), X_MASK,	M601,		{ RA, RS, RB } },
-
-{ "icbte",   X(31,30),	X_MASK,		BOOKE64,	{ CT, RA, RB } },
-
-{ "lwzxe",   X(31,31),	X_MASK,		BOOKE64,	{ RT, RA0, RB } },
-
-{ "cmplw",   XOPL(31,32,0), XCMPL_MASK, PPCCOM,	{ OBF, RA, RB } },
-{ "cmpld",   XOPL(31,32,1), XCMPL_MASK, PPC64,		{ OBF, RA, RB } },
-{ "cmpl",    X(31,32),	XCMP_MASK,	 PPC,		{ BF, L, RA, RB } },
-{ "cmpl",    X(31,32),	XCMPL_MASK,	 PWRCOM,	{ BF, RA, RB } },
-
-{ "subf",    XO(31,40,0,0), XO_MASK,	PPC,		{ RT, RA, RB } },
-{ "sub",     XO(31,40,0,0), XO_MASK,	PPC,		{ RT, RB, RA } },
-{ "subf.",   XO(31,40,0,1), XO_MASK,	PPC,		{ RT, RA, RB } },
-{ "sub.",    XO(31,40,0,1), XO_MASK,	PPC,		{ RT, RB, RA } },
-{ "subfo",   XO(31,40,1,0), XO_MASK,	PPC,		{ RT, RA, RB } },
-{ "subo",    XO(31,40,1,0), XO_MASK,	PPC,		{ RT, RB, RA } },
-{ "subfo.",  XO(31,40,1,1), XO_MASK,	PPC,		{ RT, RA, RB } },
-{ "subo.",   XO(31,40,1,1), XO_MASK,	PPC,		{ RT, RB, RA } },
-
-{ "ldux",    X(31,53),	X_MASK,		PPC64,		{ RT, RAL, RB } },
-
-{ "dcbst",   X(31,54),	XRT_MASK,	PPC,		{ RA, RB } },
-
-{ "lwzux",   X(31,55),	X_MASK,		PPCCOM,		{ RT, RAL, RB } },
-{ "lux",     X(31,55),	X_MASK,		PWRCOM,		{ RT, RA, RB } },
-
-{ "dcbste",  X(31,62),	XRT_MASK,	BOOKE64,	{ RA, RB } },
-
-{ "lwzuxe",  X(31,63),	X_MASK,		BOOKE64,	{ RT, RAL, RB } },
-
-{ "cntlzd",  XRC(31,58,0), XRB_MASK,	PPC64,		{ RA, RS } },
-{ "cntlzd.", XRC(31,58,1), XRB_MASK,	PPC64,		{ RA, RS } },
-
-{ "andc",    XRC(31,60,0), X_MASK,	COM,		{ RA, RS, RB } },
-{ "andc.",   XRC(31,60,1), X_MASK,	COM,		{ RA, RS, RB } },
-
-{ "tdlgt",   XTO(31,68,TOLGT), XTO_MASK, PPC64,		{ RA, RB } },
-{ "tdllt",   XTO(31,68,TOLLT), XTO_MASK, PPC64,		{ RA, RB } },
-{ "tdeq",    XTO(31,68,TOEQ), XTO_MASK,  PPC64,		{ RA, RB } },
-{ "tdlge",   XTO(31,68,TOLGE), XTO_MASK, PPC64,		{ RA, RB } },
-{ "tdlnl",   XTO(31,68,TOLNL), XTO_MASK, PPC64,		{ RA, RB } },
-{ "tdlle",   XTO(31,68,TOLLE), XTO_MASK, PPC64,		{ RA, RB } },
-{ "tdlng",   XTO(31,68,TOLNG), XTO_MASK, PPC64,		{ RA, RB } },
-{ "tdgt",    XTO(31,68,TOGT), XTO_MASK,  PPC64,		{ RA, RB } },
-{ "tdge",    XTO(31,68,TOGE), XTO_MASK,  PPC64,		{ RA, RB } },
-{ "tdnl",    XTO(31,68,TONL), XTO_MASK,  PPC64,		{ RA, RB } },
-{ "tdlt",    XTO(31,68,TOLT), XTO_MASK,  PPC64,		{ RA, RB } },
-{ "tdle",    XTO(31,68,TOLE), XTO_MASK,  PPC64,		{ RA, RB } },
-{ "tdng",    XTO(31,68,TONG), XTO_MASK,  PPC64,		{ RA, RB } },
-{ "tdne",    XTO(31,68,TONE), XTO_MASK,  PPC64,		{ RA, RB } },
-{ "td",	     X(31,68),	X_MASK,		 PPC64,		{ TO, RA, RB } },
-
-{ "mulhd",   XO(31,73,0,0), XO_MASK,	 PPC64,		{ RT, RA, RB } },
-{ "mulhd.",  XO(31,73,0,1), XO_MASK,	 PPC64,		{ RT, RA, RB } },
-
-{ "mulhw",   XO(31,75,0,0), XO_MASK,	PPC,		{ RT, RA, RB } },
-{ "mulhw.",  XO(31,75,0,1), XO_MASK,	PPC,		{ RT, RA, RB } },
-
-{ "dlmzb",   XRC(31,78,0),  X_MASK,	PPC403|PPC440,	{ RA, RS, RB } },
-{ "dlmzb.",  XRC(31,78,1),  X_MASK,	PPC403|PPC440,	{ RA, RS, RB } },
-
-{ "mtsrd",   X(31,82),	XRB_MASK|(1<<20), PPC64,	{ SR, RS } },
-
-{ "mfmsr",   X(31,83),	XRARB_MASK,	COM,		{ RT } },
-
-{ "ldarx",   X(31,84),	XEH_MASK,	PPC64,		{ RT, RA0, RB, EH } },
-
-{ "dcbfl",   XOPL(31,86,1), XRT_MASK,	POWER5,		{ RA, RB } },
-{ "dcbf",    X(31,86),	XLRT_MASK,	PPC,		{ RA, RB, L } },
-
-{ "lbzx",    X(31,87),	X_MASK,		COM,		{ RT, RA0, RB } },
-
-{ "dcbfe",   X(31,94),	XRT_MASK,	BOOKE64,	{ RA, RB } },
-
-{ "lbzxe",   X(31,95),	X_MASK,		BOOKE64,	{ RT, RA0, RB } },
-
-{ "neg",     XO(31,104,0,0), XORB_MASK,	COM,		{ RT, RA } },
-{ "neg.",    XO(31,104,0,1), XORB_MASK,	COM,		{ RT, RA } },
-{ "nego",    XO(31,104,1,0), XORB_MASK,	COM,		{ RT, RA } },
-{ "nego.",   XO(31,104,1,1), XORB_MASK,	COM,		{ RT, RA } },
-
-{ "mul",     XO(31,107,0,0), XO_MASK,	M601,		{ RT, RA, RB } },
-{ "mul.",    XO(31,107,0,1), XO_MASK,	M601,		{ RT, RA, RB } },
-{ "mulo",    XO(31,107,1,0), XO_MASK,	M601,		{ RT, RA, RB } },
-{ "mulo.",   XO(31,107,1,1), XO_MASK,	M601,		{ RT, RA, RB } },
-
-{ "mtsrdin", X(31,114),	XRA_MASK,	PPC64,		{ RS, RB } },
-
-{ "clf",     X(31,118), XTO_MASK,	POWER,		{ RA, RB } },
-
-{ "lbzux",   X(31,119),	X_MASK,		COM,		{ RT, RAL, RB } },
-
-{ "popcntb", X(31,122), XRB_MASK,	POWER5,		{ RA, RS } },
-{ "popcntw", X(31,378), XRB_MASK,       POWER7,         { RA, RS } },
-{ "popcntd", X(31,506), XRB_MASK,       POWER7,         { RA, RS } },
-
-{ "cnttzw",  XRC(31,538,0), XRB_MASK,   POWER9,         { RA, RS } },
-{ "cnttzw.", XRC(31,538,1), XRB_MASK,   POWER9,         { RA, RS } },
-{ "cnttzd",  XRC(31,570,0), XRB_MASK,   POWER9,         { RA, RS } },
-{ "cnttzd.", XRC(31,570,1), XRB_MASK,   POWER9,         { RA, RS } },
-
-{ "not",     XRC(31,124,0), X_MASK,	COM,		{ RA, RS, RBS } },
-{ "nor",     XRC(31,124,0), X_MASK,	COM,		{ RA, RS, RB } },
-{ "not.",    XRC(31,124,1), X_MASK,	COM,		{ RA, RS, RBS } },
-{ "nor.",    XRC(31,124,1), X_MASK,	COM,		{ RA, RS, RB } },
-
-{ "lwarxe",  X(31,126),	X_MASK,		BOOKE64,	{ RT, RA0, RB } },
-
-{ "lbzuxe",  X(31,127),	X_MASK,		BOOKE64,	{ RT, RAL, RB } },
-
-{ "wrtee",   X(31,131),	XRARB_MASK,	PPC403 | BOOKE,	{ RS } },
-
-{ "dcbtstls",X(31,134),	X_MASK,		PPCCHLK,	{ CT, RA, RB }},
-
-{ "subfe",   XO(31,136,0,0), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "sfe",     XO(31,136,0,0), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "subfe.",  XO(31,136,0,1), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "sfe.",    XO(31,136,0,1), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "subfeo",  XO(31,136,1,0), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "sfeo",    XO(31,136,1,0), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "subfeo.", XO(31,136,1,1), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "sfeo.",   XO(31,136,1,1), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-
-{ "adde",    XO(31,138,0,0), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "ae",      XO(31,138,0,0), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "adde.",   XO(31,138,0,1), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "ae.",     XO(31,138,0,1), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "addeo",   XO(31,138,1,0), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "aeo",     XO(31,138,1,0), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "addeo.",  XO(31,138,1,1), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "aeo.",    XO(31,138,1,1), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-
-{ "dcbtstlse",X(31,142),X_MASK,		PPCCHLK64,	{ CT, RA, RB }},
-
-{ "mtocrf",  XFXM(31,144,0,1), XFXFXM_MASK, COM,	{ FXM, RS } },
-{ "mtcr",    XFXM(31,144,0xff,0), XRARB_MASK, COM,	{ RS }},
-{ "mtcrf",   X(31,144),	XFXFXM_MASK,	COM,		{ FXM, RS } },
-
-{ "mtmsr",   X(31,146),	XRARB_MASK,	COM,		{ RS } },
-
-{ "stdx",    X(31,149), X_MASK,		PPC64,		{ RS, RA0, RB } },
-
-{ "stwcx.",  XRC(31,150,1), X_MASK,	PPC,		{ RS, RA0, RB } },
-
-{ "stwx",    X(31,151), X_MASK,		PPCCOM,		{ RS, RA0, RB } },
-{ "stx",     X(31,151), X_MASK,		PWRCOM,		{ RS, RA, RB } },
-
-{ "stwcxe.", XRC(31,158,1), X_MASK,	BOOKE64,	{ RS, RA0, RB } },
-
-{ "stwxe",   X(31,159), X_MASK,		BOOKE64,	{ RS, RA0, RB } },
-
-{ "slq",     XRC(31,152,0), X_MASK,	M601,		{ RA, RS, RB } },
-{ "slq.",    XRC(31,152,1), X_MASK,	M601,		{ RA, RS, RB } },
-
-{ "sle",     XRC(31,153,0), X_MASK,	M601,		{ RA, RS, RB } },
-{ "sle.",    XRC(31,153,1), X_MASK,	M601,		{ RA, RS, RB } },
-
-{ "prtyw",   X(31,154),	XRB_MASK,	POWER6,		{ RA, RS } },
-
-{ "wrteei",  X(31,163),	XE_MASK,	PPC403 | BOOKE,	{ E } },
-
-{ "dcbtls",  X(31,166),	X_MASK,		PPCCHLK,	{ CT, RA, RB }},
-{ "dcbtlse", X(31,174),	X_MASK,		PPCCHLK64,	{ CT, RA, RB }},
-
-{ "mtmsrd",  X(31,178),	XRLARB_MASK,	PPC64,		{ RS, A_L } },
-
-{ "stdux",   X(31,181),	X_MASK,		PPC64,		{ RS, RAS, RB } },
-
-{ "stwux",   X(31,183),	X_MASK,		PPCCOM,		{ RS, RAS, RB } },
-{ "stux",    X(31,183),	X_MASK,		PWRCOM,		{ RS, RA0, RB } },
-
-{ "sliq",    XRC(31,184,0), X_MASK,	M601,		{ RA, RS, SH } },
-{ "sliq.",   XRC(31,184,1), X_MASK,	M601,		{ RA, RS, SH } },
-
-{ "prtyd",   X(31,186),	XRB_MASK,	POWER6,		{ RA, RS } },
-
-{ "stwuxe",  X(31,191),	X_MASK,		BOOKE64,	{ RS, RAS, RB } },
-
-{ "subfze",  XO(31,200,0,0), XORB_MASK, PPCCOM,		{ RT, RA } },
-{ "sfze",    XO(31,200,0,0), XORB_MASK, PWRCOM,		{ RT, RA } },
-{ "subfze.", XO(31,200,0,1), XORB_MASK, PPCCOM,		{ RT, RA } },
-{ "sfze.",   XO(31,200,0,1), XORB_MASK, PWRCOM,		{ RT, RA } },
-{ "subfzeo", XO(31,200,1,0), XORB_MASK, PPCCOM,		{ RT, RA } },
-{ "sfzeo",   XO(31,200,1,0), XORB_MASK, PWRCOM,		{ RT, RA } },
-{ "subfzeo.",XO(31,200,1,1), XORB_MASK, PPCCOM,		{ RT, RA } },
-{ "sfzeo.",  XO(31,200,1,1), XORB_MASK, PWRCOM,		{ RT, RA } },
-
-{ "addze",   XO(31,202,0,0), XORB_MASK, PPCCOM,		{ RT, RA } },
-{ "aze",     XO(31,202,0,0), XORB_MASK, PWRCOM,		{ RT, RA } },
-{ "addze.",  XO(31,202,0,1), XORB_MASK, PPCCOM,		{ RT, RA } },
-{ "aze.",    XO(31,202,0,1), XORB_MASK, PWRCOM,		{ RT, RA } },
-{ "addzeo",  XO(31,202,1,0), XORB_MASK, PPCCOM,		{ RT, RA } },
-{ "azeo",    XO(31,202,1,0), XORB_MASK, PWRCOM,		{ RT, RA } },
-{ "addzeo.", XO(31,202,1,1), XORB_MASK, PPCCOM,		{ RT, RA } },
-{ "azeo.",   XO(31,202,1,1), XORB_MASK, PWRCOM,		{ RT, RA } },
-
-{ "mtsr",    X(31,210),	XRB_MASK|(1<<20), COM32,	{ SR, RS } },
-
-{ "stdcx.",  XRC(31,214,1), X_MASK,	PPC64,		{ RS, RA0, RB } },
-
-{ "stbx",    X(31,215),	X_MASK,		COM,		{ RS, RA0, RB } },
-
-{ "sllq",    XRC(31,216,0), X_MASK,	M601,		{ RA, RS, RB } },
-{ "sllq.",   XRC(31,216,1), X_MASK,	M601,		{ RA, RS, RB } },
-
-{ "sleq",    XRC(31,217,0), X_MASK,	M601,		{ RA, RS, RB } },
-{ "sleq.",   XRC(31,217,1), X_MASK,	M601,		{ RA, RS, RB } },
-
-{ "stbxe",   X(31,223),	X_MASK,		BOOKE64,	{ RS, RA0, RB } },
-
-{ "icblc",   X(31,230),	X_MASK,		PPCCHLK,	{ CT, RA, RB }},
-
-{ "subfme",  XO(31,232,0,0), XORB_MASK, PPCCOM,		{ RT, RA } },
-{ "sfme",    XO(31,232,0,0), XORB_MASK, PWRCOM,		{ RT, RA } },
-{ "subfme.", XO(31,232,0,1), XORB_MASK, PPCCOM,		{ RT, RA } },
-{ "sfme.",   XO(31,232,0,1), XORB_MASK, PWRCOM,		{ RT, RA } },
-{ "subfmeo", XO(31,232,1,0), XORB_MASK, PPCCOM,		{ RT, RA } },
-{ "sfmeo",   XO(31,232,1,0), XORB_MASK, PWRCOM,		{ RT, RA } },
-{ "subfmeo.",XO(31,232,1,1), XORB_MASK, PPCCOM,		{ RT, RA } },
-{ "sfmeo.",  XO(31,232,1,1), XORB_MASK, PWRCOM,		{ RT, RA } },
-
-{ "mulld",   XO(31,233,0,0), XO_MASK,	PPC64,		{ RT, RA, RB } },
-{ "mulld.",  XO(31,233,0,1), XO_MASK,	PPC64,		{ RT, RA, RB } },
-{ "mulldo",  XO(31,233,1,0), XO_MASK,	PPC64,		{ RT, RA, RB } },
-{ "mulldo.", XO(31,233,1,1), XO_MASK,	PPC64,		{ RT, RA, RB } },
-
-{ "addme",   XO(31,234,0,0), XORB_MASK, PPCCOM,		{ RT, RA } },
-{ "ame",     XO(31,234,0,0), XORB_MASK, PWRCOM,		{ RT, RA } },
-{ "addme.",  XO(31,234,0,1), XORB_MASK, PPCCOM,		{ RT, RA } },
-{ "ame.",    XO(31,234,0,1), XORB_MASK, PWRCOM,		{ RT, RA } },
-{ "addmeo",  XO(31,234,1,0), XORB_MASK, PPCCOM,		{ RT, RA } },
-{ "ameo",    XO(31,234,1,0), XORB_MASK, PWRCOM,		{ RT, RA } },
-{ "addmeo.", XO(31,234,1,1), XORB_MASK, PPCCOM,		{ RT, RA } },
-{ "ameo.",   XO(31,234,1,1), XORB_MASK, PWRCOM,		{ RT, RA } },
-
-{ "addex",   XO(31,170,0,0), XO_MASK,   POWER9,         { RT, RA, RB } },
-
-{ "mullw",   XO(31,235,0,0), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "muls",    XO(31,235,0,0), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "mullw.",  XO(31,235,0,1), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "muls.",   XO(31,235,0,1), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "mullwo",  XO(31,235,1,0), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "mulso",   XO(31,235,1,0), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "mullwo.", XO(31,235,1,1), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "mulso.",  XO(31,235,1,1), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-
-{ "icblce",  X(31,238),	X_MASK,		PPCCHLK64,	{ CT, RA, RB }},
-{ "mtsrin",  X(31,242),	XRA_MASK,	PPC32,		{ RS, RB } },
-{ "mtsri",   X(31,242),	XRA_MASK,	POWER32,	{ RS, RB } },
-
-{ "dcbtst",  X(31,246),	X_MASK,	PPC,			{ CT, RA, RB } },
-
-{ "stbux",   X(31,247),	X_MASK,		COM,		{ RS, RAS, RB } },
-
-{ "slliq",   XRC(31,248,0), X_MASK,	M601,		{ RA, RS, SH } },
-{ "slliq.",  XRC(31,248,1), X_MASK,	M601,		{ RA, RS, SH } },
-
-{ "dcbtste", X(31,253),	X_MASK,		BOOKE64,	{ CT, RA, RB } },
-
-{ "stbuxe",  X(31,255),	X_MASK,		BOOKE64,	{ RS, RAS, RB } },
-
-{ "mfdcrx",  X(31,259),	X_MASK,		BOOKE,		{ RS, RA } },
-
-{ "doz",     XO(31,264,0,0), XO_MASK,	M601,		{ RT, RA, RB } },
-{ "doz.",    XO(31,264,0,1), XO_MASK,	M601,		{ RT, RA, RB } },
-{ "dozo",    XO(31,264,1,0), XO_MASK,	M601,		{ RT, RA, RB } },
-{ "dozo.",   XO(31,264,1,1), XO_MASK,	M601,		{ RT, RA, RB } },
-
-{ "add",     XO(31,266,0,0), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "cax",     XO(31,266,0,0), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "add.",    XO(31,266,0,1), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "cax.",    XO(31,266,0,1), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "addo",    XO(31,266,1,0), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "caxo",    XO(31,266,1,0), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-{ "addo.",   XO(31,266,1,1), XO_MASK,	PPCCOM,		{ RT, RA, RB } },
-{ "caxo.",   XO(31,266,1,1), XO_MASK,	PWRCOM,		{ RT, RA, RB } },
-
-{ "tlbiel",  X(31,274), XRTLRA_MASK,	POWER4,		{ RB, L } },
-
-{ "mfapidi", X(31,275), X_MASK,		BOOKE,		{ RT, RA } },
-
-{ "lscbx",   XRC(31,277,0), X_MASK,	M601,		{ RT, RA, RB } },
-{ "lscbx.",  XRC(31,277,1), X_MASK,	M601,		{ RT, RA, RB } },
-
-{ "dcbt",    X(31,278),	X_MASK,		PPC,		{ CT, RA, RB } },
-
-{ "lhzx",    X(31,279),	X_MASK,		COM,		{ RT, RA0, RB } },
-
-{ "eqv",     XRC(31,284,0), X_MASK,	COM,		{ RA, RS, RB } },
-{ "eqv.",    XRC(31,284,1), X_MASK,	COM,		{ RA, RS, RB } },
-
-{ "dcbte",   X(31,286),	X_MASK,		BOOKE64,	{ CT, RA, RB } },
-
-{ "lhzxe",   X(31,287),	X_MASK,		BOOKE64,	{ RT, RA0, RB } },
-
-{ "tlbie",   X(31,306),	XRTLRA_MASK,	PPC,		{ RB, L } },
-{ "tlbi",    X(31,306),	XRT_MASK,	POWER,		{ RA0, RB } },
-
-{ "eciwx",   X(31,310), X_MASK,		PPC,		{ RT, RA, RB } },
-
-{ "lhzux",   X(31,311),	X_MASK,		COM,		{ RT, RAL, RB } },
-
-{ "xor",     XRC(31,316,0), X_MASK,	COM,		{ RA, RS, RB } },
-{ "xor.",    XRC(31,316,1), X_MASK,	COM,		{ RA, RS, RB } },
-
-{ "lhzuxe",  X(31,319),	X_MASK,		BOOKE64,	{ RT, RAL, RB } },
-
-{ "mfexisr",  XSPR(31,323,64),  XSPR_MASK, PPC403,	{ RT } },
-{ "mfexier",  XSPR(31,323,66),  XSPR_MASK, PPC403,	{ RT } },
-{ "mfbr0",    XSPR(31,323,128), XSPR_MASK, PPC403,	{ RT } },
-{ "mfbr1",    XSPR(31,323,129), XSPR_MASK, PPC403,	{ RT } },
-{ "mfbr2",    XSPR(31,323,130), XSPR_MASK, PPC403,	{ RT } },
-{ "mfbr3",    XSPR(31,323,131), XSPR_MASK, PPC403,	{ RT } },
-{ "mfbr4",    XSPR(31,323,132), XSPR_MASK, PPC403,	{ RT } },
-{ "mfbr5",    XSPR(31,323,133), XSPR_MASK, PPC403,	{ RT } },
-{ "mfbr6",    XSPR(31,323,134), XSPR_MASK, PPC403,	{ RT } },
-{ "mfbr7",    XSPR(31,323,135), XSPR_MASK, PPC403,	{ RT } },
-{ "mfbear",   XSPR(31,323,144), XSPR_MASK, PPC403,	{ RT } },
-{ "mfbesr",   XSPR(31,323,145), XSPR_MASK, PPC403,	{ RT } },
-{ "mfiocr",   XSPR(31,323,160), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmacr0", XSPR(31,323,192), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmact0", XSPR(31,323,193), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmada0", XSPR(31,323,194), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmasa0", XSPR(31,323,195), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmacc0", XSPR(31,323,196), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmacr1", XSPR(31,323,200), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmact1", XSPR(31,323,201), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmada1", XSPR(31,323,202), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmasa1", XSPR(31,323,203), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmacc1", XSPR(31,323,204), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmacr2", XSPR(31,323,208), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmact2", XSPR(31,323,209), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmada2", XSPR(31,323,210), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmasa2", XSPR(31,323,211), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmacc2", XSPR(31,323,212), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmacr3", XSPR(31,323,216), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmact3", XSPR(31,323,217), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmada3", XSPR(31,323,218), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmasa3", XSPR(31,323,219), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmacc3", XSPR(31,323,220), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdmasr",  XSPR(31,323,224), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdcr",    X(31,323),	X_MASK,	PPC403 | BOOKE,	{ RT, SPR } },
-
-{ "div",     XO(31,331,0,0), XO_MASK,	M601,		{ RT, RA, RB } },
-{ "div.",    XO(31,331,0,1), XO_MASK,	M601,		{ RT, RA, RB } },
-{ "divo",    XO(31,331,1,0), XO_MASK,	M601,		{ RT, RA, RB } },
-{ "divo.",   XO(31,331,1,1), XO_MASK,	M601,		{ RT, RA, RB } },
-
-{ "mfpmr",   X(31,334),	X_MASK,		PPCPMR,		{ RT, PMR }},
-
-{ "mfmq",       XSPR(31,339,0),    XSPR_MASK, M601,	{ RT } },
-{ "mfxer",      XSPR(31,339,1),    XSPR_MASK, COM,	{ RT } },
-{ "mfrtcu",     XSPR(31,339,4),    XSPR_MASK, COM,	{ RT } },
-{ "mfrtcl",     XSPR(31,339,5),    XSPR_MASK, COM,	{ RT } },
-{ "mfdec",      XSPR(31,339,6),    XSPR_MASK, MFDEC1,	{ RT } },
-{ "mfdec",      XSPR(31,339,22),   XSPR_MASK, MFDEC2,	{ RT } },
-{ "mflr",       XSPR(31,339,8),    XSPR_MASK, COM,	{ RT } },
-{ "mfctr",      XSPR(31,339,9),    XSPR_MASK, COM,	{ RT } },
-{ "mftid",      XSPR(31,339,17),   XSPR_MASK, POWER,	{ RT } },
-{ "mfdsisr",    XSPR(31,339,18),   XSPR_MASK, COM,	{ RT } },
-{ "mfdar",      XSPR(31,339,19),   XSPR_MASK, COM,	{ RT } },
-{ "mfsdr0",     XSPR(31,339,24),   XSPR_MASK, POWER,	{ RT } },
-{ "mfsdr1",     XSPR(31,339,25),   XSPR_MASK, COM,	{ RT } },
-{ "mfsrr0",     XSPR(31,339,26),   XSPR_MASK, COM,	{ RT } },
-{ "mfsrr1",     XSPR(31,339,27),   XSPR_MASK, COM,	{ RT } },
-{ "mfcfar",     XSPR(31,339,28),   XSPR_MASK, POWER6,	{ RT } },
-{ "mfpid",      XSPR(31,339,48),   XSPR_MASK, BOOKE,    { RT } },
-{ "mfpid",      XSPR(31,339,945),  XSPR_MASK, PPC403,	{ RT } },
-{ "mfcsrr0",    XSPR(31,339,58),   XSPR_MASK, BOOKE,    { RT } },
-{ "mfcsrr1",    XSPR(31,339,59),   XSPR_MASK, BOOKE,    { RT } },
-{ "mfdear",     XSPR(31,339,61),   XSPR_MASK, BOOKE,    { RT } },
-{ "mfdear",     XSPR(31,339,981),  XSPR_MASK, PPC403,	{ RT } },
-{ "mfesr",      XSPR(31,339,62),   XSPR_MASK, BOOKE,    { RT } },
-{ "mfesr",      XSPR(31,339,980),  XSPR_MASK, PPC403,	{ RT } },
-{ "mfivpr",     XSPR(31,339,63),   XSPR_MASK, BOOKE,    { RT } },
-{ "mfcmpa",     XSPR(31,339,144),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfcmpb",     XSPR(31,339,145),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfcmpc",     XSPR(31,339,146),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfcmpd",     XSPR(31,339,147),  XSPR_MASK, PPC860,	{ RT } },
-{ "mficr",      XSPR(31,339,148),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfder",      XSPR(31,339,149),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfcounta",   XSPR(31,339,150),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfcountb",   XSPR(31,339,151),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfcmpe",     XSPR(31,339,152),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfcmpf",     XSPR(31,339,153),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfcmpg",     XSPR(31,339,154),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfcmph",     XSPR(31,339,155),  XSPR_MASK, PPC860,	{ RT } },
-{ "mflctrl1",   XSPR(31,339,156),  XSPR_MASK, PPC860,	{ RT } },
-{ "mflctrl2",   XSPR(31,339,157),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfictrl",    XSPR(31,339,158),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfbar",      XSPR(31,339,159),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfvrsave",   XSPR(31,339,256),  XSPR_MASK, PPCVEC,	{ RT } },
-{ "mfusprg0",   XSPR(31,339,256),  XSPR_MASK, BOOKE,    { RT } },
-{ "mftb",       X(31,371),	   X_MASK,    CLASSIC,	{ RT, TBR } },
-{ "mftb",       XSPR(31,339,268),  XSPR_MASK, BOOKE,    { RT } },
-{ "mftbl",      XSPR(31,371,268),  XSPR_MASK, CLASSIC,	{ RT } },
-{ "mftbl",      XSPR(31,339,268),  XSPR_MASK, BOOKE,    { RT } },
-{ "mftbu",      XSPR(31,371,269),  XSPR_MASK, CLASSIC,	{ RT } },
-{ "mftbu",      XSPR(31,339,269),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfsprg",     XSPR(31,339,256),  XSPRG_MASK, PPC,	{ RT, SPRG } },
-{ "mfsprg0",    XSPR(31,339,272),  XSPR_MASK, PPC,	{ RT } },
-{ "mfsprg1",    XSPR(31,339,273),  XSPR_MASK, PPC,	{ RT } },
-{ "mfsprg2",    XSPR(31,339,274),  XSPR_MASK, PPC,	{ RT } },
-{ "mfsprg3",    XSPR(31,339,275),  XSPR_MASK, PPC,	{ RT } },
-{ "mfsprg4",    XSPR(31,339,260),  XSPR_MASK, PPC405 | BOOKE,	{ RT } },
-{ "mfsprg5",    XSPR(31,339,261),  XSPR_MASK, PPC405 | BOOKE,	{ RT } },
-{ "mfsprg6",    XSPR(31,339,262),  XSPR_MASK, PPC405 | BOOKE,	{ RT } },
-{ "mfsprg7",    XSPR(31,339,263),  XSPR_MASK, PPC405 | BOOKE,	{ RT } },
-{ "mfasr",      XSPR(31,339,280),  XSPR_MASK, PPC64,	{ RT } },
-{ "mfear",      XSPR(31,339,282),  XSPR_MASK, PPC,	{ RT } },
-{ "mfpir",      XSPR(31,339,286),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfpvr",      XSPR(31,339,287),  XSPR_MASK, PPC,	{ RT } },
-{ "mfdbsr",     XSPR(31,339,304),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfdbsr",     XSPR(31,339,1008), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdbcr0",    XSPR(31,339,308),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfdbcr0",    XSPR(31,339,1010), XSPR_MASK, PPC405,	{ RT } },
-{ "mfdbcr1",    XSPR(31,339,309),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfdbcr1",    XSPR(31,339,957),  XSPR_MASK, PPC405,	{ RT } },
-{ "mfdbcr2",    XSPR(31,339,310),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfiac1",     XSPR(31,339,312),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfiac1",     XSPR(31,339,1012), XSPR_MASK, PPC403,	{ RT } },
-{ "mfiac2",     XSPR(31,339,313),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfiac2",     XSPR(31,339,1013), XSPR_MASK, PPC403,	{ RT } },
-{ "mfiac3",     XSPR(31,339,314),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfiac3",     XSPR(31,339,948),  XSPR_MASK, PPC405,	{ RT } },
-{ "mfiac4",     XSPR(31,339,315),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfiac4",     XSPR(31,339,949),  XSPR_MASK, PPC405,	{ RT } },
-{ "mfdac1",     XSPR(31,339,316),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfdac1",     XSPR(31,339,1014), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdac2",     XSPR(31,339,317),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfdac2",     XSPR(31,339,1015), XSPR_MASK, PPC403,	{ RT } },
-{ "mfdvc1",     XSPR(31,339,318),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfdvc1",     XSPR(31,339,950),  XSPR_MASK, PPC405,	{ RT } },
-{ "mfdvc2",     XSPR(31,339,319),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfdvc2",     XSPR(31,339,951),  XSPR_MASK, PPC405,	{ RT } },
-{ "mftsr",      XSPR(31,339,336),  XSPR_MASK, BOOKE,    { RT } },
-{ "mftsr",      XSPR(31,339,984),  XSPR_MASK, PPC403,	{ RT } },
-{ "mftcr",      XSPR(31,339,340),  XSPR_MASK, BOOKE,    { RT } },
-{ "mftcr",      XSPR(31,339,986),  XSPR_MASK, PPC403,	{ RT } },
-{ "mfivor0",    XSPR(31,339,400),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfivor1",    XSPR(31,339,401),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfivor2",    XSPR(31,339,402),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfivor3",    XSPR(31,339,403),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfivor4",    XSPR(31,339,404),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfivor5",    XSPR(31,339,405),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfivor6",    XSPR(31,339,406),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfivor7",    XSPR(31,339,407),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfivor8",    XSPR(31,339,408),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfivor9",    XSPR(31,339,409),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfivor10",   XSPR(31,339,410),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfivor11",   XSPR(31,339,411),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfivor12",   XSPR(31,339,412),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfivor13",   XSPR(31,339,413),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfivor14",   XSPR(31,339,414),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfivor15",   XSPR(31,339,415),  XSPR_MASK, BOOKE,    { RT } },
-{ "mfspefscr",  XSPR(31,339,512),  XSPR_MASK, PPCSPE,	{ RT } },
-{ "mfbbear",    XSPR(31,339,513),  XSPR_MASK, PPCBRLK,  { RT } },
-{ "mfbbtar",    XSPR(31,339,514),  XSPR_MASK, PPCBRLK,  { RT } },
-{ "mfivor32",   XSPR(31,339,528),  XSPR_MASK, PPCSPE,	{ RT } },
-{ "mfivor33",   XSPR(31,339,529),  XSPR_MASK, PPCSPE,	{ RT } },
-{ "mfivor34",   XSPR(31,339,530),  XSPR_MASK, PPCSPE,	{ RT } },
-{ "mfivor35",   XSPR(31,339,531),  XSPR_MASK, PPCPMR,	{ RT } },
-{ "mfibatu",    XSPR(31,339,528),  XSPRBAT_MASK, PPC,	{ RT, SPRBAT } },
-{ "mfibatl",    XSPR(31,339,529),  XSPRBAT_MASK, PPC,	{ RT, SPRBAT } },
-{ "mfdbatu",    XSPR(31,339,536),  XSPRBAT_MASK, PPC,	{ RT, SPRBAT } },
-{ "mfdbatl",    XSPR(31,339,537),  XSPRBAT_MASK, PPC,	{ RT, SPRBAT } },
-{ "mfic_cst",   XSPR(31,339,560),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfic_adr",   XSPR(31,339,561),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfic_dat",   XSPR(31,339,562),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfdc_cst",   XSPR(31,339,568),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfdc_adr",   XSPR(31,339,569),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmcsrr0",   XSPR(31,339,570),  XSPR_MASK, PPCRFMCI, { RT } },
-{ "mfdc_dat",   XSPR(31,339,570),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmcsrr1",   XSPR(31,339,571),  XSPR_MASK, PPCRFMCI, { RT } },
-{ "mfmcsr",     XSPR(31,339,572),  XSPR_MASK, PPCRFMCI, { RT } },
-{ "mfmcar",     XSPR(31,339,573),  XSPR_MASK, PPCRFMCI, { RT } },
-{ "mfdpdr",     XSPR(31,339,630),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfdpir",     XSPR(31,339,631),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfimmr",     XSPR(31,339,638),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmi_ctr",   XSPR(31,339,784),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmi_ap",    XSPR(31,339,786),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmi_epn",   XSPR(31,339,787),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmi_twc",   XSPR(31,339,789),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmi_rpn",   XSPR(31,339,790),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmd_ctr",   XSPR(31,339,792),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfm_casid",  XSPR(31,339,793),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmd_ap",    XSPR(31,339,794),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmd_epn",   XSPR(31,339,795),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmd_twb",   XSPR(31,339,796),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmd_twc",   XSPR(31,339,797),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmd_rpn",   XSPR(31,339,798),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfm_tw",     XSPR(31,339,799),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmi_dbcam", XSPR(31,339,816),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmi_dbram0",XSPR(31,339,817),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmi_dbram1",XSPR(31,339,818),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmd_dbcam", XSPR(31,339,824),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmd_dbram0",XSPR(31,339,825),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfmd_dbram1",XSPR(31,339,826),  XSPR_MASK, PPC860,	{ RT } },
-{ "mfummcr0",   XSPR(31,339,936),  XSPR_MASK, PPC750,   { RT } },
-{ "mfupmc1",    XSPR(31,339,937),  XSPR_MASK, PPC750,   { RT } },
-{ "mfupmc2",    XSPR(31,339,938),  XSPR_MASK, PPC750,   { RT } },
-{ "mfusia",     XSPR(31,339,939),  XSPR_MASK, PPC750,   { RT } },
-{ "mfummcr1",   XSPR(31,339,940),  XSPR_MASK, PPC750,   { RT } },
-{ "mfupmc3",    XSPR(31,339,941),  XSPR_MASK, PPC750,   { RT } },
-{ "mfupmc4",    XSPR(31,339,942),  XSPR_MASK, PPC750,   { RT } },
-{ "mfzpr",   	XSPR(31,339,944),  XSPR_MASK, PPC403,	{ RT } },
-{ "mfccr0",  	XSPR(31,339,947),  XSPR_MASK, PPC405,	{ RT } },
-{ "mfmmcr0",	XSPR(31,339,952),  XSPR_MASK, PPC750,	{ RT } },
-{ "mfpmc1",	XSPR(31,339,953),  XSPR_MASK, PPC750,	{ RT } },
-{ "mfsgr",	XSPR(31,339,953),  XSPR_MASK, PPC403,	{ RT } },
-{ "mfpmc2",	XSPR(31,339,954),  XSPR_MASK, PPC750,	{ RT } },
-{ "mfdcwr", 	XSPR(31,339,954),  XSPR_MASK, PPC403,	{ RT } },
-{ "mfsia",	XSPR(31,339,955),  XSPR_MASK, PPC750,	{ RT } },
-{ "mfsler",	XSPR(31,339,955),  XSPR_MASK, PPC405,	{ RT } },
-{ "mfmmcr1",	XSPR(31,339,956),  XSPR_MASK, PPC750,	{ RT } },
-{ "mfsu0r",	XSPR(31,339,956),  XSPR_MASK, PPC405,	{ RT } },
-{ "mfpmc3",	XSPR(31,339,957),  XSPR_MASK, PPC750,	{ RT } },
-{ "mfpmc4",	XSPR(31,339,958),  XSPR_MASK, PPC750,	{ RT } },
-{ "mficdbdr",   XSPR(31,339,979),  XSPR_MASK, PPC403,   { RT } },
-{ "mfevpr",     XSPR(31,339,982),  XSPR_MASK, PPC403,	{ RT } },
-{ "mfcdbcr",    XSPR(31,339,983),  XSPR_MASK, PPC403,	{ RT } },
-{ "mfpit",      XSPR(31,339,987),  XSPR_MASK, PPC403,	{ RT } },
-{ "mftbhi",     XSPR(31,339,988),  XSPR_MASK, PPC403,	{ RT } },
-{ "mftblo",     XSPR(31,339,989),  XSPR_MASK, PPC403,	{ RT } },
-{ "mfsrr2",     XSPR(31,339,990),  XSPR_MASK, PPC403,	{ RT } },
-{ "mfsrr3",     XSPR(31,339,991),  XSPR_MASK, PPC403,	{ RT } },
-{ "mfl2cr",     XSPR(31,339,1017), XSPR_MASK, PPC750,   { RT } },
-{ "mfdccr",     XSPR(31,339,1018), XSPR_MASK, PPC403,	{ RT } },
-{ "mficcr",     XSPR(31,339,1019), XSPR_MASK, PPC403,	{ RT } },
-{ "mfictc",     XSPR(31,339,1019), XSPR_MASK, PPC750,   { RT } },
-{ "mfpbl1",     XSPR(31,339,1020), XSPR_MASK, PPC403,	{ RT } },
-{ "mfthrm1",    XSPR(31,339,1020), XSPR_MASK, PPC750,   { RT } },
-{ "mfpbu1",     XSPR(31,339,1021), XSPR_MASK, PPC403,	{ RT } },
-{ "mfthrm2",    XSPR(31,339,1021), XSPR_MASK, PPC750,   { RT } },
-{ "mfpbl2",     XSPR(31,339,1022), XSPR_MASK, PPC403,	{ RT } },
-{ "mfthrm3",    XSPR(31,339,1022), XSPR_MASK, PPC750,   { RT } },
-{ "mfpbu2",     XSPR(31,339,1023), XSPR_MASK, PPC403,	{ RT } },
-{ "mfspr",      X(31,339),	   X_MASK,    COM,	{ RT, SPR } },
-
-{ "lwax",    X(31,341),	X_MASK,		PPC64,		{ RT, RA0, RB } },
-
-{ "dst",     XDSS(31,342,0), XDSS_MASK,	PPCVEC,		{ RA, RB, STRM } },
-{ "dstt",    XDSS(31,342,1), XDSS_MASK,	PPCVEC,		{ RA, RB, STRM } },
-
-{ "lhax",    X(31,343),	X_MASK,		COM,		{ RT, RA0, RB } },
-
-{ "lhaxe",   X(31,351),	X_MASK,		BOOKE64,	{ RT, RA0, RB } },
-
-{ "dstst",   XDSS(31,374,0), XDSS_MASK,	PPCVEC,		{ RA, RB, STRM } },
-{ "dststt",  XDSS(31,374,1), XDSS_MASK,	PPCVEC,		{ RA, RB, STRM } },
-
-{ "dccci",   X(31,454),	XRT_MASK,	PPC403|PPC440,	{ RA, RB } },
-
-{ "abs",     XO(31,360,0,0), XORB_MASK, M601,		{ RT, RA } },
-{ "abs.",    XO(31,360,0,1), XORB_MASK, M601,		{ RT, RA } },
-{ "abso",    XO(31,360,1,0), XORB_MASK, M601,		{ RT, RA } },
-{ "abso.",   XO(31,360,1,1), XORB_MASK, M601,		{ RT, RA } },
-
-{ "divs",    XO(31,363,0,0), XO_MASK,	M601,		{ RT, RA, RB } },
-{ "divs.",   XO(31,363,0,1), XO_MASK,	M601,		{ RT, RA, RB } },
-{ "divso",   XO(31,363,1,0), XO_MASK,	M601,		{ RT, RA, RB } },
-{ "divso.",  XO(31,363,1,1), XO_MASK,	M601,		{ RT, RA, RB } },
-
-{ "tlbia",   X(31,370),	0xffffffff,	PPC,		{ 0 } },
-
-{ "lwaux",   X(31,373),	X_MASK,		PPC64,		{ RT, RAL, RB } },
-
-{ "lhaux",   X(31,375),	X_MASK,		COM,		{ RT, RAL, RB } },
-
-{ "lhauxe",  X(31,383),	X_MASK,		BOOKE64,	{ RT, RAL, RB } },
-
-{ "mtdcrx",  X(31,387),	X_MASK,		BOOKE,		{ RA, RS } },
-
-{ "dcblc",   X(31,390),	X_MASK,		PPCCHLK,	{ CT, RA, RB }},
-
-{ "subfe64", XO(31,392,0,0), XO_MASK,	BOOKE64,	{ RT, RA, RB } },
-{ "subfe64o",XO(31,392,1,0), XO_MASK,	BOOKE64,	{ RT, RA, RB } },
-
-{ "adde64",  XO(31,394,0,0), XO_MASK,	BOOKE64,	{ RT, RA, RB } },
-{ "adde64o", XO(31,394,1,0), XO_MASK,	BOOKE64,	{ RT, RA, RB } },
-
-{ "dcblce",  X(31,398),	X_MASK,		PPCCHLK64,	{ CT, RA, RB }},
-
-{ "slbmte",  X(31,402), XRA_MASK,	PPC64,		{ RS, RB } },
-
-{ "sthx",    X(31,407),	X_MASK,		COM,		{ RS, RA0, RB } },
-
-{ "cmpb",    X(31,508),	X_MASK,		POWER6,		{ RA, RS, RB } },
-
-{ "lfqx",    X(31,791),	X_MASK,		POWER2,		{ FRT, RA, RB } },
-
-{ "lfdpx",   X(31,791),	X_MASK,		POWER6,		{ FRT, RA, RB } },
-
-{ "lfqux",   X(31,823),	X_MASK,		POWER2,		{ FRT, RA, RB } },
-
-{ "stfqx",   X(31,919),	X_MASK,		POWER2,		{ FRS, RA, RB } },
-
-{ "stfdpx",  X(31,919),	X_MASK,		POWER6,		{ FRS, RA, RB } },
-
-{ "stfqux",  X(31,951),	X_MASK,		POWER2,		{ FRS, RA, RB } },
-
-{ "orc",     XRC(31,412,0), X_MASK,	COM,		{ RA, RS, RB } },
-{ "orc.",    XRC(31,412,1), X_MASK,	COM,		{ RA, RS, RB } },
-
-{ "sradi",   XS(31,413,0), XS_MASK,	PPC64,		{ RA, RS, SH6 } },
-{ "sradi.",  XS(31,413,1), XS_MASK,	PPC64,		{ RA, RS, SH6 } },
-
-{ "sthxe",   X(31,415),	X_MASK,		BOOKE64,	{ RS, RA0, RB } },
-
-{ "slbie",   X(31,434),	XRTRA_MASK,	PPC64,		{ RB } },
-
-{ "ecowx",   X(31,438),	X_MASK,		PPC,		{ RT, RA, RB } },
-
-{ "sthux",   X(31,439),	X_MASK,		COM,		{ RS, RAS, RB } },
-
-{ "sthuxe",  X(31,447),	X_MASK,		BOOKE64,	{ RS, RAS, RB } },
-
-{ "cctpl",   0x7c210b78,    0xffffffff,	CELL,		{ 0 }},
-{ "cctpm",   0x7c421378,    0xffffffff,	CELL,		{ 0 }},
-{ "cctph",   0x7c631b78,    0xffffffff,	CELL,		{ 0 }},
-{ "db8cyc",  0x7f9ce378,    0xffffffff,	CELL,		{ 0 }},
-{ "db10cyc", 0x7fbdeb78,    0xffffffff,	CELL,		{ 0 }},
-{ "db12cyc", 0x7fdef378,    0xffffffff,	CELL,		{ 0 }},
-{ "db16cyc", 0x7ffffb78,    0xffffffff,	CELL,		{ 0 }},
-{ "mr",	     XRC(31,444,0), X_MASK,	COM,		{ RA, RS, RBS } },
-{ "or",      XRC(31,444,0), X_MASK,	COM,		{ RA, RS, RB } },
-{ "mr.",     XRC(31,444,1), X_MASK,	COM,		{ RA, RS, RBS } },
-{ "or.",     XRC(31,444,1), X_MASK,	COM,		{ RA, RS, RB } },
-
-{ "mtexisr",  XSPR(31,451,64),  XSPR_MASK, PPC403,	{ RS } },
-{ "mtexier",  XSPR(31,451,66),  XSPR_MASK, PPC403,	{ RS } },
-{ "mtbr0",    XSPR(31,451,128), XSPR_MASK, PPC403,	{ RS } },
-{ "mtbr1",    XSPR(31,451,129), XSPR_MASK, PPC403,	{ RS } },
-{ "mtbr2",    XSPR(31,451,130), XSPR_MASK, PPC403,	{ RS } },
-{ "mtbr3",    XSPR(31,451,131), XSPR_MASK, PPC403,	{ RS } },
-{ "mtbr4",    XSPR(31,451,132), XSPR_MASK, PPC403,	{ RS } },
-{ "mtbr5",    XSPR(31,451,133), XSPR_MASK, PPC403,	{ RS } },
-{ "mtbr6",    XSPR(31,451,134), XSPR_MASK, PPC403,	{ RS } },
-{ "mtbr7",    XSPR(31,451,135), XSPR_MASK, PPC403,	{ RS } },
-{ "mtbear",   XSPR(31,451,144), XSPR_MASK, PPC403,	{ RS } },
-{ "mtbesr",   XSPR(31,451,145), XSPR_MASK, PPC403,	{ RS } },
-{ "mtiocr",   XSPR(31,451,160), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmacr0", XSPR(31,451,192), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmact0", XSPR(31,451,193), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmada0", XSPR(31,451,194), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmasa0", XSPR(31,451,195), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmacc0", XSPR(31,451,196), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmacr1", XSPR(31,451,200), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmact1", XSPR(31,451,201), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmada1", XSPR(31,451,202), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmasa1", XSPR(31,451,203), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmacc1", XSPR(31,451,204), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmacr2", XSPR(31,451,208), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmact2", XSPR(31,451,209), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmada2", XSPR(31,451,210), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmasa2", XSPR(31,451,211), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmacc2", XSPR(31,451,212), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmacr3", XSPR(31,451,216), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmact3", XSPR(31,451,217), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmada3", XSPR(31,451,218), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmasa3", XSPR(31,451,219), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmacc3", XSPR(31,451,220), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdmasr",  XSPR(31,451,224), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdcr",    X(31,451),	X_MASK,	PPC403 | BOOKE,	{ SPR, RS } },
-
-{ "subfze64",XO(31,456,0,0), XORB_MASK, BOOKE64,	{ RT, RA } },
-{ "subfze64o",XO(31,456,1,0), XORB_MASK, BOOKE64,	{ RT, RA } },
-
-{ "divdu",   XO(31,457,0,0), XO_MASK,	PPC64,		{ RT, RA, RB } },
-{ "divdu.",  XO(31,457,0,1), XO_MASK,	PPC64,		{ RT, RA, RB } },
-{ "divduo",  XO(31,457,1,0), XO_MASK,	PPC64,		{ RT, RA, RB } },
-{ "divduo.", XO(31,457,1,1), XO_MASK,	PPC64,		{ RT, RA, RB } },
-
-{ "addze64", XO(31,458,0,0), XORB_MASK, BOOKE64,	{ RT, RA } },
-{ "addze64o",XO(31,458,1,0), XORB_MASK, BOOKE64,	{ RT, RA } },
-
-{ "divwu",   XO(31,459,0,0), XO_MASK,	PPC,		{ RT, RA, RB } },
-{ "divwu.",  XO(31,459,0,1), XO_MASK,	PPC,		{ RT, RA, RB } },
-{ "divwuo",  XO(31,459,1,0), XO_MASK,	PPC,		{ RT, RA, RB } },
-{ "divwuo.", XO(31,459,1,1), XO_MASK,	PPC,		{ RT, RA, RB } },
-
-{ "mtmq",      XSPR(31,467,0),    XSPR_MASK, M601,	{ RS } },
-{ "mtxer",     XSPR(31,467,1),    XSPR_MASK, COM,	{ RS } },
-{ "mtlr",      XSPR(31,467,8),    XSPR_MASK, COM,	{ RS } },
-{ "mtctr",     XSPR(31,467,9),    XSPR_MASK, COM,	{ RS } },
-{ "mttid",     XSPR(31,467,17),   XSPR_MASK, POWER,	{ RS } },
-{ "mtdsisr",   XSPR(31,467,18),   XSPR_MASK, COM,	{ RS } },
-{ "mtdar",     XSPR(31,467,19),   XSPR_MASK, COM,	{ RS } },
-{ "mtrtcu",    XSPR(31,467,20),   XSPR_MASK, COM,	{ RS } },
-{ "mtrtcl",    XSPR(31,467,21),   XSPR_MASK, COM,	{ RS } },
-{ "mtdec",     XSPR(31,467,22),   XSPR_MASK, COM,	{ RS } },
-{ "mtsdr0",    XSPR(31,467,24),   XSPR_MASK, POWER,	{ RS } },
-{ "mtsdr1",    XSPR(31,467,25),   XSPR_MASK, COM,	{ RS } },
-{ "mtsrr0",    XSPR(31,467,26),   XSPR_MASK, COM,	{ RS } },
-{ "mtsrr1",    XSPR(31,467,27),   XSPR_MASK, COM,	{ RS } },
-{ "mtcfar",    XSPR(31,467,28),   XSPR_MASK, POWER6,	{ RS } },
-{ "mtpid",     XSPR(31,467,48),   XSPR_MASK, BOOKE,     { RS } },
-{ "mtpid",     XSPR(31,467,945),  XSPR_MASK, PPC403,	{ RS } },
-{ "mtdecar",   XSPR(31,467,54),   XSPR_MASK, BOOKE,     { RS } },
-{ "mtcsrr0",   XSPR(31,467,58),   XSPR_MASK, BOOKE,     { RS } },
-{ "mtcsrr1",   XSPR(31,467,59),   XSPR_MASK, BOOKE,     { RS } },
-{ "mtdear",    XSPR(31,467,61),   XSPR_MASK, BOOKE,     { RS } },
-{ "mtdear",    XSPR(31,467,981),  XSPR_MASK, PPC403,	{ RS } },
-{ "mtesr",     XSPR(31,467,62),   XSPR_MASK, BOOKE,     { RS } },
-{ "mtesr",     XSPR(31,467,980),  XSPR_MASK, PPC403,	{ RS } },
-{ "mtivpr",    XSPR(31,467,63),   XSPR_MASK, BOOKE,     { RS } },
-{ "mtcmpa",    XSPR(31,467,144),  XSPR_MASK, PPC860,	{ RS } },
-{ "mtcmpb",    XSPR(31,467,145),  XSPR_MASK, PPC860,	{ RS } },
-{ "mtcmpc",    XSPR(31,467,146),  XSPR_MASK, PPC860,	{ RS } },
-{ "mtcmpd",    XSPR(31,467,147),  XSPR_MASK, PPC860,	{ RS } },
-{ "mticr",     XSPR(31,467,148),  XSPR_MASK, PPC860,	{ RS } },
-{ "mtder",     XSPR(31,467,149),  XSPR_MASK, PPC860,	{ RS } },
-{ "mtcounta",  XSPR(31,467,150),  XSPR_MASK, PPC860,	{ RS } },
-{ "mtcountb",  XSPR(31,467,151),  XSPR_MASK, PPC860,	{ RS } },
-{ "mtcmpe",    XSPR(31,467,152),  XSPR_MASK, PPC860,	{ RS } },
-{ "mtcmpf",    XSPR(31,467,153),  XSPR_MASK, PPC860,	{ RS } },
-{ "mtcmpg",    XSPR(31,467,154),  XSPR_MASK, PPC860,	{ RS } },
-{ "mtcmph",    XSPR(31,467,155),  XSPR_MASK, PPC860,	{ RS } },
-{ "mtlctrl1",  XSPR(31,467,156),  XSPR_MASK, PPC860,	{ RS } },
-{ "mtlctrl2",  XSPR(31,467,157),  XSPR_MASK, PPC860,	{ RS } },
-{ "mtictrl",   XSPR(31,467,158),  XSPR_MASK, PPC860,	{ RS } },
-{ "mtbar",     XSPR(31,467,159),  XSPR_MASK, PPC860,	{ RS } },
-{ "mtvrsave",  XSPR(31,467,256),  XSPR_MASK, PPCVEC,	{ RS } },
-{ "mtusprg0",  XSPR(31,467,256),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtsprg",    XSPR(31,467,256),  XSPRG_MASK,PPC,	{ SPRG, RS } },
-{ "mtsprg0",   XSPR(31,467,272),  XSPR_MASK, PPC,	{ RS } },
-{ "mtsprg1",   XSPR(31,467,273),  XSPR_MASK, PPC,	{ RS } },
-{ "mtsprg2",   XSPR(31,467,274),  XSPR_MASK, PPC,	{ RS } },
-{ "mtsprg3",   XSPR(31,467,275),  XSPR_MASK, PPC,	{ RS } },
-{ "mtsprg4",   XSPR(31,467,276),  XSPR_MASK, PPC405 | BOOKE, { RS } },
-{ "mtsprg5",   XSPR(31,467,277),  XSPR_MASK, PPC405 | BOOKE, { RS } },
-{ "mtsprg6",   XSPR(31,467,278),  XSPR_MASK, PPC405 | BOOKE, { RS } },
-{ "mtsprg7",   XSPR(31,467,279),  XSPR_MASK, PPC405 | BOOKE, { RS } },
-{ "mtasr",     XSPR(31,467,280),  XSPR_MASK, PPC64,	{ RS } },
-{ "mtear",     XSPR(31,467,282),  XSPR_MASK, PPC,	{ RS } },
-{ "mttbl",     XSPR(31,467,284),  XSPR_MASK, PPC,	{ RS } },
-{ "mttbu",     XSPR(31,467,285),  XSPR_MASK, PPC,	{ RS } },
-{ "mtdbsr",    XSPR(31,467,304),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtdbsr",    XSPR(31,467,1008), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdbcr0",   XSPR(31,467,308),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtdbcr0",   XSPR(31,467,1010), XSPR_MASK, PPC405,	{ RS } },
-{ "mtdbcr1",   XSPR(31,467,309),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtdbcr1",   XSPR(31,467,957),  XSPR_MASK, PPC405,	{ RS } },
-{ "mtdbcr2",   XSPR(31,467,310),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtiac1",    XSPR(31,467,312),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtiac1",    XSPR(31,467,1012), XSPR_MASK, PPC403,	{ RS } },
-{ "mtiac2",    XSPR(31,467,313),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtiac2",    XSPR(31,467,1013), XSPR_MASK, PPC403,	{ RS } },
-{ "mtiac3",    XSPR(31,467,314),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtiac3",    XSPR(31,467,948),  XSPR_MASK, PPC405,	{ RS } },
-{ "mtiac4",    XSPR(31,467,315),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtiac4",    XSPR(31,467,949),  XSPR_MASK, PPC405,	{ RS } },
-{ "mtdac1",    XSPR(31,467,316),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtdac1",    XSPR(31,467,1014), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdac2",    XSPR(31,467,317),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtdac2",    XSPR(31,467,1015), XSPR_MASK, PPC403,	{ RS } },
-{ "mtdvc1",    XSPR(31,467,318),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtdvc1",    XSPR(31,467,950),  XSPR_MASK, PPC405,	{ RS } },
-{ "mtdvc2",    XSPR(31,467,319),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtdvc2",    XSPR(31,467,951),  XSPR_MASK, PPC405,	{ RS } },
-{ "mttsr",     XSPR(31,467,336),  XSPR_MASK, BOOKE,     { RS } },
-{ "mttsr",     XSPR(31,467,984),  XSPR_MASK, PPC403,	{ RS } },
-{ "mttcr",     XSPR(31,467,340),  XSPR_MASK, BOOKE,     { RS } },
-{ "mttcr",     XSPR(31,467,986),  XSPR_MASK, PPC403,	{ RS } },
-{ "mtivor0",   XSPR(31,467,400),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtivor1",   XSPR(31,467,401),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtivor2",   XSPR(31,467,402),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtivor3",   XSPR(31,467,403),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtivor4",   XSPR(31,467,404),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtivor5",   XSPR(31,467,405),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtivor6",   XSPR(31,467,406),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtivor7",   XSPR(31,467,407),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtivor8",   XSPR(31,467,408),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtivor9",   XSPR(31,467,409),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtivor10",  XSPR(31,467,410),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtivor11",  XSPR(31,467,411),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtivor12",  XSPR(31,467,412),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtivor13",  XSPR(31,467,413),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtivor14",  XSPR(31,467,414),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtivor15",  XSPR(31,467,415),  XSPR_MASK, BOOKE,     { RS } },
-{ "mtspefscr",  XSPR(31,467,512),  XSPR_MASK, PPCSPE,   { RS } },
-{ "mtbbear",   XSPR(31,467,513),  XSPR_MASK, PPCBRLK,   { RS } },
-{ "mtbbtar",   XSPR(31,467,514),  XSPR_MASK, PPCBRLK,  { RS } },
-{ "mtivor32",  XSPR(31,467,528),  XSPR_MASK, PPCSPE,	{ RS } },
-{ "mtivor33",  XSPR(31,467,529),  XSPR_MASK, PPCSPE,	{ RS } },
-{ "mtivor34",  XSPR(31,467,530),  XSPR_MASK, PPCSPE,	{ RS } },
-{ "mtivor35",  XSPR(31,467,531),  XSPR_MASK, PPCPMR,	{ RS } },
-{ "mtibatu",   XSPR(31,467,528),  XSPRBAT_MASK, PPC,	{ SPRBAT, RS } },
-{ "mtibatl",   XSPR(31,467,529),  XSPRBAT_MASK, PPC,	{ SPRBAT, RS } },
-{ "mtdbatu",   XSPR(31,467,536),  XSPRBAT_MASK, PPC,	{ SPRBAT, RS } },
-{ "mtdbatl",   XSPR(31,467,537),  XSPRBAT_MASK, PPC,	{ SPRBAT, RS } },
-{ "mtmcsrr0",  XSPR(31,467,570),  XSPR_MASK, PPCRFMCI,  { RS } },
-{ "mtmcsrr1",  XSPR(31,467,571),  XSPR_MASK, PPCRFMCI,  { RS } },
-{ "mtmcsr",    XSPR(31,467,572),  XSPR_MASK, PPCRFMCI,  { RS } },
-{ "mtummcr0",  XSPR(31,467,936),  XSPR_MASK, PPC750,    { RS } },
-{ "mtupmc1",   XSPR(31,467,937),  XSPR_MASK, PPC750,    { RS } },
-{ "mtupmc2",   XSPR(31,467,938),  XSPR_MASK, PPC750,    { RS } },
-{ "mtusia",    XSPR(31,467,939),  XSPR_MASK, PPC750,    { RS } },
-{ "mtummcr1",  XSPR(31,467,940),  XSPR_MASK, PPC750,    { RS } },
-{ "mtupmc3",   XSPR(31,467,941),  XSPR_MASK, PPC750,    { RS } },
-{ "mtupmc4",   XSPR(31,467,942),  XSPR_MASK, PPC750,    { RS } },
-{ "mtzpr",     XSPR(31,467,944),  XSPR_MASK, PPC403,	{ RS } },
-{ "mtccr0",    XSPR(31,467,947),  XSPR_MASK, PPC405,	{ RS } },
-{ "mtmmcr0",   XSPR(31,467,952),  XSPR_MASK, PPC750,    { RS } },
-{ "mtsgr",     XSPR(31,467,953),  XSPR_MASK, PPC403,	{ RS } },
-{ "mtpmc1",    XSPR(31,467,953),  XSPR_MASK, PPC750,    { RS } },
-{ "mtdcwr",    XSPR(31,467,954),  XSPR_MASK, PPC403,	{ RS } },
-{ "mtpmc2",    XSPR(31,467,954),  XSPR_MASK, PPC750,    { RS } },
-{ "mtsler",    XSPR(31,467,955),  XSPR_MASK, PPC405,	{ RS } },
-{ "mtsia",     XSPR(31,467,955),  XSPR_MASK, PPC750,    { RS } },
-{ "mtsu0r",    XSPR(31,467,956),  XSPR_MASK, PPC405,	{ RS } },
-{ "mtmmcr1",   XSPR(31,467,956),  XSPR_MASK, PPC750,    { RS } },
-{ "mtpmc3",    XSPR(31,467,957),  XSPR_MASK, PPC750,    { RS } },
-{ "mtpmc4",    XSPR(31,467,958),  XSPR_MASK, PPC750,    { RS } },
-{ "mticdbdr",  XSPR(31,467,979),  XSPR_MASK, PPC403,	{ RS } },
-{ "mtevpr",    XSPR(31,467,982),  XSPR_MASK, PPC403,	{ RS } },
-{ "mtcdbcr",   XSPR(31,467,983),  XSPR_MASK, PPC403,	{ RS } },
-{ "mtpit",     XSPR(31,467,987),  XSPR_MASK, PPC403,	{ RS } },
-{ "mttbhi",    XSPR(31,467,988),  XSPR_MASK, PPC403,	{ RS } },
-{ "mttblo",    XSPR(31,467,989),  XSPR_MASK, PPC403,	{ RS } },
-{ "mtsrr2",    XSPR(31,467,990),  XSPR_MASK, PPC403,	{ RS } },
-{ "mtsrr3",    XSPR(31,467,991),  XSPR_MASK, PPC403,	{ RS } },
-{ "mtl2cr",    XSPR(31,467,1017), XSPR_MASK, PPC750,    { RS } },
-{ "mtdccr",    XSPR(31,467,1018), XSPR_MASK, PPC403,	{ RS } },
-{ "mticcr",    XSPR(31,467,1019), XSPR_MASK, PPC403,	{ RS } },
-{ "mtictc",    XSPR(31,467,1019), XSPR_MASK, PPC750,    { RS } },
-{ "mtpbl1",    XSPR(31,467,1020), XSPR_MASK, PPC403,	{ RS } },
-{ "mtthrm1",   XSPR(31,467,1020), XSPR_MASK, PPC750,    { RS } },
-{ "mtpbu1",    XSPR(31,467,1021), XSPR_MASK, PPC403,	{ RS } },
-{ "mtthrm2",   XSPR(31,467,1021), XSPR_MASK, PPC750,    { RS } },
-{ "mtpbl2",    XSPR(31,467,1022), XSPR_MASK, PPC403,	{ RS } },
-{ "mtthrm3",   XSPR(31,467,1022), XSPR_MASK, PPC750,    { RS } },
-{ "mtpbu2",    XSPR(31,467,1023), XSPR_MASK, PPC403,	{ RS } },
-{ "mtspr",     X(31,467),	  X_MASK,    COM,	{ SPR, RS } },
-
-{ "dcbi",    X(31,470),	XRT_MASK,	PPC,		{ RA, RB } },
-
-{ "nand",    XRC(31,476,0), X_MASK,	COM,		{ RA, RS, RB } },
-{ "nand.",   XRC(31,476,1), X_MASK,	COM,		{ RA, RS, RB } },
-
-{ "dcbie",   X(31,478),	XRT_MASK,	BOOKE64,	{ RA, RB } },
-
-{ "dcread",  X(31,486),	X_MASK,		PPC403|PPC440,	{ RT, RA, RB }},
-
-{ "mtpmr",   X(31,462),	X_MASK,		PPCPMR,		{ PMR, RS }},
-
-{ "icbtls",  X(31,486),	X_MASK,		PPCCHLK,	{ CT, RA, RB }},
-
-{ "nabs",    XO(31,488,0,0), XORB_MASK, M601,		{ RT, RA } },
-{ "subfme64",XO(31,488,0,0), XORB_MASK, BOOKE64,	{ RT, RA } },
-{ "nabs.",   XO(31,488,0,1), XORB_MASK, M601,		{ RT, RA } },
-{ "nabso",   XO(31,488,1,0), XORB_MASK, M601,		{ RT, RA } },
-{ "subfme64o",XO(31,488,1,0), XORB_MASK, BOOKE64,	{ RT, RA } },
-{ "nabso.",  XO(31,488,1,1), XORB_MASK, M601,		{ RT, RA } },
-
-{ "divd",    XO(31,489,0,0), XO_MASK,	PPC64,		{ RT, RA, RB } },
-{ "divd.",   XO(31,489,0,1), XO_MASK,	PPC64,		{ RT, RA, RB } },
-{ "divdo",   XO(31,489,1,0), XO_MASK,	PPC64,		{ RT, RA, RB } },
-{ "divdo.",  XO(31,489,1,1), XO_MASK,	PPC64,		{ RT, RA, RB } },
-
-{ "addme64", XO(31,490,0,0), XORB_MASK, BOOKE64,	{ RT, RA } },
-{ "addme64o",XO(31,490,1,0), XORB_MASK, BOOKE64,	{ RT, RA } },
-
-{ "divw",    XO(31,491,0,0), XO_MASK,	PPC,		{ RT, RA, RB } },
-{ "divw.",   XO(31,491,0,1), XO_MASK,	PPC,		{ RT, RA, RB } },
-{ "divwo",   XO(31,491,1,0), XO_MASK,	PPC,		{ RT, RA, RB } },
-{ "divwo.",  XO(31,491,1,1), XO_MASK,	PPC,		{ RT, RA, RB } },
-
-{ "icbtlse", X(31,494),	X_MASK,		PPCCHLK64,	{ CT, RA, RB }},
-
-{ "slbia",   X(31,498),	0xffffffff,	PPC64,		{ 0 } },
-
-{ "cli",     X(31,502), XRB_MASK,	POWER,		{ RT, RA } },
-
-{ "stdcxe.", XRC(31,511,1), X_MASK,	BOOKE64,	{ RS, RA, RB } },
-
-{ "mcrxr",   X(31,512),	XRARB_MASK|(3<<21), COM,	{ BF } },
-
-{ "bblels",  X(31,518),	X_MASK,		PPCBRLK,	{ 0 }},
-{ "mcrxr64", X(31,544),	XRARB_MASK|(3<<21), BOOKE64,	{ BF } },
-
-{ "clcs",    X(31,531), XRB_MASK,	M601,		{ RT, RA } },
-
-{ "ldbrx",   X(31,532),	X_MASK,		CELL,		{ RT, RA0, RB } },
-
-{ "lswx",    X(31,533),	X_MASK,		PPCCOM,		{ RT, RA0, RB } },
-{ "lsx",     X(31,533),	X_MASK,		PWRCOM,		{ RT, RA, RB } },
-
-{ "lwbrx",   X(31,534),	X_MASK,		PPCCOM,		{ RT, RA0, RB } },
-{ "lbrx",    X(31,534),	X_MASK,		PWRCOM,		{ RT, RA, RB } },
-
-{ "lfsx",    X(31,535),	X_MASK,		COM,		{ FRT, RA0, RB } },
-
-{ "srw",     XRC(31,536,0), X_MASK,	PPCCOM,		{ RA, RS, RB } },
-{ "sr",      XRC(31,536,0), X_MASK,	PWRCOM,		{ RA, RS, RB } },
-{ "srw.",    XRC(31,536,1), X_MASK,	PPCCOM,		{ RA, RS, RB } },
-{ "sr.",     XRC(31,536,1), X_MASK,	PWRCOM,		{ RA, RS, RB } },
-
-{ "rrib",    XRC(31,537,0), X_MASK,	M601,		{ RA, RS, RB } },
-{ "rrib.",   XRC(31,537,1), X_MASK,	M601,		{ RA, RS, RB } },
-
-{ "srd",     XRC(31,539,0), X_MASK,	PPC64,		{ RA, RS, RB } },
-{ "srd.",    XRC(31,539,1), X_MASK,	PPC64,		{ RA, RS, RB } },
-
-{ "maskir",  XRC(31,541,0), X_MASK,	M601,		{ RA, RS, RB } },
-{ "maskir.", XRC(31,541,1), X_MASK,	M601,		{ RA, RS, RB } },
-
-{ "lwbrxe",  X(31,542),	X_MASK,		BOOKE64,	{ RT, RA0, RB } },
-
-{ "lfsxe",   X(31,543),	X_MASK,		BOOKE64,	{ FRT, RA0, RB } },
-
-{ "bbelr",   X(31,550),	X_MASK,		PPCBRLK,	{ 0 }},
-
-{ "tlbsync", X(31,566),	0xffffffff,	PPC,		{ 0 } },
-
-{ "lfsux",   X(31,567),	X_MASK,		COM,		{ FRT, RAS, RB } },
-
-{ "lfsuxe",  X(31,575),	X_MASK,		BOOKE64,	{ FRT, RAS, RB } },
-
-{ "mfsr",    X(31,595),	XRB_MASK|(1<<20), COM32,	{ RT, SR } },
-
-{ "lswi",    X(31,597),	X_MASK,		PPCCOM,		{ RT, RA0, NB } },
-{ "lsi",     X(31,597),	X_MASK,		PWRCOM,		{ RT, RA0, NB } },
-
-{ "lwsync",  XSYNC(31,598,1), 0xffffffff, PPC,		{ 0 } },
-{ "ptesync", XSYNC(31,598,2), 0xffffffff, PPC64,	{ 0 } },
-{ "msync",   X(31,598), 0xffffffff,	BOOKE,		{ 0 } },
-{ "sync",    X(31,598), XSYNC_MASK,	PPCCOM,		{ LS } },
-{ "dcs",     X(31,598), 0xffffffff,	PWRCOM,		{ 0 } },
-
-{ "lfdx",    X(31,599), X_MASK,		COM,		{ FRT, RA0, RB } },
-
-{ "lfdxe",   X(31,607), X_MASK,		BOOKE64,	{ FRT, RA0, RB } },
-
-{ "mffgpr",  XRC(31,607,0), XRA_MASK,	POWER6,		{ FRT, RB } },
-
-{ "mfsri",   X(31,627), X_MASK,		PWRCOM,		{ RT, RA, RB } },
-
-{ "dclst",   X(31,630), XRB_MASK,	PWRCOM,		{ RS, RA } },
-
-{ "lfdux",   X(31,631), X_MASK,		COM,		{ FRT, RAS, RB } },
-
-{ "lfduxe",  X(31,639), X_MASK,		BOOKE64,	{ FRT, RAS, RB } },
-
-{ "mfsrin",  X(31,659), XRA_MASK,	PPC32,		{ RT, RB } },
-
-{ "stdbrx",  X(31,660), X_MASK,		CELL,		{ RS, RA0, RB } },
-
-{ "stswx",   X(31,661), X_MASK,		PPCCOM,		{ RS, RA0, RB } },
-{ "stsx",    X(31,661), X_MASK,		PWRCOM,		{ RS, RA0, RB } },
-
-{ "stwbrx",  X(31,662), X_MASK,		PPCCOM,		{ RS, RA0, RB } },
-{ "stbrx",   X(31,662), X_MASK,		PWRCOM,		{ RS, RA0, RB } },
-
-{ "stfsx",   X(31,663), X_MASK,		COM,		{ FRS, RA0, RB } },
-
-{ "srq",     XRC(31,664,0), X_MASK,	M601,		{ RA, RS, RB } },
-{ "srq.",    XRC(31,664,1), X_MASK,	M601,		{ RA, RS, RB } },
-
-{ "sre",     XRC(31,665,0), X_MASK,	M601,		{ RA, RS, RB } },
-{ "sre.",    XRC(31,665,1), X_MASK,	M601,		{ RA, RS, RB } },
-
-{ "stwbrxe", X(31,670), X_MASK,		BOOKE64,	{ RS, RA0, RB } },
-
-{ "stfsxe",  X(31,671), X_MASK,		BOOKE64,	{ FRS, RA0, RB } },
-
-{ "stfsux",  X(31,695),	X_MASK,		COM,		{ FRS, RAS, RB } },
-
-{ "sriq",    XRC(31,696,0), X_MASK,	M601,		{ RA, RS, SH } },
-{ "sriq.",   XRC(31,696,1), X_MASK,	M601,		{ RA, RS, SH } },
-
-{ "stfsuxe", X(31,703),	X_MASK,		BOOKE64,	{ FRS, RAS, RB } },
-
-{ "stswi",   X(31,725),	X_MASK,		PPCCOM,		{ RS, RA0, NB } },
-{ "stsi",    X(31,725),	X_MASK,		PWRCOM,		{ RS, RA0, NB } },
-
-{ "stfdx",   X(31,727),	X_MASK,		COM,		{ FRS, RA0, RB } },
-
-{ "srlq",    XRC(31,728,0), X_MASK,	M601,		{ RA, RS, RB } },
-{ "srlq.",   XRC(31,728,1), X_MASK,	M601,		{ RA, RS, RB } },
-
-{ "sreq",    XRC(31,729,0), X_MASK,	M601,		{ RA, RS, RB } },
-{ "sreq.",   XRC(31,729,1), X_MASK,	M601,		{ RA, RS, RB } },
-
-{ "stfdxe",  X(31,735),	X_MASK,		BOOKE64,	{ FRS, RA0, RB } },
-
-{ "mftgpr",  XRC(31,735,0), XRA_MASK,	POWER6,		{ RT, FRB } },
-
-{ "dcba",    X(31,758),	XRT_MASK,	PPC405 | BOOKE,	{ RA, RB } },
-
-{ "stfdux",  X(31,759),	X_MASK,		COM,		{ FRS, RAS, RB } },
-
-{ "srliq",   XRC(31,760,0), X_MASK,	M601,		{ RA, RS, SH } },
-{ "srliq.",  XRC(31,760,1), X_MASK,	M601,		{ RA, RS, SH } },
-
-{ "dcbae",   X(31,766),	XRT_MASK,	BOOKE64,	{ RA, RB } },
-
-{ "stfduxe", X(31,767),	X_MASK,		BOOKE64,	{ FRS, RAS, RB } },
-
-{ "tlbivax", X(31,786),	XRT_MASK,	BOOKE,		{ RA, RB } },
-{ "tlbivaxe",X(31,787),	XRT_MASK,	BOOKE64,	{ RA, RB } },
-
-{ "lwzcix",  X(31,789),	X_MASK,		POWER6,		{ RT, RA0, RB } },
-
-{ "lhbrx",   X(31,790),	X_MASK,		COM,		{ RT, RA0, RB } },
-
-{ "sraw",    XRC(31,792,0), X_MASK,	PPCCOM,		{ RA, RS, RB } },
-{ "sra",     XRC(31,792,0), X_MASK,	PWRCOM,		{ RA, RS, RB } },
-{ "sraw.",   XRC(31,792,1), X_MASK,	PPCCOM,		{ RA, RS, RB } },
-{ "sra.",    XRC(31,792,1), X_MASK,	PWRCOM,		{ RA, RS, RB } },
-
-{ "srad",    XRC(31,794,0), X_MASK,	PPC64,		{ RA, RS, RB } },
-{ "srad.",   XRC(31,794,1), X_MASK,	PPC64,		{ RA, RS, RB } },
-
-{ "lhbrxe",  X(31,798),	X_MASK,		BOOKE64,	{ RT, RA0, RB } },
-
-{ "ldxe",    X(31,799),	X_MASK,		BOOKE64,	{ RT, RA0, RB } },
-{ "lduxe",   X(31,831),	X_MASK,		BOOKE64,	{ RT, RA0, RB } },
-
-{ "rac",     X(31,818),	X_MASK,		PWRCOM,		{ RT, RA, RB } },
-
-{ "lhzcix",  X(31,821),	X_MASK,		POWER6,		{ RT, RA0, RB } },
-
-{ "dss",     XDSS(31,822,0), XDSS_MASK,	PPCVEC,		{ STRM } },
-{ "dssall",  XDSS(31,822,1), XDSS_MASK,	PPCVEC,		{ 0 } },
-
-{ "srawi",   XRC(31,824,0), X_MASK,	PPCCOM,		{ RA, RS, SH } },
-{ "srai",    XRC(31,824,0), X_MASK,	PWRCOM,		{ RA, RS, SH } },
-{ "srawi.",  XRC(31,824,1), X_MASK,	PPCCOM,		{ RA, RS, SH } },
-{ "srai.",   XRC(31,824,1), X_MASK,	PWRCOM,		{ RA, RS, SH } },
-
-{ "slbmfev", X(31,851), XRA_MASK,	PPC64,		{ RT, RB } },
-
-{ "lbzcix",  X(31,853),	X_MASK,		POWER6,		{ RT, RA0, RB } },
-
-{ "mbar",    X(31,854),	X_MASK,		BOOKE,		{ MO } },
-{ "eieio",   X(31,854),	0xffffffff,	PPC,		{ 0 } },
-
-{ "lfiwax",  X(31,855),	X_MASK,		POWER6,		{ FRT, RA0, RB } },
-
-{ "ldcix",   X(31,885),	X_MASK,		POWER6,		{ RT, RA0, RB } },
-
-{ "tlbsx",   XRC(31,914,0), X_MASK, 	PPC403|BOOKE,	{ RTO, RA, RB } },
-{ "tlbsx.",  XRC(31,914,1), X_MASK, 	PPC403|BOOKE,	{ RTO, RA, RB } },
-{ "tlbsxe",  XRC(31,915,0), X_MASK,	BOOKE64,	{ RTO, RA, RB } },
-{ "tlbsxe.", XRC(31,915,1), X_MASK,	BOOKE64,	{ RTO, RA, RB } },
-
-{ "slbmfee", X(31,915), XRA_MASK,	PPC64,		{ RT, RB } },
-
-{ "stwcix",  X(31,917),	X_MASK,		POWER6,		{ RS, RA0, RB } },
-
-{ "sthbrx",  X(31,918),	X_MASK,		COM,		{ RS, RA0, RB } },
-
-{ "sraq",    XRC(31,920,0), X_MASK,	M601,		{ RA, RS, RB } },
-{ "sraq.",   XRC(31,920,1), X_MASK,	M601,		{ RA, RS, RB } },
-
-{ "srea",    XRC(31,921,0), X_MASK,	M601,		{ RA, RS, RB } },
-{ "srea.",   XRC(31,921,1), X_MASK,	M601,		{ RA, RS, RB } },
-
-{ "extsh",   XRC(31,922,0), XRB_MASK,	PPCCOM,		{ RA, RS } },
-{ "exts",    XRC(31,922,0), XRB_MASK,	PWRCOM,		{ RA, RS } },
-{ "extsh.",  XRC(31,922,1), XRB_MASK,	PPCCOM,		{ RA, RS } },
-{ "exts.",   XRC(31,922,1), XRB_MASK,	PWRCOM,		{ RA, RS } },
-
-{ "sthbrxe", X(31,926),	X_MASK,		BOOKE64,	{ RS, RA0, RB } },
-
-{ "stdxe",   X(31,927), X_MASK,		BOOKE64,	{ RS, RA0, RB } },
-
-{ "tlbrehi", XTLB(31,946,0), XTLB_MASK,	PPC403,		{ RT, RA } },
-{ "tlbrelo", XTLB(31,946,1), XTLB_MASK,	PPC403,		{ RT, RA } },
-{ "tlbre",   X(31,946),	X_MASK,		PPC403|BOOKE,	{ RSO, RAOPT, SHO } },
-
-{ "sthcix",  X(31,949),	X_MASK,		POWER6,		{ RS, RA0, RB } },
-
-{ "sraiq",   XRC(31,952,0), X_MASK,	M601,		{ RA, RS, SH } },
-{ "sraiq.",  XRC(31,952,1), X_MASK,	M601,		{ RA, RS, SH } },
-
-{ "extsb",   XRC(31,954,0), XRB_MASK,	PPC,		{ RA, RS} },
-{ "extsb.",  XRC(31,954,1), XRB_MASK,	PPC,		{ RA, RS} },
-
-{ "stduxe",  X(31,959),	X_MASK,		BOOKE64,	{ RS, RAS, RB } },
-
-{ "iccci",   X(31,966),	XRT_MASK,	PPC403|PPC440,	{ RA, RB } },
-
-{ "tlbwehi", XTLB(31,978,0), XTLB_MASK,	PPC403,		{ RT, RA } },
-{ "tlbwelo", XTLB(31,978,1), XTLB_MASK,	PPC403,		{ RT, RA } },
-{ "tlbwe",   X(31,978),	X_MASK,		PPC403|BOOKE,	{ RSO, RAOPT, SHO } },
-{ "tlbld",   X(31,978),	XRTRA_MASK,	PPC,		{ RB } },
-
-{ "stbcix",  X(31,981),	X_MASK,		POWER6,		{ RS, RA0, RB } },
-
-{ "icbi",    X(31,982),	XRT_MASK,	PPC,		{ RA, RB } },
-
-{ "stfiwx",  X(31,983),	X_MASK,		PPC,		{ FRS, RA0, RB } },
-
-{ "extsw",   XRC(31,986,0), XRB_MASK,	PPC64 | BOOKE64,{ RA, RS } },
-{ "extsw.",  XRC(31,986,1), XRB_MASK,	PPC64,		{ RA, RS } },
-
-{ "icread",  X(31,998),	XRT_MASK,	PPC403|PPC440,	{ RA, RB } },
-
-{ "icbie",   X(31,990),	XRT_MASK,	BOOKE64,	{ RA, RB } },
-{ "stfiwxe", X(31,991),	X_MASK,		BOOKE64,	{ FRS, RA0, RB } },
-
-{ "tlbli",   X(31,1010), XRTRA_MASK,	PPC,		{ RB } },
-
-{ "stdcix",  X(31,1013), X_MASK,	POWER6,		{ RS, RA0, RB } },
-
-{ "dcbzl",   XOPL(31,1014,1), XRT_MASK,POWER4,            { RA, RB } },
-{ "dcbz",    X(31,1014), XRT_MASK,	PPC,		{ RA, RB } },
-{ "dclz",    X(31,1014), XRT_MASK,	PPC,		{ RA, RB } },
-
-{ "dcbze",   X(31,1022), XRT_MASK,	BOOKE64,	{ RA, RB } },
-
-{ "lvebx",   X(31,   7), X_MASK,	PPCVEC,		{ VD, RA, RB } },
-{ "lvehx",   X(31,  39), X_MASK,	PPCVEC,		{ VD, RA, RB } },
-{ "lvewx",   X(31,  71), X_MASK,	PPCVEC,		{ VD, RA, RB } },
-{ "lvsl",    X(31,   6), X_MASK,	PPCVEC,		{ VD, RA, RB } },
-{ "lvsr",    X(31,  38), X_MASK,	PPCVEC,		{ VD, RA, RB } },
-{ "lvx",     X(31, 103), X_MASK,	PPCVEC,		{ VD, RA, RB } },
-{ "lvxl",    X(31, 359), X_MASK,	PPCVEC,		{ VD, RA, RB } },
-{ "stvebx",  X(31, 135), X_MASK,	PPCVEC,		{ VS, RA, RB } },
-{ "stvehx",  X(31, 167), X_MASK,	PPCVEC,		{ VS, RA, RB } },
-{ "stvewx",  X(31, 199), X_MASK,	PPCVEC,		{ VS, RA, RB } },
-{ "stvx",    X(31, 231), X_MASK,	PPCVEC,		{ VS, RA, RB } },
-{ "stvxl",   X(31, 487), X_MASK,	PPCVEC,		{ VS, RA, RB } },
-
-/* New load/store left/right index vector instructions that are in the Cell only.  */
-{ "lvlx",    X(31, 519), X_MASK,	CELL,		{ VD, RA0, RB } },
-{ "lvlxl",   X(31, 775), X_MASK,	CELL,		{ VD, RA0, RB } },
-{ "lvrx",    X(31, 551), X_MASK,	CELL,		{ VD, RA0, RB } },
-{ "lvrxl",   X(31, 807), X_MASK,	CELL,		{ VD, RA0, RB } },
-{ "stvlx",   X(31, 647), X_MASK,	CELL,		{ VS, RA0, RB } },
-{ "stvlxl",  X(31, 903), X_MASK,	CELL,		{ VS, RA0, RB } },
-{ "stvrx",   X(31, 679), X_MASK,	CELL,		{ VS, RA0, RB } },
-{ "stvrxl",  X(31, 935), X_MASK,	CELL,		{ VS, RA0, RB } },
-
-{ "lwz",     OP(32),	OP_MASK,	PPCCOM,		{ RT, D, RA0 } },
-{ "l",	     OP(32),	OP_MASK,	PWRCOM,		{ RT, D, RA0 } },
-
-{ "lwzu",    OP(33),	OP_MASK,	PPCCOM,		{ RT, D, RAL } },
-{ "lu",      OP(33),	OP_MASK,	PWRCOM,		{ RT, D, RA0 } },
-
-{ "lbz",     OP(34),	OP_MASK,	COM,		{ RT, D, RA0 } },
-
-{ "lbzu",    OP(35),	OP_MASK,	COM,		{ RT, D, RAL } },
-
-{ "stw",     OP(36),	OP_MASK,	PPCCOM,		{ RS, D, RA0 } },
-{ "st",      OP(36),	OP_MASK,	PWRCOM,		{ RS, D, RA0 } },
-
-{ "stwu",    OP(37),	OP_MASK,	PPCCOM,		{ RS, D, RAS } },
-{ "stu",     OP(37),	OP_MASK,	PWRCOM,		{ RS, D, RA0 } },
-
-{ "stb",     OP(38),	OP_MASK,	COM,		{ RS, D, RA0 } },
-
-{ "stbu",    OP(39),	OP_MASK,	COM,		{ RS, D, RAS } },
-
-{ "lhz",     OP(40),	OP_MASK,	COM,		{ RT, D, RA0 } },
-
-{ "lhzu",    OP(41),	OP_MASK,	COM,		{ RT, D, RAL } },
-
-{ "lha",     OP(42),	OP_MASK,	COM,		{ RT, D, RA0 } },
-
-{ "lhau",    OP(43),	OP_MASK,	COM,		{ RT, D, RAL } },
-
-{ "sth",     OP(44),	OP_MASK,	COM,		{ RS, D, RA0 } },
-
-{ "sthu",    OP(45),	OP_MASK,	COM,		{ RS, D, RAS } },
-
-{ "lmw",     OP(46),	OP_MASK,	PPCCOM,		{ RT, D, RAM } },
-{ "lm",      OP(46),	OP_MASK,	PWRCOM,		{ RT, D, RA0 } },
-
-{ "stmw",    OP(47),	OP_MASK,	PPCCOM,		{ RS, D, RA0 } },
-{ "stm",     OP(47),	OP_MASK,	PWRCOM,		{ RS, D, RA0 } },
-
-{ "lfs",     OP(48),	OP_MASK,	COM,		{ FRT, D, RA0 } },
-
-{ "lfsu",    OP(49),	OP_MASK,	COM,		{ FRT, D, RAS } },
-
-{ "lfd",     OP(50),	OP_MASK,	COM,		{ FRT, D, RA0 } },
-
-{ "lfdu",    OP(51),	OP_MASK,	COM,		{ FRT, D, RAS } },
-
-{ "stfs",    OP(52),	OP_MASK,	COM,		{ FRS, D, RA0 } },
-
-{ "stfsu",   OP(53),	OP_MASK,	COM,		{ FRS, D, RAS } },
-
-{ "stfd",    OP(54),	OP_MASK,	COM,		{ FRS, D, RA0 } },
-
-{ "stfdu",   OP(55),	OP_MASK,	COM,		{ FRS, D, RAS } },
-
-{ "lq",      OP(56),	OP_MASK,	POWER4,		{ RTQ, DQ, RAQ } },
-
-{ "lfq",     OP(56),	OP_MASK,	POWER2,		{ FRT, D, RA0 } },
-
-{ "lfqu",    OP(57),	OP_MASK,	POWER2,		{ FRT, D, RA0 } },
-
-{ "lfdp",    OP(57),	OP_MASK,	POWER6,		{ FRT, D, RA0 } },
-
-{ "lbze",    DEO(58,0), DE_MASK,	BOOKE64,	{ RT, DE, RA0 } },
-{ "lbzue",   DEO(58,1), DE_MASK,	BOOKE64,	{ RT, DE, RAL } },
-{ "lhze",    DEO(58,2), DE_MASK,	BOOKE64,	{ RT, DE, RA0 } },
-{ "lhzue",   DEO(58,3), DE_MASK,	BOOKE64,	{ RT, DE, RAL } },
-{ "lhae",    DEO(58,4), DE_MASK,	BOOKE64,	{ RT, DE, RA0 } },
-{ "lhaue",   DEO(58,5), DE_MASK,	BOOKE64,	{ RT, DE, RAL } },
-{ "lwze",    DEO(58,6), DE_MASK,	BOOKE64,	{ RT, DE, RA0 } },
-{ "lwzue",   DEO(58,7), DE_MASK,	BOOKE64,	{ RT, DE, RAL } },
-{ "stbe",    DEO(58,8), DE_MASK,	BOOKE64,	{ RS, DE, RA0 } },
-{ "stbue",   DEO(58,9), DE_MASK,	BOOKE64,	{ RS, DE, RAS } },
-{ "sthe",    DEO(58,10), DE_MASK,	BOOKE64,	{ RS, DE, RA0 } },
-{ "sthue",   DEO(58,11), DE_MASK,	BOOKE64,	{ RS, DE, RAS } },
-{ "stwe",    DEO(58,14), DE_MASK,	BOOKE64,	{ RS, DE, RA0 } },
-{ "stwue",   DEO(58,15), DE_MASK,	BOOKE64,	{ RS, DE, RAS } },
-
-{ "ld",      DSO(58,0),	DS_MASK,	PPC64,		{ RT, DS, RA0 } },
-
-{ "ldu",     DSO(58,1), DS_MASK,	PPC64,		{ RT, DS, RAL } },
-
-{ "lwa",     DSO(58,2), DS_MASK,	PPC64,		{ RT, DS, RA0 } },
-
-{ "dadd",    XRC(59,2,0), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-{ "dadd.",   XRC(59,2,1), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-
-{ "dqua",    ZRC(59,3,0), Z2_MASK,	POWER6,		{ FRT, FRA, FRB, RMC } },
-{ "dqua.",   ZRC(59,3,1), Z2_MASK,	POWER6,		{ FRT, FRA, FRB, RMC } },
-
-{ "fdivs",   A(59,18,0), AFRC_MASK,	PPC,		{ FRT, FRA, FRB } },
-{ "fdivs.",  A(59,18,1), AFRC_MASK,	PPC,		{ FRT, FRA, FRB } },
-
-{ "fsubs",   A(59,20,0), AFRC_MASK,	PPC,		{ FRT, FRA, FRB } },
-{ "fsubs.",  A(59,20,1), AFRC_MASK,	PPC,		{ FRT, FRA, FRB } },
-
-{ "fadds",   A(59,21,0), AFRC_MASK,	PPC,		{ FRT, FRA, FRB } },
-{ "fadds.",  A(59,21,1), AFRC_MASK,	PPC,		{ FRT, FRA, FRB } },
-
-{ "fsqrts",  A(59,22,0), AFRAFRC_MASK,	PPC,		{ FRT, FRB } },
-{ "fsqrts.", A(59,22,1), AFRAFRC_MASK,	PPC,		{ FRT, FRB } },
-
-{ "fres",    A(59,24,0), AFRALFRC_MASK,	PPC,		{ FRT, FRB, A_L } },
-{ "fres.",   A(59,24,1), AFRALFRC_MASK,	PPC,		{ FRT, FRB, A_L } },
-
-{ "fmuls",   A(59,25,0), AFRB_MASK,	PPC,		{ FRT, FRA, FRC } },
-{ "fmuls.",  A(59,25,1), AFRB_MASK,	PPC,		{ FRT, FRA, FRC } },
-
-{ "frsqrtes", A(59,26,0), AFRALFRC_MASK,POWER5,		{ FRT, FRB, A_L } },
-{ "frsqrtes.",A(59,26,1), AFRALFRC_MASK,POWER5,		{ FRT, FRB, A_L } },
-
-{ "fmsubs",  A(59,28,0), A_MASK,	PPC,		{ FRT,FRA,FRC,FRB } },
-{ "fmsubs.", A(59,28,1), A_MASK,	PPC,		{ FRT,FRA,FRC,FRB } },
-
-{ "fmadds",  A(59,29,0), A_MASK,	PPC,		{ FRT,FRA,FRC,FRB } },
-{ "fmadds.", A(59,29,1), A_MASK,	PPC,		{ FRT,FRA,FRC,FRB } },
-
-{ "fnmsubs", A(59,30,0), A_MASK,	PPC,		{ FRT,FRA,FRC,FRB } },
-{ "fnmsubs.",A(59,30,1), A_MASK,	PPC,		{ FRT,FRA,FRC,FRB } },
-
-{ "fnmadds", A(59,31,0), A_MASK,	PPC,		{ FRT,FRA,FRC,FRB } },
-{ "fnmadds.",A(59,31,1), A_MASK,	PPC,		{ FRT,FRA,FRC,FRB } },
-
-{ "dmul",    XRC(59,34,0), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-{ "dmul.",   XRC(59,34,1), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-
-{ "drrnd",   ZRC(59,35,0), Z2_MASK,	POWER6,		{ FRT, FRA, FRB, RMC } },
-{ "drrnd.",  ZRC(59,35,1), Z2_MASK,	POWER6,		{ FRT, FRA, FRB, RMC } },
-
-{ "dscli",   ZRC(59,66,0), Z_MASK,	POWER6,		{ FRT, FRA, SH16 } },
-{ "dscli.",  ZRC(59,66,1), Z_MASK,	POWER6,		{ FRT, FRA, SH16 } },
-
-{ "dquai",   ZRC(59,67,0), Z2_MASK,	POWER6,		{ TE,  FRT, FRB, RMC } },
-{ "dquai.",  ZRC(59,67,1), Z2_MASK,	POWER6,		{ TE,  FRT, FRB, RMC } },
-
-{ "dscri",   ZRC(59,98,0), Z_MASK,	POWER6,		{ FRT, FRA, SH16 } },
-{ "dscri.",  ZRC(59,98,1), Z_MASK,	POWER6,		{ FRT, FRA, SH16 } },
-
-{ "drintx",  ZRC(59,99,0), Z2_MASK,	POWER6,		{ R, FRT, FRB, RMC } },
-{ "drintx.", ZRC(59,99,1), Z2_MASK,	POWER6,		{ R, FRT, FRB, RMC } },
-
-{ "dcmpo",   X(59,130),	   X_MASK,	POWER6,		{ BF,  FRA, FRB } },
-
-{ "dtstex",  X(59,162),	   X_MASK,	POWER6,		{ BF,  FRA, FRB } },
-{ "dtstdc",  Z(59,194),	   Z_MASK,	POWER6,		{ BF,  FRA, DCM } },
-{ "dtstdg",  Z(59,226),	   Z_MASK,	POWER6,		{ BF,  FRA, DGM } },
-
-{ "drintn",  ZRC(59,227,0), Z2_MASK,	POWER6,		{ R, FRT, FRB, RMC } },
-{ "drintn.", ZRC(59,227,1), Z2_MASK,	POWER6,		{ R, FRT, FRB, RMC } },
-
-{ "dctdp",   XRC(59,258,0), X_MASK,	POWER6,		{ FRT, FRB } },
-{ "dctdp.",  XRC(59,258,1), X_MASK,	POWER6,		{ FRT, FRB } },
-
-{ "dctfix",  XRC(59,290,0), X_MASK,	POWER6,		{ FRT, FRB } },
-{ "dctfix.", XRC(59,290,1), X_MASK,	POWER6,		{ FRT, FRB } },
-
-{ "ddedpd",  XRC(59,322,0), X_MASK,	POWER6,		{ SP, FRT, FRB } },
-{ "ddedpd.", XRC(59,322,1), X_MASK,	POWER6,		{ SP, FRT, FRB } },
-
-{ "dxex",    XRC(59,354,0), X_MASK,	POWER6,		{ FRT, FRB } },
-{ "dxex.",   XRC(59,354,1), X_MASK,	POWER6,		{ FRT, FRB } },
-
-{ "dsub",    XRC(59,514,0), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-{ "dsub.",   XRC(59,514,1), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-
-{ "ddiv",    XRC(59,546,0), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-{ "ddiv.",   XRC(59,546,1), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-
-{ "dcmpu",   X(59,642),	    X_MASK,	POWER6,		{ BF,  FRA, FRB } },
-
-{ "dtstsf",  X(59,674),	   X_MASK,	POWER6,		{ BF,  FRA, FRB } },
-
-{ "drsp",    XRC(59,770,0), X_MASK,	POWER6,		{ FRT, FRB } },
-{ "drsp.",   XRC(59,770,1), X_MASK,	POWER6,		{ FRT, FRB } },
-
-{ "dcffix",  XRC(59,802,0), X_MASK,	POWER6,		{ FRT, FRB } },
-{ "dcffix.", XRC(59,802,1), X_MASK,	POWER6,		{ FRT, FRB } },
-
-{ "denbcd",  XRC(59,834,0), X_MASK,	POWER6,		{ S, FRT, FRB } },
-{ "denbcd.", XRC(59,834,1), X_MASK,	POWER6,		{ S, FRT, FRB } },
-
-{ "diex",    XRC(59,866,0), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-{ "diex.",   XRC(59,866,1), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-
-{ "stfq",    OP(60),	OP_MASK,	POWER2,		{ FRS, D, RA } },
-
-{ "stfqu",   OP(61),	OP_MASK,	POWER2,		{ FRS, D, RA } },
-
-{ "stfdp",   OP(61),	OP_MASK,	POWER6,		{ FRT, D, RA0 } },
-
-{ "lde",     DEO(62,0), DE_MASK,	BOOKE64,	{ RT, DES, RA0 } },
-{ "ldue",    DEO(62,1), DE_MASK,	BOOKE64,	{ RT, DES, RA0 } },
-{ "lfse",    DEO(62,4), DE_MASK,	BOOKE64,	{ FRT, DES, RA0 } },
-{ "lfsue",   DEO(62,5), DE_MASK,	BOOKE64,	{ FRT, DES, RAS } },
-{ "lfde",    DEO(62,6), DE_MASK,	BOOKE64,	{ FRT, DES, RA0 } },
-{ "lfdue",   DEO(62,7), DE_MASK,	BOOKE64,	{ FRT, DES, RAS } },
-{ "stde",    DEO(62,8), DE_MASK,	BOOKE64,	{ RS, DES, RA0 } },
-{ "stdue",   DEO(62,9), DE_MASK,	BOOKE64,	{ RS, DES, RAS } },
-{ "stfse",   DEO(62,12), DE_MASK,	BOOKE64,	{ FRS, DES, RA0 } },
-{ "stfsue",  DEO(62,13), DE_MASK,	BOOKE64,	{ FRS, DES, RAS } },
-{ "stfde",   DEO(62,14), DE_MASK,	BOOKE64,	{ FRS, DES, RA0 } },
-{ "stfdue",  DEO(62,15), DE_MASK,	BOOKE64,	{ FRS, DES, RAS } },
-
-{ "std",     DSO(62,0),	DS_MASK,	PPC64,		{ RS, DS, RA0 } },
-
-{ "stdu",    DSO(62,1),	DS_MASK,	PPC64,		{ RS, DS, RAS } },
-
-{ "stq",     DSO(62,2),	DS_MASK,	POWER4,		{ RSQ, DS, RA0 } },
-
-{ "fcmpu",   X(63,0),	X_MASK|(3<<21),	COM,		{ BF, FRA, FRB } },
-
-{ "daddq",   XRC(63,2,0), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-{ "daddq.",  XRC(63,2,1), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-
-{ "dquaq",   ZRC(63,3,0), Z2_MASK,	POWER6,		{ FRT, FRA, FRB, RMC } },
-{ "dquaq.",  ZRC(63,3,1), Z2_MASK,	POWER6,		{ FRT, FRA, FRB, RMC } },
-
-{ "fcpsgn",  XRC(63,8,0), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-{ "fcpsgn.", XRC(63,8,1), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-
-{ "frsp",    XRC(63,12,0), XRA_MASK,	COM,		{ FRT, FRB } },
-{ "frsp.",   XRC(63,12,1), XRA_MASK,	COM,		{ FRT, FRB } },
-
-{ "fctiw",   XRC(63,14,0), XRA_MASK,	PPCCOM,		{ FRT, FRB } },
-{ "fcir",    XRC(63,14,0), XRA_MASK,	POWER2,		{ FRT, FRB } },
-{ "fctiw.",  XRC(63,14,1), XRA_MASK,	PPCCOM,		{ FRT, FRB } },
-{ "fcir.",   XRC(63,14,1), XRA_MASK,	POWER2,		{ FRT, FRB } },
-
-{ "fctiwz",  XRC(63,15,0), XRA_MASK,	PPCCOM,		{ FRT, FRB } },
-{ "fcirz",   XRC(63,15,0), XRA_MASK,	POWER2,		{ FRT, FRB } },
-{ "fctiwz.", XRC(63,15,1), XRA_MASK,	PPCCOM,		{ FRT, FRB } },
-{ "fcirz.",  XRC(63,15,1), XRA_MASK,	POWER2,		{ FRT, FRB } },
-
-{ "fdiv",    A(63,18,0), AFRC_MASK,	PPCCOM,		{ FRT, FRA, FRB } },
-{ "fd",      A(63,18,0), AFRC_MASK,	PWRCOM,		{ FRT, FRA, FRB } },
-{ "fdiv.",   A(63,18,1), AFRC_MASK,	PPCCOM,		{ FRT, FRA, FRB } },
-{ "fd.",     A(63,18,1), AFRC_MASK,	PWRCOM,		{ FRT, FRA, FRB } },
-
-{ "fsub",    A(63,20,0), AFRC_MASK,	PPCCOM,		{ FRT, FRA, FRB } },
-{ "fs",      A(63,20,0), AFRC_MASK,	PWRCOM,		{ FRT, FRA, FRB } },
-{ "fsub.",   A(63,20,1), AFRC_MASK,	PPCCOM,		{ FRT, FRA, FRB } },
-{ "fs.",     A(63,20,1), AFRC_MASK,	PWRCOM,		{ FRT, FRA, FRB } },
-
-{ "fadd",    A(63,21,0), AFRC_MASK,	PPCCOM,		{ FRT, FRA, FRB } },
-{ "fa",      A(63,21,0), AFRC_MASK,	PWRCOM,		{ FRT, FRA, FRB } },
-{ "fadd.",   A(63,21,1), AFRC_MASK,	PPCCOM,		{ FRT, FRA, FRB } },
-{ "fa.",     A(63,21,1), AFRC_MASK,	PWRCOM,		{ FRT, FRA, FRB } },
-
-{ "fsqrt",   A(63,22,0), AFRAFRC_MASK,	PPCPWR2,	{ FRT, FRB } },
-{ "fsqrt.",  A(63,22,1), AFRAFRC_MASK,	PPCPWR2,	{ FRT, FRB } },
-
-{ "fsel",    A(63,23,0), A_MASK,	PPC,		{ FRT,FRA,FRC,FRB } },
-{ "fsel.",   A(63,23,1), A_MASK,	PPC,		{ FRT,FRA,FRC,FRB } },
-
-{ "fre",     A(63,24,0), AFRALFRC_MASK,	POWER5,		{ FRT, FRB, A_L } },
-{ "fre.",    A(63,24,1), AFRALFRC_MASK,	POWER5,		{ FRT, FRB, A_L } },
-
-{ "fmul",    A(63,25,0), AFRB_MASK,	PPCCOM,		{ FRT, FRA, FRC } },
-{ "fm",      A(63,25,0), AFRB_MASK,	PWRCOM,		{ FRT, FRA, FRC } },
-{ "fmul.",   A(63,25,1), AFRB_MASK,	PPCCOM,		{ FRT, FRA, FRC } },
-{ "fm.",     A(63,25,1), AFRB_MASK,	PWRCOM,		{ FRT, FRA, FRC } },
-
-{ "frsqrte", A(63,26,0), AFRALFRC_MASK,	PPC,		{ FRT, FRB, A_L } },
-{ "frsqrte.",A(63,26,1), AFRALFRC_MASK,	PPC,		{ FRT, FRB, A_L } },
-
-{ "fmsub",   A(63,28,0), A_MASK,	PPCCOM,		{ FRT,FRA,FRC,FRB } },
-{ "fms",     A(63,28,0), A_MASK,	PWRCOM,		{ FRT,FRA,FRC,FRB } },
-{ "fmsub.",  A(63,28,1), A_MASK,	PPCCOM,		{ FRT,FRA,FRC,FRB } },
-{ "fms.",    A(63,28,1), A_MASK,	PWRCOM,		{ FRT,FRA,FRC,FRB } },
-
-{ "fmadd",   A(63,29,0), A_MASK,	PPCCOM,		{ FRT,FRA,FRC,FRB } },
-{ "fma",     A(63,29,0), A_MASK,	PWRCOM,		{ FRT,FRA,FRC,FRB } },
-{ "fmadd.",  A(63,29,1), A_MASK,	PPCCOM,		{ FRT,FRA,FRC,FRB } },
-{ "fma.",    A(63,29,1), A_MASK,	PWRCOM,		{ FRT,FRA,FRC,FRB } },
-
-{ "fnmsub",  A(63,30,0), A_MASK,	PPCCOM,		{ FRT,FRA,FRC,FRB } },
-{ "fnms",    A(63,30,0), A_MASK,	PWRCOM,		{ FRT,FRA,FRC,FRB } },
-{ "fnmsub.", A(63,30,1), A_MASK,	PPCCOM,		{ FRT,FRA,FRC,FRB } },
-{ "fnms.",   A(63,30,1), A_MASK,	PWRCOM,		{ FRT,FRA,FRC,FRB } },
-
-{ "fnmadd",  A(63,31,0), A_MASK,	PPCCOM,		{ FRT,FRA,FRC,FRB } },
-{ "fnma",    A(63,31,0), A_MASK,	PWRCOM,		{ FRT,FRA,FRC,FRB } },
-{ "fnmadd.", A(63,31,1), A_MASK,	PPCCOM,		{ FRT,FRA,FRC,FRB } },
-{ "fnma.",   A(63,31,1), A_MASK,	PWRCOM,		{ FRT,FRA,FRC,FRB } },
-
-{ "fcmpo",   X(63,32),	X_MASK|(3<<21),	COM,		{ BF, FRA, FRB } },
-
-{ "dmulq",   XRC(63,34,0), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-{ "dmulq.",  XRC(63,34,1), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-
-{ "drrndq",  ZRC(63,35,0), Z2_MASK,	POWER6,		{ FRT, FRA, FRB, RMC } },
-{ "drrndq.", ZRC(63,35,1), Z2_MASK,	POWER6,		{ FRT, FRA, FRB, RMC } },
-
-{ "mtfsb1",  XRC(63,38,0), XRARB_MASK,	COM,		{ BT } },
-{ "mtfsb1.", XRC(63,38,1), XRARB_MASK,	COM,		{ BT } },
-
-{ "fneg",    XRC(63,40,0), XRA_MASK,	COM,		{ FRT, FRB } },
-{ "fneg.",   XRC(63,40,1), XRA_MASK,	COM,		{ FRT, FRB } },
-
-{ "mcrfs",   X(63,64),	XRB_MASK|(3<<21)|(3<<16), COM,	{ BF, BFA } },
-
-{ "dscliq",  ZRC(63,66,0), Z_MASK,	POWER6,		{ FRT, FRA, SH16 } },
-{ "dscliq.", ZRC(63,66,1), Z_MASK,	POWER6,		{ FRT, FRA, SH16 } },
-
-{ "dquaiq",  ZRC(63,67,0), Z2_MASK,	POWER6,		{ TE,  FRT, FRB, RMC } },
-{ "dquaiq.", ZRC(63,67,1), Z2_MASK,	POWER6,		{ FRT, FRA, FRB, RMC } },
-
-{ "mtfsb0",  XRC(63,70,0), XRARB_MASK,	COM,		{ BT } },
-{ "mtfsb0.", XRC(63,70,1), XRARB_MASK,	COM,		{ BT } },
-
-{ "fmr",     XRC(63,72,0), XRA_MASK,	COM,		{ FRT, FRB } },
-{ "fmr.",    XRC(63,72,1), XRA_MASK,	COM,		{ FRT, FRB } },
-
-{ "dscriq",  ZRC(63,98,0), Z_MASK,	POWER6,		{ FRT, FRA, SH16 } },
-{ "dscriq.", ZRC(63,98,1), Z_MASK,	POWER6,		{ FRT, FRA, SH16 } },
-
-{ "drintxq", ZRC(63,99,0), Z2_MASK,	POWER6,		{ R, FRT, FRB, RMC } },
-{ "drintxq.",ZRC(63,99,1), Z2_MASK,	POWER6,		{ R, FRT, FRB, RMC } },
-
-{ "dcmpoq",  X(63,130),	   X_MASK,	POWER6,		{ BF,  FRA, FRB } },
-
-{ "mtfsfi",  XRC(63,134,0), XWRA_MASK|(3<<21)|(1<<11), COM, { BFF, U, W } },
-{ "mtfsfi.", XRC(63,134,1), XWRA_MASK|(3<<21)|(1<<11), COM, { BFF, U, W } },
-
-{ "fnabs",   XRC(63,136,0), XRA_MASK,	COM,		{ FRT, FRB } },
-{ "fnabs.",  XRC(63,136,1), XRA_MASK,	COM,		{ FRT, FRB } },
-
-{ "dtstexq", X(63,162),	    X_MASK,	POWER6,		{ BF,  FRA, FRB } },
-{ "dtstdcq", Z(63,194),	    Z_MASK,	POWER6,		{ BF,  FRA, DCM } },
-{ "dtstdgq", Z(63,226),	    Z_MASK,	POWER6,		{ BF,  FRA, DGM } },
-
-{ "drintnq", ZRC(63,227,0), Z2_MASK,	POWER6,		{ R, FRT, FRB, RMC } },
-{ "drintnq.",ZRC(63,227,1), Z2_MASK,	POWER6,		{ R, FRT, FRB, RMC } },
-
-{ "dctqpq",  XRC(63,258,0), X_MASK,	POWER6,		{ FRT, FRB } },
-{ "dctqpq.", XRC(63,258,1), X_MASK,	POWER6,		{ FRT, FRB } },
-
-{ "fabs",    XRC(63,264,0), XRA_MASK,	COM,		{ FRT, FRB } },
-{ "fabs.",   XRC(63,264,1), XRA_MASK,	COM,		{ FRT, FRB } },
-
-{ "dctfixq", XRC(63,290,0), X_MASK,	POWER6,		{ FRT, FRB } },
-{ "dctfixq.",XRC(63,290,1), X_MASK,	POWER6,		{ FRT, FRB } },
-
-{ "ddedpdq", XRC(63,322,0), X_MASK,	POWER6,		{ SP, FRT, FRB } },
-{ "ddedpdq.",XRC(63,322,1), X_MASK,	POWER6,		{ SP, FRT, FRB } },
-
-{ "dxexq",   XRC(63,354,0), X_MASK,	POWER6,		{ FRT, FRB } },
-{ "dxexq.",  XRC(63,354,1), X_MASK,	POWER6,		{ FRT, FRB } },
-
-{ "frin",    XRC(63,392,0), XRA_MASK,	POWER5,		{ FRT, FRB } },
-{ "frin.",   XRC(63,392,1), XRA_MASK,	POWER5,		{ FRT, FRB } },
-{ "friz",    XRC(63,424,0), XRA_MASK,	POWER5,		{ FRT, FRB } },
-{ "friz.",   XRC(63,424,1), XRA_MASK,	POWER5,		{ FRT, FRB } },
-{ "frip",    XRC(63,456,0), XRA_MASK,	POWER5,		{ FRT, FRB } },
-{ "frip.",   XRC(63,456,1), XRA_MASK,	POWER5,		{ FRT, FRB } },
-{ "frim",    XRC(63,488,0), XRA_MASK,	POWER5,		{ FRT, FRB } },
-{ "frim.",   XRC(63,488,1), XRA_MASK,	POWER5,		{ FRT, FRB } },
-
-{ "dsubq",   XRC(63,514,0), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-{ "dsubq.",  XRC(63,514,1), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-
-{ "ddivq",   XRC(63,546,0), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-{ "ddivq.",  XRC(63,546,1), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-
-{ "mffsl",   XRA(63,583,12), XRARB_MASK,	POWER9,	{ FRT } },
-
-{ "mffs",    XRC(63,583,0), XRARB_MASK,	COM,		{ FRT } },
-{ "mffs.",   XRC(63,583,1), XRARB_MASK,	COM,		{ FRT } },
-
-{ "dcmpuq",  X(63,642),	    X_MASK,	POWER6,		{ BF,  FRA, FRB } },
-
-{ "dtstsfq", X(63,674),	    X_MASK,	POWER6,		{ BF,  FRA, FRB } },
-
-{ "mtfsf",   XFL(63,711,0), XFL_MASK,	COM,		{ FLM, FRB, XFL_L, W } },
-{ "mtfsf.",  XFL(63,711,1), XFL_MASK,	COM,		{ FLM, FRB, XFL_L, W } },
-
-{ "drdpq",   XRC(63,770,0), X_MASK,	POWER6,		{ FRT, FRB } },
-{ "drdpq.",  XRC(63,770,1), X_MASK,	POWER6,		{ FRT, FRB } },
-
-{ "dcffixq", XRC(63,802,0), X_MASK,	POWER6,		{ FRT, FRB } },
-{ "dcffixq.",XRC(63,802,1), X_MASK,	POWER6,		{ FRT, FRB } },
-
-{ "fctid",   XRC(63,814,0), XRA_MASK,	PPC64,		{ FRT, FRB } },
-{ "fctid.",  XRC(63,814,1), XRA_MASK,	PPC64,		{ FRT, FRB } },
-
-{ "fctidz",  XRC(63,815,0), XRA_MASK,	PPC64,		{ FRT, FRB } },
-{ "fctidz.", XRC(63,815,1), XRA_MASK,	PPC64,		{ FRT, FRB } },
-
-{ "denbcdq", XRC(63,834,0), X_MASK,	POWER6,		{ S, FRT, FRB } },
-{ "denbcdq.",XRC(63,834,1), X_MASK,	POWER6,		{ S, FRT, FRB } },
-
-{ "fcfid",   XRC(63,846,0), XRA_MASK,	PPC64,		{ FRT, FRB } },
-{ "fcfid.",  XRC(63,846,1), XRA_MASK,	PPC64,		{ FRT, FRB } },
-
-{ "diexq",   XRC(63,866,0), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-{ "diexq.",  XRC(63,866,1), X_MASK,	POWER6,		{ FRT, FRA, FRB } },
-
-};
-
-const int powerpc_num_opcodes =
-  sizeof (powerpc_opcodes) / sizeof (powerpc_opcodes[0]);
-
-/* The macro table.  This is only used by the assembler.  */
-
-/* The expressions of the form (-x ! 31) & (x | 31) have the value 0
-   when x=0; 32-x when x is between 1 and 31; are negative if x is
-   negative; and are 32 or more otherwise.  This is what you want
-   when, for instance, you are emulating a right shift by a
-   rotate-left-and-mask, because the underlying instructions support
-   shifts of size 0 but not shifts of size 32.  By comparison, when
-   extracting x bits from some word you want to use just 32-x, because
-   the underlying instructions don't support extracting 0 bits but do
-   support extracting the whole word (32 bits in this case).  */
-
-const struct powerpc_macro powerpc_macros[] = {
-{ "extldi",  4,   PPC64,	"rldicr %0,%1,%3,(%2)-1" },
-{ "extldi.", 4,   PPC64,	"rldicr. %0,%1,%3,(%2)-1" },
-{ "extrdi",  4,   PPC64,	"rldicl %0,%1,(%2)+(%3),64-(%2)" },
-{ "extrdi.", 4,   PPC64,	"rldicl. %0,%1,(%2)+(%3),64-(%2)" },
-{ "insrdi",  4,   PPC64,	"rldimi %0,%1,64-((%2)+(%3)),%3" },
-{ "insrdi.", 4,   PPC64,	"rldimi. %0,%1,64-((%2)+(%3)),%3" },
-{ "rotrdi",  3,   PPC64,	"rldicl %0,%1,(-(%2)!63)&((%2)|63),0" },
-{ "rotrdi.", 3,   PPC64,	"rldicl. %0,%1,(-(%2)!63)&((%2)|63),0" },
-{ "sldi",    3,   PPC64,	"rldicr %0,%1,%2,63-(%2)" },
-{ "sldi.",   3,   PPC64,	"rldicr. %0,%1,%2,63-(%2)" },
-{ "srdi",    3,   PPC64,	"rldicl %0,%1,(-(%2)!63)&((%2)|63),%2" },
-{ "srdi.",   3,   PPC64,	"rldicl. %0,%1,(-(%2)!63)&((%2)|63),%2" },
-{ "clrrdi",  3,   PPC64,	"rldicr %0,%1,0,63-(%2)" },
-{ "clrrdi.", 3,   PPC64,	"rldicr. %0,%1,0,63-(%2)" },
-{ "clrlsldi",4,   PPC64,	"rldic %0,%1,%3,(%2)-(%3)" },
-{ "clrlsldi.",4,  PPC64,	"rldic. %0,%1,%3,(%2)-(%3)" },
-
-{ "extlwi",  4,   PPCCOM,	"rlwinm %0,%1,%3,0,(%2)-1" },
-{ "extlwi.", 4,   PPCCOM,	"rlwinm. %0,%1,%3,0,(%2)-1" },
-{ "extrwi",  4,   PPCCOM,	"rlwinm %0,%1,((%2)+(%3))&((%2)+(%3)<>32),32-(%2),31" },
-{ "extrwi.", 4,   PPCCOM,	"rlwinm. %0,%1,((%2)+(%3))&((%2)+(%3)<>32),32-(%2),31" },
-{ "inslwi",  4,   PPCCOM,	"rlwimi %0,%1,(-(%3)!31)&((%3)|31),%3,(%2)+(%3)-1" },
-{ "inslwi.", 4,   PPCCOM,	"rlwimi. %0,%1,(-(%3)!31)&((%3)|31),%3,(%2)+(%3)-1"},
-{ "insrwi",  4,   PPCCOM,	"rlwimi %0,%1,32-((%2)+(%3)),%3,(%2)+(%3)-1" },
-{ "insrwi.", 4,   PPCCOM,	"rlwimi. %0,%1,32-((%2)+(%3)),%3,(%2)+(%3)-1"},
-{ "rotrwi",  3,   PPCCOM,	"rlwinm %0,%1,(-(%2)!31)&((%2)|31),0,31" },
-{ "rotrwi.", 3,   PPCCOM,	"rlwinm. %0,%1,(-(%2)!31)&((%2)|31),0,31" },
-{ "slwi",    3,   PPCCOM,	"rlwinm %0,%1,%2,0,31-(%2)" },
-{ "sli",     3,   PWRCOM,	"rlinm %0,%1,%2,0,31-(%2)" },
-{ "slwi.",   3,   PPCCOM,	"rlwinm. %0,%1,%2,0,31-(%2)" },
-{ "sli.",    3,   PWRCOM,	"rlinm. %0,%1,%2,0,31-(%2)" },
-{ "srwi",    3,   PPCCOM,	"rlwinm %0,%1,(-(%2)!31)&((%2)|31),%2,31" },
-{ "sri",     3,   PWRCOM,	"rlinm %0,%1,(-(%2)!31)&((%2)|31),%2,31" },
-{ "srwi.",   3,   PPCCOM,	"rlwinm. %0,%1,(-(%2)!31)&((%2)|31),%2,31" },
-{ "sri.",    3,   PWRCOM,	"rlinm. %0,%1,(-(%2)!31)&((%2)|31),%2,31" },
-{ "clrrwi",  3,   PPCCOM,	"rlwinm %0,%1,0,0,31-(%2)" },
-{ "clrrwi.", 3,   PPCCOM,	"rlwinm. %0,%1,0,0,31-(%2)" },
-{ "clrlslwi",4,   PPCCOM,	"rlwinm %0,%1,%3,(%2)-(%3),31-(%3)" },
-{ "clrlslwi.",4,  PPCCOM,	"rlwinm. %0,%1,%3,(%2)-(%3),31-(%3)" },
-};
-
-const int powerpc_num_macros =
-  sizeof (powerpc_macros) / sizeof (powerpc_macros[0]);
-
-
-/* This file provides several disassembler functions, all of which use
-   the disassembler interface defined in dis-asm.h.  Several functions
-   are provided because this file handles disassembly for the PowerPC
-   in both big and little endian mode and also for the POWER (RS/6000)
-   chip.  */
-
-static int print_insn_powerpc (bfd_vma, struct disassemble_info *, int, int);
-
-/* Determine which set of machines to disassemble for.  PPC403/601 or
-   BookE.  For convenience, also disassemble instructions supported
-   by the AltiVec vector unit.  */
-
-static int
-powerpc_dialect (struct disassemble_info *info)
-{
-  int dialect = PPC_OPCODE_PPC;
-
-  if (BFD_DEFAULT_TARGET_SIZE == 64)
-    dialect |= PPC_OPCODE_64;
-
-  if (info->disassembler_options
-      && strstr (info->disassembler_options, "booke") != NULL)
-    dialect |= PPC_OPCODE_BOOKE | PPC_OPCODE_BOOKE64;
-  else if ((info->mach == bfd_mach_ppc_e500)
-	   || (info->disassembler_options
-	       && strstr (info->disassembler_options, "e500") != NULL))
-    dialect |= (PPC_OPCODE_BOOKE
-		| PPC_OPCODE_SPE | PPC_OPCODE_ISEL
-		| PPC_OPCODE_EFS | PPC_OPCODE_BRLOCK
-		| PPC_OPCODE_PMR | PPC_OPCODE_CACHELCK
-		| PPC_OPCODE_RFMCI);
-  else if (info->disassembler_options
-	   && strstr (info->disassembler_options, "efs") != NULL)
-    dialect |= PPC_OPCODE_EFS;
-  else if (info->disassembler_options
-	   && strstr (info->disassembler_options, "e300") != NULL)
-    dialect |= PPC_OPCODE_E300 | PPC_OPCODE_CLASSIC | PPC_OPCODE_COMMON;
-  else if (info->disassembler_options
-	   && strstr (info->disassembler_options, "440") != NULL)
-    dialect |= PPC_OPCODE_BOOKE | PPC_OPCODE_32
-      | PPC_OPCODE_440 | PPC_OPCODE_ISEL | PPC_OPCODE_RFMCI;
-  else
-    dialect |= (PPC_OPCODE_403 | PPC_OPCODE_601 | PPC_OPCODE_CLASSIC
-		| PPC_OPCODE_COMMON | PPC_OPCODE_ALTIVEC);
-
-  if (info->disassembler_options
-      && strstr (info->disassembler_options, "power4") != NULL)
-    dialect |= PPC_OPCODE_POWER4;
-
-  if (info->disassembler_options
-      && strstr (info->disassembler_options, "power5") != NULL)
-    dialect |= PPC_OPCODE_POWER4 | PPC_OPCODE_POWER5;
-
-  if (info->disassembler_options
-      && strstr (info->disassembler_options, "cell") != NULL)
-    dialect |= PPC_OPCODE_POWER4 | PPC_OPCODE_CELL | PPC_OPCODE_ALTIVEC;
-
-  if (info->disassembler_options
-      && strstr (info->disassembler_options, "power6") != NULL)
-    dialect |= PPC_OPCODE_POWER4 | PPC_OPCODE_POWER5 | PPC_OPCODE_POWER6 | PPC_OPCODE_ALTIVEC;
-
-  if (info->disassembler_options
-      && strstr (info->disassembler_options, "any") != NULL)
-    dialect |= PPC_OPCODE_ANY;
-
-  if (info->disassembler_options)
-    {
-      if (strstr (info->disassembler_options, "32") != NULL)
-	dialect &= ~PPC_OPCODE_64;
-      else if (strstr (info->disassembler_options, "64") != NULL)
-	dialect |= PPC_OPCODE_64;
-    }
-
-  info->private_data = (char *) 0 + dialect;
-  return dialect;
-}
-
-/* QEMU default */
-int
-print_insn_ppc (bfd_vma memaddr, struct disassemble_info *info)
-{
-  int dialect = (char *) info->private_data - (char *) 0;
-  return print_insn_powerpc (memaddr, info, info->endian == BFD_ENDIAN_BIG,
-                             dialect);
-}
-
-/* Print a big endian PowerPC instruction.  */
-
-int
-print_insn_big_powerpc (bfd_vma memaddr, struct disassemble_info *info)
-{
-  int dialect = (char *) info->private_data - (char *) 0;
-  return print_insn_powerpc (memaddr, info, 1, dialect);
-}
-
-/* Print a little endian PowerPC instruction.  */
-
-int
-print_insn_little_powerpc (bfd_vma memaddr, struct disassemble_info *info)
-{
-  int dialect = (char *) info->private_data - (char *) 0;
-  return print_insn_powerpc (memaddr, info, 0, dialect);
-}
-
-/* Print a POWER (RS/6000) instruction.  */
-
-int
-print_insn_rs6000 (bfd_vma memaddr, struct disassemble_info *info)
-{
-  return print_insn_powerpc (memaddr, info, 1, PPC_OPCODE_POWER);
-}
-
-/* Extract the operand value from the PowerPC or POWER instruction.  */
-
-static long
-operand_value_powerpc (const struct powerpc_operand *operand,
-		       unsigned long insn, int dialect)
-{
-  long value;
-  int invalid;
-  /* Extract the value from the instruction.  */
-  if (operand->extract)
-    value = (*operand->extract) (insn, dialect, &invalid);
-  else
-    {
-      value = (insn >> operand->shift) & operand->bitm;
-      if ((operand->flags & PPC_OPERAND_SIGNED) != 0)
-	{
-	  /* BITM is always some number of zeros followed by some
-	     number of ones, followed by some number of zeros.  */
-	  unsigned long top = operand->bitm;
-	  /* top & -top gives the rightmost 1 bit, so this
-	     fills in any trailing zeros.  */
-	  top |= (top & -top) - 1;
-	  top &= ~(top >> 1);
-	  value = (value ^ top) - top;
-	}
-    }
-
-  return value;
-}
-
-/* Determine whether the optional operand(s) should be printed.  */
-
-static int
-skip_optional_operands (const unsigned char *opindex,
-			unsigned long insn, int dialect)
-{
-  const struct powerpc_operand *operand;
-
-  for (; *opindex != 0; opindex++)
-    {
-      operand = &powerpc_operands[*opindex];
-      if ((operand->flags & PPC_OPERAND_NEXT) != 0
-	  || ((operand->flags & PPC_OPERAND_OPTIONAL) != 0
-	      && operand_value_powerpc (operand, insn, dialect) != 0))
-	return 0;
-    }
-
-  return 1;
-}
-
-/* Print a PowerPC or POWER instruction.  */
-
-static int
-print_insn_powerpc (bfd_vma memaddr,
-		    struct disassemble_info *info,
-		    int bigendian,
-		    int dialect)
-{
-  bfd_byte buffer[4];
-  int status;
-  unsigned long insn;
-  const struct powerpc_opcode *opcode;
-  const struct powerpc_opcode *opcode_end;
-  unsigned long op;
-
-  if (dialect == 0)
-    dialect = powerpc_dialect (info);
-
-  status = (*info->read_memory_func) (memaddr, buffer, 4, info);
-  if (status != 0)
-    {
-      (*info->memory_error_func) (status, memaddr, info);
-      return -1;
-    }
-
-  if (bigendian)
-    insn = bfd_getb32 (buffer);
-  else
-    insn = bfd_getl32 (buffer);
-
-  /* Get the major opcode of the instruction.  */
-  op = PPC_OP (insn);
-
-  /* Find the first match in the opcode table.  We could speed this up
-     a bit by doing a binary search on the major opcode.  */
-  opcode_end = powerpc_opcodes + powerpc_num_opcodes;
- again:
-  for (opcode = powerpc_opcodes; opcode < opcode_end; opcode++)
-    {
-      unsigned long table_op;
-      const unsigned char *opindex;
-      const struct powerpc_operand *operand;
-      int invalid;
-      int need_comma;
-      int need_paren;
-      int skip_optional;
-
-      table_op = PPC_OP (opcode->opcode);
-      if (op < table_op)
-	break;
-      if (op > table_op)
-	continue;
-
-      if ((insn & opcode->mask) != opcode->opcode
-	  || (opcode->flags & dialect) == 0)
-	continue;
-
-      /* Make two passes over the operands.  First see if any of them
-	 have extraction functions, and, if they do, make sure the
-	 instruction is valid.  */
-      invalid = 0;
-      for (opindex = opcode->operands; *opindex != 0; opindex++)
-	{
-	  operand = powerpc_operands + *opindex;
-	  if (operand->extract)
-	    (*operand->extract) (insn, dialect, &invalid);
-	}
-      if (invalid)
-	continue;
-
-      /* The instruction is valid.  */
-      if (opcode->operands[0] != 0)
-	(*info->fprintf_func) (info->stream, "%-7s ", opcode->name);
-      else
-	(*info->fprintf_func) (info->stream, "%s", opcode->name);
-
-      /* Now extract and print the operands.  */
-      need_comma = 0;
-      need_paren = 0;
-      skip_optional = -1;
-      for (opindex = opcode->operands; *opindex != 0; opindex++)
-	{
-	  long value;
-
-	  operand = powerpc_operands + *opindex;
-
-	  /* Operands that are marked FAKE are simply ignored.  We
-	     already made sure that the extract function considered
-	     the instruction to be valid.  */
-	  if ((operand->flags & PPC_OPERAND_FAKE) != 0)
-	    continue;
-
-	  /* If all of the optional operands have the value zero,
-	     then don't print any of them.  */
-	  if ((operand->flags & PPC_OPERAND_OPTIONAL) != 0)
-	    {
-	      if (skip_optional < 0)
-		skip_optional = skip_optional_operands (opindex, insn,
-							dialect);
-	      if (skip_optional)
-		continue;
-	    }
-
-	  value = operand_value_powerpc (operand, insn, dialect);
-
-	  if (need_comma)
-	    {
-	      (*info->fprintf_func) (info->stream, ",");
-	      need_comma = 0;
-	    }
-
-	  /* Print the operand as directed by the flags.  */
-	  if ((operand->flags & PPC_OPERAND_GPR) != 0
-	      || ((operand->flags & PPC_OPERAND_GPR_0) != 0 && value != 0))
-	    (*info->fprintf_func) (info->stream, "r%ld", value);
-	  else if ((operand->flags & PPC_OPERAND_FPR) != 0)
-	    (*info->fprintf_func) (info->stream, "f%ld", value);
-	  else if ((operand->flags & PPC_OPERAND_VR) != 0)
-	    (*info->fprintf_func) (info->stream, "v%ld", value);
-	  else if ((operand->flags & PPC_OPERAND_RELATIVE) != 0)
-	    (*info->print_address_func) (memaddr + value, info);
-	  else if ((operand->flags & PPC_OPERAND_ABSOLUTE) != 0)
-	    (*info->print_address_func) ((bfd_vma) value & 0xffffffff, info);
-	  else if ((operand->flags & PPC_OPERAND_CR) == 0
-		   || (dialect & PPC_OPCODE_PPC) == 0)
-	    (*info->fprintf_func) (info->stream, "%ld", value);
-	  else
-	    {
-	      if (operand->bitm == 7)
-		(*info->fprintf_func) (info->stream, "cr%ld", value);
-	      else
-		{
-		  static const char *cbnames[4] = { "lt", "gt", "eq", "so" };
-		  int cr;
-		  int cc;
-
-		  cr = value >> 2;
-		  if (cr != 0)
-		    (*info->fprintf_func) (info->stream, "4*cr%d+", cr);
-		  cc = value & 3;
-		  (*info->fprintf_func) (info->stream, "%s", cbnames[cc]);
-		}
-	    }
-
-	  if (need_paren)
-	    {
-	      (*info->fprintf_func) (info->stream, ")");
-	      need_paren = 0;
-	    }
-
-	  if ((operand->flags & PPC_OPERAND_PARENS) == 0)
-	    need_comma = 1;
-	  else
-	    {
-	      (*info->fprintf_func) (info->stream, "(");
-	      need_paren = 1;
-	    }
-	}
-
-      /* We have found and printed an instruction; return.  */
-      return 4;
-    }
-
-  if ((dialect & PPC_OPCODE_ANY) != 0)
-    {
-      dialect = ~PPC_OPCODE_ANY;
-      goto again;
-    }
-
-  /* We could not find a match.  */
-  (*info->fprintf_func) (info->stream, ".long 0x%lx", insn);
-
-  return 4;
-}
diff --git a/include/disas/dis-asm.h b/include/disas/dis-asm.h
index 6484caca9e..fc7cb7af5a 100644
--- a/include/disas/dis-asm.h
+++ b/include/disas/dis-asm.h
@@ -447,7 +447,6 @@ int print_insn_w65              (bfd_vma, disassemble_info*);
 int print_insn_d10v             (bfd_vma, disassemble_info*);
 int print_insn_v850             (bfd_vma, disassemble_info*);
 int print_insn_tic30            (bfd_vma, disassemble_info*);
-int print_insn_ppc              (bfd_vma, disassemble_info*);
 int print_insn_crisv32          (bfd_vma, disassemble_info*);
 int print_insn_crisv10          (bfd_vma, disassemble_info*);
 int print_insn_microblaze       (bfd_vma, disassemble_info*);
diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
index d4c7813de5..527ad40fcb 100644
--- a/target/ppc/cpu_init.c
+++ b/target/ppc/cpu_init.c
@@ -7309,8 +7309,6 @@ static void ppc_disas_set_info(CPUState *cs, disassemble_info *info)
         info->mach = bfd_mach_ppc;
 #endif
     }
-    info->disassembler_options = (char *)"any";
-    info->print_insn = print_insn_ppc;
 
     info->cap_arch = CS_ARCH_PPC;
 #ifdef TARGET_PPC64

From f07b3a830ff4ab98ab1cf9dd3f006c55b0fc61f3 Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Wed, 27 Apr 2022 14:33:16 +0200
Subject: [PATCH 178/935] softmmu/vl: Fence 'xenfb' if Xen support is not
 compiled in

The 'xenfb' parameter for the '-vga' command line option is currently
always enabled unconditionally (since the xenfb is not a proper QOM
device that could be tested via its class name). That means it also
shows up if Xen is not enabled at all, e.g. like this:

 $ ./qemu-system-sparc -vga help
 none                 no graphic card
 xenfb                Xen paravirtualized framebuffer
 tcx                  TCX framebuffer (default)
 cg3                  CG3 framebuffer

Let's avoid this situation by fencing the parameter with the
CONFIG_XEN_BACKEND switch.

Message-Id: <20220427123316.329312-1-thuth@redhat.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 softmmu/vl.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index c2919579fd..ad886fb878 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -934,10 +934,12 @@ static const VGAInterfaceInfo vga_interfaces[VGA_TYPE_MAX] = {
         .name = "CG3 framebuffer",
         .class_names = { "cgthree" },
     },
+#ifdef CONFIG_XEN_BACKEND
     [VGA_XENFB] = {
         .opt_name = "xenfb",
         .name = "Xen paravirtualized framebuffer",
     },
+#endif
 };
 
 static bool vga_interface_available(VGAInterfaceType t)

From eeb3647cbc4716e27ca8504e0fa72bcfd16fbc35 Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Wed, 27 Apr 2022 15:31:56 +0200
Subject: [PATCH 179/935] qemu-options: Limit the -xen options to x86 and arm

The Xen hypervisor is only available on x86 and arm - thus let's
limit the related options to these targets.

Message-Id: <20220427133156.344418-1-thuth@redhat.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 qemu-options.hx | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/qemu-options.hx b/qemu-options.hx
index 5f69b94b8e..796229c433 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -4186,16 +4186,17 @@ SRST
 ERST
 
 DEF("xen-domid", HAS_ARG, QEMU_OPTION_xen_domid,
-    "-xen-domid id   specify xen guest domain id\n", QEMU_ARCH_ALL)
+    "-xen-domid id   specify xen guest domain id\n",
+    QEMU_ARCH_ARM | QEMU_ARCH_I386)
 DEF("xen-attach", 0, QEMU_OPTION_xen_attach,
     "-xen-attach     attach to existing xen domain\n"
     "                libxl will use this when starting QEMU\n",
-    QEMU_ARCH_ALL)
+    QEMU_ARCH_ARM | QEMU_ARCH_I386)
 DEF("xen-domid-restrict", 0, QEMU_OPTION_xen_domid_restrict,
     "-xen-domid-restrict     restrict set of available xen operations\n"
     "                        to specified domain id. (Does not affect\n"
     "                        xenpv machine type).\n",
-    QEMU_ARCH_ALL)
+    QEMU_ARCH_ARM | QEMU_ARCH_I386)
 SRST
 ``-xen-domid id``
     Specify xen guest domain id (XEN only).

From 4c7793027982dbaa6c3ca1bef44370ca51d4bb95 Mon Sep 17 00:00:00 2001
From: Yonggang Luo <luoyonggang@gmail.com>
Date: Fri, 2 Oct 2020 00:27:05 +0800
Subject: [PATCH 180/935] doc: remove hxtool-conv.pl

This script doesn't need anymore as all texi are already convert to rST

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Message-Id: <20201001162705.219-3-luoyonggang@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 scripts/hxtool-conv.pl | 137 -----------------------------------------
 1 file changed, 137 deletions(-)
 delete mode 100755 scripts/hxtool-conv.pl

diff --git a/scripts/hxtool-conv.pl b/scripts/hxtool-conv.pl
deleted file mode 100755
index eede40b346..0000000000
--- a/scripts/hxtool-conv.pl
+++ /dev/null
@@ -1,137 +0,0 @@
-#!/usr/bin/perl -w
-#
-# Script to convert .hx file STEXI/ETEXI blocks to SRST/ERST
-#
-# Copyright (C) 2020 Linaro
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# (at your option) any later version. See the COPYING file in the
-# top-level directory.
-
-# This script was only ever intended as a one-off conversion operation.
-# Please excuse the places where it is a bit hacky.
-# Some manual intervention after the conversion is expected, as are
-# some warnings from makeinfo.
-# Warning: this script is not idempotent: don't try to run it on
-# a .hx file that already has SRST/ERST sections.
-
-# Expected usage:
-# scripts/hxtool-conv.pl file.hx > file.hx.new
-
-use utf8;
-
-my $reading_texi = 0;
-my $texiblock = '';
-my @tables = ();
-
-sub update_tables($) {
-    my ($texi) = @_;
-    # Update our list of open table directives: every @table
-    # line in the texi fragment is added to the list, and every
-    # @end table line means we remove an entry from the list.
-    # If this fragment had a completely self contained table with
-    # both the @table and @end table lines, this will be a no-op.
-    foreach (split(/\n/, $texi)) {
-        push @tables, $_ if /^\@table/;
-        pop @tables if /^\@end table/;
-    }
-}
-
-sub only_table_directives($) {
-    # Return true if every line in the fragment is a start or end table directive
-    my ($texi) = @_;
-    foreach (split(/\n/, $texi)) {
-        return 0 unless /^\@table/ or /^\@end table/;
-    }
-    return 1;
-}
-
-sub output_rstblock($) {
-    # Write the output to /tmp/frag.texi, wrapped in whatever current @table
-    # lines we need.
-    my ($texi) = @_;
-
-    # As a special case, if this fragment is only table directives and
-    # nothing else, update our set of open table directives but otherwise
-    # ignore it. This avoids emitting an empty SRST/ERST block.
-    if (only_table_directives($texi)) {
-        update_tables($texi);
-        return;
-    }
-
-    open(my $fragfh, '>', '/tmp/frag.texi');
-    # First output the currently active set of open table directives
-    print $fragfh join("\n", @tables);
-    # Next, update our list of open table directives.
-    # We need to do this before we emit the closing table directives
-    # so that we emit the right number if this fragment had an
-    # unbalanced set of directives.
-    update_tables($texi);
-    # Then emit the texi fragment itself.
-    print $fragfh "\n$texi\n";
-    # Finally, add the necessary closing table directives.
-    print $fragfh "\@end table\n" x scalar @tables;
-    close $fragfh;
-
-    # Now invoke makeinfo/pandoc on it and slurp the results into a string
-    open(my $fh, '-|', "makeinfo --force -o - --docbook "
-         . "-D 'qemu_system_x86 QEMU_SYSTEM_X86_MACRO' "
-         . "-D 'qemu_system     QEMU_SYSTEM_MACRO'  /tmp/frag.texi "
-         . " | pandoc  -f docbook -t rst")
-        or die "can't start makeinfo/pandoc: $!";
-
-    binmode $fh, ':encoding(utf8)';
-
-    print "SRST\n";
-
-    # Slurp the whole thing into a string so we can do multiline
-    # string matches on it.
-    my $rst = do {
-        local $/ = undef;
-        <$fh>;
-    };
-    $rst =~ s/^-  − /-  /gm;
-    $rst =~ s/“/"/gm;
-    $rst =~ s/”/"/gm;
-    $rst =~ s/‘/'/gm;
-    $rst =~ s/’/'/gm;
-    $rst =~ s/QEMU_SYSTEM_MACRO/|qemu_system|/g;
-    $rst =~ s/QEMU_SYSTEM_X86_MACRO/|qemu_system_x86|/g;
-    $rst =~ s/(?=::\n\n +\|qemu)/.. parsed-literal/g;
-    $rst =~ s/:\n\n::$/::/gm;
-
-    # Fix up the invalid reference format makeinfo/pandoc emit:
-    # `Some string here <#anchorname>`__
-    # should be:
-    # :ref:`anchorname`
-    $rst =~ s/\`[^<`]+\<\#([^>]+)\>\`__/:ref:`$1`/gm;
-    print $rst;
-
-    close $fh or die "error on close: $!";
-    print "ERST\n";
-}
-
-# Read the whole .hx input file.
-while (<>) {
-    # Always print the current line
-    print;
-    if (/STEXI/) {
-        $reading_texi = 1;
-        $texiblock = '';
-        next;
-    }
-    if (/ETEXI/) {
-        $reading_texi = 0;
-        # dump RST version of block
-        output_rstblock($texiblock);
-        next;
-    }
-    if ($reading_texi) {
-        # Accumulate the texi into a string
-        # but drop findex entries as they will confuse makeinfo
-        next if /^\@findex/;
-        $texiblock .= $_;
-    }
-}
-
-die "Unexpectedly still in texi block at EOF" if $reading_texi;

From 3304f5a8c97e2baea2882048141776168d7b93eb Mon Sep 17 00:00:00 2001
From: Yonggang Luo <luoyonggang@gmail.com>
Date: Wed, 4 May 2022 04:05:23 +0800
Subject: [PATCH 181/935] cirrus/win32: upgrade mingw base packages
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20220503200524.1868-2-luoyonggang@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .cirrus.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index 7552d70974..20843a420c 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -12,7 +12,7 @@ windows_msys2_task:
     CIRRUS_SHELL: powershell
     MSYS: winsymlinks:nativestrict
     MSYSTEM: MINGW64
-    MSYS2_URL: https://github.com/msys2/msys2-installer/releases/download/2021-04-19/msys2-base-x86_64-20210419.sfx.exe
+    MSYS2_URL: https://github.com/msys2/msys2-installer/releases/download/2022-05-03/msys2-base-x86_64-20220503.sfx.exe
     MSYS2_FINGERPRINT: 0
     MSYS2_PACKAGES: "
       diffutils git grep make pkg-config sed

From 5c570ef2f154ee8efe35ef939df00d2a33dee1fd Mon Sep 17 00:00:00 2001
From: Yonggang Luo <luoyonggang@gmail.com>
Date: Wed, 4 May 2022 04:05:24 +0800
Subject: [PATCH 182/935] gitlab-ci: Upgrade mingw base package.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20220503200524.1868-3-luoyonggang@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/windows.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
index 1df1630349..47f6073773 100644
--- a/.gitlab-ci.d/windows.yml
+++ b/.gitlab-ci.d/windows.yml
@@ -16,7 +16,7 @@
     }
   - If ( !(Test-Path -Path msys64\var\cache\msys2.exe ) ) {
       Invoke-WebRequest
-      "https://github.com/msys2/msys2-installer/releases/download/2021-07-25/msys2-base-x86_64-20210725.sfx.exe"
+      "https://github.com/msys2/msys2-installer/releases/download/2022-05-03/msys2-base-x86_64-20220503.sfx.exe"
       -outfile "msys64\var\cache\msys2.exe"
     }
   - msys64\var\cache\msys2.exe -y

From 26f0ee7ddb9e95b3bde1d1b51d98334bebce0e1f Mon Sep 17 00:00:00 2001
From: Brad Smith <brad@comstyle.com>
Date: Thu, 5 May 2022 19:32:07 -0400
Subject: [PATCH 183/935] tests/vm: update openbsd to release 7.1

tests/vm/openbsd: Update to release 7.1

Signed-off-by: Brad Smith <brad@comstyle.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Warner Losh <imp@bsdimp.com>
Message-Id: <YnRed7sw45lTbRjb@humpty.home.comstyle.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/vm/openbsd | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index 337fe7c303..dc34b2718b 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -22,8 +22,8 @@ class OpenBSDVM(basevm.BaseVM):
     name = "openbsd"
     arch = "x86_64"
 
-    link = "https://cdn.openbsd.org/pub/OpenBSD/7.0/amd64/install70.iso"
-    csum = "1882f9a23c9800e5dba3dbd2cf0126f552605c915433ef4c5bb672610a4ca3a4"
+    link = "https://cdn.openbsd.org/pub/OpenBSD/7.1/amd64/install71.iso"
+    csum = "d3a7c5b9bf890bc404304a1c96f9ee72e1d9bbcf9cc849c1133bdb0d67843396"
     size = "20G"
     pkgs = [
         # tools

From f9bcb2d68496a8fa620443edacb27cad1acc1492 Mon Sep 17 00:00:00 2001
From: Gautam Agrawal <gautamnagrawal@gmail.com>
Date: Sun, 1 May 2022 17:55:05 +0530
Subject: [PATCH 184/935] Warn user if the vga flag is passed but no vga device
 is created

A global boolean variable "vga_interface_created"(declared in softmmu/globals.c)
has been used to track the creation of vga interface. If the vga flag is passed
in the command line "default_vga"(declared in softmmu/vl.c) variable is set to 0.
To warn user, the condition checks if vga_interface_created is false
and default_vga is equal to 0. If "-vga none" is passed, this patch will not warn the
user regarding the creation of VGA device.

The warning "A -vga option was passed but this
machine type does not use that option; no VGA device has been created"
is logged if vga flag is passed but no vga device is created.

This patch has been tested for x86_64, i386, sparc, sparc64 and arm boards.

Signed-off-by: Gautam Agrawal <gautamnagrawal@gmail.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/581
Message-Id: <20220501122505.29202-1-gautamnagrawal@gmail.com>
[thuth: Fix wrong warning with "-device" in some cases as reported by Paolo]
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 hw/hppa/machine.c         | 1 +
 hw/isa/isa-bus.c          | 1 +
 hw/mips/fuloong2e.c       | 1 +
 hw/pci/pci.c              | 1 +
 hw/ppc/spapr.c            | 1 +
 hw/sparc/sun4m.c          | 2 ++
 hw/sparc64/sun4u.c        | 1 +
 hw/xenpv/xen_machine_pv.c | 1 +
 include/sysemu/sysemu.h   | 1 +
 softmmu/globals.c         | 1 +
 softmmu/vl.c              | 7 +++++++
 11 files changed, 18 insertions(+)

diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index ae0bc07e75..4d054ca869 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -265,6 +265,7 @@ static void machine_hppa_init(MachineState *machine)
 
     /* Graphics setup. */
     if (machine->enable_graphics && vga_interface_type != VGA_NONE) {
+        vga_interface_created = true;
         dev = qdev_new("artist");
         s = SYS_BUS_DEVICE(dev);
         sysbus_realize_and_unref(s, &error_fatal);
diff --git a/hw/isa/isa-bus.c b/hw/isa/isa-bus.c
index 0ad1c5fd65..cd5ad3687d 100644
--- a/hw/isa/isa-bus.c
+++ b/hw/isa/isa-bus.c
@@ -166,6 +166,7 @@ bool isa_realize_and_unref(ISADevice *dev, ISABus *bus, Error **errp)
 
 ISADevice *isa_vga_init(ISABus *bus)
 {
+    vga_interface_created = true;
     switch (vga_interface_type) {
     case VGA_CIRRUS:
         return isa_create_simple(bus, "isa-cirrus-vga");
diff --git a/hw/mips/fuloong2e.c b/hw/mips/fuloong2e.c
index 7b13098f9b..5ee546f5f6 100644
--- a/hw/mips/fuloong2e.c
+++ b/hw/mips/fuloong2e.c
@@ -320,6 +320,7 @@ static void mips_fuloong2e_init(MachineState *machine)
 
     /* GPU */
     if (vga_interface_type != VGA_NONE) {
+        vga_interface_created = true;
         pci_dev = pci_new(-1, "ati-vga");
         dev = DEVICE(pci_dev);
         qdev_prop_set_uint32(dev, "vgamem_mb", 16);
diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index e99417e501..9c58f02853 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -2037,6 +2037,7 @@ PCIDevice *pci_nic_init_nofail(NICInfo *nd, PCIBus *rootbus,
 
 PCIDevice *pci_vga_init(PCIBus *bus)
 {
+    vga_interface_created = true;
     switch (vga_interface_type) {
     case VGA_CIRRUS:
         return pci_create_simple(bus, -1, "cirrus-vga");
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index fe9937e811..8bbae68e1b 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1742,6 +1742,7 @@ static void spapr_rtc_create(SpaprMachineState *spapr)
 /* Returns whether we want to use VGA or not */
 static bool spapr_vga_init(PCIBus *pci_bus, Error **errp)
 {
+    vga_interface_created = true;
     switch (vga_interface_type) {
     case VGA_NONE:
         return false;
diff --git a/hw/sparc/sun4m.c b/hw/sparc/sun4m.c
index fccaed1eb4..b693eea0e0 100644
--- a/hw/sparc/sun4m.c
+++ b/hw/sparc/sun4m.c
@@ -920,6 +920,7 @@ static void sun4m_hw_init(MachineState *machine)
             /* sbus irq 5 */
             cg3_init(hwdef->tcx_base, slavio_irq[11], 0x00100000,
                      graphic_width, graphic_height, graphic_depth);
+            vga_interface_created = true;
         } else {
             /* If no display specified, default to TCX */
             if (graphic_depth != 8 && graphic_depth != 24) {
@@ -935,6 +936,7 @@ static void sun4m_hw_init(MachineState *machine)
 
             tcx_init(hwdef->tcx_base, slavio_irq[11], 0x00100000,
                      graphic_width, graphic_height, graphic_depth);
+            vga_interface_created = true;
         }
     }
 
diff --git a/hw/sparc64/sun4u.c b/hw/sparc64/sun4u.c
index 6fd08e2298..7c461d194a 100644
--- a/hw/sparc64/sun4u.c
+++ b/hw/sparc64/sun4u.c
@@ -632,6 +632,7 @@ static void sun4uv_init(MemoryRegion *address_space_mem,
     switch (vga_interface_type) {
     case VGA_STD:
         pci_create_simple(pci_busA, PCI_DEVFN(2, 0), "VGA");
+        vga_interface_created = true;
         break;
     case VGA_NONE:
         break;
diff --git a/hw/xenpv/xen_machine_pv.c b/hw/xenpv/xen_machine_pv.c
index 8df575a457..20c9611d71 100644
--- a/hw/xenpv/xen_machine_pv.c
+++ b/hw/xenpv/xen_machine_pv.c
@@ -63,6 +63,7 @@ static void xen_init_pv(MachineState *machine)
     if (vga_interface_type == VGA_XENFB) {
         xen_config_dev_vfb(0, "vnc");
         xen_config_dev_vkbd(0);
+        vga_interface_created = true;
     }
 
     /* configure disks */
diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index 10e283c170..360a408edf 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -34,6 +34,7 @@ typedef enum {
 } VGAInterfaceType;
 
 extern int vga_interface_type;
+extern bool vga_interface_created;
 
 extern int graphic_width;
 extern int graphic_height;
diff --git a/softmmu/globals.c b/softmmu/globals.c
index 3ebd718e35..98b64e0492 100644
--- a/softmmu/globals.c
+++ b/softmmu/globals.c
@@ -40,6 +40,7 @@ int nb_nics;
 NICInfo nd_table[MAX_NICS];
 int autostart = 1;
 int vga_interface_type = VGA_NONE;
+bool vga_interface_created;
 Chardev *parallel_hds[MAX_PARALLEL_PORTS];
 int win2k_install_hack;
 int singlestep;
diff --git a/softmmu/vl.c b/softmmu/vl.c
index ad886fb878..488cc4d09e 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -1353,6 +1353,7 @@ static void qemu_disable_default_devices(void)
 
     if (!vga_model && !default_vga) {
         vga_interface_type = VGA_DEVICE;
+        vga_interface_created = true;
     }
     if (!has_defaults || machine_class->no_serial) {
         default_serial = 0;
@@ -2736,6 +2737,12 @@ static void qemu_machine_creation_done(void)
     if (foreach_device_config(DEV_GDB, gdbserver_start) < 0) {
         exit(1);
     }
+    if (!vga_interface_created && !default_vga &&
+        vga_interface_type != VGA_NONE) {
+        warn_report("A -vga option was passed but this machine "
+                    "type does not use that option; "
+                    "No VGA device has been created");
+    }
 }
 
 void qmp_x_exit_preconfig(Error **errp)

From ddc5a6cc70398ed7ec76220d59c123d8cb14b0ad Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Fri, 6 May 2022 17:01:46 +0200
Subject: [PATCH 185/935] docs/devel/writing-monitor-commands: Replace obsolete
 STEXI/ETEXI tags

STEXI and ETEXI is not used anymore since we switched to Sphinx.
Replace them in the example with SRST and ERST, too.

Message-Id: <20220506150146.564244-1-thuth@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 docs/devel/writing-monitor-commands.rst | 11 ++++-------
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/docs/devel/writing-monitor-commands.rst b/docs/devel/writing-monitor-commands.rst
index 1693822f8f..4aa2bb904d 100644
--- a/docs/devel/writing-monitor-commands.rst
+++ b/docs/devel/writing-monitor-commands.rst
@@ -331,13 +331,10 @@ we should add it to the hmp-commands.hx file::
         .cmd        = hmp_hello_world,
     },
 
-::
-
- STEXI
- @item hello_world @var{message}
- @findex hello_world
- Print message to the standard output
- ETEXI
+ SRST
+ ``hello_world`` *message*
+   Print message to the standard output
+ ERST
 
 To test this you have to open a user monitor and issue the "hello-world"
 command. It might be instructive to check the command's documentation with

From 7d5983e3c8c40b1d0668faba31d79905c4fadd7d Mon Sep 17 00:00:00 2001
From: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Date: Mon, 25 Apr 2022 09:57:21 +0200
Subject: [PATCH 186/935] Introduce event-loop-base abstract class

Introduce the 'event-loop-base' abstract class, it'll hold the
properties common to all event loops and provide the necessary hooks for
their creation and maintenance. Then have iothread inherit from it.

EventLoopBaseClass is defined as user creatable and provides a hook for
its children to attach themselves to the user creatable class 'complete'
function. It also provides an update_params() callback to propagate
property changes onto its children.

The new 'event-loop-base' class will live in the root directory. It is
built on its own using the 'link_whole' option (there are no direct
function dependencies between the class and its children, it all happens
trough 'constructor' magic). And also imposes new compilation
dependencies:

    qom <- event-loop-base <- blockdev (iothread.c)

And in subsequent patches:

    qom <- event-loop-base <- qemuutil (util/main-loop.c)

All this forced some amount of reordering in meson.build:

 - Moved qom build definition before qemuutil. Doing it the other way
   around (i.e. moving qemuutil after qom) isn't possible as a lot of
   core libraries that live in between the two depend on it.

 - Process the 'hw' subdir earlier, as it introduces files into the
   'qom' source set.

No functional changes intended.

Signed-off-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20220425075723.20019-2-nsaenzju@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 event-loop-base.c                | 104 +++++++++++++++++++++++++++++++
 include/sysemu/event-loop-base.h |  36 +++++++++++
 include/sysemu/iothread.h        |   6 +-
 iothread.c                       |  65 ++++++-------------
 meson.build                      |  23 ++++---
 qapi/qom.json                    |  22 +++++--
 6 files changed, 192 insertions(+), 64 deletions(-)
 create mode 100644 event-loop-base.c
 create mode 100644 include/sysemu/event-loop-base.h

diff --git a/event-loop-base.c b/event-loop-base.c
new file mode 100644
index 0000000000..a924c73a7c
--- /dev/null
+++ b/event-loop-base.c
@@ -0,0 +1,104 @@
+/*
+ * QEMU event-loop base
+ *
+ * Copyright (C) 2022 Red Hat Inc
+ *
+ * Authors:
+ *  Stefan Hajnoczi <stefanha@redhat.com>
+ *  Nicolas Saenz Julienne <nsaenzju@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qom/object_interfaces.h"
+#include "qapi/error.h"
+#include "sysemu/event-loop-base.h"
+
+typedef struct {
+    const char *name;
+    ptrdiff_t offset; /* field's byte offset in EventLoopBase struct */
+} EventLoopBaseParamInfo;
+
+static EventLoopBaseParamInfo aio_max_batch_info = {
+    "aio-max-batch", offsetof(EventLoopBase, aio_max_batch),
+};
+
+static void event_loop_base_get_param(Object *obj, Visitor *v,
+        const char *name, void *opaque, Error **errp)
+{
+    EventLoopBase *event_loop_base = EVENT_LOOP_BASE(obj);
+    EventLoopBaseParamInfo *info = opaque;
+    int64_t *field = (void *)event_loop_base + info->offset;
+
+    visit_type_int64(v, name, field, errp);
+}
+
+static void event_loop_base_set_param(Object *obj, Visitor *v,
+        const char *name, void *opaque, Error **errp)
+{
+    EventLoopBaseClass *bc = EVENT_LOOP_BASE_GET_CLASS(obj);
+    EventLoopBase *base = EVENT_LOOP_BASE(obj);
+    EventLoopBaseParamInfo *info = opaque;
+    int64_t *field = (void *)base + info->offset;
+    int64_t value;
+
+    if (!visit_type_int64(v, name, &value, errp)) {
+        return;
+    }
+
+    if (value < 0) {
+        error_setg(errp, "%s value must be in range [0, %" PRId64 "]",
+                   info->name, INT64_MAX);
+        return;
+    }
+
+    *field = value;
+
+    if (bc->update_params) {
+        bc->update_params(base, errp);
+    }
+
+    return;
+}
+
+static void event_loop_base_complete(UserCreatable *uc, Error **errp)
+{
+    EventLoopBaseClass *bc = EVENT_LOOP_BASE_GET_CLASS(uc);
+    EventLoopBase *base = EVENT_LOOP_BASE(uc);
+
+    if (bc->init) {
+        bc->init(base, errp);
+    }
+}
+
+static void event_loop_base_class_init(ObjectClass *klass, void *class_data)
+{
+    UserCreatableClass *ucc = USER_CREATABLE_CLASS(klass);
+    ucc->complete = event_loop_base_complete;
+
+    object_class_property_add(klass, "aio-max-batch", "int",
+                              event_loop_base_get_param,
+                              event_loop_base_set_param,
+                              NULL, &aio_max_batch_info);
+}
+
+static const TypeInfo event_loop_base_info = {
+    .name = TYPE_EVENT_LOOP_BASE,
+    .parent = TYPE_OBJECT,
+    .instance_size = sizeof(EventLoopBase),
+    .class_size = sizeof(EventLoopBaseClass),
+    .class_init = event_loop_base_class_init,
+    .abstract = true,
+    .interfaces = (InterfaceInfo[]) {
+        { TYPE_USER_CREATABLE },
+        { }
+    }
+};
+
+static void register_types(void)
+{
+    type_register_static(&event_loop_base_info);
+}
+type_init(register_types);
diff --git a/include/sysemu/event-loop-base.h b/include/sysemu/event-loop-base.h
new file mode 100644
index 0000000000..8e77d8b69f
--- /dev/null
+++ b/include/sysemu/event-loop-base.h
@@ -0,0 +1,36 @@
+/*
+ * QEMU event-loop backend
+ *
+ * Copyright (C) 2022 Red Hat Inc
+ *
+ * Authors:
+ *  Nicolas Saenz Julienne <nsaenzju@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#ifndef QEMU_EVENT_LOOP_BASE_H
+#define QEMU_EVENT_LOOP_BASE_H
+
+#include "qom/object.h"
+#include "block/aio.h"
+#include "qemu/typedefs.h"
+
+#define TYPE_EVENT_LOOP_BASE         "event-loop-base"
+OBJECT_DECLARE_TYPE(EventLoopBase, EventLoopBaseClass,
+                    EVENT_LOOP_BASE)
+
+struct EventLoopBaseClass {
+    ObjectClass parent_class;
+
+    void (*init)(EventLoopBase *base, Error **errp);
+    void (*update_params)(EventLoopBase *base, Error **errp);
+};
+
+struct EventLoopBase {
+    Object parent;
+
+    /* AioContext AIO engine parameters */
+    int64_t aio_max_batch;
+};
+#endif
diff --git a/include/sysemu/iothread.h b/include/sysemu/iothread.h
index 7f714bd136..8f8601d6ab 100644
--- a/include/sysemu/iothread.h
+++ b/include/sysemu/iothread.h
@@ -17,11 +17,12 @@
 #include "block/aio.h"
 #include "qemu/thread.h"
 #include "qom/object.h"
+#include "sysemu/event-loop-base.h"
 
 #define TYPE_IOTHREAD "iothread"
 
 struct IOThread {
-    Object parent_obj;
+    EventLoopBase parent_obj;
 
     QemuThread thread;
     AioContext *ctx;
@@ -37,9 +38,6 @@ struct IOThread {
     int64_t poll_max_ns;
     int64_t poll_grow;
     int64_t poll_shrink;
-
-    /* AioContext AIO engine parameters */
-    int64_t aio_max_batch;
 };
 typedef struct IOThread IOThread;
 
diff --git a/iothread.c b/iothread.c
index 0f98af0f2a..8fa2f3bfb8 100644
--- a/iothread.c
+++ b/iothread.c
@@ -17,6 +17,7 @@
 #include "qemu/module.h"
 #include "block/aio.h"
 #include "block/block.h"
+#include "sysemu/event-loop-base.h"
 #include "sysemu/iothread.h"
 #include "qapi/error.h"
 #include "qapi/qapi-commands-misc.h"
@@ -152,10 +153,15 @@ static void iothread_init_gcontext(IOThread *iothread)
     iothread->main_loop = g_main_loop_new(iothread->worker_context, TRUE);
 }
 
-static void iothread_set_aio_context_params(IOThread *iothread, Error **errp)
+static void iothread_set_aio_context_params(EventLoopBase *base, Error **errp)
 {
+    IOThread *iothread = IOTHREAD(base);
     ERRP_GUARD();
 
+    if (!iothread->ctx) {
+        return;
+    }
+
     aio_context_set_poll_params(iothread->ctx,
                                 iothread->poll_max_ns,
                                 iothread->poll_grow,
@@ -166,14 +172,15 @@ static void iothread_set_aio_context_params(IOThread *iothread, Error **errp)
     }
 
     aio_context_set_aio_params(iothread->ctx,
-                               iothread->aio_max_batch,
+                               iothread->parent_obj.aio_max_batch,
                                errp);
 }
 
-static void iothread_complete(UserCreatable *obj, Error **errp)
+
+static void iothread_init(EventLoopBase *base, Error **errp)
 {
     Error *local_error = NULL;
-    IOThread *iothread = IOTHREAD(obj);
+    IOThread *iothread = IOTHREAD(base);
     char *thread_name;
 
     iothread->stopping = false;
@@ -189,7 +196,7 @@ static void iothread_complete(UserCreatable *obj, Error **errp)
      */
     iothread_init_gcontext(iothread);
 
-    iothread_set_aio_context_params(iothread, &local_error);
+    iothread_set_aio_context_params(base, &local_error);
     if (local_error) {
         error_propagate(errp, local_error);
         aio_context_unref(iothread->ctx);
@@ -201,7 +208,7 @@ static void iothread_complete(UserCreatable *obj, Error **errp)
      * to inherit.
      */
     thread_name = g_strdup_printf("IO %s",
-                        object_get_canonical_path_component(OBJECT(obj)));
+                        object_get_canonical_path_component(OBJECT(base)));
     qemu_thread_create(&iothread->thread, thread_name, iothread_run,
                        iothread, QEMU_THREAD_JOINABLE);
     g_free(thread_name);
@@ -226,9 +233,6 @@ static IOThreadParamInfo poll_grow_info = {
 static IOThreadParamInfo poll_shrink_info = {
     "poll-shrink", offsetof(IOThread, poll_shrink),
 };
-static IOThreadParamInfo aio_max_batch_info = {
-    "aio-max-batch", offsetof(IOThread, aio_max_batch),
-};
 
 static void iothread_get_param(Object *obj, Visitor *v,
         const char *name, IOThreadParamInfo *info, Error **errp)
@@ -288,35 +292,12 @@ static void iothread_set_poll_param(Object *obj, Visitor *v,
     }
 }
 
-static void iothread_get_aio_param(Object *obj, Visitor *v,
-        const char *name, void *opaque, Error **errp)
-{
-    IOThreadParamInfo *info = opaque;
-
-    iothread_get_param(obj, v, name, info, errp);
-}
-
-static void iothread_set_aio_param(Object *obj, Visitor *v,
-        const char *name, void *opaque, Error **errp)
-{
-    IOThread *iothread = IOTHREAD(obj);
-    IOThreadParamInfo *info = opaque;
-
-    if (!iothread_set_param(obj, v, name, info, errp)) {
-        return;
-    }
-
-    if (iothread->ctx) {
-        aio_context_set_aio_params(iothread->ctx,
-                                   iothread->aio_max_batch,
-                                   errp);
-    }
-}
-
 static void iothread_class_init(ObjectClass *klass, void *class_data)
 {
-    UserCreatableClass *ucc = USER_CREATABLE_CLASS(klass);
-    ucc->complete = iothread_complete;
+    EventLoopBaseClass *bc = EVENT_LOOP_BASE_CLASS(klass);
+
+    bc->init = iothread_init;
+    bc->update_params = iothread_set_aio_context_params;
 
     object_class_property_add(klass, "poll-max-ns", "int",
                               iothread_get_poll_param,
@@ -330,23 +311,15 @@ static void iothread_class_init(ObjectClass *klass, void *class_data)
                               iothread_get_poll_param,
                               iothread_set_poll_param,
                               NULL, &poll_shrink_info);
-    object_class_property_add(klass, "aio-max-batch", "int",
-                              iothread_get_aio_param,
-                              iothread_set_aio_param,
-                              NULL, &aio_max_batch_info);
 }
 
 static const TypeInfo iothread_info = {
     .name = TYPE_IOTHREAD,
-    .parent = TYPE_OBJECT,
+    .parent = TYPE_EVENT_LOOP_BASE,
     .class_init = iothread_class_init,
     .instance_size = sizeof(IOThread),
     .instance_init = iothread_instance_init,
     .instance_finalize = iothread_instance_finalize,
-    .interfaces = (InterfaceInfo[]) {
-        {TYPE_USER_CREATABLE},
-        {}
-    },
 };
 
 static void iothread_register_types(void)
@@ -383,7 +356,7 @@ static int query_one_iothread(Object *object, void *opaque)
     info->poll_max_ns = iothread->poll_max_ns;
     info->poll_grow = iothread->poll_grow;
     info->poll_shrink = iothread->poll_shrink;
-    info->aio_max_batch = iothread->aio_max_batch;
+    info->aio_max_batch = iothread->parent_obj.aio_max_batch;
 
     QAPI_LIST_APPEND(*tail, info);
     return 0;
diff --git a/meson.build b/meson.build
index fa672e57bc..85c151bf6c 100644
--- a/meson.build
+++ b/meson.build
@@ -3025,6 +3025,7 @@ subdir('qom')
 subdir('authz')
 subdir('crypto')
 subdir('ui')
+subdir('hw')
 
 
 if enable_modules
@@ -3032,6 +3033,18 @@ if enable_modules
   modulecommon = declare_dependency(link_whole: libmodulecommon, compile_args: '-DBUILD_DSO')
 endif
 
+qom_ss = qom_ss.apply(config_host, strict: false)
+libqom = static_library('qom', qom_ss.sources() + genh,
+                        dependencies: [qom_ss.dependencies()],
+                        name_suffix: 'fa')
+qom = declare_dependency(link_whole: libqom)
+
+event_loop_base = files('event-loop-base.c')
+event_loop_base = static_library('event-loop-base', sources: event_loop_base + genh,
+                                 build_by_default: true)
+event_loop_base = declare_dependency(link_whole: event_loop_base,
+                                     dependencies: [qom])
+
 stub_ss = stub_ss.apply(config_all, strict: false)
 
 util_ss.add_all(trace_ss)
@@ -3118,7 +3131,6 @@ subdir('monitor')
 subdir('net')
 subdir('replay')
 subdir('semihosting')
-subdir('hw')
 subdir('tcg')
 subdir('fpu')
 subdir('accel')
@@ -3243,13 +3255,6 @@ qemu_syms = custom_target('qemu.syms', output: 'qemu.syms',
                              capture: true,
                              command: [undefsym, nm, '@INPUT@'])
 
-qom_ss = qom_ss.apply(config_host, strict: false)
-libqom = static_library('qom', qom_ss.sources() + genh,
-                        dependencies: [qom_ss.dependencies()],
-                        name_suffix: 'fa')
-
-qom = declare_dependency(link_whole: libqom)
-
 authz_ss = authz_ss.apply(config_host, strict: false)
 libauthz = static_library('authz', authz_ss.sources() + genh,
                           dependencies: [authz_ss.dependencies()],
@@ -3302,7 +3307,7 @@ libblockdev = static_library('blockdev', blockdev_ss.sources() + genh,
                              build_by_default: false)
 
 blockdev = declare_dependency(link_whole: [libblockdev],
-                              dependencies: [block])
+                              dependencies: [block, event_loop_base])
 
 qmp_ss = qmp_ss.apply(config_host, strict: false)
 libqmp = static_library('qmp', qmp_ss.sources() + genh,
diff --git a/qapi/qom.json b/qapi/qom.json
index eeb5395ff3..a2439533c5 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -499,6 +499,20 @@
             '*repeat': 'bool',
             '*grab-toggle': 'GrabToggleKeys' } }
 
+##
+# @EventLoopBaseProperties:
+#
+# Common properties for event loops
+#
+# @aio-max-batch: maximum number of requests in a batch for the AIO engine,
+#                 0 means that the engine will use its default.
+#                 (default: 0)
+#
+# Since: 7.1
+##
+{ 'struct': 'EventLoopBaseProperties',
+  'data': { '*aio-max-batch': 'int' } }
+
 ##
 # @IothreadProperties:
 #
@@ -516,17 +530,15 @@
 #               algorithm detects it is spending too long polling without
 #               encountering events. 0 selects a default behaviour (default: 0)
 #
-# @aio-max-batch: maximum number of requests in a batch for the AIO engine,
-#                 0 means that the engine will use its default
-#                 (default:0, since 6.1)
+# The @aio-max-batch option is available since 6.1.
 #
 # Since: 2.0
 ##
 { 'struct': 'IothreadProperties',
+  'base': 'EventLoopBaseProperties',
   'data': { '*poll-max-ns': 'int',
             '*poll-grow': 'int',
-            '*poll-shrink': 'int',
-            '*aio-max-batch': 'int' } }
+            '*poll-shrink': 'int' } }
 
 ##
 # @MemoryBackendProperties:

From 70ac26b9e5ca8374bb3ef3f30b871726673c9f27 Mon Sep 17 00:00:00 2001
From: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Date: Mon, 25 Apr 2022 09:57:22 +0200
Subject: [PATCH 187/935] util/main-loop: Introduce the main loop into QOM

'event-loop-base' provides basic property handling for all 'AioContext'
based event loops. So let's define a new 'MainLoopClass' that inherits
from it. This will permit tweaking the main loop's properties through
qapi as well as through the command line using the '-object' keyword[1].
Only one instance of 'MainLoopClass' might be created at any time.

'EventLoopBaseClass' learns a new callback, 'can_be_deleted()' so as to
mark 'MainLoop' as non-deletable.

[1] For example:
      -object main-loop,id=main-loop,aio-max-batch=<value>

Signed-off-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20220425075723.20019-3-nsaenzju@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 event-loop-base.c                | 13 ++++++++
 include/qemu/main-loop.h         | 10 ++++++
 include/sysemu/event-loop-base.h |  1 +
 meson.build                      |  3 +-
 qapi/qom.json                    | 13 ++++++++
 util/main-loop.c                 | 56 ++++++++++++++++++++++++++++++++
 6 files changed, 95 insertions(+), 1 deletion(-)

diff --git a/event-loop-base.c b/event-loop-base.c
index a924c73a7c..e7f99a6ec8 100644
--- a/event-loop-base.c
+++ b/event-loop-base.c
@@ -73,10 +73,23 @@ static void event_loop_base_complete(UserCreatable *uc, Error **errp)
     }
 }
 
+static bool event_loop_base_can_be_deleted(UserCreatable *uc)
+{
+    EventLoopBaseClass *bc = EVENT_LOOP_BASE_GET_CLASS(uc);
+    EventLoopBase *backend = EVENT_LOOP_BASE(uc);
+
+    if (bc->can_be_deleted) {
+        return bc->can_be_deleted(backend);
+    }
+
+    return true;
+}
+
 static void event_loop_base_class_init(ObjectClass *klass, void *class_data)
 {
     UserCreatableClass *ucc = USER_CREATABLE_CLASS(klass);
     ucc->complete = event_loop_base_complete;
+    ucc->can_be_deleted = event_loop_base_can_be_deleted;
 
     object_class_property_add(klass, "aio-max-batch", "int",
                               event_loop_base_get_param,
diff --git a/include/qemu/main-loop.h b/include/qemu/main-loop.h
index 89bd9edefb..5518845299 100644
--- a/include/qemu/main-loop.h
+++ b/include/qemu/main-loop.h
@@ -26,9 +26,19 @@
 #define QEMU_MAIN_LOOP_H
 
 #include "block/aio.h"
+#include "qom/object.h"
+#include "sysemu/event-loop-base.h"
 
 #define SIG_IPI SIGUSR1
 
+#define TYPE_MAIN_LOOP  "main-loop"
+OBJECT_DECLARE_TYPE(MainLoop, MainLoopClass, MAIN_LOOP)
+
+struct MainLoop {
+    EventLoopBase parent_obj;
+};
+typedef struct MainLoop MainLoop;
+
 /**
  * qemu_init_main_loop: Set up the process so that it can run the main loop.
  *
diff --git a/include/sysemu/event-loop-base.h b/include/sysemu/event-loop-base.h
index 8e77d8b69f..fced4c9fea 100644
--- a/include/sysemu/event-loop-base.h
+++ b/include/sysemu/event-loop-base.h
@@ -25,6 +25,7 @@ struct EventLoopBaseClass {
 
     void (*init)(EventLoopBase *base, Error **errp);
     void (*update_params)(EventLoopBase *base, Error **errp);
+    bool (*can_be_deleted)(EventLoopBase *base);
 };
 
 struct EventLoopBase {
diff --git a/meson.build b/meson.build
index 85c151bf6c..864e97945f 100644
--- a/meson.build
+++ b/meson.build
@@ -3053,7 +3053,8 @@ libqemuutil = static_library('qemuutil',
                              sources: util_ss.sources() + stub_ss.sources() + genh,
                              dependencies: [util_ss.dependencies(), libm, threads, glib, socket, malloc, pixman])
 qemuutil = declare_dependency(link_with: libqemuutil,
-                              sources: genh + version_res)
+                              sources: genh + version_res,
+                              dependencies: [event_loop_base])
 
 if have_system or have_user
   decodetree = generator(find_program('scripts/decodetree.py'),
diff --git a/qapi/qom.json b/qapi/qom.json
index a2439533c5..7d4a2ac1b9 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -540,6 +540,17 @@
             '*poll-grow': 'int',
             '*poll-shrink': 'int' } }
 
+##
+# @MainLoopProperties:
+#
+# Properties for the main-loop object.
+#
+# Since: 7.1
+##
+{ 'struct': 'MainLoopProperties',
+  'base': 'EventLoopBaseProperties',
+  'data': {} }
+
 ##
 # @MemoryBackendProperties:
 #
@@ -830,6 +841,7 @@
     { 'name': 'input-linux',
       'if': 'CONFIG_LINUX' },
     'iothread',
+    'main-loop',
     { 'name': 'memory-backend-epc',
       'if': 'CONFIG_LINUX' },
     'memory-backend-file',
@@ -895,6 +907,7 @@
       'input-linux':                { 'type': 'InputLinuxProperties',
                                       'if': 'CONFIG_LINUX' },
       'iothread':                   'IothreadProperties',
+      'main-loop':                  'MainLoopProperties',
       'memory-backend-epc':         { 'type': 'MemoryBackendEpcProperties',
                                       'if': 'CONFIG_LINUX' },
       'memory-backend-file':        'MemoryBackendFileProperties',
diff --git a/util/main-loop.c b/util/main-loop.c
index 9afac10dff..e30f034815 100644
--- a/util/main-loop.c
+++ b/util/main-loop.c
@@ -33,6 +33,7 @@
 #include "qemu/error-report.h"
 #include "qemu/queue.h"
 #include "qemu/compiler.h"
+#include "qom/object.h"
 
 #ifndef _WIN32
 #include <sys/wait.h>
@@ -184,6 +185,61 @@ int qemu_init_main_loop(Error **errp)
     return 0;
 }
 
+static void main_loop_update_params(EventLoopBase *base, Error **errp)
+{
+    if (!qemu_aio_context) {
+        error_setg(errp, "qemu aio context not ready");
+        return;
+    }
+
+    aio_context_set_aio_params(qemu_aio_context, base->aio_max_batch, errp);
+}
+
+MainLoop *mloop;
+
+static void main_loop_init(EventLoopBase *base, Error **errp)
+{
+    MainLoop *m = MAIN_LOOP(base);
+
+    if (mloop) {
+        error_setg(errp, "only one main-loop instance allowed");
+        return;
+    }
+
+    main_loop_update_params(base, errp);
+
+    mloop = m;
+    return;
+}
+
+static bool main_loop_can_be_deleted(EventLoopBase *base)
+{
+    return false;
+}
+
+static void main_loop_class_init(ObjectClass *oc, void *class_data)
+{
+    EventLoopBaseClass *bc = EVENT_LOOP_BASE_CLASS(oc);
+
+    bc->init = main_loop_init;
+    bc->update_params = main_loop_update_params;
+    bc->can_be_deleted = main_loop_can_be_deleted;
+}
+
+static const TypeInfo main_loop_info = {
+    .name = TYPE_MAIN_LOOP,
+    .parent = TYPE_EVENT_LOOP_BASE,
+    .class_init = main_loop_class_init,
+    .instance_size = sizeof(MainLoop),
+};
+
+static void main_loop_register_types(void)
+{
+    type_register_static(&main_loop_info);
+}
+
+type_init(main_loop_register_types)
+
 static int max_priority;
 
 #ifndef _WIN32

From 71ad4713cc1d7fca24388b828ef31ae6cb38a31c Mon Sep 17 00:00:00 2001
From: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Date: Mon, 25 Apr 2022 09:57:23 +0200
Subject: [PATCH 188/935] util/event-loop-base: Introduce options to set the
 thread pool size

The thread pool regulates itself: when idle, it kills threads until
empty, when in demand, it creates new threads until full. This behaviour
doesn't play well with latency sensitive workloads where the price of
creating a new thread is too high. For example, when paired with qemu's
'-mlock', or using safety features like SafeStack, creating a new thread
has been measured take multiple milliseconds.

In order to mitigate this let's introduce a new 'EventLoopBase'
property to set the thread pool size. The threads will be created during
the pool's initialization or upon updating the property's value, remain
available during its lifetime regardless of demand, and destroyed upon
freeing it. A properly characterized workload will then be able to
configure the pool to avoid any latency spikes.

Signed-off-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20220425075723.20019-4-nsaenzju@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 event-loop-base.c                | 23 +++++++++++++
 include/block/aio.h              | 10 ++++++
 include/block/thread-pool.h      |  3 ++
 include/sysemu/event-loop-base.h |  4 +++
 iothread.c                       |  3 ++
 qapi/qom.json                    | 10 +++++-
 util/aio-posix.c                 |  1 +
 util/async.c                     | 20 ++++++++++++
 util/main-loop.c                 |  9 ++++++
 util/thread-pool.c               | 55 +++++++++++++++++++++++++++++---
 10 files changed, 133 insertions(+), 5 deletions(-)

diff --git a/event-loop-base.c b/event-loop-base.c
index e7f99a6ec8..d5be4dc6fc 100644
--- a/event-loop-base.c
+++ b/event-loop-base.c
@@ -14,6 +14,7 @@
 #include "qemu/osdep.h"
 #include "qom/object_interfaces.h"
 #include "qapi/error.h"
+#include "block/thread-pool.h"
 #include "sysemu/event-loop-base.h"
 
 typedef struct {
@@ -21,9 +22,22 @@ typedef struct {
     ptrdiff_t offset; /* field's byte offset in EventLoopBase struct */
 } EventLoopBaseParamInfo;
 
+static void event_loop_base_instance_init(Object *obj)
+{
+    EventLoopBase *base = EVENT_LOOP_BASE(obj);
+
+    base->thread_pool_max = THREAD_POOL_MAX_THREADS_DEFAULT;
+}
+
 static EventLoopBaseParamInfo aio_max_batch_info = {
     "aio-max-batch", offsetof(EventLoopBase, aio_max_batch),
 };
+static EventLoopBaseParamInfo thread_pool_min_info = {
+    "thread-pool-min", offsetof(EventLoopBase, thread_pool_min),
+};
+static EventLoopBaseParamInfo thread_pool_max_info = {
+    "thread-pool-max", offsetof(EventLoopBase, thread_pool_max),
+};
 
 static void event_loop_base_get_param(Object *obj, Visitor *v,
         const char *name, void *opaque, Error **errp)
@@ -95,12 +109,21 @@ static void event_loop_base_class_init(ObjectClass *klass, void *class_data)
                               event_loop_base_get_param,
                               event_loop_base_set_param,
                               NULL, &aio_max_batch_info);
+    object_class_property_add(klass, "thread-pool-min", "int",
+                              event_loop_base_get_param,
+                              event_loop_base_set_param,
+                              NULL, &thread_pool_min_info);
+    object_class_property_add(klass, "thread-pool-max", "int",
+                              event_loop_base_get_param,
+                              event_loop_base_set_param,
+                              NULL, &thread_pool_max_info);
 }
 
 static const TypeInfo event_loop_base_info = {
     .name = TYPE_EVENT_LOOP_BASE,
     .parent = TYPE_OBJECT,
     .instance_size = sizeof(EventLoopBase),
+    .instance_init = event_loop_base_instance_init,
     .class_size = sizeof(EventLoopBaseClass),
     .class_init = event_loop_base_class_init,
     .abstract = true,
diff --git a/include/block/aio.h b/include/block/aio.h
index 5634173b12..d128558f1d 100644
--- a/include/block/aio.h
+++ b/include/block/aio.h
@@ -192,6 +192,8 @@ struct AioContext {
     QSLIST_HEAD(, Coroutine) scheduled_coroutines;
     QEMUBH *co_schedule_bh;
 
+    int thread_pool_min;
+    int thread_pool_max;
     /* Thread pool for performing work and receiving completion callbacks.
      * Has its own locking.
      */
@@ -769,4 +771,12 @@ void aio_context_set_poll_params(AioContext *ctx, int64_t max_ns,
 void aio_context_set_aio_params(AioContext *ctx, int64_t max_batch,
                                 Error **errp);
 
+/**
+ * aio_context_set_thread_pool_params:
+ * @ctx: the aio context
+ * @min: min number of threads to have readily available in the thread pool
+ * @min: max number of threads the thread pool can contain
+ */
+void aio_context_set_thread_pool_params(AioContext *ctx, int64_t min,
+                                        int64_t max, Error **errp);
 #endif
diff --git a/include/block/thread-pool.h b/include/block/thread-pool.h
index 7dd7d730a0..2020bcc92d 100644
--- a/include/block/thread-pool.h
+++ b/include/block/thread-pool.h
@@ -20,6 +20,8 @@
 
 #include "block/block.h"
 
+#define THREAD_POOL_MAX_THREADS_DEFAULT         64
+
 typedef int ThreadPoolFunc(void *opaque);
 
 typedef struct ThreadPool ThreadPool;
@@ -33,5 +35,6 @@ BlockAIOCB *thread_pool_submit_aio(ThreadPool *pool,
 int coroutine_fn thread_pool_submit_co(ThreadPool *pool,
         ThreadPoolFunc *func, void *arg);
 void thread_pool_submit(ThreadPool *pool, ThreadPoolFunc *func, void *arg);
+void thread_pool_update_params(ThreadPool *pool, struct AioContext *ctx);
 
 #endif
diff --git a/include/sysemu/event-loop-base.h b/include/sysemu/event-loop-base.h
index fced4c9fea..2748bf6ae1 100644
--- a/include/sysemu/event-loop-base.h
+++ b/include/sysemu/event-loop-base.h
@@ -33,5 +33,9 @@ struct EventLoopBase {
 
     /* AioContext AIO engine parameters */
     int64_t aio_max_batch;
+
+    /* AioContext thread pool parameters */
+    int64_t thread_pool_min;
+    int64_t thread_pool_max;
 };
 #endif
diff --git a/iothread.c b/iothread.c
index 8fa2f3bfb8..529194a566 100644
--- a/iothread.c
+++ b/iothread.c
@@ -174,6 +174,9 @@ static void iothread_set_aio_context_params(EventLoopBase *base, Error **errp)
     aio_context_set_aio_params(iothread->ctx,
                                iothread->parent_obj.aio_max_batch,
                                errp);
+
+    aio_context_set_thread_pool_params(iothread->ctx, base->thread_pool_min,
+                                       base->thread_pool_max, errp);
 }
 
 
diff --git a/qapi/qom.json b/qapi/qom.json
index 7d4a2ac1b9..6a653c6636 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -508,10 +508,18 @@
 #                 0 means that the engine will use its default.
 #                 (default: 0)
 #
+# @thread-pool-min: minimum number of threads reserved in the thread pool
+#                   (default:0)
+#
+# @thread-pool-max: maximum number of threads the thread pool can contain
+#                   (default:64)
+#
 # Since: 7.1
 ##
 { 'struct': 'EventLoopBaseProperties',
-  'data': { '*aio-max-batch': 'int' } }
+  'data': { '*aio-max-batch': 'int',
+            '*thread-pool-min': 'int',
+            '*thread-pool-max': 'int' } }
 
 ##
 # @IothreadProperties:
diff --git a/util/aio-posix.c b/util/aio-posix.c
index be0182a3c6..731f3826c0 100644
--- a/util/aio-posix.c
+++ b/util/aio-posix.c
@@ -15,6 +15,7 @@
 
 #include "qemu/osdep.h"
 #include "block/block.h"
+#include "block/thread-pool.h"
 #include "qemu/main-loop.h"
 #include "qemu/rcu.h"
 #include "qemu/rcu_queue.h"
diff --git a/util/async.c b/util/async.c
index 2ea1172f3e..554ba70cca 100644
--- a/util/async.c
+++ b/util/async.c
@@ -563,6 +563,9 @@ AioContext *aio_context_new(Error **errp)
 
     ctx->aio_max_batch = 0;
 
+    ctx->thread_pool_min = 0;
+    ctx->thread_pool_max = THREAD_POOL_MAX_THREADS_DEFAULT;
+
     return ctx;
 fail:
     g_source_destroy(&ctx->source);
@@ -696,3 +699,20 @@ void qemu_set_current_aio_context(AioContext *ctx)
     assert(!get_my_aiocontext());
     set_my_aiocontext(ctx);
 }
+
+void aio_context_set_thread_pool_params(AioContext *ctx, int64_t min,
+                                        int64_t max, Error **errp)
+{
+
+    if (min > max || !max || min > INT_MAX || max > INT_MAX) {
+        error_setg(errp, "bad thread-pool-min/thread-pool-max values");
+        return;
+    }
+
+    ctx->thread_pool_min = min;
+    ctx->thread_pool_max = max;
+
+    if (ctx->thread_pool) {
+        thread_pool_update_params(ctx->thread_pool, ctx);
+    }
+}
diff --git a/util/main-loop.c b/util/main-loop.c
index e30f034815..f00a25451b 100644
--- a/util/main-loop.c
+++ b/util/main-loop.c
@@ -30,6 +30,7 @@
 #include "sysemu/replay.h"
 #include "qemu/main-loop.h"
 #include "block/aio.h"
+#include "block/thread-pool.h"
 #include "qemu/error-report.h"
 #include "qemu/queue.h"
 #include "qemu/compiler.h"
@@ -187,12 +188,20 @@ int qemu_init_main_loop(Error **errp)
 
 static void main_loop_update_params(EventLoopBase *base, Error **errp)
 {
+    ERRP_GUARD();
+
     if (!qemu_aio_context) {
         error_setg(errp, "qemu aio context not ready");
         return;
     }
 
     aio_context_set_aio_params(qemu_aio_context, base->aio_max_batch, errp);
+    if (*errp) {
+        return;
+    }
+
+    aio_context_set_thread_pool_params(qemu_aio_context, base->thread_pool_min,
+                                       base->thread_pool_max, errp);
 }
 
 MainLoop *mloop;
diff --git a/util/thread-pool.c b/util/thread-pool.c
index d763cea505..196835b4d3 100644
--- a/util/thread-pool.c
+++ b/util/thread-pool.c
@@ -58,7 +58,6 @@ struct ThreadPool {
     QemuMutex lock;
     QemuCond worker_stopped;
     QemuSemaphore sem;
-    int max_threads;
     QEMUBH *new_thread_bh;
 
     /* The following variables are only accessed from one AioContext. */
@@ -71,8 +70,27 @@ struct ThreadPool {
     int new_threads;     /* backlog of threads we need to create */
     int pending_threads; /* threads created but not running yet */
     bool stopping;
+    int min_threads;
+    int max_threads;
 };
 
+static inline bool back_to_sleep(ThreadPool *pool, int ret)
+{
+    /*
+     * The semaphore timed out, we should exit the loop except when:
+     *  - There is work to do, we raced with the signal.
+     *  - The max threads threshold just changed, we raced with the signal.
+     *  - The thread pool forces a minimum number of readily available threads.
+     */
+    if (ret == -1 && (!QTAILQ_EMPTY(&pool->request_list) ||
+            pool->cur_threads > pool->max_threads ||
+            pool->cur_threads <= pool->min_threads)) {
+            return true;
+    }
+
+    return false;
+}
+
 static void *worker_thread(void *opaque)
 {
     ThreadPool *pool = opaque;
@@ -91,8 +109,9 @@ static void *worker_thread(void *opaque)
             ret = qemu_sem_timedwait(&pool->sem, 10000);
             qemu_mutex_lock(&pool->lock);
             pool->idle_threads--;
-        } while (ret == -1 && !QTAILQ_EMPTY(&pool->request_list));
-        if (ret == -1 || pool->stopping) {
+        } while (back_to_sleep(pool, ret));
+        if (ret == -1 || pool->stopping ||
+            pool->cur_threads > pool->max_threads) {
             break;
         }
 
@@ -294,6 +313,33 @@ void thread_pool_submit(ThreadPool *pool, ThreadPoolFunc *func, void *arg)
     thread_pool_submit_aio(pool, func, arg, NULL, NULL);
 }
 
+void thread_pool_update_params(ThreadPool *pool, AioContext *ctx)
+{
+    qemu_mutex_lock(&pool->lock);
+
+    pool->min_threads = ctx->thread_pool_min;
+    pool->max_threads = ctx->thread_pool_max;
+
+    /*
+     * We either have to:
+     *  - Increase the number available of threads until over the min_threads
+     *    threshold.
+     *  - Decrease the number of available threads until under the max_threads
+     *    threshold.
+     *  - Do nothing. The current number of threads fall in between the min and
+     *    max thresholds. We'll let the pool manage itself.
+     */
+    for (int i = pool->cur_threads; i < pool->min_threads; i++) {
+        spawn_thread(pool);
+    }
+
+    for (int i = pool->cur_threads; i > pool->max_threads; i--) {
+        qemu_sem_post(&pool->sem);
+    }
+
+    qemu_mutex_unlock(&pool->lock);
+}
+
 static void thread_pool_init_one(ThreadPool *pool, AioContext *ctx)
 {
     if (!ctx) {
@@ -306,11 +352,12 @@ static void thread_pool_init_one(ThreadPool *pool, AioContext *ctx)
     qemu_mutex_init(&pool->lock);
     qemu_cond_init(&pool->worker_stopped);
     qemu_sem_init(&pool->sem, 0);
-    pool->max_threads = 64;
     pool->new_thread_bh = aio_bh_new(ctx, spawn_thread_bh_fn, pool);
 
     QLIST_INIT(&pool->head);
     QTAILQ_INIT(&pool->request_list);
+
+    thread_pool_update_params(pool, ctx);
 }
 
 ThreadPool *thread_pool_new(AioContext *ctx)

From 2f743ef6366c2df4ef51ef3ae318138cdc0125ab Mon Sep 17 00:00:00 2001
From: Stefan Hajnoczi <stefanha@redhat.com>
Date: Wed, 27 Apr 2022 15:35:36 +0100
Subject: [PATCH 189/935] virtio-scsi: fix ctrl and event handler functions in
 dataplane mode

Commit f34e8d8b8d48d73f36a67b6d5e492ef9784b5012 ("virtio-scsi: prepare
virtio_scsi_handle_cmd for dataplane") prepared the virtio-scsi cmd
virtqueue handler function to be used in both the dataplane and
non-datpalane code paths.

It failed to convert the ctrl and event virtqueue handler functions,
which are not designed to be called from the dataplane code path but
will be since the ioeventfd is set up for those virtqueues when
dataplane starts.

Convert the ctrl and event virtqueue handler functions now so they
operate correctly when called from the dataplane code path. Avoid code
duplication by extracting this code into a helper function.

Fixes: f34e8d8b8d48d73f36a67b6d5e492ef9784b5012 ("virtio-scsi: prepare virtio_scsi_handle_cmd for dataplane")
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20220427143541.119567-2-stefanha@redhat.com
[Fixed s/by used/be used/ typo pointed out by Michael Tokarev
<mjt@tls.msk.ru>.
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/scsi/virtio-scsi.c | 42 +++++++++++++++++++++++++++---------------
 1 file changed, 27 insertions(+), 15 deletions(-)

diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
index 34a968ecfb..417fbc71d6 100644
--- a/hw/scsi/virtio-scsi.c
+++ b/hw/scsi/virtio-scsi.c
@@ -472,16 +472,32 @@ bool virtio_scsi_handle_ctrl_vq(VirtIOSCSI *s, VirtQueue *vq)
     return progress;
 }
 
+/*
+ * If dataplane is configured but not yet started, do so now and return true on
+ * success.
+ *
+ * Dataplane is started by the core virtio code but virtqueue handler functions
+ * can also be invoked when a guest kicks before DRIVER_OK, so this helper
+ * function helps us deal with manually starting ioeventfd in that case.
+ */
+static bool virtio_scsi_defer_to_dataplane(VirtIOSCSI *s)
+{
+    if (!s->ctx || s->dataplane_started) {
+        return false;
+    }
+
+    virtio_device_start_ioeventfd(&s->parent_obj.parent_obj);
+    return !s->dataplane_fenced;
+}
+
 static void virtio_scsi_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq)
 {
     VirtIOSCSI *s = (VirtIOSCSI *)vdev;
 
-    if (s->ctx) {
-        virtio_device_start_ioeventfd(vdev);
-        if (!s->dataplane_fenced) {
-            return;
-        }
+    if (virtio_scsi_defer_to_dataplane(s)) {
+        return;
     }
+
     virtio_scsi_acquire(s);
     virtio_scsi_handle_ctrl_vq(s, vq);
     virtio_scsi_release(s);
@@ -720,12 +736,10 @@ static void virtio_scsi_handle_cmd(VirtIODevice *vdev, VirtQueue *vq)
     /* use non-QOM casts in the data path */
     VirtIOSCSI *s = (VirtIOSCSI *)vdev;
 
-    if (s->ctx && !s->dataplane_started) {
-        virtio_device_start_ioeventfd(vdev);
-        if (!s->dataplane_fenced) {
-            return;
-        }
+    if (virtio_scsi_defer_to_dataplane(s)) {
+        return;
     }
+
     virtio_scsi_acquire(s);
     virtio_scsi_handle_cmd_vq(s, vq);
     virtio_scsi_release(s);
@@ -855,12 +869,10 @@ static void virtio_scsi_handle_event(VirtIODevice *vdev, VirtQueue *vq)
 {
     VirtIOSCSI *s = VIRTIO_SCSI(vdev);
 
-    if (s->ctx) {
-        virtio_device_start_ioeventfd(vdev);
-        if (!s->dataplane_fenced) {
-            return;
-        }
+    if (virtio_scsi_defer_to_dataplane(s)) {
+        return;
     }
+
     virtio_scsi_acquire(s);
     virtio_scsi_handle_event_vq(s, vq);
     virtio_scsi_release(s);

From 38738f7dbbda90fbc161757b7f4be35b52205552 Mon Sep 17 00:00:00 2001
From: Stefan Hajnoczi <stefanha@redhat.com>
Date: Wed, 27 Apr 2022 15:35:37 +0100
Subject: [PATCH 190/935] virtio-scsi: don't waste CPU polling the event
 virtqueue

The virtio-scsi event virtqueue is not emptied by its handler function.
This is typical for rx virtqueues where the device uses buffers when
some event occurs (e.g. a packet is received, an error condition
happens, etc).

Polling non-empty virtqueues wastes CPU cycles. We are not waiting for
new buffers to become available, we are waiting for an event to occur,
so it's a misuse of CPU resources to poll for buffers.

Introduce the new virtio_queue_aio_attach_host_notifier_no_poll() API,
which is identical to virtio_queue_aio_attach_host_notifier() except
that it does not poll the virtqueue.

Before this patch the following command-line consumed 100% CPU in the
IOThread polling and calling virtio_scsi_handle_event():

  $ qemu-system-x86_64 -M accel=kvm -m 1G -cpu host \
      --object iothread,id=iothread0 \
      --device virtio-scsi-pci,iothread=iothread0 \
      --blockdev file,filename=test.img,aio=native,cache.direct=on,node-name=drive0 \
      --device scsi-hd,drive=drive0

After this patch CPU is no longer wasted.

Reported-by: Nir Soffer <nsoffer@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Tested-by: Nir Soffer <nsoffer@redhat.com>
Message-id: 20220427143541.119567-3-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/scsi/virtio-scsi-dataplane.c |  2 +-
 hw/virtio/virtio.c              | 13 +++++++++++++
 include/hw/virtio/virtio.h      |  1 +
 3 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/hw/scsi/virtio-scsi-dataplane.c b/hw/scsi/virtio-scsi-dataplane.c
index 29575cbaf6..8bb6e6acfc 100644
--- a/hw/scsi/virtio-scsi-dataplane.c
+++ b/hw/scsi/virtio-scsi-dataplane.c
@@ -138,7 +138,7 @@ int virtio_scsi_dataplane_start(VirtIODevice *vdev)
 
     aio_context_acquire(s->ctx);
     virtio_queue_aio_attach_host_notifier(vs->ctrl_vq, s->ctx);
-    virtio_queue_aio_attach_host_notifier(vs->event_vq, s->ctx);
+    virtio_queue_aio_attach_host_notifier_no_poll(vs->event_vq, s->ctx);
 
     for (i = 0; i < vs->conf.num_queues; i++) {
         virtio_queue_aio_attach_host_notifier(vs->cmd_vqs[i], s->ctx);
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 9d637e043e..67a873f54a 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -3534,6 +3534,19 @@ void virtio_queue_aio_attach_host_notifier(VirtQueue *vq, AioContext *ctx)
                                 virtio_queue_host_notifier_aio_poll_end);
 }
 
+/*
+ * Same as virtio_queue_aio_attach_host_notifier() but without polling. Use
+ * this for rx virtqueues and similar cases where the virtqueue handler
+ * function does not pop all elements. When the virtqueue is left non-empty
+ * polling consumes CPU cycles and should not be used.
+ */
+void virtio_queue_aio_attach_host_notifier_no_poll(VirtQueue *vq, AioContext *ctx)
+{
+    aio_set_event_notifier(ctx, &vq->host_notifier, true,
+                           virtio_queue_host_notifier_read,
+                           NULL, NULL);
+}
+
 void virtio_queue_aio_detach_host_notifier(VirtQueue *vq, AioContext *ctx)
 {
     aio_set_event_notifier(ctx, &vq->host_notifier, true, NULL, NULL, NULL);
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index b31c4507f5..b62a35fdca 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -317,6 +317,7 @@ EventNotifier *virtio_queue_get_host_notifier(VirtQueue *vq);
 void virtio_queue_set_host_notifier_enabled(VirtQueue *vq, bool enabled);
 void virtio_queue_host_notifier_read(EventNotifier *n);
 void virtio_queue_aio_attach_host_notifier(VirtQueue *vq, AioContext *ctx);
+void virtio_queue_aio_attach_host_notifier_no_poll(VirtQueue *vq, AioContext *ctx);
 void virtio_queue_aio_detach_host_notifier(VirtQueue *vq, AioContext *ctx);
 VirtQueue *virtio_vector_first_queue(VirtIODevice *vdev, uint16_t vector);
 VirtQueue *virtio_vector_next_queue(VirtQueue *vq);

From 37ce2de95169dacab3fb53d11bd4509b9c2e3a4c Mon Sep 17 00:00:00 2001
From: Stefan Hajnoczi <stefanha@redhat.com>
Date: Wed, 27 Apr 2022 15:35:38 +0100
Subject: [PATCH 191/935] virtio-scsi: clean up virtio_scsi_handle_event_vq()

virtio_scsi_handle_event_vq() is only called from hw/scsi/virtio-scsi.c
now and its return value is no longer used. Remove the function
prototype from virtio-scsi.h and drop the return value.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20220427143541.119567-4-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/scsi/virtio-scsi.c           | 4 +---
 include/hw/virtio/virtio-scsi.h | 1 -
 2 files changed, 1 insertion(+), 4 deletions(-)

diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
index 417fbc71d6..aa03a713d8 100644
--- a/hw/scsi/virtio-scsi.c
+++ b/hw/scsi/virtio-scsi.c
@@ -856,13 +856,11 @@ void virtio_scsi_push_event(VirtIOSCSI *s, SCSIDevice *dev,
     virtio_scsi_complete_req(req);
 }
 
-bool virtio_scsi_handle_event_vq(VirtIOSCSI *s, VirtQueue *vq)
+static void virtio_scsi_handle_event_vq(VirtIOSCSI *s, VirtQueue *vq)
 {
     if (s->events_dropped) {
         virtio_scsi_push_event(s, NULL, VIRTIO_SCSI_T_NO_EVENT, 0);
-        return true;
     }
-    return false;
 }
 
 static void virtio_scsi_handle_event(VirtIODevice *vdev, VirtQueue *vq)
diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scsi.h
index 0997313f0a..85e69d0368 100644
--- a/include/hw/virtio/virtio-scsi.h
+++ b/include/hw/virtio/virtio-scsi.h
@@ -149,7 +149,6 @@ void virtio_scsi_common_realize(DeviceState *dev,
                                 Error **errp);
 
 void virtio_scsi_common_unrealize(DeviceState *dev);
-bool virtio_scsi_handle_event_vq(VirtIOSCSI *s, VirtQueue *vq);
 bool virtio_scsi_handle_cmd_vq(VirtIOSCSI *s, VirtQueue *vq);
 bool virtio_scsi_handle_ctrl_vq(VirtIOSCSI *s, VirtQueue *vq);
 void virtio_scsi_init_req(VirtIOSCSI *s, VirtQueue *vq, VirtIOSCSIReq *req);

From 73b3b49f1880f236b4d0ffd7efb00280c05a5fab Mon Sep 17 00:00:00 2001
From: Stefan Hajnoczi <stefanha@redhat.com>
Date: Wed, 27 Apr 2022 15:35:39 +0100
Subject: [PATCH 192/935] virtio-scsi: clean up virtio_scsi_handle_ctrl_vq()

virtio_scsi_handle_ctrl_vq() is only called from hw/scsi/virtio-scsi.c
now and its return value is no longer used. Remove the function
prototype from virtio-scsi.h and drop the return value.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20220427143541.119567-5-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/scsi/virtio-scsi.c           | 5 +----
 include/hw/virtio/virtio-scsi.h | 1 -
 2 files changed, 1 insertion(+), 5 deletions(-)

diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
index aa03a713d8..eefda16e4b 100644
--- a/hw/scsi/virtio-scsi.c
+++ b/hw/scsi/virtio-scsi.c
@@ -460,16 +460,13 @@ static void virtio_scsi_handle_ctrl_req(VirtIOSCSI *s, VirtIOSCSIReq *req)
     }
 }
 
-bool virtio_scsi_handle_ctrl_vq(VirtIOSCSI *s, VirtQueue *vq)
+static void virtio_scsi_handle_ctrl_vq(VirtIOSCSI *s, VirtQueue *vq)
 {
     VirtIOSCSIReq *req;
-    bool progress = false;
 
     while ((req = virtio_scsi_pop_req(s, vq))) {
-        progress = true;
         virtio_scsi_handle_ctrl_req(s, req);
     }
-    return progress;
 }
 
 /*
diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scsi.h
index 85e69d0368..ac99800ee2 100644
--- a/include/hw/virtio/virtio-scsi.h
+++ b/include/hw/virtio/virtio-scsi.h
@@ -150,7 +150,6 @@ void virtio_scsi_common_realize(DeviceState *dev,
 
 void virtio_scsi_common_unrealize(DeviceState *dev);
 bool virtio_scsi_handle_cmd_vq(VirtIOSCSI *s, VirtQueue *vq);
-bool virtio_scsi_handle_ctrl_vq(VirtIOSCSI *s, VirtQueue *vq);
 void virtio_scsi_init_req(VirtIOSCSI *s, VirtQueue *vq, VirtIOSCSIReq *req);
 void virtio_scsi_free_req(VirtIOSCSIReq *req);
 void virtio_scsi_push_event(VirtIOSCSI *s, SCSIDevice *dev,

From ad482b57ef841b2d4883c5079d20ba44ff5e4b3e Mon Sep 17 00:00:00 2001
From: Stefan Hajnoczi <stefanha@redhat.com>
Date: Wed, 27 Apr 2022 15:35:40 +0100
Subject: [PATCH 193/935] virtio-scsi: clean up virtio_scsi_handle_cmd_vq()

virtio_scsi_handle_cmd_vq() is only called from hw/scsi/virtio-scsi.c
now and its return value is no longer used. Remove the function
prototype from virtio-scsi.h and drop the return value.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20220427143541.119567-6-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/scsi/virtio-scsi.c           | 5 +----
 include/hw/virtio/virtio-scsi.h | 1 -
 2 files changed, 1 insertion(+), 5 deletions(-)

diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
index eefda16e4b..12c6a21202 100644
--- a/hw/scsi/virtio-scsi.c
+++ b/hw/scsi/virtio-scsi.c
@@ -685,12 +685,11 @@ static void virtio_scsi_handle_cmd_req_submit(VirtIOSCSI *s, VirtIOSCSIReq *req)
     scsi_req_unref(sreq);
 }
 
-bool virtio_scsi_handle_cmd_vq(VirtIOSCSI *s, VirtQueue *vq)
+static void virtio_scsi_handle_cmd_vq(VirtIOSCSI *s, VirtQueue *vq)
 {
     VirtIOSCSIReq *req, *next;
     int ret = 0;
     bool suppress_notifications = virtio_queue_get_notification(vq);
-    bool progress = false;
 
     QTAILQ_HEAD(, VirtIOSCSIReq) reqs = QTAILQ_HEAD_INITIALIZER(reqs);
 
@@ -700,7 +699,6 @@ bool virtio_scsi_handle_cmd_vq(VirtIOSCSI *s, VirtQueue *vq)
         }
 
         while ((req = virtio_scsi_pop_req(s, vq))) {
-            progress = true;
             ret = virtio_scsi_handle_cmd_req_prepare(s, req);
             if (!ret) {
                 QTAILQ_INSERT_TAIL(&reqs, req, next);
@@ -725,7 +723,6 @@ bool virtio_scsi_handle_cmd_vq(VirtIOSCSI *s, VirtQueue *vq)
     QTAILQ_FOREACH_SAFE(req, &reqs, next, next) {
         virtio_scsi_handle_cmd_req_submit(s, req);
     }
-    return progress;
 }
 
 static void virtio_scsi_handle_cmd(VirtIODevice *vdev, VirtQueue *vq)
diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scsi.h
index ac99800ee2..2040352102 100644
--- a/include/hw/virtio/virtio-scsi.h
+++ b/include/hw/virtio/virtio-scsi.h
@@ -149,7 +149,6 @@ void virtio_scsi_common_realize(DeviceState *dev,
                                 Error **errp);
 
 void virtio_scsi_common_unrealize(DeviceState *dev);
-bool virtio_scsi_handle_cmd_vq(VirtIOSCSI *s, VirtQueue *vq);
 void virtio_scsi_init_req(VirtIOSCSI *s, VirtQueue *vq, VirtIOSCSIReq *req);
 void virtio_scsi_free_req(VirtIOSCSIReq *req);
 void virtio_scsi_push_event(VirtIOSCSI *s, SCSIDevice *dev,

From 3dc584abeef0e1277c2de8c1c1974cb49444eb0a Mon Sep 17 00:00:00 2001
From: Stefan Hajnoczi <stefanha@redhat.com>
Date: Wed, 27 Apr 2022 15:35:41 +0100
Subject: [PATCH 194/935] virtio-scsi: move request-related items from .h to .c

There is no longer a need to expose the request and related APIs in
virtio-scsi.h since there are no callers outside virtio-scsi.c.

Note the block comment in VirtIOSCSIReq has been adjusted to meet the
coding style.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20220427143541.119567-7-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/scsi/virtio-scsi.c           | 45 ++++++++++++++++++++++++++++++---
 include/hw/virtio/virtio-scsi.h | 40 -----------------------------
 2 files changed, 41 insertions(+), 44 deletions(-)

diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
index 12c6a21202..db54d104be 100644
--- a/hw/scsi/virtio-scsi.c
+++ b/hw/scsi/virtio-scsi.c
@@ -29,6 +29,43 @@
 #include "hw/virtio/virtio-access.h"
 #include "trace.h"
 
+typedef struct VirtIOSCSIReq {
+    /*
+     * Note:
+     * - fields up to resp_iov are initialized by virtio_scsi_init_req;
+     * - fields starting at vring are zeroed by virtio_scsi_init_req.
+     */
+    VirtQueueElement elem;
+
+    VirtIOSCSI *dev;
+    VirtQueue *vq;
+    QEMUSGList qsgl;
+    QEMUIOVector resp_iov;
+
+    union {
+        /* Used for two-stage request submission */
+        QTAILQ_ENTRY(VirtIOSCSIReq) next;
+
+        /* Used for cancellation of request during TMFs */
+        int remaining;
+    };
+
+    SCSIRequest *sreq;
+    size_t resp_size;
+    enum SCSIXferMode mode;
+    union {
+        VirtIOSCSICmdResp     cmd;
+        VirtIOSCSICtrlTMFResp tmf;
+        VirtIOSCSICtrlANResp  an;
+        VirtIOSCSIEvent       event;
+    } resp;
+    union {
+        VirtIOSCSICmdReq      cmd;
+        VirtIOSCSICtrlTMFReq  tmf;
+        VirtIOSCSICtrlANReq   an;
+    } req;
+} VirtIOSCSIReq;
+
 static inline int virtio_scsi_get_lun(uint8_t *lun)
 {
     return ((lun[2] << 8) | lun[3]) & 0x3FFF;
@@ -45,7 +82,7 @@ static inline SCSIDevice *virtio_scsi_device_get(VirtIOSCSI *s, uint8_t *lun)
     return scsi_device_get(&s->bus, 0, lun[1], virtio_scsi_get_lun(lun));
 }
 
-void virtio_scsi_init_req(VirtIOSCSI *s, VirtQueue *vq, VirtIOSCSIReq *req)
+static void virtio_scsi_init_req(VirtIOSCSI *s, VirtQueue *vq, VirtIOSCSIReq *req)
 {
     VirtIODevice *vdev = VIRTIO_DEVICE(s);
     const size_t zero_skip =
@@ -58,7 +95,7 @@ void virtio_scsi_init_req(VirtIOSCSI *s, VirtQueue *vq, VirtIOSCSIReq *req)
     memset((uint8_t *)req + zero_skip, 0, sizeof(*req) - zero_skip);
 }
 
-void virtio_scsi_free_req(VirtIOSCSIReq *req)
+static void virtio_scsi_free_req(VirtIOSCSIReq *req)
 {
     qemu_iovec_destroy(&req->resp_iov);
     qemu_sglist_destroy(&req->qsgl);
@@ -801,8 +838,8 @@ static void virtio_scsi_reset(VirtIODevice *vdev)
     s->events_dropped = false;
 }
 
-void virtio_scsi_push_event(VirtIOSCSI *s, SCSIDevice *dev,
-                            uint32_t event, uint32_t reason)
+static void virtio_scsi_push_event(VirtIOSCSI *s, SCSIDevice *dev,
+                                   uint32_t event, uint32_t reason)
 {
     VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(s);
     VirtIOSCSIReq *req;
diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scsi.h
index 2040352102..a36aad9c86 100644
--- a/include/hw/virtio/virtio-scsi.h
+++ b/include/hw/virtio/virtio-scsi.h
@@ -92,42 +92,6 @@ struct VirtIOSCSI {
     uint32_t host_features;
 };
 
-typedef struct VirtIOSCSIReq {
-    /* Note:
-     * - fields up to resp_iov are initialized by virtio_scsi_init_req;
-     * - fields starting at vring are zeroed by virtio_scsi_init_req.
-     * */
-    VirtQueueElement elem;
-
-    VirtIOSCSI *dev;
-    VirtQueue *vq;
-    QEMUSGList qsgl;
-    QEMUIOVector resp_iov;
-
-    union {
-        /* Used for two-stage request submission */
-        QTAILQ_ENTRY(VirtIOSCSIReq) next;
-
-        /* Used for cancellation of request during TMFs */
-        int remaining;
-    };
-
-    SCSIRequest *sreq;
-    size_t resp_size;
-    enum SCSIXferMode mode;
-    union {
-        VirtIOSCSICmdResp     cmd;
-        VirtIOSCSICtrlTMFResp tmf;
-        VirtIOSCSICtrlANResp  an;
-        VirtIOSCSIEvent       event;
-    } resp;
-    union {
-        VirtIOSCSICmdReq      cmd;
-        VirtIOSCSICtrlTMFReq  tmf;
-        VirtIOSCSICtrlANReq   an;
-    } req;
-} VirtIOSCSIReq;
-
 static inline void virtio_scsi_acquire(VirtIOSCSI *s)
 {
     if (s->ctx) {
@@ -149,10 +113,6 @@ void virtio_scsi_common_realize(DeviceState *dev,
                                 Error **errp);
 
 void virtio_scsi_common_unrealize(DeviceState *dev);
-void virtio_scsi_init_req(VirtIOSCSI *s, VirtQueue *vq, VirtIOSCSIReq *req);
-void virtio_scsi_free_req(VirtIOSCSIReq *req);
-void virtio_scsi_push_event(VirtIOSCSI *s, SCSIDevice *dev,
-                            uint32_t event, uint32_t reason);
 
 void virtio_scsi_dataplane_setup(VirtIOSCSI *s, Error **errp);
 int virtio_scsi_dataplane_start(VirtIODevice *s);

From 14d5addcaedae2e8666aeda71510e1d4be5bb50d Mon Sep 17 00:00:00 2001
From: Leif Lindholm <quic_llindhol@quicinc.com>
Date: Thu, 5 May 2022 12:37:40 +0100
Subject: [PATCH 195/935] MAINTAINERS/.mailmap: update email for Leif Lindholm
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

NUVIA was acquired by Qualcomm in March 2021, but kept functioning on
separate infrastructure for a transitional period. We've now switched
over to contributing as Qualcomm Innovation Center (quicinc), so update
my email address to reflect this.

Signed-off-by: Leif Lindholm <quic_llindhol@quicinc.com>
Message-id: 20220505113740.75565-1-quic_llindhol@quicinc.com
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[Fixed commit message typo]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 .mailmap    | 3 ++-
 MAINTAINERS | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/.mailmap b/.mailmap
index 2976a675ea..8c326709cf 100644
--- a/.mailmap
+++ b/.mailmap
@@ -62,7 +62,8 @@ Greg Kurz <groug@kaod.org> <gkurz@linux.vnet.ibm.com>
 Huacai Chen <chenhuacai@kernel.org> <chenhc@lemote.com>
 Huacai Chen <chenhuacai@kernel.org> <chenhuacai@loongson.cn>
 James Hogan <jhogan@kernel.org> <james.hogan@imgtec.com>
-Leif Lindholm <leif@nuviainc.com> <leif.lindholm@linaro.org>
+Leif Lindholm <quic_llindhol@quicinc.com> <leif.lindholm@linaro.org>
+Leif Lindholm <quic_llindhol@quicinc.com> <leif@nuviainc.com>
 Radoslaw Biernacki <rad@semihalf.com> <radoslaw.biernacki@linaro.org>
 Paul Burton <paulburton@kernel.org> <paul.burton@mips.com>
 Paul Burton <paulburton@kernel.org> <paul.burton@imgtec.com>
diff --git a/MAINTAINERS b/MAINTAINERS
index 662ec47246..42f67e2b93 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -889,7 +889,7 @@ F: include/hw/ssi/imx_spi.h
 SBSA-REF
 M: Radoslaw Biernacki <rad@semihalf.com>
 M: Peter Maydell <peter.maydell@linaro.org>
-R: Leif Lindholm <leif@nuviainc.com>
+R: Leif Lindholm <quic_llindhol@quicinc.com>
 L: qemu-arm@nongnu.org
 S: Maintained
 F: hw/arm/sbsa-ref.c

From 696ba3771894f7a0b233e634dc9d401330568e35 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:19 -0500
Subject: [PATCH 196/935] target/arm: Handle cpreg registration for missing EL

More gracefully handle cpregs when EL2 and/or EL3 are missing.
If the reg is entirely inaccessible, do not register it at all.
If the reg is for EL2, and EL3 is present but EL2 is not,
either discard, squash to res0, const, or keep unchanged.

Per rule RJFFP, mark the 4 aarch32 hypervisor access registers
with ARM_CP_EL3_NO_EL2_KEEP, and mark all of the EL2 address
translation and tlb invalidation "regs" ARM_CP_EL3_NO_EL2_UNDEF.
Mark the 2 virtualization processor id regs ARM_CP_EL3_NO_EL2_C_NZ.

This will simplify cpreg registration for conditional arm features.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-2-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpregs.h |  11 +++
 target/arm/helper.c | 178 ++++++++++++++++++++++++++++++--------------
 2 files changed, 133 insertions(+), 56 deletions(-)

diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h
index 73984549d2..db03d6a7e1 100644
--- a/target/arm/cpregs.h
+++ b/target/arm/cpregs.h
@@ -102,6 +102,17 @@ enum {
     ARM_CP_SVE                   = 1 << 14,
     /* Flag: Do not expose in gdb sysreg xml. */
     ARM_CP_NO_GDB                = 1 << 15,
+    /*
+     * Flags: If EL3 but not EL2...
+     *   - UNDEF: discard the cpreg,
+     *   -  KEEP: retain the cpreg as is,
+     *   -  C_NZ: set const on the cpreg, but retain resetvalue,
+     *   -  else: set const on the cpreg, zero resetvalue, aka RES0.
+     * See rule RJFFP in section D1.1.3 of DDI0487H.a.
+     */
+    ARM_CP_EL3_NO_EL2_UNDEF      = 1 << 16,
+    ARM_CP_EL3_NO_EL2_KEEP       = 1 << 17,
+    ARM_CP_EL3_NO_EL2_C_NZ       = 1 << 18,
 };
 
 /*
diff --git a/target/arm/helper.c b/target/arm/helper.c
index b4daf4f076..9ab8b65e7b 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -5056,16 +5056,17 @@ static const ARMCPRegInfo v8_cp_reginfo[] = {
       .access = PL1_RW, .readfn = spsel_read, .writefn = spsel_write },
     { .name = "FPEXC32_EL2", .state = ARM_CP_STATE_AA64,
       .opc0 = 3, .opc1 = 4, .crn = 5, .crm = 3, .opc2 = 0,
-      .access = PL2_RW, .type = ARM_CP_ALIAS | ARM_CP_FPU,
+      .access = PL2_RW,
+      .type = ARM_CP_ALIAS | ARM_CP_FPU | ARM_CP_EL3_NO_EL2_KEEP,
       .fieldoffset = offsetof(CPUARMState, vfp.xregs[ARM_VFP_FPEXC]) },
     { .name = "DACR32_EL2", .state = ARM_CP_STATE_AA64,
       .opc0 = 3, .opc1 = 4, .crn = 3, .crm = 0, .opc2 = 0,
-      .access = PL2_RW, .resetvalue = 0,
+      .access = PL2_RW, .resetvalue = 0, .type = ARM_CP_EL3_NO_EL2_KEEP,
       .writefn = dacr_write, .raw_writefn = raw_write,
       .fieldoffset = offsetof(CPUARMState, cp15.dacr32_el2) },
     { .name = "IFSR32_EL2", .state = ARM_CP_STATE_AA64,
       .opc0 = 3, .opc1 = 4, .crn = 5, .crm = 0, .opc2 = 1,
-      .access = PL2_RW, .resetvalue = 0,
+      .access = PL2_RW, .resetvalue = 0, .type = ARM_CP_EL3_NO_EL2_KEEP,
       .fieldoffset = offsetof(CPUARMState, cp15.ifsr32_el2) },
     { .name = "SPSR_IRQ", .state = ARM_CP_STATE_AA64,
       .type = ARM_CP_ALIAS,
@@ -5542,27 +5543,27 @@ static const ARMCPRegInfo el2_cp_reginfo[] = {
       .writefn = tlbimva_hyp_is_write },
     { .name = "TLBI_ALLE2", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 0,
-      .type = ARM_CP_NO_RAW, .access = PL2_W,
+      .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF,
       .writefn = tlbi_aa64_alle2_write },
     { .name = "TLBI_VAE2", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 1,
-      .type = ARM_CP_NO_RAW, .access = PL2_W,
+      .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF,
       .writefn = tlbi_aa64_vae2_write },
     { .name = "TLBI_VALE2", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 5,
-      .access = PL2_W, .type = ARM_CP_NO_RAW,
+      .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF,
       .writefn = tlbi_aa64_vae2_write },
     { .name = "TLBI_ALLE2IS", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 0,
-      .access = PL2_W, .type = ARM_CP_NO_RAW,
+      .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF,
       .writefn = tlbi_aa64_alle2is_write },
     { .name = "TLBI_VAE2IS", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 1,
-      .type = ARM_CP_NO_RAW, .access = PL2_W,
+      .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF,
       .writefn = tlbi_aa64_vae2is_write },
     { .name = "TLBI_VALE2IS", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 5,
-      .access = PL2_W, .type = ARM_CP_NO_RAW,
+      .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF,
       .writefn = tlbi_aa64_vae2is_write },
 #ifndef CONFIG_USER_ONLY
     /* Unlike the other EL2-related AT operations, these must
@@ -5572,11 +5573,13 @@ static const ARMCPRegInfo el2_cp_reginfo[] = {
     { .name = "AT_S1E2R", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 7, .crm = 8, .opc2 = 0,
       .access = PL2_W, .accessfn = at_s1e2_access,
-      .type = ARM_CP_NO_RAW | ARM_CP_RAISES_EXC, .writefn = ats_write64 },
+      .type = ARM_CP_NO_RAW | ARM_CP_RAISES_EXC | ARM_CP_EL3_NO_EL2_UNDEF,
+      .writefn = ats_write64 },
     { .name = "AT_S1E2W", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 7, .crm = 8, .opc2 = 1,
       .access = PL2_W, .accessfn = at_s1e2_access,
-      .type = ARM_CP_NO_RAW | ARM_CP_RAISES_EXC, .writefn = ats_write64 },
+      .type = ARM_CP_NO_RAW | ARM_CP_RAISES_EXC | ARM_CP_EL3_NO_EL2_UNDEF,
+      .writefn = ats_write64 },
     /* The AArch32 ATS1H* operations are CONSTRAINED UNPREDICTABLE
      * if EL2 is not implemented; we choose to UNDEF. Behaviour at EL3
      * with SCR.NS == 0 outside Monitor mode is UNPREDICTABLE; we choose
@@ -6076,7 +6079,7 @@ static const ARMCPRegInfo debug_cp_reginfo[] = {
     { .name = "DBGVCR32_EL2", .state = ARM_CP_STATE_AA64,
       .opc0 = 2, .opc1 = 4, .crn = 0, .crm = 7, .opc2 = 0,
       .access = PL2_RW, .accessfn = access_tda,
-      .type = ARM_CP_NOP },
+      .type = ARM_CP_NOP | ARM_CP_EL3_NO_EL2_KEEP },
     /* Dummy MDCCINT_EL1, since we don't implement the Debug Communications
      * Channel but Linux may try to access this register. The 32-bit
      * alias is DBGDCCINT.
@@ -6892,11 +6895,11 @@ static const ARMCPRegInfo tlbirange_reginfo[] = {
       .access = PL2_W, .type = ARM_CP_NOP },
     { .name = "TLBI_RVAE2IS", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 2, .opc2 = 1,
-      .access = PL2_W, .type = ARM_CP_NO_RAW,
+      .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF,
       .writefn = tlbi_aa64_rvae2is_write },
    { .name = "TLBI_RVALE2IS", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 2, .opc2 = 5,
-      .access = PL2_W, .type = ARM_CP_NO_RAW,
+      .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF,
       .writefn = tlbi_aa64_rvae2is_write },
     { .name = "TLBI_RIPAS2E1", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 2,
@@ -6906,19 +6909,19 @@ static const ARMCPRegInfo tlbirange_reginfo[] = {
       .access = PL2_W, .type = ARM_CP_NOP },
    { .name = "TLBI_RVAE2OS", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 5, .opc2 = 1,
-      .access = PL2_W, .type = ARM_CP_NO_RAW,
+      .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF,
       .writefn = tlbi_aa64_rvae2is_write },
    { .name = "TLBI_RVALE2OS", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 5, .opc2 = 5,
-      .access = PL2_W, .type = ARM_CP_NO_RAW,
+      .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF,
       .writefn = tlbi_aa64_rvae2is_write },
     { .name = "TLBI_RVAE2", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 6, .opc2 = 1,
-      .access = PL2_W, .type = ARM_CP_NO_RAW,
+      .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF,
       .writefn = tlbi_aa64_rvae2_write },
    { .name = "TLBI_RVALE2", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 6, .opc2 = 5,
-      .access = PL2_W, .type = ARM_CP_NO_RAW,
+      .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF,
       .writefn = tlbi_aa64_rvae2_write },
    { .name = "TLBI_RVAE3IS", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 2, .opc2 = 1,
@@ -6973,11 +6976,11 @@ static const ARMCPRegInfo tlbios_reginfo[] = {
       .writefn = tlbi_aa64_vae1is_write },
     { .name = "TLBI_ALLE2OS", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 0,
-      .access = PL2_W, .type = ARM_CP_NO_RAW,
+      .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF,
       .writefn = tlbi_aa64_alle2is_write },
     { .name = "TLBI_VAE2OS", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 1,
-      .access = PL2_W, .type = ARM_CP_NO_RAW,
+      .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF,
       .writefn = tlbi_aa64_vae2is_write },
    { .name = "TLBI_ALLE1OS", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 4,
@@ -6985,7 +6988,7 @@ static const ARMCPRegInfo tlbios_reginfo[] = {
       .writefn = tlbi_aa64_alle1is_write },
     { .name = "TLBI_VALE2OS", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 5,
-      .access = PL2_W, .type = ARM_CP_NO_RAW,
+      .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EL3_NO_EL2_UNDEF,
       .writefn = tlbi_aa64_vae2is_write },
     { .name = "TLBI_VMALLS12E1OS", .state = ARM_CP_STATE_AA64,
       .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 1, .opc2 = 6,
@@ -7905,21 +7908,24 @@ void register_cp_regs_for_features(ARMCPU *cpu)
             { .name = "VPIDR", .state = ARM_CP_STATE_AA32,
               .cp = 15, .opc1 = 4, .crn = 0, .crm = 0, .opc2 = 0,
               .access = PL2_RW, .accessfn = access_el3_aa32ns,
-              .resetvalue = cpu->midr, .type = ARM_CP_ALIAS,
+              .resetvalue = cpu->midr,
+              .type = ARM_CP_ALIAS | ARM_CP_EL3_NO_EL2_C_NZ,
               .fieldoffset = offsetoflow32(CPUARMState, cp15.vpidr_el2) },
             { .name = "VPIDR_EL2", .state = ARM_CP_STATE_AA64,
               .opc0 = 3, .opc1 = 4, .crn = 0, .crm = 0, .opc2 = 0,
               .access = PL2_RW, .resetvalue = cpu->midr,
+              .type = ARM_CP_EL3_NO_EL2_C_NZ,
               .fieldoffset = offsetof(CPUARMState, cp15.vpidr_el2) },
             { .name = "VMPIDR", .state = ARM_CP_STATE_AA32,
               .cp = 15, .opc1 = 4, .crn = 0, .crm = 0, .opc2 = 5,
               .access = PL2_RW, .accessfn = access_el3_aa32ns,
-              .resetvalue = vmpidr_def, .type = ARM_CP_ALIAS,
+              .resetvalue = vmpidr_def,
+              .type = ARM_CP_ALIAS | ARM_CP_EL3_NO_EL2_C_NZ,
               .fieldoffset = offsetoflow32(CPUARMState, cp15.vmpidr_el2) },
             { .name = "VMPIDR_EL2", .state = ARM_CP_STATE_AA64,
               .opc0 = 3, .opc1 = 4, .crn = 0, .crm = 0, .opc2 = 5,
-              .access = PL2_RW,
-              .resetvalue = vmpidr_def,
+              .access = PL2_RW, .resetvalue = vmpidr_def,
+              .type = ARM_CP_EL3_NO_EL2_C_NZ,
               .fieldoffset = offsetof(CPUARMState, cp15.vmpidr_el2) },
         };
         define_arm_cp_regs(cpu, vpidr_regs);
@@ -8506,13 +8512,14 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
                                    int crm, int opc1, int opc2,
                                    const char *name)
 {
+    CPUARMState *env = &cpu->env;
     uint32_t key;
     ARMCPRegInfo *r2;
     bool is64 = r->type & ARM_CP_64BIT;
     bool ns = secstate & ARM_CP_SECSTATE_NS;
     int cp = r->cp;
-    bool isbanked;
     size_t name_len;
+    bool make_const;
 
     switch (state) {
     case ARM_CP_STATE_AA32:
@@ -8547,6 +8554,32 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
         }
     }
 
+    /*
+     * Eliminate registers that are not present because the EL is missing.
+     * Doing this here makes it easier to put all registers for a given
+     * feature into the same ARMCPRegInfo array and define them all at once.
+     */
+    make_const = false;
+    if (arm_feature(env, ARM_FEATURE_EL3)) {
+        /*
+         * An EL2 register without EL2 but with EL3 is (usually) RES0.
+         * See rule RJFFP in section D1.1.3 of DDI0487H.a.
+         */
+        int min_el = ctz32(r->access) / 2;
+        if (min_el == 2 && !arm_feature(env, ARM_FEATURE_EL2)) {
+            if (r->type & ARM_CP_EL3_NO_EL2_UNDEF) {
+                return;
+            }
+            make_const = !(r->type & ARM_CP_EL3_NO_EL2_KEEP);
+        }
+    } else {
+        CPAccessRights max_el = (arm_feature(env, ARM_FEATURE_EL2)
+                                 ? PL2_RW : PL1_RW);
+        if ((r->access & max_el) == 0) {
+            return;
+        }
+    }
+
     /* Combine cpreg and name into one allocation. */
     name_len = strlen(name) + 1;
     r2 = g_malloc(sizeof(*r2) + name_len);
@@ -8567,44 +8600,77 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
         r2->opaque = opaque;
     }
 
-    isbanked = r->bank_fieldoffsets[0] && r->bank_fieldoffsets[1];
-    if (isbanked) {
+    if (make_const) {
+        /* This should not have been a very special register to begin. */
+        int old_special = r2->type & ARM_CP_SPECIAL_MASK;
+        assert(old_special == 0 || old_special == ARM_CP_NOP);
         /*
-         * Register is banked (using both entries in array).
-         * Overwriting fieldoffset as the array is only used to define
-         * banked registers but later only fieldoffset is used.
+         * Set the special function to CONST, retaining the other flags.
+         * This is important for e.g. ARM_CP_SVE so that we still
+         * take the SVE trap if CPTR_EL3.EZ == 0.
          */
-        r2->fieldoffset = r->bank_fieldoffsets[ns];
-    }
+        r2->type = (r2->type & ~ARM_CP_SPECIAL_MASK) | ARM_CP_CONST;
+        /*
+         * Usually, these registers become RES0, but there are a few
+         * special cases like VPIDR_EL2 which have a constant non-zero
+         * value with writes ignored.
+         */
+        if (!(r->type & ARM_CP_EL3_NO_EL2_C_NZ)) {
+            r2->resetvalue = 0;
+        }
+        /*
+         * ARM_CP_CONST has precedence, so removing the callbacks and
+         * offsets are not strictly necessary, but it is potentially
+         * less confusing to debug later.
+         */
+        r2->readfn = NULL;
+        r2->writefn = NULL;
+        r2->raw_readfn = NULL;
+        r2->raw_writefn = NULL;
+        r2->resetfn = NULL;
+        r2->fieldoffset = 0;
+        r2->bank_fieldoffsets[0] = 0;
+        r2->bank_fieldoffsets[1] = 0;
+    } else {
+        bool isbanked = r->bank_fieldoffsets[0] && r->bank_fieldoffsets[1];
 
-    if (state == ARM_CP_STATE_AA32) {
         if (isbanked) {
             /*
-             * If the register is banked then we don't need to migrate or
-             * reset the 32-bit instance in certain cases:
-             *
-             * 1) If the register has both 32-bit and 64-bit instances then we
-             *    can count on the 64-bit instance taking care of the
-             *    non-secure bank.
-             * 2) If ARMv8 is enabled then we can count on a 64-bit version
-             *    taking care of the secure bank.  This requires that separate
-             *    32 and 64-bit definitions are provided.
+             * Register is banked (using both entries in array).
+             * Overwriting fieldoffset as the array is only used to define
+             * banked registers but later only fieldoffset is used.
              */
-            if ((r->state == ARM_CP_STATE_BOTH && ns) ||
-                (arm_feature(&cpu->env, ARM_FEATURE_V8) && !ns)) {
+            r2->fieldoffset = r->bank_fieldoffsets[ns];
+        }
+        if (state == ARM_CP_STATE_AA32) {
+            if (isbanked) {
+                /*
+                 * If the register is banked then we don't need to migrate or
+                 * reset the 32-bit instance in certain cases:
+                 *
+                 * 1) If the register has both 32-bit and 64-bit instances
+                 *    then we can count on the 64-bit instance taking care
+                 *    of the non-secure bank.
+                 * 2) If ARMv8 is enabled then we can count on a 64-bit
+                 *    version taking care of the secure bank.  This requires
+                 *    that separate 32 and 64-bit definitions are provided.
+                 */
+                if ((r->state == ARM_CP_STATE_BOTH && ns) ||
+                    (arm_feature(env, ARM_FEATURE_V8) && !ns)) {
+                    r2->type |= ARM_CP_ALIAS;
+                }
+            } else if ((secstate != r->secure) && !ns) {
+                /*
+                 * The register is not banked so we only want to allow
+                 * migration of the non-secure instance.
+                 */
                 r2->type |= ARM_CP_ALIAS;
             }
-        } else if ((secstate != r->secure) && !ns) {
-            /*
-             * The register is not banked so we only want to allow migration
-             * of the non-secure instance.
-             */
-            r2->type |= ARM_CP_ALIAS;
-        }
 
-        if (HOST_BIG_ENDIAN &&
-            r->state == ARM_CP_STATE_BOTH && r2->fieldoffset) {
-            r2->fieldoffset += sizeof(uint32_t);
+            if (HOST_BIG_ENDIAN &&
+                r->state == ARM_CP_STATE_BOTH && r2->fieldoffset) {
+                r2->fieldoffset += sizeof(uint32_t);
+            }
         }
     }
 
@@ -8615,7 +8681,7 @@ static void add_cpreg_to_hashtable(ARMCPU *cpu, const ARMCPRegInfo *r,
      * multiple times. Special registers (ie NOP/WFI) are
      * never migratable and not even raw-accessible.
      */
-    if (r->type & ARM_CP_SPECIAL_MASK) {
+    if (r2->type & ARM_CP_SPECIAL_MASK) {
         r2->type |= ARM_CP_NO_RAW;
     }
     if (((r->crm == CP_ANY) && crm != 0) ||

From 99a90811d09e58630c1c43c43b644501ce12ced0 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:20 -0500
Subject: [PATCH 197/935] target/arm: Drop EL3 no EL2 fallbacks

Drop el3_no_el2_cp_reginfo, el3_no_el2_v8_cp_reginfo, and the local
vpidr_regs definition, and rely on the squashing to ARM_CP_CONST
while registering for v8.

This is a behavior change for v7 cpus with Security Extensions and
without Virtualization Extensions, in that the virtualization cpregs
are now correctly not present.  This would be a migration compatibility
break, except that we have an existing bug in which migration of 32-bit
cpus with Security Extensions enabled does not work.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-3-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 158 ++++----------------------------------------
 1 file changed, 13 insertions(+), 145 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 9ab8b65e7b..ea2788b3d5 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -5099,124 +5099,6 @@ static const ARMCPRegInfo v8_cp_reginfo[] = {
       .fieldoffset = offsetoflow32(CPUARMState, cp15.mdcr_el3) },
 };
 
-/* Used to describe the behaviour of EL2 regs when EL2 does not exist.  */
-static const ARMCPRegInfo el3_no_el2_cp_reginfo[] = {
-    { .name = "VBAR_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 12, .crm = 0, .opc2 = 0,
-      .access = PL2_RW,
-      .readfn = arm_cp_read_zero, .writefn = arm_cp_write_ignore },
-    { .name = "HCR_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 1, .crm = 1, .opc2 = 0,
-      .access = PL2_RW,
-      .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "HACR_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 1, .crm = 1, .opc2 = 7,
-      .access = PL2_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "ESR_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 5, .crm = 2, .opc2 = 0,
-      .access = PL2_RW,
-      .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "CPTR_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 1, .crm = 1, .opc2 = 2,
-      .access = PL2_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "MAIR_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 10, .crm = 2, .opc2 = 0,
-      .access = PL2_RW, .type = ARM_CP_CONST,
-      .resetvalue = 0 },
-    { .name = "HMAIR1", .state = ARM_CP_STATE_AA32,
-      .cp = 15, .opc1 = 4, .crn = 10, .crm = 2, .opc2 = 1,
-      .access = PL2_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "AMAIR_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 10, .crm = 3, .opc2 = 0,
-      .access = PL2_RW, .type = ARM_CP_CONST,
-      .resetvalue = 0 },
-    { .name = "HAMAIR1", .state = ARM_CP_STATE_AA32,
-      .cp = 15, .opc1 = 4, .crn = 10, .crm = 3, .opc2 = 1,
-      .access = PL2_RW, .type = ARM_CP_CONST,
-      .resetvalue = 0 },
-    { .name = "AFSR0_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 5, .crm = 1, .opc2 = 0,
-      .access = PL2_RW, .type = ARM_CP_CONST,
-      .resetvalue = 0 },
-    { .name = "AFSR1_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 5, .crm = 1, .opc2 = 1,
-      .access = PL2_RW, .type = ARM_CP_CONST,
-      .resetvalue = 0 },
-    { .name = "TCR_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 2, .crm = 0, .opc2 = 2,
-      .access = PL2_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "VTCR_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 2, .crm = 1, .opc2 = 2,
-      .access = PL2_RW, .accessfn = access_el3_aa32ns,
-      .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "VTTBR", .state = ARM_CP_STATE_AA32,
-      .cp = 15, .opc1 = 6, .crm = 2,
-      .access = PL2_RW, .accessfn = access_el3_aa32ns,
-      .type = ARM_CP_CONST | ARM_CP_64BIT, .resetvalue = 0 },
-    { .name = "VTTBR_EL2", .state = ARM_CP_STATE_AA64,
-      .opc0 = 3, .opc1 = 4, .crn = 2, .crm = 1, .opc2 = 0,
-      .access = PL2_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "SCTLR_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 1, .crm = 0, .opc2 = 0,
-      .access = PL2_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "TPIDR_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 13, .crm = 0, .opc2 = 2,
-      .access = PL2_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "TTBR0_EL2", .state = ARM_CP_STATE_AA64,
-      .opc0 = 3, .opc1 = 4, .crn = 2, .crm = 0, .opc2 = 0,
-      .access = PL2_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "HTTBR", .cp = 15, .opc1 = 4, .crm = 2,
-      .access = PL2_RW, .type = ARM_CP_64BIT | ARM_CP_CONST,
-      .resetvalue = 0 },
-    { .name = "CNTHCTL_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 14, .crm = 1, .opc2 = 0,
-      .access = PL2_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "CNTVOFF_EL2", .state = ARM_CP_STATE_AA64,
-      .opc0 = 3, .opc1 = 4, .crn = 14, .crm = 0, .opc2 = 3,
-      .access = PL2_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "CNTVOFF", .cp = 15, .opc1 = 4, .crm = 14,
-      .access = PL2_RW, .type = ARM_CP_64BIT | ARM_CP_CONST,
-      .resetvalue = 0 },
-    { .name = "CNTHP_CVAL_EL2", .state = ARM_CP_STATE_AA64,
-      .opc0 = 3, .opc1 = 4, .crn = 14, .crm = 2, .opc2 = 2,
-      .access = PL2_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "CNTHP_CVAL", .cp = 15, .opc1 = 6, .crm = 14,
-      .access = PL2_RW, .type = ARM_CP_64BIT | ARM_CP_CONST,
-      .resetvalue = 0 },
-    { .name = "CNTHP_TVAL_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 14, .crm = 2, .opc2 = 0,
-      .access = PL2_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "CNTHP_CTL_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 14, .crm = 2, .opc2 = 1,
-      .access = PL2_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "MDCR_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 1, .crm = 1, .opc2 = 1,
-      .access = PL2_RW, .accessfn = access_tda,
-      .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "HPFAR_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 6, .crm = 0, .opc2 = 4,
-      .access = PL2_RW, .accessfn = access_el3_aa32ns,
-      .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "HSTR_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 1, .crm = 1, .opc2 = 3,
-      .access = PL2_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "FAR_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 6, .crm = 0, .opc2 = 0,
-      .access = PL2_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "HIFAR", .state = ARM_CP_STATE_AA32,
-      .type = ARM_CP_CONST,
-      .cp = 15, .opc1 = 4, .crn = 6, .crm = 0, .opc2 = 2,
-      .access = PL2_RW, .resetvalue = 0 },
-};
-
-/* Ditto, but for registers which exist in ARMv8 but not v7 */
-static const ARMCPRegInfo el3_no_el2_v8_cp_reginfo[] = {
-    { .name = "HCR2", .state = ARM_CP_STATE_AA32,
-      .cp = 15, .opc1 = 4, .crn = 1, .crm = 1, .opc2 = 4,
-      .access = PL2_RW,
-      .type = ARM_CP_CONST, .resetvalue = 0 },
-};
-
 static void do_hcr_write(CPUARMState *env, uint64_t value, uint64_t valid_mask)
 {
     ARMCPU *cpu = env_archcpu(env);
@@ -7902,7 +7784,17 @@ void register_cp_regs_for_features(ARMCPU *cpu)
         define_arm_cp_regs(cpu, v8_idregs);
         define_arm_cp_regs(cpu, v8_cp_reginfo);
     }
-    if (arm_feature(env, ARM_FEATURE_EL2)) {
+
+    /*
+     * Register the base EL2 cpregs.
+     * Pre v8, these registers are implemented only as part of the
+     * Virtualization Extensions (EL2 present).  Beginning with v8,
+     * if EL2 is missing but EL3 is enabled, mostly these become
+     * RES0 from EL3, with some specific exceptions.
+     */
+    if (arm_feature(env, ARM_FEATURE_EL2)
+        || (arm_feature(env, ARM_FEATURE_EL3)
+            && arm_feature(env, ARM_FEATURE_V8))) {
         uint64_t vmpidr_def = mpidr_read_val(env);
         ARMCPRegInfo vpidr_regs[] = {
             { .name = "VPIDR", .state = ARM_CP_STATE_AA32,
@@ -7946,33 +7838,9 @@ void register_cp_regs_for_features(ARMCPU *cpu)
             };
             define_one_arm_cp_reg(cpu, &rvbar);
         }
-    } else {
-        /* If EL2 is missing but higher ELs are enabled, we need to
-         * register the no_el2 reginfos.
-         */
-        if (arm_feature(env, ARM_FEATURE_EL3)) {
-            /* When EL3 exists but not EL2, VPIDR and VMPIDR take the value
-             * of MIDR_EL1 and MPIDR_EL1.
-             */
-            ARMCPRegInfo vpidr_regs[] = {
-                { .name = "VPIDR_EL2", .state = ARM_CP_STATE_BOTH,
-                  .opc0 = 3, .opc1 = 4, .crn = 0, .crm = 0, .opc2 = 0,
-                  .access = PL2_RW, .accessfn = access_el3_aa32ns,
-                  .type = ARM_CP_CONST, .resetvalue = cpu->midr,
-                  .fieldoffset = offsetof(CPUARMState, cp15.vpidr_el2) },
-                { .name = "VMPIDR_EL2", .state = ARM_CP_STATE_BOTH,
-                  .opc0 = 3, .opc1 = 4, .crn = 0, .crm = 0, .opc2 = 5,
-                  .access = PL2_RW, .accessfn = access_el3_aa32ns,
-                  .type = ARM_CP_NO_RAW,
-                  .writefn = arm_cp_write_ignore, .readfn = mpidr_read },
-            };
-            define_arm_cp_regs(cpu, vpidr_regs);
-            define_arm_cp_regs(cpu, el3_no_el2_cp_reginfo);
-            if (arm_feature(env, ARM_FEATURE_V8)) {
-                define_arm_cp_regs(cpu, el3_no_el2_v8_cp_reginfo);
-            }
-        }
     }
+
+    /* Register the base EL3 cpregs. */
     if (arm_feature(env, ARM_FEATURE_EL3)) {
         define_arm_cp_regs(cpu, el3_cp_reginfo);
         ARMCPRegInfo el3_regs[] = {

From 60360d82a1022f8db9cbe5ed312de14b490fa46e Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:21 -0500
Subject: [PATCH 198/935] target/arm: Merge zcr reginfo

Drop zcr_no_el2_reginfo and merge the 3 registers into one array,
now that ZCR_EL2 can be squashed to RES0 and ZCR_EL3 dropped
while registering.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-4-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 55 ++++++++++++++-------------------------------
 1 file changed, 17 insertions(+), 38 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index ea2788b3d5..72d05070f0 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -6122,35 +6122,22 @@ static void zcr_write(CPUARMState *env, const ARMCPRegInfo *ri,
     }
 }
 
-static const ARMCPRegInfo zcr_el1_reginfo = {
-    .name = "ZCR_EL1", .state = ARM_CP_STATE_AA64,
-    .opc0 = 3, .opc1 = 0, .crn = 1, .crm = 2, .opc2 = 0,
-    .access = PL1_RW, .type = ARM_CP_SVE,
-    .fieldoffset = offsetof(CPUARMState, vfp.zcr_el[1]),
-    .writefn = zcr_write, .raw_writefn = raw_write
-};
-
-static const ARMCPRegInfo zcr_el2_reginfo = {
-    .name = "ZCR_EL2", .state = ARM_CP_STATE_AA64,
-    .opc0 = 3, .opc1 = 4, .crn = 1, .crm = 2, .opc2 = 0,
-    .access = PL2_RW, .type = ARM_CP_SVE,
-    .fieldoffset = offsetof(CPUARMState, vfp.zcr_el[2]),
-    .writefn = zcr_write, .raw_writefn = raw_write
-};
-
-static const ARMCPRegInfo zcr_no_el2_reginfo = {
-    .name = "ZCR_EL2", .state = ARM_CP_STATE_AA64,
-    .opc0 = 3, .opc1 = 4, .crn = 1, .crm = 2, .opc2 = 0,
-    .access = PL2_RW, .type = ARM_CP_SVE,
-    .readfn = arm_cp_read_zero, .writefn = arm_cp_write_ignore
-};
-
-static const ARMCPRegInfo zcr_el3_reginfo = {
-    .name = "ZCR_EL3", .state = ARM_CP_STATE_AA64,
-    .opc0 = 3, .opc1 = 6, .crn = 1, .crm = 2, .opc2 = 0,
-    .access = PL3_RW, .type = ARM_CP_SVE,
-    .fieldoffset = offsetof(CPUARMState, vfp.zcr_el[3]),
-    .writefn = zcr_write, .raw_writefn = raw_write
+static const ARMCPRegInfo zcr_reginfo[] = {
+    { .name = "ZCR_EL1", .state = ARM_CP_STATE_AA64,
+      .opc0 = 3, .opc1 = 0, .crn = 1, .crm = 2, .opc2 = 0,
+      .access = PL1_RW, .type = ARM_CP_SVE,
+      .fieldoffset = offsetof(CPUARMState, vfp.zcr_el[1]),
+      .writefn = zcr_write, .raw_writefn = raw_write },
+    { .name = "ZCR_EL2", .state = ARM_CP_STATE_AA64,
+      .opc0 = 3, .opc1 = 4, .crn = 1, .crm = 2, .opc2 = 0,
+      .access = PL2_RW, .type = ARM_CP_SVE,
+      .fieldoffset = offsetof(CPUARMState, vfp.zcr_el[2]),
+      .writefn = zcr_write, .raw_writefn = raw_write },
+    { .name = "ZCR_EL3", .state = ARM_CP_STATE_AA64,
+      .opc0 = 3, .opc1 = 6, .crn = 1, .crm = 2, .opc2 = 0,
+      .access = PL3_RW, .type = ARM_CP_SVE,
+      .fieldoffset = offsetof(CPUARMState, vfp.zcr_el[3]),
+      .writefn = zcr_write, .raw_writefn = raw_write },
 };
 
 void hw_watchpoint_update(ARMCPU *cpu, int n)
@@ -8233,15 +8220,7 @@ void register_cp_regs_for_features(ARMCPU *cpu)
     }
 
     if (cpu_isar_feature(aa64_sve, cpu)) {
-        define_one_arm_cp_reg(cpu, &zcr_el1_reginfo);
-        if (arm_feature(env, ARM_FEATURE_EL2)) {
-            define_one_arm_cp_reg(cpu, &zcr_el2_reginfo);
-        } else {
-            define_one_arm_cp_reg(cpu, &zcr_no_el2_reginfo);
-        }
-        if (arm_feature(env, ARM_FEATURE_EL3)) {
-            define_one_arm_cp_reg(cpu, &zcr_el3_reginfo);
-        }
+        define_arm_cp_regs(cpu, zcr_reginfo);
     }
 
 #ifdef TARGET_AARCH64

From 52d187275b3a306c6bc83c083885ae34e0939f0d Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:22 -0500
Subject: [PATCH 199/935] target/arm: Adjust definition of CONTEXTIDR_EL2

This register is present for either VHE or Debugv8p2.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-5-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 72d05070f0..7b31c71980 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -7246,11 +7246,14 @@ static const ARMCPRegInfo jazelle_regs[] = {
       .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
 };
 
+static const ARMCPRegInfo contextidr_el2 = {
+    .name = "CONTEXTIDR_EL2", .state = ARM_CP_STATE_AA64,
+    .opc0 = 3, .opc1 = 4, .crn = 13, .crm = 0, .opc2 = 1,
+    .access = PL2_RW,
+    .fieldoffset = offsetof(CPUARMState, cp15.contextidr_el[2])
+};
+
 static const ARMCPRegInfo vhe_reginfo[] = {
-    { .name = "CONTEXTIDR_EL2", .state = ARM_CP_STATE_AA64,
-      .opc0 = 3, .opc1 = 4, .crn = 13, .crm = 0, .opc2 = 1,
-      .access = PL2_RW,
-      .fieldoffset = offsetof(CPUARMState, cp15.contextidr_el[2]) },
     { .name = "TTBR1_EL2", .state = ARM_CP_STATE_AA64,
       .opc0 = 3, .opc1 = 4, .crn = 2, .crm = 0, .opc2 = 1,
       .access = PL2_RW, .writefn = vmsa_tcr_ttbr_el2_write,
@@ -8215,6 +8218,10 @@ void register_cp_regs_for_features(ARMCPU *cpu)
         define_one_arm_cp_reg(cpu, &ssbs_reginfo);
     }
 
+    if (cpu_isar_feature(aa64_vh, cpu) ||
+        cpu_isar_feature(aa64_debugv8p2, cpu)) {
+        define_one_arm_cp_reg(cpu, &contextidr_el2);
+    }
     if (arm_feature(env, ARM_FEATURE_EL2) && cpu_isar_feature(aa64_vh, cpu)) {
         define_arm_cp_regs(cpu, vhe_reginfo);
     }

From 7c1aaf98ff6b06e96de1a25fa8e3214eb402bd77 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:23 -0500
Subject: [PATCH 200/935] target/arm: Move cortex impdef sysregs to cpu_tcg.c

Previously we were defining some of these in user-only mode,
but none of them are accessible from user-only, therefore
define them only in system mode.

This will shortly be used from cpu_tcg.c also.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-6-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu64.c     | 64 +++---------------------------------------
 target/arm/cpu_tcg.c   | 59 ++++++++++++++++++++++++++++++++++++++
 target/arm/internals.h |  6 ++++
 3 files changed, 69 insertions(+), 60 deletions(-)

diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index c841d55d0e..33a0a71900 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -34,65 +34,9 @@
 #include "hvf_arm.h"
 #include "qapi/visitor.h"
 #include "hw/qdev-properties.h"
-#include "cpregs.h"
+#include "internals.h"
 
 
-#ifndef CONFIG_USER_ONLY
-static uint64_t a57_a53_l2ctlr_read(CPUARMState *env, const ARMCPRegInfo *ri)
-{
-    ARMCPU *cpu = env_archcpu(env);
-
-    /* Number of cores is in [25:24]; otherwise we RAZ */
-    return (cpu->core_count - 1) << 24;
-}
-#endif
-
-static const ARMCPRegInfo cortex_a72_a57_a53_cp_reginfo[] = {
-#ifndef CONFIG_USER_ONLY
-    { .name = "L2CTLR_EL1", .state = ARM_CP_STATE_AA64,
-      .opc0 = 3, .opc1 = 1, .crn = 11, .crm = 0, .opc2 = 2,
-      .access = PL1_RW, .readfn = a57_a53_l2ctlr_read,
-      .writefn = arm_cp_write_ignore },
-    { .name = "L2CTLR",
-      .cp = 15, .opc1 = 1, .crn = 9, .crm = 0, .opc2 = 2,
-      .access = PL1_RW, .readfn = a57_a53_l2ctlr_read,
-      .writefn = arm_cp_write_ignore },
-#endif
-    { .name = "L2ECTLR_EL1", .state = ARM_CP_STATE_AA64,
-      .opc0 = 3, .opc1 = 1, .crn = 11, .crm = 0, .opc2 = 3,
-      .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "L2ECTLR",
-      .cp = 15, .opc1 = 1, .crn = 9, .crm = 0, .opc2 = 3,
-      .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "L2ACTLR", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 1, .crn = 15, .crm = 0, .opc2 = 0,
-      .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "CPUACTLR_EL1", .state = ARM_CP_STATE_AA64,
-      .opc0 = 3, .opc1 = 1, .crn = 15, .crm = 2, .opc2 = 0,
-      .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "CPUACTLR",
-      .cp = 15, .opc1 = 0, .crm = 15,
-      .access = PL1_RW, .type = ARM_CP_CONST | ARM_CP_64BIT, .resetvalue = 0 },
-    { .name = "CPUECTLR_EL1", .state = ARM_CP_STATE_AA64,
-      .opc0 = 3, .opc1 = 1, .crn = 15, .crm = 2, .opc2 = 1,
-      .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "CPUECTLR",
-      .cp = 15, .opc1 = 1, .crm = 15,
-      .access = PL1_RW, .type = ARM_CP_CONST | ARM_CP_64BIT, .resetvalue = 0 },
-    { .name = "CPUMERRSR_EL1", .state = ARM_CP_STATE_AA64,
-      .opc0 = 3, .opc1 = 1, .crn = 15, .crm = 2, .opc2 = 2,
-      .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "CPUMERRSR",
-      .cp = 15, .opc1 = 2, .crm = 15,
-      .access = PL1_RW, .type = ARM_CP_CONST | ARM_CP_64BIT, .resetvalue = 0 },
-    { .name = "L2MERRSR_EL1", .state = ARM_CP_STATE_AA64,
-      .opc0 = 3, .opc1 = 1, .crn = 15, .crm = 2, .opc2 = 3,
-      .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
-    { .name = "L2MERRSR",
-      .cp = 15, .opc1 = 3, .crm = 15,
-      .access = PL1_RW, .type = ARM_CP_CONST | ARM_CP_64BIT, .resetvalue = 0 },
-};
-
 static void aarch64_a57_initfn(Object *obj)
 {
     ARMCPU *cpu = ARM_CPU(obj);
@@ -143,7 +87,7 @@ static void aarch64_a57_initfn(Object *obj)
     cpu->gic_num_lrs = 4;
     cpu->gic_vpribits = 5;
     cpu->gic_vprebits = 5;
-    define_arm_cp_regs(cpu, cortex_a72_a57_a53_cp_reginfo);
+    define_cortex_a72_a57_a53_cp_reginfo(cpu);
 }
 
 static void aarch64_a53_initfn(Object *obj)
@@ -196,7 +140,7 @@ static void aarch64_a53_initfn(Object *obj)
     cpu->gic_num_lrs = 4;
     cpu->gic_vpribits = 5;
     cpu->gic_vprebits = 5;
-    define_arm_cp_regs(cpu, cortex_a72_a57_a53_cp_reginfo);
+    define_cortex_a72_a57_a53_cp_reginfo(cpu);
 }
 
 static void aarch64_a72_initfn(Object *obj)
@@ -247,7 +191,7 @@ static void aarch64_a72_initfn(Object *obj)
     cpu->gic_num_lrs = 4;
     cpu->gic_vpribits = 5;
     cpu->gic_vprebits = 5;
-    define_arm_cp_regs(cpu, cortex_a72_a57_a53_cp_reginfo);
+    define_cortex_a72_a57_a53_cp_reginfo(cpu);
 }
 
 void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp)
diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
index 9338088b22..d078f06931 100644
--- a/target/arm/cpu_tcg.c
+++ b/target/arm/cpu_tcg.c
@@ -20,6 +20,65 @@
 #endif
 #include "cpregs.h"
 
+#ifndef CONFIG_USER_ONLY
+static uint64_t l2ctlr_read(CPUARMState *env, const ARMCPRegInfo *ri)
+{
+    ARMCPU *cpu = env_archcpu(env);
+
+    /* Number of cores is in [25:24]; otherwise we RAZ */
+    return (cpu->core_count - 1) << 24;
+}
+
+static const ARMCPRegInfo cortex_a72_a57_a53_cp_reginfo[] = {
+    { .name = "L2CTLR_EL1", .state = ARM_CP_STATE_AA64,
+      .opc0 = 3, .opc1 = 1, .crn = 11, .crm = 0, .opc2 = 2,
+      .access = PL1_RW, .readfn = l2ctlr_read,
+      .writefn = arm_cp_write_ignore },
+    { .name = "L2CTLR",
+      .cp = 15, .opc1 = 1, .crn = 9, .crm = 0, .opc2 = 2,
+      .access = PL1_RW, .readfn = l2ctlr_read,
+      .writefn = arm_cp_write_ignore },
+    { .name = "L2ECTLR_EL1", .state = ARM_CP_STATE_AA64,
+      .opc0 = 3, .opc1 = 1, .crn = 11, .crm = 0, .opc2 = 3,
+      .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
+    { .name = "L2ECTLR",
+      .cp = 15, .opc1 = 1, .crn = 9, .crm = 0, .opc2 = 3,
+      .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
+    { .name = "L2ACTLR", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 1, .crn = 15, .crm = 0, .opc2 = 0,
+      .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
+    { .name = "CPUACTLR_EL1", .state = ARM_CP_STATE_AA64,
+      .opc0 = 3, .opc1 = 1, .crn = 15, .crm = 2, .opc2 = 0,
+      .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
+    { .name = "CPUACTLR",
+      .cp = 15, .opc1 = 0, .crm = 15,
+      .access = PL1_RW, .type = ARM_CP_CONST | ARM_CP_64BIT, .resetvalue = 0 },
+    { .name = "CPUECTLR_EL1", .state = ARM_CP_STATE_AA64,
+      .opc0 = 3, .opc1 = 1, .crn = 15, .crm = 2, .opc2 = 1,
+      .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
+    { .name = "CPUECTLR",
+      .cp = 15, .opc1 = 1, .crm = 15,
+      .access = PL1_RW, .type = ARM_CP_CONST | ARM_CP_64BIT, .resetvalue = 0 },
+    { .name = "CPUMERRSR_EL1", .state = ARM_CP_STATE_AA64,
+      .opc0 = 3, .opc1 = 1, .crn = 15, .crm = 2, .opc2 = 2,
+      .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
+    { .name = "CPUMERRSR",
+      .cp = 15, .opc1 = 2, .crm = 15,
+      .access = PL1_RW, .type = ARM_CP_CONST | ARM_CP_64BIT, .resetvalue = 0 },
+    { .name = "L2MERRSR_EL1", .state = ARM_CP_STATE_AA64,
+      .opc0 = 3, .opc1 = 1, .crn = 15, .crm = 2, .opc2 = 3,
+      .access = PL1_RW, .type = ARM_CP_CONST, .resetvalue = 0 },
+    { .name = "L2MERRSR",
+      .cp = 15, .opc1 = 3, .crm = 15,
+      .access = PL1_RW, .type = ARM_CP_CONST | ARM_CP_64BIT, .resetvalue = 0 },
+};
+
+void define_cortex_a72_a57_a53_cp_reginfo(ARMCPU *cpu)
+{
+    define_arm_cp_regs(cpu, cortex_a72_a57_a53_cp_reginfo);
+}
+#endif /* !CONFIG_USER_ONLY */
+
 /* CPU models. These are not needed for the AArch64 linux-user build. */
 #if !defined(CONFIG_USER_ONLY) || !defined(TARGET_AARCH64)
 
diff --git a/target/arm/internals.h b/target/arm/internals.h
index 255833479d..343b465d51 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -1307,4 +1307,10 @@ int aarch64_fpu_gdb_get_reg(CPUARMState *env, GByteArray *buf, int reg);
 int aarch64_fpu_gdb_set_reg(CPUARMState *env, uint8_t *buf, int reg);
 #endif
 
+#ifdef CONFIG_USER_ONLY
+static inline void define_cortex_a72_a57_a53_cp_reginfo(ARMCPU *cpu) { }
+#else
+void define_cortex_a72_a57_a53_cp_reginfo(ARMCPU *cpu);
+#endif
+
 #endif

From 7e834daf0abe785041804a63f6b65b095ece8e6c Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:24 -0500
Subject: [PATCH 201/935] target/arm: Update qemu-system-arm -cpu max to
 cortex-a57

Instead of starting with cortex-a15 and adding v8 features to
a v7 cpu, begin with a v8 cpu stripped of its aarch64 features.
This fixes the long-standing to-do where we only enabled v8
features for user-only.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-7-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu_tcg.c | 151 ++++++++++++++++++++++++++-----------------
 1 file changed, 92 insertions(+), 59 deletions(-)

diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
index d078f06931..f9094c1752 100644
--- a/target/arm/cpu_tcg.c
+++ b/target/arm/cpu_tcg.c
@@ -994,71 +994,104 @@ static void arm_v7m_class_init(ObjectClass *oc, void *data)
 static void arm_max_initfn(Object *obj)
 {
     ARMCPU *cpu = ARM_CPU(obj);
+    uint32_t t;
 
-    cortex_a15_initfn(obj);
+    /* aarch64_a57_initfn, advertising none of the aarch64 features */
+    cpu->dtb_compatible = "arm,cortex-a57";
+    set_feature(&cpu->env, ARM_FEATURE_V8);
+    set_feature(&cpu->env, ARM_FEATURE_NEON);
+    set_feature(&cpu->env, ARM_FEATURE_GENERIC_TIMER);
+    set_feature(&cpu->env, ARM_FEATURE_CBAR_RO);
+    set_feature(&cpu->env, ARM_FEATURE_EL2);
+    set_feature(&cpu->env, ARM_FEATURE_EL3);
+    set_feature(&cpu->env, ARM_FEATURE_PMU);
+    cpu->midr = 0x411fd070;
+    cpu->revidr = 0x00000000;
+    cpu->reset_fpsid = 0x41034070;
+    cpu->isar.mvfr0 = 0x10110222;
+    cpu->isar.mvfr1 = 0x12111111;
+    cpu->isar.mvfr2 = 0x00000043;
+    cpu->ctr = 0x8444c004;
+    cpu->reset_sctlr = 0x00c50838;
+    cpu->isar.id_pfr0 = 0x00000131;
+    cpu->isar.id_pfr1 = 0x00011011;
+    cpu->isar.id_dfr0 = 0x03010066;
+    cpu->id_afr0 = 0x00000000;
+    cpu->isar.id_mmfr0 = 0x10101105;
+    cpu->isar.id_mmfr1 = 0x40000000;
+    cpu->isar.id_mmfr2 = 0x01260000;
+    cpu->isar.id_mmfr3 = 0x02102211;
+    cpu->isar.id_isar0 = 0x02101110;
+    cpu->isar.id_isar1 = 0x13112111;
+    cpu->isar.id_isar2 = 0x21232042;
+    cpu->isar.id_isar3 = 0x01112131;
+    cpu->isar.id_isar4 = 0x00011142;
+    cpu->isar.id_isar5 = 0x00011121;
+    cpu->isar.id_isar6 = 0;
+    cpu->isar.dbgdidr = 0x3516d000;
+    cpu->clidr = 0x0a200023;
+    cpu->ccsidr[0] = 0x701fe00a; /* 32KB L1 dcache */
+    cpu->ccsidr[1] = 0x201fe012; /* 48KB L1 icache */
+    cpu->ccsidr[2] = 0x70ffe07a; /* 2048KB L2 cache */
+    define_cortex_a72_a57_a53_cp_reginfo(cpu);
 
-    /* old-style VFP short-vector support */
-    cpu->isar.mvfr0 = FIELD_DP32(cpu->isar.mvfr0, MVFR0, FPSHVEC, 1);
+    /* Add additional features supported by QEMU */
+    t = cpu->isar.id_isar5;
+    t = FIELD_DP32(t, ID_ISAR5, AES, 2);
+    t = FIELD_DP32(t, ID_ISAR5, SHA1, 1);
+    t = FIELD_DP32(t, ID_ISAR5, SHA2, 1);
+    t = FIELD_DP32(t, ID_ISAR5, CRC32, 1);
+    t = FIELD_DP32(t, ID_ISAR5, RDM, 1);
+    t = FIELD_DP32(t, ID_ISAR5, VCMA, 1);
+    cpu->isar.id_isar5 = t;
+
+    t = cpu->isar.id_isar6;
+    t = FIELD_DP32(t, ID_ISAR6, JSCVT, 1);
+    t = FIELD_DP32(t, ID_ISAR6, DP, 1);
+    t = FIELD_DP32(t, ID_ISAR6, FHM, 1);
+    t = FIELD_DP32(t, ID_ISAR6, SB, 1);
+    t = FIELD_DP32(t, ID_ISAR6, SPECRES, 1);
+    t = FIELD_DP32(t, ID_ISAR6, BF16, 1);
+    t = FIELD_DP32(t, ID_ISAR6, I8MM, 1);
+    cpu->isar.id_isar6 = t;
+
+    t = cpu->isar.mvfr1;
+    t = FIELD_DP32(t, MVFR1, FPHP, 3);     /* v8.2-FP16 */
+    t = FIELD_DP32(t, MVFR1, SIMDHP, 2);   /* v8.2-FP16 */
+    cpu->isar.mvfr1 = t;
+
+    t = cpu->isar.mvfr2;
+    t = FIELD_DP32(t, MVFR2, SIMDMISC, 3); /* SIMD MaxNum */
+    t = FIELD_DP32(t, MVFR2, FPMISC, 4);   /* FP MaxNum */
+    cpu->isar.mvfr2 = t;
+
+    t = cpu->isar.id_mmfr3;
+    t = FIELD_DP32(t, ID_MMFR3, PAN, 2); /* ATS1E1 */
+    cpu->isar.id_mmfr3 = t;
+
+    t = cpu->isar.id_mmfr4;
+    t = FIELD_DP32(t, ID_MMFR4, HPDS, 1); /* AA32HPD */
+    t = FIELD_DP32(t, ID_MMFR4, AC2, 1); /* ACTLR2, HACTLR2 */
+    t = FIELD_DP32(t, ID_MMFR4, CNP, 1); /* TTCNP */
+    t = FIELD_DP32(t, ID_MMFR4, XNX, 1); /* TTS2UXN */
+    cpu->isar.id_mmfr4 = t;
+
+    t = cpu->isar.id_pfr0;
+    t = FIELD_DP32(t, ID_PFR0, DIT, 1);
+    cpu->isar.id_pfr0 = t;
+
+    t = cpu->isar.id_pfr2;
+    t = FIELD_DP32(t, ID_PFR2, SSBS, 1);
+    cpu->isar.id_pfr2 = t;
 
 #ifdef CONFIG_USER_ONLY
     /*
-     * We don't set these in system emulation mode for the moment,
-     * since we don't correctly set (all of) the ID registers to
-     * advertise them.
+     * Break with true ARMv8 and add back old-style VFP short-vector support.
+     * Only do this for user-mode, where -cpu max is the default, so that
+     * older v6 and v7 programs are more likely to work without adjustment.
      */
-    set_feature(&cpu->env, ARM_FEATURE_V8);
-    {
-        uint32_t t;
-
-        t = cpu->isar.id_isar5;
-        t = FIELD_DP32(t, ID_ISAR5, AES, 2);
-        t = FIELD_DP32(t, ID_ISAR5, SHA1, 1);
-        t = FIELD_DP32(t, ID_ISAR5, SHA2, 1);
-        t = FIELD_DP32(t, ID_ISAR5, CRC32, 1);
-        t = FIELD_DP32(t, ID_ISAR5, RDM, 1);
-        t = FIELD_DP32(t, ID_ISAR5, VCMA, 1);
-        cpu->isar.id_isar5 = t;
-
-        t = cpu->isar.id_isar6;
-        t = FIELD_DP32(t, ID_ISAR6, JSCVT, 1);
-        t = FIELD_DP32(t, ID_ISAR6, DP, 1);
-        t = FIELD_DP32(t, ID_ISAR6, FHM, 1);
-        t = FIELD_DP32(t, ID_ISAR6, SB, 1);
-        t = FIELD_DP32(t, ID_ISAR6, SPECRES, 1);
-        t = FIELD_DP32(t, ID_ISAR6, BF16, 1);
-        t = FIELD_DP32(t, ID_ISAR6, I8MM, 1);
-        cpu->isar.id_isar6 = t;
-
-        t = cpu->isar.mvfr1;
-        t = FIELD_DP32(t, MVFR1, FPHP, 3);     /* v8.2-FP16 */
-        t = FIELD_DP32(t, MVFR1, SIMDHP, 2);   /* v8.2-FP16 */
-        cpu->isar.mvfr1 = t;
-
-        t = cpu->isar.mvfr2;
-        t = FIELD_DP32(t, MVFR2, SIMDMISC, 3); /* SIMD MaxNum */
-        t = FIELD_DP32(t, MVFR2, FPMISC, 4);   /* FP MaxNum */
-        cpu->isar.mvfr2 = t;
-
-        t = cpu->isar.id_mmfr3;
-        t = FIELD_DP32(t, ID_MMFR3, PAN, 2); /* ATS1E1 */
-        cpu->isar.id_mmfr3 = t;
-
-        t = cpu->isar.id_mmfr4;
-        t = FIELD_DP32(t, ID_MMFR4, HPDS, 1); /* AA32HPD */
-        t = FIELD_DP32(t, ID_MMFR4, AC2, 1); /* ACTLR2, HACTLR2 */
-        t = FIELD_DP32(t, ID_MMFR4, CNP, 1); /* TTCNP */
-        t = FIELD_DP32(t, ID_MMFR4, XNX, 1); /* TTS2UXN */
-        cpu->isar.id_mmfr4 = t;
-
-        t = cpu->isar.id_pfr0;
-        t = FIELD_DP32(t, ID_PFR0, DIT, 1);
-        cpu->isar.id_pfr0 = t;
-
-        t = cpu->isar.id_pfr2;
-        t = FIELD_DP32(t, ID_PFR2, SSBS, 1);
-        cpu->isar.id_pfr2 = t;
-    }
-#endif /* CONFIG_USER_ONLY */
+    cpu->isar.mvfr0 = FIELD_DP32(cpu->isar.mvfr0, MVFR0, FPSHVEC, 1);
+#endif
 }
 #endif /* !TARGET_AARCH64 */
 

From e14cc941f109cb93fab28a2b3fe79c700c7b6eeb Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:25 -0500
Subject: [PATCH 202/935] target/arm: Set ID_DFR0.PerfMon for qemu-system-arm
 -cpu max

We set this for qemu-system-aarch64, but failed to do so
for the strictly 32-bit emulation.

Fixes: 3bec78447a9 ("target/arm: Provide ARMv8.4-PMU in '-cpu max'")
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-8-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu_tcg.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
index f9094c1752..9aa2f737c1 100644
--- a/target/arm/cpu_tcg.c
+++ b/target/arm/cpu_tcg.c
@@ -1084,6 +1084,10 @@ static void arm_max_initfn(Object *obj)
     t = FIELD_DP32(t, ID_PFR2, SSBS, 1);
     cpu->isar.id_pfr2 = t;
 
+    t = cpu->isar.id_dfr0;
+    t = FIELD_DP32(t, ID_DFR0, PERFMON, 5); /* v8.4-PMU */
+    cpu->isar.id_dfr0 = t;
+
 #ifdef CONFIG_USER_ONLY
     /*
      * Break with true ARMv8 and add back old-style VFP short-vector support.

From b6f8b358c2a2784433f51299279cd8e04fc981ac Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:26 -0500
Subject: [PATCH 203/935] target/arm: Split out aa32_max_features

Share the code to set AArch32 max features so that we no
longer have code drift between qemu{-system,}-{arm,aarch64}.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-9-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu64.c     |  50 +-----------------
 target/arm/cpu_tcg.c   | 114 ++++++++++++++++++++++-------------------
 target/arm/internals.h |   2 +
 3 files changed, 65 insertions(+), 101 deletions(-)

diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 33a0a71900..6da42af56e 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -682,7 +682,6 @@ static void aarch64_max_initfn(Object *obj)
 {
     ARMCPU *cpu = ARM_CPU(obj);
     uint64_t t;
-    uint32_t u;
 
     if (kvm_enabled() || hvf_enabled()) {
         /* With KVM or HVF, '-cpu max' is identical to '-cpu host' */
@@ -799,57 +798,12 @@ static void aarch64_max_initfn(Object *obj)
     t = FIELD_DP64(t, ID_AA64ZFR0, F64MM, 1);
     cpu->isar.id_aa64zfr0 = t;
 
-    /* Replicate the same data to the 32-bit id registers.  */
-    u = cpu->isar.id_isar5;
-    u = FIELD_DP32(u, ID_ISAR5, AES, 2); /* AES + PMULL */
-    u = FIELD_DP32(u, ID_ISAR5, SHA1, 1);
-    u = FIELD_DP32(u, ID_ISAR5, SHA2, 1);
-    u = FIELD_DP32(u, ID_ISAR5, CRC32, 1);
-    u = FIELD_DP32(u, ID_ISAR5, RDM, 1);
-    u = FIELD_DP32(u, ID_ISAR5, VCMA, 1);
-    cpu->isar.id_isar5 = u;
-
-    u = cpu->isar.id_isar6;
-    u = FIELD_DP32(u, ID_ISAR6, JSCVT, 1);
-    u = FIELD_DP32(u, ID_ISAR6, DP, 1);
-    u = FIELD_DP32(u, ID_ISAR6, FHM, 1);
-    u = FIELD_DP32(u, ID_ISAR6, SB, 1);
-    u = FIELD_DP32(u, ID_ISAR6, SPECRES, 1);
-    u = FIELD_DP32(u, ID_ISAR6, BF16, 1);
-    u = FIELD_DP32(u, ID_ISAR6, I8MM, 1);
-    cpu->isar.id_isar6 = u;
-
-    u = cpu->isar.id_pfr0;
-    u = FIELD_DP32(u, ID_PFR0, DIT, 1);
-    cpu->isar.id_pfr0 = u;
-
-    u = cpu->isar.id_pfr2;
-    u = FIELD_DP32(u, ID_PFR2, SSBS, 1);
-    cpu->isar.id_pfr2 = u;
-
-    u = cpu->isar.id_mmfr3;
-    u = FIELD_DP32(u, ID_MMFR3, PAN, 2); /* ATS1E1 */
-    cpu->isar.id_mmfr3 = u;
-
-    u = cpu->isar.id_mmfr4;
-    u = FIELD_DP32(u, ID_MMFR4, HPDS, 1); /* AA32HPD */
-    u = FIELD_DP32(u, ID_MMFR4, AC2, 1); /* ACTLR2, HACTLR2 */
-    u = FIELD_DP32(u, ID_MMFR4, CNP, 1); /* TTCNP */
-    u = FIELD_DP32(u, ID_MMFR4, XNX, 1); /* TTS2UXN */
-    cpu->isar.id_mmfr4 = u;
-
     t = cpu->isar.id_aa64dfr0;
     t = FIELD_DP64(t, ID_AA64DFR0, PMUVER, 5); /* v8.4-PMU */
     cpu->isar.id_aa64dfr0 = t;
 
-    u = cpu->isar.id_dfr0;
-    u = FIELD_DP32(u, ID_DFR0, PERFMON, 5); /* v8.4-PMU */
-    cpu->isar.id_dfr0 = u;
-
-    u = cpu->isar.mvfr1;
-    u = FIELD_DP32(u, MVFR1, FPHP, 3);      /* v8.2-FP16 */
-    u = FIELD_DP32(u, MVFR1, SIMDHP, 2);    /* v8.2-FP16 */
-    cpu->isar.mvfr1 = u;
+    /* Replicate the same data to the 32-bit id registers.  */
+    aa32_max_features(cpu);
 
 #ifdef CONFIG_USER_ONLY
     /*
diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
index 9aa2f737c1..b0dbf2c991 100644
--- a/target/arm/cpu_tcg.c
+++ b/target/arm/cpu_tcg.c
@@ -20,6 +20,66 @@
 #endif
 #include "cpregs.h"
 
+
+/* Share AArch32 -cpu max features with AArch64. */
+void aa32_max_features(ARMCPU *cpu)
+{
+    uint32_t t;
+
+    /* Add additional features supported by QEMU */
+    t = cpu->isar.id_isar5;
+    t = FIELD_DP32(t, ID_ISAR5, AES, 2);
+    t = FIELD_DP32(t, ID_ISAR5, SHA1, 1);
+    t = FIELD_DP32(t, ID_ISAR5, SHA2, 1);
+    t = FIELD_DP32(t, ID_ISAR5, CRC32, 1);
+    t = FIELD_DP32(t, ID_ISAR5, RDM, 1);
+    t = FIELD_DP32(t, ID_ISAR5, VCMA, 1);
+    cpu->isar.id_isar5 = t;
+
+    t = cpu->isar.id_isar6;
+    t = FIELD_DP32(t, ID_ISAR6, JSCVT, 1);
+    t = FIELD_DP32(t, ID_ISAR6, DP, 1);
+    t = FIELD_DP32(t, ID_ISAR6, FHM, 1);
+    t = FIELD_DP32(t, ID_ISAR6, SB, 1);
+    t = FIELD_DP32(t, ID_ISAR6, SPECRES, 1);
+    t = FIELD_DP32(t, ID_ISAR6, BF16, 1);
+    t = FIELD_DP32(t, ID_ISAR6, I8MM, 1);
+    cpu->isar.id_isar6 = t;
+
+    t = cpu->isar.mvfr1;
+    t = FIELD_DP32(t, MVFR1, FPHP, 3);     /* v8.2-FP16 */
+    t = FIELD_DP32(t, MVFR1, SIMDHP, 2);   /* v8.2-FP16 */
+    cpu->isar.mvfr1 = t;
+
+    t = cpu->isar.mvfr2;
+    t = FIELD_DP32(t, MVFR2, SIMDMISC, 3); /* SIMD MaxNum */
+    t = FIELD_DP32(t, MVFR2, FPMISC, 4);   /* FP MaxNum */
+    cpu->isar.mvfr2 = t;
+
+    t = cpu->isar.id_mmfr3;
+    t = FIELD_DP32(t, ID_MMFR3, PAN, 2); /* ATS1E1 */
+    cpu->isar.id_mmfr3 = t;
+
+    t = cpu->isar.id_mmfr4;
+    t = FIELD_DP32(t, ID_MMFR4, HPDS, 1); /* AA32HPD */
+    t = FIELD_DP32(t, ID_MMFR4, AC2, 1); /* ACTLR2, HACTLR2 */
+    t = FIELD_DP32(t, ID_MMFR4, CNP, 1); /* TTCNP */
+    t = FIELD_DP32(t, ID_MMFR4, XNX, 1); /* TTS2UXN */
+    cpu->isar.id_mmfr4 = t;
+
+    t = cpu->isar.id_pfr0;
+    t = FIELD_DP32(t, ID_PFR0, DIT, 1);
+    cpu->isar.id_pfr0 = t;
+
+    t = cpu->isar.id_pfr2;
+    t = FIELD_DP32(t, ID_PFR2, SSBS, 1);
+    cpu->isar.id_pfr2 = t;
+
+    t = cpu->isar.id_dfr0;
+    t = FIELD_DP32(t, ID_DFR0, PERFMON, 5); /* v8.4-PMU */
+    cpu->isar.id_dfr0 = t;
+}
+
 #ifndef CONFIG_USER_ONLY
 static uint64_t l2ctlr_read(CPUARMState *env, const ARMCPRegInfo *ri)
 {
@@ -994,7 +1054,6 @@ static void arm_v7m_class_init(ObjectClass *oc, void *data)
 static void arm_max_initfn(Object *obj)
 {
     ARMCPU *cpu = ARM_CPU(obj);
-    uint32_t t;
 
     /* aarch64_a57_initfn, advertising none of the aarch64 features */
     cpu->dtb_compatible = "arm,cortex-a57";
@@ -1035,58 +1094,7 @@ static void arm_max_initfn(Object *obj)
     cpu->ccsidr[2] = 0x70ffe07a; /* 2048KB L2 cache */
     define_cortex_a72_a57_a53_cp_reginfo(cpu);
 
-    /* Add additional features supported by QEMU */
-    t = cpu->isar.id_isar5;
-    t = FIELD_DP32(t, ID_ISAR5, AES, 2);
-    t = FIELD_DP32(t, ID_ISAR5, SHA1, 1);
-    t = FIELD_DP32(t, ID_ISAR5, SHA2, 1);
-    t = FIELD_DP32(t, ID_ISAR5, CRC32, 1);
-    t = FIELD_DP32(t, ID_ISAR5, RDM, 1);
-    t = FIELD_DP32(t, ID_ISAR5, VCMA, 1);
-    cpu->isar.id_isar5 = t;
-
-    t = cpu->isar.id_isar6;
-    t = FIELD_DP32(t, ID_ISAR6, JSCVT, 1);
-    t = FIELD_DP32(t, ID_ISAR6, DP, 1);
-    t = FIELD_DP32(t, ID_ISAR6, FHM, 1);
-    t = FIELD_DP32(t, ID_ISAR6, SB, 1);
-    t = FIELD_DP32(t, ID_ISAR6, SPECRES, 1);
-    t = FIELD_DP32(t, ID_ISAR6, BF16, 1);
-    t = FIELD_DP32(t, ID_ISAR6, I8MM, 1);
-    cpu->isar.id_isar6 = t;
-
-    t = cpu->isar.mvfr1;
-    t = FIELD_DP32(t, MVFR1, FPHP, 3);     /* v8.2-FP16 */
-    t = FIELD_DP32(t, MVFR1, SIMDHP, 2);   /* v8.2-FP16 */
-    cpu->isar.mvfr1 = t;
-
-    t = cpu->isar.mvfr2;
-    t = FIELD_DP32(t, MVFR2, SIMDMISC, 3); /* SIMD MaxNum */
-    t = FIELD_DP32(t, MVFR2, FPMISC, 4);   /* FP MaxNum */
-    cpu->isar.mvfr2 = t;
-
-    t = cpu->isar.id_mmfr3;
-    t = FIELD_DP32(t, ID_MMFR3, PAN, 2); /* ATS1E1 */
-    cpu->isar.id_mmfr3 = t;
-
-    t = cpu->isar.id_mmfr4;
-    t = FIELD_DP32(t, ID_MMFR4, HPDS, 1); /* AA32HPD */
-    t = FIELD_DP32(t, ID_MMFR4, AC2, 1); /* ACTLR2, HACTLR2 */
-    t = FIELD_DP32(t, ID_MMFR4, CNP, 1); /* TTCNP */
-    t = FIELD_DP32(t, ID_MMFR4, XNX, 1); /* TTS2UXN */
-    cpu->isar.id_mmfr4 = t;
-
-    t = cpu->isar.id_pfr0;
-    t = FIELD_DP32(t, ID_PFR0, DIT, 1);
-    cpu->isar.id_pfr0 = t;
-
-    t = cpu->isar.id_pfr2;
-    t = FIELD_DP32(t, ID_PFR2, SSBS, 1);
-    cpu->isar.id_pfr2 = t;
-
-    t = cpu->isar.id_dfr0;
-    t = FIELD_DP32(t, ID_DFR0, PERFMON, 5); /* v8.4-PMU */
-    cpu->isar.id_dfr0 = t;
+    aa32_max_features(cpu);
 
 #ifdef CONFIG_USER_ONLY
     /*
diff --git a/target/arm/internals.h b/target/arm/internals.h
index 343b465d51..c563b3735f 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -1313,4 +1313,6 @@ static inline void define_cortex_a72_a57_a53_cp_reginfo(ARMCPU *cpu) { }
 void define_cortex_a72_a57_a53_cp_reginfo(ARMCPU *cpu);
 #endif
 
+void aa32_max_features(ARMCPU *cpu);
+
 #endif

From ef696cfbae6290d448ca0b36f9e41e3e0ec3d50d Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:27 -0500
Subject: [PATCH 204/935] target/arm: Annotate arm_max_initfn with FEAT
 identifiers

Update the legacy feature names to the current names.
Provide feature names for id changes that were not marked.
Sort the field updates into increasing bitfield order.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-10-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu64.c   | 100 +++++++++++++++++++++----------------------
 target/arm/cpu_tcg.c |  48 ++++++++++-----------
 2 files changed, 74 insertions(+), 74 deletions(-)

diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 6da42af56e..5fce40a6bc 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -713,51 +713,51 @@ static void aarch64_max_initfn(Object *obj)
     cpu->midr = t;
 
     t = cpu->isar.id_aa64isar0;
-    t = FIELD_DP64(t, ID_AA64ISAR0, AES, 2); /* AES + PMULL */
-    t = FIELD_DP64(t, ID_AA64ISAR0, SHA1, 1);
-    t = FIELD_DP64(t, ID_AA64ISAR0, SHA2, 2); /* SHA512 */
+    t = FIELD_DP64(t, ID_AA64ISAR0, AES, 2);      /* FEAT_PMULL */
+    t = FIELD_DP64(t, ID_AA64ISAR0, SHA1, 1);     /* FEAT_SHA1 */
+    t = FIELD_DP64(t, ID_AA64ISAR0, SHA2, 2);     /* FEAT_SHA512 */
     t = FIELD_DP64(t, ID_AA64ISAR0, CRC32, 1);
-    t = FIELD_DP64(t, ID_AA64ISAR0, ATOMIC, 2);
-    t = FIELD_DP64(t, ID_AA64ISAR0, RDM, 1);
-    t = FIELD_DP64(t, ID_AA64ISAR0, SHA3, 1);
-    t = FIELD_DP64(t, ID_AA64ISAR0, SM3, 1);
-    t = FIELD_DP64(t, ID_AA64ISAR0, SM4, 1);
-    t = FIELD_DP64(t, ID_AA64ISAR0, DP, 1);
-    t = FIELD_DP64(t, ID_AA64ISAR0, FHM, 1);
-    t = FIELD_DP64(t, ID_AA64ISAR0, TS, 2); /* v8.5-CondM */
-    t = FIELD_DP64(t, ID_AA64ISAR0, TLB, 2); /* FEAT_TLBIRANGE */
-    t = FIELD_DP64(t, ID_AA64ISAR0, RNDR, 1);
+    t = FIELD_DP64(t, ID_AA64ISAR0, ATOMIC, 2);   /* FEAT_LSE */
+    t = FIELD_DP64(t, ID_AA64ISAR0, RDM, 1);      /* FEAT_RDM */
+    t = FIELD_DP64(t, ID_AA64ISAR0, SHA3, 1);     /* FEAT_SHA3 */
+    t = FIELD_DP64(t, ID_AA64ISAR0, SM3, 1);      /* FEAT_SM3 */
+    t = FIELD_DP64(t, ID_AA64ISAR0, SM4, 1);      /* FEAT_SM4 */
+    t = FIELD_DP64(t, ID_AA64ISAR0, DP, 1);       /* FEAT_DotProd */
+    t = FIELD_DP64(t, ID_AA64ISAR0, FHM, 1);      /* FEAT_FHM */
+    t = FIELD_DP64(t, ID_AA64ISAR0, TS, 2);       /* FEAT_FlagM2 */
+    t = FIELD_DP64(t, ID_AA64ISAR0, TLB, 2);      /* FEAT_TLBIRANGE */
+    t = FIELD_DP64(t, ID_AA64ISAR0, RNDR, 1);     /* FEAT_RNG */
     cpu->isar.id_aa64isar0 = t;
 
     t = cpu->isar.id_aa64isar1;
-    t = FIELD_DP64(t, ID_AA64ISAR1, DPB, 2);
-    t = FIELD_DP64(t, ID_AA64ISAR1, JSCVT, 1);
-    t = FIELD_DP64(t, ID_AA64ISAR1, FCMA, 1);
-    t = FIELD_DP64(t, ID_AA64ISAR1, SB, 1);
-    t = FIELD_DP64(t, ID_AA64ISAR1, SPECRES, 1);
-    t = FIELD_DP64(t, ID_AA64ISAR1, BF16, 1);
-    t = FIELD_DP64(t, ID_AA64ISAR1, FRINTTS, 1);
-    t = FIELD_DP64(t, ID_AA64ISAR1, LRCPC, 2); /* ARMv8.4-RCPC */
-    t = FIELD_DP64(t, ID_AA64ISAR1, I8MM, 1);
+    t = FIELD_DP64(t, ID_AA64ISAR1, DPB, 2);      /* FEAT_DPB2 */
+    t = FIELD_DP64(t, ID_AA64ISAR1, JSCVT, 1);    /* FEAT_JSCVT */
+    t = FIELD_DP64(t, ID_AA64ISAR1, FCMA, 1);     /* FEAT_FCMA */
+    t = FIELD_DP64(t, ID_AA64ISAR1, LRCPC, 2);    /* FEAT_LRCPC2 */
+    t = FIELD_DP64(t, ID_AA64ISAR1, FRINTTS, 1);  /* FEAT_FRINTTS */
+    t = FIELD_DP64(t, ID_AA64ISAR1, SB, 1);       /* FEAT_SB */
+    t = FIELD_DP64(t, ID_AA64ISAR1, SPECRES, 1);  /* FEAT_SPECRES */
+    t = FIELD_DP64(t, ID_AA64ISAR1, BF16, 1);     /* FEAT_BF16 */
+    t = FIELD_DP64(t, ID_AA64ISAR1, I8MM, 1);     /* FEAT_I8MM */
     cpu->isar.id_aa64isar1 = t;
 
     t = cpu->isar.id_aa64pfr0;
+    t = FIELD_DP64(t, ID_AA64PFR0, FP, 1);        /* FEAT_FP16 */
+    t = FIELD_DP64(t, ID_AA64PFR0, ADVSIMD, 1);   /* FEAT_FP16 */
     t = FIELD_DP64(t, ID_AA64PFR0, SVE, 1);
-    t = FIELD_DP64(t, ID_AA64PFR0, FP, 1);
-    t = FIELD_DP64(t, ID_AA64PFR0, ADVSIMD, 1);
-    t = FIELD_DP64(t, ID_AA64PFR0, SEL2, 1);
-    t = FIELD_DP64(t, ID_AA64PFR0, DIT, 1);
+    t = FIELD_DP64(t, ID_AA64PFR0, SEL2, 1);      /* FEAT_SEL2 */
+    t = FIELD_DP64(t, ID_AA64PFR0, DIT, 1);       /* FEAT_DIT */
     cpu->isar.id_aa64pfr0 = t;
 
     t = cpu->isar.id_aa64pfr1;
-    t = FIELD_DP64(t, ID_AA64PFR1, BT, 1);
-    t = FIELD_DP64(t, ID_AA64PFR1, SSBS, 2);
+    t = FIELD_DP64(t, ID_AA64PFR1, BT, 1);        /* FEAT_BTI */
+    t = FIELD_DP64(t, ID_AA64PFR1, SSBS, 2);      /* FEAT_SSBS2 */
     /*
      * Begin with full support for MTE. This will be downgraded to MTE=0
      * during realize if the board provides no tag memory, much like
      * we do for EL2 with the virtualization=on property.
      */
-    t = FIELD_DP64(t, ID_AA64PFR1, MTE, 3);
+    t = FIELD_DP64(t, ID_AA64PFR1, MTE, 3);       /* FEAT_MTE3 */
     cpu->isar.id_aa64pfr1 = t;
 
     t = cpu->isar.id_aa64mmfr0;
@@ -769,37 +769,37 @@ static void aarch64_max_initfn(Object *obj)
     cpu->isar.id_aa64mmfr0 = t;
 
     t = cpu->isar.id_aa64mmfr1;
-    t = FIELD_DP64(t, ID_AA64MMFR1, HPDS, 1); /* HPD */
-    t = FIELD_DP64(t, ID_AA64MMFR1, LO, 1);
-    t = FIELD_DP64(t, ID_AA64MMFR1, VH, 1);
-    t = FIELD_DP64(t, ID_AA64MMFR1, PAN, 2); /* ATS1E1 */
-    t = FIELD_DP64(t, ID_AA64MMFR1, VMIDBITS, 2); /* VMID16 */
-    t = FIELD_DP64(t, ID_AA64MMFR1, XNX, 1); /* TTS2UXN */
+    t = FIELD_DP64(t, ID_AA64MMFR1, VMIDBITS, 2); /* FEAT_VMID16 */
+    t = FIELD_DP64(t, ID_AA64MMFR1, VH, 1);       /* FEAT_VHE */
+    t = FIELD_DP64(t, ID_AA64MMFR1, HPDS, 1);     /* FEAT_HPDS */
+    t = FIELD_DP64(t, ID_AA64MMFR1, LO, 1);       /* FEAT_LOR */
+    t = FIELD_DP64(t, ID_AA64MMFR1, PAN, 2);      /* FEAT_PAN2 */
+    t = FIELD_DP64(t, ID_AA64MMFR1, XNX, 1);      /* FEAT_XNX */
     cpu->isar.id_aa64mmfr1 = t;
 
     t = cpu->isar.id_aa64mmfr2;
-    t = FIELD_DP64(t, ID_AA64MMFR2, UAO, 1);
-    t = FIELD_DP64(t, ID_AA64MMFR2, CNP, 1); /* TTCNP */
-    t = FIELD_DP64(t, ID_AA64MMFR2, ST, 1); /* TTST */
-    t = FIELD_DP64(t, ID_AA64MMFR2, VARANGE, 1); /* FEAT_LVA */
-    t = FIELD_DP64(t, ID_AA64MMFR2, TTL, 1); /* FEAT_TTL */
-    t = FIELD_DP64(t, ID_AA64MMFR2, BBM, 2); /* FEAT_BBM at level 2 */
+    t = FIELD_DP64(t, ID_AA64MMFR2, CNP, 1);      /* FEAT_TTCNP */
+    t = FIELD_DP64(t, ID_AA64MMFR2, UAO, 1);      /* FEAT_UAO */
+    t = FIELD_DP64(t, ID_AA64MMFR2, VARANGE, 1);  /* FEAT_LVA */
+    t = FIELD_DP64(t, ID_AA64MMFR2, ST, 1);       /* FEAT_TTST */
+    t = FIELD_DP64(t, ID_AA64MMFR2, TTL, 1);      /* FEAT_TTL */
+    t = FIELD_DP64(t, ID_AA64MMFR2, BBM, 2);      /* FEAT_BBM at level 2 */
     cpu->isar.id_aa64mmfr2 = t;
 
     t = cpu->isar.id_aa64zfr0;
     t = FIELD_DP64(t, ID_AA64ZFR0, SVEVER, 1);
-    t = FIELD_DP64(t, ID_AA64ZFR0, AES, 2);  /* PMULL */
-    t = FIELD_DP64(t, ID_AA64ZFR0, BITPERM, 1);
-    t = FIELD_DP64(t, ID_AA64ZFR0, BFLOAT16, 1);
-    t = FIELD_DP64(t, ID_AA64ZFR0, SHA3, 1);
-    t = FIELD_DP64(t, ID_AA64ZFR0, SM4, 1);
-    t = FIELD_DP64(t, ID_AA64ZFR0, I8MM, 1);
-    t = FIELD_DP64(t, ID_AA64ZFR0, F32MM, 1);
-    t = FIELD_DP64(t, ID_AA64ZFR0, F64MM, 1);
+    t = FIELD_DP64(t, ID_AA64ZFR0, AES, 2);       /* FEAT_SVE_PMULL128 */
+    t = FIELD_DP64(t, ID_AA64ZFR0, BITPERM, 1);   /* FEAT_SVE_BitPerm */
+    t = FIELD_DP64(t, ID_AA64ZFR0, BFLOAT16, 1);  /* FEAT_BF16 */
+    t = FIELD_DP64(t, ID_AA64ZFR0, SHA3, 1);      /* FEAT_SVE_SHA3 */
+    t = FIELD_DP64(t, ID_AA64ZFR0, SM4, 1);       /* FEAT_SVE_SM4 */
+    t = FIELD_DP64(t, ID_AA64ZFR0, I8MM, 1);      /* FEAT_I8MM */
+    t = FIELD_DP64(t, ID_AA64ZFR0, F32MM, 1);     /* FEAT_F32MM */
+    t = FIELD_DP64(t, ID_AA64ZFR0, F64MM, 1);     /* FEAT_F64MM */
     cpu->isar.id_aa64zfr0 = t;
 
     t = cpu->isar.id_aa64dfr0;
-    t = FIELD_DP64(t, ID_AA64DFR0, PMUVER, 5); /* v8.4-PMU */
+    t = FIELD_DP64(t, ID_AA64DFR0, PMUVER, 5);    /* FEAT_PMUv3p4 */
     cpu->isar.id_aa64dfr0 = t;
 
     /* Replicate the same data to the 32-bit id registers.  */
diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
index b0dbf2c991..bc8f9d0edf 100644
--- a/target/arm/cpu_tcg.c
+++ b/target/arm/cpu_tcg.c
@@ -28,55 +28,55 @@ void aa32_max_features(ARMCPU *cpu)
 
     /* Add additional features supported by QEMU */
     t = cpu->isar.id_isar5;
-    t = FIELD_DP32(t, ID_ISAR5, AES, 2);
-    t = FIELD_DP32(t, ID_ISAR5, SHA1, 1);
-    t = FIELD_DP32(t, ID_ISAR5, SHA2, 1);
+    t = FIELD_DP32(t, ID_ISAR5, AES, 2);          /* FEAT_PMULL */
+    t = FIELD_DP32(t, ID_ISAR5, SHA1, 1);         /* FEAT_SHA1 */
+    t = FIELD_DP32(t, ID_ISAR5, SHA2, 1);         /* FEAT_SHA256 */
     t = FIELD_DP32(t, ID_ISAR5, CRC32, 1);
-    t = FIELD_DP32(t, ID_ISAR5, RDM, 1);
-    t = FIELD_DP32(t, ID_ISAR5, VCMA, 1);
+    t = FIELD_DP32(t, ID_ISAR5, RDM, 1);          /* FEAT_RDM */
+    t = FIELD_DP32(t, ID_ISAR5, VCMA, 1);         /* FEAT_FCMA */
     cpu->isar.id_isar5 = t;
 
     t = cpu->isar.id_isar6;
-    t = FIELD_DP32(t, ID_ISAR6, JSCVT, 1);
-    t = FIELD_DP32(t, ID_ISAR6, DP, 1);
-    t = FIELD_DP32(t, ID_ISAR6, FHM, 1);
-    t = FIELD_DP32(t, ID_ISAR6, SB, 1);
-    t = FIELD_DP32(t, ID_ISAR6, SPECRES, 1);
-    t = FIELD_DP32(t, ID_ISAR6, BF16, 1);
-    t = FIELD_DP32(t, ID_ISAR6, I8MM, 1);
+    t = FIELD_DP32(t, ID_ISAR6, JSCVT, 1);        /* FEAT_JSCVT */
+    t = FIELD_DP32(t, ID_ISAR6, DP, 1);           /* Feat_DotProd */
+    t = FIELD_DP32(t, ID_ISAR6, FHM, 1);          /* FEAT_FHM */
+    t = FIELD_DP32(t, ID_ISAR6, SB, 1);           /* FEAT_SB */
+    t = FIELD_DP32(t, ID_ISAR6, SPECRES, 1);      /* FEAT_SPECRES */
+    t = FIELD_DP32(t, ID_ISAR6, BF16, 1);         /* FEAT_AA32BF16 */
+    t = FIELD_DP32(t, ID_ISAR6, I8MM, 1);         /* FEAT_AA32I8MM */
     cpu->isar.id_isar6 = t;
 
     t = cpu->isar.mvfr1;
-    t = FIELD_DP32(t, MVFR1, FPHP, 3);     /* v8.2-FP16 */
-    t = FIELD_DP32(t, MVFR1, SIMDHP, 2);   /* v8.2-FP16 */
+    t = FIELD_DP32(t, MVFR1, FPHP, 3);            /* FEAT_FP16 */
+    t = FIELD_DP32(t, MVFR1, SIMDHP, 2);          /* FEAT_FP16 */
     cpu->isar.mvfr1 = t;
 
     t = cpu->isar.mvfr2;
-    t = FIELD_DP32(t, MVFR2, SIMDMISC, 3); /* SIMD MaxNum */
-    t = FIELD_DP32(t, MVFR2, FPMISC, 4);   /* FP MaxNum */
+    t = FIELD_DP32(t, MVFR2, SIMDMISC, 3);        /* SIMD MaxNum */
+    t = FIELD_DP32(t, MVFR2, FPMISC, 4);          /* FP MaxNum */
     cpu->isar.mvfr2 = t;
 
     t = cpu->isar.id_mmfr3;
-    t = FIELD_DP32(t, ID_MMFR3, PAN, 2); /* ATS1E1 */
+    t = FIELD_DP32(t, ID_MMFR3, PAN, 2);          /* FEAT_PAN2 */
     cpu->isar.id_mmfr3 = t;
 
     t = cpu->isar.id_mmfr4;
-    t = FIELD_DP32(t, ID_MMFR4, HPDS, 1); /* AA32HPD */
-    t = FIELD_DP32(t, ID_MMFR4, AC2, 1); /* ACTLR2, HACTLR2 */
-    t = FIELD_DP32(t, ID_MMFR4, CNP, 1); /* TTCNP */
-    t = FIELD_DP32(t, ID_MMFR4, XNX, 1); /* TTS2UXN */
+    t = FIELD_DP32(t, ID_MMFR4, HPDS, 1);         /* FEAT_AA32HPD */
+    t = FIELD_DP32(t, ID_MMFR4, AC2, 1);          /* ACTLR2, HACTLR2 */
+    t = FIELD_DP32(t, ID_MMFR4, CNP, 1);          /* FEAT_TTCNP */
+    t = FIELD_DP32(t, ID_MMFR4, XNX, 1);          /* FEAT_XNX*/
     cpu->isar.id_mmfr4 = t;
 
     t = cpu->isar.id_pfr0;
-    t = FIELD_DP32(t, ID_PFR0, DIT, 1);
+    t = FIELD_DP32(t, ID_PFR0, DIT, 1);           /* FEAT_DIT */
     cpu->isar.id_pfr0 = t;
 
     t = cpu->isar.id_pfr2;
-    t = FIELD_DP32(t, ID_PFR2, SSBS, 1);
+    t = FIELD_DP32(t, ID_PFR2, SSBS, 1);          /* FEAT_SSBS */
     cpu->isar.id_pfr2 = t;
 
     t = cpu->isar.id_dfr0;
-    t = FIELD_DP32(t, ID_DFR0, PERFMON, 5); /* v8.4-PMU */
+    t = FIELD_DP32(t, ID_DFR0, PERFMON, 5);       /* FEAT_PMUv3p4 */
     cpu->isar.id_dfr0 = t;
 }
 

From b13c91c04b15f5216de9df3878da7dc4c1395979 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:28 -0500
Subject: [PATCH 205/935] target/arm: Use field names for manipulating EL2 and
 EL3 modes

Use FIELD_DP{32,64} to manipulate id_pfr1 and id_aa64pfr0
during arm_cpu_realizefn.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-11-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu.c | 22 +++++++++++++---------
 1 file changed, 13 insertions(+), 9 deletions(-)

diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index 01176b2569..7995ff2712 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -1801,11 +1801,13 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp)
          */
         unset_feature(env, ARM_FEATURE_EL3);
 
-        /* Disable the security extension feature bits in the processor feature
-         * registers as well. These are id_pfr1[7:4] and id_aa64pfr0[15:12].
+        /*
+         * Disable the security extension feature bits in the processor
+         * feature registers as well.
          */
-        cpu->isar.id_pfr1 &= ~0xf0;
-        cpu->isar.id_aa64pfr0 &= ~0xf000;
+        cpu->isar.id_pfr1 = FIELD_DP32(cpu->isar.id_pfr1, ID_PFR1, SECURITY, 0);
+        cpu->isar.id_aa64pfr0 = FIELD_DP64(cpu->isar.id_aa64pfr0,
+                                           ID_AA64PFR0, EL3, 0);
     }
 
     if (!cpu->has_el2) {
@@ -1836,12 +1838,14 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp)
     }
 
     if (!arm_feature(env, ARM_FEATURE_EL2)) {
-        /* Disable the hypervisor feature bits in the processor feature
-         * registers if we don't have EL2. These are id_pfr1[15:12] and
-         * id_aa64pfr0_el1[11:8].
+        /*
+         * Disable the hypervisor feature bits in the processor feature
+         * registers if we don't have EL2.
          */
-        cpu->isar.id_aa64pfr0 &= ~0xf00;
-        cpu->isar.id_pfr1 &= ~0xf000;
+        cpu->isar.id_aa64pfr0 = FIELD_DP64(cpu->isar.id_aa64pfr0,
+                                           ID_AA64PFR0, EL2, 0);
+        cpu->isar.id_pfr1 = FIELD_DP32(cpu->isar.id_pfr1,
+                                       ID_PFR1, VIRTUALIZATION, 0);
     }
 
 #ifndef CONFIG_USER_ONLY

From 033a4f15336646c5dbc07587a7924d71c12a9525 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:29 -0500
Subject: [PATCH 206/935] target/arm: Enable FEAT_Debugv8p2 for -cpu max

The only portion of FEAT_Debugv8p2 that is relevant to QEMU
is CONTEXTIDR_EL2, which is also conditionally implemented
with FEAT_VHE.  The rest of the debug extension concerns the
External debug interface, which is outside the scope of QEMU.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-12-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 docs/system/arm/emulation.rst | 1 +
 target/arm/cpu.c              | 1 +
 target/arm/cpu64.c            | 1 +
 target/arm/cpu_tcg.c          | 2 ++
 4 files changed, 5 insertions(+)

diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst
index c3bd0676a8..965f35d8c9 100644
--- a/docs/system/arm/emulation.rst
+++ b/docs/system/arm/emulation.rst
@@ -14,6 +14,7 @@ the following architecture extensions:
 - FEAT_BTI (Branch Target Identification)
 - FEAT_DIT (Data Independent Timing instructions)
 - FEAT_DPB (DC CVAP instruction)
+- FEAT_Debugv8p2 (Debug changes for v8.2)
 - FEAT_DotProd (Advanced SIMD dot product instructions)
 - FEAT_FCMA (Floating-point complex number instructions)
 - FEAT_FHM (Floating-point half-precision multiplication instructions)
diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index 7995ff2712..2667aaf28b 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -1806,6 +1806,7 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp)
          * feature registers as well.
          */
         cpu->isar.id_pfr1 = FIELD_DP32(cpu->isar.id_pfr1, ID_PFR1, SECURITY, 0);
+        cpu->isar.id_dfr0 = FIELD_DP32(cpu->isar.id_dfr0, ID_DFR0, COPSDBG, 0);
         cpu->isar.id_aa64pfr0 = FIELD_DP64(cpu->isar.id_aa64pfr0,
                                            ID_AA64PFR0, EL3, 0);
     }
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 5fce40a6bc..202fd5c46e 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -799,6 +799,7 @@ static void aarch64_max_initfn(Object *obj)
     cpu->isar.id_aa64zfr0 = t;
 
     t = cpu->isar.id_aa64dfr0;
+    t = FIELD_DP64(t, ID_AA64DFR0, DEBUGVER, 8);  /* FEAT_Debugv8p2 */
     t = FIELD_DP64(t, ID_AA64DFR0, PMUVER, 5);    /* FEAT_PMUv3p4 */
     cpu->isar.id_aa64dfr0 = t;
 
diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
index bc8f9d0edf..b6fc3752f2 100644
--- a/target/arm/cpu_tcg.c
+++ b/target/arm/cpu_tcg.c
@@ -76,6 +76,8 @@ void aa32_max_features(ARMCPU *cpu)
     cpu->isar.id_pfr2 = t;
 
     t = cpu->isar.id_dfr0;
+    t = FIELD_DP32(t, ID_DFR0, COPDBG, 8);        /* FEAT_Debugv8p2 */
+    t = FIELD_DP32(t, ID_DFR0, COPSDBG, 8);       /* FEAT_Debugv8p2 */
     t = FIELD_DP32(t, ID_DFR0, PERFMON, 5);       /* FEAT_PMUv3p4 */
     cpu->isar.id_dfr0 = t;
 }

From 8fc756b6be0d0de777b2092d324907ced7365543 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:30 -0500
Subject: [PATCH 207/935] target/arm: Enable FEAT_Debugv8p4 for -cpu max

This extension concerns changes to the External Debug interface,
with Secure and Non-secure access to the debug registers, and all
of it is outside the scope of QEMU.  Indicating support for this
is mandatory with FEAT_SEL2, which we do implement.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-13-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 docs/system/arm/emulation.rst | 1 +
 target/arm/cpu64.c            | 2 +-
 target/arm/cpu_tcg.c          | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst
index 965f35d8c9..0acac6347c 100644
--- a/docs/system/arm/emulation.rst
+++ b/docs/system/arm/emulation.rst
@@ -15,6 +15,7 @@ the following architecture extensions:
 - FEAT_DIT (Data Independent Timing instructions)
 - FEAT_DPB (DC CVAP instruction)
 - FEAT_Debugv8p2 (Debug changes for v8.2)
+- FEAT_Debugv8p4 (Debug changes for v8.4)
 - FEAT_DotProd (Advanced SIMD dot product instructions)
 - FEAT_FCMA (Floating-point complex number instructions)
 - FEAT_FHM (Floating-point half-precision multiplication instructions)
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 202fd5c46e..88d3cef93e 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -799,7 +799,7 @@ static void aarch64_max_initfn(Object *obj)
     cpu->isar.id_aa64zfr0 = t;
 
     t = cpu->isar.id_aa64dfr0;
-    t = FIELD_DP64(t, ID_AA64DFR0, DEBUGVER, 8);  /* FEAT_Debugv8p2 */
+    t = FIELD_DP64(t, ID_AA64DFR0, DEBUGVER, 9);  /* FEAT_Debugv8p4 */
     t = FIELD_DP64(t, ID_AA64DFR0, PMUVER, 5);    /* FEAT_PMUv3p4 */
     cpu->isar.id_aa64dfr0 = t;
 
diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
index b6fc3752f2..337598e949 100644
--- a/target/arm/cpu_tcg.c
+++ b/target/arm/cpu_tcg.c
@@ -76,8 +76,8 @@ void aa32_max_features(ARMCPU *cpu)
     cpu->isar.id_pfr2 = t;
 
     t = cpu->isar.id_dfr0;
-    t = FIELD_DP32(t, ID_DFR0, COPDBG, 8);        /* FEAT_Debugv8p2 */
-    t = FIELD_DP32(t, ID_DFR0, COPSDBG, 8);       /* FEAT_Debugv8p2 */
+    t = FIELD_DP32(t, ID_DFR0, COPDBG, 9);        /* FEAT_Debugv8p4 */
+    t = FIELD_DP32(t, ID_DFR0, COPSDBG, 9);       /* FEAT_Debugv8p4 */
     t = FIELD_DP32(t, ID_DFR0, PERFMON, 5);       /* FEAT_PMUv3p4 */
     cpu->isar.id_dfr0 = t;
 }

From 58e93b48aa1b9f0e6de7e57f6f68b6dda7a8198a Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:31 -0500
Subject: [PATCH 208/935] target/arm: Add minimal RAS registers

Add only the system registers required to implement zero error
records.  This means that all values for ERRSELR are out of range,
which means that it and all of the indexed error record registers
need not be implemented.

Add the EL2 registers required for injecting virtual SError.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-14-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu.h    |  5 +++
 target/arm/helper.c | 84 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 89 insertions(+)

diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index ca01f909a8..a55980d66d 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -525,6 +525,11 @@ typedef struct CPUArchState {
         uint64_t tfsr_el[4]; /* tfsre0_el1 is index 0.  */
         uint64_t gcr_el1;
         uint64_t rgsr_el1;
+
+        /* Minimal RAS registers */
+        uint64_t disr_el1;
+        uint64_t vdisr_el2;
+        uint64_t vsesr_el2;
     } cp15;
 
     struct {
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 7b31c71980..37c5e42bc0 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -5980,6 +5980,87 @@ static const ARMCPRegInfo debug_lpae_cp_reginfo[] = {
       .access = PL0_R, .type = ARM_CP_CONST|ARM_CP_64BIT, .resetvalue = 0 },
 };
 
+/*
+ * Check for traps to RAS registers, which are controlled
+ * by HCR_EL2.TERR and SCR_EL3.TERR.
+ */
+static CPAccessResult access_terr(CPUARMState *env, const ARMCPRegInfo *ri,
+                                  bool isread)
+{
+    int el = arm_current_el(env);
+
+    if (el < 2 && (arm_hcr_el2_eff(env) & HCR_TERR)) {
+        return CP_ACCESS_TRAP_EL2;
+    }
+    if (el < 3 && (env->cp15.scr_el3 & SCR_TERR)) {
+        return CP_ACCESS_TRAP_EL3;
+    }
+    return CP_ACCESS_OK;
+}
+
+static uint64_t disr_read(CPUARMState *env, const ARMCPRegInfo *ri)
+{
+    int el = arm_current_el(env);
+
+    if (el < 2 && (arm_hcr_el2_eff(env) & HCR_AMO)) {
+        return env->cp15.vdisr_el2;
+    }
+    if (el < 3 && (env->cp15.scr_el3 & SCR_EA)) {
+        return 0; /* RAZ/WI */
+    }
+    return env->cp15.disr_el1;
+}
+
+static void disr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t val)
+{
+    int el = arm_current_el(env);
+
+    if (el < 2 && (arm_hcr_el2_eff(env) & HCR_AMO)) {
+        env->cp15.vdisr_el2 = val;
+        return;
+    }
+    if (el < 3 && (env->cp15.scr_el3 & SCR_EA)) {
+        return; /* RAZ/WI */
+    }
+    env->cp15.disr_el1 = val;
+}
+
+/*
+ * Minimal RAS implementation with no Error Records.
+ * Which means that all of the Error Record registers:
+ *   ERXADDR_EL1
+ *   ERXCTLR_EL1
+ *   ERXFR_EL1
+ *   ERXMISC0_EL1
+ *   ERXMISC1_EL1
+ *   ERXMISC2_EL1
+ *   ERXMISC3_EL1
+ *   ERXPFGCDN_EL1  (RASv1p1)
+ *   ERXPFGCTL_EL1  (RASv1p1)
+ *   ERXPFGF_EL1    (RASv1p1)
+ *   ERXSTATUS_EL1
+ * and
+ *   ERRSELR_EL1
+ * may generate UNDEFINED, which is the effect we get by not
+ * listing them at all.
+ */
+static const ARMCPRegInfo minimal_ras_reginfo[] = {
+    { .name = "DISR_EL1", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 1, .opc2 = 1,
+      .access = PL1_RW, .fieldoffset = offsetof(CPUARMState, cp15.disr_el1),
+      .readfn = disr_read, .writefn = disr_write, .raw_writefn = raw_write },
+    { .name = "ERRIDR_EL1", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 0, .crn = 5, .crm = 3, .opc2 = 0,
+      .access = PL1_R, .accessfn = access_terr,
+      .type = ARM_CP_CONST, .resetvalue = 0 },
+    { .name = "VDISR_EL2", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 4, .crn = 12, .crm = 1, .opc2 = 1,
+      .access = PL2_RW, .fieldoffset = offsetof(CPUARMState, cp15.vdisr_el2) },
+    { .name = "VSESR_EL2", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 4, .crn = 5, .crm = 2, .opc2 = 3,
+      .access = PL2_RW, .fieldoffset = offsetof(CPUARMState, cp15.vsesr_el2) },
+};
+
 /* Return the exception level to which exceptions should be taken
  * via SVEAccessTrap.  If an exception should be routed through
  * AArch64.AdvSIMDFPAccessTrap, return 0; fp_exception_el should
@@ -8217,6 +8298,9 @@ void register_cp_regs_for_features(ARMCPU *cpu)
     if (cpu_isar_feature(aa64_ssbs, cpu)) {
         define_one_arm_cp_reg(cpu, &ssbs_reginfo);
     }
+    if (cpu_isar_feature(any_ras, cpu)) {
+        define_arm_cp_regs(cpu, minimal_ras_reginfo);
+    }
 
     if (cpu_isar_feature(aa64_vh, cpu) ||
         cpu_isar_feature(aa64_debugv8p2, cpu)) {

From da3d8b13624246702f7b8b88e37ee525a2f39ad2 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:32 -0500
Subject: [PATCH 209/935] target/arm: Enable SCR and HCR bits for RAS

Enable writes to the TERR and TEA bits when RAS is enabled.
These bits are otherwise RES0.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-15-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 37c5e42bc0..b6faebf4a7 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -1755,6 +1755,9 @@ static void scr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value)
         }
         valid_mask &= ~SCR_NET;
 
+        if (cpu_isar_feature(aa64_ras, cpu)) {
+            valid_mask |= SCR_TERR;
+        }
         if (cpu_isar_feature(aa64_lor, cpu)) {
             valid_mask |= SCR_TLOR;
         }
@@ -1769,6 +1772,9 @@ static void scr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value)
         }
     } else {
         valid_mask &= ~(SCR_RW | SCR_ST);
+        if (cpu_isar_feature(aa32_ras, cpu)) {
+            valid_mask |= SCR_TERR;
+        }
     }
 
     if (!arm_feature(env, ARM_FEATURE_EL2)) {
@@ -5126,6 +5132,9 @@ static void do_hcr_write(CPUARMState *env, uint64_t value, uint64_t valid_mask)
         if (cpu_isar_feature(aa64_vh, cpu)) {
             valid_mask |= HCR_E2H;
         }
+        if (cpu_isar_feature(aa64_ras, cpu)) {
+            valid_mask |= HCR_TERR | HCR_TEA;
+        }
         if (cpu_isar_feature(aa64_lor, cpu)) {
             valid_mask |= HCR_TLOR;
         }

From 3c29632feba7724be447b621f3527136e5c32744 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:33 -0500
Subject: [PATCH 210/935] target/arm: Implement virtual SError exceptions

Virtual SError exceptions are raised by setting HCR_EL2.VSE,
and are routed to EL1 just like other virtual exceptions.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-16-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu.c       | 38 +++++++++++++++++++++++++++++++++++++-
 target/arm/cpu.h       |  2 ++
 target/arm/helper.c    | 40 +++++++++++++++++++++++++++++++++++++++-
 target/arm/internals.h |  8 ++++++++
 target/arm/syndrome.h  |  5 +++++
 5 files changed, 91 insertions(+), 2 deletions(-)

diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index 2667aaf28b..652a84cf84 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -85,7 +85,7 @@ static bool arm_cpu_has_work(CPUState *cs)
     return (cpu->power_state != PSCI_OFF)
         && cs->interrupt_request &
         (CPU_INTERRUPT_FIQ | CPU_INTERRUPT_HARD
-         | CPU_INTERRUPT_VFIQ | CPU_INTERRUPT_VIRQ
+         | CPU_INTERRUPT_VFIQ | CPU_INTERRUPT_VIRQ | CPU_INTERRUPT_VSERR
          | CPU_INTERRUPT_EXITTB);
 }
 
@@ -511,6 +511,12 @@ static inline bool arm_excp_unmasked(CPUState *cs, unsigned int excp_idx,
             return false;
         }
         return !(env->daif & PSTATE_I);
+    case EXCP_VSERR:
+        if (!(hcr_el2 & HCR_AMO) || (hcr_el2 & HCR_TGE)) {
+            /* VIRQs are only taken when hypervized.  */
+            return false;
+        }
+        return !(env->daif & PSTATE_A);
     default:
         g_assert_not_reached();
     }
@@ -632,6 +638,17 @@ static bool arm_cpu_exec_interrupt(CPUState *cs, int interrupt_request)
             goto found;
         }
     }
+    if (interrupt_request & CPU_INTERRUPT_VSERR) {
+        excp_idx = EXCP_VSERR;
+        target_el = 1;
+        if (arm_excp_unmasked(cs, excp_idx, target_el,
+                              cur_el, secure, hcr_el2)) {
+            /* Taking a virtual abort clears HCR_EL2.VSE */
+            env->cp15.hcr_el2 &= ~HCR_VSE;
+            cpu_reset_interrupt(cs, CPU_INTERRUPT_VSERR);
+            goto found;
+        }
+    }
     return false;
 
  found:
@@ -684,6 +701,25 @@ void arm_cpu_update_vfiq(ARMCPU *cpu)
     }
 }
 
+void arm_cpu_update_vserr(ARMCPU *cpu)
+{
+    /*
+     * Update the interrupt level for VSERR, which is the HCR_EL2.VSE bit.
+     */
+    CPUARMState *env = &cpu->env;
+    CPUState *cs = CPU(cpu);
+
+    bool new_state = env->cp15.hcr_el2 & HCR_VSE;
+
+    if (new_state != ((cs->interrupt_request & CPU_INTERRUPT_VSERR) != 0)) {
+        if (new_state) {
+            cpu_interrupt(cs, CPU_INTERRUPT_VSERR);
+        } else {
+            cpu_reset_interrupt(cs, CPU_INTERRUPT_VSERR);
+        }
+    }
+}
+
 #ifndef CONFIG_USER_ONLY
 static void arm_cpu_set_irq(void *opaque, int irq, int level)
 {
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index a55980d66d..aade9237bd 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -56,6 +56,7 @@
 #define EXCP_LSERR          21   /* v8M LSERR SecureFault */
 #define EXCP_UNALIGNED      22   /* v7M UNALIGNED UsageFault */
 #define EXCP_DIVBYZERO      23   /* v7M DIVBYZERO UsageFault */
+#define EXCP_VSERR          24
 /* NB: add new EXCP_ defines to the array in arm_log_exception() too */
 
 #define ARMV7M_EXCP_RESET   1
@@ -89,6 +90,7 @@ enum {
 #define CPU_INTERRUPT_FIQ   CPU_INTERRUPT_TGT_EXT_1
 #define CPU_INTERRUPT_VIRQ  CPU_INTERRUPT_TGT_EXT_2
 #define CPU_INTERRUPT_VFIQ  CPU_INTERRUPT_TGT_EXT_3
+#define CPU_INTERRUPT_VSERR CPU_INTERRUPT_TGT_INT_0
 
 /* The usual mapping for an AArch64 system register to its AArch32
  * counterpart is for the 32 bit world to have access to the lower
diff --git a/target/arm/helper.c b/target/arm/helper.c
index b6faebf4a7..4857d2dbb8 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -1863,7 +1863,12 @@ static uint64_t isr_read(CPUARMState *env, const ARMCPRegInfo *ri)
         }
     }
 
-    /* External aborts are not possible in QEMU so A bit is always clear */
+    if (hcr_el2 & HCR_AMO) {
+        if (cs->interrupt_request & CPU_INTERRUPT_VSERR) {
+            ret |= CPSR_A;
+        }
+    }
+
     return ret;
 }
 
@@ -5175,6 +5180,7 @@ static void do_hcr_write(CPUARMState *env, uint64_t value, uint64_t valid_mask)
     g_assert(qemu_mutex_iothread_locked());
     arm_cpu_update_virq(cpu);
     arm_cpu_update_vfiq(cpu);
+    arm_cpu_update_vserr(cpu);
 }
 
 static void hcr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value)
@@ -9331,6 +9337,7 @@ void arm_log_exception(CPUState *cs)
             [EXCP_LSERR] = "v8M LSERR UsageFault",
             [EXCP_UNALIGNED] = "v7M UNALIGNED UsageFault",
             [EXCP_DIVBYZERO] = "v7M DIVBYZERO UsageFault",
+            [EXCP_VSERR] = "Virtual SERR",
         };
 
         if (idx >= 0 && idx < ARRAY_SIZE(excnames)) {
@@ -9843,6 +9850,31 @@ static void arm_cpu_do_interrupt_aarch32(CPUState *cs)
         mask = CPSR_A | CPSR_I | CPSR_F;
         offset = 4;
         break;
+    case EXCP_VSERR:
+        {
+            /*
+             * Note that this is reported as a data abort, but the DFAR
+             * has an UNKNOWN value.  Construct the SError syndrome from
+             * AET and ExT fields.
+             */
+            ARMMMUFaultInfo fi = { .type = ARMFault_AsyncExternal, };
+
+            if (extended_addresses_enabled(env)) {
+                env->exception.fsr = arm_fi_to_lfsc(&fi);
+            } else {
+                env->exception.fsr = arm_fi_to_sfsc(&fi);
+            }
+            env->exception.fsr |= env->cp15.vsesr_el2 & 0xd000;
+            A32_BANKED_CURRENT_REG_SET(env, dfsr, env->exception.fsr);
+            qemu_log_mask(CPU_LOG_INT, "...with IFSR 0x%x\n",
+                          env->exception.fsr);
+
+            new_mode = ARM_CPU_MODE_ABT;
+            addr = 0x10;
+            mask = CPSR_A | CPSR_I;
+            offset = 8;
+        }
+        break;
     case EXCP_SMC:
         new_mode = ARM_CPU_MODE_MON;
         addr = 0x08;
@@ -10063,6 +10095,12 @@ static void arm_cpu_do_interrupt_aarch64(CPUState *cs)
     case EXCP_VFIQ:
         addr += 0x100;
         break;
+    case EXCP_VSERR:
+        addr += 0x180;
+        /* Construct the SError syndrome from IDS and ISS fields. */
+        env->exception.syndrome = syn_serror(env->cp15.vsesr_el2 & 0x1ffffff);
+        env->cp15.esr_el[new_el] = env->exception.syndrome;
+        break;
     default:
         cpu_abort(cs, "Unhandled exception 0x%x\n", cs->exception_index);
     }
diff --git a/target/arm/internals.h b/target/arm/internals.h
index c563b3735f..6ca0e95746 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -947,6 +947,14 @@ void arm_cpu_update_virq(ARMCPU *cpu);
  */
 void arm_cpu_update_vfiq(ARMCPU *cpu);
 
+/**
+ * arm_cpu_update_vserr: Update CPU_INTERRUPT_VSERR bit
+ *
+ * Update the CPU_INTERRUPT_VSERR bit in cs->interrupt_request,
+ * following a change to the HCR_EL2.VSE bit.
+ */
+void arm_cpu_update_vserr(ARMCPU *cpu);
+
 /**
  * arm_mmu_idx_el:
  * @env: The cpu environment
diff --git a/target/arm/syndrome.h b/target/arm/syndrome.h
index 8cde8e7243..0cb26dde7d 100644
--- a/target/arm/syndrome.h
+++ b/target/arm/syndrome.h
@@ -287,4 +287,9 @@ static inline uint32_t syn_pcalignment(void)
     return (EC_PCALIGNMENT << ARM_EL_EC_SHIFT) | ARM_EL_IL;
 }
 
+static inline uint32_t syn_serror(uint32_t extra)
+{
+    return (EC_SERROR << ARM_EL_EC_SHIFT) | ARM_EL_IL | extra;
+}
+
 #endif /* TARGET_ARM_SYNDROME_H */

From 13954587ea240c305d43cffd1adc8959146f43fd Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:34 -0500
Subject: [PATCH 211/935] target/arm: Implement ESB instruction

Check for and defer any pending virtual SError.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-17-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/a32.decode      | 16 ++++++++------
 target/arm/helper.h        |  1 +
 target/arm/op_helper.c     | 43 ++++++++++++++++++++++++++++++++++++++
 target/arm/t32.decode      | 18 ++++++++--------
 target/arm/translate-a64.c | 17 +++++++++++++++
 target/arm/translate.c     | 23 ++++++++++++++++++++
 6 files changed, 103 insertions(+), 15 deletions(-)

diff --git a/target/arm/a32.decode b/target/arm/a32.decode
index fcd8cd4f7d..f2ca480949 100644
--- a/target/arm/a32.decode
+++ b/target/arm/a32.decode
@@ -187,13 +187,17 @@ SMULTT           .... 0001 0110 .... 0000 .... 1110 ....      @rd0mn
 
 {
   {
-    YIELD        ---- 0011 0010 0000 1111 ---- 0000 0001
-    WFE          ---- 0011 0010 0000 1111 ---- 0000 0010
-    WFI          ---- 0011 0010 0000 1111 ---- 0000 0011
+    [
+      YIELD      ---- 0011 0010 0000 1111 ---- 0000 0001
+      WFE        ---- 0011 0010 0000 1111 ---- 0000 0010
+      WFI        ---- 0011 0010 0000 1111 ---- 0000 0011
 
-    # TODO: Implement SEV, SEVL; may help SMP performance.
-    # SEV        ---- 0011 0010 0000 1111 ---- 0000 0100
-    # SEVL       ---- 0011 0010 0000 1111 ---- 0000 0101
+      # TODO: Implement SEV, SEVL; may help SMP performance.
+      # SEV      ---- 0011 0010 0000 1111 ---- 0000 0100
+      # SEVL     ---- 0011 0010 0000 1111 ---- 0000 0101
+
+      ESB        ---- 0011 0010 0000 1111 ---- 0001 0000
+    ]
 
     # The canonical nop ends in 00000000, but the whole of the
     # rest of the space executes as nop if otherwise unsupported.
diff --git a/target/arm/helper.h b/target/arm/helper.h
index b463d9343b..b1334e0c42 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -54,6 +54,7 @@ DEF_HELPER_1(wfe, void, env)
 DEF_HELPER_1(yield, void, env)
 DEF_HELPER_1(pre_hvc, void, env)
 DEF_HELPER_2(pre_smc, void, env, i32)
+DEF_HELPER_1(vesb, void, env)
 
 DEF_HELPER_3(cpsr_write, void, env, i32, i32)
 DEF_HELPER_2(cpsr_write_eret, void, env, i32)
diff --git a/target/arm/op_helper.c b/target/arm/op_helper.c
index 76499ffa14..390b6578a8 100644
--- a/target/arm/op_helper.c
+++ b/target/arm/op_helper.c
@@ -960,3 +960,46 @@ void HELPER(probe_access)(CPUARMState *env, target_ulong ptr,
                      access_type, mmu_idx, ra);
     }
 }
+
+/*
+ * This function corresponds to AArch64.vESBOperation().
+ * Note that the AArch32 version is not functionally different.
+ */
+void HELPER(vesb)(CPUARMState *env)
+{
+    /*
+     * The EL2Enabled() check is done inside arm_hcr_el2_eff,
+     * and will return HCR_EL2.VSE == 0, so nothing happens.
+     */
+    uint64_t hcr = arm_hcr_el2_eff(env);
+    bool enabled = !(hcr & HCR_TGE) && (hcr & HCR_AMO);
+    bool pending = enabled && (hcr & HCR_VSE);
+    bool masked  = (env->daif & PSTATE_A);
+
+    /* If VSE pending and masked, defer the exception.  */
+    if (pending && masked) {
+        uint32_t syndrome;
+
+        if (arm_el_is_aa64(env, 1)) {
+            /* Copy across IDS and ISS from VSESR. */
+            syndrome = env->cp15.vsesr_el2 & 0x1ffffff;
+        } else {
+            ARMMMUFaultInfo fi = { .type = ARMFault_AsyncExternal };
+
+            if (extended_addresses_enabled(env)) {
+                syndrome = arm_fi_to_lfsc(&fi);
+            } else {
+                syndrome = arm_fi_to_sfsc(&fi);
+            }
+            /* Copy across AET and ExT from VSESR. */
+            syndrome |= env->cp15.vsesr_el2 & 0xd000;
+        }
+
+        /* Set VDISR_EL2.A along with the syndrome. */
+        env->cp15.vdisr_el2 = syndrome | (1u << 31);
+
+        /* Clear pending virtual SError */
+        env->cp15.hcr_el2 &= ~HCR_VSE;
+        cpu_reset_interrupt(env_cpu(env), CPU_INTERRUPT_VSERR);
+    }
+}
diff --git a/target/arm/t32.decode b/target/arm/t32.decode
index 78fadef9d6..f21ad0167a 100644
--- a/target/arm/t32.decode
+++ b/target/arm/t32.decode
@@ -364,17 +364,17 @@ CLZ              1111 1010 1011 ---- 1111 .... 1000 ....      @rdm
   [
     # Hints, and CPS
     {
-      YIELD      1111 0011 1010 1111 1000 0000 0000 0001
-      WFE        1111 0011 1010 1111 1000 0000 0000 0010
-      WFI        1111 0011 1010 1111 1000 0000 0000 0011
+      [
+        YIELD    1111 0011 1010 1111 1000 0000 0000 0001
+        WFE      1111 0011 1010 1111 1000 0000 0000 0010
+        WFI      1111 0011 1010 1111 1000 0000 0000 0011
 
-      # TODO: Implement SEV, SEVL; may help SMP performance.
-      # SEV      1111 0011 1010 1111 1000 0000 0000 0100
-      # SEVL     1111 0011 1010 1111 1000 0000 0000 0101
+        # TODO: Implement SEV, SEVL; may help SMP performance.
+        # SEV    1111 0011 1010 1111 1000 0000 0000 0100
+        # SEVL   1111 0011 1010 1111 1000 0000 0000 0101
 
-      # For M-profile minimal-RAS ESB can be a NOP, which is the
-      # default behaviour since it is in the hint space.
-      # ESB      1111 0011 1010 1111 1000 0000 0001 0000
+        ESB      1111 0011 1010 1111 1000 0000 0001 0000
+      ]
 
       # The canonical nop ends in 0000 0000, but the whole rest
       # of the space is "reserved hint, behaves as nop".
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index b80313670f..5a02e076b7 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -1454,6 +1454,23 @@ static void handle_hint(DisasContext *s, uint32_t insn,
             gen_helper_autib(cpu_X[17], cpu_env, cpu_X[17], cpu_X[16]);
         }
         break;
+    case 0b10000: /* ESB */
+        /* Without RAS, we must implement this as NOP. */
+        if (dc_isar_feature(aa64_ras, s)) {
+            /*
+             * QEMU does not have a source of physical SErrors,
+             * so we are only concerned with virtual SErrors.
+             * The pseudocode in the ARM for this case is
+             *   if PSTATE.EL IN {EL0, EL1} && EL2Enabled() then
+             *      AArch64.vESBOperation();
+             * Most of the condition can be evaluated at translation time.
+             * Test for EL2 present, and defer test for SEL2 to runtime.
+             */
+            if (s->current_el <= 1 && arm_dc_feature(s, ARM_FEATURE_EL2)) {
+                gen_helper_vesb(cpu_env);
+            }
+        }
+        break;
     case 0b11000: /* PACIAZ */
         if (s->pauth_active) {
             gen_helper_pacia(cpu_X[30], cpu_env, cpu_X[30],
diff --git a/target/arm/translate.c b/target/arm/translate.c
index 4e19191ed5..87a899d638 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -6239,6 +6239,29 @@ static bool trans_WFI(DisasContext *s, arg_WFI *a)
     return true;
 }
 
+static bool trans_ESB(DisasContext *s, arg_ESB *a)
+{
+    /*
+     * For M-profile, minimal-RAS ESB can be a NOP.
+     * Without RAS, we must implement this as NOP.
+     */
+    if (!arm_dc_feature(s, ARM_FEATURE_M) && dc_isar_feature(aa32_ras, s)) {
+        /*
+         * QEMU does not have a source of physical SErrors,
+         * so we are only concerned with virtual SErrors.
+         * The pseudocode in the ARM for this case is
+         *   if PSTATE.EL IN {EL0, EL1} && EL2Enabled() then
+         *      AArch32.vESBOperation();
+         * Most of the condition can be evaluated at translation time.
+         * Test for EL2 present, and defer test for SEL2 to runtime.
+         */
+        if (s->current_el <= 1 && arm_dc_feature(s, ARM_FEATURE_EL2)) {
+            gen_helper_vesb(cpu_env);
+        }
+    }
+    return true;
+}
+
 static bool trans_NOP(DisasContext *s, arg_NOP *a)
 {
     return true;

From e95c74c5e5522f6270092b788c3a96dfd8a93671 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:35 -0500
Subject: [PATCH 212/935] target/arm: Enable FEAT_RAS for -cpu max

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-18-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 docs/system/arm/emulation.rst | 1 +
 target/arm/cpu64.c            | 1 +
 target/arm/cpu_tcg.c          | 1 +
 3 files changed, 3 insertions(+)

diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst
index 0acac6347c..8110408000 100644
--- a/docs/system/arm/emulation.rst
+++ b/docs/system/arm/emulation.rst
@@ -42,6 +42,7 @@ the following architecture extensions:
 - FEAT_PMULL (PMULL, PMULL2 instructions)
 - FEAT_PMUv3p1 (PMU Extensions v3.1)
 - FEAT_PMUv3p4 (PMU Extensions v3.4)
+- FEAT_RAS (Reliability, availability, and serviceability)
 - FEAT_RDM (Advanced SIMD rounding double multiply accumulate instructions)
 - FEAT_RNG (Random number generator)
 - FEAT_SB (Speculation Barrier)
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 88d3cef93e..35881c74b2 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -744,6 +744,7 @@ static void aarch64_max_initfn(Object *obj)
     t = cpu->isar.id_aa64pfr0;
     t = FIELD_DP64(t, ID_AA64PFR0, FP, 1);        /* FEAT_FP16 */
     t = FIELD_DP64(t, ID_AA64PFR0, ADVSIMD, 1);   /* FEAT_FP16 */
+    t = FIELD_DP64(t, ID_AA64PFR0, RAS, 1);       /* FEAT_RAS */
     t = FIELD_DP64(t, ID_AA64PFR0, SVE, 1);
     t = FIELD_DP64(t, ID_AA64PFR0, SEL2, 1);      /* FEAT_SEL2 */
     t = FIELD_DP64(t, ID_AA64PFR0, DIT, 1);       /* FEAT_DIT */
diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
index 337598e949..c5cf7efe95 100644
--- a/target/arm/cpu_tcg.c
+++ b/target/arm/cpu_tcg.c
@@ -69,6 +69,7 @@ void aa32_max_features(ARMCPU *cpu)
 
     t = cpu->isar.id_pfr0;
     t = FIELD_DP32(t, ID_PFR0, DIT, 1);           /* FEAT_DIT */
+    t = FIELD_DP32(t, ID_PFR0, RAS, 1);           /* FEAT_RAS */
     cpu->isar.id_pfr0 = t;
 
     t = cpu->isar.id_pfr2;

From 880cd10e84775d3cca599bf872a9362b02322747 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:36 -0500
Subject: [PATCH 213/935] target/arm: Enable FEAT_IESB for -cpu max

This feature is AArch64 only, and applies to physical SErrors,
which QEMU does not implement, thus the feature is a nop.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-19-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 docs/system/arm/emulation.rst | 1 +
 target/arm/cpu64.c            | 1 +
 2 files changed, 2 insertions(+)

diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst
index 8110408000..b200012d89 100644
--- a/docs/system/arm/emulation.rst
+++ b/docs/system/arm/emulation.rst
@@ -25,6 +25,7 @@ the following architecture extensions:
 - FEAT_FlagM2 (Enhancements to flag manipulation instructions)
 - FEAT_HPDS (Hierarchical permission disables)
 - FEAT_I8MM (AArch64 Int8 matrix multiplication instructions)
+- FEAT_IESB (Implicit error synchronization event)
 - FEAT_JSCVT (JavaScript conversion instructions)
 - FEAT_LOR (Limited ordering regions)
 - FEAT_LPA (Large Physical Address space)
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 35881c74b2..10410619f9 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -781,6 +781,7 @@ static void aarch64_max_initfn(Object *obj)
     t = cpu->isar.id_aa64mmfr2;
     t = FIELD_DP64(t, ID_AA64MMFR2, CNP, 1);      /* FEAT_TTCNP */
     t = FIELD_DP64(t, ID_AA64MMFR2, UAO, 1);      /* FEAT_UAO */
+    t = FIELD_DP64(t, ID_AA64MMFR2, IESB, 1);     /* FEAT_IESB */
     t = FIELD_DP64(t, ID_AA64MMFR2, VARANGE, 1);  /* FEAT_LVA */
     t = FIELD_DP64(t, ID_AA64MMFR2, ST, 1);       /* FEAT_TTST */
     t = FIELD_DP64(t, ID_AA64MMFR2, TTL, 1);      /* FEAT_TTL */

From 74b17e16695b93261b1c3ffb73031d574aba2b8e Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:37 -0500
Subject: [PATCH 214/935] target/arm: Enable FEAT_CSV2 for -cpu max

This extension concerns branch speculation, which TCG does
not implement.  Thus we can trivially enable this feature.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-20-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 docs/system/arm/emulation.rst | 1 +
 target/arm/cpu64.c            | 1 +
 target/arm/cpu_tcg.c          | 1 +
 3 files changed, 3 insertions(+)

diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst
index b200012d89..b2a3e2a437 100644
--- a/docs/system/arm/emulation.rst
+++ b/docs/system/arm/emulation.rst
@@ -12,6 +12,7 @@ the following architecture extensions:
 - FEAT_BBM at level 2 (Translation table break-before-make levels)
 - FEAT_BF16 (AArch64 BFloat16 instructions)
 - FEAT_BTI (Branch Target Identification)
+- FEAT_CSV2 (Cache speculation variant 2)
 - FEAT_DIT (Data Independent Timing instructions)
 - FEAT_DPB (DC CVAP instruction)
 - FEAT_Debugv8p2 (Debug changes for v8.2)
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 10410619f9..25fe74f928 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -748,6 +748,7 @@ static void aarch64_max_initfn(Object *obj)
     t = FIELD_DP64(t, ID_AA64PFR0, SVE, 1);
     t = FIELD_DP64(t, ID_AA64PFR0, SEL2, 1);      /* FEAT_SEL2 */
     t = FIELD_DP64(t, ID_AA64PFR0, DIT, 1);       /* FEAT_DIT */
+    t = FIELD_DP64(t, ID_AA64PFR0, CSV2, 1);      /* FEAT_CSV2 */
     cpu->isar.id_aa64pfr0 = t;
 
     t = cpu->isar.id_aa64pfr1;
diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
index c5cf7efe95..762b961707 100644
--- a/target/arm/cpu_tcg.c
+++ b/target/arm/cpu_tcg.c
@@ -68,6 +68,7 @@ void aa32_max_features(ARMCPU *cpu)
     cpu->isar.id_mmfr4 = t;
 
     t = cpu->isar.id_pfr0;
+    t = FIELD_DP32(t, ID_PFR0, CSV2, 2);          /* FEAT_CVS2 */
     t = FIELD_DP32(t, ID_PFR0, DIT, 1);           /* FEAT_DIT */
     t = FIELD_DP32(t, ID_PFR0, RAS, 1);           /* FEAT_RAS */
     cpu->isar.id_pfr0 = t;

From 7cb1e61851332ea661d4ef6c1d958e3cdbffe2d8 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:38 -0500
Subject: [PATCH 215/935] target/arm: Enable FEAT_CSV2_2 for -cpu max

There is no branch prediction in TCG, therefore there is no
need to actually include the context number into the predictor.
Therefore all we need to do is add the state for SCXTNUM_ELx.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-21-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 docs/system/arm/emulation.rst |  3 ++
 target/arm/cpu.c              |  5 +++
 target/arm/cpu.h              | 16 +++++++++
 target/arm/cpu64.c            |  3 +-
 target/arm/helper.c           | 61 ++++++++++++++++++++++++++++++++++-
 5 files changed, 86 insertions(+), 2 deletions(-)

diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst
index b2a3e2a437..9765ee3eaf 100644
--- a/docs/system/arm/emulation.rst
+++ b/docs/system/arm/emulation.rst
@@ -13,6 +13,9 @@ the following architecture extensions:
 - FEAT_BF16 (AArch64 BFloat16 instructions)
 - FEAT_BTI (Branch Target Identification)
 - FEAT_CSV2 (Cache speculation variant 2)
+- FEAT_CSV2_1p1 (Cache speculation variant 2, version 1.1)
+- FEAT_CSV2_1p2 (Cache speculation variant 2, version 1.2)
+- FEAT_CSV2_2 (Cache speculation variant 2, version 2)
 - FEAT_DIT (Data Independent Timing instructions)
 - FEAT_DPB (DC CVAP instruction)
 - FEAT_Debugv8p2 (Debug changes for v8.2)
diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index 652a84cf84..59df597e05 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -230,6 +230,11 @@ static void arm_cpu_reset(DeviceState *dev)
              */
             env->cp15.gcr_el1 = 0x1ffff;
         }
+        /*
+         * Disable access to SCXTNUM_EL0 from CSV2_1p2.
+         * This is not yet exposed from the Linux kernel in any way.
+         */
+        env->cp15.sctlr_el[1] |= SCTLR_TSCXT;
 #else
         /* Reset into the highest available EL */
         if (arm_feature(env, ARM_FEATURE_EL3)) {
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index aade9237bd..18ca61e8e2 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -688,6 +688,8 @@ typedef struct CPUArchState {
         ARMPACKey apdb;
         ARMPACKey apga;
     } keys;
+
+    uint64_t scxtnum_el[4];
 #endif
 
 #if defined(CONFIG_USER_ONLY)
@@ -1211,6 +1213,7 @@ void pmu_init(ARMCPU *cpu);
 #define SCTLR_WXN     (1U << 19)
 #define SCTLR_ST      (1U << 20) /* up to ??, RAZ in v6 */
 #define SCTLR_UWXN    (1U << 20) /* v7 onward, AArch32 only */
+#define SCTLR_TSCXT   (1U << 20) /* FEAT_CSV2_1p2, AArch64 only */
 #define SCTLR_FI      (1U << 21) /* up to v7, v8 RES0 */
 #define SCTLR_IESB    (1U << 21) /* v8.2-IESB, AArch64 only */
 #define SCTLR_U       (1U << 22) /* up to v6, RAO in v7 */
@@ -4022,6 +4025,19 @@ static inline bool isar_feature_aa64_dit(const ARMISARegisters *id)
     return FIELD_EX64(id->id_aa64pfr0, ID_AA64PFR0, DIT) != 0;
 }
 
+static inline bool isar_feature_aa64_scxtnum(const ARMISARegisters *id)
+{
+    int key = FIELD_EX64(id->id_aa64pfr0, ID_AA64PFR0, CSV2);
+    if (key >= 2) {
+        return true;      /* FEAT_CSV2_2 */
+    }
+    if (key == 1) {
+        key = FIELD_EX64(id->id_aa64pfr1, ID_AA64PFR1, CSV2_FRAC);
+        return key >= 2;  /* FEAT_CSV2_1p2 */
+    }
+    return false;
+}
+
 static inline bool isar_feature_aa64_ssbs(const ARMISARegisters *id)
 {
     return FIELD_EX64(id->id_aa64pfr1, ID_AA64PFR1, SSBS) != 0;
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 25fe74f928..07b44a62be 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -748,7 +748,7 @@ static void aarch64_max_initfn(Object *obj)
     t = FIELD_DP64(t, ID_AA64PFR0, SVE, 1);
     t = FIELD_DP64(t, ID_AA64PFR0, SEL2, 1);      /* FEAT_SEL2 */
     t = FIELD_DP64(t, ID_AA64PFR0, DIT, 1);       /* FEAT_DIT */
-    t = FIELD_DP64(t, ID_AA64PFR0, CSV2, 1);      /* FEAT_CSV2 */
+    t = FIELD_DP64(t, ID_AA64PFR0, CSV2, 2);      /* FEAT_CSV2_2 */
     cpu->isar.id_aa64pfr0 = t;
 
     t = cpu->isar.id_aa64pfr1;
@@ -760,6 +760,7 @@ static void aarch64_max_initfn(Object *obj)
      * we do for EL2 with the virtualization=on property.
      */
     t = FIELD_DP64(t, ID_AA64PFR1, MTE, 3);       /* FEAT_MTE3 */
+    t = FIELD_DP64(t, ID_AA64PFR1, CSV2_FRAC, 0); /* FEAT_CSV2_2 */
     cpu->isar.id_aa64pfr1 = t;
 
     t = cpu->isar.id_aa64mmfr0;
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 4857d2dbb8..432bd81919 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -1770,6 +1770,9 @@ static void scr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value)
         if (cpu_isar_feature(aa64_mte, cpu)) {
             valid_mask |= SCR_ATA;
         }
+        if (cpu_isar_feature(aa64_scxtnum, cpu)) {
+            valid_mask |= SCR_ENSCXT;
+        }
     } else {
         valid_mask &= ~(SCR_RW | SCR_ST);
         if (cpu_isar_feature(aa32_ras, cpu)) {
@@ -5149,6 +5152,9 @@ static void do_hcr_write(CPUARMState *env, uint64_t value, uint64_t valid_mask)
         if (cpu_isar_feature(aa64_mte, cpu)) {
             valid_mask |= HCR_ATA | HCR_DCT | HCR_TID5;
         }
+        if (cpu_isar_feature(aa64_scxtnum, cpu)) {
+            valid_mask |= HCR_ENSCXT;
+        }
     }
 
     /* Clear RES0 bits.  */
@@ -5800,6 +5806,10 @@ static void define_arm_vh_e2h_redirects_aliases(ARMCPU *cpu)
         { K(3, 0,  5, 6, 0), K(3, 4,  5, 6, 0), K(3, 5, 5, 6, 0),
           "TFSR_EL1", "TFSR_EL2", "TFSR_EL12", isar_feature_aa64_mte },
 
+        { K(3, 0, 13, 0, 7), K(3, 4, 13, 0, 7), K(3, 5, 13, 0, 7),
+          "SCXTNUM_EL1", "SCXTNUM_EL2", "SCXTNUM_EL12",
+          isar_feature_aa64_scxtnum },
+
         /* TODO: ARMv8.2-SPE -- PMSCR_EL2 */
         /* TODO: ARMv8.4-Trace -- TRFCR_EL2 */
     };
@@ -7223,7 +7233,52 @@ static const ARMCPRegInfo mte_el0_cacheop_reginfo[] = {
     },
 };
 
-#endif
+static CPAccessResult access_scxtnum(CPUARMState *env, const ARMCPRegInfo *ri,
+                                     bool isread)
+{
+    uint64_t hcr = arm_hcr_el2_eff(env);
+    int el = arm_current_el(env);
+
+    if (el == 0 && !((hcr & HCR_E2H) && (hcr & HCR_TGE))) {
+        if (env->cp15.sctlr_el[1] & SCTLR_TSCXT) {
+            if (hcr & HCR_TGE) {
+                return CP_ACCESS_TRAP_EL2;
+            }
+            return CP_ACCESS_TRAP;
+        }
+    } else if (el < 2 && (env->cp15.sctlr_el[2] & SCTLR_TSCXT)) {
+        return CP_ACCESS_TRAP_EL2;
+    }
+    if (el < 2 && arm_is_el2_enabled(env) && !(hcr & HCR_ENSCXT)) {
+        return CP_ACCESS_TRAP_EL2;
+    }
+    if (el < 3
+        && arm_feature(env, ARM_FEATURE_EL3)
+        && !(env->cp15.scr_el3 & SCR_ENSCXT)) {
+        return CP_ACCESS_TRAP_EL3;
+    }
+    return CP_ACCESS_OK;
+}
+
+static const ARMCPRegInfo scxtnum_reginfo[] = {
+    { .name = "SCXTNUM_EL0", .state = ARM_CP_STATE_AA64,
+      .opc0 = 3, .opc1 = 3, .crn = 13, .crm = 0, .opc2 = 7,
+      .access = PL0_RW, .accessfn = access_scxtnum,
+      .fieldoffset = offsetof(CPUARMState, scxtnum_el[0]) },
+    { .name = "SCXTNUM_EL1", .state = ARM_CP_STATE_AA64,
+      .opc0 = 3, .opc1 = 0, .crn = 13, .crm = 0, .opc2 = 7,
+      .access = PL1_RW, .accessfn = access_scxtnum,
+      .fieldoffset = offsetof(CPUARMState, scxtnum_el[1]) },
+    { .name = "SCXTNUM_EL2", .state = ARM_CP_STATE_AA64,
+      .opc0 = 3, .opc1 = 4, .crn = 13, .crm = 0, .opc2 = 7,
+      .access = PL2_RW, .accessfn = access_scxtnum,
+      .fieldoffset = offsetof(CPUARMState, scxtnum_el[2]) },
+    { .name = "SCXTNUM_EL3", .state = ARM_CP_STATE_AA64,
+      .opc0 = 3, .opc1 = 6, .crn = 13, .crm = 0, .opc2 = 7,
+      .access = PL3_RW,
+      .fieldoffset = offsetof(CPUARMState, scxtnum_el[3]) },
+};
+#endif /* TARGET_AARCH64 */
 
 static CPAccessResult access_predinv(CPUARMState *env, const ARMCPRegInfo *ri,
                                      bool isread)
@@ -8365,6 +8420,10 @@ void register_cp_regs_for_features(ARMCPU *cpu)
         define_arm_cp_regs(cpu, mte_tco_ro_reginfo);
         define_arm_cp_regs(cpu, mte_el0_cacheop_reginfo);
     }
+
+    if (cpu_isar_feature(aa64_scxtnum, cpu)) {
+        define_arm_cp_regs(cpu, scxtnum_reginfo);
+    }
 #endif
 
     if (cpu_isar_feature(any_predinv, cpu)) {

From 3082b86b107ec7a26352bd18ada295ac1cc4faca Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:39 -0500
Subject: [PATCH 216/935] target/arm: Enable FEAT_CSV3 for -cpu max

This extension concerns cache speculation, which TCG does
not implement.  Thus we can trivially enable this feature.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-22-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 docs/system/arm/emulation.rst | 1 +
 target/arm/cpu64.c            | 1 +
 target/arm/cpu_tcg.c          | 1 +
 3 files changed, 3 insertions(+)

diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst
index 9765ee3eaf..48522b8e1c 100644
--- a/docs/system/arm/emulation.rst
+++ b/docs/system/arm/emulation.rst
@@ -16,6 +16,7 @@ the following architecture extensions:
 - FEAT_CSV2_1p1 (Cache speculation variant 2, version 1.1)
 - FEAT_CSV2_1p2 (Cache speculation variant 2, version 1.2)
 - FEAT_CSV2_2 (Cache speculation variant 2, version 2)
+- FEAT_CSV3 (Cache speculation variant 3)
 - FEAT_DIT (Data Independent Timing instructions)
 - FEAT_DPB (DC CVAP instruction)
 - FEAT_Debugv8p2 (Debug changes for v8.2)
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 07b44a62be..40f77defb5 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -749,6 +749,7 @@ static void aarch64_max_initfn(Object *obj)
     t = FIELD_DP64(t, ID_AA64PFR0, SEL2, 1);      /* FEAT_SEL2 */
     t = FIELD_DP64(t, ID_AA64PFR0, DIT, 1);       /* FEAT_DIT */
     t = FIELD_DP64(t, ID_AA64PFR0, CSV2, 2);      /* FEAT_CSV2_2 */
+    t = FIELD_DP64(t, ID_AA64PFR0, CSV3, 1);      /* FEAT_CSV3 */
     cpu->isar.id_aa64pfr0 = t;
 
     t = cpu->isar.id_aa64pfr1;
diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
index 762b961707..ea4eccddc3 100644
--- a/target/arm/cpu_tcg.c
+++ b/target/arm/cpu_tcg.c
@@ -74,6 +74,7 @@ void aa32_max_features(ARMCPU *cpu)
     cpu->isar.id_pfr0 = t;
 
     t = cpu->isar.id_pfr2;
+    t = FIELD_DP32(t, ID_PFR2, CSV3, 1);          /* FEAT_CSV3 */
     t = FIELD_DP32(t, ID_PFR2, SSBS, 1);          /* FEAT_SSBS */
     cpu->isar.id_pfr2 = t;
 

From 6d9650191ae301dc545dd9fd0727c57ec935503e Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:40 -0500
Subject: [PATCH 217/935] target/arm: Enable FEAT_DGH for -cpu max

This extension concerns not merging memory access, which TCG does
not implement.  Thus we can trivially enable this feature.
Add a comment to handle_hint for the DGH instruction, but no code.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-23-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 docs/system/arm/emulation.rst | 1 +
 target/arm/cpu64.c            | 1 +
 target/arm/translate-a64.c    | 1 +
 3 files changed, 3 insertions(+)

diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst
index 48522b8e1c..8ed466bf68 100644
--- a/docs/system/arm/emulation.rst
+++ b/docs/system/arm/emulation.rst
@@ -17,6 +17,7 @@ the following architecture extensions:
 - FEAT_CSV2_1p2 (Cache speculation variant 2, version 1.2)
 - FEAT_CSV2_2 (Cache speculation variant 2, version 2)
 - FEAT_CSV3 (Cache speculation variant 3)
+- FEAT_DGH (Data gathering hint)
 - FEAT_DIT (Data Independent Timing instructions)
 - FEAT_DPB (DC CVAP instruction)
 - FEAT_Debugv8p2 (Debug changes for v8.2)
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 40f77defb5..f55121060d 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -738,6 +738,7 @@ static void aarch64_max_initfn(Object *obj)
     t = FIELD_DP64(t, ID_AA64ISAR1, SB, 1);       /* FEAT_SB */
     t = FIELD_DP64(t, ID_AA64ISAR1, SPECRES, 1);  /* FEAT_SPECRES */
     t = FIELD_DP64(t, ID_AA64ISAR1, BF16, 1);     /* FEAT_BF16 */
+    t = FIELD_DP64(t, ID_AA64ISAR1, DGH, 1);      /* FEAT_DGH */
     t = FIELD_DP64(t, ID_AA64ISAR1, I8MM, 1);     /* FEAT_I8MM */
     cpu->isar.id_aa64isar1 = t;
 
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 5a02e076b7..6a27234a5c 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -1427,6 +1427,7 @@ static void handle_hint(DisasContext *s, uint32_t insn,
         break;
     case 0b00100: /* SEV */
     case 0b00101: /* SEVL */
+    case 0b00110: /* DGH */
         /* we treat all as NOP at least for now */
         break;
     case 0b00111: /* XPACLRI */

From 2f6283fc8e5219ce5f82ba60216ed5145a246470 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:41 -0500
Subject: [PATCH 218/935] target/arm: Define cortex-a76

Enable the a76 for virt and sbsa board use.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-24-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 docs/system/arm/virt.rst |  1 +
 hw/arm/sbsa-ref.c        |  1 +
 hw/arm/virt.c            |  1 +
 target/arm/cpu64.c       | 66 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 69 insertions(+)

diff --git a/docs/system/arm/virt.rst b/docs/system/arm/virt.rst
index 5fe045cbf0..3e264d85af 100644
--- a/docs/system/arm/virt.rst
+++ b/docs/system/arm/virt.rst
@@ -55,6 +55,7 @@ Supported guest CPU types:
 - ``cortex-a53`` (64-bit)
 - ``cortex-a57`` (64-bit)
 - ``cortex-a72`` (64-bit)
+- ``cortex-a76`` (64-bit)
 - ``a64fx`` (64-bit)
 - ``host`` (with KVM only)
 - ``max`` (same as ``host`` for KVM; best possible emulation with TCG)
diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c
index 2387401963..2ddde88f5e 100644
--- a/hw/arm/sbsa-ref.c
+++ b/hw/arm/sbsa-ref.c
@@ -145,6 +145,7 @@ static const int sbsa_ref_irqmap[] = {
 static const char * const valid_cpus[] = {
     ARM_CPU_TYPE_NAME("cortex-a57"),
     ARM_CPU_TYPE_NAME("cortex-a72"),
+    ARM_CPU_TYPE_NAME("cortex-a76"),
     ARM_CPU_TYPE_NAME("max"),
 };
 
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index f94278935f..12bc2318ec 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -202,6 +202,7 @@ static const char *valid_cpus[] = {
     ARM_CPU_TYPE_NAME("cortex-a53"),
     ARM_CPU_TYPE_NAME("cortex-a57"),
     ARM_CPU_TYPE_NAME("cortex-a72"),
+    ARM_CPU_TYPE_NAME("cortex-a76"),
     ARM_CPU_TYPE_NAME("a64fx"),
     ARM_CPU_TYPE_NAME("host"),
     ARM_CPU_TYPE_NAME("max"),
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index f55121060d..adfe6b26be 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -194,6 +194,71 @@ static void aarch64_a72_initfn(Object *obj)
     define_cortex_a72_a57_a53_cp_reginfo(cpu);
 }
 
+static void aarch64_a76_initfn(Object *obj)
+{
+    ARMCPU *cpu = ARM_CPU(obj);
+
+    cpu->dtb_compatible = "arm,cortex-a76";
+    set_feature(&cpu->env, ARM_FEATURE_V8);
+    set_feature(&cpu->env, ARM_FEATURE_NEON);
+    set_feature(&cpu->env, ARM_FEATURE_GENERIC_TIMER);
+    set_feature(&cpu->env, ARM_FEATURE_AARCH64);
+    set_feature(&cpu->env, ARM_FEATURE_CBAR_RO);
+    set_feature(&cpu->env, ARM_FEATURE_EL2);
+    set_feature(&cpu->env, ARM_FEATURE_EL3);
+    set_feature(&cpu->env, ARM_FEATURE_PMU);
+
+    /* Ordered by B2.4 AArch64 registers by functional group */
+    cpu->clidr = 0x82000023;
+    cpu->ctr = 0x8444C004;
+    cpu->dcz_blocksize = 4;
+    cpu->isar.id_aa64dfr0  = 0x0000000010305408ull;
+    cpu->isar.id_aa64isar0 = 0x0000100010211120ull;
+    cpu->isar.id_aa64isar1 = 0x0000000000100001ull;
+    cpu->isar.id_aa64mmfr0 = 0x0000000000101122ull;
+    cpu->isar.id_aa64mmfr1 = 0x0000000010212122ull;
+    cpu->isar.id_aa64mmfr2 = 0x0000000000001011ull;
+    cpu->isar.id_aa64pfr0  = 0x1100000010111112ull; /* GIC filled in later */
+    cpu->isar.id_aa64pfr1  = 0x0000000000000010ull;
+    cpu->id_afr0       = 0x00000000;
+    cpu->isar.id_dfr0  = 0x04010088;
+    cpu->isar.id_isar0 = 0x02101110;
+    cpu->isar.id_isar1 = 0x13112111;
+    cpu->isar.id_isar2 = 0x21232042;
+    cpu->isar.id_isar3 = 0x01112131;
+    cpu->isar.id_isar4 = 0x00010142;
+    cpu->isar.id_isar5 = 0x01011121;
+    cpu->isar.id_isar6 = 0x00000010;
+    cpu->isar.id_mmfr0 = 0x10201105;
+    cpu->isar.id_mmfr1 = 0x40000000;
+    cpu->isar.id_mmfr2 = 0x01260000;
+    cpu->isar.id_mmfr3 = 0x02122211;
+    cpu->isar.id_mmfr4 = 0x00021110;
+    cpu->isar.id_pfr0  = 0x10010131;
+    cpu->isar.id_pfr1  = 0x00010000; /* GIC filled in later */
+    cpu->isar.id_pfr2  = 0x00000011;
+    cpu->midr = 0x414fd0b1;          /* r4p1 */
+    cpu->revidr = 0;
+
+    /* From B2.18 CCSIDR_EL1 */
+    cpu->ccsidr[0] = 0x701fe01a; /* 64KB L1 dcache */
+    cpu->ccsidr[1] = 0x201fe01a; /* 64KB L1 icache */
+    cpu->ccsidr[2] = 0x707fe03a; /* 512KB L2 cache */
+
+    /* From B2.93 SCTLR_EL3 */
+    cpu->reset_sctlr = 0x30c50838;
+
+    /* From B4.23 ICH_VTR_EL2 */
+    cpu->gic_num_lrs = 4;
+    cpu->gic_vpribits = 5;
+    cpu->gic_vprebits = 5;
+
+    /* From B5.1 AdvSIMD AArch64 register summary */
+    cpu->isar.mvfr0 = 0x10110222;
+    cpu->isar.mvfr1 = 0x13211111;
+    cpu->isar.mvfr2 = 0x00000043;
+}
+
 void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp)
 {
     /*
@@ -881,6 +946,7 @@ static const ARMCPUInfo aarch64_cpus[] = {
     { .name = "cortex-a57",         .initfn = aarch64_a57_initfn },
     { .name = "cortex-a53",         .initfn = aarch64_a53_initfn },
     { .name = "cortex-a72",         .initfn = aarch64_a72_initfn },
+    { .name = "cortex-a76",         .initfn = aarch64_a76_initfn },
     { .name = "a64fx",              .initfn = aarch64_a64fx_initfn },
     { .name = "max",                .initfn = aarch64_max_initfn },
 #if defined(CONFIG_KVM) || defined(CONFIG_HVF)

From 5db6de806ab6f6db457300fc527f9b367fd97f21 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 6 May 2022 13:02:42 -0500
Subject: [PATCH 219/935] target/arm: Define neoverse-n1

Enable the n1 for virt and sbsa board use.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220506180242.216785-25-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 docs/system/arm/virt.rst |  1 +
 hw/arm/sbsa-ref.c        |  1 +
 hw/arm/virt.c            |  1 +
 target/arm/cpu64.c       | 66 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 69 insertions(+)

diff --git a/docs/system/arm/virt.rst b/docs/system/arm/virt.rst
index 3e264d85af..3d1058a80c 100644
--- a/docs/system/arm/virt.rst
+++ b/docs/system/arm/virt.rst
@@ -58,6 +58,7 @@ Supported guest CPU types:
 - ``cortex-a76`` (64-bit)
 - ``a64fx`` (64-bit)
 - ``host`` (with KVM only)
+- ``neoverse-n1`` (64-bit)
 - ``max`` (same as ``host`` for KVM; best possible emulation with TCG)
 
 Note that the default is ``cortex-a15``, so for an AArch64 guest you must
diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c
index 2ddde88f5e..dac8860f2d 100644
--- a/hw/arm/sbsa-ref.c
+++ b/hw/arm/sbsa-ref.c
@@ -146,6 +146,7 @@ static const char * const valid_cpus[] = {
     ARM_CPU_TYPE_NAME("cortex-a57"),
     ARM_CPU_TYPE_NAME("cortex-a72"),
     ARM_CPU_TYPE_NAME("cortex-a76"),
+    ARM_CPU_TYPE_NAME("neoverse-n1"),
     ARM_CPU_TYPE_NAME("max"),
 };
 
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 12bc2318ec..da7e3ede56 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -204,6 +204,7 @@ static const char *valid_cpus[] = {
     ARM_CPU_TYPE_NAME("cortex-a72"),
     ARM_CPU_TYPE_NAME("cortex-a76"),
     ARM_CPU_TYPE_NAME("a64fx"),
+    ARM_CPU_TYPE_NAME("neoverse-n1"),
     ARM_CPU_TYPE_NAME("host"),
     ARM_CPU_TYPE_NAME("max"),
 };
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index adfe6b26be..04427e073f 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -259,6 +259,71 @@ static void aarch64_a76_initfn(Object *obj)
     cpu->isar.mvfr2 = 0x00000043;
 }
 
+static void aarch64_neoverse_n1_initfn(Object *obj)
+{
+    ARMCPU *cpu = ARM_CPU(obj);
+
+    cpu->dtb_compatible = "arm,neoverse-n1";
+    set_feature(&cpu->env, ARM_FEATURE_V8);
+    set_feature(&cpu->env, ARM_FEATURE_NEON);
+    set_feature(&cpu->env, ARM_FEATURE_GENERIC_TIMER);
+    set_feature(&cpu->env, ARM_FEATURE_AARCH64);
+    set_feature(&cpu->env, ARM_FEATURE_CBAR_RO);
+    set_feature(&cpu->env, ARM_FEATURE_EL2);
+    set_feature(&cpu->env, ARM_FEATURE_EL3);
+    set_feature(&cpu->env, ARM_FEATURE_PMU);
+
+    /* Ordered by B2.4 AArch64 registers by functional group */
+    cpu->clidr = 0x82000023;
+    cpu->ctr = 0x8444c004;
+    cpu->dcz_blocksize = 4;
+    cpu->isar.id_aa64dfr0  = 0x0000000110305408ull;
+    cpu->isar.id_aa64isar0 = 0x0000100010211120ull;
+    cpu->isar.id_aa64isar1 = 0x0000000000100001ull;
+    cpu->isar.id_aa64mmfr0 = 0x0000000000101125ull;
+    cpu->isar.id_aa64mmfr1 = 0x0000000010212122ull;
+    cpu->isar.id_aa64mmfr2 = 0x0000000000001011ull;
+    cpu->isar.id_aa64pfr0  = 0x1100000010111112ull; /* GIC filled in later */
+    cpu->isar.id_aa64pfr1  = 0x0000000000000020ull;
+    cpu->id_afr0       = 0x00000000;
+    cpu->isar.id_dfr0  = 0x04010088;
+    cpu->isar.id_isar0 = 0x02101110;
+    cpu->isar.id_isar1 = 0x13112111;
+    cpu->isar.id_isar2 = 0x21232042;
+    cpu->isar.id_isar3 = 0x01112131;
+    cpu->isar.id_isar4 = 0x00010142;
+    cpu->isar.id_isar5 = 0x01011121;
+    cpu->isar.id_isar6 = 0x00000010;
+    cpu->isar.id_mmfr0 = 0x10201105;
+    cpu->isar.id_mmfr1 = 0x40000000;
+    cpu->isar.id_mmfr2 = 0x01260000;
+    cpu->isar.id_mmfr3 = 0x02122211;
+    cpu->isar.id_mmfr4 = 0x00021110;
+    cpu->isar.id_pfr0  = 0x10010131;
+    cpu->isar.id_pfr1  = 0x00010000; /* GIC filled in later */
+    cpu->isar.id_pfr2  = 0x00000011;
+    cpu->midr = 0x414fd0c1;          /* r4p1 */
+    cpu->revidr = 0;
+
+    /* From B2.23 CCSIDR_EL1 */
+    cpu->ccsidr[0] = 0x701fe01a; /* 64KB L1 dcache */
+    cpu->ccsidr[1] = 0x201fe01a; /* 64KB L1 icache */
+    cpu->ccsidr[2] = 0x70ffe03a; /* 1MB L2 cache */
+
+    /* From B2.98 SCTLR_EL3 */
+    cpu->reset_sctlr = 0x30c50838;
+
+    /* From B4.23 ICH_VTR_EL2 */
+    cpu->gic_num_lrs = 4;
+    cpu->gic_vpribits = 5;
+    cpu->gic_vprebits = 5;
+
+    /* From B5.1 AdvSIMD AArch64 register summary */
+    cpu->isar.mvfr0 = 0x10110222;
+    cpu->isar.mvfr1 = 0x13211111;
+    cpu->isar.mvfr2 = 0x00000043;
+}
+
 void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp)
 {
     /*
@@ -948,6 +1013,7 @@ static const ARMCPUInfo aarch64_cpus[] = {
     { .name = "cortex-a72",         .initfn = aarch64_a72_initfn },
     { .name = "cortex-a76",         .initfn = aarch64_a76_initfn },
     { .name = "a64fx",              .initfn = aarch64_a64fx_initfn },
+    { .name = "neoverse-n1",        .initfn = aarch64_neoverse_n1_initfn },
     { .name = "max",                .initfn = aarch64_max_initfn },
 #if defined(CONFIG_KVM) || defined(CONFIG_HVF)
     { .name = "host",               .initfn = aarch64_host_initfn },

From 90ea2cceb286c66b8460725f5573d8a157e65bdf Mon Sep 17 00:00:00 2001
From: Leif Lindholm <quic_llindhol@quicinc.com>
Date: Thu, 5 May 2022 12:39:47 +0100
Subject: [PATCH 220/935] hw/arm: add versioning to sbsa-ref machine DT
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The sbsa-ref machine is continuously evolving. Some of the changes we
want to make in the near future, to align with real components (e.g.
the GIC-700), will break compatibility for existing firmware.

Introduce two new properties to the DT generated on machine generation:
- machine-version-major
  To be incremented when a platform change makes the machine
  incompatible with existing firmware.
- machine-version-minor
  To be incremented when functionality is added to the machine
  without causing incompatibility with existing firmware.
  to be reset to 0 when machine-version-major is incremented.

This versioning scheme is *neither*:
- A QEMU versioned machine type; a given version of QEMU will emulate
  a given version of the platform.
- A reflection of level of SBSA (now SystemReady SR) support provided.

The version will increment on guest-visible functional changes only,
akin to a revision ID register found on a physical platform.

These properties are both introduced with the value 0.
(Hence, a machine where the DT is lacking these nodes is equivalent
to version 0.0.)

Signed-off-by: Leif Lindholm <quic_llindhol@quicinc.com>
Message-id: 20220505113947.75714-1-quic_llindhol@quicinc.com
Cc: Peter Maydell <peter.maydell@linaro.org>
Cc: Radoslaw Biernacki <rad@semihalf.com>
Cc: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/arm/sbsa-ref.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c
index dac8860f2d..4bb444684f 100644
--- a/hw/arm/sbsa-ref.c
+++ b/hw/arm/sbsa-ref.c
@@ -192,6 +192,20 @@ static void create_fdt(SBSAMachineState *sms)
     qemu_fdt_setprop_cell(fdt, "/", "#address-cells", 0x2);
     qemu_fdt_setprop_cell(fdt, "/", "#size-cells", 0x2);
 
+    /*
+     * This versioning scheme is for informing platform fw only. It is neither:
+     * - A QEMU versioned machine type; a given version of QEMU will emulate
+     *   a given version of the platform.
+     * - A reflection of level of SBSA (now SystemReady SR) support provided.
+     *
+     * machine-version-major: updated when changes breaking fw compatibility
+     *                        are introduced.
+     * machine-version-minor: updated when features are added that don't break
+     *                        fw compatibility.
+     */
+    qemu_fdt_setprop_cell(fdt, "/", "machine-version-major", 0);
+    qemu_fdt_setprop_cell(fdt, "/", "machine-version-minor", 0);
+
     if (ms->numa_state->have_numa_distance) {
         int size = nb_numa_nodes * nb_numa_nodes * 3 * sizeof(uint32_t);
         uint32_t *matrix = g_malloc0(size);

From 1dcf7001d4bae651129d46d5628b29e93a411d0b Mon Sep 17 00:00:00 2001
From: Gavin Shan <gshan@redhat.com>
Date: Tue, 3 May 2022 22:02:59 +0800
Subject: [PATCH 221/935] qapi/machine.json: Add cluster-id

This adds cluster-id in CPU instance properties, which will be used
by arm/virt machine. Besides, the cluster-id is also verified or
dumped in various spots:

  * hw/core/machine.c::machine_set_cpu_numa_node() to associate
    CPU with its NUMA node.

  * hw/core/machine.c::machine_numa_finish_cpu_init() to record
    CPU slots with no NUMA mapping set.

  * hw/core/machine-hmp-cmds.c::hmp_hotpluggable_cpus() to dump
    cluster-id.

Signed-off-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Yanan Wang <wangyanan55@huawei.com>
Acked-by: Igor Mammedov <imammedo@redhat.com>
Message-id: 20220503140304.855514-2-gshan@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/core/machine-hmp-cmds.c |  4 ++++
 hw/core/machine.c          | 16 ++++++++++++++++
 qapi/machine.json          |  6 ++++--
 3 files changed, 24 insertions(+), 2 deletions(-)

diff --git a/hw/core/machine-hmp-cmds.c b/hw/core/machine-hmp-cmds.c
index 4e2f319aeb..5cb5eecbfc 100644
--- a/hw/core/machine-hmp-cmds.c
+++ b/hw/core/machine-hmp-cmds.c
@@ -77,6 +77,10 @@ void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict)
         if (c->has_die_id) {
             monitor_printf(mon, "    die-id: \"%" PRIu64 "\"\n", c->die_id);
         }
+        if (c->has_cluster_id) {
+            monitor_printf(mon, "    cluster-id: \"%" PRIu64 "\"\n",
+                           c->cluster_id);
+        }
         if (c->has_core_id) {
             monitor_printf(mon, "    core-id: \"%" PRIu64 "\"\n", c->core_id);
         }
diff --git a/hw/core/machine.c b/hw/core/machine.c
index cb9bbc844d..700c1e76b8 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -682,6 +682,11 @@ void machine_set_cpu_numa_node(MachineState *machine,
             return;
         }
 
+        if (props->has_cluster_id && !slot->props.has_cluster_id) {
+            error_setg(errp, "cluster-id is not supported");
+            return;
+        }
+
         if (props->has_socket_id && !slot->props.has_socket_id) {
             error_setg(errp, "socket-id is not supported");
             return;
@@ -701,6 +706,11 @@ void machine_set_cpu_numa_node(MachineState *machine,
                 continue;
         }
 
+        if (props->has_cluster_id &&
+            props->cluster_id != slot->props.cluster_id) {
+                continue;
+        }
+
         if (props->has_die_id && props->die_id != slot->props.die_id) {
                 continue;
         }
@@ -995,6 +1005,12 @@ static char *cpu_slot_to_string(const CPUArchId *cpu)
         }
         g_string_append_printf(s, "die-id: %"PRId64, cpu->props.die_id);
     }
+    if (cpu->props.has_cluster_id) {
+        if (s->len) {
+            g_string_append_printf(s, ", ");
+        }
+        g_string_append_printf(s, "cluster-id: %"PRId64, cpu->props.cluster_id);
+    }
     if (cpu->props.has_core_id) {
         if (s->len) {
             g_string_append_printf(s, ", ");
diff --git a/qapi/machine.json b/qapi/machine.json
index d25a481ce4..4c417e32a5 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -868,10 +868,11 @@
 # @node-id: NUMA node ID the CPU belongs to
 # @socket-id: socket number within node/board the CPU belongs to
 # @die-id: die number within socket the CPU belongs to (since 4.1)
-# @core-id: core number within die the CPU belongs to
+# @cluster-id: cluster number within die the CPU belongs to (since 7.1)
+# @core-id: core number within cluster the CPU belongs to
 # @thread-id: thread number within core the CPU belongs to
 #
-# Note: currently there are 5 properties that could be present
+# Note: currently there are 6 properties that could be present
 #       but management should be prepared to pass through other
 #       properties with device_add command to allow for future
 #       interface extension. This also requires the filed names to be kept in
@@ -883,6 +884,7 @@
   'data': { '*node-id': 'int',
             '*socket-id': 'int',
             '*die-id': 'int',
+            '*cluster-id': 'int',
             '*core-id': 'int',
             '*thread-id': 'int'
   }

From ac7199a2523ce2ccf8e685087a5d177eeca89b09 Mon Sep 17 00:00:00 2001
From: Gavin Shan <gshan@redhat.com>
Date: Tue, 3 May 2022 22:03:00 +0800
Subject: [PATCH 222/935] qtest/numa-test: Specify CPU topology in
 aarch64_numa_cpu()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The CPU topology isn't enabled on arm/virt machine yet, but we're
going to do it in next patch. After the CPU topology is enabled by
next patch, "thread-id=1" becomes invalid because the CPU core is
preferred on arm/virt machine. It means these two CPUs have 0/1
as their core IDs, but their thread IDs are all 0. It will trigger
test failure as the following message indicates:

  [14/21 qemu:qtest+qtest-aarch64 / qtest-aarch64/numa-test  ERROR
  1.48s   killed by signal 6 SIGABRT
  >>> G_TEST_DBUS_DAEMON=/home/gavin/sandbox/qemu.main/tests/dbus-vmstate-daemon.sh \
      QTEST_QEMU_STORAGE_DAEMON_BINARY=./storage-daemon/qemu-storage-daemon         \
      QTEST_QEMU_BINARY=./qemu-system-aarch64                                       \
      QTEST_QEMU_IMG=./qemu-img MALLOC_PERTURB_=83                                  \
      /home/gavin/sandbox/qemu.main/build/tests/qtest/numa-test --tap -k
  ――――――――――――――――――――――――――――――――――――――――――――――
  stderr:
  qemu-system-aarch64: -numa cpu,node-id=0,thread-id=1: no match found

This fixes the issue by providing comprehensive SMP configurations
in aarch64_numa_cpu(). The SMP configurations aren't used before
the CPU topology is enabled in next patch.

Signed-off-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Yanan Wang <wangyanan55@huawei.com>
Message-id: 20220503140304.855514-3-gshan@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 tests/qtest/numa-test.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/qtest/numa-test.c b/tests/qtest/numa-test.c
index 749429dd27..976526e527 100644
--- a/tests/qtest/numa-test.c
+++ b/tests/qtest/numa-test.c
@@ -223,7 +223,8 @@ static void aarch64_numa_cpu(const void *data)
     QTestState *qts;
     g_autofree char *cli = NULL;
 
-    cli = make_cli(data, "-machine smp.cpus=2 "
+    cli = make_cli(data, "-machine "
+        "smp.cpus=2,smp.sockets=1,smp.clusters=1,smp.cores=1,smp.threads=2 "
         "-numa node,nodeid=0,memdev=ram -numa node,nodeid=1 "
         "-numa cpu,node-id=1,thread-id=0 "
         "-numa cpu,node-id=0,thread-id=1");

From c9ec4cb5e4936f980889e717524e73896b0200ed Mon Sep 17 00:00:00 2001
From: Gavin Shan <gshan@redhat.com>
Date: Tue, 3 May 2022 22:03:01 +0800
Subject: [PATCH 223/935] hw/arm/virt: Consider SMP configuration in CPU
 topology

Currently, the SMP configuration isn't considered when the CPU
topology is populated. In this case, it's impossible to provide
the default CPU-to-NUMA mapping or association based on the socket
ID of the given CPU.

This takes account of SMP configuration when the CPU topology
is populated. The die ID for the given CPU isn't assigned since
it's not supported on arm/virt machine. Besides, the used SMP
configuration in qtest/numa-test/aarch64_numa_cpu() is corrcted
to avoid testing failure

Signed-off-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Yanan Wang <wangyanan55@huawei.com>
Acked-by: Igor Mammedov <imammedo@redhat.com>
Message-id: 20220503140304.855514-4-gshan@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/arm/virt.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index da7e3ede56..c25023a083 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -2562,6 +2562,7 @@ static const CPUArchIdList *virt_possible_cpu_arch_ids(MachineState *ms)
     int n;
     unsigned int max_cpus = ms->smp.max_cpus;
     VirtMachineState *vms = VIRT_MACHINE(ms);
+    MachineClass *mc = MACHINE_GET_CLASS(vms);
 
     if (ms->possible_cpus) {
         assert(ms->possible_cpus->len == max_cpus);
@@ -2575,8 +2576,20 @@ static const CPUArchIdList *virt_possible_cpu_arch_ids(MachineState *ms)
         ms->possible_cpus->cpus[n].type = ms->cpu_type;
         ms->possible_cpus->cpus[n].arch_id =
             virt_cpu_mp_affinity(vms, n);
+
+        assert(!mc->smp_props.dies_supported);
+        ms->possible_cpus->cpus[n].props.has_socket_id = true;
+        ms->possible_cpus->cpus[n].props.socket_id =
+            n / (ms->smp.clusters * ms->smp.cores * ms->smp.threads);
+        ms->possible_cpus->cpus[n].props.has_cluster_id = true;
+        ms->possible_cpus->cpus[n].props.cluster_id =
+            (n / (ms->smp.cores * ms->smp.threads)) % ms->smp.clusters;
+        ms->possible_cpus->cpus[n].props.has_core_id = true;
+        ms->possible_cpus->cpus[n].props.core_id =
+            (n / ms->smp.threads) % ms->smp.cores;
         ms->possible_cpus->cpus[n].props.has_thread_id = true;
-        ms->possible_cpus->cpus[n].props.thread_id = n;
+        ms->possible_cpus->cpus[n].props.thread_id =
+            n % ms->smp.threads;
     }
     return ms->possible_cpus;
 }

From e280ecb39bc1629f74ea5479d464fd1608dc8f76 Mon Sep 17 00:00:00 2001
From: Gavin Shan <gshan@redhat.com>
Date: Tue, 3 May 2022 22:03:02 +0800
Subject: [PATCH 224/935] qtest/numa-test: Correct CPU and NUMA association in
 aarch64_numa_cpu()

In aarch64_numa_cpu(), the CPU and NUMA association is something
like below. Two threads in the same core/cluster/socket are
associated with two individual NUMA nodes, which is unreal as
Igor Mammedov mentioned. We don't expect the association to break
NUMA-to-socket boundary, which matches with the real world.

    NUMA-node  socket  cluster   core   thread
    ------------------------------------------
        0       0        0        0      0
        1       0        0        0      1

This corrects the topology for CPUs and their association with
NUMA nodes. After this patch is applied, the CPU and NUMA
association becomes something like below, which looks real.
Besides, socket/cluster/core/thread IDs are all checked when
the NUMA node IDs are verified. It helps to check if the CPU
topology is properly populated or not.

    NUMA-node  socket  cluster   core   thread
    ------------------------------------------
       0        1        0        0       0
       1        0        0        0       0

Suggested-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Gavin Shan <gshan@redhat.com>
Acked-by: Igor Mammedov <imammedo@redhat.com>
Message-id: 20220503140304.855514-5-gshan@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 tests/qtest/numa-test.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/tests/qtest/numa-test.c b/tests/qtest/numa-test.c
index 976526e527..c5eb13f349 100644
--- a/tests/qtest/numa-test.c
+++ b/tests/qtest/numa-test.c
@@ -224,17 +224,17 @@ static void aarch64_numa_cpu(const void *data)
     g_autofree char *cli = NULL;
 
     cli = make_cli(data, "-machine "
-        "smp.cpus=2,smp.sockets=1,smp.clusters=1,smp.cores=1,smp.threads=2 "
+        "smp.cpus=2,smp.sockets=2,smp.clusters=1,smp.cores=1,smp.threads=1 "
         "-numa node,nodeid=0,memdev=ram -numa node,nodeid=1 "
-        "-numa cpu,node-id=1,thread-id=0 "
-        "-numa cpu,node-id=0,thread-id=1");
+        "-numa cpu,node-id=0,socket-id=1,cluster-id=0,core-id=0,thread-id=0 "
+        "-numa cpu,node-id=1,socket-id=0,cluster-id=0,core-id=0,thread-id=0");
     qts = qtest_init(cli);
     cpus = get_cpus(qts, &resp);
     g_assert(cpus);
 
     while ((e = qlist_pop(cpus))) {
         QDict *cpu, *props;
-        int64_t thread, node;
+        int64_t socket, cluster, core, thread, node;
 
         cpu = qobject_to(QDict, e);
         g_assert(qdict_haskey(cpu, "props"));
@@ -242,12 +242,18 @@ static void aarch64_numa_cpu(const void *data)
 
         g_assert(qdict_haskey(props, "node-id"));
         node = qdict_get_int(props, "node-id");
+        g_assert(qdict_haskey(props, "socket-id"));
+        socket = qdict_get_int(props, "socket-id");
+        g_assert(qdict_haskey(props, "cluster-id"));
+        cluster = qdict_get_int(props, "cluster-id");
+        g_assert(qdict_haskey(props, "core-id"));
+        core = qdict_get_int(props, "core-id");
         g_assert(qdict_haskey(props, "thread-id"));
         thread = qdict_get_int(props, "thread-id");
 
-        if (thread == 0) {
+        if (socket == 0 && cluster == 0 && core == 0 && thread == 0) {
             g_assert_cmpint(node, ==, 1);
-        } else if (thread == 1) {
+        } else if (socket == 1 && cluster == 0 && core == 0 && thread == 0) {
             g_assert_cmpint(node, ==, 0);
         } else {
             g_assert(false);

From 4c18bc192386dfbca530e7f550e0992df657818a Mon Sep 17 00:00:00 2001
From: Gavin Shan <gshan@redhat.com>
Date: Tue, 3 May 2022 22:03:03 +0800
Subject: [PATCH 225/935] hw/arm/virt: Fix CPU's default NUMA node ID

When CPU-to-NUMA association isn't explicitly provided by users,
the default one is given by mc->get_default_cpu_node_id(). However,
the CPU topology isn't fully considered in the default association
and this causes CPU topology broken warnings on booting Linux guest.

For example, the following warning messages are observed when the
Linux guest is booted with the following command lines.

  /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64 \
  -accel kvm -machine virt,gic-version=host               \
  -cpu host                                               \
  -smp 6,sockets=2,cores=3,threads=1                      \
  -m 1024M,slots=16,maxmem=64G                            \
  -object memory-backend-ram,id=mem0,size=128M            \
  -object memory-backend-ram,id=mem1,size=128M            \
  -object memory-backend-ram,id=mem2,size=128M            \
  -object memory-backend-ram,id=mem3,size=128M            \
  -object memory-backend-ram,id=mem4,size=128M            \
  -object memory-backend-ram,id=mem4,size=384M            \
  -numa node,nodeid=0,memdev=mem0                         \
  -numa node,nodeid=1,memdev=mem1                         \
  -numa node,nodeid=2,memdev=mem2                         \
  -numa node,nodeid=3,memdev=mem3                         \
  -numa node,nodeid=4,memdev=mem4                         \
  -numa node,nodeid=5,memdev=mem5
         :
  alternatives: patching kernel code
  BUG: arch topology borken
  the CLS domain not a subset of the MC domain
  <the above error log repeats>
  BUG: arch topology borken
  the DIE domain not a subset of the NODE domain

With current implementation of mc->get_default_cpu_node_id(),
CPU#0 to CPU#5 are associated with NODE#0 to NODE#5 separately.
That's incorrect because CPU#0/1/2 should be associated with same
NUMA node because they're seated in same socket.

This fixes the issue by considering the socket ID when the default
CPU-to-NUMA association is provided in virt_possible_cpu_arch_ids().
With this applied, no more CPU topology broken warnings are seen
from the Linux guest. The 6 CPUs are associated with NODE#0/1, but
there are no CPUs associated with NODE#2/3/4/5.

Signed-off-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Yanan Wang <wangyanan55@huawei.com>
Message-id: 20220503140304.855514-6-gshan@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/arm/virt.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index c25023a083..1a45f44435 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -2554,7 +2554,9 @@ virt_cpu_index_to_props(MachineState *ms, unsigned cpu_index)
 
 static int64_t virt_get_default_cpu_node_id(const MachineState *ms, int idx)
 {
-    return idx % ms->numa_state->num_nodes;
+    int64_t socket_id = ms->possible_cpus->cpus[idx].props.socket_id;
+
+    return socket_id % ms->numa_state->num_nodes;
 }
 
 static const CPUArchIdList *virt_possible_cpu_arch_ids(MachineState *ms)

From ae9141d4a3265553503bf07d3574b40f84615a34 Mon Sep 17 00:00:00 2001
From: Gavin Shan <gshan@redhat.com>
Date: Tue, 3 May 2022 22:03:04 +0800
Subject: [PATCH 226/935] hw/acpi/aml-build: Use existing CPU topology to build
 PPTT table

When the PPTT table is built, the CPU topology is re-calculated, but
it's unecessary because the CPU topology has been populated in
virt_possible_cpu_arch_ids() on arm/virt machine.

This reworks build_pptt() to avoid by reusing the existing IDs in
ms->possible_cpus. Currently, the only user of build_pptt() is
arm/virt machine.

Signed-off-by: Gavin Shan <gshan@redhat.com>
Tested-by: Yanan Wang <wangyanan55@huawei.com>
Reviewed-by: Yanan Wang <wangyanan55@huawei.com>
Acked-by: Igor Mammedov <imammedo@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Message-id: 20220503140304.855514-7-gshan@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/acpi/aml-build.c | 111 +++++++++++++++++++-------------------------
 1 file changed, 48 insertions(+), 63 deletions(-)

diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index 4086879ebf..e6bfac95c7 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -2002,86 +2002,71 @@ void build_pptt(GArray *table_data, BIOSLinker *linker, MachineState *ms,
                 const char *oem_id, const char *oem_table_id)
 {
     MachineClass *mc = MACHINE_GET_CLASS(ms);
-    GQueue *list = g_queue_new();
-    guint pptt_start = table_data->len;
-    guint parent_offset;
-    guint length, i;
-    int uid = 0;
-    int socket;
+    CPUArchIdList *cpus = ms->possible_cpus;
+    int64_t socket_id = -1, cluster_id = -1, core_id = -1;
+    uint32_t socket_offset = 0, cluster_offset = 0, core_offset = 0;
+    uint32_t pptt_start = table_data->len;
+    int n;
     AcpiTable table = { .sig = "PPTT", .rev = 2,
                         .oem_id = oem_id, .oem_table_id = oem_table_id };
 
     acpi_table_begin(&table, table_data);
 
-    for (socket = 0; socket < ms->smp.sockets; socket++) {
-        g_queue_push_tail(list,
-            GUINT_TO_POINTER(table_data->len - pptt_start));
-        build_processor_hierarchy_node(
-            table_data,
-            /*
-             * Physical package - represents the boundary
-             * of a physical package
-             */
-            (1 << 0),
-            0, socket, NULL, 0);
-    }
-
-    if (mc->smp_props.clusters_supported) {
-        length = g_queue_get_length(list);
-        for (i = 0; i < length; i++) {
-            int cluster;
-
-            parent_offset = GPOINTER_TO_UINT(g_queue_pop_head(list));
-            for (cluster = 0; cluster < ms->smp.clusters; cluster++) {
-                g_queue_push_tail(list,
-                    GUINT_TO_POINTER(table_data->len - pptt_start));
-                build_processor_hierarchy_node(
-                    table_data,
-                    (0 << 0), /* not a physical package */
-                    parent_offset, cluster, NULL, 0);
-            }
+    /*
+     * This works with the assumption that cpus[n].props.*_id has been
+     * sorted from top to down levels in mc->possible_cpu_arch_ids().
+     * Otherwise, the unexpected and duplicated containers will be
+     * created.
+     */
+    for (n = 0; n < cpus->len; n++) {
+        if (cpus->cpus[n].props.socket_id != socket_id) {
+            assert(cpus->cpus[n].props.socket_id > socket_id);
+            socket_id = cpus->cpus[n].props.socket_id;
+            cluster_id = -1;
+            core_id = -1;
+            socket_offset = table_data->len - pptt_start;
+            build_processor_hierarchy_node(table_data,
+                (1 << 0), /* Physical package */
+                0, socket_id, NULL, 0);
         }
-    }
 
-    length = g_queue_get_length(list);
-    for (i = 0; i < length; i++) {
-        int core;
-
-        parent_offset = GPOINTER_TO_UINT(g_queue_pop_head(list));
-        for (core = 0; core < ms->smp.cores; core++) {
-            if (ms->smp.threads > 1) {
-                g_queue_push_tail(list,
-                    GUINT_TO_POINTER(table_data->len - pptt_start));
-                build_processor_hierarchy_node(
-                    table_data,
-                    (0 << 0), /* not a physical package */
-                    parent_offset, core, NULL, 0);
-            } else {
-                build_processor_hierarchy_node(
-                    table_data,
-                    (1 << 1) | /* ACPI Processor ID valid */
-                    (1 << 3),  /* Node is a Leaf */
-                    parent_offset, uid++, NULL, 0);
+        if (mc->smp_props.clusters_supported) {
+            if (cpus->cpus[n].props.cluster_id != cluster_id) {
+                assert(cpus->cpus[n].props.cluster_id > cluster_id);
+                cluster_id = cpus->cpus[n].props.cluster_id;
+                core_id = -1;
+                cluster_offset = table_data->len - pptt_start;
+                build_processor_hierarchy_node(table_data,
+                    (0 << 0), /* Not a physical package */
+                    socket_offset, cluster_id, NULL, 0);
             }
+        } else {
+            cluster_offset = socket_offset;
         }
-    }
 
-    length = g_queue_get_length(list);
-    for (i = 0; i < length; i++) {
-        int thread;
+        if (ms->smp.threads == 1) {
+            build_processor_hierarchy_node(table_data,
+                (1 << 1) | /* ACPI Processor ID valid */
+                (1 << 3),  /* Node is a Leaf */
+                cluster_offset, n, NULL, 0);
+        } else {
+            if (cpus->cpus[n].props.core_id != core_id) {
+                assert(cpus->cpus[n].props.core_id > core_id);
+                core_id = cpus->cpus[n].props.core_id;
+                core_offset = table_data->len - pptt_start;
+                build_processor_hierarchy_node(table_data,
+                    (0 << 0), /* Not a physical package */
+                    cluster_offset, core_id, NULL, 0);
+            }
 
-        parent_offset = GPOINTER_TO_UINT(g_queue_pop_head(list));
-        for (thread = 0; thread < ms->smp.threads; thread++) {
-            build_processor_hierarchy_node(
-                table_data,
+            build_processor_hierarchy_node(table_data,
                 (1 << 1) | /* ACPI Processor ID valid */
                 (1 << 2) | /* Processor is a Thread */
                 (1 << 3),  /* Node is a Leaf */
-                parent_offset, uid++, NULL, 0);
+                core_offset, n, NULL, 0);
         }
     }
 
-    g_queue_free(list);
     acpi_table_end(linker, &table);
 }
 

From 52581c718c5cd55595ca032a56f1e194c5716456 Mon Sep 17 00:00:00 2001
From: Markus Armbruster <armbru@redhat.com>
Date: Fri, 6 May 2022 15:49:08 +0200
Subject: [PATCH 227/935] Clean up header guards that don't match their file
 name

Header guard symbols should match their file name to make guard
collisions less likely.

Cleaned up with scripts/clean-header-guards.pl, followed by some
renaming of new guard symbols picked by the script to better ones.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220506134911.2856099-2-armbru@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
[Change to generated file ebpf/rss.bpf.skeleton.h backed out]
---
 accel/tcg/tcg-accel-ops-icount.h            | 6 +++---
 accel/tcg/tcg-accel-ops-mttcg.h             | 6 +++---
 accel/tcg/tcg-accel-ops-rr.h                | 6 +++---
 accel/tcg/tcg-accel-ops.h                   | 6 +++---
 block/coroutines.h                          | 6 +++---
 bsd-user/arm/target_syscall.h               | 6 +++---
 bsd-user/i386/target.h                      | 7 +++----
 bsd-user/i386/target_arch_sysarch.h         | 6 +++---
 bsd-user/x86_64/target_arch_sysarch.h       | 6 +++---
 hw/i386/e820_memory_layout.h                | 4 ++--
 hw/net/can/ctu_can_fd_frame.h               | 4 ++--
 hw/net/can/ctu_can_fd_regs.h                | 4 ++--
 hw/nvme/nvme.h                              | 6 +++---
 hw/usb/hcd-dwc2.h                           | 4 ++--
 include/block/block-hmp-cmds.h              | 4 ++--
 include/crypto/tls-cipher-suites.h          | 6 +++---
 include/hw/acpi/generic_event_device.h      | 4 ++--
 include/hw/i2c/arm_sbcon_i2c.h              | 7 ++++---
 include/hw/misc/bcm2835_cprman.h            | 4 ++--
 include/hw/misc/bcm2835_cprman_internals.h  | 4 ++--
 include/hw/misc/stm32f4xx_exti.h            | 4 ++--
 include/hw/misc/stm32f4xx_syscfg.h          | 4 ++--
 include/hw/misc/xlnx-versal-pmc-iou-slcr.h  | 6 +++---
 include/hw/net/mv88w8618_eth.h              | 5 +++--
 include/hw/nubus/mac-nubus-bridge.h         | 4 ++--
 include/hw/pci-host/remote.h                | 4 ++--
 include/hw/riscv/boot_opensbi.h             | 5 +++--
 include/hw/riscv/shakti_c.h                 | 4 ++--
 include/hw/rtc/sun4v-rtc.h                  | 4 ++--
 include/hw/rtc/xlnx-zynqmp-rtc.h            | 4 ++--
 include/hw/rx/rx62n.h                       | 4 ++--
 include/hw/sensor/emc141x_regs.h            | 4 ++--
 include/hw/ssi/xlnx-versal-ospi.h           | 6 +++---
 include/hw/timer/bcm2835_systmr.h           | 4 ++--
 include/hw/tricore/tricore_testdevice.h     | 5 ++---
 include/hw/usb/dwc2-regs.h                  | 6 +++---
 include/hw/usb/hcd-musb.h                   | 4 ++--
 include/hw/usb/xlnx-usb-subsystem.h         | 4 ++--
 include/hw/usb/xlnx-versal-usb2-ctrl-regs.h | 4 ++--
 include/hw/watchdog/wdt_imx2.h              | 6 +++---
 include/qemu/help-texts.h                   | 4 ++--
 include/qemu/qemu-plugin.h                  | 7 ++++---
 include/sysemu/block-backend-global-state.h | 6 +++---
 plugins/plugin.h                            | 6 +++---
 target/arm/translate-a32.h                  | 4 ++--
 target/arm/vec_internal.h                   | 6 +++---
 target/avr/cpu-qom.h                        | 6 +++---
 target/hexagon/hex_arch_types.h             | 4 ++--
 target/hexagon/hex_regs.h                   | 4 ++--
 target/i386/hax/hax-accel-ops.h             | 6 +++---
 target/i386/nvmm/nvmm-accel-ops.h           | 6 +++---
 target/i386/sev.h                           | 4 ++--
 target/i386/whpx/whpx-accel-ops.h           | 6 +++---
 target/i386/whpx/whpx-internal.h            | 6 +++---
 target/xtensa/core-de233_fpu/core-isa.h     | 7 +++----
 target/xtensa/core-dsp3400/core-isa.h       | 8 +++-----
 tests/qtest/fuzz/fuzz.h                     | 5 ++---
 tools/virtiofsd/passthrough_seccomp.h       | 7 +++----
 58 files changed, 148 insertions(+), 151 deletions(-)

diff --git a/accel/tcg/tcg-accel-ops-icount.h b/accel/tcg/tcg-accel-ops-icount.h
index d884aa2aaa..1b6fd9c607 100644
--- a/accel/tcg/tcg-accel-ops-icount.h
+++ b/accel/tcg/tcg-accel-ops-icount.h
@@ -7,8 +7,8 @@
  * See the COPYING file in the top-level directory.
  */
 
-#ifndef TCG_CPUS_ICOUNT_H
-#define TCG_CPUS_ICOUNT_H
+#ifndef TCG_ACCEL_OPS_ICOUNT_H
+#define TCG_ACCEL_OPS_ICOUNT_H
 
 void icount_handle_deadline(void);
 void icount_prepare_for_run(CPUState *cpu);
@@ -16,4 +16,4 @@ void icount_process_data(CPUState *cpu);
 
 void icount_handle_interrupt(CPUState *cpu, int mask);
 
-#endif /* TCG_CPUS_ICOUNT_H */
+#endif /* TCG_ACCEL_OPS_ICOUNT_H */
diff --git a/accel/tcg/tcg-accel-ops-mttcg.h b/accel/tcg/tcg-accel-ops-mttcg.h
index 9fdc5a2ab5..8ffa7a9a9f 100644
--- a/accel/tcg/tcg-accel-ops-mttcg.h
+++ b/accel/tcg/tcg-accel-ops-mttcg.h
@@ -7,8 +7,8 @@
  * See the COPYING file in the top-level directory.
  */
 
-#ifndef TCG_CPUS_MTTCG_H
-#define TCG_CPUS_MTTCG_H
+#ifndef TCG_ACCEL_OPS_MTTCG_H
+#define TCG_ACCEL_OPS_MTTCG_H
 
 /* kick MTTCG vCPU thread */
 void mttcg_kick_vcpu_thread(CPUState *cpu);
@@ -16,4 +16,4 @@ void mttcg_kick_vcpu_thread(CPUState *cpu);
 /* start an mttcg vCPU thread */
 void mttcg_start_vcpu_thread(CPUState *cpu);
 
-#endif /* TCG_CPUS_MTTCG_H */
+#endif /* TCG_ACCEL_OPS_MTTCG_H */
diff --git a/accel/tcg/tcg-accel-ops-rr.h b/accel/tcg/tcg-accel-ops-rr.h
index 54f6ae6e86..2a76a29612 100644
--- a/accel/tcg/tcg-accel-ops-rr.h
+++ b/accel/tcg/tcg-accel-ops-rr.h
@@ -7,8 +7,8 @@
  * See the COPYING file in the top-level directory.
  */
 
-#ifndef TCG_CPUS_RR_H
-#define TCG_CPUS_RR_H
+#ifndef TCG_ACCEL_OPS_RR_H
+#define TCG_ACCEL_OPS_RR_H
 
 #define TCG_KICK_PERIOD (NANOSECONDS_PER_SECOND / 10)
 
@@ -18,4 +18,4 @@ void rr_kick_vcpu_thread(CPUState *unused);
 /* start the round robin vcpu thread */
 void rr_start_vcpu_thread(CPUState *cpu);
 
-#endif /* TCG_CPUS_RR_H */
+#endif /* TCG_ACCEL_OPS_RR_H */
diff --git a/accel/tcg/tcg-accel-ops.h b/accel/tcg/tcg-accel-ops.h
index 6a5fcef889..f9bc6330e2 100644
--- a/accel/tcg/tcg-accel-ops.h
+++ b/accel/tcg/tcg-accel-ops.h
@@ -9,8 +9,8 @@
  * See the COPYING file in the top-level directory.
  */
 
-#ifndef TCG_CPUS_H
-#define TCG_CPUS_H
+#ifndef TCG_ACCEL_OPS_H
+#define TCG_ACCEL_OPS_H
 
 #include "sysemu/cpus.h"
 
@@ -19,4 +19,4 @@ int tcg_cpus_exec(CPUState *cpu);
 void tcg_handle_interrupt(CPUState *cpu, int mask);
 void tcg_cpu_init_cflags(CPUState *cpu, bool parallel);
 
-#endif /* TCG_CPUS_H */
+#endif /* TCG_ACCEL_OPS_H */
diff --git a/block/coroutines.h b/block/coroutines.h
index 8ea70d45f9..830ecaa733 100644
--- a/block/coroutines.h
+++ b/block/coroutines.h
@@ -22,8 +22,8 @@
  * THE SOFTWARE.
  */
 
-#ifndef BLOCK_COROUTINES_INT_H
-#define BLOCK_COROUTINES_INT_H
+#ifndef BLOCK_COROUTINES_H
+#define BLOCK_COROUTINES_H
 
 #include "block/block_int.h"
 
@@ -129,4 +129,4 @@ blk_do_pdiscard(BlockBackend *blk, int64_t offset, int64_t bytes);
 
 int generated_co_wrapper blk_do_flush(BlockBackend *blk);
 
-#endif /* BLOCK_COROUTINES_INT_H */
+#endif /* BLOCK_COROUTINES_H */
diff --git a/bsd-user/arm/target_syscall.h b/bsd-user/arm/target_syscall.h
index a5f2bb4e01..5804a53541 100644
--- a/bsd-user/arm/target_syscall.h
+++ b/bsd-user/arm/target_syscall.h
@@ -17,8 +17,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_ARCH_SYSCALL_H_
-#define _TARGET_ARCH_SYSCALL_H_
+#ifndef ARM_TARGET_SYSCALL_H
+#define ARM_TARGET_SYSCALL_H
 
 struct target_pt_regs {
     abi_long uregs[17];
@@ -52,4 +52,4 @@ struct target_pt_regs {
 #define TARGET_HW_MACHINE       "arm"
 #define TARGET_HW_MACHINE_ARCH  "armv7"
 
-#endif /* !_TARGET_ARCH_SYSCALL_H_ */
+#endif /* ARM_TARGET_SYSCALL_H */
diff --git a/bsd-user/i386/target.h b/bsd-user/i386/target.h
index 9b9df047a3..ddd3b8ec08 100644
--- a/bsd-user/i386/target.h
+++ b/bsd-user/i386/target.h
@@ -6,8 +6,8 @@
  * SPDX-License-Identifier: GPL-2.0-or-later
  */
 
-#ifndef TARGET_ARCH_H
-#define TARGET_ARCH_H
+#ifndef TARGET_H
+#define TARGET_H
 
 /*
  * i386 doesn't 'lump' the registers for 64-bit args.
@@ -17,5 +17,4 @@ static inline bool regpairs_aligned(void *cpu_env)
     return false;
 }
 
-#endif /* ! TARGET_ARCH_H */
-
+#endif /* TARGET_H */
diff --git a/bsd-user/i386/target_arch_sysarch.h b/bsd-user/i386/target_arch_sysarch.h
index e9ab98ec32..db8fee6380 100644
--- a/bsd-user/i386/target_arch_sysarch.h
+++ b/bsd-user/i386/target_arch_sysarch.h
@@ -17,8 +17,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef BSD_USER_ARCH_SYSARCH_H_
-#define BSD_USER_ARCH_SYSARCH_H_
+#ifndef TARGET_ARCH_SYSARCH_H
+#define TARGET_ARCH_SYSARCH_H
 
 #include "target_syscall.h"
 
@@ -74,4 +74,4 @@ static inline void do_freebsd_arch_print_sysarch(
         TARGET_ABI_FMT_lx ")", name->name, (int)arg1, arg2, arg3, arg4);
 }
 
-#endif /* !BSD_USER_ARCH_SYSARCH_H_ */
+#endif /* TARGET_ARCH_SYSARCH_H */
diff --git a/bsd-user/x86_64/target_arch_sysarch.h b/bsd-user/x86_64/target_arch_sysarch.h
index 5c36fc0752..152cb8bcb8 100644
--- a/bsd-user/x86_64/target_arch_sysarch.h
+++ b/bsd-user/x86_64/target_arch_sysarch.h
@@ -16,8 +16,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef BSD_USER_ARCH_SYSARCH_H_
-#define BSD_USER_ARCH_SYSARCH_H_
+#ifndef TARGET_ARCH_SYSARCH_H
+#define TARGET_ARCH_SYSARCH_H
 
 #include "target_syscall.h"
 
@@ -73,4 +73,4 @@ static inline void do_freebsd_arch_print_sysarch(
         TARGET_ABI_FMT_lx ")", name->name, (int)arg1, arg2, arg3, arg4);
 }
 
-#endif /*! BSD_USER_ARCH_SYSARCH_H_ */
+#endif /* TARGET_ARCH_SYSARCH_H */
diff --git a/hw/i386/e820_memory_layout.h b/hw/i386/e820_memory_layout.h
index 2a0ceb8b9c..04f93780f9 100644
--- a/hw/i386/e820_memory_layout.h
+++ b/hw/i386/e820_memory_layout.h
@@ -6,8 +6,8 @@
  * SPDX-License-Identifier: MIT
  */
 
-#ifndef HW_I386_E820_H
-#define HW_I386_E820_H
+#ifndef HW_I386_E820_MEMORY_LAYOUT_H
+#define HW_I386_E820_MEMORY_LAYOUT_H
 
 /* e820 types */
 #define E820_RAM        1
diff --git a/hw/net/can/ctu_can_fd_frame.h b/hw/net/can/ctu_can_fd_frame.h
index 04d956c84e..459c4a0ada 100644
--- a/hw/net/can/ctu_can_fd_frame.h
+++ b/hw/net/can/ctu_can_fd_frame.h
@@ -29,8 +29,8 @@
 
 /* This file is autogenerated, DO NOT EDIT! */
 
-#ifndef __CTU_CAN_FD_CAN_FD_FRAME_FORMAT__
-#define __CTU_CAN_FD_CAN_FD_FRAME_FORMAT__
+#ifndef HW_CAN_CTU_CAN_FD_FRAME_H
+#define HW_CAN_CTU_CAN_FD_FRAME_H
 
 /* CAN_Frame_format memory map */
 enum ctu_can_fd_can_frame_format {
diff --git a/hw/net/can/ctu_can_fd_regs.h b/hw/net/can/ctu_can_fd_regs.h
index 450f4b9fb3..57859b87bc 100644
--- a/hw/net/can/ctu_can_fd_regs.h
+++ b/hw/net/can/ctu_can_fd_regs.h
@@ -29,8 +29,8 @@
 
 /* This file is autogenerated, DO NOT EDIT! */
 
-#ifndef __CTU_CAN_FD_CAN_FD_REGISTER_MAP__
-#define __CTU_CAN_FD_CAN_FD_REGISTER_MAP__
+#ifndef HW_CAN_CTU_CAN_FD_REGS_H
+#define HW_CAN_CTU_CAN_FD_REGS_H
 
 /* CAN_Registers memory map */
 enum ctu_can_fd_can_registers {
diff --git a/hw/nvme/nvme.h b/hw/nvme/nvme.h
index 739c8b8f79..6773819325 100644
--- a/hw/nvme/nvme.h
+++ b/hw/nvme/nvme.h
@@ -15,8 +15,8 @@
  * This code is licensed under the GNU GPL v2 or later.
  */
 
-#ifndef HW_NVME_INTERNAL_H
-#define HW_NVME_INTERNAL_H
+#ifndef HW_NVME_NVME_H
+#define HW_NVME_NVME_H
 
 #include "qemu/uuid.h"
 #include "hw/pci/pci.h"
@@ -519,4 +519,4 @@ void nvme_rw_complete_cb(void *opaque, int ret);
 uint16_t nvme_map_dptr(NvmeCtrl *n, NvmeSg *sg, size_t len,
                        NvmeCmd *cmd);
 
-#endif /* HW_NVME_INTERNAL_H */
+#endif /* HW_NVME_NVME_H */
diff --git a/hw/usb/hcd-dwc2.h b/hw/usb/hcd-dwc2.h
index 6998b04706..9c3d88ea14 100644
--- a/hw/usb/hcd-dwc2.h
+++ b/hw/usb/hcd-dwc2.h
@@ -16,8 +16,8 @@
  * GNU General Public License for more details.
  */
 
-#ifndef HW_USB_DWC2_H
-#define HW_USB_DWC2_H
+#ifndef HW_USB_HCD_DWC2_H
+#define HW_USB_HCD_DWC2_H
 
 #include "qemu/timer.h"
 #include "hw/irq.h"
diff --git a/include/block/block-hmp-cmds.h b/include/block/block-hmp-cmds.h
index 3412e108ca..50ce0247c3 100644
--- a/include/block/block-hmp-cmds.h
+++ b/include/block/block-hmp-cmds.h
@@ -12,8 +12,8 @@
  * the COPYING file in the top-level directory.
  */
 
-#ifndef BLOCK_HMP_COMMANDS_H
-#define BLOCK_HMP_COMMANDS_H
+#ifndef BLOCK_BLOCK_HMP_CMDS_H
+#define BLOCK_BLOCK_HMP_CMDS_H
 
 void hmp_drive_add(Monitor *mon, const QDict *qdict);
 
diff --git a/include/crypto/tls-cipher-suites.h b/include/crypto/tls-cipher-suites.h
index 7eb1b76122..3bd2003f32 100644
--- a/include/crypto/tls-cipher-suites.h
+++ b/include/crypto/tls-cipher-suites.h
@@ -8,8 +8,8 @@
  * SPDX-License-Identifier: GPL-2.0-or-later
  */
 
-#ifndef QCRYPTO_TLSCIPHERSUITES_H
-#define QCRYPTO_TLSCIPHERSUITES_H
+#ifndef QCRYPTO_TLS_CIPHER_SUITES_H
+#define QCRYPTO_TLS_CIPHER_SUITES_H
 
 #include "qom/object.h"
 #include "crypto/tlscreds.h"
@@ -31,4 +31,4 @@ DECLARE_INSTANCE_CHECKER(QCryptoTLSCipherSuites, QCRYPTO_TLS_CIPHER_SUITES,
 GByteArray *qcrypto_tls_cipher_suites_get_data(QCryptoTLSCipherSuites *obj,
                                                Error **errp);
 
-#endif /* QCRYPTO_TLSCIPHERSUITES_H */
+#endif /* QCRYPTO_TLS_CIPHER_SUITES_H */
diff --git a/include/hw/acpi/generic_event_device.h b/include/hw/acpi/generic_event_device.h
index d49217c445..d831bbd889 100644
--- a/include/hw/acpi/generic_event_device.h
+++ b/include/hw/acpi/generic_event_device.h
@@ -56,8 +56,8 @@
  *
  */
 
-#ifndef HW_ACPI_GED_H
-#define HW_ACPI_GED_H
+#ifndef HW_ACPI_GENERIC_EVENT_DEVICE_H
+#define HW_ACPI_GENERIC_EVENT_DEVICE_H
 
 #include "hw/sysbus.h"
 #include "hw/acpi/memory_hotplug.h"
diff --git a/include/hw/i2c/arm_sbcon_i2c.h b/include/hw/i2c/arm_sbcon_i2c.h
index ad96781e7a..f54d1e5413 100644
--- a/include/hw/i2c/arm_sbcon_i2c.h
+++ b/include/hw/i2c/arm_sbcon_i2c.h
@@ -9,8 +9,9 @@
  *
  * SPDX-License-Identifier: GPL-2.0-or-later
  */
-#ifndef HW_I2C_ARM_SBCON_H
-#define HW_I2C_ARM_SBCON_H
+
+#ifndef HW_I2C_ARM_SBCON_I2C_H
+#define HW_I2C_ARM_SBCON_I2C_H
 
 #include "hw/sysbus.h"
 #include "hw/i2c/bitbang_i2c.h"
@@ -34,4 +35,4 @@ struct ArmSbconI2CState {
     int in;
 };
 
-#endif /* HW_I2C_ARM_SBCON_H */
+#endif /* HW_I2C_ARM_SBCON_I2C_H */
diff --git a/include/hw/misc/bcm2835_cprman.h b/include/hw/misc/bcm2835_cprman.h
index 3df4ceedd2..0d38036728 100644
--- a/include/hw/misc/bcm2835_cprman.h
+++ b/include/hw/misc/bcm2835_cprman.h
@@ -6,8 +6,8 @@
  * SPDX-License-Identifier: GPL-2.0-or-later
  */
 
-#ifndef HW_MISC_CPRMAN_H
-#define HW_MISC_CPRMAN_H
+#ifndef HW_MISC_BCM2835_CPRMAN_H
+#define HW_MISC_BCM2835_CPRMAN_H
 
 #include "hw/sysbus.h"
 #include "hw/qdev-clock.h"
diff --git a/include/hw/misc/bcm2835_cprman_internals.h b/include/hw/misc/bcm2835_cprman_internals.h
index 339759b307..7617aff96f 100644
--- a/include/hw/misc/bcm2835_cprman_internals.h
+++ b/include/hw/misc/bcm2835_cprman_internals.h
@@ -6,8 +6,8 @@
  * SPDX-License-Identifier: GPL-2.0-or-later
  */
 
-#ifndef HW_MISC_CPRMAN_INTERNALS_H
-#define HW_MISC_CPRMAN_INTERNALS_H
+#ifndef HW_MISC_BCM2835_CPRMAN_INTERNALS_H
+#define HW_MISC_BCM2835_CPRMAN_INTERNALS_H
 
 #include "hw/registerfields.h"
 #include "hw/misc/bcm2835_cprman.h"
diff --git a/include/hw/misc/stm32f4xx_exti.h b/include/hw/misc/stm32f4xx_exti.h
index ea6b0097b0..fc11c595fa 100644
--- a/include/hw/misc/stm32f4xx_exti.h
+++ b/include/hw/misc/stm32f4xx_exti.h
@@ -22,8 +22,8 @@
  * THE SOFTWARE.
  */
 
-#ifndef HW_STM_EXTI_H
-#define HW_STM_EXTI_H
+#ifndef HW_STM32F4XX_EXTI_H
+#define HW_STM32F4XX_EXTI_H
 
 #include "hw/sysbus.h"
 #include "qom/object.h"
diff --git a/include/hw/misc/stm32f4xx_syscfg.h b/include/hw/misc/stm32f4xx_syscfg.h
index 6f8ca49228..9fce67f4b4 100644
--- a/include/hw/misc/stm32f4xx_syscfg.h
+++ b/include/hw/misc/stm32f4xx_syscfg.h
@@ -22,8 +22,8 @@
  * THE SOFTWARE.
  */
 
-#ifndef HW_STM_SYSCFG_H
-#define HW_STM_SYSCFG_H
+#ifndef HW_STM32F4XX_SYSCFG_H
+#define HW_STM32F4XX_SYSCFG_H
 
 #include "hw/sysbus.h"
 #include "qom/object.h"
diff --git a/include/hw/misc/xlnx-versal-pmc-iou-slcr.h b/include/hw/misc/xlnx-versal-pmc-iou-slcr.h
index ab4e4b4f18..2170420f01 100644
--- a/include/hw/misc/xlnx-versal-pmc-iou-slcr.h
+++ b/include/hw/misc/xlnx-versal-pmc-iou-slcr.h
@@ -51,8 +51,8 @@
  *   1: OSPI direct access mode.
  */
 
-#ifndef XILINX_VERSAL_PMC_IOU_SLCR_H
-#define XILINX_VERSAL_PMC_IOU_SLCR_H
+#ifndef XLNX_VERSAL_PMC_IOU_SLCR_H
+#define XLNX_VERSAL_PMC_IOU_SLCR_H
 
 #include "hw/register.h"
 
@@ -75,4 +75,4 @@ struct XlnxVersalPmcIouSlcr {
     RegisterInfo regs_info[XILINX_VERSAL_PMC_IOU_SLCR_R_MAX];
 };
 
-#endif /* XILINX_VERSAL_PMC_IOU_SLCR_H */
+#endif /* XLNX_VERSAL_PMC_IOU_SLCR_H */
diff --git a/include/hw/net/mv88w8618_eth.h b/include/hw/net/mv88w8618_eth.h
index 8f4c746092..41074940ec 100644
--- a/include/hw/net/mv88w8618_eth.h
+++ b/include/hw/net/mv88w8618_eth.h
@@ -4,8 +4,9 @@
  *
  * Copyright (c) 2008-2021 QEMU contributors
  */
-#ifndef HW_NET_MV88W8618_H
-#define HW_NET_MV88W8618_H
+
+#ifndef HW_NET_MV88W8618_ETH_H
+#define HW_NET_MV88W8618_ETH_H
 
 #define TYPE_MV88W8618_ETH "mv88w8618_eth"
 
diff --git a/include/hw/nubus/mac-nubus-bridge.h b/include/hw/nubus/mac-nubus-bridge.h
index 70ab50ab2d..be4dd83530 100644
--- a/include/hw/nubus/mac-nubus-bridge.h
+++ b/include/hw/nubus/mac-nubus-bridge.h
@@ -6,8 +6,8 @@
  *
  */
 
-#ifndef HW_NUBUS_MAC_H
-#define HW_NUBUS_MAC_H
+#ifndef HW_NUBUS_MAC_NUBUS_BRIDGE_H
+#define HW_NUBUS_MAC_NUBUS_BRIDGE_H
 
 #include "hw/nubus/nubus.h"
 #include "qom/object.h"
diff --git a/include/hw/pci-host/remote.h b/include/hw/pci-host/remote.h
index 3dcf6aa51d..690a01f0fe 100644
--- a/include/hw/pci-host/remote.h
+++ b/include/hw/pci-host/remote.h
@@ -8,8 +8,8 @@
  *
  */
 
-#ifndef REMOTE_PCIHOST_H
-#define REMOTE_PCIHOST_H
+#ifndef PCI_HOST_REMOTE_H
+#define PCI_HOST_REMOTE_H
 
 #include "exec/memory.h"
 #include "hw/pci/pcie_host.h"
diff --git a/include/hw/riscv/boot_opensbi.h b/include/hw/riscv/boot_opensbi.h
index 0d5ddd6c3d..c19cad4818 100644
--- a/include/hw/riscv/boot_opensbi.h
+++ b/include/hw/riscv/boot_opensbi.h
@@ -4,8 +4,9 @@
  *
  * Based on include/sbi/{fw_dynamic.h,sbi_scratch.h} from the OpenSBI project.
  */
-#ifndef OPENSBI_H
-#define OPENSBI_H
+
+#ifndef RISCV_BOOT_OPENSBI_H
+#define RISCV_BOOT_OPENSBI_H
 
 /** Expected value of info magic ('OSBI' ascii string in hex) */
 #define FW_DYNAMIC_INFO_MAGIC_VALUE     0x4942534f
diff --git a/include/hw/riscv/shakti_c.h b/include/hw/riscv/shakti_c.h
index 50a2b79086..daf0aae13f 100644
--- a/include/hw/riscv/shakti_c.h
+++ b/include/hw/riscv/shakti_c.h
@@ -16,8 +16,8 @@
  * this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef HW_SHAKTI_H
-#define HW_SHAKTI_H
+#ifndef HW_SHAKTI_C_H
+#define HW_SHAKTI_C_H
 
 #include "hw/riscv/riscv_hart.h"
 #include "hw/boards.h"
diff --git a/include/hw/rtc/sun4v-rtc.h b/include/hw/rtc/sun4v-rtc.h
index fd868f6ed2..fc54dfcba4 100644
--- a/include/hw/rtc/sun4v-rtc.h
+++ b/include/hw/rtc/sun4v-rtc.h
@@ -9,8 +9,8 @@
  * version.
  */
 
-#ifndef HW_RTC_SUN4V
-#define HW_RTC_SUN4V
+#ifndef HW_RTC_SUN4V_RTC_H
+#define HW_RTC_SUN4V_RTC_H
 
 #include "exec/hwaddr.h"
 
diff --git a/include/hw/rtc/xlnx-zynqmp-rtc.h b/include/hw/rtc/xlnx-zynqmp-rtc.h
index 5f1ad0a946..f0c6a2d78a 100644
--- a/include/hw/rtc/xlnx-zynqmp-rtc.h
+++ b/include/hw/rtc/xlnx-zynqmp-rtc.h
@@ -24,8 +24,8 @@
  * THE SOFTWARE.
  */
 
-#ifndef HW_RTC_XLNX_ZYNQMP_H
-#define HW_RTC_XLNX_ZYNQMP_H
+#ifndef HW_RTC_XLNX_ZYNQMP_RTC_H
+#define HW_RTC_XLNX_ZYNQMP_RTC_H
 
 #include "hw/register.h"
 #include "hw/sysbus.h"
diff --git a/include/hw/rx/rx62n.h b/include/hw/rx/rx62n.h
index 3ed80dba0d..73ceeb58e5 100644
--- a/include/hw/rx/rx62n.h
+++ b/include/hw/rx/rx62n.h
@@ -21,8 +21,8 @@
  * this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef HW_RX_RX62N_MCU_H
-#define HW_RX_RX62N_MCU_H
+#ifndef HW_RX_RX62N_H
+#define HW_RX_RX62N_H
 
 #include "target/rx/cpu.h"
 #include "hw/intc/rx_icu.h"
diff --git a/include/hw/sensor/emc141x_regs.h b/include/hw/sensor/emc141x_regs.h
index 0560fb7c5c..e509a43d55 100644
--- a/include/hw/sensor/emc141x_regs.h
+++ b/include/hw/sensor/emc141x_regs.h
@@ -9,8 +9,8 @@
  * later. See the COPYING file in the top-level directory.
  */
 
-#ifndef TMP105_REGS_H
-#define TMP105_REGS_H
+#ifndef EMC141X_REGS_H
+#define EMC141X_REGS_H
 
 #define EMC1413_DEVICE_ID                0x21
 #define EMC1414_DEVICE_ID                0x25
diff --git a/include/hw/ssi/xlnx-versal-ospi.h b/include/hw/ssi/xlnx-versal-ospi.h
index 14d1263497..5d131d351d 100644
--- a/include/hw/ssi/xlnx-versal-ospi.h
+++ b/include/hw/ssi/xlnx-versal-ospi.h
@@ -49,8 +49,8 @@
  * + Property "indac-write-disabled": Disable indirect access writes.
  */
 
-#ifndef XILINX_VERSAL_OSPI_H
-#define XILINX_VERSAL_OSPI_H
+#ifndef XLNX_VERSAL_OSPI_H
+#define XLNX_VERSAL_OSPI_H
 
 #include "hw/register.h"
 #include "hw/ssi/ssi.h"
@@ -108,4 +108,4 @@ struct XlnxVersalOspi {
     uint8_t stig_membank[512];
 };
 
-#endif /* XILINX_VERSAL_OSPI_H */
+#endif /* XLNX_VERSAL_OSPI_H */
diff --git a/include/hw/timer/bcm2835_systmr.h b/include/hw/timer/bcm2835_systmr.h
index bd3097d746..a8f605beeb 100644
--- a/include/hw/timer/bcm2835_systmr.h
+++ b/include/hw/timer/bcm2835_systmr.h
@@ -6,8 +6,8 @@
  * SPDX-License-Identifier: GPL-2.0-or-later
  */
 
-#ifndef BCM2835_SYSTIMER_H
-#define BCM2835_SYSTIMER_H
+#ifndef BCM2835_SYSTMR_H
+#define BCM2835_SYSTMR_H
 
 #include "hw/sysbus.h"
 #include "hw/irq.h"
diff --git a/include/hw/tricore/tricore_testdevice.h b/include/hw/tricore/tricore_testdevice.h
index 2c56c51bcb..1e2b8942ac 100644
--- a/include/hw/tricore/tricore_testdevice.h
+++ b/include/hw/tricore/tricore_testdevice.h
@@ -15,9 +15,8 @@
  * License along with this library; if not, see <http://www.gnu.org/licenses/>.
  */
 
-
-#ifndef HW_TRICORE_TESTDEV_H
-#define HW_TRICORE_TESTDEV_H
+#ifndef HW_TRICORE_TESTDEVICE_H
+#define HW_TRICORE_TESTDEVICE_H
 
 #include "hw/sysbus.h"
 #include "hw/hw.h"
diff --git a/include/hw/usb/dwc2-regs.h b/include/hw/usb/dwc2-regs.h
index a7eb531485..4015c1d691 100644
--- a/include/hw/usb/dwc2-regs.h
+++ b/include/hw/usb/dwc2-regs.h
@@ -39,8 +39,8 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
-#ifndef DWC2_HW_H
-#define DWC2_HW_H
+#ifndef DWC2_REGS_H
+#define DWC2_REGS_H
 
 #define HSOTG_REG(x)	(x)
 
@@ -896,4 +896,4 @@ struct dwc2_dma_desc {
 #define MAX_DMA_DESC_NUM_GENERIC	64
 #define MAX_DMA_DESC_NUM_HS_ISOC	256
 
-#endif /* __DWC2_HW_H__ */
+#endif /* DWC2_REGS_H */
diff --git a/include/hw/usb/hcd-musb.h b/include/hw/usb/hcd-musb.h
index c874b9f292..f30a26f7f4 100644
--- a/include/hw/usb/hcd-musb.h
+++ b/include/hw/usb/hcd-musb.h
@@ -10,8 +10,8 @@
  * SPDX-License-Identifier: GPL-2.0-or-later
  */
 
-#ifndef HW_USB_MUSB_H
-#define HW_USB_MUSB_H
+#ifndef HW_USB_HCD_MUSB_H
+#define HW_USB_HCD_MUSB_H
 
 enum musb_irq_source_e {
     musb_irq_suspend = 0,
diff --git a/include/hw/usb/xlnx-usb-subsystem.h b/include/hw/usb/xlnx-usb-subsystem.h
index 999e423951..5b730abd84 100644
--- a/include/hw/usb/xlnx-usb-subsystem.h
+++ b/include/hw/usb/xlnx-usb-subsystem.h
@@ -22,8 +22,8 @@
  * THE SOFTWARE.
  */
 
-#ifndef XLNX_VERSAL_USB_SUBSYSTEM_H
-#define XLNX_VERSAL_USB_SUBSYSTEM_H
+#ifndef XLNX_USB_SUBSYSTEM_H
+#define XLNX_USB_SUBSYSTEM_H
 
 #include "hw/usb/xlnx-versal-usb2-ctrl-regs.h"
 #include "hw/usb/hcd-dwc3.h"
diff --git a/include/hw/usb/xlnx-versal-usb2-ctrl-regs.h b/include/hw/usb/xlnx-versal-usb2-ctrl-regs.h
index b76dce0419..633bf3013a 100644
--- a/include/hw/usb/xlnx-versal-usb2-ctrl-regs.h
+++ b/include/hw/usb/xlnx-versal-usb2-ctrl-regs.h
@@ -23,8 +23,8 @@
  * THE SOFTWARE.
  */
 
-#ifndef XLNX_USB2_REGS_H
-#define XLNX_USB2_REGS_H
+#ifndef XLNX_VERSAL_USB2_CTRL_REGS_H
+#define XLNX_VERSAL_USB2_CTRL_REGS_H
 
 #define TYPE_XILINX_VERSAL_USB2_CTRL_REGS "xlnx.versal-usb2-ctrl-regs"
 
diff --git a/include/hw/watchdog/wdt_imx2.h b/include/hw/watchdog/wdt_imx2.h
index 023d83f48f..600a552d2e 100644
--- a/include/hw/watchdog/wdt_imx2.h
+++ b/include/hw/watchdog/wdt_imx2.h
@@ -9,8 +9,8 @@
  * See the COPYING file in the top-level directory.
  */
 
-#ifndef IMX2_WDT_H
-#define IMX2_WDT_H
+#ifndef WDT_IMX2_H
+#define WDT_IMX2_H
 
 #include "qemu/bitops.h"
 #include "hw/sysbus.h"
@@ -88,4 +88,4 @@ struct IMX2WdtState {
     bool wcr_wdt_locked;        /* affects WDT (never cleared) */
 };
 
-#endif /* IMX2_WDT_H */
+#endif /* WDT_IMX2_H */
diff --git a/include/qemu/help-texts.h b/include/qemu/help-texts.h
index ba32cc8b1f..4f265fed8d 100644
--- a/include/qemu/help-texts.h
+++ b/include/qemu/help-texts.h
@@ -1,5 +1,5 @@
-#ifndef QEMU_COMMON_H
-#define QEMU_COMMON_H
+#ifndef QEMU_HELP_TEXTS_H
+#define QEMU_HELP_TEXTS_H
 
 /* Copyright string for -version arguments, About dialogs, etc */
 #define QEMU_COPYRIGHT "Copyright (c) 2003-2022 " \
diff --git a/include/qemu/qemu-plugin.h b/include/qemu/qemu-plugin.h
index 535ddbf0ae..d0e9d03adf 100644
--- a/include/qemu/qemu-plugin.h
+++ b/include/qemu/qemu-plugin.h
@@ -7,8 +7,9 @@
  *
  * SPDX-License-Identifier: GPL-2.0-or-later
  */
-#ifndef QEMU_PLUGIN_API_H
-#define QEMU_PLUGIN_API_H
+
+#ifndef QEMU_QEMU_PLUGIN_H
+#define QEMU_QEMU_PLUGIN_H
 
 #include <inttypes.h>
 #include <stdbool.h>
@@ -624,4 +625,4 @@ uint64_t qemu_plugin_end_code(void);
  */
 uint64_t qemu_plugin_entry_code(void);
 
-#endif /* QEMU_PLUGIN_API_H */
+#endif /* QEMU_QEMU_PLUGIN_H */
diff --git a/include/sysemu/block-backend-global-state.h b/include/sysemu/block-backend-global-state.h
index 2e93a74679..415f0c91d7 100644
--- a/include/sysemu/block-backend-global-state.h
+++ b/include/sysemu/block-backend-global-state.h
@@ -10,8 +10,8 @@
  * or later.  See the COPYING.LIB file in the top-level directory.
  */
 
-#ifndef BLOCK_BACKEND_GS_H
-#define BLOCK_BACKEND_GS_H
+#ifndef BLOCK_BACKEND_GLOBAL_STATE_H
+#define BLOCK_BACKEND_GLOBAL_STATE_H
 
 #include "block-backend-common.h"
 
@@ -113,4 +113,4 @@ const BdrvChild *blk_root(BlockBackend *blk);
 
 int blk_make_empty(BlockBackend *blk, Error **errp);
 
-#endif /* BLOCK_BACKEND_GS_H */
+#endif /* BLOCK_BACKEND_GLOBAL_STATE_H */
diff --git a/plugins/plugin.h b/plugins/plugin.h
index b13677d0dc..5eb2fdbc85 100644
--- a/plugins/plugin.h
+++ b/plugins/plugin.h
@@ -9,8 +9,8 @@
  * SPDX-License-Identifier: GPL-2.0-or-later
  */
 
-#ifndef PLUGIN_INTERNAL_H
-#define PLUGIN_INTERNAL_H
+#ifndef PLUGIN_H
+#define PLUGIN_H
 
 #include <gmodule.h>
 #include "qemu/qht.h"
@@ -97,4 +97,4 @@ void plugin_register_vcpu_mem_cb(GArray **arr,
 
 void exec_inline_op(struct qemu_plugin_dyn_cb *cb);
 
-#endif /* _PLUGIN_INTERNAL_H_ */
+#endif /* PLUGIN_H */
diff --git a/target/arm/translate-a32.h b/target/arm/translate-a32.h
index 09010ad2da..78a84c1414 100644
--- a/target/arm/translate-a32.h
+++ b/target/arm/translate-a32.h
@@ -17,8 +17,8 @@
  * License along with this library; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef TARGET_ARM_TRANSLATE_A64_H
-#define TARGET_ARM_TRANSLATE_A64_H
+#ifndef TARGET_ARM_TRANSLATE_A32_H
+#define TARGET_ARM_TRANSLATE_A32_H
 
 /* Prototypes for autogenerated disassembler functions */
 bool disas_m_nocp(DisasContext *dc, uint32_t insn);
diff --git a/target/arm/vec_internal.h b/target/arm/vec_internal.h
index fb43a2380e..1d63402042 100644
--- a/target/arm/vec_internal.h
+++ b/target/arm/vec_internal.h
@@ -17,8 +17,8 @@
  * License along with this library; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef TARGET_ARM_VEC_INTERNALS_H
-#define TARGET_ARM_VEC_INTERNALS_H
+#ifndef TARGET_ARM_VEC_INTERNAL_H
+#define TARGET_ARM_VEC_INTERNAL_H
 
 /*
  * Note that vector data is stored in host-endian 64-bit chunks,
@@ -217,4 +217,4 @@ uint64_t pmull_h(uint64_t op1, uint64_t op2);
  */
 uint64_t pmull_w(uint64_t op1, uint64_t op2);
 
-#endif /* TARGET_ARM_VEC_INTERNALS_H */
+#endif /* TARGET_ARM_VEC_INTERNAL_H */
diff --git a/target/avr/cpu-qom.h b/target/avr/cpu-qom.h
index 32a1c762e6..b5c3507d6d 100644
--- a/target/avr/cpu-qom.h
+++ b/target/avr/cpu-qom.h
@@ -18,8 +18,8 @@
  * <http://www.gnu.org/licenses/lgpl-2.1.html>
  */
 
-#ifndef QEMU_AVR_QOM_H
-#define QEMU_AVR_QOM_H
+#ifndef TARGET_AVR_CPU_QOM_H
+#define TARGET_AVR_CPU_QOM_H
 
 #include "hw/core/cpu.h"
 #include "qom/object.h"
@@ -44,4 +44,4 @@ struct AVRCPUClass {
 };
 
 
-#endif /* !defined (QEMU_AVR_CPU_QOM_H) */
+#endif /* TARGET_AVR_CPU_QOM_H */
diff --git a/target/hexagon/hex_arch_types.h b/target/hexagon/hex_arch_types.h
index 78ad607f53..885f68f760 100644
--- a/target/hexagon/hex_arch_types.h
+++ b/target/hexagon/hex_arch_types.h
@@ -15,8 +15,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef HEXAGON_ARCH_TYPES_H
-#define HEXAGON_ARCH_TYPES_H
+#ifndef HEXAGON_HEX_ARCH_TYPES_H
+#define HEXAGON_HEX_ARCH_TYPES_H
 
 #include "qemu/osdep.h"
 #include "mmvec/mmvec.h"
diff --git a/target/hexagon/hex_regs.h b/target/hexagon/hex_regs.h
index e1b3149b07..a63c2c0fd5 100644
--- a/target/hexagon/hex_regs.h
+++ b/target/hexagon/hex_regs.h
@@ -15,8 +15,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef HEXAGON_REGS_H
-#define HEXAGON_REGS_H
+#ifndef HEXAGON_HEX_REGS_H
+#define HEXAGON_HEX_REGS_H
 
 enum {
     HEX_REG_R00              = 0,
diff --git a/target/i386/hax/hax-accel-ops.h b/target/i386/hax/hax-accel-ops.h
index c7698519cd..9e357e7b40 100644
--- a/target/i386/hax/hax-accel-ops.h
+++ b/target/i386/hax/hax-accel-ops.h
@@ -7,8 +7,8 @@
  * See the COPYING file in the top-level directory.
  */
 
-#ifndef HAX_CPUS_H
-#define HAX_CPUS_H
+#ifndef TARGET_I386_HAX_ACCEL_OPS_H
+#define TARGET_I386_HAX_ACCEL_OPS_H
 
 #include "sysemu/cpus.h"
 
@@ -28,4 +28,4 @@ int hax_vcpu_destroy(CPUState *cpu);
 void hax_raise_event(CPUState *cpu);
 void hax_reset_vcpu_state(void *opaque);
 
-#endif /* HAX_CPUS_H */
+#endif /* TARGET_I386_HAX_ACCEL_OPS_H */
diff --git a/target/i386/nvmm/nvmm-accel-ops.h b/target/i386/nvmm/nvmm-accel-ops.h
index 43e24adcaf..7c5461bd75 100644
--- a/target/i386/nvmm/nvmm-accel-ops.h
+++ b/target/i386/nvmm/nvmm-accel-ops.h
@@ -7,8 +7,8 @@
  * See the COPYING file in the top-level directory.
  */
 
-#ifndef NVMM_CPUS_H
-#define NVMM_CPUS_H
+#ifndef TARGET_I386_NVMM_ACCEL_OPS_H
+#define TARGET_I386_NVMM_ACCEL_OPS_H
 
 #include "sysemu/cpus.h"
 
@@ -21,4 +21,4 @@ void nvmm_cpu_synchronize_post_reset(CPUState *cpu);
 void nvmm_cpu_synchronize_post_init(CPUState *cpu);
 void nvmm_cpu_synchronize_pre_loadvm(CPUState *cpu);
 
-#endif /* NVMM_CPUS_H */
+#endif /* TARGET_I386_NVMM_ACCEL_OPS_H */
diff --git a/target/i386/sev.h b/target/i386/sev.h
index 83e82aa42c..7b1528248a 100644
--- a/target/i386/sev.h
+++ b/target/i386/sev.h
@@ -11,8 +11,8 @@
  *
  */
 
-#ifndef QEMU_SEV_I386_H
-#define QEMU_SEV_I386_H
+#ifndef I386_SEV_H
+#define I386_SEV_H
 
 #ifndef CONFIG_USER_ONLY
 #include CONFIG_DEVICES /* CONFIG_SEV */
diff --git a/target/i386/whpx/whpx-accel-ops.h b/target/i386/whpx/whpx-accel-ops.h
index b5102dd1ee..7a1bb1ab57 100644
--- a/target/i386/whpx/whpx-accel-ops.h
+++ b/target/i386/whpx/whpx-accel-ops.h
@@ -7,8 +7,8 @@
  * See the COPYING file in the top-level directory.
  */
 
-#ifndef WHPX_CPUS_H
-#define WHPX_CPUS_H
+#ifndef TARGET_I386_WHPX_ACCEL_OPS_H
+#define TARGET_I386_WHPX_ACCEL_OPS_H
 
 #include "sysemu/cpus.h"
 
@@ -30,4 +30,4 @@ void whpx_cpu_synchronize_pre_resume(bool step_pending);
 /* full state set, modified during initialization or on vmload */
 #define WHPX_SET_FULL_STATE      3
 
-#endif /* WHPX_CPUS_H */
+#endif /* TARGET_I386_WHPX_ACCEL_OPS_H */
diff --git a/target/i386/whpx/whpx-internal.h b/target/i386/whpx/whpx-internal.h
index dbb7e7ba82..06429d8ccd 100644
--- a/target/i386/whpx/whpx-internal.h
+++ b/target/i386/whpx/whpx-internal.h
@@ -1,5 +1,5 @@
-#ifndef WHP_INTERNAL_H
-#define WHP_INTERNAL_H
+#ifndef TARGET_I386_WHPX_INTERNAL_H
+#define TARGET_I386_WHPX_INTERNAL_H
 
 #include <windows.h>
 #include <WinHvPlatform.h>
@@ -116,4 +116,4 @@ typedef enum WHPFunctionList {
     WINHV_PLATFORM_FNS_SUPPLEMENTAL
 } WHPFunctionList;
 
-#endif /* WHP_INTERNAL_H */
+#endif /* TARGET_I386_WHPX_INTERNAL_H */
diff --git a/target/xtensa/core-de233_fpu/core-isa.h b/target/xtensa/core-de233_fpu/core-isa.h
index f125619e8d..40543b2c5e 100644
--- a/target/xtensa/core-de233_fpu/core-isa.h
+++ b/target/xtensa/core-de233_fpu/core-isa.h
@@ -28,8 +28,8 @@
    TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
    SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.  */
 
-#ifndef XTENSA_CORE_CONFIGURATION_H_
-#define XTENSA_CORE_CONFIGURATION_H_
+#ifndef XTENSA_CORE_DE233_FPU_CORE_ISA_H
+#define XTENSA_CORE_DE233_FPU_CORE_ISA_H
 
 //depot/dev/Homewood/Xtensa/SWConfig/hal/core-common.h.tph#24 - edit change 444323 (text+ko)
 
@@ -723,5 +723,4 @@
 #endif /* !XTENSA_HAL_NON_PRIVILEGED_ONLY */
 
 
-#endif /* XTENSA_CORE_CONFIGURATION_H_ */
-
+#endif /* XTENSA_CORE_DE233_FPU_CORE_ISA_H */
diff --git a/target/xtensa/core-dsp3400/core-isa.h b/target/xtensa/core-dsp3400/core-isa.h
index 336b2467c6..1499ef2914 100644
--- a/target/xtensa/core-dsp3400/core-isa.h
+++ b/target/xtensa/core-dsp3400/core-isa.h
@@ -28,9 +28,8 @@
    TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
    SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.  */
 
-#ifndef _XTENSA_CORE_CONFIGURATION_H
-#define _XTENSA_CORE_CONFIGURATION_H
-
+#ifndef XTENSA_CORE_DSP3400_CORE_ISA_H
+#define XTENSA_CORE_DSP3400_CORE_ISA_H
 
 /****************************************************************************
 	    Parameters Useful for Any Code, USER or PRIVILEGED
@@ -448,5 +447,4 @@
 #endif /* !XTENSA_HAL_NON_PRIVILEGED_ONLY */
 
 
-#endif /* _XTENSA_CORE_CONFIGURATION_H */
-
+#endif /* XTENSA_CORE_DSP3400_CORE_ISA_H */
diff --git a/tests/qtest/fuzz/fuzz.h b/tests/qtest/fuzz/fuzz.h
index c5f0b7227a..327c1c5a55 100644
--- a/tests/qtest/fuzz/fuzz.h
+++ b/tests/qtest/fuzz/fuzz.h
@@ -11,8 +11,8 @@
  *
  */
 
-#ifndef FUZZER_H_
-#define FUZZER_H_
+#ifndef QTEST_FUZZ_H
+#define QTEST_FUZZ_H
 
 #include "qemu/units.h"
 #include "qapi/error.h"
@@ -122,4 +122,3 @@ int LLVMFuzzerTestOneInput(const unsigned char *Data, size_t Size);
 int LLVMFuzzerInitialize(int *argc, char ***argv, char ***envp);
 
 #endif
-
diff --git a/tools/virtiofsd/passthrough_seccomp.h b/tools/virtiofsd/passthrough_seccomp.h
index a3ab073f08..12674fc050 100644
--- a/tools/virtiofsd/passthrough_seccomp.h
+++ b/tools/virtiofsd/passthrough_seccomp.h
@@ -6,10 +6,9 @@
  * SPDX-License-Identifier: GPL-2.0-or-later
  */
 
-#ifndef VIRTIOFSD_SECCOMP_H
-#define VIRTIOFSD_SECCOMP_H
-
+#ifndef VIRTIOFSD_PASSTHROUGH_SECCOMP_H
+#define VIRTIOFSD_PASSTHROUGH_SECCOMP_H
 
 void setup_seccomp(bool enable_syslog);
 
-#endif /* VIRTIOFSD_SECCOMP_H */
+#endif /* VIRTIOFSD_PASSTHROUGH_SECCOMP_H */

From 9c0928045ce4b97fc59cb285b7a4b559d21e5da0 Mon Sep 17 00:00:00 2001
From: Markus Armbruster <armbru@redhat.com>
Date: Fri, 6 May 2022 15:49:09 +0200
Subject: [PATCH 228/935] Clean up ill-advised or unusual header guards

Leading underscores are ill-advised because such identifiers are
reserved.  Trailing underscores are merely ugly.  Strip both.

Our header guards commonly end in _H.  Normalize the exceptions.

Macros should be ALL_CAPS.  Normalize the exception.

Done with scripts/clean-header-guards.pl.

include/hw/xen/interface/ and tools/virtiofsd/ left alone, because
these were imported from Xen and libfuse respectively.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220506134911.2856099-3-armbru@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
---
 block/copy-on-read.h                   | 6 +++---
 bsd-user/arm/target_arch.h             | 6 +++---
 bsd-user/arm/target_arch_cpu.h         | 6 +++---
 bsd-user/arm/target_arch_elf.h         | 7 ++++---
 bsd-user/arm/target_arch_reg.h         | 6 +++---
 bsd-user/arm/target_arch_signal.h      | 7 ++++---
 bsd-user/arm/target_arch_sigtramp.h    | 6 +++---
 bsd-user/arm/target_arch_sysarch.h     | 6 +++---
 bsd-user/arm/target_arch_thread.h      | 7 ++++---
 bsd-user/arm/target_arch_vmparam.h     | 7 ++++---
 bsd-user/bsd-file.h                    | 6 +++---
 bsd-user/errno_defs.h                  | 6 +++---
 bsd-user/freebsd/host-os.h             | 6 +++---
 bsd-user/freebsd/target_os_elf.h       | 7 ++++---
 bsd-user/freebsd/target_os_siginfo.h   | 7 ++++---
 bsd-user/freebsd/target_os_signal.h    | 6 +++---
 bsd-user/freebsd/target_os_stack.h     | 6 +++---
 bsd-user/freebsd/target_os_thread.h    | 6 +++---
 bsd-user/freebsd/target_os_user.h      | 6 +++---
 bsd-user/freebsd/target_os_vmparam.h   | 7 ++++---
 bsd-user/i386/target_arch.h            | 6 +++---
 bsd-user/i386/target_arch_cpu.h        | 6 +++---
 bsd-user/i386/target_arch_elf.h        | 7 ++++---
 bsd-user/i386/target_arch_reg.h        | 6 +++---
 bsd-user/i386/target_arch_sigtramp.h   | 6 +++---
 bsd-user/i386/target_arch_thread.h     | 7 ++++---
 bsd-user/i386/target_arch_vmparam.h    | 7 ++++---
 bsd-user/netbsd/host-os.h              | 6 +++---
 bsd-user/netbsd/target_os_elf.h        | 7 ++++---
 bsd-user/netbsd/target_os_siginfo.h    | 6 +++---
 bsd-user/netbsd/target_os_signal.h     | 6 +++---
 bsd-user/netbsd/target_os_stack.h      | 6 +++---
 bsd-user/netbsd/target_os_thread.h     | 6 +++---
 bsd-user/openbsd/host-os.h             | 6 +++---
 bsd-user/openbsd/target_os_elf.h       | 7 ++++---
 bsd-user/openbsd/target_os_siginfo.h   | 6 +++---
 bsd-user/openbsd/target_os_signal.h    | 6 +++---
 bsd-user/openbsd/target_os_stack.h     | 6 +++---
 bsd-user/openbsd/target_os_thread.h    | 6 +++---
 bsd-user/syscall_defs.h                | 6 +++---
 bsd-user/x86_64/target_arch.h          | 6 +++---
 bsd-user/x86_64/target_arch_cpu.h      | 6 +++---
 bsd-user/x86_64/target_arch_elf.h      | 7 ++++---
 bsd-user/x86_64/target_arch_reg.h      | 6 +++---
 bsd-user/x86_64/target_arch_signal.h   | 7 ++++---
 bsd-user/x86_64/target_arch_sigtramp.h | 6 +++---
 bsd-user/x86_64/target_arch_thread.h   | 7 ++++---
 bsd-user/x86_64/target_arch_vmparam.h  | 7 ++++---
 crypto/ivgen-plain.h                   | 6 +++---
 include/chardev/char-socket.h          | 7 ++++---
 include/hw/i2c/i2c_mux_pca954x.h       | 4 ++--
 include/hw/input/lm832x.h              | 4 ++--
 include/hw/intc/exynos4210_combiner.h  | 4 ++--
 include/hw/intc/nios2_vic.h            | 6 +++---
 include/hw/ppc/pnv_pnor.h              | 8 +++++---
 include/hw/s390x/s390-pci-clp.h        | 4 ++--
 include/hw/tricore/tc27x_soc.h         | 4 ++--
 include/hw/virtio/vhost-user-fs.h      | 6 +++---
 include/hw/virtio/vhost-user-i2c.h     | 6 +++---
 include/hw/virtio/vhost-user-rng.h     | 6 +++---
 include/hw/virtio/vhost-user-vsock.h   | 6 +++---
 include/hw/virtio/vhost-vsock-common.h | 6 +++---
 include/qemu/cpu-float.h               | 6 +++---
 include/qemu/crc-ccitt.h               | 6 +++---
 include/qemu/keyval.h                  | 7 ++++---
 include/ui/dbus-display.h              | 6 +++---
 include/ui/dbus-module.h               | 6 +++---
 target/ppc/power8-pmu.h                | 4 ++--
 target/riscv/sbi_ecall_interface.h     | 4 ++--
 tests/qtest/libqmp.h                   | 7 ++++---
 tests/qtest/migration-helpers.h        | 7 ++++---
 ui/dbus.h                              | 7 ++++---
 72 files changed, 232 insertions(+), 209 deletions(-)

diff --git a/block/copy-on-read.h b/block/copy-on-read.h
index 7bf405dccd..1d8ad38c74 100644
--- a/block/copy-on-read.h
+++ b/block/copy-on-read.h
@@ -22,11 +22,11 @@
  * along with this program. If not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef BLOCK_COPY_ON_READ
-#define BLOCK_COPY_ON_READ
+#ifndef BLOCK_COPY_ON_READ_H
+#define BLOCK_COPY_ON_READ_H
 
 #include "block/block_int.h"
 
 void bdrv_cor_filter_drop(BlockDriverState *cor_filter_bs);
 
-#endif /* BLOCK_COPY_ON_READ */
+#endif /* BLOCK_COPY_ON_READ_H */
diff --git a/bsd-user/arm/target_arch.h b/bsd-user/arm/target_arch.h
index 93cfaea098..561934bbd2 100644
--- a/bsd-user/arm/target_arch.h
+++ b/bsd-user/arm/target_arch.h
@@ -17,12 +17,12 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_ARCH_H_
-#define _TARGET_ARCH_H_
+#ifndef TARGET_ARCH_H
+#define TARGET_ARCH_H
 
 #include "qemu.h"
 
 void target_cpu_set_tls(CPUARMState *env, target_ulong newtls);
 target_ulong target_cpu_get_tls(CPUARMState *env);
 
-#endif /* !_TARGET_ARCH_H_ */
+#endif /* TARGET_ARCH_H */
diff --git a/bsd-user/arm/target_arch_cpu.h b/bsd-user/arm/target_arch_cpu.h
index afb7814a8d..517d008764 100644
--- a/bsd-user/arm/target_arch_cpu.h
+++ b/bsd-user/arm/target_arch_cpu.h
@@ -17,8 +17,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_ARCH_CPU_H_
-#define _TARGET_ARCH_CPU_H_
+#ifndef TARGET_ARCH_CPU_H
+#define TARGET_ARCH_CPU_H
 
 #include "target_arch.h"
 #include "signal-common.h"
@@ -210,4 +210,4 @@ static inline void target_cpu_reset(CPUArchState *env)
 {
 }
 
-#endif /* !_TARGET_ARCH_CPU_H */
+#endif /* TARGET_ARCH_CPU_H */
diff --git a/bsd-user/arm/target_arch_elf.h b/bsd-user/arm/target_arch_elf.h
index 4a0215d02e..935bce347f 100644
--- a/bsd-user/arm/target_arch_elf.h
+++ b/bsd-user/arm/target_arch_elf.h
@@ -16,8 +16,9 @@
  *  You should have received a copy of the GNU General Public License
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
-#ifndef _TARGET_ARCH_ELF_H_
-#define _TARGET_ARCH_ELF_H_
+
+#ifndef TARGET_ARCH_ELF_H
+#define TARGET_ARCH_ELF_H
 
 #define ELF_START_MMAP 0x80000000
 #define ELF_ET_DYN_LOAD_ADDR    0x500000
@@ -125,4 +126,4 @@ static uint32_t get_elf_hwcap2(void)
 #undef GET_FEATURE
 #undef GET_FEATURE_ID
 
-#endif /* _TARGET_ARCH_ELF_H_ */
+#endif /* TARGET_ARCH_ELF_H */
diff --git a/bsd-user/arm/target_arch_reg.h b/bsd-user/arm/target_arch_reg.h
index ef5ed5154f..070fa24da1 100644
--- a/bsd-user/arm/target_arch_reg.h
+++ b/bsd-user/arm/target_arch_reg.h
@@ -17,8 +17,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_ARCH_REG_H_
-#define _TARGET_ARCH_REG_H_
+#ifndef TARGET_ARCH_REG_H
+#define TARGET_ARCH_REG_H
 
 /* See sys/arm/include/reg.h */
 typedef struct target_reg {
@@ -57,4 +57,4 @@ static inline void target_copy_regs(target_reg_t *regs, const CPUARMState *env)
 
 #undef tswapreg
 
-#endif /* !_TARGET_ARCH_REG_H_ */
+#endif /* TARGET_ARCH_REG_H */
diff --git a/bsd-user/arm/target_arch_signal.h b/bsd-user/arm/target_arch_signal.h
index f1844dbf22..02b2b33e07 100644
--- a/bsd-user/arm/target_arch_signal.h
+++ b/bsd-user/arm/target_arch_signal.h
@@ -16,8 +16,9 @@
  *  You should have received a copy of the GNU General Public License
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
-#ifndef _TARGET_ARCH_SIGNAL_H_
-#define _TARGET_ARCH_SIGNAL_H_
+
+#ifndef TARGET_ARCH_SIGNAL_H
+#define TARGET_ARCH_SIGNAL_H
 
 #include "cpu.h"
 
@@ -85,4 +86,4 @@ struct target_sigframe {
     target_mcontext_vfp_t sf_vfp; /* actual saved VFP context */
 };
 
-#endif /* !_TARGET_ARCH_SIGNAL_H_ */
+#endif /* TARGET_ARCH_SIGNAL_H */
diff --git a/bsd-user/arm/target_arch_sigtramp.h b/bsd-user/arm/target_arch_sigtramp.h
index 5d434a9e7e..06198045ed 100644
--- a/bsd-user/arm/target_arch_sigtramp.h
+++ b/bsd-user/arm/target_arch_sigtramp.h
@@ -17,8 +17,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_ARCH_SIGTRAMP_H_
-#define _TARGET_ARCH_SIGTRAMP_H_
+#ifndef TARGET_ARCH_SIGTRAMP_H
+#define TARGET_ARCH_SIGTRAMP_H
 
 /* Compare to arm/arm/locore.S ENTRY_NP(sigcode) */
 static inline abi_long setup_sigtramp(abi_ulong offset, unsigned sigf_uc,
@@ -46,4 +46,4 @@ static inline abi_long setup_sigtramp(abi_ulong offset, unsigned sigf_uc,
 
     return memcpy_to_target(offset, sigtramp_code, TARGET_SZSIGCODE);
 }
-#endif /* _TARGET_ARCH_SIGTRAMP_H_ */
+#endif /* TARGET_ARCH_SIGTRAMP_H */
diff --git a/bsd-user/arm/target_arch_sysarch.h b/bsd-user/arm/target_arch_sysarch.h
index 8cc6bff207..5cb7864197 100644
--- a/bsd-user/arm/target_arch_sysarch.h
+++ b/bsd-user/arm/target_arch_sysarch.h
@@ -17,8 +17,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_ARCH_SYSARCH_H_
-#define _TARGET_ARCH_SYSARCH_H_
+#ifndef TARGET_ARCH_SYSARCH_H
+#define TARGET_ARCH_SYSARCH_H
 
 #include "target_syscall.h"
 #include "target_arch.h"
@@ -75,4 +75,4 @@ static inline void do_freebsd_arch_print_sysarch(
     }
 }
 
-#endif /*!_TARGET_ARCH_SYSARCH_H_ */
+#endif /* TARGET_ARCH_SYSARCH_H */
diff --git a/bsd-user/arm/target_arch_thread.h b/bsd-user/arm/target_arch_thread.h
index fcafca2408..fd257f313d 100644
--- a/bsd-user/arm/target_arch_thread.h
+++ b/bsd-user/arm/target_arch_thread.h
@@ -16,8 +16,9 @@
  *  You should have received a copy of the GNU General Public License
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
-#ifndef _TARGET_ARCH_THREAD_H_
-#define _TARGET_ARCH_THREAD_H_
+
+#ifndef TARGET_ARCH_THREAD_H
+#define TARGET_ARCH_THREAD_H
 
 /* Compare to arm/arm/vm_machdep.c cpu_set_upcall_kse() */
 static inline void target_thread_set_upcall(CPUARMState *env, abi_ulong entry,
@@ -77,4 +78,4 @@ static inline void target_thread_init(struct target_pt_regs *regs,
      */
 }
 
-#endif /* !_TARGET_ARCH_THREAD_H_ */
+#endif /* TARGET_ARCH_THREAD_H */
diff --git a/bsd-user/arm/target_arch_vmparam.h b/bsd-user/arm/target_arch_vmparam.h
index 4bbc04ddf5..3fb69aff51 100644
--- a/bsd-user/arm/target_arch_vmparam.h
+++ b/bsd-user/arm/target_arch_vmparam.h
@@ -16,8 +16,9 @@
  *  You should have received a copy of the GNU General Public License
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
-#ifndef _TARGET_ARCH_VMPARAM_H_
-#define _TARGET_ARCH_VMPARAM_H_
+
+#ifndef TARGET_ARCH_VMPARAM_H
+#define TARGET_ARCH_VMPARAM_H
 
 #include "cpu.h"
 
@@ -45,4 +46,4 @@ static inline void set_second_rval(CPUARMState *state, abi_ulong retval2)
     state->regs[1] = retval2;
 }
 
-#endif  /* ! _TARGET_ARCH_VMPARAM_H_ */
+#endif /* TARGET_ARCH_VMPARAM_H */
diff --git a/bsd-user/bsd-file.h b/bsd-user/bsd-file.h
index f0c3f347ec..a6bff3b8c2 100644
--- a/bsd-user/bsd-file.h
+++ b/bsd-user/bsd-file.h
@@ -17,8 +17,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef BSD_FILE_H_
-#define BSD_FILE_H_
+#ifndef BSD_FILE_H
+#define BSD_FILE_H
 
 #include "qemu/path.h"
 
@@ -27,4 +27,4 @@ extern struct iovec *lock_iovec(int type, abi_ulong target_addr, int count,
 extern void unlock_iovec(struct iovec *vec, abi_ulong target_addr, int count,
         int copy);
 
-#endif /* !BSD_FILE_H_ */
+#endif /* BSD_FILE_H */
diff --git a/bsd-user/errno_defs.h b/bsd-user/errno_defs.h
index 73cfa24b7f..f3e8ac3488 100644
--- a/bsd-user/errno_defs.h
+++ b/bsd-user/errno_defs.h
@@ -34,8 +34,8 @@
  *      @(#)errno.h     8.5 (Berkeley) 1/21/94
  */
 
-#ifndef _ERRNO_DEFS_H_
-#define _ERRNO_DEFS_H_
+#ifndef ERRNO_DEFS_H
+#define ERRNO_DEFS_H
 
 #define TARGET_EPERM            1               /* Operation not permitted */
 #define TARGET_ENOENT           2               /* No such file or directory */
@@ -157,4 +157,4 @@
 _Static_assert(TARGET_ERESTART == QEMU_ERESTARTSYS,
                "TARGET_ERESTART and QEMU_ERESTARTSYS expected to match");
 
-#endif /* !  _ERRNO_DEFS_H_ */
+#endif /* ERRNO_DEFS_H */
diff --git a/bsd-user/freebsd/host-os.h b/bsd-user/freebsd/host-os.h
index dfb8344b7b..40cae72ec9 100644
--- a/bsd-user/freebsd/host-os.h
+++ b/bsd-user/freebsd/host-os.h
@@ -17,9 +17,9 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _HOST_OS_H_
-#define _HOST_OS_H_
+#ifndef HOST_OS_H
+#define HOST_OS_H
 
 #define HOST_DEFAULT_BSD_TYPE target_freebsd
 
-#endif /*!_HOST_OS_H_ */
+#endif /* HOST_OS_H */
diff --git a/bsd-user/freebsd/target_os_elf.h b/bsd-user/freebsd/target_os_elf.h
index e5ac8e8e50..9df17d56d8 100644
--- a/bsd-user/freebsd/target_os_elf.h
+++ b/bsd-user/freebsd/target_os_elf.h
@@ -16,8 +16,9 @@
  *  You should have received a copy of the GNU General Public License
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
-#ifndef _TARGET_OS_ELF_H_
-#define _TARGET_OS_ELF_H_
+
+#ifndef TARGET_OS_ELF_H
+#define TARGET_OS_ELF_H
 
 #include "target_arch_elf.h"
 #include "elf.h"
@@ -134,4 +135,4 @@ static abi_ulong target_create_elf_tables(abi_ulong p, int argc, int envc,
         return sp;
 }
 
-#endif /* _TARGET_OS_ELF_H_ */
+#endif /* TARGET_OS_ELF_H */
diff --git a/bsd-user/freebsd/target_os_siginfo.h b/bsd-user/freebsd/target_os_siginfo.h
index d50a3034a8..4573738752 100644
--- a/bsd-user/freebsd/target_os_siginfo.h
+++ b/bsd-user/freebsd/target_os_siginfo.h
@@ -16,8 +16,9 @@
  *  You should have received a copy of the GNU General Public License
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
-#ifndef _TARGET_OS_SIGINFO_H_
-#define _TARGET_OS_SIGINFO_H_
+
+#ifndef TARGET_OS_SIGINFO_H
+#define TARGET_OS_SIGINFO_H
 
 #define TARGET_NSIG         128
 #define TARGET_NSIG_BPW     (sizeof(uint32_t) * 8)
@@ -155,4 +156,4 @@ struct target_sigevent {
 #define TARGET_FPE_FLTINV   (7) /* Invalid floating point operation. */
 #define TARGET_FPE_FLTSUB   (8) /* Subscript out of range. */
 
-#endif /* !_TARGET_OS_SIGINFO_H_ */
+#endif /* TARGET_OS_SIGINFO_H */
diff --git a/bsd-user/freebsd/target_os_signal.h b/bsd-user/freebsd/target_os_signal.h
index 43700d08f7..5030abb52b 100644
--- a/bsd-user/freebsd/target_os_signal.h
+++ b/bsd-user/freebsd/target_os_signal.h
@@ -1,5 +1,5 @@
-#ifndef _TARGET_OS_SIGNAL_H_
-#define _TARGET_OS_SIGNAL_H_
+#ifndef TARGET_OS_SIGNAL_H
+#define TARGET_OS_SIGNAL_H
 
 #include "target_os_siginfo.h"
 #include "target_arch_signal.h"
@@ -78,4 +78,4 @@ abi_long setup_sigframe_arch(CPUArchState *env, abi_ulong frame_addr,
 #define TARGET_SS_ONSTACK 0x0001  /* take signals on alternate stack */
 #define TARGET_SS_DISABLE 0x0004  /* disable taking signals on alternate stack*/
 
-#endif /* !_TARGET_OS_SIGNAL_H_ */
+#endif /* TARGET_OS_SIGNAL_H */
diff --git a/bsd-user/freebsd/target_os_stack.h b/bsd-user/freebsd/target_os_stack.h
index 1bb1a2bf56..0590133291 100644
--- a/bsd-user/freebsd/target_os_stack.h
+++ b/bsd-user/freebsd/target_os_stack.h
@@ -17,8 +17,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_OS_STACK_H_
-#define _TARGET_OS_STACK_H_
+#ifndef TARGET_OS_STACK_H
+#define TARGET_OS_STACK_H
 
 #include <sys/param.h>
 #include "target_arch_sigtramp.h"
@@ -178,4 +178,4 @@ static inline int setup_initial_stack(struct bsd_binprm *bprm,
     return 0;
  }
 
-#endif /* !_TARGET_OS_STACK_H_ */
+#endif /* TARGET_OS_STACK_H */
diff --git a/bsd-user/freebsd/target_os_thread.h b/bsd-user/freebsd/target_os_thread.h
index 77433acdff..1b32cebd26 100644
--- a/bsd-user/freebsd/target_os_thread.h
+++ b/bsd-user/freebsd/target_os_thread.h
@@ -17,9 +17,9 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_OS_THREAD_H_
-#define _TARGET_OS_THREAD_H_
+#ifndef TARGET_OS_THREAD_H
+#define TARGET_OS_THREAD_H
 
 #include "target_arch_thread.h"
 
-#endif /* !_TARGET_OS_THREAD_H_ */
+#endif /* TARGET_OS_THREAD_H */
diff --git a/bsd-user/freebsd/target_os_user.h b/bsd-user/freebsd/target_os_user.h
index 19892c5071..f036a32343 100644
--- a/bsd-user/freebsd/target_os_user.h
+++ b/bsd-user/freebsd/target_os_user.h
@@ -17,8 +17,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_OS_USER_H_
-#define _TARGET_OS_USER_H_
+#ifndef TARGET_OS_USER_H
+#define TARGET_OS_USER_H
 
 /*
  * from sys/priority.h
@@ -326,4 +326,4 @@ struct target_kinfo_vmentry {
     char  kve_path[PATH_MAX];  /* Path to VM obj, if any. */
 };
 
-#endif /* ! _TARGET_OS_USER_H_ */
+#endif /* TARGET_OS_USER_H */
diff --git a/bsd-user/freebsd/target_os_vmparam.h b/bsd-user/freebsd/target_os_vmparam.h
index 990300c619..8457dd3913 100644
--- a/bsd-user/freebsd/target_os_vmparam.h
+++ b/bsd-user/freebsd/target_os_vmparam.h
@@ -16,8 +16,9 @@
  *  You should have received a copy of the GNU General Public License
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
-#ifndef _TARGET_OS_VMPARAM_H_
-#define _TARGET_OS_VMPARAM_H_
+
+#ifndef TARGET_OS_VMPARAM_H
+#define TARGET_OS_VMPARAM_H
 
 #include "target_arch_vmparam.h"
 
@@ -35,4 +36,4 @@ extern abi_ulong target_stksiz;
 #define TARGET_PS_STRINGS  ((target_stkbas + target_stksiz) - \
                             sizeof(struct target_ps_strings))
 
-#endif /* !TARGET_OS_VMPARAM_H_ */
+#endif /* TARGET_OS_VMPARAM_H */
diff --git a/bsd-user/i386/target_arch.h b/bsd-user/i386/target_arch.h
index 73e9a028fe..9595e60f09 100644
--- a/bsd-user/i386/target_arch.h
+++ b/bsd-user/i386/target_arch.h
@@ -17,8 +17,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_ARCH_H_
-#define _TARGET_ARCH_H_
+#ifndef TARGET_ARCH_H
+#define TARGET_ARCH_H
 
 /* target_arch_cpu.c */
 void bsd_i386_write_dt(void *ptr, unsigned long addr, unsigned long limit,
@@ -28,4 +28,4 @@ void bsd_i386_set_idt_base(uint64_t base);
 
 #define target_cpu_set_tls(env, newtls)
 
-#endif /* ! _TARGET_ARCH_H_ */
+#endif /* TARGET_ARCH_H */
diff --git a/bsd-user/i386/target_arch_cpu.h b/bsd-user/i386/target_arch_cpu.h
index 9da22202d4..d792dc720f 100644
--- a/bsd-user/i386/target_arch_cpu.h
+++ b/bsd-user/i386/target_arch_cpu.h
@@ -16,8 +16,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_ARCH_CPU_H_
-#define _TARGET_ARCH_CPU_H_
+#ifndef TARGET_ARCH_CPU_H
+#define TARGET_ARCH_CPU_H
 
 #include "target_arch.h"
 #include "signal-common.h"
@@ -195,4 +195,4 @@ static inline void target_cpu_reset(CPUArchState *env)
     cpu_reset(env_cpu(env));
 }
 
-#endif /* ! _TARGET_ARCH_CPU_H_ */
+#endif /* TARGET_ARCH_CPU_H */
diff --git a/bsd-user/i386/target_arch_elf.h b/bsd-user/i386/target_arch_elf.h
index eb760e07fa..cbcd1f08e2 100644
--- a/bsd-user/i386/target_arch_elf.h
+++ b/bsd-user/i386/target_arch_elf.h
@@ -16,8 +16,9 @@
  *  You should have received a copy of the GNU General Public License
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
-#ifndef _TARGET_ARCH_ELF_H_
-#define _TARGET_ARCH_ELF_H_
+
+#ifndef TARGET_ARCH_ELF_H
+#define TARGET_ARCH_ELF_H
 
 #define ELF_START_MMAP 0x80000000
 #define ELF_ET_DYN_LOAD_ADDR    0x01001000
@@ -32,4 +33,4 @@
 #define USE_ELF_CORE_DUMP
 #define ELF_EXEC_PAGESIZE       4096
 
-#endif /* _TARGET_ARCH_ELF_H_ */
+#endif /* TARGET_ARCH_ELF_H */
diff --git a/bsd-user/i386/target_arch_reg.h b/bsd-user/i386/target_arch_reg.h
index 1fce1daf01..8123109697 100644
--- a/bsd-user/i386/target_arch_reg.h
+++ b/bsd-user/i386/target_arch_reg.h
@@ -18,8 +18,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_ARCH_REG_H_
-#define _TARGET_ARCH_REG_H_
+#ifndef TARGET_ARCH_REG_H
+#define TARGET_ARCH_REG_H
 
 /* See sys/i386/include/reg.h */
 typedef struct target_reg {
@@ -79,4 +79,4 @@ static inline void target_copy_regs(target_reg_t *regs, const CPUX86State *env)
     regs->r_gs = env->segs[R_GS].selector & 0xffff;
 }
 
-#endif /* !_TARGET_ARCH_REG_H_ */
+#endif /* TARGET_ARCH_REG_H */
diff --git a/bsd-user/i386/target_arch_sigtramp.h b/bsd-user/i386/target_arch_sigtramp.h
index cb4e89b0b0..ef94cc864f 100644
--- a/bsd-user/i386/target_arch_sigtramp.h
+++ b/bsd-user/i386/target_arch_sigtramp.h
@@ -17,8 +17,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_ARCH_SIGTRAMP_H_
-#define _TARGET_ARCH_SIGTRAMP_H_
+#ifndef TARGET_ARCH_SIGTRAMP_H
+#define TARGET_ARCH_SIGTRAMP_H
 
 static inline abi_long setup_sigtramp(abi_ulong offset, unsigned sigf_uc,
         unsigned sys_sigreturn)
@@ -26,4 +26,4 @@ static inline abi_long setup_sigtramp(abi_ulong offset, unsigned sigf_uc,
 
     return 0;
 }
-#endif /* _TARGET_ARCH_SIGTRAMP_H_ */
+#endif /* TARGET_ARCH_SIGTRAMP_H */
diff --git a/bsd-user/i386/target_arch_thread.h b/bsd-user/i386/target_arch_thread.h
index e65e476f75..cee2148d94 100644
--- a/bsd-user/i386/target_arch_thread.h
+++ b/bsd-user/i386/target_arch_thread.h
@@ -16,8 +16,9 @@
  *  You should have received a copy of the GNU General Public License
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
-#ifndef _TARGET_ARCH_THREAD_H_
-#define _TARGET_ARCH_THREAD_H_
+
+#ifndef TARGET_ARCH_THREAD_H
+#define TARGET_ARCH_THREAD_H
 
 /* Compare to vm_machdep.c cpu_set_upcall_kse() */
 static inline void target_thread_set_upcall(CPUX86State *regs, abi_ulong entry,
@@ -44,4 +45,4 @@ static inline void target_thread_init(struct target_pt_regs *regs,
     regs->edx = 0;
 }
 
-#endif /* !_TARGET_ARCH_THREAD_H_ */
+#endif /* TARGET_ARCH_THREAD_H */
diff --git a/bsd-user/i386/target_arch_vmparam.h b/bsd-user/i386/target_arch_vmparam.h
index bb7718265b..79db420e59 100644
--- a/bsd-user/i386/target_arch_vmparam.h
+++ b/bsd-user/i386/target_arch_vmparam.h
@@ -16,8 +16,9 @@
  *  You should have received a copy of the GNU General Public License
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
-#ifndef _TARGET_ARCH_VMPARAM_H_
-#define _TARGET_ARCH_VMPARAM_H_
+
+#ifndef TARGET_ARCH_VMPARAM_H
+#define TARGET_ARCH_VMPARAM_H
 
 #include "cpu.h"
 
@@ -43,4 +44,4 @@ static inline void set_second_rval(CPUX86State *state, abi_ulong retval2)
     state->regs[R_EDX] = retval2;
 }
 
-#endif /* !_TARGET_ARCH_VMPARAM_H_ */
+#endif /* TARGET_ARCH_VMPARAM_H */
diff --git a/bsd-user/netbsd/host-os.h b/bsd-user/netbsd/host-os.h
index c0be51a7ef..7c14b1ea78 100644
--- a/bsd-user/netbsd/host-os.h
+++ b/bsd-user/netbsd/host-os.h
@@ -17,9 +17,9 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _HOST_OS_H_
-#define _HOST_OS_H_
+#ifndef HOST_OS_H
+#define HOST_OS_H
 
 #define HOST_DEFAULT_BSD_TYPE target_netbsd
 
-#endif /*!_HOST_OS_H_ */
+#endif /* HOST_OS_H */
diff --git a/bsd-user/netbsd/target_os_elf.h b/bsd-user/netbsd/target_os_elf.h
index 21b475f458..2f3cb20871 100644
--- a/bsd-user/netbsd/target_os_elf.h
+++ b/bsd-user/netbsd/target_os_elf.h
@@ -16,8 +16,9 @@
  *  You should have received a copy of the GNU General Public License
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
-#ifndef _TARGET_OS_ELF_H_
-#define _TARGET_OS_ELF_H_
+
+#ifndef TARGET_OS_ELF_H
+#define TARGET_OS_ELF_H
 
 #include "target_arch_elf.h"
 #include "elf.h"
@@ -143,4 +144,4 @@ static abi_ulong target_create_elf_tables(abi_ulong p, int argc, int envc,
         return sp;
 }
 
-#endif /* _TARGET_OS_ELF_H_ */
+#endif /* TARGET_OS_ELF_H */
diff --git a/bsd-user/netbsd/target_os_siginfo.h b/bsd-user/netbsd/target_os_siginfo.h
index 667c19cc7c..eb57e0a309 100644
--- a/bsd-user/netbsd/target_os_siginfo.h
+++ b/bsd-user/netbsd/target_os_siginfo.h
@@ -1,5 +1,5 @@
-#ifndef _TARGET_OS_SIGINFO_H_
-#define _TARGET_OS_SIGINFO_H_
+#ifndef TARGET_OS_SIGINFO_H
+#define TARGET_OS_SIGINFO_H
 
 #define TARGET_NSIG     32  /* counting 0; could be 33 (mask is 1-32) */
 #define TARGET_NSIG_BPW     (sizeof(uint32_t) * 8)
@@ -79,4 +79,4 @@ typedef union target_siginfo {
 #define TARGET_TRAP_TRACE   2
 
 
-#endif /* ! _TARGET_OS_SIGINFO_H_ */
+#endif /* TARGET_OS_SIGINFO_H */
diff --git a/bsd-user/netbsd/target_os_signal.h b/bsd-user/netbsd/target_os_signal.h
index a373922f7e..4ee4f768e0 100644
--- a/bsd-user/netbsd/target_os_signal.h
+++ b/bsd-user/netbsd/target_os_signal.h
@@ -1,5 +1,5 @@
-#ifndef _TARGET_OS_SIGNAL_H_
-#define _TARGET_OS_SIGNAL_H_
+#ifndef TARGET_OS_SIGNAL_H
+#define TARGET_OS_SIGNAL_H
 
 #include "target_os_siginfo.h"
 #include "target_arch_signal.h"
@@ -66,4 +66,4 @@
 #define TARGET_SS_ONSTACK 0x0001 /* take signals on alternate stack */
 #define TARGET_SS_DISABLE 0x0004 /* disable taking signals on alternate stack */
 
-#endif /* !_TARGET_OS_SIGNAL_H_ */
+#endif /* TARGET_OS_SIGNAL_H */
diff --git a/bsd-user/netbsd/target_os_stack.h b/bsd-user/netbsd/target_os_stack.h
index 503279c1a9..8349e9149b 100644
--- a/bsd-user/netbsd/target_os_stack.h
+++ b/bsd-user/netbsd/target_os_stack.h
@@ -17,8 +17,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_OS_STACK_H_
-#define _TARGET_OS_STACK_H_
+#ifndef TARGET_OS_STACK_H
+#define TARGET_OS_STACK_H
 
 #include "target_arch_sigtramp.h"
 
@@ -53,4 +53,4 @@ static inline int setup_initial_stack(struct bsd_binprm *bprm, abi_ulong *p,
     return 0;
 }
 
-#endif /* !_TARGET_OS_STACK_H_ */
+#endif /* TARGET_OS_STACK_H */
diff --git a/bsd-user/netbsd/target_os_thread.h b/bsd-user/netbsd/target_os_thread.h
index 904dd1bf78..8ccfa16e4b 100644
--- a/bsd-user/netbsd/target_os_thread.h
+++ b/bsd-user/netbsd/target_os_thread.h
@@ -17,9 +17,9 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_OS_THREAD_H_
-#define _TARGET_OS_THREAD_H_
+#ifndef TARGET_OS_THREAD_H
+#define TARGET_OS_THREAD_H
 
 #include "target_arch_thread.h"
 
-#endif /* !_TARGET_OS_THREAD_H_ */
+#endif /* TARGET_OS_THREAD_H */
diff --git a/bsd-user/openbsd/host-os.h b/bsd-user/openbsd/host-os.h
index eb8fdf1567..b9222335d4 100644
--- a/bsd-user/openbsd/host-os.h
+++ b/bsd-user/openbsd/host-os.h
@@ -17,9 +17,9 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _HOST_OS_H_
-#define _HOST_OS_H_
+#ifndef HOST_OS_H
+#define HOST_OS_H
 
 #define HOST_DEFAULT_BSD_TYPE target_openbsd
 
-#endif /*!_HOST_OS_H_ */
+#endif /* HOST_OS_H */
diff --git a/bsd-user/openbsd/target_os_elf.h b/bsd-user/openbsd/target_os_elf.h
index a5cfcd3aff..6dca9c5a85 100644
--- a/bsd-user/openbsd/target_os_elf.h
+++ b/bsd-user/openbsd/target_os_elf.h
@@ -16,8 +16,9 @@
  *  You should have received a copy of the GNU General Public License
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
-#ifndef _TARGET_OS_ELF_H_
-#define _TARGET_OS_ELF_H_
+
+#ifndef TARGET_OS_ELF_H
+#define TARGET_OS_ELF_H
 
 #include "target_arch_elf.h"
 #include "elf.h"
@@ -143,4 +144,4 @@ static abi_ulong target_create_elf_tables(abi_ulong p, int argc, int envc,
         return sp;
 }
 
-#endif /* _TARGET_OS_ELF_H_ */
+#endif /* TARGET_OS_ELF_H */
diff --git a/bsd-user/openbsd/target_os_siginfo.h b/bsd-user/openbsd/target_os_siginfo.h
index baf646a5ab..732009a820 100644
--- a/bsd-user/openbsd/target_os_siginfo.h
+++ b/bsd-user/openbsd/target_os_siginfo.h
@@ -1,5 +1,5 @@
-#ifndef _TARGET_OS_SIGINFO_H_
-#define _TARGET_OS_SIGINFO_H_
+#ifndef TARGET_OS_SIGINFO_H
+#define TARGET_OS_SIGINFO_H
 
 #define TARGET_NSIG     32   /* counting 0; could be 33 (mask is 1-32) */
 #define TARGET_NSIG_BPW     (sizeof(uint32_t) * 8)
@@ -79,4 +79,4 @@ typedef union target_siginfo {
 #define TARGET_TRAP_TRACE   2
 
 
-#endif /* ! _TARGET_OS_SIGINFO_H_ */
+#endif /* TARGET_OS_SIGINFO_H */
diff --git a/bsd-user/openbsd/target_os_signal.h b/bsd-user/openbsd/target_os_signal.h
index a373922f7e..4ee4f768e0 100644
--- a/bsd-user/openbsd/target_os_signal.h
+++ b/bsd-user/openbsd/target_os_signal.h
@@ -1,5 +1,5 @@
-#ifndef _TARGET_OS_SIGNAL_H_
-#define _TARGET_OS_SIGNAL_H_
+#ifndef TARGET_OS_SIGNAL_H
+#define TARGET_OS_SIGNAL_H
 
 #include "target_os_siginfo.h"
 #include "target_arch_signal.h"
@@ -66,4 +66,4 @@
 #define TARGET_SS_ONSTACK 0x0001 /* take signals on alternate stack */
 #define TARGET_SS_DISABLE 0x0004 /* disable taking signals on alternate stack */
 
-#endif /* !_TARGET_OS_SIGNAL_H_ */
+#endif /* TARGET_OS_SIGNAL_H */
diff --git a/bsd-user/openbsd/target_os_stack.h b/bsd-user/openbsd/target_os_stack.h
index 4b37955d3b..264a658608 100644
--- a/bsd-user/openbsd/target_os_stack.h
+++ b/bsd-user/openbsd/target_os_stack.h
@@ -17,8 +17,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_OS_STACK_H_
-#define _TARGET_OS_STACK_H_
+#ifndef TARGET_OS_STACK_H
+#define TARGET_OS_STACK_H
 
 #include "target_arch_sigtramp.h"
 
@@ -53,4 +53,4 @@ static inline int setup_initial_stack(struct bsd_binprm *bprm, abi_ulong *p,
     return 0;
 }
 
-#endif /* !_TARGET_OS_STACK_H_ */
+#endif /* TARGET_OS_STACK_H */
diff --git a/bsd-user/openbsd/target_os_thread.h b/bsd-user/openbsd/target_os_thread.h
index 01ed0d9fc8..c3adc6712f 100644
--- a/bsd-user/openbsd/target_os_thread.h
+++ b/bsd-user/openbsd/target_os_thread.h
@@ -17,9 +17,9 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_OS_THREAD_H_
-#define _TARGET_OS_THREAD_H_
+#ifndef TARGET_OS_THREAD_H
+#define TARGET_OS_THREAD_H
 
 #include "target_arch_thread.h"
 
-#endif /* !_TARGET_OS_THREAD_H_ */
+#endif /* TARGET_OS_THREAD_H */
diff --git a/bsd-user/syscall_defs.h b/bsd-user/syscall_defs.h
index c3bf14f38f..f5797b28e3 100644
--- a/bsd-user/syscall_defs.h
+++ b/bsd-user/syscall_defs.h
@@ -17,8 +17,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _SYSCALL_DEFS_H_
-#define _SYSCALL_DEFS_H_
+#ifndef SYSCALL_DEFS_H
+#define SYSCALL_DEFS_H
 
 #include <sys/syscall.h>
 #include <sys/resource.h>
@@ -226,4 +226,4 @@ type safe_##name(type1 arg1, type2 arg2, type3 arg3, type4 arg4, \
     return safe_syscall(SYS_##name, arg1, arg2, arg3, arg4, arg5, arg6); \
 }
 
-#endif /* ! _SYSCALL_DEFS_H_ */
+#endif /* SYSCALL_DEFS_H */
diff --git a/bsd-user/x86_64/target_arch.h b/bsd-user/x86_64/target_arch.h
index e558e1b956..09bd974889 100644
--- a/bsd-user/x86_64/target_arch.h
+++ b/bsd-user/x86_64/target_arch.h
@@ -17,8 +17,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_ARCH_H_
-#define _TARGET_ARCH_H_
+#ifndef TARGET_ARCH_H
+#define TARGET_ARCH_H
 
 /* target_arch_cpu.c */
 void bsd_x86_64_write_dt(void *ptr, unsigned long addr, unsigned long limit,
@@ -28,4 +28,4 @@ void bsd_x86_64_set_idt_base(uint64_t base);
 
 #define target_cpu_set_tls(env, newtls)
 
-#endif /* !_TARGET_ARCH_H_ */
+#endif /* TARGET_ARCH_H */
diff --git a/bsd-user/x86_64/target_arch_cpu.h b/bsd-user/x86_64/target_arch_cpu.h
index 5be2f02416..4094d61da1 100644
--- a/bsd-user/x86_64/target_arch_cpu.h
+++ b/bsd-user/x86_64/target_arch_cpu.h
@@ -16,8 +16,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_ARCH_CPU_H_
-#define _TARGET_ARCH_CPU_H_
+#ifndef TARGET_ARCH_CPU_H
+#define TARGET_ARCH_CPU_H
 
 #include "target_arch.h"
 #include "signal-common.h"
@@ -174,4 +174,4 @@ static inline void target_cpu_reset(CPUArchState *env)
     cpu_reset(env_cpu(env));
 }
 
-#endif /* ! _TARGET_ARCH_CPU_H_ */
+#endif /* TARGET_ARCH_CPU_H */
diff --git a/bsd-user/x86_64/target_arch_elf.h b/bsd-user/x86_64/target_arch_elf.h
index c2f8553962..b244711888 100644
--- a/bsd-user/x86_64/target_arch_elf.h
+++ b/bsd-user/x86_64/target_arch_elf.h
@@ -16,8 +16,9 @@
  *  You should have received a copy of the GNU General Public License
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
-#ifndef _TARGET_ARCH_ELF_H_
-#define _TARGET_ARCH_ELF_H_
+
+#ifndef TARGET_ARCH_ELF_H
+#define TARGET_ARCH_ELF_H
 
 #define ELF_START_MMAP 0x2aaaaab000ULL
 #define ELF_ET_DYN_LOAD_ADDR    0x01021000
@@ -32,4 +33,4 @@
 #define USE_ELF_CORE_DUMP
 #define ELF_EXEC_PAGESIZE       4096
 
-#endif /* _TARGET_ARCH_ELF_H_ */
+#endif /* TARGET_ARCH_ELF_H */
diff --git a/bsd-user/x86_64/target_arch_reg.h b/bsd-user/x86_64/target_arch_reg.h
index 00e9624517..7a766de918 100644
--- a/bsd-user/x86_64/target_arch_reg.h
+++ b/bsd-user/x86_64/target_arch_reg.h
@@ -18,8 +18,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_ARCH_REG_H_
-#define _TARGET_ARCH_REG_H_
+#ifndef TARGET_ARCH_REG_H
+#define TARGET_ARCH_REG_H
 
 /* See sys/amd64/include/reg.h */
 typedef struct target_reg {
@@ -89,4 +89,4 @@ static inline void target_copy_regs(target_reg_t *regs, const CPUX86State *env)
     regs->r_ss = env->segs[R_SS].selector & 0xffff;
 }
 
-#endif /* !_TARGET_ARCH_REG_H_ */
+#endif /* TARGET_ARCH_REG_H */
diff --git a/bsd-user/x86_64/target_arch_signal.h b/bsd-user/x86_64/target_arch_signal.h
index b4a0ebf2bd..ca24bf1e7f 100644
--- a/bsd-user/x86_64/target_arch_signal.h
+++ b/bsd-user/x86_64/target_arch_signal.h
@@ -15,8 +15,9 @@
  *  You should have received a copy of the GNU General Public License
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
-#ifndef _TARGET_ARCH_SIGNAL_H_
-#define _TARGET_ARCH_SIGNAL_H_
+
+#ifndef TARGET_ARCH_SIGNAL_H
+#define TARGET_ARCH_SIGNAL_H
 
 #include "cpu.h"
 
@@ -96,4 +97,4 @@ struct target_sigframe {
     uint32_t    __spare__[2];
 };
 
-#endif /* !TARGET_ARCH_SIGNAL_H_ */
+#endif /* TARGET_ARCH_SIGNAL_H */
diff --git a/bsd-user/x86_64/target_arch_sigtramp.h b/bsd-user/x86_64/target_arch_sigtramp.h
index 29d4a8b55f..01da614098 100644
--- a/bsd-user/x86_64/target_arch_sigtramp.h
+++ b/bsd-user/x86_64/target_arch_sigtramp.h
@@ -17,8 +17,8 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef _TARGET_ARCH_SIGTRAMP_H_
-#define _TARGET_ARCH_SIGTRAMP_H_
+#ifndef TARGET_ARCH_SIGTRAMP_H
+#define TARGET_ARCH_SIGTRAMP_H
 
 static inline abi_long setup_sigtramp(abi_ulong offset, unsigned sigf_uc,
         unsigned sys_sigreturn)
@@ -26,4 +26,4 @@ static inline abi_long setup_sigtramp(abi_ulong offset, unsigned sigf_uc,
 
     return 0;
 }
-#endif /* _TARGET_ARCH_SIGTRAMP_H_ */
+#endif /* TARGET_ARCH_SIGTRAMP_H */
diff --git a/bsd-user/x86_64/target_arch_thread.h b/bsd-user/x86_64/target_arch_thread.h
index b745d7ffeb..52c28906d6 100644
--- a/bsd-user/x86_64/target_arch_thread.h
+++ b/bsd-user/x86_64/target_arch_thread.h
@@ -16,8 +16,9 @@
  *  You should have received a copy of the GNU General Public License
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
-#ifndef _TARGET_ARCH_THREAD_H_
-#define _TARGET_ARCH_THREAD_H_
+
+#ifndef TARGET_ARCH_THREAD_H
+#define TARGET_ARCH_THREAD_H
 
 /* Compare to vm_machdep.c cpu_set_upcall_kse() */
 static inline void target_thread_set_upcall(CPUX86State *regs, abi_ulong entry,
@@ -35,4 +36,4 @@ static inline void target_thread_init(struct target_pt_regs *regs,
     regs->rdi = infop->start_stack;
 }
 
-#endif /* !_TARGET_ARCH_THREAD_H_ */
+#endif /* TARGET_ARCH_THREAD_H */
diff --git a/bsd-user/x86_64/target_arch_vmparam.h b/bsd-user/x86_64/target_arch_vmparam.h
index 81a915f2e5..6797623a6b 100644
--- a/bsd-user/x86_64/target_arch_vmparam.h
+++ b/bsd-user/x86_64/target_arch_vmparam.h
@@ -16,8 +16,9 @@
  *  You should have received a copy of the GNU General Public License
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
-#ifndef _TARGET_ARCH_VMPARAM_H_
-#define _TARGET_ARCH_VMPARAM_H_
+
+#ifndef TARGET_ARCH_VMPARAM_H
+#define TARGET_ARCH_VMPARAM_H
 
 #include "cpu.h"
 
@@ -43,4 +44,4 @@ static inline void set_second_rval(CPUX86State *state, abi_ulong retval2)
     state->regs[R_EDX] = retval2;
 }
 
-#endif /* !_TARGET_ARCH_VMPARAM_H_ */
+#endif /* TARGET_ARCH_VMPARAM_H */
diff --git a/crypto/ivgen-plain.h b/crypto/ivgen-plain.h
index 43db898809..ebae6acd04 100644
--- a/crypto/ivgen-plain.h
+++ b/crypto/ivgen-plain.h
@@ -17,11 +17,11 @@
  * License along with this library; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef QCRYPTO_IVGEN_PLAIN_H__
-#define QCRYPTO_IVGEN_PLAIN_H__
+#ifndef QCRYPTO_IVGEN_PLAIN_H
+#define QCRYPTO_IVGEN_PLAIN_H
 
 #include "ivgenpriv.h"
 
 extern struct QCryptoIVGenDriver qcrypto_ivgen_plain;
 
-#endif /* QCRYPTO_IVGEN_PLAIN_H__ */
+#endif /* QCRYPTO_IVGEN_PLAIN_H */
diff --git a/include/chardev/char-socket.h b/include/chardev/char-socket.h
index 6b6e2ceba1..0708ca6fa9 100644
--- a/include/chardev/char-socket.h
+++ b/include/chardev/char-socket.h
@@ -21,8 +21,9 @@
  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  * THE SOFTWARE.
  */
-#ifndef CHAR_SOCKET_H_
-#define CHAR_SOCKET_H_
+
+#ifndef CHAR_SOCKET_H
+#define CHAR_SOCKET_H
 
 #include "io/channel-socket.h"
 #include "io/channel-tls.h"
@@ -83,4 +84,4 @@ typedef struct SocketChardev SocketChardev;
 DECLARE_INSTANCE_CHECKER(SocketChardev, SOCKET_CHARDEV,
                          TYPE_CHARDEV_SOCKET)
 
-#endif /* CHAR_SOCKET_H_ */
+#endif /* CHAR_SOCKET_H */
diff --git a/include/hw/i2c/i2c_mux_pca954x.h b/include/hw/i2c/i2c_mux_pca954x.h
index 8aaf9bbc39..3dd25ec983 100644
--- a/include/hw/i2c/i2c_mux_pca954x.h
+++ b/include/hw/i2c/i2c_mux_pca954x.h
@@ -1,5 +1,5 @@
-#ifndef QEMU_I2C_MUX_PCA954X
-#define QEMU_I2C_MUX_PCA954X
+#ifndef QEMU_I2C_MUX_PCA954X_H
+#define QEMU_I2C_MUX_PCA954X_H
 
 #include "hw/i2c/i2c.h"
 
diff --git a/include/hw/input/lm832x.h b/include/hw/input/lm832x.h
index 2a58ccf891..e0e5d5ef20 100644
--- a/include/hw/input/lm832x.h
+++ b/include/hw/input/lm832x.h
@@ -18,8 +18,8 @@
  * with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef HW_INPUT_LM832X
-#define HW_INPUT_LM832X
+#ifndef HW_INPUT_LM832X_H
+#define HW_INPUT_LM832X_H
 
 #define TYPE_LM8323 "lm8323"
 
diff --git a/include/hw/intc/exynos4210_combiner.h b/include/hw/intc/exynos4210_combiner.h
index 429844fed4..bd207a7e6e 100644
--- a/include/hw/intc/exynos4210_combiner.h
+++ b/include/hw/intc/exynos4210_combiner.h
@@ -20,8 +20,8 @@
  * with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef HW_INTC_EXYNOS4210_COMBINER
-#define HW_INTC_EXYNOS4210_COMBINER
+#ifndef HW_INTC_EXYNOS4210_COMBINER_H
+#define HW_INTC_EXYNOS4210_COMBINER_H
 
 #include "hw/sysbus.h"
 
diff --git a/include/hw/intc/nios2_vic.h b/include/hw/intc/nios2_vic.h
index af1517a967..ac507b9d74 100644
--- a/include/hw/intc/nios2_vic.h
+++ b/include/hw/intc/nios2_vic.h
@@ -32,8 +32,8 @@
  * THE SOFTWARE.
  */
 
-#ifndef HW_INTC_NIOS2_VIC
-#define HW_INTC_NIOS2_VIC
+#ifndef HW_INTC_NIOS2_VIC_H
+#define HW_INTC_NIOS2_VIC_H
 
 #define TYPE_NIOS2_VIC "nios2-vic"
 OBJECT_DECLARE_SIMPLE_TYPE(Nios2VIC, NIOS2_VIC)
@@ -61,4 +61,4 @@ struct Nios2VIC {
     uint32_t vec_tbl_addr;
 };
 
-#endif /* HW_INTC_NIOS2_VIC */
+#endif /* HW_INTC_NIOS2_VIC_H */
diff --git a/include/hw/ppc/pnv_pnor.h b/include/hw/ppc/pnv_pnor.h
index 99f9a3adfb..bab2f79844 100644
--- a/include/hw/ppc/pnv_pnor.h
+++ b/include/hw/ppc/pnv_pnor.h
@@ -6,8 +6,10 @@
  * This code is licensed under the GPL version 2 or later. See the
  * COPYING file in the top-level directory.
  */
-#ifndef _PPC_PNV_PNOR_H
-#define _PPC_PNV_PNOR_H
+
+#ifndef PPC_PNV_PNOR_H
+#define PPC_PNV_PNOR_H
+
 #include "qom/object.h"
 
 /*
@@ -28,4 +30,4 @@ struct PnvPnor {
     MemoryRegion   mmio;
 };
 
-#endif /* _PPC_PNV_PNOR_H */
+#endif /* PPC_PNV_PNOR_H */
diff --git a/include/hw/s390x/s390-pci-clp.h b/include/hw/s390x/s390-pci-clp.h
index cc8c8662b8..03b7f9ba5f 100644
--- a/include/hw/s390x/s390-pci-clp.h
+++ b/include/hw/s390x/s390-pci-clp.h
@@ -9,8 +9,8 @@
  * directory.
  */
 
-#ifndef HW_S390_PCI_CLP
-#define HW_S390_PCI_CLP
+#ifndef HW_S390_PCI_CLP_H
+#define HW_S390_PCI_CLP_H
 
 /* CLP common request & response block size */
 #define CLP_BLK_SIZE 4096
diff --git a/include/hw/tricore/tc27x_soc.h b/include/hw/tricore/tc27x_soc.h
index 6a7e5b54f5..dd3a7485c8 100644
--- a/include/hw/tricore/tc27x_soc.h
+++ b/include/hw/tricore/tc27x_soc.h
@@ -18,8 +18,8 @@
  * License along with this library; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef TC27X_SoC_H
-#define TC27X_SoC_H
+#ifndef TC27X_SOC_H
+#define TC27X_SOC_H
 
 #include "hw/sysbus.h"
 #include "target/tricore/cpu.h"
diff --git a/include/hw/virtio/vhost-user-fs.h b/include/hw/virtio/vhost-user-fs.h
index 0d62834c25..94c3aaa84e 100644
--- a/include/hw/virtio/vhost-user-fs.h
+++ b/include/hw/virtio/vhost-user-fs.h
@@ -11,8 +11,8 @@
  * top-level directory.
  */
 
-#ifndef _QEMU_VHOST_USER_FS_H
-#define _QEMU_VHOST_USER_FS_H
+#ifndef QEMU_VHOST_USER_FS_H
+#define QEMU_VHOST_USER_FS_H
 
 #include "hw/virtio/virtio.h"
 #include "hw/virtio/vhost.h"
@@ -44,4 +44,4 @@ struct VHostUserFS {
     /*< public >*/
 };
 
-#endif /* _QEMU_VHOST_USER_FS_H */
+#endif /* QEMU_VHOST_USER_FS_H */
diff --git a/include/hw/virtio/vhost-user-i2c.h b/include/hw/virtio/vhost-user-i2c.h
index d8372f3b43..0f7acd40e3 100644
--- a/include/hw/virtio/vhost-user-i2c.h
+++ b/include/hw/virtio/vhost-user-i2c.h
@@ -6,8 +6,8 @@
  * SPDX-License-Identifier: GPL-2.0-or-later
  */
 
-#ifndef _QEMU_VHOST_USER_I2C_H
-#define _QEMU_VHOST_USER_I2C_H
+#ifndef QEMU_VHOST_USER_I2C_H
+#define QEMU_VHOST_USER_I2C_H
 
 #include "hw/virtio/vhost.h"
 #include "hw/virtio/vhost-user.h"
@@ -28,4 +28,4 @@ struct VHostUserI2C {
 /* Virtio Feature bits */
 #define VIRTIO_I2C_F_ZERO_LENGTH_REQUEST		0
 
-#endif /* _QEMU_VHOST_USER_I2C_H */
+#endif /* QEMU_VHOST_USER_I2C_H */
diff --git a/include/hw/virtio/vhost-user-rng.h b/include/hw/virtio/vhost-user-rng.h
index 071539996d..ddd9f01eea 100644
--- a/include/hw/virtio/vhost-user-rng.h
+++ b/include/hw/virtio/vhost-user-rng.h
@@ -6,8 +6,8 @@
  * SPDX-License-Identifier: GPL-2.0-or-later
  */
 
-#ifndef _QEMU_VHOST_USER_RNG_H
-#define _QEMU_VHOST_USER_RNG_H
+#ifndef QEMU_VHOST_USER_RNG_H
+#define QEMU_VHOST_USER_RNG_H
 
 #include "hw/virtio/virtio.h"
 #include "hw/virtio/vhost.h"
@@ -30,4 +30,4 @@ struct VHostUserRNG {
     /*< public >*/
 };
 
-#endif /* _QEMU_VHOST_USER_RNG_H */
+#endif /* QEMU_VHOST_USER_RNG_H */
diff --git a/include/hw/virtio/vhost-user-vsock.h b/include/hw/virtio/vhost-user-vsock.h
index 4cfd558245..67aa46c952 100644
--- a/include/hw/virtio/vhost-user-vsock.h
+++ b/include/hw/virtio/vhost-user-vsock.h
@@ -8,8 +8,8 @@
  * top-level directory.
  */
 
-#ifndef _QEMU_VHOST_USER_VSOCK_H
-#define _QEMU_VHOST_USER_VSOCK_H
+#ifndef QEMU_VHOST_USER_VSOCK_H
+#define QEMU_VHOST_USER_VSOCK_H
 
 #include "hw/virtio/vhost-vsock-common.h"
 #include "hw/virtio/vhost-user.h"
@@ -33,4 +33,4 @@ struct VHostUserVSock {
     /*< public >*/
 };
 
-#endif /* _QEMU_VHOST_USER_VSOCK_H */
+#endif /* QEMU_VHOST_USER_VSOCK_H */
diff --git a/include/hw/virtio/vhost-vsock-common.h b/include/hw/virtio/vhost-vsock-common.h
index d8b565b4da..456a9c2365 100644
--- a/include/hw/virtio/vhost-vsock-common.h
+++ b/include/hw/virtio/vhost-vsock-common.h
@@ -8,8 +8,8 @@
  * top-level directory.
  */
 
-#ifndef _QEMU_VHOST_VSOCK_COMMON_H
-#define _QEMU_VHOST_VSOCK_COMMON_H
+#ifndef QEMU_VHOST_VSOCK_COMMON_H
+#define QEMU_VHOST_VSOCK_COMMON_H
 
 #include "hw/virtio/virtio.h"
 #include "hw/virtio/vhost.h"
@@ -49,4 +49,4 @@ void vhost_vsock_common_unrealize(VirtIODevice *vdev);
 uint64_t vhost_vsock_common_get_features(VirtIODevice *vdev, uint64_t features,
                                          Error **errp);
 
-#endif /* _QEMU_VHOST_VSOCK_COMMON_H */
+#endif /* QEMU_VHOST_VSOCK_COMMON_H */
diff --git a/include/qemu/cpu-float.h b/include/qemu/cpu-float.h
index 911099499f..a26b9faf68 100644
--- a/include/qemu/cpu-float.h
+++ b/include/qemu/cpu-float.h
@@ -1,5 +1,5 @@
-#ifndef QEMU_CPU_FLOAT_H_
-#define QEMU_CPU_FLOAT_H_
+#ifndef QEMU_CPU_FLOAT_H
+#define QEMU_CPU_FLOAT_H
 
 #include "fpu/softfloat-types.h"
 
@@ -61,4 +61,4 @@ typedef union {
 #endif
 } CPU_QuadU;
 
-#endif /* QEMU_CPU_FLOAT_H_ */
+#endif /* QEMU_CPU_FLOAT_H */
diff --git a/include/qemu/crc-ccitt.h b/include/qemu/crc-ccitt.h
index 06ee55b159..d6eb49146d 100644
--- a/include/qemu/crc-ccitt.h
+++ b/include/qemu/crc-ccitt.h
@@ -11,8 +11,8 @@
  * SPDX-License-Identifier: GPL-2.0
  */
 
-#ifndef _CRC_CCITT_H
-#define _CRC_CCITT_H
+#ifndef CRC_CCITT_H
+#define CRC_CCITT_H
 
 extern uint16_t const crc_ccitt_table[256];
 extern uint16_t const crc_ccitt_false_table[256];
@@ -30,4 +30,4 @@ static inline uint16_t crc_ccitt_false_byte(uint16_t crc, const uint8_t c)
     return (crc << 8) ^ crc_ccitt_false_table[(crc >> 8) ^ c];
 }
 
-#endif /* _CRC_CCITT_H */
+#endif /* CRC_CCITT_H */
diff --git a/include/qemu/keyval.h b/include/qemu/keyval.h
index 2d263286d7..1187b68303 100644
--- a/include/qemu/keyval.h
+++ b/include/qemu/keyval.h
@@ -2,8 +2,9 @@
  * This work is licensed under the terms of the GNU GPL, version 2 or later.
  * See the COPYING file in the top-level directory.
  */
-#ifndef KEYVAL_H_
-#define KEYVAL_H_
+
+#ifndef KEYVAL_H
+#define KEYVAL_H
 
 QDict *keyval_parse_into(QDict *qdict, const char *params, const char *implied_key,
                          bool *p_help, Error **errp);
@@ -11,4 +12,4 @@ QDict *keyval_parse(const char *params, const char *implied_key,
                     bool *help, Error **errp);
 void keyval_merge(QDict *old, const QDict *new, Error **errp);
 
-#endif /* KEYVAL_H_ */
+#endif /* KEYVAL_H */
diff --git a/include/ui/dbus-display.h b/include/ui/dbus-display.h
index 88f153c237..7c9ec1a069 100644
--- a/include/ui/dbus-display.h
+++ b/include/ui/dbus-display.h
@@ -1,5 +1,5 @@
-#ifndef DBUS_DISPLAY_H_
-#define DBUS_DISPLAY_H_
+#ifndef DBUS_DISPLAY_H
+#define DBUS_DISPLAY_H
 
 #include "qapi/error.h"
 #include "ui/dbus-module.h"
@@ -14,4 +14,4 @@ static inline bool qemu_using_dbus_display(Error **errp)
     return true;
 }
 
-#endif /* DBUS_DISPLAY_H_ */
+#endif /* DBUS_DISPLAY_H */
diff --git a/include/ui/dbus-module.h b/include/ui/dbus-module.h
index ace4a17a5c..5442ee0bb6 100644
--- a/include/ui/dbus-module.h
+++ b/include/ui/dbus-module.h
@@ -1,5 +1,5 @@
-#ifndef DBUS_MODULE_H_
-#define DBUS_MODULE_H_
+#ifndef DBUS_MODULE_H
+#define DBUS_MODULE_H
 
 struct QemuDBusDisplayOps {
     bool (*add_client)(int csock, Error **errp);
@@ -8,4 +8,4 @@ struct QemuDBusDisplayOps {
 extern int using_dbus_display;
 extern struct QemuDBusDisplayOps qemu_dbus_display;
 
-#endif /* DBUS_MODULE_H_*/
+#endif /* DBUS_MODULE_H */
diff --git a/target/ppc/power8-pmu.h b/target/ppc/power8-pmu.h
index 256d90f523..9692dd765e 100644
--- a/target/ppc/power8-pmu.h
+++ b/target/ppc/power8-pmu.h
@@ -10,8 +10,8 @@
  * See the COPYING file in the top-level directory.
  */
 
-#ifndef POWER8_PMU
-#define POWER8_PMU
+#ifndef POWER8_PMU_H
+#define POWER8_PMU_H
 
 #if defined(TARGET_PPC64) && !defined(CONFIG_USER_ONLY)
 void cpu_ppc_pmu_init(CPUPPCState *env);
diff --git a/target/riscv/sbi_ecall_interface.h b/target/riscv/sbi_ecall_interface.h
index fb1a3fa8f2..77574ed4cb 100644
--- a/target/riscv/sbi_ecall_interface.h
+++ b/target/riscv/sbi_ecall_interface.h
@@ -7,8 +7,8 @@
  *   Anup Patel <anup.patel@wdc.com>
  */
 
-#ifndef __SBI_ECALL_INTERFACE_H__
-#define __SBI_ECALL_INTERFACE_H__
+#ifndef SBI_ECALL_INTERFACE_H
+#define SBI_ECALL_INTERFACE_H
 
 /* clang-format off */
 
diff --git a/tests/qtest/libqmp.h b/tests/qtest/libqmp.h
index 94aa97328a..5cb7eeaa18 100644
--- a/tests/qtest/libqmp.h
+++ b/tests/qtest/libqmp.h
@@ -14,8 +14,9 @@
  * See the COPYING file in the top-level directory.
  *
  */
-#ifndef LIBQMP_H_
-#define LIBQMP_H_
+
+#ifndef LIBQMP_H
+#define LIBQMP_H
 
 #include "qapi/qmp/qdict.h"
 
@@ -47,4 +48,4 @@ bool qmp_rsp_is_err(QDict *rsp);
  */
 void qmp_expect_error_and_unref(QDict *rsp, const char *class);
 
-#endif /* LIBQMP_H_ */
+#endif /* LIBQMP_H */
diff --git a/tests/qtest/migration-helpers.h b/tests/qtest/migration-helpers.h
index c7872e8442..2731399aaa 100644
--- a/tests/qtest/migration-helpers.h
+++ b/tests/qtest/migration-helpers.h
@@ -9,8 +9,9 @@
  * See the COPYING file in the top-level directory.
  *
  */
-#ifndef MIGRATION_HELPERS_H_
-#define MIGRATION_HELPERS_H_
+
+#ifndef MIGRATION_HELPERS_H
+#define MIGRATION_HELPERS_H
 
 #include "libqtest.h"
 
@@ -34,4 +35,4 @@ void wait_for_migration_complete(QTestState *who);
 
 void wait_for_migration_fail(QTestState *from, bool allow_active);
 
-#endif /* MIGRATION_HELPERS_H_ */
+#endif /* MIGRATION_HELPERS_H */
diff --git a/ui/dbus.h b/ui/dbus.h
index 5f5c1f759c..c001c11f70 100644
--- a/ui/dbus.h
+++ b/ui/dbus.h
@@ -21,8 +21,9 @@
  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  * THE SOFTWARE.
  */
-#ifndef UI_DBUS_H_
-#define UI_DBUS_H_
+
+#ifndef UI_DBUS_H
+#define UI_DBUS_H
 
 #include "chardev/char-socket.h"
 #include "qemu/dbus.h"
@@ -144,4 +145,4 @@ void dbus_chardev_init(DBusDisplay *dpy);
 
 void dbus_clipboard_init(DBusDisplay *dpy);
 
-#endif /* UI_DBUS_H_ */
+#endif /* UI_DBUS_H */

From 4f31b54bfe52c72be9c17c158b38924e6cefcd0f Mon Sep 17 00:00:00 2001
From: Markus Armbruster <armbru@redhat.com>
Date: Fri, 6 May 2022 15:49:10 +0200
Subject: [PATCH 229/935] Normalize header guard symbol definition

We commonly define the header guard symbol without an explicit value.
Normalize the exceptions.

Done with scripts/clean-header-guards.pl.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220506134911.2856099-4-armbru@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/memopidx.h       | 2 +-
 include/tcg/tcg-ldst.h        | 2 +-
 target/alpha/cpu-param.h      | 2 +-
 target/arm/cpu-param.h        | 2 +-
 target/cris/cpu-param.h       | 2 +-
 target/hppa/cpu-param.h       | 2 +-
 target/i386/cpu-param.h       | 2 +-
 target/m68k/cpu-param.h       | 2 +-
 target/microblaze/cpu-param.h | 2 +-
 target/mips/cpu-param.h       | 2 +-
 target/nios2/cpu-param.h      | 2 +-
 target/openrisc/cpu-param.h   | 2 +-
 target/ppc/cpu-param.h        | 2 +-
 target/riscv/cpu-param.h      | 2 +-
 target/s390x/cpu-param.h      | 2 +-
 target/sh4/cpu-param.h        | 2 +-
 target/sparc/cpu-param.h      | 2 +-
 target/tricore/cpu-param.h    | 2 +-
 target/xtensa/cpu-param.h     | 2 +-
 tcg/tcg-internal.h            | 2 +-
 20 files changed, 20 insertions(+), 20 deletions(-)

diff --git a/include/exec/memopidx.h b/include/exec/memopidx.h
index 83bce97874..eb7f1591a3 100644
--- a/include/exec/memopidx.h
+++ b/include/exec/memopidx.h
@@ -9,7 +9,7 @@
  */
 
 #ifndef EXEC_MEMOPIDX_H
-#define EXEC_MEMOPIDX_H 1
+#define EXEC_MEMOPIDX_H
 
 #include "exec/memop.h"
 
diff --git a/include/tcg/tcg-ldst.h b/include/tcg/tcg-ldst.h
index 121a156933..2ba22bd5fe 100644
--- a/include/tcg/tcg-ldst.h
+++ b/include/tcg/tcg-ldst.h
@@ -23,7 +23,7 @@
  */
 
 #ifndef TCG_LDST_H
-#define TCG_LDST_H 1
+#define TCG_LDST_H
 
 #ifdef CONFIG_SOFTMMU
 
diff --git a/target/alpha/cpu-param.h b/target/alpha/cpu-param.h
index 1153992e42..17cd14e590 100644
--- a/target/alpha/cpu-param.h
+++ b/target/alpha/cpu-param.h
@@ -6,7 +6,7 @@
  */
 
 #ifndef ALPHA_CPU_PARAM_H
-#define ALPHA_CPU_PARAM_H 1
+#define ALPHA_CPU_PARAM_H
 
 #define TARGET_LONG_BITS 64
 #define TARGET_PAGE_BITS 13
diff --git a/target/arm/cpu-param.h b/target/arm/cpu-param.h
index b59d505761..68ffb12427 100644
--- a/target/arm/cpu-param.h
+++ b/target/arm/cpu-param.h
@@ -6,7 +6,7 @@
  */
 
 #ifndef ARM_CPU_PARAM_H
-#define ARM_CPU_PARAM_H 1
+#define ARM_CPU_PARAM_H
 
 #ifdef TARGET_AARCH64
 # define TARGET_LONG_BITS             64
diff --git a/target/cris/cpu-param.h b/target/cris/cpu-param.h
index 36a3058761..12ec22d8df 100644
--- a/target/cris/cpu-param.h
+++ b/target/cris/cpu-param.h
@@ -6,7 +6,7 @@
  */
 
 #ifndef CRIS_CPU_PARAM_H
-#define CRIS_CPU_PARAM_H 1
+#define CRIS_CPU_PARAM_H
 
 #define TARGET_LONG_BITS 32
 #define TARGET_PAGE_BITS 13
diff --git a/target/hppa/cpu-param.h b/target/hppa/cpu-param.h
index a97d1428df..a48a2701ae 100644
--- a/target/hppa/cpu-param.h
+++ b/target/hppa/cpu-param.h
@@ -6,7 +6,7 @@
  */
 
 #ifndef HPPA_CPU_PARAM_H
-#define HPPA_CPU_PARAM_H 1
+#define HPPA_CPU_PARAM_H
 
 #ifdef TARGET_HPPA64
 # define TARGET_LONG_BITS             64
diff --git a/target/i386/cpu-param.h b/target/i386/cpu-param.h
index 57abc64c0d..9740bd7abd 100644
--- a/target/i386/cpu-param.h
+++ b/target/i386/cpu-param.h
@@ -6,7 +6,7 @@
  */
 
 #ifndef I386_CPU_PARAM_H
-#define I386_CPU_PARAM_H 1
+#define I386_CPU_PARAM_H
 
 #ifdef TARGET_X86_64
 # define TARGET_LONG_BITS             64
diff --git a/target/m68k/cpu-param.h b/target/m68k/cpu-param.h
index 06556dfbf3..44a8d193f0 100644
--- a/target/m68k/cpu-param.h
+++ b/target/m68k/cpu-param.h
@@ -6,7 +6,7 @@
  */
 
 #ifndef M68K_CPU_PARAM_H
-#define M68K_CPU_PARAM_H 1
+#define M68K_CPU_PARAM_H
 
 #define TARGET_LONG_BITS 32
 /*
diff --git a/target/microblaze/cpu-param.h b/target/microblaze/cpu-param.h
index 4d8297fa94..5e54ea0108 100644
--- a/target/microblaze/cpu-param.h
+++ b/target/microblaze/cpu-param.h
@@ -6,7 +6,7 @@
  */
 
 #ifndef MICROBLAZE_CPU_PARAM_H
-#define MICROBLAZE_CPU_PARAM_H 1
+#define MICROBLAZE_CPU_PARAM_H
 
 /*
  * While system mode can address up to 64 bits of address space,
diff --git a/target/mips/cpu-param.h b/target/mips/cpu-param.h
index 1aebd01df9..f4c76994ea 100644
--- a/target/mips/cpu-param.h
+++ b/target/mips/cpu-param.h
@@ -5,7 +5,7 @@
  */
 
 #ifndef MIPS_CPU_PARAM_H
-#define MIPS_CPU_PARAM_H 1
+#define MIPS_CPU_PARAM_H
 
 #ifdef TARGET_MIPS64
 # define TARGET_LONG_BITS 64
diff --git a/target/nios2/cpu-param.h b/target/nios2/cpu-param.h
index 38bedbfd61..177d720864 100644
--- a/target/nios2/cpu-param.h
+++ b/target/nios2/cpu-param.h
@@ -6,7 +6,7 @@
  */
 
 #ifndef NIOS2_CPU_PARAM_H
-#define NIOS2_CPU_PARAM_H 1
+#define NIOS2_CPU_PARAM_H
 
 #define TARGET_LONG_BITS 32
 #define TARGET_PAGE_BITS 12
diff --git a/target/openrisc/cpu-param.h b/target/openrisc/cpu-param.h
index 06ee64d171..73be699f36 100644
--- a/target/openrisc/cpu-param.h
+++ b/target/openrisc/cpu-param.h
@@ -6,7 +6,7 @@
  */
 
 #ifndef OPENRISC_CPU_PARAM_H
-#define OPENRISC_CPU_PARAM_H 1
+#define OPENRISC_CPU_PARAM_H
 
 #define TARGET_LONG_BITS 32
 #define TARGET_PAGE_BITS 13
diff --git a/target/ppc/cpu-param.h b/target/ppc/cpu-param.h
index 37b458d33d..ea377b7d06 100644
--- a/target/ppc/cpu-param.h
+++ b/target/ppc/cpu-param.h
@@ -6,7 +6,7 @@
  */
 
 #ifndef PPC_CPU_PARAM_H
-#define PPC_CPU_PARAM_H 1
+#define PPC_CPU_PARAM_H
 
 #ifdef TARGET_PPC64
 # define TARGET_LONG_BITS 64
diff --git a/target/riscv/cpu-param.h b/target/riscv/cpu-param.h
index 80eb615f93..ebaf26d26d 100644
--- a/target/riscv/cpu-param.h
+++ b/target/riscv/cpu-param.h
@@ -6,7 +6,7 @@
  */
 
 #ifndef RISCV_CPU_PARAM_H
-#define RISCV_CPU_PARAM_H 1
+#define RISCV_CPU_PARAM_H
 
 #if defined(TARGET_RISCV64)
 # define TARGET_LONG_BITS 64
diff --git a/target/s390x/cpu-param.h b/target/s390x/cpu-param.h
index 472db648d7..bf951a002e 100644
--- a/target/s390x/cpu-param.h
+++ b/target/s390x/cpu-param.h
@@ -6,7 +6,7 @@
  */
 
 #ifndef S390_CPU_PARAM_H
-#define S390_CPU_PARAM_H 1
+#define S390_CPU_PARAM_H
 
 #define TARGET_LONG_BITS 64
 #define TARGET_PAGE_BITS 12
diff --git a/target/sh4/cpu-param.h b/target/sh4/cpu-param.h
index 81ace3503b..98a02509bb 100644
--- a/target/sh4/cpu-param.h
+++ b/target/sh4/cpu-param.h
@@ -6,7 +6,7 @@
  */
 
 #ifndef SH4_CPU_PARAM_H
-#define SH4_CPU_PARAM_H 1
+#define SH4_CPU_PARAM_H
 
 #define TARGET_LONG_BITS 32
 #define TARGET_PAGE_BITS 12  /* 4k */
diff --git a/target/sparc/cpu-param.h b/target/sparc/cpu-param.h
index 4746d89411..72ddc4a34f 100644
--- a/target/sparc/cpu-param.h
+++ b/target/sparc/cpu-param.h
@@ -5,7 +5,7 @@
  */
 
 #ifndef SPARC_CPU_PARAM_H
-#define SPARC_CPU_PARAM_H 1
+#define SPARC_CPU_PARAM_H
 
 #ifdef TARGET_SPARC64
 # define TARGET_LONG_BITS 64
diff --git a/target/tricore/cpu-param.h b/target/tricore/cpu-param.h
index cf5d9af89d..2727913047 100644
--- a/target/tricore/cpu-param.h
+++ b/target/tricore/cpu-param.h
@@ -6,7 +6,7 @@
  */
 
 #ifndef TRICORE_CPU_PARAM_H
-#define TRICORE_CPU_PARAM_H 1
+#define TRICORE_CPU_PARAM_H
 
 #define TARGET_LONG_BITS 32
 #define TARGET_PAGE_BITS 14
diff --git a/target/xtensa/cpu-param.h b/target/xtensa/cpu-param.h
index 4fde21b941..b53e9a3e08 100644
--- a/target/xtensa/cpu-param.h
+++ b/target/xtensa/cpu-param.h
@@ -6,7 +6,7 @@
  */
 
 #ifndef XTENSA_CPU_PARAM_H
-#define XTENSA_CPU_PARAM_H 1
+#define XTENSA_CPU_PARAM_H
 
 #define TARGET_LONG_BITS 32
 #define TARGET_PAGE_BITS 12
diff --git a/tcg/tcg-internal.h b/tcg/tcg-internal.h
index 92c91dcde9..cc82088d52 100644
--- a/tcg/tcg-internal.h
+++ b/tcg/tcg-internal.h
@@ -23,7 +23,7 @@
  */
 
 #ifndef TCG_INTERNAL_H
-#define TCG_INTERNAL_H 1
+#define TCG_INTERNAL_H
 
 #define TCG_HIGHWATER 1024
 

From ea9cea93c69d508a333dd1b0cb1a44f6daf80b63 Mon Sep 17 00:00:00 2001
From: Markus Armbruster <armbru@redhat.com>
Date: Fri, 6 May 2022 15:49:11 +0200
Subject: [PATCH 230/935] Clean up decorations and whitespace around header
 guards

Cleaned up with scripts/clean-header-guards.pl.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220506134911.2856099-5-armbru@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
---
 bsd-user/arm/target.h                      | 2 +-
 bsd-user/x86_64/target.h                   | 2 +-
 chardev/chardev-internal.h                 | 3 ++-
 include/block/block_int-global-state.h     | 3 ++-
 include/exec/translator.h                  | 2 +-
 include/fpu/softfloat-helpers.h            | 2 +-
 include/hw/gpio/aspeed_gpio.h              | 2 +-
 include/hw/intc/rx_icu.h                   | 2 +-
 include/hw/misc/aspeed_hace.h              | 2 +-
 include/hw/misc/aspeed_lpc.h               | 2 +-
 include/hw/misc/aspeed_sbc.h               | 2 +-
 include/hw/net/allwinner-sun8i-emac.h      | 2 +-
 include/hw/rtc/m48t59.h                    | 2 +-
 include/hw/rtc/mc146818rtc.h               | 2 +-
 include/qemu/plugin-memory.h               | 2 +-
 include/qemu/selfmap.h                     | 2 +-
 include/user/syscall-trace.h               | 2 +-
 linux-user/hexagon/target_signal.h         | 2 +-
 target/avr/cpu.h                           | 2 +-
 target/hexagon/attribs.h                   | 2 +-
 target/xtensa/core-de233_fpu/core-matmap.h | 4 +---
 target/xtensa/core-dsp3400/core-matmap.h   | 4 +---
 22 files changed, 24 insertions(+), 26 deletions(-)

diff --git a/bsd-user/arm/target.h b/bsd-user/arm/target.h
index 419c039b68..7c423ec575 100644
--- a/bsd-user/arm/target.h
+++ b/bsd-user/arm/target.h
@@ -17,5 +17,5 @@ static inline bool regpairs_aligned(void *cpu_env)
     return true;
 }
 
-#endif /* ! TARGET_H */
+#endif /* TARGET_H */
 
diff --git a/bsd-user/x86_64/target.h b/bsd-user/x86_64/target.h
index 8956631db1..0cf0e2a14a 100644
--- a/bsd-user/x86_64/target.h
+++ b/bsd-user/x86_64/target.h
@@ -17,5 +17,5 @@ static inline bool regpairs_aligned(void *cpu_env)
     return false;
 }
 
-#endif /* ! TARGET_H */
+#endif /* TARGET_H */
 
diff --git a/chardev/chardev-internal.h b/chardev/chardev-internal.h
index aba0240759..4e03af3147 100644
--- a/chardev/chardev-internal.h
+++ b/chardev/chardev-internal.h
@@ -21,6 +21,7 @@
  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  * THE SOFTWARE.
  */
+
 #ifndef CHARDEV_INTERNAL_H
 #define CHARDEV_INTERNAL_H
 
@@ -64,4 +65,4 @@ void mux_chr_send_all_event(Chardev *chr, QEMUChrEvent event);
 
 Object *get_chardevs_root(void);
 
-#endif /* CHAR_MUX_H */
+#endif /* CHARDEV_INTERNAL_H */
diff --git a/include/block/block_int-global-state.h b/include/block/block_int-global-state.h
index 8b2e95f5ff..b49f4eb35b 100644
--- a/include/block/block_int-global-state.h
+++ b/include/block/block_int-global-state.h
@@ -21,6 +21,7 @@
  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  * THE SOFTWARE.
  */
+
 #ifndef BLOCK_INT_GLOBAL_STATE_H
 #define BLOCK_INT_GLOBAL_STATE_H
 
@@ -326,4 +327,4 @@ static inline void assert_bdrv_graph_writable(BlockDriverState *bs)
     assert(qemu_in_main_thread());
 }
 
-#endif /* BLOCK_INT_GLOBAL_STATE */
+#endif /* BLOCK_INT_GLOBAL_STATE_H */
diff --git a/include/exec/translator.h b/include/exec/translator.h
index 31d3fa76ff..7db6845535 100644
--- a/include/exec/translator.h
+++ b/include/exec/translator.h
@@ -187,4 +187,4 @@ FOR_EACH_TRANSLATOR_LD(GEN_TRANSLATOR_LD)
 
 #undef GEN_TRANSLATOR_LD
 
-#endif  /* EXEC__TRANSLATOR_H */
+#endif /* EXEC__TRANSLATOR_H */
diff --git a/include/fpu/softfloat-helpers.h b/include/fpu/softfloat-helpers.h
index a98d759cd3..94cbe073ec 100644
--- a/include/fpu/softfloat-helpers.h
+++ b/include/fpu/softfloat-helpers.h
@@ -141,4 +141,4 @@ static inline bool get_default_nan_mode(float_status *status)
     return status->default_nan_mode;
 }
 
-#endif /* _SOFTFLOAT_HELPERS_H_ */
+#endif /* SOFTFLOAT_HELPERS_H */
diff --git a/include/hw/gpio/aspeed_gpio.h b/include/hw/gpio/aspeed_gpio.h
index 801846befb..6dee3cd438 100644
--- a/include/hw/gpio/aspeed_gpio.h
+++ b/include/hw/gpio/aspeed_gpio.h
@@ -93,4 +93,4 @@ struct AspeedGPIOState {
     } sets[ASPEED_GPIO_MAX_NR_SETS];
 };
 
-#endif /* _ASPEED_GPIO_H_ */
+#endif /* ASPEED_GPIO_H */
diff --git a/include/hw/intc/rx_icu.h b/include/hw/intc/rx_icu.h
index 7f5889b36f..b23504f3dd 100644
--- a/include/hw/intc/rx_icu.h
+++ b/include/hw/intc/rx_icu.h
@@ -73,4 +73,4 @@ struct RXICUState {
 #define TYPE_RX_ICU "rx-icu"
 OBJECT_DECLARE_SIMPLE_TYPE(RXICUState, RX_ICU)
 
-#endif /* RX_ICU_H */
+#endif /* HW_INTC_RX_ICU_H */
diff --git a/include/hw/misc/aspeed_hace.h b/include/hw/misc/aspeed_hace.h
index 4f9ce179bf..ecb1b67de8 100644
--- a/include/hw/misc/aspeed_hace.h
+++ b/include/hw/misc/aspeed_hace.h
@@ -47,4 +47,4 @@ struct AspeedHACEClass {
     uint32_t hash_mask;
 };
 
-#endif /* _ASPEED_HACE_H_ */
+#endif /* ASPEED_HACE_H */
diff --git a/include/hw/misc/aspeed_lpc.h b/include/hw/misc/aspeed_lpc.h
index df418cfcd3..fd228731d2 100644
--- a/include/hw/misc/aspeed_lpc.h
+++ b/include/hw/misc/aspeed_lpc.h
@@ -44,4 +44,4 @@ typedef struct AspeedLPCState {
     uint32_t hicr7;
 } AspeedLPCState;
 
-#endif /* _ASPEED_LPC_H_ */
+#endif /* ASPEED_LPC_H */
diff --git a/include/hw/misc/aspeed_sbc.h b/include/hw/misc/aspeed_sbc.h
index 651747e28f..67e43b53ec 100644
--- a/include/hw/misc/aspeed_sbc.h
+++ b/include/hw/misc/aspeed_sbc.h
@@ -29,4 +29,4 @@ struct AspeedSBCClass {
     SysBusDeviceClass parent_class;
 };
 
-#endif /* _ASPEED_SBC_H_ */
+#endif /* ASPEED_SBC_H */
diff --git a/include/hw/net/allwinner-sun8i-emac.h b/include/hw/net/allwinner-sun8i-emac.h
index 460a58f1ca..185895f4e1 100644
--- a/include/hw/net/allwinner-sun8i-emac.h
+++ b/include/hw/net/allwinner-sun8i-emac.h
@@ -101,4 +101,4 @@ struct AwSun8iEmacState {
 
 };
 
-#endif /* HW_NET_ALLWINNER_SUN8I_H */
+#endif /* HW_NET_ALLWINNER_SUN8I_EMAC_H */
diff --git a/include/hw/rtc/m48t59.h b/include/hw/rtc/m48t59.h
index d9b45eb161..c14937476c 100644
--- a/include/hw/rtc/m48t59.h
+++ b/include/hw/rtc/m48t59.h
@@ -47,4 +47,4 @@ struct NvramClass {
     void (*toggle_lock)(Nvram *obj, int lock);
 };
 
-#endif /* HW_M48T59_H */
+#endif /* HW_RTC_M48T59_H */
diff --git a/include/hw/rtc/mc146818rtc.h b/include/hw/rtc/mc146818rtc.h
index deef93f89a..33d85753c0 100644
--- a/include/hw/rtc/mc146818rtc.h
+++ b/include/hw/rtc/mc146818rtc.h
@@ -56,4 +56,4 @@ ISADevice *mc146818_rtc_init(ISABus *bus, int base_year,
 void rtc_set_memory(ISADevice *dev, int addr, int val);
 int rtc_get_memory(ISADevice *dev, int addr);
 
-#endif /* MC146818RTC_H */
+#endif /* HW_RTC_MC146818RTC_H */
diff --git a/include/qemu/plugin-memory.h b/include/qemu/plugin-memory.h
index 0f59226727..8ad13c110c 100644
--- a/include/qemu/plugin-memory.h
+++ b/include/qemu/plugin-memory.h
@@ -37,4 +37,4 @@ struct qemu_plugin_hwaddr {
 bool tlb_plugin_lookup(CPUState *cpu, target_ulong addr, int mmu_idx,
                        bool is_store, struct qemu_plugin_hwaddr *data);
 
-#endif /* _PLUGIN_MEMORY_H_ */
+#endif /* PLUGIN_MEMORY_H */
diff --git a/include/qemu/selfmap.h b/include/qemu/selfmap.h
index 80cf920fba..3479a2a618 100644
--- a/include/qemu/selfmap.h
+++ b/include/qemu/selfmap.h
@@ -41,4 +41,4 @@ GSList *read_self_maps(void);
  */
 void free_self_maps(GSList *info);
 
-#endif /* _SELFMAP_H_ */
+#endif /* SELFMAP_H */
diff --git a/include/user/syscall-trace.h b/include/user/syscall-trace.h
index 614cfacfa5..b4e53d3870 100644
--- a/include/user/syscall-trace.h
+++ b/include/user/syscall-trace.h
@@ -39,4 +39,4 @@ static inline void record_syscall_return(void *cpu, int num, abi_long ret)
 }
 
 
-#endif /* _SYSCALL_TRACE_H_ */
+#endif /* SYSCALL_TRACE_H */
diff --git a/linux-user/hexagon/target_signal.h b/linux-user/hexagon/target_signal.h
index 193abac340..68fb71312e 100644
--- a/linux-user/hexagon/target_signal.h
+++ b/linux-user/hexagon/target_signal.h
@@ -22,4 +22,4 @@
 
 #define TARGET_ARCH_HAS_SIGTRAMP_PAGE 1
 
-#endif /* TARGET_SIGNAL_H */
+#endif /* HEXAGON_TARGET_SIGNAL_H */
diff --git a/target/avr/cpu.h b/target/avr/cpu.h
index 55497f851d..d304f33301 100644
--- a/target/avr/cpu.h
+++ b/target/avr/cpu.h
@@ -247,4 +247,4 @@ bool avr_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
 
 #include "exec/cpu-all.h"
 
-#endif /* !defined (QEMU_AVR_CPU_H) */
+#endif /* QEMU_AVR_CPU_H */
diff --git a/target/hexagon/attribs.h b/target/hexagon/attribs.h
index 54576f4143..d51bb4f732 100644
--- a/target/hexagon/attribs.h
+++ b/target/hexagon/attribs.h
@@ -32,4 +32,4 @@ extern DECLARE_BITMAP(opcode_attribs[XX_LAST_OPCODE], A_ZZ_LASTATTRIB);
 #define GET_ATTRIB(opcode, attrib) \
     test_bit(attrib, opcode_attribs[opcode])
 
-#endif /* ATTRIBS_H */
+#endif /* HEXAGON_ATTRIBS_H */
diff --git a/target/xtensa/core-de233_fpu/core-matmap.h b/target/xtensa/core-de233_fpu/core-matmap.h
index cca51c7af1..e99e7d3123 100644
--- a/target/xtensa/core-de233_fpu/core-matmap.h
+++ b/target/xtensa/core-de233_fpu/core-matmap.h
@@ -43,11 +43,9 @@
    TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
    SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.  */
 
-
 #ifndef XTENSA_CONFIG_CORE_MATMAP_H
 #define XTENSA_CONFIG_CORE_MATMAP_H
 
-
 /*----------------------------------------------------------------------
 			CACHE (MEMORY ACCESS) ATTRIBUTES
   ----------------------------------------------------------------------*/
@@ -713,5 +711,5 @@
 
 
 
-#endif /*XTENSA_CONFIG_CORE_MATMAP_H*/
+#endif /* XTENSA_CONFIG_CORE_MATMAP_H */
 
diff --git a/target/xtensa/core-dsp3400/core-matmap.h b/target/xtensa/core-dsp3400/core-matmap.h
index 8d1aa8336e..692012f9f4 100644
--- a/target/xtensa/core-dsp3400/core-matmap.h
+++ b/target/xtensa/core-dsp3400/core-matmap.h
@@ -43,11 +43,9 @@
    TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
    SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.  */
 
-
 #ifndef XTENSA_CONFIG_CORE_MATMAP_H
 #define XTENSA_CONFIG_CORE_MATMAP_H
 
-
 /*----------------------------------------------------------------------
 			CACHE (MEMORY ACCESS) ATTRIBUTES
   ----------------------------------------------------------------------*/
@@ -308,5 +306,5 @@
 
 
 
-#endif /*XTENSA_CONFIG_CORE_MATMAP_H*/
+#endif /* XTENSA_CONFIG_CORE_MATMAP_H */
 

From 43924d1e53fd77c63b6f995556545e12f9fb11b6 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Tue, 29 Mar 2022 13:01:58 +0200
Subject: [PATCH 231/935] pc-bios/optionrom: detect -fno-pie

Do not rely on the detection that was done in the configure script,
since in the future we may want to cross-compile this file.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure                  | 1 -
 pc-bios/optionrom/Makefile | 3 ++-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/configure b/configure
index e77b29b093..c8b5b99532 100755
--- a/configure
+++ b/configure
@@ -2038,7 +2038,6 @@ echo "CCAS=$ccas" >> $config_host_mak
 echo "CPP=$cpp" >> $config_host_mak
 echo "OBJCOPY=$objcopy" >> $config_host_mak
 echo "LD=$ld" >> $config_host_mak
-echo "CFLAGS_NOPIE=$CFLAGS_NOPIE" >> $config_host_mak
 echo "QEMU_CFLAGS=$QEMU_CFLAGS" >> $config_host_mak
 echo "QEMU_CXXFLAGS=$QEMU_CXXFLAGS" >> $config_host_mak
 echo "QEMU_OBJCFLAGS=$QEMU_OBJCFLAGS" >> $config_host_mak
diff --git a/pc-bios/optionrom/Makefile b/pc-bios/optionrom/Makefile
index f1ef898073..8de5a9461c 100644
--- a/pc-bios/optionrom/Makefile
+++ b/pc-bios/optionrom/Makefile
@@ -22,7 +22,8 @@ override CFLAGS += $(call cc-option, -fcf-protection=none)
 override CPPFLAGS += -MMD -MP -MT $@ -MF $(@D)/$(*F).d
 
 override CFLAGS += $(filter -W%, $(QEMU_CFLAGS))
-override CFLAGS += $(CFLAGS_NOPIE) -ffreestanding -I$(TOPSRC_DIR)/include
+override CFLAGS += $(call cc-option, -fno-pie)
+override CFLAGS += -ffreestanding -I$(TOPSRC_DIR)/include
 override CFLAGS += $(call cc-option, -fno-stack-protector)
 override CFLAGS += $(call cc-option, -m16)
 

From 236d15222e06750d6b889030ed04c849b1dc279e Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 13 Apr 2022 14:22:12 +0200
Subject: [PATCH 232/935] pc-bios/optionrom: compile with -Wno-array-bounds
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Avoids the following bogus warning:

pvh_main.c: In function ‘pvh_load_kernel’:
pvh_main.c:101:42: warning: array subscript 0 is outside array bounds of ‘uint16_t[0]’ {aka ‘short unsigned int[]’} [-Warray-bounds]
  101 |         uint32_t ebda_paddr = ((uint32_t)*((uint16_t *)EBDA_BASE_ADDR)) << 4;
      |                                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Cc: qemu-stable@nongnu.org
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 pc-bios/optionrom/Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/pc-bios/optionrom/Makefile b/pc-bios/optionrom/Makefile
index 8de5a9461c..2494ad9c25 100644
--- a/pc-bios/optionrom/Makefile
+++ b/pc-bios/optionrom/Makefile
@@ -26,6 +26,7 @@ override CFLAGS += $(call cc-option, -fno-pie)
 override CFLAGS += -ffreestanding -I$(TOPSRC_DIR)/include
 override CFLAGS += $(call cc-option, -fno-stack-protector)
 override CFLAGS += $(call cc-option, -m16)
+override CFLAGS += $(call cc-option, -Wno-array-bounds)
 
 ifeq ($(filter -m16, $(CFLAGS)),)
 # Attempt to work around compilers that lack -m16 (GCC <= 4.8, clang <= ??)

From 798d8ec0dacd4cc0034298d94f430c14f23e2919 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 29 Apr 2022 21:16:28 +0200
Subject: [PATCH 233/935] target/i386: do not consult nonexistent host leaves

When cache_info_passthrough is requested, QEMU passes the host values
of the cache information CPUID leaves down to the guest.  However,
it blindly assumes that the CPUID leaf exists on the host, and this
cannot be guaranteed: for example, KVM has recently started to
synthesize AMD leaves up to 0x80000021 in order to provide accurate
CPU bug information to guests.

Querying a nonexistent host leaf fills the output arguments of
host_cpuid with data that (albeit deterministic) is nonsensical
as cache information, namely the data in the highest Intel CPUID
leaf.  If said highest leaf is not ECX-dependent, this can even
cause an infinite loop when kvm_arch_init_vcpu prepares the input
to KVM_SET_CPUID2.  The infinite loop is only terminated by an
abort() when the array gets full.

Reported-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Cc: qemu-stable@nongnu.org
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/cpu.c | 41 ++++++++++++++++++++++++++++++++++++-----
 1 file changed, 36 insertions(+), 5 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 62c240fa91..c4a17c93f6 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5022,6 +5022,37 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
     return r;
 }
 
+static void x86_cpu_get_cache_cpuid(uint32_t func, uint32_t index,
+                                    uint32_t *eax, uint32_t *ebx,
+                                    uint32_t *ecx, uint32_t *edx)
+{
+    uint32_t level, unused;
+
+    /* Only return valid host leaves.  */
+    switch (func) {
+    case 2:
+    case 4:
+        host_cpuid(0, 0, &level, &unused, &unused, &unused);
+        break;
+    case 0x80000005:
+    case 0x80000006:
+    case 0x8000001d:
+        host_cpuid(0x80000000, 0, &level, &unused, &unused, &unused);
+        break;
+    default:
+        return;
+    }
+
+    if (func > level) {
+        *eax = 0;
+        *ebx = 0;
+        *ecx = 0;
+        *edx = 0;
+    } else {
+        host_cpuid(func, index, eax, ebx, ecx, edx);
+    }
+}
+
 /*
  * Only for builtin_x86_defs models initialized with x86_register_cpudef_types.
  */
@@ -5280,7 +5311,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
     case 2:
         /* cache info: needed for Pentium Pro compatibility */
         if (cpu->cache_info_passthrough) {
-            host_cpuid(index, 0, eax, ebx, ecx, edx);
+            x86_cpu_get_cache_cpuid(index, 0, eax, ebx, ecx, edx);
             break;
         } else if (cpu->vendor_cpuid_only && IS_AMD_CPU(env)) {
             *eax = *ebx = *ecx = *edx = 0;
@@ -5300,7 +5331,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
     case 4:
         /* cache info: needed for Core compatibility */
         if (cpu->cache_info_passthrough) {
-            host_cpuid(index, count, eax, ebx, ecx, edx);
+            x86_cpu_get_cache_cpuid(index, count, eax, ebx, ecx, edx);
             /* QEMU gives out its own APIC IDs, never pass down bits 31..26.  */
             *eax &= ~0xFC000000;
             if ((*eax & 31) && cs->nr_cores > 1) {
@@ -5702,7 +5733,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
     case 0x80000005:
         /* cache info (L1 cache) */
         if (cpu->cache_info_passthrough) {
-            host_cpuid(index, 0, eax, ebx, ecx, edx);
+            x86_cpu_get_cache_cpuid(index, 0, eax, ebx, ecx, edx);
             break;
         }
         *eax = (L1_DTLB_2M_ASSOC << 24) | (L1_DTLB_2M_ENTRIES << 16) |
@@ -5715,7 +5746,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
     case 0x80000006:
         /* cache info (L2 cache) */
         if (cpu->cache_info_passthrough) {
-            host_cpuid(index, 0, eax, ebx, ecx, edx);
+            x86_cpu_get_cache_cpuid(index, 0, eax, ebx, ecx, edx);
             break;
         }
         *eax = (AMD_ENC_ASSOC(L2_DTLB_2M_ASSOC) << 28) |
@@ -5775,7 +5806,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
     case 0x8000001D:
         *eax = 0;
         if (cpu->cache_info_passthrough) {
-            host_cpuid(index, count, eax, ebx, ecx, edx);
+            x86_cpu_get_cache_cpuid(index, count, eax, ebx, ecx, edx);
             break;
         }
         switch (count) {

From dfdb4f3c28737879685398c4a610a220e1ecf9c2 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 6 Apr 2022 14:27:52 +0200
Subject: [PATCH 234/935] checkpatch: fix g_malloc check

Use the string equality operator "eq", and ensure that $1 is defined by
using "(try|)" instead of "(try)?".  The alternative "((?:try)?)" is
longer and less readable.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 scripts/checkpatch.pl | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 4763d02ae7..d900d18048 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -2831,8 +2831,8 @@ sub process {
 		}
 
 # check for pointless casting of g_malloc return
-		if ($line =~ /\*\s*\)\s*g_(try)?(m|re)alloc(0?)(_n)?\b/) {
-			if ($2 == 'm') {
+		if ($line =~ /\*\s*\)\s*g_(try|)(m|re)alloc(0?)(_n)?\b/) {
+			if ($2 eq 'm') {
 				ERROR("unnecessary cast may hide bugs, use g_$1new$3 instead\n" . $herecurr);
 			} else {
 				ERROR("unnecessary cast may hide bugs, use g_$1renew$3 instead\n" . $herecurr);

From 354d2d9b87658bab5da58b4251aeb612fa7dc6e4 Mon Sep 17 00:00:00 2001
From: Juan Quintela <quintela@redhat.com>
Date: Mon, 2 May 2022 15:11:19 +0200
Subject: [PATCH 235/935] meson: Make mremap() detecting works correctly

Without this (at least in Fedora 35) it don't detect mremap()
correctly.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20220502131119.2345-1-quintela@redhat.com>
[Also switch the LEGACY_RDMA_REG_MR test to cc.links, otherwise
 Debian fails to build. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 meson.build | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/meson.build b/meson.build
index 864e97945f..03f63e95e2 100644
--- a/meson.build
+++ b/meson.build
@@ -2179,7 +2179,8 @@ config_host_data.set('CONFIG_AVX512F_OPT', get_option('avx512f') \
 
 have_pvrdma = get_option('pvrdma') \
   .require(rdma.found(), error_message: 'PVRDMA requires OpenFabrics libraries') \
-  .require(cc.compiles('''
+  .require(cc.compiles(gnu_source_prefix + '''
+    #include <sys/mman.h>
     int main(void)
     {
       char buf = 0;
@@ -2190,7 +2191,7 @@ have_pvrdma = get_option('pvrdma') \
     }'''), error_message: 'PVRDMA requires mremap').allowed()
 
 if have_pvrdma
-  config_host_data.set('LEGACY_RDMA_REG_MR', not cc.compiles('''
+  config_host_data.set('LEGACY_RDMA_REG_MR', not cc.links('''
     #include <infiniband/verbs.h>
     int main(void)
     {

From 4a8027363e9680d00786622ee786fc3ccd6b970b Mon Sep 17 00:00:00 2001
From: Bernhard Beschow <shentey@gmail.com>
Date: Sat, 26 Mar 2022 17:58:23 +0100
Subject: [PATCH 236/935] hw/xen/xen_pt: Confine igd-passthrough-isa-bridge to
 XEN

igd-passthrough-isa-bridge is only requested in xen_pt but was
implemented in pc_piix.c. This caused xen_pt to dependend on i386/pc
which is hereby resolved.

Signed-off-by: Bernhard Beschow <shentey@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Message-Id: <20220326165825.30794-2-shentey@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/i386/pc_piix.c        | 118 --------------------------------------
 hw/xen/xen_pt.c          |   1 -
 hw/xen/xen_pt.h          |   1 +
 hw/xen/xen_pt_graphics.c | 119 +++++++++++++++++++++++++++++++++++++++
 include/hw/i386/pc.h     |   1 -
 5 files changed, 120 insertions(+), 120 deletions(-)

diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index 4c185c72d0..f843dd906f 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -813,124 +813,6 @@ static void pc_i440fx_1_4_machine_options(MachineClass *m)
 DEFINE_I440FX_MACHINE(v1_4, "pc-i440fx-1.4", pc_compat_1_4_fn,
                       pc_i440fx_1_4_machine_options);
 
-typedef struct {
-    uint16_t gpu_device_id;
-    uint16_t pch_device_id;
-    uint8_t pch_revision_id;
-} IGDDeviceIDInfo;
-
-/* In real world different GPU should have different PCH. But actually
- * the different PCH DIDs likely map to different PCH SKUs. We do the
- * same thing for the GPU. For PCH, the different SKUs are going to be
- * all the same silicon design and implementation, just different
- * features turn on and off with fuses. The SW interfaces should be
- * consistent across all SKUs in a given family (eg LPT). But just same
- * features may not be supported.
- *
- * Most of these different PCH features probably don't matter to the
- * Gfx driver, but obviously any difference in display port connections
- * will so it should be fine with any PCH in case of passthrough.
- *
- * So currently use one PCH version, 0x8c4e, to cover all HSW(Haswell)
- * scenarios, 0x9cc3 for BDW(Broadwell).
- */
-static const IGDDeviceIDInfo igd_combo_id_infos[] = {
-    /* HSW Classic */
-    {0x0402, 0x8c4e, 0x04}, /* HSWGT1D, HSWD_w7 */
-    {0x0406, 0x8c4e, 0x04}, /* HSWGT1M, HSWM_w7 */
-    {0x0412, 0x8c4e, 0x04}, /* HSWGT2D, HSWD_w7 */
-    {0x0416, 0x8c4e, 0x04}, /* HSWGT2M, HSWM_w7 */
-    {0x041E, 0x8c4e, 0x04}, /* HSWGT15D, HSWD_w7 */
-    /* HSW ULT */
-    {0x0A06, 0x8c4e, 0x04}, /* HSWGT1UT, HSWM_w7 */
-    {0x0A16, 0x8c4e, 0x04}, /* HSWGT2UT, HSWM_w7 */
-    {0x0A26, 0x8c4e, 0x06}, /* HSWGT3UT, HSWM_w7 */
-    {0x0A2E, 0x8c4e, 0x04}, /* HSWGT3UT28W, HSWM_w7 */
-    {0x0A1E, 0x8c4e, 0x04}, /* HSWGT2UX, HSWM_w7 */
-    {0x0A0E, 0x8c4e, 0x04}, /* HSWGT1ULX, HSWM_w7 */
-    /* HSW CRW */
-    {0x0D26, 0x8c4e, 0x04}, /* HSWGT3CW, HSWM_w7 */
-    {0x0D22, 0x8c4e, 0x04}, /* HSWGT3CWDT, HSWD_w7 */
-    /* HSW Server */
-    {0x041A, 0x8c4e, 0x04}, /* HSWSVGT2, HSWD_w7 */
-    /* HSW SRVR */
-    {0x040A, 0x8c4e, 0x04}, /* HSWSVGT1, HSWD_w7 */
-    /* BSW */
-    {0x1606, 0x9cc3, 0x03}, /* BDWULTGT1, BDWM_w7 */
-    {0x1616, 0x9cc3, 0x03}, /* BDWULTGT2, BDWM_w7 */
-    {0x1626, 0x9cc3, 0x03}, /* BDWULTGT3, BDWM_w7 */
-    {0x160E, 0x9cc3, 0x03}, /* BDWULXGT1, BDWM_w7 */
-    {0x161E, 0x9cc3, 0x03}, /* BDWULXGT2, BDWM_w7 */
-    {0x1602, 0x9cc3, 0x03}, /* BDWHALOGT1, BDWM_w7 */
-    {0x1612, 0x9cc3, 0x03}, /* BDWHALOGT2, BDWM_w7 */
-    {0x1622, 0x9cc3, 0x03}, /* BDWHALOGT3, BDWM_w7 */
-    {0x162B, 0x9cc3, 0x03}, /* BDWHALO28W, BDWM_w7 */
-    {0x162A, 0x9cc3, 0x03}, /* BDWGT3WRKS, BDWM_w7 */
-    {0x162D, 0x9cc3, 0x03}, /* BDWGT3SRVR, BDWM_w7 */
-};
-
-static void isa_bridge_class_init(ObjectClass *klass, void *data)
-{
-    DeviceClass *dc = DEVICE_CLASS(klass);
-    PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
-
-    dc->desc        = "ISA bridge faked to support IGD PT";
-    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
-    k->vendor_id    = PCI_VENDOR_ID_INTEL;
-    k->class_id     = PCI_CLASS_BRIDGE_ISA;
-};
-
-static const TypeInfo isa_bridge_info = {
-    .name          = "igd-passthrough-isa-bridge",
-    .parent        = TYPE_PCI_DEVICE,
-    .instance_size = sizeof(PCIDevice),
-    .class_init = isa_bridge_class_init,
-    .interfaces = (InterfaceInfo[]) {
-        { INTERFACE_CONVENTIONAL_PCI_DEVICE },
-        { },
-    },
-};
-
-static void pt_graphics_register_types(void)
-{
-    type_register_static(&isa_bridge_info);
-}
-type_init(pt_graphics_register_types)
-
-void igd_passthrough_isa_bridge_create(PCIBus *bus, uint16_t gpu_dev_id)
-{
-    struct PCIDevice *bridge_dev;
-    int i, num;
-    uint16_t pch_dev_id = 0xffff;
-    uint8_t pch_rev_id = 0;
-
-    num = ARRAY_SIZE(igd_combo_id_infos);
-    for (i = 0; i < num; i++) {
-        if (gpu_dev_id == igd_combo_id_infos[i].gpu_device_id) {
-            pch_dev_id = igd_combo_id_infos[i].pch_device_id;
-            pch_rev_id = igd_combo_id_infos[i].pch_revision_id;
-        }
-    }
-
-    if (pch_dev_id == 0xffff) {
-        return;
-    }
-
-    /* Currently IGD drivers always need to access PCH by 1f.0. */
-    bridge_dev = pci_create_simple(bus, PCI_DEVFN(0x1f, 0),
-                                   "igd-passthrough-isa-bridge");
-
-    /*
-     * Note that vendor id is always PCI_VENDOR_ID_INTEL.
-     */
-    if (!bridge_dev) {
-        fprintf(stderr, "set igd-passthrough-isa-bridge failed!\n");
-        return;
-    }
-    pci_config_set_device_id(bridge_dev->config, pch_dev_id);
-    pci_config_set_revision(bridge_dev->config, pch_rev_id);
-}
-
 #ifdef CONFIG_ISAPC
 static void isapc_machine_options(MachineClass *m)
 {
diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
index 027190fa44..829ea9985f 100644
--- a/hw/xen/xen_pt.c
+++ b/hw/xen/xen_pt.c
@@ -60,7 +60,6 @@
 #include "hw/qdev-properties.h"
 #include "hw/qdev-properties-system.h"
 #include "hw/xen/xen.h"
-#include "hw/i386/pc.h"
 #include "hw/xen/xen-legacy-backend.h"
 #include "xen_pt.h"
 #include "qemu/range.h"
diff --git a/hw/xen/xen_pt.h b/hw/xen/xen_pt.h
index 6b8e13cdee..806d832c94 100644
--- a/hw/xen/xen_pt.h
+++ b/hw/xen/xen_pt.h
@@ -43,6 +43,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(XenPCIPassthroughState, XEN_PT_DEVICE)
 
 uint32_t igd_read_opregion(XenPCIPassthroughState *s);
 void igd_write_opregion(XenPCIPassthroughState *s, uint32_t val);
+void igd_passthrough_isa_bridge_create(PCIBus *bus, uint16_t gpu_dev_id);
 
 /* function type for config reg */
 typedef int (*xen_pt_conf_reg_init)
diff --git a/hw/xen/xen_pt_graphics.c b/hw/xen/xen_pt_graphics.c
index a3bc7e3921..f1fbb98912 100644
--- a/hw/xen/xen_pt_graphics.c
+++ b/hw/xen/xen_pt_graphics.c
@@ -289,3 +289,122 @@ void igd_write_opregion(XenPCIPassthroughState *s, uint32_t val)
                     (unsigned long)(igd_host_opregion >> XC_PAGE_SHIFT),
                     (unsigned long)(igd_guest_opregion >> XC_PAGE_SHIFT));
 }
+
+typedef struct {
+    uint16_t gpu_device_id;
+    uint16_t pch_device_id;
+    uint8_t pch_revision_id;
+} IGDDeviceIDInfo;
+
+/*
+ * In real world different GPU should have different PCH. But actually
+ * the different PCH DIDs likely map to different PCH SKUs. We do the
+ * same thing for the GPU. For PCH, the different SKUs are going to be
+ * all the same silicon design and implementation, just different
+ * features turn on and off with fuses. The SW interfaces should be
+ * consistent across all SKUs in a given family (eg LPT). But just same
+ * features may not be supported.
+ *
+ * Most of these different PCH features probably don't matter to the
+ * Gfx driver, but obviously any difference in display port connections
+ * will so it should be fine with any PCH in case of passthrough.
+ *
+ * So currently use one PCH version, 0x8c4e, to cover all HSW(Haswell)
+ * scenarios, 0x9cc3 for BDW(Broadwell).
+ */
+static const IGDDeviceIDInfo igd_combo_id_infos[] = {
+    /* HSW Classic */
+    {0x0402, 0x8c4e, 0x04}, /* HSWGT1D, HSWD_w7 */
+    {0x0406, 0x8c4e, 0x04}, /* HSWGT1M, HSWM_w7 */
+    {0x0412, 0x8c4e, 0x04}, /* HSWGT2D, HSWD_w7 */
+    {0x0416, 0x8c4e, 0x04}, /* HSWGT2M, HSWM_w7 */
+    {0x041E, 0x8c4e, 0x04}, /* HSWGT15D, HSWD_w7 */
+    /* HSW ULT */
+    {0x0A06, 0x8c4e, 0x04}, /* HSWGT1UT, HSWM_w7 */
+    {0x0A16, 0x8c4e, 0x04}, /* HSWGT2UT, HSWM_w7 */
+    {0x0A26, 0x8c4e, 0x06}, /* HSWGT3UT, HSWM_w7 */
+    {0x0A2E, 0x8c4e, 0x04}, /* HSWGT3UT28W, HSWM_w7 */
+    {0x0A1E, 0x8c4e, 0x04}, /* HSWGT2UX, HSWM_w7 */
+    {0x0A0E, 0x8c4e, 0x04}, /* HSWGT1ULX, HSWM_w7 */
+    /* HSW CRW */
+    {0x0D26, 0x8c4e, 0x04}, /* HSWGT3CW, HSWM_w7 */
+    {0x0D22, 0x8c4e, 0x04}, /* HSWGT3CWDT, HSWD_w7 */
+    /* HSW Server */
+    {0x041A, 0x8c4e, 0x04}, /* HSWSVGT2, HSWD_w7 */
+    /* HSW SRVR */
+    {0x040A, 0x8c4e, 0x04}, /* HSWSVGT1, HSWD_w7 */
+    /* BSW */
+    {0x1606, 0x9cc3, 0x03}, /* BDWULTGT1, BDWM_w7 */
+    {0x1616, 0x9cc3, 0x03}, /* BDWULTGT2, BDWM_w7 */
+    {0x1626, 0x9cc3, 0x03}, /* BDWULTGT3, BDWM_w7 */
+    {0x160E, 0x9cc3, 0x03}, /* BDWULXGT1, BDWM_w7 */
+    {0x161E, 0x9cc3, 0x03}, /* BDWULXGT2, BDWM_w7 */
+    {0x1602, 0x9cc3, 0x03}, /* BDWHALOGT1, BDWM_w7 */
+    {0x1612, 0x9cc3, 0x03}, /* BDWHALOGT2, BDWM_w7 */
+    {0x1622, 0x9cc3, 0x03}, /* BDWHALOGT3, BDWM_w7 */
+    {0x162B, 0x9cc3, 0x03}, /* BDWHALO28W, BDWM_w7 */
+    {0x162A, 0x9cc3, 0x03}, /* BDWGT3WRKS, BDWM_w7 */
+    {0x162D, 0x9cc3, 0x03}, /* BDWGT3SRVR, BDWM_w7 */
+};
+
+static void isa_bridge_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+    PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
+
+    dc->desc        = "ISA bridge faked to support IGD PT";
+    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
+    k->vendor_id    = PCI_VENDOR_ID_INTEL;
+    k->class_id     = PCI_CLASS_BRIDGE_ISA;
+};
+
+static const TypeInfo isa_bridge_info = {
+    .name          = "igd-passthrough-isa-bridge",
+    .parent        = TYPE_PCI_DEVICE,
+    .instance_size = sizeof(PCIDevice),
+    .class_init = isa_bridge_class_init,
+    .interfaces = (InterfaceInfo[]) {
+        { INTERFACE_CONVENTIONAL_PCI_DEVICE },
+        { },
+    },
+};
+
+static void pt_graphics_register_types(void)
+{
+    type_register_static(&isa_bridge_info);
+}
+type_init(pt_graphics_register_types)
+
+void igd_passthrough_isa_bridge_create(PCIBus *bus, uint16_t gpu_dev_id)
+{
+    struct PCIDevice *bridge_dev;
+    int i, num;
+    uint16_t pch_dev_id = 0xffff;
+    uint8_t pch_rev_id = 0;
+
+    num = ARRAY_SIZE(igd_combo_id_infos);
+    for (i = 0; i < num; i++) {
+        if (gpu_dev_id == igd_combo_id_infos[i].gpu_device_id) {
+            pch_dev_id = igd_combo_id_infos[i].pch_device_id;
+            pch_rev_id = igd_combo_id_infos[i].pch_revision_id;
+        }
+    }
+
+    if (pch_dev_id == 0xffff) {
+        return;
+    }
+
+    /* Currently IGD drivers always need to access PCH by 1f.0. */
+    bridge_dev = pci_create_simple(bus, PCI_DEVFN(0x1f, 0),
+                                   "igd-passthrough-isa-bridge");
+
+    /*
+     * Note that vendor id is always PCI_VENDOR_ID_INTEL.
+     */
+    if (!bridge_dev) {
+        fprintf(stderr, "set igd-passthrough-isa-bridge failed!\n");
+        return;
+    }
+    pci_config_set_device_id(bridge_dev->config, pch_dev_id);
+    pci_config_set_revision(bridge_dev->config, pch_rev_id);
+}
diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index 637367dc5f..aff8add155 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -315,5 +315,4 @@ extern const size_t pc_compat_1_4_len;
     } \
     type_init(pc_machine_init_##suffix)
 
-extern void igd_passthrough_isa_bridge_create(PCIBus *bus, uint16_t gpu_dev_id);
 #endif

From 76acef2b735729c642c54385b7566de509506d9a Mon Sep 17 00:00:00 2001
From: Bernhard Beschow <shentey@gmail.com>
Date: Sat, 26 Mar 2022 17:58:24 +0100
Subject: [PATCH 237/935] hw/xen/xen_pt: Resolve
 igd_passthrough_isa_bridge_create() indirection

Now that igd_passthrough_isa_bridge_create() is implemented within the
xen context it may use Xen* data types directly and become
xen_igd_passthrough_isa_bridge_create(). This resolves an indirection.

Signed-off-by: Bernhard Beschow <shentey@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Message-Id: <20220326165825.30794-3-shentey@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/xen/xen_pt.c          | 11 -----------
 hw/xen/xen_pt.h          |  3 ++-
 hw/xen/xen_pt_graphics.c |  5 ++++-
 3 files changed, 6 insertions(+), 13 deletions(-)

diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
index 829ea9985f..0ec7e52183 100644
--- a/hw/xen/xen_pt.c
+++ b/hw/xen/xen_pt.c
@@ -701,17 +701,6 @@ static const MemoryListener xen_pt_io_listener = {
     .priority = 10,
 };
 
-static void
-xen_igd_passthrough_isa_bridge_create(XenPCIPassthroughState *s,
-                                      XenHostPCIDevice *dev)
-{
-    uint16_t gpu_dev_id;
-    PCIDevice *d = &s->dev;
-
-    gpu_dev_id = dev->device_id;
-    igd_passthrough_isa_bridge_create(pci_get_bus(d), gpu_dev_id);
-}
-
 /* destroy. */
 static void xen_pt_destroy(PCIDevice *d) {
 
diff --git a/hw/xen/xen_pt.h b/hw/xen/xen_pt.h
index 806d832c94..e7c4316a7d 100644
--- a/hw/xen/xen_pt.h
+++ b/hw/xen/xen_pt.h
@@ -43,7 +43,8 @@ OBJECT_DECLARE_SIMPLE_TYPE(XenPCIPassthroughState, XEN_PT_DEVICE)
 
 uint32_t igd_read_opregion(XenPCIPassthroughState *s);
 void igd_write_opregion(XenPCIPassthroughState *s, uint32_t val);
-void igd_passthrough_isa_bridge_create(PCIBus *bus, uint16_t gpu_dev_id);
+void xen_igd_passthrough_isa_bridge_create(XenPCIPassthroughState *s,
+                                           XenHostPCIDevice *dev);
 
 /* function type for config reg */
 typedef int (*xen_pt_conf_reg_init)
diff --git a/hw/xen/xen_pt_graphics.c b/hw/xen/xen_pt_graphics.c
index f1fbb98912..f303f67c9c 100644
--- a/hw/xen/xen_pt_graphics.c
+++ b/hw/xen/xen_pt_graphics.c
@@ -375,10 +375,13 @@ static void pt_graphics_register_types(void)
 }
 type_init(pt_graphics_register_types)
 
-void igd_passthrough_isa_bridge_create(PCIBus *bus, uint16_t gpu_dev_id)
+void xen_igd_passthrough_isa_bridge_create(XenPCIPassthroughState *s,
+                                           XenHostPCIDevice *dev)
 {
+    PCIBus *bus = pci_get_bus(&s->dev);
     struct PCIDevice *bridge_dev;
     int i, num;
+    const uint16_t gpu_dev_id = dev->device_id;
     uint16_t pch_dev_id = 0xffff;
     uint8_t pch_rev_id = 0;
 

From 3df72d1c5500347eac0b5b6d9894713dc443a079 Mon Sep 17 00:00:00 2001
From: Eric Auger <eric.auger@redhat.com>
Date: Wed, 4 May 2022 17:20:23 +0200
Subject: [PATCH 238/935] tests/qtest/libqos/pci: Introduce pio_limit
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

At the moment the IO space limit is hardcoded to
QPCI_PIO_LIMIT = 0x10000. When accesses are performed to a bar,
the base address of this latter is compared against the limit
to decide whether we perform an IO or a memory access.

On ARM, we cannot keep this PIO limit as the arm-virt machine
uses [0x3eff0000, 0x3f000000 ] for the IO space map and we
are mandated to allocate at 0x0.

Add a new flag in QPCIBar indicating whether it is an IO bar
or a memory bar. This flag is set on QPCIBar allocation and
provisionned based on the BAR configuration. Then the new flag
is used in access functions and in iomap() function.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220504152025.1785704-2-eric.auger@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/qtest/libqos/pci-pc.c    |  1 +
 tests/qtest/libqos/pci-spapr.c |  1 +
 tests/qtest/libqos/pci.c       | 78 ++++++++++++++++++++++------------
 tests/qtest/libqos/pci.h       |  5 +--
 4 files changed, 54 insertions(+), 31 deletions(-)

diff --git a/tests/qtest/libqos/pci-pc.c b/tests/qtest/libqos/pci-pc.c
index e9dd5a57ec..81c2c055ca 100644
--- a/tests/qtest/libqos/pci-pc.c
+++ b/tests/qtest/libqos/pci-pc.c
@@ -150,6 +150,7 @@ void qpci_init_pc(QPCIBusPC *qpci, QTestState *qts, QGuestAllocator *alloc)
 
     qpci->bus.qts = qts;
     qpci->bus.pio_alloc_ptr = 0xc000;
+    qpci->bus.pio_limit = 0x10000;
     qpci->bus.mmio_alloc_ptr = 0xE0000000;
     qpci->bus.mmio_limit = 0x100000000ULL;
 
diff --git a/tests/qtest/libqos/pci-spapr.c b/tests/qtest/libqos/pci-spapr.c
index 76bf9a855d..0f1023e4a7 100644
--- a/tests/qtest/libqos/pci-spapr.c
+++ b/tests/qtest/libqos/pci-spapr.c
@@ -197,6 +197,7 @@ void qpci_init_spapr(QPCIBusSPAPR *qpci, QTestState *qts,
 
     qpci->bus.qts = qts;
     qpci->bus.pio_alloc_ptr = 0xc000;
+    qpci->bus.pio_limit = 0x10000;
     qpci->bus.mmio_alloc_ptr = qpci->mmio32.pci_base;
     qpci->bus.mmio_limit = qpci->mmio32.pci_base + qpci->mmio32.size;
 
diff --git a/tests/qtest/libqos/pci.c b/tests/qtest/libqos/pci.c
index 3a9076ae58..b23d72346b 100644
--- a/tests/qtest/libqos/pci.c
+++ b/tests/qtest/libqos/pci.c
@@ -398,44 +398,56 @@ void qpci_config_writel(QPCIDevice *dev, uint8_t offset, uint32_t value)
 
 uint8_t qpci_io_readb(QPCIDevice *dev, QPCIBar token, uint64_t off)
 {
-    if (token.addr < QPCI_PIO_LIMIT) {
-        return dev->bus->pio_readb(dev->bus, token.addr + off);
+    QPCIBus *bus = dev->bus;
+
+    if (token.is_io) {
+        return bus->pio_readb(bus, token.addr + off);
     } else {
         uint8_t val;
-        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
+
+        bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
         return val;
     }
 }
 
 uint16_t qpci_io_readw(QPCIDevice *dev, QPCIBar token, uint64_t off)
 {
-    if (token.addr < QPCI_PIO_LIMIT) {
-        return dev->bus->pio_readw(dev->bus, token.addr + off);
+    QPCIBus *bus = dev->bus;
+
+    if (token.is_io) {
+        return bus->pio_readw(bus, token.addr + off);
     } else {
         uint16_t val;
-        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
+
+        bus->memread(bus, token.addr + off, &val, sizeof(val));
         return le16_to_cpu(val);
     }
 }
 
 uint32_t qpci_io_readl(QPCIDevice *dev, QPCIBar token, uint64_t off)
 {
-    if (token.addr < QPCI_PIO_LIMIT) {
-        return dev->bus->pio_readl(dev->bus, token.addr + off);
+    QPCIBus *bus = dev->bus;
+
+    if (token.is_io) {
+        return bus->pio_readl(bus, token.addr + off);
     } else {
         uint32_t val;
-        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
+
+        bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
         return le32_to_cpu(val);
     }
 }
 
 uint64_t qpci_io_readq(QPCIDevice *dev, QPCIBar token, uint64_t off)
 {
-    if (token.addr < QPCI_PIO_LIMIT) {
-        return dev->bus->pio_readq(dev->bus, token.addr + off);
+    QPCIBus *bus = dev->bus;
+
+    if (token.is_io) {
+        return bus->pio_readq(bus, token.addr + off);
     } else {
         uint64_t val;
-        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
+
+        bus->memread(bus, token.addr + off, &val, sizeof(val));
         return le64_to_cpu(val);
     }
 }
@@ -443,57 +455,65 @@ uint64_t qpci_io_readq(QPCIDevice *dev, QPCIBar token, uint64_t off)
 void qpci_io_writeb(QPCIDevice *dev, QPCIBar token, uint64_t off,
                     uint8_t value)
 {
-    if (token.addr < QPCI_PIO_LIMIT) {
-        dev->bus->pio_writeb(dev->bus, token.addr + off, value);
+    QPCIBus *bus = dev->bus;
+
+    if (token.is_io) {
+        bus->pio_writeb(bus, token.addr + off, value);
     } else {
-        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
+        bus->memwrite(bus, token.addr + off, &value, sizeof(value));
     }
 }
 
 void qpci_io_writew(QPCIDevice *dev, QPCIBar token, uint64_t off,
                     uint16_t value)
 {
-    if (token.addr < QPCI_PIO_LIMIT) {
-        dev->bus->pio_writew(dev->bus, token.addr + off, value);
+    QPCIBus *bus = dev->bus;
+
+    if (token.is_io) {
+        bus->pio_writew(bus, token.addr + off, value);
     } else {
         value = cpu_to_le16(value);
-        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
+        bus->memwrite(bus, token.addr + off, &value, sizeof(value));
     }
 }
 
 void qpci_io_writel(QPCIDevice *dev, QPCIBar token, uint64_t off,
                     uint32_t value)
 {
-    if (token.addr < QPCI_PIO_LIMIT) {
-        dev->bus->pio_writel(dev->bus, token.addr + off, value);
+    QPCIBus *bus = dev->bus;
+
+    if (token.is_io) {
+        bus->pio_writel(bus, token.addr + off, value);
     } else {
         value = cpu_to_le32(value);
-        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
+        bus->memwrite(bus, token.addr + off, &value, sizeof(value));
     }
 }
 
 void qpci_io_writeq(QPCIDevice *dev, QPCIBar token, uint64_t off,
                     uint64_t value)
 {
-    if (token.addr < QPCI_PIO_LIMIT) {
-        dev->bus->pio_writeq(dev->bus, token.addr + off, value);
+    QPCIBus *bus = dev->bus;
+
+    if (token.is_io) {
+        bus->pio_writeq(bus, token.addr + off, value);
     } else {
         value = cpu_to_le64(value);
-        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
+        bus->memwrite(bus, token.addr + off, &value, sizeof(value));
     }
 }
 
 void qpci_memread(QPCIDevice *dev, QPCIBar token, uint64_t off,
                   void *buf, size_t len)
 {
-    g_assert(token.addr >= QPCI_PIO_LIMIT);
+    g_assert(!token.is_io);
     dev->bus->memread(dev->bus, token.addr + off, buf, len);
 }
 
 void qpci_memwrite(QPCIDevice *dev, QPCIBar token, uint64_t off,
                    const void *buf, size_t len)
 {
-    g_assert(token.addr >= QPCI_PIO_LIMIT);
+    g_assert(!token.is_io);
     dev->bus->memwrite(dev->bus, token.addr + off, buf, len);
 }
 
@@ -534,9 +554,10 @@ QPCIBar qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
         loc = QEMU_ALIGN_UP(bus->pio_alloc_ptr, size);
 
         g_assert(loc >= bus->pio_alloc_ptr);
-        g_assert(loc + size <= QPCI_PIO_LIMIT); /* Keep PIO below 64kiB */
+        g_assert(loc + size <= bus->pio_limit);
 
         bus->pio_alloc_ptr = loc + size;
+        bar.is_io = true;
 
         qpci_config_writel(dev, bar_reg, loc | PCI_BASE_ADDRESS_SPACE_IO);
     } else {
@@ -547,6 +568,7 @@ QPCIBar qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
         g_assert(loc + size <= bus->mmio_limit);
 
         bus->mmio_alloc_ptr = loc + size;
+        bar.is_io = false;
 
         qpci_config_writel(dev, bar_reg, loc);
     }
@@ -562,7 +584,7 @@ void qpci_iounmap(QPCIDevice *dev, QPCIBar bar)
 
 QPCIBar qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr)
 {
-    QPCIBar bar = { .addr = addr };
+    QPCIBar bar = { .addr = addr, .is_io = true };
     return bar;
 }
 
diff --git a/tests/qtest/libqos/pci.h b/tests/qtest/libqos/pci.h
index e705e06598..a3c657d962 100644
--- a/tests/qtest/libqos/pci.h
+++ b/tests/qtest/libqos/pci.h
@@ -16,8 +16,6 @@
 #include "../libqtest.h"
 #include "qgraph.h"
 
-#define QPCI_PIO_LIMIT    0x10000
-
 #define QPCI_DEVFN(dev, fn) (((dev) << 3) | (fn))
 
 typedef struct QPCIDevice QPCIDevice;
@@ -51,7 +49,7 @@ struct QPCIBus {
                           uint8_t offset, uint32_t value);
 
     QTestState *qts;
-    uint16_t pio_alloc_ptr;
+    uint64_t pio_alloc_ptr, pio_limit;
     uint64_t mmio_alloc_ptr, mmio_limit;
     bool has_buggy_msi; /* TRUE for spapr, FALSE for pci */
 
@@ -59,6 +57,7 @@ struct QPCIBus {
 
 struct QPCIBar {
     uint64_t addr;
+    bool is_io;
 };
 
 struct QPCIDevice

From 02ee7a8a97d3ddeda887b596005b462f680dc89c Mon Sep 17 00:00:00 2001
From: Eric Auger <eric.auger@redhat.com>
Date: Wed, 4 May 2022 17:20:24 +0200
Subject: [PATCH 239/935] tests/qtest/libqos: Skip hotplug tests if pci root
 bus is not hotpluggable
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

ARM does not not support hotplug on pcie.0. Add a flag on the bus
which tells if devices can be hotplugged and skip hotplug tests
if the bus cannot be hotplugged. This is a temporary solution to
enable the other pci tests on aarch64.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

Message-Id: <20220504152025.1785704-3-eric.auger@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/qtest/e1000e-test.c         |  6 ++++++
 tests/qtest/libqos/pci.h          |  1 +
 tests/qtest/vhost-user-blk-test.c | 10 ++++++++++
 tests/qtest/virtio-blk-test.c     |  5 +++++
 tests/qtest/virtio-net-test.c     |  5 +++++
 tests/qtest/virtio-rng-test.c     |  5 +++++
 6 files changed, 32 insertions(+)

diff --git a/tests/qtest/e1000e-test.c b/tests/qtest/e1000e-test.c
index ddd6983ede..c98779c7c0 100644
--- a/tests/qtest/e1000e-test.c
+++ b/tests/qtest/e1000e-test.c
@@ -233,6 +233,12 @@ static void test_e1000e_multiple_transfers(void *obj, void *data,
 static void test_e1000e_hotplug(void *obj, void *data, QGuestAllocator * alloc)
 {
     QTestState *qts = global_qtest;  /* TODO: get rid of global_qtest here */
+    QE1000E_PCI *dev = obj;
+
+    if (dev->pci_dev.bus->not_hotpluggable) {
+        g_test_skip("pci bus does not support hotplug");
+        return;
+    }
 
     qtest_qmp_device_add(qts, "e1000e", "e1000e_net", "{'addr': '0x06'}");
     qpci_unplug_acpi_device_test(qts, "e1000e_net", 0x06);
diff --git a/tests/qtest/libqos/pci.h b/tests/qtest/libqos/pci.h
index a3c657d962..8389614523 100644
--- a/tests/qtest/libqos/pci.h
+++ b/tests/qtest/libqos/pci.h
@@ -52,6 +52,7 @@ struct QPCIBus {
     uint64_t pio_alloc_ptr, pio_limit;
     uint64_t mmio_alloc_ptr, mmio_limit;
     bool has_buggy_msi; /* TRUE for spapr, FALSE for pci */
+    bool not_hotpluggable; /* TRUE if devices cannot be hotplugged */
 
 };
 
diff --git a/tests/qtest/vhost-user-blk-test.c b/tests/qtest/vhost-user-blk-test.c
index 659b5050d8..a81c2a2715 100644
--- a/tests/qtest/vhost-user-blk-test.c
+++ b/tests/qtest/vhost-user-blk-test.c
@@ -676,6 +676,11 @@ static void pci_hotplug(void *obj, void *data, QGuestAllocator *t_alloc)
     QVirtioPCIDevice *dev;
     QTestState *qts = dev1->pdev->bus->qts;
 
+    if (dev1->pdev->bus->not_hotpluggable) {
+        g_test_skip("pci bus does not support hotplug");
+        return;
+    }
+
     /* plug secondary disk */
     qtest_qmp_device_add(qts, "vhost-user-blk-pci", "drv1",
                          "{'addr': %s, 'chardev': 'char2'}",
@@ -703,6 +708,11 @@ static void multiqueue(void *obj, void *data, QGuestAllocator *t_alloc)
     uint64_t features;
     uint16_t num_queues;
 
+    if (pdev1->pdev->bus->not_hotpluggable) {
+        g_test_skip("bus pci.0 does not support hotplug");
+        return;
+    }
+
     /*
      * The primary device has 1 queue and VIRTIO_BLK_F_MQ is not enabled. The
      * VIRTIO specification allows VIRTIO_BLK_F_MQ to be enabled when there is
diff --git a/tests/qtest/virtio-blk-test.c b/tests/qtest/virtio-blk-test.c
index f22594a1a8..dc5eed31c8 100644
--- a/tests/qtest/virtio-blk-test.c
+++ b/tests/qtest/virtio-blk-test.c
@@ -701,6 +701,11 @@ static void pci_hotplug(void *obj, void *data, QGuestAllocator *t_alloc)
     QVirtioPCIDevice *dev;
     QTestState *qts = dev1->pdev->bus->qts;
 
+    if (dev1->pdev->bus->not_hotpluggable) {
+        g_test_skip("pci bus does not support hotplug");
+        return;
+    }
+
     /* plug secondary disk */
     qtest_qmp_device_add(qts, "virtio-blk-pci", "drv1",
                          "{'addr': %s, 'drive': 'drive1'}",
diff --git a/tests/qtest/virtio-net-test.c b/tests/qtest/virtio-net-test.c
index fc9f2b9498..6ded252901 100644
--- a/tests/qtest/virtio-net-test.c
+++ b/tests/qtest/virtio-net-test.c
@@ -173,6 +173,11 @@ static void hotplug(void *obj, void *data, QGuestAllocator *t_alloc)
     QTestState *qts = dev->pdev->bus->qts;
     const char *arch = qtest_get_arch();
 
+    if (dev->pdev->bus->not_hotpluggable) {
+        g_test_skip("pci bus does not support hotplug");
+        return;
+    }
+
     qtest_qmp_device_add(qts, "virtio-net-pci", "net1",
                          "{'addr': %s}", stringify(PCI_SLOT_HP));
 
diff --git a/tests/qtest/virtio-rng-test.c b/tests/qtest/virtio-rng-test.c
index 092ba13068..64e131cd8c 100644
--- a/tests/qtest/virtio-rng-test.c
+++ b/tests/qtest/virtio-rng-test.c
@@ -20,6 +20,11 @@ static void rng_hotplug(void *obj, void *data, QGuestAllocator *alloc)
     QVirtioPCIDevice *dev = obj;
     QTestState *qts = dev->pdev->bus->qts;
 
+   if (dev->pdev->bus->not_hotpluggable) {
+        g_test_skip("pci bus does not support hotplug");
+        return;
+    }
+
     const char *arch = qtest_get_arch();
 
     qtest_qmp_device_add(qts, "virtio-rng-pci", "rng1",

From 70be1d93f9c2dbf6793830d482e91bb33f921348 Mon Sep 17 00:00:00 2001
From: Eric Auger <eric.auger@redhat.com>
Date: Wed, 4 May 2022 17:20:25 +0200
Subject: [PATCH 240/935] tests/qtest/libqos: Add generic pci host bridge in
 arm-virt machine

Up to now the virt-machine node contains a virtio-mmio node.
However no driver produces any PCI interface node. Hence, PCI
tests cannot be run with aarch64 binary.

Add a GPEX driver node that produces a pci interface node. This latter
then can be consumed by all the pci tests. One of the first motivation
was to be able to run the virtio-iommu-pci tests.

We still face an issue with pci hotplug tests as hotplug cannot happen
on the pcie root bus and require a generic root port. This will be
addressed later on.

We force cpu=max along with aarch64/virt machine as some PCI tests
require high MMIO regions to be available.

Signed-off-by: Eric Auger <eric.auger@redhat.com>

Message-Id: <20220504152025.1785704-4-eric.auger@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/qtest/libqos/arm-virt-machine.c |  19 ++-
 tests/qtest/libqos/generic-pcihost.c  | 231 ++++++++++++++++++++++++++
 tests/qtest/libqos/generic-pcihost.h  |  54 ++++++
 tests/qtest/libqos/meson.build        |   1 +
 4 files changed, 301 insertions(+), 4 deletions(-)
 create mode 100644 tests/qtest/libqos/generic-pcihost.c
 create mode 100644 tests/qtest/libqos/generic-pcihost.h

diff --git a/tests/qtest/libqos/arm-virt-machine.c b/tests/qtest/libqos/arm-virt-machine.c
index 2e0beaefb8..139eaba142 100644
--- a/tests/qtest/libqos/arm-virt-machine.c
+++ b/tests/qtest/libqos/arm-virt-machine.c
@@ -22,6 +22,8 @@
 #include "malloc.h"
 #include "qgraph.h"
 #include "virtio-mmio.h"
+#include "generic-pcihost.h"
+#include "hw/pci/pci_regs.h"
 
 #define ARM_PAGE_SIZE               4096
 #define VIRTIO_MMIO_BASE_ADDR       0x0A003E00
@@ -35,6 +37,7 @@ struct QVirtMachine {
     QOSGraphObject obj;
     QGuestAllocator alloc;
     QVirtioMMIODevice virtio_mmio;
+    QGenericPCIHost bridge;
 };
 
 static void virt_destructor(QOSGraphObject *obj)
@@ -57,11 +60,13 @@ static void *virt_get_driver(void *object, const char *interface)
 static QOSGraphObject *virt_get_device(void *obj, const char *device)
 {
     QVirtMachine *machine = obj;
-    if (!g_strcmp0(device, "virtio-mmio")) {
+    if (!g_strcmp0(device, "generic-pcihost")) {
+        return &machine->bridge.obj;
+    } else if (!g_strcmp0(device, "virtio-mmio")) {
         return &machine->virtio_mmio.obj;
     }
 
-    fprintf(stderr, "%s not present in arm/virtio\n", device);
+    fprintf(stderr, "%s not present in arm/virt\n", device);
     g_assert_not_reached();
 }
 
@@ -76,16 +81,22 @@ static void *qos_create_machine_arm_virt(QTestState *qts)
     qvirtio_mmio_init_device(&machine->virtio_mmio, qts, VIRTIO_MMIO_BASE_ADDR,
                              VIRTIO_MMIO_SIZE);
 
+    qos_create_generic_pcihost(&machine->bridge, qts, &machine->alloc);
+
     machine->obj.get_device = virt_get_device;
     machine->obj.get_driver = virt_get_driver;
     machine->obj.destructor = virt_destructor;
     return machine;
 }
 
-static void virtio_mmio_register_nodes(void)
+static void virt_machine_register_nodes(void)
 {
     qos_node_create_machine("arm/virt", qos_create_machine_arm_virt);
     qos_node_contains("arm/virt", "virtio-mmio", NULL);
+
+    qos_node_create_machine_args("aarch64/virt", qos_create_machine_arm_virt,
+                                 " -cpu max");
+    qos_node_contains("aarch64/virt", "generic-pcihost", NULL);
 }
 
-libqos_init(virtio_mmio_register_nodes);
+libqos_init(virt_machine_register_nodes);
diff --git a/tests/qtest/libqos/generic-pcihost.c b/tests/qtest/libqos/generic-pcihost.c
new file mode 100644
index 0000000000..3124b0e46b
--- /dev/null
+++ b/tests/qtest/libqos/generic-pcihost.c
@@ -0,0 +1,231 @@
+/*
+ * libqos PCI bindings for generic PCI
+ *
+ * Copyright Red Hat Inc., 2022
+ *
+ * Authors:
+ *  Eric Auger   <eric.auger@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "../libqtest.h"
+#include "generic-pcihost.h"
+#include "qapi/qmp/qdict.h"
+#include "hw/pci/pci_regs.h"
+#include "qemu/host-utils.h"
+
+#include "qemu/module.h"
+
+/* QGenericPCIHost */
+
+QOSGraphObject *generic_pcihost_get_device(void *obj, const char *device)
+{
+    QGenericPCIHost *host = obj;
+    if (!g_strcmp0(device, "pci-bus-generic")) {
+        return &host->pci.obj;
+    }
+    fprintf(stderr, "%s not present in generic-pcihost\n", device);
+    g_assert_not_reached();
+}
+
+void qos_create_generic_pcihost(QGenericPCIHost *host,
+                                QTestState *qts,
+                                QGuestAllocator *alloc)
+{
+    host->obj.get_device = generic_pcihost_get_device;
+    qpci_init_generic(&host->pci, qts, alloc, false);
+}
+
+static uint8_t qpci_generic_pio_readb(QPCIBus *bus, uint32_t addr)
+{
+    QGenericPCIBus *s = container_of(bus, QGenericPCIBus, bus);
+
+    return qtest_readb(bus->qts, s->gpex_pio_base + addr);
+}
+
+static void qpci_generic_pio_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
+{
+    QGenericPCIBus *s = container_of(bus, QGenericPCIBus, bus);
+
+    qtest_writeb(bus->qts, s->gpex_pio_base + addr,  val);
+}
+
+static uint16_t qpci_generic_pio_readw(QPCIBus *bus, uint32_t addr)
+{
+    QGenericPCIBus *s = container_of(bus, QGenericPCIBus, bus);
+
+    return qtest_readw(bus->qts, s->gpex_pio_base + addr);
+}
+
+static void qpci_generic_pio_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
+{
+    QGenericPCIBus *s = container_of(bus, QGenericPCIBus, bus);
+
+    qtest_writew(bus->qts, s->gpex_pio_base + addr, val);
+}
+
+static uint32_t qpci_generic_pio_readl(QPCIBus *bus, uint32_t addr)
+{
+    QGenericPCIBus *s = container_of(bus, QGenericPCIBus, bus);
+
+    return qtest_readl(bus->qts, s->gpex_pio_base + addr);
+}
+
+static void qpci_generic_pio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
+{
+    QGenericPCIBus *s = container_of(bus, QGenericPCIBus, bus);
+
+    qtest_writel(bus->qts, s->gpex_pio_base + addr, val);
+}
+
+static uint64_t qpci_generic_pio_readq(QPCIBus *bus, uint32_t addr)
+{
+    QGenericPCIBus *s = container_of(bus, QGenericPCIBus, bus);
+
+    return qtest_readq(bus->qts, s->gpex_pio_base + addr);
+}
+
+static void qpci_generic_pio_writeq(QPCIBus *bus, uint32_t addr, uint64_t val)
+{
+    QGenericPCIBus *s = container_of(bus, QGenericPCIBus, bus);
+
+    qtest_writeq(bus->qts, s->gpex_pio_base + addr, val);
+}
+
+static void qpci_generic_memread(QPCIBus *bus, uint32_t addr, void *buf, size_t len)
+{
+    qtest_memread(bus->qts, addr, buf, len);
+}
+
+static void qpci_generic_memwrite(QPCIBus *bus, uint32_t addr,
+                                  const void *buf, size_t len)
+{
+    qtest_memwrite(bus->qts, addr, buf, len);
+}
+
+static uint8_t qpci_generic_config_readb(QPCIBus *bus, int devfn, uint8_t offset)
+{
+    QGenericPCIBus *gbus = container_of(bus, QGenericPCIBus, bus);
+    uint64_t addr = gbus->ecam_alloc_ptr + ((0 << 20) | (devfn << 12) | offset);
+    uint8_t val;
+
+    qtest_memread(bus->qts, addr, &val, 1);
+    return val;
+}
+
+static uint16_t qpci_generic_config_readw(QPCIBus *bus, int devfn, uint8_t offset)
+{
+    QGenericPCIBus *gbus = container_of(bus, QGenericPCIBus, bus);
+    uint64_t addr = gbus->ecam_alloc_ptr + ((0 << 20) | (devfn << 12) | offset);
+    uint16_t val;
+
+    qtest_memread(bus->qts, addr, &val, 2);
+    return le16_to_cpu(val);
+}
+
+static uint32_t qpci_generic_config_readl(QPCIBus *bus, int devfn, uint8_t offset)
+{
+    QGenericPCIBus *gbus = container_of(bus, QGenericPCIBus, bus);
+    uint64_t addr = gbus->ecam_alloc_ptr + ((0 << 20) | (devfn << 12) | offset);
+    uint32_t val;
+
+    qtest_memread(bus->qts, addr, &val, 4);
+    return le32_to_cpu(val);
+}
+
+static void
+qpci_generic_config_writeb(QPCIBus *bus, int devfn, uint8_t offset, uint8_t value)
+{
+    QGenericPCIBus *gbus = container_of(bus, QGenericPCIBus, bus);
+    uint64_t addr = gbus->ecam_alloc_ptr + ((0 << 20) | (devfn << 12) | offset);
+
+    qtest_memwrite(bus->qts, addr, &value, 1);
+}
+
+static void
+qpci_generic_config_writew(QPCIBus *bus, int devfn, uint8_t offset, uint16_t value)
+{
+    QGenericPCIBus *gbus = container_of(bus, QGenericPCIBus, bus);
+    uint64_t addr = gbus->ecam_alloc_ptr + ((0 << 20) | (devfn << 12) | offset);
+    uint16_t val = cpu_to_le16(value);
+
+    qtest_memwrite(bus->qts, addr, &val, 2);
+}
+
+static void
+qpci_generic_config_writel(QPCIBus *bus, int devfn, uint8_t offset, uint32_t value)
+{
+    QGenericPCIBus *gbus = container_of(bus, QGenericPCIBus, bus);
+    uint64_t addr = gbus->ecam_alloc_ptr + ((0 << 20) | (devfn << 12) | offset);
+    uint32_t val = cpu_to_le32(value);
+
+    qtest_memwrite(bus->qts, addr, &val, 4);
+}
+
+static void *qpci_generic_get_driver(void *obj, const char *interface)
+{
+    QGenericPCIBus *qpci = obj;
+    if (!g_strcmp0(interface, "pci-bus")) {
+        return &qpci->bus;
+    }
+    fprintf(stderr, "%s not present in pci-bus-generic\n", interface);
+    g_assert_not_reached();
+}
+
+void qpci_init_generic(QGenericPCIBus *qpci, QTestState *qts,
+                       QGuestAllocator *alloc, bool hotpluggable)
+{
+    assert(qts);
+
+    qpci->gpex_pio_base = 0x3eff0000;
+    qpci->bus.not_hotpluggable = !hotpluggable;
+    qpci->bus.has_buggy_msi = false;
+
+    qpci->bus.pio_readb = qpci_generic_pio_readb;
+    qpci->bus.pio_readw = qpci_generic_pio_readw;
+    qpci->bus.pio_readl = qpci_generic_pio_readl;
+    qpci->bus.pio_readq = qpci_generic_pio_readq;
+
+    qpci->bus.pio_writeb = qpci_generic_pio_writeb;
+    qpci->bus.pio_writew = qpci_generic_pio_writew;
+    qpci->bus.pio_writel = qpci_generic_pio_writel;
+    qpci->bus.pio_writeq = qpci_generic_pio_writeq;
+
+    qpci->bus.memread = qpci_generic_memread;
+    qpci->bus.memwrite = qpci_generic_memwrite;
+
+    qpci->bus.config_readb = qpci_generic_config_readb;
+    qpci->bus.config_readw = qpci_generic_config_readw;
+    qpci->bus.config_readl = qpci_generic_config_readl;
+
+    qpci->bus.config_writeb = qpci_generic_config_writeb;
+    qpci->bus.config_writew = qpci_generic_config_writew;
+    qpci->bus.config_writel = qpci_generic_config_writel;
+
+    qpci->bus.qts = qts;
+    qpci->bus.pio_alloc_ptr = 0x0000;
+    qpci->bus.pio_limit = 0x10000;
+    qpci->bus.mmio_alloc_ptr = 0x10000000;
+    qpci->bus.mmio_limit = 0x2eff0000;
+    qpci->ecam_alloc_ptr = 0x4010000000;
+
+    qpci->obj.get_driver = qpci_generic_get_driver;
+}
+
+static void qpci_generic_register_nodes(void)
+{
+    qos_node_create_driver("pci-bus-generic", NULL);
+    qos_node_produces("pci-bus-generic", "pci-bus");
+}
+
+static void qpci_generic_pci_register_nodes(void)
+{
+    qos_node_create_driver("generic-pcihost", NULL);
+    qos_node_contains("generic-pcihost", "pci-bus-generic", NULL);
+}
+
+libqos_init(qpci_generic_register_nodes);
+libqos_init(qpci_generic_pci_register_nodes);
diff --git a/tests/qtest/libqos/generic-pcihost.h b/tests/qtest/libqos/generic-pcihost.h
new file mode 100644
index 0000000000..c693c769df
--- /dev/null
+++ b/tests/qtest/libqos/generic-pcihost.h
@@ -0,0 +1,54 @@
+/*
+ * libqos Generic PCI bindings and generic pci host bridge
+ *
+ * Copyright Red Hat Inc., 2022
+ *
+ * Authors:
+ *  Eric Auger <eric.auger@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef LIBQOS_GENERIC_PCIHOST_H
+#define LIBQOS_GENERIC_PCIHOST_H
+
+#include "pci.h"
+#include "malloc.h"
+#include "qgraph.h"
+
+typedef struct QGenericPCIBus {
+    QOSGraphObject obj;
+    QPCIBus bus;
+    uint64_t gpex_pio_base;
+    uint64_t ecam_alloc_ptr;
+} QGenericPCIBus;
+
+/*
+ * qpci_init_generic():
+ * @ret: A valid QGenericPCIBus * pointer
+ * @qts: The %QTestState
+ * @alloc: A previously initialized @alloc providing memory for @qts
+ * @bool: devices can be hotplugged on this bus
+ *
+ * This function initializes an already allocated
+ * QGenericPCIBus object.
+ */
+void qpci_init_generic(QGenericPCIBus *ret, QTestState *qts,
+                       QGuestAllocator *alloc, bool hotpluggable);
+
+/* QGenericPCIHost */
+
+typedef struct QGenericPCIHost QGenericPCIHost;
+
+struct QGenericPCIHost {
+    QOSGraphObject obj;
+    QGenericPCIBus pci;
+};
+
+QOSGraphObject *generic_pcihost_get_device(void *obj, const char *device);
+void qos_create_generic_pcihost(QGenericPCIHost *host,
+                                QTestState *qts,
+                                QGuestAllocator *alloc);
+
+#endif
diff --git a/tests/qtest/libqos/meson.build b/tests/qtest/libqos/meson.build
index 9f292339f9..fd5d6e5ae1 100644
--- a/tests/qtest/libqos/meson.build
+++ b/tests/qtest/libqos/meson.build
@@ -45,6 +45,7 @@ libqos_srcs = files(
         'virtio-scsi.c',
         'virtio-serial.c',
         'virtio-iommu.c',
+        'generic-pcihost.c',
 
         # qgraph machines:
         'aarch64-xlnx-zcu102-machine.c',

From 98e3ab35054b946f7c2aba5408822532b0920b53 Mon Sep 17 00:00:00 2001
From: Kevin Wolf <kwolf@redhat.com>
Date: Tue, 10 May 2022 17:10:19 +0200
Subject: [PATCH 241/935] coroutine: Rename qemu_coroutine_inc/dec_pool_size()

It's true that these functions currently affect the batch size in which
coroutines are reused (i.e. moved from the global release pool to the
allocation pool of a specific thread), but this is a bug and will be
fixed in a separate patch.

In fact, the comment in the header file already just promises that it
influences the pool size, so reflect this in the name of the functions.
As a nice side effect, the shorter function name makes some line
wrapping unnecessary.

Cc: qemu-stable@nongnu.org
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20220510151020.105528-2-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 hw/block/virtio-blk.c    | 6 ++----
 include/qemu/coroutine.h | 6 +++---
 util/qemu-coroutine.c    | 4 ++--
 3 files changed, 7 insertions(+), 9 deletions(-)

diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index 540c38f829..6a1cc41877 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -1215,8 +1215,7 @@ static void virtio_blk_device_realize(DeviceState *dev, Error **errp)
     for (i = 0; i < conf->num_queues; i++) {
         virtio_add_queue(vdev, conf->queue_size, virtio_blk_handle_output);
     }
-    qemu_coroutine_increase_pool_batch_size(conf->num_queues * conf->queue_size
-                                            / 2);
+    qemu_coroutine_inc_pool_size(conf->num_queues * conf->queue_size / 2);
     virtio_blk_data_plane_create(vdev, conf, &s->dataplane, &err);
     if (err != NULL) {
         error_propagate(errp, err);
@@ -1253,8 +1252,7 @@ static void virtio_blk_device_unrealize(DeviceState *dev)
     for (i = 0; i < conf->num_queues; i++) {
         virtio_del_queue(vdev, i);
     }
-    qemu_coroutine_decrease_pool_batch_size(conf->num_queues * conf->queue_size
-                                            / 2);
+    qemu_coroutine_dec_pool_size(conf->num_queues * conf->queue_size / 2);
     qemu_del_vm_change_state_handler(s->change);
     blockdev_mark_auto_del(s->blk);
     virtio_cleanup(vdev);
diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h
index 284571badb..031cf23711 100644
--- a/include/qemu/coroutine.h
+++ b/include/qemu/coroutine.h
@@ -334,12 +334,12 @@ void coroutine_fn yield_until_fd_readable(int fd);
 /**
  * Increase coroutine pool size
  */
-void qemu_coroutine_increase_pool_batch_size(unsigned int additional_pool_size);
+void qemu_coroutine_inc_pool_size(unsigned int additional_pool_size);
 
 /**
- * Devcrease coroutine pool size
+ * Decrease coroutine pool size
  */
-void qemu_coroutine_decrease_pool_batch_size(unsigned int additional_pool_size);
+void qemu_coroutine_dec_pool_size(unsigned int additional_pool_size);
 
 #include "qemu/lockable.h"
 
diff --git a/util/qemu-coroutine.c b/util/qemu-coroutine.c
index f3e8300c8d..ea23929a74 100644
--- a/util/qemu-coroutine.c
+++ b/util/qemu-coroutine.c
@@ -212,12 +212,12 @@ AioContext *coroutine_fn qemu_coroutine_get_aio_context(Coroutine *co)
     return co->ctx;
 }
 
-void qemu_coroutine_increase_pool_batch_size(unsigned int additional_pool_size)
+void qemu_coroutine_inc_pool_size(unsigned int additional_pool_size)
 {
     qatomic_add(&pool_batch_size, additional_pool_size);
 }
 
-void qemu_coroutine_decrease_pool_batch_size(unsigned int removing_pool_size)
+void qemu_coroutine_dec_pool_size(unsigned int removing_pool_size)
 {
     qatomic_sub(&pool_batch_size, removing_pool_size);
 }

From 9ec7a59b5aad4b736871c378d30f5ef5ec51cb52 Mon Sep 17 00:00:00 2001
From: Kevin Wolf <kwolf@redhat.com>
Date: Tue, 10 May 2022 17:10:20 +0200
Subject: [PATCH 242/935] coroutine: Revert to constant batch size

Commit 4c41c69e changed the way the coroutine pool is sized because for
virtio-blk devices with a large queue size and heavy I/O, it was just
too small and caused coroutines to be deleted and reallocated soon
afterwards. The change made the size dynamic based on the number of
queues and the queue size of virtio-blk devices.

There are two important numbers here: Slightly simplified, when a
coroutine terminates, it is generally stored in the global release pool
up to a certain pool size, and if the pool is full, it is freed.
Conversely, when allocating a new coroutine, the coroutines in the
release pool are reused if the pool already has reached a certain
minimum size (the batch size), otherwise we allocate new coroutines.

The problem after commit 4c41c69e is that it not only increases the
maximum pool size (which is the intended effect), but also the batch
size for reusing coroutines (which is a bug). It means that in cases
with many devices and/or a large queue size (which defaults to the
number of vcpus for virtio-blk-pci), many thousand coroutines could be
sitting in the release pool without being reused.

This is not only a waste of memory and allocations, but it actually
makes the QEMU process likely to hit the vm.max_map_count limit on Linux
because each coroutine requires two mappings (its stack and the guard
page for the stack), causing it to abort() in qemu_alloc_stack() because
when the limit is hit, mprotect() starts to fail with ENOMEM.

In order to fix the problem, change the batch size back to 64 to avoid
uselessly accumulating coroutines in the release pool, but keep the
dynamic maximum pool size so that coroutines aren't freed too early
in heavy I/O scenarios.

Note that this fix doesn't strictly make it impossible to hit the limit,
but this would only happen if most of the coroutines are actually in use
at the same time, not just sitting in a pool. This is the same behaviour
as we already had before commit 4c41c69e. Fully preventing this would
require allowing qemu_coroutine_create() to return an error, but it
doesn't seem to be a scenario that people hit in practice.

Cc: qemu-stable@nongnu.org
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2079938
Fixes: 4c41c69e05fe28c0f95f8abd2ebf407e95a4f04b
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20220510151020.105528-3-kwolf@redhat.com>
Tested-by: Hiroki Narukawa <hnarukaw@yahoo-corp.jp>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 util/qemu-coroutine.c | 22 ++++++++++++++--------
 1 file changed, 14 insertions(+), 8 deletions(-)

diff --git a/util/qemu-coroutine.c b/util/qemu-coroutine.c
index ea23929a74..4a8bd63ef0 100644
--- a/util/qemu-coroutine.c
+++ b/util/qemu-coroutine.c
@@ -21,14 +21,20 @@
 #include "qemu/coroutine-tls.h"
 #include "block/aio.h"
 
-/** Initial batch size is 64, and is increased on demand */
+/**
+ * The minimal batch size is always 64, coroutines from the release_pool are
+ * reused as soon as there are 64 coroutines in it. The maximum pool size starts
+ * with 64 and is increased on demand so that coroutines are not deleted even if
+ * they are not immediately reused.
+ */
 enum {
-    POOL_INITIAL_BATCH_SIZE = 64,
+    POOL_MIN_BATCH_SIZE = 64,
+    POOL_INITIAL_MAX_SIZE = 64,
 };
 
 /** Free list to speed up creation */
 static QSLIST_HEAD(, Coroutine) release_pool = QSLIST_HEAD_INITIALIZER(pool);
-static unsigned int pool_batch_size = POOL_INITIAL_BATCH_SIZE;
+static unsigned int pool_max_size = POOL_INITIAL_MAX_SIZE;
 static unsigned int release_pool_size;
 
 typedef QSLIST_HEAD(, Coroutine) CoroutineQSList;
@@ -57,7 +63,7 @@ Coroutine *qemu_coroutine_create(CoroutineEntry *entry, void *opaque)
 
         co = QSLIST_FIRST(alloc_pool);
         if (!co) {
-            if (release_pool_size > qatomic_read(&pool_batch_size)) {
+            if (release_pool_size > POOL_MIN_BATCH_SIZE) {
                 /* Slow path; a good place to register the destructor, too.  */
                 Notifier *notifier = get_ptr_coroutine_pool_cleanup_notifier();
                 if (!notifier->notify) {
@@ -95,12 +101,12 @@ static void coroutine_delete(Coroutine *co)
     co->caller = NULL;
 
     if (CONFIG_COROUTINE_POOL) {
-        if (release_pool_size < qatomic_read(&pool_batch_size) * 2) {
+        if (release_pool_size < qatomic_read(&pool_max_size) * 2) {
             QSLIST_INSERT_HEAD_ATOMIC(&release_pool, co, pool_next);
             qatomic_inc(&release_pool_size);
             return;
         }
-        if (get_alloc_pool_size() < qatomic_read(&pool_batch_size)) {
+        if (get_alloc_pool_size() < qatomic_read(&pool_max_size)) {
             QSLIST_INSERT_HEAD(get_ptr_alloc_pool(), co, pool_next);
             set_alloc_pool_size(get_alloc_pool_size() + 1);
             return;
@@ -214,10 +220,10 @@ AioContext *coroutine_fn qemu_coroutine_get_aio_context(Coroutine *co)
 
 void qemu_coroutine_inc_pool_size(unsigned int additional_pool_size)
 {
-    qatomic_add(&pool_batch_size, additional_pool_size);
+    qatomic_add(&pool_max_size, additional_pool_size);
 }
 
 void qemu_coroutine_dec_pool_size(unsigned int removing_pool_size)
 {
-    qatomic_sub(&pool_batch_size, removing_pool_size);
+    qatomic_sub(&pool_max_size, removing_pool_size);
 }

From 22d92e71c77c8690995315c9ef5a1136c6300047 Mon Sep 17 00:00:00 2001
From: Hanna Reitz <hreitz@redhat.com>
Date: Fri, 6 May 2022 15:42:15 +0200
Subject: [PATCH 243/935] iotests/testrunner: Flush after run_test()

When stdout is not a terminal, the buffer may not be flushed at each end
of line, so we should flush after each test is done.  This is especially
apparent when run by check-block, in two ways:

First, when running make check-block -jX with X > 1, progress indication
was missing, even though testrunner.py does theoretically print each
test's status once it has been run, even in multi-processing mode.
Flushing after each test restores this progress indication.

Second, sometimes make check-block failed altogether, with an error
message that "too few tests [were] run".  I presume that's because one
worker process in the job pool did not get to flush its stdout before
the main process exited, and so meson did not get to see that worker's
test results.  In any case, by flushing at the end of run_test(), the
problem has disappeared for me.

Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220506134215.10086-1-hreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 tests/qemu-iotests/testrunner.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py
index aae70a8341..10d9e8ef27 100644
--- a/tests/qemu-iotests/testrunner.py
+++ b/tests/qemu-iotests/testrunner.py
@@ -378,6 +378,7 @@ class TestRunner(ContextManager['TestRunner']):
             else:
                 print(res.casenotrun)
 
+        sys.stdout.flush()
         return res
 
     def run_tests(self, tests: List[str], jobs: int = 1) -> bool:

From 5e781c700a6ebf089cb01eb4f612479bfcfe186d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Mon, 9 May 2022 13:41:33 +0100
Subject: [PATCH 244/935] tests/qemu-iotests: print intent to run a test in TAP
 mode
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

When running I/O tests using TAP output mode, we get a single TAP test
with a sub-test reported for each I/O test that is run. The output looks
something like this:

 1..123
 ok qcow2 011
 ok qcow2 012
 ok qcow2 013
 ok qcow2 217
 ...

If everything runs or fails normally this is fine, but periodically we
have been seeing the test harness abort early before all 123 tests have
been run, just leaving a fairly useless message like

  TAP parsing error: Too few tests run (expected 123, got 107)

we have no idea which tests were running at the time the test harness
abruptly exited. This change causes us to print a message about our
intent to run each test, so we have a record of what is active at the
time the harness exits abnormally.

 1..123
 # running qcow2 011
 ok qcow2 011
 # running qcow2 012
 ok qcow2 012
 # running qcow2 013
 ok qcow2 013
 # running qcow2 217
 ok qcow2 217
 ...

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220509124134.867431-2-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 tests/qemu-iotests/testrunner.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py
index 10d9e8ef27..5a771da86e 100644
--- a/tests/qemu-iotests/testrunner.py
+++ b/tests/qemu-iotests/testrunner.py
@@ -361,6 +361,9 @@ class TestRunner(ContextManager['TestRunner']):
                                      starttime=start,
                                      lasttime=last_el,
                                      end = '\n' if mp else '\r')
+        else:
+            testname = os.path.basename(test)
+            print(f'# running {self.env.imgfmt} {testname}')
 
         res = self.do_run_test(test, mp)
 

From 29a493765ecd63e2b8c1593c777877ee1f8f1cd1 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Mon, 9 May 2022 13:41:34 +0100
Subject: [PATCH 245/935] .gitlab-ci.d: export meson testlog.txt as an artifact
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

When running 'make check' we only get a summary of progress on the
console. Fortunately meson/ninja have saved the raw test output to a
logfile. Exposing this log will make it easier to debug failures that
happen in CI.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220509124134.867431-3-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 2c7980a4f6..dc6d67aacf 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -26,7 +26,7 @@
         make -j"$JOBS" $MAKE_CHECK_ARGS ;
       fi
 
-.native_test_job_template:
+.common_test_job_template:
   stage: test
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
   script:
@@ -37,8 +37,16 @@
     # Avoid recompiling by hiding ninja with NINJA=":"
     - make NINJA=":" $MAKE_CHECK_ARGS
 
+.native_test_job_template:
+  extends: .common_test_job_template
+  artifacts:
+    name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
+    expire_in: 7 days
+    paths:
+      - build/meson-logs/testlog.txt
+
 .avocado_test_job_template:
-  extends: .native_test_job_template
+  extends: .common_test_job_template
   cache:
     key: "${CI_JOB_NAME}-cache"
     paths:

From 97ec4d21e09b5e4a59f00c471a7f76533b08ce56 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Thu, 14 Apr 2022 12:52:56 -0400
Subject: [PATCH 246/935] machine: use QAPI struct for boot configuration

As part of converting -boot to a property with a QAPI type, define
the struct and use it throughout QEMU to access boot configuration.
machine_boot_parse takes care of doing the QemuOpts->QAPI conversion by
hand, for now.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220414165300.555321-2-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/arm/nseries.c        |  2 +-
 hw/core/machine.c       | 68 +++++++++++++++++++++++++++++++++++++++--
 hw/hppa/machine.c       |  6 ++--
 hw/i386/pc.c            |  2 +-
 hw/nvram/fw_cfg.c       | 27 +++++-----------
 hw/ppc/mac_newworld.c   |  2 +-
 hw/ppc/mac_oldworld.c   |  2 +-
 hw/ppc/prep.c           |  2 +-
 hw/ppc/spapr.c          |  4 +--
 hw/s390x/ipl.c          | 20 ++++--------
 hw/sparc/sun4m.c        |  4 +--
 hw/sparc64/sun4u.c      |  4 +--
 include/hw/boards.h     |  4 +--
 include/sysemu/sysemu.h |  2 --
 qapi/machine.json       | 30 ++++++++++++++++++
 softmmu/bootdevice.c    |  3 +-
 softmmu/globals.c       |  2 --
 softmmu/vl.c            | 25 +--------------
 18 files changed, 127 insertions(+), 82 deletions(-)

diff --git a/hw/arm/nseries.c b/hw/arm/nseries.c
index 9c1cafae86..692c94ceb4 100644
--- a/hw/arm/nseries.c
+++ b/hw/arm/nseries.c
@@ -1365,7 +1365,7 @@ static void n8x0_init(MachineState *machine,
     }
 
     if (option_rom[0].name &&
-        (machine->boot_order[0] == 'n' || !machine->kernel_filename)) {
+        (machine->boot_config.order[0] == 'n' || !machine->kernel_filename)) {
         uint8_t *nolo_tags = g_new(uint8_t, 0x10000);
         /* No, wait, better start at the ROM.  */
         s->mpu->cpu->env.regs[15] = OMAP2_Q2_BASE + 0x400000;
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 700c1e76b8..b3deb8146f 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -784,6 +784,68 @@ static void machine_set_smp(Object *obj, Visitor *v, const char *name,
     machine_parse_smp_config(ms, config, errp);
 }
 
+void machine_boot_parse(MachineState *ms, QemuOpts *opts, Error **errp)
+{
+    MachineClass *machine_class = MACHINE_GET_CLASS(ms);
+    const char *s;
+    ERRP_GUARD();
+
+    ms->boot_config = (BootConfiguration) {
+        .has_order = true,
+        .order = (char *)machine_class->default_boot_order,
+        .has_strict = true,
+        .strict = false,
+    };
+    if (!opts) {
+        return;
+    }
+
+    s = qemu_opt_get(opts, "order");
+    if (s) {
+        validate_bootdevices(s, errp);
+        if (*errp) {
+            return;
+        }
+        ms->boot_config.order = (char *)s;
+    }
+
+    s = qemu_opt_get(opts, "once");
+    if (s) {
+        validate_bootdevices(s, errp);
+        if (*errp) {
+            return;
+        }
+        ms->boot_config.has_once = true;
+        ms->boot_config.once = (char *)s;
+    }
+
+    s = qemu_opt_get(opts, "splash");
+    if (s) {
+        ms->boot_config.has_splash = true;
+        ms->boot_config.splash = (char *)s;
+    }
+
+    s = qemu_opt_get(opts, "splash-time");
+    if (s) {
+        ms->boot_config.has_splash_time = true;
+        ms->boot_config.splash_time = qemu_opt_get_number(opts, "splash-time", -1);
+    }
+
+    s = qemu_opt_get(opts, "reboot-timeout");
+    if (s) {
+        ms->boot_config.has_reboot_timeout = true;
+        ms->boot_config.reboot_timeout = qemu_opt_get_number(opts, "reboot-timeout", -1);
+    }
+
+    s = qemu_opt_get(opts, "menu");
+    if (s) {
+        ms->boot_config.has_menu = true;
+        ms->boot_config.menu = qemu_opt_get_bool(opts, "menu", false);
+    }
+
+    ms->boot_config.strict = qemu_opt_get_bool(opts, "strict", false);
+}
+
 static void machine_class_init(ObjectClass *oc, void *data)
 {
     MachineClass *mc = MACHINE_CLASS(oc);
@@ -1229,9 +1291,9 @@ void qdev_machine_creation_done(void)
 {
     cpu_synchronize_all_post_init();
 
-    if (current_machine->boot_once) {
-        qemu_boot_set(current_machine->boot_once, &error_fatal);
-        qemu_register_reset(restore_boot_order, g_strdup(current_machine->boot_order));
+    if (current_machine->boot_config.has_once) {
+        qemu_boot_set(current_machine->boot_config.once, &error_fatal);
+        qemu_register_reset(restore_boot_order, g_strdup(current_machine->boot_config.order));
     }
 
     /*
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 4d054ca869..d1e174b1f4 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -147,7 +147,7 @@ static FWCfgState *create_fw_cfg(MachineState *ms)
     fw_cfg_add_file(fw_cfg, "/etc/power-button-addr",
                     g_memdup(&val, sizeof(val)), sizeof(val));
 
-    fw_cfg_add_i16(fw_cfg, FW_CFG_BOOT_DEVICE, ms->boot_order[0]);
+    fw_cfg_add_i16(fw_cfg, FW_CFG_BOOT_DEVICE, ms->boot_config.order[0]);
     qemu_register_boot_set(fw_cfg_boot_set, fw_cfg);
 
     return fw_cfg;
@@ -391,8 +391,8 @@ static void machine_hppa_init(MachineState *machine)
          * mode (kernel_entry=1), and to boot from CD (gr[24]='d')
          * or hard disc * (gr[24]='c').
          */
-        kernel_entry = boot_menu ? 1 : 0;
-        cpu[0]->env.gr[24] = machine->boot_order[0];
+        kernel_entry = machine->boot_config.has_menu ? machine->boot_config.menu : 0;
+        cpu[0]->env.gr[24] = machine->boot_config.order[0];
     }
 
     /* We jump to the firmware entry routine and pass the
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 23bba9d82c..305d2c0820 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -675,7 +675,7 @@ void pc_cmos_init(PCMachineState *pcms,
     object_property_set_link(OBJECT(pcms), "rtc_state", OBJECT(s),
                              &error_abort);
 
-    set_boot_dev(s, MACHINE(pcms)->boot_order, &error_fatal);
+    set_boot_dev(s, MACHINE(pcms)->boot_config.order, &error_fatal);
 
     val = 0;
     val |= 0x02; /* FPU is there */
diff --git a/hw/nvram/fw_cfg.c b/hw/nvram/fw_cfg.c
index 4125cbebcd..d605f3f45a 100644
--- a/hw/nvram/fw_cfg.c
+++ b/hw/nvram/fw_cfg.c
@@ -178,21 +178,13 @@ error:
 
 static void fw_cfg_bootsplash(FWCfgState *s)
 {
-    const char *boot_splash_filename = NULL;
-    const char *boot_splash_time = NULL;
     char *filename, *file_data;
     gsize file_size;
     int file_type;
 
-    /* get user configuration */
-    QemuOptsList *plist = qemu_find_opts("boot-opts");
-    QemuOpts *opts = QTAILQ_FIRST(&plist->head);
-    boot_splash_filename = qemu_opt_get(opts, "splash");
-    boot_splash_time = qemu_opt_get(opts, "splash-time");
-
     /* insert splash time if user configurated */
-    if (boot_splash_time) {
-        int64_t bst_val = qemu_opt_get_number(opts, "splash-time", -1);
+    if (current_machine->boot_config.has_splash_time) {
+        int64_t bst_val = current_machine->boot_config.splash_time;
         uint16_t bst_le16;
 
         /* validate the input */
@@ -208,7 +200,8 @@ static void fw_cfg_bootsplash(FWCfgState *s)
     }
 
     /* insert splash file if user configurated */
-    if (boot_splash_filename) {
+    if (current_machine->boot_config.has_splash) {
+        const char *boot_splash_filename = current_machine->boot_config.splash;
         filename = qemu_find_file(QEMU_FILE_TYPE_BIOS, boot_splash_filename);
         if (filename == NULL) {
             error_report("failed to find file '%s'", boot_splash_filename);
@@ -238,17 +231,11 @@ static void fw_cfg_bootsplash(FWCfgState *s)
 
 static void fw_cfg_reboot(FWCfgState *s)
 {
-    const char *reboot_timeout = NULL;
     uint64_t rt_val = -1;
     uint32_t rt_le32;
 
-    /* get user configuration */
-    QemuOptsList *plist = qemu_find_opts("boot-opts");
-    QemuOpts *opts = QTAILQ_FIRST(&plist->head);
-    reboot_timeout = qemu_opt_get(opts, "reboot-timeout");
-
-    if (reboot_timeout) {
-        rt_val = qemu_opt_get_number(opts, "reboot-timeout", -1);
+    if (current_machine->boot_config.has_reboot_timeout) {
+        rt_val = current_machine->boot_config.reboot_timeout;
 
         /* validate the input */
         if (rt_val > 0xffff && rt_val != (uint64_t)-1) {
@@ -1133,7 +1120,7 @@ static void fw_cfg_common_realize(DeviceState *dev, Error **errp)
     fw_cfg_add_bytes(s, FW_CFG_SIGNATURE, (char *)"QEMU", 4);
     fw_cfg_add_bytes(s, FW_CFG_UUID, &qemu_uuid, 16);
     fw_cfg_add_i16(s, FW_CFG_NOGRAPHIC, (uint16_t)!machine->enable_graphics);
-    fw_cfg_add_i16(s, FW_CFG_BOOT_MENU, (uint16_t)boot_menu);
+    fw_cfg_add_i16(s, FW_CFG_BOOT_MENU, (uint16_t)(machine->boot_config.has_menu && machine->boot_config.menu));
     fw_cfg_bootsplash(s);
     fw_cfg_reboot(s);
 
diff --git a/hw/ppc/mac_newworld.c b/hw/ppc/mac_newworld.c
index e8ef1a9e5d..c865921bdc 100644
--- a/hw/ppc/mac_newworld.c
+++ b/hw/ppc/mac_newworld.c
@@ -111,7 +111,7 @@ static void ppc_core99_init(MachineState *machine)
     const char *kernel_filename = machine->kernel_filename;
     const char *kernel_cmdline = machine->kernel_cmdline;
     const char *initrd_filename = machine->initrd_filename;
-    const char *boot_device = machine->boot_order;
+    const char *boot_device = machine->boot_config.order;
     Core99MachineState *core99_machine = CORE99_MACHINE(machine);
     PowerPCCPU *cpu = NULL;
     CPUPPCState *env = NULL;
diff --git a/hw/ppc/mac_oldworld.c b/hw/ppc/mac_oldworld.c
index fe2adb057b..d62fdf0db3 100644
--- a/hw/ppc/mac_oldworld.c
+++ b/hw/ppc/mac_oldworld.c
@@ -82,7 +82,7 @@ static void ppc_heathrow_init(MachineState *machine)
 {
     ram_addr_t ram_size = machine->ram_size;
     const char *bios_name = machine->firmware ?: PROM_FILENAME;
-    const char *boot_device = machine->boot_order;
+    const char *boot_device = machine->boot_config.order;
     PowerPCCPU *cpu = NULL;
     CPUPPCState *env = NULL;
     char *filename;
diff --git a/hw/ppc/prep.c b/hw/ppc/prep.c
index bf622aa38f..a1cd4505cc 100644
--- a/hw/ppc/prep.c
+++ b/hw/ppc/prep.c
@@ -381,7 +381,7 @@ static void ibm_40p_init(MachineState *machine)
         }
         boot_device = 'm';
     } else {
-        boot_device = machine->boot_order[0];
+        boot_device = machine->boot_config.order[0];
     }
 
     fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS, (uint16_t)machine->smp.max_cpus);
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 8bbae68e1b..6de800524a 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1044,8 +1044,8 @@ static void spapr_dt_chosen(SpaprMachineState *spapr, void *fdt, bool reset)
                 _FDT(fdt_setprop(fdt, chosen, "qemu,boot-kernel-le", NULL, 0));
             }
         }
-        if (boot_menu) {
-            _FDT((fdt_setprop_cell(fdt, chosen, "qemu,boot-menu", boot_menu)));
+        if (machine->boot_config.has_menu && machine->boot_config.menu) {
+            _FDT((fdt_setprop_cell(fdt, chosen, "qemu,boot-menu", true)));
         }
         _FDT(fdt_setprop_cell(fdt, chosen, "qemu,graphic-width", graphic_width));
         _FDT(fdt_setprop_cell(fdt, chosen, "qemu,graphic-height", graphic_height));
diff --git a/hw/s390x/ipl.c b/hw/s390x/ipl.c
index 4b5eb77afd..8612684d48 100644
--- a/hw/s390x/ipl.c
+++ b/hw/s390x/ipl.c
@@ -290,13 +290,10 @@ static Property s390_ipl_properties[] = {
 
 static void s390_ipl_set_boot_menu(S390IPLState *ipl)
 {
-    QemuOptsList *plist = qemu_find_opts("boot-opts");
-    QemuOpts *opts = QTAILQ_FIRST(&plist->head);
-    const char *tmp;
     unsigned long splash_time = 0;
 
     if (!get_boot_device(0)) {
-        if (boot_menu) {
+        if (current_machine->boot_config.has_menu && current_machine->boot_config.menu) {
             error_report("boot menu requires a bootindex to be specified for "
                          "the IPL device");
         }
@@ -306,7 +303,7 @@ static void s390_ipl_set_boot_menu(S390IPLState *ipl)
     switch (ipl->iplb.pbt) {
     case S390_IPL_TYPE_CCW:
         /* In the absence of -boot menu, use zipl parameters */
-        if (!qemu_opt_get(opts, "menu")) {
+        if (!current_machine->boot_config.has_menu) {
             ipl->qipl.qipl_flags |= QIPL_FLAG_BM_OPTS_ZIPL;
             return;
         }
@@ -314,26 +311,21 @@ static void s390_ipl_set_boot_menu(S390IPLState *ipl)
     case S390_IPL_TYPE_QEMU_SCSI:
         break;
     default:
-        if (boot_menu) {
+        if (current_machine->boot_config.has_menu && current_machine->boot_config.menu) {
             error_report("boot menu is not supported for this device type");
         }
         return;
     }
 
-    if (!boot_menu) {
+    if (!current_machine->boot_config.has_menu || !current_machine->boot_config.menu) {
         return;
     }
 
     ipl->qipl.qipl_flags |= QIPL_FLAG_BM_OPTS_CMD;
 
-    tmp = qemu_opt_get(opts, "splash-time");
-
-    if (tmp && qemu_strtoul(tmp, NULL, 10, &splash_time)) {
-        error_report("splash-time is invalid, forcing it to 0");
-        ipl->qipl.boot_menu_timeout = 0;
-        return;
+    if (current_machine->boot_config.has_splash_time) {
+        splash_time = current_machine->boot_config.splash_time;
     }
-
     if (splash_time > 0xffffffff) {
         error_report("splash-time is too large, forcing it to max value");
         ipl->qipl.boot_menu_timeout = 0xffffffff;
diff --git a/hw/sparc/sun4m.c b/hw/sparc/sun4m.c
index b693eea0e0..9d57491f68 100644
--- a/hw/sparc/sun4m.c
+++ b/hw/sparc/sun4m.c
@@ -1050,7 +1050,7 @@ static void sun4m_hw_init(MachineState *machine)
                                     machine->ram_size, &initrd_size);
 
     nvram_init(nvram, (uint8_t *)&nd->macaddr, machine->kernel_cmdline,
-               machine->boot_order, machine->ram_size, kernel_size,
+               machine->boot_config.order, machine->ram_size, kernel_size,
                graphic_width, graphic_height, graphic_depth,
                hwdef->nvram_machine_id, "Sun4m");
 
@@ -1091,7 +1091,7 @@ static void sun4m_hw_init(MachineState *machine)
     }
     fw_cfg_add_i32(fw_cfg, FW_CFG_INITRD_ADDR, INITRD_LOAD_ADDR);
     fw_cfg_add_i32(fw_cfg, FW_CFG_INITRD_SIZE, initrd_size);
-    fw_cfg_add_i16(fw_cfg, FW_CFG_BOOT_DEVICE, machine->boot_order[0]);
+    fw_cfg_add_i16(fw_cfg, FW_CFG_BOOT_DEVICE, machine->boot_config.order[0]);
     qemu_register_boot_set(fw_cfg_boot_set, fw_cfg);
 }
 
diff --git a/hw/sparc64/sun4u.c b/hw/sparc64/sun4u.c
index 7c461d194a..d1bc77d27e 100644
--- a/hw/sparc64/sun4u.c
+++ b/hw/sparc64/sun4u.c
@@ -695,7 +695,7 @@ static void sun4uv_init(MemoryRegion *address_space_mem,
                                     &kernel_addr, &kernel_entry);
 
     sun4u_NVRAM_set_params(nvram, NVRAM_SIZE, "Sun4u", machine->ram_size,
-                           machine->boot_order,
+                           machine->boot_config.order,
                            kernel_addr, kernel_size,
                            machine->kernel_cmdline,
                            initrd_addr, initrd_size,
@@ -727,7 +727,7 @@ static void sun4uv_init(MemoryRegion *address_space_mem,
     }
     fw_cfg_add_i64(fw_cfg, FW_CFG_INITRD_ADDR, initrd_addr);
     fw_cfg_add_i64(fw_cfg, FW_CFG_INITRD_SIZE, initrd_size);
-    fw_cfg_add_i16(fw_cfg, FW_CFG_BOOT_DEVICE, machine->boot_order[0]);
+    fw_cfg_add_i16(fw_cfg, FW_CFG_BOOT_DEVICE, machine->boot_config.order[0]);
 
     fw_cfg_add_i16(fw_cfg, FW_CFG_SPARC64_WIDTH, graphic_width);
     fw_cfg_add_i16(fw_cfg, FW_CFG_SPARC64_HEIGHT, graphic_height);
diff --git a/include/hw/boards.h b/include/hw/boards.h
index d64b5481e8..6cda7e4308 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -26,6 +26,7 @@ OBJECT_DECLARE_TYPE(MachineState, MachineClass, MACHINE)
 extern MachineState *current_machine;
 
 void machine_run_board_init(MachineState *machine);
+void machine_boot_parse(MachineState *ms, QemuOpts *opts, Error **errp);
 bool machine_usb(MachineState *machine);
 int machine_phandle_start(MachineState *machine);
 bool machine_dump_guest_core(MachineState *machine);
@@ -350,8 +351,7 @@ struct MachineState {
     ram_addr_t ram_size;
     ram_addr_t maxram_size;
     uint64_t   ram_slots;
-    const char *boot_order;
-    const char *boot_once;
+    BootConfiguration boot_config;
     char *kernel_filename;
     char *kernel_cmdline;
     char *initrd_filename;
diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index 360a408edf..b4030acd74 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -46,8 +46,6 @@ extern int alt_grab;
 extern int ctrl_grab;
 extern int graphic_rotate;
 extern int old_param;
-extern int boot_menu;
-extern bool boot_strict;
 extern uint8_t *boot_splash_filedata;
 extern bool enable_mlock;
 extern bool enable_cpu_pm;
diff --git a/qapi/machine.json b/qapi/machine.json
index 4c417e32a5..e3dcf5a119 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1395,6 +1395,36 @@
   'data': { 'device': 'str', 'msg': 'str' },
   'features': ['deprecated'] }
 
+##
+# @BootConfiguration:
+#
+# Schema for virtual machine boot configuration.
+#
+# @order: Boot order (a=floppy, c=hard disk, d=CD-ROM, n=network)
+#
+# @once: Boot order to apply on first boot
+#
+# @menu: Whether to show a boot menu
+#
+# @splash: The name of the file to be passed to the firmware as logo picture, if @menu is true.
+#
+# @splash-time: How long to show the logo picture, in milliseconds
+#
+# @reboot-timeout: Timeout before guest reboots after boot fails
+#
+# @strict: Whether to attempt booting from devices not included in the boot order
+#
+# Since: 7.1
+##
+{ 'struct': 'BootConfiguration', 'data': {
+     '*order': 'str',
+     '*once': 'str',
+     '*menu': 'bool',
+     '*splash': 'str',
+     '*splash-time': 'int',
+     '*reboot-timeout': 'int',
+     '*strict': 'bool' } }
+
 ##
 # @SMPConfiguration:
 #
diff --git a/softmmu/bootdevice.c b/softmmu/bootdevice.c
index c0713bfa9f..2106f1026f 100644
--- a/softmmu/bootdevice.c
+++ b/softmmu/bootdevice.c
@@ -268,7 +268,8 @@ char *get_boot_devices_list(size_t *size)
 
     *size = total;
 
-    if (boot_strict && *size > 0) {
+    if (current_machine->boot_config.has_strict &&
+        current_machine->boot_config.strict && *size > 0) {
         list[total-1] = '\n';
         list = g_realloc(list, total + 5);
         memcpy(&list[total], "HALT", 5);
diff --git a/softmmu/globals.c b/softmmu/globals.c
index 98b64e0492..916bc12e2b 100644
--- a/softmmu/globals.c
+++ b/softmmu/globals.c
@@ -54,8 +54,6 @@ int alt_grab;
 int ctrl_grab;
 unsigned int nb_prom_envs;
 const char *prom_envs[MAX_PROM_ENVS];
-int boot_menu;
-bool boot_strict;
 uint8_t *boot_splash_filedata;
 int only_migratable; /* turn it off unless user states otherwise */
 int icount_align_option;
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 488cc4d09e..dd90df3ed1 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -1884,9 +1884,6 @@ static bool object_create_early(const char *type)
 
 static void qemu_apply_machine_options(QDict *qdict)
 {
-    MachineClass *machine_class = MACHINE_GET_CLASS(current_machine);
-    const char *boot_order = NULL;
-    const char *boot_once = NULL;
     QemuOpts *opts;
 
     object_set_properties_from_keyval(OBJECT(current_machine), qdict, false, &error_fatal);
@@ -1895,27 +1892,7 @@ static void qemu_apply_machine_options(QDict *qdict)
     current_machine->ram_slots = ram_slots;
 
     opts = qemu_opts_find(qemu_find_opts("boot-opts"), NULL);
-    if (opts) {
-        boot_order = qemu_opt_get(opts, "order");
-        if (boot_order) {
-            validate_bootdevices(boot_order, &error_fatal);
-        }
-
-        boot_once = qemu_opt_get(opts, "once");
-        if (boot_once) {
-            validate_bootdevices(boot_once, &error_fatal);
-        }
-
-        boot_menu = qemu_opt_get_bool(opts, "menu", boot_menu);
-        boot_strict = qemu_opt_get_bool(opts, "strict", false);
-    }
-
-    if (!boot_order) {
-        boot_order = machine_class->default_boot_order;
-    }
-
-    current_machine->boot_order = boot_order;
-    current_machine->boot_once = boot_once;
+    machine_boot_parse(current_machine, opts, &error_fatal);
 
     if (semihosting_enabled() && !semihosting_get_argc()) {
         /* fall back to the -kernel/-append */

From 8c4da4b52186e9d0c7233b0ffc796e78fdf3e7b3 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Thu, 14 Apr 2022 12:52:57 -0400
Subject: [PATCH 247/935] machine: add boot compound property

Make -boot syntactic sugar for a compound property "-machine boot.{order,menu,...}".
machine_boot_parse is replaced by the setter for the property.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220414165300.555321-3-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/core/machine.c   | 100 +++++++++++++++++++++++---------------------
 include/hw/boards.h |   1 -
 softmmu/vl.c        |  16 +++----
 3 files changed, 58 insertions(+), 59 deletions(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index b3deb8146f..8cea94537d 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -784,66 +784,63 @@ static void machine_set_smp(Object *obj, Visitor *v, const char *name,
     machine_parse_smp_config(ms, config, errp);
 }
 
-void machine_boot_parse(MachineState *ms, QemuOpts *opts, Error **errp)
+static void machine_get_boot(Object *obj, Visitor *v, const char *name,
+                            void *opaque, Error **errp)
+{
+    MachineState *ms = MACHINE(obj);
+    BootConfiguration *config = &ms->boot_config;
+    visit_type_BootConfiguration(v, name, &config, &error_abort);
+}
+
+static void machine_free_boot_config(MachineState *ms)
+{
+    g_free(ms->boot_config.order);
+    g_free(ms->boot_config.once);
+    g_free(ms->boot_config.splash);
+}
+
+static void machine_copy_boot_config(MachineState *ms, BootConfiguration *config)
 {
     MachineClass *machine_class = MACHINE_GET_CLASS(ms);
-    const char *s;
-    ERRP_GUARD();
 
-    ms->boot_config = (BootConfiguration) {
-        .has_order = true,
-        .order = (char *)machine_class->default_boot_order,
-        .has_strict = true,
-        .strict = false,
-    };
-    if (!opts) {
+    machine_free_boot_config(ms);
+    ms->boot_config = *config;
+    if (!config->has_order) {
+        ms->boot_config.has_order = true;
+        ms->boot_config.order = g_strdup(machine_class->default_boot_order);
+    }
+}
+
+static void machine_set_boot(Object *obj, Visitor *v, const char *name,
+                            void *opaque, Error **errp)
+{
+    ERRP_GUARD();
+    MachineState *ms = MACHINE(obj);
+    BootConfiguration *config = NULL;
+
+    if (!visit_type_BootConfiguration(v, name, &config, errp)) {
         return;
     }
-
-    s = qemu_opt_get(opts, "order");
-    if (s) {
-        validate_bootdevices(s, errp);
+    if (config->has_order) {
+        validate_bootdevices(config->order, errp);
         if (*errp) {
-            return;
+            goto out_free;
         }
-        ms->boot_config.order = (char *)s;
     }
-
-    s = qemu_opt_get(opts, "once");
-    if (s) {
-        validate_bootdevices(s, errp);
+    if (config->has_once) {
+        validate_bootdevices(config->once, errp);
         if (*errp) {
-            return;
+            goto out_free;
         }
-        ms->boot_config.has_once = true;
-        ms->boot_config.once = (char *)s;
     }
 
-    s = qemu_opt_get(opts, "splash");
-    if (s) {
-        ms->boot_config.has_splash = true;
-        ms->boot_config.splash = (char *)s;
-    }
+    machine_copy_boot_config(ms, config);
+    /* Strings live in ms->boot_config.  */
+    free(config);
+    return;
 
-    s = qemu_opt_get(opts, "splash-time");
-    if (s) {
-        ms->boot_config.has_splash_time = true;
-        ms->boot_config.splash_time = qemu_opt_get_number(opts, "splash-time", -1);
-    }
-
-    s = qemu_opt_get(opts, "reboot-timeout");
-    if (s) {
-        ms->boot_config.has_reboot_timeout = true;
-        ms->boot_config.reboot_timeout = qemu_opt_get_number(opts, "reboot-timeout", -1);
-    }
-
-    s = qemu_opt_get(opts, "menu");
-    if (s) {
-        ms->boot_config.has_menu = true;
-        ms->boot_config.menu = qemu_opt_get_bool(opts, "menu", false);
-    }
-
-    ms->boot_config.strict = qemu_opt_get_bool(opts, "strict", false);
+out_free:
+    qapi_free_BootConfiguration(config);
 }
 
 static void machine_class_init(ObjectClass *oc, void *data)
@@ -884,6 +881,12 @@ static void machine_class_init(ObjectClass *oc, void *data)
     object_class_property_set_description(oc, "dumpdtb",
         "Dump current dtb to a file and quit");
 
+    object_class_property_add(oc, "boot", "BootConfiguration",
+        machine_get_boot, machine_set_boot,
+        NULL, NULL);
+    object_class_property_set_description(oc, "boot",
+        "Boot configuration");
+
     object_class_property_add(oc, "smp", "SMPConfiguration",
         machine_get_smp, machine_set_smp,
         NULL, NULL);
@@ -1017,12 +1020,15 @@ static void machine_initfn(Object *obj)
     ms->smp.clusters = 1;
     ms->smp.cores = 1;
     ms->smp.threads = 1;
+
+    machine_copy_boot_config(ms, &(BootConfiguration){ 0 });
 }
 
 static void machine_finalize(Object *obj)
 {
     MachineState *ms = MACHINE(obj);
 
+    machine_free_boot_config(ms);
     g_free(ms->kernel_filename);
     g_free(ms->initrd_filename);
     g_free(ms->kernel_cmdline);
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 6cda7e4308..910c3ffde2 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -26,7 +26,6 @@ OBJECT_DECLARE_TYPE(MachineState, MachineClass, MACHINE)
 extern MachineState *current_machine;
 
 void machine_run_board_init(MachineState *machine);
-void machine_boot_parse(MachineState *ms, QemuOpts *opts, Error **errp);
 bool machine_usb(MachineState *machine);
 int machine_phandle_start(MachineState *machine);
 bool machine_dump_guest_core(MachineState *machine);
diff --git a/softmmu/vl.c b/softmmu/vl.c
index dd90df3ed1..13ae31e92f 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -1884,16 +1884,11 @@ static bool object_create_early(const char *type)
 
 static void qemu_apply_machine_options(QDict *qdict)
 {
-    QemuOpts *opts;
-
     object_set_properties_from_keyval(OBJECT(current_machine), qdict, false, &error_fatal);
     current_machine->ram_size = ram_size;
     current_machine->maxram_size = maxram_size;
     current_machine->ram_slots = ram_slots;
 
-    opts = qemu_opts_find(qemu_find_opts("boot-opts"), NULL);
-    machine_boot_parse(current_machine, opts, &error_fatal);
-
     if (semihosting_enabled() && !semihosting_get_argc()) {
         /* fall back to the -kernel/-append */
         semihosting_arg_fallback(current_machine->kernel_filename, current_machine->kernel_cmdline);
@@ -2189,7 +2184,8 @@ static bool is_qemuopts_group(const char *group)
 {
     if (g_str_equal(group, "object") ||
         g_str_equal(group, "machine") ||
-        g_str_equal(group, "smp-opts")) {
+        g_str_equal(group, "smp-opts") ||
+        g_str_equal(group, "boot-opts")) {
         return false;
     }
     return true;
@@ -2211,6 +2207,8 @@ static void qemu_record_config_group(const char *group, QDict *dict,
         keyval_merge(machine_opts_dict, dict, errp);
     } else if (g_str_equal(group, "smp-opts")) {
         machine_merge_property("smp", dict, &error_fatal);
+    } else if (g_str_equal(group, "boot-opts")) {
+        machine_merge_property("boot", dict, &error_fatal);
     } else {
         abort();
     }
@@ -2956,11 +2954,7 @@ void qemu_init(int argc, char **argv, char **envp)
                 drive_add(IF_DEFAULT, 2, optarg, CDROM_OPTS);
                 break;
             case QEMU_OPTION_boot:
-                opts = qemu_opts_parse_noisily(qemu_find_opts("boot-opts"),
-                                               optarg, true);
-                if (!opts) {
-                    exit(1);
-                }
+                machine_parse_property_opt(qemu_find_opts("boot-opts"), "boot", optarg);
                 break;
             case QEMU_OPTION_fda:
             case QEMU_OPTION_fdb:

From ce9d03fb3f7a87f46a1a2fc3597f2f44541a0c9c Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Thu, 14 Apr 2022 12:52:58 -0400
Subject: [PATCH 248/935] machine: add mem compound property

Make -m syntactic sugar for a compound property "-machine
mem.{size,max-size,slots}".  The new property does not have
the magic conversion to megabytes of unsuffixed arguments,
and also does not understand that "0" means the default size
(you have to leave it out to get the default).  This means
that we need to convert the QemuOpts by hand to a QDict.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220414165300.555321-4-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/core/machine.c |  80 ++++++++++++++++++++++++++++++
 qapi/machine.json |  18 +++++++
 softmmu/vl.c      | 123 +++++++++++++++-------------------------------
 3 files changed, 138 insertions(+), 83 deletions(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index 8cea94537d..46b8d0effa 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -523,6 +523,78 @@ static void machine_set_hmat(Object *obj, bool value, Error **errp)
     ms->numa_state->hmat_enabled = value;
 }
 
+static void machine_get_mem(Object *obj, Visitor *v, const char *name,
+                            void *opaque, Error **errp)
+{
+    MachineState *ms = MACHINE(obj);
+    MemorySizeConfiguration mem = {
+        .has_size = true,
+        .size = ms->ram_size,
+        .has_max_size = !!ms->ram_slots,
+        .max_size = ms->maxram_size,
+        .has_slots = !!ms->ram_slots,
+        .slots = ms->ram_slots,
+    };
+    MemorySizeConfiguration *p_mem = &mem;
+
+    visit_type_MemorySizeConfiguration(v, name, &p_mem, &error_abort);
+}
+
+static void machine_set_mem(Object *obj, Visitor *v, const char *name,
+                            void *opaque, Error **errp)
+{
+    MachineState *ms = MACHINE(obj);
+    MachineClass *mc = MACHINE_GET_CLASS(obj);
+    MemorySizeConfiguration *mem;
+
+    ERRP_GUARD();
+
+    if (!visit_type_MemorySizeConfiguration(v, name, &mem, errp)) {
+        return;
+    }
+
+    if (!mem->has_size) {
+        mem->has_size = true;
+        mem->size = mc->default_ram_size;
+    }
+    mem->size = QEMU_ALIGN_UP(mem->size, 8192);
+    if (mc->fixup_ram_size) {
+        mem->size = mc->fixup_ram_size(mem->size);
+    }
+    if ((ram_addr_t)mem->size != mem->size) {
+        error_setg(errp, "ram size too large");
+        goto out_free;
+    }
+
+    if (mem->has_max_size) {
+        if (mem->max_size < mem->size) {
+            error_setg(errp, "invalid value of maxmem: "
+                       "maximum memory size (0x%" PRIx64 ") must be at least "
+                       "the initial memory size (0x%" PRIx64 ")",
+                       mem->max_size, mem->size);
+            goto out_free;
+        }
+        if (mem->has_slots && mem->slots && mem->max_size == mem->size) {
+            error_setg(errp, "invalid value of maxmem: "
+                       "memory slots were specified but maximum memory size "
+                       "(0x%" PRIx64 ") is equal to the initial memory size "
+                       "(0x%" PRIx64 ")", mem->max_size, mem->size);
+            goto out_free;
+        }
+        ms->maxram_size = mem->max_size;
+    } else {
+        if (mem->has_slots) {
+            error_setg(errp, "slots specified but no max-size");
+            goto out_free;
+        }
+        ms->maxram_size = mem->size;
+    }
+    ms->ram_size = mem->size;
+    ms->ram_slots = mem->has_slots ? mem->slots : 0;
+out_free:
+    qapi_free_MemorySizeConfiguration(mem);
+}
+
 static char *machine_get_nvdimm_persistence(Object *obj, Error **errp)
 {
     MachineState *ms = MACHINE(obj);
@@ -953,6 +1025,12 @@ static void machine_class_init(ObjectClass *oc, void *data)
     object_class_property_set_description(oc, "memory-backend",
                                           "Set RAM backend"
                                           "Valid value is ID of hostmem based backend");
+
+    object_class_property_add(oc, "memory", "MemorySizeConfiguration",
+        machine_get_mem, machine_set_mem,
+        NULL, NULL);
+    object_class_property_set_description(oc, "memory",
+        "Memory size configuration");
 }
 
 static void machine_class_base_init(ObjectClass *oc, void *data)
@@ -983,6 +1061,8 @@ static void machine_initfn(Object *obj)
     ms->mem_merge = true;
     ms->enable_graphics = true;
     ms->kernel_cmdline = g_strdup("");
+    ms->ram_size = mc->default_ram_size;
+    ms->maxram_size = mc->default_ram_size;
 
     if (mc->nvdimm_supported) {
         Object *obj = OBJECT(ms);
diff --git a/qapi/machine.json b/qapi/machine.json
index e3dcf5a119..92480d4044 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1614,3 +1614,21 @@
 ##
 { 'enum': 'SmbiosEntryPointType',
   'data': [ '32', '64' ] }
+
+##
+# @MemorySizeConfiguration:
+#
+# Schema for memory size configuration.
+#
+# @size: memory size in bytes
+#
+# @max-size: maximum hotpluggable memory size in bytes
+#
+# @slots: number of available memory slots for hotplug
+#
+# Since: 7.1
+##
+{ 'struct': 'MemorySizeConfiguration', 'data': {
+     '*size': 'size',
+     '*max-size': 'size',
+     '*slots': 'uint64' } }
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 13ae31e92f..65a665e0bc 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -159,11 +159,10 @@ static const char *mem_path;
 static const char *incoming;
 static const char *loadvm;
 static const char *accelerators;
+static bool have_custom_ram_size;
 static QDict *machine_opts_dict;
 static QTAILQ_HEAD(, ObjectOption) object_opts = QTAILQ_HEAD_INITIALIZER(object_opts);
 static QTAILQ_HEAD(, DeviceOption) device_opts = QTAILQ_HEAD_INITIALIZER(device_opts);
-static ram_addr_t maxram_size;
-static uint64_t ram_slots;
 static int display_remote;
 static int snapshot;
 static bool preconfig_requested;
@@ -171,7 +170,6 @@ static QemuPluginList plugin_list = QTAILQ_HEAD_INITIALIZER(plugin_list);
 static BlockdevOptionsQueue bdo_queue = QSIMPLEQ_HEAD_INITIALIZER(bdo_queue);
 static bool nographic = false;
 static int mem_prealloc; /* force preallocation of physical target memory */
-static ram_addr_t ram_size;
 static const char *vga_model = NULL;
 static DisplayOptions dpy;
 static int num_serial_hds;
@@ -1736,6 +1734,7 @@ static void keyval_dashify(QDict *qdict, Error **errp)
 static void qemu_apply_legacy_machine_options(QDict *qdict)
 {
     const char *value;
+    QObject *prop;
 
     keyval_dashify(qdict, &error_fatal);
 
@@ -1768,6 +1767,13 @@ static void qemu_apply_legacy_machine_options(QDict *qdict)
                                    false);
         qdict_del(qdict, "kernel-irqchip");
     }
+
+    prop = qdict_get(qdict, "memory");
+    if (prop) {
+        have_custom_ram_size =
+            qobject_type(prop) == QTYPE_QDICT &&
+            qdict_haskey(qobject_to(QDict, prop), "size");
+    }
 }
 
 static void object_option_foreach_add(bool (*type_opt_predicate)(const char *))
@@ -1885,9 +1891,6 @@ static bool object_create_early(const char *type)
 static void qemu_apply_machine_options(QDict *qdict)
 {
     object_set_properties_from_keyval(OBJECT(current_machine), qdict, false, &error_fatal);
-    current_machine->ram_size = ram_size;
-    current_machine->maxram_size = maxram_size;
-    current_machine->ram_slots = ram_slots;
 
     if (semihosting_enabled() && !semihosting_get_argc()) {
         /* fall back to the -kernel/-append */
@@ -1998,12 +2001,6 @@ static void qemu_create_late_backends(void)
     qemu_semihosting_console_init();
 }
 
-static bool have_custom_ram_size(void)
-{
-    QemuOpts *opts = qemu_find_opts_singleton("memory");
-    return !!qemu_opt_get_size(opts, "size", 0);
-}
-
 static void qemu_resolve_machine_memdev(void)
 {
     if (current_machine->ram_memdev_id) {
@@ -2018,7 +2015,7 @@ static void qemu_resolve_machine_memdev(void)
             exit(EXIT_FAILURE);
         }
         backend_size = object_property_get_uint(backend, "size",  &error_abort);
-        if (have_custom_ram_size() && backend_size != ram_size) {
+        if (have_custom_ram_size && backend_size != current_machine->ram_size) {
                 error_report("Size specified by -m option must match size of "
                              "explicitly specified 'memory-backend' property");
                 exit(EXIT_FAILURE);
@@ -2028,95 +2025,58 @@ static void qemu_resolve_machine_memdev(void)
                          "'-machine memory-backend'");
             exit(EXIT_FAILURE);
         }
-        ram_size = backend_size;
+        current_machine->ram_size = backend_size;
     }
 
     if (!xen_enabled()) {
         /* On 32-bit hosts, QEMU is limited by virtual address space */
-        if (ram_size > (2047 << 20) && HOST_LONG_BITS == 32) {
+        if (current_machine->ram_size > (2047 << 20) && HOST_LONG_BITS == 32) {
             error_report("at most 2047 MB RAM can be simulated");
             exit(1);
         }
     }
 }
 
-static void set_memory_options(MachineClass *mc)
+static void parse_memory_options(const char *arg)
 {
-    uint64_t sz;
+    QemuOpts *opts;
+    QDict *dict, *prop;
     const char *mem_str;
-    const ram_addr_t default_ram_size = mc->default_ram_size;
-    QemuOpts *opts = qemu_find_opts_singleton("memory");
-    Location loc;
 
-    loc_push_none(&loc);
-    qemu_opts_loc_restore(opts);
+    opts = qemu_opts_parse_noisily(qemu_find_opts("memory"), arg, true);
+    if (!opts) {
+        exit(EXIT_FAILURE);
+    }
 
-    sz = 0;
-    mem_str = qemu_opt_get(opts, "size");
-    if (mem_str) {
+    prop = qdict_new();
+
+    if (qemu_opt_get_size(opts, "size", 0) != 0) {
+        mem_str = qemu_opt_get(opts, "size");
         if (!*mem_str) {
             error_report("missing 'size' option value");
             exit(EXIT_FAILURE);
         }
 
-        sz = qemu_opt_get_size(opts, "size", ram_size);
-
         /* Fix up legacy suffix-less format */
         if (g_ascii_isdigit(mem_str[strlen(mem_str) - 1])) {
-            uint64_t overflow_check = sz;
-
-            sz *= MiB;
-            if (sz / MiB != overflow_check) {
-                error_report("too large 'size' option value");
-                exit(EXIT_FAILURE);
-            }
+            g_autofree char *mib_str = g_strdup_printf("%sM", mem_str);
+            qdict_put_str(prop, "size", mib_str);
+        } else {
+            qdict_put_str(prop, "size", mem_str);
         }
     }
 
-    /* backward compatibility behaviour for case "-m 0" */
-    if (sz == 0) {
-        sz = default_ram_size;
-    }
-
-    sz = QEMU_ALIGN_UP(sz, 8192);
-    if (mc->fixup_ram_size) {
-        sz = mc->fixup_ram_size(sz);
-    }
-    ram_size = sz;
-    if (ram_size != sz) {
-        error_report("ram size too large");
-        exit(EXIT_FAILURE);
-    }
-
-    maxram_size = ram_size;
-
     if (qemu_opt_get(opts, "maxmem")) {
-        uint64_t slots;
-
-        sz = qemu_opt_get_size(opts, "maxmem", 0);
-        slots = qemu_opt_get_number(opts, "slots", 0);
-        if (sz < ram_size) {
-            error_report("invalid value of -m option maxmem: "
-                         "maximum memory size (0x%" PRIx64 ") must be at least "
-                         "the initial memory size (0x" RAM_ADDR_FMT ")",
-                         sz, ram_size);
-            exit(EXIT_FAILURE);
-        } else if (slots && sz == ram_size) {
-            error_report("invalid value of -m option maxmem: "
-                         "memory slots were specified but maximum memory size "
-                         "(0x%" PRIx64 ") is equal to the initial memory size "
-                         "(0x" RAM_ADDR_FMT ")", sz, ram_size);
-            exit(EXIT_FAILURE);
-        }
-
-        maxram_size = sz;
-        ram_slots = slots;
-    } else if (qemu_opt_get(opts, "slots")) {
-        error_report("invalid -m option value: missing 'maxmem' option");
-        exit(EXIT_FAILURE);
+        qdict_put_str(prop, "max-size", qemu_opt_get(opts, "maxmem"));
+    }
+    if (qemu_opt_get(opts, "slots")) {
+        qdict_put_str(prop, "slots", qemu_opt_get(opts, "slots"));
     }
 
-    loc_pop(&loc);
+    dict = qdict_new();
+    qdict_put(dict, "memory", prop);
+    keyval_merge(machine_opts_dict, dict, &error_fatal);
+    qobject_unref(dict);
 }
 
 static void qemu_create_machine(QDict *qdict)
@@ -2124,8 +2084,6 @@ static void qemu_create_machine(QDict *qdict)
     MachineClass *machine_class = select_machine(qdict, &error_fatal);
     object_set_machine_compat_props(machine_class->compat_props);
 
-    set_memory_options(machine_class);
-
     current_machine = MACHINE(object_new_with_class(OBJECT_CLASS(machine_class)));
     object_property_add_child(object_get_root(), "machine",
                               OBJECT(current_machine));
@@ -2185,7 +2143,8 @@ static bool is_qemuopts_group(const char *group)
     if (g_str_equal(group, "object") ||
         g_str_equal(group, "machine") ||
         g_str_equal(group, "smp-opts") ||
-        g_str_equal(group, "boot-opts")) {
+        g_str_equal(group, "boot-opts") ||
+        g_str_equal(group, "memory")) {
         return false;
     }
     return true;
@@ -2209,6 +2168,8 @@ static void qemu_record_config_group(const char *group, QDict *dict,
         machine_merge_property("smp", dict, &error_fatal);
     } else if (g_str_equal(group, "boot-opts")) {
         machine_merge_property("boot", dict, &error_fatal);
+    } else if (g_str_equal(group, "memory")) {
+        machine_merge_property("memory", dict, &error_fatal);
     } else {
         abort();
     }
@@ -3009,11 +2970,7 @@ void qemu_init(int argc, char **argv, char **envp)
                 exit(0);
                 break;
             case QEMU_OPTION_m:
-                opts = qemu_opts_parse_noisily(qemu_find_opts("memory"),
-                                               optarg, true);
-                if (!opts) {
-                    exit(EXIT_FAILURE);
-                }
+                parse_memory_options(optarg);
                 break;
 #ifdef CONFIG_TPM
             case QEMU_OPTION_tpmdev:

From 26f88d84dab62e6eb3ec72737ccb155d06049e3a Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Thu, 14 Apr 2022 12:52:59 -0400
Subject: [PATCH 249/935] machine: make memory-backend a link property

Handle HostMemoryBackend creation and setting of ms->ram entirely in
machine_run_board_init.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220414165300.555321-5-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/core/machine.c   | 70 ++++++++++++++++++++++++++++++---------------
 hw/core/numa.c      |  2 +-
 hw/sparc/sun4m.c    |  5 ++--
 include/hw/boards.h |  4 +--
 softmmu/vl.c        | 62 ++++++++++++++-------------------------
 5 files changed, 74 insertions(+), 69 deletions(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index 46b8d0effa..8aab5416dd 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -36,6 +36,7 @@
 #include "exec/confidential-guest-support.h"
 #include "hw/virtio/virtio.h"
 #include "hw/virtio/virtio-pci.h"
+#include "qom/object_interfaces.h"
 
 GlobalProperty hw_compat_7_0[] = {};
 const size_t hw_compat_7_0_len = G_N_ELEMENTS(hw_compat_7_0);
@@ -653,21 +654,6 @@ bool device_type_is_dynamic_sysbus(MachineClass *mc, const char *type)
     return allowed;
 }
 
-static char *machine_get_memdev(Object *obj, Error **errp)
-{
-    MachineState *ms = MACHINE(obj);
-
-    return g_strdup(ms->ram_memdev_id);
-}
-
-static void machine_set_memdev(Object *obj, const char *value, Error **errp)
-{
-    MachineState *ms = MACHINE(obj);
-
-    g_free(ms->ram_memdev_id);
-    ms->ram_memdev_id = g_strdup(value);
-}
-
 HotpluggableCPUList *machine_query_hotpluggable_cpus(MachineState *machine)
 {
     int i;
@@ -1020,8 +1006,9 @@ static void machine_class_init(ObjectClass *oc, void *data)
     object_class_property_set_description(oc, "memory-encryption",
         "Set memory encryption object to use");
 
-    object_class_property_add_str(oc, "memory-backend",
-                                  machine_get_memdev, machine_set_memdev);
+    object_class_property_add_link(oc, "memory-backend", TYPE_MEMORY_BACKEND,
+                                   offsetof(MachineState, memdev), object_property_allow_set_link,
+                                   OBJ_PROP_LINK_STRONG);
     object_class_property_set_description(oc, "memory-backend",
                                           "Set RAM backend"
                                           "Valid value is ID of hostmem based backend");
@@ -1270,7 +1257,40 @@ MemoryRegion *machine_consume_memdev(MachineState *machine,
     return ret;
 }
 
-void machine_run_board_init(MachineState *machine)
+static bool create_default_memdev(MachineState *ms, const char *path, Error **errp)
+{
+    Object *obj;
+    MachineClass *mc = MACHINE_GET_CLASS(ms);
+    bool r = false;
+
+    obj = object_new(path ? TYPE_MEMORY_BACKEND_FILE : TYPE_MEMORY_BACKEND_RAM);
+    if (path) {
+        if (!object_property_set_str(obj, "mem-path", path, errp)) {
+            goto out;
+        }
+    }
+    if (!object_property_set_int(obj, "size", ms->ram_size, errp)) {
+        goto out;
+    }
+    object_property_add_child(object_get_objects_root(), mc->default_ram_id,
+                              obj);
+    /* Ensure backend's memory region name is equal to mc->default_ram_id */
+    if (!object_property_set_bool(obj, "x-use-canonical-path-for-ramblock-id",
+                             false, errp)) {
+        goto out;
+    }
+    if (!user_creatable_complete(USER_CREATABLE(obj), errp)) {
+        goto out;
+    }
+    r = object_property_set_link(OBJECT(ms), "memory-backend", obj, errp);
+
+out:
+    object_unref(obj);
+    return r;
+}
+
+
+void machine_run_board_init(MachineState *machine, const char *mem_path, Error **errp)
 {
     MachineClass *machine_class = MACHINE_GET_CLASS(machine);
     ObjectClass *oc = object_class_by_name(machine->cpu_type);
@@ -1281,11 +1301,11 @@ void machine_run_board_init(MachineState *machine)
        clock values from the log. */
     replay_checkpoint(CHECKPOINT_INIT);
 
-    if (machine->ram_memdev_id) {
-        Object *o;
-        o = object_resolve_path_type(machine->ram_memdev_id,
-                                     TYPE_MEMORY_BACKEND, NULL);
-        machine->ram = machine_consume_memdev(machine, MEMORY_BACKEND(o));
+    if (machine_class->default_ram_id && machine->ram_size &&
+        numa_uses_legacy_mem() && !machine->memdev) {
+        if (!create_default_memdev(current_machine, mem_path, errp)) {
+            return;
+        }
     }
 
     if (machine->numa_state) {
@@ -1295,6 +1315,10 @@ void machine_run_board_init(MachineState *machine)
         }
     }
 
+    if (!machine->ram && machine->memdev) {
+        machine->ram = machine_consume_memdev(machine, machine->memdev);
+    }
+
     /* If the machine supports the valid_cpu_types check and the user
      * specified a CPU with -cpu check here that the user CPU is supported.
      */
diff --git a/hw/core/numa.c b/hw/core/numa.c
index 1aa05dcf42..26d8e5f616 100644
--- a/hw/core/numa.c
+++ b/hw/core/numa.c
@@ -695,7 +695,7 @@ void numa_complete_configuration(MachineState *ms)
         }
 
         if (!numa_uses_legacy_mem() && mc->default_ram_id) {
-            if (ms->ram_memdev_id) {
+            if (ms->memdev) {
                 error_report("'-machine memory-backend' and '-numa memdev'"
                              " properties are mutually exclusive");
                 exit(1);
diff --git a/hw/sparc/sun4m.c b/hw/sparc/sun4m.c
index 9d57491f68..d9288326d6 100644
--- a/hw/sparc/sun4m.c
+++ b/hw/sparc/sun4m.c
@@ -831,8 +831,7 @@ static void sun4m_hw_init(MachineState *machine)
     SysBusDevice *s;
     unsigned int smp_cpus = machine->smp.cpus;
     unsigned int max_cpus = machine->smp.max_cpus;
-    Object *ram_memdev = object_resolve_path_type(machine->ram_memdev_id,
-                                                  TYPE_MEMORY_BACKEND, NULL);
+    HostMemoryBackend *ram_memdev = machine->memdev;
     NICInfo *nd = &nd_table[0];
 
     if (machine->ram_size > hwdef->max_mem) {
@@ -852,7 +851,7 @@ static void sun4m_hw_init(MachineState *machine)
 
     /* Create and map RAM frontend */
     dev = qdev_new("memory");
-    object_property_set_link(OBJECT(dev), "memdev", ram_memdev, &error_fatal);
+    object_property_set_link(OBJECT(dev), "memdev", OBJECT(ram_memdev), &error_fatal);
     sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
     sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, 0);
 
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 910c3ffde2..7b416c9787 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -25,7 +25,7 @@ OBJECT_DECLARE_TYPE(MachineState, MachineClass, MACHINE)
 
 extern MachineState *current_machine;
 
-void machine_run_board_init(MachineState *machine);
+void machine_run_board_init(MachineState *machine, const char *mem_path, Error **errp);
 bool machine_usb(MachineState *machine);
 int machine_phandle_start(MachineState *machine);
 bool machine_dump_guest_core(MachineState *machine);
@@ -339,7 +339,7 @@ struct MachineState {
     bool suppress_vmdesc;
     bool enable_graphics;
     ConfidentialGuestSupport *cgs;
-    char *ram_memdev_id;
+    HostMemoryBackend *memdev;
     /*
      * convenience alias to ram_memdev_id backend memory region
      * or to numa container memory region
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 65a665e0bc..f6deec9380 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -160,6 +160,7 @@ static const char *incoming;
 static const char *loadvm;
 static const char *accelerators;
 static bool have_custom_ram_size;
+static const char *ram_memdev_id;
 static QDict *machine_opts_dict;
 static QTAILQ_HEAD(, ObjectOption) object_opts = QTAILQ_HEAD_INITIALIZER(object_opts);
 static QTAILQ_HEAD(, DeviceOption) device_opts = QTAILQ_HEAD_INITIALIZER(device_opts);
@@ -1768,6 +1769,19 @@ static void qemu_apply_legacy_machine_options(QDict *qdict)
         qdict_del(qdict, "kernel-irqchip");
     }
 
+    value = qdict_get_try_str(qdict, "memory-backend");
+    if (value) {
+        if (mem_path) {
+            error_report("'-mem-path' can't be used together with"
+                         "'-machine memory-backend'");
+            exit(EXIT_FAILURE);
+        }
+
+        /* Resolved later.  */
+        ram_memdev_id = g_strdup(value);
+        qdict_del(qdict, "memory-backend");
+    }
+
     prop = qdict_get(qdict, "memory");
     if (prop) {
         have_custom_ram_size =
@@ -2003,29 +2017,25 @@ static void qemu_create_late_backends(void)
 
 static void qemu_resolve_machine_memdev(void)
 {
-    if (current_machine->ram_memdev_id) {
+    if (ram_memdev_id) {
         Object *backend;
         ram_addr_t backend_size;
 
-        backend = object_resolve_path_type(current_machine->ram_memdev_id,
+        backend = object_resolve_path_type(ram_memdev_id,
                                            TYPE_MEMORY_BACKEND, NULL);
         if (!backend) {
-            error_report("Memory backend '%s' not found",
-                         current_machine->ram_memdev_id);
+            error_report("Memory backend '%s' not found", ram_memdev_id);
             exit(EXIT_FAILURE);
         }
         backend_size = object_property_get_uint(backend, "size",  &error_abort);
         if (have_custom_ram_size && backend_size != current_machine->ram_size) {
-                error_report("Size specified by -m option must match size of "
-                             "explicitly specified 'memory-backend' property");
-                exit(EXIT_FAILURE);
-        }
-        if (mem_path) {
-            error_report("'-mem-path' can't be used together with"
-                         "'-machine memory-backend'");
+            error_report("Size specified by -m option must match size of "
+                         "explicitly specified 'memory-backend' property");
             exit(EXIT_FAILURE);
         }
         current_machine->ram_size = backend_size;
+        object_property_set_link(OBJECT(current_machine),
+                                 "memory-backend", backend, &error_fatal);
     }
 
     if (!xen_enabled()) {
@@ -2376,27 +2386,6 @@ static void configure_accelerators(const char *progname)
     }
 }
 
-static void create_default_memdev(MachineState *ms, const char *path)
-{
-    Object *obj;
-    MachineClass *mc = MACHINE_GET_CLASS(ms);
-
-    obj = object_new(path ? TYPE_MEMORY_BACKEND_FILE : TYPE_MEMORY_BACKEND_RAM);
-    if (path) {
-        object_property_set_str(obj, "mem-path", path, &error_fatal);
-    }
-    object_property_set_int(obj, "size", ms->ram_size, &error_fatal);
-    object_property_add_child(object_get_objects_root(), mc->default_ram_id,
-                              obj);
-    /* Ensure backend's memory region name is equal to mc->default_ram_id */
-    object_property_set_bool(obj, "x-use-canonical-path-for-ramblock-id",
-                             false, &error_fatal);
-    user_creatable_complete(USER_CREATABLE(obj), &error_fatal);
-    object_unref(obj);
-    object_property_set_str(OBJECT(ms), "memory-backend", mc->default_ram_id,
-                            &error_fatal);
-}
-
 static void qemu_validate_options(const QDict *machine_opts)
 {
     const char *kernel_filename = qdict_get_try_str(machine_opts, "kernel");
@@ -2581,18 +2570,11 @@ static void qemu_init_displays(void)
 
 static void qemu_init_board(void)
 {
-    MachineClass *machine_class = MACHINE_GET_CLASS(current_machine);
-
-    if (machine_class->default_ram_id && current_machine->ram_size &&
-        numa_uses_legacy_mem() && !current_machine->ram_memdev_id) {
-        create_default_memdev(current_machine, mem_path);
-    }
-
     /* process plugin before CPUs are created, but once -smp has been parsed */
     qemu_plugin_load_list(&plugin_list, &error_fatal);
 
     /* From here on we enter MACHINE_PHASE_INITIALIZED.  */
-    machine_run_board_init(current_machine);
+    machine_run_board_init(current_machine, mem_path, &error_fatal);
 
     drive_check_orphaned();
 

From fb56b7a052e1443409334c579e617e287a7250d3 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Thu, 14 Apr 2022 12:53:00 -0400
Subject: [PATCH 250/935] machine: move more memory validation to Machine
 object

This allows setting memory properties without going through
vl.c, and have them validated just the same.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220414165300.555321-6-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/core/machine.c | 21 +++++++++++++++++++--
 softmmu/vl.c      | 17 +++--------------
 2 files changed, 22 insertions(+), 16 deletions(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index 8aab5416dd..3264c1e11d 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -21,12 +21,14 @@
 #include "qapi/qapi-visit-common.h"
 #include "qapi/qapi-visit-machine.h"
 #include "qapi/visitor.h"
+#include "qom/object_interfaces.h"
 #include "hw/sysbus.h"
 #include "sysemu/cpus.h"
 #include "sysemu/sysemu.h"
 #include "sysemu/reset.h"
 #include "sysemu/runstate.h"
 #include "sysemu/numa.h"
+#include "sysemu/xen.h"
 #include "qemu/error-report.h"
 #include "sysemu/qtest.h"
 #include "hw/pci/pci.h"
@@ -1301,8 +1303,23 @@ void machine_run_board_init(MachineState *machine, const char *mem_path, Error *
        clock values from the log. */
     replay_checkpoint(CHECKPOINT_INIT);
 
-    if (machine_class->default_ram_id && machine->ram_size &&
-        numa_uses_legacy_mem() && !machine->memdev) {
+    if (!xen_enabled()) {
+        /* On 32-bit hosts, QEMU is limited by virtual address space */
+        if (machine->ram_size > (2047 << 20) && HOST_LONG_BITS == 32) {
+            error_setg(errp, "at most 2047 MB RAM can be simulated");
+            return;
+        }
+    }
+
+    if (machine->memdev) {
+        ram_addr_t backend_size = object_property_get_uint(OBJECT(machine->memdev),
+                                                           "size",  &error_abort);
+        if (backend_size != machine->ram_size) {
+            error_setg(errp, "Machine memory size does not match the size of the memory backend");
+            return;
+        }
+    } else if (machine_class->default_ram_id && machine->ram_size &&
+               numa_uses_legacy_mem()) {
         if (!create_default_memdev(current_machine, mem_path, errp)) {
             return;
         }
diff --git a/softmmu/vl.c b/softmmu/vl.c
index f6deec9380..edba74f075 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -2027,24 +2027,13 @@ static void qemu_resolve_machine_memdev(void)
             error_report("Memory backend '%s' not found", ram_memdev_id);
             exit(EXIT_FAILURE);
         }
-        backend_size = object_property_get_uint(backend, "size",  &error_abort);
-        if (have_custom_ram_size && backend_size != current_machine->ram_size) {
-            error_report("Size specified by -m option must match size of "
-                         "explicitly specified 'memory-backend' property");
-            exit(EXIT_FAILURE);
+        if (!have_custom_ram_size) {
+            backend_size = object_property_get_uint(backend, "size",  &error_abort);
+            current_machine->ram_size = backend_size;
         }
-        current_machine->ram_size = backend_size;
         object_property_set_link(OBJECT(current_machine),
                                  "memory-backend", backend, &error_fatal);
     }
-
-    if (!xen_enabled()) {
-        /* On 32-bit hosts, QEMU is limited by virtual address space */
-        if (current_machine->ram_size > (2047 << 20) && HOST_LONG_BITS == 32) {
-            error_report("at most 2047 MB RAM can be simulated");
-            exit(1);
-        }
-    }
 }
 
 static void parse_memory_options(const char *arg)

From 0c1450e2045e1a046854c34f34ed9f03dcdab0fb Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 4 May 2022 22:15:14 +0200
Subject: [PATCH 251/935] slirp: bump submodule past 4.7 release
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Version 4.7 of slirp provides a new timer API that works better with CFI,
together with several other improvements:

* Allow disabling the internal DHCP server !22
* Support Unix sockets in hostfwd !103
* IPv6 DNS proxying support !110
* bootp: add support for UEFI HTTP boot !111

and bugfixes.

The submodule update also includes 2 commits to fix warnings in the
Win32 build.

Reviewed-by: Marc-André Lureau <malureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 slirp | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/slirp b/slirp
index a88d9ace23..9d59bb775d 160000
--- a/slirp
+++ b/slirp
@@ -1 +1 @@
-Subproject commit a88d9ace234a24ce1c17189642ef9104799425e0
+Subproject commit 9d59bb775d6294c8b447a88512f7bb43f12a25a8

From ad2e5b87d7d070cb4c0a3ba1c20a4ec5e6429301 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Mon, 11 Apr 2022 09:26:06 +0200
Subject: [PATCH 252/935] net: slirp: introduce a wrapper struct for QemuTimer
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This struct will be extended in the next few patches to support the
new slirp_handle_timer() call.  For that we need to store an additional
"int" for each SLIRP timer, in addition to the cb_opaque.

Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Reviewed-by: Marc-André Lureau <malureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 net/slirp.c | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/net/slirp.c b/net/slirp.c
index bc5e9e4f77..f1e25d741f 100644
--- a/net/slirp.c
+++ b/net/slirp.c
@@ -184,23 +184,32 @@ static int64_t net_slirp_clock_get_ns(void *opaque)
     return qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
 }
 
+struct SlirpTimer {
+    QEMUTimer timer;
+}
+
 static void *net_slirp_timer_new(SlirpTimerCb cb,
                                  void *cb_opaque, void *opaque)
 {
-    return timer_new_full(NULL, QEMU_CLOCK_VIRTUAL,
-                          SCALE_MS, QEMU_TIMER_ATTR_EXTERNAL,
-                          cb, cb_opaque);
+    SlirpTimer *t = g_new(SlirpTimer, 1);
+    timer_init_full(&t->timer, NULL, QEMU_CLOCK_VIRTUAL,
+                    SCALE_MS, QEMU_TIMER_ATTR_EXTERNAL,
+                    cb, cb_opaque);
+    return t;
 }
 
 static void net_slirp_timer_free(void *timer, void *opaque)
 {
-    timer_free(timer);
+    SlirpTimer *t = timer;
+    timer_del(&t->timer);
+    g_free(t);
 }
 
 static void net_slirp_timer_mod(void *timer, int64_t expire_timer,
                                 void *opaque)
 {
-    timer_mod(timer, expire_timer);
+    SlirpTimer *t = timer;
+    timer_mod(&t->timer, expire_timer);
 }
 
 static void net_slirp_register_poll_fd(int fd, void *opaque)

From bce63ded2066a21cc1f5922d0bf667123123d6f6 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Mon, 11 Apr 2022 10:16:36 +0200
Subject: [PATCH 253/935] net: slirp: switch to slirp_new

Replace slirp_init with slirp_new, so that a more recent cfg.version
can be specified.  The function appeared in version 4.1.0.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 meson.build |  2 +-
 net/slirp.c | 27 +++++++++++++++++++++------
 2 files changed, 22 insertions(+), 7 deletions(-)

diff --git a/meson.build b/meson.build
index 03f63e95e2..ea72ba7c2f 100644
--- a/meson.build
+++ b/meson.build
@@ -2638,7 +2638,7 @@ if have_system
   if slirp_opt in ['enabled', 'auto', 'system']
     have_internal = fs.exists(meson.current_source_dir() / 'slirp/meson.build')
     slirp = dependency('slirp', kwargs: static_kwargs,
-                       method: 'pkg-config',
+                       method: 'pkg-config', version: '>=4.1.0',
                        required: slirp_opt == 'system' or
                                  slirp_opt == 'enabled' and not have_internal)
     if slirp.found()
diff --git a/net/slirp.c b/net/slirp.c
index f1e25d741f..b7464be86b 100644
--- a/net/slirp.c
+++ b/net/slirp.c
@@ -389,6 +389,7 @@ static int net_slirp_init(NetClientState *peer, const char *model,
 #if defined(CONFIG_SMBD_COMMAND)
     struct in_addr smbsrv = { .s_addr = 0 };
 #endif
+    SlirpConfig cfg = { 0 };
     NetClientState *nc;
     SlirpState *s;
     char buf[20];
@@ -577,12 +578,26 @@ static int net_slirp_init(NetClientState *peer, const char *model,
 
     s = DO_UPCAST(SlirpState, nc, nc);
 
-    s->slirp = slirp_init(restricted, ipv4, net, mask, host,
-                          ipv6, ip6_prefix, vprefix6_len, ip6_host,
-                          vhostname, tftp_server_name,
-                          tftp_export, bootfile, dhcp,
-                          dns, ip6_dns, dnssearch, vdomainname,
-                          &slirp_cb, s);
+    cfg.version = 1;
+    cfg.restricted = restricted;
+    cfg.in_enabled = ipv4;
+    cfg.vnetwork = net;
+    cfg.vnetmask = mask;
+    cfg.vhost = host;
+    cfg.in6_enabled = ipv6;
+    cfg.vprefix_addr6 = ip6_prefix;
+    cfg.vprefix_len = vprefix6_len;
+    cfg.vhost6 = ip6_host;
+    cfg.vhostname = vhostname;
+    cfg.tftp_server_name = tftp_server_name;
+    cfg.tftp_path = tftp_export;
+    cfg.bootfile = bootfile;
+    cfg.vdhcp_start = dhcp;
+    cfg.vnameserver = dns;
+    cfg.vnameserver6 = ip6_dns;
+    cfg.vdnssearch = dnssearch;
+    cfg.vdomainname = vdomainname;
+    s->slirp = slirp_new(&cfg, &slirp_cb, s);
     QTAILQ_INSERT_TAIL(&slirp_stacks, s, entry);
 
     /*

From 6222e55d134162e87062326a773eb7c9f9d13834 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Mon, 11 Apr 2022 09:39:16 +0200
Subject: [PATCH 254/935] net: slirp: add support for CFI-friendly timer API
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

libslirp 4.7 introduces a CFI-friendly version of the .timer_new callback.
The new callback replaces the function pointer with an enum; invoking the
callback is done with a new function slirp_handle_timer.

Support the new API so that CFI can be made compatible with using a system
libslirp.

Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Reviewed-by: Marc-André Lureau <malureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 net/slirp.c | 41 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 40 insertions(+), 1 deletion(-)

diff --git a/net/slirp.c b/net/slirp.c
index b7464be86b..8679be6444 100644
--- a/net/slirp.c
+++ b/net/slirp.c
@@ -184,10 +184,43 @@ static int64_t net_slirp_clock_get_ns(void *opaque)
     return qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
 }
 
+typedef struct SlirpTimer SlirpTimer;
 struct SlirpTimer {
     QEMUTimer timer;
+#if SLIRP_CHECK_VERSION(4,7,0)
+    Slirp *slirp;
+    SlirpTimerId id;
+    void *cb_opaque;
+#endif
+};
+
+#if SLIRP_CHECK_VERSION(4,7,0)
+static void net_slirp_init_completed(Slirp *slirp, void *opaque)
+{
+    SlirpState *s = opaque;
+    s->slirp = slirp;
 }
 
+static void net_slirp_timer_cb(void *opaque)
+{
+    SlirpTimer *t = opaque;
+    slirp_handle_timer(t->slirp, t->id, t->cb_opaque);
+}
+
+static void *net_slirp_timer_new_opaque(SlirpTimerId id,
+                                        void *cb_opaque, void *opaque)
+{
+    SlirpState *s = opaque;
+    SlirpTimer *t = g_new(SlirpTimer, 1);
+    t->slirp = s->slirp;
+    t->id = id;
+    t->cb_opaque = cb_opaque;
+    timer_init_full(&t->timer, NULL, QEMU_CLOCK_VIRTUAL,
+                    SCALE_MS, QEMU_TIMER_ATTR_EXTERNAL,
+                    net_slirp_timer_cb, t);
+    return t;
+}
+#else
 static void *net_slirp_timer_new(SlirpTimerCb cb,
                                  void *cb_opaque, void *opaque)
 {
@@ -197,6 +230,7 @@ static void *net_slirp_timer_new(SlirpTimerCb cb,
                     cb, cb_opaque);
     return t;
 }
+#endif
 
 static void net_slirp_timer_free(void *timer, void *opaque)
 {
@@ -231,7 +265,12 @@ static const SlirpCb slirp_cb = {
     .send_packet = net_slirp_send_packet,
     .guest_error = net_slirp_guest_error,
     .clock_get_ns = net_slirp_clock_get_ns,
+#if SLIRP_CHECK_VERSION(4,7,0)
+    .init_completed = net_slirp_init_completed,
+    .timer_new_opaque = net_slirp_timer_new_opaque,
+#else
     .timer_new = net_slirp_timer_new,
+#endif
     .timer_free = net_slirp_timer_free,
     .timer_mod = net_slirp_timer_mod,
     .register_poll_fd = net_slirp_register_poll_fd,
@@ -578,7 +617,7 @@ static int net_slirp_init(NetClientState *peer, const char *model,
 
     s = DO_UPCAST(SlirpState, nc, nc);
 
-    cfg.version = 1;
+    cfg.version = SLIRP_CHECK_VERSION(4,7,0) ? 4 : 1;
     cfg.restricted = restricted;
     cfg.in_enabled = ipv4;
     cfg.vnetwork = net;

From bf2f69d08bf158bfdbbe251357452f9576489b6d Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Mon, 11 Apr 2022 09:41:27 +0200
Subject: [PATCH 255/935] net: slirp: allow CFI with libslirp >= 4.7
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

slirp 4.7 introduces a new CFI-friendly timer callback that does
not pass function pointers within libslirp as callbacks for timers.
Check the version number and, if it is new enough, allow using CFI
even with a system libslirp.

Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Reviewed-by: Marc-André Lureau <malureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 meson.build | 31 +++++++++++++++++--------------
 1 file changed, 17 insertions(+), 14 deletions(-)

diff --git a/meson.build b/meson.build
index ea72ba7c2f..9b20dcd143 100644
--- a/meson.build
+++ b/meson.build
@@ -2637,10 +2637,25 @@ if have_system
   slirp_opt = get_option('slirp')
   if slirp_opt in ['enabled', 'auto', 'system']
     have_internal = fs.exists(meson.current_source_dir() / 'slirp/meson.build')
+    slirp_dep_required = (slirp_opt == 'system' or
+                          slirp_opt == 'enabled' and not have_internal)
     slirp = dependency('slirp', kwargs: static_kwargs,
                        method: 'pkg-config', version: '>=4.1.0',
-                       required: slirp_opt == 'system' or
-                                 slirp_opt == 'enabled' and not have_internal)
+                       required: slirp_dep_required)
+    # slirp <4.7 is incompatible with CFI support in QEMU.  This is because
+    # it passes function pointers within libslirp as callbacks for timers.
+    # When using a system-wide shared libslirp, the type information for the
+    # callback is missing and the timer call produces a false positive with CFI.
+    # Do not use the "version" keyword argument to produce a better error.
+    # with control-flow integrity.
+    if get_option('cfi') and slirp.found() and slirp.version().version_compare('<4.7')
+      if slirp_dep_required
+        error('Control-Flow Integrity requires libslirp 4.7.')
+      else
+        warning('Control-Flow Integrity requires libslirp 4.7, not using system-wide libslirp.')
+        slirp = not_found
+      endif
+    endif
     if slirp.found()
       slirp_opt = 'system'
     elif have_internal
@@ -2713,18 +2728,6 @@ if have_system
   endif
 endif
 
-# For CFI, we need to compile slirp as a static library together with qemu.
-# This is because we register slirp functions as callbacks for QEMU Timers.
-# When using a system-wide shared libslirp, the type information for the
-# callback is missing and the timer call produces a false positive with CFI.
-#
-# Now that slirp_opt has been defined, check if the selected slirp is compatible
-# with control-flow integrity.
-if get_option('cfi') and slirp_opt == 'system'
-  error('Control-Flow Integrity is not compatible with system-wide slirp.' \
-         + ' Please configure with --enable-slirp=git')
-endif
-
 fdt = not_found
 if have_system
   fdt_opt = get_option('fdt')

From 248af9e80a04c5ab4ffec789aa24345d3d86b42b Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 27 Apr 2022 15:08:28 +0200
Subject: [PATCH 256/935] coroutine-lock: qemu_co_queue_next is a
 coroutine-only qemu_co_enter_next

qemu_co_queue_next is basically the same as qemu_co_enter_next but
without a QemuLockable argument.  That's perfectly fine, but only
as long as the function is marked coroutine_fn.  If used outside
coroutine context, qemu_co_queue_wait will attempt to take the lock
and that is just broken: if you are calling qemu_co_queue_next outside
coroutine context, the lock is going to be a QemuMutex which cannot be
taken twice by the same thread.

The patch adds the marker and reimplements qemu_co_queue_next in terms of
qemu_co_enter_next_impl, to remove duplicated code and to clarify that the
latter also works in coroutine context.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20220427130830.150180-2-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 include/qemu/coroutine.h   |  7 ++++---
 util/qemu-coroutine-lock.c | 21 +++++++--------------
 2 files changed, 11 insertions(+), 17 deletions(-)

diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h
index 284571badb..c23d41e1ff 100644
--- a/include/qemu/coroutine.h
+++ b/include/qemu/coroutine.h
@@ -208,11 +208,12 @@ void qemu_co_queue_init(CoQueue *queue);
 void coroutine_fn qemu_co_queue_wait_impl(CoQueue *queue, QemuLockable *lock);
 
 /**
- * Removes the next coroutine from the CoQueue, and wake it up.
+ * Removes the next coroutine from the CoQueue, and queue it to run after
+ * the currently-running coroutine yields.
  * Returns true if a coroutine was removed, false if the queue is empty.
- * OK to run from coroutine and non-coroutine context.
+ * Used from coroutine context, use qemu_co_enter_next outside.
  */
-bool qemu_co_queue_next(CoQueue *queue);
+bool coroutine_fn qemu_co_queue_next(CoQueue *queue);
 
 /**
  * Empties the CoQueue; all coroutines are woken up.
diff --git a/util/qemu-coroutine-lock.c b/util/qemu-coroutine-lock.c
index 2669403839..5705cfea2e 100644
--- a/util/qemu-coroutine-lock.c
+++ b/util/qemu-coroutine-lock.c
@@ -67,7 +67,7 @@ void coroutine_fn qemu_co_queue_wait_impl(CoQueue *queue, QemuLockable *lock)
     }
 }
 
-static bool qemu_co_queue_do_restart(CoQueue *queue, bool single)
+void qemu_co_queue_restart_all(CoQueue *queue)
 {
     Coroutine *next;
 
@@ -78,23 +78,10 @@ static bool qemu_co_queue_do_restart(CoQueue *queue, bool single)
     while ((next = QSIMPLEQ_FIRST(&queue->entries)) != NULL) {
         QSIMPLEQ_REMOVE_HEAD(&queue->entries, co_queue_next);
         aio_co_wake(next);
-        if (single) {
-            break;
-        }
     }
     return true;
 }
 
-bool qemu_co_queue_next(CoQueue *queue)
-{
-    return qemu_co_queue_do_restart(queue, true);
-}
-
-void qemu_co_queue_restart_all(CoQueue *queue)
-{
-    qemu_co_queue_do_restart(queue, false);
-}
-
 bool qemu_co_enter_next_impl(CoQueue *queue, QemuLockable *lock)
 {
     Coroutine *next;
@@ -115,6 +102,12 @@ bool qemu_co_enter_next_impl(CoQueue *queue, QemuLockable *lock)
     return true;
 }
 
+bool coroutine_fn qemu_co_queue_next(CoQueue *queue)
+{
+    /* No unlock/lock needed in coroutine context.  */
+    return qemu_co_enter_next_impl(queue, NULL);
+}
+
 bool qemu_co_queue_empty(CoQueue *queue)
 {
     return QSIMPLEQ_FIRST(&queue->entries) == NULL;

From d6ee15adec5a2345e88c680cd15ed48796c89c14 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 27 Apr 2022 15:08:29 +0200
Subject: [PATCH 257/935] coroutine-lock: introduce qemu_co_queue_enter_all

Because qemu_co_queue_restart_all does not release the lock, it should
be used only in coroutine context.  Introduce a new function that,
like qemu_co_enter_next, does release the lock, and use it whenever
qemu_co_queue_restart_all was used outside coroutine context.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20220427130830.150180-3-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 include/qemu/coroutine.h   | 13 +++++++++++++
 ui/console.c               |  2 +-
 util/qemu-coroutine-lock.c |  7 +++++++
 3 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h
index c23d41e1ff..e5954635f6 100644
--- a/include/qemu/coroutine.h
+++ b/include/qemu/coroutine.h
@@ -234,6 +234,19 @@ void qemu_co_queue_restart_all(CoQueue *queue);
     qemu_co_enter_next_impl(queue, QEMU_MAKE_LOCKABLE(lock))
 bool qemu_co_enter_next_impl(CoQueue *queue, QemuLockable *lock);
 
+/**
+ * Empties the CoQueue, waking the waiting coroutine one at a time.  Unlike
+ * qemu_co_queue_all, this function releases the lock during aio_co_wake
+ * because it is meant to be used outside coroutine context; in that case, the
+ * coroutine is entered immediately, before qemu_co_enter_all returns.
+ *
+ * If used in coroutine context, qemu_co_enter_all is equivalent to
+ * qemu_co_queue_all.
+ */
+#define qemu_co_enter_all(queue, lock) \
+    qemu_co_enter_all_impl(queue, QEMU_MAKE_LOCKABLE(lock))
+void qemu_co_enter_all_impl(CoQueue *queue, QemuLockable *lock);
+
 /**
  * Checks if the CoQueue is empty.
  */
diff --git a/ui/console.c b/ui/console.c
index 15d0f6affd..36c80cd1de 100644
--- a/ui/console.c
+++ b/ui/console.c
@@ -221,7 +221,7 @@ static void gui_setup_refresh(DisplayState *ds)
 void graphic_hw_update_done(QemuConsole *con)
 {
     if (con) {
-        qemu_co_queue_restart_all(&con->dump_queue);
+        qemu_co_enter_all(&con->dump_queue, NULL);
     }
 }
 
diff --git a/util/qemu-coroutine-lock.c b/util/qemu-coroutine-lock.c
index 5705cfea2e..5b0342faed 100644
--- a/util/qemu-coroutine-lock.c
+++ b/util/qemu-coroutine-lock.c
@@ -108,6 +108,13 @@ bool coroutine_fn qemu_co_queue_next(CoQueue *queue)
     return qemu_co_enter_next_impl(queue, NULL);
 }
 
+void qemu_co_enter_all_impl(CoQueue *queue, QemuLockable *lock)
+{
+    while (qemu_co_enter_next_impl(queue, lock)) {
+        /* just loop */
+    }
+}
+
 bool qemu_co_queue_empty(CoQueue *queue)
 {
     return QSIMPLEQ_FIRST(&queue->entries) == NULL;

From f0d43b1ecef04105e0d0f55658182510b4e0f58e Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 27 Apr 2022 15:08:30 +0200
Subject: [PATCH 258/935] coroutine-lock: qemu_co_queue_restart_all is a
 coroutine-only qemu_co_enter_all

qemu_co_queue_restart_all is basically the same as qemu_co_enter_all
but without a QemuLockable argument.  That's perfectly fine, but only as
long as the function is marked coroutine_fn.  If used outside coroutine
context, qemu_co_queue_wait will attempt to take the lock and that
is just broken: if you are calling qemu_co_queue_restart_all outside
coroutine context, the lock is going to be a QemuMutex which cannot be
taken twice by the same thread.

The patch adds the marker to qemu_co_queue_restart_all and to its sole
non-coroutine_fn caller; it then reimplements the function in terms of
qemu_co_enter_all_impl, to remove duplicated code and to clarify that the
latter also works in coroutine context.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20220427130830.150180-4-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 block/io.c                 |  2 +-
 include/qemu/coroutine.h   |  7 ++++---
 util/qemu-coroutine-lock.c | 21 ++++++---------------
 3 files changed, 11 insertions(+), 19 deletions(-)

diff --git a/block/io.c b/block/io.c
index 9769ec53b0..789e6373d5 100644
--- a/block/io.c
+++ b/block/io.c
@@ -751,7 +751,7 @@ void bdrv_drain_all(void)
  *
  * This function should be called when a tracked request is completing.
  */
-static void tracked_request_end(BdrvTrackedRequest *req)
+static void coroutine_fn tracked_request_end(BdrvTrackedRequest *req)
 {
     if (req->serialising) {
         qatomic_dec(&req->bs->serialising_in_flight);
diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h
index e5954635f6..43df7a7e66 100644
--- a/include/qemu/coroutine.h
+++ b/include/qemu/coroutine.h
@@ -216,10 +216,11 @@ void coroutine_fn qemu_co_queue_wait_impl(CoQueue *queue, QemuLockable *lock);
 bool coroutine_fn qemu_co_queue_next(CoQueue *queue);
 
 /**
- * Empties the CoQueue; all coroutines are woken up.
- * OK to run from coroutine and non-coroutine context.
+ * Empties the CoQueue and queues the coroutine to run after
+ * the currently-running coroutine yields.
+ * Used from coroutine context, use qemu_co_enter_all outside.
  */
-void qemu_co_queue_restart_all(CoQueue *queue);
+void coroutine_fn qemu_co_queue_restart_all(CoQueue *queue);
 
 /**
  * Removes the next coroutine from the CoQueue, and wake it up.  Unlike
diff --git a/util/qemu-coroutine-lock.c b/util/qemu-coroutine-lock.c
index 5b0342faed..9ad24ab1af 100644
--- a/util/qemu-coroutine-lock.c
+++ b/util/qemu-coroutine-lock.c
@@ -67,21 +67,6 @@ void coroutine_fn qemu_co_queue_wait_impl(CoQueue *queue, QemuLockable *lock)
     }
 }
 
-void qemu_co_queue_restart_all(CoQueue *queue)
-{
-    Coroutine *next;
-
-    if (QSIMPLEQ_EMPTY(&queue->entries)) {
-        return false;
-    }
-
-    while ((next = QSIMPLEQ_FIRST(&queue->entries)) != NULL) {
-        QSIMPLEQ_REMOVE_HEAD(&queue->entries, co_queue_next);
-        aio_co_wake(next);
-    }
-    return true;
-}
-
 bool qemu_co_enter_next_impl(CoQueue *queue, QemuLockable *lock)
 {
     Coroutine *next;
@@ -115,6 +100,12 @@ void qemu_co_enter_all_impl(CoQueue *queue, QemuLockable *lock)
     }
 }
 
+void coroutine_fn qemu_co_queue_restart_all(CoQueue *queue)
+{
+    /* No unlock/lock needed in coroutine context.  */
+    qemu_co_enter_all_impl(queue, NULL);
+}
+
 bool qemu_co_queue_empty(CoQueue *queue)
 {
     return QSIMPLEQ_FIRST(&queue->entries) == NULL;

From d93e839ccd26ac95650fe31e45c91394936ddc27 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 11 May 2022 09:40:35 +0200
Subject: [PATCH 259/935] vhost-backend: do not depend on CONFIG_VHOST_VSOCK

The vsock callbacks .vhost_vsock_set_guest_cid and
.vhost_vsock_set_running are the only ones to be conditional
on #ifdef CONFIG_VHOST_VSOCK.  This is different from any other
device-dependent callbacks like .vhost_scsi_set_endpoint, and it
also broke when CONFIG_VHOST_VSOCK was changed to a per-target
symbol.

It would be possible to also use the CONFIG_DEVICES include, but
really there is no reason for most virtio files to be per-target
so just remove the #ifdef to fix the issue.

Reported-by: Dov Murik <dovmurik@linux.ibm.com>
Fixes: 9972ae314f ("build: move vhost-vsock configuration to Kconfig")
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/virtio/vhost-backend.c | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/hw/virtio/vhost-backend.c b/hw/virtio/vhost-backend.c
index e409a865ae..4de8b6b3b0 100644
--- a/hw/virtio/vhost-backend.c
+++ b/hw/virtio/vhost-backend.c
@@ -203,7 +203,6 @@ static int vhost_kernel_get_vq_index(struct vhost_dev *dev, int idx)
     return idx - dev->vq_index;
 }
 
-#ifdef CONFIG_VHOST_VSOCK
 static int vhost_kernel_vsock_set_guest_cid(struct vhost_dev *dev,
                                             uint64_t guest_cid)
 {
@@ -214,7 +213,6 @@ static int vhost_kernel_vsock_set_running(struct vhost_dev *dev, int start)
 {
     return vhost_kernel_call(dev, VHOST_VSOCK_SET_RUNNING, &start);
 }
-#endif /* CONFIG_VHOST_VSOCK */
 
 static void vhost_kernel_iotlb_read(void *opaque)
 {
@@ -319,10 +317,8 @@ const VhostOps kernel_ops = {
         .vhost_set_owner = vhost_kernel_set_owner,
         .vhost_reset_device = vhost_kernel_reset_device,
         .vhost_get_vq_index = vhost_kernel_get_vq_index,
-#ifdef CONFIG_VHOST_VSOCK
         .vhost_vsock_set_guest_cid = vhost_kernel_vsock_set_guest_cid,
         .vhost_vsock_set_running = vhost_kernel_vsock_set_running,
-#endif /* CONFIG_VHOST_VSOCK */
         .vhost_set_iotlb_callback = vhost_kernel_set_iotlb_callback,
         .vhost_send_device_iotlb_msg = vhost_kernel_send_device_iotlb_msg,
 };

From e23a13c0426774443543d78eef84ec8404534f0b Mon Sep 17 00:00:00 2001
From: Kshitij Suri <kshitij.suri@nutanix.com>
Date: Tue, 10 May 2022 16:19:32 +0000
Subject: [PATCH 260/935] meson: link libpng independent of vnc

Currently png support is dependent on vnc for linking object file to
libpng. This commit makes the parameter independent of vnc as it breaks
system emulator with --disable-vnc unless --disable-png is added.

Fixes: 9a0a119a38 ("Added parameter to take screenshot with screendump as PNG", 2022-04-27)
Signed-off-by: Kshitij Suri <kshitij.suri@nutanix.com>
Message-Id: <20220510161932.228481-1-kshitij.suri@nutanix.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 ui/meson.build | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/ui/meson.build b/ui/meson.build
index eba93b41e3..e9f48c5315 100644
--- a/ui/meson.build
+++ b/ui/meson.build
@@ -2,6 +2,7 @@ softmmu_ss.add(pixman)
 specific_ss.add(when: ['CONFIG_SOFTMMU'], if_true: pixman)   # for the include path
 specific_ss.add(when: ['CONFIG_SOFTMMU'], if_true: opengl)   # for the include path
 
+softmmu_ss.add(png)
 softmmu_ss.add(files(
   'clipboard.c',
   'console.c',
@@ -40,7 +41,7 @@ vnc_ss.add(files(
   'vnc-jobs.c',
   'vnc-clipboard.c',
 ))
-vnc_ss.add(zlib, png, jpeg, gnutls)
+vnc_ss.add(zlib, jpeg, gnutls)
 vnc_ss.add(when: sasl, if_true: files('vnc-auth-sasl.c'))
 softmmu_ss.add_all(when: vnc, if_true: vnc_ss)
 softmmu_ss.add(when: vnc, if_false: files('vnc-stubs.c'))

From defac5e2fbddf8423a354ff0454283a2115e1367 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Date: Thu, 18 Nov 2021 12:57:32 +0100
Subject: [PATCH 261/935] hw/block/fdc: Prevent end-of-track overrun
 (CVE-2021-3507)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Per the 82078 datasheet, if the end-of-track (EOT byte in
the FIFO) is more than the number of sectors per side, the
command is terminated unsuccessfully:

* 5.2.5 DATA TRANSFER TERMINATION

  The 82078 supports terminal count explicitly through
  the TC pin and implicitly through the underrun/over-
  run and end-of-track (EOT) functions. For full sector
  transfers, the EOT parameter can define the last
  sector to be transferred in a single or multisector
  transfer. If the last sector to be transferred is a par-
  tial sector, the host can stop transferring the data in
  mid-sector, and the 82078 will continue to complete
  the sector as if a hardware TC was received. The
  only difference between these implicit functions and
  TC is that they return "abnormal termination" result
  status. Such status indications can be ignored if they
  were expected.

* 6.1.3 READ TRACK

  This command terminates when the EOT specified
  number of sectors have been read. If the 82078
  does not find an I D Address Mark on the diskette
  after the second· occurrence of a pulse on the
  INDX# pin, then it sets the IC code in Status Regis-
  ter 0 to "01" (Abnormal termination), sets the MA bit
  in Status Register 1 to "1", and terminates the com-
  mand.

* 6.1.6 VERIFY

  Refer to Table 6-6 and Table 6-7 for information
  concerning the values of MT and EC versus SC and
  EOT value.

* Table 6·6. Result Phase Table

* Table 6-7. Verify Command Result Phase Table

Fix by aborting the transfer when EOT > # Sectors Per Side.

Cc: qemu-stable@nongnu.org
Cc: Hervé Poussineau <hpoussin@reactos.org>
Fixes: baca51faff0 ("floppy driver: disk geometry auto detect")
Reported-by: Alexander Bulekov <alxndr@bu.edu>
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/339
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20211118115733.4038610-2-philmd@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 hw/block/fdc.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/hw/block/fdc.c b/hw/block/fdc.c
index 347875a0cd..57bb355794 100644
--- a/hw/block/fdc.c
+++ b/hw/block/fdc.c
@@ -1530,6 +1530,14 @@ static void fdctrl_start_transfer(FDCtrl *fdctrl, int direction)
         int tmp;
         fdctrl->data_len = 128 << (fdctrl->fifo[5] > 7 ? 7 : fdctrl->fifo[5]);
         tmp = (fdctrl->fifo[6] - ks + 1);
+        if (tmp < 0) {
+            FLOPPY_DPRINTF("invalid EOT: %d\n", tmp);
+            fdctrl_stop_transfer(fdctrl, FD_SR0_ABNTERM, FD_SR1_MA, 0x00);
+            fdctrl->fifo[3] = kt;
+            fdctrl->fifo[4] = kh;
+            fdctrl->fifo[5] = ks;
+            return;
+        }
         if (fdctrl->fifo[0] & 0x80)
             tmp += fdctrl->fifo[6];
         fdctrl->data_len *= tmp;

From 46609b90d9e3a6304def11038a76b58ff43f77bc Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Date: Thu, 18 Nov 2021 12:57:33 +0100
Subject: [PATCH 262/935] tests/qtest/fdc-test: Add a regression test for
 CVE-2021-3507
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add the reproducer from https://gitlab.com/qemu-project/qemu/-/issues/339

Without the previous commit, when running 'make check-qtest-i386'
with QEMU configured with '--enable-sanitizers' we get:

  ==4028352==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x619000062a00 at pc 0x5626d03c491a bp 0x7ffdb4199410 sp 0x7ffdb4198bc0
  READ of size 786432 at 0x619000062a00 thread T0
      #0 0x5626d03c4919 in __asan_memcpy (qemu-system-i386+0x1e65919)
      #1 0x5626d1c023cc in flatview_write_continue softmmu/physmem.c:2787:13
      #2 0x5626d1bf0c0f in flatview_write softmmu/physmem.c:2822:14
      #3 0x5626d1bf0798 in address_space_write softmmu/physmem.c:2914:18
      #4 0x5626d1bf0f37 in address_space_rw softmmu/physmem.c:2924:16
      #5 0x5626d1bf14c8 in cpu_physical_memory_rw softmmu/physmem.c:2933:5
      #6 0x5626d0bd5649 in cpu_physical_memory_write include/exec/cpu-common.h:82:5
      #7 0x5626d0bd0a07 in i8257_dma_write_memory hw/dma/i8257.c:452:9
      #8 0x5626d09f825d in fdctrl_transfer_handler hw/block/fdc.c:1616:13
      #9 0x5626d0a048b4 in fdctrl_start_transfer hw/block/fdc.c:1539:13
      #10 0x5626d09f4c3e in fdctrl_write_data hw/block/fdc.c:2266:13
      #11 0x5626d09f22f7 in fdctrl_write hw/block/fdc.c:829:9
      #12 0x5626d1c20bc5 in portio_write softmmu/ioport.c:207:17

  0x619000062a00 is located 0 bytes to the right of 512-byte region [0x619000062800,0x619000062a00)
  allocated by thread T0 here:
      #0 0x5626d03c66ec in posix_memalign (qemu-system-i386+0x1e676ec)
      #1 0x5626d2b988d4 in qemu_try_memalign util/oslib-posix.c:210:11
      #2 0x5626d2b98b0c in qemu_memalign util/oslib-posix.c:226:27
      #3 0x5626d09fbaf0 in fdctrl_realize_common hw/block/fdc.c:2341:20
      #4 0x5626d0a150ed in isabus_fdc_realize hw/block/fdc-isa.c:113:5
      #5 0x5626d2367935 in device_set_realized hw/core/qdev.c:531:13

  SUMMARY: AddressSanitizer: heap-buffer-overflow (qemu-system-i386+0x1e65919) in __asan_memcpy
  Shadow bytes around the buggy address:
    0x0c32800044f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x0c3280004500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0c3280004510: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0c3280004520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0c3280004530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  =>0x0c3280004540:[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x0c3280004550: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x0c3280004560: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x0c3280004570: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x0c3280004580: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x0c3280004590: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  Shadow byte legend (one shadow byte represents 8 application bytes):
    Addressable:           00
    Heap left redzone:       fa
    Freed heap region:       fd
  ==4028352==ABORTING

[ kwolf: Added snapshot=on to prevent write file lock failure ]

Reported-by: Alexander Bulekov <alxndr@bu.edu>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 tests/qtest/fdc-test.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/tests/qtest/fdc-test.c b/tests/qtest/fdc-test.c
index 0b3c2c0d52..52ade90a7d 100644
--- a/tests/qtest/fdc-test.c
+++ b/tests/qtest/fdc-test.c
@@ -582,6 +582,26 @@ static void test_cve_2021_20196(void)
     qtest_quit(s);
 }
 
+static void test_cve_2021_3507(void)
+{
+    QTestState *s;
+
+    s = qtest_initf("-nographic -m 32M -nodefaults "
+                    "-drive file=%s,format=raw,if=floppy,snapshot=on",
+                    test_image);
+    qtest_outl(s, 0x9, 0x0a0206);
+    qtest_outw(s, 0x3f4, 0x1600);
+    qtest_outw(s, 0x3f4, 0x0000);
+    qtest_outw(s, 0x3f4, 0x0000);
+    qtest_outw(s, 0x3f4, 0x0000);
+    qtest_outw(s, 0x3f4, 0x0200);
+    qtest_outw(s, 0x3f4, 0x0200);
+    qtest_outw(s, 0x3f4, 0x0000);
+    qtest_outw(s, 0x3f4, 0x0000);
+    qtest_outw(s, 0x3f4, 0x0000);
+    qtest_quit(s);
+}
+
 int main(int argc, char **argv)
 {
     int fd;
@@ -613,6 +633,7 @@ int main(int argc, char **argv)
     qtest_add_func("/fdc/read_no_dma_19", test_read_no_dma_19);
     qtest_add_func("/fdc/fuzz-registers", fuzz_registers);
     qtest_add_func("/fdc/fuzz/cve_2021_20196", test_cve_2021_20196);
+    qtest_add_func("/fdc/fuzz/cve_2021_3507", test_cve_2021_3507);
 
     ret = g_test_run();
 

From a5fced40212ed73c715ca298a2929dd4d99c9999 Mon Sep 17 00:00:00 2001
From: Eric Blake <eblake@redhat.com>
Date: Wed, 11 May 2022 19:49:23 -0500
Subject: [PATCH 263/935] qemu-nbd: Pass max connections to blockdev layer

The next patch wants to adjust whether the NBD server code advertises
MULTI_CONN based on whether it is known if the server limits to
exactly one client.  For a server started by QMP, this information is
obtained through nbd_server_start (which can support more than one
export); but for qemu-nbd (which supports exactly one export), it is
controlled only by the command-line option -e/--shared.  Since we
already have a hook function used by qemu-nbd, it's easiest to just
alter its signature to fit our needs.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20220512004924.417153-2-eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 blockdev-nbd.c      | 8 ++++----
 include/block/nbd.h | 2 +-
 qemu-nbd.c          | 2 +-
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/blockdev-nbd.c b/blockdev-nbd.c
index 7f6531cba0..711e0e72bd 100644
--- a/blockdev-nbd.c
+++ b/blockdev-nbd.c
@@ -30,18 +30,18 @@ typedef struct NBDServerData {
 } NBDServerData;
 
 static NBDServerData *nbd_server;
-static bool is_qemu_nbd;
+static int qemu_nbd_connections = -1; /* Non-negative if this is qemu-nbd */
 
 static void nbd_update_server_watch(NBDServerData *s);
 
-void nbd_server_is_qemu_nbd(bool value)
+void nbd_server_is_qemu_nbd(int max_connections)
 {
-    is_qemu_nbd = value;
+    qemu_nbd_connections = max_connections;
 }
 
 bool nbd_server_is_running(void)
 {
-    return nbd_server || is_qemu_nbd;
+    return nbd_server || qemu_nbd_connections >= 0;
 }
 
 static void nbd_blockdev_client_closed(NBDClient *client, bool ignored)
diff --git a/include/block/nbd.h b/include/block/nbd.h
index a98eb665da..c5a29ce1c6 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -344,7 +344,7 @@ void nbd_client_new(QIOChannelSocket *sioc,
 void nbd_client_get(NBDClient *client);
 void nbd_client_put(NBDClient *client);
 
-void nbd_server_is_qemu_nbd(bool value);
+void nbd_server_is_qemu_nbd(int max_connections);
 bool nbd_server_is_running(void);
 void nbd_server_start(SocketAddress *addr, const char *tls_creds,
                       const char *tls_authz, uint32_t max_connections,
diff --git a/qemu-nbd.c b/qemu-nbd.c
index 2382b5042a..0cd5aa6f02 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -1095,7 +1095,7 @@ int main(int argc, char **argv)
 
     bs->detect_zeroes = detect_zeroes;
 
-    nbd_server_is_qemu_nbd(true);
+    nbd_server_is_qemu_nbd(shared);
 
     export_opts = g_new(BlockExportOptions, 1);
     *export_opts = (BlockExportOptions) {

From 58a6fdcc9efb2a7c1ef4893dca4aa5e8020ca3dc Mon Sep 17 00:00:00 2001
From: Eric Blake <eblake@redhat.com>
Date: Wed, 11 May 2022 19:49:24 -0500
Subject: [PATCH 264/935] nbd/server: Allow MULTI_CONN for shared writable
 exports

According to the NBD spec, a server that advertises
NBD_FLAG_CAN_MULTI_CONN promises that multiple client connections will
not see any cache inconsistencies: when properly separated by a single
flush, actions performed by one client will be visible to another
client, regardless of which client did the flush.

We always satisfy these conditions in qemu - even when we support
multiple clients, ALL clients go through a single point of reference
into the block layer, with no local caching.  The effect of one client
is instantly visible to the next client.  Even if our backend were a
network device, we argue that any multi-path caching effects that
would cause inconsistencies in back-to-back actions not seeing the
effect of previous actions would be a bug in that backend, and not the
fault of caching in qemu.  As such, it is safe to unconditionally
advertise CAN_MULTI_CONN for any qemu NBD server situation that
supports parallel clients.

Note, however, that we don't want to advertise CAN_MULTI_CONN when we
know that a second client cannot connect (for historical reasons,
qemu-nbd defaults to a single connection while nbd-server-add and QMP
commands default to unlimited connections; but we already have
existing means to let either style of NBD server creation alter those
defaults).  This is visible by no longer advertising MULTI_CONN for
'qemu-nbd -r' without -e, as in the iotest nbd-qemu-allocation.

The harder part of this patch is setting up an iotest to demonstrate
behavior of multiple NBD clients to a single server.  It might be
possible with parallel qemu-io processes, but I found it easier to do
in python with the help of libnbd, and help from Nir and Vladimir in
writing the test.

Signed-off-by: Eric Blake <eblake@redhat.com>
Suggested-by: Nir Soffer <nsoffer@redhat.com>
Suggested-by: Vladimir Sementsov-Ogievskiy <v.sementsov-og@mail.ru>
Message-Id: <20220512004924.417153-3-eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 MAINTAINERS                                   |   1 +
 blockdev-nbd.c                                |   5 +
 docs/interop/nbd.txt                          |   1 +
 docs/tools/qemu-nbd.rst                       |   3 +-
 include/block/nbd.h                           |   3 +-
 nbd/server.c                                  |  10 +-
 qapi/block-export.json                        |   8 +-
 tests/qemu-iotests/tests/nbd-multiconn        | 145 ++++++++++++++++++
 tests/qemu-iotests/tests/nbd-multiconn.out    |   5 +
 .../tests/nbd-qemu-allocation.out             |   2 +-
 10 files changed, 172 insertions(+), 11 deletions(-)
 create mode 100755 tests/qemu-iotests/tests/nbd-multiconn
 create mode 100644 tests/qemu-iotests/tests/nbd-multiconn.out

diff --git a/MAINTAINERS b/MAINTAINERS
index 571556d279..fbc0662627 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3367,6 +3367,7 @@ F: qemu-nbd.*
 F: blockdev-nbd.c
 F: docs/interop/nbd.txt
 F: docs/tools/qemu-nbd.rst
+F: tests/qemu-iotests/tests/*nbd*
 T: git https://repo.or.cz/qemu/ericb.git nbd
 T: git https://src.openvz.org/scm/~vsementsov/qemu.git nbd
 
diff --git a/blockdev-nbd.c b/blockdev-nbd.c
index 711e0e72bd..012256bb02 100644
--- a/blockdev-nbd.c
+++ b/blockdev-nbd.c
@@ -44,6 +44,11 @@ bool nbd_server_is_running(void)
     return nbd_server || qemu_nbd_connections >= 0;
 }
 
+int nbd_server_max_connections(void)
+{
+    return nbd_server ? nbd_server->max_connections : qemu_nbd_connections;
+}
+
 static void nbd_blockdev_client_closed(NBDClient *client, bool ignored)
 {
     nbd_client_put(client);
diff --git a/docs/interop/nbd.txt b/docs/interop/nbd.txt
index bdb0f2a41a..f5ca25174a 100644
--- a/docs/interop/nbd.txt
+++ b/docs/interop/nbd.txt
@@ -68,3 +68,4 @@ NBD_CMD_BLOCK_STATUS for "qemu:dirty-bitmap:", NBD_CMD_CACHE
 * 4.2: NBD_FLAG_CAN_MULTI_CONN for shareable read-only exports,
 NBD_CMD_FLAG_FAST_ZERO
 * 5.2: NBD_CMD_BLOCK_STATUS for "qemu:allocation-depth"
+* 7.1: NBD_FLAG_CAN_MULTI_CONN for shareable writable exports
diff --git a/docs/tools/qemu-nbd.rst b/docs/tools/qemu-nbd.rst
index 4c950f6199..8e08a29e89 100644
--- a/docs/tools/qemu-nbd.rst
+++ b/docs/tools/qemu-nbd.rst
@@ -139,8 +139,7 @@ driver options if :option:`--image-opts` is specified.
 .. option:: -e, --shared=NUM
 
   Allow up to *NUM* clients to share the device (default
-  ``1``), 0 for unlimited. Safe for readers, but for now,
-  consistency is not guaranteed between multiple writers.
+  ``1``), 0 for unlimited.
 
 .. option:: -t, --persistent
 
diff --git a/include/block/nbd.h b/include/block/nbd.h
index c5a29ce1c6..c74b7a9d2e 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -1,5 +1,5 @@
 /*
- *  Copyright (C) 2016-2020 Red Hat, Inc.
+ *  Copyright (C) 2016-2022 Red Hat, Inc.
  *  Copyright (C) 2005  Anthony Liguori <anthony@codemonkey.ws>
  *
  *  Network Block Device
@@ -346,6 +346,7 @@ void nbd_client_put(NBDClient *client);
 
 void nbd_server_is_qemu_nbd(int max_connections);
 bool nbd_server_is_running(void);
+int nbd_server_max_connections(void);
 void nbd_server_start(SocketAddress *addr, const char *tls_creds,
                       const char *tls_authz, uint32_t max_connections,
                       Error **errp);
diff --git a/nbd/server.c b/nbd/server.c
index 4cdbc062c1..213e00e761 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -1,5 +1,5 @@
 /*
- *  Copyright (C) 2016-2021 Red Hat, Inc.
+ *  Copyright (C) 2016-2022 Red Hat, Inc.
  *  Copyright (C) 2005  Anthony Liguori <anthony@codemonkey.ws>
  *
  *  Network Block Device Server Side
@@ -1642,7 +1642,6 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args,
     int64_t size;
     uint64_t perm, shared_perm;
     bool readonly = !exp_args->writable;
-    bool shared = !exp_args->writable;
     BlockDirtyBitmapOrStrList *bitmaps;
     size_t i;
     int ret;
@@ -1693,11 +1692,12 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args,
     exp->description = g_strdup(arg->description);
     exp->nbdflags = (NBD_FLAG_HAS_FLAGS | NBD_FLAG_SEND_FLUSH |
                      NBD_FLAG_SEND_FUA | NBD_FLAG_SEND_CACHE);
+
+    if (nbd_server_max_connections() != 1) {
+        exp->nbdflags |= NBD_FLAG_CAN_MULTI_CONN;
+    }
     if (readonly) {
         exp->nbdflags |= NBD_FLAG_READ_ONLY;
-        if (shared) {
-            exp->nbdflags |= NBD_FLAG_CAN_MULTI_CONN;
-        }
     } else {
         exp->nbdflags |= (NBD_FLAG_SEND_TRIM | NBD_FLAG_SEND_WRITE_ZEROES |
                           NBD_FLAG_SEND_FAST_ZERO);
diff --git a/qapi/block-export.json b/qapi/block-export.json
index 1de16d2589..7776248435 100644
--- a/qapi/block-export.json
+++ b/qapi/block-export.json
@@ -22,7 +22,9 @@
 #             recreated on the fly while the NBD server is active.
 #             If missing, it will default to denying access (since 4.0).
 # @max-connections: The maximum number of connections to allow at the same
-#                   time, 0 for unlimited. (since 5.2; default: 0)
+#                   time, 0 for unlimited. Setting this to 1 also stops
+#                   the server from advertising multiple client support
+#                   (since 5.2; default: 0)
 #
 # Since: 4.2
 ##
@@ -51,7 +53,9 @@
 #             recreated on the fly while the NBD server is active.
 #             If missing, it will default to denying access (since 4.0).
 # @max-connections: The maximum number of connections to allow at the same
-#                   time, 0 for unlimited. (since 5.2; default: 0)
+#                   time, 0 for unlimited. Setting this to 1 also stops
+#                   the server from advertising multiple client support
+#                   (since 5.2; default: 0).
 #
 # Returns: error if the server is already running.
 #
diff --git a/tests/qemu-iotests/tests/nbd-multiconn b/tests/qemu-iotests/tests/nbd-multiconn
new file mode 100755
index 0000000000..b121f2e363
--- /dev/null
+++ b/tests/qemu-iotests/tests/nbd-multiconn
@@ -0,0 +1,145 @@
+#!/usr/bin/env python3
+# group: rw auto quick
+#
+# Test cases for NBD multi-conn advertisement
+#
+# Copyright (C) 2022 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import os
+from contextlib import contextmanager
+import iotests
+from iotests import qemu_img_create, qemu_io
+
+
+disk = os.path.join(iotests.test_dir, 'disk')
+size = '4M'
+nbd_sock = os.path.join(iotests.sock_dir, 'nbd_sock')
+nbd_uri = 'nbd+unix:///{}?socket=' + nbd_sock
+
+
+@contextmanager
+def open_nbd(export_name):
+    h = nbd.NBD()
+    try:
+        h.connect_uri(nbd_uri.format(export_name))
+        yield h
+    finally:
+        h.shutdown()
+
+class TestNbdMulticonn(iotests.QMPTestCase):
+    def setUp(self):
+        qemu_img_create('-f', iotests.imgfmt, disk, size)
+        qemu_io('-c', 'w -P 1 0 2M', '-c', 'w -P 2 2M 2M', disk)
+
+        self.vm = iotests.VM()
+        self.vm.launch()
+        result = self.vm.qmp('blockdev-add', {
+            'driver': 'qcow2',
+            'node-name': 'n',
+            'file': {'driver': 'file', 'filename': disk}
+        })
+        self.assert_qmp(result, 'return', {})
+
+    def tearDown(self):
+        self.vm.shutdown()
+        os.remove(disk)
+        try:
+            os.remove(nbd_sock)
+        except OSError:
+            pass
+
+    @contextmanager
+    def run_server(self, max_connections=None):
+        args = {
+            'addr': {
+                'type': 'unix',
+                'data': {'path': nbd_sock}
+            }
+        }
+        if max_connections is not None:
+            args['max-connections'] = max_connections
+
+        result = self.vm.qmp('nbd-server-start', args)
+        self.assert_qmp(result, 'return', {})
+        yield
+
+        result = self.vm.qmp('nbd-server-stop')
+        self.assert_qmp(result, 'return', {})
+
+    def add_export(self, name, writable=None):
+        args = {
+            'type': 'nbd',
+            'id': name,
+            'node-name': 'n',
+            'name': name,
+        }
+        if writable is not None:
+            args['writable'] = writable
+
+        result = self.vm.qmp('block-export-add', args)
+        self.assert_qmp(result, 'return', {})
+
+    def test_default_settings(self):
+        with self.run_server():
+            self.add_export('r')
+            self.add_export('w', writable=True)
+            with open_nbd('r') as h:
+                self.assertTrue(h.can_multi_conn())
+            with open_nbd('w') as h:
+                self.assertTrue(h.can_multi_conn())
+
+    def test_limited_connections(self):
+        with self.run_server(max_connections=1):
+            self.add_export('r')
+            self.add_export('w', writable=True)
+            with open_nbd('r') as h:
+                self.assertFalse(h.can_multi_conn())
+            with open_nbd('w') as h:
+                self.assertFalse(h.can_multi_conn())
+
+    def test_parallel_writes(self):
+        with self.run_server():
+            self.add_export('w', writable=True)
+
+            clients = [nbd.NBD() for _ in range(3)]
+            for c in clients:
+                c.connect_uri(nbd_uri.format('w'))
+                self.assertTrue(c.can_multi_conn())
+
+            initial_data = clients[0].pread(1024 * 1024, 0)
+            self.assertEqual(initial_data, b'\x01' * 1024 * 1024)
+
+            updated_data = b'\x03' * 1024 * 1024
+            clients[1].pwrite(updated_data, 0)
+            clients[2].flush()
+            current_data = clients[0].pread(1024 * 1024, 0)
+
+            self.assertEqual(updated_data, current_data)
+
+            for i in range(3):
+                clients[i].shutdown()
+
+
+if __name__ == '__main__':
+    try:
+        # Easier to use libnbd than to try and set up parallel
+        # 'qemu-nbd --list' or 'qemu-io' processes, but not all systems
+        # have libnbd installed.
+        import nbd  # type: ignore
+
+        iotests.main(supported_fmts=['qcow2'])
+    except ImportError:
+        iotests.notrun('libnbd not installed')
diff --git a/tests/qemu-iotests/tests/nbd-multiconn.out b/tests/qemu-iotests/tests/nbd-multiconn.out
new file mode 100644
index 0000000000..8d7e996700
--- /dev/null
+++ b/tests/qemu-iotests/tests/nbd-multiconn.out
@@ -0,0 +1,5 @@
+...
+----------------------------------------------------------------------
+Ran 3 tests
+
+OK
diff --git a/tests/qemu-iotests/tests/nbd-qemu-allocation.out b/tests/qemu-iotests/tests/nbd-qemu-allocation.out
index 0bf1abb063..9d938db24e 100644
--- a/tests/qemu-iotests/tests/nbd-qemu-allocation.out
+++ b/tests/qemu-iotests/tests/nbd-qemu-allocation.out
@@ -17,7 +17,7 @@ wrote 2097152/2097152 bytes at offset 1048576
 exports available: 1
  export: ''
   size:  4194304
-  flags: 0x58f ( readonly flush fua df multi cache )
+  flags: 0x48f ( readonly flush fua df cache )
   min block: 1
   opt block: 4096
   max block: 33554432

From 7adb75d6be0b11ace0fabfb8cddc890243773782 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 11 May 2022 13:50:43 -0400
Subject: [PATCH 265/935] vl: make machine type deprecation a warning

error_report should generally be followed by a failure; if we can proceed
anyway, that is just a warning and should be communicated properly to
the user with warn_report.

Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220511175043.27327-1-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 softmmu/vl.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index edba74f075..817d515783 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -3631,7 +3631,7 @@ void qemu_init(int argc, char **argv, char **envp)
 
     machine_class = MACHINE_GET_CLASS(current_machine);
     if (!qtest_enabled() && machine_class->deprecation_reason) {
-        error_report("Machine type '%s' is deprecated: %s",
+        warn_report("Machine type '%s' is deprecated: %s",
                      machine_class->name, machine_class->deprecation_reason);
     }
 

From 333dbac358acc6cc174029263d633a22f66584b4 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 11 May 2022 18:39:12 +0200
Subject: [PATCH 266/935] vmxcap: add tertiary execution controls

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 scripts/kvm/vmxcap | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/scripts/kvm/vmxcap b/scripts/kvm/vmxcap
index f140040104..ce27f5e635 100755
--- a/scripts/kvm/vmxcap
+++ b/scripts/kvm/vmxcap
@@ -23,6 +23,7 @@ MSR_IA32_VMX_TRUE_PROCBASED_CTLS = 0x48E
 MSR_IA32_VMX_TRUE_EXIT_CTLS = 0x48F
 MSR_IA32_VMX_TRUE_ENTRY_CTLS = 0x490
 MSR_IA32_VMX_VMFUNC = 0x491
+MSR_IA32_VMX_PROCBASED_CTLS3 = 0x492
 
 class msr(object):
     def __init__(self):
@@ -71,6 +72,13 @@ class Control(object):
                 s = 'yes'
             print('  %-40s %s' % (self.bits[bit], s))
 
+# All 64 bits in the tertiary controls MSR are allowed-1
+class Allowed1Control(Control):
+    def read2(self, nr):
+        m = msr()
+        val = m.read(nr, 0)
+        return (0, val)
+
 class Misc(object):
     def __init__(self, name, bits, msr):
         self.name = name
@@ -135,6 +143,7 @@ controls = [
             12: 'RDTSC exiting',
             15: 'CR3-load exiting',
             16: 'CR3-store exiting',
+            17: 'Activate tertiary controls',
             19: 'CR8-load exiting',
             20: 'CR8-store exiting',
             21: 'Use TPR shadow',
@@ -186,6 +195,14 @@ controls = [
         cap_msr = MSR_IA32_VMX_PROCBASED_CTLS2,
         ),
 
+    Allowed1Control(
+        name = 'tertiary processor-based controls',
+        bits = {
+            4: 'Enable IPI virtualization'
+            },
+        cap_msr = MSR_IA32_VMX_PROCBASED_CTLS3,
+        ),
+
     Control(
         name = 'VM-Exit controls',
         bits = {

From f70625299ecc9ba577c87f3d1d75012c747c7d88 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Thu, 5 May 2022 11:47:23 +0200
Subject: [PATCH 267/935] qemu-iotests: inline common.config into common.rc

common.rc has some complicated logic to find the common.config that
dates back to xfstests and is completely unnecessary now.  Just include
the contents of the file.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220505094723.732116-1-pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 tests/qemu-iotests/common.config | 41 --------------------------------
 tests/qemu-iotests/common.rc     | 31 ++++++++++++++----------
 2 files changed, 19 insertions(+), 53 deletions(-)
 delete mode 100644 tests/qemu-iotests/common.config

diff --git a/tests/qemu-iotests/common.config b/tests/qemu-iotests/common.config
deleted file mode 100644
index 9bd1a5a6fc..0000000000
--- a/tests/qemu-iotests/common.config
+++ /dev/null
@@ -1,41 +0,0 @@
-#!/usr/bin/env bash
-#
-# Copyright (C) 2009 Red Hat, Inc.
-# Copyright (c) 2000-2003,2006 Silicon Graphics, Inc.  All Rights Reserved.
-#
-# This program is free software; you can redistribute it and/or
-# modify it under the terms of the GNU General Public License as
-# published by the Free Software Foundation.
-#
-# This program is distributed in the hope that it would be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-#
-# all tests should use a common language setting to prevent golden
-# output mismatches.
-export LANG=C
-
-PATH=".:$PATH"
-
-HOSTOS=$(uname -s)
-arch=$(uname -m)
-[[ "$arch" =~ "ppc64" ]] && qemu_arch=ppc64 || qemu_arch="$arch"
-
-# make sure we have a standard umask
-umask 022
-
-_optstr_add()
-{
-    if [ -n "$1" ]; then
-        echo "$1,$2"
-    else
-        echo "$2"
-    fi
-}
-
-# make sure this script returns success
-true
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index 227e0a5be9..165b54a61e 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -17,6 +17,17 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
+export LANG=C
+
+PATH=".:$PATH"
+
+HOSTOS=$(uname -s)
+arch=$(uname -m)
+[[ "$arch" =~ "ppc64" ]] && qemu_arch=ppc64 || qemu_arch="$arch"
+
+# make sure we have a standard umask
+umask 022
+
 # bail out, setting up .notrun file
 _notrun()
 {
@@ -120,18 +131,14 @@ peek_file_raw()
     dd if="$1" bs=1 skip="$2" count="$3" status=none
 }
 
-config=common.config
-test -f $config || config=../common.config
-if ! test -f $config
-then
-    echo "$0: failed to find common.config"
-    exit 1
-fi
-if ! . $config
-    then
-    echo "$0: failed to source common.config"
-    exit 1
-fi
+_optstr_add()
+{
+    if [ -n "$1" ]; then
+        echo "$1,$2"
+    else
+        echo "$2"
+    fi
+}
 
 # Set the variables to the empty string to turn Valgrind off
 # for specific processes, e.g.

From 06134e2bc35dc21543d4cbcf31f858c03d383442 Mon Sep 17 00:00:00 2001
From: Halil Pasic <pasic@linux.ibm.com>
Date: Mon, 7 Mar 2022 12:29:39 +0100
Subject: [PATCH 268/935] virtio: fix feature negotiation for ACCESS_PLATFORM

Unlike most virtio features ACCESS_PLATFORM is considered mandatory by
QEMU, i.e. the driver must accept it if offered by the device. The
virtio specification says that the driver SHOULD accept the
ACCESS_PLATFORM feature if offered, and that the device MAY fail to
operate if ACCESS_PLATFORM was offered but not negotiated.

While a SHOULD ain't exactly a MUST, we are certainly allowed to fail
the device when the driver fences ACCESS_PLATFORM. With commit
2943b53f68 ("virtio: force VIRTIO_F_IOMMU_PLATFORM") we already made the
decision to do so whenever the get_dma_as() callback is implemented (by
the bus), which in practice means for the entirety of virtio-pci.

That means, if the device needs to translate I/O addresses, then
ACCESS_PLATFORM is mandatory. The aforementioned commit tells us in the
commit message that this is for security reasons. More precisely if we
were to allow a less then trusted driver (e.g. an user-space driver, or
a nested guest) to make the device bypass the IOMMU by not negotiating
ACCESS_PLATFORM, then the guest kernel would have no ability to
control/police (by programming the IOMMU) what pieces of guest memory
the driver may manipulate using the device. Which would break security
assumptions within the guest.

If ACCESS_PLATFORM is offered not because we want the device to utilize
an IOMMU and do address translation, but because the device does not
have access to the entire guest RAM, and needs the driver to grant
access to the bits it needs access to (e.g. confidential guest support),
we still require the guest to have the corresponding logic and to accept
ACCESS_PLATFORM. If the driver does not accept ACCESS_PLATFORM, then
things are bound to go wrong, and we may see failures much less graceful
than failing the device because the driver didn't negotiate
ACCESS_PLATFORM.

So let us make ACCESS_PLATFORM mandatory for the driver regardless
of whether the get_dma_as() callback is implemented or not.

Signed-off-by: Halil Pasic <pasic@linux.ibm.com>
Fixes: 2943b53f68 ("virtio: force VIRTIO_F_IOMMU_PLATFORM")

Message-Id: <20220307112939.2780117-1-pasic@linux.ibm.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
---
 hw/virtio/virtio-bus.c | 22 ++++++++++++++--------
 1 file changed, 14 insertions(+), 8 deletions(-)

diff --git a/hw/virtio/virtio-bus.c b/hw/virtio/virtio-bus.c
index 0f69d1c742..d7ec023adf 100644
--- a/hw/virtio/virtio-bus.c
+++ b/hw/virtio/virtio-bus.c
@@ -78,17 +78,23 @@ void virtio_bus_device_plugged(VirtIODevice *vdev, Error **errp)
         return;
     }
 
-    vdev_has_iommu = virtio_host_has_feature(vdev, VIRTIO_F_IOMMU_PLATFORM);
-    if (klass->get_dma_as != NULL && has_iommu) {
+    vdev->dma_as = &address_space_memory;
+    if (has_iommu) {
+        vdev_has_iommu = virtio_host_has_feature(vdev, VIRTIO_F_IOMMU_PLATFORM);
+        /*
+         * Present IOMMU_PLATFORM to the driver iff iommu_plattform=on and
+         * device operational. If the driver does not accept IOMMU_PLATFORM
+         * we fail the device.
+         */
         virtio_add_feature(&vdev->host_features, VIRTIO_F_IOMMU_PLATFORM);
-        vdev->dma_as = klass->get_dma_as(qbus->parent);
-        if (!vdev_has_iommu && vdev->dma_as != &address_space_memory) {
-            error_setg(errp,
+        if (klass->get_dma_as) {
+            vdev->dma_as = klass->get_dma_as(qbus->parent);
+            if (!vdev_has_iommu && vdev->dma_as != &address_space_memory) {
+                error_setg(errp,
                        "iommu_platform=true is not supported by the device");
-            return;
+                return;
+            }
         }
-    } else {
-        vdev->dma_as = &address_space_memory;
     }
 }
 

From 250227f4fa4efd6032da6c39d8fb2e6c9192d6ce Mon Sep 17 00:00:00 2001
From: Jason Wang <jasowang@redhat.com>
Date: Fri, 1 Apr 2022 10:28:24 +0800
Subject: [PATCH 269/935] intel-iommu: correct the value used for
 error_setg_errno()

error_setg_errno() expects a normal errno value, not a negated
one, so we should use ENOTSUP instead of -ENOSUP.

Fixes: Coverity CID 1487174
Fixes: ("intel_iommu: support snoop control")
Signed-off-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20220401022824.9337-1-jasowang@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Peter Xu <peterx@redhat.com>
---
 hw/i386/intel_iommu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index c64aa81a83..e05d69a2c0 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -3032,7 +3032,7 @@ static int vtd_iommu_notify_flag_changed(IOMMUMemoryRegion *iommu,
 
     /* TODO: add support for VFIO and vhost users */
     if (s->snoop_control) {
-        error_setg_errno(errp, -ENOTSUP,
+        error_setg_errno(errp, ENOTSUP,
                          "Snoop Control with vhost or VFIO is not supported");
         return -ENOTSUP;
     }

From cf04aba2a9d276336c45d2434f94458838a01034 Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:26 +0100
Subject: [PATCH 270/935] hw/pci/cxl: Add a CXL component type (interface)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

A CXL component is a hardware entity that implements CXL component
registers from the CXL 2.0 spec (8.2.3). Currently these represent 3
general types.
1. Host Bridge
2. Ports (root, upstream, downstream)
3. Devices (memory, other)

A CXL component can be conceptually thought of as a PCIe device with
extra functionality when enumerated and enabled. For this reason, CXL
does here, and will continue to add on to existing PCI code paths.

Host bridges will typically need to be handled specially and so they can
implement this newly introduced interface or not. All other components
should implement this interface. Implementing this interface allows the
core PCI code to treat these devices as special where appropriate.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Adam Manzanares <a.manzanares@samsung.com>
Message-Id: <20220429144110.25167-2-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/pci/pci.c         | 10 ++++++++++
 include/hw/pci/pci.h |  8 ++++++++
 2 files changed, 18 insertions(+)

diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index 9c58f02853..eb884adef9 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -200,6 +200,11 @@ static const TypeInfo pci_bus_info = {
     .class_init = pci_bus_class_init,
 };
 
+static const TypeInfo cxl_interface_info = {
+    .name          = INTERFACE_CXL_DEVICE,
+    .parent        = TYPE_INTERFACE,
+};
+
 static const TypeInfo pcie_interface_info = {
     .name          = INTERFACE_PCIE_DEVICE,
     .parent        = TYPE_INTERFACE,
@@ -2182,6 +2187,10 @@ static void pci_qdev_realize(DeviceState *qdev, Error **errp)
         pci_dev->cap_present |= QEMU_PCI_CAP_EXPRESS;
     }
 
+    if (object_class_dynamic_cast(klass, INTERFACE_CXL_DEVICE)) {
+        pci_dev->cap_present |= QEMU_PCIE_CAP_CXL;
+    }
+
     pci_dev = do_pci_register_device(pci_dev,
                                      object_get_typename(OBJECT(qdev)),
                                      pci_dev->devfn, errp);
@@ -2938,6 +2947,7 @@ static void pci_register_types(void)
     type_register_static(&pci_bus_info);
     type_register_static(&pcie_bus_info);
     type_register_static(&conventional_pci_interface_info);
+    type_register_static(&cxl_interface_info);
     type_register_static(&pcie_interface_info);
     type_register_static(&pci_device_type_info);
 }
diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h
index 3a32b8dd40..98f0d1b844 100644
--- a/include/hw/pci/pci.h
+++ b/include/hw/pci/pci.h
@@ -194,6 +194,8 @@ enum {
     QEMU_PCIE_LNKSTA_DLLLA = (1 << QEMU_PCIE_LNKSTA_DLLLA_BITNR),
 #define QEMU_PCIE_EXTCAP_INIT_BITNR 9
     QEMU_PCIE_EXTCAP_INIT = (1 << QEMU_PCIE_EXTCAP_INIT_BITNR),
+#define QEMU_PCIE_CXL_BITNR 10
+    QEMU_PCIE_CAP_CXL = (1 << QEMU_PCIE_CXL_BITNR),
 };
 
 #define TYPE_PCI_DEVICE "pci-device"
@@ -201,6 +203,12 @@ typedef struct PCIDeviceClass PCIDeviceClass;
 DECLARE_OBJ_CHECKERS(PCIDevice, PCIDeviceClass,
                      PCI_DEVICE, TYPE_PCI_DEVICE)
 
+/*
+ * Implemented by devices that can be plugged on CXL buses. In the spec, this is
+ * actually a "CXL Component, but we name it device to match the PCI naming.
+ */
+#define INTERFACE_CXL_DEVICE "cxl-device"
+
 /* Implemented by devices that can be plugged on PCI Express buses */
 #define INTERFACE_PCIE_DEVICE "pci-express-device"
 

From 9e58f52d3f809712762bef735a02290fd4436b0c Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:27 +0100
Subject: [PATCH 271/935] hw/cxl/component: Introduce CXL components (8.1.x,
 8.2.5)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

A CXL 2.0 component is any entity in the CXL topology. All components
have a analogous function in PCIe. Except for the CXL host bridge, all
have a PCIe config space that is accessible via the common PCIe
mechanisms. CXL components are enumerated via DVSEC fields in the
extended PCIe header space. CXL components will minimally implement some
subset of CXL.mem and CXL.cache registers defined in 8.2.5 of the CXL
2.0 specification. Two headers and a utility library are introduced to
support the minimum functionality needed to enumerate components.

The cxl_pci header manages bits associated with PCI, specifically the
DVSEC and related fields. The cxl_component.h variant has data
structures and APIs that are useful for drivers implementing any of the
CXL 2.0 components. The library takes care of making use of the DVSEC
bits and the CXL.[mem|cache] registers. Per spec, the registers are
little endian.

None of the mechanisms required to enumerate a CXL capable hostbridge
are introduced at this point.

Note that the CXL.mem and CXL.cache registers used are always 4B wide.
It's possible in the future that this constraint will not hold.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Adam Manzanares <a.manzanares@samsung.com>
Message-Id: <20220429144110.25167-3-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/Kconfig                     |   1 +
 hw/cxl/Kconfig                 |   3 +
 hw/cxl/cxl-component-utils.c   | 313 +++++++++++++++++++++++++++++++++
 hw/cxl/meson.build             |   4 +
 hw/meson.build                 |   1 +
 include/hw/cxl/cxl.h           |  16 ++
 include/hw/cxl/cxl_component.h | 213 ++++++++++++++++++++++
 include/hw/cxl/cxl_pci.h       | 146 +++++++++++++++
 8 files changed, 697 insertions(+)
 create mode 100644 hw/cxl/Kconfig
 create mode 100644 hw/cxl/cxl-component-utils.c
 create mode 100644 hw/cxl/meson.build
 create mode 100644 include/hw/cxl/cxl.h
 create mode 100644 include/hw/cxl/cxl_component.h
 create mode 100644 include/hw/cxl/cxl_pci.h

diff --git a/hw/Kconfig b/hw/Kconfig
index ad20cce0a9..50e0952889 100644
--- a/hw/Kconfig
+++ b/hw/Kconfig
@@ -6,6 +6,7 @@ source audio/Kconfig
 source block/Kconfig
 source char/Kconfig
 source core/Kconfig
+source cxl/Kconfig
 source display/Kconfig
 source dma/Kconfig
 source gpio/Kconfig
diff --git a/hw/cxl/Kconfig b/hw/cxl/Kconfig
new file mode 100644
index 0000000000..8e67519b16
--- /dev/null
+++ b/hw/cxl/Kconfig
@@ -0,0 +1,3 @@
+config CXL
+    bool
+    default y if PCI_EXPRESS
diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c
new file mode 100644
index 0000000000..9457a1b029
--- /dev/null
+++ b/hw/cxl/cxl-component-utils.c
@@ -0,0 +1,313 @@
+/*
+ * CXL Utility library for components
+ *
+ * Copyright(C) 2020 Intel Corporation.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See the
+ * COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "hw/pci/pci.h"
+#include "hw/cxl/cxl.h"
+
+static uint64_t cxl_cache_mem_read_reg(void *opaque, hwaddr offset,
+                                       unsigned size)
+{
+    CXLComponentState *cxl_cstate = opaque;
+    ComponentRegisters *cregs = &cxl_cstate->crb;
+
+    if (size == 8) {
+        qemu_log_mask(LOG_UNIMP,
+                      "CXL 8 byte cache mem registers not implemented\n");
+        return 0;
+    }
+
+    if (cregs->special_ops && cregs->special_ops->read) {
+        return cregs->special_ops->read(cxl_cstate, offset, size);
+    } else {
+        return cregs->cache_mem_registers[offset / sizeof(*cregs->cache_mem_registers)];
+    }
+}
+
+static void cxl_cache_mem_write_reg(void *opaque, hwaddr offset, uint64_t value,
+                                    unsigned size)
+{
+    CXLComponentState *cxl_cstate = opaque;
+    ComponentRegisters *cregs = &cxl_cstate->crb;
+    uint32_t mask;
+
+    if (size == 8) {
+        qemu_log_mask(LOG_UNIMP,
+                      "CXL 8 byte cache mem registers not implemented\n");
+        return;
+    }
+    mask = cregs->cache_mem_regs_write_mask[offset / sizeof(*cregs->cache_mem_regs_write_mask)];
+    value &= mask;
+    /* RO bits should remain constant. Done by reading existing value */
+    value |= ~mask & cregs->cache_mem_registers[offset / sizeof(*cregs->cache_mem_registers)];
+    if (cregs->special_ops && cregs->special_ops->write) {
+        cregs->special_ops->write(cxl_cstate, offset, value, size);
+    } else {
+        cregs->cache_mem_registers[offset / sizeof(*cregs->cache_mem_registers)] = value;
+    }
+}
+
+/*
+ * 8.2.3
+ *   The access restrictions specified in Section 8.2.2 also apply to CXL 2.0
+ *   Component Registers.
+ *
+ * 8.2.2
+ *   • A 32 bit register shall be accessed as a 4 Bytes quantity. Partial
+ *   reads are not permitted.
+ *   • A 64 bit register shall be accessed as a 8 Bytes quantity. Partial
+ *   reads are not permitted.
+ *
+ * As of the spec defined today, only 4 byte registers exist.
+ */
+static const MemoryRegionOps cache_mem_ops = {
+    .read = cxl_cache_mem_read_reg,
+    .write = cxl_cache_mem_write_reg,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 4,
+        .max_access_size = 8,
+        .unaligned = false,
+    },
+    .impl = {
+        .min_access_size = 4,
+        .max_access_size = 8,
+    },
+};
+
+void cxl_component_register_block_init(Object *obj,
+                                       CXLComponentState *cxl_cstate,
+                                       const char *type)
+{
+    ComponentRegisters *cregs = &cxl_cstate->crb;
+
+    memory_region_init(&cregs->component_registers, obj, type,
+                       CXL2_COMPONENT_BLOCK_SIZE);
+
+    /* io registers controls link which we don't care about in QEMU */
+    memory_region_init_io(&cregs->io, obj, NULL, cregs, ".io",
+                          CXL2_COMPONENT_IO_REGION_SIZE);
+    memory_region_init_io(&cregs->cache_mem, obj, &cache_mem_ops, cregs,
+                          ".cache_mem", CXL2_COMPONENT_CM_REGION_SIZE);
+
+    memory_region_add_subregion(&cregs->component_registers, 0, &cregs->io);
+    memory_region_add_subregion(&cregs->component_registers,
+                                CXL2_COMPONENT_IO_REGION_SIZE,
+                                &cregs->cache_mem);
+}
+
+static void ras_init_common(uint32_t *reg_state, uint32_t *write_msk)
+{
+    /*
+     * Error status is RW1C but given bits are not yet set, it can
+     * be handled as RO.
+     */
+    reg_state[R_CXL_RAS_UNC_ERR_STATUS] = 0;
+    /* Bits 12-13 and 17-31 reserved in CXL 2.0 */
+    reg_state[R_CXL_RAS_UNC_ERR_MASK] = 0x1cfff;
+    write_msk[R_CXL_RAS_UNC_ERR_MASK] = 0x1cfff;
+    reg_state[R_CXL_RAS_UNC_ERR_SEVERITY] = 0x1cfff;
+    write_msk[R_CXL_RAS_UNC_ERR_SEVERITY] = 0x1cfff;
+    reg_state[R_CXL_RAS_COR_ERR_STATUS] = 0;
+    reg_state[R_CXL_RAS_COR_ERR_MASK] = 0x7f;
+    write_msk[R_CXL_RAS_COR_ERR_MASK] = 0x7f;
+    /* CXL switches and devices must set */
+    reg_state[R_CXL_RAS_ERR_CAP_CTRL] = 0x00;
+}
+
+static void hdm_init_common(uint32_t *reg_state, uint32_t *write_msk)
+{
+    int decoder_count = 1;
+    int i;
+
+    ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_CAPABILITY, DECODER_COUNT,
+                     cxl_decoder_count_enc(decoder_count));
+    ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_CAPABILITY, TARGET_COUNT, 1);
+    ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_CAPABILITY, INTERLEAVE_256B, 1);
+    ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_CAPABILITY, INTERLEAVE_4K, 1);
+    ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_CAPABILITY, POISON_ON_ERR_CAP, 0);
+    ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_GLOBAL_CONTROL,
+                     HDM_DECODER_ENABLE, 0);
+    write_msk[R_CXL_HDM_DECODER_GLOBAL_CONTROL] = 0x3;
+    for (i = 0; i < decoder_count; i++) {
+        write_msk[R_CXL_HDM_DECODER0_BASE_LO + i * 0x20] = 0xf0000000;
+        write_msk[R_CXL_HDM_DECODER0_BASE_HI + i * 0x20] = 0xffffffff;
+        write_msk[R_CXL_HDM_DECODER0_SIZE_LO + i * 0x20] = 0xf0000000;
+        write_msk[R_CXL_HDM_DECODER0_SIZE_HI + i * 0x20] = 0xffffffff;
+        write_msk[R_CXL_HDM_DECODER0_CTRL + i * 0x20] = 0x13ff;
+    }
+}
+
+void cxl_component_register_init_common(uint32_t *reg_state, uint32_t *write_msk,
+                                        enum reg_type type)
+{
+    int caps = 0;
+
+    /*
+     * In CXL 2.0 the capabilities required for each CXL component are such that,
+     * with the ordering chosen here, a single number can be used to define
+     * which capabilities should be provided.
+     */
+    switch (type) {
+    case CXL2_DOWNSTREAM_PORT:
+    case CXL2_DEVICE:
+        /* RAS, Link */
+        caps = 2;
+        break;
+    case CXL2_UPSTREAM_PORT:
+    case CXL2_TYPE3_DEVICE:
+    case CXL2_LOGICAL_DEVICE:
+        /* + HDM */
+        caps = 3;
+        break;
+    case CXL2_ROOT_PORT:
+        /* + Extended Security, + Snoop */
+        caps = 5;
+        break;
+    default:
+        abort();
+    }
+
+    memset(reg_state, 0, CXL2_COMPONENT_CM_REGION_SIZE);
+
+    /* CXL Capability Header Register */
+    ARRAY_FIELD_DP32(reg_state, CXL_CAPABILITY_HEADER, ID, 1);
+    ARRAY_FIELD_DP32(reg_state, CXL_CAPABILITY_HEADER, VERSION, 1);
+    ARRAY_FIELD_DP32(reg_state, CXL_CAPABILITY_HEADER, CACHE_MEM_VERSION, 1);
+    ARRAY_FIELD_DP32(reg_state, CXL_CAPABILITY_HEADER, ARRAY_SIZE, caps);
+
+#define init_cap_reg(reg, id, version)                                        \
+    QEMU_BUILD_BUG_ON(CXL_##reg##_REGISTERS_OFFSET == 0);                     \
+    do {                                                                      \
+        int which = R_CXL_##reg##_CAPABILITY_HEADER;                          \
+        reg_state[which] = FIELD_DP32(reg_state[which],                       \
+                                      CXL_##reg##_CAPABILITY_HEADER, ID, id); \
+        reg_state[which] =                                                    \
+            FIELD_DP32(reg_state[which], CXL_##reg##_CAPABILITY_HEADER,       \
+                       VERSION, version);                                     \
+        reg_state[which] =                                                    \
+            FIELD_DP32(reg_state[which], CXL_##reg##_CAPABILITY_HEADER, PTR,  \
+                       CXL_##reg##_REGISTERS_OFFSET);                         \
+    } while (0)
+
+    init_cap_reg(RAS, 2, 2);
+    ras_init_common(reg_state, write_msk);
+
+    init_cap_reg(LINK, 4, 2);
+
+    if (caps < 3) {
+        return;
+    }
+
+    init_cap_reg(HDM, 5, 1);
+    hdm_init_common(reg_state, write_msk);
+
+    if (caps < 5) {
+        return;
+    }
+
+    init_cap_reg(EXTSEC, 6, 1);
+    init_cap_reg(SNOOP, 8, 1);
+
+#undef init_cap_reg
+}
+
+/*
+ * Helper to creates a DVSEC header for a CXL entity. The caller is responsible
+ * for tracking the valid offset.
+ *
+ * This function will build the DVSEC header on behalf of the caller and then
+ * copy in the remaining data for the vendor specific bits.
+ * It will also set up appropriate write masks.
+ */
+void cxl_component_create_dvsec(CXLComponentState *cxl,
+                                enum reg_type cxl_dev_type, uint16_t length,
+                                uint16_t type, uint8_t rev, uint8_t *body)
+{
+    PCIDevice *pdev = cxl->pdev;
+    uint16_t offset = cxl->dvsec_offset;
+    uint8_t *wmask = pdev->wmask;
+
+    assert(offset >= PCI_CFG_SPACE_SIZE &&
+           ((offset + length) < PCI_CFG_SPACE_EXP_SIZE));
+    assert((length & 0xf000) == 0);
+    assert((rev & ~0xf) == 0);
+
+    /* Create the DVSEC in the MCFG space */
+    pcie_add_capability(pdev, PCI_EXT_CAP_ID_DVSEC, 1, offset, length);
+    pci_set_long(pdev->config + offset + PCIE_DVSEC_HEADER1_OFFSET,
+                 (length << 20) | (rev << 16) | CXL_VENDOR_ID);
+    pci_set_word(pdev->config + offset + PCIE_DVSEC_ID_OFFSET, type);
+    memcpy(pdev->config + offset + sizeof(DVSECHeader),
+           body + sizeof(DVSECHeader),
+           length - sizeof(DVSECHeader));
+
+    /* Configure write masks */
+    switch (type) {
+    case PCIE_CXL_DEVICE_DVSEC:
+        break;
+    case NON_CXL_FUNCTION_MAP_DVSEC:
+        break; /* Not yet implemented */
+    case EXTENSIONS_PORT_DVSEC:
+        wmask[offset + offsetof(CXLDVSECPortExtensions, control)] = 0x0F;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, control) + 1] = 0x40;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_bus_base)] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_bus_limit)] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_memory_base)] = 0xF0;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_memory_base) + 1] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_memory_limit)] = 0xF0;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_memory_limit) + 1] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_prefetch_base)] = 0xF0;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_prefetch_base) + 1] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_prefetch_limit)] = 0xF0;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_prefetch_limit) + 1] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_prefetch_base_high)] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_prefetch_base_high) + 1] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_prefetch_base_high) + 2] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_prefetch_base_high) + 3] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_prefetch_limit_high)] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_prefetch_limit_high) + 1] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_prefetch_limit_high) + 2] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECPortExtensions, alt_prefetch_limit_high) + 3] = 0xFF;
+        break;
+    case GPF_PORT_DVSEC:
+        wmask[offset + offsetof(CXLDVSECPortGPF, phase1_ctrl)] = 0x0F;
+        wmask[offset + offsetof(CXLDVSECPortGPF, phase1_ctrl) + 1] = 0x0F;
+        wmask[offset + offsetof(CXLDVSECPortGPF, phase2_ctrl)] = 0x0F;
+        wmask[offset + offsetof(CXLDVSECPortGPF, phase2_ctrl) + 1] = 0x0F;
+        break;
+    case GPF_DEVICE_DVSEC:
+        wmask[offset + offsetof(CXLDVSECDeviceGPF, phase2_duration)] = 0x0F;
+        wmask[offset + offsetof(CXLDVSECDeviceGPF, phase2_duration) + 1] = 0x0F;
+        wmask[offset + offsetof(CXLDVSECDeviceGPF, phase2_power)] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECDeviceGPF, phase2_power) + 1] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECDeviceGPF, phase2_power) + 2] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECDeviceGPF, phase2_power) + 3] = 0xFF;
+        break;
+    case PCIE_FLEXBUS_PORT_DVSEC:
+        switch (cxl_dev_type) {
+        case CXL2_ROOT_PORT:
+            /* No MLD */
+            wmask[offset + offsetof(CXLDVSECPortFlexBus, ctrl)] = 0xbd;
+            break;
+        case CXL2_DOWNSTREAM_PORT:
+            wmask[offset + offsetof(CXLDVSECPortFlexBus, ctrl)] = 0xfd;
+            break;
+        default: /* Registers are RO for other component types */
+            break;
+        }
+        /* There are rw1cs bits in the status register but never set currently */
+        break;
+    }
+
+    /* Update state for future DVSEC additions */
+    range_init_nofail(&cxl->dvsecs[type], cxl->dvsec_offset, length);
+    cxl->dvsec_offset += length;
+}
diff --git a/hw/cxl/meson.build b/hw/cxl/meson.build
new file mode 100644
index 0000000000..3231b5de1e
--- /dev/null
+++ b/hw/cxl/meson.build
@@ -0,0 +1,4 @@
+softmmu_ss.add(when: 'CONFIG_CXL',
+               if_true: files(
+                   'cxl-component-utils.c',
+               ))
diff --git a/hw/meson.build b/hw/meson.build
index b3366c888e..9992c5101e 100644
--- a/hw/meson.build
+++ b/hw/meson.build
@@ -6,6 +6,7 @@ subdir('block')
 subdir('char')
 subdir('core')
 subdir('cpu')
+subdir('cxl')
 subdir('display')
 subdir('dma')
 subdir('gpio')
diff --git a/include/hw/cxl/cxl.h b/include/hw/cxl/cxl.h
new file mode 100644
index 0000000000..8c738c7a2b
--- /dev/null
+++ b/include/hw/cxl/cxl.h
@@ -0,0 +1,16 @@
+/*
+ * QEMU CXL Support
+ *
+ * Copyright (c) 2020 Intel
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See the
+ * COPYING file in the top-level directory.
+ */
+
+#ifndef CXL_H
+#define CXL_H
+
+#include "cxl_pci.h"
+#include "cxl_component.h"
+
+#endif
diff --git a/include/hw/cxl/cxl_component.h b/include/hw/cxl/cxl_component.h
new file mode 100644
index 0000000000..7d8f395cbe
--- /dev/null
+++ b/include/hw/cxl/cxl_component.h
@@ -0,0 +1,213 @@
+/*
+ * QEMU CXL Component
+ *
+ * Copyright (c) 2020 Intel
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See the
+ * COPYING file in the top-level directory.
+ */
+
+#ifndef CXL_COMPONENT_H
+#define CXL_COMPONENT_H
+
+/* CXL 2.0 - 8.2.4 */
+#define CXL2_COMPONENT_IO_REGION_SIZE 0x1000
+#define CXL2_COMPONENT_CM_REGION_SIZE 0x1000
+#define CXL2_COMPONENT_BLOCK_SIZE 0x10000
+
+#include "qemu/compiler.h"
+#include "qemu/range.h"
+#include "qemu/typedefs.h"
+#include "hw/register.h"
+
+enum reg_type {
+    CXL2_DEVICE,
+    CXL2_TYPE3_DEVICE,
+    CXL2_LOGICAL_DEVICE,
+    CXL2_ROOT_PORT,
+    CXL2_UPSTREAM_PORT,
+    CXL2_DOWNSTREAM_PORT
+};
+
+/*
+ * Capability registers are defined at the top of the CXL.cache/mem region and
+ * are packed. For our purposes we will always define the caps in the same
+ * order.
+ * CXL 2.0 - 8.2.5 Table 142 for details.
+ */
+
+/* CXL 2.0 - 8.2.5.1 */
+REG32(CXL_CAPABILITY_HEADER, 0)
+    FIELD(CXL_CAPABILITY_HEADER, ID, 0, 16)
+    FIELD(CXL_CAPABILITY_HEADER, VERSION, 16, 4)
+    FIELD(CXL_CAPABILITY_HEADER, CACHE_MEM_VERSION, 20, 4)
+    FIELD(CXL_CAPABILITY_HEADER, ARRAY_SIZE, 24, 8)
+
+#define CXLx_CAPABILITY_HEADER(type, offset)                  \
+    REG32(CXL_##type##_CAPABILITY_HEADER, offset)             \
+        FIELD(CXL_##type##_CAPABILITY_HEADER, ID, 0, 16)      \
+        FIELD(CXL_##type##_CAPABILITY_HEADER, VERSION, 16, 4) \
+        FIELD(CXL_##type##_CAPABILITY_HEADER, PTR, 20, 12)
+CXLx_CAPABILITY_HEADER(RAS, 0x4)
+CXLx_CAPABILITY_HEADER(LINK, 0x8)
+CXLx_CAPABILITY_HEADER(HDM, 0xc)
+CXLx_CAPABILITY_HEADER(EXTSEC, 0x10)
+CXLx_CAPABILITY_HEADER(SNOOP, 0x14)
+
+/*
+ * Capability structures contain the actual registers that the CXL component
+ * implements. Some of these are specific to certain types of components, but
+ * this implementation leaves enough space regardless.
+ */
+/* 8.2.5.9 - CXL RAS Capability Structure */
+
+/* Give ample space for caps before this */
+#define CXL_RAS_REGISTERS_OFFSET 0x80
+#define CXL_RAS_REGISTERS_SIZE   0x58
+REG32(CXL_RAS_UNC_ERR_STATUS, CXL_RAS_REGISTERS_OFFSET)
+REG32(CXL_RAS_UNC_ERR_MASK, CXL_RAS_REGISTERS_OFFSET + 0x4)
+REG32(CXL_RAS_UNC_ERR_SEVERITY, CXL_RAS_REGISTERS_OFFSET + 0x8)
+REG32(CXL_RAS_COR_ERR_STATUS, CXL_RAS_REGISTERS_OFFSET + 0xc)
+REG32(CXL_RAS_COR_ERR_MASK, CXL_RAS_REGISTERS_OFFSET + 0x10)
+REG32(CXL_RAS_ERR_CAP_CTRL, CXL_RAS_REGISTERS_OFFSET + 0x14)
+/* Offset 0x18 - 0x58 reserved for RAS logs */
+
+/* 8.2.5.10 - CXL Security Capability Structure */
+#define CXL_SEC_REGISTERS_OFFSET \
+    (CXL_RAS_REGISTERS_OFFSET + CXL_RAS_REGISTERS_SIZE)
+#define CXL_SEC_REGISTERS_SIZE   0 /* We don't implement 1.1 downstream ports */
+
+/* 8.2.5.11 - CXL Link Capability Structure */
+#define CXL_LINK_REGISTERS_OFFSET \
+    (CXL_SEC_REGISTERS_OFFSET + CXL_SEC_REGISTERS_SIZE)
+#define CXL_LINK_REGISTERS_SIZE   0x38
+
+/* 8.2.5.12 - CXL HDM Decoder Capability Structure */
+#define HDM_DECODE_MAX 10 /* 8.2.5.12.1 */
+#define CXL_HDM_REGISTERS_OFFSET \
+    (CXL_LINK_REGISTERS_OFFSET + CXL_LINK_REGISTERS_SIZE)
+#define CXL_HDM_REGISTERS_SIZE (0x10 + 0x20 * HDM_DECODE_MAX)
+#define HDM_DECODER_INIT(n)                                                    \
+  REG32(CXL_HDM_DECODER##n##_BASE_LO,                                          \
+        CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x10)                          \
+            FIELD(CXL_HDM_DECODER##n##_BASE_LO, L, 28, 4)                      \
+  REG32(CXL_HDM_DECODER##n##_BASE_HI,                                          \
+        CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x14)                          \
+  REG32(CXL_HDM_DECODER##n##_SIZE_LO,                                          \
+        CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x18)                          \
+  REG32(CXL_HDM_DECODER##n##_SIZE_HI,                                          \
+        CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x1C)                          \
+  REG32(CXL_HDM_DECODER##n##_CTRL,                                             \
+        CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x20)                          \
+            FIELD(CXL_HDM_DECODER##n##_CTRL, IG, 0, 4)                         \
+            FIELD(CXL_HDM_DECODER##n##_CTRL, IW, 4, 4)                         \
+            FIELD(CXL_HDM_DECODER##n##_CTRL, LOCK_ON_COMMIT, 8, 1)             \
+            FIELD(CXL_HDM_DECODER##n##_CTRL, COMMIT, 9, 1)                     \
+            FIELD(CXL_HDM_DECODER##n##_CTRL, COMMITTED, 10, 1)                 \
+            FIELD(CXL_HDM_DECODER##n##_CTRL, ERR, 11, 1)                       \
+            FIELD(CXL_HDM_DECODER##n##_CTRL, TYPE, 12, 1)                      \
+  REG32(CXL_HDM_DECODER##n##_TARGET_LIST_LO,                                   \
+        CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x24)                          \
+  REG32(CXL_HDM_DECODER##n##_TARGET_LIST_HI,                                   \
+        CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x28)
+
+REG32(CXL_HDM_DECODER_CAPABILITY, CXL_HDM_REGISTERS_OFFSET)
+    FIELD(CXL_HDM_DECODER_CAPABILITY, DECODER_COUNT, 0, 4)
+    FIELD(CXL_HDM_DECODER_CAPABILITY, TARGET_COUNT, 4, 4)
+    FIELD(CXL_HDM_DECODER_CAPABILITY, INTERLEAVE_256B, 8, 1)
+    FIELD(CXL_HDM_DECODER_CAPABILITY, INTERLEAVE_4K, 9, 1)
+    FIELD(CXL_HDM_DECODER_CAPABILITY, POISON_ON_ERR_CAP, 10, 1)
+REG32(CXL_HDM_DECODER_GLOBAL_CONTROL, CXL_HDM_REGISTERS_OFFSET + 4)
+    FIELD(CXL_HDM_DECODER_GLOBAL_CONTROL, POISON_ON_ERR_EN, 0, 1)
+    FIELD(CXL_HDM_DECODER_GLOBAL_CONTROL, HDM_DECODER_ENABLE, 1, 1)
+
+HDM_DECODER_INIT(0);
+
+/* 8.2.5.13 - CXL Extended Security Capability Structure (Root complex only) */
+#define EXTSEC_ENTRY_MAX        256
+#define CXL_EXTSEC_REGISTERS_OFFSET \
+    (CXL_HDM_REGISTERS_OFFSET + CXL_HDM_REGISTERS_SIZE)
+#define CXL_EXTSEC_REGISTERS_SIZE   (8 * EXTSEC_ENTRY_MAX + 4)
+
+/* 8.2.5.14 - CXL IDE Capability Structure */
+#define CXL_IDE_REGISTERS_OFFSET \
+    (CXL_EXTSEC_REGISTERS_OFFSET + CXL_EXTSEC_REGISTERS_SIZE)
+#define CXL_IDE_REGISTERS_SIZE   0x20
+
+/* 8.2.5.15 - CXL Snoop Filter Capability Structure */
+#define CXL_SNOOP_REGISTERS_OFFSET \
+    (CXL_IDE_REGISTERS_OFFSET + CXL_IDE_REGISTERS_SIZE)
+#define CXL_SNOOP_REGISTERS_SIZE   0x8
+
+QEMU_BUILD_BUG_MSG((CXL_SNOOP_REGISTERS_OFFSET + CXL_SNOOP_REGISTERS_SIZE) >= 0x1000,
+                   "No space for registers");
+
+typedef struct component_registers {
+    /*
+     * Main memory region to be registered with QEMU core.
+     */
+    MemoryRegion component_registers;
+
+    /*
+     * 8.2.4 Table 141:
+     *   0x0000 - 0x0fff CXL.io registers
+     *   0x1000 - 0x1fff CXL.cache and CXL.mem
+     *   0x2000 - 0xdfff Implementation specific
+     *   0xe000 - 0xe3ff CXL ARB/MUX registers
+     *   0xe400 - 0xffff RSVD
+     */
+    uint32_t io_registers[CXL2_COMPONENT_IO_REGION_SIZE >> 2];
+    MemoryRegion io;
+
+    uint32_t cache_mem_registers[CXL2_COMPONENT_CM_REGION_SIZE >> 2];
+    uint32_t cache_mem_regs_write_mask[CXL2_COMPONENT_CM_REGION_SIZE >> 2];
+    MemoryRegion cache_mem;
+
+    MemoryRegion impl_specific;
+    MemoryRegion arb_mux;
+    MemoryRegion rsvd;
+
+    /* special_ops is used for any component that needs any specific handling */
+    MemoryRegionOps *special_ops;
+} ComponentRegisters;
+
+/*
+ * A CXL component represents all entities in a CXL hierarchy. This includes,
+ * host bridges, root ports, upstream/downstream switch ports, and devices
+ */
+typedef struct cxl_component {
+    ComponentRegisters crb;
+    union {
+        struct {
+            Range dvsecs[CXL20_MAX_DVSEC];
+            uint16_t dvsec_offset;
+            struct PCIDevice *pdev;
+        };
+    };
+} CXLComponentState;
+
+void cxl_component_register_block_init(Object *obj,
+                                       CXLComponentState *cxl_cstate,
+                                       const char *type);
+void cxl_component_register_init_common(uint32_t *reg_state,
+                                        uint32_t *write_msk,
+                                        enum reg_type type);
+
+void cxl_component_create_dvsec(CXLComponentState *cxl_cstate,
+                                enum reg_type cxl_dev_type, uint16_t length,
+                                uint16_t type, uint8_t rev, uint8_t *body);
+
+static inline int cxl_decoder_count_enc(int count)
+{
+    switch (count) {
+    case 1: return 0;
+    case 2: return 1;
+    case 4: return 2;
+    case 6: return 3;
+    case 8: return 4;
+    case 10: return 5;
+    }
+    return 0;
+}
+
+#endif
diff --git a/include/hw/cxl/cxl_pci.h b/include/hw/cxl/cxl_pci.h
new file mode 100644
index 0000000000..3f7e44daac
--- /dev/null
+++ b/include/hw/cxl/cxl_pci.h
@@ -0,0 +1,146 @@
+/*
+ * QEMU CXL PCI interfaces
+ *
+ * Copyright (c) 2020 Intel
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See the
+ * COPYING file in the top-level directory.
+ */
+
+#ifndef CXL_PCI_H
+#define CXL_PCI_H
+
+#include "qemu/compiler.h"
+#include "hw/pci/pci.h"
+#include "hw/pci/pcie.h"
+
+#define CXL_VENDOR_ID 0x1e98
+
+#define PCIE_DVSEC_HEADER1_OFFSET 0x4 /* Offset from start of extend cap */
+#define PCIE_DVSEC_ID_OFFSET 0x8
+
+#define PCIE_CXL_DEVICE_DVSEC_LENGTH 0x38
+#define PCIE_CXL1_DEVICE_DVSEC_REVID 0
+#define PCIE_CXL2_DEVICE_DVSEC_REVID 1
+
+#define EXTENSIONS_PORT_DVSEC_LENGTH 0x28
+#define EXTENSIONS_PORT_DVSEC_REVID 0
+
+#define GPF_PORT_DVSEC_LENGTH 0x10
+#define GPF_PORT_DVSEC_REVID  0
+
+#define GPF_DEVICE_DVSEC_LENGTH 0x10
+#define GPF_DEVICE_DVSEC_REVID 0
+
+#define PCIE_FLEXBUS_PORT_DVSEC_LENGTH_2_0 0x14
+#define PCIE_FLEXBUS_PORT_DVSEC_REVID_2_0  1
+
+#define REG_LOC_DVSEC_LENGTH 0x24
+#define REG_LOC_DVSEC_REVID  0
+
+enum {
+    PCIE_CXL_DEVICE_DVSEC      = 0,
+    NON_CXL_FUNCTION_MAP_DVSEC = 2,
+    EXTENSIONS_PORT_DVSEC      = 3,
+    GPF_PORT_DVSEC             = 4,
+    GPF_DEVICE_DVSEC           = 5,
+    PCIE_FLEXBUS_PORT_DVSEC    = 7,
+    REG_LOC_DVSEC              = 8,
+    MLD_DVSEC                  = 9,
+    CXL20_MAX_DVSEC
+};
+
+typedef struct DVSECHeader {
+    uint32_t cap_hdr;
+    uint32_t dv_hdr1;
+    uint16_t dv_hdr2;
+} QEMU_PACKED DVSECHeader;
+QEMU_BUILD_BUG_ON(sizeof(DVSECHeader) != 10);
+
+/*
+ * CXL 2.0 devices must implement certain DVSEC IDs, and can [optionally]
+ * implement others.
+ *
+ * CXL 2.0 Device: 0, [2], 5, 8
+ * CXL 2.0 RP: 3, 4, 7, 8
+ * CXL 2.0 Upstream Port: [2], 7, 8
+ * CXL 2.0 Downstream Port: 3, 4, 7, 8
+ */
+
+/* CXL 2.0 - 8.1.5 (ID 0003) */
+typedef struct CXLDVSECPortExtensions {
+    DVSECHeader hdr;
+    uint16_t status;
+    uint16_t control;
+    uint8_t alt_bus_base;
+    uint8_t alt_bus_limit;
+    uint16_t alt_memory_base;
+    uint16_t alt_memory_limit;
+    uint16_t alt_prefetch_base;
+    uint16_t alt_prefetch_limit;
+    uint32_t alt_prefetch_base_high;
+    uint32_t alt_prefetch_limit_high;
+    uint32_t rcrb_base;
+    uint32_t rcrb_base_high;
+} CXLDVSECPortExtensions;
+QEMU_BUILD_BUG_ON(sizeof(CXLDVSECPortExtensions) != 0x28);
+
+#define PORT_CONTROL_OFFSET          0xc
+#define PORT_CONTROL_UNMASK_SBR      1
+#define PORT_CONTROL_ALT_MEMID_EN    4
+
+/* CXL 2.0 - 8.1.6 GPF DVSEC (ID 0004) */
+typedef struct CXLDVSECPortGPF {
+    DVSECHeader hdr;
+    uint16_t rsvd;
+    uint16_t phase1_ctrl;
+    uint16_t phase2_ctrl;
+} CXLDVSECPortGPF;
+QEMU_BUILD_BUG_ON(sizeof(CXLDVSECPortGPF) != 0x10);
+
+/* CXL 2.0 - 8.1.7 GPF DVSEC for CXL Device */
+typedef struct CXLDVSECDeviceGPF {
+    DVSECHeader hdr;
+    uint16_t phase2_duration;
+    uint32_t phase2_power;
+} CXLDVSECDeviceGPF;
+QEMU_BUILD_BUG_ON(sizeof(CXLDVSECDeviceGPF) != 0x10);
+
+/* CXL 2.0 - 8.1.8/8.2.1.3 Flex Bus DVSEC (ID 0007) */
+typedef struct CXLDVSECPortFlexBus {
+    DVSECHeader hdr;
+    uint16_t cap;
+    uint16_t ctrl;
+    uint16_t status;
+    uint32_t rcvd_mod_ts_data_phase1;
+} CXLDVSECPortFlexBus;
+QEMU_BUILD_BUG_ON(sizeof(CXLDVSECPortFlexBus) != 0x14);
+
+/* CXL 2.0 - 8.1.9 Register Locator DVSEC (ID 0008) */
+typedef struct CXLDVSECRegisterLocator {
+    DVSECHeader hdr;
+    uint16_t rsvd;
+    uint32_t reg0_base_lo;
+    uint32_t reg0_base_hi;
+    uint32_t reg1_base_lo;
+    uint32_t reg1_base_hi;
+    uint32_t reg2_base_lo;
+    uint32_t reg2_base_hi;
+} CXLDVSECRegisterLocator;
+QEMU_BUILD_BUG_ON(sizeof(CXLDVSECRegisterLocator) != 0x24);
+
+/* BAR Equivalence Indicator */
+#define BEI_BAR_10H 0
+#define BEI_BAR_14H 1
+#define BEI_BAR_18H 2
+#define BEI_BAR_1cH 3
+#define BEI_BAR_20H 4
+#define BEI_BAR_24H 5
+
+/* Register Block Identifier */
+#define RBI_EMPTY          0
+#define RBI_COMPONENT_REG  (1 << 8)
+#define RBI_BAR_VIRT_ACL   (2 << 8)
+#define RBI_CXL_DEVICE_REG (3 << 8)
+
+#endif

From 502730ee3cc3acbf421f07c26a15652f4929e309 Mon Sep 17 00:00:00 2001
From: Jonathan Cameron <jonathan.cameron@huawei.com>
Date: Fri, 29 Apr 2022 15:40:28 +0100
Subject: [PATCH 272/935] MAINTAINERS: Add entry for Compute Express Link
 Emulation
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The CXL emulation will be jointly maintained by Ben Widawsky
and Jonathan Cameron.  Broken out as a separate patch
to improve visibility.

Signed-off-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-4-Jonathan.Cameron@huawei.com>
---
 MAINTAINERS | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index fbc0662627..dff0200f70 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2545,6 +2545,13 @@ F: qapi/block*.json
 F: qapi/transaction.json
 T: git https://repo.or.cz/qemu/armbru.git block-next
 
+Compute Express Link
+M: Ben Widawsky <ben.widawsky@intel.com>
+M: Jonathan Cameron <jonathan.cameron@huawei.com>
+S: Supported
+F: hw/cxl/
+F: include/hw/cxl/
+
 Dirty Bitmaps
 M: Eric Blake <eblake@redhat.com>
 M: Vladimir Sementsov-Ogievskiy <v.sementsov-og@mail.ru>

From cd90126b4ced427697d79eb5002544a7621ec647 Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:29 +0100
Subject: [PATCH 273/935] hw/cxl/device: Introduce a CXL device (8.2.8)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

A CXL device is a type of CXL component. Conceptually, a CXL device
would be a leaf node in a CXL topology. From an emulation perspective,
CXL devices are the most complex and so the actual implementation is
reserved for discrete commits.

This new device type is specifically catered towards the eventual
implementation of a Type3 CXL.mem device, 8.2.8.5 in the CXL 2.0
specification.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Adam Manzanares <a.manzanares@samsung.com>
Message-Id: <20220429144110.25167-5-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/cxl/cxl.h        |   1 +
 include/hw/cxl/cxl_device.h | 166 ++++++++++++++++++++++++++++++++++++
 2 files changed, 167 insertions(+)
 create mode 100644 include/hw/cxl/cxl_device.h

diff --git a/include/hw/cxl/cxl.h b/include/hw/cxl/cxl.h
index 8c738c7a2b..b9d1ac3fad 100644
--- a/include/hw/cxl/cxl.h
+++ b/include/hw/cxl/cxl.h
@@ -12,5 +12,6 @@
 
 #include "cxl_pci.h"
 #include "cxl_component.h"
+#include "cxl_device.h"
 
 #endif
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
new file mode 100644
index 0000000000..9513aaac77
--- /dev/null
+++ b/include/hw/cxl/cxl_device.h
@@ -0,0 +1,166 @@
+/*
+ * QEMU CXL Devices
+ *
+ * Copyright (c) 2020 Intel
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See the
+ * COPYING file in the top-level directory.
+ */
+
+#ifndef CXL_DEVICE_H
+#define CXL_DEVICE_H
+
+#include "hw/register.h"
+
+/*
+ * The following is how a CXL device's Memory Device registers are laid out.
+ * The only requirement from the spec is that the capabilities array and the
+ * capability headers start at offset 0 and are contiguously packed. The headers
+ * themselves provide offsets to the register fields. For this emulation, the
+ * actual registers  * will start at offset 0x80 (m == 0x80). No secondary
+ * mailbox is implemented which means that the offset of the start of the
+ * mailbox payload (n) is given by
+ * n = m + sizeof(mailbox registers) + sizeof(device registers).
+ *
+ *                       +---------------------------------+
+ *                       |                                 |
+ *                       |    Memory Device Registers      |
+ *                       |                                 |
+ * n + PAYLOAD_SIZE_MAX  -----------------------------------
+ *                  ^    |                                 |
+ *                  |    |                                 |
+ *                  |    |                                 |
+ *                  |    |                                 |
+ *                  |    |                                 |
+ *                  |    |         Mailbox Payload         |
+ *                  |    |                                 |
+ *                  |    |                                 |
+ *                  |    |                                 |
+ *                  n    -----------------------------------
+ *                  ^    |       Mailbox Registers         |
+ *                  |    |                                 |
+ *                  |    -----------------------------------
+ *                  |    |                                 |
+ *                  |    |        Device Registers         |
+ *                  |    |                                 |
+ *                  m    ---------------------------------->
+ *                  ^    |  Memory Device Capability Header|
+ *                  |    -----------------------------------
+ *                  |    |     Mailbox Capability Header   |
+ *                  |    -----------------------------------
+ *                  |    |     Device Capability Header    |
+ *                  |    -----------------------------------
+ *                  |    |     Device Cap Array Register   |
+ *                  0    +---------------------------------+
+ *
+ */
+
+#define CXL_DEVICE_CAP_HDR1_OFFSET 0x10 /* Figure 138 */
+#define CXL_DEVICE_CAP_REG_SIZE 0x10 /* 8.2.8.2 */
+#define CXL_DEVICE_CAPS_MAX 4 /* 8.2.8.2.1 + 8.2.8.5 */
+
+#define CXL_DEVICE_STATUS_REGISTERS_OFFSET 0x80 /* Read comment above */
+#define CXL_DEVICE_STATUS_REGISTERS_LENGTH 0x8 /* 8.2.8.3.1 */
+
+#define CXL_MAILBOX_REGISTERS_OFFSET \
+    (CXL_DEVICE_STATUS_REGISTERS_OFFSET + CXL_DEVICE_STATUS_REGISTERS_LENGTH)
+#define CXL_MAILBOX_REGISTERS_SIZE 0x20 /* 8.2.8.4, Figure 139 */
+#define CXL_MAILBOX_PAYLOAD_SHIFT 11
+#define CXL_MAILBOX_MAX_PAYLOAD_SIZE (1 << CXL_MAILBOX_PAYLOAD_SHIFT)
+#define CXL_MAILBOX_REGISTERS_LENGTH \
+    (CXL_MAILBOX_REGISTERS_SIZE + CXL_MAILBOX_MAX_PAYLOAD_SIZE)
+
+typedef struct cxl_device_state {
+    MemoryRegion device_registers;
+
+    /* mmio for device capabilities array - 8.2.8.2 */
+    MemoryRegion device;
+    MemoryRegion caps;
+
+    /* mmio for the mailbox registers 8.2.8.4 */
+    MemoryRegion mailbox;
+
+    /* memory region for persistent memory, HDM */
+    uint64_t pmem_size;
+} CXLDeviceState;
+
+/* Initialize the register block for a device */
+void cxl_device_register_block_init(Object *obj, CXLDeviceState *dev);
+
+/* Set up default values for the register block */
+void cxl_device_register_init_common(CXLDeviceState *dev);
+
+/*
+ * CXL 2.0 - 8.2.8.1 including errata F4
+ * Documented as a 128 bit register, but 64 bit accesses and the second
+ * 64 bits are currently reserved.
+ */
+REG64(CXL_DEV_CAP_ARRAY, 0) /* Documented as 128 bit register but 64 byte accesses */
+    FIELD(CXL_DEV_CAP_ARRAY, CAP_ID, 0, 16)
+    FIELD(CXL_DEV_CAP_ARRAY, CAP_VERSION, 16, 8)
+    FIELD(CXL_DEV_CAP_ARRAY, CAP_COUNT, 32, 16)
+
+/*
+ * Helper macro to initialize capability headers for CXL devices.
+ *
+ * In the 8.2.8.2, this is listed as a 128b register, but in 8.2.8, it says:
+ * > No registers defined in Section 8.2.8 are larger than 64-bits wide so that
+ * > is the maximum access size allowed for these registers. If this rule is not
+ * > followed, the behavior is undefined
+ *
+ * CXL 2.0 Errata F4 states futher that the layouts in the specification are
+ * shown as greater than 128 bits, but implementations are expected to
+ * use any size of access up to 64 bits.
+ *
+ * Here we've chosen to make it 4 dwords. The spec allows any pow2 multiple
+ * access to be used for a register up to 64 bits.
+ */
+#define CXL_DEVICE_CAPABILITY_HEADER_REGISTER(n, offset)  \
+    REG32(CXL_DEV_##n##_CAP_HDR0, offset)                 \
+        FIELD(CXL_DEV_##n##_CAP_HDR0, CAP_ID, 0, 16)      \
+        FIELD(CXL_DEV_##n##_CAP_HDR0, CAP_VERSION, 16, 8) \
+    REG32(CXL_DEV_##n##_CAP_HDR1, offset + 4)             \
+        FIELD(CXL_DEV_##n##_CAP_HDR1, CAP_OFFSET, 0, 32)  \
+    REG32(CXL_DEV_##n##_CAP_HDR2, offset + 8)             \
+        FIELD(CXL_DEV_##n##_CAP_HDR2, CAP_LENGTH, 0, 32)
+
+CXL_DEVICE_CAPABILITY_HEADER_REGISTER(DEVICE_STATUS, CXL_DEVICE_CAP_HDR1_OFFSET)
+CXL_DEVICE_CAPABILITY_HEADER_REGISTER(MAILBOX, CXL_DEVICE_CAP_HDR1_OFFSET + \
+                                               CXL_DEVICE_CAP_REG_SIZE)
+
+/* CXL 2.0 8.2.8.4.3 Mailbox Capabilities Register */
+REG32(CXL_DEV_MAILBOX_CAP, 0)
+    FIELD(CXL_DEV_MAILBOX_CAP, PAYLOAD_SIZE, 0, 5)
+    FIELD(CXL_DEV_MAILBOX_CAP, INT_CAP, 5, 1)
+    FIELD(CXL_DEV_MAILBOX_CAP, BG_INT_CAP, 6, 1)
+    FIELD(CXL_DEV_MAILBOX_CAP, MSI_N, 7, 4)
+
+/* CXL 2.0 8.2.8.4.4 Mailbox Control Register */
+REG32(CXL_DEV_MAILBOX_CTRL, 4)
+    FIELD(CXL_DEV_MAILBOX_CTRL, DOORBELL, 0, 1)
+    FIELD(CXL_DEV_MAILBOX_CTRL, INT_EN, 1, 1)
+    FIELD(CXL_DEV_MAILBOX_CTRL, BG_INT_EN, 2, 1)
+
+/* CXL 2.0 8.2.8.4.5 Command Register */
+REG64(CXL_DEV_MAILBOX_CMD, 8)
+    FIELD(CXL_DEV_MAILBOX_CMD, COMMAND, 0, 8)
+    FIELD(CXL_DEV_MAILBOX_CMD, COMMAND_SET, 8, 8)
+    FIELD(CXL_DEV_MAILBOX_CMD, LENGTH, 16, 20)
+
+/* CXL 2.0 8.2.8.4.6 Mailbox Status Register */
+REG64(CXL_DEV_MAILBOX_STS, 0x10)
+    FIELD(CXL_DEV_MAILBOX_STS, BG_OP, 0, 1)
+    FIELD(CXL_DEV_MAILBOX_STS, ERRNO, 32, 16)
+    FIELD(CXL_DEV_MAILBOX_STS, VENDOR_ERRNO, 48, 16)
+
+/* CXL 2.0 8.2.8.4.7 Background Command Status Register */
+REG64(CXL_DEV_BG_CMD_STS, 0x18)
+    FIELD(CXL_DEV_BG_CMD_STS, OP, 0, 16)
+    FIELD(CXL_DEV_BG_CMD_STS, PERCENTAGE_COMP, 16, 7)
+    FIELD(CXL_DEV_BG_CMD_STS, RET_CODE, 32, 16)
+    FIELD(CXL_DEV_BG_CMD_STS, VENDOR_RET_CODE, 48, 16)
+
+/* CXL 2.0 8.2.8.4.8 Command Payload Registers */
+REG32(CXL_DEV_CMD_PAYLOAD, 0x20)
+
+#endif

From 6364adacdfa6a24e3f5b08f6b5ffa789a5d828a7 Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:30 +0100
Subject: [PATCH 274/935] hw/cxl/device: Implement the CAP array (8.2.8.1-2)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This implements all device MMIO up to the first capability. That
includes the CXL Device Capabilities Array Register, as well as all of
the CXL Device Capability Header Registers. The latter are filled in as
they are implemented in the following patches.

Endianness and alignment are managed by softmmu memory core.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-6-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/cxl/cxl-device-utils.c   | 109 ++++++++++++++++++++++++++++++++++++
 hw/cxl/meson.build          |   1 +
 include/hw/cxl/cxl_device.h |  31 +++++++++-
 3 files changed, 140 insertions(+), 1 deletion(-)
 create mode 100644 hw/cxl/cxl-device-utils.c

diff --git a/hw/cxl/cxl-device-utils.c b/hw/cxl/cxl-device-utils.c
new file mode 100644
index 0000000000..241f9f82e3
--- /dev/null
+++ b/hw/cxl/cxl-device-utils.c
@@ -0,0 +1,109 @@
+/*
+ * CXL Utility library for devices
+ *
+ * Copyright(C) 2020 Intel Corporation.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See the
+ * COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "hw/cxl/cxl.h"
+
+/*
+ * Device registers have no restrictions per the spec, and so fall back to the
+ * default memory mapped register rules in 8.2:
+ *   Software shall use CXL.io Memory Read and Write to access memory mapped
+ *   register defined in this section. Unless otherwise specified, software
+ *   shall restrict the accesses width based on the following:
+ *   • A 32 bit register shall be accessed as a 1 Byte, 2 Bytes or 4 Bytes
+ *     quantity.
+ *   • A 64 bit register shall be accessed as a 1 Byte, 2 Bytes, 4 Bytes or 8
+ *     Bytes
+ *   • The address shall be a multiple of the access width, e.g. when
+ *     accessing a register as a 4 Byte quantity, the address shall be
+ *     multiple of 4.
+ *   • The accesses shall map to contiguous bytes.If these rules are not
+ *     followed, the behavior is undefined
+ */
+
+static uint64_t caps_reg_read(void *opaque, hwaddr offset, unsigned size)
+{
+    CXLDeviceState *cxl_dstate = opaque;
+
+    if (size == 4) {
+        return cxl_dstate->caps_reg_state32[offset / sizeof(*cxl_dstate->caps_reg_state32)];
+    } else {
+        return cxl_dstate->caps_reg_state64[offset / sizeof(*cxl_dstate->caps_reg_state64)];
+    }
+}
+
+static uint64_t dev_reg_read(void *opaque, hwaddr offset, unsigned size)
+{
+    return 0;
+}
+
+static const MemoryRegionOps dev_ops = {
+    .read = dev_reg_read,
+    .write = NULL, /* status register is read only */
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+        .unaligned = false,
+    },
+    .impl = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+    },
+};
+
+static const MemoryRegionOps caps_ops = {
+    .read = caps_reg_read,
+    .write = NULL, /* caps registers are read only */
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+        .unaligned = false,
+    },
+    .impl = {
+        .min_access_size = 4,
+        .max_access_size = 8,
+    },
+};
+
+void cxl_device_register_block_init(Object *obj, CXLDeviceState *cxl_dstate)
+{
+    /* This will be a BAR, so needs to be rounded up to pow2 for PCI spec */
+    memory_region_init(&cxl_dstate->device_registers, obj, "device-registers",
+                       pow2ceil(CXL_MMIO_SIZE));
+
+    memory_region_init_io(&cxl_dstate->caps, obj, &caps_ops, cxl_dstate,
+                          "cap-array", CXL_CAPS_SIZE);
+    memory_region_init_io(&cxl_dstate->device, obj, &dev_ops, cxl_dstate,
+                          "device-status", CXL_DEVICE_STATUS_REGISTERS_LENGTH);
+
+    memory_region_add_subregion(&cxl_dstate->device_registers, 0,
+                                &cxl_dstate->caps);
+    memory_region_add_subregion(&cxl_dstate->device_registers,
+                                CXL_DEVICE_STATUS_REGISTERS_OFFSET,
+                                &cxl_dstate->device);
+}
+
+static void device_reg_init_common(CXLDeviceState *cxl_dstate) { }
+
+void cxl_device_register_init_common(CXLDeviceState *cxl_dstate)
+{
+    uint64_t *cap_hdrs = cxl_dstate->caps_reg_state64;
+    const int cap_count = 1;
+
+    /* CXL Device Capabilities Array Register */
+    ARRAY_FIELD_DP64(cap_hdrs, CXL_DEV_CAP_ARRAY, CAP_ID, 0);
+    ARRAY_FIELD_DP64(cap_hdrs, CXL_DEV_CAP_ARRAY, CAP_VERSION, 1);
+    ARRAY_FIELD_DP64(cap_hdrs, CXL_DEV_CAP_ARRAY, CAP_COUNT, cap_count);
+
+    cxl_device_cap_init(cxl_dstate, DEVICE_STATUS, 1);
+    device_reg_init_common(cxl_dstate);
+}
diff --git a/hw/cxl/meson.build b/hw/cxl/meson.build
index 3231b5de1e..dd7c6f8e5a 100644
--- a/hw/cxl/meson.build
+++ b/hw/cxl/meson.build
@@ -1,4 +1,5 @@
 softmmu_ss.add(when: 'CONFIG_CXL',
                if_true: files(
                    'cxl-component-utils.c',
+                   'cxl-device-utils.c',
                ))
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 9513aaac77..599c887616 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -58,6 +58,8 @@
 #define CXL_DEVICE_CAP_HDR1_OFFSET 0x10 /* Figure 138 */
 #define CXL_DEVICE_CAP_REG_SIZE 0x10 /* 8.2.8.2 */
 #define CXL_DEVICE_CAPS_MAX 4 /* 8.2.8.2.1 + 8.2.8.5 */
+#define CXL_CAPS_SIZE \
+    (CXL_DEVICE_CAP_REG_SIZE * (CXL_DEVICE_CAPS_MAX + 1)) /* +1 for header */
 
 #define CXL_DEVICE_STATUS_REGISTERS_OFFSET 0x80 /* Read comment above */
 #define CXL_DEVICE_STATUS_REGISTERS_LENGTH 0x8 /* 8.2.8.3.1 */
@@ -70,12 +72,22 @@
 #define CXL_MAILBOX_REGISTERS_LENGTH \
     (CXL_MAILBOX_REGISTERS_SIZE + CXL_MAILBOX_MAX_PAYLOAD_SIZE)
 
+#define CXL_MMIO_SIZE                                           \
+    (CXL_DEVICE_CAP_REG_SIZE + CXL_DEVICE_STATUS_REGISTERS_LENGTH + \
+     CXL_MAILBOX_REGISTERS_LENGTH)
+
 typedef struct cxl_device_state {
     MemoryRegion device_registers;
 
     /* mmio for device capabilities array - 8.2.8.2 */
     MemoryRegion device;
-    MemoryRegion caps;
+    struct {
+        MemoryRegion caps;
+        union {
+            uint32_t caps_reg_state32[CXL_CAPS_SIZE / 4];
+            uint64_t caps_reg_state64[CXL_CAPS_SIZE / 8];
+        };
+    };
 
     /* mmio for the mailbox registers 8.2.8.4 */
     MemoryRegion mailbox;
@@ -128,6 +140,23 @@ CXL_DEVICE_CAPABILITY_HEADER_REGISTER(DEVICE_STATUS, CXL_DEVICE_CAP_HDR1_OFFSET)
 CXL_DEVICE_CAPABILITY_HEADER_REGISTER(MAILBOX, CXL_DEVICE_CAP_HDR1_OFFSET + \
                                                CXL_DEVICE_CAP_REG_SIZE)
 
+#define cxl_device_cap_init(dstate, reg, cap_id)                           \
+    do {                                                                   \
+        uint32_t *cap_hdrs = dstate->caps_reg_state32;                     \
+        int which = R_CXL_DEV_##reg##_CAP_HDR0;                            \
+        cap_hdrs[which] =                                                  \
+            FIELD_DP32(cap_hdrs[which], CXL_DEV_##reg##_CAP_HDR0,          \
+                       CAP_ID, cap_id);                                    \
+        cap_hdrs[which] = FIELD_DP32(                                      \
+            cap_hdrs[which], CXL_DEV_##reg##_CAP_HDR0, CAP_VERSION, 1);    \
+        cap_hdrs[which + 1] =                                              \
+            FIELD_DP32(cap_hdrs[which + 1], CXL_DEV_##reg##_CAP_HDR1,      \
+                       CAP_OFFSET, CXL_##reg##_REGISTERS_OFFSET);          \
+        cap_hdrs[which + 2] =                                              \
+            FIELD_DP32(cap_hdrs[which + 2], CXL_DEV_##reg##_CAP_HDR2,      \
+                       CAP_LENGTH, CXL_##reg##_REGISTERS_LENGTH);          \
+    } while (0)
+
 /* CXL 2.0 8.2.8.4.3 Mailbox Capabilities Register */
 REG32(CXL_DEV_MAILBOX_CAP, 0)
     FIELD(CXL_DEV_MAILBOX_CAP, PAYLOAD_SIZE, 0, 5)

From 464e14ac438e21c532889d78caf54a1dc837012a Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:31 +0100
Subject: [PATCH 275/935] hw/cxl/device: Implement basic mailbox (8.2.8.4)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This is the beginning of implementing mailbox support for CXL 2.0
devices. The implementation recognizes when the doorbell is rung,
handles the command/payload, clears the doorbell while returning error
codes and data.

Generally the mailbox mechanism is designed to permit communication
between the host OS and the firmware running on the device. For our
purposes, we emulate both the firmware, implemented primarily in
cxl-mailbox-utils.c, and the hardware.

No commands are implemented yet.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-7-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/cxl/cxl-device-utils.c   | 122 ++++++++++++++++++++++++++-
 hw/cxl/cxl-mailbox-utils.c  | 164 ++++++++++++++++++++++++++++++++++++
 hw/cxl/meson.build          |   1 +
 include/hw/cxl/cxl.h        |   3 +
 include/hw/cxl/cxl_device.h |  19 ++++-
 5 files changed, 307 insertions(+), 2 deletions(-)
 create mode 100644 hw/cxl/cxl-mailbox-utils.c

diff --git a/hw/cxl/cxl-device-utils.c b/hw/cxl/cxl-device-utils.c
index 241f9f82e3..f6c3e0f095 100644
--- a/hw/cxl/cxl-device-utils.c
+++ b/hw/cxl/cxl-device-utils.c
@@ -44,6 +44,108 @@ static uint64_t dev_reg_read(void *opaque, hwaddr offset, unsigned size)
     return 0;
 }
 
+static uint64_t mailbox_reg_read(void *opaque, hwaddr offset, unsigned size)
+{
+    CXLDeviceState *cxl_dstate = opaque;
+
+    switch (size) {
+    case 1:
+        return cxl_dstate->mbox_reg_state[offset];
+    case 2:
+        return cxl_dstate->mbox_reg_state16[offset / size];
+    case 4:
+        return cxl_dstate->mbox_reg_state32[offset / size];
+    case 8:
+        return cxl_dstate->mbox_reg_state64[offset / size];
+    default:
+        g_assert_not_reached();
+    }
+}
+
+static void mailbox_mem_writel(uint32_t *reg_state, hwaddr offset,
+                               uint64_t value)
+{
+    switch (offset) {
+    case A_CXL_DEV_MAILBOX_CTRL:
+        /* fallthrough */
+    case A_CXL_DEV_MAILBOX_CAP:
+        /* RO register */
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s Unexpected 32-bit access to 0x%" PRIx64 " (WI)\n",
+                      __func__, offset);
+        return;
+    }
+
+    reg_state[offset / sizeof(*reg_state)] = value;
+}
+
+static void mailbox_mem_writeq(uint64_t *reg_state, hwaddr offset,
+                               uint64_t value)
+{
+    switch (offset) {
+    case A_CXL_DEV_MAILBOX_CMD:
+        break;
+    case A_CXL_DEV_BG_CMD_STS:
+        /* BG not supported */
+        /* fallthrough */
+    case A_CXL_DEV_MAILBOX_STS:
+        /* Read only register, will get updated by the state machine */
+        return;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s Unexpected 64-bit access to 0x%" PRIx64 " (WI)\n",
+                      __func__, offset);
+        return;
+    }
+
+
+    reg_state[offset / sizeof(*reg_state)] = value;
+}
+
+static void mailbox_reg_write(void *opaque, hwaddr offset, uint64_t value,
+                              unsigned size)
+{
+    CXLDeviceState *cxl_dstate = opaque;
+
+    if (offset >= A_CXL_DEV_CMD_PAYLOAD) {
+        memcpy(cxl_dstate->mbox_reg_state + offset, &value, size);
+        return;
+    }
+
+    switch (size) {
+    case 4:
+        mailbox_mem_writel(cxl_dstate->mbox_reg_state32, offset, value);
+        break;
+    case 8:
+        mailbox_mem_writeq(cxl_dstate->mbox_reg_state64, offset, value);
+        break;
+    default:
+        g_assert_not_reached();
+    }
+
+    if (ARRAY_FIELD_EX32(cxl_dstate->mbox_reg_state32, CXL_DEV_MAILBOX_CTRL,
+                         DOORBELL)) {
+        cxl_process_mailbox(cxl_dstate);
+    }
+}
+
+static const MemoryRegionOps mailbox_ops = {
+    .read = mailbox_reg_read,
+    .write = mailbox_reg_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+        .unaligned = false,
+    },
+    .impl = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+    },
+};
+
 static const MemoryRegionOps dev_ops = {
     .read = dev_reg_read,
     .write = NULL, /* status register is read only */
@@ -84,20 +186,33 @@ void cxl_device_register_block_init(Object *obj, CXLDeviceState *cxl_dstate)
                           "cap-array", CXL_CAPS_SIZE);
     memory_region_init_io(&cxl_dstate->device, obj, &dev_ops, cxl_dstate,
                           "device-status", CXL_DEVICE_STATUS_REGISTERS_LENGTH);
+    memory_region_init_io(&cxl_dstate->mailbox, obj, &mailbox_ops, cxl_dstate,
+                          "mailbox", CXL_MAILBOX_REGISTERS_LENGTH);
 
     memory_region_add_subregion(&cxl_dstate->device_registers, 0,
                                 &cxl_dstate->caps);
     memory_region_add_subregion(&cxl_dstate->device_registers,
                                 CXL_DEVICE_STATUS_REGISTERS_OFFSET,
                                 &cxl_dstate->device);
+    memory_region_add_subregion(&cxl_dstate->device_registers,
+                                CXL_MAILBOX_REGISTERS_OFFSET,
+                                &cxl_dstate->mailbox);
 }
 
 static void device_reg_init_common(CXLDeviceState *cxl_dstate) { }
 
+static void mailbox_reg_init_common(CXLDeviceState *cxl_dstate)
+{
+    /* 2048 payload size, with no interrupt or background support */
+    ARRAY_FIELD_DP32(cxl_dstate->mbox_reg_state32, CXL_DEV_MAILBOX_CAP,
+                     PAYLOAD_SIZE, CXL_MAILBOX_PAYLOAD_SHIFT);
+    cxl_dstate->payload_size = CXL_MAILBOX_MAX_PAYLOAD_SIZE;
+}
+
 void cxl_device_register_init_common(CXLDeviceState *cxl_dstate)
 {
     uint64_t *cap_hdrs = cxl_dstate->caps_reg_state64;
-    const int cap_count = 1;
+    const int cap_count = 2;
 
     /* CXL Device Capabilities Array Register */
     ARRAY_FIELD_DP64(cap_hdrs, CXL_DEV_CAP_ARRAY, CAP_ID, 0);
@@ -106,4 +221,9 @@ void cxl_device_register_init_common(CXLDeviceState *cxl_dstate)
 
     cxl_device_cap_init(cxl_dstate, DEVICE_STATUS, 1);
     device_reg_init_common(cxl_dstate);
+
+    cxl_device_cap_init(cxl_dstate, MAILBOX, 2);
+    mailbox_reg_init_common(cxl_dstate);
+
+    assert(cxl_initialize_mailbox(cxl_dstate) == 0);
 }
diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
new file mode 100644
index 0000000000..2557f41f61
--- /dev/null
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -0,0 +1,164 @@
+/*
+ * CXL Utility library for mailbox interface
+ *
+ * Copyright(C) 2020 Intel Corporation.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See the
+ * COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/cxl/cxl.h"
+#include "hw/pci/pci.h"
+#include "qemu/log.h"
+#include "qemu/uuid.h"
+
+/*
+ * How to add a new command, example. The command set FOO, with cmd BAR.
+ *  1. Add the command set and cmd to the enum.
+ *     FOO    = 0x7f,
+ *          #define BAR 0
+ *  2. Implement the handler
+ *    static ret_code cmd_foo_bar(struct cxl_cmd *cmd,
+ *                                  CXLDeviceState *cxl_dstate, uint16_t *len)
+ *  3. Add the command to the cxl_cmd_set[][]
+ *    [FOO][BAR] = { "FOO_BAR", cmd_foo_bar, x, y },
+ *  4. Implement your handler
+ *     define_mailbox_handler(FOO_BAR) { ... return CXL_MBOX_SUCCESS; }
+ *
+ *
+ *  Writing the handler:
+ *    The handler will provide the &struct cxl_cmd, the &CXLDeviceState, and the
+ *    in/out length of the payload. The handler is responsible for consuming the
+ *    payload from cmd->payload and operating upon it as necessary. It must then
+ *    fill the output data into cmd->payload (overwriting what was there),
+ *    setting the length, and returning a valid return code.
+ *
+ *  XXX: The handler need not worry about endianess. The payload is read out of
+ *  a register interface that already deals with it.
+ */
+
+/* 8.2.8.4.5.1 Command Return Codes */
+typedef enum {
+    CXL_MBOX_SUCCESS = 0x0,
+    CXL_MBOX_BG_STARTED = 0x1,
+    CXL_MBOX_INVALID_INPUT = 0x2,
+    CXL_MBOX_UNSUPPORTED = 0x3,
+    CXL_MBOX_INTERNAL_ERROR = 0x4,
+    CXL_MBOX_RETRY_REQUIRED = 0x5,
+    CXL_MBOX_BUSY = 0x6,
+    CXL_MBOX_MEDIA_DISABLED = 0x7,
+    CXL_MBOX_FW_XFER_IN_PROGRESS = 0x8,
+    CXL_MBOX_FW_XFER_OUT_OF_ORDER = 0x9,
+    CXL_MBOX_FW_AUTH_FAILED = 0xa,
+    CXL_MBOX_FW_INVALID_SLOT = 0xb,
+    CXL_MBOX_FW_ROLLEDBACK = 0xc,
+    CXL_MBOX_FW_REST_REQD = 0xd,
+    CXL_MBOX_INVALID_HANDLE = 0xe,
+    CXL_MBOX_INVALID_PA = 0xf,
+    CXL_MBOX_INJECT_POISON_LIMIT = 0x10,
+    CXL_MBOX_PERMANENT_MEDIA_FAILURE = 0x11,
+    CXL_MBOX_ABORTED = 0x12,
+    CXL_MBOX_INVALID_SECURITY_STATE = 0x13,
+    CXL_MBOX_INCORRECT_PASSPHRASE = 0x14,
+    CXL_MBOX_UNSUPPORTED_MAILBOX = 0x15,
+    CXL_MBOX_INVALID_PAYLOAD_LENGTH = 0x16,
+    CXL_MBOX_MAX = 0x17
+} ret_code;
+
+struct cxl_cmd;
+typedef ret_code (*opcode_handler)(struct cxl_cmd *cmd,
+                                   CXLDeviceState *cxl_dstate, uint16_t *len);
+struct cxl_cmd {
+    const char *name;
+    opcode_handler handler;
+    ssize_t in;
+    uint16_t effect; /* Reported in CEL */
+    uint8_t *payload;
+};
+
+#define DEFINE_MAILBOX_HANDLER_ZEROED(name, size)                         \
+    uint16_t __zero##name = size;                                         \
+    static ret_code cmd_##name(struct cxl_cmd *cmd,                       \
+                               CXLDeviceState *cxl_dstate, uint16_t *len) \
+    {                                                                     \
+        *len = __zero##name;                                              \
+        memset(cmd->payload, 0, *len);                                    \
+        return CXL_MBOX_SUCCESS;                                          \
+    }
+#define DEFINE_MAILBOX_HANDLER_NOP(name)                                  \
+    static ret_code cmd_##name(struct cxl_cmd *cmd,                       \
+                               CXLDeviceState *cxl_dstate, uint16_t *len) \
+    {                                                                     \
+        return CXL_MBOX_SUCCESS;                                          \
+    }
+
+static QemuUUID cel_uuid;
+
+static struct cxl_cmd cxl_cmd_set[256][256] = {};
+
+void cxl_process_mailbox(CXLDeviceState *cxl_dstate)
+{
+    uint16_t ret = CXL_MBOX_SUCCESS;
+    struct cxl_cmd *cxl_cmd;
+    uint64_t status_reg;
+    opcode_handler h;
+    uint64_t command_reg = cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_CMD];
+
+    uint8_t set = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND_SET);
+    uint8_t cmd = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND);
+    uint16_t len = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, LENGTH);
+    cxl_cmd = &cxl_cmd_set[set][cmd];
+    h = cxl_cmd->handler;
+    if (h) {
+        if (len == cxl_cmd->in) {
+            cxl_cmd->payload = cxl_dstate->mbox_reg_state +
+                A_CXL_DEV_CMD_PAYLOAD;
+            ret = (*h)(cxl_cmd, cxl_dstate, &len);
+            assert(len <= cxl_dstate->payload_size);
+        } else {
+            ret = CXL_MBOX_INVALID_PAYLOAD_LENGTH;
+        }
+    } else {
+        qemu_log_mask(LOG_UNIMP, "Command %04xh not implemented\n",
+                      set << 8 | cmd);
+        ret = CXL_MBOX_UNSUPPORTED;
+    }
+
+    /* Set the return code */
+    status_reg = FIELD_DP64(0, CXL_DEV_MAILBOX_STS, ERRNO, ret);
+
+    /* Set the return length */
+    command_reg = FIELD_DP64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND_SET, 0);
+    command_reg = FIELD_DP64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND, 0);
+    command_reg = FIELD_DP64(command_reg, CXL_DEV_MAILBOX_CMD, LENGTH, len);
+
+    cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_CMD] = command_reg;
+    cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_STS] = status_reg;
+
+    /* Tell the host we're done */
+    ARRAY_FIELD_DP32(cxl_dstate->mbox_reg_state32, CXL_DEV_MAILBOX_CTRL,
+                     DOORBELL, 0);
+}
+
+int cxl_initialize_mailbox(CXLDeviceState *cxl_dstate)
+{
+    /* CXL 2.0: Table 169 Get Supported Logs Log Entry */
+    const char *cel_uuidstr = "0da9c0b5-bf41-4b78-8f79-96b1623b3f17";
+
+    for (int set = 0; set < 256; set++) {
+        for (int cmd = 0; cmd < 256; cmd++) {
+            if (cxl_cmd_set[set][cmd].handler) {
+                struct cxl_cmd *c = &cxl_cmd_set[set][cmd];
+                struct cel_log *log =
+                    &cxl_dstate->cel_log[cxl_dstate->cel_size];
+
+                log->opcode = (set << 8) | cmd;
+                log->effect = c->effect;
+                cxl_dstate->cel_size++;
+            }
+        }
+    }
+
+    return qemu_uuid_parse(cel_uuidstr, &cel_uuid);
+}
diff --git a/hw/cxl/meson.build b/hw/cxl/meson.build
index dd7c6f8e5a..e68eea2358 100644
--- a/hw/cxl/meson.build
+++ b/hw/cxl/meson.build
@@ -2,4 +2,5 @@ softmmu_ss.add(when: 'CONFIG_CXL',
                if_true: files(
                    'cxl-component-utils.c',
                    'cxl-device-utils.c',
+                   'cxl-mailbox-utils.c',
                ))
diff --git a/include/hw/cxl/cxl.h b/include/hw/cxl/cxl.h
index b9d1ac3fad..554ad93b6b 100644
--- a/include/hw/cxl/cxl.h
+++ b/include/hw/cxl/cxl.h
@@ -14,4 +14,7 @@
 #include "cxl_component.h"
 #include "cxl_device.h"
 
+#define CXL_COMPONENT_REG_BAR_IDX 0
+#define CXL_DEVICE_REG_BAR_IDX 2
+
 #endif
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 599c887616..35489f635a 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -90,7 +90,21 @@ typedef struct cxl_device_state {
     };
 
     /* mmio for the mailbox registers 8.2.8.4 */
-    MemoryRegion mailbox;
+    struct {
+        MemoryRegion mailbox;
+        uint16_t payload_size;
+        union {
+            uint8_t mbox_reg_state[CXL_MAILBOX_REGISTERS_LENGTH];
+            uint16_t mbox_reg_state16[CXL_MAILBOX_REGISTERS_LENGTH / 2];
+            uint32_t mbox_reg_state32[CXL_MAILBOX_REGISTERS_LENGTH / 4];
+            uint64_t mbox_reg_state64[CXL_MAILBOX_REGISTERS_LENGTH / 8];
+        };
+        struct cel_log {
+            uint16_t opcode;
+            uint16_t effect;
+        } cel_log[1 << 16];
+        size_t cel_size;
+    };
 
     /* memory region for persistent memory, HDM */
     uint64_t pmem_size;
@@ -140,6 +154,9 @@ CXL_DEVICE_CAPABILITY_HEADER_REGISTER(DEVICE_STATUS, CXL_DEVICE_CAP_HDR1_OFFSET)
 CXL_DEVICE_CAPABILITY_HEADER_REGISTER(MAILBOX, CXL_DEVICE_CAP_HDR1_OFFSET + \
                                                CXL_DEVICE_CAP_REG_SIZE)
 
+int cxl_initialize_mailbox(CXLDeviceState *cxl_dstate);
+void cxl_process_mailbox(CXLDeviceState *cxl_dstate);
+
 #define cxl_device_cap_init(dstate, reg, cap_id)                           \
     do {                                                                   \
         uint32_t *cap_hdrs = dstate->caps_reg_state32;                     \

From ce3b4e5c152fe1c05aaf03b5bbe7579338446318 Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:32 +0100
Subject: [PATCH 276/935] hw/cxl/device: Add memory device utilities
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Memory devices implement extra capabilities on top of CXL devices. This
adds support for that.

A large part of memory devices is the mailbox/command interface. All of
the mailbox handling is done in the mailbox-utils library. Longer term,
new CXL devices that are being emulated may want to handle commands
differently, and therefore would need a mechanism to opt in/out of the
specific generic handlers. As such, this is considered sufficient for
now, but may need more depth in the future.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-8-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/cxl/cxl-device-utils.c   | 38 ++++++++++++++++++++++++++++++++++++-
 include/hw/cxl/cxl_device.h | 21 +++++++++++++++++---
 2 files changed, 55 insertions(+), 4 deletions(-)

diff --git a/hw/cxl/cxl-device-utils.c b/hw/cxl/cxl-device-utils.c
index f6c3e0f095..687759b301 100644
--- a/hw/cxl/cxl-device-utils.c
+++ b/hw/cxl/cxl-device-utils.c
@@ -131,6 +131,31 @@ static void mailbox_reg_write(void *opaque, hwaddr offset, uint64_t value,
     }
 }
 
+static uint64_t mdev_reg_read(void *opaque, hwaddr offset, unsigned size)
+{
+    uint64_t retval = 0;
+
+    retval = FIELD_DP64(retval, CXL_MEM_DEV_STS, MEDIA_STATUS, 1);
+    retval = FIELD_DP64(retval, CXL_MEM_DEV_STS, MBOX_READY, 1);
+
+    return retval;
+}
+
+static const MemoryRegionOps mdev_ops = {
+    .read = mdev_reg_read,
+    .write = NULL, /* memory device register is read only */
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+        .unaligned = false,
+    },
+    .impl = {
+        .min_access_size = 8,
+        .max_access_size = 8,
+    },
+};
+
 static const MemoryRegionOps mailbox_ops = {
     .read = mailbox_reg_read,
     .write = mailbox_reg_write,
@@ -188,6 +213,9 @@ void cxl_device_register_block_init(Object *obj, CXLDeviceState *cxl_dstate)
                           "device-status", CXL_DEVICE_STATUS_REGISTERS_LENGTH);
     memory_region_init_io(&cxl_dstate->mailbox, obj, &mailbox_ops, cxl_dstate,
                           "mailbox", CXL_MAILBOX_REGISTERS_LENGTH);
+    memory_region_init_io(&cxl_dstate->memory_device, obj, &mdev_ops,
+                          cxl_dstate, "memory device caps",
+                          CXL_MEMORY_DEVICE_REGISTERS_LENGTH);
 
     memory_region_add_subregion(&cxl_dstate->device_registers, 0,
                                 &cxl_dstate->caps);
@@ -197,6 +225,9 @@ void cxl_device_register_block_init(Object *obj, CXLDeviceState *cxl_dstate)
     memory_region_add_subregion(&cxl_dstate->device_registers,
                                 CXL_MAILBOX_REGISTERS_OFFSET,
                                 &cxl_dstate->mailbox);
+    memory_region_add_subregion(&cxl_dstate->device_registers,
+                                CXL_MEMORY_DEVICE_REGISTERS_OFFSET,
+                                &cxl_dstate->memory_device);
 }
 
 static void device_reg_init_common(CXLDeviceState *cxl_dstate) { }
@@ -209,10 +240,12 @@ static void mailbox_reg_init_common(CXLDeviceState *cxl_dstate)
     cxl_dstate->payload_size = CXL_MAILBOX_MAX_PAYLOAD_SIZE;
 }
 
+static void memdev_reg_init_common(CXLDeviceState *cxl_dstate) { }
+
 void cxl_device_register_init_common(CXLDeviceState *cxl_dstate)
 {
     uint64_t *cap_hdrs = cxl_dstate->caps_reg_state64;
-    const int cap_count = 2;
+    const int cap_count = 3;
 
     /* CXL Device Capabilities Array Register */
     ARRAY_FIELD_DP64(cap_hdrs, CXL_DEV_CAP_ARRAY, CAP_ID, 0);
@@ -225,5 +258,8 @@ void cxl_device_register_init_common(CXLDeviceState *cxl_dstate)
     cxl_device_cap_init(cxl_dstate, MAILBOX, 2);
     mailbox_reg_init_common(cxl_dstate);
 
+    cxl_device_cap_init(cxl_dstate, MEMORY_DEVICE, 0x4000);
+    memdev_reg_init_common(cxl_dstate);
+
     assert(cxl_initialize_mailbox(cxl_dstate) == 0);
 }
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 35489f635a..954205653e 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -72,15 +72,20 @@
 #define CXL_MAILBOX_REGISTERS_LENGTH \
     (CXL_MAILBOX_REGISTERS_SIZE + CXL_MAILBOX_MAX_PAYLOAD_SIZE)
 
-#define CXL_MMIO_SIZE                                           \
-    (CXL_DEVICE_CAP_REG_SIZE + CXL_DEVICE_STATUS_REGISTERS_LENGTH + \
-     CXL_MAILBOX_REGISTERS_LENGTH)
+#define CXL_MEMORY_DEVICE_REGISTERS_OFFSET \
+    (CXL_MAILBOX_REGISTERS_OFFSET + CXL_MAILBOX_REGISTERS_LENGTH)
+#define CXL_MEMORY_DEVICE_REGISTERS_LENGTH 0x8
+
+#define CXL_MMIO_SIZE                                                   \
+    (CXL_DEVICE_CAP_REG_SIZE + CXL_DEVICE_STATUS_REGISTERS_LENGTH +     \
+     CXL_MAILBOX_REGISTERS_LENGTH + CXL_MEMORY_DEVICE_REGISTERS_LENGTH)
 
 typedef struct cxl_device_state {
     MemoryRegion device_registers;
 
     /* mmio for device capabilities array - 8.2.8.2 */
     MemoryRegion device;
+    MemoryRegion memory_device;
     struct {
         MemoryRegion caps;
         union {
@@ -153,6 +158,9 @@ REG64(CXL_DEV_CAP_ARRAY, 0) /* Documented as 128 bit register but 64 byte access
 CXL_DEVICE_CAPABILITY_HEADER_REGISTER(DEVICE_STATUS, CXL_DEVICE_CAP_HDR1_OFFSET)
 CXL_DEVICE_CAPABILITY_HEADER_REGISTER(MAILBOX, CXL_DEVICE_CAP_HDR1_OFFSET + \
                                                CXL_DEVICE_CAP_REG_SIZE)
+CXL_DEVICE_CAPABILITY_HEADER_REGISTER(MEMORY_DEVICE,
+                                      CXL_DEVICE_CAP_HDR1_OFFSET +
+                                          CXL_DEVICE_CAP_REG_SIZE * 2)
 
 int cxl_initialize_mailbox(CXLDeviceState *cxl_dstate);
 void cxl_process_mailbox(CXLDeviceState *cxl_dstate);
@@ -209,4 +217,11 @@ REG64(CXL_DEV_BG_CMD_STS, 0x18)
 /* CXL 2.0 8.2.8.4.8 Command Payload Registers */
 REG32(CXL_DEV_CMD_PAYLOAD, 0x20)
 
+REG64(CXL_MEM_DEV_STS, 0)
+    FIELD(CXL_MEM_DEV_STS, FATAL, 0, 1)
+    FIELD(CXL_MEM_DEV_STS, FW_HALT, 1, 1)
+    FIELD(CXL_MEM_DEV_STS, MEDIA_STATUS, 2, 2)
+    FIELD(CXL_MEM_DEV_STS, MBOX_READY, 4, 1)
+    FIELD(CXL_MEM_DEV_STS, RESET_NEEDED, 5, 3)
+
 #endif

From 57c02b355fd38c279bf54b5f492e6fdc7114a188 Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:33 +0100
Subject: [PATCH 277/935] hw/cxl/device: Add cheap EVENTS implementation
 (8.2.9.1)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Using the previously implemented stubbed helpers, it is now possible to
easily add the missing, required commands to the implementation.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-9-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/cxl/cxl-mailbox-utils.c | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 2557f41f61..fb1f53f48e 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -38,6 +38,14 @@
  *  a register interface that already deals with it.
  */
 
+enum {
+    EVENTS      = 0x01,
+        #define GET_RECORDS   0x0
+        #define CLEAR_RECORDS   0x1
+        #define GET_INTERRUPT_POLICY   0x2
+        #define SET_INTERRUPT_POLICY   0x3
+};
+
 /* 8.2.8.4.5.1 Command Return Codes */
 typedef enum {
     CXL_MBOX_SUCCESS = 0x0,
@@ -93,9 +101,26 @@ struct cxl_cmd {
         return CXL_MBOX_SUCCESS;                                          \
     }
 
+DEFINE_MAILBOX_HANDLER_ZEROED(events_get_records, 0x20);
+DEFINE_MAILBOX_HANDLER_NOP(events_clear_records);
+DEFINE_MAILBOX_HANDLER_ZEROED(events_get_interrupt_policy, 4);
+DEFINE_MAILBOX_HANDLER_NOP(events_set_interrupt_policy);
+
 static QemuUUID cel_uuid;
 
-static struct cxl_cmd cxl_cmd_set[256][256] = {};
+#define IMMEDIATE_CONFIG_CHANGE (1 << 1)
+#define IMMEDIATE_LOG_CHANGE (1 << 4)
+
+static struct cxl_cmd cxl_cmd_set[256][256] = {
+    [EVENTS][GET_RECORDS] = { "EVENTS_GET_RECORDS",
+        cmd_events_get_records, 1, 0 },
+    [EVENTS][CLEAR_RECORDS] = { "EVENTS_CLEAR_RECORDS",
+        cmd_events_clear_records, ~0, IMMEDIATE_LOG_CHANGE },
+    [EVENTS][GET_INTERRUPT_POLICY] = { "EVENTS_GET_INTERRUPT_POLICY",
+        cmd_events_get_interrupt_policy, 0, 0 },
+    [EVENTS][SET_INTERRUPT_POLICY] = { "EVENTS_SET_INTERRUPT_POLICY",
+        cmd_events_set_interrupt_policy, 4, IMMEDIATE_CONFIG_CHANGE },
+};
 
 void cxl_process_mailbox(CXLDeviceState *cxl_dstate)
 {

From 557a79c83e73e5c524a1d44626b509644efa0103 Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:34 +0100
Subject: [PATCH 278/935] hw/cxl/device: Timestamp implementation (8.2.9.3)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Errata F4 to CXL 2.0 clarified the meaning of the timer as the
sum of the value set with the timestamp set command and the number
of nano seconds since it was last set.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-10-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/cxl/cxl-mailbox-utils.c  | 42 +++++++++++++++++++++++++++++++++++++
 include/hw/cxl/cxl_device.h |  6 ++++++
 2 files changed, 48 insertions(+)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index fb1f53f48e..4584aa31f7 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -44,6 +44,9 @@ enum {
         #define CLEAR_RECORDS   0x1
         #define GET_INTERRUPT_POLICY   0x2
         #define SET_INTERRUPT_POLICY   0x3
+    TIMESTAMP   = 0x03,
+        #define GET           0x0
+        #define SET           0x1
 };
 
 /* 8.2.8.4.5.1 Command Return Codes */
@@ -106,9 +109,46 @@ DEFINE_MAILBOX_HANDLER_NOP(events_clear_records);
 DEFINE_MAILBOX_HANDLER_ZEROED(events_get_interrupt_policy, 4);
 DEFINE_MAILBOX_HANDLER_NOP(events_set_interrupt_policy);
 
+/* 8.2.9.3.1 */
+static ret_code cmd_timestamp_get(struct cxl_cmd *cmd,
+                                  CXLDeviceState *cxl_dstate,
+                                  uint16_t *len)
+{
+    uint64_t time, delta;
+    uint64_t final_time = 0;
+
+    if (cxl_dstate->timestamp.set) {
+        /* First find the delta from the last time the host set the time. */
+        time = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
+        delta = time - cxl_dstate->timestamp.last_set;
+        final_time = cxl_dstate->timestamp.host_set + delta;
+    }
+
+    /* Then adjust the actual time */
+    stq_le_p(cmd->payload, final_time);
+    *len = 8;
+
+    return CXL_MBOX_SUCCESS;
+}
+
+/* 8.2.9.3.2 */
+static ret_code cmd_timestamp_set(struct cxl_cmd *cmd,
+                                  CXLDeviceState *cxl_dstate,
+                                  uint16_t *len)
+{
+    cxl_dstate->timestamp.set = true;
+    cxl_dstate->timestamp.last_set = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
+
+    cxl_dstate->timestamp.host_set = le64_to_cpu(*(uint64_t *)cmd->payload);
+
+    *len = 0;
+    return CXL_MBOX_SUCCESS;
+}
+
 static QemuUUID cel_uuid;
 
 #define IMMEDIATE_CONFIG_CHANGE (1 << 1)
+#define IMMEDIATE_POLICY_CHANGE (1 << 3)
 #define IMMEDIATE_LOG_CHANGE (1 << 4)
 
 static struct cxl_cmd cxl_cmd_set[256][256] = {
@@ -120,6 +160,8 @@ static struct cxl_cmd cxl_cmd_set[256][256] = {
         cmd_events_get_interrupt_policy, 0, 0 },
     [EVENTS][SET_INTERRUPT_POLICY] = { "EVENTS_SET_INTERRUPT_POLICY",
         cmd_events_set_interrupt_policy, 4, IMMEDIATE_CONFIG_CHANGE },
+    [TIMESTAMP][GET] = { "TIMESTAMP_GET", cmd_timestamp_get, 0, 0 },
+    [TIMESTAMP][SET] = { "TIMESTAMP_SET", cmd_timestamp_set, 8, IMMEDIATE_POLICY_CHANGE },
 };
 
 void cxl_process_mailbox(CXLDeviceState *cxl_dstate)
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 954205653e..797a22ddb4 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -111,6 +111,12 @@ typedef struct cxl_device_state {
         size_t cel_size;
     };
 
+    struct {
+        bool set;
+        uint64_t last_set;
+        uint64_t host_set;
+    } timestamp;
+
     /* memory region for persistent memory, HDM */
     uint64_t pmem_size;
 } CXLDeviceState;

From 056172691bbfc2cd7c7e8ac6a14f78b45ddadba1 Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:35 +0100
Subject: [PATCH 279/935] hw/cxl/device: Add log commands (8.2.9.4) + CEL
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

CXL specification provides for the ability to obtain logs from the
device. Logs are either spec defined, like the "Command Effects Log"
(CEL), or vendor specific. UUIDs are defined for all log types.

The CEL is a mechanism to provide information to the host about which
commands are supported. It is useful both to determine which spec'd
optional commands are supported, as well as provide a list of vendor
specified commands that might be used. The CEL is already created as
part of mailbox initialization, but here it is now exported to hosts
that use these log commands.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-11-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/cxl/cxl-mailbox-utils.c | 69 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 69 insertions(+)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 4584aa31f7..db473135c7 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -47,6 +47,9 @@ enum {
     TIMESTAMP   = 0x03,
         #define GET           0x0
         #define SET           0x1
+    LOGS        = 0x04,
+        #define GET_SUPPORTED 0x0
+        #define GET_LOG       0x1
 };
 
 /* 8.2.8.4.5.1 Command Return Codes */
@@ -147,6 +150,70 @@ static ret_code cmd_timestamp_set(struct cxl_cmd *cmd,
 
 static QemuUUID cel_uuid;
 
+/* 8.2.9.4.1 */
+static ret_code cmd_logs_get_supported(struct cxl_cmd *cmd,
+                                       CXLDeviceState *cxl_dstate,
+                                       uint16_t *len)
+{
+    struct {
+        uint16_t entries;
+        uint8_t rsvd[6];
+        struct {
+            QemuUUID uuid;
+            uint32_t size;
+        } log_entries[1];
+    } QEMU_PACKED *supported_logs = (void *)cmd->payload;
+    QEMU_BUILD_BUG_ON(sizeof(*supported_logs) != 0x1c);
+
+    supported_logs->entries = 1;
+    supported_logs->log_entries[0].uuid = cel_uuid;
+    supported_logs->log_entries[0].size = 4 * cxl_dstate->cel_size;
+
+    *len = sizeof(*supported_logs);
+    return CXL_MBOX_SUCCESS;
+}
+
+/* 8.2.9.4.2 */
+static ret_code cmd_logs_get_log(struct cxl_cmd *cmd,
+                                 CXLDeviceState *cxl_dstate,
+                                 uint16_t *len)
+{
+    struct {
+        QemuUUID uuid;
+        uint32_t offset;
+        uint32_t length;
+    } QEMU_PACKED QEMU_ALIGNED(16) *get_log = (void *)cmd->payload;
+
+    /*
+     * 8.2.9.4.2
+     *   The device shall return Invalid Parameter if the Offset or Length
+     *   fields attempt to access beyond the size of the log as reported by Get
+     *   Supported Logs.
+     *
+     * XXX: Spec is wrong, "Invalid Parameter" isn't a thing.
+     * XXX: Spec doesn't address incorrect UUID incorrectness.
+     *
+     * The CEL buffer is large enough to fit all commands in the emulation, so
+     * the only possible failure would be if the mailbox itself isn't big
+     * enough.
+     */
+    if (get_log->offset + get_log->length > cxl_dstate->payload_size) {
+        return CXL_MBOX_INVALID_INPUT;
+    }
+
+    if (!qemu_uuid_is_equal(&get_log->uuid, &cel_uuid)) {
+        return CXL_MBOX_UNSUPPORTED;
+    }
+
+    /* Store off everything to local variables so we can wipe out the payload */
+    *len = get_log->length;
+
+    memmove(cmd->payload, cxl_dstate->cel_log + get_log->offset,
+           get_log->length);
+
+    return CXL_MBOX_SUCCESS;
+}
+
 #define IMMEDIATE_CONFIG_CHANGE (1 << 1)
 #define IMMEDIATE_POLICY_CHANGE (1 << 3)
 #define IMMEDIATE_LOG_CHANGE (1 << 4)
@@ -162,6 +229,8 @@ static struct cxl_cmd cxl_cmd_set[256][256] = {
         cmd_events_set_interrupt_policy, 4, IMMEDIATE_CONFIG_CHANGE },
     [TIMESTAMP][GET] = { "TIMESTAMP_GET", cmd_timestamp_get, 0, 0 },
     [TIMESTAMP][SET] = { "TIMESTAMP_SET", cmd_timestamp_set, 8, IMMEDIATE_POLICY_CHANGE },
+    [LOGS][GET_SUPPORTED] = { "LOGS_GET_SUPPORTED", cmd_logs_get_supported, 0, 0 },
+    [LOGS][GET_LOG] = { "LOGS_GET_LOG", cmd_logs_get_log, 0x18, 0 },
 };
 
 void cxl_process_mailbox(CXLDeviceState *cxl_dstate)

From 25a2e524e34e9a7cdf709a380b9deaada9dc4995 Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:36 +0100
Subject: [PATCH 280/935] hw/pxb: Use a type for realizing expanders
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This opens up the possibility for more types of expanders (other than
PCI and PCIe). We'll need this to create a CXL expander.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-12-Jonathan.Cameron@huawei.com>
---
 hw/pci-bridge/pci_expander_bridge.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c
index de932286b5..d4514227a8 100644
--- a/hw/pci-bridge/pci_expander_bridge.c
+++ b/hw/pci-bridge/pci_expander_bridge.c
@@ -24,6 +24,8 @@
 #include "hw/boards.h"
 #include "qom/object.h"
 
+enum BusType { PCI, PCIE };
+
 #define TYPE_PXB_BUS "pxb-bus"
 typedef struct PXBBus PXBBus;
 DECLARE_INSTANCE_CHECKER(PXBBus, PXB_BUS,
@@ -221,7 +223,8 @@ static gint pxb_compare(gconstpointer a, gconstpointer b)
            0;
 }
 
-static void pxb_dev_realize_common(PCIDevice *dev, bool pcie, Error **errp)
+static void pxb_dev_realize_common(PCIDevice *dev, enum BusType type,
+                                   Error **errp)
 {
     PXBDev *pxb = convert_to_pxb(dev);
     DeviceState *ds, *bds = NULL;
@@ -246,7 +249,7 @@ static void pxb_dev_realize_common(PCIDevice *dev, bool pcie, Error **errp)
     }
 
     ds = qdev_new(TYPE_PXB_HOST);
-    if (pcie) {
+    if (type == PCIE) {
         bus = pci_root_bus_new(ds, dev_name, NULL, NULL, 0, TYPE_PXB_PCIE_BUS);
     } else {
         bus = pci_root_bus_new(ds, "pxb-internal", NULL, NULL, 0, TYPE_PXB_BUS);
@@ -295,7 +298,7 @@ static void pxb_dev_realize(PCIDevice *dev, Error **errp)
         return;
     }
 
-    pxb_dev_realize_common(dev, false, errp);
+    pxb_dev_realize_common(dev, PCI, errp);
 }
 
 static void pxb_dev_exitfn(PCIDevice *pci_dev)
@@ -348,7 +351,7 @@ static void pxb_pcie_dev_realize(PCIDevice *dev, Error **errp)
         return;
     }
 
-    pxb_dev_realize_common(dev, true, errp);
+    pxb_dev_realize_common(dev, PCIE, errp);
 }
 
 static void pxb_pcie_dev_class_init(ObjectClass *klass, void *data)

From 9dccb1216bf9d120579612979f4d58fd97bf0150 Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:37 +0100
Subject: [PATCH 281/935] hw/pci/cxl: Create a CXL bus type
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The easiest way to differentiate a CXL bus, and a PCIE bus is using a
flag. A CXL bus, in hardware, is backward compatible with PCIE, and
therefore the code tries pretty hard to keep them in sync as much as
possible.

The other way to implement this would be to try to cast the bus to the
correct type. This is less code and useful for debugging via simply
looking at the flags.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-13-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/pci-bridge/pci_expander_bridge.c | 9 ++++++++-
 include/hw/pci/pci_bus.h            | 7 +++++++
 2 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c
index d4514227a8..a6caa1e7b5 100644
--- a/hw/pci-bridge/pci_expander_bridge.c
+++ b/hw/pci-bridge/pci_expander_bridge.c
@@ -24,7 +24,7 @@
 #include "hw/boards.h"
 #include "qom/object.h"
 
-enum BusType { PCI, PCIE };
+enum BusType { PCI, PCIE, CXL };
 
 #define TYPE_PXB_BUS "pxb-bus"
 typedef struct PXBBus PXBBus;
@@ -35,6 +35,10 @@ DECLARE_INSTANCE_CHECKER(PXBBus, PXB_BUS,
 DECLARE_INSTANCE_CHECKER(PXBBus, PXB_PCIE_BUS,
                          TYPE_PXB_PCIE_BUS)
 
+#define TYPE_PXB_CXL_BUS "pxb-cxl-bus"
+DECLARE_INSTANCE_CHECKER(PXBBus, PXB_CXL_BUS,
+                         TYPE_PXB_CXL_BUS)
+
 struct PXBBus {
     /*< private >*/
     PCIBus parent_obj;
@@ -251,6 +255,9 @@ static void pxb_dev_realize_common(PCIDevice *dev, enum BusType type,
     ds = qdev_new(TYPE_PXB_HOST);
     if (type == PCIE) {
         bus = pci_root_bus_new(ds, dev_name, NULL, NULL, 0, TYPE_PXB_PCIE_BUS);
+    } else if (type == CXL) {
+        bus = pci_root_bus_new(ds, dev_name, NULL, NULL, 0, TYPE_PXB_CXL_BUS);
+        bus->flags |= PCI_BUS_CXL;
     } else {
         bus = pci_root_bus_new(ds, "pxb-internal", NULL, NULL, 0, TYPE_PXB_BUS);
         bds = qdev_new("pci-bridge");
diff --git a/include/hw/pci/pci_bus.h b/include/hw/pci/pci_bus.h
index 347440d42c..eb94e7e85c 100644
--- a/include/hw/pci/pci_bus.h
+++ b/include/hw/pci/pci_bus.h
@@ -24,6 +24,8 @@ enum PCIBusFlags {
     PCI_BUS_IS_ROOT                                         = 0x0001,
     /* PCIe extended configuration space is accessible on this bus */
     PCI_BUS_EXTENDED_CONFIG_SPACE                           = 0x0002,
+    /* This is a CXL Type BUS */
+    PCI_BUS_CXL                                             = 0x0004,
 };
 
 struct PCIBus {
@@ -53,6 +55,11 @@ struct PCIBus {
     Notifier machine_done;
 };
 
+static inline bool pci_bus_is_cxl(PCIBus *bus)
+{
+    return !!(bus->flags & PCI_BUS_CXL);
+}
+
 static inline bool pci_bus_is_root(PCIBus *bus)
 {
     return !!(bus->flags & PCI_BUS_IS_ROOT);

From abb3009baf90e5984ff1c230af0bc92a45e64864 Mon Sep 17 00:00:00 2001
From: Jonathan Cameron <jonathan.cameron@huawei.com>
Date: Fri, 29 Apr 2022 15:40:38 +0100
Subject: [PATCH 282/935] cxl: Machine level control on whether CXL support is
 enabled
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

There are going to be some potential overheads to CXL enablement,
for example the host bridge region reserved in memory maps.
Add a machine level control so that CXL is disabled by default.

Signed-off-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-14-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/core/machine.c    | 28 ++++++++++++++++++++++++++++
 hw/i386/pc.c         |  1 +
 include/hw/boards.h  |  2 ++
 include/hw/cxl/cxl.h |  4 ++++
 4 files changed, 35 insertions(+)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index 3264c1e11d..b03d9192ba 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -33,6 +33,7 @@
 #include "sysemu/qtest.h"
 #include "hw/pci/pci.h"
 #include "hw/mem/nvdimm.h"
+#include "hw/cxl/cxl.h"
 #include "migration/global_state.h"
 #include "migration/vmstate.h"
 #include "exec/confidential-guest-support.h"
@@ -625,6 +626,20 @@ static void machine_set_nvdimm_persistence(Object *obj, const char *value,
     nvdimms_state->persistence_string = g_strdup(value);
 }
 
+static bool machine_get_cxl(Object *obj, Error **errp)
+{
+    MachineState *ms = MACHINE(obj);
+
+    return ms->cxl_devices_state->is_enabled;
+}
+
+static void machine_set_cxl(Object *obj, bool value, Error **errp)
+{
+    MachineState *ms = MACHINE(obj);
+
+    ms->cxl_devices_state->is_enabled = value;
+}
+
 void machine_class_allow_dynamic_sysbus_dev(MachineClass *mc, const char *type)
 {
     QAPI_LIST_PREPEND(mc->allowed_dynamic_sysbus_devices, g_strdup(type));
@@ -911,6 +926,8 @@ static void machine_class_init(ObjectClass *oc, void *data)
     mc->default_ram_size = 128 * MiB;
     mc->rom_file_has_mr = true;
 
+    /* Few machines support CXL, so default to off */
+    mc->cxl_supported = false;
     /* numa node memory size aligned on 8MB by default.
      * On Linux, each node's border has to be 8MB aligned
      */
@@ -1071,6 +1088,16 @@ static void machine_initfn(Object *obj)
                                         "Valid values are cpu, mem-ctrl");
     }
 
+    if (mc->cxl_supported) {
+        Object *obj = OBJECT(ms);
+
+        ms->cxl_devices_state = g_new0(CXLState, 1);
+        object_property_add_bool(obj, "cxl", machine_get_cxl, machine_set_cxl);
+        object_property_set_description(obj, "cxl",
+                                        "Set on/off to enable/disable "
+                                        "CXL instantiation");
+    }
+
     if (mc->cpu_index_to_instance_props && mc->get_default_cpu_node_id) {
         ms->numa_state = g_new0(NumaState, 1);
         object_property_add_bool(obj, "hmat",
@@ -1108,6 +1135,7 @@ static void machine_finalize(Object *obj)
     g_free(ms->device_memory);
     g_free(ms->nvdimms_state);
     g_free(ms->numa_state);
+    g_free(ms->cxl_devices_state);
 }
 
 bool machine_usb(MachineState *machine)
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 305d2c0820..45e2d6092f 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1761,6 +1761,7 @@ static void pc_machine_class_init(ObjectClass *oc, void *data)
     mc->default_cpu_type = TARGET_DEFAULT_CPU_TYPE;
     mc->nvdimm_supported = true;
     mc->smp_props.dies_supported = true;
+    mc->cxl_supported = true;
     mc->default_ram_id = "pc.ram";
 
     object_class_property_add(oc, PC_MACHINE_MAX_RAM_BELOW_4G, "size",
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 7b416c9787..fa57bac4fb 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -269,6 +269,7 @@ struct MachineClass {
     bool ignore_boot_device_suffixes;
     bool smbus_no_migration_support;
     bool nvdimm_supported;
+    bool cxl_supported;
     bool numa_mem_supported;
     bool auto_enable_numa;
     SMPCompatProps smp_props;
@@ -359,6 +360,7 @@ struct MachineState {
     CPUArchIdList *possible_cpus;
     CpuTopology smp;
     struct NVDIMMState *nvdimms_state;
+    struct CXLState *cxl_devices_state;
     struct NumaState *numa_state;
 };
 
diff --git a/include/hw/cxl/cxl.h b/include/hw/cxl/cxl.h
index 554ad93b6b..31af92fd5e 100644
--- a/include/hw/cxl/cxl.h
+++ b/include/hw/cxl/cxl.h
@@ -17,4 +17,8 @@
 #define CXL_COMPONENT_REG_BAR_IDX 0
 #define CXL_DEVICE_REG_BAR_IDX 2
 
+typedef struct CXLState {
+    bool is_enabled;
+} CXLState;
+
 #endif

From 4f8db8711cbd27c9acf17e685987e9e74815e087 Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:39 +0100
Subject: [PATCH 283/935] hw/pxb: Allow creation of a CXL PXB (host bridge)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This works like adding a typical pxb device, except the name is
'pxb-cxl' instead of 'pxb-pcie'. An example command line would be as
follows:
  -device pxb-cxl,id=cxl.0,bus="pcie.0",bus_nr=1

A CXL PXB is backward compatible with PCIe. What this means in practice
is that an operating system that is unaware of CXL should still be able
to enumerate this topology as if it were PCIe.

One can create multiple CXL PXB host bridges, but a host bridge can only
be connected to the main root bus. Host bridges cannot appear elsewhere
in the topology.

Note that as of this patch, the ACPI tables needed for the host bridge
(specifically, an ACPI object in _SB named ACPI0016 and the CEDT) aren't
created. So while this patch internally creates it, it cannot be
properly used by an operating system or other system software.

Also necessary is to add an exception to scripts/device-crash-test
similar to that for exiting pxb as both must created on a PCIexpress
host bus.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan.Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-15-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/pci-bridge/pci_expander_bridge.c | 86 ++++++++++++++++++++++++++++-
 hw/pci/pci.c                        |  7 +++
 include/hw/pci/pci.h                |  6 ++
 scripts/device-crash-test           |  1 +
 4 files changed, 98 insertions(+), 2 deletions(-)

diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c
index a6caa1e7b5..f762eb4a6e 100644
--- a/hw/pci-bridge/pci_expander_bridge.c
+++ b/hw/pci-bridge/pci_expander_bridge.c
@@ -17,6 +17,7 @@
 #include "hw/pci/pci_host.h"
 #include "hw/qdev-properties.h"
 #include "hw/pci/pci_bridge.h"
+#include "hw/cxl/cxl.h"
 #include "qemu/range.h"
 #include "qemu/error-report.h"
 #include "qemu/module.h"
@@ -56,6 +57,16 @@ DECLARE_INSTANCE_CHECKER(PXBDev, PXB_DEV,
 DECLARE_INSTANCE_CHECKER(PXBDev, PXB_PCIE_DEV,
                          TYPE_PXB_PCIE_DEVICE)
 
+#define TYPE_PXB_CXL_DEVICE "pxb-cxl"
+DECLARE_INSTANCE_CHECKER(PXBDev, PXB_CXL_DEV,
+                         TYPE_PXB_CXL_DEVICE)
+
+typedef struct CXLHost {
+    PCIHostState parent_obj;
+
+    CXLComponentState cxl_cstate;
+} CXLHost;
+
 struct PXBDev {
     /*< private >*/
     PCIDevice parent_obj;
@@ -68,6 +79,11 @@ struct PXBDev {
 
 static PXBDev *convert_to_pxb(PCIDevice *dev)
 {
+    /* A CXL PXB's parent bus is PCIe, so the normal check won't work */
+    if (object_dynamic_cast(OBJECT(dev), TYPE_PXB_CXL_DEVICE)) {
+        return PXB_CXL_DEV(dev);
+    }
+
     return pci_bus_is_express(pci_get_bus(dev))
         ? PXB_PCIE_DEV(dev) : PXB_DEV(dev);
 }
@@ -112,11 +128,20 @@ static const TypeInfo pxb_pcie_bus_info = {
     .class_init    = pxb_bus_class_init,
 };
 
+static const TypeInfo pxb_cxl_bus_info = {
+    .name          = TYPE_PXB_CXL_BUS,
+    .parent        = TYPE_CXL_BUS,
+    .instance_size = sizeof(PXBBus),
+    .class_init    = pxb_bus_class_init,
+};
+
 static const char *pxb_host_root_bus_path(PCIHostState *host_bridge,
                                           PCIBus *rootbus)
 {
-    PXBBus *bus = pci_bus_is_express(rootbus) ?
-                  PXB_PCIE_BUS(rootbus) : PXB_BUS(rootbus);
+    PXBBus *bus = pci_bus_is_cxl(rootbus) ?
+                      PXB_CXL_BUS(rootbus) :
+                      pci_bus_is_express(rootbus) ? PXB_PCIE_BUS(rootbus) :
+                                                    PXB_BUS(rootbus);
 
     snprintf(bus->bus_path, 8, "0000:%02x", pxb_bus_num(rootbus));
     return bus->bus_path;
@@ -218,6 +243,10 @@ static int pxb_map_irq_fn(PCIDevice *pci_dev, int pin)
     return pin - PCI_SLOT(pxb->devfn);
 }
 
+static void pxb_dev_reset(DeviceState *dev)
+{
+}
+
 static gint pxb_compare(gconstpointer a, gconstpointer b)
 {
     const PXBDev *pxb_a = a, *pxb_b = b;
@@ -389,13 +418,66 @@ static const TypeInfo pxb_pcie_dev_info = {
     },
 };
 
+static void pxb_cxl_dev_realize(PCIDevice *dev, Error **errp)
+{
+    MachineState *ms = MACHINE(qdev_get_machine());
+
+    /* A CXL PXB's parent bus is still PCIe */
+    if (!pci_bus_is_express(pci_get_bus(dev))) {
+        error_setg(errp, "pxb-cxl devices cannot reside on a PCI bus");
+        return;
+    }
+    if (!ms->cxl_devices_state->is_enabled) {
+        error_setg(errp, "Machine does not have cxl=on");
+        return;
+    }
+
+    pxb_dev_realize_common(dev, CXL, errp);
+    pxb_dev_reset(DEVICE(dev));
+}
+
+static void pxb_cxl_dev_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc   = DEVICE_CLASS(klass);
+    PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
+
+    k->realize             = pxb_cxl_dev_realize;
+    k->exit                = pxb_dev_exitfn;
+    /*
+     * XXX: These types of bridges don't actually show up in the hierarchy so
+     * vendor, device, class, etc. ids are intentionally left out.
+     */
+
+    dc->desc = "CXL Host Bridge";
+    device_class_set_props(dc, pxb_dev_properties);
+    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
+
+    /* Host bridges aren't hotpluggable. FIXME: spec reference */
+    dc->hotpluggable = false;
+    dc->reset = pxb_dev_reset;
+}
+
+static const TypeInfo pxb_cxl_dev_info = {
+    .name          = TYPE_PXB_CXL_DEVICE,
+    .parent        = TYPE_PCI_DEVICE,
+    .instance_size = sizeof(PXBDev),
+    .class_init    = pxb_cxl_dev_class_init,
+    .interfaces =
+        (InterfaceInfo[]){
+            { INTERFACE_CONVENTIONAL_PCI_DEVICE },
+            {},
+        },
+};
+
 static void pxb_register_types(void)
 {
     type_register_static(&pxb_bus_info);
     type_register_static(&pxb_pcie_bus_info);
+    type_register_static(&pxb_cxl_bus_info);
     type_register_static(&pxb_host_info);
     type_register_static(&pxb_dev_info);
     type_register_static(&pxb_pcie_dev_info);
+    type_register_static(&pxb_cxl_dev_info);
 }
 
 type_init(pxb_register_types)
diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index eb884adef9..6c0656f604 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -228,6 +228,12 @@ static const TypeInfo pcie_bus_info = {
     .class_init = pcie_bus_class_init,
 };
 
+static const TypeInfo cxl_bus_info = {
+    .name       = TYPE_CXL_BUS,
+    .parent     = TYPE_PCIE_BUS,
+    .class_init = pcie_bus_class_init,
+};
+
 static PCIBus *pci_find_bus_nr(PCIBus *bus, int bus_num);
 static void pci_update_mappings(PCIDevice *d);
 static void pci_irq_handler(void *opaque, int irq_num, int level);
@@ -2946,6 +2952,7 @@ static void pci_register_types(void)
 {
     type_register_static(&pci_bus_info);
     type_register_static(&pcie_bus_info);
+    type_register_static(&cxl_bus_info);
     type_register_static(&conventional_pci_interface_info);
     type_register_static(&cxl_interface_info);
     type_register_static(&pcie_interface_info);
diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h
index 98f0d1b844..44dacfa224 100644
--- a/include/hw/pci/pci.h
+++ b/include/hw/pci/pci.h
@@ -408,6 +408,7 @@ typedef PCIINTxRoute (*pci_route_irq_fn)(void *opaque, int pin);
 #define TYPE_PCI_BUS "PCI"
 OBJECT_DECLARE_TYPE(PCIBus, PCIBusClass, PCI_BUS)
 #define TYPE_PCIE_BUS "PCIE"
+#define TYPE_CXL_BUS "CXL"
 
 typedef void (*pci_bus_dev_fn)(PCIBus *b, PCIDevice *d, void *opaque);
 typedef void (*pci_bus_fn)(PCIBus *b, void *opaque);
@@ -770,6 +771,11 @@ static inline void pci_irq_pulse(PCIDevice *pci_dev)
     pci_irq_deassert(pci_dev);
 }
 
+static inline int pci_is_cxl(const PCIDevice *d)
+{
+    return d->cap_present & QEMU_PCIE_CAP_CXL;
+}
+
 static inline int pci_is_express(const PCIDevice *d)
 {
     return d->cap_present & QEMU_PCI_CAP_EXPRESS;
diff --git a/scripts/device-crash-test b/scripts/device-crash-test
index 4bfc68c008..a203b3fdea 100755
--- a/scripts/device-crash-test
+++ b/scripts/device-crash-test
@@ -93,6 +93,7 @@ ERROR_RULE_LIST = [
     {'device':'pci-bridge', 'expected':True},              # Bridge chassis not specified. Each bridge is required to be assigned a unique chassis id > 0.
     {'device':'pci-bridge-seat', 'expected':True},         # Bridge chassis not specified. Each bridge is required to be assigned a unique chassis id > 0.
     {'device':'pxb', 'expected':True},                     # Bridge chassis not specified. Each bridge is required to be assigned a unique chassis id > 0.
+    {'device':'pxb-cxl', 'expected':True},                 # pxb-cxl devices cannot reside on a PCI bus.
     {'device':'scsi-block', 'expected':True},              # drive property not set
     {'device':'scsi-generic', 'expected':True},            # drive property not set
     {'device':'scsi-hd', 'expected':True},                 # drive property not set

From 33c47ab967558dfb02b3439b37e7dcfcdced0157 Mon Sep 17 00:00:00 2001
From: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Date: Fri, 29 Apr 2022 15:40:40 +0100
Subject: [PATCH 284/935] qtest/cxl: Introduce initial test for pxb-cxl only.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Initial test with just pxb-cxl.  Other tests will be added
alongside functionality.

Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-16-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/cxl-test.c  | 22 ++++++++++++++++++++++
 tests/qtest/meson.build |  4 ++++
 2 files changed, 26 insertions(+)
 create mode 100644 tests/qtest/cxl-test.c

diff --git a/tests/qtest/cxl-test.c b/tests/qtest/cxl-test.c
new file mode 100644
index 0000000000..c031049930
--- /dev/null
+++ b/tests/qtest/cxl-test.c
@@ -0,0 +1,22 @@
+/*
+ * QTest testcase for CXL
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "libqtest-single.h"
+
+static void cxl_basic_pxb(void)
+{
+    qtest_start("-machine q35,cxl=on -device pxb-cxl,bus=pcie.0");
+    qtest_end();
+}
+
+int main(int argc, char **argv)
+{
+    g_test_init(&argc, &argv, NULL);
+    qtest_add_func("/pci/cxl/basic_pxb", cxl_basic_pxb);
+    return g_test_run();
+}
diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
index 3551b9c946..71e86bc2a3 100644
--- a/tests/qtest/meson.build
+++ b/tests/qtest/meson.build
@@ -35,6 +35,9 @@ qtests_pci = \
   (config_all_devices.has_key('CONFIG_VGA') ? ['display-vga-test'] : []) +                  \
   (config_all_devices.has_key('CONFIG_IVSHMEM_DEVICE') ? ['ivshmem-test'] : [])
 
+qtests_cxl = \
+  (config_all_devices.has_key('CONFIG_CXL') ? ['cxl-test'] : [])
+
 qtests_i386 = \
   (slirp.found() ? ['pxe-test', 'test-netfilter'] : []) +             \
   (config_host.has_key('CONFIG_POSIX') ? ['test-filter-mirror'] : []) +                     \
@@ -74,6 +77,7 @@ qtests_i386 = \
    slirp.found() ? ['virtio-net-failover'] : []) +                                          \
   (unpack_edk2_blobs ? ['bios-tables-test'] : []) +                                         \
   qtests_pci +                                                                              \
+  qtests_cxl +                                                                              \
   ['fdc-test',
    'ide-test',
    'hd-geo-test',

From d86d30192b7bc5a10fa6c82c073f55aea25f9291 Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:41 +0100
Subject: [PATCH 285/935] hw/cxl/rp: Add a root port
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This adds just enough of a root port implementation to be able to
enumerate root ports (creating the required DVSEC entries). What's not
here yet is the MMIO nor the ability to write some of the DVSEC entries.

This can be added with the qemu commandline by adding a rootport to a
specific CXL host bridge. For example:
  -device cxl-rp,id=rp0,bus="cxl.0",addr=0.0,chassis=4

Like the host bridge patch, the ACPI tables aren't generated at this
point and so system software cannot use it.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-17-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/pci-bridge/Kconfig          |   5 +
 hw/pci-bridge/cxl_root_port.c  | 236 +++++++++++++++++++++++++++++++++
 hw/pci-bridge/meson.build      |   1 +
 hw/pci-bridge/pcie_root_port.c |   6 +-
 hw/pci/pci.c                   |   4 +-
 5 files changed, 250 insertions(+), 2 deletions(-)
 create mode 100644 hw/pci-bridge/cxl_root_port.c

diff --git a/hw/pci-bridge/Kconfig b/hw/pci-bridge/Kconfig
index f8df4315ba..02614f49aa 100644
--- a/hw/pci-bridge/Kconfig
+++ b/hw/pci-bridge/Kconfig
@@ -27,3 +27,8 @@ config DEC_PCI
 
 config SIMBA
     bool
+
+config CXL
+    bool
+    default y if PCI_EXPRESS && PXB
+    depends on PCI_EXPRESS && MSI_NONBROKEN && PXB
diff --git a/hw/pci-bridge/cxl_root_port.c b/hw/pci-bridge/cxl_root_port.c
new file mode 100644
index 0000000000..fb213fa06e
--- /dev/null
+++ b/hw/pci-bridge/cxl_root_port.c
@@ -0,0 +1,236 @@
+/*
+ * CXL 2.0 Root Port Implementation
+ *
+ * Copyright(C) 2020 Intel Corporation.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "qemu/range.h"
+#include "hw/pci/pci_bridge.h"
+#include "hw/pci/pcie_port.h"
+#include "hw/qdev-properties.h"
+#include "hw/sysbus.h"
+#include "qapi/error.h"
+#include "hw/cxl/cxl.h"
+
+#define CXL_ROOT_PORT_DID 0x7075
+
+/* Copied from the gen root port which we derive */
+#define GEN_PCIE_ROOT_PORT_AER_OFFSET 0x100
+#define GEN_PCIE_ROOT_PORT_ACS_OFFSET \
+    (GEN_PCIE_ROOT_PORT_AER_OFFSET + PCI_ERR_SIZEOF)
+#define CXL_ROOT_PORT_DVSEC_OFFSET \
+    (GEN_PCIE_ROOT_PORT_ACS_OFFSET + PCI_ACS_SIZEOF)
+
+typedef struct CXLRootPort {
+    /*< private >*/
+    PCIESlot parent_obj;
+
+    CXLComponentState cxl_cstate;
+    PCIResReserve res_reserve;
+} CXLRootPort;
+
+#define TYPE_CXL_ROOT_PORT "cxl-rp"
+DECLARE_INSTANCE_CHECKER(CXLRootPort, CXL_ROOT_PORT, TYPE_CXL_ROOT_PORT)
+
+static void latch_registers(CXLRootPort *crp)
+{
+    uint32_t *reg_state = crp->cxl_cstate.crb.cache_mem_registers;
+    uint32_t *write_msk = crp->cxl_cstate.crb.cache_mem_regs_write_mask;
+
+    cxl_component_register_init_common(reg_state, write_msk, CXL2_ROOT_PORT);
+}
+
+static void build_dvsecs(CXLComponentState *cxl)
+{
+    uint8_t *dvsec;
+
+    dvsec = (uint8_t *)&(CXLDVSECPortExtensions){ 0 };
+    cxl_component_create_dvsec(cxl, CXL2_ROOT_PORT,
+                               EXTENSIONS_PORT_DVSEC_LENGTH,
+                               EXTENSIONS_PORT_DVSEC,
+                               EXTENSIONS_PORT_DVSEC_REVID, dvsec);
+
+    dvsec = (uint8_t *)&(CXLDVSECPortGPF){
+        .rsvd        = 0,
+        .phase1_ctrl = 1, /* 1μs timeout */
+        .phase2_ctrl = 1, /* 1μs timeout */
+    };
+    cxl_component_create_dvsec(cxl, CXL2_ROOT_PORT,
+                               GPF_PORT_DVSEC_LENGTH, GPF_PORT_DVSEC,
+                               GPF_PORT_DVSEC_REVID, dvsec);
+
+    dvsec = (uint8_t *)&(CXLDVSECPortFlexBus){
+        .cap                     = 0x26, /* IO, Mem, non-MLD */
+        .ctrl                    = 0x2,
+        .status                  = 0x26, /* same */
+        .rcvd_mod_ts_data_phase1 = 0xef,
+    };
+    cxl_component_create_dvsec(cxl, CXL2_ROOT_PORT,
+                               PCIE_FLEXBUS_PORT_DVSEC_LENGTH_2_0,
+                               PCIE_FLEXBUS_PORT_DVSEC,
+                               PCIE_FLEXBUS_PORT_DVSEC_REVID_2_0, dvsec);
+
+    dvsec = (uint8_t *)&(CXLDVSECRegisterLocator){
+        .rsvd         = 0,
+        .reg0_base_lo = RBI_COMPONENT_REG | CXL_COMPONENT_REG_BAR_IDX,
+        .reg0_base_hi = 0,
+    };
+    cxl_component_create_dvsec(cxl, CXL2_ROOT_PORT,
+                               REG_LOC_DVSEC_LENGTH, REG_LOC_DVSEC,
+                               REG_LOC_DVSEC_REVID, dvsec);
+}
+
+static void cxl_rp_realize(DeviceState *dev, Error **errp)
+{
+    PCIDevice *pci_dev     = PCI_DEVICE(dev);
+    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
+    CXLRootPort *crp       = CXL_ROOT_PORT(dev);
+    CXLComponentState *cxl_cstate = &crp->cxl_cstate;
+    ComponentRegisters *cregs = &cxl_cstate->crb;
+    MemoryRegion *component_bar = &cregs->component_registers;
+    Error *local_err = NULL;
+
+    rpc->parent_realize(dev, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
+    int rc =
+        pci_bridge_qemu_reserve_cap_init(pci_dev, 0, crp->res_reserve, errp);
+    if (rc < 0) {
+        rpc->parent_class.exit(pci_dev);
+        return;
+    }
+
+    if (!crp->res_reserve.io || crp->res_reserve.io == -1) {
+        pci_word_test_and_clear_mask(pci_dev->wmask + PCI_COMMAND,
+                                     PCI_COMMAND_IO);
+        pci_dev->wmask[PCI_IO_BASE]  = 0;
+        pci_dev->wmask[PCI_IO_LIMIT] = 0;
+    }
+
+    cxl_cstate->dvsec_offset = CXL_ROOT_PORT_DVSEC_OFFSET;
+    cxl_cstate->pdev = pci_dev;
+    build_dvsecs(&crp->cxl_cstate);
+
+    cxl_component_register_block_init(OBJECT(pci_dev), cxl_cstate,
+                                      TYPE_CXL_ROOT_PORT);
+
+    pci_register_bar(pci_dev, CXL_COMPONENT_REG_BAR_IDX,
+                     PCI_BASE_ADDRESS_SPACE_MEMORY |
+                         PCI_BASE_ADDRESS_MEM_TYPE_64,
+                     component_bar);
+}
+
+static void cxl_rp_reset(DeviceState *dev)
+{
+    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
+    CXLRootPort *crp = CXL_ROOT_PORT(dev);
+
+    rpc->parent_reset(dev);
+
+    latch_registers(crp);
+}
+
+static Property gen_rp_props[] = {
+    DEFINE_PROP_UINT32("bus-reserve", CXLRootPort, res_reserve.bus, -1),
+    DEFINE_PROP_SIZE("io-reserve", CXLRootPort, res_reserve.io, -1),
+    DEFINE_PROP_SIZE("mem-reserve", CXLRootPort, res_reserve.mem_non_pref, -1),
+    DEFINE_PROP_SIZE("pref32-reserve", CXLRootPort, res_reserve.mem_pref_32,
+                     -1),
+    DEFINE_PROP_SIZE("pref64-reserve", CXLRootPort, res_reserve.mem_pref_64,
+                     -1),
+    DEFINE_PROP_END_OF_LIST()
+};
+
+static void cxl_rp_dvsec_write_config(PCIDevice *dev, uint32_t addr,
+                                      uint32_t val, int len)
+{
+    CXLRootPort *crp = CXL_ROOT_PORT(dev);
+
+    if (range_contains(&crp->cxl_cstate.dvsecs[EXTENSIONS_PORT_DVSEC], addr)) {
+        uint8_t *reg = &dev->config[addr];
+        addr -= crp->cxl_cstate.dvsecs[EXTENSIONS_PORT_DVSEC].lob;
+        if (addr == PORT_CONTROL_OFFSET) {
+            if (pci_get_word(reg) & PORT_CONTROL_UNMASK_SBR) {
+                /* unmask SBR */
+                qemu_log_mask(LOG_UNIMP, "SBR mask control is not supported\n");
+            }
+            if (pci_get_word(reg) & PORT_CONTROL_ALT_MEMID_EN) {
+                /* Alt Memory & ID Space Enable */
+                qemu_log_mask(LOG_UNIMP,
+                              "Alt Memory & ID space is not supported\n");
+            }
+        }
+    }
+}
+
+static void cxl_rp_write_config(PCIDevice *d, uint32_t address, uint32_t val,
+                                int len)
+{
+    uint16_t slt_ctl, slt_sta;
+
+    pcie_cap_slot_get(d, &slt_ctl, &slt_sta);
+    pci_bridge_write_config(d, address, val, len);
+    pcie_cap_flr_write_config(d, address, val, len);
+    pcie_cap_slot_write_config(d, slt_ctl, slt_sta, address, val, len);
+    pcie_aer_write_config(d, address, val, len);
+
+    cxl_rp_dvsec_write_config(d, address, val, len);
+}
+
+static void cxl_root_port_class_init(ObjectClass *oc, void *data)
+{
+    DeviceClass *dc        = DEVICE_CLASS(oc);
+    PCIDeviceClass *k      = PCI_DEVICE_CLASS(oc);
+    PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(oc);
+
+    k->vendor_id = PCI_VENDOR_ID_INTEL;
+    k->device_id = CXL_ROOT_PORT_DID;
+    dc->desc     = "CXL Root Port";
+    k->revision  = 0;
+    device_class_set_props(dc, gen_rp_props);
+    k->config_write = cxl_rp_write_config;
+
+    device_class_set_parent_realize(dc, cxl_rp_realize, &rpc->parent_realize);
+    device_class_set_parent_reset(dc, cxl_rp_reset, &rpc->parent_reset);
+
+    rpc->aer_offset = GEN_PCIE_ROOT_PORT_AER_OFFSET;
+    rpc->acs_offset = GEN_PCIE_ROOT_PORT_ACS_OFFSET;
+
+    dc->hotpluggable = false;
+}
+
+static const TypeInfo cxl_root_port_info = {
+    .name = TYPE_CXL_ROOT_PORT,
+    .parent = TYPE_PCIE_ROOT_PORT,
+    .instance_size = sizeof(CXLRootPort),
+    .class_init = cxl_root_port_class_init,
+    .interfaces = (InterfaceInfo[]) {
+        { INTERFACE_CXL_DEVICE },
+        { }
+    },
+};
+
+static void cxl_register(void)
+{
+    type_register_static(&cxl_root_port_info);
+}
+
+type_init(cxl_register);
diff --git a/hw/pci-bridge/meson.build b/hw/pci-bridge/meson.build
index daab8acf2a..b6d26a03d5 100644
--- a/hw/pci-bridge/meson.build
+++ b/hw/pci-bridge/meson.build
@@ -5,6 +5,7 @@ pci_ss.add(when: 'CONFIG_IOH3420', if_true: files('ioh3420.c'))
 pci_ss.add(when: 'CONFIG_PCIE_PORT', if_true: files('pcie_root_port.c', 'gen_pcie_root_port.c', 'pcie_pci_bridge.c'))
 pci_ss.add(when: 'CONFIG_PXB', if_true: files('pci_expander_bridge.c'))
 pci_ss.add(when: 'CONFIG_XIO3130', if_true: files('xio3130_upstream.c', 'xio3130_downstream.c'))
+pci_ss.add(when: 'CONFIG_CXL', if_true: files('cxl_root_port.c'))
 
 # NewWorld PowerMac
 pci_ss.add(when: 'CONFIG_DEC_PCI', if_true: files('dec.c'))
diff --git a/hw/pci-bridge/pcie_root_port.c b/hw/pci-bridge/pcie_root_port.c
index f1cfe9d14a..460e48269d 100644
--- a/hw/pci-bridge/pcie_root_port.c
+++ b/hw/pci-bridge/pcie_root_port.c
@@ -67,7 +67,11 @@ static void rp_realize(PCIDevice *d, Error **errp)
     int rc;
 
     pci_config_set_interrupt_pin(d->config, 1);
-    pci_bridge_initfn(d, TYPE_PCIE_BUS);
+    if (d->cap_present & QEMU_PCIE_CAP_CXL) {
+        pci_bridge_initfn(d, TYPE_CXL_BUS);
+    } else {
+        pci_bridge_initfn(d, TYPE_PCIE_BUS);
+    }
     pcie_port_init_reg(d);
 
     rc = pci_bridge_ssvid_init(d, rpc->ssvid_offset, dc->vendor_id,
diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index 6c0656f604..a9b37f8000 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -2762,7 +2762,9 @@ static void pci_device_class_base_init(ObjectClass *klass, void *data)
             object_class_dynamic_cast(klass, INTERFACE_CONVENTIONAL_PCI_DEVICE);
         ObjectClass *pcie =
             object_class_dynamic_cast(klass, INTERFACE_PCIE_DEVICE);
-        assert(conventional || pcie);
+        ObjectClass *cxl =
+            object_class_dynamic_cast(klass, INTERFACE_CXL_DEVICE);
+        assert(conventional || pcie || cxl);
     }
 }
 

From e1706ea83da0120be6708b66394ec3a9f3ec48ca Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:42 +0100
Subject: [PATCH 286/935] hw/cxl/device: Add a memory device (8.2.8.5)

A CXL memory device (AKA Type 3) is a CXL component that contains some
combination of volatile and persistent memory. It also implements the
previously defined mailbox interface as well as the memory device
firmware interface.

Although the memory device is configured like a normal PCIe device, the
memory traffic is on an entirely separate bus conceptually (using the
same physical wires as PCIe, but different protocol).

Once the CXL topology is fully configure and address decoders committed,
the guest physical address for the memory device is part of a larger
window which is owned by the platform.  The creation of these windows
is later in this series.

The following example will create a 256M device in a 512M window:
-object "memory-backend-file,id=cxl-mem1,share,mem-path=cxl-type3,size=512M"
-device "cxl-type3,bus=rp0,memdev=cxl-mem1,id=cxl-pmem0"

Note: Dropped PCDIMM info interfaces for now.  They can be added if
appropriate at a later date.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Message-Id: <20220429144110.25167-18-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/cxl/cxl-component-utils.c |  18 ++++
 hw/cxl/cxl-mailbox-utils.c   |  46 ++++++++++
 hw/mem/Kconfig               |   5 ++
 hw/mem/cxl_type3.c           | 162 +++++++++++++++++++++++++++++++++++
 hw/mem/meson.build           |   1 +
 include/hw/cxl/cxl_device.h  |  15 ++++
 include/hw/cxl/cxl_pci.h     |  21 +++++
 include/hw/pci/pci_ids.h     |   1 +
 8 files changed, 269 insertions(+)
 create mode 100644 hw/mem/cxl_type3.c

diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c
index 9457a1b029..afc97b17c2 100644
--- a/hw/cxl/cxl-component-utils.c
+++ b/hw/cxl/cxl-component-utils.c
@@ -252,6 +252,24 @@ void cxl_component_create_dvsec(CXLComponentState *cxl,
     /* Configure write masks */
     switch (type) {
     case PCIE_CXL_DEVICE_DVSEC:
+        /* Cntrl RW Lock - so needs explicit blocking when lock is set */
+        wmask[offset + offsetof(CXLDVSECDevice, ctrl)] = 0xFD;
+        wmask[offset + offsetof(CXLDVSECDevice, ctrl) + 1] = 0x4F;
+        /* Status is RW1CS */
+        wmask[offset + offsetof(CXLDVSECDevice, ctrl2)] = 0x0F;
+       /* Lock is RW Once */
+        wmask[offset + offsetof(CXLDVSECDevice, lock)] = 0x01;
+        /* range1/2_base_high/low is RW Lock */
+        wmask[offset + offsetof(CXLDVSECDevice, range1_base_hi)] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECDevice, range1_base_hi) + 1] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECDevice, range1_base_hi) + 2] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECDevice, range1_base_hi) + 3] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECDevice, range1_base_lo) + 3] = 0xF0;
+        wmask[offset + offsetof(CXLDVSECDevice, range2_base_hi)] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECDevice, range2_base_hi) + 1] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECDevice, range2_base_hi) + 2] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECDevice, range2_base_hi) + 3] = 0xFF;
+        wmask[offset + offsetof(CXLDVSECDevice, range2_base_lo) + 3] = 0xF0;
         break;
     case NON_CXL_FUNCTION_MAP_DVSEC:
         break; /* Not yet implemented */
diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index db473135c7..4ae0561dfc 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -50,6 +50,8 @@ enum {
     LOGS        = 0x04,
         #define GET_SUPPORTED 0x0
         #define GET_LOG       0x1
+    IDENTIFY    = 0x40,
+        #define MEMORY_DEVICE 0x0
 };
 
 /* 8.2.8.4.5.1 Command Return Codes */
@@ -214,6 +216,48 @@ static ret_code cmd_logs_get_log(struct cxl_cmd *cmd,
     return CXL_MBOX_SUCCESS;
 }
 
+/* 8.2.9.5.1.1 */
+static ret_code cmd_identify_memory_device(struct cxl_cmd *cmd,
+                                           CXLDeviceState *cxl_dstate,
+                                           uint16_t *len)
+{
+    struct {
+        char fw_revision[0x10];
+        uint64_t total_capacity;
+        uint64_t volatile_capacity;
+        uint64_t persistent_capacity;
+        uint64_t partition_align;
+        uint16_t info_event_log_size;
+        uint16_t warning_event_log_size;
+        uint16_t failure_event_log_size;
+        uint16_t fatal_event_log_size;
+        uint32_t lsa_size;
+        uint8_t poison_list_max_mer[3];
+        uint16_t inject_poison_limit;
+        uint8_t poison_caps;
+        uint8_t qos_telemetry_caps;
+    } QEMU_PACKED *id;
+    QEMU_BUILD_BUG_ON(sizeof(*id) != 0x43);
+
+    uint64_t size = cxl_dstate->pmem_size;
+
+    if (!QEMU_IS_ALIGNED(size, 256 << 20)) {
+        return CXL_MBOX_INTERNAL_ERROR;
+    }
+
+    id = (void *)cmd->payload;
+    memset(id, 0, sizeof(*id));
+
+    /* PMEM only */
+    snprintf(id->fw_revision, 0x10, "BWFW VERSION %02d", 0);
+
+    id->total_capacity = size / (256 << 20);
+    id->persistent_capacity = size / (256 << 20);
+
+    *len = sizeof(*id);
+    return CXL_MBOX_SUCCESS;
+}
+
 #define IMMEDIATE_CONFIG_CHANGE (1 << 1)
 #define IMMEDIATE_POLICY_CHANGE (1 << 3)
 #define IMMEDIATE_LOG_CHANGE (1 << 4)
@@ -231,6 +275,8 @@ static struct cxl_cmd cxl_cmd_set[256][256] = {
     [TIMESTAMP][SET] = { "TIMESTAMP_SET", cmd_timestamp_set, 8, IMMEDIATE_POLICY_CHANGE },
     [LOGS][GET_SUPPORTED] = { "LOGS_GET_SUPPORTED", cmd_logs_get_supported, 0, 0 },
     [LOGS][GET_LOG] = { "LOGS_GET_LOG", cmd_logs_get_log, 0x18, 0 },
+    [IDENTIFY][MEMORY_DEVICE] = { "IDENTIFY_MEMORY_DEVICE",
+        cmd_identify_memory_device, 0, 0 },
 };
 
 void cxl_process_mailbox(CXLDeviceState *cxl_dstate)
diff --git a/hw/mem/Kconfig b/hw/mem/Kconfig
index 03dbb3c7df..73c5ae8ad9 100644
--- a/hw/mem/Kconfig
+++ b/hw/mem/Kconfig
@@ -11,3 +11,8 @@ config NVDIMM
 
 config SPARSE_MEM
     bool
+
+config CXL_MEM_DEVICE
+    bool
+    default y if CXL
+    select MEM_DEVICE
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
new file mode 100644
index 0000000000..290c088dd6
--- /dev/null
+++ b/hw/mem/cxl_type3.c
@@ -0,0 +1,162 @@
+#include "qemu/osdep.h"
+#include "qemu/units.h"
+#include "qemu/error-report.h"
+#include "hw/mem/memory-device.h"
+#include "hw/mem/pc-dimm.h"
+#include "hw/pci/pci.h"
+#include "hw/qdev-properties.h"
+#include "qapi/error.h"
+#include "qemu/log.h"
+#include "qemu/module.h"
+#include "qemu/range.h"
+#include "qemu/rcu.h"
+#include "sysemu/hostmem.h"
+#include "hw/cxl/cxl.h"
+
+static void build_dvsecs(CXLType3Dev *ct3d)
+{
+    CXLComponentState *cxl_cstate = &ct3d->cxl_cstate;
+    uint8_t *dvsec;
+
+    dvsec = (uint8_t *)&(CXLDVSECDevice){
+        .cap = 0x1e,
+        .ctrl = 0x2,
+        .status2 = 0x2,
+        .range1_size_hi = ct3d->hostmem->size >> 32,
+        .range1_size_lo = (2 << 5) | (2 << 2) | 0x3 |
+        (ct3d->hostmem->size & 0xF0000000),
+        .range1_base_hi = 0,
+        .range1_base_lo = 0,
+    };
+    cxl_component_create_dvsec(cxl_cstate, CXL2_TYPE3_DEVICE,
+                               PCIE_CXL_DEVICE_DVSEC_LENGTH,
+                               PCIE_CXL_DEVICE_DVSEC,
+                               PCIE_CXL2_DEVICE_DVSEC_REVID, dvsec);
+
+    dvsec = (uint8_t *)&(CXLDVSECRegisterLocator){
+        .rsvd         = 0,
+        .reg0_base_lo = RBI_COMPONENT_REG | CXL_COMPONENT_REG_BAR_IDX,
+        .reg0_base_hi = 0,
+        .reg1_base_lo = RBI_CXL_DEVICE_REG | CXL_DEVICE_REG_BAR_IDX,
+        .reg1_base_hi = 0,
+    };
+    cxl_component_create_dvsec(cxl_cstate, CXL2_TYPE3_DEVICE,
+                               REG_LOC_DVSEC_LENGTH, REG_LOC_DVSEC,
+                               REG_LOC_DVSEC_REVID, dvsec);
+    dvsec = (uint8_t *)&(CXLDVSECDeviceGPF){
+        .phase2_duration = 0x603, /* 3 seconds */
+        .phase2_power = 0x33, /* 0x33 miliwatts */
+    };
+    cxl_component_create_dvsec(cxl_cstate, CXL2_TYPE3_DEVICE,
+                               GPF_DEVICE_DVSEC_LENGTH, GPF_PORT_DVSEC,
+                               GPF_DEVICE_DVSEC_REVID, dvsec);
+}
+
+static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
+{
+    MemoryRegion *mr;
+
+    if (!ct3d->hostmem) {
+        error_setg(errp, "memdev property must be set");
+        return false;
+    }
+
+    mr = host_memory_backend_get_memory(ct3d->hostmem);
+    if (!mr) {
+        error_setg(errp, "memdev property must be set");
+        return false;
+    }
+    memory_region_set_nonvolatile(mr, true);
+    memory_region_set_enabled(mr, true);
+    host_memory_backend_set_mapped(ct3d->hostmem, true);
+    ct3d->cxl_dstate.pmem_size = ct3d->hostmem->size;
+
+    return true;
+}
+
+static void ct3_realize(PCIDevice *pci_dev, Error **errp)
+{
+    CXLType3Dev *ct3d = CXL_TYPE3(pci_dev);
+    CXLComponentState *cxl_cstate = &ct3d->cxl_cstate;
+    ComponentRegisters *regs = &cxl_cstate->crb;
+    MemoryRegion *mr = &regs->component_registers;
+    uint8_t *pci_conf = pci_dev->config;
+
+    if (!cxl_setup_memory(ct3d, errp)) {
+        return;
+    }
+
+    pci_config_set_prog_interface(pci_conf, 0x10);
+    pci_config_set_class(pci_conf, PCI_CLASS_MEMORY_CXL);
+
+    pcie_endpoint_cap_init(pci_dev, 0x80);
+    cxl_cstate->dvsec_offset = 0x100;
+
+    ct3d->cxl_cstate.pdev = pci_dev;
+    build_dvsecs(ct3d);
+
+    cxl_component_register_block_init(OBJECT(pci_dev), cxl_cstate,
+                                      TYPE_CXL_TYPE3);
+
+    pci_register_bar(
+        pci_dev, CXL_COMPONENT_REG_BAR_IDX,
+        PCI_BASE_ADDRESS_SPACE_MEMORY | PCI_BASE_ADDRESS_MEM_TYPE_64, mr);
+
+    cxl_device_register_block_init(OBJECT(pci_dev), &ct3d->cxl_dstate);
+    pci_register_bar(pci_dev, CXL_DEVICE_REG_BAR_IDX,
+                     PCI_BASE_ADDRESS_SPACE_MEMORY |
+                         PCI_BASE_ADDRESS_MEM_TYPE_64,
+                     &ct3d->cxl_dstate.device_registers);
+}
+
+static void ct3d_reset(DeviceState *dev)
+{
+    CXLType3Dev *ct3d = CXL_TYPE3(dev);
+    uint32_t *reg_state = ct3d->cxl_cstate.crb.cache_mem_registers;
+    uint32_t *write_msk = ct3d->cxl_cstate.crb.cache_mem_regs_write_mask;
+
+    cxl_component_register_init_common(reg_state, write_msk, CXL2_TYPE3_DEVICE);
+    cxl_device_register_init_common(&ct3d->cxl_dstate);
+}
+
+static Property ct3_props[] = {
+    DEFINE_PROP_LINK("memdev", CXLType3Dev, hostmem, TYPE_MEMORY_BACKEND,
+                     HostMemoryBackend *),
+    DEFINE_PROP_END_OF_LIST(),
+};
+
+static void ct3_class_init(ObjectClass *oc, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(oc);
+    PCIDeviceClass *pc = PCI_DEVICE_CLASS(oc);
+
+    pc->realize = ct3_realize;
+    pc->class_id = PCI_CLASS_STORAGE_EXPRESS;
+    pc->vendor_id = PCI_VENDOR_ID_INTEL;
+    pc->device_id = 0xd93; /* LVF for now */
+    pc->revision = 1;
+
+    set_bit(DEVICE_CATEGORY_STORAGE, dc->categories);
+    dc->desc = "CXL PMEM Device (Type 3)";
+    dc->reset = ct3d_reset;
+    device_class_set_props(dc, ct3_props);
+}
+
+static const TypeInfo ct3d_info = {
+    .name = TYPE_CXL_TYPE3,
+    .parent = TYPE_PCI_DEVICE,
+    .class_init = ct3_class_init,
+    .instance_size = sizeof(CXLType3Dev),
+    .interfaces = (InterfaceInfo[]) {
+        { INTERFACE_CXL_DEVICE },
+        { INTERFACE_PCIE_DEVICE },
+        {}
+    },
+};
+
+static void ct3d_registers(void)
+{
+    type_register_static(&ct3d_info);
+}
+
+type_init(ct3d_registers);
diff --git a/hw/mem/meson.build b/hw/mem/meson.build
index 82f86d117e..609b2b36fc 100644
--- a/hw/mem/meson.build
+++ b/hw/mem/meson.build
@@ -3,6 +3,7 @@ mem_ss.add(files('memory-device.c'))
 mem_ss.add(when: 'CONFIG_DIMM', if_true: files('pc-dimm.c'))
 mem_ss.add(when: 'CONFIG_NPCM7XX', if_true: files('npcm7xx_mc.c'))
 mem_ss.add(when: 'CONFIG_NVDIMM', if_true: files('nvdimm.c'))
+mem_ss.add(when: 'CONFIG_CXL_MEM_DEVICE', if_true: files('cxl_type3.c'))
 
 softmmu_ss.add_all(when: 'CONFIG_MEM_DEVICE', if_true: mem_ss)
 
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 797a22ddb4..d8da2c7b68 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -230,4 +230,19 @@ REG64(CXL_MEM_DEV_STS, 0)
     FIELD(CXL_MEM_DEV_STS, MBOX_READY, 4, 1)
     FIELD(CXL_MEM_DEV_STS, RESET_NEEDED, 5, 3)
 
+struct CXLType3Dev {
+    /* Private */
+    PCIDevice parent_obj;
+
+    /* Properties */
+    HostMemoryBackend *hostmem;
+
+    /* State */
+    CXLComponentState cxl_cstate;
+    CXLDeviceState cxl_dstate;
+};
+
+#define TYPE_CXL_TYPE3 "cxl-type3"
+OBJECT_DECLARE_SIMPLE_TYPE(CXLType3Dev, CXL_TYPE3)
+
 #endif
diff --git a/include/hw/cxl/cxl_pci.h b/include/hw/cxl/cxl_pci.h
index 3f7e44daac..01cf002096 100644
--- a/include/hw/cxl/cxl_pci.h
+++ b/include/hw/cxl/cxl_pci.h
@@ -67,6 +67,27 @@ QEMU_BUILD_BUG_ON(sizeof(DVSECHeader) != 10);
  * CXL 2.0 Downstream Port: 3, 4, 7, 8
  */
 
+/* CXL 2.0 - 8.1.3 (ID 0001) */
+typedef struct CXLDVSECDevice {
+    DVSECHeader hdr;
+    uint16_t cap;
+    uint16_t ctrl;
+    uint16_t status;
+    uint16_t ctrl2;
+    uint16_t status2;
+    uint16_t lock;
+    uint16_t cap2;
+    uint32_t range1_size_hi;
+    uint32_t range1_size_lo;
+    uint32_t range1_base_hi;
+    uint32_t range1_base_lo;
+    uint32_t range2_size_hi;
+    uint32_t range2_size_lo;
+    uint32_t range2_base_hi;
+    uint32_t range2_base_lo;
+} CXLDVSECDevice;
+QEMU_BUILD_BUG_ON(sizeof(CXLDVSECDevice) != 0x38);
+
 /* CXL 2.0 - 8.1.5 (ID 0003) */
 typedef struct CXLDVSECPortExtensions {
     DVSECHeader hdr;
diff --git a/include/hw/pci/pci_ids.h b/include/hw/pci/pci_ids.h
index 11abe22d46..898083b86f 100644
--- a/include/hw/pci/pci_ids.h
+++ b/include/hw/pci/pci_ids.h
@@ -53,6 +53,7 @@
 #define PCI_BASE_CLASS_MEMORY            0x05
 #define PCI_CLASS_MEMORY_RAM             0x0500
 #define PCI_CLASS_MEMORY_FLASH           0x0501
+#define PCI_CLASS_MEMORY_CXL             0x0502
 #define PCI_CLASS_MEMORY_OTHER           0x0580
 
 #define PCI_BASE_CLASS_BRIDGE            0x06

From 947515fc4274bd46cc93f82c3121df6ee4968786 Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:43 +0100
Subject: [PATCH 287/935] hw/cxl/device: Implement MMIO HDM decoding (8.2.5.12)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

A device's volatile and persistent memory are known Host Defined Memory
(HDM) regions. The mechanism by which the device is programmed to claim
the addresses associated with those regions is through dedicated logic
known as the HDM decoder. In order to allow the OS to properly program
the HDMs, the HDM decoders must be modeled.

There are two ways the HDM decoders can be implemented, the legacy
mechanism is through the PCIe DVSEC programming from CXL 1.1 (8.1.3.8),
and MMIO is found in 8.2.5.12 of the spec. For now, 8.1.3.8 is not
implemented.

Much of CXL device logic is implemented in cxl-utils. The HDM decoder
however is implemented directly by the device implementation.
Whilst the implementation currently does no validity checks on the
encoder set up, future work will add sanity checking specific to
the type of cxl component.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Co-developed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-19-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/mem/cxl_type3.c | 55 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 290c088dd6..318b6f9fbc 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -52,6 +52,48 @@ static void build_dvsecs(CXLType3Dev *ct3d)
                                GPF_DEVICE_DVSEC_REVID, dvsec);
 }
 
+static void hdm_decoder_commit(CXLType3Dev *ct3d, int which)
+{
+    ComponentRegisters *cregs = &ct3d->cxl_cstate.crb;
+    uint32_t *cache_mem = cregs->cache_mem_registers;
+
+    assert(which == 0);
+
+    /* TODO: Sanity checks that the decoder is possible */
+    ARRAY_FIELD_DP32(cache_mem, CXL_HDM_DECODER0_CTRL, COMMIT, 0);
+    ARRAY_FIELD_DP32(cache_mem, CXL_HDM_DECODER0_CTRL, ERR, 0);
+
+    ARRAY_FIELD_DP32(cache_mem, CXL_HDM_DECODER0_CTRL, COMMITTED, 1);
+}
+
+static void ct3d_reg_write(void *opaque, hwaddr offset, uint64_t value,
+                           unsigned size)
+{
+    CXLComponentState *cxl_cstate = opaque;
+    ComponentRegisters *cregs = &cxl_cstate->crb;
+    CXLType3Dev *ct3d = container_of(cxl_cstate, CXLType3Dev, cxl_cstate);
+    uint32_t *cache_mem = cregs->cache_mem_registers;
+    bool should_commit = false;
+    int which_hdm = -1;
+
+    assert(size == 4);
+    g_assert(offset < CXL2_COMPONENT_CM_REGION_SIZE);
+
+    switch (offset) {
+    case A_CXL_HDM_DECODER0_CTRL:
+        should_commit = FIELD_EX32(value, CXL_HDM_DECODER0_CTRL, COMMIT);
+        which_hdm = 0;
+        break;
+    default:
+        break;
+    }
+
+    stl_le_p((uint8_t *)cache_mem + offset, value);
+    if (should_commit) {
+        hdm_decoder_commit(ct3d, which_hdm);
+    }
+}
+
 static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
 {
     MemoryRegion *mr;
@@ -95,6 +137,9 @@ static void ct3_realize(PCIDevice *pci_dev, Error **errp)
     ct3d->cxl_cstate.pdev = pci_dev;
     build_dvsecs(ct3d);
 
+    regs->special_ops = g_new0(MemoryRegionOps, 1);
+    regs->special_ops->write = ct3d_reg_write;
+
     cxl_component_register_block_init(OBJECT(pci_dev), cxl_cstate,
                                       TYPE_CXL_TYPE3);
 
@@ -109,6 +154,15 @@ static void ct3_realize(PCIDevice *pci_dev, Error **errp)
                      &ct3d->cxl_dstate.device_registers);
 }
 
+static void ct3_exit(PCIDevice *pci_dev)
+{
+    CXLType3Dev *ct3d = CXL_TYPE3(pci_dev);
+    CXLComponentState *cxl_cstate = &ct3d->cxl_cstate;
+    ComponentRegisters *regs = &cxl_cstate->crb;
+
+    g_free(regs->special_ops);
+}
+
 static void ct3d_reset(DeviceState *dev)
 {
     CXLType3Dev *ct3d = CXL_TYPE3(dev);
@@ -131,6 +185,7 @@ static void ct3_class_init(ObjectClass *oc, void *data)
     PCIDeviceClass *pc = PCI_DEVICE_CLASS(oc);
 
     pc->realize = ct3_realize;
+    pc->exit = ct3_exit;
     pc->class_id = PCI_CLASS_STORAGE_EXPRESS;
     pc->vendor_id = PCI_VENDOR_ID_INTEL;
     pc->device_id = 0xd93; /* LVF for now */

From 092c6b11f2ab835f50b3167d0559c197a21a051e Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:44 +0100
Subject: [PATCH 288/935] hw/cxl/device: Add some trivial commands

GET_FW_INFO and GET_PARTITION_INFO, for this emulation, is equivalent to
info already returned in the IDENTIFY command. To have a more robust
implementation, add those.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Message-Id: <20220429144110.25167-20-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/cxl/cxl-mailbox-utils.c | 69 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 69 insertions(+)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 4ae0561dfc..c8188d7087 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -10,6 +10,7 @@
 #include "qemu/osdep.h"
 #include "hw/cxl/cxl.h"
 #include "hw/pci/pci.h"
+#include "qemu/cutils.h"
 #include "qemu/log.h"
 #include "qemu/uuid.h"
 
@@ -44,6 +45,8 @@ enum {
         #define CLEAR_RECORDS   0x1
         #define GET_INTERRUPT_POLICY   0x2
         #define SET_INTERRUPT_POLICY   0x3
+    FIRMWARE_UPDATE = 0x02,
+        #define GET_INFO      0x0
     TIMESTAMP   = 0x03,
         #define GET           0x0
         #define SET           0x1
@@ -52,6 +55,8 @@ enum {
         #define GET_LOG       0x1
     IDENTIFY    = 0x40,
         #define MEMORY_DEVICE 0x0
+    CCLS        = 0x41,
+        #define GET_PARTITION_INFO     0x0
 };
 
 /* 8.2.8.4.5.1 Command Return Codes */
@@ -114,6 +119,39 @@ DEFINE_MAILBOX_HANDLER_NOP(events_clear_records);
 DEFINE_MAILBOX_HANDLER_ZEROED(events_get_interrupt_policy, 4);
 DEFINE_MAILBOX_HANDLER_NOP(events_set_interrupt_policy);
 
+/* 8.2.9.2.1 */
+static ret_code cmd_firmware_update_get_info(struct cxl_cmd *cmd,
+                                             CXLDeviceState *cxl_dstate,
+                                             uint16_t *len)
+{
+    struct {
+        uint8_t slots_supported;
+        uint8_t slot_info;
+        uint8_t caps;
+        uint8_t rsvd[0xd];
+        char fw_rev1[0x10];
+        char fw_rev2[0x10];
+        char fw_rev3[0x10];
+        char fw_rev4[0x10];
+    } QEMU_PACKED *fw_info;
+    QEMU_BUILD_BUG_ON(sizeof(*fw_info) != 0x50);
+
+    if (cxl_dstate->pmem_size < (256 << 20)) {
+        return CXL_MBOX_INTERNAL_ERROR;
+    }
+
+    fw_info = (void *)cmd->payload;
+    memset(fw_info, 0, sizeof(*fw_info));
+
+    fw_info->slots_supported = 2;
+    fw_info->slot_info = BIT(0) | BIT(3);
+    fw_info->caps = 0;
+    pstrcpy(fw_info->fw_rev1, sizeof(fw_info->fw_rev1), "BWFW VERSION 0");
+
+    *len = sizeof(*fw_info);
+    return CXL_MBOX_SUCCESS;
+}
+
 /* 8.2.9.3.1 */
 static ret_code cmd_timestamp_get(struct cxl_cmd *cmd,
                                   CXLDeviceState *cxl_dstate,
@@ -258,6 +296,33 @@ static ret_code cmd_identify_memory_device(struct cxl_cmd *cmd,
     return CXL_MBOX_SUCCESS;
 }
 
+static ret_code cmd_ccls_get_partition_info(struct cxl_cmd *cmd,
+                                           CXLDeviceState *cxl_dstate,
+                                           uint16_t *len)
+{
+    struct {
+        uint64_t active_vmem;
+        uint64_t active_pmem;
+        uint64_t next_vmem;
+        uint64_t next_pmem;
+    } QEMU_PACKED *part_info = (void *)cmd->payload;
+    QEMU_BUILD_BUG_ON(sizeof(*part_info) != 0x20);
+    uint64_t size = cxl_dstate->pmem_size;
+
+    if (!QEMU_IS_ALIGNED(size, 256 << 20)) {
+        return CXL_MBOX_INTERNAL_ERROR;
+    }
+
+    /* PMEM only */
+    part_info->active_vmem = 0;
+    part_info->next_vmem = 0;
+    part_info->active_pmem = size / (256 << 20);
+    part_info->next_pmem = 0;
+
+    *len = sizeof(*part_info);
+    return CXL_MBOX_SUCCESS;
+}
+
 #define IMMEDIATE_CONFIG_CHANGE (1 << 1)
 #define IMMEDIATE_POLICY_CHANGE (1 << 3)
 #define IMMEDIATE_LOG_CHANGE (1 << 4)
@@ -271,12 +336,16 @@ static struct cxl_cmd cxl_cmd_set[256][256] = {
         cmd_events_get_interrupt_policy, 0, 0 },
     [EVENTS][SET_INTERRUPT_POLICY] = { "EVENTS_SET_INTERRUPT_POLICY",
         cmd_events_set_interrupt_policy, 4, IMMEDIATE_CONFIG_CHANGE },
+    [FIRMWARE_UPDATE][GET_INFO] = { "FIRMWARE_UPDATE_GET_INFO",
+        cmd_firmware_update_get_info, 0, 0 },
     [TIMESTAMP][GET] = { "TIMESTAMP_GET", cmd_timestamp_get, 0, 0 },
     [TIMESTAMP][SET] = { "TIMESTAMP_SET", cmd_timestamp_set, 8, IMMEDIATE_POLICY_CHANGE },
     [LOGS][GET_SUPPORTED] = { "LOGS_GET_SUPPORTED", cmd_logs_get_supported, 0, 0 },
     [LOGS][GET_LOG] = { "LOGS_GET_LOG", cmd_logs_get_log, 0x18, 0 },
     [IDENTIFY][MEMORY_DEVICE] = { "IDENTIFY_MEMORY_DEVICE",
         cmd_identify_memory_device, 0, 0 },
+    [CCLS][GET_PARTITION_INFO] = { "CCLS_GET_PARTITION_INFO",
+        cmd_ccls_get_partition_info, 0, 0 },
 };
 
 void cxl_process_mailbox(CXLDeviceState *cxl_dstate)

From 639daf8e93bcf266d0518eecbcfe12d26644a0a9 Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:45 +0100
Subject: [PATCH 289/935] hw/cxl/device: Plumb real Label Storage Area (LSA)
 sizing

This should introduce no change. Subsequent work will make use of this
new class member.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Message-Id: <20220429144110.25167-21-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/cxl/cxl-mailbox-utils.c  |  3 +++
 hw/mem/cxl_type3.c          |  9 +++++++++
 include/hw/cxl/cxl_device.h | 11 ++++++++++-
 3 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index c8188d7087..492739aef3 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -277,6 +277,8 @@ static ret_code cmd_identify_memory_device(struct cxl_cmd *cmd,
     } QEMU_PACKED *id;
     QEMU_BUILD_BUG_ON(sizeof(*id) != 0x43);
 
+    CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
+    CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
     uint64_t size = cxl_dstate->pmem_size;
 
     if (!QEMU_IS_ALIGNED(size, 256 << 20)) {
@@ -291,6 +293,7 @@ static ret_code cmd_identify_memory_device(struct cxl_cmd *cmd,
 
     id->total_capacity = size / (256 << 20);
     id->persistent_capacity = size / (256 << 20);
+    id->lsa_size = cvc->get_lsa_size(ct3d);
 
     *len = sizeof(*id);
     return CXL_MBOX_SUCCESS;
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 318b6f9fbc..b5aa1067df 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -179,10 +179,16 @@ static Property ct3_props[] = {
     DEFINE_PROP_END_OF_LIST(),
 };
 
+static uint64_t get_lsa_size(CXLType3Dev *ct3d)
+{
+    return 0;
+}
+
 static void ct3_class_init(ObjectClass *oc, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(oc);
     PCIDeviceClass *pc = PCI_DEVICE_CLASS(oc);
+    CXLType3Class *cvc = CXL_TYPE3_CLASS(oc);
 
     pc->realize = ct3_realize;
     pc->exit = ct3_exit;
@@ -195,11 +201,14 @@ static void ct3_class_init(ObjectClass *oc, void *data)
     dc->desc = "CXL PMEM Device (Type 3)";
     dc->reset = ct3d_reset;
     device_class_set_props(dc, ct3_props);
+
+    cvc->get_lsa_size = get_lsa_size;
 }
 
 static const TypeInfo ct3d_info = {
     .name = TYPE_CXL_TYPE3,
     .parent = TYPE_PCI_DEVICE,
+    .class_size = sizeof(struct CXLType3Class),
     .class_init = ct3_class_init,
     .instance_size = sizeof(CXLType3Dev),
     .interfaces = (InterfaceInfo[]) {
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index d8da2c7b68..ea2571a69b 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -236,6 +236,7 @@ struct CXLType3Dev {
 
     /* Properties */
     HostMemoryBackend *hostmem;
+    HostMemoryBackend *lsa;
 
     /* State */
     CXLComponentState cxl_cstate;
@@ -243,6 +244,14 @@ struct CXLType3Dev {
 };
 
 #define TYPE_CXL_TYPE3 "cxl-type3"
-OBJECT_DECLARE_SIMPLE_TYPE(CXLType3Dev, CXL_TYPE3)
+OBJECT_DECLARE_TYPE(CXLType3Dev, CXLType3Class, CXL_TYPE3)
+
+struct CXLType3Class {
+    /* Private */
+    PCIDeviceClass parent_class;
+
+    /* public */
+    uint64_t (*get_lsa_size)(CXLType3Dev *ct3d);
+};
 
 #endif

From 3ebe676a3463b886cfc112b3eff58e4991051b0d Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:46 +0100
Subject: [PATCH 290/935] hw/cxl/device: Implement get/set Label Storage Area
 (LSA)

Implement get and set handlers for the Label Storage Area
used to hold data describing persistent memory configuration
so that it can be ensured it is seen in the same configuration
after reboot.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Message-Id: <20220429144110.25167-22-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/cxl/cxl-mailbox-utils.c  | 60 +++++++++++++++++++++++++++++++++++++
 hw/mem/cxl_type3.c          | 56 +++++++++++++++++++++++++++++++++-
 include/hw/cxl/cxl_device.h |  5 ++++
 3 files changed, 120 insertions(+), 1 deletion(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 492739aef3..bb66c765a5 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -57,6 +57,8 @@ enum {
         #define MEMORY_DEVICE 0x0
     CCLS        = 0x41,
         #define GET_PARTITION_INFO     0x0
+        #define GET_LSA       0x2
+        #define SET_LSA       0x3
 };
 
 /* 8.2.8.4.5.1 Command Return Codes */
@@ -326,7 +328,62 @@ static ret_code cmd_ccls_get_partition_info(struct cxl_cmd *cmd,
     return CXL_MBOX_SUCCESS;
 }
 
+static ret_code cmd_ccls_get_lsa(struct cxl_cmd *cmd,
+                                 CXLDeviceState *cxl_dstate,
+                                 uint16_t *len)
+{
+    struct {
+        uint32_t offset;
+        uint32_t length;
+    } QEMU_PACKED *get_lsa;
+    CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
+    CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
+    uint32_t offset, length;
+
+    get_lsa = (void *)cmd->payload;
+    offset = get_lsa->offset;
+    length = get_lsa->length;
+
+    if (offset + length > cvc->get_lsa_size(ct3d)) {
+        *len = 0;
+        return CXL_MBOX_INVALID_INPUT;
+    }
+
+    *len = cvc->get_lsa(ct3d, get_lsa, length, offset);
+    return CXL_MBOX_SUCCESS;
+}
+
+static ret_code cmd_ccls_set_lsa(struct cxl_cmd *cmd,
+                                 CXLDeviceState *cxl_dstate,
+                                 uint16_t *len)
+{
+    struct set_lsa_pl {
+        uint32_t offset;
+        uint32_t rsvd;
+        uint8_t data[];
+    } QEMU_PACKED;
+    struct set_lsa_pl *set_lsa_payload = (void *)cmd->payload;
+    CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
+    CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
+    const size_t hdr_len = offsetof(struct set_lsa_pl, data);
+    uint16_t plen = *len;
+
+    *len = 0;
+    if (!plen) {
+        return CXL_MBOX_SUCCESS;
+    }
+
+    if (set_lsa_payload->offset + plen > cvc->get_lsa_size(ct3d) + hdr_len) {
+        return CXL_MBOX_INVALID_INPUT;
+    }
+    plen -= hdr_len;
+
+    cvc->set_lsa(ct3d, set_lsa_payload->data, plen, set_lsa_payload->offset);
+    return CXL_MBOX_SUCCESS;
+}
+
 #define IMMEDIATE_CONFIG_CHANGE (1 << 1)
+#define IMMEDIATE_DATA_CHANGE (1 << 2)
 #define IMMEDIATE_POLICY_CHANGE (1 << 3)
 #define IMMEDIATE_LOG_CHANGE (1 << 4)
 
@@ -349,6 +406,9 @@ static struct cxl_cmd cxl_cmd_set[256][256] = {
         cmd_identify_memory_device, 0, 0 },
     [CCLS][GET_PARTITION_INFO] = { "CCLS_GET_PARTITION_INFO",
         cmd_ccls_get_partition_info, 0, 0 },
+    [CCLS][GET_LSA] = { "CCLS_GET_LSA", cmd_ccls_get_lsa, 0, 0 },
+    [CCLS][SET_LSA] = { "CCLS_SET_LSA", cmd_ccls_set_lsa,
+        ~0, IMMEDIATE_CONFIG_CHANGE | IMMEDIATE_DATA_CHANGE },
 };
 
 void cxl_process_mailbox(CXLDeviceState *cxl_dstate)
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index b5aa1067df..6c6ed9a776 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -8,6 +8,7 @@
 #include "qapi/error.h"
 #include "qemu/log.h"
 #include "qemu/module.h"
+#include "qemu/pmem.h"
 #include "qemu/range.h"
 #include "qemu/rcu.h"
 #include "sysemu/hostmem.h"
@@ -113,6 +114,11 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
     host_memory_backend_set_mapped(ct3d->hostmem, true);
     ct3d->cxl_dstate.pmem_size = ct3d->hostmem->size;
 
+    if (!ct3d->lsa) {
+        error_setg(errp, "lsa property must be set");
+        return false;
+    }
+
     return true;
 }
 
@@ -176,12 +182,58 @@ static void ct3d_reset(DeviceState *dev)
 static Property ct3_props[] = {
     DEFINE_PROP_LINK("memdev", CXLType3Dev, hostmem, TYPE_MEMORY_BACKEND,
                      HostMemoryBackend *),
+    DEFINE_PROP_LINK("lsa", CXLType3Dev, lsa, TYPE_MEMORY_BACKEND,
+                     HostMemoryBackend *),
     DEFINE_PROP_END_OF_LIST(),
 };
 
 static uint64_t get_lsa_size(CXLType3Dev *ct3d)
 {
-    return 0;
+    MemoryRegion *mr;
+
+    mr = host_memory_backend_get_memory(ct3d->lsa);
+    return memory_region_size(mr);
+}
+
+static void validate_lsa_access(MemoryRegion *mr, uint64_t size,
+                                uint64_t offset)
+{
+    assert(offset + size <= memory_region_size(mr));
+    assert(offset + size > offset);
+}
+
+static uint64_t get_lsa(CXLType3Dev *ct3d, void *buf, uint64_t size,
+                    uint64_t offset)
+{
+    MemoryRegion *mr;
+    void *lsa;
+
+    mr = host_memory_backend_get_memory(ct3d->lsa);
+    validate_lsa_access(mr, size, offset);
+
+    lsa = memory_region_get_ram_ptr(mr) + offset;
+    memcpy(buf, lsa, size);
+
+    return size;
+}
+
+static void set_lsa(CXLType3Dev *ct3d, const void *buf, uint64_t size,
+                    uint64_t offset)
+{
+    MemoryRegion *mr;
+    void *lsa;
+
+    mr = host_memory_backend_get_memory(ct3d->lsa);
+    validate_lsa_access(mr, size, offset);
+
+    lsa = memory_region_get_ram_ptr(mr) + offset;
+    memcpy(lsa, buf, size);
+    memory_region_set_dirty(mr, offset, size);
+
+    /*
+     * Just like the PMEM, if the guest is not allowed to exit gracefully, label
+     * updates will get lost.
+     */
 }
 
 static void ct3_class_init(ObjectClass *oc, void *data)
@@ -203,6 +255,8 @@ static void ct3_class_init(ObjectClass *oc, void *data)
     device_class_set_props(dc, ct3_props);
 
     cvc->get_lsa_size = get_lsa_size;
+    cvc->get_lsa = get_lsa;
+    cvc->set_lsa = set_lsa;
 }
 
 static const TypeInfo ct3d_info = {
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index ea2571a69b..4285fbda08 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -252,6 +252,11 @@ struct CXLType3Class {
 
     /* public */
     uint64_t (*get_lsa_size)(CXLType3Dev *ct3d);
+
+    uint64_t (*get_lsa)(CXLType3Dev *ct3d, void *buf, uint64_t size,
+                        uint64_t offset);
+    void (*set_lsa)(CXLType3Dev *ct3d, const void *buf, uint64_t size,
+                    uint64_t offset);
 };
 
 #endif

From 92fd46b68a0ca2f37d46fd69616034b2703966b2 Mon Sep 17 00:00:00 2001
From: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Date: Fri, 29 Apr 2022 15:40:47 +0100
Subject: [PATCH 291/935] qtests/cxl: Add initial root port and CXL type3 tests
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

At this stage we can boot configurations with host bridges,
root ports and type 3 memory devices, so add appropriate
tests.

Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-23-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/cxl-test.c | 127 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 127 insertions(+)

diff --git a/tests/qtest/cxl-test.c b/tests/qtest/cxl-test.c
index c031049930..5f0794e816 100644
--- a/tests/qtest/cxl-test.c
+++ b/tests/qtest/cxl-test.c
@@ -8,15 +8,142 @@
 #include "qemu/osdep.h"
 #include "libqtest-single.h"
 
+#define QEMU_PXB_CMD "-machine q35,cxl=on " \
+                     "-device pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52 "
+
+#define QEMU_2PXB_CMD "-machine q35,cxl=on " \
+                      "-device pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52 "  \
+                      "-device pxb-cxl,id=cxl.1,bus=pcie.0,bus_nr=53 "
+
+#define QEMU_RP "-device cxl-rp,id=rp0,bus=cxl.0,chassis=0,slot=0 "
+
+/* Dual ports on first pxb */
+#define QEMU_2RP "-device cxl-rp,id=rp0,bus=cxl.0,chassis=0,slot=0 " \
+                 "-device cxl-rp,id=rp1,bus=cxl.0,chassis=0,slot=1 "
+
+/* Dual ports on each of the pxb instances */
+#define QEMU_4RP "-device cxl-rp,id=rp0,bus=cxl.0,chassis=0,slot=0 " \
+                 "-device cxl-rp,id=rp1,bus=cxl.0,chassis=0,slot=1 " \
+                 "-device cxl-rp,id=rp2,bus=cxl.1,chassis=0,slot=2 " \
+                 "-device cxl-rp,id=rp3,bus=cxl.1,chassis=0,slot=3 "
+
+#define QEMU_T3D "-object memory-backend-file,id=cxl-mem0,mem-path=%s,size=256M " \
+                 "-object memory-backend-file,id=lsa0,mem-path=%s,size=256M "    \
+                 "-device cxl-type3,bus=rp0,memdev=cxl-mem0,lsa=lsa0,id=cxl-pmem0 "
+
+#define QEMU_2T3D "-object memory-backend-file,id=cxl-mem0,mem-path=%s,size=256M "    \
+                  "-object memory-backend-file,id=lsa0,mem-path=%s,size=256M "    \
+                  "-device cxl-type3,bus=rp0,memdev=cxl-mem0,lsa=lsa0,id=cxl-pmem0 " \
+                  "-object memory-backend-file,id=cxl-mem1,mem-path=%s,size=256M "    \
+                  "-object memory-backend-file,id=lsa1,mem-path=%s,size=256M "    \
+                  "-device cxl-type3,bus=rp1,memdev=cxl-mem1,lsa=lsa1,id=cxl-pmem1 "
+
+#define QEMU_4T3D "-object memory-backend-file,id=cxl-mem0,mem-path=%s,size=256M " \
+                  "-object memory-backend-file,id=lsa0,mem-path=%s,size=256M "    \
+                  "-device cxl-type3,bus=rp0,memdev=cxl-mem0,lsa=lsa0,id=cxl-pmem0 " \
+                  "-object memory-backend-file,id=cxl-mem1,mem-path=%s,size=256M "    \
+                  "-object memory-backend-file,id=lsa1,mem-path=%s,size=256M "    \
+                  "-device cxl-type3,bus=rp1,memdev=cxl-mem1,lsa=lsa1,id=cxl-pmem1 " \
+                  "-object memory-backend-file,id=cxl-mem2,mem-path=%s,size=256M "    \
+                  "-object memory-backend-file,id=lsa2,mem-path=%s,size=256M "    \
+                  "-device cxl-type3,bus=rp2,memdev=cxl-mem2,lsa=lsa2,id=cxl-pmem2 " \
+                  "-object memory-backend-file,id=cxl-mem3,mem-path=%s,size=256M "    \
+                  "-object memory-backend-file,id=lsa3,mem-path=%s,size=256M "    \
+                  "-device cxl-type3,bus=rp3,memdev=cxl-mem3,lsa=lsa3,id=cxl-pmem3 "
+
+static void cxl_basic_hb(void)
+{
+    qtest_start("-machine q35,cxl=on");
+    qtest_end();
+}
+
 static void cxl_basic_pxb(void)
 {
     qtest_start("-machine q35,cxl=on -device pxb-cxl,bus=pcie.0");
     qtest_end();
 }
 
+static void cxl_pxb_with_window(void)
+{
+    qtest_start(QEMU_PXB_CMD);
+    qtest_end();
+}
+
+static void cxl_2pxb_with_window(void)
+{
+    qtest_start(QEMU_2PXB_CMD);
+    qtest_end();
+}
+
+static void cxl_root_port(void)
+{
+    qtest_start(QEMU_PXB_CMD QEMU_RP);
+    qtest_end();
+}
+
+static void cxl_2root_port(void)
+{
+    qtest_start(QEMU_PXB_CMD QEMU_2RP);
+    qtest_end();
+}
+
+static void cxl_t3d(void)
+{
+    g_autoptr(GString) cmdline = g_string_new(NULL);
+    char template[] = "/tmp/cxl-test-XXXXXX";
+    const char *tmpfs;
+
+    tmpfs = mkdtemp(template);
+
+    g_string_printf(cmdline, QEMU_PXB_CMD QEMU_RP QEMU_T3D, tmpfs, tmpfs);
+
+    qtest_start(cmdline->str);
+    qtest_end();
+}
+
+static void cxl_1pxb_2rp_2t3d(void)
+{
+    g_autoptr(GString) cmdline = g_string_new(NULL);
+    char template[] = "/tmp/cxl-test-XXXXXX";
+    const char *tmpfs;
+
+    tmpfs = mkdtemp(template);
+
+    g_string_printf(cmdline, QEMU_PXB_CMD QEMU_2RP QEMU_2T3D,
+                    tmpfs, tmpfs, tmpfs, tmpfs);
+
+    qtest_start(cmdline->str);
+    qtest_end();
+}
+
+static void cxl_2pxb_4rp_4t3d(void)
+{
+    g_autoptr(GString) cmdline = g_string_new(NULL);
+    char template[] = "/tmp/cxl-test-XXXXXX";
+    const char *tmpfs;
+
+    tmpfs = mkdtemp(template);
+
+    g_string_printf(cmdline, QEMU_2PXB_CMD QEMU_4RP QEMU_4T3D,
+                    tmpfs, tmpfs, tmpfs, tmpfs, tmpfs, tmpfs,
+                    tmpfs, tmpfs);
+
+    qtest_start(cmdline->str);
+    qtest_end();
+}
+
 int main(int argc, char **argv)
 {
     g_test_init(&argc, &argv, NULL);
+
+    qtest_add_func("/pci/cxl/basic_hostbridge", cxl_basic_hb);
     qtest_add_func("/pci/cxl/basic_pxb", cxl_basic_pxb);
+    qtest_add_func("/pci/cxl/pxb_with_window", cxl_pxb_with_window);
+    qtest_add_func("/pci/cxl/pxb_x2_with_window", cxl_2pxb_with_window);
+    qtest_add_func("/pci/cxl/rp", cxl_root_port);
+    qtest_add_func("/pci/cxl/rp_x2", cxl_2root_port);
+    qtest_add_func("/pci/cxl/type3_device", cxl_t3d);
+    qtest_add_func("/pci/cxl/rp_x2_type3_x2", cxl_1pxb_2rp_2t3d);
+    qtest_add_func("/pci/cxl/pxb_x2_root_port_x4_type3_x4", cxl_2pxb_4rp_4t3d);
     return g_test_run();
 }

From 6e4e3ae936e6bc1501fc0d67444738cec7a1e78a Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:48 +0100
Subject: [PATCH 292/935] hw/cxl/component: Implement host bridge MMIO (8.2.5,
 table 142)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

CXL host bridges themselves may have MMIO. Since host bridges don't have
a BAR they are treated as special for MMIO.  This patch includes
i386/pc support.
Also hook up the device reset now that we have have the MMIO
space in which the results are visible.

Note that we duplicate the PCI express case for the aml_build but
the implementations will diverge when the CXL specific _OSC is
introduced.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Co-developed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-24-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/acpi-build.c                | 25 ++++++++++-
 hw/i386/pc.c                        | 27 +++++++++++-
 hw/pci-bridge/pci_expander_bridge.c | 66 ++++++++++++++++++++++++++---
 include/hw/cxl/cxl.h                | 14 ++++++
 4 files changed, 122 insertions(+), 10 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index dcf6ece3d0..2d81b0f40c 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -28,6 +28,7 @@
 #include "qemu/bitmap.h"
 #include "qemu/error-report.h"
 #include "hw/pci/pci.h"
+#include "hw/cxl/cxl.h"
 #include "hw/core/cpu.h"
 #include "target/i386/cpu.h"
 #include "hw/misc/pvpanic.h"
@@ -1572,10 +1573,21 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
             }
 
             scope = aml_scope("\\_SB");
-            dev = aml_device("PC%.02X", bus_num);
+
+            if (pci_bus_is_cxl(bus)) {
+                dev = aml_device("CL%.02X", bus_num);
+            } else {
+                dev = aml_device("PC%.02X", bus_num);
+            }
             aml_append(dev, aml_name_decl("_UID", aml_int(bus_num)));
             aml_append(dev, aml_name_decl("_BBN", aml_int(bus_num)));
-            if (pci_bus_is_express(bus)) {
+            if (pci_bus_is_cxl(bus)) {
+                aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0A08")));
+                aml_append(dev, aml_name_decl("_CID", aml_eisaid("PNP0A03")));
+
+                /* Expander bridges do not have ACPI PCI Hot-plug enabled */
+                aml_append(dev, build_q35_osc_method(true));
+            } else if (pci_bus_is_express(bus)) {
                 aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0A08")));
                 aml_append(dev, aml_name_decl("_CID", aml_eisaid("PNP0A03")));
 
@@ -1595,6 +1607,15 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
             aml_append(dev, aml_name_decl("_CRS", crs));
             aml_append(scope, dev);
             aml_append(dsdt, scope);
+
+            /* Handle the ranges for the PXB expanders */
+            if (pci_bus_is_cxl(bus)) {
+                MemoryRegion *mr = &machine->cxl_devices_state->host_mr;
+                uint64_t base = mr->addr;
+
+                crs_range_insert(crs_range_set.mem_ranges, base,
+                                 base + memory_region_size(mr) - 1);
+            }
         }
     }
 
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 45e2d6092f..03d14f6564 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -75,6 +75,7 @@
 #include "acpi-build.h"
 #include "hw/mem/pc-dimm.h"
 #include "hw/mem/nvdimm.h"
+#include "hw/cxl/cxl.h"
 #include "qapi/error.h"
 #include "qapi/qapi-visit-common.h"
 #include "qapi/qapi-visit-machine.h"
@@ -816,6 +817,7 @@ void pc_memory_init(PCMachineState *pcms,
     MachineClass *mc = MACHINE_GET_CLASS(machine);
     PCMachineClass *pcmc = PC_MACHINE_GET_CLASS(pcms);
     X86MachineState *x86ms = X86_MACHINE(pcms);
+    hwaddr cxl_base;
 
     assert(machine->ram_size == x86ms->below_4g_mem_size +
                                 x86ms->above_4g_mem_size);
@@ -905,6 +907,26 @@ void pc_memory_init(PCMachineState *pcms,
                                     &machine->device_memory->mr);
     }
 
+    if (machine->cxl_devices_state->is_enabled) {
+        MemoryRegion *mr = &machine->cxl_devices_state->host_mr;
+        hwaddr cxl_size = MiB;
+
+        if (pcmc->has_reserved_memory && machine->device_memory->base) {
+            cxl_base = machine->device_memory->base;
+            if (!pcmc->broken_reserved_end) {
+                cxl_base += memory_region_size(&machine->device_memory->mr);
+            }
+        } else if (pcms->sgx_epc.size != 0) {
+            cxl_base = sgx_epc_above_4g_end(&pcms->sgx_epc);
+        } else {
+            cxl_base = 0x100000000ULL + x86ms->above_4g_mem_size;
+        }
+
+        e820_add_entry(cxl_base, cxl_size, E820_RESERVED);
+        memory_region_init(mr, OBJECT(machine), "cxl_host_reg", cxl_size);
+        memory_region_add_subregion(system_memory, cxl_base, mr);
+    }
+
     /* Initialize PC system firmware */
     pc_system_firmware_init(pcms, rom_memory);
 
@@ -965,7 +987,10 @@ uint64_t pc_pci_hole64_start(void)
     X86MachineState *x86ms = X86_MACHINE(pcms);
     uint64_t hole64_start = 0;
 
-    if (pcmc->has_reserved_memory && ms->device_memory->base) {
+    if (ms->cxl_devices_state->host_mr.addr) {
+        hole64_start = ms->cxl_devices_state->host_mr.addr +
+            memory_region_size(&ms->cxl_devices_state->host_mr);
+    } else if (pcmc->has_reserved_memory && ms->device_memory->base) {
         hole64_start = ms->device_memory->base;
         if (!pcmc->broken_reserved_end) {
             hole64_start += memory_region_size(&ms->device_memory->mr);
diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c
index f762eb4a6e..8fb4f2ea91 100644
--- a/hw/pci-bridge/pci_expander_bridge.c
+++ b/hw/pci-bridge/pci_expander_bridge.c
@@ -61,12 +61,6 @@ DECLARE_INSTANCE_CHECKER(PXBDev, PXB_PCIE_DEV,
 DECLARE_INSTANCE_CHECKER(PXBDev, PXB_CXL_DEV,
                          TYPE_PXB_CXL_DEVICE)
 
-typedef struct CXLHost {
-    PCIHostState parent_obj;
-
-    CXLComponentState cxl_cstate;
-} CXLHost;
-
 struct PXBDev {
     /*< private >*/
     PCIDevice parent_obj;
@@ -75,6 +69,9 @@ struct PXBDev {
     uint8_t bus_nr;
     uint16_t numa_node;
     bool bypass_iommu;
+    struct cxl_dev {
+        CXLHost *cxl_host_bridge;
+    } cxl;
 };
 
 static PXBDev *convert_to_pxb(PCIDevice *dev)
@@ -197,6 +194,52 @@ static const TypeInfo pxb_host_info = {
     .class_init    = pxb_host_class_init,
 };
 
+static void pxb_cxl_realize(DeviceState *dev, Error **errp)
+{
+    MachineState *ms = MACHINE(qdev_get_machine());
+    SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
+    CXLHost *cxl = PXB_CXL_HOST(dev);
+    CXLComponentState *cxl_cstate = &cxl->cxl_cstate;
+    struct MemoryRegion *mr = &cxl_cstate->crb.component_registers;
+    hwaddr offset;
+
+    cxl_component_register_block_init(OBJECT(dev), cxl_cstate,
+                                      TYPE_PXB_CXL_HOST);
+    sysbus_init_mmio(sbd, mr);
+
+    offset = memory_region_size(mr) * ms->cxl_devices_state->next_mr_idx;
+    if (offset > memory_region_size(&ms->cxl_devices_state->host_mr)) {
+        error_setg(errp, "Insufficient space for pxb cxl host register space");
+        return;
+    }
+
+    memory_region_add_subregion(&ms->cxl_devices_state->host_mr, offset, mr);
+    ms->cxl_devices_state->next_mr_idx++;
+}
+
+static void pxb_cxl_host_class_init(ObjectClass *class, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(class);
+    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(class);
+
+    hc->root_bus_path = pxb_host_root_bus_path;
+    dc->fw_name = "cxl";
+    dc->realize = pxb_cxl_realize;
+    /* Reason: Internal part of the pxb/pxb-pcie device, not usable by itself */
+    dc->user_creatable = false;
+}
+
+/*
+ * This is a device to handle the MMIO for a CXL host bridge. It does nothing
+ * else.
+ */
+static const TypeInfo cxl_host_info = {
+    .name          = TYPE_PXB_CXL_HOST,
+    .parent        = TYPE_PCI_HOST_BRIDGE,
+    .instance_size = sizeof(CXLHost),
+    .class_init    = pxb_cxl_host_class_init,
+};
+
 /*
  * Registers the PXB bus as a child of pci host root bus.
  */
@@ -245,6 +288,13 @@ static int pxb_map_irq_fn(PCIDevice *pci_dev, int pin)
 
 static void pxb_dev_reset(DeviceState *dev)
 {
+    CXLHost *cxl = PXB_CXL_DEV(dev)->cxl.cxl_host_bridge;
+    CXLComponentState *cxl_cstate = &cxl->cxl_cstate;
+    uint32_t *reg_state = cxl_cstate->crb.cache_mem_registers;
+    uint32_t *write_msk = cxl_cstate->crb.cache_mem_regs_write_mask;
+
+    cxl_component_register_init_common(reg_state, write_msk, CXL2_ROOT_PORT);
+    ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_CAPABILITY, TARGET_COUNT, 8);
 }
 
 static gint pxb_compare(gconstpointer a, gconstpointer b)
@@ -281,12 +331,13 @@ static void pxb_dev_realize_common(PCIDevice *dev, enum BusType type,
         dev_name = dev->qdev.id;
     }
 
-    ds = qdev_new(TYPE_PXB_HOST);
+    ds = qdev_new(type == CXL ? TYPE_PXB_CXL_HOST : TYPE_PXB_HOST);
     if (type == PCIE) {
         bus = pci_root_bus_new(ds, dev_name, NULL, NULL, 0, TYPE_PXB_PCIE_BUS);
     } else if (type == CXL) {
         bus = pci_root_bus_new(ds, dev_name, NULL, NULL, 0, TYPE_PXB_CXL_BUS);
         bus->flags |= PCI_BUS_CXL;
+        PXB_CXL_DEV(dev)->cxl.cxl_host_bridge = PXB_CXL_HOST(ds);
     } else {
         bus = pci_root_bus_new(ds, "pxb-internal", NULL, NULL, 0, TYPE_PXB_BUS);
         bds = qdev_new("pci-bridge");
@@ -475,6 +526,7 @@ static void pxb_register_types(void)
     type_register_static(&pxb_pcie_bus_info);
     type_register_static(&pxb_cxl_bus_info);
     type_register_static(&pxb_host_info);
+    type_register_static(&cxl_host_info);
     type_register_static(&pxb_dev_info);
     type_register_static(&pxb_pcie_dev_info);
     type_register_static(&pxb_cxl_dev_info);
diff --git a/include/hw/cxl/cxl.h b/include/hw/cxl/cxl.h
index 31af92fd5e..8d1a7245d0 100644
--- a/include/hw/cxl/cxl.h
+++ b/include/hw/cxl/cxl.h
@@ -10,6 +10,7 @@
 #ifndef CXL_H
 #define CXL_H
 
+#include "hw/pci/pci_host.h"
 #include "cxl_pci.h"
 #include "cxl_component.h"
 #include "cxl_device.h"
@@ -17,8 +18,21 @@
 #define CXL_COMPONENT_REG_BAR_IDX 0
 #define CXL_DEVICE_REG_BAR_IDX 2
 
+#define CXL_WINDOW_MAX 10
+
 typedef struct CXLState {
     bool is_enabled;
+    MemoryRegion host_mr;
+    unsigned int next_mr_idx;
 } CXLState;
 
+struct CXLHost {
+    PCIHostState parent_obj;
+
+    CXLComponentState cxl_cstate;
+};
+
+#define TYPE_PXB_CXL_HOST "pxb-cxl-host"
+OBJECT_DECLARE_SIMPLE_TYPE(CXLHost, PXB_CXL_HOST)
+
 #endif

From 2a3282c68ee21cd04ee9d9ef62549f1771929828 Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:49 +0100
Subject: [PATCH 293/935] acpi/cxl: Add _OSC implementation (9.14.2)

CXL 2.0 specification adds 2 new dwords to the existing _OSC definition
from PCIe. The new dwords are accessed with a new uuid. This
implementation supports what is in the specification.

iasl -d decodes the result of this patch as:

Name (SUPP, Zero)
Name (CTRL, Zero)
Name (SUPC, Zero)
Name (CTRC, Zero)
Method (_OSC, 4, NotSerialized)  // _OSC: Operating System Capabilities
{
    CreateDWordField (Arg3, Zero, CDW1)
    If (((Arg0 == ToUUID ("33db4d5b-1ff7-401c-9657-7441c03dd766") /* PCI Host Bridge Device */) || (Arg0 == ToUUID ("68f2d50b-c469-4d8a-bd3d-941a103fd3fc") /* Unknown UUID */)))
    {
        CreateDWordField (Arg3, 0x04, CDW2)
        CreateDWordField (Arg3, 0x08, CDW3)
        Local0 = CDW3 /* \_SB_.PC0C._OSC.CDW3 */
        Local0 &= 0x1F
        If ((Arg1 != One))
        {
            CDW1 |= 0x08
        }

        If ((CDW3 != Local0))
        {
            CDW1 |= 0x10
        }

        SUPP = CDW2 /* \_SB_.PC0C._OSC.CDW2 */
        CTRL = CDW3 /* \_SB_.PC0C._OSC.CDW3 */
        CDW3 = Local0
        If ((Arg0 == ToUUID ("68f2d50b-c469-4d8a-bd3d-941a103fd3fc") /* Unknown UUID */))
        {
            CreateDWordField (Arg3, 0x0C, CDW4)
            CreateDWordField (Arg3, 0x10, CDW5)
            SUPC = CDW4 /* \_SB_.PC0C._OSC.CDW4 */
            CTRC = CDW5 /* \_SB_.PC0C._OSC.CDW5 */
            CDW5 |= One
        }

        Return (Arg3)
    }
    Else
    {
        CDW1 |= 0x04
        Return (Arg3)
    }

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Message-Id: <20220429144110.25167-25-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/acpi/Kconfig       |   5 ++
 hw/acpi/cxl-stub.c    |  12 ++++
 hw/acpi/cxl.c         | 130 ++++++++++++++++++++++++++++++++++++++++++
 hw/acpi/meson.build   |   4 +-
 hw/i386/acpi-build.c  |  13 +++--
 include/hw/acpi/cxl.h |  23 ++++++++
 6 files changed, 182 insertions(+), 5 deletions(-)
 create mode 100644 hw/acpi/cxl-stub.c
 create mode 100644 hw/acpi/cxl.c
 create mode 100644 include/hw/acpi/cxl.h

diff --git a/hw/acpi/Kconfig b/hw/acpi/Kconfig
index 19caebde6c..3703aca212 100644
--- a/hw/acpi/Kconfig
+++ b/hw/acpi/Kconfig
@@ -5,6 +5,7 @@ config ACPI_X86
     bool
     select ACPI
     select ACPI_NVDIMM
+    select ACPI_CXL
     select ACPI_CPU_HOTPLUG
     select ACPI_MEMORY_HOTPLUG
     select ACPI_HMAT
@@ -66,3 +67,7 @@ config ACPI_ERST
     bool
     default y
     depends on ACPI && PCI
+
+config ACPI_CXL
+    bool
+    depends on ACPI
diff --git a/hw/acpi/cxl-stub.c b/hw/acpi/cxl-stub.c
new file mode 100644
index 0000000000..15bc21076b
--- /dev/null
+++ b/hw/acpi/cxl-stub.c
@@ -0,0 +1,12 @@
+
+/*
+ * Stubs for ACPI platforms that don't support CXl
+ */
+#include "qemu/osdep.h"
+#include "hw/acpi/aml-build.h"
+#include "hw/acpi/cxl.h"
+
+void build_cxl_osc_method(Aml *dev)
+{
+    g_assert_not_reached();
+}
diff --git a/hw/acpi/cxl.c b/hw/acpi/cxl.c
new file mode 100644
index 0000000000..ca1f04f359
--- /dev/null
+++ b/hw/acpi/cxl.c
@@ -0,0 +1,130 @@
+/*
+ * CXL ACPI Implementation
+ *
+ * Copyright(C) 2020 Intel Corporation.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>
+ */
+
+#include "qemu/osdep.h"
+#include "hw/cxl/cxl.h"
+#include "hw/acpi/acpi.h"
+#include "hw/acpi/aml-build.h"
+#include "hw/acpi/bios-linker-loader.h"
+#include "hw/acpi/cxl.h"
+#include "qapi/error.h"
+#include "qemu/uuid.h"
+
+static Aml *__build_cxl_osc_method(void)
+{
+    Aml *method, *if_uuid, *else_uuid, *if_arg1_not_1, *if_cxl, *if_caps_masked;
+    Aml *a_ctrl = aml_local(0);
+    Aml *a_cdw1 = aml_name("CDW1");
+
+    method = aml_method("_OSC", 4, AML_NOTSERIALIZED);
+    /* CDW1 is used for the return value so is present whether or not a match occurs */
+    aml_append(method, aml_create_dword_field(aml_arg(3), aml_int(0), "CDW1"));
+
+    /*
+     * Generate shared section between:
+     * CXL 2.0 - 9.14.2.1.4 and
+     * PCI Firmware Specification 3.0
+     * 4.5.1. _OSC Interface for PCI Host Bridge Devices
+     * The _OSC interface for a PCI/PCI-X/PCI Express hierarchy is
+     * identified by the Universal Unique IDentifier (UUID)
+     * 33DB4D5B-1FF7-401C-9657-7441C03DD766
+     * The _OSC interface for a CXL Host bridge is
+     * identified by the UUID 68F2D50B-C469-4D8A-BD3D-941A103FD3FC
+     * A CXL Host bridge is compatible with a PCI host bridge so
+     * for the shared section match both.
+     */
+    if_uuid = aml_if(
+        aml_lor(aml_equal(aml_arg(0),
+                          aml_touuid("33DB4D5B-1FF7-401C-9657-7441C03DD766")),
+                aml_equal(aml_arg(0),
+                          aml_touuid("68F2D50B-C469-4D8A-BD3D-941A103FD3FC"))));
+    aml_append(if_uuid, aml_create_dword_field(aml_arg(3), aml_int(4), "CDW2"));
+    aml_append(if_uuid, aml_create_dword_field(aml_arg(3), aml_int(8), "CDW3"));
+
+    aml_append(if_uuid, aml_store(aml_name("CDW3"), a_ctrl));
+
+    /*
+     *
+     * Allows OS control for all 5 features:
+     * PCIeHotplug SHPCHotplug PME AER PCIeCapability
+     */
+    aml_append(if_uuid, aml_and(a_ctrl, aml_int(0x1F), a_ctrl));
+
+    /*
+     * Check _OSC revision.
+     * PCI Firmware specification 3.3 and CXL 2.0 both use revision 1
+     * Unknown Revision is CDW1 - BIT (3)
+     */
+    if_arg1_not_1 = aml_if(aml_lnot(aml_equal(aml_arg(1), aml_int(0x1))));
+    aml_append(if_arg1_not_1, aml_or(a_cdw1, aml_int(0x08), a_cdw1));
+    aml_append(if_uuid, if_arg1_not_1);
+
+    if_caps_masked = aml_if(aml_lnot(aml_equal(aml_name("CDW3"), a_ctrl)));
+
+    /* Capability bits were masked */
+    aml_append(if_caps_masked, aml_or(a_cdw1, aml_int(0x10), a_cdw1));
+    aml_append(if_uuid, if_caps_masked);
+
+    aml_append(if_uuid, aml_store(aml_name("CDW2"), aml_name("SUPP")));
+    aml_append(if_uuid, aml_store(aml_name("CDW3"), aml_name("CTRL")));
+
+    /* Update DWORD3 (the return value) */
+    aml_append(if_uuid, aml_store(a_ctrl, aml_name("CDW3")));
+
+    /* CXL only section as per CXL 2.0 - 9.14.2.1.4 */
+    if_cxl = aml_if(aml_equal(
+        aml_arg(0), aml_touuid("68F2D50B-C469-4D8A-BD3D-941A103FD3FC")));
+    /* CXL support field */
+    aml_append(if_cxl, aml_create_dword_field(aml_arg(3), aml_int(12), "CDW4"));
+    /* CXL capabilities */
+    aml_append(if_cxl, aml_create_dword_field(aml_arg(3), aml_int(16), "CDW5"));
+    aml_append(if_cxl, aml_store(aml_name("CDW4"), aml_name("SUPC")));
+    aml_append(if_cxl, aml_store(aml_name("CDW5"), aml_name("CTRC")));
+
+    /* CXL 2.0 Port/Device Register access */
+    aml_append(if_cxl,
+               aml_or(aml_name("CDW5"), aml_int(0x1), aml_name("CDW5")));
+    aml_append(if_uuid, if_cxl);
+
+    aml_append(if_uuid, aml_return(aml_arg(3)));
+    aml_append(method, if_uuid);
+
+    /*
+     * If no UUID matched, return Unrecognized UUID via Arg3 DWord 1
+     * ACPI 6.4 - 6.2.11
+     * Unrecognised UUID - BIT(2)
+     */
+    else_uuid = aml_else();
+
+    aml_append(else_uuid,
+               aml_or(aml_name("CDW1"), aml_int(0x4), aml_name("CDW1")));
+    aml_append(else_uuid, aml_return(aml_arg(3)));
+    aml_append(method, else_uuid);
+
+    return method;
+}
+
+void build_cxl_osc_method(Aml *dev)
+{
+    aml_append(dev, aml_name_decl("SUPP", aml_int(0)));
+    aml_append(dev, aml_name_decl("CTRL", aml_int(0)));
+    aml_append(dev, aml_name_decl("SUPC", aml_int(0)));
+    aml_append(dev, aml_name_decl("CTRC", aml_int(0)));
+    aml_append(dev, __build_cxl_osc_method());
+}
diff --git a/hw/acpi/meson.build b/hw/acpi/meson.build
index 8bea2e6933..cea2f5f93a 100644
--- a/hw/acpi/meson.build
+++ b/hw/acpi/meson.build
@@ -13,6 +13,7 @@ acpi_ss.add(when: 'CONFIG_ACPI_MEMORY_HOTPLUG', if_false: files('acpi-mem-hotplu
 acpi_ss.add(when: 'CONFIG_ACPI_NVDIMM', if_true: files('nvdimm.c'))
 acpi_ss.add(when: 'CONFIG_ACPI_NVDIMM', if_false: files('acpi-nvdimm-stub.c'))
 acpi_ss.add(when: 'CONFIG_ACPI_PCI', if_true: files('pci.c'))
+acpi_ss.add(when: 'CONFIG_ACPI_CXL', if_true: files('cxl.c'), if_false: files('cxl-stub.c'))
 acpi_ss.add(when: 'CONFIG_ACPI_VMGENID', if_true: files('vmgenid.c'))
 acpi_ss.add(when: 'CONFIG_ACPI_HW_REDUCED', if_true: files('generic_event_device.c'))
 acpi_ss.add(when: 'CONFIG_ACPI_HMAT', if_true: files('hmat.c'))
@@ -33,4 +34,5 @@ softmmu_ss.add_all(when: 'CONFIG_ACPI', if_true: acpi_ss)
 softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('acpi-stub.c', 'aml-build-stub.c',
                                                   'acpi-x86-stub.c', 'ipmi-stub.c', 'ghes-stub.c',
                                                   'acpi-mem-hotplug-stub.c', 'acpi-cpu-hotplug-stub.c',
-                                                  'acpi-pci-hotplug-stub.c', 'acpi-nvdimm-stub.c'))
+                                                  'acpi-pci-hotplug-stub.c', 'acpi-nvdimm-stub.c',
+                                                  'cxl-stub.c'))
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 2d81b0f40c..59ede8b2e9 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -67,6 +67,7 @@
 #include "hw/acpi/aml-build.h"
 #include "hw/acpi/utils.h"
 #include "hw/acpi/pci.h"
+#include "hw/acpi/cxl.h"
 
 #include "qom/qom-qobject.h"
 #include "hw/i386/amd_iommu.h"
@@ -1582,11 +1583,15 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
             aml_append(dev, aml_name_decl("_UID", aml_int(bus_num)));
             aml_append(dev, aml_name_decl("_BBN", aml_int(bus_num)));
             if (pci_bus_is_cxl(bus)) {
-                aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0A08")));
-                aml_append(dev, aml_name_decl("_CID", aml_eisaid("PNP0A03")));
+                struct Aml *pkg = aml_package(2);
 
-                /* Expander bridges do not have ACPI PCI Hot-plug enabled */
-                aml_append(dev, build_q35_osc_method(true));
+                aml_append(dev, aml_name_decl("_HID", aml_string("ACPI0016")));
+                aml_append(pkg, aml_eisaid("PNP0A08"));
+                aml_append(pkg, aml_eisaid("PNP0A03"));
+                aml_append(dev, aml_name_decl("_CID", pkg));
+                aml_append(dev, aml_name_decl("_ADR", aml_int(0)));
+                aml_append(dev, aml_name_decl("_UID", aml_int(bus_num)));
+                build_cxl_osc_method(dev);
             } else if (pci_bus_is_express(bus)) {
                 aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0A08")));
                 aml_append(dev, aml_name_decl("_CID", aml_eisaid("PNP0A03")));
diff --git a/include/hw/acpi/cxl.h b/include/hw/acpi/cxl.h
new file mode 100644
index 0000000000..7b8f3b8a2e
--- /dev/null
+++ b/include/hw/acpi/cxl.h
@@ -0,0 +1,23 @@
+/*
+ * Copyright (C) 2020 Intel Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef HW_ACPI_CXL_H
+#define HW_ACPI_CXL_H
+
+void build_cxl_osc_method(Aml *dev);
+
+#endif

From 3d6a69b6eb973e5761c8dc489990dcd983f6c099 Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:50 +0100
Subject: [PATCH 294/935] acpi/cxl: Create the CEDT (9.14.1)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The CXL Early Discovery Table is defined in the CXL 2.0 specification as
a way for the OS to get CXL specific information from the system
firmware.

CXL 2.0 specification adds an _HID, ACPI0016, for CXL capable host
bridges, with a _CID of PNP0A08 (PCIe host bridge). CXL aware software
is able to use this initiate the proper _OSC method, and get the _UID
which is referenced by the CEDT. Therefore the existence of an ACPI0016
device allows a CXL aware driver perform the necessary actions. For a
CXL capable OS, this works. For a CXL unaware OS, this works.

CEDT awaremess requires more. The motivation for ACPI0017 is to provide
the possibility of having a Linux CXL module that can work on a legacy
Linux kernel. Linux core PCI/ACPI which won't be built as a module,
will see the _CID of PNP0A08 and bind a driver to it. If we later loaded
a driver for ACPI0016, Linux won't be able to bind it to the hardware
because it has already bound the PNP0A08 driver. The ACPI0017 device is
an opportunity to have an object to bind a driver will be used by a
Linux driver to walk the CXL topology and do everything that we would
have preferred to do with ACPI0016.

There is another motivation for an ACPI0017 device which isn't
implemented here. An operating system needs an attach point for a
non-volatile region provider that understands cross-hostbridge
interleaving. Since QEMU emulation doesn't support interleaving yet,
this is more important on the OS side, for now.

As of CXL 2.0 spec, only 1 sub structure is defined, the CXL Host Bridge
Structure (CHBS) which is primarily useful for telling the OS exactly
where the MMIO for the host bridge is.

Link: https://lore.kernel.org/linux-cxl/20210115034911.nkgpzc756d6qmjpl@intel.com/T/#t
Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-26-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/acpi/cxl.c                       | 68 +++++++++++++++++++++++++++++
 hw/i386/acpi-build.c                | 27 ++++++++++++
 hw/pci-bridge/pci_expander_bridge.c | 17 --------
 include/hw/acpi/cxl.h               |  5 +++
 include/hw/pci/pci_bridge.h         | 20 +++++++++
 5 files changed, 120 insertions(+), 17 deletions(-)

diff --git a/hw/acpi/cxl.c b/hw/acpi/cxl.c
index ca1f04f359..aa4af86a4c 100644
--- a/hw/acpi/cxl.c
+++ b/hw/acpi/cxl.c
@@ -18,7 +18,11 @@
  */
 
 #include "qemu/osdep.h"
+#include "hw/sysbus.h"
+#include "hw/pci/pci_bridge.h"
+#include "hw/pci/pci_host.h"
 #include "hw/cxl/cxl.h"
+#include "hw/mem/memory-device.h"
 #include "hw/acpi/acpi.h"
 #include "hw/acpi/aml-build.h"
 #include "hw/acpi/bios-linker-loader.h"
@@ -26,6 +30,70 @@
 #include "qapi/error.h"
 #include "qemu/uuid.h"
 
+static void cedt_build_chbs(GArray *table_data, PXBDev *cxl)
+{
+    SysBusDevice *sbd = SYS_BUS_DEVICE(cxl->cxl.cxl_host_bridge);
+    struct MemoryRegion *mr = sbd->mmio[0].memory;
+
+    /* Type */
+    build_append_int_noprefix(table_data, 0, 1);
+
+    /* Reserved */
+    build_append_int_noprefix(table_data, 0, 1);
+
+    /* Record Length */
+    build_append_int_noprefix(table_data, 32, 2);
+
+    /* UID - currently equal to bus number */
+    build_append_int_noprefix(table_data, cxl->bus_nr, 4);
+
+    /* Version */
+    build_append_int_noprefix(table_data, 1, 4);
+
+    /* Reserved */
+    build_append_int_noprefix(table_data, 0, 4);
+
+    /* Base - subregion within a container that is in PA space */
+    build_append_int_noprefix(table_data, mr->container->addr + mr->addr, 8);
+
+    /* Length */
+    build_append_int_noprefix(table_data, memory_region_size(mr), 8);
+}
+
+static int cxl_foreach_pxb_hb(Object *obj, void *opaque)
+{
+    Aml *cedt = opaque;
+
+    if (object_dynamic_cast(obj, TYPE_PXB_CXL_DEVICE)) {
+        cedt_build_chbs(cedt->buf, PXB_CXL_DEV(obj));
+    }
+
+    return 0;
+}
+
+void cxl_build_cedt(MachineState *ms, GArray *table_offsets, GArray *table_data,
+                    BIOSLinker *linker, const char *oem_id,
+                    const char *oem_table_id)
+{
+    Aml *cedt;
+    AcpiTable table = { .sig = "CEDT", .rev = 1, .oem_id = oem_id,
+                        .oem_table_id = oem_table_id };
+
+    acpi_add_table(table_offsets, table_data);
+    acpi_table_begin(&table, table_data);
+    cedt = init_aml_allocator();
+
+    /* reserve space for CEDT header */
+
+    object_child_foreach_recursive(object_get_root(), cxl_foreach_pxb_hb, cedt);
+
+    /* copy AML table into ACPI tables blob and patch header there */
+    g_array_append_vals(table_data, cedt->buf->data, cedt->buf->len);
+    free_aml_allocator();
+
+    acpi_table_end(linker, &table);
+}
+
 static Aml *__build_cxl_osc_method(void)
 {
     Aml *method, *if_uuid, *else_uuid, *if_arg1_not_1, *if_cxl, *if_caps_masked;
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 59ede8b2e9..c125939ed6 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -77,6 +77,7 @@
 #include "hw/acpi/ipmi.h"
 #include "hw/acpi/hmat.h"
 #include "hw/acpi/viot.h"
+#include "hw/acpi/cxl.h"
 
 #include CONFIG_DEVICES
 
@@ -1411,6 +1412,22 @@ static void build_smb0(Aml *table, I2CBus *smbus, int devnr, int func)
     aml_append(table, scope);
 }
 
+static void build_acpi0017(Aml *table)
+{
+    Aml *dev, *scope, *method;
+
+    scope =  aml_scope("_SB");
+    dev = aml_device("CXLM");
+    aml_append(dev, aml_name_decl("_HID", aml_string("ACPI0017")));
+
+    method = aml_method("_STA", 0, AML_NOTSERIALIZED);
+    aml_append(method, aml_return(aml_int(0x01)));
+    aml_append(dev, method);
+
+    aml_append(scope, dev);
+    aml_append(table, scope);
+}
+
 static void
 build_dsdt(GArray *table_data, BIOSLinker *linker,
            AcpiPmInfo *pm, AcpiMiscInfo *misc,
@@ -1430,6 +1447,7 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
 #ifdef CONFIG_TPM
     TPMIf *tpm = tpm_find();
 #endif
+    bool cxl_present = false;
     int i;
     VMBusBridge *vmbus_bridge = vmbus_bridge_find();
     AcpiTable table = { .sig = "DSDT", .rev = 1, .oem_id = x86ms->oem_id,
@@ -1618,12 +1636,17 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
                 MemoryRegion *mr = &machine->cxl_devices_state->host_mr;
                 uint64_t base = mr->addr;
 
+                cxl_present = true;
                 crs_range_insert(crs_range_set.mem_ranges, base,
                                  base + memory_region_size(mr) - 1);
             }
         }
     }
 
+    if (cxl_present) {
+        build_acpi0017(dsdt);
+    }
+
     /*
      * At this point crs_range_set has all the ranges used by pci
      * busses *other* than PCI0.  These ranges will be excluded from
@@ -2688,6 +2711,10 @@ void acpi_build(AcpiBuildTables *tables, MachineState *machine)
                           machine->nvdimms_state, machine->ram_slots,
                           x86ms->oem_id, x86ms->oem_table_id);
     }
+    if (machine->cxl_devices_state->is_enabled) {
+        cxl_build_cedt(machine, table_offsets, tables_blob, tables->linker,
+                       x86ms->oem_id, x86ms->oem_table_id);
+    }
 
     acpi_add_table(table_offsets, tables_blob);
     build_waet(tables_blob, tables->linker, x86ms->oem_id, x86ms->oem_table_id);
diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c
index 8fb4f2ea91..22feda1ff0 100644
--- a/hw/pci-bridge/pci_expander_bridge.c
+++ b/hw/pci-bridge/pci_expander_bridge.c
@@ -57,23 +57,6 @@ DECLARE_INSTANCE_CHECKER(PXBDev, PXB_DEV,
 DECLARE_INSTANCE_CHECKER(PXBDev, PXB_PCIE_DEV,
                          TYPE_PXB_PCIE_DEVICE)
 
-#define TYPE_PXB_CXL_DEVICE "pxb-cxl"
-DECLARE_INSTANCE_CHECKER(PXBDev, PXB_CXL_DEV,
-                         TYPE_PXB_CXL_DEVICE)
-
-struct PXBDev {
-    /*< private >*/
-    PCIDevice parent_obj;
-    /*< public >*/
-
-    uint8_t bus_nr;
-    uint16_t numa_node;
-    bool bypass_iommu;
-    struct cxl_dev {
-        CXLHost *cxl_host_bridge;
-    } cxl;
-};
-
 static PXBDev *convert_to_pxb(PCIDevice *dev)
 {
     /* A CXL PXB's parent bus is PCIe, so the normal check won't work */
diff --git a/include/hw/acpi/cxl.h b/include/hw/acpi/cxl.h
index 7b8f3b8a2e..0c496538c0 100644
--- a/include/hw/acpi/cxl.h
+++ b/include/hw/acpi/cxl.h
@@ -18,6 +18,11 @@
 #ifndef HW_ACPI_CXL_H
 #define HW_ACPI_CXL_H
 
+#include "hw/acpi/bios-linker-loader.h"
+
+void cxl_build_cedt(MachineState *ms, GArray *table_offsets, GArray *table_data,
+                    BIOSLinker *linker, const char *oem_id,
+                    const char *oem_table_id);
 void build_cxl_osc_method(Aml *dev);
 
 #endif
diff --git a/include/hw/pci/pci_bridge.h b/include/hw/pci/pci_bridge.h
index 30691a6e57..ba4bafac7c 100644
--- a/include/hw/pci/pci_bridge.h
+++ b/include/hw/pci/pci_bridge.h
@@ -28,6 +28,7 @@
 
 #include "hw/pci/pci.h"
 #include "hw/pci/pci_bus.h"
+#include "hw/cxl/cxl.h"
 #include "qom/object.h"
 
 typedef struct PCIBridgeWindows PCIBridgeWindows;
@@ -80,6 +81,25 @@ struct PCIBridge {
 #define PCI_BRIDGE_DEV_PROP_CHASSIS_NR "chassis_nr"
 #define PCI_BRIDGE_DEV_PROP_MSI        "msi"
 #define PCI_BRIDGE_DEV_PROP_SHPC       "shpc"
+typedef struct CXLHost CXLHost;
+
+struct PXBDev {
+    /*< private >*/
+    PCIDevice parent_obj;
+    /*< public >*/
+
+    uint8_t bus_nr;
+    uint16_t numa_node;
+    bool bypass_iommu;
+    struct cxl_dev {
+        CXLHost *cxl_host_bridge; /* Pointer to a CXLHost */
+    } cxl;
+};
+
+typedef struct PXBDev PXBDev;
+#define TYPE_PXB_CXL_DEVICE "pxb-cxl"
+DECLARE_INSTANCE_CHECKER(PXBDev, PXB_CXL_DEV,
+                         TYPE_PXB_CXL_DEVICE)
 
 int pci_bridge_ssvid_init(PCIDevice *dev, uint8_t offset,
                           uint16_t svid, uint16_t ssid,

From 829de299d1a4dcac3cd6c494d1c974bdbc608611 Mon Sep 17 00:00:00 2001
From: Jonathan Cameron <jonathan.cameron@huawei.com>
Date: Fri, 29 Apr 2022 15:40:51 +0100
Subject: [PATCH 295/935] hw/cxl/component: Add utils for interleave parameter
 encoding/decoding
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Both registers and the CFMWS entries in CDAT use simple encodings
for the number of interleave ways and the interleave granularity.
Introduce simple conversion functions to/from the unencoded
number / size.  So far the iw decode has not been needed so is
it not implemented.

Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-27-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/cxl/cxl-component-utils.c   | 34 ++++++++++++++++++++++++++++++++++
 include/hw/cxl/cxl_component.h |  8 ++++++++
 2 files changed, 42 insertions(+)

diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c
index afc97b17c2..69cb07171c 100644
--- a/hw/cxl/cxl-component-utils.c
+++ b/hw/cxl/cxl-component-utils.c
@@ -9,6 +9,7 @@
 
 #include "qemu/osdep.h"
 #include "qemu/log.h"
+#include "qapi/error.h"
 #include "hw/pci/pci.h"
 #include "hw/cxl/cxl.h"
 
@@ -329,3 +330,36 @@ void cxl_component_create_dvsec(CXLComponentState *cxl,
     range_init_nofail(&cxl->dvsecs[type], cxl->dvsec_offset, length);
     cxl->dvsec_offset += length;
 }
+
+uint8_t cxl_interleave_ways_enc(int iw, Error **errp)
+{
+    switch (iw) {
+    case 1: return 0x0;
+    case 2: return 0x1;
+    case 4: return 0x2;
+    case 8: return 0x3;
+    case 16: return 0x4;
+    case 3: return 0x8;
+    case 6: return 0x9;
+    case 12: return 0xa;
+    default:
+        error_setg(errp, "Interleave ways: %d not supported", iw);
+        return 0;
+    }
+}
+
+uint8_t cxl_interleave_granularity_enc(uint64_t gran, Error **errp)
+{
+    switch (gran) {
+    case 256: return 0;
+    case 512: return 1;
+    case 1024: return 2;
+    case 2048: return 3;
+    case 4096: return 4;
+    case 8192: return 5;
+    case 16384: return 6;
+    default:
+        error_setg(errp, "Interleave granularity: %" PRIu64 " invalid", gran);
+        return 0;
+    }
+}
diff --git a/include/hw/cxl/cxl_component.h b/include/hw/cxl/cxl_component.h
index 7d8f395cbe..4f69688c47 100644
--- a/include/hw/cxl/cxl_component.h
+++ b/include/hw/cxl/cxl_component.h
@@ -210,4 +210,12 @@ static inline int cxl_decoder_count_enc(int count)
     return 0;
 }
 
+uint8_t cxl_interleave_ways_enc(int iw, Error **errp);
+uint8_t cxl_interleave_granularity_enc(uint64_t gran, Error **errp);
+
+static inline hwaddr cxl_decode_ig(int ig)
+{
+    return 1 << (ig + 8);
+}
+
 #endif

From aadfe320919b15e24ac070e9b25085e07599a613 Mon Sep 17 00:00:00 2001
From: Jonathan Cameron <jonathan.cameron@huawei.com>
Date: Fri, 29 Apr 2022 15:40:52 +0100
Subject: [PATCH 296/935] hw/cxl/host: Add support for CXL Fixed Memory
 Windows.

The concept of these is introduced in [1] in terms of the
description the CEDT ACPI table. The principal is more general.
Unlike once traffic hits the CXL root bridges, the host system
memory address routing is implementation defined and effectively
static once observable by standard / generic system software.
Each CXL Fixed Memory Windows (CFMW) is a region of PA space
which has fixed system dependent routing configured so that
accesses can be routed to the CXL devices below a set of target
root bridges. The accesses may be interleaved across multiple
root bridges.

For QEMU we could have fully specified these regions in terms
of a base PA + size, but as the absolute address does not matter
it is simpler to let individual platforms place the memory regions.

ExampleS:
-cxl-fixed-memory-window targets.0=cxl.0,size=128G
-cxl-fixed-memory-window targets.0=cxl.1,size=128G
-cxl-fixed-memory-window targets.0=cxl0,targets.1=cxl.1,size=256G,interleave-granularity=2k

Specifies
* 2x 128G regions not interleaved across root bridges, one for each of
  the root bridges with ids cxl.0 and cxl.1
* 256G region interleaved across root bridges with ids cxl.0 and cxl.1
with a 2k interleave granularity.

When system software enumerates the devices below a given root bridge
it can then decide which CFMW to use. If non interleave is desired
(or possible) it can use the appropriate CFMW for the root bridge in
question.  If there are suitable devices to interleave across the
two root bridges then it may use the 3rd CFMS.

A number of other designs were considered but the following constraints
made it hard to adapt existing QEMU approaches to this particular problem.
1) The size must be known before a specific architecture / board brings
   up it's PA memory map.  We need to set up an appropriate region.
2) Using links to the host bridges provides a clean command line interface
   but these links cannot be established until command line devices have
   been added.

Hence the two step process used here of first establishing the size,
interleave-ways and granularity + caching the ids of the host bridges
and then, once available finding the actual host bridges so they can
be used later to support interleave decoding.

[1] CXL 2.0 ECN: CEDT CFMWS & QTG DSM (computeexpresslink.org / specifications)

Signed-off-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Acked-by: Markus Armbruster <armbru@redhat.com> # QAPI Schema
Message-Id: <20220429144110.25167-28-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/cxl/cxl-host-stubs.c | 14 ++++++
 hw/cxl/cxl-host.c       | 94 +++++++++++++++++++++++++++++++++++++++++
 hw/cxl/meson.build      |  6 +++
 include/hw/cxl/cxl.h    | 21 +++++++++
 qapi/machine.json       | 21 +++++++++
 qemu-options.hx         | 38 +++++++++++++++++
 softmmu/vl.c            | 47 +++++++++++++++++++++
 7 files changed, 241 insertions(+)
 create mode 100644 hw/cxl/cxl-host-stubs.c
 create mode 100644 hw/cxl/cxl-host.c

diff --git a/hw/cxl/cxl-host-stubs.c b/hw/cxl/cxl-host-stubs.c
new file mode 100644
index 0000000000..f8fd278d5d
--- /dev/null
+++ b/hw/cxl/cxl-host-stubs.c
@@ -0,0 +1,14 @@
+/*
+ * CXL host parameter parsing routine stubs
+ *
+ * Copyright (c) 2022 Huawei
+ */
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/cxl/cxl.h"
+
+void cxl_fixed_memory_window_config(MachineState *ms,
+                                    CXLFixedMemoryWindowOptions *object,
+                                    Error **errp) {};
+
+void cxl_fixed_memory_window_link_targets(Error **errp) {};
diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c
new file mode 100644
index 0000000000..ec5a75cbf5
--- /dev/null
+++ b/hw/cxl/cxl-host.c
@@ -0,0 +1,94 @@
+/*
+ * CXL host parameter parsing routines
+ *
+ * Copyright (c) 2022 Huawei
+ * Modeled loosely on the NUMA options handling in hw/core/numa.c
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/units.h"
+#include "qemu/bitmap.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
+#include "sysemu/qtest.h"
+#include "hw/boards.h"
+
+#include "qapi/qapi-visit-machine.h"
+#include "hw/cxl/cxl.h"
+
+void cxl_fixed_memory_window_config(MachineState *ms,
+                                    CXLFixedMemoryWindowOptions *object,
+                                    Error **errp)
+{
+    CXLFixedWindow *fw = g_malloc0(sizeof(*fw));
+    strList *target;
+    int i;
+
+    for (target = object->targets; target; target = target->next) {
+        fw->num_targets++;
+    }
+
+    fw->enc_int_ways = cxl_interleave_ways_enc(fw->num_targets, errp);
+    if (*errp) {
+        return;
+    }
+
+    fw->targets = g_malloc0_n(fw->num_targets, sizeof(*fw->targets));
+    for (i = 0, target = object->targets; target; i++, target = target->next) {
+        /* This link cannot be resolved yet, so stash the name for now */
+        fw->targets[i] = g_strdup(target->value);
+    }
+
+    if (object->size % (256 * MiB)) {
+        error_setg(errp,
+                   "Size of a CXL fixed memory window must my a multiple of 256MiB");
+        return;
+    }
+    fw->size = object->size;
+
+    if (object->has_interleave_granularity) {
+        fw->enc_int_gran =
+            cxl_interleave_granularity_enc(object->interleave_granularity,
+                                           errp);
+        if (*errp) {
+            return;
+        }
+    } else {
+        /* Default to 256 byte interleave */
+        fw->enc_int_gran = 0;
+    }
+
+    ms->cxl_devices_state->fixed_windows =
+        g_list_append(ms->cxl_devices_state->fixed_windows, fw);
+
+    return;
+}
+
+void cxl_fixed_memory_window_link_targets(Error **errp)
+{
+    MachineState *ms = MACHINE(qdev_get_machine());
+
+    if (ms->cxl_devices_state && ms->cxl_devices_state->fixed_windows) {
+        GList *it;
+
+        for (it = ms->cxl_devices_state->fixed_windows; it; it = it->next) {
+            CXLFixedWindow *fw = it->data;
+            int i;
+
+            for (i = 0; i < fw->num_targets; i++) {
+                Object *o;
+                bool ambig;
+
+                o = object_resolve_path_type(fw->targets[i],
+                                             TYPE_PXB_CXL_DEVICE,
+                                             &ambig);
+                if (!o) {
+                    error_setg(errp, "Could not resolve CXLFM target %s",
+                               fw->targets[i]);
+                    return;
+                }
+                fw->target_hbs[i] = PXB_CXL_DEV(o);
+            }
+        }
+    }
+}
diff --git a/hw/cxl/meson.build b/hw/cxl/meson.build
index e68eea2358..f117b99949 100644
--- a/hw/cxl/meson.build
+++ b/hw/cxl/meson.build
@@ -3,4 +3,10 @@ softmmu_ss.add(when: 'CONFIG_CXL',
                    'cxl-component-utils.c',
                    'cxl-device-utils.c',
                    'cxl-mailbox-utils.c',
+                   'cxl-host.c',
+               ),
+               if_false: files(
+                   'cxl-host-stubs.c',
                ))
+
+softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('cxl-host-stubs.c'))
diff --git a/include/hw/cxl/cxl.h b/include/hw/cxl/cxl.h
index 8d1a7245d0..dce38124db 100644
--- a/include/hw/cxl/cxl.h
+++ b/include/hw/cxl/cxl.h
@@ -10,6 +10,9 @@
 #ifndef CXL_H
 #define CXL_H
 
+
+#include "qapi/qapi-types-machine.h"
+#include "hw/pci/pci_bridge.h"
 #include "hw/pci/pci_host.h"
 #include "cxl_pci.h"
 #include "cxl_component.h"
@@ -20,10 +23,23 @@
 
 #define CXL_WINDOW_MAX 10
 
+typedef struct CXLFixedWindow {
+    uint64_t size;
+    char **targets;
+    struct PXBDev *target_hbs[8];
+    uint8_t num_targets;
+    uint8_t enc_int_ways;
+    uint8_t enc_int_gran;
+    /* Todo: XOR based interleaving */
+    MemoryRegion mr;
+    hwaddr base;
+} CXLFixedWindow;
+
 typedef struct CXLState {
     bool is_enabled;
     MemoryRegion host_mr;
     unsigned int next_mr_idx;
+    GList *fixed_windows;
 } CXLState;
 
 struct CXLHost {
@@ -35,4 +51,9 @@ struct CXLHost {
 #define TYPE_PXB_CXL_HOST "pxb-cxl-host"
 OBJECT_DECLARE_SIMPLE_TYPE(CXLHost, PXB_CXL_HOST)
 
+void cxl_fixed_memory_window_config(MachineState *ms,
+                                    CXLFixedMemoryWindowOptions *object,
+                                    Error **errp);
+void cxl_fixed_memory_window_link_targets(Error **errp);
+
 #endif
diff --git a/qapi/machine.json b/qapi/machine.json
index 92480d4044..3f1eab3482 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -502,6 +502,27 @@
    'dst': 'uint16',
    'val': 'uint8' }}
 
+##
+# @CXLFixedMemoryWindowOptions:
+#
+# Create a CXL Fixed Memory Window
+#
+# @size: Size of the Fixed Memory Window in bytes. Must be a multiple
+#        of 256MiB.
+# @interleave-granularity: Number of contiguous bytes for which
+#                          accesses will go to a given interleave target.
+#                          Accepted values [256, 512, 1k, 2k, 4k, 8k, 16k]
+# @targets: Target root bridge IDs from -device ...,id=<ID> for each root
+#           bridge.
+#
+# Since 7.1
+##
+{ 'struct': 'CXLFixedMemoryWindowOptions',
+  'data': {
+      'size': 'size',
+      '*interleave-granularity': 'size',
+      'targets': ['str'] }}
+
 ##
 # @X86CPURegister32:
 #
diff --git a/qemu-options.hx b/qemu-options.hx
index 796229c433..315bb18595 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -467,6 +467,44 @@ SRST
         -numa hmat-cache,node-id=1,size=10K,level=1,associativity=direct,policy=write-back,line=8
 ERST
 
+DEF("cxl-fixed-memory-window", HAS_ARG, QEMU_OPTION_cxl_fixed_memory_window,
+    "-cxl-fixed-memory-window targets.0=firsttarget,targets.1=secondtarget,size=size[,interleave-granularity=granularity]\n",
+    QEMU_ARCH_ALL)
+SRST
+``-cxl-fixed-memory-window targets.0=firsttarget,targets.1=secondtarget,size=size[,interleave-granularity=granularity]``
+    Define a CXL Fixed Memory Window (CFMW).
+
+    Described in the CXL 2.0 ECN: CEDT CFMWS & QTG _DSM.
+
+    They are regions of Host Physical Addresses (HPA) on a system which
+    may be interleaved across one or more CXL host bridges.  The system
+    software will assign particular devices into these windows and
+    configure the downstream Host-managed Device Memory (HDM) decoders
+    in root ports, switch ports and devices appropriately to meet the
+    interleave requirements before enabling the memory devices.
+
+    ``targets.X=firsttarget`` provides the mapping to CXL host bridges
+    which may be identified by the id provied in the -device entry.
+    Multiple entries are needed to specify all the targets when
+    the fixed memory window represents interleaved memory. X is the
+    target index from 0.
+
+    ``size=size`` sets the size of the CFMW. This must be a multiple of
+    256MiB. The region will be aligned to 256MiB but the location is
+    platform and configuration dependent.
+
+    ``interleave-granularity=granularity`` sets the granularity of
+    interleave. Default 256KiB. Only 256KiB, 512KiB, 1024KiB, 2048KiB
+    4096KiB, 8192KiB and 16384KiB granularities supported.
+
+    Example:
+
+    ::
+
+        -cxl-fixed-memory-window targets.0=cxl.0,targets.1=cxl.1,size=128G,interleave-granularity=512k
+
+ERST
+
 DEF("add-fd", HAS_ARG, QEMU_OPTION_add_fd,
     "-add-fd fd=fd,set=set[,opaque=opaque]\n"
     "                Add 'fd' to fd 'set'\n", QEMU_ARCH_ALL)
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 817d515783..2390c13fb6 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -93,6 +93,7 @@
 #include "qemu/config-file.h"
 #include "qemu/qemu-options.h"
 #include "qemu/main-loop.h"
+#include "hw/cxl/cxl.h"
 #ifdef CONFIG_VIRTFS
 #include "fsdev/qemu-fsdev.h"
 #endif
@@ -118,6 +119,7 @@
 #include "qapi/qapi-events-run-state.h"
 #include "qapi/qapi-visit-block-core.h"
 #include "qapi/qapi-visit-compat.h"
+#include "qapi/qapi-visit-machine.h"
 #include "qapi/qapi-visit-ui.h"
 #include "qapi/qapi-commands-block-core.h"
 #include "qapi/qapi-commands-migration.h"
@@ -143,6 +145,12 @@ typedef struct BlockdevOptionsQueueEntry {
 
 typedef QSIMPLEQ_HEAD(, BlockdevOptionsQueueEntry) BlockdevOptionsQueue;
 
+typedef struct CXLFMWOptionQueueEntry {
+    CXLFixedMemoryWindowOptions *opts;
+    Location loc;
+    QSIMPLEQ_ENTRY(CXLFMWOptionQueueEntry) entry;
+} CXLFMWOptionQueueEntry;
+
 typedef struct ObjectOption {
     ObjectOptions *opts;
     QTAILQ_ENTRY(ObjectOption) next;
@@ -169,6 +177,8 @@ static int snapshot;
 static bool preconfig_requested;
 static QemuPluginList plugin_list = QTAILQ_HEAD_INITIALIZER(plugin_list);
 static BlockdevOptionsQueue bdo_queue = QSIMPLEQ_HEAD_INITIALIZER(bdo_queue);
+static QSIMPLEQ_HEAD(, CXLFMWOptionQueueEntry) CXLFMW_opts =
+    QSIMPLEQ_HEAD_INITIALIZER(CXLFMW_opts);
 static bool nographic = false;
 static int mem_prealloc; /* force preallocation of physical target memory */
 static const char *vga_model = NULL;
@@ -1153,6 +1163,24 @@ static void parse_display(const char *p)
     }
 }
 
+static void parse_cxl_fixed_memory_window(const char *optarg)
+{
+    CXLFMWOptionQueueEntry *cfmws_entry;
+    Visitor *v;
+
+    v = qobject_input_visitor_new_str(optarg, "cxl-fixed-memory-window",
+                                      &error_fatal);
+    cfmws_entry = g_new(CXLFMWOptionQueueEntry, 1);
+    visit_type_CXLFixedMemoryWindowOptions(v, NULL, &cfmws_entry->opts,
+                                           &error_fatal);
+    if (!cfmws_entry->opts) {
+        exit(1);
+    }
+    visit_free(v);
+    loc_save(&cfmws_entry->loc);
+    QSIMPLEQ_INSERT_TAIL(&CXLFMW_opts, cfmws_entry, entry);
+}
+
 static inline bool nonempty_str(const char *str)
 {
     return str && *str;
@@ -2015,6 +2043,20 @@ static void qemu_create_late_backends(void)
     qemu_semihosting_console_init();
 }
 
+static void cxl_set_opts(void)
+{
+    while (!QSIMPLEQ_EMPTY(&CXLFMW_opts)) {
+        CXLFMWOptionQueueEntry *cfmws_entry = QSIMPLEQ_FIRST(&CXLFMW_opts);
+
+        loc_restore(&cfmws_entry->loc);
+        QSIMPLEQ_REMOVE_HEAD(&CXLFMW_opts, entry);
+        cxl_fixed_memory_window_config(current_machine, cfmws_entry->opts,
+                                       &error_fatal);
+        qapi_free_CXLFixedMemoryWindowOptions(cfmws_entry->opts);
+        g_free(cfmws_entry);
+    }
+}
+
 static void qemu_resolve_machine_memdev(void)
 {
     if (ram_memdev_id) {
@@ -2661,6 +2703,7 @@ void qmp_x_exit_preconfig(Error **errp)
 
     qemu_init_board();
     qemu_create_cli_devices();
+    cxl_fixed_memory_window_link_targets(errp);
     qemu_machine_creation_done();
 
     if (loadvm) {
@@ -2841,6 +2884,9 @@ void qemu_init(int argc, char **argv, char **envp)
                     exit(1);
                 }
                 break;
+            case QEMU_OPTION_cxl_fixed_memory_window:
+                parse_cxl_fixed_memory_window(optarg);
+                break;
             case QEMU_OPTION_display:
                 parse_display(optarg);
                 break;
@@ -3652,6 +3698,7 @@ void qemu_init(int argc, char **argv, char **envp)
 
     qemu_resolve_machine_memdev();
     parse_numa_opts(current_machine);
+    cxl_set_opts();
 
     if (vmstate_dump_file) {
         /* dump and exit */

From 21df6ab97ff2c0de76ce683808356aaf98d9d1d2 Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:53 +0100
Subject: [PATCH 297/935] acpi/cxl: Introduce CFMWS structures in CEDT
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The CEDT CXL Fixed Window Memory Window Structures (CFMWs)
define regions of the host phyiscal address map which
(via an impdef means) are configured such that they have
a particular interleave setup across one or more CXL Host Bridges.

Reported-by: Alison Schofield <alison.schofield@intel.com>
Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-29-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/acpi/cxl.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)

diff --git a/hw/acpi/cxl.c b/hw/acpi/cxl.c
index aa4af86a4c..31d5235136 100644
--- a/hw/acpi/cxl.c
+++ b/hw/acpi/cxl.c
@@ -60,6 +60,64 @@ static void cedt_build_chbs(GArray *table_data, PXBDev *cxl)
     build_append_int_noprefix(table_data, memory_region_size(mr), 8);
 }
 
+/*
+ * CFMWS entries in CXL 2.0 ECN: CEDT CFMWS & QTG _DSM.
+ * Interleave ways encoding in CXL 2.0 ECN: 3, 6, 12 and 16-way memory
+ * interleaving.
+ */
+static void cedt_build_cfmws(GArray *table_data, MachineState *ms)
+{
+    CXLState *cxls = ms->cxl_devices_state;
+    GList *it;
+
+    for (it = cxls->fixed_windows; it; it = it->next) {
+        CXLFixedWindow *fw = it->data;
+        int i;
+
+        /* Type */
+        build_append_int_noprefix(table_data, 1, 1);
+
+        /* Reserved */
+        build_append_int_noprefix(table_data, 0, 1);
+
+        /* Record Length */
+        build_append_int_noprefix(table_data, 36 + 4 * fw->num_targets, 2);
+
+        /* Reserved */
+        build_append_int_noprefix(table_data, 0, 4);
+
+        /* Base HPA */
+        build_append_int_noprefix(table_data, fw->mr.addr, 8);
+
+        /* Window Size */
+        build_append_int_noprefix(table_data, fw->size, 8);
+
+        /* Host Bridge Interleave Ways */
+        build_append_int_noprefix(table_data, fw->enc_int_ways, 1);
+
+        /* Host Bridge Interleave Arithmetic */
+        build_append_int_noprefix(table_data, 0, 1);
+
+        /* Reserved */
+        build_append_int_noprefix(table_data, 0, 2);
+
+        /* Host Bridge Interleave Granularity */
+        build_append_int_noprefix(table_data, fw->enc_int_gran, 4);
+
+        /* Window Restrictions */
+        build_append_int_noprefix(table_data, 0x0f, 2); /* No restrictions */
+
+        /* QTG ID */
+        build_append_int_noprefix(table_data, 0, 2);
+
+        /* Host Bridge List (list of UIDs - currently bus_nr) */
+        for (i = 0; i < fw->num_targets; i++) {
+            g_assert(fw->target_hbs[i]);
+            build_append_int_noprefix(table_data, fw->target_hbs[i]->bus_nr, 4);
+        }
+    }
+}
+
 static int cxl_foreach_pxb_hb(Object *obj, void *opaque)
 {
     Aml *cedt = opaque;
@@ -86,6 +144,7 @@ void cxl_build_cedt(MachineState *ms, GArray *table_offsets, GArray *table_data,
     /* reserve space for CEDT header */
 
     object_child_foreach_recursive(object_get_root(), cxl_foreach_pxb_hb, cedt);
+    cedt_build_cfmws(cedt->buf, ms);
 
     /* copy AML table into ACPI tables blob and patch header there */
     g_array_append_vals(table_data, cedt->buf->data, cedt->buf->len);

From fc1e01e009fd4b1e08399253591b32e01f10808f Mon Sep 17 00:00:00 2001
From: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Date: Fri, 29 Apr 2022 15:40:54 +0100
Subject: [PATCH 298/935] hw/pci-host/gpex-acpi: Add support for dsdt
 construction for pxb-cxl
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This adds code to instantiate the slightly extended ACPI root port
description in DSDT as per the CXL 2.0 specification.

Basically a cut and paste job from the i386/pc code.

Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-30-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/arm/Kconfig          |  1 +
 hw/pci-host/gpex-acpi.c | 20 +++++++++++++++++---
 2 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig
index 97f3b38019..219262a8da 100644
--- a/hw/arm/Kconfig
+++ b/hw/arm/Kconfig
@@ -29,6 +29,7 @@ config ARM_VIRT
     select ACPI_APEI
     select ACPI_VIOT
     select VIRTIO_MEM_SUPPORTED
+    select ACPI_CXL
 
 config CHEETAH
     bool
diff --git a/hw/pci-host/gpex-acpi.c b/hw/pci-host/gpex-acpi.c
index e7e162a00a..7c7316bc96 100644
--- a/hw/pci-host/gpex-acpi.c
+++ b/hw/pci-host/gpex-acpi.c
@@ -5,6 +5,7 @@
 #include "hw/pci/pci_bus.h"
 #include "hw/pci/pci_bridge.h"
 #include "hw/pci/pcie_host.h"
+#include "hw/acpi/cxl.h"
 
 static void acpi_dsdt_add_pci_route_table(Aml *dev, uint32_t irq)
 {
@@ -139,6 +140,7 @@ void acpi_dsdt_add_gpex(Aml *scope, struct GPEXConfig *cfg)
         QLIST_FOREACH(bus, &bus->child, sibling) {
             uint8_t bus_num = pci_bus_num(bus);
             uint8_t numa_node = pci_bus_numa_node(bus);
+            bool is_cxl = pci_bus_is_cxl(bus);
 
             if (!pci_bus_is_root(bus)) {
                 continue;
@@ -154,8 +156,16 @@ void acpi_dsdt_add_gpex(Aml *scope, struct GPEXConfig *cfg)
             }
 
             dev = aml_device("PC%.02X", bus_num);
-            aml_append(dev, aml_name_decl("_HID", aml_string("PNP0A08")));
-            aml_append(dev, aml_name_decl("_CID", aml_string("PNP0A03")));
+            if (is_cxl) {
+                struct Aml *pkg = aml_package(2);
+                aml_append(dev, aml_name_decl("_HID", aml_string("ACPI0016")));
+                aml_append(pkg, aml_eisaid("PNP0A08"));
+                aml_append(pkg, aml_eisaid("PNP0A03"));
+                aml_append(dev, aml_name_decl("_CID", pkg));
+            } else {
+                aml_append(dev, aml_name_decl("_HID", aml_string("PNP0A08")));
+                aml_append(dev, aml_name_decl("_CID", aml_string("PNP0A03")));
+            }
             aml_append(dev, aml_name_decl("_BBN", aml_int(bus_num)));
             aml_append(dev, aml_name_decl("_UID", aml_int(bus_num)));
             aml_append(dev, aml_name_decl("_STR", aml_unicode("pxb Device")));
@@ -175,7 +185,11 @@ void acpi_dsdt_add_gpex(Aml *scope, struct GPEXConfig *cfg)
                             cfg->pio.base, 0, 0, 0);
             aml_append(dev, aml_name_decl("_CRS", crs));
 
-            acpi_dsdt_add_pci_osc(dev);
+            if (is_cxl) {
+                build_cxl_osc_method(dev);
+            } else {
+                acpi_dsdt_add_pci_osc(dev);
+            }
 
             aml_append(scope, dev);
         }

From aa970ed586f9c7f178b813bda2919e329b841e3c Mon Sep 17 00:00:00 2001
From: Jonathan Cameron <jonathan.cameron@huawei.com>
Date: Fri, 29 Apr 2022 15:40:55 +0100
Subject: [PATCH 299/935] pci/pcie_port: Add pci_find_port_by_pn()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Simple function to search a PCIBus to find a port by
it's port number.

CXL interleave decoding uses the port number as a target
so it is necessary to locate the port when doing interleave
decoding.

Signed-off-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-31-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/pci/pcie_port.c         | 25 +++++++++++++++++++++++++
 include/hw/pci/pcie_port.h |  2 ++
 2 files changed, 27 insertions(+)

diff --git a/hw/pci/pcie_port.c b/hw/pci/pcie_port.c
index e95c1e5519..687e4e763a 100644
--- a/hw/pci/pcie_port.c
+++ b/hw/pci/pcie_port.c
@@ -136,6 +136,31 @@ static void pcie_port_class_init(ObjectClass *oc, void *data)
     device_class_set_props(dc, pcie_port_props);
 }
 
+PCIDevice *pcie_find_port_by_pn(PCIBus *bus, uint8_t pn)
+{
+    int devfn;
+
+    for (devfn = 0; devfn < ARRAY_SIZE(bus->devices); devfn++) {
+        PCIDevice *d = bus->devices[devfn];
+        PCIEPort *port;
+
+        if (!d || !pci_is_express(d) || !d->exp.exp_cap) {
+            continue;
+        }
+
+        if (!object_dynamic_cast(OBJECT(d), TYPE_PCIE_PORT)) {
+            continue;
+        }
+
+        port = PCIE_PORT(d);
+        if (port->port == pn) {
+            return d;
+        }
+    }
+
+    return NULL;
+}
+
 static const TypeInfo pcie_port_type_info = {
     .name = TYPE_PCIE_PORT,
     .parent = TYPE_PCI_BRIDGE,
diff --git a/include/hw/pci/pcie_port.h b/include/hw/pci/pcie_port.h
index e25b289ce8..7b8193061a 100644
--- a/include/hw/pci/pcie_port.h
+++ b/include/hw/pci/pcie_port.h
@@ -39,6 +39,8 @@ struct PCIEPort {
 
 void pcie_port_init_reg(PCIDevice *d);
 
+PCIDevice *pcie_find_port_by_pn(PCIBus *bus, uint8_t pn);
+
 #define TYPE_PCIE_SLOT "pcie-slot"
 OBJECT_DECLARE_SIMPLE_TYPE(PCIESlot, PCIE_SLOT)
 

From 0b4aec246972f238a22d04403289eee97e8c8be6 Mon Sep 17 00:00:00 2001
From: Jonathan Cameron <jonathan.cameron@huawei.com>
Date: Fri, 29 Apr 2022 15:40:56 +0100
Subject: [PATCH 300/935] CXL/cxl_component: Add cxl_get_hb_cstate()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Accessor to get hold of the cxl state for a CXL host bridge
without exposing the internals of the implementation.

Signed-off-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220429144110.25167-32-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/pci-bridge/pci_expander_bridge.c | 7 +++++++
 include/hw/cxl/cxl_component.h      | 2 ++
 2 files changed, 9 insertions(+)

diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c
index 22feda1ff0..69244decdb 100644
--- a/hw/pci-bridge/pci_expander_bridge.c
+++ b/hw/pci-bridge/pci_expander_bridge.c
@@ -72,6 +72,13 @@ static GList *pxb_dev_list;
 
 #define TYPE_PXB_HOST "pxb-host"
 
+CXLComponentState *cxl_get_hb_cstate(PCIHostState *hb)
+{
+    CXLHost *host = PXB_CXL_HOST(hb);
+
+    return &host->cxl_cstate;
+}
+
 static int pxb_bus_num(PCIBus *bus)
 {
     PXBDev *pxb = convert_to_pxb(bus->parent_dev);
diff --git a/include/hw/cxl/cxl_component.h b/include/hw/cxl/cxl_component.h
index 4f69688c47..70b5018156 100644
--- a/include/hw/cxl/cxl_component.h
+++ b/include/hw/cxl/cxl_component.h
@@ -218,4 +218,6 @@ static inline hwaddr cxl_decode_ig(int ig)
     return 1 << (ig + 8);
 }
 
+CXLComponentState *cxl_get_hb_cstate(PCIHostState *hb);
+
 #endif

From 5fcc499ee3457709657b23655e385613a437068d Mon Sep 17 00:00:00 2001
From: Jonathan Cameron <jonathan.cameron@huawei.com>
Date: Fri, 29 Apr 2022 15:40:57 +0100
Subject: [PATCH 301/935] mem/cxl_type3: Add read and write functions for
 associated hostmem.

Once a read or write reaches a CXL type 3 device, the HDM decoders
on the device are used to establish the Device Physical Address
which should be accessed.  These functions peform the required maths
and then use a device specific address space to access the
hostmem->mr to fullfil the actual operation.  Note that failed writes
are silent, but failed reads return poison.  Note this is based
loosely on:

https://lore.kernel.org/qemu-devel/20200817161853.593247-6-f4bug@amsat.org/
[RFC PATCH 0/9] hw/misc: Add support for interleaved memory accesses

Only lightly tested so far.  More complex test cases yet to be written.

Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Message-Id: <20220429144110.25167-33-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/mem/cxl_type3.c          | 91 +++++++++++++++++++++++++++++++++++++
 include/hw/cxl/cxl_device.h |  6 +++
 2 files changed, 97 insertions(+)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 6c6ed9a776..3bf2869573 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -97,7 +97,9 @@ static void ct3d_reg_write(void *opaque, hwaddr offset, uint64_t value,
 
 static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
 {
+    DeviceState *ds = DEVICE(ct3d);
     MemoryRegion *mr;
+    char *name;
 
     if (!ct3d->hostmem) {
         error_setg(errp, "memdev property must be set");
@@ -112,6 +114,15 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
     memory_region_set_nonvolatile(mr, true);
     memory_region_set_enabled(mr, true);
     host_memory_backend_set_mapped(ct3d->hostmem, true);
+
+    if (ds->id) {
+        name = g_strdup_printf("cxl-type3-dpa-space:%s", ds->id);
+    } else {
+        name = g_strdup("cxl-type3-dpa-space");
+    }
+    address_space_init(&ct3d->hostmem_as, mr, name);
+    g_free(name);
+
     ct3d->cxl_dstate.pmem_size = ct3d->hostmem->size;
 
     if (!ct3d->lsa) {
@@ -167,6 +178,86 @@ static void ct3_exit(PCIDevice *pci_dev)
     ComponentRegisters *regs = &cxl_cstate->crb;
 
     g_free(regs->special_ops);
+    address_space_destroy(&ct3d->hostmem_as);
+}
+
+/* TODO: Support multiple HDM decoders and DPA skip */
+static bool cxl_type3_dpa(CXLType3Dev *ct3d, hwaddr host_addr, uint64_t *dpa)
+{
+    uint32_t *cache_mem = ct3d->cxl_cstate.crb.cache_mem_registers;
+    uint64_t decoder_base, decoder_size, hpa_offset;
+    uint32_t hdm0_ctrl;
+    int ig, iw;
+
+    decoder_base = (((uint64_t)cache_mem[R_CXL_HDM_DECODER0_BASE_HI] << 32) |
+                    cache_mem[R_CXL_HDM_DECODER0_BASE_LO]);
+    if ((uint64_t)host_addr < decoder_base) {
+        return false;
+    }
+
+    hpa_offset = (uint64_t)host_addr - decoder_base;
+
+    decoder_size = ((uint64_t)cache_mem[R_CXL_HDM_DECODER0_SIZE_HI] << 32) |
+        cache_mem[R_CXL_HDM_DECODER0_SIZE_LO];
+    if (hpa_offset >= decoder_size) {
+        return false;
+    }
+
+    hdm0_ctrl = cache_mem[R_CXL_HDM_DECODER0_CTRL];
+    iw = FIELD_EX32(hdm0_ctrl, CXL_HDM_DECODER0_CTRL, IW);
+    ig = FIELD_EX32(hdm0_ctrl, CXL_HDM_DECODER0_CTRL, IG);
+
+    *dpa = (MAKE_64BIT_MASK(0, 8 + ig) & hpa_offset) |
+        ((MAKE_64BIT_MASK(8 + ig + iw, 64 - 8 - ig - iw) & hpa_offset) >> iw);
+
+    return true;
+}
+
+MemTxResult cxl_type3_read(PCIDevice *d, hwaddr host_addr, uint64_t *data,
+                           unsigned size, MemTxAttrs attrs)
+{
+    CXLType3Dev *ct3d = CXL_TYPE3(d);
+    uint64_t dpa_offset;
+    MemoryRegion *mr;
+
+    /* TODO support volatile region */
+    mr = host_memory_backend_get_memory(ct3d->hostmem);
+    if (!mr) {
+        return MEMTX_ERROR;
+    }
+
+    if (!cxl_type3_dpa(ct3d, host_addr, &dpa_offset)) {
+        return MEMTX_ERROR;
+    }
+
+    if (dpa_offset > int128_get64(mr->size)) {
+        return MEMTX_ERROR;
+    }
+
+    return address_space_read(&ct3d->hostmem_as, dpa_offset, attrs, data, size);
+}
+
+MemTxResult cxl_type3_write(PCIDevice *d, hwaddr host_addr, uint64_t data,
+                            unsigned size, MemTxAttrs attrs)
+{
+    CXLType3Dev *ct3d = CXL_TYPE3(d);
+    uint64_t dpa_offset;
+    MemoryRegion *mr;
+
+    mr = host_memory_backend_get_memory(ct3d->hostmem);
+    if (!mr) {
+        return MEMTX_OK;
+    }
+
+    if (!cxl_type3_dpa(ct3d, host_addr, &dpa_offset)) {
+        return MEMTX_OK;
+    }
+
+    if (dpa_offset > int128_get64(mr->size)) {
+        return MEMTX_OK;
+    }
+    return address_space_write(&ct3d->hostmem_as, dpa_offset, attrs,
+                               &data, size);
 }
 
 static void ct3d_reset(DeviceState *dev)
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 4285fbda08..1e141b6621 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -239,6 +239,7 @@ struct CXLType3Dev {
     HostMemoryBackend *lsa;
 
     /* State */
+    AddressSpace hostmem_as;
     CXLComponentState cxl_cstate;
     CXLDeviceState cxl_dstate;
 };
@@ -259,4 +260,9 @@ struct CXLType3Class {
                     uint64_t offset);
 };
 
+MemTxResult cxl_type3_read(PCIDevice *d, hwaddr host_addr, uint64_t *data,
+                           unsigned size, MemTxAttrs attrs);
+MemTxResult cxl_type3_write(PCIDevice *d, hwaddr host_addr, uint64_t data,
+                            unsigned size, MemTxAttrs attrs);
+
 #endif

From eb19d9079efc4e986a37b0c3172ecd9b617fd04a Mon Sep 17 00:00:00 2001
From: Jonathan Cameron <jonathan.cameron@huawei.com>
Date: Fri, 29 Apr 2022 15:40:58 +0100
Subject: [PATCH 302/935] cxl/cxl-host: Add memops for CFMWS region.

These memops perform interleave decoding, walking down the
CXL topology from CFMWS described host interleave
decoder via CXL host bridge HDM decoders, through the CXL
root ports and finally call CXL type 3 specific read and write
functions.

Note that, whilst functional the current implementation does
not support:
* switches
* multiple HDM decoders at a given level.
* unaligned accesses across the interleave boundaries

Signed-off-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Message-Id: <20220429144110.25167-34-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/cxl/cxl-host-stubs.c |   2 +
 hw/cxl/cxl-host.c       | 128 ++++++++++++++++++++++++++++++++++++++++
 include/hw/cxl/cxl.h    |   2 +
 3 files changed, 132 insertions(+)

diff --git a/hw/cxl/cxl-host-stubs.c b/hw/cxl/cxl-host-stubs.c
index f8fd278d5d..24465a52ab 100644
--- a/hw/cxl/cxl-host-stubs.c
+++ b/hw/cxl/cxl-host-stubs.c
@@ -12,3 +12,5 @@ void cxl_fixed_memory_window_config(MachineState *ms,
                                     Error **errp) {};
 
 void cxl_fixed_memory_window_link_targets(Error **errp) {};
+
+const MemoryRegionOps cfmws_ops;
diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c
index ec5a75cbf5..469b3c4ced 100644
--- a/hw/cxl/cxl-host.c
+++ b/hw/cxl/cxl-host.c
@@ -15,6 +15,10 @@
 
 #include "qapi/qapi-visit-machine.h"
 #include "hw/cxl/cxl.h"
+#include "hw/pci/pci_bus.h"
+#include "hw/pci/pci_bridge.h"
+#include "hw/pci/pci_host.h"
+#include "hw/pci/pcie_port.h"
 
 void cxl_fixed_memory_window_config(MachineState *ms,
                                     CXLFixedMemoryWindowOptions *object,
@@ -92,3 +96,127 @@ void cxl_fixed_memory_window_link_targets(Error **errp)
         }
     }
 }
+
+/* TODO: support, multiple hdm decoders */
+static bool cxl_hdm_find_target(uint32_t *cache_mem, hwaddr addr,
+                                uint8_t *target)
+{
+    uint32_t ctrl;
+    uint32_t ig_enc;
+    uint32_t iw_enc;
+    uint32_t target_reg;
+    uint32_t target_idx;
+
+    ctrl = cache_mem[R_CXL_HDM_DECODER0_CTRL];
+    if (!FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED)) {
+        return false;
+    }
+
+    ig_enc = FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IG);
+    iw_enc = FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IW);
+    target_idx = (addr / cxl_decode_ig(ig_enc)) % (1 << iw_enc);
+
+    if (target_idx > 4) {
+        target_reg = cache_mem[R_CXL_HDM_DECODER0_TARGET_LIST_LO];
+        target_reg >>= target_idx * 8;
+    } else {
+        target_reg = cache_mem[R_CXL_HDM_DECODER0_TARGET_LIST_LO];
+        target_reg >>= (target_idx - 4) * 8;
+    }
+    *target = target_reg & 0xff;
+
+    return true;
+}
+
+static PCIDevice *cxl_cfmws_find_device(CXLFixedWindow *fw, hwaddr addr)
+{
+    CXLComponentState *hb_cstate;
+    PCIHostState *hb;
+    int rb_index;
+    uint32_t *cache_mem;
+    uint8_t target;
+    bool target_found;
+    PCIDevice *rp, *d;
+
+    /* Address is relative to memory region. Convert to HPA */
+    addr += fw->base;
+
+    rb_index = (addr / cxl_decode_ig(fw->enc_int_gran)) % fw->num_targets;
+    hb = PCI_HOST_BRIDGE(fw->target_hbs[rb_index]->cxl.cxl_host_bridge);
+    if (!hb || !hb->bus || !pci_bus_is_cxl(hb->bus)) {
+        return NULL;
+    }
+
+    hb_cstate = cxl_get_hb_cstate(hb);
+    if (!hb_cstate) {
+        return NULL;
+    }
+
+    cache_mem = hb_cstate->crb.cache_mem_registers;
+
+    target_found = cxl_hdm_find_target(cache_mem, addr, &target);
+    if (!target_found) {
+        return NULL;
+    }
+
+    rp = pcie_find_port_by_pn(hb->bus, target);
+    if (!rp) {
+        return NULL;
+    }
+
+    d = pci_bridge_get_sec_bus(PCI_BRIDGE(rp))->devices[0];
+
+    if (!d || !object_dynamic_cast(OBJECT(d), TYPE_CXL_TYPE3)) {
+        return NULL;
+    }
+
+    return d;
+}
+
+static MemTxResult cxl_read_cfmws(void *opaque, hwaddr addr, uint64_t *data,
+                                  unsigned size, MemTxAttrs attrs)
+{
+    CXLFixedWindow *fw = opaque;
+    PCIDevice *d;
+
+    d = cxl_cfmws_find_device(fw, addr);
+    if (d == NULL) {
+        *data = 0;
+        /* Reads to invalid address return poison */
+        return MEMTX_ERROR;
+    }
+
+    return cxl_type3_read(d, addr + fw->base, data, size, attrs);
+}
+
+static MemTxResult cxl_write_cfmws(void *opaque, hwaddr addr,
+                                   uint64_t data, unsigned size,
+                                   MemTxAttrs attrs)
+{
+    CXLFixedWindow *fw = opaque;
+    PCIDevice *d;
+
+    d = cxl_cfmws_find_device(fw, addr);
+    if (d == NULL) {
+        /* Writes to invalid address are silent */
+        return MEMTX_OK;
+    }
+
+    return cxl_type3_write(d, addr + fw->base, data, size, attrs);
+}
+
+const MemoryRegionOps cfmws_ops = {
+    .read_with_attrs = cxl_read_cfmws,
+    .write_with_attrs = cxl_write_cfmws,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+        .unaligned = true,
+    },
+    .impl = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+        .unaligned = true,
+    },
+};
diff --git a/include/hw/cxl/cxl.h b/include/hw/cxl/cxl.h
index dce38124db..21d28ca110 100644
--- a/include/hw/cxl/cxl.h
+++ b/include/hw/cxl/cxl.h
@@ -56,4 +56,6 @@ void cxl_fixed_memory_window_config(MachineState *ms,
                                     Error **errp);
 void cxl_fixed_memory_window_link_targets(Error **errp);
 
+extern const MemoryRegionOps cfmws_ops;
+
 #endif

From 3540bf56e47dce55e5035a039db97c0affc66fba Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:40:59 +0100
Subject: [PATCH 303/935] hw/cxl/component Add a dumb HDM decoder handler

Add a trivial handler for now to cover the root bridge
where we could do some error checking in future.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Message-Id: <20220429144110.25167-35-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/cxl/cxl-component-utils.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c
index 69cb07171c..7985c9bfca 100644
--- a/hw/cxl/cxl-component-utils.c
+++ b/hw/cxl/cxl-component-utils.c
@@ -32,6 +32,31 @@ static uint64_t cxl_cache_mem_read_reg(void *opaque, hwaddr offset,
     }
 }
 
+static void dumb_hdm_handler(CXLComponentState *cxl_cstate, hwaddr offset,
+                             uint32_t value)
+{
+    ComponentRegisters *cregs = &cxl_cstate->crb;
+    uint32_t *cache_mem = cregs->cache_mem_registers;
+    bool should_commit = false;
+
+    switch (offset) {
+    case A_CXL_HDM_DECODER0_CTRL:
+        should_commit = FIELD_EX32(value, CXL_HDM_DECODER0_CTRL, COMMIT);
+        break;
+    default:
+        break;
+    }
+
+    memory_region_transaction_begin();
+    stl_le_p((uint8_t *)cache_mem + offset, value);
+    if (should_commit) {
+        ARRAY_FIELD_DP32(cache_mem, CXL_HDM_DECODER0_CTRL, COMMIT, 0);
+        ARRAY_FIELD_DP32(cache_mem, CXL_HDM_DECODER0_CTRL, ERR, 0);
+        ARRAY_FIELD_DP32(cache_mem, CXL_HDM_DECODER0_CTRL, COMMITTED, 1);
+    }
+    memory_region_transaction_commit();
+}
+
 static void cxl_cache_mem_write_reg(void *opaque, hwaddr offset, uint64_t value,
                                     unsigned size)
 {
@@ -50,6 +75,12 @@ static void cxl_cache_mem_write_reg(void *opaque, hwaddr offset, uint64_t value,
     value |= ~mask & cregs->cache_mem_registers[offset / sizeof(*cregs->cache_mem_registers)];
     if (cregs->special_ops && cregs->special_ops->write) {
         cregs->special_ops->write(cxl_cstate, offset, value, size);
+        return;
+    }
+
+    if (offset >= A_CXL_HDM_DECODER_CAPABILITY &&
+        offset <= A_CXL_HDM_DECODER0_TARGET_LIST_HI) {
+        dumb_hdm_handler(cxl_cstate, offset, value);
     } else {
         cregs->cache_mem_registers[offset / sizeof(*cregs->cache_mem_registers)] = value;
     }

From 6d302cf46dfffa85a1a460d4431b1633c9ef2ff3 Mon Sep 17 00:00:00 2001
From: Jonathan Cameron <jonathan.cameron@huawei.com>
Date: Fri, 29 Apr 2022 15:41:00 +0100
Subject: [PATCH 304/935] i386/pc: Enable CXL fixed memory windows

Add the CFMWs memory regions to the memorymap and adjust the
PCI window to avoid hitting the same memory.

Signed-off-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Message-Id: <20220429144110.25167-36-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/pc.c | 31 ++++++++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 03d14f6564..312eb9e400 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -817,7 +817,7 @@ void pc_memory_init(PCMachineState *pcms,
     MachineClass *mc = MACHINE_GET_CLASS(machine);
     PCMachineClass *pcmc = PC_MACHINE_GET_CLASS(pcms);
     X86MachineState *x86ms = X86_MACHINE(pcms);
-    hwaddr cxl_base;
+    hwaddr cxl_base, cxl_resv_end = 0;
 
     assert(machine->ram_size == x86ms->below_4g_mem_size +
                                 x86ms->above_4g_mem_size);
@@ -925,6 +925,24 @@ void pc_memory_init(PCMachineState *pcms,
         e820_add_entry(cxl_base, cxl_size, E820_RESERVED);
         memory_region_init(mr, OBJECT(machine), "cxl_host_reg", cxl_size);
         memory_region_add_subregion(system_memory, cxl_base, mr);
+        cxl_resv_end = cxl_base + cxl_size;
+        if (machine->cxl_devices_state->fixed_windows) {
+            hwaddr cxl_fmw_base;
+            GList *it;
+
+            cxl_fmw_base = ROUND_UP(cxl_base + cxl_size, 256 * MiB);
+            for (it = machine->cxl_devices_state->fixed_windows; it; it = it->next) {
+                CXLFixedWindow *fw = it->data;
+
+                fw->base = cxl_fmw_base;
+                memory_region_init_io(&fw->mr, OBJECT(machine), &cfmws_ops, fw,
+                                      "cxl-fixed-memory-region", fw->size);
+                memory_region_add_subregion(system_memory, fw->base, &fw->mr);
+                e820_add_entry(fw->base, fw->size, E820_RESERVED);
+                cxl_fmw_base += fw->size;
+                cxl_resv_end = cxl_fmw_base;
+            }
+        }
     }
 
     /* Initialize PC system firmware */
@@ -954,6 +972,10 @@ void pc_memory_init(PCMachineState *pcms,
         if (!pcmc->broken_reserved_end) {
             res_mem_end += memory_region_size(&machine->device_memory->mr);
         }
+
+        if (machine->cxl_devices_state->is_enabled) {
+            res_mem_end = cxl_resv_end;
+        }
         *val = cpu_to_le64(ROUND_UP(res_mem_end, 1 * GiB));
         fw_cfg_add_file(fw_cfg, "etc/reserved-memory-end", val, sizeof(*val));
     }
@@ -990,6 +1012,13 @@ uint64_t pc_pci_hole64_start(void)
     if (ms->cxl_devices_state->host_mr.addr) {
         hole64_start = ms->cxl_devices_state->host_mr.addr +
             memory_region_size(&ms->cxl_devices_state->host_mr);
+        if (ms->cxl_devices_state->fixed_windows) {
+            GList *it;
+            for (it = ms->cxl_devices_state->fixed_windows; it; it = it->next) {
+                CXLFixedWindow *fw = it->data;
+                hole64_start = fw->mr.addr + memory_region_size(&fw->mr);
+            }
+        }
     } else if (pcmc->has_reserved_memory && ms->device_memory->base) {
         hole64_start = ms->device_memory->base;
         if (!pcmc->broken_reserved_end) {

From 502f99abaa58e82b186cc914e7d39d5f2bc0f1b4 Mon Sep 17 00:00:00 2001
From: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Date: Fri, 29 Apr 2022 15:41:01 +0100
Subject: [PATCH 305/935] tests/acpi: q35: Allow addition of a CXL test.

Add exceptions for the DSDT and the new CEDT tables
specific to a new CXL test in the following patch.

Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Message-Id: <20220429144110.25167-37-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/data/acpi/q35/CEDT.cxl                | 0
 tests/data/acpi/q35/DSDT.cxl                | 0
 tests/qtest/bios-tables-test-allowed-diff.h | 2 ++
 3 files changed, 2 insertions(+)
 create mode 100644 tests/data/acpi/q35/CEDT.cxl
 create mode 100644 tests/data/acpi/q35/DSDT.cxl

diff --git a/tests/data/acpi/q35/CEDT.cxl b/tests/data/acpi/q35/CEDT.cxl
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/data/acpi/q35/DSDT.cxl b/tests/data/acpi/q35/DSDT.cxl
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index dfb8523c8b..7c7f9fbc44 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1 +1,3 @@
 /* List of comma-separated changed AML files to ignore */
+"tests/data/acpi/q35/DSDT.cxl",
+"tests/data/acpi/q35/CEDT.cxl",

From 65fc04ff3cf5b91c0490c80714f97ae6a43d7ee8 Mon Sep 17 00:00:00 2001
From: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Date: Fri, 29 Apr 2022 15:41:02 +0100
Subject: [PATCH 306/935] qtests/bios-tables-test: Add a test for CXL
 emulation.

The DSDT includes several CXL specific elements and the CEDT
table is only present if we enable CXL.

The test exercises all current functionality with several
CFMWS, CHBS structures in CEDT and ACPI0016/ACPI00017 and _OSC
entries in DSDT.

Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Message-Id: <20220429144110.25167-38-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/bios-tables-test.c | 44 ++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/tests/qtest/bios-tables-test.c b/tests/qtest/bios-tables-test.c
index 5dddedabcd..a4a46e97f0 100644
--- a/tests/qtest/bios-tables-test.c
+++ b/tests/qtest/bios-tables-test.c
@@ -1536,6 +1536,49 @@ static void test_acpi_q35_viot(void)
     free_test_data(&data);
 }
 
+static void test_acpi_q35_cxl(void)
+{
+    gchar *tmp_path = g_dir_make_tmp("qemu-test-cxl.XXXXXX", NULL);
+    gchar *params;
+
+    test_data data = {
+        .machine = MACHINE_Q35,
+        .variant = ".cxl",
+    };
+    /*
+     * A complex CXL setup.
+     */
+    params = g_strdup_printf(" -machine cxl=on"
+                             " -object memory-backend-file,id=cxl-mem1,mem-path=%s,size=256M"
+                             " -object memory-backend-file,id=cxl-mem2,mem-path=%s,size=256M"
+                             " -object memory-backend-file,id=cxl-mem3,mem-path=%s,size=256M"
+                             " -object memory-backend-file,id=cxl-mem4,mem-path=%s,size=256M"
+                             " -object memory-backend-file,id=lsa1,mem-path=%s,size=256M"
+                             " -object memory-backend-file,id=lsa2,mem-path=%s,size=256M"
+                             " -object memory-backend-file,id=lsa3,mem-path=%s,size=256M"
+                             " -object memory-backend-file,id=lsa4,mem-path=%s,size=256M"
+                             " -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1"
+                             " -device pxb-cxl,bus_nr=222,bus=pcie.0,id=cxl.2"
+                             " -device cxl-rp,port=0,bus=cxl.1,id=rp1,chassis=0,slot=2"
+                             " -device cxl-type3,bus=rp1,memdev=cxl-mem1,lsa=lsa1"
+                             " -device cxl-rp,port=1,bus=cxl.1,id=rp2,chassis=0,slot=3"
+                             " -device cxl-type3,bus=rp2,memdev=cxl-mem2,lsa=lsa2"
+                             " -device cxl-rp,port=0,bus=cxl.2,id=rp3,chassis=0,slot=5"
+                             " -device cxl-type3,bus=rp3,memdev=cxl-mem3,lsa=lsa3"
+                             " -device cxl-rp,port=1,bus=cxl.2,id=rp4,chassis=0,slot=6"
+                             " -device cxl-type3,bus=rp4,memdev=cxl-mem4,lsa=lsa4"
+                             " -cxl-fixed-memory-window targets.0=cxl.1,size=4G,interleave-granularity=8k"
+                             " -cxl-fixed-memory-window targets.0=cxl.1,targets.1=cxl.2,size=4G,interleave-granularity=8k",
+                             tmp_path, tmp_path, tmp_path, tmp_path,
+                             tmp_path, tmp_path, tmp_path, tmp_path);
+    test_acpi_one(params, &data);
+
+    g_free(params);
+    g_assert(g_rmdir(tmp_path) == 0);
+    g_free(tmp_path);
+    free_test_data(&data);
+}
+
 static void test_acpi_virt_viot(void)
 {
     test_data data = {
@@ -1741,6 +1784,7 @@ int main(int argc, char *argv[])
             qtest_add_func("acpi/q35/kvm/dmar", test_acpi_q35_kvm_dmar);
         }
         qtest_add_func("acpi/q35/viot", test_acpi_q35_viot);
+        qtest_add_func("acpi/q35/cxl", test_acpi_q35_cxl);
         qtest_add_func("acpi/q35/slic", test_acpi_q35_slic);
     } else if (strcmp(arch, "aarch64") == 0) {
         if (has_tcg) {

From aa48799530ad506c311566da5ea8ef654cdb8a4e Mon Sep 17 00:00:00 2001
From: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Date: Fri, 29 Apr 2022 15:41:03 +0100
Subject: [PATCH 307/935] tests/acpi: Add tables for CXL emulation.

Tables that differ from normal Q35 tables when running the CXL test.

Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Message-Id: <20220429144110.25167-39-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/data/acpi/q35/CEDT.cxl                | Bin 0 -> 184 bytes
 tests/data/acpi/q35/DSDT.cxl                | Bin 0 -> 9615 bytes
 tests/qtest/bios-tables-test-allowed-diff.h |   2 --
 3 files changed, 2 deletions(-)

diff --git a/tests/data/acpi/q35/CEDT.cxl b/tests/data/acpi/q35/CEDT.cxl
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..b8fa06b00e65712e91e0a5ea0d9277e0146d1c00 100644
GIT binary patch
literal 184
zcmZ>EbqU$Qz`(%x(aGQ0BUr&HBEVSz2pEB4AU23*U{GMV2P7eE5T6mshKVRJ@Sw=U
r)I#JL88kqeKtKSd14gp~1^Iy(qF)E31_T6{AT-z>kXmGQAh!SjnYIc6

literal 0
HcmV?d00001

diff --git a/tests/data/acpi/q35/DSDT.cxl b/tests/data/acpi/q35/DSDT.cxl
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..c1206defed0154e9024702bba88453b2790a306d 100644
GIT binary patch
literal 9615
zcmeHN&2JmW9i1g9X|-HQONzE^`9p-`^eaU|`6EeNq%pZmk+ejbCaE|94R9$bt!$^r
zB8h=GhEZ7o632(O3FDx*(k=t^*8T%U4YY^$W}qk<pg_<bddaax5g!t>?>Dm}&yW-l
z9}AR+<^E>ho8P?InSIL{dUdby)5jSzBDphev7XMoSas9*7>qGGr*EeeJI|V1UartG
z;*prqydLN0IONRKx4qnI!T9;6|B>&%@vd*Q1GaX@xwX~~-oD|lF#=s)3oMIHocwgF
zo@+I?U90MrGG?n-^6czA%QRcAIE$LCtXE@ZYqjLD)XGHbOx=y$yu@7Z++w#f*4a$V
zT28b4x8t8L96a^Wxi_+RpZn_%ZeFrt035@&iSOe99sH}lb=dDZA9OAl*ND!qEp}%=
z=<v&;%%LlRPbIf)QAAZO_AWKIpY>{4m$z}MTB&B-%p%(5*6N7`>)^X{jM>yV^!ZJ{
z-~XLBWzH4mlue;BZx*ZhhE!=l8>wn;6|0Rhvl+YhAkJdV>kh@UFXSs;x?1yE>D1G$
zdLzpMD)9p<YQN=Mj^DN4@A$0a$NT*r>rTJlX|oU8Hv7ka#(J!0&4{)otm$_qsV(;&
zuoz=#&DWl!)=+;px93asY>Rg>(l4MX)l%(j#PTiMS)O?+DuIM*Zl74rc>s%h6h-UN
zDw$@VwWnbC%x8vCFgDl*zK=wZt+{=)d}eirH8ZQROl#~2^-y#B*h;mrDC>@i`)z1g
z$C@e_Z${sYn&y!$Uh^^cOnHYh1~hte1m}MAew3L<9L{;X)^K-P6A$knuR34>Gt48*
zKo?aK5Bq4V>ed@Z{H|@8xHS~G=)2W44qm#sRnMQsEcl~s;l{-&aC4dbX<W!v$~7Z?
zPtYiEbCuO{x?1vHOgD|(io48Gk5}FN=}7Z2iu(OvlMulb;{u~PnqQ~l9!ouu1!okB
z>I&%Khgd)5jaR*TdAhq2PK|rd{OO<dlW&#ZpZl!5|MzY=pt@f9k>7o+bCGrTcP>~F
z%z$fr9N8GQeb!4vjq7w^x97ThIv1^pAUPIcQ>-2MH`<K#+Ge%nF0qu|!nYA@Qsn&P
zC4&Zw<9@~`a<$f|CYN8|O2%UtoouDDoM+<P!1)qO+Ei{u5ENt(MNnkPsTkUUnIv#w
zL@WTO#Wl_YREESv{Tv(NOkl%^m=YRcoCzprA|xhS!?97$lr4xdVoGRKFahOEgv69t
zFs}rb*AwGR35^M+Y&=3$=a{B*j58&4OfY5R5vn@JHJ#&}DWT)KPK2t?2~FpOrgK8q
ziBQ#PYC26#r>W~isOrQMNa^WGP3NSp6QQcp(sWvyPD|H`P}LdNbjCHEaa|`uRVSW>
z%Kfx8owlwMp{jFA(>bN-oYHk7RCP{kI;S<A)4EQCs?Hfr=ZvOvM%Rf@)#+$D9ZjdB
z>qMyPOlUe2n$Co-6QQazsp(8=I+MCigsRS2P3Nqpb5_@hP}P~zbfz?&DP1Q*Rp%Z}
z=N?Vx9$hCwRp(w!=Uz?cUR@_bRVUsgF#6<kXin2Pr|U$h>fER4+^6Z>r|U$h>fFzn
z+3-oYpEGme!*0J|x(`EQdLedRW6o>Ld7X(+WggI&2Q=mZorzFo9@LlzHReH`iBM%8
z(wK)d<{_PlP-PzGOzAR*Ia5C44-2Mza3dt9yn_o`&IK*!f}Rth$Z5hrvx<Qh>LsM+
zW@_LZi9-WLfV3irB9KX8paRD&$za5i?K6;6Kz9rjp_B##6<BGY3X}{~Kz9rjAzC91
zRA9r1a;gR@pgRVN5G~?Z7^uL8QK*qa$v_2EhN?~&sHm1RVW0|>3>2YsMKVx?QYQ>l
zU^x>8szAv=1(fF`14Srx!axO<Ghv_#lnhirIg<<&q0|Wj6<E%MfhtfkPyyvkGEju_
ze!@TnmNQ|X3X}{~Ksl2P6rt1!0~J`#gn=qhGEf2KOfpb}QYQ>lU^x>8szAv=1(Y+%
zKoLrvFi?TzOc<yFB?A>u&LjgxD0RX>1(q{mpbC@>R6seC3>2Z%2?G^a&V+#~P%=;f
z<xDbAgi<FARA4z12C6{GKn0XD$v_cGoiI>=<xCi;0wn_#P|hR+MJRQ`Kn0dFVW0|>
z3{*fllMEE0)CmI>Sk8ojDo`>|0p(0GP=rz^3{+q_69%e4$v_2^Gs!>^N}VuJf#pmX
zr~)Me6;RG314Srx!axO<Ghv_#lnhirIg<<&q0|Wj6<E%MfhtfkPyyvkGEjt4Ck#|z
zITHq|K*>M_lrzad5lWpfP=V!47^ngz0~JutBm+e#b;3XqDF%v2HBf}Afg)54RAItE
z6($*|!XyJ#m@rU<2?JG_WS|O@3{+vlKouqoRAG{VDoip^g$V;ym@rU<Nd~Gg$v_n*
z3>1;PyD(5hxMRFC$v_d}Hpq~evTtFah-BZwKoQBlB?Cn$`<4t8A(o2fTd+{pwLARB
zYL9-9-X5o~Z1ehepNi72R9e-b^$w$2JDY{$p3Tw0rGsZOti7Dg)A<q|3Unx%soBz5
ztOj%^uUcE1SY(;(S@G0~(pk4Q2|z8An6~2Yjk8I<5_E;#q%vy^$5L>0)0$y)hDOw|
z^s+L6cZ955^02X7LyJKsnq5!qwPxR&W|L948^iOP;Yp0ui_{EX2kKE1(3)&2(eg@l
zc8$)hEnYH1>ro5{x5neSR=rj?Zf=Hcp!8H8X3q^|$KuIX&Yn}U=XmxU+NWmErABAZ
zHMh&8T`Z+xxi8diMIIr&tE6{%dUqhbyGwdEOz#bdx3Hk~mDPJndXJ~~2GV=Gr1!$~
z`N8zDM@XMn(&u^l{6PBrF6r}O`lZ42V~>!2NlCxN(=QFAU)m-8QkcFln11{b(ifEU
z1)jbzkiM`>`a+n#IGBFo5z-fx^hKV&IFP=$OZp<x$LLWunBEcT<--H76`1Z~{Z=na
zjq@pwFPhh3b%WE`O6b+$U1O*^Q8u!=p?R6qtHTS(P<5hgWOYOH3`@W?&!jNCl?+uU
z%0^Z<G;dORb$C@7s!o)RtZrx??9r>k`^!*uqHJV!L*sg{UL9U)hN=@~BdZ%4r*nFB
zc+(lGPL%1@IR_rDXK&Bc%Q<YdU|YqGV$I(<^L?>9QMArKT=B2g@^v{%Elf^$)0A6X
zO&4E%BY5TV57V!`{Q9exUt53qb=EZ8>dJCBTBj_lV0>bKVjDJA_2EghbpDBL+0E<U
zpdicAZZ%&q+)|}(xV7>N1{T0GbXYQ2)OE`q7TJ@8jJoywN*Zu4+el-rxl}2c0i!~U
z`s3%h9yCZaaw_XqOPS1KhMFNZj>b|6x3Tn6q-%9H**k6~lev&8j$`#cJK22f{8KTx
zwLwBj04=`{79&}}{O){b@B&E)tQjo34_#SV?)j<m7|D9HM8;rMs9|LNsinPc-L06r
z$S~bX{#-KB57>HOY~1mZoed*k8-d{mtbJ$2{#nO2Zmpxp57q}$a>0XzxGCMaTZ5&z
zE$bh5Cp&Gck2D_#qpo+44)q+}_h9`7wXw}Ex6!KG&vZ`?!T89)?OV5^!BEHGB6yeA
zX=5=T6FZAk;Tl_~TMljO&Bf8;hU!9lF0YO5=JKR_rrxl3>E_X+WvG61UT|SV-vm}<
zu#}(|2Mf{7BU{&e`&k;(nE&`-dSG0eJsRDZ0p)BX5w}c+)dqSGO-*Cdv=JvUZ1cj!
z)B#MMLN(vYXO6LO#?wTiG3A_z(Ir0d!#S0Cnx(!2>>{I%*x3;jJ61|T)vfTBY6xff
z=y~yQ$sKi9JXDP5i1n%2%H=Bbg{?$6b)ObfH)%)s1~47)LbkK_3v_+8T4ko&sI4$M
zclo)CO<~ffYzkvnbmXqED2@N+Uz->HaI^f|=RTZwp8xX~yY3Y)e8s(D><%BJrtwW?
zYMt0U?s~CZ6WZXMKw{c#K1sY2q=!yUq5_w;olhEc=v}%!t!+Z5PEEo*8!Sjhw)lLm
zOx$W)TftjxE5g=-tFN`!@C%5ocb(2UK$Bu;%3~W;VC)oRQIP1YTalfTTv!s_DRJ@4
zxOQdDav1I4-Pm9(xY|bDH#Q6wY~1i^`u2SBlH!e9zP1DYWxxOYmwarpQ%t>*e$el?
z9*ny3OI+!NS0e1j6pLULiG9lcj$;%qr;nu!-co2R*<vi=>k-D1{r|CqKQ#Q0jHOc;
zOFtetmVAtuUyK|}w{hjp&{(n=)(FK|Ix3z@hujxe@aGI==sd&lK1q+!p5d&_i`NXk
zqs{;5QF7ujLk>;xv)pC{<Gy|0HcXToc%#DSbBx^k_D(#7`NP=1`843y=|_FPPe1DE
z6VWCJGqDN(YjK7BN}W3FF0OnLUU^QvvV|^M2<3LQ+Pdh7Zg|Deu6&Fuyj`fUY217Q
ze_sjP#xtCv0D$LZE5p2GDwvG;$4(4JA^x$${|%FcQHa+>Dp%eQ#h*Ch;z7aR57Xo6
zafAQs<8-WAtk%R0puf~StOR{N;$3sNuDkYK+t`Pv!#B?(ef@YVIUY0MdN@DPN}4e%
zf(Ii-C+P}_aK88Ot~R%yTsr59-vCo*^W{}o>M=s&k1cA8oiS&O-e{du6Wq_7;y4Y8
z=61ZE$%y~Ypi910&payv+<$`)q(zV64;-lQm^?X7Cr!MBFNQ>5Bck9TIm%K<P-(FL
E0+(|YhyVZp

literal 0
HcmV?d00001

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index 7c7f9fbc44..dfb8523c8b 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1,3 +1 @@
 /* List of comma-separated changed AML files to ignore */
-"tests/data/acpi/q35/DSDT.cxl",
-"tests/data/acpi/q35/CEDT.cxl",

From 3b503d56a1880aecb94399054ae5137051de935a Mon Sep 17 00:00:00 2001
From: Ben Widawsky <ben.widawsky@intel.com>
Date: Fri, 29 Apr 2022 15:41:04 +0100
Subject: [PATCH 308/935] qtest/cxl: Add more complex test cases with CFMWs

Add CXL Fixed Memory Windows to the CXL tests.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Co-developed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Message-Id: <20220429144110.25167-40-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/cxl-test.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/tests/qtest/cxl-test.c b/tests/qtest/cxl-test.c
index 5f0794e816..079011af6a 100644
--- a/tests/qtest/cxl-test.c
+++ b/tests/qtest/cxl-test.c
@@ -9,11 +9,13 @@
 #include "libqtest-single.h"
 
 #define QEMU_PXB_CMD "-machine q35,cxl=on " \
-                     "-device pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52 "
+                     "-device pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52 "  \
+                     "-cxl-fixed-memory-window targets.0=cxl.0,size=4G "
 
-#define QEMU_2PXB_CMD "-machine q35,cxl=on " \
+#define QEMU_2PXB_CMD "-machine q35,cxl=on "                            \
                       "-device pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52 "  \
-                      "-device pxb-cxl,id=cxl.1,bus=pcie.0,bus_nr=53 "
+                      "-device pxb-cxl,id=cxl.1,bus=pcie.0,bus_nr=53 " \
+                      "-cxl-fixed-memory-window targets.0=cxl.0,targets.1=cxl.1,size=4G "
 
 #define QEMU_RP "-device cxl-rp,id=rp0,bus=cxl.0,chassis=0,slot=0 "
 

From aa69abe6e8b0f2dbc3022359a8947fa37bf988b9 Mon Sep 17 00:00:00 2001
From: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Date: Fri, 29 Apr 2022 15:41:07 +0100
Subject: [PATCH 309/935] docs/cxl: Add initial Compute eXpress Link (CXL)
 documentation.

Provide an introduction to the main components of a CXL system,
with detailed explanation of memory interleaving, example command
lines and kernel configuration.

This was a challenging document to write due to the need to extract
only that subset of CXL information which is relevant to either
users of QEMU emulation of CXL or to those interested in the
implementation.  Much of CXL is concerned with specific elements of
the protocol, management of memory pooling etc which is simply
not relevant to what is currently planned for CXL emulation
in QEMU.  All comments welcome

Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Message-Id: <20220429144110.25167-43-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 docs/system/device-emulation.rst |   1 +
 docs/system/devices/cxl.rst      | 302 +++++++++++++++++++++++++++++++
 2 files changed, 303 insertions(+)
 create mode 100644 docs/system/devices/cxl.rst

diff --git a/docs/system/device-emulation.rst b/docs/system/device-emulation.rst
index ae8dd233e8..3b729b920d 100644
--- a/docs/system/device-emulation.rst
+++ b/docs/system/device-emulation.rst
@@ -84,6 +84,7 @@ Emulated Devices
 
    devices/can.rst
    devices/ccid.rst
+   devices/cxl.rst
    devices/ivshmem.rst
    devices/net.rst
    devices/nvme.rst
diff --git a/docs/system/devices/cxl.rst b/docs/system/devices/cxl.rst
new file mode 100644
index 0000000000..9293cbf01a
--- /dev/null
+++ b/docs/system/devices/cxl.rst
@@ -0,0 +1,302 @@
+Compute Express Link (CXL)
+==========================
+From the view of a single host, CXL is an interconnect standard that
+targets accelerators and memory devices attached to a CXL host.
+This description will focus on those aspects visible either to
+software running on a QEMU emulated host or to the internals of
+functional emulation. As such, it will skip over many of the
+electrical and protocol elements that would be more of interest
+for real hardware and will dominate more general introductions to CXL.
+It will also completely ignore the fabric management aspects of CXL
+by considering only a single host and a static configuration.
+
+CXL shares many concepts and much of the infrastructure of PCI Express,
+with CXL Host Bridges, which have CXL Root Ports which may be directly
+attached to CXL or PCI End Points. Alternatively there may be CXL Switches
+with CXL and PCI Endpoints attached below them.  In many cases additional
+control and capabilities are exposed via PCI Express interfaces.
+This sharing of interfaces and hence emulation code is is reflected
+in how the devices are emulated in QEMU. In most cases the various
+CXL elements are built upon an equivalent PCIe devices.
+
+CXL devices support the following interfaces:
+
+* Most conventional PCIe interfaces
+
+  - Configuration space access
+  - BAR mapped memory accesses used for registers and mailboxes.
+  - MSI/MSI-X
+  - AER
+  - DOE mailboxes
+  - IDE
+  - Many other PCI express defined interfaces..
+
+* Memory operations
+
+  - Equivalent of accessing DRAM / NVDIMMs. Any access / feature
+    supported by the host for normal memory should also work for
+    CXL attached memory devices.
+
+* Cache operations. The are mostly irrelevant to QEMU emulation as
+  QEMU is not emulating a coherency protocol. Any emulation related
+  to these will be device specific and is out of the scope of this
+  document.
+
+CXL 2.0 Device Types
+--------------------
+CXL 2.0 End Points are often categorized into three types.
+
+**Type 1:** These support coherent caching of host memory.  Example might
+be a crypto accelerators.  May also have device private memory accessible
+via means such as PCI memory reads and writes to BARs.
+
+**Type 2:** These support coherent caching of host memory and host
+managed device memory (HDM) for which the coherency protocol is managed
+by the host. This is a complex topic, so for more information on CXL
+coherency see the CXL 2.0 specification.
+
+**Type 3 Memory devices:**  These devices act as a means of attaching
+additional memory (HDM) to a CXL host including both volatile and
+persistent memory. The CXL topology may support interleaving across a
+number of Type 3 memory devices using HDM Decoders in the host, host
+bridge, switch upstream port and endpoints.
+
+Scope of CXL emulation in QEMU
+------------------------------
+The focus of CXL emulation is CXL revision 2.0 and later. Earlier CXL
+revisions defined a smaller set of features, leaving much of the control
+interface as implementation defined or device specific, making generic
+emulation challenging with host specific firmware being responsible
+for setup and the Endpoints being presented to operating systems
+as Root Complex Integrated End Points. CXL rev 2.0 looks a lot
+more like PCI Express, with fully specified discoverability
+of the CXL topology.
+
+CXL System components
+----------------------
+A CXL system is made up a Host with a number of 'standard components'
+the control and capabilities of which are discoverable by system software
+using means described in the CXL 2.0 specification.
+
+CXL Fixed Memory Windows (CFMW)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+A CFMW consists of a particular range of Host Physical Address space
+which is routed to particular CXL Host Bridges.  At time of generic
+software initialization it will have a particularly interleaving
+configuration and associated Quality of Serice Throtling Group (QTG).
+This information is available to system software, when making
+decisions about how to configure interleave across available CXL
+memory devices.  It is provide as CFMW Structures (CFMWS) in
+the CXL Early Discovery Table, an ACPI table.
+
+Note: QTG 0 is the only one currently supported in QEMU.
+
+CXL Host Bridge (CXL HB)
+~~~~~~~~~~~~~~~~~~~~~~~~
+A CXL host bridge is similar to the PCIe equivalent, but with a
+specification defined register interface called CXL Host Bridge
+Component Registers (CHBCR). The location of this CHBCR MMIO
+space is described to system software via a CXL Host Bridge
+Structure (CHBS) in the CEDT ACPI table.  The actual interfaces
+are identical to those used for other parts of the CXL heirarchy
+as CXL Component Registers in PCI BARs.
+
+Interfaces provided include:
+
+* Configuration of HDM Decoders to route CXL Memory accesses with
+  a particularly Host Physical Address range to the target port
+  below which the CXL device servicing that address lies.  This
+  may be a mapping to a single Root Port (RP) or across a set of
+  target RPs.
+
+CXL Root Ports (CXL RP)
+~~~~~~~~~~~~~~~~~~~~~~~
+A CXL Root Port servers te same purpose as a PCIe Root Port.
+There are a number of CXL specific Designated Vendor Specific
+Extended Capabilities (DVSEC) in PCIe Configuration Space
+and associated component register access via PCI bars.
+
+CXL Switch
+~~~~~~~~~~
+Not yet implemented in QEMU.
+
+Here we consider a simple CXL switch with only a single
+virtual hierarchy. Whilst more complex devices exist, their
+visibility to a particular host is generally the same as for
+a simple switch design. Hosts often have no awareness
+of complex rerouting and device pooling, they simply see
+devices being hot added or hot removed.
+
+A CXL switch has a similar architecture to those in PCIe,
+with a single upstream port, internal PCI bus and multiple
+downstream ports.
+
+Both the CXL upstream and downstream ports have CXL specific
+DVSECs in configuration space, and component registers in PCI
+BARs.  The Upstream Port has the configuration interfaces for
+the HDM decoders which route incoming memory accesses to the
+appropriate downstream port.
+
+CXL Memory Devices - Type 3
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+CXL type 3 devices use a PCI class code and are intended to be supported
+by a generic operating system driver. They have HDM decoders
+though in these EP devices, the decoder is reponsible not for
+routing but for translation of the incoming host physical address (HPA)
+into a Device Physical Address (DPA).
+
+CXL Memory Interleave
+---------------------
+To understand the interaction of different CXL hardware components which
+are emulated in QEMU, let us consider a memory read in a fully configured
+CXL topology.  Note that system software is responsible for configuration
+of all components with the exception of the CFMWs. System software is
+responsible for allocating appropriate ranges from within the CFMWs
+and exposing those via normal memory configurations as would be done
+for system RAM.
+
+Example system Topology. x marks the match in each decoder level::
+
+  |<------------------SYSTEM PHYSICAL ADDRESS MAP (1)----------------->|
+  |    __________   __________________________________   __________    |
+  |   |          | |                                  | |          |   |
+  |   | CFMW 0   | |  CXL Fixed Memory Window 1       | | CFMW 1   |   |
+  |   | HB0 only | |  Configured to interleave memory | | HB1 only |   |
+  |   |          | |  memory accesses across HB0/HB1  | |          |   |
+  |   |__________| |_____x____________________________| |__________|   |
+           |             |                     |             |
+           |             |                     |             |
+           |             |                     |             |
+           |       Interleave Decoder          |             |
+           |       Matches this HB             |             |
+           \_____________|                     |_____________/
+               __________|__________      _____|_______________
+              |                     |    |                     |
+       (2)    | CXL HB 0            |    | CXL HB 1            |
+              | HB IntLv Decoders   |    | HB IntLv Decoders   |
+              | PCI/CXL Root Bus 0c |    | PCI/CXL Root Bus 0d |
+              |                     |    |                     |
+              |___x_________________|    |_____________________|
+                  |                |       |               |
+                  |                |       |               |
+       A HB 0 HDM Decoder          |       |               |
+       matches this Port           |       |               |
+                  |                |       |               |
+       ___________|___   __________|__   __|_________   ___|_________
+   (3)|  Root Port 0  | | Root Port 1 | | Root Port 2| | Root Port 3 |
+      |  Appears in   | | Appears in  | | Appears in | | Appear in   |
+      |  PCI topology | | PCI Topology| | PCI Topo   | | PCI Topo    |
+      |  As 0c:00.0   | | as 0c:01.0  | | as de:00.0 | | as de:01.0  |
+      |_______________| |_____________| |____________| |_____________|
+            |                  |               |              |
+            |                  |               |              |
+       _____|_________   ______|______   ______|_____   ______|_______
+   (4)|     x         | |             | |            | |              |
+      | CXL Type3 0   | | CXL Type3 1 | | CXL type3 2| | CLX Type 3 3 |
+      |               | |             | |            | |              |
+      | PMEM0(Vol LSA)| | PMEM1 (...) | | PMEM2 (...)| | PMEM3 (...)  |
+      | Decoder to go | |             | |            | |              |
+      | from host PA  | | PCI 0e:00.0 | | PCI df:00.0| | PCI e0:00.0  |
+      | to device PA  | |             | |            | |              |
+      | PCI as 0d:00.0| |             | |            | |              |
+      |_______________| |_____________| |____________| |______________|
+
+Notes:
+
+(1) **3 CXL Fixed Memory Windows (CFMW)** corresponding to different
+    ranges of the system physical address map.  Each CFMW has
+    particular interleave setup across the CXL Host Bridges (HB)
+    CFMW0 provides uninterleaved access to HB0, CFW2 provides
+    uninterleaved acess to HB1. CFW1 provides interleaved memory access
+    across HB0 and HB1.
+
+(2) **Two CXL Host Bridges**. Each of these has 2 CXL Root Ports and
+    programmable HDM decoders to route memory accesses either to
+    a single port or interleave them across multiple ports.
+    A complex configuration here, might be to use the following HDM
+    decoders in HB0. HDM0 routes CFMW0 requests to RP0 and hence
+    part of CXL Type3 0. HDM1 routes CFMW0 requests from a
+    different region of the CFMW0 PA range to RP2 and hence part
+    of CXL Type 3 1.  HDM2 routes yet another PA range from within
+    CFMW0 to be interleaved across RP0 and RP1, providing 2 way
+    interleave of part of the memory provided by CXL Type3 0 and
+    CXL Type 3 1. HDM3 routes those interleaved accesses from
+    CFMW1 that target HB0 to RP 0 and another part of the memory of
+    CXL Type 3 0 (as part of a 2 way interleave at the system level
+    across for example CXL Type3 0 and CXL Type3 2.
+    HDM4 is used to enable system wide 4 way interleave across all
+    the present CXL type3 devices, by interleaving those (interleaved)
+    requests that HB0 receives from from CFMW1 across RP 0 and
+    RP 1 and hence to yet more regions of the memory of the
+    attached Type3 devices.  Note this is a representative subset
+    of the full range of possible HDM decoder configurations in this
+    topology.
+
+(3) **Four CXL Root Ports.** In this case the CXL Type 3 devices are
+    directly attached to these ports.
+
+(4) **Four CXL Type3 memory expansion devices.**  These will each have
+    HDM decoders, but in this case rather than performing interleave
+    they will take the Host Physical Addresses of accesses and map
+    them to their own local Device Physical Address Space (DPA).
+
+Example command lines
+---------------------
+A very simple setup with just one directly attached CXL Type 3 device::
+
+  qemu-system-aarch64 -M virt,gic-version=3,cxl=on -m 4g,maxmem=8G,slots=8 -cpu max \
+  ...
+  -object memory-backend-file,id=cxl-mem1,share=on,mem-path=/tmp/cxltest.raw,size=256M \
+  -object memory-backend-file,id=cxl-lsa1,share=on,mem-path=/tmp/lsa.raw,size=256M \
+  -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
+  -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
+  -device cxl-type3,bus=root_port13,memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem0 \
+  -cxl-fixed-memory-window targets.0=cxl.1,size=4G
+
+A setup suitable for 4 way interleave. Only one fixed window provided, to enable 2 way
+interleave across 2 CXL host bridges.  Each host bridge has 2 CXL Root Ports, with
+the CXL Type3 device directly attached (no switches).::
+
+  qemu-system-aarch64 -M virt,gic-version=3,cxl=on -m 4g,maxmem=8G,slots=8 -cpu max \
+  ...
+  -object memory-backend-file,id=cxl-mem1,share=on,mem-path=/tmp/cxltest.raw,size=256M \
+  -object memory-backend-file,id=cxl-mem2,share=on,mem-path=/tmp/cxltest2.raw,size=256M \
+  -object memory-backend-file,id=cxl-mem3,share=on,mem-path=/tmp/cxltest3.raw,size=256M \
+  -object memory-backend-file,id=cxl-mem4,share=on,mem-path=/tmp/cxltest4.raw,size=256M \
+  -object memory-backend-file,id=cxl-lsa1,share=on,mem-path=/tmp/lsa.raw,size=256M \
+  -object memory-backend-file,id=cxl-lsa2,share=on,mem-path=/tmp/lsa2.raw,size=256M \
+  -object memory-backend-file,id=cxl-lsa3,share=on,mem-path=/tmp/lsa3.raw,size=256M \
+  -object memory-backend-file,id=cxl-lsa4,share=on,mem-path=/tmp/lsa4.raw,size=256M \
+  -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
+  -device pxb-cxl,bus_nr=222,bus=pcie.0,id=cxl.2 \
+  -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
+  -device cxl-type3,bus=root_port13,memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem0 \
+  -device cxl-rp,port=1,bus=cxl.1,id=root_port14,chassis=0,slot=3 \
+  -device cxl-type3,bus=root_port14,memdev=cxl-mem2,lsa=cxl-lsa2,id=cxl-pmem1 \
+  -device cxl-rp,port=0,bus=cxl.2,id=root_port15,chassis=0,slot=5 \
+  -device cxl-type3,bus=root_port15,memdev=cxl-mem3,lsa=cxl-lsa3,id=cxl-pmem2 \
+  -device cxl-rp,port=1,bus=cxl.2,id=root_port16,chassis=0,slot=6 \
+  -device cxl-type3,bus=root_port16,memdev=cxl-mem4,lsa=cxl-lsa4,id=cxl-pmem3 \
+  -cxl-fixed-memory-window targets.0=cxl.1,targets.1=cxl.2,size=4G,interleave-granularity=8k
+
+Kernel Configuration Options
+----------------------------
+
+In Linux 5.18 the followings options are necessary to make use of
+OS management of CXL memory devices as described here.
+
+* CONFIG_CXL_BUS
+* CONFIG_CXL_PCI
+* CONFIG_CXL_ACPI
+* CONFIG_CXL_PMEM
+* CONFIG_CXL_MEM
+* CONFIG_CXL_PORT
+* CONFIG_CXL_REGION
+
+References
+----------
+
+ - Consortium website for specifications etc:
+   http://www.computeexpresslink.org
+ - Compute Express link Revision 2 specification, October 2020
+ - CEDT CFMWS & QTG _DSM ECN May 2021

From 495fe3a78749c39c0e772c4e1a55d6cb8a7e5292 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= <eperezma@redhat.com>
Date: Thu, 12 May 2022 19:57:42 +0200
Subject: [PATCH 310/935] vhost: Track descriptor chain in private at SVQ
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The device could have access to modify them, and it definitely have
access when we implement packed vq. Harden SVQ maintaining a private
copy of the descriptor chain. Other fields like buffer addresses are
already maintained sepparatedly.

Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
Message-Id: <20220512175747.142058-2-eperezma@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/vhost-shadow-virtqueue.c | 12 +++++++-----
 hw/virtio/vhost-shadow-virtqueue.h |  6 ++++++
 2 files changed, 13 insertions(+), 5 deletions(-)

diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
index 1e5cfe2af6..1d6552b0fe 100644
--- a/hw/virtio/vhost-shadow-virtqueue.c
+++ b/hw/virtio/vhost-shadow-virtqueue.c
@@ -138,6 +138,7 @@ static void vhost_vring_write_descs(VhostShadowVirtqueue *svq, hwaddr *sg,
     for (n = 0; n < num; n++) {
         if (more_descs || (n + 1 < num)) {
             descs[i].flags = flags | cpu_to_le16(VRING_DESC_F_NEXT);
+            descs[i].next = cpu_to_le16(svq->desc_next[i]);
         } else {
             descs[i].flags = flags;
         }
@@ -145,10 +146,10 @@ static void vhost_vring_write_descs(VhostShadowVirtqueue *svq, hwaddr *sg,
         descs[i].len = cpu_to_le32(iovec[n].iov_len);
 
         last = i;
-        i = cpu_to_le16(descs[i].next);
+        i = cpu_to_le16(svq->desc_next[i]);
     }
 
-    svq->free_head = le16_to_cpu(descs[last].next);
+    svq->free_head = le16_to_cpu(svq->desc_next[last]);
 }
 
 static bool vhost_svq_add_split(VhostShadowVirtqueue *svq,
@@ -336,7 +337,6 @@ static void vhost_svq_disable_notification(VhostShadowVirtqueue *svq)
 static VirtQueueElement *vhost_svq_get_buf(VhostShadowVirtqueue *svq,
                                            uint32_t *len)
 {
-    vring_desc_t *descs = svq->vring.desc;
     const vring_used_t *used = svq->vring.used;
     vring_used_elem_t used_elem;
     uint16_t last_used;
@@ -365,7 +365,7 @@ static VirtQueueElement *vhost_svq_get_buf(VhostShadowVirtqueue *svq,
         return NULL;
     }
 
-    descs[used_elem.id].next = svq->free_head;
+    svq->desc_next[used_elem.id] = svq->free_head;
     svq->free_head = used_elem.id;
 
     *len = used_elem.len;
@@ -540,8 +540,9 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev,
     svq->vring.used = qemu_memalign(qemu_real_host_page_size(), device_size);
     memset(svq->vring.used, 0, device_size);
     svq->ring_id_maps = g_new0(VirtQueueElement *, svq->vring.num);
+    svq->desc_next = g_new0(uint16_t, svq->vring.num);
     for (unsigned i = 0; i < svq->vring.num - 1; i++) {
-        svq->vring.desc[i].next = cpu_to_le16(i + 1);
+        svq->desc_next[i] = cpu_to_le16(i + 1);
     }
 }
 
@@ -574,6 +575,7 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq)
         virtqueue_detach_element(svq->vq, next_avail_elem, 0);
     }
     svq->vq = NULL;
+    g_free(svq->desc_next);
     g_free(svq->ring_id_maps);
     qemu_vfree(svq->vring.desc);
     qemu_vfree(svq->vring.used);
diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h
index e5e24c536d..c132c994e9 100644
--- a/hw/virtio/vhost-shadow-virtqueue.h
+++ b/hw/virtio/vhost-shadow-virtqueue.h
@@ -53,6 +53,12 @@ typedef struct VhostShadowVirtqueue {
     /* Next VirtQueue element that guest made available */
     VirtQueueElement *next_guest_avail_elem;
 
+    /*
+     * Backup next field for each descriptor so we can recover securely, not
+     * needing to trust the device access.
+     */
+    uint16_t *desc_next;
+
     /* Next head to expose to the device */
     uint16_t shadow_avail_idx;
 

From 81abfa5724c9a6502d7a1d3a67c55f2a303a1170 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= <eperezma@redhat.com>
Date: Thu, 12 May 2022 19:57:43 +0200
Subject: [PATCH 311/935] vhost: Fix device's used descriptor dequeue
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Only the first one of them were properly enqueued back.

Fixes: 100890f7ca ("vhost: Shadow virtqueue buffers forwarding")

Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
Message-Id: <20220512175747.142058-3-eperezma@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/vhost-shadow-virtqueue.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
index 1d6552b0fe..a8376ef82b 100644
--- a/hw/virtio/vhost-shadow-virtqueue.c
+++ b/hw/virtio/vhost-shadow-virtqueue.c
@@ -334,12 +334,22 @@ static void vhost_svq_disable_notification(VhostShadowVirtqueue *svq)
     svq->vring.avail->flags |= cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT);
 }
 
+static uint16_t vhost_svq_last_desc_of_chain(const VhostShadowVirtqueue *svq,
+                                             uint16_t num, uint16_t i)
+{
+    for (uint16_t j = 0; j < (num - 1); ++j) {
+        i = le16_to_cpu(svq->desc_next[i]);
+    }
+
+    return i;
+}
+
 static VirtQueueElement *vhost_svq_get_buf(VhostShadowVirtqueue *svq,
                                            uint32_t *len)
 {
     const vring_used_t *used = svq->vring.used;
     vring_used_elem_t used_elem;
-    uint16_t last_used;
+    uint16_t last_used, last_used_chain, num;
 
     if (!vhost_svq_more_used(svq)) {
         return NULL;
@@ -365,7 +375,10 @@ static VirtQueueElement *vhost_svq_get_buf(VhostShadowVirtqueue *svq,
         return NULL;
     }
 
-    svq->desc_next[used_elem.id] = svq->free_head;
+    num = svq->ring_id_maps[used_elem.id]->in_num +
+          svq->ring_id_maps[used_elem.id]->out_num;
+    last_used_chain = vhost_svq_last_desc_of_chain(svq, num, used_elem.id);
+    svq->desc_next[last_used_chain] = svq->free_head;
     svq->free_head = used_elem.id;
 
     *len = used_elem.len;

From 639036477ef890958415967e753ca2cbb348c16c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= <eperezma@redhat.com>
Date: Thu, 12 May 2022 19:57:44 +0200
Subject: [PATCH 312/935] vdpa: Fix bad index calculus at
 vhost_vdpa_get_vring_base
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Fixes: 6d0b222666 ("vdpa: Adapt vhost_vdpa_get_vring_base to SVQ")

Acked-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
Message-Id: <20220512175747.142058-4-eperezma@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/vhost-vdpa.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c
index a30510ed17..493269b0b5 100644
--- a/hw/virtio/vhost-vdpa.c
+++ b/hw/virtio/vhost-vdpa.c
@@ -1172,11 +1172,11 @@ static int vhost_vdpa_get_vring_base(struct vhost_dev *dev,
                                        struct vhost_vring_state *ring)
 {
     struct vhost_vdpa *v = dev->opaque;
+    int vdpa_idx = ring->index - dev->vq_index;
     int ret;
 
     if (v->shadow_vqs_enabled) {
-        VhostShadowVirtqueue *svq = g_ptr_array_index(v->shadow_vqs,
-                                                      ring->index);
+        VhostShadowVirtqueue *svq = g_ptr_array_index(v->shadow_vqs, vdpa_idx);
 
         /*
          * Setting base as last used idx, so destination will see as available

From 1c82fdfef8a227518ffecae9d419bcada995c202 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= <eperezma@redhat.com>
Date: Thu, 12 May 2022 19:57:45 +0200
Subject: [PATCH 313/935] vdpa: Fix index calculus at vhost_vdpa_svqs_start
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

With the introduction of MQ the index of the vq needs to be calculated
with the device model vq_index.

Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20220512175747.142058-5-eperezma@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/vhost-vdpa.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c
index 493269b0b5..ed106bff47 100644
--- a/hw/virtio/vhost-vdpa.c
+++ b/hw/virtio/vhost-vdpa.c
@@ -1018,7 +1018,7 @@ static bool vhost_vdpa_svqs_start(struct vhost_dev *dev)
         VirtQueue *vq = virtio_get_queue(dev->vdev, dev->vq_index + i);
         VhostShadowVirtqueue *svq = g_ptr_array_index(v->shadow_vqs, i);
         struct vhost_vring_addr addr = {
-            .index = i,
+            .index = dev->vq_index + i,
         };
         int r;
         bool ok = vhost_vdpa_svq_setup(dev, svq, i, &err);

From d792199de55ca5cb5334016884039c740290b5c7 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Date: Thu, 12 May 2022 19:57:46 +0200
Subject: [PATCH 314/935] hw/virtio: Replace g_memdup() by g_memdup2()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Per https://discourse.gnome.org/t/port-your-module-from-g-memdup-to-g-memdup2-now/5538

  The old API took the size of the memory to duplicate as a guint,
  whereas most memory functions take memory sizes as a gsize. This
  made it easy to accidentally pass a gsize to g_memdup(). For large
  values, that would lead to a silent truncation of the size from 64
  to 32 bits, and result in a heap area being returned which is
  significantly smaller than what the caller expects. This can likely
  be exploited in various modules to cause a heap buffer overflow.

Replace g_memdup() by the safer g_memdup2() wrapper.

Acked-by: Jason Wang <jasowang@redhat.com>
Acked-by: Eugenio Pérez <eperezma@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20220512175747.142058-6-eperezma@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/net/virtio-net.c       | 3 ++-
 hw/virtio/virtio-crypto.c | 6 +++---
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 1067e72b39..e4748a7e6c 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -1443,7 +1443,8 @@ static void virtio_net_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq)
         }
 
         iov_cnt = elem->out_num;
-        iov2 = iov = g_memdup(elem->out_sg, sizeof(struct iovec) * elem->out_num);
+        iov2 = iov = g_memdup2(elem->out_sg,
+                               sizeof(struct iovec) * elem->out_num);
         s = iov_to_buf(iov, iov_cnt, 0, &ctrl, sizeof(ctrl));
         iov_discard_front(&iov, &iov_cnt, sizeof(ctrl));
         if (s != sizeof(ctrl)) {
diff --git a/hw/virtio/virtio-crypto.c b/hw/virtio/virtio-crypto.c
index dcd80b904d..0e31e3cc04 100644
--- a/hw/virtio/virtio-crypto.c
+++ b/hw/virtio/virtio-crypto.c
@@ -242,7 +242,7 @@ static void virtio_crypto_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq)
         }
 
         out_num = elem->out_num;
-        out_iov_copy = g_memdup(elem->out_sg, sizeof(out_iov[0]) * out_num);
+        out_iov_copy = g_memdup2(elem->out_sg, sizeof(out_iov[0]) * out_num);
         out_iov = out_iov_copy;
 
         in_num = elem->in_num;
@@ -605,11 +605,11 @@ virtio_crypto_handle_request(VirtIOCryptoReq *request)
     }
 
     out_num = elem->out_num;
-    out_iov_copy = g_memdup(elem->out_sg, sizeof(out_iov[0]) * out_num);
+    out_iov_copy = g_memdup2(elem->out_sg, sizeof(out_iov[0]) * out_num);
     out_iov = out_iov_copy;
 
     in_num = elem->in_num;
-    in_iov_copy = g_memdup(elem->in_sg, sizeof(in_iov[0]) * in_num);
+    in_iov_copy = g_memdup2(elem->in_sg, sizeof(in_iov[0]) * in_num);
     in_iov = in_iov_copy;
 
     if (unlikely(iov_to_buf(out_iov, out_num, 0, &req, sizeof(req))

From 5181db132b587754dda3a520eec923b87a65bbb7 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= <eperezma@redhat.com>
Date: Thu, 12 May 2022 19:57:47 +0200
Subject: [PATCH 315/935] vhost: Fix element in vhost_svq_add failure
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Coverity rightly reports that is not free in that case.

Fixes: Coverity CID 1487559
Fixes: 100890f7ca ("vhost: Shadow virtqueue buffers forwarding")

Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
Message-Id: <20220512175747.142058-7-eperezma@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/vhost-shadow-virtqueue.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
index a8376ef82b..56c96ebd13 100644
--- a/hw/virtio/vhost-shadow-virtqueue.c
+++ b/hw/virtio/vhost-shadow-virtqueue.c
@@ -199,11 +199,19 @@ static bool vhost_svq_add_split(VhostShadowVirtqueue *svq,
     return true;
 }
 
+/**
+ * Add an element to a SVQ.
+ *
+ * The caller must check that there is enough slots for the new element. It
+ * takes ownership of the element: In case of failure, it is free and the SVQ
+ * is considered broken.
+ */
 static bool vhost_svq_add(VhostShadowVirtqueue *svq, VirtQueueElement *elem)
 {
     unsigned qemu_head;
     bool ok = vhost_svq_add_split(svq, elem, &qemu_head);
     if (unlikely(!ok)) {
+        g_free(elem);
         return false;
     }
 

From e4082063e47e9731dbeb1c26174c17f6038f577f Mon Sep 17 00:00:00 2001
From: Alex Williamson <alex.williamson@redhat.com>
Date: Fri, 13 May 2022 08:20:08 -0600
Subject: [PATCH 316/935] linux-headers: Update to v5.18-rc6

Update to c5eb0a61238d ("Linux 5.18-rc6").  Mechanical search and
replace of vfio defines with white space massaging.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/common.c                              |   6 +-
 hw/vfio/migration.c                           |  27 +-
 .../linux/input-event-codes.h                 |  25 +-
 .../standard-headers/linux/virtio_config.h    |   6 +
 .../standard-headers/linux/virtio_crypto.h    |  82 +++-
 linux-headers/asm-arm64/kvm.h                 |  16 +
 linux-headers/asm-generic/mman-common.h       |   2 +
 linux-headers/asm-mips/mman.h                 |   2 +
 linux-headers/linux/kvm.h                     |  27 +-
 linux-headers/linux/psci.h                    |   4 +
 linux-headers/linux/userfaultfd.h             |   8 +-
 linux-headers/linux/vfio.h                    | 406 +++++++++---------
 linux-headers/linux/vhost.h                   |   7 +
 13 files changed, 383 insertions(+), 235 deletions(-)

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index 159f910421..29982c7af8 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -355,7 +355,7 @@ static bool vfio_devices_all_dirty_tracking(VFIOContainer *container)
             }
 
             if ((vbasedev->pre_copy_dirty_page_tracking == ON_OFF_AUTO_OFF)
-                && (migration->device_state & VFIO_DEVICE_STATE_RUNNING)) {
+                && (migration->device_state & VFIO_DEVICE_STATE_V1_RUNNING)) {
                 return false;
             }
         }
@@ -381,8 +381,8 @@ static bool vfio_devices_all_running_and_saving(VFIOContainer *container)
                 return false;
             }
 
-            if ((migration->device_state & VFIO_DEVICE_STATE_SAVING) &&
-                (migration->device_state & VFIO_DEVICE_STATE_RUNNING)) {
+            if ((migration->device_state & VFIO_DEVICE_STATE_V1_SAVING) &&
+                (migration->device_state & VFIO_DEVICE_STATE_V1_RUNNING)) {
                 continue;
             } else {
                 return false;
diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c
index ff6b45de6b..a6ad1f8945 100644
--- a/hw/vfio/migration.c
+++ b/hw/vfio/migration.c
@@ -432,7 +432,7 @@ static int vfio_save_setup(QEMUFile *f, void *opaque)
     }
 
     ret = vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_MASK,
-                                   VFIO_DEVICE_STATE_SAVING);
+                                   VFIO_DEVICE_STATE_V1_SAVING);
     if (ret) {
         error_report("%s: Failed to set state SAVING", vbasedev->name);
         return ret;
@@ -531,8 +531,8 @@ static int vfio_save_complete_precopy(QEMUFile *f, void *opaque)
     uint64_t data_size;
     int ret;
 
-    ret = vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_RUNNING,
-                                   VFIO_DEVICE_STATE_SAVING);
+    ret = vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_RUNNING,
+                                   VFIO_DEVICE_STATE_V1_SAVING);
     if (ret) {
         error_report("%s: Failed to set state STOP and SAVING",
                      vbasedev->name);
@@ -569,7 +569,7 @@ static int vfio_save_complete_precopy(QEMUFile *f, void *opaque)
         return ret;
     }
 
-    ret = vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_SAVING, 0);
+    ret = vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_SAVING, 0);
     if (ret) {
         error_report("%s: Failed to set state STOPPED", vbasedev->name);
         return ret;
@@ -609,7 +609,7 @@ static int vfio_load_setup(QEMUFile *f, void *opaque)
     }
 
     ret = vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_MASK,
-                                   VFIO_DEVICE_STATE_RESUMING);
+                                   VFIO_DEVICE_STATE_V1_RESUMING);
     if (ret) {
         error_report("%s: Failed to set state RESUMING", vbasedev->name);
         if (migration->region.mmaps) {
@@ -717,20 +717,20 @@ static void vfio_vmstate_change(void *opaque, bool running, RunState state)
          * In both the above cases, set _RUNNING bit.
          */
         mask = ~VFIO_DEVICE_STATE_MASK;
-        value = VFIO_DEVICE_STATE_RUNNING;
+        value = VFIO_DEVICE_STATE_V1_RUNNING;
     } else {
         /*
          * Here device state could be either _RUNNING or _SAVING|_RUNNING. Reset
          * _RUNNING bit
          */
-        mask = ~VFIO_DEVICE_STATE_RUNNING;
+        mask = ~VFIO_DEVICE_STATE_V1_RUNNING;
 
         /*
          * When VM state transition to stop for savevm command, device should
          * start saving data.
          */
         if (state == RUN_STATE_SAVE_VM) {
-            value = VFIO_DEVICE_STATE_SAVING;
+            value = VFIO_DEVICE_STATE_V1_SAVING;
         } else {
             value = 0;
         }
@@ -768,8 +768,9 @@ static void vfio_migration_state_notifier(Notifier *notifier, void *data)
     case MIGRATION_STATUS_FAILED:
         bytes_transferred = 0;
         ret = vfio_migration_set_state(vbasedev,
-                      ~(VFIO_DEVICE_STATE_SAVING | VFIO_DEVICE_STATE_RESUMING),
-                      VFIO_DEVICE_STATE_RUNNING);
+                                       ~(VFIO_DEVICE_STATE_V1_SAVING |
+                                         VFIO_DEVICE_STATE_V1_RESUMING),
+                                       VFIO_DEVICE_STATE_V1_RUNNING);
         if (ret) {
             error_report("%s: Failed to set state RUNNING", vbasedev->name);
         }
@@ -864,8 +865,10 @@ int vfio_migration_probe(VFIODevice *vbasedev, Error **errp)
         goto add_blocker;
     }
 
-    ret = vfio_get_dev_region_info(vbasedev, VFIO_REGION_TYPE_MIGRATION,
-                                   VFIO_REGION_SUBTYPE_MIGRATION, &info);
+    ret = vfio_get_dev_region_info(vbasedev,
+                                   VFIO_REGION_TYPE_MIGRATION_DEPRECATED,
+                                   VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATED,
+                                   &info);
     if (ret) {
         goto add_blocker;
     }
diff --git a/include/standard-headers/linux/input-event-codes.h b/include/standard-headers/linux/input-event-codes.h
index b5e86b40ab..50790aee5a 100644
--- a/include/standard-headers/linux/input-event-codes.h
+++ b/include/standard-headers/linux/input-event-codes.h
@@ -278,7 +278,8 @@
 #define KEY_PAUSECD		201
 #define KEY_PROG3		202
 #define KEY_PROG4		203
-#define KEY_DASHBOARD		204	/* AL Dashboard */
+#define KEY_ALL_APPLICATIONS	204	/* AC Desktop Show All Applications */
+#define KEY_DASHBOARD		KEY_ALL_APPLICATIONS
 #define KEY_SUSPEND		205
 #define KEY_CLOSE		206	/* AC Close */
 #define KEY_PLAY		207
@@ -612,6 +613,7 @@
 #define KEY_ASSISTANT		0x247	/* AL Context-aware desktop assistant */
 #define KEY_KBD_LAYOUT_NEXT	0x248	/* AC Next Keyboard Layout Select */
 #define KEY_EMOJI_PICKER	0x249	/* Show/hide emoji picker (HUTRR101) */
+#define KEY_DICTATE		0x24a	/* Start or Stop Voice Dictation Session (HUTRR99) */
 
 #define KEY_BRIGHTNESS_MIN		0x250	/* Set Brightness to Minimum */
 #define KEY_BRIGHTNESS_MAX		0x251	/* Set Brightness to Maximum */
@@ -660,6 +662,27 @@
 /* Select an area of screen to be copied */
 #define KEY_SELECTIVE_SCREENSHOT	0x27a
 
+/* Move the focus to the next or previous user controllable element within a UI container */
+#define KEY_NEXT_ELEMENT               0x27b
+#define KEY_PREVIOUS_ELEMENT           0x27c
+
+/* Toggle Autopilot engagement */
+#define KEY_AUTOPILOT_ENGAGE_TOGGLE    0x27d
+
+/* Shortcut Keys */
+#define KEY_MARK_WAYPOINT              0x27e
+#define KEY_SOS                                0x27f
+#define KEY_NAV_CHART                  0x280
+#define KEY_FISHING_CHART              0x281
+#define KEY_SINGLE_RANGE_RADAR         0x282
+#define KEY_DUAL_RANGE_RADAR           0x283
+#define KEY_RADAR_OVERLAY              0x284
+#define KEY_TRADITIONAL_SONAR          0x285
+#define KEY_CLEARVU_SONAR              0x286
+#define KEY_SIDEVU_SONAR               0x287
+#define KEY_NAV_INFO                   0x288
+#define KEY_BRIGHTNESS_MENU            0x289
+
 /*
  * Some keyboards have keys which do not have a defined meaning, these keys
  * are intended to be programmed / bound to macros by the user. For most
diff --git a/include/standard-headers/linux/virtio_config.h b/include/standard-headers/linux/virtio_config.h
index 22e3a85f67..7acd8d4abc 100644
--- a/include/standard-headers/linux/virtio_config.h
+++ b/include/standard-headers/linux/virtio_config.h
@@ -80,6 +80,12 @@
 /* This feature indicates support for the packed virtqueue layout. */
 #define VIRTIO_F_RING_PACKED		34
 
+/*
+ * Inorder feature indicates that all buffers are used by the device
+ * in the same order in which they have been made available.
+ */
+#define VIRTIO_F_IN_ORDER		35
+
 /*
  * This feature indicates that memory accesses by the driver and the
  * device are ordered in a way described by the platform.
diff --git a/include/standard-headers/linux/virtio_crypto.h b/include/standard-headers/linux/virtio_crypto.h
index 5ff0b4ee59..68066dafb6 100644
--- a/include/standard-headers/linux/virtio_crypto.h
+++ b/include/standard-headers/linux/virtio_crypto.h
@@ -37,6 +37,7 @@
 #define VIRTIO_CRYPTO_SERVICE_HASH   1
 #define VIRTIO_CRYPTO_SERVICE_MAC    2
 #define VIRTIO_CRYPTO_SERVICE_AEAD   3
+#define VIRTIO_CRYPTO_SERVICE_AKCIPHER 4
 
 #define VIRTIO_CRYPTO_OPCODE(service, op)   (((service) << 8) | (op))
 
@@ -57,6 +58,10 @@ struct virtio_crypto_ctrl_header {
 	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x02)
 #define VIRTIO_CRYPTO_AEAD_DESTROY_SESSION \
 	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x03)
+#define VIRTIO_CRYPTO_AKCIPHER_CREATE_SESSION \
+	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AKCIPHER, 0x04)
+#define VIRTIO_CRYPTO_AKCIPHER_DESTROY_SESSION \
+	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AKCIPHER, 0x05)
 	uint32_t opcode;
 	uint32_t algo;
 	uint32_t flag;
@@ -180,6 +185,58 @@ struct virtio_crypto_aead_create_session_req {
 	uint8_t padding[32];
 };
 
+struct virtio_crypto_rsa_session_para {
+#define VIRTIO_CRYPTO_RSA_RAW_PADDING   0
+#define VIRTIO_CRYPTO_RSA_PKCS1_PADDING 1
+	uint32_t padding_algo;
+
+#define VIRTIO_CRYPTO_RSA_NO_HASH   0
+#define VIRTIO_CRYPTO_RSA_MD2       1
+#define VIRTIO_CRYPTO_RSA_MD3       2
+#define VIRTIO_CRYPTO_RSA_MD4       3
+#define VIRTIO_CRYPTO_RSA_MD5       4
+#define VIRTIO_CRYPTO_RSA_SHA1      5
+#define VIRTIO_CRYPTO_RSA_SHA256    6
+#define VIRTIO_CRYPTO_RSA_SHA384    7
+#define VIRTIO_CRYPTO_RSA_SHA512    8
+#define VIRTIO_CRYPTO_RSA_SHA224    9
+	uint32_t hash_algo;
+};
+
+struct virtio_crypto_ecdsa_session_para {
+#define VIRTIO_CRYPTO_CURVE_UNKNOWN   0
+#define VIRTIO_CRYPTO_CURVE_NIST_P192 1
+#define VIRTIO_CRYPTO_CURVE_NIST_P224 2
+#define VIRTIO_CRYPTO_CURVE_NIST_P256 3
+#define VIRTIO_CRYPTO_CURVE_NIST_P384 4
+#define VIRTIO_CRYPTO_CURVE_NIST_P521 5
+	uint32_t curve_id;
+	uint32_t padding;
+};
+
+struct virtio_crypto_akcipher_session_para {
+#define VIRTIO_CRYPTO_NO_AKCIPHER    0
+#define VIRTIO_CRYPTO_AKCIPHER_RSA   1
+#define VIRTIO_CRYPTO_AKCIPHER_DSA   2
+#define VIRTIO_CRYPTO_AKCIPHER_ECDSA 3
+	uint32_t algo;
+
+#define VIRTIO_CRYPTO_AKCIPHER_KEY_TYPE_PUBLIC  1
+#define VIRTIO_CRYPTO_AKCIPHER_KEY_TYPE_PRIVATE 2
+	uint32_t keytype;
+	uint32_t keylen;
+
+	union {
+		struct virtio_crypto_rsa_session_para rsa;
+		struct virtio_crypto_ecdsa_session_para ecdsa;
+	} u;
+};
+
+struct virtio_crypto_akcipher_create_session_req {
+	struct virtio_crypto_akcipher_session_para para;
+	uint8_t padding[36];
+};
+
 struct virtio_crypto_alg_chain_session_para {
 #define VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_HASH_THEN_CIPHER  1
 #define VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_CIPHER_THEN_HASH  2
@@ -247,6 +304,8 @@ struct virtio_crypto_op_ctrl_req {
 			mac_create_session;
 		struct virtio_crypto_aead_create_session_req
 			aead_create_session;
+		struct virtio_crypto_akcipher_create_session_req
+			akcipher_create_session;
 		struct virtio_crypto_destroy_session_req
 			destroy_session;
 		uint8_t padding[56];
@@ -266,6 +325,14 @@ struct virtio_crypto_op_header {
 	VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x00)
 #define VIRTIO_CRYPTO_AEAD_DECRYPT \
 	VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x01)
+#define VIRTIO_CRYPTO_AKCIPHER_ENCRYPT \
+	VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AKCIPHER, 0x00)
+#define VIRTIO_CRYPTO_AKCIPHER_DECRYPT \
+	VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AKCIPHER, 0x01)
+#define VIRTIO_CRYPTO_AKCIPHER_SIGN \
+	VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AKCIPHER, 0x02)
+#define VIRTIO_CRYPTO_AKCIPHER_VERIFY \
+	VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AKCIPHER, 0x03)
 	uint32_t opcode;
 	/* algo should be service-specific algorithms */
 	uint32_t algo;
@@ -390,6 +457,16 @@ struct virtio_crypto_aead_data_req {
 	uint8_t padding[32];
 };
 
+struct virtio_crypto_akcipher_para {
+	uint32_t src_data_len;
+	uint32_t dst_data_len;
+};
+
+struct virtio_crypto_akcipher_data_req {
+	struct virtio_crypto_akcipher_para para;
+	uint8_t padding[40];
+};
+
 /* The request of the data virtqueue's packet */
 struct virtio_crypto_op_data_req {
 	struct virtio_crypto_op_header header;
@@ -399,6 +476,7 @@ struct virtio_crypto_op_data_req {
 		struct virtio_crypto_hash_data_req hash_req;
 		struct virtio_crypto_mac_data_req mac_req;
 		struct virtio_crypto_aead_data_req aead_req;
+		struct virtio_crypto_akcipher_data_req akcipher_req;
 		uint8_t padding[48];
 	} u;
 };
@@ -408,6 +486,8 @@ struct virtio_crypto_op_data_req {
 #define VIRTIO_CRYPTO_BADMSG    2
 #define VIRTIO_CRYPTO_NOTSUPP   3
 #define VIRTIO_CRYPTO_INVSESS   4 /* Invalid session id */
+#define VIRTIO_CRYPTO_NOSPC     5 /* no free session ID */
+#define VIRTIO_CRYPTO_KEY_REJECTED 6 /* Signature verification failed */
 
 /* The accelerator hardware is ready */
 #define VIRTIO_CRYPTO_S_HW_READY  (1 << 0)
@@ -438,7 +518,7 @@ struct virtio_crypto_config {
 	uint32_t max_cipher_key_len;
 	/* Maximum length of authenticated key */
 	uint32_t max_auth_key_len;
-	uint32_t reserve;
+	uint32_t akcipher_algo;
 	/* Maximum size of each crypto request's content */
 	uint64_t max_size;
 };
diff --git a/linux-headers/asm-arm64/kvm.h b/linux-headers/asm-arm64/kvm.h
index 3d2ce9912d..5c28a9737a 100644
--- a/linux-headers/asm-arm64/kvm.h
+++ b/linux-headers/asm-arm64/kvm.h
@@ -281,6 +281,11 @@ struct kvm_arm_copy_mte_tags {
 #define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2_NOT_REQUIRED	3
 #define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2_ENABLED     	(1U << 4)
 
+#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3	KVM_REG_ARM_FW_REG(3)
+#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_AVAIL		0
+#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_AVAIL		1
+#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_REQUIRED	2
+
 /* SVE registers */
 #define KVM_REG_ARM64_SVE		(0x15 << KVM_REG_ARM_COPROC_SHIFT)
 
@@ -362,6 +367,7 @@ struct kvm_arm_copy_mte_tags {
 #define   KVM_ARM_VCPU_PMU_V3_IRQ	0
 #define   KVM_ARM_VCPU_PMU_V3_INIT	1
 #define   KVM_ARM_VCPU_PMU_V3_FILTER	2
+#define   KVM_ARM_VCPU_PMU_V3_SET_PMU	3
 #define KVM_ARM_VCPU_TIMER_CTRL		1
 #define   KVM_ARM_VCPU_TIMER_IRQ_VTIMER		0
 #define   KVM_ARM_VCPU_TIMER_IRQ_PTIMER		1
@@ -411,6 +417,16 @@ struct kvm_arm_copy_mte_tags {
 #define KVM_PSCI_RET_INVAL		PSCI_RET_INVALID_PARAMS
 #define KVM_PSCI_RET_DENIED		PSCI_RET_DENIED
 
+/* arm64-specific kvm_run::system_event flags */
+/*
+ * Reset caused by a PSCI v1.1 SYSTEM_RESET2 call.
+ * Valid only when the system event has a type of KVM_SYSTEM_EVENT_RESET.
+ */
+#define KVM_SYSTEM_EVENT_RESET_FLAG_PSCI_RESET2	(1ULL << 0)
+
+/* run->fail_entry.hardware_entry_failure_reason codes. */
+#define KVM_EXIT_FAIL_ENTRY_CPU_UNSUPPORTED	(1ULL << 0)
+
 #endif
 
 #endif /* __ARM_KVM_H__ */
diff --git a/linux-headers/asm-generic/mman-common.h b/linux-headers/asm-generic/mman-common.h
index 1567a3294c..6c1aa92a92 100644
--- a/linux-headers/asm-generic/mman-common.h
+++ b/linux-headers/asm-generic/mman-common.h
@@ -75,6 +75,8 @@
 #define MADV_POPULATE_READ	22	/* populate (prefault) page tables readable */
 #define MADV_POPULATE_WRITE	23	/* populate (prefault) page tables writable */
 
+#define MADV_DONTNEED_LOCKED	24	/* like DONTNEED, but drop locked pages too */
+
 /* compatibility flags */
 #define MAP_FILE	0
 
diff --git a/linux-headers/asm-mips/mman.h b/linux-headers/asm-mips/mman.h
index 40b210c65a..1be428663c 100644
--- a/linux-headers/asm-mips/mman.h
+++ b/linux-headers/asm-mips/mman.h
@@ -101,6 +101,8 @@
 #define MADV_POPULATE_READ	22	/* populate (prefault) page tables readable */
 #define MADV_POPULATE_WRITE	23	/* populate (prefault) page tables writable */
 
+#define MADV_DONTNEED_LOCKED	24	/* like DONTNEED, but drop locked pages too */
+
 /* compatibility flags */
 #define MAP_FILE	0
 
diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index d232feaae9..0d05d02ee4 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -445,7 +445,11 @@ struct kvm_run {
 #define KVM_SYSTEM_EVENT_RESET          2
 #define KVM_SYSTEM_EVENT_CRASH          3
 			__u32 type;
-			__u64 flags;
+			__u32 ndata;
+			union {
+				__u64 flags;
+				__u64 data[16];
+			};
 		} system_event;
 		/* KVM_EXIT_S390_STSI */
 		struct {
@@ -562,9 +566,12 @@ struct kvm_s390_mem_op {
 	__u32 op;		/* type of operation */
 	__u64 buf;		/* buffer in userspace */
 	union {
-		__u8 ar;	/* the access register number */
+		struct {
+			__u8 ar;	/* the access register number */
+			__u8 key;	/* access key, ignored if flag unset */
+		};
 		__u32 sida_offset; /* offset into the sida */
-		__u8 reserved[32]; /* should be set to 0 */
+		__u8 reserved[32]; /* ignored */
 	};
 };
 /* types for kvm_s390_mem_op->op */
@@ -572,9 +579,12 @@ struct kvm_s390_mem_op {
 #define KVM_S390_MEMOP_LOGICAL_WRITE	1
 #define KVM_S390_MEMOP_SIDA_READ	2
 #define KVM_S390_MEMOP_SIDA_WRITE	3
+#define KVM_S390_MEMOP_ABSOLUTE_READ	4
+#define KVM_S390_MEMOP_ABSOLUTE_WRITE	5
 /* flags for kvm_s390_mem_op->flags */
 #define KVM_S390_MEMOP_F_CHECK_ONLY		(1ULL << 0)
 #define KVM_S390_MEMOP_F_INJECT_EXCEPTION	(1ULL << 1)
+#define KVM_S390_MEMOP_F_SKEY_PROTECTION	(1ULL << 2)
 
 /* for KVM_INTERRUPT */
 struct kvm_interrupt {
@@ -1134,6 +1144,12 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_VM_GPA_BITS 207
 #define KVM_CAP_XSAVE2 208
 #define KVM_CAP_SYS_ATTRIBUTES 209
+#define KVM_CAP_PPC_AIL_MODE_3 210
+#define KVM_CAP_S390_MEM_OP_EXTENSION 211
+#define KVM_CAP_PMU_CAPABILITY 212
+#define KVM_CAP_DISABLE_QUIRKS2 213
+/* #define KVM_CAP_VM_TSC_CONTROL 214 */
+#define KVM_CAP_SYSTEM_EVENT_DATA 215
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
@@ -1624,9 +1640,6 @@ struct kvm_enc_region {
 #define KVM_S390_NORMAL_RESET	_IO(KVMIO,   0xc3)
 #define KVM_S390_CLEAR_RESET	_IO(KVMIO,   0xc4)
 
-/* Available with KVM_CAP_XSAVE2 */
-#define KVM_GET_XSAVE2		  _IOR(KVMIO,  0xcf, struct kvm_xsave)
-
 struct kvm_s390_pv_sec_parm {
 	__u64 origin;
 	__u64 length;
@@ -1973,6 +1986,8 @@ struct kvm_dirty_gfn {
 #define KVM_BUS_LOCK_DETECTION_OFF             (1 << 0)
 #define KVM_BUS_LOCK_DETECTION_EXIT            (1 << 1)
 
+#define KVM_PMU_CAP_DISABLE                    (1 << 0)
+
 /**
  * struct kvm_stats_header - Header of per vm/vcpu binary statistics data.
  * @flags: Some extra information for header, always 0 for now.
diff --git a/linux-headers/linux/psci.h b/linux-headers/linux/psci.h
index a6772d508b..213b2a0f70 100644
--- a/linux-headers/linux/psci.h
+++ b/linux-headers/linux/psci.h
@@ -82,6 +82,10 @@
 #define PSCI_0_2_TOS_UP_NO_MIGRATE		1
 #define PSCI_0_2_TOS_MP				2
 
+/* PSCI v1.1 reset type encoding for SYSTEM_RESET2 */
+#define PSCI_1_1_RESET_TYPE_SYSTEM_WARM_RESET	0
+#define PSCI_1_1_RESET_TYPE_VENDOR_START	0x80000000U
+
 /* PSCI version decoding (independent of PSCI version) */
 #define PSCI_VERSION_MAJOR_SHIFT		16
 #define PSCI_VERSION_MINOR_MASK			\
diff --git a/linux-headers/linux/userfaultfd.h b/linux-headers/linux/userfaultfd.h
index 8479af5f4c..769b8379e4 100644
--- a/linux-headers/linux/userfaultfd.h
+++ b/linux-headers/linux/userfaultfd.h
@@ -32,7 +32,8 @@
 			   UFFD_FEATURE_SIGBUS |		\
 			   UFFD_FEATURE_THREAD_ID |		\
 			   UFFD_FEATURE_MINOR_HUGETLBFS |	\
-			   UFFD_FEATURE_MINOR_SHMEM)
+			   UFFD_FEATURE_MINOR_SHMEM |		\
+			   UFFD_FEATURE_EXACT_ADDRESS)
 #define UFFD_API_IOCTLS				\
 	((__u64)1 << _UFFDIO_REGISTER |		\
 	 (__u64)1 << _UFFDIO_UNREGISTER |	\
@@ -189,6 +190,10 @@ struct uffdio_api {
 	 *
 	 * UFFD_FEATURE_MINOR_SHMEM indicates the same support as
 	 * UFFD_FEATURE_MINOR_HUGETLBFS, but for shmem-backed pages instead.
+	 *
+	 * UFFD_FEATURE_EXACT_ADDRESS indicates that the exact address of page
+	 * faults would be provided and the offset within the page would not be
+	 * masked.
 	 */
 #define UFFD_FEATURE_PAGEFAULT_FLAG_WP		(1<<0)
 #define UFFD_FEATURE_EVENT_FORK			(1<<1)
@@ -201,6 +206,7 @@ struct uffdio_api {
 #define UFFD_FEATURE_THREAD_ID			(1<<8)
 #define UFFD_FEATURE_MINOR_HUGETLBFS		(1<<9)
 #define UFFD_FEATURE_MINOR_SHMEM		(1<<10)
+#define UFFD_FEATURE_EXACT_ADDRESS		(1<<11)
 	__u64 features;
 
 	__u64 ioctls;
diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
index e680594f27..e9f7795c39 100644
--- a/linux-headers/linux/vfio.h
+++ b/linux-headers/linux/vfio.h
@@ -323,7 +323,7 @@ struct vfio_region_info_cap_type {
 #define VFIO_REGION_TYPE_PCI_VENDOR_MASK	(0xffff)
 #define VFIO_REGION_TYPE_GFX                    (1)
 #define VFIO_REGION_TYPE_CCW			(2)
-#define VFIO_REGION_TYPE_MIGRATION              (3)
+#define VFIO_REGION_TYPE_MIGRATION_DEPRECATED   (3)
 
 /* sub-types for VFIO_REGION_TYPE_PCI_* */
 
@@ -405,225 +405,29 @@ struct vfio_region_gfx_edid {
 #define VFIO_REGION_SUBTYPE_CCW_CRW		(3)
 
 /* sub-types for VFIO_REGION_TYPE_MIGRATION */
-#define VFIO_REGION_SUBTYPE_MIGRATION           (1)
-
-/*
- * The structure vfio_device_migration_info is placed at the 0th offset of
- * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set VFIO device related
- * migration information. Field accesses from this structure are only supported
- * at their native width and alignment. Otherwise, the result is undefined and
- * vendor drivers should return an error.
- *
- * device_state: (read/write)
- *      - The user application writes to this field to inform the vendor driver
- *        about the device state to be transitioned to.
- *      - The vendor driver should take the necessary actions to change the
- *        device state. After successful transition to a given state, the
- *        vendor driver should return success on write(device_state, state)
- *        system call. If the device state transition fails, the vendor driver
- *        should return an appropriate -errno for the fault condition.
- *      - On the user application side, if the device state transition fails,
- *	  that is, if write(device_state, state) returns an error, read
- *	  device_state again to determine the current state of the device from
- *	  the vendor driver.
- *      - The vendor driver should return previous state of the device unless
- *        the vendor driver has encountered an internal error, in which case
- *        the vendor driver may report the device_state VFIO_DEVICE_STATE_ERROR.
- *      - The user application must use the device reset ioctl to recover the
- *        device from VFIO_DEVICE_STATE_ERROR state. If the device is
- *        indicated to be in a valid device state by reading device_state, the
- *        user application may attempt to transition the device to any valid
- *        state reachable from the current state or terminate itself.
- *
- *      device_state consists of 3 bits:
- *      - If bit 0 is set, it indicates the _RUNNING state. If bit 0 is clear,
- *        it indicates the _STOP state. When the device state is changed to
- *        _STOP, driver should stop the device before write() returns.
- *      - If bit 1 is set, it indicates the _SAVING state, which means that the
- *        driver should start gathering device state information that will be
- *        provided to the VFIO user application to save the device's state.
- *      - If bit 2 is set, it indicates the _RESUMING state, which means that
- *        the driver should prepare to resume the device. Data provided through
- *        the migration region should be used to resume the device.
- *      Bits 3 - 31 are reserved for future use. To preserve them, the user
- *      application should perform a read-modify-write operation on this
- *      field when modifying the specified bits.
- *
- *  +------- _RESUMING
- *  |+------ _SAVING
- *  ||+----- _RUNNING
- *  |||
- *  000b => Device Stopped, not saving or resuming
- *  001b => Device running, which is the default state
- *  010b => Stop the device & save the device state, stop-and-copy state
- *  011b => Device running and save the device state, pre-copy state
- *  100b => Device stopped and the device state is resuming
- *  101b => Invalid state
- *  110b => Error state
- *  111b => Invalid state
- *
- * State transitions:
- *
- *              _RESUMING  _RUNNING    Pre-copy    Stop-and-copy   _STOP
- *                (100b)     (001b)     (011b)        (010b)       (000b)
- * 0. Running or default state
- *                             |
- *
- * 1. Normal Shutdown (optional)
- *                             |------------------------------------->|
- *
- * 2. Save the state or suspend
- *                             |------------------------->|---------->|
- *
- * 3. Save the state during live migration
- *                             |----------->|------------>|---------->|
- *
- * 4. Resuming
- *                  |<---------|
- *
- * 5. Resumed
- *                  |--------->|
- *
- * 0. Default state of VFIO device is _RUNNING when the user application starts.
- * 1. During normal shutdown of the user application, the user application may
- *    optionally change the VFIO device state from _RUNNING to _STOP. This
- *    transition is optional. The vendor driver must support this transition but
- *    must not require it.
- * 2. When the user application saves state or suspends the application, the
- *    device state transitions from _RUNNING to stop-and-copy and then to _STOP.
- *    On state transition from _RUNNING to stop-and-copy, driver must stop the
- *    device, save the device state and send it to the application through the
- *    migration region. The sequence to be followed for such transition is given
- *    below.
- * 3. In live migration of user application, the state transitions from _RUNNING
- *    to pre-copy, to stop-and-copy, and to _STOP.
- *    On state transition from _RUNNING to pre-copy, the driver should start
- *    gathering the device state while the application is still running and send
- *    the device state data to application through the migration region.
- *    On state transition from pre-copy to stop-and-copy, the driver must stop
- *    the device, save the device state and send it to the user application
- *    through the migration region.
- *    Vendor drivers must support the pre-copy state even for implementations
- *    where no data is provided to the user before the stop-and-copy state. The
- *    user must not be required to consume all migration data before the device
- *    transitions to a new state, including the stop-and-copy state.
- *    The sequence to be followed for above two transitions is given below.
- * 4. To start the resuming phase, the device state should be transitioned from
- *    the _RUNNING to the _RESUMING state.
- *    In the _RESUMING state, the driver should use the device state data
- *    received through the migration region to resume the device.
- * 5. After providing saved device data to the driver, the application should
- *    change the state from _RESUMING to _RUNNING.
- *
- * reserved:
- *      Reads on this field return zero and writes are ignored.
- *
- * pending_bytes: (read only)
- *      The number of pending bytes still to be migrated from the vendor driver.
- *
- * data_offset: (read only)
- *      The user application should read data_offset field from the migration
- *      region. The user application should read the device data from this
- *      offset within the migration region during the _SAVING state or write
- *      the device data during the _RESUMING state. See below for details of
- *      sequence to be followed.
- *
- * data_size: (read/write)
- *      The user application should read data_size to get the size in bytes of
- *      the data copied in the migration region during the _SAVING state and
- *      write the size in bytes of the data copied in the migration region
- *      during the _RESUMING state.
- *
- * The format of the migration region is as follows:
- *  ------------------------------------------------------------------
- * |vfio_device_migration_info|    data section                      |
- * |                          |     ///////////////////////////////  |
- * ------------------------------------------------------------------
- *   ^                              ^
- *  offset 0-trapped part        data_offset
- *
- * The structure vfio_device_migration_info is always followed by the data
- * section in the region, so data_offset will always be nonzero. The offset
- * from where the data is copied is decided by the kernel driver. The data
- * section can be trapped, mmapped, or partitioned, depending on how the kernel
- * driver defines the data section. The data section partition can be defined
- * as mapped by the sparse mmap capability. If mmapped, data_offset must be
- * page aligned, whereas initial section which contains the
- * vfio_device_migration_info structure, might not end at the offset, which is
- * page aligned. The user is not required to access through mmap regardless
- * of the capabilities of the region mmap.
- * The vendor driver should determine whether and how to partition the data
- * section. The vendor driver should return data_offset accordingly.
- *
- * The sequence to be followed while in pre-copy state and stop-and-copy state
- * is as follows:
- * a. Read pending_bytes, indicating the start of a new iteration to get device
- *    data. Repeated read on pending_bytes at this stage should have no side
- *    effects.
- *    If pending_bytes == 0, the user application should not iterate to get data
- *    for that device.
- *    If pending_bytes > 0, perform the following steps.
- * b. Read data_offset, indicating that the vendor driver should make data
- *    available through the data section. The vendor driver should return this
- *    read operation only after data is available from (region + data_offset)
- *    to (region + data_offset + data_size).
- * c. Read data_size, which is the amount of data in bytes available through
- *    the migration region.
- *    Read on data_offset and data_size should return the offset and size of
- *    the current buffer if the user application reads data_offset and
- *    data_size more than once here.
- * d. Read data_size bytes of data from (region + data_offset) from the
- *    migration region.
- * e. Process the data.
- * f. Read pending_bytes, which indicates that the data from the previous
- *    iteration has been read. If pending_bytes > 0, go to step b.
- *
- * The user application can transition from the _SAVING|_RUNNING
- * (pre-copy state) to the _SAVING (stop-and-copy) state regardless of the
- * number of pending bytes. The user application should iterate in _SAVING
- * (stop-and-copy) until pending_bytes is 0.
- *
- * The sequence to be followed while _RESUMING device state is as follows:
- * While data for this device is available, repeat the following steps:
- * a. Read data_offset from where the user application should write data.
- * b. Write migration data starting at the migration region + data_offset for
- *    the length determined by data_size from the migration source.
- * c. Write data_size, which indicates to the vendor driver that data is
- *    written in the migration region. Vendor driver must return this write
- *    operations on consuming data. Vendor driver should apply the
- *    user-provided migration region data to the device resume state.
- *
- * If an error occurs during the above sequences, the vendor driver can return
- * an error code for next read() or write() operation, which will terminate the
- * loop. The user application should then take the next necessary action, for
- * example, failing migration or terminating the user application.
- *
- * For the user application, data is opaque. The user application should write
- * data in the same order as the data is received and the data should be of
- * same transaction size at the source.
- */
+#define VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATED (1)
 
 struct vfio_device_migration_info {
 	__u32 device_state;         /* VFIO device state */
-#define VFIO_DEVICE_STATE_STOP      (0)
-#define VFIO_DEVICE_STATE_RUNNING   (1 << 0)
-#define VFIO_DEVICE_STATE_SAVING    (1 << 1)
-#define VFIO_DEVICE_STATE_RESUMING  (1 << 2)
-#define VFIO_DEVICE_STATE_MASK      (VFIO_DEVICE_STATE_RUNNING | \
-				     VFIO_DEVICE_STATE_SAVING |  \
-				     VFIO_DEVICE_STATE_RESUMING)
+#define VFIO_DEVICE_STATE_V1_STOP      (0)
+#define VFIO_DEVICE_STATE_V1_RUNNING   (1 << 0)
+#define VFIO_DEVICE_STATE_V1_SAVING    (1 << 1)
+#define VFIO_DEVICE_STATE_V1_RESUMING  (1 << 2)
+#define VFIO_DEVICE_STATE_MASK      (VFIO_DEVICE_STATE_V1_RUNNING | \
+				     VFIO_DEVICE_STATE_V1_SAVING |  \
+				     VFIO_DEVICE_STATE_V1_RESUMING)
 
 #define VFIO_DEVICE_STATE_VALID(state) \
-	(state & VFIO_DEVICE_STATE_RESUMING ? \
-	(state & VFIO_DEVICE_STATE_MASK) == VFIO_DEVICE_STATE_RESUMING : 1)
+	(state & VFIO_DEVICE_STATE_V1_RESUMING ? \
+	(state & VFIO_DEVICE_STATE_MASK) == VFIO_DEVICE_STATE_V1_RESUMING : 1)
 
 #define VFIO_DEVICE_STATE_IS_ERROR(state) \
-	((state & VFIO_DEVICE_STATE_MASK) == (VFIO_DEVICE_STATE_SAVING | \
-					      VFIO_DEVICE_STATE_RESUMING))
+	((state & VFIO_DEVICE_STATE_MASK) == (VFIO_DEVICE_STATE_V1_SAVING | \
+					      VFIO_DEVICE_STATE_V1_RESUMING))
 
 #define VFIO_DEVICE_STATE_SET_ERROR(state) \
-	((state & ~VFIO_DEVICE_STATE_MASK) | VFIO_DEVICE_SATE_SAVING | \
-					     VFIO_DEVICE_STATE_RESUMING)
+	((state & ~VFIO_DEVICE_STATE_MASK) | VFIO_DEVICE_STATE_V1_SAVING | \
+					     VFIO_DEVICE_STATE_V1_RESUMING)
 
 	__u32 reserved;
 	__u64 pending_bytes;
@@ -1002,6 +806,186 @@ struct vfio_device_feature {
  */
 #define VFIO_DEVICE_FEATURE_PCI_VF_TOKEN	(0)
 
+/*
+ * Indicates the device can support the migration API through
+ * VFIO_DEVICE_FEATURE_MIG_DEVICE_STATE. If this GET succeeds, the RUNNING and
+ * ERROR states are always supported. Support for additional states is
+ * indicated via the flags field; at least VFIO_MIGRATION_STOP_COPY must be
+ * set.
+ *
+ * VFIO_MIGRATION_STOP_COPY means that STOP, STOP_COPY and
+ * RESUMING are supported.
+ *
+ * VFIO_MIGRATION_STOP_COPY | VFIO_MIGRATION_P2P means that RUNNING_P2P
+ * is supported in addition to the STOP_COPY states.
+ *
+ * Other combinations of flags have behavior to be defined in the future.
+ */
+struct vfio_device_feature_migration {
+	__aligned_u64 flags;
+#define VFIO_MIGRATION_STOP_COPY	(1 << 0)
+#define VFIO_MIGRATION_P2P		(1 << 1)
+};
+#define VFIO_DEVICE_FEATURE_MIGRATION 1
+
+/*
+ * Upon VFIO_DEVICE_FEATURE_SET, execute a migration state change on the VFIO
+ * device. The new state is supplied in device_state, see enum
+ * vfio_device_mig_state for details
+ *
+ * The kernel migration driver must fully transition the device to the new state
+ * value before the operation returns to the user.
+ *
+ * The kernel migration driver must not generate asynchronous device state
+ * transitions outside of manipulation by the user or the VFIO_DEVICE_RESET
+ * ioctl as described above.
+ *
+ * If this function fails then current device_state may be the original
+ * operating state or some other state along the combination transition path.
+ * The user can then decide if it should execute a VFIO_DEVICE_RESET, attempt
+ * to return to the original state, or attempt to return to some other state
+ * such as RUNNING or STOP.
+ *
+ * If the new_state starts a new data transfer session then the FD associated
+ * with that session is returned in data_fd. The user is responsible to close
+ * this FD when it is finished. The user must consider the migration data stream
+ * carried over the FD to be opaque and must preserve the byte order of the
+ * stream. The user is not required to preserve buffer segmentation when writing
+ * the data stream during the RESUMING operation.
+ *
+ * Upon VFIO_DEVICE_FEATURE_GET, get the current migration state of the VFIO
+ * device, data_fd will be -1.
+ */
+struct vfio_device_feature_mig_state {
+	__u32 device_state; /* From enum vfio_device_mig_state */
+	__s32 data_fd;
+};
+#define VFIO_DEVICE_FEATURE_MIG_DEVICE_STATE 2
+
+/*
+ * The device migration Finite State Machine is described by the enum
+ * vfio_device_mig_state. Some of the FSM arcs will create a migration data
+ * transfer session by returning a FD, in this case the migration data will
+ * flow over the FD using read() and write() as discussed below.
+ *
+ * There are 5 states to support VFIO_MIGRATION_STOP_COPY:
+ *  RUNNING - The device is running normally
+ *  STOP - The device does not change the internal or external state
+ *  STOP_COPY - The device internal state can be read out
+ *  RESUMING - The device is stopped and is loading a new internal state
+ *  ERROR - The device has failed and must be reset
+ *
+ * And 1 optional state to support VFIO_MIGRATION_P2P:
+ *  RUNNING_P2P - RUNNING, except the device cannot do peer to peer DMA
+ *
+ * The FSM takes actions on the arcs between FSM states. The driver implements
+ * the following behavior for the FSM arcs:
+ *
+ * RUNNING_P2P -> STOP
+ * STOP_COPY -> STOP
+ *   While in STOP the device must stop the operation of the device. The device
+ *   must not generate interrupts, DMA, or any other change to external state.
+ *   It must not change its internal state. When stopped the device and kernel
+ *   migration driver must accept and respond to interaction to support external
+ *   subsystems in the STOP state, for example PCI MSI-X and PCI config space.
+ *   Failure by the user to restrict device access while in STOP must not result
+ *   in error conditions outside the user context (ex. host system faults).
+ *
+ *   The STOP_COPY arc will terminate a data transfer session.
+ *
+ * RESUMING -> STOP
+ *   Leaving RESUMING terminates a data transfer session and indicates the
+ *   device should complete processing of the data delivered by write(). The
+ *   kernel migration driver should complete the incorporation of data written
+ *   to the data transfer FD into the device internal state and perform
+ *   final validity and consistency checking of the new device state. If the
+ *   user provided data is found to be incomplete, inconsistent, or otherwise
+ *   invalid, the migration driver must fail the SET_STATE ioctl and
+ *   optionally go to the ERROR state as described below.
+ *
+ *   While in STOP the device has the same behavior as other STOP states
+ *   described above.
+ *
+ *   To abort a RESUMING session the device must be reset.
+ *
+ * RUNNING_P2P -> RUNNING
+ *   While in RUNNING the device is fully operational, the device may generate
+ *   interrupts, DMA, respond to MMIO, all vfio device regions are functional,
+ *   and the device may advance its internal state.
+ *
+ * RUNNING -> RUNNING_P2P
+ * STOP -> RUNNING_P2P
+ *   While in RUNNING_P2P the device is partially running in the P2P quiescent
+ *   state defined below.
+ *
+ * STOP -> STOP_COPY
+ *   This arc begin the process of saving the device state and will return a
+ *   new data_fd.
+ *
+ *   While in the STOP_COPY state the device has the same behavior as STOP
+ *   with the addition that the data transfers session continues to stream the
+ *   migration state. End of stream on the FD indicates the entire device
+ *   state has been transferred.
+ *
+ *   The user should take steps to restrict access to vfio device regions while
+ *   the device is in STOP_COPY or risk corruption of the device migration data
+ *   stream.
+ *
+ * STOP -> RESUMING
+ *   Entering the RESUMING state starts a process of restoring the device state
+ *   and will return a new data_fd. The data stream fed into the data_fd should
+ *   be taken from the data transfer output of a single FD during saving from
+ *   a compatible device. The migration driver may alter/reset the internal
+ *   device state for this arc if required to prepare the device to receive the
+ *   migration data.
+ *
+ * any -> ERROR
+ *   ERROR cannot be specified as a device state, however any transition request
+ *   can be failed with an errno return and may then move the device_state into
+ *   ERROR. In this case the device was unable to execute the requested arc and
+ *   was also unable to restore the device to any valid device_state.
+ *   To recover from ERROR VFIO_DEVICE_RESET must be used to return the
+ *   device_state back to RUNNING.
+ *
+ * The optional peer to peer (P2P) quiescent state is intended to be a quiescent
+ * state for the device for the purposes of managing multiple devices within a
+ * user context where peer-to-peer DMA between devices may be active. The
+ * RUNNING_P2P states must prevent the device from initiating
+ * any new P2P DMA transactions. If the device can identify P2P transactions
+ * then it can stop only P2P DMA, otherwise it must stop all DMA. The migration
+ * driver must complete any such outstanding operations prior to completing the
+ * FSM arc into a P2P state. For the purpose of specification the states
+ * behave as though the device was fully running if not supported. Like while in
+ * STOP or STOP_COPY the user must not touch the device, otherwise the state
+ * can be exited.
+ *
+ * The remaining possible transitions are interpreted as combinations of the
+ * above FSM arcs. As there are multiple paths through the FSM arcs the path
+ * should be selected based on the following rules:
+ *   - Select the shortest path.
+ * Refer to vfio_mig_get_next_state() for the result of the algorithm.
+ *
+ * The automatic transit through the FSM arcs that make up the combination
+ * transition is invisible to the user. When working with combination arcs the
+ * user may see any step along the path in the device_state if SET_STATE
+ * fails. When handling these types of errors users should anticipate future
+ * revisions of this protocol using new states and those states becoming
+ * visible in this case.
+ *
+ * The optional states cannot be used with SET_STATE if the device does not
+ * support them. The user can discover if these states are supported by using
+ * VFIO_DEVICE_FEATURE_MIGRATION. By using combination transitions the user can
+ * avoid knowing about these optional states if the kernel driver supports them.
+ */
+enum vfio_device_mig_state {
+	VFIO_DEVICE_STATE_ERROR = 0,
+	VFIO_DEVICE_STATE_STOP = 1,
+	VFIO_DEVICE_STATE_RUNNING = 2,
+	VFIO_DEVICE_STATE_STOP_COPY = 3,
+	VFIO_DEVICE_STATE_RESUMING = 4,
+	VFIO_DEVICE_STATE_RUNNING_P2P = 5,
+};
+
 /* -------- API for Type1 VFIO IOMMU -------- */
 
 /**
diff --git a/linux-headers/linux/vhost.h b/linux-headers/linux/vhost.h
index c998860d7b..5d99e7c242 100644
--- a/linux-headers/linux/vhost.h
+++ b/linux-headers/linux/vhost.h
@@ -150,4 +150,11 @@
 /* Get the valid iova range */
 #define VHOST_VDPA_GET_IOVA_RANGE	_IOR(VHOST_VIRTIO, 0x78, \
 					     struct vhost_vdpa_iova_range)
+
+/* Get the config size */
+#define VHOST_VDPA_GET_CONFIG_SIZE	_IOR(VHOST_VIRTIO, 0x79, __u32)
+
+/* Get the count of all virtqueues */
+#define VHOST_VDPA_GET_VQS_COUNT	_IOR(VHOST_VIRTIO, 0x80, __u32)
+
 #endif

From f000bc74589247244943085b210cee32bac28c89 Mon Sep 17 00:00:00 2001
From: Ivan Shcherbakov <ivan@sysprogs.com>
Date: Sat, 14 May 2022 07:50:21 +0200
Subject: [PATCH 317/935] WHPX: fixed TPR/CR8 translation issues affecting VM
 debugging

This patch fixes the following error that would occur when trying to resume
a WHPX-accelerated VM from a breakpoint:

    qemu: WHPX: Failed to set interrupt state registers, hr=c0350005

The error arises from an incorrect CR8 value being passed to
WHvSetVirtualProcessorRegisters() that doesn't match the
value set via WHvSetVirtualProcessorInterruptControllerState2().

Signed-off-by: Ivan Shcherbakov <ivan@sysprogs.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/whpx/whpx-all.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/target/i386/whpx/whpx-all.c b/target/i386/whpx/whpx-all.c
index 23ae639b23..b22a3314b4 100644
--- a/target/i386/whpx/whpx-all.c
+++ b/target/i386/whpx/whpx-all.c
@@ -373,6 +373,8 @@ static int whpx_set_tsc(CPUState *cpu)
  *
  * This mechanism is described in section 10.8.6.1 of Volume 3 of Intel 64
  * and IA-32 Architectures Software Developer's Manual.
+ *
+ * The functions below translate the value of CR8 to TPR and vice versa.
  */
 
 static uint64_t whpx_apic_tpr_to_cr8(uint64_t tpr)
@@ -380,6 +382,11 @@ static uint64_t whpx_apic_tpr_to_cr8(uint64_t tpr)
     return tpr >> 4;
 }
 
+static uint64_t whpx_cr8_to_apic_tpr(uint64_t cr8)
+{
+    return cr8 << 4;
+}
+
 static void whpx_set_registers(CPUState *cpu, int level)
 {
     struct whpx_state *whpx = &whpx_global;
@@ -687,7 +694,7 @@ static void whpx_get_registers(CPUState *cpu)
     tpr = vcxt.values[idx++].Reg64;
     if (tpr != vcpu->tpr) {
         vcpu->tpr = tpr;
-        cpu_set_apic_tpr(x86_cpu->apic_state, tpr);
+        cpu_set_apic_tpr(x86_cpu->apic_state, whpx_cr8_to_apic_tpr(tpr));
     }
 
     /* 8 Debug Registers - Skipped */
@@ -1547,7 +1554,7 @@ static void whpx_vcpu_pre_run(CPUState *cpu)
      }
 
     /* Sync the TPR to the CR8 if was modified during the intercept */
-    tpr = cpu_get_apic_tpr(x86_cpu->apic_state);
+    tpr = whpx_apic_tpr_to_cr8(cpu_get_apic_tpr(x86_cpu->apic_state));
     if (tpr != vcpu->tpr) {
         vcpu->tpr = tpr;
         reg_values[reg_count].Reg64 = tpr;
@@ -1596,7 +1603,7 @@ static void whpx_vcpu_post_run(CPUState *cpu)
     if (vcpu->tpr != tpr) {
         vcpu->tpr = tpr;
         qemu_mutex_lock_iothread();
-        cpu_set_apic_tpr(x86_cpu->apic_state, vcpu->tpr);
+        cpu_set_apic_tpr(x86_cpu->apic_state, whpx_cr8_to_apic_tpr(vcpu->tpr));
         qemu_mutex_unlock_iothread();
     }
 

From 580ea136eb37b9a059bdba9c20260dc825bcc2de Mon Sep 17 00:00:00 2001
From: Konstantin Kostiuk <kkostiuk@redhat.com>
Date: Thu, 12 May 2022 18:49:06 +0300
Subject: [PATCH 318/935] qga-vss: Add auto generated headers to dependencies

Signed-off-by: Konstantin Kostiuk <kkostiuk@redhat.com>
Message-Id: <20220512154906.331399-1-kkostiuk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 qga/vss-win32/meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/qga/vss-win32/meson.build b/qga/vss-win32/meson.build
index 26c5dd6e0e..9483ccd3b8 100644
--- a/qga/vss-win32/meson.build
+++ b/qga/vss-win32/meson.build
@@ -7,7 +7,7 @@ link_args = cc.get_supported_link_arguments([
 
 qga_vss = shared_module(
   'qga-vss',
-  ['requester.cpp', 'provider.cpp', 'install.cpp'],
+  ['requester.cpp', 'provider.cpp', 'install.cpp', genh],
   name_prefix: '',
   cpp_args: ['-Wno-unknown-pragmas', '-Wno-delete-non-virtual-dtor', '-Wno-non-virtual-dtor'],
   link_args: link_args,

From 6c1d88c72bd740cc431a13a02145168cf87866fc Mon Sep 17 00:00:00 2001
From: Konstantin Kostiuk <kkostiuk@redhat.com>
Date: Thu, 12 May 2022 18:49:09 +0300
Subject: [PATCH 319/935] qga-vss: Use the proper operator to free memory
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

volume_name_wchar is allocated by 'void* operator new [](long long unsigned int)

Signed-off-by: Konstantin Kostiuk <kkostiuk@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20220512154909.331481-1-kkostiuk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 qga/vss-win32/requester.cpp | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/qga/vss-win32/requester.cpp b/qga/vss-win32/requester.cpp
index 4513324dd2..b371affeab 100644
--- a/qga/vss-win32/requester.cpp
+++ b/qga/vss-win32/requester.cpp
@@ -354,12 +354,12 @@ void requester_freeze(int *num_vols, void *mountpoints, ErrorSet *errset)
             if (FAILED(hr)) {
                 err_set(errset, hr, "failed to add %S to snapshot set",
                         volume_name_wchar);
-                delete volume_name_wchar;
+                delete[] volume_name_wchar;
                 goto out;
             }
             num_mount_points++;
 
-            delete volume_name_wchar;
+            delete[] volume_name_wchar;
         }
 
         if (num_mount_points == 0) {

From 6df39f5e583ca0f67bd934d1327f9ead2e3bd49c Mon Sep 17 00:00:00 2001
From: Robert Hoo <robert.hu@linux.intel.com>
Date: Mon, 14 Mar 2022 16:50:59 +0800
Subject: [PATCH 320/935] i386/cpu: Remove the deprecated cpu model
 'Icelake-Client'

Icelake, is the codename for Intel 3rd generation Xeon Scalable server
processors. There isn't ever client variants. This "Icelake-Client" CPU
model was added wrongly and imaginarily.

It has been deprecated since v5.2, now it's time to remove it completely
from code.

Signed-off-by: Robert Hoo <robert.hu@linux.intel.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <1647247859-4947-1-git-send-email-robert.hu@linux.intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/about/deprecated.rst       |   6 --
 docs/about/removed-features.rst |   6 ++
 target/i386/cpu.c               | 122 --------------------------------
 3 files changed, 6 insertions(+), 128 deletions(-)

diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst
index 896e5a97ab..44f52c172d 100644
--- a/docs/about/deprecated.rst
+++ b/docs/about/deprecated.rst
@@ -270,12 +270,6 @@ from Linux upstream kernel, declare it deprecated.
 System emulator CPUS
 --------------------
 
-``Icelake-Client`` CPU Model (since 5.2)
-''''''''''''''''''''''''''''''''''''''''
-
-``Icelake-Client`` CPU Models are deprecated. Use ``Icelake-Server`` CPU
-Models instead.
-
 MIPS ``I7200`` CPU Model (since 5.2)
 ''''''''''''''''''''''''''''''''''''
 
diff --git a/docs/about/removed-features.rst b/docs/about/removed-features.rst
index 4a0b270296..ba78bc1a19 100644
--- a/docs/about/removed-features.rst
+++ b/docs/about/removed-features.rst
@@ -557,6 +557,12 @@ Support for this CPU was removed from the upstream Linux kernel, and
 there is no available upstream toolchain to build binaries for it.
 Removed without replacement.
 
+x86 ``Icelake-Client`` CPU (removed in 7.1)
+'''''''''''''''''''''''''''''''''''''''''''
+
+There isn't ever Icelake Client CPU, it is some wrong and imaginary one.
+Use ``Icelake-Server`` instead.
+
 System emulator machines
 ------------------------
 
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index c4a17c93f6..6083e556f5 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -3258,128 +3258,6 @@ static const X86CPUDefinition builtin_x86_defs[] = {
             { /* end of list */ }
         }
     },
-    {
-        .name = "Icelake-Client",
-        .level = 0xd,
-        .vendor = CPUID_VENDOR_INTEL,
-        .family = 6,
-        .model = 126,
-        .stepping = 0,
-        .features[FEAT_1_EDX] =
-            CPUID_VME | CPUID_SSE2 | CPUID_SSE | CPUID_FXSR | CPUID_MMX |
-            CPUID_CLFLUSH | CPUID_PSE36 | CPUID_PAT | CPUID_CMOV | CPUID_MCA |
-            CPUID_PGE | CPUID_MTRR | CPUID_SEP | CPUID_APIC | CPUID_CX8 |
-            CPUID_MCE | CPUID_PAE | CPUID_MSR | CPUID_TSC | CPUID_PSE |
-            CPUID_DE | CPUID_FP87,
-        .features[FEAT_1_ECX] =
-            CPUID_EXT_AVX | CPUID_EXT_XSAVE | CPUID_EXT_AES |
-            CPUID_EXT_POPCNT | CPUID_EXT_X2APIC | CPUID_EXT_SSE42 |
-            CPUID_EXT_SSE41 | CPUID_EXT_CX16 | CPUID_EXT_SSSE3 |
-            CPUID_EXT_PCLMULQDQ | CPUID_EXT_SSE3 |
-            CPUID_EXT_TSC_DEADLINE_TIMER | CPUID_EXT_FMA | CPUID_EXT_MOVBE |
-            CPUID_EXT_PCID | CPUID_EXT_F16C | CPUID_EXT_RDRAND,
-        .features[FEAT_8000_0001_EDX] =
-            CPUID_EXT2_LM | CPUID_EXT2_RDTSCP | CPUID_EXT2_NX |
-            CPUID_EXT2_SYSCALL,
-        .features[FEAT_8000_0001_ECX] =
-            CPUID_EXT3_ABM | CPUID_EXT3_LAHF_LM | CPUID_EXT3_3DNOWPREFETCH,
-        .features[FEAT_8000_0008_EBX] =
-            CPUID_8000_0008_EBX_WBNOINVD,
-        .features[FEAT_7_0_EBX] =
-            CPUID_7_0_EBX_FSGSBASE | CPUID_7_0_EBX_BMI1 |
-            CPUID_7_0_EBX_HLE | CPUID_7_0_EBX_AVX2 | CPUID_7_0_EBX_SMEP |
-            CPUID_7_0_EBX_BMI2 | CPUID_7_0_EBX_ERMS | CPUID_7_0_EBX_INVPCID |
-            CPUID_7_0_EBX_RTM | CPUID_7_0_EBX_RDSEED | CPUID_7_0_EBX_ADX |
-            CPUID_7_0_EBX_SMAP,
-        .features[FEAT_7_0_ECX] =
-            CPUID_7_0_ECX_AVX512_VBMI | CPUID_7_0_ECX_UMIP | CPUID_7_0_ECX_PKU |
-            CPUID_7_0_ECX_AVX512_VBMI2 | CPUID_7_0_ECX_GFNI |
-            CPUID_7_0_ECX_VAES | CPUID_7_0_ECX_VPCLMULQDQ |
-            CPUID_7_0_ECX_AVX512VNNI | CPUID_7_0_ECX_AVX512BITALG |
-            CPUID_7_0_ECX_AVX512_VPOPCNTDQ,
-        .features[FEAT_7_0_EDX] =
-            CPUID_7_0_EDX_SPEC_CTRL | CPUID_7_0_EDX_SPEC_CTRL_SSBD,
-        /* XSAVES is added in version 3 */
-        .features[FEAT_XSAVE] =
-            CPUID_XSAVE_XSAVEOPT | CPUID_XSAVE_XSAVEC |
-            CPUID_XSAVE_XGETBV1,
-        .features[FEAT_6_EAX] =
-            CPUID_6_EAX_ARAT,
-        /* Missing: Mode-based execute control (XS/XU), processor tracing, TSC scaling */
-        .features[FEAT_VMX_BASIC] = MSR_VMX_BASIC_INS_OUTS |
-             MSR_VMX_BASIC_TRUE_CTLS,
-        .features[FEAT_VMX_ENTRY_CTLS] = VMX_VM_ENTRY_IA32E_MODE |
-             VMX_VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL | VMX_VM_ENTRY_LOAD_IA32_PAT |
-             VMX_VM_ENTRY_LOAD_DEBUG_CONTROLS | VMX_VM_ENTRY_LOAD_IA32_EFER,
-        .features[FEAT_VMX_EPT_VPID_CAPS] = MSR_VMX_EPT_EXECONLY |
-             MSR_VMX_EPT_PAGE_WALK_LENGTH_4 | MSR_VMX_EPT_WB | MSR_VMX_EPT_2MB |
-             MSR_VMX_EPT_1GB | MSR_VMX_EPT_INVEPT |
-             MSR_VMX_EPT_INVEPT_SINGLE_CONTEXT | MSR_VMX_EPT_INVEPT_ALL_CONTEXT |
-             MSR_VMX_EPT_INVVPID | MSR_VMX_EPT_INVVPID_SINGLE_ADDR |
-             MSR_VMX_EPT_INVVPID_SINGLE_CONTEXT | MSR_VMX_EPT_INVVPID_ALL_CONTEXT |
-             MSR_VMX_EPT_INVVPID_SINGLE_CONTEXT_NOGLOBALS | MSR_VMX_EPT_AD_BITS,
-        .features[FEAT_VMX_EXIT_CTLS] =
-             VMX_VM_EXIT_ACK_INTR_ON_EXIT | VMX_VM_EXIT_SAVE_DEBUG_CONTROLS |
-             VMX_VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL |
-             VMX_VM_EXIT_LOAD_IA32_PAT | VMX_VM_EXIT_LOAD_IA32_EFER |
-             VMX_VM_EXIT_SAVE_IA32_PAT | VMX_VM_EXIT_SAVE_IA32_EFER |
-             VMX_VM_EXIT_SAVE_VMX_PREEMPTION_TIMER,
-        .features[FEAT_VMX_MISC] = MSR_VMX_MISC_ACTIVITY_HLT |
-             MSR_VMX_MISC_STORE_LMA | MSR_VMX_MISC_VMWRITE_VMEXIT,
-        .features[FEAT_VMX_PINBASED_CTLS] = VMX_PIN_BASED_EXT_INTR_MASK |
-             VMX_PIN_BASED_NMI_EXITING | VMX_PIN_BASED_VIRTUAL_NMIS |
-             VMX_PIN_BASED_VMX_PREEMPTION_TIMER,
-        .features[FEAT_VMX_PROCBASED_CTLS] = VMX_CPU_BASED_VIRTUAL_INTR_PENDING |
-             VMX_CPU_BASED_USE_TSC_OFFSETING | VMX_CPU_BASED_HLT_EXITING |
-             VMX_CPU_BASED_INVLPG_EXITING | VMX_CPU_BASED_MWAIT_EXITING |
-             VMX_CPU_BASED_RDPMC_EXITING | VMX_CPU_BASED_RDTSC_EXITING |
-             VMX_CPU_BASED_CR8_LOAD_EXITING | VMX_CPU_BASED_CR8_STORE_EXITING |
-             VMX_CPU_BASED_TPR_SHADOW | VMX_CPU_BASED_MOV_DR_EXITING |
-             VMX_CPU_BASED_UNCOND_IO_EXITING | VMX_CPU_BASED_USE_IO_BITMAPS |
-             VMX_CPU_BASED_MONITOR_EXITING | VMX_CPU_BASED_PAUSE_EXITING |
-             VMX_CPU_BASED_VIRTUAL_NMI_PENDING | VMX_CPU_BASED_USE_MSR_BITMAPS |
-             VMX_CPU_BASED_CR3_LOAD_EXITING | VMX_CPU_BASED_CR3_STORE_EXITING |
-             VMX_CPU_BASED_MONITOR_TRAP_FLAG |
-             VMX_CPU_BASED_ACTIVATE_SECONDARY_CONTROLS,
-        .features[FEAT_VMX_SECONDARY_CTLS] =
-             VMX_SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES |
-             VMX_SECONDARY_EXEC_WBINVD_EXITING | VMX_SECONDARY_EXEC_ENABLE_EPT |
-             VMX_SECONDARY_EXEC_DESC | VMX_SECONDARY_EXEC_RDTSCP |
-             VMX_SECONDARY_EXEC_ENABLE_VPID | VMX_SECONDARY_EXEC_UNRESTRICTED_GUEST |
-             VMX_SECONDARY_EXEC_RDRAND_EXITING | VMX_SECONDARY_EXEC_ENABLE_INVPCID |
-             VMX_SECONDARY_EXEC_ENABLE_VMFUNC | VMX_SECONDARY_EXEC_SHADOW_VMCS |
-             VMX_SECONDARY_EXEC_RDSEED_EXITING | VMX_SECONDARY_EXEC_ENABLE_PML,
-        .features[FEAT_VMX_VMFUNC] = MSR_VMX_VMFUNC_EPT_SWITCHING,
-        .xlevel = 0x80000008,
-        .model_id = "Intel Core Processor (Icelake)",
-        .versions = (X86CPUVersionDefinition[]) {
-            {
-                .version = 1,
-                .note = "deprecated"
-            },
-            {
-                .version = 2,
-                .note = "no TSX, deprecated",
-                .alias = "Icelake-Client-noTSX",
-                .props = (PropValue[]) {
-                    { "hle", "off" },
-                    { "rtm", "off" },
-                    { /* end of list */ }
-                },
-            },
-            {
-                .version = 3,
-                .note = "no TSX, XSAVES, deprecated",
-                .props = (PropValue[]) {
-                    { "xsaves", "on" },
-                    { "vmx-xsaves", "on" },
-                    { /* end of list */ }
-                },
-            },
-            { /* end of list */ }
-        },
-        .deprecation_note = "use Icelake-Server instead"
-    },
     {
         .name = "Icelake-Server",
         .level = 0xd,

From 18c22d7112a76fabeee5022a3bdd9e8c3a37c8d2 Mon Sep 17 00:00:00 2001
From: Yang Weijiang <weijiang.yang@intel.com>
Date: Tue, 15 Feb 2022 14:52:51 -0500
Subject: [PATCH 321/935] qdev-properties: Add a new macro with bitmask check
 for uint64_t property

The DEFINE_PROP_UINT64_CHECKMASK maro applies certain mask check agaist
user-supplied property value, reject the value if it violates the bitmask.

Co-developed-by: Like Xu <like.xu@linux.intel.com>
Signed-off-by: Like Xu <like.xu@linux.intel.com>
Signed-off-by: Yang Weijiang <weijiang.yang@intel.com>
Message-Id: <20220215195258.29149-2-weijiang.yang@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/core/qdev-properties.c    | 19 +++++++++++++++++++
 include/hw/qdev-properties.h | 12 ++++++++++++
 2 files changed, 31 insertions(+)

diff --git a/hw/core/qdev-properties.c b/hw/core/qdev-properties.c
index c34aac6ebc..357b8761b5 100644
--- a/hw/core/qdev-properties.c
+++ b/hw/core/qdev-properties.c
@@ -428,6 +428,25 @@ const PropertyInfo qdev_prop_int64 = {
     .set_default_value = qdev_propinfo_set_default_value_int,
 };
 
+static void set_uint64_checkmask(Object *obj, Visitor *v, const char *name,
+                      void *opaque, Error **errp)
+{
+    Property *prop = opaque;
+    uint64_t *ptr = object_field_prop_ptr(obj, prop);
+
+    visit_type_uint64(v, name, ptr, errp);
+    if (*ptr & ~prop->bitmask) {
+        error_setg(errp, "Property value for '%s' has bits outside mask '0x%" PRIx64 "'",
+                   name, prop->bitmask);
+    }
+}
+
+const PropertyInfo qdev_prop_uint64_checkmask = {
+    .name  = "uint64",
+    .get   = get_uint64,
+    .set   = set_uint64_checkmask,
+};
+
 /* --- string --- */
 
 static void release_string(Object *obj, const char *name, void *opaque)
diff --git a/include/hw/qdev-properties.h b/include/hw/qdev-properties.h
index f7925f67d0..e1df08876c 100644
--- a/include/hw/qdev-properties.h
+++ b/include/hw/qdev-properties.h
@@ -17,6 +17,7 @@ struct Property {
     const PropertyInfo *info;
     ptrdiff_t    offset;
     uint8_t      bitnr;
+    uint64_t     bitmask;
     bool         set_default;
     union {
         int64_t i;
@@ -54,6 +55,7 @@ extern const PropertyInfo qdev_prop_uint16;
 extern const PropertyInfo qdev_prop_uint32;
 extern const PropertyInfo qdev_prop_int32;
 extern const PropertyInfo qdev_prop_uint64;
+extern const PropertyInfo qdev_prop_uint64_checkmask;
 extern const PropertyInfo qdev_prop_int64;
 extern const PropertyInfo qdev_prop_size;
 extern const PropertyInfo qdev_prop_string;
@@ -103,6 +105,16 @@ extern const PropertyInfo qdev_prop_link;
                 .set_default = true,                         \
                 .defval.u    = (bool)_defval)
 
+/**
+ * The DEFINE_PROP_UINT64_CHECKMASK macro checks a user-supplied value
+ * against corresponding bitmask, rejects the value if it violates.
+ * The default value is set in instance_init().
+ */
+#define DEFINE_PROP_UINT64_CHECKMASK(_name, _state, _field, _bitmask)   \
+    DEFINE_PROP(_name, _state, _field, qdev_prop_uint64_checkmask, uint64_t, \
+                .bitmask    = (_bitmask),                     \
+                .set_default = false)
+
 #define PROP_ARRAY_LEN_PREFIX "len-"
 
 /**

From f06d8a18abc9d29d052f628eefd1d4a86c99fcea Mon Sep 17 00:00:00 2001
From: Yang Weijiang <weijiang.yang@intel.com>
Date: Tue, 15 Feb 2022 14:52:52 -0500
Subject: [PATCH 322/935] target/i386: Add lbr-fmt vPMU option to support guest
 LBR

The Last Branch Recording (LBR) is a performance monitor unit (PMU)
feature on Intel processors which records a running trace of the most
recent branches taken by the processor in the LBR stack. This option
indicates the LBR format to enable for guest perf.

The LBR feature is enabled if below conditions are met:
1) KVM is enabled and the PMU is enabled.
2) msr-based-feature IA32_PERF_CAPABILITIES is supporterd on KVM.
3) Supported returned value for lbr_fmt from above msr is non-zero.
4) Guest vcpu model does support FEAT_1_ECX.CPUID_EXT_PDCM.
5) User-provided lbr-fmt value doesn't violate its bitmask (0x3f).
6) Target guest LBR format matches that of host.

Co-developed-by: Like Xu <like.xu@linux.intel.com>
Signed-off-by: Like Xu <like.xu@linux.intel.com>
Signed-off-by: Yang Weijiang <weijiang.yang@intel.com>
Message-Id: <20220215195258.29149-3-weijiang.yang@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/cpu.c | 40 ++++++++++++++++++++++++++++++++++++++++
 target/i386/cpu.h | 10 ++++++++++
 2 files changed, 50 insertions(+)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 6083e556f5..856a8659e8 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6275,6 +6275,7 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
     CPUX86State *env = &cpu->env;
     Error *local_err = NULL;
     static bool ht_warned;
+    unsigned requested_lbr_fmt;
 
     if (cpu->apic_id == UNASSIGNED_APIC_ID) {
         error_setg(errp, "apic-id property was not initialized properly");
@@ -6292,6 +6293,42 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
         goto out;
     }
 
+    /*
+     * Override env->features[FEAT_PERF_CAPABILITIES].LBR_FMT
+     * with user-provided setting.
+     */
+    if (cpu->lbr_fmt != ~PERF_CAP_LBR_FMT) {
+        if ((cpu->lbr_fmt & PERF_CAP_LBR_FMT) != cpu->lbr_fmt) {
+            error_setg(errp, "invalid lbr-fmt");
+            return;
+        }
+        env->features[FEAT_PERF_CAPABILITIES] &= ~PERF_CAP_LBR_FMT;
+        env->features[FEAT_PERF_CAPABILITIES] |= cpu->lbr_fmt;
+    }
+
+    /*
+     * vPMU LBR is supported when 1) KVM is enabled 2) Option pmu=on and
+     * 3)vPMU LBR format matches that of host setting.
+     */
+    requested_lbr_fmt =
+        env->features[FEAT_PERF_CAPABILITIES] & PERF_CAP_LBR_FMT;
+    if (requested_lbr_fmt && kvm_enabled()) {
+        uint64_t host_perf_cap =
+            x86_cpu_get_supported_feature_word(FEAT_PERF_CAPABILITIES, false);
+        unsigned host_lbr_fmt = host_perf_cap & PERF_CAP_LBR_FMT;
+
+        if (!cpu->enable_pmu) {
+            error_setg(errp, "vPMU: LBR is unsupported without pmu=on");
+            return;
+        }
+        if (requested_lbr_fmt != host_lbr_fmt) {
+            error_setg(errp, "vPMU: the lbr-fmt value (0x%x) does not match "
+                        "the host value (0x%x).",
+                        requested_lbr_fmt, host_lbr_fmt);
+            return;
+        }
+    }
+
     x86_cpu_filter_features(cpu, cpu->check_cpuid || cpu->enforce_cpuid);
 
     if (cpu->enforce_cpuid && x86_cpu_have_filtered_features(cpu)) {
@@ -6644,6 +6681,8 @@ static void x86_cpu_initfn(Object *obj)
     object_property_add_alias(obj, "sse4_2", obj, "sse4.2");
 
     object_property_add_alias(obj, "hv-apicv", obj, "hv-avic");
+    cpu->lbr_fmt = ~PERF_CAP_LBR_FMT;
+    object_property_add_alias(obj, "lbr_fmt", obj, "lbr-fmt");
 
     if (xcc->model) {
         x86_cpu_load_model(cpu, xcc->model);
@@ -6798,6 +6837,7 @@ static Property x86_cpu_properties[] = {
 #endif
     DEFINE_PROP_INT32("node-id", X86CPU, node_id, CPU_UNSET_NUMA_NODE_ID),
     DEFINE_PROP_BOOL("pmu", X86CPU, enable_pmu, false),
+    DEFINE_PROP_UINT64_CHECKMASK("lbr-fmt", X86CPU, lbr_fmt, PERF_CAP_LBR_FMT),
 
     DEFINE_PROP_UINT32("hv-spinlocks", X86CPU, hyperv_spinlock_attempts,
                        HYPERV_SPINLOCK_NEVER_NOTIFY),
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 9661f9fbd1..6730df5dbf 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -386,6 +386,7 @@ typedef enum X86Seg {
 #define ARCH_CAP_TSX_CTRL_MSR		(1<<7)
 
 #define MSR_IA32_PERF_CAPABILITIES      0x345
+#define PERF_CAP_LBR_FMT                0x3f
 
 #define MSR_IA32_TSX_CTRL		0x122
 #define MSR_IA32_TSCDEADLINE            0x6e0
@@ -1810,6 +1811,15 @@ struct ArchCPU {
      */
     bool enable_pmu;
 
+    /*
+     * Enable LBR_FMT bits of IA32_PERF_CAPABILITIES MSR.
+     * This can't be initialized with a default because it doesn't have
+     * stable ABI support yet. It is only allowed to pass all LBR_FMT bits
+     * returned by kvm_arch_get_supported_msr_feature()(which depends on both
+     * host CPU and kernel capabilities) to the guest.
+     */
+    uint64_t lbr_fmt;
+
     /* LMCE support can be enabled/disabled via cpu option 'lmce=on/off'. It is
      * disabled by default to avoid breaking migration between QEMU with
      * different LMCE configurations.

From 5a778a5f820fdd907b95e93560637a61f6ea3c71 Mon Sep 17 00:00:00 2001
From: Yang Weijiang <weijiang.yang@intel.com>
Date: Tue, 15 Feb 2022 14:52:53 -0500
Subject: [PATCH 323/935] target/i386: Add kvm_get_one_msr helper

When try to get one msr from KVM, I found there's no such kind of
existing interface while kvm_put_one_msr() is there. So here comes
the patch. It'll remove redundant preparation code before finally
call KVM_GET_MSRS IOCTL.

No functional change intended.

Signed-off-by: Yang Weijiang <weijiang.yang@intel.com>
Message-Id: <20220215195258.29149-4-weijiang.yang@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/kvm/kvm.c | 46 ++++++++++++++++++++++++-------------------
 1 file changed, 26 insertions(+), 20 deletions(-)

diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index c885763a5b..536cbe5fad 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -141,6 +141,7 @@ static struct kvm_msr_list *kvm_feature_msrs;
 
 #define BUS_LOCK_SLICE_TIME 1000000000ULL /* ns */
 static RateLimit bus_lock_ratelimit_ctrl;
+static int kvm_get_one_msr(X86CPU *cpu, int index, uint64_t *value);
 
 int kvm_has_pit_state2(void)
 {
@@ -211,28 +212,21 @@ static int kvm_get_tsc(CPUState *cs)
 {
     X86CPU *cpu = X86_CPU(cs);
     CPUX86State *env = &cpu->env;
-    struct {
-        struct kvm_msrs info;
-        struct kvm_msr_entry entries[1];
-    } msr_data = {};
+    uint64_t value;
     int ret;
 
     if (env->tsc_valid) {
         return 0;
     }
 
-    memset(&msr_data, 0, sizeof(msr_data));
-    msr_data.info.nmsrs = 1;
-    msr_data.entries[0].index = MSR_IA32_TSC;
     env->tsc_valid = !runstate_is_running();
 
-    ret = kvm_vcpu_ioctl(CPU(cpu), KVM_GET_MSRS, &msr_data);
+    ret = kvm_get_one_msr(cpu, MSR_IA32_TSC, &value);
     if (ret < 0) {
         return ret;
     }
 
-    assert(ret == 1);
-    env->tsc = msr_data.entries[0].data;
+    env->tsc = value;
     return 0;
 }
 
@@ -1566,21 +1560,14 @@ static int hyperv_init_vcpu(X86CPU *cpu)
          * the kernel doesn't support setting vp_index; assert that its value
          * is in sync
          */
-        struct {
-            struct kvm_msrs info;
-            struct kvm_msr_entry entries[1];
-        } msr_data = {
-            .info.nmsrs = 1,
-            .entries[0].index = HV_X64_MSR_VP_INDEX,
-        };
+        uint64_t value;
 
-        ret = kvm_vcpu_ioctl(cs, KVM_GET_MSRS, &msr_data);
+        ret = kvm_get_one_msr(cpu, HV_X64_MSR_VP_INDEX, &value);
         if (ret < 0) {
             return ret;
         }
-        assert(ret == 1);
 
-        if (msr_data.entries[0].data != hyperv_vp_index(CPU(cpu))) {
+        if (value != hyperv_vp_index(CPU(cpu))) {
             error_report("kernel's vp_index != QEMU's vp_index");
             return -ENXIO;
         }
@@ -2839,6 +2826,25 @@ static int kvm_put_one_msr(X86CPU *cpu, int index, uint64_t value)
     return kvm_vcpu_ioctl(CPU(cpu), KVM_SET_MSRS, cpu->kvm_msr_buf);
 }
 
+static int kvm_get_one_msr(X86CPU *cpu, int index, uint64_t *value)
+{
+    int ret;
+    struct {
+        struct kvm_msrs info;
+        struct kvm_msr_entry entries[1];
+    } msr_data = {
+        .info.nmsrs = 1,
+        .entries[0].index = index,
+    };
+
+    ret = kvm_vcpu_ioctl(CPU(cpu), KVM_GET_MSRS, &msr_data);
+    if (ret < 0) {
+        return ret;
+    }
+    assert(ret == 1);
+    *value = msr_data.entries[0].data;
+    return ret;
+}
 void kvm_put_apicbase(X86CPU *cpu, uint64_t value)
 {
     int ret;

From 301e90675c3fed6cdc48682021a1ab42bc0e0d76 Mon Sep 17 00:00:00 2001
From: Yang Weijiang <weijiang.yang@intel.com>
Date: Tue, 15 Feb 2022 14:52:54 -0500
Subject: [PATCH 324/935] target/i386: Enable support for XSAVES based features

There're some new features, including Arch LBR, depending
on XSAVES/XRSTORS support, the new instructions will
save/restore data based on feature bits enabled in XCR0 | XSS.
This patch adds the basic support for related CPUID enumeration
and meanwhile changes the name from FEAT_XSAVE_COMP_{LO|HI} to
FEAT_XSAVE_XCR0_{LO|HI} to differentiate clearly the feature
bits in XCR0 and those in XSS.

Signed-off-by: Yang Weijiang <weijiang.yang@intel.com>
Message-Id: <20220215195258.29149-5-weijiang.yang@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/cpu.c | 104 +++++++++++++++++++++++++++++++++++-----------
 target/i386/cpu.h |  14 ++++++-
 2 files changed, 92 insertions(+), 26 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 856a8659e8..643536d05d 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -937,6 +937,34 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
         },
         .tcg_features = TCG_XSAVE_FEATURES,
     },
+    [FEAT_XSAVE_XSS_LO] = {
+        .type = CPUID_FEATURE_WORD,
+        .feat_names = {
+            NULL, NULL, NULL, NULL,
+            NULL, NULL, NULL, NULL,
+            NULL, NULL, NULL, NULL,
+            NULL, NULL, NULL, NULL,
+            NULL, NULL, NULL, NULL,
+            NULL, NULL, NULL, NULL,
+            NULL, NULL, NULL, NULL,
+            NULL, NULL, NULL, NULL,
+        },
+        .cpuid = {
+            .eax = 0xD,
+            .needs_ecx = true,
+            .ecx = 1,
+            .reg = R_ECX,
+        },
+    },
+    [FEAT_XSAVE_XSS_HI] = {
+        .type = CPUID_FEATURE_WORD,
+        .cpuid = {
+            .eax = 0xD,
+            .needs_ecx = true,
+            .ecx = 1,
+            .reg = R_EDX
+        },
+    },
     [FEAT_6_EAX] = {
         .type = CPUID_FEATURE_WORD,
         .feat_names = {
@@ -952,7 +980,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
         .cpuid = { .eax = 6, .reg = R_EAX, },
         .tcg_features = TCG_6_EAX_FEATURES,
     },
-    [FEAT_XSAVE_COMP_LO] = {
+    [FEAT_XSAVE_XCR0_LO] = {
         .type = CPUID_FEATURE_WORD,
         .cpuid = {
             .eax = 0xD,
@@ -965,7 +993,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
             XSTATE_OPMASK_MASK | XSTATE_ZMM_Hi256_MASK | XSTATE_Hi16_ZMM_MASK |
             XSTATE_PKRU_MASK,
     },
-    [FEAT_XSAVE_COMP_HI] = {
+    [FEAT_XSAVE_XCR0_HI] = {
         .type = CPUID_FEATURE_WORD,
         .cpuid = {
             .eax = 0xD,
@@ -1382,6 +1410,9 @@ static const X86RegisterInfo32 x86_reg_info_32[CPU_NB_REGS32] = {
 };
 #undef REGISTER
 
+/* CPUID feature bits available in XSS */
+#define CPUID_XSTATE_XSS_MASK    (0)
+
 ExtSaveArea x86_ext_save_areas[XSAVE_STATE_AREA_COUNT] = {
     [XSTATE_FP_BIT] = {
         /* x87 FP state component is always enabled if XSAVE is supported */
@@ -1424,15 +1455,18 @@ ExtSaveArea x86_ext_save_areas[XSAVE_STATE_AREA_COUNT] = {
     },
 };
 
-static uint32_t xsave_area_size(uint64_t mask)
+static uint32_t xsave_area_size(uint64_t mask, bool compacted)
 {
+    uint64_t ret = x86_ext_save_areas[0].size;
+    const ExtSaveArea *esa;
+    uint32_t offset = 0;
     int i;
-    uint64_t ret = 0;
 
-    for (i = 0; i < ARRAY_SIZE(x86_ext_save_areas); i++) {
-        const ExtSaveArea *esa = &x86_ext_save_areas[i];
+    for (i = 2; i < ARRAY_SIZE(x86_ext_save_areas); i++) {
+        esa = &x86_ext_save_areas[i];
         if ((mask >> i) & 1) {
-            ret = MAX(ret, esa->offset + esa->size);
+            offset = compacted ? ret : esa->offset;
+            ret = MAX(ret, offset + esa->size);
         }
     }
     return ret;
@@ -1443,10 +1477,10 @@ static inline bool accel_uses_host_cpuid(void)
     return kvm_enabled() || hvf_enabled();
 }
 
-static inline uint64_t x86_cpu_xsave_components(X86CPU *cpu)
+static inline uint64_t x86_cpu_xsave_xcr0_components(X86CPU *cpu)
 {
-    return ((uint64_t)cpu->env.features[FEAT_XSAVE_COMP_HI]) << 32 |
-           cpu->env.features[FEAT_XSAVE_COMP_LO];
+    return ((uint64_t)cpu->env.features[FEAT_XSAVE_XCR0_HI]) << 32 |
+           cpu->env.features[FEAT_XSAVE_XCR0_LO];
 }
 
 /* Return name of 32-bit register, from a R_* constant */
@@ -1458,6 +1492,12 @@ static const char *get_register_name_32(unsigned int reg)
     return x86_reg_info_32[reg].name;
 }
 
+static inline uint64_t x86_cpu_xsave_xss_components(X86CPU *cpu)
+{
+    return ((uint64_t)cpu->env.features[FEAT_XSAVE_XSS_HI]) << 32 |
+           cpu->env.features[FEAT_XSAVE_XSS_LO];
+}
+
 /*
  * Returns the set of feature flags that are supported and migratable by
  * QEMU, for a given FeatureWord.
@@ -4511,8 +4551,8 @@ static const char *x86_cpu_feature_name(FeatureWord w, int bitnr)
     /* XSAVE components are automatically enabled by other features,
      * so return the original feature name instead
      */
-    if (w == FEAT_XSAVE_COMP_LO || w == FEAT_XSAVE_COMP_HI) {
-        int comp = (w == FEAT_XSAVE_COMP_HI) ? bitnr + 32 : bitnr;
+    if (w == FEAT_XSAVE_XCR0_LO || w == FEAT_XSAVE_XCR0_HI) {
+        int comp = (w == FEAT_XSAVE_XCR0_HI) ? bitnr + 32 : bitnr;
 
         if (comp < ARRAY_SIZE(x86_ext_save_areas) &&
             x86_ext_save_areas[comp].bits) {
@@ -5408,25 +5448,36 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         }
 
         if (count == 0) {
-            *ecx = xsave_area_size(x86_cpu_xsave_components(cpu));
-            *eax = env->features[FEAT_XSAVE_COMP_LO];
-            *edx = env->features[FEAT_XSAVE_COMP_HI];
+            *ecx = xsave_area_size(x86_cpu_xsave_xcr0_components(cpu), false);
+            *eax = env->features[FEAT_XSAVE_XCR0_LO];
+            *edx = env->features[FEAT_XSAVE_XCR0_HI];
             /*
              * The initial value of xcr0 and ebx == 0, On host without kvm
              * commit 412a3c41(e.g., CentOS 6), the ebx's value always == 0
              * even through guest update xcr0, this will crash some legacy guest
              * (e.g., CentOS 6), So set ebx == ecx to workaroud it.
              */
-            *ebx = kvm_enabled() ? *ecx : xsave_area_size(env->xcr0);
+            *ebx = kvm_enabled() ? *ecx : xsave_area_size(env->xcr0, false);
         } else if (count == 1) {
+            uint64_t xstate = x86_cpu_xsave_xcr0_components(cpu) |
+                              x86_cpu_xsave_xss_components(cpu);
+
             *eax = env->features[FEAT_XSAVE];
+            *ebx = xsave_area_size(xstate, true);
+            *ecx = env->features[FEAT_XSAVE_XSS_LO];
+            *edx = env->features[FEAT_XSAVE_XSS_HI];
         } else if (count < ARRAY_SIZE(x86_ext_save_areas)) {
-            if ((x86_cpu_xsave_components(cpu) >> count) & 1) {
-                const ExtSaveArea *esa = &x86_ext_save_areas[count];
+            const ExtSaveArea *esa = &x86_ext_save_areas[count];
+
+            if (x86_cpu_xsave_xcr0_components(cpu) & (1ULL << count)) {
                 *eax = esa->size;
                 *ebx = esa->offset;
                 *ecx = esa->ecx &
                        (ESA_FEATURE_ALIGN64_MASK | ESA_FEATURE_XFD_MASK);
+            } else if (x86_cpu_xsave_xss_components(cpu) & (1ULL << count)) {
+                *eax = esa->size;
+                *ebx = 0;
+                *ecx = 1;
             }
         }
         break;
@@ -5477,8 +5528,8 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         } else {
             *eax &= env->features[FEAT_SGX_12_1_EAX];
             *ebx &= 0; /* ebx reserve */
-            *ecx &= env->features[FEAT_XSAVE_COMP_LO];
-            *edx &= env->features[FEAT_XSAVE_COMP_HI];
+            *ecx &= env->features[FEAT_XSAVE_XSS_LO];
+            *edx &= env->features[FEAT_XSAVE_XSS_HI];
 
             /* FP and SSE are always allowed regardless of XSAVE/XCR0. */
             *ecx |= XSTATE_FP_MASK | XSTATE_SSE_MASK;
@@ -5874,6 +5925,9 @@ static void x86_cpu_reset(DeviceState *dev)
     }
     for (i = 2; i < ARRAY_SIZE(x86_ext_save_areas); i++) {
         const ExtSaveArea *esa = &x86_ext_save_areas[i];
+        if (!((1 << i) & CPUID_XSTATE_XCR0_MASK)) {
+            continue;
+        }
         if (env->features[esa->feature] & esa->bits) {
             xcr0 |= 1ull << i;
         }
@@ -5988,8 +6042,8 @@ static void x86_cpu_enable_xsave_components(X86CPU *cpu)
     static bool request_perm;
 
     if (!(env->features[FEAT_1_ECX] & CPUID_EXT_XSAVE)) {
-        env->features[FEAT_XSAVE_COMP_LO] = 0;
-        env->features[FEAT_XSAVE_COMP_HI] = 0;
+        env->features[FEAT_XSAVE_XCR0_LO] = 0;
+        env->features[FEAT_XSAVE_XCR0_HI] = 0;
         return;
     }
 
@@ -6007,8 +6061,10 @@ static void x86_cpu_enable_xsave_components(X86CPU *cpu)
         request_perm = true;
     }
 
-    env->features[FEAT_XSAVE_COMP_LO] = mask;
-    env->features[FEAT_XSAVE_COMP_HI] = mask >> 32;
+    env->features[FEAT_XSAVE_XCR0_LO] = mask & CPUID_XSTATE_XCR0_MASK;
+    env->features[FEAT_XSAVE_XCR0_HI] = mask >> 32;
+    env->features[FEAT_XSAVE_XSS_LO] = mask & CPUID_XSTATE_XSS_MASK;
+    env->features[FEAT_XSAVE_XSS_HI] = mask >> 32;
 }
 
 /***** Steps involved on loading and filtering CPUID data
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 6730df5dbf..453b80eae2 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -568,6 +568,14 @@ typedef enum X86Seg {
 #define ESA_FEATURE_XFD_MASK            (1U << ESA_FEATURE_XFD_BIT)
 
 
+/* CPUID feature bits available in XCR0 */
+#define CPUID_XSTATE_XCR0_MASK  (XSTATE_FP_MASK | XSTATE_SSE_MASK | \
+                                 XSTATE_YMM_MASK | XSTATE_BNDREGS_MASK | \
+                                 XSTATE_BNDCSR_MASK | XSTATE_OPMASK_MASK | \
+                                 XSTATE_ZMM_Hi256_MASK | \
+                                 XSTATE_Hi16_ZMM_MASK | XSTATE_PKRU_MASK | \
+                                 XSTATE_XTILE_CFG_MASK | XSTATE_XTILE_DATA_MASK)
+
 /* CPUID feature words */
 typedef enum FeatureWord {
     FEAT_1_EDX,         /* CPUID[1].EDX */
@@ -586,8 +594,8 @@ typedef enum FeatureWord {
     FEAT_SVM,           /* CPUID[8000_000A].EDX */
     FEAT_XSAVE,         /* CPUID[EAX=0xd,ECX=1].EAX */
     FEAT_6_EAX,         /* CPUID[6].EAX */
-    FEAT_XSAVE_COMP_LO, /* CPUID[EAX=0xd,ECX=0].EAX */
-    FEAT_XSAVE_COMP_HI, /* CPUID[EAX=0xd,ECX=0].EDX */
+    FEAT_XSAVE_XCR0_LO, /* CPUID[EAX=0xd,ECX=0].EAX */
+    FEAT_XSAVE_XCR0_HI, /* CPUID[EAX=0xd,ECX=0].EDX */
     FEAT_ARCH_CAPABILITIES,
     FEAT_CORE_CAPABILITY,
     FEAT_PERF_CAPABILITIES,
@@ -604,6 +612,8 @@ typedef enum FeatureWord {
     FEAT_SGX_12_0_EAX,  /* CPUID[EAX=0x12,ECX=0].EAX (SGX) */
     FEAT_SGX_12_0_EBX,  /* CPUID[EAX=0x12,ECX=0].EBX (SGX MISCSELECT[31:0]) */
     FEAT_SGX_12_1_EAX,  /* CPUID[EAX=0x12,ECX=1].EAX (SGX ATTRIBUTES[31:0]) */
+    FEAT_XSAVE_XSS_LO,     /* CPUID[EAX=0xd,ECX=1].ECX */
+    FEAT_XSAVE_XSS_HI,     /* CPUID[EAX=0xd,ECX=1].EDX */
     FEATURE_WORDS,
 } FeatureWord;
 

From 10f0abcb3b8a74a4db1412e844b9192dc9768e94 Mon Sep 17 00:00:00 2001
From: Yang Weijiang <weijiang.yang@intel.com>
Date: Tue, 15 Feb 2022 14:52:55 -0500
Subject: [PATCH 325/935] target/i386: Add XSAVES support for Arch LBR

Define Arch LBR bit in XSS and save/restore structure
for XSAVE area size calculation.

Signed-off-by: Yang Weijiang <weijiang.yang@intel.com>
Message-Id: <20220215195258.29149-6-weijiang.yang@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/cpu.c |  6 +++++-
 target/i386/cpu.h | 23 +++++++++++++++++++++++
 2 files changed, 28 insertions(+), 1 deletion(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 643536d05d..1816c37852 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1411,7 +1411,7 @@ static const X86RegisterInfo32 x86_reg_info_32[CPU_NB_REGS32] = {
 #undef REGISTER
 
 /* CPUID feature bits available in XSS */
-#define CPUID_XSTATE_XSS_MASK    (0)
+#define CPUID_XSTATE_XSS_MASK    (XSTATE_ARCH_LBR_MASK)
 
 ExtSaveArea x86_ext_save_areas[XSAVE_STATE_AREA_COUNT] = {
     [XSTATE_FP_BIT] = {
@@ -1445,6 +1445,10 @@ ExtSaveArea x86_ext_save_areas[XSAVE_STATE_AREA_COUNT] = {
     [XSTATE_PKRU_BIT] =
           { .feature = FEAT_7_0_ECX, .bits = CPUID_7_0_ECX_PKU,
             .size = sizeof(XSavePKRU) },
+    [XSTATE_ARCH_LBR_BIT] = {
+            .feature = FEAT_7_0_EDX, .bits = CPUID_7_0_EDX_ARCH_LBR,
+            .offset = 0 /*supervisor mode component, offset = 0 */,
+            .size = sizeof(XSavesArchLBR) },
     [XSTATE_XTILE_CFG_BIT] = {
         .feature = FEAT_7_0_EDX, .bits = CPUID_7_0_EDX_AMX_TILE,
         .size = sizeof(XSaveXTILECFG),
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 453b80eae2..dba92936a2 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -544,6 +544,7 @@ typedef enum X86Seg {
 #define XSTATE_ZMM_Hi256_BIT            6
 #define XSTATE_Hi16_ZMM_BIT             7
 #define XSTATE_PKRU_BIT                 9
+#define XSTATE_ARCH_LBR_BIT             15
 #define XSTATE_XTILE_CFG_BIT            17
 #define XSTATE_XTILE_DATA_BIT           18
 
@@ -556,6 +557,7 @@ typedef enum X86Seg {
 #define XSTATE_ZMM_Hi256_MASK           (1ULL << XSTATE_ZMM_Hi256_BIT)
 #define XSTATE_Hi16_ZMM_MASK            (1ULL << XSTATE_Hi16_ZMM_BIT)
 #define XSTATE_PKRU_MASK                (1ULL << XSTATE_PKRU_BIT)
+#define XSTATE_ARCH_LBR_MASK            (1ULL << XSTATE_ARCH_LBR_BIT)
 #define XSTATE_XTILE_CFG_MASK           (1ULL << XSTATE_XTILE_CFG_BIT)
 #define XSTATE_XTILE_DATA_MASK          (1ULL << XSTATE_XTILE_DATA_BIT)
 
@@ -870,6 +872,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
 #define CPUID_7_0_EDX_SERIALIZE         (1U << 14)
 /* TSX Suspend Load Address Tracking instruction */
 #define CPUID_7_0_EDX_TSX_LDTRK         (1U << 16)
+/* Architectural LBRs */
+#define CPUID_7_0_EDX_ARCH_LBR          (1U << 19)
 /* AVX512_FP16 instruction */
 #define CPUID_7_0_EDX_AVX512_FP16       (1U << 23)
 /* AMX tile (two-dimensional register) */
@@ -1376,6 +1380,24 @@ typedef struct XSaveXTILEDATA {
     uint8_t xtiledata[8][1024];
 } XSaveXTILEDATA;
 
+typedef struct {
+       uint64_t from;
+       uint64_t to;
+       uint64_t info;
+} LBREntry;
+
+#define ARCH_LBR_NR_ENTRIES            32
+
+/* Ext. save area 19: Supervisor mode Arch LBR state */
+typedef struct XSavesArchLBR {
+    uint64_t lbr_ctl;
+    uint64_t lbr_depth;
+    uint64_t ler_from;
+    uint64_t ler_to;
+    uint64_t ler_info;
+    LBREntry lbr_records[ARCH_LBR_NR_ENTRIES];
+} XSavesArchLBR;
+
 QEMU_BUILD_BUG_ON(sizeof(XSaveAVX) != 0x100);
 QEMU_BUILD_BUG_ON(sizeof(XSaveBNDREG) != 0x40);
 QEMU_BUILD_BUG_ON(sizeof(XSaveBNDCSR) != 0x40);
@@ -1385,6 +1407,7 @@ QEMU_BUILD_BUG_ON(sizeof(XSaveHi16_ZMM) != 0x400);
 QEMU_BUILD_BUG_ON(sizeof(XSavePKRU) != 0x8);
 QEMU_BUILD_BUG_ON(sizeof(XSaveXTILECFG) != 0x40);
 QEMU_BUILD_BUG_ON(sizeof(XSaveXTILEDATA) != 0x2000);
+QEMU_BUILD_BUG_ON(sizeof(XSavesArchLBR) != 0x328);
 
 typedef struct ExtSaveArea {
     uint32_t feature, bits;

From 12703d4e7523a7841764200e1a7838736495da10 Mon Sep 17 00:00:00 2001
From: Yang Weijiang <weijiang.yang@intel.com>
Date: Tue, 15 Feb 2022 14:52:56 -0500
Subject: [PATCH 326/935] target/i386: Add MSR access interface for Arch LBR

In the first generation of Arch LBR, the max support
Arch LBR depth is 32, both host and guest use the value
to set depth MSR. This can simplify the implementation
of patch given the side-effect of mismatch of host/guest
depth MSR: XRSTORS will reset all recording MSRs to 0s
if the saved depth mismatches MSR_ARCH_LBR_DEPTH.

In most of the cases Arch LBR is not in active status,
so check the control bit before save/restore the big
chunck of Arch LBR MSRs.

Signed-off-by: Yang Weijiang <weijiang.yang@intel.com>
Message-Id: <20220215195258.29149-7-weijiang.yang@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/cpu.h     | 10 +++++++
 target/i386/kvm/kvm.c | 67 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 77 insertions(+)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index dba92936a2..0d528ac58f 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -391,6 +391,11 @@ typedef enum X86Seg {
 #define MSR_IA32_TSX_CTRL		0x122
 #define MSR_IA32_TSCDEADLINE            0x6e0
 #define MSR_IA32_PKRS                   0x6e1
+#define MSR_ARCH_LBR_CTL                0x000014ce
+#define MSR_ARCH_LBR_DEPTH              0x000014cf
+#define MSR_ARCH_LBR_FROM_0             0x00001500
+#define MSR_ARCH_LBR_TO_0               0x00001600
+#define MSR_ARCH_LBR_INFO_0             0x00001200
 
 #define FEATURE_CONTROL_LOCKED                    (1<<0)
 #define FEATURE_CONTROL_VMXON_ENABLED_INSIDE_SMX  (1ULL << 1)
@@ -1650,6 +1655,11 @@ typedef struct CPUArchState {
     uint64_t msr_xfd;
     uint64_t msr_xfd_err;
 
+    /* Per-VCPU Arch LBR MSRs */
+    uint64_t msr_lbr_ctl;
+    uint64_t msr_lbr_depth;
+    LBREntry lbr_records[ARCH_LBR_NR_ENTRIES];
+
     /* exception/interrupt handling */
     int error_code;
     int exception_is_int;
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 536cbe5fad..a9ee8eebd7 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -3367,6 +3367,38 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
                               env->msr_xfd_err);
         }
 
+        if (kvm_enabled() && cpu->enable_pmu &&
+            (env->features[FEAT_7_0_EDX] & CPUID_7_0_EDX_ARCH_LBR)) {
+            uint64_t depth;
+            int i, ret;
+
+            /*
+             * Only migrate Arch LBR states when: 1) Arch LBR is enabled
+             * for migrated vcpu. 2) the host Arch LBR depth equals that
+             * of source guest's, this is to avoid mismatch of guest/host
+             * config for the msr hence avoid unexpected misbehavior.
+             */
+            ret = kvm_get_one_msr(cpu, MSR_ARCH_LBR_DEPTH, &depth);
+
+            if (ret == 1 && (env->msr_lbr_ctl & 0x1) && !!depth &&
+                depth == env->msr_lbr_depth) {
+                kvm_msr_entry_add(cpu, MSR_ARCH_LBR_CTL, env->msr_lbr_ctl);
+                kvm_msr_entry_add(cpu, MSR_ARCH_LBR_DEPTH, env->msr_lbr_depth);
+
+                for (i = 0; i < ARCH_LBR_NR_ENTRIES; i++) {
+                    if (!env->lbr_records[i].from) {
+                        continue;
+                    }
+                    kvm_msr_entry_add(cpu, MSR_ARCH_LBR_FROM_0 + i,
+                                      env->lbr_records[i].from);
+                    kvm_msr_entry_add(cpu, MSR_ARCH_LBR_TO_0 + i,
+                                      env->lbr_records[i].to);
+                    kvm_msr_entry_add(cpu, MSR_ARCH_LBR_INFO_0 + i,
+                                      env->lbr_records[i].info);
+                }
+            }
+        }
+
         /* Note: MSR_IA32_FEATURE_CONTROL is written separately, see
          *       kvm_put_msr_feature_control. */
     }
@@ -3767,6 +3799,26 @@ static int kvm_get_msrs(X86CPU *cpu)
         kvm_msr_entry_add(cpu, MSR_IA32_XFD_ERR, 0);
     }
 
+    if (kvm_enabled() && cpu->enable_pmu &&
+        (env->features[FEAT_7_0_EDX] & CPUID_7_0_EDX_ARCH_LBR)) {
+        uint64_t ctl, depth;
+        int i, ret2;
+
+        ret = kvm_get_one_msr(cpu, MSR_ARCH_LBR_CTL, &ctl);
+        ret2 = kvm_get_one_msr(cpu, MSR_ARCH_LBR_DEPTH, &depth);
+        if (ret == 1 && ret2 == 1 && (ctl & 0x1) &&
+            depth == ARCH_LBR_NR_ENTRIES) {
+            kvm_msr_entry_add(cpu, MSR_ARCH_LBR_CTL, 0);
+            kvm_msr_entry_add(cpu, MSR_ARCH_LBR_DEPTH, 0);
+
+            for (i = 0; i < ARCH_LBR_NR_ENTRIES; i++) {
+                kvm_msr_entry_add(cpu, MSR_ARCH_LBR_FROM_0 + i, 0);
+                kvm_msr_entry_add(cpu, MSR_ARCH_LBR_TO_0 + i, 0);
+                kvm_msr_entry_add(cpu, MSR_ARCH_LBR_INFO_0 + i, 0);
+            }
+        }
+    }
+
     ret = kvm_vcpu_ioctl(CPU(cpu), KVM_GET_MSRS, cpu->kvm_msr_buf);
     if (ret < 0) {
         return ret;
@@ -4072,6 +4124,21 @@ static int kvm_get_msrs(X86CPU *cpu)
         case MSR_IA32_XFD_ERR:
             env->msr_xfd_err = msrs[i].data;
             break;
+        case MSR_ARCH_LBR_CTL:
+            env->msr_lbr_ctl = msrs[i].data;
+            break;
+        case MSR_ARCH_LBR_DEPTH:
+            env->msr_lbr_depth = msrs[i].data;
+            break;
+        case MSR_ARCH_LBR_FROM_0 ... MSR_ARCH_LBR_FROM_0 + 31:
+            env->lbr_records[index - MSR_ARCH_LBR_FROM_0].from = msrs[i].data;
+            break;
+        case MSR_ARCH_LBR_TO_0 ... MSR_ARCH_LBR_TO_0 + 31:
+            env->lbr_records[index - MSR_ARCH_LBR_TO_0].to = msrs[i].data;
+            break;
+        case MSR_ARCH_LBR_INFO_0 ... MSR_ARCH_LBR_INFO_0 + 31:
+            env->lbr_records[index - MSR_ARCH_LBR_INFO_0].info = msrs[i].data;
+            break;
         }
     }
 

From f2e7c2fc8945943699f745c405be82ac7e698275 Mon Sep 17 00:00:00 2001
From: Yang Weijiang <weijiang.yang@intel.com>
Date: Tue, 15 Feb 2022 14:52:57 -0500
Subject: [PATCH 327/935] target/i386: Enable Arch LBR migration states in
 vmstate

The Arch LBR record MSRs and control MSRs will be migrated
to destination guest if the vcpus were running with Arch
LBR active.

Signed-off-by: Yang Weijiang <weijiang.yang@intel.com>
Message-Id: <20220215195258.29149-8-weijiang.yang@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/machine.c | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/target/i386/machine.c b/target/i386/machine.c
index 7c54bada81..cecd476e98 100644
--- a/target/i386/machine.c
+++ b/target/i386/machine.c
@@ -136,6 +136,22 @@ static const VMStateDescription vmstate_mtrr_var = {
 #define VMSTATE_MTRR_VARS(_field, _state, _n, _v)                    \
     VMSTATE_STRUCT_ARRAY(_field, _state, _n, _v, vmstate_mtrr_var, MTRRVar)
 
+static const VMStateDescription vmstate_lbr_records_var = {
+    .name = "lbr_records_var",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT64(from, LBREntry),
+        VMSTATE_UINT64(to, LBREntry),
+        VMSTATE_UINT64(info, LBREntry),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+#define VMSTATE_LBR_VARS(_field, _state, _n, _v)                    \
+    VMSTATE_STRUCT_ARRAY(_field, _state, _n, _v, vmstate_lbr_records_var, \
+                         LBREntry)
+
 typedef struct x86_FPReg_tmp {
     FPReg *parent;
     uint64_t tmp_mant;
@@ -1525,6 +1541,27 @@ static const VMStateDescription vmstate_amx_xtile = {
 };
 #endif
 
+static bool arch_lbr_needed(void *opaque)
+{
+    X86CPU *cpu = opaque;
+    CPUX86State *env = &cpu->env;
+
+    return !!(env->features[FEAT_7_0_EDX] & CPUID_7_0_EDX_ARCH_LBR);
+}
+
+static const VMStateDescription vmstate_arch_lbr = {
+    .name = "cpu/arch_lbr",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .needed = arch_lbr_needed,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT64(env.msr_lbr_ctl, X86CPU),
+        VMSTATE_UINT64(env.msr_lbr_depth, X86CPU),
+        VMSTATE_LBR_VARS(env.lbr_records, X86CPU, ARCH_LBR_NR_ENTRIES, 1),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 const VMStateDescription vmstate_x86_cpu = {
     .name = "cpu",
     .version_id = 12,
@@ -1668,6 +1705,7 @@ const VMStateDescription vmstate_x86_cpu = {
 #ifdef TARGET_X86_64
         &vmstate_amx_xtile,
 #endif
+        &vmstate_arch_lbr,
         NULL
     }
 };

From d19d6ffa07100f5015dc1c708d6c811354a13d7f Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 29 Apr 2022 20:52:52 +0200
Subject: [PATCH 328/935] target/i386: introduce helper to access supported
 CPUID

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/cpu.c | 41 +++++++++++++++++++++++++----------------
 1 file changed, 25 insertions(+), 16 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 1816c37852..dcc770ecea 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -4944,6 +4944,28 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
     return r;
 }
 
+static void x86_cpu_get_supported_cpuid(uint32_t func, uint32_t index,
+                                        uint32_t *eax, uint32_t *ebx,
+                                        uint32_t *ecx, uint32_t *edx)
+{
+    if (kvm_enabled()) {
+        *eax = kvm_arch_get_supported_cpuid(kvm_state, func, index, R_EAX);
+        *ebx = kvm_arch_get_supported_cpuid(kvm_state, func, index, R_EBX);
+        *ecx = kvm_arch_get_supported_cpuid(kvm_state, func, index, R_ECX);
+        *edx = kvm_arch_get_supported_cpuid(kvm_state, func, index, R_EDX);
+    } else if (hvf_enabled()) {
+        *eax = hvf_get_supported_cpuid(func, index, R_EAX);
+        *ebx = hvf_get_supported_cpuid(func, index, R_EBX);
+        *ecx = hvf_get_supported_cpuid(func, index, R_ECX);
+        *edx = hvf_get_supported_cpuid(func, index, R_EDX);
+    } else {
+        *eax = 0;
+        *ebx = 0;
+        *ecx = 0;
+        *edx = 0;
+    }
+}
+
 static void x86_cpu_get_cache_cpuid(uint32_t func, uint32_t index,
                                     uint32_t *eax, uint32_t *ebx,
                                     uint32_t *ecx, uint32_t *edx)
@@ -5359,18 +5381,8 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         break;
     case 0xA:
         /* Architectural Performance Monitoring Leaf */
-        if (kvm_enabled() && cpu->enable_pmu) {
-            KVMState *s = cs->kvm_state;
-
-            *eax = kvm_arch_get_supported_cpuid(s, 0xA, count, R_EAX);
-            *ebx = kvm_arch_get_supported_cpuid(s, 0xA, count, R_EBX);
-            *ecx = kvm_arch_get_supported_cpuid(s, 0xA, count, R_ECX);
-            *edx = kvm_arch_get_supported_cpuid(s, 0xA, count, R_EDX);
-        } else if (hvf_enabled() && cpu->enable_pmu) {
-            *eax = hvf_get_supported_cpuid(0xA, count, R_EAX);
-            *ebx = hvf_get_supported_cpuid(0xA, count, R_EBX);
-            *ecx = hvf_get_supported_cpuid(0xA, count, R_ECX);
-            *edx = hvf_get_supported_cpuid(0xA, count, R_EDX);
+        if (accel_uses_host_cpuid() && cpu->enable_pmu) {
+            x86_cpu_get_supported_cpuid(0xA, count, eax, ebx, ecx, edx);
         } else {
             *eax = 0;
             *ebx = 0;
@@ -5521,10 +5533,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
          * supports.  Features can be further restricted by userspace, but not
          * made more permissive.
          */
-        *eax = kvm_arch_get_supported_cpuid(cs->kvm_state, 0x12, count, R_EAX);
-        *ebx = kvm_arch_get_supported_cpuid(cs->kvm_state, 0x12, count, R_EBX);
-        *ecx = kvm_arch_get_supported_cpuid(cs->kvm_state, 0x12, count, R_ECX);
-        *edx = kvm_arch_get_supported_cpuid(cs->kvm_state, 0x12, count, R_EDX);
+        x86_cpu_get_supported_cpuid(0x12, index, eax, ebx, ecx, edx);
 
         if (count == 0) {
             *eax &= env->features[FEAT_SGX_12_0_EAX];

From c3c67679f65903b7d1fe25da8fc4e163878ab2b9 Mon Sep 17 00:00:00 2001
From: Yang Weijiang <weijiang.yang@intel.com>
Date: Tue, 15 Feb 2022 14:52:58 -0500
Subject: [PATCH 329/935] target/i386: Support Arch LBR in CPUID enumeration

If CPUID.(EAX=07H, ECX=0):EDX[19] is set to 1, the processor
supports Architectural LBRs. In this case, CPUID leaf 01CH
indicates details of the Architectural LBRs capabilities.
XSAVE support for Architectural LBRs is enumerated in
CPUID.(EAX=0DH, ECX=0FH).

Signed-off-by: Yang Weijiang <weijiang.yang@intel.com>
Message-Id: <20220215195258.29149-9-weijiang.yang@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/cpu.c | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index dcc770ecea..35c3475e6c 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -855,7 +855,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
             "fsrm", NULL, NULL, NULL,
             "avx512-vp2intersect", NULL, "md-clear", NULL,
             NULL, NULL, "serialize", NULL,
-            "tsx-ldtrk", NULL, NULL /* pconfig */, NULL,
+            "tsx-ldtrk", NULL, NULL /* pconfig */, "arch-lbr",
             NULL, NULL, "amx-bf16", "avx512-fp16",
             "amx-tile", "amx-int8", "spec-ctrl", "stibp",
             NULL, "arch-capabilities", "core-capability", "ssbd",
@@ -5420,6 +5420,13 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         assert(!(*eax & ~0x1f));
         *ebx &= 0xffff; /* The count doesn't need to be reliable. */
         break;
+    case 0x1C:
+        if (accel_uses_host_cpuid() && cpu->enable_pmu &&
+            (env->features[FEAT_7_0_EDX] & CPUID_7_0_EDX_ARCH_LBR)) {
+            x86_cpu_get_supported_cpuid(0x1C, 0, eax, ebx, ecx, edx);
+            *edx = 0;
+        }
+        break;
     case 0x1F:
         /* V2 Extended Topology Enumeration Leaf */
         if (env->nr_dies < 2) {
@@ -5482,6 +5489,17 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
             *ebx = xsave_area_size(xstate, true);
             *ecx = env->features[FEAT_XSAVE_XSS_LO];
             *edx = env->features[FEAT_XSAVE_XSS_HI];
+            if (kvm_enabled() && cpu->enable_pmu &&
+                (env->features[FEAT_7_0_EDX] & CPUID_7_0_EDX_ARCH_LBR) &&
+                (*eax & CPUID_XSAVE_XSAVES)) {
+                *ecx |= XSTATE_ARCH_LBR_MASK;
+            } else {
+                *ecx &= ~XSTATE_ARCH_LBR_MASK;
+            }
+        } else if (count == 0xf &&
+                   accel_uses_host_cpuid() && cpu->enable_pmu &&
+                   (env->features[FEAT_7_0_EDX] & CPUID_7_0_EDX_ARCH_LBR)) {
+            x86_cpu_get_supported_cpuid(0xD, count, eax, ebx, ecx, edx);
         } else if (count < ARRAY_SIZE(x86_ext_save_areas)) {
             const ExtSaveArea *esa = &x86_ext_save_areas[count];
 

From 0310641c06dd5f7ea031b2b6170cb2edc63e4cea Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Mon, 9 May 2022 11:42:23 +0200
Subject: [PATCH 330/935] crypto: make loaded property read-only

The ``loaded=on`` option in the command line or QMP ``object-add`` either had
no effect (if ``loaded`` was the last option) or caused options to be
effectively ignored as if they were not given.  The property is therefore
useless and was deprecated in 6.0; make it read-only now.

The patch is best reviewed with "-b".

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 crypto/secret_common.c          | 84 ++++++++++++++-------------------
 crypto/tlscredsanon.c           | 20 ++------
 crypto/tlscredspsk.c            | 20 ++------
 crypto/tlscredsx509.c           | 20 ++------
 docs/about/deprecated.rst       | 10 ----
 docs/about/removed-features.rst |  9 ++++
 6 files changed, 56 insertions(+), 107 deletions(-)

diff --git a/crypto/secret_common.c b/crypto/secret_common.c
index 714a15d5e5..3441c44ca8 100644
--- a/crypto/secret_common.c
+++ b/crypto/secret_common.c
@@ -138,36 +138,44 @@ static void qcrypto_secret_decode(const uint8_t *input,
 
 
 static void
-qcrypto_secret_prop_set_loaded(Object *obj,
-                               bool value,
-                               Error **errp)
+qcrypto_secret_complete(UserCreatable *uc, Error **errp)
 {
-    QCryptoSecretCommon *secret = QCRYPTO_SECRET_COMMON(obj);
+    QCryptoSecretCommon *secret = QCRYPTO_SECRET_COMMON(uc);
     QCryptoSecretCommonClass *sec_class
-                                = QCRYPTO_SECRET_COMMON_GET_CLASS(obj);
+                                = QCRYPTO_SECRET_COMMON_GET_CLASS(uc);
 
-    if (value) {
-        Error *local_err = NULL;
-        uint8_t *input = NULL;
-        size_t inputlen = 0;
-        uint8_t *output = NULL;
-        size_t outputlen = 0;
+    Error *local_err = NULL;
+    uint8_t *input = NULL;
+    size_t inputlen = 0;
+    uint8_t *output = NULL;
+    size_t outputlen = 0;
 
-        if (sec_class->load_data) {
-            sec_class->load_data(secret, &input, &inputlen, &local_err);
-            if (local_err) {
-                error_propagate(errp, local_err);
-                return;
-            }
-        } else {
-            error_setg(errp, "%s provides no 'load_data' method'",
-                             object_get_typename(obj));
+    if (sec_class->load_data) {
+        sec_class->load_data(secret, &input, &inputlen, &local_err);
+        if (local_err) {
+            error_propagate(errp, local_err);
             return;
         }
+    } else {
+        error_setg(errp, "%s provides no 'load_data' method'",
+                         object_get_typename(OBJECT(uc)));
+        return;
+    }
 
-        if (secret->keyid) {
-            qcrypto_secret_decrypt(secret, input, inputlen,
-                                   &output, &outputlen, &local_err);
+    if (secret->keyid) {
+        qcrypto_secret_decrypt(secret, input, inputlen,
+                               &output, &outputlen, &local_err);
+        g_free(input);
+        if (local_err) {
+            error_propagate(errp, local_err);
+            return;
+        }
+        input = output;
+        inputlen = outputlen;
+    } else {
+        if (secret->format == QCRYPTO_SECRET_FORMAT_BASE64) {
+            qcrypto_secret_decode(input, inputlen,
+                                  &output, &outputlen, &local_err);
             g_free(input);
             if (local_err) {
                 error_propagate(errp, local_err);
@@ -175,26 +183,11 @@ qcrypto_secret_prop_set_loaded(Object *obj,
             }
             input = output;
             inputlen = outputlen;
-        } else {
-            if (secret->format == QCRYPTO_SECRET_FORMAT_BASE64) {
-                qcrypto_secret_decode(input, inputlen,
-                                      &output, &outputlen, &local_err);
-                g_free(input);
-                if (local_err) {
-                    error_propagate(errp, local_err);
-                    return;
-                }
-                input = output;
-                inputlen = outputlen;
-            }
         }
-
-        secret->rawdata = input;
-        secret->rawlen = inputlen;
-    } else if (secret->rawdata) {
-        error_setg(errp, "Cannot unload secret");
-        return;
     }
+
+    secret->rawdata = input;
+    secret->rawlen = inputlen;
 }
 
 
@@ -268,13 +261,6 @@ qcrypto_secret_prop_get_keyid(Object *obj,
 }
 
 
-static void
-qcrypto_secret_complete(UserCreatable *uc, Error **errp)
-{
-    object_property_set_bool(OBJECT(uc), "loaded", true, errp);
-}
-
-
 static void
 qcrypto_secret_finalize(Object *obj)
 {
@@ -294,7 +280,7 @@ qcrypto_secret_class_init(ObjectClass *oc, void *data)
 
     object_class_property_add_bool(oc, "loaded",
                                    qcrypto_secret_prop_get_loaded,
-                                   qcrypto_secret_prop_set_loaded);
+                                   NULL);
     object_class_property_add_enum(oc, "format",
                                    "QCryptoSecretFormat",
                                    &QCryptoSecretFormat_lookup,
diff --git a/crypto/tlscredsanon.c b/crypto/tlscredsanon.c
index 6fb83639ec..c0d23a0ef3 100644
--- a/crypto/tlscredsanon.c
+++ b/crypto/tlscredsanon.c
@@ -119,16 +119,11 @@ qcrypto_tls_creds_anon_unload(QCryptoTLSCredsAnon *creds G_GNUC_UNUSED)
 
 
 static void
-qcrypto_tls_creds_anon_prop_set_loaded(Object *obj,
-                                       bool value,
-                                       Error **errp)
+qcrypto_tls_creds_anon_complete(UserCreatable *uc, Error **errp)
 {
-    QCryptoTLSCredsAnon *creds = QCRYPTO_TLS_CREDS_ANON(obj);
+    QCryptoTLSCredsAnon *creds = QCRYPTO_TLS_CREDS_ANON(uc);
 
-    qcrypto_tls_creds_anon_unload(creds);
-    if (value) {
-        qcrypto_tls_creds_anon_load(creds, errp);
-    }
+    qcrypto_tls_creds_anon_load(creds, errp);
 }
 
 
@@ -163,13 +158,6 @@ qcrypto_tls_creds_anon_prop_get_loaded(Object *obj G_GNUC_UNUSED,
 #endif /* ! CONFIG_GNUTLS */
 
 
-static void
-qcrypto_tls_creds_anon_complete(UserCreatable *uc, Error **errp)
-{
-    object_property_set_bool(OBJECT(uc), "loaded", true, errp);
-}
-
-
 static void
 qcrypto_tls_creds_anon_finalize(Object *obj)
 {
@@ -188,7 +176,7 @@ qcrypto_tls_creds_anon_class_init(ObjectClass *oc, void *data)
 
     object_class_property_add_bool(oc, "loaded",
                                    qcrypto_tls_creds_anon_prop_get_loaded,
-                                   qcrypto_tls_creds_anon_prop_set_loaded);
+                                   NULL);
 }
 
 
diff --git a/crypto/tlscredspsk.c b/crypto/tlscredspsk.c
index 752f2d92be..a4f9891274 100644
--- a/crypto/tlscredspsk.c
+++ b/crypto/tlscredspsk.c
@@ -188,16 +188,11 @@ qcrypto_tls_creds_psk_unload(QCryptoTLSCredsPSK *creds G_GNUC_UNUSED)
 
 
 static void
-qcrypto_tls_creds_psk_prop_set_loaded(Object *obj,
-                                      bool value,
-                                      Error **errp)
+qcrypto_tls_creds_psk_complete(UserCreatable *uc, Error **errp)
 {
-    QCryptoTLSCredsPSK *creds = QCRYPTO_TLS_CREDS_PSK(obj);
+    QCryptoTLSCredsPSK *creds = QCRYPTO_TLS_CREDS_PSK(uc);
 
-    qcrypto_tls_creds_psk_unload(creds);
-    if (value) {
-        qcrypto_tls_creds_psk_load(creds, errp);
-    }
+    qcrypto_tls_creds_psk_load(creds, errp);
 }
 
 
@@ -232,13 +227,6 @@ qcrypto_tls_creds_psk_prop_get_loaded(Object *obj G_GNUC_UNUSED,
 #endif /* ! CONFIG_GNUTLS */
 
 
-static void
-qcrypto_tls_creds_psk_complete(UserCreatable *uc, Error **errp)
-{
-    object_property_set_bool(OBJECT(uc), "loaded", true, errp);
-}
-
-
 static void
 qcrypto_tls_creds_psk_finalize(Object *obj)
 {
@@ -276,7 +264,7 @@ qcrypto_tls_creds_psk_class_init(ObjectClass *oc, void *data)
 
     object_class_property_add_bool(oc, "loaded",
                                    qcrypto_tls_creds_psk_prop_get_loaded,
-                                   qcrypto_tls_creds_psk_prop_set_loaded);
+                                   NULL);
     object_class_property_add_str(oc, "username",
                                   qcrypto_tls_creds_psk_prop_get_username,
                                   qcrypto_tls_creds_psk_prop_set_username);
diff --git a/crypto/tlscredsx509.c b/crypto/tlscredsx509.c
index 32948a6bdc..d14313925d 100644
--- a/crypto/tlscredsx509.c
+++ b/crypto/tlscredsx509.c
@@ -687,16 +687,11 @@ qcrypto_tls_creds_x509_unload(QCryptoTLSCredsX509 *creds G_GNUC_UNUSED)
 
 
 static void
-qcrypto_tls_creds_x509_prop_set_loaded(Object *obj,
-                                       bool value,
-                                       Error **errp)
+qcrypto_tls_creds_x509_complete(UserCreatable *uc, Error **errp)
 {
-    QCryptoTLSCredsX509 *creds = QCRYPTO_TLS_CREDS_X509(obj);
+    QCryptoTLSCredsX509 *creds = QCRYPTO_TLS_CREDS_X509(uc);
 
-    qcrypto_tls_creds_x509_unload(creds);
-    if (value) {
-        qcrypto_tls_creds_x509_load(creds, errp);
-    }
+    qcrypto_tls_creds_x509_load(creds, errp);
 }
 
 
@@ -814,13 +809,6 @@ qcrypto_tls_creds_x509_reload(QCryptoTLSCreds *creds, Error **errp)
 #endif /* ! CONFIG_GNUTLS */
 
 
-static void
-qcrypto_tls_creds_x509_complete(UserCreatable *uc, Error **errp)
-{
-    object_property_set_bool(OBJECT(uc), "loaded", true, errp);
-}
-
-
 static void
 qcrypto_tls_creds_x509_init(Object *obj)
 {
@@ -852,7 +840,7 @@ qcrypto_tls_creds_x509_class_init(ObjectClass *oc, void *data)
 
     object_class_property_add_bool(oc, "loaded",
                                    qcrypto_tls_creds_x509_prop_get_loaded,
-                                   qcrypto_tls_creds_x509_prop_set_loaded);
+                                   NULL);
     object_class_property_add_bool(oc, "sanity-check",
                                    qcrypto_tls_creds_x509_prop_get_sanity,
                                    qcrypto_tls_creds_x509_prop_set_sanity);
diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst
index 44f52c172d..2a022f2300 100644
--- a/docs/about/deprecated.rst
+++ b/docs/about/deprecated.rst
@@ -99,16 +99,6 @@ other options have been processed.  This will either have no effect (if
 ``opened`` was the last option) or cause errors.  The property is therefore
 useless and should not be specified.
 
-``loaded`` property of ``secret`` and ``secret_keyring`` objects (since 6.0)
-''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
-
-The only effect of specifying ``loaded=on`` in the command line or QMP
-``object-add`` is that the secret is loaded immediately, possibly before all
-other options have been processed.  This will either have no effect (if
-``loaded`` was the last option) or cause options to be effectively ignored as
-if they were not given.  The property is therefore useless and should not be
-specified.
-
 ``-display sdl,window_close=...`` (since 6.1)
 '''''''''''''''''''''''''''''''''''''''''''''
 
diff --git a/docs/about/removed-features.rst b/docs/about/removed-features.rst
index ba78bc1a19..81ceeb9000 100644
--- a/docs/about/removed-features.rst
+++ b/docs/about/removed-features.rst
@@ -355,6 +355,15 @@ The ``-writeconfig`` option was not able to serialize the entire contents
 of the QEMU command line.  It is thus considered a failed experiment
 and removed without a replacement.
 
+``loaded`` property of ``secret`` and ``secret_keyring`` objects (removed in 7.1)
+'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
+
+The ``loaded=on`` option in the command line or QMP ``object-add`` either had
+no effect (if ``loaded`` was the last option) or caused options to be
+effectively ignored as if they were not given.  The property is therefore
+useless and should simply be removed.
+
+
 QEMU Machine Protocol (QMP) commands
 ------------------------------------
 

From 6e577937485080f2dcebc6a3e5a4a9e8db841762 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Mon, 9 May 2022 11:46:03 +0200
Subject: [PATCH 331/935] rng: make opened property read-only

The ``opened=on`` option in the command line or QMP ``object-add`` either had
no effect (if ``opened`` was the last option) or caused errors.  The property
is therefore useless and was deprecated in 6.0; make it read-only now.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 backends/rng.c                  | 18 ++----------------
 docs/about/deprecated.rst       |  9 ---------
 docs/about/removed-features.rst |  6 ++++++
 3 files changed, 8 insertions(+), 25 deletions(-)

diff --git a/backends/rng.c b/backends/rng.c
index 3757b04485..6c7bf64426 100644
--- a/backends/rng.c
+++ b/backends/rng.c
@@ -48,24 +48,10 @@ static bool rng_backend_prop_get_opened(Object *obj, Error **errp)
 
 static void rng_backend_complete(UserCreatable *uc, Error **errp)
 {
-    object_property_set_bool(OBJECT(uc), "opened", true, errp);
-}
-
-static void rng_backend_prop_set_opened(Object *obj, bool value, Error **errp)
-{
-    RngBackend *s = RNG_BACKEND(obj);
+    RngBackend *s = RNG_BACKEND(uc);
     RngBackendClass *k = RNG_BACKEND_GET_CLASS(s);
     Error *local_err = NULL;
 
-    if (value == s->opened) {
-        return;
-    }
-
-    if (!value && s->opened) {
-        error_setg(errp, QERR_PERMISSION_DENIED);
-        return;
-    }
-
     if (k->opened) {
         k->opened(s, &local_err);
         if (local_err) {
@@ -122,7 +108,7 @@ static void rng_backend_class_init(ObjectClass *oc, void *data)
 
     object_class_property_add_bool(oc, "opened",
                                    rng_backend_prop_get_opened,
-                                   rng_backend_prop_set_opened);
+                                   NULL);
 }
 
 static const TypeInfo rng_backend_info = {
diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst
index 2a022f2300..1f961b9e6e 100644
--- a/docs/about/deprecated.rst
+++ b/docs/about/deprecated.rst
@@ -90,15 +90,6 @@ the process listing. This is replaced by the new ``password-secret``
 option which lets the password be securely provided on the command
 line using a ``secret`` object instance.
 
-``opened`` property of ``rng-*`` objects (since 6.0)
-''''''''''''''''''''''''''''''''''''''''''''''''''''
-
-The only effect of specifying ``opened=on`` in the command line or QMP
-``object-add`` is that the device is opened immediately, possibly before all
-other options have been processed.  This will either have no effect (if
-``opened`` was the last option) or cause errors.  The property is therefore
-useless and should not be specified.
-
 ``-display sdl,window_close=...`` (since 6.1)
 '''''''''''''''''''''''''''''''''''''''''''''
 
diff --git a/docs/about/removed-features.rst b/docs/about/removed-features.rst
index 81ceeb9000..69498efd51 100644
--- a/docs/about/removed-features.rst
+++ b/docs/about/removed-features.rst
@@ -363,6 +363,12 @@ no effect (if ``loaded`` was the last option) or caused options to be
 effectively ignored as if they were not given.  The property is therefore
 useless and should simply be removed.
 
+``opened`` property of ``rng-*`` objects (removed in 7.1)
+'''''''''''''''''''''''''''''''''''''''''''''''''''''''''
+
+The ``opened=on`` option in the command line or QMP ``object-add`` either had
+no effect (if ``opened`` was the last option) or caused errors.  The property
+is therefore useless and should simply be removed.
 
 QEMU Machine Protocol (QMP) commands
 ------------------------------------

From 9c50b8aae2b411b253edaee77d76d297e5fa2fab Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 27 Apr 2022 12:04:47 +0200
Subject: [PATCH 332/935] soundhw: remove ability to create multiple soundcards

The usefulness of enabling a dozen soundcards is dubious.  Simplify the
code by allowing a single instance of -soundhw, with no support for
parsing either comma-separated values or 'soundhw all'.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/audio/soundhw.c | 88 +++++++++++++++++-----------------------------
 1 file changed, 32 insertions(+), 56 deletions(-)

diff --git a/hw/audio/soundhw.c b/hw/audio/soundhw.c
index f7d94d7dfa..097501fee1 100644
--- a/hw/audio/soundhw.c
+++ b/hw/audio/soundhw.c
@@ -25,6 +25,7 @@
 #include "qemu/option.h"
 #include "qemu/help_option.h"
 #include "qemu/error-report.h"
+#include "qapi/error.h"
 #include "qom/object.h"
 #include "hw/isa/isa.h"
 #include "hw/pci/pci.h"
@@ -34,7 +35,6 @@ struct soundhw {
     const char *name;
     const char *descr;
     const char *typename;
-    int enabled;
     int isa;
     int (*init_pci) (PCIBus *bus);
 };
@@ -64,10 +64,16 @@ void deprecated_register_soundhw(const char *name, const char *descr,
     soundhw_count++;
 }
 
+static struct soundhw *selected = NULL;
+
 void select_soundhw(const char *optarg)
 {
     struct soundhw *c;
 
+    if (selected) {
+        error_setg(&error_fatal, "only one -soundhw option is allowed");
+    }
+
     if (is_help_option(optarg)) {
     show_valid_cards:
 
@@ -84,44 +90,15 @@ void select_soundhw(const char *optarg)
         exit(!is_help_option(optarg));
     }
     else {
-        size_t l;
-        const char *p;
-        char *e;
-        int bad_card = 0;
-
-        if (!strcmp(optarg, "all")) {
-            for (c = soundhw; c->name; ++c) {
-                c->enabled = 1;
+        for (c = soundhw; c->name; ++c) {
+            if (g_str_equal(c->name, optarg)) {
+                selected = c;
+                break;
             }
-            return;
         }
 
-        p = optarg;
-        while (*p) {
-            e = strchr(p, ',');
-            l = !e ? strlen(p) : (size_t) (e - p);
-
-            for (c = soundhw; c->name; ++c) {
-                if (!strncmp(c->name, p, l) && !c->name[l]) {
-                    c->enabled = 1;
-                    break;
-                }
-            }
-
-            if (!c->name) {
-                if (l > 80) {
-                    error_report("Unknown sound card name (too big to show)");
-                }
-                else {
-                    error_report("Unknown sound card name `%.*s'",
-                                 (int) l, p);
-                }
-                bad_card = 1;
-            }
-            p += l + (e != NULL);
-        }
-
-        if (bad_card) {
+        if (!c->name) {
+            error_report("Unknown sound card name `%s'", optarg);
             goto show_valid_cards;
         }
     }
@@ -129,30 +106,29 @@ void select_soundhw(const char *optarg)
 
 void soundhw_init(void)
 {
-    struct soundhw *c;
+    struct soundhw *c = selected;
     ISABus *isa_bus = (ISABus *) object_resolve_path_type("", TYPE_ISA_BUS, NULL);
     PCIBus *pci_bus = (PCIBus *) object_resolve_path_type("", TYPE_PCI_BUS, NULL);
 
-    for (c = soundhw; c->name; ++c) {
-        if (c->enabled) {
-            if (c->typename) {
-                warn_report("'-soundhw %s' is deprecated, "
-                            "please use '-device %s' instead",
-                            c->name, c->typename);
-                if (c->isa) {
-                    isa_create_simple(isa_bus, c->typename);
-                } else {
-                    pci_create_simple(pci_bus, -1, c->typename);
-                }
-            } else {
-                assert(!c->isa);
-                if (!pci_bus) {
-                    error_report("PCI bus not available for %s", c->name);
-                    exit(1);
-                }
-                c->init_pci(pci_bus);
-            }
+    if (!c) {
+        return;
+    }
+    if (c->typename) {
+        warn_report("'-soundhw %s' is deprecated, "
+                    "please use '-device %s' instead",
+                    c->name, c->typename);
+        if (c->isa) {
+            isa_create_simple(isa_bus, c->typename);
+        } else {
+            pci_create_simple(pci_bus, -1, c->typename);
         }
+    } else {
+        assert(!c->isa);
+        if (!pci_bus) {
+            error_report("PCI bus not available for %s", c->name);
+            exit(1);
+        }
+        c->init_pci(pci_bus);
     }
 }
 

From eef5fdf3d57dccc25505179f717adf636764ba02 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 27 Apr 2022 12:26:48 +0200
Subject: [PATCH 333/935] soundhw: extract soundhw help to a separate function

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/audio/soundhw.c         | 33 +++++++++++++++++++--------------
 include/hw/audio/soundhw.h |  1 +
 2 files changed, 20 insertions(+), 14 deletions(-)

diff --git a/hw/audio/soundhw.c b/hw/audio/soundhw.c
index 097501fee1..0fb64bdc8f 100644
--- a/hw/audio/soundhw.c
+++ b/hw/audio/soundhw.c
@@ -64,6 +64,21 @@ void deprecated_register_soundhw(const char *name, const char *descr,
     soundhw_count++;
 }
 
+void show_valid_soundhw(void)
+{
+    struct soundhw *c;
+
+    if (soundhw_count) {
+         printf("Valid sound card names (comma separated):\n");
+         for (c = soundhw; c->name; ++c) {
+             printf ("%-11s %s\n", c->name, c->descr);
+         }
+    } else {
+         printf("Machine has no user-selectable audio hardware "
+                "(it may or may not have always-present audio hardware).\n");
+    }
+}
+
 static struct soundhw *selected = NULL;
 
 void select_soundhw(const char *optarg)
@@ -75,19 +90,8 @@ void select_soundhw(const char *optarg)
     }
 
     if (is_help_option(optarg)) {
-    show_valid_cards:
-
-        if (soundhw_count) {
-             printf("Valid sound card names (comma separated):\n");
-             for (c = soundhw; c->name; ++c) {
-                 printf ("%-11s %s\n", c->name, c->descr);
-             }
-             printf("\n-soundhw all will enable all of the above\n");
-        } else {
-             printf("Machine has no user-selectable audio hardware "
-                    "(it may or may not have always-present audio hardware).\n");
-        }
-        exit(!is_help_option(optarg));
+        show_valid_soundhw();
+        exit(0);
     }
     else {
         for (c = soundhw; c->name; ++c) {
@@ -99,7 +103,8 @@ void select_soundhw(const char *optarg)
 
         if (!c->name) {
             error_report("Unknown sound card name `%s'", optarg);
-            goto show_valid_cards;
+            show_valid_soundhw();
+            exit(1);
         }
     }
 }
diff --git a/include/hw/audio/soundhw.h b/include/hw/audio/soundhw.h
index e68685fcda..dec5c0cdca 100644
--- a/include/hw/audio/soundhw.h
+++ b/include/hw/audio/soundhw.h
@@ -7,6 +7,7 @@ void deprecated_register_soundhw(const char *name, const char *descr,
                                  int isa, const char *typename);
 
 void soundhw_init(void);
+void show_valid_soundhw(void);
 void select_soundhw(const char *optarg);
 
 #endif

From bf521c5655f7f821603d921e6de77e4e05fa44b9 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 27 Apr 2022 12:06:59 +0200
Subject: [PATCH 334/935] soundhw: unify initialization for ISA and PCI soundhw

Use qdev_new instead of distinguishing isa_create_simple/pci_create_simple.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/audio/soundhw.c | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/hw/audio/soundhw.c b/hw/audio/soundhw.c
index 0fb64bdc8f..a1558dab3a 100644
--- a/hw/audio/soundhw.c
+++ b/hw/audio/soundhw.c
@@ -114,25 +114,30 @@ void soundhw_init(void)
     struct soundhw *c = selected;
     ISABus *isa_bus = (ISABus *) object_resolve_path_type("", TYPE_ISA_BUS, NULL);
     PCIBus *pci_bus = (PCIBus *) object_resolve_path_type("", TYPE_PCI_BUS, NULL);
+    BusState *bus;
 
     if (!c) {
         return;
     }
-    if (c->typename) {
-        warn_report("'-soundhw %s' is deprecated, "
-                    "please use '-device %s' instead",
-                    c->name, c->typename);
-        if (c->isa) {
-            isa_create_simple(isa_bus, c->typename);
-        } else {
-            pci_create_simple(pci_bus, -1, c->typename);
+    if (c->isa) {
+        if (!isa_bus) {
+            error_report("ISA bus not available for %s", c->name);
+            exit(1);
         }
+        bus = BUS(isa_bus);
     } else {
-        assert(!c->isa);
         if (!pci_bus) {
             error_report("PCI bus not available for %s", c->name);
             exit(1);
         }
+        bus = BUS(pci_bus);
+    }
+
+    if (c->typename) {
+        DeviceState *dev = qdev_new(c->typename);
+        qdev_realize_and_unref(dev, bus, &error_fatal);
+    } else {
+        assert(!c->isa);
         c->init_pci(pci_bus);
     }
 }

From 67aaa96ae451913ffd25766dc59341fe6b63619d Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 27 Apr 2022 13:15:23 +0200
Subject: [PATCH 335/935] soundhw: move help handling to vl.c

This will allow processing "-audio model=help" even if the backend
part of the option is missing.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/audio/soundhw.c | 24 +++++++++---------------
 softmmu/vl.c       |  4 ++++
 2 files changed, 13 insertions(+), 15 deletions(-)

diff --git a/hw/audio/soundhw.c b/hw/audio/soundhw.c
index a1558dab3a..ebbd0958ba 100644
--- a/hw/audio/soundhw.c
+++ b/hw/audio/soundhw.c
@@ -89,23 +89,17 @@ void select_soundhw(const char *optarg)
         error_setg(&error_fatal, "only one -soundhw option is allowed");
     }
 
-    if (is_help_option(optarg)) {
-        show_valid_soundhw();
-        exit(0);
+    for (c = soundhw; c->name; ++c) {
+        if (g_str_equal(c->name, optarg)) {
+            selected = c;
+            break;
+        }
     }
-    else {
-        for (c = soundhw; c->name; ++c) {
-            if (g_str_equal(c->name, optarg)) {
-                selected = c;
-                break;
-            }
-        }
 
-        if (!c->name) {
-            error_report("Unknown sound card name `%s'", optarg);
-            show_valid_soundhw();
-            exit(1);
-        }
+    if (!c->name) {
+        error_report("Unknown sound card name `%s'", optarg);
+        show_valid_soundhw();
+        exit(1);
     }
 }
 
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 817d515783..0363cbd9b4 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -2931,6 +2931,10 @@ void qemu_init(int argc, char **argv, char **envp)
                 audio_parse_option(optarg);
                 break;
             case QEMU_OPTION_soundhw:
+                if (is_help_option(optarg)) {
+                    show_valid_soundhw();
+                    exit(0);
+                }
                 select_soundhw (optarg);
                 break;
             case QEMU_OPTION_h:

From 039a68373c4544ff94871f945a733928b6dcfe93 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Wed, 27 Apr 2022 12:27:46 +0200
Subject: [PATCH 336/935] introduce -audio as a replacement for -soundhw

-audio is used like "-audio pa,model=sb16".  It is almost as simple as
-soundhw, but it reuses the -audiodev parsing machinery and attaches an
audiodev to the newly-created device.  The main 'feature' is that
it knows about adding the codec device for model=intel-hda, and adding
the audiodev to the codec device.

In the future, it could be extended to support default models or
builtin devices, just like -nic, or even a default backend.  For now,
keep it simple.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 audio/audio.c                   |  8 +++++-
 audio/audio.h                   |  1 +
 docs/about/deprecated.rst       |  9 ------
 docs/about/removed-features.rst |  7 +++++
 hw/audio/intel-hda.c            |  5 ++--
 hw/audio/soundhw.c              | 12 +++++---
 include/hw/audio/soundhw.h      |  4 +--
 qemu-options.hx                 | 51 ++++++++++++++++-----------------
 softmmu/vl.c                    | 28 ++++++++++++++++--
 9 files changed, 76 insertions(+), 49 deletions(-)

diff --git a/audio/audio.c b/audio/audio.c
index 9e91a5a4f2..a02f3ce5c6 100644
--- a/audio/audio.c
+++ b/audio/audio.c
@@ -2099,13 +2099,19 @@ static void audio_validate_opts(Audiodev *dev, Error **errp)
 
 void audio_parse_option(const char *opt)
 {
-    AudiodevListEntry *e;
     Audiodev *dev = NULL;
 
     Visitor *v = qobject_input_visitor_new_str(opt, "driver", &error_fatal);
     visit_type_Audiodev(v, NULL, &dev, &error_fatal);
     visit_free(v);
 
+    audio_define(dev);
+}
+
+void audio_define(Audiodev *dev)
+{
+    AudiodevListEntry *e;
+
     audio_validate_opts(dev, &error_fatal);
 
     e = g_new0(AudiodevListEntry, 1);
diff --git a/audio/audio.h b/audio/audio.h
index 3d5ecdecd5..b5e17cd218 100644
--- a/audio/audio.h
+++ b/audio/audio.h
@@ -168,6 +168,7 @@ void audio_sample_to_uint64(const void *samples, int pos,
 void audio_sample_from_uint64(void *samples, int pos,
                             uint64_t left, uint64_t right);
 
+void audio_define(Audiodev *audio);
 void audio_parse_option(const char *opt);
 void audio_init_audiodevs(void);
 void audio_legacy_help(void);
diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst
index 1f961b9e6e..a92ae0f162 100644
--- a/docs/about/deprecated.rst
+++ b/docs/about/deprecated.rst
@@ -39,15 +39,6 @@ should specify an ``audiodev=`` property.  Additionally, when using
 vnc, you should specify an ``audiodev=`` property if you plan to
 transmit audio through the VNC protocol.
 
-Creating sound card devices using ``-soundhw`` (since 5.1)
-''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
-
-Sound card devices should be created using ``-device`` instead.  The
-names are the same for most devices.  The exceptions are ``hda`` which
-needs two devices (``-device intel-hda -device hda-duplex``) and
-``pcspk`` which can be activated using ``-machine
-pcspk-audiodev=<name>``.
-
 ``-chardev`` backend aliases ``tty`` and ``parport`` (since 6.0)
 ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
 
diff --git a/docs/about/removed-features.rst b/docs/about/removed-features.rst
index 69498efd51..eb76974347 100644
--- a/docs/about/removed-features.rst
+++ b/docs/about/removed-features.rst
@@ -654,6 +654,13 @@ tripped up the CI testing and was suspected to be quite broken. For that
 reason the maintainers strongly suspected no one actually used it.
 
 
+Creating sound card devices using ``-soundhw`` (removed in 7.1)
+'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
+
+Sound card devices should be created using ``-device`` or ``-audio``.
+The exception is ``pcspk`` which can be activated using ``-machine
+pcspk-audiodev=<name>``.
+
 TCG introspection features
 --------------------------
 
diff --git a/hw/audio/intel-hda.c b/hw/audio/intel-hda.c
index bc77e3d8c9..f38117057b 100644
--- a/hw/audio/intel-hda.c
+++ b/hw/audio/intel-hda.c
@@ -1311,17 +1311,16 @@ static const TypeInfo hda_codec_device_type_info = {
  * create intel hda controller with codec attached to it,
  * so '-soundhw hda' works.
  */
-static int intel_hda_and_codec_init(PCIBus *bus)
+static int intel_hda_and_codec_init(PCIBus *bus, const char *audiodev)
 {
     DeviceState *controller;
     BusState *hdabus;
     DeviceState *codec;
 
-    warn_report("'-soundhw hda' is deprecated, "
-                "please use '-device intel-hda -device hda-duplex' instead");
     controller = DEVICE(pci_create_simple(bus, -1, "intel-hda"));
     hdabus = QLIST_FIRST(&controller->child_bus);
     codec = qdev_new("hda-duplex");
+    qdev_prop_set_string(codec, "audiodev", audiodev);
     qdev_realize_and_unref(codec, hdabus, &error_fatal);
     return 0;
 }
diff --git a/hw/audio/soundhw.c b/hw/audio/soundhw.c
index ebbd0958ba..94d9463e42 100644
--- a/hw/audio/soundhw.c
+++ b/hw/audio/soundhw.c
@@ -27,6 +27,7 @@
 #include "qemu/error-report.h"
 #include "qapi/error.h"
 #include "qom/object.h"
+#include "hw/qdev-properties.h"
 #include "hw/isa/isa.h"
 #include "hw/pci/pci.h"
 #include "hw/audio/soundhw.h"
@@ -36,14 +37,14 @@ struct soundhw {
     const char *descr;
     const char *typename;
     int isa;
-    int (*init_pci) (PCIBus *bus);
+    int (*init_pci) (PCIBus *bus, const char *audiodev);
 };
 
 static struct soundhw soundhw[9];
 static int soundhw_count;
 
 void pci_register_soundhw(const char *name, const char *descr,
-                          int (*init_pci)(PCIBus *bus))
+                          int (*init_pci)(PCIBus *bus, const char *audiodev))
 {
     assert(soundhw_count < ARRAY_SIZE(soundhw) - 1);
     soundhw[soundhw_count].name = name;
@@ -80,8 +81,9 @@ void show_valid_soundhw(void)
 }
 
 static struct soundhw *selected = NULL;
+static const char *audiodev_id;
 
-void select_soundhw(const char *optarg)
+void select_soundhw(const char *optarg, const char *audiodev)
 {
     struct soundhw *c;
 
@@ -92,6 +94,7 @@ void select_soundhw(const char *optarg)
     for (c = soundhw; c->name; ++c) {
         if (g_str_equal(c->name, optarg)) {
             selected = c;
+            audiodev_id = audiodev;
             break;
         }
     }
@@ -129,10 +132,11 @@ void soundhw_init(void)
 
     if (c->typename) {
         DeviceState *dev = qdev_new(c->typename);
+        qdev_prop_set_string(dev, "audiodev", audiodev_id);
         qdev_realize_and_unref(dev, bus, &error_fatal);
     } else {
         assert(!c->isa);
-        c->init_pci(pci_bus);
+        c->init_pci(pci_bus, audiodev_id);
     }
 }
 
diff --git a/include/hw/audio/soundhw.h b/include/hw/audio/soundhw.h
index dec5c0cdca..270717a06a 100644
--- a/include/hw/audio/soundhw.h
+++ b/include/hw/audio/soundhw.h
@@ -2,12 +2,12 @@
 #define HW_SOUNDHW_H
 
 void pci_register_soundhw(const char *name, const char *descr,
-                          int (*init_pci)(PCIBus *bus));
+                          int (*init_pci)(PCIBus *bus, const char *audiodev));
 void deprecated_register_soundhw(const char *name, const char *descr,
                                  int isa, const char *typename);
 
 void soundhw_init(void);
 void show_valid_soundhw(void);
-void select_soundhw(const char *optarg);
+void select_soundhw(const char *optarg, const char *audiodev);
 
 #endif
diff --git a/qemu-options.hx b/qemu-options.hx
index 796229c433..60d2d728d6 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -661,6 +661,30 @@ SRST
     (deprecated) environment variables.
 ERST
 
+DEF("audio", HAS_ARG, QEMU_OPTION_audio,
+    "-audio [driver=]driver,model=value[,prop[=value][,...]]\n"
+    "                specifies the audio backend and device to use;\n"
+    "                apart from 'model', options are the same as for -audiodev.\n"
+    "                use '-audio model=help' to show possible devices.\n",
+    QEMU_ARCH_ALL)
+SRST
+``-audio [driver=]driver,model=value[,prop[=value][,...]]``
+    This option is a shortcut for configuring both the guest audio
+    hardware and the host audio backend in one go.
+    The host backend options are the same as with the corresponding
+    ``-audiodev`` options below. The guest hardware model can be set with
+    ``model=modelname``. Use ``model=help`` to list the available device
+    types.
+
+    The following two example do exactly the same, to show how ``-audio``
+    can be used to shorten the command line length:
+
+    .. parsed-literal::
+
+        |qemu_system| -audiodev pa,id=pa -device sb16,audiodev=pa
+        |qemu_system| -audio pa,model=sb16
+ERST
+
 DEF("audiodev", HAS_ARG, QEMU_OPTION_audiodev,
     "-audiodev [driver=]driver,id=id[,prop[=value][,...]]\n"
     "                specifies the audio backend to use\n"
@@ -892,33 +916,6 @@ SRST
         ``qemu.wav``.
 ERST
 
-DEF("soundhw", HAS_ARG, QEMU_OPTION_soundhw,
-    "-soundhw c1,... enable audio support\n"
-    "                and only specified sound cards (comma separated list)\n"
-    "                use '-soundhw help' to get the list of supported cards\n"
-    "                use '-soundhw all' to enable all of them\n", QEMU_ARCH_ALL)
-SRST
-``-soundhw card1[,card2,...] or -soundhw all``
-    Enable audio and selected sound hardware. Use 'help' to print all
-    available sound hardware. For example:
-
-    .. parsed-literal::
-
-        |qemu_system_x86| -soundhw sb16,adlib disk.img
-        |qemu_system_x86| -soundhw es1370 disk.img
-        |qemu_system_x86| -soundhw ac97 disk.img
-        |qemu_system_x86| -soundhw hda disk.img
-        |qemu_system_x86| -soundhw all disk.img
-        |qemu_system_x86| -soundhw help
-
-    Note that Linux's i810\_audio OSS kernel (for AC97) module might
-    require manually specifying clocking.
-
-    ::
-
-        modprobe i810_audio clocking=48000
-ERST
-
 DEF("device", HAS_ARG, QEMU_OPTION_device,
     "-device driver[,prop[=value][,...]]\n"
     "                add device (based on driver)\n"
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 0363cbd9b4..219b23a573 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -116,6 +116,8 @@
 #include "crypto/init.h"
 #include "sysemu/replay.h"
 #include "qapi/qapi-events-run-state.h"
+#include "qapi/qapi-types-audio.h"
+#include "qapi/qapi-visit-audio.h"
 #include "qapi/qapi-visit-block-core.h"
 #include "qapi/qapi-visit-compat.h"
 #include "qapi/qapi-visit-ui.h"
@@ -2930,13 +2932,33 @@ void qemu_init(int argc, char **argv, char **envp)
             case QEMU_OPTION_audiodev:
                 audio_parse_option(optarg);
                 break;
-            case QEMU_OPTION_soundhw:
-                if (is_help_option(optarg)) {
+            case QEMU_OPTION_audio: {
+                QDict *dict = keyval_parse(optarg, "driver", NULL, &error_fatal);
+                char *model;
+                Audiodev *dev = NULL;
+                Visitor *v;
+
+                if (!qdict_haskey(dict, "id")) {
+                    qdict_put_str(dict, "id", "audiodev0");
+                }
+                if (!qdict_haskey(dict, "model")) {
+                    error_setg(&error_fatal, "Parameter 'model' is missing");
+                }
+                model = g_strdup(qdict_get_str(dict, "model"));
+                qdict_del(dict, "model");
+                if (is_help_option(model)) {
                     show_valid_soundhw();
                     exit(0);
                 }
-                select_soundhw (optarg);
+                v = qobject_input_visitor_new_keyval(QOBJECT(dict));
+                qobject_unref(dict);
+                visit_type_Audiodev(v, NULL, &dev, &error_fatal);
+                visit_free(v);
+                audio_define(dev);
+                select_soundhw(model, dev->id);
+                g_free(model);
                 break;
+            }
             case QEMU_OPTION_h:
                 help(0);
                 break;

From 40b3ea76425435ee18395d619e4b3f8f652e25b0 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 13 May 2022 18:06:29 +0200
Subject: [PATCH 337/935] build: remove useless dependency

qemu-plugins.symbols is now processed in Meson.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 Makefile | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/Makefile b/Makefile
index e5fd1ebdf6..b842dbccdb 100644
--- a/Makefile
+++ b/Makefile
@@ -165,10 +165,7 @@ ifneq ($(filter $(ninja-targets), $(ninja-cmd-goals)),)
 endif
 endif
 
-# Force configure to re-run if the API symbols are updated
 ifeq ($(CONFIG_PLUGIN),y)
-config-host.mak: $(SRC_PATH)/plugins/qemu-plugins.symbols
-
 .PHONY: plugins
 plugins:
 	$(call quiet-command,\

From 008f6f20a544cbffe6152aacb69b2774978b49e0 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 13 May 2022 18:06:37 +0200
Subject: [PATCH 338/935] configure: remove another dead variable

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure | 1 -
 1 file changed, 1 deletion(-)

diff --git a/configure b/configure
index c8b5b99532..dda25f05bf 100755
--- a/configure
+++ b/configure
@@ -1992,7 +1992,6 @@ fi
 if test "$static" = "yes" ; then
   echo "CONFIG_STATIC=y" >> $config_host_mak
 fi
-qemu_version=$(head $source_path/VERSION)
 echo "SRC_PATH=$source_path" >> $config_host_mak
 echo "TARGET_DIRS=$target_list" >> $config_host_mak
 if test "$modules" = "yes"; then

From 8eccdb9eb84615291faef1257d5779ebfef7a0d0 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 13 May 2022 11:37:57 +0200
Subject: [PATCH 339/935] configure: remove duplicate help messages

These messages are already emitted by scripts/meson-parse-buildoptions.sh.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/configure b/configure
index dda25f05bf..0cc8c876f7 100755
--- a/configure
+++ b/configure
@@ -1043,10 +1043,6 @@ Advanced options (experts only):
   --enable-tsan            enable thread sanitizer
   --disable-werror         disable compilation abort on warning
   --disable-stack-protector disable compiler-provided stack protection
-  --audio-drv-list=LIST    set audio drivers to try if -audiodev is not used
-  --block-drv-whitelist=L  Same as --block-drv-rw-whitelist=L
-  --with-trace-file=NAME   Full PATH,NAME of file to store traces
-                           Default:trace-<pid>
   --cpu=CPU                Build for host CPU [$cpu]
   --with-coroutine=BACKEND coroutine backend. Supported options:
                            ucontext, sigaltstack, windows

From 777784bda46847fc0587d711c3aacff8fff3c3f9 Mon Sep 17 00:00:00 2001
From: "Jason A. Donenfeld" <Jason@zx2c4.com>
Date: Tue, 3 May 2022 01:20:56 +0200
Subject: [PATCH 340/935] hw/openrisc: support 4 serial ports in or1ksim

The 8250 serial controller supports 4 serial ports, so wire them all up,
so that we can have more than one basic I/O channel.

Cc: Stafford Horne <shorne@gmail.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
[smh:Fixup indentation and lines over 80 chars]
Signed-off-by: Stafford Horne <shorne@gmail.com>
---
 hw/openrisc/openrisc_sim.c | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/hw/openrisc/openrisc_sim.c b/hw/openrisc/openrisc_sim.c
index 99b14940f4..6873124f74 100644
--- a/hw/openrisc/openrisc_sim.c
+++ b/hw/openrisc/openrisc_sim.c
@@ -71,6 +71,10 @@ enum {
     OR1KSIM_ETHOC_IRQ = 4,
 };
 
+enum {
+    OR1KSIM_UART_COUNT = 4
+};
+
 static const struct MemmapEntry {
     hwaddr base;
     hwaddr size;
@@ -239,11 +243,13 @@ static void openrisc_sim_ompic_init(Or1ksimState *state, hwaddr base,
 
 static void openrisc_sim_serial_init(Or1ksimState *state, hwaddr base,
                                      hwaddr size, int num_cpus,
-                                     OpenRISCCPU *cpus[], int irq_pin)
+                                     OpenRISCCPU *cpus[], int irq_pin,
+                                     int uart_idx)
 {
     void *fdt = state->fdt;
     char *nodename;
     qemu_irq serial_irq;
+    char alias[sizeof("uart0")];
     int i;
 
     if (num_cpus > 1) {
@@ -258,7 +264,8 @@ static void openrisc_sim_serial_init(Or1ksimState *state, hwaddr base,
         serial_irq = get_cpu_irq(cpus, 0, irq_pin);
     }
     serial_mm_init(get_system_memory(), base, 0, serial_irq, 115200,
-                   serial_hd(0), DEVICE_NATIVE_ENDIAN);
+                   serial_hd(OR1KSIM_UART_COUNT - uart_idx - 1),
+                   DEVICE_NATIVE_ENDIAN);
 
     /* Add device tree node for serial. */
     nodename = g_strdup_printf("/serial@%" HWADDR_PRIx, base);
@@ -271,7 +278,8 @@ static void openrisc_sim_serial_init(Or1ksimState *state, hwaddr base,
 
     /* The /chosen node is created during fdt creation. */
     qemu_fdt_setprop_string(fdt, "/chosen", "stdout-path", nodename);
-    qemu_fdt_setprop_string(fdt, "/aliases", "uart0", nodename);
+    snprintf(alias, sizeof(alias), "uart%d", uart_idx);
+    qemu_fdt_setprop_string(fdt, "/aliases", alias, nodename);
     g_free(nodename);
 }
 
@@ -414,9 +422,11 @@ static void openrisc_sim_init(MachineState *machine)
                                 smp_cpus, cpus, OR1KSIM_OMPIC_IRQ);
     }
 
-    openrisc_sim_serial_init(state, or1ksim_memmap[OR1KSIM_UART].base,
-                             or1ksim_memmap[OR1KSIM_UART].size, smp_cpus, cpus,
-                             OR1KSIM_UART_IRQ);
+    for (n = 0; n < OR1KSIM_UART_COUNT; ++n)
+        openrisc_sim_serial_init(state, or1ksim_memmap[OR1KSIM_UART].base +
+                                        or1ksim_memmap[OR1KSIM_UART].size * n,
+                                 or1ksim_memmap[OR1KSIM_UART].size,
+                                 smp_cpus, cpus, OR1KSIM_UART_IRQ, n);
 
     load_addr = openrisc_load_kernel(ram_size, kernel_filename);
     if (load_addr > 0) {

From a92162f4f1dfc2daf1bc8f2bbc3cad3c9d1ea729 Mon Sep 17 00:00:00 2001
From: "Jason A. Donenfeld" <Jason@zx2c4.com>
Date: Tue, 3 May 2022 11:45:33 +0200
Subject: [PATCH 341/935] hw/openrisc: use right OMPIC size variable

This appears to be a copy and paste error. The UART size was used
instead of the much smaller OMPIC size. But actually that smaller OMPIC
size is wrong too and doesn't allow the IPI to work in Linux. So set it
to the old value.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
[smh:Updated OR1KSIM_OMPIC size to use OR1KSIM_CPUS_MAX]
Signed-off-by: Stafford Horne <shorne@gmail.com>
---
 hw/openrisc/openrisc_sim.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/openrisc/openrisc_sim.c b/hw/openrisc/openrisc_sim.c
index 6873124f74..35adce17ac 100644
--- a/hw/openrisc/openrisc_sim.c
+++ b/hw/openrisc/openrisc_sim.c
@@ -82,7 +82,7 @@ static const struct MemmapEntry {
     [OR1KSIM_DRAM] =      { 0x00000000,          0 },
     [OR1KSIM_UART] =      { 0x90000000,      0x100 },
     [OR1KSIM_ETHOC] =     { 0x92000000,      0x800 },
-    [OR1KSIM_OMPIC] =     { 0x98000000,         16 },
+    [OR1KSIM_OMPIC] =     { 0x98000000, OR1KSIM_CPUS_MAX * 8 },
 };
 
 static struct openrisc_boot_info {
@@ -418,7 +418,7 @@ static void openrisc_sim_init(MachineState *machine)
 
     if (smp_cpus > 1) {
         openrisc_sim_ompic_init(state, or1ksim_memmap[OR1KSIM_OMPIC].base,
-                                or1ksim_memmap[OR1KSIM_UART].size,
+                                or1ksim_memmap[OR1KSIM_OMPIC].size,
                                 smp_cpus, cpus, OR1KSIM_OMPIC_IRQ);
     }
 

From e8f0ab0cd674241cbab7231ce05ac1bfa0b4f5ed Mon Sep 17 00:00:00 2001
From: Stafford Horne <shorne@gmail.com>
Date: Wed, 11 May 2022 20:47:50 +0900
Subject: [PATCH 342/935] target/openrisc: Do not reset delay slot flag on
 early tb exit

This was found when running linux crypto algorithm selftests used by
wireguard.  We found that randomly the tests would fail.  We found
through investigation that a combination of a tick timer interrupt,
raised when executing a delay slot instruction at a page boundary caused
the issue.

This was caused when handling the TB_EXIT_REQUESTED case in cpu_tb_exec.
On OpenRISC, which doesn't implement synchronize_from_tb, set_pc was
being used as a fallback.  The OpenRISC set_pc implementation clears
dflag, which caused the exception handling logic to not account for the
delay slot.  This was the bug, because it meant when execution resumed
after the interrupt was handling it resumed in the wrong place.

Fix this by implementing synchronize_from_tb which simply updates pc,
and not clear the delay slot flag.

Reported-by: Jason A. Donenfeld <Jason@zx2c4.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Stafford Horne <shorne@gmail.com>
---
 target/openrisc/cpu.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/target/openrisc/cpu.c b/target/openrisc/cpu.c
index dfbafc5236..41d1b2a24a 100644
--- a/target/openrisc/cpu.c
+++ b/target/openrisc/cpu.c
@@ -21,6 +21,7 @@
 #include "qapi/error.h"
 #include "qemu/qemu-print.h"
 #include "cpu.h"
+#include "exec/exec-all.h"
 
 static void openrisc_cpu_set_pc(CPUState *cs, vaddr value)
 {
@@ -30,6 +31,15 @@ static void openrisc_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.dflag = 0;
 }
 
+static void openrisc_cpu_synchronize_from_tb(CPUState *cs,
+                                             const TranslationBlock *tb)
+{
+    OpenRISCCPU *cpu = OPENRISC_CPU(cs);
+
+    cpu->env.pc = tb->pc;
+}
+
+
 static bool openrisc_cpu_has_work(CPUState *cs)
 {
     return cs->interrupt_request & (CPU_INTERRUPT_HARD |
@@ -186,6 +196,7 @@ static const struct SysemuCPUOps openrisc_sysemu_ops = {
 
 static const struct TCGCPUOps openrisc_tcg_ops = {
     .initialize = openrisc_translate_init,
+    .synchronize_from_tb = openrisc_cpu_synchronize_from_tb,
 
 #ifndef CONFIG_USER_ONLY
     .tlb_fill = openrisc_cpu_tlb_fill,

From 433a4fdc42042e3d5a6c3dc741215f78b43b3a2e Mon Sep 17 00:00:00 2001
From: Markus Armbruster <armbru@redhat.com>
Date: Fri, 22 Apr 2022 15:28:07 +0200
Subject: [PATCH 343/935] qapi: Fix malformed "Since:" section tags

"Since X.Y" is not recognized as a tagged section, and therefore not
formatted as such in generated documentation.  Fix by adding the
required colon.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220422132807.1704411-1-armbru@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
---
 qapi/crypto.json     | 3 +--
 qapi/machine.json    | 2 +-
 qapi/misc.json       | 2 +-
 qga/qapi-schema.json | 2 +-
 4 files changed, 4 insertions(+), 5 deletions(-)

diff --git a/qapi/crypto.json b/qapi/crypto.json
index 1ec54c15ca..8e0b4764e5 100644
--- a/qapi/crypto.json
+++ b/qapi/crypto.json
@@ -357,8 +357,7 @@
 #                 password to use to retrieve current master key.
 #                 Defaults to the same secret that was used to open the image
 #
-#
-# Since 5.1
+# Since: 5.1
 ##
 { 'struct': 'QCryptoBlockAmendOptionsLUKS',
   'data': { 'state': 'QCryptoBlockLUKSKeyslotState',
diff --git a/qapi/machine.json b/qapi/machine.json
index 92480d4044..c9bdae40eb 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -972,7 +972,7 @@
 # preconfigure stage to configure numa mapping before initializing
 # machine.
 #
-# Since 3.0
+# Since: 3.0
 ##
 { 'command': 'set-numa-node', 'boxed': true,
   'data': 'NumaOptions',
diff --git a/qapi/misc.json b/qapi/misc.json
index b83cc39029..f8a9feda30 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -188,7 +188,7 @@
 # Features:
 # @unstable: This command is experimental.
 #
-# Since 3.0
+# Since: 3.0
 #
 # Returns: nothing
 #
diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json
index 4d8e506c9e..f989597b0c 100644
--- a/qga/qapi-schema.json
+++ b/qga/qapi-schema.json
@@ -950,7 +950,7 @@
 #         by device mapper
 # @smart: disk smart information (Since 7.1)
 #
-# Since 5.2
+# Since: 5.2
 ##
 { 'struct': 'GuestDiskInfo',
   'data': {'name': 'str', 'partition': 'bool', '*dependencies': ['str'],

From f39057d58b9edc37fd82e3ea8a034fc7353c46e8 Mon Sep 17 00:00:00 2001
From: Andrea Bolognani <abologna@redhat.com>
Date: Tue, 3 May 2022 09:37:30 +0200
Subject: [PATCH 344/935] qapi: Drop stray trailing symbol

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220503073737.84223-2-abologna@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qapi/run-state.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/qapi/run-state.json b/qapi/run-state.json
index 8124220bd9..15d6c9a2ed 100644
--- a/qapi/run-state.json
+++ b/qapi/run-state.json
@@ -348,7 +348,7 @@
 #
 # @poweroff: Shutdown the VM and exit
 #
-# @pause: pause the VM#
+# @pause: pause the VM
 #
 # Since: 6.0
 ##

From 406dfba614edc8629f60f03eb7c26bf4e426137d Mon Sep 17 00:00:00 2001
From: Andrea Bolognani <abologna@redhat.com>
Date: Tue, 3 May 2022 09:37:31 +0200
Subject: [PATCH 345/935] qapi: Fix comment indentation

It should start on the very first column.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220503073737.84223-3-abologna@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qapi/ui.json | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/qapi/ui.json b/qapi/ui.json
index 059302a5ef..43e62efd76 100644
--- a/qapi/ui.json
+++ b/qapi/ui.json
@@ -1250,21 +1250,21 @@
                 '*p2p': 'bool',
                 '*audiodev': 'str' } }
 
- ##
- # @DisplayGLMode:
- #
- # Display OpenGL mode.
- #
- # @off: Disable OpenGL (default).
- # @on: Use OpenGL, pick context type automatically.
- #      Would better be named 'auto' but is called 'on' for backward
- #      compatibility with bool type.
- # @core: Use OpenGL with Core (desktop) Context.
- # @es: Use OpenGL with ES (embedded systems) Context.
- #
- # Since: 3.0
- #
- ##
+##
+# @DisplayGLMode:
+#
+# Display OpenGL mode.
+#
+# @off: Disable OpenGL (default).
+# @on: Use OpenGL, pick context type automatically.
+#      Would better be named 'auto' but is called 'on' for backward
+#      compatibility with bool type.
+# @core: Use OpenGL with Core (desktop) Context.
+# @es: Use OpenGL with ES (embedded systems) Context.
+#
+# Since: 3.0
+#
+##
 { 'enum'    : 'DisplayGLMode',
   'data'    : [ 'off', 'on', 'core', 'es' ] }
 

From 4ae65a52658c264caebcb3a31effccf582822a6d Mon Sep 17 00:00:00 2001
From: Andrea Bolognani <abologna@redhat.com>
Date: Tue, 3 May 2022 09:37:32 +0200
Subject: [PATCH 346/935] qapi: Add missing separators between sections

This only affects readability. The generated documentation
doesn't change.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220503073737.84223-4-abologna@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qapi/block-core.json | 5 +++++
 qapi/block.json      | 1 +
 qapi/crypto.json     | 7 +++++++
 qapi/machine.json    | 1 +
 qapi/migration.json  | 4 ++++
 5 files changed, 18 insertions(+)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index b66494e8c5..34dae298ee 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1744,6 +1744,7 @@
 # Since: 2.3
 #
 # Example:
+#
 # -> { "execute": "blockdev-backup",
 #      "arguments": { "device": "src-id",
 #                     "sync": "full",
@@ -2008,6 +2009,7 @@
 # @on-target-error: the action to take on an error on the target,
 #                   default 'report' (no limitations, since this applies to
 #                   a different block device than @device).
+#
 # @unmap: Whether to try to unmap target sectors where source has
 #         only zero. If true, and target unallocated sectors will read as zero,
 #         target image sectors will be unmapped; otherwise, zeroes will be
@@ -2029,6 +2031,7 @@
 #                When true, this job will automatically disappear from the query
 #                list without user intervention.
 #                Defaults to true. (Since 3.1)
+#
 # Since: 1.3
 ##
 { 'struct': 'DriveMirror',
@@ -2342,6 +2345,7 @@
 #                When true, this job will automatically disappear from the query
 #                list without user intervention.
 #                Defaults to true. (Since 3.1)
+#
 # Returns: nothing on success.
 #
 # Since: 2.6
@@ -4139,6 +4143,7 @@
 # @throttle-group: the name of the throttle-group object to use. It
 #                  must already exist.
 # @file: reference to or definition of the data source block device
+#
 # Since: 2.11
 ##
 { 'struct': 'BlockdevOptionsThrottle',
diff --git a/qapi/block.json b/qapi/block.json
index 3f100d4887..e0f7898ed1 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -105,6 +105,7 @@
 #
 # Returns: - Nothing on success
 #          - If @device is not a valid block device, DeviceNotFound
+#
 # Notes:    Ejecting a device with no media results in success
 #
 # Since: 0.14
diff --git a/qapi/crypto.json b/qapi/crypto.json
index 8e0b4764e5..21c670eefa 100644
--- a/qapi/crypto.json
+++ b/qapi/crypto.json
@@ -32,6 +32,7 @@
 #
 # @raw: raw bytes. When encoded in JSON only valid UTF-8 sequences can be used
 # @base64: arbitrary base64 encoded binary data
+#
 # Since: 2.6
 ##
 { 'enum': 'QCryptoSecretFormat',
@@ -51,6 +52,7 @@
 # @sha384: SHA-384. (since 2.7)
 # @sha512: SHA-512. (since 2.7)
 # @ripemd160: RIPEMD-160. (since 2.7)
+#
 # Since: 2.6
 ##
 { 'enum': 'QCryptoHashAlgorithm',
@@ -75,6 +77,7 @@
 # @twofish-128: Twofish with 128 bit / 16 byte keys
 # @twofish-192: Twofish with 192 bit / 24 byte keys
 # @twofish-256: Twofish with 256 bit / 32 byte keys
+#
 # Since: 2.6
 ##
 { 'enum': 'QCryptoCipherAlgorithm',
@@ -95,6 +98,7 @@
 # @cbc: Cipher Block Chaining
 # @xts: XEX with tweaked code book and ciphertext stealing
 # @ctr: Counter (Since 2.8)
+#
 # Since: 2.6
 ##
 { 'enum': 'QCryptoCipherMode',
@@ -114,6 +118,7 @@
 # @plain: 64-bit sector number truncated to 32-bits
 # @plain64: 64-bit sector number
 # @essiv: 64-bit sector number encrypted with a hash of the encryption key
+#
 # Since: 2.6
 ##
 { 'enum': 'QCryptoIVGenAlgorithm',
@@ -170,6 +175,7 @@
 # @key-secret: the ID of a QCryptoSecret object providing the
 #              decryption key. Mandatory except when probing image for
 #              metadata only.
+#
 # Since: 2.6
 ##
 { 'struct': 'QCryptoBlockOptionsLUKS',
@@ -194,6 +200,7 @@
 # @iter-time: number of milliseconds to spend in
 #             PBKDF passphrase processing. Currently defaults
 #             to 2000. (since 2.8)
+#
 # Since: 2.6
 ##
 { 'struct': 'QCryptoBlockCreateOptionsLUKS',
diff --git a/qapi/machine.json b/qapi/machine.json
index c9bdae40eb..79901e1c09 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -299,6 +299,7 @@
 #        returning does not indicate that a guest has accepted the request or
 #        that it has shut down.  Many guests will respond to this command by
 #        prompting the user in some way.
+#
 # Example:
 #
 # -> { "execute": "system_powerdown" }
diff --git a/qapi/migration.json b/qapi/migration.json
index 409eb086a2..fc1c157d3f 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -1422,7 +1422,9 @@
 # @state: The state the migration is currently expected to be in
 #
 # Returns: nothing on success
+#
 # Since: 2.11
+#
 # Example:
 #
 # -> { "execute": "migrate-continue" , "arguments":
@@ -1736,6 +1738,7 @@
 # Since: 4.2
 #
 # Example:
+#
 # <- { "event": "UNPLUG_PRIMARY",
 #      "data": { "device-id": "hostdev0" },
 #      "timestamp": { "seconds": 1265044230, "microseconds": 450486 } }
@@ -1845,6 +1848,7 @@
 # Since: 5.2
 #
 # Example:
+#
 #   {"execute": "calc-dirty-rate", "arguments": {"calc-time": 1,
 #                                                'sample-pages': 512} }
 #

From c6487a88871ed299f78641f81f2c0aaff9b6c8bb Mon Sep 17 00:00:00 2001
From: Andrea Bolognani <abologna@redhat.com>
Date: Tue, 3 May 2022 09:37:33 +0200
Subject: [PATCH 347/935] qapi: Drop unnecessary empty lines in comments

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220503073737.84223-5-abologna@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qapi/block-core.json  |  4 ----
 qapi/block.json       |  1 -
 qapi/char.json        |  1 -
 qapi/common.json      |  2 --
 qapi/control.json     |  2 --
 qapi/machine.json     |  2 --
 qapi/migration.json   |  7 -------
 qapi/misc-target.json |  3 ---
 qapi/replay.json      |  1 -
 qapi/run-state.json   |  3 ---
 qapi/ui.json          | 22 ----------------------
 11 files changed, 48 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 34dae298ee..27832a1244 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -237,7 +237,6 @@
 #                   information (since 1.7)
 #
 # Since: 1.3
-#
 ##
 { 'struct': 'ImageInfo',
   'data': {'filename': 'str', 'format': 'str', '*dirty-flag': 'bool',
@@ -288,7 +287,6 @@
 #                       supports it
 #
 # Since: 1.4
-#
 ##
 { 'struct': 'ImageCheck',
   'data': {'filename': 'str', 'format': 'str', 'check-errors': 'int',
@@ -328,7 +326,6 @@
 # @filename: filename that is referred to by @offset
 #
 # Since: 2.6
-#
 ##
 { 'struct': 'MapEntry',
   'data': {'start': 'int', 'length': 'int', 'data': 'bool',
@@ -445,7 +442,6 @@
 #                 has one or more dirty bitmaps) (Since 4.2)
 #
 # Since: 0.14
-#
 ##
 { 'struct': 'BlockDeviceInfo',
   'data': { 'file': 'str', '*node-name': 'str', 'ro': 'bool', 'drv': 'str',
diff --git a/qapi/block.json b/qapi/block.json
index e0f7898ed1..5de15c6070 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -300,7 +300,6 @@
 # @read-write: Makes the device writable
 #
 # Since: 2.3
-#
 ##
 { 'enum': 'BlockdevChangeReadOnlyMode',
   'data': ['retain', 'read-only', 'read-write'] }
diff --git a/qapi/char.json b/qapi/char.json
index 7b42151575..f0fd0d1c9f 100644
--- a/qapi/char.json
+++ b/qapi/char.json
@@ -413,7 +413,6 @@
 # @clipboard: enable/disable clipboard, default is disabled.
 #
 # Since: 6.1
-#
 ##
 { 'struct': 'ChardevQemuVDAgent',
   'data': { '*mouse': 'bool',
diff --git a/qapi/common.json b/qapi/common.json
index 412cc4f5ae..356db3f670 100644
--- a/qapi/common.json
+++ b/qapi/common.json
@@ -192,7 +192,6 @@
 # Keys to toggle input-linux between host and guest.
 #
 # Since: 4.0
-#
 ##
 { 'enum': 'GrabToggleKeys',
   'data': [ 'ctrl-ctrl', 'alt-alt', 'shift-shift','meta-meta', 'scrolllock',
@@ -204,7 +203,6 @@
 # @human-readable-text: Formatted output intended for humans.
 #
 # Since: 6.2
-#
 ##
 { 'struct': 'HumanReadableText',
   'data': { 'human-readable-text': 'str' } }
diff --git a/qapi/control.json b/qapi/control.json
index 71a838d49e..8c9122ef7a 100644
--- a/qapi/control.json
+++ b/qapi/control.json
@@ -33,7 +33,6 @@
 #        all the QMP capabilities will be turned off by default.
 #
 # Since: 0.13
-#
 ##
 { 'command': 'qmp_capabilities',
   'data': { '*enable': [ 'QMPCapability' ] },
@@ -49,7 +48,6 @@
 #       (Please refer to qmp-spec.txt for more information on OOB)
 #
 # Since: 2.12
-#
 ##
 { 'enum': 'QMPCapability',
   'data': [ 'oob' ] }
diff --git a/qapi/machine.json b/qapi/machine.json
index 79901e1c09..c8f07c43cd 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -77,7 +77,6 @@
 #          additional fields will be listed (since 3.0)
 #
 # Since: 2.12
-#
 ##
 { 'union'         : 'CpuInfoFast',
   'base'          : { 'cpu-index'    : 'int',
@@ -1022,7 +1021,6 @@
 #          Formula used: logical_vm_size = vm_ram_size - balloon_size
 #
 # Since: 0.14
-#
 ##
 { 'struct': 'BalloonInfo', 'data': {'actual': 'int' } }
 
diff --git a/qapi/migration.json b/qapi/migration.json
index fc1c157d3f..dd4dde6361 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -151,7 +151,6 @@
 #               (since 4.2)
 #
 # Since: 2.3
-#
 ##
 { 'enum': 'MigrationStatus',
   'data': [ 'none', 'setup', 'cancelling', 'cancelled',
@@ -166,7 +165,6 @@
 # @transferred: amount of bytes transferred to the target VM by VFIO devices
 #
 # Since: 5.2
-#
 ##
 { 'struct': 'VfioStats',
   'data': {'transferred': 'int' } }
@@ -546,7 +544,6 @@
 # @zstd: use zstd compression method.
 #
 # Since: 5.0
-#
 ##
 { 'enum': 'MultiFDCompression',
   'data': [ 'none', 'zlib',
@@ -1757,7 +1754,6 @@
 # @dirty-rate: dirty rate.
 #
 # Since: 6.2
-#
 ##
 { 'struct': 'DirtyRateVcpu',
   'data': { 'id': 'int', 'dirty-rate': 'int64' } }
@@ -1774,7 +1770,6 @@
 # @measured: the dirtyrate thread has measured and results are available.
 #
 # Since: 5.2
-#
 ##
 { 'enum': 'DirtyRateStatus',
   'data': [ 'unstarted', 'measuring', 'measured'] }
@@ -1791,7 +1786,6 @@
 # @dirty-bitmap: calculate dirtyrate by dirty bitmap.
 #
 # Since: 6.2
-#
 ##
 { 'enum': 'DirtyRateMeasureMode',
   'data': ['page-sampling', 'dirty-ring', 'dirty-bitmap'] }
@@ -1821,7 +1815,6 @@
 #                   mode specified (Since 6.2)
 #
 # Since: 5.2
-#
 ##
 { 'struct': 'DirtyRateInfo',
   'data': {'*dirty-rate': 'int64',
diff --git a/qapi/misc-target.json b/qapi/misc-target.json
index ed4a468aab..ae2c483a68 100644
--- a/qapi/misc-target.json
+++ b/qapi/misc-target.json
@@ -110,7 +110,6 @@
 # @data: the measurement value encoded in base64
 #
 # Since: 2.12
-#
 ##
 { 'struct': 'SevLaunchMeasureInfo', 'data': {'data': 'str'},
   'if': 'TARGET_I386' }
@@ -194,7 +193,6 @@
 # @gpa: the guest physical address where secret will be injected.
 #
 # Since: 6.0
-#
 ##
 { 'command': 'sev-inject-launch-secret',
   'data': { 'packet-header': 'str', 'secret': 'str', '*gpa': 'uint64' },
@@ -208,7 +206,6 @@
 #
 # @data:  guest attestation report (base64 encoded)
 #
-#
 # Since: 6.1
 ##
 { 'struct': 'SevAttestationReport',
diff --git a/qapi/replay.json b/qapi/replay.json
index 351898f60d..729470300d 100644
--- a/qapi/replay.json
+++ b/qapi/replay.json
@@ -40,7 +40,6 @@
 # @icount: current number of executed instructions.
 #
 # Since: 5.2
-#
 ##
 { 'struct': 'ReplayInfo',
   'data': { 'mode': 'ReplayMode', '*filename': 'str', 'icount': 'int' } }
diff --git a/qapi/run-state.json b/qapi/run-state.json
index 15d6c9a2ed..a5d2db3b91 100644
--- a/qapi/run-state.json
+++ b/qapi/run-state.json
@@ -592,7 +592,6 @@
 # @guest: memory failure at guest memory,
 #
 # Since: 5.2
-#
 ##
 { 'enum': 'MemoryFailureRecipient',
   'data': [ 'hypervisor',
@@ -619,7 +618,6 @@
 #         to handle memory failures.
 #
 # Since: 5.2
-#
 ##
 { 'enum': 'MemoryFailureAction',
   'data': [ 'ignore',
@@ -639,7 +637,6 @@
 #             failure was still in progress.
 #
 # Since: 5.2
-#
 ##
 { 'struct': 'MemoryFailureFlags',
   'data': { 'action-required': 'bool',
diff --git a/qapi/ui.json b/qapi/ui.json
index 43e62efd76..0e903340fc 100644
--- a/qapi/ui.json
+++ b/qapi/ui.json
@@ -15,7 +15,6 @@
 # Display protocols which support changing password options.
 #
 # Since: 7.0
-#
 ##
 { 'enum': 'DisplayProtocol',
   'data': [ 'vnc', 'spice' ] }
@@ -32,7 +31,6 @@
 # @disconnect: disconnect existing clients
 #
 # Since: 7.0
-#
 ##
 { 'enum': 'SetPasswordAction',
   'data': [ 'keep', 'fail', 'disconnect' ] }
@@ -52,7 +50,6 @@
 #             For VNC, only 'keep' is currently implemented.
 #
 # Since: 7.0
-#
 ##
 { 'union': 'SetPasswordOptions',
   'base': { 'protocol': 'DisplayProtocol',
@@ -70,7 +67,6 @@
 #           Defaults to the first.
 #
 # Since: 7.0
-#
 ##
 { 'struct': 'SetPasswordOptionsVnc',
   'data': { '*display': 'str' } }
@@ -115,7 +111,6 @@
 #        sure you are on the same machine as the QEMU instance.
 #
 # Since: 7.0
-#
 ##
 { 'union': 'ExpirePasswordOptions',
   'base': { 'protocol': 'DisplayProtocol',
@@ -132,7 +127,6 @@
 #           Defaults to the first.
 #
 # Since: 7.0
-#
 ##
 
 { 'struct': 'ExpirePasswordOptionsVnc',
@@ -167,7 +161,6 @@
 # @ppm: PPM format
 #
 # Since: 7.1
-#
 ##
 { 'enum': 'ImageFormat',
   'data': ['ppm', 'png'] }
@@ -902,7 +895,6 @@
 # are effectively synonyms.
 #
 # Since: 1.3
-#
 ##
 { 'enum': 'QKeyCode',
   'data': [ 'unmapped',
@@ -1206,7 +1198,6 @@
 #               Since 3.1
 #
 # Since: 2.12
-#
 ##
 { 'struct'  : 'DisplayGTK',
   'data'    : { '*grab-on-hover' : 'bool',
@@ -1221,7 +1212,6 @@
 #              available node on the host.
 #
 # Since: 3.1
-#
 ##
 { 'struct'  : 'DisplayEGLHeadless',
   'data'    : { '*rendernode' : 'str' } }
@@ -1242,7 +1232,6 @@
 # @audiodev: Use the specified DBus audiodev to export audio.
 #
 # Since: 7.0
-#
 ##
 { 'struct'  : 'DisplayDBus',
   'data'    : { '*rendernode' : 'str',
@@ -1263,7 +1252,6 @@
 # @es: Use OpenGL with ES (embedded systems) Context.
 #
 # Since: 3.0
-#
 ##
 { 'enum'    : 'DisplayGLMode',
   'data'    : [ 'off', 'on', 'core', 'es' ] }
@@ -1276,7 +1264,6 @@
 # @charset:       Font charset used by guest (default: CP437).
 #
 # Since: 4.0
-#
 ##
 { 'struct'  : 'DisplayCurses',
   'data'    : { '*charset'       : 'str' } }
@@ -1346,7 +1333,6 @@
 # @dbus: Start a D-Bus service for the display. (Since 7.0)
 #
 # Since: 2.12
-#
 ##
 { 'enum'    : 'DisplayType',
   'data'    : [
@@ -1376,7 +1362,6 @@
 # @gl:            Enable OpenGL support (default: off).
 #
 # Since: 2.12
-#
 ##
 { 'union'   : 'DisplayOptions',
   'base'    : { 'type'           : 'DisplayType',
@@ -1403,7 +1388,6 @@
 # Returns: @DisplayOptions
 #
 # Since: 3.1
-#
 ##
 { 'command': 'query-display-options',
   'returns': 'DisplayOptions' }
@@ -1416,7 +1400,6 @@
 # @vnc: VNC display
 #
 # Since: 6.0
-#
 ##
 { 'enum': 'DisplayReloadType',
   'data': ['vnc'] }
@@ -1429,7 +1412,6 @@
 # @tls-certs: reload tls certs or not.
 #
 # Since: 6.0
-#
 ##
 { 'struct': 'DisplayReloadOptionsVNC',
   'data': { '*tls-certs': 'bool' } }
@@ -1442,7 +1424,6 @@
 # @type: Specify the display type.
 #
 # Since: 6.0
-#
 ##
 { 'union': 'DisplayReloadOptions',
   'base': {'type': 'DisplayReloadType'},
@@ -1477,7 +1458,6 @@
 # @vnc: VNC display
 #
 # Since: 7.1
-#
 ##
 { 'enum': 'DisplayUpdateType',
   'data': ['vnc'] }
@@ -1492,7 +1472,6 @@
 #             for websockets are not touched.
 #
 # Since: 7.1
-#
 ##
 { 'struct': 'DisplayUpdateOptionsVNC',
   'data': { '*addresses': ['SocketAddress'] } }
@@ -1505,7 +1484,6 @@
 # @type: Specify the display type.
 #
 # Since: 7.1
-#
 ##
 { 'union': 'DisplayUpdateOptions',
   'base': {'type': 'DisplayUpdateType'},

From 15cc783cfbd8949c5477c134f66f1fdc599a129e Mon Sep 17 00:00:00 2001
From: Andrea Bolognani <abologna@redhat.com>
Date: Tue, 3 May 2022 09:37:34 +0200
Subject: [PATCH 348/935] qapi: Drop unnecessary empty lines outside of
 comments

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220503073737.84223-6-abologna@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qapi/audio.json          |  1 -
 qapi/block-core.json     | 11 -----------
 qapi/block.json          |  3 ---
 qapi/char.json           |  1 -
 qapi/control.json        |  1 -
 qapi/crypto.json         | 12 ------------
 qapi/job.json            |  1 -
 qapi/machine-target.json |  1 -
 qapi/machine.json        |  1 -
 qapi/misc-target.json    |  4 ----
 qapi/run-state.json      |  1 -
 qapi/ui.json             |  1 -
 12 files changed, 38 deletions(-)

diff --git a/qapi/audio.json b/qapi/audio.json
index 0785e70a50..8099e3d7f1 100644
--- a/qapi/audio.json
+++ b/qapi/audio.json
@@ -352,7 +352,6 @@
     '*out':  'AudiodevPerDirectionOptions',
     '*path': 'str' } }
 
-
 ##
 # @AudioFormat:
 #
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 27832a1244..2bce5bb0ae 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -739,7 +739,6 @@
 ##
 { 'command': 'query-block', 'returns': ['BlockInfo'] }
 
-
 ##
 # @BlockDeviceTimedStats:
 #
@@ -1512,7 +1511,6 @@
 { 'command': 'blockdev-snapshot-sync',
   'data': 'BlockdevSnapshotSync' }
 
-
 ##
 # @blockdev-snapshot:
 #
@@ -1751,7 +1749,6 @@
 { 'command': 'blockdev-backup', 'boxed': true,
   'data': 'BlockdevBackup' }
 
-
 ##
 # @query-named-block-nodes:
 #
@@ -3067,7 +3064,6 @@
   'base': 'BlockdevOptionsGenericFormat',
   'data': { '*key-secret': 'str' } }
 
-
 ##
 # @BlockdevOptionsGenericCOWFormat:
 #
@@ -3182,8 +3178,6 @@
   'base': 'BlockdevOptionsGenericCOWFormat',
   'data': { '*encrypt': 'BlockdevQcowEncryption' } }
 
-
-
 ##
 # @BlockdevQcow2EncryptionFormat:
 #
@@ -3357,7 +3351,6 @@
             '*user': 'str',
             '*host-key-check': 'SshHostKeyCheck' } }
 
-
 ##
 # @BlkdebugEvent:
 #
@@ -3721,7 +3714,6 @@
             '*header-digest': 'IscsiHeaderDigest',
             '*timeout': 'int' } }
 
-
 ##
 # @RbdAuthMode:
 #
@@ -4564,7 +4556,6 @@
 { 'enum': 'BlockdevQcow2Version',
   'data': [ 'v2', 'v3' ] }
 
-
 ##
 # @Qcow2CompressionType:
 #
@@ -4738,7 +4729,6 @@
             '*toolsversion':    'str',
             '*zeroed-grain':    'bool' } }
 
-
 ##
 # @BlockdevCreateOptionsSsh:
 #
@@ -4973,7 +4963,6 @@
 { 'enum': 'BlockErrorAction',
   'data': [ 'ignore', 'report', 'stop' ] }
 
-
 ##
 # @BLOCK_IMAGE_CORRUPTED:
 #
diff --git a/qapi/block.json b/qapi/block.json
index 5de15c6070..41b73c9934 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -286,7 +286,6 @@
   'data': { 'id': 'str',
             'node-name': 'str'} }
 
-
 ##
 # @BlockdevChangeReadOnlyMode:
 #
@@ -304,7 +303,6 @@
 { 'enum': 'BlockdevChangeReadOnlyMode',
   'data': ['retain', 'read-only', 'read-write'] }
 
-
 ##
 # @blockdev-change-medium:
 #
@@ -375,7 +373,6 @@
             '*force': 'bool',
             '*read-only-mode': 'BlockdevChangeReadOnlyMode' } }
 
-
 ##
 # @DEVICE_TRAY_MOVED:
 #
diff --git a/qapi/char.json b/qapi/char.json
index f0fd0d1c9f..8414ef2bc2 100644
--- a/qapi/char.json
+++ b/qapi/char.json
@@ -329,7 +329,6 @@
   'data': { '*signal': 'bool' },
   'base': 'ChardevCommon' }
 
-
 ##
 # @ChardevSpiceChannel:
 #
diff --git a/qapi/control.json b/qapi/control.json
index 8c9122ef7a..53461cec05 100644
--- a/qapi/control.json
+++ b/qapi/control.json
@@ -68,7 +68,6 @@
 { 'struct': 'VersionTriple',
   'data': {'major': 'int', 'minor': 'int', 'micro': 'int'} }
 
-
 ##
 # @VersionInfo:
 #
diff --git a/qapi/crypto.json b/qapi/crypto.json
index 21c670eefa..9ec0aca1c6 100644
--- a/qapi/crypto.json
+++ b/qapi/crypto.json
@@ -24,7 +24,6 @@
   'prefix': 'QCRYPTO_TLS_CREDS_ENDPOINT',
   'data': ['client', 'server']}
 
-
 ##
 # @QCryptoSecretFormat:
 #
@@ -39,7 +38,6 @@
   'prefix': 'QCRYPTO_SECRET_FORMAT',
   'data': ['raw', 'base64']}
 
-
 ##
 # @QCryptoHashAlgorithm:
 #
@@ -59,7 +57,6 @@
   'prefix': 'QCRYPTO_HASH_ALG',
   'data': ['md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512', 'ripemd160']}
 
-
 ##
 # @QCryptoCipherAlgorithm:
 #
@@ -88,7 +85,6 @@
            'serpent-128', 'serpent-192', 'serpent-256',
            'twofish-128', 'twofish-192', 'twofish-256']}
 
-
 ##
 # @QCryptoCipherMode:
 #
@@ -105,7 +101,6 @@
   'prefix': 'QCRYPTO_CIPHER_MODE',
   'data': ['ecb', 'cbc', 'xts', 'ctr']}
 
-
 ##
 # @QCryptoIVGenAlgorithm:
 #
@@ -181,7 +176,6 @@
 { 'struct': 'QCryptoBlockOptionsLUKS',
   'data': { '*key-secret': 'str' }}
 
-
 ##
 # @QCryptoBlockCreateOptionsLUKS:
 #
@@ -212,7 +206,6 @@
             '*hash-alg': 'QCryptoHashAlgorithm',
             '*iter-time': 'int'}}
 
-
 ##
 # @QCryptoBlockOpenOptions:
 #
@@ -227,7 +220,6 @@
   'data': { 'qcow': 'QCryptoBlockOptionsQCow',
             'luks': 'QCryptoBlockOptionsLUKS' } }
 
-
 ##
 # @QCryptoBlockCreateOptions:
 #
@@ -242,7 +234,6 @@
   'data': { 'qcow': 'QCryptoBlockOptionsQCow',
             'luks': 'QCryptoBlockCreateOptionsLUKS' } }
 
-
 ##
 # @QCryptoBlockInfoBase:
 #
@@ -256,7 +247,6 @@
 { 'struct': 'QCryptoBlockInfoBase',
   'data': { 'format': 'QCryptoBlockFormat' }}
 
-
 ##
 # @QCryptoBlockInfoLUKSSlot:
 #
@@ -276,7 +266,6 @@
            '*stripes': 'int',
            'key-offset': 'int' } }
 
-
 ##
 # @QCryptoBlockInfoLUKS:
 #
@@ -330,7 +319,6 @@
 { 'enum': 'QCryptoBlockLUKSKeyslotState',
   'data': [ 'active', 'inactive' ] }
 
-
 ##
 # @QCryptoBlockAmendOptionsLUKS:
 #
diff --git a/qapi/job.json b/qapi/job.json
index 1a6ef03451..d5f84e9615 100644
--- a/qapi/job.json
+++ b/qapi/job.json
@@ -173,7 +173,6 @@
 ##
 { 'command': 'job-cancel', 'data': { 'id': 'str' } }
 
-
 ##
 # @job-complete:
 #
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index f5ec4bc172..06b0d2ca61 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -54,7 +54,6 @@
 { 'enum': 'CpuModelExpansionType',
   'data': [ 'static', 'full' ] }
 
-
 ##
 # @CpuModelCompareResult:
 #
diff --git a/qapi/machine.json b/qapi/machine.json
index c8f07c43cd..3fe977e903 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1365,7 +1365,6 @@
 { 'event': 'MEMORY_DEVICE_SIZE_CHANGE',
   'data': { '*id': 'str', 'size': 'size', 'qom-path' : 'str'} }
 
-
 ##
 # @MEM_UNPLUG_ERROR:
 #
diff --git a/qapi/misc-target.json b/qapi/misc-target.json
index ae2c483a68..2fa68a6796 100644
--- a/qapi/misc-target.json
+++ b/qapi/misc-target.json
@@ -21,7 +21,6 @@
 { 'command': 'rtc-reset-reinjection',
   'if': 'TARGET_I386' }
 
-
 ##
 # @SevState:
 #
@@ -101,7 +100,6 @@
 { 'command': 'query-sev', 'returns': 'SevInfo',
   'if': 'TARGET_I386' }
 
-
 ##
 # @SevLaunchMeasureInfo:
 #
@@ -132,7 +130,6 @@
 { 'command': 'query-sev-launch-measure', 'returns': 'SevLaunchMeasureInfo',
   'if': 'TARGET_I386' }
 
-
 ##
 # @SevCapability:
 #
@@ -304,7 +301,6 @@
 { 'command': 'query-gic-capabilities', 'returns': ['GICCapability'],
   'if': 'TARGET_ARM' }
 
-
 ##
 # @SGXEPCSection:
 #
diff --git a/qapi/run-state.json b/qapi/run-state.json
index a5d2db3b91..30a2f5231d 100644
--- a/qapi/run-state.json
+++ b/qapi/run-state.json
@@ -597,7 +597,6 @@
   'data': [ 'hypervisor',
             'guest' ] }
 
-
 ##
 # @MemoryFailureAction:
 #
diff --git a/qapi/ui.json b/qapi/ui.json
index 0e903340fc..9f19beea6d 100644
--- a/qapi/ui.json
+++ b/qapi/ui.json
@@ -128,7 +128,6 @@
 #
 # Since: 7.0
 ##
-
 { 'struct': 'ExpirePasswordOptionsVnc',
   'data': { '*display': 'str' } }
 

From 23e46452589d8c62d69ae993d16e6d15fd15f526 Mon Sep 17 00:00:00 2001
From: Andrea Bolognani <abologna@redhat.com>
Date: Tue, 3 May 2022 09:37:35 +0200
Subject: [PATCH 349/935] qapi: Drop unnecessary whitespace in comments

The only instances that get changed are those in which the
additional whitespace was not (or couldn't possibly be) used for
alignment purposes.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Message-Id: <20220503073737.84223-7-abologna@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qapi/block-core.json   | 24 ++++++++++++------------
 qapi/block-export.json |  2 +-
 qapi/block.json        |  2 +-
 qapi/char.json         |  2 +-
 qapi/dump.json         |  4 ++--
 qapi/machine.json      |  8 ++++----
 qapi/misc-target.json  |  6 +++---
 qapi/misc.json         |  6 +++---
 qapi/run-state.json    |  4 ++--
 qapi/sockets.json      |  2 +-
 qapi/ui.json           |  2 +-
 11 files changed, 31 insertions(+), 31 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 2bce5bb0ae..e110af2f1d 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -604,7 +604,7 @@
 # @inserted: @BlockDeviceInfo describing the device if media is
 #            present
 #
-# Since:  0.14
+# Since: 0.14
 ##
 { 'struct': 'BlockInfo',
   'data': {'device': 'str', '*qdev': 'str', 'type': 'str', 'removable': 'bool',
@@ -795,9 +795,9 @@
 #
 # Statistics of a virtual block device or a block backing device.
 #
-# @rd_bytes:      The number of bytes read by the device.
+# @rd_bytes: The number of bytes read by the device.
 #
-# @wr_bytes:      The number of bytes written by the device.
+# @wr_bytes: The number of bytes written by the device.
 #
 # @unmap_bytes: The number of bytes unmapped by the device (Since 4.2)
 #
@@ -970,7 +970,7 @@
 # @qdev: The qdev ID, or if no ID is assigned, the QOM path of the block
 #        device. (since 3.0)
 #
-# @stats:  A @BlockDeviceStats for the device.
+# @stats: A @BlockDeviceStats for the device.
 #
 # @driver-specific: Optional driver-specific stats. (Since 4.2)
 #
@@ -1275,7 +1275,7 @@
 #
 # @node-name: graph node name to get the image resized (Since 2.0)
 #
-# @size:  new image size in bytes
+# @size: new image size in bytes
 #
 # Returns: - nothing on success
 #          - If @device is not a valid block device, DeviceNotFound
@@ -1960,8 +1960,8 @@
 # @job-id: identifier for the newly-created block job. If
 #          omitted, the device name will be used. (Since 2.7)
 #
-# @device:  the device name or node-name of a root node whose writes should be
-#           mirrored.
+# @device: the device name or node-name of a root node whose writes should be
+#          mirrored.
 #
 # @target: the target of the new image. If the file exists, or if it
 #          is a device, the existing file/device will be used as the new
@@ -1981,7 +1981,7 @@
 # @mode: whether and how QEMU should create a new image, default is
 #        'absolute-paths'.
 #
-# @speed:  the maximum speed, in bytes per second
+# @speed: the maximum speed, in bytes per second
 #
 # @sync: what parts of the disk image should be copied to the destination
 #        (all the disk, only the sectors allocated in the topmost image, or
@@ -2296,7 +2296,7 @@
 #            broken Quorum files.  By default, @device is replaced, although
 #            implicitly created filters on it are kept.
 #
-# @speed:  the maximum speed, in bytes per second
+# @speed: the maximum speed, in bytes per second
 #
 # @sync: what parts of the disk image should be copied to the destination
 #        (all the disk, only the sectors allocated in the topmost image, or
@@ -4548,8 +4548,8 @@
 ##
 # @BlockdevQcow2Version:
 #
-# @v2:  The original QCOW2 format as introduced in qemu 0.10 (version 2)
-# @v3:  The extended QCOW2 format as introduced in qemu 1.1 (version 3)
+# @v2: The original QCOW2 format as introduced in qemu 0.10 (version 2)
+# @v3: The extended QCOW2 format as introduced in qemu 1.1 (version 3)
 #
 # Since: 2.12
 ##
@@ -4905,7 +4905,7 @@
 #
 # Options for amending an image format
 #
-# @driver:          Block driver of the node to amend.
+# @driver: Block driver of the node to amend.
 #
 # Since: 5.1
 ##
diff --git a/qapi/block-export.json b/qapi/block-export.json
index 7776248435..0685cb8b9a 100644
--- a/qapi/block-export.json
+++ b/qapi/block-export.json
@@ -391,7 +391,7 @@
 #                 block-export-del command, but before the shutdown has
 #                 completed)
 #
-# Since:  5.2
+# Since: 5.2
 ##
 { 'struct': 'BlockExportInfo',
   'data': { 'id': 'str',
diff --git a/qapi/block.json b/qapi/block.json
index 41b73c9934..96f557b3bb 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -106,7 +106,7 @@
 # Returns: - Nothing on success
 #          - If @device is not a valid block device, DeviceNotFound
 #
-# Notes:    Ejecting a device with no media results in success
+# Notes: Ejecting a device with no media results in success
 #
 # Since: 0.14
 #
diff --git a/qapi/char.json b/qapi/char.json
index 8414ef2bc2..a40fe4b7bd 100644
--- a/qapi/char.json
+++ b/qapi/char.json
@@ -216,7 +216,7 @@
 #
 # Configuration info for file chardevs.
 #
-# @in:  The name of the input file
+# @in: The name of the input file
 # @out: The name of the output file
 # @append: Open the file in append mode (default false to
 #          truncate) (Since 2.6)
diff --git a/qapi/dump.json b/qapi/dump.json
index 29441af9d8..90859c5483 100644
--- a/qapi/dump.json
+++ b/qapi/dump.json
@@ -186,8 +186,8 @@
 #
 # Returns the available formats for dump-guest-memory
 #
-# Returns:  A @DumpGuestMemoryCapability object listing available formats for
-#           dump-guest-memory
+# Returns: A @DumpGuestMemoryCapability object listing available formats for
+#          dump-guest-memory
 #
 # Since: 2.0
 #
diff --git a/qapi/machine.json b/qapi/machine.json
index 3fe977e903..1e5bf02480 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -315,9 +315,9 @@
 # query-current-machine), wake-up guest from suspend if the guest is
 # in SUSPENDED state. Return an error otherwise.
 #
-# Since:  1.1
+# Since: 1.1
 #
-# Returns:  nothing.
+# Returns: nothing.
 #
 # Note: prior to 4.0, this command does nothing in case the guest
 #       isn't suspended.
@@ -368,9 +368,9 @@
 # Injects a Non-Maskable Interrupt into the default CPU (x86/s390) or all CPUs (ppc64).
 # The command fails when the guest doesn't support injecting.
 #
-# Returns:  If successful, nothing
+# Returns: If successful, nothing
 #
-# Since:  0.14
+# Since: 0.14
 #
 # Note: prior to 2.1, this command was only supported for x86 and s390 VMs
 #
diff --git a/qapi/misc-target.json b/qapi/misc-target.json
index 2fa68a6796..4944c0528f 100644
--- a/qapi/misc-target.json
+++ b/qapi/misc-target.json
@@ -136,9 +136,9 @@
 # The struct describes capability for a Secure Encrypted Virtualization
 # feature.
 #
-# @pdh:  Platform Diffie-Hellman key (base64 encoded)
+# @pdh: Platform Diffie-Hellman key (base64 encoded)
 #
-# @cert-chain:  PDH certificate chain (base64 encoded)
+# @cert-chain: PDH certificate chain (base64 encoded)
 #
 # @cpu0-id: Unique ID of CPU0 (base64 encoded) (since 7.1)
 #
@@ -201,7 +201,7 @@
 # The struct describes attestation report for a Secure Encrypted
 # Virtualization feature.
 #
-# @data:  guest attestation report (base64 encoded)
+# @data: guest attestation report (base64 encoded)
 #
 # Since: 6.1
 ##
diff --git a/qapi/misc.json b/qapi/misc.json
index f8a9feda30..45344483cd 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -136,7 +136,7 @@
 #
 # Stop all guest VCPU execution.
 #
-# Since:  0.14
+# Since: 0.14
 #
 # Notes: This function will succeed even if the guest is already in the stopped
 #        state.  In "inmigrate" state, it will ensure that the guest
@@ -156,9 +156,9 @@
 #
 # Resume guest VCPU execution.
 #
-# Since:  0.14
+# Since: 0.14
 #
-# Returns:  If successful, nothing
+# Returns: If successful, nothing
 #
 # Notes: This command will succeed if the guest is currently running.  It
 #        will also succeed if the guest is in the "inmigrate" state; in
diff --git a/qapi/run-state.json b/qapi/run-state.json
index 30a2f5231d..6e2162d7b3 100644
--- a/qapi/run-state.json
+++ b/qapi/run-state.json
@@ -104,7 +104,7 @@
 #
 # @status: the virtual machine @RunState
 #
-# Since:  0.14
+# Since: 0.14
 #
 # Notes: @singlestep is enabled through the GDB stub
 ##
@@ -118,7 +118,7 @@
 #
 # Returns: @StatusInfo reflecting all VCPUs
 #
-# Since:  0.14
+# Since: 0.14
 #
 # Example:
 #
diff --git a/qapi/sockets.json b/qapi/sockets.json
index fccc38584b..b4f84d3334 100644
--- a/qapi/sockets.json
+++ b/qapi/sockets.json
@@ -189,7 +189,7 @@
 # Captures the address of a socket, which could also be a named file
 # descriptor
 #
-# @type:       Transport type
+# @type: Transport type
 #
 # Since: 2.9
 ##
diff --git a/qapi/ui.json b/qapi/ui.json
index 9f19beea6d..c844237434 100644
--- a/qapi/ui.json
+++ b/qapi/ui.json
@@ -1260,7 +1260,7 @@
 #
 # Curses display options.
 #
-# @charset:       Font charset used by guest (default: CP437).
+# @charset: Font charset used by guest (default: CP437).
 #
 # Since: 4.0
 ##

From c0ac533b6f97fe978a9d9b33078ee36dd54cedb2 Mon Sep 17 00:00:00 2001
From: Andrea Bolognani <abologna@redhat.com>
Date: Tue, 3 May 2022 09:37:36 +0200
Subject: [PATCH 350/935] qapi: Stop using whitespace for alignment in comments

Perfectly aligned things look pretty, but keeping them that
way as the schema evolves requires churn, and in some cases
newly-added lines are not aligned properly.

Overall, trying to align things is just not worth the trouble.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Message-Id: <20220503073737.84223-8-abologna@redhat.com>
Message-Id: <20220503073737.84223-9-abologna@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Two patches squashed together]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qapi/block-core.json | 53 ++++++++++++++++++++++----------------------
 qapi/block.json      |  6 ++---
 qapi/char.json       |  6 ++---
 qapi/control.json    | 10 ++++-----
 qapi/crypto.json     | 42 +++++++++++++++++------------------
 qapi/migration.json  |  8 +++----
 qapi/sockets.json    |  4 ++--
 qapi/ui.json         | 17 +++++++-------
 8 files changed, 72 insertions(+), 74 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index e110af2f1d..f0383c7925 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -337,9 +337,9 @@
 #
 # Cache mode information for a block device
 #
-# @writeback:   true if writeback mode is enabled
-# @direct:      true if the host page cache is bypassed (O_DIRECT)
-# @no-flush:    true if flush requests are ignored for the device
+# @writeback: true if writeback mode is enabled
+# @direct: true if the host page cache is bypassed (O_DIRECT)
+# @no-flush: true if flush requests are ignored for the device
 #
 # Since: 2.3
 ##
@@ -3333,15 +3333,14 @@
 ##
 # @BlockdevOptionsSsh:
 #
-# @server:              host address
+# @server: host address
 #
-# @path:                path to the image on the host
+# @path: path to the image on the host
 #
-# @user:                user as which to connect, defaults to current
-#                       local user name
+# @user: user as which to connect, defaults to current local user name
 #
-# @host-key-check:      Defines how and what to check the host key against
-#                       (default: known_hosts)
+# @host-key-check: Defines how and what to check the host key against
+#                  (default: known_hosts)
 #
 # Since: 2.9
 ##
@@ -4662,18 +4661,18 @@
 #
 # Subformat options for VMDK images
 #
-# @monolithicSparse:     Single file image with sparse cluster allocation
+# @monolithicSparse: Single file image with sparse cluster allocation
 #
-# @monolithicFlat:       Single flat data image and a descriptor file
+# @monolithicFlat: Single flat data image and a descriptor file
 #
 # @twoGbMaxExtentSparse: Data is split into 2GB (per virtual LBA) sparse extent
 #                        files, in addition to a descriptor file
 #
-# @twoGbMaxExtentFlat:   Data is split into 2GB (per virtual LBA) flat extent
-#                        files, in addition to a descriptor file
+# @twoGbMaxExtentFlat: Data is split into 2GB (per virtual LBA) flat extent
+#                      files, in addition to a descriptor file
 #
-# @streamOptimized:      Single file image sparse cluster allocation, optimized
-#                        for streaming over network.
+# @streamOptimized: Single file image sparse cluster allocation, optimized
+#                   for streaming over network.
 #
 # Since: 4.0
 ##
@@ -4764,7 +4763,7 @@
 # @BlockdevVhdxSubformat:
 #
 # @dynamic: Growing image file
-# @fixed:   Preallocated fixed-size image file
+# @fixed: Preallocated fixed-size image file
 #
 # Since: 2.12
 ##
@@ -4802,7 +4801,7 @@
 # @BlockdevVpcSubformat:
 #
 # @dynamic: Growing image file
-# @fixed:   Preallocated fixed-size image file
+# @fixed: Preallocated fixed-size image file
 #
 # Since: 2.12
 ##
@@ -4865,9 +4864,9 @@
 # Starts a job to create an image format on a given node. The job is
 # automatically finalized, but a manual job-dismiss is required.
 #
-# @job-id:          Identifier for the newly created job.
+# @job-id: Identifier for the newly created job.
 #
-# @options:         Options for the image creation.
+# @options: Options for the image creation.
 #
 # Since: 3.0
 ##
@@ -4923,17 +4922,17 @@
 # Starts a job to amend format specific options of an existing open block device
 # The job is automatically finalized, but a manual job-dismiss is required.
 #
-# @job-id:          Identifier for the newly created job.
+# @job-id: Identifier for the newly created job.
 #
-# @node-name:       Name of the block node to work on
+# @node-name: Name of the block node to work on
 #
-# @options:         Options (driver specific)
+# @options: Options (driver specific)
 #
-# @force:           Allow unsafe operations, format specific
-#                   For luks that allows erase of the last active keyslot
-#                   (permanent loss of data),
-#                   and replacement of an active keyslot
-#                   (possible loss of data if IO error happens)
+# @force: Allow unsafe operations, format specific
+#         For luks that allows erase of the last active keyslot
+#         (permanent loss of data),
+#         and replacement of an active keyslot
+#         (possible loss of data if IO error happens)
 #
 # Features:
 # @unstable: This command is experimental.
diff --git a/qapi/block.json b/qapi/block.json
index 96f557b3bb..19326641ac 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -50,9 +50,9 @@
 #
 # Type of Floppy drive to be emulated by the Floppy Disk Controller.
 #
-# @144:  1.44MB 3.5" drive
-# @288:  2.88MB 3.5" drive
-# @120:  1.2MB 5.25" drive
+# @144: 1.44MB 3.5" drive
+# @288: 2.88MB 3.5" drive
+# @120: 1.2MB 5.25" drive
 # @none: No drive connected
 # @auto: Automatically determined by inserted media at boot
 #
diff --git a/qapi/char.json b/qapi/char.json
index a40fe4b7bd..923dc5056d 100644
--- a/qapi/char.json
+++ b/qapi/char.json
@@ -376,10 +376,10 @@
 #
 # Configuration info for virtual console chardevs.
 #
-# @width:  console width,  in pixels
+# @width: console width, in pixels
 # @height: console height, in pixels
-# @cols:   console width,  in chars
-# @rows:   console height, in chars
+# @cols: console width, in chars
+# @rows: console height, in chars
 #
 # Since: 1.5
 ##
diff --git a/qapi/control.json b/qapi/control.json
index 53461cec05..afca2043af 100644
--- a/qapi/control.json
+++ b/qapi/control.json
@@ -192,14 +192,14 @@
 #
 # Options to be used for adding a new monitor.
 #
-# @id:          Name of the monitor
+# @id: Name of the monitor
 #
-# @mode:        Selects the monitor mode (default: readline in the system
-#               emulator, control in qemu-storage-daemon)
+# @mode: Selects the monitor mode (default: readline in the system
+#           emulator, control in qemu-storage-daemon)
 #
-# @pretty:      Enables pretty printing (QMP only)
+# @pretty: Enables pretty printing (QMP only)
 #
-# @chardev:     Name of a character device to expose the monitor on
+# @chardev: Name of a character device to expose the monitor on
 #
 # Since: 5.0
 ##
diff --git a/qapi/crypto.json b/qapi/crypto.json
index 9ec0aca1c6..15c24f0078 100644
--- a/qapi/crypto.json
+++ b/qapi/crypto.json
@@ -311,8 +311,8 @@
 #
 # Defines state of keyslots that are affected by the update
 #
-# @active:    The slots contain the given password and marked as active
-# @inactive:  The slots are erased (contain garbage) and marked as inactive
+# @active: The slots contain the given password and marked as active
+# @inactive: The slots are erased (contain garbage) and marked as inactive
 #
 # Since: 5.1
 ##
@@ -327,30 +327,30 @@
 #
 # @state: the desired state of the keyslots
 #
-# @new-secret:    The ID of a QCryptoSecret object providing the password to be
-#                 written into added active keyslots
+# @new-secret: The ID of a QCryptoSecret object providing the password to be
+#              written into added active keyslots
 #
-# @old-secret:    Optional (for deactivation only)
-#                 If given will deactivate all keyslots that
-#                 match password located in QCryptoSecret with this ID
+# @old-secret: Optional (for deactivation only)
+#              If given will deactivate all keyslots that
+#              match password located in QCryptoSecret with this ID
 #
-# @iter-time:     Optional (for activation only)
-#                 Number of milliseconds to spend in
-#                 PBKDF passphrase processing for the newly activated keyslot.
-#                 Currently defaults to 2000.
+# @iter-time: Optional (for activation only)
+#             Number of milliseconds to spend in
+#             PBKDF passphrase processing for the newly activated keyslot.
+#             Currently defaults to 2000.
 #
-# @keyslot:       Optional. ID of the keyslot to activate/deactivate.
-#                 For keyslot activation, keyslot should not be active already
-#                 (this is unsafe to update an active keyslot),
-#                 but possible if 'force' parameter is given.
-#                 If keyslot is not given, first free keyslot will be written.
+# @keyslot: Optional. ID of the keyslot to activate/deactivate.
+#           For keyslot activation, keyslot should not be active already
+#           (this is unsafe to update an active keyslot),
+#           but possible if 'force' parameter is given.
+#           If keyslot is not given, first free keyslot will be written.
 #
-#                 For keyslot deactivation, this parameter specifies the exact
-#                 keyslot to deactivate
+#           For keyslot deactivation, this parameter specifies the exact
+#           keyslot to deactivate
 #
-# @secret:        Optional. The ID of a QCryptoSecret object providing the
-#                 password to use to retrieve current master key.
-#                 Defaults to the same secret that was used to open the image
+# @secret: Optional. The ID of a QCryptoSecret object providing the
+#          password to use to retrieve current master key.
+#          Defaults to the same secret that was used to open the image
 #
 # Since: 5.1
 ##
diff --git a/qapi/migration.json b/qapi/migration.json
index dd4dde6361..1ff4d7eaff 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -1191,10 +1191,10 @@
 # ask the client to automatically reconnect using the new parameters
 # once migration finished successfully.  Only implemented for SPICE.
 #
-# @protocol:     must be "spice"
-# @hostname:     migration target hostname
-# @port:         spice tcp port for plaintext channels
-# @tls-port:     spice tcp port for tls-secured channels
+# @protocol: must be "spice"
+# @hostname: migration target hostname
+# @port: spice tcp port for plaintext channels
+# @tls-port: spice tcp port for tls-secured channels
 # @cert-subject: server certificate subject
 #
 # Since: 0.14
diff --git a/qapi/sockets.json b/qapi/sockets.json
index b4f84d3334..bad74e34d3 100644
--- a/qapi/sockets.json
+++ b/qapi/sockets.json
@@ -167,9 +167,9 @@
 #
 # Available SocketAddress types
 #
-# @inet:  Internet address
+# @inet: Internet address
 #
-# @unix:  Unix domain socket
+# @unix: Unix domain socket
 #
 # @vsock: VMCI address
 #
diff --git a/qapi/ui.json b/qapi/ui.json
index c844237434..11a827d10f 100644
--- a/qapi/ui.json
+++ b/qapi/ui.json
@@ -1018,8 +1018,8 @@
 #
 # Keyboard input event.
 #
-# @key:    Which key this event is for.
-# @down:   True for key-down and false for key-up events.
+# @key: Which key this event is for.
+# @down: True for key-down and false for key-up events.
 #
 # Since: 2.0
 ##
@@ -1033,7 +1033,7 @@
 # Pointer button input event.
 #
 # @button: Which button this event is for.
-# @down:   True for key-down and false for key-up events.
+# @down: True for key-down and false for key-up events.
 #
 # Since: 2.0
 ##
@@ -1353,12 +1353,11 @@
 #
 # Display (user interface) options.
 #
-# @type:          Which DisplayType qemu should use.
-# @full-screen:   Start user interface in fullscreen mode (default: off).
-# @window-close:  Allow to quit qemu with window close button (default: on).
-# @show-cursor:   Force showing the mouse cursor (default: off).
-#                 (since: 5.0)
-# @gl:            Enable OpenGL support (default: off).
+# @type: Which DisplayType qemu should use.
+# @full-screen: Start user interface in fullscreen mode (default: off).
+# @window-close: Allow to quit qemu with window close button (default: on).
+# @show-cursor: Force showing the mouse cursor (default: off). (since: 5.0)
+# @gl: Enable OpenGL support (default: off).
 #
 # Since: 2.12
 ##

From 9e191d33688125c0cadff96b389126664054046f Mon Sep 17 00:00:00 2001
From: Markus Armbruster <armbru@redhat.com>
Date: Tue, 10 May 2022 08:16:44 +0200
Subject: [PATCH 351/935] qapi/expr: Enforce feature naming rules again

Commit e42648dccd "qapi/expr.py: Remove single-letter variable"
accidentally removed the check for "only lower case letters and
hyphens".  Restore it.

Fixes: e42648dccdd1defe8f35f247966cd7283f865cd6
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220510061645.3209195-2-armbru@redhat.com>
---
 scripts/qapi/expr.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/qapi/expr.py b/scripts/qapi/expr.py
index 48578e1698..5a1782b57e 100644
--- a/scripts/qapi/expr.py
+++ b/scripts/qapi/expr.py
@@ -443,7 +443,7 @@ def check_features(features: Optional[object],
         check_keys(feat, info, source, ['name'], ['if'])
         check_name_is_str(feat['name'], info, source)
         source = "%s '%s'" % (source, feat['name'])
-        check_name_str(feat['name'], info, source)
+        check_name_lower(feat['name'], info, source)
         check_if(feat, info, source)
 
 

From 1524559f58b1fe14e9b1c9b5be09a82696867eb1 Mon Sep 17 00:00:00 2001
From: Markus Armbruster <armbru@redhat.com>
Date: Tue, 10 May 2022 08:16:45 +0200
Subject: [PATCH 352/935] docs/devel/qapi-code-gen: Belatedly document feature
 naming rules

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220510061645.3209195-3-armbru@redhat.com>
---
 docs/devel/qapi-code-gen.rst | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index 7b968433a6..cd9b544376 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -739,10 +739,11 @@ Type names ending with ``Kind`` or ``List`` are reserved for the
 generator, which uses them for implicit union enums and array types,
 respectively.
 
-Command names, and member names within a type, should be all lower
-case with words separated by a hyphen.  However, some existing older
-commands and complex types use underscore; when extending them,
-consistency is preferred over blindly avoiding underscore.
+Command names, member names within a type, and feature names should be
+all lower case with words separated by a hyphen.  However, some
+existing older commands and complex types use underscore; when
+extending them, consistency is preferred over blindly avoiding
+underscore.
 
 Event names should be ALL_CAPS with words separated by underscore.
 

From dd8f0f332f34a2466d855d1067386b04ff2eb6d9 Mon Sep 17 00:00:00 2001
From: Markus Armbruster <armbru@redhat.com>
Date: Tue, 10 May 2022 10:14:33 +0200
Subject: [PATCH 353/935] qapi/pragma: Tidy up comments

Commit 05ebf841ef "qapi: Enforce command naming rules" inserted new
code between a comment and the code it applies to.  Move the comment
back to its code, and add one for the new code.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220510081433.3289762-1-armbru@redhat.com>
---
 qapi/pragma.json | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/qapi/pragma.json b/qapi/pragma.json
index e6a021c19c..7f810b0e97 100644
--- a/qapi/pragma.json
+++ b/qapi/pragma.json
@@ -6,7 +6,7 @@
 # Whitelists to permit QAPI rule violations; think twice before you
 # add to them!
 { 'pragma': {
-    # Commands allowed to return a non-dictionary:
+    # Command names containing '_'
     'command-name-exceptions': [
         'add_client',
         'block_resize',
@@ -24,6 +24,7 @@
         'system_powerdown',
         'system_reset',
         'system_wakeup' ],
+    # Commands allowed to return a non-dictionary
     'command-returns-exceptions': [
         'human-monitor-command',
         'qom-get',

From dc89f32d92bba795b0665f075b78d8881cf67ab3 Mon Sep 17 00:00:00 2001
From: David Woodhouse <dwmw2@infradead.org>
Date: Mon, 14 Mar 2022 14:25:41 +0000
Subject: [PATCH 354/935] target/i386: Fix sanity check on max APIC ID / X2APIC
 enablement

The check on x86ms->apic_id_limit in pc_machine_done() had two problems.

Firstly, we need KVM to support the X2APIC API in order to allow IRQ
delivery to APICs >= 255. So we need to call/check kvm_enable_x2apic(),
which was done elsewhere in *some* cases but not all.

Secondly, microvm needs the same check. So move it from pc_machine_done()
to x86_cpus_init() where it will work for both.

The check in kvm_cpu_instance_init() is now redundant and can be dropped.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Acked-by: Claudio Fontana <cfontana@suse.de>
Message-Id: <20220314142544.150555-1-dwmw2@infradead.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/pc.c              |  8 --------
 hw/i386/x86.c             | 16 ++++++++++++++++
 target/i386/kvm/kvm-cpu.c |  2 +-
 3 files changed, 17 insertions(+), 9 deletions(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 312eb9e400..15f37d8dc6 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -744,14 +744,6 @@ void pc_machine_done(Notifier *notifier, void *data)
         /* update FW_CFG_NB_CPUS to account for -device added CPUs */
         fw_cfg_modify_i16(x86ms->fw_cfg, FW_CFG_NB_CPUS, x86ms->boot_cpus);
     }
-
-
-    if (x86ms->apic_id_limit > 255 && !xen_enabled() &&
-        !kvm_irqchip_in_kernel()) {
-        error_report("current -smp configuration requires kernel "
-                     "irqchip support.");
-        exit(EXIT_FAILURE);
-    }
 }
 
 void pc_guest_info_init(PCMachineState *pcms)
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 79ebdface6..f79e720cc2 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -38,6 +38,7 @@
 #include "sysemu/replay.h"
 #include "sysemu/sysemu.h"
 #include "sysemu/cpu-timers.h"
+#include "sysemu/xen.h"
 #include "trace.h"
 
 #include "hw/i386/x86.h"
@@ -122,6 +123,21 @@ void x86_cpus_init(X86MachineState *x86ms, int default_cpu_version)
      */
     x86ms->apic_id_limit = x86_cpu_apic_id_from_index(x86ms,
                                                       ms->smp.max_cpus - 1) + 1;
+
+    /*
+     * Can we support APIC ID 255 or higher?
+     *
+     * Under Xen: yes.
+     * With userspace emulated lapic: no
+     * With KVM's in-kernel lapic: only if X2APIC API is enabled.
+     */
+    if (x86ms->apic_id_limit > 255 && !xen_enabled() &&
+        (!kvm_irqchip_in_kernel() || !kvm_enable_x2apic())) {
+        error_report("current -smp configuration requires kernel "
+                     "irqchip and X2APIC API support.");
+        exit(EXIT_FAILURE);
+    }
+
     possible_cpus = mc->possible_cpu_arch_ids(ms);
     for (i = 0; i < ms->smp.cpus; i++) {
         x86_cpu_new(x86ms, possible_cpus->cpus[i].arch_id, &error_fatal);
diff --git a/target/i386/kvm/kvm-cpu.c b/target/i386/kvm/kvm-cpu.c
index 5eb955ce9a..7237378a7d 100644
--- a/target/i386/kvm/kvm-cpu.c
+++ b/target/i386/kvm/kvm-cpu.c
@@ -171,7 +171,7 @@ static void kvm_cpu_instance_init(CPUState *cs)
         /* only applies to builtin_x86_defs cpus */
         if (!kvm_irqchip_in_kernel()) {
             x86_cpu_change_kvm_default("x2apic", "off");
-        } else if (kvm_irqchip_is_split() && kvm_enable_x2apic()) {
+        } else if (kvm_irqchip_is_split()) {
             x86_cpu_change_kvm_default("kvm-msi-ext-dest-id", "on");
         }
 

From 8646d9c773d88c3f9a20919a96afa35e6230c1ee Mon Sep 17 00:00:00 2001
From: David Woodhouse <dwmw@amazon.co.uk>
Date: Mon, 14 Mar 2022 14:25:42 +0000
Subject: [PATCH 355/935] intel_iommu: Support IR-only mode without DMA
 translation

By setting none of the SAGAW bits we can indicate to a guest that DMA
translation isn't supported. Tested by booting Windows 10, as well as
Linux guests with the fix at https://git.kernel.org/torvalds/c/c40aaaac10

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Peter Xu <peterx@redhat.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20220314142544.150555-2-dwmw2@infradead.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/intel_iommu.c         | 14 ++++++++++----
 include/hw/i386/intel_iommu.h |  1 +
 2 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index e05d69a2c0..b22376a45d 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -2214,7 +2214,7 @@ static void vtd_handle_gcmd_write(IntelIOMMUState *s)
     uint32_t changed = status ^ val;
 
     trace_vtd_reg_write_gcmd(status, val);
-    if (changed & VTD_GCMD_TE) {
+    if ((changed & VTD_GCMD_TE) && s->dma_translation) {
         /* Translation enable/disable */
         vtd_handle_gcmd_te(s, val & VTD_GCMD_TE);
     }
@@ -3122,6 +3122,7 @@ static Property vtd_properties[] = {
     DEFINE_PROP_BOOL("x-scalable-mode", IntelIOMMUState, scalable_mode, FALSE),
     DEFINE_PROP_BOOL("snoop-control", IntelIOMMUState, snoop_control, false),
     DEFINE_PROP_BOOL("dma-drain", IntelIOMMUState, dma_drain, true),
+    DEFINE_PROP_BOOL("dma-translation", IntelIOMMUState, dma_translation, true),
     DEFINE_PROP_END_OF_LIST(),
 };
 
@@ -3627,12 +3628,17 @@ static void vtd_init(IntelIOMMUState *s)
     s->next_frcd_reg = 0;
     s->cap = VTD_CAP_FRO | VTD_CAP_NFR | VTD_CAP_ND |
              VTD_CAP_MAMV | VTD_CAP_PSI | VTD_CAP_SLLPS |
-             VTD_CAP_SAGAW_39bit | VTD_CAP_MGAW(s->aw_bits);
+             VTD_CAP_MGAW(s->aw_bits);
     if (s->dma_drain) {
         s->cap |= VTD_CAP_DRAIN;
     }
-    if (s->aw_bits == VTD_HOST_AW_48BIT) {
-        s->cap |= VTD_CAP_SAGAW_48bit;
+    if (s->dma_translation) {
+            if (s->aw_bits >= VTD_HOST_AW_39BIT) {
+                    s->cap |= VTD_CAP_SAGAW_39bit;
+            }
+            if (s->aw_bits >= VTD_HOST_AW_48BIT) {
+                    s->cap |= VTD_CAP_SAGAW_48bit;
+            }
     }
     s->ecap = VTD_ECAP_QI | VTD_ECAP_IRO;
 
diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index bfa982a419..67653b0f9b 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -267,6 +267,7 @@ struct IntelIOMMUState {
     bool buggy_eim;                 /* Force buggy EIM unless eim=off */
     uint8_t aw_bits;                /* Host/IOVA address width (in bits) */
     bool dma_drain;                 /* Whether DMA r/w draining enabled */
+    bool dma_translation;           /* Whether DMA translation supported */
 
     /*
      * Protects IOMMU states in general.  Currently it protects the

From 175f3a59863d7263597011ae86bc57108539ba35 Mon Sep 17 00:00:00 2001
From: David Woodhouse <dwmw@amazon.co.uk>
Date: Mon, 14 Mar 2022 14:25:43 +0000
Subject: [PATCH 356/935] intel_iommu: Only allow interrupt remapping to be
 enabled if it's supported

We should probably check if we were meant to be exposing IR, before
letting the guest turn the IRE bit on.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Peter Xu <peterx@redhat.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20220314142544.150555-3-dwmw2@infradead.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/intel_iommu.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index b22376a45d..d3361c8313 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -2209,6 +2209,7 @@ static void vtd_handle_gcmd_ire(IntelIOMMUState *s, bool en)
 /* Handle write to Global Command Register */
 static void vtd_handle_gcmd_write(IntelIOMMUState *s)
 {
+    X86IOMMUState *x86_iommu = X86_IOMMU_DEVICE(s);
     uint32_t status = vtd_get_long_raw(s, DMAR_GSTS_REG);
     uint32_t val = vtd_get_long_raw(s, DMAR_GCMD_REG);
     uint32_t changed = status ^ val;
@@ -2230,7 +2231,8 @@ static void vtd_handle_gcmd_write(IntelIOMMUState *s)
         /* Set/update the interrupt remapping root-table pointer */
         vtd_handle_gcmd_sirtp(s);
     }
-    if (changed & VTD_GCMD_IRE) {
+    if ((changed & VTD_GCMD_IRE) &&
+        x86_iommu_ir_supported(x86_iommu)) {
         /* Interrupt remap enable/disable */
         vtd_handle_gcmd_ire(s, val & VTD_GCMD_IRE);
     }

From 77250171bdc02aee106083fd2a068147befa1a38 Mon Sep 17 00:00:00 2001
From: David Woodhouse <dwmw2@infradead.org>
Date: Mon, 14 Mar 2022 14:25:44 +0000
Subject: [PATCH 357/935] intel_iommu: Fix irqchip / X2APIC configuration
 checks

We don't need to check kvm_enable_x2apic(). It's perfectly OK to support
interrupt remapping even if we can't address CPUs above 254. Kind of
pointless, but still functional.

The check on kvm_enable_x2apic() needs to happen *anyway* in order to
allow CPUs above 254 even without an IOMMU, so allow that to happen
elsewhere.

However, we do require the *split* irqchip in order to rewrite I/OAPIC
destinations. So fix that check while we're here.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Acked-by: Claudio Fontana <cfontana@suse.de>
Message-Id: <20220314142544.150555-4-dwmw2@infradead.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/intel_iommu.c | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index d3361c8313..d310532108 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -3786,15 +3786,10 @@ static bool vtd_decide_config(IntelIOMMUState *s, Error **errp)
                                               ON_OFF_AUTO_ON : ON_OFF_AUTO_OFF;
     }
     if (s->intr_eim == ON_OFF_AUTO_ON && !s->buggy_eim) {
-        if (!kvm_irqchip_in_kernel()) {
+        if (!kvm_irqchip_is_split()) {
             error_setg(errp, "eim=on requires accel=kvm,kernel-irqchip=split");
             return false;
         }
-        if (!kvm_enable_x2apic()) {
-            error_setg(errp, "eim=on requires support on the KVM side"
-                             "(X2APIC_API, first shipped in v4.7)");
-            return false;
-        }
     }
 
     /* Currently only address widths supported are 39 and 48 bits */

From a48a5bcd641773dca66840908d2c6d9552e3d1c6 Mon Sep 17 00:00:00 2001
From: Jason Wang <jasowang@redhat.com>
Date: Thu, 10 Feb 2022 17:28:14 +0800
Subject: [PATCH 358/935] intel-iommu: remove VTD_FR_RESERVED_ERR

This fault reason is not used and is duplicated with SPT.2 condition
code. So let's remove it.

Signed-off-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20220210092815.45174-1-jasowang@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
---
 hw/i386/intel_iommu.c          | 6 ------
 hw/i386/intel_iommu_internal.h | 5 -----
 2 files changed, 11 deletions(-)

diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index d310532108..8af2e39151 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -469,11 +469,6 @@ static void vtd_report_dmar_fault(IntelIOMMUState *s, uint16_t source_id,
 
     assert(fault < VTD_FR_MAX);
 
-    if (fault == VTD_FR_RESERVED_ERR) {
-        /* This is not a normal fault reason case. Drop it. */
-        return;
-    }
-
     trace_vtd_dmar_fault(source_id, fault, addr, is_write);
 
     if (fsts_reg & VTD_FSTS_PFO) {
@@ -1637,7 +1632,6 @@ static const bool vtd_qualified_faults[] = {
     [VTD_FR_PAGING_ENTRY_RSVD] = true,
     [VTD_FR_CONTEXT_ENTRY_TT] = true,
     [VTD_FR_PASID_TABLE_INV] = false,
-    [VTD_FR_RESERVED_ERR] = false,
     [VTD_FR_MAX] = false,
 };
 
diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h
index 1ff13b40f9..b6a2da8e83 100644
--- a/hw/i386/intel_iommu_internal.h
+++ b/hw/i386/intel_iommu_internal.h
@@ -304,11 +304,6 @@ typedef enum VTDFaultReason {
 
     VTD_FR_PASID_TABLE_INV = 0x58,  /*Invalid PASID table entry */
 
-    /* This is not a normal fault reason. We use this to indicate some faults
-     * that are not referenced by the VT-d specification.
-     * Fault event with such reason should not be recorded.
-     */
-    VTD_FR_RESERVED_ERR,
     VTD_FR_MAX,                 /* Guard */
 } VTDFaultReason;
 

From ea97a1bde7b7940bb2100b645b6f1abc7330e248 Mon Sep 17 00:00:00 2001
From: Jason Wang <jasowang@redhat.com>
Date: Thu, 10 Feb 2022 17:28:15 +0800
Subject: [PATCH 359/935] intel-iommu: block output address in interrupt
 address range

According to vtd spec v3.3 3.14:

"""
Software must not program paging-structure entries to remap any
address to the interrupt address range. Untranslated requests and
translation requests that result in an address in the interrupt range
will be blocked with condition code LGN.4 or SGN.8.
"""

This patch blocks the request that result in interrupt address range.

Signed-off-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20220210092815.45174-2-jasowang@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
---
 hw/i386/intel_iommu.c          | 27 ++++++++++++++++++++++++++-
 hw/i386/intel_iommu_internal.h |  4 ++++
 2 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 8af2e39151..695dd51dc3 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -1020,6 +1020,7 @@ static int vtd_iova_to_slpte(IntelIOMMUState *s, VTDContextEntry *ce,
     uint32_t offset;
     uint64_t slpte;
     uint64_t access_right_check;
+    uint64_t xlat, size;
 
     if (!vtd_iova_range_check(s, iova, ce, aw_bits)) {
         error_report_once("%s: detected IOVA overflow (iova=0x%" PRIx64 ")",
@@ -1064,11 +1065,33 @@ static int vtd_iova_to_slpte(IntelIOMMUState *s, VTDContextEntry *ce,
         if (vtd_is_last_slpte(slpte, level)) {
             *slptep = slpte;
             *slpte_level = level;
-            return 0;
+            break;
         }
         addr = vtd_get_slpte_addr(slpte, aw_bits);
         level--;
     }
+
+    xlat = vtd_get_slpte_addr(*slptep, aw_bits);
+    size = ~vtd_slpt_level_page_mask(level) + 1;
+
+    /*
+     * From VT-d spec 3.14: Untranslated requests and translation
+     * requests that result in an address in the interrupt range will be
+     * blocked with condition code LGN.4 or SGN.8.
+     */
+    if ((xlat > VTD_INTERRUPT_ADDR_LAST ||
+         xlat + size - 1 < VTD_INTERRUPT_ADDR_FIRST)) {
+        return 0;
+    } else {
+        error_report_once("%s: xlat address is in interrupt range "
+                          "(iova=0x%" PRIx64 ", level=0x%" PRIx32 ", "
+                          "slpte=0x%" PRIx64 ", write=%d, "
+                          "xlat=0x%" PRIx64 ", size=0x%" PRIx64 ")",
+                          __func__, iova, level, slpte, is_write,
+                          xlat, size);
+        return s->scalable_mode ? -VTD_FR_SM_INTERRUPT_ADDR :
+                                  -VTD_FR_INTERRUPT_ADDR;
+    }
 }
 
 typedef int (*vtd_page_walk_hook)(IOMMUTLBEvent *event, void *private);
@@ -1628,10 +1651,12 @@ static const bool vtd_qualified_faults[] = {
     [VTD_FR_PAGING_ENTRY_INV] = true,
     [VTD_FR_ROOT_TABLE_INV] = false,
     [VTD_FR_CONTEXT_TABLE_INV] = false,
+    [VTD_FR_INTERRUPT_ADDR] = true,
     [VTD_FR_ROOT_ENTRY_RSVD] = false,
     [VTD_FR_PAGING_ENTRY_RSVD] = true,
     [VTD_FR_CONTEXT_ENTRY_TT] = true,
     [VTD_FR_PASID_TABLE_INV] = false,
+    [VTD_FR_SM_INTERRUPT_ADDR] = true,
     [VTD_FR_MAX] = false,
 };
 
diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h
index b6a2da8e83..930ce61feb 100644
--- a/hw/i386/intel_iommu_internal.h
+++ b/hw/i386/intel_iommu_internal.h
@@ -289,6 +289,8 @@ typedef enum VTDFaultReason {
      * context-entry.
      */
     VTD_FR_CONTEXT_ENTRY_TT,
+    /* Output address in the interrupt address range */
+    VTD_FR_INTERRUPT_ADDR = 0xE,
 
     /* Interrupt remapping transition faults */
     VTD_FR_IR_REQ_RSVD = 0x20, /* One or more IR request reserved
@@ -304,6 +306,8 @@ typedef enum VTDFaultReason {
 
     VTD_FR_PASID_TABLE_INV = 0x58,  /*Invalid PASID table entry */
 
+    /* Output address in the interrupt address range for scalable mode */
+    VTD_FR_SM_INTERRUPT_ADDR = 0x87,
     VTD_FR_MAX,                 /* Guard */
 } VTDFaultReason;
 

From ceb058955ab529937498acfd3c9259a2a64b2d4e Mon Sep 17 00:00:00 2001
From: Jason Wang <jasowang@redhat.com>
Date: Thu, 17 Mar 2022 16:05:21 +0800
Subject: [PATCH 360/935] intel-iommu: update root_scalable before switching as
 during post_load

We need check whether passthrough is enabled during
vtd_switch_address_space() by checking the context entries. This
requires the root_scalable to be set correctly otherwise we may try to
check legacy rsvd bits instead of scalable ones.

Fixing this by updating root_scalable before switching the address
spaces during post_load.

Fixes: fb43cf739e ("intel_iommu: scalable mode emulation")
Signed-off-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20220317080522.14621-1-jasowang@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
---
 hw/i386/intel_iommu.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 695dd51dc3..53e3f5035d 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -3073,13 +3073,6 @@ static int vtd_post_load(void *opaque, int version_id)
 {
     IntelIOMMUState *iommu = opaque;
 
-    /*
-     * Memory regions are dynamically turned on/off depending on
-     * context entry configurations from the guest. After migration,
-     * we need to make sure the memory regions are still correct.
-     */
-    vtd_switch_address_space_all(iommu);
-
     /*
      * We don't need to migrate the root_scalable because we can
      * simply do the calculation after the loading is complete.  We
@@ -3089,6 +3082,13 @@ static int vtd_post_load(void *opaque, int version_id)
      */
     vtd_update_scalable_state(iommu);
 
+    /*
+     * Memory regions are dynamically turned on/off depending on
+     * context entry configurations from the guest. After migration,
+     * we need to make sure the memory regions are still correct.
+     */
+    vtd_switch_address_space_all(iommu);
+
     return 0;
 }
 

From 147a372e7580dc5cf4310d159db2f374e38ac30e Mon Sep 17 00:00:00 2001
From: Jason Wang <jasowang@redhat.com>
Date: Thu, 17 Mar 2022 16:05:22 +0800
Subject: [PATCH 361/935] intel-iommu: update iq_dw during post load

We need to update iq_dw according to the DMA_IRQ_REG during post
load. Otherwise we may get wrong IOTLB invalidation descriptor after
migration.

Fixes: fb43cf739e ("intel_iommu: scalable mode emulation")
Signed-off-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20220317080522.14621-2-jasowang@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
---
 hw/i386/intel_iommu.c | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)

diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 53e3f5035d..2162394e08 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -181,6 +181,18 @@ static void vtd_update_scalable_state(IntelIOMMUState *s)
     }
 }
 
+static void vtd_update_iq_dw(IntelIOMMUState *s)
+{
+    uint64_t val = vtd_get_quad_raw(s, DMAR_IQA_REG);
+
+    if (s->ecap & VTD_ECAP_SMTS &&
+        val & VTD_IQA_DW_MASK) {
+        s->iq_dw = true;
+    } else {
+        s->iq_dw = false;
+    }
+}
+
 /* Whether the address space needs to notify new mappings */
 static inline gboolean vtd_as_has_map_notifier(VTDAddressSpace *as)
 {
@@ -2904,12 +2916,7 @@ static void vtd_mem_write(void *opaque, hwaddr addr,
         } else {
             vtd_set_quad(s, addr, val);
         }
-        if (s->ecap & VTD_ECAP_SMTS &&
-            val & VTD_IQA_DW_MASK) {
-            s->iq_dw = true;
-        } else {
-            s->iq_dw = false;
-        }
+        vtd_update_iq_dw(s);
         break;
 
     case DMAR_IQA_REG_HI:
@@ -3082,6 +3089,8 @@ static int vtd_post_load(void *opaque, int version_id)
      */
     vtd_update_scalable_state(iommu);
 
+    vtd_update_iq_dw(iommu);
+
     /*
      * Memory regions are dynamically turned on/off depending on
      * context entry configurations from the guest. After migration,

From 1eed051c8c8cd84e69f619b0adc22df543ed7f06 Mon Sep 17 00:00:00 2001
From: Ilya Maximets <i.maximets@ovn.org>
Date: Fri, 18 Mar 2022 15:04:40 +0100
Subject: [PATCH 362/935] vhost_net: Print feature masks in hex
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

"0x200000000" is much more readable than "8589934592".
The change saves one step (conversion) while debugging.

Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Message-Id: <20220318140440.596019-1-i.maximets@ovn.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 hw/net/vhost_net.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 30379d2ca4..df0f050548 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -201,7 +201,7 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
             net->dev.features &= ~(1ULL << VIRTIO_NET_F_MRG_RXBUF);
         }
         if (~net->dev.features & net->dev.backend_features) {
-            fprintf(stderr, "vhost lacks feature mask %" PRIu64
+            fprintf(stderr, "vhost lacks feature mask 0x%" PRIx64
                    " for backend\n",
                    (uint64_t)(~net->dev.features & net->dev.backend_features));
             goto fail;
@@ -213,7 +213,7 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
     if (net->nc->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
         features = vhost_user_get_acked_features(net->nc);
         if (~net->dev.features & features) {
-            fprintf(stderr, "vhost lacks feature mask %" PRIu64
+            fprintf(stderr, "vhost lacks feature mask 0x%" PRIx64
                     " for backend\n",
                     (uint64_t)(~net->dev.features & features));
             goto fail;

From e1b1f5341a860e87c6f208e2eea7a3b2bab454dd Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Mon, 21 Mar 2022 15:30:25 +0000
Subject: [PATCH 363/935] hw/virtio: move virtio-pci.h into shared include
 space
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This allows other device classes that will be exposed via PCI to be
able to do so in the appropriate hw/ directory. I resisted the
temptation to re-order headers to be more aesthetically pleasing.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200925125147.26943-4-alex.bennee@linaro.org>

Message-Id: <20220321153037.3622127-2-alex.bennee@linaro.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/vhost-scsi-pci.c             | 2 +-
 hw/virtio/vhost-user-blk-pci.c         | 2 +-
 hw/virtio/vhost-user-fs-pci.c          | 2 +-
 hw/virtio/vhost-user-i2c-pci.c         | 2 +-
 hw/virtio/vhost-user-input-pci.c       | 2 +-
 hw/virtio/vhost-user-rng-pci.c         | 2 +-
 hw/virtio/vhost-user-scsi-pci.c        | 2 +-
 hw/virtio/vhost-user-vsock-pci.c       | 2 +-
 hw/virtio/vhost-vsock-pci.c            | 2 +-
 hw/virtio/virtio-9p-pci.c              | 2 +-
 hw/virtio/virtio-balloon-pci.c         | 2 +-
 hw/virtio/virtio-blk-pci.c             | 2 +-
 hw/virtio/virtio-input-host-pci.c      | 2 +-
 hw/virtio/virtio-input-pci.c           | 2 +-
 hw/virtio/virtio-iommu-pci.c           | 2 +-
 hw/virtio/virtio-net-pci.c             | 2 +-
 hw/virtio/virtio-pci.c                 | 2 +-
 hw/virtio/virtio-rng-pci.c             | 2 +-
 hw/virtio/virtio-scsi-pci.c            | 2 +-
 hw/virtio/virtio-serial-pci.c          | 2 +-
 {hw => include/hw}/virtio/virtio-pci.h | 0
 21 files changed, 20 insertions(+), 20 deletions(-)
 rename {hw => include/hw}/virtio/virtio-pci.h (100%)

diff --git a/hw/virtio/vhost-scsi-pci.c b/hw/virtio/vhost-scsi-pci.c
index cb71a294fa..08980bc23b 100644
--- a/hw/virtio/vhost-scsi-pci.c
+++ b/hw/virtio/vhost-scsi-pci.c
@@ -21,7 +21,7 @@
 #include "hw/virtio/vhost-scsi.h"
 #include "qapi/error.h"
 #include "qemu/module.h"
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "qom/object.h"
 
 typedef struct VHostSCSIPCI VHostSCSIPCI;
diff --git a/hw/virtio/vhost-user-blk-pci.c b/hw/virtio/vhost-user-blk-pci.c
index 33b404d8a2..eef8641a98 100644
--- a/hw/virtio/vhost-user-blk-pci.c
+++ b/hw/virtio/vhost-user-blk-pci.c
@@ -26,7 +26,7 @@
 #include "qapi/error.h"
 #include "qemu/error-report.h"
 #include "qemu/module.h"
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "qom/object.h"
 
 typedef struct VHostUserBlkPCI VHostUserBlkPCI;
diff --git a/hw/virtio/vhost-user-fs-pci.c b/hw/virtio/vhost-user-fs-pci.c
index 2ed8492b3f..6829b8b743 100644
--- a/hw/virtio/vhost-user-fs-pci.c
+++ b/hw/virtio/vhost-user-fs-pci.c
@@ -14,7 +14,7 @@
 #include "qemu/osdep.h"
 #include "hw/qdev-properties.h"
 #include "hw/virtio/vhost-user-fs.h"
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "qom/object.h"
 
 struct VHostUserFSPCI {
diff --git a/hw/virtio/vhost-user-i2c-pci.c b/hw/virtio/vhost-user-i2c-pci.c
index 70b7b65fd9..00ac10941f 100644
--- a/hw/virtio/vhost-user-i2c-pci.c
+++ b/hw/virtio/vhost-user-i2c-pci.c
@@ -9,7 +9,7 @@
 #include "qemu/osdep.h"
 #include "hw/qdev-properties.h"
 #include "hw/virtio/vhost-user-i2c.h"
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 
 struct VHostUserI2CPCI {
     VirtIOPCIProxy parent_obj;
diff --git a/hw/virtio/vhost-user-input-pci.c b/hw/virtio/vhost-user-input-pci.c
index c9d3e9113a..b858898a36 100644
--- a/hw/virtio/vhost-user-input-pci.c
+++ b/hw/virtio/vhost-user-input-pci.c
@@ -9,7 +9,7 @@
 #include "hw/virtio/virtio-input.h"
 #include "qapi/error.h"
 #include "qemu/error-report.h"
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "qom/object.h"
 
 typedef struct VHostUserInputPCI VHostUserInputPCI;
diff --git a/hw/virtio/vhost-user-rng-pci.c b/hw/virtio/vhost-user-rng-pci.c
index c83dc86813..f64935453b 100644
--- a/hw/virtio/vhost-user-rng-pci.c
+++ b/hw/virtio/vhost-user-rng-pci.c
@@ -9,7 +9,7 @@
 #include "qemu/osdep.h"
 #include "hw/qdev-properties.h"
 #include "hw/virtio/vhost-user-rng.h"
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 
 struct VHostUserRNGPCI {
     VirtIOPCIProxy parent_obj;
diff --git a/hw/virtio/vhost-user-scsi-pci.c b/hw/virtio/vhost-user-scsi-pci.c
index d5343412a1..75882e3cf9 100644
--- a/hw/virtio/vhost-user-scsi-pci.c
+++ b/hw/virtio/vhost-user-scsi-pci.c
@@ -30,7 +30,7 @@
 #include "hw/pci/msix.h"
 #include "hw/loader.h"
 #include "sysemu/kvm.h"
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "qom/object.h"
 
 typedef struct VHostUserSCSIPCI VHostUserSCSIPCI;
diff --git a/hw/virtio/vhost-user-vsock-pci.c b/hw/virtio/vhost-user-vsock-pci.c
index 72a96199cd..e5a86e8013 100644
--- a/hw/virtio/vhost-user-vsock-pci.c
+++ b/hw/virtio/vhost-user-vsock-pci.c
@@ -10,7 +10,7 @@
 
 #include "qemu/osdep.h"
 
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "hw/qdev-properties.h"
 #include "hw/virtio/vhost-user-vsock.h"
 #include "qom/object.h"
diff --git a/hw/virtio/vhost-vsock-pci.c b/hw/virtio/vhost-vsock-pci.c
index 205da8d1f5..9f34414d38 100644
--- a/hw/virtio/vhost-vsock-pci.c
+++ b/hw/virtio/vhost-vsock-pci.c
@@ -13,7 +13,7 @@
 
 #include "qemu/osdep.h"
 
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "hw/qdev-properties.h"
 #include "hw/virtio/vhost-vsock.h"
 #include "qemu/module.h"
diff --git a/hw/virtio/virtio-9p-pci.c b/hw/virtio/virtio-9p-pci.c
index e07adcd9ea..94c14f0b98 100644
--- a/hw/virtio/virtio-9p-pci.c
+++ b/hw/virtio/virtio-9p-pci.c
@@ -15,7 +15,7 @@
 
 #include "qemu/osdep.h"
 
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "hw/9pfs/virtio-9p.h"
 #include "hw/qdev-properties.h"
 #include "qemu/module.h"
diff --git a/hw/virtio/virtio-balloon-pci.c b/hw/virtio/virtio-balloon-pci.c
index 79a3ba979a..ce2645ba71 100644
--- a/hw/virtio/virtio-balloon-pci.c
+++ b/hw/virtio/virtio-balloon-pci.c
@@ -14,7 +14,7 @@
 
 #include "qemu/osdep.h"
 
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "hw/qdev-properties.h"
 #include "hw/virtio/virtio-balloon.h"
 #include "qapi/error.h"
diff --git a/hw/virtio/virtio-blk-pci.c b/hw/virtio/virtio-blk-pci.c
index 9d5795810c..9743bee965 100644
--- a/hw/virtio/virtio-blk-pci.c
+++ b/hw/virtio/virtio-blk-pci.c
@@ -19,7 +19,7 @@
 
 #include "hw/qdev-properties.h"
 #include "hw/virtio/virtio-blk.h"
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "qapi/error.h"
 #include "qemu/module.h"
 #include "qom/object.h"
diff --git a/hw/virtio/virtio-input-host-pci.c b/hw/virtio/virtio-input-host-pci.c
index 0ac360de4f..cf8a9cf9e8 100644
--- a/hw/virtio/virtio-input-host-pci.c
+++ b/hw/virtio/virtio-input-host-pci.c
@@ -8,7 +8,7 @@
 
 #include "qemu/osdep.h"
 
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "hw/virtio/virtio-input.h"
 #include "qemu/module.h"
 #include "qom/object.h"
diff --git a/hw/virtio/virtio-input-pci.c b/hw/virtio/virtio-input-pci.c
index 48e9ff38e2..a9d0992389 100644
--- a/hw/virtio/virtio-input-pci.c
+++ b/hw/virtio/virtio-input-pci.c
@@ -8,7 +8,7 @@
 
 #include "qemu/osdep.h"
 
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "hw/qdev-properties.h"
 #include "hw/virtio/virtio-input.h"
 #include "qemu/module.h"
diff --git a/hw/virtio/virtio-iommu-pci.c b/hw/virtio/virtio-iommu-pci.c
index 6a1df7fe50..844d647704 100644
--- a/hw/virtio/virtio-iommu-pci.c
+++ b/hw/virtio/virtio-iommu-pci.c
@@ -11,7 +11,7 @@
 
 #include "qemu/osdep.h"
 
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "hw/virtio/virtio-iommu.h"
 #include "hw/qdev-properties.h"
 #include "hw/qdev-properties-system.h"
diff --git a/hw/virtio/virtio-net-pci.c b/hw/virtio/virtio-net-pci.c
index aa0b3caecb..e03543a70a 100644
--- a/hw/virtio/virtio-net-pci.c
+++ b/hw/virtio/virtio-net-pci.c
@@ -19,7 +19,7 @@
 
 #include "hw/qdev-properties.h"
 #include "hw/virtio/virtio-net.h"
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "qapi/error.h"
 #include "qemu/module.h"
 #include "qom/object.h"
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index 7cf1231c1c..602be7f83d 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -33,7 +33,7 @@
 #include "hw/pci/msix.h"
 #include "hw/loader.h"
 #include "sysemu/kvm.h"
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "qemu/range.h"
 #include "hw/virtio/virtio-bus.h"
 #include "qapi/visitor.h"
diff --git a/hw/virtio/virtio-rng-pci.c b/hw/virtio/virtio-rng-pci.c
index c1f916268b..151ece6f94 100644
--- a/hw/virtio/virtio-rng-pci.c
+++ b/hw/virtio/virtio-rng-pci.c
@@ -11,7 +11,7 @@
 
 #include "qemu/osdep.h"
 
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "hw/virtio/virtio-rng.h"
 #include "qapi/error.h"
 #include "qemu/module.h"
diff --git a/hw/virtio/virtio-scsi-pci.c b/hw/virtio/virtio-scsi-pci.c
index 97fab74236..e8e3442f38 100644
--- a/hw/virtio/virtio-scsi-pci.c
+++ b/hw/virtio/virtio-scsi-pci.c
@@ -18,7 +18,7 @@
 #include "hw/qdev-properties.h"
 #include "hw/virtio/virtio-scsi.h"
 #include "qemu/module.h"
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "qom/object.h"
 
 typedef struct VirtIOSCSIPCI VirtIOSCSIPCI;
diff --git a/hw/virtio/virtio-serial-pci.c b/hw/virtio/virtio-serial-pci.c
index 35bcd961c9..cea31adcc4 100644
--- a/hw/virtio/virtio-serial-pci.c
+++ b/hw/virtio/virtio-serial-pci.c
@@ -20,7 +20,7 @@
 #include "hw/qdev-properties.h"
 #include "hw/virtio/virtio-serial.h"
 #include "qemu/module.h"
-#include "virtio-pci.h"
+#include "hw/virtio/virtio-pci.h"
 #include "qom/object.h"
 
 typedef struct VirtIOSerialPCI VirtIOSerialPCI;
diff --git a/hw/virtio/virtio-pci.h b/include/hw/virtio/virtio-pci.h
similarity index 100%
rename from hw/virtio/virtio-pci.h
rename to include/hw/virtio/virtio-pci.h

From 31cc62bb5bd43b6c36b1f42d5bde6baca4dffaee Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Mon, 21 Mar 2022 15:30:26 +0000
Subject: [PATCH 364/935] virtio-pci: add notification trace points
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200925125147.26943-6-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220321153037.3622127-3-alex.bennee@linaro.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/trace-events | 7 ++++++-
 hw/virtio/virtio-pci.c | 3 +++
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events
index 333348d9d5..7a6576d833 100644
--- a/hw/virtio/trace-events
+++ b/hw/virtio/trace-events
@@ -89,7 +89,12 @@ virtio_mmio_guest_page(uint64_t size, int shift) "guest page size 0x%" PRIx64 "
 virtio_mmio_queue_write(uint64_t value, int max_size) "mmio_queue write 0x%" PRIx64 " max %d"
 virtio_mmio_setting_irq(int level) "virtio_mmio setting IRQ %d"
 
-# virtio-iommu.c
+# virtio-pci.c
+virtio_pci_notify(uint16_t vector) "virtio_pci_notify vec 0x%x"
+virtio_pci_notify_write(uint64_t addr, uint64_t val, unsigned int size) "0x%" PRIx64" = 0x%" PRIx64 " (%d)"
+virtio_pci_notify_write_pio(uint64_t addr, uint64_t val, unsigned int size) "0x%" PRIx64" = 0x%" PRIx64 " (%d)"
+
+# hw/virtio/virtio-iommu.c
 virtio_iommu_device_reset(void) "reset!"
 virtio_iommu_system_reset(void) "system reset!"
 virtio_iommu_get_features(uint64_t features) "device supports features=0x%"PRIx64
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index 602be7f83d..0566ad7d00 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -38,6 +38,7 @@
 #include "hw/virtio/virtio-bus.h"
 #include "qapi/visitor.h"
 #include "sysemu/replay.h"
+#include "trace.h"
 
 #define VIRTIO_PCI_REGION_SIZE(dev)     VIRTIO_PCI_CONFIG_OFF(msix_present(dev))
 
@@ -1380,6 +1381,7 @@ static void virtio_pci_notify_write(void *opaque, hwaddr addr,
     unsigned queue = addr / virtio_pci_queue_mem_mult(proxy);
 
     if (vdev != NULL && queue < VIRTIO_QUEUE_MAX) {
+        trace_virtio_pci_notify_write(addr, val, size);
         virtio_queue_notify(vdev, queue);
     }
 }
@@ -1393,6 +1395,7 @@ static void virtio_pci_notify_write_pio(void *opaque, hwaddr addr,
     unsigned queue = val;
 
     if (vdev != NULL && queue < VIRTIO_QUEUE_MAX) {
+        trace_virtio_pci_notify_write_pio(addr, val, size);
         virtio_queue_notify(vdev, queue);
     }
 }

From 6ca6d8ee9dd053eaa625aad48d95d12f4418ef4c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Mon, 21 Mar 2022 15:30:27 +0000
Subject: [PATCH 365/935] hw/virtio: add vhost_user_[read|write] trace points
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

These are useful when trying to debug the initial vhost-user
negotiation, especially when it hard to get logging from the low level
library on the other side.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Message-Id: <20220321153037.3622127-4-alex.bennee@linaro.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 hw/virtio/trace-events | 2 ++
 hw/virtio/vhost-user.c | 4 ++++
 2 files changed, 6 insertions(+)

diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events
index 7a6576d833..65e3b704ec 100644
--- a/hw/virtio/trace-events
+++ b/hw/virtio/trace-events
@@ -21,6 +21,8 @@ vhost_user_set_mem_table_withfd(int index, const char *name, uint64_t memory_siz
 vhost_user_postcopy_waker(const char *rb, uint64_t rb_offset) "%s + 0x%"PRIx64
 vhost_user_postcopy_waker_found(uint64_t client_addr) "0x%"PRIx64
 vhost_user_postcopy_waker_nomatch(const char *rb, uint64_t rb_offset) "%s + 0x%"PRIx64
+vhost_user_read(uint32_t req, uint32_t flags) "req:%d flags:0x%"PRIx32""
+vhost_user_write(uint32_t req, uint32_t flags) "req:%d flags:0x%"PRIx32""
 
 # vhost-vdpa.c
 vhost_vdpa_dma_map(void *vdpa, int fd, uint32_t msg_type, uint64_t iova, uint64_t size, uint64_t uaddr, uint8_t perm, uint8_t type) "vdpa:%p fd: %d msg_type: %"PRIu32" iova: 0x%"PRIx64" size: 0x%"PRIx64" uaddr: 0x%"PRIx64" perm: 0x%"PRIx8" type: %"PRIu8
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index afd51f79b3..6c8f722262 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -489,6 +489,8 @@ static int vhost_user_write(struct vhost_dev *dev, VhostUserMsg *msg,
         return ret < 0 ? -saved_errno : -EIO;
     }
 
+    trace_vhost_user_write(msg->hdr.request, msg->hdr.flags);
+
     return 0;
 }
 
@@ -542,6 +544,8 @@ static int vhost_user_set_log_base(struct vhost_dev *dev, uint64_t base,
         }
     }
 
+    trace_vhost_user_read(msg.hdr.request, msg.hdr.flags);
+
     return 0;
 }
 

From 43725d4844cc516e8c1b92c7903aeb0e88360f1b Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Mon, 21 Mar 2022 15:30:28 +0000
Subject: [PATCH 366/935] docs: vhost-user: clean up request/reply description
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

It is not necessary to mention which side is sending/receiving
each payload; it is more interesting to say which is the request
and which is the reply.  This also matches what vhost-user-gpu.rst
already does.

While at it, ensure that all messages list both the request and
the reply payload.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20210226143413.188046-2-pbonzini@redhat.com>
Message-Id: <20220321153037.3622127-5-alex.bennee@linaro.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 docs/interop/vhost-user.rst | 163 +++++++++++++++++++++---------------
 1 file changed, 95 insertions(+), 68 deletions(-)

diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
index f9e721ba5f..1039e7a34e 100644
--- a/docs/interop/vhost-user.rst
+++ b/docs/interop/vhost-user.rst
@@ -866,8 +866,8 @@ Master message types
 ``VHOST_USER_GET_FEATURES``
   :id: 1
   :equivalent ioctl: ``VHOST_GET_FEATURES``
-  :master payload: N/A
-  :slave payload: ``u64``
+  :request payload: N/A
+  :reply payload: ``u64``
 
   Get from the underlying vhost implementation the features bitmask.
   Feature bit ``VHOST_USER_F_PROTOCOL_FEATURES`` signals slave support
@@ -877,7 +877,8 @@ Master message types
 ``VHOST_USER_SET_FEATURES``
   :id: 2
   :equivalent ioctl: ``VHOST_SET_FEATURES``
-  :master payload: ``u64``
+  :request payload: ``u64``
+  :reply payload: N/A
 
   Enable features in the underlying vhost implementation using a
   bitmask.  Feature bit ``VHOST_USER_F_PROTOCOL_FEATURES`` signals
@@ -887,8 +888,8 @@ Master message types
 ``VHOST_USER_GET_PROTOCOL_FEATURES``
   :id: 15
   :equivalent ioctl: ``VHOST_GET_FEATURES``
-  :master payload: N/A
-  :slave payload: ``u64``
+  :request payload: N/A
+  :reply payload: ``u64``
 
   Get the protocol feature bitmask from the underlying vhost
   implementation.  Only legal if feature bit
@@ -903,7 +904,8 @@ Master message types
 ``VHOST_USER_SET_PROTOCOL_FEATURES``
   :id: 16
   :equivalent ioctl: ``VHOST_SET_FEATURES``
-  :master payload: ``u64``
+  :request payload: ``u64``
+  :reply payload: N/A
 
   Enable protocol features in the underlying vhost implementation.
 
@@ -917,7 +919,8 @@ Master message types
 ``VHOST_USER_SET_OWNER``
   :id: 3
   :equivalent ioctl: ``VHOST_SET_OWNER``
-  :master payload: N/A
+  :request payload: N/A
+  :reply payload: N/A
 
   Issued when a new connection is established. It sets the current
   *master* as an owner of the session. This can be used on the *slave*
@@ -925,7 +928,8 @@ Master message types
 
 ``VHOST_USER_RESET_OWNER``
   :id: 4
-  :master payload: N/A
+  :request payload: N/A
+  :reply payload: N/A
 
 .. admonition:: Deprecated
 
@@ -938,8 +942,8 @@ Master message types
 ``VHOST_USER_SET_MEM_TABLE``
   :id: 5
   :equivalent ioctl: ``VHOST_SET_MEM_TABLE``
-  :master payload: memory regions description
-  :slave payload: (postcopy only) memory regions description
+  :request payload: memory regions description
+  :reply payload: (postcopy only) memory regions description
 
   Sets the memory map regions on the slave so it can translate the
   vring addresses. In the ancillary data there is an array of file
@@ -962,8 +966,8 @@ Master message types
 ``VHOST_USER_SET_LOG_BASE``
   :id: 6
   :equivalent ioctl: ``VHOST_SET_LOG_BASE``
-  :master payload: u64
-  :slave payload: N/A
+  :request payload: u64
+  :reply payload: N/A
 
   Sets logging shared memory space.
 
@@ -975,44 +979,48 @@ Master message types
 ``VHOST_USER_SET_LOG_FD``
   :id: 7
   :equivalent ioctl: ``VHOST_SET_LOG_FD``
-  :master payload: N/A
+  :request payload: N/A
+  :reply payload: N/A
 
   Sets the logging file descriptor, which is passed as ancillary data.
 
 ``VHOST_USER_SET_VRING_NUM``
   :id: 8
   :equivalent ioctl: ``VHOST_SET_VRING_NUM``
-  :master payload: vring state description
+  :request payload: vring state description
+  :reply payload: N/A
 
   Set the size of the queue.
 
 ``VHOST_USER_SET_VRING_ADDR``
   :id: 9
   :equivalent ioctl: ``VHOST_SET_VRING_ADDR``
-  :master payload: vring address description
-  :slave payload: N/A
+  :request payload: vring address description
+  :reply payload: N/A
 
   Sets the addresses of the different aspects of the vring.
 
 ``VHOST_USER_SET_VRING_BASE``
   :id: 10
   :equivalent ioctl: ``VHOST_SET_VRING_BASE``
-  :master payload: vring state description
+  :request payload: vring state description
+  :reply payload: N/A
 
   Sets the base offset in the available vring.
 
 ``VHOST_USER_GET_VRING_BASE``
   :id: 11
   :equivalent ioctl: ``VHOST_USER_GET_VRING_BASE``
-  :master payload: vring state description
-  :slave payload: vring state description
+  :request payload: vring state description
+  :reply payload: vring state description
 
   Get the available vring base offset.
 
 ``VHOST_USER_SET_VRING_KICK``
   :id: 12
   :equivalent ioctl: ``VHOST_SET_VRING_KICK``
-  :master payload: ``u64``
+  :request payload: ``u64``
+  :reply payload: N/A
 
   Set the event file descriptor for adding buffers to the vring. It is
   passed in the ancillary data.
@@ -1030,7 +1038,8 @@ Master message types
 ``VHOST_USER_SET_VRING_CALL``
   :id: 13
   :equivalent ioctl: ``VHOST_SET_VRING_CALL``
-  :master payload: ``u64``
+  :request payload: ``u64``
+  :reply payload: N/A
 
   Set the event file descriptor to signal when buffers are used. It is
   passed in the ancillary data.
@@ -1048,7 +1057,8 @@ Master message types
 ``VHOST_USER_SET_VRING_ERR``
   :id: 14
   :equivalent ioctl: ``VHOST_SET_VRING_ERR``
-  :master payload: ``u64``
+  :request payload: ``u64``
+  :reply payload: N/A
 
   Set the event file descriptor to signal when error occurs. It is
   passed in the ancillary data.
@@ -1065,8 +1075,8 @@ Master message types
 ``VHOST_USER_GET_QUEUE_NUM``
   :id: 17
   :equivalent ioctl: N/A
-  :master payload: N/A
-  :slave payload: u64
+  :request payload: N/A
+  :reply payload: u64
 
   Query how many queues the backend supports.
 
@@ -1077,7 +1087,8 @@ Master message types
 ``VHOST_USER_SET_VRING_ENABLE``
   :id: 18
   :equivalent ioctl: N/A
-  :master payload: vring state description
+  :request payload: vring state description
+  :reply payload: N/A
 
   Signal slave to enable or disable corresponding vring.
 
@@ -1087,7 +1098,8 @@ Master message types
 ``VHOST_USER_SEND_RARP``
   :id: 19
   :equivalent ioctl: N/A
-  :master payload: ``u64``
+  :request payload: ``u64``
+  :reply payload: N/A
 
   Ask vhost user backend to broadcast a fake RARP to notify the migration
   is terminated for guest that does not support GUEST_ANNOUNCE.
@@ -1102,7 +1114,8 @@ Master message types
 ``VHOST_USER_NET_SET_MTU``
   :id: 20
   :equivalent ioctl: N/A
-  :master payload: ``u64``
+  :request payload: ``u64``
+  :reply payload: N/A
 
   Set host MTU value exposed to the guest.
 
@@ -1119,7 +1132,8 @@ Master message types
 ``VHOST_USER_SET_SLAVE_REQ_FD``
   :id: 21
   :equivalent ioctl: N/A
-  :master payload: N/A
+  :request payload: N/A
+  :reply payload: N/A
 
   Set the socket file descriptor for slave initiated requests. It is passed
   in the ancillary data.
@@ -1134,8 +1148,8 @@ Master message types
 ``VHOST_USER_IOTLB_MSG``
   :id: 22
   :equivalent ioctl: N/A (equivalent to ``VHOST_IOTLB_MSG`` message type)
-  :master payload: ``struct vhost_iotlb_msg``
-  :slave payload: ``u64``
+  :request payload: ``struct vhost_iotlb_msg``
+  :reply payload: ``u64``
 
   Send IOTLB messages with ``struct vhost_iotlb_msg`` as payload.
 
@@ -1149,7 +1163,8 @@ Master message types
 ``VHOST_USER_SET_VRING_ENDIAN``
   :id: 23
   :equivalent ioctl: ``VHOST_SET_VRING_ENDIAN``
-  :master payload: vring state description
+  :request payload: vring state description
+  :reply payload: N/A
 
   Set the endianness of a VQ for legacy devices. Little-endian is
   indicated with state.num set to 0 and big-endian is indicated with
@@ -1164,8 +1179,8 @@ Master message types
 ``VHOST_USER_GET_CONFIG``
   :id: 24
   :equivalent ioctl: N/A
-  :master payload: virtio device config space
-  :slave payload: virtio device config space
+  :request payload: virtio device config space
+  :reply payload: virtio device config space
 
   When ``VHOST_USER_PROTOCOL_F_CONFIG`` is negotiated, this message is
   submitted by the vhost-user master to fetch the contents of the
@@ -1178,8 +1193,8 @@ Master message types
 ``VHOST_USER_SET_CONFIG``
   :id: 25
   :equivalent ioctl: N/A
-  :master payload: virtio device config space
-  :slave payload: N/A
+  :request payload: virtio device config space
+  :reply payload: N/A
 
   When ``VHOST_USER_PROTOCOL_F_CONFIG`` is negotiated, this message is
   submitted by the vhost-user master when the Guest changes the virtio
@@ -1191,8 +1206,8 @@ Master message types
 ``VHOST_USER_CREATE_CRYPTO_SESSION``
   :id: 26
   :equivalent ioctl: N/A
-  :master payload: crypto session description
-  :slave payload: crypto session description
+  :request payload: crypto session description
+  :reply payload: crypto session description
 
   Create a session for crypto operation. The server side must return
   the session id, 0 or positive for success, negative for failure.
@@ -1204,7 +1219,8 @@ Master message types
 ``VHOST_USER_CLOSE_CRYPTO_SESSION``
   :id: 27
   :equivalent ioctl: N/A
-  :master payload: ``u64``
+  :request payload: ``u64``
+  :reply payload: N/A
 
   Close a session for crypto operation which was previously
   created by ``VHOST_USER_CREATE_CRYPTO_SESSION``.
@@ -1216,8 +1232,8 @@ Master message types
 
 ``VHOST_USER_POSTCOPY_ADVISE``
   :id: 28
-  :master payload: N/A
-  :slave payload: userfault fd
+  :request payload: N/A
+  :reply payload: userfault fd
 
   When ``VHOST_USER_PROTOCOL_F_PAGEFAULT`` is supported, the master
   advises slave that a migration with postcopy enabled is underway,
@@ -1226,7 +1242,8 @@ Master message types
 
 ``VHOST_USER_POSTCOPY_LISTEN``
   :id: 29
-  :master payload: N/A
+  :request payload: N/A
+  :reply payload: N/A
 
   Master advises slave that a transition to postcopy mode has
   happened.  The slave must ensure that shared memory is registered
@@ -1237,10 +1254,11 @@ Master message types
 
 ``VHOST_USER_POSTCOPY_END``
   :id: 30
-  :slave payload: ``u64``
+  :request payload: N/A
+  :reply payload: ``u64``
 
   Master advises that postcopy migration has now completed.  The slave
-  must disable the userfaultfd. The response is an acknowledgement
+  must disable the userfaultfd. The reply is an acknowledgement
   only.
 
   When ``VHOST_USER_PROTOCOL_F_PAGEFAULT`` is supported, this message
@@ -1252,7 +1270,8 @@ Master message types
 ``VHOST_USER_GET_INFLIGHT_FD``
   :id: 31
   :equivalent ioctl: N/A
-  :master payload: inflight description
+  :request payload: inflight description
+  :reply payload: N/A
 
   When ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD`` protocol feature has
   been successfully negotiated, this message is submitted by master to
@@ -1263,7 +1282,8 @@ Master message types
 ``VHOST_USER_SET_INFLIGHT_FD``
   :id: 32
   :equivalent ioctl: N/A
-  :master payload: inflight description
+  :request payload: inflight description
+  :reply payload: N/A
 
   When ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD`` protocol feature has
   been successfully negotiated, this message is submitted by master to
@@ -1273,7 +1293,8 @@ Master message types
 ``VHOST_USER_GPU_SET_SOCKET``
   :id: 33
   :equivalent ioctl: N/A
-  :master payload: N/A
+  :request payload: N/A
+  :reply payload: N/A
 
   Sets the GPU protocol socket file descriptor, which is passed as
   ancillary data. The GPU protocol is used to inform the master of
@@ -1282,8 +1303,8 @@ Master message types
 ``VHOST_USER_RESET_DEVICE``
   :id: 34
   :equivalent ioctl: N/A
-  :master payload: N/A
-  :slave payload: N/A
+  :request payload: N/A
+  :reply payload: N/A
 
   Ask the vhost user backend to disable all rings and reset all
   internal device state to the initial state, ready to be
@@ -1296,8 +1317,8 @@ Master message types
 ``VHOST_USER_VRING_KICK``
   :id: 35
   :equivalent ioctl: N/A
-  :slave payload: vring state description
-  :master payload: N/A
+  :request payload: vring state description
+  :reply payload: N/A
 
   When the ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` protocol
   feature has been successfully negotiated, this message may be
@@ -1310,7 +1331,8 @@ Master message types
 ``VHOST_USER_GET_MAX_MEM_SLOTS``
   :id: 36
   :equivalent ioctl: N/A
-  :slave payload: u64
+  :request payload: N/A
+  :reply payload: u64
 
   When the ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol
   feature has been successfully negotiated, this message is submitted
@@ -1323,7 +1345,8 @@ Master message types
 ``VHOST_USER_ADD_MEM_REG``
   :id: 37
   :equivalent ioctl: N/A
-  :slave payload: single memory region description
+  :request payload: N/A
+  :reply payload: single memory region description
 
   When the ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol
   feature has been successfully negotiated, this message is submitted
@@ -1346,7 +1369,8 @@ Master message types
 ``VHOST_USER_REM_MEM_REG``
   :id: 38
   :equivalent ioctl: N/A
-  :slave payload: single memory region description
+  :request payload: N/A
+  :reply payload: single memory region description
 
   When the ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol
   feature has been successfully negotiated, this message is submitted
@@ -1369,8 +1393,8 @@ Master message types
 ``VHOST_USER_SET_STATUS``
   :id: 39
   :equivalent ioctl: VHOST_VDPA_SET_STATUS
-  :slave payload: N/A
-  :master payload: ``u64``
+  :request payload: ``u64``
+  :reply payload: N/A
 
   When the ``VHOST_USER_PROTOCOL_F_STATUS`` protocol feature has been
   successfully negotiated, this message is submitted by the master to
@@ -1380,8 +1404,8 @@ Master message types
 ``VHOST_USER_GET_STATUS``
   :id: 40
   :equivalent ioctl: VHOST_VDPA_GET_STATUS
-  :slave payload: ``u64``
-  :master payload: N/A
+  :request payload: N/A
+  :reply payload: ``u64``
 
   When the ``VHOST_USER_PROTOCOL_F_STATUS`` protocol feature has been
   successfully negotiated, this message is submitted by the master to
@@ -1392,11 +1416,14 @@ Master message types
 Slave message types
 -------------------
 
+For this type of message, the request is sent by the slave and the reply
+is sent by the master.
+
 ``VHOST_USER_SLAVE_IOTLB_MSG``
   :id: 1
   :equivalent ioctl: N/A (equivalent to ``VHOST_IOTLB_MSG`` message type)
-  :slave payload: ``struct vhost_iotlb_msg``
-  :master payload: N/A
+  :request payload: ``struct vhost_iotlb_msg``
+  :reply payload: N/A
 
   Send IOTLB messages with ``struct vhost_iotlb_msg`` as payload.
   Slave sends such requests to notify of an IOTLB miss, or an IOTLB
@@ -1410,8 +1437,8 @@ Slave message types
 ``VHOST_USER_SLAVE_CONFIG_CHANGE_MSG``
   :id: 2
   :equivalent ioctl: N/A
-  :slave payload: N/A
-  :master payload: N/A
+  :request payload: N/A
+  :reply payload: N/A
 
   When ``VHOST_USER_PROTOCOL_F_CONFIG`` is negotiated, vhost-user
   slave sends such messages to notify that the virtio device's
@@ -1425,8 +1452,8 @@ Slave message types
 ``VHOST_USER_SLAVE_VRING_HOST_NOTIFIER_MSG``
   :id: 3
   :equivalent ioctl: N/A
-  :slave payload: vring area description
-  :master payload: N/A
+  :request payload: vring area description
+  :reply payload: N/A
 
   Sets host notifier for a specified queue. The queue index is
   contained in the ``u64`` field of the vring area description. The
@@ -1448,8 +1475,8 @@ Slave message types
 ``VHOST_USER_SLAVE_VRING_CALL``
   :id: 4
   :equivalent ioctl: N/A
-  :slave payload: vring state description
-  :master payload: N/A
+  :request payload: vring state description
+  :reply payload: N/A
 
   When the ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` protocol
   feature has been successfully negotiated, this message may be
@@ -1462,8 +1489,8 @@ Slave message types
 ``VHOST_USER_SLAVE_VRING_ERR``
   :id: 5
   :equivalent ioctl: N/A
-  :slave payload: vring state description
-  :master payload: N/A
+  :request payload: vring state description
+  :reply payload: N/A
 
   When the ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` protocol
   feature has been successfully negotiated, this message may be
@@ -1489,7 +1516,7 @@ client MUST respond with a Payload ``VhostUserMsg`` indicating success
 or failure. The payload should be set to zero on success or non-zero
 on failure, unless the message already has an explicit reply body.
 
-The response payload gives QEMU a deterministic indication of the result
+The reply payload gives QEMU a deterministic indication of the result
 of the command. Today, QEMU is expected to terminate the main vhost-user
 loop upon receiving such errors. In future, qemu could be taught to be more
 resilient for selective requests.

From 0647098d2ed69171b8bdce495cfdd1c5223010f1 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Mon, 21 Mar 2022 15:30:29 +0000
Subject: [PATCH 367/935] docs: vhost-user: rewrite section on ring state
 machine

This section is using the word "back-end" to refer to the
"slave's back-end", and talking about the "client" for
what the rest of the document calls the "slave".

Rework it to free the use of the term "back-end", which in
the next patch will replace "slave".

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20210226143413.188046-3-pbonzini@redhat.com>
Message-Id: <20220321153037.3622127-6-alex.bennee@linaro.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 docs/interop/vhost-user.rst | 46 +++++++++++++++++--------------------
 1 file changed, 21 insertions(+), 25 deletions(-)

diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
index 1039e7a34e..1151326d6f 100644
--- a/docs/interop/vhost-user.rst
+++ b/docs/interop/vhost-user.rst
@@ -332,40 +332,36 @@ bit was dedicated for this purpose::
 
   #define VHOST_USER_F_PROTOCOL_FEATURES 30
 
-Starting and stopping rings
----------------------------
+Ring states
+-----------
 
-Client must only process each ring when it is started.
+Rings can be in one of three states:
 
-Client must only pass data between the ring and the backend, when the
-ring is enabled.
+* stopped: the slave must not process the ring at all.
 
-If ring is started but disabled, client must process the ring without
-talking to the backend.
+* started but disabled: the slave must process the ring without
+  causing any side effects.  For example, for a networking device,
+  in the disabled state the slave must not supply any new RX packets,
+  but must process and discard any TX packets.
 
-For example, for a networking device, in the disabled state client
-must not supply any new RX packets, but must process and discard any
-TX packets.
+* started and enabled.
+
+Each ring is initialized in a stopped state.  The slave must start
+ring upon receiving a kick (that is, detecting that file descriptor is
+readable) on the descriptor specified by ``VHOST_USER_SET_VRING_KICK``
+or receiving the in-band message ``VHOST_USER_VRING_KICK`` if negotiated,
+and stop ring upon receiving ``VHOST_USER_GET_VRING_BASE``.
+
+Rings can be enabled or disabled by ``VHOST_USER_SET_VRING_ENABLE``.
 
 If ``VHOST_USER_F_PROTOCOL_FEATURES`` has not been negotiated, the
-ring is initialized in an enabled state.
+ring starts directly in the enabled state.
 
 If ``VHOST_USER_F_PROTOCOL_FEATURES`` has been negotiated, the ring is
-initialized in a disabled state. Client must not pass data to/from the
-backend until ring is enabled by ``VHOST_USER_SET_VRING_ENABLE`` with
-parameter 1, or after it has been disabled by
-``VHOST_USER_SET_VRING_ENABLE`` with parameter 0.
+initialized in a disabled state and is enabled by
+``VHOST_USER_SET_VRING_ENABLE`` with parameter 1.
 
-Each ring is initialized in a stopped state, client must not process
-it until ring is started, or after it has been stopped.
-
-Client must start ring upon receiving a kick (that is, detecting that
-file descriptor is readable) on the descriptor specified by
-``VHOST_USER_SET_VRING_KICK`` or receiving the in-band message
-``VHOST_USER_VRING_KICK`` if negotiated, and stop ring upon receiving
-``VHOST_USER_GET_VRING_BASE``.
-
-While processing the rings (whether they are enabled or not), client
+While processing the rings (whether they are enabled or not), the slave
 must support changing some configuration aspects on the fly.
 
 Multiple queue support

From bd59f2a1823a458ac27c09e1f27f30098745381e Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Mon, 21 Mar 2022 15:30:30 +0000
Subject: [PATCH 368/935] docs: vhost-user: replace master/slave with
 front-end/back-end

This matches the nomenclature that is generally used.  Also commonly used
is client/server, but it is not as clear because sometimes the front-end
exposes a passive (server) socket that the back-end connects to.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20210226143413.188046-4-pbonzini@redhat.com>
Message-Id: <20220321153037.3622127-7-alex.bennee@linaro.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 docs/interop/vhost-user-gpu.rst |  10 +-
 docs/interop/vhost-user.rst     | 350 ++++++++++++++++----------------
 2 files changed, 180 insertions(+), 180 deletions(-)

diff --git a/docs/interop/vhost-user-gpu.rst b/docs/interop/vhost-user-gpu.rst
index 71a2c52b31..1640553729 100644
--- a/docs/interop/vhost-user-gpu.rst
+++ b/docs/interop/vhost-user-gpu.rst
@@ -13,10 +13,10 @@ Introduction
 ============
 
 The vhost-user-gpu protocol is aiming at sharing the rendering result
-of a virtio-gpu, done from a vhost-user slave process to a vhost-user
-master process (such as QEMU). It bears a resemblance to a display
+of a virtio-gpu, done from a vhost-user back-end process to a vhost-user
+front-end process (such as QEMU). It bears a resemblance to a display
 server protocol, if you consider QEMU as the display server and the
-slave as the client, but in a very limited way. Typically, it will
+back-end as the client, but in a very limited way. Typically, it will
 work by setting a scanout/display configuration, before sending flush
 events for the display updates. It will also update the cursor shape
 and position.
@@ -26,8 +26,8 @@ socket ancillary data to share opened file descriptors (DMABUF fds or
 shared memory). The socket is usually obtained via
 ``VHOST_USER_GPU_SET_SOCKET``.
 
-Requests are sent by the *slave*, and the optional replies by the
-*master*.
+Requests are sent by the *back-end*, and the optional replies by the
+*front-end*.
 
 Wire format
 ===========
diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
index 1151326d6f..9159476678 100644
--- a/docs/interop/vhost-user.rst
+++ b/docs/interop/vhost-user.rst
@@ -23,19 +23,19 @@ space process on the same host. It uses communication over a Unix
 domain socket to share file descriptors in the ancillary data of the
 message.
 
-The protocol defines 2 sides of the communication, *master* and
-*slave*. *Master* is the application that shares its virtqueues, in
-our case QEMU. *Slave* is the consumer of the virtqueues.
+The protocol defines 2 sides of the communication, *front-end* and
+*back-end*. The *front-end* is the application that shares its virtqueues, in
+our case QEMU. The *back-end* is the consumer of the virtqueues.
 
-In the current implementation QEMU is the *master*, and the *slave* is
-the external process consuming the virtio queues, for example a
+In the current implementation QEMU is the *front-end*, and the *back-end*
+is the external process consuming the virtio queues, for example a
 software Ethernet switch running in user space, such as Snabbswitch,
-or a block device backend processing read & write to a virtual
-disk. In order to facilitate interoperability between various backend
+or a block device back-end processing read & write to a virtual
+disk. In order to facilitate interoperability between various back-end
 implementations, it is recommended to follow the :ref:`Backend program
 conventions <backend_conventions>`.
 
-*Master* and *slave* can be either a client (i.e. connecting) or
+The *front-end* and *back-end* can be either a client (i.e. connecting) or
 server (listening) in the socket communication.
 
 Support for platforms other than Linux
@@ -77,7 +77,7 @@ Header
 :flags: 32-bit bit field
 
 - Lower 2 bits are the version (currently 0x01)
-- Bit 2 is the reply flag - needs to be sent on each reply from the slave
+- Bit 2 is the reply flag - needs to be sent on each reply from the back-end
 - Bit 3 is the need_reply flag - see :ref:`REPLY_ACK <reply_ack>` for
   details.
 
@@ -222,8 +222,8 @@ Virtio device config space
 :size: a 32-bit configuration space access size in bytes
 
 :flags: a 32-bit value:
-  - 0: Vhost master messages used for writeable fields
-  - 1: Vhost master messages used for live migration
+  - 0: Vhost front-end messages used for writeable fields
+  - 1: Vhost front-end messages used for live migration
 
 :payload: Size bytes array holding the contents of the virtio
           device's configuration space
@@ -290,8 +290,8 @@ vhost for the Linux Kernel. Most messages that can be sent via the
 Unix domain socket implementing vhost-user have an equivalent ioctl to
 the kernel implementation.
 
-The communication consists of *master* sending message requests and
-*slave* sending message replies. Most of the requests don't require
+The communication consists of the *front-end* sending message requests and
+the *back-end* sending message replies. Most of the requests don't require
 replies. Here is a list of the ones that do:
 
 * ``VHOST_USER_GET_FEATURES``
@@ -305,7 +305,7 @@ replies. Here is a list of the ones that do:
    :ref:`REPLY_ACK <reply_ack>`
        The section on ``REPLY_ACK`` protocol extension.
 
-There are several messages that the master sends with file descriptors passed
+There are several messages that the front-end sends with file descriptors passed
 in the ancillary data:
 
 * ``VHOST_USER_ADD_MEM_REG``
@@ -318,16 +318,16 @@ in the ancillary data:
 * ``VHOST_USER_SET_SLAVE_REQ_FD``
 * ``VHOST_USER_SET_INFLIGHT_FD`` (if ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD``)
 
-If *master* is unable to send the full message or receives a wrong
+If *front-end* is unable to send the full message or receives a wrong
 reply it will close the connection. An optional reconnection mechanism
 can be implemented.
 
-If *slave* detects some error such as incompatible features, it may also
+If *back-end* detects some error such as incompatible features, it may also
 close the connection. This should only happen in exceptional circumstances.
 
 Any protocol extensions are gated by protocol feature bits, which
-allows full backwards compatibility on both master and slave.  As
-older slaves don't support negotiating protocol features, a feature
+allows full backwards compatibility on both front-end and back-end.  As
+older back-ends don't support negotiating protocol features, a feature
 bit was dedicated for this purpose::
 
   #define VHOST_USER_F_PROTOCOL_FEATURES 30
@@ -337,16 +337,16 @@ Ring states
 
 Rings can be in one of three states:
 
-* stopped: the slave must not process the ring at all.
+* stopped: the back-end must not process the ring at all.
 
-* started but disabled: the slave must process the ring without
+* started but disabled: the back-end must process the ring without
   causing any side effects.  For example, for a networking device,
-  in the disabled state the slave must not supply any new RX packets,
+  in the disabled state the back-end must not supply any new RX packets,
   but must process and discard any TX packets.
 
 * started and enabled.
 
-Each ring is initialized in a stopped state.  The slave must start
+Each ring is initialized in a stopped state.  The back-end must start
 ring upon receiving a kick (that is, detecting that file descriptor is
 readable) on the descriptor specified by ``VHOST_USER_SET_VRING_KICK``
 or receiving the in-band message ``VHOST_USER_VRING_KICK`` if negotiated,
@@ -361,53 +361,53 @@ If ``VHOST_USER_F_PROTOCOL_FEATURES`` has been negotiated, the ring is
 initialized in a disabled state and is enabled by
 ``VHOST_USER_SET_VRING_ENABLE`` with parameter 1.
 
-While processing the rings (whether they are enabled or not), the slave
+While processing the rings (whether they are enabled or not), the back-end
 must support changing some configuration aspects on the fly.
 
 Multiple queue support
 ----------------------
 
-Many devices have a fixed number of virtqueues.  In this case the master
+Many devices have a fixed number of virtqueues.  In this case the front-end
 already knows the number of available virtqueues without communicating with the
-slave.
+back-end.
 
 Some devices do not have a fixed number of virtqueues.  Instead the maximum
-number of virtqueues is chosen by the slave.  The number can depend on host
-resource availability or slave implementation details.  Such devices are called
+number of virtqueues is chosen by the back-end.  The number can depend on host
+resource availability or back-end implementation details.  Such devices are called
 multiple queue devices.
 
-Multiple queue support allows the slave to advertise the maximum number of
-queues.  This is treated as a protocol extension, hence the slave has to
+Multiple queue support allows the back-end to advertise the maximum number of
+queues.  This is treated as a protocol extension, hence the back-end has to
 implement protocol features first. The multiple queues feature is supported
 only when the protocol feature ``VHOST_USER_PROTOCOL_F_MQ`` (bit 0) is set.
 
-The max number of queues the slave supports can be queried with message
-``VHOST_USER_GET_QUEUE_NUM``. Master should stop when the number of requested
+The max number of queues the back-end supports can be queried with message
+``VHOST_USER_GET_QUEUE_NUM``. Front-end should stop when the number of requested
 queues is bigger than that.
 
-As all queues share one connection, the master uses a unique index for each
+As all queues share one connection, the front-end uses a unique index for each
 queue in the sent message to identify a specified queue.
 
-The master enables queues by sending message ``VHOST_USER_SET_VRING_ENABLE``.
+The front-end enables queues by sending message ``VHOST_USER_SET_VRING_ENABLE``.
 vhost-user-net has historically automatically enabled the first queue pair.
 
-Slaves should always implement the ``VHOST_USER_PROTOCOL_F_MQ`` protocol
+Back-ends should always implement the ``VHOST_USER_PROTOCOL_F_MQ`` protocol
 feature, even for devices with a fixed number of virtqueues, since it is simple
 to implement and offers a degree of introspection.
 
-Masters must not rely on the ``VHOST_USER_PROTOCOL_F_MQ`` protocol feature for
+Front-ends must not rely on the ``VHOST_USER_PROTOCOL_F_MQ`` protocol feature for
 devices with a fixed number of virtqueues.  Only true multiqueue devices
 require this protocol feature.
 
 Migration
 ---------
 
-During live migration, the master may need to track the modifications
-the slave makes to the memory mapped regions. The client should mark
+During live migration, the front-end may need to track the modifications
+the back-end makes to the memory mapped regions. The front-end should mark
 the dirty pages in a log. Once it complies to this logging, it may
 declare the ``VHOST_F_LOG_ALL`` vhost feature.
 
-To start/stop logging of data/used ring writes, server may send
+To start/stop logging of data/used ring writes, the front-end may send
 messages ``VHOST_USER_SET_FEATURES`` with ``VHOST_F_LOG_ALL`` and
 ``VHOST_USER_SET_VRING_ADDR`` with ``VHOST_VRING_F_LOG`` in ring's
 flags set to 1/0, respectively.
@@ -421,7 +421,7 @@ Dirty pages are of size::
   #define VHOST_LOG_PAGE 0x1000
 
 The log memory fd is provided in the ancillary data of
-``VHOST_USER_SET_LOG_BASE`` message when the slave has
+``VHOST_USER_SET_LOG_BASE`` message when the back-end has
 ``VHOST_USER_PROTOCOL_F_LOG_SHMFD`` protocol feature.
 
 The size of the log is supplied as part of ``VhostUserMsg`` which
@@ -447,26 +447,26 @@ the bit offset of the last byte of the ring must fall within the size
 supplied by ``VhostUserLog``.
 
 ``VHOST_USER_SET_LOG_FD`` is an optional message with an eventfd in
-ancillary data, it may be used to inform the master that the log has
+ancillary data, it may be used to inform the front-end that the log has
 been modified.
 
 Once the source has finished migration, rings will be stopped by the
 source. No further update must be done before rings are restarted.
 
-In postcopy migration the slave is started before all the memory has
+In postcopy migration the back-end is started before all the memory has
 been received from the source host, and care must be taken to avoid
-accessing pages that have yet to be received.  The slave opens a
+accessing pages that have yet to be received.  The back-end opens a
 'userfault'-fd and registers the memory with it; this fd is then
-passed back over to the master.  The master services requests on the
+passed back over to the front-end.  The front-end services requests on the
 userfaultfd for pages that are accessed and when the page is available
 it performs WAKE ioctl's on the userfaultfd to wake the stalled
-slave.  The client indicates support for this via the
+back-end.  The front-end indicates support for this via the
 ``VHOST_USER_PROTOCOL_F_PAGEFAULT`` feature.
 
 Memory access
 -------------
 
-The master sends a list of vhost memory regions to the slave using the
+The front-end sends a list of vhost memory regions to the back-end using the
 ``VHOST_USER_SET_MEM_TABLE`` message.  Each region has two base
 addresses: a guest address and a user address.
 
@@ -491,60 +491,60 @@ IOMMU support
 -------------
 
 When the ``VIRTIO_F_IOMMU_PLATFORM`` feature has been negotiated, the
-master sends IOTLB entries update & invalidation by sending
-``VHOST_USER_IOTLB_MSG`` requests to the slave with a ``struct
+front-end sends IOTLB entries update & invalidation by sending
+``VHOST_USER_IOTLB_MSG`` requests to the back-end with a ``struct
 vhost_iotlb_msg`` as payload. For update events, the ``iotlb`` payload
 has to be filled with the update message type (2), the I/O virtual
 address, the size, the user virtual address, and the permissions
 flags. Addresses and size must be within vhost memory regions set via
 the ``VHOST_USER_SET_MEM_TABLE`` request. For invalidation events, the
 ``iotlb`` payload has to be filled with the invalidation message type
-(3), the I/O virtual address and the size. On success, the slave is
+(3), the I/O virtual address and the size. On success, the back-end is
 expected to reply with a zero payload, non-zero otherwise.
 
-The slave relies on the slave communication channel (see :ref:`Slave
-communication <slave_communication>` section below) to send IOTLB miss
+The back-end relies on the back-end communication channel (see :ref:`Back-end
+communication <backend_communication>` section below) to send IOTLB miss
 and access failure events, by sending ``VHOST_USER_SLAVE_IOTLB_MSG``
-requests to the master with a ``struct vhost_iotlb_msg`` as
+requests to the front-end with a ``struct vhost_iotlb_msg`` as
 payload. For miss events, the iotlb payload has to be filled with the
 miss message type (1), the I/O virtual address and the permissions
 flags. For access failure event, the iotlb payload has to be filled
 with the access failure message type (4), the I/O virtual address and
-the permissions flags.  For synchronization purpose, the slave may
-rely on the reply-ack feature, so the master may send a reply when
+the permissions flags.  For synchronization purpose, the back-end may
+rely on the reply-ack feature, so the front-end may send a reply when
 operation is completed if the reply-ack feature is negotiated and
-slaves requests a reply. For miss events, completed operation means
-either master sent an update message containing the IOTLB entry
-containing requested address and permission, or master sent nothing if
+back-ends requests a reply. For miss events, completed operation means
+either front-end sent an update message containing the IOTLB entry
+containing requested address and permission, or front-end sent nothing if
 the IOTLB miss message is invalid (invalid IOVA or permission).
 
-The master isn't expected to take the initiative to send IOTLB update
-messages, as the slave sends IOTLB miss messages for the guest virtual
+The front-end isn't expected to take the initiative to send IOTLB update
+messages, as the back-end sends IOTLB miss messages for the guest virtual
 memory areas it needs to access.
 
-.. _slave_communication:
+.. _backend_communication:
 
-Slave communication
--------------------
+Back-end communication
+----------------------
 
-An optional communication channel is provided if the slave declares
+An optional communication channel is provided if the back-end declares
 ``VHOST_USER_PROTOCOL_F_SLAVE_REQ`` protocol feature, to allow the
-slave to make requests to the master.
+back-end to make requests to the front-end.
 
 The fd is provided via ``VHOST_USER_SET_SLAVE_REQ_FD`` ancillary data.
 
-A slave may then send ``VHOST_USER_SLAVE_*`` messages to the master
+A back-end may then send ``VHOST_USER_SLAVE_*`` messages to the front-end
 using this fd communication channel.
 
 If ``VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD`` protocol feature is
-negotiated, slave can send file descriptors (at most 8 descriptors in
-each message) to master via ancillary data using this fd communication
+negotiated, back-end can send file descriptors (at most 8 descriptors in
+each message) to front-end via ancillary data using this fd communication
 channel.
 
 Inflight I/O tracking
 ---------------------
 
-To support reconnecting after restart or crash, slave may need to
+To support reconnecting after restart or crash, back-end may need to
 resubmit inflight I/Os. If virtqueue is processed in order, we can
 easily achieve that by getting the inflight descriptors from
 descriptor table (split virtqueue) or descriptor ring (packed
@@ -552,18 +552,18 @@ virtqueue). However, it can't work when we process descriptors
 out-of-order because some entries which store the information of
 inflight descriptors in available ring (split virtqueue) or descriptor
 ring (packed virtqueue) might be overridden by new entries. To solve
-this problem, slave need to allocate an extra buffer to store this
-information of inflight descriptors and share it with master for
+this problem, the back-end need to allocate an extra buffer to store this
+information of inflight descriptors and share it with front-end for
 persistent. ``VHOST_USER_GET_INFLIGHT_FD`` and
 ``VHOST_USER_SET_INFLIGHT_FD`` are used to transfer this buffer
-between master and slave. And the format of this buffer is described
+between front-end and back-end. And the format of this buffer is described
 below:
 
 +---------------+---------------+-----+---------------+
 | queue0 region | queue1 region | ... | queueN region |
 +---------------+---------------+-----+---------------+
 
-N is the number of available virtqueues. Slave could get it from num
+N is the number of available virtqueues. The back-end could get it from num
 queues field of ``VhostUserInflight``.
 
 For split virtqueue, queue region can be implemented as:
@@ -595,8 +595,8 @@ For split virtqueue, queue region can be implemented as:
        * Zero value indicates an uninitialized buffer */
       uint16_t version;
 
-      /* The size of DescStateSplit array. It's equal to the virtqueue
-       * size. Slave could get it from queue size field of VhostUserInflight. */
+      /* The size of DescStateSplit array. It's equal to the virtqueue size.
+       * The back-end could get it from queue size field of VhostUserInflight. */
       uint16_t desc_num;
 
       /* The head of list that track the last batch of used descriptors. */
@@ -702,8 +702,8 @@ For packed virtqueue, queue region can be implemented as:
        * Zero value indicates an uninitialized buffer */
       uint16_t version;
 
-      /* The size of DescStatePacked array. It's equal to the virtqueue
-       * size. Slave could get it from queue size field of VhostUserInflight. */
+      /* The size of DescStatePacked array. It's equal to the virtqueue size.
+       * The back-end could get it from queue size field of VhostUserInflight. */
       uint16_t desc_num;
 
       /* The head of free DescStatePacked entry list */
@@ -795,7 +795,7 @@ When reconnecting:
    #. Use ``old_used_wrap_counter`` to calculate the available flags
 
    #. If ``d.flags`` is not equal to the calculated flags value (means
-      slave has submitted the buffer to guest driver before crash, so
+      back-end has submitted the buffer to guest driver before crash, so
       it has to commit the in-progres update), set ``old_free_head``,
       ``old_used_idx``, ``old_used_wrap_counter`` to ``free_head``,
       ``used_idx``, ``used_wrap_counter``
@@ -824,11 +824,11 @@ cause the sending application(s) to block, it is not advised to use
 this feature unless absolutely necessary. It is also considered an
 error to negotiate this feature without also negotiating
 ``VHOST_USER_PROTOCOL_F_SLAVE_REQ`` and ``VHOST_USER_PROTOCOL_F_REPLY_ACK``,
-the former is necessary for getting a message channel from the slave
-to the master, while the latter needs to be used with the in-band
+the former is necessary for getting a message channel from the back-end
+to the front-end, while the latter needs to be used with the in-band
 notification messages to block until they are processed, both to avoid
 blocking later and for proper processing (at least in the simulation
-use case.) As it has no other way of signalling this error, the slave
+use case.) As it has no other way of signalling this error, the back-end
 should close the connection as a response to a
 ``VHOST_USER_SET_PROTOCOL_FEATURES`` message that sets the in-band
 notifications feature flag without the other two.
@@ -856,8 +856,8 @@ Protocol features
   #define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS  15
   #define VHOST_USER_PROTOCOL_F_STATUS               16
 
-Master message types
---------------------
+Front-end message types
+-----------------------
 
 ``VHOST_USER_GET_FEATURES``
   :id: 1
@@ -866,7 +866,7 @@ Master message types
   :reply payload: ``u64``
 
   Get from the underlying vhost implementation the features bitmask.
-  Feature bit ``VHOST_USER_F_PROTOCOL_FEATURES`` signals slave support
+  Feature bit ``VHOST_USER_F_PROTOCOL_FEATURES`` signals back-end support
   for ``VHOST_USER_GET_PROTOCOL_FEATURES`` and
   ``VHOST_USER_SET_PROTOCOL_FEATURES``.
 
@@ -878,7 +878,7 @@ Master message types
 
   Enable features in the underlying vhost implementation using a
   bitmask.  Feature bit ``VHOST_USER_F_PROTOCOL_FEATURES`` signals
-  slave support for ``VHOST_USER_GET_PROTOCOL_FEATURES`` and
+  back-end support for ``VHOST_USER_GET_PROTOCOL_FEATURES`` and
   ``VHOST_USER_SET_PROTOCOL_FEATURES``.
 
 ``VHOST_USER_GET_PROTOCOL_FEATURES``
@@ -893,7 +893,7 @@ Master message types
   ``VHOST_USER_GET_FEATURES``.
 
 .. Note::
-   Slave that reported ``VHOST_USER_F_PROTOCOL_FEATURES`` must
+   Back-ends that report ``VHOST_USER_F_PROTOCOL_FEATURES`` must
    support this message even before ``VHOST_USER_SET_FEATURES`` was
    called.
 
@@ -909,7 +909,7 @@ Master message types
   ``VHOST_USER_GET_FEATURES``.
 
 .. Note::
-   Slave that reported ``VHOST_USER_F_PROTOCOL_FEATURES`` must support
+   Back-ends that report ``VHOST_USER_F_PROTOCOL_FEATURES`` must support
    this message even before ``VHOST_USER_SET_FEATURES`` was called.
 
 ``VHOST_USER_SET_OWNER``
@@ -918,8 +918,8 @@ Master message types
   :request payload: N/A
   :reply payload: N/A
 
-  Issued when a new connection is established. It sets the current
-  *master* as an owner of the session. This can be used on the *slave*
+  Issued when a new connection is established. It marks the sender
+  as the front-end that owns of the session. This can be used on the *back-end*
   as a "session start" flag.
 
 ``VHOST_USER_RESET_OWNER``
@@ -930,9 +930,9 @@ Master message types
 .. admonition:: Deprecated
 
    This is no longer used. Used to be sent to request disabling all
-   rings, but some clients interpreted it to also discard connection
+   rings, but some back-ends interpreted it to also discard connection
    state (this interpretation would lead to bugs).  It is recommended
-   that clients either ignore this message, or use it to disable all
+   that back-ends either ignore this message, or use it to disable all
    rings.
 
 ``VHOST_USER_SET_MEM_TABLE``
@@ -941,14 +941,14 @@ Master message types
   :request payload: memory regions description
   :reply payload: (postcopy only) memory regions description
 
-  Sets the memory map regions on the slave so it can translate the
+  Sets the memory map regions on the back-end so it can translate the
   vring addresses. In the ancillary data there is an array of file
   descriptors for each memory mapped region. The size and ordering of
   the fds matches the number and ordering of memory regions.
 
   When ``VHOST_USER_POSTCOPY_LISTEN`` has been received,
   ``SET_MEM_TABLE`` replies with the bases of the memory mapped
-  regions to the master.  The slave must have mmap'd the regions but
+  regions to the front-end.  The back-end must have mmap'd the regions but
   not yet accessed them and should not yet generate a userfault
   event.
 
@@ -967,7 +967,7 @@ Master message types
 
   Sets logging shared memory space.
 
-  When slave has ``VHOST_USER_PROTOCOL_F_LOG_SHMFD`` protocol feature,
+  When the back-end has ``VHOST_USER_PROTOCOL_F_LOG_SHMFD`` protocol feature,
   the log memory fd is provided in the ancillary data of
   ``VHOST_USER_SET_LOG_BASE`` message, the size and offset of shared
   memory area provided in the message.
@@ -1074,7 +1074,7 @@ Master message types
   :request payload: N/A
   :reply payload: u64
 
-  Query how many queues the backend supports.
+  Query how many queues the back-end supports.
 
   This request should be sent only when ``VHOST_USER_PROTOCOL_F_MQ``
   is set in queried protocol features by
@@ -1086,7 +1086,7 @@ Master message types
   :request payload: vring state description
   :reply payload: N/A
 
-  Signal slave to enable or disable corresponding vring.
+  Signal the back-end to enable or disable corresponding vring.
 
   This request should be sent only when
   ``VHOST_USER_F_PROTOCOL_FEATURES`` has been negotiated.
@@ -1097,7 +1097,7 @@ Master message types
   :request payload: ``u64``
   :reply payload: N/A
 
-  Ask vhost user backend to broadcast a fake RARP to notify the migration
+  Ask vhost user back-end to broadcast a fake RARP to notify the migration
   is terminated for guest that does not support GUEST_ANNOUNCE.
 
   Only legal if feature bit ``VHOST_USER_F_PROTOCOL_FEATURES`` is
@@ -1105,7 +1105,7 @@ Master message types
   ``VHOST_USER_PROTOCOL_F_RARP`` is present in
   ``VHOST_USER_GET_PROTOCOL_FEATURES``.  The first 6 bytes of the
   payload contain the mac address of the guest to allow the vhost user
-  backend to construct and broadcast the fake RARP.
+  back-end to construct and broadcast the fake RARP.
 
 ``VHOST_USER_NET_SET_MTU``
   :id: 20
@@ -1121,7 +1121,7 @@ Master message types
   ``VHOST_USER_PROTOCOL_F_NET_MTU`` is present in
   ``VHOST_USER_GET_PROTOCOL_FEATURES``.
 
-  If ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` is negotiated, slave must
+  If ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` is negotiated, the back-end must
   respond with zero in case the specified MTU is valid, or non-zero
   otherwise.
 
@@ -1131,14 +1131,14 @@ Master message types
   :request payload: N/A
   :reply payload: N/A
 
-  Set the socket file descriptor for slave initiated requests. It is passed
+  Set the socket file descriptor for back-end initiated requests. It is passed
   in the ancillary data.
 
   This request should be sent only when
   ``VHOST_USER_F_PROTOCOL_FEATURES`` has been negotiated, and protocol
   feature bit ``VHOST_USER_PROTOCOL_F_SLAVE_REQ`` bit is present in
   ``VHOST_USER_GET_PROTOCOL_FEATURES``.  If
-  ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` is negotiated, slave must
+  ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` is negotiated, the back-end must
   respond with zero for success, non-zero otherwise.
 
 ``VHOST_USER_IOTLB_MSG``
@@ -1149,8 +1149,8 @@ Master message types
 
   Send IOTLB messages with ``struct vhost_iotlb_msg`` as payload.
 
-  Master sends such requests to update and invalidate entries in the
-  device IOTLB. The slave has to acknowledge the request with sending
+  The front-end sends such requests to update and invalidate entries in the
+  device IOTLB. The back-end has to acknowledge the request with sending
   zero as ``u64`` payload for success, non-zero otherwise.
 
   This request should be send only when ``VIRTIO_F_IOMMU_PLATFORM``
@@ -1170,7 +1170,7 @@ Master message types
   ``VHOST_USER_PROTOCOL_F_CROSS_ENDIAN`` has been negotiated.
   Backends that negotiated this feature should handle both
   endiannesses and expect this message once (per VQ) during device
-  configuration (ie. before the master starts the VQ).
+  configuration (ie. before the front-end starts the VQ).
 
 ``VHOST_USER_GET_CONFIG``
   :id: 24
@@ -1179,11 +1179,11 @@ Master message types
   :reply payload: virtio device config space
 
   When ``VHOST_USER_PROTOCOL_F_CONFIG`` is negotiated, this message is
-  submitted by the vhost-user master to fetch the contents of the
-  virtio device configuration space, vhost-user slave's payload size
-  MUST match master's request, vhost-user slave uses zero length of
-  payload to indicate an error to vhost-user master. The vhost-user
-  master may cache the contents to avoid repeated
+  submitted by the vhost-user front-end to fetch the contents of the
+  virtio device configuration space, vhost-user back-end's payload size
+  MUST match the front-end's request, vhost-user back-end uses zero length of
+  payload to indicate an error to the vhost-user front-end. The vhost-user
+  front-end may cache the contents to avoid repeated
   ``VHOST_USER_GET_CONFIG`` calls.
 
 ``VHOST_USER_SET_CONFIG``
@@ -1193,10 +1193,10 @@ Master message types
   :reply payload: N/A
 
   When ``VHOST_USER_PROTOCOL_F_CONFIG`` is negotiated, this message is
-  submitted by the vhost-user master when the Guest changes the virtio
+  submitted by the vhost-user front-end when the Guest changes the virtio
   device configuration space and also can be used for live migration
-  on the destination host. The vhost-user slave must check the flags
-  field, and slaves MUST NOT accept SET_CONFIG for read-only
+  on the destination host. The vhost-user back-end must check the flags
+  field, and back-ends MUST NOT accept SET_CONFIG for read-only
   configuration space fields unless the live migration bit is set.
 
 ``VHOST_USER_CREATE_CRYPTO_SESSION``
@@ -1205,7 +1205,7 @@ Master message types
   :request payload: crypto session description
   :reply payload: crypto session description
 
-  Create a session for crypto operation. The server side must return
+  Create a session for crypto operation. The back-end must return
   the session id, 0 or positive for success, negative for failure.
   This request should be sent only when
   ``VHOST_USER_PROTOCOL_F_CRYPTO_SESSION`` feature has been
@@ -1231,9 +1231,9 @@ Master message types
   :request payload: N/A
   :reply payload: userfault fd
 
-  When ``VHOST_USER_PROTOCOL_F_PAGEFAULT`` is supported, the master
-  advises slave that a migration with postcopy enabled is underway,
-  the slave must open a userfaultfd for later use.  Note that at this
+  When ``VHOST_USER_PROTOCOL_F_PAGEFAULT`` is supported, the front-end
+  advises back-end that a migration with postcopy enabled is underway,
+  the back-end must open a userfaultfd for later use.  Note that at this
   stage the migration is still in precopy mode.
 
 ``VHOST_USER_POSTCOPY_LISTEN``
@@ -1241,8 +1241,8 @@ Master message types
   :request payload: N/A
   :reply payload: N/A
 
-  Master advises slave that a transition to postcopy mode has
-  happened.  The slave must ensure that shared memory is registered
+  The front-end advises back-end that a transition to postcopy mode has
+  happened.  The back-end must ensure that shared memory is registered
   with userfaultfd to cause faulting of non-present pages.
 
   This is always sent sometime after a ``VHOST_USER_POSTCOPY_ADVISE``,
@@ -1253,7 +1253,7 @@ Master message types
   :request payload: N/A
   :reply payload: ``u64``
 
-  Master advises that postcopy migration has now completed.  The slave
+  The front-end advises that postcopy migration has now completed.  The back-end
   must disable the userfaultfd. The reply is an acknowledgement
   only.
 
@@ -1270,9 +1270,9 @@ Master message types
   :reply payload: N/A
 
   When ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD`` protocol feature has
-  been successfully negotiated, this message is submitted by master to
-  get a shared buffer from slave. The shared buffer will be used to
-  track inflight I/O by slave. QEMU should retrieve a new one when vm
+  been successfully negotiated, this message is submitted by the front-end to
+  get a shared buffer from back-end. The shared buffer will be used to
+  track inflight I/O by back-end. QEMU should retrieve a new one when vm
   reset.
 
 ``VHOST_USER_SET_INFLIGHT_FD``
@@ -1282,9 +1282,9 @@ Master message types
   :reply payload: N/A
 
   When ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD`` protocol feature has
-  been successfully negotiated, this message is submitted by master to
-  send the shared inflight buffer back to slave so that slave could
-  get inflight I/O after a crash or restart.
+  been successfully negotiated, this message is submitted by the front-end to
+  send the shared inflight buffer back to the back-end so that the back-end
+  could get inflight I/O after a crash or restart.
 
 ``VHOST_USER_GPU_SET_SOCKET``
   :id: 33
@@ -1293,7 +1293,7 @@ Master message types
   :reply payload: N/A
 
   Sets the GPU protocol socket file descriptor, which is passed as
-  ancillary data. The GPU protocol is used to inform the master of
+  ancillary data. The GPU protocol is used to inform the front-end of
   rendering state and updates. See vhost-user-gpu.rst for details.
 
 ``VHOST_USER_RESET_DEVICE``
@@ -1302,13 +1302,13 @@ Master message types
   :request payload: N/A
   :reply payload: N/A
 
-  Ask the vhost user backend to disable all rings and reset all
+  Ask the vhost user back-end to disable all rings and reset all
   internal device state to the initial state, ready to be
-  reinitialized. The backend retains ownership of the device
+  reinitialized. The back-end retains ownership of the device
   throughout the reset operation.
 
   Only valid if the ``VHOST_USER_PROTOCOL_F_RESET_DEVICE`` protocol
-  feature is set by the backend.
+  feature is set by the back-end.
 
 ``VHOST_USER_VRING_KICK``
   :id: 35
@@ -1318,9 +1318,9 @@ Master message types
 
   When the ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` protocol
   feature has been successfully negotiated, this message may be
-  submitted by the master to indicate that a buffer was added to
+  submitted by the front-end to indicate that a buffer was added to
   the vring instead of signalling it using the vring's kick file
-  descriptor or having the slave rely on polling.
+  descriptor or having the back-end rely on polling.
 
   The state.num field is currently reserved and must be set to 0.
 
@@ -1332,9 +1332,9 @@ Master message types
 
   When the ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol
   feature has been successfully negotiated, this message is submitted
-  by master to the slave. The slave should return the message with a
+  by the front-end to the back-end. The back-end should return the message with a
   u64 payload containing the maximum number of memory slots for
-  QEMU to expose to the guest. The value returned by the backend
+  QEMU to expose to the guest. The value returned by the back-end
   will be capped at the maximum number of ram slots which can be
   supported by the target platform.
 
@@ -1346,19 +1346,19 @@ Master message types
 
   When the ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol
   feature has been successfully negotiated, this message is submitted
-  by the master to the slave. The message payload contains a memory
+  by the front-end to the back-end. The message payload contains a memory
   region descriptor struct, describing a region of guest memory which
-  the slave device must map in. When the
+  the back-end device must map in. When the
   ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol feature has
   been successfully negotiated, along with the
   ``VHOST_USER_REM_MEM_REG`` message, this message is used to set and
-  update the memory tables of the slave device.
+  update the memory tables of the back-end device.
 
   Exactly one file descriptor from which the memory is mapped is
   passed in the ancillary data.
 
-  In postcopy mode (see ``VHOST_USER_POSTCOPY_LISTEN``), the slave
-  replies with the bases of the memory mapped region to the master.
+  In postcopy mode (see ``VHOST_USER_POSTCOPY_LISTEN``), the back-end
+  replies with the bases of the memory mapped region to the front-end.
   For further details on postcopy, see ``VHOST_USER_SET_MEM_TABLE``.
   They apply to ``VHOST_USER_ADD_MEM_REG`` accordingly.
 
@@ -1370,21 +1370,21 @@ Master message types
 
   When the ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol
   feature has been successfully negotiated, this message is submitted
-  by the master to the slave. The message payload contains a memory
+  by the front-end to the back-end. The message payload contains a memory
   region descriptor struct, describing a region of guest memory which
-  the slave device must unmap. When the
+  the back-end device must unmap. When the
   ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol feature has
   been successfully negotiated, along with the
   ``VHOST_USER_ADD_MEM_REG`` message, this message is used to set and
-  update the memory tables of the slave device.
+  update the memory tables of the back-end device.
 
   The memory region to be removed is identified by its guest address,
   user address and size. The mmap offset is ignored.
 
   No file descriptors SHOULD be passed in the ancillary data. For
-  compatibility with existing incorrect implementations, the slave MAY
+  compatibility with existing incorrect implementations, the back-end MAY
   accept messages with one file descriptor. If a file descriptor is
-  passed, the slave MUST close it without using it otherwise.
+  passed, the back-end MUST close it without using it otherwise.
 
 ``VHOST_USER_SET_STATUS``
   :id: 39
@@ -1393,8 +1393,8 @@ Master message types
   :reply payload: N/A
 
   When the ``VHOST_USER_PROTOCOL_F_STATUS`` protocol feature has been
-  successfully negotiated, this message is submitted by the master to
-  notify the backend with updated device status as defined in the Virtio
+  successfully negotiated, this message is submitted by the front-end to
+  notify the back-end with updated device status as defined in the Virtio
   specification.
 
 ``VHOST_USER_GET_STATUS``
@@ -1404,16 +1404,16 @@ Master message types
   :reply payload: ``u64``
 
   When the ``VHOST_USER_PROTOCOL_F_STATUS`` protocol feature has been
-  successfully negotiated, this message is submitted by the master to
-  query the backend for its device status as defined in the Virtio
+  successfully negotiated, this message is submitted by the front-end to
+  query the back-end for its device status as defined in the Virtio
   specification.
 
 
-Slave message types
--------------------
+Back-end message types
+----------------------
 
-For this type of message, the request is sent by the slave and the reply
-is sent by the master.
+For this type of message, the request is sent by the back-end and the reply
+is sent by the front-end.
 
 ``VHOST_USER_SLAVE_IOTLB_MSG``
   :id: 1
@@ -1422,9 +1422,9 @@ is sent by the master.
   :reply payload: N/A
 
   Send IOTLB messages with ``struct vhost_iotlb_msg`` as payload.
-  Slave sends such requests to notify of an IOTLB miss, or an IOTLB
+  The back-end sends such requests to notify of an IOTLB miss, or an IOTLB
   access failure. If ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` is
-  negotiated, and slave set the ``VHOST_USER_NEED_REPLY`` flag, master
+  negotiated, and back-end set the ``VHOST_USER_NEED_REPLY`` flag, the front-end
   must respond with zero when operation is successfully completed, or
   non-zero otherwise.  This request should be send only when
   ``VIRTIO_F_IOMMU_PLATFORM`` feature has been successfully
@@ -1437,12 +1437,12 @@ is sent by the master.
   :reply payload: N/A
 
   When ``VHOST_USER_PROTOCOL_F_CONFIG`` is negotiated, vhost-user
-  slave sends such messages to notify that the virtio device's
+  back-end sends such messages to notify that the virtio device's
   configuration space has changed, for those host devices which can
   support such feature, host driver can send ``VHOST_USER_GET_CONFIG``
-  message to slave to get the latest content. If
-  ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` is negotiated, and slave set the
-  ``VHOST_USER_NEED_REPLY`` flag, master must respond with zero when
+  message to the back-end to get the latest content. If
+  ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` is negotiated, and the back-end sets the
+  ``VHOST_USER_NEED_REPLY`` flag, the front-end must respond with zero when
   operation is successfully completed, or non-zero otherwise.
 
 ``VHOST_USER_SLAVE_VRING_HOST_NOTIFIER_MSG``
@@ -1460,7 +1460,7 @@ is sent by the master.
   description. QEMU can mmap the file descriptor based on the size and
   offset to get a memory range. Registering a host notifier means
   mapping this memory range to the VM as the specified queue's notify
-  MMIO region. Slave sends this request to tell QEMU to de-register
+  MMIO region. The back-end sends this request to tell QEMU to de-register
   the existing notifier if any and register the new notifier if the
   request is sent with a file descriptor.
 
@@ -1476,9 +1476,9 @@ is sent by the master.
 
   When the ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` protocol
   feature has been successfully negotiated, this message may be
-  submitted by the slave to indicate that a buffer was used from
+  submitted by the back-end to indicate that a buffer was used from
   the vring instead of signalling this using the vring's call file
-  descriptor or having the master relying on polling.
+  descriptor or having the front-end relying on polling.
 
   The state.num field is currently reserved and must be set to 0.
 
@@ -1490,9 +1490,9 @@ is sent by the master.
 
   When the ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` protocol
   feature has been successfully negotiated, this message may be
-  submitted by the slave to indicate that an error occurred on the
+  submitted by the back-end to indicate that an error occurred on the
   specific vring, instead of signalling the error file descriptor
-  set by the master via ``VHOST_USER_SET_VRING_ERR``.
+  set by the front-end via ``VHOST_USER_SET_VRING_ERR``.
 
   The state.num field is currently reserved and must be set to 0.
 
@@ -1503,12 +1503,12 @@ VHOST_USER_PROTOCOL_F_REPLY_ACK
 
 The original vhost-user specification only demands replies for certain
 commands. This differs from the vhost protocol implementation where
-commands are sent over an ``ioctl()`` call and block until the client
+commands are sent over an ``ioctl()`` call and block until the back-end
 has completed.
 
 With this protocol extension negotiated, the sender (QEMU) can set the
 ``need_reply`` [Bit 3] flag to any command. This indicates that the
-client MUST respond with a Payload ``VhostUserMsg`` indicating success
+back-end MUST respond with a Payload ``VhostUserMsg`` indicating success
 or failure. The payload should be set to zero on success or non-zero
 on failure, unless the message already has an explicit reply body.
 
@@ -1517,7 +1517,7 @@ of the command. Today, QEMU is expected to terminate the main vhost-user
 loop upon receiving such errors. In future, qemu could be taught to be more
 resilient for selective requests.
 
-For the message types that already solicit a reply from the client,
+For the message types that already solicit a reply from the back-end,
 the presence of ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` or need_reply bit
 being set brings no behavioural change. (See the Communication_
 section for details.)
@@ -1527,26 +1527,26 @@ section for details.)
 Backend program conventions
 ===========================
 
-vhost-user backends can provide various devices & services and may
+vhost-user back-ends can provide various devices & services and may
 need to be configured manually depending on the use case. However, it
 is a good idea to follow the conventions listed here when
 possible. Users, QEMU or libvirt, can then rely on some common
 behaviour to avoid heterogeneous configuration and management of the
-backend programs and facilitate interoperability.
+back-end programs and facilitate interoperability.
 
-Each backend installed on a host system should come with at least one
+Each back-end installed on a host system should come with at least one
 JSON file that conforms to the vhost-user.json schema. Each file
-informs the management applications about the backend type, and binary
+informs the management applications about the back-end type, and binary
 location. In addition, it defines rules for management apps for
-picking the highest priority backend when multiple match the search
+picking the highest priority back-end when multiple match the search
 criteria (see ``@VhostUserBackend`` documentation in the schema file).
 
-If the backend is not capable of enabling a requested feature on the
+If the back-end is not capable of enabling a requested feature on the
 host (such as 3D acceleration with virgl), or the initialization
-failed, the backend should fail to start early and exit with a status
+failed, the back-end should fail to start early and exit with a status
 != 0. It may also print a message to stderr for further details.
 
-The backend program must not daemonize itself, but it may be
+The back-end program must not daemonize itself, but it may be
 daemonized by the management layer. It may also have a restricted
 access to the system.
 
@@ -1554,7 +1554,7 @@ File descriptors 0, 1 and 2 will exist, and have regular
 stdin/stdout/stderr usage (they may have been redirected to /dev/null
 by the management layer, or to a log handler).
 
-The backend program must end (as quickly and cleanly as possible) when
+The back-end program must end (as quickly and cleanly as possible) when
 the SIGTERM signal is received. Eventually, it may receive SIGKILL by
 the management layer after a few seconds.
 
@@ -1568,15 +1568,15 @@ are mandatory, unless explicitly said differently:
 
 --fd=FDNUM
 
-  When this argument is given, the backend program is started with the
+  When this argument is given, the back-end program is started with the
   vhost-user socket as file descriptor FDNUM. It is incompatible with
   --socket-path.
 
 --print-capabilities
 
-  Output to stdout the backend capabilities in JSON format, and then
+  Output to stdout the back-end capabilities in JSON format, and then
   exit successfully. Other options and arguments should be ignored, and
-  the backend program should not perform its normal function.  The
+  the back-end program should not perform its normal function.  The
   capabilities can be reported dynamically depending on the host
   capabilities.
 

From fa9972662ce5b6d50fe7190efeb48a5a9fecca91 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Mon, 21 Mar 2022 15:30:31 +0000
Subject: [PATCH 369/935] vhost-user.rst: add clarifying language about
 protocol negotiation
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Make the language about feature negotiation explicitly clear about the
handling of the VHOST_USER_F_PROTOCOL_FEATURES feature bit. Try and
avoid the sort of bug introduced in vhost.rs REPLY_ACK processing:

  https://github.com/rust-vmm/vhost/pull/24

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Jiang Liu <gerry@linux.alibaba.com>
Message-Id: <20210226111619.21178-1-alex.bennee@linaro.org>

Message-Id: <20220321153037.3622127-8-alex.bennee@linaro.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 docs/interop/vhost-user.rst | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
index 9159476678..73e710fe32 100644
--- a/docs/interop/vhost-user.rst
+++ b/docs/interop/vhost-user.rst
@@ -332,6 +332,18 @@ bit was dedicated for this purpose::
 
   #define VHOST_USER_F_PROTOCOL_FEATURES 30
 
+Note that VHOST_USER_F_PROTOCOL_FEATURES is the UNUSED (30) feature
+bit defined in `VIRTIO 1.1 6.3 Legacy Interface: Reserved Feature Bits
+<https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/virtio-v1.1-cs01.html#x1-4130003>`_.
+VIRTIO devices do not advertise this feature bit and therefore VIRTIO
+drivers cannot negotiate it.
+
+This reserved feature bit was reused by the vhost-user protocol to add
+vhost-user protocol feature negotiation in a backwards compatible
+fashion. Old vhost-user master and slave implementations continue to
+work even though they are not aware of vhost-user protocol feature
+negotiation.
+
 Ring states
 -----------
 
@@ -890,7 +902,8 @@ Front-end message types
   Get the protocol feature bitmask from the underlying vhost
   implementation.  Only legal if feature bit
   ``VHOST_USER_F_PROTOCOL_FEATURES`` is present in
-  ``VHOST_USER_GET_FEATURES``.
+  ``VHOST_USER_GET_FEATURES``.  It does not need to be acknowledged by
+  ``VHOST_USER_SET_FEATURES``.
 
 .. Note::
    Back-ends that report ``VHOST_USER_F_PROTOCOL_FEATURES`` must
@@ -906,7 +919,8 @@ Front-end message types
   Enable protocol features in the underlying vhost implementation.
 
   Only legal if feature bit ``VHOST_USER_F_PROTOCOL_FEATURES`` is present in
-  ``VHOST_USER_GET_FEATURES``.
+  ``VHOST_USER_GET_FEATURES``.  It does not need to be acknowledged by
+  ``VHOST_USER_SET_FEATURES``.
 
 .. Note::
    Back-ends that report ``VHOST_USER_F_PROTOCOL_FEATURES`` must support

From 467eeb0f80798457ba83040f7adb1e6b29c7c6a9 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Mon, 21 Mar 2022 15:30:32 +0000
Subject: [PATCH 370/935] libvhost-user: expose vu_request_to_string
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This is useful for more human readable debug messages in vhost-user
programs.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220321153037.3622127-9-alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 subprojects/libvhost-user/libvhost-user.c | 2 +-
 subprojects/libvhost-user/libvhost-user.h | 9 +++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/subprojects/libvhost-user/libvhost-user.c b/subprojects/libvhost-user/libvhost-user.c
index d0041c864b..b4cc3c2d68 100644
--- a/subprojects/libvhost-user/libvhost-user.c
+++ b/subprojects/libvhost-user/libvhost-user.c
@@ -99,7 +99,7 @@ static inline bool vu_has_protocol_feature(VuDev *dev, unsigned int fbit)
     return has_feature(dev->protocol_features, fbit);
 }
 
-static const char *
+const char *
 vu_request_to_string(unsigned int req)
 {
 #define REQ(req) [req] = #req
diff --git a/subprojects/libvhost-user/libvhost-user.h b/subprojects/libvhost-user/libvhost-user.h
index cde9f07bb3..aea7ec5061 100644
--- a/subprojects/libvhost-user/libvhost-user.h
+++ b/subprojects/libvhost-user/libvhost-user.h
@@ -473,6 +473,15 @@ bool vu_init(VuDev *dev,
  */
 void vu_deinit(VuDev *dev);
 
+
+/**
+ * vu_request_to_string: return string for vhost message request
+ * @req: VhostUserMsg request
+ *
+ * Returns a const string, do not free.
+ */
+const char *vu_request_to_string(unsigned int req);
+
 /**
  * vu_dispatch:
  * @dev: a VuDev context

From bcf317f8591083fc11c3167f8048c826475efc39 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Mon, 21 Mar 2022 15:30:33 +0000
Subject: [PATCH 371/935] docs/devel: start documenting writing VirtIO devices
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

While writing my own VirtIO devices I've gotten confused with how
things are structured and what sort of shared infrastructure there is.
If we can document how everything is supposed to work we can then
maybe start cleaning up inconsistencies in the code.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Viresh Kumar <viresh.kumar@linaro.org>
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20220309164929.19395-1-alex.bennee@linaro.org>

Message-Id: <20220321153037.3622127-10-alex.bennee@linaro.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 docs/devel/index-internals.rst |   1 +
 docs/devel/virtio-backends.rst | 214 +++++++++++++++++++++++++++++++++
 2 files changed, 215 insertions(+)
 create mode 100644 docs/devel/virtio-backends.rst

diff --git a/docs/devel/index-internals.rst b/docs/devel/index-internals.rst
index a50889c556..e1a93df263 100644
--- a/docs/devel/index-internals.rst
+++ b/docs/devel/index-internals.rst
@@ -18,3 +18,4 @@ Details about QEMU's various subsystems including how to add features to them.
    tracing
    vfio-migration
    writing-monitor-commands
+   virtio-backends
diff --git a/docs/devel/virtio-backends.rst b/docs/devel/virtio-backends.rst
new file mode 100644
index 0000000000..9ff092e7a0
--- /dev/null
+++ b/docs/devel/virtio-backends.rst
@@ -0,0 +1,214 @@
+..
+   Copyright (c) 2022, Linaro Limited
+   Written by Alex Bennée
+
+Writing VirtIO backends for QEMU
+================================
+
+This document attempts to outline the information a developer needs to
+know to write device emulations in QEMU. It is specifically focused on
+implementing VirtIO devices. For VirtIO the frontend is the driver
+running on the guest. The backend is the everything that QEMU needs to
+do to handle the emulation of the VirtIO device. This can be done
+entirely in QEMU, divided between QEMU and the kernel (vhost) or
+handled by a separate process which is configured by QEMU
+(vhost-user).
+
+VirtIO Transports
+-----------------
+
+VirtIO supports a number of different transports. While the details of
+the configuration and operation of the device will generally be the
+same QEMU represents them as different devices depending on the
+transport they use. For example -device virtio-foo represents the foo
+device using mmio and -device virtio-foo-pci is the same class of
+device using the PCI transport.
+
+Using the QEMU Object Model (QOM)
+---------------------------------
+
+Generally all devices in QEMU are super classes of ``TYPE_DEVICE``
+however VirtIO devices should be based on ``TYPE_VIRTIO_DEVICE`` which
+itself is derived from the base class. For example:
+
+.. code:: c
+
+  static const TypeInfo virtio_blk_info = {
+      .name = TYPE_VIRTIO_BLK,
+      .parent = TYPE_VIRTIO_DEVICE,
+      .instance_size = sizeof(VirtIOBlock),
+      .instance_init = virtio_blk_instance_init,
+      .class_init = virtio_blk_class_init,
+  };
+
+The author may decide to have a more expansive class hierarchy to
+support multiple device types. For example the Virtio GPU device:
+
+.. code:: c
+
+  static const TypeInfo virtio_gpu_base_info = {
+      .name = TYPE_VIRTIO_GPU_BASE,
+      .parent = TYPE_VIRTIO_DEVICE,
+      .instance_size = sizeof(VirtIOGPUBase),
+      .class_size = sizeof(VirtIOGPUBaseClass),
+      .class_init = virtio_gpu_base_class_init,
+      .abstract = true
+  };
+
+  static const TypeInfo vhost_user_gpu_info = {
+      .name = TYPE_VHOST_USER_GPU,
+      .parent = TYPE_VIRTIO_GPU_BASE,
+      .instance_size = sizeof(VhostUserGPU),
+      .instance_init = vhost_user_gpu_instance_init,
+      .instance_finalize = vhost_user_gpu_instance_finalize,
+      .class_init = vhost_user_gpu_class_init,
+  };
+
+  static const TypeInfo virtio_gpu_info = {
+      .name = TYPE_VIRTIO_GPU,
+      .parent = TYPE_VIRTIO_GPU_BASE,
+      .instance_size = sizeof(VirtIOGPU),
+      .class_size = sizeof(VirtIOGPUClass),
+      .class_init = virtio_gpu_class_init,
+  };
+
+defines a base class for the VirtIO GPU and then specialises two
+versions, one for the internal implementation and the other for the
+vhost-user version.
+
+VirtIOPCIProxy
+^^^^^^^^^^^^^^
+
+[AJB: the following is supposition and welcomes more informed
+opinions]
+
+Probably due to legacy from the pre-QOM days PCI VirtIO devices don't
+follow the normal hierarchy. Instead the a standalone object is based
+on the VirtIOPCIProxy class and the specific VirtIO instance is
+manually instantiated:
+
+.. code:: c
+
+  /*
+   * virtio-blk-pci: This extends VirtioPCIProxy.
+   */
+  #define TYPE_VIRTIO_BLK_PCI "virtio-blk-pci-base"
+  DECLARE_INSTANCE_CHECKER(VirtIOBlkPCI, VIRTIO_BLK_PCI,
+                           TYPE_VIRTIO_BLK_PCI)
+
+  struct VirtIOBlkPCI {
+      VirtIOPCIProxy parent_obj;
+      VirtIOBlock vdev;
+  };
+
+  static Property virtio_blk_pci_properties[] = {
+      DEFINE_PROP_UINT32("class", VirtIOPCIProxy, class_code, 0),
+      DEFINE_PROP_BIT("ioeventfd", VirtIOPCIProxy, flags,
+                      VIRTIO_PCI_FLAG_USE_IOEVENTFD_BIT, true),
+      DEFINE_PROP_UINT32("vectors", VirtIOPCIProxy, nvectors,
+                         DEV_NVECTORS_UNSPECIFIED),
+      DEFINE_PROP_END_OF_LIST(),
+  };
+
+  static void virtio_blk_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp)
+  {
+      VirtIOBlkPCI *dev = VIRTIO_BLK_PCI(vpci_dev);
+      DeviceState *vdev = DEVICE(&dev->vdev);
+
+      ...
+
+      qdev_realize(vdev, BUS(&vpci_dev->bus), errp);
+  }
+
+  static void virtio_blk_pci_class_init(ObjectClass *klass, void *data)
+  {
+      DeviceClass *dc = DEVICE_CLASS(klass);
+      VirtioPCIClass *k = VIRTIO_PCI_CLASS(klass);
+      PCIDeviceClass *pcidev_k = PCI_DEVICE_CLASS(klass);
+
+      set_bit(DEVICE_CATEGORY_STORAGE, dc->categories);
+      device_class_set_props(dc, virtio_blk_pci_properties);
+      k->realize = virtio_blk_pci_realize;
+      pcidev_k->vendor_id = PCI_VENDOR_ID_REDHAT_QUMRANET;
+      pcidev_k->device_id = PCI_DEVICE_ID_VIRTIO_BLOCK;
+      pcidev_k->revision = VIRTIO_PCI_ABI_VERSION;
+      pcidev_k->class_id = PCI_CLASS_STORAGE_SCSI;
+  }
+
+  static void virtio_blk_pci_instance_init(Object *obj)
+  {
+      VirtIOBlkPCI *dev = VIRTIO_BLK_PCI(obj);
+
+      virtio_instance_init_common(obj, &dev->vdev, sizeof(dev->vdev),
+                                  TYPE_VIRTIO_BLK);
+      object_property_add_alias(obj, "bootindex", OBJECT(&dev->vdev),
+                                "bootindex");
+  }
+
+  static const VirtioPCIDeviceTypeInfo virtio_blk_pci_info = {
+      .base_name              = TYPE_VIRTIO_BLK_PCI,
+      .generic_name           = "virtio-blk-pci",
+      .transitional_name      = "virtio-blk-pci-transitional",
+      .non_transitional_name  = "virtio-blk-pci-non-transitional",
+      .instance_size = sizeof(VirtIOBlkPCI),
+      .instance_init = virtio_blk_pci_instance_init,
+      .class_init    = virtio_blk_pci_class_init,
+  };
+
+Here you can see the instance_init has to manually instantiate the
+underlying ``TYPE_VIRTIO_BLOCK`` object and link an alias for one of
+it's properties to the PCI device.
+
+  
+Back End Implementations
+------------------------
+
+There are a number of places where the implementation of the backend
+can be done:
+
+* in QEMU itself
+* in the host kernel (a.k.a vhost)
+* in a separate process (a.k.a. vhost-user)
+
+vhost_ops vs TYPE_VHOST_USER_BACKEND
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+There are two choices to how to implement vhost code. Most of the code
+which has to work with either vhost or vhost-user uses
+``vhost_dev_init()`` to instantiate the appropriate backend. This
+means including a ``struct vhost_dev`` in the main object structure.
+
+For vhost-user devices you also need to add code to track the
+initialisation of the ``chardev`` device used for the control socket
+between QEMU and the external vhost-user process.
+
+If you only need to implement a vhost-user backed the other option is
+a use a QOM-ified version of vhost-user.
+
+.. code:: c
+
+  static void
+  vhost_user_gpu_instance_init(Object *obj)
+  {
+      VhostUserGPU *g = VHOST_USER_GPU(obj);
+
+      g->vhost = VHOST_USER_BACKEND(object_new(TYPE_VHOST_USER_BACKEND));
+      object_property_add_alias(obj, "chardev",
+                                OBJECT(g->vhost), "chardev");
+  }
+
+  static const TypeInfo vhost_user_gpu_info = {
+      .name = TYPE_VHOST_USER_GPU,
+      .parent = TYPE_VIRTIO_GPU_BASE,
+      .instance_size = sizeof(VhostUserGPU),
+      .instance_init = vhost_user_gpu_instance_init,
+      .instance_finalize = vhost_user_gpu_instance_finalize,
+      .class_init = vhost_user_gpu_class_init,
+  };
+
+Using it this way entails adding a ``struct VhostUserBackend`` to your
+core object structure and manually instantiating the backend. This
+sub-structure tracks both the ``vhost_dev`` and ``CharDev`` types
+needed for the connection. Instead of calling ``vhost_dev_init`` you
+would call ``vhost_user_backend_dev_init`` which does what is needed
+on your behalf.

From 27351992bf238f669dd08655d31ad111905bf899 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Mon, 21 Mar 2022 15:30:34 +0000
Subject: [PATCH 372/935] include/hw: start documenting the vhost API
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

While trying to get my head around the nest of interactions for vhost
devices I though I could start by documenting the key API functions.
This patch documents the main API hooks for creating and starting a
vhost device as well as how the configuration changes are handled.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20220321153037.3622127-11-alex.bennee@linaro.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/virtio/vhost.h | 132 +++++++++++++++++++++++++++++++++++---
 1 file changed, 122 insertions(+), 10 deletions(-)

diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
index 58a73e7b7a..b291fe4e24 100644
--- a/include/hw/virtio/vhost.h
+++ b/include/hw/virtio/vhost.h
@@ -61,6 +61,12 @@ typedef struct VhostDevConfigOps {
 } VhostDevConfigOps;
 
 struct vhost_memory;
+
+/**
+ * struct vhost_dev - common vhost_dev structure
+ * @vhost_ops: backend specific ops
+ * @config_ops: ops for config changes (see @vhost_dev_set_config_notifier)
+ */
 struct vhost_dev {
     VirtIODevice *vdev;
     MemoryListener memory_listener;
@@ -108,15 +114,129 @@ struct vhost_net {
     NetClientState *nc;
 };
 
+/**
+ * vhost_dev_init() - initialise the vhost interface
+ * @hdev: the common vhost_dev structure
+ * @opaque: opaque ptr passed to backend (vhost/vhost-user/vdpa)
+ * @backend_type: type of backend
+ * @busyloop_timeout: timeout for polling virtqueue
+ * @errp: error handle
+ *
+ * The initialisation of the vhost device will trigger the
+ * initialisation of the backend and potentially capability
+ * negotiation of backend interface. Configuration of the VirtIO
+ * itself won't happen until the interface is started.
+ *
+ * Return: 0 on success, non-zero on error while setting errp.
+ */
 int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
                    VhostBackendType backend_type,
                    uint32_t busyloop_timeout, Error **errp);
+
+/**
+ * vhost_dev_cleanup() - tear down and cleanup vhost interface
+ * @hdev: the common vhost_dev structure
+ */
 void vhost_dev_cleanup(struct vhost_dev *hdev);
-int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev);
-void vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev);
+
+/**
+ * vhost_dev_enable_notifiers() - enable event notifiers
+ * @hdev: common vhost_dev structure
+ * @vdev: the VirtIODevice structure
+ *
+ * Enable notifications directly to the vhost device rather than being
+ * triggered by QEMU itself. Notifications should be enabled before
+ * the vhost device is started via @vhost_dev_start.
+ *
+ * Return: 0 on success, < 0 on error.
+ */
 int vhost_dev_enable_notifiers(struct vhost_dev *hdev, VirtIODevice *vdev);
+
+/**
+ * vhost_dev_disable_notifiers - disable event notifications
+ * @hdev: common vhost_dev structure
+ * @vdev: the VirtIODevice structure
+ *
+ * Disable direct notifications to vhost device.
+ */
 void vhost_dev_disable_notifiers(struct vhost_dev *hdev, VirtIODevice *vdev);
 
+/**
+ * vhost_dev_start() - start the vhost device
+ * @hdev: common vhost_dev structure
+ * @vdev: the VirtIODevice structure
+ *
+ * Starts the vhost device. From this point VirtIO feature negotiation
+ * can start and the device can start processing VirtIO transactions.
+ *
+ * Return: 0 on success, < 0 on error.
+ */
+int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev);
+
+/**
+ * vhost_dev_stop() - stop the vhost device
+ * @hdev: common vhost_dev structure
+ * @vdev: the VirtIODevice structure
+ *
+ * Stop the vhost device. After the device is stopped the notifiers
+ * can be disabled (@vhost_dev_disable_notifiers) and the device can
+ * be torn down (@vhost_dev_cleanup).
+ */
+void vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev);
+
+/**
+ * DOC: vhost device configuration handling
+ *
+ * The VirtIO device configuration space is used for rarely changing
+ * or initialisation time parameters. The configuration can be updated
+ * by either the guest driver or the device itself. If the device can
+ * change the configuration over time the vhost handler should
+ * register a @VhostDevConfigOps structure with
+ * @vhost_dev_set_config_notifier so the guest can be notified. Some
+ * devices register a handler anyway and will signal an error if an
+ * unexpected config change happens.
+ */
+
+/**
+ * vhost_dev_get_config() - fetch device configuration
+ * @hdev: common vhost_dev_structure
+ * @config: pointer to device appropriate config structure
+ * @config_len: size of device appropriate config structure
+ *
+ * Return: 0 on success, < 0 on error while setting errp
+ */
+int vhost_dev_get_config(struct vhost_dev *hdev, uint8_t *config,
+                         uint32_t config_len, Error **errp);
+
+/**
+ * vhost_dev_set_config() - set device configuration
+ * @hdev: common vhost_dev_structure
+ * @data: pointer to data to set
+ * @offset: offset into configuration space
+ * @size: length of set
+ * @flags: @VhostSetConfigType flags
+ *
+ * By use of @offset/@size a subset of the configuration space can be
+ * written to. The @flags are used to indicate if it is a normal
+ * transaction or related to migration.
+ *
+ * Return: 0 on success, non-zero on error
+ */
+int vhost_dev_set_config(struct vhost_dev *dev, const uint8_t *data,
+                         uint32_t offset, uint32_t size, uint32_t flags);
+
+/**
+ * vhost_dev_set_config_notifier() - register VhostDevConfigOps
+ * @hdev: common vhost_dev_structure
+ * @ops: notifier ops
+ *
+ * If the device is expected to change configuration a notifier can be
+ * setup to handle the case.
+ */
+void vhost_dev_set_config_notifier(struct vhost_dev *dev,
+                                   const VhostDevConfigOps *ops);
+
+
 /* Test and clear masked event pending status.
  * Should be called after unmask to avoid losing events.
  */
@@ -136,14 +256,6 @@ int vhost_net_set_backend(struct vhost_dev *hdev,
                           struct vhost_vring_file *file);
 
 int vhost_device_iotlb_miss(struct vhost_dev *dev, uint64_t iova, int write);
-int vhost_dev_get_config(struct vhost_dev *hdev, uint8_t *config,
-                         uint32_t config_len, Error **errp);
-int vhost_dev_set_config(struct vhost_dev *dev, const uint8_t *data,
-                         uint32_t offset, uint32_t size, uint32_t flags);
-/* notifier callback in case vhost device config space changed
- */
-void vhost_dev_set_config_notifier(struct vhost_dev *dev,
-                                   const VhostDevConfigOps *ops);
 
 void vhost_dev_reset_inflight(struct vhost_inflight *inflight);
 void vhost_dev_free_inflight(struct vhost_inflight *inflight);

From 56534930b5cd3a6c7259cfeba69f08ba28c8f62a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Mon, 21 Mar 2022 15:30:36 +0000
Subject: [PATCH 373/935] hw/virtio/vhost-user: don't suppress F_CONFIG when
 supported
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Previously we would silently suppress VHOST_USER_PROTOCOL_F_CONFIG
during the protocol negotiation if the QEMU stub hadn't implemented
the vhost_dev_config_notifier. However this isn't the only way we can
handle config messages, the existing vdc->get/set_config can do this
as well.

Lightly re-factor the code to check for both potential methods and
instead of silently squashing the feature error out. It is unlikely
that a vhost-user backend expecting to handle CONFIG messages will
behave correctly if they never get sent.

Fixes: 1c3e5a2617 ("vhost-user: back SET/GET_CONFIG requests with a protocol feature")
Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Message-Id: <20220321153037.3622127-13-alex.bennee@linaro.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/scsi/vhost-user-scsi.c      |  1 +
 hw/virtio/vhost-user.c         | 44 ++++++++++++++++++++++++----------
 include/hw/virtio/vhost-user.h |  1 +
 3 files changed, 34 insertions(+), 12 deletions(-)

diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c
index 1b2f7eed98..9be21d07ee 100644
--- a/hw/scsi/vhost-user-scsi.c
+++ b/hw/scsi/vhost-user-scsi.c
@@ -121,6 +121,7 @@ static void vhost_user_scsi_realize(DeviceState *dev, Error **errp)
     vsc->dev.backend_features = 0;
     vqs = vsc->dev.vqs;
 
+    s->vhost_user.supports_config = true;
     ret = vhost_dev_init(&vsc->dev, &s->vhost_user,
                          VHOST_BACKEND_TYPE_USER, 0, errp);
     if (ret < 0) {
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index 6c8f722262..b0d417651d 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -1949,14 +1949,15 @@ static int vhost_user_postcopy_notifier(NotifierWithReturn *notifier,
 static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
                                    Error **errp)
 {
-    uint64_t features, protocol_features, ram_slots;
+    uint64_t features, ram_slots;
     struct vhost_user *u;
+    VhostUserState *vus = (VhostUserState *) opaque;
     int err;
 
     assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_USER);
 
     u = g_new0(struct vhost_user, 1);
-    u->user = opaque;
+    u->user = vus;
     u->dev = dev;
     dev->opaque = u;
 
@@ -1967,6 +1968,10 @@ static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
     }
 
     if (virtio_has_feature(features, VHOST_USER_F_PROTOCOL_FEATURES)) {
+        bool supports_f_config = vus->supports_config ||
+            (dev->config_ops && dev->config_ops->vhost_dev_config_notifier);
+        uint64_t protocol_features;
+
         dev->backend_features |= 1ULL << VHOST_USER_F_PROTOCOL_FEATURES;
 
         err = vhost_user_get_u64(dev, VHOST_USER_GET_PROTOCOL_FEATURES,
@@ -1976,19 +1981,34 @@ static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
             return -EPROTO;
         }
 
-        dev->protocol_features =
-            protocol_features & VHOST_USER_PROTOCOL_FEATURE_MASK;
+        /*
+         * We will use all the protocol features we support - although
+         * we suppress F_CONFIG if we know QEMUs internal code can not support
+         * it.
+         */
+        protocol_features &= VHOST_USER_PROTOCOL_FEATURE_MASK;
 
-        if (!dev->config_ops || !dev->config_ops->vhost_dev_config_notifier) {
-            /* Don't acknowledge CONFIG feature if device doesn't support it */
-            dev->protocol_features &= ~(1ULL << VHOST_USER_PROTOCOL_F_CONFIG);
-        } else if (!(protocol_features &
-                    (1ULL << VHOST_USER_PROTOCOL_F_CONFIG))) {
-            error_setg(errp, "Device expects VHOST_USER_PROTOCOL_F_CONFIG "
-                       "but backend does not support it.");
-            return -EINVAL;
+        if (supports_f_config) {
+            if (!virtio_has_feature(protocol_features,
+                                    VHOST_USER_PROTOCOL_F_CONFIG)) {
+                error_setg(errp, "vhost-user device %s expecting "
+                           "VHOST_USER_PROTOCOL_F_CONFIG but the vhost-user backend does "
+                           "not support it.", dev->vdev->name);
+                return -EPROTO;
+            }
+        } else {
+            if (virtio_has_feature(protocol_features,
+                                   VHOST_USER_PROTOCOL_F_CONFIG)) {
+                warn_reportf_err(*errp, "vhost-user backend supports "
+                                 "VHOST_USER_PROTOCOL_F_CONFIG for "
+                                 "device %s but QEMU does not.",
+                                 dev->vdev->name);
+                protocol_features &= ~(1ULL << VHOST_USER_PROTOCOL_F_CONFIG);
+            }
         }
 
+        /* final set of protocol features */
+        dev->protocol_features = protocol_features;
         err = vhost_user_set_protocol_features(dev, dev->protocol_features);
         if (err < 0) {
             error_setg_errno(errp, EPROTO, "vhost_backend_init failed");
diff --git a/include/hw/virtio/vhost-user.h b/include/hw/virtio/vhost-user.h
index e44a41bb70..6e0e8a71a3 100644
--- a/include/hw/virtio/vhost-user.h
+++ b/include/hw/virtio/vhost-user.h
@@ -22,6 +22,7 @@ typedef struct VhostUserState {
     CharBackend *chr;
     VhostUserHostNotifier notifier[VIRTIO_QUEUE_MAX];
     int memory_slots;
+    bool supports_config;
 } VhostUserState;
 
 bool vhost_user_init(VhostUserState *user, CharBackend *chr, Error **errp);

From 503e355465ba0eafad49c2e174995d2e234ed288 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Mon, 21 Mar 2022 15:30:37 +0000
Subject: [PATCH 374/935] virtio/vhost-user: dynamically assign
 VhostUserHostNotifiers
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

At a couple of hundred bytes per notifier allocating one for every
potential queue is very wasteful as most devices only have a few
queues. Instead of having this handled statically dynamically assign
them and track in a GPtrArray.

[AJB: it's hard to trigger the vhost notifiers code, I assume as it
requires a KVM guest with appropriate backend]

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220321153037.3622127-14-alex.bennee@linaro.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/trace-events         |  1 +
 hw/virtio/vhost-user.c         | 83 +++++++++++++++++++++++++++-------
 include/hw/virtio/vhost-user.h | 42 ++++++++++++++++-
 3 files changed, 108 insertions(+), 18 deletions(-)

diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events
index 65e3b704ec..ab8e095b73 100644
--- a/hw/virtio/trace-events
+++ b/hw/virtio/trace-events
@@ -23,6 +23,7 @@ vhost_user_postcopy_waker_found(uint64_t client_addr) "0x%"PRIx64
 vhost_user_postcopy_waker_nomatch(const char *rb, uint64_t rb_offset) "%s + 0x%"PRIx64
 vhost_user_read(uint32_t req, uint32_t flags) "req:%d flags:0x%"PRIx32""
 vhost_user_write(uint32_t req, uint32_t flags) "req:%d flags:0x%"PRIx32""
+vhost_user_create_notifier(int idx, void *n) "idx:%d n:%p"
 
 # vhost-vdpa.c
 vhost_vdpa_dma_map(void *vdpa, int fd, uint32_t msg_type, uint64_t iova, uint64_t size, uint64_t uaddr, uint8_t perm, uint8_t type) "vdpa:%p fd: %d msg_type: %"PRIu32" iova: 0x%"PRIx64" size: 0x%"PRIx64" uaddr: 0x%"PRIx64" perm: 0x%"PRIx8" type: %"PRIu8
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index b0d417651d..b040c1ad2b 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -1174,14 +1174,16 @@ static void vhost_user_host_notifier_free(VhostUserHostNotifier *n)
     n->unmap_addr = NULL;
 }
 
-static void vhost_user_host_notifier_remove(VhostUserState *user,
-                                            VirtIODevice *vdev, int queue_idx)
+/*
+ * clean-up function for notifier, will finally free the structure
+ * under rcu.
+ */
+static void vhost_user_host_notifier_remove(VhostUserHostNotifier *n,
+                                            VirtIODevice *vdev)
 {
-    VhostUserHostNotifier *n = &user->notifier[queue_idx];
-
     if (n->addr) {
         if (vdev) {
-            virtio_queue_set_host_notifier_mr(vdev, queue_idx, &n->mr, false);
+            virtio_queue_set_host_notifier_mr(vdev, n->idx, &n->mr, false);
         }
         assert(!n->unmap_addr);
         n->unmap_addr = n->addr;
@@ -1225,6 +1227,15 @@ static int vhost_user_set_vring_enable(struct vhost_dev *dev, int enable)
     return 0;
 }
 
+static VhostUserHostNotifier *fetch_notifier(VhostUserState *u,
+                                             int idx)
+{
+    if (idx >= u->notifiers->len) {
+        return NULL;
+    }
+    return g_ptr_array_index(u->notifiers, idx);
+}
+
 static int vhost_user_get_vring_base(struct vhost_dev *dev,
                                      struct vhost_vring_state *ring)
 {
@@ -1237,7 +1248,10 @@ static int vhost_user_get_vring_base(struct vhost_dev *dev,
     };
     struct vhost_user *u = dev->opaque;
 
-    vhost_user_host_notifier_remove(u->user, dev->vdev, ring->index);
+    VhostUserHostNotifier *n = fetch_notifier(u->user, ring->index);
+    if (n) {
+        vhost_user_host_notifier_remove(n, dev->vdev);
+    }
 
     ret = vhost_user_write(dev, &msg, NULL, 0);
     if (ret < 0) {
@@ -1502,6 +1516,29 @@ static int vhost_user_slave_handle_config_change(struct vhost_dev *dev)
     return dev->config_ops->vhost_dev_config_notifier(dev);
 }
 
+/*
+ * Fetch or create the notifier for a given idx. Newly created
+ * notifiers are added to the pointer array that tracks them.
+ */
+static VhostUserHostNotifier *fetch_or_create_notifier(VhostUserState *u,
+                                                       int idx)
+{
+    VhostUserHostNotifier *n = NULL;
+    if (idx >= u->notifiers->len) {
+        g_ptr_array_set_size(u->notifiers, idx);
+    }
+
+    n = g_ptr_array_index(u->notifiers, idx);
+    if (!n) {
+        n = g_new0(VhostUserHostNotifier, 1);
+        n->idx = idx;
+        g_ptr_array_insert(u->notifiers, idx, n);
+        trace_vhost_user_create_notifier(idx, n);
+    }
+
+    return n;
+}
+
 static int vhost_user_slave_handle_vring_host_notifier(struct vhost_dev *dev,
                                                        VhostUserVringArea *area,
                                                        int fd)
@@ -1521,9 +1558,12 @@ static int vhost_user_slave_handle_vring_host_notifier(struct vhost_dev *dev,
         return -EINVAL;
     }
 
-    n = &user->notifier[queue_idx];
-
-    vhost_user_host_notifier_remove(user, vdev, queue_idx);
+    /*
+     * Fetch notifier and invalidate any old data before setting up
+     * new mapped address.
+     */
+    n = fetch_or_create_notifier(user, queue_idx);
+    vhost_user_host_notifier_remove(n, vdev);
 
     if (area->u64 & VHOST_USER_VRING_NOFD_MASK) {
         return 0;
@@ -2526,6 +2566,20 @@ static int vhost_user_set_inflight_fd(struct vhost_dev *dev,
     return vhost_user_write(dev, &msg, &inflight->fd, 1);
 }
 
+static void vhost_user_state_destroy(gpointer data)
+{
+    VhostUserHostNotifier *n = (VhostUserHostNotifier *) data;
+    if (n) {
+        vhost_user_host_notifier_remove(n, NULL);
+        object_unparent(OBJECT(&n->mr));
+        /*
+         * We can't free until vhost_user_host_notifier_remove has
+         * done it's thing so schedule the free with RCU.
+         */
+        g_free_rcu(n, rcu);
+    }
+}
+
 bool vhost_user_init(VhostUserState *user, CharBackend *chr, Error **errp)
 {
     if (user->chr) {
@@ -2534,23 +2588,18 @@ bool vhost_user_init(VhostUserState *user, CharBackend *chr, Error **errp)
     }
     user->chr = chr;
     user->memory_slots = 0;
+    user->notifiers = g_ptr_array_new_full(VIRTIO_QUEUE_MAX / 4,
+                                           &vhost_user_state_destroy);
     return true;
 }
 
 void vhost_user_cleanup(VhostUserState *user)
 {
-    int i;
-    VhostUserHostNotifier *n;
-
     if (!user->chr) {
         return;
     }
     memory_region_transaction_begin();
-    for (i = 0; i < VIRTIO_QUEUE_MAX; i++) {
-        n = &user->notifier[i];
-        vhost_user_host_notifier_remove(user, NULL, i);
-        object_unparent(OBJECT(&n->mr));
-    }
+    user->notifiers = (GPtrArray *) g_ptr_array_free(user->notifiers, true);
     memory_region_transaction_commit();
     user->chr = NULL;
 }
diff --git a/include/hw/virtio/vhost-user.h b/include/hw/virtio/vhost-user.h
index 6e0e8a71a3..c6e693cd3f 100644
--- a/include/hw/virtio/vhost-user.h
+++ b/include/hw/virtio/vhost-user.h
@@ -11,21 +11,61 @@
 #include "chardev/char-fe.h"
 #include "hw/virtio/virtio.h"
 
+/**
+ * VhostUserHostNotifier - notifier information for one queue
+ * @rcu: rcu_head for cleanup
+ * @mr: memory region of notifier
+ * @addr: current mapped address
+ * @unmap_addr: address to be un-mapped
+ * @idx: virtioqueue index
+ *
+ * The VhostUserHostNotifier entries are re-used. When an old mapping
+ * is to be released it is moved to @unmap_addr and @addr is replaced.
+ * Once the RCU process has completed the unmap @unmap_addr is
+ * cleared.
+ */
 typedef struct VhostUserHostNotifier {
     struct rcu_head rcu;
     MemoryRegion mr;
     void *addr;
     void *unmap_addr;
+    int idx;
 } VhostUserHostNotifier;
 
+/**
+ * VhostUserState - shared state for all vhost-user devices
+ * @chr: the character backend for the socket
+ * @notifiers: GPtrArray of @VhostUserHostnotifier
+ * @memory_slots:
+ */
 typedef struct VhostUserState {
     CharBackend *chr;
-    VhostUserHostNotifier notifier[VIRTIO_QUEUE_MAX];
+    GPtrArray *notifiers;
     int memory_slots;
     bool supports_config;
 } VhostUserState;
 
+/**
+ * vhost_user_init() - initialise shared vhost_user state
+ * @user: allocated area for storing shared state
+ * @chr: the chardev for the vhost socket
+ * @errp: error handle
+ *
+ * User can either directly g_new() space for the state or embed
+ * VhostUserState in their larger device structure and just point to
+ * it.
+ *
+ * Return: true on success, false on error while setting errp.
+ */
 bool vhost_user_init(VhostUserState *user, CharBackend *chr, Error **errp);
+
+/**
+ * vhost_user_cleanup() - cleanup state
+ * @user: ptr to use state
+ *
+ * Cleans up shared state and notifiers, callee is responsible for
+ * freeing the @VhostUserState memory itself.
+ */
 void vhost_user_cleanup(VhostUserState *user);
 
 #endif

From 3857cd5c7ffd0bcc49bd2b66654af0909c847d02 Mon Sep 17 00:00:00 2001
From: Jonah Palmer <jonah.palmer@oracle.com>
Date: Fri, 1 Apr 2022 09:23:18 -0400
Subject: [PATCH 375/935] virtio: drop name parameter for virtio_init()

This patch drops the name parameter for the virtio_init function.

The pair between the numeric device ID and the string device ID
(name) of a virtio device already exists, but not in a way that
lets us map between them.

This patch lets us do this and removes the need for the name
parameter in the virtio_init function.

Signed-off-by: Jonah Palmer <jonah.palmer@oracle.com>
Message-Id: <1648819405-25696-2-git-send-email-jonah.palmer@oracle.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/9pfs/virtio-9p-device.c             |  2 +-
 hw/block/vhost-user-blk.c              |  2 +-
 hw/block/virtio-blk.c                  |  2 +-
 hw/char/virtio-serial-bus.c            |  3 +-
 hw/display/virtio-gpu-base.c           |  2 +-
 hw/input/virtio-input.c                |  3 +-
 hw/net/virtio-net.c                    |  2 +-
 hw/scsi/virtio-scsi.c                  |  3 +-
 hw/virtio/vhost-user-fs.c              |  3 +-
 hw/virtio/vhost-user-i2c.c             |  7 +---
 hw/virtio/vhost-user-rng.c             |  2 +-
 hw/virtio/vhost-user-vsock.c           |  2 +-
 hw/virtio/vhost-vsock-common.c         |  5 +--
 hw/virtio/vhost-vsock.c                |  2 +-
 hw/virtio/virtio-balloon.c             |  3 +-
 hw/virtio/virtio-crypto.c              |  2 +-
 hw/virtio/virtio-iommu.c               |  3 +-
 hw/virtio/virtio-mem.c                 |  3 +-
 hw/virtio/virtio-pmem.c                |  3 +-
 hw/virtio/virtio-rng.c                 |  2 +-
 hw/virtio/virtio.c                     | 55 ++++++++++++++++++++++++--
 include/hw/virtio/vhost-vsock-common.h |  2 +-
 include/hw/virtio/virtio-gpu.h         |  3 +-
 include/hw/virtio/virtio.h             |  4 +-
 24 files changed, 77 insertions(+), 43 deletions(-)

diff --git a/hw/9pfs/virtio-9p-device.c b/hw/9pfs/virtio-9p-device.c
index 54ee93b71f..5f522e68e9 100644
--- a/hw/9pfs/virtio-9p-device.c
+++ b/hw/9pfs/virtio-9p-device.c
@@ -216,7 +216,7 @@ static void virtio_9p_device_realize(DeviceState *dev, Error **errp)
     }
 
     v->config_size = sizeof(struct virtio_9p_config) + strlen(s->fsconf.tag);
-    virtio_init(vdev, "virtio-9p", VIRTIO_ID_9P, v->config_size);
+    virtio_init(vdev, VIRTIO_ID_9P, v->config_size);
     v->vq = virtio_add_queue(vdev, MAX_REQ, handle_9p_output);
 }
 
diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index 1a42ae9187..e8cb170032 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -491,7 +491,7 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    virtio_init(vdev, "virtio-blk", VIRTIO_ID_BLOCK,
+    virtio_init(vdev, VIRTIO_ID_BLOCK,
                 sizeof(struct virtio_blk_config));
 
     s->virtqs = g_new(VirtQueue *, s->num_queues);
diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index 6a1cc41877..cd804795c6 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -1206,7 +1206,7 @@ static void virtio_blk_device_realize(DeviceState *dev, Error **errp)
 
     virtio_blk_set_config_size(s, s->host_features);
 
-    virtio_init(vdev, "virtio-blk", VIRTIO_ID_BLOCK, s->config_size);
+    virtio_init(vdev, VIRTIO_ID_BLOCK, s->config_size);
 
     s->blk = conf->conf.blk;
     s->rq = NULL;
diff --git a/hw/char/virtio-serial-bus.c b/hw/char/virtio-serial-bus.c
index 6048d408b8..7d4601cb5d 100644
--- a/hw/char/virtio-serial-bus.c
+++ b/hw/char/virtio-serial-bus.c
@@ -1044,8 +1044,7 @@ static void virtio_serial_device_realize(DeviceState *dev, Error **errp)
                             VIRTIO_CONSOLE_F_EMERG_WRITE)) {
         config_size = offsetof(struct virtio_console_config, emerg_wr);
     }
-    virtio_init(vdev, "virtio-serial", VIRTIO_ID_CONSOLE,
-                config_size);
+    virtio_init(vdev, VIRTIO_ID_CONSOLE, config_size);
 
     /* Spawn a new virtio-serial bus on which the ports will ride as devices */
     qbus_init(&vser->bus, sizeof(vser->bus), TYPE_VIRTIO_SERIAL_BUS,
diff --git a/hw/display/virtio-gpu-base.c b/hw/display/virtio-gpu-base.c
index fff0fb4a82..8ba5da4312 100644
--- a/hw/display/virtio-gpu-base.c
+++ b/hw/display/virtio-gpu-base.c
@@ -173,7 +173,7 @@ virtio_gpu_base_device_realize(DeviceState *qdev,
     }
 
     g->virtio_config.num_scanouts = cpu_to_le32(g->conf.max_outputs);
-    virtio_init(VIRTIO_DEVICE(g), "virtio-gpu", VIRTIO_ID_GPU,
+    virtio_init(VIRTIO_DEVICE(g), VIRTIO_ID_GPU,
                 sizeof(struct virtio_gpu_config));
 
     if (virtio_gpu_virgl_enabled(g->conf)) {
diff --git a/hw/input/virtio-input.c b/hw/input/virtio-input.c
index 54bcb46c74..5b5398b3ca 100644
--- a/hw/input/virtio-input.c
+++ b/hw/input/virtio-input.c
@@ -257,8 +257,7 @@ static void virtio_input_device_realize(DeviceState *dev, Error **errp)
     vinput->cfg_size += 8;
     assert(vinput->cfg_size <= sizeof(virtio_input_config));
 
-    virtio_init(vdev, "virtio-input", VIRTIO_ID_INPUT,
-                vinput->cfg_size);
+    virtio_init(vdev, VIRTIO_ID_INPUT, vinput->cfg_size);
     vinput->evt = virtio_add_queue(vdev, 64, virtio_input_handle_evt);
     vinput->sts = virtio_add_queue(vdev, 64, virtio_input_handle_sts);
 }
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index e4748a7e6c..bd0224caaf 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -3393,7 +3393,7 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
     }
 
     virtio_net_set_config_size(n, n->host_features);
-    virtio_init(vdev, "virtio-net", VIRTIO_ID_NET, n->config_size);
+    virtio_init(vdev, VIRTIO_ID_NET, n->config_size);
 
     /*
      * We set a lower limit on RX queue size to what it always was.
diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
index db54d104be..4141dddd51 100644
--- a/hw/scsi/virtio-scsi.c
+++ b/hw/scsi/virtio-scsi.c
@@ -1013,8 +1013,7 @@ void virtio_scsi_common_realize(DeviceState *dev,
     VirtIOSCSICommon *s = VIRTIO_SCSI_COMMON(dev);
     int i;
 
-    virtio_init(vdev, "virtio-scsi", VIRTIO_ID_SCSI,
-                sizeof(VirtIOSCSIConfig));
+    virtio_init(vdev, VIRTIO_ID_SCSI, sizeof(VirtIOSCSIConfig));
 
     if (s->conf.num_queues == VIRTIO_SCSI_AUTO_NUM_QUEUES) {
         s->conf.num_queues = 1;
diff --git a/hw/virtio/vhost-user-fs.c b/hw/virtio/vhost-user-fs.c
index c595957983..b875640147 100644
--- a/hw/virtio/vhost-user-fs.c
+++ b/hw/virtio/vhost-user-fs.c
@@ -219,8 +219,7 @@ static void vuf_device_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    virtio_init(vdev, "vhost-user-fs", VIRTIO_ID_FS,
-                sizeof(struct virtio_fs_config));
+    virtio_init(vdev, VIRTIO_ID_FS, sizeof(struct virtio_fs_config));
 
     /* Hiprio queue */
     fs->hiprio_vq = virtio_add_queue(vdev, fs->conf.queue_size, vuf_handle_output);
diff --git a/hw/virtio/vhost-user-i2c.c b/hw/virtio/vhost-user-i2c.c
index 42c7f6d9e5..6020eee093 100644
--- a/hw/virtio/vhost-user-i2c.c
+++ b/hw/virtio/vhost-user-i2c.c
@@ -14,11 +14,6 @@
 #include "qemu/error-report.h"
 #include "standard-headers/linux/virtio_ids.h"
 
-/* Remove this once the header is updated in Linux kernel */
-#ifndef VIRTIO_ID_I2C_ADAPTER
-#define VIRTIO_ID_I2C_ADAPTER                34
-#endif
-
 static const int feature_bits[] = {
     VIRTIO_I2C_F_ZERO_LENGTH_REQUEST,
     VHOST_INVALID_FEATURE_BIT
@@ -227,7 +222,7 @@ static void vu_i2c_device_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    virtio_init(vdev, "vhost-user-i2c", VIRTIO_ID_I2C_ADAPTER, 0);
+    virtio_init(vdev, VIRTIO_ID_I2C_ADAPTER, 0);
 
     i2c->vhost_dev.nvqs = 1;
     i2c->vq = virtio_add_queue(vdev, 4, vu_i2c_handle_output);
diff --git a/hw/virtio/vhost-user-rng.c b/hw/virtio/vhost-user-rng.c
index 209ee5bf9a..08bccba9dc 100644
--- a/hw/virtio/vhost-user-rng.c
+++ b/hw/virtio/vhost-user-rng.c
@@ -203,7 +203,7 @@ static void vu_rng_device_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    virtio_init(vdev, "vhost-user-rng", VIRTIO_ID_RNG, 0);
+    virtio_init(vdev, VIRTIO_ID_RNG, 0);
 
     rng->req_vq = virtio_add_queue(vdev, 4, vu_rng_handle_output);
     if (!rng->req_vq) {
diff --git a/hw/virtio/vhost-user-vsock.c b/hw/virtio/vhost-user-vsock.c
index 52bd682c34..0f8ff99f85 100644
--- a/hw/virtio/vhost-user-vsock.c
+++ b/hw/virtio/vhost-user-vsock.c
@@ -107,7 +107,7 @@ static void vuv_device_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    vhost_vsock_common_realize(vdev, "vhost-user-vsock");
+    vhost_vsock_common_realize(vdev);
 
     vhost_dev_set_config_notifier(&vvc->vhost_dev, &vsock_ops);
 
diff --git a/hw/virtio/vhost-vsock-common.c b/hw/virtio/vhost-vsock-common.c
index ed706681ac..ad5c8ff5d5 100644
--- a/hw/virtio/vhost-vsock-common.c
+++ b/hw/virtio/vhost-vsock-common.c
@@ -224,12 +224,11 @@ int vhost_vsock_common_post_load(void *opaque, int version_id)
     return 0;
 }
 
-void vhost_vsock_common_realize(VirtIODevice *vdev, const char *name)
+void vhost_vsock_common_realize(VirtIODevice *vdev)
 {
     VHostVSockCommon *vvc = VHOST_VSOCK_COMMON(vdev);
 
-    virtio_init(vdev, name, VIRTIO_ID_VSOCK,
-                sizeof(struct virtio_vsock_config));
+    virtio_init(vdev, VIRTIO_ID_VSOCK, sizeof(struct virtio_vsock_config));
 
     /* Receive and transmit queues belong to vhost */
     vvc->recv_vq = virtio_add_queue(vdev, VHOST_VSOCK_QUEUE_SIZE,
diff --git a/hw/virtio/vhost-vsock.c b/hw/virtio/vhost-vsock.c
index 714046210b..0338de892f 100644
--- a/hw/virtio/vhost-vsock.c
+++ b/hw/virtio/vhost-vsock.c
@@ -169,7 +169,7 @@ static void vhost_vsock_device_realize(DeviceState *dev, Error **errp)
         }
     }
 
-    vhost_vsock_common_realize(vdev, "vhost-vsock");
+    vhost_vsock_common_realize(vdev);
 
     ret = vhost_dev_init(&vvc->vhost_dev, (void *)(uintptr_t)vhostfd,
                          VHOST_BACKEND_TYPE_KERNEL, 0, errp);
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index 8f1b38ef5c..73ac5eb675 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -882,8 +882,7 @@ static void virtio_balloon_device_realize(DeviceState *dev, Error **errp)
     VirtIOBalloon *s = VIRTIO_BALLOON(dev);
     int ret;
 
-    virtio_init(vdev, "virtio-balloon", VIRTIO_ID_BALLOON,
-                virtio_balloon_config_size(s));
+    virtio_init(vdev, VIRTIO_ID_BALLOON, virtio_balloon_config_size(s));
 
     ret = qemu_add_balloon_handler(virtio_balloon_to_target,
                                    virtio_balloon_stat, s);
diff --git a/hw/virtio/virtio-crypto.c b/hw/virtio/virtio-crypto.c
index 0e31e3cc04..df9adb2c07 100644
--- a/hw/virtio/virtio-crypto.c
+++ b/hw/virtio/virtio-crypto.c
@@ -810,7 +810,7 @@ static void virtio_crypto_device_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    virtio_init(vdev, "virtio-crypto", VIRTIO_ID_CRYPTO, vcrypto->config_size);
+    virtio_init(vdev, VIRTIO_ID_CRYPTO, vcrypto->config_size);
     vcrypto->curr_queues = 1;
     vcrypto->vqs = g_new0(VirtIOCryptoQueue, vcrypto->max_queues);
     for (i = 0; i < vcrypto->max_queues; i++) {
diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
index 57c09d98a9..2597e166f9 100644
--- a/hw/virtio/virtio-iommu.c
+++ b/hw/virtio/virtio-iommu.c
@@ -1033,8 +1033,7 @@ static void virtio_iommu_device_realize(DeviceState *dev, Error **errp)
     VirtIODevice *vdev = VIRTIO_DEVICE(dev);
     VirtIOIOMMU *s = VIRTIO_IOMMU(dev);
 
-    virtio_init(vdev, "virtio-iommu", VIRTIO_ID_IOMMU,
-                sizeof(struct virtio_iommu_config));
+    virtio_init(vdev, VIRTIO_ID_IOMMU, sizeof(struct virtio_iommu_config));
 
     memset(s->iommu_pcibus_by_bus_num, 0, sizeof(s->iommu_pcibus_by_bus_num));
 
diff --git a/hw/virtio/virtio-mem.c b/hw/virtio/virtio-mem.c
index 5aca408726..30d03e987a 100644
--- a/hw/virtio/virtio-mem.c
+++ b/hw/virtio/virtio-mem.c
@@ -867,8 +867,7 @@ static void virtio_mem_device_realize(DeviceState *dev, Error **errp)
                         vmem->block_size;
     vmem->bitmap = bitmap_new(vmem->bitmap_size);
 
-    virtio_init(vdev, TYPE_VIRTIO_MEM, VIRTIO_ID_MEM,
-                sizeof(struct virtio_mem_config));
+    virtio_init(vdev, VIRTIO_ID_MEM, sizeof(struct virtio_mem_config));
     vmem->vq = virtio_add_queue(vdev, 128, virtio_mem_handle_request);
 
     host_memory_backend_set_mapped(vmem->memdev, true);
diff --git a/hw/virtio/virtio-pmem.c b/hw/virtio/virtio-pmem.c
index 5dd21c2c44..a1abfe0e1b 100644
--- a/hw/virtio/virtio-pmem.c
+++ b/hw/virtio/virtio-pmem.c
@@ -122,8 +122,7 @@ static void virtio_pmem_realize(DeviceState *dev, Error **errp)
     }
 
     host_memory_backend_set_mapped(pmem->memdev, true);
-    virtio_init(vdev, TYPE_VIRTIO_PMEM, VIRTIO_ID_PMEM,
-                sizeof(struct virtio_pmem_config));
+    virtio_init(vdev, VIRTIO_ID_PMEM, sizeof(struct virtio_pmem_config));
     pmem->rq_vq = virtio_add_queue(vdev, 128, virtio_pmem_flush);
 }
 
diff --git a/hw/virtio/virtio-rng.c b/hw/virtio/virtio-rng.c
index cc8e9f775d..7e12fc03bf 100644
--- a/hw/virtio/virtio-rng.c
+++ b/hw/virtio/virtio-rng.c
@@ -215,7 +215,7 @@ static void virtio_rng_device_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    virtio_init(vdev, "virtio-rng", VIRTIO_ID_RNG, 0);
+    virtio_init(vdev, VIRTIO_ID_RNG, 0);
 
     vrng->vq = virtio_add_queue(vdev, 8, handle_input);
     vrng->quota_remaining = vrng->conf.max_bytes;
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 67a873f54a..7f9fcfd06d 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -132,6 +132,56 @@ struct VirtQueue
     QLIST_ENTRY(VirtQueue) node;
 };
 
+const char *virtio_device_names[] = {
+    [VIRTIO_ID_NET] = "virtio-net",
+    [VIRTIO_ID_BLOCK] = "virtio-blk",
+    [VIRTIO_ID_CONSOLE] = "virtio-serial",
+    [VIRTIO_ID_RNG] = "virtio-rng",
+    [VIRTIO_ID_BALLOON] = "virtio-balloon",
+    [VIRTIO_ID_IOMEM] = "virtio-iomem",
+    [VIRTIO_ID_RPMSG] = "virtio-rpmsg",
+    [VIRTIO_ID_SCSI] = "virtio-scsi",
+    [VIRTIO_ID_9P] = "virtio-9p",
+    [VIRTIO_ID_MAC80211_WLAN] = "virtio-mac-wlan",
+    [VIRTIO_ID_RPROC_SERIAL] = "virtio-rproc-serial",
+    [VIRTIO_ID_CAIF] = "virtio-caif",
+    [VIRTIO_ID_MEMORY_BALLOON] = "virtio-mem-balloon",
+    [VIRTIO_ID_GPU] = "virtio-gpu",
+    [VIRTIO_ID_CLOCK] = "virtio-clk",
+    [VIRTIO_ID_INPUT] = "virtio-input",
+    [VIRTIO_ID_VSOCK] = "vhost-vsock",
+    [VIRTIO_ID_CRYPTO] = "virtio-crypto",
+    [VIRTIO_ID_SIGNAL_DIST] = "virtio-signal",
+    [VIRTIO_ID_PSTORE] = "virtio-pstore",
+    [VIRTIO_ID_IOMMU] = "virtio-iommu",
+    [VIRTIO_ID_MEM] = "virtio-mem",
+    [VIRTIO_ID_SOUND] = "virtio-sound",
+    [VIRTIO_ID_FS] = "virtio-user-fs",
+    [VIRTIO_ID_PMEM] = "virtio-pmem",
+    [VIRTIO_ID_RPMB] = "virtio-rpmb",
+    [VIRTIO_ID_MAC80211_HWSIM] = "virtio-mac-hwsim",
+    [VIRTIO_ID_VIDEO_ENCODER] = "virtio-vid-encoder",
+    [VIRTIO_ID_VIDEO_DECODER] = "virtio-vid-decoder",
+    [VIRTIO_ID_SCMI] = "virtio-scmi",
+    [VIRTIO_ID_NITRO_SEC_MOD] = "virtio-nitro-sec-mod",
+    [VIRTIO_ID_I2C_ADAPTER] = "vhost-user-i2c",
+    [VIRTIO_ID_WATCHDOG] = "virtio-watchdog",
+    [VIRTIO_ID_CAN] = "virtio-can",
+    [VIRTIO_ID_DMABUF] = "virtio-dmabuf",
+    [VIRTIO_ID_PARAM_SERV] = "virtio-param-serv",
+    [VIRTIO_ID_AUDIO_POLICY] = "virtio-audio-pol",
+    [VIRTIO_ID_BT] = "virtio-bluetooth",
+    [VIRTIO_ID_GPIO] = "virtio-gpio"
+};
+
+static const char *virtio_id_to_name(uint16_t device_id)
+{
+    assert(device_id < G_N_ELEMENTS(virtio_device_names));
+    const char *name = virtio_device_names[device_id];
+    assert(name != NULL);
+    return name;
+}
+
 /* Called within call_rcu().  */
 static void virtio_free_region_cache(VRingMemoryRegionCaches *caches)
 {
@@ -3207,8 +3257,7 @@ void virtio_instance_init_common(Object *proxy_obj, void *data,
     qdev_alias_all_properties(vdev, proxy_obj);
 }
 
-void virtio_init(VirtIODevice *vdev, const char *name,
-                 uint16_t device_id, size_t config_size)
+void virtio_init(VirtIODevice *vdev, uint16_t device_id, size_t config_size)
 {
     BusState *qbus = qdev_get_parent_bus(DEVICE(vdev));
     VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus);
@@ -3237,7 +3286,7 @@ void virtio_init(VirtIODevice *vdev, const char *name,
         vdev->vq[i].host_notifier_enabled = false;
     }
 
-    vdev->name = name;
+    vdev->name = virtio_id_to_name(device_id);
     vdev->config_len = config_size;
     if (vdev->config_len) {
         vdev->config = g_malloc0(config_size);
diff --git a/include/hw/virtio/vhost-vsock-common.h b/include/hw/virtio/vhost-vsock-common.h
index 456a9c2365..93c782101d 100644
--- a/include/hw/virtio/vhost-vsock-common.h
+++ b/include/hw/virtio/vhost-vsock-common.h
@@ -44,7 +44,7 @@ int vhost_vsock_common_start(VirtIODevice *vdev);
 void vhost_vsock_common_stop(VirtIODevice *vdev);
 int vhost_vsock_common_pre_save(void *opaque);
 int vhost_vsock_common_post_load(void *opaque, int version_id);
-void vhost_vsock_common_realize(VirtIODevice *vdev, const char *name);
+void vhost_vsock_common_realize(VirtIODevice *vdev);
 void vhost_vsock_common_unrealize(VirtIODevice *vdev);
 uint64_t vhost_vsock_common_get_features(VirtIODevice *vdev, uint64_t features,
                                          Error **errp);
diff --git a/include/hw/virtio/virtio-gpu.h b/include/hw/virtio/virtio-gpu.h
index 2179b75703..afff9e158e 100644
--- a/include/hw/virtio/virtio-gpu.h
+++ b/include/hw/virtio/virtio-gpu.h
@@ -22,6 +22,7 @@
 #include "sysemu/vhost-user-backend.h"
 
 #include "standard-headers/linux/virtio_gpu.h"
+#include "standard-headers/linux/virtio_ids.h"
 #include "qom/object.h"
 
 #define TYPE_VIRTIO_GPU_BASE "virtio-gpu-base"
@@ -37,8 +38,6 @@ OBJECT_DECLARE_SIMPLE_TYPE(VirtIOGPUGL, VIRTIO_GPU_GL)
 #define TYPE_VHOST_USER_GPU "vhost-user-gpu"
 OBJECT_DECLARE_SIMPLE_TYPE(VhostUserGPU, VHOST_USER_GPU)
 
-#define VIRTIO_ID_GPU 16
-
 struct virtio_gpu_simple_resource {
     uint32_t resource_id;
     uint32_t width;
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index b62a35fdca..b6890ffa70 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -165,8 +165,8 @@ struct VirtioDeviceClass {
 void virtio_instance_init_common(Object *proxy_obj, void *data,
                                  size_t vdev_size, const char *vdev_name);
 
-void virtio_init(VirtIODevice *vdev, const char *name,
-                         uint16_t device_id, size_t config_size);
+void virtio_init(VirtIODevice *vdev, uint16_t device_id, size_t config_size);
+
 void virtio_cleanup(VirtIODevice *vdev);
 
 void virtio_error(VirtIODevice *vdev, const char *fmt, ...) G_GNUC_PRINTF(2, 3);

From c255488d67dc0d52052a12e0242bf00dcbe5223f Mon Sep 17 00:00:00 2001
From: Jonah Palmer <jonah.palmer@oracle.com>
Date: Fri, 1 Apr 2022 09:23:19 -0400
Subject: [PATCH 376/935] virtio: add vhost support for virtio devices

This patch adds a get_vhost() callback function for VirtIODevices that
returns the device's corresponding vhost_dev structure, if the vhost
device is running. This patch also adds a vhost_started flag for
VirtIODevices.

Previously, a VirtIODevice wouldn't be able to tell if its corresponding
vhost device was active or not.

Signed-off-by: Jonah Palmer <jonah.palmer@oracle.com>
Message-Id: <1648819405-25696-3-git-send-email-jonah.palmer@oracle.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/block/vhost-user-blk.c      |  7 +++++++
 hw/display/vhost-user-gpu.c    |  7 +++++++
 hw/input/vhost-user-input.c    |  7 +++++++
 hw/net/virtio-net.c            |  9 +++++++++
 hw/scsi/vhost-scsi.c           |  8 ++++++++
 hw/virtio/vhost-user-fs.c      |  7 +++++++
 hw/virtio/vhost-user-rng.c     |  7 +++++++
 hw/virtio/vhost-vsock-common.c |  7 +++++++
 hw/virtio/vhost.c              |  4 +++-
 hw/virtio/virtio-crypto.c      | 10 ++++++++++
 hw/virtio/virtio.c             |  1 +
 include/hw/virtio/virtio.h     |  3 +++
 12 files changed, 76 insertions(+), 1 deletion(-)

diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index e8cb170032..5dca4eab09 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -569,6 +569,12 @@ static void vhost_user_blk_instance_init(Object *obj)
                                   "/disk@0,0", DEVICE(obj));
 }
 
+static struct vhost_dev *vhost_user_blk_get_vhost(VirtIODevice *vdev)
+{
+    VHostUserBlk *s = VHOST_USER_BLK(vdev);
+    return &s->dev;
+}
+
 static const VMStateDescription vmstate_vhost_user_blk = {
     .name = "vhost-user-blk",
     .minimum_version_id = 1,
@@ -603,6 +609,7 @@ static void vhost_user_blk_class_init(ObjectClass *klass, void *data)
     vdc->get_features = vhost_user_blk_get_features;
     vdc->set_status = vhost_user_blk_set_status;
     vdc->reset = vhost_user_blk_reset;
+    vdc->get_vhost = vhost_user_blk_get_vhost;
 }
 
 static const TypeInfo vhost_user_blk_info = {
diff --git a/hw/display/vhost-user-gpu.c b/hw/display/vhost-user-gpu.c
index 09818231bd..96e56c4467 100644
--- a/hw/display/vhost-user-gpu.c
+++ b/hw/display/vhost-user-gpu.c
@@ -565,6 +565,12 @@ vhost_user_gpu_device_realize(DeviceState *qdev, Error **errp)
     g->vhost_gpu_fd = -1;
 }
 
+static struct vhost_dev *vhost_user_gpu_get_vhost(VirtIODevice *vdev)
+{
+    VhostUserGPU *g = VHOST_USER_GPU(vdev);
+    return &g->vhost->dev;
+}
+
 static Property vhost_user_gpu_properties[] = {
     VIRTIO_GPU_BASE_PROPERTIES(VhostUserGPU, parent_obj.conf),
     DEFINE_PROP_END_OF_LIST(),
@@ -586,6 +592,7 @@ vhost_user_gpu_class_init(ObjectClass *klass, void *data)
     vdc->guest_notifier_pending = vhost_user_gpu_guest_notifier_pending;
     vdc->get_config = vhost_user_gpu_get_config;
     vdc->set_config = vhost_user_gpu_set_config;
+    vdc->get_vhost = vhost_user_gpu_get_vhost;
 
     device_class_set_props(dc, vhost_user_gpu_properties);
 }
diff --git a/hw/input/vhost-user-input.c b/hw/input/vhost-user-input.c
index aeb0624fe5..1352e372ff 100644
--- a/hw/input/vhost-user-input.c
+++ b/hw/input/vhost-user-input.c
@@ -78,6 +78,12 @@ static void vhost_input_set_config(VirtIODevice *vdev,
     virtio_notify_config(vdev);
 }
 
+static struct vhost_dev *vhost_input_get_vhost(VirtIODevice *vdev)
+{
+    VHostUserInput *vhi = VHOST_USER_INPUT(vdev);
+    return &vhi->vhost->dev;
+}
+
 static const VMStateDescription vmstate_vhost_input = {
     .name = "vhost-user-input",
     .unmigratable = 1,
@@ -92,6 +98,7 @@ static void vhost_input_class_init(ObjectClass *klass, void *data)
     dc->vmsd = &vmstate_vhost_input;
     vdc->get_config = vhost_input_get_config;
     vdc->set_config = vhost_input_set_config;
+    vdc->get_vhost = vhost_input_get_vhost;
     vic->realize = vhost_input_realize;
     vic->change_active = vhost_input_change_active;
 }
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index bd0224caaf..a788f1c5c7 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -3620,6 +3620,14 @@ static bool dev_unplug_pending(void *opaque)
     return vdc->primary_unplug_pending(dev);
 }
 
+static struct vhost_dev *virtio_net_get_vhost(VirtIODevice *vdev)
+{
+    VirtIONet *n = VIRTIO_NET(vdev);
+    NetClientState *nc = qemu_get_queue(n->nic);
+    struct vhost_net *net = get_vhost_net(nc->peer);
+    return &net->dev;
+}
+
 static const VMStateDescription vmstate_virtio_net = {
     .name = "virtio-net",
     .minimum_version_id = VIRTIO_NET_VM_VERSION,
@@ -3722,6 +3730,7 @@ static void virtio_net_class_init(ObjectClass *klass, void *data)
     vdc->post_load = virtio_net_post_load_virtio;
     vdc->vmsd = &vmstate_virtio_net_device;
     vdc->primary_unplug_pending = primary_unplug_pending;
+    vdc->get_vhost = virtio_net_get_vhost;
 }
 
 static const TypeInfo virtio_net_info = {
diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c
index 778f43e4c1..3059068175 100644
--- a/hw/scsi/vhost-scsi.c
+++ b/hw/scsi/vhost-scsi.c
@@ -273,6 +273,13 @@ static void vhost_scsi_unrealize(DeviceState *dev)
     virtio_scsi_common_unrealize(dev);
 }
 
+static struct vhost_dev *vhost_scsi_get_vhost(VirtIODevice *vdev)
+{
+    VHostSCSI *s = VHOST_SCSI(vdev);
+    VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s);
+    return &vsc->dev;
+}
+
 static Property vhost_scsi_properties[] = {
     DEFINE_PROP_STRING("vhostfd", VirtIOSCSICommon, conf.vhostfd),
     DEFINE_PROP_STRING("wwpn", VirtIOSCSICommon, conf.wwpn),
@@ -307,6 +314,7 @@ static void vhost_scsi_class_init(ObjectClass *klass, void *data)
     vdc->get_features = vhost_scsi_common_get_features;
     vdc->set_config = vhost_scsi_common_set_config;
     vdc->set_status = vhost_scsi_set_status;
+    vdc->get_vhost = vhost_scsi_get_vhost;
     fwc->get_dev_path = vhost_scsi_common_get_fw_dev_path;
 }
 
diff --git a/hw/virtio/vhost-user-fs.c b/hw/virtio/vhost-user-fs.c
index b875640147..e513e4fdda 100644
--- a/hw/virtio/vhost-user-fs.c
+++ b/hw/virtio/vhost-user-fs.c
@@ -276,6 +276,12 @@ static void vuf_device_unrealize(DeviceState *dev)
     fs->vhost_dev.vqs = NULL;
 }
 
+static struct vhost_dev *vuf_get_vhost(VirtIODevice *vdev)
+{
+    VHostUserFS *fs = VHOST_USER_FS(vdev);
+    return &fs->vhost_dev;
+}
+
 static const VMStateDescription vuf_vmstate = {
     .name = "vhost-user-fs",
     .unmigratable = 1,
@@ -313,6 +319,7 @@ static void vuf_class_init(ObjectClass *klass, void *data)
     vdc->set_status = vuf_set_status;
     vdc->guest_notifier_mask = vuf_guest_notifier_mask;
     vdc->guest_notifier_pending = vuf_guest_notifier_pending;
+    vdc->get_vhost = vuf_get_vhost;
 }
 
 static const TypeInfo vuf_info = {
diff --git a/hw/virtio/vhost-user-rng.c b/hw/virtio/vhost-user-rng.c
index 08bccba9dc..3a7bf8e32d 100644
--- a/hw/virtio/vhost-user-rng.c
+++ b/hw/virtio/vhost-user-rng.c
@@ -247,6 +247,12 @@ static void vu_rng_device_unrealize(DeviceState *dev)
     vhost_user_cleanup(&rng->vhost_user);
 }
 
+static struct vhost_dev *vu_rng_get_vhost(VirtIODevice *vdev)
+{
+    VHostUserRNG *rng = VHOST_USER_RNG(vdev);
+    return &rng->vhost_dev;
+}
+
 static const VMStateDescription vu_rng_vmstate = {
     .name = "vhost-user-rng",
     .unmigratable = 1,
@@ -272,6 +278,7 @@ static void vu_rng_class_init(ObjectClass *klass, void *data)
     vdc->set_status = vu_rng_set_status;
     vdc->guest_notifier_mask = vu_rng_guest_notifier_mask;
     vdc->guest_notifier_pending = vu_rng_guest_notifier_pending;
+    vdc->get_vhost = vu_rng_get_vhost;
 }
 
 static const TypeInfo vu_rng_info = {
diff --git a/hw/virtio/vhost-vsock-common.c b/hw/virtio/vhost-vsock-common.c
index ad5c8ff5d5..7394818e00 100644
--- a/hw/virtio/vhost-vsock-common.c
+++ b/hw/virtio/vhost-vsock-common.c
@@ -258,6 +258,12 @@ void vhost_vsock_common_unrealize(VirtIODevice *vdev)
     virtio_cleanup(vdev);
 }
 
+static struct vhost_dev *vhost_vsock_common_get_vhost(VirtIODevice *vdev)
+{
+    VHostVSockCommon *vvc = VHOST_VSOCK_COMMON(vdev);
+    return &vvc->vhost_dev;
+}
+
 static Property vhost_vsock_common_properties[] = {
     DEFINE_PROP_ON_OFF_AUTO("seqpacket", VHostVSockCommon, seqpacket,
                             ON_OFF_AUTO_AUTO),
@@ -273,6 +279,7 @@ static void vhost_vsock_common_class_init(ObjectClass *klass, void *data)
     set_bit(DEVICE_CATEGORY_MISC, dc->categories);
     vdc->guest_notifier_mask = vhost_vsock_common_guest_notifier_mask;
     vdc->guest_notifier_pending = vhost_vsock_common_guest_notifier_pending;
+    vdc->get_vhost = vhost_vsock_common_get_vhost;
 }
 
 static const TypeInfo vhost_vsock_common_info = {
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 2bc72c27c5..dd3263df56 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1738,6 +1738,7 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev)
     /* should only be called after backend is connected */
     assert(hdev->vhost_ops);
 
+    vdev->vhost_started = true;
     hdev->started = true;
     hdev->vdev = vdev;
 
@@ -1810,7 +1811,7 @@ fail_vq:
 
 fail_mem:
 fail_features:
-
+    vdev->vhost_started = false;
     hdev->started = false;
     return r;
 }
@@ -1841,6 +1842,7 @@ void vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev)
     }
     vhost_log_put(hdev, true);
     hdev->started = false;
+    vdev->vhost_started = false;
     hdev->vdev = NULL;
 }
 
diff --git a/hw/virtio/virtio-crypto.c b/hw/virtio/virtio-crypto.c
index df9adb2c07..c3829e7498 100644
--- a/hw/virtio/virtio-crypto.c
+++ b/hw/virtio/virtio-crypto.c
@@ -961,6 +961,15 @@ static bool virtio_crypto_guest_notifier_pending(VirtIODevice *vdev, int idx)
     return cryptodev_vhost_virtqueue_pending(vdev, queue, idx);
 }
 
+static struct vhost_dev *virtio_crypto_get_vhost(VirtIODevice *vdev)
+{
+    VirtIOCrypto *vcrypto = VIRTIO_CRYPTO(vdev);
+    CryptoDevBackend *b = vcrypto->cryptodev;
+    CryptoDevBackendClient *cc = b->conf.peers.ccs[0];
+    CryptoDevBackendVhost *vhost_crypto = cryptodev_get_vhost(cc, b, 0);
+    return &vhost_crypto->dev;
+}
+
 static void virtio_crypto_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -977,6 +986,7 @@ static void virtio_crypto_class_init(ObjectClass *klass, void *data)
     vdc->set_status = virtio_crypto_set_status;
     vdc->guest_notifier_mask = virtio_crypto_guest_notifier_mask;
     vdc->guest_notifier_pending = virtio_crypto_guest_notifier_pending;
+    vdc->get_vhost = virtio_crypto_get_vhost;
 }
 
 static void virtio_crypto_instance_init(Object *obj)
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 7f9fcfd06d..5d607aeaa0 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -3271,6 +3271,7 @@ void virtio_init(VirtIODevice *vdev, uint16_t device_id, size_t config_size)
 
     vdev->start_on_kick = false;
     vdev->started = false;
+    vdev->vhost_started = false;
     vdev->device_id = device_id;
     vdev->status = 0;
     qatomic_set(&vdev->isr, 0);
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index b6890ffa70..db1c0ddf6b 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -22,6 +22,7 @@
 #include "standard-headers/linux/virtio_config.h"
 #include "standard-headers/linux/virtio_ring.h"
 #include "qom/object.h"
+#include "hw/virtio/vhost.h"
 
 /* A guest should never accept this.  It implies negotiation is broken. */
 #define VIRTIO_F_BAD_FEATURE		30
@@ -102,6 +103,7 @@ struct VirtIODevice
     bool started;
     bool start_on_kick; /* when virtio 1.0 feature has not been negotiated */
     bool disable_legacy_check;
+    bool vhost_started;
     VMChangeStateEntry *vmstate;
     char *bus_name;
     uint8_t device_endian;
@@ -160,6 +162,7 @@ struct VirtioDeviceClass {
     int (*post_load)(VirtIODevice *vdev);
     const VMStateDescription *vmsd;
     bool (*primary_unplug_pending)(void *opaque);
+    struct vhost_dev *(*get_vhost)(VirtIODevice *vdev);
 };
 
 void virtio_instance_init_common(Object *proxy_obj, void *data,

From c98ce274dbd6373589ae01b652d88f93633db830 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Tue, 26 Apr 2022 17:00:40 +0100
Subject: [PATCH 377/935] tests: fix encoding of IP addresses in x509 certs
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

We need to encode just the address bytes, not the whole struct sockaddr
data. Add a test case to validate that we're matching on SAN IP
addresses correctly.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220426160048.812266-2-berrange@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 tests/unit/crypto-tls-x509-helpers.c | 16 +++++++++++++---
 tests/unit/test-crypto-tlssession.c  | 11 +++++++++--
 2 files changed, 22 insertions(+), 5 deletions(-)

diff --git a/tests/unit/crypto-tls-x509-helpers.c b/tests/unit/crypto-tls-x509-helpers.c
index fc609b3fd4..e9937f60d8 100644
--- a/tests/unit/crypto-tls-x509-helpers.c
+++ b/tests/unit/crypto-tls-x509-helpers.c
@@ -168,9 +168,19 @@ test_tls_get_ipaddr(const char *addrstr,
     hints.ai_flags = AI_NUMERICHOST;
     g_assert(getaddrinfo(addrstr, NULL, &hints, &res) == 0);
 
-    *datalen = res->ai_addrlen;
-    *data = g_new(char, *datalen);
-    memcpy(*data, res->ai_addr, *datalen);
+    if (res->ai_family == AF_INET) {
+        struct sockaddr_in *in = (struct sockaddr_in *)res->ai_addr;
+        *datalen = sizeof(in->sin_addr);
+        *data = g_new(char, *datalen);
+        memcpy(*data, &in->sin_addr, *datalen);
+    } else if (res->ai_family == AF_INET6) {
+        struct sockaddr_in6 *in = (struct sockaddr_in6 *)res->ai_addr;
+        *datalen = sizeof(in->sin6_addr);
+        *data = g_new(char, *datalen);
+        memcpy(*data, &in->sin6_addr, *datalen);
+    } else {
+        g_assert_not_reached();
+    }
     freeaddrinfo(res);
 }
 
diff --git a/tests/unit/test-crypto-tlssession.c b/tests/unit/test-crypto-tlssession.c
index a266dc32da..f222959d36 100644
--- a/tests/unit/test-crypto-tlssession.c
+++ b/tests/unit/test-crypto-tlssession.c
@@ -512,12 +512,19 @@ int main(int argc, char **argv)
                   false, true, "wiki.qemu.org", NULL);
 
     TEST_SESS_REG(altname4, cacertreq.filename,
+                  servercertalt1req.filename, clientcertreq.filename,
+                  false, false, "192.168.122.1", NULL);
+    TEST_SESS_REG(altname5, cacertreq.filename,
+                  servercertalt1req.filename, clientcertreq.filename,
+                  false, false, "fec0::dead:beaf", NULL);
+
+    TEST_SESS_REG(altname6, cacertreq.filename,
                   servercertalt2req.filename, clientcertreq.filename,
                   false, true, "qemu.org", NULL);
-    TEST_SESS_REG(altname5, cacertreq.filename,
+    TEST_SESS_REG(altname7, cacertreq.filename,
                   servercertalt2req.filename, clientcertreq.filename,
                   false, false, "www.qemu.org", NULL);
-    TEST_SESS_REG(altname6, cacertreq.filename,
+    TEST_SESS_REG(altname8, cacertreq.filename,
                   servercertalt2req.filename, clientcertreq.filename,
                   false, false, "wiki.qemu.org", NULL);
 

From 5bc6364bfb496623cc7f856bdb0358ffbe3c18d2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Tue, 26 Apr 2022 17:00:41 +0100
Subject: [PATCH 378/935] tests: add more helper macros for creating TLS x509
 certs
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

These macros are more suited to the general consumers of certs in the
test suite, where we don't need to exercise every single possible
permutation.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220426160048.812266-3-berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 tests/unit/crypto-tls-x509-helpers.h | 53 ++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/tests/unit/crypto-tls-x509-helpers.h b/tests/unit/crypto-tls-x509-helpers.h
index cf6329e653..247e7160eb 100644
--- a/tests/unit/crypto-tls-x509-helpers.h
+++ b/tests/unit/crypto-tls-x509-helpers.h
@@ -26,6 +26,9 @@
 #include <libtasn1.h>
 
 
+#define QCRYPTO_TLS_TEST_CLIENT_NAME "ACME QEMU Client"
+#define QCRYPTO_TLS_TEST_CLIENT_HOSTILE_NAME "ACME Hostile Client"
+
 /*
  * This contains parameter about how to generate
  * certificates.
@@ -118,6 +121,56 @@ void test_tls_cleanup(const char *keyfile);
     };                                                                  \
     test_tls_generate_cert(&varname, NULL)
 
+# define TLS_ROOT_REQ_SIMPLE(varname, fname)                            \
+    QCryptoTLSTestCertReq varname = {                                   \
+        .filename = fname,                                              \
+        .cn = "qemu-CA",                                                \
+        .basicConstraintsEnable = true,                                 \
+        .basicConstraintsCritical = true,                               \
+        .basicConstraintsIsCA = true,                                   \
+        .keyUsageEnable = true,                                         \
+        .keyUsageCritical = true,                                       \
+        .keyUsageValue = GNUTLS_KEY_KEY_CERT_SIGN,                      \
+    };                                                                  \
+    test_tls_generate_cert(&varname, NULL)
+
+# define TLS_CERT_REQ_SIMPLE_CLIENT(varname, cavarname, cname, fname)   \
+    QCryptoTLSTestCertReq varname = {                                   \
+        .filename = fname,                                              \
+        .cn = cname,                                                    \
+        .basicConstraintsEnable = true,                                 \
+        .basicConstraintsCritical = true,                               \
+        .basicConstraintsIsCA = false,                                  \
+        .keyUsageEnable = true,                                         \
+        .keyUsageCritical = true,                                       \
+        .keyUsageValue =                                                \
+        GNUTLS_KEY_DIGITAL_SIGNATURE | GNUTLS_KEY_KEY_ENCIPHERMENT,     \
+        .keyPurposeEnable = true,                                       \
+        .keyPurposeCritical = true,                                     \
+        .keyPurposeOID1 = GNUTLS_KP_TLS_WWW_CLIENT,                     \
+    };                                                                  \
+    test_tls_generate_cert(&varname, cavarname.crt)
+
+# define TLS_CERT_REQ_SIMPLE_SERVER(varname, cavarname, fname,          \
+                                    hostname, ipaddr)                   \
+    QCryptoTLSTestCertReq varname = {                                   \
+        .filename = fname,                                              \
+        .cn = hostname ? hostname : ipaddr,                             \
+        .altname1 = hostname,                                           \
+        .ipaddr1 = ipaddr,                                              \
+        .basicConstraintsEnable = true,                                 \
+        .basicConstraintsCritical = true,                               \
+        .basicConstraintsIsCA = false,                                  \
+        .keyUsageEnable = true,                                         \
+        .keyUsageCritical = true,                                       \
+        .keyUsageValue =                                                \
+        GNUTLS_KEY_DIGITAL_SIGNATURE | GNUTLS_KEY_KEY_ENCIPHERMENT,     \
+        .keyPurposeEnable = true,                                       \
+        .keyPurposeCritical = true,                                     \
+        .keyPurposeOID1 = GNUTLS_KP_TLS_WWW_SERVER,                     \
+    };                                                                  \
+    test_tls_generate_cert(&varname, cavarname.crt)
+
 extern const asn1_static_node pkix_asn1_tab[];
 
 #endif

From 58d25e97f38214c04d5ba55950cc6d9fff574236 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Tue, 26 Apr 2022 17:00:42 +0100
Subject: [PATCH 379/935] tests: add migration tests of TLS with PSK
 credentials
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This validates that we correctly handle migration success and failure
scenarios when using TLS with pre shared keys.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220426160048.812266-4-berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 tests/qtest/meson.build             |   7 +-
 tests/qtest/migration-test.c        | 161 +++++++++++++++++++++++++++-
 tests/unit/crypto-tls-psk-helpers.c |  18 +++-
 tests/unit/crypto-tls-psk-helpers.h |   1 +
 4 files changed, 179 insertions(+), 8 deletions(-)

diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
index 3551b9c946..166450135d 100644
--- a/tests/qtest/meson.build
+++ b/tests/qtest/meson.build
@@ -273,13 +273,18 @@ endif
 
 tpmemu_files = ['tpm-emu.c', 'tpm-util.c', 'tpm-tests.c']
 
+migration_files = [files('migration-helpers.c')]
+if gnutls.found()
+  migration_files += [files('../unit/crypto-tls-psk-helpers.c'), gnutls]
+endif
+
 qtests = {
   'bios-tables-test': [io, 'boot-sector.c', 'acpi-utils.c', 'tpm-emu.c'],
   'cdrom-test': files('boot-sector.c'),
   'dbus-vmstate-test': files('migration-helpers.c') + dbus_vmstate1,
   'erst-test': files('erst-test.c'),
   'ivshmem-test': [rt, '../../contrib/ivshmem-server/ivshmem-server.c'],
-  'migration-test': files('migration-helpers.c'),
+  'migration-test': migration_files,
   'pxe-test': files('boot-sector.c'),
   'qos-test': [chardev, io, qos_test_ss.apply(config_host, strict: false).sources()],
   'tpm-crb-swtpm-test': [io, tpmemu_files],
diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index cba6023eb5..2eefc9c1ff 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -23,9 +23,13 @@
 #include "qapi/qapi-visit-sockets.h"
 #include "qapi/qobject-input-visitor.h"
 #include "qapi/qobject-output-visitor.h"
+#include "crypto/tlscredspsk.h"
 
 #include "migration-helpers.h"
 #include "tests/migration/migration-test.h"
+#ifdef CONFIG_GNUTLS
+# include "tests/unit/crypto-tls-psk-helpers.h"
+#endif /* CONFIG_GNUTLS */
 
 /* For dirty ring test; so far only x86_64 is supported */
 #if defined(__linux__) && defined(HOST_X86_64)
@@ -640,6 +644,100 @@ static void test_migrate_end(QTestState *from, QTestState *to, bool test_dest)
     cleanup("dest_serial");
 }
 
+#ifdef CONFIG_GNUTLS
+struct TestMigrateTLSPSKData {
+    char *workdir;
+    char *workdiralt;
+    char *pskfile;
+    char *pskfilealt;
+};
+
+static void *
+test_migrate_tls_psk_start_common(QTestState *from,
+                                  QTestState *to,
+                                  bool mismatch)
+{
+    struct TestMigrateTLSPSKData *data =
+        g_new0(struct TestMigrateTLSPSKData, 1);
+    QDict *rsp;
+
+    data->workdir = g_strdup_printf("%s/tlscredspsk0", tmpfs);
+    data->pskfile = g_strdup_printf("%s/%s", data->workdir,
+                                    QCRYPTO_TLS_CREDS_PSKFILE);
+    mkdir(data->workdir, 0700);
+    test_tls_psk_init(data->pskfile);
+
+    if (mismatch) {
+        data->workdiralt = g_strdup_printf("%s/tlscredspskalt0", tmpfs);
+        data->pskfilealt = g_strdup_printf("%s/%s", data->workdiralt,
+                                           QCRYPTO_TLS_CREDS_PSKFILE);
+        mkdir(data->workdiralt, 0700);
+        test_tls_psk_init_alt(data->pskfilealt);
+    }
+
+    rsp = wait_command(from,
+                       "{ 'execute': 'object-add',"
+                       "  'arguments': { 'qom-type': 'tls-creds-psk',"
+                       "                 'id': 'tlscredspsk0',"
+                       "                 'endpoint': 'client',"
+                       "                 'dir': %s,"
+                       "                 'username': 'qemu'} }",
+                       data->workdir);
+    qobject_unref(rsp);
+
+    rsp = wait_command(to,
+                       "{ 'execute': 'object-add',"
+                       "  'arguments': { 'qom-type': 'tls-creds-psk',"
+                       "                 'id': 'tlscredspsk0',"
+                       "                 'endpoint': 'server',"
+                       "                 'dir': %s } }",
+                       mismatch ? data->workdiralt : data->workdir);
+    qobject_unref(rsp);
+
+    migrate_set_parameter_str(from, "tls-creds", "tlscredspsk0");
+    migrate_set_parameter_str(to, "tls-creds", "tlscredspsk0");
+
+    return data;
+}
+
+static void *
+test_migrate_tls_psk_start_match(QTestState *from,
+                                 QTestState *to)
+{
+    return test_migrate_tls_psk_start_common(from, to, false);
+}
+
+static void *
+test_migrate_tls_psk_start_mismatch(QTestState *from,
+                                    QTestState *to)
+{
+    return test_migrate_tls_psk_start_common(from, to, true);
+}
+
+static void
+test_migrate_tls_psk_finish(QTestState *from,
+                            QTestState *to,
+                            void *opaque)
+{
+    struct TestMigrateTLSPSKData *data = opaque;
+
+    test_tls_psk_cleanup(data->pskfile);
+    if (data->pskfilealt) {
+        test_tls_psk_cleanup(data->pskfilealt);
+    }
+    rmdir(data->workdir);
+    if (data->workdiralt) {
+        rmdir(data->workdiralt);
+    }
+
+    g_free(data->workdiralt);
+    g_free(data->pskfilealt);
+    g_free(data->workdir);
+    g_free(data->pskfile);
+    g_free(data);
+}
+#endif /* CONFIG_GNUTLS */
+
 static int migrate_postcopy_prepare(QTestState **from_ptr,
                                     QTestState **to_ptr,
                                     MigrateStart *args)
@@ -911,7 +1009,7 @@ static void test_precopy_common(MigrateCommon *args)
     test_migrate_end(from, to, args->result == MIG_TEST_SUCCEED);
 }
 
-static void test_precopy_unix(void)
+static void test_precopy_unix_plain(void)
 {
     g_autofree char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
     MigrateCommon args = {
@@ -922,6 +1020,21 @@ static void test_precopy_unix(void)
     test_precopy_common(&args);
 }
 
+#ifdef CONFIG_GNUTLS
+static void test_precopy_unix_tls_psk(void)
+{
+    g_autofree char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
+    MigrateCommon args = {
+        .connect_uri = uri,
+        .listen_uri = uri,
+        .start_hook = test_migrate_tls_psk_start_match,
+        .finish_hook = test_migrate_tls_psk_finish,
+    };
+
+    test_precopy_common(&args);
+}
+#endif
+
 static void test_precopy_unix_dirty_ring(void)
 {
     g_autofree char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
@@ -1026,7 +1139,7 @@ static void test_xbzrle_unix(void)
     test_xbzrle(uri);
 }
 
-static void test_precopy_tcp(void)
+static void test_precopy_tcp_plain(void)
 {
     MigrateCommon args = {
         .listen_uri = "tcp:127.0.0.1:0",
@@ -1035,6 +1148,34 @@ static void test_precopy_tcp(void)
     test_precopy_common(&args);
 }
 
+#ifdef CONFIG_GNUTLS
+static void test_precopy_tcp_tls_psk_match(void)
+{
+    MigrateCommon args = {
+        .listen_uri = "tcp:127.0.0.1:0",
+        .start_hook = test_migrate_tls_psk_start_match,
+        .finish_hook = test_migrate_tls_psk_finish,
+    };
+
+    test_precopy_common(&args);
+}
+
+static void test_precopy_tcp_tls_psk_mismatch(void)
+{
+    MigrateCommon args = {
+        .start = {
+            .hide_stderr = true,
+        },
+        .listen_uri = "tcp:127.0.0.1:0",
+        .start_hook = test_migrate_tls_psk_start_mismatch,
+        .finish_hook = test_migrate_tls_psk_finish,
+        .result = MIG_TEST_FAIL,
+    };
+
+    test_precopy_common(&args);
+}
+#endif /* CONFIG_GNUTLS */
+
 static void *test_migrate_fd_start_hook(QTestState *from,
                                         QTestState *to)
 {
@@ -1497,8 +1638,20 @@ int main(int argc, char **argv)
     qtest_add_func("/migration/postcopy/unix", test_postcopy);
     qtest_add_func("/migration/postcopy/recovery", test_postcopy_recovery);
     qtest_add_func("/migration/bad_dest", test_baddest);
-    qtest_add_func("/migration/precopy/unix", test_precopy_unix);
-    qtest_add_func("/migration/precopy/tcp", test_precopy_tcp);
+    qtest_add_func("/migration/precopy/unix/plain", test_precopy_unix_plain);
+#ifdef CONFIG_GNUTLS
+    qtest_add_func("/migration/precopy/unix/tls/psk",
+                   test_precopy_unix_tls_psk);
+#endif /* CONFIG_GNUTLS */
+
+    qtest_add_func("/migration/precopy/tcp/plain", test_precopy_tcp_plain);
+#ifdef CONFIG_GNUTLS
+    qtest_add_func("/migration/precopy/tcp/tls/psk/match",
+                   test_precopy_tcp_tls_psk_match);
+    qtest_add_func("/migration/precopy/tcp/tls/psk/mismatch",
+                   test_precopy_tcp_tls_psk_mismatch);
+#endif /* CONFIG_GNUTLS */
+
     /* qtest_add_func("/migration/ignore_shared", test_ignore_shared); */
     qtest_add_func("/migration/xbzrle/unix", test_xbzrle_unix);
     qtest_add_func("/migration/fd_proto", test_migrate_fd_proto);
diff --git a/tests/unit/crypto-tls-psk-helpers.c b/tests/unit/crypto-tls-psk-helpers.c
index 4bea7c6fa2..511e08cc9c 100644
--- a/tests/unit/crypto-tls-psk-helpers.c
+++ b/tests/unit/crypto-tls-psk-helpers.c
@@ -24,7 +24,8 @@
 #include "crypto-tls-psk-helpers.h"
 #include "qemu/sockets.h"
 
-void test_tls_psk_init(const char *pskfile)
+static void
+test_tls_psk_init_common(const char *pskfile, const char *user, const char *key)
 {
     FILE *fp;
 
@@ -33,11 +34,22 @@ void test_tls_psk_init(const char *pskfile)
         g_critical("Failed to create pskfile %s: %s", pskfile, strerror(errno));
         abort();
     }
-    /* Don't do this in real applications!  Use psktool. */
-    fprintf(fp, "qemu:009d5638c40fde0c\n");
+    fprintf(fp, "%s:%s\n", user, key);
     fclose(fp);
 }
 
+void test_tls_psk_init(const char *pskfile)
+{
+    /* Don't hard code a key like this in real applications!  Use psktool. */
+    test_tls_psk_init_common(pskfile, "qemu", "009d5638c40fde0c");
+}
+
+void test_tls_psk_init_alt(const char *pskfile)
+{
+    /* Don't hard code a key like this in real applications!  Use psktool. */
+    test_tls_psk_init_common(pskfile, "qemu", "10ffa6a2c42f0388");
+}
+
 void test_tls_psk_cleanup(const char *pskfile)
 {
     unlink(pskfile);
diff --git a/tests/unit/crypto-tls-psk-helpers.h b/tests/unit/crypto-tls-psk-helpers.h
index faa645c629..67f8bdda71 100644
--- a/tests/unit/crypto-tls-psk-helpers.h
+++ b/tests/unit/crypto-tls-psk-helpers.h
@@ -24,6 +24,7 @@
 #include <gnutls/gnutls.h>
 
 void test_tls_psk_init(const char *keyfile);
+void test_tls_psk_init_alt(const char *keyfile);
 void test_tls_psk_cleanup(const char *keyfile);
 
 #endif

From d47b83b118eed26e10ff4693fb1f0a433a38d0a0 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Tue, 26 Apr 2022 17:00:43 +0100
Subject: [PATCH 380/935] tests: add migration tests of TLS with x509
 credentials
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This validates that we correctly handle migration success and failure
scenarios when using TLS with x509 certificates. There are quite a few
different scenarios that matter in relation to hostname validation.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220426160048.812266-5-berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  dgilbert: Manual merge due to ifdef change in 3
---
 meson.build                  |   1 +
 tests/qtest/meson.build      |   5 +
 tests/qtest/migration-test.c | 405 +++++++++++++++++++++++++++++++++--
 3 files changed, 397 insertions(+), 14 deletions(-)

diff --git a/meson.build b/meson.build
index 9b20dcd143..93aa31a9e4 100644
--- a/meson.build
+++ b/meson.build
@@ -1742,6 +1742,7 @@ config_host_data.set('CONFIG_KEYUTILS', keyutils.found())
 config_host_data.set('CONFIG_GETTID', has_gettid)
 config_host_data.set('CONFIG_GNUTLS', gnutls.found())
 config_host_data.set('CONFIG_GNUTLS_CRYPTO', gnutls_crypto.found())
+config_host_data.set('CONFIG_TASN1', tasn1.found())
 config_host_data.set('CONFIG_GCRYPT', gcrypt.found())
 config_host_data.set('CONFIG_NETTLE', nettle.found())
 config_host_data.set('CONFIG_QEMU_PRIVATE_XTS', xts == 'private')
diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
index 166450135d..b425484920 100644
--- a/tests/qtest/meson.build
+++ b/tests/qtest/meson.build
@@ -276,6 +276,11 @@ tpmemu_files = ['tpm-emu.c', 'tpm-util.c', 'tpm-tests.c']
 migration_files = [files('migration-helpers.c')]
 if gnutls.found()
   migration_files += [files('../unit/crypto-tls-psk-helpers.c'), gnutls]
+
+  if tasn1.found()
+    migration_files += [files('../unit/crypto-tls-x509-helpers.c',
+                              '../unit/pkix_asn1_tab.c'), tasn1]
+  endif
 endif
 
 qtests = {
diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index 2eefc9c1ff..5a3edf2da6 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -29,6 +29,9 @@
 #include "tests/migration/migration-test.h"
 #ifdef CONFIG_GNUTLS
 # include "tests/unit/crypto-tls-psk-helpers.h"
+# ifdef CONFIG_TASN1
+#  include "tests/unit/crypto-tls-x509-helpers.h"
+# endif /* CONFIG_TASN1 */
 #endif /* CONFIG_GNUTLS */
 
 /* For dirty ring test; so far only x86_64 is supported */
@@ -736,6 +739,234 @@ test_migrate_tls_psk_finish(QTestState *from,
     g_free(data->pskfile);
     g_free(data);
 }
+
+#ifdef CONFIG_TASN1
+typedef struct {
+    char *workdir;
+    char *keyfile;
+    char *cacert;
+    char *servercert;
+    char *serverkey;
+    char *clientcert;
+    char *clientkey;
+} TestMigrateTLSX509Data;
+
+typedef struct {
+    bool verifyclient;
+    bool clientcert;
+    bool hostileclient;
+    bool authzclient;
+    const char *certhostname;
+    const char *certipaddr;
+} TestMigrateTLSX509;
+
+static void *
+test_migrate_tls_x509_start_common(QTestState *from,
+                                   QTestState *to,
+                                   TestMigrateTLSX509 *args)
+{
+    TestMigrateTLSX509Data *data = g_new0(TestMigrateTLSX509Data, 1);
+    QDict *rsp;
+
+    data->workdir = g_strdup_printf("%s/tlscredsx5090", tmpfs);
+    data->keyfile = g_strdup_printf("%s/key.pem", data->workdir);
+
+    data->cacert = g_strdup_printf("%s/ca-cert.pem", data->workdir);
+    data->serverkey = g_strdup_printf("%s/server-key.pem", data->workdir);
+    data->servercert = g_strdup_printf("%s/server-cert.pem", data->workdir);
+    if (args->clientcert) {
+        data->clientkey = g_strdup_printf("%s/client-key.pem", data->workdir);
+        data->clientcert = g_strdup_printf("%s/client-cert.pem", data->workdir);
+    }
+
+    mkdir(data->workdir, 0700);
+
+    test_tls_init(data->keyfile);
+    g_assert(link(data->keyfile, data->serverkey) == 0);
+    if (args->clientcert) {
+        g_assert(link(data->keyfile, data->clientkey) == 0);
+    }
+
+    TLS_ROOT_REQ_SIMPLE(cacertreq, data->cacert);
+    if (args->clientcert) {
+        TLS_CERT_REQ_SIMPLE_CLIENT(servercertreq, cacertreq,
+                                   args->hostileclient ?
+                                   QCRYPTO_TLS_TEST_CLIENT_HOSTILE_NAME :
+                                   QCRYPTO_TLS_TEST_CLIENT_NAME,
+                                   data->clientcert);
+    }
+
+    TLS_CERT_REQ_SIMPLE_SERVER(clientcertreq, cacertreq,
+                               data->servercert,
+                               args->certhostname,
+                               args->certipaddr);
+
+    rsp = wait_command(from,
+                       "{ 'execute': 'object-add',"
+                       "  'arguments': { 'qom-type': 'tls-creds-x509',"
+                       "                 'id': 'tlscredsx509client0',"
+                       "                 'endpoint': 'client',"
+                       "                 'dir': %s,"
+                       "                 'sanity-check': true,"
+                       "                 'verify-peer': true} }",
+                       data->workdir);
+    qobject_unref(rsp);
+    migrate_set_parameter_str(from, "tls-creds", "tlscredsx509client0");
+    if (args->certhostname) {
+        migrate_set_parameter_str(from, "tls-hostname", args->certhostname);
+    }
+
+    rsp = wait_command(to,
+                       "{ 'execute': 'object-add',"
+                       "  'arguments': { 'qom-type': 'tls-creds-x509',"
+                       "                 'id': 'tlscredsx509server0',"
+                       "                 'endpoint': 'server',"
+                       "                 'dir': %s,"
+                       "                 'sanity-check': true,"
+                       "                 'verify-peer': %i} }",
+                       data->workdir, args->verifyclient);
+    qobject_unref(rsp);
+    migrate_set_parameter_str(to, "tls-creds", "tlscredsx509server0");
+
+    if (args->authzclient) {
+        rsp = wait_command(to,
+                           "{ 'execute': 'object-add',"
+                           "  'arguments': { 'qom-type': 'authz-simple',"
+                           "                 'id': 'tlsauthz0',"
+                           "                 'identity': %s} }",
+                           "CN=" QCRYPTO_TLS_TEST_CLIENT_NAME);
+        migrate_set_parameter_str(to, "tls-authz", "tlsauthz0");
+    }
+
+    return data;
+}
+
+/*
+ * The normal case: match server's cert hostname against
+ * whatever host we were telling QEMU to connect to (if any)
+ */
+static void *
+test_migrate_tls_x509_start_default_host(QTestState *from,
+                                         QTestState *to)
+{
+    TestMigrateTLSX509 args = {
+        .verifyclient = true,
+        .clientcert = true,
+        .certipaddr = "127.0.0.1"
+    };
+    return test_migrate_tls_x509_start_common(from, to, &args);
+}
+
+/*
+ * The unusual case: the server's cert is different from
+ * the address we're telling QEMU to connect to (if any),
+ * so we must give QEMU an explicit hostname to validate
+ */
+static void *
+test_migrate_tls_x509_start_override_host(QTestState *from,
+                                          QTestState *to)
+{
+    TestMigrateTLSX509 args = {
+        .verifyclient = true,
+        .clientcert = true,
+        .certhostname = "qemu.org",
+    };
+    return test_migrate_tls_x509_start_common(from, to, &args);
+}
+
+/*
+ * The unusual case: the server's cert is different from
+ * the address we're telling QEMU to connect to, and so we
+ * expect the client to reject the server
+ */
+static void *
+test_migrate_tls_x509_start_mismatch_host(QTestState *from,
+                                          QTestState *to)
+{
+    TestMigrateTLSX509 args = {
+        .verifyclient = true,
+        .clientcert = true,
+        .certipaddr = "10.0.0.1",
+    };
+    return test_migrate_tls_x509_start_common(from, to, &args);
+}
+
+static void *
+test_migrate_tls_x509_start_friendly_client(QTestState *from,
+                                            QTestState *to)
+{
+    TestMigrateTLSX509 args = {
+        .verifyclient = true,
+        .clientcert = true,
+        .authzclient = true,
+        .certipaddr = "127.0.0.1",
+    };
+    return test_migrate_tls_x509_start_common(from, to, &args);
+}
+
+static void *
+test_migrate_tls_x509_start_hostile_client(QTestState *from,
+                                           QTestState *to)
+{
+    TestMigrateTLSX509 args = {
+        .verifyclient = true,
+        .clientcert = true,
+        .hostileclient = true,
+        .authzclient = true,
+        .certipaddr = "127.0.0.1",
+    };
+    return test_migrate_tls_x509_start_common(from, to, &args);
+}
+
+/*
+ * The case with no client certificate presented,
+ * and no server verification
+ */
+static void *
+test_migrate_tls_x509_start_allow_anon_client(QTestState *from,
+                                              QTestState *to)
+{
+    TestMigrateTLSX509 args = {
+        .certipaddr = "127.0.0.1",
+    };
+    return test_migrate_tls_x509_start_common(from, to, &args);
+}
+
+/*
+ * The case with no client certificate presented,
+ * and server verification rejecting
+ */
+static void *
+test_migrate_tls_x509_start_reject_anon_client(QTestState *from,
+                                               QTestState *to)
+{
+    TestMigrateTLSX509 args = {
+        .verifyclient = true,
+        .certipaddr = "127.0.0.1",
+    };
+    return test_migrate_tls_x509_start_common(from, to, &args);
+}
+
+static void
+test_migrate_tls_x509_finish(QTestState *from,
+                             QTestState *to,
+                             void *opaque)
+{
+    TestMigrateTLSX509Data *data = opaque;
+
+    test_tls_cleanup(data->keyfile);
+    unlink(data->cacert);
+    unlink(data->servercert);
+    unlink(data->serverkey);
+    unlink(data->clientcert);
+    unlink(data->clientkey);
+    rmdir(data->workdir);
+
+    g_free(data->workdir);
+    g_free(data->keyfile);
+    g_free(data);
+}
+#endif /* CONFIG_TASN1 */
 #endif /* CONFIG_GNUTLS */
 
 static int migrate_postcopy_prepare(QTestState **from_ptr,
@@ -1020,20 +1251,6 @@ static void test_precopy_unix_plain(void)
     test_precopy_common(&args);
 }
 
-#ifdef CONFIG_GNUTLS
-static void test_precopy_unix_tls_psk(void)
-{
-    g_autofree char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
-    MigrateCommon args = {
-        .connect_uri = uri,
-        .listen_uri = uri,
-        .start_hook = test_migrate_tls_psk_start_match,
-        .finish_hook = test_migrate_tls_psk_finish,
-    };
-
-    test_precopy_common(&args);
-}
-#endif
 
 static void test_precopy_unix_dirty_ring(void)
 {
@@ -1049,6 +1266,53 @@ static void test_precopy_unix_dirty_ring(void)
     test_precopy_common(&args);
 }
 
+#ifdef CONFIG_GNUTLS
+static void test_precopy_unix_tls_psk(void)
+{
+    g_autofree char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
+    MigrateCommon args = {
+        .connect_uri = uri,
+        .listen_uri = uri,
+        .start_hook = test_migrate_tls_psk_start_match,
+        .finish_hook = test_migrate_tls_psk_finish,
+    };
+
+    test_precopy_common(&args);
+}
+
+#ifdef CONFIG_TASN1
+static void test_precopy_unix_tls_x509_default_host(void)
+{
+    g_autofree char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
+    MigrateCommon args = {
+        .start = {
+            .hide_stderr = true,
+        },
+        .connect_uri = uri,
+        .listen_uri = uri,
+        .start_hook = test_migrate_tls_x509_start_default_host,
+        .finish_hook = test_migrate_tls_x509_finish,
+        .result = MIG_TEST_FAIL_DEST_QUIT_ERR,
+    };
+
+    test_precopy_common(&args);
+}
+
+static void test_precopy_unix_tls_x509_override_host(void)
+{
+    g_autofree char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
+    MigrateCommon args = {
+        .connect_uri = uri,
+        .listen_uri = uri,
+        .start_hook = test_migrate_tls_x509_start_override_host,
+        .finish_hook = test_migrate_tls_x509_finish,
+    };
+
+    test_precopy_common(&args);
+}
+#endif /* CONFIG_TASN1 */
+#endif /* CONFIG_GNUTLS */
+
 #if 0
 /* Currently upset on aarch64 TCG */
 static void test_ignore_shared(void)
@@ -1174,6 +1438,97 @@ static void test_precopy_tcp_tls_psk_mismatch(void)
 
     test_precopy_common(&args);
 }
+
+#ifdef CONFIG_TASN1
+static void test_precopy_tcp_tls_x509_default_host(void)
+{
+    MigrateCommon args = {
+        .listen_uri = "tcp:127.0.0.1:0",
+        .start_hook = test_migrate_tls_x509_start_default_host,
+        .finish_hook = test_migrate_tls_x509_finish,
+    };
+
+    test_precopy_common(&args);
+}
+
+static void test_precopy_tcp_tls_x509_override_host(void)
+{
+    MigrateCommon args = {
+        .listen_uri = "tcp:127.0.0.1:0",
+        .start_hook = test_migrate_tls_x509_start_override_host,
+        .finish_hook = test_migrate_tls_x509_finish,
+    };
+
+    test_precopy_common(&args);
+}
+
+static void test_precopy_tcp_tls_x509_mismatch_host(void)
+{
+    MigrateCommon args = {
+        .start = {
+            .hide_stderr = true,
+        },
+        .listen_uri = "tcp:127.0.0.1:0",
+        .start_hook = test_migrate_tls_x509_start_mismatch_host,
+        .finish_hook = test_migrate_tls_x509_finish,
+        .result = MIG_TEST_FAIL_DEST_QUIT_ERR,
+    };
+
+    test_precopy_common(&args);
+}
+
+static void test_precopy_tcp_tls_x509_friendly_client(void)
+{
+    MigrateCommon args = {
+        .listen_uri = "tcp:127.0.0.1:0",
+        .start_hook = test_migrate_tls_x509_start_friendly_client,
+        .finish_hook = test_migrate_tls_x509_finish,
+    };
+
+    test_precopy_common(&args);
+}
+
+static void test_precopy_tcp_tls_x509_hostile_client(void)
+{
+    MigrateCommon args = {
+        .start = {
+            .hide_stderr = true,
+        },
+        .listen_uri = "tcp:127.0.0.1:0",
+        .start_hook = test_migrate_tls_x509_start_hostile_client,
+        .finish_hook = test_migrate_tls_x509_finish,
+        .result = MIG_TEST_FAIL,
+    };
+
+    test_precopy_common(&args);
+}
+
+static void test_precopy_tcp_tls_x509_allow_anon_client(void)
+{
+    MigrateCommon args = {
+        .listen_uri = "tcp:127.0.0.1:0",
+        .start_hook = test_migrate_tls_x509_start_allow_anon_client,
+        .finish_hook = test_migrate_tls_x509_finish,
+    };
+
+    test_precopy_common(&args);
+}
+
+static void test_precopy_tcp_tls_x509_reject_anon_client(void)
+{
+    MigrateCommon args = {
+        .start = {
+            .hide_stderr = true,
+        },
+        .listen_uri = "tcp:127.0.0.1:0",
+        .start_hook = test_migrate_tls_x509_start_reject_anon_client,
+        .finish_hook = test_migrate_tls_x509_finish,
+        .result = MIG_TEST_FAIL,
+    };
+
+    test_precopy_common(&args);
+}
+#endif /* CONFIG_TASN1 */
 #endif /* CONFIG_GNUTLS */
 
 static void *test_migrate_fd_start_hook(QTestState *from,
@@ -1642,6 +1997,12 @@ int main(int argc, char **argv)
 #ifdef CONFIG_GNUTLS
     qtest_add_func("/migration/precopy/unix/tls/psk",
                    test_precopy_unix_tls_psk);
+#ifdef CONFIG_TASN1
+    qtest_add_func("/migration/precopy/unix/tls/x509/default-host",
+                   test_precopy_unix_tls_x509_default_host);
+    qtest_add_func("/migration/precopy/unix/tls/x509/override-host",
+                   test_precopy_unix_tls_x509_override_host);
+#endif /* CONFIG_TASN1 */
 #endif /* CONFIG_GNUTLS */
 
     qtest_add_func("/migration/precopy/tcp/plain", test_precopy_tcp_plain);
@@ -1650,6 +2011,22 @@ int main(int argc, char **argv)
                    test_precopy_tcp_tls_psk_match);
     qtest_add_func("/migration/precopy/tcp/tls/psk/mismatch",
                    test_precopy_tcp_tls_psk_mismatch);
+#ifdef CONFIG_TASN1
+    qtest_add_func("/migration/precopy/tcp/tls/x509/default-host",
+                   test_precopy_tcp_tls_x509_default_host);
+    qtest_add_func("/migration/precopy/tcp/tls/x509/override-host",
+                   test_precopy_tcp_tls_x509_override_host);
+    qtest_add_func("/migration/precopy/tcp/tls/x509/mismatch-host",
+                   test_precopy_tcp_tls_x509_mismatch_host);
+    qtest_add_func("/migration/precopy/tcp/tls/x509/friendly-client",
+                   test_precopy_tcp_tls_x509_friendly_client);
+    qtest_add_func("/migration/precopy/tcp/tls/x509/hostile-client",
+                   test_precopy_tcp_tls_x509_hostile_client);
+    qtest_add_func("/migration/precopy/tcp/tls/x509/allow-anon-client",
+                   test_precopy_tcp_tls_x509_allow_anon_client);
+    qtest_add_func("/migration/precopy/tcp/tls/x509/reject-anon-client",
+                   test_precopy_tcp_tls_x509_reject_anon_client);
+#endif /* CONFIG_TASN1 */
 #endif /* CONFIG_GNUTLS */
 
     /* qtest_add_func("/migration/ignore_shared", test_ignore_shared); */

From 83bcba1ec1f706eb3ec0d6fdeee2bc32f95d9b1a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Tue, 26 Apr 2022 17:00:44 +0100
Subject: [PATCH 381/935] tests: convert XBZRLE migration test to use common
 helper
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Most of the XBZRLE migration test logic is common with the rest of the
precopy tests, so it can use the helper with just one small tweak.

Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220426160048.812266-6-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 tests/qtest/migration-test.c | 67 ++++++++++++++----------------------
 1 file changed, 25 insertions(+), 42 deletions(-)

diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index 5a3edf2da6..a1dc08a93e 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -1174,6 +1174,9 @@ typedef struct {
         /* This test should fail, dest qemu should fail with abnormal status */
         MIG_TEST_FAIL_DEST_QUIT_ERR,
     } result;
+
+    /* Optional: set number of migration passes to wait for */
+    unsigned int iterations;
 } MigrateCommon;
 
 static void test_precopy_common(MigrateCommon *args)
@@ -1219,7 +1222,13 @@ static void test_precopy_common(MigrateCommon *args)
             qtest_set_expected_status(to, 1);
         }
     } else {
-        wait_for_migration_pass(from);
+        if (args->iterations) {
+            while (args->iterations--) {
+                wait_for_migration_pass(from);
+            }
+        } else {
+            wait_for_migration_pass(from);
+        }
 
         migrate_set_parameter_int(from, "downtime-limit", CONVERGE_DOWNTIME);
 
@@ -1350,57 +1359,31 @@ static void test_ignore_shared(void)
 }
 #endif
 
-static void test_xbzrle(const char *uri)
+static void *
+test_migrate_xbzrle_start(QTestState *from,
+                          QTestState *to)
 {
-    MigrateStart args = {};
-    QTestState *from, *to;
-
-    if (test_migrate_start(&from, &to, uri, &args)) {
-        return;
-    }
-
-    /*
-     * We want to pick a speed slow enough that the test completes
-     * quickly, but that it doesn't complete precopy even on a slow
-     * machine, so also set the downtime.
-     */
-    /* 1 ms should make it not converge*/
-    migrate_set_parameter_int(from, "downtime-limit", 1);
-    /* 1GB/s */
-    migrate_set_parameter_int(from, "max-bandwidth", 1000000000);
-
     migrate_set_parameter_int(from, "xbzrle-cache-size", 33554432);
 
     migrate_set_capability(from, "xbzrle", true);
     migrate_set_capability(to, "xbzrle", true);
-    /* Wait for the first serial output from the source */
-    wait_for_serial("src_serial");
 
-    migrate_qmp(from, uri, "{}");
-
-    wait_for_migration_pass(from);
-    /* Make sure we have 2 passes, so the xbzrle cache gets a workout */
-    wait_for_migration_pass(from);
-
-    /* 1000ms should converge */
-    migrate_set_parameter_int(from, "downtime-limit", 1000);
-
-    if (!got_stop) {
-        qtest_qmp_eventwait(from, "STOP");
-    }
-    qtest_qmp_eventwait(to, "RESUME");
-
-    wait_for_serial("dest_serial");
-    wait_for_migration_complete(from);
-
-    test_migrate_end(from, to, true);
+    return NULL;
 }
 
-static void test_xbzrle_unix(void)
+static void test_precopy_unix_xbzrle(void)
 {
     g_autofree char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
+    MigrateCommon args = {
+        .connect_uri = uri,
+        .listen_uri = uri,
 
-    test_xbzrle(uri);
+        .start_hook = test_migrate_xbzrle_start,
+
+        .iterations = 2,
+    };
+
+    test_precopy_common(&args);
 }
 
 static void test_precopy_tcp_plain(void)
@@ -1994,6 +1977,7 @@ int main(int argc, char **argv)
     qtest_add_func("/migration/postcopy/recovery", test_postcopy_recovery);
     qtest_add_func("/migration/bad_dest", test_baddest);
     qtest_add_func("/migration/precopy/unix/plain", test_precopy_unix_plain);
+    qtest_add_func("/migration/precopy/unix/xbzrle", test_precopy_unix_xbzrle);
 #ifdef CONFIG_GNUTLS
     qtest_add_func("/migration/precopy/unix/tls/psk",
                    test_precopy_unix_tls_psk);
@@ -2030,7 +2014,6 @@ int main(int argc, char **argv)
 #endif /* CONFIG_GNUTLS */
 
     /* qtest_add_func("/migration/ignore_shared", test_ignore_shared); */
-    qtest_add_func("/migration/xbzrle/unix", test_xbzrle_unix);
     qtest_add_func("/migration/fd_proto", test_migrate_fd_proto);
     qtest_add_func("/migration/validate_uuid", test_validate_uuid);
     qtest_add_func("/migration/validate_uuid_error", test_validate_uuid_error);

From 490facffcf21647e7088f4f0c2c5a8f5f45625e3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Tue, 26 Apr 2022 17:00:45 +0100
Subject: [PATCH 382/935] tests: convert multifd migration tests to use common
 helper
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Most of the multifd migration test logic is common with the rest of the
precopy tests, so it can use the helper without difficulty. The only
exception of the multifd cancellation test which tries to run multiple
migrations in a row.

Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220426160048.812266-7-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 tests/qtest/migration-test.c | 83 +++++++++++++++++++-----------------
 1 file changed, 43 insertions(+), 40 deletions(-)

diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index a1dc08a93e..f551c8d030 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -1740,26 +1740,12 @@ static void test_migrate_auto_converge(void)
     test_migrate_end(from, to, true);
 }
 
-static void test_multifd_tcp(const char *method)
+static void *
+test_migrate_precopy_tcp_multifd_start_common(QTestState *from,
+                                              QTestState *to,
+                                              const char *method)
 {
-    MigrateStart args = {};
-    QTestState *from, *to;
     QDict *rsp;
-    g_autofree char *uri = NULL;
-
-    if (test_migrate_start(&from, &to, "defer", &args)) {
-        return;
-    }
-
-    /*
-     * We want to pick a speed slow enough that the test completes
-     * quickly, but that it doesn't complete precopy even on a slow
-     * machine, so also set the downtime.
-     */
-    /* 1 ms should make it not converge*/
-    migrate_set_parameter_int(from, "downtime-limit", 1);
-    /* 1GB/s */
-    migrate_set_parameter_int(from, "max-bandwidth", 1000000000);
 
     migrate_set_parameter_int(from, "multifd-channels", 16);
     migrate_set_parameter_int(to, "multifd-channels", 16);
@@ -1775,41 +1761,58 @@ static void test_multifd_tcp(const char *method)
                            "  'arguments': { 'uri': 'tcp:127.0.0.1:0' }}");
     qobject_unref(rsp);
 
-    /* Wait for the first serial output from the source */
-    wait_for_serial("src_serial");
-
-    uri = migrate_get_socket_address(to, "socket-address");
-
-    migrate_qmp(from, uri, "{}");
-
-    wait_for_migration_pass(from);
-
-    migrate_set_parameter_int(from, "downtime-limit", CONVERGE_DOWNTIME);
-
-    if (!got_stop) {
-        qtest_qmp_eventwait(from, "STOP");
-    }
-    qtest_qmp_eventwait(to, "RESUME");
-
-    wait_for_serial("dest_serial");
-    wait_for_migration_complete(from);
-    test_migrate_end(from, to, true);
+    return NULL;
 }
 
+static void *
+test_migrate_precopy_tcp_multifd_start(QTestState *from,
+                                       QTestState *to)
+{
+    return test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
+}
+
+static void *
+test_migrate_precopy_tcp_multifd_zlib_start(QTestState *from,
+                                            QTestState *to)
+{
+    return test_migrate_precopy_tcp_multifd_start_common(from, to, "zlib");
+}
+
+#ifdef CONFIG_ZSTD
+static void *
+test_migrate_precopy_tcp_multifd_zstd_start(QTestState *from,
+                                            QTestState *to)
+{
+    return test_migrate_precopy_tcp_multifd_start_common(from, to, "zstd");
+}
+#endif /* CONFIG_ZSTD */
+
 static void test_multifd_tcp_none(void)
 {
-    test_multifd_tcp("none");
+    MigrateCommon args = {
+        .listen_uri = "defer",
+        .start_hook = test_migrate_precopy_tcp_multifd_start,
+    };
+    test_precopy_common(&args);
 }
 
 static void test_multifd_tcp_zlib(void)
 {
-    test_multifd_tcp("zlib");
+    MigrateCommon args = {
+        .listen_uri = "defer",
+        .start_hook = test_migrate_precopy_tcp_multifd_zlib_start,
+    };
+    test_precopy_common(&args);
 }
 
 #ifdef CONFIG_ZSTD
 static void test_multifd_tcp_zstd(void)
 {
-    test_multifd_tcp("zstd");
+    MigrateCommon args = {
+        .listen_uri = "defer",
+        .start_hook = test_migrate_precopy_tcp_multifd_zstd_start,
+    };
+    test_precopy_common(&args);
 }
 #endif
 

From 4d6d2e872a9f5aef4b08eb285a1fd7221312d562 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Tue, 26 Apr 2022 17:00:46 +0100
Subject: [PATCH 383/935] tests: add multifd migration tests of TLS with PSK
 credentials
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This validates that we correctly handle multifd migration success
and failure scenarios when using TLS with pre shared keys.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220426160048.812266-8-berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 tests/qtest/migration-test.c | 60 +++++++++++++++++++++++++++++++++---
 1 file changed, 56 insertions(+), 4 deletions(-)

diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index f551c8d030..133665b500 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -1816,6 +1816,48 @@ static void test_multifd_tcp_zstd(void)
 }
 #endif
 
+#ifdef CONFIG_GNUTLS
+static void *
+test_migrate_multifd_tcp_tls_psk_start_match(QTestState *from,
+                                             QTestState *to)
+{
+    test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
+    return test_migrate_tls_psk_start_match(from, to);
+}
+
+static void *
+test_migrate_multifd_tcp_tls_psk_start_mismatch(QTestState *from,
+                                                QTestState *to)
+{
+    test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
+    return test_migrate_tls_psk_start_mismatch(from, to);
+}
+
+static void test_multifd_tcp_tls_psk_match(void)
+{
+    MigrateCommon args = {
+        .listen_uri = "defer",
+        .start_hook = test_migrate_multifd_tcp_tls_psk_start_match,
+        .finish_hook = test_migrate_tls_psk_finish,
+    };
+    test_precopy_common(&args);
+}
+
+static void test_multifd_tcp_tls_psk_mismatch(void)
+{
+    MigrateCommon args = {
+        .start = {
+            .hide_stderr = true,
+        },
+        .listen_uri = "defer",
+        .start_hook = test_migrate_multifd_tcp_tls_psk_start_mismatch,
+        .finish_hook = test_migrate_tls_psk_finish,
+        .result = MIG_TEST_FAIL,
+    };
+    test_precopy_common(&args);
+}
+#endif /* CONFIG_GNUTLS */
+
 /*
  * This test does:
  *  source               target
@@ -2026,12 +2068,22 @@ int main(int argc, char **argv)
                    test_validate_uuid_dst_not_set);
 
     qtest_add_func("/migration/auto_converge", test_migrate_auto_converge);
-    qtest_add_func("/migration/multifd/tcp/none", test_multifd_tcp_none);
-    qtest_add_func("/migration/multifd/tcp/cancel", test_multifd_tcp_cancel);
-    qtest_add_func("/migration/multifd/tcp/zlib", test_multifd_tcp_zlib);
+    qtest_add_func("/migration/multifd/tcp/plain/none",
+                   test_multifd_tcp_none);
+    qtest_add_func("/migration/multifd/tcp/plain/cancel",
+                   test_multifd_tcp_cancel);
+    qtest_add_func("/migration/multifd/tcp/plain/zlib",
+                   test_multifd_tcp_zlib);
 #ifdef CONFIG_ZSTD
-    qtest_add_func("/migration/multifd/tcp/zstd", test_multifd_tcp_zstd);
+    qtest_add_func("/migration/multifd/tcp/plain/zstd",
+                   test_multifd_tcp_zstd);
 #endif
+#ifdef CONFIG_GNUTLS
+    qtest_add_func("/migration/multifd/tcp/tls/psk/match",
+                   test_multifd_tcp_tls_psk_match);
+    qtest_add_func("/migration/multifd/tcp/tls/psk/mismatch",
+                   test_multifd_tcp_tls_psk_mismatch);
+#endif /* CONFIG_GNUTLS */
 
     if (kvm_dirty_ring_supported()) {
         qtest_add_func("/migration/dirty_ring",

From ff32f1dd3281d77b86d195dd62c6ec66a881b6fe Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Tue, 26 Apr 2022 17:00:47 +0100
Subject: [PATCH 384/935] tests: add multifd migration tests of TLS with x509
 credentials
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This validates that we correctly handle multifd migration success
and failure scenarios when using TLS with x509 certificates. There
are quite a few different scenarios that matter in relation to
hostname validation, but we skip a couple as we can assume that
the non-multifd coverage applies to some extent.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220426160048.812266-9-berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 tests/qtest/migration-test.c | 127 +++++++++++++++++++++++++++++++++++
 1 file changed, 127 insertions(+)

diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index 133665b500..efc6ec1614 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -1833,6 +1833,48 @@ test_migrate_multifd_tcp_tls_psk_start_mismatch(QTestState *from,
     return test_migrate_tls_psk_start_mismatch(from, to);
 }
 
+#ifdef CONFIG_TASN1
+static void *
+test_migrate_multifd_tls_x509_start_default_host(QTestState *from,
+                                                 QTestState *to)
+{
+    test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
+    return test_migrate_tls_x509_start_default_host(from, to);
+}
+
+static void *
+test_migrate_multifd_tls_x509_start_override_host(QTestState *from,
+                                                  QTestState *to)
+{
+    test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
+    return test_migrate_tls_x509_start_override_host(from, to);
+}
+
+static void *
+test_migrate_multifd_tls_x509_start_mismatch_host(QTestState *from,
+                                                  QTestState *to)
+{
+    test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
+    return test_migrate_tls_x509_start_mismatch_host(from, to);
+}
+
+static void *
+test_migrate_multifd_tls_x509_start_allow_anon_client(QTestState *from,
+                                                      QTestState *to)
+{
+    test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
+    return test_migrate_tls_x509_start_allow_anon_client(from, to);
+}
+
+static void *
+test_migrate_multifd_tls_x509_start_reject_anon_client(QTestState *from,
+                                                       QTestState *to)
+{
+    test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
+    return test_migrate_tls_x509_start_reject_anon_client(from, to);
+}
+#endif /* CONFIG_TASN1 */
+
 static void test_multifd_tcp_tls_psk_match(void)
 {
     MigrateCommon args = {
@@ -1856,6 +1898,79 @@ static void test_multifd_tcp_tls_psk_mismatch(void)
     };
     test_precopy_common(&args);
 }
+
+#ifdef CONFIG_TASN1
+static void test_multifd_tcp_tls_x509_default_host(void)
+{
+    MigrateCommon args = {
+        .listen_uri = "defer",
+        .start_hook = test_migrate_multifd_tls_x509_start_default_host,
+        .finish_hook = test_migrate_tls_x509_finish,
+    };
+    test_precopy_common(&args);
+}
+
+static void test_multifd_tcp_tls_x509_override_host(void)
+{
+    MigrateCommon args = {
+        .listen_uri = "defer",
+        .start_hook = test_migrate_multifd_tls_x509_start_override_host,
+        .finish_hook = test_migrate_tls_x509_finish,
+    };
+    test_precopy_common(&args);
+}
+
+static void test_multifd_tcp_tls_x509_mismatch_host(void)
+{
+    /*
+     * This has different behaviour to the non-multifd case.
+     *
+     * In non-multifd case when client aborts due to mismatched
+     * cert host, the server has already started trying to load
+     * migration state, and so it exits with I/O failure.
+     *
+     * In multifd case when client aborts due to mismatched
+     * cert host, the server is still waiting for the other
+     * multifd connections to arrive so hasn't started trying
+     * to load migration state, and thus just aborts the migration
+     * without exiting.
+     */
+    MigrateCommon args = {
+        .start = {
+            .hide_stderr = true,
+        },
+        .listen_uri = "defer",
+        .start_hook = test_migrate_multifd_tls_x509_start_mismatch_host,
+        .finish_hook = test_migrate_tls_x509_finish,
+        .result = MIG_TEST_FAIL,
+    };
+    test_precopy_common(&args);
+}
+
+static void test_multifd_tcp_tls_x509_allow_anon_client(void)
+{
+    MigrateCommon args = {
+        .listen_uri = "defer",
+        .start_hook = test_migrate_multifd_tls_x509_start_allow_anon_client,
+        .finish_hook = test_migrate_tls_x509_finish,
+    };
+    test_precopy_common(&args);
+}
+
+static void test_multifd_tcp_tls_x509_reject_anon_client(void)
+{
+    MigrateCommon args = {
+        .start = {
+            .hide_stderr = true,
+        },
+        .listen_uri = "defer",
+        .start_hook = test_migrate_multifd_tls_x509_start_reject_anon_client,
+        .finish_hook = test_migrate_tls_x509_finish,
+        .result = MIG_TEST_FAIL,
+    };
+    test_precopy_common(&args);
+}
+#endif /* CONFIG_TASN1 */
 #endif /* CONFIG_GNUTLS */
 
 /*
@@ -2083,6 +2198,18 @@ int main(int argc, char **argv)
                    test_multifd_tcp_tls_psk_match);
     qtest_add_func("/migration/multifd/tcp/tls/psk/mismatch",
                    test_multifd_tcp_tls_psk_mismatch);
+#ifdef CONFIG_TASN1
+    qtest_add_func("/migration/multifd/tcp/tls/x509/default-host",
+                   test_multifd_tcp_tls_x509_default_host);
+    qtest_add_func("/migration/multifd/tcp/tls/x509/override-host",
+                   test_multifd_tcp_tls_x509_override_host);
+    qtest_add_func("/migration/multifd/tcp/tls/x509/mismatch-host",
+                   test_multifd_tcp_tls_x509_mismatch_host);
+    qtest_add_func("/migration/multifd/tcp/tls/x509/allow-anon-client",
+                   test_multifd_tcp_tls_x509_allow_anon_client);
+    qtest_add_func("/migration/multifd/tcp/tls/x509/reject-anon-client",
+                   test_multifd_tcp_tls_x509_reject_anon_client);
+#endif /* CONFIG_TASN1 */
 #endif /* CONFIG_GNUTLS */
 
     if (kvm_dirty_ring_supported()) {

From fd3540adb96d6f7ee7541be14f972a74c93faf3a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Tue, 26 Apr 2022 17:00:48 +0100
Subject: [PATCH 385/935] tests: ensure migration status isn't reported as
 failed
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Various methods in the migration test call 'query_migrate' to fetch the
current status and then access a particular field. Almost all of these
cases expect the migration to be in a non-failed state. In the case of
'wait_for_migration_pass' in particular, if the status is 'failed' then
it will get into an infinite loop. By validating that the status is
not 'failed' the test suite will assert rather than hang when getting
into an unexpected state.

Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220426160048.812266-10-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 tests/qtest/migration-helpers.c | 13 +++++++++++++
 tests/qtest/migration-helpers.h |  1 +
 tests/qtest/migration-test.c    |  6 +++---
 3 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/tests/qtest/migration-helpers.c b/tests/qtest/migration-helpers.c
index 4ee26014b7..a6aa59e4e6 100644
--- a/tests/qtest/migration-helpers.c
+++ b/tests/qtest/migration-helpers.c
@@ -107,6 +107,19 @@ QDict *migrate_query(QTestState *who)
     return wait_command(who, "{ 'execute': 'query-migrate' }");
 }
 
+QDict *migrate_query_not_failed(QTestState *who)
+{
+    const char *status;
+    QDict *rsp = migrate_query(who);
+    status = qdict_get_str(rsp, "status");
+    if (g_str_equal(status, "failed")) {
+        g_printerr("query-migrate shows failed migration: %s\n",
+                   qdict_get_str(rsp, "error-desc"));
+    }
+    g_assert(!g_str_equal(status, "failed"));
+    return rsp;
+}
+
 /*
  * Note: caller is responsible to free the returned object via
  * g_free() after use
diff --git a/tests/qtest/migration-helpers.h b/tests/qtest/migration-helpers.h
index 2731399aaa..78587c2b82 100644
--- a/tests/qtest/migration-helpers.h
+++ b/tests/qtest/migration-helpers.h
@@ -27,6 +27,7 @@ G_GNUC_PRINTF(3, 4)
 void migrate_qmp(QTestState *who, const char *uri, const char *fmt, ...);
 
 QDict *migrate_query(QTestState *who);
+QDict *migrate_query_not_failed(QTestState *who);
 
 void wait_for_migration_status(QTestState *who,
                                const char *goal, const char **ungoals);
diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index efc6ec1614..d33e8060f9 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -181,7 +181,7 @@ static int64_t read_ram_property_int(QTestState *who, const char *property)
     QDict *rsp_return, *rsp_ram;
     int64_t result;
 
-    rsp_return = migrate_query(who);
+    rsp_return = migrate_query_not_failed(who);
     if (!qdict_haskey(rsp_return, "ram")) {
         /* Still in setup */
         result = 0;
@@ -198,7 +198,7 @@ static int64_t read_migrate_property_int(QTestState *who, const char *property)
     QDict *rsp_return;
     int64_t result;
 
-    rsp_return = migrate_query(who);
+    rsp_return = migrate_query_not_failed(who);
     result = qdict_get_try_int(rsp_return, property, 0);
     qobject_unref(rsp_return);
     return result;
@@ -213,7 +213,7 @@ static void read_blocktime(QTestState *who)
 {
     QDict *rsp_return;
 
-    rsp_return = migrate_query(who);
+    rsp_return = migrate_query_not_failed(who);
     g_assert(qdict_haskey(rsp_return, "postcopy-blocktime"));
     qobject_unref(rsp_return);
 }

From 354081d43de44ebd3497fe08f7f0121a5517d528 Mon Sep 17 00:00:00 2001
From: Leonardo Bras <leobras@redhat.com>
Date: Fri, 13 May 2022 03:28:30 -0300
Subject: [PATCH 386/935] meson.build: Fix docker-test-build@alpine when
 including linux/errqueue.h

A build error happens in alpine CI when linux/errqueue.h is included
in io/channel-socket.c, due to redefining of 'struct __kernel_timespec':

===
ninja: job failed: [...]
In file included from /usr/include/linux/errqueue.h:6,
                 from ../io/channel-socket.c:29:
/usr/include/linux/time_types.h:7:8: error: redefinition of 'struct __kernel_timespec'
    7 | struct __kernel_timespec {
      |        ^~~~~~~~~~~~~~~~~
In file included from /usr/include/liburing.h:19,
                 from /builds/user/qemu/include/block/aio.h:18,
                 from /builds/user/qemu/include/io/channel.h:26,
                 from /builds/user/qemu/include/io/channel-socket.h:24,
                 from ../io/channel-socket.c:24:
/usr/include/liburing/compat.h:9:8: note: originally defined here
    9 | struct __kernel_timespec {
      |        ^~~~~~~~~~~~~~~~~
ninja: subcommand failed
===

As above error message suggests, 'struct __kernel_timespec' was already
defined by liburing/compat.h.

Fix alpine CI by adding test to disable liburing in configure step if a
redefinition happens between linux/errqueue.h and liburing/compat.h.

[dgilbert: This has been fixed in Alpine issue 13813 and liburing]

Signed-off-by: Leonardo Bras <leobras@redhat.com>
Message-Id: <20220513062836.965425-2-leobras@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 meson.build | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/meson.build b/meson.build
index 93aa31a9e4..53a4728250 100644
--- a/meson.build
+++ b/meson.build
@@ -515,12 +515,23 @@ if not get_option('linux_aio').auto() or have_block
                            required: get_option('linux_aio'),
                            kwargs: static_kwargs)
 endif
+
+linux_io_uring_test = '''
+  #include <liburing.h>
+  #include <linux/errqueue.h>
+
+  int main(void) { return 0; }'''
+
 linux_io_uring = not_found
 if not get_option('linux_io_uring').auto() or have_block
   linux_io_uring = dependency('liburing', version: '>=0.3',
                               required: get_option('linux_io_uring'),
                               method: 'pkg-config', kwargs: static_kwargs)
+  if not cc.links(linux_io_uring_test)
+    linux_io_uring = not_found
+  endif
 endif
+
 libnfs = not_found
 if not get_option('libnfs').auto() or have_block
   libnfs = dependency('libnfs', version: '>=1.9.3',

From b88651cb4d4fa416fdbb6afaf5b26ec8c035eaad Mon Sep 17 00:00:00 2001
From: Leonardo Bras <leobras@redhat.com>
Date: Fri, 13 May 2022 03:28:31 -0300
Subject: [PATCH 387/935] QIOChannel: Add flags on io_writev and introduce
 io_flush callback
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add flags to io_writev and introduce io_flush as optional callback to
QIOChannelClass, allowing the implementation of zero copy writes by
subclasses.

How to use them:
- Write data using qio_channel_writev*(...,QIO_CHANNEL_WRITE_FLAG_ZERO_COPY),
- Wait write completion with qio_channel_flush().

Notes:
As some zero copy write implementations work asynchronously, it's
recommended to keep the write buffer untouched until the return of
qio_channel_flush(), to avoid the risk of sending an updated buffer
instead of the buffer state during write.

As io_flush callback is optional, if a subclass does not implement it, then:
- io_flush will return 0 without changing anything.

Also, some functions like qio_channel_writev_full_all() were adapted to
receive a flag parameter. That allows shared code between zero copy and
non-zero copy writev, and also an easier implementation on new flags.

Signed-off-by: Leonardo Bras <leobras@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20220513062836.965425-3-leobras@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 chardev/char-io.c                   |  2 +-
 hw/remote/mpqemu-link.c             |  2 +-
 include/io/channel.h                | 38 +++++++++++++++++++++-
 io/channel-buffer.c                 |  1 +
 io/channel-command.c                |  1 +
 io/channel-file.c                   |  1 +
 io/channel-socket.c                 |  2 ++
 io/channel-tls.c                    |  1 +
 io/channel-websock.c                |  1 +
 io/channel.c                        | 49 +++++++++++++++++++++++------
 migration/rdma.c                    |  1 +
 scsi/pr-manager-helper.c            |  2 +-
 tests/unit/test-io-channel-socket.c |  1 +
 13 files changed, 88 insertions(+), 14 deletions(-)

diff --git a/chardev/char-io.c b/chardev/char-io.c
index 8ced184160..4451128cba 100644
--- a/chardev/char-io.c
+++ b/chardev/char-io.c
@@ -122,7 +122,7 @@ int io_channel_send_full(QIOChannel *ioc,
 
         ret = qio_channel_writev_full(
             ioc, &iov, 1,
-            fds, nfds, NULL);
+            fds, nfds, 0, NULL);
         if (ret == QIO_CHANNEL_ERR_BLOCK) {
             if (offset) {
                 return offset;
diff --git a/hw/remote/mpqemu-link.c b/hw/remote/mpqemu-link.c
index 2a4aa651ca..9bd98e8219 100644
--- a/hw/remote/mpqemu-link.c
+++ b/hw/remote/mpqemu-link.c
@@ -68,7 +68,7 @@ bool mpqemu_msg_send(MPQemuMsg *msg, QIOChannel *ioc, Error **errp)
     }
 
     if (!qio_channel_writev_full_all(ioc, send, G_N_ELEMENTS(send),
-                                    fds, nfds, errp)) {
+                                    fds, nfds, 0, errp)) {
         ret = true;
     } else {
         trace_mpqemu_send_io_error(msg->cmd, msg->size, nfds);
diff --git a/include/io/channel.h b/include/io/channel.h
index 88988979f8..c680ee7480 100644
--- a/include/io/channel.h
+++ b/include/io/channel.h
@@ -32,12 +32,15 @@ OBJECT_DECLARE_TYPE(QIOChannel, QIOChannelClass,
 
 #define QIO_CHANNEL_ERR_BLOCK -2
 
+#define QIO_CHANNEL_WRITE_FLAG_ZERO_COPY 0x1
+
 typedef enum QIOChannelFeature QIOChannelFeature;
 
 enum QIOChannelFeature {
     QIO_CHANNEL_FEATURE_FD_PASS,
     QIO_CHANNEL_FEATURE_SHUTDOWN,
     QIO_CHANNEL_FEATURE_LISTEN,
+    QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY,
 };
 
 
@@ -104,6 +107,7 @@ struct QIOChannelClass {
                          size_t niov,
                          int *fds,
                          size_t nfds,
+                         int flags,
                          Error **errp);
     ssize_t (*io_readv)(QIOChannel *ioc,
                         const struct iovec *iov,
@@ -136,6 +140,8 @@ struct QIOChannelClass {
                                   IOHandler *io_read,
                                   IOHandler *io_write,
                                   void *opaque);
+    int (*io_flush)(QIOChannel *ioc,
+                    Error **errp);
 };
 
 /* General I/O handling functions */
@@ -228,6 +234,7 @@ ssize_t qio_channel_readv_full(QIOChannel *ioc,
  * @niov: the length of the @iov array
  * @fds: an array of file handles to send
  * @nfds: number of file handles in @fds
+ * @flags: write flags (QIO_CHANNEL_WRITE_FLAG_*)
  * @errp: pointer to a NULL-initialized error object
  *
  * Write data to the IO channel, reading it from the
@@ -260,6 +267,7 @@ ssize_t qio_channel_writev_full(QIOChannel *ioc,
                                 size_t niov,
                                 int *fds,
                                 size_t nfds,
+                                int flags,
                                 Error **errp);
 
 /**
@@ -837,6 +845,7 @@ int qio_channel_readv_full_all(QIOChannel *ioc,
  * @niov: the length of the @iov array
  * @fds: an array of file handles to send
  * @nfds: number of file handles in @fds
+ * @flags: write flags (QIO_CHANNEL_WRITE_FLAG_*)
  * @errp: pointer to a NULL-initialized error object
  *
  *
@@ -846,6 +855,14 @@ int qio_channel_readv_full_all(QIOChannel *ioc,
  * to be written, yielding from the current coroutine
  * if required.
  *
+ * If QIO_CHANNEL_WRITE_FLAG_ZERO_COPY is passed in flags,
+ * instead of waiting for all requested data to be written,
+ * this function will wait until it's all queued for writing.
+ * In this case, if the buffer gets changed between queueing and
+ * sending, the updated buffer will be sent. If this is not a
+ * desired behavior, it's suggested to call qio_channel_flush()
+ * before reusing the buffer.
+ *
  * Returns: 0 if all bytes were written, or -1 on error
  */
 
@@ -853,6 +870,25 @@ int qio_channel_writev_full_all(QIOChannel *ioc,
                                 const struct iovec *iov,
                                 size_t niov,
                                 int *fds, size_t nfds,
-                                Error **errp);
+                                int flags, Error **errp);
+
+/**
+ * qio_channel_flush:
+ * @ioc: the channel object
+ * @errp: pointer to a NULL-initialized error object
+ *
+ * Will block until every packet queued with
+ * qio_channel_writev_full() + QIO_CHANNEL_WRITE_FLAG_ZERO_COPY
+ * is sent, or return in case of any error.
+ *
+ * If not implemented, acts as a no-op, and returns 0.
+ *
+ * Returns -1 if any error is found,
+ *          1 if every send failed to use zero copy.
+ *          0 otherwise.
+ */
+
+int qio_channel_flush(QIOChannel *ioc,
+                      Error **errp);
 
 #endif /* QIO_CHANNEL_H */
diff --git a/io/channel-buffer.c b/io/channel-buffer.c
index baa4e2b089..bf52011be2 100644
--- a/io/channel-buffer.c
+++ b/io/channel-buffer.c
@@ -81,6 +81,7 @@ static ssize_t qio_channel_buffer_writev(QIOChannel *ioc,
                                          size_t niov,
                                          int *fds,
                                          size_t nfds,
+                                         int flags,
                                          Error **errp)
 {
     QIOChannelBuffer *bioc = QIO_CHANNEL_BUFFER(ioc);
diff --git a/io/channel-command.c b/io/channel-command.c
index 4a1f969aaa..9f2f4a1793 100644
--- a/io/channel-command.c
+++ b/io/channel-command.c
@@ -276,6 +276,7 @@ static ssize_t qio_channel_command_writev(QIOChannel *ioc,
                                           size_t niov,
                                           int *fds,
                                           size_t nfds,
+                                          int flags,
                                           Error **errp)
 {
     QIOChannelCommand *cioc = QIO_CHANNEL_COMMAND(ioc);
diff --git a/io/channel-file.c b/io/channel-file.c
index d146ace7db..b67687c2aa 100644
--- a/io/channel-file.c
+++ b/io/channel-file.c
@@ -114,6 +114,7 @@ static ssize_t qio_channel_file_writev(QIOChannel *ioc,
                                        size_t niov,
                                        int *fds,
                                        size_t nfds,
+                                       int flags,
                                        Error **errp)
 {
     QIOChannelFile *fioc = QIO_CHANNEL_FILE(ioc);
diff --git a/io/channel-socket.c b/io/channel-socket.c
index e531d7bd2a..05c425abb8 100644
--- a/io/channel-socket.c
+++ b/io/channel-socket.c
@@ -524,6 +524,7 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,
                                          size_t niov,
                                          int *fds,
                                          size_t nfds,
+                                         int flags,
                                          Error **errp)
 {
     QIOChannelSocket *sioc = QIO_CHANNEL_SOCKET(ioc);
@@ -619,6 +620,7 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,
                                          size_t niov,
                                          int *fds,
                                          size_t nfds,
+                                         int flags,
                                          Error **errp)
 {
     QIOChannelSocket *sioc = QIO_CHANNEL_SOCKET(ioc);
diff --git a/io/channel-tls.c b/io/channel-tls.c
index 2ae1b92fc0..4ce890a538 100644
--- a/io/channel-tls.c
+++ b/io/channel-tls.c
@@ -301,6 +301,7 @@ static ssize_t qio_channel_tls_writev(QIOChannel *ioc,
                                       size_t niov,
                                       int *fds,
                                       size_t nfds,
+                                      int flags,
                                       Error **errp)
 {
     QIOChannelTLS *tioc = QIO_CHANNEL_TLS(ioc);
diff --git a/io/channel-websock.c b/io/channel-websock.c
index 55145a6a8c..9619906ac3 100644
--- a/io/channel-websock.c
+++ b/io/channel-websock.c
@@ -1127,6 +1127,7 @@ static ssize_t qio_channel_websock_writev(QIOChannel *ioc,
                                           size_t niov,
                                           int *fds,
                                           size_t nfds,
+                                          int flags,
                                           Error **errp)
 {
     QIOChannelWebsock *wioc = QIO_CHANNEL_WEBSOCK(ioc);
diff --git a/io/channel.c b/io/channel.c
index e8b019dc36..0640941ac5 100644
--- a/io/channel.c
+++ b/io/channel.c
@@ -72,18 +72,32 @@ ssize_t qio_channel_writev_full(QIOChannel *ioc,
                                 size_t niov,
                                 int *fds,
                                 size_t nfds,
+                                int flags,
                                 Error **errp)
 {
     QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc);
 
-    if ((fds || nfds) &&
-        !qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_FD_PASS)) {
+    if (fds || nfds) {
+        if (!qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_FD_PASS)) {
+            error_setg_errno(errp, EINVAL,
+                             "Channel does not support file descriptor passing");
+            return -1;
+        }
+        if (flags & QIO_CHANNEL_WRITE_FLAG_ZERO_COPY) {
+            error_setg_errno(errp, EINVAL,
+                             "Zero Copy does not support file descriptor passing");
+            return -1;
+        }
+    }
+
+    if ((flags & QIO_CHANNEL_WRITE_FLAG_ZERO_COPY) &&
+        !qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY)) {
         error_setg_errno(errp, EINVAL,
-                         "Channel does not support file descriptor passing");
+                         "Requested Zero Copy feature is not available");
         return -1;
     }
 
-    return klass->io_writev(ioc, iov, niov, fds, nfds, errp);
+    return klass->io_writev(ioc, iov, niov, fds, nfds, flags, errp);
 }
 
 
@@ -217,14 +231,14 @@ int qio_channel_writev_all(QIOChannel *ioc,
                            size_t niov,
                            Error **errp)
 {
-    return qio_channel_writev_full_all(ioc, iov, niov, NULL, 0, errp);
+    return qio_channel_writev_full_all(ioc, iov, niov, NULL, 0, 0, errp);
 }
 
 int qio_channel_writev_full_all(QIOChannel *ioc,
                                 const struct iovec *iov,
                                 size_t niov,
                                 int *fds, size_t nfds,
-                                Error **errp)
+                                int flags, Error **errp)
 {
     int ret = -1;
     struct iovec *local_iov = g_new(struct iovec, niov);
@@ -237,8 +251,10 @@ int qio_channel_writev_full_all(QIOChannel *ioc,
 
     while (nlocal_iov > 0) {
         ssize_t len;
-        len = qio_channel_writev_full(ioc, local_iov, nlocal_iov, fds, nfds,
-                                      errp);
+
+        len = qio_channel_writev_full(ioc, local_iov, nlocal_iov, fds,
+                                            nfds, flags, errp);
+
         if (len == QIO_CHANNEL_ERR_BLOCK) {
             if (qemu_in_coroutine()) {
                 qio_channel_yield(ioc, G_IO_OUT);
@@ -277,7 +293,7 @@ ssize_t qio_channel_writev(QIOChannel *ioc,
                            size_t niov,
                            Error **errp)
 {
-    return qio_channel_writev_full(ioc, iov, niov, NULL, 0, errp);
+    return qio_channel_writev_full(ioc, iov, niov, NULL, 0, 0, errp);
 }
 
 
@@ -297,7 +313,7 @@ ssize_t qio_channel_write(QIOChannel *ioc,
                           Error **errp)
 {
     struct iovec iov = { .iov_base = (char *)buf, .iov_len = buflen };
-    return qio_channel_writev_full(ioc, &iov, 1, NULL, 0, errp);
+    return qio_channel_writev_full(ioc, &iov, 1, NULL, 0, 0, errp);
 }
 
 
@@ -473,6 +489,19 @@ off_t qio_channel_io_seek(QIOChannel *ioc,
     return klass->io_seek(ioc, offset, whence, errp);
 }
 
+int qio_channel_flush(QIOChannel *ioc,
+                                Error **errp)
+{
+    QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc);
+
+    if (!klass->io_flush ||
+        !qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY)) {
+        return 0;
+    }
+
+    return klass->io_flush(ioc, errp);
+}
+
 
 static void qio_channel_restart_read(void *opaque)
 {
diff --git a/migration/rdma.c b/migration/rdma.c
index ef1e65ec36..672d1958a9 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -2840,6 +2840,7 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *ioc,
                                        size_t niov,
                                        int *fds,
                                        size_t nfds,
+                                       int flags,
                                        Error **errp)
 {
     QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(ioc);
diff --git a/scsi/pr-manager-helper.c b/scsi/pr-manager-helper.c
index 451c7631b7..3be52a98d5 100644
--- a/scsi/pr-manager-helper.c
+++ b/scsi/pr-manager-helper.c
@@ -77,7 +77,7 @@ static int pr_manager_helper_write(PRManagerHelper *pr_mgr,
         iov.iov_base = (void *)buf;
         iov.iov_len = sz;
         n_written = qio_channel_writev_full(QIO_CHANNEL(pr_mgr->ioc), &iov, 1,
-                                            nfds ? &fd : NULL, nfds, errp);
+                                            nfds ? &fd : NULL, nfds, 0, errp);
 
         if (n_written <= 0) {
             assert(n_written != QIO_CHANNEL_ERR_BLOCK);
diff --git a/tests/unit/test-io-channel-socket.c b/tests/unit/test-io-channel-socket.c
index c49eec1f03..6713886d02 100644
--- a/tests/unit/test-io-channel-socket.c
+++ b/tests/unit/test-io-channel-socket.c
@@ -444,6 +444,7 @@ static void test_io_channel_unix_fd_pass(void)
                             G_N_ELEMENTS(iosend),
                             fdsend,
                             G_N_ELEMENTS(fdsend),
+                            0,
                             &error_abort);
 
     qio_channel_readv_full(dst,

From 2bc58ffc2926a4efdd03edfb5909861fefc68c3d Mon Sep 17 00:00:00 2001
From: Leonardo Bras <leobras@redhat.com>
Date: Fri, 13 May 2022 03:28:32 -0300
Subject: [PATCH 388/935] QIOChannelSocket: Implement io_writev zero copy flag
 & io_flush for CONFIG_LINUX
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

For CONFIG_LINUX, implement the new zero copy flag and the optional callback
io_flush on QIOChannelSocket, but enables it only when MSG_ZEROCOPY
feature is available in the host kernel, which is checked on
qio_channel_socket_connect_sync()

qio_channel_socket_flush() was implemented by counting how many times
sendmsg(...,MSG_ZEROCOPY) was successfully called, and then reading the
socket's error queue, in order to find how many of them finished sending.
Flush will loop until those counters are the same, or until some error occurs.

Notes on using writev() with QIO_CHANNEL_WRITE_FLAG_ZERO_COPY:
1: Buffer
- As MSG_ZEROCOPY tells the kernel to use the same user buffer to avoid copying,
some caution is necessary to avoid overwriting any buffer before it's sent.
If something like this happen, a newer version of the buffer may be sent instead.
- If this is a problem, it's recommended to call qio_channel_flush() before freeing
or re-using the buffer.

2: Locked memory
- When using MSG_ZERCOCOPY, the buffer memory will be locked after queued, and
unlocked after it's sent.
- Depending on the size of each buffer, and how often it's sent, it may require
a larger amount of locked memory than usually available to non-root user.
- If the required amount of locked memory is not available, writev_zero_copy
will return an error, which can abort an operation like migration,
- Because of this, when an user code wants to add zero copy as a feature, it
requires a mechanism to disable it, so it can still be accessible to less
privileged users.

Signed-off-by: Leonardo Bras <leobras@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20220513062836.965425-4-leobras@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 include/io/channel-socket.h |   2 +
 io/channel-socket.c         | 116 ++++++++++++++++++++++++++++++++++--
 2 files changed, 114 insertions(+), 4 deletions(-)

diff --git a/include/io/channel-socket.h b/include/io/channel-socket.h
index e747e63514..513c428fe4 100644
--- a/include/io/channel-socket.h
+++ b/include/io/channel-socket.h
@@ -47,6 +47,8 @@ struct QIOChannelSocket {
     socklen_t localAddrLen;
     struct sockaddr_storage remoteAddr;
     socklen_t remoteAddrLen;
+    ssize_t zero_copy_queued;
+    ssize_t zero_copy_sent;
 };
 
 
diff --git a/io/channel-socket.c b/io/channel-socket.c
index 05c425abb8..dc9c165de1 100644
--- a/io/channel-socket.c
+++ b/io/channel-socket.c
@@ -25,6 +25,14 @@
 #include "io/channel-watch.h"
 #include "trace.h"
 #include "qapi/clone-visitor.h"
+#ifdef CONFIG_LINUX
+#include <linux/errqueue.h>
+#include <sys/socket.h>
+
+#if (defined(MSG_ZEROCOPY) && defined(SO_ZEROCOPY))
+#define QEMU_MSG_ZEROCOPY
+#endif
+#endif
 
 #define SOCKET_MAX_FDS 16
 
@@ -54,6 +62,8 @@ qio_channel_socket_new(void)
 
     sioc = QIO_CHANNEL_SOCKET(object_new(TYPE_QIO_CHANNEL_SOCKET));
     sioc->fd = -1;
+    sioc->zero_copy_queued = 0;
+    sioc->zero_copy_sent = 0;
 
     ioc = QIO_CHANNEL(sioc);
     qio_channel_set_feature(ioc, QIO_CHANNEL_FEATURE_SHUTDOWN);
@@ -153,6 +163,16 @@ int qio_channel_socket_connect_sync(QIOChannelSocket *ioc,
         return -1;
     }
 
+#ifdef QEMU_MSG_ZEROCOPY
+    int ret, v = 1;
+    ret = setsockopt(fd, SOL_SOCKET, SO_ZEROCOPY, &v, sizeof(v));
+    if (ret == 0) {
+        /* Zero copy available on host */
+        qio_channel_set_feature(QIO_CHANNEL(ioc),
+                                QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY);
+    }
+#endif
+
     return 0;
 }
 
@@ -533,6 +553,7 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,
     char control[CMSG_SPACE(sizeof(int) * SOCKET_MAX_FDS)];
     size_t fdsize = sizeof(int) * nfds;
     struct cmsghdr *cmsg;
+    int sflags = 0;
 
     memset(control, 0, CMSG_SPACE(sizeof(int) * SOCKET_MAX_FDS));
 
@@ -557,15 +578,31 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,
         memcpy(CMSG_DATA(cmsg), fds, fdsize);
     }
 
+#ifdef QEMU_MSG_ZEROCOPY
+    if (flags & QIO_CHANNEL_WRITE_FLAG_ZERO_COPY) {
+        sflags = MSG_ZEROCOPY;
+    }
+#endif
+
  retry:
-    ret = sendmsg(sioc->fd, &msg, 0);
+    ret = sendmsg(sioc->fd, &msg, sflags);
     if (ret <= 0) {
-        if (errno == EAGAIN) {
+        switch (errno) {
+        case EAGAIN:
             return QIO_CHANNEL_ERR_BLOCK;
-        }
-        if (errno == EINTR) {
+        case EINTR:
             goto retry;
+#ifdef QEMU_MSG_ZEROCOPY
+        case ENOBUFS:
+            if (sflags & MSG_ZEROCOPY) {
+                error_setg_errno(errp, errno,
+                                 "Process can't lock enough memory for using MSG_ZEROCOPY");
+                return -1;
+            }
+            break;
+#endif
         }
+
         error_setg_errno(errp, errno,
                          "Unable to write to socket");
         return -1;
@@ -659,6 +696,74 @@ static ssize_t qio_channel_socket_writev(QIOChannel *ioc,
 }
 #endif /* WIN32 */
 
+
+#ifdef QEMU_MSG_ZEROCOPY
+static int qio_channel_socket_flush(QIOChannel *ioc,
+                                    Error **errp)
+{
+    QIOChannelSocket *sioc = QIO_CHANNEL_SOCKET(ioc);
+    struct msghdr msg = {};
+    struct sock_extended_err *serr;
+    struct cmsghdr *cm;
+    char control[CMSG_SPACE(sizeof(*serr))];
+    int received;
+    int ret = 1;
+
+    msg.msg_control = control;
+    msg.msg_controllen = sizeof(control);
+    memset(control, 0, sizeof(control));
+
+    while (sioc->zero_copy_sent < sioc->zero_copy_queued) {
+        received = recvmsg(sioc->fd, &msg, MSG_ERRQUEUE);
+        if (received < 0) {
+            switch (errno) {
+            case EAGAIN:
+                /* Nothing on errqueue, wait until something is available */
+                qio_channel_wait(ioc, G_IO_ERR);
+                continue;
+            case EINTR:
+                continue;
+            default:
+                error_setg_errno(errp, errno,
+                                 "Unable to read errqueue");
+                return -1;
+            }
+        }
+
+        cm = CMSG_FIRSTHDR(&msg);
+        if (cm->cmsg_level != SOL_IP &&
+            cm->cmsg_type != IP_RECVERR) {
+            error_setg_errno(errp, EPROTOTYPE,
+                             "Wrong cmsg in errqueue");
+            return -1;
+        }
+
+        serr = (void *) CMSG_DATA(cm);
+        if (serr->ee_errno != SO_EE_ORIGIN_NONE) {
+            error_setg_errno(errp, serr->ee_errno,
+                             "Error on socket");
+            return -1;
+        }
+        if (serr->ee_origin != SO_EE_ORIGIN_ZEROCOPY) {
+            error_setg_errno(errp, serr->ee_origin,
+                             "Error not from zero copy");
+            return -1;
+        }
+
+        /* No errors, count successfully finished sendmsg()*/
+        sioc->zero_copy_sent += serr->ee_data - serr->ee_info + 1;
+
+        /* If any sendmsg() succeeded using zero copy, return 0 at the end */
+        if (serr->ee_code != SO_EE_CODE_ZEROCOPY_COPIED) {
+            ret = 0;
+        }
+    }
+
+    return ret;
+}
+
+#endif /* QEMU_MSG_ZEROCOPY */
+
 static int
 qio_channel_socket_set_blocking(QIOChannel *ioc,
                                 bool enabled,
@@ -789,6 +894,9 @@ static void qio_channel_socket_class_init(ObjectClass *klass,
     ioc_klass->io_set_delay = qio_channel_socket_set_delay;
     ioc_klass->io_create_watch = qio_channel_socket_create_watch;
     ioc_klass->io_set_aio_fd_handler = qio_channel_socket_set_aio_fd_handler;
+#ifdef QEMU_MSG_ZEROCOPY
+    ioc_klass->io_flush = qio_channel_socket_flush;
+#endif
 }
 
 static const TypeInfo qio_channel_socket_info = {

From abb6295b3ace5d17c3a65936913fc346616dbf14 Mon Sep 17 00:00:00 2001
From: Leonardo Bras <leobras@redhat.com>
Date: Fri, 13 May 2022 03:28:33 -0300
Subject: [PATCH 389/935] migration: Add zero-copy-send parameter for QMP/HMP
 for Linux
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add property that allows zero-copy migration of memory pages
on the sending side, and also includes a helper function
migrate_use_zero_copy_send() to check if it's enabled.

No code is introduced to actually do the migration, but it allow
future implementations to enable/disable this feature.

On non-Linux builds this parameter is compiled-out.

Signed-off-by: Leonardo Bras <leobras@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220513062836.965425-5-leobras@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 migration/migration.c | 32 ++++++++++++++++++++++++++++++++
 migration/migration.h |  5 +++++
 migration/socket.c    | 11 +++++++++--
 monitor/hmp-cmds.c    |  6 ++++++
 qapi/migration.json   | 24 ++++++++++++++++++++++++
 5 files changed, 76 insertions(+), 2 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index 5a31b23bd6..3e91f4b5e2 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -910,6 +910,10 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp)
     params->multifd_zlib_level = s->parameters.multifd_zlib_level;
     params->has_multifd_zstd_level = true;
     params->multifd_zstd_level = s->parameters.multifd_zstd_level;
+#ifdef CONFIG_LINUX
+    params->has_zero_copy_send = true;
+    params->zero_copy_send = s->parameters.zero_copy_send;
+#endif
     params->has_xbzrle_cache_size = true;
     params->xbzrle_cache_size = s->parameters.xbzrle_cache_size;
     params->has_max_postcopy_bandwidth = true;
@@ -1567,6 +1571,11 @@ static void migrate_params_test_apply(MigrateSetParameters *params,
     if (params->has_multifd_compression) {
         dest->multifd_compression = params->multifd_compression;
     }
+#ifdef CONFIG_LINUX
+    if (params->has_zero_copy_send) {
+        dest->zero_copy_send = params->zero_copy_send;
+    }
+#endif
     if (params->has_xbzrle_cache_size) {
         dest->xbzrle_cache_size = params->xbzrle_cache_size;
     }
@@ -1679,6 +1688,11 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp)
     if (params->has_multifd_compression) {
         s->parameters.multifd_compression = params->multifd_compression;
     }
+#ifdef CONFIG_LINUX
+    if (params->has_zero_copy_send) {
+        s->parameters.zero_copy_send = params->zero_copy_send;
+    }
+#endif
     if (params->has_xbzrle_cache_size) {
         s->parameters.xbzrle_cache_size = params->xbzrle_cache_size;
         xbzrle_cache_resize(params->xbzrle_cache_size, errp);
@@ -2563,6 +2577,17 @@ int migrate_multifd_zstd_level(void)
     return s->parameters.multifd_zstd_level;
 }
 
+#ifdef CONFIG_LINUX
+bool migrate_use_zero_copy_send(void)
+{
+    MigrationState *s;
+
+    s = migrate_get_current();
+
+    return s->parameters.zero_copy_send;
+}
+#endif
+
 int migrate_use_xbzrle(void)
 {
     MigrationState *s;
@@ -4206,6 +4231,10 @@ static Property migration_properties[] = {
     DEFINE_PROP_UINT8("multifd-zstd-level", MigrationState,
                       parameters.multifd_zstd_level,
                       DEFAULT_MIGRATE_MULTIFD_ZSTD_LEVEL),
+#ifdef CONFIG_LINUX
+    DEFINE_PROP_BOOL("zero_copy_send", MigrationState,
+                      parameters.zero_copy_send, false),
+#endif
     DEFINE_PROP_SIZE("xbzrle-cache-size", MigrationState,
                       parameters.xbzrle_cache_size,
                       DEFAULT_MIGRATE_XBZRLE_CACHE_SIZE),
@@ -4303,6 +4332,9 @@ static void migration_instance_init(Object *obj)
     params->has_multifd_compression = true;
     params->has_multifd_zlib_level = true;
     params->has_multifd_zstd_level = true;
+#ifdef CONFIG_LINUX
+    params->has_zero_copy_send = true;
+#endif
     params->has_xbzrle_cache_size = true;
     params->has_max_postcopy_bandwidth = true;
     params->has_max_cpu_throttle = true;
diff --git a/migration/migration.h b/migration/migration.h
index a863032b71..e8f2941a55 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -375,6 +375,11 @@ MultiFDCompression migrate_multifd_compression(void);
 int migrate_multifd_zlib_level(void);
 int migrate_multifd_zstd_level(void);
 
+#ifdef CONFIG_LINUX
+bool migrate_use_zero_copy_send(void);
+#else
+#define migrate_use_zero_copy_send() (false)
+#endif
 int migrate_use_xbzrle(void);
 uint64_t migrate_xbzrle_cache_size(void);
 bool migrate_colo_enabled(void);
diff --git a/migration/socket.c b/migration/socket.c
index 05705a32d8..3754d8f72c 100644
--- a/migration/socket.c
+++ b/migration/socket.c
@@ -74,9 +74,16 @@ static void socket_outgoing_migration(QIOTask *task,
 
     if (qio_task_propagate_error(task, &err)) {
         trace_migration_socket_outgoing_error(error_get_pretty(err));
-    } else {
-        trace_migration_socket_outgoing_connected(data->hostname);
+           goto out;
     }
+
+    trace_migration_socket_outgoing_connected(data->hostname);
+
+    if (migrate_use_zero_copy_send()) {
+        error_setg(&err, "Zero copy send not available in migration");
+    }
+
+out:
     migration_channel_connect(data->s, sioc, data->hostname, err);
     object_unref(OBJECT(sioc));
 }
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index 93061a11af..622c783c32 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -1309,6 +1309,12 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict)
         p->has_multifd_zstd_level = true;
         visit_type_uint8(v, param, &p->multifd_zstd_level, &err);
         break;
+#ifdef CONFIG_LINUX
+    case MIGRATION_PARAMETER_ZERO_COPY_SEND:
+        p->has_zero_copy_send = true;
+        visit_type_bool(v, param, &p->zero_copy_send, &err);
+        break;
+#endif
     case MIGRATION_PARAMETER_XBZRLE_CACHE_SIZE:
         p->has_xbzrle_cache_size = true;
         if (!visit_type_size(v, param, &cache_size, &err)) {
diff --git a/qapi/migration.json b/qapi/migration.json
index 409eb086a2..2222f44250 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -741,6 +741,13 @@
 #                      will consume more CPU.
 #                      Defaults to 1. (Since 5.0)
 #
+# @zero-copy-send: Controls behavior on sending memory pages on migration.
+#                  When true, enables a zero-copy mechanism for sending
+#                  memory pages, if host supports it.
+#                  Requires that QEMU be permitted to use locked memory
+#                  for guest RAM pages.
+#                  Defaults to false. (Since 7.1)
+#
 # @block-bitmap-mapping: Maps block nodes and bitmaps on them to
 #                        aliases for the purpose of dirty bitmap migration.  Such
 #                        aliases may for example be the corresponding names on the
@@ -780,6 +787,7 @@
            'xbzrle-cache-size', 'max-postcopy-bandwidth',
            'max-cpu-throttle', 'multifd-compression',
            'multifd-zlib-level' ,'multifd-zstd-level',
+           { 'name': 'zero-copy-send', 'if' : 'CONFIG_LINUX'},
            'block-bitmap-mapping' ] }
 
 ##
@@ -906,6 +914,13 @@
 #                      will consume more CPU.
 #                      Defaults to 1. (Since 5.0)
 #
+# @zero-copy-send: Controls behavior on sending memory pages on migration.
+#                  When true, enables a zero-copy mechanism for sending
+#                  memory pages, if host supports it.
+#                  Requires that QEMU be permitted to use locked memory
+#                  for guest RAM pages.
+#                  Defaults to false. (Since 7.1)
+#
 # @block-bitmap-mapping: Maps block nodes and bitmaps on them to
 #                        aliases for the purpose of dirty bitmap migration.  Such
 #                        aliases may for example be the corresponding names on the
@@ -960,6 +975,7 @@
             '*multifd-compression': 'MultiFDCompression',
             '*multifd-zlib-level': 'uint8',
             '*multifd-zstd-level': 'uint8',
+            '*zero-copy-send': { 'type': 'bool', 'if': 'CONFIG_LINUX' },
             '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } }
 
 ##
@@ -1106,6 +1122,13 @@
 #                      will consume more CPU.
 #                      Defaults to 1. (Since 5.0)
 #
+# @zero-copy-send: Controls behavior on sending memory pages on migration.
+#                  When true, enables a zero-copy mechanism for sending
+#                  memory pages, if host supports it.
+#                  Requires that QEMU be permitted to use locked memory
+#                  for guest RAM pages.
+#                  Defaults to false. (Since 7.1)
+#
 # @block-bitmap-mapping: Maps block nodes and bitmaps on them to
 #                        aliases for the purpose of dirty bitmap migration.  Such
 #                        aliases may for example be the corresponding names on the
@@ -1158,6 +1181,7 @@
             '*multifd-compression': 'MultiFDCompression',
             '*multifd-zlib-level': 'uint8',
             '*multifd-zstd-level': 'uint8',
+            '*zero-copy-send': { 'type': 'bool', 'if': 'CONFIG_LINUX' },
             '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } }
 
 ##

From d2fafb6a6814a8998607d0baf691265032996a0f Mon Sep 17 00:00:00 2001
From: Leonardo Bras <leobras@redhat.com>
Date: Fri, 13 May 2022 03:28:34 -0300
Subject: [PATCH 390/935] migration: Add migrate_use_tls() helper
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

A lot of places check parameters.tls_creds in order to evaluate if TLS is
in use, and sometimes call migrate_get_current() just for that test.

Add new helper function migrate_use_tls() in order to simplify testing
for TLS usage.

Signed-off-by: Leonardo Bras <leobras@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220513062836.965425-6-leobras@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 migration/channel.c   | 3 +--
 migration/migration.c | 9 +++++++++
 migration/migration.h | 1 +
 migration/multifd.c   | 5 +----
 4 files changed, 12 insertions(+), 6 deletions(-)

diff --git a/migration/channel.c b/migration/channel.c
index c6a8dcf1d7..a162d00fea 100644
--- a/migration/channel.c
+++ b/migration/channel.c
@@ -38,8 +38,7 @@ void migration_channel_process_incoming(QIOChannel *ioc)
     trace_migration_set_incoming_channel(
         ioc, object_get_typename(OBJECT(ioc)));
 
-    if (s->parameters.tls_creds &&
-        *s->parameters.tls_creds &&
+    if (migrate_use_tls() &&
         !object_dynamic_cast(OBJECT(ioc),
                              TYPE_QIO_CHANNEL_TLS)) {
         migration_tls_channel_process_incoming(s, ioc, &local_err);
diff --git a/migration/migration.c b/migration/migration.c
index 3e91f4b5e2..4b6df2eb5e 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -2588,6 +2588,15 @@ bool migrate_use_zero_copy_send(void)
 }
 #endif
 
+int migrate_use_tls(void)
+{
+    MigrationState *s;
+
+    s = migrate_get_current();
+
+    return s->parameters.tls_creds && *s->parameters.tls_creds;
+}
+
 int migrate_use_xbzrle(void)
 {
     MigrationState *s;
diff --git a/migration/migration.h b/migration/migration.h
index e8f2941a55..485d58b95f 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -380,6 +380,7 @@ bool migrate_use_zero_copy_send(void);
 #else
 #define migrate_use_zero_copy_send() (false)
 #endif
+int migrate_use_tls(void);
 int migrate_use_xbzrle(void);
 uint64_t migrate_xbzrle_cache_size(void);
 bool migrate_colo_enabled(void);
diff --git a/migration/multifd.c b/migration/multifd.c
index 9ea4f581e2..2a8c8570c3 100644
--- a/migration/multifd.c
+++ b/migration/multifd.c
@@ -782,15 +782,12 @@ static bool multifd_channel_connect(MultiFDSendParams *p,
                                     QIOChannel *ioc,
                                     Error *error)
 {
-    MigrationState *s = migrate_get_current();
-
     trace_multifd_set_outgoing_channel(
         ioc, object_get_typename(OBJECT(ioc)),
         migrate_get_current()->hostname, error);
 
     if (!error) {
-        if (s->parameters.tls_creds &&
-            *s->parameters.tls_creds &&
+        if (migrate_use_tls() &&
             !object_dynamic_cast(OBJECT(ioc),
                                  TYPE_QIO_CHANNEL_TLS)) {
             multifd_tls_channel_connect(p, ioc, &error);

From 33d70973a3a6e8c6b62bcbc64d9e488961981007 Mon Sep 17 00:00:00 2001
From: Leonardo Bras <leobras@redhat.com>
Date: Fri, 13 May 2022 03:28:35 -0300
Subject: [PATCH 391/935] multifd: multifd_send_sync_main now returns negative
 on error
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Even though multifd_send_sync_main() currently emits error_reports, it's
callers don't really check it before continuing.

Change multifd_send_sync_main() to return -1 on error and 0 on success.
Also change all it's callers to make use of this change and possibly fail
earlier.

(This change is important to next patch on  multifd zero copy
implementation, to make it sure an error in zero-copy flush does not go
unnoticed.

Signed-off-by: Leonardo Bras <leobras@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20220513062836.965425-7-leobras@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 migration/multifd.c | 10 ++++++----
 migration/multifd.h |  2 +-
 migration/ram.c     | 29 ++++++++++++++++++++++-------
 3 files changed, 29 insertions(+), 12 deletions(-)

diff --git a/migration/multifd.c b/migration/multifd.c
index 2a8c8570c3..15fb668e64 100644
--- a/migration/multifd.c
+++ b/migration/multifd.c
@@ -566,17 +566,17 @@ void multifd_save_cleanup(void)
     multifd_send_state = NULL;
 }
 
-void multifd_send_sync_main(QEMUFile *f)
+int multifd_send_sync_main(QEMUFile *f)
 {
     int i;
 
     if (!migrate_use_multifd()) {
-        return;
+        return 0;
     }
     if (multifd_send_state->pages->num) {
         if (multifd_send_pages(f) < 0) {
             error_report("%s: multifd_send_pages fail", __func__);
-            return;
+            return -1;
         }
     }
     for (i = 0; i < migrate_multifd_channels(); i++) {
@@ -589,7 +589,7 @@ void multifd_send_sync_main(QEMUFile *f)
         if (p->quit) {
             error_report("%s: channel %d has already quit", __func__, i);
             qemu_mutex_unlock(&p->mutex);
-            return;
+            return -1;
         }
 
         p->packet_num = multifd_send_state->packet_num++;
@@ -608,6 +608,8 @@ void multifd_send_sync_main(QEMUFile *f)
         qemu_sem_wait(&p->sem_sync);
     }
     trace_multifd_send_sync_main(multifd_send_state->packet_num);
+
+    return 0;
 }
 
 static void *multifd_send_thread(void *opaque)
diff --git a/migration/multifd.h b/migration/multifd.h
index 7d0effcb03..bcf5992945 100644
--- a/migration/multifd.h
+++ b/migration/multifd.h
@@ -20,7 +20,7 @@ int multifd_load_cleanup(Error **errp);
 bool multifd_recv_all_channels_created(void);
 bool multifd_recv_new_channel(QIOChannel *ioc, Error **errp);
 void multifd_recv_sync_main(void);
-void multifd_send_sync_main(QEMUFile *f);
+int multifd_send_sync_main(QEMUFile *f);
 int multifd_queue_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset);
 
 /* Multifd Compression flags */
diff --git a/migration/ram.c b/migration/ram.c
index a2489a2699..5f5e37f64d 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -2909,6 +2909,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
 {
     RAMState **rsp = opaque;
     RAMBlock *block;
+    int ret;
 
     if (compress_threads_save_setup()) {
         return -1;
@@ -2943,7 +2944,11 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
     ram_control_before_iterate(f, RAM_CONTROL_SETUP);
     ram_control_after_iterate(f, RAM_CONTROL_SETUP);
 
-    multifd_send_sync_main(f);
+    ret =  multifd_send_sync_main(f);
+    if (ret < 0) {
+        return ret;
+    }
+
     qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
     qemu_fflush(f);
 
@@ -3052,7 +3057,11 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
 out:
     if (ret >= 0
         && migration_is_setup_or_active(migrate_get_current()->state)) {
-        multifd_send_sync_main(rs->f);
+        ret = multifd_send_sync_main(rs->f);
+        if (ret < 0) {
+            return ret;
+        }
+
         qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
         qemu_fflush(f);
         ram_transferred_add(8);
@@ -3112,13 +3121,19 @@ static int ram_save_complete(QEMUFile *f, void *opaque)
         ram_control_after_iterate(f, RAM_CONTROL_FINISH);
     }
 
-    if (ret >= 0) {
-        multifd_send_sync_main(rs->f);
-        qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
-        qemu_fflush(f);
+    if (ret < 0) {
+        return ret;
     }
 
-    return ret;
+    ret = multifd_send_sync_main(rs->f);
+    if (ret < 0) {
+        return ret;
+    }
+
+    qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
+    qemu_fflush(f);
+
+    return 0;
 }
 
 static void ram_save_pending(QEMUFile *f, void *opaque, uint64_t max_size,

From b7dbdd8e76cd03453c234dbb9578d20969859d74 Mon Sep 17 00:00:00 2001
From: Leonardo Bras <leobras@redhat.com>
Date: Fri, 13 May 2022 03:28:36 -0300
Subject: [PATCH 392/935] multifd: Send header packet without flags if
 zero-copy-send is enabled
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Since d48c3a0445 ("multifd: Use a single writev on the send side"),
sending the header packet and the memory pages happens in the same
writev, which can potentially make the migration faster.

Using channel-socket as example, this works well with the default copying
mechanism of sendmsg(), but with zero-copy-send=true, it will cause
the migration to often break.

This happens because the header packet buffer gets reused quite often,
and there is a high chance that by the time the MSG_ZEROCOPY mechanism get
to send the buffer, it has already changed, sending the wrong data and
causing the migration to abort.

It means that, as it is, the buffer for the header packet is not suitable
for sending with MSG_ZEROCOPY.

In order to enable zero copy for multifd, send the header packet on an
individual write(), without any flags, and the remanining pages with a
writev(), as it was happening before. This only changes how a migration
with zero-copy-send=true works, not changing any current behavior for
migrations with zero-copy-send=false.

Signed-off-by: Leonardo Bras <leobras@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220513062836.965425-8-leobras@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 migration/multifd.c | 22 +++++++++++++++++++---
 1 file changed, 19 insertions(+), 3 deletions(-)

diff --git a/migration/multifd.c b/migration/multifd.c
index 15fb668e64..2541cd2322 100644
--- a/migration/multifd.c
+++ b/migration/multifd.c
@@ -617,6 +617,7 @@ static void *multifd_send_thread(void *opaque)
     MultiFDSendParams *p = opaque;
     Error *local_err = NULL;
     int ret = 0;
+    bool use_zero_copy_send = migrate_use_zero_copy_send();
 
     trace_multifd_send_thread_start(p->id);
     rcu_register_thread();
@@ -639,9 +640,14 @@ static void *multifd_send_thread(void *opaque)
         if (p->pending_job) {
             uint64_t packet_num = p->packet_num;
             uint32_t flags = p->flags;
-            p->iovs_num = 1;
             p->normal_num = 0;
 
+            if (use_zero_copy_send) {
+                p->iovs_num = 0;
+            } else {
+                p->iovs_num = 1;
+            }
+
             for (int i = 0; i < p->pages->num; i++) {
                 p->normal[p->normal_num] = p->pages->offset[i];
                 p->normal_num++;
@@ -665,8 +671,18 @@ static void *multifd_send_thread(void *opaque)
             trace_multifd_send(p->id, packet_num, p->normal_num, flags,
                                p->next_packet_size);
 
-            p->iov[0].iov_len = p->packet_len;
-            p->iov[0].iov_base = p->packet;
+            if (use_zero_copy_send) {
+                /* Send header first, without zerocopy */
+                ret = qio_channel_write_all(p->c, (void *)p->packet,
+                                            p->packet_len, &local_err);
+                if (ret != 0) {
+                    break;
+                }
+            } else {
+                /* Send header using the same writev call */
+                p->iov[0].iov_len = p->packet_len;
+                p->iov[0].iov_base = p->packet;
+            }
 
             ret = qio_channel_writev_all(p->c, p->iov, p->iovs_num,
                                          &local_err);

From 5b1d9bab2da4fca3a3caee97c430e5709cb32b7b Mon Sep 17 00:00:00 2001
From: Leonardo Bras <leobras@redhat.com>
Date: Fri, 13 May 2022 03:28:37 -0300
Subject: [PATCH 393/935] multifd: Implement zero copy write in multifd
 migration (multifd-zero-copy)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Implement zero copy send on nocomp_send_write(), by making use of QIOChannel
writev + flags & flush interface.

Change multifd_send_sync_main() so flush_zero_copy() can be called
after each iteration in order to make sure all dirty pages are sent before
a new iteration is started. It will also flush at the beginning and at the
end of migration.

Also make it return -1 if flush_zero_copy() fails, in order to cancel
the migration process, and avoid resuming the guest in the target host
without receiving all current RAM.

This will work fine on RAM migration because the RAM pages are not usually freed,
and there is no problem on changing the pages content between writev_zero_copy() and
the actual sending of the buffer, because this change will dirty the page and
cause it to be re-sent on a next iteration anyway.

A lot of locked memory may be needed in order to use multifd migration
with zero-copy enabled, so disabling the feature should be necessary for
low-privileged users trying to perform multifd migrations.

Signed-off-by: Leonardo Bras <leobras@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220513062836.965425-9-leobras@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 migration/migration.c | 11 ++++++++++-
 migration/multifd.c   | 37 +++++++++++++++++++++++++++++++++++--
 migration/multifd.h   |  2 ++
 migration/socket.c    |  5 +++--
 4 files changed, 50 insertions(+), 5 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index 4b6df2eb5e..31739b2af9 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1497,7 +1497,16 @@ static bool migrate_params_check(MigrationParameters *params, Error **errp)
         error_prepend(errp, "Invalid mapping given for block-bitmap-mapping: ");
         return false;
     }
-
+#ifdef CONFIG_LINUX
+    if (params->zero_copy_send &&
+        (!migrate_use_multifd() ||
+         params->multifd_compression != MULTIFD_COMPRESSION_NONE ||
+         (params->tls_creds && *params->tls_creds))) {
+        error_setg(errp,
+                   "Zero copy only available for non-compressed non-TLS multifd migration");
+        return false;
+    }
+#endif
     return true;
 }
 
diff --git a/migration/multifd.c b/migration/multifd.c
index 2541cd2322..9282ab6aa4 100644
--- a/migration/multifd.c
+++ b/migration/multifd.c
@@ -569,6 +569,7 @@ void multifd_save_cleanup(void)
 int multifd_send_sync_main(QEMUFile *f)
 {
     int i;
+    bool flush_zero_copy;
 
     if (!migrate_use_multifd()) {
         return 0;
@@ -579,6 +580,20 @@ int multifd_send_sync_main(QEMUFile *f)
             return -1;
         }
     }
+
+    /*
+     * When using zero-copy, it's necessary to flush the pages before any of
+     * the pages can be sent again, so we'll make sure the new version of the
+     * pages will always arrive _later_ than the old pages.
+     *
+     * Currently we achieve this by flushing the zero-page requested writes
+     * per ram iteration, but in the future we could potentially optimize it
+     * to be less frequent, e.g. only after we finished one whole scanning of
+     * all the dirty bitmaps.
+     */
+
+    flush_zero_copy = migrate_use_zero_copy_send();
+
     for (i = 0; i < migrate_multifd_channels(); i++) {
         MultiFDSendParams *p = &multifd_send_state->params[i];
 
@@ -600,6 +615,17 @@ int multifd_send_sync_main(QEMUFile *f)
         ram_counters.transferred += p->packet_len;
         qemu_mutex_unlock(&p->mutex);
         qemu_sem_post(&p->sem);
+
+        if (flush_zero_copy && p->c) {
+            int ret;
+            Error *err = NULL;
+
+            ret = qio_channel_flush(p->c, &err);
+            if (ret < 0) {
+                error_report_err(err);
+                return -1;
+            }
+        }
     }
     for (i = 0; i < migrate_multifd_channels(); i++) {
         MultiFDSendParams *p = &multifd_send_state->params[i];
@@ -684,8 +710,8 @@ static void *multifd_send_thread(void *opaque)
                 p->iov[0].iov_base = p->packet;
             }
 
-            ret = qio_channel_writev_all(p->c, p->iov, p->iovs_num,
-                                         &local_err);
+            ret = qio_channel_writev_full_all(p->c, p->iov, p->iovs_num, NULL,
+                                              0, p->write_flags, &local_err);
             if (ret != 0) {
                 break;
             }
@@ -913,6 +939,13 @@ int multifd_save_setup(Error **errp)
         /* We need one extra place for the packet header */
         p->iov = g_new0(struct iovec, page_count + 1);
         p->normal = g_new0(ram_addr_t, page_count);
+
+        if (migrate_use_zero_copy_send()) {
+            p->write_flags = QIO_CHANNEL_WRITE_FLAG_ZERO_COPY;
+        } else {
+            p->write_flags = 0;
+        }
+
         socket_send_channel_create(multifd_new_send_channel_async, p);
     }
 
diff --git a/migration/multifd.h b/migration/multifd.h
index bcf5992945..4d8d89e5e5 100644
--- a/migration/multifd.h
+++ b/migration/multifd.h
@@ -92,6 +92,8 @@ typedef struct {
     uint32_t packet_len;
     /* pointer to the packet */
     MultiFDPacket_t *packet;
+    /* multifd flags for sending ram */
+    int write_flags;
     /* multifd flags for each packet */
     uint32_t flags;
     /* size of the next packet that contains pages */
diff --git a/migration/socket.c b/migration/socket.c
index 3754d8f72c..4fd5e85f50 100644
--- a/migration/socket.c
+++ b/migration/socket.c
@@ -79,8 +79,9 @@ static void socket_outgoing_migration(QIOTask *task,
 
     trace_migration_socket_outgoing_connected(data->hostname);
 
-    if (migrate_use_zero_copy_send()) {
-        error_setg(&err, "Zero copy send not available in migration");
+    if (migrate_use_zero_copy_send() &&
+        !qio_channel_has_feature(sioc, QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY)) {
+        error_setg(&err, "Zero copy send feature not detected in host kernel");
     }
 
 out:

From 041eb5bcf7394898786c50f448ea358e5067144d Mon Sep 17 00:00:00 2001
From: Helge Deller <deller@gmx.de>
Date: Mon, 16 May 2022 15:51:02 +0200
Subject: [PATCH 394/935] seabios-hppa: Update SeaBIOS-hppa to VERSION 5

New features and fixes in SeaBIOS for hppa/parisc:

* STI firmware now contains additional fonts built-in, which
  can be selected with qemu command-line options:
        -fw_cfg opt/font,string=1       - a HP 8x16 font
        -fw_cfg opt/font,string=2       - a HP 6x13 font
        -fw_cfg opt/font,string=3       - a HP 10x20 font
        -fw_cfg opt/font,string=4       - a Linux 16x32 font

* Fixed PS/2 keyboard emulation when running in graphical mode.
  This allows to type boot commands in the firmware boot menu if
  qemu was started with "-boot menu=on" (and no linux kernel was
  given on the qemu command line).

* Fix firmware rendenzvous code to clear all pending external intrrupts
  before entering the waiting loop.

Signed-off-by: Helge Deller <deller@gmx.de>
---
 pc-bios/hppa-firmware.img | Bin 701964 -> 719040 bytes
 roms/seabios-hppa         |   2 +-
 2 files changed, 1 insertion(+), 1 deletion(-)

diff --git a/pc-bios/hppa-firmware.img b/pc-bios/hppa-firmware.img
index 0ec2d96b8f48a46dc9c0fabd50d14d5a2212a071..392bce072bad607b1baaff0136f35844fa5d439e 100644
GIT binary patch
literal 719040
zcmeFaeSB2ay*IjMGfDRB8D=Jv0Fz{r*-69(Ep3p2A)JsI2xwreVgV(25;7Q&U}{AP
z7QFE|kN_dPC5B+)GaO<oisv4x9;DTC;Spaz)KX)@i&#+Xr`EBy&qHtJH1?p}?|1Dz
zA^396bMO5;_mBHI`LNl0uf1M>>-YPu-`iUA@T{9}v?!JlBKfBX7alDagoP;qq5Y*>
z2*r*kT_~bR<l~O-xMCOZA1|RCcvg(KdlH_~aK)B1esmj?zu}MH-NVxpV#NQM5SBm8
zoj(4Dv*(q6zhds(>H6H6<@$}w@4x3Ki|=2m7tLF`c*g8o=jnIddw;_?QQz3OV&&BD
ze}CEX#`;xvey{G{d%j=4Wa*E7wDkV(uUxwL&gJ*69ACd;#bPo1W&8s-jGr=o*`0Sy
z3={{7i^nfnet+Yd@i!FTfPck-+8ZWJoKjpee#M01;^AHU|LEDP3$YMaH?DVaO~JJq
z*9*7?bwxO%;v;WZpR)aOARoXTs~=N^5*Fvx7lp+c)kBw&?)p;s&>L3HtG&v3<kg*)
zK)!m|@-m+Vo-twgE-WLX3BUiMZH_X>f_yA*6+a9YhVv}JD$!dpLg;!JWoIZW64Aqo
z9?U~oMFh%3tgx}n68zsr#Pn6)vjpnwG5x>dTZ{GXSo7|`I-;JvdY}w>N5mXcJ$oG`
zxTfF=;+l+W60QKQiO6?1^3|yqP`|)9X5^1@S|+Kf(UD@*8|(GZpUkm6{Z3JKlRlZ_
zNbOxOboV4RgmGG*MgPyK7G<swUVV<LEZiyz`+uYh^+)-*3UC$Tx(3%+T-W0I4z40x
z<8WPv>v~+@#Wfz+_i%k5S23;$xNg8T5mx}$BwUkm1#wNmRf20OuD`=oifbCKGF%}H
zL#TV%hS1G>*t|#0d(6D|n)k!z{k(Cv2=lI+_po`7n)jG_?=|m-&HH)dt_bt4oA<DJ
zkDB+GdG9svht2zW<32)|cip^)&3n|m$IN@Lc|UC4&l`6ktj74oO!MAi-d{8C*O>Pu
z=KTrt{yTZU+pC1{&QdB=w^DWai$C<crh8f&Dnz(pwy0>BD`E||iPnaN;&4NyIF`i2
z!qdRn_Z5+tqx~-UzKH1+z{eTt;HR;}iWv51(1LOyp-X%V5Bnq^|MNR(>_6f2pZM|r
z5Bg|$C3L3YH=&ms-V43ha3=Iz!@HrU8%~EF555;#o8%pdIjJv@X9V95t&JdmB79gx
zP)66mL-Iv0a6!?p<MW7K{W{WzGhw`^w3_!m{X1C4@>8S6d#cyCrxhjh|IhS8$GVHP
z)=S*`rv7}DSN*SeAE>fjIvsa<2OiTd0S=0qtyHU7O3gsr_LoyDo!<T??b5&@^Z6m|
z3+5?|ZNJR%-Fp-;+#rrM90_gGdjY3C{>u%og*s5jOw>`;@M@?h*eu=+R_hk~`qo%N
zh#OopJu$$%GJ-M@<c%N?;eYGV(Aq?;f0ml6RAP<qQb#CN|4}=^7V$3IKzrqZCf(A!
zK32?ftmAapmzSu`+tXK<^Cjv#q&90K98tIL+uHtr-&XGNue4SDzpgFc#jms#|F*Uo
zFi!&|{z2lGE9~C+`@;~H{K^dGC)`N`%oBXy5^FeC_F=>8WuF8fr3X}z*>JS%&w%ZX
z4M)m8SH~!`Ft--ftIPz<ZNXQ|&ea_*`#?=o?oc6zsVSJZ$mrKpt5Ov_B%;Z5$3q@r
zeAAlvt-V(lzyqy47W3eq9jFUkR(;AW%+XRcSDCpbs%!fFz@<7NFke~h`*vNgz?@b^
zUMZ`LppOysF@nBD&@bSS1vq3g#!?0x${U5U3hG<W@*Lat8)DvIsfccbfQI|eas7Cp
z$3F-doJsg|4^ZahCE?%=UB35-h+*6u?<S0!<L%h!i0Q!T^1?@)mS9}CUEO+|vIf2{
z1m2DxDDiT93;Wi1FX4S8#$08*XEmWue;W3V?|H|syl3<KzG3hDzV*s`4!_?s?EMn@
zU5RmTVj9NYk?`k@zx7CHEyv)haK~M9-JQ*yy*gp%>hTDsq0fq|+S5w<#n`G{F?Ytc
zXy|`1ov^c4TYUka5W$kr<;^}_<1^-Gs_QY4Da-bVjCGx%BWxq!3tiSaUx*<O%PP!^
zd>O7Dk=eh}KiI#+Ke(<@&zAWbR}bYwd;4U2l;JbLXV8yMA@M=h69nw-J}m=tcig!K
zc!Tdu3uFkfZnds!O<JaNjTqJc3;*CFz+>VO`|gZ+gtN;dP;TGkl|tb>-utSE0d8Ya
zM|mJqgoB;vbDfsaR@YnL1RofP`v*7g2zw{53|(d!)a6DyVbpQJ8TBaS70lBaE0*nS
z?yNmcyaIobbj@~eeKoWeu+BjJ`ntwgG2v0z3SDcZHyn&x?i`2-pFL)a55#jW6HeDJ
zc8kD7@wXXGqIgE5Wirwq>hH?=qPa^ix<0Clib<}bZN0ktRLqyF#g*zin=F%u%H@l{
zg(r&dg-b>8sg*W$pd{yl)}&Mqcyk8B6D@&_&=C9onsdQkDzfYuB4eO5I}t4r#nDpn
zJ@z$<JlZ8CZqHEWVT_0DrAj5?bh_5}abe7rU-y_eAAC&w7PtkP7?qNw3A8f^z7gLT
z)3aSW+?j1Xx*NFD>6+(>v2UQ=D%QvObm_poc3jn-Sb1FNxT`KJ;h1v?&supY_<yy>
zfqOCTPM7M=;8<Lm+sCmWZLTX-=Kqp?#(QS@JW;%1iQet{uG?qM#e<k9WgzzAh12mi
z_b*loeIWM2;OY1#70+%wH=T~Rw%}RCa~Ymz;MqD5o8re^VL#dL&RAH*e5vUBxnMl>
zd1Ri@BlAVk<R;<KI<?HUc<g@HI(Nn+(P9<2QMK)EeX6Sk{f+vv`d8)*wyg#{Hn=nJ
z{H3;LN(_0jNmHnAJ@8c0659cO#DgWGDAFm4v^83%v)-NQyxZ+NwZrZ`9oO8PXXlUH
znWuKxRQnEPzVpX`1=2Wv;tk73;-ZLk*n70Ns}=YLdT@0Dwr%cA+%?x0nP$ba!?jI5
zD|mLYeMt-CXI_(^Wf}a8bdjj_bwTgf9ea&=aQve&58+O6gX|mT1o%)HUMa3SwNm|<
zbA)fjkF3-M=KyoJk7=iO=YHU5^28i5^c8%GIKV#FH;E~dKap0zpUkloV;nu@6GbcV
zpzYL36Aq{cX<LX7_D*FUzHO6r$9%Qr0ns|p=^d;?nSsu%LA<x&xow~`o6lSDyai>q
z)uGIPJm&0uv~`46iJzZZW!o~dNj#ezE5`Xxa*Qj*Ke8Wu#<*GwqN;GL^t5AqoWITu
zQT>Ow?{EUv+m`4*b+jXWi#u~N@axnL&3$U6_E!yKIQ6jq16l4#15dU^^{Rq9E#i^h
zVr#*jU|weZgfZbZ99G1mIh~>Vw%rvv!tah1?sy<$io!DZ)`fnxllOo&B;O&AYircK
z<i$5t3%xUrIWqb4O$#hx%vavdMp0&I^oIMN&lLSTaxQO+>3V-t&LDXpc=Hi{PtphB
z2)n{T`(s+RykmXX9;^L`d?5*A;7}#oi#(R*{qzrGo<Kj(U=6P-EZ{og%>VR{)>6qw
zkcK|jBzX$UH6ZV>vfoXX{q&Do=9Mye7n5a}pFCaSP_rmj*J#b*CeeC2Zuer|fJd5}
zc!RVpr{l)lA$=RuIHuFFTtDW#bzp~gFxn_u1KGlg_qKr@S%U*{@XAKf7H$+<q9~6#
zwnb4E>&G6pk9t~>b``E7Z3tJHAM-M=zcJ_XhM11^E_C_z{oxfDpH;TKJNG5jEpZt2
zN*u;}8=fT&%XU#8>fDC*x3N6nA>}i!F&sO`O&Jb!WzbJ4=m#{w@h8U)ni(29>49T6
z`_!&xD%Am>R|EiEr297SU_Yn*_0+?;FZDNhNk^@qpVs~+lfFQk{Y@r)p<dD>=<KS#
zQFcUK8K}cQybi#2$$wN&qBcj``iK|A)8V*8eEakd8P+5oeEP?fKk^xPC+xd4AK`)g
z^MbcqV)q`mh=yN<Yz=RP(i`3kWi`AJ%5OLp8rSf8XhOr$PziLC>5$`QLx#I;PNP`S
zzr(g0*ICeSSO1Ref3#ywggmetJU5Q-752}x%>Dzmm$bO%>px(}b567UGi8VNJT9c&
z-6Ue5bG7Y(&@$GCHLulG<ldM+vCIOQx5icM-r$<zUJrS-7V_Dg$Te733PllQ!TFFK
zrbh}yab${^qAJSm;K?^bc3uz|BgCX}_V$2X#QJw=o&7uPrLH^lt-z^x{|<G4_mM%p
z)3#sB;M$_StaO6EWM~7p<J~6vs@;tK?zeR+o#_8}?O!M(`HcI`+CM1{+dnBcPyVY=
zY@J$$ty&wYjz&7lwtcE6pbyNkw`!{Pl$Pcyf_?^g+Mm=??H_3=uJ3qSBY>^#3C*fO
zhjLx(X^#NTYJsM>#(KK7cQtQ>Fy}Z4cZ@Yc*mInOKgJ5V{S5TI>ELm*!Q*b@Jl=X7
zx`YKV$OW7-mFemzrA8g8%!fRF5i-RBbB=4Nmjq{6MMFR4@g2-#U+A`mw?mZ;zYZ;K
zI38M#xxWu{zXo%^7IVJ|bKilv@4?(Zj=6stbN?LX{>8{8aX#{eNDLfQZyeaA&Kfvq
zYaZBTYjIt0Kk6EEcdM@{E$Yvd=Ez5)J@R|eJ#f%I4eRf<_Ma&`?JY`|>pbL|gnQS(
z82hX0xzt?sV4AOg(Ei%M8119}LGAY%(ln&uyEj=E<gqmS2;j}{Fb~hC=Gr^cd<~a?
zuU}Zy;AP?7{M)kAvR@McyMQPH?sNcm9?@6Aw%`|{VE}MF5gHenC|2P6>Ofs;JNU^>
zfL8@zF##|siA(~VCyOZoYl;92e}H+tB{Ejbja(}hMka`8<OWd>Je&=DoTH{I5!IvI
z44mB#oNbSkig08aY^(!i?=+mi9Gwp3N8SyMQ(fShZe@PMNzl}(P*&vS&;;<Kl7_P(
zOT#(P+L5w9LpS`q;T`CU{bipF9MmQZ?9y%kO!ok$PpZ!;>(zc`1JX|+{iJqO*{SVO
zx*ATEy=?!RmI8TWgQM5IzTsrq8|uT_DCb`HPG_&XOFNo6THBL03VPkk(C^+*Po<7l
z52THP487joYujL3s-)XWl@!*GK0IgJprqPr@cyKHe=+h-*>jQpvKRY1?WtNn>Uu71
zRDYv-zjh4IPvN=KzSrhaTGeC92K4{&{!aD2{zm&#xKh+VD;u>Bl@01svaYAKyOdPB
zqO3<+nzkNo{9f6J_f+V|9qJKTz6b4fpuHZ>CHnk0`uw!opQ@_QrKM@VPEFPJrKKd>
zhMxEX(EBZ_9rNc<D(t^2g3PEW+UQib?P8iMauP5;6)MqAq-JXS(=sCOU@rPG_a~v>
zo(kCjn^eFirC}`W;@64_ZAPl1<)>ND?w`@_=SiGIpFTvNK5<Qid=T(-M+QRakrSaT
ztvl7L{UmKnL#deCFikAf<^W&ErTHS5`v~UV^?gswRSa4uo#hK1^jm~J%trnw^4C-M
z*0!Kb18_JfiX~l>7E8qnd|w>_O-4YI5zr*~$9>=#YrrSgf;VgeU+4f&=m9@?9DLwu
z@PX&R2VNu}2)<voR<!{yHSj_UYzYc%2|nO!F7P!2d?8br0sO57{?-7$=K;ULBXV+y
zbI`}D1Ea+Jz-X}uI!Y$xZ|a_qYl^6cLUxzB+apnZ5oHaur)XV-yO^;d8hR9Rpt9}(
z$ni7WnJm}V8M@chPiy$CtyA~#jx}nUnZAnky!BXb5oK`pA8?NGo3>wDFnNFIar7ax
zzso;3c}M7R$XJ8h9@+hpwo>cd)*ISw*0J5>ZPlht<}t|2J?MX@bA@~7hQqoW_6g;i
z>@?eF>T|EZ%PlHOUG>3Eq0~=w)o+jK1x!PKu0{Ha%}{Lkjbq(DBXpeK%ra4#4%rdq
zqPSMba`~>+YN_@z{B|*nb1Qszt?VCdAmE*wz&ksR{fpiWSkLS%EEa(gLfNS6tMLxI
zhggL*t&+NHpfqiD{WWQ;UF$uqxVjsfq5m~|+LQV+=KQPj?QzOEuCPb;kIxkoT@`G5
zLo|P(OE=P?kEs;_*W7>?@-p<Y>mUB$GW4)9lP`)??VvWQt*iHMoOh{5u5<eXlT*Ul
z!&p;BD>a-W=VAAZ)>^%tvN`2k%*8zWer=cQ33moyJ{oK2?U<KT*X<qwJQ8jH8oSPU
zR`@rb4}CoOlu*DceUzgx&$KV300*nLtqA=9-@JHV1)82moQR;`ggN^!Ws~`exc6=F
z`d@=z9tR)&6=ajQz|Y^r8u<oz_c8F(*FkGXL1RZiN3UT`do{Fp&MFa}(<JJ(gG!uo
z2IUk!JJ%TZfw<~xYYgp!9I_gHTMItk4H^(!Gq8R<iuGd`c+Gs!&%8(o1}`txSU>Es
z7G*{tRa^_(-Yu~0JrJ;o)PN?ekU12L>E=LPiW*1}z3ZM_8`T<>IC#`8+F6BqdEB)?
z-`+Pv-D>-jHqyRT^ErO3Z=HMqvcy5iNT=-kRLCEcXWI6_H=TjD7HVGGFSRkQDc~nq
z`|WEri?&P4b>&0006esZHNPteIRbETO#_c2Y|vNmjC}Bnao`yfz%xp~Gp2)Q%m&Z6
z4fs<Dp0SuTckA1b9WucSe98>izPczkta~cv)pnxacI6hPvES?V#;(Iys&S3vSiocW
zt;Ti`Y0PVr@6(t!=tbZIWbD%=>c{phEz4CZ;+)T~rrS$3pX*V>_VLzZ`8SmBfISAf
zd^vPx%Hh=ekQSgmU*09Yi?-*VI)LX+G1>g~J;;*tK|_0q&q4T`+d6xDi9@aecLwI%
z3Ohl@y2jph&WGL5xqyejUFSpc`E7~Et5BvZHic!Jce+P`_l(lIx?|Kez}vL8#+MF|
zH?sf0(LXUC@e;C>fL~->!(rgz8NmGAQ2O<Yp{u<YO0Osp#}c*f>98|+BWFYDr(-@p
z+P=v7Bg`S|TR<;KyTm^LRzBAc-5JC657&IRFR6REZg*#pZlEjw)VZALcCWL6b`I!i
zv~%2J<aOTbmbNbEVt1x}mr_GmI_vn&w2dqsW*b>HWE+t-M~v;GKrUz$ePR9VPf`~e
z-j~4W(96mG5EdvSzzbg19*}E+<dx3VYzJ*W=pJ?I0NU<S7C4)XwmDzUzXzPVl!f5Y
zM!x%p<!c<4?*W<bjv0+2%=%`m5;N>CK=yky&6lhf-)7EODQ+Z=ux{c7@bVn{gt25`
z9KPXp5zL7j<H_JS$marnZ>r=;w;T2v*sor0-kE<t@J_Y~T$?cr7iXHdc!Pn9nj7D5
zG;vYlF2|b0S@s8bI~i&7q09aY*E86`MsY03{)7Jl21egu10!7K-?&2*t=p|vMY=?V
zbH68sc~^tzk8>5)1Kr)#&?71~^y+Fu7t)?bTEG(n4a3)yw|;U{=w;$6arE9dful!(
zpGSbFuZ1QAXF}F^HB=HT7hdp98~A2A_-0n3Hv1yh7TVaWzz1gG{UmtajrR53(iUf5
zFXy(v{$wlPJ;+y${4<dkcCVZ+S?)^N8d<j2nrn}x_@piNJ1Ngr6s5eN9<yeuPo)r-
zQyl9(srKiU)V{OcK}X!9_MOe*U1LA`&Svx3(RWaO6MYBeH>K|&(l@{sSDU>L->pg6
z#&)AM)iyRIB~hETl;!Pa+xGQJ25d&D_9vCh{y6Gt@~HiB)YatC3>$IE;r=-4Z1Om;
z)*Ytajb}l9+tKK8;#r~Ij=Oc;{#X(4U1{5|dz{CxempN>BWY*dGJG2axn-Gzlcu$r
z->{Zoy@B1Y><sMRm%)o|;Kk|S#aZCR`QXLlz>6n<7nhJP-1=t7!aLfr4pij~7RGM<
z&FL!J7Obgm+K$*h?6^`^BJG)ZB|qYLSSDO*?Ys9F=Kb}M4eQlSnE!OFVK-v_i3eGj
z|8d*@0X`teAck!Ox(f25Kk4XC7W$Kq{*06TITo_m<68CUxZ1+{1BKRpJM>oAXo|@r
zU}KSdn)HLTK==VMImxr^JC)m$au;~t*uYXz?10VJvEgcZbif8nI)vS3hFMRTbvxSJ
zSxNqex@V&98)e;btT$_vO5)trZImtLc#wCl*|#4i`4`gWvn=Yk9qUu|)#avLG|MeC
z%T*04hq23Yi&!r5URlpq%06V4tuV{Z8CLdYv+OM_3z@qTHjS(Le5~o<7cB2M?2d1L
zB>u5{-fQq**pG+!uj8K#{_glWWUR0BKjHULZmbSuSk@)irDf1lAg_HFvQsslv+(>T
z>_`I7`LOw3Savov4)~+uK4G9V>jH3NHgJT#x!Zs<<;XX7*%|POGvE)WL(_w}mc17m
zg)0}=c$Aw2x*v)4uLm?Z54PAS`wsnRdfO6xj^js|XV~6{^Ih0n<veqa9rqi&mUIDp
z8?s|?UE~;G!^z|N8pu)#(5DGOtgRRecun4Tj6-`X-dj{Y5E>8Kj>$TZhxp3!SjSGX
zFF~)!9PnXm##$X?J(SVW_fZ`4KzYs}o^!*cu7%4Ehtik58_L3%^Ea;3k1abL8Vmo3
z6};|E$jY%m7S?>|@`+m8xZr!xQ_q%V0Vd<1!}kQ=E$bkh61Cnxk@uMJ(AkHZ2FiBb
zbhhkqxqc9T=yM(FYuVa>&$15vorHd*7slVdjA!^ygmQfp>niBG0(Oj3zx1}lzUYR0
zp20dXPWlSAG|vke_G9>qSf=2v?U(6$vR4<y?DM9?$tw!#T>pmW#{e^~%fkMc;U_GJ
zx-11T=i1XTf5<)``8$pLtUK8@Vjhc6HK{Y4i^UG@G3k@hw#7mx*_O6gv)o)N)Xtbr
zolvg%4&5C`|BpLLJw37x@IKVh4f&u9^BHbCl=OpI?nK@UM*-G7_$x@`TuU9-82G`p
z)N!pl6KRll^4*ZNs~uDL4Lo5g*Ge;g>9G7|GXH$|5JtOhf!=xwepBeIZ3B8ca9?pY
z3cvF~F?!%K((!Hczv@xSe%pQ=dYj+i%f12m+<4xNd^_=7$9J{uC!r6R=9*;4sFc&O
z#x6x%%Urp>g3sn#Ub;+PtN#`;r5_jD$#2PL!C&d~&JS3DJK(1V{${?6xhMjT+!$Nm
zThDbgvB-CXe1ZNc_yu*?FsqQZE6^eou0`~d(uS*^l|F;5EF*FL-{IM^Z-^&>j*^YO
zY=SOw7=7u`*P)Ls=p*7h2Fm?|?NL1w^QXbb-Ua@Ic60X0wfrD#+UrAy3fI1L8G0x6
zv2TnqxgItFwiTG_=%qg);I#_O7wA+uLz*GJ1{OPdffG*nPJbOOMUZT%<rd&(9^U^O
z>m^<$?edI)L7y|tK3ip<TU`HU^f%{%quU)<(KqPn_akOw(9!Hs^(C%<1B^2rtKHH!
zZ2C3lIO@MWJ|$5*GF^_>@4q8a>-d~w2JZeu!VTk)b!MmaAr@EQ`-`-flb!<EBfi9(
zxWR{ZNgFz13j*1eKj8hLKz9Cl-0zXJR-0FY^aRT2^iy-3z?r{c>;a_FzEFzs2XIY6
znu^PT^rq-UF&WoX_?I=LIfuiB_KnR`dy9gj!2@PyOB;R?e$sa!teM(-uxFpqMxlNI
zdy5-3>CqV1KHA!ytAw9AJKm2tKY%>|>p90;F(vK1y-Jg@0a)kj>hX?x&)64Qy}=$)
z%zSpQwlFX??ZRoVT4$fCWl<IZ9ULbNc(I<~`y<5l550XEa^>Pc{zw6x7%?!R<<O7+
z9`tdaDT}ND9UulKv^IDIJUs=rY1lGM8tDMdlmiaMjw1JdT+c)H-TO>O?K#M}d(~#E
zueo*Y3+e_ft8YopwcxdtC#|+|r*&WUi5B&!{aP&xd91<rrln&(7ba$Up8$^hEjWPp
zPWX^BpmTM4#E#YC@3)I^k!^%_EO>TWI_ZqKnpl*(3c5FCtRJ^FA@+$eIJF~RWPMlz
zR-1Xo^Bv==<ag+Pw?V(VEpiU_hO^VwHk_Nb31x1ReZY6pbz<h29Vj1$?6(-Q-}1;S
zrIfesgS@pS0ysp5*>ZyKPqRReM&HMLH(3Yoz`>`{PuP@T&j%a~c&7^VBR!x>SuYE=
zh%do=@&QA_cN}P50JakVZ?5YlT-Vi~0frmbwyIA8hAVT@`kK5or}x|D^exT#obbe$
z76P7GCsWiq#w`)1wp`d?FQug=W{w7Ir=|x7rlza=hr#m&@g?DjvgN_EQ_}<gl7wgW
zNqe<~=lc?#R>ITyA?pZuxwaGL*<1PE@cQ3b8jk&)4fDo4Jxl|v3_1Cgk~4tS*JS13
z`%^89l}gOaJi+`d%R9<v1@>5AUqZYJ#!FZcFQq(@KQc_75_}Cjnz3$HWdY%WIbZRO
zI0w5N;D<itjzV7QN0nUT6EidZh&hd7?qtjz`7voVDZdPBPx9UW`|bHAeWg7I<&bY{
zk9?N=mupPLN{hN~b&T;<Rj^?IKAs&FYb@T1CW{KXzKY+Hwq@DZEke@gYmgOTcLbm9
z1bp&=Pw-EK(y<P!kPpittH&q*UU(}erL6{Tq{5q)Y2RVZY<s@<80!g+6>h?dc$Ao#
z{sh{uQV(j`z_UT>Eq&0daA(}l!anF#u5wQ*XrvNpR^*=v`>;wLUmq8`dKR{^xThQO
z05aYR&+V8e@KVT(=oe@w9q`DCpr4XYzHZ2l!S}7A0J0S3xn%MVVFer(Vl2~PZ;^Hl
z*jxo{)I+khYo;MvQ?7R1=+0PnqBNZ{>Oh=wa_LyK=?P?xur3=Y9f$V`j5R==Gx1!4
zZ%VLIxbbaL@V(NTK%<jUK7ZM(rMbAq<C<1?wsd;ktEJ_MnXVSxXZ0ENj)MF+0p&B7
zy;hn9`F0Fs$qB(TrRfsCYOM>1pOB{~Kvr0=tx>Nhe9O;<3L*E7rLK|GGs;hdz9a1i
zkfHEx9OWl%KXjbtBFKRBeg7S0>=M9fx`Csm7L#9-Kj(u#$H1S*f!_Zf^G!bh`A3P#
zKc<`fWA@kh$I_g?4dWTv7=K4%=E#HO8Nq?lj^R84_;AyX7Yx38qE4*~o-OUT!Y4fI
zO+NAC)=RN4$4@#-)Mk9nI<N3v4`JLP;j_p#i~IvHlXMS1S{A>72U}2X6ZkOo?0oRr
zO7Ploz?TWYmlD8XI^Zzdgu_~ce>mpLdL{qB7-}<Yfj#84-^4$h<R2v_Jx@nJll;T`
zP5gstT*HWiCLc+;ny&5D7++Cxe!j*>ps!FrxwjuY{%y#VUgcKUz{%rp0e-B3-_iyj
zZaQq@<6sk?0GoITY~s_g2G7Ra-iCRt#GEe1d@e`7?}KimlSgc9(T`CVg6s#I2IzJn
z=+;G;=jEzSr3!diB<T$GM^HcfQCYAx=flQ44))~<uq~Iswmcnc+HAo2Ho&+NFkTE8
zFUMH!1B};5n(Pl*Dz0@^v>XnNg<LqBZNjfIH!vV*9~brwS}J`I!D`_Jj?6`#Il+i>
zKH#u?IeW6J0{#ZWKKRNo``~L=>}AfQhP`ZECgKp6xHBhj6{+y+k5rGpKs(hBTL;P}
z_3ejEXQjstIojQ~A9kIko>aUmefwe8SqZ+wd2X`$HpKF;bKd37#68;?HQInZSe-Nz
zK9E+Wf-)86ZoV_-UeLBzPe|Lb=>vg$(TTEYfses&E@^48Pj}6FCnG!IYV|0=4#8M5
z5qLx#M$A+7=2ra%@HG0SV(=j^B-{z}q^$%z#M}Bv=o#h(9xC9E)sx-SJ91CLHa~CM
zin5E~Yw(x&4sh-q8{A&_7b+8r>=#iV<=BT<2YGvVqP3rTQ=rt^54&R~;@x0-L`>BI
zNOa&AG6s*aS>PG4JyPeIk2pBV8^J^7Ar=nyTYO)oEMP2L;SQ&T>A`sF+(){wcF^V(
zC!Z0Dy-V>yf2m-e&am_q-hgx;c<m%ylMz!|@XRjAayvZ*%=1Vi%4dRq##5(gO-3Dv
z8%f3`PVLlX9oLaZ!j>PvHObj%*0IV{0Q{1%x5H!WnMU4*b?!_ePkb?4J}rzP)_9_B
zp)W27TO8<N)4<G}LC{f08|6L7o~~<5U1yj~##lSOcl%}XoxqFYn2d#AB(^eU9PjUv
zp1H09?>E^CHLgd9A@+|-#t^eV15^EjoR5kUSNlL|&IRgu>?3%4C2e-VK^ZsA{=+6P
zXzx*^+=6cx*k9NHRoUN^oI&XD+zZg5W8VP#e7e?uInV=t!y^A;*{2@S2U_}uKIQ%?
z*{83y@tWDjs~7|7A6|E2QQmW`AM;=hJSO_=&B}<xA~DHcu4QAan;18Es%fN`FhKm<
z1xfd>g=pKjdK~{w8%F<gKa~AP8>}l?e<Ck$?X9RUsb|~^Sp_`JHgIjuAl3i};F5)T
zyuf(?TxjEj445}*U_{Q~K<tBu!3Sn=&By#1dh@r)4v3Xk18b!0O1-=ab*v(f3Zz-S
z#5|R^L_<GeJ+Y^1Yfu+?rIgW{H-u)foLOchX%_Rpi)G~8SLb{|y%znXenVNj2YuWG
z9d<8yF~;D7e7TEr%^2zJvFL;3Rc!mjZ@g~GNt;{rXy1c?ZLM!T=C%?(ip#JUT-dxJ
zHl?pA=L^_ZpP`+A?Xs-o0eBCLz<vNT#u~hN6L3_@t8$L8M?}U@8?n&m!=>W4!|iq9
ziQ;#(%TgA?x8H9J>-Ch8$Oq@FL5$)jsnxUv!PZ~(>8Dvz4=mJU3S!E#gC&;er=N`2
zBIz&J@SPDB(r0Gqg!8dB-A5e*bGZia0F50MfZflD=Xxxr5XS-I1A!Jxl>G!eZwwTQ
zzfoSKk1erC8O8M*Ht6&0H|)p1#T-<Uzl9$Zzb7xlH#r6=6FmsI0(z_Zo;CZ^PvFyq
z-rBrp_fpnp$VreZsh@qTZkCiUWh}gmzkDkmRtDmE3F0eZN<CbzaTA>}*hGtHj|*gm
z{B=9T`MOT=Tk^^JQfG{G&U{?kVXshtquE%K7aH=owClqT2mT0oH#{-z{&h{Eld|4Y
zS9mBE)$e}>^L#E`nsz^7D`N}*Gs5q!9>f#?x6Hl_!hckJ{bJy<S6JCs;DHf8;7?sl
zo&bH`S}_(hzDKC7&3X}W>8gCR(I8)9(U|4rEogTi>>z&GF6bI_wdt*?=)YXwoxouo
zb0+2Fa;)hyD;`7L*qf4AH2M?D_*<f(511y~K-nRl;e}iPJM~byKP5j&`vuWke*<5B
z6nK9LcE>!x>^i&`4aB_<ZQKCbC4T~Z30u&^|B%ms57XbO(hqZvua5!-w7<|UIa-v#
zx6yg3Q>}BZ6;jSnCu##&^RM_UAyZ&o&q(^`h)1xeQnn}nOx?u)%{xP@SP$!G+k}b4
zbI6G%y-x>zPoQ6Q;|9pLl)u)+?jOQAe8YSUlg~pI&{%kLXXquFr&1Kb9-vD8;`etU
z##)!pi+t&nfoVs<yy|jZ7Ug`xG}_uR&bf%yAx!4iK~An9tPI<K7VP_@9F6X3@a_b@
z<<|{_{K)5X{M0=WSHRI}th0_Co&wm>VAsvtb5mTDa%}WTQP+$#i&pUK%Pmp80DST5
zdpTea-rlzRZGIzOVBEJ?lsfl<j%zVK)%9Df>A%IAe!^4m^M@c8ob(hx7wg`x?Ow(*
z@N;$oR~+_?7wC&$i1>#X_OD!ozJCFBDF<*cm%NRAlJ!HzM*T0aZ<~91!^7IGteEP2
z7Ug23dG;UiG~M`at_fFkV$vLKVS!23e)_!ZU5HO;h1}HU9xZ*gr(k=AUkh@P;j=vf
zc?mXWq|Ha1RO_h+)%?QGAZPq*>XgXk)G6#&eaLwp{=(gX0PcN=&D-f|@81c1cqe?3
zJF^CzyWzjx>CyUkLLc6NeH=SI4*NU}GML(;t&(=T49afgaY@_vH^^=D!L+iyf=0Q=
z^6=!9sdHUxp<i17?@ZXX>)i)%{laz8v-jt)sk#2uv$t;|`T^OrZzA;aPRPU)(T7gR
z#3mjSRx(byQ@kL@GST@g_Fg<|KEHtHK0YIk`dIx$=U%L1t7!iujgf{)*W`%?P2k)E
z)JOVu#b`s#&E<R`ujDn!yr$fc#WGU<874PKxdiqa`&7tcQsx=ChVoCM*8UN2p(+Ai
zg7iM%n3R7Kc_T}b{Dbv@PCTHG)%Th_3pRn7u-8f34(Fb9HM20~Jo3a)A7Zld&YkxA
z{7gq#1$MYYfh>#67g&w69I`Ar@^mqeQMcs@Nt?Bfd%#0g)=4<j7p5fGH;$40VLJOW
zbBb&$Oq`XmF=2N(>jG?~FNtlkFRX*(OZe@FXe&41CgrOua6%oF6ZPo!%Mw0oC==RC
zBzzW)d^+%;Ay*ML3BPT9vL5zB@+&h=06c5fiC$kG){zOr<SOzq*!AUJJlHzHbDk6i
zPpZxPh`vpvW%aGe`7%h~Bj(gTQ5y~Y;4b@4#N~ibZr>LAi2KTr?sH8P8K6I1t5ZgR
z7etAJh?S*17;;t4MZys8S*#cDC;1Nftb#hJlv$>Z$mG3HcU=xl8R4N#Ix*96m^N$E
zKJ*RtLSy~r+?#xP6MTS>skA+WE!wDjy(v7G&)k<7*=b2&{l1L;k51}loI}n<avu3C
z=hdHk&u~05`M-o;18uK>9poI!rsHtF1MNq%I3~gwGTl;+dpJyuar-7w9wuCqFjZH=
zwx1~qz?U+HVA|*r{Elz1am#g}G6_$_L)1uEGR}bgpby%F?GT(W&k29lNZKxmSHuU5
z`*SI053>ORAF`UO3#M@W;{IMdPc2;g4DT}vH^}!nh28T0!+;O_hpP*)=h&5x`<=M|
z8}4@(uKPak_ZK$H_Xi7`<@>tAwer2KaGktAQV5?9@;w%eD+2eQqCD%{E7x!M^OAni
zb)B)fh}ZIwp5QmqE3QeahfG-<XbFA3rAKe&csZ9?ucrj#BZ}aU-_qI?+D_e_b>SWj
zl%^gAyxb-{zR|7-T^unnS);hdB9_Y+Ys%kCxl#Ijkp{bH3)Z7iY*)62eHhrYc)@(0
z#c`O=<MHe@pGV<&it(KH1)dk0&mZBr-h4ie=a~6?0MG5_^DaE^HJ{hx`Iz~955}pQ
z&kK;AFrTL(J(*`Lp7)yRBk|m7Jm+4-bG`X|4$lTI2theR2d^Yg#eT-jtM>SUM}3$0
zjywW5BI7O-i!{b<&V=8$g1&qB@!ilP4}kZp$C`R8WSKLe@rju!3yF`QIYT!a7MDnT
z1Wb=mhapZ%nt~0J_y<0eLLC5dW-Dx>qfDRQ4d4;Q=*Rm*^$poPLzW$D#Ch%wfzPG~
zw(Nz_mk;1hSyt+5xNl9iC1q*I$e`Vh#3GeGlv%_v?2%V03e(<YJLva9qn%;$v?(V;
zA5;^I(*8_**%%8QU_RE5eXz|#V}WnjjJZHu0r)D`wYRGEe~eUPKSO8e@`fMB+M#=4
zU$G9qD)d3ht%$WFpP;=na=XRNxPv5bg5FpxaTsZl_wYR#$K%g^N7A+tvsEMWB)=K!
zvxR-)8pN20M2&wB=E({;TH(X|Z&-`g*sGOn&M%%9BW*tNz`u$9WLV~7J(&+*<6QPn
z@@DwW>Fb0nJ<kdMH7?2nMJi$j;O`vd+5kK<c&ND_i|`K45(@BV)4(-3gOVQt_QqWM
z*04W`n!HsKo-NQ%O!)$1ygClL2Js;eB6cdC^SA9Sp<gl|>%toHXZBb2kA1sFs|Ihq
z0N=LKw-R<b+UH<rQW0AVzbkx4{%P!|Iae{tIMmh9KgLfnMuhUE*w~Wa4R|K?N9t9y
z|44rXWAP!&tXp=rtONEh8MlBs;WsE_e7f{SFm`j<t7Tn7K8U#VLBOY?%$NR$6J<|h
z?u(?4;oUNvyHU0e`FFuzTAC=&`yJ}q13yAK`jw72l#}RNPw-3`a5m#4aIT8`73BOZ
zDLX`60<kC`G8PnZDIZa<hdwjpCxDFq5XUk!2LH%Pxu3vX6X4&CVeR=I+E~kZ;ht{#
zA<*_3$DLRFDOls~b1b>yQ^8tS<$w+a{H|o1#QAlNYiUoq0X79U;=&*crjNrsEev>J
z`z4+&qK;qM!8Ta;aDT_}x>+yktZslm!ErZYO5pRSUxfMD7UGc5N7yf6w?I5nGH(xI
zD)FM+SB-Wa-xAZ`MY~1A(&2wCeH?LPQ&<=52vd{(7NnKRxUQdwGHDN4EyDIkl{;7-
z@j<iM@1%|n9m4QI049Lz8sNrSL(VP5*$O$+m>2wE6x#*P?Ey>z1x;^r54|<XC!nh_
zmMe@q;<%I<fIVZn@{#5(SSMq;#^KpR{6_i&;KOq8+}rS6f^>yE0{Y!_x#xEvbQAad
zLg)Jd(q_|khc+|8Q<medEIWa?uM?q3_!hY;-s`>4w7T~~<+x_yx;3%LH40_upIT1*
zf`1C})5Nb(je%dG#Yy}!U_n3BSA3)<O_A?_o{GtvVFRfmpCx~Y>S^~&Ig32P<k^7n
z?-PqMXPJC^AM@%_m3(O{=zcWr58)0uJ@+2mxu;+$^BAxk5#zJ8mmI@<94lq)>zc^x
zrCE^8h|~0Y@r(+_?Q{TNrL7LOTJF~<AHvO=+_9XuhF43mFAn~xG0=T^W`~pq9Gm#<
zSW}p1r-1K_S;hA^FX;=Vuc<aKg))}(OR*1ZA8kjzVw(%TU%J+)&;AL^16S`b*BjJn
zUCT6#Z58X`*rZI0^_H{-`rMSL$v)0LlOLcT+yit6=BoqnL#$8kIVl5vMV4hf#4Xmr
zIYF#)C)S<<SGRiyblU>XFXSj`vw0*M<g@3d`9jD3iKifo7C}$$-0-l@v}=&|uuQ{V
z?wiW7SHDhaMm?RB_hI|FiQghSg}&jiu48WrV~H!1>lmIP3tYWE05`D*TKtIp0!|l^
z|F}x^`Tbq6JwHFvt2Sx#DG$SDavVBFI2sjm0fY6Z%akpy)W^C2k9qcq!{XH+6q%F>
z?K?Cd;Cck>$KYF!>0$Eyiiyr%#?3%)$*}9zEc-<46>ZXdu;ca0Jp!h#4A|=EH{>J#
zdd_iDmS&%^z8_$pA+JC`N$PU`t<=GWaD3Q2at=9vD1-TieuZ(MY*Df-=FikS^7c{h
z0ACr>Em5Cbw}$*3(08Q2gLR@0oL~4misbrN=m9R8I*h5CQpZlzIFD1mUV`(6FrGuq
z)4=}0j*apLofv1j4ABYK!&dB{CtsCyeg&s_CXwS+op$v@rmcGS*L(z5`#LUsldmID
zGqQ_)!#rB)i(p&S`N3a@{4ag~l0A5O)<`ey2k@JnOZq)HhC;;SVXk5l$DwbdKT_8=
z(_2^{;f?ihNY`dL$|wUVIgFL%+>keP82aN#nV)ll*tCT#&%Uq@oF{Sta|2q<8#}xY
z;P)!mL-v7l0{oan`I<bK<$2b_aJ;}inta#~dG>`4J6sWYJ<ff|LSCFl#4-9tU4?T=
z`v%IKSfl1MzbscLbjELU?y>&tLmw+4SABu+jzo=PEBg%Em^sjiz4uXFC(X+|wK)rr
zN9~Je%f0l&@lC=8<6!?}TWEuA4YgBaKPO><I!Sx$n)E3gqfu`*WKF9;9~d{cBsu=!
zW50f}U|jnJ?y2E;6aMUJ!^cYd6y~EE^U*_kB|Jg1w0Tb<jJX~G-kEF@{qC{*Ap7Te
z@1tF6;4#9%KOV8YGOi$U2L2H<U&5b0n|YFSOjr!J9Yp3M&ZawdjO~?ITfCGL%I8_K
zlX3#mr0h{%Y00CUfNurx>s9hQzUfIB1<%(cWfbJOmhuc}eG2w-(EcQSOv%`t;cy2Z
zKQu5k=R&y2s4KkEQjlyLX|m4ngO+RAHufStPFg}7keaA<G46?a&jQxl0GM7MErwrS
z?Y{Rd8B3NwaFc%!YvBYL1M((f4Bmi@c?`1eQMr$#1hAf7@q|NdUXR11S`p)j@>~9f
zl^C&~^<!04>KG`8_yERnWnrDm$2vC->)chg?pxmqS+-ozi)>a+Mf)nw#DFe~bp~r&
zJ>FR_>YPno9rrm~J}dABu1$;1o~VlYrK(Dvy&TVuU%{3g_g?_+R*lr$ncNHW=^qTA
zRNF&;{}KDcy0o~}=XlpWuVN?czlAE!34*P>(NlpqfcdEZroeS+0`UPeVK=yg_=)G6
zvCpkadrcdS=eh8Cz{Z>=POo>pRFAvazeoU>mR!GB9(~AQpESf3nDw+ej=ATvF4m2D
z=CTiyR;ylFccW)M`mnI#66#*(f2kh#q56L1=9sbus{9vlE=V`=F?fhD${oKPlz@7x
zMyf;K7xH^Wld3XL^TDvUVvUM9_c3lhlg}J8_r%sCPhesi<E!UaAl?f7X}7mXdw2HT
z@ih0R31@uL9cfYLL!P;_OH=_Hcs9}+*<X$uW1fp|^I4yBnWqA4VI^Wji<AAvzGAuG
zjWkajBd=r3G0r!MLB5ygRY11#k~fg%K-V8}J(SORsjG(DCzEBj;7k_g2hIKf<H^2K
z)<1rD{z4J{ip?Tp_SHNiRqAxW<Bp0#R|WL>BCH9}0V$`JkWX}w2L<0RSsU<<$WGMQ
zAgh;j6#iq2TG*`pI2hNC1`ny>hO;Fd4d+UF3LpR8hQiK@pCkQj@DSoM&z3xn^rta)
zYu(wB|3k_VlU#H0d?*0@So@ha3ePVkbtuNbK+lmnB6YtZ9S8a#beu7PtPx(RV*r-G
z@%if>)fw;N5B@9m?G9+8wO`11mw#?sRkrE`V)oJg%fW$?4#+8S$cq1aT?}U$K(|WN
zr0p}tDFouu_6L3<rs94e_-+Yh!y-IWXHI1q$oL2F{vy{;!erxl{T2Aau$Nc=XJ^!%
zd-{+~g|EyHc}&<}f{$TZN*KQ2g|1$A)#h3~M%tv@B=-nsXx)=L1<vwOXRKSZYdp@S
z%EkU^2iv9GX3s$U%)_1v;M2{BZGj&fd~Y;kT|zZD|BkUPa(^@Nh_MvF5v&6_N3Nbz
zoVkZH3j*ol_YHkzF9zJAx8d!wmjf>B@&9$%8v&<yw&8f$JAsin=kQl$X95nKwfI)q
zhuE_l#aVryBMxp3^Z|?XNyDcGzD_-War|8C^Rbqc<9vpv8{RDYq~Q&o*$}e9E-25`
z+o)fAn7Z@kqWo^iKUINF=mak1hmhf`0jtdZN3ZN*e<arKN#G1DbMJ(veQNj<;X{mT
zm$W-3UkW`A{<oFz!~WYJ;`towY5Nmo<qYLUz<f;qUvVbfpY$1%@4|kxKSH-i!@lXd
z(B~(jzC7R*b~P(=a1MC=z$Tv`cw*%LV}23xTd5PS`(yan<j<f(W{PU<Gq!J?p?$WH
ze$A=t68Z}0DwXoWSg<xTF84;vNp9dZl>P4o{lJl<ZI?oam}gkKJv<{XQIq>LVF0{2
z#Qfa1zoC714co(g0m~Hpv(qy9z=(wNdA;ioy@wUl@n`fs7k%FZ9lmY79@T-5RkpBI
zv*{@|)M;Uzh>gAjJai%6X)iz<zKd)FYkk46Ivxeg4xpayVSVc+Ob5omPXjnPUUB;f
ztIh2?V*K4LfZrXM%Sy-zR#|UN-kO1u?1b^`v!O4I{LXyC@=Sp2^<RxVIWHqmwXEYP
z<}FQGh;Lt@pVZyZ&zwE%C*~9KR*v+cBmV_F-^TA|y>7~oD6`fqGmT}m|30M!_8yLL
z-LNrsVtifLYd#<SQE(PVBhF4+0(;Kw+YX05z}_dt?z18uD;$>fTa`O7{#nRBe7?Gn
zwyJ->&yTZ19>-ZGl?~X-1icCSn#y{x$Ehd6y-mh?yflJ5gxlthsUz91VX$r462m!@
zu(KoXOeARn=S+G6fCYRD`{dXOhnlSWkcV+92bgbo9m#SV^^#kM<(KjK$bW?2aTdqZ
z@>~+ocLcN@0bNHx(-F{f1hgChZAL(wIM?!X$Q%~P9AC5N)22l}Lb(xp+w!@u3;VYx
zU@uz<>?S30O^4nUn5w{@GiDZLL%Ajywo+NPFaRGf$_GnQjurk8{)Jlqf?$=knCX0H
zetgrB7DeB4hs%-RE7TFrFdx3J23?&S&d1n(MX6{{`Y5SaZEn`XSf}Bu95FVK?H1Vo
zR>8Kg20j=pQ3lcetgB*uhU<galZANdc8;0nFKqvZSTW*lA2Z5ZF0e1C<2?Fti2ZOa
z#MvdtN51STmGN+Ak~-zReNsN0jWOH=-nbYssW!x;GX69R@u&HSKOKko(+Q9lOCT@u
zJcipKN75&-9QqB<Gu#k9EA)*UzW*%kD;ry3H^%sky>{xH=0cnwg17<N9(WcA&fAQ&
z$8-U{)W&>R?+rfz+C@BSI^t2Y5RaOVc+_!-N1Xt9vjp<ybj<y1%>8YUH>ry)#(KV7
z>J)t;i(`XZeMimta%<|?cFE%mntK^EScN^ODd3$ISQ|2J@zpUr-wfZ5l;g3Ncve>&
zXTCLg>9ZD*o;2^t0-P0s{al~^!6qy}!a0>MhA!Jmw5$PLWI<QMej(^_wf+~ehD1(A
z;YTK4uJ!kvj;Jk?F2N5zP2?Oy`YlLb&R8lu|DfZrE;_0qV?iIJ-p{$}cn0q-p=|Ai
zZh)9kWB!zS#9BQIT@LdPyD4}sVFaEqg|cR%X3QItu-6-WMcSvN>{OF8o%?M@-r{MU
zK2KaTeLl{y_+{wwq;o=_UM&h>vy^8O&KwX$$kTr6ui63XCrfd5B-<^Bj$o|t!buN?
zE<b{~!FxN_%zmC5(f@GPh5kFeg9Ddy2Kyh#8O-mEiA`(t7ZBgN0R5@1I3=cR75UGe
z-t7xH4^!U383C`D@uslXI+L-zjfk^7>V{N+bN`|~AJ)(vBj<Z!@Fm15&Whr7O@H@s
z{hOi~e3iOowd;A0-(Fc11N;`a4tUb~pFv#1Gad`)p#K@fHaz1Q0XahNe<pj-K1Nwk
z_e|&r;#C*)KZ|n^pAEgEUQ}-Fe-?JppN8J(Kaf3GcOZ1q_JY#g--Yi7LTCEB@a?70
zTX_BrZBFX*T4vo7q2qPG2)&K+1E}vb=w~dxpX=Hp!o-DBce({+v)zbs8HE_58Qkyh
z+-bxa@r(f1?>K&Q-~aE8IA)G9Fe>fKfky$uT{uf`cQ($`%NayHJDiVtgbg+*=Pr+e
zm=SB+s@mhuxF;3-mA3wB%r*VHRmYn2qU3xdZMmFte-?H7>a7BP3yG829i(-sXJWs<
zflDKBerL{7;L~~RufL7o!MlD*T*6vKxfA%5F)r}4EBKVC8T~ZtC!RVEdRk|m6-7%9
ziek=@agGemkznkKxo7j6;#M3u-+(dN^uaJb8+RF-jXT$3ocrhtZ52~e@Q$&LxmbTh
z6yv;)ukneNSnv>O0Q8;Gw<qVzEl+~JVT&@>)X_Xoj`Z!oI3n1arrAm*ea{}E?=8yB
zU8Ps(`{WG7wSdMA`nH5E%R%?qVqtJEXuLv$PUW?|1F%d2O`^{7lW(gnST8<gEH~n@
zt4J%KCNkZmXQV%8()O&5UR|^|C24!|9qe=avv^00cT4Z&@i;4tatLG`_icnN`pq%m
zee~yiuU{&{l<{x|qdX^c(uvTA@9>-i>`T<PYMJlwtb`_yhq2=C9F%7lWU|iW`3HEu
zU7mq}^9=(2(1*yY!LO+E9EAFNMFn6sf6}9&%k}q&xsH20qw(FbxoB}1X9wvOCq&VV
zxUu)Y?;-Et4D*~8+WL_m=zAdR0(nK>s_X>q4){K~@8O)mt<n5vnTN8lr1?9^QxQuA
z{RZ->qyy?b&@be9t**!1V_b2>UHt>j@91`W9Y678J63zV(A9m=lk;4=JYK{}WxKjO
zKG$wfUc#So1U5yT^olbwR7v-sSv+f`SKJ)~#y9tXk?t8V(lfQsv{C&ppd9M&e*t9x
z7uyb;jl9~E-Twm0J`5k`L!h|_JXTz(;6bDD4f`qCZtfAx@4T*Ypn2fO>2;v_CeZ3S
zx7Uetl}?vp|1{$25$}?P{%1SZU<^OO7#cl!@Eer%p8}se0QzQ`U7+u3#Cz>TTU{RO
z=0`!_yNUCjksMp1#<k|_^zC^k@L=de*s6|`zPI%13yGtNTF*Nsea|Isn)E#vaCrgy
z<IJ-n)0eUg@~$yyeK+xrw4SK3SE3waVhq1VjmJXVL;Xdd?IP%IVZ?lefg6ykeP1~1
z5S!Ntn>+U2BYo~S(`WHRSJUS;@|<&nKAp++S3|oI=xUj^vywh<8KTcNWlq=GVf2Z$
zcZfbe$+lX4jP(~b58E^?moZucy}|)lV7%_dryS~0jEn0zY_0C;pn)oBi^0B$PZJKV
zzdrzd-pah+OsBeDII+yqz87=~V43Hx!{;^vemnOU!4lVD!0rXg6Nwu2L&6Ge(C5SV
zmqCl<2g&!x<=Q_4+w=*{4|>C1^&D}!O-sjfE9nQc-aTo*h~ZtIv;%vovEO2CYj`i}
zyPZ5XvKzh-^7W|R$2HY;-k>k6ImR9i(A>~EgEiIl`=K>8hifYEP_C&7|HxUuvEI9v
zhI(wXv=mz=*3t~+_PbYwHr@R|Xze778Tx1pYup!iH-*x>ehqvFyt&5N|Dx^08h&Ms
z+kIt?8(Pnwm+QGa+lFmkwZ<Xtk!u|4xq6M;Ied+Cq_BUm!`x@m0OCAk9G;1vPQ<t2
zJS)HbKz)>P;||6Qynyvh=bUryFjuc&t>j!?;9MDap0WwwhvB(1i*v#C3^sV4xvT@u
zQ@ED>&%|JYH^urIv6k(?dFvyTd3}vpgIvGx&NHM_`x>(c!83C4F8#ZZ3#Bd~`{K6|
z%tL-7(2y7X6qL^se)S7}lh0wpze9fn${XkE!}kXn8{^hK#JE==#)fCStuV%&(w(SH
zOTih!tBEV9L;5Em6H6Nx{SpQ&M%>4-!xk9DxT83mx<0y6)PoOagyW*#-l<h1y%=Xr
z!q%GO2d&pc;iqUo{}?MqodB^RMYM~+he{t&y4gP&GXdZD8Or?Z8{0AG1hzk8KWDIH
zL=oimbja&j=v#impR(&#+GD(yqC|~S!al;Mt8(u!-l>aMM(?sX`_AGFV%V{-kLrSF
z2;o`d`9ZkL^Mi2r^86s&vw40H?!LabETdyDCf^I-*Wmpc#y&;j*7Nk0M^{?h^pzuh
zG@d>5mBaS?Zgi<7FS^8XEz&Z<|6=TaJz|2sN!Mim2_N<ic9$}gKhHCnpr>wvj%0&Q
z$9;M{pQi%0C+_d#c|9^t5NA<#;4I1>=vfmmw<Vi9!}WmiOapG>MhQ365iChv0iEZY
z&lK{H7{xP%*uDzeH*8GRfkN!#_o;P=F6Mq)1@X>Rr>n3hvoiD!_uL@1Cztz})Fs->
zv(aA%&akB}=)l>vj<Gnm`&v(T{dM$pmYrFS@3L*4YxM7&X~gyk2kaX@fA6oML;ean
z<69W7MTy97Z^9OppLPM`p98pzL;B5t31P$7E5e2`l!Ogq>1W$2Qd4c;!TFu9LC1Ry
zFnSd}K*&+hLv8A+dfKu6xn;KwdMP`DJez<k@^>{*cL(&-9@xXVUK+TEGSDH@p+jat
zhs=i#ISyl*fbo=IPNxGWX9Fj115O$-OTUH=iP&^U5%_1ZJC3tSc4@nCmS<C%FLo)W
zBlf9Sy(E3h*|ywP_C*8VJP!V)#QqX1QtPw@t$%^du1>pUQ)X=L$)BnDOtW=qC+GYE
z{`A+N-{G6K!K!IXQ*7YF@{C&MgH5l(woUuB){3(pmZfAk_j}r%&wIL?*P~t3k25>=
zvFzq2cRwZTJ!i$)x+y-KJG%*M&K=;L3mFH1_%XmB`v_qGI-bQcBM-adBagXvHLu?#
z{nzH1Ut05%yOxsojO=rFj(pd>BYF0&c{bRbEBRtm)O+xI6E!)rSnuqnwDYY^5C4E=
zAtQXiZ>rfJoSQu0T#B{lE>A1usdmU$-E&q7y|uG+EA!iGQX!wG!S0v6&*-DCmiPnR
ztr|W9#A-_WuTl3fe#p57E+GENeGltVeFd`s{~L%KsOKAbU&Rv^wIvpkx&zMOT#P+#
z3V4)^?=xu)a=mXY=&S;;l4mWUuF8sESlnF4lm3qFv4_XwcX6<v)IZjN-_T%x>9?T2
z8@OBkR$QOMexN@(*Avo|R;SIITqn;t#cv<rtkr1fZOlb;vaiGaprC`{cB2IRCClX)
zF;3YQ)&q{6K54=M_~1xXyJhS^Ap1LR*#6(ZS~m;VOr$d=v1;>ropn|!Lfx-+S`lkl
zd&H0x#g|M6-fx0!NCocGhb{LHGEKfSUI1%{Y@2)XhTF>F69uixvjgB4USDw#GUG(}
zggdqPiNB~b5G#_wcwPDqurA8qUckC_4A1=i1^y0${Otu7&w8s#t>ic>_CwbB3-n+7
z=7ZEB?jE*J2fz7%-+H)epN=}9U@x!UjWgT7<oN8y_<q8A(AG|l6F$^g(03WnUzw=R
zq3r}~zY(hh+W(sEko<hx2H*<vV4nBkY`72P_>+G8niSgPjs-@GFl4n@#Y$0({sP+-
z{`Ljue$KGIyw2|iye)Gen|ld!&`K_SS+Z;tW%nzYICp6)<cJR{8c|;-<flv#9;zEL
zqS)udx~G0i-AhR0tiJ;5H)M}i*$(HAu~o!V#C32#C3x$mfmu0&r~gY<7UmN%PKY0Z
z-y=~o;x^!u)4&}&^#+{lpk%DOi!u3E;m;wHW^M<6*$P?9^r-=tz9AljZ4<A#wxgai
zkRM>f$~!q6uOW+x!fRq4(o!Uz=B1y67s{0fOU8@S)oKg&^U3q=6V<unrTmWmv0oVX
zq~U%++H7HGlY9ev_jJZNAjauB$TPb;<J8GRf0gA-TQACE|5)gA_VeC1fa5s3i?ob&
z$%Zv49XOr^nB@aj<091}T7DzsrI>p?Xnq!X!&~tg8sZXtu($mK&OOIotkB}XQ^L`H
zShh8tHi<;FdIaBQ_O&4H6?+_ki&E!LEK1$Pv(H~&GChj3YtAv2G4M2aMw8+T?1dla
zDV#ZrJs6;&ip`z+>cA61gMS43z*Dmj>zFNdk432#>;=oFtdv-k@(Ipe%f=eO=XbFG
zD_i3IqLddM58zprO?e!)1+#3*Cio<iWf{|%EL#bmWU}mZ_#v{f588=3$Dxne<Zqla
z@XUt{7`(rp{g?6z;D&WI9ki7Ne8>kc;l2cUcH6CQnENwK`2c4Cc;x!;#rlsm*pwx2
z8?bu)cB^K^TvYb8_+KU-aNVaofweuK@&n}s%*}LH1#HRl-8Vufjk?0_P2f9iljdMQ
ze$2hGr8l4a`xd}pp)vN+lQ<vJZd#qP%VyJTz~KW-dn<mpFK`t47yKE3+dOr^n!%U>
zIi~*y@<XCJMb>Nko;GjNuW>$ElPBD|Ep%MA?a#a1a#){g!(RA}?PdE&3*4*6^S>Z(
z;=FGENZ`>$=;D}r%MWpRh!fjmWgj5@FQgwS_Yh}~0cUysqY<-}1z61j+{Tfn0V5mm
zHXktB2%eQ$aS1TOJ_h(_tOj0KHUUnppmoHcWnCm3r0h;!S({Y@{Ng*;N!Y8bLgH1e
zPsV5ikIT(E$Pe9^b9ufb>N4#2-*C<(X$(4x*W8OVj(sLyz}(CDd-PStokYqZd(K3>
zY_&FudjK5IyXPT}?8|jcvAN*K3CK$|#6SBJI48C_jc0}7cUiI%r+0g;n-8vK?0A&#
z-1i7RDyR$Kxe)uTnf9H&cuu13)zDb-I>eFiY@mAe#FQBB75JXp*KE7QJq?JF&FE{^
z{=#u1Jqz_*f)4&c-D@GaPXyn`fCm1AZ#h`|83PagE_I!+#lbK>-{1-IH?t<O9l}Gx
z0zM2mZ}4Ma-ewZkn8W4dLsD<6R#V822@AllI8mMUkqNgfw5i~1LBy+ursJ9W2e_{~
z9Wvkq$bcn;H}bD%e)RhX7}s=JXKl_ytPeD5_)pkx(}%*oN`6S(;vNR-p2Q{d+&kiz
z<b^jGyb$xxJ(SH@KM=$3qdbDO)-wv}JnI$q5#J%=AJUfMZpHa4^7q`ZpP`T6Th8aJ
z;ByYHNr~#wvvA&36mxSE@egH<-#Dzz*h5}}GWACNdE7fzWp9<f^`CNGRR1F7n@3sB
zT%&%6y?<C2oe7pf+r}QCZ#q{mH)qwYI15$ccufw^b_>HF$vw8=wx-ZVS)TSKo_kcA
zz5sJWxq1Q939IYx(uHxho#jj5^CsvwGR7F^@j3yo9w`@A`<?^MR1I|c5xW+RBDS#{
zae9L>tg)!48e=#J*`<#$MTnEyWsD)U9eW=SVGP}%eIqvSTh>{gTQqs4%R>DV=eX?w
zpR|F$G6tG)T~g=7d`evt^c4f0MKO=7BRAsrGUkH@fMbx$!uXAY9>#SvbXvRx^>PnY
zjAwQ5x#3aZM9iXUtFfkM2#t6Ao?I(=fwLQO3-&3)XVDH>^f{z=BQBaY4(?|L9;Cg<
zHW*Khv=wHWjLS3T-tz<4-gKU`&iN16Ah$8@+9;dOvd0b<T)-SiU2k~0Z5GcaDR?A2
zAZFSol2?YdvTPvJ@(O%R{LQ)8y2b*;7a}eMG{my353=WC$f?CL4KdVAKXo2{JmXuX
z{si<2=u}2qsWzq?=M#obG9Bfj#<!Gr_)Wf>^(1Oi&X{TD_+g7O^3Y}_+d>{Wev}KF
z^-Py}GAuaTPoB5Zif?B7&=rmHjQ6ITZ}j&v_LD*`<+nt&eHZXtA&jn+v%bM^tY7A-
zv97_ltH&(su&$Bcg}`qyg)YO#a=E2f_W-8Y`<A<aa3zkamx!O$#|d8pM=dubYExGQ
zORTb-<bkvoe4EXJaw%*U=6Vp5YZzz}G<B2YYh2HaSbfT<`Q$fP!&;N;!t4~W8gk$n
z@&m^SoS~Cp`LY@N1i9B8Fqu!CLu<kBg4CsPZxv*~?8K>uz1FtQwZK`+4!*-Dl@0k_
z^uZ2<XE*l<;hx#onUiq*489ef%RdVIH$&X5Uc}mvA?Dz_6?mjzefW#xdD!}zwa+Mb
z!7ql`P}?}bR5+gSsN_HRt|7kz^<Sb~k7p;&0Dcrcr9I%yJzOt4<59??Lu&?kaaLs5
zzAf^j_VbV<Gc4`kH7RU=XwBeSVb}zhgT|+W#%F`Zr7s?G(E7fEIhV06@(k#5IUnEV
zGxz)cM1GGlalm*N`X=|DF-C#$zOIEbEgx$MdA=U05k(c(BGw?ezF_^~y0Bf(@8&ur
z=TNl@#}1r>)Vx+Fe6U7xeM)~T{<y|DlQc~kb;c-Z(}<w12-m7TR`JV2z+E4HGsUbQ
z<6g>GX7c*PjLgHR&#<wZ>lflAqO^-7>-cJU*2(hhPjmBb{w^r?a#0t-diQNGF?0h9
zd3GQTvCBMf2|`+OO$?}^b6gL(50zurk0ReuQ2@R#@f9$jt=2dXwpvB}^9bVQEZKMR
zvg+K2Bt4#0G7$3<wfU4W!@kk}P$k#1{c=6);x~iNlr7M09tWKX?xg{~(GSe}MzRdv
z4L(!tlX@<A?_7>EN%zD#=82%sSK*q^x0SfY?_c{yTE#bx9ry-04EzvlAN4HyX83!W
z_}!*K@`X=7&KjgFTUdZRh;xv?Y4-_!g9&LkGbYywpQgQA@ijjRA2sS>8T&!G)*Ug|
z>z_hj{}A?w)%ZTNhmf%~Y`^dt#I>T`z!(e0Zqb7!*DZz~cBC+T>u+E)(&gC$a&8Sf
z9?MZLfDU*Gx;f7inSr~4yLPnngMI7+%M`4@y4ES81<{ziGY#h+9EPoC8S_MTiFZ*?
zpP@g>*rsU_=$`BoZJU9BwI64)_roW6)jl@(2W4Cd_;@vZgc*o&A7tz|&wGd5-NW-v
zv<LBfPS~4*cEa$FW-{J_{VZ5_>u<1*ih{NGns@U(X1qh6An&j`55jl(py0mnu(@7f
z&bhxE=Q_ZTYn~|&m?3_@+KBam-E|>s5DPdj$+#bmll`F%PTRw&gSIU=Pj-Zc{TGOx
zy^gl5?QNlxOb5)RJPMqX_O{_REEzv<@Fkp+3cDlt59^(b--?5OsTh2c=d$RCgv^9b
zl>O%K7I$g;r5$xD&R+9tOSDmpjdZ?*-{OrSZsMdeYFlsUm&`~1i<}d{KQb%^igQ7{
zAbu2<#stS0EGYzjWUnWFycySZ_9I!AeI|a)Y049ff5Y#{L_6af{|(=j{$J!=+VFnp
z0OxqaE1{=Y=IDYC9_9V+M&L;-Chy=2_b?CJmUxg)JP`FbSLtrX{xen|-_Ou5PI|?+
zAA$Z9z(DR-M7|#rA6$>=J?tZF_3*W3jpe-X`bxb~<i9gnFY82oONP~Tm#nMZE$d=C
z4GZPD5<F{Npl<l~yoSxxtlO}a<Ve4h>_5&#;dye-pThn&>@1Sg)wXUYe*cF!#q*+w
zSB%?nK8fET!FfwO$J<%&?g4y;<5{JFXJghA&dKqZZI0smzt<+)V_V5~WE(3Cnjs!>
zuGufyM(yZBprdTgMRH$ea()Rz;xzDJJos06a-JMTh~K~N>*I8M_v_<yd~et|>vWDY
z44CSqr$p_jBkaRh+a9vJVV{n<9)KP&x99_Le$InwUhuzP%Kl6S+`&Vnk9FAZhJ2H6
z)!FUt`SfEK&pGJ6iq9r;UPG+q<pRvnWc;QGVyufb{N9uHg@)gY!S4-a;P*-JJFJYo
zb~cOA&R=+-D}*j1ru2eow|uOvCG@(C%@#N_0k{s@>+D~deF1#%QKoZEvKwoX$*+|P
z#1_DA4mnuLF;~+h>XY*fIyu7nI2UsMtFz&|{f=AK*D2Syk4zgW=>>6>;ymIVxTk#x
zZveyjg)#iDONG0Kb@2Q=@ce}ZE5c6JqZ##_#9I3!!|t1Y8|=QlNu4Q)5BOaPthuyR
zg(_uS=OX)_U{wly{C=zlp)ZU+j))<i{6oy^Er=ukGi**4v_D76rjTo)qt8aHJ#}N+
zEx6W~1Ya$E2Jyx|#E-v>81*+0pZ*SF)6XCt{S(BXfBscH%ylz<4`{CY2RO^#?waFH
zb5*-jPkgL?%yk_0!hHKi<tE1#*xh{JdABo6TcYK=iE6J6as=ZCZ^pUk9qaXL=AQT%
zzi$H{HtZ6$j^#K@7ye4XiLsqksG}P1?S(PK@1+Q(12P?+-AHTI562K|XnCbTAK}J6
z*{qH*)-jjp@4$M7wDyi0@a}=026<QgABdX}7E#cPGSA{2F%^g>e(}T}HLm_l!*8<q
z4p6Sc@8UhTJytjual|h|#=BiRj`$A5G1x}RxN~hoSogx-GVjz1oWhOY7KdD13z;@H
z`7HLJy{*hcywCy1&+!|XST7+{w>zHmv}#W(8{iFdYR9Z9;w@%v6=i?LZ!bQFZAsm-
zee?x=7-8;pgMFZY=M*Rf(d}Ol7LIxk^k&N|9TJ9!VZ4Oj7GeDoE~4FliI6a9KZ-G@
z!v41b@jD$uGV}$EMe63ZxRg~)Tpdx07zFrEEUh@txE$lhZ|3@F2ichTFEU263chSB
zp7&tBZiJlJ1DUo1vfLfWyD-ob`ozG$RN^1$jpu7s;S6NPHe^A@yv%mxK4Rb$Vxr%O
z0H+WWeFibnA0j6D6U0RS88OkH!>4J1PxIf^x47RpIX>7zXCV&oixWq+l|~;@cA*c4
z*$0dTc8$D??8gu-yk7Qa;J>Mxm3<EUzY+K!hP-P5jnK|v3%&;Y$M``D)ZNMx)r#DM
z20P~k*r#vDUe)P16V{1(V}NgKAPzl0S6F%Gb3w0&;ykkM$cfS{%3y%c9qs2CcM4nh
zddLa;v3_<d;ZqO$<AwPU-VR$wYoG0nK~Ab_Ih?;D(5c`owa2l(Z(D5ekr98xw+ir)
zZpgmXXh#KoPlFDzU~_lxivEi^pPBjIWuC+$>sGEu2Cpcsj9|R{jY!DI7U*L($QkJd
z|0rFneP;D*FIq=wFV)AiUnqY3wplA^O}GCQzu(sBSv&b6exr|bi1R)XyViqquv)3Z
z6-1qNc<x3X#!>0G^S4Q>9rw9AL1R|n<0jfERIICzaXK7}hvNU%@^=pq57$m!K>Pa@
ze&G6q`~Y=Gf0<+Dm6#yMDyG>~M+4+1#04=n)$tP}HpsEu$j=y6#NOsP?lJOeh~fFa
zSbO{UD62F7|DMStnVC$I$rC0aAu|amXw`rdLNE`5JPo9@AX<`;$WTCn)Ga8q(8cc#
zl0bMSMk@FXYwVUSc9&gjwTu11Hnc_Y0h<s}aj``!?b!a<#qRnov?{;%=iC!OtNZZn
z?~i%ibD#U%=RW5;=UnG{zI<ObcB*%JS4yttcmv$}l;VGHi_JTkfEj0L%5LD+!FQ37
ze<BAA-=%${rxOife4UmO<6(h6&)5TM@7>FLe@VPG&i6#WuHn0OJcs$BnaKmP*k2&~
z-B*dn=3A7YqYv@!YUu=tZTkb~mwXTCKQzKg=bgU4B1YQ}oOjCql02-ty~JyfY|38W
zC8t>5=QY2;wTt}Azt|9~9s!)!a(*4x+(Qp1>pAO$n7e~#l`Dn}O|~1dQ;?rf#GC>6
zI<lWfRZcN+mq)W7CgVe<^MJU~)tT&rLiWJ~vk&GLDW8XAFH_M+jMHA={CBe6wFhEa
z?_=*Tw#>d@O@xU1%le-kdfM`DTG?};tjAKmfI0QQxV4}&5B!9ESlYYZ5{)~wG07@(
z2F!k-U(T|IpM4Z^5&+<m*&E;3(Ib7N<2Rc(KHj5m+B}wPAA19v^2{FT8OM^Fp0i#1
zqnNdjtNp@%$+ufJZ|sSk_;>b-v8{5o=^bMHjJ~p0ZfykbWWh&&rvGQ<&u?Apy@V~p
zDp!to1$*Xe&fHg5kkf9qGY8xM4(;tUe(u?dIgEb03!P*)xg)wXhu+*x|Iy>+%)qy0
z*cboe>y~@$@8OT|cR)rb9dfKRW+~@G2X@cyv0Vk$*sh_{H}6ne8Z+&!9o#pq3+vWi
z**n;ov3Ejw$QZHjgeQsl&imWRyDPfljQ6NBS8Lz9BC+;?L8P^Abz})K77ATn><Bu6
zM+_Xe5m`W|=16i4*1&G?OoFy3Bo<~$M0mAzP2F048zXD#IyomC@G_6$>oI{ca7FPV
zt-bZoGsKscjCt_=CA@#|)81#jzOQ#TURV2w`prE)u(ujs1R5!4{oU~X$eGr+2BOe9
zr=am%%)2*Ul2M`g<1E^(a$-H*`0@L~<iYs`=NXv?{u;TWxmf3OGNF$%Z_=K(c^LMD
z>`ivdRu8`L!!#Bk?HW58(Qi6K<l7C7dpp4!Hq(x1rj5X%j`t^$`)C|I`gxtD@ww@e
zUs|Enj+pk1@c3)<iiUMHm0vHp=5}ZZa8E2$<Ss=n|0C7udvxeJ>Xh)V;hoANmWBT9
z$;zhOBfd6&J~H5s(y(`=EhAS+*a@7p*R|tXa`9j@m|@xY|Kl|52CI%-Q~0kGl^dHu
z<jQu;A>=MLZn-AN9AWcU(ysS4UiN*XiLIu%opz26>E$M=KI0bd@DL}J{-utS-XL<F
zbxeE!ai{uXXT#D3Oo0xO+?n&==#xIxPJuOK&RA?cCLjX~(6^n5*u-DG+z<5rW{oP|
z$(26r=G*w_<~<=9m+8X@tJn15+SFsQzX|&jupcR(BVs!2jyGqnmYj`PHixx726=zZ
z$UliWtZ&~CW#Xz+UKj7ReNP~OK4hYS7xEgu-P^h*<@E@^dc(6V1NO*O;K?uZP141G
ziVhz5jgHUFJ<D2HhE8Akl#E?qYaq3ka>XC}hMw`?#QY+^Ra>ZNm<4Z~qw%@#cM`2m
zI*p7H`t+sNTT&0OHZ48l|3Pq#Vwb)Ef9Hg5=o{FHEa2Ov$N1L5hkD9WBp23Ibg^U<
zn;xh;tU8k0kk9vE*O`EI&(YKaW#92@OqRwBewkBw(AA6nLt`hNX26uU)?mAQ*O1AS
zUBIV<c>Uma;fiXjSNbDhe1~Mzy#FQSBF28mwgYke_@N%YcgX6E{LrG^A&0T=yUmQD
zI#ssLleWi#J+v{^UgOST@w>8*;IG!#`faP%UblJpv$?~^4kf+|u57)<I>tMbz(03_
z4=vFr89jEcXIn1i_}xRje7DOwrr5smmb~M9w=MBq#pERZNwDwAcO5sy?}Cr3O5WGH
zPO%+~%=JID58#{7rp1c?GjH39+H~=q;Bx#4$%`|@d8hg=-zNG_ygC0UJe3`+)ktWs
zyFBtAsmEUZw)-}3n|HnMC(xP1ma^k*2fhq$o*18-F(TtB>rvlnWJvfZ(qCJzau@Lu
z;9rPa^MKmmTZi?X4G#w|;L9yv#zpjN;<krt-c)`t%{z6DYF@7DPZWP@PvC^<&ot!4
z;n1`0Smc+feb|g85i=l$U2Vb!EbxM8aGe3j-3QBl%r}}$T)AdXC;bcX-yd)GtmZo^
zM~~~7um7S4p(7rVOk6&&!oTsmGO~mZw)E5$>znj%gy6k?1!u}r>hG`*S=*BzY`vVL
zJ6+%25GF1#vFwyrT6JVE>gzH8539b(k5y9KuJN-rr)Y0WXN~?kOKTN5lm1IjDm`=_
zF-yb)D1XI7#igRGfb*!E{CmadeOus<9%av#8@V*PQ^BvE`poCP_%Qe>bP>8w@av9G
z`DV)CHx=Um`fS?3M(p9pPk}t%4X%^ke6qgHS-f0058IKC=gv^x4CTxqhsEXoJy(~f
zzCjP>sjtV@Jg#_KBja-mMufa|Q`w94z<vsTgMm=GqnG*h;<wPf;048HqyLJ*^<rI)
z+F1M(@+dy9oU2(6EPk%87(L9YsRMOEPnXGScD>#se{A`-^?~nszHjJ_eJ559y#&2+
z$j$*5jwa3~i#2r=yJ`8HC(ax11Yji{sPrl(ZYt**JTJ8CTy*5pnPZEpZ>Y`nJ%M~f
zTM$2+`E156;^$@`{txqfXkKV%>ecfv_-pKfA@k%0p0eo}ocChQ`+{B@Cd4FJ1YGl&
z`^Z%6GJt1dJnqLUn>^*}qv;oI49-3K?xb+M?rA?EKXPzh`DODWy<iow9h3)EHd-eC
z?UrcJ0{={*eRR7yHNp=!%m<e#m#yM|?DZ^*`~loGx0o}KTtw}b*?-tq?GjGd(oy%E
zo@-qa-x2!D6XT3j?Ei+=pJb{X^w`<QjqL<`;JMSirHQ&()ZM4PwU%XEFz*c7!hD9^
zv|uWDil>mRKkHTbD$X;m&UiC+HqbVqoUvco+!f5%IZZtm<tDCdSFAjrc^|XkL3~H)
zE8o6=U6@B_Gv&SXv)R0pOfFx=2h)0KF0N6VoHhH{^C^Sh*#01AI&~;h`!iIBwcZHK
zYu8m{&rDnThT6jhjP>VTrZSz~^`Ed6HLxfdc9uG4G&a*WXh(SL{>rAbakAsldx<$V
z^kKptsGt(P6!RDiJjJ*6x_f|!bZDX#N9)_Hr7W!{-y$o2plIlakKn6|jYiWzPRTm?
z?0WC^=E^@S<8F)iU5tOV)hiuJytyDxw9A&(y6t**qKRMZ>48r$=kUbWg}*%CqJO(G
zcV$}>*~?}ve$s|*Sd!ueISjq5dCRwI)px@aVrNNTWfQxEF)XL=t6Bf+<8v+YpBcNI
zd~n3g*jXAK`?2ocaM?p{V{?)C){^?GV+am)?A5m_yRZE&wqZ@+c#{VyRFwMiZ-15a
zIqnP4$!Z@p#pik_qi>r$DEpRwFF1Kr<PED8-<D<kzd<-taU_}3eC}_A1~>DI-^fm#
zAJ|bhux@V0=4=`7MX6V)EAM<!&-FfaYeeT`>l6cqi_Ki$<p=_|Gw_*iwaxchf!`?E
zi|DKZ7Q2xb9OrC$1ln<q+GSsF)w{@}_i60w7uMaY`?gphrZ#AMo@58)y_!GhS4!Ce
z@f;K1s9PkR0Bf|-OTLHS#?$*WAF|n#Z2Gdz9`o=qc!EPN@3(ww(pB@OGeYy$NMGkO
zZ%dfBHt+-artC8gc$xV7Wq(bMx$c@K@Le80G#$+6Xr7hDn@dh=9*!OA4by+o#)CdD
zvWcC~9$8ZVle*>cp~V-~2WY2O!BcS6oDlDCtLEd_X>x$kH?wD&eEEu*rLj?#&+|cE
z;Nxv3#>KpR6rVfv$wZ$O->8A_EW7fJq@!l;iURps-)8Qxovm#0MVKoeb3dJH2G{Ry
zzc%RC^NHA~sh?@PM#`=ypL6AX^+W4X_~f_08kP?nR%vcoiwPdk`M%~IJbz$tEWEdf
z@2tA=UhzKO6Mbgh)7Z%;+Mak%<25i4&BuE^{Q9s1v-OoF#b?zQd%OkxgX7g$1<NaI
z)q#zi!@_2AgvPt!+S)}LPh#9Hl}+h>;U|N8z|WPot#2^!3b^N4a1UpTa1Lw2oGZ{u
zlP=?*yRBsGD-8bW`+frVEKcB_#UIB#z^MWLEYINb)h2I`XijAOndcHXX|a#?z&}00
zOM^H`_(*t(eny3p@RPVx;}ET__Z3&3^URJV-{P1B4l?|*@ttbUtl(ZYSxvy;Oj$<e
zzr<S{9>ok{eYymrvVX`lx=`Wy${%vJpd&1E3cu&i*P6o?prD>JcG8;UGx{bpxLbR*
zwK_S@njXcP9?hDr8a%57OZHKNi624S2K+li&|L+@;*l><FxXY=z<zfWv3SOyPZ=HZ
zVkbu|hmgnThR>S--4LnR?L5O-J`Flf`C%JrZ?fi#JvkpZ^t{qejwEapO-u@GUE(eI
zHxiHgl6|(}2VtLE<nfvB?z<bC5!R^6f@3(#oBex(v=zLM-}m^PFnrn6P^QmaF)IC7
z#q`X0#nQ}O75g*86)$I;Z#y&J-}ftG`-x}yGh$I7@8{h0y(wCK^G0LmC;Q_D6DOvb
zoCcJmBWb{=oO)?T&Ao7XbEfRev2P#N!W_vC6#Gfv4Aai`QrW@ChP**Aga0drpJ1+K
z8|pi*I_ug?cWG|5{>++h$vX)Q1ryogQ_p)&_4q%AcNXb?y5-N;8j(#$dOrS*e0N<D
z@!)AE(Z-C*ri^9OOVWGbIo7tfCDBfZ{>vr~I~0>oX(n|j^A@WNx|BU`;t}dQuGo&?
z0p^CX^!N3gGo_FF2J(Km<IUI?P`2T*z_$iuGmoDoJXEgY40#uJXr0eSZ+YRtuAY3g
zA%FaMOKzU{Lh2RKeh$8Lvhg$go$<|u2ZJ6wPF~pyrW}8H?)NhfP3bPiN?p6Z?!%_P
zz!$jG*mFP(3fZGFA2u9vYJkHy@RRIn$0W+~C_6~mInz!{_WQQ2H@*n8t^F}*tIgWJ
zY}cl<r|kA&4%R)sP$@^CNk?igg5v~#@Zc~9a)Viv%g?(}b~VVh-q#-GesWtZIF7O@
zkq(Eu-3pF$ABFBI2!Jn!REZZj_Cb&Ar3USXqUdUOm^J-rwm!w#4;UL*)@<aP4kH8l
zRWJCa#ChBH*!&K?7jJf*)Y=2D$d`fn&u;;bH$`?i-0CZHHD@5d@VjQ+xA@P|?^Wv4
zhFH+x59-Q>D6nZw;5<0SQx<?XZOMI0FhADa6A*q%$Sl^g_V=p4z+JIzC08|mB*eac
z9GC)!q{F%b?}K(<t`SeI__({(m%e{0SkZUXvx+**btdcIviI|+ja{-Q?3{$_G*^-F
znL9WeVzjkdcvgHScxo2&)CjFGpE)%77UOfV$<!I0<2aGlH8R#3&D=qMpgCRdgVUN`
z{mXpChnO^&&ud-RT8%M5%e(f&W9q+r`t{CM>T0hybDhk+VrE*QoQinI_o3NyN-gmz
z>SLrj`E_)@CE(-NLq)^?BDN+FDSHqauRCWgxeE?zZ))83`bmGYvdM9bvt73F(7eMO
z+((JCy5IC&Wn)-_(AWH`fO&v8<emJVK%6LYZsLDQj(h0LwF^$yJ^?>lo@n>R$|mQy
ztNJrf{ehm@t9GFQ&a;l)jAsnzbQSozG4Q=%Mt*~QGX|c|4(-Ku;&feAth*=1{DK3$
z*b3H4Pipflb3LQDe;l6Wdhut_j(b(#e7|JaO2)Bk>KnDYkqgK16YaG7m4+O{i44$(
ztR7mc(7K2JlA}y~Eh!N@e3!FfgH`rmSq$CXE^og0Helmy9hVu<d}|H`d#yWUD@U{^
zBdaq5iT2t5#04O%ht?~5ZAst+Yu{drBY*qr{J$su$Vh0SD%0-pOuxB4>pT`p#mB@=
z+aodq8b`;Po(^!gXmOW0gRQXPC(6P<BbSzEn!aB@n1fTX+dm@@r(#}V=f7X{5bchJ
zKXlT++li5Xv*--d4zaMvxiX3vs3w0v{4R2&Gf(WZv#K*#es$gWUB2g4hc<)QXpX9f
z#-fb^>exP}EuInDyFAb0c>(dlgY~aK|Hr^V*s<YDIwY+t75lA#ANki~n||@vnfjJ@
zkBP&ajGjL>=x+)y*Oj&ppN<WwMcy?=hlPy4#^k+2p3(K-1BV~cyGB>GC%Xe&yHfY@
zx$Yy(Rg~YC`TvBO=Wg%quPpFPmyJDh%(^M6AYN_0p;ORVi0&I`EI4n^XX+m2Q?}f9
zXx`b=K?6tOyQ}zy!Mud*>nYGAMy3ybkX?ONW|;gT&BSX%j>!4fG_Yalr7dRylEqGZ
z?ojWl&?%j3{yX)YeG!$di2ZHp|C8d{mj*9?hEJmEmqsl&V<FbIa>LYJ!XHMovWxGn
z5YAM4*oI<zpUnDC0zM^os$It9Cf@UC(>Ja0Zemtlm0L@G3yeXrqm9pk@F(M$5^v5h
zXEW`O;T*mMe^##X(3fHO+KdiiKSZ2q@hr#z9o1L$;b6SQ^}hOGj|(}$r|1dnex=Lj
z!j8vq$n<Lu`3}!(o}_d7FXFca@I^lIY4O`Mmxi{74`;q6X<ixET9f}`u4BID7ayRi
zP?5`mpPL^l%8ZGx(>{c!tLN(EI*l=lf1aRmOzlegK=x8)>xE}?|Bg7vVc|Z(U2{jQ
zw*%0{tLym=dupY4Jv~=EdgdHF?XbYj*}}EVQ#tFEd2v+*?gJ-&!*wCvR5BvuN?OL+
z>I|)N>3p374u-F?3W<j>TJ=>o-ZboF=o`t>H}6JQvBT6G&^yVR>mq1<{46c+Yjp*4
zk-05b--v&=gq$JEuk^3rtoo<EX#dN0UogKR3ZARB%D;Nl`2NTyU-B2pxekg}sh`Xo
zHD|8WTEagyuylZRBR>S_O72H;-#K{h$bZ$?R?RDFp?o)IbGKk6{6hIYWY>=%$0l|K
zaUoav(MLjSUuAEB@3E1OCFW5v>C~55$T^9*UpE06Z8>;n0<evkwcnINJinps!dIMs
zHoyE+eEaoIVjstwv+va0<JWc&eo*pAcwNDuq#Zq8l=gS?uJrZtp8&Vy+z3qHKDOpZ
zEoyTMahnA*dk$0Vcjpy?KT>wlMxN?$J&rHA<mu-I^UatsAjewzTJ7V+V0>cfsoE!(
zzEP{VakDGQ0V3Fn4}Rf(BLlIpXUr*kh;`MS(ns7v;oB_aw+-q$uneMm>0b(-W&8%`
zde?Ns#;(Ek1iy6UIySIO_+1XlJ+#ja?$TbzW<g`sTFZ3%yMcpZPbg0LUhMvM`Su|T
zpIac^)g<>%h!w#cmt}Lt?RFJZ=BIGR4>NK?;t40&u>SNtZtr7k<(_Bk$MEla4w~Xw
zXsM&nQse@n57_ADPO#~P5v%HdTqAo`aGJqe21f$hIA`oR`Qq@*=p67zb=h;Cir@Q?
zr_CPHd*4VMr*|pa%DfHJ-r?RaxIsrZ>YUa&YwExgChGfNdBD9ubs5VXvmRQ8q@$N>
z2(7|rjE$scum0Y>Ao0zZbYI8^)KA4QQ@+w0q2G5K9wZ4l^RuZv__0~Yx^L|0Ug#G+
z99U?+>q0K+RDJb@cSZ+(N1u{u3;qSV!N<FCy{oe=^y}2#L>mD^CtRVYSOdw=NJhbX
zG5S53wGixB^X>TH{sVuHfDUS4pX^e<>8sn=&>L7F$E~tbcqeN^Y<QpW6zwGZ39do*
zRSO+aJG%b0T4;t^;)OzooTBWF+5%)Td(}6+i%-QKeJ|cra2TBP1AYhjFPpcC{Qn{U
zMbl0a{i8M1TAAq&RcHD|hrQC4mhXlRL%yMN!kM7E6jPD;8~UccseVVkp6QRQ%0wQP
zKTUc+Ge1}n@xOs}TW<PZ<g*r5C!aCnEb#IEHSh5}wz#)evODEE0@o%nPqkU($YIQZ
z9<4Pq2k@<-H{1uLr%{>Re%gNC?>ly=XQs-KixZcV+*aM+I#agxN0-WOsm1TrF^qk#
zei<G>^`O;8*R#%{(<UH4nZ~&{3!Elf_4zgqsznx&tMw8TjuB5|^TyXeyU|yCD5pZF
zF`qX5+d}Lq%6aFlsl3Bl2-<Y;(8=V)NQyV-98p`~vJv#ZXX0yh*HLGbd@@8s3x;<4
zL(ZA8({|{f<s19^y7&18e375H*T94}Dx1i?*_><QvFbb62qvesdy`FD^ppKM*cLdH
zckH%^Uske+_WlTNeM04|nJk?LQ%}`u9kEU`Z91)_@Gxs8DBk6{L;tc<XRVpzf8K8L
z2CzOpE{kf(nPJnXL+gNb06BiKXg_dw0D8m3>taqmk}V86W?LuLlx5<H@Z1WWavz8{
zWgS;cq1J^C^k7`q*l~nx-0Hr-y<ahZ6yJw-R?22nIP0s=I^^e~7)P@A6<w4Ct}4jP
z_wU+Z1up0;GH2(!l0B7G9wRF;ag*AIeu&QMe;c@Sj)dCK>M7RzLRW$CCpyqKEAu^(
zHe!4#e}c-H2gy?ne&zWdo`=n|!843Lu!gc&Lj_tx<k@hP{*E(<_Qw9+gUrS=j`ujH
zocN#luUUVF+)p-MZc?2`b~};XXP;BMoJsrDwvCsQ;CZlFp>4*fv$f#GLSjQphxbNJ
zmOZ}WHyNKj_sDhWtLdloowOIF-I_Gcg!o-U@zt)$!d_Kp_~pJ<{=6n}hX1#|mHeN4
zD~3LGk^QYg*6e@&EzTI*Cmwwwb7<r((6owAs~>Y1jOe_(GR~rOzP*n%*aV%jOF4J>
z-qy6qiTMGC!T-LM*6vECZDV&Md?xq`{}Wg4M&Lm{&VppMNj!lD!Gd>;zK6Xoo!V9P
zGmldron8Lk%VYVZUzVX+Y8=Ezd}RHZx~DZ)roVhsbB#`QGHo2F9H_}+Ubhl^rEqXw
z8MEe8>sEG)n%m%ug+1f2t3Z!0T`TiM9p<Wlb(PS!*Ep`gqu`vG3-mXFiEx$1dS%`-
zX4rEv#LWAP-O}wqf54}qgP8=3yXbc}{}n?pBHBna^#6bT%isT>^pCZYo^JOq_oO`^
z|C9bf`#uZ3auj;yr_d`ufnNCp-B$T*?UTX*(87}QOa>=Kg;NqXviA{R@h$ZB0rtzF
z-exrLt^(c@fcLaPdbFMxnY3S^xI*BJEOQ>Pp7^feL81E{^yN?JJm5K5w4UI}`Bq-}
z^W+%xl>^zrOlT~R-ltvZPbcDk@Gdd!-y^2|MePg5RRA4t><sItSW#pnjfMx4U$o?$
ztW&M&S1%Pk#<N+MpHXK1Ut<lYJm~#3c9~5>WQ&!{JCh`5G<#ug+B2#L9{g~_`alQ$
zi-H9Q;}1QpdiEL|--!MCyTnR+kC@&UIX6Gk2eI<!;}tW#F+MlV@QVq*@@S&Wqwg6X
z@}|6_=orH27bJI^hI}{CCvT_fc$%Fqc)C?Qi>ObYTa5q7)2fH>$lvQ+Cr=JIFX@c>
z8Ci`#+3nzMBZKwkYoDp_g1Nq(fXB7$bBEv&S!}K1S&Xx{Dc<DTDcXqo6MY|KyDDEB
z-zk4m${Ls>;Sl1qSZjs<@tcqgKrsnBrk)<(NnFTsY$8jD&o>S}s$*MpM0sSP>>jNi
z_LDVr;8x<+xU7Zbs-yl+u9LX#;Tq=p1lI^QlEf-C_9g!Mfm_$spRNwp4^)rh8^p`-
zb__gRxS70P`K|D6uRfOVCsvss`t#8fYdjII+bU0u#}2`Lm@~Yy@(unsrOW1ND(lSl
zW6(IBCk|Di)nm@_G2&~zt@t_JEB^()vjbSjo*(>}a}NLKM=9Gy-xodi{oa|-o35#+
zZ+#E@xr>a;_sGM<{#=nCKJiqpg{&WY*wUE;9VHi354(8$c1jy0KjWRPz~;jfD@wj)
z+of5B&kcnA)N@KsA)OF)I-fgD88$((A*tM8xgtxL8_D&UpLmm_#pq0m>VA=4jbC?;
zl@eKFtwR5n(q0nWrFqx<YwodS=v92Alf*bPbvm;1#Y2cU2cJqNjSRa;^c1q4=ty#I
z7~Vu}uz%k|f9Y45U?CXbGj`6vq@>_|;lxO5V$HA~S{GU!@g|E5JCcdiok}mV*G*!*
z7@02LDz?A%m;b1>g3q9_AC^pL`(rgL1xI9KqPxMn!_rxclZcf!PV3GdPZ~Uv<H=VZ
zEBcYfL!Fk)w(R^Z?}W#RW=zy+rOuL9E*1S%qE24shU|QMFF2~u`5&~)i_e0G@`2yC
z!1dNZci|sHIeu&5{Z=(Gfi8t|SqI1oT31*jH*5-&YfWk0+wE9S0GkG4KA+W|LY{y2
z%35=uGiy!sYbyANzH6P?G)XSF-?y*zW9uvPMgv>y-YDN?<3%?*e~*!;F6~G6m<6rV
zhwjmj4)SNnelo2hV%twZSLsJrISoDKEbP0&=qVdn*Yc}hUU{rCOLGDISA#p(f;%^Y
zTRT~6+Q(Cto-lO$G<2G>C)*uwDtdzQ8G}5g8E112wlBU5$b$<EuQDRFRaf|(`p|W$
zQQsMV_qxt7d$g`#)7|0LX{%DBV@KmBI?LmBX9?$BgPgtY(yPnew*ixh@c&tD(daS7
zJ=k<h-3Gm{cLw2F|KqX9qS6EP^sSz8*E42xm{r8npHR;@iG?_;o-yJ#G#`Ft33zvT
z{io$AEGoXY^5>QCE6y7g@A`%2o&#JHx+E*2@8}!Ij?Vm<-`T3~pr>mjM+JDZFgSKP
zwpl&??u^OVJK)va8O_-pnVX1D`Ca01GX9KKYbP@0uy`TQXzUXdw+T3;b+*QQ?`2Du
z(2V?9@SBD&X;H?V*}M7{p%d0OkX>a=%nqYR8%?{eyPK_ma<NJWt7nuuC>wrHGyE_y
zzT<PVN2vVE&EVwjp%)XnoC_Nsj2ZhK{L-kC^``1<?hFU0HzvI(@Xc+#u<S+Nof+T5
z_VWy~?XHFx=YV+iCVO6U9zaKky=#%t@oLR}^(BXeU)OThpog<@H|t7ttYT&=cBphv
zL2_SovK~j_r?gRY3cgBOmsOngqW31C2U*QpoQ59t8TQOfXp;_oQ)gSKFeMsqE@b~W
z4^U>)Ci)iNVJ#G}7J{sWQT1x~6ZFa#Us9}AVn8<yc}sXSy3nfaZ#Hqst^Q^clib<g
zY~qo-`m0U3yT96$d;6<RxvzhXDbMd;W6BHr*O>C6{vuQE?=P}roA(#lan1WaP#jll
zWZwsh(`sGc_krTLTA#CH!dd}49;{Vr$Ah(o_jjB6<^A2J{+Rx5Q$J+KgSE!l@nEg-
ziU;evCp$$kQfaUHl{WTPySES@$_#S*_`Qp~IliUY!|)#*5?yTF6usNJHu@E7EIymY
z*DCY&EAh>VKW2Qb<m;6AB|E-3YxafAjfuLOuf#FT+-ld&+>XBz{#Kc5@wfV>^Got+
z%3P;s#7tLtd&0L#c?352b5-VkWtV&y3f-@a)b%#^|LM?m4EQw?T471w27haX75|Sl
zRYRQfn0LLYdt%dbffpt|Tz5)w6({bj`xp6L$^VP8H{~Zb@tbwO&^LHCVE2a{b`RI}
ztA3*1$-!I()On4vJJ#ktmZ@(mmNfnFuKn6y|C{c?QL{OJ-p`EryUm)G+!3Ea;ZD)Y
z!k?m@S)1bw&TKANBEE!J9$|Zpx^AkMZ6WpH>pSYHUoYHRlvkr1lbltt%=YYE#3k?(
zKe8imAN<gQ^i{&COV?R}$StY+c03kN36&r}3#RVF&MZ4)x&L8rdqn+gTDlf``C)4$
z<!<6Wj08>{zL4Vle!2{EQ+xCq(^u4;da!NYcD>t$KOgPfzNsxRJ!3vGVtf5VGKf>K
z>Ew^TsOOtb1_~o5t@7#E_-=aYN7tzg`{@asvu87+{w`qAEVuxRIeZhDJ7)p5Gm$EM
z5eh9=S8SQ&3D|xRCq%MreyWcQIi@^f_!#fz-P`dqew<ublkhcuf-^0QzwwitX$|-;
zKf{?e6W`+_*x!5}ALi#d*COx$r-INk71Pu4Ya=h#sN|9G2L0s5dUq*4k4y1+WR7HS
z<6&O1kWW`-mfQY#nPc$#8S76P%s-eJO8DeqXDJ;UzUN0c!;i$)T=$zWYmj`(w|y*D
z0raV26=Y0jE%QzWK1#Cj3oo{&4BGf*eEuq*|1VH)g)@?Yjk@Y(kY8Iq(W*;dJLS`u
zG3l!MQ%wD-=n+hP-!~F_iF&>@SL*rxm#No+-odVS-&OTiTvhLDrd~7p$Q1OE$$bOl
z515_p?i(O~!0c>q-vD_7W)t7%`=;FFwcy_5waA!~oxy(>XIBb34*NUzT=ku=T=kv5
zAol@z;pD4`+`+DUx4&QzCd6c<%#MQ)USW;Lf4x!9fbTfLMR4GM^;LChx0jS2puFC*
z)O+W|TdXn2ajVGD#`!~*k(K0OYt7E+&$II=^yk?*6#Db*914B!n{un~eaaV;3t=T=
zdfz;AJ@srVwpPi>#G+UJph^At<d!Z#A2io_C$vYC;h_rPC!2GXPpql$b@O~Y&yD?C
z;`u^kme1?C#-K529E0Qi<X9NT>}_8TjO)+meAYSJoHNUAn|skl%{in0M{O-EEjMl9
zH;%mWnagcK7jSROb!2-Wd#t(Hd-a@iy}{4{mSdmpw_6XK)t*ehA<)G+y&8IKIsDIB
z=sV*J$QivGI|gI_L!3_6<#}bE2gf-$uf*BRlRc|<B|faIk?<mG?BJTYT=%N=;$7&R
z1kA<&r}9t0Ei||W+O`Dtc^Bc6+Ge?Ru4VN2r-+xK?~KX+p?Fs0kBZHNA38J=_mm&k
zn4swucQ#^ZE@YwbLmjLic#Qm-%5`-|Zu>s5H(b^-^q?2<MRM{#iY%f=-$3WoNdB(f
z_^LmSulf`Csy~UZ`ZM^dAAzQM9(tx9ULP4b{_EhJ&-Syn^LL5HFtI!4maHtBfXw~2
zx<88kjn6H)b6_{}ZQ{p>z8+YI?_g7g8+wL0bSI<tA9cfg>^XwgABdR}AWjN-Q$5%p
z)inzDkXNO~%@tbiykdbxc^;*nk*(Tx+{6^Ktj#$A;<R{kDR*Y9woFVK#;W|&M(!ei
zM$Uf8);7kcI3*KS4CsKqBYR};!g1KRDu$?$|8F$ylgoJ!mn+5uczU>r5rHp%<x90s
zgO4vtegWO*AP%1NW}e^}aQ3K-9jvRYN}Kj8Fgl}OXIy~(3kH4)Py6ikvPl|+9w~+O
zIs+VT;{2&kibD}^%E1=qI>|&PA@?|>Sn=Cq;U4HY+oq*Cc_%Q8iVrjM_<tQ|pmweJ
zrFfH5dTVU<YdfK%{GvYv3x2WIR{Uz_6|Y<vwF1ZuWn&x=f5AJW%{#|yk#$1P6AM-E
z@qhS&munBJ9q6%G>sR!C(fi0a@Oi9lnK;1tS7fcgrmTD-_DP|=*o0ya2|sE0?6#iT
z87kl2QJXyRXSIK~_3avCKeshpWA-uoKy`Pr$ETw&m|*rpN%4NcN^_-HXq7+DVm&!_
zF(0$|Rsgs!K}Mo;dkQ#U0`n!>yb-!w^I+`%Sr==4qXJ#Afk0u!D9gFAy6)Gy4|#0)
zk;V$IKMnm&R(svGk~tQ%^^W+R?qw^fm>r9c){WqMvPlS5{^Vxa?<>Y<sX3pq^8x2n
z+4T!gvKLd3Bl#zuzB!Al^m%Ua%aw23oP)1&B;Ht(Oqut^?r$BNIB@f06JNRc@vCtN
zWq+f60xlaZoW%3J!oTVRIIfBqI_dh>vF>l9JLek@4%anjdyMPA%`@oRcQ>sWet`d9
z#Ma>s{_o;H@N)S1ze99R!WJSXm_qB={j19CfXBp_4>?<7adGH}nZ1k;+Xh=_$hSK~
z2g$!8+lFQ<$a`MB-+C>wu7R8HB*ve6;tMy=4&`LVPOK=5Pd$2bi)p{iiXf9H?E5~k
z{Z=5O0CwCL)eqb}{{%TaD0816e+O4D?_I!0Zy5fwDa1Ye7G*wiA(fxlIrNBRQLIJf
zf*8X48NQXA>rvL?itKE`MexFJqJwMJ)^Kp7zFkpWcZB+ivkQ-P6S)iF%UC1uShL`<
zWXClh9_v$l6mE848y6Ow-A7kRMmweNMdh%*CU5bJ((#r0S)Y-5zKOk674j`>&}PcB
z$d#tPXidEGtY;~8vijk9p*s(|PkW27>llX}X3q`t{Z{*wSg-ISV|EEgYTkt>jo(a8
z-2;*vf&(AqIvbfz9Q;@co(xbnynRwYu>9+-Qv!-jJ&a!j9%?vc&glKrV_)*kWcVxW
z&B>(z9BBJ@^5dk9--Qn)tydkc9b8MfZUV3FR2$H|J6WUhuWaX<!F459Ki5TEv$@{M
zHJ9sTu0yz1a78X#Jc6sye}GSrhqGVVU#z291_vHxk75fFZ!EZ1u(kJ0?pdA}aMp#5
z{vBPq%@Z?U)Q9i5%65c#8Qgmb81p^n0{oS<FBkvR#NcJ$<{Z`D#2358-iLKF2KQjR
zF(3U_W`t+hI2zv)?d5>l7@Rk?LmM@A8?`4ZU%z=i>%CfSkk{!MbROf(8Q7RF`JR0J
z_NdP`o(dSdP{}*xPsX^#YsuC_XH+NSH1Qz8z58|6BSRm_yA5iaZ}gxuxk&%csC~ne
zpzDJtIjHk%hj<d`hkDupF3L3_c^tZ5SA}u}4Du+zep%(uQ*78Due})j-c1hVYL1Hi
zmKA&m+aF{N6<adT2fxhRAakf#YCRad>t~n1s|6L)YkwRh4m@}?STX8G;ns-c9MUOk
zor52ZP9~HL{!BrZC%sHRwoQGtZ*j)G%X#`9<DC$1%-ai2p9o*GJl(>Vr^WxIq1$3#
z9vAXEqM>cXXFmh}@jJpcotL&6+EV9?B^wL!SLc&6<x%}!!@v2rze@ILeBkQQ#{Rm+
zpJlf<f0zBf_oCkCe^b2CcP_LYc{Oq);P?e#^j&lV&xATTmyImX`#8GnJE*sZI+6qF
z9r?a|u`4$Ej?NDP-_<@!*iXLU%`m+63tQlI7d%(H8r=l?e&CVsxD8n<HZ5sObkE%O
znES@GbL6+k;693Nh?6}Kqp#R$mz;xVRQ%+Ty!*QSuJ=9r-O^*aSN*oPOGlCmnz#_`
zzf|^NbB1KH*h|$-reC+Qmo3!|FT<|32stmZ6P~X{4xKfyt?<Ih)9Lx>E&{E06DM>H
z@y5ITnen;#75y(4{t+EQ7Wln}u})&1c5~*>*1BWucXsv$yvXKegHJaC%Q?V?b$JtO
z-i`mbh0VuJwk?wQ;^tiB&sM|M7XlsV6592h@G4?aKk6?arba>iqsZ-m{b0Fd@Ak8l
zi{@EIf)<{KZ3^^-+SqcYHd*7xKlZ|FUFrGgiSQjCdVcffTUQDWf(dl(Dd^g_pljcS
zu6+->_9C)W2e|#eB|k%lg)G%K*@kCLG4Pab6kX1EbfdN|XYl)!6$9HzSKW`|K0J8O
z{|NnV8N3hWxE15TwIgO?zPwO>>ek2W-?;S&Y*}|}?E;V4tlehhGsMI7ZsWYObw9|~
zjPB<ZbU&HaqXS)@FL2!^*lC`bKWwM4EuNdZKfIi{F=h@6o;Gv%GIN#1T#*M&^NCD-
zt>#nnra6tzOKxQDq930-{}IjI;M|ybxvS*q&z`fS&X3PovC*d^-`c5pi_cAZl5xt0
zQGT(rg$Lzl0e`U@+#&l#<rI=GT=J&^<V?bec1&~m8qdw0OrOh<kv8buF1tlC($voz
z`#IhxF32&?=}*K<^^cOTfNybjVWisa$7e;fYLRH?)+)Cj{i54<H+cYKe&y2&82-`i
zoP%x$U)`I~Cs_wgJb+f@PSNZg^bZ?*c!=hVjnXMW8<;rHrVpR|c1hli#z%uW)mYiz
zt7NOf*d()&P4Yy>mqnh6&&B7up2X%sb_bLDPMcU69dPbJ-U-C-vSZ5gUwdT|zRrr<
z-bCN?^<8X<lqV7!2G;zi*q?ysi$dMlskJ4Yq3`0G=g<~53AP^Wxxek*k@!|QZQrQ2
zd9Jfd-!k!P$;mTW?c*0(p>JK)j+1s4=^f_vL)GDX!$RHI0hSP_ug!TzW0y@O<@wC>
ze9g1I4=k4e1I_hv(a*qSZKYx{;X?#mR<q~)8PV*0|JXV2oUXF1WZoul9tY!%9>wLC
zzQ3w0=915w?B;{yLv_Bgz`a0mgz@bs{)pf79~hPMJj-4)j(^4fN$02hvRgy0x=YyJ
z&aml>q&>Vh>sa@Xj@CmP$PO0VvaaLMk0xtw;A4-v7a2T%8*sE`@}Ef`vhG&-XTx7M
zUyi4>#@~cIYxZkR$aJCC(jIj-gXhcT9}BIhJZfM1Rd2xf@!%I=J%!E$ILas5Z^O}P
zO*~!m80YZ>c+N0q@icfFC+F}i_DuvBmjmO*%I9k3BfA7xPd2a~jIZz#=kklt!HJj(
z3+7le7c^V9#~V|2f;&2wlD|tl7VQKmyI1cncmdg^_`1`zvdQb@Ke-ob=U1Mt&BE8j
z+%JB<b}zENXX5i5#fwkZK8YQp(OJ+=cIByB*+(SNUOIl`Bh;?$c~;7^(ecLQH);0>
z<=6@#2hF#<l+U7kzv1tAPuGbr)@Dt7sWzv7>Civ+ZyB1(I69$MHdek?dxY2qNsMVX
zYZHAXXHj!1_J@xAt@i}o(Jj_EbN+Q()mppS`=Mg?e>Z2}9&#a_=9@a(?7b~N-z0OM
z-8Hm=I7iA$-N4+V7w7DkjLGO?upb1LoF`TA3KQTJrok)B0v|Ni16%gtl6qjy8d`1J
zkJLK0M(f&TV~pJ{d}M8>$qj5`E2zK8?;5`!Y(>=ml{rXSRQXa}USbZWBX96)9t3Z-
zfxH;~lHfjU)Z!Nf_kTVd%5hog9scI@E&c|ryM?=~>UBARF!7w52X@)MmV%M#OS9`D
zcBis`zEj`Dme9n0OK<bfNh2<s>B}|Fmcdxvg6|;Q#{S-N+OT#lr;Tvg82oWHcU7Kc
z@kR%>Ms-PwZ6o+1FZm_UY}rAhLoo~*7ra=DF4=mD@%&O_;lG>zzveo2!B1-6<Nv>F
z+=;cNu`<r0My)UQ^Le#9@pSE1C96iRaPOeJo3iENjYMZne5Lk&Vs$+f>9PXwp0UVI
zaA+=b$bA0xouvOT=WQPM#pcYV%u%hf?LC3(7j|2UE8m1Z0RL%!!@`HH@WMUBD&)Hc
z2VS*a2G3Gr1RY0Tx9ZB8$y}~AgMH>s!BTrzv4bjq?!4@43ys~tT2`JM*;*T4TJ#3*
zK`&oK2kzjkzLMAdllk27w*?0-sh@#<W(K*Q#}kjWF*8pzxcCC%4h2FtWCl9U1-?bz
zuX6aA_l@l<?L#A7MBnT{-~4}-za2j&2Q*7DV;{$vT+aNCOW0H$i#KC;Q043acO?@i
zO|-2AJ!x_qnDcZoF++R&-K=w?$3_42%|X1kAr>}yL~Xj-aY8aMa%}{ZYlGAQ=yM$3
z1^>&&*Gu~=)sATkoh~+Qm)kkkJ)sG`XSr}VHg4-#S09icp%h=73%r{fZ*=cbo+tX9
zVfU|SBkS<-0J+FQL2uYMJkZTrdxCjG=5)=I;H1Z)9XeQ7vPlvT;zedAn`Mi&niD<^
zKI0ss9<g+4iA^K>Yu9nE#m`p<zKpCGS@SiGYVTuX-*u(Wnj`6~Mbk5P<?+VUJL#)y
zux-ww3sJtKHsHa>RK6}>zmhlAw%xZnol~+$anL8?-5A{MQ~%VjP>~ZKnp)ZS`Po<W
zFKH|H#mBmdd-F=|UU;fWc0Y$0x@Jo(aQ%PM*D%pxi@+UQ=_`Jf$saw~wKmHB`$y5^
z+qXT|qPFP6FFBi(>%mK&k<x#7fb*OyKT9%Q`gYF1c<!(n3kGU`YmejR9>y^xF^;3O
zLEkNn<2lB0)b<6ceO~7R<=MdaI5sfvGnRhjj4P+UP@4x_-jZMM^F;~2TFq;r;}6=O
z#9#Q3JvxcLk5C^UAwG%Xl5pRyxFm~TsO{HxXmf_Xu{~CEd@68DclmSEQ_%CEhn3uI
zb63rtv<`1kMT(W3?)K*ri>n7+=1<qhYQAb<*-ISHyS?~0AO51bCpYi5eWh>sUhwBV
z)Ki`f{c-)a1Er7pUPia@1I6*QyqkVh+B25?!s%XejOAF_^zTu###n2bLtvDnIkeZ8
zm9F)5tozV*d+x{wdbbszpGFr&U-8dF@7LjVyNSyyc`n~B+Wt`K??g+fZD2YG+Z<vX
zD;}Z$wao>$+CHk;4t)z4XD|mR^-cAGyrcz`7au$Q|9r>HvEg$q<d)=ihmKkLliC8c
zEjwP}A#}T8Xa#KFkpUbrbi=gp8t~Ejz$)w@r<gt3;orr(&D$;FQ4k}aaUL~uonz(m
zt>?LaZt?4y=V3vv%=7rYdKda91zO0Dd|tZVgYZPww!aNz`(E<rt^Zcd*9Awx6F%fP
zxO_^c<?qry1$T_dASM@W9~Tdc?xbMBG5ebfp09n*#D{vJ_W5{A@r1=M>w8s4WxJv=
zbe#UHt)bKWf1~yn@kT2h*+UTD7|}0T_+HF1=i*$&i*xp1<1uST8!#}kvVzHi*Y+LP
zzQLSJHrKPR2z&x9$-Q#l{B+ZbalfyG?u#rVKabB}u>ErZe@2Ese_hVC;}mPqZaZ%)
zK5TAuJ$uz2V=btxuKT_4Lsu*&f70tW3@6WmXrZ}z74!qyVo8Oeee$8j3oKW>@tPyr
zcUoKQwP}g6-70Gjn3x3UmP(D!YrHxANtLr_BrCyQCEeu0X%7UYr-%&ZTWGiNBhJBw
zQtN8K;Gd)1f2cL2b;A`J-cpRFFKQhVD`O*N#iDhQ_aCKPIpC(FBafjk$G<bbQSut>
znRbbHMGk>pEVrF~Jd)gT&W&H3_q5LIt-UpWB^cV{&AAhQ#)Iq!<PZ1~C;H9*`TB3<
zkI;Zd#?$D3Ty{+9BiXw}*4yBV#(Sn&QEU~-4``jaCt?}9A#9I`x8dA*&vf!QkAY_<
zZ>`~{@vF!FG>Yuwu-e%WtNj`4(58##W;6du%fKo7)mPT4=q%aPPD|V2^=CN!MQPhi
z?A{-)+a5IhCUEdF&RLv&USMEtv+BNs$y-r-IOta2Is?X@;dh;uxxP)z<`dKxU5>ma
z)5Bcu01qj~+ppMz%T<o<Ph)vU`@HNS^}Tj={qB?qb_HvZyJ44v&TyV<KeD;q8)CCB
zX`LE4vMg9*$N1r&?Kg5r^?}%SCV!F*U(YjF;OlwZ{5QVgo{cuFa~5fRX1M&sIj_64
zVf|yWqiPiF^*jJ9L&21pzUsi*c%T;<7WvwlPvDfyx|+`Y5&ORQtiA5sk0<VrYTfaC
zZQ_0}_fvp<1RSE@G=9_h&0@aq1t&z=BSqUE98+QZO$v~QHfEXi&-swpC&<s5lG#&P
zADTV&vg!w8!e5bbnIpGbqYq;TI0g8cTp{3<0>v<9-@RwfI|Mqwd5B;~JLuXoCfj|>
zGUuT71hBmg+?B`A;8Cu{3w|nmn!B3$$$7Ep!27h3Yxd9OGn?~>`d8HrYCmZ&#+y^t
z3XU6M<?XGfGZQk4<Q>FUone35-=g{4-d1kTpLF~swnJj*toW31{LzeKQ%rd2HnRpw
zhT*%Fd@Ru)bN*<I^d(RIRlkVM;ZWbU$8LVW?o+av+YPasThh@>C)zoQZy>V&%o_3A
zqJvEiPvq(1QQ`Mx|2_#m)&m`E=km1qcjbcH0e!hlYdgWeZ^5Sh0r>Z@@R^Z&{1>?_
zBY$f}Bj2%YiX%@DYil1l&4ddY-H%8165l9p?-_82vFjb&Gn}PS)=(2Pi6vUGb&1u=
z8M>?$+wu5ah0Bqj#*BTjWTfB%;Re(0JZq0~f!lGuboLA*M+s-2#;yFk^7%mK&)8y&
zy#U@*GNB-AW;O6oUiK*CjU~ogi(H!aGED41Gv3eeJ&&wsoEwm7KyQcJh^^X~@|OA|
zI!a^E_^^R$mYfyZcv;^;bn}c4*;l#78eL@F$GDC#zE;lX`@&x*zgR*}7+Hm!u_}f5
zaK*+i7&-2JjCoRcHQ%{w$P<xoAltp)en+s-w~!I<hBq~9^eR|9@F%da*MRM(e;F3O
z`GU_UV1W;WyB%1Jygc`x1q*x5ZCLCTEbQEByq9EPfghJ(0UYiF1`TF@fy3Z<1rNsC
z157mEt@ur4F;}|<J2P&^Z;w5Hz8U-F``pgYhDFg5&Bv#~0{ds#dMSRt&f;Z*YXn~K
z33$OL!Ozd2J2`^x<az86`r(I9VT14%`21b)_(k@O{7>XhQedwo?~+iV(*k}6LizY&
z8hNRMxGH01`|(6&Rr>4jfKMVPe+D`E5#;30BPZ|2{sSE-vUA$S{^LF5vlo$TIFM_2
zkZWA=LmHZl-~8CMT5FYWjO~m!I>{|Cwv%<cOl8Ob-ql!v@0Y+CUcsCgxha}A{>zTP
zlyYZuB`_yeI%OH$ThKu6Nk6oP!{5j!$K-?M|44q<Q+G_?PGdJ3;(jc@27XO_*sAhu
zX5UU@lj`roW>xx4zTv0*_P(9QMm3wfq8Fkotk3c6Cd%*Ne>4B@;@i3L7SDUW1=&N6
zo$gsC7zOXU$q}55jM;}?CIvh3jJo7drW0M1wX9-kN-y+D5B7NnL)nhr`qN`O*R7B3
z7=-Pg1y|!sHg+xf9g5&Jr~7vJQ+*Hl-SN386J&eAe2(MVpmP*DXdgCPi-hY%n}TP~
zVoMZ|?R30hnBl8@f01p$!%4nfZggGHQ<Dw>-^JDnbQ8;>t=8Az<3|g(Ay>(#UeuiP
zl0`w&nDs6hnBrsIl`VZ@6L{LT^+%qDtr+FGr#RDzeOUWAXF#HEhWvm`UGh{u&iiu=
zUO=7_pEtyu5%hPb`u$(yMc1@X!HZ83v+-WmF|l0|=fEe|sC+P4yMKZg8_0uV<3*3w
z<$n(^`a3@xFQ!5tX7@`j{&BqMTm9L%Aa~ZM;es3HBOB+O2vu9>B_|5y1Qc^B$iA++
zq4nlVp<qUw{XN0t1wR2U=m!_{feU^HE_fMSa2(muOW=bS!3QsZ4~YBEeCdplf92?<
zYaPglyw5N0a>ONzB8Ii`DUS?KZH2y>ie4cST32^b_N$@cDUr6-fgbva-}tcOz)5k8
z*TG91$By)3AetErtx1h*+@Y^0onh}636{u8t5i3%CM6#FIywM7mtH{rH}iZZ{*G)L
zn76f&v#G7oYRA7y|E&EMyr16*{{K6BZ*SejP`1-ou|G9XG2IoYSekUO;^ov;760Z6
z@94SiV%&0Gk6-?hpoI*R-1I@tUt%h^A}7TszwTm%8(RnB09TB{Uf0W6olZ>9H0KwB
zzq`q?W%bBSsv9`rbOp1aL$MLkJFG{^)&mvp)C;r|3;F7jpqFN-Pg_=x+822zwKaG<
z{kxBM+UhO_V=36_x(@{RSyshT_o^+suiwR-S%z*@Uq+^12>k9Q6JNx%?+sX?U%SuH
zb~$|;85+e|mTtwitiBn24svDY6#UT-UM8L$c$hf_FTV#~zKFcpfxOv+yg3Vba{+ic
z2wolqULIY~`lx4J49c6?<0Cge6xb9#WPJ{PLOkJp@wtW5ge%ZF$QLRSZ^_!DJ?btR
zGm+<G;!VX5aQ=-XzE?TVR|#JbTjKFhVX~Vz6XY%^PS>1(Pnvw~+4yE<P9VS04uhxh
zMa1XZO?$kPKMfp&t)j!kvH^#6!%ug=_E^a!{&$fRufvqZkll5Y3rWv+`Mbs6O1{io
z^ni0CA+-76+0utkTeXtmN=7633^EO5frbYfj~~xu=0|vOCf5XBbbgOMj2^^`A%_+2
zI8wJ#?SOOc)mRLz+K{_fdyum}Cf+hs=*G@#=!fY4qToSj2c(ZhPW&i_o&^7hPU37s
zc8d&{vi*s&Rp7TH(2UkYIoK)?pB-6qzHq4U=^(D=yX3rbK5KtB=V|e^%1xEH-wB?8
z#=U&MlKWMWufu-|pNgky%<t5mqD-`~Y>QO3#Oz66hd*0{x{8G&J5fCwt!Ig{{6v|t
z_Z3dUNB@b4{NC>J9x-?*7M1*jHBhip@O+`DCv4y;{605tH$IncQ@^>O%Di7vJVUtK
z;D)<=XEmPb*f0=h?&5S}JPDrhrhIsHw{;JByRX14zeRAPY#Fd6eiwVUDZe67wu+bp
zM|KF7zXX=(`LU<S*BN7ADOjjK_)O5B>&^EXhs@Xare9;jB~ezTGGI||-{-wMc<-%t
z&PkNLXWoOaF!yt_-;!O(^dhT+`yCn&-waI8SKgtLx9szE=)GD?_=4Y?xSyf>2(-$&
z4*JQMZTxsmjrKY=AX^37W8HPHsa*eAH|gj*b>=h}*v-v)Pw>XRcVxUN`5D%Y?6Ym2
zz9DlF_YDh<*Gd=JJnf>VTs%GN58Y?&@&!k0*J5wE4*mToba$)S*QLrUvE?7?9<bpw
z?5w`4eZ~AY8QI;ulymC)mcF_-RR%8Kqj$_0)-wjytv!Yz#r7D|lK*rJ1??XngMSI@
zOk>E$msatQTzucqZt(_U+R^6)FD1rs$}`T~(|3LM@>q^*97)8vsa?J0KwZ1uXV08d
zUnYSQ)^!E^;MJLm-N!oJMA>j`DM~4qy<vG*S9xz@@AYuKK>WVnaXmx6;Rrcu>dBc?
zr<hRql0+%Lp7&<zJ>r#-GdXvwa5*xHl2A^LcTgVOoAZn3^j!5=qoWkVnp`M4$I69&
zxCY`aZpp2PDO~qe)z$xa(>0Sr9(O6Y(C|Tq2cDY|Bu+{Br1jSD9le3;-OrEPDBUgj
z(cGaQV(aun&I4pV@rLAkL&xgsPg>+$VSn_yhn|0>x2TR-UM|+;7j`^b{x<XJ^!UeZ
zN3P-Q_=`Zfy^hl7YwZBD7JE#?DuB<mv9FZMKA<u7l~`X{%<U+PLC~%VTquh<9qu(o
z=KPzo2xs4uj;Qjm#2X5ChQ5a$`E>0wV539ewr8Y!K`#^LJI!0^ujoS2d+`QW9%Wwu
zCa&0;x;?_T!gK03--wY*%+Y~8lG>LaI&&)dTd{$u-|}xg6Y#%UQuG*dEH`n2!shH~
zu+C}ksU5wyr8nSLdFL5}?>EI8EX88XK-TEzY#2$s2Y_2f#rM>AVj6RvA;%!!NZnDp
zkGaRk-d=ZUTe$A9>QR4$886?Xtvt0wpWZjm=DE(A`70TEQhl7nzTUj4^vjd(w}un5
zsgZHclZ*tqJfOLM=>Br8B_}z1z@5%)^ga9Sy%JO`RMtUGMF^gpHl<5#n6%m&F5NEt
zyeF}Lpwmk{Wq1y;cUqy@-=ja-23I!>JH>xPn^JzHY-S|v+h;vWTcYhBXYaT>{;uv9
zDpOmG<wNzI7*(C*V$0#VtK;wMj&s(A6YW*0J=Nt~lIt>m!xaBzCcKS@>nzblz&?R{
zQtsspZvejx;vQ^WKY@Gt*|T>QRU~lF_1ez{_t1uoLo#L<n&)Hq$6fph{9|(Ah~~W^
z3NJ}4ImI;t#`$h%LZ9D~gkP=ajM+ymNqP8r`!9X?n?7*Y@Ack)BmNw?b=XnGpR*hp
z$ZJcAwN~}h`)XG{TeLe9+PV>*`z`3|ccH7_gRZ`a@38~FV-NNQS<ux5(A7cc>QT_u
zqoK`ZPcZ@7dKxr3G0#$F1){{sSxxSOhM}{{!j>c6P<({gHpfb%mSb~Q>7(+681y^F
zhZfzx?}F^_(L<Ay0bMxr#M$vUb}Sd#qP84Al;rpbeM9HwiKRcb@k#Pp`n!WT|I5wy
z;GN5QGA4-bB(`U7YfbVo@zZ5JneVBL?~|`4`2ghy%L2r#Y$z<i<||A;m+@b^F>*PP
zqo$$Y-Ph*jm+BqKi|e7xcyE`v2TrEVz0hgZ$vw0M|0(^y_n4l;Lqy>}o%m|D;m@)l
z-sDNwn#3Mt2iJr@vgd2ae<XuY+zmgp?z_;s??LNc#P8XG-?InXiY#c|0%+YJwC*Tq
z-O<pxRnWQ<pmnD~>&}AKZHy0fEm1s8Gk;BK{=s(tyf&TWGpz--Mwi>ET+qk{HZB16
z&>bIY4%qwJZ#Qa>+jHSuf}edvFo1`eiC<&u*!$`(Zfgu&2<7<P1~xaQMm_tzZQ`rk
z3%x%L-IP4Cy=U@F`+Wy%Aaf@2$Sk!9{=CDqMIMmIEy+ERQ1ZS|9(F530uJvtQ*ym4
zQ^}jKi*qpZcJCT<9(o_#vpozhGO}!Z%KgN4II8tMr5Qe;D-bzxNB)J|@OMt@_I6{(
zbb7(V7G(t&7CdBqf&bG_+);Re>*w*~o7C4<7}p%CpW$vR66U$_5yYP@?P2`bx~<Pk
z-zM9OP-fjFZ@06VI$z*fv(81vFiGDH<@wy3cLw|wr|UjMFO>q0{BrwCr7?J)WfiYj
z?vpF<c?$a#`vY3hvqU&)>r17g3qK#qFBh-YXz)~1%3GoIatHX{32$a4>U;EzbE@Ca
z*rtsX<#%gp50}R3->mJgSPU&JJlEuY(%2fH=cDb2k!3WcROy-h-@V-Y7k$aM?Xu*H
zDub6kqP9GTo$2t_&(}lybN_r=4<t@3In|SDk2y{}{w-$$<snDOM;?df#wvU$V9qSB
z>@h;_fb6K3Lt{2@el`;4U^Tb{9n4j}Xm=$|(0<tv3+>Zd?hFU4ikB^?ep_E^nJ1WH
zOEMXs%r55A__&L=1D;XoIRs0cebB96?(ZD>7<<*H_ylQNoL+d}w>qbnyMdom`c4~$
zuF(cQCB-ee2fh~dE`nE|=IopfJu#JWr*UT9=!v@AW-W9j)&e=Rv16UBwV-t%J#)B=
zHP9uSwm?3#w^gxUvhPC0et!}hkNW=B!Z<ZM+^ybY6R;zjkn{1jD7)Mi&rDl7XA^4*
z{c=t^{S<wSO-8QhjA>zVC~MC6_Vg1^551uNXf4W5Td~K1#|7C2nLd7>c;t#rPG8Yo
z2DFy(2SyLzKUYyj+>czVaG<MXA$}B>(C_DPzRPdI$(V{Xhs+H*HdqVSjWo0y>*RK1
zwnsu)_{*?Qv`5qreJ{Mx^#5b<X*&~GmA272t?iw_GUg%D84Ex+SZ|m!nKKgGmfM&Q
z@#gvHrz+I9!CWEC!G~rYd9lreMo$l@-x*J%&-zIBjNwL=VdG<TKeD^2_%|y*lno8_
ztJ=e*0m}vcdQ)@K_I6oE#cnIm_D<P8t|R&R@#$J3xbi<=a5XS)a!#N>Q?@?_4Tk+=
zdmT0+;1tfY{j4G3sc9XaKtAWz^i4G}FY9`m)(vz;d*b;WX;0x78pTExTm~)K5B*(E
z-r*zbd!XMqpJE@FwiV;0nQ^qZ57~RieMopM=^O4hJ9ND#{kZ@!NsYa-?EFkTKIrTR
zl+$E~O&gAX>djR7y%wc?m)OJ`Q;R%dSFR_JtQ<4R-aXEgKbfP>Pnn~4N)M)WI%lbG
z$gP%#@@-u)dv!Hu{#y3%M$Y<9&iviPK0GVF=f&<DZ)1JuWIl-B=v6+lnQd2mf-~0}
zKjqoCt+vKY?6Q4=$t^X0(07!w>WSE)4Ely@ji#RtbW3(NI9_vBH0AAtUYKckVCFd9
zNWL)UZNp=?-phPUq5QC+2OE*s+4EVhd7_?)q0jt|<9@oC-$v%i>zu;<<j(5r6x+|~
znasM6UUy0O3mPn2^G}<3YV)D{sl$Hg44CKMZP8oLYCUX-jbEwj`snyYb{}(RFfZu7
z3#UGP2Xo?K4)&xJd)K65FI4oijDY`k);Rl_d}Z(w#1oL7CZhjk?#xT^#-YiT`E`n2
zp*l4xXWz-TSUeYHg-2BeE<QEz(9jRI-B!I<^Tpl^Gsm}S4hC&*IX90`K1py*;HxQu
zExg_l&ZcaQnLgd9xn>T^nUgVB&mm=({dIXZR(^Ktqqjb%@9`|Gx|H8`)jUpO9;aMA
zkKof0`ku~b^s(}7^Z6EL3%8)}DEWy0-oe<~f*<!8Hhmbd_^+RnpNn4#{j}qGH)XZ6
z4)z(^0=h0Dy3XDo=D8`aOzw@0OFo9LB=ocIb?CgaqA%99mCc0Cd;*#<AKLwyctieS
zu8(tH&i}olL8)tG7g_^mPD?WHgszwwZ@|X!4fIFQL0UK5?`Qu{Q5pAB;|+N$xsN2C
z*K>a-_n#l6cd0W?_`+T%E@FSC@1o4km(YlBC+jE?gLHPJf_S21GlgI7&^+OTWqIFZ
z9UW5KaT`x0=$}IG9^nbgi;Yh+xTfXlhgo-N$vJ~?^~790Pc%8rv&NYxxZ_LV^z3mC
z@xDTP-%l-~4|DiV1M%aYc+Dzux?e~sc0Z9?L=503|LHJo6@ga{f>$0fc*P4|@p!78
zd7hS!wV|<o$&<^w|DIa(YHyK~{Og@dU#;Dp_5<GC$#<|d)f^&^Fm@J3N7|Hr4mcds
zoPbkSYL3htv<y`&W7!-}(*CpOIyKsHD13&qsBwA{^6T#*Tb8U|vg^Ghb4ewyrW;-N
zwa)V)k5jf$Zs-FSvgS2gR+GB}S-##2rQrLecT%B|)V|wVgFM2&Wp}VARFq=vh+Zf8
z`WyD#d$4862xG5Pqy88<PfOv5#DB>jn#B9XTM9me{s%V=+i&Psc=1VJ$5*h)cjKg0
zRsb2+Btu&@I!}tud9m7#%dPh#W0Q9c+JJ&n8?jZs5O1{JS3RCXAEq07Kb?=}ylcvN
zOV7gSwQPPrAUUCJBh-}hgr3<v=&*E+6M50d&FhhspvRIN2mNF+btd!fN6;70rq*~{
zZh|ZTSrE^+O3uw3Bm1cPnV$I$`rFCdcvSu4U1MuL$a6ry|9QMjp@%r%wQDC4<BfRb
zvin2NQbg{dx{Gbm>T`mD=7M=x1+Ba_@DRLMKKfMR<pNjYmuB>`&!1tP_>m8-VO~?1
zQ#U>c>yV-D())vb{j>O^a4$SMenw61cEOml$(Bzy`H!d{z}JqG+vI;-_o7?W2gBdb
zzs%ns<h+z`g=kg$*y?P%eZ~E4B6sz?l2gJ1wl2f=!~a-J|AhSn{I+5(nV6aN*CEeo
zOs(Pj>I1TkBE=<Bp3_}r58nL8*ayL$vbFzsiL1ex#N`U#5C#8|_CvaH1KTQ#7{Sh2
z^x*<_3;pVYdx3XdDCpkhF7qDojUzw$cz@1dycA!pKS^hbulnOLR)EQ!f;Tv8rtg^l
z`~05D_`dOFc{Agw?0q~Z4-Rsxe2YF9eyK4u-q)jh%4fJA^%lbG+WL0+^;{MEQ~Jys
zqJC`2N^T65IpNFA|GZFHCg&~o)@9Di@!|hemV(W83;D2{EaJnN|Eu{w6TAAu8mrDU
zo$pJ#Qb!_R>8HKlOHXIZSRArd866sK{Xu<(9x4c3=bUN&6B9x45acUb@s(lXa~#1*
znSX>{^1$C@g+_wY^$sx*LZh7J$ol2?Bby(^Hm_RRTl;Q$!{zm`OWzE+9A`ph)*m?I
z-%VTJ3T{s+)PJ*v`5rb37m*J*kPmo}4`d-9FnO?$4}2o$RqfjJ2zihOb4D~+&**F-
zUu@X3jy#+22<yKbD-Z8-gm$bREm^V?xrj4u95{#koYCEm5Odkow|eNs*TO{w135!K
zf`*MqF3@#L`FO#Ldq3ruwnc02HESl{IUD>wEZXG=@qEj=TVlk3v+m&<tGtEW&pFok
zb+-gtx!#d>%Q&4|f3Wl4K5!0uGSOK7AiiaDSNjEQiiuIkTs4trunCz`N%42!QvwgJ
zc<eE~Q~FS?W9-9@kr`cqhkq06b@J}tuDb;uJIC6$`IfpnRYzlwqa$T(ipw73-sYhj
ztoPJk@XRHixft^`+s;&<Rjfqj{zuexc`ul<!-=w;g3n<2`ZIyT#JP^$TuBRiu`y8(
zzayvns(RU~N1gqA+i%x_pDA$p$zgSAyLIb_Jf|PXe1m2Ux`?xrPyFTsloRu<<idv7
zUp}q=haYx$!`%+I_&NAD+vgp+EZS{_#;&nk!0kNoGUMEj*SMn(%m1g&k+C{EOxbVZ
zbBYdY{^6sqF>>F6fc4z#KgqV?|BZ77R_dp|1w9C?^Ti8pT2cB!qK;_I;k5Pta#ak?
zV*dXtxkLN|dxm}luM)}V@Xvm(xi<j+HoLN}?*4&Gtd%n{`I}w;Ty<|4S#lkA;$t+&
zX^+`-Ojl39VvbJmJC@PrpS``O&d}D}f0y6+SO2Q$@cL-gez)e!a=tDY#OD+z>pr~F
z3We{t#=iPA+%-Hdac9PHF2Y-I-|cLEa7|Uad2de9UQ_>mo2I4CSkc0!{{4yi;InmI
z#2lwxaP;gGD~4VmepWto)g7<Kit@3IwV<zNhgUk>;S~<A`Q}s3JFuyCg}>(T3I06G
zc=c7jebRZS_Yir&;Qg@E8LN3Yv1TZ9Ic^cQHzDn3(G9Z4_I*FQWGVO9G+$%G`RgaO
z_Pwt=hpg`nG`KH%U+JnYXD%=MTo-bGk9}|CEGVCycXR~;<De1lVD4@TcPsV|GO9e{
z#4%3qA!pwDo@xVwj4l4z>w9bNWWU|7dDyUPPmRXBzNe;`y{p{yJ+D=H+yl2Tp2T_!
zf79VjBL*DphP3{WJ&a8wHXHZj%su>^k&o?h_{IW{*A7$XF-L}AlJ+C#ytEI1#SrG>
z&{!jX&gjYxo47Wt)p$wX`%(D91g@2hpnN4|8!10YopH<2hg!&@3y?(zkwuR}7CjnV
zTLqq-0G^!&PFB9;M&CjUpXibc$O3+?`2QV?!hZL+y*Zpa6VV4qjv@I;JM^9vobZpA
zpcS0KFW>w}=mtMDLl|1|BWx0`N5)q>t=b(H&9rV)z~p8~+dNYE>Wz3qR=VIN*$lpz
zS;V!^8hEtiLRUCY$nzh^7KQim&17VVN%+p{o3_6s{YS<dM8-P`8SiLtcojH&0yumc
zID8i8p5n>P=geDzjCXm02Ee~O`>ehP4q@Mxv?%vAspI^7%bWIj)2G|pqP2Te$LKYi
zyt|>DF2$R?8^xn1%9h)04Eti*4Er5uBQG`#;HF{MCF+byyjP(2c;92nOG>^ITlB!2
zdVlj5T8=ZW9Kn{p%n(cseS<v>K3E483u*hjKpa^xIpe_H%~|{L?>i7#K+c6me^BR%
zIpY-fC}f=>uE*Po-zT||{7T9KF%Dzv9sCcYW)CZN*e%wt^epWbZ%$g5tMIpP=e(o7
zl*OROx~+eJUKh_ko&NkjRFK;D+joXFnCE;So&#H9Kj#OwKja2gKh_<3Y^U8f|4RAq
z5u<#o>gzYS2E~U(sxxJaxSDmimNmF>-J@auYmXw^pEoQ|byX)6%v_bZ*6QA{J{WM1
z_r9@VeRvr$Pc7DllRiDTp)GJs-x6*WUX9PmdRqRr%=2pIc`ax9DdyV5T~ZE|bzP<K
z#*Q<ayXy8N@NtRV*UV!2+92LSIuc@D^b*(mUiE`<Pd0U%Gfq;cL+2D_<hpQJnGydw
z`~3{^zc_ozbD&rp=p~es!lu6x&s%tY9vJDIx`VhYkp#`>(*D+W)7r3kwetkUw5I9D
zmDo6qIqwI)$9&JpI2-z5QUmZJHjYa+By-^Nh>b(;D)N&c+dHTC^!{tZ-4CNz%w3H9
zL3w|ON0w}lr+BaWaHa3*Bk(6!MVt7T@WUUP@jW&;SIm)ebBdm_<EWSzDoOC_iUZX1
z8}?^`_A2u+{<&X<m2=VFkAD(BrOrX_-n{MTOU?Mmt%L9N?zAN5Bj|e<(f@Z|?EX=^
z`)`~{ZPB`~tBwu3w6pZDFFvP$v5AH!Z>2piH_ATG?q`x{z%2A@PoiIY2L0L*^lQ&!
zx7Lqr{S>nHx1b}h#6ZV)ADrf4Yz3^pAY&Uv>^FnIZ<r6A5VX#RZt+X!(#&rP@r`5b
z1!5T0Db{gg=oajHc(ycVT`pf7!>-AcrE1@px{G5e7g__zW~!}a3j@|j{?Fvv&>C=9
z3j+=}S3lQ$t^uw*ca(D-$2H6~!j<|C)sJ#*<+_aPDy}iE9b9*C-Nm(s>%P_iF(d-`
zV+G*BgO<Oey6)`4K=K8H6Y`z;t-Yx`=;OrDc<T=fdsBDWdnOHCDzvv~<6zQ|R{}*t
zLcwv4zUqSWeL)kGbr@&KO`J93eDjHMfSd{2)@zv`@v5>lnjW9yJ<2(Z?KCu;XhHUn
zuL!$bXtOeZ40}9e#7${WOXi6ERE_2j``r-bf#@31vUO|uUrpIKo#*_<M;AKWJo8e%
zhW|ePW5?@w%4cO4691Dn9e=M}xm@?@9$S?xwX1KkzAdwM8ZFk2%isuj{&fCdP&xZ%
zGO@7GjU|`2^#ppkpVYMhy9>@M<Wk>?9CWz(ZtdoxK!<p$$PXO;%2PF2zv>MbANokU
z!#(wMO^q!}wr;~m=CJBLU$j3U9^STzPWoTyA+hmF>o@ClPMU1sa&>-o8JnnVXvQ@8
zVG3R@{wpquZ1<tR(nhFH{FE!xs^6WO4}G&rIyU%<4~R)O`?W4KX=_|Lecg6BcIF@8
zH#v#C#H-w0-Zs^JZIZi<SXLjnA9WSF4|ylMf9QSmw{Lr&s{Cn9R@z}ZzD**=&4S;)
zQyf!2bw+^Kn{-C}_U+V{%v?QS{c7Nlv5!7O{}v7$GO=~v9ynBZAwI`*ocVf3IIL{?
zvyo_ZCVsl&1q1tp!(Z%vvV%1@wd^+c^_0)YUt^!UsFc{u*G#`MSHl$dYa@0n2L|>I
zJ>S*?zQ;Dh%wdyT?;MQ0?yw@KmAeJJ>L4!A>or-EPBBlTENgop5LkM;rl3nUkm#Wx
zufaz}*W2%>zQz0J>rdBYEdiE#pZ^8v)ppLIXtkBCvd}YDN^39sve)5WI08T3hM}69
z6v|!v$1Y6%OX8POCy*Bk+xNjU$7SqS<)f@Tu*9on{7&=-uJXvQGDm7Zm~+0tBbnGu
znq$WIV`M=10aaTXr|(<-qRkurcbql@ns3P?p;@z*zFAA&ui7BjQONDG;&YPnu;=j;
z!|OKuj&gk8^XIBhzNh^8>lem|r)Ax=ZQ=NP<n#AXH0F(4g|~|a_5Hw4J9Skj9zl<V
z-6sDhD911Vujl_Xt|Ph5;(9IDMy|uT&gWXnbqQDCTesYI$X~>DweO$JbuDr$-EZ{$
z&|Eux2h4Rh^UV0MbG7$kV*eF9&;0fn+u$1OOt80?xd>R1k;XoFhRTD#<18O#y&ilx
z16c$*N;kH)FQ<gjAEs3NoBLRWVlI?tmTxVnIxy{Tv6qeZ5>pDh-d8dlrMVf7(6iKE
z3hqOnTd~FCu6Wrwa;)DO*>b9M__Xg(KR|up6<m?w2tKH_z<T*L?~F>pJ==0sO!q`8
zmO92&Y{>|QZp@sy<BVjLsb9mEW~14|cO~02L1O_noy@DjBLx!%9vy1%igG1kPtw7j
zOXMPQzWd5v&&%*orD1G-IO_|=FSrlcu_xI#rJ~c3e-?mK(Xn5)-$4Iz%@Wpa{Kdn8
z4(%iOI9J?Czm~qtP+zvkZuA-WRj2OT5c}N!k=$#0>~rtfYt%vf1<wKVzqy$G<2Ct5
z!_}D$hIVo|&zm-5<*v9RDUAR2ma<!bAGimc1udDB7Z_kJkPWkD-Jxfbm6KRDxQc^s
zbX2VA2Ka6CgN|flpK|~^DCz{rt&JVhhoK*)_P$sh9%*3wPpQ4a$;7I{_Q-KYWx@Mu
z9g%BO_XV?S9XHIktl${F_bm45K7Q9(=PQ=FVij)JvWiiz9qyN)sU67&?DMK<hW29O
ze+B<tT07xtD-P~nrnZr3jIv=AooVhHUBbOl@Xo}zBja`LcRQv4e0vkPs><1`dS$aS
z9MFDegkLxxpkH0c!2*_OuTWX)`M|%kF8|jh`7&Cey{`X{y?2j}y1e)OzcU$<$xJRo
zE|7$TOcJpX#YRcENs{4W2~b;<TY^x7XpPeDqNqhrtP=<jZUTn;4ic^GO}n?P7O3EY
znzRL>J=Q?D>4INF-5$ryaqn2Q#VwlO>-~JcLk6l{`<&Hv{@FWOv%b&$@_9bb=ed0@
z`%14(T2T6~=g7od^0d^^#*{5)--^A8x@%IxMaA^F#(l1+8eE@?g65%dS?saXYx%0&
zs-iv}>uMKRYLj%!sD9qlma;CvbAo*y_A}h6?uU=19X^($@N6H0XB!<M@NfTT?54y8
zPRRZe<5qpOJ?HsH%E##Z1;5Er*~j6&RBy30?BlW+cQ^Wd`*0_HP;g|8d~w2W%F7Kt
z&CGt2_SF?lOELPm*q7~>ET7`AIpx8<h_LnC*N-?|nH3-`H?hW(Egd1NvO{72O6(ev
zR=5?Z$>sdgXWfLi%FB_xf6+g?NoT;v!KplZ{ESY7@y>dB7#YerT2t@qEH9tXSzh2<
zv@3ky+42I`;)y!zvv0i@KW~!OcIC(a{l?b95NpF2*~%HdlzW$4ehKVPTGHIY!KCGB
zp0daL)J0~(hq*j0e)tM_A`02ZCZM|meAl}`*<-}dNk5u6$k&JaFE@7<6Ox+KyrLtK
znf>>Z|A>=M_Xwit(R@e9SEDy`<?XlxEpAC;Pg8kJXJp(-qF-g(1hf9vC9OzH8NQ18
z@1}m0!Mj7*R^sNQ9W!OzOBwG`#)`B6I^HsvQ-P##S{ixVH6LojMh9bnawT`{#b3-F
z(zV(r;~tFr27Wh|J`X=#m+bNK2KbNYXnfxC@b)2|QtprYexf}L<po)j=L$E*{knX>
zxX$uy;cn^j%JcJZ8{DbMLkCa!O~l=zIKpoc{&Th~UK_qKlX-M&(aPUOCS#v!c+0iF
zSXn>heY@P;mu#FH@_NzD;DXWb5+97r%)DLYLEE<5<>#t?`JXW3S(E#o+9*5We4{qa
zzZ1z%9s*v3wMR2JTGRhCXlKc|GvxymhW@|XP2cj9-miO}H95x`7@67U5pdAhEn@7)
z2;SP8Ef0U^b=3urYSdnlnFEphq#ei_q^GXD>MeeC%v=0Z_FFu6dz)51{8Ya7JiOtC
zdCy%Odk!b8QRnY*8?C2#_nk)kI`iJU)mmos@}moyunpt3S)0szFL!S~WP&3z(+5eX
z6Y#iKWs8QC4)Pq=I?Ni}`+o3?05+F-KVZGq`BH0+?w6wbSB=r~@E|YaBi%n1dcvzt
zdcBQ_eSBVcWs3C<0iVp^sO}rWZWz<gy73Fkq7AjzdpbzN`eyQ>YuV=EA?%uH9`avH
zeG_BbF?V)q_44rbdQYh`ec}AD)O}5DE$84hp6d@5ZD`{TFHw6Tvkxk)4_m_)7R}XO
zuV@QzJ}o(6+yzCAa-;wFZ25I#MteHg`=T4m+c+Ox0j$Z`EMqSP4kkIe?1c+S=(CTu
zGbu)webO*s7XvF}f0um>Yn@~ssw342iQGXq{dV@wZq_^L$TBv7BGvKeVskGeysfC^
zjULe><f)|t;|8Oj!0Y0y;{FJC0lK9o67Iu&o$wfU;J&vUT?TJe<l!E~ZwSAe_?3}v
zB(j(jBQx{g*O?i8yB8-MD$kq@96FaKz7yWnct(5)qKS+<ymCSIzn-71{zc<1<Z9kt
z*NP~z<#`%C8uts&HU4#@Q_;h4O&Hjlx3<U}Um|nmt0r!ccm|!lj^#-MU%w7Me&Z?}
zp5ox}w;UXv?cneoF*sZc4&Qwx4kvFH4#V5ARAU~=PxyECmx~OK({AhW=$H^zFWd?J
z-1@=Q_zPc$Pq;(g@52l71UPJLh8lZJwyX+!_PG6}*{+*^JNvoT0&brKH^wUftx;S<
zS&fy!e&LAv%Nnh5kX#1(jLVDoK0WS8IeJCP2l<=aRq)DRTw?_#lO0lijk)r}7yg0n
z^5l<}?S!{0oN|JMk5(OwF?_Pfmg6^Lo{fxIwRJnR$j4h^HQ}X3A5AvvR<*6;_-O94
z#bbtgI?&xe_0&Snd)OpNS%58@hXUx+!Y0DQ&?4-3XdF@=1#h`C_UX?2&59w=%^|Ym
z5uNkY@1adI*{{onr|yedBh{`4)z05sYOO8`0;AJ5E^|lSk2vvl$D_}@cKDg{0{F!q
zS08{`NPcvd-ej+_p6}V~v*+$eP4Z`)e@a+)U**k4XV<J)U3+p9R;>Mn=J5PO<$bFz
zr501>kHIga&mNk1gYN#pxtn8nUwV0EwpurWQ^S1wwRg#C5H4|tBlwRHT<L@OXp<r}
zoV&qwv%ctjU6Xu2a@S3f{J2zmjqvZ~j8c0)e4m7UoIE(K%AA#6w9iVN{d><!$Ke?O
zAOFwJO7<D)G3U(G;GB_GIA^ByoXOx>>a$bh5t)fyH6t?<C;bz*-U^JMzc)l_=c=cI
zcbj!{w);Je7jkvC>b{Nh)-rhY+8L*+BbQoB>^-fUaqzv*7*?Z;f&TW{u5%Q4TSXb%
zktW>F+B&QD1NbH)=*!v_XsLV0DyHw3>Mo&=lyfOGGppk;_TJlq#dR0Wp2?aAedkxm
z?;3vB!V}mc8b^C-I7^4oos}uxJnkU1e?ZqGbGs(_Cdpqh4=wsN8Gm2e${nile+?Y^
zKz{WZ*%fPjJ?C(6jCm7~%+tD7%HVB_vf1;yereZf^d8)*I;$>OB|7(DW9;v%KD0`B
zlgd54BUiR!KjQ8@%zolW!co>^kK$8m11o(i-DyvU%zfopt?}BoJXL$wYl^pJ%aYEM
z{}15sAl0WjfL)WuC-v#Hb&;8#_uvt$NjlEmQ#x;+w70g?8b<sy!rZ5*pRcFm_gNP^
zP9Zmb78^rpHh(DLfl_X_`KIy9Ag*8SajS&i&HPIF74sX+?*{a7-N-k{uaMt0_^<6a
zoO3bpAzw@Cjx=l79qiFJrMa<x<5@Yq@^x(f#H+3IS24e6ug047??>Dj7};v(!fe4N
z=|EaM{E|QF*pqcp@0Uq?*aIHGj>(>E{OGLMgT0SE$hGfI^YZiYOR3#W8+X&j-L&z5
zHO%yRH*MWbTX)me-L!Kz?c8l$BREoqhjcgcyB6FU@c_0At3Cpb>DJx82i#M6p9(7-
z?`PHn^}w)S(29<&Gp-@u0nStKtui-00Y}s)2Y-?OhgQ#jt-OH#hKyT#{yDDq=XaEI
z_g{W{)yL3+OwQTTc?;hb<DQXnH|u@OTL?Uq#%7b&E_f)@^#;QG+<?Wq3&u`**lJz8
zymGCRxA{1_;(@Pq99{9U4~W|>eevK(8E%i@?%(KZfo3(S4rtj*tz#;au+`DG7Q&xW
zn6J~F@l?3ttHbtM?o&_Z?HISiYEm8Z$geu^uf)Alb>Lo&do|@eHIMwfEgw`q-j9xI
z9w&7f+wEzZ2ejv$?2D~#9g(WJuXddb<>muFFSgT|5B|9}&F=rY;>Vx0iqT!yOJ`OE
zrK2^39<$el@6eeE%)gu23up{N*uuV9ay-nBORSlLR2Doa%V^V0x_e=aLWai77`Rik
z_nv=5FQWK%kQ+_<2jqEW<DtenRQr<77KuMn9ngX>`Z8t}d7w=-30sF}4YbDZhabHr
z{wC_=UNo_RuTNooNfXZ<8@e1!nm+H@VV?bbgQd_M#(V-gIrL^Ri?9*68wi`^xbMeZ
z)z}pb@x5bdh&NsAyezdU=Fq45<`vK%WasSt6Y^5B0fStZ;^>>Pzb=?ZX5_vj8M&nQ
zbpMRriA&hGfNvCcJMJ{n>Kynv?hMY4;@3G&TBY|S{d-zG`R^sJpVFmFNHe;yQoe~@
zIOryEL$DjP<<wos>3QMRO-*akyiHC^3m?nRx|rN2ZOgHK*%xK6N9$E-hSy4*h)tg-
z)2y=VEO*MIX>M%xcvh4e8LjxbA6p~pb}`q`d17Ez-S=7Gmo;8vBe}jAX$MO`VPEu#
z*37txq;udO2mD=&X`?;wX5<(-FxKn>`s2~wjWv8CZ*&Xhx9g0RW9H@L*O-5M=ofs;
z+7G5I))~a;rJW7`fzcnQvq{P_TZbHPRJq`8^Rl~xCCtCvltqCJ;XRe7qjL|OKFFL4
zbEoqG`a6nQpO8ro_Tu{i@V#r)pM{4uPPp3580F`@Zo)2s;~&Q0_zGt&y)$FF(t+b&
z(^?8%3g1e=aYx^zdoXw4)!b3sZty%3SdN`a&f%4x2)BVtNn}RWN#Q$Z=yl+-(N9tP
z6YxZEt;zYI?jMB5ac`;~bWQR;QrQE&D>m+Ix5XN@1)g@Jmz}zU-0ckkFTc68D;VVa
zwdN-9k;*UQqhwKxytnHQQJHSD=FaFZ-oYtmK7pr^8U1(SuSq%rZ9gI#3|6?~6mL0a
zS(i>`U+g#meLs>G4?PaC&R<N7u%>JRf2b>UBWnsYZACh3$}-lJ16J|mW#~5I%lWrS
zIuh9vdu@MOlM~(&nd$x!eYj^*PJ*P(4`?lQ-WCcjydA$<F!XQt<te@HP#Dv7Chxky
zx~DC(9<NkB{{h*hx$6zZ+4~t6dz`$3j72b7m)<@(6XXhxz_7Os`MG=1xwKsA=C4D3
z+Wt;4Uldd+J$kVJ=xs|}#u;%I@~<xG*c(1;*mW1)PMzjo=Kk90gT8?GoO{j#Pgr%N
zdF#U4{%OjGKipft>+j_D)w!!un|kY47_c(^Dy(DuZMIq!R=DVmo{qP&F5=&!`XV!}
z+XZ*cEz#4+jEq~&I(E1mU9IH>a-+Lz;{4ZH&yFykC<E9_*Jv$mgl{xBSZ!jwv1OF|
zj9>uoX1QQo^>eFu*hwptV$YMuCpW{>b#HJMdJxQfMBn;l8&v3aPfvX&kb4zR`O*~T
z6!Y$p;Tx^22hLhmrvK=~(KqGWX-#Xak>}VmkS6z>+Gg(Zs{0zfEX%`BzGLsNvo><?
zZ0zLHPdE5{s|l}8JV<yI_EG}8Gkwk6t?u*`GpBg(;T`DCfbP&ou%G80FTKghGo3uh
z69)F{ZK~<)&2!C}N_U9Wf%Te?m+QSVmHShlxwkDS<vw>1cg);3>Q0xl@SD2(Gk46%
z*7&mP62sj4Z!4SRJL3Ik?wBq2e^A~E!BI5E)P1M)xB1Sjd#i*qrhi|-9`5|AlP4B*
zrntD5cua2=g=ze_$0~avF<kb$KDA~4o>V(5kW@A7rX=Qf@=^53cc}lI|A#dlDEx7#
z-s&>v-G%UHICmw4{fjy8J`O(y_ceNlW1sdIy1dO>oPU%r$M-n=8RrAft9?=a4EQa|
zI`$5Xzz_alq&l@PZx0vO?JkLEj$to(3S~TqzQs`RIQMMy9UX4BmcHc+M5<E?M3)3N
z!7_AxYW4bB@m07Mueqn7OZf_#c_-P;kG=*EywM|m@UqqJlCl@vQ;;14p74KP(+3?v
zzCdYQLI<+gYs?yt9yMD}N78qt`#|s#U&QhqHt*T;#_Otd2eN$G^;<ZresAsK%BAWP
zcNG5`et-1Gl4VPJ&+=F87cEP<P4nT{{QQf+(8!k=`g*5-ukJ+AvsMfqKB2K~Y<=Q=
z!GisWIpgVW6?qYK9Qo&ztxj6Rd3_E%dVL*!g&*7F>mP&+gIE1scK49=%rR?Xe8R4>
zgoIt@JtET@&-vobB^yWIt2TyDjGnG<>KZHk_`>{);oV>TP1RxaeRCFK&b**J(84n6
znFya$*|@_MndtV+M!w#Xe7)*E@%!A$$!F<*xG0Ob{><YE*jc{I!0Jx!c>f5`XNad{
z<kby*zccY1@0y;(32xsSg|m;63_5!i`j)se@hk2`bZfS%Kl;(1Ho_lKI~xy2@i>t=
zl-zCiIk6mm?{&*Ug@17TdH9de_mE^9S=a4*)H}V%52WZ^@U7RD|B^RC?%gOKU1`i=
z?oo5Gqs;vq{fbB506MJ6H&yRH-Y3Kh%soM0@Nl+2-q#}8Y^~$bH2!72Wu&Ph?VzN#
zv~X!cLI>r9=+kWa>MKpNzC~UyXSYJ}*}=1y25)0LZ=;_1oL_{y^G~2_@+7(@Paz9&
z4xPo--pA(u9{Y2|PbLm|uoaVuL$2~!aBt0I;{401b*}i$k_}^m!#7u+1NZJzd693$
z8~V9C3{O1cA=<{6Jc-;j^FQh0Ys=egdXS%YD{vfP;E<m;1h-A2?nKx9tkgx|-bvt6
zVZ#ZX^s}&A1#bg=LryO5g<crOml4-Z+%>`r;vNy)mtA^hg4z=t%6nWvpIu4AZ5@`-
zy_te%sukF}JXqeNG^@5&o*Rx{mSKOc>{-^m&9uL8VTx<m!uVjy_k+#<THEGcvVwDr
z&pmU=)9CI<ch97rrKZ2)tIe06_r!kYTh5tD{vo_O%!JSV80)#-*|_t8&y%?a+!mPe
zt<K#_9QP*qhluwmP2^ksH)_vK*mnGqA5hqLopg`jhp!>mrTNGCz771>H>mR99}6Jw
z&B+)Vke~Ha-rL_)-Q4qq$>*bPbb<9d>4YsdcUY3Ep-sfU5}7&RJZ{N*O)~AHZ<HxL
z9C~X(#)h{7gO_o)5e^?#|95HU>xz%unV>gB;6HE(e%g_cX}1~qj%;u)*BZf_QntdN
z-`~)CQ}7RR^ZpWtj%e<%?t7~|6THpRUJZAi?(yz>t$dK)9_~9@ejQ;$?)z2wO@x*4
z9f`ku9`A4S-YOr*?^b>jSWCFy&A3kYl#@FvCO-9WLGgsnio6MLRTP-BB==le3)Y5^
z{m!+9sZI7hvis|5lcUQTdvUzAs9xHnwx}KdnKtmYaFgI_*BvZDZ+9N`7nnOZ_UpP=
zMt*p>=)7bX1v?w2)=nFy>7OZ%@%?TAI0y}VwqR(AYuv9YzE=S3%jUbA%C2*FmraWE
z8X3Vmtun!jed+{g;CG?%Q?c)=cPlI99&G9E3GNe3MZT>^_xbR)NY@j3MuWSww>|N$
zw?O;VHEESkbiB*^yBXhncu3-c=r#yAe%{b|7diqRV(lx?xb?N*m-blDm-w~oDhCZ)
zh@W>um4|L`Bm3=r`?4M=@MECS_s#d1_Sd9MR$86C#$g{|bBh&{{;1>i85Uva2PvwO
zE-7r{avmQz2-}nw*RCsA#F?FXT`zKvTfm%kV;6k(eH&dUdv|Bfd)noBdT$UthLt^%
zHJ-Q870=I0`YOIY|8}!C+16NFD7z6;Su^Jg2d=zZ9f}S__Y8vzZ&ln+d>U@qx)1Oj
ziQnJ`_>gzs8{ot-@bWeA@{RJc$c+5k=J7^4?iBg5fDwBK-=k)qIf-$ZZpP&KK>n%^
zEAG>nIPWWCQyZ=6PqkLe!3r&R|K5UO0ra6*l}n9$bT0Q9XUiw(?WHppeE?_5Z{tq-
zn}kj5%+0w-p6XMJyao6tLzl2mm7OTM1pfVqF{wCSpWRKnGpRd+_g>-pT;|AJ@O+}i
z9~%a`H#K)4(2-hjb@~KsSeUuO{)w<rO2axK-Br*i_eyYlz1CFlIM2|UnYnq|n=M~G
zr;K%F^zy~CzbSlUO*mxY@8nEt#ysU#;dW!&4RdU|p2M5BFHm)svm7=8OFJ#yVS7_r
zu$PMMZTJCpv8HXQd(D!*5pNyoSo@kH19RWMKRsa=@1yW<1bvZ#d70w7aNd}7$K@&F
z9^TNOnQ7QM;EiEm?(2p(gT1cCHhAd`Y@b=h%=66B2=rNRN8tB;hs(1G!%j;1AkI-g
z;`a(?`~BKi!{@kFyz989M&)H$Ypxr9)Vxn+%s*FtlW2)(klwo)bKV;L(%~`-eVr56
zOe@^-MYL$ut%Y^q+@J2_ZHhOTEZ(Ya<17#|bnv$ow`<HOtD5lX;K4-Rv1;x+Trp?<
zp^CZi4BoDH6UKY)%6kjW@OG6qfwPYpKWx|fm%49V-CD4V_RL_<a2>GmsW0GW7Ia__
zzr86Z(supi!Mo0-oJ`xLF!>`hvXLPzFE{YHWO;x`-$LYvfYaT$vl-87+I)v#>D<kv
zfnS5jGnsFhf#GkN8;6)1%pZOu4Xr*@J^@?9?ejRJ8C?90jf>{}&%g^^5P}!|dFtWd
z2<Yc@U}kJJ>&{Q~(;oBeAsSEgp}b^pY+!b>{M>=T=X_l)cNR;RFAH&3>T)k%99$<}
zmF0~Ee;|A-^q_=v+u&b*SAO*Dc0-2^-wJZInb5j3z{%!?p3z@6nRK6Lalg?9PI+iw
zb?HcVmfq!6E_|SzPX??}J3<HCS!)*ux52Aq+k%8QN#_uFGmR}s)=*m}20kWaM)HAY
z3Oq~LdURc@Iw;>AnUQJMaP5a91GCm>FT8f~j@K0bW#uxSi>YU-$})B)DIXduTcZD_
zHijOs{%G1g%9TYQ%g`nB$E4>2ye+vt$L7Z1q<=>5aJp5mc?<pRUWM^==CYT2BIzp+
zVH5UeRj(k+Icvak>^C3c&gOo}SaFu+y~$g#30mV?m=)^W4*fnj;Pcvp)~N9>V}J8t
zWzYD1*7)&zIR757g3JG5X`9Z!LB<N*;&n;o_P%WVPuVkf6;zM^i8X$G2QUrrhTT>$
z0y-nRXf@+s;Eb;`(3qTSr1!{#N9B^oWM5)`uXA;9W&NuOwA)R4({!d@wfz_G$iMy|
zW%6Cx8blwd*INH!I4Js@c($p9{<(Dz0*wjkoUs0d@DGr$aIa~t_`do<y)_rk_v<KH
zXIcIORohh#Yf{<pgXTL(e%_KL*D||kOL*Y$7O%CqzA;OFXtHqU2N&Ls!v=!u^P(+n
zDZ^Wa_W<(+>_w+5{&Z#btim?Sd*N;O5z-(iMEs-SlFkF*Wj`ylWK%ec^R;ACzJZ<=
z@z@28?Hjf0ZFenqs6S9$z`a}VB9wa@{Ok`V?%fwA-EG*Yc?`J}-7C~+Jx4x;_1zO>
zpMo5DM%nxmm0Od3k~Ti+g|w=rV`mLY`f1wNwT6yAz_{(ULY<p(A{xIFKlV1>v)5W(
zn(0~xUnFlZu5j&7t!HZYT2CL_n-e)b$=g`_b88#;vDmkEMA-Lt#YP7j?-wnkSzP;q
z)hL}}-0dy%uQ76UlkPcSZD4OX1{mq>V@vHG3mszpUF2yB?Y3GCpBG=*p2z0AxnsdL
zY}lSt*5Z3EdDwSk9jyD%BYhRYm6wA1imKpan3dn-ucrT5@T1Hc(9L~O<At}qg55yD
ziMh-Hkn!SdOPV?GC46Iei>Frb1LmKv+8fR}`8#jHvEL2oUU6#2Ir6l#=PyTA>YGZ>
z+<HWLl%Kc1Im{*H-@<w4W9HH~qv@1}FzI@jt#Q(diT?%R0{HhF?XP1WT~7OJ#eY{i
z(u&wL#RW}4rUqIQ$?vyObPAeOTUW%LxM<Og6r-z}K8K|vH){7#`*dSJi}_#ogy=`$
zobH&B_1%oj@)qoywIJ`i-JgeC{iTFlY?KFwJ;J$X7uV)}czZh;+@N&}oM<p-!_LYT
z(itUO!5;2b?nX|+=Uzg&I}N{{q1k<mUbWKnYyJ+seO}u4Gu|WML&`1FeFbT;Yc$&E
zSC<|y$@`+u;JqO7zUWE(0QsqL($fNO#!>8L9mD3qo2+f;Cx4Cir(YL`y=O->!Yk;W
ze5+NmnmL+U%^mVg&wlMGoI8j~>spPB_;c&RLDA_P;vb8N_Yn`@r|WLs@jg-b(cEC5
z0J&nqN$>ilY3H3u4P_(U#ZFrC1ml$lxGe>KOMp*RV>noy^fqq{@Q?);Lc`|8H#PJL
z-jQqtvdd^==Sc2X)aN@BUBIylJb9Hm=c-QbF*Yin_#TOqd`AsByxY;?eH0t+$0pP7
z$?7+-nG9SUU9k<+8Ke!{;I*1dJGKh<c=rewHKu;rbhh$7m4_|B^G@8Yid$+GtfRfU
z*W6*>d%GT@jOOd-n)l50+yw;D<Mtvr=wZLI6?-4gl7GVl?65WM4Yq+N&80a3y#v);
z_(OJ!?h(Q1noibA&8Z9LlU~$%2Y%OdKF@p_lk$;h*?qsZZJ%LB;8V(dDKgu06L$*x
zDZ3f_8-0>K)O{!RBj~@%rp#Fv{*ZOnl!aeqsT{)gs4V(A2OPH=ca?8eKQ%rMe6OR9
z&zO@s^BCJ*;-BDt`{MTwqqqN1<q9Le{A=(QIiJbntr6f;3HK!X!Jj4Un+B1OJ#1~D
z5IxCF&A|@l&ikdEm7gKbybt$hMa`V4TjJNHHV0cSgxlPb<z^0+u=ekQH+>}QVbI8~
zw*+65J$56%j@^3No5EROh{})Z2V#!OM&%aSsZ@U^%a%QDM80bMWZI{({^sGz2B(aG
z%2VG}KHn!ynfdwPp$W%d`R|h1F?~iSALngsY`VC6s>IGCy4NTVS*?aE)5N!vhWfuG
z4YCRbcC^XJG}1l;JLci!ae`g^7+|*)-iH^2&zE7>87!g9_mS1R!{`cLk`_+-q4_oa
zwZ5*tyAoSb8pHe-;Gqm(ZtF-_$@hM3g{iBN`d6U;{C3q(`B_SjJYX0cn9a9IeFc6-
zHtm~|fiwCBUDoxr;Q0|`*j>mt9LB!)wfK>_GX44u@=8aMS9%TkOysMm<95y<Mo$Ix
zj9?!SnLL4Uu9vL0s|sDEXM(#Op5=msD(s;wty*anuU=5FuWAMRteNTWu>PV02b_~0
zb?L>|dEsc~db8eQj|N@n=i#M1lN1iLm@w`L#5aT9ip{`m3-QmQQ+_M(S{$x#U~laj
z+<13GfHr(qwFR0_JBCBsmo#rZV|b_Sc5JO`F){_F9`4r`;g<}8>T^|XhKJya`jUf}
z>zkE-NcE}T$Uz)8yhm9{UC8mD0Y|*r-<0l*-={f%`&Q;b2eje`OPZf~SM4bzeAtHM
za7v3U*B-79SJC#*o9dt0&UY{}nyzsDGiy!Sg}O^i4X4br@6zbo*P7}pW-6{~D|V)W
z!!N4c+)<`bM+RrVu+DjWnR{tl=5GbYmTMp1V)RUxDDAR)g2h!!tl+YHs5ir9p~I@M
z4p@}7SXt6VeSBf|Ie3e=Nal>Z<~@D!!@&UM>pp-tZr+=>1h%Lit*3m$k?Hw|L|d3+
z<LQI$p_ejmL)2Ho86sG<)GB7ZA-`3~54o?(&o~3slCDheI98X=iv3QCmaTmz9Jqci
zJaLk7WNvWQZ#h-(Z7eN{3$mxlMaSQ-svdJ!uMGw{`&Ir~`MLMAW&XfVWNr&4gTKh!
z-h|BUNNB|vXhn=2b$Cv;1eVd>nyPJ9a9!KaYD?p3%bnTD%*|@}1Pq<6t7TsdjF%5<
zs3u);QS(4(8vH(6qcn{(hu)?~CPa`a+o-lP-tPz|<O?nj8hZA%<%Q-OAnpbF{R!Xg
zsk2o_Ub()&E0=Nx%1yhr37;a<^X}#B%{#(C^bJrR<Dz+K+@rlg=?0>#H;GSH-+|Sf
z(p+Q|?Q@Gqw1oL8d$tC@DYu0=vrqU9Zu8DNaKGK&yrmio@If?UpZY0UT{YURb6mHn
zb7Y^6`J=5`e!Dcbd|UZNCXa@{ep7xfZmbBQ108oNGEH^J6wEJTZWS?~ir@z-lHQ!W
zy?m>`s9hQdwF}vt68dvU?X%l+%TD19xbZyq2-~$L{I;cSls(_P(q(a`=#CqaheEHD
z^qS1cEKr}4o)>I$mVy^elv7;QsJ+iE=vV;fvRv9f)I_G|+$($qe|lWAvqIpi_JX3h
zZgipRdu=2?`!n3@3=RZ?Q&iUS;=%`@15rAJjG1?r`lR*)<IVK*R_10<cvXa*<0Z3$
zXS9dpe%QeKw=J`F3J1XHW&_*N-Yx!S-&yVsuHfrVeJYS`_<~}1yM2%Gp1wtTdV(R|
znm(l+(7yDY)W1z_VlTg4{Qw7EU@zR!_|IFpU#|RfX_m)IT15Yv*q7wSeZI1OEBE7-
z9~WB#U8N&kt(1{x%0Pd;!IPHCPfc8btHe9n*A6U_k_rQ*{K`@uPAmNBcJ4bKNh?&p
zwKqW)Fg*cUkq~xE7B#rIe#beDLp<xP#XYCa7{Xhvzx<57Y4+{6;@)ClTNtbcX2sZb
zs};>;UcV?hU)1>3PsLM|nhh<=g|218`<9<Ih;NhP*xMgW-6FZz<OSm20EQb<(@B>>
z{HEm(4R6V?$hWQtIji9<rDsS#2mcoJ(Z;vLDtj(uJ!{Ve-U>7qppl}L%j!1YCVwIG
ze%J<UT~rSV`LTll4??alV{!B5KBRfyoI6G)wfeXEvgzA7jsKPDHVOYsI?;wwbmEY1
zjURnD;C7i^e?RJHu3xTylc~RGH+1KqwJ&vdaQlVUHuqKKUap_H^I7RgZ|jxu8xO_S
z3vMKrj`Fsu{pbcS7wtA}2S-%5;v&;ChS=@Nq<+($lC|NoGQonj7f)mlwrce5L&eEA
zau0#KpcPzC>06wpdkOhB{9gIYz4_>-t+-haG7gGPX+2oGxK#LO;Jh|mx)gU$bPwU2
zNp7~+1K(b?L%5**Ze&*AB=7QPBGU(+Lq~ojGCkYy#YFPC_g6ciT{hhqdQRm-zk<L-
zdd6yc*S_YRiet@7V?Co?X=Q@JQ+Edgnvcxi*U?LRgz!G-0X_o#DQSGJP~{dkK3~*L
zneNrCL-SMya9+;bsA#$r4tU@7G5^xuC;ZPUhxKY+>9yW1Dc`mX&x7|Ya~vIt@cH$L
z@)XDFQ1o%%*w|Y6$jJonSZw(Qm$e4NqE%nUGa-Hn@l6<BK66LLx@6x;{yYUeOy*AV
z=kPiM7ty)MbnHSzd|uh8jm+>GS>Zy?Gu|V<cEMb*KzCnLXtWg!4Ydlz8*rH~V05nx
z1##Ev9ua$IhoCj<u*s!8dqw!t_RoO9a`pZ6a=Y)T%Pjb={8`@Dd`FWD1L?`v2E5TS
zldp-93xU6Vsa1~7D3`&($n;dBe;D|}-^*SDI~rq#TEQ`;7IGc53thdUjf7nj<r^XH
zTEehf+8XdL_o;m+8H@L!M_ypxkW`F(%HUr46!Cv0-O$^Hq#Fb1p5R<%_}}86pl@NF
z{esU5Cz9U>ZcW_#WkqHrBx`Q}w&kn+HP+A_o2NzkF`ag4Tp8ypcv_%!>r^JN0_T!%
z!`<YpD|MRhYYT7uzHkb6Q0GkeF!HNPt$pyYHS?CyqdS|(oP<HT-;d;{fPWV9eBk_0
z^ls!Qe<mCjyr66A*+(1u3o5I3zwAT*VahS+T`}pmQ&*4HR`tc6GeaIx{ln3}5Sh{E
zR&b^o-JhaAf5yhJ-N&b6`nbX7qf6a%1r8>G<E8w{;J=kU+;d5L(+ZJWI%wu6bN_TW
zuX63;;?vr{628W?gS`juWyR7>qIBzO*YW17vrW=;(G2>vS!=5L!kb42yc}ahhlo#B
zdg8T?n0Vq6;kQ87Si(-x4nuFk*Hga@+roKY{d86HEeFloTi`883c~kSd;OEdzmIN(
zbI>m@w4zw|1Cjj1&*nD=v+Xjz@;$rF{z0bA+Sj(MyjOG~IFxm?!0L+HdhEsHM5Bv?
zG50uQ-^P9d9B@IG5^pv8jLSHmD44S!`NFgtl{S(eC@TuD?XmYi{^vCA?1A=a9qt`x
z{~5sxT6CL%*-)K9;s+sjuutc^@4eyDSx&aN$8)cl$bDvC^zCG$Z>Io#JJ)gUx{0&j
zNX~a-IQzj1hTffDRnCPT|A=$B(LoI@o^UePJ68L0?bA<aZ+<O%^Ch0RqOj#U@k8%C
z@2`BnLP12RuAck;tmG`Ux#*4?#W&lEEfDw@v|ocyk};ADBw=}FkHhz-w;yCei>3~A
z6*RN%&4T}Nmgk)UVC-%6Bm<-I@Mc(Lt6fLRzUsQjon(mqHYYys%atAy_zY@*N3-}-
zq<<3oqyhFln<-;X(gAA=V>v?i)LQ?!n+qka=M1@mebg*ZUQ$C~>xyqw-ly`^mdLEQ
zj|*r==}(|bx<8rZ4whX<ok{TZEp%li{?MtD{UrW?c!E?X<p!$0&pB{b+#|_nsk7dt
z`)#|<dF1`R)s4-;2=Qguo#|>?U3s6%!=Gtj5b@=nLUzo*!IN9M+il6tGBoV)q<VBH
zovnO~FE%W=@1SkybeV!*=hhW`&AoKWB71*>oX+MH?k&-Od~em8Zgdy8tCUB5CtqvT
zY39cn*E-?{RlVtIO6mw~RXqMa#2+ztHd+?~ftI?V?yl?a=3Fo|B{*zhWltSnXRThW
zu%~J(-#5B$z$tVf^+g9#Had_B(1COv=dqhOkB#I!Hiq-qM9yQka~{*(!(4NB^9FXz
zMyCXqhl5^IKYD(UiT;c>bfMEPQ#hiZ=B#~3ft@1GcHB=)Fn17~?a<w4=pnjS&ZB$f
zJ#?>pfbNw~z;74$od|yS1;6dP3(khR*U;UC_RZM%)f=W8yjfN^+B$<fC>pS`J}mjf
zTkEtR<EQ@D(N5$DEZP6S_6oYRw}k61Q8)Z+q@8W<p$n{ko!^ksDqItdCVUd*qN_Ik
z9rXiU9fnWF<}cLS88Dxj#aSZm$l4!=NALy#uLyVte*<p~$dV5{Z}QC^aIe00K6f-<
z0clz_R^+RxD|eqU<yE^sA-uz>16~Bx(f=s2&L*GI;$}^NXBORCrk!!TQ;)FPoU{)T
zU+08VuC<STOt#BTNG5Iza4Au|!nj+PPN*tuyu3i%KJ*VgP&AY|YxW-b*kkC^!P}!(
zaK1m+iuA?}JWpJg6X#K!Xu4fbd^u&IJI26p95Cz>4k)hlx`YUKLhumw`FP?;D|r8C
zOZ>`)ZXh?tyofJWIq1N&*e|YN9hp_g_~~539qEkH0vGpTR>tLX%?amRb26<^=a%?d
zH+SvY!v?Z+w$V9;^p?hc{)YHge!KX+Shxg#3$%4Zf>pR>Z4X~>;nN|{unqn(=$Z`R
z&zel!;^y#R+VXFrgV4YQqJ_L=7&<6i)0|rNgB=G+!#k7}F5LEBapC+)-x;>h-8gJR
zyp^&$EzG{IY2sAt$ov*-9rr!xvb45LT*w~bVfGcWrPrJgtoo4lG$)ks8^KzUz@D(V
zXkD`SSdv!OU*BZ*?4vEq?A3?zCWq|N@rQHs(H;Mh?_poY@eKpNvkaYhMek!D<Kv{z
zsXa+)^Z8aiVs#DM3vb0ga5uvksQ$W#tuBR^4vX)qDvdYsDJO&5eJ6bz>&oN1Xm7Zx
zEWYddMgQL8-{{FGElLWLCzx_1u-$jilR^Btl#aj#-yTo@)R%Fe32gHH+>=>!;EkS?
zjcMzKJzUu{{Gc~Lo|3}m8(ot&da5ZWQ|`J)uw}K;QycY9+T^Jt>_z`3Pd_K^g1Seo
zu;Pb*loW!NW>5}yZ<Mpav&iwUbNpe)-&(llM$URk7G-U5!m1=A<b+4lO<}w~b@Dbj
z>2^8(b&h|Z<Byip=J;EQ&t*N2co#y~GvSGloKT$muEkIO;}*&382AoPsD-C`tT~&F
z@Yb5JkqIGveZTVcD;<?^=6|&XjV>U~DW$99E<rdte@T)T{`>1hD<(3Z)`p93{!3i=
z%0A`eXXc#Ivz)~n{YV#kr!?^khwL!!|3+HS;k0LE?@aYqjqcO7HvE-cRcqLT&B)kq
z=tphPDr$JO2l>)<g^#ycRrLv3!xnnN+9NREYV<`OdMtBv)`hMtI831FlKUkNS~L?q
zKUE4tUI-qh?1x1IwN7cSo4wx5Z20NoYpXtCZ@eSRyK3}m_7$A(W@hg!NN0~TD{+PP
z2+A*7Ogn<;ru#VfA$0G~z+A24k=Z#P6f7KQ_){aZGDeDjMl{yYnpqjy+>h;Y+|Uni
zt=yyuVe9C8>AP_yW#0IXvg_hTz(0nL7QeYKZmRz$*-<z2nsm_cqyR7tCWW~p;N3@k
z=*-G~*e*BycH*<H2Ub<+Oo3ib5N#8FvDcR$9TM0IA`G0E!QQ@AG_|xpJmdNBJZ%;a
zmhTTww+5~IN&j>k-*l^WBKLCRR$5cWt+wu9yj#bu!TpqVhj3x?k_66sNpFe{uMH25
zXkQzyAN<E4a44M@x5%Vf<qeT$y?EkC)1iA#rE91k{2TkuGj5Ug7w`_uWM3uSZ)TsR
zKCy?~N}t$c^xfL=TlBH!T8CHE2fb>C+GYBO%}8kXi}Yoj)Au_1`N{IF!NC(Y@b26^
zSh6&;wT8hz(Na6w{k-zB9~g|h+7yMOcm9sj0{51rp`4>er)|0ZNqCUNCpI#*Eixzh
ze(<%`%#{Lk4Ucwd&Ap2=ZAoebKAlKlZ|Vm7OpI*FF13xab5wTiP**>djh)V{s(R~1
z!V8gen?hdefrk@bRNMpg3H!89Si89C417iQ+B7S5KYX{X6LS(Q-664O+`|50E9o{x
z=Oz2~&8c01{=Sdg(~iG9pl4ZEZ~=6-8$FtO6AX54$?s;LSFqfw4IYPY1i4Mo;co0o
zY>;0(wNG_bzN~c2m1)C!baw3jf|0$Ok@#RyDAg62kvO+#eCntA;hxBziSue!CVh<D
z&#!l0TVZtHz(d+QuSs_sb33@UvGQf5qwJ6Mtqb;P<L>g)$t}c%3j)Fg?hMu!jpfd5
zhPR<;+*dy>8e_ul*PJ!9i9LDg_2`oJxxX$yxbzL&<Al@jqf9BdxV9&8wYkTimoneJ
z6G^PUd?(_KJv&Fw^pb5^ip+c2XzqoXBD&+7%z3Qnnb*z@Zz+3XaeZYUwS_yLcN@~b
z8q`@|Y1oUJ`$u#&0k8aI<OHTP))sY@X2n(WHk+ID&{g*U569dCJZj$q6fRoZW6soz
z6z9)MdR*sSTc-s46uIhSR5lvkiuBTAPe}I%1{Ud7Ejpw^<WB(x-zeDf)1Hmrgw_Iw
z!eCf3PFaa>_;MQ#gyBbpSB?Ew;tz|%Wk0O!NgU^kV-AJF^<x(?=2h_YPSJTMTvs;8
zoQ>xBwoztH;^V%zBGZ%Jgr7itd^PcRzUIV*zD{_1s`UoUe8;wB!u_(ZLpf8AwPamn
zJ+Y3pWM54A4)VZ>?z`~2kG13|+yh*=zo4HFslVK>v~cd0Y(MLf^u8uN$~m7f;JYC?
zgF9C4TKP6{H_&n{1DQbXS=omK&{G^ZmTqKJ-N({%E+!|L_b3lCO7ZYcn0o|ZdHJqH
zyeZM~296oJ!(nex!ntKHzn`am%)DKH*SXX`!C!}N8s7~*WFB(9lRPvn<=9YUV$p$9
zwmN7X8*1wG9UCf{*gW0oB{ANO+HY}A&STFN620minf)+l@9EHbOXp1BFjI7oI|u35
zty#x93VdgxW9<_gwi#vG`_u33=)Z~w4+nF1=+nLm-t`&G5$+)}4vB8BeZKNt`JW0`
zp4L||Vr=%3Zo23#dT_Bj8CUJN``}(c81I8|)uOYwhu~g7KJF4T4~j-k^r6e~Y{mWp
z&NtAcEZ*dFxA(rj`hhDciGzLKdDw1(2jnI7(LOh)aDI({5SvGDV%K*U=j3hZ{(lp_
zVpUCR&}oExOVuU94_6LV+u>PyQ+QI^&pN_db6B!rWlvNNUHJ(2kjS2_+E6%&ak1!s
z6Z@zaMGtl7B3}B)bZ_Fgwbm5Q)isn+%~`%d=WpI!fK7g$>Rs_@!2`%ZZ+L21!Q+hU
z29*Ji*%|bIm@`sv_l>S#CqMRQ1q-=X-s!@ov2}ClVb{LW@4#!(h#sQ(ZgW0F9%KRM
zb@QIWTQPcUJ<noet}ko(jp#qU&a`P}?(OgcW(-IF3ial8Zpgp*RJg!vc&4QjCEa>n
zx^`AI7X)#yga137?@BwYvRrTUkvVbMqHimn3?hIY*s!dzVy)WV_&>ur`i|XfO<D4C
z#rM>gNek82yzT4@-xLkv`x&~|(`+0VI7n%1T;W|0cn{xDSt)x>i`P8!D!j(xF>lxW
zgqBf;<;Jer&9QA-(p-^h_7LL#pl!6Z5gyEQPMT-zc9$EOY4D|Q>f(~2$g;jF`5@@w
zyYeT~uCb3<Q)nN1{``x}!a<!oo;7;nAKLLc^wT|T1OBb~7Y%MO?rN)lgZ3R(4)U+Z
z^ae*iGX;0efw9=8Us4uKv-_66Q|XhrizMx1npe)aU_ZnEi9Iga1sWIHtua^zkIk!U
z`;z9--&2`{{fxc`wHBMb1AA2V4juAyM&y2@Vt>(8z4@W*ewO=8B;WPMxUJR{CroGP
zfYMUeOW1ry-)F^s*07}tlbgQh3f97d<%`4r%KQ-yEA5<l%ps){?~c+=*lLZwa6Y*a
zzGBaP=PKa0U=B(zBJ*tCZ<%k{%Yr}WP3$zkqcV=PwC<H2pos~tu8DnIk>tTv%@XW@
zOpJGRoA%Y@LwDkM3$^Ye&&Q@-=7;LkPxV2QWDjWnh4X!H02V3m7PTuK_w@0q3%;I1
z<n03w*fHf1%#<EExOnj&G5>1RK4kR%`9y2(K($+CDx9!J({A#rEW6(WzKhPKJ}OK5
zp}J;!&r{cuu&=5m0UcVr$qi*c$WMCt?0)1Oq8}Y*p9-%AdZ9bO$;*0Pz~LV<?{|`Z
zSK_3YbqStJtxed4EazU%otQczOuNRhPkKRX_3(nq9`DFP(U`#QAZMFV?z{_KZSJ+f
zv4c04b5?C-jhz+m@&`S0yafp^fBApCZ*yy{<u8vna1m|tkJ^yOUG5ygNq&pJB{i*C
zxaassxwrA#s_&9e?T8a2y(9d=hh5=OwK@KfC;PW!!agr$ot~BGRb0dsvN}iRM4GZI
z&nYZgCjPztIlXn_-{^lO{&Jo8zstCcA-$#Y@J}axdrbZz#5Y9am6kNg{%7qnY=6Z&
z$`i2${zp@G#Yx=fdh=JjK$`ZL{0*d8b5;I(m52O~JLTO*ylDe|r61(Pk5asUO#B$q
zmZ%InzWg0;fx`0s<HRU$V^cP?)PXbc?f7?kN5!`j|BB;}tzYr?+GXace(Ec~s($o1
zjfrm{zv^#r(q69L@q6_peoRyL=+|kRfq`A#=$+mI<<C|9YQNksI_+2bdx@ugxvJmk
zU+x0IAjV&%d@7H$V@PZ2r@uR!vTu3EKgQ#VSOe65`7ih1o1^~UveT|}z$=0eZl#UZ
z-)@&Tpveo2C|~Jg`!D|>C%^jNj+^=inEGS<(f&6yWskb5|D#lYd&J67{~PSGa-#J|
z{Y@&@iKqOW1y28|f7BqafnScP-*Fd{-lh5-zt=k|3jdLZP84_>BUbiXz46&ch>yar
zfw*Y>?S#{|>?X=l{bMLs@zMID{wjq#<<Y+E5@-BWzx-a}qxtb?brXM1=@pLuh|)Rb
zEB)p2ocM?n-=Ht)qxDDqCcVbj9_uVqKmNl_*~8zl!}@Fd<-cnDhdc1^ukn}Lh5qCh
z)qddz{s#O_e8)I`>hEvrkMT$AKkNmL#252bf7-n+!LPqpZIHhydsyuFt9;`7b!bj1
z-mb5osXxYV>UaFq-_O(^<2Uuk_>ZW)^rz8laHF58zt^AH9j%Xe>d%bU-#|Y3qxDDq
z(fZ|YApWZUXI7~m>W{{o`qdt%{5r3LU*xC#8QsqKE4}<m-|nQR{ETS*$}fLx{iM$f
zE1x}H9vAVd--frHUifVMN=Lj`<!HQ=E@Gvt{_Qb-)!(4<ocLYJ?~HE)>6;Wz`x^Ml
zZ~7nOuezL{e8d-DmA+2#q#vVi#7c|pzx+o?9}}N$$|HTd*WgoHgnaGBZ`z+0t>5WS
zT5ozg{s`&ED7{mkm-6LT{j{%dL^L2KzSxOZdg9ZnuJVT*zmq>r@NaO+qy2rAUhPx7
z{B>XAZy=oV1DE^n_>YiJ`JMQ_RrpnYgThJQq;$RG9f<Z{>8USJr7!V^?YSN((fq~#
zZd3MGo%x@t`7i(F`S19hIiITePkzn+)E8ssKlP_J5gvmNsiyv3{75zJjqx8*dfMNv
zc9Gx9H=2J;#7gNF{yXXASNir|{}tsCpBf@w@EF5a{xIb=#H3%Oa8sX4@KSjm7yZ%v
zm){$uKQ=u|R{h!^IP@u5^_%>BNuO-`uXxAb6cgV-yy?FaFMk#BYM;`}A6uUMZ87=B
zM64v!e<!{CO5fh=x67mbjE%=de3O^{#>5N%V(=kpm*SoNQlH8*{nvb!-|O_hf&Sz7
zoB9PmJKj%vyL|lZ#9!{e<8LBf@M++yc+-Ce{wlAEbZTETUggE6kM>{jr1wSpAN4DJ
zyZY-LrSbLca>{F0Iry9Kw>#ylKGT2v(Rii*i~PO#;g8~nQ=jp>V#-@UJyE~vtHK|x
zPyUeOZ>JvWPc;2k{f=Mh8)E!N@Q;b{U+zEY+bPc?UiGQ{l;=G{xW>;J@5CznCchIO
zQG85!VJ~pP@ASv3^zBZ0wD)p(hgA-Kug2dQZ~O@c{xSX|)MMtKlfDXnbbhsyzDeP|
z>AeR2F@6JI+OPFr<rhcOs~r4o@~gdk@yA~tf5&g??~P9|^~d-_Do6G4Rrz7^Mf=}Q
zc~vUMsbA$cDLy8>MZSiZ^3+~ej6Zh#mEV-7c;&+%qCDjv<M_k+#`sm9{O#URCcWxw
z#~;lf#lLnh?LtRDwEt265huOl?^6{mPvN9D{ZW17$KOZwkBRYLE?@abe}r_3Z+HA{
zQNO<AkM>{nkv>l88&sa3a>?&yecY-1PJG0PS9yvL<Bygv|Dvd0>F{qMoy)0T@q$Ne
z`KmwGFZf6O<d3WJ(od&7@`oJ1Q-9n({9_nzXMYzLa_F1duk@DE_r@b1)!SYd@t506
zdifVP@eTS$EO!;*s=pn7JF*c?SH*{kH~6S<((l7RhV%`5mHvprWBhIM(;jDjTW0)Y
z{6~~e=}Ff>{wjTA{2}>C@2v0E0@knXG5#v@EB;-<fqRw`t#2nrC3-oJpEqIo`90Bd
zHepAj_W-z`ReVgG`Vi}%>t#;I#E-$>5aYj2b>VkYRw8*Xr&T`A=kN8#R~)4um*a_h
zN&T?%R;*VaqUAbyAB>5=oxGRh$vcubbB@NHP2TosUfq@GFZHdohOe{-`lX&W-<hLM
z&+Pxz_O$+~?SYM{M+2Ku-!$RXsb|c$|1JF|Y#~02_^rfm8+Cd@mg8^1zukA%mn*&|
z|7`ff;JFhIBX=}oB_GCDJdM-fTUr4ho{_Kn&7Ys!2j5uxs_Z34kPn=}J)`@0+sN+Y
zr?z%;CmTH8HtOQ>Q`;`$4jga0rT+M-zem>G^5Xsl?qA^c96vSS_r9|p$zfM*-h@2)
zqgkQN=mj~o)v}};q%&mYDvURrX{UKNJ@pFj?77GbB14#Z*6x32zwB=0q-!qpWS!St
zlH%k)elYQ)3+KIFg{7YR3m6#~EZ){y!khhUe}*q(Nk%v@{ME{ymD_%_Yxsf6&v^fs
z?Vsk!P+3N1u_vnl9V4c0g`L{&3tTv#P=(&;>0es@E3L)8eZI`aue1h+@2Turl^OOT
zAO9KU%{j$;mF{MFH=Q&2OJoCAeiD|v<uu*js-0h!j<-1X@i4I8u(g}_pWyK@u-~xl
zBJRNPFtFe7_w~5FxPO8B7q~s#<Nw|V@8jy)Yq~c)g1rAHt;LdS_vbVjdG{qbJMvVg
z!nhahqqeTKg2jw=jdT`VINxWAfj2x4X8hr;iH<*V2baem_vP`&eR=$G+v9)2gHG3i
zu1!m<3V*IAqqE<D?$lGx*!Iuv7G9Y#Meg)MPo{a>i251hKV3NQF?@EH$KX#BHaqjc
z9^<#n7*F%}^YvT&7URGeuj;q7oAJ;XPxH_4^pi|0V>;_XPezRyW84Zmbxva(7xw);
zW{j^c-x=d2nc&aQ!Jk)J?J@4h7{gor*T&a1FDtZZqZJZvyB-1GWBVS)<`w;q_El*w
z^rUB>XaN_5<Je$V8!w!9hm`kn|DgjNA*-Lp&Vj>H=Jzt?_oCPQ_=&{-Sn{>jl2gxt
zdpW)w+%|6JEQuJr$2O?<E%072k*sb{+D**U%k8l7-c{{8f&SkwpI3t~&ndxV+YhH0
zyay(Fr`5cIkL))80rzaReZxGfqBCe^sEw!o?7#x%HQ&Fu$9HV%#Rs*gV>2*OzkB++
zOrLC6%s3^u7<gF0FMvmW3_J{8D6ema`>xJEC4RQ#72vUt{OH-l9pIg-r)hh`8Qxqp
zS7-a@^unX_PXoFy^aK(w(;DhDFzL!K{SugL2L4+XZ)qLu-QddzZ*cHrOY6lbOj1wU
z_-<g6;mH6la~#;t7<FpGCdRQ}l&-Jb*jlW%pB~Gd`Ha?}=B00=dF%2?_lEpmMkkOZ
znyzs{&MaB;LGxN|Vk|O+3%r305)7oTK<Q%EfoaY<FhgsC)&bUoyZB9ySr1-x)`M-(
zgW0^vx;vpsoh@6thn<3N1e(;@vh5=7KqoY*v*qvWaeHz90{1U)%U1C3#S8bFQ^9J{
z`Auh86Icf}onTGa4_-WM)`Sd&iSBCNoS0>=37RX>H32>(Gk-7iB-!t+j=y6LYeLrm
zW2f^AaM!GZi?_EHMsb&Krf)ln65x09@9D(>%1!JTd)1ui8c^z;_JwoeigGDGpmO~g
zO_$5n8mDnP9=7tnZ0r_)vg456Ou+H!rEX|#EA)OZ^nNU~wiSB67kWPyTH6Y}-wVAT
z3$1O1-tUFpkA>Fmh2B5sajy<t^WuhEtqR{KyLcZ>cx6@2@}Fs~gZ_Q#It1M>VN9oA
zru|*nvtwxgbd7biJsRt1do<S3_Gql5?a^49_WWF9{p^-azp_Gx_Ge~yYuqo7qvm#|
z=6a&x3p8{Ozdepw)2>+Cd+G8O@YcExyz|XCir#A+DZ{31-p4v-JL5>0J&*f8FW!;M
z_|aRgvu8N4L-b2H!u~;J=bEvE*NJ&^ui`fS61=^eIV!r&T<y%KPrrB8w6N8`lYTOe
zPTvw9R9ZvV6$W3S>2p`V4a$4D9nf{~Bj2IvrjI7yH0ah2c(pb|*XOjB_+Rnl8oHiC
zzjJ)~{yE@pzI&F!v12EmET>-f<JvFbSHJB#Gyhq9U$J$@*0*^3fzyJY>AQ```KIr{
zmA-FRoa<ib`FQ55==eSK-Lm3*Z@b5JJladgq4k*^-v$Pnm*_ZGo(nyF=AIS}SD0Wx
z_=T>F)hZ*lJ-{H7_!&;SY+6o!+3*m&kH75D^9_{sgfB1rL^v?)ENlKN&t(le1wBXC
z8}vL^<*V-(dOQ!BHD7Qc?6mfEUFj{@rk)*JZ)|-_wn5A5eYt!C!{IOX);BSiTw`hb
zoKvS*-+u0%;q1HnIeVEI{@I?~t7$pu?^?B^HK@6_YGbfi<LTS9Pkak${(hc3?<0Nk
z6}F*~z0n>|rvFl(+~cjRlfdNk0pP3j+ud_I!&dH>(G7owr=RaqpM3b{$~4cl=0N9P
zQ2XJDct<c2oge=Hx#7pQ{(#-rqu7Ie7MX+}A(QYDG70;UNodCw>@jS?zB&Il$SNE~
zR^bS;3a=rna2Q#IL&z%p8d-&3Rc5DF1sap!S6I?KXbSY|wk6GB<W61Enf;u<c?5ln
zdz`Q;<PgRX*1Dv*Kst9rq+iE(teqzQkDRMZ;9+nZK99T6O_dHmFK2hctR>Aw7x`wP
zry(%B8D8#m>wlq#rdGOhJDbHjGB?25jodpjFyvd(d`(DktD1KV!5`xHFRPl17O4Em
zzbvo8Ufi?Ti~AAw;$FgD+<xrEwPP>tDE8uxVK44Ybh({JZ_9g==`(s;K0zMDHJLsm
z57HNTknBj7>mc^x8r0UO?tbP4^dxy#Z7h6I<*Oabo1U4YxmNfPGPw&=Mx@l+dJ$5m
z1ag-*2457<KXNJXX}NAzJO3taO3&Y@HvLV?>hF3&WwBqFLtEWyH~sn)KA1|wlM=q^
zIhY<7!P}Y@nQjfi_GkD={kQJLA6DNCzthy*_cdnd36XBmG03@l#P6s5J?{99dUUH}
zPa}CxVAh4t^Usmjo!lDOEFDBD16h-2#mBQe{6F3|bPN4Tf0Mo$y(#4le>1pe<=iA#
zCBGQ(Fa0)pQh?XeZ-?9HXAnKn7Gs>V=AO|*3|#(JW8FXXCDpm2{+6Zc2fTl^@C<E3
zFIP%rYFc@z-!%<BxG>|Dqw!j`p?oQPPRN_)eU~u-wrMLI9;xXHr|nmb?-2M<(M$Yh
z#hA$S-1}9>k$Q9j5N7IX3Lp=S+{s(z>#28HWNLP%bX?E-Rr%(Q`hgL2Rc-|b7;EM{
zd{oHFCK9$CeXHxyE%Tu2K3d<pMQI{4<KKi1%fHCv$+D76y)#_bDLgVYu-)L(%swyK
z=|aI{_0k_2tSWtrv+C5qoKin@US&;B8A*MCrM0*0wEH_?jc^b<*{_0olNpnMf0gvW
zVb|PW`6*%2iwiswv(-oN{kyCYlZU@nF($OsS`cco9-_}d@nE8hp~k@BZ1qM1^CebY
zWSZ;Q6FYr@`EBJ}qk4eEhbx@M?uztR)xg_#?}g8E^02G+CcjU>@n%0XTy24u3)$?-
z9=@wiyqf=ezAKPTY~!v(cAuVwM{eoK{M>hO|A_Dq;qaD~zl5H;4+`6S@PCy{??DN6
zir3>lN}4Wo(R)n(&WgmsTJ&L@ttc?Odz6jN{Pn>8R`iGVrCqCtyRIm_#5M0o#nSmF
z%a6@JReqknyodV(c&}WflRdec=C@Uhj7<06gs#AC%-Jww?QQzU%9F?%R}+4F2X<X?
zBfsq**|9B0?%B8tI<{de=P~PB9ow*>bDwC1?~~FxtC9PNW$+Ap(a-6NOiemxc)ujm
zJT=MKbz)rPeu1$VFMIFUY`(ZUJbF&zAa@`1N@Sr=c12`r{!W7**sdH0{3I(adi?bK
zprtW~Pkg&@VEEDU^`T{q<#J~%u`?2x-ml$^<$-CRsSM;}>u6U{ytR?(i7%CY+X|>n
zxVLHy$DJ*2MwjhQ=F+E{Ki*63+2C_?^8Hqh;z>8LY(Dw~7ghdVVd^V**l6QTVqf@L
zE$>f?n^}k4^jmtl*TXLzglDw4^n1WzfwiFYLFA<#LVk9t@1W$n*R*}btN!EPE<RPQ
zW3~>%uVf{TccUXa^Q7jv@czQ*nRC$Zz186G)Z8c7FO3(iCI2_zd+n?75#MWMdcr3f
zAL(1feO}{(?(ePWNUw=Z^?qjdV%TiL{f_1+c1_~fw;^|reKXQGFqUEUiSQu)x%7*>
zqJf(uQxm5Pj{4!2eeQAMQ8u(DKVyv<hr{LT&0I3@N_XIYM;~V{)f)Q9xQ}0MEnrSR
zLfQr3c04l4#`d%332u*I7;3Q|#_i?%C|@7*ZKn3uq_-}7o_?#XZxi_Da}NDE_B@_U
zOX}RqSz~ujr1nv4cpV&YvGx&k7(H%<NPC0Q)jna}h%SI^bX{a={;b+jc@7;O7V;3`
zj(T{<vG-M5Z`CGagPQN%_*>A!c@g(La#L;{?r+luH)TISx!<A8zoUIA%Fj0-dILXl
zT}M6kuIkg$gkxqMoR`)QxyO$Q%Q#y9v#;3ro0ezbcBDe<s)L(-Z@2OGn`?@y&=tB8
zyk22FG<;S@dt~asLBiwVuQA85i2_V&D1U0lE9i`5EsE8FT$&ra27RqtWq%nvALpSR
zqBUbKd>(im_j~hxQ@*|^>??py%|_>~;y+Ly`F?_q+4gxy%m0aS$V&DE9^v;W@+$ZI
z<T3c)H>KUv*vi=v{EJNO|6R2e-BVuZQc(1Z{QDgonuUx?ATrgH$y}+09yFm3_a1cd
z9`Ks`;VF6GwiTHgZ*0#b=3dq_y^S!VcfqC+@tK+<lWBjr<*R#zZ;fHwmT~E*l;X(L
zxapb)=vjC|Fjf8Z>siqh@b;bgzb^lPb{alm{K%H14k2B<?;f{jb$HZm@ZEody?No9
zUqb89E2Vk28vTG~UQg|}lX>A|pS`&;Y|C<p?k^7o1L(G0|5WXW*MQkZd%mDgYP!p1
z%FE9huKjnBrxpEe>{qq^O!aLn8VB8q>ZG0Odt~@q>@|N0->%uOu;(^=cI0EAHQEoM
zcm4x?zeuJxa;Nk)+xJ4Tdg8u!g!;VK{2g-1iKPkf2p_7<4stJ8x--3j{d_?XzF_wB
zk~>DfLej|6Mpu$)-*nGor8%5?nAeT9wPrmuvY|8lj}4!dSOtt0cYGTifzvb2iw1Ny
zGuK*>5o#P^>t3xh_f?ZvvqO~kGw83;(FhHB$@HN%Xh|Pa<gS56r_0KFgPJSSr-&XD
z(VDo&49w9Tqc*5dq6aoiT-NpE50xb>pzZgI|DSWnIQnoq{i~s0bH`1!rbvEc+-&O(
z>8M~|l;7DrfHvkt=1sNkm^a&+vUpE0MEr;)8>FwxvgS=QwyDPQ9mQ`n`?z(??~3N|
zx)gX~1{Xer4Ccbb*67||d59lj+A`GBfqT?bKR_31J$Y^YeqRHJ7DE?;N`t#qbgHx<
zVG-_SxUb>6i|@63-F$E6>rtJ!OZXP^8_e$pem5o+Lw}l~D+|*Cz+3dBAMP~lq+bk;
zM%GL8akN#)*T?r7;xce2;l7q{ijji{Mx1MEIM>YOTyuXE*ErK5UxV(JNk-0Ss`mfL
z7Gx+)vNoxA1U#BUDR<lQHj;$_Zs=(f{EF7GuHP(p(0(s^Q7rVdx%qlH3uf_+A6swR
zz#aQAx>9s!&ixBv#s==#d(n~7;+{=hv2<HY-iNrsQF$N2Z-~nK5Pl<i1Ts(UMN2w)
zQlc_Dd)(7f>I0cc&jtF)My8Luf|MU1&%;@rd&Ft%W9&Sf+tKTan=~ig)06%o&`)>&
zEB919p+1E6S|dVxtWhf8$bONRu+h;niK|d~ylsS5wp8S)&*(GA!0u?S+8u4Pu|Epj
zQXU9os;|gp7<nyhk!Hy*sqQjTz5|T^E|6{HwGMEfip^Z#7I(E@_Kc^aZyz4|BWDj9
z9rM&<@C;bGg+ECp$cwe4d7&T0*z1+s$C*sFuLH8#i$4i_y}3!t7z^wm@{7k_*JFgI
zbezr78aATiZ1%<4Z-Z;#ZO2*3Ck@z(do(aMc8-Dj;Fz&8w(uCM8=bK#i5aV#qx~jN
zX>_c(FWO02!(wnNTYVF58C-L43VZ6dzc7#?Skc$Ql`RcVr<S6xhCWEQjwzpcoI1jm
z%hKL2briTr7;AT6WlO;b^4*NSx7<{tPY3xh(meI;26t+SEiaY|oJ2d-{!nRZ_0wV5
z^s$~puDs*aj;y}OoP0tV?A5sEIE(z~>B_#PTP!a!T7k&4)ayiFv7_<h$;7k2_Okcv
z2G2v-86RI;YmMMHO1iW7jvn4p@ucYC+QuC-opNu6CT!RI@U{4IP2A!g`HCZ4G8~HE
z%eRD|8#~Y13t97^Py8si-#NZmCjLL3rz2KvDC$u8jK!Ca>*Myg%8o|rSoA@CEs&WS
zvT-(w1F7Tfx})<>IIKA4qrq*1=g1%$d<Mpve}v6md7#kLZ@1l_1&uLsc1oZ6bvu3P
zHv*XsUKK{mrM*F&t&|S8Z7-X0!Le(da&rhTQ5wRGO<X5!soH=H&@h#$^2R~a#{o;F
zjrIrs=xAGsuTWkyulE!P_Ba0x=e4DI9hu&LuJ9im`Wbi-(3&7R>CESJ++OCgzCPw~
zI&)R=n#Xc`_{HN-(Hg+|FhXkp-%+Cf=+)Vdd$ih3+0ujHA+42!Jx;!y9rD-kjb{vu
zTq9#(=8j~pv`$I~nDhrrf7Qy31(Gwq896E=7isLeM|DU!djHS^m}7K&VS9!BPq#_`
z*Yz=FoAl@kME|7j0dM}#>JVGo#2pLiB_pfxLKZYDGA;g)AD9wuMdzO89dH-|94caH
zOggYIv_SI-Sx~p++4y<*#m{r*7hm!QWK$y^yeMJ+08Q)FH3ohRCLG%43mBZKch4qG
zG4RY3eG=S&Y4*5kYZ&1-L}f?GccWxSho7h{;0+}!C5)WpvAf>aS3l7j`oNly=*69l
z+pBc|_eAy2*m9np_JQ_>zil~?s<9W|R1Hl)@6v#G>X7e4c4;=Ur`p#{!5${}z=1*Z
z=Z@%}iLYBfvj>%)4|EHo+XP)7$e>Q|e-4?cU5;DwQ*A1LGW-fvqY_%VUvk&+jSO(_
zmF{8f-L5L9+R2Okpv&cyMavP*tP3WzsyxaV5-r2xZ+Yc457@F*H(M_U7jSQ8Eq-~<
z6y&So^EwN7S3H>4?Hy^(o?AK#cw0O;Fk<{J*R;+c?~4cX@A20ZVw)iUImu`76%UR3
z<zPr<-GMxoD?BndwzRym$N#L6-x`d35%0gj9L_7VkOAkMke{hGtgTo))*qUYxA`D%
z=H&ARk?A@JPk+!e#(yv&&tLFg^gHT~F!F7Af~z@$WQWYx$iaaNhG&U4OL+2fOuW@+
zrSpyY|7iQ;AF#iPkK13*f7oB&Z|rZvSo1~KzQ%!f%j1s!r0w@R<$2Q-u5nb|{!v`C
zIDav6dQGN%w9CdL-l49*qnxYoXn=!90~|aWVDL!wtgRTEwYh-vh45k4e{Fm)@MbQo
z4bBaivB?fu`d*C-O4m6D+X~o{Q@rDUE@HWFHu)^~&+Kpfb20Ht7;|Tg-6q_~^8$C?
zgbdEHpZr+kKsukD-!oWW|0s{8I=of_TvWY(-kj?2%g`^o{`_{wAMf~|bNnfee~IHC
z==kpm=Y$HCzjIDj1!KQW=Rt2?Lf+;p!rTFcDV^%?oZ}toq#fZePspQ;oR@2?c>k#Q
zS>y+9(2|fR$CMlLN@hsmO;?qXZHJ>bXum0=5PKKxl;fiu+QFS1<<Jg~!aGCWa=T9J
zqRKI8)yCfN;N@_qElyh>>&WqnKQc#crR|^KKi}(5#Gh#Vf{D`~Bg3gWemXY)5w$bg
z4!J4gKV7EEGvz{GfK#;G=YBeN;33M|M!8<<c@7$VoBVQLhC%+oC)5Y>2^Z&(=U&y#
zozBPPbHB_FH!3Ymn!)k|57RdIMa2h`=gsb8kL8u&*y`&OJDxFl^7i`o_fefsJLO3R
z<1GW*Io|!i?pg0;SSFG$F$&vkwXw6N_>+IyZ~7FW-|l~^$v@XwGvcD!OWXc$$_fAz
zl_eSEJjy$%^3*nGe6cee{}z2c=-^Ql4mGrOj01<IF>r_t$M*6SaEOh|bKt;TSl&Cr
z1^U1qbKrO7?wx`E?fP#B{5J&t8v_4tAP~o&<!MZ~{<inO8*24z{eM&I->#NPA6q4p
zE@2bBj(fmIeCd-G76wx-bb!~^W^smfb5FM6>F~Pvh~>^n`3Sv>U4ady{Q{Qs${&9?
z`=gXUrB#>q%d&{8I{q+v6Fc4MxWE3i-sLj9R!SGTkUmPA&Am?Z@!Hut*hl;r8+JVp
z?DmHyeS|K<?^xEbN4Nvn<I`P$n>(&z^c{v3hYrSc-jl+UKD0_6K<`<~rNFe~8}hqT
zE(WF=yM)}eDcu9j8D5otsJZQz(K4mKvG5<i^03O{p3Qse)$G9hjb8NdS#oFL4xV~7
z3!Ri+Z{e1<f%uEzulXE5cen7UvfCl71pf%&Q-WUq+`{_?osb?Z`;5zbifjvKSn+~a
zGz@rc;yw_2c<_uZq^-XTWFCKYz-PmIDtngy?{G=Vkw8Y#)3OJI4zC@y?TpjS-g&-y
z_~=@HzeDy9UVEg?d-_9UZ7=z9Uz={+JB_=nE$jHJ`4@>hEWg~#yK2!Rj<#{y$DQL{
zojv*8r+d6(xyR(48l7qj6JPLu2%LJ*llx&}Nbr_>*yC0(^@c#6|1MvC>SySsx-pP9
z>@llYasEm>Y}gal2;+a%mtWNCsU3bW-cs6OJFpY+WL8LVr(Z=s%xC@(X}!H^2b}=U
zB}%I>-;=&{-ab=Oo|Jt8E9n2om+s%>>qq;Rz@sKv01J7IjKzCGkH^B&w@|vv^Ik_+
zVgcWS_*?kC0KCTX-R{J%!LNAPQ=YvH-Wbl=)4)I3S4n#cz03IT!kvR2@(lEm=b!H3
z9v&WT-9MRoxsSL<2`ijC6?h)i=K1K!WN-1h;41tpS`8n{2Kv%WUot9txYIFsN!SR5
z6^5-^qnorhJYa_pIsvbQH=5qJ8=NK$VL_EEKRA<N!dieoVc<Y+D)-Xp=?jC`Ex(@K
zX66IqHJdS>s(WdTt?r>8Xkjjnu*aG)pB=rkHgl$)c@c}7#4BxVe&MRhq8xAPHCN{k
z1XQ2mRri%;_XGd)Z&mrRbwuYuN+d8_^E5R#tt#bzQ*U}&HSVt;uV+qNB0N1!b|M4w
z)8REqXU=y8n)!0)R`MObiwf^f^r7$d9+i!MMB&}b;)gBFT80}wyLkBeg0-dTUU>F`
zBTCcz=sT<E>G*4hx0KACpKDdkv#_%g()%4c3Kz#?za$@>PS@OD)bN#f+=rBg?>j{e
z-#!fdTc8`<Q+ucAo*bAoYo05w%82dX)s%ON$66OAFcfEhOWf7QU+n5KUye=gkhKnJ
zzQy4bcgRK3|NgrY0zHACE2JP-{*@6=UV8G<g51CUuQXTY>h)D#eDYHFz^)Yi{~Ld{
zt1IQ-|Lxu1+3xOvPhP6L*qh@@m-6>~ZvVgi)Z0%*-CZf&120zCzQ2}Vr7aqL=?9-X
zWhjT*TX~WB)RoW{7rwH-FZRWS6WUU{^1Gw0`OoK9zUaSdyJcT*f1bQ_%f7g9Tv+j6
zTrUjP#P`<So44ol|8c<1{nzuymK)cW(3RS4|0i_CwZ+E8x_iFl&jy-Zz{UJam0yyq
zC$_9ux9U$>kW`XXlCt3cw*C*+e)z_RZ+x)!KU;s?=W(A~1FZ_{QR{!X4#zF4?0T}h
zdr{Yj-v91JPj**!#kMontubFTx)=W!jjjZ9G%9gn;gRk7axAL6RC#GcuKjP^)`-9O
zzi4&GR(XZ{$<Lqsd_ju+AM1{W@vZ0{-E~EtD?N4#7q%>Hxn-YlzMenf@-6!mX2)Kc
z<7!_U?Ym_4SUqh$-94Ags9(a>)<bBw)pm9Azx2mJzALSsxSmvYVEU`X6-R-Drdn}-
znV~l%iv4Qsywy=?>pO3Cv`2q!7JoVEvg?S(_tto&EC2J#&ma5UgO~pw`(5Sl@_*<0
zi~lQg**?v-fr}V?5HJ-wklURrAO8bKq>f1Fj_bA-S@JDP?M~?Wq7uG5Ot=vH|79t^
z7=Bm6?yu4BFV6q}v-dvmQ61O)=&V*Ngb+d?*_Le?T?xqujHHzSTVMhD2S`{#vLtN$
zFZ#QZHb~k<y8>Zj*f`iBshg0*?ISTsNo>c&c2j<--MX#oD*ZVPNu2yhK56Q<{8F{?
zYn{Y(Hov5;8uQ?N&zYG!cUQYBLj2yR@BMODcki5;bIzPObLPyMx%aM|{gB#%=40xd
zSQ(PZukvTatO#P@6v{S>LZpMpn^}NP6}!bEk;6oi9*h7lN^N>Etnt|9yiY2<#Sn{(
zzMNoTZEayNht2<TFsHV#ur?={(HD!C7;;5i5%&yvPwBsl=8ERFUEFq&+CuyiQ<+@C
zAzY#<0xit|#EW<mZ-e>Xr*v=6zskCFexjQ7*raQs@+-+RJgBz}R2Ynf6Y-L=(G2j=
zEAW*O`aBfb7A=}{zv4bcpU|m?UfE{P2Z?wT6>fvyz5QiSk-g5kW?VDGcaeZa&Mmum
z(}kNZY`R#RH2zsOQ#4x})Stpx_>oA_w`<=C9@d%Dhqt{`^zB4-Yu(Z{QTY|+&62z<
zEk;cIhw|MK*9=;g<ls>T491xSUd%^ogU04U2M->O*Qnnh(M-;4VX(G#Av#@dbRp?A
ztjn+Vs`*?=8)O|+|JBwupHA05zeME}vfCqXIp6-EcU_!}-Vy)Q!R9XRzYscgu<%IX
z;lgP(m}Eqhj2=A|fyLsZC7pjQiWVgefE@LsK5h%z;hF(omYy1xN~4CJn_7q`spgi=
zi8={UT$fsY;MIx4;-c$<`_!VsoI<#5)o!kpB$A&b3Lq@!%yBe&)UKT<QGX$MkR;>+
zo_=oHG4l|8u+GGz3(3#*ONlA)jI#pM7fI_+5dS8Mh_95M+k3I-qOCXRciY^yr0sX|
z)zU7M=PY-oQ_&|KzbevjTAUp{MA)lwT;i-JS=wo*&U7+-%1>BV0-2KWF!oP2Bcmp2
z3w?-2rer!14~jIV5vc6H6uNZqqX%;wpU;La?f>^6LfCDWiY_IhrM%esrQJGgQd%ZS
zmgI=D_gcVF%zQQ-Qguh?jwZDqNBxdbWRFPjNj?6L&Y-^OxEh_c*9NhEJUWv^eD+Ot
zZ7jjj*`u>@IVolY)>Vw;^XZ9OvSWVx_g~z9(W$@G?RR3zbfVr%-hZ^Wp=~VLHP!xy
zxP;IAB{S#92xxj5vDIv{_6M^eb!{*hoZUYYiX0S2;LH!>`=AI#_Rj=oQ9{;~os>RC
z_p$w^p;Tj;*6GtL4{tq!<KR1?Z->6U|D%qzyIR=?lLkOW|I?Urj}#s(44vA4A$SpU
zah)kku75}WJFV7{PTl`;<B#Zgyy>><Ycl;URDbByp^H5Kk)um+B&$EZ2NRHTmZN^|
zNq8&7?^izmQ{N`DTW5Z<`kj50IGtMmhNjj&bm#G=Dq>>#B(2}JFk^N5KT39Z@Ex+r
z!&|3UQb|&JR7Mq3om8K!jiPUgzbOK*WpA8{em)1M=43h<J&!FJVNvul>zZ*!e$x6<
zjXzrJ#;;o9VZupPKRFtzo@f@~L(<Z!Ma<$u)oX+#44AhbBuTEx>PJh}Dku5)PEzev
zWXEbz?T<vRf>BbEZHOi1$ScB5guQ+!#I8ir6_HbA348&WV9NR5DU9t4xZ@P(1?GQ8
zoLcCV+KfxY?<wR@SU;pDZA%H5KI!^jTykB)K23x^h?D1JYW}ygo@D8i@{^G#h{Ufz
z|I`NweK*OWC8hK-tOPaeyr!K0_1sSk=J3&Q+?D)L-8AiQ6QVg{(8!YoYiBpj+#cD6
zt>@b#n`Uq$fP7d|`IE_f#qt|b1le!s720*5|8bxtO(wDbl=^ovv+bWLv<0W0f1LD5
z>W_;DoD{L?1?^Az`B)+c6uNluBE@gc^{}>22U^mkYClH5GYUxEe)9SHgZ4*t+HsAz
zFS?TYFQ+hC<FVH(_aBmuEM3mW&KcQRPqMVGH`e;(4rVeMXTGkV>ZLe9?!d`hCtk9&
zu0Qsw+MmS#IC=35l^_a;ow=M8xSx%?EQ=zJ-omZGt#*%ce#9MhC7u7B<2+S5wpIge
zg`ZgzaZ<Wan-g9X*#?Oc9V-jB6&^ely0CvPm{h(*bcyoZh$WWIu8h`APuJ=D@d)Ag
za~b!<6VXtf!)G{WO`V1(k>f|UX#OsWb~VH+w%;|hfc>kOFHV*#o<FJGWT!;G?JD9A
zr~Fq%&vB#NU%4838EMmayE^%)-TJWmQuiKR7wA8y{%m?E<x1lhr~a;r-l4y%qE}`{
zI!HE`r;0zE?Y|m&XB>Uy^hRsWHtP|?y4NV`s{2@8jn!NoJq%;kHA7Zw`dQDk^Y@=p
z`Cq48NAN6<`?Kzkx<9(@qen0O{L&92^!fR>k6zlEbpG6i`<6eO`A5fxyp<D1`qDop
z^<U2ZOQZ{J|4uobX{~0;yDIu5_N)7Hq57Tj8yaUSwf}V5pOea@7ob;&)P6(bOey-9
z`P*nMvCd-XvFeDw|6%Bzn2FP=<2RC9uR=%`b0=5c&t2#au6c$-2B(FB>|dvG#gWSX
zNoF0d>HQO$EA?7VU!|P$zwwYv<cALH{~<ktgKf(nUi#ssSzP&ZWHxaC0d}rA_eH)}
ztzr<m*XE2qbegnr6p_fh!}bJMIsZ85uiXDRq;SwDwxtF7-$9?mep0UAi)!ia{U(mT
zlB|E&SRE20X{bz{zuNw75uVVZweB)JuR-=xTZ`v5c$a`ICbqUj5Ix}~>GZc=8($f{
z=Be$2V41e>yQ)FDfc`ZjT;t~wN6{4}d7L?h-jxvX?b{6d#_jA)nAOmNi#J``glD??
z<Q*~Wu_WV_!w(N)_b($SC#Mj<?@9LbZTDvvZOu7S_|B$phb|3Xf{RbOe>z`(Iw$s;
ztbS)c>OHjx5){sEnxXN!X{Hd*Q!c7?pK^WMiI*%LOA*V>N%P!uctPaXI``~(?zyW!
zFJFG{xnJju)$Dr<U5?P_>UbJjlS;%X!Ir(fd4DNLpHvvI&=dz|4}7kU-bigyzl!`)
zc3N~?Nh?yzn5r~c^JUwwzJ4RW;*_MKRI;U}lnfK;l=6($WhzPaDYFfaWVI}ikLPyr
z<Zj9%6^F}wq$E#SDCtwxus|8hpBg8X09PV^D(=6Q{6=<@O5lr9*<a34{w&}@l5mi;
znCzC}`R!#3SiyWPN-Dnlc2!bfDToj8y^>y-U%#P}oRYLMxo+(Cr`F>9nq`iu-=q@Z
zCX>swKehbH?9bF{Qol<5Bv)$6MBGfCp_Qp*avF;HDM^`3Lks6$iRC7fi##7bG~zfl
zU%J#e@$|{+NySBtq2xBYK*^QpFPU7X*GwuIZnFHVpSh$~96C(k`DCq9x~u<ArGF}0
zkTbn2^J{#54N|UlA^8?+U$XL|ceY)^yHNUJbalo0?^4k_$!NCWeJg%`PGSe?9vVM!
zrMGzIJx>(NDSpkUcOm1iu~dm6=4I-QNhPMxYu31@FlSveyo$svbGjYH>(XQl9Qj3f
zmuQ-_QD=4q7TW)FkSE%I<7cvuUO0Nn=)*}7n<hCEjLR7p-B(X?5Wc-{Ht)W*dHW#=
zd4fIDgBK27xbpTV-t&?4`YtHF(U!b^yz49$33_Tr?uj_-ru3EWUu?zh^}l?Z-a)0$
zzkK`X#jQ!T=8%T=D`+1FPo33Xti8DD!lnzipSu0jw!&?N^d^ofb`T^=F2WNgb!=<W
z+Nif{@hDBv#YIVX&SUv3nkmA)Kl!Xyyaou4#g2!Z7M?5}PZJ9pyNk6#!g6wqr4mq{
z?md3qiXP`sJ9=^-itxALX`*;K8;--D$8Q1p`2ADUj`t>&bl=r(03~!?l-AmE=28Aa
z$JWB5uRL@y$TsUtC#R$F0@L7+-aSl?eI+IG9X9#y*_GhW$N3dnO*i^u9{-V$oWG`P
z>*QOQFT3ZG`9Io&EyVko&d+>_lI3Xa$vbIet*PE~NybIz7hxYb(MOm6KYYEMmG3I2
zI$ZeORe3avA<-k+L^D44oy3U#EYb+-S|{qE_;h@x72Yz18;cl^=Mnf<c)|nDv3Q5U
zFNm`qPXbZBSBAGNLfqsh6~9H`NS~|-+7VN8Ew}Wr-QLCcZ|FG!|Jo!5OJKjaIPTMr
zJ1f~D{t@$OaH5mG1)dSXFSk*R_$80ngD_;Rr03-AAdScm=><B5bea5_&yRSPi`<m_
zjGpUL@}qu}pE|;WJ~GjU`Fe@%U#;g%(sRrakEeSr9#85{>rb|nzL!N}*eM-ryfl#^
zB*)bJNIn-Z(-+`}-jbFwuis+jbKzZM545rjTw{;;ag@xiV)a5=mX@Cu%MP~U93Rvl
zx=&&eiJ$4W%t$tDelfVl+jA_x%*axFvJ&9M%9p&}Sbk*p^0O!sE1$uuM?v!OLG{+N
zA0qopj;F_33v53X9$AX~)TYEk;T0V;@?$?lMme#^3L!tSxrv~BL7$`iZLaZ{q}8pW
zRkYx<6_&Rc>8-5mSWK^IiL|1u2DYA!@;p%*DUCEo;(wL_KhDped{KZ;Ids$*X%bDM
zE)vTxgGX38IHf*lY>|w)_J^Q!jJen&mvbp!e`qctx<p5&Sc{mEK5K!K{E#+x6gc#`
z(t~;}dEJQ3u9T6DH{TA(>yU1Yb}V7BjoU<=pn0Y~(hzAxU$sO!BYly2e2Ovu#2%U4
zTe6+b@AyXsBtM#gT<Agk8zQc=UL1|Y^4lsJ1Yr?gj&P}T18ghy&>n9&<ZJ*J7jWe#
z7Vl`JUe$EO<|ZQZ`QaZoDZKZIezaVE=C>E}a$V9|NRLvJ@x3hDt93@Y)ah9H3?8%_
zSqbj4?eXn3RL(wirVWNnnKDWxv@49lY~e+_y|`jTM&fxGSof2@ni7*Td^BHErc*g$
zxv*3lr5#cUX)*qgn%N!cQlED8<oy1iwvlI}sE9O-{#Kt@en(JlMd(uuTQcb*v1lA$
zfrGLIbp_4JI6T8oa@D-!`5kp|qB%!@;>U6_y!yC$rB*bTCe_yY@@Xx^>V%+JhK0&U
zIma%g3uDm)b~x1dGS>!CG7k?vLo|wJ%nCu#CpvM3C&vhluUEGq{u*yz^^t!5kQKDy
zW7-ufU_CrkAN*7&yj5GI8Gfq~-m3xlv3N><4nD+jzW$D|%(AuyywPdoT$o>5WR)n2
z8%^>2lIexz<Jv6Pob-Q*<kNFYe80u}A)_y6YJPilsa##iSb@e?d>o<i=t})T{X#ty
zuV<3Em`7&(2#wWW1N2OGRDyUTwk@%E<af#Q%6UE4_SW(CveKUK&CE}!`0-m04i@85
zt>k~mqmymN9xFLt#|Gva&o2}5m!KuT-f?+G)5~_w*S{{Gf2EnzCAP6Xo5YqlgnHY8
z?-v7%LwX+&_s8KG&{$$3ziYg}O=6YJZ<UDoseHo>DdLBjn&p^pe;J4%W&g`4(%3=_
z69*7qFG*zlk!df@=hRmHaj;Oy0=^c+*k8Q=Ba{rr(<?k<3uy`Km0qw@j0))inb`h{
zt<AJHQN5Dr-w{PSKQ^nl#KP^B1Qc;=ofy5vF#{NHFIuzln2Gvh{&*6C<15=5>*Lwj
zFxhiG=4#?GKVBoW`S!9{^^wiQk<e|W>-_Oy5UE`HBk7S$No>m57p>yf*g^+H+$5*e
z9>T~+qezagME!CU9!W-X7W5MzM`YMmT!5&~Sedk463f_H>Xd9k0sBVsj~R#%Gi*qX
z8Amf!vKDi~&iUDP&f3Txk@&PxI3d^FWNUSRjimAWFSWM6r}Z@LFVKhfE@BT_pX)Wc
zT%TtO+K-6er-|j4e4jzD;+wJiIA1>F$7A`$%V+RX%V+Rn`Nhj;@JPNMF@{gG(o5of
zQLPh-NA&(0NttM$koNTGLx<S`jO`J#UqX7(dqau$>!^2C|EfMCpBj6_>P72se!n$y
zTYJz7YKPqSNz}jb{F3d5)iS3#*!wi`6Pb}Ek(`9~LN#i<<okc~`6b@}i^WU6{}+p=
zSHtu7|6=i|*NEr*ec)I;(x2?R`QyXcUi5v0Jk|Vui{l6V$vxTB{1nc@@_`?06leT?
zVr;$f<HOiiES|I%@-p%Jgt2&1CyB;K93J#1_s3&L2mRtwto}4!eB5p%O`f7vyWHzD
zE8s-yUs@qw37+I<R-(z)tCHWscqA3=TFG%vcT{vLcHSxwrNB$P5_QHyrB{fj=SMJ2
zc~JaMvHwDcjO`b~!cWn2dl$yLUfjGe5-?<cI9FUIz0JWv_J=8Rrkr?0V&d$TXgi&m
z*lpb#S?%CwW)Vkbf+X$o2t-RdUNW;ZrQ#3SD-vJEI`MIi+{uYwamMqJuv*7EEDE;8
z>v&qpCE^jko-uf`KWGPG;r#=uUE(hsXY17)U1LA7ZPL46bfukVi$0O%@F4L=eWU_5
zCRf%KXm5PEvwwB(&hKBMl06{fT-r-8`w8(psaI&6Q1n`i-McJEnTEVr9cLh)dOdda
zRt_6Ze7InHtJoLP9fUhUXT>4vQI~pMOR*P8w$S?0;4QT7(fmw2-T+DGuWPkEFC?GF
zOTGT3Jy3bqVIlc6-a_(e-?or^(hDxEKh19;`83`_@@ap$kbD|%A^iznwb4AX(7Kka
zUXK7NWo-XyKP0bmXF1WlDh*cKi&<;YIB?EegtrpDiS{I$@oC{ZDs)e!7^~sfV_Aee
zg}ev18gXSEkCA4vGZKqOb7m>lcLF1)0TEcTeIH|2spT`=Z-REVW2fOx?3<LN*dMxG
zG)K}S+oXJv+YJNwhAWFVB#E}a<nl>=%Hv0zvs>&wlEibCo3wKtr29`-5|`*E$+#oA
ze1u0<CqL_OmAk0LVv`uwOmHQ8lH<(CB~!8Okn<|p0ktT({mFS~VLUB@#7}K6M^K->
zqcI)PsbcJa#~-w(fqx$b&uI4lV)rYgo)?bmWSp}wp47|2b|CpJj3@Q6FrIF?#82*D
z<ebR-L_OxSRQvbjtw@7%{3hP#lI@MR6$v}-+4wU`$-O2Mi)rc2)!-ZN7r);$jhWjE
z-%>5S5pSd#DT2Mc8aAASj8*u1t8F=ShYb;o`Z%#EP4;>vM<ebV>CZsq0cbS#*d`uS
zpUsFp%qJErcE3cfFKPW2zYcX$rHX$o+|@$MBE@17GxlOU&rk+mIe*<9i?<|_UvEVk
zFj^MPIBTLb*^3MJi;7j0Q527$V8YWMy8Dm}kI*v_J(j>Tmmprtj8IPeaTIq-6My0{
z3D%q>*F+dHPg<RhCCSIeAKP%R{%GWVkq098MedI5#P<V$68<)TA-$Zj)U}jA7n#n9
z^Jh<-5G^<mTm~b3xw8PKTUs=5ywmDDE<`jk8;Ob-V9vs!OM=Ibp9$uQz7tX6*4fty
z2Et2hFUl>h*%SDP+Inwh5S22TVi3WB|E^IVtgjd1`0>t8OE_3ZbV(3sc$T^)NUWux
z#NU>>Ha9mHyJWK`PN1H89N21-K(5!6cv~{PrX-lj;z+d4&goppsuUudAnQT19W8Bv
z3?u=G(=xi3RHd}@#+l<#m81a@p~;e3hy&ROh-e)IBWF+G$#AI<=;rhZ4WI=l`ZNGF
znu9csLs*p6+1Y7I0M2#dk8`ZxnbX1YGeL`>mNT8Xmeo0WLKZ>2eJ4+hGLUQ4^Oe4U
z10gLdmjJrvQVsz^SZ9L4GZrTSSm+&<mKU!#S7pq_<ih~Gl#!Xqywn0Kc>Ls<bH|Sd
z8I`&Q2XuoYcsvslMI;jd-pouJt@Fej@E9!$bFoW;$0O0>n%f*|qGm7}EE0c;0aDX8
zjg(8u22@JuCXfIY{#h{}5X3ZyvbDZgP}K_}c=F7d6A@F-94dts;DBKGJ04{Kg5pqU
zlx>&Hp*<*)RRMsFz)I*Knv=Q-z&EUeumT)XiDdb?xiA?zj-NO^bNYmmfH>ihtdAcj
z`&1>z&z(DS=Hy8!XppjXJ&=vWi7+==LL(aomDsEyThP`hk5|aUG?JNWDFE21qvXsR
zS^^qXoi@N@Ofr~bGthPbtHdE&8i}xp=4Mh4)n@Wh%ayW|;*{g-BGTwLM-Gy~%<1FP
zoP|cBNveO4GmQ*c0>;xQkpR&gpHn5?miqcx$iobXhR`ypv5K42C`M?X-9HFS3RqNy
z#y!Lgyxeoj{E>713=Ylu>^bn(0Cj}a0dlB>`iKihPqJ;3MyNz*0|=srqMVIUOI>7Z
z0?bU~q{~%!BtWN1Yj-a}rs0+Xp2JX>qx=G}t;bn_G%n^4eP|m7er_M3;pjZg7fzn(
z<PW8T5Eze$<AAnjahX+f0Gv0?7~cB5<l|+Da;nln0{2&-kquK*n{>B9Odmmv7}%l3
zvkWBYgIDe|vkb|myuTEOW;BSyLkTWg%Ep4KNw4oj5KUGylLXZKqf2rvs)y0k{38K1
z|40DSVJ2i#H1$EMRFz0Kq)J-6ro`J)n`xtY<s>fAl-?wuCL>+KI*<U(!M1E_Lbf{%
z2^bLUa1NG0K&KPP71E8($r1zLsU-st4Vws>28(b2nYoms4()&$%q`d1baes_i(=Wx
zLggqyJq8fD<<@l6>U$bZ6Zki1=D@TC<0pZT$8#7_$>EG1gf7*gowubXI`$L*Ai?o-
z+K;k0X3$!KTWT>20_wH*hxr6MJA<S|lFgddvXQ9;@nIToi$yZzN-;3bNOLnpgM(&x
zp5M_@j3MRHB%pki0&In#R%#s3xe=O10*Xcfl7<0IQ*Sxyk$|F6fTUqS(+H<*63~8>
zOMn*yS1!>sn2cjg3|5b3Gn&}YvUl(3Xv<lm5!0tnM^B$V{W6j~lgG5j(4ZJ(qml+d
zXZiGLwgad2veY&Wml)YdGFu_Wlv!$<1_K^W;P^SJ$8s`&5>0zxAu)jIIiy-z-je<Z
zEOKjegB%2MXPF@bK@A{TE7_>T7F5=wO4ux^jFi|(XqXl>fO^#=0d7pMyTl9((?3K8
zM+|R5Zki7TNdTSIr&GA11?SI1DC6*3Z;h7YptSVl%0W_wJtFI8>RUR!!SmBJAu?bE
z&#|{u7CuMl5SbgJ08!p~x>I|FIdw=p<a6T*l2NN>Fo)3R%p55s!Ere=*Og28F1l?m
zBm5OO<U(5p>?>BRTtVAUh(b7kFUil$FJV~v(0F(&&Hs+_!jh7;h3S+b{q&0bf_%wO
z)Hd^RiA6wTc^#3Ho1G>%LFEi)_{mvtWp1I0j+>jy%X5V(mu}pYOYdK}qEOW*<<E|5
zUru^@y1ZcmyKD&xic{o6dJ<00jw#Pxk(M1(UYMrkmH7GjE7Q5TTAy}%ZT-dSX9Xl=
zeMxy&+;FYZx6<#eYfGfQB|WeezAVpyyi$JIexpB{vulNcp<gU04UFc(;~JQSaDI_0
z=n%4llvNvy($XWPIPfL&a%+1!y#@I_wW>o)d&+4&RJyBKmSZGfiT#mdk55V%)4NQl
zZ(ckhcGHt0rEq|SM$7G~JzU$P*dr6dT_OQv552_U-1!@jYORJH?sR5u=@_@5w6XEt
zdm9@|3nafx8Ovs7%Ki~`jcMq7pevpP7pA49HP%V|^z8i7<|D8t+5h4QEcURYysi;&
zBMyasBrQ!zAnEI<$Wcz|tqyu%@dUtMl0?BuLliIBYdS`Nqz(n3EZw1Wk{__{U$k#`
zS65eCy_S#aKSJ^}%E}HSMQ7t-2~&IM(D+>DDChPQf)at?X@ct=^t(Zz^g*P~`s8X4
z{aY*XX{gfCtl-AR=9bQ2OS6`bS<{!Qud#8Mx)X<_ZyXrt>e{ty#|{Nk`}k1xbNiTy
zmQNP%s!LC!94_xP^-=rl9Q{M4C=XSiVz1lx$RqG~7)9D(x%l3Dhi&=D^vDR6e1s2V
z9FiZ~i_`urbtw7uic#BNom5wPx+>2tR|8S9bADe2K-S^R<8)oxRv4koCe?5sA2w*D
z1KmETg0O9T!m4XLVwV?|K;CJR+0aY;v@{aY=C9h1@h!a6eAVA{PYT|c4;rF`4+|J=
zR``0j>hVgoy!{C}C?${k2S`8-k$gCmekhmA4}-Bn*}MbgGzb+uRue0K9W1~>pL>|(
zmGdW45PPjZ%AhXAtYsf8-k1#vX#ozH>7{-s_XrJOW#8FU7o4Q5j{xTZXF=W*9)^}m
zeAKk5guJZn(I1B8{6j2OGEDJTP?VJ9=gawA&d=Go*Q)qHmYexV&L@(d?8!;bFwgIj
z-jqvvg5|o^#2a$HD#UzU$nww<0T1M3{+IH}^)l$~`JJ{N5xpq;m-3o;My_NX^0Pc9
z9+8Vm6CX&}%>TMPJyX@si_k($0a89$ZsvapL*AJ3?A$`7KUGdHnA@lMXUCLhugFQ0
z_JsHeJ(+FCV-nx!U&1CH(B)cx5_ag1<u~yX3t_%63)A2@LIy~=nfb9W%C$phd&KHB
z1{S5UFv_(<m*Y7OHnY5x!R+BkICgc6yi%F3zvGM>&LfjVr-R2g=p|&2_v{t9b@&87
zKEXW52AzkrAJhLrq-Er++`2L+Lpg(jQXMHsvbsj{NaPQ(9i%u>o`fM^qnv+i|AQ>2
zKf*+;4oS?gMf4JGC_yk?g0BV%Lw<;F%MagD4*x?Qh|3ERN|A_}>B@gD!dy?K_S#xx
zZf$L8Y27Nvx9nnvzn5K(PTeZ&!xBE$Uuu==a<tkxUkHNX|807Pb$@^-VD1u4PbTNA
zpH0v~uS`<w13er|50i+YC)im(!;bm^oB9h%3u5dY@}YfR?h)DkEsZ;N?ds|pXnaM&
z+u-qzkk{*wa3&mOZ2f2;Gmz!<y`^#Yq^2i$>*8(b07{XpvGEUPWJmq7oaJSEP_k17
z!>B8E{DEhz`~{`WaCyzmLwl({-I)%0TL3DDamK(Z%h5lFLHbWPbv!827RmWv%M&x+
z(U=%m8w2PWUw5vCRsF=D%Z*49E+^yS0L(6@ta?2kXXX+qA532=8XB-z<%8Nshwgtl
zzd}6BHieFOLuW9uTwi6c(EeY-X8k8&<NxLSXXX=T;&3unUrA|s{<@x(YW>M@?z%3`
zpKWQYIKp)FMcq;7fETYQZ7E%`*e<94oX3aCp+cb8<>+%7|Ce;ek$dmGYxiCE-fL>m
zC3JnOentm9&g|c$f1`olXy6JoP`-OlTiY7kOB74j;Vi_N9TnLZalVc70?yZ?u0K4E
zGlFv(XAtKo&ikXTFE_t`@js9J*~ph3t(*Ay<TD-rw)xJSuA@(SMjtQw;NL!f-|V-p
z|K^{4=7FzxKKsVNr`N2#_3&RW`uIJrd%eN`dEGTtZ@=*$IrnTDS%34$<(VbbAANDx
zher2av;6O~E0+%a=f|Gi`BKYGul};*Y2yF$SH7M}bsYP~v$s<{m!6orh3e|P>DHf7
zeZT+Dk9>#fyk+ZH5!L%{!=LY?y7P_{9~#-d%JtK4J^$g8e|q6kZcWdD|Nh;7j7CNC
zhMnC_y)B&$qPcZ<2PdE_PVZ^x>QNaCcQ<vnD!8e052xC<w|8{iBQsl@w{xPc!A29D
zcJN`?!H40tPF;<}Z)@1Csu6p3-?6*n?%fv2hqJe{skOJMp{uzU6;qco{mvbGH9o_g
z-8JnUQa%me(b#NcYuMA=s#+Qq50pm5{cZAm-&vKOGkFhuuiMw(*xJ!;l^^g;hWvpE
zt1Pr83w1Z{X>DuX3<$v84PCoicki?+tUVL`17kib9I*Pw#sY)=VV^bW3k9Yo2Ypt5
z7}%T3hKh+CcD84)q3iLF`vOzpIGlt1{%~)2$ns521}4Eh6z&gCg|?JUgM+4kM2r^n
z2Ypsq&?*lPZNY`GtU-KLl})p}Q=^0a-jHu%sCQs$Y@apU?;rCGflmPWmT$=NPYef$
zCX52Qj{5q+Xxf_aO^0!1nSF#y6;Ta?Q<GcDytSYR?2`$FYAFefute9JqA}aMx;nZx
zTRSHE$9<M}x@>x#YAOR+0(~$!IO?Amu?GBsP{oKZFzyRa9<qi5ld1=V6DJS|2PTJn
zlQHEj4Q)N7cVBq0Y5*7&q=5<&N(h3P0_Bvp7}(5hE*p|n#MDf9ki^oG02N(pm6uJe
zv*gJ--+?NoCVdwAXmo183I--&0z=l2KQtJanD7mTQ9}|OSdEX0la&wo!jk^Z?d^Mj
zCUyj-CWb76tl@*bgTo^rc0g8hdjtA7?3*0!hb@Mtf+P)F0B*iCwQiQAwy{$ElLvg(
zV1LjW@*VJlrnT7`3iv`3tHaiKe|T`TlJ&Cy?U-+3Bs@y3_J>gaf&MW+S>1wlPM{Zt
zsY!$!B-0%WgnLJP;a>mben>S2>$ML0!Fp<9-$dZxgf)rr0}~<q&eqnw>#OK{1M2jL
zH(S23L9*7GX^XxHXHR#d6`t&$2#xzgJg)5#A;eTaI&&~EKHd+N(wO1lqVnY+#IPPb
zpt2?YP=B}|x%~>6d2rsrNk7~Gu}8HMSIew;P5CYV5Oidh`=*2FU9=FDhkVwQf1Nb~
zg*fr<T))w3>ugHeKJJ&YkTozBvbLykWakYH$L0km{p0-@PzKqm*s6#i6w}SeG5r|w
z!Jvnda$x#IIN-o#sJB;J986>io-E#A7>^A)k&SLvqL)o^|C2q3rUo#K{S)MaQ?)BG
z9=DBrg@0tIA`qPPjllO)|LDAdq0o3m$p4@Zd94^%V^-_VW~(FE#h8|lCxRhz=x}Io
zlW=K3M!W84Tt_1czPghtF|>`%-R;Tg#>R%)os?Zu6BFbS;agZM<^ByDyu;Gh&nw%%
zej_=gdHLE7CkEi;yc?!B(-*rgamQWl@((gP=#ZDiI`Gxj@IK{!-}qD&0h^WIv4%0T
zSfnP(wVVUlf!jYcG|8H&zIAMD+Jc*KK=*Yuv_C-d_K$Z%X`7V_t@{SY10fVN!belv
zhbIH$M(!-fJj|YMqJK<Hozh!kJjKqzP5l!r-#DDyxNkf_BcOlM3Sg?YHth7XFVS^1
zHr=Hiq1QWVmG2k}1cQgx*s?gvYpEOycSMtj6KFn>zGX94)^E`i!(jp@f_+(NXM1B8
zomMA8uGXH`j@`XI4UKKvMOJG^a}$CF<?l9Iqrv_yG(Tdd39YdL!7z+cX7o>v)Yz%@
zG9@rlrpCf{_C`B(tDU-yQ;<O!tbj};8~dX6sSsrMPvUzL>cMmXN9&tp$KTW0ZjGDr
z0dzT=JQcQ1z_E>u^^<s3U|3S29aM(6y0Z$iVSU*&+fisVaIk_jQZeqEm{Q|wa=*8N
zdSTK(MB|H$X}o{BVlX%rBApUEF*V*#85QWfn7oO=kPle>gF$~4t6BQh&^XQ5Rl|PY
z*ihL}1!t-}g(dTp&2^ym4LhafNfXt^r|M^-im^aH0u8FaJ#c_bGC1Bl&<~f52#`8T
zjulL$@Q)QV<B9_#{jl0lRcNH21gaeHPe3QtzrY+<F@W}tputseTY(DXj0QpoPY_!O
zu@3sGDhrX?@T+vdSzGVLiCBqph-T7%zy}z_p?!=dra}XRSLUt0RpNvY=Jby+mL&Hw
zbrldRh!^-!|1`8TwN_E!qh@{@!!)xULUcLI?UFvi#)FAA^fW+$R!jfnP(?E&pkT-<
z8}v6G3j0DB2{vxGFDxhCVT`hIbQbAq55g0W!#+~uwZoqM6&g7;Zf9$UWVry(A>VL6
z*jm6(UB4W~Fk-u2O!+~-){>t?qkd~}2ptSgSV!H}T-DTU-Dl%VJ$1H{rexs*gvZWu
z8qo;#dBr3KLrBGQz{A9+Y6w~77zLa`6B+_43TDt1+rWe8M{;DU&jyBu5v6xp?9s}G
z)>-UvF%!Yv>bZyk>pmIc!mr905_Q60{cxUSu(ZC3x;{}Cbv=<8b^XqXsOz`SMqR&q
z4!w$i1i_WE3>`i-%V2TR$HZ3q_U6_-?bhy&9;>r!d-wL;Jy^$}hVZ1{7lPX#rXcrj
zMB(huvC2{6!zQHDhGQP`PmNbrR$8K|+3T%#ZEv%BI=WhWI;`%_?M<yaTALckXF><s
zSM$68#dmeID}E^JfC#{BXzSQXI!9NK6IFeLzGAsepP_5nO;Get#w|OoEJPhfi@x~s
zRK^c-gp(8D$pFIENoqdLL$ox*pH)r~E%MNEWYWk&jRz2Q1}3+xE(@);f|G%vsll)W
zCVdBFB7BI}VMEe0lgkDGT5WUS(M%qJ#OFT<9vTe!d;Jt@(O91Jg{LMbd=z+cotx2}
z_PW3t!18sUj#;Ek17rIrhM`Ik%)ymlk;(Dayt<&HR+t8C2>3%5#o8@<8lmtZX;aoz
z$TuW?$R0<X?cGg`t8Abgq1oVAe+bo3<PJ52u|$<7&+$9(g;{Y!!^8d&T5nN(#EV@b
z#%oF(?suA*uqJTGvZtY2+hR&Nx~GEJ7zp~PnKns?XL_T|@<YBbnwwA#BBG9h2)hGN
z%z!nD#Q*}BNqlXSa-lu@e1`@CaIV(D_WtS4{=t2|FvUjbjN7e#M7nzILv6QlY4M{v
zff!NQcmTRzfQiA2We^oV!oJwQY{1!!vj^u8;iBPUEaaz<7RH1v4%Fof1!LfL-$;K~
z1J;n;XmB|>HDIr?+L{%`co0LvK~Fx@IcR9a1UuJR{s7OfWkDo)+@N8$f2A-zgf%Xl
z$!4phEL4&ucoAtqx{d1xYd3A|=b<MV;prQ>u_19tVpEGQlw~o}&qGrC7sCh9V~F)=
zGNF~7k8~bW#<{Xm$rr7zpix=Qfp>!HF(3apKPrA=Ma56o8T*-<;Ngi46Sz|uHpE2Q
z<g-7G{mCccw?A_->iX<CoNq>5pDT^JKA(-Vh4w>u@#u%3ShI2{9pt?h-tRitKM@XD
z-S>3&Y;U&+xsoDP+!5Qd#h!s~)>V_4SSSI8E>~k1gBB6ah%D4)6TX9be~co;AcnZL
zs#@YS^Tq}wnn1)vW)eDt9+mSc&vApJJ~$@H9)6hf5-ka7zS*(9#-cSmC2!q8<}^I!
z=WQ2Bmts9BS8H;HBv%V3%I)X+V%M_pYKgHKx3vRP$Oc;EVB({zl6id%t+`}ui4vJo
zWwaHDxVg2egQsmD^b-nV-v$yePgTCYW&nDpiQS1I;|XNZ2F?(|YFgM-U_Ozww{A#Y
zFZZL49Q@e-(`!N{7aY`GI~%m}?F|~1X*-4g)YUYKu`>sMEjJZFq2)&G#}hp<k;?^I
zzbg9$oyOY}O3Emu4b?~+ii+YF^uFzQKkXd)Fz+11)EmN<(cma{OhXV4O7jho=jSm>
zW0&`5vjhby2?q!4U-OL_A52i0wKn;IT7>BB#c&uzwY~7r_Qq9cuzzBZ;$WJV<%*+b
znm37~uAJARu4OOaOh?*_6JwdLlwROmRt~y4beUOiI_A0r`Igm1+p45`(gn}**w*1?
zo9OZp8<as|F>#h$2qQj=%cqrqMJoX=ilvc3icMp$kDCHhV>I$*P$`{KQ!|Be<*XsK
zJz*li)=qjxDn=e)%Nj8bXup#>723Bfy1|3K_3fYX(*$ABZXLCWck2hk{sXjA6dv_i
zSlW_R2$9vv18X}%2{LF#rd2Ar!ZED4X>B@L;Ts&q`V8)hKruGZFKF3TL9_pupBK{V
zK89S!h9-N5eb~pJ^!2Kkkn`1AR9-4z5u*WKSJK)bwv1Pvybe~jD7+i0ywz1T-s%m!
z_MRFZ_7D2ec<D4a>>A)@7&moj$m(uqAJ2Zxi!Fb;XF<W4#FDn!+}_%Y_A~{8Y!QkC
zT(L_M3?M$ya|SUj!_0xP5$?whcL#4vaPZv?(?-5r|02HRVeJpl(m*Yb?J^y`DeDVC
zcVS)~mSe@to9(tU;=ob!bkgytmQDwY;NV#Qp)ApZ7?*g$&w!B?#_bHb>I7jZFvfFq
zyYdlx_8>l?0MmI^Xdp#Gqv$J?LH$O{>M-u?V4rZLKO7d@DW+i`3u!pqhpQ|{(1Hjz
zNbzC9azEXd;Wl?9EI0H!_#OcIaSXwfTD)~kj2%Mbu?I?~ZqLe?9VWt+R`))C&}rAO
zdGr^_PUyTPgRf8sn+Egp;lt+T$@?kYzA>y1oHcId;Gt}=lBdxu)J*$FO)6sNjUrUs
zLxXP;n@T=R+1Nh9pc}W^vG^hL1i>0w^Y&Y#@GJ-W5$CJcz*&VyQKDRZWman#J7&1&
z;u{NMAvNhAKE&giHU&qp84rC7tFXcf1%|`Ki=!Ko34F-^91KkEBeNL{Ok$@$Ja&jY
zf@Sd}$79+e-==oTIL7EF7sBQw7#{L*4n<d_Q_Je0y|d6L6g~<=9q?hxGu*$=H<2~3
zj0A`+Z=e;^B)5ua<c=EiLQ|B_B0fF+EH!V_#TA=cF*aCm5=&(54urA8+7p2Kz?c>X
zV2Ry3Yj|n`O$uNa6)Ib&Ymu9N_I4kZvWTx_+u3PmH5|a54)!Q=e-b@5>67=;tf5M+
zHO-D?unr*KR+lNVf|V9rFxJ9h44c?JMf5MXl64!Zv>I@~mE!MYSXMu9;=DZ5`YKhg
zI_ia;HjUw?qtUa}RqHT}!x|jzpBV9xrC_JgKM7;S<lv`%AS1;5!0`oUbA+noQ!U(L
zqF!4kV-AbvlW-tQ&pt4E+j21mV1sm<jjeFB9}9ThQaLBF72)Q_AjJ{GQ)3w2gCr_N
zGWP^fwI5wKLCaCNWf@~tX+zX4wE9Q-{Sz{v@4zw*QAfjs+A0qW;>IqxaUg<NwOXqJ
zf{_wx0L%8vPPaOny0FJIu@B*KR!?AL1Y<S6gml2+7L6!u+z!B$;XT+VSX_M>&u3P-
z6nF^xg;~(jX0=B_O#%G@Kh~~F!MO8BL9px!ip(1<YqPAFrtuJaAQ&(ATNBKgs&3ug
zV^wb&HTN^*Ohro{TGN?e7wzOVIp(#^Xkbe>Haf5&ia)t^DYv$_VBe#S3#o(()rJV)
z_@KQw8tBNz{l9SvLR4GhgMQqL1Cm-_m^EJN6<!uOOYSO~=V`S`<5sx>*d~p*sB3dy
z)U^q*W1@3&=luN5Euh(p{5f1laUR3@1kN)!Ux~VI!+Mg=+d+FfcsJv^BQxsS{ykh<
z$_A;<T^n!$cNbz+K3`PvPciu$I<i#Yvf1k9h@lgAIVduQ4iJUt8yln)#9#IujE1hA
z>c$52E&y$icQKGscQ%kxcQ%l+?`)txQ{kQo|9EiBhf&ItOmkCjdq*?kN$u>oCvcZo
z{lTVF(Ka<SwQRpiTE4eqszuxjq1V$R7r1Uv*Ny6WE3OO3($mrGkVLYpQRGgxunW^9
zJjnwslQJelQ1---Z{ooNG|I*TfuQu~23bFjW4dc>%IfH(%~z`oN#|X8v>|cj4t;`8
zZ)?YukvaQ;=o-5^8k(CLx_f%%VhvGL2sakwX2F_xp7HH>v}v9_wC=_&xCZPn&(o%p
z3w0884*M4@{H(jxGeB$C!m8ynLz}I6)|T4LW#4l((!OPe(+H^E+jPA&mg)obbiuf5
ze>v*fy$<KwQCA1XW9JY~tQ|X}QCC+R&e^D|8)LQ`a2xXJY`++F^<X^n72}nzoPH7c
zZ%18s!yhmn(v+|1{wrPYhTMFfODx~s+Nf(U{M6nPvV5;<yR!{@9Xg@>d*CzaN^S0d
zf9rS}XDLqb=?JUy30zO%q%w8xu7`fn7v1GJUjse#)(x8O2+q@vyz{8TLSNOVKJS?Z
z9%wb*J#B#7R?%T$?2@;_gK-#r)3(*&=13EUvfK!@hNi}Y7URQGcXnZcf|(uY&Qmx6
zN|RAHCBg0P!kzv&2JtY_w>I1fyy;5zr1+-1uFmqwnl+Sd?=x<;x@h3?u8`iU=^v(L
zJDNcb$Rb`t2r7`^_zeq@om~x`Ev-%6<<)Bh9`_U7+q+sD+Ip)iH{I&>R@PSPwRm^O
z4u0mV1<wqEji{o94en=;Q^a2}?w_c@>VO{pL78?5bibR+DZW$1Q1d!GLlSntR65C9
z2?)HfSn<(K->D!T_@Rw_T^H_8SB1w0df{N{ei!A-hqDlHw>6UA<=bF%1BPWI{DD{k
zjo%ngpzGj`tqMpN#)c{F1>%{$V7MwcG}t@j8<-ja9>Hq4OY3di5fEZzvOhTLA7q)R
zd>HqqNIpu_{)Hc#__}W0F)|;$jk`WscvztofvnyS+o9Bk2jDRG;4WpuZR@?>wkF!^
z-&@gEvxY;gp~^DvtwUCO%Y%eBWwke2T@Ae3qV>PoYFDdqy|VYI2ff09fQ6+<7`C7u
z)+nc{+8fubLxc@KJ~d9gf+5k~C}UeKTW3QB%c^Cx?vmXO+^nqTDZ~TQv=*i901FdR
zvoFXqVHla%mMf>3dJTH}fS(@}tLC`1;ypdGEbMTtL4CnPlm3y>u!YT>+bU|jH8oa?
zZw!l#W*^@=tQ+EM{mA%q<&bYHFTQjuu^RT_p%gT&GV47Ru7VnjpG1;-mssmqRyW`9
zLP8%1<Pd?H11+)9Raq0Qhb+2jh1S{vnk0zrTxTLdwlj-m18rFdJ<mtYMlV?yr4cE{
zA3N-2t+T{y-O<S7R<_kFVx_5VX56wdHd%<jGY}ZkR;b+o{2G}a7?7^i19i0N66_C;
zGJ_p=)5Fh|Wf%oRVA3!KKPR70bD+f`X}h$UNJ&DVzeC>=j48u9f*R5=6%NoI4y}9I
z{1a2tSaIQQ@0fKEqh9WS^TR8=K&1z>Xw#H+Y1;1CGE#e}MmR5)TeM`+riNjQeVBGy
z{W?4ES59Z4Ve)8F4`6CI{@Uyd4Nm%b31`!&>f%vt*ddNh)7XJNy05dV^)T*o9j+r~
zZhgR}u&hSv9KJt>+q{E*JVAuJ*g&!wv~0&d(|vWc^51&6PCYm3V1bPy+KZI?QM8fF
zdjz;DRGX0uWgS(=B{5a<dWlwf>b96_P?oUOzZu&ZG=J*nSGQWY?<^mYRaw}Fkn<I7
z<vO+kaL*ouSiH*zYq7U(mEW_@+P#kUH}I_7pl@u=Hn5}n)J{WC+i?B%#*f}Cw<pH9
zitPAU-GopZd2+o$>L#OQjXSH6yZYF!!WN+R=xm`FY9rXiX=~kqjg%4pAk6~mDROKQ
zV2Z$Qm3O+<i`W46s`wK(?%6AF_#4+}Wx<mWI^QtD`yT00*GmOa*V!4o|M4c^T)>n^
zSHyiUy%kNXo1ukDXUE;WI~rQ)HiaO#simR2yS1CsovrPB<t3;trzNFI(|Zq0)6v=0
z(In|<GSa-T<m2*g`M4mbci+{~hUa1M^q|CVXm0MktD&ux0wChmy1TolOQj`Oncvvf
z(R4>If(%tgDZ-~$E$G_WeK%j_9pT>ghE7RGF%~s)JD9SpT@BqWy}043%0bV_RJMIP
zYQ&oxl-_-p<lDV{dvh<}fRp*HcsrBI;}&?FN3VvUt^9-}HHhnL*$%n88g}p8-VM<e
z@80h1O@yZOyuE8jM^}5p?xyWruG-MLdq)SuUCmv5#r{xN>&`u-I*HfQ%2(Xqgx;FB
z?^XF-+o7z69vjbIGOF^n*4=jy4}>;?#&p+R+d2MhXzFaG{M~nT<K31$-TaUzV1Gae
z%rgReT*GwjZfU3{JX~uw;$KeTy1o|oc5vNTU4v`;`U)zbd`?K*^%aaI@hcb;ao*q{
zUQF28gb1d(y0WIS8lgo^wfDA~TC2PZo~#8sUrmz%++mi*71e9-mc;~~r60pX@Hzml
zsa(IYvZkWCa(Hn42E>qjN2X#l80@b&P*YJ`F)}c`euH;|*IO~<p9~-3$i-XZ-OyW8
zy}s7FsUlcSKYbESo9w|kgfob98s{;bCvcv``83Y6IA6y38qV)U(+2OC=j5A!qbhwd
znik5&nUAv&XDQANvMkhWXf*9_uKzW35<=gH`f!e^z6u+;LdTH*xXM3`^ouHe4(T^=
z!k>obqG@5+P&gN7fh-S~8y>$(KZk1-eG7O&uJB%*LpUK@IEZl=#0s&OW|v-?>M_3L
z1JHmb@%9c@R`f~?EuMHr*4cXPi2QJ1uowH}Sh-FLT8q&;EPm{-(KEIL<56z02~$$l
zZMW4{HCEs1#SEdCQ!y<K>Gqn=^9PWLrzi2co%GQP*O|Y)8>`*<`Ar?WceL^|kMqmA
zF~cOHA&-_YAKn>l!!v~RS`5A3LNLGDg)N6#r0FKxSTEk107iH!$nWmZZbV?BcM!Kk
z!amhB(g)qvg;Ma6n|uKT+sE=<AEp$7!qQ&s&eIzOy;#~(UGiSIohvIr{vei4y|Pf?
zzH2WYG3QrGXi?iw@1N2AP<rx_-ph;0lPsJ#$Z^m}3NFWzR6WG0TBmrczzS?xet8$8
z7p@$y6U64x<7<?~cvvD0_#C!$?<0pkHSxDU`P|2DeC5J72EOwB%Q(Wp@ft6`_!JzR
z<ohf_NW1E--UL26O}Zz0Wy4X*q5!`UI^L;Xn4@|*4|pB)h8$otDFnU5D0yBUI*IhC
zv4{ut3Gu@nHy;$MUk4G5AE`R|H|1X{*?(K}$5;K-_pO&^-u)kMx8Gj%qnG~vnGbyU
z?_b$;*I(ZE^Ol!t@A=vCpZ$^N{PmyM_Q=zJaDB&<&lkS$um5iR0|jp^dhX;??zF}g
zAH2`CJs5bkwEEFkcfa}N$M<AE{NMlj@{9j8`RW7V|Mk#^1Cby6w@bHQ`@8?`$qO0J
zmVIr-#;-j2v+)1$-~7azPx`;|*`MwD?oVIc`B-g7*$<a}<uiZvTaW+sOQqSLd;Q=G
zr+gJZS{JzImb9r|;bYHDeBsRx-hcLs7yi8A-R~Xv&*Ojh!R6gAo!ok7*E_@i@cfzT
zzZ$Q2u_pJdd-b`K?-p<Si+hfK?bV;=y!ltNOMbrP%*X!nub=(+&QE4NTbA*S!8>Q&
zj|B49tQpD5nFy}_<E%$_e7EXi<&DL)_uu^Wo7>loU%dCVwg1{(yEN}Jzw?Lx;L8g2
z{MH`~{@}p-v;M5^;Fr&Q;4i;kx$|GY^v|D|T=!`HBO7o3{9nBFxwfCVhtAwEe8($|
zUe{mV@ms<3bN|Q3zczN~QyD$4dcX6Dx32&GPrC}f|EY}9J-#pI{vv1Nqj&GO*5<zd
zbg<^nzVq_+Pk-gwm%n@d+{)X=GVg47zI%1c!M1<+=ubYi;p{6f{o|7@XEqgP|KV4^
ze(&J!H=f(|!-kK1^S)Ex=r}m|!_a7D;OF1}^2<+!KlaEw5B&7Y7pI>Y3^ZL=)pzXN
zbpGd}&rd%7wY1C|hBm!@;?>K)Ts63|{OQiIpZwsFKmXCI|M}E^eEF{qUGrD_r~lUv
zC%bR?#_Fx#{_)NIr@r&p%U>&=`O8H={=0|2|E<}dto*l!JC43H@tx_6=*V-E*9HDG
zYw+Q(e*WCaFD&17GX2&6T>7OO_b=I)SGe=W^1na$Z>!QDK34OM=O3#7TInb2KKp9c
zHD7=Am8P3UUOSb$w5{!{-+T3&TO0rAo}+6{UGyDU+xyMH^P9i_%eTjxuYY{k&b`->
z{gLgF-I2|ay^*bvoso@^eUWXEU6D<ZJ&`Su9gz)@{gCaD-H^?Yy^yVtosf-?eUNRC
zU64(XJ&-Ms9gq!B|5M*nzf+%6e^XymKT{u5|5D#lzfzxaf1)p`AE^(i|ETY%->A>1
zzo@UMpQw+hf2eP$U#L&0Kd3LLAE*!TL}@QS=M8_^J2u7dtc~N@+faXqyaH!wn4UcA
z#flQIRU-bBIGDolVA44IT09Xuj~AY#otMWh9CHERGlfs!Ey<~1Z$D2A*p2NS2+?H5
zc*chckQg7$xbl4;mLGF435%HqQu2F~w*2JZ$1w||*!IKDcZ|s|F-((GFTJp=V0Pu?
z%qCT?%2(5Z&a`=j{e%0!3nI%W%X_Ir(U^AFGH?^{bO^mujZC|4hU4nsJR(qaFzlGL
zWL!bH&iW<HxXJ35&6RU<cu4Sj-wE2y`EfFFo<k<nZYn_^P`eV;ORwe1h=@v|D(H;*
z2pb~f*iZ=5M8R*x;Q2>ikY*{&o3KS~FDft?z~f7Uh`Ofezv(<Uf#*jj!uCJbi6Af4
z@t>+NDl*R_=}Lu5WtZo$Ai<&V^=~S1UZSr!{6&dyoOzeeM;N?P{&R|0;V=HLv~CkC
zk40S76~g_=4+*hk|5<|hN@)>!3?p7ptU^tQic`=(<?YX%pSPyK<b4hiFVp6tyIhnX
z72TGiUGB`IGCE9NS2}(jCm+@Az|T@~-BEFm$|F2y9?Oy^=}Eq(LRrTeuOsga_|Z_m
z4wj+#wREbylGwb+W0KCT@Sb@Ybn}j^tR+jZmN<+vg7cV2cje>ycvNic5lhl6q@Re2
z%4z&Wq8I5?cn2_FEX}M(`e~$(i50no^8zzoZBhC~PM6mc{tTzrL<s+NN>|sc->~u4
z+iExU4-Ddswvkc)`}d8FPXvPdCqq*Q4o)9>5Uo&$(c>H0Xd`|%sOjTI7kMX{_xvv!
zBXrga0TTRQWt@+jfwTAo;Ij@G@@C=Z%$Nq*S<rjdF$a7caMS_+0>*W}ftU0Fj(`w<
zbS^oD|0;BUCjIMc{JcNXD9=8w%8`}*xDZR*GTj<~Df(||kdWoEGz^&VHB9(34)`^|
za}M~T5IF?~&Vl?noesDM@GF@vU0==&;I|y*Z{t5BTLxZM3V7N99|n91u*zC?8t^$s
z`RnMLLIcw;?&ThHz{de!aKN)dEO+292mbQ=P5JUZz;7A-EQfq69PL?w_T<%?^1M30
zeTF`72=EgQ7<Q6((gB|W{GtQKxX6nFR#|!I-~3Vo=a&m{jb-$A4fJzux+=%zS~|V>
zQpdF$(Czt|OPF6S(y)QTTKp~<p->v~7M?Zug)ai8`qfd0`fn-+Oj$T@T8Hz5DZl9n
zz!y#XR|0M2>k6OJ=W!S8Smsh)zjYk&DMN3a2K=%Eenp7lAp;kW0zLznvLJu)3pgG6
zDu%vF>J5HLvk<GY4ZI38rI`jU&BmL8GssZ<%3cF}&XkwE4)|>c90mLf2RtXl>Nz!@
zz<c#Y`~uxfW}4<#4t{GgRsW!T4eDF-nu4M4HRo{78GdU(zt%GFTJ+~yM|;+yJ!_9S
z%8v_i%cy~G8N*x7=|-M)C|WmW{K2{);FpcPtb=|l>P&e>J>c(|@`^Wvs4NE)ssd-_
zI-FqzBMZJ)RK1yre-;M7i}YLYqPeR6KBV(;MvOk-lSOsZ0mGhZ%1wPW;9qkw)5H9r
zuk~QNp;Y15BV8`U#!kh*7U>?G=M2A%uk*i-k@gJw+H9PT{;Q1&{HH+@Rvw$u0p|l&
zS)0%&n@$+}-2{8yG~+0LO^D5<rhIcb;Nz-(plyB}=LH8mE5z-OoA%s(0>5du&gi?&
z3)nIK>rUbAFYTWp|CT<S5mi4fTcE!!XOK|GmKOj!>~YH+eqV9S$hS2J_zfe^)(b*x
zqxPy}8``^#_-h#a>+e_esK34sXVmmhJ^HJm&cF@zcypx1&^NXLegZIM;cPsKbH)Kf
ze@z8O9*QHHo_4@z0KaAUH@%IwalHm^-hela3YGpK-*yY9wl7q(9rEv}H|0B;@gx6j
zM*kF#w2UcvAYu#j)ABu4Klru0fhT3^RegZz4oT~2qtDi-h1j*h;P0vheAdXf>qQ~%
zaLk`~puV;qg%4e~?ZvMg%qV`KZGR1SGM-lQpuXK`e@DLI*8!T2sIm7B=)bep=&Q31
z@OgvZ`91t%kz+o%vlOtSf9^#8+!;oF>bP?n@Z&}wcb)+3@DF#sf;%w6)JHKReyvfH
zi!LO)_BzV%2mFGu@2<1BlTv8(N3m&lkCC?<`sjA(w;THHcC@D(?dc&yQdRb#{+^(z
zuO|%nb;GacJYX8{8b69Rb{+NYL49|54gFmkgxGr`b1~!RBK-tTNBw(I|2@L+r}+3D
z%kaOaRET>osPa*yXK~VaRQ2CGhd0(r&HUP14!BR1<I+2XljcAC(b;<%=PL?ER__e{
z?S^B#^kKa8?KS0n(D%MGnK?`k`S-ye?0e0W?>h(hO$YoIb}xgbKgOWH@e^izjHCS%
zPh{Hj;lxR-OWw%L*8GDP06W?jMEmw1H~jWL4%jh&?0*BW1Ajm8Cmr=qqW)0O)K5Rv
z5%wB=hc^Ir*i-l{;MYxi!soF{E-?60(9cwlD#vANuMh`bQS%e@a{&6B&NS_r&IbIl
znJ=eb5#rD}<3A3)j#c!KY9Hty92MfB69)ew=<9I4vG>CT_`ULTMt?^@fApfkkDyOK
zFm2!u9LDdXhYkG5G+?bS$o~lR^}*w&Js*UAk98^-ypHwYJZaj0?356X<{SHd6!JVu
z@frT;d=&CN`j(-8^lc#?Yc}v>Er2Qh!XKTFK_4GlXW$Qc0iQATedq=J=b@KPeaC@+
zoZ>}AfBY9he3;^A1%J3th{s3Oc&J5sOo)#Sss0E2u~8vDQK<T>1!)WCn4*VBp9tc7
zLBY6u;;ay#JZ0+t<Y^(EC^h!<L^<HorhlGz8t|-<|A{w+I5}<X<0Sgy<lCwo{e3bj
z#HXiCy!>gDe>!T)KaKW%+7Z8f`l1j|zHaJ&^1KkI&Y1CW3i>+rx{;sO`Jd72In?)=
z<3jwNj>iE1UcC^XYccjn>;2C^VfcUkq!7=%XvROSqn~xe56|`q@qC?XUjfqK_k38@
z5B|?j3-N+uete-5@S8?oFT5qh?^{OR-!B#7OWNOq{!91ceA|qVFGYp;gR`nXQ2qxm
z;@{S7F#0-M3)nH9&z=?Hr9LH}h4hdRf7EQ+`$sK+H9s%X;P=Ocia+2#2F=Sl9sv9@
z!LKNJ2%Zt*Pjoy3_)q$T_|t5||4(y;_={5I|MwzYF2q;fQ1*EO=?g-<_CjVB+b?im
zJ1fLjXN-Qn`Wj$8UIo%~LVV**<L|!#{e82~*xNVJzHc6bKGpHf<AC*g0omUKzi*AH
z^-%%RK_R|VDCKjZ{_j{qd^caVkKpeD=esA3eSP<o5Pwscxs2uQLmK#RWE%hSMz#=t
zn{L|wx2W^)9R2@yokIK|SG5<kKgbv2M+K@qfPYjd#E)~$c=<8Pe^O}d@h8A}yWGHU
zuM^^@_?I{=&q<^KM>m-AXsr<Mt~31K^#UG(Q23+s-BFxRJK!_IwItituWLyz;1;{w
zwWLkBvZrnSuI$5rUo+*|=Y(s;2Gza-q-%vM?`?ZNaOFjXE8oOFuKZVo>ssUgUDu8Z
z*LB&py}PbM{X2t79xu{i;c9)|9&fJJ^TM^u*o$k|Yk=Q0^6Yv`xbDDoq1x-ZBOCAu
z+rC_PJONn8r<q8<Azb&o2m+8`{p-5tW#PI%Y_F$X_fHGgM33r!OSpc!P`FN>RPvQ0
zeF~@cKcGJe`976x>i-n-KJ|ueFRo8r06a^T@!|T^o5J;!89%P4fd5p!DSxT}aDf9x
z{ZAD-V94~8>0j4V(C1TUP);al-`N@A`Z5EuzAvW(Ci&!nXT*f-4`&Vhhmhwb2mVXI
zf60OW67XMg$oCTDduhhdzw{bl>JP>LrMCdTZNqr}OyIO(*HdSO>zQA)*!W+-HsBdy
z;EBD$6*y_&GpPUhV+M|#7p@PKXKH%h!vKp1yYQcx(p)!Y(|!r<Ylwo&m;U7#uJEsX
zSomItyw`=uz`p=kp7u~<Uhpxo{ZZGkv=6!^4#uYFynN|j&kEsrOEl*Nf%TZ!sZd-W
zSSMT`@*3Ov&<5CtV{cJ1N%g(p4S`g|zbX6wg9iS;ZUQ8~3m?%?Z|VL(BAwD{N%OJj
z5=((*4yPU@&nv)Dzxmxe4L-P6a4DL$?sPP*2=6BSM_<(S$Mq4zA1PCXBi|iO`^=~*
zi6LNs?%~bBgUn%OO?E`nUb=YEF7LF^{tpl?{^wKL!6*KP{ySk5ASvZf>AXAxdFSZ#
zip%G5J>Q8F_p8oZ;_|zIan}jOfFn4Wl^~jE+Vf|kX)oY8v*)Qe;X!;+j%U!GKZ$cX
zArdmF?r7R`;P*WEJ(q&t4B>&_bKv(}a(-w>H0_Jv_Z;|rk@%%uK0m{9J7u7@q8$9b
z2!3Bo!LJW^XTa}^;CCjue9#-p!S4+Cok_vZ0^YOWcLw~PRr2LRzh0+!fa?hl{GJ8B
zXH)Qt0PhRn_bm8*A%%P&AUyE<0{DF)DZgJrifGz1;P(aadnPHrUr0Rgdj|ZTNy?8H
zMAM!Izh}Vj>7@L~-cSyHPlMmnN%>uVHwwI`!0&1Bdn!3U3>K8vM&15&T#v^n_cy?A
zG;p;VfP;6$c?r0%7w4}uE3!+7-%WsXm05g`I(h9G!J-D-XTD123COs<%i9u^a2Ab4
z9Nbn)V8*G(^b!&8)gkNOnv5O!3><d-u0HV*?1OQ}$a&2BzwROHxB17jlg`JYY>87(
z^&b@E8{fP7X&IULpn8O`_?y2}{p7>n!}>XhGL@JV)#I$+eV%?I>6k(IbyKAO%-p2y
zS22m*Kd2x2m-SD0-lP7{%uV_{lX{NH)c?H*+W*k6+iF8E+i&wX1u{i30uH;}eV*zM
z%fk2KTh^bH6UB)*sD2)6s2}?O2m^2Y@;%dk233;ipjZ`$v;C<55%DQxwKu$H^?MT4
zO?j8k_tC{sKiS{ooH4lgJ*z+I_@fv$w*Cm-t@;4vy*CH-KL%hjX0&F&;pl%Fd>@XG
z{=fR8_o)4sv5S}FxI8yQCAR+4EUACW99i=o)t|{fAx<pk9qPG9yZ%wPz#DuRg74KK
z^`Dd*t+8;}^>@~bq`oRClT-y<fAVmL)`B?f`j;_?uP7eSh>AxGqGBulIanj^wtNr|
z_dV7U6(4#bDvsfSzQ<lc+8c}i$XryEy$1X?Twe$M+fh;f3!Hs`qkx}a)^x9hj??^o
zX@)Oi78eqiptJPlI7+wgXcdDpBkIVHrrnEosea!U!%_7AYBDN5eK^YZq)2siuPXJS
z+X<`@oWnTnK1faZo6_HC;D0X-ATYsGdun(jok^jSG*M1VCzlJUP61a28cHEAEsawv
zD1%d_xVV8(PPZsrw7+O*Ro)+@m8Gq?@$%TeFU#zZ+mQ419u?P=Mg=|VN#lMw++FT$
zRM4{%3|mpr6UNC><N5FJVQuv*t!aLh$sms^{omw>;(4B^*!09~)oj)8h^NKuQhc&-
z&qp1u8~GDWt4=G%b0orD<F0p$Ln3{B`YJptB0L*Buz-g|#>R{?JSQR+-MZ*~i&j1?
zGH=Vg8PAD`#kGsq;`tDfwJ8gIi`N0XOV&_amR*tEn_cvXSh{8DEkb-y<YeM=G$*=j
z@iOs*cqW>=Kljeu<+)<{!R2=^zizo$;ajnNg=dAxo6PIVTahR7!}&e=`T63SL)YAM
z&GpxaYp1T=bL}<PitGH>wO+UQI#CcP=qOlLAg&Ky-+6uR_2P#6Zg}X18*hl-_`r=1
z-+0rF(ZZ$pWEI*|;L84$4J%zMMNz1zyC|<nSol$DE8P;s!^Jy_Gm1sYNXgEUMI~a@
z=&F`gnX5$U`%8C~W|fMvePwr)Eh!T>kKNpMbN0<*_3f)S;~Cj#`Rei#JSQ7n)3@db
zo`1FLx#j*_4&PFEOLX0X>+W55!#YthU9q>Kph8p*RyI|-D@E0Ts=KPLtrFe=Z=*NO
zE2<}|cUR|Bi<<G8_L`+NV*R@Hb$Dhpx}kCd_5wCUH&$)Lp2S9*|7~k;Tl>4>HXGh#
z$&c>O+wrW~`^2cI6MICj@QW?tE^Pe2Uu+e3i+-^W0p4CQAjU*Jo(UTi<Dx;_D~7~`
zXcYGep9qL1+^!oILD4K8z=XSBv$d}~#m?If0%{R0wZDyPt7zSH4A)&1yDB~^c17<H
zcWej&ZWC=AKZa|&Xup*P`R-eG-*Q6ij&_KSYW#aAIGfIzkK=l$xO4phT)RY<_Y=5w
zi|)!#;u?S4b^l!)D?BPbBt9&DOFSlyBLIG!(Da(g<*lytR!hS7I`40Uhh0-;y36%$
z&QiEeI8XMQNEF$gDh=sn1T7&kmbio`je5%cx^R`cm!)lU70Y9F>8h&Y(p9BZ#l@vn
z_%1D8RlI6dbxBoO)hhgmcQH!kQBqaxzTN^<Qe5IiRtXSSXTF_Qf5X2Ouc8bb?rf{7
z3MB5lUGBwc7hNvSEe2@`Qf1)JM~y-+0WuY&;T{sTVXbiEyEHw%U7<;Yk?MyZDhFlM
z<=XrlVQ1c(5WBJpG*$BLa>0Q7t{Oe462k`tq2Pia#h$Dh^AA+v1*a<h0SWvx>(q3`
z$i!4tb#;}0VsLB<|MW|;4B-VfqSIGZ1!aLXe$N{zpsIwRpD*Ws-C2oS^LRBHKTm-a
zx&=QK8=MNGF=#^wzqy8AvFG#v{l=<H?i(1A3Hs4){09~Q==jO-piCgwPE^qQcLt(&
z0wu)k@b099@OGn2&`*F*A_+l4qy6~d4x~wt(SH0!hfLEuaWWMi9O3^9rctV8Hobn1
z6hxv@`4jMfv22G<`VZhoHxxAJr?)I^0PP3O7=D0Q>IAQo%h&GkD`Ct|suXERhyS(X
z4~9r25;g3{``$85Z+GMOYym)S^%l4U<j=|Z1AtHuUh0>LL8I6p#8tSX{X>C+NJDB#
zC-Xri-yW2-<NUTF(hx$WN2y-w7<vOyVnIBGB41pUx!ieEj;_LMqNW7@lT!k=&PK_J
zkVX&mPeMVcMgK<@iHNsd`Tw&>2=Bu>Abz6}1t1#ce+DR#s4x713jDLA7?|E^l^OU+
zasIypfT;V%Nyp$ZK)>TIWuR6}+btao5eDguQIWuJ^x^ju&=1rckPeM2OM_PN0;b{z
zlcL|9!kgWUWVFmCfgyz#e^DO-DEdDDR5iv7ej9urQV975hG_60VRx~j!~Y)Aa^go~
zfJ4G_e{(X3_S4UQqX1zg{Tn{ThW`PdO5>$hXaNBFn8pdSW*g^}Lx)b7b;ud^NMneF
z=0<AUWJ%x_Mgsu|%SD614eM`{NgB*dNQ&m+YeTGd8^Zgz29Za-UBs5ikRBYYKm2wF
zD6l;;?VCWt*;xRgP3mt>92lVXBj^1=pLR#%YKS>UDc1q7pvVsxLmGQ<ynhJ)d3A^w
zp{8;A=XmT%SoH>E>uJL{89YQk2LWvK3=G|LlT!HjI7ad)O=3!sz5E+-G&I1Un#8QA
zQ!9o-L+~lSX&OP?%hE=XTR}hlr;9?AfgeDR&44kE1~rOpG0Y*ZX3kUMvEycas0{Kq
z3{M=ue_6om!V)r*7fcgYKtE=L66~=J;D=Tyi4H<OJ6g^MfLv$=3FSYRGB8XAAd`$e
zr~vx8OfoDgLPv4}LyGt~()1536c&D@eVY6r2tvS;CCqvtfPWxj6Alj8$^M{n4e-hI
z>m`r`K33Mk9+!9>#BYFy0yMoyDIu^<jvfRB8coVUCTJaMbHZvuB<6CEveLDIFyOE?
zhC@Sx_%95SEB*_W8i6zg_?5$@1PhFxL7};WhvaboWH78eZQa0>e=Lk%llQ(nl^irJ
zj<}b*Jf1v{D<jWSmVwv?Kwh3F&pm<cYxDXtTp0y`#u;h`bUmS_r+GGJxKFz*Pg=%x
zo{cwp($>>=X0c~uMV%*Ywa1g;e$etP&Tx4$=~Cq>M$)w!Wf`?z&uaG#l}x#sDe(=;
z)fJw_t5I~wvOr#}iPJJ_+9;i|#*@oDJmndqOkB>y_y%z~6PI5J@yJyZr@J%UKRz-H
z4Hb~W+|y7>xyR+6hQPRP$#7eBq&}48q6yWwF7xEA_qf)079I3tsb)dBXvg(?Jxhx{
z*$^8|U5bpw8CyKr8Lmnsyq+v%crtE8LB1!u2+0ko<{sqbdooe9qzb6TBtq$~U3H*v
zKTwJ6<rx+D&W55h+-oXjuP!I;WPzouql&Mdj8(uX_IQdaalKh%dcYEu-Q&s07|F0Q
zs_x(e7+z1#I*+Tw^5o<af%{wNc-Tyd3lil4n|s-lu@w@K+{K<{xVpBar)Ri->tRC8
z&qL927TU8F9P%LZI>O0dD_r`xJD*uCLEiOHOGZ%!DMD%{+wxp*dGNgyc7zsM9@mD9
zGF8*!JZ@1wN-Yn*71}Cdz5y(5luBidC=w{h<5fbe^xVJ<Gu;1|sFUS+o+TNUX9X1C
z@?0a*($OU=<i(TWl>=pQ#x2xp2$JDmVJmuB7200qxel$nrpU8IcK3DI{7HAAHOtA0
zE5Mh!+Ur?P+CZ08f*c@JQkjlHa}l*crKG>yJ3X%Vr)Olizs@QIU#NMD$4ll=3?m~$
z@MLU*VMLB1m7l%{$$zFU$<I&EK(YIGkGNou8Scx%bq%R$xrchVIK%yE*?%-VU3YsH
z7ZXQ_kn71T;Uux9K}!RUdV#K<jGL>#!Bv7B_k)#=GU`kV@bdIbQpm-_wnmmPXE=*|
z_jjbNq~&?8hmivla((Fttd(Ryj~AscCgR6PVp8&Q&vjC)#b_!ak<#c&Mv%`}68T!>
zuOxMmwrN0;F@g{56l`t+Kq<QkRh~7-%8^+#45`DU(N;0`HSmAHCM;JKMjef2sf{h3
zk_;D`lE?k&9)Sa+R6c!i9@|RBo3eO?#a9=#>Kd>t@~ot>4axFYGP1SxC7!aC^i9RE
zV{Tm)8dS~6wHT|Fo}2Ma9@GoBN8-8<!p31?`LLPt^sEf`caHE_EONQHJdX?-VlH-n
zKFzYyvphGrfABENo#Fmm8lzITdoq3>%_sW*`7kLgFJnC^%y5BHsNXW&A4(&E^E@{~
z$JPsA=yK0JjCMRKnW3lP5R0M4Vn|iq>ZvLA6rg8HJ=<soXtF#7C7#>xy~}gWI#j#L
zt|Q-51dDU6@?2Zw$%W5%6<e<KB^mDDJHi^WMWM+i&;1tLPx=y=57{-h@W*W4zef!L
zoy^U5f1iz;1%3heF83FYK-^zSlh@*UJ*5!$7Ryso<f*X8)+$Oo*IAy@B2PJk`8Pq>
zGPX3b5g0AtDrE~*o>i>w#pIsJ_Q)tJGxCZ&w-tM8iab`ar?6-hndCKUm>SAL)LT&D
zap6C^kTJUtR(e*0c~-_oKyug>u&ETMXJ@!SB9%c_T}mc}&Zyo(t`(G@OGgd)uniKe
z7}ieIxt^t~a3PT}xavIRL<GNc6J5y{rJ;=72KwCn$I2q|VFLN-IT`M6(p-k|=iU#d
z`OxxBw5dYXWV>AOi0L^XcGpu)T&pJ=EYUUW<=h95M=;pY#eGo2#E5g4nM9G4lxAc-
zU~)V~jQJ;UYK(%J)$1uhUqG&cjOCW62o?qt!Dt%sc$On?t19voRq2vDP(qzA%i?V5
zku=Da$HnenK8)_WzJ%@RSQ@!YZBLglFlb1k+fkbD{#Uj@wky(t`x8g%EEL@6e#hmi
z+Y4?`%qL)(IUaJfc^Q>3O44AS`(36~D%0#g2<JzW0pcSH6{M5#a!qgYB)~OQo&aMw
z+$Vv};ZJFXyOUjHT$?`QXwyGH^6N`Hu9e`n7^B|Vs((V6U2P5fK1cwqa-1gXKgL#J
zk04b4E~vPAH%BqG<O^x4h262l*bPII6YJ&?sT#DGRO8MeOjeG2k`;_xjL&=?^yK$%
zfcMC7cLPpre|hc)T%OhF{VnOsGTiSYy1cyW+&8&Ac`i@>HAK7@_6Z|GxVGl|o?K3$
zROZ|XsVT)>Lkaf@2+!!`%iU<0#LQKhp4BoteVJ!1<3D+{QZs9t$Lx1fGqYLYSpro3
z4!ZR^N?D%GoB>}{%87210}ZnX?*CI%LI{cY<WVv{oo^wZM!MVmBTNU^N{*}KMdH^W
zc$07;T`=QtYtb2vo<(fs#rD<9y|l=^-zugaw(}Qd<QAvSDW^VI1gGXIMkwydb}zF$
z-eM0NBZU!p?l1q}#JvZcTt}7fuRGOsJ#BT*^i1~zX;SwDW%bNRvW%@61(RnalR+}^
zu^`#N5@j13TLzh6lh&BbVwoHsalyc178WpRSy*6~v?Q~n<&hrPEIGXI@0@$PTe6J3
z@ALospX|@{4RtG=I(2fLa!Eyt;uQjv7<m<%b=(DGfi(Rapb`1iD8pNlgr}wOPC!2x
zuJf^nS0)<wRQMiaE<vXd0K@AqTj2WZD<l=q(#%`iEna&YEq=AL#Wz6isC+Qoc*Fo5
zUY#}ewIlVJSR0AsO_u(bXtN}V2!Tv;qv&FGS%+hW3+e($41G<>C=C<cK+sjO7v|U)
zH|R>i>K$!}-w5-EV(++L_rg6AWa-iD%|PtU3)Lu;Jw;fYpx9D#A+8L6U7s$%Q`7K4
z0OjkV0fh5)ao^Uu*xbEv;=<U+7ebAiBX@Wh9k+YK7izIz6dNxjaYu_~vwp+pRxIUf
z;%+C@^T@wgPgct+t7SZiyX|7WnbmubrsH+9{lXd!f5!m5?B|$+TRvNe8?C0LW*pW<
z-R}{o{La7K-t<+`jJ_&1AGa!Mo8xJrw;4}C=o&%64Mo33{G9`}O1k4@&DqVf5{Tgw
z;J_(qb$<P(dEIf{n_mC}dtkq-H6sXo)$n%1PA(qBi+nMhc!B(=R|Dm^D%238!eao6
z2<AfgJa`TnUS|vAG;Tr1IwIezFFP8yHl+`3UC`fBxRH_r;gvwG6uu{jbzJ*Oc~jn(
z$Nh!Atj_u(R=-NC8)<1EJOM4n?pPP=SRL!NI%sxWEq{H*{#f1Q%VfE|OaPXz7%dYB
zYLZIy&Q{87b9QcN)BN)KxxKTq;UWhvc+E<YR4G{cN|9{PJF!7eRG&0UEr;2KrRGxe
zpjJe43mtsnWz7P*OS%m>mZn{`7}XR=utK@)F}Rs5-vmFBa4P&|3yYcgH6cI(o*1C8
zCs~4RF2(_Fi-^sV5s=Saey^b?qp_`^$8<+bJ*L~eUr20EmfBRUtBYk8`)I|Kx>2dR
zXXnlE2{pW;C0)nf#aKb&naEhQ_alKue<qk9pho7@t!5O(H<@^gHQjHYBuMtDsIdy#
z=mgFO#WmJB*VSXXEK#7)ueNUxyj=(DMvh9)!}v05#n<xV>SvUKErON*q8`}2ppi71
z=!24MsqOND)h1}EyhLV(Q)H4v*QPTV&BsklUEK+!=x8at9kq95_*b+T%0Xj+=NvCs
zkK!c`^?B=7k=`fa8e|kyU#+bYGHza|%u9w^*Jt5Iq$G**6I<dc=0BaR+N_ENCz6>R
zxP1&40f&bBQnOoyLuhMhc-N(dr!1ktOs)>!Mj%H?dNM*}f+ej0<ND}RZ1Sts+F+aV
zJ_#J55hRkHoAhf+jV{8A;7UePjlQ1j5A#xWo*sSxi!#2w!=&WUP!;sse&CeWMtFIm
zX%2NhbF$X{j<&XT@N=J&5^D5709#H8j}owN$?+#Ol2g+1)MzZNjn%2C*f%paHpW+w
z0BnVw;Fdydt59o39qI}tad8c;EBs-F54X#1Pxi-#s}XgDKf9u5*H4B2V#M(4%cLy;
z`VqT*f<vH-&SWs?3MUlzju2bHXWnS5NB?#EdSoL(9llK33T6y%`dsIW9hy}b5wEYp
z>(POGN2ZKrYc;{XCvv8JOa6UyA(q4Atp8cxD{1&Iv_DxX1W5t4Y(}XHclpzGyoG-C
z$x?=ThM7Qb{FZL?6{Oh(s4)w=WcpbRI!VD{ZXeB5)F6S(4_Z|?437zzt$G^j2AKRi
zp{A@mh9-Qa+QM^Cr>DHW$EyObk2)BDu`z0-OEFwTeQT&qm#UF<@^c!>^n|)q_F86m
zAPsM}=NT=Si!!}U91=O0XgE=r#aN(T%vl*sBvsR5V0JQ%h&@8W#m7cWnZ(|<-7rLd
z7F_oP$GM{+v3_u8hkjs?{!>oRum-SHicTi;u`*s51bCb2uz-jx=*(OI)*4Zx5T+Vt
zbHH1t8`9&|hm~SA^M#=_e91Jq!f@;l&%cD?f+3G@3JK4+;9+41nBE-fX?Pl)0yxPr
zriXBcSe!E+lXPPb0-Ms^dRx_t^nYT()Dobe{`#MJ$}n3z2q{a`erZ#`-8-B{Ec$I~
zLYuw{_9tD^<pSStS08I}?EV^}+A3jTjAMF#Tw5xiYe8t?x}9(Vu#w&Ak&L+dMbxod
zs?+aLn>1;mrK`ByRaiwc;dd7?H%Z##D3#6gtUyo)MiB3306e@8;m;rzc#D<9=0Pl6
zo$dU>Uce%<FMl81FsOcZL9LB?Xyn<tWxnC~Y50w!p|sH916GHOq8PBmfYi0(?l{Ak
z>Tw2k$~wP5OHD6ju<e@_V;5@V7Dc8T!!PzRo>i%|50<iWG4^2{zMO~4430$&hJiGU
z6M%~RPI6u05~JD9h+1A9kHf6%@z{MqTz5Do&sx}DFRV_(|CZ1x)MTle6<WG)5AsdZ
z7u$1O2arLqSwL`uo<QoX_$N(3@$1j#*L`uUAqRl}?Y=x#kg~#fy$SE+<3}viG%%!>
zPjS0w;Eu>YaO-9)@0Y<FOXW(g`KvJ>$coHEoA_+y(T>kLpDmxHoPX<fcjig+l5TOv
zbrm)prq*SZEm|_NO{(J~^mlU{X)G-9yTAn>Z5J_?xS=`FERD#5+8c@}1_Jda9?3#r
zER$x8b0WCBsL6_aF>@?(B`oNbLFL51IYE*INA`Kg6HL67^?VebhhR(hhT*S05_lMe
zbcmjjm~4nW(=pEII;MN*4$T;$ZDgG_Vorzg@2KmdHqz_!ekQw{A_U=aD8#%frqi2s
zQ5Zo;3V(O0BtzT~ydi4IR$qe3Ujl{+4WzJfUfD88jSg~0f3GugWis9A`^}+6ZyA8_
z8N+{{oIq(PBd9|XRe5nd4oo6Zp#*TZ3dtxvhl+O71kNjH>8Mxy2ec15q(MNkfzM#z
zT)V7aG6@w~6`H4Xzcp6c&*i*H=TL@+9K#jqp08s1DC_!M`pDQ1MFKWk4SmYcUkT`2
z)4Rt(`CAaT@hPcAiQ4iM7)MCM`95rO-XKio^IO{vvSa=1pOL?=x-TYUG7M5cw56Xe
z&Kk=Rffr?=T4y{gJ+%O{rO#8cW`#G1){9rl0D5vO)<nR-$!wLvR|D%S8C<B>>*>6@
z3mqSer8Rtxs$<;|_A_mni+~OkXPy6bc1DXnd|X0sHM*OnPW^!4dW2rYkEJwxTk|W5
zfgED!jF`Ju^nv;pv){%Ezms;K%oaw|@TQA3jeO5g@jr~m86Z1B!^Lo{t6mtT%ip<F
zG|tR|)$mF)N&*%&&=B(*jWgjt=%abZ+}ij8D`b9)SX@Uf;fE@as@`gN*-5Hc8@}IJ
zMSpme7ZT%cwUfy2LDS*lgS%t0hkhG?uCSu&Y8rlJj*%Fi-G%L5!{l)czrIw$%Ia{X
z{3kp}dBX4o-DkGDYDIsW6@6B#=x-^ie>mWhuuOMqNm8wPb6c7`jp1xOtLb86azol^
zO!}M6b-7Ak>8nrB?UoxT)ARx)l}=o^0Qqx7$nABeCrRv+Zetuozw$3v3ga5!Tah1`
zFOfaSV{e0c4)DAtAH^sn5wiz1vJ!U2Si#!GC$Dci;6(m$&475}L6iwNG}idlH0m7{
zZH|5UJr7iSfZ3J99*rAOr8*GTXbSFy7vUl>N5<nZ*~DVHHXXnLT#L0W+}vf*G22Yg
zh;Y*0R0!@z&x`3!9II!WbuCWCl_pT{h2Hw|55vOl<t2%K;qLnv(<Y$UrXjCM%oZjP
zcxM<M9>QWMBI$?=tjV1BeRtyi$@K--?$dR60_kmfM}8&55|u!^#j)7W5b@2kcv$)B
zc({0}M&IRXxwXFUR^O|0yXdYU&pCLRgccsM067nf`jZ;hq2H(JuGei;R7!`}*bE9M
z0I*NTE&m$eQ|JdXmM%9XviT~#{ofQOu<{>a*x837574-V24;TmGQuEaA9Ty`5-p+M
zJi+LO_&6nc^iNwaM`AxqZbR(e&8PFw-~mCedpZ`ioRFBez)x#bEtJ`cGeC99S1T$f
zZWfp26qHmn2-m{1IEQyFi6Ed#xXEEQqf;gfz}5BxtY?GK*#OMunxIjEc71g~Cna?l
zCz8x|!!qk|bvV09L$Pi`et6trD?f|gF@)GRmL75;ycc88L>v(AVKX>hqyS7i-5xt>
zs4&kYBFfAD*mW$zs8L+g@LEZ0|6N3kx4=O#v9=hCcM&95W5x_`4fl2;?)$m{;~j<h
zDy@yDlUhE)$0aN^t}%qaG9)}6AGh5CAEz`5pKSyfY&a+V8`EbSI_{+n0vo`S_sI+8
zxTPQjWL{>!f&p|%>-LMtpo!_D1ItWwML|i)jIHU9^WqIMO@c(cGIX4W&BFlexxuIc
zhV>g=fJz8SGP};!Fo^se{&WPMgqiUth`&O!k+)LSY%Ol!+SJB|z{tSTFhdJGHMB%P
z!sw>hn?C3Py}YmF8$K7g-5Pgt3C?})2;xv~c0w%J8Q#W9(U|0wpbQ2m{_^0hNwTVn
zEh>EG2<0Y_wkV*Ax6U9!iHs0kM1x)#;)Z{X`H$~qliGS$mYR13s}Tsx3r4n_hI3@S
z+6sf0!9bSOzBFvZG?CsJ!JkgP$5ep<#T6ZxXlQ92vKXWC1xC)0gEJaN#K9&F5AEkr
z$1|j9JxkI<1?XzlPZ_Wu0;~qB_ZBF^puK{|GtX-vSV)Ye3F#u+(r^u9<?%N0wuY$t
zwh3M6?;`P7?qE;WLMy_B`(xKo+HA7uWzXo6&>6eNn}wi}d{hu}g}h!sME~pxhnUM4
zxto0_*r)QiiW0o!MVv$EK>34)+b=eyvg;*b7@9mD&IOWOCox6kFAgD?J)E)Ch!e0f
zRAT5}oECpk$BNdmBDNK}g&B<DQR9778W6_3ma|oh)v+m|nfqEZTuyUI$L44iOXRdj
z;C70^khj<QORcr^!mQG}t5SRwg}l-`wJP1NN|&rsE~mMq(lfJ4)m0%60IVPo_}?@P
zYAI?hR$@OZbUgN(Fvs6qpE1p^QeUt&@x+CU4`XVlveoy%tmU>8obttGLlcOq8^|al
zx@UF?tSfS6b`dS*4@Ebm03nOXByy#ewWPuZ!;@|bM!?*<4`7I87~*4n>FQW&0Nk{U
z{A*wR5*ujtRXIF|wPr@3d5UV4IVRbo-B$cJ?c$~IEQ(`HbBRd?FVG&7tl4lc07Vnb
zSW5+Toa<c-!df4@-$JhmT^TI&gpXcIYy*m3vjBfY&yqYy+yp=lpGDe4^PY_J;oV&e
z2dV1Rq`)$KPur3o!=og53^E)UTJ5T|6?KEJwudhuOtNUgJNTftQJ4&$zZ9oMcqf9j
zu0d$R3cO7;fh0UaZCCNAKnj@ip+tQOpQamNttM)5A%d3M564A)S_;=ef|D)#(}w^)
zdfyiZ6tLzNHT^=W3scx(E-^}-5!)m^Gsa;>c7^|KRl6AdPXLlyT+hhDx!!Bm+FJNj
z=ev3VTVWbVp6F_%yKbst3E%WZlcY~!(KFvz+rs<Lx9&I6@bOcntAF79gK&dS;*L>F
zOA_Aa?@{>8I2V3&u>=EYFD5oOxZJ`}5yJ%q5zuanR8n+hg14b~2Y%#GLJE{kD6>o6
zPdG`4i)L2c*d&8BmZJOZw>nO>r)gNUV%8F@)ml>c6fF^L_iB)#lI{bgs&rM@2r;c#
zNM<sRBMC-_^$NyOTrYDB-T0u<GFqyJ4?D~<DyB7of#(KLSJo|f9s&f;BX!z}zrVA%
zDORfiQ1}ENNSiCwQDznPDg7^D`9Xlt1!?k~Zez3sJ!5vo8B9~|o>xd^=~L9>fp**Z
z4qn0-z{+(BMta$$GR(?uB@igwQByLo1Op6@)(nN71%wj}dqxw986F@$uU1_=;Kl-M
z1HMS5?DxegjDTs8VXWAIA&HwgU~aZv4tV6(w`>9Xi&qi{AZ-(mzaCfwKz2{RGC<4+
zdHMlbVqt)3c+!%Mml2>uA@(KL8by{Y>CAo}8!iL0csx)I`2jK4p#aPttXX%Fr))d`
zNfw&T-x5K|akUGu^s5#ugaFbCer6`OH(8GnK6qta#?+mL4_qo6idjdxzXud%t5>`A
z_{_L=l2^zxvYd2(iuscttq3s@Q{9QhCB1+PX!^M*u*Ovf3?*J`#MHs`)fEn=HxVcy
zFnx7}@VpqfZ-c2Ez6Qx<J=*85vCrv>9%X>+$9LN<@#>S5=m}(fwW?oZP4$Fti63X`
z|Euf^xf_JdH?F`}Yu3l7No1<3+qLjK!({I^(J(wCvMR~-ajcf%W(&YrO-V?gnK5rZ
z;x!c-Y51<y1?Oe4B}jYYFwg|7vnfi^9*mXat<G3^ah*{D*f$JOe`SA9!*|+hZ$vqu
zE;%%6B9IL^X80ZpS2}n8a@-*avz{)d`_r&eG8S_&Df6s~+W7j6fX=Yyc_oj^3`Qq{
zt%Z@)YIj3yC*%tRFN?Gv0O7FB;fev-q@7Tb_tYTfh~>pqGX#tWYjK51%}XYy5bcyn
z7(5*h4?!3xhBXof&ao@{s<oxzW>P?<0C<oZ=_fWi`8HNz@7fqw$@GNPYbp+D6*SCO
zIwX8_t;@9XUHD98ScB<EI{+Hyk)#24Ibbr-b9XFgj9WyL0I6ybMMfuF-fJsQq^1G_
z^v@>v7dLXH0LJ9JrFz0w&ZU;zZXtunDu_9LYy@U-s&4Y=AcGYkfP0LaMRri}WO1v(
zENeE*Ug6X;3e!>w6nji~XKQc^`L*Nyu4)Z<*yvW#4EDF`kbxE#6%AJ;Wy8!c$S@8n
znTjQ89aG{hY9Es+Anj{9t$n+#PNN)UnR*jUy$?=j7tKI4OB6G46HIZ%Ek;yHqyiej
zq^Lb%HV8@pq!$#4<FT@a2N$E^%P|BYlf0FN$1LO~!9{b2Fn7Z;Od{ZbBmK#K@H9Ls
zCm)gtAh=RA5C=qj-DGMuqljsQnAUoDM04@@fAq7TNJm>H@0K1R!{)a|lnr;n-_oNP
zzJCNi0WXf(-FH~C7E5qrK{y6@7?4Z~`QJGOOsGRKJtrr^I|W6;l1}G0yi}%lM^3~T
z$q+8v=dqIOwF+4COveTI9%PV>=>ubVP|XR;D&dLEg4|}|3s0n&OFV0r<B-B<wuKs<
z1vY$T0v#WNoHe+e;I>c~q-0x{T4Xxf92aGvieA;KZCpO0R<~q#Gqxk=nrV{?Ah8P^
zL%b5>%=wLan4sZ-GP4CJ#n)C~co2`lB7G{OZKiwC@>L(RXJH%+{E+PsfwMEKpO$G)
z@`Nl@@ie*(f@Z&5tfgP>81w$gzs39jEyRe97kV_mm@Z)!=))>0$oa&*08nH=i|9M(
z>->EvUAhlV-GwG=VsdeKOB~r?fA0{7r{l<dsc)Nf?Tab01_4gdI@vGM3pZmv^H2||
zq3)9@pVMhSISt#%k{kd57l-`jm5o~1;Y*5lHW)=*qS(lL_{&BN_OhJO@t5a3!QLdK
z$^FGz5xSFfG`<{HUlS)VY`>CcP{QzP%z{ythctZS6eRr21i+h@x8Hp4lmx>SSo=48
zWk^e}rWQnxEP#ocrG6M?prWrR3<LuxkLlzZx_Z~RX6hiWCxUI)2)DQ*98D&wVyHm!
zvCrBalhk_*AwoD)MxPRA$t+8POib<w60)ourDMZpr-AL2SM(3>BC;QyI=^|&9K?oE
zz!z0o%Bd~Juz}?wWovWnejByV8-+F!J<VlL9K<&TeRVjv+LvQGW0d|oD4jE=XF~ce
z&bvaS66%}G%pg1%!|W#MUbr^I3l{riMD+#e)Rc{ExGqzu11z3SL>n^mY`p74B*+Ly
z9<`6u<NA8M^JW&Ids-?It7pY<=mDyyT=s=BZP+HFdwwx|R8H!+aypLE@<^<h5KxiW
z@3PivoI_NC`Bri${bR;87)(?J)}UGF0MnCLcKD0fy;2du*V0dnAK)GmK^P)@3;Q6O
zl$fAuVJ$VZKx{D;GWxfcI*mj&zRd_+4Uaej6BE!3RDlYuF08{6g+EMS+~A~{3B+`w
z$JnQ`$3Q2a17WgJ5Q_CdL6UenZaJIe(^Jr30RKN<seT?ZlzbuwECj@vhDZYJh9@F=
zrzY&auw8vuxhQlRJpn~nvZtA;<}?MMxm!Xb%4{`Dveigdt{EL-RaY_BYFMk{9;toS
zsXf$%$D6P$eR*1w$wj#W+yS{q<{##<At)X-jYxKsCr}W@T$UMm3c2pE%sj^M*ex>v
zD7VaHHWY@#t_ZVtwErSGSDc5v&&H?4<2(t~eo|iw6{4*jwhmZtZkEtx%!_3U8<uaS
zh)9XJCti<@LcAVF9dYKGrj9zfouob)vzbeb*-0ZQtWU#P29I{^KHv7nO_LFufQTdT
z_3E>JsOm@CQg3}C0*Hrwa4v4*1M^G7UG(3*lWoWRuAEsA*YL2Dv`Q6tzpdH+MQWnA
z!jj5l>=wZ}Z4t9Z8dd{oMeEb`gK$C{PGK+&-|CQ;8vvX&55CgdkFKCD5G?$wE5<x}
z3yN_md<m5&;aY@8ymwP94%8qvl~_dLNr><OhWaTtQ<O+#ZODUfH)LN9RZ8cicgPTB
zf)CVVL<sjvFe|mAv1gpB^tbye9%$8y7n$VSfRhx{^|No|c4uLOI{HU~`LbGzGo~$$
z#4TvH6LAXxEv(d>4QOu$U;;i``wA=M>*?eug~CH~)0pofLu6N|mYR&|ZoZS*R4jpx
zx{fUfO77A^*wl<$>Y4%s41EUI+zUYo+v>3~#gp(MscS=nnX)=5hP!5DYO-!jRZ7}$
z9QDZK24Ul)=%OMpQXIK3_KXh=k2+Ot!I+YUyCm3|;8@f2lS#%E35P~*Xy_rO{@xo*
zp$4jsXkMAp`3DJs&8yZEYd1&x9Wa$^K=rr-)3c^#oA$fK4NUe@JcGZs%0+nT#cS0{
z_$G)#lP@=ka){>xaW!^A+|MQ#xOg!=6-UOHr+hePXWdX?v-g|CHdmn5PZN*Ce2UR0
zpyF)9<R)h^5+aHS9?YD{9Z9+;oBO8GI1`w)0<ul9PALBIQu--jS5cRlu&eDIG5k0a
zc3)<9Kg@;QkG2;D)_X9Cot2E-jHx$hU8Lb3@pqg;Z?=Y0yUD|(kW2ibbxB73gk8FG
z)Vxu?d$AM-j^Q(XG__1HrIIX#j3;+qjEQe)^0z{K^oxf`YAX@pQ|`NJXvNXX5*(ix
zgDHWW>qwoBnc)FTc4g6x0-pOhN=zEeVkff(c-f<;DLlbX;bFE~DKZKS8+-E_wy6s&
z-EAyTfV`Eyv5aICUuHmXf`?ly7M;L_@eU9IWe|(%uFT~Bar0u+S|S=0#R;^9LQ#cc
z3`pfMJPAfezn)J+IjIO=Ss`odo9SvL>{Q9Kh;0nePB^@x)<4aZB;mchFOmF*%R_WN
ztkD&U^K>8>D`7{4B3@0pE=wg(P|l?gpKGj6l5{S^Xju!016yJb;+obwtJtlcSUN2V
zB@izPN<m1}DI6wcOgbVLed)fqkzd>aC~+!Vl&@Nj##Imr&4cPj5;D0^Pw=kr#FSyk
z%huR8AoUjQmOv5;qFkz|v^_*iE2*W7b<WJ6YgJ_7Dmd~Y?|Oo4HN`Lm!{s@Y2?5Fa
zVeWLM^xk^$;@DS}`P&*GLXt1#m(q^szRg^eYo}@b5m9<K#_1c)sE(Zo?V?oM$Ds4B
zQmgMm!&Qyss*xM{45*se?<?VLtggTe|65XD=D|>S&yfXt@R_83z)HN=9`uKo!dv>p
zp1~o3nV0>9KyX<l&iaeUWcK9U`IFD)Pp-2k$RId~((pztYQkbyyBqBXvRs$v7G>}a
zuVZ&nGHWoxoMl$cjNmhRn7u%Tq#v%f^#QbtlIMI(Ad*PHPVIW=gQiQ0k|0-MKZe9&
zK9)5DSikJD=0LjN==RTO1$d}lW**C`fgTow@z}bl1@Gwq=q{+pWVg})0Q!$2+;x5>
zld?z|V%iC~o3es>5&fAq^jKCC-zHzi;t&Rq+qLWvB%ZW{0jtV**dS{3thWvZAr?Vc
zVe}Su2KdCPrI(oReo0}Zxp0Mug}|y^0g`IGXdOjuoRx)CRbdmnUO(@W);e|?N>vqf
zIkAR$!{%IGVN#GIi>u^g#qJl=`)ATEo)<=ub9f`fE_C1;G&uu%I68!x5C%+j7ne~O
z8@83CQD7ckZ-mE2GkmkQv|#Tprsoxi?F%14sEGqL?e7k+Zcl{?C8BKVAHQ`9)5Whb
zQq2^V-i>O)MuLK@pLMI?p9w^=av$k}D>w~zaZJP|VRw%j)6QvOrmBtXM>6)@K#|y)
z^LpynCd&z2HBBlkh8M_?jy1L?`~=SxFW0Bxg<UE$oQ9uXnk-PNFI>%7kg3KSapevb
zH{PvZY51sP5!3>S_+8ZMr<E6^3)b=9Svw5y%C6>ex?qcd=wWudyK>KFZA_$FQAM=}
zToSG%lz!+1J_oh13jf}8gLR4n$Yc$fa^zYCDOT-)n2pBDJr%JF(3qEXF7jAC89?W7
zDkt#{$LLkVq!6S<E{#svGpC!2>TeqU-5||0ya>n)QQH;?Qv>2tn*6@FYI7WD#JPRq
zm#d~Fxv!dwgX3{%T=H^d3(DW=GpAFWrv?fl(N{sLxgIhd5d)5qcg>u*@X+<%oe{Za
z8oXN)3y#}1YoqHnOqU~;B!GB<9F`xWYd+Od5+zvW$WE(n%EhSKj(0YC$+B<+o4uP8
z&~%UHgFu?>d!N90!*D@f%4_wQPl%VfJg-3=1YAJrCG_c35b!%4*sguiV4-6vL<zLA
z4hk(P3^M#N!*)@(TO#v&|GUWi3F}0WI7XXykog9=VKsnZwKyLB0naAPV|+GlakGPj
z&lUoUavzVtF}-?m88BI*4zOB6^N?y7Mvf<r9ri~MBYbHr4)J$1|2xgKWMZL;?E1ZC
z*Oy^AvnN1V`%vgA3NYAivS>`dl7V$DVy*zF*ILg9@vWONmNh1MBJ6ba3ZN@kW01a$
zY!b??B1-&F9K2f`n2SRj`DWS7*jzZZ8c7w#!e{78EyQCd!jkEf=>h7IbS}(m8Zw=^
zS@{XhG|U4oG`))57Dif5f^SOlV)e`7ZmNwa3PEofwq?lH!afMY%5a9rJ!mQ9p=Wn8
z9>!;TmiPIGr*76%;%4x9%D2vXRuxCR-(JE7dQk6WfvL<0?I->c>e&YT_kgGwX8wPH
zdd~;-*nb~xYN6h9I#6#VwDab!1?)Ga@MbI77rv5}T!(=9Ms?T7e3l;I-yywv@rKq5
zYF`_^W>k8~Jb<VK9b`7fUCO~@7&E2tN$644X{9_e@?o>j&dDKm7(N9TiSUYp4FpUu
zk(kS-1a3?_Fz7c*ERj%Rs{=N#ycu6X#8?OVcaQMmFHuuf9~N6^dfjCRm^C!R*$r`U
z*3?^JrtC4K6k3G^m}aU<(4qi~7HZy3n&_B0eETqyI2YT<m!W2Hz@Yq9+i@c)S(8ZL
z-vlb<XdCwdR!Q9E3_?9}m{@#RP8LPkW^5xX#=DxaYJwy$*`G-jb;UQy^ijf&+mS{I
ztC=XDbWPclW|gbsE4~X7!DxSM=zWX5%bC7f4hK4TadXiWY}`O1GU5}qBcy>vEWyeX
z?_gF@-W7Ngej+2gqOk6vUuLrm>%Rd3?vDOFBljqYFJCGVZraSV9s=AauABEl5dw<7
zK<|er$(W5jP+A1nNQHQ#p_a}g6rb6LTcr}d!}=T~j{g|@53B4Rg+?8+7dKvVlo!=-
zEiW{Rj5Jo|j0s=2$ao3fd`rv(8=+L$jgt2mm387Z_OLH}>k>@IipJmpy}>>?PUP7o
z8j6BPH6;=;Wh{~~@d5^5Cd*9B(<l%=iV{BLg<aBk&bRRlnuN0j2*5OG8NG0t4WW$K
z?ody9SVI`zQRMt@xq%#pv%;vqWH%FdC$D0GX$?|35ih8SE96(wK4}}b$Mtm2gycZ@
z%F3|P-e`fUc$B)8qh*p_aL^(m@4dvHPezJ-s_m^V#2kv_F12rKJrIsm;C`7rBKtUE
zZ%s-v!AIvW$QrAS8N6^-J&+zso9vLODPm-^KP#CjUZE@EBzbG+QWen$0%6Jd=PD#M
zWh_(thDzokiaSj_lqQsfd$?dPvuSU{2Sec60**MsA5ACXaV3uL6+mpzW*iwbm0FIu
zd{dXx@cB&a>{u+Bup<Mzl-jnJVhg-i2ybC7UqTAYME?%6dC8_jfWRV$N|l-9dZ9J4
zFXPJ&R}iMt(o0ZCCYVmp1T(K~==peCFNp_O-vTbC+g{1Yu4RjOqg)A+$g2=kWNpHh
zOd;jswl;*Uhn0MC1&x}ct}Nt|OS??#(4aEdkf7fe$lLS*8P1fki2{9b;&QXCNBYFn
zK<x>n{4;8TjM7LLV#K}SX-5w7W<)Rc^8kYL)FUl8VtUSVN5LtcD>i1Ff?rRGW32op
zI2V*Ai@CG0xiS8>hjuDMgS(-~E-#wUC?dnI7wTW1@oGZ1=sa8>aY&;xpYz31<JP1M
zL=#>ma+!Hy2UBNv-)%z$+`<TpUa_n>|BPbR>g`tR?F(-2Osu54KV>EitJzDW{ka#R
z4L=r_gh3c6{oP5Lh1h?VW+mCk)nqiaH6<@@PYJ{93qL)gBACTGK>ym4R0<FMI+beG
zVMNHW>goTTvHUI@OT1?rL#}9dr!KIEUlW-ucJ-qpqO!f`U~LEZn3@2+E*k2oSY^KU
z*_>fuh0vlC+%^)Ffhxu~CYTt!^6Ew9b4rGA2q`emS1fjthH=#n-WpKqu;R!gp#Z?D
zdHoa&CuLXi551=KVm*I=Vqg!NJcyKNE@@cwQiPNhE$<qgc&5WzE2v*<*a?kn3{HTy
zI%(emVSh42I}P^$3P$~z+sZTCAkm<Z@NVijY9$dsY6$elO<Pd9?lTRTUc6zZHE(CT
zlir40i$$`A$8PXDwK$9zciOM-a{v*blG5e!8A!+kvLDmT$5*Ip+KDl35_=R+k{@1N
zX`NW$eoRAQHT(L)K}E%{2J+z`0>#C#C^NfF?@Dab1PMg~j`ZDE1RoZZI36?aTnOJK
zR0fMWyB2vHUZik#Iz}EWYWy4URJsafMgJ^qmVdX?7DX8XNxxPl`^D4iiVw#Ed?)z=
z4G1gYMWqxs!vZtN_iJfP--Gm72NYeD&g!ql7}+0!IP5VD(|(~a**&VVE2{#SL*8eZ
zB$DBBVRaRV(IrHpYcnERLmVD4R-X>0XD}FNHTj;`IWtnUvBmBM3Q1US7Ev{Zh#Kh_
ztq2I)%ooq{UxKDVj~b*2B$2@bVMghju*@r9noe1^G{NtH8DZ4-GE*};=2~3?ITZ^B
z)Cqx1R`rIn90~B4oH4G#h0S?9%vZfL$Z0a6uzqE$?OhZ5N3^wm4H)6`-is-uFZV0C
zebC43D<PTJCSYcRU?Q(ybNEbYuI`v)w?G1~jD)7lAq~2*!Gk-16G}oELk?q*pf*cH
zsF5-e7ENuk3?8K6ZI(F0W@6-W>#Q%1e0?==t84$!@~rzNannQHn_AetZfowG5`8Oh
z0AzXL(0^&wB&rM#6jdOCYI^fLbRv<WdON9DMKSisR5CjPXxeATzlB8f3QUO5ad)mG
zBjJ;e$*N|w$Qo(*Q*7Zz%c~Ge*><${6ZTUn(6k|7*aGZa_>kU=5J1jsuv=^pKlAPJ
zw~rQIrHCi&x=bq)kdxz83k!vs$5dQ09RAjn49zk-;jM?wI2j;+Nq9WV{$eFk5n_RJ
zC;J7L$kGOUhE-fMJ^^7GZqD7oN@wyOsZ!Ra`2K1bcW01ZvB}wxd%>KV)o&pcj*=~D
zFvdj!hiG@~ETkLR@#M#0I?knIEC{y&Dglhvy=KeQJJf~&gg^i)62Y+b<9|FG6nmTX
zh;MrIt19OPNO)X{-9(JqtKIEybVbrGdu+9UZgw+!p}HXXLtPN+0ktBoQFR$H`+@nz
zgYpZ*N5n>_VJDC{s`57c^-jOV8*Wc0_4k1OAjq{V98S-r(y#}LLZhvbmmS})xxE4e
zTaChhAV3%lV>q4Zq&l*WEg%t*zX&QYWi+%<fGxa)SYQbQKU%@QR=X#D&>tjm>Of9p
zwNT`G84jNj8O9?B5-KNb9H`d#Fb!W`VXLMH?3z@-lQ+=goM1vM8I}`CxutU=IXTAu
z8SWa`U$N{NksLjaNV>N^re}#9lJH<A4!L3nlxp$R@BtXw5aIWRg$ViKgRr(!CwTU;
z?3n~`o(+YMXAd$<mc@Hv+(JlKcNM4axtw>(wuT>?s3)hS@58wu^26%;7tv9@Wk1f<
z3VVVbXfV8LfHdKu#0DzkiBJgEfeomLwr!-8D1|D6teG|4`CLDdKnC__UWTTCofM>?
zrr!cfjQpY?nahgbISMA>4}v)%Gum~<2hh_$QFxeo2NitPu*qf%!^KPXF#+lJnQh;(
z&!89j>KA<F^}Cf?nAruDRRXrI)Y9%?+0@P54Qsoo+BQonTF|<wt@+yFRtmU)K`)?K
zBg)l=^8&-Fep)g4agDG}!7x2w4R(t}zH>O6G1{uRY@5UmgCe3%kb`X1*@847dcL}B
z1oP*z#tA)dOF6@(9iES31pHv&WC#%^=24wKVIIIk{wrn>N7pQpfMVZok^zRsM#7lw
z{Rn@>bGy}hn*J+Q)L_Sh*bshU>(ANxMLrtS_BA~s2dhe_;m%U=N@0h*Z>qPZ&1wiv
z+m#uFnv{r#?NZuV;(OM?Cu;#}L;)5aIef)6glNt-9+@?ITR>nx7F3paHL;9dScd9_
zzbhyy8RmKi$zOFI&(Yt6H_L5+36S)28H|<LjE2Jxh^<I0%b8tcQu7To;@V~sKsqJN
z`N4=WNbNQdxs**KrL1pINJWT*aM^kh7pyqTXINYLsutcR5(lkE2}ahmq5qq0S*V4#
zv|nYLH86!*Q-5!7GMLRe%({`VHPm@4VIH}>H(r8uOaTU99Q8dcZ}8JiMT(7^Gn9O*
ze)}dBo+e)CiXkrfr-}l)z>trQVD~?=Fo#m?GMdU-_+RP(W5MR54r2{_5gi?3LI3~$
zUTD^CpABN_-}jF1A5;2DwqJJP-Lo89#}5ymA)PmHY3abBRR>#GafWAuq>U13ojo1H
zk9_+FsCzBnmB@_+Fhn#7`q2GN5aPrFKy+;k#Wt77P=`0BO0XG@5Q;-<O7~)JRc(Lx
zF4W{iz?Z@+XhNy}y1-@&g%M-#GF0D|hhC=P!`9V~IVNSh)V?~D6Q!X(u3UrlQKd*J
zD$J^?NYq7Zmi^5qwmKZJq#y$gyD8m@Avc{^3;>XJ%a*RTRycPdERN852KQn^JFPe<
zfP;rRO7hkx+N7#0TWny>32W^^T5F!EEqnuz(r_oyAa2XBQPWfmU&l_c8Vr5CvA-=m
ztCDzIALZM9$hv*v+Jw|(lB%yw-^J!@liC>BC$lYt@AIl`)xrlvLjTED2*4H<{#=$|
z1(yak{U%iZF|jELuoV6c6J0aB4XN$K>m@OXciW54@;%V{k!?z+&I~W+)<m<_PBI>r
zn=s5t#{%60onK;@h6^YI?XQiNU1_xJ*En%X;mS*W`5%+w%<`w=a;#w50LACPpV9?!
z4z!r;_&cX-bB%kmi{7eyuqVNPdU2K}^j}-(|4!@F=MDYeNwcN!d7(eU`|T||A>jSd
z`#&cOp{jb?k?Z3iVPAC+APrAvOhP7XASVt$HX#Pkv;nmLlAPhV0qno60jyK21SA+s
z6TK8@vu5eGl9W2>w0R*6s&a1u(3#^$eBcUApzPTcJ&{F<C9aNh+KOVV@nejyiB7Yc
zIsF2yjfcm}LS7LE-X7jTasXb9hmQb^qKrw%%dC_^OZ{P9)hc_wGU{Y@D5Jpo2$|<x
z!*a<4L<bmwxVI4bf#O7nCAKi;1d!tv6SeP?9M-UJkgARElRexJ>ox@ujpQdxE@}+c
zW}>7ZIHaGfHyHuOREPuSU7+&`%T2%)k<N2iP+bTTE*6onC2C{|Xm4Bgk@Z&TW!)T#
zMcTq=Fi5*${Ec0TQy9chhSd%uh4v4OF=m<+SI#Y#PtBIjZH`cO@IN7HF$gmhBIj`W
zgoqfSsWCm>;3l?j0;4pgL?qh(q%KA(p9RJv%kJgaqD(r_kMYbvhPBKXh-_pr^d6*k
z=p<Jn6kJWRpOq^lo!T3mV>j|yG&@RvLy2?{`9~ljvjY|p^Pd{wuxKlQ=_S~P3e!W?
zr0ypH$ciO`*N3i)y>N{B8gYT?tM1^E06nXCovbxO>mmt=73T+DX%At~;W)S}R<!X!
zR_Jw%i7m8~o0R}k)kxD6D|LYHsuF27!`23m8r>9U1dh4bFZM#fd74a)s}J%hOF5dN
zZ8<!%2JP3KO%0>t>j*-HxCoU3{v<IEH?LrBGLCgZ`-+dTtQ_i^QowASh$J2$6R@`>
zLsV^tpfkWG<Fqaeej037W-cOd+$w%N_Pp%BgFQcRLESA)tTY9iZNbP_o?S@aN`|n;
z1Z^^i7=F1z;R?{a!`LJ1b<CtYi3&xp+{T4jBElzDuorKE-O!D6g8j<Ax+tTG!No2t
zLHjUgyI-ZV7)i}$0kTP<6(%EEPud<RYmSW<_6b?P0DTTmO~H0@yf|j~l*-!2Toa8X
ztFUNBE%v)Yi;LMKrMJqJDE}tWR$Pt9)g{GeD^!SYg-{FY;@Z(TJY^J)zF3N!yj>xU
z*7jYXU2Gr$$&m4TJmCq?*<GaQE-+7u*(CZ6CQjWxNI@mlxYmUh!#!1lz9qeM47(7s
zZ-c?i$`dKCOoj&5Pw2mBq?Oz~5&KCMl$F$Y@vEt1resZHf8iM^k|r=R>ZRo5EWBA1
z%|&Sh$m=AM@J159d$~RXBT$^)b+$H*IQE8{?O}I#p6I_(%U@`wEOi3$s&jGnu<SK#
z^3dOAT)m*{m_9&b>!fMF%4Ct-q-i3dS0H`>pT#8U+SH$g^1=P2-+2T=+qlJ0-geoN
z+%jqS&lQep>7r3D9%Z2&hHc~X>KF)4SdODv8xyC5aFBPRw|NNQEI>*kPd@VsH5%I-
ztI!0kO(5=AH;HT+{s)sB<OF4}12xoIX1+dU2B7DY!U8rI)Z+&2RT>es6yC9QiNI`5
z^TMYKk(q47Gnh75tE{$4SD81TBVIAweaWc$7$Lm2vx@R#eerH8T%0)`8T*Bdku>KX
zn-qQy{MeC9KL_{gTov%@oLfcA!L(Lq_um;9kH&1+zLaDEVE=`UQSfX;8>25cB@VJ~
z(I=~ZhhnsJg#jMduYS(}kWR*1IIAL(ToCL@u2Do_{=p^kw8ijpGQQUJh`qEzAna*8
z?+vd&o*fL=uuY)XbrVg<ut^gfWuPQhgAJ-Y32(jb_Tr&Do1b<()T#j=8LIzQLCmWJ
zi1P%9qzT_5zX&A=0;W}1fRz&wSZhp_1IlK31iY0@K%2rT^=62tPcKxEIEExV^tkGZ
z%oov4*pq!-e*P&fKLY*?53&`;l^C}6(8Fge*AZzbcmO36<B_nHtl=ICEmux^Tdv%P
zpzI9Jg&o*LlDWdh6ag-FZ=)xaW-qn>X7@%&j8<njDPdzxES(O?fgdl$)4WAlxQhh+
z<RQpIFk_nwQG_xpN?npoO@fuBV-O@oXip}g%5$j{kQ*7o*Ac7`0tr+Uk6!I^iyeNW
zL2C07;kw$LXcj(zr4p$neRruRdSPv+z%-x;Ycs*T@KvORffz#A<ll+{jP`pVp=moW
zvde7Rq~luf8}l@wo1tzL!T1K8&tO1E>q~JxRdbGz{Z(z@XQntGS98SEL7k0sA**xU
z@pW?Mo2o;KU{>arEKMCCf|gaMWW(`IyAokU3)u>;MW`i|G}7=IcC(`ukWWOdqr{i5
zP(5-dy7AZJrHBE&6R!H5akZ^#-gC^;-5Cke5O%eVG{$an&PW;@<6Rr^iKd%3ubVa|
z5<97ZN{sAE0k`e9r%$k|{jAHY;V@O)ouU$G-pYv0)Zk~Mt71h$BQw}4B$g}5KWu<x
zhnM_aSnPRE*JFAQTX(%0s2G?<FEM-~JY`;ordUPcKLq9QLX&<xwU44qTCKS0S=%tI
zk?~tJBAKG!8&OB_CwU{%p}diV7)0P(R;m=%@@v@mnA7EUp%E|$AG)#<oWhtO(xe}d
zxW@iVUa+!Q2QtSY=25NLh*6I~S`zClftA!usHuj=hGa@m(J)k%<S$gzQ#C8^BT+;J
z=)OMx;Q7#!yeo32dw<7UUD>ZL4Dx9yd}oF792FJiD5x9d$YW*Q;-B|QdpC~jma?C)
zom5@c?j=6&?DXq->%`tYhzfwp?vvy&TSbiRexl3F?rFFR8%#C^y6f_Iz9+*6#b&D3
zpb&M)xS@e<kh@4ILpS;Yee}WvD0V(SHz|;KOo>mxAko%)Yb+o)z)8XeDX#YyZn@On
zJ0n-G?p!$)mZ;oXK-8<`ot51GOLAw~B;ht08bcJc&(SG4D*+){e=~w(KMc+kFmyh>
z0<ZKVyLUYc=LpvWyLR`qJ9LnJjVJ4%6oMq|lILX)v_Mytuv=Fj=Br*p1a!S1p0MXC
zE#k^(-e0+fCXVko<$Qq<D6awa3o5;WPiY1NEHpZX2O#;e)p!cvtLb}sPvr_9bh%#+
zY&8B1f^tTBn;9@Af3|_e42Cr<Y#I}P;+H&bxV$0l5}chaSn&JpVGA6X5<bPaB`i9<
zCs9Zn{<^ZH?DDE4Z1vTTwTy9t!Es6$qY+-tHo+I}*#YVR(Y{pP(M8e5WX&-I{X0V)
z7z`1aj~90I@aS>OQHEARpCNP(!F(*Dx>|TB(1aC(G;EFTfRJR9<KSXZ%4+hYy1Ead
zt)wfzB%$fJ6$*0h1X4eUe`zFPhHzGhlW+7CL(6tq8Y?4@AokEjaUOADWOUo5zfoLF
z+?9u%0mQoFs2YH!Gx5wL#f&M#Qbb&pif1LG0{y9Os_IcU?q5idha~@^9q2q<rEgO*
z|0RMzF)E0S5l54ZLHyum8pa;43_pE)_<J=`K9OW?2crqFkT$OrC)O?Kp?<+Ey9IOY
zciDW=QdBJ|x}^Adp{>)QEM$|CNvaH<`6^WqOahORCoUPc8?%b|bH*?%wxBT4;%Z|u
zFSX=vx&*u~FGK>BG(6(nrX{D@SlK|-N~1gE&bC-O%XSrmEMjm%8fHR+A^(3KonyCj
zNXB(J*wrUQQF}aTj>0gWKg`Z;%8F!nR?=OHl+3EjDzihRuz#7KoF1H}J#I4Pl1h!G
z#Bu<vnn3pVGU|Uodw}mK&om`Onp!1PfS3)r2v+G^rN7||qBsmR4GUZBE06>Lw-tK=
z5lm!Df?D!G0Ta@}Ee^PSHT{sx@jWQY>M~BJ^UwrZ2owi-Rl@xRK;Kz8g5Q~~SQBy^
zjf}_oe69i5f<@>Ssx6~Q;4j;d(5Jp|fAPOrf3#kY2EzkX3b9U>$!r5)bc@wTGiFF=
z3MCokgK|pYrwEY?K-3X?rG3X~RIawPza#U{f`5wB!HEl|16xf_4a2!gI-_Ixdc!&Z
zLn>9V*y^C^KkbTejZ+VF^x;lNBWX31$D~EEymF685^I%6zTv9Ksny9HWs@jS{E>_l
zIT5NZ->?p57M;p}ryi8e!hp|0F0&0l-xT-;YZr&a0!1jEaQH;E+^1<l;p7zy7&1VU
zYEJMG)n8x`0=80EwAXzn@3hsJa!?F-iyE!Jm`e4N(RhfLy|VCM^MX^|zNYPAW8<?m
zvy?Z3Z}H^K#NA=$puL0TC9*Gu2Q~$9pi-lYy&O*o*4(YRz)cb6rpfZYb@sNn$7TsV
zEc*(BlD-QUaORwSDMjs8zSS`H=uew>Axy)BZM4{a7}#TdU`ocCLs0ed2C>Lz+%eHW
z)G39qDwD8E%a6I{%#K!M{ZMZ8Xa!ZlDkR+4-fLZDG>XJ+x0gnplpLLeIBj(@#%>zM
z80^TQu+uIVjPnp>PIy>OquA9bBBQY~njZF`J2^5(Lo1KJQJ-@mS4NPAM2>oha`K|G
zg^9E|jXF;hZxsQv8O=HmlGSAMeA%-945Ulx?tMyK2Qwu_N*4Q}z`A2%W=D;_)#BtP
z?L3)etpXvF3xRMIvEW(3d4QDzkt{;3tdwX*?zqYcU-sry-CuwpBkPl0m9oj9+oZf3
zjeYqNwYxaYF73YX_fJv(lkl+u#dfF70iIUFbx#3DSh$6u@XrN9sdQ;OxQm00RF~m&
z9Yc3<9f4N_Mh7#Z>V?yJV2`OuXb1H%I{o1l&8s{X=lz&lBq2z!H$=7o56X7kMSncy
zGI?t|%Q!MBe5C`oO`7rWF_Xruxwdq6i^Zk)Eio?ab(F#W&2w=_LlRD!(=9rzJ#FZ(
z69^SKX_d+>4?D`ayqB>LvnGaGO<1K_6E1p$)Jhu460&QJvcC0e;;73Uik=iH#B71;
ztlyo|UF4Fg+D1p55@(>;s3+;U`fJl=f3oi3k(W7XlO?V5f}M0ny-<9tz0o?$NagXP
zvNKWIRr>^}Fx|9o>>VBFCP3qDys9EdvF8_k|16c!EIXiBlH7On*ZoPHpQy(~DeEWQ
z-Y(-W5cjh0>mA=)gBk{C*KsoX<oPm{U=<|?FCLsWqamgom}KF<ChAFmowW^Z&y}6Q
zjL!?K4aoT~S%n$C(tFP85%U4#&qqe(H4Mm14a%jq^;{cJAe4-b6R_B=ADS{8lfu4i
zZTwkQhR!yghGRBsBNop_aygfp4)b9$AWnxs{sq@W%i(|s_#g{5Q7D5Ye}!Uc2iGQY
zb)%Wz*Y|VfciKSKeg~@(@88Ko%kiPEBh*{Su(M&ra2*Pv7d?{pEt4HP4+T9WJ$^kl
z!^)j4KCTZ?z8)(xB&KLgmh%|Atf<V>Wl}Au&p1OzU^kdqVR4KIPE?%X(S*_jvXy{M
zjNvglO{I@ud)-p_P||2Lz$f)m(8kn>df?WQEGn2ZC<8iY?I+lDX+)B<!Zl1c*n*Kq
zU@bNj!(CN6q(#Me1MDS!I1fRRiI@|nwx`hZ@DfrcXr*DqFt3j>eI*iGkR%iS1ZmU)
z#6_}oN8C=GLYzix?zC8oY$Y-phTY5jFx;U<HPQnhQVf3x2i;DhT~+KZ?FSJtN{q&{
zI1*z6_8wAki542>grl-I>vB+xL>Cs0WRGlt*)#j&>+P_n+sY}BiE;fd9P<=exG2i$
z-t#3wugC39Pq8=`n>;wmR#t8+qR(@2Gna_F1BBD+l1wRm$h2Gi=~=`I>fQN*l3*Bq
z1V|*l4dJ-i&Nf)~3IdjfrP<~zx`>Q{*_V}xPjhF=2~5jX9jPETOOE^B>c|qit;F7^
zdlC|YJjiCpK@+SJ&6QzVCY&&5f`4xDaazzp&=`4Qlus*;F)E1(bIX=stk@(%qRM0@
zOVWr!n#JDa&7i!}n3e}gO2Q9Vl<GS{OfrGL{8GjAkQ+d-62=R&zzG?G@(~~{&|r8@
z@Da#sB@D|(khdnJLMyeVX055ChP-Mj-S0j9BcH0Lw6yWKCRJX!F7Q>wIG^4PwcT=s
zj8bx%?`;cYO-a$;Sy)gR#8Mfj7R^113kiQUr{|ecreWm1KRxqtWxp5%YEUp90&N)p
zSv5i-HX)dFS<W-))_IB@P+c8TUd!-l0~{A`Hft(%gfHn)x{p4n!afn>somP`{^Dx-
zR(1BZm+Gtq6g+@{nBz$UAaV_|z)@)>{0o*p+jjV>j`NXh0Yc+BLN&EyXCzojKad_J
zIUO@V$ltG&2p<azh69{EsQ-*zJrEVkhKAwKwulO+G_qtN4G#k=xoX44ziI147z>fB
z98%|IHOf?wa*`O_t0;%VxmjwZZydI{IxD@in^;%~nRA3lcXi}UIs_+=`(Wi}DJ0M#
zZ@NX4)jfEIhoZTq5vPSf$G!qyNH#B@@2KY4TTYcUUbioZB|tDd<sc6ll&rE@6bk%F
z%~=-Sd+LisGE?Q+Hianstr6|P^Ur@^^?>;<q()EhIe*T1R<%mb0}{eb%CX$TH!Bxt
zicJ*)++oAQlHxDO6<1z2hi<6AydLNps}WU{ukG`O9{0w0M=iI^z;DD}1-H?#;9lOH
z)))D7D!o5(m|h3Ukc{A~u-mcYgxCmzH0C$PJ3w>Aj*zEqlEDvb-?5P{C&_*h@ZTFz
zWJQV@XLM0q1u;dN=!F;!PaunD1fKzi)fRDs+P3Q7NiMeI`6$(LA!E&>0F*{-_UeMs
z+*e$7tUAjItKi3mksR<nQOW`b#zSb8!{oGTzOi>vxyI2Q#<%f!mxUn|NeN_%H_l#}
z3-ytdU+C?{&2ue7s%@+dI~HhYiElPi1V)B=;%Ve`!X*^mBGzGIB3<hZUz7!iFl6Z?
zip&#blRg5&*~t!k+>RQ&&P_<N1X`A^y**IylO5`P^pwp|0s!>%Z)CK$U~vcxI2fL7
zSR*_)xwSz-DumX9r!!!i5uu(~dU8hnL{^ZdIID_D(s`Y%S(JX8zJ8iw092Hn%CJ?y
zWqwa3;DNuIXvYvurDPL35z!Gmi=03S)Qid?zC>B!QfyzS)5{(`OQ}N}ZdfFO5K7s-
zi)z^ikc8ihdJPFrl@N;6lqJ*kC3wi>flM|eT@`AZB?nasHH9OdMhzE1<oD#&*~t!n
zxw>a(VxU*IsMR^7>XfCdc1c`~jYgAbA`D%M=^fQMo#f<QyE?a&=M;&UpF_^4X0dn?
zYCtk<bNhQWVx=qhYl6bdOulqLbYG~@3~lFEh&@PgAwpLYeyatV4IrMncoJ|t6>!K}
zlrIkvx7@TNk%&c#NF*ARFHh&8XqMR=+)C5GxXq@o_`hk|c`50;)aS7BAVz~FaJNR{
zJ5#j#<a2ASbR4f@4@9^E5)exxJ(C|mL>{G$8W-U>Y5mq{x5=Vx;-Hp;np!UKCAReR
ziefM<_@H=oT)7r)S{60x<llg^8ul@|`-rSV^Y9w_=B70!!&jUV3kbHh_rmua+-!@T
zns3WVj>QYf1uAZ$Q@Aq8jApqmD1=bfeu&Tyk030=0u}b;Owe7W#-w)HqkevjnW<}9
zgX)?Vo^{Q)Mp;k>0J&XVQw|IkuPC`Zc#0^~L|jdV6d<9bmqs5yKxi@whITf6TP4sQ
z262qadu2S0F4zR#fXL-58K!C7%dN$l5k*nKRfMYBcqy?7rsCyrduQcKc(KOJC(BH@
z+Tdp>3l7jT#+>h`W$;=FI&f2QSEwUJiwa-?z-1_|xAK+CFq*{<#@bC>t}%hLCZ92(
zi`t50FywSGn;4UJ3B=H($^3Ylt@p9BWmn>4^*F;Ra13S!(>6ewadUz$7zM_3o@tls
zw7;GvT8xUqR{d$K7Gy=!1NG2;!!Mr#Ky+z<)`bJx9(ffq450oC@OI_Ew@a%S;oF}-
z8=k2k09EggcDW%WYwkFgDCLj<?5|-bx{TcjX3F&Kh~_l!q{@Lu8dQ@51|~kb3P0%t
z3`%?GuHkebgitbZNi}F7&%DPL&k5L+ZIhFPWY|P4{{%K2*ETVHs8iB*0Xdv!!zLJ+
zSfHlwq&>^bTKRa$p?kZ_dVRYD3BlI?WCozTzG8Z(4!j*uSeCF<$jHicJ^2>>%B)-P
z7w|-SPwjdx+3BpyCC|!ygY77W9EhMH%gH^L{Vis4?pW9&vEpncVMR?$!oBd?2E2wz
zxS#U0D^x|=MBJq`?z8t)VshVNmQ@hvrGmKPbqGh7*i3JnAWemayI{uvVuuik5l1oY
z9fsV)qEXK8I?h(!QWith69u9!l#C(uR^M~#mwp^{?q$bDX|HmpyWU%&qq_1nrz>9u
zXdn<p(iSKYy%w{tGSesKlm_6Q8sJ>LeiWY;%Tj)0a#$`G;4?iUQ3}@(tq}@|<exy1
z8XA?f&~jR-V$ah~FkJB-C+50<OoU?Rm<Guo%3SyjvN4KxU%K14RhhjpIs|Vulev*P
z<XaalQ<uffM1PbzYP7*6=a%{_3Jp_9nH@fAaa{C@E>_@Pn`vBRX3_A7UvF3YWTCPH
zJeLqDGL&^kHS!W8q5c{n$X0#0m)$`zEFsDBXx~|6{ie8v*`SLVnKnM$pDfj#6F;7O
zPJQGOsEiy^f~5Wr#J&%3&e44YND?2)$2^Vn%#jRwunOoCE~)CojPxtXgjZN4{Gj*r
znEIyK?jV6#FB!IV9}=uzvof8>G~#&s!9{W|#PAC21$Jh{cM<>0B9(XCXgn>kPDqA7
zWK<8Gf#zig?plg>(2wfWZ~934n;BBRWSO*GJ{!MKOKiNY&1xYdRbZ+qa>692HYPOV
zS_AVFp4F@mn5kpY6X&5;$g3}dYeAx_-J4z7tp5H-WV|3_ivOXn8hAlG(#j(na*yGH
zzqblAgOx!Um!(7NmrqgTO$^U1WYGGBSc>d-w}6)6KJhN$N{~w9GL_fYx!WS4PUfzQ
ztFc)jC4iDJRMcNO#fm?y_1a~Mh|=UbS~UURxX7`^+%ol{&T1xhU-?lD1)qUA70ZqX
z_Vo&$4;NV)*A>oui9=jLw(J<8qV3fvN*W+y(zc?JzMllc+^^%;zlE>lwO}NkdaJ?C
z36Cyq3Al+oAi1mm0otOUhTpMQxa01c<L;bE(<zm}Vxu3RZnxCto_m&~o=X9?_cvRt
zgUZ?atNnJ|?>I6~g^v0i20q=nV=+7xGB6kK$cI4Wj<lM7A4tfjUgIe#MZmrAR|I>w
zTHRW{%E&@mxVYD-UH38GEBfZR+hxmHE&45pbXl#sAT(1y8kDs5tK__T(UZGL{Uzb`
ziPm;5%uSivMjuKr`|M%b48K1Gai2Z4TBG6JCKI>Fuj?+ylV5qrz_?F!Q+alL57apP
zMByx)G};Xn+}U8eAGYJ}r1`z1Fhd}OUTSB8EKjxy30!1>_9slbDHjh{R|lYRE9HNM
zXS%E`WLp|2=%fcEYnc^EA6CGE7cN-fDJPVHl8ycL<a|QdFtDzmDILO2^^f5bf{5AW
zA(`9{f{+Tv5wz4ZW?YH&7ghEh7T`lPJ2&MxiUyWWGEnO#E{$G0Y%`;|LL|t!a$(ro
zsWuaxRQKFVG)fqMk_^#PDg~Mqc9v;vd~XFkWs9e%(GR%pv(#is_#)TERt^p@CkYSN
z2>$0~y8JS4%UO1C)~3ZVg#9$leqIdUD)RjmY@zEl`U87m|8)-!Uv6dxE^bT5DS;-`
zU~A(Rf~B!_<bDm8j0?6bX>#x|S#*R0Mp|ELO|_D%pwR+rQP&b|xff=`rPzPRuL)Oz
zp9#rMxQLkZtvJxUJW)$S(E_owMR|KG9#EK7?#kUstJJPvuX3w?8Cu5LVA8lPJf=I}
z6l(kbn|1;=BCgGAXN0p2yklP%%7#61rk%lyvDwqU!ZywMz6=RW`UAI0I(k5_!Z%mc
zPjX+?RNH$d+uk344+g)1i6(5tswX=Ig#g`KE;~$}79b&6qO_WHjXT`rHLv>kj-BNq
zF52E@I$k@8H7ck~PXez6U`j%h#Z7h&9n!!@xxqD$`OV{USamelE!+PV47<pT<@0QE
zi`YGTciSamSzM`@btiJ#GAx3V8=2d`9;6l#$Nc?nvzK34q->4R{mE5z?B!3DLm=Jt
zz$z9x;_iQO0S*?ckwXcbo;Ius98tmwi)5aP!wB9i3epjcax4~m+IVw6Ru0S>{$)}8
zQ<LJ5#5%3^l%{idV={c5G=Wdmc~cm{Wb_ws2rg!gV&8H)%CRWC5F?~8I510CGgEZd
zQGuA#QZz9X&|BFjB%U~mDj5))<*GXr;7Hsio<G|c2g%i>GKWN~IB52?+VY6xNu!Fk
zBiG0x!o%`G=ACxm&MqTa(zRcgC*vd~r)v^vkjnA&Q9(nyq{bB$?&6OwG$Ii%7ageQ
zuw|W7cR7Af1E^q3JsifHTju7rL6WBwl*o21MvPQ1(r2E9o-9BEF*6y(a6Qd%g=sQ<
zx_QRY!cs0wKo-;Rh{J*{kRYjH9uchlrM+%)katF3yJ_W?NM#r)6Zsa^;_eiQ>^50h
zn~d|s&jB%Z?vyN>%m&gI<G2PgkXfSEGL<`YF0hD&TOo1_wW${J_BD_fq2W)lH0Kr`
z{x-DtYB-zD2w+LTM$1Wh$sEJfk4aM`iA%$er7MX@@L7k7*dJbOWc|kw<@Pn%*X%i|
zB_x#DAuTnF9@Moq786Mb<&WX*7kLso4q34KpSSNBDv5eCowQd>6j^LU1|u$iCcb$d
zGo@~;pltwR--GZ_h%05Y191T}v93Juv{*M@#8)JS)W;k%ZqXI7x!<#?y_%<zwcDR>
zaV?_=r8k!_2OU7!u_!aiBJq?I9-=<VFLCyOEf)wmCA8fv{JZR*w)cTvJC!kh@;a=Z
zWh@p{k*l3$;&hVmF7?k4+nltYrGS`{&VfjR9qw$k2^kH&%&K>#9FS4U8$vUohziE)
zG677=SQ?%I<Yd)q(>Y>9?OwVCo`VswKPgrN+O0ma!|pc07uiT@!Q((hnR()enO_9u
zn680E<(qc>&=N>OF@0cMPrdApd=RF+@QQ?kof6}sZjdmQ;~;^xZ>5Uxvm=LZwOdNl
zOgyJ3uzEg*S<aW3>>d-CKm5+-QM*^5JtAc}TAb&x1`TzUO<H>Gjw7G6+>Q;JZK9$6
zanvpw-|ITZSF;@D=*c=e(VQyv^uc>+AL>N(0+i}X01F;sDKI(i3UM#@l+=&KnQ1<B
z8laA`lpsNvHH5{OH6)tYxn?MO17ImJtQM@w`d_PCv4knhdd-dz7C*BwxWO(4WCReo
zce_eX_5t-0%!qI8pe{5ndTz;=!&a_*3HK?kpAM`?Z8)Gl-IrJ98xy89s`wCdMb1!t
zF^`03tqMeYL@cmV>F#<bG!N5$N~J~#d~f*)Medy<X|LiVBy2t^l+^Z7?b=cr7~K#U
zJgDQ(kp;C4BEKr`RutT**^{trU{u{_iT<QSQ;yk)>0>0Z%UmsoEa)tO*NwI<LMTfZ
z?lHxc8l^BhuJ{ZRiB=>$vy<bEn)rlwQ;`-6sxjPkf1Kx7X5dh5UubRO>ql_~O;F9@
zwDUMFy;PHz*<D7R!hA0Cd3L=rmJkJdcOjA_0XLC&lb|8vO!xyQh9sn%(Eg4;S{jYZ
zxsH$t#qj;so;<7dq<SupXo;|3C9}~o96&eCrLSDnY0U&DNR>{IvDJ*A02hO&lgm_i
z2M%VUuq|GpMz{hWmveYF=^vO~=0AYk;FSzUng$RMK!FgA^f2Bu6&6aEjKeQGOXINi
zo$bwN^wXCqk%S$|VIWKkck}bYWcldifKqpFP2|th7i6A4<!wSPmoUjVk;bcp^q4^I
z(<=a#AadEho?DDjDL;IS+tf#knb9{kloPv=ZOcHH(HoP{^Kj*Mx7@u!0+BlaMA)&G
zi`>@AwkpDgd<u-1aveCLz8?S&kqoSyfRBmFY&?u8cNvL>saJmAn(`B-43vFV=9YxT
zu*PKL+<?PjLI+18NJ^5@GlFS#DPY(UTFtmwDA{zY)|?|~X5+%>t9NzJOolWlKgpTu
zFj%0~H!b@VMm@*8tIe#d2x$(*6XO<c_r*cU5sGCG*K;s{cKNbBEEaFJQDr!5@GqP~
znX9Z!E&N=l&aM+ZM2Wo=!)h^`gYZj6S`>ZNiI3^r!Gp$SpSWZL6cS2|b+yTYj>vVW
zarEd9Z^%1Hl&4+f0t7-8C%)N~;JZP-kiFHS#&Qn?2gD<dv2X#jjC@iAZcB|V;-pY6
z90(3yvRy}$E}*axsmSPbmWbQSk^A%g4<I+{eHLdS`oQAWTjQ;cuKTREH%QtLKhpU;
z%2K35AY=MWlL`lni)84H7OQh+EKQye2~IB&zy}Y5@Xbtm&{`v%5-|{+wW=)_9t2UP
z;QAc-E5vpLovnsn9r5i}&-ad5ziy^qKF1t4Odke>>dwR3T{MdGs};$h2L2qI2D4X)
zxrQrzT8xT_!%Bg-?&~sJ+_wIeZ-u(sG)y`81h5Lu41oR96>JW(wM{KCkCq89xQuPt
zdW?e4=gJ!FP@c;G`l6Enlzu(c`h{xYTuBSI)v?}o$4r^UP>}Vl6RskCn+&imxk2aD
z*b{Bz_l@LC?n@9n5cO>oufcAUlLV)mKBz(Xk|JIi=ucLdrH9o{q>`IpWLIb6v@Yg3
zOO;db#N`a9l?gGk!@~ZQ+#W#(r6&A(1)_t*N!n$YCD!?%n>$)l&_b9t;?RV3LVjO|
z<CaWm%*8MzL_j{9w4i7L?1T^UsAPe5?CA8w$*_54jS|G@#*dD8H*zV+k;(yA+odLD
z<Dr}MNgPk4-_^6cN6&4Md$Jwo9k$u)u>koV?G0?0hlNVqt(@~UJSOgfQ&2~RkBO(>
zN=H2!D#rn}X*hsJPi`4%JCU<8nJ=Up)+JKGu}~<=W~Sar2tr@@48&r9AQfsRhX9)u
zkv+c#wHak7R0!b)7+4Xh#&#L}NGpxGseo@RVzz2JAlRzexeqbh9ST>-&=0DiIs_4+
z#72Vh2H^)>B2N+0W%T9wLhve`z$401;#nk;h&bm>X}EbsLbR<f91-SH_yV^IT_GpA
z#H^eOOE^;#q<v){3ws!>$Ad<x1?_C&K}x&>GK4{7jWk||mLmZQ2rM<>9vr9cpZXvO
zA+r$av$dmxj@5l6VE-eCf{`pXyI0zc;sJa`<whO1V58CZI4R0Nk|FDPMy+W2?=rN{
z@ht->)76WgYD|_9&Q$JqZf(&YIdF6Gp~$0;>BPJtIh`qA57^)1jLDa0NEZ5#AUtU9
zx(?l8Q6G}PPDjyH#INJFtjetla9-rNgZ*rO3cF3c=h$jRhK;ZvbtYCvO|f7*f&}Ib
zibHUUo{JN$APUtasWEKwX4asoAYdoHiD`50qW8E)3{#c8a9g%z$Ix)o&wAM?_K84S
z2%=6tW|v)Hf4HWQGU4B$0CbCJqF&oaKH{gH%KJzf<gfF#MseYrc#<W}sG3nx52WAg
zH<LBhQytFJIa#0>_**6llpnB}U_7Qi4l6~xuqcIs@!7n30~WoL(5X?@jl!qHg$E7W
zamV^O=yoCew+nexx!`XxJeuStBj`N{QpI(u^o=mwQyBIyOoDUoffKqthIb0f*S&R5
z0rrm1Pwn!)oPzy}72PIyRAmV=*aG6!?6y&o+?FH$SU!rhWiuliPxGo?)2>RQh+sn9
zOF%N?MwQ8icKje#k|hZ!;vF;EZ-jX^r^_yP^W}Z~ONj_1X{=y^16fb<<r)SBfZdQG
z3#RvX-g>x(9@=0@yl8^xAW&jCJ`J#E;9#W7%<8QVG)9tF`)FQu!t%Q<!b(fRx2#8*
z$l=@pHmm$bq}5ZPb%niOqD0^mJwqB}NQ!!wqGGZc8PDqb=y{7LJFk!yFSiip$26SR
zWK$V`#URT>+npTqD`B{<EHxuiyNDyL*wGF8+ExH$p%%__{1-v)Kzm!6{dCqIAGdg$
z1#Cm2i5-0(-zQJA!jXIgaeV${1E|(HXpIp(juHGdqtuQLs!|v5!e(L&$9rHiW{U?)
zo%j7307;5C4oIghsE@t|VQtxk)@DW7)CAq9yAdf*0JxYHI^RUBW&xhbX7iIn{nk1y
zoBm(!vt*XM*Ra8U(*MPX>peV<UwX`Ibo5>gI=k(+2tIBsONUA+Zr5ZA24ZYuZwCG4
z6Na!+Fgsn~cf?t{*G>jm`&BAw;f%`Qwl2zRCSUYb3UT2YTkvv0L2pX$#!9lFH<@=V
z<au?6j*cQds3RX$X(qfYd6b)i)X~+ANZ^h!ApCBt#riq2kEbL}<($SE*Ce%LG{lMn
z0~DG`-L95ZNebd>RI^nLtKW$&A`R6N*HBU>GaTiU2ETRev1U6|DSVskvm3GW@@_ry
zD9RLr++^-IqP-Jbp~tWj@t9qm7b;AuBQ66u-Cl^UE|OSHpo5PAc=lI1lEbSqI)OJp
z{vjT(=Oc`q8{E2S)3i(+NzRkya$qp(9U3QIg^j@DHA&Nrhb;uH+HNj0&N>;S91wpa
zY$K;kakuMv9A<3dib9rMx<0>J&{lESKpSgs=X$GU>+P}s@79|+kzayTi6D0x!+3Gd
zw4mAY*df7o)p>&tn4r49d~Ysk%5%yDqUzrg5u)<nXLG85|0(&DsyvM0d+g-Foe*At
z+d&cRo)SQ63)EE@ow?i1yZ;ixra=QZqH69AKRg8q$)dE=&3qwAGH#~7CO2K!eEJ16
zfS^Kbsg5Kp>p;R39MY=#F5xOt%-_w_=Iw%N7BoLlr4H?;FNUvPS|`*kd@-p9g^Y$7
zbl?)4E&LC)eB4{Zxfy`P=+p33&WNfOwo~JkCXAX8fz?`SdN%Z%;5DaH;F-5p;J+v!
z^Zw^-Dl*tu-`;guoxIuJm;P{&re`G6-w6esWXX8htuu|hGh`GOJq_PFp&}4iV|jEv
z+_fG)dwhL7VmYh6sQM6^^4iEn&az0dT+*H0dZ>h~J%ZD>Rar3{6YUBgSpicMg(!!6
zd(Q~u&ERx7-`lczqd|eYL3!aAc~=y5NO;Kx7H!%N7=Z2WL7EI0O*R$x*2_|u)XSP#
z&ew+5kj+T36y6M-Md`SS2}dfs7P2HgBiA<+37Oc@+w5q}37Zsdr*$pK;3UGgXv1_O
zgd^xk8=dj5n?@<S7bC~mFd;`RR8?*fVw=MIea{JWiB1!%^h_DA@{k%!&<?hK5V5En
zK4<|{;w=>A>CdIsLyi$X71c`HUy%3|-h10sB2fGzWkuaf;XSvbd{6i|<=sSzox?N}
z2KhlD5UhmdD~BCVe#`n%5Fx9B`M`*7(cpsU-y5=JP-|_@l0#VXGN(Myz+{hUEgVg|
zKj;7A!5}4OLz5Cj1R4fSv|5BsqYD0j4PbgVZ*zG$v8Id5dk^H1L5s`UuK$iM?Ei^!
zmaxbzv*=<LBcAi$dE!=IYN*HLA7@=w5K}okPG6EAM?41P`6<Jlw6s8gpYx)#+7_#|
zos==|VSD6^d+30NO8Bw^-rl`QFtK;M>gCweB_8V-RPqkkjDD51b^IZys-!pY+y+Yw
z6y0F3cj6Ei@9uK-9Pir=EF9CcO^XKE<Q?ZsW)c)^kSs{|KTd>?-g4eRMkYmH9;Pvx
zc*Bbv-{(*#mIm;kxz8dA2USYYyf+s_I#IHdyA~DxJ09-m6P5{lE+;Wb9RlE5jX=_a
zHe5*B;(vfi98(FC_M+5#o8Kln5{K@J1X7OUeIa2CVdDRO1c(0e2smI|QtSiBTFMy_
zj72(lyYDR_LCCun9R|5bn|)i|->B}r9O@!j+?K&UyF48_B`et@zN~Q$v(ZAQ#-nFi
z?&SNaQ><!D=0=%f$qOZ;*YPv49}lt;ciI$a2Jzh4mN(mvQ;%nc5wZk_N7_{#{{~D!
z*Q1;f$*IdqD93`OX=z@NazPpD+zfN%fuXuF#XeS*2v;H3oQ#0^I(*w#@NHYKwK?(_
zq5~n&a6KDzAC774U`7%$F9L0bJE_oK?*?5`O1Yu%I*gVCBg<hRkK<lIPz??unP{@7
z7PEjfi9^sZ2bju?y$U>>L_i&tJx=MuQj*g$Tt?M|O@TT$vu4N3BDOd^(deJt#OKKH
zH&aS`ykHyLDR;2|+qaWtPjF@8=UOowll9x{q6!Z`sJ(%T&Z}XD1&0@jbN5Uvy60?i
z2)6VLL<N#31;qt+Q9}uP*Mc|qB!aSOZAf0n%?K=r`|OgyI0w}B+M;7W=c|}-W9aKe
zYPZP{bQd+MRd_EhPPPg+33Kmzo2zi$ajS5v(3nmFk6z<vwiX|D%e6Q!fO4*<ZQQ*A
zkU`DlxnV=TcI`HRB)0KIg+C{axKcmbvYmu$5^;&5w9UIA0A)Lzi}?=cIA|dI7cdTb
zR};c^)%Y-mPr@J(m=qZLF>Mt!0g)MH5h`U=zMmlf{SCH-0-XuJF_#NYIGXLw0)YQ`
z3Khr^L8P8XX31#8USsz))F5iksD@`@=~{!~w*h!Gt9Y}@;pb|wR+G2-{<_LA<ADF}
zc3;Dm#a+8I!nn^xh;S_tCAIJ$7yEgw*G8)m)dsXw8@@eN!YgBfzZJEq&(59T>U+&X
zgNTR=`No5oUya&T3^!lY&E_&_h)#`{pn~5TAEH*;FeX|${)__ZPsuqpY+=EKS`MB<
zA|n4V%L+FEHH6P8{BllK%nSjp>GJ-S$w+IXEO~|k#a0;|pTX=jN)yNu0~Y?!d+?xE
zknD8ys0?A}?pZ8CttX`#Rqzo3M8)*!+7KF!daR8En<0`N{}J?Gga1JVoU>ORgW7;4
zq!_+|Ex#|k4m!d{Wb$#CAi`d4k=X5ugi#i7q<F1(k;uzkw#k(>*rg%$@0Hoac@WtO
za$Qaiv-{wl61ec-W$)Y5A-G~(1Fm$;PhTbAC%dIxqKoC6kQk!uWE#9QMfI)BzTZv!
z4j$ri)~tQy^i|obdHpuArAxYy)s|#kd`Es^ZDeT0ARCeTGkTF^NvIvMKGMb6uIm#m
z$7BCk96;g+ZF{jw^|OnS&Jq@7HidN*SPdJnwI2RKnKm@W(y?aTlWG%_0>g;+_FcL+
zYIh>TU`{rKSDk`N-I@~tV(9gBkO(|Yf+3FwJ9%8$Uf%f`u{Flph~00Y9A*W+L(HzR
z<k;6`xkY!3VNF6LUavU&I@`-tqgiyvfSP8ckGkMP6*3U13scI+uwE%BxpI%R2r_E?
z+b_F{NoUd8Lin8Q=K_7-W?Q(g19^3;^#vYMNC^Ni7}V*kCaw~8*_2`FIy3Bn1=fK|
zc)S^Xhm_xUV6QQl$wh6Z!%nK*FT^ZZ{MCrhN;!SHAvZ|wB0upN>j`hhNS23h^`(bQ
zK{z;I>r~r$MEHmIk<L}aA%C9_=kg;Oj^U=<kRa0^KnnRM+L#0DNxS>i-(zCykN{Ct
z)k!4^x7cbAre`BEfPyf@Atz_E0W@~-Y|^YK40@R91vUh2f&vk-Y7`7o+!Rk~M!rLx
zsbwQ(fjpa)n8Pz$UT*G$;=@lRw+Oi`Gsc{h)aEhK8l<>@qN_>%8?2~WCZ%JPmYRN8
zOFFy^!NF9md&C+Bs2T3?9nRQIdJhS-qF7&A0abddD@vk(vij6wc0k?SE?`8(LIluq
z&gKLYm(GO+2m<gbK;(DY6R?`?QGO>B9mdwBB<>#kw&8r0+`wl_tivuMoRGb7c<+}R
zo3V9k*CtkS>g<iHEcyGY$%E9rPaB3=%j%fzFc}tC1Yu~VkuKGg@1PWZaS6*2#{QC!
z_`XGCkaGC0#r6qiSc-KXufo8TOuUx^1+dgkqHb*w9i!_Qh+dhpn#$on?JG!oL0$y1
z4_oT8@GYJAv=f!#by&Hyed!x$i)5J!Z)O5ZjV!+zrw*~w$;t9=ov|kN$V{NAFjM95
zt+)`++^m%CEXXr=WEX2V@16|MXCjpIW{q$G^x(4>VNWc(dcbTE-k1^6oXpyWTRZf2
z@2-8>!)4>Ht2AU)!I3XJA>-GYB4$P88}WL~d4Mm`vblYo9`X&P`RYwaplC*<nRV7Q
zsX}_4!dY=vkQEnOeG$S<Is63r%KnnH{$<t|cS<5dJ?}z~<8uHktq7<oy-6Onyp=J@
zEN~W}yqHrbxEyLwTvk-0rU<G8WAQmP{qdsiCNt8k$rMaRY?qUMP5np<@*SK?=Ie3&
z1yw-fGB7O;>ik<c@Vj~_2J)z6ITl|`f1DkH(28lxY_6w*r^5F97w5dhtSp35DZl8u
zLXVI<JqxPYM8)Mkr1Lut?K(HEVX)PRe}||*lIOrjuz1(!>e^ZoK8E;lD`K6Kbmm4a
zw&c4}0AR;Hq5-GE#rK7zpogC%Z9fmf`@;g(sjm^?tEWH34w!QPLbhXVowP?kwnyXP
zgN3Huvjr7-uoym!$x)wt7?Wel867Xq-v5~Qe(&%3KKA34!LFE{dw0DQ=Hu)tqwGt)
z0r_&J_vwZe&`Uc}kX(9ccnHM<`9XvSN#m$t<zFw+iT-edS0$pyjEF;~0@Dv!E6QJ9
ztPvxCDyU=U#yJ~VpEg*`WF624lGSfA@x)_C+GeN4hDqFRbLwd*{X)0)9Iz2ImtSP?
zE~bx)B&Gbwieq}LY0R_MK&IJ^1#Q0mG8|5z-@JA$nU+YNP8-6eX4+hn&W_shpvW$0
z>68t!nC{~`i|a_2)B13_!M<rf5>~1fdtw*)7fxt1hpx#}-Y=mj43|TPl%XX>SF2Br
zr}f$-P{Zn<(?+n)ddC4C)ab{XiE<O>>>?)v6`^HwQDXj2q#q8C#G1i659mx<cu-nC
z*1<rUvoq+@vi*gNW$`OiY2}ogABGP-pbM}|_0i;f4(=RPGv9cMjJ3>a_=-)ctdMxd
zb;P6{S*SVulhG#1fmB+f1JIh%t~kK4UON}OnulR1vs(VOqZZiohmIu6G<oONw@z#n
z%IW|kytWtCN#8g(fxdCDZw5)vPQ$r}M1F`IXuC=`alsAl&{4cy6713}cyI%;7fiKs
z@KSOopu?^(5y34NS_Mll`73sdW!Q9@&<&+gow;0T!8DZ;?gS>10_mlMIkRY@pg_2I
z1wYC$Su=+orBX67-{>NEYXI4WDwJSGXf+T=lj_r^N`(BrZ9GL})XZpDu7SxF%Sl6Z
z-T*D9qwgh(Rs@pO*Bg5-lqb`-Grdq5Q&a&wJlEHSx_#lBhnsfh*GA4;*Zq4|*4d{b
zIku{;cN=8(0B{eZ^Wv!=zJNi?47|xkA6|H*u2tM;r?hvEY0Nnw@<oWt!7vcU&bZ#o
z&L7QgXQD?$429`T)$7;W4q?pN_eAmuU!XC7szfs%;xl;d1TurjdJBKuvq&&yT7xXG
zd!Q_bWhFcTV-Ky=#0s|4Js49~pt8%}U}QmB)1DN`8#)yLiuGSoF9f|lg?T2#g9xD*
zz#c8A!esa&nYgwemc@EI>JWdI;vz|k`=Fd}k3t6|B42Jr4ioE5FD$^$h*D525pe1#
z$o_}~j+y16s<3?u5O65>ZUJeS;<$)SP~ejB4<99q9E%3Rz^g|{ojhkT-JQ;Enw4cm
zdkImz{p9Z4Fh<W=Rm5k5SnRs$4xh)Gip&+>NdPo>W;?MyBigOy`#dN=7ci$^m?_nF
zVnO)d-p*47v?@qj+CVliNgX(Su;JLQjsU*nBv3ajC`G_%SPMUiU^PM_IC)T!ReASB
zrZ+Q3Nf=l{&<DvL>tW0Q>;gVfCPg>kdXRjZtTW5ioo2}T>=S86hqd<$Y}b{gp(~4e
zfjC*L|H>1lv6+`;z;=4-Jef#nOt&(2DMnKCt1_(SKfc1m?SH^%Qz00eFr0`ZyLb=6
zjVH)GT8|Cxc6HvxEsL`q|K>AOurhFsQeKA7t;$nR?7oE73z1E+J|)UnEtgu2JzQ>F
z9um`>l3@%hxlo9=7)?a$Cg;*kbJOV#1w8<*nC`WNR!_*PAbTF)^`PuaGSe`si3DsT
zvpR27Rbfr4PT<_NjKH5oAch!X5~w5WG#Yb-DME}F!WV!{68=%23>1V^cE;@hAJxJe
z=ne}5fk_fJZ)pc7u)TYI=exSU`$qQeHJ$IW@4lJ6d-d_}gcv$Iw~B|;GK*o&LwZ;>
zz%<FWq@Deq*%hJ-pauq&$v<43<8Q_o5eV2uA$pnn!o`b3`sxRHylbzKX;fL$!`k~L
zoj7tJ*{_dyax?4CjYq_>Si|92IVfw@LbtN@StcN*)4_Ho)+3z7@nklt$7>nDdo?eL
zghIFC=5hJc|3luJ$46aV|Nk?2XC`~d$^=4SCWM_#Agn@wfXoO1hCo~<A*gHtEaC#9
z;*O}eqf&68ZgoNHz65JqYg=3QrB$l6t<{fRZ0+KwTD#cR-}80udlEp~&*%H?<NMc-
zhfdz-eee6;bI(2Z+;h%7_g<zl*;sm|p`nZ=!~_!WS0Tg9FOTie4Ez&<a{VCFuttY*
zxg;(Du9@t4g-{)>w?jSZoTR5xp6n5`jYoz_$*@pemvH1px$s(XSBPB`<&B}xfKYie
z8(kPZBfPpOSxR+x2W`pM9d=@@|6;-{qxsk8T5s^Q)7*wQD|mruWb%{H_kK-=pm)Lk
ziDs9?8HyChVO4$0vzvLk$2yFw*lilfmtpMYHe35`VrG}_9tsoYBHKJKyapw%!b1Kz
zY@2Wd#F~6o+#?<v$YYe_;F^Rg*RW;J1c%q09)jjqIj_93-MHjZ$0ac<nd4LnDcCHc
z5IY>rH=(g5uQu}y!(K0?eUxB@O(dRQa~cLnQSRe@d;Ux%5HiTEFd__bX+ajx^(eiT
z|INa#RO`54w1CtX4J|F)(r~(fIA-g9!h6`-VbObqE@aR9o=#uGGme*+-q2ABY%oS^
zVVAJ=&i6FbEhPMY#_|XX-*Hgk;)Xn;@S}FgNl3?by8@4=0If~>Nmd*u47$=#*t5B^
z&b#m6($=XoPT4KSSMdkTXk9s{Xk+h*9!wSaq=G+_%c^+7VN5TvVYQ*bcJn_kNn8P~
zm>W>Lpwu+?mvcE8!5B|Aj`@;cM0Y|!=IUI|#KVW)qF$d7%%)>z!mQlv0zc+?+`~;_
zeJeGkdq38$VwMBs#ZVxMhIwYQa*UhWbT$XW?-iS%RlN=$$@A`KGOsgbM!s#N&otdg
z?pPUJn7H<!Zf%gH)aZE8jSjD13p>NOs7G}o$73uSP#^A%c({wu*?QhESJ{BL3fo!;
zz9O{&BS~6RCJxY4js<Xv;w1jU_RdmoTR57|4gqjBK6COmWCmzgHTk@M<>p}acuH$l
zVNTpJQ7}{_o+#HH!IhbLaY)U}v}S@7d!3NDMxXL`k1y!D+=?#t;$fk@!FU71eK@Jc
za3?6<q^Qc(`hZ;m<xObGW;?KN52#zM@#c!+`nRTXee0>yO^5k38@>OHV7%S@ak5Kg
zvT$Li-Df<Ec^t>GGXrL3%)K%0vc_q;tdYoz;qlO6gF?e&p@A$&alMD>C5qg0`lTXT
z<;^P^Xz`v<K_^T|qd@d)6~--F^{T0MP1P{cGn;x8Uxm_%7}5u`{SRjK?%rhJ8579K
z!hv<R0Ss<t)z^hh$rVkwLv1)3Y>Kn+6{puUlx$rF$7DH2>!tb2;uR7@TZ*`$CQjZK
z|D}qxIJLpL=eaTzRTXG-W5`--w>_J^FcN`M>t;Rn1CYh`1@Kn~xJle|K!!iT*$|V@
z4<gynd@c~2=h&~A%?pD<b1Wu9MZ_q|)?eg#dc>I8ag@GuJ4bow=;bp1-~jeH5|;wo
zX(6eH1;^|+W#~0p^#U8)oHMPZas)6FE1mRLoI&CR`K1xD5)HP|P#Rybk7H7sX3#!r
zJY8`K-xdb7d`1p;`&}-Mye5Vw#YNSh^tpVhGj#fgxMTE2trwt+QL=u!WO#XRF^rvn
z7|$)t9EWMkS^fgQDdX74TR2Hun+Y7l7CjD+h)}ZewAvrLlGC86xzzgy2W$7vZN$y?
z^#{!uNslFlc3z6d`SwT@v|Di##X{kkkf>-J4`(=3oJ@g+rJ)>^pXYrXh?A7V0i^T?
zE=|4TuvdF$8%^RNqQ62Lp@SB?aF$xZT-?Cv;jWpP?SgbfW~Yt<^#0-6Ei{*b3zweo
zWu7*)4AvIYK02~sur6QtyeH_GC-X#&i1~GnK@Db%pyji7X97L1FfSyBPO9-yVdcih
zDI>lbo$4g<m5D_m-~_EsZ@l+)WIlu0I_@5={{73IMLYQ;Jxjksn_+C8Jp<#A1|gwE
zlA`S_ok<6x)bn6n-N4P78xqxwir*{3nH6|-lIC?s#Wn7nXiX(S-TMFwx`ZBE<+Fzo
zxb}^MHs#^=C1QAKq*_8yibjO!EWa+HfW`}%+CEe0eEXza=X2-;=_<~n0c$q<7@Fl=
zq=$Pbv?YpdO6UK^bF`xDO&KchZ(3>TCp;dB-JmXcgAtYIy}dcg93lMP%hn9+O_6^0
zypsZ&)k5JN2>9Yql=!!Bt;BtXr=;5!#X|i!xQQc}5KGbU^zcSo`omgjrlsCfo9QrI
zbBrAHGbXg}aWx>$#@uI|0$iX>1@Y`zcp+YpVd_?{2Adi@P*<H6dJnVi?R;YGWZ(i@
zL#ih_2kB7m)!C;$GfJf`ushV<=J#H%at*BpC$)5SoFz^8m0hr~u<k|y8A28c(AbC{
z;hmM;Pi1S8Bw?br5@&xVtxVU~g0drsIYbNad}_!uBsA0843z2bcCi9Csk_ZqkTPB&
zuCl46qSB(Bu$fdKES#n`I_1kK2MUA2(DD;>r7sD$`3XPnNoaBjGSk#EdEry!CBd@Q
zs*ea5($TIcNNN?`9iSQJGZ3~u&33|9)5(q<GM!|_6B2Kyvx^`B<>Sj|Eu6R2Y%M_+
z87f*G*Ptj@OanTuoxJBNv9pp6tVBeXy(->9Hsh$0y!XEAIPQNAu@Osrl_`xDp|n?N
z)|()K>rj0<gQ|SN4pA=O_7UZcJN}@8xEcbzYjpxaxm<5UhPuCf?XQLjD_m>{V;zni
zOBDlnfwx1qUvgx<xh6SR64L7OW1V!N8DRwXWW^BN^-7?ZdvMa;ZDf!|D;mKP9NQ(E
z$DL+isugZYe15&ZwITMPvv?|1QL2|FNjXufSe0cBt2$KAnK+c|^BI5DemkS&fT~iy
z2Iu#r9qzI}Dg@8;;K}JZu9q1+d(C*S_MSy&alK5i8|!uN(M=`ZwWd<YcR&O=lF4Wh
znoUQBEhtW&8VYF$vLN78{+RW5$mlM1v;7&u-@6ii?UlQkjWRK{+O-dU8YSbiaSNn2
zeMU^<E)g>?1_KQg3*9QVJNGjZjmOPzj<-MqyWC@)&#}8Y_a--*;lw@Qug-%LHAFOK
z(F%>0-?lG|SoctVoe{;l0@EXo1>#Pa+#jcMJ4EJJXS1IR0;qLArI5^g0HJ$^l)P$;
zsA5j}^(oDLus)@iO25%X<c2Y81#xUYaQzlTeilvH>JlVOdIaSN+W(0g6;SgSzRU#W
zWCZ0{cce@292@qUF{E6BIE%{3#C6C-JDr@K8Y%V)SB3{NAI8}R;U7j8Su$;`m`3uR
z#awsNN5-DuJY9N2p`2TYQyhVs_D_vSNGd62`+rTmku=yl*QR*l8RSmU!k(KK9md1d
ze;gDz)9pIu>J7$}4rbWz*Ch8ern5+>+P2xU|D#HSRcJ>QZGw_s=cm!nnN%JtL;X#S
z?-|A=I$~I({<9<54M3Mh*_BK|v`I^#HWx+1GiGQke4>7lL8NuC{n?(av2KnrofI{v
zm;3)FJIRk{)ZBUq)la)eEf9Z6p*@hl<bT+xYM`?E97{)WMG7z!V{>TMzhukZ+GeOe
zfw#WW%a&npL=%QS{58PM!@vHVd038sojvog(`gT!hd+nCIxW6g)e%8!GwvF=OLE^j
zC{!`Z9xAY!;IDDLf@pvZ4hhUSnDIS6!5%3^e6>anEG84eiFnTVkh6lEt3=LOB9NO<
zUvOhZI$=J{_=wMO2Py_|*^G8~iuC{%vT8yD28SvQIpdB2>I`~Z(JBq~p#d>7@g?M?
zL_fk@#$ZF9OU@9gD6w_Gl*bNViAnm%<+$myIZ(t*lCo~p)6k18n=h=baUWBP%qOeo
zy3dvp_I`mdk?b&su%s|`zlW`$8CI7RgQ)zJXn!fN{QP^x<J$1SHa4{QiA_FPe+rxS
zAf}aFe?s&7=kFi}-$~0e+^+sCZep<Cl1)}wX3qZ*7>xesIbZB|DP<y}ePwmBR#ExV
zv0@Kt-L2wOK*3I_fwP^qjNt2OfnCv@nJ103+D_U-+TVUgHIldN>T{e$vOhCwzrRlk
zpHez{p{JgYKa+g%{>gv(O!8%x4EH!Ae<<>+Iu;r_NWX-D#j%10LSl5X|NhLk+xfQ9
zdVu8Lv&L~CVv0)ZX`u<_gPC-zRZfF^Wx;8P`!k>pqwFceHSPdf>llQfY>Pn{B0=xJ
ztbZjCd)3b0e3mr`N@0&sFV$ZE41Z~s?jh<Ady_Th70v@mP)Id_T4&znj{>>6SUI&(
z&|kU;%On04c6XhD)K=Q-cJ<@Xqj@I4HAoyMf8+De{TAp*zMIcWt1T7Y-Mz|!FHQHU
z@y>6?`l>I`<#pO$pqoFrOSMv=xSUhfs~Wmsbo^*Kkd6z+B((E+qk+ulrR3(a6IFU`
zZ>4_AQ9XJIwX2s<ezLT;9y;#)Ofqg)bsoxqbQWR)6oL30p4A$pp9U`Lz8PO28eAp2
zQW1?#pi2{IcW2Duf6bl=?pK_lh9MXzC**yAXTnXS9OfgHs8G(z{mX4&4MuMBA>kpq
zk1<cbxCd<|lC9Z8a68*1$E;{hdyR6}1-h#j8Gt#@ifWB+wv?0P>injG!>D_hcitww
z33-3Y!{8V)aG0FG^!jf&cmc8HtR=g;QA<F`Dcxf14E>2)SornVfv1TeG4E<(c=Eyv
zt=;sco26q~NmxxK%u0VF@`f9WybE<(5U)6v4hNZ>zco(&85zeg+anW?O2-I9^uEr<
z7f}9IRx|_RS=vGTQjNz7G`MbvvXYWl-Y}-IBGhP;iHkfK8{Q~K@z#xuASYYLhPSYW
zAui!AG|juJhNDh22b!njLL!KRo|stPYZc1@msH|SkA|B?5Eg%IB)BvY+15#Z#bGd%
zf!~}EElB68n?;vlh;e#*-XQA2kBWp){xNnmn=P@p<4pMMbf$Wt;l0hG%QW$i&J$(E
z6<diZ%wOnS407EvKCNo4=JvRpGiL>0Zo1Gov$D0s$97y4L_cssMHPh=Q;{i}=&?q(
z1D~|a1pz&7<cxpM?Zf6Q13Th<a42T<$3XD$b0@-cZ7hq6@@E_SXq$RJwuP$`S!anh
zBJ-k18{!1}DLS=Cd&2uEW~9hFNl#VL8hmUuc)BhAm!?nH0=lQNfB>Doy^#e3qJEg;
zL}l5!Sj-|1XP~(SE#6^cdVQPdlh6?58S7dATA)lsvyW(0h!*K4bPl$WXz}{tLPf-}
zhFESYH<P_C!jti%2-=|c6RCg|C}w1L8`};e2O;_86WIV@5rnH5Czttgtg@W&^0OTh
zf?%E)sjS~txj<Jkux3v2=&JiYWESi*?yrog(z8bnZeo|Z8Y6}VbIt~VPm6;b#mrlA
zs$0&0PpH~`A0(78#J!2~8A30pCKAd^pfv$Gvel8|3JNGH_wWx`nQDRpjddTM`q2&r
zQg^Yqrv+X=87$<vcCG1YR)f0Bj;6|!f9|J_fT1Go)^MBd<k6&5kK37Q#R7f2k}jTD
zz~d?Eo}gawjXl${IaS2XG6TkP6$DaAu5C$Cuo%UHj3{712By~8(bm<jaJEnIgoj-`
zXYVMy=yBf{dO)cNRq>UCig#XCT4m!d&n$ULJPd~dgwnfysT8_-atm|wu7!u_<_U<&
z)<3yE3W?b);%*nj0DlST^xi%ZdNUyWuE<P)oNuGG5UQuf-F*;X?vn}978KmWAW%)k
z@yvNR+Ti5fzX5SHL`MY>bFocbK^QYfE7M|lbTh-lF|r-e)SqtVZFw9qhjJ9EWCKdt
zxLghx0>k+&v8zt8@9AWU+{?N<MdIwZlGrdl-oVNr-c=%#2<^~NQ4JNLA;h8~aa36;
zn;bINLHK{gA44<?JPa)<1UZ-)UxnVQ8$=K&)2zQ7Y#yZlzra!J?2wAvBOfxA)!M=^
z-LZvZ$g#ljqU%&#I{?^1mWGPAy)7Jr*h|!Dy+r+#5oywx{-tr|=s*xbRjWwkW^1mU
zv!#U@l+Ffo-J{9-_n5;P<3g3Pt?M*OVFn3U7vZj<q&6vhABJL9cIyoiylz0+P}AzV
zca2alHxo)TL+No6Xmopn7D{bK*@Ic@2I|<cEY?;bNH*iijFB4hK0zn71>7w->ulD_
zNMriY)^g$QzlL=5Js)0<PrnrX=tH&}p`qwY?~4>>BsbD$R-&1`GqX;{$X6h+n<rnO
z$AyvjG!#A)5|5;yPQRRAyHy;mMzKCcz}sfb_MQbQTV-I)S83jTu5N$rs*Ffltp@jL
zM0^u}*N1sb%WOK;5WT-o4cl0p`tOK19|R5_|156c>(}Ua2Wz8S$fV8VESOr%TsE-u
z8Lq#KfBP#sjHdzM-qU$bu1qkwqIg(sMM0+Iv2%3`*}OkwnG(O(M44OE^Gddm36T{N
zKOg)AlW!sF&~JxS!weVnG=Lpel>As0dPxPV%+~QcsX%OmZU#e?%FD;mB|ML@Q256!
zx}TO}mD2Ovk%fB&wZ;82eT2_m4F~&A;v5CcG}Ag+!Qnr2(gk7==Z7Cd7*>UMv-BRs
zpo=YFv7HwG3>HE;@NPL96T;49{~3BpU#W?d{SFHwwpp7T$D2k)9`wB1xE@F?lsT)C
zckf^FJS&;w6>!>7H0M!cl&G$EacWqxKPQA+2!CYU0k+!5;dadfa6FBydB?NI_<vp0
z4L5jgr^Nml-kl$kQOri&{+hKK%ukG$1W_jyO3g<&@Cy<$FYF0T$^OQ0opEf5nJ4Z=
zCZWPbAL_H1Xzd2ds)4mlp~m*m(2cxLnHQV0zDLFt05irW#gb_)3PGV0E46os0VLrx
z)0ntWp$bHGYgF~+Y|OEtzMKlCYg{B&WmLISUyE5AE5}7A{1wqr>>b~N<I`fL{gw<;
zOg5)jZSQAdO$~6dAEbxb%O`#DYo^6!rh93#)t_x{RpwPKsl*sy&@_hz4q#@J-=Fhi
zDx9B?m3MMhO)^?JVbMT|iawJyAa9sta1EWv310oNUx4H?;%}u*&x-a8&wv}?o%f9!
za=kAabqHY)my~-h@CBweCfq4ynNy<g#t0P6G(Z}NJJ3*XYzalQu;3ky^tJHe&``E(
z4@Z6vff8J2kD29^6plHQ8lC4Y4A!&kHHqnrv(<FSm{4>a9OG3V;5tR(do5tiOiJ-p
zp`r<)2q%&$9*a{F*`8+_ym9cYp%&xd^Y$BZ*iWnD+0webcQ7YeG%;DmWDL?$(KOWk
zzr>@2yoCI`ggR()BCMHR*XUpzKSJeZnQ#JAXrMw<!W?%mWC@|GQ}I7TZ*!p?8J&KH
z0>t?CyLh~~T=bWiPk7>p^3|5Pf1ag?%q=)E_kS)wg`#3;!JH+GhRs>Rl2Bs!3h$${
z5K_nvPnTELH^Ai~yU8pb>SEUiy1@bOI314+Y$jljLa1oIjk7_J*S0A7MiD$!We2oK
z%{B!D<;V7`OmP=|p(Hrnt2UZ8ahOiK<LIPx$il@SF5^-Bm${pYS0Vg{nR%>qh8Z4&
zi9}gY!nUjl$)uFB!fdC}qf4vgSz3%Pq|I?|eR9NAp}~T*gvG)_eJC<4G@uT9SBf6$
zoc1^Bo;~>wGuK{mJ&^X=>5!;VydE)c|03w3g0w&Q8f~i~T|uOS^$7cw)g8>{oad_^
z5UYytXgL{9p+#!i*ig}?4wm_Jk#dFAGu`_!-B{|~eike`T_@*+bn_332)bs9k95!y
z=O5}|rEKQA!S<Iu1^NqF4P;QNcY}jD2HaGUO^gej=|d&LK;=e5qrBZ{n0nTXG|s3{
zp`aIp3{zmHF)?lWhPaCVEEfy-jr>>d|JRLo*KHux-2l;m6{Yt3i@4*4Ftvn}g=q<7
zC&(?s4ayu0PwYXc4yq3jTd?^GRRC)5pHsyuSl&iExQ8o=Wb*zV$}|*7qnjWO_$@!p
zJmndiodNFcVMgP=9YV}s<I}wM(0q3UbXGl-&kA213SEHzGEL5hw2!tDo0o_&8nSgO
z8IXv4c>!m(61o|hxiM`vs~nnz-tXLu=rao&6}TtQg0K|IB@X>Jf3<$=ya@x)>%147
zQsx(B+e_g;id<tEDG4b~iP+MFKvHgbBvXTCnAdP;S$QGv#xsI(?=CbtbcOdeYt4y)
zL|_D)F8Qp1sR4Sb0G>*e=|-qh=>7Tt1SD}nS%i6@7kS^KaPmEYg5gHGc4zcMW^0=q
zijC}dyB@>-x`1(a_#(jyI08b=>?=a=u7{Inn1g~5OWvUObBE@Qn;&<R1MI{&kvBvY
zX%lW~r4;oER+;w)f?^82>rB48>BOf|xXF74cchuUEUWA5Xc)?A!mC6|6lot%#1+D5
zv98HbBup3_hO>B<t*KZaRWWMT@hlE`{SAc`=ibs{r#PC&fxV(orO^A1^D6rNrVC_?
z#)Oi5S#E;M?4}X2n`C=Rl!+zW-v<A$HoV9fR1WuddQVQp)%AXn`O?MN@xGFbeZY<=
znlKX+opgP67g+tqk%&%2m63(>*<NA;n)eFZT?r3_vAhScAHdmXT0PjDdYMuD?>@yP
zp(GVup(&iMe*T`nncxohH^%&8#r&D+snEO9*(g1<f?8f9_W)ZPz1A%z=+s*JLQ>v^
zk2(X_NqdS_EAO|Q*!|?0)>j;*R(jj<4GZro?;Xz5;RV*Kx>YRb_{s4mn{ei#w^U(a
zVx<>N3$H;+UBuXDig!*KLJBKlP8q2HpHM?BK1B-rh|&&j^1&qgnzi+X1KO%_%+AU?
zobe)kGs4k9Y@W+7p*g<Pbdi`2^X_z6&GkUVI%kMp(m6#yKhEOB+*(F&qyx~|#`1cp
z@yt+SRqsz6G0LGbMu*Yj)c-P!+{wCf6c%l=p$HXm7V=B2u*3j_$?B%hrZic~y0{Io
ziy?HO_xA$|B;xh61-j9`e-&<ALGKHLQO#_9)Sh$CiG%SW3}Cwpz<Y$&7WyaMWg-Np
zOaC_0Nug?*CK{w0*~Vpa65;ORBS&I{R}tPHqvdqd5ttptNF&RQvp8^uA)c+Z$-irB
znbs!JQIYd`L2KbFR@kG~3AW@!Rm9VZNZyMmWrf}ce%gT&+g)AU>*H+zg^VYVX%om6
zOl)1;R7|R5E-&=XJzy$9{31x!ST|$_1SbQ?xo_)+^38JfrnPUZFEnh}SR*I#zWPS8
ztksF_;uyl>_?sYVB^FAs$4JeO`QP1SY@-WnlqXXg4^H(d3csGI$i*&a#8k<RJs3^p
z!+u}m6HN<NPlg-qYz!GC`YMAjQh8%2GTknT${;9-W3$oAOp1lxV+W9f-dtqvd;CGe
z7&Xxlw#H~{M1w9FRi!YoaJNZNDiwUpmLom%{@f^#xUfR+YgWFSAdqNQf+U-hmpx8l
zALTK!aiBSuyvworRZIfuo$yo<wpO9{^#0h^GUkHbJ`+^ly}G|UY<Jn>li3dyLn>|f
zu+P?e$x7(qT1>K+PnBa3mX_U|@v(c4vcfdh$jKUCi;vz?3V7W$2(H<b>2y1M0t_>M
zGt#*6f6AZ=BI>fSFt68N5;_|9&Z9nvT7)75d-9JZDcMFG+!PG;`7I%gQUsmzTw754
zK!;cY*Uu2?hb*WvfOQ3GW=FhT=zu=Pt1Mkon}t!PKjB&K9oo1+`=JfemKl_m9l@=`
zNV8(@iy`jWa!7}lJ<yN%PMM}Bgi42)g!I@$$=23E+WxQB<xK=*ilrN2+tMLO;{Sth
z8HJKZC?Sf|(WyiqhDxiphVm9*5Mh@qLMp~V@<tPPl!%TemgLuEPk0MNxPqoFGMCx2
zDVG#IZFYm-SZJHa5h3_pWCR+$rOt4d%z&BpJC2ZVXS9rbGBjW+a>h*`@~JsB+T8=M
z_YwnmVB{5XNtij6wxoNpdBdH9%KVTBLd4>qSQyYpMEZ?&0rr`FEaFDpmkR!4>>@{I
z4NjMn94t-i#trOSs<iD>PGM-5m&~YDx{C0p0VLPaNyT*`Huei!88`)Wrt>qFGK&Fa
z#HJ4D3Ja5_kjJBJ?krkluUdLusYuC};uaZcgp2W9sz7HD0bpK?g_|lD+(gsf_x=Ai
z#iIv=%BRzR-leFB63N6dx{zA4nMFM3qlz&ckx~vvjZ7uE25RO-O?L~>j%s!@bCbK7
z1#gr%6g5_VS~8HVLpa*fbo+1CDUQi0+QtK9)LUwC$3;+sU=TRqwO;}~Yso^)(i=j$
z9VQD{_iUnUWIhuE>a~M8bwIGESPy%{u=X?96|mkP3GK*Z4ZQ}^{gD+wf0b53eLQW!
z8|^p>rwJlJ9m=W4BqFKQbtqKg2NujF8$`VZTb#fpIH0G?#n{TD9u!&STNq?3h04zI
zN0#&5S<fpozQs5ft^Q9bPQX{|sG|Nls>uHaKT;EgRO{UI0iFU)satNW8h;a~>L*~Y
z+V!X;+aQo~kn%<G%yFAN@9+lUS8EEPE!K<_?hI5|&Cpj%aQw`~f59%XP)xLmA-=T)
z$6a=|d`JO$X<xWU^Q*W=F`m0j3=5`P9mEX5J5?hE5EmH{3xj#02{-_<M-uX~Do9sV
zd4KLtoUz<~-WN{^W{(u#v#B#w5vd4wVTCc|nq2ZFERtygwQ)Vs<{VbA?N~hTQ=3)7
z-lqXg2krhe*~kpgP8nJ*FY<nIl8V)YzzTp<(S*IXqc|90&xso;Gw6K9<i~WmpZ}}F
z{|OG*;to}y&QN%EIVVZD7S0&cFlJhcjk6|YjU(sc%bIo(?&`4joT3ix=(H`N0iApa
zQ^Yyeg*a7$8+7*?yG#e--l-0a=0-`>yc3ZN=C<T!rxtlq&S$)n@X7uOQ1S1@U0W=k
z!*R7cUQGI#<ccY4Mc&7z`w?5orFpVmdM9W}0z6NqQ=^VpU)i-zzV51=qj@_k78*S$
zG)p5Z7Mf+*1~0kEC!dvPn_ozrNFs5y(%APaLz}i#mh8o$O0Dxp;j~I3!=;>XyU%Ji
zm9P}_Ll%yN$w21f7+<*{7wH8Ol2V$goro-@4%46Hgp=hdx4n`zglogxOraIfG2zK_
zPjE&IZcU2w?m8uI275DQIVDoVDX%Y6Ov12&Zht!ok2xhC&<u95!>Z^y&sXV0o<ZI>
zS8oiJ$hgG~Pv()&5Tx5FFJ|U+QJ$7#-$L|yW2{2l3cO>TDBp>h#t=3i?k-KVu^Xw*
zG^cw)G4<XIUa~xZKfDgdG;8^%(1DZye#}`cI^lKZBn1NbrKxkxGZqSM*LiQ#Ogv}`
zuYk7(NmCd|;QNgsuF@k4SiQ(PNT}v0|53d&#=P+4QQ$DCgLKv&!4G8Cw850aIyLLO
zg(ftYLRy;gaZ!^uuqC{a<a$vIZs+*)A8}(UEhH9B`$N@r8nJ9|kZMK-t=SQ8<dPT)
zP;;{P$eG+G<OsBl**wIZbn=!{?_FkcJTLo=F{_^L8}oNJPFO`zF@F(5;Yy|6^QV%p
zf+AUiz9&~0g}m2TdFJeTJ9=pb&IM3sr#BpyM|?zxL+pa#3A~Mlhmq&>D6$ldy>Mu#
zG^S)UM$xwfHlkPp_}R>Ei*G+T6|bjw8o3q<pX;{{e<82>$C>O=P3?=@*jXiB(jX17
zCl;lYt-)H7VJ-ehs?zpl{3^A>_={r2l;h3kVoA1nb3UEgL)3f=&e7<YX-r%>C<ui#
zJJbpbn2}j*TlM>iz1nR}xXHT|J(mN@bef9Ti(YUet;DgAHiKDb=r~h(({ZbTF!^{4
zk5nS4)5UM+IEw|eGFA~onOvG&eb>2FjN_KGKu93ECr9T&;E_-j+GDHf*SH(S%I#QH
zSz7iHcZ4Er-hJg%JFpww9b&W0XO-WV8ytg2mgLvzR?W8<PKF6zm4KwW(z#3EQ=>Nh
zb$>42Y*7Z@nC6%ddEE~hY4l`tN;}p+uJ8c~0O?X<B_~G-UviWCU=G+L#T2~E#k;Nm
zO(&U;>d9ikbh0BftsuLsya|?NFsG{9Iv9QvRw>c!QN_&h>(4i`H!S~{9*>T(3Qg1e
zD!lx^l_r;H4SnEnq>I*Cb=(_chnumaA-rmOE^pAPTBT0&j8QuJlEg=iLaG?!yv*#u
zbIlplgDZgt94<0erzoP_*sYE+<O)Uvfpsyytao=*X_rk+MhQNYiIvJhYM&0xHi0j$
zbmsd|;a$u632dMv#<H~}lpUU-Uo>0&;phcLy1^ITB}Cw$zQ|poPBeD{W#?C9WwGM|
zSsV<6vOWktl<mFgZ!>d2Fls0(2?yqS_~+g)Ef<E<T1Z;SWk(7yU{3w*8WcnHAcSX}
zQemJwv9lr;9d$r96a|;9$z|JP*;XH1T2D4|4fn2&xCq<duVH!IO25lK>pJ)|%>;&o
zpY3k44fhXSu4-bYwtaQ(sFB}VbngDSzv6NW8IJ)-?v)I^f0g@Qa_=`kkAbY%zK2Bh
z+l)F8*3AT;|25rne<Y|4-%#7me}p^u;AnlYKT&}XXwO%v(+w)?9`!RJ7OHx7G&DJ?
zU(`-Y>O8n=$aW~IQbhe#?Y*!DRZ+Lk=#AaAHOV&I@#T_#ivwRgD$J$YgE{hmNHxA<
zvGCtu9`vq>1P|n;hf2Jw`$Oo=AyFDv#Lj3)73j!^DlxD|mMNL#{;4db=i4CW04K2+
zN{oiMY$ZIODoda{h>C>^drcLv^8e<2e?Px{XLZ|WGO-k2Dtk?`lHzno2$LvYE3_p}
z>of{{<(z*@0?_@XW3PM8(d_T(Ks5+=*Z=t0ulOv&X$hI)H56>b9n9?SSw=f2<Nzk>
z8B6|<`?bU4E}a?ev5Qh@aN@_m;8hU+jTOu-VF&ebPr?6UkiKu<*^cG|5b8nP7Ztya
zAm-pKBBLt2BIFyv#&7{o5F3)`)got&{3T#qz;<4(>l4$KWP3R2T8MUk-5RyrzWc#y
zrWOQswLCV-DYn$o53W!i6n7M~gCOdX5w2xhnay^i4fzCuoxSDUNc~lOyfdO{q06rB
zu_uHgEzqNo#~I*T=K^f>xm<sNO1jKAiz_zljs3>=7taqo>3zoymIfL=+@>7bBW83y
z?+lM?BF_YZNMjku6z@;WWkXqxK#3JB{jUuBp%!i1Ud2qi$Q|M$?OeDGTT8W7Yl2l|
zvn8jw`WKw5jWiDgdSyQJ-A#B;#Is#B2XuJP-0*pf!gpV?@7&pp`-{IGM)8H!jh64f
zWuhKXKCS8%lazN;6(UMkbfcGH(ZD{i`0oh|{x_e&=eJROk3?5vEgI&4ntCu2>r&I5
ze1emxmPzV8?8QoM9f&pzhz*Vn7$O12q%AX<6O5s_X7J442sbljTG2D0dz#^oiw7xO
zoX$Mn5m&z+C_*32_Wo*IA;;`~v%*2Ss=Nykln_W;TXAR6=>u^vfeXt5_Q0u8+&qOs
z#l0wO;49HKZ8*v>FE=~i+JG@|dQg!sGedc&+ME92;^Cocoq8Q(jIqukY@rI5rSXsD
zt74$p?oq)#tRi#2p%VHWgk*{LZA0?2O2?y5dCxigX@IOKOb$Z8_!ZL!QG+<A^K(9X
z&)fI;C1y1}>AhJUEYAvRh!Fs18qnnfyeG3oCj6q5hM>V;J22GeBh7Kd-5&FJi*8ZW
zfq#-GcoJw%#YVibbNLFqS}bmmQ-m>d@>%VGidduEJZBlT;lX8$Z`6w+?#`qCa&85u
z=|)@Jg#<cVf$p@(rZ0KXjl3=K-aYfiIOp;6Dc5-ViFaoOH}bB8+mB>&+R`Uqk%yC<
zNeJnzJt8v)W0xuXSr!CWRY+(!AB~G!k~&}H**^Y?>}<xtG-Y?!oQb-2<t4F!3fO1}
zHFLl*<(0EvP+j1Hyu>eTJPeYGLsnJT2}mx!@WU)tQb20s<Hn`L>~KaZ!PUaqOY8n9
zuASHJlsd!HCq}5`LZd)!bmMlil$dffQlGGMBj~;8Oj&8GSgbK>G_NFafaKWYam0iw
z69T~N-OW~m)vn6@$l%7Mly4~rTEp{EkUK@}rC1bBn<_CsA1jrtSNc{4?m+VBY$~QG
zRvyqw2V+|LHcO_!j4hJcSU?z3(EGs|6s{|Xz>m+pyWu-me{T?#TSlN?7*DEK`HJ0D
z$ZAu_+soks$OV{DpUN>LgN0})F3J`X3XBsa*aWRJ9xkz!TRIj0DSA>RD05)Fn&^zI
zScVkNw=0%#^9v4Jm9LXn8Wf3DO(c!3C$3f`dY*^hyxKcXPufr81V?5?)P)Wii^zqN
zJAnG|5`}{^329@7&l~R5Y~kEruiqALb=0eg>RDGbVr1Ig$3*>y$x;b@DhPZHl@Xxg
z96ey830h4zZ#gSZG?Uz%NjOaoJlrESA-41u*0JOp;;^u2cB4ie1khlWO^-dc-vck4
zcMP3Qej&}C%%-+r+-fwV?<qu;ut>(IETK2}RKAB(md8QmTmG-9%ray=Jkbuc<nncT
zWsoid^_}PRS)SJQ`^!NDE@L{dvvlTBm6zXwJNpE<&^sa;i{Waa&uOKzAMLc@&;Se!
z9HrR$-q#IJ6sH%@^aFdQnR7umQ<iF<7qV*(g;hYrc{)p;tc^iVB-Rw9ODDu*MXY_v
z-=M+47CmFL-4v{g$o}96kYT3GF=#N8<2!lMVLGG?c&7Qr8?GS`b8=@Pi6(b+T?}*{
ztuxUGgC0Ow1A?C8waV>9IoGuCqP}*JPU-8k;{`eMi_LsUZ`<WTb&!h?QyN2Xgo`Mk
zhw-JH!X`03=9tPRMFz0l5?lz79<9^r8PvpCm=0hh152XN5x_z3Tocte&z$JYG)<d$
zj6F1B_Zv@)D~(Yqz;%tPAM_q}`m@H{D2Ax7i^-HySB_H5{E*@9Z?0F-#R>vrWR$ds
z!5VNw1}QBb8ZW(g6njD>WwDfyP@HT}&8vGPf&5QM0-5|$pQXc9Dg;keR54JaHYTt`
zvRRqnvN@W-o6M#=9!^z9pyU0kQ?In1uFbi+wL;!+jUAE2_bOQ^5U2jS*qaKb8)ZT(
z=gcv2(ifa}T7GC#@55fC#Ri%kn62rTk*0a=3MS}rl<wVwtwO2JwRt?3RZtg2R@qvr
zE)G&$huzHVe{Ek4BL}(6ZKn}|DoE`GE#>pv+)t>k&HdNz_V&7|$0mLkn^>z`e9L<d
z!_D%fE%bcxiuZsP@>Cnn(T4EYhV01@&Qw!LSv!O7a=y?W+AC~LW&Z_NJKgRB+%iEg
zAzCBlRg^b5eqcS<m9QgI^}u%=MNiyu#QXNyI@ynlsM#}6hfe=ZwjT@rmcbG4Ss&iZ
z?8nhD?6Y*~-xM>B3y-bjZIDgNs+S|3`j)q_Kal3<c_*|mz}2e_7J4DA&g7i&(tPeb
z!0E4^grNL}IN}0|KzEvn4(L6O0<2!qiN`2DnB-2$JH#jRbw42lNR5$-<=R{9tF-J<
zwVt@{=ibNkE$z*o0+J7G5RwEDXcKGLdzz%Y@gZyx1@4lg();QLZLH;d`@#&|vV|9L
z!;6JM<=|e04ea-1MtKqLRA>Q`AtUSItR`#|Zw{~4*tu5EI@T|f!LXN#xVbIDOl|kK
z`EA#o7FZW>{s8~C?>y*IL%UI#orr{Tj!>Dvi+14BhNEJjI<Codl*O_Z{$xT$RMX{P
z3cfT7R_OHJLS`+x&wsLj-x%azi!%TGa~w^%9G+%o&>+;|_xkH~G^yjRQpmh@VW{j9
ztyK$>m@3wV+GTP|^3>#>5Fq{SKF?6@me806HrJrDNqRNvJf?VvN~Uvv4PGLiqoI5)
zwD)dg)Vwel2e4E`+11D-z9kF=$;2-Q6HSSc)KETFWGVOV5x<2~%u~f&sKL?Cu+D)P
z(~8%E+ab~KKbJ%nnDP_kDr3h-yCpP<0~@ys>r&=-`W4T*lr%K?fkLcSTOYNIrZq8V
zLrYO2CF6}AXUCs_Cx;=1i-qxUL-#yGm;D5~aRv>9jZG{+aCsA_*Lp|6nRtPevoM<0
zsSu>>fMlF+XM@aQQYn<ya^j4VJ89R?Ket_>eD4OvX(3y0Aad`sb^jOnSnIBci+sdR
zt`J=4d%JHoXMJ>gh-Hi@3e<KHp^jhHj~LS1QC30|w}%=~x?QhM2HznY<un|E>oBfg
z>r{8*oBF*L+loW=)uXkvnpK0!Bh7r&s<@K`EXWvv{+pgC^!{@8Mzx);cdN_u{5T5b
zsWSWT=K*p3&py+1GC;*|So>@QV5St#3&p<_DkKt1zgt3sRIP3(aG_k@cY`e2DKpL?
z^QNJ;v5etgt!kx|doSQ?E94|2NJ0hAxeA;<t^P2>LGIIXPp4Ang_lymYYsimkNOI-
zx6s>n);)NmzJlvV&vE~`j0s(Vx3`|X#mas|Wpk$%g%ytrl_{iVi28(@MJ=hSGB89L
zx=2I?bsN7NRcg}i_bd)d+Bp>?xWGxzslI6ZGTVal$dD18{e97iWoe@G%T?YTHdROk
za?-Uj=;k@Lz`860bVOB83G7mF=^PL1(xcw9T4h5(XM3>kgECZD^Jb+E;=Pcqw;}Hl
zZV2H8a5MvlTbQ17GM%As2kuG!>0E%H>Ao+;Zv<J*mc^l)vB^mL<t)HMa}W+|yjS<f
z;m!ka_@R%(-98RK^l<?GH#q!SI9RR7KR58L?m;%$1_FT}1wz^U;hf80@RQ;qW_sfG
z0s;Q#VNe#2W%@sh=RSg(`9IL?6YT+A=vs^Nt^W`BcfgBU(rXWRb3oqz1AtY|&(?L&
z_k&|Gqbqnm^B*+sj(~S3*UNm)KU(siH-=R-iU!wz`)^opTXy~jZCRhzU20a-e~!5o
zQ^&7Q<>0XY0JA+M-qw6!Hq`%r2!FYG%Fs0Gs3wy2!coh;P(I%5mql-sEAxNMGLj?n
zLVsg1`%vFmvY^3wbmjF{q0C;?@W^~$M@hAVV}pTJtzO$JR{+q#W$-FCgb#X2b>)9R
zsuOPZt}G6!Ox5iF*3J9>_rLi@ivOFRjEuN0;2|j%Wgjg4;_(Le-|7`B>c@_aH^#@-
zH;!o>J7!Yj<c4)?m-9LhaDN-)D<m*8z#0Joe_}cJ6>mFE98bRAz7np;{yw2v)HB`g
z%5JfN<3oK93lnxsCOmY7qz3D3{tE=WKL2l#J8v`x^$0nTV#0cbvkW}v^yXI+-|bAR
z?7(3ArpK(MC{zz8OH@gpf-q-RaHebyN~Gd-vcrvpRQy~Q!{q?xqJx$)&ta&diQ%*^
zsK;lKi=|+CTXq=VY%q;mf&<C9J~7<X#JzHWK~C$Ax|cYHn%HQJiI!Hd1_yYDODPT<
zurH;`1ig3AU2+JjI=jnoAc;_y-lS}u|JJ3C!OPC<1p|5X+}4*h%1NI4h^Z0IVdiLD
zAWgjEjM(rWd)p!>pS}TqeNcw-oXbw%?1=g_m=3obguk{L9fW?=Fn^aTB6~0gm;1q7
zaTWOkDX_Mu4y&s!R4`dJhKXh_9TmzQ+=BQ25dMY=<`^l_P~1NYdoj{MNyr(FcNy7u
zj*4<RX{Ayde%OXGFjPPkGKaa-BKc6+hu#$`J&a;|N>9j<IS6(8v*kxyT=_jcgDb35
zJ!btw{w_V(-l7qq3eJ(zO~Qm6(E?pm?csvzX+DI!>TKobZg8V5M~#aIy!+WB!=LZ(
z2W5uDr2n@=;-c~1g?2#~;n$oJ!h!sp$V0~+Ff6XKVeu;zVhsxrz1Cj5XD|Hm@u7{6
z%dzeb93NuDg9Zp5$bUIN)SVg(E!LmvSQ{+X#hET>u`Ujldm|VN5$-OQr`S$g%LYC9
zB^VZ5d<Ws((bG}^88ol@gNAM>*UpXT=Zulu@M0E1Bo5?Sqnu#}7Q?r<Qp|pXhpl;T
z^aTIi@ZlVpTn?-089y10<y3w<#XmE4Ksg%JrVI@|mXwUKF!C6v2Mv)>=@GCnzCnM4
zaCs!WeJ}@8ei~{z()AYZ1_<)1XCTV=&p5$+X8)%R5*v(lgkSYhJ)wc;*g#CMy_@9E
zA1mD7tS-xR6g9`<cyKi$bx4M;An?bEJCK;6k~`LXA#!dix{xb{6lreHJmM+P_ExvC
zpJy*zbwIBol0Q7VOCEEmRqaH9vz9iTyrSX6WhbxQup+Q#)v}fGw!r^lhHF@H!uqbY
ztNAc+{MvQv)=nJHcl?Vny_wmdnJ(}F|L>NWpnp01Uw^LNpXO((2m`d&@N)?NEB(i^
zBxdq|TyyWZi8f&l%+4n5!JiUp5(peus1l}x_*tZi7-=75oiaM}U6*~=X5SsDOo!>G
zOPMyl!Myp;fB1DT{UPPdzmv)r^InxV<not)!MyW=xvKEh^MbD8^MYBx$NcPd)*sid
z-x!z^2;4sY(Bai&ZhJ$`z4z;SHTBRHRlGXv!r&>B19=f1H%;KD>%3r)qL1?gY`QSG
zVR9h9lE*7}uLgz4KXziEK;KRxndid3Jl{vPAb)us>Ejv$MS9nuGP=pci{c8NXXwR@
z>qV0p<eNA?&}RV8!}R(s^6{diKhK}gCJGB)526FBXO|Z7t~;4KyFxF=M%JAWm|dZc
z>JCKKtmRpib!P`|px`66<<2gr{lP_D{PfwdY1z8K?8-bcyd_G!eADde3~Iqe6d5Ba
zcp)7!@s`=OG%C0#5ZSP5bzt`3Qoh-l9XKwRkSyNcqVkrf1A(LJW)Crh0yL@k!Q9z{
zpjYtG)%;u-2qd$o^jG#bA}iMh>@l!2K=Fsx`ao7DOIBZA-*c~f&2bQP%pe~K0zCVD
zA7I7^e5b$%yufj;f~<)fR`t}-E!-Z}&l@Ve8U(W1BO5?Kj}C#Yojza)Se9LseCFTW
zYeV3y79S7-SG?;3LSS2&4;TXW_1+DudTI~?SrhfsE(8V$fj1+|PYl?jLm+R14;TW<
z;#E;{{a*Ll5Qta!fDo8@xeo||sdIe55U{T&Z&=k+gCVe22)ro-YE=4?$nw(y_UI6}
zbD<9y0+wZ0WcfNjOIIMWd`;$A)!#keW$#k;x8CCeR=s^o`+Le)_1<LtEK&6XRsEZh
zl~k`sSAAvN2dsL%&+Z#pN%fY6>Q_>|dsg)qP4)p*f9_R2VAb2V>i<mnR=rx!^houC
zRCar01<87J)ju=Q2dsL_^2Nvss<$juzk=%Bv#P)LOFp3Lue!?zta|(Q(G9D5%2)N-
z6ZK=gui=vYMr0Y)>(Nzz(VaeE)$2XmS{R$|E!8ihdduFW>c4)i52*S_ul50}-o72b
zVO3B0R=v1C`wb1rn~xefCBl#l-56Q9Auy#%V}EBLvWn+^_MF8~&FY)2-(9)<dR2w`
z-BrSGcHav9ezA&Q4)3Az@&!|32*BViQGhenPpP#shO4H^f{H|qYWf{X7hjSXs3ha#
zqxU5S8KAbDygd<h6$8}2lyJpmN7gN07D!leZxyPzF;|o%hEiNu<8u6!iDCAU1jG1U
ziQ&B^lqW`5+1aFA_DEu+0inp+lTQjHMi~$$CHtzxXajO0E03q5dINGJC$2pqkcb<Q
zN48}f6Ac1(<^zEC7y}9c_AE_|wQR)zu@4iC2J{IeOA?3i7=DJ==VmA7NKU>bEGLyG
z<_hS_bsw}CkQYc+CFbjczq$_=@H+fg|AUm}DRCbxG@#U6b#F1t<!O>>2nLR;N+e;J
zoKZ;Mi`FLQ6^KR~HjW(^NVMwt8BuS<oWm0np<b>yhR&EIbWQYb{!(JH0ZJ(<OdKMh
zTh!Y;GBL$~+{lI#HwF?@4N&9rXC|5iyd~70nwpp<V5j6&&6vb=<A?r{FRVH}keFd9
zwWQRQC7K0v)d5WWG%?eFApn!|60<DZP=FENPaF!6n>8FDXK7-#fHLU>xkn}v%J$ZH
zxGyKK$p`^o=av~EydGE+RiGh9&7M<Rk~l)met`mL+>us~s7vFHA|YV%>+l(gv;iI|
zdlHFu14M(T9!hlBHz5ERV2P#V06ct0qSFBNXHiaKsbv$>Y#x$0+ENMtQpY5Yu@qyP
zmc+4^;xo-M1NsQly}Jq$s{r%X@=^A-M3*HA(~h-?)dpy^9Qtu$jRESdQ*#q*4KTD@
z62}=Jv~L)h_<{jK`>Ded>kJU%U-b9H@dl_3`L&7l2B;0)6^RoJP#enH5+@pvFXU$4
zSe)1ZnD>>)%H?pv#(pBHkSjVrvB~nOt<PVOIMo2PweYyaX$BOjgmq7NiE~xL+ay8z
z^DIdvgua&8Vt|^r{^Z2@24v}@uER<a7Xs!tku>RtiHmwZsy;li4IJ{mqK{s?C~>i+
zsO8CP5|>y?$Vre(Mf`5H4?%O80fm9%5or6e0*NbC_;%9D7bLdZb6{~Guw_T$YRedW
z3@G@`#18Ag;Maj>?Mdu}z4C7fJ^^&n?!>h&?VlO$%fIDVabIw+UgW(^)(3s|q(R4>
zT%Nd<dJ1MzC!_GohPJ4%<9CVM3{ZoH-jKN60Cn=Y$;2H7sFM%BJaMO?rlvkRCb7qA
zSL<$hHF1{#l2j$%OWbXMI)cIa6$6Ty#5M#H_ZWFfgkSI$a9L#deJY|<{v=qK>`QIM
zH|xVtQqx26%|h^%^-0YS#Sf^mM~5Z*yCMOOt4<CuK*C`8aY@PiKvx;Sx66_y^mmp6
z92H7R6%2G$0xbJwGHQUdygP18O3w`Ji~#gmk*u|pegL<>o*Zl`{Q;UzNY)vkd1vRn
zNlgaDPYwjAy*@e2zL^g2{Oif#2Fw8P<|IcN&<t=_S8|jAqW%RRCr1MmzcLG8@NLO@
zOPLFhzddOaz|Izc*t(>f0OkX%dOhhTfE2)Ook_|3;<sA?3eHWAH{6Z|xbmIk1S@Sh
zz!5)6PPCL20C%(}Ct1oWfY8+BWCOYY2Aq*R#N_5`fG3A1r&!7f0FN$DI>CD)z~N((
z(=25p!0t<v(+xNUV9FgyCu=tW+<9-Z*-}mem_9H$(}2?f`rnY8Wx!^D_JPSm4LAef
z@T-%v0dlj>1X%raGGV~E07orJYDx?2+zN2!_~aY|E(G}YuH;-RU>m^FH<K-vaxuWH
zW0UhN<x+rqOOo>qxD4R5ZzUHPa5=z(HzrdCTmkU>>B)r#TnVtOZ?e?@4bR74OfE9u
zDu6v-O)fUzYJh&nC)*6TMk((mP1tAc066Bp<dLKle{e0p9ea{a+FuV))Hms*{S5#!
zewl1H!tVxH`rk>D_T9GuoOfl?3HUt#6Aw);RcWu>1(4239&O(|05Iw9<T0e=W<3Tl
zxiBf2-Y4*NlLtHb;sJm7Gu)VP;aG#SGV-GM6@v%$z;7EI?)m-$gL5+JeF6ptd*Bj-
z7uxWBH8>=Bh3XdpF1{ow`Q}uviJvAV;QH*4d>OJed6nT`3$SfL@@mUAQb2q18UscF
zwDe8x0C2PKo=cNERlsfINm;Zad0jccBN|D2u1@Y&%2UgFaJtQKiuS;d7+fegw^msX
zYQt*=e<#ow2t0k<m?%)eOvL3q59RhNpy(3GoW_s(6_QZGNhomP<NXSw0A<;M36%GC
zAOEMMh|hlTOm@FM`mB+h)ra>Jh4@T4OCqF4PmRvXQTjWP<tw`avvQ4VRqV4rnw3X_
zQwr{RY*s#of1kG{Yv)XuC8NIN9ZA$Z56vpJZ%P14I%gG2GutVN*ZJnGK9*7e@KWDd
zC03e*+@7VgN=$*1kh}fovq~*RDpU4fXO$TsiT91gv&<ZyF#skCluUvm{qkl-F#Ag)
zJRes&tE!O7X3&fy)>h1ts4QK;cd4gmnWXGiGw%3cmPty};y`JLP_jAjoE5W%8lYwz
z(K%}vg_U-d7-{~w6$e4-im7TPKcy=IKMH1pMCoEGzHZ%Nwe)y!(QYzb*v6XyhYZgh
zMCz$LU$AL-?g;;R^B;$o2*vI)eo)U!@4Y0bjy)rHPC4pm=`B1z{j)jcG^zBv6d&kP
zk!$awMOF7?&ynCOeIM**KQm{Fc`oRBM&|svhww%IgJf&kH>PC_*~-M_uU*tK)~Kh3
zulPwzqXFXbD{5NCfy3Y`&7yl+T1-Hj#yYa3Wr9&b<bUWJEfeh<O)Eow)^dmeH2_ad
zZ880Kr{<WAt6Qd8X;Fah+|=U4@bJjGQ`QGsrj@82BSgm1KZETPedbPqTgq-1`Th*b
zU%xXtubhI*598~TcF(J@>V@*l$Ir74-zhG;_>y^jm9kTsT7GI?#BdS!U$Asul>vDG
zecqqvY&FgGcl=_W&Bfj7(z(B#S8YX$GmqXkZvfxyx=zd(`*5C{n6=!0@$7j65dgdH
zii);3esgXs{;%>jur0eAY)da6KHspF3-6@e^9@@wVqzc8H*C#_x$fop!gge~aN4tU
zend5QDS)7T{(LhmyM@NciSrHHZlSUKx%r0eTO$4C@%e^rmuPzQzWGukYIekg)53S>
zE{=lJufVB1&5Nd|=C&DI@%*iYa}U?^PH-*UGiZTiM8z|dankMuh1PsgHum9yA|sh7
zJ96TJVgsZQw~bmL`4H$5nNRi>xKS#F_?h&AQWdaMH1B?Pfg6}2^U)OxM3UqnDZ0Df
zSWuB42poU4+IP`K3o5NNDK=N%w7~VC#(d7x3ydb+YV3WdEijtw6rb%`yr7?bGf<7~
zcy4aTV4Cn6jjhP$#hwS|9#7jUexVnChN~)m;X4=$1^j_5-|av5c!poaFL<7sKX;qx
z@fOePe>!)YoP>IQ=;gT=XWl=2`rM1<O?<12pOaAa=D?tV22VU&IhoAM_RPHm0aZDN
z=M&DJdt-*!Yu=uFR}@utE$?GxbKlVW8E;Bdd_4C}O3ZyBNXm;R&;2o@JJ7}Ru;p{#
zw_Ihyo+9r@0;rlVFC)jIxxsuSN#*?l4g(k{Aeo(#W3cb&h4M9~su9<L?&1K@xo4&Z
z7!)i9+I3T^#zB369%xMsbWjOU@|x5j2bBVioS2FlR9t4R&oST6PW9_U6>Vu6duA$@
zL%lEYJm&i;Uv$pFvGY!<4!3jP(cs&+jXtPcT#}Ne+4o7ht!7xtDVpN1E00V$MN`7&
zh;OFM?QpAv&E?-pncLxMao&z#%G?enf1kZBWp0OVb7`Eqt0~n$*^#B>n|FW8l*^}8
zcJS9J^Df+~5`rJ4%)4-_vFqPb<HVu2ibFSqQzq4J6^D*IA!SnSRtbWUr>0Epyj7Ow
z%grg-`;nz$*PI7alU2aeV*qY!OF4UgJV5;Vl*zm9$u@C!^TqMTBxXu4`!qKtrLyWO
zh}#uNnf=}!R2QA~PRa~;P6HEkd`cF4pgWia^p$r~GU0(dprTV#X2W*}vw;RpOeI}f
z2<WPQDN{4MgJGcab5f>fb_a8SUU@V%*X7Iwy5i=PX`0=^JfIhEOwDs?`9Lj)rQ}Yb
zz5<}44^s;qR0wqaE2)%&ihwqLKef<7Vyw%4n`(8C7;9BJwa7tYto!?=7CWf)(%^~9
zRtDNsL9r%$I`nW$%9g=;=M%Xp>6ulpl63OUR9Z=R@AOTo9iZw}F-!U0REJV_idiUO
ziC+TaMKfmAyGC^e4@8-oo02o3pTrI<V!i}p5qj%b|2y)1@27>SmCDylz9Y+0tCVE<
z_^8VOu@aT7wvTdE_>RkSQ}PV-+e9Pj(&H^D7;ueTPYK<FgfCy6U`f`<r?RqBC+jOI
z<0NgcB&H7K=k+ayy&3SED`%(9By0b)!|E($%`o`c0x}GKjsZS{pKE~6;O80OGx!z*
zd<H+?05SLzfz(z3Pm93^Kask?0L?e^9!XtjfJE@Lhf{JP6hA2uy!DOLHl=ho0YJ=)
z4QLJ|b5n8?RQI_wcv|3cWhjIorw6XE$HGg4Yc>b2)N@W1R6AyCYP)?H90W9RQ|c-Q
zMS)g-J>~lmVnBH#QrEb&TA+&Wq;@!HFi@YbroQN)I-m(prgl1Li0g@6s;<val8)$`
zy4I4s2|($1>N<mRCkN(Wuq;kpkL>E7Ms`*Ap?|CAKbewhUbB@ZzIIXSOFTnWfZab%
z-3Soq5>wp2Aa#=gnj6RGrfvo({=ig_t*Kk6pk}N3B6LLR%T}9+-0+jsZ3c+QT_aMr
z8z3T|e`V?p14QIIf0^1NU}qFy@ZVB*86dOcws%tIqUf#z$op!_Tom2I0dC!wy2lC_
z4e-RK)O`j>S|Mcbw*n-scHNbFz_6Yu;P%vm2227-y`Or>fXM*QKbv|Opk~J*0N-Ag
zdc;aopa0;#RJQ>dGE1vddkvTku)JUDF$3lRJQ+-V)qwc`w{A>5Za^zQL1XG`1{@Bs
zwJr5^1H|f$4XJM!&<POPmwLj0V*$KAsc#yv5+GWbdeVS30Db<J`j!Ff09MzhzHPvX
zkyY!F9p5ovBPn10W$L>CHCs;uSg|Gbw56O2(0yX+83VQgw0t}DtO45qu3emZ&Vb7R
z2A-XI-hk}@eb%R5FklD3b5m0<8gMPZ=C@NX8L%7Rp7PYo2HXtr%<$AJ05!MWmKpLj
zJ8a1J*rYYi{&O}iBB93Fe__d-?=+UAUZX<>iqO0Mk=mzn%@Mt{BlSZABo)uUCH1=X
zVK%_-K<W(x%$vD1^&<nspigg1y=j1yx16@rj}6OQfEzwe{lroV0j^q|`l$g$0ADIj
zy#-MGL9y`p>fcII?*I;57g@9N<Us1b@Q&3y5+L7K|CV|eMP=YRbCm@)@p$HC=skf3
zeRs|iC8_uH-7m;mwLA4|t6ASMDt{xuMBCoGQtw;M!uHEkQ@=Gp*ydi7`kiGHwz=1*
zes6%}95VF}R+=dZ3y;oC{gJN*jbOwhq(8JIn|GUHsgE2Kbb{hfRNFn!pH)7#Boo=m
zTXIu>RT+yYp4s7VN-{Z2U7z^*CI$jGy_TB_29)u!lvNfCy7B{ndmB>0EC&VwnaYEn
z1G50ZC7A6%?_QfEgIe9{9*~U^MlHdxC1wYx=9S8dR8WR?G(jRH3g!dQI3Q>gls+Hm
z3IdHjB^4~xhqq=4IwKV<GDw9&z+kaKVL{iYf_)6i1%iEoB?jdK!9PJ$>u)Ut5;A2<
z>n;&8ed4rP3VifL5+P2oFMpwqpEV+g<#V#0y@jm(mGv2)Jk?fCP~=&<FBLRDC|VHU
zv4^CBH7+&Frw58|pu5KN%N>LP5q&0GW`5vl3cvT)6fS!{`WJGbm<0!0jtm}ketxn}
zzL>01L}P==DwKz-rXG|>^k(f>LDtdAy3|Lx-bdL-DlVi3iPoY^L+_V|U0;|AHtNfr
zB#v603Yy+&m3-liRB*hNlj+?F_F=B5bjvQPlxH*cEgw5mF?+C^;@5%x|I)xza3(&M
z>ZnYGKk&($m<lA!dodN9#n-Wy0Iqp86+F}c#@+UJvOSqjwX!ph==AzyY1EE1=T7V+
zc`#~+;JmOO3WT_^kNByr7>(%Z@)8KD*hh*WdFVCXtUk|M*ZbxbkRLy|&<kFs@-L&t
z^*4FJ%PaLspn1c_lLKDx3XGiE%d(p#m4PsrYROFv6@N(Pwp2iq+DAd`<=?eb%G+at
zZQ!*neGw$3D-<r#Qo^vPePx4|H(Sc6Bk&d~U+q<rDW|?y)gs=<Wr1w0X(^PyWQAl(
z?wXb&@F=@o;;^No#o4lw94BvYF}B~C&#GAMqa`Y5-FI_aMyQ+)GBoaJ8L6%^RA&6M
z#f%ZtJm-vP8Eq-qrl7VMXY35iB3)J0vOZ(ZZP?MWUh-Y0-LAeZCqVGh?^4S{yQ&uE
zs+NzaW&D*3^KgO?A0lAa!h8dS?SzJf1qS5W8tg(F&Rvqq<Bwfvqq$2`dGC=6Z7_F9
zpKEw+p*3oTRE^q87n%p}oeH(B=?9Y+$`G#`LDKHW7gq4D^v|mGzV9xq#EnrmLaM?m
z3m0k~BhW<<4Je*Xt_!}{+$y)<kbaSqSFH}T3U_1<nb%*^>fC-(c3V1Hoivgd{rab^
zP8ezN<(w_8y$*$neyzO@h5Avgy$*$v&emRs!g0;5Wmb>+YSph=g?gY%4wENVwN@A)
zk-w+9)%g&l=I76Bb&ekqDd((K=lBtkR-V}E+y)}j(9TxLBIj{gxv5pYo+16*WCcqN
z8DW#MO;-7PD_XVqGDJ#4?WL{qu$H_NunuXBi3dy(ynJ12t+AX<pwn9iE5-au$8Bq^
zGoXY8?$%MT@Q`kbI<lyBv{=}@jw7>M>kY_;z8eCqaRXG;8S7gc46q69;^)q7mGBz!
zmq^!I@(pLkt>casoUCWJifxOZY<4AR>8AL}V@bLGzSc3=>qGu>u5^wer_uV{-nn@4
z{n6I(<Q;mHtIW(`!|D0kS|=Lb;zY2O8SFfvU~6V@ccH6GGHU2ij(XiZt~Tm<a;qPE
zPB75gr1IsLoU^Z0zQ3XG3u8J&OE9@vQg%n?x6V+?CsCzrYHbD>Rxlpmf(Ki*1T*xL
zW~H3jswJ4*tiu6r>S)yxOm0~k;P&EHEx`<{R+Cq5Y1I<Uuu(UgnnjNe{lsvp&cMSg
z%&eij2d=jCf*yF3!83Z`sRkdTVRT(dYwH{p{{&U-e!O)qNp2WLe$?7Rj|_W4!|2p>
z>pTJ98SnaPzJO;YM^+raGSIp}z>7_h6W5;{Xicf?ZnckITxh^-fRkQoZ6#anWpe-`
z$F$mF<Ba(lltV`FGc;E}s|9w0*2ZnoMcJS=Tq|D>Ua-iN2J<{Vy=+lVPv=d0YEiBM
z8X-l6i}D2Ql(5)5a*-RoGS2t3EOMh)>yLZCx5z|~xgj5ae32W<lC;;Kwa5)+IpBAB
zi<~^wlH(l@Eix^ldw^(`d~n=i36<f`nb&u*25D{?qiAJdag{K5OUyp>PmA3c6*`e)
z7P~Q88d-7j`oQ99K4EP%vSQ`(z~TW+gu|b+?$g{i{7vhgL8@xiq3ae$$vZ;E&F$we
zwlQiP-M)Nrtu<X7ebchVHb%`|Iq8Rs>#TSQlsQ?8hZrD%a`(-P<-!^^sxYuzvqET4
zwoQRA2<qbIKOpb8@r<mp+o^Mn;gMy(eFBL<;F8NiF=n$7i|wmIeRXn(!!b3D=AH%3
zUI+zmr3*v9)2j#nLIVnil(dQVBZX#5M_ZvG69714eVZeF&=%<1OzL+_*nW3MTdx#7
zd3&3Yey4QhiTQ1Ay(G_Osy2z#k=gkf#vWN=xQ0Zz2hS{NQ^$-v(pBt)r(dz_7)kl>
zZfjG=<d$W*64fy(krKZ3kKDF_0?#Jjo|d*jMsAhx)YP`9qrLL2{ZUDq<k-kJT}{Jz
zcJm2MlpI5aC1t`RZEpTm6Gu*L8)YdX%Y=Px(#(ixbvV=mjC|9r{4v}|{>ECBr@DTy
zrL0X$9;1%uyD|UW*1P0U@_d_=G*=fDk264Zt-q<w>1dj}cKoz$f~AO@-x%EHq+)Jl
z8Ii(mlPpDJX-&FKnp?o8<VnNYOmo8z3kq!m>e?jUM!g$3aplH9o1~j-?nB>ba|_IV
zb0yu1KM)Gk>K2&&=1RKN?2r|==b^R)>_6&V4Vn9&YD)_EumoAWDbRKpz-TY3?$y8;
z^`Q~MmOurM_I%d#(?^#Z><P=eca^j)pq9~RQZ9U#>cJA)EHpq^=I(84H9%O7zPZgg
z`$Q=C&U}3tzH75=KHnV<;B!fEPj%Z-fb~Q9XzgFy(mdDf5SN^^v8~-cQgN}|Hgoma
zQXI9q_#rhKR_L^Bd65;TuM4y-wE{GDH;-&P+EO&-&pWs67)#NVKV??iu?Ca^Y`VW~
znYeLh1;CPc+j2|k8#(#J;{$CgETvyy4wFx)?RW-feWQLRoxv9~c$kNl9O&kUA-^2i
z|Ni)X<u-bM&VYaT`{VmF;LC2eH*b?(cHQ{uQiFa1)#>*@{l$JSAOOBF;PBvHVyEj*
zw=6H9NPp#ab_AEH%6FOi2mOXnctc6%Oy6n5!y6>4ZfIvXSZPpp^Rjgt1L0s_gKQww
z@1_LT|LMR8{3px9!3a*S`gc{+L+?@fT@%B>4PaZp+g`oRt2lccHub)G1IC#6(q%_T
z`-`uKT8$qak&D?8-=O~Lzwd}V7%qOAnDpXHj>uQ--CF@3dG?3`^r84I+q8TUA5ClG
zqkX1}Z?oo~6YvQdzXe*xH}GRisHjBBy$xb6E@&Q$&K%fjw0Vv<uP$wFH0|2nJj<K!
zyxcsl_suiB8NRD|JQHW|7R&P_Z)yiMH}~fGI&YHOnrHUD`5JG&vA%g$@0-VYQ}nCm
zLwnzRl{YQj&9i&oJce3z_kiXk)XFV`bd&$uoHdMOzbp6fs^GHbtl_<1+<ta$bJoaW
zwDkL%H>^4-(3~~O#%rK?-KzCKqk+&T>Dd>CHfPlf!l;034V(G@n?|<EK$vB#Gn=!t
zH=yBuuqvyo9jHA37T*8hZB!fCRLg--yy`X*+)SqLEj_ZY-psH!yLof*1xF6=eRC~u
zhF)^yaMj`R?BLCpo;&j4-lDJO%@-CQ`LL{USJ_om7Q6h&V7NEab}{)*M6Fpq4WDXH
zLDn%x74Uk@(L6tOS*)-6w41`pOLJR7Jr!OCuRXjl+8~2stj6ri$2XWpVzc*UcQlwr
zV&Y)Qy$v=kn%dC(jRu<*ZGn2iI}N6h*uvqpuQ%AVc)oPZGuAga<D$mqj0R1M#UBim
z=)0n}qyhCmF!oUL4ee~`Z+t8cA33qXIV!~Ahn&-3ZVG&02!|_b8*1pGvBizZoxd~;
zvPSXz#3Kz+J!=`U&-WT)dY-{^d|N|fU-F;h@27ZUISl`W=$b<7k!{Cir}Lo2*u7~M
z@VBHDKhapl^M&2%0#+p(&ljS19#N651@A_wWq15DJ(vPX0eJM2bR8K=U%?_>u{Mw%
zLPwUoBWXG7l=M(ouyloHsn;$_YvK)bNyXZ+FYRXDQVr6vdl`0P_fkme;=;5{yvDOv
z^X7{xp6{IMksVoeVj$gXza4c*x>xg<_G!A;ek&|Zo9D<htP6jhcE()3s%osEs>T|s
z+Hv%Nv=qa}e@51vvOJKE(4cYWs>V~_PFGp$MEUWFbU*78HR!qL)BOjN?GrWVjK8O=
zhmzrZHRyZu($+yUB-cTv#+oWc6*Q%o1iJc(jIDWfY3UE+&W)TvYwD2}<DXNdo6bzf
z1-z_E=>;qVdO?*IUY;If^i`$zT$CPb=%`X~YcxQWp1nD3z8+)0RfnXV8X@*O`>wPz
zuA&lejsK*ul2`N6cMFYANT2e4+8p_2R_{I{eUHjCHE7B5^nC)%^Dyd>^!=7147<jq
zA22|A!L7eeKWKpbr8C>o4;dh(Wo%>mVFS$FGVTJZzWW<_>29hXe?I9{Z65qg)#kys
zsvo;1ZBBa&_5JcU(qFS;JiZx|pEjqxS-%(mHf>J(F5z+8eQ9&rTZlmTva~tvyM)Ix
zyVB;g?=n2bt0$UEOVWo3kLO6g^!fA@ICcDc>WZ`8Nl!IE4IQ;4-DH5eV)|q0X$-3I
z|11<G$8QrkxA}9)`1hDb#+}QLn?}aJCdbzJe;)MqcU1k=Xm0u}s-CczGB<8apUtz=
z%Eo<~KF0v*JL|L4=c?64{liX7pJ#wk|K+qTsv)A`=?NZXrn}10SMce?YQBW5R~o(4
z9C~!S0V;D$GJSPVnRAz?udx)B+4xv`hXJDGx@*#3G(bH%=f~-t28fb}y^-D}z{b`M
zUr1kTfHa_%ZRzU_5TB+a>FezqNz?fg(>E9(_usrD)4L522d{c6{UrlR)g^sz$Vq=k
zZ8()A+VGSm`EAewisQ=GpQddA#aIJ&vjvncvBr68)3$(OaXDwalYY*=u_cTf4ykC@
zaG$7E-WShlm$~6+K4DM0vo}Pa6VGn9;chfv{$#t8$D+>z!`q!a79%`3uieRGbB<j2
z{_OU_ls9PwNp)-5o!3QpKXF3)5aD1`$CmTkhZ>*~F1fmWm;s_z^~dez47Sjlp~Koo
zSc*pVSAN}YhKP;oqxZGT5jbguM)k6f+RYJY=KIY*YL_E$(uUb)UUlnH^JznF54>fK
zV3GNl*Jrm&Bb}_CZv3d-6jGb%PJ6U{xmB$OpZ2wOQ%EfaqN2HdrKJen#t+)N3{V%I
z_~Z7~1_<44h3#t$sERB<r7O_B7GQF|(7o@~_Tvl~BbJ@CrA$;IvA?029e9Gl^8DWB
z=Jrd4*%w?bmswdNB(+>_fH0$$D-6hWwTRaz=L<7x5wB0qkIDx$C95RRey_^Ah_uab
zwo8~zz6ia=ul(9$MH8--fhsF+9MB=AI79>J#dA8$Ibd>W!k!M}36o1Fp50+QVGMHL
zUpji{==;46;|ZhJ{ly(_AwgI@FucP;s;zrR#5<gEDtd)ibvWa+N+{e@-_#-Z*&&aS
zRJW$XyuQ{kPn^)<I$O<;o!ntwU#ke@;<Tz}t124RzoP+wT>$`tPwN<CfM%gZhjqAl
zLwzyfkq*rpLyw9AJfG@t^Trr}d#~tl^Tv39=brCy^TuQ$ci+^K4#~?y{zlTxKkArb
zWvg$#{Bp-s1BBep(H%_&h{-{2ngKRUuDw0GLpJ9W#aylaamOrIf(pO+g=oh-C0$C=
z_Xc;&cS!*G-|1LjfC^uCSx3qMzwm_ysO3j}siV~ZvE+S=I~ExrGF)5TvDg5SVf8~D
zZ3dWcXx`3fhX%rwHzO-Ho)+lPK&aueU9__e9d00~qMIr@+(1x8BPVvafuM?L-zvW8
zca?d$S3Hv2v6kYd%06AQv_t&vysRaE@A!fNwy?MM;%LVS$~P&pa@D#($B8aqWZBA<
zfsT`0K3Dk308=saeSV#Km<<+lnVj@+NyjD?a<QxIR96T9m7Ug8$fO^3oNj;`GPJW}
zvjNr}m;bV)Lo?6R=UoZs>|cUSJSMd+zNEt@o(#9y#N$Gt&iq-<66sb=!zf`&+mdpX
z;7{dC%vQ7^^Hj|eGZjr<f9;he&Qg@Te&CfQ5&K4xsIzj38*?VFpE)Iai4>)#g(Te<
zTVfo)Q+=};yICLYv}l#%UMcC67rse>Ef3t-X@%REnQ(Kb6>hC~;LJ`d+*+}ztkVWG
ztw0tvErbCtC@bld@2%+#SBw+XsxCRA(+O%Z;P&C2PEd=SOAqUGg4*_`w6z5To$|Fz
zlK|QMm(FHX^l6-~&Hy>JbEYh+?sEaI`)=nf1GWMjy|437WgE3E)5p`C|F<W!e46wB
zo~lu@)7##eq=xCM`E<|HPPtqhgAMKMoMV8H-27zcTmyt;*IAt{28emb{kn6W0b<_7
zeVy|S&?sq3ccu)`D5;p#xzGR!i^oQEwi*xxXqnKt$bdS4NM+|@1BL@Eig&gdFdAUn
zL!E~kAenO5q|PG^kWA@+MCVZkOa{3A7oBMXng9lj>TEZl86fLtogD_u1}HzJv(tb%
z0?z4NYQTJeNiTICZ9pr)xSGym0H&`#JVVRrr`mWa)?84zq9m|%u84OHc{jhgv;`j0
zTmW7BK3uYNy^{8kbj@8$PY`<<4^8^vQa3t9*KLJM-RKluujpQSGI}AljY=rpo4fQB
zmGBQ&!ls@Q2L58H8=4mMK7F5t@~fC-nf-4t;Eb#hC`>ox1*=J%Q6{78f+(+!o0}K>
zq1kr4@{W@caM>r(5z=C2oD(@^!}7q02w#ldD;~Ri-3ViFb1ZGm8PTs&xzw9eaz^wg
z<a5SWi%y@u52bbbsg(87eFH|wOqua|<fJu$5d(QOeV>Z@W@5x3ef_%HnY(C26riz2
z{6i%%wQQ%#eQMYU8H9nZ{!)ggUo#p7ym>^rT8gn5mZwqT?IWu4rCl}mHH7o~^&ouP
zrTP7_G@JWY=<{h)$$Zk+N9PNp=1q}RD+Bq$sA-`v%3YK%j2dgiw|p4255=AJFXhLq
z8=?RwZ_lq4dNb-G8`&qDKbUWtHw{<$O*=s^@0tAeUi8*b{N;bFE0FYP{)wSi(6<M@
z8>SaTRLf5^I&QnPpi00yYWK!>3q;)JU#k9@V+%|jm|?cn#8ati;;Z~j+Cu5ePwOmD
zduARY3Kyt7lePr->YIszL0FhG51DO<77U6C5PR3ZR1mfHh`n<c6~wIBt!l#L{xl)+
zt%5qSgd8Q_aie(?`+LFhy^Tua+qialP9NIQc)s<*;k-#4pHtHNrWHIwjk!7UEzkTK
z^(?8*naHyf-AC`sVa5TFI(+PZ=g76J1pz)fe1n&x`EurONxEl9&NBB=<g`G}a`#E(
zbRNN<%@{Tacar(<HV9Y98}t2Ba_$8CnZM=v#oV07crJcBnVa(`dTmz1Ve&EF5Qj<5
zj+{Rm;A8R^0AX@mZq7&eD`p*wLvvO|T13nF8=fn+?1E&fDmUjtu$i@SwF;&2ECR33
z`3Ls&tYeKLbIA4Nb<OqV%JmGj?tZ+!cSZX9@%l>orty5quC^2ED+$<cJb%2c9@X3I
zGQ;$tc=4cm@&BP?$+tAGey}*tgvM9?S???W35{VtsCO2C2GfKO>$Rp6u$A<+XVg1~
zh&;xRkEwSK5!p-Y2i9vTqW*N-WV+;)qWZWh+Cb7}Z`C^uPi(ZLsNQLKs_3q}>z#(D
zibf^s8(l?go{wvmI`qq0@`{hN&e6w5?~9KzAQ;)WY&AYo0nDj$@V)&wUJrEWhU9>_
zYMZSwP<K&$OwY$VX2r+4k4bs<n7H$dO7DrC5qC~e&nOljPqy5$Y+F2wPXMStU6Q0}
zMtmZ`?1}-#C-mg(I)7~#1nQ5BPa^fu`;!CXhX{c)_~fQ#aoOlDCSuYL<Fe8JKg!+&
ztjcO_AAjF>gKiNJ5COpf6%|DpL=nXS1r%kYD2O=Z1~#w(H-mx-D57c3m8Ft7PnnsP
zl~Fn5oDFK0)v43TQ=K}ezUtJ;I(h0;|NDN{W&>Tn@4CLfS8dk4hG#v`TF;uFwO)1U
z2cmF|?tPPUibwJyM@uxWNo%f0jsZwsD%Y5xR$B5#QAc0Oc`G$?EJLFwm?Fn98#acg
zpN*8=Sea{dPL5O?u~IG*e~IKKZ1N)$JR>I{ehQ~8<3b}RGC!I3fRj`KRN2C?$Vmi^
zhAN8504NIPyz){cuY4z8E$tCG6}p^~1~r~MJ2C_NZOLJ9AKe!@jqZ(bXJ<!Fhdad`
z?o=Z(lm4f*3B4mZF-|EK6S&AJQ0k<E8gxFzSF*@TkDP%lQqmAGY)NY5Of$eoU~AW@
z$Q<~ogX6`^B4^3A6SW--%of$>NR1Mn9ZJ>bNZoUL<m>?i9H~c6j+`A!kf`LE12(j#
zWH!DBuKRdiL`D;MVP*1cPp;-w$qYN<d07HjG6Um0WjWu}PBuh6UqMK#qrnMLK}Z%Q
zljZrH&7Q9@;bh6o^L1qAy-$YiuJ@WfuQRAYk-Q;%DBbpaQve&r1Mhp@L@zo48!PAo
zCq&`@2c4>xk9k4iu;E>$TNn!dhaT}5Hkl;fmTR}e(vYGLK2(d9TIetf0@O<YZ`?O5
zT|fsRxf=;S6_(BHZK|7w<uJpyN%q8_;d7s3*hSYq^$DJAWS{r+dK4rko&fDLzkBTv
za?HqMoYzhPEU1Qiy><y;)AH}?wOatyd~k%<9sv~5XRg;?0aVDzv%U5S;NH!V5U&FQ
zxOdaO-s_+Mb`hnCUWWwGuBGrFUXKZ2qj|Z|>#%^1v<$oYli5pWuRrla^s$e0_tNuJ
zIRh=+?4_%9lAAUFm*U2Fnc_G)ar56)8jiyAO-I7tq-+dMwkQ4*@lHFuOn0!4I`KD_
z(CW`n9uc;ndE8e|O#=hbIrF@jhu9FT-!RWx0GWPg?YvF|umS|}R48leOm$OjN#?vj
zC~|7r5U{mzp6kNh+0?`#*agv2mCeKu*-HHh*8R2;ZAc6l^ZA6Tf4^f0m&95VRsYi7
zA+b^b@uG7>Xr`hSji7>C>k?x}15RS6z3Wu$82F(j3Axy@t;oeDwjvj+j#?GD*zrzs
zPhrbLi;5VGQ<+KAqS3>CrQvt|e|7wh{Ih}I;O|WMO~`h`@6vbN@w>NOJN#z)w8!tf
zGiGf<w+OD4C+z||Z-htiAqq7Jo&6<ZiVudLj?|CTYZ3Agg=|2Cjf%*SX#UdpBBlul
z(k6sPaBV#4A4u^0j)*K5si&?-%n+b@@rZ2D9bC)m0Nt4?H2~=52<WCw2#A<XQopGj
z<{-hO-_*7Xy5Py)q7PlP($I)BbbgaZ<)G25ikOb>YSJ{g{nkdzqI*}B9fwm9bLsvE
zOFt?xK7xA?lm8>eED<B6B-v^nTNRNYfUTzH9}%?tQG<hiIf50Y$p)eR*9b1raMC3;
z8<8l0t;XyA2vtdJH65EGR3)+1?D;4nS?W>-*1tuh2;d;Lv^-)WL8A|V{f&rJ0W@Mc
zlMpdUqVee!$~(a)!Uj>(rXlds-iQM959k*a6NLhFywS8kizp(H&7}p$BkU5w8r(WQ
zq8LD(!>`{EQNobhX0B{P9&j^9uz{wnMZ|@hBl6HPl@j#Ii^vxMaWNr|D8;-hZLRc!
zlm7$fBT^A#S+vIK#0YLmq}>6LU8f={m0|-#St1r-4wQCBwUY>LN~m`7{3{VvA~J^v
zNL4MRIZ{c*<&ajCz?tjhQ6P5GGckr!9Lq$>AZCEdKw3`BKrqtW5M$J~muiSH+yl^x
zRLhAOEMSDT1ubXal9*`psb)jWVotmRa=pI~iW!Fd(pd!?d&CT9UnJvl^r)D40aTA=
zzsHOaz)sb1FlM9xcB;ef#v}+}kx$<jGfDu}Bk_kA?)<CUhjEseF#@O_D~896C6Jwk
z?yF)F2^y&$jj=J~1aMC0J3D5)M5C@4kH;hl@Ye>B?U-bos9NU8cyR`T-1okJ%oNC)
zeglyq$W%sd79>Le1$n4r%rpT`L8c3E3L+OT<%|S^$jwVRBY_|@BpUS`f@BMD3Nn*G
z1Oe|k1R}`NkukFbP!Pu(F|#F_Sgm^GN5|ZQ<~`+M1ii32W*&o}*y9&fX)#s-&LYnh
z;N&z<069IM6O%81MgHcZ7_6s3fJJ_7W=w$qC#Qu1oSYUBG?@b}Gp1-Ub^x;<i+tg#
zm|_X>4@^oPuf>!A46fy>ZR?$wQV9tM*!XtLd;x>BRG1a*i*aBDm3{*yzujz(siZuw
zAljLz7`5PM53{Z?MrppxuKE?kR3XijJx@awSGa}hRk;32tMjGhR=pYoaP=TD-=<mh
zMF3L+^~AD@0-(hb)E*HZK4aCFFbs<TjAgv4Ftxw5M|Z)Pr%WC0#LVOg|7g7qemg$Y
z3o1crDSCh@Z=1EmVmtw8)$5`6T2yFam7oSeVaTS_->mvdLE(wK(Pq`}Wg~iuCZM)z
ztG<d|_V7rK%~OZ-8M~>&|AOCF4wyBwRbPX+)0QG?<$9~m<1U;a0FC(Cs&5cPivi0z
ztG-DPt@z)}vFi5=qTRm#_g4KuLA=awebB0JZU-u3{ed<eu<8#1O<NiSbZVkie^_FY
z?M+Ww^(_qB7*2H9s^hFno3=ACgU^*%^+$=GG&I02saAa#pf=W&0r+H3tG-*oEI`Ls
zR(+3xIe?vmt@>Wc#WWZ2!KbYHLBMIJO}T)%|F-IfROkx8z+P7Uu!5@rvo2WmBMPnq
ztlDMOpHOfU;MK>i`f&v}1C~E+)t^%NZUxL3W7VHlq1ypXuUYjI3hp|A;@%OdKgXgs
z?IUu#pq~_UlW5&Q{dpoc3VRjx@FK`gd&4c7bUwA}r$J}hhwT`;ae`GpBV^mT0lGoh
zvkd*JozgMb5KU9-AGJIu-q}j$h@TW(Av#EWT5x8wPELD6uzK2p;U5aFa)o~-SPk}3
z2>(TJJ*Se%{|>Y2*Pv3<*P&7u4YumneXxJkK$WWVwCXnmQKj^oR{c#uRH>!V+M9x?
zQV)-?>Te06N*#O1s=qCWDmCgJtA0xmRcd7itNxB4UPiv=Z`I!w6iR0L`?^{6_rXkN
zk13es$657{&<V#hxba@def_QaZ3W!`Jx^QpzX~*&0Ru)_^^XM_9RY7NTJ=u^>Ry0X
zS6TH>Rj4=M8N;goO{Mh#%(q+h&jfmO27G>$RsURtb|+WmIpFFGa@B}XV}n)y5*rsW
z4W8s`5oG;FQT7D1jkW6k6sR*J^GU1zt%98Z2Y~1A6lLb?w#=&kOQ1XRU3Ac@e=pD+
z#C+5KTw~RLM7}d7p{Pc@ZPmH;qN--{4_5td0Bem^V|-!N|07|nn!RYXKMP{jyb)#9
ze-Xs0c@~oYR}ibF%NthxH$kkL$2VK`--RZtX5JL5p-EUM&?9%OhAxObLH!=9VF+SR
zFyfrmFbQG@_GU+`;Vvi|=z&<P(GGohW{;sj{g+ye_7WBk6t%!=m<5eu%u=htn-4J!
zi9kCL#;Xr84aq=nU9lRx`w-KR3e@X@)$kN#0qSwXYIq6C0Lt!fH985(0`l)=HGCxZ
z9H2eDt%k3J%?0Y|uo`}Xa)E;XY&ALyDgX*kup0h?ih;)cW;MD9asVAiegT3ifoe}!
zjX*)w8b^fP&t@1HgR)s>n>SgFZfY0-Lenz3vtwVZs{z62A&8cH@2s^NJq4Md2h+fH
zsPq_ap@287R->1|&S8L0Ct8i(0?oq{Y2|4(`hWp#u{p5-#yD0ZOi%}qy>QTK^cBR*
zTj-*VeuCIQdZ9x53mKnWz=3gABV55sH6|GYz<PG5GUShfGg>+gZ)#@OzqnZqPW-Z$
zfUw^ps}U)LH6UNe5+#Utp?nuwjc7rXa}9JOMv#eQ8ou+f8eGU_zl=crJ*&Z)q8iTn
zK<YR_RPcr_R%3W~GC>8mcCZ@pf;>s)t>d0ngBGRPKOwLjdCWi_s-5(_YBjP2kxX&4
z)tD(rE?Dh6Z?+mmNH~+XDlkhh>;SNgm05yOEFsQWf>A0#F5B6eTv%#R?qs0yZm89$
zWb9JViF(#*EWplm_D?JhOeu_okb_eSL{0o0S@Mi<<~B7+FgV{(yx!PlHEI;E%{}+b
z$0QeZ^3)%xp8o^BBDlSFPDhQrfGJT<=6!m}g2kHg64}!UFiAbt3@Q<F+6n9n0nRvq
zjo_>kR1v)F1l0uRoPg66<2*soeR?(A7bM^#oe$tzjf<-OAn2r6Fw_J=bBeLtnt26H
z1I^>YTRN)X75ILXXkUl~SCL@mHZ9Rc@T!0nd-N)z*MO#-#*GoIea>5r*M+nm3fOtN
z)p$d}FhKuIt8q=iaKOVT;p^Cs%gNLSNv4-7jU20S6R)XcsZ8I(G;QW%+8$kUeOr(Y
zDB0d3d|qZT%=R5Xt-q1z%=TRcQ=QqqC)pYq&TRiA*&12gahl=jJrJu)WpsJsMf5_~
zhlO@xUoop8uvqKFlcB(SuwF+%T&J$+_h%I$(#e2CA9gRZ4k3~c4@+kqQUmvGIMw~%
z_Dpm#&?ktAHez$dJ2$sOfOL;Iu=y^x&KsLIchG#f88hn(r^-Bo@=u-F!w>$m-siIB
z_>1}QvGu^mb%;(3Lc<R2=?xhotHkFR_!(y3+>;Gqb`q59mV3_<kc=F}$@kzJJxj3*
z;A`8&wd%reyXBb~Z!dxt?8@sPi%`Z}zah^90czzx?oJ-<Hq8dt8S~!F<Nng@y<9*H
z3eJ-aCOOAiyE{)dm>PqnB@`jj(;|i}aQv<Z<?*0kPLaw>j<uVZ*O6!Q<XBsbjeO?j
zsbg&}R(9C)<m9=LI|ff5&r=6;+*+G9I<Kqn$HmI*zvuBfgR<MtD$Ns9l18qvX6?xn
zQxe%&$ayGFOiARes(04r^^i2&lHc)MUQYousDOZ>QVszuqGMH|c{E9yn}Hw{kvfnN
zw(rSPmrSrHfZdh6enNo@t}SJGa?&ji84dd_FI++>L$5n|JdjX#;0}M6H;@@MQJ|i0
z=E-th*v^`lCsr!L_L4X9)N-AR@yf6A)R}!}fM`b^cjWM>U|_y&v6d%i7>#&17;ZVj
zXoU3;2Ae$Dk!$3go_)RYVnsNfxoz^#6C;(z9z0X(F>Va6lWJq(K0JC%F#2k7AJ{)8
zcp%;9p^IG}8KWO{zW|qM+!*~yl*>l6rrAj-sr(cE`BbL4UxW05UHPI+Qo(PQ<tt_4
z(sW>4z8C?^(zMr|d@7UL4T!Mk^O~o=pPDuA-F$C^Oz0^`H+^6C@tuzbc26XNg5ULZ
zK!<y8Vl7nk_brn`B<&@-Z@B=L_02@z3IU#Md3|5^twhV4`!yQOo-chDLNV{LNQ2=@
z<Q}vREyUzoB?SO)Q)w7{{9p3@4?`D8L@sE0Z-Vz&FW;YAB8Cn?UpL5Cf1|~#8@&3?
z@zt+2dl|>~VXd)lov(gVyb%8OKYaDKTEch1Yspz({p}X7?MRYw!&kp0UgvbPul_4K
zr+Iy!Mez^5q5p=FN^2qrcj&)k&sN`0u<Mke0pMms999auyZP$BpdjZhMC^Cg`s)Af
zz`Er5#y#J2c-^qd*D%D76VHV?zJ>{t8voBZ@jUjTui=JKvD3%?+?KX@`5Nue)A$5R
zd2B_e#21d{ZIf2si%Z{m_aWZE?Y_n{m~GG7#(1mGgO7v^v#-fYKHgV+m~s^#K;aHw
zliCV*W5+S?IR01tUuSL6_>^qT*JOiU%wzv&Dp`gM9v<s!k}VB6y)CJMSMI02CfV3f
zUe)k=CJ|m_O?g$;V=kZR>z2X}o?gcB&1IN|Kdk%PI)F3l=TMpjyKEk0%&E7wjs(&%
zO-;3VigvPNvd*wczt_Z0rq3apw}h~PPU~Qkjdf{l_eR=$gaBLHp20R>iN*)jDn{D;
z1oQ)V>wj#W1yH-^y=(IqZsMAIQR|OVz8`UqazSU^%@&C8{B*>f>0uN5n?|;T)wgV_
z7v$DZ%sE>KVyVsA*L`fN7i1gR^KYB#1u2L3r#3ODlMXiO1)CVu$#c&^(`__Y(D$>0
zeb~p=OXOf@IrN}Sj%XVL00wTcNslN4fVbJEdPMd#1?z08M+^Z-J88pI3hb~TUe;`?
zN9+zTWT8#<h%A-L@wR~yji+9*oox{ULII3fHnoA-3n1=Co9YpJ1FU||raHks05?Cd
zMN676fVBBGbyKV_fN6kD9ptjWyFY4E=Mwz^qA%NqN}6zh6P;~5&BvoI02g9y>NtNO
z!0U5u>Nr0FV1IiXk5jd#K>%^T+msC-_jQMNu!#*{Bez{2?qC}&<rW8EWZA|DX>Rgn
zCfd~gIUi<Oeaxoz&qo5>eA}k>&l3RB3T)#g%_sm<Z(EXp(E!~aw(&lhRx}16`m8NQ
zz*vA2p0)`B5&<p@vP~2)4xsp!O?K9s#sfglCLtt0eL}OI<=-QEcGiW}`&a!rI0W|`
zC!3IC_v$6(T2R3#q<lNDQY#qEp8sv?U-#z<#-RD)xDjB_*n+W?d@Ot7(!2sW#2HIn
znlZP4$HMu4=l<>B8wETT_O*%aine$ulQ1xWp4{@z{}aF5@P@Zr!DO&u^UFcw`@VqJ
za&6gg?|-3yw{mUxcn8+WNxrKK*vZ(kLA#Gn0s9!6ywtql!2)(Mw!b7J<mA+Xd~{E?
zuUAh;2HpiWux<OVzMA>j{P-Z_PG0D=&4FxD!QBd`lH|3(0-IKlAwp5y`=Se`Q}(YY
zZokF?K9`pN6~!I$LqV2A^9;1XRG=VRfKOoYVy$4NxcvjG7ia}J;trDHhS0XZNb#$+
z8}ZkIJ4yIrU}gXDyWl7^Ft1#ewlca+Tjg26)6RlKmFF&@jZa`8&)wp7lRWo`Ti(Bl
zFYqroz$6<3%OTl8XNvMFt>BO|MZv-nt>7`0B(S_pD>y7}TpEY_adG>w;5)3F8D4M{
zv<j(nZaWH|L|9>>mR+%^S}Qn4Fs)r83fm{C;5bpfCz1Q0f~Q4&IuVucEjR&Gkk~01
zD6+8N8H_`PHJx&RX8c_6EYZ47>wtWc3!X#$6>jfz7YORW8Ft}QK6kUBn+4BPYiehj
z5wuGyctN<F849#&f58h}uaBJ>)?DU=)1(ab2VEZd%A>*MQ6sgW?gNjdFZUV>*NLyZ
zx^BZnw{Uw4<_2`(Qwqzkq@s=$zWj$+H=N-Z=--}QV7w8)I34Rs>{@||H)jf8Mk+Rl
z!n+c)y@r@enhH#P$io*XEn}yDp%)6iC<_IDtF(|Of<-bMxDP7qB!x@;T(!SY1|n&W
zyR<@IDHTtAg`-j{^b;VH^gcmuh1?Y=YDA)t$U@b#Qr+8cEL1%!Pa)1dRH%AZ4mH=Z
z3RTZ)V*4q|+KTpb|Lfp8s;E8KEjkyCUGNP&ku6-#6WPa_#0KPwUgX7zO%dbomgQ}c
zo$-G}oT|}`UzNf5RW_dBRxOqrt|KZuEgY=vY{wbZ78$JNK<DZ;iwxFcZnZYu;z_B*
zQWkG(%3v)Amm?Z1>J(gzA|9@<w$NO<<RZx7t<Q8>{83)K+o=GH56^oFaDKhO33%R*
zS3ns8n+t%)z7t^fW7l?Xq>3gN4th>jA@Wi}{tjnID5Bl+uuLJR@)Fp_i<YSZI20xS
zY?0okNR2U;X(G^Igq+E=OebjMd|~Xn7M?b1O&opR{?Q^`PZ39-D<4~CNSa{)Bkx$G
zA1dNdG`xdFI-(*z7}n)pi}XZI9G4vamRUlL1J0o|3s0Zb!v_6FSe*7L2+5ygnJXcw
z01Y0NdjwbjvJPA33CI9g6lk#upmsjl*&=hkq8xz2!4{eF70m*ed(tAt4@I*9-1b^*
zOj9%mVCxD?fq=OH0bg1Q1>6IWZMMkOiK2M`!#i8-5`x`NZQgelxjNC93vgzVrGz1k
z`2Zt_TS^5K0C)|w%ok7uaCVf%Aq0v6iZU&-q$nx@7+YnLB}GvwK(~b!SyB|u2T0Ag
zR7x5L!0_&t1p>+dvie%&gsZ3=z^Jy!*=|t<z|wL{HKR3Ea%~x06=qoi?JG?|5VWth
zwe~6C)II^t+NXe1`v_dxN8r*viPo%r64I=FDkPLFWPcoHSwR-wRV>`wngs=%EC^_3
zK>;TV1TGc`Tr5bmW)>u*nFSTX2JmcO%PJH}>AR{^Zi4f(Vib|wu!(##%(4;e&97GM
z-QSu$1)S^&Xl73VCwl}g_6S_;NwjA6B&3-=72;&?0kAi}TCs=yocXI<?Cm+4XlW#S
zUn};uOWkS!=sq45K=$_BW7#2qvxB7dmYo7T0WKf5$Ve^rBv1|+s^!wjvnwndt2rl&
z!?xJ6Ar_qzQ^!H{{7@BrxGD)CWaUI$&Y`PPsEC`cc(K@txV4PVN)So|2)&gc6bBHx
zD;2YerwZt=RG7h21$0=>Fz%4oceLp7G!}0R1KPFIqK_bI>IZc6utnz!SK94?WP4s8
zi$&)Iw(Ma9lI`R+Y%A2vwjdYV3N^D$)WSAV3)>RY#kPdG*j8c9Z1d3t#WuzSov$uv
zjbwZH4;EeS0Nf4-n*FUsSC;^yfzDsF=)47hw`_nYuDS-02t;w^9>DEn&J6fGL)ift
zsbzE-^_^Qf&6dmlgsOgLZMxcUQtFYeHk>$`y4^?*rAA=PMT<P!J}5m5iON?Z=-SEj
zzF{QFL`$mE)ryKbbu}tot*E%+w4!Ue+@_EXrwgak)ryMR7X5p=T2ZkvOe;_4ipuQA
z!D_?T>0D8Hf5%npHe0${QTYQrHz1v6vg%tEk4;ZklYjt(l+8|8lYl_sv`a`jS56h=
z2b>Obak}|tx>`6fr09uswQwS*SBIw44q9#axPO{XBP303mgatxPAep350&_SdV++|
z;^4)Z>B<U;FVyUPDP3711><dDoOa%i^ilTlm^f5Sz%*g&NlqNzy2Xiu_ph8dbR2EM
zZ|NsA{W_RI)32p#Y5JABUmo_IeNT{`^ZSaYL43yp_Gws-D9hM+@7ifdrEG{Qp0a0R
z?pSe>5)I0;XE8(!Jl6QzX&$Oxidr(to-NTh0~qoj`%DS(=gvsQ1k8>qUb?NLj)Sb-
z?6Z++B~6}@DGjq+ncgFSGb>~|4**Z6sZ6axjn#@wa|N&y8Q#I3CxDrP1KN#aMFjv3
zY!V{3;>Lf|&0fS@Y5&|e!Y&3LGFODCH1JTOj<nj9fd_N#`Kz4<9{qliCBx>$W_vj)
zaJmG(D<u(i6BHJ>D1i1t0i*!#WLHge^S~$eYJ^l?RLAH=aK2THjHR>RyvEJW;cx+W
zCP1UMHT60d_1Y_TSRo^po6ONcV?h@ejjflv*_V^XM#bTZ)-+bSXrN^8MT2!!=PHIY
zSLbR0EZL#;_WJ}lOLmQ<ahB{_2?=7|F6e?v-TH;uzJXa^QS5JGcysaIFMySb;(b6;
zx@mj$;=B^g$NmuZ-AEKAx}|lZt*%7icUx<I8-)Tj;n+C)BLd_K)`Ba0_39GMY~KNr
zc13bmYm&QNBvHV71UM^cuK*T(p~=1vV4>YrNedUN0tWF4&CPxgfkzP$Bp#DgP7OFL
zfI_+-xAT6&!lSM_TKJNyj(Y43wR5pp<<#V3Ox8piLs!_33vjC5Qvgssgv<}OKP@5L
zK<c;BegdG%9NMe_RXBHXYSF=OJnZk0x<$<K?SEo2sfO2{wZAVk$l>d6+dmK>m*gTk
zdf5NML@QOIk6I_X?Mf8=FZ*9x5_O2}ZU2;sn#6M4{<jdg11SfZ=Vua+9W9#B=K%eF
zk%xqAyL;HbV4~Y9(U(kQVIsu)ABiXLFjeOS*uO!%>QEK$pDyCJ`q{sA5#Ro-{W}5N
zh;-j%|CazicF%)u2H5{g3aLt_Ul}9?v9#R&8$iEb4DGMzL#I64-i@Vam#=tMZ$am1
z{)OG>>Mz-io;|~Ev@Dq2=)`vHMnfmD8?C*U-RKst_V|6_6FDetJRN4&xg4#&mdN*H
z?Yg%T4`b!}4R*bgaL36<<Lh?aCzRsJ+4PxbJK1%PdW+nh3H!E5*w2-4?0@Wff1)Bu
z_y9gKu5*N2G{%`QqD{g<u7qCG?D}A!elsQETlWXsbq;2WmN*l}v`IL`mGG@yc0CrT
z>qC+-zKg$IU&e$doC)ihFi>;KxLk&E%ILPruFG9eC=ez?=h8jxVl%erLubNOu7q>v
z+x68dA)`L=R)Ae!i#!(daaWeyItgSR4>Z~J^$23^A?kH&s9m1yTpZ)1dcTq*aUmY*
zwCfLG6L-;v9*kQ5cn7=w5UJLXyBl`>VNz`()s<GezD4rjY^>)qc6}=%VweWHxyi0?
zld$$AV?GmO*SC|*b!U!`y5w*}wW_Hu3*gN!?Yf%ldIQxTv+KJMb8!qG{tR`q>$`=w
z5E31i6k^x+f#{OnPNMr=MA4e`1Hx%DQQ50%Y9@NfMHEd)r}_Po-mYf?m&6o9jwt4_
zZBRG6euTkmoOwLq%7cs_ZIcJLu<*>ilksD&Jg$6h*N=mURwRr^od@G5NVK^$o@qn$
z*)~L<YeV#;B1+y;uA>@X0GXO-CvPvdA@fojGN;;*Iql+Y^Ci1}M#A_Bi3P{)`dQ($
zC$o6zc_`I+X3?zi7hGA)-(lA;y0SQS$gW><WpM;;`m!KDl8GooQ@u(uFFQGXm1Hc`
zqP)R&{WZyvHASMYOTvyogQ0J4NHVpfiE+E;dU12zgcS6#>o*Ww+bsW^F6LM_H(kt8
zsoruiN2Pk(CI7<BcKw!wQT{cNcKscYG#es+#E&Ms{wHQpg1`|k*!A}rwULaEon_ZQ
zKoB;JfWl7M^*>8Q*Jl-LYl@p$UHn(5?neyfT8vrVmMlqU&2YQ^*PaZLfv=X`<g>1n
z^RF3p{a;M^5#o}A?_C@mj<@S~BuH5lqIhb*cBRZdjy(U3;5wSdGS8n}c~a@`_F$08
z^I16ixl&I3*=#oqrnHhjf4gCFrNnS+xCxR*|Et|_mx!)$zpmUx=Y<Jwc7w;eb&rA$
zhGWBdz6*r$*zk}b*Kl8V#FcXTAf)Vq4z%tT;*S2#ZpahUQtnx=*$qm>wGUNvRX4lA
z@nh+9crMswH@pD`*Rp6|J7hQHWUYzI)h<9j2-Ava66aB6H+%(k0kS=8H~a)KqcK0&
z4cZ-OMa<~*bi3g%B6FF$WTM>&!m+~A>2iCn?g*Uu&MQq|$NfIA(+g>r?nLB1lk7$x
zSJ|-y!rGKwUm(ttk#OZZp>~63TuZ-mCLGu%VT3Co=FNsW>Nh7=7UgMnV=%)S*?v&5
zMx+q+)F$}*^~O-WjE6$IcKU^3`&&(!jvn^w58!)+Q>Mrl{02%0r%Z1i^&2c9oXLh=
z_KOzaqov>&bc|mN4#SszC#Rsx1|f2Z-&TYdos%<G2Yhxd^xK9?Q&*5mis92c%dfEy
zc;IzdQ%M1qGqp8kFrLrQEB$ss*k#YiRK;&M+GAZm_1zD@Jrdy3sAU&(K{HCXbB40h
z*VFGYi~;q&;P8QbKRI2H8Oqc9{N#8+_HW<z^*h2sk^S4?cm1Bg3d{d<$~z;+?<jbz
z$6^z+@MRbIP*DA#1m}-D@aKDlyVBYMgxB*4yL#SCt_OwsH0PE&LlPj~PG7$#Q7H9k
zER@x`17NK>%FlFxWuOj_b8&!d+UsX(xq)`P7GAHs;^)4G6DWC4?Rd>%^yW2w?OtU<
z8VAk%r+;VKCDvcSB(*awUATLaihyrB2Qb4U`8d+}dp{p}f$(@W(f9p)m3<FykMtcI
zTAYbuT+VI6Js%az9R#`3wXMEb?jXn;YOjS9%N+#SOX?F*EO!v(7W?$2#p(`1fLtRg
z<{gCYt7T1?@Y<&0xunhgzvoXD-$SltF23oJ;&}pC&{IArmg7HJB9)&lmTL#{HgeLw
zV!3u8xhDL*Sgsv3sT)zng&?r}U9RD$4lA|`wG{yQLyOhrg4F;E-z%22XVbdo$^7!q
zrDjA$7TsD=>;QiaParB}FC!6=J)*I=TmW?)1yLb@3$+uE70bd@zOsc<k%ehfC_<K&
z7t6x5DNL@>7IR@bxHcRig+;|&m^NRKDP9C%_8TEH=i<c@G7fL#RB6Ra1dK<TOM8oJ
z1klP#f2dg2t3}BGn@$#Uz1r|ZO0!H2FNsVesrFOac^21Grhg)8?-j+%(RnxglS_ax
z{fZkfdNkZ&nUB9yyn^7aJhinv1<Jepb87IHpt0lq#Nt(;vBH9wyG|9ahM&5KdcULi
zKG^_>MhIA3gIIV7T<#JTb55xh@jSrNP`nP*S6JfNJ{n%)X37H(AT%mrA1u*U4#Utj
z<`P!{nZI%s;5U@3fb}nM6>!R%tALHZTm__Nausm&J+1<#Tg>>qR@?!<#}|0ux8YDn
z{Py4LiQn_<z3@9|tv7!2H*~^pk-HCmw{`W!@AN=F{0>{+8Na(X_~ZA9^e*^q8Xth)
zc8);&4$lk1Z|~@?`0e{uFn*`53c>HR)7|mgbgU<SJ>Lw)?~@Pr!f)W(-uNB-uRi$g
zwjqosm8<jh(1gFB6joxy(N^0NK7u{V%0?}_aB;Pka2p_DqOMIxnKmZ;6$oEa@&QUa
zo$#?j{@J!FE#VUdf)M0+f5N9gzBhFlLlP!}-eIY1v0U0!Xe4~j{B9t>=$wSVOAWBH
z7j91YhX9>QvGq4g!v6r?`-D88knj~~ss|~emnQsEvf~y(>AHk(1+Y=APfqww02jrh
zwkG_Gpooh8?28HC1FSsU3t)R#!W{v$kk~mU;os<=?|mYi@8%`^ME6sJ0xK7433us!
zE*kHmYYG3M`^BO7;IWqQGu>z7k=Xyw3BM9t<mpgMw}jsat|kKXj7j)iKr#ldX|ob^
zjp$k`l`DQbeny1w=SBRC>_XSibp07WD``HQu%3pdFU-|PCFtl9S0#YTn1}@3jeHN&
zGSC>OCFrz08q!^?NsC7&=<NYfKZ6i@z&$}XD;T{;=ZjehdIxaeYtx4!xbfWt-Gjkn
zb@{rN|L4d<_5YoCq#6{Mpwr}h6+d#wS4CD8J8KYw!>XL;z}p)o33?DhR~0Azj(>>8
ze+XbzY9{*qU;mNZ%E$!07s!pJHG2ygxZS|dwe|`6BVd2%x<Sx)eNdygWw6doML9K;
zbcI!xwwT=g3=gl;5_$e>INUxjl!OEzFbeL1(vohVznb6uoWH`eWB@^4pu?t>L}1#3
zljT51rB*UX+<0{|Un?0bZh0MZ{>qX_2)8;4sh`w*O5&J$8+_)kDT&8qVRas5H+3uF
z{{L!B7EqC^J0jaB^^#Gj#nt?Z-?e9g;)BgFm*A$$f?e?;9LZ&Xd7&u2n*erpGs5D#
z3t(^l#-8{d0yuMdIwoGMl4O9f&xn_;a=exf+DW&A;`@>|k1?Zj;`=d?^R9EeSYXP)
z_|DpR*&T0^Bkue2&GCbo$RVA3JZD&nq4gW$qXf7GmgQAy@zHqf<vt8PsHXdn!F@}M
z5mR<5pX%fpA5R*uD;mlch@}A<BNYvRSAU34Adq9JdD-!!1aK$bs>kzfh5KHYSG;*{
zaNh?m>b_Hb;>Uyf8gE5CNeDWrC%dToPLG!xw6|qzHet0-{A4CdQi;-9C$hK_nLm$D
zCs2vne`}7P&O|ke^-Ncymxsp7L9)F0)NWKf50cl^xTbMy)+%l<LX{Iv#LvPAwq_ga
ze@znfC!|aK0kpO?yz+J5@CE2MK8Qc)#lQcJKiD1f;}!RX#_P0YTFc8P6Sv0eN-WOv
z-z<yQMXn|;N{)7n*Cz;9JkBcpK3<<Fh_;G>mGOG2Ag&1F_Q&g!1o_Kdh<JT65T`lF
zFy;QBc%9q)Yac{ZW++GNx7Dk?@w%L?H!%w+i$2|zMXx*Yx+;|BEL5p*(J}T~ygmb(
zzxF{f;bA4N-J^;HImW%VD_+lGj&~6EmV3ND3#~G85JmL*G+v*L2E6vpCZNGL;`KQ)
zVs8dokCK@y`ELbUS`NfMscAb)ckLih_IbHmy#6;(UN;9Z$nR$opOhc{EMEUy5DSk*
z@^?X$?D3KD`ag(nd&*nI@%k4MMp+uJ$Ln7TlHy8sLl)mN3+@y()W+*~dJ%|C``Wv3
zD*G3i#twO}bRK)l#i^(uTnK_Wmd1`~-H_5;_EPMK&b(2Yht{O^N2|SWoKGpIKkI&m
z($(Bw+JJIdw~y-W5nQ@L0DFWEmr7R(V2?2G-O_snh%N8#ex<9?F|MP1{_FssQqF4D
zrv{cUD$`2WyQr<-P`aTtwT&)nogXdT)Iu#HwDf^k5NKEGQ+g2H*80Uc7%_e>eXI|8
zg!`F0rH2PH2Yel`^g50n*Y5yJG1E%lgpw=H4|OZONwC#To6xTGZN~W?al(HseFw#{
zZXKoDlvDby0G|1+{H63g0qhn#e_Q$|0kp60{$lC-0$2#6x0ik(z#WhEl)eG(HnhXe
z!1~4f*}*>WnV6c=&5(cN5=38GUb@94|6#w<Mwk4aA*GMF<X@Csx<kp2G}q6S?iRqQ
z^U!Zf_X*%!!)tiyegQ#&i?q_mDO46`8WWe79u@yV04I8sJ}H3HO;c>?Qv!wpEY2x?
zS^#Gb{RfpkD}XbH4gE`>7mx^0Tv+;|fMkG*fYQ?fQUMlpE<GcFb6oEurDqAYW&lhH
zFMV0Sbbx-pm!1=l1u!SB^t^y<fOdOJFA^Nf0a#UCdWm3PF2FM{m0lK50I;;W^ooFD
zfGPheeWeSl%mGmTMd|AT%IvIBG1};G*r)Ud7Whku;NHIUCrQc*p8Jnd-tJqsk2Qy4
z{Y8LT)?B5(36M9=W*pl*zY{5Zf_S&w=ldWC9RtAN8}ogoC)mVQdjB5tN1@R>SKxEu
z6aKgPqtMkleR5J^0(GCGPha>H!F{XJ5hm^lHb*oX^8;)-_joym(Nl)piTxZSB?dWs
z@Umm90G`XvGdYq4@BpZ2o@0^#8sbkJ@0cQh)aFz;G6irDyEfC2Lm>9TuRQ6vM*zp{
z$^b{c0G>)rs&y0#pn|#o;3yZsQ|HZgM>T;MJly=mQ7a$_Az4p5RtliLr9?W`NSaW9
zQx7@rmyq576VEsv7SIo1;AqFA0tNy+c*?Oy0B11v&m4~l7y?jI<~S-K4q)T=j;9G4
zX?eT#amPsk)KQzs@sa@A*Df!2yeuFQ;CPthk^rs&x4!0hRRFd3>IKJj0TTc^20Gpr
zKvninb^M8-X)-{+qmEAmOabUW)A4tLBC2vwZ^u^>k_m7;#qphhnE<m+IDQmBt0=?Y
z@v{Icw-M%k>-Ys7!=~=baSdP{>-{Bk+SlS8`X-6QapAcnhyIWtH=x0l4!uzj$AzlC
z4t*z4BL~&Q`yKi|LEQSjn&{9~(Q*8{;qK5?*>TK>{lTG2;fd)*=w*j4rPoA@k<b?$
zx)fhiDA182hc4yU6h`C7N5?w!bKv*^|K)BD*;;<!7sLtzA6F!2EpgJq1)4*Dg-CW}
zuXsB2t3aFFO{7o<ab6{bSC)gNKSAaP)0`CEZ$se&MIkY{ED!YlEXahIr1v4|;fW{E
z%Ut5n{{ngsnn~|}NN=Z;p43NUGe=*yq4$l8Ud0`U{!c+dZ^k%>+#r51hZ6okdLK`L
z#SPe2wJ6`ryE((5|C@=_H}lY(^q&OwFv!&7cOCj&GIeD60Ehl7h-@AMrrtj2(0@a;
z&0|=xCr!BeK{SOGRPhUXEuaT?@>9@>-5rKbf<<9K!|pl^gJB={<&5@$UzmvFHok%l
z{9qcejN?&~`+keVFfsRU_F=qo7-r_bCUJV+5`Ye%6Fm=IJo?x_ISdcP+dO}t?$*U2
zTeJ`LKz>U{JB%*O9fik7EF4BKQ4=@VCvS8ZJq7W6$#KwONXOO0UC(~;4r73%qXMS*
zJB&y{lyA|u4kK0&kC0v+<uIgglRe-okdK|)L-@8k<k`Fi&Pb6@C!h0D97Y29+_Kzo
z7@QD4v=3F{)x%+=3q`rE9u1Px+cmoJy85nL4r8i>>Sn;o&m2aEf*k-aCpnDS0u2wq
z##o0jN0IVmj;kFGV=i)hC}BDBrnTc1Hn*i;IgCPy)?6;7M6PnF5Y${Qs()m;96IeV
zY8h6<ayj&(!&ok;6VOxh9L6f5MwXK4Nr$mc5KF0IhQp8^vyqK99VN&v^Pzp*8()lC
z*)B1;8=g?&F!l%v25KMYFb)c0gFg3~!#E;{b^8#|aUyBnFJE*Rs_9eDW({)~rzI>L
z=;41mjEjOe)1SB0VZ0_N8tC%t4&!yAqM<<B2RMv3By1Rv{{s%=nxGLtBj-De>w-oT
zRXU6tg2u6&UwG4DyvcGt&WSQ6MqBzywKr({^w3`&ru``St+XTS+rwcxz;tr-@Hmi?
zj~aRXs>5_hXtIMmJ=kGV-5zVK=WK`Ru!OOLeEczoiJjo)c`WNOzd1}#3Qh94b(h0*
zOb|CP(0Q6vDUs_Az7Er~2;16WTn2LRa+scD%%;#U0f#^5FsUL8{RVK&afj)775Xng
z^t+}e1%Cvb{-nd?>|FmtRevZFRJ)4I?O7oEK6aQ+qd>Ow-Gb%Wg8`$uBdl|dkL({r
z)305pQ9ZGx@PJwTj-4-uAofuMV869>Hf9_TyabQ%cSa4Q$9#Bf?fNlNtw|dt$LWo~
z|AL9<rWBtto(FCH5hK!qU1iSWAb`4~WgaMV^;Y}9xU!CjrPjblJC}J1;Pj!_oiZNv
zXhk%VIO12vqaLlOBSA<R=SS*w$9eCT@t~*sRxeI>^S>Wd)&+3eF~mcza#vL5HpnQD
zxyog&=ZBXC2?b7PkdeA8%IVDU`DMXaw5d-o1$|u>0<i5ErzRoamZ|*+&N<GHFO&U<
zM$S39cPo?qh(^vihCEuP_9Mao+*8Vw>0UU%(~)I7EYgZN0Rh{+1w^olxH#N)MyjYI
z#c1^MFN;8MV_!slXKmRaiK+oidA^Lh6YAB_L*}wb0bC{Q=vNjcfUAVf|5L^(o+{(x
zZe=k7I7LIoLjdrOIWv0r5p&rvq-vyTH8R3U31~B;n6YKbERV~PsflGgl$eRWABj#+
zG?&r*u92JWNHofo2;7Ypz(n9~i~w@Cw7g7RXd-vdoG43F+>u;Bl(~$H{YN<G1G!`u
zxdpe%Qe5PsdzZ;dU%lD&W!Xdt;Y@YZ-^)@3aHiUMZrLOOtlEU8GH!anf{+=#{0~!F
zJHSUco7?h4T6+niu$OwIaRAdAWv_MZYd&dI%174AmVTPjNez#PFV@nOPEx+QKc{&I
zFc#&jNKaEb$tmr;chl72qa0;Ff(m}*QK?+ldGRAp2$t=gM@Az2#bRV1rR76_iCqHx
z*%ptzh%J%MA9gA3ianFQow{7=4?%u}JLQosm$3}ciWvLrE?49z=-5f3yIo#U=)@(o
zNdH&6=wTpbD#<ECyT{ITc{?9y$_wCe;@0x^QU{d(&9ZWlU$%2*l$EQ-N_CmFwOln;
z&fCx2E%%f(Y!6A{<#KI8X6KG$<=zs)*?HNo<?1AsEoEYUxvzwT0u(+{?k9ka<H)1s
zodvLQRGcbT=dObQx_2m74L2I#k@9kBxbkA)&A9R)NyA0so3EF56~GDkmZ{}x?=TTy
zZijN!Rypln7*wwI4pRZDg38t2p#`9`Te;dU%mAp?%6kblt`n;Vm-iOHrQsr<a&@pf
z7hu%-a-QUBO}PO5UMiPO#ijy)-doCLQ?aQSVDX~zaEaysc=?0!fdVQ4mV}n8^WAEK
z!R2zi+f)Nk6Iw2(yG_dg&TTJOyNxRV_6;sq&3QGW?I~Byc^yFQjdC_;t!Wd$^4H6U
z3fK&A<+<`<0=5DS-di3gU^~Fl$IFKc*aa|lS-IMt+y@Y5l&kH@LjW&SmCN>I(-8pM
zYvr;%*>nsb>z(p3Lf`~I+WGRa0!{)f$}d+f`$d5A#PV?xat5GeZ#g$Acf4>O;OZCU
zY7g%&K+>3UwT1Uzc7<4pYCEn<>+s_8W#7c|i5RT5`%V~L!Nbv=Zb)*=y@KboJKZ#Z
zz{-k19N21dL1l`)A_yw36^&}{S9kggvEkZ;MHQ^zom@%zI4XL(2;9;u`Ur?eNZkI4
zFrmh}TEC$}-dZWD)+QuZus(LOXpw!iD>V|txU!GVsTd+)6rv4`tB4hls7<J?V2SPg
z5Z=gptSdG0PIToxxUgcJEAP1772^d^Pryi$h*hJNF01f#$K1154wgzQE4rd5*l`Q)
zz~?LY5vZMfsJTa41>XbSIUVlf=@ooTcjsojSXQ7_O!Q{j=EB*<mpQelrR*MW{CPzM
z%`|qgfF{qb=m>^U8`#>|`JtPp9?`5wYljFm(b%NP=CR+~cnJnak)AyE+c?J+YdT{+
z48P2m87t?M!JaK<YCtG^CsRoV(VPJJN~WnKmB=0trh2YpDVUVJ8_*9jIuF(p;d66Y
zKzNJKC`>@MPYReBifq2d1SEQU9J<IwTLbhe2(Wt~=2N%a1N3Ti&uU9GX?TFXi1Li(
zmg>1RKrGv_o4Nbj&CNeR40?9k5ICYCK>v_YZ*u`PFfKs<iy%(*-dP)<e<Z~sUq|0|
zU%LR=YTbR*8CAAgW&YT=G(hLUo?4&|F$d`CXphUw7eWHmR%@etIepH@i2;U%jQ<OP
z>o)`#=@@KL^gv+TkPikla^gAf-2g+r7|_TCKrj$L8L-pMtnn6X{A_a=%~kgBqhj60
zhjCVeZ|9;i{}vVuuJOqYY>So_hshzgTyZ~kga^A}N0D7rU>Is`PZsLS+MNfxiwCmq
zs=Kg;7oC-%%l~u4C=it~Nb6Q)U<5R0PZr0k)mf;=F6O`(Rge3K`~!z5w0!|kXke^6
z{|*Wq>W(@sw*?Mko!*wwZZil?{%?37XB~UH%U~NAkF5A9=KQ>cTA*}OW7+)IZwQob
zN{(*=D+8sQl5fME?-MB9)Yu?IduMInXzaS~8Q2Zr*o%Q<u-)A0<522)^mWfbUZdWd
zf}m`#KsiH^qoE}^fys#H6DY?fmdNlxPBiw;4+ICP2-rKHeNNIp11AY@VQfizyW{_*
z&SJ@jfa!rn9#SM3OCjG_gziqDB=`YB!3}i;E<i)v`?QG8NB#D$N6n@G68M}a<De#b
zJ|3vIYX+u;fqFLq$c}IQ?p5DE^`c?R-aiijM^6Uo8GRT)lB?oD!NWUH7YhWE+tDS^
z=z;Kwzg^aQp<BHJDQ5nCbZ;NJDbj@>M)#5ahJCU<ZnWwJ|2rKjaA7oi!Tly3=iqrY
z+UR}~PrsR6Xd^-^!yd&u*@Xo}SU3Q^pIw+w6e{R1*@ab=Kz#*$j4+~sf<8l72~d=v
zFF=K8sGzU23riStgrM)hOIKrbzcGS-0D3EFbboMW_S5gO9=N*OufBi!lMAcwpT4PG
z)=^3a(v;W(k?KGi#fGF2A`T@*84Oag@r=Lv!_k8oZ;tK;)c4ZpNQT|kE3+4pD}F`}
zs3v=1%rGEsyx!JV01X8~Ndv6|iUkT6v>9j!&|pE^fntD$2-*h}4K!TPk?e&d0C0;Q
zL{4Ncj0Z>*@M88tmQR%8<b3u*mR%HU;I{s1_Cl6x6l>r%YG)zKJStSs`v^mQM6m{L
z>mP#(5o_SK{zdje)?*ZF;I{r<_CnTi6l>siOSvAHBl`VIbGaUvFL=Io1#{uip=iek
z9;_;fg?#)P=Du!KC2=SRzGT0@Gc7B6FGe(8YC3QTBg%pKn)V5L9?xtZZjT-c|AT?3
zjfJm{9tL3!Jjn7taCUT@0E1|2&}cPlaU*KKWHe7$4+griZuZ|lT6%}uD;RdF|LBnn
z3twT*wpA{|H!p<BZ*n|E9XS}D3*n-cx`a76LOkk!(p~{OZ?64Pni}2pabNaE+_4yN
z4#ZV<VRzNE0Mp|0l>xjyed%AJm0U?4yaDDH?5gaF?)cyh>eaxwN}g+DI|iWFoyrga
zod7a=SE{+C4?xK5N;S9i2bg%gQq3UQdtK>Osb-L3H^LR-!P`>gUQz{%fxi{5eTQmS
z!Rgb>7a-Na+rGn+Gn2E1K|%D5<8v~SFg)kbN0|{!nv!J+GIt5}ndCNCo3kQ$Jl+-T
zXt?QfR(2FOe#V>q2PKc6F;U-MoZFb2TfH_{$8mT50iSL8d6lcy<>qUQv?#yQ!-Ec=
z-qy_V);afvWF$?BHd|9CC3}v$>0vTk$Kjs7**arVvhFjbOC%#njHKx)nZD!Odz7#9
z(hQcfrir<PK2g8xdDl}n!(aC_(f=m|-}Q81FR`n*vlhLu3P%^G{r)>tO#T^~grF$o
z-^aiIehyX6f9HGDYUZTraYKD$J)$>6v?N!-EAU5*GZb|F(!W;oALn6Bnx3g`Ambi$
zGcsd`C}(a)O0%;#!xB5BG9x<E%3wHBtPHGNSS?QYL(LgypXZj0m~5+dh{PTwOCH`C
znQ=pvGaw_S+1W3{5;wFmBZd^kL5kwASQ)gM1eiYeP_9q9Sye%bIW;3CtGNzz)B#E#
zwVaxfm9C*IZ~oR%bDOi$hq!Zg%9P~LjOoeNQ2#h<X4W{Zl0LbW^vbKt_3+8v=lRf<
zTy0Bk$KQWHr0w&5XiJ{9HP2J~<VW{I+98ilW)_RDuZQ;<&WJ!<gOy{g=9AdYRoQdy
zO>)nkIL<1{00s;&10_f@G!O96kxrBF;SUL7lC?t`9Ye&?)tVWbjBK|sf@fTarXTV!
z?Kd6LeEPPc-!&sACRUL*p^QO3CYI!X_nF|{D%vEsNtrEdNW7uZNrp!*`RF$<l2n)y
zC)QC~drL;j6kUtgOiAPDFeQ!0AM=D{{Q6izqD;-YOqz4<9XvfN2@)#B?+BG=u>^T^
znW$H4m70&yTZ3vZ)D~(+yIO6|y~ESSC%dAXO#U6D@LlC833IhvO-Cxc4uZefR${{b
z=~HL85_anR$Ao4N_uiThBTjakkeuXdirD%HQuRv#a`&}EogUKmySZ`<@d%oTH%jqJ
zsoO;TC(}<T$KHv!O)Hs>v1DYqGR3V`Cn=WD|AUl|hu1{C##Cc6n|rU2^eJvhQ`2Lb
zOUGST9YQl1mhb;Y=B^2+8)nQ;DAELy8S)1*9)^n!D(HWuV_-sK#%3E-bCS75C!mO#
zS$aq9DGxK61VIw?aYiV9l0)aL$V{1xp3(f2C@MkH_~?|$q5hpRl2Rt?m9=`YMi2kE
zjHD?OP|vvs25rmr=(#9&pYKxAsnv7Q)hA8K%GLMz(I>gZ$HU|^c^{R?XJR!%P*oYz
zv;5MkR`_*vLu1n{Q__+(<<{XIr}=r(-@^bH-waFuNt&ORo9FMpqt64v+rdu6REX%U
z!XzD{HbbTaeTj=9L&bF>rfiOiX2cv3RZ;>nCAg8e=%H0yCt}Lxs3w&XQ6(iHQ-ULj
zi*7>2bt0#X64gzmY-S#?wVW7!k+PWscdfOY?kZ(7^MI}8#0<C<2kj(fbJTWi<pgXk
zC#K~{iMYaTdr8?WXZyBt0=AYDBYrD6&62XYNX%{J1Z*uQI@DHjc4#AKhqiJ8ww4p~
zk5+Pew2{-Jt(<_Z<-{=BO3sdL<m}j1PQcc3qGN9*r)L{EJ=@9&*ji33Vq3|HabNK+
zn&8z|PQcc3V)3NnO09Uek<+`aoPe$6#99j}o9nYv8#z0*l@qYFoLCDWWpm5%X(Ojk
zTR8z+%ZbStQa0<YZyPy%+sX;pT28+<a{9HA)32?ZfUV_(1!gNbJGYUub6Yt9Tg!=g
zZ7Vtb+sNtPR!+dya>5v<m7HDL$l0Z>oPe$6#0(uNn|p|WHgX2El@qYFoR|%^k~6T4
zoPllS1Z*uQX49?Y3~D21P+K_xTg!=+Un@Dgwvn@ITR8z+%ZdGnR&oZnku$ihoPe$6
z#Hy~9oR|(N-leUCw3QREwVbf5Q*ouObZaALx3+Quww4q78c5l!x82*w*}bitfUV`k
zY8WY->$682IeWB~6R@?MxOIS(&F!;i8##Nnl@n0OIcH^9l!+rLXJi?e+36Y6r$w9b
z6T_b&{E0RDcb+s2L;k)FOFdQ;V+PydkVi-VxJlEJrguPu>CwL49v#=W(>llsqE_2L
zKM!~OPTJRTTd~KA%3`_?(R<Flmi8=}<{616X2YCob0p974e;o^-dpQn;%X5cvc?q(
z780G*Opi>EJ%3+o@}x|yo^Fp7;?qf18i&Y|;6Q8U^h|w2vEl+#^XZf5@63JtA9Ba!
z>i?X3Osm{uB=;Eb$8}C~?lJ!V@7!bN+?!6>lm3vqhw=X-`ys8eA0pWg`9t<Y{+H~B
zII~awWA<3>{GX+dZIykjWFPy7>|_6z>|^KLJJk)+XZ|sJH<f*}3UNt<DsP_Lcl2*3
zmF?eAl`&a`7i-sNqBp04{=<?o(#PpoU?*ivol>mjR{D0tZajAl@Gq~jcwHV2!sU-i
ztIP#*(+=hN2Pb7frCU<%bI--Zp35YEkED#0$;O=C9<O3mG+pMpPkV&US?MVNKh{Ja
z|H<v(NYbEMnbtzA@;Q;6;Jz()ExB-#&DHYuc`VKKXs^}f>-qaV;a!LYA|`<Ol}qz=
z&9<-i!&~xfl{R5LSKH^Y#imcmnlLL@-*1+`9;QXPK2thaCuXEbxp|FC%9s=#9UZ5k
z5G>KrntvB-N_31iw`Fa`X|=x(e1~Xr)ubJKO6*vbXsG%#O#N|E8m<sN97($Pk{JJt
zQ-8**KS}COvig&v{!HK_mQQ2ns}FZbG3e4z;(2BNgnJU$&iS;;z!>%X7@c2RlTe53
z#w=lI+Wx9`44qq(iJg`jM9@z+@4e8c?cU3rRpZ^0GRBQN>T?3N7UoIGR;$O+Hst~_
zI$6`lWnrHHO2G)gT%!YLW;t=n<Y=8$Ojryi3eHt^RV>B>O|Glzu22ooYh$Z=DHH}Y
zFrwNi48E14ZGNU&m>;7*2o!w1+C#-$1{9rM%`<L@9R;*DxY|#lBxfNsEzE_mQF@+i
zK~4RdCoWBSdOn4QW@vgt>te{9!LHS{1vTD%u4d+@iDymrP(Qd*nqAl;@qh<*Kco>L
z5&yv><&PegHXc3$KoR=}R=;*`I`V>hsMXJm*Y{{d0yK;vq~(+7*-F{eLO$IiBPCmH
zpLk76NuL1)jX|a<nORtTp$(->ovsscv&;!HvdqnEP=>TwgZg_<PtM3f(@M$6!Y+-!
z=gjFDXsX4;YLmdr!vkmX_^0c&+D7&a9>%1p=~I-`4bB`;pEP9>iJ;yQ)a=MXb?oX<
zJwa8+O`v)4)sq!k2lP@-^%RBDfjS?lp03b5pp(6;vlSWvv@Up&QyF4`F61q8Dnl|*
z|ILe>$}rPe;V<Tk)@M`e*ZjPQ%JJg#nXK-al%s#^>dr`NQ{6+H?HI%KoO}Cbj^lnz
z=8P#+eQXtC5ph)W_qI~2xx{7=@<=?isGH(78t74DQK&+rfqJ~RC`_Tryff9%xX?(I
z_KSV<6>Df9&Rsp0ulh&70YPp3n!{TDV)k2qe#K&1bG&2=ti)}%#Xgv0`W@x2?)v1#
ze5Cm$TPQbSX}R%Il@y3O3nVgiu|KAUFD0xXIk2>X{kLBvGycOrbMEcCdAIN8-@a?R
zeHURqE`R`}3{oK~7BUV+G*=Szyx}f@05>$$*6HQL^|8_BCA4BW6^`V$+?RMlh{!4m
zG5E$3FHA}BZWNEgI$v>HLSav3An38QB|aRw#xh>#cb52uGQf?(o`UzOT<7g~q&xd*
z;1Zfzo#M+_fzK}yiw!wKi`&0MS!|dC=NHy$OT=QM(G$_SYfF@=6*t9?y|{!Qrtb7{
zCw9K;UR}}+lVcy=MGHLZQxi(kzam<POEqFCbz75n^lSPcK)uDZw7e#aKu8byp@w(8
zv_^SGXVgTu8eXtJJp>dGZ=eKe_%v^gRzu4w^_JCYdksyiP7e_YVfd$=9#@P&xy2Rw
zPjgKaLyP%E-<lYvYH~6$M1YiE$IsnrXhL;*x5`SarW&1D@r=Ti)d)=CPw$qjPLG50
zcuQgWj`XRa8P(};<(f*(Xsj5)1t>1KQNwe~)8F#ob5wc_KT&e}2QxsQs+vTG{6}uq
zo#N-IPW_6X)5WQ<!C6+r)5<d*&USkyB$pJ!O+1Sk{@K9Ln#suS%tp{Twz4J-i^bEw
z@$AHRWQ_%RpFSSWbCr;~nskvM+#e{rp=OFg0YIByubB!IJujTsT>Gr7nZ~fO1Bqt8
zTf=)}K;5Wf3(V%48Ng?rLw2aoY^Ydod7gh@T+K`@tDWDVy?DaEwiDo4+TNFZRV%6}
z)_Y5QYn2rdOJPfPt+FCwDcp>!6)Pf9f(ETttcc_*37y?)X)vO`8ae#ET2as<zrYfF
z5~NlXw5T%(_{^=Pf+pk3)Tx?w@kA{@+i~VOu-(|jr*;6?KKmk~jp<S=N+iZ{2Ts<C
z63Jt-BbU}H`#jo#?Oar=?DI@=XkDx9^T_qY*|pRr^<~Mp{k75LO)kt2jH^`!Jz|o#
zKhVFH=V50*4z%IirFIy!=InT?!m;#PWyB*F72o*5zjg%Ty=+$TM!MoHxm8Q^JhLJ3
z?DOMm$DmNp@k^>n;kC*>hxE?etyT6pWUug%+VL)qCg#`5iFgwm?IUrua{aE6=T&3A
ztQE^0`AEjgAJk4jMtH^mAxUFul}TO~fbJb?CrjA}0IYnmR<0X1@>pxb+qJZ=*5m>F
z&1Y-nx?v+v=x4uMJ4HfZ-wyMq+Nr`$D8RnKwHX4!0DMAfrwL%c)?s|@bOD0^68hDu
z%XrZMvAb%sBxES7)jq#=2CDUCGpe=w`+6;v?%YtN3OOQz+*RxTe(fwoJJ+?;tu`O4
zw{zt=;54k(hGyk!gS%j0Z2>l|v?jPu7uFVvn=1amlODBsAaw2$PB+gb!epVKNUODz
zgGPMLl@(bm`LqkP71{B|jDYq^NlFD!n^OkV&KJ;uL`$OMYs&%8kDx#sYb(%9&y}<4
zI(yVspuW$Qdng$y`Vg=hpw1QH1T4iTht^g^6R@nHK9y1;>=So?S-U_oVpAJ!u3abq
z^G?hT4>;m@T6x}qpk1fpy1|d1-d(&vi|dXZf%A^&z{QnXTn~)B=PnK9#ltfpi1YaN
ztl^UGb4fS6IUT<MJe)Sh>cgYs7Y*+rqt8%83x0QaPppt5?!=S)@K7P2O7g>dq5C@j
zxdo9f?Hw)$ri~f$O2KeBFm0U9jL%&{#;12h2XiF4uoaQLrUeIp;)Sg&rnhzl2MVBu
z_c{<9gjvRgqb%rz&|q3lU3i`ve)d>!uphvAYIxMa;1CI6Y0l~r+>PKJs_xkX!QCZ<
z+PW$+m||S`h6-Bwc5qLCi|weOjh5h0iN-<?E)4F)kl;aNS8L)EBo~e*;AgA+(D%Zl
zufkVrqVs7wi_Xz`YAHS8cn2|YY)!u_&3@m&uRewWO%m{dGvH!7`1xMJ4@!V3zHbf)
zhF{T6`d#b@VbA{$91-f9&ZA7hVHkTa_JqH75&um@ZSmp<j7u&K9|ZHCWx<0Gc5$NE
zmdFZ-(aI&7d~f>VJVbkFQE)7yl}og4@G1HwcqGca>qGD{l|;km(db})`nBIo@d<^`
z^D}}+!^e4Rjj7rP2amx9udhww(4^_*9>HTJ4nhxI2u^GX^>3C%eg9C`K1Bguj4*=H
zRxW-;Mve^_fCt2$4mQ+>?8Pe_Y7-V4U&oh!E}fC~Stlb0W*LjCv^p6$q;r1wQk@JO
z(iKL3RHtS(K7lryR;Nr+sV5J7RHsZ)sVCkC>u7GJHBnD8)9XS6ga+m<$kXbSU20fh
zUeyAvPV7=|hX>}_0K_i!cC^NigI?-2A-s-urk8_3qv>*;GP$8npW0p5pR)t8)kVJH
z1Y!vlnN>GH0M&TGeRX2jBCniWHR{B!<#wo6npoEdJ=^7B=)v~;*6~Wl<zYq0@ggUA
z)VIqiQmyhl#rc@TI?lc>&qB9$89Fk-r*0V8dPcDoC!CNia5CJ*)~P#n@dS-*Y8j8!
zjS%1=-yWzNDS+*6_<MB;0=%_SpSoDGbxpCw8z22<g00K5*d=+)@~PvL>+;8B;~7{s
zqdQyHu8t;LSIl^Y!lBiTceQ%^FLg3{!MYCdMsN12<0y57ra!S8>n5UKx%{yfXj@RI
z)$wCmSJG%IK4fd18l*g?VwxXZHx(*|g>5by%;jsKv*WB^9Vazc_9!};PC8YjlSMjv
z(6^%=XJUbH<scWFpgEg$FVm4xOX}_s;3FjYUdt5@M^9dxQkM(5uk?^lG}q;!^l9_9
zu+WbE7VL(;(u2L%$>ntgOfE_}<ncN_9dspwr5LEy*$K?-@!$BWu0*0m07O1hH(#Q`
zZeM%tlRAe)8^`=F@$<(I?YLQ2!K}|CeAlVEO8BW)Vgf&?<LB;PInSZA<F>jgp~)UQ
zY+juhQr?aS=rOHsiG(Bq9RIMcMnE#awY_z<0#X4Uc(!gCI>FwX*qa}|uTEY)Y{~#w
zu%T`RLVRsm04LJwRwB(SS8~YVl_S*0+raONK6+C>X$JF(djF@t;Gcli_iy6ZTfOo*
zzlhf5<We{E7O(uswtnwtOow<^IR<G_nT&VUOX6Kv>W+9<yGy*y=v&5Z>(3pIm<ITm
zN__Ic+P41b%7ig&2%au+UnWl_HNkD2wu~<&Y2uDvHoMvVy63VvbT=YjS{q$e-#_|Y
zLz+<^E#qUpSMP({-f>wjW#13?OH-ERx!fho<cVJT?`pq{FZ4>BHTNznXm)3RuuR@d
z8jCn<y_d;zNn^qPtA6!<{V`L2S$er3_{1)%4}=oU6m!of@4{#4+4>Na#EeOd^W+iu
z?E17m(v|AT?eLj-q&~JKRSkURTI%_gWM`^y_>7oUKe;6p^Qu`=Kh>4$DDzqtRG;Ze
zb(DGSKU-hqN_CWZdH1NF-;#=X4fv+Mq9qmc8lF~vZ%ZoXwJ5QEtt-_N%&V@lexobZ
z6U^)6PxX6Tsh(h710JhC*piBQ_4!-<<1ML}R}c64Q!S~OS5ssCIajJ9%<D{t`b(}<
zN0`@`==%3usg5u&uS4}8wxnWS%O9@)YfCETwdJY$?^{wauLpb9|Kv*bIP*&DQO~hW
zmHgvJ2H>t{*LvOCmF)5D@S458UhmwJt_EJYLG^lIOTut?8L!pr16vXX!E5Z!dOgaO
z@bF!DC5P1Ov95%Nnd9O|>-7n)gol~qbKlhKX)Otv<BLz!>r-12GRJO@)$92!37O;W
zAL@0xE8%0zv3qL0PJ2Qn<73QmY`1!Sxhvsg%<=rOdVN((LgskyllA)AmW0gl^}hA`
zj+TVXap>fFeXlFwA?7$Tv|c~xN_dDlntki_CRf5k%rRwVy?(kSA#+Rx6X#kIGRL(G
z>h-r<5;DilDAhl?5*}oZ+bs3^hpvPNnd6DS*6aUlPG~A|gI7pgy?&=9;W~Krc%feZ
zsU=}NI%)HMkLBD4xhCezH<!ych^$Ok?O)Dm18y>*L*1n<m+L8wCV=JLmiI>z_09Lj
z*~_^-qTX2_a$-4`x$2$u>>roQU6n?8Gw+SRAUC&E_vPIExYhtNOUsvYBu2*u;C*Cy
ztSh&~*OtrelScUxOVe$i2AX?b=c@Q{aYHEM$y(k%rh)h1x^LyM*1fiY^9z-Cs$YZb
z?#P2UsT~`XSRI&aNrMuL!)o!`hJHxn{T)ZY#%~*xSlm)7-rJzW>cr00@!s<6USvBT
zL&AcPaCYOS5)K>emah-ag@a4-*Dz%G9uGlLU+=a6&X+<4x;ICfE|D63*LW{-P*z9;
zrUlo>C9>=MQcFQw84@y>GX?qh<`*~N<GDU062q7JgxtYsn8)UXL?Q9@r*bfU6o>GW
zyE(;hXFe6e+ZQ<>k)!Jy@W1;0PbbN1@GBuhuoS)il$m*5%AgDzL*m>7S<12Wtqq}(
zxBs_r_xL6x9>bQnqfdm4X!fu52pNG2^#2xoMIQ|r5lXjENJFZj-XS9;CPF8C8<HSy
z37rg|_&<dtfIVmYNr=C|9x|#qer{#R7_2!vPGkIwqY*#X5i%Ah&DSQ2Pdo~;OMOUU
z7o<D1F=U)KE<+B5yWq2s@hF{`2DrzUh9rp_vUPeVB-xiKPsO7&@a<gf#;a;^vf<T`
zLdbUG4Dx7vG^EPCxol4jg^!UIBA+Xf;q6p3eAbtREJJSij4!DTg3qk*kop#%aO4(K
z71Gd2!ov{0VM)jeSNOp};B3R<kd@*y1aV74)?hZe?HKst^yAIKTvFF%Jg-_GvKE1F
zW+CFj!yzzL#chm}sBP>1ki#;OfScQ!_n!<of_b49h5Q~FAM&(2+=}N=_{4t`asnm$
zrVQJsx{3Or=Z`rV@&l^C?=fMOqeJ!n5YBd-KK}3teK+LaEj~W*8UJg@-IgB2b9s1(
zuA}kYER~NFhv)`YM(X3lyPHCE6S%xt%F88hoe9z1`T|v$fgbA|qPt@`ezTglnmZSS
z=<V2o$MRP5(fSa*JxSKeciCo40>jxa<7Xx>Hu1j0&Gq1kyLyXpzjHx|ZiXt~<g)YT
zdi<~6zw4<Dz^r<2h|VL1n~julk0+wA^8Y`=z63g|q6zo*%w&N~LPFSY0tpFZAqipM
zNo3!Iu<uJq2nn)-?2s@rGb9iKh=>XT3W|ct5)jlM5k!$6kxdo_MFmAgji{)A^1iR`
zy+iolJLlz`o^(}LS5;S6cUSk_PJJZxog{hchOQnX$pMmVxJVITd+Mh#6oGyKP2^48
zc8@fdNwYr`s0Ceh8^3&Rp!S8LS{3YhPsa%707ikQ!}+Ds9TRmsPVG;}pi-whx_twH
zzIR1dDNv8?)&o9-J^MUMlxqx7Pd}jteEa~F3e?hhdcfHSDC*R%r5<p}hgy6Js24Bl
z0et_j!E5m*^j5o!#L7-IM4Vxt6PV2ViB)j>MD<Cc!VX~4n<Pd>!I0U+?7auf?57iB
zFxaXc5}K*NynHY*zAmmCaHnnCkPFPJHi=2#Q{x-Ki~;6@D~XLDy=IW`DFdd@sKlnA
zseN5&h5(a)FtH^u3k1^*m@b<W+o4vS1gUicn2qNW(}|J6zabr%Q&kf?5wl-t&H+;?
zC9x}Xj_NF!6SO{J@H3mm`uA!5GpqK(`h{6B@Uk<TL8kVlgRI(UfX5zm(wY4vdtzBV
z>_^TV6S62`PLE{kYIh?4q)q#P$!xB0#$jOU6c7`#k{@mtjQVp7F`EuJUb8T7Qry`V
zOd?J`P*%GW`e|Kyq?7L!IvYps8~CuF4n}83n{?3rY#-3M^CO){Fl5e-7QHk+bQ}F%
zr1Lnzof>A6${!W*1M$P&Kz`z)f&m=&YmwhMp<n{=YBT1^53dwF!{?$t{J!8>KTOxh
ze@@R2#JY5jA9C#cR(=IpAi3VY)h@rH09@T0@66}*9pfWr>7(-H(Og;TgHPx4G37v;
zAIN6BoF5`&Tt}O%&aVXETsy9#{z>_KOu7Eu7yw&5`C%-1w1u@mo_kbUYkYt(2SxbS
z5C)t)7d0_DHJ?Xr=lGVREqC*)VQy2Ksp80w@~aELt^Z3&`85RKiA1My`85UL8A+c>
z`LzUy0x;^+d>$MIR*vU}ecgXL<@0p?(=8~Pcs@T#=*as_Qa%sYKixv!2dd}ua6NKO
zeE_?j&*w^Re3x^^`TSG?5>c`!IKNQ^8l4+r+t7R?c;RQ#+7;wC5wa8jZy(9$p}UsD
zt@6ws`OO62?su~--*_rsKF0EMzVTE%&rn~ym(S}vA!SVf9C<09r}4(yQxTN#2^3Af
zrWSTJpU3f<d`+!Q(|lg%i3^e!O)?Jov{cL*$)>;YeVxUAC_c{*bsoF4*dIQBUarM$
zt-Y8tu5s$qFk!LrxOO04(z)1h31{^=+G2TTb8T?-xyU*}zP1t=9e8=MxI<;WM|n!K
z#qy-~+L{15FI^nM#b&JzVE>lI^4Rv;NN&F+^4t2cV=*3t)6TodoDaG{V=}LSji~Vs
zg1|Qy*9akClNVVnu7PIYRm^P7)LxFo?AP<}<A80twm8D54^aynE#_N83=RvE7e`=8
z@^Og3d|+~z_EJ;VS=73~3pHnwT<4Il|07-DKy}xr01_QMV++~lIxl^UJLR=z!LAG9
zF|h=~T^9vN(hgy4er$6&LU`0m!(T`lgG|GK`w{?txcLbRinI!QN@4JQZ8Pe9_?kk4
zFSND_bEUA7S$Mz(16&v^gh%bTvQXo!<WKWU$u-V$WDi_YLq>S!lA7{=(vn*8f9?`p
z{y()O!qFTS+uX&idP9$kkHI|qTDaw<gisbx>|bt`fg%_U&}09)m0WO>_a8m|o?G#3
z3T8vuJNMnHzd&5pgHOBF06>@x0Ihw@tp-XNUwP7GwOb7Wg!urFf0A1b280O#&=!wd
zUQmb`0nnJSZZ%YFoDOK`K(`tuJUeK7GwULMyTh$ELhr*UjYiE)B7=JG{?LuvfGoW~
z{Kc(a5XhFEotdS%)r){G9=2zI@(Z`xB4v29kZJ0RXT{Vu<FlsdwTs`#aCw(ORo|yQ
z)zB{&&oaNsli<t0;%V#4@AmXz{xE1}9q>Gbe4q{aiOoH0kw4cC`P_Fr&mmv`F7i*k
z<Jn03qojY&v)M=A-t&S*KkFmU%RYP;&o&F6wZroo@&=#xJ-dh(J{LTDeDotdr9S%i
zJO_OACp`bL=qHZzylc_S{K&(XJs3RCd5&2$b3#4Gk(ahiI_UYYMW4ORbIPKBv4-cI
zkG_WIypR3^&jpKq(n-%{E1$j7^A++UYu;0y>sEeps^@zj&2OHcE&QCWo?FO^4wGAY
ze)G}X_55YgR7&&QBc67c-PD68kYIDTy1F0hQU2)d&s)PuIwyD}qBP#UGkK#&1!DRA
z?`0dxKK#X_%qdP>#-ljPg&hq>+3L?c%2@Zp00p1+D87>->}WWk883U3vGVcF{!LbU
z6sNbaqkNZ<f09S>>T=jo&O=)~9u+1~6#ecqIP&K`#LeXOzb|EbR2580pL4e~x2i{F
z#-b4;L3tK8MSMO}C?hl!p8VURvPhXt_g#z3Dk)U-*8d*z^7_-~i;<rb=dFo+$R_0X
z*YVbpyjAC@)N!&-p99`V*4b;7eI;dKEQ_4rjbYhotLz^stIo2qvED?M{e*lYmzUqC
z`TR8U6DN6jJx1_NI(a)<_@}medsz6Pk9qrB__x0C@?2Bs=N<R*Rqn#4^jmM9g<mku
zJI%uH8|~#2$3j1(hnJt%68aA9yj}}Gcbd1v!bhL<@>^9xKR?|2yoIm#f_JNhU;UBy
zH48uL3va1~k3Z~v$HEsp<2_>GTekF`u<*~k;yo*PEJtsRitv68hr2Ql3q$7wuRPYj
zmW#>wUf#>H4DzgE@<#6!@jV_-w4deu5{`ir{OU8tP1C$ragpK5JVrBT_ISS%fR}&5
zfAD@SKn*UPa`JlxbN>~*`V}jT+~4B*-FqFLaU~yavZ#w!1rrFPT6djG9aQTS7<Eoq
z>L>E3*5r*#c`p`50lo6dHl)kOp7^D#r4^!Cr7<pzxtCVN5{HEhq56PA<0*0EJI56n
z8w=#yO(+V-uI9g&kuP!<@g^n7Z;dFb3iIF%i&%T!tylCi^!QTA8-k0rQ4cw=DJ&{_
zMF3i4Xy2mk1mud-lVgf@NC_?Sh`s1F0cernw~AgDfWh8Dt>_JKHa_p(VNcOc0U}Yd
zJiBO@0K9gQw4-RZ0CfO_-!0lBKoo#i))$ou5DlQ+mZH4^aJ8S`sc4@7jA7>wDtc3Z
zSnUO@O0T@-RriCZV{dw|;O#JCmKErZ6Jc*6m+YO~6K!pZ4a1@9_`%o2LHJaMc7?$#
zF8Hha+bHgj68TI7UVMP!wx!*({8{|^XWhR-@gx@Ku%ywwWi7(+KV|5$ME>t|X?eOo
z{s$eZd)Mz9)OfF;TL@$SeW>=j$g@}jbqxfePjzY;+!8AGDY^=fo9L@`oLOUYwW9A3
z1zxR#uJcwXx`|!i)mY>!94Wdd{ov$Bnv4FCcEh(u-6@jozP1lQ<4=l|O}YWV*4jnN
zE&!c$3E#f_1#WQE`n+89HY8px0t;|`8`88dLWJwvQOM^253X+~AWvI^>D$fBJb3a}
zCVbs6Pb>NlnWu$8x$uoeCviG-wFvt_dM?^kaZAxAt6e+5<=Ca7P4y+ZMPbRzqQzF>
z9iwn@T^1rge{#{{hGY!B4R01PO5^*Sw18<v%OoG>L=ar`q|Q(pM=98ICKb&!xqa0z
zBx_Q(Ry4N;IfTr>HMOF-(a5hw)#Q^!bCaa&QK`z*q8U~t-Vp-*X2em2kXe}%wW1kK
z$$;FmA1#_>5k77ZP7*>Ew({a*TtcaR#ZQ}n%lO0xT*hOwX>e&+rzq1ReA^(*6hd-w
z^eY-}6<#w6hsQtQGMv^IU5Xwll3QxwEMF^waG(&9OQYzbJ|>s1n;V6FuyBb{Uo0x>
zgRX~3q;-ATSsY(1DC%xi8f{eSUanGisifg|`kO_GApH8UVbnxSRASV;+@i!N^!Yqg
z?($?&Vgu&UU7OAoB^mxfSK&MSe6G?0=DS)1n$8JD#${s^q5<J}&j&UJ*Z^r?{|R;|
zt`1TyjU41^x#et8rnBM3&(}K^s-|WOu2l${K5nK~sN}XOz6E1IV4?YjZ~$}C3e6XU
z2k~y5LgW45BwMhnP-V(c2^u#U1rwNGi-M*b3)MsmXt}shO%fnz5{RbMVKYE<u&5B9
zyg`kIn3umP1!mjsLd7?b3XfJ{g{qgdKZx%&0f5;T{Ym1VLHxw#LN(nYUXSt_7O*Iy
zP#K>g37RyHA|GeTJ}oRL4?w=~1cSC;6{-RNVMoP`2@!?7qZAUfYkaZxu}g)2;mpDu
zT!nu%W*MptI#sCXM7StmC?R`Hwwg**Fd90Rc4CYrO&K$`P|2-$YlCJ1&#fZnLv&sD
z_X;_wiOl0N1PiZWfSN<F@ETmo9D;?{Fu23AoYGO>k}(JR<c)<t$bzbM#onW+@RZ5y
zdZ4&l;VDdm!T^M)>eI!X?5H!M7~G@qqHtpE7IO;^TDAKbwGU$bvuYoN>zK6<`D!1P
z+F?#K>R1?Q#gN0M7s^|ZV94tp!;r5(7~F-g%itD0b_*vO;^lzCt(NrHjXGPUj?l@n
zx|Vcyci~!#?pK3uZL)937p`qh!-UCPU-+zaG_16>b|K$GODluL$9f^61A)G6ZLla3
z7DlZV(k7!;iR7Jgw8E7}Zmw3iN^)eh_4&eli_tWLQ9g#TITaS>gTxrS2VO3G;(`1j
z*9vD_)m9kQW=l2ltg^Lmf>n6LD4Y;yI^l~&g%f0YmC2}ZLb@~pl{WrfIL4}U)2KA2
zT%|F_oZJ<0^jn2859VmvcY$JAW3Q@V5n13Y?2Q5TU2+KTs3`0$u4hQ<4fCLH!Ju!o
z!tOkSKu3Sq!yxRAx_~j&4Ln-d1ZsS@(eR!omiOf47B&&@foffzENqe@o`iAPOlE=r
znF*T9M8HL{b3&nU>x=Z($ohr#&B^V%Ghl#Hc_B`k6Aa?ZF-Q;wRC08BVQofh=<e_C
z8s?~t!E>Gi26Il7L5MkUOQ_e@&otbPM`RB4+T?CXV-ED%a@pDYWZLuE@}%c6&KK<R
z+9nfNtIfv#qM6s$g<N3O8+jPuUR!ra#XC6XAlpN-+Hnwg8+dJLBCZ1>9;erq0N}=I
zFxs*V0PY&Pv6^v!ACH=M9`V{@SpKeA&U5S=tI>{EIwA83DH!hZwD`s+H6X8{e7g3=
zyQq9B&})li<zLLoky2Sh(TQKZw(2PQzM@%F-B)z-W3MfYMfJ?0um?qlytY6VJ!%#O
zN)c>vsI}Ktu{y0w?#r%w)!(F};m|7ew~xir`(E`Mi=HrxetS@K#j9?z=y|h<(Hm?5
zhG{uob)7{A%%bbQqNQ(p)u$}t&V#Z(l_GY4e4h~g(H2*Ep7AP%ZS;m8_|}+Xmw=F<
z3Bdfx0AN#@*wGYq7_;vWabaj!N<ToD|9?Q}gHE{;hMyCiq0;Ujy^8bO54_K4WjFvd
zU#9u#E4^wLkXjlR8n2SsElqn7wC@4jrcil%U#}`=<rhF#BM<;)xgX*nZ_p{P+F<z2
zl*u4xScE2^PTZSb#jx*(OQ@545<oOZ5P+tg0buUX)`kT!K=G<2oM$jP(ehQdy=o?^
zvE@Hj4rP1?1U)4!lmQ+9@X^+qOlKM)9x5*U#jC97>_^`0MUxrfS*Rh!h|i$(n6X|Z
z!(uHZz6)avvm}<96^F?*ZqkkbQd=7|wlSoSG#Wd$5%QzVd=umuPGRtjZHhc^_(I;0
zDv?GS1<iQRt1>J>Kbe9u;^~E^h8Yt3nNs_SR91ZC3ol+50-mtRO<s~`)4EDUleW9i
z0+^2eFa|GxDIa=OPtk>3A~L<Iy~TxL4!N|K@DIR?MF1rJGj(ZiM2)C8v#D3Lv?{JM
zE4D0Ov87bhFhL~O@~V2Ez4@KlTlFM#M5Vl3lth^>(B(-05<H5}(ZjLmKxORQRl);r
zNb2NOHNcKh;!okAVmU85_*}o1xIWuA8okPifL8jcv8iU5j2X`JH9jw<|Nk^a=K12+
zMK2g<eOmGBbr?<|pE{)YbvTr4il(nDHr9Mrb{{L=$jUp6${VQxHyNnB5x#C!-bk<J
zjtZ4GHWyV;`MG7qPqQ*5Ja(~|-*&<f@dCBDZShL{({ey{II-A`{LegxACgk+rih%7
zDLFY>u^aj^KN;CLX0ap$Jy*OGlz2_@EUlQEJFo<oR-YBmCl}7;;4+`Bq(YE8KZ<#y
z>LQ~8R^eIgi>HVRsI<VSG(|)kgwutPGG-c8AY;Ow;vC9249u9Z#W|KGAuk7BVQHO%
z71Pu@hqe$+^L$)7^e*P||MPDK7x`KYO$sh!Fo9WGkKsHgQD?;G#Y0U^f8oUjh~`&y
zq#saesEDQw#{W{xVfM=)gIh*9OJ+Q<WQJIhEi+EjSu^Cv;vTFy->Ata?kqieL|J;&
zdabylrN=shu%n0|hmIM=t;vClK9p)*PIl`DvRnIPw*#Hgb6u_!b7%7l_d=-Iw4Cgw
zKH0|?7AKps%c>ZJ$v)X}U5e|Yu&kxwHT6x|@S6HKjxwY)&n}K4O<z=oPw;s=6wnu6
zFD<SmtkH^rtBb=)HQS)#>5@rxYIAY0P(6j;jO`v<xWdp|lIT%Kh)&bAUpZSIy95NH
zr}Jq5=1&HI2Nkq5^d;kfUnjB(MtiMqJ?bM2hhL`w*s{!{z5;pJQOpS*^)<3G%6ZaD
zCy|}Yn6V!Dq=KwmgHCzm)3aI*n)+)!@JQ(ghA=LcKh7q5ay;rZW|Od^0LI++s53IP
z2><iUriBGfA3qk-E@Cx|Tmy_5ocwA$|67TdvVR?cMf5X{b6azd+KqPp%5a)(+#Mxz
z6DGcJoK?TfBI?Zp@0Sr7X>5M@4*=ktq690wB7p}qV)J)M^Npn&kfA$g(##_s#bI{q
zGf;M%<WZBL#jVdo_nDS$Udr{T4FvYV0=KvzH~hq-a{c6bEm1twCThY1)N!Dw;4Co!
zfLG>v)Lb~{tsekT!FkZ>JcG$6Hha`Ui)=m2w8#QrdX?{lU~VTH@ASx)RJx0eS_W>v
zPNV#{P`q}SN4djj4e1p2(K1Yw3{i|J-=j8??^>}w91WUCXc)l|=ImQA#_@X|#hCZE
zp0ca}kmv<1&w~Z}Mf;W&0APD%o|wr9iYbFi4HBgQu<?UUfpFg1BRr}Lcyo>UZ3$Si
zybA{g10s)ch#-p-x%3b&xY~iMlyE2`GDgK0QIWPwGZvixg5mbp-RcV2paIYIa;x)f
z054@8y$%4QgE0cGkzEw$<`UMbsoNL!{T?B~4-g!EM(_gw?9Jc@Kr$?t6*P6q3~=G~
zC7g4@OAr3)R(l}p_fBFg06<Em$r}N*VhJsK@iE{VqUtshPcw<Pad~2RMdEGEg&1@t
z&D?4s>2{fP3puA)bPJ*U@4H|wxAD<?Iu9=@y1@ti3Sk#Y!Q@y+w;Dwzw@oIaI7loe
zqo~;~v*sA7DICW$D-o3AQ@84j`kZI};8PsOE&+itgT8|X^)I-69rDK`W^r0PizTQ*
zDTcGbYUvMd)yiT;VPw@xXvylM18&9p`b5$*W^qG!t&dehES}=<Y-b}-pq;3=4tzNa
zOT{y0#rjgwY+1a}0;q7rtr7@`^3UT+YHh3Hzh=eS3FRc#MrRnb1%us++aU_M%^NzY
ze08CfsMD>C2XU!T|CMfESadtfqz%IgD|7RUMJR1dPNs#z+FL>ro^-1qO9;;tC?u$S
z#UQCjAvF%Wl_D+1@a<z}L*+OcXGhuN;1&)rKny)))<gij3&L2B+CA>RgL=2W0;ZkY
zeTV)aflN-g`&ShF>5qa|`R-romD~!$+P?`auD0__+~1RkH<SbwxW5;dG>CpQh|o#J
z>F&>{8f^7vHxQy3{1y;ghRfc2Q{AUox)$sX{Osmo(w`?J%5(E8V-j7Yyx=|<&C!M0
z(^t9=GO%H&k~+lAn}sFc@>%!09GeWbUY+EAm+Kvat;xOJheZ|SM^AGfkx_~>Ecg3{
zf~a%+Gxsiv{RY8^`%MGQ-PXoy?yU-HCBEr?65wC7TH*)pCuLz_^Z&ER&4-Nts*QrZ
zE!@j}5zve;+%s4(-cXW9-J&FvpTRy`O9f_85>B1C0J?cyFLDiZ>=o-?V9t$@u(5$V
zi;Q^TzFw(2D;ic{7~3Y@oyF<<C~9nJ;~oc;v^Kx9n+7ntaLO`we^%zL9BF^M`%?>+
zCZBZo7xOr|Lvj!IRp?dC{U|H^Y*ctO!s6urh`R#{{*E*{rb848jPTptA%SYJJ)NXI
zv`fj8?iM8PWstWhN8Z9m-pWV5d#js6{qHFdJM&j}LoO{j<S3)saFgzVZk~Hcr2gV6
zcMR#D0{xo}-7#<`qrrQpx~s8(M_?$ZCi**PHPYM+>J6UqP*@rlcz3fUYCp-pfF@g(
z0f3Ji=LeIwEKzicJG?8CMZ3`FzGIX-@XN_ttVrcfHCZbwGE%wYkJdN;VaY$N%T7Vv
zf28UNTxow{iK>dBdF;{><tHJCoE<HZOXU&?;WT}T;+-QLD_g*0f6j{G;OHq?!3FY;
zM>wwNf(AoRcy8SjE8<x2846Y&SrLceNe1cs$tx11#c+>fmsTW;G-uXSZAB_{Ki>)H
z(bHD6m$88S!bU52EeJ*=&toepf#+Si1?pCUQ_EpC_*a51IAkWDN<B<IK?N(q*pq9)
zCw0h*%EAYSgxT7P%HRg0<NDExa2&>cS_+cJJ6BYbjz|Bls42NH=LBs<q}giBWjSw^
z*p@<r^Db!dd?=&=2CMU(#CRpPB54kQ(bGz7t13#kF$+YiMGtmu>Gl%abT;BAqxy7)
z!kt31GC?)HIa!;mW|n8QQ0&2QWqC}AO%D9m;wT7l_Rl4@L11?;C3JS)Wbj}pbDsXb
z{A~c}cmdkEOKgL27>c=EBn{>nhKmuws)Q072hhDjlR*bs)nd@0H36eeM;2Jxb+)v7
z`>7IJZ!s2>TiB??*2icnST`P0V&mw!cMGVzTmaHm0DhMM2x|b;tzkFnX12UhdETwb
zjwLG>l*ogeVOanQLIH4=$>I`oxO<6>3o9V)pG3p1^~DY#rGm1ql1a1UB{o@C_p#m!
zol9(VD0r9l&tsu;V7^LlLcBO&F0-;^pwcmD8zwph!C^{lRmEdy{S8?q>Q6BEhx7QG
z4FRxv4mqhmqb*MA&v<4HPU`OmoYX(U2?ZOcm)IQAQbv#cMwh6gU`sdqm&anzKph1?
zL>Tlr#m9Wb=U_YaA^bXdE|@glS)z^!6B*Ja>Q~WMeJoOfcq?9sV%#D2Oa%2=84{@a
zQ7`}~hOjdaL|+I#XackFJlRD3I{|dggC*)J)&p3K&UoNXiMojxRBcAAj7O^8C{e4R
z_x;AE&a3J^(0R4^mAH~x-GYrX89a^6fZA-Ry3R5P+wxp_RgIHGQ`HTaxMX_)yD6Tr
zN<3boc1o6-O>0)7c8j9mem_Q<vM-E;xM;?9y=4G$uvqQw5;>1#Gw$bwW}$d7=PPbt
zMt)YJp0E^QdzXm1Y_8+`5;d94J%Vy}(Pa8}nlX;l<aX>MlwcsLDdieJ1)g`_Wwdu%
z`S!}=m2&DF)}Ta<RGQLio190AApu|*sgYzr&KO2&lr^U43ZrBeG+2)*&pI2_XgXZ>
zjjv01V2HOIhtio<SG=MR*J%)je_D#-nte)CI~Hdf#qIFcK&_>zMLPrth>g&6&#)5Z
zB-wR?%!$_$YAu6u#)4yr%vCVmqKFc?yGq<GqFae^#gM}gv!e>P$dXT%sH$+dI9+_t
zn1k7e98i2rS?*{^d7(t{@fwctt|*Huk!zbdc!koNs0>+up}`s#3)K4yI%@kcspr)w
zxg%{yexA4FPAF-RpB-0nhyI=xG#&G^apOc<(0Jx;LLW4id5Kd&gRC(;gT^B(cj}}C
zv9ZRp5ka}g%5A!7c=x4R1Lf!5Mipb_AnIik#al7ygC@wYd2mHViw4&!`3BB}tGddT
zZ?xzev{zb$dG#A^^%ymxMc2!<=z6&pU6&R)q0CL8a6)e$CgZ|6)cy%GB686BCoBud
zvE^lEF|xlJS+<)Kw#KH|MnNK)QidB{p(im@4a~0whS#@He#?NkNbh{31iM*fI~I~X
zW2RPe9+68}R>+L$7**%%Gmo})c9)!QNRG&3em)O}mOOm;d^eE<M}B&5$w9oY*B%cd
zxILedvKSr2a6*0}F8!7qOh$!$&hgn0h6}l%hf4D&mwdoB>~r$F6(t`U`5BnoWP;qs
z2nmlED?2|lf+vQk_<<88o1wUULI?-OX7OmSg0H@4tiAx?g}Y2AM2%toB?y6)BVKA`
z;YFfaOS$G+GENL=)XwqMhNmx=ImzJkv~Yq=?Wcf1;UxAY`}l>7lF7#W#L}OQ(&@x7
zmO<%EU#Y`slrAh+cac%bz5&-K%GFz9lxS@4hWwJo)PARE-?&_Rn~TP%o%lgX3uy^i
zq3we$jU6t7H{UN~X`J>zr%gs_*K(!Z9@43=l%Ov3czbyX7H{R}Cn0%>oK3DZoTq-2
z^fUJYCG`{7eE34uM@s5rBC?!Ez6wG|g7efz=m$3i)6`KVb;xv>!L$zN29s$W%nfiJ
zVOj?sDH{(kty9h;>tKPkJhDz#i)j-_NjRClVK5D6KxH!JBe<4#h9kJL+&sLA$p~&9
zo^Ix$X?QQ@IU#qS;&R|*S^1rXrMD~tfRTP<^(Z+HRes-?!8~%}sud8-gBEQ#T-)X7
zh>HH9X2p)ONkA)bkgEKnNMTj>kCMtktn43(%K4wV9I32aXjV>@%0ax2=5lcTmVhPs
zq|4F7ny4fGb~&0#Nf58Sxg5=4buCAN4VR<6Y#Ce*E*5ARWb8&`a1^2ZKQ?7lFE2wP
zYz9S>H@Y0P@QPXgk2xzg{KVym^fN*Vc-y%HU=+69<#3P@Co`JJA&e-a=2n;eUlaxK
z$OlFCe-X7=+aVQz?MTh7mJtGs?|<^sIiN@7v5&g!w^_NfS^2h9Cf~7XF8ePm;`tBx
z{^BcI^qI?kfkhvgMHh@_Au@B>FChSw$ZYybm;Eykh#C9zciBG&Am_Xb8@cS4#d2X1
zlez3)$fN)O4ET-%#(mIXCv-SOLDx({hkVT*R_e09g`z;GS@hQbHhWkpSv;2OvhN`s
z`;E=m^S^YD<&rKNqIQsuC-kJ-@xOH0(xutZeJkns_6O2!^_k_dPA>a;7QJE?t(PJk
zW4i3mOLK!}HAa4;<V9$N%l;&5yl>Wc(#I@2#APpH(Vu2f(F117g#lb;e7wtE{D2#H
zPehdpNoJLaQpKd3Y*kvH;j(i#j|dj+9y7pY$L)Bk0?&ilo=mIKn6WN<7LYi$0^Op6
zE;~*aRfQ=gU1wicq!zgBIO<guo;HipgoQZ_>4B*QWO4YW%g&Ig0wZvWata+L_zFjY
zW=w(hp++PfW<b>*#MSTBcG>+|)%1V6zf>jH{kL8G`~~ilH@V_fd)$Ge=+G^f4F_nd
zVvJE_yYryvn#=Y(i`tt-zZ*rEj=~~cw$EAgvYf*Lkf}42R~rFD&?2a`o&Z*1Y<BPt
z7UVg3=7eTTzB%(S{NW8Q+ZD03IkIhEnj@R)op;%Aw}Yy9+SK7w(ZQm;@PKmBZkKHV
zDH+wka<&CN;}34&vduzKP!+RimM|%2{J{;R|HiuTtsBraH0d79Ad3dMc!!4y8fz4(
zzs=!{(Ww5x6pvV)LtFhTLz~>3M_lR_t1L9D-0~^3sI^P|$Rb`0hpy_!2Sq6^b(uwb
z%%aP_qMr9%yab?vI7zV9`TuQ3&->Dhx-RuD=$w4B5b546r|Wy=bUj>d8dBVz7&G#Y
z|G3m%)*o)x-&?N!o8{{7FJJ$Ful|CqF7+DgyUhBpm8<`T)Tf#Yo^kPlmvZv@6V-g#
z*U|;WF2yLzx!){8oMrl}`-V&5*i{8DFpIomgwSlfi4!GpY(&uG6XyUhE@}gqOMpvT
zPSC{ZQW9?hQqqzD9C5@vms$p{G8uQB?NV}292cK1(Jr-$CES)IwFQ9SL4bB{ms%qw
zK|EP;sb?*lEPvajp0fbV=4!pK<vZ57xRX@D=47B2`<l04u}jTk(Yt2zcqz@dQgOLn
zu$abc74K40NymF4X}qaIXZYX@5i2$x=Tdnf3kf#K^2(9%sp<2bglwowF-8rEH_5Vm
z3I$$r@tzYEGSV!{kRsTjbAn6t15)fTf3i#U7l215C9_-#QxS?ZRjD1&r1fVsZCuU;
zz$t>(JO=(1up>E-{S%H4UB=y`EO}zH>%m?SGl>efgbh07QoMLAz1DcAOL1A3&Am%W
zP{K_~$nW5>c9=`W`Sw?p0)Rs3pjsKz2B<34GEvkLJA0y%O;oypGUpoA(E#Ppzg0Ka
zeYEd<8hqzeJr@S7(blbUGpp+|ENkA(>blGg6(dqaNtY$&3y^5$D&Fp2X)8l!JfHXr
zb!5~IbMQ(9FC`+bIsZQV%<p*@M^mU4GL<vTYxrOg@(crBYYq~|*zjB#zH7sr;)NgL
z<U<Zg;e^MmJbaLk%`<Eu>rP-@y34hZtP>2@xI%5Q-pEb7i!t*4maZwJ;c0N-o31JF
zVu|IU>EqIGJeGCM6JLTVt@2&-r8jt6=awO^9GVG{W+>NS)X(8Uk>i|$k!RdCO&K}0
zTmvZs<^!*R(NYEY`D<M`&sU)bQ8%c-^(f6M_rs2O(Iwa3)@mv&2vSklhEi$5gGsKo
zpcf@mhqxH@Lm&@yUFjAcPQ%N42D7(IT@BcQWJU_OR|C-!c{pJMsAF6#&|D3YtV(ax
zcSW!gJUoo|q`({z(s9sWQG_d2B=K58?Z*o?pbcT)0kdUU!3K1JZ#?5|S}2#Bn+skO
zt4%i_M}hedwkbG@j?!|FA6Td0DAmYu&Uk#1R=~qN9=EBmi%#B~Ua(dCN2512{G{L=
zb5*P?_erWSz8;28W;df6#n3wDTtT6NeH%v<%z$o{he7Nk3kqiQ*_29^3+9l|JDUpT
z${`l=&-N^sCwYjh+|v*;53>*kj45&fWC4;x|Ge4-lfXP6w_q@tuR{A~Xa!tMw0#he
zv8rH9m{~2;)D*rlcYi?&4T_VEaNeC$SX7V#rt(b5Cr=lo$b<oW=eq?d3BdDwrupK6
zRAC$DoH_}1;r28<0tF4m7qpPb2>JfK3))!2tIp+uHWH}-KYwyTTfrOp$vyY`WNeIB
ziG5dYOl0sKoGgUnn-5y3e`N4IYSS>2FBORl{)byr&Qc8*MF#VY@Ml{9cqRe>)*}J@
z1_FSlXItRJDl+&!73J+BQ?<wtzi=`ZVb%J#!UKnIM+Scp3~Pno51l?`63D-ZClHi{
z>{Zm1J;AhS08m+g@JAzqdAEzKR#7PVN*V?&_S6J`4#3q?3jSII<2VypZXwb*{Pn2o
zF!s36$l%Yam)Y7L0^Xt4p8&4<>EKV{$XZ&N+Is{41J(Kg@RSCT!F<hypx?*;ShZeL
zRi4YE9Z`f4UgbG8VJOzX$lzmyHmSU!Qy!lQ@T1NEx2oKse6?w0@Nt5#s`2QbS7GB5
z6>$x94LH)G@~-0O7$_J$l0R$lQ|%Dd3Q7q+49kjqQbMinxCy9FWbhMgDz&AT@1R;$
zBTcoQ=t96!YYCycKDFFk0mk{%@=$50MeEc|=g%zuY=)E<rbY(O!U?KKap1pGq&#6t
znbi%9p0K3MCbZNiWlnd1#Xc!>p@Su*I&a9S!Q;D{L6CCh)yUv<2Fqy6&O;bZohYSd
zV^d0ckMdF;iF!y%hv<i-bi_nn&cvN96W0UxXxRP9v5~=DDgc1LbG_DD;s7`1$1=66
zsix!?296BIlCP?Hv2%!}Yh@tyCIA=HMj{Od5c4-Ok0Ist)a<*DNnO0$0({89dCt!c
z^fv%zA<ZOk9+@YQ7NciPp=~uA+^<t)@F)uW-4eJ8R3(<c4FI0A1il7f`$$tD#wu!O
zloz%in0=Nodh6i_!iI>j5s|^8A*}j1OV}r%`q&coIe<%+upa?@Z?$d=ZfaAz@T)n>
z3%m^k^p@5sGzJc<StiZaY^ahA3`~lH0#*GpQz)O*S3#ywg?>@hO`$3dKs`%nt_a2N
zLqb(!(4<;I+XFyPT0#q?3VtaRdlprrk0liLQavo8g8|^x$I_2#9DpoK=n~<8--m@x
z1<hnj=zIWk9|&C|&Ld+vGB}q)-?xMof>JyQ&ZC||dfF2DGJvg?(Dfn|zYjT&+6|hW
zmQeJiIz$gNJoXxm?Z}M`o>svU_Aw~_OMt?zAboBL`w760mav;548IQx`wKL;O<^`H
zI5zq_#^>0ok-@x}%<;J!Q;<5x@mX8N8&e=ID9i0PHuR${EKw%HP~uSC)(t7$oDRc!
z0vY6+5h`Q8FvseFJCVVBVJ%-eW1E3AC3@^st^h`S3&c_+4+K21IWpL|1OmjfNUNiH
zb{-kb_v4Q}1H@6J_o9ia!GVr=()KgbwdlD-ISrH@>#n^q`4s?(iHLOep-2Ne!Kf`X
zRclW)k3b`X8^c#QUD&51O+XVkUF<o>9>=e^Su3jp>?NQnHaox$_3Z1+4zO<n@RHd9
zdoHpA`1_EH+4q2Em)QaKg8<$E1?Gy{p)h=`27B6nhzyQ|f7QAUzI@2VeiG?8fx5`t
zLi*Vn3cDc^_3+D~P|r_O`Y@`7Da`>^4hI2daP;h`1b|bT152GF($sVJ4&i{`hxBwH
zn0Mff`7-DoxLD$7;+ti59|EyvzZ)5h&(bShoRI=MB0XXW><yroC2$mg5thLBMIe43
z7MKTkyd`iZ0Q7}9;CG*pD)`l~rYAsXL^|bAXg<<HOK1^*rIt{1jboi9RC;6g|MrGs
zD`;M{gzg0J`U9btIkoM+452k5gA*xquO;+tP#&;^{ujVUmQYNtj!Tx%8=^LTHD|f`
z)bTB7u317cxjKILnRbr@#F`@@GPnT*M%Fh4`f(PxZwlmVTmA3_BI!%NS^#R80#O?(
z3y<;>xF1$VzZg@XUn+o<2Lc0S#R!iKPNKjkErFPE{n}ar(U*Q!U;3dh{YIF}6TcJy
z&FYkEogezq&+13Nxd3MSrn5a!A`H=1GL}y1KffiQ;Q1ZB)Nduy3IOy{zfDNbTP=&H
zDfi&_q3O(TJ7`|9TJ|P@y`aEQuYH|A3-D7X0j+=c7Vcnyw`gf#Ub~&#%m)mH6Yd6?
z^f;#gtpNc}h4C{Yo!`U)+6Rf(Fcn5|DvX(hh9MeseuE$<W|kQcIj<3#Zv{lo>j=AI
z3ashR`E5sI`ZN6)+}J<h372x-fWhkovgy#w4->!NAp*3+CrDhc#SUKrxMJA>t^yAD
zeb^2^faZo}hd%-Q;q&7?_eAJDL+JMuD&qx0`R-?b%@oQL4F6D5s6R~OA7u(H4Wv;1
zKIF&#$)IUq3iWRXppDP1N~^G^?Q0{QKTzNgltF>rkh)j`hX5F43CsnMW4cvo6hvwE
z7s|Pn{|uB*vxFhU@_!Iwl_rU>*CU-jLRb_dHxaf1RLd=4&jVO*3BxG&-(j_`sk9Ei
z56wsZ2SD?tCGaBvA3kUungRXZigezjz!R3hbD%n72}HN~U$F%K0^lc0U^jM1DS4D1
zrT%w7gU|lTj2qyGEvEf}z!4(wgGlF35E$Lg6c~VoIv~Up7@z~d_dCRC0^rR736{VS
zB9J`F3k+xunx>Y(M*yVxoTjuujDlZQ!4+~9z1tGn3zSxf6)+fp6=DU917L+%;J{Xz
z9DJd2z*Nv!Ay&YA0GJ}o;ZeFygyNUZeG5YC##urOLFu(xx(2{%tEDdh*krYIy>P(q
z!!3OcG~2C~?gxOi-i*&n4}n;-pNVw-L4j*5fggZM0z8bzfYV5ySOU?p0aq+fJtVI8
zMLACmz*r2h#$o`*VZd(>T6j_P!!KKS8v<jjumTeSwtZ#`1495fO@VwYGq9G`!pp(|
zzYn!A5FvG7J+p;@jRB<M<;h@=IHvS>se)gup3c8~0@Fd&9=BLx$_e5mz{w`CCsKDy
z)?ZQ)zYohA1eyVstV{r-LBXlEFMsyqC#o~%(UH|7ofkNdGNg3izicdHE}Qe{1*&)1
znny1Z`pP$tUg`z#Ti-nTnHfe>O4L;Te9WJ}TH#W7(@5uOUSf@n4wWsq^K>gtx}kjH
zInsG1o(dQfzVi%DtPti+!B%g6M0@`-Ejb;Cb)4SoHPtclW(kTj*OZ8KjI`KXHWOR5
zK-3AGL|_RraNEGSnpI(hhSk}=D53)^JY!W@2AV>v!qWhrVujIlAYr9cD2nLD3je{h
zXfMJYmG;qmr=tVq7_P6JM$s9iCfF<5ZwPWKbp-M|LE_Dyq?~CY?*eHH{6laH3|zfX
zAq*`O<PsAZ10-&mu86wJiIRHojQ;p&UcU%_g#f*!CB1|HOK6t*qf>aDJHmMG8?D%m
zw3RK70NGV($l!=(LEt4VPeCz8PT>1&)nGr+VND1;k91Dc>H~y@lgbcR^B1j+XaO74
zJ82p1D-eEZ8SN&3AH>R(bx&kvMYO_~lItUSu<xd<dq{UEE6bEs0l`s)iXv;*VYJdo
zS@@+ae0xLHZ;h(EpbU@YD^x|TT@YFU!>2+$u@u%48oz#Tnr0T>-Gb^Zi6cA{eEQ;H
zup1F6Vk*!r4TR~2MWob35l>??&qt)x=P4F#+G#~Zm|;`pJf>-WQ)$x%wNtXRh%f|h
z_zsj-p*d1&$|T&Q6A>0oCV(-*D)bKEk!nOZmSw-DNxZc_g4-wUqHPs@b?B`!6}%GT
zR;x2A0=poU&;%_TdLK2{BF(9$*@x*M*``6MI<NWL2Eoz^HqZ=9K&x=XQL!EjSLZdD
zU>I}=T7)!$Ku3&J$O*bfAOk=HBz!l+J`9zd!;tzBn1BrCUgxtN0nCP^QdGkIKXD~2
zU4WAJkq%*=wGTU2rb6Iv5cjXYEd<KOVVSfK`x*q;!8&<;wGNIJ?ZbYttqQDIrYf!1
z?9XnY#aBUltPl2K_kit%&g3ud!}qu0d_lz{Ua=2HcRS%t&YvN8IJz|$MhRA6H$0a0
zS$hZSl}c@D8R~DtecUP@n-%-8w?K^PHW=5U?ZZA~1vaRt3>OP12xODkAdKhWdPsE$
ztV5<BQZF(e^+A~mJ3;37#cRg=Z5`0U_zPe@>O6>GsNh`G8inQtKaPagINL}4j7$O2
zA_7?YgG-Q#0gT3i6TBX29Rc)s@asrB;G8OC0D_`#!24&IhHq!Zmk?_Ky#q~cgiNKI
zn#u)J7iJT_x-3fC6l~ZOJbgA*D$l;EslQUWw!lQ_Yz$}>oXXWKs!l|8CwyfB8o)so
zd;sZ9bWs%B&H)|#5fZ{7R2%*!47EnX^d8k9A{OmV#Gb`I{3Zx6$b!$XT7$LFXXFs%
z+WX_y8_?)Yff)BK(4%?@dbb}iqo(m^9DjzgUVEBmA-H^p^bLVRWd1?AOW-M7Ttdf%
zR3xw&8FWtwRUC!oJ_L>%(whLZ4;hX$6zznKLScpxXr-llw&>nBq7HSxY3d$=J`2gE
z4t?RiBip0eL@~{1e}$k+C4L3Mkv%~)4{1&hi0LIXw8LnBysR0RX(%c}T1sFXG8mU3
zh%eAOWL`kpL<ys*Bw87=8)+v2v@+y9q<5nLJjI{Q{CSN(XpOXDqS1;N%wLBw_LkHA
z@XKkgMpek*6c=&|=_ELEiVL}l^aY-J$Gd-?N_+x~-=L<NW*fs9DfubPNQqC>GTuB9
z@+T<2kL5{yBA&CxK|NOYW=MR0@_{>v9$ec?hVd))M(Xm|^eMBoL@x)6yvBL&p~teO
zOa!K?5A!}QTqJ)}rVWk<N(=ZtaXkiK5`)-e%-@xsKw1nSrAm3A695c}O2t_8Qd+V&
z3D9#$SZGvAlvPXsb3~;bNUs7&E<@%`q`fe10Jl3SaD_^kiz*#j03DWaR+*<^1mF+~
z1AxP5?+K;GbF!yf7^-d}5b#Ih3skj?e5vZ^NS7#)s`>z`3Ky+(oh2=i`3>n;0#F$<
zlorakjw*+uUT9?kP&pL)r%>#lMCH&{09sJxKTMTFy8zRHDxdrRs+>|#rVUA@%7Ko=
zU0{&xrjT9WFcOJ7qLi)3z^y}b2t0$#5+v@tQ;Ly!5@{902RbPJZ9tol))T-$4~0QP
z_d=H;%gvc*!w^`Iw%gFnGU5#wQ7}`X2bRqt5nSs)uUf+%cz~)DOWXFvi8Fa0Px7G<
zaj=<jbg%%-Ssy3PLKu<!g*G46ap5ufi#EI_z@Mf3DaKEgSHTE%n_x@-PvS^kQ%V_W
zFdCUeMlfsA<B22Tt+E&=%}*SeZsr4@OdQqF%0HVps*Ra{VsqkX3%_1V91UfqZt(0x
zdAgpL;vj4+luMa!DVH)I+QiJ_kK_t`%B75iavwEG>><jXGnDI*K;@9fQ0dXw6wu<u
z#2%tV2nJsd45li7Le$=+iM=T57V;QueT~6p+9c%`<kVLD!Dbr=tq;VW`wq+NkfwTI
z0~{odn1cx9Du;I4W6^n<9+-@E1P`-np{I}z)zq}pF9FFR3MAix5*7^P*)%;c1x~=X
z&@(;>s{v%xcR)s($T%QlqG!yU4xSBzj0h$SjR@lfNzWWTuu(;{;};<MBlU!bAAb%?
z$QL5!v>aH*Q+RZdrW*1w$SGs=3KcM&w*3ohJ}$xA@eG!+uuX=Zpq_LN)V(y7S{dv!
zm+BR&fVy2Zu*P5wn~5cHvV*lifd?#QNt|{=JAk=xaumih2l?aR07rc6pE5NPhZ6D&
z74^Wj@a@#5ypwNYMN;JUE>oir(aJ9(89i_!DQ07Y<c^25#Yp)Ch9gsgv<$_$Gmv>1
zX{$B|n%yi@qqmU-elZp6f!+$(W`;)taY#e|Bu~==mtvu5v<6mAo(2ZtSYX5cgS3Z0
zH$3Cr3JG^(tMGxyOhH0utHQ@3vlHo60>plXbc))Cx9NzVc@S{}GM!)^N*Z`fIHn%l
ztcz<2;Va5iCiYMeS`--)kL5{)Bg6=Y1B8cBl_FMo1yBp5Mg&Tc8I07Qz<bDGISbDt
za1xm{NGl0^j?6nqI|x7lE*9Z`68H^P#{5?W!LV$$lb?kSvWg~`>VdfDRyD$f2$c-$
zRl!tOr4#iiMa|?rfDkQI8L$8~R$3Fz4r{_0V@)_^<tH4|{F(b{e*EH34_t$Hh^lc2
zPBZUG;}7Y9FF-<*u@F4Y4G}J+1r&S;=y9l31xBdy9D$?Ae24TEfiu`>Rse9Hz!d<!
zkh%~+o2xEB%6<ubk<+<M<-j)ZFU6Ik@j9$tIfJ9{lda)`1K-4u{59~GP?c{6sY+g#
z;Qiwf!~NS8ZFMfnUzFcCwr-Qy`C47yDrGAGQ@H!Mzff-~7@+MF{K5)0y|;+y9L+#C
z2{6!Cfkv#8gVB;R73JfXGxkHnTT{Y}{232P;A)s_o6AXCP3~mrF)x4KLsW}|ih3ZV
zx21M0Q(5U^)Zg{MFVUPPIP9{eW}rQPAYu8lrH%wCT3k(B>~}w4umDwShSaD9fZ3Xs
z5`c=Zy)6|stcHN9T34+dAcWs}PBxiE7;M$@kT`dsKXYD%Xp9Ba4%Y+OJx%MNKXX1p
zX#vs#0`SRd&myfso3ar3=UhO`dD+Rwfm4f?Qs3l|X>q_xS8b2Q;S&J=C5MX^2MjB6
zNE=e7vRktGDVTE0sKZrMN<&9fyN-nTP^Cc{4kol}e8erKDQMCPfQd$m><FM(SZKI4
zKs7zW($y%z_^aLoCQxacV1hA_#sxWTE6@Xxu=y16xE7r^QN(xFE8x~K)$$ig{2e3{
zk#KCG(%`(+DR2=1Oc&KxA(c><&S04i&8lxh+C(4(nGcbU;wJQT-M;FgMr=GVpCX+m
zfNraP6X{z59WX%#BH`=lwuY^s%sNqKwjMB{vZk8vK<_t1Z`O!G;)4e%v(b?aF{o=G
zSgO$!z29&SuxmBd43DMT8ZN{h1%5iA3a+Lt1(HV}K+!L;FGUYbh&5LS+X9)^u|i=$
z*&4zpQeJ^XI!Z%$R}HvvjoDI!=qF__im)ZHHQWdEawLxwAsS414@Iq61iz^P+8S?Q
z<7aEQ46@K}><U!#p)6jBqW6&gBgGfW6hA8{#dBDE8ATV7&apTZYwu$%&Els~oLUt{
zzaag{M&x0T`<6Jl0~<Rdl|BsGe4+>7?Q^Onm*136XyRPlIf2Errj1?lIE+Cg=Bt`*
zky7#jz=0+#n2XF)SiU;o#)JtAG#U{@3112hsE-OQYG4|efYFaNlB0Gv^ut96$oNSQ
z=v@&6xZ-0=g*|FwG}fF!;50H%Au$|G!FjVS72AfIuOMyZsQnRG%%&}7v7`)GtX?$_
z$f*6dOv#I3;5@cQSbS<;LV^Qn)M|~fP`7xQUHiC!+XUrb2d_g`cju|DLX1T;b2|#o
zq5$C|8&QVUwg8MCyJs@u6vz`9;3mc!xRws$Fh+^P@Feq=kq?;j<V#J&w3WHtU~Et>
zHEVYUk`e5*iNXLoMTQnFM_A~kf<`A!%>jBE&>Sg(yl5gl<&cx$%|iKHd1PCY>u!td
zHZqumivnN$0j?jGr(I$)xNISz{-jexS_s(<BF0svbz!8D;L_qG*kQMA>v2m{tzu0n
zdDoQx_$M8?Pde<W+u~@@k#ndBGX=2hC01H#RvK<rnvh{t@~fycR3p(gRQm>;r&~n1
zf_}%W`;>()u<F`dnl+)g)Wk|IW}dK4YThtw9<k7+M32G%Y^OucRSS(x8ux5ch^_Fs
zMkxH(D!lxNX6t!MrAjGI5S|;Ysn+=T6X$$0aG1b@R5dEYENuC3A<s$GD9ou8+IoP+
z_Z}*nTvwW9kC|nU1xl4IMyqhsyU{9459Cy8thlW=1I_3ZTriv2D`M7!%`<msIIsp?
z<~4A=1BoX=hr`}Yrn0UvocjXDkXPnkCe1fO;}8yGv1|-phSIU@M|4OVyQM%XC5*+{
z;P%tZU}`?Fo*Bqo*^n9NHYA&wcW`c9OdiL;1FqhN5eP>B-s49!yjsv%7He!p^Heh_
zyJnWnGs`B5?Z^IZqF0&dNl~QEiZIdJOf()wA=(@aZH`)Kr?Rc+rP5iLUBj${J`klQ
z_kwiUyXVR(Fy6LMBRQ4`{Oe}=M)n%RT5ao~8Z3IswCG}(HrMFNaadv)MYU}P5=VO1
z*gBLq4jn95h+%A1a9CoiaqCUA7$K~{Wuo6P#dr-dzK+w(j>AyRy}=fs98(gss5%>)
zgc<qiJ!9)S3pA}lQao-53n%%Ee1x*WbG3Qe46RX$pb_39jQo?Ery7_+@jB&Z$RAsR
z1*7d>ZMYr#pC9gw+g@RLyO1`%A8oP1(x|D8Ed6HyZr}Y@4~(hA`eoWQtcY4KZWG(R
zi#4y-rzmO$sfb`|a%R5?<g2rQ>~3uTYE=M{-Q51w3o^EUwXnw5ie~iN?p?6l0hSSZ
zU>w}D-Ajl(+F?1V)fg!W%TYV5B(?e=b;m@kO<TuNO!K{{CRpQF!-{VIUJv9AJB{+m
z^=7EC->eZqf$|0)J&>Qc#x`{YqL}w@>4D)+h)u&nI{Z&P;8t*%3hISMr?P2jt<Z|R
zT%cPMfbgkExj-(D&;x#p2C_X6>yTChS-L_G_#KfsH^#O0Bkin-eRZN9fU9*pm(Xfm
zMfxl{cWS1l2mEQMq1E~a$iG2}PsauP6{+FVaoB&L2X&tEa1Wf;pW|wd6OM*=$f|L@
zG}WF<fDP*|aTtB$ar1lo56a;%JURT@{{$Rs59VxSuz}ZLE^%<A2OTr#Zzp1;e{lms
zS~c=`oGvG`ALfxLE__amHrZnY&eXtde`vY=1t@q%(+7TnFSMDim&WhcY7N8adTAYD
zI_GLV;9DMSr16gYt}wv2siXkVWgUQa(nI{{y&iA_bwj_Z3ffKn2pRh60lXC;?TALy
zo`+F<3ZWbYH3<!@hfiA#)B{FI*Ft7`2E^cqA^@LTHvk+>1dQrz0yupL7>zi#$}jS)
zg&vRza=DB7$VNS2Ebd889|$p(cIg3G=-;}h0PM`v1J<FjT0;P<M(F|1SisRmdH~G;
zH-IvY^6{fJgaxYI)&pwO0_o4A$_t>1vc$M9=>gFQsnVYh%0m0=%J>+eX;a?Q1M1=b
ztUG!@yo~B!K=itpwmNS(=@F#+-+<<ImzSx^E#bHO>9-y1i8JTS)KFB1MQ|kyG+l>_
zhGqeY|MW(^H&Taa2=wPm!+G?gmmn>U=6la{{~tv=AU;7lf}Kld0fZcr!|X}A{~y?-
zKH`QGN8uue%3KYi`$%^Yb7pQt%!vx)`=i1MNYkPg>9*DgQXj#67-~GmX+#)OCHi3H
z8@RdB51H^Gye1|tU70H*-1m`Z=#EC977mz!-6O9NH%sWkmz%ivh?|3f1U4{0=5~w=
z94;6Yz9?7WeG_-ftN`B*KS7*Hqus!?Q2UPVe=b;I%G(#H`_IAH#FVFo?#H3`A=<ao
z?8K>glVGAbZ3&<yQAo9nf$cx9y38+H1Z*7929@{u>HhOD1MzGuqAyaP=&Y%aqx$?{
zqbDLp0XYnJAOc0lVMQ;}wCJfoEvSJ$&j9!h(x>2uuc7)cj4~ugV$zLF1`?k|^<PBf
z0w7*MdLGDWuj&5z(U^q#192AVV{=OJkCM3{16!)(ahmE_DLM=1a{i0qtJE^`HqtFl
zE<^8W3eQHVjxjXQT(SDFPDbx^>;c@vlS_t7F8<5e5x4+o1c$?hnDi4&dLu*_hN*;|
z+>y&5+2kU+%QGjnVf)QO20`Y=ZQdMeH0+{TvX>=LmbjZH4uPJqG9(O7GST0XlxG!r
zwwc0YcrUZ$4oi$?4Ig3PI)y#V1R)RA>=bXJ^WrIT_&F1uW}!0#ZN@g823craCBms{
zu1P(~LQkcMM$CkX3OG#+$>q-#xZJo&x?dRp-uuw=qNWAQVQj|G#LVgAH58@58wSEh
zYa#h3W~LI?*l?QKaK+kF5;Gf>$87;_ePU+g@;EH4wLc2YY~sU(W;QLCZ&ohfyj;FT
zxqQoV`BvrfjUFfzn%U@qCIB9&0N0#URrkAPE`fELLZ90}9=)Xd{bnMgu(U=sHT^Zi
zSi6=)AaZ@AO!bQ8Q5}A9)j{{;%`lyhV`3Q$H;=+jJ}QT&vO>6%_o4K{FHU9sc`Doa
zH;b?UgbR2ik}<hVJ-SrGz)z~6`{z~BRF_9UltF^ENUNbyQU%n_C<e3xX&ZrOk$E5K
zAUd)iof6{{+y?ytZu>b)CPnD}_@JNavW+4GKza@73v!D9gTYlaE!qyu-vls9qZ5&0
z38Wy?87U3ZQ|}tq#aG#mAPf~T8NZgPAv-iG-9YyrMltswrY{&qW5tLbMlrk-lrM{j
z9*;EcW2`+qW*ovPxr*P;%1r?e#+-o@Mnm=a5Qh~KHCyTacJS=V^Q=K=b~IcodK!7+
z89_A&ZI51qw1U9P$h?8{3V~8&jv;+O;5}qMMdB&YAV`h=4(U39OUPi7i7x91;9H0t
z02Le%xSFLPt6Kx98V5=5F`R;EiOWV1VudENP(>_W6J<9=?F`-DA8L2w&4_~<fCD0d
zy3tfSgM51e>Vnjqzz}4-NDINb560)ZrB;YE6SJ7YBuwT5H>32NbSD<Q3S^F3RMo&e
zsQDVwb~5J*IPegl14wTY_!yZFk&d7NeIlqgGbemrGBC1CWn_!aZr$HWwttXqL)1Z}
zTlXT_y2+_6p!-Pll7U$AV=zu)aQBHCI0%_!qyz$4$UK774h9{#5X`4Se;BmeH!NL*
z5=<H~eHqMUAc)M94em_P$OzbmnI6QDxk~pR3vIgdZnliKP-Q&QIBK&Bj55%s7#xzs
zOe1g(nT<#rFvDf6F=n{#easnd@I7{3#wK$S7>c===g8e>qukh8%eK}QSU{bI4*4H1
z=iUZISn@PA;5-<?*X38!+Uow5pdv1vqv!_YqaR}Ed;|VK217Ci$HHnLMqtbZq)!Q8
zLXG(v=_=cgEvs>hh>R%27nU&e_kal?C=*lc(fun^&X<&fWi{qkq@QRCUU3;vh=;up
zH^pL;qz0@-260s^hwp$Fk%9ZfwjqGICl<~fO9veAA7o(A*l`4oBQqUoDw@bkNd26g
ztoW-X>5dG<S3TZ>XkJ2!MF+<6befltVvCTLf&(uh#Xg1fBu(A7E%ye@VE_JaS2VOQ
z)+k2pi1%R(>FEKfkkCt-cU1SUistnk!{&7WmrY2|vw26UT^~R@ka*c>0A`d}gg>$S
z;5vihUHx*|8~BBf>5kl*J%P>yVu_%a80d+(n??<uf@<iK*u#bb{cnTps*I;kb$@*J
zRP}tF3Iu}WED~S0mc%X_P!$j?9{W9kx|+s&j_L)Yef<$A4qhhB>HiMEF~aS-?yr*@
z=lA|6L7IdVPi~xW`hN}xQ)<0-1mHCFa*)OflYZG{YCtJ%l8hLr37GWaT?73vzto$9
zGz(0Ud8*wHy<881>n$dLUat2#5{v}D=^q3p>!?X`xbB}wCLU^n4P8CV3iWswOmaAd
zW0tQ6gVZ}rpan9gkWR8)H)M#=<NKEYEaNpt{9>c6``0GNcgS%w$l01ptWO2LMY<-n
zdO(M^tc71Z{jB@PqSm7`SqnW_?-mj-IP{-^%s)tXI|4vdKOmbu2GUTLF2+4@i2dsa
zfT!)13=|izTk*@~)^P!j`6%w<wY@SDL{*R~!`B8Z0b&Ejw!NbIT>|j|Q#s9GYQWf$
z2<lNQ{>#2ee;u$AA9V|-?fU)>NG9;)Xx+a#bncDgWYrJbr24T)QM6q&RO^RK{f<a2
z2t*-+meij@025*T<wza^t&!P|w3R?tWH24qKZ<*sR6h*GWconAfk6F&^b<53xZjwZ
zdRO_sCnpxak2UfFv|<Q|-LVGTplq`W&%OR<K4bAUvkDJHxif)@25Xwi?#3W=hLbBy
zBiNJO*7q%xb)$*#i{}Y-|5h}?yD&lD6VUJu(qB-yFBZ+X03<sB^m-heE3P_BId}mk
zgHME4Pe8|n*D>&#s{8i<uRgeSQT0W4#Wh1}!s-){>41bgrBq)m?s2e3Tz>*fkQswC
ziU9g3?lGi1Z0q`Nz_yO<K--LV7?)Yl2hxkR5n3SThk=Lc{{1M{O|hw{j-eDc6P=el
zluX+JawFw~*^nh%6JaFo0LJBl6|C0_Y{p}9?j!rZxdv{g;YEFLDksEQ6KP7GRp1m1
zzvHoWkTX35`aIJ!tC{OkpYN<{J4J#Sr}wdrfpgjt{iwqbl2O&9&RI$H;CHB`rZRZ)
zD~i^TZWP%t9!Q34GD)oUZ`>w2IYYGE0_#97Bd-P;*f3PXsYu39#3pZ?)BVmnflGsL
zW^4;JV*K;*jPc?9jE5NsG|C4M^Re!C30B2k5H<7b>wX7(HGeg09w=Y)?eaC>k(wAK
zJW&d$$41N<K}PmFy5Ba7QQz7oqixCMRCrYwk+|izy5BQC;+`h)GtuRUpYsu)`at({
z`-r!j#P0g#h`pAIPgm0YmU1j?56Z@-*RQZ9@;igF$ISvz1l{}Yg$ahF=zhx#{#eil
zPry&_Gx#}G2;DLR@KI==xZh22(-O*wn_-EgEob^{nft!(*W9YvFWaoyynM};<!f?y
zre{DO9NGD`#vnX3Ge{2g{MuM#IQH{vD+efHW+TS2b$c=CLh&zjzZjpqU8cMkoZ+Pp
zOu*skaNVz7E%rKlY5f+{5%}FJpCiQil;1T|_Y3eb`_W_;Ak55~70TDF=&Lz%qVD*^
zs@XrtsOk7aYEsQH6?Mm#zVgmy`IqHf%<+}QdGbcx@ioo~`gi7TcF=58pA@D$j{B;Y
znAMNRu$3=hmN335YN+Vovfn5luk^w(gyT~zkvxU~lhe&~#~vS(cT6UDn#(sp9K2f(
z)F2+}(H&3wh;N(3PnRct#!?s_;CR+&ijm*yj(Jwi0jXxqdG*SP;k_P~`5g;=HH-hz
z9httG6U>^K<;?HMDql0(S98N|-4X4pdETrUE%q_35>vh=*YflX!_gcJhD4jdOx;nx
z8eEVsn1$Cl;>7ECj_8QT;*5u0VVPz&-4SHT8W?HH3M${eV2dLy6C$#}ar|A~QAtY3
z=IlV-e%r@pxxvPMyIf=Ke2=JQCi~yzRJQYG`$qYA^@47{D>|B!(ZKaG8BH`g-+tea
zW=)v`*UOX{z$tU!a_G=tqi*Miie<_i<S=zO5&NL4>?h0X@QLVvVb${s-F~VTHhvBn
zR@>0J0}kDO8U`O!Wt1^7`pF9e;Y~n^+(^1}|Fw|Y>9%g)=#!gc%H1e(!M)&1-M+ax
z>dlhY%rtt?{z7?GUKCZVmJOOE5<=LrK{=FgTDMR1N%+H*FtNOKCzof3SI+nx!ah|>
z#JbbOx=>*HO5Hx42Z}tSKE47#O#-H&mq-m6Wostr_Fk6c!EH^+y~@qh_TJ?s_Yuip
zx^#<f&u}6fZ%aFAucH;~kLz{>tGE<|C2eqK=w!T@1(*ixe61C(xlGsX;XDQ(yilI4
z0ThL*`{aU5@7B6K1m4^ze;7_6ztin17!PnvMY&vS8hjYnev+1(+cz3|e<&7W^ZF=a
zZ;jzyQ{8rh+VB8v4gSmfB8L299H7~5G@+*;`oPXL?m0O?v)w=#&QsU8eMr0I0PO(|
zv18mva)9;#cMZ6!a)4&y<N(c=#{rrzj{`Jc9tUW?JPy!&c^shm@;pF$0N?=a0e}NE
z6F@&AHn9C*c46G#(C1GeF?MY~8eXQwJMciN%^YOreU$4EarNAZ^KRUOE7#6oJ@MCd
zV!Dv-y6w1SI{L#>9(NC0XPWMKQ^R!eRlzyfFx~N%hUwyA!uTeJ>3q08!1Xjt=fhz)
z8=qyE&cGR_v+^{Zm8a>fJWXfiX*w%U(^+{joe!Ytd;m>n0LCf3?SyH%c&sz=^G$mi
zs}K-RB9)l-JZahUWguS!60@xB6B%Pb96);0H1e=*V#}Sl{j?|cTfC!k*aTEa572Fw
z%v1dM_fg?6BgURB*$*J$`L`TG7+tZGdxf6q+|088p|dq(4BI9%Mm5eD!`8voCoR)$
z@0<PA1Djgo?K8s<n*H^DbF;tBK;}uKzuw2*h^`+0J<@ffzbsq=isJ7X{pG_|*R+Ii
zqrVKC(O*`c{bl9ZUsj&|W#!plR-XN3<)y!T0Q<`au)hqzwC(VdV%uY|O^?6+9}bM&
z4rV%D^&WmwtW<+m8h#K~dUB3#J7`%6<~OV~;+kotgXV;gfK%6mM8ir4%?Tl)GjNX>
zR`TJ{wFyHEEBSCUfSY1i$-o&_vhuW&m8X@gJgsEqX(cO9D_ME5k`JJjd;qOv01prI
zYqZk2|A&<@Z^0n8GTpY$G)VVTFv#y5=A%9{4YJN0<_Rm%D34)~b>=WnC<Sh(VGtkg
z1aQX;gZOZelwi(~2F@^um8U_hJPl&yX%H(<gIIYQ#LA07d;ksN185KfFo*fG<}gpV
z2Yvp+Smd;6Zc(&E6ed;zlG7$$L&&X_mWbdpu|AL-<{Qjmo`?~X*zVzBeh()Xyu#T7
zcTOQJ`A4_CicQO?{&wREXJQXj>Bc>|5aDPDr0Za8hi=<LLLTSwQfML;qs0E45xU<q
zW`t3)?R^}(OJhd3$HE>C<I?{qx&Ap*Hk~~>@qiD!J~ByP!~sUG>c$zQM>g2wGQIKW
zA$on#aNU++_Ig*;w~b)0XLdAuJ)^NPStRB_ol!=wXEcYtZ2EGfrADv&a9e=eWc0cZ
z_YQFTjbUKmj9$0$>~$;8Ubph>bt}(axAN?DD=)q71K8_6fW2-2X0P*{h#QE+b4aJm
zUhi)qe+2S7v)2chz0MciCH@o5S1szcfo88aKqS(jN)TU^4rB;z!OP{gK|wf=he;YB
zsA*6KvnK8gunh)r-BUOrF});H?%J4J#q^S{`*DJ(HTY4t4GY!On7L->H$cZU=*Rx;
z`o3{uG^W^`^@+P?;^ZnQRs!PSB)r{Yj9mXTaNV%Da48vchIM%uB91dx?kDS(ZJcS0
zk*lCHgh{ulCS$q&2_`tI=(gXy5Z?_mNByP1bRPis>I<3OCu(v#VrByQuL`7pb%(wZ
zMm!|XnuIfr`#-9163*<{3wJ7P(`_C70TdX~rLCjHwXpQS5h!WGfRS5_hIe%vKXYG&
z?~l&hZbY%R7Sbc!)Yw{9WTA$bFzbeHOUBb^L(t2b+_01RZz4&Ov3ch03=J^48jN&+
z0*8y4|03Oh{;HkU9w?4A9{vR24x(<c-Rk3o^tk!Fnr@3l$T?&USf+D)j2$2>V=*fT
z%ObFJ)dB^gsGJM)kU7L>l|aa=;XY;)QE-B_$@SQN-By)>Y#_Mil53WqaIG3^a(xY4
zUxt6$JLA37_<osgtA#e;{>rj3(GzD(N1Jpf>SAa%_z>wRt|*nB1j7M1>Eo}=Dz$Mi
zI~Er!?4{p<25WwUQv`5U+~6yuFX26SNvkNO#{R~hM8`9r#w=2buQ(Vp1#c@^*+{q9
zt8omKX$>ZV`+6#!-A$CX!@<S-Z-MC#@p4m%&EDP^aY+ablPXz_@B<H{5q!<0Zu2*X
zU{YgHrD6zThhht`kez{S(+NgE*pl7SY<C5--AQ>U9ghR)?Mcw-BHN8$9!T3Nq21Zx
zIFR1n7Bn-FrpEy2E);f?&Joh0G9G!{dskpTA2!w5PP6ZGS}-1Y+z0PV!sL;(5bSBQ
z1~?rqErE<U+Gw`etN||N88({(A{ep37R|QpA_x>(02B79s;ghHAMCrg3{xj)uF-_!
z0?>@77oeycPmR<BfH>4rWi7e*s62qH_W{HiH}<!=(Q^K!Zefz*(NPi><fNt1*g>n`
zDv$!;4y0`($r(BT2lFN9>JDG+F%E+&=@8OE0&wu8{~~=v0H#kmk93Yc$LojV(g9sZ
z`Vt&43={Kpb&;m$!Q{AJkk)WFFo*(9%>=+ET(SUa{24r1-ZZPL%Z6%TGNhiad?!HT
zW{1idnL3_8SK&@ooiIpS;Z2(Bbmir)FsB}HO~wIA1;>V@ou{k91}tF&_}o>XnkVY2
zC<drzL_KpAvqZOAx>_c7GEPX;a&apWQ!HYFaIUC|rA<MY<yEn)h2VDVzOGh=Qh#z=
zzX|?b?$NWTFq7eKM8aHD5M~ZKNZB>0U?swX(*SoiVKlt^E4rF%Suba*+3>j-D`vyz
zi$<Iqp1Q27NthcF;WoxjHRsi#hM^|m0s-!0oFE-O#i*erJJ3o0vWs3?p{uMam_WGE
z&beR;$->ZTln>%g&>^c0Tg8yC4$L#sj}Sz$ucGLV<87v>k(k~nN*jk=%Qw0jETrH&
zzOKoa-@HQW!D>yCuJ9PYuwq!H#S5$ciH6qwU{RVWxi(TQGF<M`)uTR!Gfak$iZ$4f
z|Dd-ctcK*)G|MC4&9Ii-5vXdzwxM51XX0sVh^`9NjzChocXZVpKyFR;-G(UMA~^m$
z`~mj_Dqh<#?iw6_J8054oI|KI=|q%wcpN~4sLJXU9<E~ERcLXLRSmb^<t9iUSK#|=
zU+tuSi}ftGcZhKQ77Z{zs-k}zCwwtHpI)P1;4`ba1%X(ihUgb?SX`H{;7@F>UqAzu
zU0%SeUkZ{rmET-k_?3PXZ(ndOKr`jgBli=mz|Qr?BA)v?rWiJLh?Bv9bDN2_?t&f?
z&!ya13Xz}St2%gt-H-Y^!RiN<I}tv=TQAe@a$L1!m+|79b{99b^Y*5^0F&{6GEQfA
zM-XkI214OSG3pxY8^o#05blFXQ(-b_wDkK1=`K8MgK_=5bP>8AyQt(woTfg0L?2E&
z<ShUzOnAd#Ep4BCMMNK72l+2Z6}(U%Bvc<5RD(Du_l58yH2NU))q89H4`Js4o>j3t
z{BzD5LNF1N@Fp*Y8i)yC2t5f>LMQYp(xjKrLk%Duq@$4(rAjXXf{Lj0q5_Htg4hui
z+Z8podo8HnZ)VR4!TZ0@m*?3xyR*Bqv$M0av-Ol5%9-^tJ>rdju=uHtz=w$cKJi;T
zo^B0pSEjoWxxutclQP|c`9JKPGF^iyr`lg-x@b9nQO@qOWxAkBM*Jl!SP>V9esy-4
z&JsPcoQduntI@<fJF!eBjY%OU`FNR5vGQoKkly--G95Lhi;3xo^b<p&Av4Q#kiu)M
zEz@59Pfjk=PX6zFzf9X;A@$C@GHsAPE8hm%Z*!S+i9J=SOl$e?%r4U^IGa>62hM7a
zMzhjd!_AA?+3JVJ_w;4$>}<NH`?7X+b{)Y69o5fgXEQEG4CcJ8b@y}GvXKq3_9%J2
z@yvVKQyTK1begt8_LMdnN5Z=nXOH$0Rxk-iYeK<NuW5EqFH&BU21`%4&b!=1cI=ql
z!HXPkB0D5YB>v4yvpY0b8s*)@?2byWWI8@E+f)p?R+~)Ci)3o9nJi(!kt^A?6g{~f
zTb5ngE9!#DRl2TQ)I(#lOE>juvuk{ISgaCgkNTE`EvsgS)ze70J9TTq1H$zd`w}|K
zfuDtMCvcI+vPTEdq7QURXd^?!FlpZbzb9}3l9*cHsD3`-J~{$?O$A5o1o^zU@O5-Y
z6Iqubfd$QOEQqd1s7_?W-~y{ldw_}3Wkz3_B+ob}RF?`Iutz3XRv<H>I;w$8R&}yf
z?|?*(zM`2NH*1xI8h(cE(O(dqiQ^?Kd-Pv;X5pqw;72?;I60^@<_<jTaclM2nFpFC
z#6Smn*&Y*s$d1DkVz61&4<#{Gq-@~OAt&zj<6qRWEEJMQI!4wK@^60jF<%H8z4*tE
zIsev5!;&%m<tt_Ypm|LH(6JNo|4|nI5Tiv2(SgQ{K(b_p#Lug8iy$%=hO}xL<4hAA
zn@|~db|HX*Js}?sxVV-;RSASn;;9nnq0*8{P#pgkpNO#b6W>iqD6sDNm5vf5tQ|X_
znpBJ@N_abr`tQPRuQp-mcyM&%-p|<i08ZoN+QLik#5mofwI!|U-vay`VDxN&w23^v
ztas`{A?HNj80UV*s$pwlm7Ly&94iS~N~#q*V!~kX_a&h;VbVC<;Q5%S@i!^4A8J$%
zF$*ApiPzXn0KUiC2U|Rgkg!i;*|}yfdkWd9;KaZoDpntaC|wLN7#{ghjMEt4uwyW5
z>`4Z~7^jIy8H@Q!PcvcoCd+EWRBm95qkBCE3ap2zgVj?wGj1|flUjt<ky=3DlejgH
z+feSPo;?r#it$baIfh{$4u3tyiS#P8pR~_nWzJ}zI8j$&y<;5z9en_`x8yg|mfsl`
zf;eLcsn6n03#ey%W?K(`W%0dp8BQY3&ABi|Pd~6@uZJ1mq}=>;Cs>9SNWQ0l|GZ)i
zvop`f*xxFcT93N~F^?PoLPv>sC0u6Z1kfWV<co1My8UB_&`uz9LyY~qp&c~y8^+jI
zJv5o}#fR!3zFHNfTFXUq6{hHi@&wHnibhy@WPOZ%-lgl|8zj*slgv+)YxX6{W4LBt
zmXhJxV|I-Fwkg?4x(W|IWU9vd&(^{?nX>Q;>tgJ8=%SfRnVmiR8EL-8eJOz}cz(hC
zBscGc*9-Moov?8+_PhU^*IfCxYWS4bP2jaW*zNNA_&<3qmtLZq$Ji&eQq-_lM2!8S
zTf=+KL0p^&%lNJci?HHFMqTMfHITI$CFOCqDYVcv&Z+4ykIUIdiV5Y95l^3~agPKR
z;DMnvu#zZZN~aimr&~m`_TW>iW<ocf$C0PN0QoE{_^ufHX_iXHZxsZr#MWqxYgkHR
zn3KuVK{Yz!$S(|)d_#nx{4z(2vG-#7kN?2S+l!Q#uqN9G0>cY)(f`PW0+n3*n4nFF
z@N&^bYfSKRNgd{Kti_Zy$AWRe<zwt;Fcv2?_i{W+q7_18d0v2ewt+obr-IEerydeJ
zO-}fTbRXGaZzk0+E;-4`FIT#|?T@i%DPNH837bPjh1d&95Ptv>K8=*{(t!z-I2B{B
zmB3YDXwuy=_HfNuU@fW%bcVFq#I2@-+rweKbnqIRaqD6g?P$eB_!#aGaw?5G#>lx<
z<2``y0Hn*?TqUMV0M~J!O8sf^Mbrqn-4SDFBDfQ^{?g*#<GvMU2)AzHewTwlGnSBp
z3!&toiW4}UJ#mewg`=3j#c>FXJ>%J;fo%j}P|b+C#H_~rT$F2MIKyrtYNClOP+5x_
zTK3dj;lO}qF*XJ;gd*;T`9DcGFlnTzEYGuR6>3%lWm&hf{oKmd#9FDD3Q&4#e=lWA
zfX!Xf7oJnn*T&df1F|O#@JN4vH1b}hl3o}P(ud#%Nz7_IDDj$;qza4>(hcPps~k=W
znWL2eyIYBA9b@b?w-O5&rcE<W9!E2Znv)M<MZ5yChjFv38FiPY>J9*HTXUUJLoKAX
zR4934j4hvHlh&(w3U`R>b`y~)??wBXp8~w*@}srKj~@XR(5K|hy9L*zckrvdqb-}|
zlWWkt&!F9E{)v;vv!3|`n*i$qL&(b|2NkUaqPOW&?YS;Ed8nJKRul<2^=NWcBv4*j
zasi8HrX}ymi?Jg;S>DZrC|p`H>o80ZB^_B+lr-7IRdtyViGWKbYdwg&-!Oq`T_F?n
zVQ-0tbG2qu-;006*!sad0WwI|+5j*F8IoIxRz852aC=<NlyNyz>juDo0W4oB#xCnJ
zw$^_D{t_$8Sur+GZ7U(QnZ4C6&awfY4z#{{Q>JIr61g+K;}Bx-vP#o&Qp1=j9O!j+
zyiW4+xU%jt)uMBvQ03PODmfsvvRS4g3|n1j;WCv8!%<nLs!i{&jj}XKk&e1JT`<Y7
zG({~I#-<cV|D?0lMupa{F9AAh?Y21CoKW?)c(gP-znUu+!OfDV*hCH2M$l{D?<&RG
zg9r>DmoCCBo}syBkc(lUGKw+l32G;caJ9$aM!B^k9gA@MO6>|4;Rv$`=Ryl>XTo0X
zrh;V(*|hT?7m=Y~E#^Sq`k`V$-C=V@-7It`>PQpXJH~dxNH1zZc%W)wYOWCxRF8^?
zsJ;-ljcd=Rh`F?J_J3mR5?*sY0&uyj`W7OND6&~Gb||WEYGaR^t4Q;>$4!(#?Jb__
zTew48d})jw#+AOQOJ!g_+C+YzZ6ZoS5<i36tIAFW?EQtBLbRO|(!^+rp4v&U0SAZ2
z6u8w|(4AVOOCOmQQbE=DFJ4tng7Y8`Wok#JE$L?bA_u>g1krz>Rr+!5&v2i(J+k&s
z1b&eD{({AVQB2W^<e|(%+hZ=eWzJs?=!}#({RU$^{JU5d$S~IF_*eU8oD4BnT;|n@
z2JDX;I2hPFE@^ct1B;CtZN<DRshCdHVd7LLkqVUiBj!CX=Wt+y-3rWy)B;O>slW%J
z5ci^6fjS6Mo#}1`>MSDgFi)@PFg2g|0BEbQn2!<sX<xfdRR^tHC);hRI&Tm-<-I4m
zKo(Ifeno_@)4tQoyP4`DEp-{Fw71kPMIb^lDR1R1C~JXVN&5oQrnBXzysb;8uFKUP
z)pm8O5=ih!TTqweR#<GzSBxps7kH%rty|kGr73|lt1TkJ0J@-w)}?yP*Rqr=V`ts=
zxVG*9bSr>3)$N6Qz!Rqhon8B{Bs&`+f5$Y83$yS2`>V1#6cq}XMaPca6myED$5g4m
zE>AzyjrL_4c8ZF~^rtQ(Tirpxn2nvP;li+;>W+<@G>)O=4C8>Nn+0ro+>pt@&U!H9
zvF>pZrDF$T4oDI7%4SdSUnjv2J?%CcARiV~wU|Tj|Dp9BXU>!RoX45>3A`&}+KjGZ
zOe-`W(igcRW~;0kKlBx%cI1JWtq`t@gAct&148wVl1ia`R*8D;H6d9j=Zc~{XrmLV
z8Nqz&Xt))lpRzk3u``^Q5lS++XEX>C0oMiC1K_E!A!f3QEPkvsUCPx=&1%LBF3hw9
zg6Ju{UO*uw75(4@{IV|7q*u&f=+_zd@IPV(*OxTFPaTdK+)VsJ!j6B(^ee>ny1~{@
zu^9~g1e?s+!O%~zxfuE>Ho@@XLovNP3^VUE7<$p?G$l0js&)%QFX_rIhF%TDFBr~#
z8Plu~!+cYVW-(%y7#f<<u{|1^CEmi&OjcNiDcf2rt8|q!CSCp2y>IjCIN4B|X4Jbe
z?W7!3N_$M`GbuKYhS)P@UH|n!iXBF2e4|j7f>Ufcm#3G*XC5IsscVWIh2WTF-kEYU
zy9mfEKoq8sW!s11QoaE9tXAM`HaX=Bv9bn-O&M>c@S2cye0129Oe^JIm=ni8N8lOI
zc|}2TL1oI7lmexsFAcvbg|%Dg(P?H{*raij@Jk(ZC2U$!iX3s3MWI<c`G_0|H^HCq
zPfV$sQl!+9Qq-nSQdCJHU^C<$A5Dr=Po`{^R+O){wJMXc?QY=X5mAbGN?52x>@G-Q
z<wT2^U905i$z!dQX(ZI<2-%e~Es5T-6$CKwA*CAsrXDFXFzRO4!XN0LGE0#Oyidw<
zg}YUr-HLV50#oASnrIG1QRlVNS}=GfTFhP1IX<4bp`5#!-HMj6YLt>K-`UBPENgYw
zlp(Ns_EQv32}9V!fl_70(5knXF%%`}GGkbkLS_uZl=7G{9FgNw(OSph@#2@Kl(Qa8
zNpKl3XSjTYJ0+nSRP{IYxmW9>tyx`LQ`=ZxBczRGIVhz@MYb-+k$TH3DK%s9zaKVn
zxRp|iIpLh)fZzN!rLNXJOl<FzhT@n0@7$TvNa6S^9ZqTN`CD#FNmD<T(43w`!ly*T
zC*j(Vb1CU}f`ZWxh8<o;qAX>YBQn2zcCsA?v*uhzf?p3L+a<NlROXZIXiw(nJfi}h
zYzGQue$LCcgN)1<rVyF$`$+Pu;GQe%_{}CKzls)#7z{}%SCY?YG58zrOn$A5q;W-H
zU^mI9lAdfA6Fw%tr+g&siOtFHOHHrhUpOxL1JD1$tH~dF{_*FN{|ePjjk{oZecCJL
z;bgO5M=4XJy^_0CmG-hwdH$O#?cMCA%u5KTB>V{?2;m|t1RCq+&)|U)^RyC7l^@x)
zQfK}15(u=cy3*bkA$ws2k~gljpAkA_9^K<k0>oGcLgi~61gM3^lsL80epbmP+2utm
z?dKFVtv4?KM52FS`oc^@SSrEa$;zUytkE4Ci;_GxVWpi~j3jjPdFu`3sR;h@&p}&x
zE|H>PMJNW(Txr**w)579%6Zt8b^|FZKa_62(r!$993O4?ZvBq^bZmn1eWkrEbjk!0
zOe;ms&ARqEnz9e=fn)0?ue76uV;`AvqqW>{MHmgkj*k|qg%xE)>JHHImem<Hg-E9V
zm`ovv{qfIb6}!zREB_24-7w3ZUrH3y%0EpU;`7|Xl?7y)pQ7-Bc*z6#DOXk&pv%cZ
z{%+37&%ONJG~MTMiktj_dMhtV{uK(pSjEkMV)M$2GHzWB8&=a=DXV+dfv{n<@EU@_
zT4VXjBf{US<UXdkkEmd1j$^lS95Xp6Yv6|~CsWpfq>?HwlPgf!1c-$ZvH+^uammt^
zlXbXb=L7amZX_kp><jKho@wV92Ae=C7>H~Y)9})biC=HxrP^7KtSsjexL~6xvK%+1
zydulNE3e3MP{)|4MOF~fwa7UJW!$YIR~ktD+luVy7731fcjhdS8h$DK$XUV&Y=)1V
zC1@7anA2<KENM*vX<<lD4)<jc9*{Qdkh7ct;?U2c5)O{e84Q+(AD59QXE0-vX3grE
zGbCQY@`=Knp~wL1etfelM-CInw=&ti%NbT$u;ZtsVPXQ9k}|X;2F08aQoppYfm876
zm`L%pb8|+73mE^5LpdXmbM=oupEDAChrTx?j#3g$XV5Ru9~>=$e|Q<(Uba7{o!i|W
z{!0mOhmmr8bV*UrIqe!*9PHLQvkPVFT=BP@j<m<ae}TO6#GFo2&~ITQhFCfBA`J2-
zgKAC}C0B|$)D+(M4(HT{Hw)9Xh}v2Nm8Al;>GW0&ErL0LDFXWCB*3BXrQ*q`cHVt~
z*2)<<m0emF&QL^^s|&4$kb9I6{3R~s$X!b`2>o|-iK%_i(+k-conTGv!w5ESb4k$`
zQ~N-Ka)0;SseNeYL2pso)R9m{PD;D*bJhj7v_-N*74vr1g({*BDdOJYS#QPhl{p#o
zmn_ZFjwX$eyfNz%Rn|t>+Aix-8P{Lxa@J+eW-Ur4P4)9xZ==N{2D=T?dxw;@!yY<e
zf|X^fCR|$!BnPWN{`rPkYnXT~IsyR{x~7d3iXVh)Ii#ov!$)VWM~fy+1o<U9>&aM2
zF9i+F+JJFx)eyqgWo`8Qf#<U}34gBc)H2UG#6LAW>o`KX=%h>3qIaZhX}*K{Gr%=S
z7QF+Ptr}&;AX#)80%NabEr7tqT@8T?(t*1KKHLhw)c8`dtVJGCaA>hdNX?#EOVqE{
zK-N+Z4u_UOV8kniz{hQusKtY%JUuvoZM1lhl2S%US=<#;0-Lgi3n>>3DZ_<7E-4u;
zT~grHNRJeFHOlit)@ZMI_%o&mf94bx|A~}QSo|kie2f-<k>WGPWu;R5k|Cz}RLoqj
z`1;M<;yVt{YT##p4j8U9^opm(jZ`MI^JEp;KY@{1X-byi#9)t<C6lB)F(9Apk~PpH
zX~`rdDIg>*8A7ica4aj4Ubp0BibHy8X$zx+mx%64>Y7!%(BfJ0{B00pl6vv7rF>#Z
zRt1|{93l6As3E<f0-DS(qZYzgk&~{AlIcKMl}y*e{@V0wuKhQ8mcGiUS+i%Z!-s7+
zsnO5UDZuPjK;QU1*OAR5H6D&65>RtHbi9@8l+@ImFv5!*^46HF5av2}DV)sPujV?1
z&ppMI&UN_U0bP$6|D{|<&OGVqsa&Uu@s1>O6;++EVCi<!^{{iDYGxRak^S-NxlXbM
zB_xrl5kead3uWUsQy!EJ?FWbC@uHVyFBhx$wb%2?v+29Mzgw%~_W&H9pLefTDol=6
z<Y6|8S{%yJioAO(h@Uhk*5%#HXs-UdTj$kQ5`n*RDz82))h|l7Xq49wQkM_mv)7ZY
zyoNQ^Kf=l@w3W-1&1>W3?6fhjy`&xyHi$KYy!IIG%SV8*R>QmpyfmwB<_++$A8MC3
zkY&l`V@VU4m^VuNqC%qU<c;FA;PQ!Ka#Ai&w>d<&^*))$6Pnm4VPZn&jYHPLKLI|W
zdfrkGf3@Vi1D^k`-g$?qZSHI#%1^?iu6eT6VWySK8|&Ak^TrFm^TRS35A!C8KP_y;
zgz;A1WcAC@qvf4%%WFi_!z9Wn_ZNQh4)c?LkNotOpPdNcv#ArDF0|B&c(wT_IK7l+
z+O~702~L0J5i8;eyzm|YI<N|1%_$R{3{>@sWMVEAo8XLB)oPj-OA=^))&ys@Bxyik
zP3#0`4MJu62yCx6!P$o8S*j;=uaLB*Lnb&=Didg;&!A0k-1n(Oz?Pky;H=O%oxUJv
zl&U?rlvHbY3w}nn<Y&Z5`tto>WG<#Jub4w8%PeI0*e5fWA%82x=4rhrb2$^O6%R**
z4Ye{?GV@sRuns4ihk_(M8YE4|Wj-WGULrYJXL?yXSIT7D47ebltt}`dL<~mW=Wov(
zB*$<bi42odLzzP)8gWLnGlz(BGNPS1q?GY<YABP(B8_M#P01XJzt=fIJd6|^eYlr?
zzcaJ1_C&!uR*&0c_Dvv9T1mOCo!Pgx6fRA@|9qw_+0>eifBhrk=RrmM)A|(gk97Uy
zx2)!GWu_>-qV;;*nOWbZS7;Xt(4)P+2(wFjeN{VQa^fkoetDPn@oO{hW3~B_msP)V
zRPB+=+B#ukSFEg@c`w@jk#9;)vNG>wop8k*nRA9ZjckWXJsL~{hxct{he+x9LRW07
zMmI!nJemky%i<gT05a`B&5k$v0aY6Qi5QW0{M_g~E%Yd_ZCMwNH99YHDQ01l7LCp)
ziJ#1)-feVID<y<H)xFVL!8(tKwdsv^(v2UT7dDYkls9@x{nC(|wvclF;YOTZvmf1U
zQm!nLGRLGGhZ%XiQ5vr%Y(|PLJ^qnhqyONER--hrGxK?_Ts}b6sJSW;Q%VbiX9CKi
zQA;JOu(Qw$vZb+V{nF@xib9w4=j@6=wk&Hj2;CZZP$vL{CKKwlMc1n0-VFrFQ~KGx
z;9;kvK+sE)t5<b1ZWQ}W*SHo-wVrHQA8p$>5HwmAEVKVb>5_glgHd*`P*H<{AhXsY
z7~Tini^~E*6S2ykEJvVrLjova4J_Ca2%05I=PEHLuLOc-dx4AX0zq@UKqrWss{x(D
zUS1RkS|vdmxHnryF9(A9vi$qXuW+wGmiKd?uVRSmi;QIVV$t<G+*j2mKw95U63+oJ
zcw6s|*s*~6iBN#A9->fB-k`g8cp#`h-8A<gfR?!CE$AA1Ec>xcVw1WCf(96!YSrsP
zY$r>i50ZW!(Irj=f(9C5t$Gg<-G}|yT(->5uw_rWzXCzS(1N+mSOL#{pZE#5(GqwA
z&qkbF>xuY_5^}#JbOI+!%DF$|`3ooab@p;y;5E325~~vkY5~5Lmq4*M1*?-rO_mpH
z_R{B*@4@jnm%X<Fo;J8N2~a@ti@5!GBIOy4z0C=|hC71>+snH_Aa{sM3!vETmCsA2
zF+_iW(}D&vCRN1A$*RaS<ZSP>%5Kg%vg7d@zgF2m`!F|W3d$v=rUWEsW1M_^V(&CO
zP?ge})GUj?#tHsbEoAp*iX{h$Ch?o$OeIqGH_oua-trpc{3<CVZiLO%!i<oqR*9eE
zO5D`1m!07xX{$)f&f#SXH<1%kQ_wtQkZ=fIkmNKfjJG=ok|{-zoN<vnSeVEOseKE{
z*&sKbM{B)8<_N7pQ>`XS##z6cLT|h%t<GFcF~v*4fmJ2+)GZt{ZpSgeP4W4y6f<wf
zkx2?{-rQohQp}PRa=IUmo!!mp*UyMgk@)GByeHz@T0Ft_J#u)qyvXZ&<i**1%dd9a
zG^<juY)`%>Wg^ZTk=e_|1A^=k@r!*I2pV33jf}lLXs<pvo<Q;KB_<^UH%tPf@l3>x
zXXGRo_Od+2le_gFohD@#X=Z3wF4~twx3ITtt>0fC%k#ZJP(#t&c}@w_Asf~K)j<N%
z!5Sh#*45BKLx$0RbvZsC4N~jQK#-bddb}j^sX$N#hA}xR9>;(4rfA+D)r^j+(7@=1
zlt-XuF(kh7V<0FNy(-!vWiM_gz+O87L6tn<X@IZ9PGlk*2&xJttmjxMZ{seKGLRVv
zs^$SvMJWX|KwLx1UUQvVtd<6VHv((#!sKWTVyv*Kz-Y+4lSN=`Woq6lMPN7FbpLYk
z(P{31$7`;;VL6As4;ABR#o=Fmy>L@s=RT<~T-x?}Am}cwQmCm)xS^Vy8CJkI!^`=&
zJaCI@_sxLwsCKbH1HQSUEI$EF&~E|X7Rf1R*z@NCp)q~)=+(y9%zxVq2EK({5+u81
zUM_kuP3$mqxbHDEl};UwS0;eTr-3C)1HP3KBVz)Z*td!qlxSip=BxiM=8xM!5upq~
zKDHW6Wz*{ge6*tpNTGX`1S2`VeG(AKxzH-$dqyfgjKH!+0pGJ;;E`DY-~K|fX=L94
zk2s<9pau*heII)<C%|w`0x~g@+I{N<q(TK+A>l$&TEKT$*JcUyXdCbykpNPqcUPXj
zGvFIm*cyN8Dl(9I(f}|%?&*MUI0}D#HOWlc5o#ca1Uxb&;A`pSd}54|0AEWT4aq6B
zZ7EAYQrprc13q(zMktEB9`LpGxN?rh*XI-@UETWvzK$MO#uW?rdTBB}u&}dyz{kx)
znyJ7#@gZ9amt`1U^}Xq8zI&@m18`cu1}>qROmRrxRLw6dRU*1n8^jW3z?aCrh79Ye
z^Kr5xDLhV$2Y8i$a6eI(S_w#FCN_}d1cb+lEhRut6dotJ!*}W)5N)k)%$OyU)Qss#
zni&TJK6%qNoHdqI2ENoY`dV<n*U-!OF2Kvk+^W7*UPQZ*5pB*>?B!|jm&I7KBJu*h
zrodNi6wL@}slVg+sE=<iMDw9et9~dWWJI2*M>t-e-cvtXyZp(TmT%8<WYznRA@Ufj
zbuyhZA!mTeM;~~&opH;Y3+XAN<H^PDgApha8T)ipBcm8W!ygOy?#2kJEGFYk;r^Ny
zBiPmW1#nZ*Y4*r!lg7c}7!5%M`RZ2^`^Q&Sy0nlA1M4@3DXnq<(h|a)`uF2Hxm+#h
zGN(Q^OZ`lS2_MG>RzFOvFsQs?kX0Z3UVnxR0cxS3c~|fFXOg8PSu9<8^&ywNZf=h9
z$iP$iU5leUl`m79@PLm+2767AJNVSkBX15|^|55FBT)SfK-Rk>Q2kv5cA624cB*{c
zVg%zCNe}q$(GiSxs(%2t-)kpqtop~iCOSs|iD+Y48y*b!?h=u|T3}V8w$?di?DbwE
zs(0~#t+3UM2fhfTP<n3t_i*yMu}ER0$FG|Je+2`i*lV75nZbE7(UsPHD2My*5+l<2
zo9N5`j4n@fS#P-XdAXo3;p>2}m`h*mi<G}y=vzBn=_|&{h0y1cQrsowzG`5rLfdGD
zDkN{=mKWlYgHE}xp0v%{2i&%Cqf}+l&|XS3&GE2+qi1^Le%;#d!r_clz{r~Wyir%q
z51zWBzdAoOhDgzgp;ZIUFT$|{1mKkOr;;aSzHZ8N!~ixU$yyl+5HJtqlHmpAF_!sn
z2OOS_x9oLZxvxT8_$T42R-9LL7DsstKMy#k8EM`;O5n}G1h87yJx?J2c>?ICH;;yn
z8^#dgoK^D7K;gV5f&9>MqlXaly2d~gTE{V0{|HF^&LNdrzu|uboHr!dReh{K;C!qB
zq5S^s0q0YCn^wJgjev7q1G;EnU8OEv3j)qFT9?mAzAZN3m<>Gw?UMq|exv2-^9|Pr
zoPP9@O;6AlDoWFDy4z@6r(YQ%gmz5F$w!PNXTLZ!(JKJxz4wibRpZGSAcRTE^|)L_
zq3`<vN8gRl2k<KHWObuG&C@H>q}$)ccGO0P#K%VmoXI*Uknls?6@azR1e}>3@K1oh
z030<i;4F244Pq^;K^eir8tsz<h~1nXaP;cpFk+YtG-wbj`(6R(G0|rN9ETetS-O#>
zhGc0GaB`!BkHk#JO+|rj`3md)1ybB0_Rthnv|CXrVgx@~USP9yqOrA!z$82q43hD(
zaF0aI-Hhnltyt(L5tcHu5w-BeVABlnyyYWgm}WAFLcUKUO1s6am56DQQ%9Z5TW0x|
zIPwl9uY2`f@}_IvhHhTD|1P;&G1bX%CYusDU@kM4Cx5tABCjwOM)x#GH@H<7q-_-G
z9D{U&T>BNIty76}jnOTfLKLJMTHQk0Mvxj?b;|}9Y5!Z%Z58Q$H{Yxx5t1)NKg+%4
zINb`KZ+JqYb-0pg9E{#N#Lf7^KQbCacq?O^G}Z<_vto$lyqCO=sCn1Ak>7ZcePvwS
zs<RwH#|%ejjq;A8Z|~{?i<hb#WTsTWVS-|BtSbut1!U!6+<XZzSTuMH_o(3C9B%M$
z?Cs)jbpd?0TF+3yGL4U$UiL<wSN0}<ZVb(*F^~LLjLbTbmW?|k#!POrv`(8Z7=CO#
zb1O!~ErZG^P1oG`uWrT9(s(3SXue^}+~i2S;WX3;TQVg^pjpZmpfad!%m_Gqt(;y}
zV6C~p{|`lTY_3<$=!1H70nzmvY{0FipWQzuV4f`y;4$1Gw`;aEipXm44!}1VQP%b)
zYa;#V6xr^3DByJP`uk6$_yJ<qO5-E=oAwG>W>!4W5n|Ua5Y__~ZnsR;s;!}F!n92v
z7N+K+ji3o~BYwFRQCnuOYvTlqq-bLKTCicdS^K;xXtHt*xt8MS+AZbqtw)590h$e#
zngJ(XmbEr7G}9fYN@*EJ7?(D)&8&kEyA1BQw0X+yAa7i9QwYMhE(M(8ig0hZn!QeO
z_{+E?E3QPLc-fDqRq8SZ+QKd-tFgQJA11A`r9v;AbvbrD$z_l1C2RJiY%D455%9Uy
z+JNJ8b$Eln;PcG~E#JO?VnOts>qLhm>H8Q28v1eAbmcj7_*B4_)1_MmhHJaqA2T@U
zREX)dEz=8~e}z>D*q=&3#_4U*0sAuzNY#p+4A}dmYFnN)yt1G5c(vtG!x(#i;TXPU
z)9oOaS6&K1xcZNPy+aUg^|=UHOEt{ja?Q=X>lVVFZU=b?-4uc_E-zrORD|`7F=DS&
zHkvWrenP5}AF3R+S4kkv%Cp(Q_%mQX6wLTKcu2rbz^2~%qGj1Tq&>E8GLsiO0TYt7
zW2>QXwY<JOLt17da;&CdPh5A}rq|hkT@`C1388D4i5m?tc}KwZ)(IM-hZ@c`<E!<<
zXw@|7GRO_F>}@MZERB(Lmy31_i8r8p_Vh2Xw#A8k%rBBc5|#wFRiTyl%peU4t|2RS
zzCBXpN!)QHej6QgbVreRmKyl>`5ob8m79Fq2ybh7t1F1I&^0%@(KdSJ3+ssQCGkdK
zZDYicHK%Q3MAwGgl<u@`Uy<IM&bp0!{z8s`Q*p6t?-~#9`nmOK>ec5hNINg}2?fIi
zt<MEhpQciuU!@Ad-|M(fB~TX6_qcDRNG1rTKB`T9O`5<9UMtx)K&DX8zCFZpI-n1?
zN6BNlc5P7{+rv$Lw(I*ZX2jWE(bQ)<3xd=~N0jX-@<MHtK2;*nA@ZFqsSt>^UbCF`
zWkEg}#z?cJ%M&%$)?Y3C=G=A}_7q&D@Dp(6n{(E-Fh~fNDcf810(M2jtN9LLl8D!i
zL|42j)^f$`Z)*F8D_#{F8aa@U|2B$zP?D4RWm!}X9v!gbiq!>lA5=6yXKJ(~N?Oef
z2_Y5bv6$x9QG3_4o=s?FeYv^5W%XGTup_0T>_~UD0Z)v`C;*kGy9VqMLM6;M^NXEL
ztEp15OVlt_%6n3cs=8E`xX-1sCBUXoxw9#mAq^@A1ne-Oa%V@OGF51y%;Q2O>=o`K
zIvI9-VP*5~ZY4e~5-mgr@qMsZO)<nE*l8C@F~?L(5KJ`)c5=C*5Pvq&#U{>iz<>-j
zUvAh@7>(H_#p{vm*HZ(32v2tQDXALrj|64O1D2K9J@5-0N$(6m&F-DQBq>LJsqHf4
zLF(QsNp<`$ja+^pZH+QH6faBnWJNa@?-ls9SRWuM)&&aMd+(e%v<03DoFvyp`bNvh
zw5XIMT`Vb@htr7Zqorc7%NBMN3;A0<-o_u4Rn8B54`qA;!KLh}Y(v@iOn^jRHQIz*
zZz%hoii)~w^b+oQmol<KSx9)`$J;4e@K4I<sf}d16B2%lvR>MiJj#Okeq8o~sV-?#
z-wk{PY27Lk)>j*Ll`*7!<w?<(VEoLG_LV0@3}}shk5m5tOVu?8JSnts1P$B`0g?su
zT|Q}GSImVazB}fSgai!ots+&fnt5QW65Gd52i|As+tozR@CM%3R?|Ju!271PVC8+U
z1YUsNjqTC>?WJ?>nU<s@_zOCQP(UoMoT2xHCWe)bVQJ%fy2&1RN$KSoiGVCx%M`2e
z{kRTsveh0qsAE0h$+(G$2-Q!4Lm`$`vcP#Pgc}9&D$>Sl<g-ft8aT(Ow@dCn$%9CZ
z=i+8d-~&95;#SDz49!p=8SV-Syg`QDZ2OoDYe=)o%dnHcc8e9rJ=2&uT5_d{U(0F|
zcuO*^rhD#L2zNJ?Hr;2p*uq}-Y$E+`Qx((ic26%7gFNbYF6dR8P@ZNJ$L-VwBBl{@
z=bUCf7B5PfG)-hFgkO#;2QE|ap3+>o>hJ`_J&$`%0?YB7!0~>((_s#tw{e#$64)X?
z`=qE0%lFo+cJPRtppiL2p<CpHq-Fhek=cj)sn23}G~uhA?g${%Z+y{Ezo&j7^}dOy
zA;_mGBmvs9C4fD*D(;*3rI&z0H{L8VIJDY_d=<~Ka&2y?sNNX!O+c@5?8z~DW2;f6
zh*Zjoz)D1FtqgMdeY2<cnJ%z0+4RoF-%`UbjYzF*YD7wIR;GoB7|Ck_B1UOda21Uh
z<+@^&)(%)pBSv|i7;(=>q+(sVY{<#46{4Rye!77x8EFrZYnm$=X-fz!@<eGJHVm@2
zF|d)wd7AA>6{WO1(&Ts<HWSz=EL%rU6qY^x7|ifZ{8*S3HrbkJRjO6iFoxc^ZcY)l
zwLB+mdnCkVTL-otng&K{Wnj>5iq`wvdxH&wM!F12dxj$SB&o?Z5>_ZX&hYPL+GXwG
zq6Mtg##r)ppr<UQ?Cos&de3`>(<OCPsApY40JAW%deZN$tN2;c>DgM~VY&*`TX?RQ
zV)~q6GgAVBZK1)|o*_Z6LgY6Jw67_-id(V{)D1)|S}=&ObK;(o403+)+@ZLA%)7Xy
zkuf3DiZds><+w%J+Hptk6x8@z-ye5WFdbmZvG;8>Q>*Xejxyl$W?KZI)OX_KolJSb
zP3pe6V%+0ey}!vC-ZyTOkgEqD9-I)z>jaj)FHN;koH<`2r7gb@r+evA+Ky-A=1Ix<
zCi}bu%`PKJ{q=DjF%Daz_pRfi{+^kZBx*RWBa;W2AEteV`+y9-2je<Qz60nGs@J3#
zQrP|Bxb9KZ{s)v<T;{+rR-BxMlC`HM#5RfBZ(G*lpg5PicUhJIcKs)=cbp;ufWXJ~
zm)cOvCL?eInP9fO4j_a|{X0(HDvHx>b}4r`Zn1UI<~Av_K!|p*HeQJ9QVij2DIK0A
zp=T-=c6hg^x4huS+}a9umyBBZjq-cuf!p#R#jQN$CNJdO@%P^`$Z^PKdqjzwF_nXK
z46fEAuAy+QBD4+>&OLjd;T$^)%DE;_LG2d9xrQF+&I3CcC)cau%t1uLH*vCP9J~u8
z(Q)!M!?+eID3;Z<G%mUV)APP@EyIaN0W__Gi+@Q9A5hHqeL;)5LcJLnu|A}yGPBkR
z__H-{i^6VhIRhk}E%|urv+bD3So!jEfD<270D-91J+1<nTS%w5A=P^Bl&MyQijtU;
zF;AN&n`%|4s(CH!Kdb2jMj$HPE)e&)0?~8^Nv241n-hbC_ThF=wXtz=)m;&2dJ&kc
zF|=?+;MpHc-$)ME-av18&SxMgkqVKtH=x2Irkv?HsU}9!)-Y*PQ3v!T7}G+GNu=aC
z7QejV&Wr~wrn)hMZ^cYeX0|wEq~y7ow_@Bp@47McP7fP-P*N@FF!}*fnH`+ssYRBT
zzW-4?8KDAcqu{cZ?Baz#9PK<2Mh#?beLws2oNSABR+pn=$(G;QUeV4P#9-Z11e&Kr
zJ8OkmGErZ$lmP73z=Z5*^Ze@pwqWtXFq*YJXLvq{cE$#Sc?ek~)pX4tI+Rr0i!!+<
zMn*d;B#<u?Y$<X!#H~YwA#mP^Xs25tfono1ju=kf-e>_1j!I#DBxhPE=hUN}A&OGw
z<-6{Ub}-8<`}xw~+<7|MDJOBV%AdbA+Nq$K2*kV{?Ihg6^m%IN;IWfHp%Rj>m~{p%
zI|<1fqn&C>1MT{Jh(2-}?IfyJ(>;@O`=g!ujKS*$>k?1R&!TOX4@ujH+?6Xw+xk^o
zVO#YY(N1xtMCy9)@M!zDSmCu4X#Xx4QAS94>Y-@+brcp-Z$Hl+29C8y+q&AXOv-sV
z+J4KFZ@oscj6k&gic(14(j%kogI-?Q6u49j03A1JB8cFxeYh-GKdxcc+~xLC>5*n_
zBicTJ)%rXg0R6c32Y!;q!p(N|qHU%?mVF={bnwjPI2sK=0HLsVRuZ}i99u}h+Xq2H
z?479+<CeNhORa87HD4*5mMfKgqg}MUMhH%$gf-z4xk3t{jErM0^=s%$NfQas6!sdj
zSO@eq8UpDOaH}GB@!n`BD34W(wpVy4KXFm6fX;});iHC51OE#8rIqiZTp=pQL%E`f
z1Z2f5eO0t=E)Eu0_xA*<>zFUBhL^(AY4Ji8TH{OH$JK^RM1bsRkSbH2W)Zj&q#RcV
zpiy*IGyXTLbGK=QnaU3U!*Bx-tryJtU$o76HE9HN#{HK8e~rsBy<AoxQ9Jfa<?!&%
zoBd)UJ-!EfY%jVRLYLm!bckk_DOuT<9|lz!Wwq+`J3E73Wu>|7%Se=f%RZ5WHXTF~
z;`>J1qeQ+K`Dp*Be@EMVoJmS^@%7Qx2^V~Q;)>wY%NKh3LQwR(h2q%EXgNp)igdT=
z_Dr&9aYn4s9iwetY@!M-_Qugltz1<h1~pwsVh>c@=#E0ugY~0jcbZ(=+*~zLM>Ma=
zRZ~X7G+AA!^<K1Ho?#_61BkAHyi|^6sXrSqdQRJF9xX5fV6|j`>|RQ}n{C9c0Vp*o
zpX>sU0z6oind;(byJ9rhn`3{_1hLt?M8Qa{PyR}=n_h{wOH*w33*^)({7Vm3lcp#w
zS~DLsZ{sd9-t##x-~OM;>W%k8xHaST=(AG2m*({|J!W<*F|9ftj{aJ>N{Tv6Y-UEE
zkOhR7))G~2bMz_XnH}PByc^azdOf9YAHZO6O~%qAhg;CAKeFtDUE?N9w4$GO+osuf
zknt6wwzH<?YsQk=OS&2oQdh)pC#HzP{w*~Nkx$ZFJEt-;V#Gff?M;e&tUKxSr<TUM
z+bajFlShW2Hn{@cRob9B*i=!~l#yyzoWL}UAzMa>RFW~R?wTLHfoZ{^E=b8;^YI-y
zi?TuOH!m)MOL*wb&1Fab$VYfkG|lTu;43`sac!BfrtMX9#XzwpdN#2KPta|)KLNjW
zGiLj4#$8tb555NtB26545L&m3)h2M+1~YK8ws+0I56S+Q!kv&r%~0S*hymzEBuT`M
zRGBbP583<D)jCd5d4~=sqDx7_Dild%zI?QY1|?s28nSb=%#04pcM^}HQJP~SHNU?c
zfpNaau36%F$oJTX>RIG_?2LMHbZ*b%k%}XPn(v8j41z;HvEDP{B`^r4ffCq*XFP74
z1Q7P-vvD&ekcVdlZkYs@;8}-TqxG3%`PTG@sBr2-QEFV!#NpDMJLm(2OPX!aIJD?<
zr+{y<qzMj3@8;O`k*3n3hoSztw_T#2`xH#j_}upt^E_^+1a9K_2KO0B_m_~00^LuS
zYe5HUQ9?MkTPzFV)^VdH56jk<q?DG#%F3`+ig#QW-2%}k3yF4(@Et!>oejjO%s)n@
zWFL@>tSw7*cd`y4&3E`WxB1$nN|Wt;LNqY(PgOLSkCfAbIaULLY1*t$b(d~J;-|E~
zz@DdaUG_-f*aZ>~F6=n{Ws(+sxR|>{(V{tZZ0asiwCF^jgS$l0g7#}M@U|t2ok-t*
zFH!sr0}GcZnlz5PwL~GBBk*<fG|?PKeh!;rMb8txkspQ)^|0|x1h~-k0$rqD$u}xF
zx|ifT+Cy&*MEAkq<(i+|8i?)_F#f^fr}+iW$@}O&vFaZ)aRPqEOue}_>fPv(g7rXH
zhxF-Ibf)^Hgph*h`dY#UQ$m9xB{V2fLW3eDG|&>zf#cUkH#BGa<+;!cS<yArD)KEN
z^yr0XW5qRN3+vY?hxKCn=#_Ap^AMmEh5~uHvEP{}`&S0ZrO!p#y9ujpd~6ECV&hg(
z_S2A{hE(T@QTAR9gbvNLqU?Q)Q~sdPNj335gXC+DsXs=Ui_`gJuXBL_0<3#r3(G~>
z&r#`!SBU9ci2(f~;+5*2Q6|b_+720vUwS&qep9q{J{^Y-nY;sS_b!OCXG2@WU@CFE
zEO=`61ah}P2<&(^%AO<id`{raF$7?&22OrSAYB67z>2cxDWQ4;E6Sdadabma5?lGA
z%<F1$?(*1gK$a|xvX|0{8Fv634|74w3`nSy6e~<nEdK#-N7*u`I6feB(xB0l)RdBr
z4<KQSMNxJ$$TEjC2YeJ|x5QLZHUxf&vRg5Es7@E6reo)VXmePU&D#a`@$IIi>;&P{
zV5R=vic~~i0XR23%C4&cmw)D??bD9SBg`f1M%f4ynF_3mhbefsjE5(hjsSTCQ4I0^
z%l1hA%{W9bSyDpgL%7K>R2UlpXB4;wV2(a%6=g@dz?MFMf5&DFnLstm-y^8GWshXE
zECs2~yKYQ?6N(xGi7gXMQ*L6|+gLs7d+K*$0peS8O4JXPxO0+>fBoN4KMR{=5vAeT
zD07r4LhCKe5poy-!@rKYF1dH=iz89jSzE;~@5V-5Z%FQ8l6KfHlr#D9C}tG)iPPXK
zaVko0nq0LQH>(VYdS1l*0FecTh!;E}YAlL6i2p=^pcys48!H8WMJ;uix%y{v@%*G^
zpIkUe`d-vhR5Qz0Bf*lN74@iKX@I}jq^NBCCl?|+C;NxVeZQ#3wGLb#ih4q1J1rti
z<z#IFd74l?lWJ>Ws%=G6?Ql~GRU@xchwb-9&7dHdVxPPwb?6y23+C6FZP3g%e(maW
z)Gw4=#8fXaEov6Fth@GA{@W<&L$dY-$Iiw@je=vR%9qrPqv*Q&7DTn=sBtR56olkW
zP&ue!I<6R~XXE5%<>5Y2-G!}8YEC_9Fmy*{MGR(8rn7VxGqXk*ov;U1XsIv31>RoT
z;!^lsPe#>{!k;l&YxpIrB(KGobLttBJk=AB)=Q)GK9sDoK~ghKV@TqaHNhsV%AF13
z$}6Jo7Q{aq#COx*4B|4Sp`)8Y5LdKL5&w}~GOZd@yP-YYbaMOUs=DJ1psFlUh7KAu
z(Q0UqVxj2es!V4a+UAC#l$#Sm-mNPd+Wow|lia-hG%vNN+mcw?!yL6;KtQB7&BZ!M
zu?hvPPdBu?d00-nSa=)*3IxjlVm&MuIuU4G1j_@81$EIgsiEDI<g`r|-?|}FGaV)%
z{R_Y&-vuDEntUsh=@fLw%hf5^V+Dz{zjEgYx#<L$<@m2w@)UfUWj+7Ks5G(HUl|=X
zX3z+0RMWER$Jaqi3L7(Sh&8Gs4fD!)*P`z+MJ)Pp)kk#}=oI|>&yMPXoJYK(H>XE+
z6`4Pvs$yv{d|Z|#KQAHgrCaAOa=CE&_mWd4TKS7ATlVSSk*(_6@)yV8{~LeYoczVe
z*@DXW56{kDT;H<K1mO=0%3p#&4SEZI!@KjB+~cO1Fd=_QJxPP0<uAJfXdYC>U&xoK
z{({Q*pO~A!f=u&&$G`1p{v(n`Zar_hBmYtLhm9U=<!1viFBa^{&++^*NAq*N<T0o6
zb0I*p#+=E|E${kM&gSP<5x+7ZUrJXzz7O*AStdMF0zg+gALU{EO-JWH;o*E_W&SGl
z)3;xJ0nL*CB(17N`0mc%pne&>UOg>wThZFIo3y~wW%HksSShVfRQ@h6SF>01cWc{`
z>2xZQ>)Yq=r(@`Kgws_4yf8gqm(|f;gGNlU@=ww}O7F${^G{(>EXboVNk{Mv`k4d;
zS36|zv_Ib#qhmopK{<eep31i?s+uE^ypce(pd)a6OTHbe2qNfO`F2Aki;ol1Od1f(
zx)imzWxm}^aY#YOHFfKvsU;w(Wz626nQymMk`3CS5mH_#uNCFn!_+USDjmqT<p~Nc
zg9Kz)uv&7>5}&+Jy&xEoOka>c))mP!oe+`pTl2@>OEZc8Y^VIO?Eb1uE?JsCwwIIx
zUj`k_&y;eeDAIxLxp>_s)u~1|IwSYEFBMC5lGMZ!I>uE`b?VSlG%D>}s#7ltRV}aB
zj6a|1Btu7~<&rL`YpTN&bh?Zsco(h5;9D1!YX1kkXGKlk`6&T9_@bXeGY3v%ulOb%
z=4}1YiL8yM+J7=soo&n|57wbl?Z3egAqz7jGEyBur-7uDRL9odNzBGiQyoVG@ce8Q
zAour8b?)#JXlX_01>saD0I{Mc1jXV$Le{T0Y{dAi%vNW<mTK=3>W+qr8Ifx5<DBT(
zqhNX}G}S&P?I){;$3954k9z^B#0i+Q=qFd5oV_HqZ=IWpf+B;gb;2_HjIvA@EmLiM
zF9s?of8imdPadN>dr34K&lcCie~QSVyHo!ZRrXrx(7}VoK#h-L7H&h9QtdmWcoE47
z*#zh*jNGA;CM;GmDCUP9kQDzj^#i%C`&x&Ru<?TmGOE(zN9(5Q$uF^L9vYkasn$mL
zvb|60b}jTo*x*4KR;rlR*DB0m1U<be^)fIDc~0q|_olv}WPk%QUQjCVPq~=-G7b6K
z3H&QproJxzS0lp6_>vL_`0}FEgW6Jt1@ZvF!w-o*wNi5^>RJV|{A>uwQv&ewXwMU+
z1uk5BSU%-a=iO8`F^~kye!Xi+C3gYhbge?kNdu9f1=V580kvmR7osmJEjNVfEb4%e
zzgWqSFtu8W{jqQxEyvKkJ9Vr}#_JpDHziJ`jzvxKVFy{&N*#-uWZO{er&Kdx5)?3N
z0w}Ikpp**NQ}u@Z0Y!H<JGC$9WD)uGYb9kRFtsmQM8B#5<N9LifG*QY?T6m{XnL63
z)k^JengprbHRN%pGEI1{E$w}&*yBzWSfX>EeYGEV^mbaB9;18Qk!M}>4CA>3(k69%
z+(~A(ZqC-9yYF_8>7#`y<g+K~-#l)Y7D{ExaBhS=k`Dk=dQ5+LzN~M2;(CoGQodj?
zWk%xq%8d4JQseRG6Sp8-4PMtLHWIfgXTvfk53&+>(&cK)b2fEbCdy%aM3um-vWd@#
z2#Eekub22N)>qv(Evo~?G)@%m%lXJB3M^hIkiq`Vwn>U?Y(?6A4yJoLu33U29b1*u
zu)5_4++eoWCh-AQEr{?^+yZuHk>AAZ0&CT;qvVO<#K~nDauYao9w*J1I7N~Rip{tu
z5gd6&&wO)y+Bv377xybM4%3(ciJ9n&H{S9AWyy=JB&(&|n!R`t&nvhS5)jXOxVJHL
zOAcV|<l@(6b>-qOK)=C#&054N_M&R*v1k1*J<DFYT63`!QOh|RWR;wnIF-`Qvj$;b
z#O`YOJML!*khB$yZB<GFNPVk{xbjlkR4FapEv-AydvR5zw0*agW=`Cfv7DXU0$z=s
zMBW97^Evc-eln{ZJ0B!lGOjLJSF_TwbDgZ)Ooc>_cE~raR(&WZ`S@k}k;r{+`}`N=
zlh<@xq3T;TsZ4+iJ|tjiMUxeIO_y-7LTKS@x+WF(a}FRItq_J*?Z7T;K&vKMe65Ny
zWDUq}p&*ux)(=HCcrb)z(K+ra*tByJxjM+K8NFGDNSVdA(8GO`b#HDpCg_{&#Cbb*
zKxHeu@xH{N)aZ#B1oa`_3-=hQ8+wnCx>l?xx9VY}Zm8_AATq6{;l>!L8}3S7tL?xx
zi`;PZ8gwEzT@r`Wk{9lf_Ii})mvApgU_PGHxKk3y!t(*{9cjrf(voZ3mK2n9hgRRw
z98Vm-ttC}%Si*jSh0(&5zZTu$hmQ?w?No2Ql-SelRI8<WQ>9Z~EM=<KGuEx%pWy$^
zRIg_>Q@z%xl-A*<)ATOubsFtsX3vl4`nY;hTjX~2V)9ufS0*yJ+82J4Iw26PJLBXf
zSdrV-V{juRfZVp8i<>F6UHK2WZJkeyyq~<9;~qtNiIH1@UoKi*4r1z62YCQvWX;5M
ztu1=&;=q!sInpahB~e?bp>)#7w`W8Xq_>eY`oLmhTPj%3XH-}h)0PZ24HHOS;o-#k
zg(&wMl=W3C49fbt;fDY4---1ZoY3g<W@DnPk9jD~dW#U)?#@KaD-uEATaLk24PELH
zSS?W^4Yq1@LJwPF5o~n~fnb~EPb^=EEtOm$T@uSzMC;DSkD}GRlzc-^Ma3XAjDI!p
zuEJbnO)jqcm|W=dyP81TSd%Nl<boLuUPq;bj&BgOO7k)y3x_;iez+9qm1z+XD>(E+
zan;WgM37h?8^P!k(xpKVZ)w?=<h8=rCp8Fqk^%ivSv86p1Z|Z19N;SPFpz8tbxpkd
zw;Kd))|$HZRgWo{cbAImU<NvFlU<5+ugrlE%i&VIS{3syLhao1{L~f;xpt|cSlR;{
z1kLd3eCd;LA$Nw78$M*j1PN3TUWdyo90ZW-ar-G-sxgy(po@C7ZxTS3)clef%`IA^
zd8S5#2b2HN|0Y{($)v92F=R_kF1nz3hIBDh$CG&>Yb%#N(VEoj3hB=sYY>zI>2TJv
zFAGPe%m5;t^-TizXAx*3(FCe~(I6-lGxG9ga4)>xAgCFwt5<--N;U{;u4EA-#Irzl
zt2uZMmF95yoYpOr8wd@8Is?4CLl##X1a%kxo=~k_4@E-_F3Y4zD%JA#wrF*$Et1QK
z<T(7?w+xGBL~aA;399jSS+!dlIO|}pUNzra(7@5w&JUF%m<^mQSm1ASpv1l`>z|b3
ztS_`^-j?;xA|PYY6sE9vxW%d*gc^FgEQM~rn4wFdZ)a(t8MbIPHJ0A)u6Xzg?Jd;9
zZwDFFg(<`udiX%bB?`c3wy%WB<DeOr)KV%AP4hF}!RoydiogBrjQ8kNbJ~YZv@+hu
za-97g(45s79|-@%{(keLj1SeXT`l7yQqTS=Y{En^UKRhZq)EAw@o{1LZ!^A9-isNP
z)HUN>Mr`+G+u480%K5woe&;!4w@MekvsH}wD<ieaerJbQ<tweVett*xc#M5`rT6V1
zQ{}=Gve`Smx8KpTW>-GbB_Bd`Zf&@_E&a|^+Um+@dJWR=OjB}0GsjM_2$Yn%hK?CH
zRRSn`oky)0?RRE*{XFoV-;qaE5{_0k)|a`2IK85n`w0yw{Sm6A+u!fVA(HCn32C2%
zY$^RgQ#!Lnztd0oW3^%TAmF!URY9!DgJ`N^rDfo|dYOJsCi|(HLNepbcHEN`4;l7u
zB?G`4xU;cZ1$&Ry2f%gQwW_Q&|L(V+W|QOUW#40qg8)2oz;Ev*Rn2~k*s-EQ{PsRK
zm%O9fT0VFG+980-4b0w5+}K!onGm2(FaSJ+n-)81n83&&CPw;V+yePjh~GRDn?`su
zZjG*i`|U$sF;JX-$}rHoWm3=r#f2IDelr-gqpjcm6<Ty!+P=Tv{!0Ut)8~rc{u{z%
z(KY=K+)n^U{p@$bgnQU1ZD{H?@@<m#X@2KU4>$tgK!CYZ{7y*^xCY=d49Xi@3aled
z<V>pkTBZGRuE&0_DntK`Bc%Bd_r3(4!Sgll%gWq)pP<P(G_0Az{pHz>^R0fI=$~;v
zN?;jZ=S9%l{4Igm1gJ&ZP)vd&?E!2N!)}`HKC+q2VPz>Clko5)%Xh991aJa^{mI2a
zVlg}x($sQZyXu&0`G@CN&g(%W86Y?w3(;&lEZ>F&z_JAv(%CgLJ8<SMLyt=jcd{%t
z-do6)qZD)4avGzt-s7x}yN_x1vyaR$Ch2>}K_n(=hXIoQ2m<-JL;9};zaHYVYss!D
z#Phu=-O|_n@FV+8{`aYx=I^NV9i`5jKC|ri10LGNg7#>p3r_OD`(=0dsGbY(K(qkj
z5&kW?zd<~5KJ`0oq!ot=f(tGJ@8(O#M5)jut>jTmP2uJHm|g#amq%|HUcSGZBrY$H
zg&1DG|F*yidD#(L^%gIM%wv4AqcOKyZZXjNh{wSfW!Awj1DM}F$+Y8x2i<8$Tl7KO
zXm{E{3Z@;JuE?~5Fw+hfS~%@6VRzcm7CWqMEa<%w54D#P`PCBNmJ;9eN=zbcqFW*<
zC{fcDDUmQGy3oQBP1q~3K3GyriP~unZIMnRzjPY=6Lj>4{oPK}miB5ZFG@+LY0Eg&
zwv*dwNKU8GOhr0P+rA{}?R6TuYun*or(xF9b|Sn!#JtdS8su18xUknmyPcS=d$fq(
zZ+}S#{_snWmN{UU<<YW&z_PZWk`8=bX0H(yZ*^YzkbK*GkB+SbHhXkPMfOVP{mP?b
z>Obi~91r%i?0-FO*&hXq=uFWRllhg8$4fL<5Wc3<e<bH!ZjYuL1!Y-H^eWN8<!&-0
z?*B~I%p}V-v(1l&lg!-qKDfgzf6+g3%9I!a4xu)bKGx4}!_1>p8jtOy1fKpM@o{dE
zatGFT$4rB?p^Nn9KQl2*NTC^K3Vc=0Jn?y%E3}_(Sr=%mt8lKS5UUC&Oj_2Z7quhw
zT-A^L-NsMDu;$8%LmYh*?R0gSP~aUgzSUh>xRBd&%v&Mn$QjFN`X`B>D$;ZHcA#GU
zJ<5sh&z=7t9`8NCtJmqyR*foh{&dr$ep?ReUHJ@i=Hu!NcQSg<Y^Dasq%oNcjO-YS
z^<l={Ii#=-R5zAHn)$B%hhoCa?Bus6$+?D)<@qQ<GO3Q502DH`z;Clx=oYe)c(De$
z6O(W>+*kqKYY1Lrao9byL{o@6+D;FbLuZc;@!P4WjJY#NtxvRkBA=%dY^jP3fqxZu
zu$sXtujR-%->w+I-(qFk*l*vbRRGuom+B7m`ZTGe9EKZod(ob{n){WgD}Gg7wM1QA
zqmSykOgp5e-85HSkpgw4>58Z;!l)}3TBxo}*i~2U7J_8~=%vZ$?l)q4EyqaU&kE!S
zj`W3V&;7HpjhxIypNQ;TyYzp@&tvgMR(7t*$uGp3V_PcL)3r#MV@d8$irjjn_oE58
zn<QUogsgd03<aMsO5Y~7b`!ZUHV<X;^)x}w;eL?II*A~F*F&+$%lYl9C_J5>e==Uy
zK><|Ob<j4JwP&>-DUs*`YYzYH8Ddj@)})%IvzWNCUTU>lg?p4{-!y>a(!yx~F5*rZ
zORl&yFtGM_;!0FyUn|pZheWe5e1XyOZq(;GzkcCKZd0^pKx$tOVAWE7UHw;JD!{r7
z2%(28`_mj^pFHok<$)hLq|&|<t{u-Ktojdh{wfrQp7wM7le9BL_m(AoAdaEM$BLNM
zejIL$1cLCu>-JO8%5yEtoMSJlRHeVmFg-U+DlRP6Luk4>$idh-8;I^qRDw%Wj7QTV
zu+H~rdRb}O>$fXF(`RybU1?fHn#VnwFwEPtNoZ(#mVn#_n}cbhXY2Ktn0$uR)oVi2
zoELAWNl2y1=X7_$MVeff##(;q{}$dzYtA}HYu1(4yuLF@TJzf)rZL-}qU1vf+L+&_
zNMgd@;y$lHJB{>zuc`oa%xyZ{6{ku_ULZ|)gD{43hxj<L-hXtPyaU-gG?IoJ15xjT
zBdM$Zdl9e^L||xiNS9N4H~JJ<v$@<%x@rEOu>!8Ikim8~@q=+{5@3#XK+SZRA~6|w
zp1|coM(8v$+`v-quoown<!?NW=REG5J4%BaRTjFzU&S-AJG@8qJHl?s_yz8|1f;|t
zaRsn<b{GIPv1Ro2-ll2-PlWliX;Y)(r%J3?#$w5uPx_wmUqQpkjQMj%X3YL8)h+r;
zab@8L%0hhr)&G&Q4)~kj`gNa*#WdI_$EeKi0qpv~V;8%ME&Lf`wbFt!$m#5t7W{e|
z%|+8>P=-wLI{c0M!!%*WeM*MaF~CvYQow3f^JlbnVYPr&2gZDE1TDk%lGU*dPF}&7
z`lx@zy`(!3r%HO15QIAJI0o0B#3@(&qnJb~Yy+^hz#8}T%b690?FaU3aoKb8k0Q!C
zz)yS$|EPF3{sQr5fJL15j}nz3xC?MMfH8c?s{#o}d8?21?G$0eY>dDVvrZ)GR4#7V
zh>4betia9?PQ+CqBCDr=oWKy{PEB!XaXRw2$Nj8Mh+L<(x)tI#+hc@#;(AoqA=Ldu
z;Pg~?<$u;38DjXAA?67-dB3>kjxe2ukZX`T!jOOwMw1m8VF)wAxX{88#)RDwrW2Z|
z6FgOQKV!F1<zM{$->Mu<KjXL=4ZhR|K#CYBb`u#0W?;X_VDV-1f8=&TQf0Kz!_qUZ
znL+H!o-!o3w%3fCrra;5{xj!PrE8|&#k~H%<HhE|yr%q_s2v#!zdZWSR8vfRri<n3
z|A>dxn*Xo=5s#=#{8#1w8E?$anfna>uNwc4c-W`;`~5TCt)<JquV(#^c!qM#zy6=`
zAr+<mvn)6M=zqoQneAC+UHZQG?|8TvG}(OvU>4@_&UMsN_kXn2W~G{}Usw1)v$_p1
zE8P(AHRpnEvjiA5l)wQlmj3@oJQ{7*TCqhWE@YKwhNS9t(JM**f!y+~xB)pWD=Oz-
zkE4z0rUPqBH?>D5X(t=lka#+BC)9MOsY!D4&p)uG3(EoaXn$2E13L=L9!__CJkCGZ
zRV|%%lVTf#!8hx*C<X)Ppnov3_pf_V`QIEO`a_(oqkMy=@AM_^bM%9A5S9PUX+r<T
zp+q?O@y%t$0cz*0e+Ww7{^mCZ>U(gNCGahty0|0>e1eDG+?n32--E7*@*+FGR*=85
zj2O?NRy)&GJB!gEpMvf@9f#b=HtvY~{Cz|zA0W?S{_DcqH|q_D`Y<Bg;!vOaBpq27
z;pQ6-^=a<H)&pB@c=Djjlg=*ylf%8rlf^PXI&1w8!jo^(gmQS&88&qOTmtZ<Gh<ih
z0^!M8mnXbiZ*~4dd2%CI2^5~}^7oThjBiwCxPlE`gbg<sBD%naF3JYJ;#~0w(S7_1
z8`_axzdG7wJW0l&zT~^lT^_+LCjmpn^B?;4;+)(j{^oK?m56SYq;3o+@i+Y4bT}!l
zZGOM0UmZ@2jq`VuS;~!IQXx?tRigB8slTJjlO8VhcQkSYj*~O}x^g4s+^A`CwNWmc
zTx|@G!Tsbcf12X%QBntsG~V;R(SsZewrNn0?RXayB$2di2Y)3^y2T`|lq^Yg=&sDP
zX-t1|Vcu4jTxnsW2Mx6R_edS&Gh1zY`{Rk`=qrdCMfl4qqSp+fvgFn>NmW*I53{=Q
z#m<N0chE3$LU@-*(Yw;AzKhqJ;qf~dBjsU>E>GefPmog?@jGge3c=}e8h0#CE_=uC
za)qSJH^9C$<CkrwfYZ~lO?Mq84@&3TSqN428u5AxL2&$qyNSlyb(~uyFOYzV!AQ@*
z3=-pJSJ4eeS&ihdknhQRiP!?pp!hvxT2~hk@2~ovOvQxL*xd*w8#x8>2U$G%_FqWM
zlQdgb=<Hft0%()249;Bx6tTOCEV@y!OxN4I>?xddRaLt-7|On5<+6}6HM@2<w0!69
zD}-XhVs+JSO7f;+e8Jo2LgI6w!pf(N&isEuc>8^-;cYG&KzQ3V5f^88o0~)`_|g@%
z)U~bQt-kgyuwgiN`e~axgwy+a%-BsAhZ(DAx~GaTgEtC<RlBwO@QX%>Uk^jR`@>@h
zP2LqV)-Yr-frY}5-Tx;;`YJ<qv&Jb+^}U_|&@~L*@s@nFmEnkj4M%Dfas;hadUpIW
zdU7{uyPC?4S%w?Ss=C}*1r0gi>UP-R8ey>1EyP00z4~pu+yH=k_V=Hc95G~~6+fE}
zg<l?piJ#p_<XOMz5kI%Iq+@0Hd---kQrGxfPbhzX<u-`f+416~J4STq!Uw?Li}%j<
z6;v=~jjg3+$)hy!<8(8|lsm4Tl&hz9F2)aW2`rdShg{Mpeu$qc^HmVZBwxl-q6&H$
z5;$1w?g$l(x((tI<;Gjoz09R}IaDBZFZcx19Z$#eX%0~1-&PPW#|pe+dKVUR+#v09
ztC+WLgSf@G@p6f-|F7{;G}N&wsAx6qe>#JlzI-yibZN`}=?qg(dh(2e@iBCw@JQN>
zy@jsN0NWNDUq-Y+8sVF`f;f2~J6`T-2&@~5r&}G;P5CiiUbk1+aA19aby^o+wxSDL
z2P_ZRgzWgTqH!eeY21<9wM%9Tc%BGJnfi`RJhNJp7ERIZJBfUN$Odt;vl(9r<kIE3
zhvULPe)hfi>JcV`EID<L1(sefKFLd$0<0de<rm@`OFFt?_hz^zz*5`9H}zm0fVGRG
zgjQfiuUg%E;(AE&i!A%c2T5N0OgyhxnaB(xhaux9Fmud1c{;6g<`7Spl?RZI-;fcH
zU+J<{?gYt`^wPcPmF^R9<0XJ{?vAO~eFj6|UIwV{OK^+CzB)eD($|lF)U}`)>>>i=
zw$*yph<>~vDk~(;Q*)V`L6?DvERe`~I*ptu(@2NRAUx68bo!s}DAafIwx`6f8Y=4G
zClGr25u&7a5K**tv#ARij{R!sGmby?B17<8D>bf`p2nJMGO@~Fo>ern3ZR;4i%fti
zg*Q;#H`a6;h}VdMOoA6JeH-kj{Ui`Q-f@O^15EB{%3{QE6jz~|@0iVN+@>4-^e+9x
zs`0moeaFhB=E3)R-GNLM{q%EIg6w;inn%OVqz80=0=E)awd7LsVn`(-A#NYAZK<W^
z(I~P^)crE<{HVc`rdp*Il_c*8x(v0iM<ssF!m+FBo`f=MBN?6&t+(G>XDK?bLX;i=
z%a^4dfV{f&N;^@=`{ibm7V$tS@lzyX-~9%pwEe8q16Vvl^8Mv;(T3y)+yUnPbYLRI
z$s0?#7cZ>)CjiTclP}4Y(ziqM39rS8C0OpcQhM}=s@}f~w{=vORjMyT$-iwXHcFO*
zOR1@Nm79$Y6|K8KssUB$Qc+#03Yln?I36>_sw8q{W@>d`H0}D!I8qgV%$jUfs$2re
zlzliJ-%6W|)gZ_&P!LnzUzQXibN9sRUUL!1{YO)b=H;<7xvp6EhUI&)gz!DY#qfGz
zIu}E|RF}}ty%=4sM}X@wi}`ni2i09T85U-qKyYYx;#6=tNy|KWwp0u)Q~m-i^P;rO
zf4=SyhhoqWCvxqP{k4^1IIQu@O?yP#2&<GTMB(No+}XH+Rw>zG7jfzl4CJr4K|n%X
zNG+iE#0>@_>P^z1LVEOz8v+E16p{;MdE8JSu2$-SO78JY+%O=bXeHNoxG&<gvC6xp
zJ>W;OO2&;Cf}mFLVBLVVL39?ZD-|1!o!eS^)bs_|R?n|2uquSB^*^;#wQ%am2{YgG
zPm_WE{Q!4pea~;gvkbR{Vx6fdr{@n6+K$^SfmiW-h5J;_T^fY!qLFk#sb(A;`Og{`
zVNYt^vxbXMo)GHUKoBl)5%z2gq?L=XC*ss|l*}y*!ZWfJ=46*@bEk)FFA4T|$PN*B
zNswiG$j$(H*+cd*fsbI>A)Bq*W|1XA@Tb{w%%gbq$dI4wqY*AehMbpU9=}$lwc4+X
z47nh@lhffVYej}!L_%h+kxi70e@BL7lTnwoe|?qs4LOk^D+QxG`=99$fWsQ7Jt8th
zhH-0{oCDdrAu?oDn1n({Sdk&C)eAf(Br>G%V+x}liVS&HhdfN2U!NyB?YGE~Cs9_4
z@g7RwzzFUCTu*BS_l*qMFVc~QqZZpbMus$zTKpDEPKh<A<p?ZYO#r2$F;b=GS|wt}
z?}-d)p_Ns-S|C5>Sk`@!AuSayij^Yl-)d?92$rIXSBfmIC+mM3N*Zt!{}a0O$`eHu
z|1-{#x524s@xKiYcvrP<Wbscu-u<@TlwJI~DLZo*X+N*Xq0x#&%==elak;-lv1FXz
zCbIas!i=2Nkwz(gj>I$95QqsUP+b!11%=4s=L>V*IX<%ZUN7hG6+$QRnp|XYroC3y
z!caLL6j}Tzfr;%Yb3=p3;>RTC0T+$lPf!BhtR7kXu|hPmG4{&mK$zBA!(0L$qs9<0
zi8a>9;yI}5tXggX8;cgOsb~RuN=pma`DJABL4^e{$I%)Ns#v6kGF2$7p*|0zIdf7X
zi*o{!DBAsxJIPsfK9F((aY;+7A_14HO@#oDQARvMmG|u`3h#y8>$*f1zq=6SKr(C_
z3q)!wDA!LVP{$1j?_)H^#cvElmxHPTmCSZuM;0$UDfx%AT8qO#U`-*JccGvPDbo)G
zXwI22k--Ip)O<(IUQs|q6onel1Q*2LLK9q&qG+T}!9P}%IvHYue<HP!r(K64gWvN|
z-mD8e`xFrAw=U0u-=hT~MzMTQ6w8MOOB-bL<H+FG6-$9t<z<qm%Q$g!7u-gCgI}j^
zZyrVSFogHKQI(&Yk-=}k-bv>e*J9=LZ)C75sa7wPSFZ@hs4CYjr(|QsD7zlRCfjz5
zR~b4E4!7y$nQS$@5g9y%OMI)YGkq7M!Lc_{<d$WCF$%s~2_|H*S6Oz5o_Rpip*zd0
zj|_f@>B67#Nrz`Z(<>d<N&-XhbjNkIrdsyc#Rc|imQ(c?ODr;YK}n901Z^*n$7%rc
zD>H=Pys98;!-{p#OXTg3dyr?uouEY|X)km2GW#Nf^DC2RtRz}Rf|0lkN#q+tq9K~-
zSY&W6*6?4|N#t8jf-Kxb3CzZmjav$9%J8`f$0s=U<90}3E1qv~pG)9bzUomN=aaxO
zJY{h)5;%tk?(~kAz<YR-akZxr_(HyZhIM-O7m5!Tt3Zr)S<Jqpz+M;U8Euo~VF+h3
zzif?Cb(b0IWD0KctISwa4;(#QX7nCQ5u-IB%viTW6Yekx*Gj_KE<<`xBVm>>#QBbd
zFDgUci42y{>HZxdJYXc~y%0A~0$}L95w}Vh@{ZI8d$sp#xR)yuh?h#$5pV-QKjFSC
zN5EWSaD4Z3JgpP11GQVva_yMyB4lEgIg?ll87?wrUjD^WhN5dEV|Jmx>`+7oZ-{cM
zzMj=;z=)s7!4{UY{_f&dy-#mS=;_J9`k%E%-$Vv)l|(E{s2ubeN+RAxqL3*CHWwR3
zSbhgn6p6kjk&{P)2{@4)XEB~7xQC_Mzf(VF6QM0Q79t%7OQjEu+~<r04&nI>_X)gk
z&QQ}0%rnmZBF$P8a@`7SjFSrRs}*=E0b?c7tHAf<{?@C&#=1gV*}Ecxcg2z@#jC*Y
zB>Gh=&;mR>%t4}Zk-<;hNumit^I&e^6vO%cA7f`8UPY1p{qDX=2uVQhl`ViIAz=?8
zVP8TZY=VYOL_tBe1OfyIBm`vN3HuH~QNfi_5L5(I6ciXy5m6C!bQDLN!96ZBq5{63
zZ*_NwGrvFHJWq9MId$sPsdG-9s;;i9#BD^d1w(7p0=Sel&j&vK2TkAWB!?pQHKr=>
zYYl^jW&+*W;J>caelXq;(JmCRw;DjN2_;j(&@2MJn&Tu8xt%b`phZz1#LtDI?xBch
zTT<gYp?wYj^lzxqKl?#|jvAob0E)(haC3@dUjzWhYiL0$5+5n&G^xde`AWP@<X*xy
zB~Zkne<3`=u+-tZZ^!;OBBVjU^&5JW@Dlug@MD+%t}8F);@!P#mma*tQ7fA#E8<YZ
zi@seN`UX|$q_Imw-zV{&rAevPv!MtxL$W#RIM-FwyG3%ac)=4SCOr^}7)`=DK%z;T
zP(-$$SXwm{k>e+3{}_rGgFSe86Vp{)>g3|vOnoj|vv$Nps^V7DSkLyMh#~NhR|z!M
zp-Lf+OvI3?ZhW%hiNrf=^)NM7k0L7VjtE6$l~tNjC3mE1WucqS6fi%^2xc`lt4Y<U
zN{wcpgd&p4YP=25rsL!wkd8@QB+-GyIhVc$hCT>iz9JOSP>C&GFB)AB)JH>kVE~u9
zimWLs^;oBto?Al`ou3Lt++dob-TR|L;a~gh2K1I>T@g97>n4_xNGFkN+718Kw2Nz#
zYlXtUtHUI*$ZP8QXniFZ3jf6?rqe$|5s@yqJHb^{!Z0eo0AljrrukMchQjxiHUB*j
z&GwShniiGl`fVuuDfA)mv)jC`7BT0A>P`%W@2^RLj<HWq4~0LA?>l)vi59<x!Zq+N
z*aM;PgN7lLF#6$#Oo>Z!_-EqI<h`OrKVd%k1qj-EITWsMCe3h%>E$h=5xy}_OeGQZ
zHHjA9cMR*glIS8>N*XK(h3i*f%ts9~pCcio(wUDjV9LFi=h^%qt`)H_6#jr{@Z7HO
zT_)jvP8$k;2rCbZBzId@g~C@?6FyWd>@mQY*27n0yO`D~pF}^4%gSH%%8Q$u@~*hB
z*N4J6edUCG<@!9~W{uz3>-sDUn=$5#P<RhO!sDo2@fe8&oXlhrXia#Jrs^3q_sa@}
z_kq2vG^n)a$xwJ3RgyDUa}x=)-f%z|+B7JKp^Xf!k3pMkE{2_5LgBT9K|ZfAXeC{S
zp%!c_qoGzxISln&4D_w*C!z3)!XW=&%~q_VAu*Svk$?u%Z1|E;cx8rU*1hHioO|bn
z!hUFvQ(PDd`;euP6A^M*6Q+^1Jl2GLMVB{T4uyRh$)NtOE|<I+3OgwHW*+=u18+&0
zc0L>mW2c_#jLC+{p|Ck7A?VQ_^n&u(AJW*R{1^%wSO&M!gLA(CaR-R^w7_*-9|~(-
z26xbdYisZse>5yT-gxTt>QGE5j2E}Gd!7!^^{@m6X4WV^|7K|uRv*BsXSG}#d1ok0
zmq<Dlu4~L|l&nx#Rm7-TFS8FITUh6XP*^p!aX^Rf4#$VWDzN?058u_ewN8cYSWN*{
zsA_<$q5s4*pC=PZ39Hb=jI%W?jLfSE`ocF9W>o(G*W0mQHLMPMHvKvEvb`5H+%0db
z!0Zq9Z$x^#(uLZjl-H|Y(=q2Efe<mV$)Uh^B1SI1t}S#b)}dWAJMdk+OUy7dU|2&k
zdtD*Z$PXDzio*twiIxVw=RcBY4Vyv8C-HO?nJvBQcof6#q4#LIes9o~MKnE7+DZ`9
zq|bqW8+~SyL%LAuz^`UVsEG?U?6~XD2QK@WxFN$n#gVG`xS^a=#fE<H{1^o<`mxP=
zmA9^uW&_l0rx|AqkKb(I_FHT@*H5DSfOz8`3}c%S=HR6JM=ttt!ZsSGAo9#LAehEJ
zS-)N4lG-`z;raC%h>u5`y8SL%PV9wiW!j_kPmxcf@9@83Y$)&s{O`MqbiU#L<6!Pf
z%@EUI#i7&|?ff7?0hJZ~(~<kWm#x4X4*+Pc$18}=w4A7>?g_(wVP4S&QBYv}MIsDg
zdE)_-9ph&k<*QN@3Y_qjZ{xp00r$R5WIHMb=A*@ddyzPE3sdbqp}>8Oc{wq|U5f|y
z;)wZ+{lGqrmABXrJmn4?3YgU^6qw~JV3eGKw-=FvwK4XJy^g>v<ehySGb0A(A~4h6
z^cq-QzJWF68(3S`z^$K!0(X@)aN6AYNRI5fT$#$q$X)b+Ut2Ld6oC<O^dJHyJ=QD~
z$RROp9|^WA0`A!<*F*%2`(l>NcWnp-y88`8R|7bV9`uZ-=5!Bd|MGMWC{L%}sWKh)
z=9(LnshvSnYTewQjZl4|=)^=fG3wB+p5ojw2E?$R&?xCdJ28g2?5>*#8Ti|knoxHg
z{<fZcJQxb3v)dSBGrv{p1<XQ%PS#dRC$gQe1y{1tAS7P@M>*@8=s&jvp0w~SE6oPt
zIl@5%@a9sj(a-=6Y@;H)z8n|MNz_DdsfG-31-ZM<kwrm1rA92Kn;@V<SY~AM0OdC(
zVyehDm|aZQXFd}ORF6?#N-Y@)yOFwHsjA!c>dn2bpM;cGyslSo>vjDHGXI5*sPtMW
zP!p*sdXv|0$jI<Ijg8Y9s2?d*P?JzNRfeU3x`^P7xRsDK%^mb+W`F|gR|cr1u}-N7
z{sV#aB>~UWG*<?^9x;n>Hw<aXt8FnP=AvN;7dT#Hh*Ov&b#Hm<|KNxkF)S$wl(7W2
znD<_gHG)WUd!aZ}q}_`>X`z7Q%T}coOuzKN;b2irH1tRYoMh33a;PLs-B{oWU5pa)
z4b-1XrazwSR(AWsOqH-b#IgWwuJjCmZ<Er2&)D@ODh&uZ@-!RVu6eRjX2^NHjF3xS
z`6)b!Y2hwb_csN#@g+3xP-j|(wj>>S5O9u+@L<R>7Yws|#=eWIiVpXLakB(KsG{?{
zTq1WT$a$#&Gw((Z`w<sQZ(bedg`7t{MQL~$@RO33>!O;B9?3N9Jj%qO?fMZb2#XPy
zRqvCT=w&`e=0i=)pgWItRW&kRCcFS-<rpAu_8|X3=EsfM#XlKxc8QgrAVZ{Isuyxx
zTjs2fcZI|8%$YUf4=P<@<QuW6Q?};!RY7XO%0J$${BhhE<bo{N62CCym<?*N<+^k_
z$}YE~%YuD->0Y00c$oZc$T4?w2)K$CDFPm<*;-@$>~|sOE*LuF%{Gv}JR;<bj{)gU
zyx#T)LyozdcZ)@;^N^5}C!HJPkV<+!<V+-W!=`{0U6@}0A1751elvoTX~k@5va9ao
zN2(G{br}<KruYfA;+-i*j9uw-)`?}_Mz6D82t3>GY&6)kNxx=P$eAwB)Z9%y_V<u8
zQ~u5vLI|RBA)t*q#%qk;>>e};Iqki1m%+BqXxxJx$5GUZA~Ow{kGRK0l4J=HJYkPZ
zD9Fj9H<>I9gB@|rpmAiR`8FZsbj6@@<iHx4K}f%zHyZu1B;rgD!x?fh5+i$&nNT<6
z^kl#c_!2S~ktzD!A0#GYBVjF>;?d21b;NDtLkZG?eqv5@k}nYs4Z{!E#0#&%+7@ue
zQlmCzIrb6CJ|LVGEgV~LVxJ=Q9pNIiB`|lVA}}pq3^~;eCy;^QL*nFtkW-Vf*jNoq
zz#7qs#YO}l@^q(;;W%K&>W7?qN=Q^#*E;SZx0yLI%Y8n^K`l``Gp5td)5Amd6~CQO
zA{8gxLk^uZbMx+pNi_BnQ`eD@Q9e+`XqSC8S=vP%w93BPS|uKB7Ropy6$+$EuHWhr
zDsXo;%ziIqf9wr*V{!Aa2K)L+jZl{c6u}_L$ta3Og^~ICJ7h{K(EL%jk)u+`yaD?E
zUfrOyMhzmfADJW1k=cOEEo2lC@@e$tp=5Ri<-K#E_ER_?`YUQt?^wt_j|)&xLn@=1
zpYh+?5@k)(H}4_)AC-Z8K+_w1sB(SCE{)`vU~Mb~Oe)zEvVW)}F3`$-g!KTm`YdFd
zL-kB9XD!!$*yfH#CYMof5ME<)X`F3d%sSVbOOLNPtokw(^U`?!pC%SFy=tyy#$~QO
zYv$U(dTC#8CX(!sebQrP>r)6;7qPP1duDdEPd0LA*HPzbS|1tXmD?xV$Sdbp=1}6%
z*OPg?KbhU>`T;U}5IT`re?Z87P5KGtqYDUQ>1=hJk2dY(g7AbVA^S~Skg6RK@9M&c
zMvQ($3D`RNX~I4w7|ziz5?-KlZMtjMPLt|f<44Tn$tnfLqBYe!0I*3~6SgX3zr`0A
zPW7o&q+g@|szeSE6lOGs)|~2t`I(Dw{nUjLSwxVK?3y$dwu&LHK95u;p`#M3h}=cE
zLy2uf=u!4$B_1WRfv{AGXNbH`;B}`{{V0)(gnz`7cyl4zSRM5ZyH{l&@?j1Ov>Z!`
ze-eRHIn|Z;i3kkNNo7;E=1-+o+Y%UMh3yO3J1R0T_d#qd?9D-9bF|kR%P8cGBjhMi
znaCW%-AcfRob`m&>eD`u$D(LC4-t0i_w(8w2j7zaX5|927Hnrp#8kGxuo-91a5$E$
zwA&6f6e0bqtErhcL5L;F*`sETsTuX|HNr6gn%$I&{|wnjp+~RAkOJeX*hkYmSEX+v
zrN{|VRi7QC9wFZI`>-8c*h5V`SV`b>jFj1<kmNmw;2?QajQb&EPxC9*u-uB%%2zBZ
zU$MwktmukrQHC01vCe#8&tk-a9Z?VryFJ^?1LiVY5*TXl-nNiE$CLmj^EMAe#&X`x
zFL%wWqC|8NxSj-84fBeasl(<2BX@p$qCV`Cx#(=5HYWmwFv^I3qsP;|*~NS-WY;nM
z;g`{Hrfg&8YW`z~ZLbd532?eb-!8h@Eo9egjHCRn)g^yqHmO8r^_u^vBeTPT+fHJ#
zu2El3uG_t4rhAOfzX+dUi`2~&Z@L?USdOvWInYiq-6aE{G1Yjer>Ul=WTX+AlBpi2
znjVvZT#o5Y=2LY-{%x4nm?dOtPMq~*$kzFn;0ZFe5;mLlynUBh&j;3T4q!$&{^r!2
z3$z&4!Y~ZTCkT&`DaLg&j3eVH;U%xJ?tbQ3GEZv`MON=Hf*aYII=-xq`8VZ1AghtB
zJy{(~*O^a|UiduJ-3+K{&i`Mtt!xIYRyjZPV>SzF9zi)oFrNLj-2x2M9jso=D&8;b
zZWcN+zD~gn=42H7MYTuJU~_gD+T*k~4%yM1Cw5{pVGgdC9UV{U781=)lW6IS`>iB8
zQKAh`HnHvKo5f4HVRj`{9~YuNujs3Vc>k<D7GcZ-6tBa!Fsbf&2itK$o8lQ{g?^TR
zZ4u-qj3|&l;~-jNV!-*IGPox?hAs+jjNlFf2aOjoJRFAv?yEAmcPfP3d0B85T)2<b
zuz~x;h3kV<-W|&K1iOW_@T?}CTBjT8$4f%_e3et{G($piYyuBfGWp1wW|FZz30=rT
zuBX7w$@@u<Bj|<0up^-<m_`G)g1p7Fj347(-9h>>!XxAj{Cj8?T+wtjR`qqq_PE^8
zLc~BG+*q}JDv5W(i&+iUdY9Bkg!dW!7VF$)U9E!*yLY%u$E_Im7DrX2rLDWCQ$PbK
z^2`Q$aAl>{u}0_?(`R^7>pvAZ*K|B|D+~8p{}Ei?Lx41XCDhFasT~W*n-@Yo0I3}-
zkO|+wxr41kEu|7HhHu!!|2%Q7_8fZu_@kkg39f35Ma0K`OWyW-L#@bDPjgYR+z^D`
z0HNhRcr)v9Z8IT;?BpHE87EEX4AOIzM{)ElXPgK)S2=&&XyVLF=6csfJ9EY{9Oh1^
z?N-is?1YKWt&=kz8*Ad@hUAQIY;ZyRm~ck0M}?e`5tdV@ri*!GLt#crz&4WNxA>si
z0^3ML{H>QNkIJX@kvExF_mCDBp3}!8t<Dq|K_C2BgP_^zoIZdU(q5XC(+6kG+{Cv!
zBFF5ZY!SOR=eQpv5YM<LXRs>))Edz|hpXp61Z8VtOrLS;l|~a51ams_F0szHm347v
zPDkV`cmguV<>qvRWD|${jvc*DLw-697zS+Zo1=Au&x~|Nn9mBvSveVM+x$q$vyWwm
zMS}l^N2^SpR%nqx7T@CYY`9GV#UJ$!z>9X0NLM9vJv+?xgrmnRYJ?$rf}*h5EW{T5
zGdnE0s;V+KRc%26RW*s&-0ZNJD78Szqjg9Gm53famHK8EpmX%-v5FgG8a;ZH;=s`G
z-HKN8318MFJMcY(*B$N>9{9c-;bu42Cp_?d8R3B+Ovgod;D<881NucEpYXts<p>Y_
zSdQ?(Pvr;?{B(`*fVt4zB|Px+HNpcwmk}QLxs32_FJuSKL3mtEkMMJKu5mnYF7+D6
z1LrW!F2@7zHiaSlgKLC;P@eD)%M<=#dBXozp76hwBiw!OVK^T6s2t%RT_gOXGQw-e
zX9ot0@U$vZ;NRd{L^HkfV8m)dYjeHb0{{(N^lEmXwV;a(bZbNzu_{2S0<D{rb<2H~
zp<l~cbSgX0RPg(&OyQm^63irK2u}|ok!HjM*$XtgHiXR#Cscd-p=?J?jeop~Dm&+D
znC6T~JLlq+pjwsglq9<)W5x&!?c;hf-J?Vz(~!k49b`Knz%e*qn5MX94dN{nH+CkP
zI}V6fck^2*u5BG)S}ShsPjq1$#Wk0a-&XMkZd_5913Ijk?R*#|mZ1-?$CF?X&0)E0
z=Wk^lb^c~L3do{6vz?F3IIGt?_hma1>2*T8D!J3ZG%<yxh2A19SxtiFuIcsuo3ous
zjXedtzE8HJB_%|mvJO{oDhgkne79?;hLdl`JUUxZ4QuCU@h2aVBFYFl`ED1B6qR{5
zrsTHf*-jr(bJ86H8AsyTqcJN^pJs;b8k;`YaF{MpaQgHF=1u51_;9wPA4sI{rYqX&
z7=4=|%<M?>x)ksMuILNl0GD#M9J8HUj9HEzrEa)!O;@At<}rHAbdF%v&vt6Lvf|Wh
z>rz(>9~A0LW3><i!?zcH1Q0PmQbirpo+uUC+VP6&Wp1`}L%HVMxb9mYKP21!!*67S
zX~g~mPa@$YAEjqIwqd#eV_Ih0pZkEjUBJ&>YJ}eY91~Bg>W%%mEI{*Utf2jcJ8Iyv
zzi?~Au7&5aZ4}q3_qL1YqXg3{#LE7tm4;lG%TFlEw%2$wSK?;SteQ#&qm#JVwe$8G
zn(@u0y{4<$2W&f7)*4YDF}Ds0Gu9-=(bu)fh9jcKY@lgxb&*ZN5J#dQj)a+hq9-Ch
z_Ig)GCSu3!^+tdIn%RMbQ4$iZ7n1PMN+s<Lv^8L*1~YplR8?P{ZSxY{NqmcZb)uM~
z0j5XHPj)dKHUclYi@PYH-e>$h+ivRhKB=!)u_*&X?{zmKr5)9K$(nu5xY(|7(b(C|
zTv;m|iBQ<hJQI*@5;eqxV0118uK&+!il`PNxF<BfZFaU@pQUu-X3I&MNGC;iLu@A*
z2*b@iv+bJha5_m_J?d(*E=}BQ2&zSvt6r(wv+c?Zb<&F-ROQq%s46bhjT|nYll`iP
zu6`kI0<FEuR#)O?;$v53ziM=kyQ8yTGgV^ph#7d?g6!8!KVvWfubEE9<YDMvGZGM!
zuc*<7m_kMIeeop<VZ!S!rb2k|x|u^_3Q^G4%@m@Kk1$4Gf@Ac@F<6{0%~+F*Rr-n<
zd-OM4<i-YHO!g6<Y<_-TWFIkO8#5Xc{Ic7^Xe{u{rUg~GdsO!GewE{HmFG<r3m3Xc
z&CFkI24F@`<|w^DXmFY)&o0Z%1<)Epy#I;JaRw{#-J>(d6AzvsK5AiRfgfKrHM7tR
zl<~<j`PCGBGGMv~Gm98egL6ua%;{YD+u+^1++E7f1Lxbk7wD1sUCgMdR_1*87~YCH
zyPnQmg?n%A%-(xEbG3<cL1CelDIqt`;-a0I8_Zk-{C4sU<B4Z)$aHt5i9b0h^I0E$
zc&p6A;4wemF}F^p<ig-dx*_wpiA!;Y->ApG;l+SYGaDIR{9s6HYDm)?t-IgL<hmn^
zUyQ6Qnrq>YaKt2^?YZ}Z98=&*(p)g>H|Cs%uVcq`mvi{%W;b>>2TLE|!y+J9w#T0i
zh7Dp(+wghLY(4M?{yugkm!on4{Y*h_4??#(BwABpu}*yP#2yChKU9gGrNEAEzziY`
z<iOZNtIB=UGvvHhIrc;I%jccdYb|}guv=G>*bhByIgPflF4hS@=znE(Fs!%ekEa=(
z$ffkp^dDkJVu}?E%X0Z;<&LE7;eMy$O{b0o!?;MuX|&ewR30$n{Z7pwQRH_j-E<1A
z_dC^yBY<wFih<+%OZ%bj<vZYYiqBNazFrzq)~Q3RuBb4Awzui!`}GrpVK>m7s)yN%
z*{$w0y{-}BafobJEv(#oXl6sgxE%V_fOAhx8T{NMgcqhUw85~>X3*J*sbe|G9D1#`
zj&IP-)7ljbOJjqtNj6(*$$;hli=Y#=$#Fz}CX_0XjP*wj$Av4w@Q%Yl7*|z^o<!jL
zxD<3FJPxcMYZD;;-eBM>ikt9dw-c@kh~l_TN~prkgx*RB?+^lCJ5h%U;R8ucBILOz
zCxkCxu)^L@csou`*wr{USOJf-@v9p2R-QEUJI+Rljdx~Fm^A)`G6~6$-4dYc%(Ui^
zi2=7eu{q)vdapbA*DDa^(1z8PPC-QcV^AMQwd7lcw^|8<h4eOQ<KrD+vUga4ym@5b
zL^f+$wN>qqKz%T-y>Ks7hCfY@LP?eyiGL;-h8OHKc?K>tJxzb-5@w4F&#?M$`VoW0
zVbSVctibtCME|YjaUuyE@vy621hT4#i+6YCn|vfHuI(N<kxdMdgFV`R^odGV^=kQa
zZnOxVT^>1GVTjLNdl0=>?+b=iMD)0j+Okd=(K~9k=z?KEsm=jtUQSqs>h$RhK|9E_
z;?ckBvB^z`@!c{X+(Ka0DlP?03q1-3e_*oFpgzNaEEe5xPoSGTP@VkH7!B4XY<AV%
zUbardn4ya|Ea1p=?tE=#C0_^z{*XOudJ)nZ<&lzAJrG?m84Yl-s`gI7WljwSI1WtL
z!+?7?oD`GVvY%8I|8Xz?3&j@7wm(k_s~CEO`Md48VBj+{&4RJ!reNS-W=^4G!tr20
zO`4;a)BX|+d}|VVm)QKRV4&1(S(^Oj@?c;;JK%egAf}zznW78Lgl)fQ9Rn*%2ph<}
za|oHn^(F&V9`{VF+m!a;^qOegCBlzJzXQ+t`VEXIc-Mm&4}$Tr8qmv@)9kv0iLwTP
z*J`L?N*WUy@)_kG!&{H9Hx0(<vuI-4YnB2&_2Hb?hw*Kx(Nc$u!$(qWA4iL=D$&8f
z85B3AvGjQs0K*7Flqe=LjWC77#Nm^W?JCjI@XcgxC9L+&MuCStE}r~_!t3a4)E>?(
zNyD45=ZuD%^7F1@H)9XX2*6FVO*TEN04JO2TfaXm_f50im)g>8Z8U-R$w&{0E&`Mk
z-85gDXixIK$+Zsp<bt1Xb*;lq^Nn@jSeh_I_JTgq*2`74lwRQYFJ<Ex$&*=9S*cJ|
zIwH09hhX3liByVC6Fd_~3U}JxNkZfPNOO>mi8kZ1%Z#E<9@hCL7?`d)Um`>?BZGn2
z8c<Fk@aulTzygLQ1>ed^(Z^>gw*&)aU%)#lAO1N@QdkbcXm))>@x&d$KrW(q0|wa2
zl_)lEKTf_9*=ABP@2sUoxix?|z&q<vd~Y2L<a^W0cx=}AQ@lyC(jE;4TI2SG9)aT2
zmQcEM203)B>h~VS9ej$L&+zEs?Uws_ywiLoSyN621L?fBOl@h=ujcpvFVs{2n|0jF
z>Us^=`((ZFOE7SgaYGF*o$C|_%0XTEpRCj>UKZ;}kFM0jKWF)z^dPhY!X>{>YBy#=
zt9h1&Z6yQB$AeDl9+*(LhXiwpN$fcg3^b{$gvXI&pChR&yk?qRYx%xEXZbCA5E{eK
zBQXBh+F+orS>nL>v;^8r+fPn2ugzl*kYG!*Om5=K+PvyDz`Go;w`qPOUe<pl)w^nI
zgASihIxRA(y?kQOaZkUH=+Q6eoHnj4iKUMO9WB(1q3m@m=on+9cl%lQ1)cZ&lGnq6
zj%!a{jyV4?)B)D1OVIfT?_!MbcgYGm?mioGktQsRrSthUx?dv;>2WTUNl)6x{|oi~
zpR?4v0jq<~e~hS;XuLe=TrmlnX>o%K9pD&%$DS5C86YY0vV8V<NxsnLL)B41=RsT_
z?U8pH!T%(wRf`ggdBKAeVgIx^9RCI3GYqQ;c-XLmjNb^Cutc4hlUtNvShUM;(C!%=
zbj-<g(57z($8n);d7FoJqr?xix@2NPz@3~Ot@brJPhe)7P#bgIavGH{SOJzLTZ7J4
z){`ycpeKAQzDCPs7%`eo=P;eSL2n!g?|s4yFk$dDy<6!&TeK4XXwcav4Bc^Zb)Uur
zNSV+;b~OF0=JPV@>$9*rA9R)(rXV>jrx`=%EO!kZ#RC=vow9?LEjPKvbKK&tTsuD?
z=uB3{*XU3)MtL$5kvVCvA)nk(FPkW-F`Pu2;X^)$RW}EnVVJge#F@onRx9nTFq=EW
z;78~YsKqc(NF!5dhT*;HT<`?utqHf2DRjd+7<ASI`afYR_uah1oM^Fm(rODsH}7iD
z8I2m&TSe(c!g@Z5$g2TVt_~US<S*!qflIA8_+jTISxg-7Ff&ZJjjSR~988RG?41!T
z6yz&jyT!R5XWw)HnrXmt+Hi@_JIr0~Q%HGF;<P%&^x9@24c|}Lt_0{N{DttC?nQWq
zQ^9T4fz@saE?1P`Oz^9qa~t%XI1X~%&*-#X?Qupw+h{m*2pms)oVm^C%sDd8c$~S-
z=giM!@+pWR@x*zNsHaFA42iAvIgv=r!?VfLDnKOa=DoZcBJsq}K8Y<n5_L@EA$uIh
z{PAyQ2c3QleCt;jUX5HoCe6?zBs60ET9q64ei^jcS(RS?SJ1hUN^L?_^xJ175F~3$
zm2vrs%a_udO(D23(MxEkZ)_$6(r8>A=O!c!M^X4r3<^4(gm1Kmuak?<OoUFS@(4R6
zl|$Gmr98qe9zvvj&DTMvtuX^)$2y2vf&90P6F7-?P|dc67&(jgJsfm`4A${|5L8uz
z)%G+)zzppK4GuU-LwWVQaS1YlG6Z>J33rkyeF>r>G6Z=$2%E@M?}JFc1U~P5!kf(N
zu!ooI7+lMmh_2>cfrx9r+%)qzgEpHnsK<X@3^TDhA(UX85hu~Fh^$OZ#(K8B(>0Y&
zl53=~9lHJG1TzS-u`}qjL^|5;z;^6am41Yqm1sa@6k!CN3-08*{k`d=CK4tXx9j)^
zmM%bX+Uf!nJIlzBrTb4vvU028?+GB=`5ZZXC&(?%rPihk!03`Dr^Da6b7J%%%W0=;
z9Zy{i+MiZ(nKKcmb>j4daaPd&Tv;%2;zq)1Rd|*Pt<7D+LHi4N6>xJRLo#u<RGR%Y
z{iM9puD4O)b|x?0%Nu6$I!VnpCo9i`k^h|eI+<rIxLxV5LHkEo(taQvuk=@9nT*TC
zvxHZa_=ja_8-G$Pj4T;mYHdw6wLS~lv6z>3=ipt*Y|7EDNmG=VLS!|;TpP0)i+q-S
z#dbf_xE$VYmoqKr<}+-S@QuKhnpWV++Vo>fy17ZlX<r*K&Kz9lbMvBlv{kFO&mj?2
zHAuQgs`l0{R$%MT^lq!xm%42Wqo&m^_DXYx(_<i*+4!Je-v;fU=uL-1>g2b?dW?WX
zcsc1|C79|ahmp99!_;FD$MdQJJ6t7;DM3yqrzo+8NKZlsZkaH=IHQ&-w+cRj9curX
z$BM~tVDdzd6_ZzxSgLC&OohW@UgaLa2jN|ZkYC{eV01@?csCgfIQePQZI;(PBsmS#
zse&+*huJcD+!97!r1DBEFR!o3i_w6Wu(EYoQUrzKFved4(P56y!}#(8vCbce{q8_?
zcu4Gw+zLjG+RU>;x^;(#R9W5OiV>}&7T4r>NV{G1ugD3H`f3E1<j65;^Fh#lJRE4I
z@PqMK6HEG#-W_vLG@GKPOa^b!=?@5Jn&eHJgd#lNR`3?LS&;b~@`vHDpW>BOdUw2q
z1T3okRCRX(%x?l%k~uUOwDrEt)ayoOCjCA*)Czq0w&iqGzp3hVDC;b}?|A@GGasuQ
z#O+DHRp9e2rxVR|yhq>#hII$GTY;Sg<Q-QYt1&n)K{GozSr^P24KAxTJAs))fHWud
z)|=5exZlJ0O*xFGT#V_+`Tt_PY*3wbQKcse1XS?<5PGna4_v%{h<Kk0c!y=wf*_q=
z_hBmeL#G_Oz7@20(8GGGO09kE$kqi!^skBDKv3j!@D`Yr)kN<j$&B+jm`~>TCi>AW
zGF!VpEtZc4<QE(EKVlqil8+M}rSto;)i}Xy0=b_%z!nsBAKdK}MF;k+@;hL-$n8Ca
zRvq{D9*NY=d6xAp<TkxMXqUu@j8d!4&-^!g6`j&OfhkFL1t$MxT7E%qJJvrHv^l(}
zk#{=jZj4&*k%810_w4Xm5-nBA%pf^hZ{#RiLbTpx0XXVZ3)=I29LLQM<B-7&XMPJ=
zIOZFf^l&V>hGR(?j>2DpcD{!rqnd|9XL#J-Q|>{j@m`?t=wcEW_%fnqUPCz3MTqKD
zxhZIm@>J)*exS{{m(Djf(WP$`8Pss$Q^JR^r_~5DJqIcO3You<d8Qwkm@VnW6ugEh
z)!>ZyFu^p5aACcAd|7H8K<3PooAiQq9u}{28$5-c>M{Z_rKJ*R+Z2>*N~RKZiS#FA
zDS-e^8ABLBqQ`GUu$@!pf<?bSIOPGteTd0{^R7a6u2v?<8v87MUc`8c*YpyBIcT{8
z<Eq+GbC}HXHbso}IT$H5@l-^juzGXQ#{V;-(0MhC?mCSOh&FCJYf^h~E0tDp?abgV
zN_b;B)bK+lYdOQ6<@3E4AcpT<j(B{}YM~*eLr)TR!D8Xf>L^rXoFe?yvtwB|dYNC5
zdBNlRz;b-=nF&jB4c~JCh3~h*u`W-Fp?6u<RA!~AT)g1)Tu1~CPu1z(o<&6B2s$9&
zb374-XDVL=I6YD2sRIdp#CNoH>Lfy*_+IGpz02zV556DRRG#nq{+I6uup@usd#6)D
zyPL=NE?;|m?^a)Y_qcnb%U#AOeq7LQ>%nH$@YJNOk!24yqiiJREC|{uFu~#{7&kr4
ze{)bfa|o<F`DD;Nf7w|AbYC*^vgx=-mU6nCH%Gl|+!3^suqv`9azU-xGJ=_}6RnC8
zB-L=e(W!G_l<S3=58D~ZsrL~!!@EuWT?ZudIe2GKBZ|h)P@A*iwD$9i-+`T}J~Vhm
zJ8qd*qGe?rV<>s88nh$ei`BW=pS;_Py4ExAm*9C%+zuTDW`{`Z*3VpI89eVv+tgPe
z=XvH-wEVm$Y*W7`?{o4{^7Edo6$CA-pkm7CTsG@IYbHFTv0VktF`VfAm%jLMAz1M1
zY8)WUqVRUYAl^^Q5Il!RVV)95y+-MlA=zSZ3Pus6;Iac;USt*Mh78$(0=(D)oxheH
zC@3Z@CLudeu#=#7E3yLxhY8Qa)8}GbdF*z>|A#zs*xAf{<?Mjj70uEzR#W41tt_Xz
zciozKe~?%cw%(o}Jms@B-e+s~M_sm_VyL9V1#c2gz(SPx6w)ajF8B}OyA)$1PW5v0
z3hP)_VO5u{Z>G3xE$m8O2bZnym1pbw9$O3HSK$cQDm_0mLTu$JwsMEQ(|xn}g_;*m
zBur3ZArUxPXgG<Q7cM1;lT!1-jRal@8Z|H6P2y2c%?n>8aRRm;%5&M;U2i(d`8)ro
zc|GLK$1Yoqn*W)thc=aGD;~uDwBU=L=63(0vYCZmG~!v=h}(-j#Fq+2%w9J9h(X4V
zxy<N;%Dx_a22~uERcdYA0UL@X=RF3(d{prn7_4<+A<j(UnIsg(ylY@M3NI7>L+>`8
zauu;hS6>nNaJx(qYr>fFig^DQ!KG5f9@}G3#1%#lIdBkMQ(g_ddvDJMW3sKFZZ;d1
z1^w?N1RwD>SRnhg--1PuEjOx;W?^H4dVHBOdK&LDicmLg#7wJ22o1-3EvjwlioS74
z6RqG3wRM0JBx_n5T_xRoSMW}Q49a?Dnjex5-dSBZDjN?>*Ov0DriQme6D6aOM&*Yh
zX68&Ksy<ZCoPKONIKuQwczbmVvSk4p;oauq?T2ym@%D?qhPQu`hZmXe-^}D2S?}N8
z$9roT-rPOGj%9c+yLda6qqn1>7aCh!40dEuP}P%*j#+9$G-eoFF}g4{c%$Kc%ox~o
zV`DX@OU9d34kq~gg-@=VdSho-ED{(KiN&<r2)CpZ6vFca`fKv?$Qz@h7fVmJe@sU&
z7Orf6ug3hw(K-au{H^v6!UAomU=G@LX}^?axXau&zfc@0>AUgaLiR!4Y`>IY{QFF<
zjAX;7Lwmg};q<mnO0-|=4&i319=VWVf5YP63agZ1AETLG(HO-ZJu~bRreEd<8Z+#3
zHPv-CBW}D%amp<j_WNZ!#y9@{=PY+Cw+uo%-A}E_u=9j`FQ5;#%&@2Orq(2Sch9g3
z4LM#*(@jfe3p>MJYD#Ff*Kn^5u1fTFkb6!2->fBn&f+$Z$rG&%yBIwjx{}{~%qOvi
zqbR+O0C;$9hP|bd88Iz2tWT6r>sIsOdLwY>m;sy&{4L|G7>pa<>vto;8E26Wq(DEV
zl3`AhZqc~Z_%P$|!i8eoXjwhaWc2i^^-is9i0GLh8e(*fLxxUI@eA<!J@t%!$Qkb`
z8R%mdGdgRadk?Ct-L{O*l_}X$na#G0&I~!PDARana4dt*$DQ6YU99a)&nl0~9%E(b
zC!7Fc5~%ia28%49-gco9nazf-q@gxDGZGsbtfodHu*igIt-fg5q7y9{jo?OjD_7bw
zQelOpO_JC%jvw1%aU23wt@q+qvO#8kEPlcVaVv%P0E}pMI&O88tOZJ_Eid=yGH1nz
zuH?wLSq6dbX9vM76u}^%(k!SCjwXNkinv+LL8c#IpWG~N4jbw>4fl|F*QH<5_}o#u
z590f1+$6*3QY-CeDw%cFO+P}+yWw$@ur(<K=+iR7f|TrnLMv_(sP%2Yv_}XJ05`d1
z+$7l^25#ERgcliJnw@HzCH*epZ6xrf^U#xWOI!gLi9LHkGjt2~dd!dgxSOm4xk~F;
z+#pf&NvYLap4}#?<IVeNh0YFwTSmjrgYjeX9xsX;#6RTDns$lsGc7~)?TodkF5-XD
z_2l(G5qGBshY={kN)$Dmpr2)k8=*XXt!#ecJ8q;8BCt-(EWHm~PF5Y--^9#fcC7WS
zSntc^JgzB5I|sRTJC4R!{61meV1zE^F3FE;Ll58(b=^T~gYZ=Sg7{mb<JvUl-%Bt8
zI@=jKi9?^hW;O%HKz1><(Hk&#E}E5O>`t6(DrLA{T^E-^J1GTF)SbE`#Hp;8VF#Q_
zVcrp^idcvh_2N`m)>nXK17=mojZ5X5sH}Z=T27zq*$8v`Kta(I0{c@=*6&1?5f)r8
zSX_xdFtCWRE_#3w>%Gnj+`B_=guAU)5C#j}%{7&>>9*axf|d)0Or$5rH8Mns0vOQj
z`nX2%9y4aY5Z9=K87PJ%y-6JR@X#BSqffF6S*xB0`ON2W=JuV=<WsBG%>s<kL>pJf
z9co4_xpEwr5>mS-P%t+L*$4rG+OtyK&AQ_2uHkjp2!I_jD|K+)H9=_ng!u9mb=P17
zkRs63f4**igXVh|s=rK#2c?t1CVqBV-Tr6>1}JAzwpG_$ZTtydL|uK*Wpzf@`<#o(
zvFdWcn!S1n>|Ok3-C>nf*$OIuYPtL(H($zqXZ^ayeD#eJuC4}`f$v@po?m?v9?kB@
zZm*jL)6B<BeLF!fbfr}lUBXs3xo*6A(sx5;u*IvlVjw+_mx=cJ6t8|LPThD!uI#7v
zBI32Enbl78HmRY0=mO#!AFW#h9Y{$AGD{NI_xlvE*D%<Tk(UV1^C~5Um1W7-$VrfQ
zhG|JW>a|P)-=0HU%RKJCGtV&6tQ%_dNVGFK+Ow?+*v-+Ne!o<=?7e@G)l26(YUd`}
zTEzCdfN<7EIPHBgH25Cj?Y1Q7SM5a<v#>C~uv>JIl7A|JX)OAQ@PiV_!gN|F;wK^O
zZ8I!gEbjC(dx1rk^qNAI?6B;o?xw`tg;RuHk+<7&`hQ_cS^jn7+(Ex*r3@80<x^IV
zJ9M-&-R~47;^D0iW9#f~Xa0QUxF|3v*^GYTZQ(vlonOR<N*^Wi9bKtiat09NSN8vc
zmwB)EA?qS9^Yr8|(QX4T`uB;ah)Wyo*T*O}NspO&EY@S49@}|sXHU3dDu#R2t0ro_
z?ew3_5eYjlN-y^MbKhghWz)cA@JA-v9R%?Xcg#BdFGq0XtwxmxFdG9RENuILke3rr
zJWh{fJ=*KhQx6@S+-6zMErVf%&Oy@2lJlVYLJ>Oh+;(#rR@2jocD?Tq<myq0@R16c
zmB(0#f3QLbjyAt}koX4{)NC3I8p^bc`MPBw?nm5biSF`Q@8GL0PP8jXWt5n-AkmJ*
zikdpnrHOV$Ke4(X(Kct4Uqwp?6pXa2Ar4s38a7ImwUn&*d&oi=ysT|x4Z9EYP~>Ip
z_Mj?W%Q`~V^(_<a5VpjFdWWprYLeBqEb9xhIxJ7Lt6qb;MArO?WR*eP5=~am?-T9n
z_z^CyTk4WE?j^Fy=(;71tT%=v+BID2gCy0NnwloR*L!C8)Tg7kVBKTb%IT$B`h0Km
z8Znr70y(OIqNQd_jko48No>7nvQV`6zbESE%3CXwTQZp35lj?T{P&4d$h~DR+86&l
zv637#iN>dcNA*if?V4E@_!A-$r&2!feIOGe5SFc15Ekc^P7f4ANJ&Lom0{iV6hb4D
zm%z;hR$$+g1_f8^f=DvnH>n$-%}5Vc8l3Su(^Jwy4fZJ@f46C{dZJ#<3@izdTLRKS
zgn>$|A_8Nk4_9J7k!gei(N!HG?enl6{P3V%^RN|o=dX6exQ$`sH-_mp|FYK>-8(0d
z@~<JqG}OG+J#IDdkd)tSYooC2RYoD3!d2oQTX5eD=&3@ikly+X=SXVJlU5is)Jx>I
z{G5Tmi=KM$Auv1{QAa=ExV^s~5deA1YS?>@)MeShgp|ZLs5UAUWSPTEu8<8$Tutt+
ze~`QXbg3Q7J}9|4oimloPC{V+zv%mN!ZIa3BC>_BNi?Yf$kCmIDV;m10<ZAh3J(FZ
z)330X#2&vwY?x_bPUj3&c+;=&0x-|{6;6`)tCi@s5L<=a4x1;|rE}MCzrtBy-tjAZ
zM&eWP$$pNm#nv=5El=D`6L(Y;pO!bEXdy-MGQ>PBt7`R6C>}HfWB1%z1|T*Schi3X
zbw{>8lIAxWg!v^{8EbAxO?a$26NEG5>M(LYHM!!!W7X{6Ok<BRVZ8JMBz;TxT8XPf
z*uk3qpBpUuNHk6$nMZ1{K*dv-5u*fC>Wtci8cJaIW}t&JnkmtV$V@_^5;qaqO5lBi
zeKe=kuD8QT@`-BpPf&DwG(S;wbSmX95~OkoC#ZFFo@F62){I~1>(Lc{)o);emGD6|
zyCKSP8@x1C@9qJxuCq`0K+yA4e+5~S2;-5h*h6}=2NUa+;<zoK<{}C7`kL2fGx`#E
zau6lF_A|SgKqoeGXSmy*Cr>lnou-0uhvf@00Ez7^*Rv-rWW200lWM(_UPJN?i~~K+
z?xG6y&ijs~Y29G0BKNvA6`9+QjG-4_!<J&PZY&*GOqdHb@r&K_leedP)ZDROauLsV
z^>)05+Xs5t3on@LU0(KhANsJGk7sr+JNJGE@9)P&ME&u{OlCz*okwPs+6lU=Lr=?&
zuV#PEy*oqLcDIk*0~z-cwkh!#k==yHmDo>&vAg>PB~}wTO<?cQKJqe=zY#uA;uMkZ
z2p5$2(6VOMBrs#xNB&8df3=*!{Z#Tbxh)8(^eurGnCjc$=6)yi1Mqgj>@ucnK*Ym4
z_RBxN8y<E8$FIO(5An>M!Mx+Pk08r4k;$1Ih16r%VE>pn3=`7*K2qcH3+ciOK2oSW
z8p8^dyLt!z8-zzIfvgvyrx1SZQTd)vrD1i#rT;;A$6+Cy1!a<inq|f_ie4DQ$|B3?
zdOGU}9F-vreT+J9YsY4J*}9?J^ty!azsO$<=pgdE#fo|PYR8u7y*4HMEU-E9B+z)c
zIfRWt|IO<Rq}KcO5Ig=>I>yQSA?E!rNYI@4Z$L7YNJ6so10w2Wt6M<H?enDG#e%}C
zJRfeLVne(gX?4w&{B<f&r82#P5z<a{b{&gttaHD6jIyHl^=c&EgLN1hhE~_Zzy$gs
zb2A4L1}O0ik;#OKN*p1wlCWHfH;4%I5zQHw3_6Wm(|Qd-cV}avYYJRf6m=%v-^}0G
zpnf75P1f--Ju%sl?Kq-v#}CY;Po!JctO|tiL=wFX0REu?yY_>iO5<;Ox&9A=dTAOv
zfVj-U#hFzF>}DjHgu8Nf<xZBB?vJzEi45+p56I#)&3Jb6kdVZ|f)dA=%ME|}lN<Yi
z67|~G)3w8Q_F+2dj}e}A1EIEfXkacFn7;MsuXCr4rWlw;z%(d>DK#*6)N2w1hTV7<
zrX?_Gz&KBEdp(DTm~iWjsn=96%yrHaV=2!dq?>-6@%kZQFqcZ&v1o(o+Na8DVW2L0
zS(>RhQBEgnp@iO{9F(0O8sTPGavwI$O-;NnT;n9yDk6@y(B)Z6In+|?0|Uc&)GH2!
z9mdTYDi5a~w$iEfB|uLQ9zzIg?Xv8b|4nf&Er!Rm=+pmJNm>$qRc-tZq52)DLz>p>
zZ3~6HBvda6)izMoMqp-PyJo#8R7X8jI{OqJli9zULDhh_Z7wQ%(iSksXils5SSaim
z93N&2m8R55yFl|1;ax5RvL`)5_5o#MGiM{^v;Ls-P!oaUmvr7xb21cm1~W7AA!T<Y
zyVPL*I286&70VeuT9`i;WBPysPkMH<c_i*AwHvW_=~P-_Y9ON2v#j}{FeXE1*c-T}
zl}ZBZi)}?7VM4+Um^i7_u8;14@avFs6)(k}SPa6^gkefxsAkV6%vGE62_|kPg{IA3
zKAWofFvK+gLQHGwNw5ngaGx5#g~GPcli?Lq{Wls8-eSB6Cpa7w4ApEn<OK7G93{M@
z1QvYuTZGq?SW1K$WA;~cEV#?yMa#fWUdXA#nBDavA{l(%vgR<5bF|45e3eLLf{r}~
zak%ExBGgm@RC5{-l9c$2NJ~PR==$B`Oeiw2{$cs-p=uO$A*3U#)G>GWjT4m2rKWnW
z)q0ptA|MUA$V5WR+u{eBs=T4Dk-UrPAEyA7Hbxd?jdol+H<#v3UNcMR5w+@L$3uy5
zOlQ3`ALth-x~r}XZ24PV?fTn2(7`TLH=u0ramTwkA~3^P)&}|90>l>U%}B`TQoCVI
znK+)7JtGvhw+bDNfZjQG5(Y?$a5jw;MXUs@odIqKX<Xhsl7SqdEs2Qx0A^t3EQE0p
zk29bU-jG?v5YC5R<D8jHXZ&=dh|!@uyCG)m)f05fT!PEzmzGnxhTYQ?t#d1ms4w2B
zm*uNFk@0?K;9SF9g8jeX_@0MF3)L;H?khb5EJ|yhIFO;R9O+YIN&X2}04u#i2euQ;
zb*G<dTv;Af`UV$BO40CVtD<@GHBQLUJKzzYdCD>8Ahn+Ilw-~ZB;N7#V$M$_en2l8
zwG)lzLe#+eEv}3-JVT!|pdvye+u9woY8qK1OLkw`#D*6IR1P2>09WS5+Bnf-WwOh!
zk#hEogqWZ7b-6A6Y4MgS$VUA=$a$j1<!oeswW#=5AJ*9CMtL4=iLkabgAgry*@)rM
zJ>PIO;vh*zz&v?*845cKUq)4v6nI>XfU9%aAe(y~?KiGs>8BQ)k^fSg>#L}PjiKJX
zZQ8h4HlOqP49GGg2s8NFNAH*foes1PiAj`<lXqu#Mp)C=0DhRE)+sBDb#ukUccEi+
zBsM0oVmw84m~RI(h6?Qb+*(SsAp**|^3`g6Z0U#XoKZRnA{Q!a@She<0{PkX`bV+%
z?Rxz~#SvGrYq#<WU-k=Uh}*UEwC)4#-$bwB;Ix1#nBf)tqyj0tGtDu*%PQxH-?`89
z%W1&;R(rOlfyA1L;%~MtlSyQw!)fpZqRWczXdOVzRzPBMW-E35qCk?7!lUYSE+Jb9
z^?Dv*mVDQ4fOG-&pi1E4T!eq_CKh%_2S~u^0i(=@Hy{r|sQJMa-`u(0TOHODWgk}F
zdD%!~?ZY6zVH|x%b<Cb3!l$A8Km}!)uKOZ3v4qi<;b^wMT-KUT5o_ZN@N<Mtc3oAG
zM!4RvKA{9#?ea}`t?3B^Ub|$$*~(9))Y+|Vo5X&Jyk&FdrXwMBnGnfkicMUZxBxcI
z8PDW&Wiq)Z4S!#)X#Y<3#AtEvRbp3W!;)i!Bb<Z1!sbEZ1t{9Bk~QSMPk2ug+?6Br
zkIh2>eu}pH;V9;vU_!a_CIA-++Okd5!uHDFNHM?9{Z)y7@#R7+p_&pu6T#t~*BE?P
zgQa#8GhrrOA8mK1c*08XT}6E7-AL#GzNG8PzS@M;Kmrc1bCuzp$22-`m=c*pCJ^*}
z!d3i&c}%18rm?4cHCH+Lq}CBuDM2^qJwn*2L@^PTZu6d1Vk41H2_JFV*?zp#PQIXC
zCZ$B%Y<oLnxJ@x&zd-rF2|tJ`&IJc-=G*zvgbFv1cpD*Y%m2Sq+Lt<a3E~q<69`<~
z5wO1^LbvAk;Fmd{MwXi~_U36>snN6G$CzZ=c{&A<TL}HBqu-;RKbDZA?Ukob0)AHJ
zohPqDqUTmph0iIUO2}8@9FaK$92mU&zx1ascv0qQS&`9_hcP;S@|R%%Z6K^e%#wfB
z&$A<e`k^6&wON=em19_1{#h!MB$dtTSb@5u-6Gd2?HO)H604LnH2UueT-bUY#O#6L
z^Y15UpJ@+6;U>V)WFH&SoLlD5H^x$veU);E4oggva-%<q0}(%N<RaQleHPs=qUT5)
z6e4CjgUE@*Ti_!2`Nm2bdQKTM)1yCSf*X6EkLIt`Jno};m&DsbbLM~1c%#%5%1UiL
zt+41XalFQA8_SH+Kd8;7lsig@#{8dTp*2cPlK{7p7Da0Y%?+pmqqKm$`B#--lolX4
z3o0^5Mi!c;n^t(z%D1$V+s<ocL3QA&daW!#m=?roWVLQ<%0{etYGg}M$K1a#vRbz-
zNG3E%B*D53FyxFUxGT5CSqVJL3V<{orI%Z_ZgbPLrghr_eC!3SsaDplLM4*cMoX`A
z`O$i}TU`O`qJ+BDmypG@Y50&FuWso{x9l)#jblf}t%d8gV2EF942kTrTKkQ}?sscd
zQmvkTtpWhGs55QNB{93K)^XDoPqifvZ`^FZ)=B`E`?a=^*n~ZEDLe(dm2_{kEkiSo
z1DGy#I{|vYud#>36S9C;cOv~sMy?)<wrf&j{0>>HCjr2iEYMFuCdswGietV2>00oX
z63E<w?+IUYBk>L>QfV`r%6(OeeO<^PFBHjF5tD^O2!j%>K-$xQ=F0Kl$zSxV()YB}
z6mKM~SK<{S(7NyeZT~n|ua?>=64k08Un6{2;f1>ikD~;CL?<HVH2`6F%u|B%ZpBvQ
zBCzN|N(@lR((LJK(Bn}`D<zVijkfE8a=ej$(Rq;YtjD8;BIKxeB%70h14E6MqwP3q
zOo)I-GB^w0B)p+^y88wP)EJy5m)!3UCd8}l$j_w87!OLCa>7r00~1=AlHW`o`!M(C
zkS7b`ME+0?npsxU>4ZrtabNZ6H;8jF59(hsK8*XZq3edR&@e^v^Nb!gVKECiG4vN9
zYDRY^ysRE#lox(Tc;8dRh5saR!7vC*-ej<uF5Zl`+tsATuYL_Sl^1?5k$BUqw#X)j
zA)&RWUZLIy)7l8Pwe}2A9;bN&Po3p*BS=^jMYX3sBT|!4-7ty;%r~zCXI{Y@-`4J8
z3aYwWUU_(I@I+t!*-aX`T+N$F9_uN8(KSEWly-M{m8UyQ7;m^}Hsbus+g*nx?->>A
z;^A?4&SGMdhb?}4Ub&=AU!FXRW5ASYo)+KR+kNF6@cUnZC%=I7L_&hqiRphk)sxj-
z#NrBay9iBKqzptYc3Y`^0|E%YKCEb824ZS8G4DgT5c?=_OrDFnF~`O}L+%fFFzlKJ
z*x=@(GQtpp<4c4q+l<DQ#&!*;nDQ1>#Qq0>JVLGzUZGkonpw1nFh_~XM9`o`+aRM>
zlTy1m`olqinvLxk$jCnkidq=MMM(UjXF*XP)eqF_M(P;hWhMF%VfnsDLtRVsd_)lG
zwXkiA{zc$=lRzyd>P0^hez=>&YUOO^H6G5|B108KEUr&TWFpZd);vjlso%2k^GvFf
zvY|th>f$tlBt?^|n_yg;H}WR2KTWDkFtzV5wNvvoJUls6*qATMoJq&1Uwbe0I}<wL
z!qk4jUDBq`b_HY7(;DOQKB<}gQxS}7vp?u-VbM}iI`g!w{Kk6QGWiMd>{C#)d|TX0
z9sQcfAi}_IBz^%JG1s@PlSiubG9}pr`AxMWiKw45m8eQ&8DUAHYZ6n>%X;zTuki-o
zBre98S-g%$WfB*QkN3$W25L7i)yWWJ5+~#N1!^O5iytTKV$D=L-7|@4F+$1{li2uv
z-z2gSOuY=$Ff9pF)=8bvtL8P`ZI8+3A{e87%`q(637KqGL0QtT=3Tt3Z($<~J7pES
zoXw9BXN}6SagPnMT_h*H;-)H=-5Tc=yR|QP#W$(g=zjU)RP<R3vHZJK9BDA7-pXXp
zQNH{c%A090PS^KIcIpVAar^SuQ{pbCr$?BAJg<Q3BS7=AG(Ig!c}h<gC+jd=yWs~;
zlX<dN(AdUSIaB(>|5_*D?X!eul)zD2{4(JPZKg1A920k@R8Y?%kJ9l#)LM)IMc=rs
zPN^kw%+fps>i+UJ(9M+9+VC&TgX*HJRFT%aibonrgDBUszr#P8GG64mGeBg#DPcXT
zmjYAfs>I7<eWm$gikx58YS(O_TjjWG1E;*G;@O6MLYXT*(0Ixz-x3yie32zg$&xPc
z#1b~PtIGymf)%oai%$_=#}Wo&{|d-DK(K^?*f+>#dRhD)8JM7$d$5FN3a#I{qK#p2
z=Jb};=?rYlW5AUXzQGd4>?e09hH{8X;PsL)f@5%e#kln{)+0B{ZUGf@<cG#w0-zc}
zU-Q;yjTFGYvX&$h8YmGW(u>d)GAc=R*uDX<ILdAa8MEU+Q7N7B5rn&hf(__ECDq6y
zj921TBFqd+ij)xCV!}e~U8Pyd!RRj8Opx)bw3^5P!qf9eVE<cS_5+n3BnMYv3BP0;
zsPrDW7%!UI6ZJcu@pLT$D*Z??7Iw+|N-!BOQO0*7Xpb;l<!Ep;fs3O8m7|Ez!KICq
zz|&gVm5`wXhGFR-!a(F=*9$bnHUKD&MQMY*;1}GMT@1}qI<fSwMAw^c@vz)XJ@KYT
zv`cioY0`|jms{pdyJ;GdH@y@FEgera-{)^pXfW}#!lIsvwVXnK{kIhHS~`t-TK_GD
zeM=XXRl+RN5Kk&SjWQLEgah(VmZC{Z#Z*&iJBfQtC6*&CR+&mXsl-NzQ@BNyJ_iPA
zTKb3*=ZHK_*jLulCR2$gl@6&Ai+#5x6ng24%otC1wDSNrE?<ihDH%@;Xw-88zv6j)
zOW6%r`T^?FV!P+{Igu?~E{L|{rWu~pdy#t6TKKdvX)gW3r|l;aKggh0PN&i-z*8-!
z2%kAnxf2&DkzRV$TTHb$V^H#h@+0_g)0TLYFN*<GUX^&XES6A1^lLHI;$vAeo}%Lw
z)MeU#pkIrrWl4m3{_uaSdBT8|TTGE=G3Di68-5{^erx!bH77J9Lx!V;{M{U5hmK)$
zv>P`cV<7rrNy|o}Ur*@Q>wuyj9!r*W0Et-{GF;1g61taFN;Q>uQt5S7n(0>>2;2ZA
zRB1S2SXrfZrV>vo>AXqNUcXWvaO3?-$m6nUWtDPGC7x8$`ljfDUkOoNw$QJHJT6;Z
zRtbwhpLkM<jbmqetX~OHU3QOO33*)hh^d5M-EzLE#FI+e?3~_9a0vUdg9NTD^F`3*
z8!V&}Pr!Nc?o6NKw{ZsW(|#M+jb$I24HdOfVk+^Z(nsMw{;z=fr;pz(fGgv&wA^l0
zz&@Of`U4MYhGg@)NOpt+1ikmF&&Flox~inA5Mro)T|7ROCzdUDn3j0b(pU`4j5=Ox
z#SC?^%&)|<xDBCI8Q*r9N<0xuZF0>pBZ=x3XA(MleJbupqL05$Z@C|EtNs^J*kfmg
z84y)q1y+2gUxD#09#1Pqd0QTp`Qm9=F;Np#iOJh6*@|ZaHv{jdC7XwTxlcOY@&k9$
zobj#%*!YLed^0Pu>_E6%GM8U-5Ji(bStb%Dn)*LPeaD4Fcc(yxJlSH*OYsIJ^8ExD
z($1E7vijCp?}DHVx1-WtbVnab>uFhyqU;EWy;}>eN{2zf#8JEhK2|zOWG~?fpZ7pR
z8BcisIcD?j5-|cvD#r2@AN6_vcM>0!@t#qn5>F~!P(yF3l300>@J}UBsbWl1@egJ4
zRMS-ANu@7*d0Ni2Qhc>co|s%K-wbM89C&ht?ty*Q2@qBu%xUG?gjmUw)&!NaNi`(Y
zR{}OIryI-D{El5R&GDqUO;s#s<`l1E%QFB=_d2%xW)i*qj<qtYT%H<kZYObO-jDzK
z^yglOmfr^Ot;W{QF11@V0;~(<)mHm+C>l$U&t7>skwpaFEt<`WR;?sPJo$^-feoFh
z-7aldEWekqO^H{CAkE7kg<)nhqSXjfX#{m5qaLi}ZAPejn8H?O*}_xZ+a)H+#ncci
ze}?dsPw^2FFP3$0hIrD7Czbx9N=y7or+|Ciuk-<lb7ht0n@T*X^tdYNUQyE$j@$Aw
z-jtH~PRoH-EFV6pWnMO>;aZxNK^h*emI1Ta!5D9MF@jpt)t1?gw8dRV8*MQ{t+{|z
zy1OUgdaI_8x<=r|gES=DeZn9+3DRZ{@=3b_HNSiPS%IIkB2vDesf7wKqeO8eQ8jcN
zP@P#(gHX+{!OXUzo~TS?D^dFjvv-I}cQe3{pElI>nrCYp!0u&WSqI5!p8d99uQ|<L
z>2B9&p3SyQ*^<LphaxlerS129=IJ63rIyc}ma5s8Cq6S974lZ)49RC+fs?kPxqRlB
zR{=4jitxoSlq(SN6-<e;I&}_P*UkeD?eD+Z3X9CveH0j>+m7N?FP0<OpnBqBcrrI0
z6>Cr(T({wDt{6fX0Ng%g{)YWh%m(%9e+1l|(<sNj&5*MXfnPy4RurmcJ!<aTNs3u-
z#X`h}dMxlxo}<e5;#O)VsK0jq{leC|05-V*bNMmSKb6sg8}^BFRFpd~6tg5LyMa|l
zQi39jt-##XieP>RXVW7mErArC2ehOlo`=#^Ux_Z>O*m;-wiO!*>y=<uS#dvMyAr*K
zJVDSlWg5=!ilc;=lo&<iZGv2mw8@sW63i<&^W~&rsaN(TbXNi<th}3mgX*MhAwoMV
zS2K`%?=H3LR*|skX>rxs2?lcRzv0~86~H`2*lY2QDeZg6yw`G;zbzDK)XHOoBT699
zD>1?=->YufzcAyrF-w?+p<uuQ*K%GP*_T*4wPmlvvKlT6MgD>LT6CU%H(bUWzDsAh
z&KYlO<rUuWUHXj4SDD|DIhxR0iQkEQLO4g_*GR4o0MV*HlmN-<>j=>_@@pE6d@Lzw
zcs3NNo95@Uon`-;MR_Aak`kSXDCqoUL*{<_SA=MFKH(O|y6pwVbFt;jlQ8fDUK-A}
zmy}1OT~$AlstDli(p~WR37!btbx9J_u9ugs9XCnFaHE9n27B54{OoK4o$qCj@w3hG
zxpv5<NSUusAPkaC26~s5y~)cS5u@z(g<kd(UiMuEN&D?y_A5S;!v<PnE8HLY+2^H?
z?N2G&&$@&>h^slUAYEK-6d8)l!Zlwo7~VC~DE*2(Sp5s(TZZn}#d<lb$U2=i5jVTI
zM=2Kc0H+H5vlO(|c9&&e)&{YAGDR)@How&FP*p<Gs8%R)DaZa7@dCj9?E&t?N+ht5
zwSU_{1a(-W9~E!J3v>Io-K3r+YzD>el|XUWplBbeFpI-Y3vnUs-{ZJmGnIgWvVTt|
zf*-!Ns}k+~65(qP^}qKNr6B%&AVqf(2G=1m62uoY>+!?{hAK3~8(et5P!s?&jZmP(
zd?E`7^MqkF5rvgLhM%Qr9e7$XAx;Z`0VD5XC;^Too{QmO5)Z};1FY(B*$P}S=j_rl
z$$C|LW6HjWT^a!gZDn7sq->bp;iwo3W?6rAMKauhN$$bLRO-lPw~nQ-mop411fM4~
zz+e0mDwQCCDp{myP$toBL?wvTl^T>t?#&*xp}~-*V#Lc7cQF;ULsLdM#T^j~N#CN|
zDVAi9Fj~{`Rx9w?H)KySNNMi359yPL&Q^)R<>wmG+qUx%X$BOcVT;wHwjeV24|>^m
zg(<t;Sugvvmp#I`zU|{Jr$r`N?nW6$U~EZ#qh6tk4t8pJO$kOXM)Oi-24z3tNhR?2
z*1ku0Q!2t{ZO42o48Xae$nUAOWHclB>psdcU2E}m?O$KCtaX^KbrHfh7Zks~PO2K=
zddYsAK><LcwV?{{A-Ib_rq*wa>AHG^cqKCYgix$SR)4!0YpMT(J=Oay$2}5e|27p^
zRc<E!o#&*G4mh}*hUP@LDsZw!;NCGDP3-osoaso9v?`dV4z5*rABkDgO=|s>+t{za
z_>G16jRk3_2ca8{r4Mx*^FGA<?{=U`K$|oc{y1EcgmBC#moK(pJ?yyD7?>SKk9UM3
zzr>UmzphU2fR0NY1M3gU&RpshSbsp8dT9`C-$v-m>5)rJiH+EUEZ(OQrjT{>2-B6o
z%CCErz?^DdnnmOc;j|LPL_Q;YBm|q;qvoB)(m5)*huo`#-;{Wa3ydIR{dHKu*i1--
zu>hxpB6lQFA%g{2>_E!#Ue+f9*jV#=><Chw2yKnfb{gS|;nKm9Ic+7Imox}yr$VnJ
zPbD)AqdLv@7{$J*#XM|ZdKR=D=-w0&bO5x>U+V`#&~?|9+8JSnppl_SeMnV&GX!0i
zK>0Mn6cNN4z_|`bWBn$=I@L-ywE~pQ4LRE}YQ;5F3)f-&F2cjAHJ2kj*WGD`-??t5
zhE!;tq!=%3{V^q;BZ922|4E>K4Xl5b5pu43H?V%ciJX_HDCgV2`hBMMZ<e(oo=^wp
zqDloC*<v}%o&i-AlwbqA+K`4W^7C*MJ;o31;yf+eid~DHBZ}k+#p2a$xP!pZccNZ3
zSSS~)k&sj;pYh&6by%A*n5;Hncnj8wCb%vdP1(op>BnVXiS6qfnj*|5V%oWir#P;A
zZ8q>Z<A%yebM_g{VI9pMHn#l;%g&vzT3uWlJKij@wGw9j^A5e?rt9#ZsRLb|`^)&i
zbJ_7gRN$g%)i?wqMq>_Cx|x@OTB1w@m%L8#uCNquCr`F-<T};ul<LWg$vWyP+irDU
z4WKTjcL$MLA!)P2L-L6b<1s4Fb1;g3RNfpn&+`|S>p-B~B3f#5$hVsXATv2^qF=M5
zbu4qyd+U>$^7OR21f5^%V^MFxx$HB5u~|e|z?#)*0lv1`fS`8E5p1UgKHUbU`VDJL
z@eLGjS1S}3%U+uWP~BE8^~)cCDr5f5Ym(M11lZGbmNUvOPZ^vx@%W}+itO&?7&Ht~
zt7AF!Pq8$-Y_Xc*$zRaHF?3c8pr!gB>hXmhKj?9VM}vxbRMR6tk7gWQYS2}&oAel@
z$4EUU>T$OoOY~T;$9;PIMUQ9oc-i>Y4<XUd(qbOXrtPBJsMdzvrU(she4L`EQJysK
zZC{(8`HZ)iOzzKccXM*RL1`P~_0D^7gsg~xX|KV+D#XL6WE}X@G;oCQg4UjUIdRgt
zgkr1l2f=`jNUW@`1ps=r;Vj`DX&jq#fxRP0eM<Pd5~$V&_S`mHG%rA&{1GI(jrrkz
zeLOQ~C7W!4y%mA@H{mA|Ph#FTvO?JyxQUWjQ$d#JLTs9LUG}C@1eqHvE75>RJfW^y
zXDcVLH=R^dLSraJQg8UiGN&U~`abW?h5iyUkj4s<VXqshVF$KuBN+2JG`-=;%20Eb
zuhn*B1^rv8<NvISg|^jh_x{C|S3J;RF+kjW#yzRpiowv4%vJGzY0{q>6#{P&v8s~?
zl7)`?M>1Po^6HQB>$mgkheuUB?vmZw5?P>XUDc&;l{pvd($_l475i0Nd<<FYYU^CD
ztF66hWEINpszsi<)f#V3zo4+{J+doBg@2q{WTiD`O~$VmqZgeiE}N017h0`u<@3Y3
z?X+5LT$;!bI9q-pHyj^<Tx@oe9*GZ=yRHsO6mg#1JSh=R%W4%0cPLstnMVc8%|@J;
zjjd>??xwP1wRFn=G0Gz5!LhQkvW_HljsA4dfH?d!sJkS@JT3ctDC`Oz)>@8rIN>Zx
zH!|OE9Km_;@XMuk=1J5f0>7k~Bce_e<Wq;uf7P_Mhr&K$ep|B|n5gsaY!=0QWIi*9
z`ptD_oM`&BaSCA~7EGtEx+KYh>1l<90yndZyrv#I8wX<iOhi*}3pOqxVAA|*NE{H2
z+-iN<zg#oKuVym+YU`-B#>^mHFy4nM!lN$m(Ku}GR*l)P)%RlM95x9ny6Z$u1bmal
zfHfH$3Omo)aqfC2dlgvkC9r|#MBxE#+(md4RcbO=-QVVQ|50yVj(WzMm!pmXx1aE|
z=s4y1a_*d0_AP-o^UZ(+(NOUaoM%1)c}^(of@omZ)v1VsyAdO^@kKS8L$ej%cblzv
z!E3hS4_>ns@v%0(LEz`Z?Mx(`!=BFC`G~5so^!U+9?hZVm2&2Mcfww~!xYz7v3Vu0
z`4Xx#xBnm9b9-=Y);6@GHkrIE<<K~-*5%lieo12OPoOow4`JrekBxR<YRqYV<3WeV
zbfF=CPUo8k_Z+)4CS?_fc4)QSWNM<El+Axp=k;w+$~L-pj-bgU<xwJ^6F$ZLNqGj4
zgXod$f65_pe<A!^rg`(AQoHLnvZ_sv1p}*bgg3UqIBrJhH!<d$!h{_MEU<YpsT&B|
z^4&a}ND3ht6kEoZ+TDy_*yP7>^Ui-m7AUr$UYoF}o7xG*kJQ?N@NODO&>5XAYl-09
zZQ82@lC*g&p^p-e5%~|{A4)*_7Kqt$lM)Q}*6JiKvnRLZZ>Cv<qG>|-6r^MW0utEr
zH6SH~O|(h@IrMewlLR(!16u*#dV=t>5;y@{KO&&&fvvTP{6P3tiH6+hf>kOBDAAG#
z*h;Qr1A1#;(<*QF=rTuD!f(M=tUpf6TZaSEpU{^XF|f5W5bc0yMDaJm!jP5>BMd<(
zy5S^roi3}|?J%-7K#?<w$OR8<K~9@Cr$b`(mQ&P$_azf0Cp|R(`ul;Ss(6Cytm%#;
zg_Lw*nzjRL4$os#nLmy18&s+()oI!%+^)p<Z>$Qjn=gQoK`fb}#0Nyy6V{4nLuqt#
zDXIGj_b73h$g_l}L=Hkxg5H&!RH7=8_X)g*3v7ue@(tml63Lde4WTO0+lnplVOvcC
zpSK3Kbf=Ac$Oxd&TYA6E$M5_2$fU<{EpMCVhksMi^b^N%`vSUW!!~7fvmj&ziX)wQ
zI*XD(!LL$TW2a=sd#57FF*4&k8JS>U3DUfYt~<Ru(Rf=zYbAOU0o}IDL=rO%IEK`%
zl#z*_;PhTbaX&&IB_1fB*hu0|0*5P{-cJxgYPZSs^gf8`LCd>hp5+K^^haqV8w>6!
zI=(LA>*8AKPS<3AgTIhY_=oJ8K)1$>wR+cd>15N{!N79$;tdUxc#2ithOnC*>8{xZ
zW_h@sgb;2T2h6_Blou10DzTi%Cc*}d>vBdyThrSV=#!7+kU_FJ+-@W!Px?sq0c8?G
zViH2K*+=psxi6TJ>B_F-wULRTH)e=V8OJb%ec7%L(=8msO<&3SAlutc6JFOmwz)&8
z-E))0nDMl%_+Vf?z1_?(1vcLZ(C38DU{}vw<=a^0wh<W5_MSWbvaY)<M;9({MmCI)
zbUSa1?oo^T?a4N$KO;+JR8yJeM57B42Wcta0io}J-MuFB$xvJsFaLh!>!TYSG!&KY
z2p(e`m^`o4j>Ch3F*_6WerQW0F=;1En!F0qe<XZEl{iLwD>tv&acn5*v--{7fzyd}
z+qu;}k+~(><W2J!xL}0jq~~#8POe=yp?b!9>eCa9gXKLn-J7^a&40p|qkX<NxfPUK
zcdNb}ZHkF?`k=-4R3lVmLg*Vo?kiM4i=DoJ-g6T{JNbQsL@*xr@cz!}iy+>JGPzGH
ztZq+4WD76YoW7874;K8M)uQsobk8cu!D6y31OeG`dF72&7DSue{oaSqfxexSfDUv?
zPf~a6y<bq}q>;wIC;kgswvH0<ECuYnaPppe2qj|T1y{qiYRRc_99685zOla*;97QP
zYk%?4y>z<Jw0I+)>%PuzE;Qto+VQEt<yn~3NmUaMB7P;?DKNi7vGw!>u5wF!F{G>9
zT$L`_=f2$9s>6?gC|*B$8XFe59NOt=$x(lp{vTiG0bWJXHU8PXV3K=tlhCUOCJ2|1
zgd~7~Ku|ysP-!A63P^9#I|>LYC?KFBFH%D9MM|U>X`zJ{AVBCvsz$+vsQ=%Y*-ha4
z-tV91nVXq2?aY}or_9dI>TAV?!#G3Ca&E_=toI{lTc5L-dmf5JfB|V2h#W5eSN4qT
zD9@i8Opa_I7vM~NCG_3)ZMpn}r{Pm&Rz|jqG~b>jzOGCpss`OhJiK1yTV*KIj6ImC
zNeLW#SrXYCS<Lk>c}Vr{6!~stos#R`eDa~%@Cf=T)FiT$^z)zZxo9f#8EH)}t(rQL
zCgToAvi=np)(PYb11X;anQtJmIgsxRBp&v2YV9+SgdE6O14+z*+%=GjIgo!1q*6BI
zKjs3x$jUj8NK@w797rVtsgeV!XCPH`Aa58*wH%0inud`+mjij<KzJtZjN(6^8%T{D
z$ZP}QbeLOaje*ErU5wqq_eGO0Ts5jk#DyogS#LBX?j;xPOnbMC;)cJ73x#+svOJ8u
zm@pSuxylh2N>M6>^j^XaU<;~5T!>+pU^nS&gbUr;^&7<1i4jV@u~xIBia)6AZ*1C?
zCm-g#h1^sJ+vz*?xRfnEQ;gV0mxAWWTJWJ#eQrdg3V!g6!S5T};Fs#)_l<{s|2Fug
zI{1CV|8I&J{8AnKzDWXB)!?`PQT$pt_+9%y@k2QL7yOVbuXF8XnTTorEERVH`j!{^
zP5&@f(_mVp#Obp&;dO9na3x|IEH5<uW-MW(!Fd|qpwRT2HH4*4DK2n~n9giz*acuG
zqTcJ@Lq*IG*c4nhFG0RM|Kys8nF2dc8d^-x*KMGRn1$GzF5f1Kk`HD_B)bKwQsCKK
zS*k)C84DvmC8O0MAh@pz8Bx?+XfGy;l$vP7$T$B`Drl1Gubz$!aYae);Nktzh*Avw
zGW}~p;q1nkV4Z{Hl!xropUuv3B+Uq&o$6F<lAY~V#77nUrgw95<h@q}mx>RTIxf_F
zF9UJ$##+Q5G$0SVO~t(`lai7ZQG-b-3?!HSpj7gh))1%0Q_7mL&g$A)PY_?6VS>2u
zwx5bfatJ=XA#|!2)mBB+aw*pl$U9s*9Z?JQqKQ2h$bX{R0ik~lVe+S+k7^IZAfskP
z^DUals186-WKw1Z?aXKs)e(r`DG;2X8J|aW0)j!WAs2yUM0EyI!a!!?#>^}b)jVEB
zJntYhvpJxqQOy$o;c830ncWHR!dIhbM7)T5kvvs&%}}{kF?j}L=tVJQ4UFi9oiMWu
zIx!h1W9Bx(7K+U7p^IP|SRY4J3sv?E9(midG0kQkCW!kw`vW3p33Bett&rPkIXI$P
zF}K2P%G`7-{6XS3t^SiYP?iSiVsb>c7_eAOq2&Ea%?c(w;bJjH3>0f#L?y78In-%%
z7WA6MCY;7%77BRQ({k;EMth8y6xN!EN=4nm70FL<3)do1(_nBSOBK<EyR|K11UnG3
zT2UzZIt3aNUU4v>2=<1R79lrK&XgmdLWfyx2=BO!LhM;x#laN}sKC5fXCk=i%AOVL
zRv1PZ4lo*&rG<}Ol&Jt8S=S>flyeJDC4Y)rcs>ajOq#Eun}^4&Um`~EGU2RgQs`X@
ztRu(~x7jU;AbV%+l*B7UP7sbu?c|@cR8jU$sQqum$S?rnA<Txpvk=s?t~mHp1pMv%
z5z*!7;72JU=>AGzmr~=OOr2zd5d)?I0@sd=h=xMRJcrg5lHvR02)6>vn_S!-Fcl!F
zoe&Z2a!GP5`8<9$tv^Qs=_E7ClBG1{v7e6^sY@d-lUb6<FG=aLMBXC2AqnZD8=;F*
zvgU4q%v2af1VAO?V_1N#wnvCVug+6Gc|#a_sfy@_^Kvug<Yp0x*?A`<FEKl>&drF}
z?7W{PFBWlY>NTGkAve+JHfQ@KFOZ$rWmH5|c3ut1i(>hz^I?OD!MDVQ5Tkd~mRCi{
z_vB1lJ*50o#oh8xMY`pm!ZtJIi*1Q0nqB@2DPPpI{~j?Z-%%81^AEzeqA+6zG4ooX
z&uxvOiAf9J>L|?Qk&tb$RzBfdWzZ;}<fQ~r@(ZejZ*>%AGQ5<0NGr_n?V>P;vp;CV
zH*;Ile5SBgu72UNg+BKMC_EPnBKco}oI9UO@3Tu2ghA%AVrD;2r~&r5`bmX}q*@W0
zOJX{a5d@wD+jAEYnLzjyM|G~45b}SkXVM}Yg(I&^h<2DG`@Kech4A&WknOJeGTJAu
zmeD?Q<i!WnS3lHOaE|&)>TA?jTyvVfNNYK|(9wT!?VSdG%kIZfU)4o@@gj__I5#3Y
z+pT!eAUL<2o1^cxLVd+$N*x!fx4^Ypi<>Gn1a2G>y}2jvcc?_JKHe5S*^e1K_duAa
ztnkT^5)T$zB78Dy<9)e^wp``#$p|@pGjrwH;a{?zx%X9UhH(8!LovgjToW#zaRWY3
zg@28WF!xBP7zp801*$*1`o?fsM=64(UPnoyNRNCOUX@i@5UFZi73*f<Q50!-Rd}Ae
z;*BB=uZkj_dq6AF@aJ?d%-<41GKw@@-x~~z%|?IBK7k?)ug(e=rknj2;U0=Kyt*h-
zAakB36pd;R#2};QybR=d6lr+%0#2C^feb;BhF2FnrPgA?Y!qoY7YG>05g@53((vlY
z30;Q1=KLgzG`xm`;#}m(+-F3QhSv~9N=tL0`P}&OqDaGeg~VQXNfhb(sWAP*>yV)r
z#rT{+csms7{9ds3!r!ToM5uv}y{?Ncg1A&W!z&{m=htL+VBJS#Hz3I2?{$NTv>-H>
z1Gi2E6sc}GS9s+jZiTj#Y3)|%L86<Mz>93kqFnep9NrF*J^vT5Sj4)L6*reV{7s|7
zM3JiV;k*%M&zCJfjm0R6jC33QoWyv<u3I5D6>f#c7j_Hd!_Jjc#v0`XB<2|mN@S@*
zJPW4uZ{dSbqzmQ?25}Vlmax*nfFh^~D|dK#Fjye(KWhv&Q)HvtXex<4=qil?6-4vq
z2`^um3Ip8=$0>8vt#E<FITvLrFp<`i;pI!ag>R96!!7(9iTeiSk2K0~ofQZlRGj9o
z2+E6;n#VBa{ws-@Bq%V?D~T_OAS~w<5JkGE8nx?dMOrC*Fp6}c=;}oc$%k9#m2~m1
zuLoQwybJ?g_^gA{ycmi^J1EUV&CPo@d%*SefE$N%t&hF1vs<AKWe}&D?DHCvcqMzl
z_4R;Tg_m&$Jnt>?(T1jQClVdBB3;x%qbvjN96p#CwHFN`vkyU-b`fkm?|p(;Sc^Im
z8AJHkC{ngs7G81$0E%?sQPkDOPs1z1Vk$+-+bEv!ie_0!-l*`n?7R|^7nhy){-*G9
z*?H9@uUv6yMilV-DdDBF^IAw=X_L3idbw>^6<$&ov=@COSrD0O=+W?!(45#1VgX%Z
znN^!*Zq~iMXpNLER@5zBtdv{27>k{b+bLBlGOUBe?sdIOVebeRBk>kx1mjxu4tvLl
zm}W{n-7V}5oyl?|PifC?abpzZlLf=xh|ty4ysshKq)7c@l`5E@%_se<a)rGW7s;gj
zrPH!v$ph-XS>vSHdXr~m^H?_XR#HGANA~3pYxV>Je{olO?K3f~8GBq{NKVZ*VU2ap
zXp_^}<ixW(+BNLuC*<u0lku|2=*j;4xv-b81oE*8%!bk8qYQvnUou71yzO+ov3r)v
zAEoRiPl@#~YeLwIQfEDNs*Av1lH1gI(XDfyIvLc-m!<1~+K{<nwFR{$PZ<nqo1&<X
zpiN=VaT#~M>9kf{T3cG4*2=yW_MF?=16up-DYk`LhPiTCK4CK7A0-!;ht+gs^?asu
zeqr|cxDi5{gQ&XETwyhM2)SgdLvCQGXD26{+$QaidwywJ5@Si`tvrSCQuQzA_rsF>
zaJalQHNOHO)|pz0NUO*{CegitLefGg1I_76r`^7uqZJlCe@)oa8X8tfcq&Oo5vGS%
z$&g_(;00&GVl~GJZyqlkR#N0Kyt%Y*SdYFctR(V?BzYq*g>j<5<jE<z%28oO4c{^q
zvzCT&ITf{*kW(`)Ox{3e(*U-gwLC0>*Lc;y;qjO<%3fyck8#9XRaxx%vmZ+@Ab?|=
z#><`B&i6N#{_;4;eD))|gko$V8S$1!x?J`)a}|?D3T*PUb6U@>Tc?$zbMe*_Uin%H
ziSgazbFRYvh!(u8{g<Qo+*G_(PNqCbte;ND<NvmpkY%4UnAcjx|1HDpsPEg3|67Di
zN8mIin?L5G7`wZ~b5w^KhxB(m;(tL#=r7Sw>KE)Kl|qf3v*LfLA+>|h^H<{U7v!3?
zKjech!z;#{4}oN<pUNXULxp2kEYKfRichB)6QJfp!})dSF?eZwI&v)UeA4|0UE$)%
zm*Y<sk~*Yeg!$71_CtJz3p)aAKPo{!nqfYh@Ifj?ik^tS>_g~`Bl`y7I<P{m<F6V@
zsQLGR{j5I$5r55v{R8ab2smc%ocKFzgf5@XaG1SQe15w}7g1a*mNE`md8%+!JX_QD
zig1Xzf({nY$bukAWD-F*Eyybguv<XS3!aq3E+XX!WhH?$SWt<Oh#~t^ve?FAus*ua
zUyHRtUSspnU<Jo&otL=PPo2#}gB7(jj5o~nQ<o1?b%k_g*TxKAp|3iXlMk}7!}ee9
z{wnxi-lpzk_qBDw`-)FKQ&1j#<b3wI*$AF<s4TXIk~ki$vXEnE7G>imdrGpe-0%;$
z>GjXz_X=;U_!S;tQudZ6zQFLw-kLdlBKwLipQMVm=@=$o8H$&EL_923;NZ*gYn_F>
zvQt>kUVK5m3`B3u1eBN>zk%js1`|JgH-4j@E#gg@#N&R^A946^yTGS}>UMYNIB<CM
zdx@vWCM`3$&CH}I%NK^k570e=|Eh9yK-mG>Le||6C@<ZIwQK*biXR9#&u)s)%|KC$
zQw;h8%SaqJXbnWV(Y=VE_`$kMdTdoOekj>>hHj;&E=Q{5iXVn0v&ynzv?t=<FQwEg
zlU1R<ojR!a4<xatg5rxGy;c11Xl1WFB}k`Uil2){Gb5O{JAS?qZ%VOdn*S4R4HNW-
zXvq+5jdtbEs@JuUiMBq<mOHDuJ`OT+C%c5mopC#(d9)!7Zeav5X{2CeNA5&h-E`}|
zciZ++(N=doMlE-ut)4oePy?+uMO%eK!8eI!zOAM|=?4U%t-pNPz8r}<G8}4FsuFD#
zb(;zQfP`ifa;k@u&{jzvpHMW~YG!EQ>q<PSh_p|!PG_U7T9QD9h-cMeeDpm$t2^Zt
zcB2TIwSidqdRFwkiqb!etpTAXvo^ka^gWSRR9rBYAREYUhDHC1N^IB!CzvH0*C<+)
zXhQ_Wg6)LOj*J=pqWCW#++h@jt}A=Z+okBh@(F%P6gxpU`nGhPrG9J-P!;+!+R7t`
zX4bw#-W>w33EFF$Dz%WoFZ4*_MIupzl9H%KgxOeFDW6nE$gJ&3inbPV3c_AHfJkRT
zM@f80q%WbjB=GSUjvx%feqKJ7{+Ef%QaDfac@BK99*<GGd;!^C5x$VbG$L~d$t?39
zS2-_O+9R~Y7!Kx0pC5TVDUU2y&lU+VOwJLhAA6a1+tiPvjZ43Je~x(IAOa7+C@7UP
zj3naHA}Z_G!n8;3Ub|?-(2rS046S)mN{!UL57n(mjqFofgN!uHC1MG(%i*{G(WLBX
z|5Wl(Y4vDj#=`-QX?@@iIqfCCFh%s=5N?gQPt(4J5~~RE*n8S%M79t%Ndj6d+(+0;
z2X}J;_+O&Bibn1Z^jL<tTq9@T<Yhh)y}?z~5PG|6B=#D8>@`@9q#HI8I^97KPn*XC
z=9NL^90~AvbLr3h-d(Hgbx%oS<{T0eE)uLxpXfu#6}{BfF40mq`Vd^?Xy@odaI7PB
zqYr_hBXy$>)s{H2r9wvZVU*h1CB%!Kh~}nU^d<2^t)q|1x=hlaABsMTxOYERsebOR
z@y^sb;A<pz{tMBwH5P)Sh|cKQT-D#Bb*PBW==q30M{-Atiqu~_s(B)MvD+`pOdilQ
zyhp$5_Czm}GE;a!8NFPWHgY?9i6ggP=9?TVq6cXKlcgpzT|Av;p{J}UqpJ@>ipm3>
zg(nHefFW=OX)=)hg>VZPT@HTMtn5X_m0Faa{-bwB50wrP2#Z*Zi)wz)n5z8~JxpU^
zQR4Nj(eE=D)RkJ)ozPiZ)6wJHTCBuHAJA2f9Woq-)AsNewS?jL#lnf6jB%Zk#5xz7
zSlr@8vI8YHI0ZaTu}5rTploL7%8$4^Chge7i)PWv3|bdvnhdAO1urRYz)Z5k#!;DF
z@|KSJB1&>7@<n;^&nPlb^XaWNQtt~6a?I=i@JnIB;4d5w9j<rOzWDNyXZf{<2`h>n
zkfhXzBZ*f<Wu!a^6;Ydbr?4;}##72^W;C{GA_uA*%QxlrV-Pd7ZoU|&jq@h*0;#gU
zt1MiTmdN+R&_1T!<GNiM7~efHUDF&UNj$HoN9w*gD^Z&w-}TcAA#raYdm$w5O#oM8
zrY7#qF;f%w6;!JJRxwk@Hb`8enK(<`F6o-78yadeHE{{WG-IrW%<}4J@X|#2R=Jp|
zi>?yRz#fw?CoaPp+0YOUKxr=a3hal(6)vn6uxgL9hwxp|6NxE$*^%9n&;nSY)``Z*
zA&r-@xUDu*6E~T$s>SaC>u$`{#2rw2eQiTmR*Yk&ZfMLnCW5spl$IzjJ*<~)h7IGX
zIF!InkiB6f5sECHAc_7&z9GnQ&JA6NpeGj3lf>IZz9p<g_T7dH{<rL7WU$ZeIz~~f
z*Sa9MmWH)5LT~*Knx$5`ER|nprW>)pp^(mO>|{10(piq^BKeX3a$XlslMe0}WwIgK
zv0&lo+YcQh6|VblMr!@7a7L+>i8F+&HcW%7n35T#i8nM{G9$-GO`K5=dO2J&Q>NS)
zsfja1rf4HIesW?H%_Yj-_=<?S#3rth+N4S1YkI1gik-6(UzfQu#%N+o9S2@wYT_F?
z;xO?|J)gkgXz}ydNQR@|aiP7JC%&WQ93`GTnb=y(9%g%EJOI{J3p3lYiuP|x>@7t*
zN}|ui#P?Y@+9qQ36aG$oQAWRU91YNue3X~X9{KKj;)^A8T)wTC_+lA}Q|6=Ni7!?l
zo+PXL!|4g>x^{+Mg$bGN>fR{lKVq9E-~&0Ud!wBH$bp#E?dHo0Exa~ihqSdRzYrlU
z0oHX_cf!t`)t#`*rA0!TriIikzbS#^=G1Mf=Pe;6kiEJSQgT*zLQ2l+PFOFiyQi$~
zkE{eajVRxJzQx)!PUx0sy}A>);7t>2@%M1Kd{jAjX~KV;)xG!@u#2*iFDHD%^lkyZ
z1RHTlgus4CnBl@u8B1zCzPig&r09u+h0f|;g7LSc6|h3B6Be6ssU=;2b<(Rl0S%|+
z=@PhoNguPi6IL<bn@btGa(=;C-CHU!j?olXg*GJ2VRdi*l9jrp2Nj1CxZ%X!(w4}l
zgfWtMjmTF7j6QqIOGJ_hGbI7JmMkSKW_8~h^M9}ITh7U~&Ds$(tNRv?>C9Am->nz$
zEO&L6mDPG{vsta1^-;I~j}v-}gDT>LZZYTIvc*6x(O9?G57SQQZQQ2+;)K@k32$s`
zlfaQcS{MRfF)8C%-&>*$pNz{{-wEUDK{2zw6UOJP?}YK$>pShk1U?cf(_nAyBkMb%
zO3wODsIDiQ$=Eq7LGPiN^_`&4ImiQ{5>pdu32PfiH9;RPNCG~-dqN$V-IUPwJ=&^-
zx;hSgdCi1+ln=`y{_M$whQzg5*7>W1MlQTuX2Q!ZyngkBS9#2|wNGe!KDeD=JTQk-
zK7muVHl!g|Ae(-|DOxgYOc<yo13A_DCuqY-EWc`_5(aCy*`Q`mZ0m<CK^0d(Rl!Q?
zKDD(=hnYf$sjan1ClX=P)v0<nY+C`EIi7oJF?_bzzQmIbPA!oKbMPARv7b(T${ppJ
z4O2@=oR1Xu?5n1hWj*9+$6^pWDX6rzFsD|~1*kZE<J3xsg}id!tasYfYG&~?LcLe@
z=ZavGw<U<*8h6;A>q(}N(^ulR#ufB?9tsO=8>Jm#f36T$PHk&L@z|#RT;6O!wXN^t
zAma#UmyiZWSMz&r>jsx;50+Fw1WrWTKAjYaiZTZECdb!b_Is{j+hqtbABrWq<H>Sj
z2LZ+#7N2v;X2RNXrsg#yD>c=U^Mo_Fi94R8a(ODN0#EuqKjdan?qCL%JS50ts2$Nt
zEiFrsn<)xB$ztD8msCSSU0x!zcQjQ3YfZWnp`9eU6ZwEJND_mIOeK8DqRgnk8OaXH
z-p*-Q925QAHKUQzx8rU9L{_oce$vW2qm`49oRd*UmUhNG<+=S5t?bFks31C7f6QRV
zHT@!!%HUwJ+`6~@LQbjrx;ke(bp}3=+hHSd-7@Y-yE(x4$J>NN%uH;w-?K?(X2%p~
zW;T@+I>SI|aKR=>r8+Xdum9oqY|#scM5TDYXRBs05{I|?J*kqAEr`RrNJL1&*QHNy
zX3gwW<<XlQe$NiSWa*BFH}iX-gtEVX+UYo1yIQnVu?<-Wf0Yzr8c7CGC1OGPn#}@H
z^8Iq@YxdJlLQtA5bwrbW*TV0aD_A6nZ?!HJ#8&L~d*-p#6cwNlOU3cKhPM~4`;1$E
zo56L*rehh67OqQui-E_a0Y<$GgDqV}SX9I`^qOu6z!}0ZIvf}2_p|`7$om{W$VI8|
zyZSxG9#Lu;WLx?W;E`ATp0_j&0pc?*s|p_Z=h9mZaw~rB_q-$8v<W$b2;Bibd(!V|
zEkr|7EW>PEwj5x=!hX-YvOqclI7HY{s(oi@-OYh5FM%v9SgLP3VDGuGdccxO^?eUm
zPZ!n|SZC0mkmC0Yr<V%vuvJ3`NAvnUBZ><ZC@_@J7hq(n-}9xQG6g^~VWwdJr?PiX
zDa99!{hm=maZp;mgRrfXeBs;g8H)%L<d+{L?5E9)g;^@Sg7|Oz!f6yi5A5Xac^ej2
z%TEzbz`f}lf<BpFxHq_#US0gk3r)`}`}8Xw@B7r{;dkm{O6mun1%mvZ2|#v#L`$iu
zwDKF_q9iskYb@v$O=QUjv4~SokeW-FC5g*Kq{uq9I#Ol7BlRKReQ?-{(}d$3?$}q*
zGiiG+fkp!Pm2giI3}7XAtoVmjoZeX32RJalOV+(`fgo)@<`#TM*6E2|)YwFUIgb?(
zwR{q5;JYU%b8?vS9F#`VbeZi=BaC_fc{^Ae0w+1waUd5(E(>k+$GFtVRYvFUE-5hi
z5Qp9jX^a%<Y>!Z~?%4)1ra%Ac$ZIX~cFS<>2%Owy4uJ2LEg)Gijmrz}*iC{OZf(s1
zg4%s$KmRxMebaFlfSm%5idTRQk$&AX;?gPNlFFy@GcH7{Oeenb+Rvp}KU5L(w67+>
zV0j*${}9|(TG@N!nXG;IM=PHsgh-+>kpYCR%v}CMLULGk<x;{zNem>ijj%})qlp|N
z9Fh5&<j&VjAlC?2Y6D%$EU>~<EoJX%D|MEUlSTLuWKTpnBRxU3jC4;OXQZHig4s>n
z_g_XSGn<~@O#Pk@Deyn0ezq>qkG`1tJ@`$J6wq1g&5ZBauCxBzjHkmmGUI#j9NZaC
zzi(!IZ$W|O%y_U6Uv9waB=ep#<NjfMz~}>2Df<OdbpqOFzxtFt<9ceAus?DfZ3WG5
z`aSJ2k@j-T!(M_YT=^H_PpEZ#gnqKh=#N)n#k8SJUqJ<~5g2<Pa#8MH`4Eccz%IfP
zk^}qd(t7{XDLvIGjUxIVrBRJ*Kc^kJ2f%Fsis`m*3M}|F;Z?)P$6;j2N#$7|IZGsG
zr0JC-Pts1R_Kls=84e8pSGGwtmqO+fqB3O`HRO0%$B1|5Rk)m&5wI!P#f!X;QPS<a
z-%|}@>3zf0#mJ2oejsQjtH~3bjjrwYa7Bkvqax31=tPXM*vx0?hc}YF{uaIb9vh9A
z`W8R?#3UBIVn+&ugh**SPM%KsjBL#fQ19D|y!PU;r@lphiyHVn`6cH(iAJsco}zBz
zEjlmeWmbP8r_KYvr=(X|=hI2_M$?t@DRwJK6rxT*H!Voj@9VcuXCozhCLm>>E}esv
zeL6k|DLYfwq#2O1Iiz8b`r@+R{=p>zUE4Pv>Ds=jyB52VzU_OWWRjG6i8g$GJ5_g{
zrMle!JWiDJ{%vIq7Cic*d8Jk{6{~^(LLxggR<Hq3i4bG7nY~{l0zgVPb7&t_TBP0Z
z>cx>#9kW#T_M(;e<<D;)=P>L3-t1O?hm=_*p?{?EY$&Kpu&rpyS+znbvY`ywV+o&V
zZtI>)={^4aA89_UB7EPSPQ<y}KUcWRYaeuE^ziD!ciq{k{NEcv@K^^tu($p8CfMug
zaS-A_F32nN+r+@0mc7!kpOT0IWig3l{F@AmU^3grMpMmUwGaLFGR>nL2apl!mvauh
z(Z$qRR#a$ab*0gL1N`<%ni4g?>Knp;03Pn)w^!-u1F(XyKxQY7$(rgkIvD+~rpxRc
zY{*2&<WI#~)w6S)3WuE8nVRU#&V7JC0&MUmC7ErR@o#Zez76l+9uLWHPcO&_kD%fD
zx9icK^xWc{hk3%DR1#_q{zK>mIpp16$H*3C2VwOL;JNDjgLx!B=OrS2f1Vmr#mSA+
zxh0tjeR9&$b1|fmveP;~UXNb&TpY~w+YQ~NekdeDBg%N$o`-o$O!eDMp;8$iiNkmO
zb~B-)JXrfKkKcaXT??3^_8Y~-6eS0%&wew)5fv4`_S<jigwKrmZ9QhQ-0hCKwN~;p
z-EWVjF2cm_rGYrB2mb_uV`je|WxQ2ihdzBEn{ih)_DSRXcKK}V>lp0IKkZ;IIL2!n
z9S6?lPIB>kWvXXR_gjBCQ$1)i=tjs?r-hai3Rr)Yk~Ww-IQH8LOzNCA00YjXz6<cp
zQr+KeOUb_l-SYs(5JqpL#o8VH*1rNx8tzOqR{`jW4mKvFT~g;#ej#B#vp<LjTr&HI
zetLYC%=``h(w#r^fK2vE_(mA>f;7t$>bLH?BuHzkNnqWTakh@oB)9IO{nOf-IkSG&
z^f2_d?rA(6;#faB#6k4bd*UU0q85JZm`fosuvZC%4j(fVI;Oh-{}jS1heF57I}|zv
z@Mt;H|FM^)e*k|Feqrq~FIKuEzE)!gt<D1w9<okq9s*E;P#hq<WS!FN10X;sBfziy
zR)!0#3^2iv)jH>pb#*g<FA3`Km30mBj;q6KZf!xnIrU*F3*${Q!fN#O>OLaQ2C-Kq
zs`hXXrogHun%3y@L9LX1Jc<nM{+i>$ef63g)Lq%fj*z2oK^bIVqm_MRA+XIdlaI`o
zUdRZ>LF|yP6=RsM@R>ej7C-m;`TU;{50Q9ej+w<yyg1XNHG>l7>x4u$9^+jS6(zxx
zcIoWTdCo3$?e)l-wEDPfvQnJi>g%q_1M~dW03lV1Pj;L9*3cr-C#|Ke_gll<M8A=K
z>kD{Hb!06b`N*tAtCu01@JyeHeru#?wiI8Fwu5NrL!HoOM7>onu|4sz-^%YW(17hM
zu|~o`$9ftD%I`4H>W`R$_Z<ex?=aBnl>iq4MCe%gtI`R=boFV%QD#sE$_K!;EMEOL
zz`p<r0~K=^Xbr<#69iBgsJO!`YY@L{@(K_JdeQ}=IoAkR$!JOlRp@9<BSL+EUq9)$
zo<fz2zq|$(Uh_6Ugp}p3;5FEQYx=yFrDS~k*+)|-Askl3;jlGpDY07ogN*mH)WG`k
zyxm8BB8Pa7p=#}nX%wLLnyr$UK;#hN04zDMk+J8F{w2JG0Ra09Y)&D)or(fi3wSwx
z%^aYMNR;$jaXdLZ))f7{W<PCY5>85D3#X?U@7mysB>1Sk8n}#>ajMLfa{@qJuX)Y@
zge}ceWEOyI_HF3vzp1BD*d305{5M$;sc(hp1%8uP1Jc?C=^41KCxAGFzoSVk@vr@N
zbzE=d`G3(=Hd5cpExYfpEcCla!1^O9%@N4sQE)uXq@&4Y2Vw0iga%J(`k!<rYAy4!
zwk`PgFArHFz+YZ+RMKYqw*I9K@AYp<3!R1cP6W{==r6^i5=FkY7omHs9>!8FCP{xv
z7)?Jg)N+TP)-D7#9~c`RE1-aE3anj6_!byAuGGULol4kaa9r);coNt#U~F=%bL-v#
zb_3Xk`u^`67|p4*51ID<<7A2vI{F|$e=Sz}i;IA{pZ<h1X;SWpND^Qi*pgdpM2Zc-
zOwh1cGm9FA&#GZB2+WA70VN$+i%05~cVJxwHo&xpVWI1OATT2k2h?$3lLa=|lyjtK
z`T~JDeYA4QZ4{V;UuOq~1kCQEp991H&+cP{1LIW?-r5Kf)*mp@nPbGNKHYGzjeo43
zVr1rtmmniEJGQvYEKJAjjeCR{$GgnP>YZRH{yN8@&-t5UArI*cgXqJ?84-p-n)8OW
zFbI#y)w;Y)6wS48m|<N_VAX-a46nPB_bRY@z+i@!4l}Il4XlU2zV^%EWntoV;|RiB
zpwZ4@kag>Ut(J$&{p$KhWT~N5VvHx1`--;a`%ygo6aO%`tD_V@jK~?n`c^2XQz81C
zfxok#bPAd02xpyx1w#vq3&1Z=t^B<a4<~uMKp!kvcY|WramD0d0R`{?bg@Sc7Ldkq
zcA(jZZB5!aSg`IMt^7>u!f8XUno|QOm&;0&9W4RgH&c0XudwbQY!`Rphdh)C1*Nl+
zV;FsCD5YWaliCu337Pnzh%M*C2Ysk*h$3fo<QjT4WV4KrUm0N=_CQY6VubSLuyy|s
z9@4;&?Q&3?PAnb-QcZU3h7Qw&8Ai3j{tpnwr#StjH|$byC{u!^-!baHY}oaGbi+=@
zCmub^@D!}y)v77OGv>jNW%PQKj17_@7YC^!vW%tdF#%~K!zqg+C2RiFbuxq$L+IXF
zKd1hb!c{aS^zxA+!O0FCMb=Gh>Tl%8x_(C(kk+#q@59QlW1~`r^-^Hplzbejk<Ac_
zi5{xue+7bNbYtYQ7m`wqP=&pa{<-vC2pd}c*$LA6#t7DEE<I@De?fO9<XQ+*(#H!W
zr4NaJ?SBE=1XibHWRm|mU6S>gA%`Q`MR=~D&IRe*9sLzuq*sA-NfFSei-0u6Dmp?x
z1x=jtB6##WsNA_2!N|3DBPczevw2F7!DIcAE^epg$bP54{E=ZaE@zg9zUPlD<izPB
zGT`9c>3}~n5x6;;=`StIGUf3lI6|J3v1ED6M9cmcl-l~AO7<xRc6t`3P^*iT>~nO1
zULfnGKqc+tpWgY7yO4by`}9?n?8Bx^b-;gCa@<#%xFq%->C$5FiN<Yrs?usb4xVYs
zv+%K>R$2|=^z;8pYqi|S6WUHyN@1h9HlinqM~H(faUh5CVy>M@1mC#BmA=l+81;<}
zXX*k;|D@6^GXg2|x`E6$5E&sgmKV@91c_ypD!m61oMGX?DoDiVtMpzTkVqoFW<#a-
z5KH>3*7S{)`co_K{5Uh(CrV0P@zK+wL#<NI9G_{dlxLUB@r7^8%40mTp4P3al<thI
zeYw60P$f2Vu92KVm_WHgtpP$W8Dh$I!g_!^X927W5H*}~op3oq4)p@06V*|>+o9Go
zQ0(a=o=K^SnT|;dwF0bw=pHyu+FuBD&%CSepg_t43N2b4YQ=a3L(im<7J!wz00aQ+
z@Jza>`uFKDfKFnO4?QM(CQY_V^=jV_KqWmPsGp{USFrz9Zwa+3yTHx>+tb#2gF~%X
zO7Y@g3(uraElTwUrNJ1Nasd6ij|rm@>SsxCaM#X!*(s3}?v$8BiO-QCXW0{!$Kjck
zwH&~TGh5@S;uPxhkb=DIlnT!LuP?eBadtM{-SO3Dy+unadOXMI)j`_(6C_I(4z)U$
z;WWiCds2DHl%17+WR3?Ne`u)HO*kM0ROS$-$(PcvF41l9I)`qlaj4ZrPP|{;LxIZ#
zd0=sM6A`q;cegRTt{(GD`dOsmId+i5>&?_c4|%LbC&RF63BKoADo?MD3{s|$KTPOr
zBeT?*^&+|oBrt#B%0Abb=C93AYJGl!*wfc06RAp&H;oG<(BZYk42J&BKso8}96PM;
z_X=c>#dLS(kH@<!4yISLRAzO>(D=m^4z*t8YPoaR)HXb-*0&@ym*M?Nq&K0P^cHG*
z%dGG8wkx|gy|3eRm)YyF?gYJ6PK95cF}rfk?D3q%jGM|{^Z5cgn<<7|ldyR^TUCXc
zXQAfT_sK8k<ZB={ovzeC1K|g$#KjSvxe8q>p;~pN)=s_#V&LgY7=7;N&?c3YI$DXH
zZRFI!{?j+aN;}wVbFR?2C^r4Xg##DPoq3ua<+@tUjoFtga)4JJ+@OG|p9!@u{Yw2G
zG#<>i04tbAFYwUFG<sDUH5xN>k&{`<$!wy>*5A~;<7A?ab<HLv4LZ^+=8^t%_l35)
zFHJ~6xz76RA)bR5WQ|>zMv<I{5lyO7;{UtG&Iil$BdL_L#!#%>yMqiV49~;`%Zu|m
zeN?9-WJmP9)KDvbZnDMwzbG1L`@2BOiF_ns*Cqi=>mAXLyrEVBmT1f~dKV|uDntpD
z<TDbmuq0AcGmc8O4Lz@$%u=U=5$g3t_FnpqnzVhstP#EIzl3YY0i)IPwWLwfTL~%p
z%u(n?-8ShP1UcpeDpyPwY6GHRgCAJ2vY}U9SZQF9oKg$D8X<^ulOw*N*NQ7u_nbYc
zWH<C&=#7F%G}#8F>b%{dKb91<s8)dx>*(Em$F-F=Zjsfqzlhi{WYL*iv{;A2uRCK6
z@dG%I4{TrB>|{rUS|OZdzw}~7e&q4mrPH)po4~a{_N6OCdJsCP4cLrlYT=a#Umzca
zF;2Hp`qsxw{bd4WX=U%CN-oym@-;LvR&os!o=!G<;FrYqvx9%9hQV9p*J+GBEx+_u
z9b){&2AC)@4MLBL#62%)^9qgXmM`=;D&|s_Q0;`KxAv1mwUet~5`Qs{`5bvSbfTk6
z5A0$-m&trytq?0Do#<>=ZdlFWmO$6NjY9Q1=1HW_5YmAiZWQ_-XF50h4eVF#HiS-d
zrfy>-ua!Q<Mj0D3_SF(*{$~q`X=Gy+LdEjz=_H3PV2;b(Vo=zU*Xm`dgBc=_^EV7#
zh&;ahy9md2lx{$%FA1dN#@7hV@<}O?kUi~5a>)4LMI-PoUlZzS`EPcuwfH|2>XHvy
z>o<z8m7vZ%{S>ct=xnC>%InfTX2-@h1o^UH{?^idHIS3`mznmj)Rp$N=kZAU2M<2h
zzT+<)gu$o@8B}oSbU%Hh(uW+x-q?-MCDQmy6p#+2*dzWDX<klrpxOSCNjv`1MtE&w
zKdR~aSok6}ugD1)VVqax!Z&ruk$YFf@Yh>0`r-n~aVlK(OAanIY<8%<I9-w!HU@AI
zjimt|Z@w`~M8VqTp_~L_B6(E8NK{$PIxSlZ>x?NWMkkvlP1pl_Dzt|#2LmjTQ(d7w
zG-JV~C8mb<fP3?_Bi?UXXpd*5Ic(4#?Nw;s5?s&w4Lv7ivViIdDtd&JPx(8vg>DgU
zzj{G8*+L(fqZv%H7TiB_^@3@oCDvK_eDVyk{4i>MJ1n%GTk~2GU9+CHk4?>bFEY<X
zOwE_*zg&K)S*`^&cbyM4XY}avT0LFU9MUtTt4Zk(#6L;v7g{#Ei($H8S$Di3SM~+!
z4>Og@nM%^dEov5iHnebd%@kd;aHP=DAXm61HB(H@qS-a0dqCoOePc6pO=Vw~V|xRJ
z#b&VZDjp8PL3tQ^9{F2wRA`4@D)t9Pn?4tCgEwPEWG0Lv{_M%vn{NE6sj;`*c(n$x
zx7~Q}wAeeW={&nz$p(0=RqP3+MqiD!^5Fc`?WdJStd-A5f|oJ*md0+^7-gxO#hKMV
zggI`M(MMlnw<F`l+}rpWVK{msn9|@V2irFyg*I+Pux#_je$Q^~4YD`>Mz{lP`o`E@
z4s6rYoCz!*<2nagKqgb9Q0rLd_^#T7<GHDd7CEtp4Q6UnU0}6wRDMtlJho}<&g|o@
zH+X~irWOQ+=Ecf~hkjVcBE2z^3QY;GNCHD(Q+vX@lITLDC!vQV*csR~gfK`FOv$E?
z2%{t+Jx(OZf%hBe+fCS&oBqQ)L5Z)>;WF7U;FrHxHkj=nI12Ace1{?{3Cm^a-NY#Q
z-%ma3>&1Q(dr;n!`avJM2-YXvZlZDI8R`$(0uJ8e>W^d*12?ZKyV@Br!P0ZgFztlX
z&n*=3BZpmuq|JDo&<XM89EX?HP`6(qKK|lZc4KHocD<OD<)w(1F`Ty?VNg+gQdn#S
zGm<chI4rh;vFkW2w$g>eVk<RVzu6eO3NywS8Yv@U%~{YaHLDhgg$e`RJ{ybth;6N(
z#X+7ug!MGF=1gpRhc7ng0`^b0{#{gTCp2PRd89>&!*^r#xz!}4Hp4laiyOA+>9ECS
zW_WXz5pXQLy-0S=rtOZMo28PwBaFjvac(|KBd|+*S&&TpW_04_Wo-L+;?IKNE}2Ds
zS;VnDQCK$@GK+HC|K@{){gUtyIY&4}BK~(Ge-M6?#BHUvu$s2m%)zV-W`Ab3jesN3
zi2-Q(2ba`ul1YCY0i-`U2&jt34#Io5RmACUOA#6><n*@%Q*TR@)87{Oa7%5szvd(w
z(_eCRr$0M*gqWe`1qioCD8(y%y2sI?A;T`lj$(AT`nkP9i7g%6-o}s^?e_K!31MYD
zx+NqQNN?<F>Cw?!PScxp(|2M&lcrIOns8gTQAgf_6T)pdPdMW?{WFQ5+@}8}@fVv*
z$!LkhneK$l&MySwR|qr}0^P3aw7V63zLmEkb-P=k;MNjOyIbLntu>r>xAI(IYXeZ8
zT|Nh8vE(0{5$MB`n$_J{1h-qtlFE6CZU@Wv9Mr6-sH2vkAa6ZsXU%n%YRQJ&tnJG4
zK|8YXZOnrPMl*};%{f=7oLtgrdtG+hcJRM4Q#4M&*-8ouo)NB)&<Nxd5*7Sy$Cy=2
zA2yeL=RLhjV^-zphnN&(Z$(iAyWfr3;R^gYonv+Y*9s!Kam@Fw;z;{BW|xke)e<AG
zTkRVgb4afw{Tg(P5x~H!H;YM=jWTq)9L0$_;uwA?yQF$C6EsRlkvmtwiM}T=9a=)r
z_k<cwe88}n&&!BJ(Gnu&bEJ>6K@>BYHGk)dqXUwfX&tbPZVR=JNjB){UT5pL$gUSY
zrssVlV?J<uzFUr-gPX*BP)d3O$=)i46P>D($BgP@xJ3;x$bVv*X@<{IKgMiUW~mu4
zy|rFsQ`RbxO))PxvT18K7^pQuW5T_dmo%QFrxT`jYxnLz74tHZ?uArWIK{kb8dqDt
zrO0xs9heu>RA8vdt;m(F-)n&tBL|$sbl!TBkS2m^@0^%{S!(9EZteQgY-gIS9na@p
z>abduEBo$Gv=!Sl<|QPC$gQnxDs25(t`x{xMJ;_KG;Bl6TP(dF*P@`ZuAzCM$+jmX
zaheGHur0474iQ11Y%42?9YiV<<Sv=84JgYjR^_(m2}xW-Boc$R5o)wuLyrE4EZo+J
z(7;)G+ukJ6!dZIT-Xqc7S$f-0quU0-(%g>tf5<}XRj43)?s^cN`LAYn22wnx3v2zS
zVeUZC!P`U!>w$bl;&XQ(t4OSH2eO5PoQ>3D*iT|lq%(%jDk!_@oN*w=x1DvzahJpo
z&N#M1x$RI+OTFz_M%(krI7a+85mH8kx&@bKPsSn{R9I@vyG%w_WoHcAA^CPqawcOt
z2J-eqXAIjLkf`sBVLQIicJYP6<Zfjd$o6*$@3;e5B$8;R^pHPiLbi{j$Z%&M+b59t
z)E&rd60_WatRS&O27>R2_!eUTLx?}xoUzsjsI2DM0?N2Uuj<@gM)qatLyzX<>m&{r
zcIFHUvTw#6Bm1((xUx^MWVKwaz#yhei+#2CG4o3ibEIXB*Ui!o>KaXqsceCS>~3WJ
z1Law1m9azaTnTI6rH_iKjO(wG^c!a}m7jCG43<@8mWIAyjdfoI?oJq`zM`FS2ZO8~
zpruO853*+JhL!zuMb=w`Hvur2`k+MHlO&K+d3LJ<6gvESkOgC@Bn<NBcLrG>g3Ql@
zVEo`FLDt8*bve}#lNMxshB*?PS6j+KmV6mf)-lzp4-2yTNwpOEl7jsh#Lp?dK`O`^
z#IV#T-(ZOrkSJ{>{TXBp7C6_Y(dniR9P}6!WWB9Xh8=#s;o$tX)R5N!1n0NWRp2Zq
z<i>v-WTF36l84@7X!?12|0Tvjy6$60Hvs9F!4A?5bC8}~KL`VoYU1b%(iSfAFVlQI
zIqw%_Jpr{|XhVZr-U+h2lwc>R3vi!~VtxPesjpiH8I_;*RY$H@2(ltf>v^1<vN}hf
z`*{9ikZ*Z9=!SHd#?W^Mf^IN&b7{<(pc}I1)sdZ!pj)~FMe(RILFXub&ljdUIv0^s
z{G2XsnmJci;0XICS_R3=931DPa+|wBJEU@T3X!oxbl4Q)<Ie=`z@gOrf`hyy!z`d+
z`#cx)iNiAYdQ)(~u%J(J6l~D=XB-6^xE}O{R-WQ1?#UO_Syz&-<e3+>I+u2gXG(TS
z6kkm%pCEm;N0QX4ygI06b}e~4M6H^-FXGfniqEO#XccOG{6f$(Zms)yDG7f*!%z}N
z$(2Mw&(N)Y?Y_4PDy#LMboF7Ypd#6o>YGYM&?i(H(6+k@DiRR7XssDk%+zB2?CIsR
z<q?lq8~1;V=%q-Ol1h*_CXECVEl)emZ9cm%&zkQ4=<C;>QvKk(=c|%2Ce&y5m&uS<
zWkU1#Z1Xm}PhRBm+3!n^98+5Ng3tcI=QNgZ-)DcQ8-Ts;%bDKTrZitdS;tGMJZ%*(
z$@}c#(1~4ASvR$Avjlc5^V#OnUY7d2GhT+)<bTdDCrC$Xq!E3`2?xrFOaWL{4@|)x
zYWuJHEE>S7TyMRW=4uiuQbL9q1()S%2Vel9D>!$H0SI#`xkn_piO(*p$4)Lff5+kg
zJ1Cocb_{#Hf6m7)DAljMj5bz}i2_#$X8{VVz7R^O?`iG(2wD>wB?wuj0O(5S2yj;!
zpPeYc6aZrh!vS{b?6c*>v<7Yf_$|O*VE~1%rQ}sYMvU>RTMW=P)sa$i8H&$-*QvOZ
z8Qd9FstX|ZxWUf4fNCRI{YdZMI?$+)zpNp@+W9)Uud>am{E1}CalMg{zOExrcD_&c
zV1V)0efEnka4Ntl4WUL#mYR4&P7?5oj`!Iw%S#V`qJ?+PCCrw@5hAMy%OtUj$oGWp
zsKP&Q5jjdYAc^0J2!MtJnfl89<C1=E=d(Mw{r*Y*y)4$`gWMG9Ku$jPeIHbipD6h$
zgqrntpt%QV?_JPsS0SDvJ(!oJCf%1!OaJ>mJ1?&M@9jb60gmD>G|{eDRh(Tu_82C0
z7vF?mxWH#y><0dRl7g}uxa&DW)kw1&ND0QQ%RRCiNSY_@4m5i=(4?K+z+ErW3d7T%
z!g%lorH2u3$yj#_(imR55LvtU9FF~<6OmSgHzm=Jhye81MiS{m=;e-WtsdKFK06qc
z|B&MldTh+yuA!<nsR{ISP`V|%wY2v+Sz`zvizzT^oSe5H>wVGnUOTtDEwNqa@|a3C
zE&tf$DEZn^<ZE}-y|snYV`dwIfASSA{?!pX1TSb32M_)&FMB1EvZNiijaJeIMfhVB
z!bqPB3n_n;&#uM{{VAgmM%wiiVUi?Zq+P7XT{!TXk#?ORaa3kJiXjO%?Yc|2CJDG{
zH(axuWx;1nDpaQXTAD~-vsSnY<Ig@=P!*a|0-?E^tGGNBn3~;4*4+Ua^yiN4^rwF2
z_2~U8)(9EaL>>;zOqKER?3Rl&u<cPdjH3;aIBTn_s?_RNeRdJi#1C>uXyH%<q40cF
zP?Duygq5nF@5st3QjtWGBSqx}haF$)TaJ8oG1eZ7A0<%kXP@-|*`}?k2eW7fNo74K
zsYSW8Z#{Tc+K<o@!FqrRqowq-Cw<mKEgoXD=ukHfYtgYikFQ;S5uA1&De3Fivl|JI
zP9UR2YUgqjbw2djA@uZMmN`63ie25~&rneW_IO4+v9QnjS&w$RgWAuLf|@+;TEE?A
zeGlphqoBrLL&+a73`iMKs3cw+NTQabhXU7q*3L?zC_&=mt3GRgFulkNoFHKxLWVt%
z6Byqf@L5L@zzL&B2qDgDk!^-(IUeU9<n_Davu+ju$?IsJzghu7{8@)MY76=@8mXL)
z*LD9#XKYqFjv}f_AGglc$tRYhLH*%q2lWq`JdL_wJF1F<tswZ3i(rW=KFj!}^nn(z
z<W&de7@h93Cg?aBeJ1*>&)tMzGSSt#Qfv|f)M^*4Wlh%l)EQt~<5FKlK<u64vo>PG
z`7GfHdg$1l_!30g!_f@+;j7M&KQcoWP+P6lt=UDjvJyx?3Oyh2S@Lpbt?QaD-fw)?
zYSxh0!c20NZb+vQH+@z+L}Xqxw7Ma__1_)K8WOhSO%`Tb?M7Jc7ELQt+>S)W!vH!G
z-T^4|YM&tV0x*#<h7M=V^I7llUzgkna5V{;ddHSlyII@2FN@r4LyvM$xBD8O)k{wr
zRt;3z6Y;a4wV5J{Y?<O-><7v$?x{^c9Uy}8_G6vP;WJO)Bl<z0d1%|8N^-EU#?wBl
zMI)t>gMB<brKfHB-Ep7Q2!lc2VDtBMhnO$tb}za3d*$OGmx-JbA_#UT`79o%EBl{3
z`bA+<@K3}gDt=SbXO$}i%`-`aboN;>GKXSZ2{XlMW^$(7yk|9{yh`NG$&o&u0MX!V
z76;6c0SWT73(~dD=t`l&1su452>>&-=k*#O&Vsn0<4(KAk{D6kVWpD#F^2D^#zS9T
z=ew=rn#C;5VxT9CYaT=kKJP5^Z5Dj~m5pXF-y9{&l+G%&^ljIj=~w-IJ9Hd|`{y4r
z5ivWNtXkKJ!~1*8A|%1|?SY;CEyJitvewgJRlVOuxnqx(JLaxcVbR>LgIIgt7q9~F
z^I`_Vp&n}q&4AMvv?yP8-_wQAI!c~o`M%L3BmD(oQdHl5th#T+rAQ&2Oqi~pnfkbC
z!@$-8TjdbWynHnxl%3|)#p&@&Zl2P=Dwl628wLMjJ9t*dQez(h1@Bpn5wYho;iBx2
zh{e%&o>(0GLIK}Azw)ShXg4u9_S_}hMuf_7O$uOe=wgo;9HcQg9B8(|VbYGlvF8t3
z`Hj|P`&DC!7NriqV40kc4%w2?SnkE9+ROQDjpbfAes4itDV5?^btQhOw3H(1Td8z~
zGLeK*k~m3(vF<IO-BXsX#4nXrgj4Cd+Y@Bi`>fj&qGWGVmmbU-^x#+M@h$YQ<dLt?
zV{dE1+YUV_06lcEN9aLXjyg)^NAATkX@?$ryV6Q$DIxS=X#JuDOZf%M^^~&4xkKwu
z(Z24`J|r<hEQ7wIwZivU$;c5qxIJHwQhigk0fqI@^?hG|B#G!~kM)OH4E;+B<KYAQ
z1_>J_g?8@$uJVmyDJ1+O>^-xNZwNfM<4fYnr+q{H6oUWY<xPWp9Uw=ZJ3!<-yT|^$
zwXWRZ87{6LVRePGE7tXOh%+_zP6EmCPib!Nq@@7q1%!FP=%uq)dLg}*unHKxb#XX*
zuT178PF?8EAg?`v0xU#%DMb|h-X95f<e`JG5v9OZHTQMNOVQo##6G0VANeH1ncY-q
zk{aM>sG)XVO>OyA<$dfJ0j1y6%7lszrBk0Lk)-69>->H`^QszGCzqv)+@|5lH8NI{
zpkXN?0cGDIylDv05V0zxOC3Py1O6=gmmNZ+jsx~7FbMIALx@xynbax3AcS_vrLKe(
z5JGneAv|AG07AU%5F+(71x~qypp+1zyYE#9;Yo7{k$R0XSA-CJU{f!s)W3oJrJt+_
z+vUg8Aa{_j8Tp>y9YX8_uYFHcq{d0vdX&@7`O&cy9PJZ|<|@Zxkdw~)uq5}DM5E=O
z_t;5iJvRc=Ik>I97GYF-U7F;}@EVhStkQku-5JS$L-=2QnUPj_Z+b@dp|$osYY>r{
z7~j-aL9=z1Dj<_5A1UzUzF=me0?5cr>}x=H!OTPj=3QoDAB$*TZ!;4K&P?q44A@vR
z6N%1D?E4znS7s(;V^7yz18g}np}S-za+fkQk>Je4z5^84=gtJBWF{v1Dl!wfTRJnb
z?-XTD$V^Oh$Nv)$gsUR}3XB#4$S(w_9zxZRTp~Q6;2-X6XlYR(RZrV=UmY6D%`Ua3
z)_x@Cevc{00SB430)OafTjZ+?lj|wlA3>oormUcHrpyjbpi@L!o<Ux_peC>KJh7Z@
z8Io|fvn@lSoM?BHZg3N?UA&MK!hq{bu_r~J4j?DtORrtRO;jJ~wV!emeI|OH_Ybvu
z;k8R?AtZI~d+n#)Vxf7wws~cmoQ*@gb~&HZY1mM&9jy}u=y?X&u}wka(d`?rT^>E)
zZr>DZ&K|-HuXV`bm~q`1v0l3&XV|_WLk|veP-%ZLy?p~vR(KKs*lmAZy?p}^K9ud7
z{R0U-DG`(AwbIZ{vVF6EHbG7}b&CNA6FS>BUMpSGh+G(C|4zhD*-c*SRIu`doN>2r
zyjF%DIt5sn`+t?~8(=RAJ$b~#tQ;tdZ30=WOd;4507#GnVz2Hh<2COd!r%uM6J`Uf
zIT0XD3g;e3C+r8<rL)(%A=LpMgmMSu5|v(I0E;=mYGjm;d2~-_L?gZJ;kCo0A{03|
zhJY#W2`*}gZ>1aJ*8x9vUxMWb^2lXg>nFccVI~!(0gS)ywX$3whV{Yq>~v^G9*<p+
zrQ}!T^4g(rbFe%g7HNENH({qFP7=XTIGE=8(BmHpSBL%!Zs_yi6bgwCeefJXp1Oz+
zeefpX2dNQqJ4;Q-B@4#yGc`hR-957DeDF8IFC;=BK0ci~l%FN)_koAEAgOpl9LB4d
zNE3RANx(1s@3ppJ6NJe73t|%-l2FaD2`GV0pvygC6OhIxaG=>Xl1bB>d>Y_TJwjcY
z7sq}A@24!vK+jJQ55EJQ$qoQFw{CYfMMl&SPvY|`k8EbVpvjn}#x2tiTs$Fr#WZ+q
z)1aiZJ-(H*Y4CYV<vCx}*))JoZRk7BiWM)v?PRaDK`$IE2ybFU2Vu(%MI;X0IvpUv
z>cPtsFW+lz5*~41UU+S~IQAfjEnNN84y?#`yw;~QCVscadyldMh5-=~eVD{+14-0!
z1Wf>5`7Fop_F9vJY1;VRUhAvE(g5uDEf~oCc#hxgwdUxx=5Vm_yGMwZjYP6mYUvK=
zdgY1+#D;~4&+)sx)^yB?CzczRiPR&ueA15)V^3wT)xgEy_oIV<1H_@@e0!~iInKA&
zLf0ECOH}b%TA|Bkw5;LgLMQEfd#x5aP6ndUdfiPRTdkIl$X4qOS2T*uc~c9(MV7pg
z?zKMj>1k;%BTK|D@>-qUW}3D1TD@t;_am9U1J1A=+d)9G|9x<u*XpO|)bX^v)?jCu
z%G71N$p-{}@me*+_G<&;<3!ztPAjW5fK}sZAsRf}g3uVCOll35g80vekd%jd1C;r!
z>3G_QrU0CXK&bPf*Q%q51YiT<TV#l;Pvh^CD-%@5(XEHBk$X;9J<kXhwti!)ubhz8
zd~H6m+lV8syS-L4%oTb|yUJd1S+%R|3Ek_Ay@Km1dqU4V4sz$+DIu=%i+#KnXC0Nt
zAE#YqQp~n?mFIotwZaP_p9|883?!c(h{Jh~cf3rz(X!>eH+35;VPR%ZPS82tvG=yF
zgrUGh{ix@OIP>w7o=c}O$0eTLN~tyvyjD;sQ~bN0Vg1O*`<ITp+>o0VV5NUGQ2MWU
z%6m!oe}ewO^im07OG>4D?sX1RNAtAk4ev^Kp@+#$f0X3#6(xy--O6W$i>OtvqW4>A
zJH@oHTF13#?%k~825a@8pkDqHLpy?Qn$9t=F?kW1S_Oey-f7;6f?L>p)8oV_2e-aU
zykDA@<ynq*nqIn6A2VUP=2hY>l38ZSl7L<KMj$QGxUlUcn%`8_%S!|Rf2Vn7Oipjq
zDo9}lG}0yaxPjge9~p!^eae<$j#JH@(Mr8l-`m!uP&iLtghFjg3%;R!I=5BcPNorY
z2fXjGS`+>eTXm4-HBaHPRG)shlS&p(#P8Zx_0|NR#1R|?WoI*uJxtsuZ%sH%6m%MH
zX<8yCCf(F>7Kz+rkoLCT81TNJyCXe_&`)m+cx$^bY=g8hkFyVgSoz8-Pk4qlTfMcM
zrIj|DvNN>ViYL%#I=w4jMep;dqX^dkPZNAN^`Vt;+Ab2?UB9}|MN!@ItIB)pqK6}D
zN;$0hwA+LqXh!xK4r6Z~4s-qLzJ>KZ1B(+S`Vu-cVx^-)x$avXF5>7=ipf5M6njL6
zl9qi21I^Z<CQWamLk~Ygh@zT8b@s_LE^@?9$3+&m8S_}wX~abhYv^jz5h7yx)*^$$
z^zIofrcK=6?^b&&iwG=9vX;zZp0Bn0>#a<0+3v5mavb^4>oELxIL^#p6=(hq*8^5p
z+*)Y_A0kf&GZ|H!$vE7M0!>_RC{J0Tt^BHr-m0#Ddbl-Z-gYMAaBmXbw14`1gf~X>
zXO@b%NEMA^-j@uHG1@C~Ty$>??5uHIOzrswM?@v8OnM7pli@La?WK%mA7L+e>xRKQ
z?~?{^#;4u!!<d+dk10;{h&vu}hvr#=-U{L3hQ#KRgv$sQDS2K1N_A-vu%FT>)8b`3
z4U(lqnZw30X(U(*j;9f@OS4|{UY4!mfc*^1E9n%1_t}#HyNqUf5@SLGc3C&E?uCFI
z=X2^O+z;3lbsgkt-eZ)FZAy!eIt>B4LXOjrG`2Y&*^Yp<$06w0HVj%$mU;@6z-b^u
z<H=BdCfw9c13;OnB&E19O&uwqod$sD5pf!hF!x82D8a}q+0YWF;Yd$H7l7Sj0BWZJ
zz<9z~G+=O(fOS9*8^9t$GQiM00ZWTh=_rGc4)D_|0ZVKs-TO}fZv%Y2YQQ?;0FUP4
z4j~)ht}+4Zm<ucf&>s+oBw(Q(7@X^n1gw*qUKD6ec#WEiLIYNYTXPP;Df($|z&a~H
zxa8;w!qH=pZ10?aT?h-bF)Smb<Ij`732i`j>wxpD_h=TyZ^%1<pXR4PbBc@+zb9Z_
zrexA7rH&OO<R`J?V!*n>x|Uv!VTo~{0n#k=Fjf9CReo}-)FQX0c3A@1&c?BZ0AHd+
z{Plo!U)q6o$66AaJB|PDHr^FrXJo*qQ=x{w&d6M%GK@-K`Wa%3_ai+8>g#UpV7J>3
zs5i`XYwH(-q=&JDPiXA3;%;N+V3RZkNS*_Zy(7CL{35ELWKo*_fTb;d%&{*ClO^#Y
zkz~S5sa1$We4pu)Z2T%UEMON5Ri1(w6f87_LQ4sYB_X{Fz1H%;$Wv%F^vYxn7Fq*n
zCt-)<(0s<5DYQ6<wt@rJ41{sP>W(lzmPWaQjxeSK!dRDkL>QAs7(38xVQkWlFg|vg
zR+vqmXwdz3N{=DlN^sdh&UM0NNl0h-U&nq&K7TgR$mfD15yQAUa?n|u6MW5=+(o>{
z-WWu`GD2Fz);h4o8fJ?i|1AAcw`i3%7yHSPSpA-=t)C3b3xV=Svtw@<hpFHn*|J}s
zP?Ha3h{N<L8wZOrh<rxDVsF#IxR#H0a1nckmO3()xHd%MtT6qg^tXVuf)?~HY@vbj
z@D+e&9Q|;VL@9FRDGTDvTBTQ0jPb!8AL@v-v*1X;`Z~KExot?~!`Dx0*(v@P2~3D=
z+r`}9Fk+Y3wO=d<Skp9*OKN=gfHez~Lru}*)JjHP$$FEAqZ6%wz6wY?X@v*S%Yb1(
z#5OTb8i`ttj1I7zI_EfP0jp;)jTk2_VD&+v6qenfZw~~l0R^QF3BmaTEz_JKI!;;^
zd82~@)_4&+!;LLY8;*X#pi8O&4f6gJuu4G#ja?DhZ4vBBm2<EYgr3fE!2%X%YYYx0
z5RlIZWsM6Ku%dNBo__|+Q!dHqGcjPrx}^lu3a-+TV)0rcQcNUxg4Qk07|ip}5!^^D
zqR(ng8Ij2Ha9wvOl9*!zCLt174%^`XcA?c;Cdvb{fc0{5pt848MCc~MGRhD@KypLb
z^MQcXSXgJc<EjO$*JWCfHjOZQGXmTrM$wzv{Rvq4Q8V(S&{JfO(_VpW_ov9|$3ZU1
zof6_XJd-ED1t`i>w4A=IJn$QIj<^)08R!o|dkXP79|j(3Dm%0=uFuFQX<+TCz*^nF
zG8hA@t%VC5*C&voE78E)O#>S=W#u7ZV524h;9PwIn>6)_f4VrZ`B9!R9?{7BBVYvx
zeix4@U_J;#VJ7jKrm0|X=!?KyjlnT60M)s;G=&F*z1zN{3d{}25Tw)_zXd*#QpH-D
zQlH?46ds@-5(Ylg3<(X3#nJc|iv#0@;NtyYRfrdxG@LlhG6`2C?`82(n7B!}BE`@!
zl7#>NQ&f1?l$-HrxfzdaGSvJVR66zy4A7{|1QizQfP4;}#|;b&)->0?R$z$4-K`em
zri{!bZpzcO18rO>Q7qMrr%h=+9&t?q9T^X<$?I(2<PS7K^fybVr3zx(6mO&5l0cJZ
zj9YTNq*BL=ml3xl(8Muok3S2{xFvyRdJ0Lyd&lK&q?%~~y|pogG=o3h0+p$oNB<XZ
zOaUm<%vsXMv0RV$R9vpnoZPkdk-1g{ZO9&+7~p{wge42JMfDQgo@`-oODg5yHk|51
z4Q@&BrI-fCrQgYP!$d#t;)XsvzS!WVT_!2>9bp|X<ZV4oQ_@EWY2d571fwSv1;F=t
z7vHlKI4ydjc?H3j5*(*7AF#3eY6a4~3VZ2C$>6@<u2yATww6&#Pu4OzMbuLBR_=H!
zib|rcIEpoM2g<vn?gyxqjJm{|X4G6Ns2#=QH$mc>8FhIkTShI@cl@Clb*wY$bS61H
zpBZ(WGs1Kzm;RI)wKnXbL^>P3=~ZOZ(xr@AJ}Jhi`S`gu?9=N~ptksJ@*JW0da)Jw
z6<dK<nmr|AF%{$yP<j)>t1`IFk3LTjYE4yIS(o7cC-;<gb5YhZ<8m#XO9MIc@YI$N
z>s~JoD-{Z{PSe%-{5Ye*LW!8A<%E3!mo%hEZ%>b{SbrU=hGd(d>FDV6k4tr-+(j97
z63N+wZye#|ER@Drj|J+|sBD!F7#3n(Mgz)zRr+QE-{qjYEOmi*Qz^G5#NxEDC+0Pb
zs+bMrOSwZV%K~elK+GOeutxf6NgOA_6s2?O%M){v2nJ31U&Be<%~BUD3iMZ{PN2?C
z1PxZ6SR063=y9SVp#uHJmZ97*1=|-a9%Ajyqdes<(r;{4@?Rp<rQd?Z(fP42dN|A`
z)tdnxLEvPbC$^1Ja*g51JW`~G+;kz(AwW74T1w&*B5MeXCGjPZ9|)Hu0i{mS{;3c_
zZ3PG6Zu5G^RsC$R;71|WWS-AQhB4u{Z&RoS;W@$PBTl&8{+$%~ovKITj)%w_1fC|^
zcNpub!Gyk&0JT$J6TXlHY<y}ZVexPhiCOAeKY>06<RC#_Pq<T$$Q8mlNifh1=JXT?
z%j`R?=^4%{by0E{bp}mm6ykZ;o%d*|j<U&L8gnEc=-!!2RwY8bB<Mb)0pUeSd`5(p
zGTxHJa3U;&j9xM-&K1~qz%E0I$Pn-BCn7D)mc&USD+tSYZYQs2URx$BgJ1q)rot|z
zsxWQxdS(VJlkuI%f@}4ayn^W|)ks#t2rV+MAq#i~(^IOWz=FS&#W2j|EHc(;sgEV+
zL%mE!@+%jR>(<is`<b3n$&zmz3+7q+$&%MIX>)&eO|hMuU;t;|TTV$9>-AiZ*so1w
zS<6blbAvXBnqO5h<`9oCpROS!`jfHjB7Dyw{=2zx7Uh2)&!o$+nSGZ<lyR7Fh^DWV
z<z1fQG8}o$tg{r+$X(@h=bS6F*Bs<DT7v}Vs~jnbI|n%D{SPZa1{*v=XRmj%CuI*-
zXQMHs@)7#_k;+T6b8T2oaxOYKyK`~|86pjH@W^!W$dn4##!z80O=0-BRHZ2y(3g^v
z1Em?*M9%(asUIeaq$s#E#G1kmSm^{vbq}4GagmTA+_V$&+-s@SX%=?IKayxq0)zH6
zcShRxdS|H{U+N}q1kH$0o+z)hFoA5=)M+_`bMHeUuMy<coq{){g=wVP68Q9qeLpWP
zpux<qDj75wJR*@C_Wd}DbSHF`L?jUqKRtkXyZ;<h!@zFbVPMWoX+P&iu#g5n=f-G>
zNRfv^#{V&PL3|-&KPZS73J&^45dTMTg+oq%L>MWF^+W{KDIi`bSP%!#)2yu1OI^ef
z^7O$kh`$QrQM?ao5Z^(uZ7$-6NE~<+arAHw;xu;e9L?0yjqyuk&3rV*IUoDp1L|fF
z<N)=(ABn)=r>{t3!o@N~7Gh3g0_gl-#teu2;Bac`C$s5cB|4l+wI5Jz+2@Lzjs}h5
zu<jf!)%o-%vK?b9#Ck*c<P$_h+%^!oPxw(1Ym_>JV|FGS@f~--V0Lj)4rY2KK|gt*
z8CN$;U8<vrUm(QlQC@kTX(Vm5pneHLQ6YE%Y3N-VlK=0Sh3bi8^`A*3#3A+LJ|+U2
zovF>>F7=bXK<%LnPJb0i>d5dW1<U%l1WT;IfPIFCFZxnYfh8Lm_Dqbxg8ydG%fcS1
zZ-2g&YOwzwt-*w55WXyRnWX}%Pls3^GhfflmO+#Qp*DnfB!OHx(~ZzYM)b6d<XJ<S
z%Q8nfLPqRIC}SP$LZkCEWnx3D&mnc$xgNkQ<TG@9rk}JE`y`N$Jd<uAx~zkf0DndJ
zOrSGpU4V-LY$9yn<3h)((B>lHE%{Z|%g?9TGB45WF_z^Sl*5@_k`PLwP|sYWp<`_w
zG(`&KpecP~d(y!=@Tf;Ko255Qxih~JvLx}nQfDIwp^-Gh4$+lbqF?xht;?_HhAWFb
zFm?86LP<$5qO;gIXQQRtXqhv3@rujSM*6bHZyj9=c#$Sev+~=8GBH#?dXB192$dyq
zlt>-I3zFDP<W<7ULbGPHe)Oi{BJ1d{fLjsXl&%gsLv+~%WvX9T6tnTlRo!9h@^b31
z?3*k>F&WQxA#^~@TwxDv@eJXv%YPW1S9XYSl6gPbTsIvYCHYtGV{*2mjR_`S26;8t
zBMYucV|qm>U)w|mBm?JeA=Z}|F6H*p?X?4RIhHU=M$-*pbnO(W1qAs_*|jS~wh}mb
zXJ11~oIOK0DGB7q*`En_NnC$IsZ1QJO!<Jxb+&vmnTpI}l3?9sRwg81(p)cxf=x%n
zTu;#aWM5Arry=1bFqBIqu6GwRrO>EYD=&lzJOo3zMIw_GoY`Cw{fWFwXd?;8mDz(J
z#|`8biA-2IbATk85E(@fc`LU_WTH7TIXI|qk;t4y;%moDxvn?+_~lP-k?;gAu)xGj
z$=pWRqKyJ%fVoA&Ouz`e{DbQoh1y1_4LS2M1G37S=8}oIzDjX%6yOTk6Gx`p=)wG6
zPnSAUDibC_xt41tr5-g|uHSHSdgSC74St;|*L85v-8!I8YpgPpGdm||fL35XR3}G1
zLGnW*HzF%RL%K^uEA&ehm8W78w=Nb6^DA${cw*VQzEF}lwWE36*MnE3G7l0`ojubF
z5Q8%Os?ZQ?MPbVIg4}WlB=bBWGm-=cWhueh)#V=9GbIiB4m5ku)THT69$;l6Au?~!
zJlaEhUKd))`sbG&m=KExu%75e^eF2nlXaMRkCkwN<LKs|(Ms$Kljcqi_Y&}dR228(
zJHTT!*L@XYZA09a{}%>`-N#1+JT&vKtc$N?$R~jmB@{&5#$FL$Ln2w4lRdG2laok@
zanVcG=q1xjaEKK{1LehP5%kVAAk-K1vJVGV%O2K6^cPAIX(RROO1<H3JyyZF=G40g
z5wovP6~e9Mnx8S0WK)>S`Sj6w^Ct1Lny!v9Sbl>$wOyo;WL*qq@{3dRShihi49JVj
zpfFka>!yw@gNty=W9&g2yjLr;FakHO;A6BE@J0A3Myo*c$N-W{pW98-G0!_qGcgx0
zK2}+mJI@Lb#TwI@Jgbo&2@Ed2L5z3+Y(mTRb6T-B7W^*ccbBPNNQMGvQrQ0dF#W`{
zDh-zkvF6BjQmi~B<GuxJ4wI^%sfpV=uZXz4KI2%=FDQ;RvaMp>5NiR&RSN9ZkHkU!
zT5pK8UVupecfK5A9i?QSiTJ1*n8zuy1m+d5yEK<no(!>?B7yb3ckFi*lP;T<lP;+n
zPogFItJ3RE)i-oQ<b>u4v06!D3W>n=5UYI@=M&@}q5TIibg*F){~|H;j}WV`Tj$`>
z5UZch8QRc?Ay)rq=w}qT?wu3D{sH)W6%t~VcJYbxIrx+&N8?lH!w{=1?4MBB>9vCH
z736xen^3j5-1XpKYIbj>&9>i$SibBw>(XX$6Cl(~7!@I}zLJQQkvPrxbu+R}w|{%c
zT`DH>5eQG*oM2s%?<P<$v|YzeD&!|fllYN-Wj5qzT}vPK2r&mgf;27u(xgD`^ZhmF
zB_oV38t>U7<We?P8K&_|6xMfUK=?}-9dQ|^@hb+UaG8l_+%z2%a+pI3>2rCEI|j^K
z%H~Uj%J{|1-eKCPc-ZPnD*cm?!?+eqZkN{b#%su7*o{=@r}Ug1Ep6VaIfwc>*GAra
z{?Ew}dFcw~@KlhCsvy+&B^{I!9)^4mUyug|n|p<Prwf7s1<X<N3TF)l-$5b=12|*7
zK2c<Kq0h~OLN;n<pufI^Zjs8f?l?_U6j3d|s#eJ6D1_a*t7Ok2eDnVZdk^rciY@MY
z@3V74LP8)RLO|3c5D7&H5UN0q^eV*|K#|^ylz^bp6cjKjy^4Z}ihzKKC{0lj5mB*T
zuU8QT6c9D^diDMNXZ8t^`@Qd%=b4k4S!LF&S<}m=xp{-W;D?<!#(5(Us#WwWROR9{
z_D;5PUZWi8BI+wh!lr|Ajvij21w)F#gpgv5xk&8}lJHw@jb4oKHK)R41x%_MCetfA
ztVu>M78ClCANJw)K>YZT=*5-58ep<8zJBylR=I0T2YMLyLrFjx<c;)~QEh8ZCGsln
zqB-GGb8433ymUVLDb?>9o2*KoqJ+#sGs(??p5)N|`;(&|_jO<6KOh2p9KJP^EYah2
z408}0b&sCxBi1Yj@QpLklO^<2^HM_eWNghmz)MRqxt?kpI74rXp4>$7%KxoxqaW}K
zZ|+DU%^`%EbICuMcmUH>CJr)YIv7{Y)T{?Ou+aDuVPFT-z=QQTY+t+M2V3Ks($>vg
z!63yn_q@5RInWF4U=}m@H^>y*GsmKbQ*1-`l47znUh@w)QBZ0uw61q>Ag;HSVt7Me
zipPkX%=+`tLw-xI_pNASEq7k<ag?L&H%Ip;M=H^w>)>Bb{A=B5`S&;G2%qP1&k~0@
z4DgS+9o$CTX7S(sPyTjOkt{eWoBfYOvn*%oMXjM$<y~o6&=U$CaWuN6QK#~6fi05-
z#E+s{QUl}r#*b!^v|sdHB^<3IuSt)tX-H5Ts7^=8o7oB3D9oCM7`YDNzNls@ZcS6D
zjv4a&m|(s}5M8q|X>TU37p^n3Bvz}mEtMZv5U1mq6nw1|2gx5Isa9-${7OQN)mQyP
zk18cwSOaxCWo}o;i`*m+o7iX1NaUwVMkgb(RNSjOc`+hq<yjRct0AWde`H|w;}^tX
zNXb~f4CxU0KU9)9)Si-QTbx5%iCargTAI~>Ub{30i25`7Ru(7f>S3rP8VR+Hht3fw
zO>O$A5ssBXuk@dI9>$F-$>le;!Mr4T#Gh8<Rw;lf{j>wOO#y7<Cu;FedlV1{r2FYR
z1-jumh5NG{wem?L0>c0V;Q(wUR~Im9Go1HoECEN!UnZu;i;nYi5nND%w4{ggo!;aI
zDMFRpQIee3TfC&DAkcdJTtb1T@KEc1ZYW{3nV$4AAu7$!+^F`FD7&90;T}-nT|BSi
zwBnoeDV_tk{R(`C=Q!@L0>3)WVaoV$Nd?a0sftS{ptl_k-+^nyN_Lr>$j$eP#Yzt|
zNITqFftq+8z>QXb;yL^*Zjl1*@VtlHqCgis=W(Z*0<Fy@pPWl30}r$B%361w;SB!P
zQm{vg;-Z)%vl9yV2|zPMPVpQmg-g^pz0Ayfr|&fMIuh{90zUzWVmfk*hEKg4s;2?;
zu0~sI_M6k92C|s3)+``j6^yldulA!(5Eb#TlWFFgy&1d-v93#OaIEt%&#*AQ>U~uM
zGm_|I#$doJDT#+n5|cGuVgN_8)h8VZ>{(R<k_>9Q1|c)l{TScXz)I@&5w%%t;h9ha
zfp5ltfIKRQohTV%j=LIoh8--{)d2|7LZ*HlC91ggvvcUWJ49=SqT#V2BJpDQ|B588
z<}OfC7CHJIGbOSxIvvBiJq=qcil9fZ@gsT~ttcgU<Q`l{1=`@j5|7N36>gviijE@m
z5^jY8Sm2S}xLpe5;rS2lYXuhL!NiWR*x?o3>^MhbaS?u%K?G&w7(yi$b!UN6Zyu?s
z${fWek5(o|Rpw}2Ty0gEq8Ia>{)tBLPl~$pk-Pyn^}HC&`Dim-NC9l>=)Jfu680x}
zXq<MG#`Ea?3e><uUXHTZ@5MC6LuEh8N`M#B7SE5k|0qEIk7B_`Pb+|Cekq2FR)DPk
z0>@vH6(H-s+>WcuUP?@!V&)U-jN_XJUd#$Sxw!rcti`htx5lp&{X3eRT;x1Bm7@Nm
zex>;3GB_7y`7s~oI|E?BTMb_Q=f#2={c9vH!hjFvI|H#KEX=Df-NzvPujKmI;!<pX
z)U1k8|2tVo;fm3Jq_y|@X011>s==OS;~E5nO}4~gnIESe(BK4TGW9~y2dwEes4rA&
zw*KmYd2^vZ9c|2+Yp?>7Hq-8+Um^8Lq{jFgyoS_8zeZwO_+KlL<)T0PP%2G>`GOt>
zLfi6dD^<0DXA4-5R7JnW8o36Nt}0rCkCnEqXe_M&8HA;1*44m#1ey#}lLj?Z<^*K^
z&x7sW?L`ViHZ>^#9!U=BUXbLq+HI>6+fbKxfiXknSTrKKGEH{iGI{&82d=vUDv^=6
zK?;0<XAy1zWmFXN30%HJ2n+v}!VPG$^XoR8mZh$!8293KE0B!mJKVPlG{o}@?g&{`
z&qMOc#SE!5#{CeT-L4)`)#z7xs9*miM%Cz7x}je$Q;m2G0sPHI=sRNE{V1ogUPZgI
z)Nzi{ZypOOKpi<oy2r{%ST-F4SLk_;Q5BEXQh>^Q3^PC0MuB(nJc66306ot!>iscF
zFK`8^k8QwdpX~}g&#^tY_Y|O6IQ9dM4OQ-G;i0x2%YTr-O^Tt{InMa-c%lLsc&O6H
zyZAL{P`(lV0iH3S<}}=ayy{Ml-;W!nhT>KFjKL)g5;xjsa}>T&1Uh;3B@iCMjaPu|
z9-oeTj8Yz)<haA`lmSF_mtjRsF$*b%PX#Tc6rID%2xz=fyD%Rr`f;7xGklg4cvgJU
z!+caD;1kjkFz+G2;aR8$Dm&RAug9%%V5pO{EMzl#(J%Lb-8J+ljh=<Gj#nT50LQ+P
zIbMAn3p#$lFzgM(E#S#mt_~m?T^|1r%>vIV<U2#u_lyVpFM+89$4}sXF^00kHVch-
zH{+kp!r*4Mx;N%2mLY-3rsfQuVJk=Dw1Ub(=cSE;sP5u6m_AmOr*U(^Cz?E4I0_V;
z4%fi@3vO3TEFAvQ*IURp^BV7K6<S%j(w(eGD@mllJM)1n_c_jqNSuKJGEmxz#<YKd
zD?@=`I44rXddO9uwU@hb@;@63)UJ|c9FB+@jdSnY{gHZ*RoO4y6Bzo5+J5Ptpl><R
z1eSa${C})OO-WUMpfKc!ZGp1dD@TP}prM&o4_U(s^6ED--6iW#cC!C3YGA_U+%P)%
zK~4X%HpKOHk{@Jfl=x>6#>>r<&Er=KoxYCK`Ge%4F1SOTWZoSG_j51?ew=KUSh5_t
zz4gx|cM#y1m~==I6Z0}!r{pF?hhh}ngxYR+L)n71fYjkxv}$rS@eH+wXP*(twWKn=
zlJ@$@WIi!x{a@UbTsIt_u^_o_3B@~esoLEisa_erJTRxw@b=X!UJG?ySG~Lqtpe8h
z@g*blKe5u1L8QB6`V#Dlbl-sSlBbt6S6W&4@9w^&c_qhjpGSuDVN2RrNzp=!>Ps3+
z3ts*b25V3r276-jC6(eN4gO}QmQ*TjBts){pD*EHx_3w2xDf*eI7@CpvChJ{5%=T0
zMP9hCTDqh>WG1(FG+a{N&=OxOdr5gl0Ve*9cb1f|BfqpfGGhtXYP~x;LN)Npl2RgA
z=u9V53r;VI<Dat-IhWj;)qt-pIk#kG**CtScpHl7_p_3uxGnfc&(2CJi(eV8IX){1
z%jO=xj(}#}s0>TI0hO}K$*+TIYwBbrgKEXt-zw@JiSaK0qi09;uo^<XPkThQ7a-pP
zIu)pHSt4JI@8meyr=skG<&M|n29sP=rb$A%8?rr08f(&tgt}IXx`k1ZGr(vs>K3CK
zX)nrXZ$PY8#-gnDWJSBjMe)=l#N#Y+X;UWV>A0xUm?MJU<v4d{MHQEp2C%d;f&UdW
zv2CJ>KIs)V=}{X&Jkf#Oj?JHfHchL`{8Ejhw8hdy{om$aLFtd{s{kX?6EuS-a&;1|
zI2_GK)r0Y;rc7zYA|-1lKcw&^uGcge`+$?8RTiXq894^bNO@7>WVs~yikD))1()yK
zKAXrDlC9ZI)pX7<Wu~%2Hcvc?n^djO*j(bI6UEUPpO{B85*$ey|EWM4i<FgxL~p@u
zCMz3j!}TkjzUG&|l=&ReZn_xhH?{!fUEDjI+weAA0^_2x%&+kHfPmL@m+%<+p4gAu
zr#9tw8l&b<)7(UIv+D1^%c==P1qCc-YtzDH!%4|wr=gh_X7Mjrd~I0lfQ4G_Yb;!b
z%xGfhoRkp}Fr$<j9OC~*qlpVv6C=tIqpbacJ4DHKn2<8UR2IkiElAwo)%x}x>7<Nc
z)Q6g#-+=vgbA`c9$|!4%&Tk=N8c+x8ev>j<ad7*M5x{R<Xs~aORmKo_liE^dmaGz)
zQ+T&`l(+!)Sb$ph+flfQJ5+$okkw>vnx8q#B9KclV+gw+Ng0SCG}pRV^!q6T%aHm(
z{AIsR8CZpu>8Q%H$cmJnGUev|%^A~_o~0zCrO+e44$t+|qNMb!q<Gbdl%6%^HFYAT
zx6Kljt$yhga}sd?yq>nabXg(HE1Mw@hyOtQ<ai20EdU_CX%X9~aqtsuQ$CFN-sO~#
z5E&daKV%U@g%Cp)kvx2OQ%apk^3Wo|V%0IFX7f-dO&m<sOsT^tgi;d4hLoy?qM4<r
zT8N@*A&ROh*M*`evo9spP}sA$DXE1hQVUU}ieih_wC5a8xhYlwNw&hKbJI=YXC=Go
z)-dLt^pqs3Tzp?)?z5Ol$=5KGr28#SLkj!5F4}rE%4=~P-FuUXkeW@4UH`+pFX7QL
zh7?xi<$Z~^j?HBR&vHB08)Xw6Yv!?5?X}c(Krr6C@fwC3nXFDUAtvuNVccagUPIzz
zbIBAu+#+N&Ax2m;U&(t#Si>yVD+bE|>g25y;4uqWc@2=Bw?qJ~3X<}Y0zij-d32^;
z%Qr1xZUJCsY~EA>4p_j{0>GFec@qRUZUGYt0Mq}-lf}0RSinFFAlpM<%u^rI>c%3R
zg!m}WZq`WL>4);N1<14%+1FTok=IUu_7>2t08lb5ueku~g2_ko0>Jd`d9?+YZUMCo
zU^F%HYF^!Vlh&(wbVpvR)fOQ=j8LsxUS)%j@J(Kd5Z<*2DFv+C&d4h%z;_l<(gK<|
z&U?4#Xv@lLb<SM(B|x4^{G_ky-L*2u)o7^oO_<55JOZYyBe}ygBFAm5dYdJ@ZV%<S
zHH4?lAmP<0j91U%-Aq#N<!Cp`Yu(FFDlOe41*ra|9QQ_n9`;$>NFR7?E^}<B%+ee;
zL7;^`D8bmIKpXnyxX}W!IcyS+wjgp)qj}C>g6uQL_;dbBHYovEepAkQ0S;Thd7{WI
zzb`k-IVr$Z3(zMuEa1k&IfntXxxoSs7qA+)Hs@0Tsu`W;=!GmqhE;`TIeP?XX#snv
z=oT>K`JA@}=w$(K7Xa?vo})_MM(?em!7T-VRkL#z39!s2zKC3x_$juZlk*G;LcW%c
zQ}J?+nrq6!YqK?WWZ%I~&e#Irt<!Uchk@T%9@-$Vly5$j)6)mGjj_O<Hi6Oo2Rb=@
z5XOw@?%tj=s5k?P1Z8gbvpIJmQQHbuR1U=ZncI6jM}w)hjVz#jfjaIznUf{J!xoTb
z092zYf8^X@j01n8t2uX6QYLIWp|-}k_oUK$FDHkuR(fscTWWo$(kk<w&N)>Cc+~=`
z6aY5N&C$55?UxpCi?Q`W?au~NNbjx8VUgEsd&y#zxgP7im9p)K%v5(DXzgC@VH>dE
zVeG%{1F7!k(3y)rp{c2^u39()x@+$^)zzbip>C=EyCYnqXxA~O&uAyr)gy7iOh++&
z!Flj}s(V}m+$<BNhh4f<NOg6O(S4r4-Mv%YlV)T@VE2Ml_jdyE9}w7jN2>dWdWHe!
zPK+3xjW>);y*-tx7c9JXdrh?<VCti4Oxc#H?k0iqebA=svt72OSY*ei>fH*jz4m=q
z<q<F}d?iLE3qwK;Om*jk^)PyNsyi2SUb57u1ybF43S>F`8FaT!<<&xk#IvWl14c}u
zkOfa2l<LyD+McfcWaw@xMNG*sybd9Z&WA4tuEg$AueHTIP~PcuDlc^-c*{+xZgnNH
zzo5LV?rSq$Ligz4y&qHEvO>}7;TxoECk@JWn-i&S5kPlpt6bSGLK5Nr&kY_HdHRM_
zHjBZj<>PS144gu$N#7v5eWP{iIickk`$|1$u*|;IuGA}Zz@XeWC-sm}Y{f`DWKgIK
zs~$-`X|!sP9!q^kNY>0#-zkLju|X=7@_mnqV&v<oD@5_5O?k!j@{N4mrm>|Ujgbpd
zXF<^+Xenm>55)rUo!UEfl&~sWtWkv|QD2QQD)G^&J%ywx9~$iWKdh(-U=2uBBhq24
z#kwoZs!oN}yHP8ps`N5aTL@{XMQTw9$#gzjFfM1~wA2)hK06jAQ&oqjrW<nn1N){{
z3j4d<omx5UCy6S?Pc`gt9K`77Q)}6}q?jwjRR1is!40Z}<}MSVhNen78k0S7BvpGu
zW=z&G-HZxT%UJRzER%f{3eetq9j9Si(eH->cEC<x|E-~bIl7?}9z`pL0%MHI2#lUh
zz)b&QayfVrvd?>G#q=FCS^+zg8plgm_lE*~loAbn#}|Cue#%Og19%$9cZb2>m$0nv
zHghhYRTqOBGpzsppmwo*G3Zw>g#x@@?{z8;yhlF*w4~;yUWnJE<weM{T?kn2?WiZu
zgxtR)A<@OhO2a~d2%|0nRTmS0scFEZfS@=mM6;c809XGk6o@ZwRqF>?;gqIT{j%zZ
z+=FPZ(<2bA$|GQhDe&vtHROIPAz$%9-&VY?*7fbG+F9-Wko$@7cKdjr7(9VeONQJ%
z0{!TN_7s4U&WGHM0$uh&8w)^j(?ae%K%G;4&^!Y|c*@OPKqRAkxEE;|(uWJY&W(IL
zedE!NVA8yhUYGGYclALJ7J!Pxh1^jB$uyMXQ3as5gpfN_pr?J%&;n58>5$t`ps+If
zT?73Ra%&58(Wk0yK#1ERI^^0lDFV$ah1_~!1AOn<kXt_-D7uh<u}0-;)LkL>R@mKj
zTM1*1?yY9(3h#+EfK(;-0|*3H5vZ+zh?;H-xn)GuDyHv<VVsA!(J&V%=k}0WRG_{w
zqeqMb6l*~*P^B8ghumN>>iz0ZLYIW|c&s76Vo?B0Sr<Ajz}h%Ne7dqyj5A25HKQ~Q
z(83==hlF$>)=2!bp)O(s{@JdWz^P1rXt$8g#Trg~43bUm!-C`<m>Sv+Qg&IJ-1b|q
zCAYm=IJqY)h1LowE6yah7UpKmRP=D@RWyTf5L4`)&<Y`Sg*S>@ksPM(*f#Wp0Mp|*
z02O+|rqD;;+k^sonFXOIGQ;RqD}*KreVvtPV!8lU0TVL}(o-mM&9Ts!V)6tkhlUwX
z?uO6+X?F5Yq4v<<9chK-Q6wuY6}5vA)+}*_I?BH#K6Y3xEl@Yr`T@En?KmsTgj$(K
z0A3F~5vnL&&8#dH4KFK8MQo05PUsn%kZFS#YMgpc(@;u`;pN@k+XBm54FO#BMd-#b
zaFPXT)`9wzZqeDHQU<EPdy7KxCeQ?bZkm~T+`R+jRe1#dx5K{W!8Errt;*d;e4NVH
z_er-rDBJ9IBF#+(FDKT=OE!4amgx7>TvdNt)4Lr^bBha4gIA?lyf9w8!GmbkBWdmp
z1-8`fi?rWG^8_)_{2uPhVEw0|Fq%(0Bfr)!p+0Yj<5`1CyXuz#idr!&?MIY=C|=Gk
z%j`!p^0mVNGjLup3=VKUYB;NxoFA9=nR>~b?|_V+o%XebRB@cePp9oN{D5{T0mIDZ
zX}dHq(;CCTkJEP11RFzyYF8!swFuC9d738gU20khUb#-|m`CYKX`01%k;#+cmDlUM
zbR|`cRRhyBz3DR1Vl679mL*n1kScwbHc?1BEYie6NE#~opOH^{RDR!T)28_vgM8}t
zv_2wNJAx#A3Xz-M(|(0MZD3*Y!Pm&6Mx|*Q&w{HVZ&!%?PLsOsJ8wx&%P}K$MxZo@
z^U@moWz@BwC2M?r83pIrGHPac=pbI*9cfir$lP~~Y)sG+VmG}Si`r{i6z8{5Ecy*)
z>$ck}g9d|ywrRC^U$pBPmMgn`0(d4)cWk=7i)R5&rwF>ef#-SLaz@k%ZIg-lw|<VY
zGP>`g<9(zf;Supu@_T9dPz{Nn$j1@d?n}5YEt5pLaX#NBvJqD6!-;Gsur-{>hXg)Q
zBKQ3x5eBL@kzo9kjqgsf32fw8@W<&Ufx&4FSgP(en~kB33*fa;*<j`MH@zF0Fc{cI
ziGLJ#*sNE6T*0|^<)}l18JT@LAdSri%z@JCb{e%~$)!-ctMz#-xs+D>)TE-)Qq&fG
z6{r7bC+%jU<mW_ZT8i;I?KtMTEHa?c=RG%;&_{9zXr?wGUbBhFKzlor(}QWfmqxmK
zRk!YWnR1HkPKc?7nHg?e6zSSAI_<$z-$nYb&=9zPd1N51lxPTSoE>Q&@zKz5|E-bs
z8BmRx?s<ZkSBplvpIUp=oGVm0GEm(F2~>J2GLYfhD#NPVmLX8-gOM&@ih%kn1X8lw
zC3=>T+}&m8LEvp&7U^y<cvpSA4TbPF72qw6jC7X?@7@|d-ZHfh8WnbXA<})GNDli-
zE$(VdbPqM<k4QHUYc8(wf;muo?;C@yG<i}_mOCV!kLZ*GOU1H~l0A<*4bJ2?L}m~<
zz#T+zX<dSih2v`d8D+UonIs+OBy--A(TYUTPurq%PSzr_ik3R}-xTRS#OP9O<-Hp8
zp)L0z(D|s+V5FTOHfB$+^_!7gessL<cG)lTJV=G7eO(xUUWhzGe&S9-Q%|+C`)kzB
z$xlT75>B@p>E2CTu~L!0hT|S4ZY*)1m5k(yAqf*+hg)sFSQmNHO5mLQn7EJ1J6?}e
zRK~`t`Q=!6<jLYxhaOpgsSYQ<RzUgt6?b$8@4cVm;wl7el<>=6<o&P&Bj#e{L6f6z
zNp@9U<U!MH**-$cv>*tr!O}=QWAAlWeF05ZiW#&5*}F3GfFSp2ga+h*h0vZ3*&X?z
z0BIKRVF6&<`be#A-PhOx-Y(#^ZDr(}hL`OOBJG;7QptHfa-)Hawlo{BG3y-3pwPST
z1q*n=0BG@ge-rs4QE~a^Y<Q$Ow4WtbTQefLstUzUOEI^A)m=v;^91<D0`dyvdh0~w
zR6~+)AyW%T-Z~hm_2C}T7BJx&;ETwy0;F2NSPP&ZU4Jrim|CAKLE8asb5G<jvnGQd
zp@%WTi2p!>Zg@PB!*yV3)QM*68XowEtdFdlCKmWVIu%)0ZB!OnzUOMBc}YckoIGuF
zyNpJDnu^w8Kc?LVGBfm~R`v<UfAD)`U*p%lN$|nQ6v@a2OV3-thCzx%6f=nIXbe9@
z6~YWi{MN9a#4CpV!<$C>LpzLfR{KaUN5Qpj?06@#lufV+hrRn}d)o5od1sIC>Y3(w
ztruhmjw<?DFjwUf_-B2iWqCX;J78;!+PtS1We4(Y`%CDx6WM_?DRc}kB0KClUdayd
zJu|Q82R^$W3_IW0S+xN}1H9EWvjaQ9>vgk_w-akVrX?$-wv$412KYoDeJOO-^FCae
z9nh<5QG>DrpD@Wycv12ww1-3_0gRXQ&JL^=W362%KdZ0r?U(e9?+wRE*?~2rz*z&4
zwHcWmm?yG{KG{4YHv#dV2O|8ztGPVeEGqPR(Z?Iktlr2iyy46gEZ&W;X9u)-(d#=O
zPwiy^-ax~TA8o@(DL1|@=GE(D2kwQacfcpQw=mJY1wz!BlO51LNAJ2mUbZn?0umy-
zhJ^5$M<zQ?ojGFOyg_#0P7w|BiS8^+bm#R%%^QelNq%;qm59`{NSmz+6ScaYXi2_^
z-hV$kpuLLT-}pp!ufi9v$@N6<zi)^(W(OKV)W<IO5h$Ee!~Y@LD59EEvIEsc6!KH5
zURYt(uh(+TDI&_em>sAnqB%ZM#ll1t4Ut(j&Gz2__f@HK%DgC(tNs}v*4njH0>cb!
zS(BAMTYZMfg&8IrhQhmTN4EPHczqKsp8HoJJom5brM+zjNiBacI}i!F>Eef`W(SIx
zmPK8Bb|Bt%@e&~^+trRqUo}FMlY6o->yy{B&Pb9xUGK?uRrUK$@`;WXQn`EddX;y*
zM?|Hgv)%7S^odXOePN>S4Uv`GJy=L?R|}v8Ra#bAI$G?e-JI=y47+|>_?6%v7iRZy
zVRlL|tOobf>)B1a*|1B<X1oQvp+39qh1qQ{%x-64c5nX=y9BYT2|I0{_M7Lk(^u%i
zwb*^Fpk%9`%68WXZ@Z7TrZAqG8QUskW@NkSN&6l4@m?s5_d)?)jnA@OZLjv%bpdH*
zNnyMt*B57v&m?Sm%WRjG2Csj6pXjN=L{D8$l-^QAX|=Q6aUy!(CmL6nXx#NgX|-9u
z%;j#<qnKX#q?r4M^(8PBT85qX4;!Qa8aDx<O|4`C!}|0l@VL;nDAij^WV<7ks$Cf)
zP&n0**Qa_*36sB)+02L>Z@_K7sDld=4Zfa;%XCr-=Bg%jfCiY-WZ@>&wM}YFzY!x5
zPiG}k4`91>zsYv{N>+aZ%(T#~Wy3BxE8A7eJK!~+oeo757)18<G9+h-sN$$>x4npd
z@rl|OCTf2@QN>XrO8Pw8)tqObj#NqT7KMpgTu+qrxrkD_WV<Crw8AGUS(vD#A@U2b
zRN(@oE?z6Z(uE6<r2!x)W%TT9w`}2BqUJEAi$5Z8Yx)x(dhwLH*%y#u&>dhS_yr>Z
ze)^^h*NdG}mz6;b<%*Fe)|iuKK&&GeYr1M=!4J+=BML(zMET1|f<No8?7xj45o^w;
zXiUH_(ISOQw3ygtX`v(Kw(NWfsQD)X<`)t$-v}tAa#OO83n|Yc9WR7*yZ|Zbw(OsU
zw9z8z;m~7q@srh`3y{+O&i+zJY5_^^%R)$B8YEVgvcCq%SdfI1|H{^oesCp={$U~X
z4=uVe*N;rz&;ZtITSkE4;Y9LZ;xv4Hwi=tkeinFpu0XmMSRhW&*uBj?*)KFT`Zio&
z40D}0Kl>#Pw+_y;T%A+H#Lp(DZVpWrcbwpJBy;bh+2cuONDC|Q_;S~h8DH~SGUG$n
zl6fec3`IG~<lGjeT|&a&s#5k8V+Qys)<=xr7v<PzO4HiS6NT(1&mdX5(Y(Ug5O$MS
zNq#H(QsHE2U8wlF?4cTevw<_DPn;Tnp~;Rryr*E@`({)1Xy1S>K|G36wJ+gC<~cBI
zWfIj=Q*e9so!1OM{;TYoQp7&1h?>Y?tRbsrcFpiyqD?TnW_@#73(c)qk*$HqQ1$6p
zM#XfW;D+C`E224b*b;(@twca_CEI7qDy*HX{U%#`=0lrV;xcBKiyz`Ln&@js5#llp
zt`Yld|5}b7uwrRd^%l;6MN7-O*{p)h%XGDwmdeTf&y%6QsEPHk$?1->Y#SiN)d>6K
ze{pBV%2+bpn@ll!b7qt<PtRt$DOSDC?~LDnFOS}z-jeCwiXi{GiJSYVaQ&;9ZdQy^
z0iiXnB}d^tZ)VRR56x%kOWe%Lm##6HmkK`b=mdT`hb%!KtmAW^uX4PR%$<gDe9+LC
z$X$;23^z-LF{k7Gj%fJk!g_~Q26)V;(DPSHMUh`Pj<<sIcEj!`(!J9sXvY!@FV1A>
zG~O3X=ojg&xIivSxr<TU{LIJDlb#3sWF+L~gyD$E6^))A!}z79-<hub-g&4|(~JTw
zQ301fm^m}7B<Wy=S$xQx6^>g=-14eMaXMsS^j1OLXXYxQYytco?i*Om|2<QS&RWO#
z{XFgu;>IRtz9+nej`Ihi{E<Z5lecH;@PUb|OWbX|xA)l*A~=;ed>ja)XJ>w+1=B2m
z{c(C)_Oq+fR#;Pdd-%XcP|_h;nlhZxXQs>Tra-mM;LpTO*D~dz#pq-{XinnD%$X$|
z@BSDTNfxaJ&hqCUTK-(L8PD@LEq^ACWHoWoE<)>ZYk0nU(Z_Ju=%C-k8NnLGMDQ!1
zcX7PK<SqISp8dFeq``|z5hwDUPj_jFhF@BZOkT3_?th<a7eUq`|M(X74S|RYcz(eh
z;V^zM9?vP<pW~fC^cq9Tl8s^DBm`R)#VkJk-U$@pb(;G*pFhOS4BOFenarn09B)}G
zU<-j6^ExCu`Hbb+BG}Ckx3<ied#Hmf(0mM*%Se0-)+-u?lqgEetp~DAHWy`{=8Wrz
zBIIr(O8S$m|DiWAwCrjFQq&@U+4WC$1OJS5B*!Nl$<+;p5x4rKq+~M+YRxCSYNJ{+
z;&vZwWG`pyz=)28BSW^<j4+R%6s|R7>(eR!umemaVpAr!lwrJ@7Kx@I*UZG3_^R{q
zUq<ca)MNks=FD#7MlI}!-D+VoyHyCcu&pm=-t5Cha!ec8%{<FJyeBMQ|1LAxu#7de
zC&knr;;~Y*_@PypTHP~swjllk)@4WDW>%jvs~HmpIAdvM4OB$E5VWm@W)*@qwa|DA
zVG|A+EnP<oacB>-W(NZdQSq=cK7;=a)gF#KP#Ix?7-=bs6x@xb^$P|bP%T44f8LI(
zPir@250ToxRb&fXGc@|KVT1lngF;F#3=RfDHYN8dDs9XZr8KFYO(~RObmsi|F!Xn`
zOg*VyzWbE9B$+Ta80bPRo4`FIZ&Dks>d@@w>l@GO&*i%d_nPJW*MfmeaB_E254a)e
z<UfGZ3seao@;1<h65gEC4z*$f3FlA4O;I2n4^2qEu1aml#Pd9ExdI_Pn{XQyXou$`
z9N(G`Y#2zAy!;2_pKH8NqnVqh6ki8?7MHKU%Z_u3aoQ=aJOws9gNL<~Q#U9u3r}U-
ztqLH<sU|pXO$IiM$AgkiwUfwPP!4Q(htNda!;=4FZ_Xi}h6!wR>0r3f#K{3C;Vh}Q
z26ZF4KSez{HC+%o$dc@RjGiaRjl|B!&5g)+7k$sMdp1=@o4Y`{s$K3PdJ7ZB{T5SF
zMf2UoP`{!6MiqM_mv`;ssbv0@AwaodMGA>EHc+2O>$#^_#*x5s92c<M#Z`&;T+GH|
z5VT!<&B>)T447ZW7&Vr&gPU6qI=$m_Y8`G(c>=8sf<)R3;k9>s6pk#24Zq{#W6@mb
z9iLNg;@*Ia7Rp_;S6Pw-$fZ0kJnQhFIEnTk(kptRm&y1vCvYT#$kD<?&!SxIXB-`q
zYcF?~SU!~RK8>a)^#yr_IB@M=*bBdf16O_iaqY?tL4jp{5}ohP9%(d{5)7nanPYOv
z@+T^fy|`Tzb4fnV7tmReQzvo1@W#*AMW8xKJFj0>JBkGwijpgDfv;JI;V!Im+Jorp
z3V2wXJRPS%1|H1%bXf(^>FG+iie+JPRrf5Ef!T*hsTr(#7s|k#Wbj)n17k=`Qtxk@
z@6MqBk*#qspqt4L3@7dV_rb6ZuBN1Jj~VXgv7bY#Px~Jrx&uyjxPL01p*Y>L+dmu6
z6x<{Qp2hPNZl(fj@T|tI5T`fITStNYyNRG9JH3xLz4m{O9NbwB?EkmpoM9YwCWv;v
zok;9nX@_6ODN!sik{2__4n@Lmmx9C4?M$))Pvc<_edaa_f6jiR18?j-$|yET$<5(S
z&u>3a80np+By)+#WB9TBv2#(&{>wr;DN=)m#fw)vl{hHo;)A(tCpD-qYG~jiZC5q+
zfQXC@*`CTmXT$DOsE-468>m_#j@T}90$)}@*Ph76z)!Dv8pt{)@WlZjrHvF8d$;4g
zemiky@MrG0g@w>140p~&o6H01`&{0pGRcsgxfxF2z{_G}BsY?owwYUyft?u1+=kLE
zzmB^<7^p_+kJru4oll@jM%QQRsr2<B`JD?0(axV~rT}f`8HPt^Ix6rgo<6u<6z$Hf
zzLD(wfEe<6W|UAq$0L@HEARuJl{lHgPM*Ab)RpI)H|H^V-geb&%TvsoGtI=5vF7r2
z8blj$>oAwMYvjB0qi&@wpwF1cFxvdGwor2f4aB%nbQ@bbf%`FzL6f(ox#OIrw9j0n
zlFZ9?yf>fKozA!0lD}c(n!5^tw`ugxCg4ga&=t>3xEmGdhlfV|?5%!_I1dwm^5L3H
z<-BqKMrCRd&!We(4bc0%BjFZN)<y}<TW3vj-0vz3P!;!<jT^5g5V+wfGo43CTl>qk
zgmbg4{f)0tP>PVt{6gX}mzO@&q>XQG!G<sSk<pFU;|IYpPL#cel&d@E?!56nQ12f^
zjd7ySR*1fuCX30v&A_Mm@RdINWC6UI5pc2Kz#_&ttt=hYP7TS{r798AkW3E-GBBlw
z*{=_5rRJZNVq3_Ro<W&g-yk#yH$Z`R@t~x$qZRlB&s^MW%75#>ZLx1XN(4>E*%iWI
zbbIzQ9Cxk*TmOsaZ`>II+Zg7Y3*mUeAh3<J&SA0V=26DmYLM_I#0d#flh}2*=LBgC
z1dTuRf*_1!&K<`6L>X_pGv8ehMaG<xpVykih3AL3z8u&#8So#tlL}Bm=PuzcWD*#t
zn0p=PJcEGq0TjI;)++MBWB)7#Q}p`Eo{H!p`ka530x5{Nf)7u>z7(cdc&2(S=o_l!
z9<gzZbxm9jG=7uvs4=Pqn7b*n*r=(sCOH|^=*~;Ak>>Yyq&NSnHKT`~6zxPy)z7C2
zM9o$z7_IttGal%rR%3B5Vq$BtvE@C)?LSik&eLU`FNGm5?&sG4x3-aP{tmeD-p50K
zC<TmNsqVLN4+j)C!RBJ5A6H6o7>2Yl!H=u#$BF-Z;$(-M$M%I2N7`|<)(>-2EZj#w
zc&)a~Zzv+bxuWrfXywQ8*i-m!woP`vL=pD$!X}Pn=RfHu8;EKWPS<Jc1U7F#%-L#%
z_;uWR!9YztOtASK)IrC6K2^s56IQ)>pS9}E-w=x}p4Wh2^Uru_pw2f{00TQu({;Xs
z0+;dh#NA8a4cXRM97m$PrK3q}hpBFVRAKb%O_)YAw{#KU8Mwz4$iqV|IIr31QY`B{
z?e_WQWMjcDB33EG%-;ODRUoRK)%FUj!wD~3)1kT?wK$M2+>9W%gmpE@_9oPS_ovPb
zn&umUE(ZfGss9sB@XYB?G@a)e-ko1FEZ?0KB4QMsk9*`5LhWQm{IXXP477$~Qalt#
zwu1d5POo$&#srCBpm6>-+;IgC;Gy1~Kc~R=j&q?n&Y&H`lY+a2=U7i>z?y6Cut-W|
za_mjyRD*l6HQ;JEE}gq48{rAz8ka{M=o5U;{4&x=yn`nhj6N^WM_k}JTBFYkT?yp)
z`n*8iF7!buvuHJ~KCitBW8NW<`OX?<6*}=;fkM5pyNpKlj;}jwx8oi)Rltk`?C^m$
z%Zx&MIe}wOqZl(2)+n?wBRd%WuDW1`r4HR<kD<XpGFdgp0w(n>VJ33P<u$#E=Xjj+
zLib>xehF?<ux@rB7~oi;nf%4=;=g%EZPHWS3{7#~;~n0UWJWs9?K^|+Rn{N!v5TT+
z$!zY+wDaF3ny7%|B-IbPf1%k)`yb_q$ag8k+(`xe?pD(uymJG1%N&o`dxGxY;7$Gm
z@^@+yy-wwlG_qJk8AX3jIVX+8z~2d>&k?xc<+yYEph0B)0(2UhV%}f0nv-jz`wfgG
zE<A)AuaVq4y+HXElnB7tV$&bclQ{EANB&oix1T}d<grq}9ctw|cVK!6_K8Zm6z{jz
zAgsBDAhbbR=@{tc=6hP_@N&-<xx)#(dlVw3x~GuvH-8|l8btRz$j!3&0UtjyjIXA{
z$){ee<`fLm5gJ(4m~TqJC&OrHLNLJ4+MC|>KBCK*=vb68Au+HmDD6zZ*1uueaNz~q
z^11{viGAMjrZg9%KC#rA3vU><L-L)|^Nof#CI;5iJWp?cwSO}ngwJvMz(UeSkv&G}
z7aVg>_Z#}%3s-QL6j;t8Dc#@22nA68#d0_%8}2tGbMZD@MFsZYp+;S7N^X8^o$s6(
zWRm?lF|dr<{3ye8_s2oN`{H^kKo5BlIWB6Jp2TxS?vJ!*7oWn-WZmZsBbQfScf3dF
zoQCL~c^-N=GsCejLY-lzrbVGg==1z`izXbk`w!Be+2MHWDgZbA-vl7x_x^bn-9j|)
z(<i3@nJ-T%$s<1O>HmeDP4-h<uN0>kjy6TpS2<@XJ0sCYO3OgPY|qje8<1WWy$+P^
zxW{?A=+Ovqwf1P*SKA?JXk_8#uY>b;0Ro?$=7U>Z4`wZIpm<<wXBP|UdIS__5UlZW
zqU)@!p63z+%dwJamt`gADT@`j=Tt&!V*f=AzPJguMgba;i(lXlP~r!Vz~U{Hc$pg#
z0|O}WX=`9{P%M7J{Rb9hZcrhM#eZ=Z6oAF0(zudz13%sdi?67Q`lXU5x{o2r)bg^G
z+rh!oE>*z_&VEnCT)o}$YR`P<oH{D~8j3ko?x`ba7G9-TF4e=;QGgcA2GGa3sPH2q
z1$IGAvAjx{4Af3^=Q7Njsw1X`PYcMKh549(_#m=*vZk;V6-dP)Q&7^aRP*3AO-rAK
z8x8NJ&S7pQAZ{7|U))s2n6f@+G3`bbA@x9U81Zv^iK{~#$xoAEJ#0bwJlo5=qtlG#
zTRT<1Zfb^<cU8)h#)etjna22MkZ*41z?V5rdt5D;F|l*A9rwf{qLwJiJOXuYl{F{@
zO|?8rgyDIXw)|f_U_9+Z$(n0RX0wjX5@%j$n`Gk-IDt&X{VBXDhRL}@PN3FDY)H}<
zM^YDKWeEH-DNhW{<_YD;dSeJD%cHzLxLyj#a_+|sD{tTNIfFSt#IF;P*ftg&>j|Gr
zXy(#5T2y@@_0oJC%^S)5_7%8EhJs%uGY6!{G&K0_5?Hi*mv{!q{q1)=%v>(5RsdtZ
zv<0_WY3_vst)cxT&5jx;Y&s11UEDhge1m5{Zr`=6(Jw-gY}1^F#XjCpxlz{sjiZv;
ztQ{qASjm(Fospyc{YTbLgYt);waY=rq1qN`u&U8uRhx_+XMg+!KbgyD=rW&}G09ve
z1DDH&lgTj2@LM>U%d99}P75c~gh0rY*=gF0MZ++r)2rdNj}-Q(K<zU_QRZ#G({EYx
zef-}*+-9=g(+u@xy^m|ViBA(5>t#eL&@o<3=uaobT!_b;v~J>@p-YwlP1pQ~`G_D&
z!3;O4IcKQPf3nVDR~(&2v|BCVbU<04iTS1-;&#9>ax=P_GL9~FkT~;8u?GVzsd>|<
zCp+$xY5JK%HDMn(a7SsWa^h4%BUysfuR({aH+>736KCt^S)q?H=nQGLv9EZF1BtYD
zcLW1LP-mV}({~4Bk!?#5V*sv?=9@1P+YL?>TVNaJb(ulT<?(Er1h##f@0^b?O;P`v
zKhXxx{7N%Q%;EA(+;s8lFDgbWmlxo))Vqx~_3{h2<pgv-=ki;)&9K|fAniP=gzDj%
z2gL4v*ln*5_!Hbm3edJ*#<DN}pg;;98q~{1a4kP?Z_Y#oPCGLUCuY}W=G5G#ywRuy
zb`EzOU17X}@dtMH#)E)Y7`+E}-i?Rz8du6G&>9bhbfuyKjqtFjbfqRGc)pe6kC^RK
z_z#=XMIFvs#*#ns1=I6Ss_bg+B*nX!VgHz~pa1ITFqP?0QL=o9#^6dXTvu^qNo{*)
z*~9~EZJFgXb%p0gg2zIIzc|PGwsRs8&e+0vtmqGxkY>&h*vy3NEYvxV5&T&Xh<3gF
zM(uo6Ym+l~C{D9Vt&z(n$Cy6%Josit>c3VwMyWJ%0^8oC;k~F7Rbo4kb_$NJiXJGi
zWj&^|T9c+SF9!pWs5kFN74^&DZrKjrBAga6wtPtJHphFSwjeYvSJvWQk?Cx?Ojh<9
z)7ew&6lI<F9@>6WrLzt9miX;~>P;2V9^7sPZo%^v?tlVS@SMaQf!&*R@|_Dv3)R1B
z9TL0y#O_|lxmpYttpIiFDi(9~76qE(sfyEkQ)TE10&fnqrn9x9;Uv@fz1GLdTNCiN
zJ_T-FTx|s&!P6XfhXP~qbi?t$WMJ!HJj}1I_EF$IJc16VL@&ht({#)PUtOV!`vM<|
zyRNu@O9rIk-fxRLZ``%w#^3}wWQ#j5uTXK*sj3OeGhaT^;O@HOzUmjZzLqpNFea8n
zDD5r(h8=}$N_)#eM5Y9<&Le`--dLH^{$8bBmL{kO8hQFMN+SSFDp`+PLuqepMr>op
zd$PPBRHCcz;dU7w50Oc70z#WAuHGwpS`~QX3=qD=eI|xHTNBWh>(vvuUldsBIRB-e
zxcXlOHV~+ct00+n<~tXYj7$$!amP{KPirk}<M+Vn>;J1F+xQtC8nOTO67vVe;y9rv
za9Zi#$bj#^yKuZi8+eUW{+o~cg?zkL0{qpmY#~Uz<1mGD{wh_@(H4<&akn}}ei_D9
zaqnip^i+3J(;m@ZX}I#`2{5YwgF-TR*Y=1Mrh)cj!+S)27|m*r=&zc%4A?tON&e=1
z=Mq)bVUF<B=aPpi@K+Y@b_LMKUp;a6BAmXga%qOTQhsUwtLTK9_tY+g({~sC!V>?=
z#n>)ANyM|V8O&JLcWFFE5vK@HjL<vmKGajc5a~@kl&vYvMS8r=ab)RIqP4N7%`N~I
zthk1D8SiPc{y)XWb@$^k;Nr3g;IHp9JQ3hpWHRc8KP)!KA<o4y;xBRR1G>2gr8F)u
zC2syE#qtoW>`9ahCvD|;TJs>aXwtZ2tHsk=3l7JHMgD*|Z5C!H8S^&{A(@sE-a?)1
zb%ZB3QiZC<#xA71S`wh|jos1VRNQ1D?xUPH(SOtHlY!&GK(r>kc!+-ooTa#hnA2L?
zy|UE1Sq%jFoLDN%U+YvY)?rDP&Z=767*oS#OWd0~lj2&30sbWx+hKSki$sBSHxXih
z_SY8*sMP<&DXDeTz`t+7@kU2rT{CaaUPrN=h@t2DyDk%k*Ms@a<!n`!8!Oas%VK7;
ze)Wh>22O_l&dwxon_@`r?*+JdDCBZ4TS~MNWLCn?A`&gZrd|!VDs^s7F$udF0xIg?
z&nbYW{$|km_a+6F;n|1VqiocZe7kHLyMC=<6(qW2jq?aJ`z7$#9|QM$+;<AdqX>Pj
zYyAN{XK<$!pz8ho7w)P8n#mQ%6_Loh|JlY)G37fE@x$Ho6*YL}YFr}4)BpGG*>!5e
z@isVKp<b=~L{{;S?wO}suWzsaq%&a@7Fc&5a#T<g!7oij4L6?r%~9*L?oqhjh3ljM
zvuCFVPL0R9A$a8UsqxEn*`|7Up*9t)nO@`EO7;74E6cf*IO}kY;mih$Gd9dwNlgiJ
zewx6O;@r{a9E@$kxG&6G_4lH!NOQhdUd^Dw4XHL=X0)L(#hiQ#FmcJY0hn7tarPlj
zkqq=}+hy>zra!k5aJRlmCesi$^8rmoXb0a9m`Zz@I<MXRxm^UY75VafAA6+O(1`fU
z+0Nx~`(ej3-WavTs;g^%q{QS`U0r*GTtACrEX!dL$B7FC-Cw9`I{z~F%M$(j^>c_Q
ziym~BB5DwU>YP=0QSU|6trc`%@^R;#_HkcgfM{@E{|;y^;iAPi<Ad%xx}v@J5?KF1
z(B=JLr$QwH2Ra1Z4N)YhhafT+1?{K27sd=8J&KfHD}c-jx|@(d_XK$!FzCLnfNl^*
zP7Au5!)U$71l=tbEqCm2(Eb&MbO;6A?*!QbO?;3*IR!XuGw)Sms&z)d3apS0*7=)x
zyNF8rG3dUfs$h=f%=@ypuHV2B$+>=GZ&U=`Lv(xY^Y(6z4e4%<)Uzit$gM{u;&}6K
zE@8!9mO#Wp<dE3<J_X#tZI&H$xkqBuzM!nH%7&?h)ZUOLEJ4?IUd|7?jY!l{J}W*>
zKvie}fv6k;Xxu=iL=b4I0PlPZL=#zHm^0B6=Lg+TlH)BXOD1PN5p?e`keD$wYJt`y
zjP}*^ple<N9T0Q>$i6_@TD1E|4kFMl9LOcm-Uf8wFX-N>z(H8=zCGx6Fc1QjJ`cJN
zF-k~0N}$MnLH98e&~e-%Roc;uj}MyXyB0KnV%atVWF9jEwCweudA@6b?nViefQebq
z-I+mlSjoy0KFBp7Ohl6Kib!Dbout<)Bj^StiMi(Y=z~F)>mky`+S2`kto#Hr^fu1u
z*}-!abv_F}>6~k<WNr(ePgalk$VdnMcIpw_D+UYuTg1IsggKT>I(u(bGUiBl&>V2K
zN5X@jDtY<Q*Jr+8TKzoiN2y<g{os6Q{EV+V^$mVa$GosVX1Oqji0YpOb?LL<X_?@k
zHY8O>r6OM1zk?%;czUUge2geB@u0+@M-gRi_&roPI3}D#vDn~P<B!c9ImQV-5RPBn
zCdi<X!i09zr@`^2#B_jo*w)~5#$Y<v$@C)cwqQq@?f_J>=uVqv#}q$JtSGxsnmxk2
zNwa66G<${PNwar2%_Z}LeT-jeB77HH8I|UO#lhC5x+u+IHqF-51I&U>{jU3hb?7$L
zhAmng+h;88#qBVP|G+QIth-EMP7rRjp~0$QTp7fweZi_Fl^e^nYE?0f9qa_FOOz~;
zY&;iCBXQE=jIAX}lR8B5z>UGQGV*IfX5Z=v^PwZ1PL3n&4AU98#XT!aBtRf?q}WA|
zuCp;kuj)j&#cVRJ^9hk3r|5m!2sbXq$NY}SZ|J1r7<VjdrEVY>w*9%$c!f1*gi3u9
zjVzu~A{Qz*QT44vUwJW6gujcsqx&Py8$6h(1vEF27ym|YGI&tLd29#{L#pMDhU$EZ
z=t=Q3?nE^s-+5H%vee{O#NAW@;V$~JxqXSbS4njFJK_jQ#P`j2Zu^!b%r}9S+*b|c
zBP0+n#3i_y)sXTCHI|R2xb7+3QJct-bel+kRn$NQvddC3%b;Xe+hX)_{2R>loPUEH
zJti@~e8fKTz2u2F(%8qqFurdby!SEW8?iag2)<7hbBn}_JP`4+FW$%fh%`^)F1gUN
zUkxYX<v7v0Lwe>aU_^5VTcKXA<O?+tp#~}Ed<o8!UtlFMODmAzu5irP#C(Fnow+9?
zo<U#UQXPSB+B@QzcqM>;z`lrQsF-2#o?$Kn9(k#^Dq;qCSvne|>c2$HAUEbEjd$`Q
zW-!1t#CNuen33+s_iY|A<MwNeX0$epNP6$kh)I#+v?^}on9)weq*6vNdPX>65?M3$
zI`fH$NsN?98qM7JQUtfENFv`Ub`^wVnd{QyJ;~X)uwt-_aW??t6IRu*SDnO`J$Ni|
zd8RDZ2O`uGJfqkBkbi*b@bDx50FWseSPURdUUD@hGaxgZ^*g?d=pxddmb6QutamY4
z7ipJ@65Wz^L0?w%QIjItAiB=I7Sl+_nDsL&s;L~?n0f#rOFv#>wcMIS(N@T`JwBx5
zwN&1lI}x@6hQX5s49*w^4PhV~^%yXE4;cmxVPK-(AnH|VqG5Gk6DNrI*{R3d1n)&u
zGcu{JlBLjpBRGbod`qforu2!9tP@eKij79HYV<l71q`d1l8oo6#)ukfUMN!J9JeZ>
zJWO2{yNXe1og|gfvPV=#<xM`_Ar(eC5f!mBWiX-)?gkO2oQb$e(hSIVIw8E1F2ZLM
zdYZQACgd`9Z%X7QwP0J^!5nRWHo>4XpD8jf;ufvJKbs22+i;Z>Xn?0FE=z%Gc)H;9
zx#c3`7<WJ0g3t)uKm|JDnT5-Pdnd>Fcuhot5eU$I=;7@-5eZ>qnch1hA;lPr5h$UW
z1fms=xIxY1mRJq$B1%C-L)!$>cAmLf{OmfwtMUk>>rx7V8-6EHQ#1tH1PQc~Vi**Z
zZ0`kf3<m;PHN8L=!&>ZmxAFp5llSZupIsXUBL(c*&?OYGYXf=#J6+m0<5;mXgQ6{B
z*Cxda=u-c489uu#ir%mjUKTB?qrsV^%euo98$`R>c!9bi>hBZPr3Mrb)m0@36V;{q
zQ`RGS?R%IP$YAa>O3kjRz0d7M6G|on<G_U6h!=6oiBxhK{5MweE|EKV%X`#We_25t
zWRo%fXljQzN+SIxFOW_WJy(R2SYE|XB0Zc0y;Q_keiG^7B+e6gnk0JeD4YZvmxWq$
zl4wV%zvBgx$@=oU$hu@t);Kww+hlA3nZZlw<pp@J#alkZS5YbKx<EyxWHVu+Ql^+x
z7!yZ(fmjjgCap{*mZq?PC>Ci8h+>%!krG6s(!D^0h)(-Ne01j;QG^-u0g*~08id8$
zal+$}1vhodS|ayqxCFs-uhLvt=BrkE?iu0Lwk+H;R6juVX^^LRE{i<CI{7>f1LJuO
zWD33Ext|I&twcWuFOBku%_US_2&iY*<Y@G4&t;E*fW{-G_j-D}lTIt&+1iU(a|`JC
zTrxfL2|%<NYRg%nVYpm*z;Nc9?j~#%Vcg&yxcZCTlwA4}&wbPH>4F1^?uiM%(#&z=
zFPL%F>`FX5-NtiwAZzMfEM^dX;uFu^saEWsz??)gs`1=+!#t+K;}Pc3Tba!2kihe1
z|7VPu^ln{89oS2pc0k!2-HO53RJ0(q6?<L9z`@@F@OP4!E-SkY2YBvswJ>xF8#eQQ
znwq?osg-ROmeb~G_!XRsd(zjzax`aj@H&xeeH|>fCg%pfCGv~KuAe`B2()niUc4@U
zuY~h=2C`E~yOHJXLegsZqD_W5$a)X6lp&rwmzq_qBchr?$;yWY>!BZ_8NQ{axkI}3
zMfGzrx5PhP-6NKk=d*~gl10GaU%0E9mR9RbY$~yn9Zd>S@6u0k>hAEDJ$C{rtUE>u
z=4Qys!+sVgB-&z*Eate8MNAe|WRv|Y>0|G&Ci>D#{hJQhipyw>40Jb5!|+v}J3vi?
zDv@FT!ream0i}KRRbZc1z<x}aJ(HUkGGqFVm1zxO%lU=7Rk}U13#X{73#hChMRFCt
z?=<E6NV@0VO}bYGk#0z3yvp&@y_=p$)*IOx*K{#EmQJ}ouz%k-$*!2ZOj7x~g7ulw
zG5NbbwQ{UGR{&|TUdrJY<MiBKbZal1-$5<hQq9D&h-hJfj0mmQUV-7De>+`xgs6#k
z0vYeQ2w{d~T-}#8d`G%>!jvuIQIjHjS2MNlgl!z$$j6C$6g<k}L_>*6cn?maO;3*(
zDRV`p<Hl7Oi*fuO4HB(<k=?lW$b8gY6^PaaES4qQBER51tw0lTp5{WdQFq~LGcKF@
zjCVn$0ddq1xDSbYZMApNr;M)ZxY3n~+g0AX6bs$}!feh)*NxGD(sScPtZTo~ojlik
zSc3GT$y0O}Omfyq&n;n+WK(W6<I8)OX^oCw>G3L?<}cP_1hIMy`1^7DSWER*PtJGl
zW38TCtnl0_Y$CigUr38Vpe9ChNzPkM1&%(9<7lt><^u$bL!;Jt-(#*XPRS&;g}J;|
z+Au2u>@Dx0V9Nwc7fj!PfR5@Kwc9%+82bVG{(~F+2ks<?W4%|{kpL&o{BnrUJ4D&P
zlm^}_oY{)LjJv1+23|B0$890+m0R#6;5abqy+W&6^j4fEH22LXqF{k<kU4=%19)FM
zq9ftkOih~B{sw6eYM|f(cXhlq8v*O;i@!7&)xbbT`h*k1kZXiWZh{EIAhzOZ(4^oO
z0iC4?oTvky+klnzm#ZV`>YtIa7OpxPT5Ytvk5C6(Q-R;cGYL0AflYWgR8sUw85=u}
z-s;~8t;VfT;1Ztiao<RMnsMdsWwZY=*6b+sX!TFY9+foIwKa56J;poO2*2E&@!S>?
zkEL60&5Q86h&!zS6Umr(Tye75W7RcgmP;ohcNi`^;8MYJ+lb45$k&=7qGoM5hFL+3
z*8A6Bd@&SXOj;uX*dCN{5o8Ln&2ai^QID^P@O25Ce0BGW<z;jJg?uRvGgMKn_N>-T
z-1mOmO@&HANAsQTC(+DvV4JG|mcxCkeeAZ^upc%$yKnO~31|}oQPNNq$Jzg`r+GIY
z27=RM=1SfXTT^TCOxG6gI}IdOzZhj&9UbXRO*1dHd*9Krl=w+2TVJ&B_6zHkNQ<?f
z3J;d4;rmI!e7a-m^WOeUB?AsE#Qu=lmN>OO``arzcFb_B(JXjr<%l83XSP)$f$sY}
z_Si|FsZGEP5&Q&pq(QxfcoNuwfjU`+>5lsHstMg)&D(%M#P%Y>9BqB&IopJ8Fk(_Z
zop9}|8F4pATozVh2I2ZulX$ulsP-*p0&e^;IN3sib-m8sv#^et2FCYlo!4}-te<7*
znVJUvBHS#?`dK<)=OE#Ca9f9J0_CkJ>NtD8rXyitz<ZvSL?eqBrh+kSyc=tdEuoeS
z8|1K^x27>=Btwpk!>O&1A;;FhRma9+vxzV^rlAb|NX!8iKu*eorTleXWmRXU$GLX!
zp0uib%1FwxhxZi9OFSC;0Nz<flK5D1KPS%g0V2IO%A0CPH(JuErA;w|KNYRfEufM0
ztG%((NH57OjjSzWH8R#9BWPn>gKBUd%dlI$OYGgaPL##n-MxueUTS5+LvVw*I2<z`
zGq&CL;36#Fw5rewHIh*JDX%XH-E)Y9%w%?Lo=vDP19PdK2f^IfF-oYfq*Fq(aCyv5
z*X9|*2%4T|9>YRn76YQwja|u9<DRpQ+gqFb*Ai6rvv2Y`DEn()P#x)D^)idX8Ct|+
z<O#<5m0lHt@wrt{6-MsV1F);GaDfOT9OtpJUU}s1*$=tRU}s$gYI4sBn%y|wx*{a2
zfkYE+qRR8=5q&JFC*udRZ);!~Vc2X(pY$EOF}*3Bq}E?LsqJYcy}ufRI!cdBZz`+9
zpkue<UL|hk6X{KB_;FIoVRI=zy%|NWCAMNW;mXi+y&40%0}kiS^y|c?H=`$A_W{a!
zH60XeuGnn~&<+-Bj=KW`?DTE;RAnxmklo!x->=pNbVPPsWNLa#tp>c>8H#&xcPr2W
z&x5#Ja@q;y+XIzmvCa(LNS-2$?vfrd*%V1_G)(S7=+#tyauup=as}qT;pvSEFtsWK
zW+R4fVa!IV(dI)pi81MVa6P`Sc+Rw#4Ny`Tvw>LyQ0mn?r`IjOeBEN!HM+%5>UFJ7
zDUL?X(`y(^GD88`HfQ~fR`oS7dKE`8cF&5<v8t~bR{h7seTX%5qWtuw%=OV~!*d;3
zKqq>I^_t+QGML9+_j>8oOgaKsA-KmQ?k)L-+wk-%Vc^{sSjFT^z^y;0r-y-BFGT!w
z(~1f>{;%}3FmRRyrdc3)yYtiZR0BjHbswhYt^W$n@JJ2k?ML$QAIVH(7A{Mzzq%n)
zknd+TDO*y4qV)BrO=r@iEJb2FlbTPIsm@f&+mOg=R+;L|BTE#E8-%aE;xscUQJT7z
z(7H?ogVtJT6{z$jiXrWUZDlId#-C7xw^YuSxiv}gruJr+DQoZ_6n<HQZ*j^Q9Q@E~
z39Cec%DZW)GKp$<)hljTAORAo6dM*80cwKqd6^`Wi-khi9j7miC6%oz3dE@TNK#qE
z;28=q6Uro&R(=iUC5u@)RhV)8`aj?#l?Dg@fcx><zJ`crK21v0vep3DZ!B#-tdx`}
z1sj%cznG-=&@8mJg~qFt4D_*;NqjQJhu&qOQ5HIzJbgSX<6$GuTmq%O&*0&Lz~Z~i
zX93?RpP{)AqN3v1u^G)0mHc{gRBT*EGv$s|NWQU=(X6igvHeF4a59>!pEe1n7Rg}g
zlHD^=?WgYa8ktcGq3K~;!te4kMa7(r&5ggtJ>WdUxK?crs%ST`fmY65izTI1c@5fB
z)N%#>lI<I`$$-Q>78F&nK{FwxS)^vrv8KUq;|>iP3DDRA8YPDT({?n_V%Te4EWkc^
zDQl>8e}gKrhSw%qNEH^jO@b4uH>d(R{XGJ&cx2hPepPsFNnGv-XW6%8+kIYZEPX!@
zxTB?=*YpNS*J^~YDQ8Ow4I>DsxjCqtIQ3Hl?!DpCc<oP+qu(b6t&G@gCP4wWJ3^2}
zkUu#aaC2+{=)|v&oCBl)*{nA%6!QDQfZNF^6frjGF+6eZR2aE4k?5PKR~80bJuBcG
z6tXt!gxsz$vRQgOsLXws9&lSJbDNg?h^<moRDR}Ksns1|5Vr+fV<VgN>WVlvD~#Ao
zmc3AjWxfu$A;XdLhz2oK7%?QoEe<a@InT`xxD6BK^NIxA`iT(N_%q;E6Y=ZXkIQUM
z2)SJ?Zr%1N0W9R*`2<GKCeXleVqxUY=L2pUr(QdEC&p|hy}mNtlB+@ED~oP1=3m85
z6Dc-nu=#-%kvOJbaixhAyEMzrjbjQOcPCw3XU5l?IkRjITq$Q;8wT72Biy&Xa0!%c
zfp7`T$$xmr7cK#OvdM4e69G3)RGZ^`s<^^bag4>{Kk%vI>KUpt?%?}nTAnHLe5dGH
zqRrOe=8k^8JSv0o6}KGs4AREbv!_zb@eb!pIE>?(1#}CcpHV<Ri~5FIFNMkJpyD<`
zUUdKU<b9k!j4cBzS3v%$CI5pGHL0@)8TXOM*>2Qn8=HD9hUkw64x3DC|8wZKfy3gV
z{ZA4<OqQVrt5IIyTVcJSC)mLHmSL3H%L40L)Cm>?z?WJFKB07@Hv=4`5ziZg5d-v3
zgsvPM!yU4sePYSn;zb;{xF^v{6c%lrJ9@-uC$Ps5dT*%Zc|IYqhb0y>5Gd6sum|a!
z>W&a8gSUOeFU=e$pa+NfP@e@+j5LgTJ1g)G1!x}CPq`!T4&z0n!p<`;1zs>ks4QNI
z8}9^OFcO;=S^_Udf|n({n3RA%4_9L{cw5Rc7)QcMjE`$fW?_WtQEg42W1=VDBU1xZ
z591pjNJ>=O{q#<^yL7mv2YG4L(CuDHbp-#%tK9Bch6zc`d)n<PQPd`Ec?eP5uJpIG
zTflZ4bWK;FUV6(y!R{8!WTEqXx3lQfqixwFSZBdZA8<!6w?hHXPX+5xz;k>D+Xfk)
zCj@I(z;o<Kw{@83n~{RGwmcbj+&jx{Zg}Q9^&?0qQRV$++iD;+x6Iw*H&e1@8FbW7
zbiB8s-N?c7B8JS12)^LGs2NC0_H^EGn}`YB&zl3pzez#bw}0nm6{P*BVA^XTKaj1g
z>^3xM>xBS;8rl>QYss^2eZw)|sh3|M*6%$*s&9C^L&_rz%O!w0wPuh?xD7M{ehUjN
z-Vj$WnZSEyVD!xQZk@0gZ$&GuI#vwQ`rv?D(TIV`&n7K1xq9oxp6YJgipj(r^s~%<
zm#g8OTO)P^K1MhCE}jtsN~tf_tsG0lgJCgfF^abr^H^UN#_W<hnuQT!liVr=688hO
zk5H3*RGG@m32u_&RIG1(X6l3soQNWPnCzrmPEAI2VYcYy-e7Ve%-=2M4VWe-M;@C@
zcM}XBlAc0j2^I6!QZ|nXLfgXF7dEfYcZ&;A_O~_DvMDb3LZ^Y2A(#q3Nocp|x8;GP
zRkY$Pj1?cQ01bR`#^1#kDZtWA@wahrDZrXs@t<)2W)vQ~mx!h$u!t5cBG+lhodI*S
zU$Ad|*fx#Owc<87nCnCzt568#n2++qKTu8<M!9HFwwbMn0xtQ5P#Du1A-0wO2bZ&j
zQIdR=7XLuGSQsVUN8$KGA#tyUQK&Y>S_uWPKiLj4{mP>4)@g}>ECkV-P0e>oE=H{p
zMy(s>Fw%QxjyuHsf6g5&)$2>Z^|3hn9f|Gj(COy?IBtm;oTtFAcrtM5n3zHC3bGUN
zFTY<9I0x4;OgN6fC=_0QnA&T{+g=e7d7yo8^i_2%-q<5%c2xgKv9q8d_IZmKkZP5U
zb&C8wU%d>%t$h_f^YNR7^zHAG##&TgVjgam0(<bR!@Y!XS*#J%UuJ~+#jCD0f~>`W
zSkWx88`F#64851Nk`TR2iJ#T(?7AkxU_=lpi8en3XgB_T(%O!VFlx?HCkI#Ec#S}t
zU&6y2IzEX2hvMCMY%sn$5*SLdM`EY>NNgC~9dLG-a3TR!W5aSjf!Wmb77vgl?~U+e
zW3d!QOx_OeYj`clX8bF-moo`W*ZLu28>7@6d1&qi^c8irEVD~}r?_5)lu+9Wx1>nN
zW2}xFP~*co9(28q7yCMH{xodU!`a<ANwPghqO^ALn-o}s=QG?#%At_!DTl#wd{{RO
zmU?uET7(JFxS*dYaFwYVDILhv&IX_;Q=IE_6Ur$t9Zw}(MU%O%Hgh`%lDWiO=l`BJ
zh64>Sugx7nERyT+1;O%zkV%EtACOuSGkkoTv&CL>s%b(UTulWY!P6esTDcLjv6x^}
zz9$?8(_*sY>=SaSDPdeVH_J?JdZctCH*ZIQVsgW(c*4^PC^ygGmYCdNU9$hTb=mX(
zzo=-Q%%d^aDnYn_+V{+N8Zy$gBgeP7DdM#sMJlUttJ)F}yq}hR^y}~y%6nv{2In1c
zwsTs2E$8-usw^g_x;^Mi-rmM}=(PiZe}(&k^ZQ<Vmckp-H-W_W411^tX+H<xkGLO<
zdIKGg<~xn1;Wvj$-g%y9GfWh96o6`KVtF^!MXDyO$P+;yYNBJyyXi8QCM**sT)>@E
zpoim@qym+6nW(F60X66oL`7+-{vEacYJy6Z#i_!naVl9Em+qAJi6|9_RE`l*H!2ZB
zK&32MJ511$Kywf<5%<Dzu-D;lr~3WQRjRUy+Hdl}ualGz9mOQ{onIO0=?TV0`%MZ0
zISJ9M?|wx?Y5@60TW=*q$CVK4w<dyo5v=iIz3V^hvA>l^(%o>FqWcyNQb{bZ<b#Y9
z-d>UKG-j>X{33fPw@!2^%cHTaV!KOm&>KEG!=x`m`h1w_UIOn?Y_ER{hh~y3c%YP9
zpUYhyBW$<iZ@9w*UO(fwi7-w~P~aCls4wwW>QG~ru<c7lV?|Dr*f<jbUczAgK81fe
zf9J9=pzKuzV<ebn4)ZQ6f$m#GC$_>h2<K>;$&qZtDdl!HITG0}MFhVu<+dY7@3sw-
zPJ;9yC5#oCgzeKAnPxG(F{2qCoFR#akAd<%9LxgjdjX8*VVdDY8k5917>};n0>=&v
zR#e$JJdgP<H`#TQ*iEEaN?e9Hag*3hT#sAlwDFm06zBwxm7P_RJ|cK5%y=tEZ<hBN
zo1q{=Qb=VN7Y{St4a$3Arih>Tg;*hM|91IKRuX=rruSOX-MamIxI>cpH}n$)gpdqE
zcXwYb8t>nOWcv?=1ywDPsz%>^q$@bmqS>W*KbhKkWDrm+F7cQGxsH1STaGuxDKHsN
zbzJ$T1n38|zJiir^?QrCXL;mFs)}WJOofwm4B*Jv;cl3+H<WZZmM7(S!x#lf?S`jt
zdE6y1JPjH5R*pQ$_6MC&R=&<0I^O$c>uZdLRQ6$wP=o+jksW8*A`OwKv@_z#d?(Zk
z|9sVv_m5)<BPgF6<QN^gBi7>i5cdwJBi#{iIjC*K2W(=hj3JDmn9i1U&(H+FKgTbN
z8%~0MLIrr%FOVCq5_8$%T_2m^VpZ-CzfSkE?(Z~hCY@6BRi$V#j0B}p36xW6Z&4g0
zJ|GmrHKd}17#W&lsf@9Pl-GPu8Tk^xS^TCQ+5Mu}9EG3@PL+McX*~2jrACP7-;xB1
zJ4#3r{W{5I-GfFFA!5m;o(i*gjlfz7@i(52aXhDOZmr)@K@Co%SJuT+z1<opZ#oS8
zU%0Oo_y&(6e$ufDu#MQ^(-f-BR`!S20}VNBCa4fTxJ4%V@OF1dQ~rGjNQT#p1b#$_
zhxfyeHl)-k%*YjG6UxB;I?c+ySj-cx<Cf-sX}1vp?oSHEX(sCLh-L4qbgW6)Ytang
zZv0)9@*Zr^Yr$-$G&(A6h6MNFq3tfMV*)J(<LQWNLvL*;WrI#+{ghNQx)S()+#n5*
z%#cZ}p1?isciByjsnd(>l)~3&ybsn2Z(hM@s>xZ~MSq}6?*H#D8gp-QRr5)>nD^gu
zO?%liQ5-P*i1=eudLf#IAY2x1dJ@0c)!FkA`Dl(kmwq1itO8%+c?-8$Ll7Zf1(|yy
zDIvZ(3^+`{psO_Bn=qq8aFmZFDJ<Q3dmdI6=1D=*!8wBaSwoIyMjGI+{6F)PdxVAz
z>~gfO+%nqpw%Yq#G&_vQs${aQvG0Mo`A&;wj>9QBO6tQ8DBgjOk<>Zd?-bX-x%yd7
zc^S;jCKK=9=$2y@!(b+*IBrrIYGtI}AQ?E95c@_+AqAG>>5IGXES5KL4f{-$DfEHw
z<EcYpU4*k2_(QnyjzHf4Y7U5N)$TwB0ZFrP(<Q<&GSjk`bP0HNL9=6@`*K>25#(Cz
zION1g2cCy$JbrFRxdX4jYcp=6GC}KG>|(ytY9xNMPPLcOU?8q2aEc^J57Wj~pb6w!
zRa^!URNthN_Yx>w3Q1Zqu~DnI_hzN(w~FdRTS)1xR)EK?;Qzzkn?T7`)%(Bubaw_4
zz(62@5J&=nFeT~kjLhkDcSu8b(xj6Binpk)s_rh*HKeLK1A{~XMX%QzF32E*sNm%~
zfe;X<D_&7>1}`coC@Lr*N~>H@#`pRD_C8hB!TW#f|Nig4)_<+{SnSRj_SwU4p3bQU
zJGg6~Zl=_2qI^)jLqFKS*B4;i)w_bc@y|D0G0hJkvGeL5x&J6jGwt%=PXIRb6CJn@
zZNkBv%&?_JuN}dK6KPA|!Nt8IP5!*_$mI8EX;(~a)0$ZqGj=t*hJEF+dSOkza*!jw
znS?|aQE&X&s5@;rvtB4-_%`ybkvF`f1J9(L@ec04yr2I5UVA(Z$5h{m`}Y1Ck5XqY
zVcP3EY~MaoO2S`Chf8f*%*0Pswa010dfb*`Q|l)TPZM?BU{n7WQ}Zi@(SaAx3Oe~d
z2cM-u8T?uy2j8K6OD9ir@Hp*ftnKWt>(kk%$<Ee7L=pbAZ)_jkcv7=+<GWgpzjk_4
zxb{n3D>n#Gr;BUkw+c``qzt_@zA@^$&jJmGl<$5NdTGM6JMa&*r*#5iX6#SHdT3}m
zngi7Sj0HOB;9vty-)C()!;@7LOf`f3&7iirq3JvhcGB=M8k%5*8Q0UUo9P<%|M95n
zIZ`T}ze<CR`zU(iJ71yS{s`<Dc*ZlZfg1Mz9S2x6GrqtEdXOEa%Nnkj;&&k0g3sdt
zSJ;3n{D6Y#caiXZK-}#a_s0Vw#%4UM0lWX30oz&bR_=d>p$=e}Hf;v;V#brSU+Bp@
zT*G7$i4lKRivSbdUvbf2K+|_q-0=pe6qr>|dc1Uhj~Oqg$0fCA!Nw4*R1}VSE_DqD
zK2^I1ukqE9*-o52X;1Fo&x)QBHGL18xM9*Etmg&Vv!Gzoau%b1`F6&U`gdK!6^mK;
zq=btOLIfN%5l)|UjHX$Guax^&rtzd828NBHaK4?e*m#EwGaUQEbEB^JM5esfraT+I
zdEpgCI*>M%lhN@UKo18kfKx~Jbrezb-e23h(#cx+cfQ8;M>y`aVT9Q4Jj_$D;6d<X
z!*_nqL5ViZa=-Ie4j`I?KFD&v^NS-{?imEoYd;kLPw#q><j6z@G++7q8{wt9k;Vtz
zO8b~j_Tu0_X!jl#I)fk{;jfz#0{-d(ksl=QUk}fmKpQ7_PJ{P~l=ur?8-Cf~8oz&H
z7?${AG*iWL_6&Mr<}5*U9i-%6_BFr52Qswz##OwYuujsssLlVtA_=p24C{X8(Tua|
z7!FRQwdn*VnVF#_WH#u@hj`LW_~$vtn%SV+m(VVXAN&|6*Xu+>ewFscpmuo>wL9UC
zIW4H&@Bdj{uH^QQXb-_hJ}9#e>rkHl5A;dJ|6$w$AEFOVuxykZ{Z@;1T|1M-vBypf
z+&+YeHz<Gb@66UoaAf9hXt<J%ol9JF@I)Fh-p0-z4%qa;`o2%+SsZlH+7Wb}ZQ`Q7
zZ5(}wMSV<sP>zc`v;qLQYG1BWEMt))q=2^!q17+o9Iqc`4W{^h2pL3Qc$tA2;NUDL
z_cn1r5bfY=b@Db2Ai;wlz#%q&mPZAzuT6{IDbG)%ZP<^|+=TB=vmZM8;d|rfFwhTv
zDU5i?0WLa3W`pKBWFhSsK1Uec3t%Sv4E=mB;`5NxXe(L&hGV%f)<>0WQ0dQ#xyR38
zvKe$*?xHUqtlrSzy(*Sui&S9%=4&u>68F*V9FfvPdjh{}E@W)AZBVY=2L$vIAg1UQ
z!^R)JLM-i?3!%}(YpPQhP$!-Q<W%1C?hCD4FYy%B5mH?KhZLCV!yg!R=L)+kY^s~V
z+lQ}VnyY9Z)X66}_!4cGPHyMmQQD87T6^+Xo^%tR<So|t!56tb8XlZA*+sJ^ga>Eg
zyv#ZTTWjKzZ2H`e>KgwVud%4X!0_GRWVnK=nyocG{4=0_H&CB-3T+7|_kiJ9n`whO
z!K#?WB4=Hv6XfKqAJg~_Omq*lhf^BOdL2C5m=g1D(tk{g{tXZR;SBdQ?(D;8v-C9M
z&Sw7EOLfAyv(KRQ=!9`+<897Pv;Il{F<XPPiQ$LBCJ^hN4ZGNWl~X}$L(j3JE=O9A
z{tr)!?g2p$e*h8Ka~z{wM7uyNFnlv5LU(cNCd1ZMcI|O*ojv?jT{E^Yi>G1YRT4nM
z&+Da1yVb;$;dTCrI|R9gH77AC3y3V7{-bX(;Tni?HrSjESvRa%&jAkD>>uzkg*E5u
zk3{hrZ1g$E$vKnxGLMb%l;giizdK|$fcg<mpA9pog)Ya0VdlWybGmeLE(bXp*kCvg
z{)zT}0qS?S_G{h;pz`}wv5f_16nTM6HTILo6ub8K>}Ov&u95v|nS1CWs>%<gD@ps^
zudaq%*=rBRfBssG{|-^ffBt~kdv-AMr)W3n<by6c6pYS!iEZ{=%Qky%;Yxz0&)lNf
zqeJO`==pr|qvuPk<t0Yhs5@WdD)RQwYjpB$4ql<X#9E7(ple_$L(BNVE1QO{Y-+r9
z+&qW*-_V2kKY0RUd|Z$H)J2D(JrCo<gbh8v;-G_eGAJ)z&KUnfw=4WOf<f^ST?^;B
zP&oGQO*CO1DtmR~t$XO-4h-)q9Ey)~JX3IZq$R-NunH6A0~`(`baB|V28Y?K=NA3B
z1H2Sp1ijxHo7bfJ-!`T!=XACPs88QT*S|7_1w<Zsj!{p)pAj*(4kNzQaQfpMJV`?m
zHk|$p2e8#)xKs_NBM#<*gt?BBZiMk%cyTU<c*E%riP$$!;qpwDeKWf^`KOD>29>Hk
zvP&STMAq>fQ4rT2);0SqvgFzYb7hNczEBrOOoNDFEThK48xP{+9-Fmr^9g=}{BAtx
zJQi>`Z5Awf&=wBbX{YGqJsg}#>+?S@;Yvv8QT;_`9rZ4%wSW9!cFu=`qPe*Ba|aLO
zq{})qhHiBq4&pHa#rAO1-qjBW`Ew@Y!$HwpP&Bv1=qmizQcJ6KGWo!xw!n5j{u%1~
zp!49P$p_BXQuXIOAgy#RpvV{)^a(N(?@c@h<Vx__u3nz~qi&b|HIRK9#_CTRxz;#o
z>g6J^(y{6oItj&(9XRSLiy6fr_0!AP=wnB-yh~^pAq|f$<p8FbixJZB*h&uIlDS_5
ztB<{%gP+qLyO)^rV{0yQ?ff&saA}5{>l%OBs9u8fZ)r~j;8wN?xcmjUdmRmMe}e7e
zf%_-Ay>OTZE*$`_b*lp|=KwcctpeAd$H0A!Q3$V@kf{u@UFkiuR9S&-_i|~_3V$i*
z<3-lSpO)c}$Bxz?G=S~$aE!&n$LVAZ2m8_X1)Pt){UrWqi}tXtYyqg1omIpf6A*Lp
zj<A4pgakvtMO*>k51u>fE|9LcRBZbwxx$7AF^vyDf;Jm~J$Nw(n2U$YCwUNAb~uvj
z@OR<@>7!N`z;X7M{Q6#vrvko5#rUGl!y)j)hYs`LqaWboudDmv6P*LTaJ33Pe;&Yh
z_;yB7AVN-;t<~&}3(VfwtMPci^(curzvfHn`yQb1;Kll*FV|e~*8qyfNsX&xSacgM
zF1Rnog%@DaMSG39_lOR52ogU-OGX!=Zx8=44X{KPL5GL$qJ2&$(Ba_^)4s10=<x7o
zX-}i>FA|2{r19~w=%TY-bOhGN5qqG~Zu>G5T!l&MCjaM*)r8$2TgS?8`!2T+pzU{>
zhZSAM(dFsA&s~mrbN8$E5XA;uY2FiKukoHebrJw{#C#s5Y=EjxrmN8TD;&e4KU2Qw
zB81Tqh>s(dhR{EEL4@^bnsGX8`q-@qBZdBt;Q1p!vW5PSU=v4lJ7Etjzvp|xGJoNb
z8NKpxh2SH!w=>lR;loXht1fl?!8r&HSE~s2=kfZ}jMC3zj@dj8pb{PutxTD*Qej`A
z;fT$&oK8T)5$~m4x|oxt`m-{C{fMi${4cY)a<+`(i$Lt;Kk0M?QlQv87FTBkO4xp)
zrp3}KoyYyZa%_s_acq&ObRlDXoOTlflrGnw4^MN^g(p}~;qdht(WL|he@^%{x=_9E
zrro8JA_xCLyH6+D-w$ablnc+|;0YQ|N_1g&e8TE3*^g!J1!IoeXUC3*wy?m)Nh9*q
z?e_?XcI~hTy%u^&v*4q@iqOAnPc2Pf%r0Hdcu?UHm^&<XCYZ!cq93^)>)kXX!W?S+
zxk^I}<1f<r&jI7}np`xmLGKyqo3EFcKgwVDdPa05F5S;rzeUjJp*`n80~SG_hr2QF
zpz2dd7oI(brzXybKFm{(t1#lu-_Qv%orlflpE`k)<LoJ|kZh~{wWqGrQ^%<zG&%2d
zT35V6eExZ7lc%5_l>QabB!BIx8>Z+9br_(f@%!!^8b5qxQ|?M>Brts8_)&M+0UVoN
z`2`u%=)&oA1#j~z!0W<8^=H9B{Q0i+1jBotGNS;3pV$LrAuXeM*U<o8bRmGNonU9o
z`y}la-FcjYduY4B>1B&|$LY8|-Q>-87*=;!40DH3y)6z{^N1kcF(Zqs<<|!p&AU%0
zhjIWJ&67uQp+F;`{wi=aj{!n^xM{`+uMR$OK@7n^((U;ef^Ts0nzXzq<1#3dO|F?i
z!F1yjcZLV}IkxD?326I0*Mz5zJdi8Xo!<BpX<W8LfafoqKI460>j_0PgySRU(hk+h
zPdI3%Er6%}L&TYQ=u#e<GUNRK)IYF-J}1WyAaaiErUUlU-k%zEmw$!h<r2lej9AIu
z-{AH?(9Y2bkT|kH%jx9j99&9!4<qgU1rGk1_D?~-Uv7elzs^mZ#3y<hfAOOrzmEJU
z-EIu>iw@(Mb2s_L`Go4#s{Hc3m|4ctk)NX7#^|zpE`QM5IKTLvsaR|r?4--_l!Bdv
zzhrjOj-qXI`a<r$$fFhueR3M>pWeoBU#1~Q8mISg@HFk`Og0_+<|u+eM^T(=Qu+P+
z2@(8tn1vA`sK#G@Hf-%EME+4r!`A45XtsWJqM36x9IjTk=Fh{{j%sI=Hf;^5ptZh#
zu4q($-ISxE&!LK*YzS*U3W0G{cUbdLgPf$$LDSz38@wMI&-RaZN;usE;33n&?_Xd4
z>d#fisk_pn?jx@;#UsM~lfQu8cWq%jIQ6LWb#e&@AEJF=HYXpN$seA$en?BxU$<gf
zL*Je}d7tiMt?~p?YS)K|R)V7Ii~en-C-vhmvROOs6Xvc*cz)qop85=zeUR0B-uQdB
z*Rz)DOG+J&7`Cs6#J`BBd6E~S&7vcka(%a~3K>=l&Kz~0Ucq{dhJUqErDO{-4Og5+
zsC2=_OaZPIU}+tNHG9-YS;D8!@+<q*5n*LrpOK0DX>3NHBCh@3R>MT9)RAgQQs1AG
zxHY}?tLGcJlwmkD(NVYSMB~GZM?Hifwaz;PrqA)$opVq_`}nxef6?bp;pwB107s8w
zL7xLAgS-LvEAxMRK>2g`vdBxh_S3M))2;%gmV6mfE`SAp?ooA#Zv!f=^p!7hW0d3j
zh1FkUV@I!GRUAEs#uq4~SAN@Y1?Fb-%3rw9Nh9kYz3PzYuRIn_<1f$OzJll7l*9L(
zpihVV`d1p`F}j{ZdwV?K5GMnOuPKK!z$;X;`60J+c{B68>VQ0Wy8TEkl1<)@llQ<k
z8SZ8wv6ku4t%@Vl!!2F_HCLOBPTY+2*6&~FqlJoG%aUi)w_jVg>C+{~ba5j2q{TMw
zY){*V^2LKYAf8=<!m|&HI2Y>AiTZOY``o1-PuV^fPix4Jr`H48&0>-8!p19hA`9ux
zZf>MqsjH{5YCU<qJxQQv$~n^-NQX2&t<O|XIY)i|jo-<&C-32@2Wa2a)o<{m{(Y$N
zimUDJ6WsYVjhB|AUC+4am=@aM%Q*QjMjGMm*<;oJ6|DmR-Swa%hOY59`ud};i4-u_
zb?b57`Gj_!vY&eKU%lRDF7X!r#$SBA3q^W2T=x_qmYD0XSe1jY*yLTWF~PA)YTiAO
z1zbYU`&q!<dxQn-LIi37Prb8l0o!c>z{oG)qT0}Y0e*qI)`zL7fQ_g2lkP$<-g_+Q
z+jY5ysNJrvUuds@BldX*E3_RvM?Z;^(<=bMel=FnvO%B0`}_gE*VFsNy`%{mMgY!`
z@5a}UM&0L+4c(T--EPoer&N1cI`H!=yr_D5BfW%I^&uJ*e$U|Z*aD~h8SHPb4HCzQ
zns2l6!Ostec|`NJXrx{|hOh%5w44uyZoHrDyWMBq?B2~`5l`46t~S(HQ3voo|A;!&
z-=vF`M+u!k-%YvBHU73u^t1swWWHUDDQy1sXddGdlL^ei#JyUHU7f-x5;Vj%qq`3n
zb$1>qD`v{SPHVUfl7FUYs!l{BM+B0;dl5AJPTD)#xw?WUBru_l-+ooy+L#DxzZ>i0
z7+j2FHnEU9hr&Yc8nY0@#+^jXddhB_Yg~a1y6b*vVEzKGY0;ZgP|S)3?s~{BqvG6=
z(;6Bvk$%^Qu<yB+;XgpTTqnTxn5$`5>0}27*b&FvhTj^ZV%qhK^-;0Ox6FeFoO|=t
z`&>0PHfe>sO;NaR)tzbOSo?}&{f<`>4P{;W9Xxkux5w7+{*lXpWEv;!AQCea*%wyt
zyn!0StvvTp?U=-!9=a`DGbl_v$FA)P*B-Eb@3d>rvuIfTcQ3>Nys%X8;!H+HEkfd5
z4iWjAu&!?Hh46G2-@d;K#sX{iOyifnk#|P{k*;3=4fc7AGPGUKaquMVaUquwc0<=o
zE;^R6k9h^R`ZUs3yI@1^eS6lG5W%xlKQ^2;i!NBM$4=J?BzNplw0Sx?hJz)vMH=qg
z40jroaqLRk$qc7}`Q7UkFrV_PS??1~eTjz@FFzKMaqK$fOzr|)H$4c9?q*L4oIfLV
zh^r=e+)b=WWev}~tZNoHx0s+@Il<xX#bYAp;=k^*MAeEt7@#iHpgat2JXWW8^gX+p
zi$TstT~*}WwWl+rKbzpT4QEKd3%l#%zX2SQfOdHW9M#CDIJC;UR)jIH=dNcq6<LTw
zb@!|=<el-eXW7a~By`V=FAcFefWgPpF~MZ=th$cfrMdc))Z6d7fZHm_`dG}tW8bM0
zyqROKqg|nsA8_yu+Lyt=s-JQ20_`_CdCEoeXVdV~8di;Ra3*bo!g{OjiR6deF`bES
zmMNpZyiRs1(Uj-<F+cfwNi?7N=4aGTUooKv=CAc@+UL{-^dqukUox2wZsu<`lwAdw
z9_ALlS*S!YfbF2}W7R#_tp>8Y@AoR@IjK93*l`tCCgAOVBE0(+$<8gl?^C|-?~!7E
zw=bf1P;v}l-!n7xU1<0rauLS6fj+j54+{AAoEWPk(Ukotc{jTU?q9!p)ctfeaClyD
ze|{>^{rWBJ?R_)_O~3wm4!%gkCT#fnzjN?=8unAe*T3tc1&F}~Q;9Zy{ZZge96efq
zwYZ@39b9=@>-*_?;@}%V=$F|`w3=UPb3a8jDh&R6)q@dRBgtrX{l$ElkGoHL%6onf
z?rH<Q=dfU70{tYD$t#u2tQN3_N8O`M%zLQJ#GQB<uinNtnk=dM+V`M>ohMSM(T=Zz
zPXJTS+&sPMvxtf3_2$HD6Ma7Uc~O3oiuwgXRc!&U*QQ{K@2&Oeg7k$(e@$?5GcTUr
zbSp;WYxg1pW?symcb>?ev$PZ)Z~T&V)O0UTZ@L2^@CQ|A&3u`TH)8ZGApW%AJ;3co
zecwV0AyFcw`@90)+VPqTV;S8Tl<sRCbcY@PFo7;{9fa2TI_ly^<f8u(53TbPOBk`J
z-5xw{dXsW?fA|v9A3P3le4-s)_KEje2RNAy4J`zQqfcDLlF%><uGR@Yg+I~SkVQ(m
z|KSoQZor;V4c^T=io(=ucZ7R4>fXd1T19!eLGLTNquuOz@tbeHVQ_s?pwj|xNsdMn
zAvNaTXL=JNwDE<1(gOA7$1V8f3+|wO9HD#5g<L&}x!=&8cXRbw+HWNJSKmJx?Xgp0
z`{1tW=Dz-MGZeWRyJ6v;w22y!*iXaiCpkNWHk0fYg<0-kK3Da}gzo%-tBCc5^Y~%G
zJ>bqx(h>Gkm+T<Y8(y6s(giF&=dOmYMrZMKaeDOS$&B!Y#avjZS&A%f^aRFHTXqsB
zXOBjcI`ow85Or912Cav4G{r6d%-Fj)Zeoz1XkUPSAtuAZw21k20Qd3V@N2`9kn@ti
ze|$+$geK0^FZ4x|&h?jHm>OR`(V#9-5KY3e_yGZs7Z&+-!#2^PHdQK6U8eg_g@=2D
zuL^QNztdrNUJOs>b#03z1Pm8H3OFXS+#ejTn{XgEw^nZ&O0S0IUVp~W@viAjyxrOO
z*9lP4)u*zri)nhR{@{19+pE`e_F>u;I!SWyY1+R6n5**)@gji1`+ngC-9=C>{2J}6
zviYw5F(Z5tVa$)&L{}s57oq_ce$Q^b%B=@n<BO0|bTztsA^LwI;wZXi0tbJky`+=<
zT+}?C)}WI)92`XBR~4gc7H|M1G#{=L(Aa!3?L?id<^bcac{NM8W&`v5#Wnuro4T`+
zs}O7RpiWX8ASRnXrjsJC6&_8S3&UM=k*<7zvvssyom|5KT-fpsSmBy)j7AOnv1xb6
z-06+?5_|j8V&HZSZdA*8v;m#`n1grG&IgEx%+(O+x)zAP<wGE~VF&~xGXJDRbHn+D
zor!OXgu!s2>GL&%c5n+LG`q$>^Xm@LHMcSXQLL8hbaE#L2>F(~HRLWHq{#D6_A`(d
z7@b^bzqs9i)V4d8h6;D~6a_U6hdci!+6s53*quwmop03L+0X7=74FClh@UyY?%Wpc
zm=zR8o2gBa+^E@s8}2pgyJ2uNYUGg3ojkqq4+nsv{Q=w!SFoSQXg}7;r5p%uzt_oj
z7qznM)<_^dnWGzS;tY}8I$bBwWb1NTD=@o(fe+aT%r4QLuX1%Qtw+3k!*^Maw8N3y
z{<v;G#I18^Z`UTqaaq9n^H;RJ_ZS$dJJp>Plm>Nh=NI*N;H##6M&E3=9z=rdS?Wsa
zo)rL)0INFcuZqo^S%ol%w7VWI|I1P8aIsxO&PLZW*<VhoyEfz{!C!J*lXTfITGcp|
z0!YVB-0E8AzRMjtUgI{x+t=n8<5rC!5ul$^mLS;UO{rql5Ag2_OY1j&=qnc^AcU=^
zbLz0Q@yPl)P>4<+n6s~H`@ejSrt#m7W|lw7###j&k&CSZ(izvhP8S#pT=Y+G+%*es
z{|Jb@b|xKg99t3o(Y0^m;2&x46IT1d$F(qZD<s}}BZ$0qDOZW#y(mOp%Y{~~%hua;
zf^u!Wn|7B@5Z0~Vp*^URf8gLL+OOG6gu3%8VLQr|3k^WE<6vOHPYnPHLI(!$=qz1k
zqI(4_Bi?9*N0LTKyXfNw`a8yySHGKeUFaHLCYcgljcw5SU$npI<VqJUVq>judQiDH
z?nR;KFY`=qeD6Hh_+JMGP%WBH-~F93S6AP}fFIWLY<<yTv^hHY3<n6hMaSv{R$p`i
zZJAEK#lhLMGXeAsls&%*2QkFwr#D_nT=?Z(EaApx?hnu~lOw|3(IU*lMVIJ=hZfyN
zQwHP4JvjI_?VFt3fGA!BK`xT#b;#$L=|;5UqL*pf-HoTYXz>)9B>IhKazK`6@liSf
zM2mHIg-$kea51f{6Xf9HyJ)xT1Y9qMzZd@$WZrl!b9~=5zC<N$bmMhgeUA2wPCm&2
zAYJ?yOqMrydcJz8R~XoCgzQ5)Z{Fg0@}*+^9pTm3$h|4-!_6;!p#DB&;NHAH=E!(F
zJ+%!a0h-%;4W5XtLw-KJ;oEo`FCPkQZqK=BNdt>|Lnr9LB?r*<6QF*M+PHoDVfJX`
z&c#o$zkC9xTj@-D@>n__9Y2YrTC$Xl-p=$(+G%Y@hwjbAe#I|Co$+2T5QVxod;MNt
zKBxYUR!yc#_tW9YE%o>94km^q*UQ9gYGdk2Y&EFcA-KJC@~{x~8wCAbmU^v3PB1<s
z6NehWsfiG)3_^tPuZa)@Oyh^{y%?Rwe=>DP>x9V$RGHwr^B-Uc+z$Zv?DVE@U`)R9
zX#lfx7&x3w)2Fi!c^2sG6m(FJOG-TT;aBacCg*02pWc+0z48T~x_U3}-%PuKr)G`k
zsjFvjb`R}Noy_9^vtx;z_cz#!#OhxOVarF;?#BbSYLH^%`dHkJ<C_}9&Wzi4zL$~r
zW0eP^ZFc@6AMe$_*ZMfp&Rboy6slM%%W)?vnU5YsOJx`D{2B*G&<@qf_c&NZTRDSu
zT+Sy`-;RtPe}{{%d%-pSjj|0Ew5MeB6{cs$e<j@Mv^xjb9n<II?*|qX4FB!yunCRx
zxZl{{3{hc(=fkyktKSv=S=d{;=5HUiYc|t_J;OD#0(`#-M>_Lv{-!{<bj`-^#egyI
zX2xhV;Y8ks8PD7ko4sE8+w*bXr=))PrORmNa{z6;rot}tTLa<~G0=7@dKzCPGK4hP
z`7@x5czoqO0@u$0)p@jj4fSXkiZp)wP{qXdMC80pvZ?OQo<i%yzv@oi)uKHxey3mT
zE7x($Ha2WTaIIkKdwUwa<xa*RPtBSZO>e?_Y<%q?5!fOYcguY&?1Qw+S@td8=PFkB
zE3$uY`4Lwk#-%rj&Tr+{*7s<V0-V(|J>s+Mjju{{+`11PZl`@#1NQLPt%q=inX;6!
zrs&pVICzNm1Gc`$6u;hAyR|s9CQyWKb!RsGo?*xV-SLhAQt=Xga()j`^Xk^H8+Bbx
z*C)AlMXf8MPIuP(9}j<3Q6KXmG<+Kx%l~+Iv<KWF_vBT%{b3ewuWqk;6F_^99p2T5
zyIg2O;(}@y=g=azJW0n(0k2*wy5(th_yp}SUBE9jHU!G8Ok~#0YK59!8)hxP(*5T>
zAYNM+-?zZS>l1xA?zI2~g(_Yj@}Y{?l#*(k)I<=&LKWNPjk(t$e+BDa`-FC6bSd9F
zPW!!K)EwqGlfZ<e*sHH;@_6DSEweN9Lp-c0+&@tvnAfKt<|4a<RmTnfy8RCN%3i>D
z@^T)&lrC_$?d;5l*qMDxb%@O$Pm1dL(P-KbPu!-V-Z&K+dG)h^24-DKMRUaOTt<A6
zFIsjKa_7}Yyxe*9NiJ-lt$ha<ABbl1SK~aW`SsVW*mpd_z448O8sY(ZqQ91HipN8=
zF54PrTJ~N}-ZNW+gBlz8damd_8h$G<ng%b~U*{(7JAtobH@^M{9Cv;zFj|J9UUmx<
zFzq!iD5lUjX~IKBThku($+S0!$dE=W0b@&}HI-e9k`mFh7d>F!I6MI6>}Ld+=@WtE
z4sqo)#N-3t<R&{NE@$s!xgWz$<C$Unx4z2J++V9*$7{^{7E{?oCZFsd&Lnb~TtAu6
z>!PVQ?NfI-ThLVvaL^}dH(VF(dDEn!WIoL!EyvNl?sPe)$KAB7oJsa)3+4Gk#bR>)
zme%>p=MVM|ELz;WxVd?LI$Nr2pWoWt%Ky!a6Rj<amNy?azu3~;?6fpIWtxsgp7zbT
zIr>E==cYV>$#=pRMx#&M@gS%F`7YY!U~K*ldRIIBoK7cRH5z@=UHad5dVx+SojV%k
z&$=I3_4h~Gr%|0=8I5k6@>e2A|4paqbUNkm(Wr3qZHMmODSFl~e4k?`a_gCc$<=?0
zgUd&w9iYOU<DwP!MiVJn`}}|I(KIfbM;tZj?1>0ocYJ_;qW<|vG?9%w@}($Z6h8DG
zv6%Dr^gSQ-XXBy<dj8YgsDZ~5zr>{d&tL6$n?R$(C*M7dL%ePG$Yq!6>Pj76dYP^s
z5jAP>t0L0u?vW4rslLXCS~&akBp&|h#D?hi9JoKwZ~j6~W^i)8n><|D>NuONvnDsb
zA$oBS7P~i}z-fqHLPWd&7&lB9JbGCV?<CgY9=Wg~`YQvwKkTKRd<<*uucL-3Lf+08
zB+(7}L{9_XU(`d<vj3p{`;;xZG*g#ChOS{`1UA`+VGe58vc)w_^@przzsVXKFf9Cd
zLn5IIY%^RKPXt0|PmD_>1{hK&?)rTuhRZ(`Bt9B0Uq;x<J@S$GtjV#WM{Wr{Z*z=I
z*4sw=7Y5<K@o#u!@4J$oKb}>G>~|l_UD&g2Lz74LRFCW?kL)Io>?V)wCXehUkL)Io
zY~S-ye>SekBfH5Xo2|yk)?q+4hcU8s7?9oMk=^8x?Wg)$lSekOnL1>H%Q|H5KNi`0
z$H<;kL-wS4WCPb4vZo9F2Z)!War+9{$XgE*<hE;SeOulsE+o-O)MD9h-j;TRA9cR`
z1lqC#)rm)zbI~olI~up&ByikBcy0MRXlLl8n*)5L<(KHBgM%N^z6_-<yc@=7dZ+#j
z=}(yjG+~Ax$D_xcs+0F~fS+~T$8>U)i;kbf$)9zCPkB6c`|<yvlTRUfFiKbvCwFr&
zi#Am!_i>P*y+bG8<p4H5p{SD|bHMN?;D)=VC*gpm-?NW%*~W>)=#SUcQ3q2uz0N}a
z%q4yCa2!_g3AfO0(#hT&+)w*=onWD#@LSsBIynL{hfjH;k}l&~Il$vR@d%xq%)wgP
z={h-$18Duke-H<bKX^2<f>F)?CFUQ0s4kB?b2Pf-9G10$%MWvT+&Nm-xNkVu4D~b~
ztxFH;4{v3;aZfTFfI9I5I(eRhf2Waka^ui=Cq7P7xA8#h#Mfzm(+Mj*X+PSYHgJ}K
zA&bT<^-!M6EcvA6I>Aah>Fu=jI(aV#7t@3^{nW-uyJ(-%$xR&mn)awpK+VZ}(#Goq
z59Z{fX@|NgOv@+#nor>J_+~D3(S-8xAo1ir+8Jze{3^Pf4i0AN&gooD(S*zKIO-=O
zgHGnF5pKM$B8*NZvF64XICz=%2c29X1wcDUC&UU)0rscpC+5aq$pKwY$=F(MW}<>%
z{ydO&R2RBO-iR81Lq|uEz8nAQUU>&l-1U2<-J$kxspuByMq@tj8lN^1*O<oQ`>j7L
zl}IK-@X_9lY3#3OBxo;dOt)}|Y~?Vot4Jdq>Z-&Yhpik+h+o#YnGVnT$2Qa9S^pSm
zgXjFkega|7A*|`Tb*pQ-%jQM>G`-${jQe)e3}mFc%yDJhnYuEg=?J@`<NfQ7r_>!!
zt{p$}x4PpO!*OFG!4EX~actzQpR2Vt*EjrJr-MFsnV+k6Jf*32Jh^tvT-EE3ycmxC
zT>iS|8h@}Kxwmc?wc{yGwd2XPV@9rCXJmiuNA|}y@<P9hvfsr%jX5e=_@s<Ln<Hg$
zg*4`6TEfkK2{+Vju68`7sdhZMcFYp0*I9x;_Dk@`ZlArQrW-}>&rdSlXD9CZnG?h1
zUm)<&eg5)gPG=v9&zhW+*)5?bfg$NWKibDw^&#5Q^|T3xdkr}C54&r?_brA(o^esf
z(Eq13;MK`!oP?A#pa+R*z_T%Vf(ASV@pZ~4-?k;zfGZIBE2bT|)kWhb!s$|qE0)rl
z;ol~bZ|_^dMc4e^Xxt=IipZiBNU9a@kRHTKTX7@pN}X)t;7_#2kUP!qMGQ4vsz3j%
zKObRDO}Ap8psd>7t&=-AK*_a{`*lrU<};a-Igzw$`W7c2q4AoRYkJs4?T67MJDU*v
z?TcxxI{6(3?X;70@@EeEY3JzVH4c!H?WBv`xCxNtxP942k!?sR;n%xy(@jat<?`cP
zzE8S}C&wMdQ=g%k23W$uy|i7r)5gK~Xb<Y7hl5|yp3n(_vG!MJQt0C{P~dFZOr31z
zfI&JIYO?ViqY*{PH4VtG{%C3M8jaq6R=E6(mUfBh%jbaMo#?MKG|&qSvYxh1C$DnA
zRy)q+Wc&mUDl{1b<EL@}d^;}F3C2Um%``!N{4pGSiDs&W*ht3@Y2UVqb~4ddnaKA4
zVJ`oX_B;=b`xpnP+m4rXg4^4P3rf(7lP_`r*>q0R$u~K`2ktyZC!nhn4c93pIPM7!
z&ZWKGl+#8g`YD&C^g7>7yGSSOr1L7;PT3AQy8I3VuF{<nSO1kJi)H+~IQR<9Ht+!s
zeoOnA?p*DnRR_`b(aFa-KnJg)D%Fku90w5Ss+>;P{i-Wy@3*x);9ScTfIUGn(UCkE
zt0*TN`rjzZ35Wh4rzpqG^<uwHQNGK|_x}xwa;KlGPEkJR=c+0`lydF(e@Rhx_>rrM
zkCAJ~|4WK;qu<5<y`p@;FQKaVSVHaie_2sZm^wyL9vl?qgmM2vMR`b_qWnd@qLgfz
z<rQVL55HcpPqgpb8uvbE@5A?=3Z=W~2-gL*6HD!y8gbA#hx3J!+jHII$tO>4#4;O4
zl4I}V=R_w=K5ovQT~kjvumPuP{5}ywaQ~par^tAWx?IBn@8U)S@ut0xpG@b;T~n7c
zQsX$+GPPm)7go}(X?1i_bg|9EsMX8kZHCDwjMFplUpTcP2x1Y<t&Zs29^o`}p5%vg
zZZc0zWa4n0uhH3Q7-X-oEW4{^?aeUi9`3*GZG7{(NwXsX>c`n<GJ;~?7(P=cM;qA3
zep92yy(drhj|*=5Pmc7@w1cCTsq3P9SB~2-{jAAnk2`a+u1;s}>QtJW&q;YGA6UQf
z0HYbJonVBaK6bFlez2L<tDtgxZIXixo*>|m1+2}(Vcvn7epWp=!z8oSo7h!7%)+fh
z!}9>fKY!TnGqMswDE06Sj~-q>b=TC{w)-Rg&f{9vyn3{T0gkN4ZMb&S7#OiZyuuoS
zLZ@Tu7ah-h?3j)7<a}M|n!4C1ctIG(2p|lzFzzrDx^F&`-ZR;1%Yx{X17i~OFlntH
z`ji7tBv!DfehGf-d~HWiUgDW-X_aUoLP22JWT<7sbR8}qJB4S)<LbfbM?8K3j|lNA
z4mjVCd%`$l;1jFQGxod#{qrYje06s2WO2!Rq60-&r&RkIiv7SV0*zMrqpi#Bh^C68
zS62Jm*jjFTJhDzY#1lH`919eQiB(ghdNw(=9!7qVtEnEW<L<8dJ$H@S>S<L%GUhTW
zJiU6Y#tnRRX6%Y*&oSog89RAB6xY;ChrK3m0_khRB_j`iskgq@a>VBu)p-n((4C)Y
zUH!A_*tdJf`pMT<ZwB1e-`p^nPc`p5Uf|uxs5LRdsAtx1sQ#g|>gRXv>@f@jR^DC@
z<oXBs;6=++k^DK6JXXdXHwW?e&MK5+vF6-A#H>Q#cMK;n{<$XKSYa|ygi)1elm5l-
z)Tv%@ia=5eTs?0AK<Co+bTpHQKxTIcd72&A%V>X4|46J3Y1(CT&_-ES6l7r6R84c<
z^f7d}i5aS()cs)z-M}9?<I{09;hwKw&nd8ap}HQ!dp!h;)mw(l`dgc8JLmU3#_n4J
zq+E|*zFi;oU<9Z&j)X-UFm+)|U>&>Q+}3)mdgj`8F!(fmq=k58;oSB~CYH{x$^=o@
z1^Y*_+_+FToVzG&J0MRpzALaZm-NiV8b~JgjwsLzCIYALj%wD!dm=C8l<U;EdhXIl
zlj-QP$TmIamkzx>r?G1)N_elydkTl|jo9AA<LBJ(VjcayXe`jAtuK$H$wad6uiX{Z
z&O6X(=>w6*`o}QM4>Z=CJ-ViPhW;QoHON2lXe@;0fr$^r=j>BJv622czGyg~&5S$8
zT+u*RP&|iK1El_Cw|nY;C2NhRksWn;G+AA*;wwCjd!IaKf*Bw+vimSY1m3W9ke`ag
zh%pZee%EM7fHXtkV_!41+iN36B_XyHlgzn~L`!Uw*Yio!u|~@cjL^8ZNbg2IT+Kbv
zJmXB%$TxYA1$#<D?4#;5F^F^D>1Hm4r;H#z#&Y29IhRBl$IPjZ@3sp~_lX!dVPCh@
z`iOvpf?Ex*>ikLDbd7$sxPPsGz@rHG-X<y(TQfIQ_;3Dn&dWwX&V8z$NJX`ut{-{w
znRE8g(4UE5BQG1~O!dfb)b-ESNQmL9B=k9u_g2*1?%_H4@@Njo@x*OVy+fm5O#4ey
zyQZp#fctq5xnPe9Z+C{S`>-7|i|-Pyt5QvJaaVlBWW-GL{N4V_oP#~jF#2qbbx-v&
zYd^$hwd-GCvsLjsXZ$=28SV96BjcGY>x;&-mx!Rg6i;T$nMri^%QY+r;sz|x@RewH
zsbi}8tJeF(1M_>!srdIQgzBv5uR)?lUreEO+{curEI4=W>$L^U*-!jKsxQ9e0YIm3
z)E=2LQ(M1ZBsynD)A4goGTzs?-_$&H0{&Zm80!J3XU?G>?Sb0RbEa{PNcn@&{!W{8
z?%VM!kgHMoci3Ev8{g%-Zv_rNcN$*=LF`FAV-4Zl_iN-h=OFnG&ix=P%-DNkVDBIL
zxs2y`8{#34I0)h48WL*O(vM;e5W2(x|LKW+PJ_s$el>#9BfAY@EBdkRz+>_FIX6Zp
zNZ|a0!D4T411_dK!k@-EfN<8M5#E*gl8@Pub3dy=7l4~66tDk$jL<<oO07TcNo5?T
z#@j|3PmJm5+%NECo%^L1rIRN!Xsz#)V<r<>|H=<Jo<3|`oBg$6)U%q%<~Q}6VW9D>
z@l=dKdve}^@8ip~Ed1%(Wy5O?tG}(uY-5Ap)dsHXd7<&$Gaj=hbJ1Vy{n=V?!w}QO
zLu=K~`P;#ivDMVy{(bG~G1TT~d)~;ItT?^uY1fb7`~0ysLOj2>y<UhPvdi_4jMg@6
z6!Rwx0Nr=)&$Yp9IlFD<FB;N|@3+w6i_v5ht6Xn>pX2X@G5+7aR<mCkyn>ZNGb!9?
zMwjOpiGrqJYujy+>$qUXh5q9aH7EhYsZch-M-V2~I7Yw`$LtX%g<&S`?kY~MA5^pK
zIo9^u%e$C+SH-yjxL&%vZHyD(!!ct*9isYu#@cuL*6&TAuZ!6TE*2Gv;j$Z?#s-k&
z29NhKk_o#3aAM61-lKj#?Q&B6wh8gng_8EHcR}~E%@LxS^P34CO;jI^0lVM6di={I
z^#~6Cl-M}+VV<`lWoiv6wV)4M7Ap6zU!9gaZ7fozd!!uj-yua~9Qb!!s~Mg|4jRjr
zp6_Ok#n8d^5F_>@IV-CC--s%at5*I$jVvD|_@6=5d;iZul~H&why(r1_&ex3$fIxO
z-$UQQW6^iWSoF=Rq7MvG&BtF=1M%Oe{oR9<MD@S@?Ec~Ye)TE_k6r&*{M;TpT$Fxv
zG`dq4_8g7wdJ5v0%{gTX(IcbL9ry7sS*W|i{kstZ?lYWo-%aG-`}voY*xk?AdBchK
z@o&>;^iQm1@{ZBy+8a1u%)bxu?~2jrIz9J2{(YQ(dHy=)+v8Yj=fZfO(D*$63GGGy
z&gS3M{5!<I9klpA?Z5HOk84?~k^ICR#FkgbH~H7!e;~g9NiH?`6W__-e?GpSKTG@E
z?fE9RPkjG2*4Mc0E%%qk_X{^89*=y>{ddIoLuFm9gXV8)SRLPg_pwO8iM89d{-!-n
zi|>b2b?7jZNv0R{xrIZ8TxMZvOL^huOm4*gDHU?%h4qDw?$k9KX1DdtZe6fwnJbr4
z3%6uS<!m9pAmvtW=<4q1TC-}c+t|6TuWRiZ>)YP4Y<5e_g4P8sv#Et_X<pX4eD;y+
zGU?f?la<-+r9!zJZqIKyYWDnGI$Ktm=KO(TDKl8+!a%ZInV-#5P?<?p=BJXw!(1Jl
zJ%26#3>0$N%KU*+GMAZOEKpjQDRGep=a(y`5xSQ$m61}OYh28fE0l9)^E^9{FU(J8
zhBKAS{QSu9aQy82)KDh1N#mxImE`;~6S6a%(?3<n<q8^#<6NdRnBhuZLsmkE{*ml(
zC7YjLEU}OQ-!V&NXt|X1C&g^WAMjLa6a5MmW+-UM{)$b1Qoi7}q>3Y}RVg`KV7c~3
zXQ^~QzqXmk71EhRIg>AE5`&pS&S?v-oGND%3{x2?yP>U#Y})0r`9wK;e#TXjthS;{
zEI0)q^89XN^>Di@SJGPslVMn=t!MLYb+$KC$|duee5EIoE0nhPX{d6h#I6&-IhRNc
zXOg9aAJFBJ+lEK-WyZ=T1`4dTKbhZ@C?*FpWtS>0UbHMxDI`Yn*_76vOBNj)$&|Ka
z(u@0Du2NzgFjaJw6wgrRv!y?i$mRzMu0K~yq%$ck%Jpf>D@O(fGNsko!J)Qvy5w@1
z!DKmEPB3|*?0}3tmKZ7~69WV3=H<;4+zyv3Lrgd1vZ+LWvYc_nWGP!tC9;KdD$%)S
z{kpRgU2D45yE1c?a{X?AI>1D#QW^%bx!zKtzwCyyd2M#kfLAP(UAmOr0xE_x`N7JN
z+c=n9x3)(Q7K_^fBuGu93L~ONCZqDDoJs<<LSj>9yCC4Y+E*Xfm*bbTS9YxO@NxZx
z)KFP?01^qfpg5e|4n9j2x4yS0v9(az1nIh-fx*5C1UA^08O(tjw?!v_2$Euhxg5aV
zrW$9L>j$XLgNsF^mja1IvQiisc0~Xo_*YyJ2n@3{5e}p-8es`AeZYr3x7>}C`x8SW
z{fT0sR0(%NY4oyY*H_HuM~a<!do<CXW(O|SAMZqv3sA}oQH2fv765&|Qc4uFMTj$%
zNHQyQQ*z~ECY2r7-s1X)HyQX`I#nsVVlGk1l>}=cWM!61skGLJh$!|K3Kggzm#~B6
zNM*>cET(EvzSdxzFgUO+kZo!xS%O+KP;)w2+Mdvlnu~fo#4E*IODiBsrW3`O3lamv
zNxpw2N-R3?nAu|B&!#g4Xm;3TwpA)tU><fWE(pUFlOynIa%(bMN#qNaM1Og^+g46j
zTqeJHBr}o`S-4^<Re@Q`BmM16z_Qy5IV16{rEDbw%OS3|3t;7Pfkh$83L|`PqF707
z2cxz6_H<?-IWk;HfB<#`kp{laCWjM%uRjA>R7$Ql-`geeH9TyXZ<pj-J1`)kNR?sk
zLavysBy2}9UE4^>RAm^*0Eu#IIWq{^rmGS}ux3DkBU53`bTi5~k}p-tg@FOr-k#{`
z>g((5Nc45C+t}6Kndn@%ZtXe;Lh?|G>+cfR3?x$-IImJj@gu}0-}{a46G<_BI*V8V
ztq{aW5rkwCsZz=%K%z-tk$<YlJ1Fbhdb<)`9k6)Us<UI+MF~Fq=;av4Ep7nil1QeC
z*|6K%y+tln>QAI{Y3B)|JcMB2kO}f5t|L=Ud0q@~t39~f<sk<z2(#Kt!cb?lujO*H
z1e-_=hz6_TqSciK`%jgSK|U5U$%^L!VHZLj$PU8Y<xN|;3GF3@w+y>V7G0Dm4`(t(
zC|){mq{M7Sh=Nt7Tz2^_xeUBgNO6)V6k$&Bi?ml^IO9@<QbxRk4Cx;kOcX~dsomx1
zfM6z|p+xfh3~v@b$pZa~6G@AB`$tOUiiwFdP-!*J_8W(akV}%ZMK3hrBNx9{!J86p
zdp0wiMmA<ko1JG7SjsU>1ubR&Y?n;(O3zht#o<J{w8d+RLD->~6Q^!Y3}s=8Qfg?s
zC}}VwNl=|ftM|J$`q{K4Cy5E*U2o^Qp2Vu|wo{>#vl1O$eQhhdJKgGZx=Y$exYn)*
zlG$P4>Qci=K$z|a?%dI=U^0W`^K6{lLO&MbfDxaf1<Jr`5W$}-4xzBzP)h0zYo+Y^
zi|TiNA)kTe27{b9ugGDn?F=U@H<3!a?I<6YgR8T4Rf1|bYXArNl1QE}3KFoZNBY+m
z!63`c<`Fo#B4P;RAkeUQB#nMC4)R8oYae2@Cfi|q*VDebCzl!+lxl*cjOz!ouyz0T
zidg`XB!O5;8O<?<hFuA{#Iht&jq*~PGp?st>1*%n>RP)l1E^TJMEyo@Irb#a<0r`n
zOG$u~Dw_&2p$hBEo<IXi;d+fSMw%zJUFm5QP8OKBExdG-E!b{@mfGF)fFjC6W=v#L
zL&qhCi)f!>E|n;wI6*WP2v!Z8+tS?9>ed%?fwN2LN;!p*RRpChiFb*%r7~bJJzR3@
z+fPmO4UJUNg{}GCQf4KrR25;cXBjd{A|ey1fsuS_Q7c*m!q&=*1z;*g=u&VG7K6!%
z(l&!Lurp8;O4&hF3(%@$(ryGx$B>G-<I01`JsBBe#};KWYNk{g0rQ^IN{d>9p@1Sv
z!}9PI5;JDQrDvVxKpMg<lgzooa2gw;h-^X^NK%Xx(Kj+a6Q!+2Y%R?Q&EesGX<`{C
z05wq@+Fs6Lol7ZWo2H-?V2TYREj5%Ju3#1yf&{RQimxDsEHFiudF6_{PhcKw@-kUs
zuRM%o8UQ%y%oaw}`&PWjRY7B$z3<>CY4_?@nw!&7F`XL``ibtfed~jIS>4-~02j!r
z{z9?@17x;jS)(XMc9~Z1^e2HfMIS(!Zc{&82DH`=r+W+Ktn4E+;YfK%`cpCrk}G@0
z1LoKZ5-BCDZsadgnSIN^Fnd{hO;*7421ui2sR^xSZ;4?2jmTKXYC|Wmb-`JhO{8AT
z7ZWH=9ygIJSz0x1mr1CJC4oKcW>7%TFp8LT^N6%B=1KwPsknh5)Mi+WOdl^AQW)Y1
zzE6*CwJEomRa9r^BllZ(pLt+;%9Me}h4h>eLn1@X{#y;}^__j|-8MXqA#9hDyf-8u
zVmpoO)ay2)f8)oR@;uHsO!l3bP%X{dT9Mvmpee6FdVrk_0v6c5YBbAWj$iR(9T>Ri
z6Ddp6m~Lw}-v)zYj(A^Zu!KS~!^-%#7qPy62*EWtw7vjEW0;A=ymB2X6gGK{9TYbF
zkAa%ok^sd;kZOaqEOHraUzgb;3t3VgtuD(6^9s8i1`zRKPs7cz?C`{s&uo?aN4BEo
zaNUIXG4_9|Ag7|}Z83OD;+u_0lbb#&Lw1>h>qWA_DY<nc`8-pu#W<3f@nX$1Fbv4#
zkTAZB-~-7As~|TBd9W|D{6zo-Z*PrJY{RAyTbpf#WuyWGih+cq2c2ym!4n1_1!MGt
zH>yxffzhCEBQCGekshYG#gegSqnJYiJvY2Pky#GLDc~=Zu$&-lz+WmM0}_QicX4Z5
zmLhwMnGkHL{sCAZxgAhrmjt&+9!oHpWRaAzMWMs2JTaqrKjIJv0@1`M=j?!)DpID%
zKUsaIS#iM`R*r+YwK|Yho$$Lc?@Q7ui&#MNWk3@P($cpgX54C+uV)}xbyrQv4i3nV
zN#Kr;RRg-1b)|AzE^lqkG9fZ)@o1gp)ZCKv=-oU7q?ileO+r79=`C1s(TJp68R^Gf
zE*H5icUp#?jEbSOt|H;FM$IG%#&4_=mMmF>Qji2jI`*fr+hK{3(snT{s-r9++kt_d
zSh-=<s?K#m$_~jY(fAM(dV2)vf+UO=i2INAmXW+itbzgr7?Kl`hYb61%W>03ykrUj
z)I&(I8*>(+R~BSCz_egVy8(#`(wNU{P9a@PqSS2#E1xGbLCY9hm@O`t1wYhVaCm1p
zpl;pzb`&K#K<ZCEGPYX%aEl$&%8yO<mxJxbc8V-RDj5NlO%^s2Ye1Z*fdt~nFmA?Y
z8hEm9%%nm^K6SEs)E|r-;~g*OhVtocT-8YeAtEP*jVxV^4om~70cc|gX`$!{dly?F
zRg<VM7;sqLq!)J(Y)L8$KMCu;-k<Uj4HJ}R=v8f2<B8#Z!a``Lbj6q{hQZKQcGTiB
z<{eA2jFd_s)phl6>Ge@K+&TP7Z_yZ)V_BHvZ=OLdI*?ALB(jn)4O=v8*|nw9rfieL
zU8T*7m#(&VuuC;jFaAZIV{bVbTZIza4U$kubA!W$ewQzaAS8W+L?0uHg&l%(OGXip
zeENL1MVx_|TsGPidABT*1qiMAD3Kf@G-j2(TzZ?x!F%Npx$JP3B$dsKDFKQL$WpBX
zt8Y-c!}F>E-VApWc6AOH_Cqou)q!Hs8gONZJtAgAD&}q08_d}}3_hG$)GFr0et^NG
zS)l*|CO+oJoNNya8Mm0#mIMcEONN5csI5WklY#1uDwtAKjUW?DaALQnhcHMand^Wj
zkHU)qnP26h>;UM)Q4Wr@t(#%MW@WpHPoZh8p(*b@RlPAEGy^YLiFLfX5fIG@k`6R_
zlpJRxX6)5VMdM+9%SRo}OsF>^Jn{JejXFp%CrncpGc5wKSvvwN+yXTu(~6a*Hk(LH
zv@BY@1hYr58yViD*tm%SLW(i{w=70BLO7zt0laivLxQgUTLETnWx=FD&_H6AZx*G(
zOvVJB&e7f2p60*Xf+bSB)@*F+?jkm_eqCo<4?bxSth+I{BzTB-R|A0MlsiA~W6wr@
zro<G)lpR|n`jRTSKz8Q&N>y!7jL7)3Rid?mg)xLzlwz@v96-nuTz4CCN!XHYqUmLO
zv@8NbvQ!1$EK#g1s-gS*T(ZN{7W|ozzhh7zB07-;(wITE06i0L$Va|n%S}!rvIz1w
zO(j3ykZan+TT3~d%WZP_#I(bTdCRwr%{oTjqUb?bqfAMQZ6jxbxd$DmiIOPfSd9T?
zT*mt!z3p8c<`~FwucWX*E#|k*i`I3Ju{r(%e=d_OgLiXY;KJboro^T}m@zC(h61Ej
zEN_KuEh1NzkF`AxrXf<0bAZzHa)|5`^ihP>BI*73VR$^*M9Y$811*bV`oc3!iu%k{
z6ag|wEmSQ17|b3MP`Io}g#OBKzs-{%Y>4q4c0Qaue|sYMp5^U%6a%}(2thiQ`U}L#
ziBu$rhl_nRM|r5tgwk$Wm^)^7R#&)Wxon4l9MB<?AnRAsL7^gXx}?o^ugZLwN*o5l
zgYz!#tdlhcB8k*eAr3&)S@0SQ@lkBnpBDXT4L05)Pq!#8OdszhLkdBU_N?kkwD)u*
zx?wb;5{a(0ZY7T8sU%Y}*!}J1RJX6}$>i26kY==|KxEz!Lx^tW@QAs8m`+uXY+$*M
zEP5*%o>D~2c34j2OXl(+X*bJc_Cau2{ASvih)Itq`yiJMHJ%_uW2RIY(OwtT1mCVw
z8cHQ&3J1;@dqi(yev<Y_HXDkJt%Ln7okE<%LL3FLJ}ML8kS@mtvutkWhcvcWKzKJ(
zUWCOQ4kr$UPn^aw6K^MeCEM1b;>+<yi%1Pw&|q1+Nv6Gz<%24s)}Xk_qydCd&Yj1$
z<IOZLk_hbU?e1EiSii1qO<!C4da{T?1cEzH!<O4g;)BW!1)5#xxox&#Zwe~CWa=zi
zGk|3G*m_)lYKyH-iWTVq643M*fu?P|5m+YFPG~SkWVY-*K~F$3b1?q%E?oq5sVote
zvI#)+zCCVEmg(BDYgEY3PmsbhPb^*BylwIF=55O@I4*r$cIz_5;q9xG$n2B*rHj}c
z{m_|zFsK=`E{qivd#N|%tm}ZqQc*!ZS`k=ExXYG<&(D_i6`V45ll<^tXk&$rg)g}?
z5=O*)Sq&;W?k1xX5<#WPMYH({PPtBe3w$+8B9svz{R-mvg$Ds_v?%s-K^sVdp~K90
z8Xh3M;T?xkEJa8{NWm!hUQ88%kSu{*0SgZp<VZ}i6h)P5;2sDZD7_%9UM9mLYk3H2
zIIr1(R0-=^%vC0|=}RBZ51B?SF{lrOZU`GygOh=qu+)f1iq2%ONz?k&pG%g9wY(z6
zHx^^sk>f+Km=x*usX{N2LIf%HDVD*<KAmdvKOoz<3Yup<=`9S_o6T<P%~{3}fmlWv
zD<mbjc)k_}ixMEo{0K=0nI$C|SFI+%j6yLSiAjg}Q++5{wqBb;<}#sWl}KzWyKHt9
zp-NeDDb!c6-ILg+3=I`k!A0b4Y&5;CyL;`KiJrEuHS0Uqw5@6H{5vuw02DV6@nbfx
z@*~x-f(+F~t)}a}#8CuP>Q@2K?!wmITn;HscG|5U8p&@G?{wu?Dkv74@8WYp=#pJo
zoKJA>Gp6hlqZgWpu??}0<n3xRcxQjaQF=w65WG^RXcCj2fjH<CA1x7(iJuVf^xN63
z=W^Y*WOsthw#BwBF)l{0-XZm~)%c??mzBcp@M=AvN}@CHAR-BsX6b%!0|5yZr+8Qp
zQzq<Mnp+odTefuhHo}ed$^q;Oer9osRV^v&a|W`@VoRCzqIHRbHxI<-f&!QXbb$y`
zx_xa=PurS~MBB=>>(+O61b@%0a*=R3tXs;?i&K#*At!Bu+<0<0W$-cq>@0=t4<)OJ
zVhS#arORVuiP8^j=6+&o3}eAE%q*;g{zZ_svYKc`0wJ=w1gY{?Z`JIs6wp?|&*>>v
zy4JRLVME3V3G=N3H7ZyoIZ#Ost~+(*85`QzpKJnxO+!#bpRcmO_hC`Va>!bhuxiZZ
zfK!d=49vu)EVgUh=P6IAnS=-pvk34g3hMQa;?8*ciuM)BhFcJ`Kh8^dx-kmTHof~P
z$J8kd@5d7{$;Kj4WGwHUKly!ck(KG@B@*xKR2T^h2-J&rn<mfzv#7Qqn^O^oXjNvJ
z3K?XxTV_Y`v-04uc~joNXj!skU<q0SD<Mu2IRz0sWg@C1&ue*`V54U9Vq6L10zOE2
zlZDDf3AIp9z>F>iEpJH#=&R3bYYgy&dAR0QA?T3_>g}19udR#>r?ZP(Z2DQjhcSwW
zdX3(fdJEEEskxlxr$VmE6^dw2gG;O=mbl)HvLe?ZG23&+4kCA;WU!M`qhy}c8gY<S
zRu|GoCdb>lwsdoaV$MXDqweLzSN%Grjuh-Q0h}7k)4UK`G*U3D4MKAz@2QyNFb}4?
zeRZFl7{%bQ`pZ7eZwA)-wY|NttWsg-+{JRN1)`}~Qpfl>Sl!ZO9*D8+C(G38XxhCZ
zUP(<&TksCOHd2T>lp%ysECYCIr_j=8kro9nBqZcH5&4j2M*eKfCf6l%eR4eA&`2`n
z2tBZJM70&%xZYH@a~sBGxvWVgP*571;y^dkT6`&wVjyK0Ob-^1l@Q07Ou-7RA~c_r
z@x?|sotCLbkebt!qM!y_V<6-LG8ieDq}bE)@McAUgcd?Pq_1QKgz0iBEmA<ubt5ZT
z@JeJ=pOsy;<<lKXvR5@nN|GCCZqUT{O<2x+jOBvDLfm%tCf2Mam$jyi^mH5)2o`90
zYnBL=B}RlO_J)(*zVgft4Dt0sJ@rz-_^4waAA<noZ9N@TkjFIL9IvpYB@W)^3l_Tw
z`^U6MD6Xf-pfrzdAbuJSvcv<Tf^h(+8Gn*X)FBXgQzTviGQyf$aHGluD}4|JG2tCO
zPj7)I5nscqW{Go|v4v=Ua3x(PbVPaxySk+nquTTyI!{F`CWbwxOdz1|L1O<l>~2+1
z_*#d0Bf12k?Pfz)GfW~Q<GFOdNW`c7q%uXsUU|VjEsJmt2v3L+`;_ZeCRA(j!j0s#
z|7c{aAMum4N(&5aRasFYSx#lg1T?BH6VzREH-iAjt-%*FMyjL;s!1tgW22=k0UsNf
zixto+Dy#4oO%U>V!r%zuih(SaZJrt*)EEdgXF)6U0f71tbdZAtY1a|hK>C@P8wY@q
zxXDe#FO{&Bn74sUNw`XZvLlK+{J)r#sx^Y3fbIyciKSt-LoS5f@*^S7VwruD0RVSk
z&@@C~IN}faWNC3i|7Oe_HsdA;m0Woc3)=ev=r11@Fr&xHu?Qjhh|)-2K8pde1V@IT
zL^@BoQ(fhS2q=_=hAyGjhSXPk-<iQOkV*m4l7$jC3j9e>d!-~(SEjQ(N1|+eLe%1<
zA-An$pwt7Fq6ELhYE2pi({)pph4!x*$#p4EETI)#K_4NL>4LiR5RXH-;w*)=2^9?V
zn>J@1q|L_IXLN#YVx1(lg1~^-A^q40WI{@Ze7(XT!FOd=ONFh_jMYul+`JGyhcQ@e
zTy+7-Fc8zEk<O(QG@!rRNeXTmKv%XW)^)8pHL+n$2Q^_E*2qN^{nU(PA_R2_3%o~a
zJADKYgIE=oR_SU;L|OUj)eyroSs*V!@Nbr&ta%wil&2Y)HWs%BZUb4Qz3s|EG2(sZ
z!<ezuNZ6nGpuUE5efLTP3x!c5`PLEOELUYj;W+QDk_d`(`PA0Nev1sB*kg%P5xE?R
zT3_L4vO&ah+;Wt2DA4d8v}%Z!566Of@lxzX>Lj6XGZ_3X#Xtu9nwG_%Gy}e`vwiKF
zjy7svS7OJX+10UrbqKVi`pHLY@DVFt85_4VmjkL$#!0vnMpboSB-PusF0o!33Z-Eo
zCNFiVu#rH9%gfU79V(seQXU+`gixb-u|+v_z4#J|Gdg=VB-&S<nrL6&tv(h))>K$X
zZfidUX!86M<;_7cd(*ONzbziH(#pDA0nE0g)kLdC!w{kp81PWA0dXox8EfvcoLWVS
zy);)GOH>tHUWs}HB2p%l<py=P7!?*0Ov)LXI#I&F6s%eIma^AqxPJMHk|JK~haiAO
zxD}9tlc{kiT1csyRuBVVD+%t@X{dHfBbkhZ^;Sa2m@;0ZYz`GAiA1?zwy*NGwim;W
zCCAO6suInj*h`iyGG79zg5_$tg=(mO@e&zYGB6zqLFvru0Lutm0eb4VDcVh?H}tKn
zY6uo>@D(h?ZDF^tdf>=`Bo(mRlxT>xRC7xUfe=O4AV4G`6@7>XRSV<dfC<E0mIQZV
z3snMCV*26hLuHZ(H5<p}L{!jwF69#smW)&2ifpJkc+?nBV`e%E(cp=AnAis;EoQQ#
z1Hoxre5PFlW!&GB+}2yz%KD@tDX6ealt{;71obvWTnvji^`hJ}g*QtODn9;S9~XpK
zsxH2Ku28n={b*2c#p@=|yisg)AdQ&qCpl6>TT5-%3Q?IGfGwyrM44cAW-Aekz(lYp
z&aJJ<oRgTgNjs?e#e3}#f!S?>1Gy2e&Imp$o*ElOV`56k?*Rqvs^t>Z0FN&k$(GMl
zZF7~Hjw3i23J6TQx?xM5hsA_69gL5xR^Hq7cbKcwN69o?t(;8?TsT#5$$F^POO8MT
z%W*fLJ#%<AS4e9i3<zlrN?J;YL8Z3PK&JNkg0W4Ur<W!dGciG|A?s4@-U5vyyYkPn
zsaDyE#4vs322l#-YPCwhH?VtIm5Qox$fYfal!SOSmb?!QSX>IfJSRgxhXPeCu%|UI
z=_Q@2QM0`Ab`=%r;SrJ7qo&-G8*m`&Z8;4eFS96qe;flrQ1bGL=P?EEt=z_QSvjN@
zjap;sXr;VtD<%VdF>ry;Zky8E7q%%F+|0>5m3H#URn(^yi=x1Rxqn7?M3?)K?lMO4
zM(tk;(5Q{uCVMb2CBbIogISF!RX9?5V@{-{yuI0HT8`)><#X~<LN$|5`>JZsD~)&`
z%e{$J5VKjr5zt{TwY)VZPQh|B>k2u_fwvZJiS!^R1x`#(qp747Oec>uuZU<T%dtg!
zUU9I$IP3#!L`%<FPX@SceOwzqv~@j&#l3ZQkaAsK?k1Fo74;{|)NU%S7!0;rhb6o-
zkW#5Y8O=pe1dPFZHF$`tEW26jSRbf7Zfvv;5x|>K0*I*|3z}fgB-(p7BnXl_YFQLn
zY;iqED;D90HrnwhsP~R7>=Rba3kpg6=$vU<YAwJ`O@d`m2U7iEOZ{maDPu}mY|3kU
z7Es*8++HgMi@8(Pf%VmBJYyAvEzOGunk{B)ic7h}ScSz$1Y^UsMb#s6Y*mnxxKly2
zFs!o2pugC~TPemEpe|r)7H(e%!PJ~fnI4iZM8Syildef7`l;BV#5ZiQFg%jW$6)4N
zyMCWlkX^1&o9~4;$%5In3X6iU%JHIGk`j6gvMM8F5(Qyu>3rc3y0rAF1qwYP`>is|
z7uMsdm+W;DZ##R2^8{^PH+iCMEv@`n?oo;qaUfDc7kab=LSlrn4*W1*Qjg^&=C5;p
z#NqwQ9TF8VD8!|enA{Z~*+M?^PKOzR7F`JG(P}|~f<kB{6S0B4ee!_ENmnV?L$(OR
zx+t|xR$dmDimizrAVT%Z9PW@GgqPQVy;%}L)Jhum$9N_vzG#fi$WxXGv0eFEC?6_-
zU64eGEB`*)!16_s9~Nr1GRi#2BZ?eiXLV&!^>${>kFLsOKub<SlCN!;Xp)MiNq7Y^
zM3?JLH{+AQbr!n!T&Kz>Kwz&lRLcu|7Qn0)!Bs@X#tHVTKmZ$OY@#g|>!BFhr!gq1
zAZ91B5bmU!%c*WJ2u-}B{&@Asb*<dkqCc%<7R#t8h`^+>&WJVn`Xl}Z<1V%gmnM0j
zFc24RiX=p1rcKc5D&>}Hd<)ye;a)QRXRhm7PX;Qisc*5j`ZAk+7Bi3x+6h7nlS38?
z?zD!WN}VRA61FM~7A;o&z7;jwOC4zGer()wHPqZ&S7iWeuJ-k8eW6fQ4mE6qlYyqz
z+%;PYp{`WxTPy2R{a{Cxh0>oP*J^fWEfQVsGhs5EX;~T<iKTKyv+l7IGv)5WAbLZo
z25%W?ucnMj2zd$+Jkx>6KA|@cF(`b1Ul&VP*%rY`hJMMgAzV$w*h?@-F><z+<j^}g
za=~0|LRdkPj3VAMt-cCmnO%$L2C*ff#Eg0|WlSPKQBu-Do{7?y;^j@gOmI6?Mo08Y
zx;Iv>8&`HBq+w?ii7a&)vA%X4!zH{g6{0<?7wZZcQLAAHWr4MTk=8{5o~XFuQ#sQ@
z=B4BC5ka!#mBg4gvv_60(zDoMDkgL7D-}jCK^RyQ!R=!g-e?8lTjSSe-kRU9)oIF&
zGFw&(c>P)HVOhN#MlJssou}P|q@*YosLR$(4iEBPNo6S4MdE<Bku2P9F@Nce@Me@a
zb4wndWhm{fCBrTiEM^7Ni?wvKy^AoANaj;RitS3cD+pB{Ny&Ku2KIiM!k`vE@FX{s
zP2&&C;f4@p&?=}bK%37xQdX+SlJX!k7yw(m!y}m5N(r?Wcx#Yi4K!@1okLoAQxxZq
zx@})ih!@F%tO(cGm^C3oecH)85xs;~*X#Ytl>pFcO53drXsigA{U;GFWML`?@#Mv&
zp&C2Jzl&`AC>gwVAY`a~EJ<$`EpI_k66sX|jF&22h9Sni)mU@-G`%dY>T4hNMsqp5
zZzP2!gv;FZkw6(HkRB;Lm5vF71VFjX+jmfR$*QnGj_R=SvQ3W0Z@jQ6LR99a0${-x
z^BFOwnv33`7&-Lyo+5DzlqsfyUv1)7@dm^`DH+~yP}0V7iy+@8%;209Hio?pBI#>n
zP_<kJnLSf#C!nC1P{#a|c6e%Sh(Fe<Id~9PV<@vtHB&AawKf$Fr}!(j7q=<`tpu3x
zD)kS;QY)YWA+mbgR%$n{+@cW5c7zJALlVbEvRcJ*9V@7k(QW|dQi4RY1q;m~l%Lna
zBc>-*-UU2lZ1m^aY7RMWpM-FK25Gj=D)0<xvQ~rt?Hz3S+WyTz2!UOuHd$E6zf~?k
zr~8##;02Y{X9j7laoL8^o)2MqNy<*ehKiYax|3~E=bVTSH~LgeNZ|y>m-pCLb#-_0
z?o=C5*jk>yqa)G2y0iWCz70Lqaz0>2ZQt0`Bt=cWK+i4*o5N}z6iE-k3uSXm&=Mu|
zr3|7};#OOe{ltx6YIctp_e<ACgjPLG_zVmwBuojWGI$p9$)O`K?1IwsF$f6>^HrqU
z3Pr0NKzUjH5$c<K1?)jYvdUp&Q;ih#8b4T>ED;LTd<QO#vsFGI6NDEgE!LbbovVs=
z-omm>AYs8_@n2E)r>eoU9i~RadY?9=Hz^v-@h(ROHr3nyak7kcwzKIj-UW~{49lqS
z8fv|v=Ho0Dnl~0hHdxdU%OcP{Ry-wN(x43CBt(#_!LzG>i-rPF{14MoyP1CRClwUT
zAUc2)^f3*=SOIu@L)xOw<{%kNutnu0Sq+tCbL_Q3Uy?7=K|wlYob<s8X(yc7Y_Z&G
znw8XzSjC6Ttbi7)54eSo2oR{#;Z`v(qh>K{cx%y{`h>$tXF*K%CW6l}lYwke3Irp-
z`~1QK6XRPTNu+EnDj`1wfJ@J>tjleC<?YsHa>%Hxg{|WVLZ*2s$^ah@%C;3Fb!(Ir
ztbf&qQgUH6Y?h}M)QVJEm~N6WrtKRdB*nc+V>=__IjpdqmAXg>F|S2&DqFK@LJhr$
z9O$KUb$eG$64hRa_FM5;SzECj31dxFu;}d~m{xB%sQ$!C3ZNaIm)rt&nw8|Sw{eHI
zwy*4!$O}@NZxKlK1P4ldDnZ6V&4p)Y1L)A0u?ZAb8QUZ*XTg*T!v$()Dqm+LVppt)
z)%mZA#BO0IY_uXM30+AK8<Hgu&&QMVZ>fvl>zOeG9zc3{5!b5{=D|K3Tf7!t1z&sS
zn$C6Y-JNag+<HO0UDTwyDXOyvAXrqEaAWtZP^3y4Df2lYa1i4L_0n^k=5|}V4AObb
zc*4)5hs+03%m_giv?WzHyfwsrh_q(&=B%Q;pfIbv42DPZa@R>byW1<+IOT1rW{f76
zO>bloSY%$k5GF}USrFsGD2H(fG>0S=RAz54L38%{M#{Vjf^29m$%+A=)mG}0;f7>R
zBgbG%rP6t~E~F+5AtrRO$B1BccgHy;y6WvQxL4_wP>bqKbXH16q4x4XZ%N;`sHl3^
zy!+}aD#jGM3#9OE0?8aR0aTA<HcZ-5*%qirk!H-1p)x6r34%#gv#?AucoQLzie!M^
zn6i{&zuAIN?yxrQ-|=Wxl*XdDAzDE=7wzM}xlvuSXD}08airRZslzC|Mg8|a!Vx=$
zaj!-3tSyU4Vo_FI8G^nMDb?#skvQ?UV<dw>riCNo5sr(KN-6?|X0x(fLP*ly?U6We
z17<a{T&zU%^}L|6Ko6Q3n2ddY4+R_;Sas1N1){sDp+U@?Y*mIUoUKw_ft1`Tm@jN1
zc;+n{UYV`s<;`eHQ6FQ@yvh(wc?FRW_*!uYg?dH2vF@`{0L!6`@V2x^eW*RE!?L!a
z@;#40UY<mTv3&4`jKPdiU7<&nc4`2Mt8jOxxHP}$i5wqOzs0ncsWN4gOO`e%Gt_6H
zIecyfKBOzFUl6j`jAoU|yOP2;+!GW~ygFXKWCHt0K&F`D1#Q4oG@4B$b5!rj6trpw
zvSe9|g7Od=OkormDUhUpkJsc60P}@Rb-_39npUfAtJ5|&_Y-fG|I^LFV!>G{3=UG2
zAyJp7uH0AEVMDkqjH*{@;Dv&>&twEQvxfLX9I@U>pXY;7K-T=i-992>wpf~i3B{(!
zU{@U}B~GnI-X8n^zb5wo>}z6tUTo_SUwY98$1uWjuFtYqt1S4h4^ed&Qz|MiZt?n5
zh9}s-(ZGa;X??U}h}izvLOHojvegpRcPmt|I9T^;Qb?9kmm?*Z(AQ4la<X72$U-w_
z^9*h_yS@sg>J3|<*Oam-wwh~fMLQ;wlo~`0g&Z6iUC4)d^eF|i4pmo+ZLnL&OYh+u
z4Vq6_pQly)8Rbhb$%?~#@~r+v#eT|i@x*06%WuTO69dR7=ZX!-i#1YiQusb#Vg6Uu
zc=0hcbf^jq;@9^SW?|8$FRd;!(cQV`REja;aE8M9Ou&rQ_%JT5cOX(`R44S=urUgX
zGTcSY{n*C(`V7)#z^y}qv?&8^qQq(xO_X4PN}yH)VCMISSJ>htv{R@K+sBF%yro?y
zsqCX;WlGNik6Yvzu?8rN-=#N$p@x~mH{yIzg#@({+dRoj{y>1wL3OotCwkU)bOxum
zrFqO}7dER`)tW;BO06q8Vjr8bz=tiN!%P2_of~`g$&vofP|u+4D;fo`$V!7{qIj|F
zTiomvC-fC*-dj>u8VlBRV>y(maJD6WF*MeC3>usPE9ds_O^4VMUdf<O70G~-(Lh0^
z1)k*yVQZlwAP+KL-b>*vnmRQZcE(Fiev7=s?hjhy1AAerr|qmn=UKcn)lJ55P5ar2
z^=sF1yzbOEeHi4uoNgK5_R%L5yP@Xv1sVdPEA8W2-FQqv--;(ileOBQwS3c2HCg)H
zP3J(C65Ui+x>I$#q#%4Wu}`7mF%S7qd#PSsV8l8g98rstibF*iULM7$u$RkQiJ{jb
z*TibPT~Kx#a76N&6@{G~_E*jL_WF42XNeFS3sd5DZ|yHAETB}04^>L%8x~|@Se<03
zy1^Y#em~9{^!4;Ae-G85{`@<xA#|M6H;Zu3eeqJz?e<cKav0^75JF1vrXJEnue9{z
z2|?&eHz^X#(p*QUegCX?-P(=~?ZjGo+ty(*cXrnkc@Uvhep`mcl10l}3<iNtvFC0P
z8@+YYo_PD(HJypoU8k;Ae%;t29%3oE&-j=(HBIttXm9WA>vO~Kh4`}W{!$tE)XJu+
z36I6e;7*8tWG1_0GoQpk(%6g8!J`P#6M5IHQf7jgmp)UiLI!!k!6_d~wl1N3&8G@o
zmJ&NROk5~d5qMk8a@q%rnPw}rZsRiu$g|!`DK0jZ`9pjUmepsij1FryQT=C2LLO5Z
z%f1{ZQ5{!rc@?x?FQ|0uEtjG5!<Ubg^N6cGjZfwO$!Qyk%hGqp<f?8&`PqQ6Jfzs6
zf58w@Ai~3)B3&g}Hex{laYhiYR1GFZh<}a56j00zTae59ttfG{o4$4uCt=V++PM`X
zReb;pTLt}~jRr|;F(F0!;Zo@YGq2I}Dyw2mdeIjM^Lo4r>rtnKw;^SvH1v@RSz>y1
zO)Tz%>4{v*idkxz!eKJ0p%hT~Qh_~Keaojd*ZOx;s?MK`2%HPOk0oQaiVnq-i1gw*
zROxXkK9h`03i0gwWw_or*0oBD;B5dDAK2MAn3QG1Cspuwh)ny>z(|U>pa?CDY(_ut
zIkD2Ted~x1HP#}x@>{%SMUtb_t1D@4X;!IPW{dGlkZFi#;N>VWN6ics2*p)%T{c-9
zm$vBdFfN<_WP)i&PZk6KLO|3+lR{(27~Zx(OK*KkK*<uZs!;Q8s1NX%J6-SKD)7x~
z=-OaHjtfD$WDDO-vtXj6zk!QNeE*7(j1!|rm+h&4k<f~X6v^^oBEO-4-Z*X<>hR-N
z&z4s!yOsbf8FzvBYq;P$e)cJddfc=UpNn;>{{{g;ZlcFVe5Ls7MyeD2AIQE{eVwP;
z8%t1vs(P#JA(Yky!VC7HK(8bW=Av}HuO8Q``48hMY^G8|`!Gj>Hwnjnrle*0YV#~9
zDX=_6{4unc_s|L1^az0>dr<+rZBy06@+vv;+LR?I@>>deQHS84j9F9+VnPWWbcQ~A
zD?Fo^-|}tVwqV*&!DOy%bzBm2b;m7r$F1=vNvhx0k(y2sz6|f4*VWJccUAAkcwTIV
zl5nCR2&b)(Qj+f$EE#BKO8fc)fngR7HuZs5w0kvR0XAwc=YT7-v8&lGtG9~nP@gB=
zO(c*`gSPkttp(cqdRCHH5;#I)&418GjE8ZrDk+~?A^yRu{zH5-FaTBtu%c7QLhAU}
zQ(hp2jku*ilQ%-(WP3qhnWNy&?B)##N#2OFK%@+)VAY`--PZMb=OM)D%wFlyM_ks$
zWRB1!Afdb&l4F=Wk@7kO-w0FinL>;XV)ovQ^R-+kDM*2kfk<-LK)q;*w`-K~p}YmD
zm_ARYKUM5!klJV;7+hs^6)@960GuGNk_{$~%(d~!NNFS(W;JD%^5EM;ct`xJq?kuI
zzB$=)nKSWqXk0P)v9%(+lx<OOxX#{4vk%s-^B;jE?40knmj|j$efq{8uN~^X24RUJ
z6U6?t1H_j+M6`h3yb|1ja1N1%&H;nu1q~3s_)$1_13rN*>JoQ%>hlk4dwSc}cdhK=
zV*_VnY5KBty*9;bdcl&B;|7#5-g1LELYy(p+Q<yXYm@pi&i_T*`v+%vp7);5^S%$_
z1>{&x)LM0zShkgf)HoKPVC9sZ3J|uag;9_IIiV3Cfg~)9jD(RBny8DC$X>Q3Yt0<e
zD1juj<xJ@*r=eRWp;OYbTc*p*(ivu!nPsNztY_ALuv`9MX8D6T+0Xa7?)!P(S2*3C
znO!C3eSX}}{qwr7-`90V{Um9(H*88ac5XU|<Gkb_VoQv)Ios>`+rsP>K0PAsrpk{z
z*&Z_`yg7u<W!0Y6mm(2HZ|BPToIrwfvA&@Bb`7Q`g5`vZdIweunF2fxM51#F-m;<t
z3<p7uC5xuu$YjCN*Q&2QEt8D6fCwy~QQaC68dGhwxg$bXXD2c%$=_+lcnCU!VH%2V
z76p=lz$_X@me_}Bc`vSJwOF*RHDXcMB(VU|^Gul#jbkxSjH--aIj`Q=llyg;5kTN5
z$Mk5R%!9)Kd@|>*#>9a8xhMfPU=!LA4dGTaL7pdt1~CMY6J}uBrdO1RqVM&dO-dFb
zQL##OAbUd&90nIEkh`}@kxhvZt!M9{e{P!<YLl!8sxa(?ihz{fRW4fC2U*uw;QaQz
zxy2*hgZ316ZX#<Uw`9)UW*Zge@EvSZg|*snZ!?m&nR=Q*8OhKH^u)G_$q<1q<2<j$
zGue=oF^}x!p;p7kvXeQd!yH>oL>?$2&5W5|a!Lb!hPg6hu(3*#LQVEkQHL$_=OgZn
zS;(i6nx(@IjE#iOjfxT*#C>XF1m~qqW@b0}Gk3?v9oP3F{JA-mD^|lZ94Vf^t5d2?
zgFvaG#rCj@`+&(k&RW<8k&~G-)MhEu{s;FWbOuuu6yfl`PWleexSrK?&ugbvt&suS
z_(ePrf{Ii7MhtlHO3o8QV#+j&v<%s%dlpATjkaZAE=<h@><;9NoOl?C4=`-Nt2N_3
zGc_h>H+{z(15)-4Alkb4V8&2o%atA~Sgq|+12NXe6;xwmtf2Npu(_dDKlpaoMbW{%
zUFt8WgW04b<{~pu6}zE3G!_9(D?rw=9}_47FdL$BuR{}p(B&yzg@jRakd4}gntU2L
zQ;4H&P*?%I<8U~GXptn^c<4{=FeiW$K^5^hyys|@Y6Uji_Brcnp7b*6u36n1ee!@s
zL-N5I#)d$ZhUz=8uOQVSGt)aMM8+DyoNmfKTQ~@-C4vmf6cZJkP>&AhKFcHuG2(=m
z)mu*UT$yLPM2ylVa35eM2vS61a4exqV*{;iZXKXMlJ;b^;~}#8h_r5i4?cy&6TzuR
z{{nY`abZkGIB8-8MI^Y*Q<up#X-&Z|>aO4zbq8ZkdZ8-)fF2Se5{xRk`$BcJR33C~
z5l|vk63zRhN*FK?_5pLW9xx~Cfwn}cyFK4D%2<$?R4%v+DVP{+3P)qZB!PhaDlx@d
zO085Fk>HjMt%&Mq-rRf7h$H}_z-*x(HojMZGR~@E>lRt{6)g(ri&1#l>{RXtX5r!_
zgiy?P&UlE+bKauXBF%DXv@%mcXfXyUvpt$57(mfzbwtR<4Nl6S+_`-l+(;xbTQ+rW
z#?7Y2XsmUWqg$S?9pq+dw*p6@cu+Q^xzQO)fSMJlMoer7*&LzUno;#Vpm71Mw%(4H
zG1m^fyuG_d1y!*2&dwMFh&3$Q+95eyBXjp05cv?P&Ip#0bAnil8fMo#4b`S%3dF%8
z;hNq}_|55&0+?aynW96ZAQ8Vn?dhr{V>loUsQgWrD{|+`MK&SO&os-J?&vgb#)yeM
zuyT;Cg=Wm4B&I4DI^q?A>pAj5b7VeXg_Gey;Ld;pQ!^LjdjTcoim`##DfXpYXN<->
zg3%I-0f2q8*sK628mVlfSREl*LC5AMIsyv}(FekCa1^uv3%VAEW%;ptb^8uu5HwY@
z^9@ABS{!w*G?N|y9pk{zp+krF?xV^r<9)(-D9spw<;1hJ#U?eWH$f|{teuYKAWa}P
zW>0SQHfpKCj0Xb(OsfQuz&k^m?#RJ|SVyR)CbWl8_P0#;CFKQf>A}}HYOjLn##jkO
zKBy^iY??^UAW-0T%mtG_m-bXakOnHU#yBnjM#SBQ_!fk;@lg^TOUEsrSW)tYnQn=8
zgEqB-49Efr-oZ1<MgFSnLx(LHMnG@kl;Nf2q6bO!IjY)8f<b(|d}m3m&Ppvv&&@|b
zby$c1qkEA!x>2jcd?UUXs>9a?-Hcs2eG$4GY#rGDQlfKg0CfWtRCLe;HZDNONS%Tk
z$VYPvPUXtk?!p#m0)fpl1LVP_G2xL?9|i8vC<6+2*xn83_k@EzL%XRJokI{kHS`ho
ztq==Kj7k9!QS&5qFj_Qe3TdD8nr;#A9U-Rds5El>#^CNo;Z&pL!+2HI$yTX}+mjVQ
zI3h^77;2cG9EOdrG#HLVXzkklu6!5gAM!4k(IPUAG9JKZVpJ1+m`(|>%hT0>KuGcK
zQz-S@#?Ebky%lvqO9aNg;uHx7G20WdeG$Pm$!2x9)1x>xW3@W67YV+GP_A{yWI4<w
z0~_W2(o^`o2)bQ<EfNH1|3trE<6YrhV3{FR8jOud)+Rx9je67rUN9NP^Jm_)^Ji?9
zmFe2km%CEWJ@{=rmf9p{q%Aaqth*KiO_cNWuz0oxt!u(>TOkxX+v%@~{}wtyo^^lG
z$VlRX1S@}dPp`aP;lE7XIq|h5gmy9~eg@3T(FNCd2NY+<Hp(9fvSW5YsZ%EsvN|Ch
zY?k0A;Dm5Q=UwOEp#9Q_+X9;2qfo=Lu5nunnLuj>7x;vcleM#F`-V-g8f58nbG2f%
z5cm;_ETAFRW%NVr_u>12mwWCiBX$5%fMSnLO9oLtfdKyr*Su&j3-#rM>A$@B6(j+|
zEtq;dtYd25pf2)>mC{lZjLdJp=(_@iR<fz6dCj9_bn-=kT~-YyN1cX4x(u~z_6|>O
z5M+U&Hg2$7pph6@3ouXqyC(RbNa&XXitQ_75;ToUJu3GoAev!<C5*Ge=Ug-au1?#D
zt};hLj^mm|d22hBK1kq3a72~wfg<-%_UV)>3kEX4$8!@XF+02|(qRh8x-w}&ahMR)
zvLrq6b5%0Cw}N9GG!i8itgTV5>ICI#kvWvy=NNa4UIX^#h6Y)YU@nV4fFk>r_z{_t
ztRA@1PP6m$X)rdIkqs2`bn0QXw#yUUSWF^0By|~Wc4lwD_nlG_m=OZqDE_;$Drwf-
zCvak5HNh~$I<gIpVbxxg8-XueAf*ThiX%tdD{1R~XK_JL%H|(Z`BS%M%LHytoU!~Q
ztJZFWPH%NzeDPz*p;e|S&9Jy0xLgD=>?e?WLJXk3BoGOJ7BQs6!r~gAtU>ZYxFJ=V
z57QB&DKCn&5s&lsz8&IbyMp9Z^;Qx`jW$Ngdl4Nlg%U2~G|P@jUMAQGk|5v`knS6U
zhzLIEb)Iy_(Vt0FT6_)2S00W<Gr}IUGm<Q+zQlfMj|{Ypdy+{o@=ihYvk1B<Z^{2f
z8^E+<M#vRl&5|FM_c0Cvq@q7Oj+@OM%%5m+budVkVO5i`p>w^^S`dmR5AStaXUI^1
z<v|Om5?CJFN4bX5kh+IJnI(`p$+K@jVWVB!a8q0k;o3GF`mxbNj$mpe)KGJ)xknNr
z>2U`H@`NQ;tHvgoCLHqbW*x;;7=h23eppk9f^1xAyjd5-32ijj6}m{}_={D53)C($
z&9WHa7_;X7FjeMGCEP#41ZpF!5d*ov`3KGMns&>T8y#NcrepSPorsuUliuzS)(|nI
z18s1k7&<u<05xlMTuM|H^lk1P=-a$y)7I@|t(Agsee%eo4UazB_~@g~sv*L6xZJ6o
zwW?LQXpte*S1^o6XPS<gE~0cV;v}jpTSJzhP^`P56kMvL)8!09lwz2D*`4UgAV$MB
z&My>lyqt&HAJKdEY}@qmmQ7o{utI%-Aaj=wE(&A|P-baXjRW;u8k5Kc`>9f3dN#_4
zYZ#?l+#8Glh9J=h3ACz|{=WEF*{2j)7^tEVY@sz6`oSQb8>yFR6Yu&U47@}ib%y1@
zsAN*TX(KT!39c2oBSOtn&M-5U$q2YBN<apItJk`JVKA7(QCwNOFjpV}Y|(Ld=(tEH
zp(2{HpX6pMGkMrop+@&cFbatQN?z6QzH)If&1n=sgEF4vnV><5Hs*h1R8CKjC!oip
z9PSmf%P3RkLjCl{t=nJgtIA2pJa<aDI52QzX!LF6K!mT~g2{c~;q^g3wt;H9+<qA_
z{K(PQSl&&YVm8cBugw9VEo@8x5#UQ$-|#MMfwGMwnb?=b5lIl*;_NYlqqXMi$s9$P
zE_4l=S~GY{*vThWIkss~$)$w_Me7G3C;kf~V)q2(;<IDwA>P_)OQcn-$#f;Vj5u;5
zoU4;DsS}m+NSa=-T8-vKI@*bWm)erOCJx#S@GmZF?fRHPL<qa^K0$86E#FdjN1!$&
zDxT7<AJM)mB}Q)ZcW9EvR@8xU7Kcc1b65;VAqgdWI5!jaH@LfXb(*)E(KkR${FapA
z#$EwmMj=e2Z9#ozqZE4fqx$Stw8PPfPa_?$=`_K7;3c9oT|tS?B|349oBkXMZ|t5G
z2l3|MEv|}9;^;hhp$UXTF1ZEUCPc@hH*uki(Tj50?6;$EHa&q95+0c%xKPH*{fFFb
zgUOSV0G*a$(#Gwn6cmVR79miK*%Mgzw@f=@=eNd+O`8%j3yIi&gv?_f)hwum2n37t
zn@0}f3diO}dz|wh0|OXcR!IYGr{~!%f!9nGXsO81TN``Q)Af4q=8ZT*f^&Ou=_49%
z^UIq=WC+XK_^KRn{M-t;n@BmN%o8A60$*_}6M(j}TFe*1GXR7AB?qx82-FEdcrN0e
zNw&<xlN({UbT&Ya-k5)3;TUscr=$5kVWe7d$o9dpfEX5fK)E-k>qn?17*8Ubu)sz<
zBz)PL4BP&KhrntWl)ZFaD~zAJXR%Nx4AxwE19s9<v9RKbSl1f&3a~5WkgO#J8X;kk
zgK9S!y9VFcMygRcGzli<X|XAnL>H_O`Fw=X;K$K$!q18!GP5+|X0Nxbony)?<BAoP
zSo_3a+Wc-^hf*zz1Ou(CTz$CV<uU*OU2XE#!hTU+^70JX%(PhN7aH6{BB5v(?c|_?
zaHuMLrFr)YQJ{Q*%h7>TWF1o)cqPA(Hz7Czs^g;j!m7g#6^YLso9Ik*Ys#Hh*kztw
zKJxH7X0^Bu@E^TknB3xB^|eCk+z#G~-k`W{_so${!DJkC@OFGf#Iq|OauLY)eRNQW
z>?jbwk3)QfvL{AKp$FuTz8MH#{;3;h05h_cpko~6R0I;?lUf7Yxu=co-Lj=;OQ%VB
zQfY}VR((fml!8IX$LbKmOJg2|aAX&20^l+Gf<a?OBu1K61A~^Cl;BGog9_#c<gR2A
zHZt2m8YH3-WhWs!X4ucI0~SSSL|AGKfg+5c{Jw@@Ka6$LrqE*N`RpWLbgtiMlBVw&
zmnJxtE6y;~w+K5TT%_u`9U2R!wPdt*@>Q{5l3C+I?srpLm~K~S(i+Id1RFC>VlRIL
zLs*IdI0c`Iygm>_aZSQAJ5Jg74rSJ^dWVgv>VPawG7$+dVc^H`l?-RnQFGL^td%&<
zptrG*Sr-CLDI_(91@Gk;6?9}FtgIyDj!6ZJi7tUI3d9%8`=+VvhW8**o(#Hyr}VUv
zQixb^eV7Ad|BN+Nq%Y=~-fmI=zxGHPV9lM`pgl}b0)se(Vp<eNXkMC85Y<TXGhsXr
zV>E4=8<0{i8J0D%mURK^F<W{h?j>@<A=T4H&^w4&`lg8Hql=*18JLqQu>@e0x|)AY
zCxg!3j`f3Q!?87(b?t~Pr1TJ5p1W><(W?dPfF1n|=meZL99RwPZ$THNa8SwdN+`p&
zmwLWndq|7r;wsAqCxfj5I-qrtu~;00&hg`LGDd!Y5D&x#4zLf$A^|#5(MM!$KBEE*
zBmFdq(7IR0<idHAJq4~$gyyIe>rz*`kFupk5eYXheC<Zt#NsT?j%?QX5Z4NtH69L$
z$b3DN$j_nOY7TLnW~DT5B|lDFp2GM;S@JGHJ0^0Qn82cN8WC6qorXCVdTP}L`Hurc
zlaAJO`Gc!X=ark{J=3$ZSLmZ_vd7xq()Umn*W%{dh>&AMd&f4bEC=jFR5IxHTz0@_
z1E5tUc=gcUTEr0ZaRBt_Q1WmatAUnHzbDQJByI{9bY}yktYhw~+;K2DAmQVuWSQ|0
z%dF$Vwgj`GY$IKYZAnd_d9lZ>C3go2J2Gz@!|C5d9!r+JuZ<3`9?$pU9@OO2_k~rf
zYbKt<B?d<n-@VrmVX*v}9?Dn44XL%qs4_fYJ9ec_Qjmmt$pqqV3$>K=LL{`36_xH<
zG<?i~MZU;MbTdqy9QncqmXvIgf*pjGrvm3tG@6L&$&iu3;=93fIxKz^^P<4P#z4G;
zagR_$soTt&W$Mul8+t9b-S%EPcDomwu6ZGf{%NHa<RrHF8h8V3KD>%3zAy>;1?7J*
zz!!YQJTsGrhshXkXa%Lj`f5k+F>E=++t!@+ZL!_GU7LFfAdAS{HXZ(ZuJ1!HY9JGQ
zqsUB8tU;W4bkbEOt35KhPP}uF)4OtOryYk4>9SA&SGaa=-InO^q?dc8&jEoO!m!;h
z&n)3@%RwZOMH#sz&XHZkv<9%bhwYii>qThOi8sDF(ifWmxuxoq3yv3pb3BaqEvV5#
zpMoD5x{@QD9vT>Ev4kwAz=!2hK7b%9<(hzxSR{}Nt1yboV$!pDBf5n*?35kMG4d%6
zhm42$dTFh?q2vifLb2w){bk38e5_iFzBY-+cSzO?N#7;HV5k*t?mZYz<&x}as$Kb`
znH^2YBbN^gdC{-|32CH6Gz!25Q^ss12e80eJ0nQpaL}N=@$8q)(qI@3X7X{!bxtjy
zjiBGuNs!X#92#4_dKG^hObgrZVZGu5B4^C*GTbyia%kV1IOIz5m0T0_8vFO|m;58y
z{;-)mU^T4;fIh+VG00dX$zv-;xRaKy2x!*b8e;@yf2b)`74Ac!f~nRgFD1h$tp#q(
zhLj?()^7;-v}<Y1U2AN94D*S(An#%oz$=duAuA9^(yyp#4;vThQvOVF%er2jbV5%z
z7@=A3d_*_=j_L#LIQtt_8NNkaok6U-)AYcqno^#u*X$kf3W1lkq)H@m3>;pxQ5pty
zZr4h{pZN|EN-Vt2t4h?)6^SXq|2hZnsNtIGsV{h1%_GVt#$I(aPeddpM!?~l$F0!_
zL8@{4=6Zly&Xe9G9DVWf25Tl41&sMKdJXnaj=QQ3up=zgZUgDI2Rb+PZQ7_~c#IsI
zp2JXvZA-a>nvWrULWQ_lP$r>oqq(0&Igt!`GH+t-lb|Bttde7#)~H`F)jF-u=^z}d
z#b>Z@<X5Fxtzz^R#HHmY1}Q#`NX5iBQ0epKA0yGnWYW!@TTm7656DORTgqyp{$Qot
zt=cWWp297zcvT097>ZeQiFKNaIdsTQy&4wdBcm_fw@x@|!2HuSvwhn(gxnQ7M-CY-
zfI9;dm@}bqVA$DY*l?ak>TQ&Jpi#J1n+@;Mub2c&MmW&3RYzlOQ@E=v>Yy><P}l9)
z+6!~ZkXfz&cBN`awr8{*(yc^z7L5@Yt6nL=e7CX9wBf3D$`|)Y1_KTtKBm%^^skyA
zzj;if=)oxFh6%!$fIRc?iOe^Ph3EZ|$^Gp(@XH2&=rBA~aH)i>@Fn7cP;9|m<_Vh&
z#C6Ckn-Rgn%EWWlVD8-!!=N3G9NEHLOwA>F&k)CF$v1irPNeR-qRcc_p=OE=Wm$zx
zLn58>k-aiID4?To>X%xK$rdb&07y4rPCICS#Xw>tG`ync&`>GZobPrgYQ|mI-*Xfl
zu*%z#*wIT9bZlhC#b-NgHP69>WNYR>Bogf2fdo6Ks7d9tHEn|{rA#4EU;(L|qKyJS
z9nOCX`f3R;bk{POb{?@VylI9N2qw>631K{#;EbtPTh>Q*9>U-xfiPNpX_x>6W-U6B
zA)3l+^CghN#{I*c90ThyDlXmbufh=l@|*=;$To-jkoev)y74slR*#_g7&`*IV1q^v
zne=VjuyLcb@U-dNK}am}CT>ya#%)c>XAV)75Cr*vp#XUCE+Ei1_&lj5i#E4-Tq$Op
zRHF(v!s0b6KVTkRBwrZGd@?ZVTHZrMbD8{DYu>T^A7Rm%GILZXfvmR7K2wYd8im7<
z3plV|uvRrud?AxvL=r~PY(cd6atcIO5(yP0<)caS7Ua$XDPGl2wo%H1&9#TGbi3nY
zE@rtz6Ls0<Jum@&G~KY%AhV~s+gc5{%iJ58C3V7(u3LrrI-@-<Y*(<V*4G4PSP8ja
zy1F)ZIYI!zf~v4_^U&)vMt6J?2T<9FL()Orqp=CxVIcGiZ3NCK8Cy2dHo%FyK?n}P
zQEwZGWd`#D5KZov;=@Q9rV07Ed8BeMw+-w?KDMk?1A{Srb(qLPA~Zcga5sft2o_El
zUwC)JJ+6G2uvWB{a4tm<(J5pE3%aO6JlVd(B9QoDK+7l;4S#r^8Zoit1cD@HTv`jy
z1S-zU(j+8<;s9aGYB>bGM^gnh+YH?G>?5p<l~L%6wwU!U@DHPh*e!B+hee2LK(v0^
z_X6T5Tp1A@z+h~oS3Q<b!YFea{2|QwpY9Q725&>Ag^iXgGq5M7*-}WBg>7p~LB*Br
zAgrmQ2*dtJusR09^d!~6xun9~Yeowh>CoJSJguhW6uT-Ib5R!crK0md%;+{290NuC
zDMxBZ@q(e0oMQ6lWA|7d>4gEDU+i)N=mHt>QBw@57yTG6t)LDOVXaK1z)UkHw#}Fs
z7lKRII0xHEDU>Id6iX8r`bV9;dP$|OG?Bt)z$(H786^XYgbs_PAmpc3cnIl;1`iT?
zDa)H9S1}?22_YqLDqOT-cxg29^gJxr@IDKFH3<~7VjQAk?->+J<Ng!_dq_7x&U>4<
zoKm}nKT;#%G_!uJB6B`)4i&N7)-1kIn}Z)5frw3@T*(oTtV4N>>^DI~eG!xfEojf+
zC`is^N@rSv=2*s(UNgOHluAQkrlVqOR;>*MP?Xtl_{ZX+<We-Y?+^fhPnk(0VUvp%
zy0@)iVXdb=MAIogP)QlKz+P_)+@OG$<wUz7EI>!yZACjB4sP@j&2^CNf@^Z{6H@Ys
zNS1}M=!x@@!3`&<LG2hXb5Im*92;dv*LqK4Ix#P9G&BAi@7IQVxk6ahI7z#xHZh!=
zP*m-~WvT*Gc$i!)#kN4N+EX1aisiTI6k#13#T&*KTmc*=T%b5GU6>S?8d4CKU2}!+
z0Z=R71KEi2Hil_Liy@}fgbHeAxetwxzauRe*gE*+8Mwd#DrE`lY5fwFg!CipPAJJh
zE!U#FK>=_mLo+gB7a0nIZ?90ETj8R*M*jKoPc&q~P!;D}WbB}@otDj-1+G=u5=^>q
zeCj^7V8b`j!W7uka6QU&`XHNhz+0}hm7;|q@k&}?BkCHNpx^ifPp+<+^aNgFI~h0H
z_rN9vL}q4dbAoquO>hV3>_Nf;DHaL^PtGiID3>4hp{~H;c2K$d>!2^{tv#a=cU%Gu
zWSL->tXUAPxhkSecZEZF5M81}QgVYlgBnP}-b0pnPuZP-MKRhYOX@;8z_&-y6>Q+-
zUN{1=%#wMG9W<JUlEt<tzCc8a;GRuv@}BWtp?0^^OSfh!6@v&zUpn=|vlRyz&3&OH
z?9|%6ENb8M91yf3J_Z?v9qt~4LRX>UD$)(hqIlxIt&fBh>xj`1S3GEG_Ry#9MaRJb
zkXsGNd&KQZ5(;5Bf!i24iYbxASV685PS1H;_qeS-aUrUnOeJFW6I0Ne$^pElqygex
zV3Q9r%YI~Z)b8%XF^Y&TLk4cq)tT@dJh5*NnLFSea%-q8P;|r$5&)j*Y)}<hW;pa*
z$=<h3CfskQHqrZPL%A^^BE4@D={zQgj^`DdBm`qI{HDavs4N@Jk5~=3E09SiB2pEA
zHURcfTpuvy+DWt=3S{Y5ZKI*j28E3fs^uC|+LkbSkS(W7d`f07b%illCWMeOpqfpR
zqdsTK&@7A<JIFDW5hvSGW`4C>6QPht2<MD3OhGINYw5rl$cgsd=unTl3*$$O{ZT4O
zp|D(Q89pz}Np6WJ0S~^N`}b=df^i+rD>}C1p1x+ivmv>z7~UfXA5@c~bkdqSH(#)$
z!T%sCVcVpHl@ZFJ-E@FDS%HlHljcCMjZ;S7=@V2TOemMREM-_2w9-0L>g|c6fJA5T
z{ycQpL{Zzn+R?(TqN1&}K3R@?4fYH<Czvv(<X+{+`huWG4}voT5|yv*U^E_a2=iN*
zIN$<AFicUo$!s~Kx8FC>3Y0J4nL<(mRx>E+O=9v~>i!81UY2MF&>7$Jj+9<iI?qTo
zawR|NPuIK!>5bvR8Dcyn(U0*7L2|IgWcH^pH6zJ-atc$Dv(q@lQVhpyTL*e{3L>sQ
zJzHPvjY+&j6il8Fb7JF^ja-f!S_r77P4Z?ievCV@2f;2F7zzlYf&F3l;2NJrELr+P
zu*1HKQU1`8;SpgS!6vh0sei&TwQ_k6$w>k&8QZsC>fggUdX{wTvVEAxwzHEta2A4>
znLMGCVUy(U6?^ux+t7%zXy!O}6e=s&5{KgZ0KKJ>SG!xWF;Qgl=@~Uo#%h^als7<%
zSTty-B?{cYPJe|%?FGM59XZS~MvVS>*01$C(i|%E7Oe8{Nb-iUcoa33IG6hIEA`DS
zYYR*^gbT0uoTftEB%e<V?UkuU?;+i9kpafCiHHH30(TACxfB|r5P;4TA`|WrcXg6F
zN_jyKd$0=1BoH9>QIkYNZ}yYu0R;oC+KzG$kKi35Mq=!ZgYmRIkvyYiA!;O#wAYK^
z!s{9#8>be2VvSZpvGn`|_MVgU+{J=5V1D$*><1Fa0@61vm298vWmK_3M}hvzoI!Kl
z<CE*9kz~kp9$7LWSI^d$UltL4nIv{SecR;8OH@FJNRX!*sot?vEyzI|g&Z5$s<l(7
ze+<oVdOF#~i2VR4F~I){0=;4D)<~e`n1!+*Uj*0A9c9|%cr<{JMn<Y9Q^&b)hM!vO
zqCC5DVDI4Dk0f>LT;I2TpbO19dS2CJpEGNwR@MA!VvA%EisBri>jF?9j^xi`h<$wp
zW_DhsO!3MfVo4kxL2YsXi)n4p^_hm(v@C20S%_SSc5p_4s7`_TIUX3Efhcz}9F6rH
zl7}!yT08HIlhEj-k;|N#C|Hqvv)tqSZ*wN<;ei(c!I{jh{NYHSzI!zk@qb!2ylpUN
zilUf-!^8lF2Y6JmUqL06Y&-HMp2x6_Xla8Agr#t{%!Sm|Mc7#glvX2RRgl%RIS!E`
zC>h-tVxmmj5`!LsyzVn%XX3aR7Gx+ee}0m&Fp&ps*!s~ObSah^TA9ySO|j^CxZ!$~
z0eIoUnuc`ag{^h~P&L&zpjBl)2^%qB8G7_95>Y$T<3jrSG1d%~*1le1T*YGu>M^9_
zk;U+BVojo;cjWIdc0=-OAPE5ihXz)o%!Ly~{T4MtR(-V&KWZV}NK9f+WlR}E7caM(
zt|NQ5Vgv~%@R$pgP)p&_om;QkTGtpWBHHAWPA%;|+lDffJ<4Cup;ahL^2i%yur4fE
zz472j5BZsZqD)O1PgS(4<$>6_JB}DJQ_yN=jhQtZZk%3lj5(AX5yp<>5qnq_EfE9d
z{y6k#n8T^D3cEnPes$}s?_d~3yQV~`9~t*&IU}RkpIZpD0BnLNA)4A*>*czCNPLc|
z)#<9vJeoJ4d+Tej=rAgLdZl3ulCld0zTD1niiA5BV9N{BGi|7<jaPJ1CdfhR0~Xk`
z_*6)Lr$n?e!A)8A2z6R)a?-nIn-Pf};1CLvp4ylnnd`xz#B71#LC2s6V?CG~ceBZ<
z{@#5sN|FJkq60-K$6Qbt87rq@XsudpK)ND@2ug}5uFLQ&`fC`AjA@)njSj+jTql;=
zwI%?u?Umg}wfo6T!rJ19O(~L3wBWWd4{uRS)DHJqsKWkc8W3aXYz|y~1X@E;2H|KV
z8JcZD2)x*1#ZekDD^NkCGf?XQ^-_|7sNxHkUwTImb;2OHYmQ9y#&wu_v|0tNwTyxf
zYlIe0I9u8oxl2r%f`d>qM2AC3_oi3YlP@;-ZbN!s=}jg%{1;)#NgLWczgR8nCd!q`
zcU-*~G`#_HYCujsqtF`kSQuSFiXlBNhRB|eI`-P~LIZ3Jp1jFAN0Rl`4{;f;N%e3H
znJ=7S+ql6QP@fBv$qpX@B5X%j>w82STyq{QZ5~kSFfOa7LCOSRZA4yE$J!h5AOBj^
zZR%BEV6cZu2kzuqwl%)copG(Ef*24Bzi9_2f_owxCDRC+!_bB5rF+{j&%m&8xzuG!
z@?ZdAz4i?0c()@8qYNkA$d)N6V9SwD5c!MLtFmaC@WUdL2Kor>I$Ln{-T~CMI4_P2
z#xNz{m=W;Ap@lBRN3i6gvC^t23R6oLS%5)FZ9;z_s_2Obqhb@IP$JP9JC4bKA9amq
z#}KE_grQoeq}v$8t2q?z+YjE?L6p9Oj16W}n{nMNdt@&V*P%A?!pKEIV}tA)>1fp{
z0|EIoRLKTf2xO?O+qV%v;faU?Xwcn*kX^&2M-MYDkH7DP*~UO)xI}b}oK4_c5E4Vf
z2s~n)&P{6$nl{sgmmm_8!C>$ba%X5)G=zgEd`p51vGN!zXl#{u6?VMzoTi9S-?zWk
z3E%qKrq0-TydccFR4@l@ZeP=YQn?9d$Dzg%%?w<#R30V+SrnXbopEfewKR}QdZ&bk
zru2}pQ3Bs6=Z_FL{-%QKeR<qhkQ7TI_CSiMb%N41KTr;|T4Y?*L9>Lbp|@5pBBF1F
zzN82z$C>iE*}ZWxCQ}5(p9m3}RI%Ct%iwiRZo`0o*z>sT``JuyvR+k%zKDp<FvFU-
ztSy^kk*d1aEJ+cs8sjPO6OS4Yoh_e}Enhu|IV~P>(?aerVI+<XtIi_8K=+dE2lw=v
zk1tWYo+H6rMZgk>XE;@nhyZpF(#ssNydk|izL2=xZ1KoZhti?Jy?bD8U9qdl4$By2
zU{+Zxg8G96WR(?F2j7>oBAC-)nSdu@0z}<CnrBCx#(0ZXvc?wvFfxit31vDU4rCH8
zWCw<^?TCmlg{|g&uLlTeV?b_9iG=#iwwgo3xFQpA6A+De0Eb#CX|0W6%l={J9mX^m
zW^8s1>|G&8ZzaLHi4RQd84%okPw1ye&0*nSWJr}r8g_w2)Hr<yT4c|6PKrTEJdK#m
z9MSAJC~*_f$e!m7BRVCf5K}yQ$R<W48`}W|s!=1UaO{v;mfe}B!v7=MtYJ$ms0Mx$
z8Y;u4?Gn@4aU9>OFTAuLpS=QG8Aq`G;F!5Ppm8wKTISOT`Fuda1MSS{gq+|Z#0ie9
zKM2*%?4b>uwHT=##=%UIQkPtDFcGSL)Rd&5KDlGEb!`jpL!}2O+O9WawL`NQOsw04
zppOKgkuk(kz{nuUdSPG)$3hba^f|K|J4@;lx<U3jd3Gre0hVr@q_A$t1WuO^C(>0N
z9_Z4U^3FAqG_wmj*MrQVY&{M=+JqCF!L1*b=BTBWv~^vtN&r@~EHG5*Kn=(WA3o-5
zZ_m-gTi;yGzpos^vqikT<|ukwDu|S5vwf4DhtV+@={LKat(CZ3qiQUncu7yL=`_(i
zw-rHCq^PmES9ZadLv}%_!9WL!Jhn_8#{3yGtc|1NA$^Qm9|jWKRp25)cqB_8D$6qi
zGP|Lq;<(+mgX0s%Q{`4b_pO<z7S7z#Mk6L-j~#k*xY_>6Gy4dwyCFheFG$}oYiuK|
zPiSt#5yE*YQR#EkVYql!42(fKZ?Vo`7WyAJAa}zsCQ~YjsGv5{2GGC*dPv9xgoUz~
z`PL9UokVd0)n2mW0?F(k92VB(fZs|c3!yE>i|;iK6mi3=^&p3;gR(yY4DL|ELD!lU
z7&c}WBf2pPu;56l4A#@9kx^9n9L<G84qLYTbU-*ooL(TMa?@g>2M1xCwS@4d1YXxz
zack1Jkdis_ws7AL%x{~vgua9w8hm($WhAkERP<DI1ck5$C-MlIb-z$BviJf!4AFd{
zbSsOqIVvXmP1v>V3=S-u)k3UU(2ol%B7P9iWZX=R;#UXyDW<xIeGRTdRECdsfdku_
zA{8uCc=UtD(^p+Jpe{1!Xx?06!!>@txH^_gF_LiDhMTR21?LD%b&}a{aB>6o0t{E4
zqD(|U21!O+K$?IDR>^q6gNRQcVYEz6+ifq1&s5}^@t9KHFmof$pi74GQ)Q)~2&VHG
zb%;4+0bUqt{er2$8B92O9o!gDfsyikI+aR7N{~*)BU7uAKTdl5N5q7C_U<)fxOw85
z|B2x}BgrsB1qX+E=CIspU56y{?AbQW)m#48rWOz}t8p_&d4vk52b8}V8wuE8HIArS
z1rT~QZeqd;Tc%t}#1~F-ST)~}2L-b>kiTzIs5ij|MvzYMx0n8Oa+3Wn0os&2ZatD@
zoXg7=@9mh$7@C+<*&LGXXbWd(+-D(9JU$BoafBL@Dmzt>A}CU7$P?Kh4j;96LUwvp
zRt3zS=JdR|ck9NkfeqVVei^Tfm??B{=xD1PX>r3a_KSrJa;I~1CY`BaMQ}M_yP{k<
z(!`ChvNF;SV;+FKiZW>5Sp1D6J?2VuUP*y;rj6Jz?bj%l*aO`Ft1!DD4{}fp1qMl4
z&!B;E4`b$iV2^^@A&+MD?laxz=1xJ{%NvZ74APk&`U<Cx?m223O$lZGrSgyWBalsh
zAYfC7jpwsg2%4;DwQ(<x`ZN(ET;<fHeB&~S9wb4PlCt{xvGa8FO~5}Ij9?_snWxM|
zA{bTcIPr!l9<aG;YZqZ1?zm;7#sZH`2oGk7!=P=E#X`yVnp<sgqrjD}Hu@_fFC#cL
z2rzC$Kqe<CYw?mPo)@7Lt*c1c#!3v1$ThF(v@~sZlJvLPxU_ZHb2`Vds+KRp%yF~=
z5GzrZpCQa3A(o}BI(h`i!`L5Rt##T)a1WB1i9wGZtJCYUiaCY6Ga)Uu*$vUawVG5a
zstS<D>?)D4ip7)AhUW_~rnoMN(jgcI+ARkb3(GXsm4tCcA4hIl0+nRX-VHdwuYav~
z{YJ}5>ibx^RM^GsiINs>+|ccAcxY-clT8waEZ=NHAw`hNTH=MPBJCjK;gERvSFr{{
zLoODYzaVb9%H(c`TJ{7X3jc0#sUAPPmaJpNHSA_QXu)mr``9BUEn$0u<_B2SW~{|Q
z<TP>xIJ?dK)a(c?7t|fzHte0)1Ts-*f}pyJw)WcQ=rfMPwxPgR0hUHO-JL@GF<`%S
zKM5fe)-WQ&1q#%Vb6bp!WP)4=OrPT}2Kd=l(EUISg=P#JfpJ)<s5O;nB2W0J5<^=|
zlEru2kLM`5_>C}%6h?f4s>s<Y=pCsXwE;=$;AV_OTaXu2Y!JYHbblGnRUhE2tZ@j<
z%i_z!>YHq7UMAu#lvsU@C03shF#@(kFpPPSx-=T^HKTfgLXj5PLPB>?vycnQ{3r>T
zO^gac_}EwG5<{4~BT>V(LZQ?mLy-AjnkQpH{S=raM@sxD6`$n(lre*s>+Av@52AlG
z2Z^T|z+>EzpxqJp2(8C-3ss_1ER7|66SB2Lw}C*AMm+FN=8O`SjhT~5RcdY1X9q15
z2Osz;w^iJLlr$m#dT3CfON6PoC`^M~Y2LIRQA=2h+B-LC$L=7!rc=nqU5T~c!Q<*^
zuAB%gJjQy?uY(OUcVG8Ra3LWlfzi0)DulpV2D=X4^~foKvYC)nl>@!@yig?xm8fP}
zLZx!W27cJ1tX;aIznm##S_?5eLREnR+U^p26G9+aw`7~=?%2vaw7pX6n6_$7SMcSH
zjGD95h_-JyD;}mC2r}RO!i4XB!l;8>p44QsH;Wtzx%$sJP|3ec3(m#3YHRHX8Lh_4
z;kYTmMc8Z@CZuhG*Ej7SslW3oC*DB9z#M_xad?>YSVPsOc4Pl6z8XldFcm`wra*w5
z;Q%ekrE<A}ZlG)F4TfKq$4O6a%r|2%CvA(^v0ESqaul#sxpqK<rdiZK@W4UyZO)da
z2N_tqH~L&fp%Aq_SWCOXS3f4d-`JTG39O?APzWNUGLbrPYJP*sbH7|6AU(ZZcwgXk
zyYbaceL}qshVuG4v=^L{8|GY?kMZ9&Mi>Vj=>k?X9Yv%QqVEM4pu2*G=x%kSP(k!t
zwe}4qNqh6i7!o}4cy&E((M8_%Bih#hk=ycJZkI+|dAr!6reu{9O?4P0tYxIY8oeC)
zm^CSy#MD<gMzT2sA~N4#Vh=b9>zOoyg(zi96>N&yOn{@J=us|+66=&=D5gLY$TX=<
zcwBx8lAEl9@V~;O!Z3vn0xD$Noq(ib{AgCf#<5&qGD_GwN}J<k#qgJ8a4F{VGk}=j
zc(5Fc%sVq3IzfLaI8Mp$P+chUKDJYVtcP|$Ok4ww6rvYZuE{|S4-kRU@ycD>+V)cD
z3?jYUYIN|ApgUxVh&8eD<hmD9f$}OO!do8-fv+C9YTFC!DB}nHW}Cl>_%y5)EY5!L
zN)${9jG8ioY!Zk7o&(J2hv9vTC3jt2)$#zEB8QOnT&@b5BY}rujo=pH(;?>ASRA}b
z_rcM^@*qf8Jz4`>ZXVAL&epr&*(NBn#h2>_j>kdkWw~;L)4l73oomEK%qaH0wwH7E
zl$hE5J?)evFpL7_ARS|#LV1h4>U*@d5Pj(sjWN)CnQxBiDI|RznJ^7Iv&94|&jnko
zM)7hG5}@_4yy3!y5sZ@IF?2GRh$XDDu5xHaVslRpPztzyB3X}vtIo6Ul>8PX2MKs`
zG?DL+RC}VNFyrt=0uSpjrt}@bz`I2(u+-nS_8T{_7=-*On;SV@krPQwr5!lVjCkh*
z8&~j}0sM;H2DM^ZB?3yhVoO?`H%!V=W(|iQdLtQ=1n83Q<MmonhEMix?A^S1>oyla
zz$n2BOT&aI8a0(*&J(!q2p1hkNsf>J8sk~R+DeVHnL}kJ$p~Pxg>VPz)X<bxT`=i;
zz;`y>CAc@NjZ5vu{5oR{s|91B5mJtn0Wb-gTbsfiudJ88x64U#Umk>C<f1jJjj!vY
zJ`F|>lCSN<{38)_V@Neom6SG;I(<t=Jh&O<)sab(jr(B^>PU2?(h!aI+C%3M^5m;1
z=VeBO6fs?B-C#niP=o<1sVhYu1OH{{)&Xr3K(Z;w49Yy$CK}+gC)#M<{-c-_O}$kW
zC17eS7;t<kkg(doN|6j0{x>)>e4@O^I`@5HkB%mN;|Jp`h+fMeYv~7EC)}&t+C5IH
z28h`Wr%EOhSc&A+jze`65aEAJoxVUd7IKQg!6Hsf+4AGsVCe|DhK7jGiG;y4apF4J
z>K?M;yfSJ8XCGa>k>*JdX1mDIf{iLrU<yh5I=VToW=@0Feo-a{jKtVOFe18&J=ln3
zgsW9dCU2W!2e0eUM#LqnzGyMRh=|gTfEUbAGR7)FC6&3Jtv<oIb;w9k3W@J9zpz}A
zF#-_NDHZ3B&<xlDGB7^TiIA7PB?i%qH9|sQL#frZs;JJS0ti6lyA9I-n@8N?D49~<
zkhA3eBRUnwGL=PFe2?1apkm?HD$${aO*JY8%NWj%My?y+BnH#703Yr0=VAXdSY{A~
z7H5u1t}!cxj;?~5)^I>Tg-5{_RuuFA`UA0fo)<?6<})4qr32`D_7B&=8}Vat&E;Q5
zJ1DRu9|Ng8g`Cl@?%#vq*x_Y><49g}CBy(y_1IXSv}<B;QF0)CX<<#s46J@29%xlZ
zIUg7-YO&8Xp07?v89C9~xmiI*ag+ZM)ZEXWwTB^>f<{2);{Lci1Y|66F(u#SfraqR
z9_+Uo8v}w|ft5f%LbwQZV0Vkfl1e<vH|7#z1k)@nBK(2aNeFUvxT_7L$<d?)V+G%W
zv9qv8IBKoy09f4jT&(>d$(N*oGqVClE$@3352Z*1ncGcEFQp>jD>pJbi<->2gQKI8
zpRr5Y@NLr?LEiu(?1t6rgQ;4zg&gq^^yuGaA`ClIVW@cR8P$co$Gcr=%zA@UgDfE+
zCgFlLq-TI4i-?#%p*kY+j13!+TZ%4|Fbs7QP9GEja1-Fo(Ft;|qU<(0M1x^HH09E@
z@SiyHBKfLIcghK}XSLxB)50gvesj<6JJgG_g&AdYyS5gkW{=4{BC$({b=Y}JO5jjP
zbA^ByB|_6y1c^{HXK{<#<;;CR4TipG<qeU48Z^5_h<@KIt{Lii*tj@xa1nc8Wso$N
ztVix}y;KnWWfq|#$+&jTYBlB1iX2VeG}SE1jEQ9H#1=bWZv9@wz*So?;KI$49@0^r
za$7)_TxS$yGS?*4iYz|t=m7DSaI&BRSwFaUfEHxYH@l~CxI(Z+0;cLkf;&_?86iOV
zkqfJClRnP)FTsg2zCpl(ux`sqhAYc%Xg6d6ukD?&S!{Z`r67VJVK^%AEe;48nBlEr
zt~`<5D_}?ReQRy>XONNcs8ArX0F6{?l7L(xDoj0YwTeZFtJbT5Z!q<2w-W{uN^b65
z--+E~kVNk<C!)k)OY`TIPzBo;jSc4d9Jog9M{+>&Z|*SX+@YaaCHJy|Qf21JBds?D
zkLTX@)l=tA+8#g<WC9t#sv*i7LDR|VDV!6w#2nd4g+s9fkJuji^hGpjkyjEmCdZ;!
zXM&82y&@*WNk0_nsK+~m-4%lwLJD&~4%^5?*cR+QF}6eKlHH;X5#@|l_Cv@LCj7FM
zbSFcu7>R1-g^)3|Z%|fyL)G951996gin)Zi?8k$cl-;zW<J<8iS921kQT)R$N;kd=
z-pWavt(p`T;G}9%aiDz)yW@x<&sX=oQ-x>psO*KR%_22`tjQx^vvz|WT8<f!Q;RWm
z;C>lVWl!h=HFnn?l_<~OXc!ZjfRJJ|!vpFNb~lG?ZAx<raY4Z#MQ5upPMx*wMv*oQ
zAH~VEPWNDk;Q8^tmRu;6ExJpkzgu`nbDjvOvop_tlPe+2zygjCQV5KI_`Rgpv4$qo
zmdH|38{IG~n1zf6uP`%BLH=<(X0*0(yn`V$cB5y?HeN2s$Vzx=tXe$CI#^24>Ps$1
z#i`^1d&buTd7x$sc-A7iLKgwf8NU#0<-Heth+7BWL5?`qf*1<Kwml>i=^>`PyL(gj
zz?Q9@n+QD~8zaHk=X;YE<;q2v^kKhval`h_-JLHE4-UStYSrgge}2^qt6Mot^QG18
zFOH0l?|A_rKQi?KZs-ed$42+^QPR0HFaq0qHyg;k^4j+1jjLX2_S%|bZOtoCHa2(d
znQY#8_|U{ec>Thv&o#d=9RI&{b@L1Hzy15B21%zz@}r677xr79fg`%3R(bN~p%<8s
z!8cw25h|4T&|w~u7><8sl64P||B6#15I^Z8>^DY*?Va97iS=OcW>mj$b^`-gb@^wN
z{lj&9J#^>@P6FUV++4!-=Rg0V|No;V%scm@|E&JJbZ&=LGrq>27@vJKG}^|^TYCJK
z9+Pf}e^w>`UrpP8(M;9MF-+(SBKt4YJbiRB2kgAGS2k|^cg-v4_2>T0E1l@C;{T<|
zE9+a*HJIU&WuHpzzmIzN*vHaD|9#ZE|3AL_SL>vokMZBH>P!8vcvnyV_1{N*_iuTZ
zzVsiHMH!zbNs0gJ?xlZqHA|XG-)YR6TGAnV!NUvJvgAxD9m_73F7uN=zL_N*>0oxQ
zbfJ_ECTUW+m;R@=N_ojIPv<w;(nV*AyL31ySC@)6*|*LtJ53i<!Mf$8RmF8(eUeT$
z6!+A?U@6vJuVV3<zWUew^`+KWy1(AOZY=KeuXkrE=3np5=)8YDR4Kk5s$@$mN&I%G
zl7Bl=_3cQO9as0frs)bb#mj8z(yZ!5?DvdX^D;A8HZw!-eB$OB>gs>%%spC+_gUAv
zmf|jNf4&k6dNuPEM(^KzKpmg2szdk8AN`2$^IGmwix>4;?pDIW(m4UbUp8dP`7}L{
zolmcqvY(^}PG{Fk(`6n^mv5HsflAt6D1IePN3$zld^8jvjm1Yb7hZE?mYgf4`?G7M
zhy3u6_5910E9G<_-Sj`z*pek@OX*N{wsfhK{dfK8>FiSJt{&ViT`AiGE3Tp^s4=a4
zJ2{a&`&`56>|A;wt+Tt+$)a>YokKwE(D`)9yV+c+qow0Bry1d;QvFZT(dY6f*3h6e
zRP#ceqdqm+oh2WpY>5xk`)T>a`04C^`Y?5%vQhrDBTJUkRL|Me_C`LgpPtN;W(sD_
zD;dkm_ViHJ-WON!r#JJ(<T6`Y*Bq8kF9Hbt^j2nDlUd_3`@=K!r;|FW4|BGn@~z~h
zrz*dE%I7P;t0_are2jdj**}*<E|=NTj6Ivb2+jWG`5bb&q}e60%#})dFciN6RQZPW
z&v_L;znbNxxI7d-xZaRIz3#xuhVXBB6~AS~u>~#<`ELEo+f8-#&w)_F9<_TKUtZkj
z6?_OP@S<M#hxM@sKh5&GxpZ`8W%Od$Tk<l7B2?xNb7b|?3SId8nTFT}yY~&}r7KOb
z3qNh73trT#Z&Vk&RC7b=rO@f0Hv23z758}!D=I4HU$4-D`PUyb8ODt_wgCC(9~2$9
z@0i)Yzh8X+kTH~tY5b|~tEQ3r*n^7|#^6P*7Vcv&E;iQAgxyn1C5(gnyq4zLnYh?o
z)Uq;vWy86s`>OfmKCh)MtC`P>Z8>;e?8-L?mo)&XS7A}TlsCiEfRr!pD!$$2-$sDc
ztN5*z@^85mMPwIUoUXof>XE;QUs@UeavJ=f6OW6h909AIQ*6PUc<4pg2N&my2IiqW
zA|BqrmHNCBSL$gwQl5*vh%H+gZ<#VPGm&gvyj8S(OGJ+~^ICZkTfSYie4CbM^N;U0
z<Sla<i$5reKM;xhbq9mwg^tEZVRc>i#PsyveZx^)^1?17>7rD+u6ttE-q9CBxyj1Q
zS<(XiX=zJ)v$mPkX;50C_{uw3-4iFgxZQt{&?~EH<-JPU8!xGJ#v8O!4{N`AftL76
zuU@F<l^0FyzCo|l5yP|fFhnDCW#nR-ZqKf!_m-yH!`rI&quI!XG~JP1NUy51kVGrF
zBPrDhU)TJV%lR#M{4r=m6QgW`c#LId)9Y#7L~<;72BHd;;nBl%wiF-DmcB3E?E9rN
z`J*#XNFH4&&t<XHTy~Z<;h&57ql?)!J-U|NTog;4UwmOPk1i}ew<J7DDmekod${n4
z?)6_uOl17iFwSL(E!JO%Gx&@2qJ~lVCY0%73h^9;c;3q&-P6cM<H)SkZ0SUaM<+`2
z8ks$sFP+z;^QDWW^6|A)RJx}7Yo!}`ksGC(dUUgNJAZV$^gxduz`)p7)8!Lo?oX8G
z{C=){P50Nz56iKkhg|dMVR=ULWo^%7|B&(MADFTD=vsDp5s&5;!-S1y^OcjDHWf~m
z#(B_hYMhFGtNy|SEnif=SNgmEC0pK-j$|#n!`*JCY{aK&B)gYB(A{k5REZy(0^Vul
zBbkkMB)cuvXe7I@fsVj)&6N3>E6?l4`aP0aUwOFr<dSs6`k@}&&XU<hOD2*f)6O&8
zsk>b%ZmsVsOVP*gWyvHg%dOIEnV;G6$#Ui0WKGM_<k{2N$?~itPOCT;Zv~cGCqF*<
z<Xj3v(%v%JWEa5k+44m}=*9A->?2<16Q*V#WJ#q2uiVj9nki*{WAHBL%NHGETgM)|
zk|kQ0tiHnJ-}&5gX&^cuJ5gUX6m9JE@$T-bN7~!z<0Gxcq3QBsL)F{V;G8ZrBw5nB
z_`gj$T3SDUI%{qB%g>$8I=bW2q*PtFoj$gAl)zCP-LdW`uJ4XS=84<8;CqcFf8utx
zt@z(FZ*<^^yWMcX`lHz=9%e}+Gu^l@ATyq<lcb|wKh|1<ywb<ogrS%!x;r<W)}2Yx
z&yFU^BCo0e+}U9b?aVs*DOhI(FZbKC*JFzf;I*dGi6m?CZikN8V}{hUvUDQNR(3Z|
zJQWsn;*-9hbEQBt8t%13UY5%+H1E|=YwVN}p@y*)@!sE$t&l{QUn8Hz>sW4?b$V=t
z4<xCtzJ1*gcly|g8pI7IO+f#SxNVd+AIXlkEa?M*_f0z)$)?jA5_WE+V4R&`93%{r
zd9IYNO6Iv*zLTXp!!#v3#<Qe7?aMkrseO6fb@%K=Q{7}Tr7h7p6t9M+ON$9=Zm8qw
zO!+U$@y0+P*|}nu%MH+-+Uu_@J`T(wqR_Ct{7PW_RgRYRN2@zm0-m(>ZME9EGk{jI
z^GsV&FV{g_=J}es<B0@>Q0<vDi)U)uIkV<*aMBOnb^+;l0!Qr~sI9T82NzhX*n`S*
zL8wdmTXz+c*54}fJ({(AWeJu4)z!LHvD*IDuV_X8>Wa6bub5SC7+ZfQfE#P>cqOm7
z^A*u!{&JMc{$tPl?k?>a5a~<&Gr#QoGD%wJm))#~n%@2O*(dp#eew^V<mDeedGg8f
z#5X}*Pu}7&Rk3yUu4|@VgU^FE`_mms3Sp+&gQz-^v_DJwMd<nC*;f%zKLHN=M4x?a
zkM(>PD0N<R|7?2yH~2aC)UBuZx$*47XH}t^d9$)SUVimXQ)wL>kaz#9cbLd*X;*SO
zt2FTon`Jr@N^ysvSNze^0Hs%eAXc0!ee8I$^)pK!|K(H1lV{f<)1HGKeC&8RJS*F?
zyQLH5r^={vn(CfQ{Y|{H_t%%6UG}fMPs;kU4$$`Qjx4E9_ht1>=|K<!y;ym^q-Fd;
zdv<rp)unzJyPHYLq_3I~ekt3$@-d43tQ8<w-2&W74n7FV<W<K_uV!5~xAd-8vo2Jl
z3V@<knr*%8rCPIxOKy6pR!yS*LR0FXR=>Up(9fDylvbb4R-7-D)||Eu^6QH+fN;n>
z{P&33K?$~}3sBXw3#`Kji!WEgHEW?iy0HiG-@a0b`etPKJBu$>Os34=H-j<lsZ?YB
z7r(S_0cpH{Ab#t7g^$Is<{$6Qmd#X7+xK>rR}pH;I}xfHqUZ;UnZCQKaVUGRc!nCY
zxymUN3NR+;b!W{;SL8U0-t1$0Nu8BtEsHw7!$!|qRxDa?7c1KQ$$~+L_V)SE_BnX&
z8EcJG6bR0qF(}EBZznd1prxkgSn%{hhZlQRQ?EZw2vn{i{awxTWBgoby4{riuF<VH
z)S{Rg*Lvvpa%Fl6yLo!asU^BSwdB@PZci;c{}?}3i}tksXk<U#X<+}ZK&926U3vWO
z<7t1KQecx3!ywT{0ZL<6<r~S@lI3s_*~qct`WWn_y*LfCHC@hMcpKJ8dzN&jyRuy{
zvb(bTkDqBscZIk77+eMx**E)sEC1ze*K8^MGTa@y)vsokOE*dk;6~|I$*|Kc*jVmw
zm+tB9z0!@W{0)%SjqHBL{r$&h8}x0%sfO~`czmjXM!#$$)0pP6WV%FOrb~0B(p0YG
z9b{|tuo9)8HNrtJyT2@4NBA$^H&nehk<VsxaJM!g(6#L5GQc)0xqYQ=|L1d7sb9uA
zN9zS?CmP|>$|p?e=~Y2Fr0~sVmd!775~V4l^MkyR3^cQBb{Rh>m(B4LM>YsP?$EM7
zSb9Pz-lka}>3RAUdz^m7m@=LYG0`itrKoALb4xESwX2$U8vjo|?r+1jEttLq7a-zh
zr<R^ws<HagS&cTpNvqN?ZL9FjDD%TuQ57*(Q@+}WMghVhD8Z@p8ss5KGbq96*oPId
zH!uF5G+6WCmS85|sAJ5Aw(!Ul|4hxLa35#WTl^%6eKm)stv}Af!4_fJomne@<xGpU
ze{h9VlH0SD<NPx|1HTit0A2QXJF=BMbg*ZjJoUh37LDeVxSbu@n24G+5&p3)tO78X
zO?0Tq*nQ)DyXa-t6wjt+QGX(<e1L@@`#_Szn4^r*3k?N;P?PF``&{P_v$130;v7oi
zP|!%{8<qO(>)-okj;jwasH`+;a5V{B2}M)a8;yE=+#Vn2alq_v014GlXe03@edCId
z{=3E)Wi7uAr_$9GZ@b<|N5fo&eVo$9f%8;W{w`=blcx7ndP!q_AYpBcpNHs$!66S*
z*-Y5L>|>2R)u(TK0JgDR*96KHRxJI@bausCu}U2@D?rp2_Z$G4+Nh<js+KTMEymve
z`CL+Vd^Xku$<=;KDz@J=QuwL2PGrgBFtm?%SF7;z7o*wZrgzX=)7ZmyzBLd0>vNl@
z<SmVw0SUIjIQ7f*GEJFa)xt{{ZvI&`k^H+)*=x`+U+RxxEbA7_av)m)j@-`NoD8;w
zld>T>Yn7LKktui)wtPosZwuH9^}Gld_u@<mD4Qu+JtHu4s>eBFuV*?-x`jW}J1IXO
zrg!<FL;i9yOL)AK$6=?iBbPnwujS`H4$PRc2hpGp(EuSFbWplf&f9UUPOnsSh+Yq|
zGq-2A(o-58dY@AzEE(IeWSkEV&g=6qT$>zCkSchfEjpGh?`MvN*gIp{Y&xHo+TEzZ
z?UnSl-QG4ZOEld+x&A#J=oDV;+{I{jSzm5y;HOF#%KTg?Un+~N#O3(-hvMSKvhhi+
zqU6etUpGycF?w|9^-Dsi2eJ-OR$q24y|T#RCiM4p`a6}ro=ttzR0@3mmHR^uEgH_3
zfyGM*PEM|NR&AlsXY0EPPD${j-%ag&)V`59+&EPJ+SKW6Xq*RACuCa5PNb0UKI}Dr
zElts1ga68t6uFh&U@fxSWpt~3w$lqel<GbSTH*b=k@Dp3)7j*OG@F=aV_rx<;OdT8
zy!20np8m9SvApC6EKBKX4=l^o@`qKoHe>onHG*?)YgTWkH%d<)O;$?>(#Ah_v3yO-
z+mgg?l;*OhKH??y|Gnxz&#e3_?`^U|V{_SKN0X(c{!dkZTl*+k98sT^Y^1X4^bgYH
z2Tv#2_#X<2PNhHLCocPfWit@U!6*#!)mu{c9LOf;nAL19L*Z-^@w`-Cef)FRK_R=l
zXnqL~=9k=D;w5e_xvdAcmz-Ma4^Az;^T}efXa-gJ*IC1_Uke-Jn$Y7YSn|#%`6d_(
zt9tTv_PQ5llUrfwysYp@Ar<8{4~N3Tq401lJe(JfyT<0E>Yp@qF5e&3%!+m!O>{5M
zl9O8g8L8lU>CT;!gVdeUS?#iu(zW)oz$gu1<rY0y#LsNqxyAfkT6|eQmlt1OTs~1c
zot>$i1wnIht#Tbrb#}e-&whV$$(;Dxxg~$G#NOXra!dEOmVD^<vrA7dwHNdC7eK|-
zaH0O{<CM7h_{<ah%sg@G3446%iJQ8=`NVy{zxRm;pRgCVf9?PHHM=<f>*pmHpMUbw
zlRl35rb|tJb-(Gu-w0Pvo&2O2Fy{-HoiG#Zy`9;uG8+G#*||mcp5W)+C+>fOpWDAS
z|4I0dfGWCSN|w{g!xWV5ths71E^j1z_#3lNfkHyrEwf@U`?A@lvoIF65|rHn?OF+t
z%ru>8%IR_be2nL_O+uJWC!1ghVDR*z-m@XvsBK**|8!4!g?h<Y^cwc1h3((cTAFN<
z3_1wq*FOW?8>#SG;15<(eH+@itn({uCWj5xV6QMrtYdtgW11yN`Tgf<(77EhRT+))
zf4+EarPLIC+42$oIh9^9MlO!?wN__2n?wm(UQ|o@clK&~)-(h;nM*H2X1uM}`WM5_
z#oBi}uh7tFTb{1GcB>KI4XZ<FUe#nvQz6_T|5asnMKAP$)kkUT?OhS=zO3CDPU~m!
zkA`zkQ#hKnwewH=Zq4KFNiHY9uFIK}-7=b;OwXpeJ)1(2a&{`TU=<w5+Qz{+<9yY~
zS68KxyQ<P7V0AjR^(GYw9CUr_geZCF;ns+(c<4}M9q$>^O!j3BD;yE6>#|?Nltlq`
zs$m67x2}sC+2b0sK3%~i^Gs9h&)^rt=%TF1fwJeAFIEf8FQpzWXQ1#pw98<D0tPm;
z@H)itd9~fOlXaei0%w!QbbpMpL6vLY<)iF5uXXLqDxJkJ)Kxv!hZVt_%}&pSjtGT4
z*IR-|j`xk3K1W}&3n>`ahv6^oXUQup(|OD^s_ZHfXjmq-!CQ<`Ac>U6`S$%1l<*U*
z($uu12@Q^=`O>i`(^37NON@Wj<vM|)O~seyVAsu6>t4F8er0JwpHYPC2(8;1*^gd&
zTQ_n_7}T~NS`px=w63+%x}M1lLR1l5-FC<6eV6vuIK}#GO{?G9toN|Tdq#9Qe$+Jc
zvBU?Gci7`FGTGP!oeIK3FWYM9qw3U)4pr1)cw@-wq~Sr;s4n#aOuJRvu{aBT7Y&94
z3lSdrKvRwMsd2f!#~cJoTB*yGZB`Q_?Yk@}v%S!FYPk(ePfzI!TGI7=6ANJ_dhd0|
z5?rc;f4ENS)kRi#-J0UV5g`zp&%S9?km-Hb(a`%ow!gSt?1`9xBgoj;G`Kz>N;{>s
zhktthCj*{&SSju<izVBh&1QIQ<hw}ac?%6J^VmVj5b<_qsSDj+*Ur*p1IDY<Wq!RG
zwX?mS?_6rQUP6m=vvj5$WM0joO4N&+-I+C_^w?=a0Y4{8|7*$XVS09EEpY{QcJ+j@
z@a4|#ELj8auj|e+Q+Nv8W{oUEquvD&-g#}DPlKen^IDcXtz~+8+y<jou7jR^J$BxB
z(l&;kez+o5(BIRM&m-;h_i11EW))v~sECc4H?Pi7Is5DBu;cYSJeI-H5JBNt<)ufw
zbYP3}N{0QhF+lX@)$=VhlWM=}vRGApz&yT4t1phR=wnlkJ0;0`sZhc?Fzc4>yubD(
zVwpX!>ViEV4$svG0AKaAN<7<<^`RHZ>u3`g^kzM~)xj`eo^=SkcE^c*wG(wn=s%7F
zhEc4_-}%r{(kEIRNvc{}(qrqf9v4&QrAIO=UG&!<5BuZ5GpdrSA6++(KdNpk{b=3x
zxVG-l$;~&y#G^^ro?Svu<a2)wCo>zk4MnA_x%;(~?_V?d_PiL3KWFmoc`=&J_p{`b
zNWobdzgSyelg$t8@s8{aeh1Q1`2>c?z5ZAb3h4Y!MXOMf3hH;d66)~FQ@V_Q%-buI
zTIK^;_Z5+lEBM>_xGt4IC^@jJR4-87i-$?|8u{~)@Vpq8KOYXyi*dbjE=%r7Z7@?h
zmlbgJ3ix%3X}zDGs=`KkiY&L&A4^-tST>J4=j1~O6VTNz?6O_z{{F043}ss<x0A=k
zrl9E&1zfb4uV1;8s8z%3TCk-5rZD?~46@dnwLH(&^Q|@kaf+<aTqq<?WQA?Nj_N|x
zv$7JDC+v)D_egw;j%H<77H~_lvM*co#Pq%0Zr#vrm|-<WCC`tE8-~>yqLQJp@&uyr
z*qnGN{|iNai-q_t7?JG^Z9a!HntJ|xmw=*#!8=ec9>Fw8K!?GHr(IE=L@~4i^qTeT
zqFb8M<oV0iL-0-VJZN9@qs!Z-ZO{y>Z@&pEekM)71VqB4d?}M7ClxqLmi*={Go)r3
zKS<3=t_rZo$(bQsiU$doLs|hmQZ$|>__p$KI@Qz!Rc%_KYoqE3s<q@dJ6TxOTMvTH
z+M)gGjz0X{Ln%@ZWX;Y$=6qxFxodEJ`lLZTI>6wc*hgRL7MD0mGugYz(@exlhh`b_
zk}q|y;4^ER`WF;60<9u}5+`38k!xJ&%%>lSFFBC4;|n24+@9CGgZK7Hw+6Fa@I}?z
zF!35jvi45ky>l0(5T;csDv7QCUTQdNXPvVyl|qVEsfY@a&-9@JtUr-dTBVRU|2dQy
z*~Ug)t}u6i;(6Z3(9h4Da&8%|&R90DMIJ+a!*WWahK>veC8>%dN=UxNrs!hKUHv{}
zu`PPI>^*cXYVddKo;6>FM;;m+e7oi8z%VA?ZYjLWxUPHR%Jhx5wGdSJ?bc5iqw2%I
zu5ErGtIz2NB|d$p;>HQTW_`*i%aRS<4Ruf4wu;P6Y{RaE^^&7WvSA2Egj-dwU{>R+
zabA5`^=i^yEz(!hN!=44_$$L80Lgctk*2bpWW!D#^NDcg!`_Vu4o4;{Cz96B7gqi6
zbcoxqS@_P3h9}l8`OZvzxq_iH-Y^iGf$yBGlsdv}eCOn1iiZZi&5qoy26m4vXyDt|
zx6*)B#AU?*q3*d#n9=J!<t5W`dOzZ|zU9yOwq#~LOJI|-Cy~wb5$d%qh`$}x+c18}
zw@^aW*X`;ezz;{?XB)lm7_)6do4KFWeXGjdVdkRL*|tVAce3&c8P&C5Gx!YRkZYIs
zgsc4N{UkW#gx-7yTxWd+H}--XAEbssA4E-d=$G}}%hK})qOuRt>7eYIMqSO$v@r<H
zsN6}@aGsPZU?{T!L&l`@@|5|NN0&jTMm66cm=T4cPY1JWl9zE^$FojzVl@ecXz{Q1
z#TEO`)v35*_3E*DG250UgIM9k=p77Tado_^m=N}NZY06UGnwtmQ1Z_t&wB0L+GN_y
z&W`0XI6H38t&eAqnXk27PV1>}(*7J$zm38uK<c;WfmA@!0c@Tf86fN2;}l`)B66Pr
z897xD_Fl2Vj4mr)DxXN=!j1_5@QaA|bSOtK$4^LDJdmvmtQK%jM~|PlYIE)Y)KoEO
zD8;`rlXd(<c(76%ZW;}8C}?PscR^FLESb$Mn)nu%y2IQa2dTg(pu0+$U5q*cv*R~2
z!IoX#O61tZxcq;y1#MQEu%Ct|`)O*npXL_*e8ea6sr{$7^%Yllz;Sert1G%{YUb*U
zu9{m^P8g7J0Q~h829RIVE#C-@c@JmMUIlzI)8<QKtH(`Tnwbtd%98o4b!eTeNPGRf
zYX@kkz5b4kpEr!L9m+cdybqmER||psrjd^h!|b24>#mk(^x}EDY4@9zcJmC&sjjz=
zpdlFab=z=CkdR-?g;+!6&k@c4010KY9Du+0a7pRAyRr|L+*?xmgJ`9vN1?<hCC)EN
z_h#oQ0LIM)hU6dM!I~hSO>>J&-*q)c@;}aBMSSz^N(H5N!}EV=eppwx^Zl&<{R}t`
z8~56HmdusH<7}>UuN<#WqA~sz#X&VH$&U)rmT^p)!fVIqPw8D;_MZOqL>lIAB8ihX
zQHnD+QTA!ep2pTCwt%_2HZxtS+x~vGgAVGC`u?#+?BVuo7p!v4v)IdBQ?ZZsy1Gm~
zSgyXIJb4V5d+sw6i(+rkR6>|o%89zXH;cVDa28Uj3{CwTi&r=I9J&I!$rjRB{;^F`
zY_$A@kyQ2Qzpx$G`Pg(c#C8l9{p0lV(%JxfTx+XX^V7>K-;#T)oPYhv?#A*2Thx0u
zB&y?=#?;xi`^U+81ER=Cx6}uIbEG^m2~odP6KKVlT38*lI$3uQxUOf;O^^(}W#vhL
zBTi1s401wP*K88E@YmnZz8scVWmFM(uTnSmes=KvEUT<s^vBCT3#)o;CXLT#fF*xw
zwT91raa%4f2eM8C1}UJ=1_2eF?xFxMRV8<V@ODn881SaOB&JX+`Q?Rm?!>&alYRU;
z2-50=F}0HJni}f-J1EqX4E)JXHbN)rn5_GuHiOOO-%n%t6&|qh{rv?n%i`lnU8NbP
zrfJA@yir}^`mZkMv(iLMW<e|_ja*$`9~~^Ow%|RWqWYusa}9y1Z!f=#$+7)mwMg`w
zG`3JP^M6<26<Gom#s?h}rS_(=xB(kJ*L77gjnSFsRy3F13oKI$GJT+k&)rf#yabnn
zK0rMU!1f$rJ);g~x*Vd90Ksz~Hd+De@Ux)S_N6}*CbGY*oge~VZ@Z_P&w@Hwok6%5
zmUGbY*R!41)4Wg5-fbuyZ+05|%$=d~@!bKOxv|cA-PHAC82eoShJ2Y&_+cbBJ{7M1
zOkFKtsALy&)AdlUSP!dg<dk$5mb})V0PZVU>y?vW+({;RjJ`6nShLc(MTKwaPcA!P
zbPd=h)A#i0EX#l<8bzTr+4!yh@AANie<}bfHg|{5;vAKI_PQ!QYc{=NEzh1eefe(q
zE37SjbQm{)-NL_YS|JIOFwfq|lEtcIF-RzDfxFFmOnEjz!;_HwfN;;=!ef+=Sgo^n
z*KjF)N%9knRT2urbv|Eo@%vP4ssn(g4`aw}-*7evUe7~~q@iqa-MNoV&)F#rjHcmf
zUOp`ch_NZd541KvCML`F%rocf>)(SKt@-p;^ZxfPJaaxNz2{Zs75?$1fSDklOcI88
zU*Gd7N?3G4<#Sc2iVmp!^Njs@dtQ*wKfWi&Yq<RTJ}0(Nf6{<<FXF4d%L_@;pS1cQ
ziTN>eB4hZImU?sG^GkQ=VnMX8{{$2zGax3iK8M^txlwPjs|w$!&pJAYs?{HbH=YZ_
z>U6QH>Tv_TZ_s0P9(M?p24$dL&zkUp&n58iyq@Cq_PiKuSC3ZVK-SP)Q(TX&xE@nF
zkeXjnT4th;s6>xHqLN=xy0<xiQF~2K^w=uVW9tcX#6w$t*PZ;RJ7^uQ`;NAq!I=Fy
z8V7sMx6-v}Pw3qY?1Y$LXoV29IWl%jU}Ao0;YYJ5=v_gc{OArrm)VVgSGX%3)gq|W
zJ<Jf<KgBoqDy8o(ah&(#4mtDY*EyE!&~D^Q-iK%Z`Jb*(S;WipH^keZ`a<u}$A-QX
zm(hpnud!m<b03l^xfs$9m~J0ZD?@5+w;{Fro}vYf`UmXAHPEp&V~-t4CX#1YEjJbi
z?b_tWCgfEzv-MejLV0@mgD#CPj-0#F+p0-#`ibGoN6wu_u0L9t2Om`!LnT*F<d2c^
zU13ev;~!saVl`Is#mb8s^~Ln(Vt{{+v{-<GRiSdNRM9HsdzZbA75TDm`alERm#tY>
z`K|`|T;+7Ou5THTv*xMq`fcBCzx@Ndg`x-|ymmfzPh;F0(8NJ?$8a9gwK22m=!Xn#
z(TOL2yM@<Hi%$GH7hxQF8$sA$YN)#NUMQ3Sxn-a1Vx;UR0d>~X&rSaTj|cQs7?Vn`
zV`tT&#v&9VYugyE9A|9mT2WX2mwTOU0nDoI^VT{+vKEkdM-zJo*RtwP(SQo9q}`Rr
zn9*bQ+FSSe>^zwzzeODUZ_TDPlQ@gLyk;7GW7#z7n@6Xy$)?d@r}N^FLeMO_!{**T
zlkIGZWuMX}goe&}SK;8$eH-5x*GVODJIp#LiMN}VWEQO?RBU0LBl1RN+%Rl&>GxCM
z|NBm)(9>smrBjNWIFR)Z(fTBG)LB*Q+A|{o1wUW<Ze#YmFmA5fWDfOOt_3Z;uJ_l)
z3hL$aG-+SuhJmDyef~+B`(zc~wYL5P$V^^?*UJBMnV-Am`}_!LXlMN?Bx_!Sz9C;E
zh8iA2PuS;#99}ZTWO#a4PJ!DA#ygjzvOm2K-C>5<!RN*;_NJoUe6DYK_T6jAY4VfA
zyW2H)xS-|l>dqwT<$6{!mhJc*Hi7Scs`kBf?kvo$@ay=b{+&F=Sns9(xAyf30weY8
zLFt~rL|?{5c=W42zM9ie8B-`{GjZ-J8$-1;bTS=1qStrvkkpe|Jjt1iTlk@~h{mG+
zhg-{qjpKF1#*!!(Jp1KCwPSzGck^YjjHB7EGvYzTQaDQ)``dg^Gz)w9&}ZKJ4MUtA
zE$>OfGEwrq7=rf9dm#o5U%@eok5By@i|74#;7W**abu*7(m%t_ZKks374%8Go`Wh}
zQyV?*>*Jr@Bc|U_Eez%qoi*Q;X&K5DU%CK}RbU;Tt$Qd2`5~b=6abPn^~OSe4_JO2
zocMSP^V{8TkYrVVZ(0Dj19dq(Ai=}Z7oEB{h4M)smqO!Kw`BIJXV#%917*JX9vW!L
z9cqw|zIOuJU;b_bW{=OZ7e2c}*RIz7-U;ma#ih1_zQ3sNO$Pp><qf4TfVTze-@}9#
ztU2GiV>%+Irl*bxk>EKxSv?bK-6xFKn#;Cb40qQsSHm)BfoVo?pn_%=)1wgrkz!?=
z#pjl7299+}BOgZ_YlC(%LnWJ)*n!~mV}m$#)w<|?_#;Y~@^7lPi!`WLhPp@S%*eR+
zY+Ur77B%cYx`HIC-@CCwo7F#>&1Y2+XfL(<tooGal|sU_5b9;#hluzzxIemLm4+~J
zO8u-M$lV_Ze}5234cqb*Q7>^$f7XX;NrumJ>CGjDxkit%^KSRY@Li6>{oi;eApY49
zn-)ncsb6$7>39an_(Y1Gx-r}}ws5h5$CxmOv*(Ax-Kq3!DO{cf`*69DG8aofFH!2}
zr7PhHon&M`Lwgkjpr2P<7BX=7d53A{`m?!3rxr;J6sqHoUj6)*I3D8E%m1v9;fLk-
zse&-}t6q~8^Zog=c`aIBwwp=7Wi=%T{`Krq-SlF99xgtyBz@hYK2$mNUhaXLFITpp
z7F}MhymC5Qj?wt_>^g406x61-KkJE#1A<-hq{2H;D&MX?seJo(N#(oCu<@<Z{gM^9
zOop70lS6aut>vE&jZ1GlKBY?HC%!EiigD3CZZ&ABJ+kGk47e5ct>4+V5xzsD#dHRx
zGHy!Y-_m%KvgW2c#W2@@)4}Ke68L9sG<>bW-&E^qxe=X+FEnz2_s=e6!W(vZIalES
zygQ?SJ^gu4uE78KS=#wgXonkwG^_6CXRGwV+~n<W9R<0+`Jkb^<Ze*&bHmab(fFGW
zDyB=0H~KbS*Jue0M1G&$YK;9YZI4^&KZ_LpGcNzOuVKyZ>$x)Y)NXXa4oj=g&E9=e
zaC1|ikH*hINO!QM@LUNB3=ar0^rd}zT@n<W>?i9^BuoA;7|Gq{Z+%kaOxZBJ)MfJ+
z-kzP&V?3cWufPAH+z@Ose;*_w-B0|ZPt(wvos42<zmK8+4um?p!#ex@T_sOqg=+rt
z@6fyHJ38NwtkcWR#{{(#6O?_<<GZDMqT}~eixgAV&1t$R{TkjlLiGOZWR_eLPPwhn
z3CK_J__}b4J<c`7)6mit)Z2l|ff^X)^aEs>5*$vv)9}L!(G70Y>vQ7r5IcJE@2=vY
zT6}l4!SI<X<dXK5<Py<0SOUZ17}XRiayHD6kdd`OBmB{dN}6Tf{KKZbB$Ht!qoZ>2
zjfYvnQ2T)wJe6G|#mC>QWC`H4onR5>QX{4_-@L}+=hZLxFie#`WbZ1<M2nD=tm~o1
zgeBJ79sV|--|$i1nz*U(30X9H-_QEc+p5BF9lCQ&+Cs(l(_P;{@kSdcUR&?2zGF?n
zzJ4*pb!aTKqvvQ_WUsO}1<S2B3b1ap=LOHL=l>p%&;GrC5!`W9pZ`(o@AZo>AInw>
zpLO9vosE5ye@>>Sq}jVDKLEMs{5Mk51+@8lEzShq6i+TzjG_HrYo-2pQu*JWs&stL
z1s$~jJypjqcu;oFFf|TOQs3n=WB|6PY+RP=ODb<a7esSayX5V^cXLU=V8`%vHJVCO
z$NJ>x&7U+&$LKyQxPxGbYwlO161#d~8?9f*c70$-Q++&k_ZoijzR|xiZ5|fd^WT_>
z+yeDApAqgrZL~a~71{}pGiEqjp3P2^9A~lG&tEKiH+iXL7V&x-kAE^=gj+>gEn&mk
zJJ30PNYtOmFE(DH^INp2t}fi-;hCKFLRN?B^V{kckRR-!k~ifWPW{fROV#$Hsj46O
z=fN?j`J!?t+1AlzzIB?#HfbuFYNvw7HWf_^r$R6OZC&v3&lXd`W%X3dCizs%CN)zr
zn^d>p;}^@O&i*TnH&%~d++6c48eFZReSC_K>rN!^DnuyD7Qa`i)1|Z6&GQjFRGg;%
zY$~W2%rjy-(6i|f`Kdzug>t5D9RobatQ>gF$_dAMnRD9f_*IG`@Q9xM*;TP%2E6AR
zv$``$`9C9~#rtyQXWA0}M<eHZ<luE6TRz6N8goaK>==7DNB=7E9LxCJsYogLSYI*>
zDWUtuBYHo7%9{J-Ct`D_vU25z)*Qr@PkPS=jit!RidMh^t$>r=T%Y&1jA*4|C;uE{
z*-F$3+q3I1qx@VHx_~%AE=xxes2#wT8=Z`L)5PCiQ4<HV#`9_UyC*7{Jm1TIz{N$R
zVtX=2<{^7BUz#V<^F$@~Ks~h2jo)jWO!x2+s+2t_1m?4JPb?QTk5(2GoHfpg$GiwD
z!Oy&U`8vJ4U*gOA`qCIroAKC!eRW1?=c1w%kporEeX;VTDe4cZ&7TT+U&JW|roldl
ztOC_u$u#cXZ2X3EOSh^_hgXhe2}YD`H|)k<<4XD97t2LAcVx?7vK<TH{G$GqXm|ZZ
zpSec}oPIHe0PEoMi)jJ58AFZ3`Nf%*`0C2tCLqRuOqW^~zP)m<aJi((FCepY18d#A
zjH9<kKFi;Z>Md@edK(CoL7Tqq;zgGUl6o7+l%|WX+g1dXHj5yG&TL>9W=rR*S*pAw
zO78ob|NCB&jb$ZmV?|i<I&rNtCnc*9jhL0|4H#Tt8|&1f;$J6A*}MRC9$ut;q(iI-
zePBO{ShUTLbD0?|Rev}M`_h|Dmq-D|(AJ<I3UK+@V`|`2HxRt>=7O$XF(05S;@tKj
zD?Uie-$M2JAjO+sR`aC3-e=_L4iW<3Z`m`OrrWYv!5MK>vV?6@v1QBjFMrgICprr=
zl6Au#?1w!7GPl9jGC})|!RGE^de!rKRxxQ$gBd#D4mzfA_n5knlIg+Nq2E<+Nb0HD
z<7Ro9^;^fjTsMgZW|F>>)gpzLGY&bh-rkqp(pVqLS{3OrVx{3p`#=J>q*A(-qMM6w
z?;n6v8W`FIRR8>3mo1UdY;!;iTUJACpj5;0wz}iZb)P+6;lHHrvnaz_8jiQt9be9U
z#(&53?2<a~ys{V;!(30(NDsLgWF6;6ZAnH{W0DxvnsSenDlGBS4T)(n!u2DK;!wOe
zy!-fq4y|ety{3Yv7QSd~Y<cqO=bAr_hm1Ro#W5|cV(X_FRpS$k>RU-ZC_QX!`S^mN
z@-+1LY$DZ<_x@}GfW$7IO%@$azTmw&n>@yKbw?IE8<Q_X3x!{$DdN@-C#$*Wz~A88
zb~T5?ad<e-9QxCwyhOvjmXuyY5)9X*1FEZ3!VTqK^VY8=WDr>_tn&1=EPO5i(;suO
z;%cB{?QElVcd}OK4n$GgZpDX@r}y&d_ddvyF>uk?4eh%dh~7qxkiO4<oR1`yQl?8B
zcD#n(3&Ew|UxyWzYIND%q)UW?I&Oxtgv3&P*$2vG7GX5JP%?q`e^ErTy>*`k^+g~W
z%NPc^XF6FE;!uo~sBs5j%U8>Ksc+RKNe;zl3#*P*ULz7-i0FKocup5?((m6t!S+Z>
zOiDVx9J6<%^W16U4P~u=oqp!{V?j52yguq?kAKXraMwEixLpB>HXT1Wl~2bH&e^4p
z=m(dW4(;5VMzO5Nq1SrFK9WP$pQYw|ano}L-V>I>PgPQuW<>=r8@^Kk&A>e>(+pJq
zQSgT=MgGRJ#^#T|TL~R~cS-1HwtI+IL*pOy>e#ff#WW_7k9cbgT>8Ty(2I}thbS|`
z(*F>{fPhO<M0&cOfCDZ4gYwC^^x^UM%Ph|WB_0fG?z5vW=50;q%X|sK;zt8iQ4LVL
zJg)+`6@PJD<7Vkjcl~oI!sCxxtW#wk|ENuFkOr_nYWv%C$wXzYwDgy<8zD)lx*xTf
zJw$%mf17@M0(^$B)!q`^y5f@lxE@x*HctBE=0@RYfeV*Q_##wjYyV^NfrR<^@fGZ+
zB<0W_4gL5E3sN^r_Ga{kjZX)EejYzvnsEP>P^bx3`CBT~B)4uG+>h_b<|1`d`r`);
zt`YWYYs_0nw%!-s#}H506+7LTNxf71_SUS13g5L5QR5~+1wG3j4@#U^v?;J9#*m$X
zVRTer%%5}GUZs_TL1E0<lsRqW97<lmc$C1eSw(ORR#C(UdC6WVXD_>e6cXMf9{E<%
zcBDZ`vUN7t51MJR6V@jm-YI|4_S;osv6mYBs}N?h4o5r)gL;6q{)un4pFr7ti1*Xu
zfKhfFwxfFf7F2|>Qx(A@$$&_TP5+*I`+hWj&uj$s$g3wcrA(Dg0nZr@f9e`<`e?Fm
z(Dwacm~39MG~LQMW+koc8t@$3^8nj3@A<h$I!dJ~I7M?)Gr~7Bl!lr4NeZ7ox$iW5
zn0*v_=hcUgx<9@Q5&3=g+xrNN|26-<|3ub3^8CW5gYRcw<Yj#aIx2oC4phERsP+4$
zY-Bp^Sy=u(@Ch3W+g4&1xJkZO3VZFD_sVwl3>E##N;Tp~YJK;eQ0u#xXWe4<?$=_i
z?|%LNz1E+=JpJavzW(Z7*lKFNG+W(FvE{{?GN0$;a6Wc>6A`KW2Ni96R&dR&9_K>+
ztIwi63+a6cvXAcAp-R2%XVv=cdA^lsOmB{|gh#JPn7QI1b2+~s3hZY8>_$E$ZG{&w
z?UZQ!loV#+&jrs<wa6R@4VWf>&+mzgX0MOI;-2yQGh*wcxZv|owIcn>Z@abz5N98$
zYqP%o(ur?o{lMigaEXUM&rT{Bewd#*-G*v9;Jb|>*Ytg=U*gH3Y}c6QlM_$!sbeeC
zLH%BpqS)G#M(R^1P$q~HWGi65iCo%myC+oA5-W);fK_eD)5t$KD9>A=>wLd`4mxA|
zZC+C;*2{PPS$V#h0)y^Sx61Xf7(G^it!)Kdi<}oQwM}<<7GLx3%c`h)O;p2In1!!c
z4IneCVHj4!JEKmiAp`+Tl(?B-h&q68i(Qiuff&a=yYAanz|-eutJcyMgetypU8_kh
z9TuUNMJOUFZ;F|_i5fin`bn1gUg?T!{d_6&Pf6wjMie(6KrbT+4yo6?m`{i|GLj9k
zv;!=yl?0hv$$`A2F7kSZu~suHjD76u%*5C28}6<VB_eR4JNBBXVBOcYH>3l_<yVW#
zDp&Bea{?R!x6Ww3sq0+&KqwGk*R;~KN}N$2^+pxeJpQUbrgHT(mIsZ?V}v?wp6p9#
z3ovMw^0nv191k+;i)O}`m&E@=+WSD)eN}h9zkBb|)wQj}H?a~cL?oA3)G85)Vv9y3
zN-c;WYEdTw%dwI;iNFHOt}+o=P6W|_CRSdRAV4Mp{8m-atg2JJRne+bw}>}-Ycx}_
z&>GaD2WoL!wP=cHUGTJO<AR5JhWGjIbAG>j<s`JT-pz`2e&_7-Z=Zek*=L`9_IWJM
ze|JttcaPz%yCm{w&KawIh<)byL_u$RKVRp(7_{W94b8g^>0<uvN3kx^PWdn#T01UY
zFIchYF!$6q@_D0$fm9c{5!Sm(dQy^l$fw2^(xbas60mbkyQVJwB@YO{yWizcM@_ol
zHhlCpW0yryTB$6WD6+e)X+tQm6-c_xira`r=LmYCt#MB%&Qk<UzT4m)SDdG(B(76&
z;VA6xvzoGz`l9qB(#3RwJXnt)O5*+0))hxfHOmDMM6QQtUDQSV47ACg@$_Wap<Ci*
zjnW5P_XRfbhR~4X8MFp7so6BVDPbWh$=|kmh7EwF%2YZ0{Jbr~7$faXpD6$tlnM-|
z?7zn4s4FOo9;wLZ&J?oyYW9S;8=^-l+JlW5+KoV^4{wHH>K#dGoY;eIMvqL?W<TN0
zovaa3u|6^(JF?;2KG~5WlDf)T{}qqLBZIQ;cuMD7PldKC8?i;^NE*yB&atmf_&edc
z%ur#~GsdmSX38>`%ACSHe2x{5iRafTj}u=)kLz02QIM1Y0F?4X0UTrY9s}TD48YX2
zg=QZPx{~Xc`}%0;w$l+_rbgwm6NkIqo@)0E?Fy}?Kl`9ATTpM$yg2A-Q!OUt^{@fE
zV0;9L@<=xg;dK{`XOJcX0}ep7aQRv@pA!(A9w2;2X#4Q1J+-1rw6+$`qC5Waet3d?
z58Ep|YI*8cJ9n^l_$-BzECqRg&iwf8ah9f##UmAgzj8h_U)N`8A6_u9+?)w^HJ&C?
zl@~w-mqGV)Whire5S4mvD8rL2Cz1%mU+RXj>O>$;-7pod5y05}DP`)0#k=K4@3;R-
z%>hNJ8`h*l#E9?+&YE{@M&VLN>c*PNvi_(6s<b`F4Qs8!LLBxdmf`Ex9aduC9kvI$
z;R;rTo9TyV{II5OZ1c?mPnCsMlPbaxeGw_dZXP419z;x?dxb1(BPlR{qAB5KC>0El
z;nyg*I-tWD<zwK?dJTJMdgn7Xy66%Pf65mmnV&U1f^OhijP)n_&AJ-5eK0<`{o;UT
zeUwFy19AK&#2V$4iWuw|g9;y(l>1WJ<`ht0gjWob3*l)-JMt~@hAT^nu_o#ZbN!18
zto%aN&sVYd%ZR_qoFx7{SBsi^En!Ha*+eP;*)T`1AFSFjS|kSHQmwtOE8L*1F;b!)
zILCc`UT<v>O%t*JIN|PHOr|y<P3s(|MCH;iHC?hh{b7Po<~OQj412=umrDV;zjGfO
zPV$C&J<M?|f*C`OI*(a{tU{lJu`r%Q<cpJr88e`&Z?tg+@44GI`ptum7@dOgzL4s_
zvKZ*YgrSDeFDpN!R)e(aJ7FkyDz#K(R^BQaTy-Z)n5C*}Rq~bfq&@{D<raUGyU2hV
zRoc|(uv%S5AT_Q`;yHei^*U*&&m1z^`IMY^G%F)C*@48sQ4MBo>Cp<G(eGzP1o|Zz
zvGA0akTzvRuIfM@eF#xr8J*7<@4dHV!$?V|PL>UEsH2CEp79_O#gI(&z2Kpi#ZQhk
zt=6wFt@5I*FOH)jMPhn)OuAobg4nYkjIu2--o~L+2&ynO_bY8Ar$=$clqZj|?yt1v
zZKEpmcDVEh4OFn7X_4fFdWE|aRLz3MPw`MIhBNF&^G8{{?5&EUNg|TRgHq<}p$FSC
zq{Ol87wgYIb6$t3*rbT;#`PM81l^4^_T{)><<c0jnW>CE&&t{g?~r8^Wp<5uqHL;+
z6~qsKwJ0fc9iq`(uDUln9G%p<Gl~=uhtJ`?SgG@zxIxjfPBAYKSJD&Gb&XL0R1s$P
z=inas^gT@FcK!{u(Q}KZIn6qzY;8{*RlcH!pKTW%Vh%VY3Yt-!hpbM#NudW_m+zr<
zRs?-+X2!9z;mn>B##1SLak@!o>!omJ!qI0ti(Pv@I;E~nA!TMmco~JSn3s*_=jNbx
zuUa7@W>=pBIV^MHR9rmS!#|h`Vr@N`d662qRTs8T>f?%yZUGB0Hki7?t6!WzCZbJS
z=OgC>ruv6dTT!J1x`plUAytq$6`Xp8(!Nv(7Ez_r1gq$(wiikJRj(i$;T4Swnp6w1
z<l-UYX(hOB&qS+q6n!q<5<=Bq;zoANDS(@0WeLIshMhWnD>di#srYs6qq07wiZjJ{
zRBA1psaB@dJxcWwlKAy(Z*8hWf5+qN+lHa)OF{&yJ@kNh5520o-#ZtC2xF7laFsSj
zsV3rdLvOYnD`k^PEzp=k7ai_WUK*28Q+{u?-fRZgulg@Yo5`Xb`R}d50KjIh9xU4g
zienv(bUP_;XE@B>nrhow(s9jOuQ21NvM+k~QDIbnuk|N(ht!|Y9a3MTJ3U*v!*eLb
z?)1EBcZmH7-BIdmbZ2NwcgAbOL5wYeC7vxpcgH(K{csfq_aIl+`KP9#$Q6JB<8uoE
zC(SmxuK7W5{Ge@tld&!V<R(D_<Jll+)}ZD|<Rn#Rzoa-Gkf?_Q>jlMvBPd&%wn80q
zHI1~%j*wVi1f{9PxP7@%YH1^uFGr==<%^PxZWK%q{pCor>s5$D39o_8FIN?@>3hgj
zHi`eAMTG(Wv#2nre-;%6_1dVIJDZ?lj@KJxcI5ldSBwe;6I9G~y#^|j@EWL?8@LJ;
zFHHtXC7HZ1KQ@k3<5x?D`X#YqBd<Efx~VfBVz?2h)l7f8Xk}+bc2UVyH~WQ}bu5NZ
zeir+qN@JA{d65$Az6+~>r!0df){LBF{)VT^5~g|ArNG5t8{954N-kzpY5}HF*b95u
zt5or+vN9E_v}y>adN0+}V)lJ`fAC~&{JE)c6=u9dI3=6KXOF$)dJx}l9#3U|HS=q`
z_j*>W<OOxU*X0n3<5m0TmQ)kIsrP4I9M(!sT|ajPt9&&y=HNevP{UBT-AzJii*>}c
zc&}}~K`bLxOu!nBVoI_RE7$bZUP4!DJm1tBIEpwxm96=l(imz;xC24LwuNm0V~lN<
z!&nydBfxiS^JEQ8{hx!9_H7g9gDOIxLoV9G=rtWQ^reu3Q@gt=8C3#RHBlzlrR^O*
z2Xqw$SEk>p19A{i)00O;{QY<BxDvf?JMC}gjNEFdnyJ){m^d8Yh}ljlSI4Pj+b`ZE
zd(A%%TS<LvTkr;(sj4fv^nE4ac)xj*WApu|aSmsPy}qbCMXkmOdFeE$h5p>s8l#oI
z+=L9J$+2<G!nGO-0Tep%xlS7mHHCgmDbr_eal2N^u+k|NGA-Q-kY&ThHB;B>@6@$3
z*D{7P*ADO0=kU&RJNZ1fbJadqcW&}Y(JEi{Pk>9XSXqrW>+ChtyArc*5(^hSm9$_W
zC25l9+K?WN$W(%UK;6b94?1CUsy5`08HQ3H*U{6*QL)=HWcSjJb1!YmzF(!BS(;*?
zm3D2c&%%ArQpmHzzKr3mW%?#0?Vz;{mWMO8<L*GKg<Bsa{9vcTG2kBOHyeA6_~Tez
zgp%OcinBC#q(?TJ8jLqIB3?sakDLobX%SMIu(Q|-F&~KS4UJ>keA%IH2wNq($RLEh
zp;@xtA=h|-DSDte)6ed@8NHmpu`d)$+}A0!Ae+V8np-rMZQ90Dl$zCDqdNc=oG1o2
z3VLrtu{WJRa{Z~O;f<U+-E+&S=zwmPW8mo1HX5%-ed5#luS(mjy#92Yb~;J(lzX}2
zU9U*XHQn-xvTh^t6_wxiLUhaa%!ON~@4RRF&YP!MfUM9x9Y~Z=18v^Z6JK{o?SW0e
z$i)b5!YdCHYr%~OSRdRJfLLwnrp;XQ|Bq&=O`bMvbZVdu&4|S^CLEp)wvBDGTO=An
zhU9fKep|P4-M6j}YlmVNJ*S@CeOl`*YMH=_Q*tIE?UY+8TPpK^hq5*?v!nW}&8}B=
z8Kg*U=BGV$F+7^J@Wa+jv*EF(zH!8ptmnwjLQT5<s!ArZlUZxExQMm;jRoeiJ2S#g
zsLG~9yM1_8_{KteOt5b(JoxH9<fMD1r*E8g5xW`KLD^82Of~{Pg@-o=EbjA9)d?Gi
zT&^{9YUdZ+1$Tu_-%i_NimHFALF)y3L$|5B7g_nOeB<<>44RTKbe@D8i{VUeh{vQu
z29bh)c;%-YJ8eq&shX|o5-QtU@2%an{gXA_lx>iw(*H^47Yk|5C;ZAO&^C!Yy|#%_
z$@~<%8jvh`p}rgb7@J_H_Oj|XxHF#xrw#YRX6|pXkF5aQJ-ZPl#r=%V{VfVd_JH2r
zORwpxkB-!?9w-vm&6TW<6^IDX#{WGp)t;NJ{*TpLVtjLrKsRK2|D@Z+$pkpAzRZdq
z_vL2D6b;KEclPE<MTa#;(Om3rWm|sp>|y>|Lg@d^>ny67Q#o#C?V%5Qmf7X<apiTF
zZdM>Jx_VQx1J^ZfTDuJ#$?hvvz2qN~>9^ScF6GIw9-yX`@J^0jgx=)*b%9-6cytY`
zlJey?o(ylwtJH@a5AAtFPaj(q;44w$7tZCvHKx>?rl-mY8{-XOV=-@<$C=)vw3Ifz
zY581y+@<o*L(Ni5%$Lo|LrR+eh`t?+OLtR;GZb&ItjGrG8hc$+5I=2>E1pjI<KH~4
zs_+l?90w~^D=ziE7&mXOaa>D1#3>r4kO|i7He`%>*v-mV`mlJ0s#feSR7)0%qZyV0
zV&6QDmTJfId7ieY%HFoe$(_ND=XWNbp0B@ou?3!(5vP3fczr3j=b3L_6wfr_=gq4e
zWhcQcpq9>qz3CR6wFza@mEo}_`<h2BMv|>0Z_)=-(tj-Dkt|8k2PynG4NBe4Ksu0z
zfb<Mbvem@fEB$0uu*PeFLohJ@+nXG1^svnl17VBA?c9E2%4)s1z1<cGwd!_!IK^ut
z;ml~*yQk_3E*^S<;n6EH)M4?bq42@XY2N)Y`lws9N-dsXcZgk1JWr<TmUHg(0?U^K
zQjLv`?qNoED1PWkv9;1G@<3bauVq!>`7jwQ2Whx;!{rkg@n>UFK^karJCb^HmsTPU
z&Z>V}9stZdWi&BQ8BNSnMl;@!WZOCy-=<T%S*wxtHT8&)kGhJ=e|Zt+ekS$ip2(eE
zQ7^@9`GQ0Z>338nV|7PEE*Ord6bu$%cQoa7E82wkH&uXg4TT2$O?heWQ+U`+y{R+G
zK2w>6kfTiaCcDql1&LIK9Ni)j6m+l-mj2FolDvrp#-#_Kri#n}WdyvwzTmmV%xz<}
zIx^58zP`d9D*a-@gYM|osqsU&Oeym`#bfG@UeEk@Rx>5*Q7@V|R?mg*81(D_BfCTJ
z8~NBWg1@tpI{>l`r=Act8oINFd9M75uOIT49u#iE(}AwCDcmxhf<xG*#*2mZF5U=N
za~+|&Ci7wVNDYSQLC&k68da-|WbZ^+&`WCmtG<ur)HkVq3P$5=s9(L^CQ2ZMdRQI4
zt?=og0|GX&Vfxpvvu_TWEocK$zaC{y<CHW0P@$S~cZp_wt#_>KlCKnN&mAi*YJo`Q
zjy8>Q;AVG>X`r6AfqJ@RpgOdb=53O_3wO{ZKTu8e_J~$8P<ORzu{i9Sma~2^{6O8+
z9tNtwAU1G#Sh6tO)uj!*t>A}T%{kA})#{Z+4Vn#92-_gswKi8~h5&-uKp9#(9{Ugn
zv}@B0?PL;W1{ja(1_m&dvBwJc!T2$P{;^Kh&YB6R6}tb@q-+8hRBcvuOw71FfwGpN
zeB!ET9lQ_?>TF{t_yHFosyxI|Jr_fGuoNqL_jFW%$UQOc8*vd<32s71`SHET)V>8B
zojvKgsryMk{`*@R7%C&weUk<0t!)JNO&i<QgxuH9>}0Ci?A$kCXKNNeNW4bNK87^$
ztY`a1T}jY`0Hjv4K3N_{I(y1bY{T}xv+RP|h5P)AMJ~E;5f3zQ2H(C_ZU=US`c_I<
z#=g~vc;ox~)?&A%seS9o%CW!JcHAEEwd53{G{5%Omn77aP#Y&Sx_HR<HxeH+{{3t|
zV$Aip%xU4p3K}TJ!*PuBlcHYlA9`i4eV(xdQTxZJ!&AOxIs3{iM*jOJr1?t2161%7
zl*Qmjh4QDP9X~KPG8&8hlg^FGiHVKO{wW(VYvgp!H8<77P3CJ<yxTYCTm`Pkhs^ju
zWt^*Uqst_Q<!e8b<EW<fjof8$&g=wK<Xt%FGBu$e$1BKYlxM5T{))g4tJ0@6(LFMl
zUe?tT67bD*#nH;!+HEb)gX0psRJ}5{d~BrlU&5R#Ft`0zkZGCjif=$9<mAmjE(a9b
z#}y&Pm{LPUMO+0=;y`Ny%SJ8ip|&CYd?Zz)I9Zrh4M=wd(h7A-)AK@}oi#K>nbCY$
z2Kk^ZzVJSyuQ;6fKVsxJ3=|2EaOQMf7>5QH0`V#}ekks|5Iv8Y>Bj*`AEzthkEPNS
zKE#o2HN2}{>v3SRcfm4PGmXhU14_#^B>d>{>xUUB@h~0;gd<A6#;Lt`T9|tMxoWpV
zf{cwYU*f<2<rm0Ft6smkJDv^=^Hs&MZ&<P|1n+0M)nb3HlpC;Bt9su89ZP^doBjd^
zr0xspwM^y=4!x$_!VL-_quUQAspZMq`j%Swz1mujg7~nT%W!~rm;=NMnaneA-V1UQ
zblA=FUYmec*i=%%YUOv>tXmOaCKS}{Ox&K7o&F&=>(0|qx8^qVXTEay28JP^PIazI
z&qzTNiuXVDfR)SW0SmV?m`;vn&Y_%|S8Kt}EqSLMIS11*{k&yfiaa)4b_^PNW&Qe@
zYM(Nio@YUFi`mufg1a7Fo1amkQu{;AZ>`bxxR;aluW66Ep8jk7*-ZbF{tRb6rMm}b
zd6tDgGwyeUgn9S-npYl^!l(}w%ca!D^N2m_=oG#{ut!+nJT>!6>|8fF!<G|_%RHwm
zrRq>-B%_xO$1<PG7{E;CcYI(z^Hm>M$^4lQY-GOg1DD;8odJ)OeX2|Wb2_F5Q1#s)
z*>GG_Z>#0G)H#jhl4J1=^2}SW*rW+8No^gn>slk}vt?3}goL}-WG{pAVLUS|RngYy
z!#<jMAx)c<5%K@U5HV^I<xm0GkkO#DF=tb=b}wv}BUNEzK3;v-Q<<m1(z@&*A4108
z^xxw=;i&v`y63}vvc=4DM){UA=QAqrJY=hQQLW%E(O`AN*rhc`&1eb|Uk~TX01c;D
zy?L=*&V>;G@U8-WLLcv`M7CAQ9cyV7boCG$rUk^+Zs)3PFghN!KYh>GA1P!lFwQF(
z7SLY70`uwL_r$QA{-y=u`B@~Pt3tHiw}@ureT9FIaFil{Dg+0Hq-&8Kl)kig;d7kM
z>2orL;M>bSB{p}O^zK?@yFmPWdeJmAi|J*3E@L>MT;_QImQ1%*^*kzPA)~fWxY!a4
zFtOST8j1wH2!o@nro;Tx*~3gR4Z%Gd)dEYx6l8Pzo{RBy?9@G%Y*%SaUSR_fCMZ>w
zt~C^RZrj{8{%q`cS+6itr<At_a4yWa`EB#|IQIOui#t$qxr;lNu2aqS`qRCAvuSC*
ziXWuNMMUb}KAwx_W1FPvd;7fqd5X5r4XP%(3AeR?Si76LI2R&lKk3<;Xh1%5=1VD;
zzswN~cRbmkGA`)_tWA;;79a@HtsR$dWnOYy=CB))n=4yktUJ~r4+ehoO(qNTlApJL
zwXd_vA=#K_-8aIbDN_hdK;h!P5e*nwYwA=_olOdBsH$JrvfS-@xu?(_N8ZO&$#J4~
z<S3f>P>KKx0vrU+&!mvumIR0D)j#g#I~VFj`mm=}AST$Ap~#rPeW0f~c<FQPXEU+G
zzEIf#xR%^x@C2B!nh{2~X1SD(vycv&!ng~_d~-BrMp(Ca%CHBb=Q%gJe-5g*n^k`O
zi;HAiW)Ps}obJ`><pAugj|v&ubseZ=G1cE%6QyDbyN_{;FoYb9ZqvT!{bsM~zm?Mv
zFB+g1i&&I+^0Zho7Sx_XeJn)1Wht`dEv%t;cY~bpsPWnSug9-M8)3es*6-BDZ4Btc
z-FXQhrz*7_KNSlv2hQpufQk1ChDoPjH{+?!%v8l|X#7nEn~o((@i$2}MM$EoGV7nn
zi|TJ->+Xe|8WvXEV)3$1olJ3Ltr~WH&Ew{gPuVnon*03xX&(3UM@Vi4x!$c(s<{r2
zFsN}AI}R6X(dF*&5pTzvR}dwAbKh2grG;Jk+a~=i1p1a4Ga>W?aSOED09U{Kj@1W|
zGpt*;EXIfoH23z4K^=La>})wrH*~-K;&izX^<OxCw2A)qOOxfM4YLux-{EcyH3To5
z;$q8d7d`7!XY~U`_jgGid(u_ZXFj1dpNj{YhnE}{XNs6EoArUUvUMN0fO=mZvWBuF
zSpyl*PT6?{1=_5p4$m6Y_T;C!dYI1XQw6S;V}y7N+qZM+YfeYgcW%GJKRWoqF~s>a
ztvRpfkRNv!9p3tZzl;T3`T#2gjnVg98I0<24r0*MJnmL$nPS&@@|IFB@}6t@RN7rG
z;~0hL(d<MvG3}&nzKuz{JQmoC7t*J0V>Q3PG;|l_cj-w?)iC{g4DoQ2X57ZVV`aNX
zqU&`HbI0HP_{K^F-VAZvxC=R<Y?^9sbF*b=6rOa;W#{elJdLBk^|DPqo!cxMrct-K
z;p}+UraP!Ifuz_TA*x?X`l%W&qw|`y{oP1sa$sOXFW5f80m&4_)o^sQYJXSDHp*as
z8(<)PDr+Y&N*>CNWtGhrzGXOkl<H!SP$kdfT#VyN{)TxWS9l|}o-RL?_X%rM83V0@
zh9qIEY`iQ^7%!VFCM<GFmO!o0&KPR7>_Rc&LfNGx;SvKD=MbHGNiD20v^NY~+-7m9
z^XNeQ-*tZ$%Spw0`g4+i*Z5CKYqVCA<Pk}(G*EHV8Y6#@7bmUITqUj0I*V&VL@T>u
zaaU-V#kI=7%o7ZA>j+b8G=(stO{kA$Ww=!iO?p1ZuoY1}u9rk1#oIBAMbQ(0ee7)(
z`*>1>kF9aHv%7uFbA$WG$(@gR>ivS){jq5J?&-ZBwSR9pRWqK}Cl9)l?S?IB|Hgv^
zIpTf?Csf0<Xx-nc^@t<>D(%t4=~@Q0cMhEX9aXxq&FcWN7KN=CXL<)mipfF6I|}9;
ziy0ypDUZc{2+@!G=*L6GLYrw<3{md@QF+aSyo4_OXwEy>gjWbn74dtYifphIq{g|Y
zf(YPYVOw@^*6&XJXZVI(gjvY1D{rqr1DONU^vS|mya-vihC%iX3h`wX#rQBXZ%6vJ
zm*j3nK^Y3^mUxncGr70LrF0j<=Ga#@S5uNUR}<0#M&CZA&OBYI<?v~iL(B5cabc97
zr>tA4T++qOTs?4?|JAt%OC&P!Rd8}E+)DlMrgB3ME<DJ_XW_xG>-6^P53W3Dmc1(v
zo+ny`@*~qbH(9rL^hd3_Gxfzeu3|Jli2tPG=YyR)`ahO@k_o&#kh@#cmPaVUcZF2K
zKB*&K!&H29*RYFPSYKP-MI+uNhn$IZUy^!UsXacm)RI$qB?~p1ou~5l1eRqbSN_D3
zE%h}^CiOK+wpQPHK?WDR*>**H$DibC#zX!jS2Licq)c4Rpxo8XJiCwG^XkrKtW=F;
zV+YKn#Mz8HHH4HhRpM=?zF%rUkt6uDe6igU)l$vHRQ&FXs4A+TtB$|R(IFF4Rn?wu
zmQRwK7QJW_?9wUjra?>*vQNC^y5Re*7t>kze>cI4Sfgj(|C0Lv;Stu{5Jr^e7x&jP
z^QMk#-AHHO{gQi+)(*_d<3dKt=E~x*7VZE;qcVZF$1O;|khemPd9$|-9A?A?j<CAM
z;b|4(VM-Q-=q4r?ZxcNBl6#yM%;UQ$wx3%|&$>8%orOAttphe>H=T3QOgd0H_Uc#S
zCk{|1jg%!X@NDTdo4j27CPHo^e#3?PkDx5P+p>hj5QjJ2P+9WMOG&>Mk<rrGM_+PB
zXf*f!TmW(lD5v7^c_g3!!}!Njnt^oDLOkqbruU(sd0y6x9|T%Z=J3jqW^VMl-obbv
z0}A-V7`zVUN+HfwXx)A$|L=2qf9<-xu!`2HbcKDM$q`3{D6`^|0{CQznc)UQNtP&*
zBfF};DCQsbdIsO=`@PDvG|dIrOb$sl{)jXBf<~fv$bCU0-Z2j%dsJ0S^La_bG9(Vi
z`P{Xh_TzH#OylRm&$6u!;W#a~Bc#0@`96f>v>l<GOzx+#*tn|DYp)BH_<@L$@9@M=
z@*TKpN`=3Ui!FY}`O5gt<o1S|OA>YDCg5}%i%$s08OziI%ZM#k{1GQwD^*aKHOaT1
zuD%x~6-Bw&uSawT;$b%&vU^Ux&F0X(7;8uUvT4>5S3~;y*xX=2Fai=2ZiB?k^J+-$
zV`Fe}*q!rb!XXV3y`LbFs36HZ)>wF0P$ILtot2F&C0<C4>XHO-rBFEPoRyNyQc?65
z(}<R%PBI;0x5m$wQ7b$T+rJ=hZ{q&L+=*#M*~Z0xSr8!*MR!&V$wsvdZA#o6vFlcf
zlqri%McZpeH#V@8z+l->BeMgumO^cH3`Vq!;tXcKoZiSJ`w$Adk=fgp>eTgnHt6%I
zRa7u)oN-pAFX@WoVXNy&H=A0LzI-W-U6Kx?{1!`KtMbP6kZYPsab<Rjn~3Z_=FpV|
z?*yJ(7&dQYf}45aq@=n@6r1FjO|9!|J%z0a`K1F+Jn?A=ZrsqRc!#S)SJ%z3V!`+r
z+Ixo1u8}9LWhFINfUJ7&W;x=VyW36c60z=C6<d2r2l4N5)u8nQ4qrW_!?~#Y0|E=Y
z&A0ZlY)rWZ*O5Bn2FC<HhNh5@NjZMSxtVTB>uos3X?JU*VNbz+FoBRFo^oSfgw$vp
zgJ2q=Ee;cH3_{Uxq&O@Y(=ra~T}cl-7^UxyGg`C<CK~P#hk>@ayc<&nE?jGN64Xm%
zahPbEmx{hO%ux?*97fSU7v{W=wpps+2mAz8@ZH5A$`Ub;-K}b=ucS4SU2_~J+E++4
zd^HXe?JFeuopFU$h_8^~Q=t}uPZon5s@YFOk&}eXg(|z;T4yZ5Pna94Oc50W?qa;d
z^&jqtHUz*Vb*?nEshCO?yC$UO;~|x5A8mdXc1^l?n0HN8%6MOsbl0Tnv`ZVird$#`
zU2KUF)ZekcAdjnThaI$cS+TCmxPtGP{-tKY?S<*=ZGr4t1P9OQ>f@tt)bq52Og(&y
z><*5*oi<J6y$JYLyOzxrUVVk)Rd$EB8Qd*~*0>?wV)V;9iDtA&Z#F9s|3hpo+G*d%
zznhT8%DSsyU{4Bq>zhbHPN{?s1-%V^s93$aNUzL0=#04G45t|$0rM#iOb)qWZs7Td
zafkV)ZAjwWarR0&Svg^zF@1LWIiKS2?O{$aEISdFT`Dib&Bc-6uW}>4)<#qQ`$bU?
zugb$~>|o8a>c4H*BVGO5TzLVuShpq?%Wq3!PKk#tk$xAG3ba8vsJJxpkVRT&LUgCs
zdfJVp8Z@5)|9^W)s(l-Zzsm{#647Yt-vV#fDr=U-)F~YN{_PB=3rHC8hd&KBvHRj>
z%-%?fKa?j9hcThlPtR&aJK^d#OKyFruSpuu{9tKx(9QXGfWYg5yqw#pnjR!T+9L))
z>Mr?$3S${HNZa`(kgO5AV5Pg@;c~n`5g;d0%S43ze)Li!)}-nFv}=5Vf7Kkd=cRhR
z;nK@pP_SZ?@TF#(grc;US|ZyI8nB1|K01La`c{?99&Yw@g^F$dBpbs}gK~uVcBpi|
z9eR!V_I+BUE9bkJ5w4-#W*GCCkFG1~Sm}KGe(S5xx9_jF#q;f=w%as??p&<4`4&1d
zrehr4gt2+yA-+G>Y;yx(1-&Vk1x3gqwmA2|-}fs2Wn4#a|2tggd(7v5DcAX4)2n0Q
zdlwR}^Sw(67QUZPxX$;66Rx9R;5rHhuA^Y!IttSB;@;+bg$`wSo!xw{`s;&k@JgU!
z(Uu_WHfAzzA-7T9d3wl8cQ=YFoV?MMx4<%vaFq0rTSmlEwIy{4mb%bEaH|y{x#$2W
z9tu?kc}x0fXSy6>lB~#jL_eiznlsi-XEh)EPtj<ZbVkzf0RQyr;Q&ig%DA6VcuDUG
z@K(?_ShbGOjw2&Hmgq*%j2rQ|sqFAJ&(7i!zOfnBH>?JEHv;vehg>O}l?impxuEZ~
za=!`9Yu9c*?)8KMXM?EeXXR}Nw_52Xogt&&+$@)xD&t%cHN~t5V;B{aCG!lLeP!ZX
zd5n&eZU`aYY|xGvjLh5J9*i6&UFP57hiOzwcG>tZX6<{e{9?HRakst+soe^;+Mfu>
zii=gD?L4!P#{18tPIS8{dzwwzY;N)yo<^_rYV=J|MbcDuq5SHkh4Pj1R99S{wQ|;E
z{>K^3OL77=#^#<s>)FlhRu53$Tn)B%6OwkH;AN>{%=_GY77M~BD3OW#%{3iG@r#+}
zDr~7)D4)z+Fg5T6bod~9Q|BMBb<6cforT?}qeeWb>MUUQjr#Y-TleJU+|>Y9YrUF`
z8djU*<Kt+xo{dK=MI_^_%5Ah);acG)<qjC#4uARSGWHmDbLn#W$9gdH$Dq%?r`?wv
z-(SMq!@kYx;{>K33uX4b#94#_r4^P`<j#y>HZJ=vYVsedDh^TC3F=xX^ARe{LbTR5
zwnK#6i?Zlt+rijz2BZB)OFllwDu0lT3kN$~xmIV<Jx71s#*EwYy34$D<0ilS@w_o-
zEiiw)Udu>&U6UUft2We}qXz3?&NyL}g?oDZpQk{{Q$z8UGjq88JGfo3#5r7M(yRZ(
zCRN{CDxh(zD%f|IlDYD&`!V3CrziY<ipZzf;40P{q)b)5zdU6J9nnw44FntnRwODo
zH?Zxak?7<t=np&*;nq!Dhtxl+Q-4jc_~wyX1dB0WzAB--U%U_&6I)!h^s%_Mv*j|T
z6;rOD@WAPAZH3V#&+GwJl<Fv}mrg>PZntgQ>Iv1$E2XTkqu%Z)cM8Q+ZV1OZwx8G%
zV@=)FQ=fTplCwVRtTk^}kM!FdQ<?v|>O$r;``$ub=CoNHhCY0FSv$oOt`=i1be>)q
zNn`>V<^k|*gwP;Gf=Mf-i@E^pPOUo5UNeX@Ve4!y8JJHbEGpT=zQN;aC=CPJf(Aj_
zF6N?EmWv(|dCMS8Iu+svF6x<;_+gI~xX6}=JwALk794!-6`}S?bx|8_t6d1-r{Vx;
zW`vsQBJWjrfYL4s-~lAI2Xs&Nh~-BoZj$6cV|Off3>ZhTIWy4Kqg5jm-=;Nll;;b&
zGyTEunN@9dv#9AZ(@w4-yS#j>?Cl`b!R<z182nH{QN#Q6f^IC}V0m2w!V#(O4vLdy
zlj^UAJnnkWtK;WAB*-*K(0(;!kLz7V%xA5(G7(-4(%%w_s5kv0>z=gN&s4JzTOUB*
zGj&o*9d?5>2z^nkgHUv&ZbzM&kKmGyWc?L|s^d!T0o}@_bM_?i2%85*>Wn{1p*V^s
z(laI&n75E#H!zUJWBv==q!>>zY_eOLN-669B5Dc`O0>^ja0$(}N&HM}J~TmnXwCbT
z)-#7^rjahT4Mh8H{%BM{r<$+K`41eNR2GT}rE-q)M7fVfch!7Y4^%|??IFA;tpc`x
z^kbHyugXFK7vt?zwT2#k7H_r1Vo%i>W__DZ(PP`0T<{adavGwPayKM<_qy2mdfEUW
zS?eZ==@;u`u6Jr!Y~aNP=bCMQr%Fap2tfdT**P`l*`jIMl2JVcnN={M<xDbZ@uJ%E
zkXw?=ct0DJ#Atp5USlm~-Xf3SJm1go%PGk%z)<T)M!isLrCBCbtd`|=r~4b+?jsP<
zisVEI^1fIEJ60mSR57F$Yewm(8cEtE^4hhS%AAD07E@ot3oik|D#n8v?6N<_<My8j
z<qIMXo4wEW-kR<QPMH75#Ws<zg3ngTqR(QVt=68(fIl*yi!Z82&kn}6wxK!wD}j$T
zGak(kFsu)RYxX**jrvy_OP3GY-;cCp`j4AoEXmuOMM3dT+k@<i5hgBVc$ddxbRn}?
z?iU#AA0-LW9yG*C12b9=sXeM7w{TZMn#UN-7CPOOhtwWbkntS_EAJ?}(xZ#zRC&zs
z$V?3AOlrLnPDZtuqBKNzsQ19XB-6DoA+YduIX$ei|6#doV)L9?cl=q$js*0pJlg(e
zM{s(UdK7c2D=FRo3b8Q7f$X4vYj##v5I#)Z;PTI}co#YZ|7=M?zdN?_{=8eW@XXJI
zYgo)e`EIT5qZxhu)k*xByA1%J>t`DMrK~V@{=A}{k@p4Cyg#pC*J<yO>Y4Hr?!k8d
zIB7|Ze>#Uo*2EaBnAn`$&vVdrgA_p~DMvsBUrxae%DI86`vC%=nzbe-f;%JkN4Z9n
zs7XF0)cSn2cT@k}jXqyI7UcbtT<}ASJ(|?*ws}Jixgr(ymN_a&TU$|d=^9+v=6Ogu
z9Y#2mIgv6W;3;-y7SIJ32+@j7loTqmS9_h_uzzDr{$yFAyUmzCS*-M%(6Ta_k7OwN
zPgxKA)=5W3vh%_EwooIR+?pEQM`H#85<`}rs^TJAZr5d6p^R1W$I0#EX-RO+HPZ*C
z_nN5h@k1eBgJbo9xIP-MjoNzPOY5l_z2lM`tOyQEV_JbPR52U_I25i3oAek!*n$kn
zpOcjI;~D-;Whg4XeHtydXgF|z6!@Nv@A8WFr)X{3j9Nj`^7%$4oN;Rfy<^2_d8&?}
zsZJ?Jvs~P08O5(>%O%79H`N+U6gvp#u1%-(jCtRjhG-5($kOn&m<h+Z_zOmq@lKNO
zY@?T!5mwO`q(5WZEF;Cqt{kk?Ccrk4qp6Fk$k8>x$AHNRrIOO$v;{F%u6~-AKsvS3
zhrhXyC^WujSJQ~r8P4BKY2D~>^)N7E1UW@D#UN}rj)g(RK`9mktdV$JR5=(am&+5G
zw?=<6quUpegW{4Ob$+2au<AkDIEXPTe%e#RfiQ+O{JdMC>s*G@^C@mqX?*lkZdST>
zT6|<mxQ02&*@piqb;A3ssKr018x%thX|MK2@M|7lP2Ei1?g40IU`#&N<?))2X$@r$
zE84og*5y>bhm)4GVzOuD;<tw*?qAi{jOV;^(VP&=Yqi$q<`)*(@D<{r7U&|Z(->KP
zH{5@>?ce-72G*-iJq7@*(1MKvH~(@+G+D!=pd583SemS1b+f{75^@al8Qa)d#Rfa%
zVX0P-`E-Ay%i~SN3Q*)Qkrna8J{4g~mEXhZhsqDhRdQE{_qb!eiIc;a`jnJP0afY#
zCi5?sWSjrVrILJgsq<!`6jv*NO#gz_8(7@G<HF&Vp8vZ_uwy;v<X~&h4Yt@uK^A*n
zN~U!2S4IMnc0r9e(mml{{AH5&Kvf4>=I6FJca45!RwsiGxn``xl8w@@ECq7Bm`Gut
zuY_z8s?_HH^T?7iJgYmvKdsntn=7D15Zqd@4dLfo^XUhZ5&V3s<k(|uVTyw+h*;T9
zm=?(@IQo1m0<V|LLm9z`<W*&u<Q~chVP!Oa`T3q|9iSy^<mZ`?Ae=QJK{%)NF=%q_
zJt7~?i0dO$KK*pPr#jv2Q@YCax3YwmS1mUNwQ>a;Ad7_=zx(`jWk9mhr!`?9SG^ES
z7FEWrBKZzd5KH7_10>MgwJ%~S{z6d;$J4(jBI1=z#fqQ3qS%nGVoTCry)5$Ek3v30
zFI%Cm)<sEdi~jT<DqBu->&Nu9w*3-#fHqgDliX=Vaagn}ILl2+?wVcFu@%e1C0!aa
zi85plnH!+41~Rh`_nFPmfI};uvdqj$>B`2s^cV6y2z@;`t;4QqExsJ&BNYOD3?p2f
zKR*mtwCrA7wGxzd%ygW3=yW3OPR^nL(c@v|aQ0lgd9EA_x67@W%E25Ddzj5s`I)aq
z+ZNbRZnBn^e=>YrPHouQT-r<{$B-Q@zH5Yw>#fdNzevEvd%4#!B~Cqs(LiPt(}Wcn
za>}FWq$K>w45h}`h^!f3OYx!$xn9CEXD(KXQE6O_w4ti!NvlU+vsx~mv@G5%0Rh6p
zUzSvI%=M!+7w#_W&v^P<m_+zq6h~<+C$;@kuKQ#B^RcN^4kVvx%d)-Iu3BH8w6B@e
zS7oB`)%01HfCP-tz%%fayOF$yQ@^Zh5_9HB-A_uA#2SJY(Rwow?EWxgIEV$xd{7Ue
zk&bDQBF7jR^O4Z7eI)v&X#36|zYYiJHigFK8+6RC88UVuXF==s*ac=eJ;edpu{gO#
z%VRv+V;2Q*l)bucpt4C1_LR}sW;Mm2d+_)k8<7fl+yBzMSoQayMl${`dC**(>6TUT
zGdVZ#sT6-gS1)O5+m`Z7ulcKxJa@*-AWXD7Gt{A3VzI;Rs60F@K^%vxJ7<;=X}m;u
z<{YmIdN$x=g>BISovhGJ!0rJVTKT?}r0F~vzdJ|ej{K<W?Mn|wa=3a_AW41*5b48v
z6%2j0axoj}?$$qjT#VB@++Zcn2kvqh!F2Uw!(g@<XPTq*nYJYC*+&e1eMdO)X8In4
z1Q&yRwTQCmm5_UoQ1=9B)*#_225GWZJbWs14;*C=W#SvgmkCEP$YhOEC$B#jh^Lc#
zH~6z~o!vYKcwC|Gk(7^1P$zinoV+lUpTcd&et1Ab3+QWUjlTl{3l_r6yJ0bxeC;(>
zJmqGM!Qd}m3C11_+Xf6m^R>Ac(($wiSPj}ZgF4x-=3>#&4zAieVRW$MPVkYOL8GFz
zCGO;^Y=CvCiI$u^3z^c&BVEPeJqhVb?a?v;57J;rRz5i{dKSs!2jFBC&4q9zQJ*|(
z>JAa)$*JUE`6R|0MlY6^&5SDHRO@6_$hGRzm1_g%71R=fMlF_W^R;Tcl+@Tyxtg5h
zrT=j;{WW;*Lr}Fc^%1V=tHakZOy(p~uM)8ju|@E@i#_QVbMZ=VAlJTtOCBDN-4nd>
zRP<a2{GKh59-r42)COZEp&S-@b>P*@X?T@#43XCt{HBe*TwhaXnD@a~%Rt|#p|UpY
z<ii%8*d*S(>xB)w?<LT_mlR9*svgoA{fsIqVNK>y1lH$j4wRpccKk0~6}X;JS!P*R
zLt!S|yh>r#qe@JiqzUtF>eX!K)?IyEoJU{fgz3n9hUH`oD=&BySfkX}8){gXweyl!
zdQaXHsdI|Z!`2pIumRL;T`WExWC}>SPx#fluiRY@yMd#wtDAp%(ENoShy2##SIKW3
zxS=cQzzxH7)_IV9!%{s#n|wE1j=g3i9UBEP_P+Y4)-s&`gtcrueI=}AalZs<{kkG`
z%oU99jD}g3f%F<sfzk&-7GQIQAw+vO<jaK38Kjr>OCaxO!Mh)k2rD8FiHu~Bha&mZ
zj3MCwm6;d&SoAi~pYnG@M(Z$|X5^xasa0H@_n9abGuzQg?%+$DS!HjWJ+ph;CNw0C
zO7@dJyq;Q;+w$T9ghfD6pQ-Fk<ZcR*(WHu+O#Mf{u|*(}fHZ<+L{mnvXXblG28%~I
zBVX3nGB)Jtll?ITj<VnR4U>>oQr{tgB{z)`JKYn;NVHq@b#<=w9W~rA>|~{~SEUd=
zJXnkf1LX!?-H>D_gW+q82Nd~K*|8{PELw%L;I@L~&XfFeR#bZos$Ianem_UwzmdpP
z2_}^1l6v3m+K%zhvBC0Fk(a3khf4&kVQyj<1*}mf0>B%oOD+bF2k%qO5?k<}QwSa@
ztka)>(^u2Y>RN7+jP_?sC8SS`)S3k<^!tg+v7;0Py>k~CZhzuRarGt~)@0iLNeBPf
zxqa8g+-F}b*_2508g?3=xWbE+=-fp`>5CSisg(K2%4dKwq2`l)33LA>gXqNwYxi|s
zK{6QD`X(e4=YIlV0zp3{(;r^#5H281+Nb;`1&|-2_xagIh!*q{$TD59RX*o(73tBO
z<LZp?=Y;@?YyS5yfeoN4_d~2xZ7{7i$uKBig9Vu#ac&iA1ZmIVFsQhZ^^gcPIt*pf
z1h)Di7I2eg%v>hE$rhcg@M0>L;%yn$3i+Y6M|IaYItg1&_Uc#Xw?X*OL?f2a)@n6*
z0#s~&p>kgh*O6KG(KzqW7v6yMDKIT}A6jR<-ri5bX%FV=cIYbsGijm1mIRX)I5G)q
z_enc4Q9;6HA*|5NnHPHF1-j5I-t8AzAGU^GSKGkrCMn-kpPMDol{VJV^r+bMhHLeu
zg}vB%)taaEX3A7jYku8oHA(e4vx~I`glJ><K3Xl;k8)msw4kaJP!(QAJGhUy77T;d
zxP>TY7k&ZO{7$p;eT40=K{CPyxy6WErPjgE>*mYYxXyb>__;xPW6h-YkBhr}JjF)n
zai7lHR}c&Jy*R{9Ron2D%tg5?3wN!H&Ca~o{{?G!rk{YCF81C8dsjt@w8pFHE0CST
zlKdhUjf)SGy|BuS7TogT85Bz%PG>acFKpDZI+<BXl6`{%;0jz5)zsgC_q+Syx82+y
zJHX$G)R3OpJ<qYvR)C9K<q2vJk3uzra>5pH#l<Vkif-P773RFojt=q<*MR&2RBVVx
z$_p1!n`tXt+oj`$alQM68%<fZcd-hx0p|j?+pry|>J=VyE*fzdvY#WK!x!;1{nVg;
zZuo8Y`C_FM_=!I1#Y#!KC-cbGGQ`R8dTU;o26r!xqZSUqi6(JIU(buvnk1dBqSM|a
z6y&+K?L&C=ZKyfP$+FM#af~?8$Y{2GydVn)Bjpz^7TGpo?cp>`AMY8|K~f0XVEYf=
z#^M~+3K>v$eR){YC!2$h!-RIF`zzco-h;6w8b?rcE_*7nbsv>fFX`cv_8GnoLNInH
zQD5ZXpi$BOnFo__z%UIYG7;bi?|gw%beqWI>D%^zFq@nKC=}npLS)7J<BqsiMcYdb
zijL(&B9r0?zoK{>kB-(F@MSdx__(HiNn9999@)~74x-H}RMAOI{UtT^zgA5X8Zcjz
zhd0vcn0ti)`4^SpiqC>5SB8<+1SFA>8;y7j(|^$?EzM)Df98g8)7M8M0*JPqYeA+w
zGJZo4BpHOiU}l(s(X7totoCmguIwUgtqNi1{p5eUz$}o)G6Ld|S49|v$)y>2aH7a>
z4~cAfI;Q|@oIsGbgTgDRiCt3g1XQliNzCowSvf@pfnj}6;?P0YKY5)<ddp7}N%I2Y
z_5b^+B~A7Ej0lOn?4}mcq4a_jXAAK&&+9N`x0xO0c-#$_f2kb6x$=c_Yt%yd=60gb
zSFBgq_tG^R*O0z(%_#2N-RQOB*IM-WwQKrbyLQ8WZ(Ms(-{*I(?-W+ocV5Wzc_Ggq
zKY4x2)wE4;k;VQ*JFANPwlm*~>|hz4fXr3rkq_h41?;?w>AtVN9Q&8T{@WD&RE4dm
zB0G`BM-Xr;a8`t(w9{S2>{FjhW#6~Im&#6O^?h;2X|&80jHjWF!c-K)p34ou39V&0
zo>y}Dk7b5YR*dnikp=7ov)jfd$<s-rJNqlngtOHF*|&`8q|?h96wVG^<<oCAFb~Nl
zRh)jt&E)tqxnpeye<pBtFAbO`{6?=1F31*#6|WzqY`|Mu)ZrEk2Oc)V0a44A^fslp
zkq!nhEKqu!dsge|!<fNrcrToBJc^lGz*lAM*ohm9b@mZfC{=#9K}KIsxZX8cI(nub
zqIGnk*RLP3#MqhtRf&&V;^SuTq@*s2FdCXh>8k)*yGSSA-))h2`-s^=Ke9D47)Ni7
z<TcAhUAEYg{b7qV*&vkK=8u$jRpQK8Pkg<PPJ`L+o=*;1ziYQB>?HKNGv+hJ0JuL8
zZt8ruPD}|4D_gs0>R*ouVZ5m^z&}H>;yuX56eEs<7_bRP3Cu$}v3<xV+77g&5Eipw
z(bnmzx<GsnDDKq$l2P!qaAAB%T{OC|u$Ta?zrd>p8X*N&3Vw(!d@F4Zi{)Q0%R@`2
zYnk;$_*PjYPk9rmHPSW)7Z_r(i`?y)h5JG)OA6)j^l*H6p}COB#WunhniYSIXh{7P
ztEzBkR&Z-jVecZ<+XnzNY<=)fl|jpZ{BI0&7f2c4oWk$Q6*=Is(b4N?P5^~ZGfr|V
z!0S5Qpu)QUmu<Hi6eTN|bonpg2j?{ym-G^CkzuAa1OIX(F!n*}W<N=>=j7u_=97$|
z<}2lWXIgJ!`kyi}NOkVa_vlsKy@;asHrYpL#&rCUiS_F{bzneKqGCT~<J8Hs0804|
z2nM&VyFW(ocR*ekKn3Z6B#`e+*Ge@Vhs6;>w4E55T=|_9i6CK&zeAtgjx&Gfd1&JB
zQpSMRq>W~2bq>0<46Z{C;@)K?E3`nzv}%KKC?M|5%AUMSUG|y&s#@{ywa4q;%f<_)
z@ZQoi_Hd8*Xr%RG7NQ^a(PojT8dMU|ojzI$e5*J_KjfpqcFI^}i0&eK1u>NNfRevD
z$B`>@T*2OEQF<H(J4sMnz(a60kOaj=LToH|`igobd165w%*XA}T~F;W46=v#tAWTU
z%^LgdMt|zt(5UfLdEaiuuTDOM+x%!K`bb`m&tx7%*S`(rj*AqzYUZb!kA`!gPJ9T`
zW63f?^YH5x2{(AyHTLO<L*4)OWX`|PMtHr&>kJj2A8!>}G-&*&FJF+k1veNvb1!7W
zBG%bOL|0D|k-$QZP1&#pH#L`}t9yC04z_z%u{$Tl8o19GZc9=l;zmfQA)&^UrBeE-
zF-l;@RLZuis+xb}DnH#99=2>Q3W^4<ggOo}>h?1~_A3>6aZ9!u5^DCdyR+2f*psS$
zaY-|pXMw7`=!cURZ-Qwh`&2GmDV6;tss1%_<(vsQtFLN~67n<$(~cguTd7HNLaWrr
zPlN2eRJ^y=R~*}%+3`c;1Ef5l3zeKjw6-TrdPRQjYAQcXBR?jpjsAoD$9;ZM$R9s}
z{)+rPK0g~`^7r`se2^atY}L;P`HMGwifFa=AM^Q9dy@Z{&(8<>kNNz3kUyM!hU)2k
z*AW-7ES+#eSOXzG4|8bgd%KjH<ZPCw5GQ33N%tqqS8goP(MI;x2PK7D_4->Z@Hhop
zN%VV3=B>xMjcz3oF<Fk2u;ct0-R99WiWfp6=C=*)^btclhjgD}Xy;7cN6h5cG?Z)k
z;TwF!@D2D%J?@5YSh>+6VvN3`YdyYeh{wJ`AaW_SBq4e!^Yskq{d&gQMZERU>!sr2
z)T0H-GBEr(-9}oHyoE9={e2lCeILK6R8QXd>~t>Zv@EZG1=r`rBf$V|9k3^Qg%F>M
z(0mrZt8|>qz85f%@Jnj;o)7Rznw~c&vl*+B0kJB*(j4IT)~T6jM#|=8*J>M_qgP_a
zq@n3{L43lZ-zrgDy|$NlrsJF~?9#hOkHC&hWbIl&S?`%w*AlxPg8rxaU_7k2QjOcr
zWHa?cnQi?Lg}7lPz!o771BrR2I$|K4=1F~11b9dQMaTC(sa4x8v;@dFN7xynn&A(Z
z)vg(cuD|ET%HaDf?#_F=Ty!>t>R7gsdT?L@M_>LU0qHmO*2C-$(8fHhodZ9$f`p1)
zEz@_t;9bFC-hNm0VDv_$1*XQM9$qhXc<3n|Ts`8t&wCg8K5|}LyvJP|%}m~Fe=nS^
zo^5OMvfOEf?C)bGYRwM&tyAlf+()<=tIbN1-WIgot=E6#I!n5{<|Efz0GUn0V4VAX
z^tVA`N&{mcyExub?eSeQ-!>)^7J<%b9-d20q%$v=RS2Yf98yNuJ<48&yvbVeKI`5G
zFuy}$_<&zZ-`j7O`)w(I-}yoj_$UiuD@C;Smf!W@U{qb>TL@0OFQ8=87Iitxln+zO
z7)D5}rvdesL@~^UOSod=`%;-vI1$QKoEOFN4*T-r10xdNU!j4QqO0tzJQ+`CujmSS
z*m@sZQv)AzoinkNYQuGGBTNV^P+fb;tmw&0PVjqzQkw4p-4*iVE}zwySQji@s09@^
zU%D?O^QC(;o-b-$rHf=U$*n&KJkflI=)*qxa6BDCG_tIqV*%D${Qg=^heut-imtq$
z&*0|zVSHUP9iEM@f6FVT!~2_EG?ytK4AWsD!*mEAF`>P`-9?zHKZ0@Hs2*|CPc24-
zM~nG{8_A#`d?JY+Waq0xzTwQeEJS1Ivo2bfSB?wmMO3OuG(@y2?c-W{4vP%KBxQd<
zRZ5(iO^s@s<!7LF6cTj>(1A=Rs>b{VdYIwP9DnADbh{wBJ#S{8?2AW=8+2u+Elmtg
zQ_H@marS?c&6(yJg{v&wTJr{dyv{zbhSvx08#4ReAYaOh@@;DJ8mQ6T2c!27Z~&TY
z0gerOFW3#8(EEpt-V-iz?4smR9q5JYjCQ0L<S8ZMiI+=`bld9a8R4>NabD2=o)NJN
zX2tnnR(2j`u)52Q)Oao+*;&Sf`S;AYXj0^V&Y9~3qs9~Ka2!uQlzW1y7+<#Yefq`x
zgh{iKL4UEcBWC(NBMA}rjFk{^&p8;Uh<G$JBU>;rA+^^>ANJ7+5r=5$NPyl##63%a
zh%wuJobj8Iqs^(zStlZXB~R~JXCsJ97*riYCA_w0(?u*N4`Wic$z+IW?l@DIi#|s<
zL1m7}H;c9Mc#XVVL30yYMH9k3-ej7b$2c~o6`pWCd|TirXOZnQCzm6(q2n0^Og5%a
zJLffWJq~v<%~kFQD^)k*RHw-t=6ZNo4h}l8X1*w!uuHPrOT_Erm!!~q%Jq}>RG<g^
zd_r?Z8^^KG${i{Fz>e0=;E&f-rMi+}CdWxv7>yImcrHC+X;-Lqr}5xMLmA%Vo6w}Z
z-`d9|`ByQrK2cXuWaB6LOW62Fx03Y<u+75_viFJaqsp?-kRjqC+4LV(;MR!m7xh{t
zjW2%Cc>RFh2^X=Ai2SEmS9cEQve#`64Yh{e@*QMt!0#QChKB2!EW>`}tP~&ZFKNo9
z#H0jQlkn;Ay%%&eXZJ?$UDdg+Z3geXte_ns+&ji)eQS^vb`L~m%5Ryux4+`Hq>{~U
zL8}O^m*Sd2hm>r365RmcuB30KSIFFZMB8>GtP;1sA=|yss&qAV*eaRnjy2`KGEwFI
z8#@z9t3N=RS#12*3+59%3G--59Oey;&8|QjU*iNuJ7kbRP#exToV{*n_CMQYM5Wc%
zyY1ITVSxBb9$4^0#C$O>Bs;8<ZIec?*>9pNZL8Nn`+1QfY;1g&ceWzDSb5)ac^RVA
z)P*mu3ne1P&PW0)<}`(CPJ7P0?SIzXr593Tiv~edS<dt9K(nLKa0H94F62w$6=&8B
zYMNyjj_yYGt14{I;F$RQc#1C&PM>#HX_^L#tEk9MMwugDj*@R)E&A`CMm=3n;Qw1t
z8&9}uDejwe0)pIqM4G|@yEYPE8fNwhy2FoWEo+6{X5|o$7q7H#Q2RH~LmT-@$Ip)p
zf6yoG*$LM*EvNd^DDr&`o067LD3{Ory;lMc=e(kof5M&5iszwhk(V+3DDDqwt>_7(
zM|4(~wNKF-_k1kj+<`xvI3}h!Xj66E{fF?lyEA9L9$|ipwF#>po`6G9UxzeDQt2~4
z?M9ebFyGY_?DP>ggolHol<@URIC=Wan_LB>OjE2OzAay8(&AOrb+CjVVjZ862t6ZZ
z&>=UJ8Pg?^7@VejeFKV#C3U%aXO?xsHfQAJE9lZ!7LERmU;M#%L$WjzHa6lhIv!7+
z2B}?S<M9Wx(gVjqWwQzKhYPJZWMu+afqXGIKDb^hSGy0choqH`+Qgx1#7D_Qu`0hd
z67&xZkVQk2jUYJC9GgU;!XPO)UR&N@vJr_f_1b^4o2gG!o8IXJ%<ZiAd4~D%3Cxd|
zf`Qdin!}05+zfAC5dUW!3VHfj%Uo9c<;<_+i^}~vl;z{6@l;LxYT0lgv|-$X5HXeg
zW){G2W>-V}Y8D^gI9V(IDyF#ZtK~~U2e4E=v^y-5klvQYtG$!jw4Bf*m5;gB0sdK$
z!i+y<_?mvqjc0#f)%<>TF;ugd-N+GtF~_0ZV{YPxEB;^)x38aFLUA4olBv(K!kX%s
z`&+iNO?`Hw4-rIIPJQ;mayt7F+ew9pi%aS3hvNv}NLu&VO-}YDb}po*u#2B(DVxF*
z2@&#DXAxmJGf*9+mE*0&sD+rtq)d^Ij|jDT>CSSEXt5tbh;(QE0oqDH=}VZWOjLvX
z=bB=D`<!e8Bs@{$=a$Xd#DW|3CV&xpwwAXHMbl7O>T|rx0kj3@V%4$`K%Hao8l9fD
zoMZ^FwKZ1O&mqcV$<kvmJ#dYlliYfDN|cyK!DAOyp>zraIVZ1&(fV+vzh2i%_&6ip
z6Jzj9ooVkMbL-}=Pi^2wGtRV2_~ZPPy{Htbh(3M?ptMdG!6&%+D>JY#yl0LewI;cH
zO)P^o>dG37!IlbNFi8>2CAb|p%Tu7-ix!*$R)GG=IKL-Moq<@xk<FvQ2|+U~s=eJz
z-rKuv$B%%l3%cTLYTtR!Ut#2r$~5{>kDN10d8@)O#YbRRsZTd8OQtkO`JZm0zw}5$
z^XZlXHUwd}ukY63RCXYau$n#%Cd}FZei=*fsZV#anHIgyBdofU|Br+_7+mVplSmx#
zJ+ahFAh;78VmWOG4c1{>mn{ib2A#UKpF0zsimLZ?ap)I^y9rk%@rX(G-j~Y5!ot)<
zzWpzqL!JSZRQ=Kd*X0ejpywr4Cr(22l#|vNC9hYQ8a;*>&m~mBrtL9zE;qgn3`vYh
zjpyh^Qr37LnJ@daA%$n5vM<Ky5oBd^_L=zGiq_za*90B`)$)>&4&E?=$z9G_TZ>@z
z=CDUL<?J5h6=pr{o)a?|&0Un_<R7^9yiLV|^hnYe)cvT-<ulI$n%DL1Vb}<N&X>=T
zQe=i<b=PXB-9c)6jGm}~+D5iHjqM6{;-3>s9r@NLZVhud-NKzIpDX8czMPBc#FlTA
z3wERYvOFre%jKiVkwNM+0}`~i{8-Ub;|p4JJ6LCNHq+~_WO;R=!wr>>ikXd;!_zz9
z>Btt?nU>E;A!DOBei<V}AN)+N-85IAS;1sHKEudKXItJLmdVuFt}wO&9fOZ1-1;+9
zo(^sWKY?9;W`+xRaTT*HX7LUz^_h9iSgt~rkLq2@vTRmJ`)osu^@8yf)xqmyvCBJs
zfE&+V2~%9mU2Xb&5-Fe(`syCi>N6xE?1cL*@8bjkZdM9~$2^ICs%_Y_K&r5C2jpml
zd<yXhXLL<&4-levAKMqU(cDoaM!A;6_e3x>{8Iygt023F3I^(9EPQGJ*L=<1%m+xj
zEoeNIu+@N|u(?!CPSf6Q&g3MBd=Z;9zrsYDFd%LqIPSJ&P8R+C{C%sJwdT@O+bD<Q
zESeJiJ(MohZk2z?qhA!M_<sO7y4Bo@2#FAe=7PwVM*_le0ImJ8iAXP3YQ+gk+LxQ)
z8Y0O2tdvsX|Iy1$yoSN0NPeO0nb3V5RMt<%8zEmWBW*lQmN5mMG}hYUEALTDdG<62
zdnJ+FDkCx`!{}wOj~R-qpAiWemM%?<s+uymm0CexE=_2q<6PWZPbk@qase&X1q7|W
zH0uxC*~05A`O=(Ze41-A{$|R+RF=}1P@&CTc7{q3#!I7r1dFV&B$tgwZ<9_|vgW1L
z3ff}0`NxPi3SBH;{$sw%`)D-8{$p);ztE0^U#`f-kJ3hjeQzb8znnK+685UxSn0>$
zGrqC%a-|k`>&(klqP{q2&ID{^i1za>dbv858x+i(rVz7=d%57}kW>PMOR9yFl7&js
z{WuG-^jec9ELfLIH__b?F3ASHp=GE@TFZlgK68UIYk26>Kdgv^PZ<fHDk0%vf4F@i
zH?tiKUZ+0%{Xp_Ou0DSNTZ;!hgeg*G{-7d^5_=6y0g(w53B7v8E#tpXyCQL)=-OWq
z$F5&Ya0?tt;eKi?mjM5Rrh0LlI82dA5RMTb1gtiB;!$OZb1+=q6zrI~fXkQwW7PVC
zF)b|*1+_{(^R(8N0y8Yk`G34DguQakL$#V@m$!}Ypjxxc`1@(wXX`y-L)a3GE6kD~
z_^>J#ZWsh3>%o98C_?c+RR1-{(YNp#W0O5p)Fh!h;6(t%BtF2*58)<4TnGky{m{-O
zgbvN4BwvGy^!NgTe1<v&x1dh&uYx*l^a5IbF#qakxgb`;`P}xkos{m;vSzPG8d|LT
zKU}s)LvSsaVxN>a_P9D)eNN!>dV)S*UD4V+TD%XytMl=Rt~$9HusS9Q<ZNnHn_Lh}
zi%Gp)<)2#Y8<JqcLkq8xiho~HJAA4UW7@9B6-zMjp}h<Fpl?C4!-Bjd$Rif{9{2e;
zcoL56U)9^;38ev97WJn#;;jBMmN_UN3*w_1bpG><P3??p>9Y28QSL0<6P4;jMvkbB
z+gx63%A8pvr)dB3A~qO&*k1SYl5r;vps4<T^s^L~Z~tD`bCH}%RTF$dK{Zfl`5%Lf
zBP{eTR5lKp^rZVgMsj-CDvb3Hr#067#xwehJmVh>qX*s0&a>Cki)%%J{}3gsT`2z&
zMjCVy*AbGPSu=dKS4FEemhhuKD9?o8+x9Txxylg!c^Y4r#CvdS|CO-fq1f%<%C)Ul
z9{=?2N34h5jHrV_?=nH6D3C`#Vr(L<@w4+1Ft?qGER5M<E+0PhOiI4Gm;z4d=8rls
zqKeV;BUpi$S_ke)w#IcR?kYI^tf9Y($~0_0nlq*rhsmA<xr-Yh1nm2(nXJlQWoBN*
z)cz?C)T=W!Ra>K7gYQU<(36CNDqAqEf-ycuP+rXAa3+V-^DvD61npkU9Bmk{1+#ch
za67vhdg;SFX<pmS-qSimSFUBKtI$r``5lpxk2Iv$R%?rT+*Q<NUx>H~!U1r7?ljH`
zZrg*iqWZ?Hg*^pt$)q;{a}}8ppeibIBjkla5U$T!Sb39Jx>L+s5VkPPsl`yWwvM`9
z^mWxWP#57)mmdZFtC4u~`Ji{I@_YH3I*RcAm}C%S4R31MaXK1_ZljF5n0_s7aY9>L
z&Om5iu~a5=Qf|@KrpENZ4P3HnXv;qD%O$&oO0vh;^UcEGM$N)7qQ=}GG#Cm%WkVa?
zqeW^?TL;}EY)y#sf2~`{0Fk*EYq{$3(@{ZiJa`x)4GCeqzJ~ZHJ(tU~O|)*ajZJV^
zRfus|f^nbaxUz&Hwk%<YEem0f8c=c#lf05PRTffafy36$*6wQ-L(SFH%+(?`d9_KX
zK4)=s)Ec`b3pH0$bDSj()#oe}ve>|=_R&Bd>Yrt)OzNwy+fvKcEXQFZzC@8>Wpy!J
zEz(v|mO=j{^?><2NFMd<Iu22f#c9Bqd!oN<`eU>+rTn90(Yp~H^AaGu<$JK7vsfln
zB{vpWFRU^jhbw$eb|{Y)c$B`oPMtmKDk}M>vda8-V(oR*V5rrnoBh3?$X411nG1PH
z0Dn3qj^%u}ly)<<o4>i_D1)M&n0)O_O1mnP5vbMn<t{mTz%v;eWY?}67;Gy0kfrc`
z<gKTKQTtJihsGwXOG|*Dv25r=+2Qw;0u$8Pu&=a3AM6udqu8`9wO_s>yGj28^taB*
zJn)2TAB4AI{A%@wWGhPmb~w`r4jUg$qgQ*>z)E{Bpf<48_9(dV$!3oS@13nweT)Nx
z+>T*jvFaJbD?5raZ(512TL406?N3VmC#GpJZ*ys@$@|D<+~3iufk_lLZW5o&MSvt3
z8!C%W`jPZH432%5hAkq=-{S^GBnyo2zKS-`Yzr}RILLv-FlW1b4`&wjy_i|zJBOuK
zX2c2JjXL`t<>ZC$3C!XQev%iw`JQ!i{(BC$)O=qo+vF-S`8Uhh=aQa1mo@lv*~|LA
zjCG2AU&&&ug+i~KD;uhRp&TnR(~YEJ)uwtt2OjOzV#Ny<XLSUB7DpT`uV;DDnXfAj
zU(!g8*-4q&oX>9P&xq7WVGH=!8j^?KL{r+U;pfo-ZGw+WcgGUchSgwOd-`;Q32OTE
z$6RM05q<sXrz38!=yKj<YATgJ&70|R_M!*7*<BxV7qYYHh}7Vu3JV|u7h}|yaS`xQ
zbglHEy~`f4?ixa$>tuy!pTqKaBd42dy8g}Me(LU}%&@cl(XhrtPhXF^^C%<uXDDmm
zL%aye_XrPe+V_Oap}hiC%}RdaH@c^wmeeuV8k+g1wV!{6q#Nqx;Nrag3`w5j%1jz`
z_hNTu@4AnbP3bUg*`KLRWT&$>JDDwyuAq!ZoT0n?uktUUnGruMG)RZ1mu~s%n~~ti
zAdgfq-{RM{<}n(G!^`9;`?YO*Z{?wB^o4QsTIxHg>@ojL*%;2z;_wC+BlYOg`+3Z3
zikm4RjGjmBn}`@xFG)Q%l8*%}1rJ3#t^wk)kx0%#ZQn$+WV0kp!$WxI_I`Y|-TgY#
zVaMr#pV8~b=yg3L*wQPaQ$NH1A*~AR*?I1W7MW^01g_Vk%&w({q?W_}foAa^ii&$6
ztR;o>OhYdfdO_QkBUKv1atkp$JcXfF9BOF^Arl^pc{DIi2A)*zS9e?6A4emBkd4S;
zj~OCe1#$H80i6n&H2V0wG~uj=e*66Ryu<{qeN|f)US|&T@9{;ss0tmZVD&JA<M`1U
z-!Dbm1m7EYJ$?0{*aizUv6eP8?4Wt(a{+SYRV)a`_$6C9bjmTsQ-s0A#L?8N_u@Jr
zjo!2N%Bnr9XoM(G<P08KzLTv;edf#^h^wYq6Dc!*K&eG0*dKHijj*leX8qHeImyUj
zTpZ3%EAqUoqCA4%L9fiLyJibJ_tD6hiiVq;CXdJxEAvOzgVf{2+qghQgG{$e(d4vM
z>XG$;U+apfjjG?xXxPn3)C{iSUh;6^!<Oo%W@4IURUMz-Tj|zdpGz+V(Km9P@VQvB
z$)(=<YD4$lz2*-cs&(WM$E`>%Lc)_OFzjHHwxFFi-r(!n&_)Af;_J-qs`MrMTgGa?
zrJ6Hhx~o**+AVz?3)>tiFxhOJaC<zWWy^mMyNe{3VPsOc<F!?0T+&(?zg1s?=FpM%
z9FgXLYXJ?c$PF!4lxtxlYkB!cQlJN7;b_X8<TmTm5`tS=i{aMx+(|q_D_+@r;|U~L
zxe!p3?T=2171T7ApNtCqDx*>56`*=6FNtpL2(T(lg=}Zq+s3DNOINp#yn1x?E2Bd}
z+s*~?qz&R|>x2>=t_S?0Nj*xojZhEu1o?yLw>kzOd;BGLf=RWrGW)BUU%SaX@tG#1
zaTB2wF7{SBZW>T+ske<`_N2d4rY5l{Ocd-%HA4@@RxlM0fcgW>?nBJ(@K(U$7FjRA
zH9~8R!0AZK021ja0N8Gd2j5vQ8x3G9C-HS1a|c+;<=m6HdU%(>Ah&v!dux>p#PFL3
zUR=*Jgq|5JVm;tdK;~E;hTz%{yVFCM%N`b7X}jLqDcKwE@4G#~ZcWXr0cxqnU&rth
zS2bwN-#Q*xLw#xOAt4VsFgk*4mAa+K31wqbWEMTOSEo*?MdZC%1|GltB6dbap8eLv
zz)QM|HL5<M4gHEvs`70l=>Uj4(8&PuAj1jwgn+i9{;eB2DeG~!vZA;~8*ojt0Hg|5
zMU>dMbrdTI(K(9DDH^U|7Hx0ch>8uq2T|OZUrB?lT}rUMfpN1f?WW*+8oe0kP0t#)
zRH`(qRMByz_cW?ft6xFCRZu|?u+<-07)DOfHjB3Q=N?7|Y`cPbAMmI`Y{2NIqLUgY
ziZv2UYHX_4sSnTpZe;{b-C>j05Xz8L(bP>xXp(}xau@kRbj$Re_xKm!y)x>)aj%$i
z=9l?CC@#{o5}1E)`;;y&koCUt#Aq@8wnS5G-8bQ~*ZsX&G{)Z&rtIgXh5IgtIqtVY
za+G|~wGH@Yz5?#JJ0LjLA>fXCIAKb%wdTanGCCC`R8)GX+IaA`0<hQ!fc>fgXhkV{
z5c=5`dPy0W6<AFOr2Lhp27Q4KS-VsnpzKsg@Y@w@g2f$D97{y9Oeu?U^f|wp)Mj2p
zUZ|}%v7ugaL}Rms+mBKAv8}`RvkQ_R5EQDd)qZS#7L#-=O>_eFvuljO?9j^7>WIFX
z1TsQV@KO<qS8%TzFM+ybtXEJ6>bYrWy$bC-FdUW$I}Y@e=UNb|Y+I(DcnBVRUMbmB
zSrJJheomAV^;ZkeGr32*{YUpxd_1)#E?%7D2QJ7cyvOa3YG7+L@-xm0iMBbR)?Vlf
z?Fg-H?$U(CK+&)CuCAYt{_!pAgrVO99`}GyQ5`!%u1_Jx`{JFj*W8uQUbk4pQ*&SB
ztJ6x~d{znrNUvn>nh2l-6GabZUi`=x+(D&_u=pw4qBsGMXmW*c&?0^GrBZD`TQN(L
zT0>nmW>u@yn7c|t`L=vu<o2SRDC6o>^CM_tVoW#c-rMp;3&OY6<g^nrF`jVtx@GIO
z{~m5ZDW5s6`Ksml?G6dUA9N73q(aiaw_E>kO5R!Q18r2aEjEgdn<soI!QYRB@|%^K
ztO-hm7<PiKohHyV6jPPPVqgroL~<qi{)>UtzB1Yxpy<#5L0<`lh(E+U_5Ld{BMpQ0
z&K8X8Wi+v*H*2^0bQ?wEAu!t)P~tn!$>8XO>%{hr*AQfwC9|4wZ)P7h!VgsVtx+52
z5!l;t6yqf;BcOL~iinT8`WF7_k-sOK!0)_Vsil~6yI5?bwg<Y!qk6U18})8kbZW!_
z*arJ`b^}H}4-Alg)qzpySu1RaJKFBn4jR(YWk}#&t#1yErHa-mFDBz=;Gikgl{Ng7
zd_Jrw^(5EF4B`kMM`GR;qo$RPaXgRSoz%9DgwasS&o_!+pKulZB=rya!H8?oU%`Oc
z0x$(*k}Y-`+v2B+Ya1lr-~`5PG6S<;$YGmE+u@j(!Q#;W{DyB0`vh@32>V?*vB>yG
z{<cmDZzs5`rcsyQSC61oJe(zrC&0BbKU<FheC(q4;unS~BzHl@`0$3KTXI6=)-%;}
z42{t-Y_N2v((2)gs@3CnNEGu6;OGeFfbxH)<3MZkFN}C|8=BwN$wTo!Li*rL(!Bfp
zLTz4Zx8jFAIi@N63(ch|d?>}2cg`D72n(unF)%YJgZeiTMoJm~3QL}Df$;;fy(x@f
z4Uq8N7?B}qNc?pu^-G#Xzl2LB5UHVvCH!_Kb0bNOtst7sb7^jpY$-pGd4Xfc1MS7E
zDx2x<&*)1O1@FlPN-5#JNkf%bM`P-mtRQV<dH2a~HkYz;_31q=Vee{BvE@U>B9-^F
zAg+6M`JQ%HcHPCHuE0I{45Xm~W&<<QsudOi!<G^ns28(IEOU}IP521e%q2rNf^fG|
zJ-vj~745Q7?6;8G(w?OHad;0Rf~^&{W7mu>lDyK4W%`Nk1=aVgNP>)i)P4|Zv_;fp
zVnfW91<5)fpgnY8kEs(LAoM_SUGnYpAh_n$X>7An)r(kla&RzkF5P4}Wj7i2l#fOL
zVLep*(Vh<2hHVC#DFRWg*_g{wsO4yn#T221Jb{~oa+DCRR(T{PzKf4{2|QrK=Lz8E
ziR}P=FrgAL1@ImZ?hy|(poc1jK2C(}C`3zXNDXOl4^(HxIzlv$8sO!FXss`yt#J-|
z++1_d8&J7b;`Z^l5A$9Eqg$OsSRQgb!D^^GYdY2R<#^G#t2$6wtyRLQ8#P+bOG!>p
zc4<dYprjVMFSbUhd#Tp$?SlbQ<uJwi-BlOJrpkAsB-%7PiUzGE(dJ@6(FytjW%u@n
z)uNM|&z+tOT^AORhp|YQ^z1b>N`L6Ia@g@oJ5<!J9JIY_OjaXfKF8G~>T~qCJCiky
z{&bCtwPMbvI*P3-t>c)x({5U*4w!EjOFj4Xe2!%2uNJW&-pMf3U(nJnkB_!4&%_Am
zoe}pr%t1N@F#-WEimsy7RUZGP(WY2cGz}|(#&jAd(bgI-0zBC(uCw(=gPt4@B8Q}j
zfnm?25oR%QkyEZo@mw8I*iK`LJ9oZaaXF!c(}@CIPoYS4vwJT`K&dY!@n0fqh^Luw
z!GGOfb0b~cld^yAs<~}{jU<#wlw0bCjo*{D%-U%_>I;c*MU*IQf2@t&-%u>OByWv$
zv=5<9s;Nt#X@&OFpKOhhr2KVU!%<|5^4M|`GXLwdF6xhUQ7ftWbR3@NenYGUUrGOR
zCXWAQoMEX6Xsoz@x#+ToUvix<!KJqRz?Luhm_KdkpG^)StdbCIl|1~CdxVy+@MLS;
z%h6DYaBG!#7^^}>!@PRp9uIJK;X5DrMgXX^BV~!iKxi4VO>LNAPxYWv*l{vyy+uPI
zdzt#i9Vb6l{A}#sk}7YvTO!6dVJXcJH|?n*#E)$6OAal$>nSSCg>xF#*TgB-mV&w_
zw-g+(D9~Pn&bXxdra08pTJh27-{<yzG{-;5S-jxa;Ct~X?XHT)h%*zfX)Gz(Aj1%N
zNFf?X9~bwf?{d(?Z`~SF*9Y@nDnu*BCB;E)-L+TxU?Y0aIQIuzJs;l9;!T{A2lFvY
z7A`(W@m}4OIms5VdoWBmkEst;djloJ9Irbj$A@Y~$+$*FQe_}1@<WZfRcsrs4>d(P
z@sF$TZwRGx<byA!WHS3zSE;wNl--WACof4S{E>QmY7eQIlOINsRBq#j)Y#vx%B?^B
zy^^M^9sL7IJNgIJ4j(&$$@Pbs=4xvH2==@G2HW1GiO<53Y!MpB^ixgDlSIZI2L>QA
zu5E&hn|otKKD#Joyp{UwnuD)qPSkoNJi9I>g-_Y6P_f18o*M|O%2Vu^cw#m|2_SS@
z$+itJ&A258oP<QvLs)!R8It&8p*Z-zZNi=#@AM~@bo;sa+fh?<++}spW3=0zU#oN>
z<~cNSLH+XFg7y{}(b9ACPQz)zg``u@Et^kf%Z5ZoSN)mnbCC27y0xRi_sUK#pIg%x
z)%dRqdPq&^kkR#NnLz7ry-Km8q~|W<C3+~FE7`|b@xyID#m+v4lUB}pdb**YwyTw=
z>(LjLXq8Sk`1UEOy}~u$S0jG!zhXdN#*v))fDI)cNt9E**lxbpi|Uor^cnK98Zqy_
z?t)KJzxXjE&o_p-S&`~?Rqg!K-jSx8*7oPCO8Ur0wj$c`!P(F;zP-AUMn2z-U?@to
zB0D7EBevx;quBD8;!*(b_dS2Uot_VUKGCN=&*Y?)dXe|}?kMi7-K*xCT!x0{hs?Qb
z9FD;ZCC?9+qJ*z4sUwT!<v)rt*M4bFmG2c1J3>Tu&k!UbG><H6%WB1s%r~;-BLf{3
z>5M={$c7e#iej!|eI2=^J;w=b&Ww^=+YvsJO}cPwKM*ZQQ~uuzj_(yXGFRJCK92T!
zx)asw2k;9O$lJOgSG}i9a)4t~qtBLi`A<J*U*PgYw{O@BRW{CiD0htp-~`8m)+l<F
zMDL(Jqk7S~`u)od{jv^inbW9R-M`#GL&Zk&{o<Pf`i*8`Bz?z+O8CQX9trHoUw-gf
zzEas(048Ljw82&+Q3Hl9{w7PGz59K{7^GFLIBPh&5rNr{4*e!8xG{njO0wM<TbA}k
z(~X*V0?coLGA1eJv@kzy0e#bdt62{zQV`uBZ0uh{ZuR{Zi;m?nJ<CCIAM7S$7zoe%
z|3f7(LI<XknCX;Fmhoq@Y#Qqubj;IbTvZ7R)^B&*4DVtRc+ml*Q<2|I{Pq<4Z52{s
zTf24z@dksZeuttNTXO!h1N{yBR`Weywp3={OS(?2QJ^$ywYA$9#|w}6h@rFX+&ETo
z7&XP7oXMqikWT-H)tgHe>d>byG4F$}s?Iu5hw{f`>)c9h;v0f+@=Rp2s;97G+>ifh
zfzG9juzrWNphi^G#=XCx%tj`Jx1@Uq0+#H9>Q2bduvdb74vjl-*0Cmod9(RGccrUX
zl&ZhdW!G3V6$yL75yk(m)6Tiq?dhZ^mi2c#f|Zl1;U}POsYOq?{q0oKj=H3Oi0hkY
z5#nu{pjSk}X97t~Q_5j#4Wxd&u6jtOtwF{X5J}kuwt5BIms^Q!kPI#{A$dhR8fix(
zMayP>O13~#xU(g0K@ExTo2lWGOp6LVRlfg?51K^`)+gIXqvR`BaSb9zF`1gc(C<!l
zWaCSqS^$1`GD@Ff&@63|8#30Q@DJ5|tDxRu;0t11mzWZ>OnzMP+{-0e&r6DVLJspq
z$Lp@H-Lo;*i&hs8Ts}vT34E?32j1s8L1?FPfTrG6p?B5ixOyL{R++y!(YJBdW8}be
zN>6WFt4jBg-|EyDV{J7pv`*W#ykZakCp_W_Qh5#eMv|9yU*2t#*M{(lt%L`4OXf)~
zm4L7-eFIF?Vvh+)m6Xa&iyd*YTmUr}<jA!Hv%XJdkPzkkm#--%0RVHjc&7q?VMiG!
zVC!DtETbQp<9VMi+&Ew=sO)WH%q2>wZw(VTwQbs#9~zX}w&b$j52H0oB`%1A=}lns
zbL1~3rM9=Q!_tKw?C7Z^aQ6R?xAy_7>$>hl&$(>i0+&A$NJ0{_t|TNO8_USX5_YhK
z?nN@Tv4mwgQBpO4010gV1(1yrH6i8#SJIV)WYXXUC&b1Mjd6=(+)xjl(v(bTLOUeG
z*U)}U9<OCObV{cuEq%jGY5aa`?RD-wU`^&_`n|!vy63mo+H0@9_S$RzIs2Sz(g6eK
zo;5`jw8_6L>WA6-%^`a!jMSeaw-+zL<yZ8dm-39CBO219_v6uRDR{mS1t9;jM>f&P
zwM^rz+x3b<ZRigw8Z$IR;(SbgR7)1lxQyeC(V&3S^e)HYJ#u>5V4Bk?kOJcfP8#7k
zO%`UJ8IPmqqqQhJAYugQy`i)8Q)~1>9+J|CM-tC53zx@o(8%6u*X=BfI}>hv3t2N@
zG|7!Q|EemdMOUqT6>_a8`{-M5%z%ATj@1ii#|5h7%0$2l2S-d1TSrYN9L1x-_JY&G
z6D{`j1YV@&<eO1*%BIb!g=i;jA=-+ySE{;r6ddJ-jqJ5uPO*JOugs8Zu)-!$6ME;C
zF7f-@dS{CiNAuz5*>QX({04JL`!uDAMP29cVr8i^GO0%LF^sIVcjE2UhSzqOf;hVZ
z3sGZsWjHUXJ4!k6qsXo~{^f6^hf~XSYWZ5;Qk_~FDLpz{FAOt}K!tqgF18sxpcP9w
z2U2w-s!6j}{Onl2>1y$62Z~<EaD}|+4df|lhCD_MElkv^46ZauKa_@-cWB*oD()9h
zs<^4E2(<^9B#d})NkbuBCWQEP8@c<A;`J}uT}M1AcH(0csdEqL*)p+XUoze_>^H^y
zd=huLFwW^m2|q$t(p&C$r0cVK3G(w%!}SVMQJV45Vx?Km)#J<XOVmC>->sBQ75!5~
z;SCn(XynrpnqFXl4*gNOGN#R#ek1Cieq`W=$XJwI--Y3)4u+p%&Gaz|6Ap%nbqsh)
zj9(=A*|qr)OE@_kOFqitw`Kea_Hqz3e}y`llmkLCs;-hT9Mapj@Cp)gO<fIyX~Y1$
zqLH3&J123*LCmBWk~vIolo`jHGzl%xPeOi1t<u9~2<T{Q6r}&iFp!vuNg`(|N2VDU
zp&80lj!dB_g8cHra^&T^2_F?8l1q2}=%W<_Kk+%|3geF~2q*hbq{L#g>UI|Icu_O<
z74xft>2fR^+RKyA)HW$>w|P8dw>Fi`hSc*Ym_(MP)00YCd|5h@b<izKcfUffIQkO1
zBGrvzcrLv`63^}>)2|b#{!P=HWW1xMK_|fJ9+dmMYNHKJOdribzT1a^6ZYdrGU;YG
ziNS;#Vn%3yz{@!9l1fGDLW{6jW=E!}WGmUewvxY*lJZu@{(=4OkND3rc}EJhidkH+
z115w}!pe-xxmO5XW8KQ=He-I}Rdo(=yBjY+iGC{qXO^X0GuEq&iSWMRggFE{ooU9P
z&3|0q8AJ^72zMH_DIOug%sDFuuST$?xLash^Bm;&(d|fnvxd7e^28r>i-Wm9l3^<@
zjhwX^&e~`PjJQmCp9r}a)Q$_#po_8C@jyK&GQ~7ESIfo|?b+n(du!^<6@V-KGB2f8
zv71fHq)dA-+t|Z*tMnZ<bp~9|I|PR$YV&LYJw~tP%*~Sps5<1JCb*U?P;~Po-Q&S|
zn*w{LM;vqX-h1}E%`0vdA3M#K+J%SVAwvIQ=gr<oEI7v-iTj8t_V=Aky22PLjS<u4
z9x;P@b9f(qn=Gnd+38rt;8+kgjgh|O9*DEk2jYUBh<uWjElV4fttnfm<R~-^eK2RX
zxckd*isJo~(=pWei!tK~^gWvp<P%tK3bOc99_+wg)LYP$$G#qp8dmo8lDjQVu66D8
z<FopjW$5~yhL5t0%2yR#)8iCW54wiDs2E8f#!Ct9^j<nlFX=<7JDNHU@h`@V#M;-k
zJt%!3ZX!<74Z99uMOYUsE~;4#d=1lE601=Jo+qhO%P~uFKud9b33%*oF~OoQD`AC>
z39Tfwd4SqHK@V<N#ffzwkgY^^Lf^Qk<z9qbyw#JQ%)l!^E~35K;_wxU8&ANTZ#H2*
zfoLQiOFNRw3oO!73!{D@!C0k1-P2eL;kHthmVTO~D<t;99Qf*{<uKHZnTrLQi^c0y
zs!$}VavEhVhU*~8S4nq3X?Rqa3e|u5(gnY!KRob4H`06Y@@DLgOw#F+7j*HJ+*C+p
zEx-cn0S)PgG*<CaR3K^{%s=L0LvHheTA)|FRI=MbCbbY~OL2qocnM=50Y<6RR=%LE
zK`%*!2Eo|DSWpIIA&Q<qr}vcV9)q!K$%O|g3o>ue{oD|ApvX0er%v1fJ2j9nnt(PP
zO&Cjvx`ADZF)FAI!OJv|Uw4947>_$X-sXn4z`=lGIT(rm@E|s=K`p8r)hY;$h=A5n
z!{#vF(Tk_EEmw5VD8LJgxn(bq1vPz)@{XA(tGkwo__nQ&HpU7PQzl5qZ?3=+K|1U~
ze6tBaEX(Gmw`$_I0Ws<?xS$8AkE%eFR&t!0rw*&GL81=QQ{yja0i>*Hs(wnU!<Y#v
zQZ>?WkzNvo-^IkaAa{5DtE#&J9cYq-OLiRA<d{tVVtIqNm{sB}W_Z@B0nb|D5uZvm
zPtM$cFZdY>V{W`~Y$OK7jl@jGz(A8R_{9W=wgb^{kvcrbgI_hfph7WtN7)Nnzy+?7
z3Q$s=8lvnDZKvop#V_DB#r-i6ZHGv>WPgF@2&fnLfs5LwF6u6Okw$3zF^!sox;1@M
zho)w6hbziVi{!qGv2EnHZSHxR>RZ%d#Zz@Yi~<h(47%|~8W`C<M?ODCK3@m%=-wB!
z4zz-{|DeY_dG~y5fDTFo-2GHizZ=h?y`UARgR<1Vc$pFs>(sdJL>z!BDFE>w<c!l3
z-%OR9pVY-cfJ^mJFTi%OTwNd)xTmB4jm0Ajjm3{A;MWSteJLWzB}b);;Q$Kem3Xq&
z3b^eT>M=hjB7~6%<FyIdCX|DE`AS@OJSx^5zZy>lB@-f%LTaV%>}yOo7C#q{s?NpZ
z7nCk2D?UIyc#(YTg3enPt~d;{6lcW|TU-xi>WLqrl1AcDtrxUxb~@Bn4N+p9nt}FU
zVJ#|+1Tvs)i(bmq8$THzaV_Sq*4z^RsZC<HtQo?&74$zebGL`(alB*M9<D=CJ>g-z
z2QS~U(?{HkZkvD6J&|CiPbADF*y%H<yrgp>v`UxJI9F46c*k_gj8V(3WmkA&iCbFs
z^-#i4f?QLL5$j+R4|x0Wi-&I+n8t%wu7e7^ay>zX>+7&97|1ygo)K_Pdv<^a-)!AR
z+~aQhnyaIEw7JFV4$I+w(pcDZA(|E7IXXQ<m<i7iW=KoS7BCsWlcAUy2~WSm9)-*b
zM)6{Q`+5vN&qde7Z=0H~7txXI^d<MQI|fmY(rvTj8uQ_Zl`!~IHStBZ!55ij?HmC1
zB)edJfKi@!F%eL0Gy@oo!Rwyzo2{@1zNXv|fahG%z(CZtGKXb1qyd?Y!eO0dLTWmu
zurFZofwL>8@uKf#3h|*RtcXvm8TvKGN)&~sT`?<&8KsC!CIA|X!HdQ)EXU%U-D#AM
z*VR<29ykJqxQT=*LQ|y11+7M<9m$-nVoyTYXEZ@&4$!<{04Fpch?)vty`)DCHwsWc
zhD!>U^!tvL_y<pIz5uQ8TWR$Avjx`54Xw~u(E*T;8U!?I3k)SAaSL&fG93?9YqhF1
zT2&cl^pJq(ZShR+MLg3B!Sp&~wki$gv>L^jw}f_}39W^OpxHqHbqA(uko%9hLp1#g
zQP*&Lo9HnpZV#S{QF9PeE36FzQH!3&-3Tas0sla;(<lv%ciKQK7&aH>cX*=}-fl-D
zI(ABE%-4}{&RnI@dDX<j&zOkl#1Sm0DI#G&*NI|({U9MiAl~2^(_wTOJ7_y}h#H+=
z%F3fcG~S>&LfNNWC>l*if6?IUr^J5OxMMhH7Mk|J45Kl6qJlwyQEbqDH7bqo1JU?_
zqdGN#VLqbgYIGls#eFfuF`_4rNj=o6b+{#iA-NJm@+#UGIzY-2T_YN^Ga`xpz&B5c
z^U*=a(9J9~W@{&{A|1(H$#66p1K`Dcn8C$P#YPlbiS4D)u-850=0rWDf)~#~W8NJ*
z4R27>czGWU&ZN*HUDv5@b~zb7zz#mlr!e|P0zR|Qm|-Zq$OvOk6*~>TBOGa*bOA~I
zG>?oW{}_pbFJPCReBqN6B6Z>fkChYfU_3yle-eXOJ~l><#9<@%f*OyTrJ-^*Zj?s7
z8Jbby7a-^h2dqgn=0!BL>afrj-%n#%KMi+P815E$WTQ!Am7eph86!vHMrr9W8aEe*
zOmp_2+(yG4%eE(G7)Fr@!^R_0)M_ni^%$uuH6KS=;zEuQvX}bk3%dH${14o*hsJ1n
z-VTEj_7n`d?xE|H47#RASu^lJCtdRo9pTgrU8gKCe=c3qZz#i-im#XG+QaD^$FLEh
z&S8s4|Af7v((?Jb7Pk^rN)BJQlAO$Jy5596X`5#gmhv{wrkM!OX4=W6$8)qA1W9ko
zzWEAWr)MZHOV-ev=_Jwh7Sss3LP&Z9Y6(K(is<VUHi*8?WU*U{IC~0SyTRERN_I`s
z+4<U&65&aK&a6y2bWXvWxAAZhQ3+y1WzO^<U1w$EQF~QX#?)|%W@eaIK%}h`tVuSW
z0(C$(g(hj4`a0bk;VBE61h$<av!a31(^YyNehMOzh?E>|!fj>D<H2)&OhwC4OH1Xq
zG4y7t>=L5hlf}#dQ-$bOc^`J<BFQ<3-bkf{o!kPtWp)mbL~XyF<CGsid;%UXSCk+d
zMnmORx<S1CqK8l!5|!Iz7gWquXU>qET|=6au6GyW)&o86o6EM`JxeZ@mP4dHP>Fm>
zp)$Hz_OR0_F8$tKQmlHMwZE^0wM<b3EtE4GIiV$dHC=hRn`CoN-SpYZno6Aa7mzB%
z$xe0uIR~km9g>>W97!YhVDB<VykQ=51ib?VQC@Q-k9&u{E_Pgx>mr!Pas8ci%InF+
z9<Svh;&+BQ6Pe5W&T6EP_uY}g--jYpGvaxl1-P02eV9?H;$E_b`Ch&@9A85dJHvY=
ze9aoZhr;Oi2Y8HwA<cugMX8cD7oKvNA%%>?J@i8rSTeJI#^DJvNPe+u?Q!uC$vl&k
z7e9CF!st52*^X!<!hyXii0220jh>oC=gP(jM}~bA!7{k(J;Gws&{Ak-Fcjga5Y8Jj
zTZBZAo&beEQ9~fF%Hoz?lgJ%{9@TWR<*R~1JgC4DoFr!c#1t{R^KqYw81gF_ayZSe
zrG`u6Fql6mCdw~nHm)^Aj|`PvO(~BpP(}|m3_sDBhW%f@E~Io_Q+g!O?hlV!)JEPc
z;-M$psFBd`2_DO29axmFcn!(d%4)cY&DX2_N?g}BLAl5>=uW6|5gyxNHak6&%kq1f
zt&vE6T`6tP9P~gu&&cUZtS8cQ5yvD5`NN4+1hE~5o56v(5swE*867@Oj3^^WZ&<<4
zs7<=2T>Oi!Y98}6wFFnfP32V=Zfpmma9g`4iX>5vv-B9&FO>-j!GpP7PpnAu=x%&s
zRp7ZRtO1o#1m_1oD#eZaq;}FYmCaS=mpb7sS;R@NFF;#RH{qv3*s9{3aHL}P&q}<T
zZ5;zadZNLQ7I_|h70sO{C9Z>?%3$lU^Lz@{gK97=*XC>{lempOt=6dh=~*J-TzY^G
zDXie-iB#%mS}4_xalMk^Jm<0N@GB5OA%IkoXo_yeO^c3EJ8j~=H@a^(es>qGv=oK6
z)$}UWuX<xtf6Pz}HlYS$G2NSsCE354rwmx2jIY@Nin&MpD%&$Vo73C4K=#sKouJ^A
zb5^zVaUdwmX5*&FuuwKmaiMbDg?1WNIN2~3jp_493>IH)vDm-tXCc3U3qQEEmUWKG
zK083+s7EQh?<g9lsd{5buQE-8r`?rYUd^g`WD;_|k0hmVC!${i3r1tB6U9woM_TpZ
zmDD6MOu#RJ@vncaq3Ykm!a8aW(_#FS1?8YLgML(NU2+lIjDopQjrLDbmD4W#Qq&k$
z)oL{wJ9=$g;Kyh%*ccWJ5eD?7adbSKAHm{sC`yfBgPx^=fP(K%L;c_)TzZY0ugO;^
zTTKST6uw4lQIw+S+eS}@^~-dEeh&ztppckqQ5?Iy4sNxpM5?S-jg1Nil&`e08_?qb
z%#@u=dH5u}%4>ntDW*4YQ<dSRWaHxFO>7{Pt^03N3)MykoF=psf*y}7Ez^{aBZ?`Q
zH*m_!u%@EHC6kM4^$<fAJL(sw=sG)xRQMt?qPY<e&@*792?}4+IAE{O1JVBP8a5p&
z+Xh%TOK3w7D#kf&Vxw!1YUS%!+K7iVhfVW|`brNg$-Yg8hT(E--&c?etS~VLqIR?j
z#wO}3G?wbh269On&j5@7IwP*RN%S<UBYlm8j<Z~}hYIcBLNl{ylrJcj(V;Vk`Jn)Z
zyG~fr7T(#vUU(;)E2o5{bp;}vT|vI)Ds)Xyb?}2TwT;Aft(sgJcR~nU2S9z;D4}_s
z2RBPqSqmI#l38x7Ml3_uP-cSdPOm4&V9XwROXp2fwXG!f7WiO-EY*lv?kGYTC3dq;
zlk|3>X_LpGwkX_v3G#SDM4U8OL$ott{f$?v5xPCX(<m<Twpg0ddlo3ScfwSwQD|DE
zd<7MQrkOmnal#}gT){5GFwLjX4F{u#xc`4On=GOiZ@*rMSI6le@tAUPL|Bf3(122O
zLk1yufW?0mUudB#_7r}=RP@Vo=o%Ajw66|QX?QI{uXM++j!_xf6ML@^5!T5mBF%a&
zhUhNr2pHbW*od#ezF0!Ad%gzqDAj|*gOuYpv;fTBxO=~TmWbN_UI(KANyPSb)C=XL
z;W>SV+IuqS`Wpoiw*CNJr({xkKh(6Zzd0Kb`^`nZCbuTGH&zLzb344r-tWOkiFva*
zf_p9ac{w$Y^QmeXcbJI%O_YYUQ5I$wy*dF_rYV@}ljeFmlj21|H&^fNb3|?FHetO5
zPWe`uR)r>0Im@ORL=+{#Ytq>kgJ-A$=%6}O4x*jTsYXm=EOUhBIm1zEGzvdqGmI@X
zlJJ|bQ5o89$W)8C0ZB>kpxy@V(9`w1t)v$kMo*I89ahE_^bFk$zlFTEMSj~$*WNr;
zz<p_W0Y8=)t>&W#VsOhR7^}Lmm4i*6*;w2h!D|Ocar2bq6JH<3_%(b)cn^wu;BND*
z(tA@>fAq@<)lY+`9=G_wG7c&avKN0ROBr|@L?7NHM*`l5=9#JuKg5YMic4&*_YrQ~
zxX-re!j>!dy=hMWppc}eaDe-8lCGz5i&OQuXE_{>8T7-C7`I`zQ4F{w{r6xaRGocE
zt~px$K8w;58_aqKs6P+WPYg|C0fXjWj8ayV7O{?X6ff!H?(=c}HZD^0Pb-Zk(nwD?
zEAV|?D0}1gSE%t?bp!o7XwC!3`Z*S6;w8GUN!(?`fz(w~wYdT=+^$@WQGwWHw`z;)
z;}-hGGAy_eoSZYwM>72&la|)!xEs@TihYf4L~2KNW`4juNw3d;h!(_nL}Q}<`6F6{
z7quc<!sUZ_%_OXd3E?zujbnzj;Kt^1Pn_zDYuluPn|e0!Zyr(Jn}bws%&z-WRyL}(
z)KKcW)Ugyb<|O_45@o<Icj4DyCMi~%U+t$9gpRZpEEst|!u;wGrSaOz{3<$vq9>Dx
zxx0}zuxiJ`0A>&4+vp^-Cf=#QgMdjeTV*>^?&OnAj45(O^VM!H7lZ3Lh64oiDNN!w
zU2H*^K%wGx8uDssj$}wFQD-d#WjHc3!Xyas6RctngR|!pCNo6*3fQsq9p##TWtcsi
zyxF{o4wI%TarYUgq$q)Y#slT)YSZ{<@1p&1n)SwnBAptIrt$@NQ-eB#SvK}~X3Sph
zX<;^`2eG3KnZ^$nC@VV4IMxr8@Fy*l5Lp~}B^HI#^Ye*q`V3iF&zgP=rI7{2Lm`YJ
zyplL`4s$oeyK(&TtLlzf<jx%7OvkUv@$a63VSszskLRV}E4us4cyZuI^+&BRb*6}n
zf%>hu#-sU1;7E(v6xO~>waz1g;?_SU{zaJxLaol~MaECj3*72l8BIIKP3E15#_e)m
zQgp}Smt>~n)olDyyjsRxRXRZ7QInp9WVM8IWojk4Z-W}xFuy@9Zdll;mNxdJs=m~P
zRJD{kpQaYm`ZuY;O^ch9waMB<3OGw!HUAG4G^=|4|E^KkB9Th=CXFSj)ue@FXHtKX
z>R*%ICJoZPade{^-xy3)-Kl{z6-q-Kok|O7quEStsi3bCy~j!a&r=lC3b`j3|92sc
zo2`S~-hT-I_sb#`{&Ij0pW?O+LTZ498q2l9e*RbJ`T450g_!=Tnnt&RYQA3Jn8*|2
zg#sG1=Azo7F|2X3P4Y7FuWNC3jwTS8DCZVXuKz2j40Nn9ghqmYgPw|bg`qlE<zuU*
zm=gcX6;=xwk&YppyUpJsBXmU~_G5EsGeuAQ{%%|udFU|MYzyMd3gxGyY@81qrgEI=
zGvv$Tv>KU(D--MAbNM9IQIwU(Me!;hrIl3@CnXxP`(%|7L;e+P9YEF;h9c$|QU(-b
zL;Wm^MHHZ;^15*`8m|E!jm5itN8GsEH^xJhFkwQ)L-@~fsH0}&V5IMl!teRbN2|qX
zo_!3t5jjV6o?H?PZG)-@ksrs^F=3mF#?kaew)oGWCeP5iiM|js7>nCb18z0s9@6=_
zZphE{Hk@j*=@!SCe?nZKXh;?MSu0g&*&T?(X|Yf|4mAwN+vybO_?p|wlc|Dn!FZgk
z7}6Y^nx2WfNo^lXMa0BCCfvQTv_^yEp27Iuc-%(nOJJc$!d(`gvi=NK#qu^w^~LvU
zBFD_MOx%=Rj>j30kTcbb)Wvul)?iXVljg`dt@w!q+_{}jz=4*&L^YJym!t-$uT3N^
zB&nsOP_pVt9!pje$@APLe}-JNoz42-nr2PEL{?@W^ZeqHW_3yPw!5XP8PBXo&3hI+
zYSCkP)Uqd-s0I^zDJ+a8sqv)wB(<0nOjg~=qseMKd4>i2gF#(jA~~>8g*FauB#kMF
zx{I_VK_K?7<>a=teAJY*sGmu(*y8_?P0Vc@LK{?Q!_)>fzhQL)%arsWr4MXSLmS2@
z-Fz)QqCXrfr;qj?^`v4Q31bRKBh*{W*EoD&SjsBGMj)kptvD=)VTr)BLRCB)g&(pU
zr#ZFx`Y4?u#?ZV-ivvA7`TFvjtY2TTvs&1>QHX5F_f7O1+{VRf&qi1~(AzQzzTu(3
z##^4|8@fI;MK#|jqD(s8w9vHBe4|g%%7Ugj#y>YwXTg?228Ym9SlNp;?=nZBRWJRl
zG-7Gch5Hg%H}^)3M5(c;QG_6OC{FcYqKe~(LvgrqyX;Y`p2<WtmpGoHCR0MGswZ_R
zRV}9uZ&IV1hPJAat#ez|!q&xY%G%buUG;CD+^(j#&umxS83P$=C}TN8jqh08K}u1%
zH2RP5%w-iSR;p^A(f$i8hjqlJp3Q1@OIx}MrVpp9(eyD)4bnqfRS%UhxNUkHRg)fs
zl0I&_`@`0S#e2ecEKE1Lq8q!N15{!Cq!VTt0WO%IS;CSF^XwJuLt@G>6OBN%5RIGi
zTxSG^p7<4yYD=6*R#V9XDVPr|(e%yuIqayg7@ehdGk%Wd@L*$ZDO#;WuewzrZaPja
z#7%nCw5KOo^(8MStI@PEos)S-ajPDYwx0~W_zN@WM{87JCF$=%l?&8V)DpW3JsIOc
zDCTVhMA=@wuE4V^6s6O1C<i}_Lgy>khfGj#^Nl^@=Ru67QEE17K&c^wX*Gr=AlGJ*
z>XyDhYubq@lr|HEadUx9_H(A?V8r^L(fVWukDo@sKJPqx8b3sUv;RET>Pu7oX~Xzu
z(<ILh_|X7<Fo271qZS)TnNCr&Dbu{ZaL@2`m^|2YS5hSD$B!D?2!k^P1=C$k89mv!
z?b*jf&m>iZd>`X6rWd@IKWR=zscEdoG5GLeooha&Fib`E7I?G@#NxLI`(iP=STtgo
zPH|e+EM8)b<;VZ7z@!lA7s6Z7S^P4oPd|Kw6EZpgC9;?CePA=OfS6}uW@6P0*7}q~
zC(W%%DkQFc9P(raMuUPj4wAcgOR(h7;wgb6Yi1^Kr+Jl8MAH;@6@AO!T|$)2nG|BQ
zl{e?{=*!OaCUTGjY&a3yn`i(B+>%1D2re|KcL~fbWu!scjEUnqJWuCUiJ6|u4F{JE
zanb4hjA_PUrlSoJ^>X<dZh)Pyl_)izG?=1>Qzlc?bjo;|noL_vQ&!qohMLH*bT-Xe
z7|E7PQ#3PuRjX)U+wo&7dSyR$I5648vLKM6LMf}MDv&lneb(Fx1p`>dCru`+>Ey*^
zWhKw2s>RfxrgDg}^y)z`h_`am*hXGZ=p^qIy6Y9xdv@k7dm!=Pwm;1}&0W(}X?Gr%
zhmO5KYn~<C$WNW%KCl&Om{r?omTjx=PGPlK$=MbtF`Fy8)uL;CyO^%ss<@GOw&|dm
zgCTP{U9F}MQb4v-GmyGSgR{AfOB>tLO|vXB6GP89?lMHTv0D+utubfHJf_Up{=!5}
zjo??9ITK0^V222ij^JJ|HzYbEyVPFz9t6<oCoXSF8JG*NWCU5C?JyR$Qs=i%XQ<hX
z`3z#&KBNr9o#E~M8EP<NSf`=fDeTN{TiQ-68DOCu*lmIBDztrUhnm<ivs=yWUfiRs
zJrjG?)ZSIxOFJ-qK+PVQxmV5IJNAH@cwq4XJ1=I+|9g4iq=l+)8;^bcJJjHg(cNl%
zcVG_-1`%$XqIXWs4740=o2r3rGuzbsw(cFOcgNsvHN1OyH?yM=-E;$BVtZhR3hn6I
ztp;{4YSJS4KQ^2eb}sHzOFONdYIWzBR}JmzWx780Fpkr2B<NmP*q$%kJHkC(X3gfk
z6AamV@5^NMXl8BF!3vpBn%QU(>dcgWKX@APWMCT$K8W66Q|KB@>?QY>@HOYy;h}l?
z`0n5y)xBpSQ!QooafqOFxnpYXNa1LJS>5ioJu0+kK2t4b_G-2?5`~&B?_S-l+A`-d
z)k0<ulOD+6Qc_5-9r-jCEbkxBQj=K|2h`Mop@VAV;Os%0DEAP|9R(D$(z8`CYcxwu
zWQ`wClLrP5s^Nn(nzxWt!6I-dYlJ)`ctC{?j2%!D2L=wRp@Y+!sfbF>@T$RkNA6Q|
z_swK;F?Q<gy({-}Lb0$-XAj;_2@j31#xzYg4^2k2e`r}5E$T`)$2=m~M%@vhTT8{<
zPj(<PHjp@|?U<t-=h6E|A5`NHjz6R(9||2&Jx2zPsG%dnN7U$%i6d(2$ovttcx3g6
z3gj*1sinNYBP#UB<fCf((WOV#@}pyqsfouHA6M4nE03$TqXS3P(9!9mYWC>-QMGuq
zyFm38Ec!{)d$P$SGccxFI2d?91s)hVq^v{MgKFi$vBPTO@XTQ~cX<A=T0FdbSgjuJ
zKB9V$3>{G;M|$#9UmniKjODF7qS_wqdsGcPx|FY$^Lrmx{f`emu7)3<d0fps-gZ<4
zk9Hqby+_B7s>!3XJ}UO%ZuVL@HLZ<D&1ssHVu0;YyuWS5;s=ITV^zT2?dCI$SanXi
z)wFv!4rlf->zc*yO!6UqHJ%VmQr$@-$!aW_r*C?*Ck0cBp%gWeGMAzjQkFQjh6|OV
zCS$Qp{mYj(PiwhI4f>+6MxMeAbv|N@&H1}4PkD~OkyBK4Kh_?Yi&4s>=m8nDjaJqT
z3t)<%`f!95o2YZ#2Ok|%MyATD#!Xh@K3q$#w1m^8DiA;9Q6rw&L^Yqdn5eA8o+J!w
z<4I~VX*x;GCbcE2U~)GH`NORgYOt@j64%e(=}742y3m;*g{kpWv<i8K6V+(qc%qt2
zTuM~Sdc`-Gq=u7blQ1C9^Fqs!q1)5rQT?7_j~f0Zvi5rhJZg|~je9Wj4i`wZ%ty5M
z>TMpAju^S=;S_8RxMy(Xj%->lC#dCwKCY32Ey6Jrx6HMBESId_W(G~{#NZAaHp6L>
zYd(fPhxL3O_F)#dbLxa~%=U=<aF*`R`cQ-UFgglNBPgbi=xMTfG(-~kc%ylAfa%<W
z@5xg}H`aUpQNJ4ZPduTfo>+ZC1&XX9wNf<nq#Aj$yIA!WPo7fKr&dp?KuNGfb(f5k
zsIiiX5;avaQ=;ZdhM!fV&(1!p7M=~BCVfaKj@)aR{ZCL*0otun)#~X~!3rsp#XCUe
zJ^kb<qbKH0sD%?fPpG~pf<>ykXrf3>6%9VEhM%54sTNQ6oKk(K`cJ9BQ&XqZ%&ECk
zYT?w<DYbm6hlA2R6I>-8g74vo=EZnc)b_LrK0SI;jh|dPsg_T!oK$V622QD=QzNI;
z*r|zAtgOvGrrD>9Rp{j1^ANjVQOFD7IbA}97-7&@0X+JMIpqQ^xVu%j%N$Qv#u%m)
zdU0!p;S!xh;svjHe3*wE9{7CdbJ#-XZAQ~q$^D5Vx^I-O`9Q3>GmS>}{!=ro{LZr!
zn0rr+o>CL17Ijh!Plvr~^<*E}3#2g^jkMi~s(RT3gr2w}Zjy}|A5YPqDxwrUt9~X(
z<K<-hY&@2{eLS2#12wSYh@FL6%Kyv~wc_}uRa!EcC*gFMMc_7VzACK7%}l*za1u;e
zDi1dY&Z){lT1apZon^m0g}PwYjMu}!vHM+k?roB-T7tS~$t<)4j*D3)-4I$y8Qq}9
zH!N&WOB<FpsAavyVwPCEgoN->lMfD>r;BO#wGgi%4sbcA(M|L5;~VrWAnE#C2Gz5W
zwjvSW25EsaB6}Lo!9=(PPYd=wmtM3zhukTUJ;iy?phDRy7vZnpg&T$AQ30hwSgMiS
z=iS7a&KnqfD{_(U0K#gZorS01j3LjgKaC{Zbmi;Pam@2^d!CNUEX9u{U^mO+Kv7nv
z8v~I5d%puF>bIyjW?_n{xW0Hbf{i&6T@Ft4ZEnKyRZ?LAN9-{U;nC){xB=R?8%b2d
z=`-Al7xRew9EP!J>{wI&iUPWWMAC5le2GO;5*@Q)GF&tRHz|S%sym@alTlwdQUdQ4
znw2>u(VeS?aiX1a)YKxV&<$-g?4-$ixzn_+x1@jIfzxO_WjY^=qy2oR#;ob($p#_>
z>dGt5S$)%Xfa70HKiLBNUhuEVAh$yew4xx#<)P8S*Nb%B@6uDw7tSc7Pj7i)%ViLg
zb=L%KAoOvYytqKVMZ4(S6a-mJEY7>P&Hiz8EDpjd5B`0Bq2EOyWhM$=!{ZZ#K48)Z
z=|d)g6gT}lkxBn9;@Xuzu5f)LqiqYmdi!$e-{}JU`&!Br<#=t5a=f-Mv=Lu#a`rdV
zDa)G~{I9o&uB!`iD&I&p<iT{ZQb7qN{Yf~<gE=*B#97$O$HP2$sCPOB&$;s8Gfpi!
zo4}7E@KbpF2qZtf$Imqyu=RpV-zDd>k;HA_9&d;an?Vje!(i|eIyem1LwBymW7R|~
zmi%+ED1jcR6hTXF_%1!@$I|gcTHJ_CfM)PG5IyNf`_6$iJXZV)-TlJ;HU3_ARtQb)
z#TLo3iyy{P^n4WNEWD*Zlz=C9q)vF`SG7gs|KS*%r-_+~!81imF?bHRn~uo!)3L!J
zHy-l913!4iA)ILnH~TR@W1EX_D)+_>(thGtyqb!~jX4M7So~BxHpds@NkI?2-NA&1
zxIFhTrhmv&@Rfp(2Oq-S_{zk`ix1%ud{z5X8#lPV&}sNOOn<=TZ`Y5B**x(X;Hh29
z<6F-Y{QtnS{0n%}K!ZLBW%$a}Wl;+2O6dVy&-qeIpr<eB=}Po>Ciyy&eSs8T`v&5p
zI)cwrhOg{n;KDcQO=%%~O=y`NhA&|H+V%5rVw1!Y(8(q?ZozfB15^Gy2cB4mSAo3|
zdJ>gO{8Fc+Gq|qxv5nhtX_u2Uj_Yg(Uc$AboMh7JEIv}V<SV$&l)8zo$^A%k$Vi?7
zrt*Z2qa<iw2c~u+nKpeM*98&0!7Es7x1kI;q;<cW?KUidPV$9+BbB}QKj%-)2R#cP
zDL=Im*SQYdi)-b;eYhssVuRFCT<41o?6hU1<@$7+r{NlHo<?P?I&3cKo5-dM4*wE8
zHoK9YEg$6H44uq{!qhTIy<t2RGOxR?MP0?M(5{%y*bdx3Yv*34bq7BVU>OR2OaA%n
zo%GvVySLrielz37j?g{VcV6>e-4)#3wWl+)V{c$z`+gK-*FS^o(dxh>xGum)^vzht
zb>7GI-9frK?7pK1SbU7zNnG5v8^?s{<Jv|!;DyfjOrwsYx{kMu;+y6TS19Vbx)yyE
z$E3R4opBwAi0y2ku)f|PvZAiG{@#9EQ~QV=y<@l@!bkMqMLHJx*p9dg^X(!Y$|L!9
zQM-mBcy}SLry_U{we>1KS@=NrOyo01wcKllzf*NY2V#8fv4vN0|ENpim&o2bfh=x(
zY)o>=4{U4+ozftC?+TtER>xWApmD6r^mV!l1F9pcJ(?JF{lT}N^qKTC-!p+YAG-wE
z2DB_Hvq;l??fTsSY&XyjP`P7;>?_CGH&ceNE(a!Yo$J7MokiDsNk?yles3YJ>0{G_
zxL&}A_)yk;wfO4Ob_yE4F0-)H)e+?jD4$futNF6YKGnLv@HpUej04h^*^S^SE{w!6
zJ1@0)U~S&;_#k=jF91)kqd#hWF@r%mT?>96x|`kz{tnvRoQdG!ap1xT{$O4H56J#Q
z4m$Zwr0p>W0?u^M7uV1+F9If+w%)V2KBM)<RX7eE%*J)G#GEW5;7OrY=+@{<?f(+G
zng43BFZhhF>!h#ql&_=2A9&W+j>ma9#(?($lxt@DQpdjt-ebWxOkdZV@UsEa*YTF=
z3%rdn>RTp>6@NNJ`ZS6^1vdKHQysLxU1*~Jp=Hn{e(>1i1xEP~lKwNg?{{HLj5^+~
zI?WC}E=rjXfyK=7rM6Lhjt4HbS0h2{nu~AJznf^sf@geP&-*$*>FfBEFL2h^u3sEx
z>;CW=<ZHEckM*^?f%EWz9uH5UuN7!L?ij~zMfq;3lkLg3&2HBXyujmn>^1jQ97E_z
z==5|X29hX~?JGu}2iFS@O#PVZO>GX@k8nG#oeMldb-SW{c+Gfb=jj1GXI-8Nq*Weg
zzwa=P2TVJ*ik%)Mn+QE%_}ZQA_Gk#SGdgdW=0zIxdvqRO!@6&F8OJ&?_lhcPSGo~0
zG|wlx@!hZEf2VPbJ7otIKy`<V{2YAM;*%-358o47=CNR+uPe#lncR`$4{Y$YZzLA7
z5Ar=$hObPAUyHoQ0!XL&*mwxn#rTLmkFDUkRp*2K+Fe+~@}Jn^@kSsShXTeiZ3~h0
z_#9~Y_y|5qd1~<y*+;2t%<*W<Jr;=dx4S!Z;1rz;%0TbYwqRej)0dH6tZmHcBL7$c
ze3Y(*&sR)*j<M8E??pO%ndZQ_3iml5wfot1Z722glg=USN8x&kJpUkQg${k}^d+Ph
zInuqrXp6#Jd@bpE`)DNd2V(r~@-7!C?>O;RY8}`6(Q&fdxNSEY1$jI~BL2Me4;~^b
zL_hQBKG<cz%c!&Cwm$!f@8c;z@Qd$Dxkc2TSRYWE`0d$%O`8P`ZO!e)of=r$@=26u
zl!g7|V}0hy0nmGFon<r<J3W;L+7+Fb+f!sdHG*`EWz<gGcm2{4MW3gkzv<R%gu*~-
zJByRN#b-f-t{ks??PW!Q(vHtUob4OWOyFAPdz5UapQN^Ibc}h_H(<|GR3?0t<@tl3
z^>vl{I?H?=<-WjqUwZ{Hh5hK%GOmm4w)5*gA$2Xu2d&p3-%g)FdWj=leEQi;q&xcO
zv*gp+_(&T*+luRSX+x3qY&Y>Z>?!tob`0rZy(Df)Ij0q9^by|EZd{+S{atvyNP{g-
zpCg)<Z_gdV<G<Md^e}iDwePQqp`zn+UeNO#^_6_joke=Fqs{HLo1Jd}^c+V%vBPs@
zKRb>~9kAO19qe|H*d%h!w19TbVIz@qW*X`1`otLvY1NuHTu&+EdD3@I$Cz+G6`%QJ
zF8W}Z9_y}SUa5Q?F;84GueyR!oza0<-GfA*vptYg@bOr8b_VG=AJ_BKRJV0{emWCA
zKjupvra6YM{d*j#rCy(z2fytn1-M2#er5$t4%xJZId(h7*B$G-<@Vi-^WBK|g%W(%
zJ-%SFuPepZvC$Vu^|hxRzY~2e@oG{hufIj_QmSjd){msx{iw8((jBo^^eF8|`Z~Q!
z$B}l<_BClI;V&b<C=mb7!8P<Qga0Lz=vdMXyT&~e@Z2(Qx^Cdq%Jt}LF;`=Q?#}p*
zg!V+Ri<}pSaGfD??6gUw*}91Ty|{?9kgm^~`GObeC>w`p4b!X3?!r0*^EVfU@pfts
zd#+T6wlrluan0~uHGM&suPcg!bYUPyNBUgKKtIU?A1Ty6a?|jIT-T!tuPOi4XkReK
z*M-3D?uhdT;(hH2z8lW=Y@9=OyFQ|OBifT<Y&c%p^A>5V&xJsjzQu7??C`l+q_4BX
z=SbESTh_657q(H50zUc&zw$zMFl;*<3q<+a6|AA>GE!bMcxap=eVlQ<B^T);o8w1e
zz{T53qEE{(=(av0t7RH#&b5f7U!uA@{SSSB^t*Hx`k~L#SalWil9&@gv&+>P)u94<
zQJ16nUK$4<Y(jiDjbkAf*}Wsq7l<!xPk7>nc|Gcyx*FXT+vx_s$Zn;WIjr+wz1ZpU
zVTIKm?e8)J>;|@tTE`$eMUSP&I-~s^F@C$(hsVv!Q;^lCWgY8C@C7`6c3WG<=c^$j
zJ6u*k9q%$}MaQc-FY=<UU+BSCK=-we#beR=N;>>(MElt_qe!os^(tEXqwswZ`k1*|
zXI=~Yu@>&YZ3TNREM>N#NE2<ybAf>8M0;Y;-4#c@4g`_)<tw;ewROi@l-AWG%U+90
z8@|#CzBWBp;3^zLULhZp`5&jJ(7%?^PJTSOcFYT$L0XYxTtIz^H_(SK=t({pj?YqV
z2k~6_c#P^Gd!AhzD>{~tc5ZE%okd7%(Xw$B*1Hq!ez){4@;T9k`-AA%HAH@LcO8Qy
zEAQiF1ecLs>#$iqFzS1?3~&AN*z2^L#s1LAYbE}x&-w$O^tFGg_)gL7C%d1z_4JKr
zuAd5??s~5COb1?POMlX4p<4I}+DxzWx=#8#Px(7ae1T{EyqXQixLfc`7xrT;Ue8`_
z@&%g<yIMLg`8r<m2VVBKw~|b0uRC5`m#nqjokpa|c!#U-yyMO=t}S~kkhznW7%bz}
zetZS(Iqk6y#3Wijb#h?J&^$N<6K+%P3--2#w0X|}=vaS-W0KUnX90G@eua(~j$pBF
zFQ@kk^0#OmigeV+_^iwBwa|_sO*?Tl&exUT>qztml8Wq=Z7{ym)1FMs;rjjt$&<D6
z3A>MZhyl-<QX9MR@!~^x6<;%2M+_#|{OXFLZ7`zR^YE`t<67n;9aukx_r9h6eekt4
z`|Vy}$nK+jXix6Xol(9H<qt&L8(Cu8*Mq3vsIH%{J*BWSu_Gx!)e-tP3g8p6=J2&A
z6?P_ccmgyAd+|XT-z-LciM7YNqOkPAs>d%i$k#k?^r9WSx(tl%H(kCPQNED!U61x%
zi}78J^#$Gjt~g&uLSevD*q(UH?38_3DdVkClrd<>9Nzq**oD{(`O?<kDnl9b_I}*X
zC$4S{ZtL3KnbEN$z}u$w9Oj(@<iTDF*#uYNdb~r<J*&&$bv*pgziu5rR01CO6Zsj|
z@qC|!$2e(+p+(U19sN2T7&dwrI+{}Eu|TrFJq5{P-`_fi>mD7$*R-wV{VfZ0doN(!
z{^W0ufEJMUwS8LT4(EZ^YU@h(gxaG5v@<038Xm?xam1HefVw<!bzg9QS61hNj(Y?5
zwP%9~e<J6DN?b4KGPv*I-V)bY=)Y40dadl2`2s#)yPtB0^ZyR%csIJ2m@MPF_@+4r
z@3$T6^cA)r@9+l-i5QkK>cw@x&C}uY1^m8tVi6ri$A2;XcgsMRvAwV}rUNf@Yv*^P
z<{}+Be2DglPWPuy>wUa#v#{&78R#+Fzha)ae#5+W)4U3wykmB}%25e(7M<_+vuOPF
zr_TQxjOe?K*wu}B!>ult?`F{Dzj4(Sy5=gpe%*EThAVi}b)xH*tFznHaoZKR<2rFC
z;JOxawZF=nbvatzAIw3X#Db9FyKeffxqMfnd_m>wiuQG44ioDOxP4l8ug)`pb~Mu*
z?d;t1MZKtg-=DyjeQzaS(pJ#^mfVZG6Xm<DeBIH$TQR<yvA!E_UntIZEx~`)Q*=E(
znAnv>+fRWFy!j-){sZ!L8E1Li^Juw0B!9`oN81*6r!lW@!}^8x7@xgy;7;c4z1{n6
z?Z257y7&5h*RrqPA3WIgKxa<Jp}>Rfv{@;B|HB}zr7wlYc=Bn~=ZC0|IV5YGn;!p-
z#88s&db00Yitp+Me{iF(E7jkb=I_{47}(st#dnKu{)&zNq!Dpq7;W|i=u>$0n(Dlv
zp1dAZ*Kl9xy6Ot4j+-iQOSN|^6y??B{fNF-B#*BhQ~8dd3S1?Qun+#2^bXi-W5@kT
z@s%I<!QPn9@m{g-#y3oVsNeKo|EB4`_J--d`ljg*4w(L~w@iQM+h)hN%%b++G;e>+
z?Cvvfy>8z8x*5Qk9sVQR!UjL_;%mu{iFzJJQ`YcA40b2JS!$P69q+jRwu8qJ2mag*
zTVnr!;-Rn0<L^xLbtGXR@be&mykfV%D1qG&hbUh7u4VbI9`FV4^>^Lp>$u+^ICwm?
z@B04E?Dhw!(D1(SPvI*GZa?Qa7Ro$+eQ(!+j{BaubI<La-QHWfZtlLZ=i0uj`-54X
z_Xe`t@7H;yet$#yuV}q6*Y8liK=g4v-Pe)MOYC?k5oMSj&4;+%;W{3O>NI5#7oNYx
z_Zl9G?KV@9ma~+N`XLTYQ(r!I^`^_;amRJ)PKWDur>ncmbt~w)dDV5}nk#hObv@*|
zh8}&()z$6lL~nl8)!w7`+@-v~hpaW_Id#(g0?#cPM*K=4+vZGHW?+AN)}5`lw{>s7
zm2q>&je9~nuY0fUx{9;JoqId>k&37%>WgC@_>#FirG(R`ab4gTd$b*K6a?v0PCx00
z@&_<TM|WzS$v+_XwK;sj7bK?Q)QDlkW$SzxlM87T5FlRRf0N`Azm3WNl><}z5!QMe
z|D1zsjDa+E;wq9}jccdgVxNB=#F-TAS(bqR+4k*sJhv0OlWry7Ou4ZkwDEfCwY007
zf}6Xxbf$M~4Qz|ZGZRR!d`CH6B3fobG1??2>?gd&hZ|sp3$obOa|PGVy=BrFc|DUD
zA8_A>bU7B-?C(k~3?*Gp_Fqf!U)>PgSlF57>)7OL-$LBdU(AhYU;EzXu?~+fkcipS
zQ#Ye-;NZ*km}{|D-NCr7_)ZQ@cKN9m@=rPDv1CJ(pEd?rLt6La9e7ehpOy&g{iYXe
zMUFa4ecmM77CO>J-eyXd`P8uv3@GuwfMYx~x4?!ul``Izbo%ZbzI~)S@75zXAHDI3
zQ2zDDu04MBXt1E`7!B0z#~~{dAL#i+3&yW=GNy2aPuSJ$U^kKSpPUB!vaaV{=27NT
z6C~52x76pU#dYZ-?`aYq49mmnpu^-j3i%}Tdb$T+*pDIK3y&pPLYp;=#CdI7Ul5}#
zhS<0hbWETtro$}<1jG+MJ&O8`Yu;O!hbP?icDvkc&(-qr(h6h4yu`q+eg2NDz=4yu
zGrD)&y65K38{W{a>$|V*xtbZ=+qu8}Udr;ZKXMBCi-Hr-H%^Dq7h&$fmaBjuVf+ue
ztAq6gAm0Y=ou_YL-X#HMk-)sluw1VZndp4G0yssmo4$d)g4s!cZTUP+2ksL(do%Em
zU~Y8aaly21fV}U(=3E}%3Iv`P%*!ZXYG=}i^{`wYj0@&XO1h>$(D`PVMvPPKYoW8>
z*lFP3WM2z?^E#}1uq1*%eI0%4I?RE`P6PjT`<lLi*+1<x;2pS@VVoN0qn`o0xiK^o
z(I^OTy9w`3U~UJ&`z*s`dka1c+=y%Xqx{E#;dA^)7#rQ@xQ0wGz_rNZ@;{67W86+i
zC;n=D_)qj}T8BIB^Js8B;)gTo^Js#<X`e?E{I+H2vuA?84ID(iTEST0#4ks1%&L8U
zOz7XS&y5NG&^{j~_>b(fVS@kEGW1z6!9M|Rf&6sAKLri}=L!B#z!Yz6yac=o>=pVy
zT82IgCitI$&wwuan~A{Vz}cE^dV!aKMV@&8Z!pdTJ}Y#zmp%g~_$Y9{BYz3-6!00L
zqkZ)GFTr)dWItQp7l7x13xs~%GB$V}_#41wz#*Z--uk?k;O|+6KJO*?N0y<_cL|2Q
z^!YBq{{Y+zc}im(Q#De_A2S5QfA#q;!P|hTeRBoxvd?u1&Y?FjgTA0~?DIIc<pvge
zx(e)bT*Cj1<@&Wray>X7x|Hiv_PH&gS6Z$w6d}D$==GMN&u0n!^S~pJFZOZ;fJr}T
zFITr^=yO@Z-)lwHw1BQ45AnzKCNT9c8xH|f|FiLTfv*7f3;(}?ydGfj7uOWdYyle6
zBL6>uzY^Ha|KEYDBRFcK<qDJmm+AUM?Y7T+>HJamTdvOsKu;0+5esiI22L0H6Ttnz
z(mqjVfyaP{gkFia6;pYIf<JG$KBV>#eWF72He+Cq(BA+a1r7;*pWbi`JS7<Zv?m34
zRq$Wg=e#slF3UAU{^N$e7!On;aK8h?U-nKA7XE|4)E_bhe*)NY(4Vo-dI|j;<Yj?Q
zFGDB!jlc(h$$o^d*k`_k|F(V3OEBWq0T1}c1b=|`s1<o=fAt5TPXdcSs~-bXyvP#%
zMa$6Vy99p(O!g7`MkiRV*Jmle)+c%s=+s`~|Ir66*Vj>gv?27z?DJn*Ui3-4{Milu
zYQbf|<bTD2t1Q<j?C+xd&^NjTbZYN_(Bbd*7XbGPzC$O_fvJ7*fAlx$EIQ2R75okz
zN(Y`1{2`rA2d4TE{~yyabw~c60H*^F34H+=`MEub{~v)TfHO5c2CpvoU@=0E*+A#o
zftAp=LSM3%#KRcG4}ES6bW}0se*4Up;9UD$mf*)NL!Y}6jPdbM0P-3I7Xv%`f6O`i
zT$RvYungLkFr|HCUbYN<Zc6B1!uh8&U}zN#`|9&hg1=(P87IMSSg!Ug@JoM<`7Uq<
zum_bvJd61g@LRxX!vE*%Fvcr=CQ9f(115W45&ZuEQ@qL1IMxIvdshlZeAZ{31ZM(=
zKo|SQK5U<968e*t@riEG`vsr2&npRr|K<0AJ|Vc-K9?l;%fMuBv0v=3qdx)K@g)}V
z<gp=O@!!}H(90b3KLvdSSnL}+2YSAPj`8~OM&Md0?|%T#0q1M%PO=PrMoI7vVA5w!
z@O>8EP7T~6I1lpEfF=I9kAY79A^N#bf==ya(|-k+>~G@=yxzJ1{^AjNjiBd%PVI~S
z=xzh{0#o}FzJ>Q(7X#b!F`x4F0%uG9Zvs>MofV9DtIsV7{vP7nFz7DGLws}pdwT0N
zaG}uWftP@b1jAqSStX6*@Kb+(_!Q_AFQHG|4&Wyv*!_=q4Yn6p{3k9OnCv$$`O%;B
zxg^0SE!Vp(pp!kIPu!;w?<hZh!Ig0(y{H<P;tP`EUIM1}lXw>QI`AxTt<XQT41Eqs
z@E<`w`HL<8KY%^}EbSME@kXCL68@h9kAU8%ar{opcpBxKZo&6jhCWv$_=IKXb3}q`
zEJL3k5`5V{Hzc?RnEXTR8~>(d=<`5A|9#*Y&`SjWN4$PJ6F5uoUjtM7OMAyJ!#`4h
z#h>E;1@u<nbj_cD^}-swWgW)gxQ2dc9exbBJ5pXk5%3tW=%4UB&H^baU&^}(>;{(j
zkWg!%6%u+A@G$6&f)W4pIUvEW0-pt4{5xS7nA$5x=-&q>``8%qPM`4+I>u9du1D|+
z@HF_ve>|`UmN;Sm@nl%W^HiSnZ_fjkq0j8-{GL3^^*bKW#lD_GtS=`V_-R>h+Vs=#
z-$u~$wf}lbK_`9FfYDz(4WJhQ%Y4T3CD5k?6TKVsB4Fz8=ntN6fWGLU|2F8Pubuzj
zT82J1B>Mgtu){tc#D9HGNa*OF`izjqiD@_kG!J>=FNt3JOpwsA9?|D}1V4oJHp!R%
zlnDRO=X`|zl;wJ50rVr#ALCOZ?D0xB@EO4vFH8G@r9Bc~1Re)IC;5NPa`BmgjlzEo
zXMHGs76`oubh0nS2a=C?r_TZj{X3ScZ5H%RLjQeWYHwTK6!19*{&UOqrFjScUjwf=
z=&+YQH>Bk!xqv4@7yXk`fT?{#Lca(3^MR9e`ALWu`n-_PKVi8p_kg}Z=$LQovqD0D
z4(pLT&}Dp2Dz(oC3BAEG^tm6wt-z$O#IvNYAb+MqKJ25<{0KkR7oVF2-PZpHz?A=r
z&}U&^1#IgN|Iz1sguV>>wSk_j^-ne}L!a@{^yFmFslQ16Wb`k6)<@_M!akH=+AsMy
z=)=IGPcr6TEz`i#|C67?IUZ{Ne92#KpX(9)f@Qokj`(czV|}a7<_J9mJO+A-;NP$e
zecnd!kY(ueHG+Q!`n8}uTR*H%^w}Dr&stG0WPvXIJ^3%ePw{xO*yF!ihCXK_{Qr#g
z7PV)hrl(;2tIyYHddeQlXam_4`=?+%rq9d>{R!j;-JbuYz#jUnjL?4-{zU#J`lev~
z(Pw6a-fW+h5sc2D&&ddW%|0I^80#5*CPpyguRaSS81~TTU<6NDhCTx$81YA+eG!a!
z+?50U?D}E8rO&!(`UX6gdN~_(vCoE7%h2atgpT!UTO;V=Pa7TueHpk`=swWNUnm|T
z9&9*epMMehXF;C;UF@>~@kXC-5jxhdZPb3^&l|dc3mx(>Uh8u%!v7}Po658Khk?g|
zZT)}GGVXYRON9T2z@&eH;054eU`y~nAwQrU4>qDdz1j~P)bx!h@V880@wbh;F`pR}
zjPYS3#w&f^Mfe{9zt_QE1pWaBKj!25{EP5o{r5WAU*h>jjL-TUjL@$jf4_r%9k|zl
zUjy!Q;BTRP>aVuGAHaVpK1%y-`~zSbKP3Kb#QH^_r;+k7|HPCQXJ`cfSNO-2BR~8{
zpO?|})HwT$jNq;I85zL`Fh43le$git>y@uj`^^dcGdTA$1e_`ObM`qH!SIJbiR6cW
zrFL6}J_954Z_wM^F`f1Z{w@2Ai{L-R>)K(jD2Z>W7;i2&LY~BnRK!<(UPbuf50{HT
zm-Sv6{NY<=$Um>?yBAR(TOZ69(vpCQpZuRN;_W*Hz<HWKEf2UAcvdjR=g$+rjE8BT
zw2UE=*DLgkpq~Sl_DReJo!U?0Z(1|P|7l?Hm$Wb1=T#(s&@%KH6~Vs_Y=M75@Q0S6
z&!Gs0ee~HA!9NBbhWHV|^BB)O$S?LvTY`Qoz_$LF-{~_antv1GhdyH>80&p~rbIC2
z>)-XlzYM`=En^heUaxLKywvAIgnq*^ezy$tVZpzS@v$2?PRE-~n7@CRf_(~^KW%y+
z?ML;M_S=N{ygv6K`F{lc8$lQSH~mM;(C0XWPVIjd^mHwMGunS*82YAZ`eux;pPv9-
z`qSp^=zlbykoMS|X`kN^{v2S~OG&)h`~>{@4ESY!u^IK(XEQW^D&n0!OCkBI?DG?X
zVV^&N{!wC&&1nB0c_D8_=(j9GpKTEQHDJUC6%dT^<tN0yB=~!n-%bKc|J?jT#KTHp
zvESyO*k=-iALEHWgCH2=>z6{{Pu2F?g8BVllt4f6w=IYV`fPya-?A64r%t1D27(_#
zyvm7`zr_!E6c40*wv?c~A<wq=7L2zYUSQF0OFiiM5&msxpLffIj`+Og3y{|jEcV`l
z{-MtxNco7LU&;bq`rDSbEaPvzXm5!pTWCH4eZzE&zxwQf<o`3v^~EL7X?=nCwB;{>
z=YVbfFdxz93xxk8(8=CXUU~w(6B)y!v|sub?B@cz(L*Snr0;^iEP$WZGmxK7`^PB}
z`RR|qU#UE?SNd_#y};66)1S3m!(+hY&#*^&g(dfkwLa;!*pC<lUHmuwCA^Qd8n{5{
zSAaoJuGTo}Hu$N%ZT?Q+ZeUptrGEwb1c7C|PyZ(N%TOOv`cwKa?(LKQ63^1VkNRea
zJgV=H?0f%G{$E+HZ=?ShUeW)*TP~hoN`1G+S*|xd5&Bk)&%fE~z<Kt004;AT`XAk|
zbWK7Y^w|0-;Cb+q{<xpH72`4A?<9=-om-nNIU6APzi62VapC>*twH;KztFLM{QM~B
zQva=ez~s+19>l9E$-aZa|3}!rc^X*ubGH5k><4?>>!+=Mjq!!zh4jy@KSO`XcHn=(
ze2MHQ^Pz3=mg~K<z~p~u?`@mVKgfS=I_&f1OknYkZ4V%SpM#!{{G^|qANKlu4=|Mv
z|J!yJ<>vy6zT3(muMF6hUuU^qruLHlu<bJHSM1=2zg!*%mj1Zy4&?U&OZ?dOb?8I&
zv-Nui`m_Su`k+6ykv(nt9|DvAC5V30=r7~Iy9JXyss6Ga-S%JLZ|T5d&uxhJt*gLW
zH2?N!VCruoe>?2gItjd0=zD>O9sCailm0gUam#h768_aA{OEtJ0nnwtZ!gDuZv;3K
zbhOX*i=ejvi~Y8@z@P2*C3>6X>LY(>75<yxuLUOmM0;$9y}q3VTq5)juwMedw(low
zhkx}AfKK%#c|Sq^bHFlwZU4`}CBV`i+y9&8TAUVs^v8@?*vr<3@Md7IgARZCko-aH
znQ;&}$AOPPejc#cC*ujr{2eba^=EW}4D`R(iC@|`qfE|Aj0?RE{2+(<zW`hS_rrgj
zzXRi!1v=?Z`h5lR%7Do}sBgvqF!_s(F+X^P=sBRnei`2frt)o!c;43!EcVKPeLhSR
zO!8NOQzCdr68xX|CI5~L$O8=HdqJOeVA$hp^T1R;=(od%^A^C|UWA{4y}Y1{y>?){
z`;gjK?7QOy%az>^bCJDZza3XB*UmPGvhhvJ{BLF0uCy`k4}G`5FbZru0((>&VSDU=
ze|$jxPju982lgM{A%80o{r(>Do%##e7y8{34f+bOS1{&3?>PML9_%l?L;b<#$NGoo
zzcxSCgWq<@zvmL@PWf2heA}5H<HxsK9p!x&@!gsKPeFI)UjTi@!T&#i9s2Fu0PK*z
zbHC*pbjaKJIOq;}JI{dbkhc@|A9Tw5GO$zLSAkn2<?npga(&Awe*$!;e8i`3Iptfx
z4*6d6zi&C@dv^dk<a-~oTyHz%d!GW`A>WJm{I)~B_a$Jb{2Rbd`B-1S?Uet!mg_C2
ze8ktco$|3?_m)#W`o~)i`MXkq9rAY_0CvdVb<}bVIOOm8B<K$LyD&c)aLC{F3b0fD
zYrszVzYXk^kN*9pQ~n(2PWk@;x<meMH*g->PyB!PPK>__SmMiW#QQgck@nq9V>kcl
z^EbPn!z-bdbPAo<*?kU};uHQDSFNMhYZ^MZrvI|}U;0<(|Kh(g|Mh=m{#V!K|6k$P
zF(0LPorl9@2tm8QiSts_UNo5_jPdW?RzPzX$E)4gZ+>soff3){Z3I>bc8F)Y{}S;n
z(}69^{7o;gv4cbMZp4rGdLwkarP=jvH(@P*58}yt{{~pf+q26)zo_-wbHp;=IRm=L
z+f!hfgC!39B=Qqo^x5+%%Y1Sgcv<q-gWl($f7vqM97gCb()#VWN^g{-GyB5-Rm;?8
z7zMwJ@>?9`eHWPY7klmbx6o(SLH{GjPX|_#|Hqcu(2t=ZOUnCmtS88xQvYPk=k+;U
z;m3Ua-7BD{iGBVK?E`4q<^2=Zvz1856#iBFjI_p?F{tl6=psKe-Ew^|4>%g_h5s|}
zvkW)cr#OX#8kvW|{{vu`E-w?~>CiZEg>K)>=WtdUtj3hcFGc&HzKQdK8<9U3^d#ZO
z`2D@dAlf74b;2GapiBE@-muSJOa5NiufRcn1MwjTSlTo59m`N<m}*#}@B5b7h4P{o
zH?n_a{+?xiyV#K*_Wv^GTQP2-{|NdOf$o<2A%1<w13a$vO+>uE1b*WPFy=Fv@aOj^
zKH25}1N8F(+xla@{@wy$s+PCc#C-l?qU-wZjYYgW2T1wx|K3fgKm5z~hrM33{|sTt
ze=p>Z1B?8<55d3O4xEqj27sl1>@Boh1w<x)Abl^OJxc)dwSD#$!ye>+wmggn@BJ~L
zoxcM0C;3vpy`KZUpRnljihbS|@-RQzdmH`;e~qHbpug|^8u-V-?-q>qc*hHDmp3No
zU%kjr{Qu5!{SByXwtfBs=1b$mFZP**Kfs=G+1elWV*Gw@3}n%FANG&jy})_8e)~4U
zp4GsGntxx0WxfsQQo4Wb+XMe4f0O#{I|O+vz$v<Z`yR8;ZA<>spihEsmxuBGy$>C@
z5%wg1k@{_IwFux(^!dDf-WT#vU%Fq|G7It<)DP{muO0e_fIY(hD&!ARcG2fI5HFX3
zMgG14_{U!Wen#7OAI9VN-U7Du8?{^ofLuP>XW#cN^QlEZi3j_B3jW)`rsV%S_!o+{
z<?naFK0|<FpZ#gDrx&<L+hhMdnm?*l+jBqWcYj!dc5H(@{J%dJ{*VLyEG>UO>^VsB
zO7!1<0_D#E+VY<Vy~jbv`bM9RhElNCe#~dz`(4nbe*0g5J*fR{c`d+h2mUqi-ywb}
z|0XcmSNwNB=8x|V6Bha#u+Idr*k}JJ^dbFi`G0`+t#;rapnpsO+w%Vm@(D|O?MHmj
z=eb4yzd`(+2E9P+NB&FowaZ@x?vBXIN`SqQ)o^S6tZkO5&)Ey!XPIpz@0^ZbS=e6~
zLjSYnWnumE?q!mv{U^&0`&0r;eX^bbo_F9+S=5aU*BKOQqS>k}v;HhD3w8Tu)uR74
zI{05geD?qcMV}jpXW-}ZVbUz>|0ROaepzqZ=kFmO{+<PY9VY+R`c+qzR~~FIsvGMn
zN~`Kynl2knRn1i`a$Q&6^kP+$ky+DRTGvot-c(UtTG4pP$ZW1_EUl@mttzc;XlT^1
zs<oy?;Q8ieeRZ&=p|Ya1sj9_zsj8{DrlI~|g;7z_P+9fRgBNPcTMjlHY^iHRT4`zX
z%jM@Eda$&jp}x7LyuL+J4(A#Tms)C#n)>tjyo^t+(RiulQVlrp-CR?Dv9_wZs+E9z
zRg^ZjG~tGEQ*BkfQCC%0QP)U_zJO3OLM;`ImyP;MwY6uO%4=#N<+;Z4iYlYB>O%RY
z+LqGBriO|t)B#l^Q`A;fHXARVuPh}1y)WW^HPx!T(zw)ozVsrcHdR&C*5EcZB~(^5
zS3KBQ(^wTIH&!*l5)3cY)K@Zx(NJAcQ(AqA_*<$fS{%5h8Fr`-r$yvdg|kFhI4#1`
zT+v)pisoYV%3I1yt6E!{$}3t*r3H)&4VR!nMMGU<c}vat+NyHMG+Juvs+vlhYpbdn
zjZ2kPwdI$MI{g(clHI}xLs7c9y8OY150{>=ZK!zB04RO2;b7ijx_bD~gX@wXHZImS
zoQHL*F5tj&V|i0^Rq3UN9%QpLHZ--=l{XsYm6fIEt6nmS%3rD~D)copv>3kQpY%iL
z7EU`6PBU5>>$pW~8!ncXqq>yvU=BA&^QFc{G-E~6`SSV~n}q-f{#uw7j(E4U(&n1a
zR~gmi&DEt)uDlM-($q{<IkdJSM;<iDXr-hx+qu?(Uv8>tsj4-a>(19SG?&)H`o`y~
z>MoU5T)0?TSyx_)#;9+;fId}n>hzh?XOBPge5wD$6Q%w$#YSaKeFK!PK|3_Ga6M|8
zK4(xuQ`JT2Tv}dRbFto;(0r-R3F~SuHkG$jl^)JLkG@E~xu&X_8CaQ`dZaeNq#_S}
zx{CAT3*42&Ntft>IFJG3u?}Qr+jl4pzEtxPN@0eV%WGOH;bkvDwMw#S^~<`3bvUAa
zrsSETPnA|S)gXImOGD|U<|<;56mAyy2z8%|^2YM>=!h+si2;e^y`^Z0(uSAoVJphj
zTvOMGfKyRdNrJ;Fp*vNzmNr}{^_M(fT2yijXZeni|2MZZG?ibhLg#v^23=cHVV}n8
znu=ySxv8qI;iW2c--f!<7vcP++=;o5RyQ`53tQ>=%kYSnYVuihJV}LRHEc%jF0U=E
zLDQ^BK<Glmg3V!2bTnHfqqe-crm>lVU0q!{nZ5Zzb|r+DOBAf&04>yvbr<U>TGf=E
zCnfpnLPLE^>81L{x|bNTbgq-mR#khUw51ARh;r4`e~#km1qM~E6{uZxWi7=<6hKC&
zMyG&J3FMj;2%4H1O2HAhRIyN1-h(;R70}u3&Ry9+R}baoN`k?`TQ}W>np)k~)H$nO
zLOIli=$+^f<Z0Zgw17k0y5}#|)FLj{Tx9K`9YrJ<5MF}Dv;%V_k=#h4b>$^EJ-SD;
zfuV`~tQNzILH`h-;i|UkaISp~ndcGc&tJM&ib11-dk0ym=3*&#-Sf^Aa>JL`Cuk1#
z5r^MWH|5@rKC7c?MR{$7LEimh)n$x14KKpaFyyRFg3&6`(wDS{AT&~wa-=(bslKEE
zO-a6AQC?r#T;1?8I&A$Vc7N&{jn$W%YfuVB)aI&sczHO|aP+rBPE|d*UuA^a{9;Wb
z+m*B<`F8&n$1jKPpjR0@mY^wXxz|zWC%GuBkvo6*0)<yn6yQ{JP#s8#pL#1hsk9u|
z#?f$Va7zOVU<**I^99+}**=&6)YP=*mQqCE{!(7ih)`TtdcGXtrKY0#ky6CoO9%i=
zbz~>qYca8?D@RLkK%@reo{9McVs>O=LLHDoV0fm&;W3htL^agcUWSU4u^N%09^-{k
z;(xM`<4v<c;poiSGY21f*eGwLz*SdXQPUt=!RB_5!_)vl03C#97R?8_6JY|<crb^I
z12)}!)J|)N2wBw2N~z6^(;A$n@zN&LHk4yTX@RrT^n!*oiZW|t2uTO9gWza6n6rj~
zoI|=SkBi)$xeKpnL*K5hscNDr;$_qU{$9JDg`>^|Ot>yJ>EJ+veO*;~GcX(*u86+d
zaFKn2NHo%5q^2fhAJU<Gja26Zrw*;Pw#<s^s)`qRjMQ<tuDllW6ok0)iWeI(OVk);
zzHDD&I$B+GvDyJ0#0$+;7j4QqE((`CI+sEM3eaOu4Z5~=LqwgIX)4>iCYo@-K?)oi
zvuT^x;j(q*%`d*Zrou!Nhk%oVjyZ~P=`Pd4aGIXCg`v#b!jVGHh$69pTwBJ4Fb}73
zNT;EK!zNwWais!HeJB@x_wvC*5AguP^E`wf2Ip&Eta}M9NEWAT8q+))ldCX>acl-2
z<RbhG*OPU28E;9W>hdP;jZO9C2obuyB^?H?D6Khf2MqW$oRw`~di+AobB$FMPgWkM
zG3vRNrc0QJH&<XbU$5(V{-wqybfOE08<ld!gGWvMg$5Z$Fvp}h$|sMXe(uRrCk@~8
zPZk$GdGf?5WO=!~slK$nLAxEK(x`qObJ+@fT_Twma9vt@@X(Pwp84`z$(ekFC&$Q0
zj$6uWiby7m$PRGG0S-Or067mk!C{u@kVh*PTE1xM+|kgh)@ky{L)t^^*n;jE5&H0h
zi1COam~zrAA*>T-29!_=W{E9fye3GIXRUGah=hmMFk%$NT8+X4#WgXnJ+iQe(|ok0
z21}8!h(ixP7+G2Az7V0oYk5I)muxvljB-k*j(K<G9?3H>N@A&0su}Asu|D5?*`S!j
zIeAE=v5|*CN^Zmm5MBV-2{d3bD<x1@W7ovW6B#K93e>mI)L7FIP$IQ^<cQJO@N!kt
zrN*!l^{m8mjWzX`8joXW2-o(JT)j-Or=4NKVK^T}kjk3o7b6Pj)dr>wccD4fJL^n(
z_+f+VjulH@r09o^@W_r~?K~nshW79xf&kAtIED5TFrndf00?9Zo@5Z0&P<aEDu9xS
zd|759bVXb`jjo8xu{4pCNIv2TPh7&bc<8Y97RpVIboZt|ggD35Y;oB4(4j|kB<7Y|
z*H;haX}Nc$t?9Zs+9mJCn3EeZ>7ks5xF<*Ws3SA7=T|ZeV{E!u-i-O;g_<V4ehSz2
zksRYZtp{r>s&z9MKCD`wsA|C4x9PH8bvsiGZYspV27{L74Na9*O~PeE17;<e#Dr;7
zQw;(tUsspcwj9ip0(olZr{VcTZ9`+@<>zWH8ijg3UsPB^@&)-dqS8*2PG_(zh#29>
zBaZ7lwg+#hM0Q8WD3r9`P4%vWWA{az5OYbM5=I6@88B%wZ4Y=f2s1yVHe6~#G$-**
zSV%e+1eKTS>ac7C>)Pc!5BAOFrRSS!DzQF|OjEINXyz(7wx2H1G(WN%Rfqkzn!DwV
ztnX>++f+qcYBIJwoP+3$S!!v`Mc%=}5;IK8&C~U?*Bz(J>z>0N$SJG`a-T5J6KlCG
z^?YDWS8&WjF!ecK-V_mf_z|A6$t<O&VGZ#JR*^is@&L-tM`6d{jY67TQlT>Og=zHq
zAK@Nzw%WCmyXJbh6^+b#=|~ODYxMA}g?%hs)jY?Ln0H7rQ7tIbnyQ7pmz$W^K(dLj
zmO|n9t^}yEu3MF^mlEl5|8FB_6B|cT#qo-FrCoAZMM6kOti&EHPN*|Ijy?7*@z_ZQ
zC2_EwMVw~Go*8?z9?vizw!<kLxPi0>4rEU&!2zL_h_tYGZg4^zkhmc25s3@ph?W(<
z|EsF*nvd+JrJ3$}{rYv)t5>hy*Nmz7K<3ABNJX!6B)p6@d08oL5*@VxNUIhxiQ;Z8
zx4R*96pRmc5*+>l=hl{W)S%j?%R9B2t?&nmY1%BwAV0Mb5#nXhD8h9n7>Y3k(B$c`
z<2|7e6Y)JxW0ENBHdchmRk-YT+C90PSX9NUj@fsNzZM!bt0btiv(9K{x<Ko-v#rF)
z+1dy}&XW#?86Gm8sdZtb0MGOsy^lj(jy11HC<&XI(GymrR-VQTzAZ`|Z$hMsdr8t{
zbl~Nlv?j@BoL3SV%c}%RhMi8^0uPfji!Bz6&#Os)^Gfcx?}a43vq0UdI`+Dk-OLN^
zt}&szM)>sQ&^8kVxw+~3L`7|+3vb!1Ig8<j(f2O!#^f5C3~44NNAjJ6PTAN<uGf<5
zntM2$H1_HZq8R&~A59g_>9?3ZMKumi@tQBUckW`JpZ9#|B~gEJ{w|TIL<-lv4EyC_
zT`-r9$SXF__!1*}GLeTEnsOjeWmx|*v)R{7A>bF<MN@8ix7*!Zt1#is=G`Zc>W5hD
zvcK0|fkm*E)CBWp(l9P>_{ZCobv+(qcuETp`18ykDAc2%f#WQ{S?dGWAhyOlb?OX8
zn2yM>by6O&YY+U2C|F!eSX^5V8Znwp3&8+X6>*@8)<`V$!Ha^@dd(d?)GmIf-6re|
zgE69a@7EjUyycZmZ*|<C9_=OjLtafLH?OAVM;;6%|3n9Q8H@2PjDoDT=IT4%E+~4D
zGobpx4-V`1@pdqs)gdzg<ewK<>xt1=OJ7$Bw$*$G^IG|Eu4!bQ$`>5ZPVm|-&+F*c
zBy+!}A*O9D;<PR6EN#!pPIapo*A`lH2Di$RQ&hr#%N)w8H8(jsDir5&3yC(}mzhdP
zUIe>sSCkQ3$nLR(@BCtbh${FKA`OI)FJnA(oY|zMgyu$6&B=ZBn*^|W=a?+RsgfMX
zjY*?L7~_IECj&MlgQuX>0A(O|UGDhYC{dr>2z!8!po3%=x*-FGyhGoxNFxT#LONjN
z9Umc?ich?JYnMR)YkH?&DGGIn!4phinJf6xwESEIOikesl*gE9XV~Q-AYdazI@;JA
z=pP>?FraNpM#!24S$k2@(o1|}z#{4n5M@oRnGh_9nYqW}m)&K#x8BeP*0u3jIAik)
zbYohCY+QxdNr97Bt`u`;OeJXJiqLk!E@O}MjDalZBBi6i%e>@BZjc;lIdwe9Qv%f(
zW})*Wx|_>LB+!IJfG(0S-8<Kdpph4#ChLOE5H{#M;r7zuCD4RKfG(0S#Xi@Ipzaru
zH<qxoT`8b5?=$EkiP};U1$2glK^I9>my#%;Gb9YUNaEH~5(RXIgh3ZcNHFF_t6eFe
zGb9YUgoKQ+1>SE)=qw3A=SZ~Z!eTE{Xh|YM=SXO9izHHLNg_h$NZ1Nl7)S~&Nkr%z
ziEfwCEs#i|C5Z@KAaT@PMxunKBs_G1gjl~YBi$03lJHRCM;8rBNk(hAxx8*Xs`!ik
z2>X?sHvi+$yLbW$dqv;Qv`v>&3gXCuo$2|r-qFeNah0-=Q~uY$TT8R`Cd&VsZtYUF
zL1Jt=!LGyz>x@v>I8>N{4_j9xW477a9uWOFei}Smx4Nn>H1mpkx7WJ6ZC3d<d&q%|
zRi)4Z5n|d+tTkd6ev>f%>^?f&|6%j+!FD9G<u4k|HQ9Ja19lc4MqPZW@u+d=-o1C{
z{$TL_eX_o|F^70dp1`1i-5P67nx^8(#h~{r-Avr3Pb4|u(s>LcayqDe8%?9RLD7OJ
zHjl@BVB(`wpk(wwBTx<plMv6v3`6{X_DdDOSW|;{vo>$@r0XtDh)}44!K@D}uP_#v
zs~DN2e3IjxOrfet!oD8*GF0+1j6gPoe^SnVXF?d9JFUKdq8i2#M=|Vc6q2SUyW|Ap
z!B>~4=IBDU#}q};?yY$w_#vxn3cg?(@{@yYrh|X!c}PXcj&i+wWEZBB$LepOd<zqG
zw?BoGR4Yl!Mp*2m&pWNF*pG3)hhsY=&DS5F#Fwqdi<+zR)Ku>DXkhW%*ymd0&v-~E
z6T>qG{7Lc2aez^poTm9Q&deC?@ZM~<h`BeNyf_ol#Ik2PJYOls@~rW$JC!e_Q39Kx
zpwPDlq)E3WLTGprAC+L0YsAZUIjghNhtXs&dg4NYHK#xT%P+*`b{-$@#@8lmqp=s?
z;KM($MQ+Hy&eLpSsjtdjX`QW_f$910J<idUB}XDR*PRa$HWVMwy{A+-lw6a)7>1wp
zqrUg(ZsVS7DDbt@V;)*h8*M)zCEBRxkYQ()W{)wYyMs}md^=X-$=-RKehj$KhbkE)
zWEDZQd6vYF#%=1}yfRrMj-e7CA3k95gvZw7=;e?^AgP(~*v3rvOd0GS)@2nVkII`I
zTh-Iz;0~yA39l%+SPNY8URQl*AA!=Ufm0ppbbDY660IjysKA6f==Zex1Ap9=uW2IY
zhu~0i!*HX>ku;y4X>~pBjm(Rb2EH7O-iKkaE2z{kFovjyTd>J5`qfqG`vQ@$)uK|}
zJvdBP;n5Y=0K5n}uTEHEUKDor_MOYdPx}ZAXX^Rvd>XGzHv;duEIgMzzO&yrsAtpj
zOQp-R&M7%BW$ZI!P9}c<<(=U{=Nnmeb38h}iLsv@XKT2jUwS+LvfEYScbV2>Ms`SV
zv%(YkUFkJTe-zsm1`BwtqoWbEaIr<@809gsY0oH^1CL#Haz4#05SnTG>k$|!F_8l)
zl;X(|Rg-DkbApP{S3kZ5Dz&Qajeq-Rp1JUwi$LO8I8y=}@UFe!>ILu*!Yw<!6B_zM
zdC{Zx`8MCTpYgZB6;~csUraOOBfM>^X<hj#M8D3*4#8g!p9JrhpShLi?*YHTN5{+X
zD8qd|<o;LSJ*R)qpS=R!4e&mA1-y5_`{*Tj{7xhC<#`3X7qrj##6#oNk&KuBzFzt8
zZD3zDj*r3n?o06Q0T-WMvs&<#4{!7H;BV57N2L7_K)|EsdgT+<A;-BAt_quVsLv6P
z`S|2L+RO0tTdRooPw=SwUilNSctm=?1)%5YAHP*t`LyEP@23%o^X+i`JI-%}Ye)&t
zSuJ*eMgIMPw%+THl}~@+uHE}pY;unG{ur)Bhqr%G<NJGd?Z1Sl&tJpYLtp#iC+_td
zW8bjvNbesrcyGSN?+m{2@e;g$&El<o#9$J%f|vgdc)ovceh0k2{Ut#R{kx|BUDtKM
z`+UIlttEJG0W10U=I6iWu@o=x@7gMOaZJq2|Nic--4##c5$@TuCEoR&pF5XJm&7CD
z?EpydroZ9m0pVSS8n1;@GUB@a=l$kqz`hOmlC$t`@c9}aYTv_g{m(z64qm@$h(~-E
S_46s-e|G7{_ahQ#!}~w^<hz6b

delta 377526
zcmdqK{eP3y`3HWVyJ_>#G)bS6G)?X_JhULCg_N||HsLv?q6LArX$wI>gOv>gHo&q5
zc`6E{cw~2RWt1sf{Y+dD+3G&B6$LGdu81f&(7MUI8M>8ybb~4KeP4IN+~@WEeE)&(
z4<+aMT<1F1xz6=;Zu+i`-tpO}cDq4{3WDGj3_YLh9+sE%=#Y@$TeJJF35nu=DT1(j
z+j72|`CFqpK@13kx|bx$#Jp~8lpVJQBuPM85ZK4ofWT6gDK6KS`kCTPHfRla0w(4Q
znZv;ZK~lnf0{TyuN(#~581+^PQO_#WKVp)k)fjJak|6Cx-DcBYqMP>r(-=<cmd4XO
zCdoK4t=m|}=ZjJyv)&%aonFXFbi!bYka=p0<o`Sl+_Pu$?Jbg*CSjH%&xk4F?O8nM
z3YqUiV^o=`nd|#dh)O{x^G;4Ix*F{h`H&_{WPM!GIRF3o)2|!e9LKgYQ)r|0`#$t<
ztbWCRK?(E;)qn6GkL{{CtVn&LG>R~9ERqT;>31NOH(BGqVQD0liyiTNckI{Pd%Ha!
z(B+Q2y}JRzvv3~c3jf!<N50n25k;eh*BI@htCv4yED%=|a@qKl*j{*Kd`dvfePeQq
zP&9cpn{c)>XTv*ta&B(gBjt|{OG2>3tmEzG9QMYM=gdFRWEnr973p%tkS9+Nd}Sv0
z)z55ec{pQ)$@tLis;i&5YK}aaktCXQQ;BM*`PI*I@_BdG0dbs@S7-Z0t>ws$Y`vKC
zFkp8HLi1U_yZK#zLG^fHMYUI0>s>5bj`Y~H8e7oE1u-(-9NZO^@<!ip5xnWbh+w(7
zh7aWC*aB3GBe4f90{E*@Te!`2T;rUPSQi<lt>ZOLKg;Ls&TLl1yPcz%!ryg%DL(Av
zZ@T=V(0;@y-D0f!h%wKckk}r~Pwe2*aGuijPN)64u_evuosihY`-Z>HdU)H22KK^{
zPez;-S?Q79f^5btd|=cRww7m%zQlLlQ6f4n^AmUYa--jURfu}68bQazDat}ks@I0&
z@9^L9E;2^Bb<7hcQ<W}B3<2_5Wud8<_l|LjDcAV<F*Dd3Tq>+)jr^g)>Vm%HkfG6M
z_1}8GQ@1!$s_i2bC)a7LzP&;b?N74RQ0JYf9pHn7d4`W}HJ|b?0@;MQaqK3R!Q00c
zj&L5*bsFqmhwz=YmS~YKQLM*=*L)vTq<XEuGvHDdIG6HI#^$A&wo@@R(P;tO)NBp6
z7L7~G>bzY{zskatk9lL!qhjW2K3G&Z(UFqesE+@kI=+U+x7d6iOvLz1h>f3flJd>3
z-^rT$@6_wmHs&)tSUh&jRH~s8`c)R1XL}DaL2)^U>z>v)nz7o^8$P<_`w%0!#V-C#
z@eE+*vtlRK8Y@hYc*eNtVp<Dt9Cs(v@Q!g~Ssm{m=fS_9;ol5y8DA(`UgUxCABv_g
zxa1urChp{Sd4J9B;F^+g6Vi{x;r~wC*U^A3K@7#<%PjVNP`>+H9HlR(Jk=`;3>)}E
zC3Z36IBzQ{EKEM3)~IVJ->odnnB+YuME~zKBy{mlO70d7r@426hn4cW3BOCt-`wXP
z4nnCS+6847@qFJ%P;8BF8))^s?`O`GF}hThST=Y!3s&0qz<A9I@2yI<ddoF0@Xpd{
z$xjqA!x8abUnlF}pOxCgCknZ?EYFPMCss>>H$xCn73HO6^F^_Qx0i)Pc9MTq=3^`P
zu!$j2*vF$2=V!UDc=rnZAQp}HDPbUalbE0^V9WVu6X%F78xP!>CAwZZQg`P<1}g0>
zpOEJm@O28JdcLokd+Druq^Ov@&tUhJvGCO^w(Hzdac{~SpeJmwKUG<fFpICKm?&EN
z_;VEt83?s9CnXFHDbnHt<>pfEt@OC(Jlf(v7e|N0JFx(Vddz=`19K@*uARfzR=Nt#
zZw*UDqZi?1Sp^MFMt$`O|Mu!v{ks*HcJDnW{Tq{?GH8Osg>K$gnI~Eg@J}i|1?Ry>
z!qiD;6Q=;{Q-e<l-QLCiTgn2f+ZPgaJHq~tsEil+^TecjKG$E11OA-<`B7PGK^k;s
z_e^aOBsIMtw~9#IchXPCNy6CydE~nJ=D>c|#)nOsQWnNhHKI>0NT*RHXe<<S*9FT>
z9mFOc-`f@NCI~75bat^mIMLLuOidr+U6V$$7x?K(WjTMx#rh8xt-i&ggp`Qi)Mx6q
z?Du+2eO#KnH2Gn=jZIw>R$=qoC*PafUd@FbwdHtI2hszSuI?22BjZhNPfU3@&vf|x
zg|1u5!sN?D&_LOC?E8Sufi%#lI87`0lT*gyzV4rLudScz=@Pr=gnuIuAFX0)X<{Gu
zPF-Lyu2hFwVEUMMPJKH|{ABbZi{O#Wx4fm|ACo5<Y|2z)5syx*qRZyZX}v5X=+z2E
z-p&3SJN8O#;DYs@cHBlYzSG)X$bhIO5b6+$C_T;BDGKrQ7^JQ7bSBcJv9$9h(sl9l
zWuznV^jk<r<LRSFx5v}_knWDBw;<ggPcNV1>rBwaGxJcP#M2chzujm!(%td$6r|f?
z>D(`ojv&on4^A0nOvOG<^=1n}-+9o5O|&QrT%UXGg08=%D+oHd!*{_yMwzPV=iaJy
z2@8cQyr*hJntmC&O(O*RHVb0VqwVEN)d*2vz%4V6CM*yx@cx-@{O;zT&2-@a8mn<C
z5Akqyp6FW2H&=U5)?MubR~f9{#pd!AciF|v6u#rG---#l`CYT_x96M&r#i4BDxC+>
z<^zy?^*)%A(|P3dEGH8!v-oFs&&Vw$vfTElRD>l6+crxff{6$tEaBwU+><3{Zs3u7
z{A>>IzNb=jHgV;ipNfveBRgtDCgvRD?Xzdtq7zp6ZzYd6q$f)nTk?1@-OyriM9NK3
zJ~aCS(UQnd&v{BTy>%pV?<|(QKqv-!j7n3=N!~YizL=89t@E&~gS>p6hjsJtywU7D
z-!acA+MYe~+`M##6WKR^x~NsSu69D2R@oI@h`q!icDdF>%28ZXn_Up78m}FUEY&I#
zRte%Vq>ZO7jb}SuLqzB)21|x6L#)607jrLfuXSbWMu!E#xH>IpT$MH#jA4d%RMH?n
zQJa^QH2D4&^Y0?%NrR)qMRcWCdrP&~`Pa1*TD8{=Ck$EWH5a|=f?uQQqpsgY@cZXJ
zNzl2($(WPovYidNhOYj_WFaQ$J4TS^j#*X#nEFw#yIr3$K=r{EtwL=!y&SLx*J%}H
zp<@j2lR;^M(pL_V2yMKr<CBH}&)i_AAO(LUv?p&iILvJp`-7cHLsyl=jFMVyz_cN)
zrm9RkXnxFMzp7|HBY<fg-cnr=>gLjbrU%l3x^%G~qJ-r(;EF5f%j#DuFVJcC2mumd
zBy+$*9O8oF$|xW}c~^ZlZmQn;JTZA5zgRy7B9;_-RMem3&xA&amOXqRG?pn(G=$5+
z!BxgVv~||wx?cU1HmWQTmeVlXqLNi*M8tet*<<c`Rh68<h{R)*KC-&uJ|?=(@bis@
zqH87BHMzx}LO!}_Y?|{F@m!r=tkoIBI;F{ZjX%`1pjDW5w#=@m?6@iS3_xFPj9N&n
zsinIjR_?lSwk+qz@iyDhrjgosU&Do_73)xUQ)zP5tM#s&gWeM;Plj~Ulv_+?Xg`Fs
zjneUPEvn?u<f_u~ss$UwoNK&)K?&F%zqP<4=9KYa3kya21wMD-BTUB!7TQLRBZUcy
zLKzEsYQXRoy5{*pnj#O%DNjevS0;3EOE_1oDdc0rjxj5~dqRjd6T?mrgvU2Y0lLDJ
zg$YN!c8l<4XIIcO4idFpSP$8JEnge<xL42ynk9>4*H5BFkaeDlwO~WEk!Tg>FNePo
zUEBG~iz-Dy$G=`=A0^Df4OM+E>~P_z%v2T%W8SaUZB~~G6kJu(=&raM@ZR8o`)v1M
z%H6*NMVSS0XO~o}GlQqAV2BZ4SxS+zFlQQp3z9!Zw;;Bl6uT8tnrypErU+J`t&SuF
zA_1UOf28}q_gJfqk%ZjeBIVz%NBd&6jmx%^#@Q)I0W7r?<GYEBaI=>Ju4%Ek6Lg^9
zrI<ggEVO@2^<WOarn#|o0<i$4DdU>=>2K#(nyf2nd=fT{{N|E<V)iNCxwK-+73*DW
znU&_M^L6`g>4OHHve3Hs42CJenlaXtbHCSZM%hZ83FKhJA8>G0Nw%VH7Ro1a$Nl5d
z(rh%mx>PI~(;Yt&zW*E6nmXzKz0{1o-u3>E6<6w8wA3A4QXNgNEKFY%Uuq3a8((UT
zugnNEm{cVleU?%mwAII#y&uc2qh*7`x@qa(qBQ-Kj$ev2iI(lWd|9QKYUC};a)Q_1
zGwNPLhbT<~f&Prnf=<{otPSf*p#7f(x$hp-T_gDq`DwemB%!?pbO8KeFC+lRKU$Vk
znH<3My*eEo9>cRL3M|<1Sbn=M1-;Gx&M#A`EKU(hQ`(Iqr4No`_a&FbQB>r$G2vVA
zEh~YhMT#r=Xe125Z()iaXzHce3Gz64UQt}Oo4&H-6$D2N4cghL{|_{nKjtTzf9p(M
ziS8?KP1m65idgr=4rQS>(O0Iw&buCRiiu^s@1X^GhGjadIF(w`f;{7(&)XBg94jwW
zzvkEDT=nq4a(kMF(R|+Zf{xT7rAg!F4=tZ}C&ZiTPR-z1ZoM8}#VlvJ?RvO{Wg(q*
zmfNof%eB4XrOX*#!bYGh1A1tbhK`U>&^N=%qvJJ3x|N{}SE6wrANHL|>9LbPU4`Kk
z!N9KbJ6dgg_jj_`0Pp^etK?9!$6(VfF&wW#A7g%6l*D)A$h0=&@U(X0h_sH#C_$qB
z%TQn4?@z79IJ~IUkx!EmBMFvh`@Z`&6Pi!^Q=8xP+vqk_>)!CYVX_)H68P@xOn;7W
zQq=;uEGs9AITd{FN*}w)cdYc~UV~;K0&PXKhcN^TI601u02gZP$}Pm{9(Z`PJ?#(;
z;<b;oC{6m^L|^!usx+o;;Lkm5i!b&Y;*liw<D9qg^AG<()VX=xs(DncEX<vTa}mWv
z^-5#z7T&jNvY2#=YgQL0tEgf#K@%N3uzI#RmS3WZoK)7edZH=O=3Ok9Vw0wf;mYdV
zSTkWh7|KpxnIXzsS|sjWGaWcp4_c%1OTKx{3wFcK<jvyLfz3JB(R(`PSI@k%Kx^YQ
z-}ALTYy6JIu9E+Pq}|39NQ3z)3zF6uA4VDiG17~StC1$i7bH~~zlXG5O&1%(NYl|-
zkYqF7hjgqjGcG||?VEVTct6r=-^4S<h{f)GNiZo35??YtWU(s(Px}5#DJRiSJ<v*{
zb}2vo{RN_S1HWVKPeswj2i8tuyLraCkeGbw$clAIEKN^LC`1|@jzy8us6WKp))$KU
zS-f}sgU;MFs0E7CFX8YQ)O1tsETl7(#*8sM@JQY$Yl>IL!obE1zz)XiIH$t8F#RAM
z_g&F&BsPTWW4_~&+j#y2#~C|li16M=@=T7zd;0x!CxDCOe9o0e=8MjCJka8?^%z<-
z4oHCeV&s(64#nme`dP6*&ri^=Gey%n-uPOw0(R2c&|kli)&ucYq+=d0Q}3Vm8*DZ}
zoyo%+>~`G+YS#a9-VmUy1>|+Hrn(FK$qki-x&m(!G#Ow73#+B7_XcreqKnd`b4$<Q
zdPut6MSJFnAK35L?p5ct3WN%^hti}y5HE|v2F^sV5v<>~QDH+_AO?U|ueM2N7DJ2Z
zfcVsgx;Su2XL$4n_UTEdsEN{;vJ@L368!(sM73kkP->f|q)Djz-wV0ra49I~gFh%X
ztGtbkQx+zz@s*jcbE)+~(RJX+j@CyQTX96UaTIGcEC=^f0Pr->RSb|u*EAYkcTr4-
zo*vda#O(j5tDriF8%K0UH0eJKFWf_QF)k|ZfM9U|GqsWa0mw)I0C_o8u7Z3FQH{<m
zGzK=drkWpqba;Yu-aWkOzjOG&2o1mf=!mRTvWExAPDvM(YO7RoJGpn$@YsrkpMpbt
zjB<fTHvKa3g#c_RM;x0IL}=a8)>{0#cIyKuhN*>rvh}<1RNc1kqNab_F66zaX>1#n
zl~nY8r@rHV&y;qWceX8$%_Ihug=t`LDTCa)y)e%)iMSdqKAEeFzZ@f6W+83{mKEla
z?G?nMl!d9Y(M{q#+f@nh_uFN={z~#TafWdRj<sXtc^;tqmffK=+Rhw_KDLF;R9RU<
zxw6nmOi^d-uuQ@3H@+WrkZQ0^X>h4{!<D2#dO7xNdh=`kz0DW>yRiqgXj|vq>%Rg4
zl;5-CQR2pDchriSMf{GPYhnyA;a_lC1Oca&_w2OAcxl3UHUI9;--xM)dHWBsa*{(6
zqM(|+sM2xcu7fn#kzK8#a~}7$Z;!RmbYn1xiOCW_-ChtIOtYEtPwaj}OfKW6caO=&
zSiy*YU2vIyx7V&QKp@?X76P|CVWTleKT%0P%}@MOG%w{#_(W4o67mEg@ic1J@#nZZ
zHX9qEEO0;+&`0@2UQC_LPZrXT|4FO9n=Ufw-90?~<ODHI&pVzhxD#?X#O=Ml5UDGf
z&ypNF`hGMUE_A8QM`*jFPX|c!4`Cts-UR<G<EzqMr7?99|N6;E>ezc`jL14ff(6X$
zw{Ti@)H5P!yOf5k4Jcnvbx1Ge&+QqR(hh5#ptcyV?cjrZ{KfcJ5*|FMZTI}#LeiB$
z+5vF~U$3SQyarl>7B(J9Ess<P9lYj8*{q94el&7+SMWDNm(r9y>-~^!vN~$Hwi|t?
z?uxp*aC^^)(Vpm37Ni}=9b6ZAM(9$tkp-C;R~P7iS?Z^;A!@)554+glk4BbslZ8xa
z%4))>o8zPQVALrXb>Q1kS)sBZ^$bQWifls9a(S8ykb5;yVM97tMJxqU4_2ZPgWiL>
zqTm<WZr&w3Mf2ypPxk1#Nns#5yef|%h~oXi$^x@q8c_RHd38d-`<gBq{1dj#+cD0b
zTn`(m<XtCNl*Z&q04apD73l&V?HCCoes{;Sv?=%Q-7nfd=U?nylNIA{u!EaVg6ohG
zubyUYeaa<fF6UiO9isO4{CLVl(_r)Kep~ZNe?jvJfANm6n~VX|V>~qdEn3RxMc_6G
z;#5L`_n>}7@LxbvgKH5#{o|ZT-7&Okm-#~aD4G1Q@VdyWra|<;rIg-87i{%Kf9mTO
z`|{##Q_;3tHRX9s=XrkT<T1I4L~Gvk2@gHa-BzMUNSGR>5mze8d9=t1IvfY{O+GC`
z?#wTLiz=W<bpsPgkc89$c0KARWvmWm)cF#%0k-NvUX`25qxq<7qyfI_9FYdx!<>ED
z7cvZz3`oV}_l?uOfGs(gpzw$GRXYwsRuXqa{JV@x1?!Zv5ELBxE;COflH_OiEld;V
z#l;6Jj9+6R2<!ROpNtR_mh*>xVo$Rdf&bW=FZtb=H>5NeQ+UTu))3y!`zO-;xI(%~
zFx|Suznq3_*zXfH=lPTSAq9NQKiglB&?GG4)~*qv>o^Z|4X3UTbos>Gb-b&qfO2nk
zIqBzWmq*N$xaEL5JGQK)zL0S(q^%I``_(^Z^zh&TH??d&0Ihv5Z$D5xQf&##WGKZ#
z2q4!3QG)Mv2*I!U-O@Y2#B0EWa^RlKgX;FIgE<wPl+sG0WdpzKrz)p<;HSfi`#>C1
z-gnUj%RW|hEHqNJgGTd7<lEE%Y;4eDGXDW>_VSm1>PzjTQwDKer|JRaiT!-XG%I&J
zohv?J;&(pn$uva(QWb2g86ePlG9?n8Ry{pZOkKk}pSEQcVHX=yOWatDdj<Y_@HYUw
zJl))yvK*31=<)4RkkC(-6xGzxXk0_((9r6j*WwB`&QtTWMUCcjgfP%W6P<g&4QbDK
zNgZcbULUR!5_;=O_L?H#6S)vy$z%fAF{O#xY}%67LF7hA@&Hh#G-aLxeo4|IO%)B3
zH&8>Q$6??v`T1wu#Ej8ymM>&lks??UsE%k@X*7HuZ|g<drE1#BzQF`FXoE7tA?`gm
zp6%h0gX>H5v#0?<h{kWlR*`cFjb~6g226SdCMg92+Cq~MCbNH5IliWn0m*s@ZL{Gj
z$dAh>0=iE1pbq69qntGJdXhPm#-uI0?Png_DUy2ozwRQX_qLQ1hU8)8eUI4rhd&#Z
zo4AzVfo@AG0~Tcc9D;r**uiudn7e-No0qwgpziPNl5j*daZzb2@PCj70_3$gLVM9B
zrsyw7KM*56&8!$bK?7Gu&OS#Y)15{m|KjHXxPU(U`3QT`84{Pg>Fk|ko*#{b%`32S
zT%U<s_#NGOd5IOcq=?m!9fFE;F&u4sBYvkTjhdT$MR!h3@B8~5u1I-euh%i`JJA*?
z=y?>(zB7Gew8h;^dj(*CV!~C*C0~g3tCyyFrBx!9frU|iNCkw-iYoaZx;?GNs>K&8
zpM_Fi6z@647ZOY9Km;)pW?V=!!q0k;dg6i|f|RJEjRVgqgPrAy)KjTm9}w81I3j7@
zgJK;xZCSLlD+-i>v<*jnA5@YARHqU|g8@F$$^+n<qr85Y<DnTIqz)<oF6SD6+A-sU
z%DuY+l2U!5a(4`{<eLBtH@T@ENQ&U^L8K?~m!BQEjIIhomq6&Fol+K<juQa|*O|d2
z2_ea#xrBTmq@8vcr{;z<P)WBV9fJI;pPFCv>V<qT$uhbSyu(aG>hEh}zgG*_vJPVp
zHw*l(|C%G}DtOm_{ljF~j@iQayH9B_tmM!9VobI=lXf3wssobd5=2Tv(iJ(d7rxT3
ze~~Aq4(HNyF3V-0Ap}&0-wit)fREqv+)NS`2vis)tx8iO7Msup#LOd+1Ow@mCev~L
z@^j-v^A<kz+!zw(;>`m*|IlbCm2(f>$pqehXo8qf!21u)5;1W0^HRYa%yb@dtN=1e
z!?t4zO%dS6keXA{3T)nb&2^;#o|5qMIVs1fhDuSgm$yCd6vbt{`}qP9^m6w3@#J05
z{nBj*q`_4}kuQ{(4+BFJVhr|z^gC%WQ+_!mGqVEJF{!`hKoK@}5&(b^wvzw&m-#i9
z>8gT_qIb$ge<ms{v<fVAi!YQErjw&`NsX#fiAqDpmstEC`&)k3g!0B6utL%bVyj4^
zzC67rB(_)b@AkB&+E3xUOx3*#n*jtkkEN2|>Unq+#>dzS_WX0w8$lPOnF>(cH8eNm
zJ6J)oj$+OH$A^cdULnG`&3%Ze^MS*eccLL0rJ!-0cdcNle&25cVD|U^saqT$Bns~N
z7DNUO1}7^xIc3LyYZY+mg)s(<37zk_G*roJU+{@ZRlNNLo4WKy(@_w19dOk_c{*L5
zVK~?1(rrpS!%x4E!=B~J3*!oNwqS35g}+`UltaqJ6r_KH^gLQN(JL)G;xi3-%S}T(
zc%+~e0|viu8Vas8UB^+aBFQi|sSyOGs{{VkPzSHWC@d*Y>{9`-83o9+*H9mQps$jw
z5}FtRCS1X>O`UPEG8O&0s$Z$NK&YZi!TW)(2P|YQmBq)$dD9IH>wmech6*7C!HNzi
zUFS~+JsW>_E3T9|A-;C;j2FkVD?Iq(ID6*G86WuFGcNkw5H!ITa;E883<<Cq%;rzN
zcn^b@`NfH1dKs6Fj%2+&c+}T)$k>?{1vyE$<jE)sy-4^*aXG%k$ZJ(@6G|r&iYf$n
zFX(!dQ2HJ+8Ag^cE6$cK(!x}g!P8y|v!U)f@)j}?s^-9pM~jK>&A&pJLnZhB%1J*9
ze>Ey;2sDV~u3zO*$&p`8q-KBnRTky6zn)_+0$vv8CPEpigE+ev`w{{WDoL*T^?lU%
z{a=rqW_&hzizbmYL}Em=c^H(bP%pHM@mcl#$^u&=MTWfD*;iNnO8EsyZ<FE&<aT>L
zrFSw-yRCOYY+Bpr>a-{0)8Z%*sJD8z#;UD{{;%r&FRgM~w!kwn4af`=GGfR~co@9%
zW88vWyy!PSn`3g*MG2B1v&;;Fn4o>{mSaDdvF+b=C{xqf!Z4*F<%;1`u~rFLNlnW@
zpQ);~2W?Qdk>~$+Xrl2HHILho!YTq9jAy7!wHbx7cY2edix`*%-K!u!#V-<LJ%9PX
zhm(?_G#M{J!HCe%!F34FtEuGw{BKWI@+q1;_yEFjtU^?kOF~Iayy)duO!`@LD<L*!
zw-Pe%eBxLxZHw#JFT{l5{PZzMxt08Xj!mGS(Z9V@WS{fMZ|f=d>TmO$HN-cPA$_rB
z@7{8>23v%+kQOq{+tREwbHeYYPec=?Nq80#a2@<^mnlLCH~E@x=QJE89MW|Ku*hIV
zw||$XsFH8`-TwK`l_;5Yn-9efRmLKWb{&_LjqYdfaBYx&arsqyM`{DcK}+YR-2qGW
zS-<VE`vZ}xwc5ahVnKX=wqA!A7hpU47+>4x(W!<m0`uv<x$a}+QF)`Ws|UZgZN|as
zS0-Gje#h^|Wfb8(eSThh+$`o?;gRDHu;YB-cy{XU`x}K}4>rPn1ZRxWkam<`Kb|M1
zE$5C`#)%mzyyle!PFEEcR0DcyBa)3tRY_r`(GChUmGaZCAXeqsBf3}b5p!?ovc)NI
zu?1pOrzuoz&nL<D%q)EMNw;eSSmWZ9wY1>oP4LK$*S59q4<sSLO@wW{jlX#E7p$F!
zUt44b;!9?xM4L}le(Wts95^!Y+6=MyvU<WmIACP}AwxZJ1Ly76k)NnEh^l5mgoXSP
z)YtRisXJ3Yrd|W*Z9^*l8q<%Xbj;W#c|~WeD1*BOhS{orw@Fd+1?o&B>yTLAkY2)v
zPQ7DK-AUBExl58r3_?VRj`B)FY7@Ws#@;&{1?HIqwm8}U6^LRA)Y{8u_rk;tzdmjK
z*-be^kePMBBc{|PzPngoK(QbQUVVf2ytyo4nq9*kr|Ywdu=WB|do$b%zW&PHq)5?~
z^Z4npVkXYYX(&oyJ*V$gZBB2E6BBOm@LP{M;mjgvT)Btnc#;}MUToioshEMSed2Az
zIHvNsZ#S7SzVCyv={RXYB_y2UeQ$4?Me;>d^?HNBgM1-hnA?;_@g))rm3rr8?{Ca~
zbhCTYV>c9ph27p>b01LcP#T59pb`W1kBkr`-u`<R%RkcddoxS5Zi5+xmbqnmv`<~?
zz`)<doDzQG%*y0ABuc&QEgpEMY&7z4(!B)UR;psK-cGk7*5up=ti_^(;zBKd=AB{}
z+S74^8T<y|9i%?$VFH&9mccgk#XA+I!zKn_3@bS#2<~?9|FaK>*~@wN*+S8%=jYEl
z4GuNrKnbxMPrREoIeEM4$%B_tsW%<<?t&PA!-0|S#!UaEG$dDfcMD0eqnM1K3^o_-
zmZD8a4MD3n&Ej?M&J<IQ@@L-VDJRdoZU0J*+^csb@~P(vOvPYx3w1F02o}$5(~(Ec
z`Iwkgz+ZfCx+$|Fuo9L&iu0?_*}!%0=cb`GcnjQii7<N#Vf5g>_ZvjbB>v?4{?>B<
zsR&22ju0O^blSUA?-q10*!P3*w53*`^xNP7a#z3VFHjto0&)@<A4s#~ejGyRl?d0F
zgZNscOgpq0rZ?mLX;%388zU_OS$aq#tBouX@-a8qrO<0!nie%i;1LV+Ip-V2#7Reb
z&KEM#aFY*S7@14^1b2l*=8Du9NbIjrRk1&Yjw8c9Ft98g%s8Ox6P3g==^B!hdR+;h
z`iHwGYR~yz^AD#iB&P&a<@qjD7Ay7IgXnS`<+w*HkE7gx^4Ta?)o=Ws<EQ^HMg(@g
z`om<g*u?!8AEewbF6M}dV|f3?d6aWqnx2D2fPt@suomzX!6-pPmFZU~_2MNSy<`^?
zw(!nNTMO;iNm0^eVpns;d+}F)^%H9;mF?=1MlI<SrmA&_2)X)aZJ5`5=%!g$eK=10
zgp-x*Fr4SleOR5@r_@_6L7R#hD_@4yZ8X#m<3|l+=oT%-&XG@{G=53<D~+rSOjUv)
zo#_1(ERj@?de?6L<VQ0_!{>bPqsdOgQu3Oimnl~3oDGZ|kPr~zbtks*z~EL<pL?W#
za3Zq{Ysepc%5ht%Cb5eB3nuUDM;w0)F?%-M*Y(Y>R@&eKdkT>&o$7_;*5dsB>4Qvm
z?){L?jc8410hOuMz&BA}>n9;X<sUv7HJ(t}kXZs$4gum4HG0>|V#c<5+jb%Zz)9jP
zg#c=B$QSX_%ex>Y_Fq=_>Yta#ifPMu{-28pc@2Ngqmf(xJdS>j{Mj#B&hRh(>=t*M
zc=o3wsL=oE{nYa3pFWb3u8xnzNd(ncMMA_#jZ#3Z9#Ph6bX59JWhUjPT$xTkk6$Tw
zUjf?Nf%X*utUXShJ0Om20Co1!0=VP*!?^J;cF_R1{xXYZ`~F}2PO~0D75TEm=opQ6
ztc1$A55xnf>{T~U8L}kQ>z?JhtJA30Jy%!5lWP}9to_eUVWmO01)X3ezB*1!f*0|#
z0&4I7?1cM<*DeZZ-{ZAH!dJy$3PPDsU;R$yv*=UDTmD+%$w^$}`*6ZW+J4^$6Lu^0
z31@sKCQu-5%pOsnaF}oY{8-XT6LZgT@)>`1iTY9=`D-&f!{7bun1p)aFxUKTG(>=+
zzwOVx@P91qMQDx7F`p&AuvF2xflEW`x*iy6wDtKu6k+X_$kgM8WL!2nGWHhYEBpDy
zp~(}ByC2|8A%l3N96^@Eu7_!ZWbXxc33@mAZ;WpJ=$3a_($MC0QkQxehFnd2?zP&I
zV7Q$9)_s@&$KS*FJA=Q!n_xxt-|(i1#*#x?sbr?PyQJA1D0y4Ew&ZW-em;0DQ(J6e
zFmMO?*Vo)>*Gz23y8Oo|FTEpb+^L|Czvrc4s2%GHx`Ur+gYMZFcBP3g{`)Xp(3_uf
zUb<%DyZ-I~4L$ew$3$leFa4jfKv(2{TtHLX|14(b_-Fq!u`+8p5q0E#$e<7#ymp<S
zgd97;9K&Fi5yUYPvDct%DX<zs8Foa3Ed#78<f!0FzL?mWxq+@1uN6(nM_rfM6kLyH
z-{vcjUmN_spbEisK$UuXJxL=dBeJ!`%ZjNCzN^k{(QuG>H0nqkL3`T`ReFh)S>t7A
z)G>JOm${-nk(YltbCfNUR`8%B26+rh>*oYH%2v=_O3*{(KgoN(oH)^1FcYSdnb|QY
zW+L=OskbsJ!^}moeD>knc|*Ba8!vl_5BtaW**@O+4|`ftl^P?17%5^ZN?j_L^FMwh
z+Tl+AXHF`aEi|fPK)(Ds^FhAjpRco*_}uHGE$6&?U7mU$k$r47_Urm+QCq@azWxKz
z(ZtJdAZ+~-kKBNibCkb)0~U`p{NfFJmgX2bg<^gs|4H?ZuFKlPjsLQX>=Mua*FuDO
zxBV-xRXFDf=mf}g5v5Kz20+qO#MKE0krseF03_`6)C2>@ZsDnTemMf!b#1@!e?(>8
zM6l~R@gyPI_ip8G!xlq2>NRc8`QI|Nqy^&RCDM4*7IU?c)mN!=UC`wu!V+up>jCU+
zHz*iX5=RkTjbCjsfn=4Y1laZTgS7hhzbXZ_TE5OD);9L*+7Y=n51v0^tnE;Y`}4vA
zdz|MzMRMnD^#e8rIQqVJr(q~>0{dnPZsEtL<h??}ee|`|+7+ot>?TW<r^|dzy$aUD
zp;eyPj`9wV^!r;%T@I8;sBr7xjH}Bzh3iB>x({iDOI!1(k7u_D8o0Rw{Zug@gEMu^
zg97+LViaSwDGhW18~WqCAwXsV-R4n-&F^m3EmlJMeS|@nQrhtwHwNp!2eF3uAZf83
z{Og;Qdd)K5hm{)%_`ls;^&R#l9Zi}IdI0=K$+xwt8YmgT2@m^k{IFiS?oBu9)LF-9
z1NuXT&BTci?Q_Cgm^iSdYg$dLKf8iAV>L>>{sN7OgMbiFNkA-YV{m+8H0Z@WJMfKq
z)qM3$Z|e=(!=0z|=)~ORwH7h*45<4W(ysKT6ftiqu5trth=L<S`n^<#({_e*wOE<f
zO=VjGdC}@im1jyGz(FRu2<az5LaK*sS%<NMlvY9rTR>ebt!FxY8Zs1jfyv_yw(ptB
z{=|;U`HcON?UT<lHeXELAQy>@vy<`(k>#1p(_m@&21gOx!g5UhLS+6iHvmhhWUpYw
zP^6t>e(?<En!anlL>l|FE&fW?d<+^}BZm`MF0PU)Ihw#oxRT!oMcqs24`uFwf?EUg
zO$7-t5MXxFXxsMRsk+ipeoQ?n^{%G)j72mf+Ev8!B>T@)A*gp{%K3?GHcjwIA{#x?
znW@8qp<{yE3PDPHbmm(Sm-pZi5c!INN`3BIR1OSZ0AnI?1Z6q%<iSLCfyVjoB<5{x
z4-!oST#~M#i|<vb!k5?;EKTfILa8`s#`8*@Lk+RowAoVwdGlEuEGxv;my|mD7W^JS
zz6j~%sN0RU^N3oYMy07#<TB5Lx28s^v+e<_dlr9ch!*l|bh!(}=~Rey&QR*ID<EG6
z;|;2j&qe;O+ozUrgZxZ&&`{z=4BCORnERw&Gn{17xYh99U<s^*6HM3T)7sctf39QR
zX@Rzv1B1k{W+vc9JZxg>{ks@9>OqqL>2y5|<7>Sm5<0wkF$sjKgPBI-H3}^|)=2|c
zliE*~^o&SeuGX`A?@L%pNA%5dDWIMVtUEX%aqaEEXu8?XcH~@F>e&`*M`I`<lWI)@
zLDi{r7s?>pNGQ-E6vdtXR&05ls*i^<%6?>E9`_!!n1<SK;IvJG*?*$3&9WH^?w9m9
z0wQ~^Jk`M51l0ou_8r0y^y{@aH#?=>U^`YJ3A?W&Zw?-H_9>yXD*3vBt)G-So62xp
zVlkGXR0<j+E%T;vt!0y{IWK~Ag~;PzxzJ&bS~>|tXf^p{(ZD?Nkz_VC<t>a5^ODs$
zkIEV&8!^JPgDfN}BxT8nDiGlYGhbI0I#u71Zn0sL*R5MC*BIGi7L{K#vhkT<lNLn&
zx-z{UFdY*8Jz@>-`O0RR?Ih#VZYW(=&upjcNnyiFRp2071t$vX<Ct9wC50R(XaL2=
z1%t5AA6zVS;5ad&N7B4W>z2DS*u)2I*lH*~(LQpNKHnZ5c?(#biAn$IwP^>gev<H_
zT92_v<v~Qzml*Fl5c4ICx#{Kodt&oAPr|)Z2jl1^YEORzmk(ZFB1L+9DBqRI@Wf1B
zk;&X<jEBvLxl-!bDY+|?6}PGY=Pt)UD~O2bR)o&p1^4?g8W2E%NeOe2AC5L%*h%s+
z;bvxnIcV7w1H|z;>dqr3`g2+=oKdiZ;{^f;E?$2K>IveUorhEa+(N#w)Z#Esv}kG&
zk)+-Q6p2VLrpN#SdTRtA-x@%`0;R*iY_0RA_c!8-m;pqin&@K&5Rrs#JDE}Bt}JGE
zTIa=p&pk*JehX;M8UWHRpU+~2V%i~Dm(A`j%-DdwpS;=myDhq>#e~ga|MzMB7~u92
zv{#fm1C%y13>>x!d2=?KXKoU%(FCJGpV-zuS;=M}+1^;vr|rO4f>N8a6mbS`#CBc$
zvpi^J6GZzZ*=l2X274meDdBV7E>E?w`4eK1(zh9nI&VE9JVFVOFtFx&6OgNcl>*d2
z9?W{vTgV6H0UMh>2L>64Ri_+>Bs%RevtYsxa5<k>LUy-`gW8-uz-0#BlU(P~u%HNO
ze*t`mg9GT0^N=Rv&NRX|(iL*V&O(J2qwhpTYVt9|Q+U~_{y_^}eDxFYB8^Fp=Rlc=
zRm0=CKc+b3Av?<}v|SFj2%iuUd1k%_9m3qLG@3s4g`{qx#y2|;w~_YYnXDUy^22f<
zhh>saMX5I~hkv8z?7^I&&0X$TSPLyYlyw*`v4P-uCIm0W919pA*nGYsb?ODC^A?x~
zcS(}ee6hmD4WL|uAabDy3v`T95f{oThObIjeX$~SnZrCtfFW&5DNk#IC?o>FzeiM(
ze@RG+^f6&cAJZ)vZvH~4H^86-rUj=K(l(^U*npW}0!jV<K0~z`mx*`Yeg*+-1QA0)
zyHgbWvzh$56j%wt(w-`pJJ{sR^fBJw2o_l4=(SC{@dZl;#+A8)@mAVYttCzb{pxh}
zx<e@IRaLkOq0imdClI*7d{vE`Mdj{mIMM||uN&qLsw>g?Z<M^96o9OgRPs0NV9Ac;
zUT=}vveQ=9Q}r`wT;V>!z1zv$?>Jdr8v4G5_RUFxw-v%s$qt=V@^(u11WQW)jthCi
zs(y|R5;|Zr+z;aq9&#05pQ2MilH@fBnv$8ia7nYVw`7OO?Hyw-Y9AbVLlLy!Q|e6W
ziy37>dW?<K>r}sdjDmH;&pwrNofetA#gK^2?=lWx_d&|Ipi;}^A}1U1pl*41l^}U)
z{w+B}hDn-5-p4nONo_<@yi3w3Ldpmni2@?aUC}XWusKX>ZQG)E{DW?cUC}!}QL!1a
zkjE~D*~a9f5a9D^40_5b_c@uZwYZRsm^xpXIH1(0eklP(s(PTsP5heBgRRlgS%b2@
zTlF$g$ZR$Mt~XzXdLi!U@gJi$6kVk>*k*ZuqwRaMJdm$q?AO{pPz_vx_5v_Yl@#48
z5i2-C+vZNi)-4i*R(G+w9~Znv{{`<zp|7Mlp`%2X&{k5A(5*Y2A5~alkIG!?jEjgQ
z<LN4-HAD-6a-kp0Q|01t!mNv7BwRxPDD~-0XcyKw@}?-vbpBgIrSKPs1SHM32B@KD
zb|C}%r_di5rBjZB3-wSrabK@TJ0M*{?AQ-(-J%dT#8eG<N2JtwKTQ~{a-dIV1VEZP
z2(75sM24Bdo_wsjKJ}QV4#bN)Ffd+JD3~;iUm7>)N$hD4uMiZLW;#ExIA@3eP(sEt
zI7In!L}Js$tUKf$iNQDcvHY&Y3P#VSdq9sHQ4{sdL2w<6?8u+P1r$^%YO_Iux^|Qu
zlJngx*N+aWq#H8aKt=bA4=O6W4oy#TV_Ov0?s{zC%^0qvwKQ0qiN1#CDNV#k<;`x)
zZr&hwyV-E_N+&aJOVgCB*F=?C-8A{4o7r2DZW43#!wrcPe252XGmJb<=YAMVsnwn<
zS<iyL?RavO7bqnijHrm@uab4(7ftLCiRaXJ;Om_0@W2{i06T*K3IS;&9oCYkXf@ht
z#_ee)-{TnPzCe*W2IRDoy{JoQV#DRNdCVzlFUcKwY^DkCqHrC6LL_k4Y4Q-&853oT
zhh;OJobO@I)?~=T6{$EfZ+VJ=Ak_eySgO<};{A9<G0OL;dQ5%tw0M34AxWuEjz#Hp
zc1hrUw_}7JQd}uxVEWhjE>xt(j$0CrTVim^`HCDW!-Kb2n2*c&dE^=J#&o;jRBAJK
zVrNu0CXm@ApZBnX*>wqJhV^8;9<!A03ZfG4aLo(ylf&31)+1YoGdvxYYldSfbL7Zy
zb|(s-8O|2iv)wS1w7uDQa0us6gxh8>&S#xcmvv2cj9`lr>MTr-j$oBz5@!+mh)g5p
z=KO8ZVNvWmh1Jnhr+R=pR;?RuF2cEyua01~V(w9S>PR*N6Ym%aT&ACqpBu^SY3X}Q
z>V*uPx_;^bLf6aZN8+ka*UOfCmNhEv0``II@De~OR}Thm9?Yud34s84lu~C|CfA_7
z$x?<>YB=RMkUwd>rbk|#&q8AAc6lJ5*~Qe^@@M(XY0pVjH^Qa)Zp=2RxlgHg%z{Cn
zR~}Zt#$tcx7O;t;cB$N6!0e;!hiS3u9@A}K@2K*XC3Y#U1hcwN^>*;zq_xWo=0W-0
z0`{nwdr7Vv#YRj|EhEr%j&ajeiND9P6)N2UfEPr37@q!tX6z!SwpAd)`8-~RBElwd
zA6-D8rJ%~3YO^=WCq_ZS$yOaRO1*H2#`M}GpvsX-CNA}nltZkBYDi|<ws?Lz@@M3l
z(QJg6y-!{{8YoRXDff(K<MMLI+4Dz;pu<xnOK0$F^*sPR&kmWNNW(eFWU}TC_Pm&W
zKz{KK_S;d;RE$e+h~qG&seww(uUELyH$e9{1c)l7$x<PAj$wZlZ8mvFAsZ*=tdsi+
z+3=B#1{)0yM9csp#wqm~7cd~Ajl0H?aR|VC4ds1}Y#fU_q}KEjVSCBX;makp26jN>
z&(hkVO~H+2&I})>18Ln(17f97W1l7D0Vh@Jv)1{nPGRaxLcUU;Nf1LWv+c#Nfy(fU
zHW_5~(Z$+i!0JO~>C{FdjDmz`I8fkL(Mqok>mgV|gGU2OSIK>2*|daO;|*C?#4231
z#&yQ8)3Sh=QqznyvVhf_?n#T%1}U}1N%E2+wjd<HMlM*V(sJk>WUY9{dqc3`&#2U=
z;VKj2Ag2Lj_UN}rPEJ)3d$Pg}hNqz;Oi1>+%{9%(i5ciCWJfVuAX+BLEydVm%K`bB
zVm5q6qj*47<{aau6V(A{go`2|JMNtkM<GW4Jft$!1Y%V1PQaCY#tVsr@D2cqz(}sh
zmT}BJK|d|>Gs0)uOEhNi2=EBk!h<bBzM8g)0?dKX&{9stQHH%S>E-Y^R+3(;rITBu
zZWMXOYPB!P-Q(E9V%l-pF&?Ke`4hQxJS&Mu>u<%C1y7%`OKuqtPL%On@>X&B`^O9x
z-6gT<!3{zrxzq5$)jwIDlfM|x=8Ui`8+c9i9tDYnlv<5yS#Ex<qN*7vBh=wO*(X2b
zg_e@NQ||P#AG8WfX@%ROt(W8c6_T~S2y8{bs_ALxG_}0Wx)0@Hyu-9WLZ7bQ+XoW@
z=v(ipfw0}+oQL|-SikhPc%KrKwZWjYm!$f(i{loltV%Vcx>l<5w%(q&As1JzUKI-5
zaCZfVLxh<P)dT_q+yfWL&8A8)DZ<o}0!=@J#HUJT>IZetL4py)e_@hk#J^$&8eHWK
zxitAu`v*m$*Ph(#egT(eouj~g2JaazR<0unH!s!69C^jK2@h%dOWxMN-;lg^f+aaT
zp*Xn*jh~gh6WHC(%r9u*+gdfW<dT7-PNn%=<w~V4_nO=_0SEL9*cKpE8lV(?pwCAI
zBjO;m(vkzC&854;MWQ0GM|Uo94_2S8T<P9j$lQnVhrZIwzrRIho01(YSaOI(O5Qey
z$+qkrmZ*4N)8kczc0g&cUs4ZzgZ(XfGm88%dTmRql1qJTqTQ*QZO$~iu4CI&+oO8A
z8tjSk4j;>%G8?k?ZYPW=1__enAqes9Fwv|2KUiR(;0XVv?puRx4J@n@0lD?QF}q2&
zs7lhdm1wnX@@JSxOq(U=m$Hei)_DVe&$;!(0F*>}9t$VA(qJ6}g-N5IAsTj4IXwl&
zw_6~0=n&iTrGBKS-y-F8(LD5rdCsH!Bw={-Yh+uat;;?LGng)(-idVg)<E7L<N4*t
z4+5sZmTFj3asFe1K|?m8*-RExEO>p}EBPq5#XF>+Jb?0XD03qKHff6_1%3bSm-QBG
zI7RXuWz0sFjO49`t@KSXJV#k0&n;s!#q4^ys|>5n*cl_Ra*DN{0Ym2{3M-2FNR5`)
zxAn!Kcj46~45aGh_{IU!>S_@tc^QDT!@hM6z*NS7%Q%KI1bzzT08B%{9p3_;u?2AG
zP&W*9q9WH%WEO2Pv<bJSPhK&R6>C=(vI(maOXco~Y_c<%g7-*z7q>;rwh#q?+E%D?
zZ(Z^RS$8L!Fv^$<J0yfNsjnIlf+RmsJ&Zv42oG9H%Trv|U&1VLOpe^i3hz$7iE+pt
z3;QI_4}F{@ME)d!gvnbphg#l{3g*CIhspjFZQi1KpoJnKVbgJrLEeJ=Qu&iR*?Lhk
zO@<$Ey_hgdzF5wR{WNfc>C4yv={Jeg0pxNw7N^vvz2q(~WK&+P{H`1AN}5?$Nr9*<
znJG%D0lC4nMZT+o&B{>+p@3?!baNQ-s<AwfD|W~)R<Kb{MBS5J{ZFDJPU?U#r6hJf
zQijW4SFj5SwdO_g`AWzt+9tWtkGneeW4YbW3eyq}5q)oqdUI72)fsNc1AgXo>MMwm
zD=rhB_iG0>_ubJH+f_YCPM<Fu11zVe&De)$#jguR;7q#6MIj%LLfRCk<MU<d+BTTd
zB<zAgapq=vu+Zi{MBrqrVui*d!Cys}aV?&Xl;L%O(*cpXQ2u^^`9$%Y+!Me_%qfsR
z3*f@DFO%<@1V@1Wr2Nbz2zO%Uk0#?bv7VA0lUYF;hF0oydsOJ1wiFtsb~3A+L1?Pi
zEdrW`!0lrey!qOrbW>o{h$Hj?1d5cc^Wrr%m@&T3QY3gMB9xB&Cm@)>%j~**aWb2f
zP@7mFOH)`V2`0sQ-9|Y&g-sT9TjUc{SVfF(6Qi<jDjS#OYEhq5Sj1Y8P|Val$u(E5
znab`H9r^N$Q(2{`i^yM0g@UA$Wa~8Sg|=6|YZ@Dq4v?GQLd;nxp;Qf>sTViOJEpNG
z#Kg<8G@Z@QOIeG%!Fl#r&Je%{2ZG6MH|`Ou*aWftZhWU^t=uu46`N6L?6x!+pRrJ3
zk9>YQn^MppTOPym7=7jfVS!nD@QISWKu<X05v(d&<~ujaQ)jU6WjTk#99N_`EeBOa
zmAr_x89U{H88AWBL0Ey&5|#LjNV)nM5pt}19v%BtbAROlH~Zx+1ZSMWLzOAbZ&$8t
zeyeg_^XbZs&2Lt2Z+-&-G^Z-}Hoso^Z1ZcChnr7Uz66iN@#<;#a71^0RQGae8<~2m
zrwT>*2!v#=gS=9k@hsh_aq^zo+8#5i|8%z?(MH<QK>fDBZTd$&tQ_o7r8fNvb?m<=
zS=_L0qa5uGGgaUl?9Y~zrS!zxj%treNs6HV3nEH3!=}Cogfe}LAk`WF1PO16K_jgM
zds>{>6+I!f&Vdwa^PUvMvt_uD2onw21(;|$Sj6uz>1>*GSJc0UI)cmL3p*7Ts)31W
zV4@nBs0Jpgfr)Bhq8b>h1~#gJjcQ<{ny>-v(Px?9Cu9JkA3$vdP;H>td}V&Cp-KAb
zu3Pjz3`927d$1_k?f%`Z?Fsuw612+vv`GX&+&Li;cR(3Ec^8cr(**hq3Y_>a>pIn^
z)hg>JJ2Un5Y&L(@_HyMu0-B5u-eJNU*wST~e7j0N`C`%NdvH$3vW4%hNhLd-f?6?J
zv>1CWKRokx&aIt&?l!fv2ItNryx~n`gJ8<bwwF&L?I9MKIN$mbF>GbNeY<Q;WhLXc
zNLH8u=DrtoSf7YOIL{F8ec)_mRbUO%y?0cvgV|pbSj`NE7sa_)*-+qn%qX{{GO2J^
zba-EQwcw>GaCPm3j2;E~QxI1C*k@CyQL1;95YRs@k1#QbRJNcAqGH7cd98_6PHeg9
zE5|#Fbm4i7ef@F1wZ@x;_<Uj_by*M@;97GzevQ|g(xa>Q?<d1>-frAbPbTT)ArmVe
zOR<>qv$v~U$mKBOC~1;Q#q1vr-ub1fQG+hP{4|E^YS~+IjhQWEDRPe)Y+(IK`CT)c
zws?NJ3Z~++k5oNH&`@4K|LTJ@*{<lio3tFz*C9Hl0a$GTjMc!!^v70v^5Sid(v2GK
z6h6amL|Kq>1?j)O{&Jt??7p0D6d?h!wuQM`ODDAW{fJy0M{7;q-x0+bO54>1E6!%i
zy=a!Vt;=JNrwz#0yw~bDwl!R;iZaNT209sR#Gs0^Y2=O(iK3cS>|1}mxzF8!O{Pc5
z`#JUz=Wsp9myceAw=rk23z@Bg1m{#ckgY){+v2m8ZjjfLGdhLxwmqQr6&6lkydnO1
z^^aNfu`MEo`m?<-cUtke$&dAQglDoOo_z+aa8hz72q*NwiOS)$%^0&7pE0=5e5tYw
zW890eJBcK=hD+AqYX3@Ub!D!h1F16T3=H_k%ocS(Oh^OLU?`hQ0|Mdm=qzl8h)BQ0
zF>s~F9^kt&E@8l%G30-5CsB;yyqLAMPpZ?~4Da&tA~IfyF)Rpn<R7HG{8=i~>kZ;u
zF5#m#$?`X;EZ04YATaH)+`F}}{7cZ{D%zWE9i5k`d}~+vF}XaA-8aH}PHo520{E7Q
zg6*A%?=>9?b3t{lksS_Ea#6W)CUY8$i@-cLDYd31dGk!>_Jd%3ok{IxE_Kp(TgiC`
znkw>^i3pcX{1}gkCF0=GPN@f9kd1ma{_+;m^d!v$eQU4$$xOEN9-0B(MFfx?#!NM|
ze)25Fn3aDcInz`qVr`b84W2ptf3ztxDso3PAk|+1q+68Q<jZnjHLK57mtpLs?;)XL
zGkjOu5qx$-F1m|tbRCYHQ{B2EW4W{?_?VEtV}-jnxSe!PYpy(S7qg9?jjQwxc#Ik}
zxK5MKBPz6$4l{X?;2;RY8!H>zEgh(@k~3zpk*zCK6K!nFz4Xm3x`C++gj@-ENZmAC
z!m~kHDEi;S8T-E-E@2r0oAw4Tl1$`&N41MujXM+}iQdpdKz5dR4<_|`c7w0DoYSx!
z12_gh*u+gg`83!el*X7!iX`{XV#7TvNd`7cro|HfOSn(O8>;DTEtR#@Ft}GJ+9mng
zWy{@cOTzrbV!7jPh$IeJSMP?gO&pTHz8kWW<EY$t56g97K3}JKz`XbN)~hOT&|9YM
zR~j4@a>qTaihxz-i-_+vt{~uu&jh(3jSizRrOS1Ktf^sRvVj!}-y%;7VFehG5IwY1
z4%RRi>y;xlY>_y-P=2$9jUHi5A&Cp`M<&LQq3y-^;aErjOmO64ZX!``HX9|n&dI^q
ztb+P(o6W}BUDt@%-|P$>^xFJ4U=QFbEah-R56p%fl|4(g%z;g_N6w$aMmP~z-;i|@
z+xis#xC&vX|Eqif{&cr_NPcJzTWJ6y-~j8v#>$`0VRtZ#oN+I!9O0;<nIPmZ$4obA
zyRq`3tr2+^<@0t|g4GK}^4fdh8IH=G_p;H&QMBHPuTa{R`3SC4duO3H@bvHAKT>;p
z4kIM}m{2EQzn6_fZ)q;#1ESt^jew(D0g*YX-k<oQEcNzI)J2tfwte!FxvWIA6v{uI
z%L-dJfyTq$Z2;z$GEW#2n=WIiI^9zMc9T$qdTg1Wt_ca-cal8Gv<L!c>yjr^sP2bh
zDUHnSiYsj&P5i_ABnzRG=n?@!{U~3KP^bRj7NKbKPxENp{LRmJhKew#xT9<Q78(SI
z-3FdeiEpY}be|eJ6<7K^l>d_kdap&d*gFFI^V~f4m?^7Crx<h;b*;=d?UAR>2ZdV3
z$ZhjMCTr#H`4I9m6Xj3T;tDyxmQ5UCoJZe7B918?Ah=BP(7g?zT2^Jg@iM+lAidc+
zwpDJcWp}n7MbkDq2-_qnLd)EhSNIV@h_po44gBP;pjYTHMJ@cDRXMltL6t^4Cn_;@
z0wR|Qt05r=J2oCs2QJf6LJ5%#9etQM>QoeBtZzAo9+5R^AA{iY71t<JA||^9T;&s7
zu{{`bi2%SecnCJhxfr1C&CW4L<%jCnbbHG1IC&JkAI<gedbGe#Cs}bBzm!kc;aWZh
zn_460mr)#7UJYwu#h8sXVm{*<>W=yj<kfoA*<kU&)57GVL=sqc5uA6JCWeE&#bdvE
zoZop&JvDBf#VA?$*6`?0QUCd4E&fYz!|OmI`6ToyuE~a_&0Mm@1{wK&47TK*>KZZd
zw$8`iB6^UDD{-ELD0uHi`rI|WZYRK$dFD0pbM-7I^$@~G@zI)p7w4+QlWtP6;<D`X
zM0eo<9!{k-`EEix7(xIi_CFYOGHrtlYCNi<qL{d4(oWeMf^REvx!i&f@K$jsd6Oo=
z6BXFIPwN)r(dDbrB~l>L0(H)6JVE2fVp)v&h^m<vV>F3p2u<WhMA?{<_4<5rDq*V;
zb5RKc5)LW_jf8Yxy0J=KmnH=}IF%ycV7-czjAb3rzXE`b>*dS_#P8^q%cTv>IZ8Li
zxY4qA``!KrsFDW8Ac<}a>U|z~fW5k8ChIFHU>$O_0h~^|MSc$bvb58J>x99!9sUP`
ztC&>@X@^r^d9Z;MXweylgCW_{$nx$>S`)k=!l#dKhoYRgE)?59@O}K9XYGisD+yj;
zZN>r1UaChv6ZH}GipP03S}dKB@nYZT`w-$XK4pl$-x<MgTwi*W%H$^-+2laV5b5&j
z7znLVLWxlKMY!r>(OQJPSh$P`p&#<x>;7V}0tw3V3`A2`d56jNCN_5DN&h6=*|Tvi
zZXx=5tOZfW?Ut<(;*9tsFe2rcGTOvEt#34MG(}NdN=FeR%|!4u0@-kSFdMA!Yv`AR
zKrkUeg{(HW9y)S(*UzG*FuuZ012o%&B$Rb1p@dk!M%TV?`$07&tkgQLz?)f0SV6z~
zNQKzznca$tIxWH;G?DluFtMnO5xJo0pfoi`x?ibF`k0h16;M3X@s{GfT&!&wRD|2*
z7+C1QLaNoeI(P#^iA7ku`aNSRgR<R=`k+#$zwB!bCE|^m^q{906Ok%8FEJPHrU+~&
zWFQllIfUMC>ok{9w>Dm<p*lKyv55?%r?khhSOqM?;vB1k8@gZBVd@NHKyVUHqD0a!
z^@Bk-k)ZFRfxK3@<mPF1l65~1yF`~K#Nqk4K+ns;h3w%G=7;HM!GCpKy~AA&tj>I$
zcA1E3>ksxH^IFZT<-vu_mD=6h2{9kvr*7s#N7X8!L$-w3sFYzp%};?YT`mauxZh@k
zS#E3Ip+YwESH{45?;6t{#P8_!f4fyR0TBdVAw0U}Z?}GU!cz*S(`DGGVWy2~?a5DS
zGDfdQEWU>gMpg^bgw-JE)k4vPqsBnq6DBtENx@2OP}UaDOYwY<n*WiIpS(wt8Ce6F
z>j_gUbtsBA=!kbHjpy6r`9M71mA8{LKcf)O2a|uK$%>anBi|FE@$wpPrXa-o2IFPj
z@qA4@-y6%f3U%><zIcN$@(yehzHjUQ$^#R+Zj~V&4|nF`uMuRj(05E{d9?LF(R)u)
zEEsmqhtI*bq)k?3iBeraEi2PX^lNz4kma4I?fr8f)M|XUmdJ|QMIcEbEi~6|GU}B1
zj+Jy`z@4@Dm=yQ4r}RR%Gd|kl4p)KB)X+;SjAjU6!uxEvHMN0Q84Ezii=Z4)ADRrK
z7cNDY<7DziO@{p3QZ}d6nK|JoWRCe+^`3wkB4$>VXR7%t`a>EA50Uz?KG0a(9<AI*
zzu(R!N_h5O{GW4So<5Z6=>-5SE{J#IumB7OcamwudRRT6VM(B8-tZc)$deD;$K0)O
z*I{uW`Vi=u>~ghN|Hyi&3_eZaY^QBCu@YjebSTstx7KkI=i++s1|*67R^8*}kF~49
zRc&sHI~7+_$asye3F=Rd5KM+|XJj@>*T6j*(oYfhM;v0kdnFz-FNFDB31uH3<0<hf
z^~8p<H$sLE#M8?VYon%BCq^i{%KND}P@+p3fLZuF%)&}Zb8;QA&$gCl>G6rm_Ym=Q
z1CE9vxoa`(ax`No>m?Li9}N^xP!BOwVhKcykcB88#MhKBsB9uroDbP2in1x*Y&fFu
zm^E1@r<30PM3*Ei%MjY0=#zxVDj^CIXq&Q4sG0J(Felg|M5pXPdZ#dl=tuqD@BB1<
zE1`2&v_SIg#**;|<D*3tri?I$BJ={*O?o+}@*U_^YY}n1XyBCVItF=dd%2{efoUQY
z;B-t3GXa2`5nl`eeq-BSY1E7-1?e%Ke;=Zi8i|`jSMCg#ixr!xGcK@EQ@Bt=6;nC^
zp<7&Q9FK2cmZx>Z2}|(e)4NJjDiKzfdScv02<A(f<RB0*f}KVPk7rpl0&MzL>~g!@
zi0^i0Y|Hni#93}A9UGVut@@4p!=)^jjg-Upv#b&3?VibFLEGkao+)F$@=PK+H!mXG
zsD~2Lgm*MV<j(t9E%c8s?uQ9zbd79%0J=q`T>b$2P1c0ZRk>ub;VB9@)~`2wpv*)3
zgfT)fDSPERBJ5V~gT#j+K4pRz_Y(=7EwimwTn6Y=NgZ-<8Pw{O0y6n2uFRYAie;E3
zYp49=GA6ac@Ov2_LBEb;4(UC6Iia=cIj!KkSoxBB3BXGitmbH1_wy?keonvihb=Pc
zA2R6hwU*;B2p*;xVd<WV^$+2)zT!Sy$*QJk+wc<P0zSEa0Ur#0s^pL<zwJ8+5yCTo
zs;Q`Z%|9OT&#f0=5x4-ga0({s$Nlgcxpqs+{NvkNr1jo(|AjX0ju2rfg3uKK;_1$U
zd>OOi(US{!XV4#!Z-yJ>FGPqh1Y{R1*8U6I$4d{w<g@38Ka<uHkZRwM;aNDNN@0iU
zjC|cHwBu7|I@pO`iXA<}O7c|Ue-^lByH`r1&Rd`H@1(lTr=ye6Y0u=NLe1kP(x~8C
zlDz4Z8T9BA#-vR&Xbb@71_HncTLUn9bK9fQQ87uK+Up5`ZZNu=Mi<Z?-2DF#_a@*?
zRomP6K4&LsI;Txq3Z*nH1<KSGC{VCKDNTR^X`rAfB2&vKLZOIEk^%)mnaXqknG_HZ
z#42b|QJGXgR8&+{RPcJQDCn&-UQoYxt&=oWxxfGad%oxSzC1n2dG}s>t+m%)d+p&2
zC5>_Ys0we~zSHy+Hm;spS=oF5yIIaX7!vO^m5{9|fp)`4+uj>R?$yhY%jhcPxKFyj
zLMF;iLi|R1l%7Gm;L!~1dV!(wGBG?gch~(*t8j%gRJqjGpj?TZBuW<I&iB41yoH}@
zVIGLl+k(4LGEwF`DuWs&kxVarkwDr#CgE2v$7G|uqOe_ej0nxA!b~J=+==e5pfKK~
z*ZplOL(6@l$D=VO+R~$W1}S*;vgOeE%EnDdapFnoNL$8=qb%`a1@1IV2sMOx4aAIk
z;rNP>8tR33EBax)Jv|+hpOG{nqN8&c>DFsvI)<8%@RC)}(`9iuiAlXZFyi6fohhi<
zF3{(N7KW_}tq6M;*S&Zrnzs9}vD2kvA?_)Q&Z&fAy{;syw)=hOCoBCGU*mG6cdYWE
zR<;0JwhetJ7vP>MB}VI01vpUKQ>APpv`DE?T53lYDeG0suiD_n3SOY;JGWTbttd}v
zGnb&N11?dfC`yW!_aNX@ZNr1$G(l_pkaC|gOI!PpvQjD0(v|^D@NHg(wXz!ajCN^-
z(o8Myu7#{rCWhF;26q0m+kCCfO5B#xAnac4=t`wkqtIt?+by2f96koaQzcE~X!rlh
z;E=tVVHLJX@FY#aD&;X%w$-k!!u=Qtd$p0(xPsRBH*Ib;!b0MFZF@BqB}S}6tCbWv
zSiY^*R^xuRgzD0Z-L~N7uIejySSC=zo}RmX1DNAdP&B_D1DM}(0>Ke@mZayE+rysX
z>xRXNcd-(8MC~PfdTyTm?YaDd#6H;{Atn_kCS`~B_>FlDX>bhfWmz=xwp|z`hT*j<
z^tz3NDsA~{EU{u^w3^jQYP|d>_4HF0J?pr~FT@7-;D}M$mDS4EUfXjK+p+6&B({>D
zYTMo8J?t2Gt_-n#cn&ws;vBPT+p%+-@VbRrVi?Z%p4-%r&WE4dlyE(J9CjCm*yd-C
z7gY$Di|K+sE=z}9DQ-YF^bO_vdx{&-b!UF<DbneApn|}QfR|Y-bMFdmrW$bv3Eq#_
zed(3KVWYINwb<Se1GMdHl}1t4F4<#+g_t3br)a0v;s%~rSQ&Y0>_#nO9j?UU`AXbK
zwESobGcmtX9Cw%2f1T3SJ2<2ZrX~d#@%dT)D??&7QmqhFOK?7)YQ?w242hXQwd(gC
zo=&INZ4~1+VtNWZmd|%l^WlyoQ}t0?<ixGoC>sh3&^t-!jx5Ai%U4+4(p7EkO0g}D
znvD(v82zT*Tn9%<>Z{q;V+cy@sO7C!zLOsDm6ok3sZAPv$Zd0^XnE5<xH$2kD9>9e
zrga`_J9cHT`LZ@$Q<lrYYD>+q0bHWq)6zF6P4Z*n$R2oiY~Zr>rYpr3ei(w6LB*E$
z$PWE3c6+39bLKPf2vJq3-<q4<HOhRYI4VwCvVq)H!JUO0l#VzVrG2qM$;b#>iU5PH
zg@n$$9c|c->uCPnh~l^*O?%mCDXCu?Dvm>4HY$yp$MV(y)<WO#K#lnvCLY7+CSj~Q
zc!^4zi(2JICBIW#yX-0@f;!rjA=t)f#J3Cdsv0?xUao=34eg`aFs6j|!2q`4+*n+r
z7}8{dc73DLyuDb&{bLC3732v4CONpD7mwNSrrv{Na1Rm|fw*?K3wOJrQ6d%1y$Le|
zw>Eu~a!>NqC~OIbZjZvtsr4;kA@BqNmbAE;8Mi;Ke^_a)czkUhhMiUOI?cUBNyMWp
zleW;^fcN@#Zc)M&+>lwb6_>smex+U7ih$(yxwk3r;Gz3(!Z)fVxFO<cgn0{b#2vZx
zaubg)_YtMNf`1{Rl|QO{9p88sy&UTG%}1i?9p^kZ>ku+tE8n4X^x_^z{ofu4tPe5e
zy<NPG3I0+4IcBEn1u8(74El5psl;}RIv&4Q60xMm|5PKJ=$8=c_+0Xr?OBgc%K#rZ
zGQDiIlK5RoZ2Yz_ZKpCU!dJdY{Xwb!t!=MS9`IGxU`th9mErR}t+aLc(%P!Sto5nr
zd?;>s%~Qh~MzpCF5sh&~)Qa$%IIiKih(p_vr@pJ^Z_sA;RDVysGp|+*9j_n5&g#E+
z+Jb3!)@mE_)uC$f5#Nvb>d$5+!gsZwYE{%M&-wxCgMbeWP_1h0Ll<k6<$G$Cm07jQ
z1MXU7)p%b`k@~ZY{W7Ok{L_tNB#v?%mA=Ep>S!hGN>;7-Dz{dA(??4iqGmK%YpGS%
zf{T_@t85sLqpDWfbh1`?ct4ISzVadJr>bJtiahE%pQl9qJ=EK7aIKPg{nkgjg1@J?
z%dJ(~^{-X3uHknWj+r=?;Mk0#rdG*-?sT+A+4kUl8-8<d;y6{Sbjxj4D#VBp<<q8=
zj;W{^arc;6+IJqc+3opY(raX`l7F~X>G=hIA%K4W{~Y-jL0?;|IOo<Xy$$%i<Uelu
zE|#j*igwRfHQcvzjJitk=1s{H^r=+}>6l)t^vkYQ`ZvY_1NHy0RvC~}tGL$IDn;oy
z_S7nF;66yFqYzbbSNVTIPngqCwSnMQJj_p*t)~ad)%|+?jOr5pFJmtWE{5iOz;3rv
zWH4+$xD3ZmJ%8}gztasd{5fq%f{w`={oFXN;h2PD1CAs9<6^B>>GuTyW%`f8B`|Xz
z9OW==1k4806`@cOOj>j$nC6CnA;=^8|LGXA40LOQG($!Lk8DaGdoQ;ydxC0Ee7O_V
zscMS>WA2$UZI*rTn2PC{eJ4+tK4H|P3A0B{pD<;zz1Ng`$|sGPKBg=zOlVI{QU_?i
zO;Q^v?R*LMsO?mvySRN=nC6<IcGVu7qBimf(X1@P-mD@_gym0}GJV42arUv(rrg6R
z*mwC;T5+%9zV_*(ri~jj-QKLMYk;Qbz0;?({SzVBX<tuOn`r6N)W_A3O5gcu>JO?{
z)T(*eI0|sMad>c);V8#36Gs(}RX8@_*oos%t=eIje&n45KIcEbi1TYWe#CLJR_$rP
zVbk+^rUlCc$_9SDfs_A7zuvmq{J&Aa``=CTe?=iL+Oj9wzh@qf{{HqF98{`j8PX^D
z(<^bl)_=YS=ZA5e!f^q|B^<RluIu^v<{*#%sUjJF&CE~mGf1z+mLiTE90fR_T)x}4
zY^ItceW&KA3r$LvHgdC?rj;*H!;M)8!sa!&Zzl*t7@qr6?)rslILbvVR4cTH7pj@w
zP1l5|Tr9*?%OCh~C>MdWmI=YMZR2&ia*Gg(Lg}?)AQr(~Hl}_W<<TdY=0;fy-q0?b
z&{0rct+@R<r+ESYoX8BVIs=R=qGzQpq-9nxEk6Q<?*>C$zWBq>(0Rz8mip(k%Efv)
z>8IOqo+=smtifH^+PNy#G&UcOkUtVfnJ_3e{7$MB9S)0d)sFM&wW7^M5n;%}`P^EO
zIa5TM^Kf2;b3DdsA(~~(vxS|`*YbIK9?@^-^OluFzhBE;q;`#eKOMTNLiDN*)+u%a
znyUf!`p6uZ?F#_{VXnlnj1u&Zc@=O?J&an1CYbR6GV*bRV(%ifsvcefTw4$Si1;}y
zh{Hw#SE8W5yf+MO8a~Wy;sjh73I{<rHhCFCMmVY&(cdfs1tQ$Q<y2b#L`(-J`T!>S
zqxJAf;Oq79O(7x^>abTtq6U%fx&#q982E@8yWrr6Bl0MYEA{fP;h(YDgE%S;cxF93
z7kD$UKPhTE@Tq$FXN8Dv9Kc=xJC3SA0?wD<xL6NgLgUpHz&YqG!-Dyik-%4i7P3IW
zn0gh8L4{&-g88wzz$5+TAz#Fn;aD4F7`p*@Lp{71_)tB37`PVLpA?HWwx$KKHC+fB
zE+%q5l5Mb1oWY-upEx?Q@P!Y_7jfC>SvGSx7eIe+06$2~F=Oq9Oge|gi8XowqE9@8
zgB0+85}`ojbYM!t(YOtcRl)qmYk_YDEr7i?k#v^nS-wa*Cq#0UIWnL?@)F?9K?cd&
zfe+WiM})AK1+jfR@E%}FLIv#man!Sz9TrQ;3(}_);F3}VVO_pRMWLqVAVJdzA<~W_
z!Ot-5B=D(Ve%e{!YxQs~@Q?NIbs?Hx_xCFBZhljU7DvsvT~t@F1^A_#(L~(;L68oC
z=_mbd29u|s!f`#wFdYn9+Jm?y+O%c88nr}?TGn&omP>?aH9kn+Y7+3dU>s<59#>l$
zn-wo>1iIFC9NU8gt#=C1CMW1bZQuiKC@>LE9Bo$NxZuY~LbwzectZ{6XW_iR5Si2c
z78r^1nK-@(S|Ia^5LtuG-a3<Hd62LyC@>2G+Kn_DnISCHt_;t>E%7tR!}&5HIvnx~
z$iewxA+p;91!O~sPLupjjPjkzh3M=i?e$OR!N6r~e}BSwB+LxfsPkOlAA?@o`MMA}
z^}1P3Ixyae8n8$X6w0~gR{(6V;uKx-g18Hss0*Gn4^*fNdQDd^_F4lBx}uwOy@`_k
z&u#d<?P$=6ZaWG5g<nDZ+;&BXT=rtUKrU*T>j{?69R`f|w*>g-qRqSI1{La-2mkLz
zHTHkHp~l@vKmbEP_oIFWVBQ@H^uRy<3^440ChM`i9^Q#@Y=5wPUJdYdU`oP~cT<R-
z0Sf~6Oh^CgIX9>gjs5x9z6%tfaX-I*P(Z#Lcx^qr0b^>V--1xER~3$clR-eQD|k$B
zXE48D59TB_!6tAXLjQN3110|G<AjNN=lB%>!``_#cKQVp27^9^ph^3nT%Qd=dR$i&
zeQiMn`z8R_1nK)8!rN93`{|Ka=shAt|1>{A4$jjBJwdH21c3u!fdMmv3<t~=!ZkSP
z^)3(a=Agw~+l45~4$>EaXHiu>yac#j6M2hjkZ=J4{hy+Xz!cm4Nku;j;YOecG>O{|
zTox2ab3J#xK;xbSe579fQ6UCef=)ir23%QZ5id_F2jV`CfPjIp$e?;A9t1@O)vM7U
z)M!vyP{Bb^V9@!Xz(E&;D9#G<E6xU{evd!;6c5HxClBpk41t5!1{n<AAjIuInz4Hg
zYBT}OT<@?28QhT|ga`k@hx5nd{Jan)mY~2A)bOsn;OKc*0q|UZdHmc}iQ}*zQ~bFL
z3fx6afIs@&g(euD9aLy|4)F9~{_vSXOgL$d3Me??6!4{B{)8`tm`IuUqt8SGj%9Tw
z7ZX?ETGxKP7kH_Lcf$uJ)xhhh0Pv(kLQMYA91*~iVSy=@Aj2uB;gp?0hEw(c*9I*x
z<wqgP>(Q5kzPz3S<xpViGFAZghrp>wm{zZ!PjdsG3OdcSv%pt^3QW5uL`6NHsDOnk
z%KiEHshBRr^b7vM2>hqRqBGKh6`GL_e4cs{<dXt3E(meo7r{Vr-xVQd2D}vcGZ9#3
z?GJ|HSv7clLsn41+1WzO{UT_=xhOvmFJ25-a9(4C+O<LK-GD{TD&WBQU%6I@`2mxm
z2J>OE`8#ofKRV{`!4~)=KSt7mavX<(CSPz^h=o}}pIHcn7E(XQ9~}#!aFrogz6z&R
zBh84vM1W6K8IJRQ0?LF%7WE0@Mg4&T6BHybIwi#7dfjd@=pTp((mw#JJutH#o-4$I
zRsLR)gYzXqEUz?s_2Cc+%VFYGIsP{5gY#S*OZ*I=;HqUfP6ZjRf=^TjLMf7~j|j0E
zPaAWEAbfQJjw8V)T74AwX3%15z;pdtZ_r8BZxBMGIT@IdPiPTBY*-td6>mWP#)x44
zM$l}exZwY6OaR6|R1TEicuj~+Cxg9Y(<$I9pr;}@Xx06&H!$S_vxj#Iu>}|80}Qqd
z6JlFg5N{ig>-hCXukD9~*l{ix2WX|cYi7{NcERL(aa$(ygK~SpZ|^F<0+6?NEu8Z4
z@j;CrpCrUS5j5F818}at0Dku639)~=-^svW|4bpCz}w6N3O@mhJ#jXe|HL^V4!DCB
zJ1`iyUbjAQ7XEi|u3tmD5C<!TI5aX?<3nY@0ft#P2g7Fq!!z(Rxk5af<EICHmhc6?
zLWD00;hP!6zPUmi#{WtTD15jL`u~f0{#ft9Ih^L@AN^js3g_2_IDX2ESK^_~EI2<a
z#EFYR3!S(m#K}On1<6U95T`{j_Me6YPtOfH-Dy<t^mbyVe@=UMBH?1N-=4lC#Oq7^
zD<QPm>&t|AD@RvQfdX%#hUc>M8WBF1jpLBNhTwbdun_Oz-%GQ?BXJJ;_tSztK|2v2
z*n<}Eevl@_g?c`40X}fCjlafroM#E~WwyUYz+dLza$TEXuepN!uX2Ju^c4htT@b`y
z_YvZo0)P1pI0ydLU6=0_-@++=?GqI6Yk%NMR0V%@{91+MXgz#VD52?fCRRe*0GHL}
zE1~0s61KKZfD*O=_);)G><giU?+-MI7obKck+?~n1zK>9xA<dht;10Up>!|v3(Uef
z1UPN}!NiX91fk%6wg#s0O795ZHbH^CQKq-M9v%#Q!B3C)uk^ksls=8?d_d^~rbAAd
z1H-X0<g8FU0iS?^o@GKASL<(M&@MhAl+}UddOFT?aRkN+(627X5gZGY)ipv{<Bt<G
z{wr(d;;4j%`bSw)C6vwk>zqW{i~^ev1@ku_2BsN|pMEnG*nFfOJ}Q(gfx!k9#4W_i
zmOjWQ5}el{|7iw7dt-yT==P9Gferc~gDq~MY+Ku>PQbS9LV09+5Rdy&C=;@S_z@_$
zD=mm;m*Z)<Q`(-5YO@A^SkHmaFh4Ile%t)<hb=+-VxyYU`HzM-g)m+b1=jKyb<SeZ
zdy!J5E|e$@n^y+c%Sz>W*!ggvk!hpVH*%BemTINYMr={zwF_I+C^>PDf~-U>Z-Hvi
z%v;q|bz!d7W~(|D4<OX4vphI*K#*Yh<2trF)tTdQT)=Ud5@{pX5K*h%2hy1!of#lK
z3ex*PdLKydV^Yf>s8g*v1C;lH^1d7`cbnQIYzByCfM`a5Xe5a41<?!;&G553S5Aqb
zycd-B1}N=FoDRx+L3wYEc9Gak2hns8O=ltv8_T3#1*oQjYI=a*d=#w!X$43tbbkMV
zm21^$psWC8g--dSo(Rfmpq$2(*q$byQ$aZml+*k?Z&2~6pqvWIsXEU;Zqy>N9F$W*
zIn_^j-9m{oYh@>FnU|mPjoPHD>KeP!+qZm&+C22GPPo>)FuPWC!v$tsrolX_7E_&C
zOkZlng2On!fOA9~{Zorw0?^>XbJ|JJ*WecyqpQrdqKBsz%U|GH@T;Mkh)t(7%M)s$
zsa0OBI=ft}u2Cv2wPM4DTAn!&#rh5GD3$~28QI>JRx4;{jxsFQYDM&=T0s+G#`an<
z2%6AZhA^#h3@Qq@@#Q>?>t-g;`C8HWy3cb^?ID%7w4(22N_Qe{t}4lDG;JKWBiK$6
zMzt0v67HKOqs{N9=${s8O*2!{(lVRjJEdvUcBv`aAJ3}YaZf{cpE^qIbintBPaS8*
z-gxLyH5Yd}IgjGf|2Mv_$JD_JuCA{<u70j4Z)!ci#zpt1w8baY7nCwB{}r_r{Pvz#
z)EVj9RJ+{}W^5!c->(={qtzt0s&=E%Y78|gMx*Hot4T40ndBuc{8csCR;6Yj&uUCG
zDR%pQlvCslt^2ELMeaWpkXsGm#)c-jaUQV>m0v2kV6lx@ga=Y>@?C}aGmrhCSLA-*
zH?N{jw$lPEY{m#hv7<y3ippMEk5g(Fym)lhDYd0?MB8yn9qRofAWAXG-|h#uhN&!W
zzFJ@0{CSEog6oipT$}uwOR-quE!WM<wId@@UR9J_4-|p8bx=IYXo`V;rVQw81A$du
z=aPPdY_oRMtFsvVW5M5y+L+oI%{UD;WwEHsN}ZK5R)KRUjH*3#TD7;_r24G{)jz*q
zf#yb~3Ce`R>h+cXr}m%IYE$ccZi+fnfzR(+))_T9?brXIG7I#&EY>Q{s5#cZ8r3RC
zYA>BpUCMGT@^v*+*{XGWU2UwaCe%cm`MMgde5WmWU2WU!bCF?)Hg*-NLCUROvQ-N{
z|6JISp3djjw2xj__b8h7(^)mS&HFHk(FXH?RPIp}kVVLY-V9^BJft&_8x<<tRDP+w
zc~)(%Z03S3KCWMI2ihMr@$&O|EL3jwm-)nZ+Z*aqgK|=P<!$v2<#k`uJL)}(QmsAo
zuG&WVN;~|n+RVCM$so<_2CEUb`5UB3ex=pEt1eZ3)gs<gn`_(8tEPqvyzHRJSLJW0
zj@1~+=L@x$&#S4Ce@^=LyqctZPo>)`4{BZCQ%l2i0*gsr*LJ_B&TH^XeGf??f49M}
zS7^EKtGC5|6ZCgGr$6Rf`o4Onv@U^Vs3SBoDH(=F<bT(FzkH~EsVMuk&puX%DMecM
zi|T1LB}3criJF)E790j0Q?;Ux!6b>sNRzCALn)@XLX$kD{ql+0WAdxB;YU`a#TY}g
z;9SOLDMlrkUF=cSZfFd>KE2;=H-?*1kZ6_L*(3eV&8jyxHj|$~R5&@d?$GO)h&45a
zrh|+8f(T5>+Ne*}hOsRucm(NV<xcIbPt`c@M@WUw&K{d*RC+=y2(-y%dKZ8qCiy+b
z9HU&JW|){e^fSsI?x%KdB$rcYLV^h?ir$GI3@XB;Klw=?sw4dgr1UYEjO`UUh#XQ;
ztX8VHNw&^FvQZAPn=(z^Qc_!)Qc_a&oaT0e!6bM1)_taaY*IF9Q~#y*QC{&q|1Wj1
zR9@xyne-foHJf}zZ|y)kG$jpR)24o{Zf!Y;S`-nCS~MP}&LlrmO>&TjV^J8Vn&eTT
zb-t>WC@*T&S5;f$t8o7)J(M8+&dfkK2&IBn`K)&Isv3{@_rX;)McJYKdKFQAm)7Z;
zdT%y&8Y>(J72Af+Mp5K4x(pJ4$%Krd9XD$uV>|h%_SrRcX4-qy-Si0Tcf5UEKG7tR
z#SHdqbG}jU@NV}Kr7q)Rq088Xu1qpMM#mu;z(E<2@)L^UCTcFL+{VT0J69uPto*bd
z#nxL;M9I(mqP*K!R+PLfSOzoQ!bPmq%Niky3I%R%g5BoND}=6bps~pZxrubWzvE^z
z%9TpM$<Wv2&m3GDff7!}o$u#)IwwPlN&bU*8GYxzRby2vh6|e^4N}Me#s!8nQ#0+S
zT6K!DR~z%4`i^qgm-D^)ut6=#@xAx6>QoS08(&xFD2sd>uB#JO<#(UFp{`bx_S*WJ
zY8O)jlU%E0_*(p?E>XPb#+fGhj8qH~Hhq1r9Uf##u~GaO3q#5%1r#Rvyacd;6g|pr
zG{Z$;Vw5n+b90FzRMtp4dMhWr=}&rE+B5W&x9*4i^r&EwABbFV--~9mNMG&mswq5v
zC$(W<%x8DOfc=p#B22z(P_-!Sa+K_>zIw0LC|ZvH+mMnhk7(<nWl^`A!AO^Zxk0q7
znl}~}OO(j)AOZDQI8U(}qfraEh%r>}^Y9dtT&ks5WCP_Vt&2sb-BLukMUKF;?&mBr
z9y6LREwb4y$&zDai-ZHLNqxr)r)YUj8y+JYHd`Ik6JuqJter>R)#Ok5Y}W;SsvV5^
zFM_Vd$i&!1K~{F~QsjTMgjiX4E6sfO#>zxh>8VxQWJ9HuZ=X%xuilDKyRCsdc*_hW
zK{i)sXZu<v$XexA6uvf%<PW!I+CE9%-z_*B(ECOo^5_VeV>jtjE}I;q7&9`Aojm?d
zF%kYoaoZ{vs@k!uYMgRIQ=7=tTM5@XH<4}B`8I7#6X{YMT1u+4DqXcxO=L=^$MsIj
z6Gw_U7#7~=PHvNXf?57(^bzM+<uh7xvW$y7#qpvYYw?8EGg;bj%k%BYva9zu!SR;n
zUUgB<EGxmVfoXs-QXg>sMa6@|Y!j2zh(P&2&mZ-_Er6^VRKS>kI@u81{-hv!3F0Pl
zV&(T*xn0`2EQ7wNBX@eEg`<$ogg{Fpz21cyp%FLwBifgGM1zGN)n2sA{tYfs@U-#_
zwa!Wra*Jk9kt3BTZGMWhwVa|Dqf?9t$!M4qs#UCvAf^6X2rMhKmr~>?JPu+{mA1^s
zFpxB0eM0303^|RAX1RwuQ+sZ$clBwiKUDA6N>gQO#<O}NyMpcsXZ!=}p?Wz+C56KK
zS?yq|oNE17aKy?*8JK9J*14%HOYK@`Iwb$i2B33jFE*9iQiuG1(Y4m*rOCm6aqC9L
zEcuA`&oo&aYa<)7v8kGfC3kCu&1BQ~wrnPSvE(0T<E2|$*zCVVjTf59k$;gAWuyq2
z(OeF@Rfca%bJ_ZqU3&M&=~A^mMfTPQb6Uuu%5Q1kwvy$y<TbUmy!V#8e80Apb8Z<@
zv<EWf2(|i%?@FfJVZ0^RtsUhcsT|e5%aPW0FMBZ>8>48-2`BBruBT6x3=J@z<<X>}
zA!a}rc7aVY&zI9h4mFg-E5O)_p?nl%d)Li%neuKh=E62A8YS=clUn^Ubbgbl>oi1p
zKmSR-tKDQmNa__MVg=@c7&9$KV|P=b)!NT$gW-s4b4(v*`<glAP(|6JRXXKN<sI!;
zr|h6SthMbe?cwjM85m_T+}@m}d3vKkHih{X^_I4fTNy;#?~+SymEr4MBs)diGH2DD
z@=(+*bMBZZ6Jt6|R}Af7z6h$bl+*2uM%ZPdUB+TMA7Mr2Aj*u8$&_i6_iDT5$OGLM
z)?d!Ehr+r?F^`UvFV2DGY{7`~mz4?{=e$-uSH?BC1rN*}j%zQ@l?`v186{8ozClUD
zEo&4v9|Its8p~WD8!I!k{tM&@WuazSC|fI!YPkz#Gu!J}Ibf}1lj9YmYQr4HD9`xr
zT_}&6u@cK!hEed>2yNstxmf*enijKAHl#;v;+w}{06uJDcc2AO11x>o8qFy*Suz@X
z(!9fvDlcl6SIb7qW7^HtGBN6-+2j_;h%q*ok7>=;$XRh0)Iz{exrb&=rc^^L*q+x8
zu8}Dy_1+rUL;2Acw-yu1&@-4f(~9C*?TK~JeuoybURs-7^vAPDIiA%yaSw9$&-{7Y
zm^A5<A8xHrnqlBI#Q|T>_3}GI?^8y^7+V}^2@c)-TMeku04v0UXXR|WAx2i3Fv&6k
zWaJuRkZ8hu)g%*gGflaMrWo{mvmTbe-n#QI+A6~hcrvg0QE4>{SB%NpA9G}!_Tr;5
zsUQKHH)!nfWQr8{mle}53OKy0W9%fyDaQCfUaU;!a9D3Z4mUQD!?h+m<gR}I)W@NE
zv66O$Ft&9^#bWHn?qK8Z((f`UZ6`&W<bECv6Xkh|8N5%kUyIr)8>yFt_O?aFVcRQj
z=YI*eGk40ACMN^jO!7rPuM;A}&+A3aw^PoD?E>L}D9SSWXgzkxl3`g~S$(kL^nuWo
z8p5w>TbhCJ4vnARV9tl3(9K#gQ|OF}KF=+kC|{-FnK=#7e%vKrO3(*M_$WB}H_K>x
zv`;(pm~3QyIXEWLZdn=_X}>%s3zL#{-ckClmA;V`;5>5o@ZD18Hms`?O~>IVW9#Xb
zpqEd_!c^Ymb$Ftz(l-M^IbPfJxSZc~%ireF*bBUj!8nOWeazc8Yxe!p)|(TJrVeBq
zz3z+a&x~36uPCR_UQllRtvFxNHtv%h{uVPb<QVO<eX>c@${<gFjs5^nEF)3^>Z~Mn
z;+21Eo%jEjN$RBin5146TFQQDZ7)~D{b(str}l@sujmcVYCi^&I<>#jKHD$j#;o(w
z7N1F$)-&jje=GSAO4c#{;cwYAt*gua{%_eUwUQ@f%O2nUEn^MJwGWv2s(%(pJF<Es
zudT1|4ei(y(w6lAi$aV0y;kpvez)aXZmwVI2QH;7(;6L+_A!`H!pGs0WGWs(-~<{i
z<m$9P|F?$OhlZ)kuD9h&I?DY#RYi}czQE%5YugUU!rPv|m8g?o#oHtLsuw*=;<4gD
zx61_sbe9WS+LLnhz;(e4Kj*LOE3!7Ir$75@{p>Y=%f9CO;7OSqnsIRUei|zMd(*Mh
z_U{_W)05=B+0dn4R;+y2*Yi1vCk<C<1D=;nl;hf@=Vi0-*G)>J&ITjKl1<v?=jA|5
zqQ89}OWysO=>^$6^r9hyGjI9^yda-bLpK=<886kMUy|$l9p{~Q%s>z*o60Si^eOTm
zP`CcM^#y7mBZB?IsFhXzllR|K5UXu+%{+z14>PvAwc3~Dr2Oy<a7eYUqPc9zSSz_W
z8Z47~wlN;zhjs*P`c|XSnrUp9%#DvNZ>!PP)Q;aMt^Frz8}GrnTzy5}N55>5;3`vJ
zRbqOJu$zdf$?MpW0G~L>KqSPvCl(S3#hAKbAT?uydmgsV8e{!&nl~|$<Z}l9Cg90A
zcAM1(DrmRTpcp%3VQ5JoZZ~$29y^`tXH**D_TO`;b*%C*?-<s1-o_Z{U(xp~Lpy+7
z9IODlB*7IMLl2pUh|;Z_6*-FhP;0qL_VBX%VI~mId}7)eLz7U8OdEMPn>^GMZ8Y~V
z#-&iQi5S5ppGBoHIWWmHSkE>_?Vlx^ntIY!AZ_0!%Et|q0n#&bGN5g&Jf>?~Z~TN7
zCRJYbMXiu~6(w0ayi(dzKbu260u90IHP$XX*%*~Vb5zKC%AjbgUzS4>QsitqbZkzG
zH2ij_sD|H6O+6|zBfFC#x`t1BwX>__L3zxOp>;hfZ)<qse#K^N4eIXl1xz4gV6hC0
z(DJ0V;;8JO@H}Nx<&lEy*7C^x8Jzq5QF%l8Mf>)c%nE-39V0Z38ucx$<#CxD;hSYg
zf@v%wz^mHe<8lht@h6YV?g?+9IpLjf4Y^~Ek|B3u#sjzfMr(9JrnTINnR|r%R_|t*
zcA$#v(?+~L55)?hHItvv#+{IZ%%ACncWB2>$QB`A8>sMB?cXQl;*e?(MqsCW?n${a
zWSwCwz`wQFS7h^6uS><0ixC)0&S&7zkybWYBn06khLm}L8O@)ojebS;2>D(T+qK%B
zSLBHBhp;(ER0;B5TI{Q`y|P<#yeiX_^V-B$Wz&#nnf7gM?W?jfwpLY48MwMcEZdMa
zUuw-?lZECg@Q9G}@jl+yWNy>>ik<8mO;n4#8K`5n0)Im}lRN|;i9ihb;5C`i=o1F}
z;Lhl*HY3^;{n(CT?R*1H$p_R1*Y5Lh2aAB|(bwpT7<g{*T|FazP};vu5rX0aye!g~
z+QFzK8qMj()Fe7dZN}ekkiBjGS!Xw`;Ty7P!`G&gKy0kj-a7f8?s&c^ZO9w)FwI`O
zV^a^zW5C~(f}m`LeK1Vt#tqm858XLU%X(ABsqaMkdcP@C6_bzWxZ2dWq}`hW&o(C;
zQ&P|^ld*fANyEDl^Bonztuam;Q8&sr=Hj$5ou(S?rg&p=V`CCa;}DjppCSMuOxTV1
z?~KSyPNnZA;F1gW#^hY1J<-^-If6K?0@$mp)_4R^#R%a@K!=fiAzOB~YtNsPt-V;?
zqT<GOph(1G42!7gOb836qPeg)8`0F52n%8#9~sh;VQ7V2Pb^B4;2ytOG3&%;Ep{Sj
zD~47u#N%wp1~$`JQxED;xq9~NQMfh;8N6`mgyyC_Ey!(6Tc>9EJLUwTreP&us><h#
zg?}{=1b8yEly~J_nP1G#$iz}1FUSL#3(z2VMTkLZMG{IN+GuKzx@@k{yl=~R^^*uq
zdt2svi+c12fz^n2=owT~U$E)+09Hm|Fazquq4iA*3XP4^P#o=&QD}g<Zid2mXbxjz
zW0MX>W1_J`wlR`=_H4Q^XTV!V=t@iIDkC0t#~IbJ8!v4l<f3`h3bGPIU8KBmzoJ#V
zgFdoA+y0K6-G*|WpGJ-Z(Wa&-#@Aj^8W($#Xcqff3=}|H?VId7^R85sgqI_7Wpbfu
zI^3fzm4oWI7E?V#`^S0Nu8C1z56uM|89~}Dn5UpNc65J8rx?-sJ=r4Xr<r7SaK)Ij
zAcPng<?m?NP`G%KsfU#;03LCqH=4%OiFg?84Ykeh$wY5V26qgLyo#&ISWx0}^-T1A
zjJ3A`q!^98nH)NW5(BhJ27DmPD7(=P6OR!VS<CfB9HK!1t<{YuqcO%@IUnJ)5q8lr
z>(;w<CQs;$7K-^;^>s8x+Q@3tGXi2_Gq8X_jRImw3Mg7pKDFHs2u;4{K9Fq{<+AqW
z1=&-5a+=oaLzy0WITnNn6jsf0SvK+>{6c=iDaHnJ0&?Pv>8L8Nc|M;~7?_%6pj{yU
zDGS0Qt>phq#*v3cq+FfsX(k)@U&%m~E;n0f2J20fUrfPx&Fl|yxv_sO2a5*QWlVMm
zWa(JgfDVgd#A`VQ9aO(u^f|Ua|4jXiPEFd#hD=>nk6Jf|zD8i?K8v>KBiYLK7e;9D
z=PcTZk7O!xQ2tK}ov>)Xek9wa_{*R&y-e~Y3z;!d9-o3pkwh-@l11zHv25~J2FQ8Y
zqTT<oOl|QO){5L>QSgR)><u7{(~Tv{(-yhWs=fNLy!cl(Xr5Y&_UuL3bj)AaAm=-_
z>>ir6a0B}n51&u@t8Kq$!rc_3{^SXNZQ>s+TK7+6lJ_rWf~X&v!><3%0d)zqe*MP(
ziOcQ$Te&(7{XY1!1rrw7tO64aseSZ`d|mx~injYxnWip?(cYlnGgj?~Pi1xgddg8@
z6nZR%9QX&!ssCKPI!C{iawxYuMmzVJwB`QAV943S9C2mxFC72mBcR(GquDRXrnhLd
z12Nhimt-<>{!~?zc`8PG@RICXzl`4qFT`kHUXpgT(x!!8Mva%-w9LzLN788<^x%!q
zKfBE{C*|8V?cbMW$IOejCc)i*^0G~_stUOq);8>Ji;wt<KbLX0-W#3sm3&8CovRgn
zEwj|s<=XtO<ydufjrP^oXp=Q_wP&x&6m?CN@58IIp)qxHjogR<g_fVBAolHa6>$ZV
zMQuK;*}g-8%}2D(-^nJOHy@S9%tQ}$=z{8Vm?_gl1>qN^Ph!3;(W8&d!G;-fPhxGs
zShIXDlavp%$G?-Aowu0f>-Bj&`6nK4`guI<=kcbW$J08G%UZ_wGQTly1QR)RH+hW(
z5@U6Zx45-s-^*k6|4y-OyWEW#{ITz)O`HFN+>rX;Nyq&tw}&Vx*tsKO_m8rpR`sLI
z{2#<WQsaB;M;WdB4<r-veE<1L9#ZJ;xSC(&!|Lp_z5&-|3~rh{?;CkTexPc*o12?!
zoqxxLftwa>!|$?*^G<p=l%9MG!SOEL+13+BhQwEgM4{lPiNS!pr|bsBgy&vXQc_nV
zzKRX_$}q}WwO538^LN?M+n18ZQcC_|+!J?wqrg%dNq4=6LjCW6q(Msn!7XDQe(_Nh
zdLTeN74Ow$cvYYg<KKr*PMADh^cG_01HIA{LFxX0SndCVl2-I8O+Zq?V&&c(5sGKj
zMMXz^c}u;D87eL&P_bCSQz~I`IDHUTHjumo<n20%a4Omnxj2uA$N65b3cZ&dKz4`r
zB9f9ah(ir%kPv0Vc`ha7Zlt>Bg053G9%;sT3rfBKIwV*c(X%Z-d2uH+?#_jpNk0|D
zL?WrCz+7OX#8&Z>ricPtETxI~@e^=P!fFg+6BrN4G8b4;d!=v;zRVRfMokh0aiJi%
zKz4ZZ#e&3uYIui>xgZ5u3LfVaokkU;qE1SoSHxG087B&w#-LEOK}<9k*h%gtVtA(q
zPgb2RXvP`|)G2D0xgZ(lQVzh&@m#7XFm!E1<Zs89O%Qx4ss;GxpY;PZK*D!G?yJ?4
z@v26&%V+?qf2jg=>fin>^m%dI<p&ZEM)9edSuv(gL=7o-fWF=)_KDCy9TZ;y4d_(Y
zVBx>~fHh!}Rp30gN>64D=AQ5B<r1vHihKN7q`?E5{eU&#;x!dx>O_zRY9@URA`Kdo
z2Jgm?E*E^NYY=*`AFu{QB4fb0?MgkFHE8R-=0{|Mo}K-GG`P*|2dn`XZ&xv<P6TVf
zU+<CzNyL6|{OFm2Pjw9*TkQv|0h6qXA3e!WQYzv{kM{-u$=|zDr!OV>kNn{WET0R*
z0(AmNv}{LTgGqi9lK*ae8RXNcE<Y~J4_H2>8=~XOAfHJfzl`MTfaEVe?gu2lGSLrM
zJ{Kk*2nb;L{KfT8Cgy|UOOZ{dy8PD$`vJ>mk`?i#kk2HLUrO?IK=L;<@dJ`y-N6r7
zJ{LYvF{Vxci8hc27}x`-(-_{49|iezs>^?%ogc7#N;gKvql@aPkUxs#Gd+9Y3m^Ld
z$v^a-AFzBb+_qv&odA|keZcTG^~`N=w(8mdJ=3VgmsN<a2^0gWMf@0?H{^2_Jywl2
z(Dy1czGX}VeS23~@MDO!(f5i3d>bPhL%OwR*A&D-Ws@CX_Vuo*%p{#e+5WIOl1S9Q
zkhRq2XhK=+&BKQ~k{OUOr=>XTx@3U1k2!R9hWJUNM+pbByAUC4=>J(GEso~kXQKX@
zUF2xNr6@vV-{DBFW6|cSqa`ypAZJ;3M=J)#_zBad3P)=OCgd2_INC4>i7&eclG-vb
z$Cpo-A{-eELP0jN){#lTTO9@j#j^-iM*!^J?r6t!Q2;5&9qk!Jivo+IH%?8*knCOI
z=tB|n0;$>QT}NL6rFcAq3KTL36$J^7{#4+5y}$q@o4)rKa50SquR5X<MT}wuh4~8@
z>{J?WG88c}!Qq70grp%5FaE*NFPyAYF}+;};c(OWF=~&N_Q8%$uwTdx$U%E_CVe>y
z&u-z!VL<*A8Rh6gpoZ)>>yV=>1Ft#0qI|k=+{TEiAJ)r}OW=Y9;KVpbHv-iO07+qv
z?(7qd;_*&1;po9RsmN)1*^x(}G!3BB%Z{E5nh`kd$Y-+V04?`AdI5yoNCyZR;V1~D
zXB)%Y0GT}w2hm-?%Og?$TsB21(s}5`)PhVyIFU^gLKW}*iN!Gl6*s>E2B_ZcEQkz@
z>fM11!GWy!ONWPnM9%IdjuHlBg%jr--l1H`2m~j%le0nqp8m=)i~+T2WT@jVCL_<8
z^`v7s=Y#{eq8%eRhaG3RV<hMJ9cL7SXj1jrkHQ^e07FNkkl{hcSk8i~UNO|*7{`b@
zO|K^%;~7wk%~<W2z<?EB?wH7c6yNlS<8B6|_lZgeJxC-<$^9Q#?6`*kRU$0SF_{5X
z;+Z=fQy5SsVtYHv8F<4;y`Fud92LN!JLAhn!yBgKeUE1R&o@+kkz)oEQf<%daNNg$
zY8x@hL644#(n#Vksi$<%!$G0=*9~O;`J6=@j4wJCFrfO~^OIvC1C<JmIb-oU76XTM
zL|*6hjtA;WC1yC5K#9<uRO;P>j-{MKb$7ZP4|0xC4<!$g{cEU(h&am_M2LG3+CET)
zV<j<Piacvi2R&68wp0ufVnGwfY9>?;djXYWj<wu~$`PRaL5}tCv9L|bOF&cGIyUHe
zzXx0}Y|}_c%usxk5c(lVA7`JGRHX?&Ssjl;P&i(QN}X{B>rHl8d!J(`1FBK;3db%6
zUTSFXImcs+sG$dUbnIs3sIrGD9D7(i)$ftcj=c;hWLchfJkG%1XZJCP!jM)W9Q)Zo
z7E(~z1Z4_Y!9U0>afg)%XFQ}uZ3LGvZ-|q|il~jG;#*stG+5w0wg87FIveRc0VZs9
zHfBIUVbs5z6#qqOEWoiLP96ZNtpK+_;iT!ID2*es(M~&qcz|7(oizCr)eQim?{KDa
zPD6@+yIMJ$a#kZ`<z_n57|<ZJzO$1ChN$P70HjWEw%|hD0nYs7OlQyoKo&Y%F~|eh
zv)9?0K~I21l}>udFY0tYK+_5*UOWor`T~TlcJg50tu6#gxy{L=L3MwCF+-jDVBiAS
z@uZXDf7FL=fba!Qyf7D)7zwcQZf8ekM_qczd}k-lDFxWI-`SaS#sC<*Idd3{1!z3P
z*@YwMIDnV$c6K%M{C5h_p<PZr2A2aAUvPHgLel{@tIqBW?gi*N)~Scx832#fI`cSZ
zCP24S&Ylcr0W`YTna^N0fag_bF9ve}26uH90EFC_3ox$G>vS-x1iC{yX@C>eRRD8d
za`s`c7~t4P&c4iH3BX;Y&O*+iew82Q?8i9|0qie!_GhpRU}it(00zqeYA!on3|0V~
z(VRsLR#N;Mxy?xrUW(FH$a>!59LS&=VE0WYy;wq&t_En>+gZ$D4Zxn}P73@XH`W4-
z=<K{5AnL0P0K2X_^~k>oATq(JNB)NadaQDmu<bVk+%?<DfxpJP6KH-zrylJ00CbYh
zyNKQCy#RMS=^W05Y5+P<bdF%DhY6f@Qhbm4^$5p>YMk!%hXM3DjK6SO#%dsLM4e`w
zT!%jd_KFNs9fPl!Af%4qSH?;JM~5;l;%@w;(#*L63Ph86m)e{Z>GWAxr<a`+>Y|rZ
zplMd@tmX=&0zB}6b2XE-BJdCA8hW>YC~XbY_f_XwWa)$P?l+z5iNls`<P5YrH(CLn
zpboYBJ?Cc1d2v*oQd?LldmVm)aRgyA7U9t^{-qLcFg_vL3-RibEUz6fyelH-zMsqu
z!@(Q-nRs!~p@tF2u$U+;%`IpcVFw5|h>l?RVYL6tVnVUE|7B<xO~t4di5U$odNHE3
z(EIJ|!|nMYl>bru=(4eRa}B#Lu|4%meyHC6C<gDlkspR#f#?q@?DnqBr+J6vBMPp&
zf69;ILKXmvJ3ophYt<C{hLz?=bB+z*jqH32<cLxVce}UeTX;@K;ci#w{20!mNtEG6
zek_BA07uq&^Le?R+gPAhmMj=0;YxmkMmV?Ne8SrNgb47=MMZ|h*zzfA#yC;XbzeTm
z%o?i5V;%B2X7bDqIyEDmcto7{O@4C*RF5Hs{1#vqGs42g`K1aQ2r=Ga*vGMags+%k
z;sT!H!bv|!-tdp!sc3bj@L^Cb`UEMBt?A}u<d)+c{|C(6(tkdCZn}kZtcj({Bk{2Z
z6>8kstNU0nM~&Hp^H+cCV?~`}UIKejN-QU|L6s67GI;xRr8<29{R(FH>B@T>XnxrC
zy}n&gB<yj}<$ALUvp^P0zW(mP!gg#xs(PuXuss8En&rC-J3xu17HZGk%L_SH^E9^Q
zmBNl}2Wo+*MPVl{Lqm-BEv2vvqa>gc;|h6NTulQ`#r4A5n4KM<%_D_+L{5*Nbnj$Q
z*v&#UX-SrhsfD^P4(i(#9uvEj3|I@DU-;VI&kDxYJd~c=wx5mVlg@8G)Q{V|nmo<B
z)YdPavZ`tJ7Uu5PfR&*Exa!M(2@FC3qCe}Wuj6R!e=NNp56v~ytbKR)OJv^Up2LUt
zYm745M)Js%<Nfq;IvHTeoBf&~5NLbtWZkWu`nu6Mt^K{wH+Bc~jhUa`pY=Tt;;C)>
zv%b8%Njcu1_2uQwMrVIgx8+SzYWMd34M=opG=S&J{=Dj}Aw60}^=EZ!NRQFZ{;ci=
zvi`Ee{aM{ovg+{R-u^VNNLrRcS{1G7JID^LE<q{lKqTZ&?pw^hh4Z7U`wph_YA74=
zc=7;>6}DqwGPUi12sQv2H|6+%NH!W7w^h`DC<Zi59`x}5iWB-YdFt8$dS~_0Jo&X5
z17e6nHJSa{hyi*BB}?D6WdIq)*?^|M+UNl`{6~{}9;6yR5Ho<@(Wg&!R<|6WHzM`@
zkkbR$Dm7HyCteu9R;eau+r4f;LoU;VYCCjH-=R$re{FA2b$>(3?jQQzgWB0XMS|_$
za4p-X`ZnZBI==#nlR16wK_|3*iu0~#`Ys`ZT)_F{k$sn-xA6JXv3-{Y(w{!kcPVW;
zUWmolWX%5Z;H0!u3x;tNZM>t>`aXzA6_*DvMd`aWVD9myefQciza5Qq`{lk^tD`wS
zq>%A--;co3{EC8{v#a`kj7}~}ao(b9->Xa&OWK3wZUh37^YAE86!%ra5K!WF6X*-j
zgg}AU;Gzx0__QL7yrW!+yww;bfGSO{#*CCGAg!e<Nk`E@Pp)w_(UAqn>2f9OC<dri
zl*`U2Dwg+1vvwF<4Wl8Vp9jnS>#h{+m&Uz{^Q;}NrlGJuHde7$x?!{{4I9|;Y0x;n
zA6g+U!{VaZXZ&HbT$19_C!geFD=xY8$tMM!A>&-UEwP1y&hmaP-j+B<-nZr<7jH{&
zBtGSH@wP+__B`+!qm*1%CU`d(LYuMuM!0ywd5oAVS6#gOu!UGCKf8GMVG9L^X%D$N
zke6;DFP)~jIJ|8kFKyM^#o=uW#e-HwE}n{Rp;hRc!(Fr;&|nC;Y{)@Z4ly{E1+e9e
zi?##2_{U>F8Pi-G_-b-^9Iru<G3+V4&>l6%?4pTg!cy3G%t9A0?`srlrTg!8@%p|-
zQGn+D;-Uq<s8Lj)eQ7SN@Ciwv$XPC4;nyezpyVW%lk&VZiV<+t4i`^7YZMdEf@&Af
zJ!_N@ptt_%>Z`Lc1FdM~;@M}75(;$oh^wET7Y5Y#7Z+`~Kwvmf<WH^vI*I_=)Z69K
zQ6$jx(XJv!UiPkKGhA*xlf0|!W7j|(k#`*!<{G4<n1__|QDvf-Bt+5pk47ELSq5s<
z-HO>obI*j=ku|M_%R^b%<fAgK5`cu)$ziPfTtg|RnjGeo*XO#^&jQ_s#?OS0+3@H?
zFv}}u7i~5)q=g~eg!dZQP0(N?{pCRS%8dwD8PVl{ZpiblF_gu0C^VJ<xfApq$A!$q
zd~E}>YZBgVA{tIcHPNj1aGoOcY9d;Qnq<-##ino;SMx-M!8MIa(exi#6`X}pgy@mH
zk=5t*{j6;TuDKv><kr=?pGX5<K99f_uiwqRjQnn1$-wXC^BMTvd;tT$n=fQQZvJ8~
zR~3O{<mOG6yB0B^v8P{|YcT@~#@+66(GEk@a}<oLM!S~K{*$Q51%f@75?xK6C@{Nd
z8zC{|A!VjmPH7P^$t<ygPa_^u#?KZj={zI>b{i4tTEzvGWS~x+T-7?V1C8t9^6xXG
z0EM=8t<m#Rfowk4S|Id4r72+a6Rvf7CPl)Im9F(VYNj`aM)IPYBWv(Wt__?eI|AMD
zzH1{Ra}EWu%8RZ|2x5)g2x5sLXye4@NAc!C#IL!i<GTl4TW}t7gW}BQL9VR;UQtRd
zaA1{d8;~fak+Q=n*LHxYuXswb*!2h`B+aGfFy7(X!Q#lynFn1v8IYaF?04;AKz3f(
z(DfJt9`ste_7JGHBd6(U*Iovx06Vi?k8``%qydG#;Nm@unsk6i(_Q<SLmPk>pK~2x
zKyeH4_DSYIVN2`as$sP|5y*5s#h^2Q>tWZ^45&w(`O<X|AZb|_fMYvdhnO2R``g~L
zu4ftL0S(*f@-Zj?7`4uIm_Z+amkV6aG3XEQ$OYH)49Km+vt2JR7!0uR6W0+29)Nbg
zxL#y144}aw*GmjW0>~KG%M8kB|0DSa*HK2}krn-|>llMc0Ap)h#~GBzkC}`ha)QBh
z<m~9;dIcb9?o5Euzq?-JoJxRa4!TY;r~)XQ<~q$_34r#l>kNZs08Kou*9mxQRslti
zcAe#{wE(9}TyHSg05E%^>rDom0ruVQdW*q!fY+R^a{x)3cLutC(lYM)b?eon&Hhl}
z4-HBC8Jj5h9KYA%qBq%jn@}@o54heV#=L7i>>sZ88Bl;+*xL00w<CG!!}qx^Fd#2I
zvDEb;1M<>WSGzu9K-0O9A+C>EO*56b$90i&A^=u>=lX;}B*5lHu1{(G7xh&XY4lt-
zi|aBno0P?mFPkP@pJNX#X}<vBbKPA3z|^Hl8E>eG3Y<<~hMADapmN?m7S|Oj_X$W7
z=D5CM(Nqqd^J@Y;ls|jTb(KYvx{t1KU1JbJ<G*>G>l>ycea)L)-!h<xhw%DuX2-LJ
zqA0WLdlXBiXo7hC184CteA{K$k2+HHfbkRL*5IF`GmoD+i=#I-I$XaJkG{ylNa6ht
z<#9}gup55DPPm}d&+OvY>?PN@Sel~f0tDc*(_D(GLj@2+yCQX{0zw(ZphNit519(4
zc%r7U0rN6QQB0g^5D?>i%VBdVq0q?Q4Y{zA62_JhKroX+(|lYZ25K|Fr9@E4Eh?d6
zmlDZ{xWa@=6eAO%%`PRH5nj?pE~Ho(g#p2b6rSvFi2$PUSBa&}8Vjitor+r1yxu+@
zxv-BCk6+k_XhGV55H7xhbY28V@sBx${ixqSiOfeK1Kl;;rSOI+=0$)bj=Pj3Jy-Qx
z1k+zplO(-<#>rS%*pC`$7eE{X_b1wbJ1zI^wV=RcO=-#$0Y%dM^dQaY3sRb$rqJw}
zbWSHxbvm~Uk|z8BQrZzs8RFO3+t#nNU#$#ME1Aqq)y)hR?L8#IrL?D_E0Ec`hfCqf
zC`-BfPM4C+d;%@pkqes1PTOxlCT)SHJm*)B=A8lcye_4yUkDZkZ{)g^p4f#+OhCx6
zT|t?5u!cZJzx!QEK8mNj3b5ubm(q&?y5Fk14U!gBsnqj_7-S<XK~hsOgo5}QZE>U~
zVRM7Oz!2t5xrVRQh&G5pd!@sRkd$(bZa=BCA!(dJP70=2pnivcL`Y>B38;d|$*EFV
z9!Et)Ud8ljLMkh;{7J1c<WYbI!cr==%1m->R|?F9HdHnB3263KQehnL8&P0rGOaKk
z0VQUbL3k&c3oYm!si!LF!f0VEgoq1ckg$}3%YD-l3aJNRPXpWGy9*;|M`alCFi$Lu
zgd(w9DKHj>73vFJiXhWc3fcLq!*D4r^$82{nbgNz*pm1R0zvy>g}h3kXzLX{_7w6m
zh3BKazbI@&ty68_X=@>SN41HTt7Q)tP7W;fDw-8erdUqPzV+J+r@-bhFG0*xA0-r-
zNz65f$u24i#V!f{2@!!iiozI>y4n9O3TI%B_tNFtBJS3u6rr>2Mck`PDMCNHv4}f$
zDb4XR_Z4xqa>*2_LyLH$?;{)4E_ZHD5iQfxQjoQ|U6BoGF|{Q2fVn6R`y6R0SpaW&
zi|E3KD8-#P&~XN+7I~BN+_XK|EGB*$?t^iY_WCyJ<VZJ#CH?M{!Z5cUmMEeg$#?5f
ziDu>A2zPM9!4~NbZaB1+?%;-l#qHKN9LP;4oN&i7Ycq}iV<x&uc~MGRDlhJF^Xp1W
zDg5sq?$-AhX!jwkms{T(A~S_}-TK}TnW^*#x4x}FW@?`8rl6$n$&~(z*Fsa}^pOjz
zi)JZ224|Al!k)9ay>z{%Svd-(4t3LBtmPxYm(<-U<Oe)CT=ug&mHm#qYtu9Crj)~b
zPZPu4X$&m5I__=_KW=saG((<ow;?~~-N}|0-EA2dVBrek&R{^SW?y$_Qv9O{Hq==<
zxYA8wtXXaR*a;v^#{itxoLaF@7dJU?)N^?{3%c49_1s9(rkOhn>-lE2l{5)yR*w3M
z;Lw~i#qQ1qZSx{s8ZVMrsWZpjomg>Re?VVaCF^^p(3e-qH4(ZnidM~wbOYAlbOakv
z(kA>!yrI&aO9K8uTCQ@_7Gd*$kjiKix@Z$}qcK3s&)q#J=Lb9G{OYE)bPGun*hSvk
z+;r8Z`44%N^@p3T+JxMox#ZTj-E`H)98Pn|ouk}z)ux4&DqUuD(?y#WO}F#R3$5Dx
z2UaREfLln$);eru9A1Z;GVUQ_;eWK~Y|()eMo>F!sCK*i5dUW(YjZnyUu5aMslj1)
zAsVH{v(%ereD3Z?;Dv0xx%v}0niF4oPnmEJAn;0VeEH;Q!tEm7HB><~WD$b`fT<(g
z?uL+`TGa=rL3j5+%F69eowHdAzM9in^AQ9OK(FZy?E?+aD}5ZuYfcU1Sp@G;zUmzq
zQrCXnCJ!_-pneh=H87MwH3f}XhX(4MnAYyQmk*>)jP(y)xb%e%<QT;JmCyY)Q18+S
z$l3VgK)p-Te*OC81NG2KmnwJtG?1qiHI2z?&Pg2xQFKW^&KtyosITI^#PJi##GnLH
z;R3mQ^EU?RU6u4{(0x!NT7QeunE29Zlf|G!2C?y_WuwKQ#uygUk8=~!NSOW(Stlcz
zWR<--X^<VXEi=Gn=lg@WtFm8vejAj^m8USf^}-<Ts=Te)dHtX?W>0~o_ltv?F$kym
z=iYw|qK&Q=_=g&!X^1c;<I!>qh!KQ_<1tbo*?BrTSnO5^9m$I5x2QymKDbvh-k5^1
ztz{vV^7f}v6OF03^%khl{g~a~Zd4vcD;odBNja!NL<>tXnZKp?CKM<PD~@1w1i<X;
z#k%zszP?|~BR~zs-IGrj2LtM~lw!7iH3ig;XNvVJETKGX6;pI=c{42F?ycxr5^g&s
z+ts{mDW<k*<zm6!Vm*%fCF_ktn;s`0E2cP#|I(_nq_!cJU~w$ZT-=0M%mCf)<;BTt
zapG`dT(Mm@KhaGXW+|qC*6JM=Rh*7<eQZG;DWG9&A^UfnUaXJDRLNFR#jQDuTCn4~
zVw$aCS*|P57NFHT`UOMu+E$-)y+TRY+uLLDMt#i8#-Lo*iQ?cj63ffQG;h-fGDz;g
zfMiWhE%xg3HX6y+jw|lSS!B^8V~h0wX^tONQ7Vc%a}IT>aUU1cY!3H*0Cf7Ym}he}
zyo4HbHm#TfUF(nI%gd&VVv2Kmg`XD1`o(E7E$T~QF6t}j;3cv8)oFi)Db6J=qZQun
zpNbvm1g$@&p7Uf#v6H}63&QdYQQVur4Ldb4b&u9p*%W;3l<^J5(SiInf%}MHs=Xgs
ziU&YS8?2<g<a{oGx`ruYKzf>=<NryZ-$m4BX0g60NT!1G@t$D7`HH!W-}wdu_&r5=
zJhAu=z_xTfX~NuM56+X8k*7>G7MD;VZz=J!e^AVOf_w!JV)grwsM>IcVMJFN8eckV
zk|@55IZz*-b*PxO2K9F6msLE1b7%_CRV^OLfNnRPkykv5T(a5*F!W^cXwHd`pH_a4
zC@$sw@9KuMcT}~q_#Sl9wyo$Za~6u^;-nWQ&!Prj==!5E4t$Z_(8|3yt&b1~ZP|^`
z@ng4giXHi}tFseh7&S)2(8$tuzP}+52qtbDtoU$8cH5~;5`pQp_wg%u)+v@`evGL=
z^4liG%IeZWCmxT*(F_`Ur9jQIO-dXiL*A%K(}hWiXT;s1?OJfqeZV7HVvD8Bq%@Gk
zl|)shL;9XsCZz)Ewq46fb|hsS@tTww!L$T;N2e9bhR}>KV=N5T{?HIJ)*~5rQ=2t<
za!4q=E~A`0b*XJg7>Tc`0yq>gBpfrOjEzfZ-I0-o>SUz(Ju{<{4d4}ilV)s$H8bwU
zR}8MbV*IP2VJ@Q*_8j;^UOP;sMKzlz3@Pu-$!pJ(ZcYhB%JCa{9fB!lq@+*DqyNvV
zY+@P;$SL3E<<-&5yNMKMue_eYlp9Dn@>E_vQZW8ev3dVNQsh^8y@Eu)BBgL_UO_PB
zI%ZmXN8~wSFLOAom-Aen+5*{nbI!Yhr0^AaYI?B5KPwe;p4uu3v-drD6=SA~Jhe6V
zTah<u%w(W8xc>}pLd(Y23-Z*qgs|Lz=`yF`ILxMt6DDjb<mRdLghA#W=oOog+Jqhw
z&~^U;f*TA;Rl4ym16St~h+dg<dpxD&a>_-dERDWBJ(zL<Db2m3Z%-#7I?;I~ZRvUY
z!65H<kTUVT+Yi!mU6=MYq@@hIT`>iT&XKQIBkYc2egB}R71U98gd;iY4xCR}pAt{)
zS_5Wywy!X#!&xsp?&;O`Oj>HRqrm&-Lzz5D;o*E)Rwj*-`aPR>cFN>Yk|z>*<1%@a
z<jdO~AIao73SV*D;K}4s(n~YX+1E4mWk(VZAel5uMt#+U28`wVEtw51@V|Dn1J^t|
zvk|*9IeM$8Onond9KB1AOy0@3V5TUse1B#V8mV1KdoW*+natI~`HR0~+UcAwMn*rM
znL_8eIL{cF**+fhGxdix5=X=B%k1MLxJ^z?G<ZFsutvKlJ-GgVn}_beXm7#!;yh0{
zu4lHNLz+Ie!RASY;_Yb}^jJ?%Q!qphz%$)FX&{I>jpb_T1mS50`&&MuxS5~qX%0V(
z8Ae0ay9Yfq4vSKnb*&xY(Ffuf>Z{&%Ponp=dlFn+T_QYY5;Ftq=61`8KMLw|5kvf#
za^VTC?QXy92~I`2?ezrLb`e$&Z!htrYw=`{zR(LJS?%K>s(l<pt+~?JLzBh!KYQcH
z-#c1(8lXxYW|GMJCVCRMeq{S>qo*Oa3RUU!SDr>qLG}YxY0hF#VsjA8p(?#H!NX0I
zOEDhWBynYF_6j{|rXorkk|B$pOY_itqQlJiDX30cL=A8DagsVC(vty{{TfL{Lu6um
zqP>NrMy&K?v586QzB8V7tPe?ra_t$A)OiLE?+399mL2iv(+YCIc^y3Zy30;+uKmwo
zW_c~t^El~2|LRqA-RI$rf4+tM;j^Co#FX!cA9~$$fB<hzv|i?Ul5<GKG0%Ex7|=Z7
z(Ol0{3}|aL?>EoW45%};Gk6X%;2oI`_e1vMBGmINWM|XOa9#I#J-p)|kj*>(EZg_K
zhd2587UCTi&kM|l2J6g=9^T~Vweiv!9^T~V3nx1j4{!4Gow%kI9^T|HB}HC)#>0F3
zrMe>5p-7JxES@g3|D1gs1P=}IbcMfWU!vBy|8CE145+HDJ9u&#P-}D_?dgWDmHl%B
z88f?*EL!OwSh6o+OzAKaU;2=eeU`SrvVZ<d>OUd-aH-jIKa9|^kmjG$jh=Y``V6SU
z%N~7ALNlOy9`ICBz4g7J)}HwcNUO1fJbWz<!5W^OeFIEAQL&yC;M0i~RA|(dY%!`2
z8g&%|;+d7;SzX7ouQ$oFhO>xi`<b4#45(En#e3E<pjPd(-m{(o8MF5qk4AvI*rv}t
z8yL`>s4&N~kpVgD9WQ$}aT%Hu_4}9SVFq-oN<WilGXuW=eoSxA7Dh4DN8;}b@tmMq
zOaUQkagwwAwV-P*x@)?Zd-$3Qy9a!ZuetDCuJQ*DUvuHxd*<BjIm2c6qQ|BhTM2dh
zPIQI%?3NO&L(u;0dmYD@Byby$iOQo(xZAVYN9UI4k(n&?<lQBDWF}{*Sy7@#X5Kzo
zd~ZQXQ?TfKC$iEym*^War1*<Zmoy_Kcy_rUvZOf!;_zT&Z%GSAWU$0@CA@RYcknd(
zqogJ0Q19N?ri520+`ESlFQE;?&UaGp9#vk#8-~2R-`=f+HVix8UBD|Z?3Q%Cn@2=*
z9iBZ7itu%l5hDvqMw0(`{tbNEA1dK#Di3=zdz6f3*;M73j*?Oaq_8c!q>KUS+rFk`
zECY&0<&#RrF(7>(SYI-pK|=iKd&i2B2>>}N>3bl)WFmts^4rd{y|H8%<i@h12Js^L
z%$fJ{@=6|pEptkBG0T`4*%D%wGa%I<W(5PYE`~flM<vxDhCDt;wbMRPm*W;u@&xgt
z?l!xugrZswsvfJ)|3uEJkz^M62YFywO5>rlLhVBR=<JrEynEnxlA-Ju9H7dhhq7O=
zlN`8hXmG?n;2g?+!8e{97&uhF@<3`mdG}DhS({t9rD<rge*X)Z%rtJOzI06>9rl&y
z4sAs`)F3OZ^HAO)=C*n9>7jaqlL1nC4&@zUmeg_l&<xZoX)Z}>_5ILH0Kpq7O?wZ`
zVn9RE|KjS)1EQ?f|KE2&K?DTE0Ra^i6+sZ(KmkR?9o$7lWP||+aTr#W8O7Jq+*5Ha
zchl6=iqg#7_eyig>SlZQTisi?+ifpz-Rk%GJO_sQ{lRcPXL+9I?B_h^Ip-b!bB>xr
zSRH+h>KslYUB^WN-LU7VNhAT_*y}lJ66pi*>K8d`5=o|XPcIJ0=}+mtL(tx|oB=|d
z)w91-PKp3Zx8-C`ssOeeNS7u+I?H2SJ#)Ar9bj-5Ys<?SqDYYX-h{}UF$`OZplgG3
zGF1?O&$XN^0i<qkl`~dA1NCtNnD?o}a>ff_yL@^}&IAE0<ZVypOccO^u=UNEq{?3u
zNMnXa=5X6K;GWr9U8Ci&JG5KGO!?2tQQd)@cCX1%-9bv(KSy;3azYM^kf!Y_(TQ6V
zymRJ|+%IJHaSmI(Qpg5;lQUNU4+ITuj&Ed686u|iG#3`xwVZMl(VSmcsO40sh>Ch8
zKuS+{Urre$T}HGj6&V3J^GW0(#chEi0sw9`ZX!urbDRR0ldgSoTq^%kC6AT|<Zv2F
zxuQrY%K+vdBxDwn@oQCZj?6-hO-^PZr9fUZ$7=?cTd5t8Xh7>3t;wokW}hMZNa;9N
zM$M3Iqzv80nbtGZHj-0&ed`$^l86(~jNLO-*ORgP<xQS5xOPe%fuN@gXGpu3^J?en
z8Ep5MB-wgOCI#eDX`f2-m}mc<E5xPe^qrL}#3dWgUdR>VhU8=3{#@zU$OlHH)DfVw
zY-d0&b#$qB6&*Ebvp9x*nybcaw!-z#=c+NAC7OFTSB=?n0cMhiuH{l4lg5E$XIXAK
z7Vc?y-UTbB8@YqI)v8~@0qOD7+#!ON0p#W74rRQ~D;vk`G^H$d+pVW5UGW0ucFUt1
za)*P(Kns!{+M7$QjA~_F`{a%kKq+0Lb4LlFl(V(m(E`|ljowe?juFHrJZx5OrT})9
zN#40*1+cRO$L5X`z)|Bxd+vAvkpQDU%bg$~3c&nB?nD960OLQ*og|<Kz{=Fz$pR7p
zhJTlvEuatcAJ#K>s-R?m?YX(r1f&8)G|QbXARWN;_uL!-833*4<mL((Nl=wLQ$Qv_
z(z@I{0pkIBeUNJb7-*T?SegUNq}TX!UT8B2`>&a!Sbl2|*;P1mG#U~o1{AG%eZWiy
zgKi;cQ}3B&Y-7@plD5uNos>npV(U!RNm;b((q~qxT@NV)9`&9%j}(4X6z01rw7)P@
z^-xcP4Q}8q&VskFCiM~F#5{E1FWs@G8`?q-N8lhIZrCPXh1Z<dTIhe1Z6CZmlFGqo
z?IV%hxvm+sz&x+2K<gfY6p7EWp{@O?yEJprJ}s^5-nI>6u|oU5+&v6tq{g5{V){Oi
zMz90<(aKw|MRex|X3!mTMX}bsJ-i0qBCYc)yLVvnJItx~r0$Ubz1px5flEi`O>QY(
znbe(|Ma>$<CGx;EJuoY5>OL);%VpVho&v=Ki$Z<4-by$8MP#40ZjtwY=o5x*Xgcoj
zBYj$L#IDHd;X{$q9p<7!&4(hTj-W{1lYA&ruQqJFNEj&z*+k<G`gD|<hy<wY=+lYv
zrAC>naXZ(iGt#6xqDkLKT?YYM-0+#+B;aAte{@k)OAaIHZ;E^^&2IU&ztA#-Y~Exa
z+5ch7P=a?@q}A0eS-$BXQh@aLTFUYuRkmII%D}7NP5dOeKt2D5+?LGH;6#Ik+>$v;
za%o6*er3xJ*nSR9%n(sqc8DZkBab`SGE#EHM(#bSWk<>QcxECw2zeM*)iR3Bf|?YM
zBs+XM-fUUgl&i-^V*dNYs}(ZRYoS!cX80JxI(P*%d2N6o&H8wIQ4u}(I2h$T=hY8x
zHH_!wc_}?K7s<0+UdagQ{y!OZCx?4@arPYiDT0m`d*!P{<{HhbK%BbS2`4fDb*}V9
z=xYZ55Q@I0#apScxx2~h5ab{HDcrAh^*RN&|2rApUO%GdhNLM`ehO$5#cQkA&jK1m
z!8fi%loD^RAE3|}GOY+dzSCH2UcW%Qg=@XYTGlJxUf)BUA$F)3hNNL%1OAI2c>RX$
z_K;~(C@T@|!u#oQtr_hi*zQb>YdX)qn-|wcv;!Q)CUuBw(}AVgM~<B0WU5J(6;j+k
zI>fOT4DF71xf|ouku59u=-@cDGvMIS?T8Vlwg&7|eUHcS97vOc_~PH=l+J_;q?fAV
zl+J`(snUIMJiLgjkejVDru)Uklha%Ttu2mIt3Gnd*&nA?edKibt2njlBcs@MalI5H
z&B*YK=bSZk2T#i4W2AzZIxjCiR)B7<&bMjtJqR$xLI<^LVtgFX(772A@no05F(4{C
zKEa)M(`)gGDltM{sf|}UR33SB938LJryf#f@kSrS^KR)WXKwL*0pco1F6EE-egGN1
z5z<(!a7Hv7EJK30`1mA*4n367KE6LCs6vw6_v5*RS4M0}TjRNfS7!&v3%Bt8uX2sC
z^~?A)iN+0UjWIQTAW%k?T$n;L*D{vM7#zr)-kTjih@sIL9`S?8hh5^CALFUHReV|}
z$19z!l+1{e@zmF5tk3X`AI|vPTBYBJAHn#t5`rir1+d)~9*Q?c2`Q5*jnM!o46=SG
zE1uWQGcJ_0j~|B?J!~i%^Jt&=@leZUM8N&jC-D>LUJCb=-{U9J{TtjP&c#on`?!|T
zDW1#aVY9@bE`BUp_Rt-e{<UE(gvF#E<EI$lF>EM8`X-Hx=TXV9w;<Tg#Q16OQ>V`J
z{o<#KZi;Ol668n+;I06LoD<0oz=^wq5uekYfD`usMC(nEt}-?Qa^O)F_Q7>i1AH%o
z(QsbP8g2Oaa<{Jvr*E<ERS99?#Q6JOlT^(`b?SQ^A%8kk9NsMy$>PPRu4JpX@2gBW
zM7a6hKswy0CAYS@-oCdOv_O%(EhJR~_r0U+kJ&%g75Lu8fI57EEU$-mgL6egdn>>~
zVP@Z?wlX|agJz#%pT46>^M>5^?K=kJXXgOYY~HDFCIZw;1UL5g%@WXpO?dbg{JkqO
zdut;xS&ZmA4GfLpH-v5&yq5jc_i~d`n`a@}#lC)zgT{!%Abxg;-*%x#PCi5Zb_ifm
z8NPly1+a$&`}yq>z{Vcu;kR1=^B6GRZ;t@3u1EX$?FGR31NA$H^8B8VAnJF#48Q#X
zI9ANL;&(s*tz(Kx{GJrRzH{!d-$4N_X=e6v8*e|IGr)*9Fx+lE<fn53g#{B(Wk)|<
z?Vvoh?zn=N?uVE88Q9Fjz+6>~;t3p%ls`%Fn7AYw@ef2i{j8tI_njHQGMdPtIW{FU
zk6QSVLm@#zOr9T^h!Mi7;5>f;l=_Y1d94UwRS4oEaR4%MoVpe_|AV{`wC0gRdqJ*<
zJi~pz@9diNUf5h|BbC`jFVVVwgX;s<Ia;qoNEYxeJHXnXdU3<7u5I+#*Q>V_L9#Tw
zUNm;mto_;WEFS5-(*Q@Yuy-c*9tc15Cn49HJ2yqnSo>4&^gq$-t&V9Gz1~9_=sg3a
zh!!6khkT40m4Lzaw1(eR%XRz?*lxma*bNW-rkFkPyD+aAes|Y4$M3|8Uii&}y-G$z
zEO*kQHbb5p!(;gx25N`v?{foV#|B_B@?awwxgu6x!w?lkRLj`$5-(W#VC)0|p;|^{
z>?G3q2nkMn5Ib4X0(d4Mc8UNspvPu|Zd?Ux1azmW)BvD6O+bX9Wtd}gNb4uXVg^!-
z`iZI;4hq`n=F%qGXeE)cV=xSk?lTR2CpdN@#;sAK;0{_CJDu*$RW_Vx#pcre5i6i)
zXmTu-6r;ZtlbF~6QkLvH2mThDA^?5I&}u4TX+orC3jKO4D@_wkqW<?-Zr^Y`Cbb)z
zE`asvw?0<Y6Z=lfQ?aU^*mrhMj?IwjWS(Qs#10d{8O`+|b_9W#!q~>fjuc?f7Us<2
z*ijOXufQ<J83D0YWOmFb1oqe$TZn-Ig`uP<5}@PRs0uB1Hi2lMDh9^dB!rc?>9yEm
z0CkMNDmZoyL+*QX&l6tk=^e{{I)+tWX>^Ru=kiE4HXRdU3j`p;SSQDpcrluch@-!S
zbG_8XAeOG_TzV|E7Gu5ym7R&P+=i&>qGwcW1y)64zEr&>mRbwdTV7}pTP2z1bb)MD
zBR$5S0V-eIDQVUeu4hO00kz}f(i6Gp$s8mc61yu7(t#3VAW36?OjOD-PDnlWB~nMA
zSylf@j1$mb+l2nJ=Rb)F7+t-772}%4T+?;B6lx^)1;Z?kXR8k<CUI1fc{;&AF<Ah|
z*F~!m`wQUsYVVpjKmf<rzNZpX1hCX6j!#S#z$TKuJCP>^YIyDEn>bJan}~5KaS(xM
z9*lM~6Vr+0>6<O*5(f+5Ixlc+;t+`!0APA5aj1Y`tp^29%<w?bOyam$`#)smk%HjF
zvB+)Kbwox^#xZhZPR0vhPBv^woFJefCld`p4Ox-9o^q^$tjJ|gIaWbdrbsllc4Q@6
zKtooh5=d4c{4@f|iR)_ObOFqXeQaWmL=#(BpTdO1nds$XS0l(c`Ep_&!_d5k%&XE8
zEdm-!eU^X*S@Q)@)C)Tj3k0y#Z*NMp3Sg<<oR?TAph46k0S%(gCaCu|XTtb8(FWk%
z(pNe`aAL6p1)GOv4ABzj0K`>rN4L2)u|z_m0oLRu&K1yu>$9up6YbbsWnD+%Z`<gd
zSk4@sN3=7QiAo{Bp=QO`iK_j}f~#w4VinSi-TExrqp>bhuR{AD*9u+E^}wQ6g90u$
zq!(B<i#`uvTq`}jw5$+lz65z8;>O_?eF4KdC_P{mozg9v$&d;A@o_yHFu5F_nxMPj
zxA9%QupE>YVk8)Q!&^%)F3$(7<^5xPo+G_Vph8jL)9P)D{*Z)4r}F}xMSqxm=_%Tk
z7JOsTmvZ>-8_%hFTu;8SH?HS<_`P-!U*xvv%Mo`%6{41hS@e}chU<X-$1M74K{PnX
zOt$E26-QnZxIN#ZKPrK|7!dpqi@sh^0FWirqHkypj;X;wYkOF9oO^*oD9}r*E&4`@
zNx9edwdk7|wltb3!J=;^+LS&4yKc~YoM^vE18fsz(RUiaTB^r*;3EYVeV2lh0qqWp
zzFWa*fUW&4`W|89kqfvU!tV#1aNJ`S;4H(UA5fu707Fn-2NhffIC+jmKdj(Nz^XWl
z{*;1i4dCn7Ec(+5Zvb@kvgpq!#+w1h{l}v7j2y=kj{$nz#E1D5+=(lo+T%~^$65Lw
zdx<=+=qCi-C0cPpe}Tx8*}Z<qqMriU30FK5Nas_Fex?=L--I`sGkEn~i+)z<HunVd
zjIikE04IFXTs1TpBTY~{Bei`e-t?!2N4#HfnKVS=<ANtOHp~fE1grNo82&~m_-pZ1
z?g*bq1U31iB~SQXa4pxD8HZ6_uc2K{T!M<NK5x-)p&KODuw6N~TJ+n3*sk<@7X6MO
zHY*pJ$6Y~eR&`e_`s;$&td72EG4y)^*{)LCS@bsqv0W|MWYOOg#B0zuQ7LZ;ilj7M
zzwor^m{7Gz=CN4(H?!y;U_4H&@#N)`WzSjk4;AzT^gU<MKN9HS4QMu7^p6FaT8>5j
zZ=JB{e;2I#A^7?}i~fHq*dOrBFpK_)N*n-~|A|HaRG?35z!PY8pQ+Gx6sz<-i1j(e
zT8B{6T8sWgYl`JdvF3HM=wB(yzJLX1Ec$;uQ2vQEIyrftwCG<;u-*!=JF@?cB2C7g
zi!J)M0-KTXygnBFJAvMzWISedjYa<vjHe7nVfEi=(W#wLwKMvFMSoys#j$ow|FY=+
zl`z)MZUYthi$GS-jVz1)s~}d-bIAMu1hID7p#A<Xh_!R@4U7JV5M}MmeA{BuBrFnW
zTdl>U3*v}S8*MR}1aU;@Z&*wof;f`h?he>YU;^Ny=PV|^X)?*&2dLX5i^)sEl7V`T
zvY5ODr84FZ785U5B-W$@J^s1H#M>2#H5ov6Ke3p2y&|z@Bv8i<7L%`_OrZ9MEhaxf
z<AJ8!u^6US0w)6ox3QQ4g#9$2U5{H#xHkYYxj-EtOpu^iKw&2=rq+TAfudVmOu>SR
zfd+#`8$ouUrvoh}v!HSy=KzZ-L{PQH*<n|AW4sA7QZ|cpgV|ydtHCPvDfBQ?yFd&(
zRl1rSOzj2H67Y@V7E=d79%u{WU$>YdW#sUT1iZ7}Vxsl97T-Dw@R_9+QzwDmN$Ip4
zwV1G123ha)LfG`6{G$Z60NsoKu$a0C;!QCO*QTz5&`<ENgvHcN=mg9HjPbXaq7^Jx
z6O^etwhY;!>Y_kAoMKlN@6YDh_N1r9#KmBC83+f>v6$k8u;z#MAGpwB>M4-dt^#LU
zObLRR(?_;gOo@U#NXKvmSWMjHWuHW#e&1r^>JjJZK%L_)rhbCh%op#mn3CF23O4h+
zjTTd~AYYQXd(+oqqM2&;*SvpGy3S&n0w$`vba=*M;<gEc3BqhB>ZyX{GS`kIZ;NR*
zQcmrLK&%?@L3%Wt&(*5IR4gG4s|Hhv1i4LerzXj0+KdvEuZpyo$`Ni%<%+-ODvPNC
zdhG14nG>urOqIw9*BFSKz8*YzxH$DuwQ4YN0iuMxb=qR8QNlJ3;#1XwMC!N-dX%Z^
zL82zC9whS8&g-ZY<3+5Jrj2<-pHnekGrh#Z)Cp#5rqhj}93f{KfUOeXYy+?ooNEA8
z1m_z-HNk}j!1asiB0=aQdNtgaB;b7=&-EH;kC#;)BC8|+1CdT5tJ7x9hU6f<6TRe8
zu%2KsU5D>m>9$HFco_+%KB}c#30@JfWS3q=^eWJV<2q;5ok<qc4I!;Z0=7n}@&EuC
zi~<ZEVKKeN`4>1E_{kX-(=F)jrj5~i2-nNUP17u<yP7sVQ*nJAOSh@Jv|YNey(dTq
z6uxf|J}v7S@_iFf>t;%C;QN+>BOCbsO~f#9(PNPB+rrm0SsRJ9$_f7%Y(1%)!PbI?
zQm;*lY{fxi`T}#Y){4hTAyc4}LBQlzZ7~N-&qJi+)6)Yu&P?BlNC7-&oxW4e<M-iI
z7mvM|=y*;*C=)G({>8swmeIn@wuFF+hi19_v1k4g-IxlS=^smT?Vvq}Di}GgeGvSo
z-=$Qp2L<q%(Vm}S1p1~2?*0DJ!5^83pDRB9hCqG0ZRo)6kduN2cCT}XIp7l?#Wi`{
z!44(R5(MV1#YWc9Dwjm$do$uTgjK}lw~#F=Bd!Y0_d$Tt3=H0tPg78@U%3Iv%gd({
zDrXya7O~&uixx^wzgBe17cEp>nDhrLA{~-^$kr$5QfNL87G~rrUcu`8oy-pXm@lW|
zVzv}8K3^S*a~t#chxu}DE#_9we3P#Z^tfXk|5bil5s><YoQLwm1n|IMa!`J_<b&I<
z>HYJ?9z`?{)B5L&J&HV_^~P}{U)d+}Fl&2EK6*Uz+Xetx6nm69Gq?LkPe$g`xG8re
zf>2262t(xFo3F02KxY62Onz6Pz`fYUpY!FMTwYe{dp<u}Lc9^u@p3*-FnHq*?f+o!
z{20nw&n$H~nJ?S<Ito6ydA``Xh~NvR<g4vGH74bY^3|DtYk-7-`BdCNZgYWkzLqbi
z9(87OA>4B4Q3o?5%sly`xU1vcp*@e~i=|7QSn00WoiC;?b?tc&)joV6FR^N=a6h?W
zU>L@2aqs(KU|0;@7tkQvtRJW!X?77VkD&wgqbQlB=u|l=D64`V!3Atkxu1ejMO=Y2
zDABLnUQ?hN6!)z${sm}I$iM8dJ6<kei&E->*g*xnTdHqo^UBLB@JGn-5IOGY@_0bt
zT=cV<T@e)aY@i)2xzjRMgyTqHsT3oL&tDVh5WwQTQyy3*z?U7c%j1FN=zzJOqTB4A
zWdv5DdCts`e)Bo9FtaQC!#)Vafe5k+Z;zxx_yj*4_+N&WO2kj$?6eHt178gMr77a9
z?yxo7AE@7K^2&u*mzY5PwMH+K>tk#}Rt5#?cj;xMBEr39f%@xB5#EQ_f)RoGy(X{w
zNHczMp#Fw<o!7ks_1`c`<+XbrH4t}D{~gnoR!<O}q5lE>uD+dMXQD|1z}*Q9?itw;
zf%>l~%DfW9e&cwc{y)+Gr}9W-=D#@GuJ#KwnOZR8T#Z(I5oq!VMBYE-YV_!@fhJGP
zk*(edrV@M1xj<8M3^@S-QX+3+bmF5+d5=nO@5qgDUM=Fqln0ug!y-KIQO28C2tiT;
zyaPRW){=LZ>kmlcF-@3q@{P_2^q5YBTkfEl_ay$*NZ4ANH)L407U(epZ6l8(p+~>7
z;86E!phs?FQF-(`1+Q6i13hLod7Xe){Ze>QHmTEo^|e^y#|C;DVze*T_pZC8Se|dz
zgRLzfn}yFQRK!_*L>4x%+iR>X31oyC`@rfeO_U>4-iKBh0qfamJ6Bo#C4~KS{7Y8R
z!%L@o=#(`;2=IJ-*8yuFq8a*j-pz5Gvj#~bHtW~Dt*r&b1LWmdgGHQvjRUIXD=Fd+
zsLd=tpnF<F5MD40akHPdiW!fX1udItRpTMmM~QXTaKuu2-B;tSYCJ?wF|^&K)^;Kl
zbK?Ius~G6XSle@*Rj!uF6V;)KRvJ9$+d0<OZMSxmG#objSgq=spBW%#u2sfHnGF1W
ztZHoJkW<*ws>a4}fH8$ub-F9#&$*#iH8{3IP_F@2H8`?d%HFWXh}<0*@`g25KqP=^
zhE=ILI|B53!K%i_P5{f&tZF#y3~;vv4AGeJD1b5HR&_0{3xG$g6^Fnae^}=2s;ufH
zqZ>fNRx1~|nl&2W*=%cH0X)jSINz$w0b&4Nby<@nBo<)rTq{pvwfY_a{VrIQO(1o^
z{a&@ENMTV!Ter!YCZHdHDcfoc6bjS`PyEI@NC01nS=P_0)a?TR?wYJh-OjmuOt4j{
z+j-9D5oH}JX=otO&Ti%9HO)E@AYr`~haLb_A3r<JI$S_Hz{N7_2mylujN)voXyWUK
z0HLvsLQui5;f;;0V2U)hb8f7j<?6<XqV0bNi?%X#w?4<Ag%(1kQE<att`(+n7P!Ga
z)9%s2f#}6Jrv%t_vTzXdp2|Ua&ZmVu_c1Wk08Q~M<e758ht&1P9W3OTa$v4lz-U!Y
zreR<TJ*glsxWlhc-00z1I2v+TeWszmE-d7AVCz`8_k|boHn4R%yU26P1D6$AkSpt0
z(C+k6;ViJVUZr#uk%dM+(VH1S$2S%hV4$*ox@;mgU#knP5ZwBMzKjg5K72HC2j6kC
zPD1YDzl$gwN19j7g;uR_ykv_cmz^%0$lQOz40m-DGIGHu%y6$0g_9+kuh|MygTib9
zMu53^zE(I@K(M*GLMv2bVW^ZivTc1+YC!Glh(8r9Hw*a|jP(b>e@J}n`wE-QVu34L
z-Z8y>3wcgjh<njwx>E@%Ja^G;q<RX|-4Y=0awSIv7e2u>OU(}CZGS@&N0nB1K-`|@
z!pb>X;gc$f*-@$$9uzljtiyds+yN~67OSU57ajqvA~tZ(orOmcR+Nlw=)7vJ@ECwm
zG^%+z$~z#b@M(tS_!2dXFXWAwf*Y-f9Gwd5feMpbWdOye6+VYasi?fwG@vO@6h2S1
zq}58GfaJpCsKTNRtsVfONt{606deqBkc|df_ySu_#Z+$u?O0IwqKIisjRag8TlgZV
zXsJ`98jHPXloX@B-tCd2JPfx-xfHy*h&+hCg_x3y)O9;~Fy5L+StOQvKpxdqbWmaW
z9i-9z+X@T4qLcq$wCfEKF!8@$Q)s%0<$KY|maaKAt<ZxPbc#+QCA&q@HxkoJL(B!~
zg&v*#;P)}g%d{)F$PZ;dI}>GpZ)K4`;>}*iM%paCsFf72C)b}#V~b>3l5V+UK~bO-
zj4!?>Qmz#R36RBmr+<1DQEf1L9Y}<G7pXy(^t>J|QiCjyEzZALqy|~eJ2xwfl%<Ua
zd(rI7&FDpsPKMy|v%Mhi>=%s$D9sP^G`C3SY3`FaX2&A!YKPf=oWHG1u&b9Oura|`
zh*No+@hkt%_?18K^tZ}O9zXBj(>IfIw~a&N_^3>oyX6q*`oA(|?iK^w<!@*DGFvpj
zH8AF7%H%Cap8W@9szY?KtJvsOok>gVIj2DnZ<1!o#*jJ2`=Sc4o8uW$0S(_@XaKYq
zz-y%pf!PMYlS2h~x8xZ2&?yzokSh%x5(TN(8VbxRs6G<$?nP&grL6Kq*_zDEaRN9m
zWgN?tVaKW_oXiQ5r8tC~*_t_#ppNT@K}IHzs<nDfMfX}~${1wjRCM+C%qg<}vi1cU
za6D5+Br7MTey?WAh-6IyXp@&ILsC8a@Z4>gYIVjU-oHDON7w3Yg>HeF4OT7)$$vXD
zS3*VtEZ&?sQve&{<guA~0>%SWA2u>Af+hnU$<E~1j7N0=3hOgDG-D74$W6->`w43f
zfaktUu|T)Z0NCu4St!wR0nFDjiv-LBm{Ol9w==BVYxa91(<UL%lWKX+Ou3!GXVeYt
z%-YO345}+YQ2(DZO9T`G`1Qz~D_}OjStHXf6p8_g{+20w4eK0$LG3bSuVF0#h{(*8
zy@r+MXd~NYmP;BtK)(+%D+H7ROuCgR=VDgF0c4t=DF?$=-uZDo$gF0(`f_eO!?L0>
zVUwekaDqn@bpE*sD$vjb1vEB61sa+lfx8J3xSOC;zJJu%1R3ORf<mD_k}_nkjLKX>
z8Ez>V9{#fo3N*+dpiu?|8e|}F%RtZ|gK;WT;x)=Z;Fdv!up2yAoVgT*QgTa;&-HM=
zZ$lx;O`iBeQJHHX-&__m%IMKQ%cnqtd;%Kf6JW@zH|VECZUG70g(UGB<s)#*r$QR!
zTL<~(+Le6w;aupJZ+FM^%&nB~QzhSHQoR~LjK7ijxF8C+>&?vV0=QflIwW(40AGMB
zS2Jbi7NZn25ShB=YRhw1GC6f~nTDO!w5VR0I#;Xq9he1B6nc^>3m^<x4Tw8Ej9G$U
ze?SE--dI+FR_dWq9D-0DKp3|Kp+JBzaH*K})R|-85`-cF!pPMS774UyOQxQTsZFbk
z0@@jrsrM(U?+WzP7n$<zmS)_Kq2M!*WoGJJ)|ReDC<V{>i{J`13NFYkw?d6_6E(?A
z)FijWbjwZTWw-@bfsKOm;Rhu+<_DcGKWKFnJn2NHE|(1MM+4=o&eYW{g9M<9%`$af
zGr-e2K+LeZXOIrW49h)(`x#s*Kx3_yZpVa-_xEM0N!{+5MT_XtJE-nAj%O*Ylxibc
zN-M?5)$?wa(n^WZ@!8R#Sy4#ja3bjDy{s-#q{>9|f6r2zEw-!cSy^hc)eOMcpCuPI
zL@RaicOy$}w%FtnUe8jSE%w<7?X$Sq@@~ngYxVprZnpft;GTB#S6OPa#m(RI_AFZ8
zDxKMZC$rR|fIC6^2U%)S5W+F!?6mMK?x`IALl8vm+aM~|9~*MA)Sim6&hC+=_EePh
z`ot`nSSzJavrn^V&ZNnG)>*T&XwRfftI{9NQua)tNb}<MEM?Bb_j2|mWGQo|FpUcd
z7t|M8yln^;5@iFiY}h<d<ND+7w_Hj1f6J9b%R#gwD=GBE?-ToJN45}O$%RHCMqVPG
z{vg!Gb${6*P~YCnHUaw-WeA&>XQNG)GDa#(v`xZ#vg{bM)T6&`GDE~j<dNOBDFWE7
z=U=xO*%FUSf?hA#rb<vSl_zBbv0y5D?gJf#7dkM)mIJQk-4$0FdAYgH6u`w6xaI-i
zSvLT1wFouVEV#}RK(75>wdD&SQv)*4Djl0GAc$a<Ai0D$WLt!7HraA6i~7YTMk1nH
zKxSzqqB1+6gH0KUkZp%gZ8Q?;`n_x=HfY{92Ptp?h2kogL~Nj-P(fgb@fL_zN)Smv
zK2g3&3w@%Gtr{War`0Ju^vUI~x(C(`0iHHaj^)228E82FOx`7c<>DM?TSywed&<%s
zG%EZWY2f{ru@Sb#AX2duQ6b}!KNES#O$3GeFgmWgN|!REu}YT-VBz*jvpph!g^RLX
zE@>Ldc7=q5vU)50P_LUSy$#!H@;-;y$iP}gXe{AJ1uz3B;dPSKQ`@Z<=g-jsY;{=r
zk{<-#^yfsI-H9OZmOl%;RVc7E99?GHCIGj+(ElpV?bfU3pr*EiCf7%xx%1C7ce!ao
zM0ucHRA8v5JrY92QQ>miUVut$w80eCOj@Nwl`)8yX`Z(I@Oc6eLE_0j5jSiH-PD^s
zYvT=w$|v0QRQX(EJ!$Qokv8rmtJr$bo{uq^XqEe<+nyHC(EgqQK>I_qlB2d~B_sl%
z>vS8gw;_FGql8thY#4vl8e)ICkL_<Hk9#NiHqG`n(@9Oda@TfWh_EJJ{l@l=0J)PF
z^NEk`eI|0ML?8S)(TDCt30G_%H6?1%x0CJvnCKarr5l><6C6NSwQ9)LrxKB4E_%{u
z0A0VAr;V&_+uJ^8s=tXbyX^}m%3vbI`<KL%x1p-OFx$RDyy{RD?>}zh_cCl>yNN&6
z+xCqB!yhD?t+RbAD2PLKkHco$|41TT<?1&ENl~~S*nS7-`n^f}5X0%%Azm;j#DI!V
zdL#N843pl!acj1$iUaD|KRBS;cX2=+eu)EWWC90Nr!NQ8joWCL`=WvQ3-kZhVNo`n
zo7L*8>3sdxru(Z5V(MHKY|~qbcwCKaoov$sBAG!s**^1QE1S+au==-#gkAoUu&X=a
zp#3(z8<ABKKDxfWO^@OI?|J<iQpWxzWe;~szezSd4yfxmNqP5Zm`&%@Hm|H9Vd7sB
z_Hrk@d)lV=25P%X5+=QcoGfC(!wm^*!@(bQ-;m42T=3N~m!6N<bh&_vribD=zroig
z259r%Xh^w~EG6NLA8h(Em5@=N3O3vH6=cFQ0hZlL2_%no={9{8f>?ow`dV$I&5*Z6
z=XY-)`>4uEdL>>Hwdw1i@SXRD598J*wy^1SNIRc)6rOi&`bLr!^Y$fQ*z`@pgbTC|
zkJ<Fih=^Gm=+0T2zD2^kNXGkoqD_B{WUeWWhW@xaC!Vj_bhQ9x5!`8O)71*tAE>sU
zP2Y)_^Skqf(hgtR^j#ucIH{(86>ih_g6e{>hJ5UE6U9)WKOv$v5*3A3eFITLKhQ+=
zQ=9&zP;ozGxS+clfWTz)$q1W%n8Ax1m^|fXLdlQ(g$Wg1c%{EV@?&lu*F0?c(;%W*
zMe?4;Hi%wNstpCAKlc}^&;Nz$@xM?#p{TM1h8dAI{Y8+eiEa??)L+QF^cOOxo5&a_
z*E4SEHe}iKvl7Usd@Bap^m8I>2l6=e04?hxc{H~BOKu)>+u8KXZXQSfY16N`c^tCY
z^s9n`NG5g|dhE+2a}uH&?CY<PP6k`(tdlnVRbk26BGns`vL#U5L7RS4(kb01=9rpm
z?&=92*x9DvMzFK71n#({WA)s1OUH)wdZToPw5)sX5~%EG)8CLlmcS#YZ2Fs^>Gi9W
zK<q8l-rM9soALe&Z2Emht)t|F5cCd$pj85jT4mGUm55652U(rvZdvD-d)o957`%h_
zFXZ{5@Fbx}F4^>tIxtM8L1#3aN8Cy0jx*WxZ<+Kh#H9$|xkY$#jZMe83qkI0emOsN
zr_7!LravLrMT1-N{23|n{y&*w644*vjVV_v#q<cA?cGVo9r3oAOiVhJ0`9h%Jlx`9
z_BDA5lFt9B&D2aHy4L}&*={-~S9#h@JQ{Y@fevP8Qwtg7L<G#wCLalNPyepH?xYjm
zN76PJOI<gSG;O=hBoAXt(NC_hndD(?T+d@Wy&hpRaV}Xn8c{3aY$l!`$5k+&uUxj7
z<Up>Td)YQiY=$WSftr;Tc0TiLra(b$fUN#DQ;;BX8i-7{7DP^`x7bX<l4)*&=f7?<
zg?f>nT-I~#g)^t&@)YE0wgU2WMB0U$5V>=x&D7aldL;%5A?h#1*9C~nXQX^6FVbe>
z$=Jfr8&bwJr9{G5cS0<uP0CcjJH32%zGgGUF-+zX)U7FAi2CwN-kmUWFY1k1e`ioo
zRBOyXY8Ca=7eU>iIuAV0RaE@?pcnwPzP<fQP@IHty&1JBC_zAgHWC|cb5NoO#gjAO
zvpsM~x*}*Z!e_rI$Fs-av*U}PEvPp!sPX7`7(Sgk2W{;PA$SYd@aRXoVP|<MHiGZ!
z<v}}<-9=By;w5Mox}~eV`hrQ&ZV7NVt3{`0foQyL<C10jO5dO-F(1@&=lbZsgXB0u
zmMqVF5+vsl^4P-bJ%bLjSVYwu_H58ofoT81A2P>Nw4fu9vKE_Jtj-sm=F>&BQ7H{S
z@&aMtdFrq=2f}Ol!d@*t@B@zc8`_|OXGm{kaQn)jqbQczp)8h(p50*sJ21%O5=%jy
zKJUfpv&X(5kIT6K4soRPUV`7XPlK8*=bB1+otlr4JtnC6D@<ucKBsOCZcR(a+LvHu
z)SC7()X9!Oz_!+Aa@<>hLy(WU2FM$Ut`kIGb`4Z^LOet1;vZQ&2?e>B>criXizgc-
zE%(Q^cooYn2YKG@mHx$Y%Rv-Qoqj2nTMlxGeqwC#G^T-l3GK>?dCQ^QBH1sdtXf-~
zOXAcMp7^<VCgaHxeQn2Lx#S>EPmEnvEXRiG{)Z#ISS~imd(K0r7t6&4VViQWSS~iy
ztIJo#MIg8M7WepLKP$Eg?Ii&D^~LJ;!ZLtLTd{0R>sK}|@)v&~l_Sk$Ug45rI|MZL
zAu4iT>dW=N<UZC>>>x?WJ&K}C05@^ZeqJowR{44u%0;%V^^pj1Jt&r4Ykid5-Yw>?
zHLfBWA%$NQbJyB<52tt@fOpIOvYsxUFCl~RWKWe=Y%CBo1c`d=E3OeR6hPlpEW252
z2Ef{Z#oWyr`-U~<%6Lw4HGqvj>!fdSEpzn`MC~-LcrnIt;~(4_q}?qxIskmh7CvNG
z@e+b><O#6FX=uTVKWK&g*Z!S7Xk2>nQV>}}emfJ3m%%Ua8xmRjQ}H9BAV>g!jLQ)V
zPngT4rD85LH90@X>{+}L<d<Y5vzHjpQQxvogiP5BYqp2tu-YG3%H6@Fr@1=_istTM
zl`)vxgO?_F;&=59ZVyKKb9->4Ew={~{^yO~o6B3^_t*&^{2K8s@!M^IFMcn)=7--N
z7JvK}G;f99qLl&o-Ml{#zY|^#!f)SEt?|1{5617|4Q=rI!Xsw<Hg|;JH>sl$ihmvd
zXp7%2SHkc+u6H<oC(Lb!-xt2_fM4I&Bk_CWt&aE&ncWG$aj$m9Z$x1f&p4O<TaQe6
zALZ~6&&8I0oALpyVjfzjWf#q_)>1wMNExJS6H%&uDIWphgF690W9FuO48-s61ZP{T
z@X-nt7K$+6^(p@c6!@wx6H3Y;5Zo<gDK=0$-Z!OuMuykHFd-)8A5sadtICck{}iCJ
zVQgNRnes2dhxf@F4k=%Prg|15AuHuS!Y<fAs*;u|UkhU2T6sI=8v)!ir%q1!mOwt1
z_uQ{3-vK<dyCXujJ(KdifX)EhUr6~0L-oV^qWP|0%FlEk>|rji)KVVMeKY}2x@#%_
zrMtclKJl!j{6hB&$vl4yzMb+L(dkr>O88I8?*tdp0Xl@H{2?F%)8B*^DY`~<btGF@
z@&^3$Z_OWG2N~c;*LQT?!_Px;_(^$))~hG?=&30>2FInnK_%_26y1}8$I(R9dy<i&
z(~7BA8=W?)wsR@E7a%I72jIRrDZ00U3A=Q@<dvegfCz!PdLIODwWR1i3{KVMvt_{_
zf{B_7TJbbC<fjy!_UudfRY*Q>vUFBMC1RpjIv!TC+Km<~QgnW7V(F}HyB_u&W-K|C
zqIU!yM0<J-F<7k{q}Mz_@n)A4eH#Smv!n-F?cE+*#VymvUd#d~FVAU96=xOce+Lgg
z`<yVc><f3m(K+E}xZ~lrel;fo^q27zc&>-<obCkE&2}rEV~X`cq*PqCEXPMO#f`@;
z3$!_L;+97^=X%cJ(c-dhNPc8!z?^<e{wRFrwwRNQ*?-w|O6}2p4i)~(9+kHOmwABi
zk&HR1$jh=KB)oYiG&u~dbJ;J5TM?HW&Y9Ve>Eh&5$q_^{T}*y0xt#!NbZ(qYZZCjq
zk!PxtI|$$~J^OyLXvQ(if%wpEp~+o9{E?mrN{C7B%6JXeq?5%eQwI7sjwg#Uyj~8t
z*Z$kvNRDHwLK&%(xoT1q*s9><o&r40rTOJrazX@S$zo*LFK|}b5K|7Lo}TENOr`ZB
zmlX|V6vXlXjRA@Vz$-z?DFkvDHS?q7Q~@p69rIQv^MQs(F3bDj!JotVx|@99(12u~
zlPqtp$PX2Y4dgT2<O7E%4<ooQYS@&FfaK9kl%f)i`E#O7cOvh9CL38zsl2}RPM*kA
z<x2QT?o{U%B+D_f)KBw)$x{%ryxhGsTW%<EFQNISlqXNee6;*g_SxkrUO3VGI{687
zn&tdb=_5VM(f{uy@Au>XACmVQ?Xc`!(ke1pr&-gADD>D7<C1liF)rY5uSwP=XZ74}
zJT*31A1-3?5UXTfvOYo(trtU(ZloYC_50-~>kW5|<+?<&J{n<MxPalXL7~Yy)%q)p
z6^KiYa<qP5JrkU)%h`H8d7#DU6Wu&IUQX6ku{82f<-*;>p!v!A6tw>pE5wwC^|)fI
zDj2Zrw`ymyK8-BDMBICwllAH7obg5!GwJtfvYvyEyyDBXKye3?^%*h)Zva|(Bw5cD
z0X74<9sqH4seg<Gydnyu1D=ma);|I1m6H&I0)Hy;N%`>^$@*u4SbQv$e+XjM4%Q~?
z#y<(~`^xjk$@=FK$lMrHll3nINr??T2Oi&%M>zvMlJ)O95{SwBif`alW-zii?X|Te
zkE2l42-Hzvi3O%vG9s-wTQW;dWZ-|+za$@h%FsBbEV~&{!WGTR@6hsULP`u2&C18v
z$bBM8mI&aArp2|AhXinB$jd8vSb!M$?%Y+f3`68fu6d_F7f`}=%Bpm;V_vCNvdT?u
zRdC7bKT}&{#&opuF_LTjXvx|paxo7|*7Ziv?<D~x`!Rs6a!kYUzpUiR&hY#W?&tcI
z9E>3ge7&#a77i9yZ3L0T&Lwxy)RpM_%S!GNto76~nwQ*ToPQxs^jjrwqCi$IVfNM-
z|5x&sARhKT^hn9y1W@zb`je8k1$Y3o+gNg60E;1QNXa__nrS88B{wPF??w1=@Mj}O
z!}{u)k_{+<HOjuAWRtrDo-8Wa>MjA_b|uC(K`eplf0k@lB>?c+2PL}%aH#F`bje-;
zJl62*U$Rd?sCk}Na){Z=<kDe8|B@r(-vi*;{F0*rxQ6uTTk?#6J^=H+D0x-@H!|Hk
zjgsdDaVfCsxsn$IqyyMKFF7TE>r=<$C1(VT1SmgHa#lblfd9^ta|CP017zJVIWJ%$
zK-afRE(n+mkn>u}MFH6W&G(gDCO9yS<{%IMUUG%#u~`VJ|6j>f0fhjr-%G9uC<Yk&
zRLS)=tTsD<V{XX}0j0Ac_P@k%qs8HXk{?*+&mlsy79~GRQdV+qc?quru6&G@hl2f8
zfVXU?jFR64$%}+jCU2PAiX`4a#Cx6R1|X>M3dh_)84=cU7vA-!xvA*<4Lj_e@QFS%
zHx;99gU@5=U*r4QJ9mMP74CcM>`~&*u-X&Qr^O~LcdNZ`7jB1S1|I%*`v8eT8P|`s
z4-&vL*_mtY83K6fQ}lQHC;>FoAO4$ttN^y28O`jI1aL=wbDn(~ffy2Bd(}Qu0H^S>
zL-qmzJcbz6$8Ho0Vk>L*lieYJXU^-twO12}&BNV5yHfzytCOeN9}>Vum+`TExul5%
zc<D|1qY}~yV8jCZMgd&`V%pdr7Z3xmKEu9S0GBj|&1Qd6P%oh3Z|p||&?ailBKxxh
zbprr4C)!U4U_&i1>@NwR-EHjy`*{Iuwa0AsD+0K`+dRqsiU2m@>mS)~2^bF0vaS7|
zfRUX4gH!Eq6V;Cf==!kz?*hgGbn9vV2Z5ChI#joRDIt>pj;*wRBVa1P^fC4y1<*hW
zFDv~bz(PGiZjJp{3=nJEEYYLwax$>)c?{q;v+ep?#=`mw@cHF-y-pHwmWUf^*S8Ad
zEKzyVuJ0hK<0PA&WY_l!qN@9Po?Ta^$La7^XS=S7j}u7m6Lww7PwX!uH`#S5z<Sz^
zM1r4`V7(Cuc<3p+E@fCBrOn6pfWYwrgkI;nMEA6dDtX<Hh!zTPO?YeRqwp0kyMCQW
z^siTA?E1?<Yku<}3Fj@l{t8K4SfV$x>u;mcuOHq(;{KmW82USkMtVkRJ}AB`$OBPH
z@jalmcsmLdCv~&y?}OrclTiE@DQ;_^DAiKeDAd3ILhvg$!Lo9@{vSa?aPk*+Tt7zt
zTR%y%`2#6_Fcz=1Y_;pEI_1lLcOJ3pKQWp5vLCvZ{<FaLCdzdv#jZb~TnCou-R=5s
zpt6A%c3;0_*MCR64XLc)<HPOxA4Hj~sInRiVnFM+b)Y(W(93p{PLftGzxtlCn@kM+
zpbM1`>waV^&gS?S0`U6bz`25j|K57L$%Fj=wFgs}-Q-OHmZeY3UjWbobP}dQVTwWb
z9lOZ~@ixraqkFcoi&C~Og4aP@VBd!9QG$FC!)^*As;63gbc)^7K@iWD>|O098OZ8+
zSkN`VZt5=S*c3Ca+fDI;n8SG(rh5zG>CwwMb`!=rWRMN>n%QpRSXUR(Mm|TiVHw;#
zBu6?Woq5}CN};56OLUXn#N~3`9@L9pq}`MyH09EJg1_A)!(W{zFTL*^Z8wdRP@R|F
z%jelm;}vWHc%{s4$`NSt0o-!VZki#`@bo3mr9atCx!_sXe~CWKZlWRNCU(AsAX6j}
z8_TAI$XzyNf*Q+4jhHN(0}#&1Fe}UEKuf!6u^^rVJiX6uT1r%hk}@=pr|qVd0$Eg!
z`*xEIqIE2)tUY!UN71@HREf{4vzs21m{fwNbhn#!3kn1BO17K!3u6Di;A=M>7R0Zn
z*G;mUo+heiXFuP}Zc-xwTbnWMuHAG-0;2&pqTVhG>H(DJX*azpC;{lo=XTQ#B5NO@
zZO84Vn-bO+DEMK!=`}(9fd<6dO}7N`6Q2VT?55j-2D7MN46>W<u&9r58Esfm()@}8
zmg3qTJ@PlZ$37JPW?Gj;eP#D}f(hl&;b6YqL*<YS^R>_H9tVUdN6gdZb`LcUveG&X
zwtF0uFpii9Keu~uRNRouhB@%C-Q%dxq@bI-8g`Fk0#g7noO-B&qTDT7+dZB~*ya|4
zp?f$z-tKXnG3z7008FZ~d#F;3{0eY+lHKD475Xh8Mq-bX3jPQ<afaPP4Y!g1Wgo7?
zN_lfzm1!@Q$=(dR#~GB#rY@VXg<H2bwH*RmPf$0#%IViUFtr1859>7Lr#X<7z|`(h
zAe4ip$hzm@5q&H*h8}a_vAOL>NcG5DsdB1b_XpqWUvnp*l!rl^f5bFZ5m(B4f;^4_
za`h?o0a6Np82{3ih^Ds6Pi2>?dxBh7bi7>3a~>Gb100%C%5xsg+L9ozlq)Colw@9B
zDGzztVKW7`1&$u2Z2-3%Lp-pRi>AUAjEoSn74w!8uF_B;!G#Vushg%;=p1{dGz=R%
z^<AjYo2B6ZTaIzf68>tbQe1E`a^aCuQC!q<F%t1psVFY$xESe`U#b)rQ2@<Glq!?H
zXaM7xic%gGX;v;xAb2MMv8*QUAh(>AYHG=hY}poEN)x@UT@dw+<E8346pL?cQ7P3J
z>PrFpXO!Yl1Ax1T$9I+X6u@1?hU!u-|5Pa-8&R65o&)6a4V-%+2ydyA)5eb8rL-#D
zO7m-QQs#Q(lo(j5%=Nfm8T(Bs?LDV<6XV3wtG!EUmbdjOl_*vCHi(xdfQca9Kmiob
zWjrWVcc3U={r9EmN<Io<Zs}c0rNB0>2O&g;n_fj>=`c6Fo<Ed|MnF9Z<|-W_Aza3$
zZYUiofXmp{<4Z>gVC|;NFQs|}riC~T!jKf>F{U~2Hm>wG^&aCTVU+xG{uoYZTAk=u
zS4;{R!<M;imFV)vsD`QG8T0wt7}YSDLs#J#e=}n-hmN~rRKw(gH!p7t8z#P$$@<-f
zKVzHve7*bBc-vD_;i3xKHUN=M*^nS!D?k>~+n9r?`*S|PzPt6CZHn7^VU+r?&6RGv
zC;|7dQEjebo1j@4`?EIJ<Q&NP1JSo_t}FD-WpvEoOKtQhkn%A4Z7zk`<au3lG!KAc
z_eDrKVw}TEs(~eNdyPX%KvX|d+Bn#0)iL4po(|P#xvoE(;_#Ia_KG3*9CE`#7Vq|+
z4u1*Z;=Oc<L!HL5zl><-2vo<xkw8UZjv$H0K62=NM{5D>BV}n0b@ti=AmS5;>c9yA
zTOT;2|JHM7d1tmGRMK!~dHZumTLE0JZ+yog%0~HS#Egv&)n&OVuP~l+C}rbFpvtEl
zO4*nR5cI&I^o!#Gs_Gmag&a4NRp%X@1YrNDRqu1C)7@Nv)KQKu5;6;*Yn?;1koAQC
zoeCVHg{&_InBUwHEz#^$ho6sd#0V;9&=H3^<gF%n(IIEN^)&zsjyU9qw|)`8`S%=3
zJ-Gy6&jE+(*2@5Dq8;egsG^ks&S?(zYOQ`Pz~ae{J_0rXTz%cqm%yms3>3G((NBUN
z16Y{oND{CUU{F_wQfTf4=={1vDKrlNoIL3eg=YO>0P8S^C^YMj0Zd-x7$^kl0mhU&
z1_?L;Q2o6_b?{RFrB@t-)pG!6f#!5_P_?=J#ftzhH+LwV-vfXlpF5Pw?|<wxm@l;L
zFH0Zs;|6EXdB+IMT#p52q?Pe}bcZLB-0NJ%_p4!00T42-EF_Q#lL5wtl;On`2%M@$
zSd{;cV4;?zWy~vMCGX&7D_~YxCpU$A9m+ZjAccPUWl=(o)wL?Ptc#GV)-p26SS34H
zy5OJSPK^YKZvF`|WxWKXB3g`pnbBJi)JV=U7TS*Y;1A}5+{wW_-OW5Mt!%KHdB2%u
zLj<s`K$4*%NsU&rsLZz+R-_%Lp<Eu6wZ(|A{SCN7Hka`eRXc*=Zr`Ph4-xNZJ`v#9
zhHX7}c!nCE^CU17r4uio!MgQeNf`|^cCtW5`;@hWd?<dXJa)Y2si`MA%f{f8VrNYP
z1o)Vze6QE?lyC4!9`ddJ%pGeYV{L@rBnM+nD==|DOB`vA<e=oyZv&Ar*(@LN^ys&e
zaF#hrjadI4iS32O+!c9hI8I*zpF79Q(M>*!u>RO~!#p(-eE7ZXgt~q>;LF%-)~k?%
zT`^cs-Ro@Dt1&J^qXIPK?`C}-Gn9%|f_DB@AcKXOx`7Jf@DGE{V$ZY7iqQUp%=&wb
zd!H#|{LT9Ng1Fdw<G5M>Kng~_=f0&?bF=7TcO7YnD!N!%L3ZhH)_IJlcC3l-nss%M
z$KB@1JadCCwoX3wK4VV0*_27qzeVV(V6!O;Gc8IV2%?+hLj`qQd*<btP4cCJI&K5P
zo;92Jse&Dz-WspW;>0D2rYgHbQO^+(QC!sE3%{t(O;KSG8z20@VsOrvQ8?H}Uyyt8
z&d+(48|4JQ_>d^n-0sP!v%G^m%bj<I^;dO}wLEXE%vwAJ+g&Kl#zbQGGDBi9sqUW4
zscOOlRAn12BvDo6$3(s%y%f4%4ip;FyBYsSg&2LBjl{6<kC48s*ZVTl-36u5$D%{H
z=-AUvX4;Ts@Zwja3-T+q5E-3P+38mWhsfw82R$L<LS%H3&(&R+9U`MsYAB+;aXcgq
zYT4Z}5dcSj4H*cHbgOqFu?{e_ggw3?ym-B57{aE!5F&>ya*8zn|3WelF#yvyBpUN!
zbO_fOd*+%U!bk+{kq<TwJru%^7YDo>#nELC9@5yeu%TcIki}skv*oh}6MkUX;wtB!
zT`7!>AD|v6|2{oJD*WO9tYjX~<!F0Wq4u(lh8*{0BveL+bs>85M&MB$qDKe-KRzkE
z=U>9lk47zf-tCTqvBD62d}jub<oNeN!6zU@7sCURdwgw(sXf9+{B)(4-Vs}Y?~(PX
z>(V*}(9QAlQc_xH8D=;b&weRQjeh@|h~a$Vwu(KT!A8A)H@nDMqos94_ky0{&FrE=
zB5W9d?q?Sj5Jd`lKf9=^9H@(+j}b-`Bj{6v%>n8u=yOmZ>Lcji*+p|0v%jEkz&-Tu
zX<Y{r8Tt=^cdOF6L1yok`U9CKtT@i~kiz)cjav)8qg~NaGEa<AIgLT8C&n<R$Xu-C
zl$k@edkB!T{Aznz9OKQ<JrUMrV_H1J?(5~*m2(Wp#gEwm)?`;E_64FY_P)LZs1Fc|
z7ic9=Z=h&F8-RKN#R+;0C=sZapuIo|KuLlQXIJ(IKyv|+`s~VNfOG+;vMX6aMo%Tn
z#q3HJUr$!Tef^c}N)~NTR>FN$P$i4KXQZI}2t$?hWF_3!KL!~hR>FP#^Xy7iWlvVZ
zef^v4%6uYL!u_U_ePV``<gZe)ekkfE)OWY$YFD?d+vnNMjt-Q@TTD+Jn9V><`xrwC
zK2Lz(v_A0PAA<6(oSxPf!alK{))G$yr1cYEBHHvsnp%W-5_vh(k|mKROMmaTY0~TO
zFJahACEGea;Tge={Ql@!5Ho(E5{n-F#iRCT?HU;0e)0?HDs(Tp_(dngO~tgaZ+m$g
z4mh}fsZBpsZsr;Jm3EQk+zjr&4KXX?%G+W{-G7^HB*wp-huBc80Cc=uu3p(_Wtss?
zl6&Q9T^N9%@IK{gT^I~7qPSeG{x|?#9aOGXe`4#wP2T<w8#-+e2v-4rJqwQbz&_e_
zi2BT>7en@YrU#}z^LS-Gn$n1$8lt^0R+7Q`2x6VCGNJTd7^p`#hT0``0Yk@3*tY3}
z=Q3kRRa+G)8J?qKs@f@31N6$ws*Vapk(@_$135g$s%=QAZmz=C1BD%`_EBgNP(p`l
zZg#-9C(!Z()j<jkZ4l#RrQ}7=M_xQpMmt~J{(@(zZtU1nJzT}Si*z$DRgYF^CD5rI
z)ngUP0&2apdZI#kK*x)!vlZ$OwBo?L2042JT|6+aLCy@IZpHH&<eb_d%c%k>l57--
zhqip#wpA~BTKkNh7oq4SAkAY-=S3=%2GqWOUX((kX}e)WR+>gi1#F8v<vF<j=q2;H
zwR_2GE{C<t`~XY?LHpT8R^Fb^r(RyNMzVi!7xz+?aBjxRh>zU1{gme#qt`dy3%Hv+
z9gQUSIxp}=diCAkxPuG)usFlLE>1SBU-Dc)KBvbc-htf<0=U{pC7;%>EC`HbfEdu8
zhWF`N4Q8lFcdkpw0_r_ZKZ~d#n-|EoNhY&?`3pkS%G=vKx5%k2kbRRpV;u4B0;Tt0
z^FI3P0)CmM)jO^D`p3$N3nDOa1_Xe1$j*S8NYefd(OO)q5v|32*pT2wteVaUP+LUT
zgBo7dP%k9+I#ELl4XsY@x1<_go;5UeJ<|&$5HUu=`2G4Ct%hn2b=hp`w>8vroav>E
zgzMlOY(t>fUPPYruIb6pVt&k{MxG(4Z;(QsA;6nLkhbMNJ!`1WII~N!5)Fn?*T9P-
zi<?(}43}qi39mDQ;e5KOJOkqcYN*pV^R*l+*Q9aOjb}YnSZcV6IP*1^A*p^feC__s
z58eQsht#Aq<iBDPa~ki@qR8-brg#Kqd;1?XTsxfgX=tBk!)K9Xv<J=1k`BZ~)=*1v
zb`1y}y;?H{hBjw@=a3f|Uy}*uXP%Dca2P(hCQGsq9SoG+t!Au3W}pp|YsLX3OpT_Q
zS?BRJ6Bw2n17zUKYc(|Y0F>{+l|SxXGX?nUaqvT(W}}^{m$qX3Yo=oIYq+_3>HA<O
z-|#%gL$Wyzr?f}Wr_JBt3}oXJUBSkWooYYNa=0_wDf@Y8V~bZiv7bl&<;}C82TtzW
z)HCr(Tb<HOtwH8Fc;?S3yL)SEPzV_BWGl_^4d$-z()Uij#CP^MYIn<?04MLXo;!tT
zX&0Q*kmM1FC%l}}kmM1F0Vz(UFr!vx`@fxr(wKS3GSaCuW~{#vZJcaU>M8kt`OXB%
z5i=3v?^IiB(UI+480_RscJ5=d6&6CyzGyk;hOjvt-QZNaX)!Xo85Hd7k9g<3RlETW
z@sR%oGo4hJdEo_E<YV@KlAQxlt{413W5_+H+9{LZ*%aqscexgYIfuAK8qv-vYg93&
z-Fnd}b~EyN+`vsv*+R>+cjqIV!@&sWA0+*`lWSL81n0AN5uZ3mlUyCI@IExeDW*d5
znA7T9r<e-KQ&t;BIK@(^j;rRJa_3k{gCi)YBAnwyo=AW_2b|*tL;(b}a!wGy0j@=c
zbD{t$aR&UuIZ0`L5`cQ=Iwwm|A6D(`|2U_hYR`M4NbuU3lP&K;AJrVDNhVN74029K
zv<q!ZJe>uYzAreYp(#A$glY`~C*0PjorTyCY4vcQKI|+KH_LflUms^aAA-7Y1#6QF
z>DWjV&eohZicp8oIIuE{g;8^}6?Yk&a|C#)hEgJc%^~AG=Uf3TNVWKQveN;0u|IP(
z##x5mdcnck3;NwzhC09C@KKp4!xVDCA%>V})Me2`EXX4ZoMj0FEH1R6aw!vzj=N?#
zD})n!T<Y`AN&!h^8{@L~;}Ybe9YH%2`$fPHX1wP46<WV`*neEKCz$7#YyH||9=mX*
z4{e6dh9l0w_l(I&lJGN0nADh%U;jv=`d@u*U`U6g_A(XqLA0=ElR97_6#Hc|=_f@B
z{gI@f)Dh#@#m_PksmHz~Sq0UN7mMK}Sq0TiB<BlP!1?q?31OU;j7ysl+iy~s86+=l
zW=Y*W9Tp;h{jg(2SSXfjmyWQk2YehxHO8eEd;!jW9TpY@aFOe*p02QP31QQob}lS}
z;7zWx&VClwPD0pZAHEUBd|dj9t+d=4)}bZ(-{t0Prd$3N#_x<>dV%F0HYkjrQM(-0
zg92;yycTfjND6*7%MX1oJ^nI$HCmxwdX~=FFVcCMo4R^9-b74Xh^61PM!&D%SKFNd
z^%C%o6Z{$ba&veFp2rW$fJwe<NOCz0ezP~x?=nB^e(_&$#Hw$)q+AJ$!c=^@1N^nK
zaEw5O@qiY)zRO8Hur>D!>w&P#BV^O}T4OYaM3e8$TsHC$Z^Iv9v{k+Aka!p2GkbB^
z02F)MRq*lXR}P=Y4u$cP4_(KJ&n)<ycq}XpJ`L6;x$yCd2pfofYhbR#ndXI|Fh6XN
z#6jqR;IQ<j(8*wk%oy6`Uqu{U`#Z|;a;zx~UFPzal*BowJCsP%!%XUvR^nBRwi9Ja
zcC|*^xpG#zsY}$tSa!^>(p)l$$Q6T)SuUAGWKc|a)}>U$0cNXJb14lm+lsNy=29wR
zHWvSFE~-PcdN!7czq!H%M4Iy}@->&z7e|@%t12{?sEhAMoAa#zqAtFlpz(tMS2|`y
zyQu!X8U`9Cy<A<nK#+;?rH@_R1h8R)Z#00Rt{%kqaCK*pC@Lz<F4=&})r0GiF4=(I
zkJL)iU7az;UF|y!yPtV39-Ln7TLhMuWdnw0baj}NrI?Ffs7`lrL3VXI#<#0zCm8{*
zzLblWNsz0bh+?qEL6#)9V5j@Jk_qH3!0}UD{RQ~QLt?H00@(eMwz^UT_-iEruHKaE
zH6<6%WxI}pTvw-aZ1R~A;NmLm>c^DhIh;CRT)WWRMGea}Zv$rGcFn~PfvVnbTkMj#
z3<@wrOdH_m;+%DjyY${;Tq7{RT>V%xTPun**GP<J*T(Q9r&q2^%~(F;utGiH!i`;Y
z1br5}&DGaHXM3L@7gssgb}KrQ8t7CRBs7_Xc4M4JT~5W4{n~z>EP`kb>tEI<sY$Mx
z0s@5QFc7}Rx#?)Dv94Jle7(JVp~aPt3ZPc1vdE^n3OX==wSD}OtB}d1W%jbUW+Umf
z@hC?_Gk@f=5qWd;zuCeyN8-f-#IJJAm3Xvfe&r9BU7`)909Ve!T^DuNRYu+y5xz6g
zRSrLOc_ZYMi|<-rzsQNT<u+H9(B$A973UIt=lx`W_OUKwfdr)kJ>A|_BOn9d<~f&B
zz({~~sjfvB4m&O5$o}MgmzuP>5vd4wEkQ_N?qq;x*SQ`-n(No5QN(M9Io{j{eoGpq
zKN~6cxz|4v+)c3YH9US4k8qA#*FWRC$8C-+ghd1b*yqO|U_!)unG=#0KZ)^P_LF!o
zEo_E(FSnC;<1xq#9@LFn&%_$|c=Q`z0NK`c%TkKQ!d@_ldts5>W2%Qc@7+by<sy`T
zmloxKlt%ZfF^gu<T?fW|)8OST!uSsjVW}$?@g0trAAx)Jx<!0nBXB$1r|K5*c@5=0
zwn!e=p#P4=i}<#N_%HWeRM_awerORdGvOgbL|XpXBDu_ziX6P0P#e?@EBKQ#-c5r~
z@4mGmXh~C(#3va(3rEz3L!>E#=~MGLe0EN+jd!Q2xeuSI<7<1nQ`MY;&y2Tf`4)OZ
zs$%%`Z)?<!Zc0XGHG1v1rewnGiP}l-WDCe_U&q?n?o<oN%s;Yrt~=ENGV6Y=mY<we
z%odPYQp?(hn^KWi_3O39il$^_=KQ60jXT+VGCMJ#cDFm(d@}1^Tf5(#YCf5De!lil
zQz|lR_fhTXrc`A1!aKDWno^P1natWNhCA6jGD~~6_HXWF^T^EaVC{SEWb?>uaYXG$
zO{vIiQ*!NhO{vIi-BYzcH>D!8(YI?k(W&yN{=7T1yzkWN{)RhU^?mr|MAho8-3hBt
z!OOC*Ru5@PSqv}J&00OCDd9AD4H{Of_iRd-46mVY*6O|839HC*eoU>-kNl~|RRxZQ
zc08e0ALCA0MV2rAyH+39l#(nXj9R^*DJ5C%np>;eni7&_yH2$_Re&m&m1H^adab_L
zov;!twF_U@>Ps6E8sC^I$@1a8wfc&tlw|qp&{}<aQ%bVz<5R2eakH!>%l;qN>igXZ
zE6CEjb*+BVov?x|hqb8H&$ts-kmE>Ot$u+YVpYOckmd4UYxR3gDamp}ms<Vprj%s4
z`ERxQdv2EHWLb}l|EDpbN52esg$LB?-@6l*zXGrJb+!7>O$lo-UK`%~eHK$y@){2t
zvTGKLrbPCwOJf&v9RaHYgzQ+bSdNkFXdzL1aB(*zQBP=Z`E)TADC)JOUge9q30AKq
zW$#)nN6K|_x%KAzVCOly*<$KPUZcgS>%rn)hQ#I~z<=lB-fnj3Rg2}6xK18eJlQe8
zpc?lUH_cC;F(MgHmNc_38MIqzx0aJ!yGe$!TcNegh+BqefaIpji0gdt17gv9$MwCT
z^2JHkDE2YBBFO&>E-JP<4V5wKor*gdDr2oU?%Jm>&hALT3ot#*36JKG?$Iwjh*Tk2
z+$=b_eSZbhN8rJ5l=iKNayXw4k7?EzX`)0j{*Vu&_2>~E>yP$-Yj8S8(Ua*T(Ph32
z=f?)|GB<s^;p4j|JRXyoIua`{zp9(2h4%!DThC0x-L%l~1XSm=S#VFv3+JKfw6{2|
z-CB*m^MD!>({L(ocX%%krt&7`4J(<6rQ!WNn;C@59M~Bz3{P&(2@>x1FNY^%5)*gA
z!{PlK{cARd_eUiLe-6PC7KQhZq+19sMXKJn!v{!AgbqIvo+54ueHcE;+rm>IP(%EO
z5WoCEcxrRBe<5f2FJ4l>T46L3te=MjmLJ0hMWDcyPdUo6&AjmRHb}Vt!|=iWuwaUV
zyU-Rs1m%-h1NV^N@S);H?poan&j=)~73C-od^JP6{fgSPtezNNgv{MO3m#iL8{t)0
zI;!GbVTYF~E4-%3rx>L0!7*1WAytU*)q40sFxTSgQ*H;_)mr!>@#%%Qso@V{`MHHw
zeOznY89a+|b(y-W!onX$;GG0StV{}D3il4f^8~8L5);nb4EQ7vVte5(`M~h~SO;qH
zl<~FjBTWL$f=}|(;YU%pceaaAxzao^!Y3{d{~9G6v|GG63sm<C=PRuZK9k`Sc_94T
zCLb(V3~k7v@IMU`nHkaH|3mk>JH*USm4)*Y%_uz+hTFJsUJ|%FgkA0KyWuqCy*q-3
z&j)6Q>l$|9cQa{e*m_#HuCs%t($es$i{ZM7^d`uw%2NhI+U%3~nVN(m(o#|Xce!Sz
zR&PG6t+K*(+VS4`CqnP$<8QG9cF@qFwwuHCW)wm8iE}d%g%x{upG5Ixlr5FvdLE<H
zu@L6+iX)FQ?p~Wk7N`cqX?H!`^fu$1KpeyL$awfId_COsZlj-R>3F0I{sd7tWoq$W
zT=Cug@*-;I=(FLb9~gi{8OOpsYTBY3eotm4Kfr5l-*B%M7<9*Nq}TXJcy03w_iE+#
z8utynR(ghe@e|gF#r`+$BK%VSFWf7<DcS~j?T-!j>e%GPJ#XqSF|B<elAMq)&4f?>
z!k9KV=L;zk8U78QK_A3)j6_j#7i?O}CEbFKF;VDB=2H@<9zKWknC_i$c)7|Dufy=#
za4jYdghJmFpS|$8_G?Tpuy5-jgc9I0&OfFv;<W!j;yeZ)V`NN!`c#Y0O85-VjTwkk
z9Xd*?X!z_I6Em1Tqa@A}_}m#5GmJj<lK-W_@M_g5W+bGJOp|23tbngE5IUs-{$K^%
zoAUxHVC7ex(A4fZ5luh&|JZu(z^aPpefaEcse}ZOb`nTvX%GSg2oQQn=)FTAgf2}$
zn!pJc?!6=$0yb1o5Jd4q3q?^;#0n@PiXaLoDk_L2DheXv`#f{bxryKR_s8D5vpYL8
zJ3Bi&yQk6pRv!*<bNM)Zyq0JyaV4MSf)>w`uF<I!l$ujF#(#~H?jOV{qn#ASv-BXz
z=`i?@e-?6bkn!;s7ztlkTv#vI`^P^EJ&iZ7CA=ILHjrQ`Z$qu}S^+=XH0n9zXQ+Z9
zoU!YW-?OV=I?B}(@LxwS7Cg?I431tcc)|@o`d(xw!w3z<%>XAbrMk~B!jQ*m0^sEW
zBU}Kkz0Kzsyh3B%O4%vYkXzhjA+B`V;0=Evf4hM;FVlz;G%j*21{swBoIb%t&XZ>F
zhQCI?*9WlmGb4sI7g}?r!1*il(qHr4--Le@SZ*@7`<Zlu2W6jdBlb)`qdNSOl9xS>
z9c0uHAQUy*45Ow1JPqqs$EYO$&ouhS8MOt72aua<@VGBDB8j`}$2xQ~44#djX@R;K
zBaOy_NCkY<OHRhmv>@@9Mj2dM;~r=Pp!6$)C*xXvGJttgjC28b-|DjUMzb*Lotsz3
zXoI_cEx!qX_9;dS0nz{*@)$fZ*YdfoHRhc+cu_{n=T>#|WW&74v>AXWCK={Urab)I
z+TY+6ny9Ok^v#bAo~4^_AVAE}RsbH0#(ZM%EM1c~7`MG{@JdahQy$Z%qR*I*X~(hi
zdm7hW;pU6L&YVQUqc>J~V6bQ8>h3f5tO#L8<O)VpwIWo2P(J0p!ZeG>>Wj1$awpWf
zO4S!58wvDvl|ksGA6C#LLL*`T%z9~s-1WGQZ>8+sc10A|igh}GgWs=^TOf^faoj9R
zaJDVpwW0>vo{b@OUU-Jgq+S!7M)R)T(3e->1!w>q^4!=8ek~jCHDP~dUv#bD*q$AX
zQ?a?)ideHfcr87&g16(F6qX)b5sN8iQ!rM<VmezS?rELjJxvl@(CVXJ-cOKk^bO77
zrBU890FrO>U@U5*_pFR_JpdUWcs~`ZX+Yq#_nZKYHGVDT1BcfY#Y0-^{_Jbxkx2{r
zS3<O-Pp>4SoqQ3x)P63tmHyQ>qv6qA3SB;X+^)PKm6ff^mmN^S+3y7L9cMIr=KGTW
z$cdR=^L@_np{r`jOwU_YOa5oBsxAMESLyQq$f{UZD}+GJjUGPL50ZR(3mST1iBFyf
zhCqNC9QCOIsDj=A-Mzx61`5O*|JUQwT|7^M;sCwc#is@f#9Of{o%X3AfZ!DXJ@SW7
z4V5}ROweMGPYnYEzW~US<}=iAfba|ew|?f6XM^Dy0FArmQzJ#qodE55%%?^P(Ju0t
ztYvGRE}z<j1D1iPO&IEv7onMg?+f>-r-da2KU&GBwg}|tG%+t<^Qo<X&JA=9#1mf$
zhEF{smCoS3$LA)9wjJiD1u$~w=88LdosrSNY=3nO&AAE85AY`k^CSIP!Tbb&f93}Q
zXF)IjBglvRjr`p&`yWOA^a<prcJyyRzLD`efJb`xH?iUZ8w~P49c=K1e~aB<!6^T8
z!Sdt&?RI&7l7AQSjf74=|6Z00ovHr)LHtYp13~=u{+EOJ!~Di;c7waG_+Ph?=8y98
z4r-HVuK$>gv+xQ3d&o;~W_I?!Z{tr)^`Elwx7_XjB#6Joe>RBU)BmZBKXah}f|-Z^
znb^Yr6#!wjq^|#4J3n*3|Ek?!;sXEAcKO1O{J$Y@3USE)dk|;2|E7&oxrP5W%S981
z{dWWh2JcV}{K|s?|NJ;iquY7Ep~Tb4or$yR`Bf-_u=g*}#E(|;E6dr5-S`##UCiN1
zsC#gtUzw|544_J<{fc*b#~iK-XkMmYnd@K;KrIIO6&+p7VLmP7N%JdSD2_QyhqU!G
zzcOM3jOSYTc^cAZVr%jujco`0sv5l0=f@FkFWTf+c@5B|AAxzgr>31ZZ|jQHP<gV3
zU*!|?cY2I<sEijfiYEKs7Xe;EIv<Vv!o7i7$VWYn{J{x<+L6$I)Z=!eA<`(4jrvau
z#IaE~yKauu#jq}}TcAGc#@lrdNL>xq<!uh6ux=^x&4vW{8p!kGk-xhvzzZr;-oh2=
zYL^?2v<&pJD@J+(gYAk}8U=VBDGink3Gn?$Lg+wpV47Wi-`|0I?D7}31b8=};1B;T
zz}MLde&_E40lR$B-+>alJmJXzUkz)R44w=;Wmh!3C$P<K@ZjLUF1tK;O5lK9p7ccE
zRlB^Pe&8*;y!Fk%3A_C9_`qo?$J+Gr)3Jf`u)2#3jk=u=$c?k|qAp|dhk*;SD)MY$
zR=vPQF+lEdI>ZOQgmK{fT|W9AxP)s27sD9eeDqP^D*<=~r|K7huLY>dg;dULFKh;8
z9T^n4XT`fb@GY$4ViZhe*{Oi41mMDE$n{DGBXYHif{g>EZs%9Kg&|kGS@l-)E+WhY
zn&!pUNOzkX*ljPa4ohH@=D5CfY;`ymJZ@369+eBtn=p~@_E}LCbY4E7*1lR%RqRsU
z=Ugm&w5Xb$e`a>kU9>+u-HerJK*OTvAjlVM<y{>`+bM_~u@qJ-dR_qPWW<j}I|#_#
ztq=ZCv{P!RlTN#eb_qb8RC={&w*U+PU+Y@52b8sZuH9Y67L^JR2jKpni;TSj@!CV<
zRYm&*s0*O#FGc$WhzGF!%c27UBmijNujmB<xcV<SS@fa+jBJ-&Dmo}Y1MO)n;@kfT
zsDCTngZ<_+1&3l-a(}q)dM@Tg<eH3|)f=6Cs~!4(<Xc|*9HRmA;S=o(E3v-7+w&0W
zzebI`gbjBWqdF_MXTFEkyW@I(h3ea^&Sy=tllQld!GBt_`&0Pe?fwTkdGH_cX1%}N
z-=M{jw{oJmCn(jlFWU=0leFkxy9|uz!fu(BGI5;nS<xkcyfOF7-I4hdrfNmsBQpGQ
zH-^vuP|=Ur$bGpV`LK72xKYPjGx)`nqJO04FtFT}Mdtpy6hQNmBH4ax1fIF4NI7L#
zXsN6CCX|cD>&@V7h8G<I%S*iH3S<v~opvrt$R3JEJ|E>Edng%sjvlBUYH8&`l(#5p
zrIFLLqSr`0D+bb4eW>VUeUik^j^>Mwg+E)wFo_+z#31?TjiSws(0|0ds9kfXXoX!{
zXx6T%$4cav+$>trlr2HH>03pN+;GB(C1_sJ{gRJ~L}*;}kPiH=I3Llp_^L&VZEA<3
z@@G!eiex956*UhR#)=jvfWkU7ZDKrIw79VhKU!6rSTxUW^*w4K;k-o35QR^cYDM!h
zM0z0RpDLPZA%4Z_3&fd%NZxI-i>BC>yt@aLQ&P&QOu<x!4uF_;qbSd?F?*Sqd4kDW
z*N~#ocI7;?a&%HTnb8~oAyfE%(NG(4t%*2P5J@Hd_oDuG<vV6&f2?An)-4x{`WqO2
zEz9g*|MoV?EvJim+O2*zTlK8as;4y4@H;oBC<TaL4>a|f0*@tnoqDY(B_89>9c_<(
zMJcJwW5hOR7Bx0)gyw>0W-<P@y|jh~FY#HjZs&{4E5IOzE>vAsR0BH(n2GkaAM{XN
z1E~0d7f8sZZ<|8AWpS{uUH(FqVRhk~$0FuVny(cqxgUzRn-6}e(0XMLKSaOqc%k*m
zo(SH9Q)s@ihj>fd6{<X$F%h_>uux5BejO?@#ucg=Hjvr5P|Xw|VkVGg*JVF|^x6l7
zc(Wv0q`^gh(-kGp{ZXiTBk~X$nI9FZK78_B%ZlJb1c2a(F{Z>`2JDrog=(&iy<uUY
znr8#co-I`7oBkqZP9o2HSyQUTL`(x<(gcC5`wLY8YVZn4Dl+MM;Y~{=mm_6V3vU`Y
z)v)GR;Y}<k8oNDgQK3>mz?r@&g=|3CZ%Pqnc69<Z#jr~{GwxcUk{j*TMJzx$w~_FS
zVA^wOA-x-C{pFD|6ALe6p2Frh6AO*Yuq$&aK3jMhlRRcjqzu+~q|6CFt6t#`vc76P
zG;Ms<!c#W2^(KW=aEL+xh^HFS&LTNhAB$u#k-~FAiLG0=Da0d&rYv7s{13xJ+075b
zeyrw4g3aHN<}s1z*0C7kiY`Z0DwIc$(5-LpGIhDTe3BRLmPsyx>=Q~<#k0K%w^{7J
z<&&sKZxn7bq>&)XD!Yz|&OqU#He!K^_-K>h)GvIrE&C87e!uVujy(1B%squ0Bro!v
zXj6!2LAv_wag#)mkT6@VkxrqNfo3HFL@v?_*P6M-TH!j$QJ80bEwr7~cbZAbz;w2p
zVxa*Pa|*xoLm`i}InpQ_@j&51OMvg<%w`Lv87r$jUO3&XG_>!ykA*6xV@_Lcsc^b@
zTJcJS(>n=GwAyrc;dq<meP*Na6&j5<-805HYImV{#C%QrZoTN%+{kM2IO%<~upg$`
zcdthA7KuWmpV*(tsvk6j@dbqe?-ustDFp`myU$F_o>+|l!|4s3S=a({e4l2TPz&3F
zrd}&-Ar=JLdh{!7f$apW3DdKscm!x09-);u1TK-?&KI_gf&a5CYRsO(MwXlVJ`WU7
zEf1t*kwQ`h3jE+Y?)Zh2xyK9ZNR$U*icE#n!4x`k(qt8HB4RwY<_8>uEY*E~O1xpf
zA$L0(raKHcrgFeb#Q_E!(`58LbbtZJECTDah1hsh4mf&{3>5qQO-%HFqbJzn+4n`r
zas@-paUl3#2sk<lzb=S@q5_U&09O(~XzLdMa5K=A1jYsKCI*gQOFaQceIoR;5b9$}
zUr9huwg-^OB?g0F9tdB_tqCq^_=E)U(fGs6fFq8Lms^eFq_Kvo6A-rss~)$iY6Po3
zxE654u*x`K)y0(8LBvqjahE`OL!}DZczs#G5su&pOA*N5KRclQBHj%XPyH1nvAQsz
zen-_+x<rsrzn51{38)`g)kLZcK*73>(E;@>>vF7S-v+BzcMqsDtm0^p;u)#p7|5%^
zq4+FB*IfgOksR|^Uj*jS8z_)43c!+^0H9Uz@#u?sQ^)#y^~D%;EiDBg)`F{tBw)L+
z8KZNeJ9@b<EuiS>uKo@RB~Jo?0>tIdO$eyHD8%wL4N{1o%hI+0_Xxz;uF&{@4+m5+
z3Vvt?yc*vFfV;c;BKQqk6i`pXU~v8%F?$x8^F7Px1R5ol1QbKTA83+Io&}J=i3A|S
z4FFC=TNe|-h%BI1(cxf{qQC130%|^*G5^EU(Tw>}Kx@g0=28OybTm3P5}j{GeFm_X
zEe<F<V*BB9)KhC_$QEs~F=IA}J?>gS$>dl^Hiw4=)F_MPHQqJGhNHxzTexFTsI7~b
z&>ZZ?np0&$Gvsrvd<z4>I19)?p7(M=B_>-5J+jDY-k^XQU~^h(aT<_BQ?x`JAn~BZ
zc95`T!_KP%D%)=OrqwW88VWA#ucsgbnwu9;<|GC%yIVl@7GX#xwp~DVu*v*wk?9~I
zA%HCxQ47ZiO`C#uFe6C{{!Tz;S`B~X(??WMW_d&K&y<!LJVoQ*0;(a9e_UXVSVM^?
z(P}F6-Z0)WgC6|=NJJ@yN2*wXATxIMD!JSu8pj4yO^{>c_~RB}v7T2cf)k)NKZJ4h
zMfNZk9Ox04|M3k=&={FI^!YVkDdsHr@s9ZXV!laEqJ|AG7Vm~{jyVifrKc3{hE>US
zX>LNX83VAfZ&2|jH2$fb*?1F0;O+yBjZLt4yY(j8HaA;ny{Q%QS!}(*QM{h5d8T{!
z=wiNz6%!?ZqJnpe*Wyph2i6-ii+#l6CVu$YVjp?rN6nr*Su6HIK;~y5o5(DdhlnSO
zR|6Aoo<U$!%*6vVL8i?U#rKiSIg`wN>?b7x<NM;7H`^{Z8(>ks_rv1ZB7>&=%&S!3
zJzIF2h;s#zJm#BCz+-x=;>l?8^FWjs<E|A?wv~ytlervQa!<wrYRNsB+7MBv1<7>2
zS<G<YXD>=XMc(O1t%AyUcrsh;@$`5SipIWBJkl0*huLT(-|;A8fmS1hH?=TjZZT)u
z&wNfA6$2_Lb3nN=2Z%D+H}mkFEu+3H?uC}Wgqtn-IGHU-uXtOK+JWM(mLSG2yx{~q
zx(W}H=yI#LElDs4Kt<aM%-fbTZyRLJP|_T{9*-4s&+`j+Nobi-fq6!d`SA_KO)TcW
zY~}SQU^WRdPpnhih$N1hrqjq`4%2CbLo1U_s|LmK#JL1EunRt_h6)<wt{cU*g*Lh|
zv~6)!<jZ*43oIT%Sy&%_R9s209#QcB14jEDRTXC4$^(A&7R!+@yCyRb1qi6loCL7s
zCV+I-Vmuial>N;nnDw=f_|<VtieGsWyY&md`U>bVhv6Ii>T6_Ww)0f@cTp6O#$EHP
z>#SMlLH~y>^2-+>@FWVTl44p4C_=wj4Cz+(J?~c^!L7s`1~7iLUwte-Mi`uBHY+A#
z?vx2&cMfY~+yf}FqLni9{(;JjR?4nnN&TG9<T@bBK6La~+{a|VAp(jQF9jD^^`u`t
z$GUqUAI`_sa|o8SEC5x}v*%!;A_W^gFOdjDq5^h`0?d^hkSRQUYF<ab;%xhE8Zf&y
z@~fE;<F{!d{Cs)-i4owoLw@xnHBt&4{K^Ho>9c+{6(?4>*2^LufwO3Ou;{muD58W5
zL<Io0zv@?uVWGb*20#gyK&Ue(0jx~*t7SIYhJ}8$+y)G2U&&3d+-o+U<ChJq3>mw1
zKgj(`0e<U=>PIg7l`n?+ka6KK-4B<^n8kGeezl2&*U13D+8~OAi?IwgPxpj6-mC3b
zjD&ytT$UJIe;6e34EjD5Rxv0cxW)j0Grt%nnvz&q))-1POr(N9?EY|zBdoSgE5BlV
z%mwFnK552!K3^&!lHWXJ2vX@qDh4VS;()q~Uzq_jS~f>Zs;{HD`n(5<|NGUaE|Lzq
z@Wfj_brxm6PXgXMP5_uAj2UogjcyzLhAf%aeZf%h_Z<=rp@5Un42J-KJU)a6B2AtJ
z)C5W#6ESD@Jdoi<DV&JHRu5<R)P6EME&2k0LJ6W~)dSFmHB|1Ww@{vT&WE^9)2?e4
z@^%<N5P5qmK?dHM*L`Xk@j68t49x)1xlj~RT?X;5cY@x1=EtaLpu8ICLp{cp+`59w
z(LA5ZB^95=EtgZpCXx%$u6MFpj+d4~aSF2%Q@K9%sqXgH_WCm@KY9Ztm^v6gh|p+{
zFE}RGPg&I+lUOZb4r(#2O<Jqx`&1jq!gkkxv2fc6E{l$V74O<({*NT9x+zTejA7Gi
zinUbCo;_^_6!a9>kKOH4bjZ?@@gQYv#Fm^_#3eb2$HU(7sbm7uyQez%R2{ovvDL6n
zas}z@U_9_4c2YLJ@+t0=gv-koat%S2P}HZ)$8lLdxRWm!DE;x9g&TtfSKRcLi>Pf*
zN|ugdI@l5=qbm^>lRtTG$F3L=6<bC~OLD3Cv`;A_V=n(W-0G?vUgLmi;v|@c3ltDq
z&t5PCfZMctEIaM%`fj1!pL0;szNzmPZ9*cONssz|WyK??Xw%X6D@`*CZ{DI3xwio)
zG{3Kw?<%og0ak?9cU9~%M3A^70SV*uz<a*WDI4Vc^HU(ABYa~g%!Z5Lk^R1pSi277
zUT)^&iPMeI68ZTK(-$yCNn7GOoWME8`(NiK_zpAFVMTh1?{&H?<}*k6UgsQT{PldP
z?{%(zjK5}n<YOeStwTQdmhUZ@tT^fNy=_Vuk9(JI`u39VTtp|ngJv-|Vw*<$wkgP!
zQsR4v^$(MO%2U3FWR;-;UVX#IhmCLSLdAh4z6XLa(Y)Edd93&o6_An#<02)bpT}Wa
zM+p{?6^^F39Qt@YFzx{e+4~dUQp-z%!&6&)`5<(&1qd}XR{HW2pa{m#?KEFLTyD%^
zG}*ewHwi`3-{l|rCQF|3&5rU7X6uP4>A2N5m|{>t@L_{RL6O{o_(lgC^nTaZmkpMg
z4f@8i0V#R9`no8L|IMA|;B<*c4fVd^MPHX>%E2CWlO7SjB+l2GgwC7jtt+6n4x+aS
zqVK)p<CMR72NdUfeNDN{<ddRI?kq8toVN@gk47Yd-*VZ<Q;@$JpuvlCef42crl1G*
z`>L~o2W6<JE<K1`&`k50_vnzQJP($|1%K7+tJF(G=aKZ*F95*OaohrrIV*9MqFMa)
zh*h<>Hu@htY58j#QS8{|uWhn)hBJ2g>k;(7)&Exg!^YPk2I&3+lev;G7Ej3>S*7m6
z3_W^dm2yi=A_q*X<Vw55MmT(5rFi=ZXUta6IQaaU(V*zBwZ`Cjd5b5gTfbP-16_us
z@p#l{!<s}^Fxp?+cTFPVDLLd^a&t{Gio`6A-dNK_*hS{g(blA6YMnU^%gMd9ri07~
z<d+^=(^1fc=K3|2N%IGkplxMX_L-Bke=w*FX>cS>LX~@2gd)6aVmO+dX6Y$wst6&R
zCQj7WRFP@Pi%x5*^31L)P@2zKQ(Xoh!?C88<YFSHYir`HexvJ?+m|?2Lxi(2n)Y{-
zXmBer9IHX=%xTeIiK9q51R(cTiDO;3#2*Xrw%-Ru5DxGDpGzEbLF4bG(%vzb0rKhS
z{5)XIg-1WjNa*N|636_Cq?U<BSatud5{FzMScjbkJpX{FN*u#T@3rWK(`JDPgPSv_
z!S8|Y05JFhv~ODC7%rD5Ojg5j%7JdcsMhW;adZKJ+chl$U8q=_K$o`V;&nwuP`vJ@
zc(A`$&XqWL@+Yg`(nBSV{$^)EyLo(xgOlj?3KUJf20(fXz<nNo&;~%^n&wy(*!O03
z-$}bI$uWjl>nV{(Ib+5EFrEOw`=1f#U})ZWpv1v-7La!PWmC0{L=(Uz6<KCUrDaix
zLzdZ6wtM>d5(ljcGgZ5ND|!(WSXrEW^kQaZ`9LEI6(d4Lz-USwcZt<t01R#8=OyX}
zDEu=Ja=thh0Ndx2l)90C>^hQCH<Fk&NvXfeNh#i;eTEm8p1N9MeiAQ>5#^xDCF&iJ
zrLElw18In$-T^&CFEl*$PCJeauYW;(@@2#(_W)=<r$ikSjmgw5QNN1B>H}dD!CU)E
z)W^0;Q&yL#(=sv8^jInYNQPK+8j!w_@`&lo!u(`=_0M0xi#$@IE@5qe&SJ>_<@P1&
zM?|LBYS5tSFE3H+Ao(5h6b1lih$Z!dVq9W_>cQ5QJ(0kAY!cKHrmT<IDq&|fRFsuR
zj&fCtvOWn9hVw^A0M)6d1WzO{l`Ln=oOesqJ`ok<@9dM#ya<hA=wk$U#ROQORbO8s
z2bb)}otx45==cgcE^-)~U!qppl2G1NBCp2tZ`ZXFH47d7cMuRbOtWa^C(Y@kX0_)y
zp$79&&92b>*<g7##_aJu6?-fXSY}a`F?CDSSW@0)Q64L*1c1q;#)5z$q%oURu06GA
z4!O<PN0avWinJ%9E&2-2v8Q2)oGku(G@2%@vcd1)qm0habVYTov=Y^xtbaGF+sCrn
z5~Do=21H9}VfZeWs7R$LZN7;YiPh@gqY;xQfMk?-Eog7qvn5J=F(+Vby%O_MBLVnH
zRn<mo0yHiqiB)A7O?S*;<h0`B%5ob-+L98*XKFard%*v^IK4zJi{@)e^+0PdyNNDq
zymuE3`F%>Q6b^pshLT&-d*qkIl-!CY4)P23mfWJ%XGP3~tG1qfh?v5>L+~ReFfYL?
zaFCVIH7jBYvWDF8gA2gOdra)9$c9MvE=W`#;x8^h8*>?9Q7fy6w|CMIEs!tXqZ#(V
zk;a=P-$3`cBCO;ql<3ko=&^JO9`+k<0~tw9-C&e_TcJ<iR_N2W(x*sB^P^xyLh_%?
z90dOfz7bd8A@)yLB9LR>%dBc-e>Jo0Irj)!76s?aVbPbeC>CH>uhhSWS@f$}#LHqp
z_|1g0cv#tQCD`jK$MC3$^X6+MXJH*NW1{BG#pF8Mh<OZB_tqt6o025*aMfp_1IfdX
z&*lg#So8XhB{;rO4&EUI<3EhxhJ*}KI*eHbU<R%mmmF?_7NwC>CW0F-0;2(BT5_}G
zUG}0hauVJ{RzfpIejdD>xXMz-OxighAC;KNP03%bT=Fy}|0{~qgI@;z_b5ogV7Ho!
zF@UrsM#)~w6wzel(<O+EmGhu9$p@`}kIMDh5^nXy*wB1(usQ7g0dY=3s${)Tf?A#9
zP=Lyr98Qk&(+5gsnU0FJ%goxjEHN0}pn86J^#-$gS%t>S%~}o*$gU)0wOeJ@XzcOE
zKTDcJd{;*izj=iow-S-jI_0U7*3uP{!bBWy*fP5;;(YgvEpx}9P_CvX>+A}(IsXxA
zpwyr(1bOAd60GIQ^{_<hC31+l&NQG#sK!upk5JMmnf{*<|6Oq<jo^}O3z8T5(VAdE
zjSvpPSkWv!y`(OwUNx!KrH`<v)}^ntsMdv9%8mp?>sBz!y6s5FBw9C{c@j06zg<!l
zRNWm+s#O_dSycHTE^U-1S{3n@ZR}NBSd?JwRXbUEh+4G|^YqO<Q@k!5Lo4?LD0}M{
z05IdPB;?A;sdCS?VtnNI{IQl6h9Dov=*1g(URPJP++?-viea>O2+F;O7{S(iq;Z5a
z=5-8XoY$3(#vZ=kf^5^JaRkqJysqXblnAEDvtCyV%V!%}>}jtnL#iTp&Ccs;2_0$q
z5`B1G9b_-zb!8Gm*0jARy)K+pC=ai#kx7k;OeFAL2ah<bp4U}7j%f6cO~EqGZH5@I
zxKDC8V4J)y7tVi_hZodH$t9GyEY&i+&O5An#;kJQK_nMcgXaJ-Y&Cceh#L5^OW>n%
z-f^$<Pd5I<YW$}(Cf&S4Ugs~Y`o*d;ehJnspXhadin@>yR^6v&#}KV~ouA3}20fXZ
z;C1pErD(Fh)9XABK+c4hg7yW`U5tcjUgsC$8~}g-UwXh;5JK$ndYwneXdcgh(Q)UI
zV8=)G_c~u;9m6J8z4CuMKB_-yOuy!J?kC<`7T*5<i#Pom@$%pFI(HI}rzfPh^S^jl
zfAilI!~<UEHXw#_6(r)epkk&U^*ZrMMHNaTVAV#c!hxsP`IK}xVnK7{H%VTIZSp!F
zVvE66i-&^5^4EEtMXd5m6~?)!oZ16I0a~8C!|N<AC%4k;oWVASEOIlXiG?@IZnOb%
z&JH#jchT$2W20}ZMtOFlanM~p3UQcJ2mN23<8^i?;vEaIdvG+;2YH<xQ5Dw6s_G~-
z;8k!1M9~b>2eH3zHuX9gM1}E$kQ#~<JUIKXK@sz2LpRZ8d>vy#)q~Ntp&iNdIz4P_
z%>t)K+LG)`ZM;s6RqHLXe9F^2N;uNe>%cji3fpH^Ic}9#C3+p#S@nTcb=|CjUy6zG
zI?kgiyp<f;0x-mjM)Nu)AjF7rfV*KK04&Pb2I1{2$kUJJM^BVIHn|c~zj4XyxG3tj
zX1L=^YlcJgBelE^+%KWRdGrg7IL?R=Hs+@`CiS1?bu1<7eHQA{p#F!Q_Bs}@YO7VX
zK&VvE|FF|k)UqGF4j1tbTX^N3WZ5Mz?-EhrjEEqJ;^#=OB*>&x|5O8i#PFQrici;`
z5pt35c-3!c6A@{(`7J2XvbVkJ8mkhms%zy{FL~7kR&}weE(EK3@A2}Ifr{V_j1=Om
z-4DYyq~3d^BRjq7bt3-HLVUe~v~OT-vW4ZR%jDDo{z%aWn9g}8`tbtW|6sL$p+ftE
z725NwYHV-Np(!3Yn*^3N^{QPU5Xo?#lJ2TNV2==>oJ;F_d6`T_@@`V1Js0fj(i&dH
zz$-G_s$%G68m(`xSK)+KRf@2x0&&p)m4xVtcylJI5-CN*y)za8FfWM%SWJK(K0ji{
zT&YR20IA6&08cQ+=~efGtoY^Z=3ceN2D)GHs&xdoUuj$oKs^g+-_&cUhovflhc8~m
zPv41JAL#B?8*BicU2P2ZeaCq(x0$Mv<rLJ4U<a2*c-0bC@jQiU;T5>xvc=VVX#~}m
zdBv;d5bs{YLYyOrrXkJ~zQnul7q6N|yp0yxv<hf<m!o~-RSaA!@uZn-^MewFLRef%
zSCzPlVATMrf+o75W{|x9lX_Wl(`$axQm#joKo#&wv<Ve;0xgN`VUV2#fVdvI3N}1?
z!>huuRXKCgj1<gUQdzSy)my$#ghx?T(N@|FTjW*MOs|dcZ$8JX?vg6(8of%w6mD0d
zR)WYQm%S=6xb=#96h(+7R2$QO5bJrXs69x?#ZndZs#Vm<EV7)A>S}^=^4liV`!70(
z>pw7{4~Kd&aizDFGvppw?*%B=x<}S~p)T?aQxP~_kT?**Fyb40$XQj+VBCx+B7dQf
z49cMpUfbX`$jX?ngX3{nw%N-$Rk<W;4n57TdXi^Yu*+!0IgMtpUzx`1Vj{&9kK%Ad
z&QoED$Lu@|@wgR7fcBZwpuPXPcN1y9WYR_$NIs;!2@y3S4&WTT&pVqqJUI@1**hDi
zY@OhGjeBM6c!cX+A_fIn+H~~N7i09HoUJ>&lc}lGnu?A!2~6gqksrAT6VSXPn@lD>
z=^Y9t(NG|04NZ_HD8Db>i}QaK-4|^myxzW4tX#zz)7aaF=l_6I^b=r<+ID11ExtC}
z%UE5+Oi%Her^s^n>+NKh!*DtaKE-_HhBp;r3b7A?0UJvdF_DKQra~U`;=)32YGc7e
zqutXCZ!BBE&ZBPv6N48kg9j0oJ?rJA0P`9^o&E(+q8Bj>P_p%lf+sNw!Fe6yM_AWi
zZksOPL64So5BT$;7mUDb%M0GYSZVpl4>?-E6F{hL9==Fga1^xo{!Ngb63IK?3$}?7
zX|#u?&lbFDEs|B_)=L%R2UEV|q$TAjhS>GD78I&@JpFRHU>=3L3cj5i6)fZvFqJ(8
zi%93tHwBC3s0;bW-z-=nd9wUmm>301;Ab#zES^ijjOS-tL4NAf1v5cCWO~7HP_Gh?
z9?sJWMze>d;4q+b!T1=fS)L^-bh@}pK^j$xvyUpg%crnfK^ln4gDWRrC`c2>f%5LV
z3ex!gA=y^9>Rga6bYmjt%!F#Nc|~hTnA)+RwFFDZ54uyp?_-I5)P1X<orEqZUvje`
zOUg|F`!HWBvv165>>j5)8yCI;M-NrLgIlP4D=vH`#YxNKyGG)S@KxNn@(z+YT3mQh
zRpLq8>b=|G8r5rY;gc$%_f^M4&z(IJ7?Y9+MC2no1rkf-KW85Ra223x=eY2x?8`c^
ziw^}Last>l1_0Ndb#QcX;nRp@WZ@VSdF~Xl=rHt}-$Hv6ABqdlt5N0%DL}bPZG0bP
zHA=(Bz&5q4GWEh0{D-QILooKg$A$N!gJSX9_-~*#Lc*I<sTWjrBcP^_#PSvw-k;Dv
zYTC%z_s$16pu1+MY7RAZB;Rlv7e27NrmBUgDNy+HljFh%hhw+-Fi28E^6KIQ9TFxC
z<<A=YRDX$bMWl!KL^vY?5EF9s!JPXnE<9WoTF6Di-ATEsEwSVZ??FIKH%!4I2pU1b
zBC`Rm3d&UpVpA^ar&cC^X7T57uzC8ExUhe4aw^<i_>UAemrOQc|Ky<MC9C^kcL;rJ
za|!#mr>2Z*KLpv}<Sxjj#w2j4d7f;XVDr(jxUi!Ph@s`uBbXxH$fjnA#pY<QifoR>
z|A)=-1b{&<Vei31mn+!_J7H_NA;>2{^$-3SC!srS7&d`uAy!lM`<AJ_8Jdz`=r}Iy
zG7iV?x{Uon{d<8k0I44VNTiKL8ckp{GIt|Q>#aF=8qyXo%m5vsFxu_JH=K83w)535
z+I%OFiji4~v;y{Kag~>hyIRJD;iOVkA82!22domC<C6e3*c^8O*fGZ92<5?oJI95Q
zo5}1Yl)Pv$Gc?-lo8{a_Xb#}_jSKsZ+}^ageF&@%Y;NZPd}edI2H>jQzwx-wOYOz4
z=Bmi?PZV6YIP#4iuv*Pjs-gBy)EsCt5k=7I_uyKiwZ)Z>&#MTFtD+mKVR2Q702<m{
zp>t56<|}el&4H6{bL{{CgJ}<Cfix+g{{z>n<hs-53YDo|HrL?*@Frv#OEn2VzRh)&
zkihSMxy}L3ESu|n02Y^XeOL@grgU5wjzCmRBf;ic2-JYh^>F~}ZLZG&*k*IxC|vPN
zxr1Y>_5r8V=8EA|M-b9j26UPFc8!b+yBTIP`v4gDWVNbqKx@3SN}ad4{RH5e&Fx3w
zhTs2ky9t~>EpCobIC6Mb4Qk6<7Z>(u1LQGH_rVvbPdH8Ea+AavhCPaD8kY-n45cF`
zl`hc;ThGFL6yzXvGCgD1W4%#0)%FZwkJE9Oj`8L8abX*(pmrwE=ON8*Fkuc>gD{5Z
zoUV@5NPciwnHm=+_w=z4lg#m810E&Dg*{abpCfz^1@FNBJKku(G9LFy>z`468DHH&
zQKVUf*OzF{<|G)7LU<(NIY%N5?GC+SJ*pkufCs8^VK2jA={B5mk*1>$bQ}9W;i%(R
zlz7A*17`_viVdoLDsm9e`It2Z&g}r6wZ>roIgSB;|6^y){lM94je+wpfLDP5pAk0~
zx^KXHL7nI0!f<}4YIgzMIQR=EVq)ie1Yot!-;jRB`J*OM?gLLW#4m?>ryI=<QXDup
z7ihU$1R$s@2B|UuI5!s-K3AM2=)Rpo0l)td)Rh7lFVh#_=4uI`MbO*qJA(PAX-;_D
zJ>aMV&E(h>Xq|12{Q&f_Il_EgV{DFZ3rGC^m*X_RQ*4g&0bn@nZk~`P_|>B0!tnkj
zaDCF|3SnH!Y_3HBR@+=LIxg6xIPrZlApHF=*KL5e+FW4-uHEHaFVM&ByMQJ&<HGi$
ztFcu~riSJ^1k{&puI~dlZgYj7b$w=Yy&{6+SBtDL@m=2m2dj{&UHDno^`Lh5yMU}Y
zZ^nh~Bgd5{N6k%d@UO+t;QL$M_$085r@J<QnifYihtNW#qFZ%01WtX6qdOfyS~<s1
zSv`QhpB&3<j&Q#2EQ_NX!|Aq%(`{fp-D9l9ir*1{Y#5lJV{>CH-S$|z7Xw%rbZq<M
zg<EV~7~W)N8jc%=?dG8!P1U^?X$=9Gs(UliQ+C&qsLcKNHU8t++&h5uyxq5h0A7GV
znDTWl@@GDN>LxIk)d){RVMz`!wq(!ZHp%^dD5=Mi=$ZG@9|Tef&@(3WjtiUK08lCT
zUWR9kr)R8pj{X;MV%T)VKK0JofzsWCE(Qao83@PfeI4|MGrOAJ&@yR|<2ZpRjcYa(
zUuQZyjhXAlO5i?1fa>@VN!D<x<4dHAy%~QXF4+&GF@OK7jvs(@#a71+0DlBcdjD<V
zdfVhWhg`q3xbltA9?jy)QwvWt0dVy|Ii7fH5DtWbsiFAyACvYp0aB{P)zcn8yP$y`
zsKz082I9i*0mr&xM(C#}2dRh6aRh*2Hb->aGug7Q1My&`IUlNEUxo*3i02-g8$vHn
zdFXYZv2fcI7dDsNUL#F%TZ8m~&Fv`w8*Ofw@}8Y`|1zl51Ni-qpYXg49Jp4Qu^!_%
zprhseLq{Otj<_(qmQls0*&IIs*2gwS44vnq&G8oiKiM2}I3@>3qvDM9+yV~1PA(in
z+}Ke%gA9!WV}#?uxUl);xXI!e5)HH{i(`lm0AKo$F${rCha}q^#|TH#sK_y-6>u_a
zj-3H?3>waX0#OTo*@W-ERteQ?u9ztycIXv?&@03ay+S4dutP79U_UJqL0=Iv2RL@<
z6>=W{_zP>29C%E);+Gb_m`n@F)WCR#6e0!ezCH}#LA$S80BpAVx=~2r_rHDJ1)Lpr
zUta?7Vo<vWjsRJ6z7@xd3n~F2lNeCQyTE!|7-Go@`3UJln<EA{<f3h_N5uX<tYEJp
zn35s(lnnU;!0+XKJSPI;mwj9Yj`i-gIEG@xLzQquCJHIiWC;Hk3#hi;#|uINzyIlD
zD8lW~hE^X#n*&J4U9IR0!kYuvr3rqyq8fI16WR$_9S&maUVmzYI0MZ8(B4QrZC*Fg
z5c3~@|7A7|NJDI9c>u-%gRZwHfB3<N`1UZAF`MJUa_LeTY`X9-ThIC(FyrY_b17d3
z+oO&n)HUc)$M*r41CMIR&O0p63NXngeky->XcS-8M$_&>uSW2KYyFkcCh~|j^uN)3
zemX8}WD+H?od3vX07|vl*bAn{w)T{1sn4L`F}nH&kLs9!O{9n}Y<w)vP+@-v$DBJc
zV?)s>bS43KyHFT!=wi)oaYVzC?VKIkjxGMS8{7{Z*s$4PJ%C5pV8m9ixGoLui|vdC
z4S1p6c^~e(bdD&k1689QcC~dBsxp+JraQx(SELkiYJ*3SdMXy;WuZjHT^&+57KM(`
zKcwuERW?@;qsyf5idBgF)~DkZ>u|I!W0ACjbqvNkxE-;L;eIe+T53H0SE-GNzZ$-g
z#tYxEjccI@QEKQ8q;2TRj?=)y(F4TKiA{?DG4_4Ox2VPh3VoaX%5kIQH$Z2RKGC#>
zmXe1c=GIb&fTAll9a?B88$s&%D<rH`rrLhQ{0seodJ|^jh1rnU=9SQbQHL<Q4V+tK
zhPPS@voOR=Vd27T$6~OVBh27^z$_yYO|c$0cVK*YFdv4`+BtVTh72ZASVPg(T2ucG
zOR0ay0=ydrR5r4Z+L{kS18_K+8k;EE0zuxh+bcXakt*W3Y~G;QM1&xCu>}yjWyi)?
zh>en?^0n9)gmLv&Ma{&$I<YYcY!bp#)7Yw5i``9-pPLX{4SDTj&4Af0s++P3wmJkd
zI>yshlQyU2YbmQhg^!$vA$ki-ugBxJ6hBTbtBwJ@6lqai-n*C*z%hK%X26di72iE~
z_ChUXbu_u+?Vyg8+THoHXQ1S+U<ob`s-&OGw4o0${?bA|Pk9phPRxNvN``P@?;#x{
z0Q%)Xe*hTduun2j)0Ne&0ey{hi9jkcQcVDZ74|#QuK<$%M&=IEZIp(54qHfuCkl^1
z3R!{(b?BA#om1Uajg^r~P2pJJ4A7Ks7@dk1@?FYSiU->h@pz~f9!~(`l>-njJPoNa
zYp{I@Z;O;k0AhysLX!SL%y6>DSCd4{a13_%TmT7_GrRz4IpzG$ma`Zog-gn`VVVCg
zF_Z5{&BG$*A1RFTlh%^(t7rtz6TXE&DKdMJb`fCd`$)$LV8FvaM><E2e~=^QRrvQv
z-w^m187MCNHrbcKGBKWpWBOxCP2<Q;U#;+j9EUgwnv)W+qA3q6&{U&j`T;1RG9H51
zi74)WZ>QYL>+DJIM~Nxjn@Cf_czcJZEX3qax~gHhMzoAvJ2==&{P~tYKkz36gwWP#
zfxx>{N-ocYlc$=LavPHpW<}$^DY-34gZaTJxt*+h=p!lPn%en|DdXB%`IV_D@^mcm
zH!MsU4{4=srNt?Ef)DAy4BwAX$$nEh_^V_;<f(U#KNtCfmt<MuB~OKP#|EbK7U{k;
zrR$wc>5#{?=-u4n&^kY*w@88Mn$o);CmMJict52tdHsVtCe|QxVp&>A{s(;Gj9mWa
z^Ji)~-!H#|lN^3Q51AQVrbb{|Ri4Uu-17GZ(=<I~Hm5plGXmZyA_Okv)Y~ZJd}QG;
zq#-E$c&r{W2Zq2`N=M`&jYDDliztk<3TL6vnBHLC{J9`{4<=B(RVeWyEeC4D?Rv;O
zyFE-Y;z1Pp9eT)oc*<&5P=G)wVp|hDob$4#TFFb5yWbvKrt&f&35GD`iU*hVzR^Q_
zM5DfbIBGg+&W$0a;W;`S!85gp7pBYsI+pL$wxFNq(nBW@y%T7kLOSUxQ&Tpx888y8
zrX7!IFX0Gg%Bz^(D01DaHG_3d6Gt1Xsmbm#H4+;O`Gu@{$a0u}>X>I>W0R{ef}8;J
zAB9z~Od0sYK1&bTO%z7U6J~?$c_f~QPRK{*D$=*8o^T&y5&-{d_i3ti^)fYjJ89q-
zD}^3%9Ls~Mf(_y5BV2cyjMGEjL(tZEHMH1d90*i-4Fu53$VwEY%6naLUS8!>;6t0Z
zAfOt%h;$D5r;#@Cz^}@+GBxJ5kXsfP{w(*!RsKTdexyC5y9`aLLeY_DkUk_3gUlaD
zzY;)zPzf`(61EAdDx6iN45Tyy7^O;+;jb$VBh($x^GHt<7=R3XZzT+XguY4J%G6ld
zJ2iP!4|$(``T>F@?E_93($5s+QM61tjBB?Jl-yZ@76Yu|n*O8hVK&<yCf)WhMk!dl
z3~f-IV`oT5x9A~fsLF9W%M?CNN>!HXAzy%g+F-C9_X?OaM&j#bnv|k=+)sc8BXuWm
z3mJqRQTGEFkC_*>AL)4lm^V>>B9#${!>nqAZ{Ago;%!>vA@GEnkfoZg9>Ma^q!7!}
z_y-d3^SIU;zwr1?K1TU_Y}*@c14t`j)cNU}^TwU^7n`{gI_E7#<u+h_#jeK`Y<^=o
z;scep$|}BT7GFYfZ4^JuV$|nHYtDz^R?<?}cWmyC_e{AYBqud@*EMQrjuZsM<6vo!
z;IbYX3jxwjI#Jvc5^O@+KmgOe^6N;4sFcfYl)!W=e~EOyHGnxNy@7^%S#vK+VIh@&
z)j9xLS*9kgMbBY;j+E7i`avNYHp9cmOv78I#v>krZ*l6OO~AWpUyMN}8bqfcH6rjN
zGCh#Gpk<m9IGO(gG#P0^R{#fv1RykK?3Rh;EE)j^9tr6H=soa1V)9p+AENjPq(^Q3
zzoGVo@UNza;&w3Aye3%k9qrLr1f#hK@{3y0?;yQJ0E>Jyj5qod3R4a2+Q96iVIk3+
z!);*pF$ni!!f`KUo9@na=QNfMLP>3;>IB9i(+nvUfYD|Ka($8TsVqlwLr8N|q{-4f
zFW}g^8TO|Rh6FKFktUL77WA0h4UmDfn7}|}uu#W5ijhdp1<H6$rDw7RV-)iV(#Nnd
zNAf+W!W2kf9tU87V_m2V{&CVA&G{fKGJOh+2W6N-4p4^WO<HBSQ^zv6LDT(_43wX?
z2n`-5hdrSF3({A@0S+h)#+ZJRRd3?4@{UODNb7b5*W3})bDDGU<6vWS1>!2y!!R@#
zTK>eA*kD(I!BrWgvHDb*lIIyw-7J#Ta79&SB28m;N*Jo4b+&E>l{!+QP=%GIN&)A_
zq)5}bH|t_~iw-xXc&AxxG#hD^<Kr2Sb8_Q~at<L`&^Tt%;9iA+OUD^3o}8mOr~Lq~
zPoS71Ers)~!aWrWORd7UD;AyuRou_7nw_`Ft`-V7pF6(<#SEcrvQy$s3eDYiA%vYW
zgN1ehUTlCBsgtI5S7iogJI#hc%8|H2*KM(;n)kK1v?*1dDG#}(KIC$yZ^!qHh`fj$
zWTpVi&{u4QEr+yPWwmM>YqxTTD{XrsB^|jP%_f2L1{*0uieI(b9=417*lnGeR?AaX
zOXG}<Gf|pev6{lAi>BKauy_>4wpDjcwFt9|8C3Fe4lALW53rkTk!V+L@2okx4_E0@
zn*rZ<8MWO|ix$>W(e{8;4w!A#=KZ(!RjI{X5w6Q@!pG3sBD>bAd)Th~M4CKf_N&DX
zyZDk6A21zoi}%*=ZR5Vn;6hY?C1f897u~u6q`LupY^<?>Nvz!e2&ZA_cs9<hW_tIG
zO2Bb-%5G3sN)cvoDadF*PB8@qZ^}}DOJ`<rJG0np756duWQ-#pqny!z_zH~RHYiNS
zd?JOz(ZM2eG1L1kp%_>+IM^A%9^aPwo0@amY9Nl&G_}8&8OU1{&rGc59C($P2AXqG
zRnQoh4jLi2i6P@0#luW${4hH+u9K9jjO$i$J1Ro8kh(r*UCRioE-PM&apuLKtz|u{
zxP7z~i_b!F8@pJh`#6pZv4^pk9}#JPTRGPBQJvU_@$pdFmyl!p01FL|C5d<)vLJ_J
zp++SkCc}mgD=^x@Ba++r6Q;&m?gKuC@02}!<Iw<yf;ozNaKRbX(tg@q3$;bs`XlY!
zY50;JqFROoc-<v#5o-DSD#Qt`>bGO`{C|G9muVF$^=%Nvw=#b;152u=o@QkdT-EY9
zdI)cWXtt<Ko4p-``!J|)bu|eCxd&A(5q*3CzqkQueG;7S^Dm<mt2{&aDhH5uTfzF2
z2s2o(auS6n7{9c-3Yy12Ggc3o#sK;h*1}eQqxvEeZVy(iLy<vjROLFlt<8qTe{h9b
z*{Y`7(@~qIY0hhU$S4YFECSiq)uYYr!Kherl$R&yA!D%2L8di`3f_*^Lxy5K)v`w8
z9*F`yB%@N93Y&^<r?YdJ`RLqKkj!XHK+~$~NL5jI|Fe2XON3XMz^&R9sXYq!f24=B
zs!o+-<*7>a(Fm7^f2@aC&r4`k??<7pfqcs+q>U+uR&_fHp8+Pm+!N9^4jZCAD0mHt
zZvo-<By$;!x%(;DL>Q*@xE1Ky1rGW`v6HaPn!=lERcpF*tIP<a|A$Rx)<7YfC*HQq
zr#tLusxv&K2aPif&P!Z!I3~%BJHYQmBKN`?=>cW>#?#WmN3b4sp;&^-@SPsaj0zas
zSoh~6PyHW6oLcP{>`XJ4iTTCQ{4!^@K}`R!9+Je(XeOUr>v0HBI<-Y$Fo5<lNc`Ba
z9+HB#(VyddC-XgQ57JWf5S-Pap~j+%|1IJjofR-BO0=OG;q3D<-SfN*FPOGj0zTMQ
zdY*4=0oYD@p6_k}*c*EOhqb-h6}fX&_v{3^d6V$dx@Q;e1Z%^mV=HgdJ$&6*y<-6O
zJfnL)M|ZU}0Bf)4p7S>Fwy{z7To8!jG+TrpEe*5ih2FZSAI2K9NDV!KBd#Mfw01wG
z&Y3n-Lsb?53_Yr<zH1taX8?A}oJ3WxA$^a=FXZT+0WfJP{0D`9H^`rJFPih~54<0s
zS~%ukHBSQ^+Ij|33YN7%ifb?jMT2Ux(*R45R)8QrUFI1KH$~RfzC!v8h4_Y<X9x>_
z)3m$#A@Q-LdphVI-px6pHwwVyuEmy(y7$n?dQ`(!tjS<Ro^5>%^071b443W$aSPH@
zbZf&f!Sm~Ds_o;nw8dh=!(d&s*0x9fueJ85Cf4eiKx-W~(X!TI;`e}m$-*~1{jk**
znSTtcJaZ|{)SM^7PMEdrI3?(6QL|)`i1edXlgb)&3t|q3H=#)F?9k3+G2B{A+KnM9
z0x1xgFGPlqw;IerjbU|c_#CS+fWq=6X}A^hwR_4c&Xk2@_!SHL1*^DCyoiETP((Rl
z7fU2GVyRX9ja}S{N*b{mN-|c{?naE~&l%X=#4_FeE&yJh$ce`uS8l+|9+sLncZ!Cp
zG}<>t=v}0Jsd?!vYi=4&_4=AtT~EzxR<UdV%5qZknpZ4ahO&F3^IEXXkjBw@89{{T
zyp|R6tt#YOSIB2p$hWDGZ&of*bY8P^Q2><70Q;QrnC?CX$7*nHRR0=j3rgR<p}UV;
z?zYD7npWegWwqIp#cFe65r(8gGl-nzSJOguH*R%T?K@#5>eofnYSu;4dHf=XH+h*U
z2frxvbb)JXKg~wOLRhnz!To3U-DRrhDQ>t@Z|I(mh+Nw9E|zR43b!?AHM1d7>J2o`
zE&`N?G?u`6WH2pi&ck5#kH;8Kg2bBYd$~;Yg2%w|RJiUw3u+zefLh-_f%-VoL!j0;
z90d9~(f^t+p@_#q{i2ci66rGnamZj9t$7_Ty%!AjVIGW{fh!^k;gkJynHqA0Qa7%p
zd(z2=N5VO%t%bd6t;*z63&e8ZZEC>{)vA30{*`ACLoNZSn%iF`1ODGWe=vog4u126
zXrk`^3`9F7Q~0H5-WZ9eA&qf?LG>*J)Dx*Ifk%;<h?GlUD>6`Bt@{X+A_J|}@)J0O
z%toXq2pmIZ2W+|4wyuEAfG?`Jf9n{^>a(c%0O@_sm0mcQ$mii+Zv>HC-s~l90h)-?
zLs6F@a(B?(_-Ll;=%?J+ZrA!8=`7{$K)P)KA!4a@jX*Z|*Umy}PTHm11B15MS|)?H
zjwba!Ys*xhSQ*mMy8BB~=cV*M@RhYEAn{@U#-l;7&mKUtkY*5g6`7?-i_nAK6Dc?{
zcwM<|DSaZb^X{`#q@Jg{zXsh-y!P4$=MJ?WLMkR*UeoWB3J9@TZCXm7Ok_?X@sy|!
z9C_`pk$7^|XAm;5v)aEwrF~|A8LW#c?KG0LGc`>QNAfUG=mXQquTTFgCz>)3Z~`|J
zYYvzix+fFjbPk6&*$2@k7KtBuZ)|{4_S=A9hI%6cA0Y$Rr4NEP?sLKPb)DX^d|jgK
z5BgwdCgICaxMdzfb^6J)XYj{fO2U@{N;MP<#(!+xs*=mqz25<Y>zw-dg^SeP-%`c}
zkg@lt5CB7=(*k>6Mg}uezn8$T$UJ~lMBolGn78^P;MF%;a~c(xK@T=JOyk}ofsrZ7
zcv^Q~0i!Owt)TbaX#FhG7Ak`WnY|YQdL8Ly0zPCeB5@Y?eh?W*s`DtY_a^iq49QJk
zCo)iJtWlfL%g7}nHH6#Zsc}E3)BM#M>#kIs%XCQxZyp-Qwm@n|0D)^P>@&7KDDcoY
z7EUKNhqI;cDh8j-Ah^772VL!lO&F&Q;z<WI$HDGh7sU@_{-YcFb@w%P@GLvn3}l8P
z4Q2;<M^^6+fZ+9F#}R--jD;b^-i@75KbTm*QVs}yVQ9LmYpvcW9)JRDisV;{vnzpe
zzg#rKu*5DjCFu1nPR9Eu$+UV)cmE6tx;6k_uN%NwjpU^SZ;>f>p|MbU>{A5b++yK8
zV&6m$`)>x_CDNf@I{<DMYNvJgucSs_-|ID?euZ?A)J}syulE4mLi){(O=vGzP8{|N
zaW#ZW&%81<;FxITXWbJ<D!W0Y=R9CELE_^njeiD}o(}=)gw%?_W@IoRaZnR<+Y8=k
zz)p(NL}9j($Zd*J8yN5oaWg=qiJ~a{l5`w&61R|01~M?9xRvPHfOMHDT*-PZ2bjrA
z4fw@|PIq4i$!^U_axKuA*0BAP$ZSA*Tv}xVf0eXag;q(r`xaZ_t|irLB#@p%+Q!-3
zYYuYzkV?t+8GNE;3p<hXYnd9TQ4>un>z+uo?LLQ^z^WB@1nE_FqB7d|j0W^25|zt`
z3hE$$sS`mU1(_tIdIVY{la17wKsRKjVg0L<$EWgof>9G3+o_)LV0G?CDugo`h(M6e
zqI*n*#m}0etUjk<!;b2PtiBh9R;C6mvYM11^aR}tt8ZFOMz9Gcu7Sxh7+uT7=?>Nu
zT+N-zH@(3)pa2{x4<17^#xE|e>z>L~!mCh1&T)u{MX%1okU0lSRUIsPbv6<B1{uuf
zI{Tr_!D}%!KNVUzfG!BF3!pVe_r!x%kJbi>jRN{dNW8F;GX<G1ka*2GXFf8&BK<;O
z1v0mhZV@Qfw7MZkZk#aZJb@Eib_BgMJ7ON-&I2b7q>dFI5jj+xt$UKl*GslA=ek3y
z;D=}NO|nU~EuaJ>9n^4T9}5-q#O1JM7HC0%Igeovi<#d;?$ChYvg}pU=lA%)LT<7C
zR6ln^Bga|`dR(^4ONrjIv4#G(RoqI>Y<l)Fi?gfP#c~!n_!OnUD$~*`&fs+(Ys<S(
z_z%G;gP`=WP+HEX7c_=og`2QiLl1F7g36v;6J1D)<~BEbSR?ETYv}HsktoZ80cLND
zHY4nvNt(HXWGuQHifOjUgJ-rj>TWzo%iS1SE-%&H{3Msz@@K1MNyV1ys^X$ou<3)+
z6#C{Vfrnqvb@YfQCFgD3z1*ghqg#}gH>n^)fyhAQ%x87?>>zTEg*-c<0y5udgoql*
zAO30R?$JT)?H2avMisEf+TDBfaowFui{BnG5gSi8F5;@3T@e!}EdVec!sqPs@Ri59
zJI`c*2|j!Re)_$KpHq3@H-4?|&JMEs73>UdSh6vFWkXJN_O$s?oxOtUoSLn>BkiU=
z^DIdtE4GZP*pjob%@PQNgBW)-{LQI85pq=Hj<L~jnBz98V728lVrDnmv48tC>SnlK
zcl{M)xZ7mt`U?+uwlR`5tyWvz^*3B~vqc=JY1hFC(z|#kh&`gNe}dAN9@JeIgT$^_
z#4gsu(qebq^<~ATmx4{_{iD0yx106~wVJ*!a!}5359zKKg7saj`WGtLnd_iUc~(8$
zg?ARJUR}7M95%ZV>~F@yx{F_h6Fcp-+-kqAKKqG>y{#{MS>4+%l*snfB;B<p*ldT@
zY)wVBYlFxm6Li<yAo8~s@?0Pr!S2tug~YhK?hST->}=gN)Nb0luGMsCLyBh!HN0ZW
z5y6(l0o|1qY&p_unI(#|T6U;vR2VMq>CCzNWU}tO9c=oJ+0=PkbYeDj-l@RU`LDDz
z?aQUvHKF;sOI3%ex}Xh=&1Jo_DqquG&T1MT6yZm@GTzjk-`ULiL|V+gtI$K|6`Le<
z<Gd=&Kyu1*-T8yma6mqur#n9ilDW?!^HGKFI!{+5^NEl#Orf1;qzVr&>dvz=BP>7E
z=YH`+>3ER>4uG7Wo9t|N+2?+7m;8vg>T@52NL{Eqp9>1{yCuYP4a!H$`Fuqo{%433
zm`}ae=*}IrK}Np#J?$p?_fod*+zFlcbs{FDQ*_JQb>O;CB#h+)yMxz&akuTdbBWEk
z?@){J5@8JL1=DorQmk@)<y+o8GC>`))w!(V_&b-2EOy`eP7)ST>|5WVWU*3r2Ax;m
z%NC35iVCjiyPT%)vh>@#T)%xpzmQ;Vg6=f>V&#z|$K&|qdEZ(DEM1SYbPd)IUDut7
zcJKP7TC8!KUT_jPlPjt<#Ws=EX}U8t647reC%pDNXl?AQJN|<8Rr>-fT0dhz^ejBE
zj3R6vq1iT?_SRFn<7b}i_M0L1o&t(T+fq4X@B4%9xC)zYwrDhBxi@sj1>AO#@(JqY
z3PQiVIM7civ$kh69p3<Z^JeZUh9HWF)E#e892{R6*^G4Lk<0!+n&(%Jw_31Y2r$|q
zwU+ZM$6JUCdGJ|x5K=!mzbY@AhO$X=epOypfU>1>er1)(@l`O7<15J<xWP6L=U2fB
zoL>d=IKK+!aefud^ZcqD!1+}<fb%O0z)-@(9Pe18SoaYK^biU$g&psjmZsI+%x*qq
z%{BAJr5<AtR%Ihd8=Hbih1W!&K$(PN@R&Jn8R@M%wprTC2CS{-q4U6irMzt!mh$$3
z^7E$hwq;t%JB_jrP32i-hF13)3cokCXCuZVe6RbjsXepI)SjKE_Ut^BXXmLrJ5S}=
zc`DD&i}Hd1K9X0VirO=)%+qqmbC&kvbxn(}VX4$wgtYjkC``3f`n;vm_^v4Ih(b7A
z$Nyxip#WRe_`#N%2d)=w&d2>>+0Q`#yxnQwNVMp<S9ct!3H2=j_IRXRMxxmT915T;
zz;MX!-yDtkJc-C&%}q2b7J!fyL=Idpo})TF$G`xFXl9h|FrV&2i*z8$gWQ95VaOQ&
zIPgIdG5C0F`{JjYW3~xVHVr-=TfO)d=9mS`Hlpk?bI5{a`%$*X95S=a95Or4A+z%w
zGAnQ7am?%rj+ve3nAv$5vmn4B3j!Q66R^}f=rvLA1Jq8Z|A!Sn5Y*G4*J>jE(9&vB
zQG<3tQ4cwE$K$qb)8C1r2A{TU`*F)L#A9zB|E{U0$1TSY{}swEntBSB-9*`SQ%`nT
z0>~#sn0i7P_`?C<VB29L$ia3nPes|~)RUd3p6opJWamXsL4bM+0@RZU{AacsJ+@O(
z2s{6O(|x;4cgJtKW1gjw9<;0LobE%9TPneaGsScgu!&BnXR2hLHQf^ipsbInl3>|X
zlua~M!ZP?j6civ?&eB9QRbuC9J9eHbvGY`kou^9dJXK=nMI}LiDhUEqiD5$4bidb{
z?g@`UpogvLzQ8WrgTfuwbYEyq_k?#*_@*`87g^Ii;XDdI`|or=g4I87beZabyHyaU
zT+<ys>|cf&U9ic`n+g2ykS=o<E?Ad%B+}y&^zG6eYlz7+S3$&9JK-04h92Ea&oFc~
z9l*Vs?Wd1OWgmJ5wA*O94Jmto2W%*N$wHko1V@`waT*UB=&=*@>GI{I6_a6@Q+G7B
zMxIZegp_jR$KGX)eB<Wk$k)R>Nw{T>eB)LS7z0`l0aLwE=Ew)j;GycxHAg;Jwg&cJ
zugIJYW}!LqcAg_|=Q;9ro+EGPIr4U%BX8$r<bwc5J_vB+O~9~5KFu2Bdb=Tzb(P1_
z)GoY=!i(0(^BM*{bbTxd^=mZX7cF&1Gi&7Q!@Jf`i{R@<Q5Z#SG(foRXdVFyIN++^
z6RAzIT>5piKpbP#JBDKs%T&fT6xVgmGL;TTa6_J!QmQ*}_@IVQwG5#C3eb>;n{#8!
zRL4_1h2^g{LU;7WLCXjq5OdoA39hbw5rH&hHXuEM<CEMT$Q(f8J@vW9VC3FHdV|n-
zWIjPU(Ha1ZGUZK><N0(hR)+e&BmIgC49~n!raEDrl3!lja^R!xYQzBC{`br~z`28j
zy^7<Rj|Bs-g#b1TIjVJ)`kHgkK7rk4DIOAzN{{S-;<s!xJX5mgZMa^jv;hu}<n*K?
zE?Vx!lcOUhnx;;J?G(u}!i#}~f{y1j#}?ilrEqQs9X9*{NPS2l<)n@mvS@0|SZJe>
z3xpw1gDL=whPRM|$u?+?!A>~|q(;#g5&p+I)@9)FnZXOoROfD*6SKBTY>hBYjmGnW
z&cPVX227J{0N4zaF<K~4hI$;sps-Q6E7&<0#)LOHXbr1^cIr@cZ15rM?PnnMB5(}j
z-{@yRKM*>jX^F70L_F}~9Q-vhIY^xd{DRC{q}5#k+y*PmMhycn66dkbA=o1);>aLz
zH~Q=xf(rs&u<U`~R*xeJ(vExuk_^FtUE=#l$0^CyIF!tn<2XZg9biSvfI${#@IzLV
zsjeEw7rzLsJr2CBLXG5OmP1NF0|$SJ7swf5Kq9<RB3^Xq9P&Kdvo&r8>^fCiQ_N_O
z1D^*_qi})FIiw?yZzJK2g3cj**uX%ShEzriSfrK%JZfo3G){scXi_BtI6Fz|gw&SQ
zNAxOF-QlU^SJP&A9C#(38udD;j~E4{5lGO#bHrq1@{#fg%tZzkoP-Nr&JoLzS%!ow
zf6ftOi;!c&r9bD0N0C9NlAfTq>=J68*o`_1lj6i|{s~X)y5K_RkQb5Zx>-){G#m{(
z2d{%#_o7#W9~8B!&Z5=z122VOi`NiONu7o*&Y)ArFHAbNI-<VOxVS3MVI@5WrrRI^
zpZ7{Si1Y#<+!{I&=$mD|IEqKXVZXT2<s5ns>fb~<!m&L1EZ7gbfZS&u<$<3V{ci>e
zu6~c=;qVHnpBbKgZWImwgFjdK!;1{5xc1{50Xs?h80jQ|O2}M6!UH<a5wOIhzmWbQ
z&`{HoqmaT0G(#o<sV;%`$TUX6@=xV-+79^wuy}N3F$@)!h^vSAW0z~(g&uTdm+wc5
zc1UeNg3piPnOiNnr%dXN<<^Kr%CQ`)+Enw}ABLpE3G-xp%srMHVA+pWnXkO8ql;@9
zxbdzK14|>vvtzuuT#k8>i1UKW<vFRGVIJ&`dC#nrvPBl<FD%OFV7JYh^CoUaR^1Mm
zce#$?0WmEZ4yZfSFJaVws0ZIBJGLR+U|KbXn`7_lg~#6DvFmyO3^I4rj=Hj$hvOTH
zRljZvZjv?NH9Vj@Y7p;kym=nk+NijqjmA60&gza(Zh&c1@?1=AY${Pg8wiS)j5%Q>
z55aglTYZ4R4tqy;R6=t#te|XsgBkPY0yioWjWJb{=OfL<9NX;!#g{O0{DtatM+|20
zSUlqE-2D`Aux2MOBk&+HYmgp*wzL#Xhp~5KsR*Y+u-J1a4s+Oz?|`5H#N}AC<dClZ
z#KPB%_tfPsV~5nASpVv+Lbs!b1L$BKi9uF%6PRikzR*1pXn&y_a<++s3&p1vn=Oio
zkUl6mEJGjq;ItRbzJ~YD*;l3|uLe`&GUJwUS@1^dJ*dmka2Bhk1H@Ue`X-ahxta#>
z!A=0XqD7XP+Ca`@6))ZLJU01Br1gB{Na5u{65}Mpl_wt@L(1qV%An(9UHue=Rb?Eg
zC4Y$Y0hgF@7^&pXk<PKIA2Qz}eZxUX#!@j3=9zpQ>1U8=ghdES2p+hOTLE@W3sI6u
zu>r6TCv1Qse}J*ZyDcf(ho)>GGW-Kwy~csf!h7)W_(-&zJ*(G}sWriU-NbE!uZ~sL
z)e;`Kj30%v77YQVqa|LUrZwxMtEH)|p#z+jj-r-7=xSMg6p2rqlJ3#8>@&LJ;Z@E)
z^R!9v)vp$t0`onP5je3|1tL%c0wz^ZnX-Z0u{d3=gugc*shn13g+kUtRF`8cirZ?C
zqHNuUy-4OYDPd)9d50rd8NbAW(e<7abv486Ix4nB8)nyMBn4%fg>E8D#jrdwSy$sR
zf%uSuc?vZC%aJDjc=$OO>2w)%zHFO;Z3;S;Qi;bPb(OA0xLJzU907UjMP9hodIT(j
zt#Kczp+$bHt3iSbCFiABV=+ijNb8{&b=B9Rm4x+COQOZ*jZQTs=?gWn?<vqtN;2<E
zeIQa-csv-)N$@_4V2<>VCiLodUDc>&ov|4c5-fxosiFe8DN@yf#-cwLQ>ZPygLlq*
zAFSF#C(Qs%NZ^=HpToN(^S;0^<F-h?9n-pTGVhLjpk7yPWoS^}r7nP2k(JG@J1KF*
zZnm*u3zXA}E5K_eLl?&HFg+j8zk;8eFf3~3BpeEUg)Uex@7KRd6v9|<*4NQL=F^rF
z_?6T26#e5kdJp7NqV$i^L+!S_%u)X&LSpVK2K?T#yY(Zuz!?4*s^y??!l&@z;V+p^
ze*#wob~Y)Jen0$etJuC&n3Tk|E&LRCeu^(R;B`s&>IW;St7_^D*zzvDO#2ItN6X}>
z<=2NV{L5NR@+*U|LCOuv=nwzGh2P*)e+?mWU(Bc9^vA?n%e16pppqd}a>vQoKPDrl
zrq{#leq2Tg-RyzXNrNU%jMGO`6Zz~toY`pTODmN(;OnF7B7crp)hikLFhM(MqT!xN
zpshbR)!ovVa8w^+8WOx`Ool$#ga65+^g(<(Ny=G$ARA8fpyAF)eITa&L=R7LV~(Tx
zohS7HQq7*AdO(6yqh{}mdVi_uh?-UfdjAA@PcLe+ck2D5W~9^@IzRM{SNMW!sG;{I
z_onmoKK!45RPW9I=i~HVp68(TO6zrgDpFggP@3CV&tdJG!}M(aJ67mDJZpiq^xE>S
zn59}4)O6yPwY&&~nX4v#C2wC{yOypxOWw%8c5MqRW!CFg*RI97gYSl*RjuN+-1D>5
zq;QCN^v2o+nEPi=ie}d?=qYtTth{tBZ;}v1e(M#8GX#-TTB@}p?8<It<%qVV!pgpP
z*7jv3zR{;9EjDZWwqh;vTQ08c+m&QZe*HvZwtD@=wPx4QXp7mXYlTK#rIAJ@yjE*%
zGhvQa>+V`x(-!2M*{WuS#mZQ6b8XGec7OhNU~OcAh^Qw00nBda*G9IKN+@_y)5PIa
ze76aS0~AKmKQ6Jmf^kf)+Z<;FHxl^<3k)M_#;;6lD>ZFEqsg<0ZDHQfY$iU3`D$VY
zq@%(n4`ZeAY+?pnI1cv*;}f%|5}TklW=xsZzZ8XyITI#-F`qB`Ol-m~l&a})12y_5
zHi?r)TFL}i=@^*h<S(R=!)lt8*px3`nT#bh1!J6oa79h=AXAJ~$eLToY=Li1*$nSA
z1u~@ULfRp(Cm41$u{H#O&s9_ELd<?O5^LA7@(`tV5}BhMg49Om<r6aKXDmcS8%~co
zg=oLkz0Mgr4F@M*t#ja`0q8c=HDwAvhJykKJ!Og!F>4-xzheQ+FdZZy%%dp?@<FKH
zG5ow+A3s%!d*Q{Vw`)5VlL!x%D-8w!-}we#Gkxk60LcXYKf2xnKFZ>G{C}RiG)N2~
zTyl6)LI|OS(34;Sgk~tA2}p||J@ilmQjR7?$_J8=ARxULiGqTFh=LSBny9F#sDRib
zVn_YIXP)N*`2GKQv6p9OcV}m3XJ==7@c4m-$1$a>#1a%ozxjr|_kQMabWs|LjF{hP
zE`Gw?Oz%R<67>*a!d_~(#zEXGHOCGb1ClOviLwyD`?zz&eAVM|bj2i-_jPLA#oYlI
z*&86$@<tw`ifEj4Cr8(M94wL=gL7k)o?aF`a_NOpGc{)D*a6_rP}_j2t~njY4c<jY
zmAhm)*ftmi5Ss;H5&-A2^Z<k7`mu!)nl0wLPI^<saitDYw4|dR2l^B)4~1Q$J&q;-
z2g@}rnQzn0c^s)CX{>H3LCuBbJCF@Dw<p=-2o@b&SY*r=$?GA^89I%^NjZY+OF5wM
z1>BB&6WSfBLeG&w)I*Naq!_?>X-Ef;LznZVM8|M%WEkee7K#(u3nAE5vKOGPhWt)4
zdl^yY#5v3?m=i}3{R-}V33)2LX^}C9F-&$u!OT(q9`Eu?jsTlJIucZ+xjNP`vif?=
z?-fiLhkgv#m!txrX_P(^CR0#&&>e5&kvVm2-VPE%@dR)1n12$TW(S&CCp_k-J~YGR
zy6jq8fK;c~tZt$4nxDb~ePK!P+*CZmxWk`&%yTwj(gztC!nk2yE04@~C5^=+^D^JU
zqooTy=6hDwM$PXj@V5*&xTnjQlPUuaeqofylv6sxau{RC6{0n}BCs`O_#t<Q#`)nK
zhRxbJ>4t+813c!H|FHHx&c?Y&60V$070y1Toc(^Z|0CXt9`j8t0Hx{is>eKFmnP;c
zgy`)5@Lq_46#GCqDFc`);q+#}TC}BWOuy+bHw~FAJwRk@!cog{dP`I?AI)*8Drpn0
zc}zV-f~2J)CSJ=L8PS`m8@Z+hnA0!<s5#Dsn8SC2Zs!eBBiC`Vgd4z&piUCRJqZK@
z7Vc@fQ^LK}Q&8yVF=Z3b@GpI7Gl@CamuwQAiN&cdGd3WZR1V1s88wZNxh>6dHZf=V
zk|F(Z%Vd5@GrMje_yMIn=8M>QBii{=tRd2BA+i(`Av<rIBXkUyij6lx3@q8QcAkWX
zzcn`!Yw&*f<~>f<mNa)6=rMJdQ6bViVrQ^u2vY`yDt9Mpc+3SpPl>lYrrwEpMTD@{
zPaboKU2V_cH>QdZj@(VP(R$4x^{v($_Y&moicz#9Yzd6>o&)$caw!!&GKj=d!MAWX
ztp;uCSb>^~`x*BW9FU8So>?ax2wCkhvk=sg!U3)Ie~7~#O1*%m7UpNI5;6}taTzIe
zqh-i8k2#ikoRJ%)&7YwDYoSeRNmn%y#e2_#rm2N0trgXPkPTRl&lHNx%4B^iNBhiF
znQbwi>0y{-R|^mNp7EGHY#z)(j~L^H2cyPX#eLE)Zmrs&lm$yEZ67JEm*UoHgG;m9
zW?x^_t^j#ITZx~;QCEt))MNI8_)#(uEyULvj2q~qz1SnPkHd|TkPUo1gX4Mt;~5I0
zWpD(|kn@Q_Gmw9Z0eZU_ldgEorgkxA9e_Tq%F)?WrKmP}B87W_j9PiPB{eM-*Hl#;
z0P45aE=%*!$ry!{D50OnY~?G_CBon4)ZjSe>M<MWt@RVoA8nST*#Zzx<<|OFnif|e
z&AKaXgkNnV?2a`@CyK<=FUCV^yv&Ko99BF?;@#S`NHsgeaOkh5O6T0;=#jouF+^mq
zsig86@zhCtEn&kfCbz8GtU^-=x7qN_DT52-r8dVLhYg~d!>fyGCRr*j$!0@1VvatS
zQ}agrK#L6w%!=7i)-G{8yo+Z+$+%d~V?ONlL0<&A2WU`BkEz4KWdO)J{;<uPM|`~b
z3*a9B%ir~wvQ=C%tsQ{#i;-y>kI6)(1@r)ntfddy7`OV!DCn^wZVuSc5>a4%3yIq2
zB|5yxdXu$-4TxNAqSvlztsNN8+!Ea?zbq0wl~kE^0p@h@!&w{ZTRO1)FjbZgYB$9-
zCYjWRwAZ-<1(W=Wo~z4jR+h|W?`lo|UmLrsc5CsWs%!Veb@%DVx$~v!`PF2VFk!~T
zOC_R_Ya{Np2ioef_82^)NhM1o#@rQ}3Xza5p)x9R%sPrEOCz<X;-0pPM?5DTAAh8L
zzO@nY`xirt%V)*%m2VzsXjPUHxGG{p^O{TFQSgmeN+r!0-&$aic00it9@Ak9)Ewc0
zs)w<w^Sxrsn1Mi~xrkZ_vg^5a4tXpExOAq+Ea9ut=K-#=mEfEMsszV+OxZ;}mOCm{
z3D(|5oGm_nzKmzDPYKRBuC+ehV}_I@MNTiaA5J`xDjoB|RM8fwj@n0YM^x=eSIwB)
zRvDuA96_mC<~ifFsn9F-U*svW3pOi@Vzic)GA^v57DmP}qR?w!#GS4}%}INvI;~%1
z!TWnCnEsVYr9IcajpG=Ew&ywsXdUgrW^EFbVs)a{WXOK-8;|aEqZ6M=X^m5o>G(U?
zz;N)QZ6?&gFs&0DE&Yw|Xrr0bse-E<-ERP}50p0KuTDK&o#+vU=ZbJS1z#F2nIe>b
z&-0<p`8pGUjkSv~lUgNfeksDoT&pzhs9l0Ol%mc8UkUQ@<jF#Uw1HWE%6!A~8R9?g
zC#$x+MxE_MT5DHVovU~*$?UzRs3!Bev#OCzkmoveJf15~w2tf2Yt^OK(l%2!8c&ox
z<!{WYBh8avq3tHLjqgo7;ci{pw%mH5+^w60C&BRgi0edrBP7Ceiw<Ty7m!-nv>;M5
zUp7Rk+remqn9!fj%FxnO@_a4xt<oFS#Tu&HQ#|yf_dJMH-AvqIpGeIbVTshRP_{M|
z+?niO!8^ut?4cqfAqx_&OI1$jO;s|yI#$6N)zqcl>y81&DC-#dT1Gk?$}{Rtiyl9k
z?&MTS8@3qOg6M%0fSvZikiEL+MQV=x;d#Xt)zrIWrNi3^uRJ0b)lxseoY+o}cn-no
z3EF^Klk^kZ$37N(jpr+wNNR>IvWzMijph%Z>DeqZxD$#HtHUcjo1t7E=A7_(ppv~U
zR7sRKH7M8~%O}&;3!n9P|5wHyC`}CIor@ucN8dCml;A0gJVTXekWXw8BGNosMG{06
z=XoahzOm1y=(i+l_K{~maUxk&q*)BmW)Z6B2f#|5c1d-52GpSw)Y0^i>7D_N2pB+u
zH*a|cG?#Bdx2=(<Pcb@qk{EPs1h-;>qK}}H@i-Xzzz-h=Sp&B)>_6h^>BEp6VPWV=
z1JjtPUZ<xg+4xZOgr7bXJyU!rPWyYB7o%8cm7_U5oIY<45zQ;#!_YkL9){*JfigfD
zm!=HT$w^OZ{ia3mNSBw5rP4d5=}RfYdwbeRM(C}!sGqkJO)fDpv*iY{w;Lv!AyChI
zRWgU0Xu8VE94<Xf_FDwU*GM!Y5S}DG*q(jmL69}OP#i<1Tc3U*@g}&vlRAL3dBwz=
zVqay1OdMk*e$D665g`+^jKps+BaYrc?BzYN1Z4E-7pVNi{6vv8`!&Zz(-{ImI;czu
z89#bF-%^ManYNmAAaMg0mCX7~dXcw;h`0d`g?(h@y1_qDvMpFT&HrI<-1@|gQf<q~
z;@IxQEp$Lf$LRYviS}kX5bmm<$SjB=oSYOoV!|jRaWWCL0)uuZPL8KRtN{W1nxZuF
zz5Zb0RBX4&NqoDTBu*FprwIJ@#6=3%0x}+&OjF%e-^x-R$1_r<GX--rakaD^SUe34
zWzWtW9TUzRJI6LAx2GZu_K-B>yKxlH;hat!2xlk1NcQ9~kR=o-)uIooey`|*8c4}Q
zh0BAh36~4DP4^FN3=@VRT09o2Wg8MJ-v;_`;#Y}P?Sel&N}gg(tja3qkRet%YHB$O
zexB}M$t$2uWfSXEVlm-T$haX!A`hpVPmcn8zI<W>Emw#b(1|JXEsr2ON+mW{IN#9=
z6Px(HTlGk6s^7@j)AIZ*Oo+3XAS@Z!F;UL$O2-8^4lg9q^s+Q*Bi<nbyyeXVeFXaH
zj{~Ga63kGo5f$JBGtwu(PahySzD9!SE*9XY&pZgS1h_bc2yliw;RI}+k^#=<6%$UN
z6iR16Q=&IN;gsfsr%9KDw>fh#CBqisC+USV+)qlSCYb&r;t3zBh!gkd&4iDntXKGc
z=8puKQ_=7JB@(XszQ^=U_$OGG7D|zz5!K@9n_$hYk;^oxmIQB?>QXQAV5_Sr$K1uD
z#?<&Qvf{g{RCJig0-h$i2s5CcguI{>Q|KiPb94+pRjz_)`Eia}SX$O{@FeuhF<%ry
z3h{J%84n>wVX)lEjz<lCLP~hw$}wM3g7Lk(H^<zoxT(RZ6(pSCpCTlsJ~1s`dFNWD
z^(fQpqo~UxAm?5#ZDdn7Tcjt^(;t;br7L$LP*kP}Pyau1Otv(cQ#S|8R_+`#Iml>@
zzy_m7a?B>w$<YxO>y1A)3u0w`j=3dx;#eX~E<;jlrg8+CAKDGiR)3dcMhefa7kn_&
zzr2@HKO!<#NHudS$cH&*kc9qYkrjlHAH5;-%WYQY{Ow0PY}#oaX;wLZTVcq`OQ&;+
zNHnd9!i!=h4NOVQ&yh7pl92xKmvV0S()ZGMH)4pFenW)K?aH|*2=X-GVl^AV(VICJ
zrPH|*GPsVB^R8A-$l$tsS`>q}_JEw%gq2rFeWKv?SzN1Bs8L?GqJX?#kDLi0pWZN3
zWnn@^iWd)gRQLpPQq{J=l`}#6Ce{Vu+=Rx0k}93f4Wd$|Q!Q))iM1e$VJMX<&6Myp
zR=5<@n_V`iyiL;dZC0M;+4JPfvpnqa<yju`=)pA4iYil*!#oS8_4K+d5l(;3f;4&{
z&klB;;HWt>cYze~2l!WTX|{i)ck;p91!*8p2|*Ha7pfqDB;}#pMX2*bcZBP&?#X4!
z()6wgMVxX6&~F?a0beG5kvlL}!Sdcz?jWk!h!5r2mE1wXB4J(k_jC0$T?*giG*~)l
zE2m7&Da1R}$Q>#LN(t#Vkx%Wg$iH4j?$9s+^F6gy?of<f{T|adSC*X*$$h5KI9zG8
zBwHVBeRYqtp2G{_@4_az?d--g!(S<HS5qk0hsbl=HI}WDAzmpp%Rq=f7^6<*cAy4k
z`14)4d2UC^>5q`11C88HmQ2Y`!`#kFujF&68QgIm$*l)>W^~j%>S-PnmlD*Y4KqO@
zSv2rvK>-?S0qCo96T($Vi!@YD%&qLBcD7=wTuZ36D3M(&hyLKZ#Cy51R`yYbF?`Xa
z-l*OguZ5Duq~3HrQ+I?)`=8VsDzpN29-q{^sT?yRyGg?!i#~$fF2rPC(A?;3XC?vm
z?8&}RO%xn?)NGo4DMsLYFK}gRbCP;U=$HL2h1Po5v?lxAN9^x1r?M}zX=Y|);?&xb
z{T_<Bv`^7@E|AQ&nS;iTHL^`rZL8~o<Y2WZUVQ46?Onx4Xy$P!AlFrGY-+%`I?cuy
z_Eh#7^kV#QuwSZ}y|$9Xmz)}Bufv2k;)Sv&v)B8+-Fev?ghN-hYo=!%gm2%GeH2NZ
zdAhb`LCySBGM5_k9+cmMPuWJq%uivo5noP>h?(agvC^LGIZ!xDmP(;;PHW)8FDQKG
zF}|hDmnvt^^HBwl=KCnAQzLtUerqS6{j3iTj}}5<=_63+g)arBMV4{m7s=2dTldUt
zUYs*$9ka3>BFwO}&1_|73uA`)vV}3jeczBU!j~1y7<nHvX4pBIv)+|FFc!ROJI#7m
zi|vtO&pJl_LvCj$k^k&&R{lvX@8#d9xt)IpcXqOi9#&2l!-5oF{uHmVih$a0*~K=D
zyG?derOT?A0ZNM(dv^99$q)-~&8h6-*Ux8LN3YyM((G<Dv!_~T$I;AYzf5-2L|v^<
zbl&Vd0P!`l>lGU<v$s3|v1~?PxERAn&t_LNDaCP=*X&yq5$TuH%&t$ZnvG&x^~}b!
znCxg_UW(XI8`|ZW|B`6Ve){e@HS!z*lrefvA4{Mnw+H%8**wQXl0XcD!=v%2K^#2B
z$a92hY<A%AiNJApnM~B>IlQGbfCO7B<vF4>J3J5H&U47-TfMK!bu!OUMLjH`<vFTZ
zpTp!+)g?@rvw}q3w&yw67$aS@*lGFu^Bf846`uqqmDWJ%XpQw9CqBs%ve(G?etD#<
z6wmDh*G_-EyaKDNb357P3fKp5Ox5KzCC|c;LH#Eg%Q12Fdw_h)08R|oqt}+#q?^|7
zhhvx5QyPIE|9rWuK<h)FEq+}t55Las#&e_-jO8hH`1W>#&C82z-g3V#Z{thavGekD
ziQX%uKZE(@=~&rwdx5fU=<@EqIM0n+{*({>!6nQ4F(jWW=5ct`^5OD5knhM+%ZGEg
zX70cc*-y4y*BwNi^?GvoNO>UIXc!`f;PTP5fiQVCbIg(D&-yZ`5xV>p-*>4F%MVlD
zJg-nCT||71<+Ao*jrHc%)<-^<j}ewH3(2A%Ts~gDQ$mK09b+t?px@F<&uw-uZ5W9y
z0ku&Tco(uHhQPfzWvrt!^)k<`cKTRHPo<Bl?09LcgEay|@a!*#2b)lZu1@Q*4xXUo
zsbxYgRUYdYqngvIOe{a3%t|{}N)#HCv2w*&$10JbyLh%t8td4CrC6pj_kfx60s*<d
z8ta%?8BblkLv5^Mid6^^sD<fc9g8(g2OS6<*{T{TBPw-B3O~ad^E31yt+(5ftogLw
zd1I*VtYYqtiq2YyxXl{_L((#{7BM`UH?efcAg_^?!*F396K){bD7O^04h2i<!&y@V
zi}cl`o#sp1@lqDcR=@?j&Zp~Ro24@_&t})n>My%p<~u_M^_yg54V3D%L^*4q=p()n
z<t(`@SH1`G?Hwou5xI`<kR^(!=W)S2m?&&6!00}EGAl#do*?e4yIQg`s*)xpRBi{z
z%3wmVbcR&yzS3DTaZ+~@-)sEkoBJ5~o^<2B?>qK48@~*r#lkFaqEalXuG`C5jckgA
zda>+$)HiZj-$H#ORW%m%avQM{Kc-gJqfGM7e_2L_>{d1BzoiuvB3;a~-C1&9%>1jN
z<BhDE%*@RjBYG~_ud(T1yOufsH!3--Q)4qovR@{YRf=fLoeGAzpge><TiEy~uxX2F
zKCJOi=+Cf^#9A!*YvXfN(t-{&%=14sJ}1H{_FvN<8lQ`oZxRph-T0!GN+{W0(0H|A
z9Y?^b*Bfu=L>QisaU(|>jd#$B7mO1r*szJHd;e|B-Y;{(^H$Ux4YgwCS}{jsFSc#m
zl+Ed@50jan%q2b>)px)+qj6I)D3_VTLgYP=#w}EDSZP~Yc*de18p~Rh*C;G*Gc?!*
z|B_wZla0GaDKS8vm8#wNY2*HARChOR!T2WN>$zIzjAFLB{p2FdXWPQX4xhUHUX@gB
zYZ*F&W6w3EOS#szYQ#2z^KQQpGI{u{40k(tNic#==-IZxqWj!_)+C6akOBGrb8f$J
zSYK<)<EiI|2i2>dIq`14>C#4J;_1zY-F{x5=fV`X-wdCp<9xT@O!a6_@$O!?-*e){
zUS$Ti+#0XjFN4|G;|Lgg@!;T+E!C`IWFRQdA|`eJ#{E`vEVN~GlyC?^H`aSNW)z@4
zA{atjDlP@ltl8A<*O%71WG8^GxW`-3K-L@P!hVFt*KqqiWhqjl-cy9~Y@39?K!S0E
zmw3zV*Uuu?s5gS}VJM0v%>18NWSGRe?e-gtK3r0tndc>E2%m|YBAz3B6ykQ!a!YfI
zkGbRuz6-dw#dCv+TkzKB=>l`xZ$-u_=tY}VrTi^HxJVvR+D^&qN8!qe2mM%|38VUT
z#DhH5?}6h8fw`SAYyFeBV`LRx73|w3-iNpkP-WYjDRu{8_1#=(YA#}S+1!ql)&B<f
zg?Juk<6&)F%`$w2*RKR<SY<mYQUwBjjdJev5IbpuX2fPQzm>E-9tJ!l;sdf8JV9*c
zsZTv+IGdg>xVFr{toH;v=9>;8_pfCsWxT->mC0o&nL(3`5_e)s+|~DEPO$f~X@Oxb
zmg|$PNRA-AdTWZvgK^@rg=BGjak$xDkW9QE$#ok^hvG<%pon^MlEA`pGRlsSRj??>
zzii2qb!>l5!3*kV|E5y0uiygy^pjI<|NHk4PNM*seiMyUrTgR}Vw{HVU+=}3E-|d@
zB8`)$*!fA}msBfgkn3N~drB~#mY1fS+l9MR4Tt`sE8TuWN-*KS9pPvo?961zWIKIn
zgKXRc@i3BYFbg*W6-}z8@Y)a5`Tr5;28)TaP+QskQZDV@%c`rfW?v($!EJ886j7{8
zex~(HTS=*pXD1#J{gfskzY;7@h`arC(s~(ftIlU`Ked%~14j57w_io=k&2AU{520q
zv;E(9H7)(YF?wyb4W5Ub0%&|I%<c87gxZsq+kj@$-~iB`E!=+5KH&QRFUQDJ=5D|0
zkiyi7(cn7n8Zq4+-F`J}U_*3agMSf>%FHe=e~)YzN(GElVX2&M!`51rA_7!-60n9C
z8UNdRACQG!O?>h`*mn|<cpuvc5$71#yj-pU*OkCndb(zAKV4Xrt$j;l!>kfJ9+eWq
zr!ApwzfxMJV5=-)7TI#`#*o===TzkyNtf*-09Ear3iHvtVidZaGs{Z91|Gk^+|Ess
zRBYI#V}Vei&RH}#%l=z>!qU}F9*33Wl6VpF!^JYhMa4Ro(LJa^|N3A&ZZSPm2wLEB
zJ98vRbT*ame2$@wC`!ra!~dJlcMpO}2h%|~nWVRhn^xp@Zufa4*F8#uB|XkU@rd-C
zpXl~FUzDQDX0&Hdx}7iiJWGPy&V9v%Q_;>>eAEf42i0To()pP$<me2y^O|_1L{h%*
zeI6-Mk(NmK5bt(759_RF0Ojv?+wFW!e2A8wyV%#s?HpWOAwTF0Es%y%2k<^-n%g-f
zkycX!RNY>7JBMl~E<5O!z=T%5lsSDYDR8#Z?uwL@*V&5hO^fUP-t9~a68;Ymn!>NR
zoo#)LIg{>oru$e?|4+BGgU>T+zT4STqv?A7wtjABO=+_`i;UNQXX^g4w7A@BR*Q4i
ztS%M8;DW~6FrlUlJ&2FO#z-<6X5ze0)$Gq4qT3l)9!j9CVGd4gV_|a~Win(nS}Gu}
zj_r{OjZ2p3gb17C=qC-khjqr=J#53nAlg^UvNdMwbth@%XtYyXx}C}V4GSjxL(1Rq
zgP1H9Pu<QGU&^lleu?1a%79h`TrLenz}auG?Lv|9W)j&Mr5CuJ&4B01?!av%Nr0LJ
z*>|_?6F$n}<OaynvIxHIXM9z0Ra6}uO*fodhmmxPzD*XS%iBi>@;EViBKJ7WntRdd
zYxI5vj>Oa8k8biEM;oX8mG(p=6jNmgoO&2bC7Q0-xGNQ~TzEUz3%HpmH*;9c@uMlA
zM}3fi^d!_3BgN@0Crz7TCSZ^xJPuzT>)>{lqcVg)2?KF`0E+yUx7m|`VUgg4<xUpW
z^Cl81EUI9!$VgaB=tAiZWSWzIX5MrEjq)5(3Z!M{?UJjD?n}-#8R><huNWr3mq{Mx
zb}|QG=KW&#L<yTnyaC2KnKst$C}9tf-FA1Ba1hS{?arjCqInlWe+{Fo+gU}rGp(w(
zh;>XnUa6}KI5~Kz)$}o*4^_C=UUoZ6iQr!;GAgy??^)&Tdp$*7OL4cbaMtP@oTaIJ
zX}}4$aMvlYa4g*BSR$-U41{CQVlJ1ZL@9C^;h6V(f}(I0#MpEEOZZ#&hf6RroLJM>
zds@i=r*nMu4Y$+ZCNPRZ8;gX%{2of6KQjwLpiN7lkCsMYOQd=<0XN#4`3>z1<sw<6
z+(>*3*HY>yzqMUGc91GBNcY8zrTukuJ9OiVbUgW=g~1y~5gqN4EtcML{N&SHG+W0{
z(u<3B46f;R`~uGyWZ;8Yj=z;a$@H+5spB7u9s!+y5M<Ngiy@cw%<k)UoDlP&z?bof
zDoD>qVX9snTm^0Ei@BrSj*}QM=U0;ZxtU&kn3n}x@a03rNo>#aD}zT5Mj<#(OQjZC
zy@um$@hl4-Jz^jsXEX$&P(O~d;*mm03C|*$>KQWK?RZB*uILT@ZpUZp5z@QCvo9VZ
zs8IBOtw6Wqy83jWZ(N}?kFRq(UewZDCjRE}ZilsQhNs=9Zil|-J3!k0fPrpDAKJ>g
zeYA)uss7a^E#>Rz^N3JFRd&a9L{&&idCXIGE+*256DP;?_liB`cuGqEWH)Xj;*c@G
z?O=<g40(;j_iz_$S&Gy;4wu#dVJ*dM{>q<;{Q4fZV}f=NMEn+a17N-RZpSnmn8e>C
zKY+uoyB*K^fT;7N8lp!^F){UAXM#7Zayyoa5@TvE3Bx_9ZHz2JxgE=V!07;|N)qOR
zpNo<t1#U;4be@HTEWpjfwA$#0O`BqvEA&Y0MAf+)lPtUUIXMZ%X<kvP8#9b*)}@V(
zX`mqG1{&yUW(%HcAhAd(vE&TZpn_3r{N3ISD5$2C`?+COxC|X0)r1Xf^ou26vgl~t
zNdX?8<-H-*a5njdv|H|_ZLMj0*lCXuAgLmMmsArhiZaP@5UIW(u)uGvf4FzUcpH&j
zJRzcuB3f-BT3Oac<W3^Ywm~<rD^C!uOuUzU8_8#!mCs=d>B>&`g4-(63wFBEc7T`c
z+DgJ8eZ6$URhrLb@Gg*W?VTKxESAr(Q*OHtr6e401=FR;NMz+(1WU2q<%C+R*c%Vn
zfmeNj8Cn;NUc7qErtgw^>WyxBd*>o!JsZA|3|bSL%*|C*q{A46MC9S5kKW#&k99b?
zuisc7XJxReot;7Cg$L@)B4a&Lld=MAa<ivB3iR4R)S3p_mqOp8eZ33~5t3E<6u?B0
z1$*$d{yU2!t6mj!R>%|`oIR~9S%23K|L|V;bPY#7MLdJZ(<}7+ED<bbLyk!4A|Xeh
z*_t6x=_1yhbvqi;{TKs^jOV`N?^V$e&&Sr(fs~_R13<L#B>K=KEM}uygxis5H~ge?
z0N=KoV5=ZO0^lYpE9p}@kd@T35l4I3N4iIo-H!Hl%Wgz9G!k-GA^;-stJeu#I?#5E
za|pF7*9rH5O1(!UY2j8<IAPmrIqhi0=JIwy6J-b7)Bw6HB(R?BR9=}XXe3I8;cUEt
zf@$lva+;uwL%JnD+He!ulIo+vDI^-6l5R(=%s4%tZH;ss)yhhzL6`G<S9C4iw1ZiH
zdvNpo5Czd?*f3u=WP1hU#h=`cK*hK&OwC<KAS|XEk{MBgnxcL)q+%LWbvsO97{_*_
zcJVhvDrQYsPaSbN%!ZPRmz8A-otRA|ra2T&Kj!W3b~tT?-H3jpQ80zyeooQAc?UYy
z87A`n8DSV<jVf^qF4HT++~#M@b*_0lOzYnKjIKb3K@6|W&jq23y+SIv%`e0w{qp9a
zZu3j^NZI_4yUoIf<QVswjTXDimwfD6Gs9w+xsSGIjRn`_J_xe8<%=O0SB`a?+k{(d
z?$`*K>a_U3=AxZ?=RJhi736iBHq3|7_6o+Bfo_vq=?!yj4a@#8bCi=-CvL8g!Ym6`
zrkc-*C&gH9vL_?eZB7ZGKka|bZC1sW-gp_yb(>Vl`dn+=VpgTYl5vZ-5$1EFT6+5s
zk(0GXjYdNN%E6+ZP*5EMBYrpl+DfA-024CZrf;U85eli%Qma2TivH4WB1HPM9)`J2
zUip;z;6%9yiqU8<ZcjPJ3z0>}rc^PDz5Jn`h!aTyIVQ8UK9Nq~n5uJbWt`FIBJMnL
zzlo*V!B2?XPm)&SEGZqSoMzjUN7{sY4MLPgrmeR((Kui1N;q2HetE~(B)tdIO6&SC
zWEtuRx+|?})7kr4aaksBM9<0YXd0cEv!&4P>pR*dYUV4^=g{^ECED^P7?|78UoB8G
zDbUxHVT*|F&p6o~yyaa!{=waqEblNcZk0&Yr!zGbF&;s{R2e>@3O2_Wj>rBa;{biE
zs-9?&&2d(V)_bfHD=ft-(RwEED3Eq9o3$q<j#Nt7NfLlAk+-X)L?GID*Kl}SZ3Md<
z4b)gJfp6kz)0Xu5>!lM^aJAy++v@Oq8@g&!HN#o>n%Ha?ySU9LWURR~|CS<S8_L@<
z7FE}hu}4EusU>X@i^7N#S$MPxu2Kf=Z$o)fSCeVqfG6E%jDG__H6f$9OdnE98v>-9
zR`(DTWw3v}d22#YxRf-NnwBF_(p-ASxXo~BE*l!zT7jF9q(^|v6IpJv1Y|aw0Ypa@
z8|&3XJGC=_V<t;mcofneO|t1M@u*E_SAd-|40B^WQqx;OW@on<B4loC3YpDT3o&H+
zV!?&2{t+dc(b+56?-u;Hm08ecr|j_C1*Nx1_*E->V?;rnZT=4jG%J$uf{xp;$liX!
z{g)JAV|u8nwZ9O5=*Q+AeZ&0=<Uh6#K$LKK<IQ+guD{f?i5~%-nenQbe`##fJ{y>~
zjWrWaQWR=nTleq&y_iz2W?Uen?d4{`5b=kxPui<<-jA*%Rel+X%Ny`adWXUC$1Vuf
zu-p`m`v?6iuh8)i(kc&f{|ISM{O%)d(@cxB9~q`7Uta{_Zi}=ZsVA-eQ^1bfq>&WT
zg3M(1?FVU_bB{KQG@8<*A46J@d5^U2E+MU8?RI~6@BZ#vkkXlS*1Z9nLo8Bm`KtI=
z5Zty%xn<XIGTlyciD+f`H>y(cN8_TX-Of_G%$#q!Q&4(`zsn0NI`0#ZN+0hI@vOoO
zDL4yVRooxZC2g*&J5t>rX~pQ`q5C7NLSWXT&$#zP;o3|za=O&x)?xA5weDBVPOd2C
z<T#7M{iznSl3N37DT~p(S@Wur7|mqlK}+Qu$pZ<_h?X^J_d)H2@xy}TS#gGuHOGAj
zYbW$;ZqwmdfZRf}_AUe)q2IaB(i?659IYf5&?GO#EfEisR>=ZbEr%Q=!8$=|g#PY+
z$4iPW-0)?k*g>Riz7z-Wyux_}N<@lsyg(+uhLP&NB#9Q%Ubbe#(RB@^x`2qSYFjWt
zsiJq*wIpN^A;a$rX(_5aXohI$bv=wxDul(!ibNJm&zy0R!A%v>ltkhPgZ{Eb*L|6s
zw=z%SNJChYkK^P>l)UMad;xb(Jb`>%$9+)=Pfhtrl)UB`&a=}B!gl%B@~#yV7`nFB
zD5lFf?wa;76I79|y=6zZS0v}SuY3^%TiN^imdHY6CotQ{GlTy1V9Icl=qR_IGuS%E
zYW7^yZ8!VG_U;@n^0M+fAi4msZM`MYISE$dN~SWCf3@T#r<o-$DUV<ZxJ5+d5W5VU
zQxb8tEfL9++7wtXT-P`ik>x%SsbbamDnzQtkY6s4aIb@n+v4n6PkEMP^X*z^%wVMC
z`{ZQRc;XwO%iZfG!IQoOn~1Z)m*6El@)m%xs;kAcZL?s9bKGabu8;}FI3v1lIg2&4
zwN*4fYltNMP{di`+>#)hbM2WeJnkN*#eqeq$U8=w)VAL&77epmlya0j567#WG^{DH
zr0^f$-lb|*Ub(MoB35gCEa>a*A@dd6n@Llq&Go1$%nsGrmmUoS1N_3Wk&C*FD||C^
z(j!gaAv)R8OW3X_J$iR$aZfygF5A)!>1cJ`roa58JDujP>*q|jbO8~g5_+@MlVf&B
zg7*>5Gu{tl3K{fpgdlTlmJwrZRF>81av?EC&|SJQAT2EBh@dLO+TSkEf<3k)=18RO
z5h#r>sUYSEBEXIT-H@<hTnq|~m_?uo>lCBg>vTgv#$PcLC1gh(-TM;5)(bBLN_LBS
z#pnvNY@*ouOUx|EdYP4eR#o}d-k?#Rm=0J$EhML-QX4ym#fxf<kt<0gPRecEHzaWO
zjp-oi3L%Mh(Ab42cC8iDHNr5S{|Rjsoz-uo5z}36GmuH1#@MEf>+hrgowp>$W-wNm
z0Cqka(<??X0YL0y*h@nu#Gi!AVtCiGHGm+B^;NXoffJ)^&yw*fT!C?sYu)x-RI2HS
z0=C6;_D4cnqQsba@tQk%;LfLWtQOx=lvoOdySHE}6D*Qw5+SI!ovj~8gDkhi5|u7=
zSRhBFO8)yV=G)Mqjpoo2cRiK;ly^1aV^V~7b0Bu0@NQR4c;|(A%;qce8ovs?doAXr
z_?Y(zu#3@h1}Mhbn#b;Z1g42h(gp~Yz-U<$h-s<9LI4SyR6!$l^NVQ}MmU^mf^;-#
z$fNg0i{t*zAB&pn=v$30FG^aA)`G?;Bj8<?q-d5wMiBOD3$)>`W`?{^x+{fxGM33t
zA)6s6LU(Soh>NKR>gH$*<2xzX?tPX}RE(0yWZe$e$|_gI>YCPQG6LA(IOGqhh<;Ei
zWb>Q|MU#cNXQaxlJ#_e9$L*(ZBe%wIP+Y~J$u(dy%5Pzd!LBc?7LpLAZGtwl>y8CU
z3|EN6?L=aF>D}H?f@nZVRuGYQvz<^HSW-=mOszK;BM*7l%@r*VN=oyqc1YKIA(o`H
zz>XJ5**)rBh$USuQtS}Dg4T=Gj;Tu`)K!athQ8*ccKK3Koz&KC`N#}LLoS^uKeNi|
z(p%W%NQZR^67}*z8OEFLuOb~Q%hSGOwe3`Pq+?ZCJUj3-|2)#MTG%Bc<pnN0FkC%j
zy_F&z&ub7XpnSqJs+NVYF(V=!qu3YpLO0TkCDzkctU<(@=EF61W29rTc$UckSMu~i
z-Se0Zv4%18=SW8v)!#zlqTq2uhmf`x!pn9_$*eaixy(424cn0pYoC```a4ENI(RF_
zF!z)I>GqP5j`EU)4EghGM>;BsN9M3Sdm<fGOEP@T3LY?OJSbF3mMLl-!AhirevytE
zN(5E=!XJ9gY@{Pj)tf9-c#SicA{{dM@caNBXViH)()?C)PbYcw?MU-i$xt{~D=^X#
zsH8|~Yc`EE|EMJFmK@DL1to2b|4CCK%`<2&WZ&E)@0n!Zh%|MAUD=e|A=12L5n!B_
zo3@6mj5Oa;8cAHXO{6JTvuNTA<aVhtfNC;I2NNtd50?Y&1#Dh9T=pa*t+H*b>4ix1
z4NTNM?=hUEm-&gG_`&eAExB)(5*0QA9b7ZnvPB)>Y5utvALg(wP#!sr2eV#1h>5v9
zNj!G8J2l%{R<_o=jcGlkxMzn(nyZ9jnn>ZIuyGuC1n`J-YBu?+Xi@RQ@lYG)Dv}t5
zdP@o(uQb|1yD*d63pT-dWMrhd*oX6yjdL-Cmd*$pK4>x-ET(xH%WRyBrQ7!5T%0N%
zBb`l%T_a8F)LW6!<uFj4QhZS^@DZ*~jz#`t@}tSGxF2c`9ETK{lOa|HF-@azWr;fK
z6@Zq?Gn&#enl^vfs)%XI5dbvorlXOs7p<ANNRvHXQV-~ku1SDD!ev@|K*rZyyYiNH
zsC>+z|9PoAEx+p`!+gnB6*>pjCMc4zucT#Zd@y8Xmea!1{LD=1oGZ}fC>owP@z@L$
zX=oE9(h%Dq(i|>AR;W@PaVgU5Ei&*@c^hAEt)MWkAnYyus}G@`MbJ$Uf}_tp97m2u
z%H|_*G_vzf$JVB{=)Q(uj5N7Bh%(sdn?x$P7L85N*u^xqA0*MGp=Z1qX?p7@#sWK8
z9TXE)Y$dA$V$k<y>)OspvjTlgR3;Fehu9lR%`~D3RzTxvb<xo@m71*)3a|$8D0x$H
zI{->iDkRvzcL1KQ&Q9Yi<R1m0byM^PS|e&H2J%IzC^>;kz8lI$nq|qii^!Lb%=dPv
zDYa2n6sY8Q9e0iXze^{Q?j)(N|EJq#1;1M)HUYIgu8-AjCbt)3tHX%MuZ6D&x|b0`
zwQiAb$i%?j#e|ig7I_S*W`%Y%+kQqxt|9x)Q!%KnN!Pl6cPm==MZ?_JB6{pNBk~2i
zdYb+X9ls%OThk3EqXu*DyHboGd8gLc+QWu@rLcwy%aak6VS4ukngK&|{f6cGIhp>^
ziKx07!Cv#&SAj%gpFb>bxf|?~Q~dkb8U}uO1l`!4Dlqle<xxbTQfbE8DwQ26<BOM@
zBPpf0dfD86kU=8`VQQp!j-!85Nt0SjeCPN`!?h64hkQ^2sXZB{Hfo^w{K4_RNExN>
zdx_Swc^@o)Oo-Yn9W?Xt``(5Kc@~bRCl&sk4gaDTZcVb?wc&rst_Dru2#S*wTSBam
zm&GWw1L8G+0gWsHy)0`|I&F4c`fcgGbzKU2wq$7(86_V`Q?DS*qAoi{%9!aDIcR*K
zD`gst8_2<>15Y`<%dQ#nHNv^<s(wszE<2?kb97A4MTLq4c`UU)vI!VozRnK90dIg}
zEG|ntFY_@2H(fm2`B;MEMgVib^L(tvtrQRPnz{?OLkonxmv<fg^NLA{$V;sasyR%m
zbjz;$22S$EEVa6OA8@^2y;Nc3u9DP|-2Yvw5b}4Qu&LhtE|?&4_cii4jyoWp?>XO&
zPD=epqW&o*1u7?^o7L&UPmD0{l$aXAu&b8Bd_|U|B&yVA8@Oy*vW(I8(kPgzThWwi
z+cw`CW;mBuA>`eHpAK<SZYvWGQe&^kVu}iIk|^J;`$YMY(Y6!|)BaG+i2SL~$#jqe
zWl7_Z4{aszV_Tuk56O&KF$o9P9hnH>m*HCEVK&t0R75k{db5mL_FP1>VfZrbxrk=d
z@l1VSE@B%J$6UnT!~V6sry?hd9@u{)WTFu{OSHhU5NxAotgmqZ=h~k@=jo@+FbsP&
zvZtgwkfvMaBYTIFGG0%AMD})D-)uUM>`grioGtT_y({VWNUk%E>`gx?=d!%)^Nt)Q
zXbVHyw|?4)%+hbkBItHxBh6x&l|}M>StQ?=1&^^6XOXN~P|IU#MW!Gw>}!`RDbH_>
ztgXh5a~{6`^@_C2v*zf+<O2~7=3vbOC&Oe+gNJNT?s79=pUx5H?-2TI;|OyXe!Us+
zRaxh1Tohrx01av&b-WN^?om(hAg*CG3+Z~w{2e^L4&N^##F}E#)ClXS?=q6tUy6rL
zz*r_be4g1FVeX~arH|n0cnJ?OUiwHapS2}|l&04X9{4H3JTK~(W>tz2$@y4p!i)$s
ziqK3c4n72)IyHb`FQ_fF^_K{91_Zf2CFI;fJg`<hZxU^W*g~#PIdl|Z&QenK&{2dr
zTTN!@Y4vi1b-h;##>T7P09oLQFrP&)(v=A4ZyU7G0*N6YF&0~1G3%e|6Ja(L;yMS9
z?>~Z^nvv7N&O~gP>Wwg)OO0#i|J2|J-siL#;l3PUJ{E=rJVwhpYBHEM^o=k%S#KU(
zVO5q{RXCNQ<kyU%B=3Oq%!&xJfqHBX`s*|GX>;Tb-_V2z6QLqekx}t5^>6HwKJHMx
zOt23mteRjkhw*1>E4&F3l?IfLlWTT_x255Y0$&06f?lT-VTRkl<`@voJu#UB$5PM=
z=!Qe}0Em;fD;!ts5O;Me#LLoy(HudP>RefuCFx}mKT^U&6A<D$ts{Ptb2wG`UNbS`
zjxb6lOH#%~$nGGcu(Xz3(h5D4IqZ{&>ymn<-h2^py@7sD><8Dcr;uWR#2x%S`An!5
z!Fa*+9y$QR5^qK5S&J(M)_1j}h?hm!3rQ>2l0e0N9~HIxL>%P%(C30@_>?Y&5okm_
zYxDCtYD;d3Gt9%;<E05kJS!#(w}FBs-;DU5V2R_~e`3T^z7J<d&=(x;6e8!`A_}w=
zoS}(WArhWaIz)wKbyX5UFGqePmON?2#1*l&+#icYB#k9x4ZBJqwv39HN=~rFJbY0K
zQ6pkH{I5I7LgTf*vqpPq@-x(fEFXT8dMYs`VmfvzGg~sxRpDfWG$onmf{0ViBZkAY
z*Gh$|t{6@e){6{kghq^3DJCbxZ>-8fZL9r??Rl!XoGCoCCZek_mQLWc4i<*4GIlMU
z5u|OVYYhV9LuiXnU|^QH1~L%$)S8yuOXWt?mfSa3X=}Tb6-2KaB~UARl21s|T1Cj%
zOJ~%eskz3nXyTeL!6wY|=JB+aQ8^>xVZr>Rg_-_Xm<;Ae7`2s2w@MXrf2m5(l~iI)
zj(9o69AY(g^N2^6l>t<j8Oh-O{l}p*hchK~#KVX+#k5Y=NyfQBq}_DQn_~9yB_3oa
z?xTsJy@5ZWRD!jY^&}pV;S?Kfe?==Kq|Hh(yZEpiu(5Ep10)ERC_;T$&XZx6Cik#-
z&F+c@z0v*s6tf5M!I;f8?@1|UbL}%A{sX|ndIOL_%Q9muaxo;ujOE<R0RVm?@<;y8
zl;b`?u{3X0GKYyR{&Lvxrece~o)I##|4?IivvT^)S9{GUWTbcWKx23Z>gM%+)QOC)
z+Yb_ReN^G_P68dw_r8$foe}oZNA!^J@W(~)3sqw*_y<ld94$XDBJ^cr^XJ)Yc;lPU
ziQ|m?d6f<Gjc*XITGR9Ad-(p5@0gDH^AWW<kMQk1^k)A2Mg-i!6B3ZW0HNytHs8q~
z<}avXM;hzQU(itEAQJftO9GWqL+rNvMU-OBBYdxL<S!=C?Av^AG4q#5oKk$RAC><<
z{SFy1z{p<;MCYc*G|A7kzP%VfEAsPv@jcJy=Rtv{^=!z`Lz(J#;)49VYVxft$Y1V5
z<oqN*pE<!d<lKAb7xU2})_1eW{1rZ&OUmUxr{A>jqgzoO`D>|HMeBUTn=flob4p29
zc67f4ZgvWhdR}XH-~1gCDy8kUGk>QqS@W#?U0Q!6dZQYF&mYU*N9)ji|8G1D;FZ_%
zbtWAx)_>@DBmYgRNK1a<nfznnf|B|q-UaV7`R~x$#4pH_k;4=9@=Y;9ETZ0+4akoR
z6QiWst?4B^Xdd;v{#CwNNfDH$f#sXB=^Nb{Jb;=~k6_l_vGdO7o6Qx6WK^iJ)2v(n
z?dWpxkrGCZd-vv>ZIx&X@1W9xH+ZDvF<8H)qmO<j-;{e7G!r6_U`~#tdRo37;b-<g
z$YtxM`J-&Pd{eF(f9F{KD6ZO(Z?AD`Wd5kOP$ZJMz?DC$r(^_c`X}UPNk)3IZ%)s)
zhH-jGlA{K#>CFPNzEnBM!99*xU_!{K!X!t1+KUD?otWfkNYA5Jv5e`P<Vb*!=s6NE
zzDAOxL1}BkN~oRZ&8P2Oy(r224aCRf49i2;@zBoaeH)zBZ!)XCcWF1rst1o_Iy}kz
zn||vUmlMm3Q<C{FC`xC5H}l&hhu~9Be5oXPiibu@$oj{U<lQAau>DwBAl>|u93@?N
z8XBd8y>eYdl7lx_C^Zfo^ScOPMc<kT^RfDrds^2dbElB^La>+>NhX&7;sMq64oT+g
zQsa`->lKpBqdt$6;tkj{?^|1~96KiEPwSXuz9rPi%qC1Ty{D9GI!Bpg>Rl+10ReOO
zB7$;h(y?Ro0Y-edoU%W*7o|3+cGBOX)Q$%Q59mJ<a-8HdcMSrSWR{ftMM%e1!b5uz
zS(G>-l_Y^BA%B7-@ZY45<p8Vqc;iqw`LS52x2W~kvXeg7GVuNM?@88PD^aCe)+BA!
zTnj@6^v`5V9+`gj!gQL;(WJ|O$Yih5K^somuVjD%GWIJKd{6A3beyU@E*4Zyxui4l
z{j#@o2q|Ax>VUtyH|d~Omc@er81c+FB-s5~QZ8NhXB;;?{-s60awUN8!A8=GvH};j
zJ#&%R^;aaZa)B@y=3C7|mE1W<)Mqb*j_-#M&8Y=n3e~(x^3FwXM9;BE)!|j46kxs<
zptM!4XE8_Su3>=y<Kpe4Q8pcKErp#W-bxyUzT`ClnZ8OIg}!9TP%Nb+Yak^k;MQ1B
zd?uscif5AalsWwe=uW-hO=8#}eVX~!`B0hlOUgi-=sOMYE(6mCd|5_PA2jGEB|_wQ
zRFW=+>4g1`tp$$C)Z&RoRQaXK1&(SkMaM*YCKNdIL|BS$T`O?NjV8Kv@B}vo(GCk7
z2@J?(hd=G&MD_b&9Cv@vYYat)Y$-6y3aPS_{zMPCogD!7^^hS(C&TEyCvJ^~68Jid
znfOrL8Yb8qoTSWS`o?WSzLJmWZ4+^umA4_86Z#u*+c~Ij)cci44gBMdSWL$=ZA;vX
zA_k&|(u(3<DleGyPRZUU<D>#)Ut$4*W$MnK(~aU4+o&k&eiHUP)B4Sytg1*yRVOyw
zZaxP$okg|rMx0g+&8+!m^k4I}tjroGj>au5GFG^T%f+{G6Uvc8MK*RvOI5~A6v70@
z5!^wfN9+Xa>G!myjHS+ORBD*}3hf$~h1xjz26HcG&Jz0)?t*xZ@^KA!RXqFnxPvR=
z)hp64MRMkfHHmWOx?!|{`z`#Kfhd?^IBTwxISQHd$Yg>`CuPn!2-?Q>9wT%_+$6F)
zB}ZM))Fc($Y*9))@Vf=kTf~b8`?^Ikt`Q%hBP6#(JGU0(-{LV`OUdr+1KC;o+8^P@
z*$?3ibJ`@H6gRsB`3+=lW5Z37b;flh>53G?u`!lO71^XGJ2&Kp*Mi?=`y9WFO5(gs
zkDq#vberBH&6Bv^mGO8lX$+=RG~Rs|yyEvSh8AD&YQ^F-j9g&bA`>?l^fI|sz>AR^
z`jN>k$S_+9Xa&<l1*~tL_7}>6X4unRX-hRQN~U+%u68<_P#NR9bW7a?U6xzH*5T8Q
zB$-V2>Ar8-8r7s%+#t%7zk)pVa@e<Bmgo(_R;y3hTadiP7)$g9$+`++(;^Qy-x9qc
zD&zRy#2w*MOY;m%li(BKX%#nw+C2S-)ESz!#bw+j@r>o;3*4vTp};ME#od<LY$dh%
ztX-S=jObBkElW@%`7BfZS9X}Q&Sx65cx$t(t8AC#?c$-@wl01U*TZgGE2Myv#I8Q$
zunO3tl2t&?;TtVoRsnm|unO3+KCqfr)9F>*bhN3-dY^QxExY16Nn1g9S8M`kgwBoY
zT?+oceM?G(^tK$18zdg2x8;1?9Pv=hmaB0qq`Y$<kY1oK;`WFHvhCx(@>;?x@S^(*
z%p$m+IFJh>hVG4P&30kPL!+JP6sjtub+i-{1%^Iq4NdX}ifDw^Z6po~Y>W}tmLfLz
z%Ti{Yt@6?Hq*MfF#Sw9hl!bzFqlL1OiiU-<kuJ6IeYkmCqa@IKQ!Jb^iRQy;O~?p|
zZC{JS;Ib+F$wF5Hwd$j=Mw|p%=xQX$w}mb4K5X?Z3c)sQO<V;ZwzE}971SlpTOkSs
zJOvLrSC>$hStz0ug%B~Odt9mFWKUYjN>M|a3f*2R)iBRKX{9P{rGg#F$Efk(amjwq
zX<CMAv;=E*;+=q=Sq1@>z~P;uPmI;tf}gyYL>n3)geCiNqm=nh0G_vVlKs}w!M_u%
zR#LLxdMObHpXe=<{Wc)*YW3y!P4?TUW$oh|$kDT%{S^%HH9*G!u6M%ho(+T*@uDrg
zkUHx$KQmA8Q*WaDU`ZK!HzoT`^_BXa3t>VqleiQ*Y~awb;;AO=4wHK-@HApWrwmD?
z9MfnaI`voYFdpPd4KgXx%=?QpON-<U9xwm||NB3Lzq8vWe5OS>WqIeQ4Vq;UUz!_V
z#*la}lP~XFP>ZQyTcO>N?3W1b^`;<&=W;+j@gX46W>4bjRuNCC1mp2mKa%X1glTy$
z2jp`{CHpm}hV=+;NLaF83!)U0-|BwyA5-#eRwqph<{&__UnhX)3S<^G*{`d7ueOT!
zgrcDc=VsYOXk}N2b;2^)?=ibR;D16Hejfc(erC6&nYxo5D=5hMV6|kD9nZsGJ?_1y
zezHSrd|9yUxJ-6z!Y)5wTC%xiW#d>QHjbJ&=l*&SWSKb0USAYLmb0VZMgN`;CfBX5
zOr0}5KUj0kG}AO!aG#g%IlczZ0T<&w|HOkJ3wLo0v5%ff&3sqI2?T%2-Itl~s_o<r
z1X=IQ4={$$|H*fH?aU8pTQgdQj59Jnsz3$Z1ezO@DVL7u<m35=GOy~lwz$ksi0%D0
zWb8Olek$KZ#81r6{H!>B`OI6&ei`pVHwTt|7qfbQSs*jlGv9JK_9DU6s=FMU#m2wT
zQw!~KZ1WZRLV}ji<=9?a=nHKg1X+bHjv<S^ldroRy7}zFC7l$)M;q6=YvAv4OrpBH
z7cS`mM3-Z-5*(a0YOH}LR7x8>vfm`}p!ju!wdhxuW4f=syAQY=v#^h=zG53>_Tj(l
zG7P9%pX?V?E?oe>7y|BETkvVqgl@_HNwUA-y(BMaxEy_yMY4fl!Frb|vkqcn9zehP
zmz8et$}!qIsn|zHDy+JeZ{iM;KXjP8lnwy5aMxqB4CZbv5P*Mie^uvam&<&CMUgAV
zoXh6>0a%ji^0MQF+0pLw;4z|0TxOx2tQ8ry3J3A&4*@Da04&5Uh>@%M0J?-P0O<Ey
z<;9F2EHDx%Y$LF>@~(x;x(PJ}{}J3PI$!QG4_VoGjaJay>Qjq{z5^foT2W1R4D)hb
zc(rx0%l!Qz(8`!Ky_C!RM?GZJyRXaq7s_OI_OVD@D8S*_E=P#)&kF!Nk0k<Z_oK`4
zkPnC@{@7H2dGOt<&uU84S8&{IW!}s!G7e?PZdLg;%DQBakNIv{Oo5xbi1Qnc<JRWQ
zb&L)CaE{7`GnboTh>mTciNahJSebL?(Ho^94QUUFCyWmPc*G+KVQ`u^78~JECXni4
z>`fM9Pu76hqMS<Z7?B5;8qO2UQNEi@1gSVNA(jO-GaSbUIIP3*2Up`+K$MPxgM9>&
z?6~1v83;@d;01LemDrf&^~j~zltbh+ilboH75@^C9^^^pM5OFpwNEAxD#kuo4y_RD
zg@8R^*7Q#)_<&)4`V-(Q1wUT#ujoB=!H)K$9j$`WaF}vQ>dSKALQ~I|)Q1<@;PO83
zK3V@gtgHV%&{d=YA^4osa<xY82$!RcRNw$X@STldpcUKes1PSy8LtdGjEN}>yF8E3
z|9^(P7GyE(@+zX(40|ocV%X*50xM=%1-9AAnM{jeLg#Bq!Ys+p$0&K_TSpuRF3X66
zUrY#>`6fe-_tNblM;bP2T3s1(FybHvBMyys--rW0BMuu{JmRqY_J|{`30P7<@5}Mk
zOmg%Z{Lw7mlPr(;vTQ@-G&@UTkfp}EFH8JnX+w*%wEVs-v7^!+x3bj6@#;}&9P&%!
zFh55jd*7Gqjihy?^~JGI+iD#n@NhkmwhrRcIyBY&ts|Q#6MU_MhM6|Y*E*KsSpvIX
zWmIT24&+rUxwv&;4@hf0>}4R1%>&E7qN%?BzK@typm@$l%r-n*+E~OKW^W4#T=zB8
zy#&ALBZRW0!9}Z?p2hRF$fuu=kV*d`q^n5v0c7O*T(bW~<VTNR*#e@+m=7#9%rB{|
z56b<YU_{F=+eodXShm^GS`=hl<x1rPh1RI@gC|I49i~3Oos^Q%2Xt)@WMm&udzBeI
z4bT5?I9q2R^wkD-7O(t2!c*-m&zPo{yEdk2t9rGEjg^gx59T_>3ZG+*b+42loTp9R
zw-rv>KF!nCBUfaMu9_f3YH^a46m24`xzheWsLj@q7H?W8Z(c33w9b{M3l@LkU%AR5
zuCL%&ig|@?8^w)L+NpJl_Hc|um#9(J+s?O$YTaU5P4<ee|H5U;cDf6fFgZRfM}H*!
z>$K(q2_8Q(i=K<6Kk--{;|%Sw?9M)Env^1KotPO@=^)3;PK6I;@lBA7uP^B&lR@KL
zCZk+mE;ETB9d=j7WK~}YvpVXowP{K+gDXj&&@3P94lm2Tuq8jZ%p|mncLlNa9+Quo
z(Sr!KB*li9evW&uriC?0u`+QMMOWC`6(ftlF7r_>1W+)x?qc`FjeWqG0H-~e_i<ab
z&F~Y|#;@v`R_K|JcqvI|Q(Dvix6ZTm3^C9%8t<N-@d^M%V}pwojper$P3!fbSqpv<
z81Ds32tQ&YvWU|i1zxvg@S_bPD8BvTB1ota63svlh#-CR=6}L#`yw#sA`>znU9+%0
z<`dH6n%r&}ANv(VUGw);nbFe9?&He;L1(7Qp}`q6_%AaH`d;In@S*IjNOqakQA0YM
z|M)4Hpaf9akO|6k!`StS3ki|nB4avcfH_snx=(7?)OZF%DB>Bd_u*vo;QHf4ml{Ob
zxBd?IsbxzAN>u@CL!34-)meJ_%4G(5F@B;?(UD=*lwRi2hk#@^dmD+F1hCq-E}ffK
zU>ATL>0E>N80J+mVb`{DnPFm7EhA(YZZMYy<W8j)Ttag2As5FWi8~yXm`g3SnTmT_
zJj?lb7Wrx;=XY`!;1ftL+LDEBYhU^Os!*yyx=?R5DY|gg+D(NXkUMdvALef-f+B6A
zJU*f}fRzEc5;gsr61CA~RurP-RiIi*6})Tnl8-9P<2FZqM7@LO4EavSaQN+?f(l=7
zG;;do2Wb$}sMYCNHkfItVTecZ>{P4gT;JREH}w|v*Ffs;(-rYjf8W=(s;|wbWb}Sj
zt-kLQK?wib2FY!Y6{(m$E?zJby<oJB$2x6WBU%-UT#;GzEr3)>+h+0-k?Xcy%WZoB
z>LFFt52{XsB)*30N0F&91RyVMd6%6|ux*hsjf1_!8|k`3hy3YO>GehtJ{?C*nKuXU
zQGi<}p6*mxTY9{<d&)=}QrvtR-`lto;+f0G*SN3j{u1P<r{FwSHJ@4A_71_f;jfsr
z?chsWsRuD@+firjf?@WwYQFwb?4<jxS`~q5RO6>Vi40iXsr2qT#oURd_$=4^{>nb0
z$Y|B|el6iJBS^UTv2szb=yZLe%maSDyh|63n1O<UH<W?ah|~44&9L^Y^c`|#h6rM8
zrl%XQA7`J`;+L6J8Hh8<M5?D9OZe>~fzb*xAGNBnT|Hp&z-r%hWv1CM+I73uz!*pk
zl^V{#KM?m6bw24~*HD?jD~wjnD_hZJVAO6t?&*rue`3CCxIL_DcL<=IF=^s($)*yG
zcplh0fwBzd8cvWQ+sdkZ?ix<7qT$~X{-xylrfayZ>(d#WrUwF}R~SJXs>+}yJsuZl
ziP}hkA!_N5l7D(q^x&c64A&?D9tB9L(p!*t;uo&b0)wb@N|fF!T6+|GXP%Kh1lS;5
znQ+asOO-wj*qB;6@nt_Emv+~lhMx+Diwu6H=UBnw>$UTA9Q|Y-KHZC(X7?P#py$wd
z_j(TT({b3);*P`e+Z{(b8YmsMY9H|QDNAWz`=OxJO-@HYebwp{K7YveV?;<F@p-+1
z_BZ_>8T9mBtE2fmO}hlk*qKt+Lh|{L|7d>OVwqCMMzZ+-hKm6*K}nf{DiK5M^W6m<
zZuv)+oWh2C&H5eN4`Vd^d-Z##U<Zl+i--P;)iT_soVC(_QS1K^PEj?z+EMp+kLXvr
z$0>HvzZm)7^cvpIn2K6c1dIP0At*`^Of6@J@BTmGx|4Zol1<=y1>b+=U!nX|EXj?r
z6iAmubc^@Yu2z&ULkp_i@-J6%>Oc$Sm+=-)rcSlPI~0t9XWpp?q{o)*g2c*GqwB7o
zZ9AUs>c>IWitDKNa?*;g&mFIV-;b7B^%Zkiyh?4q6vAnD=_tSS+;}<E=IVz=RoDx_
zcJ$Tu*{_4ZxT>D5YA?73*mCVnKS`7~(6KibYhLJB$3fQs`o-&v<INj82>uNxGXgi(
zu#7>3_6AzjaS%m%gZ9_HG>&%J2JbP&0dB`>*FaRMdE*v=skkQMxynaJTpRIR;KNJ%
zXixjlr*|vTzmc9_!_U=86!ET=1S9<IrT3LbciYqZw$CL3sxI?2S8q{ug`|<jq0>k=
z7F#^(UFUut^%flmkJ@j*t+9C2yM@3c|5tG@TWopKW=ngNWc#x|uFRL7$FbV=BwV>6
z3x2|t4!}Av%2KY#^CBIBg)6IVu5>5|1WhMg`9&!duIzO6k&A0@h7$-QI<)bMkFKW!
zgme%_$iuc3SBOI4R~XTb1p3rthebr0k4o5in~&|dtwgB7Caj7sJrX7-W^Wt|RjKF_
zEu{-$6Vq{57uC{%TJPPg0@i1fR1;SRz0KvuFGNG1c2Kd>eU+{bDpdOJyQ_mGOl0)N
zQI}3pNJcj+TB+J7qpehJEH;CD%`dK|GGMsbDpXZ)Q!aA8DR&6bi#1J#v}H1h7!zBc
zVwbDxiq^QfR@~?WO(&gVWri6;hLDVYs<I?Y2^rD9pW&(^rMN<g+Lm<163n(wGEF(;
zDyN8EwGfpfwPs4Ja*}$0(Zw)MZj9YVcP{&5x98DI7+0ii-HOl?(6QU-eB=g(4hL~B
zRh110vD<2s7U}8m1@5D0Inx}w)8eDik-F>Xu=)kl8Z)&H!P47)g*0fBuSOzRwE|;x
zwH+LC9SYaR65*<09hLp6r!;=GARH6@#B$hv|BBkfcQ~!fVcQii5O5TvezChjyQl%6
zQ}T02&XP;0Oq{YtaXt1Ra|B;~h}5h&MP@O$Xz|e7b!>pEM;1F;8rCYMR}{{}%m}IG
zX-#_)+72si=2v>nluer2wL5wk&Xa1>SSF85wwd*samw|CES|&Om)gbVL59KGh&x8`
zcZG2G>#7!a^Qu_f?U)7~O@+JOw_@|?zlFUWhvEiV>@D!IcP21<z-C(K<61}g*s)`&
zvg0{w@45TfQN?D*K50ZdM4Ob2T?0#QedA+EE=VyGg(a<xjvMi<7oME5dGa7jqLd{&
zn9`Hus$c6$w{`c7EbB@-s_W!W1<ii&FXOe?Fq$m;#4aTJ$3-~nC<{hfELd3GW&wnC
ze1YVTGh`E%4#nVxncQb>Gc(CBP6Wrw`T+uIetkJ~=)iGCte2L=x7=D1>uoG@tS>jj
z&MZqjuUuXA^<ElAe2v&yXd-KA=Igf}fLK<eFI*a8>Bl-zV*_G+o7lbywK5x3S2L4q
zQ({M>065XF8INu#88ebGGJ3@hv?=^%Xb`Pw{*BmyE(*=-50YxIkR?=o)7qkeowPPp
z-}HI_VpC;@8<hT$-LbM<E~WqG0*vgC9or*9Q1iW|eyl8*`?Bd}Wy3VYH+w9sz3=7o
z#sd&LA3IzQg!P>n8$n&Y))ED+=JNOZ;|$~2wb-&)N#E~h2uB;9+A`Kd+m&lkJ6^$^
z1-5y7>?0zmD9cW6T(CD<E?$n6sTKu30k8uRC(e#7SH_0T0yYs?$M&)1qHNd!U}F7_
ztrT0Xwhcqsbh`4OVrhUTE*%9;ef92@ScZq1PiLyIa|r^yrP~o&BU)DLVxz$>s;o2W
zzjIZvpZYDf7Jao));SfI0xYd4Hr^NSabTT*E$S89MB)|Ve-hUlSW=1DW<FRZu))#f
z@K_kJ7<Y9Zha02$Pch8z(urSher#(I0|}jtn~9{qj;Uka!e*(I8HCeV<)T)0G*JVx
zuZhN5jZyD+OK6|lz9Quk9LKKoTHVgi;VGaC-1ajcn{XRMUmeLa^g`Y5>gl}6RK_L%
z@6f>0!SuQsS1~>nrzf6X*QrhdKP&k6C#<m7M_A4KyFczJW6M&0ooPR5_}`bg7s8lR
zibJmV7sm_nLX6AMivhoHO@e@#Mp8^&W<-ZNW?G3Dpj-F!S>`KH5(OKt9hI>p*~}<d
zdz-=gYnKhZQR@4RU=f8>Vd?`nN->Z>f=VjhX;NV+cQ{*Z<@=K~1|$Ao5$*QxmXYhe
z>LXy*Zj&DdSCy5zGP9^U2E9h-y|~@LYJ`@V<sp_-`bE0M&SwB_URY)p9imLCbp8?d
z_3#1XCmChtg_8JH+6BdJK+XLy7}r+#GhPP5*2F@lPAtHu24#H{CHGnE>>rZtKGVjO
z=?;DMFm22@QIkJ@8!uUS%XBXz9<D+~wjU3KmbSTNx?|J`(T_L6)wPK3ZucIKBWaI|
zq$cz$W1lE69$yS>cC@?(R;HJ<$z}NW<HXu4zqm|qfl=VcPvDLY&o;_r&`<nW<XBcA
zXt+#}mQj1~3g_?~6t&(hDF_h~3p(Gyw6D($kkr|aeG0PgFqO@t2$Tgs+njt8KjT#<
z>eJn7Ozse|L+mxM6ge7KzZ3V99VLhQBWw%@i{sgAKJ^(pYTlQ9Av(DXd+YC#G(f%f
zToYM}PM`KF@<8bbRC(wq{9c<{#si@I1;{)g{Ql`mUyALaX}^(Y4(qGc>tXA_kKdX@
zqlX%0RFw&fe!yLi?q`(Idjo)ULH%}lIJ!R&bRrpfjV|ecQlkd|63rx$X!~6zM-K!7
zNebBmWNY*wAhtf~as|lQ=)pkBSaG^iFI~-Ot$_-59myPW*Xq$jIcHa<q7OC-*g$mk
zoT_CiMbiG>Op~V4x-}Njo;F2Bg)lYTCzYuYMoAl?MD`3K#Y)@?@$}?l3vLtnIwql3
z_Kd;zChmxMPypSWc>b16ffm9|if~Su=4@m5=~)|LH;Ub@t<!5^?A8O{6M}J$jj<b<
zcI)TExD?M^85dX>F*=C4v`iaWLKX75y^T9%lh^HiJeLL8QXjG}fqdpe_9vd-VVYcS
z;BE0&cnQ&<DW;t53#=U;bccpgYiD@SITKs_w(QMLZyz3XLD(mIn3o+254wnyOv{y-
z{LIPWK}$)f^SifSCVXAV@Sq&QH~`ORFCGRo>Zw;ZJSbm7WLLrqUxo)g7b5y^09$*)
zgI0#v(H{OHJgE4B{bB!v2fd^{7Usb1O{8nOF+6B(m>svux$vNM53|<&sqP^P=o22a
zPb5S}A{1lu`tYDsDaOxXq?FL+zG6HZgQpBT{WM4l)j|tJyfIC}gIa1lC9I`%NmBBl
zl<=Td3Ky+O9_G(c+DZZ>ufUk_K-~fUvq&O10{@0EJ)}ckfq$cdxFnjA2L5Z|h~+p<
zh&Md&3m^M_UTkF^c-_i8YcP>-M6st)Dq+_6@W7kJDQCSK9(cAmp==<c2!UsbJS`W`
z!{6YkCE4ryQ{jQ<ic>x`E<A9LFXbI?Xz+M0Gz$;pNT!iJJy>?Ega@(((iqp0OxL~?
z9{9Q>Ewu3*Eyi=MU3lQKVmzaPzvTpiNn^-7fl#oFGDE?5<{rZXbJ4@u)$I({`!euK
z4jb;zLH7=&h-_`I5gypTIEU>-k}~#>y047=?=Ry(L8B>i-UtumY$ZPG{?1+C)q4Xe
zFAzLJThnUd2|)gYvCTB8T}Derf|dR4qJ;fI&FWL(fe-s|{?dsAo5|&2DX-vs{zp9Z
z@5{iWA$G;7M)cK%Dnq5S-E-7`@ix9+o&`_KQ9$5MF`=bURmIg<TIm#Wrfv!kC@LoB
z6W~2w10wP$<bWoih-f}E0Y!<5MhX>hJ4y<rrS*mf$kQnn+jjgD9`K<L=dYE4FKrJ*
z+ON&FfDfrfs8KW@-$N4~aMi+sR+}|6Jm8FCDKe@aCwgn?Gk#qO!(ny6847nEZTbYd
z{fR(tbqJ>@?^J=L<8($bvOzUGz?N6z33}=$;^?uevaA`zeuiOYL$MZ0nx$ohOBPCo
zm`yQ)nTS6h9x$=U%$>ssQO$zCmXhVhPl3|w->L*6B0iDGg2~QMjfWETK9Un2Fa@*!
z*Wb{dJBj$j09-%uK<E>raU%^3)Xb$tCbRlHk0o>ahF^HVoX~QH`Ok6yR~38}a9DW2
zays|l5}Bd-XDex+#!ce3Y4e{6MEp($+mBG({K`biAd>f=dBk`IH(O%;-Mh$K&YB{z
z_J#-KVblKp2(kVaG&wj7T=VadeC)(+M#Vp}7ux?89OrSz#FLAWgxGbLqy2xc<AW;e
zUO_y&`9KW1Hx$o7K3dTKc5f!WllVI0+E2xEnYXL*vkd1ktc2hG5!*p*?;%_OT9Sa{
zKFe%8%x8s}-%PR0WadKHQNVns{wh0GHKYtJAzRqdeF1KsMuZ&&ttr3$(U_|xCab5W
zu%vq)E=O2$_bO3(C`-<S2dom7u&83*{fh|eanFkfL!<j)+&*E+87UC`O!phO&nn?@
zNvSFcSi~(9=U~V3-wBRo-XMXt05S}hN%>Y_<qVOHJ(jUG-cP2kkuz>|HS1AIWZ~{P
zqA^>pzw`^?0qY{{!mpU4U}R8aVH2}be>AlV--G$@9y}3X$-;^}Et2=M@PN${i_QC5
z@E$XWHC+on5d72tCHetW^nZVRAZ49RgvGc8;+e?DCfqtH_z#rjE~Y|{*KzyBvzm{u
za9@gNJ0Jhx?!t|`uTa#LEe$V~ELB_QUj3~+E=Ay1i?D+=*gs{)QtGc~NnDVx2rDlO
zapj!h0Xt#xpH=J<^klf*vy7IY9(XPrencuA9<YO|`|~Lw8bRwBg{vTUq26uDR|2k{
zr^sBjl+nydUk0nR^yQ+$0}5$We?H^OAcNSwB+_4hljNvU5f>h?2NwOgUoxQO_8fv6
zBp&n(k6Re*ZWh)fpsQ|ailp#>mzhd5|GX-ArF1mIp4trmqOq@jN@KMp7H^jPCEIuY
zdY=gJx91X!_P<UG1flD>7Pm?~d-!+-w@*A9_<)B!k5V(`+9Ga$eTf)R4}TQ_{TTNl
zRp4DcRoa=fF-w=#h3&knB^qiq=MVIk@ojj(Yb9(2)$=B4uiFZ$=WRS7MAbK#5FVi2
zrOdSc^>+zbfCExhhGKX|MTZ9rz+=c(#u}mF0R#U(#?Cvws$%=wyQGl>%sB}~2uUCb
zq|iY+fdJA31Vu%p1PGW=1B6}<oggit1r>Wk<SJYnA}T7_3wG>WE0(JfP|=I>KHu5<
z1n&L)^X8M;XUdv2Yu2n;v!?9X3WO9|b!#wcLO5yBqF~fS6z8*BQ6eexgHc66)ayPS
z$P7kJ1sa2#(k?w6jJlTQH&-s6%}OZ6vWdDj34Y_BE`K`y=9*N@4Cpb9Ob_%AMh&da
zG$x^FY)IJ#rU5zz)c33-kZtCU$`VhO=8pxVGODx47TG$O7^L#73CtwWnZRL>z$S)3
zC|{ctjB2Vt1p!qfi@D+1mq&4QnTGEvx@N4n2>mysOhxV6NEzL)3r00GWl``g%Yu<V
zgbV&lZxobU#89voNFdOaz}2SU$bXxHv2}`o`U!1Nw9YRpF;4Hd1tWhCQ`4<H7!~7z
z&kUK-u7*P9%bO*Wp{C?z%5S$g7)eWd4E{?4eDgKLpd?M8XJs&QAHCwQFTC;}HV9SM
zzQM>xScCfOTLN=l2u41NK|FmAfi_PBBOj{<n;(pPT=wtTmq|hbBKMmlkLXBqmBIA;
zL`*i1=?^!ttcSV>BlW(YY4X2k0}fv!w{S-M?8^jV|4jf5X^z})dW}F2!IIMC+F;~X
zjA3*BenBq+8KiXO#Q`f6jJ&x%*343fi27$R@-`9S8%mM8Oh5<fF$p5?s;ML+>f7rC
zBR8OC0vba9-Ub-MeB=g{7{ff2PFoa=EUQj`)K4#KY0`UQ*4_||<kA8A?_(h!xtY~>
z^(((n_j61v3PxTE%(<FmuW^JxBMi?B0(73pOIaT?r;`Ss4Mu7+ix~)+_WTlzY_Ck0
z16OYr0eZjTfiSd(v}z3Pp_Z*A>MaynJM&Q7es?gkzED*7C^XNmMo}NmRimhnaa2u2
zV-Ewh>sc0ztRW09_>|X3R;!r1yhwovlPMB$s-<fj7H%>3y4>_@Fya>`GT8%z5g#&X
zaxke3=OaD_kL5EbLYJLvSsje{oDsIkDOGyM;9$h#f^Xo%A2jfmy3`#p!3dV#$xg8q
ze*_~In}DE)3Ys%$5jy@=js7mRUyE0Q5m#0N&iB!J?<;t16fs+HU4{lDE~$pQ$A{}+
z@EIF5qHCJ5->KH)mj@$KO||&Qszv1^8qqePg(G-}nK@5HCXh+5YbIClK`=ts9y*rH
zg^PLwBa)D%+C9uN!EiDsZwp4$Rv~+J1ntT3!H8(qM+OfkMz*eF-+=xUP_(9wdwESB
z8n68iku-HgbdG6iYeXsup^G#0j%qN%r~rXNz=#ek>kP&K+x9xh{M~k~9<rMUat#)x
zS~HU=tUIq{oAATYZ{{{}LH8%9!H~1eEqjg|OTlz-_op<U8rp0GSHWBT_6I}mPezD{
zVT>Fxg;;v4`!m0Yqcvg+PItNO8$@JVKmSuis#DS9^@Tc5AnEaLRXah<re6o$Uro1W
zRJ+*pZ1)e-D&)i}8}Yek=DTNGhhs5JM*O+Za$H^wgLcQS0f<h<!i_<%S?-zcmg9sf
zZ;y_u6FcaH<<uNS`0mO>(NPUTgU(nEo4=0pba*H=Z<Mi4#%JhYff&jOWs~LG^B{9!
zV|Yz$@IcE+G?%q`$FH>T7(U}Xn|i_A8V!Q(t75L!DoWJU?hX;hi47+{KuGJEvKzTF
zf0oQ*yx3O8b~L9<2NhWEk*fimAR1=H<y%g~#onRvK}CMjQj=tdlW6uqh{BPpO>}%X
z+Vrl3NkR9B<UgeDwPi-o^{#(JcHKYWEdA8IIYr|sQrlon(7h#_F`>kpgu4$lI6upG
zrKiKQe0QJMD&V~AOwi?pCHf~gBcvsd-a!ls#i%5fIoyTFbBIM8GxWJj$TateU&4k9
zOW1f}3Erz!>chj<2HnxsB^)%@4-zBoEwiPTLx3vq1rD<|;TF(sX8%ono+}Bu6BO8`
z%R^a5aJ|z+o)&P8sbXf?qiMC?;SwTUvPP^5x_#2fI^9RgJCM~29<IjvYF1pKKv$hr
zyk|(z?P$<cS^XbjmY_?0oX9_N%`oP6Y^(rc0&lL<ornx)Zbx{dG3&YsxNDOwC%PV)
zPh()~$t5O1w=3(7kv}84(HOzk-GbwAwj(8R-Q&1@4GHw38|in5Q<0<JQM%+U63-+c
z-o-V08!;R(%q;ghk>;#^1*e3IBRQ^@?oZGVqyZ%sbDK&#E-EuL;1SCGTXxW*rPOXg
zx1d``V$Dg+)`iqAOSOx3R69~y`jz{V!ansYSH~<Vu$@P?kq+Iw=Bq)s9@U?7gou1x
zhiqenx|zIL5baQjjMAAx*QB8pa-vbcqo(7opY645bJGZJ#X6U4YR?v?A>4``uCFdy
zdW~U?+D^8uRJjdXGOTbrQ5>wsg62r-`6@f56QncSv{Ic-=aJ4dW_FQVQJE1RRVOqa
zx)KYhbqjmBSTW}a61k@|=sIC}iJr%}3CRv82_vP$)T~Z3F$#%SN`%N}jMB=P8?S&a
zp%DGD9I6GWmPb44z0u?#c0K(mI&=q+M-K!Y9c{dLErFU>1|1oSO)gj8sM#y%yi!ex
zj(jNnHEe*=zdSMbJ{PCeNuhZ=6Pa$WMMj4g$|QLpiA)SSTIw}jIHsA`!5l8EvKoh0
z0!0sV4#=GGmUWz`$y2({M=u-O6aS9Qg3i4LO_kN;EU_<1Kw{41jvq_KoqJLD8tO*f
zgS!n0SwEJ@bU*SrBA?DREOPGcscb}iVqI;GdLPhQwE(@$2mOnb-?Leszar@D5;xzc
z$p}9+A?SEY%Be{6<iYXvlr@^dN7tjVn|cCXvsP4ks5kXUc{k{og=ML)8aY&k<!MJt
zQQ>y+u46WgYIY>(j0`g>HpvGbSx>+)_|euuXEa=!cZ80*GBGOXOk!xMu?e%Z)7+qA
z?s%=R2z7UZ&J=HDTX&pftPMI-=^_o;kc=xWnFX_^MO$2UV%kXyr)G7T^>C*ohJe=1
zdn5#%8Q}nn=*|o#Vdz+ogWPUVt*x92A@G-H9j*O(x5jM<{TOuS%5*eW9bdmB=qyMQ
z4qOnj8H8dpR5gw!4P>UD5_CHGt=o`T>!?~cF1v`NERxJ1VqOX^T1OBwKk&3&oH1*{
z6snV-CQzE7IWFBCpAR~ro^dkJiMbZX&Ya~8K}U!=m%<GZGMe5$THilP`*+akL-#Y_
zw-b3QksGLh_jOq!9>nb?Qe3($oE@ne{d9(;{c^DfLuT}cxOYY}kgo`_V`Y|&oa@QL
zDw>nHmBdrHpG6F32Ao978UxG5xMWxVpPz-$v|Su@>V)MWPV+}5(oQ{^V@qxNJaakc
zVzfaVBa$X2PO@P-V9yl=oyH1CTtdu19d9}cTXZ+Y^UoU#a-<5HH=Tm^-5Iq13KtX~
zAm`+U#86E$xIXYLfowl8vk?I)+`S1#2<*R`sT+}p-edoLiBf#xEPCC%EX^#`XFS8w
zCpA{onbE9pc+mdDZ}2QJ^q?BN=BO-Mqb5nPNKG_`Wq~oNM7~@_q{d<T_?XLaI&0ss
zQ;=in2BkHogh<^GIQ9x6n=mA6=X8wRI!Zls8?il^R&7~o#|LdTqn!9h=^c$X2kqmG
z;<GMN*HPYo@oQDYTC?>Yb<qBz7LXe>;K7Q7=%8H{!zsN)lmdL((IRO7lI-b|G5aZM
z4?yki2-+uWXyo1NR=jI@y2Ad|tGYGj8^FF|?8quK`>RgPR+U48QMIG8mTI7=oWg=a
zXdrs@*P#89&!Bd#5SSif&_##MfM~yz<qe2q{sT5y%h<y9ODKlLDCYux3tABQ{30TI
zix46TaKnkrObgmCtLI@C6l}zCkGykH6?@)0Wf$;-cL_ng7w^={L2eW1?ghwt!65}|
z;dveRssaw4W4QOxeeHH>?MwsR5;M@Tw58br+!_gFHUMX|q7|_?Xup}D%wx&;E$&|m
z41@V_XsnFrgiv36X}DyCy5Q-AYl}=oET-WSrV*lNkG)!fg?Q%UW-CyJhdPbrDoiI~
z8=imQ?oi-PJYVC~@(B-dla)*0uSNt8E<-sdya@93N_&a=k3%BHHBjh1JZ*8U75D<r
zP~2s8EhqRzm37Hk*l0!2-nZ9)hbUMlu`cO|;<zaa#Nt_rTS_3v$TDsRZmR-Sc=q6S
zsZt^^k%k-hH16PXR;8`>H1I{G|I5u~<~&AwBRNW2W;ne_RKVoK2O#N?0uhe{?G3d>
z!9g&-hI>in@S!^=k@}7M3ip`+&2mPK6G8is5j7ZbQB}ee?gA&KwQq3@dWS%FBU&hp
zHuTQ@Vc>*pW1ZOhb9`8d;S+RIvqm8Cd!iX-m=(L!V?ld%b;h5)jI%GyI47Jjq?~h1
zPGmv?J0}k`U70GQ@9l*~-=km9+x8-(=FJ7E1kl6&)n7q-u}J|;x+|paC%riDD1ih8
z;^u(sL2xDf93X*_!Cn?m!g%e<?Jk0Y(ga-%*tT@QPJ24~EA;35-2*Yyf<Zgk)Tqk3
zh8|@b9Tt4+Hrw71v{~<W0^NG(a<HJ?I2#l9Rc>h>TSzFKfjjtCvNQ^_T|0x}IFKas
z)T>|fkf|V)(7(o=MbD_BE&TCpJhC~yfj7$8+yG`qt?})MYz3t1+RD_G2xuJNhe*}A
zm8mlk2<P}hBKM97hHqB1#@|k)2EBzHgZ7B<w^+vilUQ#$ZOg=2wA8KG=3dH<1F9jY
z*A~RHHil<FzJPm<NU^S)VI2`4<399DGhZg@O2p5&?|gwB4n1IOLY}a|7WReK!gwS$
zrim}Hg-wV_0}#b|4JS`D7Y~{P@n*8Di7ub$vNtGwIA?cE`h%6P*Nv(hHM}=c|4!xi
zb)!;ba0kCA08b0ML~AV_z6;uMoWFGv#Wd_FJ5Hs@sA+yLfwo~$UP_=FDKY_BCbr`)
zgYi$xgtBX*5*hrVa5;mFbX?MB!l>CQ_JVEEVBR4EJLGt<3=SLK51y9nu!oJcfR+UG
zbYnrrgY>s*xZO*FCj>WMa1Q_nktfi|oFWA7`)atiGK1cDD!6w%xKC8Ff&0vZ3k^cz
zh6YQ7kcn2Sa(>^{xRUe~18r{iU`Z4C;dM`(K0U*9qolQ<36I5HM;tOc10!wv9>Qq1
z!X?Cw>KB}m<-=iu7jl(~*1+e(1C2g~&*2WGhAivK4Z(%s$Tx}9wLV2Z1eg2LUHBbw
z-!{gsS;<T@N!fnGoupk`aX3zst)zRINBQTq^il_7PwNbCQ_%aEqq&D&R?dPHJto5K
z&L0G?pwfI~1ok9-9nA!(OK=!d=A^F$IOS$wvTq9ZGK>MFb_^h|JHbl<sU0Jb$zQ>{
z$L0mwO5EXHGS-|mMZBv$h5FzBU9fE<i3D$#BE1u0h}#(qwj)mUpMZfhp?NHdJ=q+Z
zlk%Qzly`WUhvSM|wbFk=ais8r^+>I8oaUNPEDDa)Dk&b1pMl-n1G*x2LUDJ&(atx|
zoiIt}r~HfWoiGU{YWzi?Oqi5ya6$cqzi7nXPlP6njk26N(H`otO@*3lpc_m2n>Tsr
z#v<r%-c)N`3B`{cV7AMlE#=1vgM8ZR6nF>*VTKw6&F`Hs2oOWtQ%_GAg!Gte+SZp(
zFbfzJVtD+7t2`}CAg}j?;hq$bZOr}&eQFC6q%Dp&weEMZ38z`s(wisn={~2<C$)6D
z<Ag3*Vls3V{V<^mG#fwkcfm>svqKle8UOZU6S|^B!tCgpi~MmMzu20Pr^3w#D<1j2
zFd~Km>TXMzKD*Q+fIz-^Q-T;y00l7a6@XXd5}*?>9uyPr8M1Mc<TU~jH(6ddZ6;GI
zN(v+5l9V+HZP0`OJ(d9F7d%rK5g)4}NV;b_fq(*W<7a}v%-FlejlW)gZ}b^IPJUo0
zdA_{0!<4UnrO^Eu%IgjGD0hFpfO4~18m8R+*{3{Y6}rEes*7~@muk{oeIzDKx_j~h
z(%q9D>4tLm)CH8gr_NLEp7yYca`*Ik%H7k|l)I<7Sei2Qm5>|T6uR$2`o+;c>F+0>
zXS(}-tMg2E-$zS(On1FC1w;9<^OPUEkn#^Nr2NARDgQ{6hkT~HA6-DYx4mGP?tXj$
z=^vjb{o`t;*Z;H79WKh#5@t}<;q~#h^3P-8s|T?yx}z@upq?v+7P^-RdV+zzB-Gec
za~7iIUXoK?F>l8}j9GDYp_{7=cO}f=@+1P8J}+LMOQ5w86$H?2e!dl(8(t{)+@6Jw
z*joR_1Z8&K#~4MFja)nLr=kDeyor1X9Z72DB~6?tI65KMR_S6L0{Mon(pLD7!8Dj(
z+FE`MA^6+KZxl`(Z`9+j<Hfg=A1gsHm&gx6^0$|tddnZ`pa3JK0`hu1(4omf=fhZW
zjb8C`9swHCoL4J!KB}&&^O314Aj`ilbUrq1t$N>)ROn2l-VK^26wL<H)E11FgUPn4
zE&*n(rrHm0EDSl*vi)9}m|y6Wc$n$B4i_|vC|K*1c$K4rI3=dt;|k?f({_y$hf0tu
zyfmUy;#G2vyi)Z>y07}6&>19h9`~9-Y7&3pc($8JX>K^L_8Ejehwl;xXHXx24nojl
z-xfOhiXOE$mC>rj_`!y<p&q?x3TLR-0mMX|q=y`q2@9PojE0UIr+RpPja}nr@EAXO
zF8y!WLxoO#PhgznCLVqDF*l*llvW>6Fsys>5&#he<VWN+fC@+z-Ueys%WHT+d7fX_
zbQgb8XrBuga=9tQK8Iz|AoMu*&K5ehsfQrrFDbOY3`5@RL4N6xqx#riqV6eH^|8N{
zCTJ0dKD50ziG|Dl+RF{YmKPM-^j|0Wkca2vMy6Vbmi=)%wYt?6{WIy0LVKe>jy2T!
z<GQm%&_WGYdOF|UNI7B6X>T+`8DKZhEwncoaUihxUIM1Q2^3M=P0b8XM38Jpqe+*q
zBVedQU}x?nkVyHo)KrAXuJF3bRCKysVWbGC`G*OZzCz%Vo&<cn>Y4UtFb`d!1}$6x
zS#^pF?J|b*hT5l4Ey}W03jz#Qb{SPRVlTqWdMIG(pSPya&h_h`(#_AAOXE;~U7km3
z7u8<^XP?(Hwx?wb%bR-wS2`BKu$%j;pwI;Bs)B*IA`(1npV<>$EF!vwDSyYQLOZjL
zinE-QkyKNZH${ySfj|uSt<bLLHK~)b+9$6bQ_+U{R9vB5pD3@FwVGCF*P^*obj6;K
z)M`}?mEb}3v8<~y3t#lnrH<FTK0v<6(pAHi*bvuWRQRIlf?PLT__E0pKLwx3fSFLJ
zI{+~lPz5iWYQ|4N>A!5GAihLi(<|aj<;5Zfv=lbTUXT|qykf?a_)_}mD-0*tpL&%C
zwe%fG#=jen>iN#JH?ioa?^B6oat`6UE-!p0Ot?-hL-#YLedEWYji2?37>_c3))b*E
z^L{Km5YF<jm*s%TqLsAls{B7}y?izI=lpS6{ZBhcneRNDUj(Ez9{-Tp`Nak;{s->O
zpM*c~H2$%F=g$iJ*PYETHEm_gwEWq|OarFZEBSM1RD-i*SN>csh)sKTw72fqaq$u@
zqv!t=KQ1)W%3lg6Bb$@wzF+g#WA&SRmLH1F-(dXQUsr17OVW)2xgs}zvl(}Qzj<!{
zX2W{?g|+j&wPyVP*q5)}U;{t;sr-XxCPDn-d-I<P$EW|8e;7afSN9pyU5|gmjG>$I
zvkWsnj~@qZxrR0iqi_D0e7@XCEzZ<ZLVfHL&UVb?4E*Lm#AQ|XE#jI!K;|@h9EGm+
z)ySD<eYJ!Go;Py55Qqrw{@fCXz-)FJ-N%Wk8$-)6#fRapP-qjL0^Dfc$+&SpInYAp
ziC%Q=bIPUT`ZvA+ObKop2exi$cEJlY*;6$8v2WpRyy}-~vJ12cr8f%5(w)TKmeY6@
z^9`M`ga6jEM*|W0oa;J71<Y}Q#;g9j0*hFNm|+DXdK>0Z?jqa*zgA>ZtyTsidO=^~
zg`!WDT7%<&oL8wW1S<SW?fgWQ;;Bl7OsyJ^4%g}~0C|nk-ucyqHSlV+^Vo%P)wP0{
zDb&hRt+XeeI5rT$n{amAUKU{Xs6jGx(xI>m(O%E6COk?p|HvrjxKAJ=B~cJWe1v;{
zHjNvI(C0uSo3j9uY&n@#RdzhHTJLG?3PfbHwwE!4#k9=7SwD7g+WTyRm`y~CCdMg%
zaWIj7JTXmyOgxCt#Fh%Q!E+g|Kd1lB@sVdc{-~A+QVRm^7o?}3;Rw;WrT~n<4Oalg
zja6n%)R9bmI(lL$Zf2tLaD49^WAencxHaBs1>tiIRyZ7t?8s3AJN`!<9s&8oYIk1I
zGy;*zN7j_cG+GBc(qAD_f|A`)LR2%!Kigs8?QSfnc!~b$4u1N%vDBVL6U%AR9df3?
zmpBI5COueovE%h>7b{sAjeWdXKbq*RmOByVWwa+cq+=o!7QaWZ;qp1+>fOJ~vTEiY
zNa>+cQs)ICT8j0ziS-Al@r}4j@nIfI1DP+<UL4u1+0Jr5+NKKhyAWqTFpwID^ko_{
zlC;{+If)FjM16kM$cf1@=>vW0KMPa8U-|j!meYj0tZlAaKTmPiQL_>_`X1!(g(m~h
zUuDOBMTxIT{xbKN#koNBnVN}rL-?I^pgy6)khIm*?utM}9LhPfKlJx`UnImucqC*Z
z>!xAYAy9Mz0AB%VR)gW6;>vu`bz#s&Ci-?iTCWtU^|JODCUA~*T$z@;Ld^~a+~1|w
zGT)_lXZ0o|^}081)^sc2{)u|EZxUdBa=`t!fvW=U?Q}wnZ)<)~TG|5vmvbs&3~4)C
z6G}$(ZjI6vqXG9rB8~l1Z&tw7DJ`?3IJs}YJ#Id1p$)Csodd41;mqb>%X<Uv39o4N
z=-2KGxO>=%c%TQ=bQDLk^61e*w@1V>m`!{b_YWdR{7hs66-Wd<dg2=wvs6P_4~mKk
zhIr)V`kJPjyZeHS|E8Y<BobZ?17QSQuNPG>s<NCW|H+stt>8Y3IA-OM(i_*aI|T9G
zQ?=CAA8uLX;D#orsa~>GCEoBWas3tK7_9TY=PoDPYxG9zqH_WFr4T(ni)BvxTrm*y
zaHR^&!BdXo1$q12NGQGt<!_%`LDWBSyZm!WAYvJeCx1cS<x>EH)zh1<r^USw<2`WJ
z>)%boZ(-J#Y*>5h(Z54C0Q!~zrO#D&^7@OsBSPxq{=0j7SW%OfXnMjIrTR;?(6)!s
z#GC*#uR-voPfttKUuv`jr^19GMx9T@6?!{dS!f{dd~qmz;ZP1r+}gvDqmQ;}K#six
zx!SZa;NBv+%9=;J{Bs;3Ol(^-R<->tEx|fI&NR*KMq%slU-An9ce?Vthgijr54fdj
zQ;rhw*|>l^ANwThJ=T`8{z-C+oKV1B0B6k`=1$fT6IUIjO@J0T)j`oUx*_s-cwWFQ
zKpvBufw@TXn4MW9(}`#^!kB$&^`!!tzkAu29>=@P0k_B>YOeo|M*P+)rvJ4DS582^
z1)6i3LF;OV7^;?Zg5qCp9B|Qj=PSUHzE^?l89sGvMEUe*uOw>5n1I`cElMu^rkdIM
z2#e})0o3jbqC#HHv!C+gI-N&$l&B}^6XuO+eMHX5{__7qc^78~-18Z8YWPqCs&VBc
z|94cFu>&I66NCaSmvB3*lhc)9(`qqL?YJ%_;HEH{$+?&6m*x>*lre!le+1mr6b1Z7
zZ4~B7&I-Sr#^;OP{@+pIqJ4;v8Z~$;<<*`XaGs<Ma$hAw+3x|zJ844TQaj+hVQf(X
zt6K(~H%)^O=(i-`yrY2jkp>P5IPZm1XgB9yCg9QNd|_w=tn2py=ZjP|B7yFq-vW-e
zQU-r>dk{rqIeI(7Yna?CiK0oJud16gcjEtrn*ZNXs^rja0p~X(jRdmC2Asc4K*ikW
zLAy0-aNOE2=fL?mMTOY}h-n~wly5*V;M|F&qQ&kmhxyHr=r?PO)}7UsB-E6&E<M4z
zUVBuc;BLbMB2k1ziRg<iyb|6_27%V=k$>>+xjEpNW8A_^X6vxQd)&vnMKU40m{Ub5
z-bm_fGh8R4Bd$Gqx!E}~GvK4o%K}a%O1Rnm5ENO4Wzal;WuUT$N&SRXkJ%asd8aM8
zI)*XzFd)M%CJK?20p~`ccmunXvDhl=h0B+|>(Wt!Z#Df*m>G`@IQm|t*-2=2&gdg&
zt*4Jj9tvx!4=grM@sbyN$vv5Wq^AWO>8s{fQl;ki5`b0aV7i)pI)<S63}oFr0<8@v
zW^(Qm0!LL@N81sxQrv2xgR+dg&L~FZ;H^*#V~Yk7DJ-L~^mTf+hykW(CXvE6sxz5G
zV(b>&2CjQKT0l$&$Wm#&5NsSz=V-i=)CX~Uc!ywODo}c(NKgI(&P3SM{0D}qiRNyp
zqiFELkBB-aHqVxAXC38@lAC1PoxiR(mE&e+Q^wrM+wup181B5Mu(Idjw73d0uT?B1
zKY`;l8mHx7co4jz=X3?&(Y++s1FhXwdwoWnU`^n&fOB;f7hgbLJF%u^ZJ#yzT%Tdh
zJD_;oV@=4pI?S72i2cFm&DG)!tSN>8#hg}mMB?*jMWUV}akxnQ-6yd)8@Q%EiN&-^
zaT}3%I3Y|T-$XVfa{SiZC}8#EBBQ8)6S|zHZ#jUbSL0pI@D{w4fErEL+*CLH<(M51
zM6PGi4}F#CrUZTKiU6`?RU{Ob$S=!B>rFzyox)x~t=+e|5U5*YmpB7_yQC-f&*51C
zr<)M|;3MqjAv8ml)9pfx-7+q~*saBd7<>2_k@$_X15O9}p4C;{sRuPT)Fg(L4D;6J
zTaOYj)L`3%tXmocoB#s#99E2-qy}q!HLbv4>I4iDnAsoK&2O3jt!SENIgxXSRG$gZ
zO|+|4{3PxnB31tYCWNHE_$2NGLof`gZPc)3O-jbqYfky+Kho2$46$F-JK!|tS68G=
zqQ6aQo?$EpCq0~#x!a_^sL@vbXy{~k+8Pa_OF*74f)p#TE#S08LRx2`L39JiB$VAG
zrak5ckV%UPETndU&^BIz*F7MU%5bZVmGs<ca}kKsT33nN10_h$Gk4xGkRTJTy5Z?O
zIgs~9YX|K2tE?>)>gt*zr`=er(3+K${VVl&EjeI+F3k^jCT+)U=6qVf{?ZHv^zun`
z?Mb_o<9>1+H22R2?60v%OpaGcebK^n+k*MW*TbFGM@>iWXyUs1angrQOgDA*E$CqX
zi{Geb6myrMME<B5Zp{M$`y@PR^AR<#$;E3<m(j@Iq@Qu0DpZW6-W%5rR#tpfWo_$c
zvb`6uFGkC>;f*(UGk7MWDkkePm5Sf+?82G*Q?^uD?&}y4e9yw0>m;7dx139_VVQ$d
z_gk4&-KRszTZfw)RUC86iW`cxFADQ>kLt9|ET@3w2_H*RjIwVVNpbV3-?maresfsf
z+pR5yQL1L%N1!gy_*3&=0s9oyX~%cR3E<14{U^VLdqx5D%oO^<l)4=(r+WzfRx8GG
z+I~Y)cVun~;x(ne0(7V;V{un;ZG+*)k1DEGR^T4=QQIP)8B^8+Qsy&b$~^?`&^-?(
z!!fb1R;R$7aIY<gxDEHFFmaypSU5xI9YU|G;@cxAGaW!~4P^2iVdBtVChl1iXKst%
zPU$A@JrftNM&Hh$=rM(fi!yFy7*JcZV<v3ook?jIwgr=YuPu&p-NcT8z~vYT;$jhR
zrD}(;Zae5>)x<RkHErbU6U*s6KitA6!q>?}#(rP03njp)*M3L9-W>@v=ES4ua1LPX
zw46&`&Ehn5O_FjM7ha^+r*J=na;MCmMt|7dK^ZJ-D%xdg?QBa5#NNmL(<QI!7!JDC
z-dD#P45!jrrgk@n_yV?8+D*REM2@89kIu8)FU!cMnv>P9QS!UgzehJtq?pf$Ke`@>
zj{y%o3T^Ex2sIM?EW^D=_cLJrM%-nJV-7`wNZQQ$OY&D{W(A{F+9Av7d>qIzUTG%o
zsE?5sKB_UE@G)-xAB^VawB)lUUzLw*w~vb^c#Pg~({ElFDnx#mjAIN{=vNx~l8@`b
z1g`2CXjwhy2khIaW8;-o)`K*Lxvz%`=4vN_q%aSs!mjIcr;eRCPqAibAZ8tIRj$6U
zMC2vjm!ziNPvl*u1@57L$Q+q^1ote}e{iWvr@E|Phi{sOG4$65VQ~8jYVhDL)u8gC
zu2C0<8ceaQA0aNcWx(DMuZF9#8l2|WtSPjA-j@(Q>SZKk>L(QQzFN1UOTgwRpjz7L
z5KWWUe~1XA#Ej2RzD=O5QW;$(H=GqkZsH^~6<&qG(ctG$z+M_ga+&$k2@wpU=JU5g
zveXEnk7U((B&({Cl$HhTk}!^)J`Nqn@jjs&Nvg42An}<O37}1@sak-p36r(J!$<=q
z<OS?;24j`=$Wg$}36_p;<m%!xk_>M6@F(sx3~HA~q;DNfqur+kiF}i)jq79hV@+#M
zWGi?hPD~E>pk$OrsDvUAGV|m@xqv+dJ=gImtc30x$TS2uSb=?b=ugu|DsT^;$++tk
zxDn4X9IMPucX@8Z-6(DO6dp9&wEc+1BWJxn*pbgaT+jzY$-LHVtg!Qa5ea!T*^_av
zcOBB(_5K@drPN3On8clr`<+7Kupu>IWBwU&=#&JndnzJ_;d~^4W_J*1r_=;bw+!#0
zfZhobOGX-&NKvi5IA9M9v%U}*!}`vBeAW+aqh_T_FXJSeLOf6tBuJRXkTUIGzVgzQ
zz>1s>vT5gh)?axc>tBK?D_QH*53sO)7)<MYiP-ox8R>x28xrUN{nMM{a+UHI9-4Z3
zdj%M_rqe>x2Pp6%o@uxuu*mY8z81GetS|Ri-#O{OSZ_EVl8ydftbb(hg{<HEe_8*?
z{{LZpw>bfueFjz6={yr^#rj^EV!hvQ`g;6@9ce=Xc84(RtqCO|Bhej<MEk&b)s3+@
zE?~Ea0P*-LYx5y~&FSjS$KmFSuLZorl3g_rO<IIVY)0oAkx)$HaVGy50i6Q2c`+QV
zvRNHZSI1-i&~i!7kyJP2nWNKpz$?!RGw*||;r8S9!oJN@G~|)12kbMrkw^2z6cxI|
za(b)~UrWJ`!l~0SpbLKZx<Z_kt7Ft-oVslkoUyvnkQZ3Dd$`&P8CwuIZg_%n{{*1h
zMP%3a&9#n!<G#pE{}g)OV>qR^ANM7$1eQ(zn>afAabMs{(3B+^EyfqIwDzy&!bA89
z+yD{~_V2<n$OT)0AM_E=Q81=tCaxqCF+w3cj%F}VN#*y5Q`aLBF!VvmVw~yd=z|i5
zh7vBAF#4cmI{_V<mOd!C3#T0c>4TDkxW~XEeNgfN?mf8rc$N{$kkj?<|0|XMM<^ff
zccD-kePEU`2Qmg5een1Ythgy2CbR6Ho&aO7sTX)H%vhS#V{Be?kFl@OTHI?1JtuI-
z;Ub;+HG$z~1aLK47-jKVKR=G4U`9)iv#+=CI6Gq$aaVbqeeXife&9291{|BQP>iL&
zKekAW<tfI#4@>ewIy|rLKBEk`nt;0d4465?F!L`wci;>&>FzU_8qC<OJaqRNFXEmB
zjk^1cFL1}<>|<qKcklL}{}0B79y4Qc*nlwI{l7W;*xm~{+mRjc(!f)G56+uX%M3|R
z85yl*WbUa;#hGY`854Nca3mf<d){NmAbReJz{9qLuF9%7i2k+8B<*=0!GF5s!^vKU
zoN)>SKQyM_n9B%gk(qUI0qR$A!t0NDFH=9$AI%qq=zr>u?cm0R{qfe3fz|4d-P7ag
zk2jd^7!TTwUT>s)bw~dK*c}13k^;IbY*cS*@K*Q0J^pG7guh%bFbBeAMKySY7BtGJ
zdr6D&v)PB3gS^NLnt2JX`54UCIZV>I`L4L}R4bsRa%(Rs*WiZgX62Tlz;!|vS}EFI
zGT#plTvta(YUzYmKo<`31xv%;kt`Y{d=z84{zzoa9F?TYkEAs?!hlUvVCp5reg6)y
zd;uEp_DJ*aUJmQRcrQ;=9uM!34ES#lBl$znOe7eIAJR#FAs$wZxTsa2OEuyZ9^x(+
z5ZuKO43TYi1-i6SD|&@=87Sg~E6-!Tt`H6MHSCX{7=lxMvsIcd-fs3ypi!8`G`VM}
z_ISY)4OW!R(g8P-Su>}$m{khf8|C_OYlvH>O{djYbh^*(#yO+fo%lkP<@ESoXE|El
z+lj3`ZJSBocCb3FrWo$`wnJOg8Dk`Z`MHSD4cBy9O;dha!=6O8oA-5EFGBTU%~Gq=
zCa)Q{k@uOVdG?R?01W<%^6cj+Ctn|7?0qKBK4NNSzQUJhzh6(4XJI0LI>{|^^6U?)
zmyz?A6BW83&RgHD2GN51-na7XDPZr}9@slq=h+OeUZCH(JlpJ|?DdP9Yl<=}+j+Lm
z<mj{MPQKm;RT=Znm;1*BQT|>-&-e@CwA36obNW;(&n}}oj$FZ){YnUI<V;D=J^()X
zcAmYxCMD2DL4J-Yp5m<*W3+UC-Dqr*J4WZdD;{?xZMl(zymyfhBtc*9$UA1(8i=O<
z)!m=>sgTh>`tZLluTMDN<FyPGeHw{~cwK>zr}Kh*09+r_&bu68qseLP=acfftFe1>
z{+RW*yzaF~si?*BTV8jXoh`=JY$n{6$D7$sFWbY~(Nrx|Yh2+(D^H&R0*Vo#_APn5
zAPm@W4=g=Jg;P@dv3cnTB4|yHbYPJSQ(UI0jLxRyWx<cg=AOvqwSpNEIVfCqG?iUo
z(B25D`VXb9Yp5<6L;st6UFtgF-HW_6zc+P5tked5QEJKiE@Ho<A}sfKDvKy!_{PJq
zknUhUHclqS7mO|9*CwSdYzZ<QuAa6xbukO=y}t30`FDsPR1=Lz9AsHff0sH<mq2UE
z)oM7uX3EujKGeLOo;nSs(_%8+>Q3Cw7KO7)t<-7Y4vZvx2zLOuX^*8&>qt)Nwoh=!
zXkyBpY04%16YfXEu(!TH(jq5y7S}sH^qv%Tt3~EzfRNdAC^p@)!7Zs*iJW6qRzK=u
zZB?(lv<t<+u&ZF077l0r2lqR1_dk((6+Z}{H7gN!5k*7xHMF<Lp2d(fs|9gGW~b^M
z56S8*<ZD*X$@*wPYJuX8IiZRe^L6Lcv0<1Izzk=Y>6b>~ci|c^%$SvI-FMrsfSA^@
z|Fn!yWOvjSX+P?+_bFXkUHMCDd#V7B$m_LJdju%yGyHEXOKqR66MnD)LOU8l^$}4h
z>}pEKPh<9oQ^Wd=<_}CKi~>saw5C+p3lFBYpr96$tE|4es7Zl%)khyxfmbc+d%T(@
zK8)v-SDzVxy#$yQeQj#1cp^HC;Bk}XGF<lA2-JEUD^^Y)R&Zz0;<I+b)97olE&AL`
z=n(Dzt=5-i&~2HOPhA#OFVy#-HlJl<%my)1(B;R}EJG8S^j>naA5yc@{K_x7G&QTU
zX&J_-F5jnec@4xP&V3&CAnP~wAs=p%YObu&n-b)zf0+Q|HOQtWdu>a1%Kaua#NB>m
ze;qKm8K9+Ub_Re;pGn!qC3M!P6Db=7@FxJRa#A*;)vaUr*CwTGr0XL$;K^K@GDK+l
z&hemzNOAa3I?`+W?>w9`gwBB4(I@3ou2owdiw_lBDI-j?A>VyR#ZRzOM)FyQHFIdi
zmDi+<swpTdp!k^=#Lw~K)uTr|nPRk7-+M7gQhFL(2EO+N@O&;zcr>&>cT-Ah*k;~a
z>U%f@#ZcBdNt6j)N>oak>eBDxTA)i)eZ@oSZZ078Yn7&&$WPuhgv{E@L%JsNA=y&>
z&Whk14n?e)!|i}p-?1rm5&LH4Kn|3Q^_$ZI>~#aO+|;E$#J$IAc(Y?HWtJcf0Vb5K
zGk7#^n*qWFXV_RQ{-4N&3AFZ1RDVXg%_+Cds1NT{PXEP}d_RY~HnWDs`pn35Tj%on
zPeK|SjARJA|1Zj`!u`l8b$d5FYcQAeMkMV<b8=9_Dzz2BGAX4oOXC%wv{KBmQa+Dh
zvvsL!x1Ij3MAzK6@66T0oxP=pWj`>96jt+<qEk6rSO^_3NvWGz%I?Z+uYVV%);JfE
z67Vy|cou2@D@YO-Gww#k*;`7KL(qDC$N(Fho4>N$d&-0yHPL^u7*X@nbh{O0l)wmM
zR}Nq^#6H}Os4{l;^yDwyZZBrM-H4<Cp+!8emUomM#d^%rBc#VV-Jduy(&Y4>2u`}D
z-cJA1ciW@|fp7-Q0%E5rVJ8K|q}!c=7;uf=cN?%QiX&^aB5p%f3|MEO(g#%fF^%wt
z?$&dk9#8A>svgIv_U79y=dw?!w$3tA%gswkm?RHXd3b}lMk@E-bi47t5$4+8q@R<}
zUiLz*iB|eKW(k3j=1TzS=g^yG0pY3$24r;BWydf*QeR5<Cer3otfYDAwwC_PUjH=U
zV$hu?Q{2gPyGA&${;zb~95#NIt~%h%SgtqR4H|k5OQl3z7DrTCF;VmdKPrW&QB6P(
zNq$soAF76l3Yna}iA!vnZU<2*KG-!xU3G}44%JbUi0a%w-A+0WwV0^T??j!4DkrKB
zgw?@@@NwNmRB=90=Lvg^s8{Bu+o8H1{Q<(NPCX5fC$#lUh2Ca?fI;X+!&%;Qx$Nzq
zY<3k}rXN9=YO$o!s!*M|{8t8tiWw$~?v{FI`b#wbpe>617qP>%car)@`V3+RwxbWG
z>gD7LM)lH`^#9Uax*y^BRrW>n9g<&7pD6@79hLej5>v4niE%c~zO4*ODmK$Ior+t@
z*dN!+#O=18<QpWrCm0l5RtqBj(g$bs0`xJ$Odt4j2Hu0zIu-gR2|AtlLARR{KTaQ`
z617m)mocD}9#lF_0Z1ragyR!#PG59#X*q7A2-7PEeRK8bfVf`7de>`<D=V*vY7)Ar
zpLZ5&5C<#$UK@UCReLJ-MeD<d6}$}qJ@rPbtVcF@hTjFqL38x>+w}P&YpKfnmWsN=
z_ae`90(fdan1}~Jc2y1k#t>o+7L}TMDxHf-o$DS4TjwuKG>w&(lD>i1!7GV<a22s<
zr5w|8y63BmD)x;Y6jSfU-KoH4Jo|7Di71ny%A)V2weQ|dXc~nxJPpjjaE4cq$I_Ro
zbDS|n^z5Fe96AVXig=$K?}l@HN#I}HBVme2V5_Ri=7|>R-jfRgybM1B^Y3tmvjqMW
zr|gFZ0~>FM8l28{t8=AVCX_ZflQeWvLXuQ2Ph`5zXCmP<wA#Z<X@dk-eycZ|va7<I
zO>Qe~HGz8ni@;>55z?mA(GK2~D@GD~)WnJlQ)=6P)}XXBR=Vx|Us%>`M#I@r1VZ~E
zcQ(CyHXGXZ{<GM4#6F?qY+O?XAZd1cTpI-%;kg>uPk|PAw&6A@K>wTlPuw2HvjbOG
z*%xoM=pSh})Hd6YL&`Z&cwju~jO?>t5ivKAD|7(qos)(OQriOy!+DRx1}p8B+ICaA
zz>sVFymJZw)OGM_oWQdWumwRDqSoS8Az%sY=<OH?8!u+O;l!0Q1dElWcaCR-2QEGV
z<nBVX4l!TW<;5p)om2E?y>kf0i5bmuSfTUnU9ED`2KCv3^6vcg`auOOQr3$x2~9~d
z1iXzlyN5Dd{I2ZA>f$v<=CEp0WM4u?2J$Dap&EQL%Nbfz+#606Zo}OO&bn;g>5StL
z_B%atdZ?(nWq$NAKYGaBCVHnI{c{+)pUSU$Sak|V4Wm9$vFFY$U<lMW`2=Ux>otS_
z=Et(rYTG|>{mm8I;PeygpyM&zBMR)m^CFJ*Q2U8{@X&s9^wH!emf-mg_pRus<>&m0
zW6Rlo;wj$ds6pVI0`#)EZE?+2=?|#%)71H0rF???&mBb4Wz??T5U(0T9|_ZJ*jNvX
z!sokM(Xg=xY;@+s#$F)#o~%ppcZB+EVMU<l!p*r?3#retEA*XV<%MPNrrxeF(y3<q
z+)D{#6`j~zX-C1hKp6i98<IPBtS}z+$h=}im`ua%dOwCSRwK;mCxnQnk%oAw(aN;p
zQME$}t-NX073>3f&8r=4JE4ak{d_n&!(#qrC(MJ6kBDQ^;Yp=|Ua!q-QhxGM3~1f1
z^!+$RctSc)%35<3(AVV@!vT!BD?_OQKz`{kb&t~<vhmbQQ?~%@;^C?YU_<{G%>qx}
z3q5(ia*~GyihOxrsRR=l?rv0FoifR?Z%8wvMSl)lC4}}^k#P5u^qbIBSFKfV{)E#f
z{aNLkI}L}9a{G70a|5nSf!=tSlFhwafgyOF!yQ7=cUcU+Jytv@sv4uKoM8_NT@`R$
zHn5gf7gsye(%07Q`lv+nq|3*|bmT0;usMvAcOL<!F^*Y+zP2tnY(Sn=w^oNBy8fYm
znlLt{9~=I#q#vfVhYo|GIS%u{uqlBdR_9cPt%W0YXEi{5+V{?}mJ=GL{Xo0B#G?K{
zfr)7}lE<P=ROvMZ(*ADD-Mr4Qw;4<N-wtkgGOxWxtNwyHCYVNvjk5*wTNtJnF!|Lm
zUm2Lx#+>YRuHp-X22W_-V1Nb!<lKE6L|<v(Xq4MHS3r4UNfqhW;;u1*h8|H(u984t
z-%3*H1173aQEg1rCw|miMYU7i0`xuq2u>U5=&#Y0*c~BLUWfEsB5A8eGS79x(Y*U`
zcW{CwiNP$eL&qN*4GKma#FD$}i~uz$N}|&9UcfyE6#<O+C%++?_W>qF#<v+XWSGRQ
z{%PLO2cb}*8v=V&Zllq`h^Ikyb+1HBjz*(}>p$Q@1I;@oT#tCTuI^}X<qzp)a5dZ?
z#$^}XT4gt2Z3|pef)UTd^s85kERCo|dqMLX?x$r;Gm4%hT36>69prVbG#qOaBFAa~
zig&r;kMs?}h@)|qa~;!8hgab2245M}cLXE$C0Nez?}Yjev8D_8ufX+N!9ZyBtFluU
zC38dGJd*`MC8K364Mv0jzGf))XY^{fqRf_O3!NOHzkW`YolGAD<KCe2cVr0NeSJA7
z(ckmuDX;?1&A1y?>Ru?n{uV;?xcPT4Ag@~4Qz*hrajHyQ3sPtSay%1^xRJUH-=OTz
zP;y)kV@5b}gFr!Z&3_K!;zr>47#I3bq3iK|kNZjihS&v+7Yib(THJPnnBE3@)<d{Q
zjD{QD3&D)r1q?R-f(8maiYFVV^Nw-P;kg9YRsmXdK|Zd#0&n6$;1>)OVc$`MkeCzq
zn_F?WZ1Z@RwCix!AhOA)sXKQf7OSFCou!seN&;y*b;EYjmJoA|IO3_*hC@7kIedoE
z&CH{X!Ub9CRv9Bea#*f4i5g`3jkK&<>MzMCb={0Qe1B#8(NT?Jlgs_^XO&||M<s_E
z7ILr8AC`3)jXt!>PKlTD;%V9Qf)NiTK)4ou7R<p-F_KZ@ONPbFB!z+9JTX3l5o2ka
zYZ-4Hi*bFy9k^RyfVG1*f{>K1UcuL5Dik{lVxiw|V>jn|R-LW0Q&3RPE&{<)PYCiY
zCnnPFW0F?-Irmf!855OM*>y}*t*Dfpid017Q1l(-r(>BXy(1Vg0a8ZnrS#~BJqd|E
z1UZl6_9^mpvAq$VkCJHz#5U!GRl_{lX9zmlmL0)?Gb6nV-X_;;zVt3QMd07Q;4Z9X
zSqtMARHfRf%|x`h@YAi>?zLgcVQSC}L>np-3ZfcBrF^1rW$S{dhOsGWUX4b4R5`pL
zDlIm(w->%Hs&aV&6-q7e!x}+#+!uz$N#~@(3O81Um1EsQ5GI%>JAJ_jvHrT9V!2;w
zi%KtShRcD$X%XgTf9JYslAMYJY0fn0@UAm9BfiZWY7)iD^s!NOLuptK7W?feUSFtk
zj?z!0^9O82lN~bZ^^Np*D0bu~b;&mxk%TgF2Ruljop5~U!X5A=9(v(ISv7&97BdVo
z=)8mkHF@}d%`yaX!q~_mXC%VxnJp*80k0bbvpfrWBu0-(%+<*~J6NEp5_rdoV8gN5
zu1T2UCoE8cx`#8{F?H{8f!Uhd)l|$$djV7>)Z1zl`ZQe(9fbzA8hr{^)80Xpm;@c*
z6~>^MAUFCYznws)3*;B1;s6SGAxdr`O48OE{=%DZ+p+Twv}55}u*~`F11OM%2;;(s
zm<k=}DPdF<m@sV11SAJao!OxnX!X^DZ7kG8?C<KguX4+brU>o@1IE@Uqbbaa8$+j>
z_BAC4leY{9_q+_4{5vO*ft_2Ae}#rkjG<f6IHT@S7HOjk-c$lz@WQf#VFMW>SMuOg
z6j#}aw`)P8aZ)<lPtM%KNJ(8j96~HMSmPn-%c%UElj!r`P7-@iEN;BMhW#I+CvJv+
zjo0J*?Ho+`4EIq7Mz7z|LXGdYoY6X1{5xx|3s2&zG${PulNwfOP-whA&b}NQYvMq3
z@b5#=<502ioG|WJwo$~;DvOd7n1qLhSky{^xp)TPdJ7){-N+0wjgQ6Iy-A)B3JT#J
z0F1*G2;nhh`v;+EIL2e=_b2cyz_A19{C*hEYTQZ%=qii0;>s5h_(3tJ2t9ziPXQ{r
z=n(E8firP<FgO++RiF-@KXIYopzA-)tL$`hthMo}IGdGWr(h6d{U?u?yAyD_BCN3v
z-~NZ1EN+5pjHLc&7?Rq7-vU;9Fd)W!Er_wCBIb)bDliof^;%qz$)_&fsj@R@fAeJd
zDsCY!vX2Qpj2ZX9*T4vNG96i8FkZYIw^R!gdzjH>yd<T|lMRWuCHfYczWuy-18$uH
z^6bEEXEy+w^g}9x`C*>e#&ME~0v*!0^BaKvfqMW^OJ{ldt|m6SSNf<89V7jWCz}Ok
zby%_)eu*walAe)(sD15;tr17}zB;BgBdC3&xKGjZ1>2$KzP1!VgDvLO3im!brxyT2
zqq|wNy&4Q|Y=W@s$Es{KF)AWtQ0<kprZ<oKjMtb7J`JOK5e&>#Jyh=!cw49#h72kv
zrnhuW%u=H;4Vps+P1m^3(Yj+Z!f5_Q&d<YWej@Or&>Z&Agg9nijmK}Q#?-6I*3*iJ
z`wo+3?6qV!EyZMV@gIImEujueYDrtGr5e`*+)D2kr?G9UUaD71Eop$eSOHpU3F5OP
zgN8B!(YTdL3F(Jo{;gu3@rzl~8o-v}Vi2h%oz=XW`!w#M=H;p8t)g3wYen;F>a(OT
z?$UGuOnm@A+IWKd{M3gq#u!h<dQ+d!^Wg}J)7)nXChC$QKvY+(NP+ZWan#!xH#S@`
zy8V*t!WAnfa6MzD;Y50pDyAp(vLlqMGMsA;fNXPk>{v#CFP(?8?J|OkVW@1Zq&VYV
z4`<s5q|WCXF5XC>vO3p(Qy5R>k~KBn4d=QSz`Mh_SW;Z_5Sq;+@euG<dO@5WN>mn(
zEqcv$5U2y;EcCi1FGv}l$)FR|Gr~0`&aOv}LYb;()J(^4`VwKf)YX}8gb=YM5`r`R
zSroyP2(h@~oVf}V85CMbaqlU01Szda4CN{`kx(hFB;9gz+XFjNh6hjn;$Bq8&!u5P
z590PH&<791h7M@y#5uF9%FdL$CP7+l;X{jr=s%$&bj~vy(-C$i?3r|A^OW$sQ==WR
zA*>g=L#73oxk4PJtyzI#Q)ZR|=?miQ6mS-*7f7f>Z{v<CfKZ1%#eFO;N&95BGF)06
zXQwL12Dl{E6FPzWM&)!gst3)}js~L}hJN(U>lfZ9{Sxypb?HSWiMIgFB)jlQlk%s)
z8?&TQ`8|kvRRHK;E`Vm#)$m&Qt5UoXdATKPE2g(9n2uHBy{YFJVy*lnmJpCXDd<L6
z<mPE4sW}>YLO<%LZXY_0<3$srpqJVN&WS}P*FcMK`sO&hBLXsEsh?vhBkR&wiN)q{
zwq#=Jd!=F6HyUP28%FsO+)ik$36Ci=mi<yx=h95&+kpqAy0o<_pLW`^zj*?<H&sO~
z2x+}WOpNgskS2U-xRPPwsFmJ>n8@S@t!wNuS%ZVjjR}o9dfb|r7E+eE2A9VYFL$|T
z!%Xa~WL>jcOo8HKqa7BXEmtgIQ=47uAdL~!JS|>xvIomrz&Lbw?Sytl(xqLkJceW>
zuu&SBq!NCoFD$(bPA)}liJ6JXRW>R^<G`&o?6ZJO)-Hbft?(g{$>Gv*xX~R5v?um4
zO5g~ho7ja|l*-algM+2e%;Ba%p3AoDLdN8)AtP}<0Bdk7gpe7mo4AV5t+*Z7k#1rI
zK04CUM?}Xzs_Z5)hK^6O?Re;z#7c^rsC49h=~3a}<svun1wx<UwB(hjr0;Pj6j1DM
zxYG)J$N3p*wG1=OP5c#4EnNH}%S}RCxuK-G1mVgu)QX#g5H9P1%VSj0kk+K0sm+<}
z1N7tTe}E7TvdaeII5urYMlV3qG-0ase+F5NHnr2L?52fkA)cJW%jRV)r%3s0w?_V!
zjld0WNT{1Pk!)JxNk>s9wR7oz2?ymgML5oz1K>{@MkFS(d0Ko<NgEf%ZNQ&27St1P
z<5jt-cuH{7dJ&)xG)2LN^m=HKxR$hpB$313lbdt{o=P0sD{j(uJa^)5Pxq8#Qx@dO
z{w|fVvNwm7<1*}<We-rQl;bkmYuSD&M>lC7u=HRl$KovE`zL8faL+g5>u^a=``Qs@
z2Uer<VzO^F+7ac*oUmyXc|Mmfla*1kE72t0TR$is42!&8*aQuuC52)$K^mw|*u-CD
zC@vM07e&KMPZe(wZ@Df_U28rpF7mM)^wa;TbZM|~vX}jspZu1R=lySTQ~I?gX~o7`
zi!s`vX+PrLBJP~BEHGK-nq}&y!;rrRA3(yHZulQ{nHwd-n--8q31-?I^Q?M)Rz1ZD
zhZ!Q!FG(yMWnxE}C-eGjJ67H01DYU-UcKxnPVYM>(Mgwmg8K;6nY3N^O29U%YB8Zh
zRNal;gjOK!xL@oYqQ|U8X3AD&4}X&O64-DN)EOV@MMJe{YZBqprd@l1jI?a7`*0>P
zLWD<ym^72ZY*mZxlOvT9lg1Jz;5$98PR>x|g#(0_XTPTnCu1m1VssMW8exjW2};jA
z6xNAoqH3L(*+V^oCpt0Pt|Mg>f*I0@%f7{ZjZSnEL;ZnS2@FcnO&m--lDe#l2y{_Y
zLv*4UQ8OZH*nh#?Id7qUG6Wly!!c3jHOr&XiBTPhMMF$Ju9U9Crs5hJ98+n!uF@hI
z&13C0kTFLyhp2@Bz^u^nX2RH9`O1i*-InJouno_6+$hn(vU%2F>BNjdv9?y{XWu6r
z&yX%W^M!&{T{r4gLThlV6!-wo7F>k_U*fq9cMBaL>ThDAqX@BUW-Z^PPy(K}aIdol
z=0-CZWLGIVlc*E8Z%~NQ!-++YQP|Ab|6nlBkdBFF;=LSgyZodA#g?_ACQf^#8Eogf
z(aQ)m!KEpH!LouXuHcOiHyT57MF=}~#kC4Of~N#Gjp_ECy(wv{sc>1W7Otn#bk3c_
zNT$9k=BImBb~fWPfIP9M3p%EI8w`YLct5s!gTV_^kc{jV@Mr~N5d6>d?1=0lgNmmW
z5xb8!gQv1(XN->(n}IDSqh^OIVBv~e!c&NB=%pl{WcnwYep9~-XW9*%_-ZmePT)~x
z(nKYDsma8XOpi1EInz?eCBtsTbGSnah)=KMUac+);UF(ha_v{9LE)n4*(;7Qd^~W{
zo&vbB1GCqwU-87!jD14jt9(;%1w-J9Q}mkba+!k0{)lN~Ra1w~CXFW<IjQeVGv+wN
ztwj7#zaDWb8AMmcOS?xu1Nb4p=_k{7P~&I<oHejkUQC5$qGiKy?+NM8VAoC8h#|Xf
zWmCqVmB!M7w=3J>+NcJaXk{OjV&f@7o}=4LUqTHu(OTIZ*DWjpM>S{|unQ(ygqdjh
zvFAlV$kcZw0xJjL`cb}7A=#M0mERkc#Gav9U>-cAnkhNSs)3%=;ANOH{dMs{Ot}ho
zWtb^r35*GIGAF}i;z_1g$W-ElGfe|-ikFG{FCa9pIu}D8Me!t;PIHuW2<Jj(SJH|`
zI94K*E6b}hjWn5flIcxlGT22Q^1AYta3+Lu<*w>XQ<N!`!w<RMR<1k3MIo>&9}VY1
zC|4d*E;VLOsVRylnY1ukqKSxU%(sDi<GdUiVb57<z<C0G4DcC|;WB;z{5zM|g(d7O
zQ6MW%Tj_ow>rF16<oZ}SQ~)~7TE&ioC+KE=9K)tcPPultGH#4LhgCImD!?)808GIp
zO9|abgU3`7%EdJ$d(2&!S>fu0wwj`NQq=X*X_~Q`;(7s}PiA%DRYP%?SF>)X$;1=c
zELN>PB$MP}6>YF;gaTA*)g)X|cpjg#3verAckCoG%#8IjgjUT1Xihi>ZM<p~#h6Z<
z^N17~Pp&<RWqrz-*~Ra~t5lyY7(_X&Gv3uieLd$jZw#Gzout?pjm}atNXD2&q#VZf
z^H!sXl6{*MI*eUxWjq%96Zh}T)C@=l?y84y|4^W5I3Npz#W3H%WpmDZdDslM(QW8#
zRYv1fL2Op6&4Cza)-n)8(^*$Ng?kcCMqh^K72FFx`?F8e|HylqToJKfqCsb+k;$<C
zbKpJ>v;Pc%->TUkVc5@;Okb&_tH>nw^9q!;+ExJeuSQ3$j<1%fc$0}InZ6Fo)ap<}
zfG(~UDhAu=H((}K4_tYUx1&Kndyp#fF@WuHI?0xyNlf&&gnHt7C~yi7=ELfNe${e*
zH0AN6ysZfk`G;S#)z<=cjbF3S>WPF3!&O5@(Rw`9<lAdlPN{6zJvW7`Gy~x2M(v&h
zEEzDYoYK2M7K4BFa@-;XsOIWhaXX+!%Ne;$=FJnR*xS*jrLam%8X;@-W58*!li9xd
zdEB$`%&b@B4l}uiktrtjPOjn8oiNp;VhMn<ki>{shB~M8n5a}Myn%ZyO!7wrj#X!x
zBCh1}B-8z53eBz=&Lwt!9nK|o{!*Q*)a2qxuH9;po_;3!i?#Z1Khv5x?ysw%c|vZI
z<$i!QgEtF<(@Yt%Fnl#Xm|aW<v3nS4jkluBY+&{cl?e;^jzl(AHD^nj;}2=9&f2Sj
z^@IU-hDz4zs4BUp9<ENf<V*tTGWSfbgTPxEZ*sJc9CI!T=V${EtA$25*U&H4bQ2L-
zJ-lUwIa%ts-ckV6ae6mvh-WL$(Xz1J&j2$5Yx*+hS^=N8l>OA(JwtiU{OybQ^jsDz
zdfr;WToFXwv{sOnp?2emUCg?K3{the?BX>8aRX!*M-2ePv=Y)3bve-p?HUF<+1rVG
zt@DTIaz6=gZ$-oudZ*~zA(vt|&tusFE_|H65+=;mM{qLKorhcjB3g3I6jadewgB!j
zeeK&!Qk}V1BksEgLeK6Vz*OM0@s{x|Id=~ybQ|s##DIJlvd?WGi#aFJDj(D(w)t7a
zvN`~vq%t33%l{-&horm3DwwE6nPv?2l9i+q%T8Y*&pgYWr*oD`Xw{+Lp)H-92MA$S
zK+LK1<B+;q_jP7yYa(YqL_LVxqree7PvDGcm&2gA<^>!I!^uIVtwD{f`A~r$@yJPm
z<@~|^W*SZlsJT&iu%pYyC_o`)TX9&RPOd;GsB9N)c|S+KYa30drxg)!j?k9#7Q?&y
z^MQFS#N!o<U6S1SblL*Tk)7@2E+Co(lCqB!p#PMiTg$5ITJ|sBRN2kVOeOP!fXhh|
zXZb#&PhyO<NLE`m85oQ?hfZ3|t#<_2=R1kISeu38{IGrUDC4RkKP7_2gwWbTg}x_5
z_10?p@Z|57bpu-KhWZ3fp<-{y!8M_jQ)!fPRFcr-#$b%DR$suNXrJmrdM{je1zO-4
zfz!FgCXDO$seXjk;il5wEsoKo+)g@Q0^uw{wb6hrFzj|l<7vqvo#)Ar2l$q)eE9o)
zF=(+;V$-s#AAKks&G+Sr9_UAZ5RNtjUduv1T8GCZOOzvMgfk>!7HFPp;?qq0P7i#6
z&P<x<9g1#M<VUlwYoc2iEUn7@=mBA9qXb$>c7<Cj9R|r1b<0+V6djIQDbrSrWpStl
zwmBSoJs2|(^LxQ@vF~-UZ=z+b3*z|nr+sRc_G0A;>6ivowV*HR#KHo-r=!GtHu!a9
zfx=qswCvwFT2H)=bf~1$)2r;(F-A~655}yn&4J9t$dP?|BPHbFIxDaY&v@KW5%D=h
zoW7aRQQThPI17%w2FKsQXf9N87F|uQUu*HT$IEbi6=<MOKjKCyfb@AO!uL1i|9Xi?
z1##%to}?lB>t`XpzYYQOF^y+DQHH_jrdWH6nh3`@V9IeD6_}0(UAUezb@s1|@a)3f
z8^&-#143(_R(vq}A{z{-cMrqE!0iiTcn0}j{}e}Sf8}-oyLFZ2{`LmkvRmg9^_X5U
zQuHS%sJhkBXB7?KTR$S+f?9fCm60H~X876}O(s6Z#JLc&cr!}<o}>by0bU1Zy%K~<
zHvzXs4o%VxN<ur@zbZ*7$kuOfkV(>SmZ#6tretKyB)?=bQrv$fw?Q-{f{Q*USt8iV
z^qn@Wb$xsn(SHyfGC5AaM0p~6xRMMqs4%gH_!iAPOp7)A2~+a~k=FuDDqwN<aCEt%
zTfXE+Yctb`{$3;cEG_1~N|aWFEQ^nb+Q0ma{5P!`tl^-smfwzaG_6UC{_+>C^d^pJ
zoP9DDPZjQjI?4u!EVR(2O?w4nekRuv)_d(!w{bm49bA$Ed+?;=8VMhZ81|_r2(`ks
zko49vC}Kz(7L2~?J;ZlOA=yr+0niPH{cfLb9u5e{FeLVLXB4M?`}#fb^f2#8m3<m9
z-!Kr@PaJ%g^EwYzezGDev)?I>_O&@{MVlvk>A~m^*#20;TNC!F4?S$BzVWf0Iu*tS
zYc>?%MuV;G5QFVsE4W6-lg74X;;qluRx8!y=Ih@z&XJGWovcT$t?HXiCj?`@)A(?O
zsxW~<Pv*E4k4t%+?BG^Brrvk5HwDeW6?Y&o6u9kV=%^GdfDKqs8`zq$PmZCZY<QN?
z;|fi}^8@ZX1!&6+XK}v?!%8r)in=sXDQhij<Hb0(+U%3t@h~!M)B(Um2F14U3E)$~
znA=&~4!yvfDG^Pe&pd3*6Tup@6R~MFVo+>^7<H6(!#u$X9hVeoLD=lLD5c0xIjs~%
zPng%P#Ag-jkQPI%ee!Pbp0k`~e~6-cz>A5qaT*lG`~~<KQIypr81o{FqRZZaq8b;G
zj?US*UdJf2nm`h_5?UJ{!aYE?8aXDLOSWD?XD6y^*=@?!mGtLv&rp%9UL1v}(cd&T
zUy1e9o>OBd$tb9ep9$xET)K<&ZmK2FN8E}>X(Xq{vu?#B#`A_mgE^2><72mCFYV^k
zAnPVZ-Ax@j5MZ$=>pIHrqLkl=g<YEl(!coj8A((404%oAveb4&Oc6;_08E?*&}<yd
z&%RSE3pDNa!>26LV(DL-7_m2DdBEID*dmMVNHZ<F2JVU~7%wVK;!*EF;If(Gd<DB+
z=D3g=pnZgq-r8%TnqJgun8n@Kii8SI%w4vIZ%?=HEsX%)6T%zoo-n!0U@c^7Hi~a+
zhD_wLrXm-bxOPcidoR~wv{X#%WYgpsev_Lh`XmLmy&dWWCYd+XJx)6daj5xL9}63R
zTC-MUbARw(u9I5MVlU1g%a^kpFQY|ho9_v0vsqv>G0gJL>$hY}H=Ekc4r(mY)9Mj$
ze$S*DA7T9Uc^~eEZsoC)iK3Is!JifgI!C*m<w&QKdm~Bzz};_>?;{yG<j+}=z!G-u
z9+Aoon@0Vvfa!(_&9*AK8Mb;ezE-xPE769lj7-gwP^)@(uX)F67KJdXv09TYC-q6D
zV^wCVh^FPmGUr{0Sn(l6rM{%cyLueg<3~Nt@JNf$L)4@tV}GQzman58z4W+TkL&cf
zUXNLN@ZO@6woZ>q6TC~$hm7@$zRg-dk?aOKxoxQU%eWU!QV){WTF%PO^nlz;=}2m&
z6TFLxa$70u#w?^1`R#&|@f4)Li-Wuv7G4Pp5o?AHXQeT*hts7ueMBkmYf5-Ot<-)!
z=~mV)0k<Ahth`l5dQAX+#{FCU<`yP6_tz!#C+>^_3^UmLRyisFB)jeh$?Zms+#hf;
zpPf|+AlY>Yhy+|M1s=nbhD+(svKm)cDu*=4{TH{^f`@LcEYHPdEAS~E`gwUL70*=M
zTv}P)hnU`w3L}z^TjnI=s<2Vopfo(yb6Ydhqv?~*s_7*!$mGrm5Ot*|FCnKHLn$p^
z4rA$ttF9L_(Fux!sMlFwS+&Z8b(-zS=Y0)b>!{TIeyO{{seAvcRDt-0{;DHBNHVVh
zgpW_={;2%eo~yK+tIpc&a2P>kY>mSn;kl<p7r2_mcm%H2z~>X#tez+Qt5d=lj;XTE
zM*5X)=8p*6sNt7zMwB<h*6Q;W6T?xQEu2&{Hu95JbF9|cS(aTVQJuIw>DA+_dWWW!
z71zMLW|~%JJ({kmVEIx`V4mZYCb7|$l^vUYH?eE)p(|OeOw}$>SK?_|?ShdGNvnV7
zVIfH6*W&acgW6>lUzfk3vUq(|4xi(AlcIlAE}oWsJQ(p8w$0iC2C8$C0rXJ$LN<Nu
zbKh6l9S<^wpF63cgaRWDnoQ_#W#was5g#)UmDQoQM7-e*I}wZ<=8SyA3D3%KA{aBu
z5%=;kWa6O^TUQ<GNm1!(MFic;Sf*X`0BZ~nJs*xl__QQnemf4urF_c9uX&nmgIOG0
z^F=sYvT%j7JwUem%>=#!N_yWRSkM7B>G=7i%JL0SsUZf`i0g<gv#`yw8wMke$59Vf
z)tm?h^m6qZ)<B$ysdyM&%b%rBWH(eb*Z9@ks@f^y4u71Cco4WZaj&bmLtb%=Pj@pA
zRG)U=Yn!Vb$IXLWda>i)3yT?Fz@8I~_zEJ{GQDx!{op}glpj;MITYg_@yd1I_RDoY
z_sex@iSlo7d|+B<Ud(q;*Vd>k-xyUpwqvO`Vy?}ud}w1-ov4oGW|(|+V^q^-9S?&x
zR)Si4Bm(mnWiNt3)!z<jU1WrmPR|&cvWd^zXYp!8?XAFC*&WjFL&`R_5q%x-WvzmO
zUvNJwuo=(axIZx&vapx4QQB*NRm$C5a}vZQNQZ6wsLJkiTskbfEa0v}$F6?{O|%iU
zRFRHL74|Y(b>nG59YZ{LW!>FGhKl~UzTnvOGdPSnmHl|6+3#NW9ym6gCH)%QmBN83
z>6X*d6^n2zx4Y#@cra`#*oSeYA~!F`6)DinvbF>Wu)FJ)cO*c6*fK$Z5PEQHYg}Dc
zs>;Wha@BI*M9yFqy>2bVmY0zHFzz80`;Br@<W>fltw$6n!*d*`BP!+e_^rR;PAYJ_
zWo?Vcv0&ns@4*96+fo!j7Pc|cY|GKsWI0<dRF};xo-8*-UP0eor}gOauYj41D`pIG
z%MVxCovDhIMe=N+IYQDrTq&BXGXpbk8OgU7fzdZrIv<utwyRq{1lrMu>rgaq`B3uQ
zfGe|t2zFNudEsTU);lXNuQ0vI-NbCVYrF(GCm3lC3$E8L)23o_R^qlOFba>Vwj1W;
zGy=Jd!fi~LwmquAGCW6duPLwr&)2vw6~IE-b`tlK0{2+fb`;@uSAl(aLdiTBM%_(M
z;c1KGeMncMdd?+?)cQsQ-gt{$?A@6VxnDE7oJ%5qtReC4idm*t>vVF<SXP}*1AURN
z$3yF7#LfBx>)*V3Vhos`kHdTvr+Lx@fsnfx(cM7%J3S9m`W3h#3Q(Etqj4kCEiMW&
z@F+0yk}EHt5!mTfi)7|F+lv%va$%r8fjPKRx=pV(c#zQTOHl>A@;tkwQ-U|dZU{~B
z$!$GP716P=sM!{8q}EprJEdub7zl3<$Rz6nIL3<DcsBs8)B|(!1MV`av2hqJwNY#>
z$E~M_X-&4%a04?q(#}BMHfU#m_dwF`!rj4Eyjh*y&Rk{tKh(B^X$-B-Za;`SU`jGr
z_82T9BJC`&Y}7`k!SX6lFNcwsfRKy`BRNL$`(ER)=61eu22xv%=OG>L>xFV*jOet3
zieYzcoF#4ZI61$=eXSv6BOBA*%G7Q=Eh{bHu2p@Xrr?dQ0ffTZeinvx+j(IjuX%;I
z9XK-N$$%8*PK(PS$*g0Ky!B;nh*bO|7T2K@|GW4dQ$-f>`+8BCc-*MkGm!NWno}a?
zt+E&^rw%lmc-hzH^%JZgx+V2dyte0xk)1kNC$MH>x;8yF?Gu|OMq=9)Cvbcl`>|3h
zn{{aM@?RjQ4ofM!F!MLfHRN~&_4CdlZrW<%X8Sz6JWaCFZGTu+%pD`Xth9n#SSEd5
z2jf6lCOM_)|DgQx>-%%-!JMj)qUn)2r74wq@i0ds+jj-jkh3y_F`zHKIcE(yhAL$v
zu@i7w&hATlRo;Z#kx78jFpt8#rxlUX2QevUmC)%+OIGg2?Gl-Jt$Zcq%+<U+<mg><
z;_DP#sGJZRFLsyz_N+U&J9D#Qw@23uRmi^oeX{&Nw$1}Q%Hnza`@RPx+~tys^d1Q9
zNP#3Y1tB&>1uUS5qJV%@r8h+n6cwdtKnW$3&{az4NRtu)A@mlIE?_{=DA-Z|pV@ui
z1b_c;o@Xxa&bHav*_qk$?z>5g<lI;`6zq?_7wYHY2z~l3b*%wb2^(|GK>O%RSSDSW
ztnU+O_Mq}`0l59q^CyLs=2n)$y2@sFhM&>#g5{4>kY%cq@E6sOJ9g7!*CB(G(4^+;
z1!?X3yfje7ZpJL?4;4}4q$kX-PFj=nFQ*hMTWk^yV;kzp^I;{@^QUk~KP)!W5;NHE
z(O4Gpy7GyN1+gLdziegf%T|g~kKP~48ZqwViHMiFb>RjKc5~BC&`G0i#6DS^s+;n>
zx_2z*11rdLRFhb~^KGY{W<ertH~OaD$k=uzIir`Sq6zi?J@%!BIwTi<g?x|l8&#Z&
z%G8UkDiuwBjBAHtYwDsYVv?6K>n@CCpv$FYf~jaQ@r9T~gGnsJq#8^TLgwUp%3zWU
zG2OzLO;L18AxdsZ0#m;bGuB`l6k_C}EYjdWxJ5597~B^ZBcDDb4U1AP=AgmcU5L2|
zCPbop3Q@OABGyElY?F%_Ow&S4S%bN^5L3rsniXQCPbm3*!w+-qTWkvxFQW6-Z-H~g
z@>ShD#9j55D_16eUrTQJ3Y;s$dk{!<I`$`h3%ELwfpb+!l|}d>a2ni#ae;FQOt`lw
zC0a85&purT4_AQ;=#CCcv`ta<*ckYD9OcT|v@sZRSs8kaf4vxEs$Qz%YhwST_n*e4
zT_Xa!g`VLqIUmd2bC9#!A!ll3sw!*9VQ#O<nTp^~ZD`2Z?UM5xxORq|J$H~(cdH@i
z<5&MDImMx;?m_7JCp|Eo9onqkf$9Ey`MQv@;)Oj^e2Kb=(_`hCI$i??Qum{i1JjXe
zVcFFAK!%}lx@aF*Hg!L+vx?%HzCZ@Op>cnL%dhq-cLfG!3NB5lX*GaY(p8!em{t59
zkSLvL6g(}l&oC93jry7rfV0))YsY~(l1mb_1D+}Zq4->KA;2e{)6Y-m&D2DE582H9
z<UN`jmZpZNYO%%9SXvJk4G#^$jFF8UC3{K?$ufm2`T}~oO2^>->jJhG#5g*9l#Y&f
zGU_W2{*$pIl!Sy7y{M^gqcaE0Ro%Xlsh~WmNmM|GC3or85LcFm#K+}+ueB+Gd+C5O
zV2V_i4wxR@2|nCgMOo<!t=^q<XYf@KI>YK!ZLkV7cbGekji+g$m#TG90lBeOb7?fF
z5nL-ANX5Ks41zmtb+xWwAV4ro*wnN?s&xZns8Q2dubAGfT6Zv|oHUd({q<@ugAqa{
z*J|LiYCXVIa4>&?xf!a~6BHsViDong>Q!r(qyqOj6wP1=&UmF-yJS$<TAk`WP;bO@
zR;gB?1(Sg3RmDC$!b_9(^qMfDKgpCeEYK&Q?DQ-Q+4M$aI0fWNl3#TaEV}h>;GPK9
zDAL~~&5UT`uK{@y2;+fS!VEbK<>n~r6dfM8r-G9qqZsq?B2ET5sGmV8Qpf$I6@N!r
zlLCDbls%O*n)dXzB!~s7x@3%HWt=qg0}XH(rs{$yAOj7bk>r-zoWQ*bqpt}cE3SaG
zD$t<3OTy#CGZoP^XFN}!gCXI?d{s7E+-~d3z$m8eQ#sXPPluZs%vUmcIV6<j=n;*u
z_6Oun!Ks`_(VWP5oirodQsH;TJGx!)jzZ}g>r|i~MND1iW=JQ^CvJv$1TtNk$$%_b
zR|56wxQSO0&nBJF98wuu2_a=d(N0}Hf@b|37*$2t(_%?A{d*D|1UN=*Prty!dEl%B
zPVkTq{GfBcmaoe771zT0H^2)9_O#=YW}H$p>4}VgoF<cm2G}J7adk)%8b%uF{FwoO
z1w75jnUx4saGOjHWZsSq#KEI!i`@+9(V1~>1|}FY8@Wv;15CA(197gT%zT7+_DxOc
z&k#T#NvDRUXrkeeeP3X-B%i*K*p~q|t?cP5co+%{mcSw&-UY@eB}4BLsH!r)Py|f6
zrbjcu==eyWF@mT#280+!Vrp2jB`CKKrkx3-6vVwKaVb@iKNI=jj6h;R<ZBWsS8tlQ
zb}a&2%i$C{MdE@5alMKKsu#rVl(_0JPvu6A;0aaF(`!7%xn+6nfGjZ>0)LGRk)aB6
zNt2-p!^32#0==g43h9CJZiX2ZrGoONf>(Kx^j*bFx2qYo1+y4`n%>t|IwP-^n8m0q
zWsJ$0nE~TI23IFCirenAd6~Hr`I{-Gd%?J<EskBBi9pTF({?dxD?-s>7>jE*!jqAg
zuM$#4zKV2@I)v+*u@@%KL@UiI58NdIB4?!n;y-8n!b3;kS;#NKYKA?t9HC)AKM7#R
zX3eO}&ol`o@$f0IfZ66uc?U)QvlqG8#`+{UOG+74(r0BBs}xF@Z){FR3uCkfW3<8x
zjaKtP#%Lw5j61UzlyojMB}1Tv0(UzF=%dHlDmg~0sgAzxWSsVYGjfW-^IHl=D`S-#
zrSC4nD@aV0OfFV01&d>@5^ig6;gz{KFLu-<KaT6nc9CM1q9(=4Gb<pEPL7&{34fzH
z&veiv?GuEFFA()Hqm_$0#RrJ`M4rWSt~4QvgL#4<rlQh|Q%Z*jvG-9^1)Hml<uDl|
zSwfQ@Bh4|EA$1@->TaxMsrtytC9#&7&tWa2?na)u>f2b$sJpS2Guvrv8P!BrL&tmt
zY{Xbb>6>1&2w@>+<zp<PScK8?G`qA?vm-H<QBBcd;=s(l7ifU7jA|;>gap+S=!mt9
zYAV)JFf)Nqu$EE!LydyT1~y?WqncK5(x3!qe~Yn<YAVK(G;^qIj#rFjlzd)ZqaFZt
zpBT%id&O8vsy0BYcrlhy_m{@nW$M|~V`CBM%ziMUKk1m2M0Lel&f;{WojHme&jHUe
z|6R`O?<B!msvS`cQGc^`F?C<wl<4k2R|(YPVF1t%W9eqVSn86CL^UkuW*ABP=e*|T
zz&y@DmkAl;NK7Pu2co*7lxM##WMq(FGLYtyF;bIpHYy1+X3OfECSw*!GTcHxBd{2~
zYRDL=ic;pasHC!P;?Norta4LtBk-jmVNt#+9gv}|evKNAwVabEB&;Vv4zR}|fg~6L
zt7ufbkZ`~)6)QAH);~2Tz9n#hW@!@0fFZD=qu9-`=WrUyA^HarU3YW*N#J*vXmZe*
zR;8%;8cy+<G$PaEB+kTAWg=U!ZEmEd8PT(XQNt@zeqU&wiyq0W2#A9)7tNL_Kw0)&
zv}P!iL6%uhbA1QdTWV|BAZi5GGP8;_`6%(oY-SU;1uR`q!-G+^X!4w`E=?^-(!!+)
zQ<nKuL6ckRCO;8XOPc(Xo1qhFUeN7FTr&Fry$YJ#Qa8C{RIP$0WAQSvc$%Y`ZxI-!
zt!3spO|vw)XVeIK)t(D+nG*nB=dm;Y;9)8-MFKzaFb~Ky#&Q#=-XTX5U@bGB!@_*A
zFG@}lEAbLD$6;Hd>YLFeaXX{x7Q`)*xVi;#Z#)!Lryy>J#MOc4j5qSVks4K_Anu&R
z)i7~Otk$6}y;W3<PD%esT#O#y7|W=bYJ$gDM#ThmiY3+q40kjB?YT`Qbp;F!X-Hk6
zs++n(ZJoNSQZYYAcDJ5T>fwyY7e$oOeRI2nvxNIs<cmhtgxV@q?ViXEI+jsKq|%eN
zxhD~miusWp0y>|{TnXov#OgPERH>3I59wc3B=Y&XvGmJI9kvUw?&}>+DpnC?FDaA3
zlevcy6(Y4eIkIi27|MU{O{yJOHL@+T42?uJ^F}@)G4uQ;<_QynD3@*#*_wL~=hZY3
ztxZHfmds~HK7zgHHWOz>XWnzPfKneZNmS-H)P0Pry-TnPG;gu^2(ycZB3no%#x<W6
zMfuO0Xfm~MGl>M<B-35_Iuo?LmJs=Xrfsny;Q^Dhy;5PDBb#usGIy51Iw@`b8cM63
z64}HlZ7xioE7w}rs}pHdbO;0@GPia$xraKknWMMoHUO7W!*HAO;HxgRNaVfj)6IX_
z;W#)<w-ZyqaT9hpJ~x$;nn93s-5L_l*M-do6CT;z&#JM0duQ%rBzwf^UeZXhlBV$O
z9SIAo47RW`9d;{whEf>({2_H^o{MZCa@ACt5vkkPz%XJG=!KOc6D{no$`T1Z+%Pgm
zbTbk<|90dngH&V;ZW=-2M)!$iXI;lFC<o6QT#77jq?g{9T`jVdtb8p<A*R{z$Wk)W
z(r^LQ?COyL>rur;jY)Viq!_2as*q&e&6w98a9Gf@81*OCNs=3qoe%Ubcpd$G2g-cF
zp&*6AtuKBm$@1truh3ly3kg9^bVsN7qPloWY5rc4Rm`jS@C1SJza|x4Yy8ejNtPEK
z(7IxX&j=-1b)?&ysuZPqJd^adMV~AzOLLwslJvJUw5z^7Iq7fFHeJC}*53#g%_K+m
zdr3bFF$<s3Ut~!78C{{j|3RvsQSK^>9Pyb+Kf^55H4MA|O42Png~#&c>l~psw<MWw
zI^?SxEm5NpBDaeZ^;dF|j*^Z(P;+|#ov1Q=Y0^;~clo;O+z)|wkmX5zla8~Ax;T;W
zm%tjq4M@s$arh~7FDKmjO_o+;Nm71!(nTNrFJD-hhti&Bfh^N3>5}1vnimN!LSN*c
zblJt>F3&6Trcyh%CdnrV7FDM`bk+&Jp4?-A7_r4Ev_nR-DqA+`3~uzIbhx>=Kl!Qy
z)g;i3hq^!lfrQ1+@^lZ-SfX0-@DT8T1X6iG{N}a6WxcUU+~f4hON*6ma)a5QEa)%N
zul<In%3!%M#w_SB+AZ#GI7>TvQ>4{c9f|H}`VAR=O7FLnow^_J8dwOm7f0&EPdJHV
zN3qwSen30jU6Lw)z<WNLASntKwZ)5lET2wqpOUmoI(_jQbS})^g{oMbXaryvqumt%
zSpswgV7Hi)u6zuJ@3SPynjeFN=8fo^wAvY^i+_yl*Iy;At_3&c;|`SvCVfHq2?KaO
zP$_AH?ggHow33)aQ6bMr)>gr%MaWlXlD6q*@JQcwdCrmrRJ!O((?wzrL-$ut8mg<L
z1)r{@1`-byZ#YZWKQvzIkL=p*C6y#Dd-N8PP!DI?s7HUzCTUnLc?J(%!|KVizQ{jm
zgszh+pWmGHI?)df+DuKo+BPndG?Fg+^nD!H(EJWbZ&X#P%?GOVpdQ^-(wh?4Qcv+g
zh5;%m6i2#FLiVnIljiw_ED6jno3y}D<MI`kb_?UIk<kB{t%rS_HOAE{pPthiB+h!L
zK&gEC^PQ-W``Pb;B%)Quw~k|tQA&J>HY8AdFRZMiRpP8Zx_IA9T?VQ+tFLaQ)+%vU
zKOImtmFo0hoK+TINPi*wv+eq`g<zV9JdmjCQUVXsetori{omuP@@_d%0|{tZq0lA=
z2x#9U+9z5&&T4D8;OotEbH<?l)?tqq<E&H(WC2*eXBx!aWFmE~rNTiJEwXe)qI~Ko
z?q+=<fPvH<=pZkbj{i08rl=!w&YKU&mmoWQ8uts%%wuwJ5=~IIdfc!4dkpO{55qn0
zgrh-5w3fBKR!bPAW!S-6a=AJ+SUw@JDarDI>m8-;e09AqNLA)WoE06-cwTyiPLz%e
ziGp1ULGw`w^D9eWKM&1;CKA}r14eBA6PS`RH_#zVe<suikSY7pe|UHs7)@ZAmxqbK
zcnP2_=g$Drag`UPLcngBpp-2dca{yRC8cm*7rj8V%=zXolhBhqYz8(m+^^^G9+#Y-
zwnxc!nydqg?zrCHD4QjE^LV73r%~6(o0q55b!HH<(!AuULN)OQxqI3ql+2tT5XG~E
zEotYyBsS~Pi>@pd@ZN|pN@&S~^eQVkR&pao^bTp|_!dX5ga+Bz>*=gO-@uF)s-qqf
zXHRS;k^I$Ap2@%EtM55;sDJ73IF4@EOYf(dA7km}?*(Lc`Qrc&xd2OXo{x+0a0$5R
z?M%KHX-z>(oaKkcx+T|W=>l4^lv7-Caf>D|ys;WiHsoz;E28xI&G6q{)hj0^)%U1c
z;n33Q=~Y+p^40eT>8C}~HhsH<-WfzCJs20a&(%oFlC>U<+lO>GdNghy(&y;WxP4Hh
z_2{cAZr=m)j8N74Ebaj2XIV1O<(J2C^(lsp=Q7RW4#^;Esz3G;7dnIncRv`Y^8eDZ
zoT1hFhflDvcO))TlOaTkE{w~pDY`I1bYa{A^qHfJ;}+^dsi0XX?o+p7#?0q>%z$50
zJZ_1kNn=AWZmCX<77L+@<M_gcSW7YgFEol9t`$bU`Y4Cg`7&;^WM7E<!%>U)7;64c
zzzuLH5V-?bhV23d+=8OusB=UkGfo%qe?eUch?^1jy3{D?x&R%v(Wra2#EmR2Lfuph
zefyW=#B_<Dw%|SBZSDWYjd!!n1UHSk<SQoTdpWJP)-WDy8r!;jF|H%I!{a954lic~
zLOZbwmXm8~thk#xfyYT!=){I3A4?HBw|m(a^Ij8nyx0XBC}usyivZDneNV{p3kaTw
zBqvd_<<$?BmMSrlq@_%&pyOXHo7Xz)d@Rt(olIHsN|7Bt*+^x`Znafbz|ykr*ds5&
zfiL~-NtyYKdM@RXSb?m!B1CznTrVq9h_92?B}@!y3JS?_RkH^D2V)Ga8!xWm<J(iX
z5QVH_Kob_G9OC64EQ%@kh%T5C#{ZgfRI?pPNjWP|+7o^5o)qn3WZl-oA!S#vU^t}g
zD)cK;c0sh`SEhW825h-j1Z(Wkl+Uyz=c}ttUAJ=O;gI$#Q{+o7S~~W@=^eGiuS}8e
zMT=j#;9uYuL}XImlqGl>D-SbQs0-8(+<=s2F7A17Pybg$<ZEQ*m#3`NqmSr8zyNS%
znxz=$B?JQZap6eqSEg(<tyK%(2lpNkJGZ86gV!te7|t?o9KUkq2^z8ls#TfoDXY+I
zEAK_<R$-wRP5~xK046W|9Qc&L%G*4w1>^uDb7Qq|8}Mam38C;7GT0XGK@hGY2##MF
zy4u;O`jt;=kx3OXx;oq_x0QPDH9X6Rk=;sOep6TTjS62mOJ^wXBYEh_S$g@*D|%eC
z^kAbz*|HRKu##JJ`1r0ZQhbd?8SQRf3)R0<!5e`@TeVz^BvpMgWu{2ds;V@SPGG5;
zjxmBUvss~F%zRinmNfC1DYK-L#ov5YrOc`-&*E+-bxV0t3kHQ96BU>8q~mUe@GDcE
z*1Ze&DWJXdj`1r~p4HFbD-TTRAksX5=L3~ep3^<ZGgAD5JZFg%zvyBIG)s9=>o-y~
znv&9qoT@7wylOoN2GcX9gJtM^9XgpZK+<-V(7>uGZ<slzl94p=y%ZMKmA$$IB~Voh
z+)}(%`Pg<!i^}?0K4q8EqLw_9=A9Q(TGZpYsf_8jY9=4mxjX5hmwehC)2o|lGLo67
zSdDQDp=|XNcc5?-{}U2Y%1%z+rVE=PY*?6#yt-pLc}L-xPR@3@k-T4XL-NK?PUejW
zDmkgUE_rRRpzD&?h6)FD^4h{doxGlYY}rEw^_X$V)0{z_N8`*~aqS8{rjw^hcV{8i
zX8`%8ZTQmUY0jADl&V_zmyF-O$usETwJVfb)BuPR+<@enf(r?<3-J7Z2X!84$}dk|
z<P7RXBS<m~T$yIcpPH7bMPtCdrN?x#d_PKD!9^be6ZDu)maitRS!7trNdjj~uU(t3
z^5&3Qm06lRw`dz^kTue^H_131m@0wqcvt|;lK_tBBAU1e=gnSwfQOC17ZTXP!%iTZ
zF`cLPV$57IubmmwYZD8{bRP3Zaah(qrg5{~F`Xx4Ixp9Z(Y1rkTrux=Gsf4Z{dbI~
z%HX}qltvHUtF{04*zG~5Z<&|O0ZL`BlT+3(dpu)z?Fu6R;|s@b^7x10gc-ZZ@@kzs
zS4^HzFnIUPNp9>6-gUpp;7xAq&J~}ulAG$Dh0F1o$$BBnjNN2?ydVvH<$=kmBD4c|
zK2RxH@94{EoAJLUKPbJH718aLE-G2(Kl;;&OGYP$9wtNNRT4FtlKdFY+SK$co7~F9
z*IAw1+QmQoTk_*<{jB>fqT3*dGt(p^0`DOL>*O3!Vx44`-b4gsY<-=ZJWLOIp6_Xp
zteqk8=k9qRd4%9YI4F9oD|>wyqe~^0QjHPj<x@L3j94EljF{R<J8Cl8H`bb}SESaP
zj{;7uz`QE44`ntVI<;~%PTpCb-!D0}irdVUx2IOsLL1S)-yk)$Hi91A@kQhNgo>@!
zp5xSdIt4j1lBPCb$lo=K=K-Cj-eZQvqfDTV`ilhU=k@Dk&8qG~f00V`qI|28*=6Dn
ze$Q=@f-l1LeAQp1xT}QLUnY5C9e)vTfoHM)cNY~hII{{?kVHy+JKOL1UYD4pCz^zU
zqD=x_2KG<{nDj+=R2b>WC;Xnvcx=aO!@k?%U~ZTjC%z5n8j^!ya{@RJXL4RfwUARS
z`U434!enm4TynQ0w<;Cw_k35B0k>f@{DHBH{SshIF2)mI+@ypg!8&i)L+E9ovwV<h
z!wDrg?C#>XfHx)Z6A#mX$rAX7hgHCG2H^2l%F_(T?Tee@m4(9f_UG}QO5dEeAwou>
zIhLvI^YP(MM2*6T2W1=|->N*9*~8zU2YJCk&K{WJ`;@)Wd}u-)=cP!wk!?e_!tC-g
zxjKHyM1(fHQ5e%oT*njPQg9(MNo;vxp)g6h+vE3eK`0efa5`sWjIbIZ+E<?_>58uO
zar-%b&t^S(2-N?{@7bcojKKcWe$VdWOc?ZH!TyVc0%Xea_2OH-1mHdEB6>Z-@7d;;
zkgoa7Gk(t;UGpNRW?uI+HP??J3f-=n223Gypj64`sZ^G!OOyb0W$NVz0wEzPQ!-@*
zIoUML@0llL=r6>-9S)t#!u_846*$UWl|;-IoJm#d>LPmMp7eX#B6<hc<Nu5i(c5;O
zrY9@}&8QYBTf7(8QO*?fv@Qt5FTf2d92Z*Z_dEk-vDex2FG8|yAdJ(&=SpLNG9V9x
z{GR7E8$mE~KW+CA<2;?}wi8?balhw9JVjZI{&X_%9>_*h{GLw2HcZv0Ilyj^rEd8>
zFKKlIA`dtns@|<9?C#?rKf7D0&yuPS>I&{v7xxml=c*5S6<j|T_YSzXX~4u(zb8aB
z_1d$nLJbG+_j^WF6h4q(3Lr1+#-{r{9}6vMAT|PD2>CaZ{bdEp9Zb1z3)4}BpPdFy
zRhKU(`#tZY%!K^Uz5^~(=;8Q$b-0zx&iJL%NrFk(+=U@}7{&P6&p<vReE3=AIj}eV
z!gjtDbohwJ`v!?gxIK|5uCJl)zx<ww9OU|v`<U$Qt(E$msy_Qm0;vRE1LU>Vl7BIH
zw?9W{3$Q@~J$N_{9IQ%Uu>8m}&32Y(KEDP0z!{3|GdyWqiwJfx^0kDPEs2!S8XoBX
zCFK}?hx;l!`>e9J$at5Y7hPb^<Aq<B@p<IDvUjjRyQS8D5{OYgipkH~Oq!!pm1k#w
zH1hVH)M*M~_-A*u!)r@1N8{Lz$$p<SJ!C|)pee{4rI&}dED&6{>(<AlSIOpkc1O>f
zbI@D16D#WnG4WFFV4Nhfvf$GwgEEd$Z<8^Ud(Lt71;+|cA`ROj^Hih6$R=}zH+AM?
z^^|7?3s%xYGPjkZB=%s;jPwnY<ipij*bAwW+Yq4|%HFzG`r{%T!yqoHE`eh_AVW*u
zk<QsBO#PA2PCzbN*?yY`wEdFf1a>gEm)ropr%QH}cDiH-BWfwqu;iZyz}AD$^$5Y|
zFL{f{-jRfFu@r;1v^Z@&0$R8A2qr?>`sLS7TWP=%+L_Qcz28^G%6JdgGo86LEd3+G
zy$^pxQDolAS4{8dTw(p;qo&KYKBS}mSC<_=Z+dKN2Vsxtu_FP~V_Rnk?v5Ud-pRXu
zN8nq*+d2ikrP(@fV!k?r34vYc&2IQETQ9-Lc{6Au#(QZ=+Ia}I#dams9cstGy-XU`
zxa3^0#<9+@>>coT32qeK*-N@pQX^WDI<&3k9m!Wa$y5Ko<QS}FEy&VNVw@sihc4XA
zKA)Dtu?SK)QmA9bO6u^>BxbaU$#crdkQh?)9=|$t%Rw@M?JWsK#$uGce?MUvUmA1h
zL75CY7%5TACRbG>Ura<}uh&$nPZz)E9@wRqxVOuh^{3Z>sV3UB7{^C9_&xH0Ty5M$
z#ojBQ<5pc0(C4fqnQw2mj*!LjXUh6{9h3H+=V!Z!<2Y2i9V-|DqB!lkvU%q-+O8-_
zz3we4)XS`S+j%NnG{NsFDKWVOT1ELi<=w#Zo&25(UOG0EN7RE${GJ%EvT}D27(i52
z?Y9vq)6DM)>avBZmWh7*q)XL~RPaz`pR7?xm3@*?;8JCu);TE#s_Y9cRg?Sr?eAPR
zgs9ujyQ6OVs;*mnIx4qs`lVq|Pa#eG_HJE!zUob%sPSSmvmdN&$Y2La8;+%Q5Y1{J
zVUoQ&QOE)D1klo$7<-SV1jJDKf9b0Z`-`pC>z%hNqGu{m1JbI0smO01;dJMYUs)7h
zL4ivr0UyOG&&qu|1uJ=^=FduvAj!(JM9&B2YLV+5qx6PcwpXMJaY5pGH#U>3?Ajp_
zD6gG^K$+@A_<D2H`hTu9_reN(=ePNQDGRu;BKv*XCLG*I)2fXj@P0`G)uAPefN)@=
z5fdC*+qh4v9pd(t-(I2xHN>WQH<H)aygb{}WLi>Q*k<*n)ZHKZ?d6mu=6C5@U^U47
z<NUTv#>DC_{TkRN9aOiG-=3;V%~!+nh|Q2fzdA@?hyYBj>2%N!WXJ<4<b|fDm>ycj
zAX-)&?4w?iHYYc|z6=Gstj>^b%wha?20EwOZj{=PuCH_>Jg=fsE&j37Vqoa-lEN`6
z(|ZX84lTD7+qx5mqvdq`6zPP*1B6zTc4&vUbD~$Lr|vCQGBmuWvUf3mxHBmgu~(<m
z-aq~JW9}f|IWeGVdyF@>d-OnND-ZPBPla=_hljv{N`AYo@UAI=we9?Ndv^@r7}_1Q
zKSvBMnEhP9Q4;mr!<6TB$Y<L5f^Mf7==RIHxEEwG&~Lv_X7q^NUnB9<cfJXR*^=F!
z3)^H8>FYjlAgnU8NKJn8hJHJ~fc&ow`P|Q7$QL4$%5v^*+j(bJWha@Uq=RRb@LPX5
z9XvD_f&<dQyVBx>2i9Lzr4Tx-EdWDQW;V#6)04PS%Z60%`%)KD{w?&9f7v`>w(M;@
zP{nWkt6LGGcex<TZ(C0Hh8u^@j?vL%*bZ!^&uNtI(_Mq7?4!?yWx0K3wuj7Z&$0_!
z6K9%QZ@8`Abw#(@x*;v=5zt(=ZeVkFU4ebD!1_t|lVQ1aQ&;S;!}`f#2jbe|A9&KB
zwBI^R$syG=Uk#<z92tQrU$-|*!7VXNIRw>v9i|+PcbJ0Dx$JtJA>eRpAwVg4EqJ+~
zK_-M8byU@I)XwsFkO+=-Tni3}yMaa^5g;pcLdy)82Z8$q+um>Gy2vL%wl<u!L?O!P
zTK+1~O^8RBtjmzU7}whOw6b@fBtLRw9d%l}{6msX5Y;u5^&zpXyE)gvrxnEzYFi&t
zLx(Fn=Tn-lr=j|_J8Y!RGnN>N{W?a5)Nq5FqwGEL;O-OL9bKk}EJ|=Fi_gp6kxCjn
zl}JZU{MgUO(S&}~-=12gBYW`jNT^2-nk6FFqn1PyVE9=*>Pvub?fJdGaA#Ur+y9QS
zXbmhFi$D6ULGD=Gv)ykE6=r4mWbMaqz0M`idumbIzT<vtq#GD~m*1KkLeTW;?H(`j
z!n^*4cOp0gtNJb8oRxxJ=`x&x-qHc>IXo=qIzIi{Z<W-|%2$I=GqPHV5beEXgs7w=
zM9b&U3sW5-D(MK(@;xB8*T#{M_Lf8@bl=_rd!1gCrj~Q0D61?Klm;nMRKby=EUZOV
zHIO1j6&-oW!i;An2ofQx<RV*w6v>k2RCa_YixHpIwH)$4Ws%>ig7KD#Yt~3$I7v}Y
zmOFB@rh%OBv}qw^+IyRjLReMK5wolVq}V4e{b4&_4e2I3)V>yeD}^(zd(-ee4_6_<
zH^4~=FjBJefGfylXh=`v#qAv>0>vy085+``RJ`P9?)X~3``t@BfX$}z7{67Q?YX@t
zu-HpSQ^;?Coc~;kIIoBUV(Sy=A*Jl5<S@4sL13Ski2Un4XtP_JKJHMl;-U3x{a1B2
ziohcgaQ|v`(Wa4lkY9aR#yFD}I^cWFf1cs{b<JUUZtuULpY^($|7Xo)qwTGtvdZ12
zz27|%HK?(4wa7`%r?qr3M?=_cnaHo`1$2j3u>ZKzQ7iD8S8%bM8q`wRhdzONd2LT`
zw06DZ|J)HiCe(^Ik|}imcnXF7&zS;kJ5Szmz<Y_h9iMAuNq7M;n|f%^QrZ*ZLVL9W
z1-)V`NERNJ!2@v;RvZL!z|q*{jySFO7Tg6x<Vu&wpTOM!$6~`8C+|x5veE-?{W$+R
z7e{3)%h2tE7E2E+d!K%Pnb+S$-wZlxL}A}Jr_%&?UDOE)Wks8QlXQde3N((F1Ed!<
zj(MiWvA^hC=cNWWb#UL@k+-FT`%!Rsl2Tp>KSDC^=LDXk7uDd-4z7&goJt%ex<5{E
zMj`Tofyvuka1McE92}GMf=WJea7@Yz>c}v-(Eis2IV4PkfAA8gpV6+orhw=2rvH82
z&8W`(`;BzIUr~A&4%36<?~up|u0$e16Ajbbb6B{Cza3uj@G%I9K8Snh6C)(;Dr+GL
zsg(`rm;^eD@}AM<5$*uA2ZunkcRLWtTlo?=1mamoAXbhCC#UZi1#+@iBzff`KqL%0
zx;jF#@(8$nvV%M5xPMf>8qqGn%%F;f28t8ix<2WTtn!a^>$*<*Yv`Tfzr%=*%04(z
zZ|M7bI$C%o_?5phjgd2`BkGIe&o7%q{sE|ngO$)V^S@GxllW?t#T-m_@JWE*uan)m
zgAalod<R`{4%LL69sE_16ca)5A~MK>J+}*$B+G!5#U?>AV;3<x*h=J=8F)eGk3f?j
z`h=Ly5@V!&L~T+tH$FI6J`X3NNav4uLU#BiQTRZz8@9+%7`Y{!ukK14`IR<yE=u07
zXrp}fX;nF(3<dCUUd}O7fR(JX$@1BVshTxY$yV0?COY|$yhS9dXRB%ev0i2qBjziX
z&veD9LU%a7aw)=nBy?8OXyI^~&(p#~41nGicOuDjTADt5H`Vg(kVA4$k|o~`QNv|4
zOWO8=(jJGC_L3R<hx$l$Fk`sX?+l|uZ%XRMniqN$Nt*CE4l6~Ket6p7%F%iQkr1U#
z?qM5&MkZUK$VC2DH3Skl!y=0zBol*F-_PY0_Mv>*E^8sHo(9^o7BVOX$C<^D<~*{h
zqYaPIu?)VVFZu7+wF$?#K}ssQKSnB%=l1^l5jZL&Wo)FsiB8EdJuI;-A~a#{Lo8Gm
z?dh-YQk?_UF`}yv&w^@_h3Y#7U{w!j>H;AH^9s6S6m8eXmV~A|x`Qgi$N6Jj;*J!j
z${ug}V-Zt*p!c=k{jp`7XX=OzIz;zq<Bv@NZw@#5YcM)Fyyp%@GZIr}8;haK>loS&
z+^^K;(+%tsH0?-r9HWlC8rWy34;w*L%S#RP4F5>W7o7n;(D(AdL8^g$z@))x{j_NV
zXX;83FN^<2xQ2GJ8&iMRwhdP5mJmCQOz)R$uoB+so%aT-wE~Hb=&Bm5#m8uRT%Yux
z;At>f_5_Z6&kX8_?7i1r*dV<qt<;wi9FgM|M^-hMZJHoyP8-YugOMhZV`(YcFB@8o
zS;`&W;8m#L#0aO-prYR82Cqg#g{=Opyxrhc^pn0_AtR~55VGAhtFF`D<&sxs==&at
zu&O%!oZD9^Pqy^)$%eIMkBz8mx^%VbJ^DEmH%NhMK>a0Idswvs_>_cYnt_F7;t#I+
z1~>vTJ`?0aAhF7Fw070s0XYN#@(DrexpstA6P{Ju?Md6M=<S4s5mpdg9@m|jvz|jl
zxQD_O$8pu_QY2dRON5ocr}LoElQ!A{vpft8bMlidJZb0Ekb&KYQcI$j>n%R;q)oD_
z_wP0sL<8L?6!Hqt3+=NKWJ4GEHpmcVy>dIkYEu;}*2a_efkmnTog%CeIGb`ndvzu-
z8~uLrp2f7Gm9mc=c9JxYa*`||$!92&ljqH1=rJ~+&<6#}*SphapDh*>LN~H9D^X_c
zc{t@!@Ha3{<tY33o6u6_-w3NG&5j>zr^QR;C?W?Fc^8%XBO|On($!g1xEWZZ9#i(&
zy1EQ}T3yl;5mv7#^uXE2Nbolxn+Io8>Ay9I)S6;=TxZ*R(tZ-{_YF%$vT|`;E4h)%
zs7iFIl`6XJo59@dB6?o>ia*3!?tjQVDw?WPO*)F(*N)$y{BwxR8suzEEeSB7*C1wV
z+LVxFRCq3t*2rpTE*#Poa!lNDaJAMj^_?vF-})*-YOj2C>RCnO_(l99tjD>e?8NiZ
zrgx|UWxb}K1n|k%j0fJ8_N7rBwVmo_YCG|W)E07QjRoyHHU2;K34J+E4mnXNt#;L#
zzRzBr#D9~k7*9;s(W5P4SXw{|yj%Er5`LzQAbz<MuaP)&k|}iJfX1HJY)&gHx$e^m
zFsC{f$KR;*ytY**JDXg&czf;_tI}#q9&NWy_B45Nas0GdmfCUD<j%ufv1MYEW5x@8
zGX3qzut`6DrdZX`sm99whTA7|`|8T*S7B5uC+GWaPDpXB=BaEaa-kE+Qj9KEa()x;
zMDB1SWA2EoXuhwUdqvkJ?5eH3g_7$F8giOGmp1}VRv}H{n5mjR>nzlsW`2_>l-@C9
z?qX#*Pey)O1R^n94k_sHGddiQWwcixi?B+fWc5V;q@0WSase2O(Ot6$*bfo7n`l|b
z9P@F6Rf-XpP+2d@L|A1&s-`|sT%ic7tc0>uTi)x5jyS8!WG9+wt<_SL<~L2rO5tZ2
z#Bzpj4N7*+B5)KdgD#;m;gi52eMTqZye^!%j|ZHb3xbw!O%-ZwJowt+D%cU1G%f^!
zLe@6pXjjA~yjGRfrz;0yBQ94|>Y-z-8ur0PMdWFPivA?;{IU^0#0XtvLj$aR#Icz>
zc4#-P)ncpP5K)bhD3{zAN_<e+r%vnEMp(uDAaa??+TYwK(bTW1gXR$TslD~VK80`c
z&5IN(&`tt9c^C`4rPky1<#u#ttwYI)c$^ri^nHM*{**qHkz%|AOP?(`fjhUTH-$t>
zEJImxY%;6Y--z=TvcrEPi=kXJ(h1x(t&wP~Uf^hPKb@LTE#e5eP2~#VLx|`-CE^Hb
z?VBOOw_}>+dd?f6=c~E0r_!PpT{?PL#7B-D{Q5jyx<tD4%$h`D>PLt<>pnDXAMh2d
z-Pt=rzfvu%{RQ|D-2UDX)0{3{hy1MzXy-p-y3?iWnt{6qi&zrh`pi5d1DV2Q3RwpQ
z>mH}mKm9ghVGZohMOuY7cs3_r?YSj7HdL~C#3FR;>5-!1@<`qt=qiCTJoE=%EkWVU
zMd{tL)5^oZJ-3YtJKbBDs&)DuDwb73hk>wLzdL(&9(lIvjkvZEne_DO(^CFw%6^^x
zUnk#kE7?{m_!9D@g56F9d!zzAfw-fBJ$e7BpoW?E?CC5joL}6*hzykBnbuSy^PY9@
z0%Ky$yoUr*fh5!O9(hNYBcvcOI}W;F-ebbfyk}h+#e7IUT>x&nD8e({piqRTid;RW
z_BvYbjCk)%mJ1ctUN^%eiD7z-sO=qv%AVUr5F*TA_F#uJ!s9K0$kThP(jz!!Luazl
zL5Hf^T7J%S68RY$BYq=`Ah<pQvLas5>EL2rIW!gVieAq~R4WgRcqLBOwRj#J7x79X
zDGo1^U5JQ5A+$$uGF3NAm3sq(mchh}BfmB^;u&2eGJm$4F7p|EB8_tJsh(lk;%qll
z3@@+qv2YRk_`&4N`ZD6-f}FE-&WE*+YjQr!Z13zWlQZ-P1k{O;taaLub;s=pa|%u>
z&CyxSF*uX_9!Wh^sM|LwqPAPdIZG$3?Y0=|YTu7fV3|yHOeU$~B00;JjwoA@Gt@+9
zEgLJWG~|_SM%E@KYx#n#aoyqayq$??02x~Lxx=I$>QBs#W&X1(jPKLGn|QV+&#y;x
z@2?WSqrLklfqvARD9SVWb)FkdNxbSlR|zFv<2j@w?(rsm??w#hlz6>3-L>TfS=dgr
zDlpF&^J=0MjnDH?Uu{7WtrDbwiq=e^UQ67nNy=B>H3s>+2*`yc`rK&ZR>zfFw-DK#
zUW=izG;u3LiXB;Z4mcJ<!)$4txE<wjVF_zgWLO^tE+Z*1+r`}ru2F*P;%imenIvVJ
zB|4{H)%vGN(pKx9!~=#jwZ1br-h{T#zeCd#>m=?dI2?L`7l^*o3ZYDBY9gN{urJ6J
z9v5lp`d5Hn1TOr^LkM_70zdHZ0Wekq7kT&um@I+gJj?-RNnj5Tp96AA{K6I<)&r|~
zfhL|MkMHoc^e<dWTvePDyq*;D#7l9Kakq@X?{G!_|GmqWdeDD|vnTH+UT7o3BwTO3
zeTVsx^QITHe;dBjwIy23<Y9d4cOmj!GscP36Kd+ot7s0D@-cY5-6vd)Wv8GAiq1Jy
zE@R@MzA4PuKTljLGrWs(K$?zOT9qP~81Y+*QmDU+XXI=dos#@I&&b&_oDcnsoGo|p
zh?#7<B)_f?jwfc}*D&Q3HM3?%qB(P!uV!|H>Ijjd%bgNYB#E8$?ivib0I-v+R#i&u
z=E%ku1ivWPXGkxV*aQ3cP-o*J9H^A2&xb<X7f8<+jf`mYb423{I{AyXZ;E_}5*N!#
z*z~4}7>QY9v15@ep<LWZxrj_R8I}V`ybsvL@{cF36BM48j^vlIoY)WJ^=%vr-*J;k
z^Br(m0`KwgJMgmvURP>E34k}%JaPSas0vh~7iQinqC5%J(zEi4m22;mJ}Xs9=a=`)
z6NfVu_;#jK`iA;sOLj`%0I3_AyQOy~@S<C~)X<00XFaP6kGWc`a!BZ&HAie<<k7^p
zh34PgQkep780D5akHBoV)U^b7$HCByCEKt~O5G}@ie(8OCFWJLWELx86n#E%97SJh
z<raO4JjW#vf*%|H0Df`G-bin4<OG;5dt-S5rC9=+^}Ceq@_9DbrsgYtg%eZZ#HGzn
z!5b6F5HA7B-Pjzs+b#Dw0?)eT_9D<7nr9bnsabK~HLlZ{Io4SBOQ&U26mDkQ;nl>3
z&RmUIJws)GNwgjuwwcPL${T4jPb<%`s|Y8ap4kErx><3|=uYNYO-<RC?=C3Y4*!>_
zfD}%`*%cY_8JeVccc5zu3R8(S&H6MU3x0~8zx<OPXbD+`dOl$-Dzv^R^z2ay+g!b!
znUk;$yw>M&wG+0x24H_wLbiU^@}6)qM84gN687n_pkM4wFlrQh(-H~$W!(z%EC*u}
z4*E3_V(*&$lrT|~gd(`|GBi>5L?#J(kW=?W#+iN|>QDHvmM8^1$P+$9fjEmb36mJL
zS3)l*7**c*4Xw)iQg4}N33CiRy4p-LPSHR2ze&}*XD7T_Q1w!(4sV<AW>u*UDhH?p
z&OEC6f0<^z%Q#bm4E21NPFUC$J2Ni9tRCj88KL<~S?fi=TuTxCl5oGHUpBsrw7kZ^
zpZG??BSPuRgx3M9`gDD1m`Z4ky1IX>t6&lyH>InMIV9OlwmsVuo)TOZ;d8(lt#lHe
zchmg@?&@b&NYqZkuzZ!VxKGzXl-rYX9|-cX4N*Rs%KmO3g(cQWc*NP&+Z0f0Q-r(#
ze=UVv-SkLXn(#a$Ew2%V=2}nUk&aDOCD4(FBtTw(zlJDnqSQ?fNZ>&po@9+`)8p_c
zaw%5t8m`tR$?zPP=!k}(@M02|aIc)i5skK~2hiOaX`6-+7~qVwP45$U&lzc(u!oz(
zjmzk-2O0{kWctAy#fpV+k2Tq2#68YH*QkmKy%@`R|GK2lA*)zVO*+im^qEWgUIM&V
zYDmu|AcqSz(U%Eah;@h_qrwV`E+ic7yXjZCgGO|OoP*wMJCtvRS({;&);*h(2_y>T
z8HIXH=I-_6z#(5i!5Uh~b^-shc~+5|5?+G;SHEx?u^BFJ)?9|2o1X&Om_~%u=I(@K
zLL(ipxj!Hin#fH~Q#QW=W`x@m?(l;WsiI_IA8gJb$#kJtTJkBdz-`GU0vp_xd`;jh
zX$ezEG_;uUAxIm46guOe(M(rQ6lf+hr+Hk*o;OPG>aTYwJr-ViGL1#)WrQ20cg^Q2
zJt33n@|iWXORsAT(i4b`EK!6clA&EHAN$VuZIznP&>g?mM$;1vu!i^$S6+_n)@z_j
zXjl&to9Y*?5*jvfW(tfZJ}ARzVq7!c8*4SEKB1gC^TVud(4{Je!mL@kV2D`9FniPs
z0ykA3U+D4_f$FqBdW+gasspvcETmM)jav7;5@x-HBD?k%;vU{M%z9TBFJ}!B7KT~l
zFizn<?dpbE;~gDHwx&(Ptifda{%$IhgmMV^`@4O^RhTuLc5!R(2ze|ekCfGXURcN)
zArV{>My;!Qv2EfHvtH0dBNE^Daj1SlvdEzeq51`u8X<ZEsU`%%tacLRq5AmTFzY_4
zKFgu{q5n|*C{!m5aHxLFp<3owX@|pDms0liN)Fi;bKUEeeEs@%gW6&+?EYsdaZ_fP
z<s}6xL%qQJ^dtWBb-%AqcWIZO3RG7v3J9|Trub+lrnZjJ%N6&%8Kcy*qr>u~!Y9Mw
zfA+&+d9+?{hD_)dmRDAuS+)qfrfX27kNq|5jHI6!scSkDC`^7vCx?d=bEdZ75r|?h
zhFy@%X_C3~(6DWi`AZTJu}ut^tesHTwhAKua;*xvw=oH&Bc5QgniKY(?gDb%B66rd
z?7c$s8aAP(V_t(>!X|6eDF*PXq_Cbk6LsCFV{-MZA%>1;Qua!b8JD&^Vfv<urjjev
za9~)of^3~lwr09AldXADVK&F4knNqEu$l$AJ|QJSTC;+fJh?w7tS0s9*Uks1u-X;H
z`bk}*uY{E=$h6mF3YDv(a}Dj%SA~@e3Tw2{469(WF^s;N=(A<JMZAq41_b&mf>k9G
z98OqBfTE*2YLWBVgIFQ|VSsONH&P8o{O(I4;=L(8dx-Rh2xdeHpFL6nO?|xb6e8Ul
zIz-g!cYOAnKBu(g)js<zT>>)pLl>eG>rj4WW$g>7=q_xDZ1dS6IK|SZjGokQ83<c{
z^x0;sEniI-!$d(F@|(5egz8vLHA;}#+U6^9qFa0+JGQoNn%Y8xw*>TQF_JP3>9+(#
z2jFp1NJFb5YI6E!OB(PFMEA}D$zW7X%SjLT?Ap5Z#O46mL6$=%`)qco?3=am3#t$9
zHV85jb)!h|4-onltRPzouUJUMlzbUSal)o#VGNah`3^7|<PPdc5o8vK`M`9LJ-_$a
z&0OSBkU1dx%?61;=#>A0yy<vVt-EWl>Z9s%y9W$>$;r9(A-LC^3pISUdk|o2XHYMK
zXJFeu=25EjKyA|lYU@B^dn<|N*7E!zM2~LhvmbI4%hstxe+)8dz0YppB3FT2@fiHL
zJ70as+$Nu|G8KW!iaxt_aTMN7sd5XjNdj$oz{YQtcl>WY%)=Sr8w}&kS9rJ%T#>+Q
zyqQxJ@G!xH8pQPHZh9ZhXLolh-d2+M$R8P+H^-8s12H8qPd8V|i_bR~$<K7Gu3x?t
zb#p6e8v@Dn=B--Xju4QQNdEwzeHXLx{O_RVRs-T610GQo&HCPlV`VbJub2kgVuA0c
z4kVNXzHQF{?PARWA1OH4pwr0$9|Eiw_&{*5!9f=+@R_i)z_$&V-`0up)dLWDn$)mZ
zJxgm_3f1stTVJ5J1g`Ng4B)#X_RZgTcncUsfC;7AHUSuC8VCOm>ehvPcDN9|!)+af
zZ<``j-HN8FJgJxHi&6#->9%=5rnm+lE|#MZP{C@VoV70ME-{SKv1}*FLgdfQ9HU>N
zx=ant=}}fr)3^V|?DNCx%Km-1tZ#f2afgFYqNAi8qN|2n5t@l2Ey`ckXWv6#{4CUp
z%xq&oY+E7$WM&(qZ`*bWAT!(Yf$P$@WrY_A&h~IXUJ4Y!*<KsCzdBP4<q2+}H61lK
z#`)}*%2U2fM}x;nau3j00(*F90q7U~COA&h%~I)^zZWb0^u+nH+QK~U!=AEGs^1>%
zN=0`=qDr*3sJl6THOXg}6LWiWY(NkFa%hnIk{E_EkjtS=^`?vrn{xFDG)<5AgAYPx
z$??oOU;C^-EkTn1RyFdVb*w*Qv|^UREWY<cVF5k3tv{PfVQ9`q-1dE25A*~*w^Dg8
zYArA-+ShdmTF|zmC8ck0zdi&!IzTp2!9`3U#5PG2pIw})ZdEgfbqTTDx=%b4QNQod
zwh!m}te>Q9w;ph4`zcmvQ@vQ4@%XIm(4PFd#Q#J5P(PSjVj&4UU4=lZV^)G&eAbQz
zV)&rq-B*3qo)Dd;U+jPiYag?(Th)kK5$UrIp-z+K;x=K$H(CLkMq17Uay@Z_2m7q6
zP?~xcV{*$}(nL8NY=wSlj$}T{6zYcoPUEbjO#84j`h0VdULl`2gogNE!yMw@qO&yd
zLhsv+9eRbrcU%f9-{rH+Jd;Y$>DGAiV*0SsK5L?WCSqV!pY@>|5K=zU?ZAvmvNT#4
z(uNc(WKGfr(rMxgHFQ7Dc|j(PK5GM>na`SSs(86C&!3^5ehs71zZ|9>^tbhnX|$j&
zX*+FYme;mNFu#!~^@z_}isDW^t2yJH>9bb)>3{6Mbadmc&nq=*kk9IhzKh1@syvx2
z|9X{?B&2ucMG<;jZEp{V>6C8nO5XarK#T@BQH4X~v$`b<$3T1z%%jHH6@1pK{MRXu
zf!s$xx_^ZDiTGw}2YP<{-)MQ&nJV`Z-M697>aRPF*d!n>U+CII5=FGMxIe3O(vv%0
z1fJFNkm$k6Q>T+CtU(9$E{HjgAgV^r@HNfxS?nLE*>rnL#PzrPL;B5WpVg|YywD1(
zeyif}vvpDB+{yA=@_$jTBsnQ$@Mpi^v)IBW`^WkXSwhHA;1n6Bukl%YFIA-;B~ZM#
z&q|PPl+h~!6{@Qx%IS9JU8d_Qg@u<mU#^t+yWEzHfaD@A(H+Bdwo^J&gh;{}EeN3L
zUIqcZ%QM5`oPKu7b)19KjyT2WO$Ogp2w@#oU$W-=UQ?yz%+hiOfg-+U&sYe0v4L-s
zCg?K=g7!`9VBe}nZT0H-w(8pSyY9Yi`WaUKKAK2$jl7Ag$wG{Fnz2X(J-D-)3faGZ
zM!N_z=JzA8dL4^|hwn^y*d4RTX`{apzyZJbCL<05K7y57MNI!nsHKUTjFjF&@$MJ{
zyjD#%n|w2L(+DpEmQ))wn87zgW(!%gf5%3U>-65HFT+iE0Nh@Ocjg7B);TQGetv{1
z$z*<25#KD<0{*}q@O*xm9G8Ie81>J8=HYMPFF80Uha(4U6(@pUhLUeS=J=02m@)Ao
zLfJ4gyBOM3PDhdgCqkzyJRK=vd<X|y;6s?O<3nU)2(v|_%NnXC6EBK9{6Z$5WVZic
z-9?j`9YfNJ4w=~@IyO5=axDpQ?2Y8+mt3D?O8=}+t|ik+(+p@TfeAc30zB+iH84VF
z;+IUkk7WPJK8mU88Pc(_t~rw3kH9M~M+R0Djts0Q<3HTD0+#%F%3(?NYa|@*aD;Sl
zgkK8Bt%@~X6=68yU<){6!c-@>(q_K{yiK-(wxwz^`Gw5&wCyj>5_v+pafng&M7MD>
z3Cs{5XTTh7?Crl|Q7CqJH@-Hc23*o!5rez8@(n?yXp{Yy?0zsLhS1-)sxim)4cB8R
zqUVs8l<#c@MDnj9<1;t<UPJJ<ea!Qmdws9_8Gj;$?O7q;=<5zg0@7ppAo=arI_ccq
zYjRu7aH}`+UB9TWdtKtmk-Y?1Tt(N~Lu&)V+kh{@QB6;;R6`h1&fW`->Uufioy`O#
z`!=T}sEqoRvTw`%wG2pkBSZ}USET$ZOcwb?mPiJ@s-3SFXIF2xbUOE|GNdVi+$l~=
z=pO#61(*k!)f-`ot}YChU-oEO7&3zN)srN8+~M_C9jX7T=ag(H&fn!T?}u@jZ*B7E
zTP6M}lBk*RSXE&HX-5D<3@aX^6ND9Cr2!LY03FcUVZ~RAz%2j=E7~}$_-Yln72sfn
zo^eaw&>?=lh8Ma{Sn+pxQoxGVE-P-4;AfW=q!L#2^*s(N{(jP7#ZFLP{UfaC>#$-c
z=42-d8xG32D$!RKZ`E*LTjcKV*Bn;ttikx-S+%|%RH!>SVq5}UM#?chIfC-{Qig;a
zFWz}C&;&QS#C(p*)<hi5@nYRQg3~cfOMTC9$M@f-DDv~mE-4Jwoe#U6QX)?}MSkg&
zj>To2R7Su~?ETKC3?<ST<4^mzUKiocSG+}|l)gm&-<M2h)I+~YXYA|_ykI(`UL2`l
z=+2LT@uoA9ozB>~5Zrvz87WR@>|6;hs|xZ*Z`5}LbLW1LyXg#FC!O(+<f1c@ozB=P
zytw3c2C1YoKJwM4GyeI;?F_W<&V1<%zUiywau;>VVoeRx870RE3*;9TgjjsD{|$F~
zWEZl(iwUG?Ij8GNvU^L)4Bvxv$iGQ$!Wa@(HVGdwJy-H~-E)h54<YKh*LJ~^T|s({
zQcbVf;mMSR&lKI)YnPUCLu?$WdMv&Wt}50KRwOJWfN(pDFJ8MMlNeP`dyQVZ5~ZcS
zT8y+Ey>?|c(6o`)uHps;R`ojX{mD`cMOPID%j;aptG#wLH(5jpuN?}yQD1EK+I6%K
zO1j85ymp)pl%nbwFo|`b<IY7IuN@y-xJXk10UBM)Ysp)QA~he36}5t8_IO@kk%kD}
zIv9c>+0{@l(tworuY@1FdIFvGA`Ms^Dp{o2H3|5jwq}U6pAjXCG`ltdaw55Rme<;k
zBsz;UUhAml60ui+vq4#;@meQX!S=l1F4B0dT-|u~mMCs_p!(oYH)zXkAuI)f9^U-`
z-U^(tP77mDExS=AyTyy$(bj9;0YuDqX9Jr+HoFHTWs2<Y{t>tgvgh|+D^Ie6{F-)u
zE%%A^n+?)gx%v8OkPnN4!3L>iyt6hDdTqmNM@mk3^7TAmHvG3{8va{H4gVj4z|)`)
zQR?W1Uh7A{li>?8tO1#{-fQK%$YUUnu$rM2(fE=g`|_)bc<qRy>`K~Uj!b`jo|I=L
zFoB2ffvfK9c|0STN^e436dydkn?y2u{`z-7UhkIK^PVDrEjc`|Qbmjmn}~l+4o@8t
zdB&3<8t_YC6b~uL|DL)G)rfx?s-9_}S^<yZ(u&tKe!Tby{Nf^btuOu5{Db2o?4hUj
zJni@hq`*he>F)3m2;(C-*a9EHgsDw@ggq~zfA(~yeA%NNAJ4Gug54j_Y@d}6kB2Lt
zSoyHFIx8P-ipWaFgm!nVWZVS@#8voUx87y(cpep3K_(r}I|E)TM#>(?F)Y1+Fk!Ir
zoH*nxAi$|mX9`HP5@i*jOQhHOLXRKbxbv(HbQjtDqMSS<5<S`wK$w_h$|``@+NdQg
zK~MaxR^w#C2!&B3c(@k>^C_?O0fou5+*4#IX@~m3h_*J<assLFUu&G;7Oyp~a9-}U
zCWTXxTqg8j6R-72St)_Q3J8=d;hbr?*P5$`oI`?{mba-$LR6Er9Id<UW5l;edoZXE
z>l35J&9vNWWuReIsGgZk5xm34Z+@jF7PXz%dem(}(E&~i9z`)ab91ltSmE5<Yhn8h
znU(MIT4qU|ie)Ib6He*5xz~C|KNEqHwA!1?O9_QDxSzd4%~~C_q96usY(1w{;bKc(
z3-?-Y`E<{8qivqG%$U4ZPq)yv2<iZeDLR12qHV}74R?GALCJde-tWBDVBM`MTDFV4
zGriUbzmTXW;!O`S5g#_vYuzjUU}x~-MUNJ%geDX+@2w`*0O=h7^Z_XydoOuqdc6mo
zxo12`>C$G-M10Q*ke{O%q&FYbYyxo<$O)lLR7*-XlWL+JQ@a=Zp5H~_1EZWdwLM5q
zxoYV)-)nu8!=|IWRvZ#m>=iwc_FBwm^+eiJOz!TG*4J|<(w<^3{1-*n%jLBNNn|G7
zBEf6<nPz&5t<*DVLiD(vNzXg$wW9DkQzI!xno>eH#SuR<cW%>HrmD4zc&|zsVadob
z9Vn;YoVm02dz~qaQ;y-@AIjkW6dgc-KKW7ir&F4vlRGkF?)<jb3d5)O{-%3b@AP>8
z($B6$6r}{@$-Ig$1k_LWUeE+Q3jvUNp|Z#(sn#6!ek)n^M?@Ny^M;nY13qlONy@1y
zlJX3F%Nsf5wfNDyLT58YtklnX;_ls~pAFgSj|Ex{JQ49S^y=ds`kwPfQ@p58ZI2-C
zu8H1{gt+k9a*+zueN^3{Zcr2N$EIlQH%!+<SSBv`4H;Tuc}50j%h0VS5%k_{<dK_?
zjO@No`5Q)hIfn}JHCjQZ$(fDRND|Yc(Ry(3Jl^uYMOE(lgZJ$_nqiqacbj^$yY<{z
zZx@$G;c|Nh1b4xeD>5`<V9zef+rtnn(**CU4BF&h#ep3*&TF>c@>TzjIQXGt2zg)A
z4y?Bs1c?K?H=g?UR<A{W74<fwE-~19p8?uv57x_f4XKClNMN{LIq=@E>mxiF_)sq#
zcx6>o=UoJD-hV|Xl@aE{{1$&+`?B6tcYtjoEhh~fKePXOicr4V-up04#m8eJ#V_A`
z2sj{t2|Szwz7aQ8W@Y_viz%0174Llrvs|2w5<TtCVR+=&6e9}}IdMRcOlBcMzoj{#
zKAFP3k6=+lUJjP&m57{%B&_e4R8q)7grvK}q!N~e2!ky!sU}QyVp4PN1DcaftYUv$
z;<;Vmdkbd*EtBdejB)jL)ws}Ou5nEjT{EDg^peDw16YG82vfiI7jHvRhA{+Fr6+ym
z_2k&wu;v|;V{b!ea-7o!A|5qe*x2d9oR`3LmPxM^qNpZM`Xf}w+t}%l96Z3BzV1XR
zfJBp6<hs81Zg=*Y!`ej7aHl_V@Jw>v)sxrM4c-JTr1>i9JXs8pIfh71wiV9cy$N)P
z$V!fIYpa}tMWE4|TF`QGK^`}PR?iWtoJ-&?Lb)y(%6)T$@;Y9w<<_(H9Q@9l8;Y|$
zA{SBDDIZ(ntrtZnetx-x1YBYGi<Ex|0w;A}(5@n<k_dPrx-$hK%)oG_AOsi?&J-kQ
z*I?X6beDDIpk0$u*8BiM)@Vx5uBE9Y@E(EMZeaC0LA$Otn#3fp4%+o~9yDyk(_j+o
zP@;BbL(Z0O(5_cFn`l0|9n166g4Rxlr|*rV*>X126UbyXL4+oh)7TL}5j~p#DV;^j
z<TCet_4H%{BsN7R6Z`1@eb131VPVkPRn|-<_I*J6_l*JDI}0rRr1LKZIUjQw-Zp6M
z)eQ%+1CaZIB1!};ty!g}o4}7C-`^gz4oKrc?ni6xF9!0N3qk9ki=@l;R|2_%G7r1R
zWc1Je+CiDB1g&FCRRWprR3&H~*DNC&GPl1!Sr<JXv~u07n?bJ7o4rBn8$lvc`}2Y8
z=4O(hT?V(bFJdFi<Im$iN*Qq8+UxA}9-xClC>PR<tw#zR+en{mTL-O+ByD~`sRKzs
zT>{%;gVwi>2|17o@*XAAkR4=s(_|4g=qz|I2Ri7vOt4@Ua^Pi<-AIwNK4{&Na$wYf
zK~jG|r}W?4(%&KGZFIrd)$n7b=mK-ii8KRJ-;d{iE9%{=>$T+uL|yMR@}--4ZT-5B
zR52fzOJVOfb_+9#MGC9P#3jVu;8-^D^)E^bW|mi;D2pL3Gn@m<0aju34ClZ`0$)hB
zz@L<;Pqy)^)X1P+n*GfH@9ue`A?LtOAX@^6=K)msfdiwlhtUn-*g=M3G^+N%S>QD5
zpIR@!|FKMU_(kY~7GIn6M1AS#<pWnq_?@GdNr7J0>F&_Wgwe|mwm>hNu%nj`+@ciB
z0W((We}L3*Ms+NT{}cV7QU@7m2l%*yC;CYOjE;kMp{w6tVsv$&2))2WXqKz1$0&)-
zR^0DFvw7c7^tHyZajJ3g8fS}6e?R}u2~(DKAqUIRSn1aE?@NW<6&hQ_OrHYW!{DfX
zeMpTu_0()EGs3xDm`8m;!jk7G^n<~77J441hURpIn3bJ*NVFpLw(9<%wM?rJxF7ga
zwq-$RdC@zm1kfWYOPp|YnU$r-QNn*_R3Aw}3mNn2Zb8wVNhYcxy^>a(G8-d+7x8~y
z<8CuX<~Eb-1+D2?wk0(F*Pt~!nxtu3ms)cwA-y3+nrx+hI4-Df>Ctnx=s&1ts2_~@
zCuYt@Al1>(!7V|nXW^7BX!Q%H5Hn>9S_82s(SK0iipZcfw6x?QAVj~Z^_kN~XUZ0m
z-Ke+Ig4P64Im671j6xjy5(TnBO<4SG(5ea-G<hW|I^<QwOgIy=pjEAKLKd_*S7V5%
zd{@w_NpigeCHgZ?2V}D}DBEEuVOg9SST$%Ry2*srdalutbV*trl1vnMvbHTwE6iqT
zo2mHfJGTa{PNWfiEQj@UxuS<TRbT?5f18`?dr@ktJcgS3*h&jptt-;fkr=7Fgmt1S
zs~Z-8DiHxH^+?cqLgZ(dvxOS8+Dq4>X&%M@%?Wa~7fJi-xlqvJ)i^U3D)ER@Ua5k)
zPzgC%B~^e5$ws5gbtj3;B5GU>axRksm+G6ngTKP+z^O1TMZXK-X*@sJBzRjh+Tn&X
z54t1;tS%p1Ed`WJL^$BvY9zs#2L;#aOcc=Zn^5oz&0N_~3~ta&0AFNqaHH-3o<B$l
zZn{%&oaxcgH#N&|Fgpq!(o7{S-FR6)3k5kVg7Y*5?Vtcs=P|p9{v-0GZZE6gyr49J
zR3X(NE%=^JcFH7sj~PkyA9_DAI8F;CT~KlbrB6-?P7s#M`A{5-k92SFjJ(J<L?gPJ
zSxGRBSxL!LG?zFA!L;%sw<aCmd@f8klVyj{z5pd6=FNVA<nBKQhYBsFVxWcbI<$nt
z>G4)jAMi6bHw0gkM!FX_I5U=PnXy#6E7;jp7Ns6F4eDG&H%MkF!Ix<evlRT=m(zn!
zqW42j;jyTC;@_0IsHZ5wCu^E1%E2b+;zqS(h7x?Tu2e|)DZtE7f^Bs#67B)W)kV#Q
z2laAC8sR~}08FMX9uJ_D+yjNowRHye!SN*c0R2OfZYb{0(Y_X*Kx?J;E(<o-Z6K;A
z)%F+SO6MBlnpbs*n@;{IhPdW5Tobnf+*0bHv+r|>+X*h)5T~b4Lfm2CAUG6n=wV@{
zAioE$3dK?<6qkO>P<)?D@vkJfC6?wXEa9@xNijncEmno*N(4J7T!rE(y-Xq+YSesV
zl}gL?x{?e7^3+zh!S%WSMQ4_F-@7~*@3uP^3OY)=%bYgt=Dt5YZ#+~)sYA-NJ3fw7
zbjqO$KpE5SM5o<{;N_u2)9$)X6Az&)4&7tgtv!629Xj+h$hOjMsgrh>8E)E5X6@o1
z>P&(cWfmu!L~ZrvDZk<^+>O;PlZCS&8%2lu0=;Qo+n77I9E2^m4G+p>7xF-aM>;)(
zakpj7Qh1{eN>rI&+`2iCk@H+}iwaNE5MQ{kqE_8Fkzueg4lhE~IE+!)ZBy|ANA%DF
zkeStck?y>-TiUz{SnKF0Gg=-G;6j$Vn@MP`8Wb?pU);KgIh19rLnnYEXq;Mg^VONA
zvX)$GQE`it!=9Qaaja_HOZ+t;uY|<Ff?AIfLOKrJk^p1tFr9Q*NuV1KWr5OfGpcJ1
z$X92d5cCL82<_o&BXNRiy(_4R1Rn<;lbSy#;Z#K_r31ySUD3)@y_?j$nRqF&Gc}hE
z-~!az<4HTHD}w05GXPHKd1_ta`jdu0yd?QauB8wxTdzmn1_lw3j{+Ro4}`KM1c^s&
zLboMQgFqrseIx-nuy~o*G%j)$kxEAtw<eWPo{DL7`sIEkdKP#_$Qi*N)#cX-LEzDs
zB=8;&gMn8iFqsEj%%c+}Fq?;!z%uNQefe{q_5j)N$iBQTUwzB&nSFT+sPBMpB_Q$8
zfAn_=99QZXB7dw@BLWwxn??8vT_xsQur&NwG8<!;f1;$$ie{Cbop7)O1k1|qu_u65
z5<n{->yG|E)<r^*gkA%NN`O4aJ_N=|!)8<Pm1IH-fjJUjtR929W1A$<frqbwT{X-r
zm*47FF8SpzVJbpXej{BcZ(JTb3LF-t@GTl|F<yK51N#1_QCfL?J6yk3=_$|hAzS_*
z%a|BxVzP~kTA{eayroCWXnuRzPEM{s;hjrQ1$|h+OfqN-6<y+#?a>5cS{LM$MY^hZ
z!acY)nfEn9t(=SZF2&s8)mB|E<kz5ieV~@XZhlq$gnf5W+E!r_v>iJKoZ)2um9fkn
z<^RW?w2KIvePs?Y*MK|}!nY6d9?2YOkG}F;l4$BKE@u2?bf-XhK|~WaT&TW?ehf-`
z3f~N|xNM6T+D7T<qb9myNokaBu8zihNapKsdmtj3SP*M7+$07yq+?nZ#tcV=5cTa;
zX*`+a{FWx<v__-6<4c1cQJ90Wq)Ay4Q-~cd{9>qF|1w`)UZRJ}y5iOpmc}ZtfmzoN
zz>2?sn<7!`V9@n%lsaAo{d>Hegb?)OsX#;a;;-MzS68xh8K=W$29&4L1d6y3O*CWc
z_)8Kf%0quZ-s>rSni6iHy^hl*$6u>K;1f!?F1;qdDkf|MgjDJ(85wrR8JNf4kpL1M
zI?l*Eo<`r@*h2TvwC`98aC=M7fUZ;*<}d@IQ+bdiDDQYKDRa2bbp`xE`HecXK2KAA
zJ8TA&R~{|ZJWLI9fD8#V;b9H1LMXq@B+$OmmLLQCIENAp>8N~a;}_B&7amr0Nk2_8
zIjke3AI}4>+(G%3R(Db^g>|Nwj=C^@DXguJ!Z;#i-*|((H-XSk5*nZs3RE5cM@kb}
zrjb_|kA-{?#F+olYJ}&qBd(PZaaSHw9C0Pv<%4$&tZHMZrXuoNQ)_rXht*N^IAF!C
z4m7li7hMs20BWKDPOp1{jd+L$>Y&Smj~mj?W*3r1MPdW=Y2(^o=c{v_h5u!W7q?!C
zr$AZ2t$m98cLNRJd70uARQn1A$^V~2X25!Cqg_w50UpH!1id^UYbRc$dFMWo$|x!%
z&688sy(Et`FfCk0$vGky9VP*Ha_MV41-Hp4vnpsBl-b{8(z^q<W$>K4WXPA&D72tB
zXApU6f0(b%YcH%!>EhPA^jS=GX~r@rdL0-cflMCW1;$95N=sWd8s?m*o7{ebNO39|
znbyu{DU`Et1gaIcK7`rTYVuvt&`v5kF&X$!3aVC#l!HBK*HB{C&QYMVfQ5n`Pw|4J
zyG~$%P8_YFtQ?N=pU)NnlV8;|egWmi<WO$TgCw~OoRdIfu2I3L<SGivdBUL+brd&8
z+D};bBKlG_qK5ep(nNWxHkaxakd2|A>yy9=9&q|{YjON^FZS(1M}-g1FTx$aK1!;>
zkmcf`<TjCj)Y%efA@!D*ZiD?7T)~zw`j3%o?VSfcTXW~(_^mXeDsG*%H;24!fhQ%f
zm4{Bi3ldn(LvNs`aP47=-<xYB$=Z7f^e|wE)RpIShAT2spQsA)B{2iC2wbLZ{8tEF
z)iaDlaU^rc0He@17g-#eS5rjnLP=u*t0G9w`_s;O|M2P(e-T|%<}zIxZQ`Yw7h~=i
zcU7ODTJ-a|R3cy+flkG(k4ta}q55Ogdye>A_?|mU+S3VzdJd-MZUZ(;03pde2^^8Y
zARc}Jev$wUI2i#T8}_+zJj4Q3B#_R-JwQVV%;VunptU$T=ayrd^U*cu)`hfG+2^u}
zdl~2kk#bkWrLp3<lo^(2-34!AM?s_96>$<xb+R7;xhvu%O6=t868M3KvB0|$fUPGd
z1LC8}T@fc|0it{5u85PL0*fSo2Xk@*{eN<04fGl(KriWqKz{j?%OX6nSPbpCoJ4_~
z%oV0yLK&F5BFrn~qqH6fN5-TVMheHtmrgIpaIaQG(3f(!SL!p+tOq!qN2FZcK_7=M
z<x3_>_B)kfY-9D?qIY^Rmy0+tya(W#9AnZi)9;d}y00(>EkbFRQL;LwMq$iQZO$$~
zOUyv3zTA_CkbYuB7}QvK&=wwv@FW~_GoxdF623fuA3Z})3zRGb*I<ZxYGuG34ERYL
zCB340rkiL4`Bf3ctz~7Q?-m@N{}pNe0Dg-l0K-XPGyl~I?^seLOamNj!IG*8Q=M#e
zox=WO2KkfKNruuH-h0u%tkM*>*st}}h^I;!PNxDuBz<>gq%$w$vNQD>;dZnx<g9Qr
zVCDQw?Iz$k)JNd`;?|d_-kNWVw#DT=RT(HR!{dFZsQnxv$~l!Js<(%jAbH?|JqCiO
z_Gn`MA6;((r(^a1|6kXA%{_}5V=x#ZGs7^VVHj(Jn-CJpZBWWG_Q<}J<yKl`C{qfh
zlAVePMNv^vlG5%|lCq?Tk-fz4`8wBikE!qfKaa<`=bZCi&-=X3dR^ySoeAy3`fWA(
zZAD6SjC~_0X}gVnG@tJW5^|;Bwy=UU8;|e+!x=+mUze~ggf*OS!DtKLkA&_3b}V``
zQKs8#?#FU!xF2LOZ{xcMj~g4ydNaft%}x1@r<oxX(gW4iUm+x?s9$!cl?Sr$3_Pud
zxd4hDKyvYxf4RtBw6q>lMJxhS*V$dOuCfo{Q#D@yS9X61-S6?<ArQQ&oCBBtg=T#E
zN7{h~+8QT-*_}*TaC7crad`dBY?B}M2HGwOQt-!mYOV8MuM<@+w0{Ba&`+;9L{y8h
zmuQ!%<|{0aZ(KpDlJ~ex>tH>7%K9kQU1~2cucbB}PnJK%UZM4L!QEVT@NJC!sX6r<
zV{cYwHkt3wjj_J~b4&_LR+CxULs?VOqMyk&$6iMCim~sY`_;S4cpG?<dB+Ws32|oU
z7`taGcl3AhXnPq>K;p{A*u9m=CsFI&7<)i<%W71X#IE0%>(J{qI74FcLoxQ)FwO2$
zG4?oCx;#q6&n7<_V~?+apvh>vb6X613Z&*e9b=~zDfFbLP+FR&P@9`#>~x&Iege!T
zww@7-cB9#0Xj&PxzdcpmtzGf#cVmnlTf}o0JflP1$bz)~<OH3IC6S@BczjA4KH9+B
zbs**sC>xx?Jhf^>YjZJ%uO2`bmpf>%6>|#XG?;46)5e@OROY3S821H{C}WMkjWKR?
zpCyF%*)NOxeT6YUhxMw}+vWZ<*ya`uZ2$8uNbT)%r#I6fUnZgryL7Xx*G9#B&Z`Wc
zJ;h^Qh8n9U=HqzVI8%Y2!z1e>dl;#&HpF~h*VOOOJ~|E?^EobOWryq`RgD3^VA`Y;
zI^RzvOtu(z7v2`5LtHqBQ|pkc)BDMuPW89OypP_%P+`l4m`w%|6+kfWs&nMVRbdlG
z@>HOjHk;Rv?4iiqvL)ticb^}DPY@ohMkhK89$D(9^=k!UwrYmC?s>p(6W*ksS+|_X
zJ_1ANNa~?u%xao&JX0bosIuR%7{S!eYE<E|gvyR9ucqrljiIDaV~sg!Z4a@!;eT%o
zjah*D^(Vt+F<k11OYPX2+OLdRATH{uj}U%_`qBGh7SsYO*A!yn2QkmlJ?p*L@0qE8
ze-W3o>QOorSVZfs&%Bs}YSs_rQ)BffH@B=6O=IS$nb&TFQ9~fkF*S{kATJ+8<YE5z
z-;a6JGk@(qU;=y;#`P22s(7@K;SOSRZp#lNE(Z8UqnMDio@}mEhza3v=DJ*Zln@hM
zbNSk^f|yWi<@Z$u)~FUU!K=Wzv&l8DD%2^D1<-CnZ6nC~0X{#}$}kOVLKpTR_mXZ$
z2!IFgiL?N{jr+p@!aekIbGOnBr_O&NGwQlfpO_I;($K>&PnLP>jDd)1QU9RLy&oPT
zOmPK{$o2#-ChxhHkX`orbKO{_b;(kDsJDVZ`M3MU4D_mR=*RG{Z1{ia@*iltk6!MF
ztuWh6Cbd7vdk5}^Z^-*x0uKILfcu~kDV8c?_e(K7{V>i(?NhFOd(sV2wODgx{}}gv
zIhEA~-jXaJag1^cO|EqB(jkV#*g-Mf5-e>fuf8{?e%RcU)#@sZZ~O-3sJi-w8pZx5
z{E=arYJF3MmbL#T!j0y`0WtMkApBL*c-a0C&?2n?o~A0;zlEUNaj517DM$!qb92Ys
z0X*Xvh9qry|J#(LhOqU-zri}uGIt#Bay3+h2ihkmNaxuv#3XxGzK5AytkJn?f@|eu
z`c~EA{!4^E$ir=tTe`LQV@JM96v`kI%X6^7bTU7FO(y5QoYH*@t$X%s-XDD^rbdG0
zWcH!Gm&OSXR3;=S5hBu)&`Aj<fd`%?%%+{h#Lyj-UP$T#!h1@rAo3mI8znXnq1hhz
z**wzTMhyK9oD*ojW&IRKC{dYqm|%2Zz(*6_lC5Ry$VSFt#MQ0>4o3Z{967aV_PYA_
zr)15@N?$3o`gE1%F<G_j(WqW;4-{oY(ST4}3F`Z&y9uElQnxEi%Ft{6G>!0p5{-$V
z|4;c!v?1~_K|8-?ZYOe_@T(HAI2b|ViW0*})F)&pF_FkEgqxLkm<Y9hu%|Y)%Pgim
zY|A=jR#E>4alnHklzM|mK4Fd$?-SWfSg*wAM1CZEp~Ux=^>b-LN!C5fE>qOGrl=!%
z9Lu(MUG6gTEysXA(-0V_8bg6?W2Rw+k|MuCb|#^&X69w%m`(RvjBQ5&uLHbnNnkke
z&&@Ph>ch;x)*(RVoiO*}VtQIZHnJhx4v0R&=gzG9eCTD3Br9UFm1P#1efE2~y57z|
z-kNM(uT@N6=J*RbMHJ>*ea(~$PI?5y)~cqFAuW>NN?+BtP0<Uj1|%hhs8ygedi2Ll
zZeaRZ?^jWqy%ue*2?F06`rx-w^Ya@;l@Ii1(x<s#W+gD;jS~HbOE=+8cUQ}qt;EH=
z!QxJTjSBteij%mM#p7uk&Xz(eH^P{GR!n+=<z%hFr<`9g8_nV8pc1O4@q~Mo*h7TM
z{&|_4Z7F7U&XU?r*g_%#SN-`X0=FtS5%EOM6AG2M&a!@Cbozz83?~Ag4E;iL{iUkc
zHqZg(`K4`?u%85{eiGu;Hh#hPe$nJ#ZR3|a2wl}SA{a_@>lw|rm$2uddV~E?L_7yV
zAA(+riI_uV0%44_ReOzCNlLkohm_h#jMDtV*1i)#A%6Lez&hWF_>9P3g!4*ZfkQaz
zA-P=ywmFnesICNsKh%<N6A3?sKa@kbV+x6Q<y0Uwf-qDGhM_~uK@Lq<A_My$+D+Ku
zIe)G>s}*yCx5IJ%22q~#A1-HEhs(;t{hbSW*oA@c<OVgwqZ^<ULRACqS7;5K2bdnX
zfz0La4=96hgE4Ir*Z&)6m)nQy-G_6IGwzi;+x6=PV+D$?W6*I#HHrp29uI%PF?X|<
z$NM<L{8Je4t~aPBT#sw;u29{zySFt^M{b{c8C@nApKEUx(GfigtwGVMY21ws5+?#z
zJ4_dTxDCaO80bMQpzM!35rcpXCgh6SASxPe?=_98(0d@9)nc-`O0wHxa8w*=P#w30
zWDO96BGE(!VK>u62C??%%5*1dc9FuSi3m>;CKc&R9q};fEUzsMT4p@$V$2N;7rwJ2
zQRjZbL?vYBJOYbl-`Va&))NZwxDtu%{+yjcY8ydMjLtqz<QszalFl;596m(&Nr|OI
z&JoTi@t$QJ!J9&duX^nR`6|{jSuNK72(vWQAg1<lgf8+(33Ajvj^J@e%4xUjEMvzY
zmcOXcFxLJQvo+zkneUt^suPlxVC*=;cz>j^wC#-Ka||6vXk$lAB1Ahqg8v>FC{P<B
ziwFypxShxb!a61HCh`ekrxJ`EM}8*!sKi7f7YRHa^__dzvX0V>j>fVXagHYEJ4f3)
z+MG~V3D_N_u^t`cwTQuSbe`zN+4U!B5{+7+vBra=81d+1>H}5=3a!C)4AKNHIpAgY
zMvr5amFRQy83H$^IxEri=n}$W>Xz?|*@IZ#;}hmCkm%`V=fm&`Xy>EEBixUG%^Q6o
z(vTRfBj{B;!{}`io5iS2n2}sv-%Zi>Ew>`TA|LV-D$sWz?;`BbuD{-q9fJ1mFK2=6
zHR+bqI2F#XFo8IFkZ^!n)XURH{~#Q<8uNhU++jGff!4UWIQ~XH<~(|y7SHVp@WQ4R
z4%@i5z<?ObtzT{9Ec@IR-*|$`TYSVD+}CZ$jb{mhljB;0m${9i@tUGW;hS{3Mkq{W
z5Ev%aY8rncgsG;~Y&-$ShGOuc2mf7g_ti-kjsRKgYi?MG3lBZ0fA%LP#Tdj&!Zq~-
z_bev(HHC1UpnrH!x`8HiXTGy1fdmNI;x?3%VdvwJ<cuva+vsL6Rfi^9t3HRCF`Hke
zeX;gW^8a<JMXEIPD<1x9SFcWg?L*>j@k|fb={4t2ztP?u-}R=P>28{QsS+0NZiaSs
z%iiSLCZ*S-^ft=!B-$wJ^^WtOZxm%t0O9)OIw$jh(Q=w*#phbd_m`)AAB$q9>rXb<
zSmj%}mUYMY<a>1|;IJ9VJq>OS7=t?}n=O%SE0YEKf#fb?GD1wcB#DWY%uVyj$*q+=
zSJ|y;<Ayhs?Yu=XOiN@W^RyJC&tTf`cya^v%(=wZZ%F1tZf^XG(aDX&`Sk`TH>Te}
zv8QOAy-U=+0mj@Ir_}Jhb!%Qr^$gan=BAbcyQ9&<k@}xlW#JIyh8A2|*j9p@HG}g)
z|H8HsmQB2S#KN|<EX(F223C5Tg&kv*FFUmTePIjP!TFPSv_&}t>?xlwtQ9YHh__j~
zuofL4xY#6q$HJ;Y+8RG{&>(AJRZMH;$B(>^XjP)2eAqAjV_`MuOnC!G7gjUG<TprM
zSWTWIJG|jrSgn!bVmfmBLe9@R&3I{=ukI`?FM@pQF^cufl!fv9v+_}N!3!;$*-b2~
zYK@le)!UYH)6G!C>}Z)J)9E-#XiQ4Wq)H@I<obb@Nz8J&V@+>$w!BV7mVN_YZK?Mi
z&0A-y?{BF$!(2Uf4~o7MJ#Ny)9Ai?V?{RH}f_tV!-zESb6%*k0BI7-u$G4rjT2^pj
zbh`A^3E-QaGxDOdj0}};Sc_;G?WS*$)Z^ReswE}R7%;l3F^&uvZ48(jr_HQr*M1h&
z^1;$*u5g6-vP)dSR7<?Us(??Tc{kRkr=m+4`MGR`Fg9A%wKB~fThj<EWdzb7?;{LL
z<2|8AKzmsG!2HsbqxDE6TVvna7eSd$n5M*2MCKBn(D$6;;b=ZN?k|0M`t<f}Euviv
zZ$3aeD{g|7vfllav$YaU2F0c<mqOV#+O2Ar;(m0k&}z!u&RQ(xnlH)Fi-ReTt3cTZ
zd4aGnqyPA^<jE?^+eld3mM=c_gP?CJLguS1`DA}V_?)t=K2c~jYoa%)`K4F1Qs(i-
zb@M9dzxp3=_7lG6%?oFB0VwA29>2olf`6xZTj6a7=U2iJbu7*3np&2nH;Ux6_0_8^
z2i)teJQkF|=as`s$>X^|bM6y&S^SCoUkeS7WXoxxKKmLEn^hCGxHmCnWCYTwsHScB
zUm#nwnBv-EWMy(F+%Zh|>)%vfmpUmUO_L!nnNTSsx8Eo$Wn^7(Al;JCG<B$zGTL>?
zkag@Xvbxh28$X*eMww#A1mM^hhLfhFR5au@R%bg-!;Sk=T~o%P7Mzl8ajOf^*wgfD
zFK=KGvbNH(P}5_A6w5{3Oqy|CN@y_N(BfrWHD*W3;EIUfN4(Oyl)>yp@nI$$I%-J@
zvt=<bFOH`4F<ppD(MNqqfoT~jeQHU0wTzTL^%XTOBc-2PCfZz+?^1%sk}8wSD;SbU
z9->QpFjdSfF!w~I@DN_6a9VOjsl2YEItdT4W3@2i`^{4VLM#Arpo<tNh8S=WDa6j3
zQW`~4h?etRtVX8d<o^h6q)w;6G|`ks8N_o{?wxy5>KKw;E=e7aL=<(3QPff0=8K}@
zvXoRq@wZEnT8tvK7)7cm%o)V|(AbnJ8vWHM18FW<l`3NB+N4U2FzSwWDM_@v(!GV*
z#l=iYzJ{43^B1TQme}7zk4%mE0{3Bl=Rq=LXw!K&eJt-Q@tBK*rJ)0PUtzFgDfgJY
za*i;{F}nPc$2PMQpo8EDVcsTItzit?zs^cVjjMUD3q#%l#_OnjEQMmN;zWHUoj~cU
z!g{<x-fCf;cCl6)ECFV2%Uc4VRSDfd2K|z2ko9>B1yP3zU||uU%L93gtxhWq6abzo
z0?d3iZ-xNfUBHYYz}N+OQv|rr1xzUdJT@s$Zr|!L)jzxjyATR~Z+e~vlvaxwH?qg(
zxldc9@MFXBf&#qd@(Nz#)jO|~0AIR*PDOyS74zB(aNGs7Edo5&Hm@Oo)^P%0|Aq!K
znilz4UgOe6SmQhfC#N-cTuQz4FhWLV-VFw!%qMv%LTKkAq!jVKwMkxC0kn$^Wmy-{
zniE9Ra`e#2Y5jmX*Gq!Nmhio$t?6!e<k*@ZwO)d+EImNN)OVz1?VXcjcToGy)|c*j
zmCLa+h4+bvmsuRIiHmnV+v)xsJx_94AMs?Rr5jm*n!c4|UkB)B%VQC`jv?__3TtSq
z;<y~Uj9?n~rA8U!m;$X`nPbNY)W`$HxF9MfvqH`lLAsci^mDF|MX~sOabC`O0fxAM
z^KN+otavQvga8i;5VCSk(1*Ds*S(W-NRa1Tl0!wjCN|9ZOn?o>tU0<V3!MR1FPpPd
zfKOe(P8z%m7*;*!T>*}`fOm@kceTlRQ-CWjAoOMtWa*(CPPsU3Qe5ftDR;v6aRaQJ
z=P?lqw&MfsU*?RpEmeoprbQg;TRG#4fHnH$j0gkoa(QUa-=%zWZ%&^uaIOpNBLSiB
zdDJ!7%IS|fW}eqQH)lvGEf!R~54PlVlS->yWqD)SE8fRZIhtR!dEW)xR%DR7#^<yI
z(AIJREe(LiRC{MmE8`%<o3F@eRZFFyzX;J`sME3T8ZUv{7v}JxNT+Q@mt0?lbnV&e
zj~pFzZ>uBa_(ts_z}jzeG*xS>quv0j8jmm55v~2k@I8{lCB07DX)ad9A}rkiqW)_8
z5qWq-qg4A7@Y>bC$L-4eV`v4p@)Z3G2+jGP0C#+=ty{Tw>nU)meG!a+lj>bL!a<3)
z-^BJGW2M@<GR}V^*Uva@kAZXl^i=z|1QlCZ?R3#%&Zns%`#2r5{T!+8F{$<mGg%_>
z!G=`(4-%!{C9(P8RQsexiCl9|MV!u-Hk_n%x0h3OafQ?FHq$srm}aSVQz<3YUN4aD
z30``DL{Y)La)&CV7#`bGbzg<k?n4i8mf_*qkuKlrwq061ooZ`fGL{$v^QS=PhD&`v
zAk}_aiEJ$pM}MEnMTSa==V5c)i=0Fu3!d6I)ux{%6$0Uw2nZ_|l2ta0&=AAoc_0X0
z9k`@*(->A@t|PyFNvhq-ru4UrNVV$<>Gq;3bIsq7g%sw;3h$?;+LeSdoFZ&|CDnEp
z6DoJd&~vGFlwfm_Aqzzz<Jwp^M)c{ArSjMp96jh44qL|H6f;ixV{)>Mvs2Fr?FZvz
zspkw<X*%}U_f!97JO<_NN~s4yY40|V)PuSy`&bHXV(HhZCyZeYR($Gv!g8HG^*zJL
zMcQqUiiy5EUKFF+q%Ic409W+l|0+1DjZ6M!k&IC_QlEq()P9yr^5p*^sUgM>9ZwxC
z#_L?H(Zy7G<Qgw0wU3ar9K=3-{tqihtVUi)RcF%vFBhwu!P0jz8XivVj%BG?34A-X
z9atTzx>)VP0zon(QGq5}>pn_N(Ui2qlaSYGo9do?CO+tBYAwTxc+NkmH-zI*n^SKE
zG`$Y@0gPFi+CVKTl&jX*VLDlLSEM#8r?zNLEs@gw)A}|ln}M_LOQrvC-Lb=KW~P{0
z(QT1<QTgQoA6>80F$M>V*%9!$^LP@w+XQ^(T@1alQKDABH_jN1#F!Kk@ml}n#*XDp
zihz5UcXn*<kTFWQi>!&3_3`Y0ufIgm>32-`a0jSh*%sg#vjV>EVQ{|$msNMO9P?Q9
zFu1Y92i^zY9xh+J`nBwU@79tWzUZh+C+_}(6rCy;wG0JKU5PopS1FT(%f2Jc<<@|G
zH4>8Jl%N)6i!cTwQD+nhxSF0!Ch&_xKDO=n1K_$V0>09vT;qDlDzpguIm(X#TO115
z2Qc8Rkx(r?K*ID>YJB~_4%qux_uZ<qi307b`Jcwu|7+o8JRh+42v1%jygdd_pp;(%
z_D+FhHGy^(fs#Vs1ndIALLRK32p0cnz<yewl^*D6144c3&VGhWX8o{_>JZT93Y=Tt
z_we)$dPXfk)xQkrZW-s+A3f0hMWCpJfIXTU2%TFmda%(&u=vD)eJ^C4%6g!Ci$IZQ
z1NH!c!sZxo4RkDEHxx))W-4Yw147@<Zv||3<BCL^9|CrhuorF*9Sqn_!>JMrNXS8Q
zrTnPF0lS9ytxqsMY1c3dTbNJT0;G=cNcdNfXc#8Cd2GP0D55>Fxg&@3dc$>wx%hRb
zl1m75EOyMuiGVZ#=Htf}D*>zN7XjN}GMP~~A#e$#+hXGk{ogJ*fa#3_e+p19-cbKZ
zt7K2cApNQ3X0CdVX9ouk3aNXXQTZ1`9c48B#obc@sp2Pr4~6tV+?bJkyfv`XV7c->
zDv~!LF|b8gFS+uz)VL;3A3}2F<!uPOBBT%Fjl5S<1wdYjBZ1ej4Gxmb9ofADi-n|b
zMq|3gWVzH`{tkp@0qGnU&x=igS+0owiry$3@GFiC%*qPm*S;K>D)dZOrK#xxxF(od
z&mhgglxu1S#+6jWS0^ysM8*aJgJj##y@A_ApXX|O8xvpbkGeG2E=FCm5f<pGctPp7
z;bZBFda3oZbukx?FHa4$S27!B6Rr!?6ti8fGBsfqR;DJ7$Hyjgi%r0E$N8GP_Bs$q
zi4_2%qb{(T+5a`rWod!yJZKlkg;sGbD8VJ33Y0fcCANPbC~Xq-9NXVaGi#DAnq4kE
zK;r+7-&Omj**DO;bQ$X5+;DAx;@OI3H&y$}Lwh}vW+w|V-$P6`h%}m*9cgx|tB&yQ
zPqRx2Z>NV>syJS0g9p*luhZ;uMgG)lMA{$FbS*^=ivHmaZ1jKdK4nOZ@zc&IUO=Tl
ze%?@*X0I;ovR4fl@})y*KVb&#j&yD7a{I~5huvv`nNGiS$W&{ZISXtyC!SuA#&?1t
z=>ud;O4>I@W=Nykf>LSQ3`2pRfn%0GZJQ=*n(GbjoVJbL*mxsk+iEGUO@a=0rmYsi
zCRc^k|1o?r&y=FmXxmO#IX*>Ra?R|n)hn?625srvX<GQ)?sL)RyOuGAS!k5BK&rJq
zZK{x3xkyusA*t*3KIoqIkm8=}rakQ04DuOm)B1~ix=Y?aOdi7eW-xV+6{ig@PH&#e
z;QG)<k4@7u|8{o;nARyw4|!)JzBE>SBR4I_OyDi~yZ@QA7G5>o{+lbd#eb{GA9}{E
zr#6O(-m+^vC9Mwoyt}khHpOPIv0E*g7M8E?qFBdfGrT;?TMON488jG_X_?j_-g55H
z-8kLu0KAN_L<#gbRzO&*L^C2=32!scF4HoFoO?q0)sfX*Jcj;&Lm0y%N>4BBt9?fG
zC}AnddMa4v_OvWX(GgH1WjDNbg{6E;;_I-KUq~F3l-rGzA|cg1A^y_S3${-~KnMwI
z8X&%nk#KifGd8xnmCGVmFCnc7!01nct`seEH?TE5E$cT3k6ocxfG$w%h(lFuhCA<h
zIE|+hxP%R>iC7+YE|2AHod(C9%WGFqohwESm8a3@!#D%SSZUXjr#SCyrlpv;E*9vQ
z6zS8HwA<nYeJ6*6R&lwdwake0-R7?H^j2}d@{#t(YGT1UR8{0Xq*#NPwbHupBW-u;
zPQNf`K%{pGjc#DOFE!Ga#%tBVI)K)F73p5%qxs_QHjzHJ`(zpwEJe<%w?*2YxlUc0
zWXrE3eRWNeM6H35zIvX+m$IxI=b=%BcOq@R84tByB9`1O(mmU#Zf`@o5KANwH~$oA
z^KEeuAM_B{6+>JPqKmh1YNWl$;OSCCF<hj6Lib1XY7uF_OeSyi$}sjSm*_6;7H$=3
z=jqXf<`3qj+3wATxYT(eSt#2cmd<6GD?n4#<Ws{eZ+|NZ(zLJ1lqcreLr5-MOR|M<
zY;9I!E_;rV33?qub!B4fC!$3{MrT4hX3Ey?n<8yq1QU0syIMT<vLC2O|H}k?DEdwj
zpKPrzI*f@tuQGOLe~^AF@;pq7F9~}<@<8NaN>uU$G)>e<a~jb|Ly3_`!-B^lcocah
zlOlf&=RHf_T=G84j6CM$g?{^p%n!^*>mv09Sn2xPQNrPBmeXSeR*N%_eN?k?ktaCd
z<n;Ie@SlW2B{o^s?=;Tett}j@InH5K2za)_FMpBR{%ybt^smSRDpJnfZjla{{tMUF
z{)-s_WB(FfvpXYo?Y)zuW<$~U89YHg+7`LjKr|)*+3P}*d02K!<VPY|;{rY^0=#uP
z@?8NwZ~^aL100Fe_HxdCcmFlg-CvfJoDPv3mjk4c4=DxLXr75&3ZO?#7qHX-==%EI
z5xJbK_}|P+;F0D9{cN$?JT~$vvFhQD7?DpEvFi3rWS#(bLKi|vo(mDpjgd19jh1OZ
zW)#tEPKnfBa*sJKV9GT>xybPXtZ)J2T>!n%+Paa$)eC70q{m*M9ZN(G$F0qiPP86=
zy>@p(Qg!XFNS^OlPEU1Eh;3|`5FeHq**MK3*;OjCvHGfP3VrANNOPr%9*%y{JW^)}
z%!5*T3@v|59RC(OGryy4_7CUph>g^=op#WbRvnrgnIbK7k-ijkIHX8PafYf|F{<j}
zIMQo`<4CU=jt}1%>CN_NL{EGd$s;+qZi$;@MV2=>TIxM=D(FNi|I2%{#ise!80B~9
z8uT3@zgG&VOAnCvcdMg^d1PJC=e8Sleov$XeTC|za+P}h#-Q&^3S-2J<iM`$D?uM0
zH*<Q)9%A>SVTS>Y!;cY$rhK85<Ac6;gt){*d?!_{orJ`_Q(FQ&st;&)E63lhlM_MT
zcL<Fh7WC=D_A)O@1*LW(MJm8~!M>nxl^FleW4!9Wga3kkrTfwUtlS>-tw97YD~PUR
za?tlQba&bw-P1;M5)$w<hzJ0$Zcfl_I@~FntArz1^c%&6H-ZI(i+A0epifUb?$pz9
z;i=;+!V3*H480Ok71nig4EOk;?=De2>QUWQoa(M3EgIb(^y$IKom)J-pz&Q2(ju5C
zEyB##SyrRl#k|dlps%xt3N`<iPMwQWb^b3^n-ij1I5_BQ57k|1o~ZW4iQ4~{XyITH
zy?-j`Yb~NXJtFtnhNoZaOyvJd!S_!Ys{KKq+U;FB3?u!ri;K$suc-I;i>QA4pieuI
zcYWv)>E^Z~L)SGX3!4Hv*KaSPP}b(4uO?Kz%Xw5ai&ND!RAzrQ=*|VSX&!drtj+Sf
zx+?&&Gq0^GIM#~@`m~7Y-P~iCT%2LDVJN&C=LPL696oh=FK`j<E5#7)D+aNcup8$g
z>&3#LFA{V!;tyLM^hKF2Mk9XESGtUuCcwMy??GG7SbFdASf41)`b06-AvJ&SrBP2j
zNnOC*K9A~1F{|50{%iG~J`t5a7PR+^Nbh)1RrdbkMEebqtGIojnBw+NM)A7Te|Q8e
ze@y&ltP0w@#c#34Z+CHiyNmNv;lhTnKXdtAV>e@!VYevA3=DQ!;z_+N#o29f{m;|y
zo#Oo7{XhH`iC^6>gZAt2yGLJ!5x>`qTlV#$+O7L>&|V`v9mx{jn&NosZrt7>)SxhE
zYhb-cD<2{1vh}dn+DnTNGhYtc3x&7H!&_J!Z(*@2H8q;~vb0UF8noGoaPIlSBbrm3
zXbwc7YpoAINUtiYw1}WR5vo47cvQOJrbwlU{}q)M!A9p;P9!~q|5Zzfy>ECfi5bu`
z6}@lx5G63WNeFF5Et44DzaNQ^`Zso-5qtfGpgl@rH4~I2bn9c0*iru#d;J9w)jS?#
zsRYsM9?{U^L__~eRP(qY`Z;K;bM0$+{hwXDceUN#HFm(rQ7EXlEmA+mLmEFDw9UX>
zya%ZHv-l;~2-@m|`!@8H2o_f&_+KSBLn6&P1nt|@{`(H|sBSAxb=!ZbYIYD+QbEwx
zVyN#+9#OmEMD6}dlw>TMf_`No>Zd7$EL*mSC^U?eq4FxReDO+@lNVKDh2oW1p_U++
zWlT!YuJo_Af*+;SP{+s&!LF}qFbjcXG9!Z*q{I%eQR0HpAw+_a>cW2&Pl;p)5pVh1
zs1s*=>ax*>I4*VBC_@~a%SIO_Oz5J=x@OANa#irEiK9sU-qemsc$FGeRH-3qwWRoF
zOU2~N!9uC{JIJV5XjCMQhJ{8$AyvLJ_!~(5-HoK+Z^e*)D?&<&3;rUcESHmR6g{RT
zEu4NSLQ4BC82U<BS_~uat72GR87y|5f_kch38mhipl160b>0c|kBXsx<f0p&-DOIL
zJ#bmKj|3PVvy%6S(}>KVI-T57vOlbrR=uX*QZ|#zyhwh>62YZ6BRfCNu+>4_)Q@31
z_4?on-j>ZR<+8O3{}w}6-dnCMOED77M^5=KgOeoZ6IaZn%Gbn9sxL8;kDN(?YjPe4
z%b_}_0S;HEyUm1{TMh@O8&@Ds)jnwAo-W7kkx<unW)<@t-aAu4jPGc@VdD5sD6f_h
zuJX?bDO-C-r4|S8wN1jwJsPi$;NE1*9z0mEyG{gYf%J5iGoT9kQKf2xWnN?j1jqJ9
zYH2w7-wAfU=KiHW4%WvM13J5=sE-nO1mj<e{lWU-MMj5h!TL?ji(kpS^;aRNnaO}W
zm!f96XS{N|f;BOod4Cgvn(ak^yt1u=wLo=!<&}y-o|riUK6Z&Kn#nJ5h%0IzKreqn
z9IDvt8nySVKzrANsy$anLx(k}(sR}9Am?_=XW4oLm-=v_^*L)~>!Oz3gPe4}4wDIp
zS-U3QzcUf~J$}3#Cd;m3YSPJ{nP4ua&9XJE*8=-@%=mi?EdNACAj__SD&M}!v3;!g
zT49#mlFen^QuF%&f=;J@-q(GxdEmHb@8ul%=ff@E4-Z(*z{lAx6L-CQ_xU!<8OjTJ
z19dE<D04rAhsk6UhI=1k*jCe}5LL}FhP@FU1?%b}ioKOBhc|wmfq#-|clHP>v**HH
zGxfCNwKv2I^ZDL&Oy&*68;55-f@SpKm))~1%Q0^#4s~i8%RGY18^cVbFeC0YhM8Gp
z7n<FRg;|e>&9V}lrDiW8OEX_%s-5J$T?bPIvgR2>>AsFsjh<P{gi`?g9N{#)=1t7n
z297%1<E05P<c)hh>wV$nlWt6CKwjRCtdGNacaYb<d!h9iI4nt>L4SiVCMD}z?W9sy
z$EOn}QE<LNgN4m~d9t}~BmV4Da4^T<>#L7o3Yulda%dvG1q90oOSFBNUj?h6BVINj
z>v7u5;3wHN$!B?VoZREu7tP1xk8dStUo>$5n~eEwN$n!+=s=<?96q(MaN-nyW=Vat
zDStSy{RD2HaPs>S`IT@4Asp?#vaitkbiX!j__eI|SsZzBhTOuHF<1Ts>kQ$v5}-3j
zwvJ!*lekJEj!<H<<#QGoQZ{i61FMYxmJ)c_r`+dm{lg>sA&z__E9^`k%Ki3G@6YT(
zjt;{aA3G>#*^_Mfc(DWYhT76?fg8C%^PPH5Dfymg@03uLdHb-RjWPwz%leZSWryTZ
zyfvLrt1_XI5&<R-Zc4Vw^EPBp)B=vbp~&t+x2+6WW7-NYMwzxU<UN!KHw!KUGHqqZ
z*Tpk?yKQBNIXzIktx%3nY3r(cmxVUzA~t3X)BeKHI#I^Y$C`CG)vAHUy&$zqvZYhl
z<^Ia*Wts=b?bVlN^{O83OFO)gb$uB2yMbNL_1uF8!*fHGtYpK}j6PW@*G7e`8uDH3
z!!)Rzl}^0$yKK-7eckLpWn~z@1vs;MR%VDgr8NlrxeIPt41BW-E$u>hH0N@Ku&WE<
zQgEwfGrx~%DIj*cacee|FvhT8ZAt>fsEhTp35`rQ=l4x8`jL?n;zyIg>0QZ<7cm+A
z2z@Z>uFc|uQGH{9AZp20zc1j5vLD^buHkfvn)t3ODv)BVW}SEz`ca%an)q&^{n$+T
zOPQg5Uk}>Wgx2`T#J!gF2OaDmT$~v4t(sbB&mCem_m}&9Ss;zj38qzZK%h(bLw8P<
z*}_FYs}?x<e_9-ISw$iLc!}`55;W33UMJ{;)hcxQL*Z>DJ|?n<uv>|*i5w=d`tz+i
zg~+)^Wa!(>57-VJ!M3Sy^-Y#_l8M2|SS1**Po@y6Dp8FHBihM&O0bP`lJ<X6YObap
zPO{^3l5>>4)mZ7|NWx(0{4HWv-$H5u;aLi>x|@@KkSkz(s~@L_<QUZIXV~5^!>iZB
z_XroWoup_dmk6{34wr#c5M(L2>j<w!6x#VyEa$ZymUHiKUd1SHrk7{#exkBUkhhAL
z`@aB3?Ri=vt=rV1?v*+N^kY>^+2~N_I@<B@ThSgN*2%XC9Ne<=xsYh-osuw~>@-W4
z_SpGB%V&O>Q)Irxv)$oaq0^l`CqE+Ws7B&TgCL#o7wg)UJxWIw<%X~9u|qBz-GuJ!
zIr$|4?{Et+M1_(Hj7;<#_S|{s-Vdq2km;28y{|I!t6RPw-yoA6d7Za1MQ`sF9WzsI
zXY4kyoK$GfoQL=QMs-&g2W~L1LX>sxBv}0DlFLlyIF=oNgu!Cx6NUDy1Y@jneqWm0
z^8$t5xfo3k5p<hG*;FcN=c|@=idKEf=31W5W<xcbE?&QExs>!}i*hfNpS=K1GU0lN
zJ`3S9r`SV0RZFQih%f}6YN*6^B5eq*E5U_kwzi#$Fl#!ZcERgNxCoSB);91zQwhc)
zO=|DhOD=VYa`pW_omd{JbK*OGf#L0h+oX1@Dzv{HC*_q0+`%mQ)Gz`AoNp&x{;5X^
z4=F)6cj|e<LM50;omxd$rUb3z)CUAL(4E=XTbUimI72uxgC`C<a|-RexyokdTUdjT
zjDfzyk@ALEcF6qWJ=2oDQDrPXO@@-X2Dy+Pypa3Zd|#qyArILK+f&(j!;Kl7`+ibc
z@l48|Hx8tZN>8BZHzQ{nnKOw;Q|JmQWT=w!W{cFI;rQ^Hbwahy6l-MT`P;`ul`oNp
zk>=GR-F^Pg<DwEnJXPXY4e9%!vYW{>kSHy(E#IfB0n8LTbLN7A8%QT0-GMM2Gx`X7
zfaSdUCwW`lJUJu=XP`9)HSD|vo)}%EWjFD}lqd|@d3+-N6LV9)IY#Inl^d%H#(<kF
zSZflYU(5c~@5?~{G4!v#O$nH*JE5B@xf`a}^OM4Ugg#2tBr<|<uM!Q2&}s@NQ-PaW
zdakhPRyJM==aM~JI6a6gC%mjgE|GT#s_{)Do&1MvMJ7A>kC@0~>ZFzPw3Gj^$$823
zicPOTw43meykgU)LVM1tG@3nq@oX+<HNR{h)E`M-Fp4G?a<l>{JW9}4@77(mtkV?Z
zbOoBj?6a2hdQc}xH_=;^W%fN<r)@WV0sKZnZ6$sr!svdQqtm`kzY(F&KHbXe|K?0L
zO?qT~7HZCD&Pkc>?=&Sn-5qb4^RRWh*Z;|@sKz;+T-O_IPJh!QDX*uOXHI`lGVVBs
z>URB1HKqDLD?dZXrunTVMwPp1HWMCoSfh8#A27WbF$BrgMjs>3KhBDF&nS)68FPD1
zk_#LYJwJaUembYRmrkGmI_c1yCQ3sXG0tkACjS{rt5hejAqDDHrW2U#_tnEc#<!%T
z8!{>7U_w9n$CH?6LpG@yga?&qPb8l(M+tiI)2|U;p-wmSb*pm2a58B1r?-F++Cafi
z|4#Tt_zzpwpDgA6<XpdR!xKdABIsc1hUbVB5Y|#{8#W_(h2@OXQRoeCk;?$`=Uaj>
znv~rPL>=OWo#b92u$$uB@J*rp1RW3X9Sx?Hww$q7D9lEE9YU|foQYD={vwh{D3e9v
zq;lx-^xWr66-@cWtFGzBs!12Kv#Eq9kg7@;`vkz~aaPRP>M@s%J((nl_(KoA)`Oog
z@a7nNx{L5N2w%W}OI?Qg9(3~#dU^XH^7@L0jH5h(53hY>{)6{NRSrF|$n}YFWrT%K
z;DS#^2(EPWT=B!@G!AR<eu@Tfel07m0dH74<N8w4r68Q4IiJad`l$$S{I(mqdh~`H
z7Zr>fCyvyZ3N+ly>t;H#DJJttyu=_D>%yL_?B((3)pTnTkVT%<<1K4XJNYUDk(_<p
zk@+JZU?n_fzpzMO5|S%+&M`M{yiVkK2d-&kQb@AF9$!<~<RRyYI-VR~bnCDn>sHIR
zwlOBEr+$N9%WmTL)la|(Yj42-EeXx#+o5mp+O>DMUb~j@=ge({4tUmD2D&qS3HK<0
zs5AKVnPEy$J!c*uOi%(3J~Njvo8`~b7ZBFlG^bx`Gu?D1=qlQIl>uY>GcPNVK%{`6
zW#haE9Oukd!rQ7(dMWo>NOSjtcksE1YxIC?wh6Ul68F%!4o(lT1G2%~AeyWUTWg4!
z$EktaKeS>_gQru}lYZYVm~X-;R*pY(L+0m%(2n6C93-bNV}^ZT0jU%69e(+WvESDL
zl8N&nIY7fYqsF6Kv&w44d0>NOouzG^jZoq}BDAZs2};n5pS^)lLy7N*(0!b3#uceY
zs}|aiC#h#mj7vPM&zDYQUThzwlxLX~oaJbyeY6e{I)$^ns#(UQ5IBa6SwLbdzh~03
zgFqSJne^;f5~GbtiD1>UQ!&fqbcC)&uif8bZWxd+JsyWiL%8BZ^U)e(Q)m1(V^kfD
zw;vyX@@6UNo|>#DbJH`Ld7{NtqMWLn_$o#>^Td`+7~pvHwQz>+u)F)L-<M2b%_IDY
zza^LjS8_G2f=a(k{*ur-zpp7SK_}XAy5Gkufo7FfavT56)n=2*_A*4JYKVyW=orho
zX{z78TxgwysD#<Wnq7y!{Hui~i>gk#(QjYD`zGD~AOq=FE68chEa$6N<PyHk&w)oz
zMs8<+$fhQp`^PGO6ZvM+pERIkL`7w*`%ltjZ!>-F*;#~{H1yT=3?iEgpwq#WbgocA
zlitSqt6PEd4B=_bvo;?A<t!+4RFnQEHVgu1R}z*|0oKVv%c;3z7qGwNx$ZP+tkuf0
zS*Dw7$WI~d2-_0(*PXv?T2!_8E&93z%g$3xlt2HQX;G;mlc$crI=NM|f0@4o9i+Vf
z^Q+)?GDRKb_c4WarvKR6viC-t>^KTHD$)10U!@#{r|vC}YVQ)>ZcL&Yxz8Z`6G76+
z{hF{>Z1?smv<eI4#7WCSiN3Y;uhZYdarZK0o~7NKJtmIJL^sZ|IM898{YQxhNg(1J
zx0%>`={L?H;#^fFo+r|nz_i}pOG7)?p3qu}H;4=%+=JZzH7T@CKWyagO7tzFWlk;(
zA@_ehK$u2&K#7h-o+fBnoU{ub{}a8KF2m@jXL5A|_liBRKsLYMT;_UunPo0svQB?!
znF;O#+XXbekh|6qqKNld&hj51{z335G?k$t{u$5-4|Ks)fGwE$`V(Y-QiUR$to~e?
z#bMd5$jJ@Fa=JjCNSE)QRG;SDlRFndirDrB6cTLc|Any$b!Z5n$HmrlEz@7RLyYw&
zW;F0s|ABMcfHYq0f%p6uoTW?u8K&^JKR1FD8uDL^Y!AbD;NI`Ra~s=li9ViqInz{Q
zd-3XXTL^Edl73S~W2AF?2s@OxO5_j1acX_vSFre6wO%nM(Km=%pH>$Z`^4fh+XLs$
z!=qvhWc%Q8K8e85U3(uq&etbo(69fmGCT_D*Y!)2O|&0Dl_{&_FZ9#rakumBxT53F
zi)1{=NZ_n2Uud1FEL-qvDDqi-O*u;=U)cccZiL%OgjUuJr!dJGw#D3QVT^OgH)t>}
zvpS{-wG!>8n3+v&5oXj6@;*w4k4cD66OT`EZz)oX7d=5?<x4?A_S0fFJIrn*Tt<Z1
znS|JN`u}35dd8tw#X-MVwdgwepLnuvgPp1!cjP}T=bCJs{0yiUt^hLrZ=5w`n#mBa
zC7oet``?2y8k<`3uc;RqG(4X>v&hYx#y!hJTt7SWs%0O3lp4KWS?%RjXBdGF;Z#bR
z*7+wHhUcGj&R?Lxv`$iXtgBIIuX|new7$xu6OeEgt{KBYG*h|hhRd1DmhVO?4QDuV
zj+UG*XW*AbaH4N6R}@b>iZdK7gZbz@&ren&lE~wP$EsQ8)}7M~%n<QwMI_!Dhm~JB
z!ccquS;A9vuDWsO{5k^50m=CQ<SawNujJ_FywIJnK5`7Kw+Wk-*iVE3=KKdr>?Oh~
z?fmBmeKiCJrUvdWLc414*LPLVi{^V0A$?f&{I3MsL%49+a*)&6uF$9P+t7BEM}<2}
zV4uSj?g9wsuOLTP3Y`uZIs9HLoMq{wh8JjCrfe6INnC$TOnYn?vQDd>C>-~oJJZuW
zITvb!S}QDvuKhy8u$)dt4!{4BgEwCYgyq~#f(Z}}vXH)G?rG|@@I!Qd3kl}72KQE=
zXl~}gLWQg<<iykO!3FF#@_<5S(DLEwddS1Kfs7ARRe(-280*@H>0E2dbxYYMj|RVJ
zAbFa+c(Y51FcV39@-)r*H>TX~ULRi|wD&eb$O+02ico)b6ABL_3?fIh@W3=i*oVcM
zU##QzIh5_eIQD-Z$kWdP%64j|<!kn$&?n}Rn$7N?ehoTYod<6MbK(O1EEoD*gU(E4
zV_8!3EZnnV!;s(S2Y1GVvPv<>*qDqSjI<Y~HYQPr+}mIk5`?+i1tupKp5{@7Z)2ZA
z>uj>=lX}(vnci?lFIEQ|hXY?mSR#JCL`7$RfeHJCwMtN$3(OEMX#05MTq2>b`1xG?
zo`+vgraBzBslOL~51kReZGdt13x}21NaQl%q7tht>tX^SPLf8l4B7Z8%Mdud`KI9%
zMfM~05#&5X@8)~+47jO;WF>I;iy4GEj3&M}e<0SB&{&DjiL@bTqvy>Jh;%31MlGHl
zV0p7)_uBc%riNL_Sg$Zm{9g_j=wwx{{|+Mev1s_;c>O%2pCQa2|EG<B|1E+Wz2U{N
z;<%aZ!HxfuV@xoYN99MGk@4(o5Z5T%NMe84I#v(X_`_LT=~y|eT~c6oiZR87(C7bF
z6G(~+G1%>0$DGkhdH18R_Ne8U<6J>O9;LZ!VE2~Dafyg&fh}7gN^u;?xAA85eV#C%
z4E(16=RNhI7DW}G_xmET@FN3M33^&kAl&r?Z9)`uA$Kh*sk#e-<nAPVDEBFN2my!L
zo&@9V2B)d-M{dF31y6$T4dE*>G-ML|GvO!kSV`m*;kXhTEb9`reDSLIy<2FVW4wiG
zcY}kld-xZzI|NEPp}G>^5usDN)JzGMSeI_0{atD+NxSi%f>W;ltY^*vD>zsDxxsEq
z_wYv$Y`_|q?jUqkf_l2tm(ZKUdR+3-IKmhu3W-ptOH-9NOaw2#G?U6cm;P_}QEd$I
z&F?!aDk^j?_<t(=H#9X>c+9Qvhc92NaC}a#aM`W!M-r}8_&NG9^+b=b*wQY<e=Gc4
zl2_mQSP}Ow#v!ByPFT<ne)}X*hNT<2L>*pQO9u75nnIn@qfe{iWxp>9i_8qkXqUlV
z{T_I`2s^0n)t}Po)<^m(K^TQD{YdyhJk~IxoTDfpw5V<Cx}ll<vCtY?@THTo*Y9ww
z)B?ISRV?c-KfzXl+0|bR8Gj`y!8q_&TS7}QY+GoZr)fboqP9Je8h`AB7!ClQOW*?`
zzBN6F(3kx+S<FX>#b{E?3EF61!*Kr>ZU3(jx4`<=<XP6=u>_lvtXTlTB6w2&zBLQU
zZ9vGZ%=4Ewkb6)AB)`m{YTMnJKt1|1;+E|ICqQUXjl`D*gL*P}*PhcTO;7FRn&&j3
zbqw`x2F6|YAuMhq+zNlK1J`f{;R0@CRY##5DMT~)`%Z#MP*Z>75r0oWJAH<M{+GFl
zdSNTNw#|zv&SPu~n9nf$4VS;4z!@*FX=%Pzpj|XQqp>c`3qK4RPma{y2MrH{9=jJ;
za;6>(YCOHb==a(Vpcm@7j`o<@*`h;JUQe8h?YYOy&en7{Z<3ey23&03u{V4ClJ}ti
zZl=CJI6fJRBNZO6?Erp$t^~-7-JWSIT1g0#FK#44$W?eT$CIYfZWEbl_Pc0_%41km
zZ#dHoyI!8B^GrR=&_lcGNv09#`#<^`ucnU|zQ(+4FNPOAGKM<D)juJciZKmj^<m8T
zBH=kQdRxBbH!+$s2cQH$`h7845ECK4JveU=UdN@DcQ*xeX1*tgYL8~~_eW|Rq2&zV
z3}$K_S2JxMP&>2Zsj}sAyzdF$isMz$tsr%p@P`r`EbDR!f*S4eZA6+8vgpc|?{Rqk
zs9dNo_axlGGUC;(NExLjb6rxVT?tove7NR=-_iaqPb7?F%kfo4TIbbINEHy)Vv>tf
z+?q1iv`XC-Nn~Ot_|(a?Iceq0dJ1h<G>70F!dpsY68VC#M~O5dM+iStj_dd!pl?M7
zx368*K&(QjYF*bR^E@iestv)aQ6OI+oKs?eW&ML~|FM;zF#p68m_GYfbs|!cP(g_R
z5xR?iQl&Gy!tTyh@lV7|rMkjnwT1<&a^@u@P7KkH{QnKs|8#|Adstp-g4#LqjDL;Q
zmw4LnpRxK6s-0<>Z~0LaVdK&K(p6;IrBy(6Q!D6b{<)7p1^AYSt`fVSFj1*9L>?jN
z8{iiiz)dT00v&mz>Q+IR^#HZ=i&#gj%fmXaeVFxA(9ab!aejgDe3<iU5-Y`-1&;gl
zI<9qFrh;MSw4IBIZhOk>r_#EcsR1>ri%dS$sPe|U0Fy^s)QFS!%>1hPJS@2w6kpQX
z#J8Iowpj*|?O6l(Sj6rOe1d7K7isFR?SN#zAZ}Z|Sm0rw6B|Y|Z@XI42Ul-*I^&E}
zqf`rBK7=}|j4PWax_kuX+)UUgFk@+aYrlPz21eYO-7{g}fB_a6l_<F0`HLP#qAqU)
zyr>%;8lCamD?Hpe6FuA&3_}KYLlvM6gp1kU4EpU?lPur%gCy2Qvmv<lIy&kUQlE$X
z_BtjC=J;V&55N0)?flrGV@4zT^&&_Yzr7w6bZU=l*8KJcC3K!I@=?FNF^tx?&~Lx#
zqKzFt0<`bKkj_B97ersN4kcV7Raq%!iF4|cm2m?qQgeex>n)f$s_f5gzr9&wzj-lc
zPS1Wi0|QG`=M0Q{q`_|=Oj3zUdqAZCS~Ad%>V6I(N(c;>Cb)VpDnmvnA5EmVJ_rGO
zklU2>+Z^REmVe?U&zjk(#VnuQSk{DaWdpz696442iJJG2s4gxfqI;6S=mrvcg2c^A
za2dy7Y|#=7^Dgq#(Di;hz$E91mni1riGI740mY8XYye*CFy2=i{B}nra%1lsl?&un
z7wx`LLr8QAC&rSv%}wZaJHKsC8t;Sk2ZQ}~7Xu+tYqj5gpfu(=pijI+_3_(}n3PqD
zRP~Mw;+cMP>DCkPL9$2*ijS`WT14%X;Vr5sRD}YS!OvV!aKGQS0X<pD1K9?In@APz
z`tarJ4Ad>h{kC7K=+IrrddTt9W}xbdgH*WL&s5fT!wTYKQvBy?TD&Sn96{$=NY0Jn
z_kuMZcBuyk>-Km*O%w*dyNDm-73OVV1btjXa?H!v{ymcEzEJD`EF2GEvd_JQ469=Y
zLXus-49CIw%EXyt-(J@LO`?S1n@|2gM%@*Dox&`-1jc{LP05(i*P-8MfBQ#Dzqx%}
z{YF-keo*4oqxCkid0qJajs9_A871TW<4rtn?5J^;-@Kfa55dYgekP7wxJZ0-AO9p%
zX{AXF&-OpY&IRsmuAbTC?<)VzmC&zTp<Povp?FfTn9PvQc$v3}5!$Di(7xe(g!T&y
zT^Q@{Z{iY)`aRq>DWT7#`a76LBcZ2Vp&e)qxCSFqx03!wjG5}p@~g!4A5XubBZ!=4
z^eWW-;!c-rA!pp|uM<X=Q!K0EuTxg#ak<v1Bd&2nEq`6<Vn_;__|uS%FkW<ZNz!Bx
zk&J)Uug?|JxPz9pYiES{%8*_PE=f9M4c8lM`G;#uDPUmgP2PW)3;eUYx~{;92)}M`
z{d0`Wqt$iQY=j-pkZ3T^k$IXCsw4x(GiC#MEYgN@!mvez+5=eF<f)PgW1+GW)$9^-
z6~#|=n`}1@AeV~Z<~b1IqQ1}^5Ppifw}u>#IFBR2Vpzu5F_4{45d+J*Lh-KnGmL`g
zF`{*ubZbJBRJ6P3DR*TWITIwJ$I6Jqh$!8=(5l1m!Z{I&{nRg^46qL)L7yhMvXQVX
z13eGZa`gd|t5on+%aw9C-Ia2cdH&Uw6u1sU=Y=bbcX}Ykh{r#6=Dja7Wb*0v*CIZl
z<WHB1N5&@!k`WK@PpXP{ywUuVkQ!)#6pZ{bVx_0xuIXf&>usN&(q}-X6|s^T6xUb|
z>oZmfqnka{)oNuePpdq%nxV4u0X9=HSDD*geIpf#+rl}8<Q%h_Th>z(BA#b@<~$u*
zi-+7C8}U4C$$Wfg&}|XV(?G)tK3~tUr!-9lM9fraE<jNCod~^8Y;L=F_p^wZRXpO`
zE=J6hm9#I?e|yBto31gM*}*VE_D7FLOpAomyjt<2#)ZaM5z}}`glTl&Z^SeTXT0|D
z#E5A$E2<USTsJ*}<4qtHT5;5~Wj@n0@4h~iZDQCo_{PL?z_PInPCF4RLu^}B3D@cN
zjn~>n43bz~%|`(SY52=m|3e7|fsG3A7$l+jZtV=pGCZJdH8rA#Xb-!zJ&F~+hbg>B
zd(;Fy*Cp)%k*oXYQV|`|T`!}?Y3^fu`<b*1Rb|IS#vi=d=FJ`3s+%#kK6!K+CTkd3
z{Z;xl>m%Ip7!I?GI7~7evT08^s%^bMc*ZcuhJnf2OV-D-h0Z_87Oc>1-iL0&onf6K
zGK^B}Cq1)=LIqYvWSBz5ORo&$0c1xuiO8r8aelm1%V=Th6OI|CHcNBYVnn9;8Cn#I
zPRxj?Ca#xJl(uUnsf|4wscNc*o{0zSVX336h?=F{653hpC-H<6%9Qsbs)#bT(CRXi
z|LKNuejn2*RY5W1{iexOQD0VI5A6W!`39YZP1KNxs;n@aXID~6_NnbQO1wwp9s*nN
z&a)ecj3Maj$5BI=eLuUKRA?4Iywc}9`wfvbgcY#wYFVFr9KkTJUekG2mm<CMbwnBC
zF~lD|9#JO6IE>M#OolX~LyjmHV`7@{MU)2-YPzI%o^;HK-}#w<mmVOIZn?+g%Iyk~
zsxKly9k!8ZFWWFnDBH^M<ro$uTI_dxJq&O0>-(YO!=;@0ysBmRbxbYd2ZZ6Fh+Rj}
zi`aE4&dyAe3e0V$9lmgUEyeE*k6%lw-yGwU+?MKGvpw}%wlY-**Z1Z)zQ!W@-6LvD
zJ18P*tfmkqYD_nPM~~rJt;&v%r|Bcl0yS-TL8ue9keGTn2ySHC+X!!yDQnc@zuJPm
zpRliGp*1SOa(5Xt{VBuqo?@NuFs4k*LeDwA^g`?Itwkc%dJ@vZ63)TwlqVsS?uqb~
zwrpQXMBKfvxCowZ{#!&x>V1RbOQz@xzNP5WJy~<-aA}kA26P6mOwjRht%|cS)-zFg
zT4s@n%FAoAL!M#Fn~GBHPPy#(;zSkjsN(1ni>TsIw@6eR%??ozje5)R@eLhkq2^6;
zv53mWh$1w|g!PMP7S?a;jgB3<EUFh=D*JM{4l%xcnO?)?zVw`9pAnv}a}?+dZ4gjX
zdggl^`;caL3%~Yw90GRCT#*^*<Jh_c_PMBp0Txku=aH`_sL7B(KP{R<OmiKZ=Le)T
zKdG?N(fyldDzIq^x#nojbLT1c<B5RiI@FW-G6>ac4;nEJUh8pJw6T4?3Bmbx^|9l|
zkiE$r^=#imWIuox>zqUBN6o~!Yb~zl`r5I#4nf=0TiM(oV^$Bxen-7o4_|(wndLb4
z_ArwdVX}xd^(K~3dZFMsJudYYznebB8F>8koE{DE2=y(T#ucM2s?GT93lGkJaB);p
zsD@*|q+W%=;I+N{e^~w8^UG_y8oxwG$CS$Uy+&B+75ODuN=5#Ru+uB@ORkgIzB6S0
zG2ixzqt9>_FU|7*l%~RJuQ*P;Wjk>wN8^xkbzkP)w7WQZljhh@u}cw059rQ7p8sbE
zN6AvjX1bPI#U9p+9?^aZk4ijd>*%h`IIj|=_y?912^F*+&A5TwR9ceH(n28hR{ccL
zK1TqUQfR#rAuFi#iw1gS3QeJbsvbF^W;>~-P^yxUSBPlTeWjk%#mX}8bT-n}IGKBK
z)T_5?{v&ES_8=N@n<Dn#dh7>P@VK{wd!UH>xG?u}426r{h|L`@-x*eg#J8%wyGIq5
znD`%wl&bUxr>W8D>m9p0hP(7If&(ho3lDmNyEEXJVzK|@+%w;9%Vtv!@HU>y^^{NJ
zav>G)67F@afd3U&*>cwX0MbU?SHmxRCyw2h(d&hYIx@fRh{^SbVZIw9v`2f1F17#9
zRu9Ws_?GiKO7_Cg5%r|x%ZKTxx7_RePUVL59dn1q;!Mdf4eNKe1F)T!$$J4rYUB58
zDNFiif+m;F32!uySYp``Po<8>J^qLRDVyX^gdZt>^ew4mpNf~$SnOqx{aY&Cz<Ig^
znZ|z~A=H&g*o-ts&!9!^zexCnyn^G-MUOP{7R!!oOWu2foJ-8j8DQ<mo`gGMHGOpK
zcoA>MEn~Vlw)rT;Z$K$jr0ykta)M(g7@27*JC@k2Jvn|cNXWU&qw^OxI)E4=B7Y?u
zVgJ-w!UOrcmN9j*qUSkwZ5|&iE(g*QoIR5E7kOHVl0>c$crn*}uK@xFg2m2$R?9Cj
z)zPOK1pCY|ivsqlb3m{vg8gK1v90-j1B5iv=*`YS0b9X)^>)jSiXlYtzN@q3F~cC<
z{PKpNbCCLfX@zv4Fj3_QWtCu37?nb(s>EU<Hxk&ab(YK{(u$xp%w71~k`L9Djpt$+
z+7TN-+Wot02>Fk}c!z-rq(@ML1s8an<t)nt?43KHS@@#O_ww$V?Gb8WWVQ{j5lY?y
z5r#n!(UHl-HTbG`Z@NE9{-I+xV@G{)fpn#QqdF5hVx%QKlyfU7rp8hCDAAJ0Lc)9{
z8W4Gfuu>lPAy!#}TSa|9;I(XL3C|^>E)ev3!Ci;Rpm>YTYhmNfql8dzMycYuJtTE#
za?45<TD>uv`Q_}2W4Duj=aIRL*A%165=tsjiAWv74HUHZwrk7`m!&*l>b=i!*$3b1
zj@=P1FZZTkOOHdnG(>CR5R@IwV|`~S9vDqe6&+{+9~=-;*Z^5Z>xu3|(5Hub7Z#VI
znz_@fnU_boMmBl)qH62qhl}N1^zvQ|S4N21xY1N@H1ey0zWy5eT_1YrpN7%ROG2vA
zz1LJQZZHW&Sw!vr$<czF5By+ii}jIn*ln;4EUVcB=X*^Wmehzg-Ou+aj1HkH=9YEm
zd%75|U<d7s0S@9+Q!d)4H2l;mW})bOS{vT`$e&T$`7}#%h|^Pi8c-(_uWsnm+mszQ
zZp0|dF&kdmH)6W+x!XM<pm#TCs}b-a#3A~@tr|c*30u=3FK{Jn#Z9ej!*y#@Mb(>r
zaHq2lr-<!KhIx5)X;nH&xU4gJQbofFLo!Uyv`+dKke)#ZWk|nuMs1Sw2~Q6X!D^<l
zGQ69VabAFT2@NOv)l)CkM`Pjr0&`7u)Y0n+Yh2zhXnu!lMSoBD`d%%XoW&(9Yv(si
zY}gWTUiOB4y)_rj<9FlIvE~65j<`nt_MT4Z%AI3s5!7kOontx?I#SEAL20JC=NW(o
zl@e6@#hHqnAo7%|t5FAsmu$owqoL{?qbge-&K%5_cpz>9(I<^6rQ;}ne!LkEL>m8;
zGlQpklHt<Ms9-7rf*BZ&F#?;scH9{+oAgBkvpcfvIoBrR4KkYcB6QDy_4w+BwH-5x
zoZ(bQ_XuYy?w6WM`Vqp62VgXb^0@Pdp;1fi(9rhP+ha*2rSEcbk<{~RB$>f{d23fv
zE^~2NKjvw|laiDx^%&G*))H2*C|%wf!o#NXZ|E=$B!>?Y$9zZF-NCZ&zQcgo--~YO
z+zr^io5sv7zjmmEjX8T3T6W)`CNU_nr&^;H)Q!**^<3?S1Bip>UNpmQX<M8wGTZVy
z)xf&A27QudH*A*E#J0hB?TAy`V9az4t1tiI;b7OMk)ruXW}(ZRYI?Qn?&)wfld|PM
z(3RXxU#&+)%il!<v*6P6S4Fxm)7O&j-Y9q&re{6|#?+C9H@;Q+&5X3#vgw)(+xBwe
z74Kul3SH7~mc!B*O59A)TJiD3^qWJOUgrH|jxcBH)7#)J?9ACESl5?WnkEkLqusc_
zR$O`;hT|1eu<{DJ&k|b*o0Ry9$mfLJ_-<Dw6Yg7@V;RTXXL3~3iW7j?D_}cdT>33>
z^8b~LvL%iXei9ERw6WAriGL_-*Ph{5D$ORJnRcQsCXDHo9xw$J$q9^1`2ysdu7h#~
z8k%B}vnwyXc^Gr0wg<p$&Lq&o45c?$hhV;PQ}$}QZdNbN`zp@LfQ#A;JB67xGkXRS
zU-yRe#zmOJT+GJCyu=aT*tIG(l6@*Y(_pe`au!ym%UY>!1+>iSEX7nMd|KR&z0S3M
zeb4%_-O0NhkDzMJ2V_@nqt<<6IciN_B6+7st0h^wZsVhr^b8|N09T8QbW<{bcT`EQ
z?EzQCxxm_{U{(m|TWRU(VW@7#K(lny4U6xjndxa^U^^F><^m~Q=V|Gw21x1lBh2hQ
zt;%ytmr8eF3OAI33`FTJv@+uoUh}sD^#54%{fAmtDl1&<zG}4@k6KrvR^0KZ{v8$T
z+Jfwkzt5^zmt|){Wc(0fO_is0PC^1ru1xca)rGd&5?Xb-jD$$kozkXa^@hY_qqqcR
zW5pUt%2y-qcDZ6D@I#z55J4qF;38EvNW>x5##chD>b&)~iizQ&b#*6ckXQq8os0%*
z)_Xf7nPTKii^=i&;8;>6?F(s;tkWy05_)jG1ej$?CRI>@!9;$6i&`O7sPO{^POy?H
z(7zEMbRSW<XSS%GZ<~}@hD5ITe`da{l$0nN8=mhROVY(=F7$*8Ev<?&&`0VdMTenl
zO1RKyK|}fG#_UgCtEbn0to)}aV)@N{U76=*`^{GZHx8<&#T3e7wYZ7(+9V=>%|Oaj
z^8R{lR6h182adDqwP~z)+`!R;ta@!V+8fC!-_~Q(lZSAknn4TdbE#efw2tj4?}`gu
ze!hgb7XW=ReuDLUi5jN!Nw%y`6`E<oZOsRWT6&;aN4(114OF&OvySy3$+bX=uGOrK
zkhr5$NNq48`yc?CUuxD|fO9UOIaw~?;dafmh4xCk0fd@0W)%j6!ZpZfR$DIgN|sAg
zn=Nu9aq?TuYC}&opz=!>7wz*J#4BCn$BwiX?W5p4iDQbAdz;T5Df_%~KL+UWEh+rU
zDoD_5goOH?eLAz#<QJdaQxy_YZN9Ps?3f*7VVIGxnpKt0?u8Z<66`%?e0Ddt4nWpZ
zK_Pde{)&);WMXiT-*4};I~$Y2`Blq9?pz$Xi;xS1yejlRpWQ*&`-FX?huxt#ws{1+
zPsQ8W&S$q(@m|gG5Zk7xw!GrC#Ys_Ei2HnYD}$)5HYwYxIASZgcfJrShSvG)<_2?>
zhuOS1W^-W{n5*|+jP=<~6PMWZhA}?7Q6kit+kJLzL;NSiS$<L|Zk}1+xPk;;vRz+|
z7?VPR9XL3#-O~AXpPiaWAG(d4ka;BaTCa4MUY|d+JJMgVArw1>Oz}yD&yNgJv8xF$
zlPP|wE<Zc=Fq!)p@NSReor_mA)QyKP+HDT`>{8P1wI@C8N>RN<+UZ)TA7f-6u`2}u
z>hYV^(PzgRvK<~-Y;m$!W_6|C^{8T-7^*+*q4&wZEK}`;oEatCJYrlM<rU1LVW?nn
zwFqgbTcU}1MMQ6vX>$1$EGAky<zeN+U}h16b~HOItM5`7`T+FV(8ql0zG8BXyuWqa
zbTPmx7m+{Wk{_p14S5g9yNSH7OV0X7;v2A}Nbnt0vDdy5%?Hlz@*NZtJxWFTK?)5y
zSj`*yzA;!|xL6#mL}M508!QSIQ_1Cve4kLm(Hj8{(Hv-9GJ3%D6QMKU#_>=hZWwtX
zxPRi(*>U+~&XIBn#mn}pJ!a$>%lCnybk=o1*y0Jk59l_`#Grgd-v_8?WmrO_8n$?d
zv&@>wr<)Y{YM%?DCg~z|0hVtob;yT>qAV*V!MBx3Bzn<+>h1O|Hr0qJ)~%19Wcd~w
zoz3Mdz9o_1nb9D&n(ux9nUjR~1yg%eoXli9la-+6q8ShQZcB6&d$6RBMq*+<U%5ne
z<4@dT_m&k4%IQ(uEf?(GwbWFIfAYTFyQSfRE>9=fJ*A8KhJs9Vv3oN7(t`opnqr$_
zL4$U|ZGv?d%#`7dHufE&(@0n_RIob)Gvh(45_Xp&o{z+b0Cp*2I`MyYCjkx9mqpmA
zh$*jAnU2aZeMhhkE>q?qcO}_+L)qF_Xf+<f|3uaK>%I(-+Pci`xK&i{GG-x-Cjr`K
zM-F{DV%XCWA^%cmc|{zd9XIgg+pUds5UNN(>muPdJ!kWMOSeR?Hx*2eYbX(9FF#>t
z8{v=z1<H0sqSwMiyQ$%b*kF-f%ld%S)bO@L!>U=%dgklEeI8bV-As?w){j8rorL#O
zNZfDcz0Y^E8x`rXO~M+vdLZn>lXgv`2aZ1%VP>_qzIPvuJ$B7xa`t(J=Bb&j8J~Sa
zTxcXw<~ueYn=pDn8Ubn?Ff!_e_4oljz7uzjR~TiO%S-KOb~(sRvTGNqd=k_XLQV2e
z<t~p$+DXb&&8~mcv<y2bnf7PCAu(>MU0HochEUsGXO}bO5b81)wH*G5-%-lenRXe&
z2-(xgOi)#?-{Y3DjL-_05yNKvcXlZu%3szWciEH@Jl|@j-3^x4vnBsSG1$mK2D3*O
zcNOg~D6!bGOEN7lnWDt2L|C<yG-#WN+)K!1avu9JP_}6{=;&a?SgTW-_VdP>{kqtF
z9&EHG?&Gx#@{hG*j#V#)Qs<b5Gw)wGCyL`-^iVd<k9#YsVhJYjuE_7_*Cic_OY+j@
z+MOaX7mKloOLtK=CjU#y<uD4(BCfqqycLBsWy#gD5}GnuAI#>F<$H@0tz{NqzzAcp
zL43Es_f~m4*hH$?LuFTe0l4uLkfsRf4F}`#wO_mBGlV%x@JOfR+l2KvkU`!GvK8^M
z*ZB*4knp`n7|+IA$-gmlW41o@X5_{L=*S)GLt`$Z-B4$I=&)H4H7*oC7dFhjxgr*<
z29@HhsH-|1FYT5OfYC+bW*`0y^_|N{TXsB~3-J||m`bD(p&r^bO(@h;d;TK6bLwhO
zp{ekK1kJA^zGDZcX-(dXj%P55AC3*1RKKRfP;?M2Qd<rLlumqrZ1Uz{IJFeXlY~4a
zrV&{|Sfa!%BHIaXp@N}oFE$Z}_%Li3d=&5jkFZqxD9bJ-JBw$j>j{-;$}Rs!W_!yA
z<&_lPdUD=Wf6q&QcBux0%q$Xd+IMU+z_sg}t>@3)5|t98Vmz#3%ztxBRMk*RjzKk{
zow&MhYA89?U=Sc^ld)x8BJ2Q_>LzuCyh!T$OY&vhFnA*1(P2XNa7ytJH&?+$QqZ58
z!i3a$^E4<%L3-*^+FD1!a(1Z=q+U0gZgn-?bp8C4JyFSeAHRv5B8p0n8f)=sziR6H
zF?fsL0TmKAle^8)@M`fFY_<`$N|8t+><yLr3Z;dt$rp-b@!{Vvm|m>Z?;c?SJ9r7a
zIc1avS4wxOo5IH`oK5e6BdrtIxJyV<f}Ns-G(t5g-Bd1t+iv=!$h=Ln$!a&h-#a}0
zYj>FMLeE3CnXPS}2YtKq(AFnp+)6_5kG1onUn@)BBMjqo0H-Z)nlC$Opi*j7s(laL
z_GYZ@TgGZT;T}S7-oEGKg&^&<m=5@-2ni;)l*e}y7zz?--<ZwUaaf^s(=p=aMUc08
za=(SiqQL-Avq`LGr`o7>)3+vr0n22^RkPDo+c%wp*@J|T5{HQ3;t6@|AgQH;>h}nu
zqxE9_Th)c99wC7TJqhxD^%V&l3G1zDE>$+>E@aBXUqtkEl41yGAqg_QA^3vC=fY@a
z*{7KUWG4{sq{jBvhRR+Rb9ey!T1jPMqFDgG_2OjC^ym^pa{nK9?;lrHl|6o+eeVJ8
z5mXQaMC7-Cii(O#ii$tM!o)<QqSVAhBW04*%(|3R=Es>Z%|k_MWsNqJXf|3|QCU&h
zWN9-tIb~&KWiwP-R#wk@?Q<?*^Z7on@9Xva_n`Zn{bTL5*IsMwwb$PJoO7iKDKolR
zVd2J8ST<e2SchTTyIEbr1^2E+fRzEYTyQ_l^ZaVpO4^p5zAuS848>1AsR%Exz3s)1
z&Z{=q(c}v8(Q23#<Kl3}$dl1Lhyx{0R|o12qHaGIYs5t)iV)pncY<P@5Et}9v^vkC
z*N^R?3`-wzjU*l=!fX}MAc^~lz`%$mhDTlt4ANhlk|l}k8|BqQf!${1{ej0U6bp0d
zAYY-(I}E5+$7j$IwW}Y<-XZJ{^y!4|ljw;NtuE4iqMCjP*$8?wS{2cyXFP7zZazbG
zGiiBLuG_Lar(VI)oa|@&M=_W2ofn$Et`(SkaO$nlXP#onEyebC!{m1Bi3Z8*vy>w!
z4dshl*Og}G(B;3dci1hHz=uHO6<XDern(I?7#|vGiBKB!Ufi9LSnZ(q0*J>0<R6>r
zV%FZ*C;>I&>iG<xgY`mSiUL>}OWj2HLhY=Zp<^X+hf%*W1*%!=)xu<MTMkG<0BQj!
zQ_qIwNFnXcLQ35jaNV2<^<V$;10;;phYkDKBJup|Yl>G8<UP56ogvads3-A(N#rTQ
zW8+CMH;tCbiG9F*Q<KSvV8HOv8?uW;K3ol>D(o7pg;Vn@YJN^aa7KDa{7Pc1G#80h
zMt0}r0F6}F<K>eNA~?ptXpAnAhX!foX|x8%hB?>yKE~(@DX<5;7!u=OumQmRw75YA
z#<N=x{sJbnNTd>(Bngao<TS!$o@P`Fnhp50;7yJoMXFT9;|%&VPJM?#eU{fVH1i|*
zltN^+i0o5W0+F|o15YoYvNU*BhO}J$Mp>NtTC`O+#1s>GXMhAex5!7Nxn=~g;7wAm
z6JB9-j45)TB*oQPm?{e^1iRpU3QCD<^YL7npb;K@M35oA;3p#Zh><^nb76>(@;;mR
zW2%J~xf)4v>H|#*kWpxFRE&!wY9tBy_{KtJ?5G)p>GHl2IxuFnwlk;V)Cs!!oH&`h
zf3BkD5N1i@k3_B^TrCer0Ii>Z4LpUxAtt_;rvAFWQoPl`g%vqG_jggt>t0d|$8&p&
zBGI)86l{cw>kfnPM#2qfj*^=B719a08WZ9k?hM)2y+pY*{y(Ehd=w~@+8>i!PuK{w
zU!0uRT8*)pBub^}=l4SPn7>nwaf*`3e2i3hf$*#(m|dg(NjS^|tZRze8Nq#%Q?!#K
z#pr3&x40=|K6X)ck1|yEaCa6STO_k^@VMj*bSCckE5)CQm5X+2#+G7Mn0248)3dH0
zJs!)3!%lSh`K_YIOSCl{yIjD~XfxtO&1?vee&n?p<BEyvvy<i;X`B(&9W&THLlUJ#
z#t}vf=>jYjjo|B&^Mi!i(m(~AqV6*sYO@7M$X%1n#;CV>u%;G3@fEr+BJdC{m^3+*
z&lakC`Ij8W8j)RtS$~;qd7;R=1Q~K3HLi~CfO3SP=G@R~OxVUWsfGT+O5`>Nnek|i
z>b?vuopY~T{#@ojPK?#$?(V5#@CY5qzX16VBQ5wdBs&Mq>yD?;-7g=dne%62UlTs(
zSu{0g7w0xThZA{&2roIkzz5dU9H67IPSGBDqIS*)6b++@Z(6B2pOD>y5Fs?Q(wm9<
zg>4iUh#Z*1oDC@Sdg2i&Y0u?1FXf#jHRo5#juPh$K6NEDN0S*X+i&#V6T<srlMWNt
z`#di)*gPSqPBiR~o`xOsb;FMFlAkYu3?}R7rI(TjiiVU)EE&YKz3{Fy>Y6s0oF)m|
z3oJff#*+o!7k;*<yc!Z8To5bc&R+a#NOak03~fOkt!n%KA}bqp8s{haKEgT%{r8q`
zRq?EiihhOAd?^rTmEbJ~<x0c*;x*FhI^S3!hbG}b!3KmFVSn^$NSwcTR;y9iKnr>y
z^^*9o#h8icj|lHyN@8V{TzRq=xJEsTdS8_11enStYady+(hARtEO4{L8<4Cs(y)(L
z=Uq!)C|<O<2^e-60`h}>{>h>z4KkKrYNTy`luz*ySQY)FByde)5OvHT%+>TOh?EeD
zC9#mmRfKYgn?6at2R#{KOS3O6(!`0p@kXQoOC;ZTd+8RT$IW9>(Ru!jlDWhqY?;ge
z&`UdU9h~h|sQHDH#Pj!I>$C`7o{0#4`IVNJ5f+p9HI&F^!ab6RCGs2ryQF>{Oyont
zKakX~V;o8SiYe#)X%#EaYyVox;E9bR#0q?GA`()0OQ`GqR^x)TGR1Ob`=MLVW_}fK
z{}S}BD<&?M)njaqBrtxl;|XKw!3Fj1RO49}?sP%aY=S(Z;vmHT1q#b82>%Qt{rmR^
zFcyR4!T`q0Nn9ovoBj_NY4TUrn?>Q^LSgrxk?tUoSR@l$LCvBl!xJQ*=BD4G0#Jc4
zE=V%e*RLAhmswQHp`yL?^^k7q7i9C8!5DH!w8M-0#^aV%BAE+L7@o*O_T#UIL{Ibn
z1a6rCj*HT~{2COSn77#L2}>A=W;zYZeTZjWrrh&A>v9;F<`e)n5bk1Nnmq(<5e8-c
z4!ipse0u8(Ue>;F3ux;F?JMEP<BKnVeu!|tBu*1Svax#^VP7g^1?P;JP9O9ao3Qj^
zCiYJtUmT`ljRwEbvwtSF7*4Uw>z@5pM9va;z1FjzX^~$7P(7kA^^=;L2~AV0<t-m!
zgM*QMYK{;p!KfFRuQFYFAKyzI7Q{X9;k}RVC&J9pgIz=i7#mF$u12`Ki7g=LYY$l+
z>&oj$+}YcO-;GlEjpE%@Fg-^-^!~d5UYx=n_Xa9qzI$vJz7?Ys&Wo_@d>~tIH8Y4F
z(qnjNe_9mrD(U{Kved>;P(6g;-I8b`!ZNDI8<N;Z1Z&pgi^;|{UR^au@PyPPXX)vg
z^0>}~5VvMrKN7uNoN)yt#=Fz>v>ciDLLNvS#Ju<Fc0gpU99KrTR3<?^TTjCgliB*!
zzdH+iRtwWs3J>Ul?BNX7dpPy+PFF@hKr!Rq@z7eX1Lx?+!FLT|o+O?oau;EZB;FzN
zBtbruIQm~i4iVnOIE{Wq)PCAgx*KVDTS^_t%HZ@@Yv<r1gCbcf@fgU$PQU4Ip)xS=
zv_=?e=RaGG=_L@MXPwuW^wrK2VE%=Gkkn4JvS$_{g#?qP>UkMq8VgIkE~Kikr72on
zF6$?aZ8_MCkbTi=%4IB|UK<sth|}8>FT_FlAc;+qps}7$5uUsml)E}plaDZ{U0ujJ
zOgI1!r`O4f%U=68wsEGTFi&)mOq6PuSYkPYm#*;M91oQorR_3Njn`g+QQ|hn2NOJ~
zqLj0#fa=QC2$y&2yszy{7BF<~3XDv>{#cZ66x@@CQFx(FwDAfUxJ{-NZzui2A#L5+
zm24S>`uU#MFGfOq)l<bM5TYfKNn`?Hv?OwfTuCUy)6z8yscHBQ^Xg`x;!PB<2voeA
z#JWJmMiSfbX?%z2wf~CQZB#P*IaKQPKC;|_EEm=xqa<O#8(>g7U~l|u1hyE|4rCes
zGl5Z~nmmT;)t`_miLOLu5oW;qCOXpVkA&NisK(hM<aYXNwA$`RqHn~zh<iN-5JN#D
z?u7~H)f_<lJ_%NV4&rY~{F@$h`fIXaryC1Ty1l^uKIV3#pp-_~1)B6kt8w*eQL9cb
zN2|Xgqc_H4*CsU^Dj|f~EFnnrec?VH58*-jH)e>w`!?vjGmabd25)wVbpx?pqf0Sj
zyxJ%45$Yn7BA!KM0`T~1M0q>~`vGWzcY06wz>Tz6fEkfrCrH3ldkW}L!ff`L5=yZI
z1)t01J1lG=ike^(?&h;Jf<z1{q`#k#1gj2A2;g}JkS7Cp{z2l9Bt9qd0|9}6&QOUl
z1X<m8dY`3mq3C%fGncEDsY*p^i5H%gCMYYXM^*OTw?l^bCoyYP?#6kZAi{t1R>8>f
zED_yH%#=hVkv|b?g!Dv6FTmI(G6N;DEcFx&H&jw5f;pVTSkUv;t2aAK$s}?CO-hzT
zF_BtA4b3lN=~5s&?eKc83ZB2-tXOY)9}dbeaGyN|mxJVIkX$15uAt6Cgw2wuAo4dt
z3xZs<m5%b&k~a$-MA9#Ir^s{Dl5q_izy<s5H{S_KA157MbjHBt&>cPDb~Nc%5T0>6
zn#}M^PSG8Oj|KhYG8|?u4o4<8+CvY8B>N|hBpn4Fz108P15}-O0oCYXGFv5{iF1kI
zYb2LTVlk1M34Tf3Km=AMuaU$?B3lTXnDr-KEPA*&i`YXvHF_~!(B?@p8MuFbhfMjv
zil>@SAxY+P5iXglB~yQqSL{yV37P#D*GuMgdSM~TtWRUa&_#X7t4u<<_Nd>InJ(%i
z6zm{uXS~RYGx<%zYb3gxL_Q+?t2bMze9oq-L|n8gzZiXApPP7!_^l%P&gX1YGAfw-
zO)3eVHU^H@1h_J3cNyL-RZekrn!QgqH}JCqZkpEknvF_E63L>W^1!;LwOkURo%^s4
z=6!1&(`D4jNWbJ@d!x>9=`I^<RV(FvFwF=a=G2OQ;eIN@rz!@);~`XjSc>smBrnhT
zV$EB!pX2QmxmpyZs!#x3AQ{*1Y-%;GEe4C`)!TXW^d0z=(iuyV!nE&shmo0rdZ#e$
zd)|4K$YjEJB>T>XMCK9Z?op%;-Dr&GhW1@s1HLrFd+=$2`Wc8S!XlTy*KQF0a)rH}
z`@-IXm`Kgu6!a$L`c#L#ejwnlhU>sz((u>KZR4+#cKMsaL@KXNYZGv7EgjZ7qi}@>
z{GASOm09juuKW-H?^Jr<?#ozshbM;VuG>w23NBg-yG!(p$R<Jq!{nV0*+0{r)U_q&
zXZU(83o}<XJP)d6GbP`pfRp_P=V%$+NDe%I;nP;*dTgole$qRWXyXN}TuKw+QCRjO
zDw*QPx~9nYmR^h?f*DHroHc|f==G?tzH&y}*;@by-<Clx3Yv0?@NKHYLEvB@*K-t<
zGz@fe+ZgDiT?WdGm&#1WA599iD{u&$D-83FfMf4)K%}Wo#lWX>Rt(j0iGUd^qFiQp
zJ9)8M4mZF+(e+0I;;05YbMmt8twxRLLXB9f!^~uASvnXK(Z5tq>eMpUaH*pSxss5`
zWds}wwd{N%4E)r^Ov=mf@l)?0+%AbZL~s&P*E8unIDr=Mml;p5#uc)ce8__I2N~t5
zI9aKiCOg~$BnXiAy3EeS3)!*DuWN<qtn4tHFYk_*BoW}z9_l_REE-RHD*--2JPGWt
z3jwSvQePxA_ja~EDMc0<#*DiF{v2Y%QSW}HVu(_av)<57JwoE|h6vD)q-*d_^p&9!
zSIojg3pw=@fIbo!tPmiA_ymo27$u|;vYXo`WG5Zyz`wzAQaV6$ju_AsDv7>?$BpyL
z)}|H(132{uNg#=|a6*?H5~JmEzRP)@!c=MTcrVN5h?}}>C7k+Hl6*JYxY=kM8`E5E
zDA4U9{>p19k0PZfnZKukud)_gS$Gm@nP8}_lgs_g`%A}bP?6&|ajLcp+QPA&S}NdU
z2;xUCT~36hSlaoLs3gMfP?{*q(%D4j6IeN@r3l|i2zlAi*`4tj6j4)ia48geJ6)Hp
z?81MBg!?spMZ%YxxytVIP4d~v%32|`4lL+j+8;ED@s6Z{^SIDvoVtT0$GfcQwW!l>
z0(hk>>NF<Ew6*dAjBE_w46XXgn2uBHSwp>BOD(_b4yWBiut~}8aN3^<54Tmrwoq+3
zr1|31-BRso0hC%OXc``sQ|o0CFG(%ghrhW-*Q#-9-6OSH0hHsFw08-I0wnT1R+aV%
z+weSINHzMauez}YygqMZHvn1*|K^Rp#>C*t_2xC}P4FVxrMyG6Du=k9S^mLC<<!zd
zL-l64k@hPbTiRDH!+LUoCLob)*pRrz2=?#I2x*y*XI^@zQX^SJc9uw6f+l-s69!1h
zaU%JI(Fk)@u?W)+JsOhg*926G>{fAQr+YzIfCf~gx9INDzC=)w-s0CU9Y*9D!qvSU
zK2Z(s>dI!^b-caykHIdRdZTB(m%41~y_&>rGMNi=SJfC;&aiQDex9@S{$oL)#(IF(
z;kky53sifEoCjo1(i*Ueg&VaSpwx`J9hv-t<)4;h@5cd@tvQi@@23fS8M02LEt<kD
zR0@x~1H0x_cpaD)IdauCuqnO&4g{W&uYap?n;5EGv9?F{LW;Zw)f+q0n=N6l?;;{U
z5`K`x6^2U3x~F3ayu5y<(t8kM+-dwaZFsn%4W0NrjQ7YLu6;`H16W2X8!BR;fWQ=K
zxDE!2bVyrnn}KqmXj<H)bPP;7r<L#n$m5PPb41>!(#{MH(d{f)+({Sz&~b>ebw9)L
z#Qe91Q&;a9zV{|Q2p60a2^b1*_(3B0tm*S;Eu8Iwbl9KH(<6F@y&c^GJ-F}y)suAY
zz4Acku<Gf}uJHu|<E!C1^o=w<zTGd?%`R=7bHD<f!|JEAwJtNPSE#)`UldiYMtC3f
zIMdC0PX&nnC$N7H5d95_uakvn)ctncIn4pqiGK~Spia-UxBII)zCXegiv%B^(Q4ci
zOB;tp{{Orit$!S&n8BFLFeOn<B$g1x*nHg2LRZ(Q7259MYUD?IriW2ejvF4IFPG|G
z0?CdaQo3iYG)eJzq~sS*J*wG|{H?O}LH=Cwo!v;i{0w|}^`5;QNR2wIC4LWM^a*Ui
zKmSMS;Jn3wr$W33I^xM>GT2@QL0Xaj`6C+{p3G^Ub*1b#W?oGbUZbBU69vw|b<OAl
z4cn_-+T@qFgtQ)w;*e~|Kjd(8+A%)Xc6r}cqfu<Bo{Zjq2;*}3aH=!AWSl372}GDj
zGA?Buq=6Z&RFN@T*3*;7mYD;C+03{hP@93DaVG>c!h|_D7~c2w0>{|}Z#)c9`0eli
zDZsEe9uJduXk<A93=}q=5*U~0IG7`XJhh!y+{1|P7eOA~PGhH@p`pJJ9zr6gcsDHx
zfu8!9jJF7{A*)mWxs*$bVlvhBg?~nPk8UAjP`9J?3k69Fu%8i*2QYq1q7}Op-a|0F
z>*8d^tTKL<hU7D`h~}{gLM7dNY_@aiaa0y}l#O*yJ*0?lE{8?X)GGU}kJ#~jY<GvA
z^_2OQ?mm{rp$aQ`>Lxil8_9e02hfS#($!B^i_#^@rN3Nq>CEFm?2x_~;hE8%MZrMb
zBowpx45)Wqof6gVZ>Y=+LNCe9g)+ZXU8}3InG#+c?ODd?{BW04Ef%1A`F%)Tl{q(a
z8eyvB?xw2TU+}Kms7wDw3DbAxrP9iKL=Z&g4ix9{&%m_6@c!!)f&WSffeSm28*I#x
z8^4}jN~TDhBQPn$Wxw}(%AT@AKM9HNv@2Y8m`J~85BdZR-No44)r;K&_4tqAUS@bd
zxIu9L<l^3i;S$^*@M?P->gzS?U8N4{4{{v2NTbYRsF~CBt}3Ug4|u+@eT>Se+r^Bw
zs~4i#6|_}w*l&Cil90OtQiQY*9spj(Q%`o4oL}^mO{MwHQ4~um2dX6iWhWo!ai|@I
zZvMXm`3+Kg4D@uYUFO)04`o+Mgds3&KDyt#-@*Jm;d(?ydkzT=ZM!Vl+cJcqJD8h=
z$`3^wHI<Em!+!EO&fSh}ZsC$ZVY6;PSKQ||&3L3~4yoXm>@@_wV!oqQ06G=|!pqob
zp@?T9;`t-(o)Pg(LS-Ok7dS;aAAUqN`67Yftkzn;>#&fbv%{qghCX|JYKPToG+}i>
zc3-sTR@C#OXGA^k1M7|$S3P$yVlqD`d^(=oLDUc%mS=d6iEZ97n8wk}9aynUT=~q?
zP|{QjC4NXb#@8JJ(uNYG-Nb&=q@N57)p>t|<YO`uaz%%t)o<ZUkg~tD>!i*;Nj4+S
zi?{MoCnm@O{JTC;gl>|EB$7l(kVFU(bfM2+7K!eDlZ8q>5w16!F8C~@zrP93dE!H$
zN`@wFp|eC1O%;wLKH~S70!rZRRO1zH>rZJmF?)<WOTC%W7vE*N(tbBj_s>_R^A5Us
zH)*)bC2yxs_tyK-Bg+2aGmheS#$%J_xdg=f=pb9J<4b|upvCZLG(Vecct1Wyr^e5t
zLo7o3$O?Y^ABZd@TqOc}l!g2F8%eDp)Upa4e-E8y$WZodB991pKYmLpJwowAga;(?
zoT2=Eo*=bTQoD#ePk2`FJuCQd>-roZ@Cib_3;cxsdTsc-iL80OAOC?$lbR|1fN)eq
zxf90J7tu?ejnkXKAImhu?|VtlpU8JMgFk*;@|K|4qTg&RvE^GQhWLr%bGCy;An_cY
z=-;2vI=i!8uu>Z_XcN<v<w)w|-^g?go0{Avr=6LOd_Re%SfkVO-H8t)PQxc+n6%Rn
zpARwKPCTe15^`<8Cl^tSDpMpsH7p}dN~>L*oPs$xONI8G?2k`;D3DbFd9dU0GYrKU
z;yvUi(Y+dS^$F&utUNsVgW%dX^uN#Nl46gmZz`duBxVqqLYRmU^5+n_o^XvMt|sy*
zVXGvT5&4Mlp=^ic^G;fCd3}$T_KBAuS6=8TU<2Yf`x;)pNum1U%=P_`hHKIg-eMQf
zj3^iY;8+2C0ia(9KWniy$s9=0d|^kmh)uJyhIuxB6mFnq+7rhdV}E?T{oxlO(b4fd
z3o~Edzky}D4Y%xhUFMJ9B4rpHDa&=sKGPCVpx3|8EWh_XU$?VX*t@$_z=QzN8wX;G
zOxC;u(`vk%M@LV}Wc}&uu<sd`1ANs(^^@JiXINwP!%X$VpYl8-5kE$?-_?>TCc;eH
z?|!!7p1B;3KEvSY$AYxqTbGhpC<MM+#ZeOu=%?4xx+>(yvy69{lw^<iQzlZD2Hr<p
zmP|?06)+5<U7t2nDm_RlgJNyT<(GGq$4>kzr0e*<C(**6#j{$2Zv42P&+6y{zwcKi
zlhvQcxANAs8b4wF@Yv_Yn2%utKFfrlpVDwiInw+r+Wx~=A(8%{vg8>ap+p|)5gRTD
ze-f4$@k?UNhjF#OU(Aq>UxHCygHzJ)JHpoh+MF{3C@p*yV>ShW^q>HK1)%+up|ZjW
zVVHFL6=y#AiCur0|2~slU+G)IO8@UCz@KIVkC*wGyk+h@R>4bD$na-OxP2I2fy*KE
z{&d=3tK0uE#*~Al&;KCp%L8!t=P(TKU#0;Vm+?Qwcm@S|$+{JqQV2<sxQp@Qq=Ylg
z4re~sj}$P)%F|a{YV>m~KOJz(N}!4(CLv=<u8{T95h;=Wg!zB_?N<uCOO6Tigq52v
zYE{8|8LmdJpcqqj@4uJ;g}s8{&?Y*XHNwv&YdEQE$R0!wo|K|%$wgDMt`TEC?t)eo
z(kPa=*Qgl1xI2M2>lt?$0Gw51-6a@Dp?>4$lfp&FdW6%OaZAYMyFw?TrG#;p^#oxj
zKkOF5SR2i@y#BpGKYdUk^n46oUw3{)(RhSm^d+x-^(v21&+ywgCl)agewiT6ltbN@
zeL?gx;f2v83R{(LwscoYFgjW96aGP(v9hTK+y{x9Ao-(o>{beK(6jz6Pm28v>--J#
z_|aY;rdyQ$1ZRg@F)~&&7L;$LoBetqn}@LU&ooQn26<e7I=U;%WL&EuW99gqhlMD&
zFdKsN3(f{z!y~IQDG7IKe$kr`{>fW@&HxM4g&|gwKsYRt!evIFP>Zz=#XD<iiCi0F
zKF`ql<~h`<F4s`Jr>gqP7vy_gOMmMolETO6-%}D75gAGt1Z(PMfg)TU5XCrr@rV?c
zQ-m4sUm*Ti-7BDYjG`B$=nb+<2{UyOUQDlN4DY{dr08q1aX9*4DG4Tw{<jdMWF4V@
z9bv5`x)FJp@Q@^gBDA4Dj~@8dIwryXEreI3=sY4v2=73{x=A$jIp1$$3s9{qAooYY
zH<BnZ)PRwMev+6&WH;e)#@)I_l5+#8qlAA-Vl|P^37^K0*xjm(Y*Ckl*ck8LZoFmw
z<yWw7-5~%@5!lyL>s~R`Kp(+`j|uYV)4F4%;s~*b&!|OP5$u=miGWsU>G^&WJvQnB
zn!Q4rkS^Q}1}xrJFA5e$%Rr3#K-slj_Yjfs^nc(O!FdpzdBA0fvOqqVSL&ZEHjcX`
z`b@?xk%vQ+hgnV6ncbmIxGve^maOPd5~WL;+>#A$iM-{yGFBJucZ;-t<2Dp8-SWnZ
zCR0I7CVOdOSvHooDs@;ha&V0I>sUU#bpY0FMEV2g5_mXQZDghzcs+q9QPjqTMD8T~
zk1%yG9M~u+KNHx%2PD-%<VC_h*tT&yFz17Fgp{zIH}E~eKV(dA+}Em1X*ZK%PJ-3O
z7bs!{KJd8o!aKp~#YtJB$fx$`pHC(><t?M+H7ecRu|)hgGotmk@fxJUt-PZnh-Lwn
zV9aYBb7i=TB8l+M8tU$M!F)^rIEN%p#gjLbyi*<WY8{L4bsBk^m#r$j!zs7&{He&d
z%oxnJcS^7z@i4~kx`F{vW;c8rGQiLL+b)Ro%(kgZpO7P@?L7GR8$>x*<mBDnm)Z}D
zQ-nh_5X(2PRiwA>-+-(Y1ClT^#=AR_!8{Ea-PIX@-v~cT!Z1{JcS3{+Y$lxc-^CEf
zMuD?4kkMTkWS3J@WOP?1xmf9J5$s(Und~tHCIxlZI3f(!?29BZmB=DOB{SD=%xR9*
zzh0irfRD2c@2Cfn7~gxx=&=kJRvN!mOEwMp7Yi@7W~zLL|AMB*Pz{}&5<TYXu7$8F
zY$8Y3s%{yr`3W~lqL#=X33s^6I(t}_Wn5_}#(P@^jU05DmHhy4n+-Xgsk;+|cZjX3
z!=7b7M)-?hyPwEj0yDp=!>i4HnV{|E9wMI*K7!X9KWsIA$&ooY;o%tXdiF-Y!Fy9S
z#wxp&a8eRch8l#+H;5xEWrq+MOpwj`jXx2YPMFFUG1Nw^?H~ryAXzgdJWRXx08%#-
zq<{8EBKHzDNTQGkgbaE@5;KU<*+H*M;z~meW}X=IizF747)3Zw5-W+!CCnL4;ttw)
zfel-ET-jb@s1bv!DY{ltHW9cw_(tUOOQSPtTJ<qG<DDVrY?1uZETaYt8Mgm4yD`6r
z#Pdjx6>+seBE(;g31r7y(EwSN!@=!iH3x$r=dBpeYb*#_--9*VuYdrQc<^dTFu4tW
zkg!QO^A^nAzA)9P`s+#=5nLIWu`xwqfvSH6nkW36sxOjgrf1vX?%+2Gvezt~D|Si#
zuAR6WoRUhXbKl^&1Ue(SS-~&3>6$f8&2Rb#$_E5~$nY0j2jR;%QvyX&B2^(3Zu}~y
z_%-{5??Yljj9*I_H&@snejk$SKO5G;0QBDKuxHIwRJd5&(1d@*m@nbvow|~4t+||D
zd_s^f9wz*YzOA8eLog|W&%)F@u@%(9lqUQXV=lsn_;wMs?)(*)zJzqC^%J%DbwV{H
zpO6=X#Oq>-l@cz3IcFhFjNG>uNI#b7L2T0Pw#g#BBw(Y0I)xkTFzErh2JJ}n8Q$*>
z_kyUEeBD&;cR1TFYnUX5Ak`sjgyM&Y+)ubq5>2!}q>0cdsb`5CBphI+dUh*~@{K5E
z9x&8;bm`lVHAP{fi|h2vXW~`plC!!bR+s1*)?{-3(|3@=Mw0G@;5p;;>^m{lE`}f<
z?MrvR%hj2>auECw+y9+@(9dZq2xF-8W_N&_VCbdCcdN(~+n;_Qz#6Y?y~VYvpi;xz
z^el}B7YPf&8CZ~~>!~eR9=%@kKKaiI-X(gV(IS>J7~gG;{#8sjmlX;lcnwON*b{w_
zvAqV?3^_$ODNPU&RHxV;2%xGKwN)XQeO;2$p(IQi3E}JD=d@B?rE8dt4!Ng_16`{W
zaC5h?)j{GY>ZXTixJDE;F(bx&8AUz)4qBF&0d<>p8Y(B6&<(mbJx4B#J|%wXrkBV?
zBXcqh*?``33QCe>>Pj3NqZ-lM(;V8W%^o1mBMcXeW2v{fD=D1Dobi%iI?lP2P%2%_
zatP)#!gL-d1tmdwr3B+7zs`8Nmy}fN5|s58evvx_=}bF4!0nH~GWUrac)GA17K0@v
zOg{3%%BN+IMZ5Qce=O@0=ZBS52xCN+4yWtvkA7lvS1%ca!wrtxA7~nDj7_+qgY(ad
zXVcpNZh@n6{<P`u^l3g}u4J$p)T0l~U(mF6F$!fT=cm;)evtB?8Fr#t-&=w5azlQ^
z^5-;X!|Qvu<FZfibFH&a@cl?&y`4<HFs!#u3W}tqvxc)B_Wh;ONv(FyvyrF~ChV-%
zVYiWtZBR$%Pq>L~wxM^XKCQU&AFY06WdV>UJ0RcxEtOYU{=pKc2&=8HMpVx|onfn0
zLT%1|zJDoz_Mi+KYc>@=(zs5qX;q!eK($R!{qQS#ak>HK+(zJ8Sao_kk!J``U~o>q
z<QSaOZ=u#X9}?a}K~DcwJU5MKpAgWQtC8O=81KaU!)JnnX(Q(c!f6-bQ0_yOBxq`A
zI6)qGIE@`vr{5ygn;@@8@Y`yt^GW;-xf+JATSzdzzz<uQ*X766)X;3g0JNYp4=d^)
zSMzp~#VbAUe84#e`hk5C+*!7A_$72UnE0x~AlbE7oliKd`QfO`nm*OSn$BlojXcJt
zI<J&NCXrP80TXAMh7tUZVRQ~|*+(>Or-XX@re8v$$JeXKO%YXd7tETsXwH?D^JXnr
zeC3>Lb&Z}iJcqZmUWIMyR552!Ic0;+;}wJs$y|~No8s-p>pXl#R#4yll>~`h$DW!5
z!$>yx5gNk?e?l2*_!F(_{^Ayr?=%n|A)KxOx((=6fO6|v)#fd=|7W200Xna&Rn0%O
z9~<LfV$S~^s{aVo_pR!IrZd3-sNVxccmW!j+^QDjZ%g?-kov-*JR$}wdx*KqIP@(r
zkF=^47(`wvS9!aY4?DUu+A{q?3l?EVOi3mwhOD6;P#iDdvnn55ImE^xXcpY26x1xZ
z(}C>iq*f~rkZT`T9;(l(z=b^2xuz>^^p|MZ&Cx`8qsBP2vsQA)Nwn!!$xT<LAa7O~
zKFQqcwDpzZ85HBtKRR*o`aJ3@BF5JM_7Aal{uYvV-l1(C#Urm|^4z@Y!X8Z8PUQ&~
zNx*!kd#a%;Cest%O)^p0B0}Na>Aqpz26|@AT4s2nAV772;(-J2UgS$nJoJXrdB&~c
zZ5(<fkTMNW96ES+lP}Fbfau{1I<oQry6H32sA;Cd%1Bdl(sVd!I-E2ePMQuUO^1`F
z!$}9Sr;}Q3I-E2ePC{IOlM)TPz+t7tT~<oeWu@t`(sVfKwDpzga54(8c22V811Gom
ze0lzMPR0l~Vnyp!P&eV^eQ;9u4kKllgS##=z+&778}W#TUemSQNOaRLlYXIx08!-#
zna03zDk!Q`C|M<xeM)#=63Ad!cS2`LOd#UFgv-P@selHVm&>I>E=!=myoo~U4trY?
zcQE`$5&B7D0}%-;C9#<?7e?ZQBpxO4C&C&@JV^vwntQ(_nu&Zt_*4??6y@SV`g4Dh
z)VmC|NCJ<28|JYvE?>1O+-Es$<L87WmygwO{RK)Hs-B0a6~t729?P2ZG9}T4$c2PL
zNyHJUB3vbjbRsNv&f6df)=TH@B{WHbpCnc1y+`U@Nu5umm2iTsE+e>Ct6I6zCE#fY
z2u_jgpo_^qOLjg)@Z^;dR3>x<y#!qwnCbdTR*PK3Xa*f)F_b`vk_0w=1Z%Ak<0OIK
zIN~b$KjI2f!8qF^ZYQjgL^zR6gpC^W0x)j{^LnYZi0l@^i;`GD<T&92NicQsjS@BD
z7fIYjVl*K~5}+DcNw`uHNM|ID^T;|$FhluA?k3e}gwyUoT52IbIGe0P1QC1iNFx6t
zyiYfSCjo+fnTb+z0lBO*M~XayFUFd|^*qts;JHLlwY+>uux`w|nXpU}tX}eV5ZJ9Z
zg6VNq9_-IMB&qd8ekA-y6SIYu&I;R4!fb<Qz>LFvHwO9k6(2rr#$g^X64=3@i#-%f
zGShXA#K(4s=XHn=?-1|ZA)aK11^eO!>)>{*FF9B*pv%Us4%YT~p4lEB-XY$*Lp;fT
zptCR5t_${=Wsq0gfxPy3UWfeQ9pb$^#FN~(ch)R^o5iUs9S<HTWSzEKgMAsJX`!J5
zsqOK+4*A17#Cvy$C)qc3@x|MJ2=T>>1vc;3nB=2}SYS708e(w|+KnN;zW(=qZ-Q$M
zBZw1*$}j)_iwVB_D-{%sPKpV3dZ$hBB}{0p2_BWtl#|_cX}|=J+Ddwhyong(!?uc*
z9)-Od^)#l{yaX)A$TYGeRZu5wl^F-47(1gr4Q00h_e_n>AoLQmd^Hh@N6$s)vm+S-
zX02TAl*_#kWj<`km(<jlP)Rfq8ABK-iD!u1O^~5xzJf&3Nc<{^w@Exk*e;2qM4lsv
zpJjfISDVj9Mn0~SVSY;lJC*;LB=8Eyve`5?gr<W+(2JmMt?Fh>g|^C5BsWIes5G+0
zRyJc4pQdI|A8HLFXah8q$T-4iDH%&-CgCDUOd)bL?T?)=DW;;awFH?{g60vySdHb3
z#|ZiZk>?3dNo&CqTGcHTw3hFh;NZ!ch?~eh<7S@_+E!_U{Uux+gZ-Q)i2ehT4+uvk
z@iP(n>L2?hso)S&zY=6V3WmLW$4-swDv3lQeF<rj=tpD%L7TSWM4)=yblp-NEuBwG
zngfVo+;ReMo)|$75V?gQu4)is7<VV(e<bk~?T<ru#%+<*%S4djxF;on$i^`*juR^!
zgasb=CE>U>jx%YgoNTeW<3kA{lDLvc0wGSOFOKQ5tC8U(DY=&1L4^L2xQWOlg6_c@
zLrouFMM|S%W*ColjlWOe9wc&z@R}qZC-Mv72T43_s0lp@k(!uSY3Y9W9m0Z|zj(3K
zcAv2SGSxRZM)m)rsg5nF)0VHJx&G3@+-{b~b}+^N*XBB2>}b1n9@`=Qe`TrdpF8{F
zJ&sLI9rvn3Q@br5+rj4kU)$x}{}+>-ASQXwyIp*%R2mCJ--ytWq34B8j?Be~#Pm{w
zA}>;V#|6!ZDGj|O=;BZnX|D|T-P&h>@^*0`GkPc_mYib@;>Ig$?HD{Ud&b4seItBH
zIYEL^r+J>b4QYhx77W@@LCYG##R+UikI9itC=tFf_S+G@sBRp)X|(NV_IwiI^Qz1N
zYD|xILXtF|(V-A6s10LGp}JAsqI&r4E|I=5b0Sq}s6({0H!)Prh>^&oj-8ntDOr4x
zD}Yu$by-Na-0n_}9Mi*tdp)_W7n(Kn!7$FZKaBKs?{Ps}k=sG|W*9{Sl)WyB+5A+b
zZybH?NI1RanJOBZh6JP`-O=12yDZB0kQD@L<H2y5{Tua@%G%Hegkl(%cM8-?R(5`O
z-{O9q4I%t0xrrp_CleW-%)r7eZ*M|+vcTTa-4|`QcK0njFUKf|bU4IMj1Dk5(wN7>
z;Mz8_gU(A<89k(b`LQd|0Q;_J-%Nji-I|BjcL%~@=L#t>3?7Aa)u|(6R(8^=bEC^+
z-F1h@H@UqEr7@0<(~Qv2(V_fylTo8wQ35P%yO|&L927aDJ8Pd?l(Z$8tCS#)P<vL!
z`0|urYi4pBtWCBiN5%^)Y(@9kJQ!vOAMQ~@HP7x<F(`LOO}P(r-Y&v`1{$U;NQ)@h
zC(Yci+)?Ao%rG_ykk3Wn0Bu%&D0Zr8inl1u>f}{Q2r@-h52!YVgDRtkwD%x+0cmBB
zeyANrm?BwQ!89mRfFD+(8SShU>K@T~D9WC{_VA5V{x$(pJy_Zhpgl^f7{+(6=H4pd
z%TuaNTJ3uFbbI@b2B5iU*E8*~Zu5FP?9<SBJr;Elp*;&*{KDAPs>3LJ4sL5!xuSgD
zVT>LEcmkx{{wGJt(BtE{fh=vMWL&)1UL`qRB>9r4M{<ohB6$?HEZLGi?w59g{O!a4
zWe1N2!8pYyvmSh<9d~jxd2CI*s=6DZ#)i=nXa<y8V6UAc907U#zu99LZv--vdyTYL
z#rq;s53~y^xu;lF!+0~$veu&b&i2FczMChBLOARjk$gWJL}=@~Dnj+{63H;=*=0sd
zgbGLJ|E}Djfm_43Rv9M3?;|>981LCP_wq&i|9Os{A_FlC@4Nc!h{P4a(Q_aT<6kPB
zhCkrVa&MQ>p`kF#u~5R+4+C^N(m;idIVgkMIsK6^E<VyRY9gPH1C?}VY1{6V;G1f%
zOz_1;d>&9(JumRtcYAw+FTwYP(wghDn@HIaiM|-?D=itGWV1Ub`g&gYwFA*ME8VVo
zIP<~3DbP9ui2;+fQtSPH+S}4XXvK_({u;(N_R>UOLh!dz)n{*t^hK-h_UugbeQVnH
zr}=t#d@Jp}X}-b1z8XW#s<vCxeCPW^YTy!^qqprdj8L8gx+{-#4tLq5%je$2{XnZ)
zFD|Z+d#zzuS*@x;@<})B<xc$u>g#(mcOEy=E2#;@1Ap?6oA<}jvKW*t<NJ$oU<>86
zG4SUA%nbl+-U5J=a3GxW)q(Q)rwqKXKxOVu`D20d2O!Y9{J+ZI36w9$H;jnPvi}13
zApqd+Q?>)>27W%?R(=3)5Z!J2lXBDZS)jb;u)QJOH{LsRz~DVc(tV{}%&P0Em+Tog
z)K?w6=Wio@Eh_BFIZGDIoi%6f+<8lu?AbiZx4^TfbH4A7L3T=kFLuwv6MZ+TJtGQy
zV}jz?ADwsYoVnEtmd-P(=3KjA$=q2>=3YB*-lDmC))e_RSRv&LmRvn+&=5O)rtdn-
zP2}Y4**?=}dF*|qzEyh)F7f?f?J1e-+ZEhpN%ewR*Dk(h)>VraRSz1rCupH>g4$Dj
zweM<mPSKFTdzveKUwQZZwaORLY0t|meBPiv@A`d*l>OMvzU%f3yv6tDAan6`)z|O&
z{C3}i9{Zc?J>B=%YkaSV+VeO13hd;sd{KKI-{?CTyyt=YePfdCFP`%q+VjfuzV#t{
zDqr>OQhNrz=KIaNXY`xCZ{qDqU;5q;T3UJCqU=36U->47hV#1+*Hz7`zIM){C6#lk
z7cAa$!}q>h)Sj+qd=;L~1sBdp8aq8{@PHvh2MpS?^e5j0kICnLL+wq!_+o=tZ-tGv
z2ma!V`Cl`G1hR{i53#%a;!Ct|rc!VywL+a*f`I7eUwjE(H8gk6(O-OveExo0qdYwC
z;6d{Z;=#Pz`QlBR4W-k~VV*(l@!*Knx(xMD$_NHN$>f{b3(d3~HQmgqRVH6h0o)^~
zCYZqyYaMW%W~Yo@D~+|pr=|y@7lj2!)H*F{u-U(>R<5h*qAH3UP~McGg=#4hO4|RR
zsifXM9A@S9EIDlQ&QpmwVXLW1+MopBu-&h-HDvsZwPn(A8S=)t@ykM18To1X>1~CV
zsq!{BkjWp2X4J7OWOet?G;`VzNgR%8q%+MS!K(~Ddfc84sqKgufUcYfFZ}BUk2JT+
zh;M^D2V;AaGowty7hG-JW~6L3eW~gyTJl5x%nfDgVj)ZDnz=!wvcOEc%mVu=Gk8!N
z-VK+@+2ASxbZfyHx7Ie($Pb{-FVLt3W?O-9nZGtZYuT$y%1k~gUty;ChjXnB+Ykm5
z%ne~@!_bA+?rWpDt&LtEP2u|J4bk2ueoCVo$Oacy_KK}ZAQrV#^Q%e~dk9`H)RVy4
zs8W-!CI&dIE=@CyNL6V@mIrE=9<NLAU1tpH<NM7Ie*?5pzZ7f6LvAz826Zr&TWyb~
z9^8)is!NbYbbDLl1JO&5#d`9n@AT=?W3bokRA|85SnlDQ4X;!Hid2|Lr~Qw%E(1Pk
zfGRhWws^X%GNz_?>iS#UDkCK?eY3g6vyD8jo9ESe+dKz6_8VQT_}CWj5%2G79Pu`p
z_E(hpHI<j02n+VMs=p@5;1t2CGM@d+mb@ki+^05p+&Dza9}t&wAQB#c2v1Xs%_g;)
zUs>r`o~GtHx$Nrd{Pvn6L23=I1fWI1@Uv1|hOPX$_*a^twZ6?7g%fM^9QLqAw0ca`
z&AUj-f!M9!%|1a@Znmhyg7LUIA*qv!$1z;|3~_fDOA80>qY+lW0Uc4z(~T*PuSFP_
zX9ak`qrVPSS@zIwR%YZC<@8=%Zq@`J3FUUYTV14G)y;~!QWpqalmyHDNH8x7PB$Au
zc85rEcL>izOgDF1Euq|`p4KzB({ow{ZK;{rsJI>ptqoJ90oIwOu;t1o`?GG=*lu(7
znZ^WFW|q;pGIMR%-Z1;TNGriJXP<pZq&3#RHf&3ny42hf))eMl!FXr_LS3qHfn{#2
zY1DYASmW8?@i5Q3VxPsxEJmoF+e<c^2d%nJZv4N?i#wDTJK74~SuLmy%x>~qtrpUB
zxu(&IIaqO6V|Ahdq~&!cGymEs(+u;jjY0*T5i*<6nPu`!4=wqQGFFJ4Qzo=kUF4Ly
zaV=cvnC-6-Mpl~5z5_m?qXSCa_->8UiE5puQhBqcgR?m3kdrA#<mCrUJ~SqS9OYtI
zn<z&4fvrKoTMX6r3@x6M@d=d_J9BR3tBM%r&{46R%W{mxbaSgZ!VPy%@SHkpB4Jnh
zS3y9My2Q+8BwV61wa#6&-{iYNC1zF?m#SJ$*j<%?W~XeXnN<vh#k0K2&(Jtk{nDQ^
zCJ#K!G^-jk-GNr-g=0-c6{C{CR|-DI__+;oqwc(TX>+<6p94%z9<jV4Vny}RgWc+k
z+|DSDa!U+42z+={2cgn_0Gr^-f)1zDtXQSo*ebPJd8?x~!;eFOyhG}!lXq0D;xma_
zU)Fl+-Dp4{^G=vX43kbwl6n0-{x*{}Z?%YTiB_W}BSep!u3Y8okl!IYSF)=$IAmyg
znb8(+v)_!dV)`xKW*S{-q-!iajV;36SRgP=Q;YYaSOTtf%2?cDca61rdbnrBT8SR3
z!k!XqrKeaGrZEr%0}sj!QRiv$pge&;xB}eWz<I0&`&sIzTa1EZN?mN85JM<4ENtx*
zTPmu;bfu*-<AD83535)5%rKL$pkHDx_bb`W2C&pu?hqxyn;C15jI%O4GuPVXan_}t
znOp7VIBQVyy46hO`@JW;)^$AJ=#gZAAYCob?$*;9GpkO(8qI@doIEJJj14|6Iubu5
zje1fZkj2bAra-gvGNY5HG{G&?b<fQ*c&4VU3r#((WY5en`%q6S#Z#DM|IpLAU`%7r
z4r)jr3X@FZ2H1B)i%0X`Y~dq}q>p+Udj^Ez5U+vlo8zt6?z*VroKAuNaf&QmG@l|n
zd4ttseUb{YzmB)M*gwWw(VoI`JEE7h+20h7uv@|p%I#qI@o?{2&t~&@_?8ILTOyhx
zbb4Rp{zx6!*8Pd@I&!A_8F4|*MAyccOKQ|+^F&N-tP^XF-QB~D#T|`9mMD#kPyU2%
zW|4zlWPZeZ##9&k&22&ZB1rCwY>wo%t^0{MG{H3`g_X{bJgaK?`LQ+>4DwhUpOzkn
zd!P(1?Bs)DhfB=EF|9G}zRhI()GT;-Fy<uV$E5=@ocu~N?qJN}80^&Hn4>X_42-Qy
z=lO>HLW0%HbG~UGOR#!Zp3jiV0uaMcX->Ed&7)gRmz*M8bg>x|Dy7GwpC!!`phHip
zL_O89(mMZX{rqb2?n=z~3NH1kx!oQ~;d1*AiB=E)`OC$g&oE=lxolATl$zlnK7UJy
zd`i(p#h?{y8y=+x1MOar6veP(b?*WQ8nY8mU4zNF3o;#pekwTlfELm&$T5vfsLGAQ
zWdGhuIuN;bYLYd^b3v|Mo@AvBDT=RiAVF1>v{nqgi>nBQu4VyKO{OC!ji9K}ekKXC
z!To5G6<c1^C{4~V^9pqj9T?{NSjfm?ks)jr(u$ZAWP(d$x^2tr2&dUTBbL^L@BC>I
zQFBJDU7BpAl<2Z{@=gisN=)AbXJzFM*CNSAT;%JvrCVq615%w5mFBg6YcmE@a_zq*
zTQUAA7=;4#tP!tF>hAaOi*$~@(Y};!mHDGlk=>@qQ<&}};m`6Knc#$8+M2Q-MU{Ii
zpL!=xqO~Gi-McAAG2pU?K=y?tS)L_Q&DUBaiGTuJScVhofM2-5o$Ca=o@RukKwLy3
z@dGB(k(ontd1|KJlw!quxc@E1O7cw2vRhNEm^o8Xv~svr1s0hBoj*D!1^LqF{6c)+
z!g7fU&|D260LXVXa;sb!=tXgh7ORLSAfn;}c15a{JgfMiNLMdhiW_kOoVk`LXh8_Q
z#Z9qr%Yn#chIS!xiv+^Su0?}!#00oDQL<fyoW@>w8iRV8X(y#w1LR(uX2p1>Ew>lw
z{7QRu8hxIYWIxK?KRq@rK=SmmVsOeIJ(@n-G=?B;9E1)wMRF088I>h-EM>0DrXR(N
z2}}mlkD7*%WF}T=MS#ZX#~9&G-=-gr(V8H+XEXf)3?=3HZsWT7k_u7W=_UtgAd1KA
z1PCh->}XV?tI{sLlI??Dt?7DF*LWQ03J2PqL+G88J>N{UTY6hz{(@6!qWq$p5^c8$
z0$L(7{z7wLC7MyWT#qAj`Rah>a3OTlt7ZS9L)*G$g*i~?cW7Ma&v)~gy~;#*f(ScS
zgj-Qk?m88cx6(8UVMOt4q0o_y<mF>c$a9%muvYwGrvqEVoZ;i3%(i!=Tipu^vR(G1
zol7Fwk<2rZlHGwqoju>lmg&`PH%ueTF2GUI9sV6Mvi~<^VRmtb)x$F>*1kH!N|XD#
z3@bWrQmil*_M8&7wKq=r3zYYmlxK3H1Jk!EoGoPHFlkc0-I`%_&z_X8i<!dJ0WrvW
z;7lqOK6u0lpHyd1$%!5#x;{chivK(s&6<roXS14|ZXQ-g#S>Y$85iUXcFR2DY4JK4
zE#6v_j9PPlsFSfjba!Vmc6V;-sxty?5sdRuz4<!7UH(pfg_|!O^gDcX3g){7E`K#P
zx4cQ)(i!G{S@JS7dbWDF)v)B}Ru^=<6SuRSPIS@Y4^-6h9^EWHWQkcTRZ7fOPpy~R
z$xcT@xoz-=HHL9-4m-*%piNTcq`0WHo+DiXF6xARv95I!^rCn22m8jrcEJ&rgrX+p
zKu8XAh*_%|#UXB0EjYwxjd|GOb~N-zXEd27MPPfgp=Qv3Hr~{jgQf2B$nXftgQCte
zF|OXFxLJj=m!pl`n1i$VQe5k7IH#)}m)Qzh*lba{;&Jp));vCjFLS2eBFdf+P=+zA
z!rVcTA8U#RT_z(uKVEP91v->pq`QBKc}g7%!TUaF9SC*SX`Z$T!-bg%OiKuqwuBrB
z(X<{3q0UTQ$DR^mrTNEBkV&1j1#@8~n#;QyON<+h<jv-R_6>_b)@sjvY>t@sdA4{0
zSzA2KUb33K8_Yn~29pg)aGAT?vUZy-lGS1!3<^LU2{{^~DlPM9$YD$J>>opLV8)i&
zBYf7!UH1l|pUojHGHP3V`|N&}HK51XGF(qtkJWWG<I@<4X|tV_eXV89%%n~Qb@o|I
zov!f5)|p0aki`$~)x^oRq`5X^wPm+jR`kW=x0*&3`!coO-Cl0Hz5BerJB%S|%Vnc+
zpLe&ZcR)2FQ6m9o7BsV~x>d()P$0`qtB%zXcy-WwOt^5&d)&Ufla-V<p6S)+sWr{~
z37%SySzN&^e8hXqY@dL~R@)DB!busIAJ$Hu-bhyCjiL@&T9d}dX9wJe@fG16N@W{Q
ztNHld&=>MYHe0g?Y&NrVooJuUcyNJSy=&11oC`_0d43Oj*f@EvZ!)oQwk8p+o4Bux
z9Z~I#Ox#x_e7^H%nv#+OMeIz;AAFg3R3_MRGYTKvg+7`G>pU_5@+;5T+gABJr*nlf
zRPwjfDlZ>e+8o%N$!}p$>1OUU^UFXMEN$htl<B;6f%ftbqpzN|h8bI|iPl8uO!^ZW
z?^!GSpyF6F2`V)CL|yV_pI(j4@35;rQ!*y-2$9Aw$c(;~;`%N$Gnv55*%blH$&$No
zqaL52H~Q{1-H|R$NK#N}vjd7g_QJkauMvG#2evqlE6p*1WS7@jEbFMam~Ji`?pO(r
zCc%E8ua%(s*vI->J%(qVmHn87W>Tg@i*}<0EEAT6i#Z98NBOXAjtj1PF}2Cg?q~HZ
zo!XQM1Uz?hWwb9aV>5XeI#IG|91t8F-H)jU!nFJ~aM~F;PgrQ?qR0A}!{w1Qon}AO
z&+4tx?AM&zfBIR;BYWc~m|<%TU$(5luV27GGRH?rv_cP*_WS@QeFt4{`+_WMpnnzf
zNFgm2mO2fJyJ(OZx`u_n;JKe?m_;xCD^qi^Hb2pIrj6SC_H!KK!4b`Tztin+ZBeB5
z;+;+&km+He8HQ`%m$Mv=DeaiA?NEa_?D3*HMt)OxaKurKNgv4MGe<CTbWsahb)-Yl
zf%c-q6rJc$bVL^g3Buz>aKv$^h=v)=Fyq!DyQaSt=i%Pa--?}lYgwk2EbzB(5z@7&
zZru{*^|4bPNVxvDaciT`lkfKQ*2WOPoi&tEd&K?)bXV7&nCcEygQ!nSDmc~W=tAv@
zVy{&bVATIoEOq&FD7|H`Y=JE_W2%`^t5<f^68dq=Ub|)hd%@hd46u4lTNQ2w-`e4H
z=oU6Ouo=R=Th5x^wQ|}ZiB+IqtPO~<DojS4lN)ZhR!7<wHnrGY23mbR+(!(w(i3lG
zG}I{>4t276FE?vEbslVhRw97>1Fh}?ym6p4$bajxk^mEKJ+?f+gfKxZxNj>q`7HZ2
z+HJnZ;mobasycu%FB`X><QGbSCwg1C8MMTxl^@u4le#;)Z?~;(>%(npwd9rW5~|dj
zibos;(QOTqt!vg+x5>J8Z_j|NYxg2+gsNptA|UJ9{p<}Itos6n<CZFWbGDUu@hz)`
z)tt+?EWL$>0}B%4=54YeyU@&QagQ%$KZM-<!aS$5x8#UhiVOmOUXiD1cOPW+AA0i%
zlP8?8BDL(D(!LhY&8?=0N91?&DMKaz=yLu!RhJF226;g7=pZZCe@mv%Gk7!QfD@;t
z%ooNj*=<N}$#I+>$t`wzuNg!l(ED4;!Zla45xb>KSfN3N$xgFtScg>|(<D*?8N=v#
zIhx@wZ&ZUt@fT{ll&@{e0v#_*OO}GGbt2bERO&>sYa50gVEsN<Zz37_XJre!EiT8!
zVrKYr2FU&>?HDh_u{y!8L6w;&IFRuC$v!sN3ia%dvcDc|C60eNO^ouzX6%L#=Yu=+
zdH>%1a%8M^7bKoLoc#sk<-D`O!56PLOZk+pvxP^uURi5T9b)zOaKC8?1MZc2d(#kW
zWS>{I)OdnttT$)s@9t_2wB(zH_`qe$U00myE%~-5hdog4={eRly#*#!%@Y`z=Aovg
zSk{=1cj=<B_vTpR{Q?>u0M**r(h%<%=5mU-NLFKLi1*zp?vk8cIyoz&yi2fOP6IV!
z3Y;j1O>-U1XsHX+3~_xNBQ?yO2rIlxsyCYzd+ilNt?-Og_*aAhDk^3cEZ)HS!NHW8
zRp)Kj`c!-8P%A~H+J}c)!#vNQvBQR0nQ|ZI-e=3bx!7Ja%<38bLIWG+B4pj&7Y^7@
z4zrTVUd$D@Zb_Qy)6=Ak-4}DiP<BoEi}|K=&`eUaEhXHYY;(PLEZ=PA^2=ZC+xICe
z*%D*WYoA?!?Losi2Tdv5k00(RR7z%&XN_x@Q<)rG6OnR6TGV4!qAsr_>he0}I8(gC
zWQn@erJRV;0GhPq2~6#sHi=T{KQkA<pD#4q2Zmdr7bdSXjT)#k*D8*R<~mxKJmG+9
zBPaP}xMx+8V~>)Km3vpsb_Gd7)9Ik4Ut4zDy>nqK_o=y9>j(1f8*;7m*awct)KO^`
z&eDPxc%UDgU_X`1HsphA?SJN?k`K1nzvi+8fAFN8cAk|!<iS%MiUSJcAMy<$D$8Z#
zfowxtE;rt`$OPd%C+!>0vwHdOMMq>GD9zcCYa{Po41G9$UU(vfiABUJmusB??Y*nT
z0BakS)EE}F#@OH)(tE02M>y>!H5#5Z4yrbU58<C`XN~og9I`cP8*BF%Va55StRHr>
z-!@sXx2?O4t>|@$%%G>*H))KmS@^^5=rv3B{eB>CEpj@PgE5l5dFb7m?;O)QQRko%
zJEUO4R$RGQwytF3T2`b2ax#o%YanXo=cj4AB~lm{7-tjOXLL6^9P+~@Td3EMFBoHM
zM9mkP?D3uhF8N`Ahv|Iac_Bv&dD3p%Ka#na`!x5i(=*RF`P_BBo6eh7D_vb^M&+J^
zU$QlR$p*c)Dr9PrIDYiz3j5cQR<wU}g?O?H&~oP#2d4vL=9uWsTbMt^Yv3H4qk<y6
zCNE-eIy*GR=B+jCWk5MR+kDnEJDCSvT_+`9|Ij5{!Z;Z=-*!t#$)_MSMO(@;hs#_S
zNVJW*e3})c0JwR-zTFJAfJM0Xr2TrH)hprNlVY+Lm^pi0`vLK?p&*jFFUbxX#WB`>
zX?Er)E73pgyA(ZFxpdrDE>s2RCB-ZH{L$!r)nOgzB_;IAA4u<QLFeJO50}{Sh}*L9
zKAk-_S9#qU<4?#z%tF(CWfVKL<4@QpAjUJH(6&ZfBjrA6G(=7)vKNeI7khlIefwwz
zH}}7cwz9^JM*`u&EE{Bjo-^K!IjT9nF*DfvpVo2pE(1HHHMv?{>+K$6Xm4D-Jz@+y
z$>UDhmyEFn%l(h!$BvIJXM@3cDoIZzx;xYN+55*}n7IGjy~pNTeR|1ea4CH$Ek_-y
zIpOKBG)%Iu$Y))aWZ#%?#SczGtNO9!z=K~MwgZjnIi}GQ1U-}5*{56lbA|mzzSYzJ
zFlGP`rNlfaXNgQ!=pt-?m@$g&F&jO5^?8Jo`-o4D{b!gBJX{k{!$-nn#R|&w^T=BL
z+>}pb`bbs0xvs_7%)3N^)X{TN$8>6%Qy9F(P*3tQY8!C=`B(r)3Bc{B!XwQ%OfopC
zs^wa}GGvV*KZCZJjk6zAAYnf`7O_6kY`2Vsp^qH1za493rTqC+yy(52sE7yjp3xS_
zv-8JU(JIltc$}5UZP_?$enBE!5Ere2y%`M&XOiG(64OyqCUXmv@n3QcXIzw&pp>&K
zwD%`EL6IG1_a2W%aG%88Z(}ODV*R?N!QY}%%{&LNL3l0?RWR+ci6>5|9k$$)BkCr5
zeWb)$=2$@-rVe>kNVtlPOQ57qt$3^AEjX37VVb<_!Z({?87RnQ8JUuB){yo#&ehtZ
z0R#sHfwso=x#6Cn`1&GVoAX$X>vi<3c8DFrwy|DzAQzhX)%Mt!PTf5l>+P8ntStY=
zV{+EOUVuz<dda%6zED=OF6B~B+Y)x@>8(e4*oLdn)mQ1z#$%5At+`ZBExllH4ilu8
z`j5&|Bxku<6>zq~Ruiqr9`#3=*uim+FMvZcrGBg3f1=ehudNF9R#+x9w6zeJafF8Y
zqq=@D$EN%NwKvPIooJ=?*jwd3@U~6KwrxOJ_g2|2Oyqfky*2ii6RkNz_sX6lH9_@M
zn&UTUG*6Xjr@YkM5L6!|`?{j+{L#Wwr|g<Z=<wdX_M?-mEYH)G_K`_=)=wX^Lkg@>
za-URSWy^hefi=eeOj)eg`j2aENw5#AZTd4EmnI~%dm@Ao2S&?w?_s^gE3)0|><CEo
zsR~mzb9EGjaDBd~nrY_&w;d8p`O=Qxc&fR>8i+(&L&O7q!9FA0Yn^mgk0j`_9+dkG
zpI$!&5>hu14{UckoDtUewdyQ1b2x*ZZXQt|vc|HXEVTN0{Pp(7h1Pi<{|UR-WGii`
z|A;Kau~}PWkDUW@=}e7jiuuw}^g{X-_yhS(@%GZmxaHg*pKOir-vkQr4mntoQK4mX
zQ=`d?5DeC%>ZBg5?b!#UHJvRJwTAXb$u8b>GasMM8CU)&raTWOP>(WFLYl}#q6{(;
zLdkv}B|VRdg~fr2Hyz`wzO5oDt#VY9C=FU|Gv~?^_LlQeAMUT8kK1}<tlfIP)qUiR
zX(sPPK;sE{WXW|VZaf~Q)l-vuW31lPFJ;4?$0|x?yb3iprrG%ySTREdfx&n;b_)YE
z7CCZoVRUvIaxt(@)wLH`z2@CmFKYz7X?A0pBk({%1RjWMs&vH_SX;dbw>zkP0t2Ds
z<7~>ZUzfNpfG)AS6!HAZiX1z=h>87%YWw^mYf$tJ`Qo_EH4n=?=j<>WHx$|HirCy*
zaln48h_&hs`SzPdR<eIZ<tY<}>(RQRN}lE5Ph8g()usW{W||vJ#<R!?&aYTLo#{bK
zZ^iQCdK^ji4Zap0V{sI2Wdo10I10CNOTZ9;FL8_OHpbJtwmx`_M0@cRE0$>}{6Z_i
zeq@RjaY-WeDhv;&LDZ6ye={g3IM_|{u!gbODdF5;cv@<W$4!Pxa<!2nIZUV;CYGI|
zHZ{?044jd;MGnn%e-b%vaJP~qsyi=2OkC{}8{3gsoozC)n6c*S8XjE_5oS5?cEp8N
zpXm<GtJ}#Z+L8R#MjQF7jdRFfZTyaW5Q~L|=Bd`eE{Uu0#HBGOZ^~3FJ~EN*D~Dn!
z0CxKZd*xJ~DdWC%s+BT+J7bCo$1L8~=2dS$CI(px@X>5Dc#GjZ^8~^YW|8*#o)%l@
z{DJV%O4};7lBPUbX&R|$U}}BaQd&3o=myyVm~NI-1;*^7oIMHB*8@m*gLIQd>Uho<
z`Rt{|R!Zoz(8dqi@{HeZ`=Meh$Mft_`@LdJJ@?pYELgcunC9fKn8w&9y>%K^@3|!V
z?P)CPp2IG(hBap&^QbGe3jo2-=gOLBrWrq6bB10$pIN~2TMPU7D!sWTeSLm4Zi&<F
z=Nrsmd|=rh(N^;LeQ9k)zSw}RdY<T|E}AeFv_?T2lZNaSCM-0Iro*A>=S<a5<diOu
z(h8@v&4t&c_RfnKjf2<PZ(l?wx&Q3mdrx=LQ@KYB;mB+TewS;K0b4!YN>{z?9n-DB
z{Fvv*)2+VTI+s|(xJ@Xr`f;l&u`;>cQ(|Rvd%c8ng9N*^#2U?Q;0&wZxP)*+`zSs*
zLOgs`PVlOv;EYv1f)Sj-5m88pTh{G&WrNaW&?VRnGpr2I?Vn*4PE6Qphy&%(Rkj-H
zqPF^5ji76dp-z>pMrYztQE4%WsY$)r+zSqIhLP5&1bf;{>%2(`Er#bpW<^UHX)!nu
z3-<Zk1RpH9&_PAP<%~fMW0F0%_c_{|VIQ1nB}8VN2-tfO8AoU9Yi)J0m2&?2+QPup
zOCF|ID}>r`l#K~3YPI1cN3@JNUF~kuH1^9O`Efa<l)Wzb1K-`IeGT|L+}B-f4V@x@
z@tEJ-|6qL`Xr^66C$OwtP}mt}qxT?Z^dR6M?CuRLd)ZTAl2&)`TkEVh9PoX1WT`c5
zsGZwJlU=Bz4m!IOnq*0HSh43Gph;J?OYNIV@yfXWS>F%X2TQF>d)J$K_g~truWiFx
zzd`qutxbrmZ?vagVs-CT&&+?Hmp+{Fp5>+oncELQZob4ypXPwD{ib08?|Z2F6CZj4
z{JeL+ZpBN3_a2t*aPf7BKCi4Lz6~C|tZ~OLx%Qb$th9;uHAV^Jg(3GfGnKVPttC9<
z;8YQ#rS3Zs*fvrPvva*m+MeVkY0tUT>V9>@>_pEJXTy!IHO$691mD3oU^qRFm18R{
zfrc7c*@<hX8tTIx$K8o*Rd57tTnM+r8R_9Xfy6@y?X$EnoKpLvORd<*3v0USIYlbz
zf+;KQu*<CU)G2K5@sN~RB&*HkM?09(cEA~zG3u${r$WyuC*9>!5*75rlmWoAcJpN{
z{U-0WKe)^)kb6cMZrtQk_N+21R_@oASv@1pUzw#>b8fXk<@Up6s08=Fm03f$@9|!4
zWhu{~CVS8oR_}trdBxgTy71{|<@Fx@(iIPT22L$H6qeGPQk~z?sHNNYUtuMwbo;q0
ztoW#m<$BLnY9U4K>-Dn#eTCI8rdM`7$3@zmI;SvumX+=AmCqSR;I!N51?O{l$E2*%
zdBa59*<N&RuH5&FTPseP_FH<b<@AGbZJL?tDl<1%AEs+L2oiZwMI#Y!tu5DiHblm3
znchZK%KmVc<x`pViCNgUOgnrw9m=!^Ik$^uV|z30>t|d2V=`BF=w1M4PPYB*Y&>)B
zXJ#W%?kRIv)^IPLV-3#8X$V)d9Ct~gSF}f+N4GR#IcMzcbF8GnIcH2`i&C@9ZHfnv
zsOHe^W?$iCgQBhRGxpDO*!CG3ZYNx6^^6(H<n7}nl^h)0%L70|W9{NA*|Qm%WG}wb
z8s;CGglY@d!k4SrW{o=NQM21ob}XIYl!_0p$sAfPHCK7eyxCmIMI=LynBl98JUNZN
zC?v2m*2Zy4S-V!6S>@iFZ`usHdOrS0tc^C*E?3Pl%c|9!bMt0N-mG);u8_Pd&dn>8
zJa@G<^sGqN5ONu$L!0eC&9$OO4Q-Ya1^UapKG$3g3EEK>TMyL5TTNp&`Hk|<$6PGV
z7LR^QO{2K@r2WxcD>6=imT?Bts5r8i$K))3y0YD+oax}=t@hw@YrwQpBiy$xDR^*N
zDlf<BRcfhG>|19n!Bb`D$7SQi$6Yp7u2#ZFy}eOdml^y=X9vwI*v0uPt+RKQTZz+4
zS&S#4%`DXxJC<1*m)v%3rVP`m|0v6@lV0&an|=oEaED!EO?P&kojMP_;XY}em6#>q
z^Z5egd{%kW%>z7PhQXUJQbNMnhpXE+0J0C;>*iUpnc2r9-S(ulfbr<(U9-}Wtq?*<
zy#4k(D^11Q-_NrW=HBni6#3vL%6re5!QwrB$ySw4r^%~+x<%D+CX^W^NU|H6+p7@Q
zRep4<G#quSNMq8R3T?8NU1g>ELyy85o@kJl7`(&w@sNbC#v8^?Yg-2JU=OS1&6hys
zuzF`f%%ewS>M18|?{Imn#h)%MN&{gB#mz1;3(G>PjnG!ZJ!w;&W21tqjT~ewQZECp
zP<o?h8RE{`)=edbYLxCKveQm0Td=L3KDjOfI_EodL1JiBA=8!f2SfjoZ(ldx>h0m)
zFy9*P`AdoY?tCjbp|LQ{?6fkl^8U+)Txnk1E$TED+EEoehxV5Prm>QIy=frjqG*Cs
z@->ONBtR_VNN?o|1pJiu20d}H?6=7qIkEXVxpK3aU#Vj?@=O3f0OkOJ7|n@a85_E`
zO$f6b%>doV{W94tWUtVjSH;GUJnXpanxv78v5`QG+~20U0>Z>N(+CmgaAc{W7pl<c
zbmCx+9lgLx@gHOfJp=-W@Ir!F>2I5!Y1S#KFo*D5vb<eI<N66x&>gH`E5sRy2YE&m
zS3!2iBpPtEsQ=P2_uNT8p_=nMpFm3ERowqW+55o7SzLSL&$BzgJ`W^Wk|k`yCfOvL
zWJxd~iiSjDRz<~%1{<QpV1kN<M675?B$gIzu#$)gY0#k+4Sh>{X-h3uTDdK4Y325!
zr52Scwb-K47A-2;)JrYu`#opoSvK1H`@Fx+C(QpdGiT16IdkUBJbOi89cI!xN{(GA
zpBk0V?X~>No`Q~1gD0;uw9x_n<$z4whk=EgtKBc`v(BY(*FGz}{+C!HR)M32$4QPP
zy&bhI0**qczk3L59V^Jy&e%RxBxl`fExPn62%{cX1FZYKNaWFZs?5t013LOtg>c;w
zZNkYCfe&|<ixzh~Ebc(W^g}Htt7XNlxc_vrQ9f`hxZcUr^6^`(ZLX8kGQQmU%)$rr
zh_7t|Nfh8Ob2~2gV4mDp4q2qBT%Ihq@?A}p^3`%{wg17PZnatE3Y>v_a9S?E%}RGY
zI3cgP%}OnJZ~{<43Kg}^bvQKTL98oWRb`!dW?hs3nY*$Yf2!+D_W_u{IX@KqVUGOa
zZCIw^{^>Sr<?0{SDpK^5c}*AHHqq*WAHwd#861>*tWd7|9K<lVS8Dgx&shuokJZEG
z9vFPA)3N^GE~m*yK8Gc@Yf$!3Zn!VQy|oMMBZ$pmumrW0lq&*s6ZLmZX0p0~cLo#!
z;Q{fao#~py(1D^onP5vi7*4DJTErZ{o0F-CF(y^saJ#i2`G+25SWg%rP^G#(rMf+(
z|7&~d&S+0vY<uedZF>;=|Iwa0)t;`FGurc&TzOpu_QYSQkat&D%UwSjk*6v^DRBEM
zpxlG|wsx1?X)QkI$wHFd8DUQbVjGAMc38BlGRPan-ID=XaVKs%z$JXldopXLe#Xd;
za&(P3y2cz`V~*q+f7BJN@kiCMH6rBHNZx3TKf)xT8sVbGg>#!eQQ?Um`NKOgc}`5o
z-`{Bk%ryD_ozTstb<zAJA;k+RBoLv+RY}lkIr^}yib+$CB*pT{Y$hgZRz)nSW)`Ul
z;|0CIO7#<Lic*81rr;_N&nikKE>(kXIDtMWF#viBtU~>05MnCTzLAlY=h4S*O;N5q
z6R$r#BGQ`7_;0wsnYs#KRS&Q<fq=^eUMN!cSy5A72Gvw90_V7o)|m$GxS+DU5*(~h
z2OVB;S-U)|5{FBd^^>lrP=v4>i%CUn0Mk66mJv#&T{itt!(~|2u+bI%AgT$@Toy4h
zAnatJ^I3G&ldKW33}R{j%9JpQk-roQSgjK{8K5P}q{k3U<yWxHlR;9F-6V3hF`n4w
zb#@`yDIw<^yi4vWp~W9R0myUmp`&$e#5G)~n?$J03r5{|IC0uoXrluWXfjv`nA7T`
z!#v0D9Ol_{Mc|W>1e(Q(In&TLzo;b*3Rt~L;cU*C^td+1_#JU%&J^UYmz@$*w0}~d
zym*t?Gc*&R1R568yDrDUEP4ae1c;VUjm+wuO`}0MQuA^D6^&#>rk2U61DMp_T<Je(
zZC>I9zR51c7ch%N^Q`uH(Sr{_OC6P!2QkwZG|8_Wv@UfmXp^rUv@UQh7?sQJvR3&Q
zU{q+iTg4-f3RD%H@`k&tl<oeER(h_$wK}4k%@M75KL=U>#!oayp2n;{gS98Dn9zQP
zs*n3kf0q39T~>IrA96&5CM3+u*PjykBsd`>I_obA(eV*`8)v-3L35`@<bceqg8tn*
zEc4+8y<@()hmEWiI3C}$_^`2s%v<<AF5z|}=?b~%kahlbM~ua`dwg;G+D+|i)7l})
zaSaRe=uQ`3;#pYi9A=pdOZa#PGK%RPz#R$}S4=OCxG*3bJ4P932Ake7nRW=P?!qa#
z_mH*7pWI5Eqg)Iozcg2~4U9u_w|w*v)Cp;Z>^@|Lv(ivFk5GGP?<vub$BJkC4e{*%
zDz(sHg&ytn%|NyE-3?`Kz$YVjTWMDZd<AF?<2m5l`tMDY)j5h8Xb}c>9=MYdR~wH)
z-0nqC*<kW%R$wj)%8t8XWwa<vvv<1~6_a$@6;IcC0xi{~jg0y+Qc*udYRB*P)I!gx
z3lOF-J*o5*&;fUm|4eZVeOe68l?Gfvkig%1+@FzopT`~5#Sx{gQAPp;h%m4i{D@A}
z{KS*-^kz}?IKo^!j#dkN$Cj)wz<n>y<-nO4vfU@<!-nl*@TQA%$!{dm04^T?q=9!0
z4U0oTOZvCW4C7o67*kT&Nr92O@AOI6Ntfsh{3LKj3^$h>oIFgIvu+dfuq6gY15@Cp
zPG{zJ3j-qNcF~;HO3`|9Q}m?VT@6y1J0uTPTPrS@H%4q2$7%~kmZ8t2q&4UDsAmAW
zn3V%JK?pIK$dohaQPUsrD0)hc09MUXK`^cn;bJY-bL!G0Y9C%`@xgR6*yYiut>)5}
zwgmW@wzQuo%hJxq1WZEZ@9lD~;iI0VSPXFAbGzt?>x@TlT=<9RWk46rg;g1?@vQUb
zYL|s+3sEk>^|bWM>zKK0Qhx0VRxmf@7o?*XWArLQ6f?_#%m~`qY2udEwV+{!MX~ds
zIpA0_!R2~G_41=HU`H0}&c!q(BLXv2pP6{C7kBUd@ynJA%q@=WrfJN~KhzHuSk!w)
zR!(0&=5ggJb6?=cr{B<^u~^!P^J^qAKjZ+W0S9W9_QLEpnnVF4;Roe|xeP-hfCT{y
zEG*B|oMj$0L&)n_yEW~Fw89x+T<mzW40?nF5}COy0ytQ<$(zfJi<-5pEU6~SQxR}C
zcr?#j?i0=^!pjeZs4lkCjCZ{5!+s}!H`hu)m&03%4v@DudcT1D;&3l`Dy_-rn;OAu
zlQD#~>IDWB9guj=WT7h*kOMW=3OiJ&4p)5vc|onUTCS+I=D0$|^1NEuSBLOb<HLgl
zlb8rX`Y6;~;fbPV1?;ZXQIbhst#)3K=_TtImKv5q8^RqY^eq*%sm0^a@a7xwclqLp
z65KkB;;Kj?Mg(8aVT>=uJ{a+&DhY!97ddpdG>Fl2Rblj!+*JTVKXlyZQ}e82S!0;C
z(u63V5hf+a-O;(aqKK4s*T9r7PR&MDc14L+Q3pWfnX>#IYj(|w3N?XnPPQF0G>)eh
zV6ODwXqZm1X8b%TEB%-<`o)Wtp<3Z!XJzJTH}eI`pWSZam{09xcxhQVMY}z$#khpC
z1LB_h5T3aSb-2tty$-Kx#;J-j(KJ{3VS_*q$S|R-D#c=<sC`v4CM~*xDpvKcVT#mD
zZPkFh@Jp~^Tr~vYTW4l3qkh$p;XaIu6Hb7D5G}8oQqm#3EBfY<*&o-qqA!<fB%)Xm
zq4Ct5U_~8`$Uz>FgE1qLNAmc!Vq{e>p6|R?4@qX`nL`qWxSxq!IRo*GK0Tb_Ts<T~
z3|AKPR&;icX3$WFLA+kfU#wBvMW{=0Jv1uZ*GI#|L!<qSlki{-Zujk02nr8Z09MQp
zq(@MIhQQ+k7xx9+RTu(183Xcj^J_)vVDLD^`<S##&F<*LUOnE5_l;;3AW1mAR2Uek
z-Pmbk^mczDS`Hny=DO0$<@jO9{OQ#KuP79W7J4KF7DV_Fgf3DH-6l!_7FRoHFjr@1
zIWsH0DM;D?9-ee{WKQ!cy$q8KGmd9WI;IQI#PJn~r=j3LB~2fOm=419&-hgy<f8iP
z<)d}DQ<&ZWVHwi8%HKW1YSOzsJfWGvruXCKW{HO17c<k-ah_R%?o;Ciol7ca`cRa=
zt`4wZt~wY%$)K)lb%+uGle->5O4v!s5YF(e&Op4w9#=zdbWl|++=mGfT)2ED9&DcZ
zBT}fBt8sJk%htRa&Q>QJysWMRSVg*=s{s^E5uoY#1BzKOM_Jui#8ZF?V|6nK1LSE0
z6Al`Q@58WB;bV1^5DjEX$=F}r%%j2;ws=G&Cxir5KY|U!XH}!ylW-}pmbJ_@_U%cc
zVE~Cls?>0hhd@1fD{yKp1gnt=X1EVb0mdYX?9(U7K9GJ?1aTw-B>C~G9mVl5pQf(K
zZvqLc(x+xd1r}DcL8Y1nP#J@ZB>AT=TdDh2kCR!J;vuUiAy;WqTN5azBQ7Ge)e{<v
z9GIkjGvinTdqxcrUoc0@)&$AE3aY*=*S&_wA)87}Z8f{sgk{NnU|DOzhP%P57a;^t
zw*pqm^yB#5UnYD~yVsP@5ElII!x>IOrDOVIuBmkTMyTt=rt)K*i2{dxPN2uLGA5}9
zNnn$_7KKB!9BR%NE|+QbkeYo}z%=oYB-*uyFa{z_`$nEBzV?7fq$Qde6GD8S^j_6y
zAKr)1B)q1R3^nLS80ujm^%~JEhGtriPe)Q_^`l%^J}|=aiF%kBW(4GG^*Be($dYT0
zSm}X`fG~z==PrhPEh#iy{6n#fI$3c9r<y6CZdxCQ&~1hMj855l1QPIBUisrAcuynK
zFW)?36}r|9$jDLakZWx~{^Y2&G?F>VxYo3}4l-yh4q#7VBhehE4xYIeYX^lv*Ahe%
zy)K|G0ipi2gEDZxwax|i%KNQ*U28Mt(EZq8t@k|C0A1O7k6h7U-Ae9P8mvP#>pOAe
zNzlU;3|lo-#iDnJr41}%xz6?7z%yto=UV(JDg$Y;G`)T}K-r>x{pzH-K59#3t{>-9
z7p5%oa~5m_8SGh|^27tM3O=h{{@?*Cbp6?-oi5s%(IqRB1GG0&0X#>CfJF=C*`>{V
zF%E6x;4l;=6g|zCPMB`bDTKO+hX#Mha^akIdCoED3eV}5#m8_W0{8GSyw{h7kCQc!
z(%0EJW>w3d9kbFdJZFfeQ>iG&#kx|4L{Ko%J<xa+-vl!>LJq|tV~{ddBOkK_6)P7W
zw-#QVg>?xGLEy$3{tmeRMmlmrEJ|jz(}fac2rmkn-HuJX27Df9jkc4xSWiXV)Qzlm
zdGB!yC*1EGr*q?WIgIdv^E@C~;NZLfb06#*xNTY8($@%mA=N~yJHH0Oe1Yr7s0N9a
zgP9Ju$>=fMC-4mqD7h%d+9I@&Ws4!v?EMd~@~R=p2q>vz$eFln1-PK=06&~|z?t>u
zHQ*F{kD9dP*OWuvQec6a-0F8Gj1z`f#WWR@<b;vL)PW?xibFH;QyZ?75}9@aXF40|
zyO>sxK;*HZRFTJqdPN?zif#a!APllM2s2JL3N$y=H<+|5A_II5IEtGMePFDrN!dP#
zn_7;{Zc;>s+$pVmB7`dAuSW<Zk%dKU>XM<Y;1S&Rlx_!IdU!XE>P_Z&Typ}L6w~>}
z(e?zw#_<E1Yi=CJJ65za;_=uxB^P`J!@sFeUhoxb-{MWhAX3+XAPi8OTsTK+a&@_8
z9X8t6+SB+WfAtls+O?@v=AN`3aBV7+FPyaYxi*!{Elt)B(l(Vt<4L1GjCTi>JId;@
z^dXt_AQ*6Z9}el_TH`3KJq}_Z=qC=(MWA_<GzdQ>4aSYDxM93+7B|gt({W1elMXJ3
z&)E?vZ3J5_1`lsA2WT6t33L-^d<i~-NtWtx_=n7!>V;9WhB~$eXDpEX2(c;}vCBA?
zo-K%t4T&{aOs`ki-c*l#UWYOX+?v!Hqy60WRbS-4iw}gFTohb<ii)M2k@f}7QnOey
zqK#7_wT93H+?1o$lYUw?h=9(optV|0n4Bbmj}GE}%?a1E3**K^pM#f9q8tAl9~%eb
zX;9Z`GP}^7(9ye8G`_pdYUCVWJ_~8w&m7>&OTG6J1EwF^lM|#>5{OE=i}unwGR!kE
zB*g8Scd5LE!eDh#r8w{lyq?(xYGI$6o*NBpOR#4r>W9C^br2r{w>c2PK&!rRrg$SR
zX(ryG=S*5Rw~M4SY+E#xq6}jA3Sv=)+jF4<rAR#haj#D62*spzgm^Jucdvugjm^}Z
zgUnbCg0V2X&Gm#-{1kc3&y>HFuu+&_D&LdVIX7$`1LsPtH1f#Q<nAe)=|{H0j&>%v
zwJ5Q&CK?1BbZGPR2D)>38{f`F_yRCZLMnOloC!{-G}evF(PkXg!Tm_Pfv;K%;wY&~
zCspaBDv6}UuC@8fk`pd0IfGEt<A?kRH+~fhHQb)BTIqAv4wE=c7*vU;ziI{7t_uX5
zOU4O?3;TB?)nDL>kg27=Pw_RF9B7DsdC}Lb9MdmrzGmgZ>HC^>5uAjFtxa$)dKiYG
zetF<w>ryy94|C3cJ`DTN1v0b6N`;fxVtuwIwZd?a!FM_lc3rGQM<XUpko7gfZV!$R
zoPbl%;KX`EpZ4%R*GL5;rVB7&&g6H}R9U1!2Ma$w3>%v`TlLfd45Lrg^9vv#Xup%@
zNz`c<ft72&la{FAxIO96mwp21>||ZZ&Xc0J21F2z*l|)^7aP_@ViNQPI9I>vq$ou?
z*M_j$vr1PTlSm2y%IwO=B+!1RU>tRU3qnt;0+-Ez<#rOJJtqZ}wW{3|zWSv2Y;0IJ
z!|*s(jYh_Z-1R69u;JEg_Xp&j6_F#4TA{^fMMylsHHJ=gjR7X6*k6Drf%cGR@hQBz
z1$W~qYnk_~ids7Biu2HRgHmrjWi2#Q<d;ubnQ)#wg*#&@@}*NSflHAeor2wQicD{{
zvf%7$<;Z)L)6vRlFSX)+)cm|`W@hOB{K_`6wPKn{sW8B~m_*=pfp0i|60(S@C>rGG
z_=(SPbTvInp)yw+t_@B>j#ilw4a!jd8jk02obRP7d1T<FEAx&Y(L5#;eZC$am752g
zqA%Cut)h9r$a{+#EAn1F<AvZugfl-6&jDlQAX<Q*ujXk#x}a)AG^Ul~N3^OD(U&?k
z0?N%AHoUjbcyF8WVlU}1hZGO7X*{ojE9rQiCqtiVVkI+D{A%l}*~nbk@_(#F=3M#X
z|6`>U&TS#l7k5zUmIl4BPbKEIIy+orZX4f2P_YDukJsn6;gp}(Nn>vN40NWixANqv
z|G=dLxL1Dz`=Yl><UQZ8R@b~$p)ITEL^^xSIbSv24rKE&4C;M548a)9ByTP@CaQ+F
zx=DLSi?TspI{VdLeeHx|v1M~&l`WUT47cdbJm@>+>NabW>&-lQOPh7>+Be(uWrDGp
zCn=2y#vxwW>(z$_YQjvE%ct9{v_%t;tP=2oo=6~7UB(D08)==#tU(s@n^2Wd7OY>K
z$g7x6u8eBo@EX}rwCitq>2e15_pK0hodkRWhx{zVh42&rmT1}=8x#FdHi3yc0j0{I
zMbr4356XQc<v1ZK<oNj<o53~?kp6I=$i$b_)uFdW5~^$*kiY#VI0)Q-vO6i4f6EHx
zoq%w$+*Rok*}L(V^CS0(e7cca4jUgdnn?yvZaCw0qC~0pV@BYO9C`d(xXuUn7vBO+
zypb>8{T68AjnaQl6K_o8oe71QHv>_ccr%-60`X|V8y#faOV$8RUx0`=IwCARXs^;S
zb1ZG?b?gj%(7HaJsmK*Blk0e%BG+;3k?0&R^%La!hKQ2u8{TNs-pGiO>v(IFT*<4+
zmAsl<$*akgyl4*Z>$q1hLc|MVh=Q`U9X$HQu>46o<dGNi<nP-dOu&u%wzYK6ix`z`
z&{+X&r?!h;dIpinjS%x95f2cP1HBU;Zz_DfSgmL<>oRnz{5aW-(T@s1XF#tr=I!JM
zvpAaK2fO71-v&uMmn5J3Ha4Zt<;y>R8%EA>gWrKQ`Evzw`*&cZ_FR!X_8lCUJXbEi
z^Brre>$w5>ukT<_{@kPteb-t#?>RbJC8&BYZakkOi@uBP&I^_D3*WVtY<NCb31JwY
z=Zk4ZFzG&@??wAb3U!JqFGZns&zH&<zH9Aq!OiKgE^<9zA!|EuH|~Wo+1z0T(uc}`
z2Dj%*Sp5gwp39)8<}C+viJ@U8FJc%&)AE%Ln6o_}k{@(f*OI&Kd)7I(JfEdhBSpA!
z#4V>Pn^c?!Q(qZ}7|#z@SzqLv+UG*@U=n*FfF~xj7w7is88+j&sxT&OBjr&)z{5TM
ziSNPe0q)$#tWe>;kS000PZ{R(W7se9QIN5(*1$(8np~b*s7`rpIU|N^o?DjbKH3Vk
z;S8$h%A<hnx@%l^J_d~p+~?q~xNDqFfMDVSJ$xbdZY_=(<czd=a9E~%AEOU<%lC0g
zc^8;tzZt*nq$oTo#GpCq_|YBU@h%+9DvILKIS4vtR?|uZ+G=*5!Ew-c$`HX)6`}JA
z(WEkv5fcA&hRe{z%g7w3xbrIALMDxauyfXP`ydkxkLtm)GCE`c04iQa%(?+z>@Iaz
zYE$zW*wr$#(^@i5!<p%H<%qno6P<?JpxqyJTFcI^#LL4+AyScqegIc<qI2g!9^In9
zR#cDj(Gx>L?FaIt{kXL#m6GXO0+dV&18y1;FL)dzcc4=4eB8=ft${ONKTzs*-;L%v
z0q!PE90J-W|MPJ?w*dF8$E}>u{%mEnI437&vEQG^ty3lTSIR4YfP?k@U<n(5><y#z
zkm;yvT))o2wII=TG}EE+{jIbqJDP<T6^TjF(C%+jFCwUUw;y~LLoMoD5Gtd1*grzz
z2Oj^#lg^B0w0Hj~(E#4Wlw&`zl3i7G@_&9{Et^->uDE7&G^#pe#uHZ1Rn;l4dIC%j
z?%pS?)i+m-D@7S>CCWh|r-)_o@E@wjBf=<7^Y@S^A2{MA=ulSlBI==R=g8cs8s-<U
zRKaFPZA_J;iFZ!OsVA&OzB?zfV`nF&49N5@Z10B4<>g(V2)O&T`$!jN6v9uD`+L89
zqsz+u?C+~!K5gz4UG$hItPd2jeh;Qw6<@7h+~_pNOzlVQX#0p?S(n25`!VvW^_5%p
zhgw<lL+gU%KZFa=zv#mI2RuyuL+po#hvd6Iw3crg9>RLTk5P!AG8s`XQ&0i);$<vi
z9&zF*EKWSxX7LQ1my6_8KLVVWLh_Cup?h#2*6x4($Xb>45?E0=wsWFfFD+=}OMI`6
z5&6<6U)OTE;_#&n)1*hshJ!Nrq!qqwIEZ=Tn604|2=TQq(#qiNm!O~SfKgz-tuA&&
zLhU*2tCl)bpWFTPs6758WJtJAK8a?+efdc+p=V0e13o1<I8<vJYJO%y8Hhyk8HH%+
zGpDn`37ERctM8>h)0+V=%ddb#<%xivf!OXaE;JWEfcz__7VZq=LlB_p{0g35qM}=V
zp&Le<f6bO(?}jk**F5<`Hx}i;mdTI1aSAgoqT;}KP;*1tQvJ1;=0Tyz#Pb$>_=A`n
zTCZ{R!(I|2wHN1&ju+ZU$4e8DcV`3&@863AvSg$)7UQq|bjD%`umZ}YeyH@3V)>2#
zv~paJ)X7)>6B5KD-SXrAgyi+elw9{?YnP6n$KL}o9vQ_-86=UBepGrC!UA3m#bJ9Y
z^pB~2@~g>*yh^%^X2h$~S#%#jKW+JF1nVvKyHQ||Mxa|`V7@c}OAE{Zd(=yrYgJ~{
z`)EY2|A{rv^=PYX`H8h?<D;!42=6z9e>rUUne-lQg;FV|1m~#Cp&X6|+<0`Di|0K$
z{ORJutn{MBQGJxbUD?YOXzp?<^Aqhql_||0m{D`WOwEX1(*rjCaGT8Q!JC>74=96)
z2sDH|U{t}wW7<%ELJ=%RtR=I>x$ucbkQH|e*fZHiX26C^<lzj0%}=Ae^-REwgqBjT
zYiks1Eu|QJTyhX|c+3=@D2uZTfFDgF1-sEwiaQsql!ljq=-UGfWCjL5SD;~NKNaLT
z)lwIr?Q4`NwbX(6*SJq$ekVC23{8oXmxs#<pdC(yDCKn>%oLEK_p!x;p#iJw!GstB
z{3*C}x=dLaF6HH%SHwbN_S(`1dlV$A41EAal!O&(526S}A6{9~$m)UJnX^8(^vNwh
zvlgvW<-xzWf-}m4t2*VQ=~T!+i>jj^+Vnu3?ED$zX1IU+8J=f?oAz^D+Jw99=a8cx
z=#a;L4nFe0u<ZRgF2ck8_~)>Gg?m{q?hU}br5987m`^^`i@OlVLh={AR_?mvyqe!6
z>dc4fvRXIaSyR|X5)Hai8LCD!$*f;kp=*wT(s5)XlCtc2<35AFCWhN-Oiagyvq(ci
zT{<p2?B$}z1??F2kiqgECz;Qz96A2DM?UroD>eBzj&kW*r=C>D+T@GBzy}15*J6|C
zy2~S~j+<~)3maj*1634p-sAObvBlm7QqnL#KHgGDmIfT#T7WYdl^0(1RRiPrIDG+x
z+4%86-2FFrFjTC@{4umVX#q5jqO^+ijwC>-ABN?akjH*$EnCQ$z?Ms^02#FG^a|<E
z{L)&SdCWtC&ds8v($PE}#{zmHUumJ>2Mcz70GllRPg&_^vb;z+<;r>JDQg`ffAth@
z=qJnfp2F?hWVxgdN0oQr+FhR&UVg`<vNz(3h<9{B{K7C5mqgY6#vN_)L?4t3cVM5!
zNAhH8vER>TVZ21#(Z$jZq3n*{=r;V0KD`a6HRukkkpvfZeEa=sVYy>crK8lH=qNOF
zL;>W*NT}uJ&ZL+cZ;P$+PN=?Mh)7l58HiSSXK)5&1P10<QWQ`Gw8;of3@;kP*B0o_
zXUP0KfZmBRdi78<2d`}WFKg|><RHOj($wjg@6;boPmyo^m$e`@8MP9#L|c;evxDT7
zi+*JVcFYTt#IR2k@}rRa>@X8I#q;-uoW~-$5r|cd008)BAAv3AQBVAGZH}op9uE}0
zhW#sR*(_avS<*cDi(gr(SIvV!K{lGInsVA`;jlL|;joeT5!t;kC!E*9QYjDLyn!gz
z=8b2>2qWmHPp<rLXjOgk>i@P<(tR-dVH}}(@c;YfajE0~ZKYnPvvcGAO?Ix_R}!t<
zSLYa)8ovHa^nqLIn{sTN&?D8UqxBE<LuNc$FIQ`~P`h7bw^O$FgRdR!kuUdyufbjL
zv~`v1=z!e&G!{&_Uw#_=4DRDkBj5c(zWy}!HTQdE>N8fp|AAuSJFr;`YP%A~<o#{(
z^UvTJko()hd?g$xz$fIiK?r%3!4;i8G~iPx&OC)*&r{%~A&60)W6zuLJap8^TpKpb
z`STzdljyD4IPS(7eJ=O@3Va93HNhVAnP6t#g<Ph&i~J2yif?FS#^qqfTSCLVPXy@t
zh^RATUuxuDP=@8$i7ls_X81IangA%SHf&yb=dZ10rdR&r*H$n|k((+<F>72Ejq>eZ
z1NCtI16Jtf3a|^PHMZlnE_4#k@vCtMh#O3WOgd;<Gc&E2+R2ZF-BIhvyFkb%%X3#N
zy0dWv<#Y*B@7D*wt}3Qw|A4jp#yfyDJ@fCth6@vq`3`Tms0^Tbfo7iH5%-aDXL<)C
zM7nA!D?eA}4QJu1sp&}nR9as78*9U7=8O{i#f7&RAah0;2p<D@F|e97=xC;A70r!T
zjk&K{KKmO;>2N>z4NwO+^ena(aPyzF0*mTfgaP5K2%i?1AnQ|YdqDj_mV2MI)?RoY
z_*IX<jrs}E5;t?D{=P|JfZgsC;{wl{Vb4e6dOfU5Xg?aW;M5!P^s`vk;pY6-y3kec
zku|@y)-S3L2?I-DF+Sfg2FD;0q@Su8jb5Up^ZHEM!bGL5`b=WV`@OJiqkXssk6?H;
zpRMmKXNiZ}(Vau|s=(Eg@PaNT74hQ;lsuSk*WwaNRtUG!b?GDcUI&&m8qOm*w108c
z&}^#5%I(Z1_-V1EjvlG2rVXX`x+^Qe**MTpi8k=caHNU4r2sZ-sw3uo1XMfEg@-Zc
z;VmI<BlC2M_QW4BtVuk3=pmDT`R1UNT=Us%y&2Q@8#Rt7THT0@Sw#>T<&f~tIby75
z2JUNeEn5yWG5zUG4xBO#UTyOzmWJT7hz^7P!V4+hS)~v=10Ro`K4+zF`)mjF&>FHz
zc0fSpVr6bIAj~X1pGEX4Kwd@BQNY77fQK1y#y0d%lQ_yyy)3V73w(+rk7UXVpT~oJ
zBMtPr>h+?sBSqUFa9>8+<QJc}7HogTqZN#uw0&n_ujEYA7fBf}+K<P#2m!C;fFMa?
zHjrbx+C6X%(!9a^141U{YLOGqLznx<A({LFc5Z)cmMdStP31rK%j;jTzUul@lAL<M
zy2ACRUU|uob-8QQBO8an?BVteVOowR$;lxry!f^HPI^NcLpo`&pX?zu9$)K{=l#xF
zoc$UIBp>pv$j7@FMff03oi;J%mb^AV`>!2PZ{bXbOnIEr!?NLbR@#Es3rOR*Ta<%S
z?$(nH9eBM!_9EE@_m$sS3wFK^m5nlCZiiBmS0H>Bk2D;wmk@KmP89dS5(ENhOe8dv
zh$Q7gOe7wqKqT}KPNcm0MJwgn*FjiB+_=wjBi<V(D|hCEub+;p561Xhi1+ue_p?b8
zq2=|!8VJ5TonFr)%D{mxa|N{R&mHooFG4o~_th7n=>2n_biD+lslRmAvgI|EXTR24
z|22sgJ*I%aOpvjdYQtY9@BuAGJi#0%bEhJai$mn1Gb8aj4I+tIux9cvKrQnkywa+H
zW-`#V+_S%ADyY<k;-K5Zx*iTtyxKCMK9J6HFIj1!KQ`m5S_GNs>rX&>36zLcu+NI_
zs{hm^y~CF8sz0?U4PhF6E;WnpT5)}&WprsQ>Wy}2w3lB8+=jRRKtO~E^>fiC{8dBV
zISdukt9|s)h58|S6s$8toOly<H5un)XMTPT2at{qjP^N~S-GvR=7G%;eI}iQ@#QbR
z5gY;z1knq&rIikXUM;3J@&J$*Z6xOTYTx=WZCG@GZf7`NR+{h<^XdfgQq9_5GvuYe
zw-#@B4MGz=x;o;9Mb|bl5H}c4Jp_HA`qzNOAf9>YjK|G^5^9q|?pU^b`1g>h#=uH(
z&FeZmHakq_P0?hI-l3w&u#TbW+(jyy_G+=w^r<HR0SZl4Zs^=4h~_tD7;$5R^rRZ^
z;l}DC>Xi&ycMw2HC!Xhyj>UVQP6nELI@VbhRZaD>`ekeJrDILXhM!yYN}<xNWU)O0
zm3sw7(Xb)odGrd%Rgc0eRbGC~Om*g!HaYsTm6G;K2j5k=2728IO@J!HAMr-eAFPzs
zuk?7-(*uC~#}Xyf6KTw8%A+0<uJOYd%#mCFfIZ+G`T0Lsi&o94RAM(L&2FXXgEM%2
zYL8Ml5k#5U^5=iB{QlYP=mTEcrqAvijRI_Jknj8fcg*2>My$Yfjo|$u&{*gk?Agzu
zQ_->*scP>Sv3`6(gdRA=N6l~{S}E$e1uAu~O2u7Iq|!rKDy?6BaRl}3?U!-{8t=v&
z`O^_x&V~D-b~nGG!e4j=8pmB#@+Yt07!YpKAFX<FANnK2pIsyJkAK7wJlutUqI@H=
z=ug&~n%$T+baQYpt|tK_&MVQ4eh1rxh8u&LW8x^tY1NJSv}(hEoK8Ry36$U1UJR7;
z3bJd&=csme@#~)$G^gZl%ti2JdO~dyj&zNJ`|mE0OGd#r;YLPr2C=(T?i_`o`0i?X
zbQB8c-Hq~lqgLvy-Hn*y^K#qQ9w9@_=a6*6?q1qqUMEV2)kP*o+HS1zIp!{r4*@ay
zR?%*ZK_*t`$N(8M(hr?00fFYl*Qq0dykgJY^P&%ntMoFt?N!L%CDrooSFzB-J@u+}
z_PP?T_7*Wpv+Wk84Bp+y<|d4dlG8yY{8J@2bUgGLOx14^a^Y*}E!+*SS?f1KA7J$0
zE|D0BhdC;*>#9*Vh4~YD+c7B3xHBZ$<EQEBklrNSJSD&W8Yagrh~SkBdUa6{GdQ<o
z$md?eS?n#<@{QN5#S3o|(c!o$i4KElIMA${GUd|Ot&{~fWeQ`AzKc8I?vFz|NCsEQ
zTVKau+>|T#y^g`SsaSsXb!+KmH<b$mkCI#`I^7L%#IAJqCSlaNvBKB8Ay4m)MmIx(
zp>wY~_o#aLQ~~dj<`~rtJPtu2i_iSw+Mr|4X_S`Bbz>li(h6BH2IXC8mwaH%%1cAt
zWYNPYE*--ety9g-t#Wb<r{K4EWYVAUedk*|u;{`vkQf!UZfR?yd(BLWFy%rAZ)uZP
z{@Gel!zl{=6x+}5e{ILU02iSN>mcs3X%J0x0o;p$W8(yz6sxxE&1i`pO7E>KbSA<r
zFyGNS&RaTSFn`OSF!%$24ekzcfo>Jmt>Uj06hAW(I{F$F?JSRs$6Lnay1!Uw)!Z_T
z#TAPO<geP8iLfUL$HtJy@CN&&TZ?6=XHN!(kUmJ?>B6&!*d*Xb(Gfg2O=H#K>WTTJ
z$etn$A?6GY9oyV`>Ztr>nogVLAN~UIXU~-U_%GImnldlR!MBS_u#ww=aUc*4aUNDc
z?LN1sSgSud0JT1AJN88USRfrQzuK&6wG0bv(0w=pv-+?&TuXNgXvHo|g5rU!;yD1V
z*zM>Eo{BKa@}O>Y`c>9QFGk@5vE>Qtw|>zG;2sXdXZ+CTy}%GIH+AAF9W)Vm4+)==
zMu682oeWuxKTNJjVK2QdjMg(<Z~(2>z-B{w^_Xk0mXO&NcYJo8Cq8+ywl+`q7h%o%
z0evGL5FEEsYBrwMyB!?I;|hw|8r4joE1U9J3r9w5o3PK<Cr_JL1Il~xji+I3$W7oL
zZOqmC9_VaE!KaS?DH&8L@TS!@B9D`2$DaIe%$4RFm`NLZ<)SyN1^$hd)F&K|;2fCs
zgIvkeLSFR-PTigsd|8E+T;#{iKgX{>9LJ;ibkKu_J6_7=_~VmnJAdIExj&tY=UrwS
z8o3BXFCcNHvqQwe_X<3b!do|-bgFIxm9ynrZ&*uL&CY}Fg-YSuiQW*#Jd~kB8Cf#~
zF<YMdrnNM2HU`np$zFNGo7T#0Dx%KVfGw~;XcHK3lZg<j0s3eA>H4J}*QBKQR>OVG
z1te)vS`uA(eM^x-v{k>&kuSfALx|s2NYCG_K+0gqs}|82o1<Sh$c=x)Im55V<levG
z++iR`*8j~qYw-Xk*9P~A-x{LJHNfvm<9iqc7@hr1hJ4{~R%p?0GD+k;u^OLsW37;?
zS7qstz6sR)tRXL&z*gkhQ64C6>#sYrqFfYyCRHx<>rMy*Btqj>DOCTYfB5wP?SLFF
z72}Z`7{P;T9*h(NM2P5;@GvmV&!{aK)k)7*gvkm$O-U9A#SAIe`E;B7Xu|T(f0`cx
zX1Va`&M=;K<7Y?lb=Xy~6EnZuBa7a$9&_Cnk)F4$%P(jwgVxAd#mz>{J8a)XT0R&y
zi7;kkT?N=Gjhfk5Kj4bLTRrQiA{ylJxA9??#zwjR9c$TM5))BHBe+BYfF!`Qi#v&!
zK3Du9Rg!wGA~dL&QuWxOiF&ioB~H8tlti;el>iz(Rk!NO#?~%%L<xW=aQ{>7mCO_T
zz7yflJkgI=00`3%m`#`fvN%s(>;u*6O#_W4AN9&_q6ar0^^&1Wp(xuAI~o3(2Q73c
zakK$P-wGo~8=&;bWVVIqdG5+gQub4xR@%+#?i?N9P1^MkZ16^lLa?f8Qf5qA3%4Ad
z5=IAcsV>-4I4C|ktxU)`=l$TQbm<ayN3dSN7Ak7sW8R-6@0|p#-ye}boW$qp?ypk3
z#p8t@5Bjg2s@3=7d7cB{3ETw)2ne<G{*eJ%fA~(%{UcpmJ)!9Su^<#z>dPmbZ@kWo
zuW}+(qtI*?maq$Q)^Rfo10|z}7Q7y;Y-0aUX;}f$n+*}{uWv)8T()6SKJYFCA-L`D
zTIbX>OhOU^21o+tjg0v~p}Ne1q7Rh75Qm0`#^(XdVQe(%o~7F7m=Dw@@jfS-qdki6
zdh|_nd73PAk$9;2Ks#D70}FFA9DmU8tfB9cgxAF%bvj5{yCU&-Ibnv))%VGJ--Gb_
zz@Y4SkB%~h9C;64pFNf-7yaGZw&1uQk29&<&R-}9&0{6<$ltAVYmPNTO^yV>Kh|Er
zg0%KV8+8oL$F~<nLV=$D=2yUw=v8Vebv(1%p&@`hj)jMiu>@~(=vjI^ABXX}isJ>%
zj#hv&7mGqN_stdw4fI+89;Bxp9UmgC7#I{b)I_sXNio)AZJ0{|;?Pb48bx%iekfj_
zB~P5T(rfA`>SzzZgv5T-g-n!Vf(>qS#B$_tj<3xlm|raLl`y_g>ya{=G@3w;`1KP2
znGP0?;Qblh$0L4N2S=3d(?kI6yG+eU1h@)t&9}z_D9W3!xBESObFYiKL2WYLJ`I(#
zYd=m8lid5&b!Uoxr&QMd!}8U<ldtw-esvYD%hBCTf?#0iBJzewjMWOmciS`q@yIHP
zc!VkAT|610uKfY*yG2+*3B}${dU}GzoOg>l<H+IxKXe(HCi``dL#CYfzLk9I<Zx82
zRw#ma-$hB<1h0;ohnFmv>nV)s$U=^Eh{<@jL7hBhb`T*jGT!TMAh|jY^OULvxc9o{
z@%OF8$?pv)6&m*Hq(6FZNdD}7=u+MriMFGJ-)AgAIWG8jpabj^$#D(Fg;0X=n)cpg
z65w+u{~o{!E}SLp?-_n&;>{TPd#=8_&ibHt!yf1H5CeWSM@hB*ZcPTA4(RLnl{mVj
z;Ud}e-ArQLG?$EbvxsyYF9CA=dMX(2W;?3_dHF&*EBD{cQPj-<aYn$H<9Z{g7JU;n
zg1<4(8FG2CU~itsm4gmze@r=$4w{>U!weBm&Bfhu)!YIqOh`wsCMSsg4lw;4h@rn$
zrSls1blAXQGWD(Iou=qRCKNs$#9E=pQu_gufSwNPfodZj=fWC6k)S;_z3CJ~S5J|!
zD;EWyii3Dd94aQ?#Sz*ISE9-UQZ}6~$|QE+_yIHO#f;ZEC^Iz*)WJ=V6CXm)a=Jrq
z{-?G0n$te&{HG@$7!fJ_rmUt!0GLIceW#CQO?t$I`#uI1;`CklsS>JVLO%XaY|#G^
zkWc>;=S=?ykob?A_&NMkCR5`-0y*3@z$LHl&_4qF9W1n$+8QmF#pSp|$boWE?-_lg
z{2n$1Q(K)N`-7L01gLq<Y0Tq))7?o-K(cBdvq$Ctyz1^0zYoCO<!=20X3if#G4YRa
zzKmHYddw;MN+petlVS7&yjJyok{<g;Iy`MQhv;d3NW^FZdi8!5>72LYu~E_s)Vo?f
zkt#3#7xtR(XURSP!uKlPuaJ-Y3u_y~zxkK7GX4G09QTP4P!Wyp`=f#bELFZg##7Mk
zrWx};jWRH8EnfLQ&FC)}!g{Q!0s?{>y8mh8hnzG{{?R8(rg3yI$qmDo%jPuC#emGF
zL7vq2wQ0<`^;1|Xuwrc!%`TEGz{dV<G+{&~G{}Kzw13Jkr>9{lIMpu~euy{LrYe9o
zS_r-{jn&R$9WdBW&nSC!fbL&<12ZX#nNL+f9q$b16v&k3PGJ>cSl7sLg%tOReE70R
z6`C9e)SYKc4e{Ke0Qpfq>V+wg6HDK2<sZ@G)R3Ax|IDDdvw-&DQxOt}n1{matj!dy
zclVU1l=ASLftT)xIexbXcgi#d{|Vy#BxN!NXBE;~gq?dSuwGhDyp>uvt29I!7y6-T
zvw%)|dOS$qR>O}^h$&t^`w=eF&64hqt@Gh*`532X89DOc$JRCEKKn6DC^B*$oMm6&
zdb;+(EA33z({&Hl*z3vpfqj;l^x&wSioa9#*=CYlW3sbLIVVgz3z5Gv?X^Wom1sU%
zDab%v-QG5~*6XF#7iP?;wUuLOgO2l0kTEx3#o^nKVcO{vsi{S6Yop9}*^9HbHiAa5
z0N_rv>N1zQ{WLA8_<%wh6w|Ta=|jhFKP?}1*{SDW-}Arcp)`COR25Y^0>&Eyf3jLm
zy6g-W+{JD?bk>tyT1J8pgeOnYm~0{@Ufa6;bl{+VsQY_^@^-hq#09s}Z7<*Oy+L8*
zqlx)DQQ6Kqeb<0XK9)%(nIk6Q>%1{)j_=}p+HD71*XOc(<v$^306YAgiC9Mpczx!}
zt=-{+DL7KAu-U<l2v4VHCIecelJ|-#j^BIgA&(W14+z`m+ToMm5cbbqJ1S*qoPEA)
zC$z6|cF?t>Nj?>4-|E_d575Wk_2m9I-oA3)jtNjUALwa@aGh5!Ot8bQ>%uZO!M<tl
zj&6g#+rcEhv(4*JK6$qriL*etJKGF0uxBQVXd;cMA6I;}e~w*=&(>~G-)VKy0fK($
zQL$HgJ@!Hu+zgL>Rz|T`(7o83akDj=gS08z=}Q-iHz1y7F-qL)vCpnS2~0Q`p$odJ
zlAEipCn>rsZlzO$LV5?RlHLK!aJo_4M7wA1Trpk{AsJYorW9jk@u3Lbt)e1$e?tlW
zM)4>ag6c*kmeA=uUd7GWe8vD#Jm0OvL!@WsEkj;pjh?n}?`^r?vR9e5eAu#A!}*nE
zFE<nAo0h$7MWR=I2Tf%JTt1iyol%`F*W0$Q#xB9RKUYU6EgGU8U=p4|TnvVic&6~(
zAsnD83dEct?&bUl#u)VjFXH0RQ7j3fQk?e$Ns<WRcv-9=BMX#LO!Y!l@c+FS@~a7|
zMyVhuAF}N%7u*5c&WIF5l$s<!+fn)fazPgHC1|us4%RC~L68~=`}-gdtCKTNW+!68
z739fl676NKt*7PPiFU^P0>}omwZ_}BQK(x-WKW{Kz*W#IUrMx>lRKSgZzOj^l6`TF
zb(*XN_KA%0Gth2<A^y8)u>Ma*>qdFH^sLVZY2_EuF&)gVqFV||W{sh?b=0L?8_MLr
zlI*~y4Ta#3m~tCnHOTTVtsI_4j54`xEHH3Xr=i|6V)O!SiSp`MAh1MvXqFv>^R-#_
zGB`h-Wv9C~RLk*M=rr6}v+dB94a{Qk@#R!;xHfo7q|%L{Sd}Yf2&4Vs4M52T=RT1d
zpAA#8Vz!;?f_r4P9jM7Rh||L`iw)hvpMaF@A#RRu7pIY$?TgQdK+I8Mc32N<=^03;
z=xmh9rs!z1m^dhD3nu@TBJc^^#luy(5DfTS*?Dr~9DB)v>}+Z{taJ)ziUK*=+42r#
z3uosUGc!B8;nTINN*ZLx9MDyEi+pB|z3}?%7V05%==3l)csT30vs(yM)T`D3s%lb0
zr54N>`A{^E`<V^fXvhn_HuUX`L{yO*Q^iJ19$f`_qg8BbqSK6h;FD)mv8i7#ob;6e
zY9@(;oBEY@pS**f*+&h$C_M9;FlRY`D(MDZ?J;f8f%B)pkX>6K<0ZMR<23V8_WY^n
zp@9?TL<W$9XZ(`i<r4)bOz`-p4z6?XJZ7@J(gk-{vK?BU1B|jopbt$2zX=m&X{1>T
zjYj0O$ZsawOI<nbvOn2Q&&p}1A=oAYI9g=5l!xQ+(su?eGEp`|$&@-ISIo7uYBq<6
zpW-Mx5QCM?d9<J9E$rq9de&Jxcu;RNH-ki!BFETVG?S|kU2KZU#fncB2sSqrXf~s3
z-aM|E12#!Q4$sABVmFV=cjnq@<R;CtGq+rTZEhmSIFW40^g@9)Q*=3|UAZ<fWB3>3
zSLi-#_!soa&(E`0x!|_VvzKRI(C3X$?we`b1w-1zmDbJ+rb&{)H>YCgju+Jnrsc=;
z?7;RdVWQVEwk4>h8t_B8Ek&`yOTNP08L1&a4e>`WDAk#!w3XRXDJy)KKX8xvY=6y`
z${9`nmU?~_k=JAR<C6!t;VL|yi~WWByb25j@wiACc&yqt!OKY{imXt{h1_l|33T)_
zGtVfRxl!~)2Jbv^k1h<$vr^Ei3-jd_DfYse3&ARQR_C1o;GzOOe?J|~a4A}M7Ol9b
z<lnG{i%Oiys*#!7&R<a7C;Cm^i8Z*ITyiYM_PZ|e%a2m*<x4K9P_hU<4OS?+X``lA
z3?8aW3T0%zosxP<v5%N2_P3haX`n9Ym*w-p+b<cBC+1^;yL3o)%*V{R%p-q0-(KXp
z4Bsx8Z>KN1tdLq-jt{MaD)Z?;I69hnEix+=>*wWuS(J+X1l*&kcG`-|^Hh-lU8Q1L
zyj-6SM2mi=M?RBkuUz^WEEstph`c@EOM52teNxp-G!og8{gAKWp6$0oHCJH%7vl{s
zGI^$_xES~q6QqV>O889Age(37!~BSz!6#C<qQUFl?}-WU{WDH!i%mg*QW##oevOzK
zi#FtgGWcW4m8Sfi-}YwYk7TN=4J!SrR^mx`F(U*lWmLu@=&Dwkyue<%Xe&r28(Fe(
zxJ2Knjltp8Y`JxT9aypz<f9ZuF>+x+xjq0bl=rN`YE@7!Pc5+1YYIRYEb8D%N|ESw
zC=2~9X!l~$GcF3cL|pPjEhyg!&al9%eVSN_VdzP~ouz~<j0c(UirgqWp#cFZC>6$Q
z3&i=QSVl`JSur7<oFz*ZVy0Xjl3!eCFAQG|7RocFMAgEJ-ql&oQmXNLb&u>_2ttJW
z=Y=3ZxSlkEGAdW5VY{)dP!^=wXOUZ@-N(}G^mFmWOh7La9u`-LOV2PwN&^puz>nye
zS+@<QL;=t`+*JWPbKx$mT%--hiK-qFwe0M|vLCQRIlC}!e8Q}x6A_MRDS&ZBsr0q*
zRg_wW#VEo?c`7YWJ{Pc47VN@Q;F0pHhUl%-t~~j10DOHPlsAhobKqX3-CFJbU=g;Y
z`+DW*BHNdA_XGx!0q^U7XtAAAysw`IaJ%rUmd&K<-Hd8dY&6cEgqNcC=oS9fdLorG
ze;DIiJLMgV?T8ER6N~M0SC%)1=7UC-oBU^eXc>L=%60c>?9I$_L(X1;EgIa_OYFtu
zenz|dmY|#EUGlyqb`iO+F0psGZcCCmK}>77*Tda;TOEX)#QibPZ)IjD?w@&nOJrA`
zky>junrB2b;v-|R1-Mt}%xx|5rJ%j&tj`q*qu`7ZC9P*b&}cxB&(+JNOYOC;&rQjz
zmxBD@etxOFvgUTsKJTw$KUcy-a(jkicMxP@M^;Tj1?i`6FN7rTH01VDMQN#+SBRtJ
zX40RiAF8czW@M>*|J}H6R*jLOiHTvS)a-~-e}Q>KH^vpYBro8BjtFgTh^JNHgcVh?
z7lH=shfo6m+*u$?mSIxC{i1eTmSHiwvz7{TgYK*+&ZDkk!QfuQxwDzzsMt1~+8oln
z^v*V%BYXj84ACAO>eMTLmAmrLDN)7R4;CkNCpZHHJZOi(Tq5Pb1Ppf2>_elp;o>?D
zO$jVj?hR$C7l)=vsN$5nladJTOntRYT5P=?+OXSeN!Jp~$!eSAiRGX@xQ{Kj=hsm7
z+hNhDaypZswzG&H8{=lw4zI!1h_<;vSS@<P8Yex$7^$`0Q5mmxgk(H74Rq}=Zp5iE
zuN{fDw|3lliP@;d?5#mJU=L1^r$~moNhBtz7PEPF?G%N;sW^0x;3v?~5;9!4hmKju
z%_rb|nskp3cbaO|QJlen#+iVy4?v1`^09=D&lID4K@jCxc)cedVrh)<dJmtH@H%l%
zd6I~14(U|`Atxh?==W6NIpr-#bwbfp!o)qb5X?qrB+7g0nRqD4e@`RQ##P$PARaE1
zJ670fHHWieI)p%cl96~vr|t#9YF_|2hihreuebMyD>R$;x*Kryhj$p1fj*5udCSqu
z+DD@z`r!&59kM?<Tx&Qjr$<m~C<4V81BYAq@~h)_@^Oz3x5oA#p)*pyJerX30Ur0*
zv>adUA<9NXqK=4PUP~;TBFEJgbMg*nLtJrtR>F>mKScFqPz|^O&EPMCYMdF&Bzd1#
zHm?Lng8R#r_8QlHVfoHV2u1g03g^=6eOO#ETLA!sb{}{KD{37=n~6@ugS1S4L~+!n
zuNG92ir&{vDCgR~T7U^av(-x{Q=IAB#nf6?cvs9U?5Ki_5{4KOrt?kF08QQ}CSpgG
zFz_(5SO?>hLQ$_W%&f^B5a|+;oLJ0^i0h^}Zh3Lc3WXkhFG+s1%3gNf_aKsyObX+)
zZ7iy&_I<CIo|;fdhM!p&;hbT9uT)+WhOpY1Den&3;YHuCNW*LgA@SA`d1xc3mjCbf
z%AberD_wABr`xONcTT1`Z|kXiXi8_byei#JE&9PkIBE_EfAqnM2mlgdXW~BoS1&41
z#_W%S8HEVpfud8Q);;Fd{+V?7({wDPkGIHIP)orR{*sv-fQm4m+Y+6?uP{er{zOn1
zpf0hz2$<uOqbKn3^4MjbGS$l5)sQ;i9$XEAg4?><UQO=6YI~jQi5~gkYWv)?pXe=&
zTBFc>dZNdmM_73xJkf)LRA3|e1Ki)`$kH`{3-=-I_G|ZpHIQb0S1H38cHr`VVfc2V
zk%Q*Qd<<ZJ>Qt(3B7l5QNMpN=Y#482>k;FFhAK7vqjdqbuyBa)lMiIr8xrHYA@ryX
zj{I!~_KgW0bfk%gGV*w_gpor+A5SX`h?y`Z;_=ybemNch3C@VBamv-eh6H?qYkEpZ
zAjfej4jRc;K}`!YVGyM0s^lZ&N<<TVq1DsKM<R2p+2O63>C7~SBZ%ucb>?84c_X)<
zGG~(TcD?A0tM$To12rJdo22(>jNXqbJ+3bahR*Omnk2a#ms4`wmElt&r`P5&N5PMC
zPFr-rLAB%K31M7H+!9tz1N4*-L;JX**W!<$FszTOg@FUz9eCMy%6Y8V__(3X()P0|
z(}Y~J79yIFB=gro95f>G*joE8ml=`?>+F?V%~7>a0Y+VTI!8$|K&FAdd0~N_n}^Ej
z7meFNade*wMT{z3fZ}A%;A)W7>#(ae#^pon?8P<a5RJ&SxH^kfHVQ_?-k@S{;L-rD
zVt1?9-7_%Vki1E@4{g97e4ctn)-v6_1RfWIVdpQ(Dv(6X+H^Ozny$(qt}9ZA><1fy
zAOXU0rok3n3NsCKs=EVwa$I$RK_RNdn10N$g(_A_RHjjiT98Z-29?pY^5Q$DWD}UR
zatjPH)d&qHd7a43HFOk`m9AGm6iq;qNuP&hSkq$-B%WzYgAW7F#u10lyOjJeJ!H7P
zq$lb}qSU-Wtw!XHTDgA&iY?WT#^TF06Q=WktTH?PsEEW<GI&GhKFkuvpcxM<(Rb<%
z(GQzy?f*9!9*oKGAS9koLHK-3hR=T@18zD1`ey3L83i$gsMeV&^dV&5UObQlb8uL4
z)2GsCf>Kj+<y-3^_I+3=?X#7;>TLTQ3KySkpY8gvST>$*FTc(BXw;j4M_W&X`5X!3
z@?m#A<`HA~!{H3f1$}#fI*(6nqXDBXwCjvmY8GN|1Qh_nN7a5h^JNh5BU;VnY8gBS
z2XNB`^1O2}rQ$2(wddHO%;^HfGe+lh(M}9TjHCqnL@c1;NAacdyXV+vUm6EWkKcY$
zT%+4<#`%U^Gh;`2&A5;-8qIimrxtIlHNu`V?wg6LmQd4FdX8M4g|#q#QeKv22e-vH
z(i*-EIbp>{Edl+<*G8?dD3}neB^waMX3PZ4@;P*;b57JaH6eo87Uhzy^2b?FBF!q6
zzs~~p;m$tSUbbykt(tirX3t<kO0qC#^+cSNoBUcYi=NEtp$#qcw-HCGKwD-F(hWyK
zxjAd%$;0Q`y`GigLIalyCW$Zw`RheTqT}yN9M}FQ_g-MHimzD$J1oXU9G#JXp+k}|
z$lpYMZ-D$zs~7<h$s_;lcIW}1=_LQ0G0vYwe(wOlAjp1jUc3s!@yTd)SQLYV`1uFK
z`*PS{4poux)?{)3a*@D+Bl+>FLNt@FpEGeD9~hyNC#3?o6G=`xC`b*b;wj+hbgN4Q
zxs(b9L$OKZpN}q~v7Q>d?m81zAaojcMftsq66Fsu-1%UsDDM?MRh)ymxJpvR*`MU)
zD%kHuqq#MyXtY<<_=C{gQ!<6I$KkB0_2ds{h^&anE8)~=jLOWx><A_K`?x6?BAXGb
zgNzL2S1^^|ADk&tQg6ac2Cpg0#!GGA<^_d>V835v^1v-9p&$(416*VQ#tR)5y2C0M
z42g^a0=P)NdMP%Ri^7y6lb%0pGAVvBqgkPUaT`HUI9l9G8F6C=d6sZrD9C+RLE#Y!
z=}TI5y*ibHl*)JyqDfhzu#n&cJ8D!`3M|cqv;m3|S&Xlx!$e5FOiEgYHWB6EgjS<|
z8IzXa>eHL4Wo5nG{MD7*Owg%c1eyHTWK)b9&^1LWm4L3H&fz1_$jY>`Ms+ZW(@x7k
zo;_ZZF{z<tWQu}X7j)D^E=tduqWe5_n}>tv0*M}xlP{{8T%y(0t0FU~>>xhbBs!ql
zref!+D*QOn74>E_H<%6`K~Nd6)u1eNt{A|nJo{@Ht`9vyzmX$P6cmagrY<@X^k8I>
z#913C=q2JtVG-t@(@|eEl1YNj<DxOhmV-}vLZ!<PVv{)Yrc8{EM|g4|`cMu!X+x0t
z$fiLWI3G{{^OH1^a`GwL`P^@sGy#!OMlm^gR4|D#OJ`s{JnPBv?G&&2N(WXBiEIX0
z`(Pr3Px0a?r;lm^`KS@xQqPTGSdq>%k|HDA4iABjOjA;HY%dH`DZ&jX44uxySpa2=
z#)rZjtmAAI<i7~Ldguz<ckx99+%ljx3-eXv<Z@~>l3-HI#jR9(5|75k<5asEnM?e_
zkPTNrCx=sy`>wFh*>efX3X#QFzob~>PSM~cB|2DKjkS4ziRRKS?j`DH%wMKx$R{!)
zR4rR12_HA1Adgy_#MFGXPcF^3m(-BbJBCiS0kwuEcveBfuK|A|xV@J9!vnY-{lUtD
zu5zQVsplj{(>0h0l%|1PJ3vX$>()Ug6H})AdA=RLyr_&~Jy=vqsm)}ifg+&4Ol%1#
zW=Bx|Ip4nG>YZp_z9=Z6xSc##^O>f0R#AzJ42}l{IgDuMC~;)mL#k}y$wUfkq7=<*
zaJlxLEA0?nvW2@q_Fidw7w<xgg(zyHY1t%1i)f9*7eJcv*@sDE*OYwwN_)|HyO|Kc
zN!3zQ(#;KKlA+t8^>O%iNKZU2j5Z{QriA7MDvRr+7hVO+0CkNMmpDfRUNq^C<Frh|
z5^i^XBD$2j!hG?T2z8}&&?SalJwo(}={QjrU(29BmDUr7uXn~~7$kub;(MK2PF-cs
zcHJ^4C$6#=rtd*(fh?|JPl)}_61*pygI;Rio)NijtDVX>20_G3;-sX<jO2-kEVue8
zduf@ds1TJ^qPmt5&Ng9`DI<&Euqqt-Zc&T<8%Hp*OK`od*Q{l7uHZgnh!i<jG!kai
zRH?WWc*X=}MtJ806Bf8L*gnS`0EPsy!VuSWFiaW3OcT}P<j=^YDyt`CVu8J)rWyc1
zWj>+S+%NWWY0grENs%uqt0_DAlSCE!YnimDZ9ozeCMi-0fs~U}pP&dv*gYsk@v!ux
zq%Q%@V2wm~#M`MmP+5MdfU5K`ulf=w8~zyLJe+j~h{KhXK{3h0Tx%$d9*Ic4=2dn1
zoF6UV30?<WIl$|BJ_)>ToLAP&0&3m8tutBgZDT)o;@(l}P);#U-YHvs9Z{fPElWob
zK#v$LN2gU8m_6bhIA{s_d2H#GC~5+gB|p4<Jj4$)A>k}7c08a8`$TyaWjuzAKtfnl
zK*QdR#gQZ1<jYq>k9rJ^!~P1igLQMjC5BwJZk0~)6FPK!kfi~p>Ejx$l<TBHxq^MB
zn0D6+jzK5Tst_JYndXX4j&Vdaz7Eq#t9geTQ@?|!=!1D85#3iJb5n_|`K-Od^<c03
z)@QM$f3T0r!Cs@;Y~{W`I83-w6!cItC8A52c~q%P;vndl#}7+J3HMwAE0D#QtRy-r
zM3dRZz?u~RNC7<Y9%f2VWb-gG8sc+md1?W7Lq$H=eo#d%YSaVdf25f5sciIID0n<*
zP(AJG_)ndt;tFo7U_X%~W&66Hf__Hn*9+w1+w7$^Uk6Btl3~&4M*ApJ2@M^jix!RE
zwg^qvNF#Tmt%_1B8%^M7jAOo8t?Hw7H}kY9+T9&InrFmwkH1wxEynawo0M<$;+$Mw
zU1$e({0F*RB09_tQ;g$22Q}l{=mtIk$jtb=z$qhI2~Xc05JolVQ=zp3Wt|RwFNgfW
zY!P9K{=Ol9RS3%*xNjGN?|mONCW%&jNt<f;KDY@+DhwvvN^67S;hmWTqCT#j$)z(_
z7$YWL!W!ftt3*YuyTvWq@PZaWc^s42gYE9PON@hZaUXw><DV!Y=w9X<-DBh*+?3GZ
z5lx<9o=QIs2&3DLb*Wv5Zmcr|>nG?ly23-P{mcY@S|We89j0z@-`H-iJ@=;!dr;JJ
zsb8YS7;rB26sM?#pwCAbtWsF&3&_T6?0_u02KZ2cb8`D5B%~KJ-ICv{{ZE5z2;|I0
z?rEkRCFws6oPco>cQ$cBG~rr%Q%yb7Y+iwA^-j$dweuS0QEE(Vk5}}>CeZ^P&`;vy
zO+BuDnzZH#G=Rz8R|%<^cPS=)dI17K8K)PJA18JcpevYRQQo5iYJV{k0RkB|#i-&t
z=1~MQL`F=^{Ug*tV}M4=e1P$n1eRXRL3B@u9Gc{~6X*bP^IbtF4rLWI9SsTAJO(fY
zq6kX44vaJ9JO-qI)%0;ARiLO?V3nlA<H(L3KsWdOcmWSQ4K{Ag5qe8YMZ2kT(j_@m
zs5*WG1L16;pGE>OhILH-tjJz;T{K*X`>}d^@(fDC!XSu+d4lRSe%(kBEHd*}E2!sK
zG(IPXvOQZt2v*DP;D9`L2UPHIcki$l?i!rdAgUE@J&#!z5v}eiCdcPXsr_IEQ?A}P
z2u5n=yqLl5C=peonB#bigen=X)_~ByA|=|5MC405>~a^}ORlr`koy?i$Vh}R)Somq
zM@-C_oI|BmRP*ef*<yP3*cnl+vqkF}@hx)uPMB|uw8*=6+Ka<~9w)-jW&EmFm*~wU
z0y<j4O$xKWSN84%+n}e8azuG0#l2cbEF?k{X}kv9(g3i(80vgHNK_jK-q9C_H;MXL
z)Euja8lBiH*7&Rl<$R?L&pw#At#g_y{A(vwm<0BgC33K5$)`Ny&D;vLk+HGl@R+>O
zCQFL3ng81eEOj7zVW(P;cN?hyMP)!W1_`X^jQB}HTQ8P!DAGr{!ne3WkAm%he!EbQ
z1OzFt{*qDxN|A=%xV_Col~WCge*u6E;Ic7i>4#ASBti1OlPhOmZ!b@NhbItZAhlz9
zCtqHKXxBRha?kZRE_$a>-lyIFfLrr-PzFet7o~Ta2;6B;oh2Hq23s`Qoi>9;ytAgg
z!?;NbiF&Kv7L9g?%E7Vzkm$8)KNSn=w}ve-YE4=Ut&c{(-fp%<t9^b;q7w(r7%!9W
zpi>Zu!=lA*R*3^tJ@j9L_Gw#8*$p~Y*6f0z&O1}`YrE{M+3!pdy41ppxRG732ZEcp
zo7`!+X}7&4G#Q|w>zh+IS2WCRpG!4U6m?FcU@{;d*$twjXr8B#gonJ7+^|Uyq7b#o
z4auS*c_3MgCQm0b2pwNf@omYXBe{p-&98K87__Ggip&;Mw3&)$ej*984@q4#rsheI
zxUraKa5zuo5j~ns%xQ>e&@h=zlk(~k5TY!i*M;!5(71^guUN=4oAprXtV-xXN#CSf
zGzic>%71N$y4+!QbA1%T!ryHwM)`0D%`o05n-8lF(9&i;JWho$jUZ}O(voGaX~_{J
z7ZBSv{@FleL>=Y1`vK4wge-ZQr^g35GWiDkVBiB3AWqfAOY{dQQYy-L`TN(9{KgIT
z>Wk@|PE(p0-!p?gbULc@G2;h_KpI_b%)H}AC`1qRQ9GuE^xkN1^iHDzLJZ>{$7abL
zH{$eiI!oSrqrEP`Cr~1nrkQD=$_PHII>ERxhveXm_WZEBkyqJFD$VCMd}_lXq(FVf
zy`FB9vv0CfulZ0=ZT>MA+R^6*upu{?(HOT#rDcL!{BLP3dein{7U8hpHOffCQNZVK
zg1FAgenAHS+>;jSZJT${D~7#;yv)ag$zV+)yh9^T1m0?D)F4DF6sLf!AW>w97AWv3
zBGCu4N@8P71xc86KzsqE%<|Jvt75Y@XE1*tM3^nTlt6+BSN4eKjNoh_!4P@qf4>sW
z%-OI>yxC4!Jtxd{LuzT@%H}k2n1nNP4n`UD#^TJJe%X7ooqqNlZVb!mrt-4_!<>Vj
zQvF;Geuq-LNiyvgST1?PawFUtFX-GOCX>hJiix>O7E)0?b7P`Nu}m*nv~rvL5d}{q
zWy4yEY9PfhHfP!7_jW+9Kr82jDH`VWG2SW8HxnA*-b1faSR+WzVB+$E{xEB3{V=^_
z<Y(cK{T_Vnf#Q=Jx#mpplhb}N?QdTc6B|J6I58?SnXBtbZDbyhW++`mTrI70CIz8m
zB<)08ZH<b;yl!$sxa~|3T?zf9;xN<DggOY%3EdnFfCPxCD2<aD0Br%n1aux;Ax0Yu
z+Ek#P8#EW_!gg*nZ7?8)1Eb6o@bNYSgx#6e8W8P)P8Bya2cjLehYKgt2+X`z`RX2f
zlMC*mGID$5jxswfe;x)NBj{E8DRUn1M|&>Ows2^n7+u)7NHi~MUnV-2onFRZfJqN+
z`}@-B7Kw&MEz3mPvT>C(B74iQlr3kn=`}l;zL%mm4kKII&M4OFoVgTJ9<;;q-ZCb3
zMN`Ylc!WrzFqfnE4y}cdrve9{tXnZxglAI74@f*DXOnIn0VYo$1|(DTE~{TI8kdiS
z#6+lt=M2Sb)u8DU^7vkRLAuXF<)Fwty=;1!s11#T#8{|_v#<0rNS?vIO!?eiJ8iZv
zlR)r_ZThn02Ya!q`?BT2efIJiUk>HRV$;65YmFFLGn7GniN+4Eo?6Wj5uI%?qx~$3
zSl!O`aL>aS7<7Z{Ts<y~N>Sa&Q-l&Ig&|ataworl{U{7s8q*oekZ(}-?!#5Vl%PE8
zR{MYpA2n~g)!t9;f^z!`a&IZO&-SfFFTA3x6$_sLWup!2J+h<RUX;APlVrJYnO<hj
zDv>Xj+gYKr%83Ac0o+1w<#xtIG?GMMo>ec;z74i1aBsQ|_pjhSdYc`beGX<qPzjJ`
zX1*M_&0eu2GoPT1BupfT5zm-MOyDUJ(P5otKv{Tj@pJb4;w*Fwl1MfrxG~0nis;1y
z*i;{V$sOpR8N|V$4uXig6!SOFt0f4m-!#u_=5*$IDH-xhpR>2EOUb}nHlpF;?u$k5
z#jTeSm<{y;%VN3Wlp7|fe3+#@Go?w2+wF^Ir*tzQo^2^Ta{KL&aNyR#_3}58am>`m
zbKv}3`HS1Z5Ob37^b2OTT93_W5XSV;xHg(DKAsaBF*eZlvPf=jEaKr{LfY6tem;*h
z=X;3dv~C^RO0AxsPxR2dwR@}R+d8hI%6R1nis_oGTDFRIO6%D=pwd7}K~cN4jYAhy
zXv{$kSHL8-1&j)q&n3{29G=M2rl@n(3(+VBgcuUzLQIO&LQIQ>IMEc>94A_#V-j6)
zE%Bl)zBfViCrl<l<{ZI@)+itVm<f^n{}+s52SyYGmRi=SH@jL$B>8`VRugSf47iJU
z4mP-&d88u1GSkuoKI$ZDU3Ds!$tT!G(xDoS2*-qDVlHi@l4`ZSe2}<KzddY=q15qI
z!ro^LVQj_TCRgN_b2y7I+mO<q!l6p$x=@PY{HY8|B_$JHflOhMoi7TDMQOV!I%#Lq
zjgNd&nJbFr#}yDi^Fa(@y;kO9ieW9pce+HMw~plC&ScS@+?_0Xllxg1zA~G|CIIU;
z`FPK~s)_b>xM)^c$8fiXDVzu0w-QOVmQu%ZY0m>L(N5Z_E-j=McyY<V0Hje`lgGJu
z0g8K(x0yp%quY?unrVoj=jnCV3DE$up$=RfkiWeHhRubj07rYg*l!;o-li7*oxQ>s
zC4Ca6$2jZ-#FV=qvSgxS4#hnCfUF8A;9WRL7`3KpUt?Y$<hjmk{q^P4_Fl7>8?y@)
zc|~n}SG*XEZ%Pm?3B8G;zb0`oQH;&4XJO(-ObCxC2Dm}!vIyMnc2Z~zS`(Hyjjfgw
z<yAXEvjnpX(n*#W_KYw`E=3Qpv28(}kX2c;)9ZpXIZ1+NivXK#<7K^cin(~1D6HV@
z;0wdpJV2g?h~45+jN@h?n8=Z8y%}r{vI<kr%^5NA`Q*o0MGNg*5?~rpd-`XxWxsvF
zWeK$+fu1p|O@;H7oGkiV5uv}G116l4)gJm=*D)7POD6?dx~T}Xt@O7y4}Tx=Jh`;e
zK8g>CCw+{ciOO6_i^(P<iT+T1aPYcrG~5V(2>*llPwDdY1K9cEa(lr+JI&IuRy_wj
zPEc+=XfI!2m!L@K;&~OxE*Kn<pZ-i5w!HtKy(%TK1fh%zRboITCCdLkh_k<>0t6|y
zZdFnx{Qj751N_-B;l59Vrx6a$q|YMol;4N{Xu+&PIOP{{%MMeNu4h&+{24~gOvD5t
zVk(+VjVi)_LJ<JXu7(rReb7K=w<0bB|DEsv{8XV1&k4d0L;GmB9)5!9q>rvZATkpH
zjjIZ4Bza2^Zn#MP=AgYG)jNRzLDPxJRMWUhteKmGKp6iu=((lvXT^k@;TJLC7WfH}
z?%v#P_;YmkoVe47%eqJnn-6}VVII{n9fMC5zZwQMNEnMD0o{-k58^ZQe*j34|EjXr
znW=KkA$!B+Y1Hg0{MWhDy5NuCzpgIrH2m40hKkdLGjI`*-#uh6Nm+zgWAGBy;!FyV
z5y_tp+2_<OsYl3<|GK6nt?&~9oNy2P9r&+n4wk{6d(pfShop7BAmt&y6DGtp%!HTb
z!apz*UPj2Arq?G4b7;?U`S9KLn)RVRga|hpPKe474xR9%PNz5&3(5bv8@I?;$|aw-
z)67-!d^j~>%G{GXuK<m_@bG-f83hpThd(POJPkj!L06DYfP*pNT=?m~lV1OSJ3Ift
zs;VpBpL+=TaftarLP$agmk>gVR3qkLs!>A-5N(8jnU*(=(~PAMY#bw`G&(Uim>Y;1
zB|_wc4y~!;OtjQVohWo3)r>=1ea%d<rOF%gUU_Y*=+O3bii#~WzRx;)o#fz~_x@<~
za?f|Kwbx#2?X}n5XP+OJ<4HbIJ}9uXg|BAQ^SiC;o=A1v{rA6gGVP_O+>8z<$G$x3
zmUeK@+vSJdx(?^BZ%FE*LINN4DCcpE4kLi0snA>2&vBiGnJ7qO{9Jn3eQY^gQRg<^
z$9)M`w!3}zIr%HDJPWyXmESAX@!8egSzwi4wuxhn6@REl!N@A>cPH*cud6m6=;RTq
z<$Z2Jr?a&1YU#N4KvYt7cQUxbR4FfWH+OPnU_~ofe4swLrh#K=PC6CQ@)V7geQ9F5
zrh~GM8b9OoSk>`Jb+;|tQ*|zITy|-l3Nd^0YOS+69x+wi`7tMDQAOW0F>Y0NM}U4L
zqd<j3NyBRQ(Oo>t%<EX2VouR=Uh-;LLuSzIwpPaNRoydsq?TH7?EuHpo5g$(a-HRl
z#&`hhx)V|s>LBI6PMj_pDy|gk>-ZG#p?+l}U+tzI7I)jMNc_KC`BxtJ=1X&M+*)Z2
zP+8YkL6g8pwdV4#XI(GuYvH59c`&=Ri*q!dsqXJ`a+cn>30j?TRXCyFc$nt_M5uHR
za;iq$lkn$PT?8%*8Kp|CEIqJ`Xjs|h7Vw>4EmdnmVpU9ay<5MF;I1BYn|BdJtKx2-
z)LV7d9Tr?O?2hxje9b8JquQ(5#2T@?*+kf?_^fW<WTo-&Z}fiTtQ~eM@5j8_WVaEV
zUweo;9Y$T>Jaq$4{`L#sO{4CJ)Z=;Gaq#jw>6gix=`^K!-y)5*nke+;2p?{_rNb@g
zc9zE0i5qK8J8^53=@YHHZV1|7sIHc;ub{j>RPGxj_xDg9ijVap;AMPtl&qiNI9GjR
zWX-eIpBJW(q3Rf3Kys$r$)8)_3_U?A@<H+X7QPy4Or-N}kkUBb=MJK}xKp@gLmXm7
zNHg`?hU1jinMN)di)zuvWawp~daCh8u{0Q?s=&r(_hj73-?H&Ew8oI)TIbes%0CtX
zw>D6wIkUQZ27o`6cdO_&VMgiHuof?3Y{Q^i-{UOWprSu>fzpqO{w!i5$E&kOI?4Uj
z-7ot?PR-hM3jQi1+VA`-Fk*Q@k=Nr?$8M_uRvHt85{#MKx;U;3&9>V9ZKCSO+P`gr
zvPR>&sX?v$e^vplN;|xaW7__+cFUSo@6V_1w|6_AxOv+vlou$IR@^p8S<p~{sd$G3
zLEnGUhf$^XyTyBm1s<oZ-NTf@Bd~3IoE15LAx;bAtE{`S-LXB4qPvFOGkctzoV(7$
z^Rf0?v)$BQc=+Y6WxdGd*Z=B!$pAdq*-ZJG?c<?3>hSI7D60#_k(TceM}!?+48|G!
z-1B%Z*RuKb>hrzaGrzOcEqwsI^R!$4fKycTm9wxNGK-IuJ+rFgsjIsEUtfuu`usxk
z0DSg6BknQzzPH?+kop~G-Gm4E?ksSZJxF<sI0EN)4N$MkWH*rzboj1pxBo#W=jL6*
zl)ZAv2(|8SrK~n&Q?69xj-g|Gb(sE`id(g}yWI7Vx3qf%o=unbZ9JK>#!K6XZ+6Qb
z!ng6$=DVlPZ33_BAvPiz|9Km`Pa~(|{|??g5<@K^8!N%+%Ws37c!*cB@gsN%efWkS
z^<38HEWNp3==GQC)89v#4jic+jMaYWb1w*X&6ZJN@T^CVyF-0W(Y5<zP;~^eF6$$I
z-la@^vu}*!s*pc57WP~2Mf55>FbKWP%oerX67>9)$v<+xj$8Gxv;2xd;n6kAnr`#v
zr(jk-DE5nwqkO1yFxMS=7`+ZkY-*nAVKp$d9#Or%imxv@cZ-6l|J8HuMe571=0+c3
zJn%ckRr39Kv)l9t<wK)x{1N>6<SF-r;HP+%(Ifcx=|(qWFMay-fLpqk@@HDyy1h=}
zif@WZ;jxfw+WBS`ea*5>7TPQ0>8rZM4jr#Y+T9`Q+B}vxwwGCtx4=!v_p=S|1u1`a
z+|BK$O`fZ(>IaW*cANa~lWv!MKVRt%^*cqkzaV`*RGl>xlSX=~tK+MxyQROhofjtl
z|Ma6B&_~R8tl6E|lN3+xPNB$OU*oG|%iQ8eofY<XYuv_1optt07u+L{GOk}9ch5ZP
z6y^Nm8R69>uINIY6WMO=KIev|uZmwZZv14OYUr<a@m<WP+{f}-^R9i)io#b9gVm2y
z<FCT!vH11vQ~S6|z>DHD_R~J^l&{%OADk?AoA=|7lgHfN{piQ@gQxf7x7SM#3{cE(
zibn^4{8W1F0M+=BvW@}7@cY@vr2K4|du9MJ?^L=8kMaF(ty}t-bF=-cS}C9B?tRQj
zvVYy+4nBqt_EowkA7kOQuN9rDLgQ3yv1AkDzyYUZ_I`1xglmB*-XC`>4mgE39~#ke
z;|6v%z~Z6+A6Km5`(YheSN$AK4Yv>D?$80k@WT`Cu>+`;vdr}ar!>3CE;cxWvOb&>
zi#U^18V`+2RcIE9M2?s~U)_@)&*;wd<?DE6n*~@Knk<b%Y`joj6Dn7Emo-v89OU%m
z>Tahxp5j}j9h{cnmJd2tR+qttpKfTjtR!NtO6K?8?CS2kPd}3OaC%?HLzxfGejsbl
zoZUG+bL0LXnsVYAf#+-74SRV=GqP4(7T%Rn4cAHw;d0e*Z3E@K#uMhep(o-mTKq=5
z$l3|$H9;2C6OmHN(6%<pTL{%#)x9&TcPF`J2N|h<GVazLbe3mq>a(nLRzhOUsy!vu
zZvR0iFQKQjdnvE&pW}`lbc$jbB1#@`@)t^;z44S{>3++zxA=~N=qEmKjnd`q^g<W?
zrLzByUhrnc(j!7|Q|u32a9lAb=g?#J7EcRi`33e?`4WL9_Ux^`hG2g#_Uta-G2k@A
z-xXDU{}<k&`_6&x*Yo6Ej{Jc&l6)w5wj8-<zUsUbOZ9ay6j{^F1^f1MEO-&vw@2|(
z@CA<LPw3ZzPy5K=o4{v6c#~&E{Qjm)_{E1R@1x)HWTJIDTYoK~3c)d22!pj#X#svP
z*h=IqR|Fmc_w!w>_$lxxxLNTto|PgBv?zWS+&hKYkb91T^HT*@;&0eTYyt-YUjR$b
zO{jnud6Z1(%M||*OtkUqutuRDdzRT{RQx7bB7vu2eT9Dx7XAvsv~Rsb!HJ-NZx4Kt
z@2FPzfNe#wz^}mfN5M9up+_>oW0b2xkz(+<Y5vG3*-b14Z&H4`%<L|L`5{p&vWkMC
zkbnm8IJiM8&_!lzQ87Odoh~j61ae<@`WbMQ()Y?Qy@v3U;3jaN(($<2T(s$bQQ-TY
zWp)>p;isNuHW(G-GPA*`_<iso0uv0jXMqoa3ltN?W`j|2Ay~S&T=C*yW05C=MFG29
zp4fw7(l8{|vT>LUR+ri};Br`%Zg24HPve1zRAxNcpX13l24_OCHqh+ZH&Ng2Qu=n!
zGFy*I?*xw`zbd8$y%dOs+Q;^QXPG@mr61yLXi^~ofsAJRZ@|YwcmzBW!v76?96Y4_
zFClLL+@kVMvQY`NVkfoW9V#5AAZXy<!Nb#dMviB1OaM0-i)NG{;3OV6X6nzl%Cl>0
zp=T)lI(Z}xoU8OtgNMM{MKc<~C>XPjC_}SnnLSX&ou0jCGZj>!8U4_YfK!$J*I@BL
zpW>tPxEy#~F)>w?0lujCZMH&Frxi{}^z3_1%Z7Y18k1lXGATGig#Z&YX4BLNNLUV*
zQK5aDa3k0gex+{!U!ZFe6gMHS8M?fNLgcrBw}Hijg1bDc<f4rKSG2-op0(rzxK=Tl
z%WSwR{tjIt9uR@VK*IN+zXH~%PIw)B5nQVL=RC`7x*E*e2&_+thiVk3d-j)NL@n<j
z@HLv52ZQw4q%tgrp-!ehrLPQjWtDy__!RtgiZ^+7Pdy$GeiWR!l_v_L(Bn$~3ivR1
zP;rkJi9MG<f%GZ=&m8a~e>aJNQN>^PBG1u>atYF>f6I%EjDf`h;Xm$0o)6dm?_e?L
zh|=ExOAJUC3IAK*6X0USA9%6I^Jk|S5)(W-sSBK77!sKs%{DR^qY|^h{m|9o#Hef?
zLuys}63@B<k42U#zRI(%Y==Lfvwv!~l$CxXsnhHuYr%TiKc<3MEDhe~Syyf5Q2R9T
z4$oTN7ixgSVm6kQ9~YOGVQiIR^2&-<<V`Am%(Lsq!8%M6W6w}9MnNjAk;xLr*kx`2
zXKRJ=kKxn7F0|4o!5#2-C}uLdhEZUZDL&7paVdDS!AXf=3<z*6iCm3WrY4)DQqTHS
z4R}N=RD#cghZL{osmo9%OCkubgB(o1Nn5}{A7fKe8(6wnJ(%<Wqo@jO@=DA~V%V7-
zW^F)H>(CT~mGlq6j<eE#1YIoF#FX^2V8>bMzhYCl3VNl%$&p}_S#btf6q*n$`G-Vv
zgJ?XU442b@bb&UITrMx+1Z&VGuY}$hqOSuBJ>dTg{D+a9s`9=7U1C7Gm|>Z`1Kb3b
z7>G&1{T|!b6a)=C1ilE?C{7-br$WJ-R3NE$Wf!<l@$+Oh=|UTUWVYm&<xx>^jnaPt
zt_RmDCT6M!!RK`TPyUEsGrOMx&1@+d;9b)=`7NF$je|9aQWk*4g=0ow3IS9zO#7*d
zSDJTY`W``{lseD8Lp;DcH$=gd&w7^GK-c*{Wg7*jU}#nWOg3vP!7Ylv1*QS3N%7y)
zz&WtETm*jKvucyU+Jz~kVzaZZ{J-_An{uHaGT5079>v~do&TNr@@h;9v_a=n9<QmS
zf{t5<>C$YmtAG~ST8Hij5r*yYEVIR~^aJ1_=$dMs=h!x{0M{#>`QP70kLj>*{s#pT
z1Iv`*ZGuGlOoPhtpih8xK6n0bDW2tg4Cj6fGgXG`&ng3-m-$~CoW+8nK1u@`B(rM4
z&7y!&Xx5(wd-94u4<3Wwp!kdOHcN1;VltuGrdPZhjE93DpT(k-?RdstnKB%sKwK1H
zGT%l~AQ1Q__%ygt`Ty0k%pSaAJYx3W4Ngr5OM}`)sYSBSPLOI0rC!bnTAj@Q>f+RE
zC`bt58}tk=psyy1rHd*}(4}sKE*58l8OEu%L+=9X0w#4kbQy+%h29Ol7cAqRpiSKe
zJtjdGFgyVR8cqrLwr8130cz1po@FirC_W?4EyG`>_?&F{V~=_;^*zrrTmMRDn0JZ7
zI%+OU;Zaj_4Ztv5#v-$x%*6{r{K@FDLa+p3gJR|eb8SHRuK`O?YL{GglV`851kYA}
zGNHLPp!_V!)|Ww-Jiugk*`2&6&Ri%^1{R&`!~=7b;UTcNJP<etE(qbLJo}cR5dU-F
zV<9>oGuItd{y&2!p+^-@#5~JfaL|HxJ-hBA3~8#s`_TKz3tAy9$+OpPf}UyUX<71q
zPv|;I(iX4`m-;%Lr(Nz@Eu#cojgg<mC^Hu-6vt|Kai#Q$X0Eh9N5I)o19w6n2G?l?
z5{<cvp_oL|D)l=QAI3u`z-mAmG1MvsSV|v*UIjhf7!XVQC(pVYh9H-xy@?CQLjumq
zyDq^&1OEZN3aovYo(Wy})kEndB6E4fG?>1C%O4fcD;0lIE{C9yN%b)+eT8SWO;Df~
zrmyv^oiZv$w8AFvY3Lb>w|JJhUZS|&v)bd(HIb#?kAmeP1s~ykm~>IF3Qj+4-iB$0
zJ>x(9St`gl%`q-X|9j867XkLDHt+-bT>A7fr8Av&&_$7QrT@mWIt#$+p$yjfu|fDR
z8hS<+^`%R6&6j~k%mo_ga{iz32^gRUF3tEay!cW2RAVCJMzF-dD=M&#t1_)%P1PBs
zUUQX3>Dzf7qj(@tr~@n>(Dg$G9x+#EbpFp6q+l})Z7SdiFfO0Mqdc%(7ZS*HYcAO+
z|4%%7jR##ll<_lp5hJ)l=^qkwWnc};%tS8JWPn?A{?AN<p`QZDgd~Q{d2$_x!&D<6
zb18KCIPg#=iRZyyx=3A|xdys)p+<G)t>A=^ye7|jnELiHm3J3iF72g@|E0kW3c4sz
z3o;+@EOWs~E4bi3=v9hGz~f*~@js9^#=x2vG8t8m9tZa-eS)A959?f#`D+$1#E=!!
zT%5`5Wv(ch3bSWYp=r9p>})DbDrWMS&7yVR7+4jW&H4Wwcr3{6vu}`x&#`S#<<(OE
zOo;wjFkKXzs_;b$CPECXQm<)L%<MIL7m4WzSo?7H!(fSWwRrYH&oY;^w80~C8H-WT
zr<f(%T5)+$pBUou7UQp78Ggz&ED4Ha#lMtmTHsTP-(yA81y&bkv934Qx(q$5glkx@
zKrdE&y<F1*YoBKk6C2KJeY!l0CEZ^gru|?GCF@J_%sN4$L6>!}XFc8wy-o%0^{k;$
zTpCr(<g>OMftok6NVQL%f`3@)#Du>kuj4+8n0SgVnsLU^i*CZ8n7Aac=wB!xs)O}F
z7OD4X>4FMVAv*y)06wjl>ADsd2eV@~E1YLqz=KM^8hRA0U6gvkv&>aFrQhh;u`Y4B
z24VKiFjQed&|rgS9chQIi75Mad1EN}gbLUVJ_(L1#zW6GfZG*+9XyWsQN>44Nch!b
z*~i#~uRTtI&hObLcqe8Tcv35nNJeGYv?%@%ECLkIVIBW`B??=HKBw5T%(XwoWKMIf
zPw{{CtfO>6Ft^OPJ;sWpo`MvU2j<)*7x&ny$khta*N#HhKAgj;*>`KE=Y~07!=iHN
zYQUVQJ?o_l&^2i1NDLG}&sF)vz{_|fk{!rrx~-Ky*I_Z|ZHD0~uy)BD*8Tr@23)Qc
zF3Oce$eMU^vWRLL3>F+Yc)(m?H2hg)K63@p;GC<#$dku@aGMp&!9_ppBPw;e%^?PU
zgiEapTH$k^by6(UKF-++K86ATX0rLI@SjuuM_AP+gEgpg_$jbkioxo^9M=7>mx4{|
zj#)WOE^ojP(b+7A>Gqa77_yAZa!&E=bqflf&<4m0Z&yGM6#OkuSC?_KiRO)*k4Rjt
z&{cuCY_r`${h*8H7SQDxVBh16|G7-R8&6Q68qU3z$>ZX5g*^6b?j2G(sdw&L1U7+n
zjW?H3aux-G^ZmJG(z>(Ib=b|l+q2%mLsN7XD0R}sLXVM&<}x3@TZ@Gn#dDv4;Rp<J
zVj*-ED0PRxfr2bheg*%Oz#l^I4K?_NJPyrdsUDj93$}9cNV0S|seJCQFj)9Gk%{>Y
zFh8#qjkJN>ncyL?Ms+S*vuh{8I&O34d;InwSfUz_<X-7T(h@jx+ibchw}K6cv(Pn&
zbE|peH4d&)dMy}_r#I{TKLZ!vGajnIsntCv!Fno{`{yW72i8Sr?wy>NOyB_>R=FKs
zBxM=&N^Rg#EN)bJSe*Mc=*{q#DgAFfyIo=^D)YZs@IB97Ee!@1|J1WvrNOAunXh*e
z^#RV~*)4M+*$B)_^CGiGXh4Tmo}}gu_*LP&A~0Prg|FbjQBg3alSSSwY`Jt(pj$9`
zoGPtt0qdDiUJG->09d0s?<+*P&;!iL<t=z5SOMkrdv@m_Sh^Gg^ZtsUmM+#6Qr<9_
zE@k||K#=9JiL_2M3K(8Sz;H;wKViWpurw(A{|g1$z=6VVF|4BCp#Jah=y9+Hao$H5
zC^4Y?`KcHfm-%0ZQGPBBhypt9^NX?Q99R|1XNA)#U7!}_SE8_ZAfR*dxvm~O%P5%t
z8KQhMxJWS`U2__&9?fr+3%0axzXC&sW#xBK5QRa4jw`zPWHSGXui!NBA%ZIK06~@i
zHwYYqe=c-9l7AE|9t#vAk*(<i=PCUSuy`!se;XVVi-QUuc=oDN6dEuU=FNbi7P=;q
zdF1*#&V!4g)2H+DpjUv^<?~7i+8`!`zTC67U4Y)M{8jLmfyE#co>$NO|CMG6>Xl(D
zxf>S-7ZB#%g8*qz`dkD)fPg4iC#!h}!P#K#!g)`7_6y0%kB8=c2ah#{Fr#cwprFuS
zWB%VG3TPD1`xk=dB3L~#?_DqQLKG~+kdZd;_g>^^6?sHr09=sbMLuYv_X3>jMMhxf
z!CXUMOnt#xzu;<c6*0(ve*Np2-@C;nVgVWzFbwY)0*i~VxS$byE`+~Cl#hYSpyR><
zJhZ+)gm;seq9M$N#d<G<@z6HG>d}H2%i=u^S|AGi2wWe+{~G~K;Gn_ZAfOC9rV9Sv
zv$ul%E<@q@r0&&>>%fEa=U~7vSPhzAioum&@fhvJ=6{M*c$R{CBXB-JagRi~ctiwl
z0ZWiY6?ecdT^8`~2cHh{V`2NT5dTY_?Vqj({I5fwmLId`pMxPJpwI?~1Qe1Nc7_xz
zyb}7f0IP5{xMI4&!q34UQm}A4I4r-{v%fqgkMUplI1FKdWWt?cflRMo4h#G_I4tl2
z_}H`p3(`EhZK?swTCf;;NdAKBz=8Z2E?%&K3ZVuTG=oD8EMQlxE!@C>XYU9%@J;C9
z23T~q1r3ON63>pXd?vFUDnBND9A!1RBh)}N-?O)e8i<m5cZ3><t_6o0h~5T&sDUV#
zTDNNhzWfKlVfjOzeYcU1|3v{(=k{;|Og7uY4X}c_JKO+Ke|M;Xg&E*b0}CY+h74GE
zJ@k<Lg|~uz`8xkE+(w0V`cR{EVHeYF4OsKS!UND>neO9-M>zKX%v~}IM|tK>@&tc!
z*UZB2f@Pcs`1>ED|H#mo|Nck;Z>WGjw*Jpdef<ANDxlW>(e=;!3ebL_fGPNY<NsjM
zSO>X+yQj!SMK=DGK9{*$FiWm`1xFV8F;(<Q)(aOxm_&E)2snYXN6ZziBC&1u+t-2(
zUgU`}7_9kzHZ3BNbe;}X_#y&M3f2ZlES-M<2MyjI+%l&M4tkNpzELXg8(!oa=sYE#
zOTV450THMQ9P=XPjy@H@T+lfYqW_%7JnPBzwJMO=arbc+JYC9v!Hbyt?hIbUqV%ck
z=>``uJ3dt=I0L1*wQK>`|DR|NF_g-arQie=z;wB-ig}{cG`Q$m*8SpA^*}nyZf6i&
zr~EZ|2!*U{<Do_CFt-alp$&39;8nrJ%6}Wz|M$skRIUXqKw?p_Hn524b#E(pCLZGd
zMSY%?Tn4UFfiC=VH=nHyK8XTA>kG!Ci$>-CQ}DRT9}DhWRQwwAe`gyEX<Fev8vHWe
zN!145#^ntnQ9bZ$c^ns*tO7oyPtQ!#i&?U@io)8Z#i^b(V~nMOXA~~Z_99)Uu;=V-
zKgNn<g<j-Ie<+y-is^!T$GHz6Ss7Tvb@n1K*;rUiCX2}^7&8jXC7V^L-~uoUl47E|
zQxa96z^y1a237-#S<u9afmudgtoU};@mTEp7a7IPWWGZ{5g>S{JZCJ}SX{h|1|+Do
zi;5p&)HH#E2Dpdh@Nfu!4gN{6j*{Z1JR4u|$G<bke~W(LSxF<{N_7!8nLH^$7zku;
zh@}7n0Vlw<;K0Hk)4&<Q>Y-og8c-Twjad9|!~{V*L#$!k7hj}4>{ha1juz7=hsUJE
zbiv~JEUBn#Mfryci<hvft&%`C9$8$@5=z(u{3&@jn=Pbe#zTvl&O0wcR)rU{YnCLV
zrotGwc!xZ#4X%M-=AXs)d6CD5P&Ppoh?AMl!=M%}-iO8&;0$BoVs6VZcMEHSe+OLz
z1`RU*cm6(vU&EtEz=45pdII>%!2hBEi%j2QxiJ-s<+@+BbU_0a6I6@e#e#F-RJDNV
zxAUyjHwu;Tio7QWz}i5`Ea>^bKQ%5dAxJxa304J57EHwe{-=*hmXSKkV9>-+!kVu0
zHL0Ki8i?W%aL@qL^*y4HdZ^?t<uQ10t?@`nkKv!uZaiATsa$_AhIAp1|4TR<zPA<r
zQn~*Ei%Sj@#RU`u4SWli3Jx0JPNyfzL-fZ2kKqxi|2-7`5PD#txV$mM|0eZ^LwFMY
zAA@6oh3`=?PJu>o376T}f1Xlc31>Ry{%&L85_0!9PEHG4Lf$wmF`x!4Vae8&3RL+^
zByR}TE?aVwIl0CE#)2jF^40+us<eU6qR=U@dSJ;Hz|vqKa62v^0c$2)vdfG72?p3>
zTK{1ZnaFE2iEIfgsC#4-1oFO)f!~qye|6cCXK+~p41oos;I0sUkvo}jsoju49m`rW
z?nSmTN+LC;kC*&}*$vWE{r`!_8i2j3(0LNuAXpUO|5DB~I{QDV1@vL*Wo#IT#o`g7
zzEq+%7ibi`>&tCd9)0*#=jNG5&;GOXg`}g~Uw1a9y3Q}yW~*4?CQLfp-Jks2S?2DX
zbT->7u68FToulp(zjRj4-_f@1^V{#b^YdHo`ir)!mvi}Z8eX-+{nuYQx4GYb%gJ(Y
z-Itu{Zg|Vt<eqrTS$wqWZRh*RGe6(<#rC$NGd<_U8Scx!<5y3fDsxZ&&MApBxgY(`
zS>T|oozZ1k)u-Ij^G>0I3H-7r3BJj#7tCGs_LfnxN^uSNL<l#4#q5B-89X_K6SuU1
z&%4?0IYo)wL2qwOa4R`>ws;)3PV;R=kF8y9pU_Aq_SSxOK9b#I?>U8uY@^s+gYJaL
z=!$YY<R-i?$E6$(yUXO5lxlakx()C101Dq7d?$9F;^yeb-C?2sU<Ylrx+nePUswrF
zx_jY$XJJy(YyZZc&Y&Cpz*#sW>Fs{E<^yMOQqns#f9elz>jymh@!~`7zz2wZam+o=
zcj8OuiJBYTv(OR~uIAi&*v+`Warp#Gj0U%Y<103mS=N<RG`Rr1?&MoSGlP#zMMQ`G
aIeS`(l&ITFot(d=&B%s;#5ZATS^o##Cv5!y

diff --git a/roms/seabios-hppa b/roms/seabios-hppa
index bf3404006f..17ca7a9998 160000
--- a/roms/seabios-hppa
+++ b/roms/seabios-hppa
@@ -1 +1 @@
-Subproject commit bf3404006fd2c832857eb57e6f853862f97dacea
+Subproject commit 17ca7a9998c1755d42321cfc0afb5f480f5a58ff

From 7e50730cb8246c33c4f79fdb130d56231ee4a67b Mon Sep 17 00:00:00 2001
From: Helge Deller <deller@gmx.de>
Date: Wed, 11 May 2022 11:34:15 +0200
Subject: [PATCH 395/935] artist: Introduce constant for max cursor size

Add the constant NGLE_MAX_SPRITE_SIZE which defines the currently
maximum supported cursor size.

Signed-off-by: Helge Deller <deller@gmx.de>
Acked-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/display/artist.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/hw/display/artist.c b/hw/display/artist.c
index 39fc0c4ca5..6333ee41db 100644
--- a/hw/display/artist.c
+++ b/hw/display/artist.c
@@ -98,6 +98,9 @@ struct ARTISTState {
     int draw_line_pattern;
 };
 
+/* hardware allows up to 64x64, but we emulate 32x32 only. */
+#define NGLE_MAX_SPRITE_SIZE    32
+
 typedef enum {
     ARTIST_BUFFER_AP = 1,
     ARTIST_BUFFER_OVERLAY = 2,
@@ -1325,11 +1328,10 @@ static void artist_realizefn(DeviceState *dev, Error **errp)
     framebuffer_update_memory_section(&s->fbsection, &buf->mr, 0,
                                       buf->width, buf->height);
     /*
-     * no idea whether the cursor is fixed size or not, so assume 32x32 which
-     * seems sufficient for HP-UX X11.
+     * Artist cursor max size
      */
-    s->cursor_height = 32;
-    s->cursor_width = 32;
+    s->cursor_height = NGLE_MAX_SPRITE_SIZE;
+    s->cursor_width = NGLE_MAX_SPRITE_SIZE;
 
     /*
      * These two registers are not initialized by seabios's STI implementation.

From e9683fbc3738ebbf9b4058413ffc53e086da95fc Mon Sep 17 00:00:00 2001
From: Helge Deller <deller@gmx.de>
Date: Wed, 11 May 2022 11:41:19 +0200
Subject: [PATCH 396/935] artist: Use human-readable variable names instead of
 reg_xxx

Convert the variable names of some registers to human-readable and
understandable names.

Signed-off-by: Helge Deller <deller@gmx.de>
Acked-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/display/artist.c | 72 ++++++++++++++++++++++++++++++---------------
 1 file changed, 49 insertions(+), 23 deletions(-)

diff --git a/hw/display/artist.c b/hw/display/artist.c
index 6333ee41db..c8b261a52e 100644
--- a/hw/display/artist.c
+++ b/hw/display/artist.c
@@ -81,9 +81,10 @@ struct ARTISTState {
     uint32_t plane_mask;
 
     uint32_t reg_100080;
-    uint32_t reg_300200;
-    uint32_t reg_300208;
-    uint32_t reg_300218;
+    uint32_t horiz_backporch;
+    uint32_t active_lines_low;
+    uint32_t misc_video;
+    uint32_t misc_ctrl;
 
     uint32_t dst_bm_access;
     uint32_t src_bm_access;
@@ -138,8 +139,14 @@ typedef enum {
     BG_COLOR = 0x118014,
     PLANE_MASK = 0x118018,
     IMAGE_BITMAP_OP = 0x11801c,
-    CURSOR_POS = 0x300100,
-    CURSOR_CTRL = 0x300104,
+    CURSOR_POS = 0x300100,      /* reg17 */
+    CURSOR_CTRL = 0x300104,     /* reg18 */
+    MISC_VIDEO = 0x300218,      /* reg21 */
+    MISC_CTRL = 0x300308,       /* reg27 */
+    HORIZ_BACKPORCH = 0x300200, /* reg19 */
+    ACTIVE_LINES_LOW = 0x300208,/* reg20 */
+    FIFO1 = 0x300008,           /* reg34 */
+    FIFO2 = 0x380008,
 } artist_reg_t;
 
 typedef enum {
@@ -177,12 +184,18 @@ static const char *artist_reg_name(uint64_t addr)
     REG_NAME(SRC_BM_ACCESS);
     REG_NAME(CURSOR_POS);
     REG_NAME(CURSOR_CTRL);
+    REG_NAME(HORIZ_BACKPORCH);
+    REG_NAME(ACTIVE_LINES_LOW);
+    REG_NAME(MISC_VIDEO);
+    REG_NAME(MISC_CTRL);
     REG_NAME(LINE_XY);
     REG_NAME(PATTERN_LINE_START);
     REG_NAME(LINE_SIZE);
     REG_NAME(LINE_END);
     REG_NAME(FONT_WRITE_INCR_Y);
     REG_NAME(FONT_WRITE_START);
+    REG_NAME(FIFO1);
+    REG_NAME(FIFO2);
     }
     return "";
 }
@@ -1028,16 +1041,20 @@ static void artist_reg_write(void *opaque, hwaddr addr, uint64_t val,
         combine_write_reg(addr, val, size, &s->transfer_data);
         break;
 
-    case 0x300200:
-        combine_write_reg(addr, val, size, &s->reg_300200);
+    case HORIZ_BACKPORCH:
+        combine_write_reg(addr, val, size, &s->horiz_backporch);
         break;
 
-    case 0x300208:
-        combine_write_reg(addr, val, size, &s->reg_300208);
+    case ACTIVE_LINES_LOW:
+        combine_write_reg(addr, val, size, &s->active_lines_low);
         break;
 
-    case 0x300218:
-        combine_write_reg(addr, val, size, &s->reg_300218);
+    case MISC_VIDEO:
+        combine_write_reg(addr, val, size, &s->misc_video);
+        break;
+
+    case MISC_CTRL:
+        combine_write_reg(addr, val, size, &s->misc_ctrl);
         break;
 
     case CURSOR_POS:
@@ -1122,12 +1139,11 @@ static uint64_t artist_reg_read(void *opaque, hwaddr addr, unsigned size)
     case 0x100000:
     case 0x300000:
     case 0x300004:
-    case 0x300308:
     case 0x380000:
         break;
 
-    case 0x300008:
-    case 0x380008:
+    case FIFO1:
+    case FIFO2:
         /*
          * FIFO ready flag. we're not emulating the FIFOs
          * so we're always ready
@@ -1135,16 +1151,25 @@ static uint64_t artist_reg_read(void *opaque, hwaddr addr, unsigned size)
         val = 0x10;
         break;
 
-    case 0x300200:
-        val = s->reg_300200;
+    case HORIZ_BACKPORCH:
+        val = s->horiz_backporch;
         break;
 
-    case 0x300208:
-        val = s->reg_300208;
+    case ACTIVE_LINES_LOW:
+        val = s->active_lines_low;
+        /* activeLinesLo for cursor is in reg20.b.b0 */
+        val |= ((s->height - 1) & 0xff);
         break;
 
-    case 0x300218:
-        val = s->reg_300218;
+    case MISC_VIDEO:
+        /* emulate V-blank */
+        val = s->misc_video ^ 0x00040000;
+        /* activeLinesHi for cursor is in reg21.b.b2 */
+        val |= ((s->height - 1) & 0xff00);
+        break;
+
+    case MISC_CTRL:
+        val = s->misc_ctrl;
         break;
 
     case 0x30023c:
@@ -1379,9 +1404,10 @@ static const VMStateDescription vmstate_artist = {
         VMSTATE_UINT32(cursor_width, ARTISTState),
         VMSTATE_UINT32(plane_mask, ARTISTState),
         VMSTATE_UINT32(reg_100080, ARTISTState),
-        VMSTATE_UINT32(reg_300200, ARTISTState),
-        VMSTATE_UINT32(reg_300208, ARTISTState),
-        VMSTATE_UINT32(reg_300218, ARTISTState),
+        VMSTATE_UINT32(horiz_backporch, ARTISTState),
+        VMSTATE_UINT32(active_lines_low, ARTISTState),
+        VMSTATE_UINT32(misc_video, ARTISTState),
+        VMSTATE_UINT32(misc_ctrl, ARTISTState),
         VMSTATE_UINT32(dst_bm_access, ARTISTState),
         VMSTATE_UINT32(src_bm_access, ARTISTState),
         VMSTATE_UINT32(control_plane, ARTISTState),

From 482afe020b558d36d3b863970ad1672981b22d48 Mon Sep 17 00:00:00 2001
From: Helge Deller <deller@gmx.de>
Date: Wed, 11 May 2022 11:49:02 +0200
Subject: [PATCH 397/935] artist: Fix vertical X11 cursor position in HP-UX

Drop the hard-coded value of 1146 lines which seems to work with HP-UX
11, but not with HP-UX 10. Instead encode the screen height in byte 0 of
active_lines_low and byte 3 of misc_video as it's expected by the Xorg
X11 graphics driver.

This potentially allows for higher vertical screen resolutions than
1280x1024 with X11.

Signed-off-by: Helge Deller <deller@gmx.de>
Acked-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/display/artist.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/hw/display/artist.c b/hw/display/artist.c
index c8b261a52e..780cb15026 100644
--- a/hw/display/artist.c
+++ b/hw/display/artist.c
@@ -337,10 +337,11 @@ static void artist_get_cursor_pos(ARTISTState *s, int *x, int *y)
     }
     *x = (lx - offset) / 2;
 
-    *y = 1146 - artist_get_y(s->cursor_pos);
-
     /* subtract cursor offset from cursor control register */
     *x -= (s->cursor_cntrl & 0xf0) >> 4;
+
+    /* height minus nOffscreenScanlines is stored in cursor control register */
+    *y = s->height - artist_get_y(s->cursor_pos);
     *y -= (s->cursor_cntrl & 0x0f);
 
     if (*x > s->width) {
@@ -1158,14 +1159,17 @@ static uint64_t artist_reg_read(void *opaque, hwaddr addr, unsigned size)
     case ACTIVE_LINES_LOW:
         val = s->active_lines_low;
         /* activeLinesLo for cursor is in reg20.b.b0 */
-        val |= ((s->height - 1) & 0xff);
+        val &= ~(0xff << 24);
+        val |= (s->height & 0xff) << 24;
         break;
 
     case MISC_VIDEO:
         /* emulate V-blank */
-        val = s->misc_video ^ 0x00040000;
+        s->misc_video ^= 0x00040000;
         /* activeLinesHi for cursor is in reg21.b.b2 */
-        val |= ((s->height - 1) & 0xff00);
+        val = s->misc_video;
+        val &= ~0xff00UL;
+        val |= (s->height & 0xff00);
         break;
 
     case MISC_CTRL:

From a377b574eb2cbcbff3fc579529e0f1b2fbda8af8 Mon Sep 17 00:00:00 2001
From: Helge Deller <deller@gmx.de>
Date: Wed, 11 May 2022 12:28:30 +0200
Subject: [PATCH 398/935] artist: Allow to turn cursor on or off

Bit 0x80 in the cursor_cntrl register specifies if the cursor
should be visible. Prevent rendering the cursor if it's invisible.

Signed-off-by: Helge Deller <deller@gmx.de>
Acked-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/display/artist.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/hw/display/artist.c b/hw/display/artist.c
index 780cb15026..b8930b7c5a 100644
--- a/hw/display/artist.c
+++ b/hw/display/artist.c
@@ -353,10 +353,20 @@ static void artist_get_cursor_pos(ARTISTState *s, int *x, int *y)
     }
 }
 
+static inline bool cursor_visible(ARTISTState *s)
+{
+    /* cursor is visible if bit 0x80 is set in cursor_cntrl */
+    return s->cursor_cntrl & 0x80;
+}
+
 static void artist_invalidate_cursor(ARTISTState *s)
 {
     int x, y;
 
+    if (!cursor_visible(s)) {
+        return;
+    }
+
     artist_get_cursor_pos(s, &x, &y);
     artist_invalidate_lines(&s->vram_buffer[ARTIST_BUFFER_AP],
                             y, s->cursor_height);
@@ -1218,6 +1228,10 @@ static void artist_draw_cursor(ARTISTState *s)
     struct vram_buffer *cursor0, *cursor1 , *buf;
     int cx, cy, cursor_pos_x, cursor_pos_y;
 
+    if (!cursor_visible(s)) {
+        return;
+    }
+
     cursor0 = &s->vram_buffer[ARTIST_BUFFER_CURSOR1];
     cursor1 = &s->vram_buffer[ARTIST_BUFFER_CURSOR2];
     buf = &s->vram_buffer[ARTIST_BUFFER_AP];

From caca6e618d5577e45be2bcdc1e626a8b747fb56b Mon Sep 17 00:00:00 2001
From: Helge Deller <deller@gmx.de>
Date: Wed, 11 May 2022 12:59:47 +0200
Subject: [PATCH 399/935] artist: Emulate screen blanking

The misc_video and misc_ctrl registers control the visibility of the
screen. Start with the screen turned on, and hide or show the screen
based on the control registers.

Signed-off-by: Helge Deller <deller@gmx.de>
Acked-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/display/artist.c | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/hw/display/artist.c b/hw/display/artist.c
index b8930b7c5a..49dad2b824 100644
--- a/hw/display/artist.c
+++ b/hw/display/artist.c
@@ -201,6 +201,8 @@ static const char *artist_reg_name(uint64_t addr)
 }
 #undef REG_NAME
 
+static void artist_invalidate(void *opaque);
+
 /* artist has a fixed line length of 2048 bytes. */
 #define ADDR_TO_Y(addr) extract32(addr, 11, 11)
 #define ADDR_TO_X(addr) extract32(addr, 0, 11)
@@ -903,6 +905,7 @@ static void artist_reg_write(void *opaque, hwaddr addr, uint64_t val,
 {
     ARTISTState *s = opaque;
     int width, height;
+    uint64_t oldval;
 
     trace_artist_reg_write(size, addr, artist_reg_name(addr & ~3ULL), val);
 
@@ -1061,7 +1064,18 @@ static void artist_reg_write(void *opaque, hwaddr addr, uint64_t val,
         break;
 
     case MISC_VIDEO:
+        oldval = s->misc_video;
         combine_write_reg(addr, val, size, &s->misc_video);
+        /* Invalidate and hide screen if graphics signal is turned off. */
+        if (((oldval & 0x0A000000) == 0x0A000000) &&
+            ((val & 0x0A000000) != 0x0A000000)) {
+            artist_invalidate(s);
+        }
+        /* Invalidate and redraw screen if graphics signal is turned back on. */
+        if (((oldval & 0x0A000000) != 0x0A000000) &&
+            ((val & 0x0A000000) == 0x0A000000)) {
+            artist_invalidate(s);
+        }
         break;
 
     case MISC_CTRL:
@@ -1263,6 +1277,12 @@ static void artist_draw_cursor(ARTISTState *s)
     }
 }
 
+static bool artist_screen_enabled(ARTISTState *s)
+{
+    /*  We could check for (s->misc_ctrl & 0x00800000) too... */
+    return ((s->misc_video & 0x0A000000) == 0x0A000000);
+}
+
 static void artist_draw_line(void *opaque, uint8_t *d, const uint8_t *src,
                              int width, int pitch)
 {
@@ -1270,6 +1290,12 @@ static void artist_draw_line(void *opaque, uint8_t *d, const uint8_t *src,
     uint32_t *cmap, *data = (uint32_t *)d;
     int x;
 
+    if (!artist_screen_enabled(s)) {
+        /* clear screen */
+        memset(data, 0, s->width * sizeof(uint32_t));
+        return;
+    }
+
     cmap = (uint32_t *)(s->vram_buffer[ARTIST_BUFFER_CMAP].data + 0x400);
 
     for (x = 0; x < s->width; x++) {
@@ -1384,6 +1410,10 @@ static void artist_realizefn(DeviceState *dev, Error **errp)
     s->image_bitmap_op = 0x23000300;
     s->plane_mask = 0xff;
 
+    /* enable screen */
+    s->misc_video |= 0x0A000000;
+    s->misc_ctrl  |= 0x00800000;
+
     s->con = graphic_console_init(dev, 0, &artist_ops, s);
     qemu_console_resize(s->con, s->width, s->height);
 }

From 9ef2c6b4ab9a430ee230ee305b6994d7eae86fe7 Mon Sep 17 00:00:00 2001
From: Helge Deller <deller@gmx.de>
Date: Thu, 12 May 2022 13:40:39 +0200
Subject: [PATCH 400/935] artist: Fix X cursor position calculation in X11

The X cursor postion can be calculated based on the backporch and
interleave values.  In the emulation we ignore the HP-UX settings for
backporch and use instead twice the size of the emulated cursor.  With
those changes the X-position of the graphics cursor is now finally
working correctly on HP-UX 10 and HP-UX 11.

Based on coding in Xorg X11R6.6

Signed-off-by: Helge Deller <deller@gmx.de>
---
 hw/display/artist.c | 38 ++++++++++++++++++++++----------------
 1 file changed, 22 insertions(+), 16 deletions(-)

diff --git a/hw/display/artist.c b/hw/display/artist.c
index 49dad2b824..eadaef0d46 100644
--- a/hw/display/artist.c
+++ b/hw/display/artist.c
@@ -1,7 +1,8 @@
 /*
  * QEMU HP Artist Emulation
  *
- * Copyright (c) 2019 Sven Schnelle <svens@stackframe.org>
+ * Copyright (c) 2019-2022 Sven Schnelle <svens@stackframe.org>
+ * Copyright (c) 2022 Helge Deller <deller@gmx.de>
  *
  * This work is licensed under the terms of the GNU GPL, version 2 or later.
  */
@@ -313,19 +314,15 @@ static void artist_rop8(ARTISTState *s, struct vram_buffer *buf,
 static void artist_get_cursor_pos(ARTISTState *s, int *x, int *y)
 {
     /*
-     * Don't know whether these magic offset values are configurable via
-     * some register. They seem to be the same for all resolutions.
-     * The cursor values provided in the registers are:
-     * X-value: -295 (for HP-UX 11) and 338 (for HP-UX 10.20) up to 2265
-     * Y-value: 1146 down to 0
      * The emulated Artist graphic is like a CRX graphic, and as such
      * it's usually fixed at 1280x1024 pixels.
-     * Because of the maximum Y-value of 1146 you can not choose a higher
-     * vertical resolution on HP-UX (unless you disable the mouse).
+     * Other resolutions may work, but no guarantee.
      */
 
-    static int offset = 338;
-    int lx;
+    unsigned int hbp_times_vi, horizBackPorch;
+    int16_t xHi, xLo;
+    const int videoInterleave = 4;
+    const int pipelineDelay = 4;
 
     /* ignore if uninitialized */
     if (s->cursor_pos == 0) {
@@ -333,16 +330,23 @@ static void artist_get_cursor_pos(ARTISTState *s, int *x, int *y)
         return;
     }
 
-    lx = artist_get_x(s->cursor_pos);
-    if (lx < offset) {
-        offset = lx;
-    }
-    *x = (lx - offset) / 2;
+    /*
+     * Calculate X position based on backporch and interleave values.
+     * Based on code from Xorg X11R6.6
+     */
+    horizBackPorch = ((s->horiz_backporch & 0xff0000) >> 16) +
+                     ((s->horiz_backporch & 0xff00) >> 8) + 2;
+    hbp_times_vi = horizBackPorch * videoInterleave;
+    xHi = s->cursor_pos >> 19;
+    *x = ((xHi + pipelineDelay) * videoInterleave) - hbp_times_vi;
+
+    xLo = (s->cursor_pos >> 16) & 0x07;
+    *x += ((xLo - hbp_times_vi) & (videoInterleave - 1)) + 8 - 1;
 
     /* subtract cursor offset from cursor control register */
     *x -= (s->cursor_cntrl & 0xf0) >> 4;
 
-    /* height minus nOffscreenScanlines is stored in cursor control register */
+    /* Calculate Y position */
     *y = s->height - artist_get_y(s->cursor_pos);
     *y -= (s->cursor_cntrl & 0x0f);
 
@@ -1056,6 +1060,8 @@ static void artist_reg_write(void *opaque, hwaddr addr, uint64_t val,
         break;
 
     case HORIZ_BACKPORCH:
+        /* overwrite HP-UX settings to fix X cursor position. */
+        val = (NGLE_MAX_SPRITE_SIZE << 16) + (NGLE_MAX_SPRITE_SIZE << 8);
         combine_write_reg(addr, val, size, &s->horiz_backporch);
         break;
 

From 37cbfcebdd0e7ffbfb8332a8794a91a4a8f1819d Mon Sep 17 00:00:00 2001
From: "Michael S. Tsirkin" <mst@redhat.com>
Date: Fri, 13 May 2022 06:56:23 -0400
Subject: [PATCH 401/935] vhost-user: more master/slave things

we switched to front-end/back-end, but newer patches
reintroduced old language. Fix this up.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 docs/interop/vhost-user.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
index 73e710fe32..3c12607517 100644
--- a/docs/interop/vhost-user.rst
+++ b/docs/interop/vhost-user.rst
@@ -340,7 +340,7 @@ drivers cannot negotiate it.
 
 This reserved feature bit was reused by the vhost-user protocol to add
 vhost-user protocol feature negotiation in a backwards compatible
-fashion. Old vhost-user master and slave implementations continue to
+fashion. Old vhost-user front-end and back-end implementations continue to
 work even though they are not aware of vhost-user protocol feature
 negotiation.
 

From 76b1b64370007234279ea4cc8b09c98cbd2523de Mon Sep 17 00:00:00 2001
From: Kevin Wolf <kwolf@redhat.com>
Date: Thu, 7 Apr 2022 15:36:55 +0200
Subject: [PATCH 402/935] docs/vhost-user: Clarifications for
 VHOST_USER_ADD/REM_MEM_REG

The specification for VHOST_USER_ADD/REM_MEM_REG messages is unclear
in several points, which has led to clients having incompatible
implementations. This changes the specification to be more explicit
about them:

* VHOST_USER_ADD_MEM_REG is not specified as receiving a file
  descriptor, though it obviously does need to do so. All
  implementations agree on this one, fix the specification.

* VHOST_USER_REM_MEM_REG is not specified as receiving a file
  descriptor either, and it also has no reason to do so. rust-vmm does
  not send file descriptors for removing a memory region (in agreement
  with the specification), libvhost-user and QEMU do (which is a bug),
  though libvhost-user doesn't actually make any use of it.

  Change the specification so that for compatibility QEMU's behaviour
  becomes legal, even if discouraged, but rust-vmm's behaviour becomes
  the explicitly recommended mode of operation.

* VHOST_USER_ADD_MEM_REG doesn't have a documented return value, which
  is the desired behaviour in the non-postcopy case. It also implemented
  like this in QEMU and rust-vmm, though libvhost-user is buggy and
  sometimes sends an unexpected reply. This will be fixed in a separate
  patch.

  However, in postcopy mode it does reply like VHOST_USER_SET_MEM_TABLE.
  This behaviour is shared between libvhost-user and QEMU; rust-vmm
  doesn't implement postcopy mode yet. Mention it explicitly in the
  spec.

* The specification doesn't mention how VHOST_USER_REM_MEM_REG
  identifies the memory region to be removed. Change it to describe the
  existing behaviour of libvhost-user (guest address, user address and
  size must match).

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20220407133657.155281-2-kwolf@redhat.com>
Reviewed-by: Raphael Norwitz <raphael.norwitz@nutanix.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 docs/interop/vhost-user.rst | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
index 3c12607517..a99ba4433c 100644
--- a/docs/interop/vhost-user.rst
+++ b/docs/interop/vhost-user.rst
@@ -1376,6 +1376,14 @@ Front-end message types
   For further details on postcopy, see ``VHOST_USER_SET_MEM_TABLE``.
   They apply to ``VHOST_USER_ADD_MEM_REG`` accordingly.
 
+  Exactly one file descriptor from which the memory is mapped is
+  passed in the ancillary data.
+
+  In postcopy mode (see ``VHOST_USER_POSTCOPY_LISTEN``), the back-end
+  replies with the bases of the memory mapped region to the front-end.
+  For further details on postcopy, see ``VHOST_USER_SET_MEM_TABLE``.
+  They apply to ``VHOST_USER_ADD_MEM_REG`` accordingly.
+
 ``VHOST_USER_REM_MEM_REG``
   :id: 38
   :equivalent ioctl: N/A
@@ -1400,6 +1408,14 @@ Front-end message types
   accept messages with one file descriptor. If a file descriptor is
   passed, the back-end MUST close it without using it otherwise.
 
+  The memory region to be removed is identified by its guest address,
+  user address and size. The mmap offset is ignored.
+
+  No file descriptors SHOULD be passed in the ancillary data. For
+  compatibility with existing incorrect implementations, the back-end MAY
+  accept messages with one file descriptor. If a file descriptor is
+  passed, the back-end MUST close it without using it otherwise.
+
 ``VHOST_USER_SET_STATUS``
   :id: 39
   :equivalent ioctl: VHOST_VDPA_SET_STATUS

From 8e58f6ec24f64b7d88f2b01d39049011f5097fae Mon Sep 17 00:00:00 2001
From: Francisco Iglesias <frasse.iglesias@gmail.com>
Date: Tue, 12 Apr 2022 00:18:35 +0200
Subject: [PATCH 403/935] include/hw/pci/pcie_host: Correct PCIE_MMCFG_BUS_MASK
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

According to [1] address bits 27 - 20 are mapped to the bus number (the
TLPs bus number field is 8 bits). Below is the formula taken from Table
7-1 in [1].

"
Memory Address | PCI Express Configuration Space
A[(20+n-1):20] | Bus Number, 1 ≤ n ≤ 8
"

[1] PCI Express® Base Specification Revision 5.0 Version 1.0

Signed-off-by: Francisco Iglesias <frasse.iglesias@gmail.com>
Message-Id: <20220411221836.17699-2-frasse.iglesias@gmail.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/pci/pcie_host.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/hw/pci/pcie_host.h b/include/hw/pci/pcie_host.h
index 076457b270..b3c8ce973c 100644
--- a/include/hw/pci/pcie_host.h
+++ b/include/hw/pci/pcie_host.h
@@ -60,7 +60,7 @@ void pcie_host_mmcfg_update(PCIExpressHost *e,
 /*
  * PCI express ECAM (Enhanced Configuration Address Mapping) format.
  * AKA mmcfg address
- * bit 20 - 28: bus number
+ * bit 20 - 27: bus number
  * bit 15 - 19: device number
  * bit 12 - 14: function number
  * bit  0 - 11: offset in configuration space of a given device
@@ -68,7 +68,7 @@ void pcie_host_mmcfg_update(PCIExpressHost *e,
 #define PCIE_MMCFG_SIZE_MAX             (1ULL << 29)
 #define PCIE_MMCFG_SIZE_MIN             (1ULL << 20)
 #define PCIE_MMCFG_BUS_BIT              20
-#define PCIE_MMCFG_BUS_MASK             0x1ff
+#define PCIE_MMCFG_BUS_MASK             0xff
 #define PCIE_MMCFG_DEVFN_BIT            12
 #define PCIE_MMCFG_DEVFN_MASK           0xff
 #define PCIE_MMCFG_CONFOFFSET_MASK      0xfff

From 1f1a7b226923c655530d6bbe7d85d87e3df2d6f1 Mon Sep 17 00:00:00 2001
From: Francisco Iglesias <frasse.iglesias@gmail.com>
Date: Tue, 12 Apr 2022 00:18:36 +0200
Subject: [PATCH 404/935] include/hw/pci/pcie_host: Correct PCIE_MMCFG_SIZE_MAX
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

According to 7.2.2 in [1] bit 27 is the last bit that can be part of the
bus number, this makes the ECAM max size equal to '1 << 28'. This patch
restores back this value into the PCIE_MMCFG_SIZE_MAX define (which was
changed in commit 58d5b22bbd5 ("ppc4xx: Add device models found in PPC440
core SoCs")).

[1] PCI Express® Base Specification Revision 5.0 Version 1.0

Signed-off-by: Francisco Iglesias <frasse.iglesias@gmail.com>
Message-Id: <20220411221836.17699-3-frasse.iglesias@gmail.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/pci/pcie_host.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/hw/pci/pcie_host.h b/include/hw/pci/pcie_host.h
index b3c8ce973c..82d92177da 100644
--- a/include/hw/pci/pcie_host.h
+++ b/include/hw/pci/pcie_host.h
@@ -65,7 +65,7 @@ void pcie_host_mmcfg_update(PCIExpressHost *e,
  * bit 12 - 14: function number
  * bit  0 - 11: offset in configuration space of a given device
  */
-#define PCIE_MMCFG_SIZE_MAX             (1ULL << 29)
+#define PCIE_MMCFG_SIZE_MAX             (1ULL << 28)
 #define PCIE_MMCFG_SIZE_MIN             (1ULL << 20)
 #define PCIE_MMCFG_BUS_BIT              20
 #define PCIE_MMCFG_BUS_MASK             0xff

From 9dee7e510969787be49771e4653a123a26d1069f Mon Sep 17 00:00:00 2001
From: Xiaoyao Li <xiaoyao.li@intel.com>
Date: Thu, 10 Mar 2022 20:28:10 +0800
Subject: [PATCH 405/935] hw/i386: Make pit a property of common x86 base
 machine type

Both pc and microvm have pit property individually. Let's just make it
the property of common x86 base machine type.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Sergio Lopez <slp@redhat.com>
Message-Id: <20220310122811.807794-2-xiaoyao.li@intel.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/microvm.c         | 27 +--------------------------
 hw/i386/pc.c              | 24 +++---------------------
 hw/i386/x86.c             | 25 +++++++++++++++++++++++++
 include/hw/i386/microvm.h |  2 --
 include/hw/i386/pc.h      |  2 --
 include/hw/i386/x86.h     |  2 ++
 6 files changed, 31 insertions(+), 51 deletions(-)

diff --git a/hw/i386/microvm.c b/hw/i386/microvm.c
index 4b3b1dd262..89b555a2f5 100644
--- a/hw/i386/microvm.c
+++ b/hw/i386/microvm.c
@@ -257,7 +257,7 @@ static void microvm_devices_init(MicrovmMachineState *mms)
         g_free(i8259);
     }
 
-    if (mms->pit == ON_OFF_AUTO_ON || mms->pit == ON_OFF_AUTO_AUTO) {
+    if (x86ms->pit == ON_OFF_AUTO_ON || x86ms->pit == ON_OFF_AUTO_AUTO) {
         if (kvm_pit_in_kernel()) {
             kvm_pit_init(isa_bus, 0x40);
         } else {
@@ -508,23 +508,6 @@ static void microvm_machine_set_pic(Object *obj, Visitor *v, const char *name,
     visit_type_OnOffAuto(v, name, &mms->pic, errp);
 }
 
-static void microvm_machine_get_pit(Object *obj, Visitor *v, const char *name,
-                                    void *opaque, Error **errp)
-{
-    MicrovmMachineState *mms = MICROVM_MACHINE(obj);
-    OnOffAuto pit = mms->pit;
-
-    visit_type_OnOffAuto(v, name, &pit, errp);
-}
-
-static void microvm_machine_set_pit(Object *obj, Visitor *v, const char *name,
-                                    void *opaque, Error **errp)
-{
-    MicrovmMachineState *mms = MICROVM_MACHINE(obj);
-
-    visit_type_OnOffAuto(v, name, &mms->pit, errp);
-}
-
 static void microvm_machine_get_rtc(Object *obj, Visitor *v, const char *name,
                                     void *opaque, Error **errp)
 {
@@ -650,7 +633,6 @@ static void microvm_machine_initfn(Object *obj)
 
     /* Configuration */
     mms->pic = ON_OFF_AUTO_AUTO;
-    mms->pit = ON_OFF_AUTO_AUTO;
     mms->rtc = ON_OFF_AUTO_AUTO;
     mms->pcie = ON_OFF_AUTO_AUTO;
     mms->ioapic2 = ON_OFF_AUTO_AUTO;
@@ -709,13 +691,6 @@ static void microvm_class_init(ObjectClass *oc, void *data)
     object_class_property_set_description(oc, MICROVM_MACHINE_PIC,
         "Enable i8259 PIC");
 
-    object_class_property_add(oc, MICROVM_MACHINE_PIT, "OnOffAuto",
-                              microvm_machine_get_pit,
-                              microvm_machine_set_pit,
-                              NULL, NULL);
-    object_class_property_set_description(oc, MICROVM_MACHINE_PIT,
-        "Enable i8254 PIT");
-
     object_class_property_add(oc, MICROVM_MACHINE_RTC, "OnOffAuto",
                               microvm_machine_get_rtc,
                               microvm_machine_set_rtc,
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 15f37d8dc6..7c39c91335 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1123,6 +1123,7 @@ void pc_basic_device_init(struct PCMachineState *pcms,
     ISADevice *pit = NULL;
     MemoryRegion *ioport80_io = g_new(MemoryRegion, 1);
     MemoryRegion *ioportF0_io = g_new(MemoryRegion, 1);
+    X86MachineState *x86ms = X86_MACHINE(pcms);
 
     memory_region_init_io(ioport80_io, NULL, &ioport80_io_ops, NULL, "ioport80", 1);
     memory_region_add_subregion(isa_bus->address_space_io, 0x80, ioport80_io);
@@ -1167,7 +1168,8 @@ void pc_basic_device_init(struct PCMachineState *pcms,
 
     qemu_register_boot_set(pc_boot_set, *rtc_state);
 
-    if (!xen_enabled() && pcms->pit_enabled) {
+    if (!xen_enabled() &&
+        (x86ms->pit == ON_OFF_AUTO_AUTO || x86ms->pit == ON_OFF_AUTO_ON)) {
         if (kvm_pit_in_kernel()) {
             pit = kvm_pit_init(isa_bus, 0x40);
         } else {
@@ -1537,20 +1539,6 @@ static void pc_machine_set_sata(Object *obj, bool value, Error **errp)
     pcms->sata_enabled = value;
 }
 
-static bool pc_machine_get_pit(Object *obj, Error **errp)
-{
-    PCMachineState *pcms = PC_MACHINE(obj);
-
-    return pcms->pit_enabled;
-}
-
-static void pc_machine_set_pit(Object *obj, bool value, Error **errp)
-{
-    PCMachineState *pcms = PC_MACHINE(obj);
-
-    pcms->pit_enabled = value;
-}
-
 static bool pc_machine_get_hpet(Object *obj, Error **errp)
 {
     PCMachineState *pcms = PC_MACHINE(obj);
@@ -1707,7 +1695,6 @@ static void pc_machine_initfn(Object *obj)
     pcms->acpi_build_enabled = PC_MACHINE_GET_CLASS(pcms)->has_acpi_build;
     pcms->smbus_enabled = true;
     pcms->sata_enabled = true;
-    pcms->pit_enabled = true;
     pcms->i8042_enabled = true;
     pcms->max_fw_size = 8 * MiB;
 #ifdef CONFIG_HPET
@@ -1836,11 +1823,6 @@ static void pc_machine_class_init(ObjectClass *oc, void *data)
     object_class_property_set_description(oc, PC_MACHINE_SATA,
         "Enable/disable Serial ATA bus");
 
-    object_class_property_add_bool(oc, PC_MACHINE_PIT,
-        pc_machine_get_pit, pc_machine_set_pit);
-    object_class_property_set_description(oc, PC_MACHINE_PIT,
-        "Enable/disable Intel 8254 programmable interval timer emulation");
-
     object_class_property_add_bool(oc, "hpet",
         pc_machine_get_hpet, pc_machine_set_hpet);
     object_class_property_set_description(oc, "hpet",
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index f79e720cc2..784d54efa9 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -1244,6 +1244,23 @@ static void x86_machine_set_acpi(Object *obj, Visitor *v, const char *name,
     visit_type_OnOffAuto(v, name, &x86ms->acpi, errp);
 }
 
+static void x86_machine_get_pit(Object *obj, Visitor *v, const char *name,
+                                    void *opaque, Error **errp)
+{
+    X86MachineState *x86ms = X86_MACHINE(obj);
+    OnOffAuto pit = x86ms->pit;
+
+    visit_type_OnOffAuto(v, name, &pit, errp);
+}
+
+static void x86_machine_set_pit(Object *obj, Visitor *v, const char *name,
+                                    void *opaque, Error **errp)
+{
+    X86MachineState *x86ms = X86_MACHINE(obj);;
+
+    visit_type_OnOffAuto(v, name, &x86ms->pit, errp);
+}
+
 static char *x86_machine_get_oem_id(Object *obj, Error **errp)
 {
     X86MachineState *x86ms = X86_MACHINE(obj);
@@ -1333,6 +1350,7 @@ static void x86_machine_initfn(Object *obj)
 
     x86ms->smm = ON_OFF_AUTO_AUTO;
     x86ms->acpi = ON_OFF_AUTO_AUTO;
+    x86ms->pit = ON_OFF_AUTO_AUTO;
     x86ms->pci_irq_mask = ACPI_BUILD_PCI_IRQS;
     x86ms->oem_id = g_strndup(ACPI_BUILD_APPNAME6, 6);
     x86ms->oem_table_id = g_strndup(ACPI_BUILD_APPNAME8, 8);
@@ -1364,6 +1382,13 @@ static void x86_machine_class_init(ObjectClass *oc, void *data)
     object_class_property_set_description(oc, X86_MACHINE_ACPI,
         "Enable ACPI");
 
+    object_class_property_add(oc, X86_MACHINE_PIT, "OnOffAuto",
+                              x86_machine_get_pit,
+                              x86_machine_set_pit,
+                              NULL, NULL);
+    object_class_property_set_description(oc, X86_MACHINE_PIT,
+        "Enable i8254 PIT");
+
     object_class_property_add_str(oc, X86_MACHINE_OEM_ID,
                                   x86_machine_get_oem_id,
                                   x86_machine_set_oem_id);
diff --git a/include/hw/i386/microvm.h b/include/hw/i386/microvm.h
index efcbd926fd..83f9ac0b2a 100644
--- a/include/hw/i386/microvm.h
+++ b/include/hw/i386/microvm.h
@@ -67,7 +67,6 @@
 #define PCIE_ECAM_SIZE        0x10000000
 
 /* Machine type options */
-#define MICROVM_MACHINE_PIT                 "pit"
 #define MICROVM_MACHINE_PIC                 "pic"
 #define MICROVM_MACHINE_RTC                 "rtc"
 #define MICROVM_MACHINE_PCIE                "pcie"
@@ -87,7 +86,6 @@ struct MicrovmMachineState {
 
     /* Machine type options */
     OnOffAuto pic;
-    OnOffAuto pit;
     OnOffAuto rtc;
     OnOffAuto pcie;
     OnOffAuto ioapic2;
diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index aff8add155..ffcac5121e 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -46,7 +46,6 @@ typedef struct PCMachineState {
     bool acpi_build_enabled;
     bool smbus_enabled;
     bool sata_enabled;
-    bool pit_enabled;
     bool hpet_enabled;
     bool i8042_enabled;
     bool default_bus_bypass_iommu;
@@ -64,7 +63,6 @@ typedef struct PCMachineState {
 #define PC_MACHINE_VMPORT           "vmport"
 #define PC_MACHINE_SMBUS            "smbus"
 #define PC_MACHINE_SATA             "sata"
-#define PC_MACHINE_PIT              "pit"
 #define PC_MACHINE_I8042            "i8042"
 #define PC_MACHINE_MAX_FW_SIZE      "max-fw-size"
 #define PC_MACHINE_SMBIOS_EP        "smbios-entry-point-type"
diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
index 4841a49f86..b81fce54a0 100644
--- a/include/hw/i386/x86.h
+++ b/include/hw/i386/x86.h
@@ -65,6 +65,7 @@ struct X86MachineState {
 
     OnOffAuto smm;
     OnOffAuto acpi;
+    OnOffAuto pit;
 
     char *oem_id;
     char *oem_table_id;
@@ -84,6 +85,7 @@ struct X86MachineState {
 
 #define X86_MACHINE_SMM              "smm"
 #define X86_MACHINE_ACPI             "acpi"
+#define X86_MACHINE_PIT              "pit"
 #define X86_MACHINE_OEM_ID           "x-oem-id"
 #define X86_MACHINE_OEM_TABLE_ID     "x-oem-table-id"
 #define X86_MACHINE_BUS_LOCK_RATELIMIT  "bus-lock-ratelimit"

From c300bbe8d2a84003775c2ebb2b2885b80b1d9c32 Mon Sep 17 00:00:00 2001
From: Xiaoyao Li <xiaoyao.li@intel.com>
Date: Thu, 10 Mar 2022 20:28:11 +0800
Subject: [PATCH 406/935] hw/i386: Make pic a property of common x86 base
 machine type

Legacy PIC (8259) cannot be supported for TDX guests since TDX module
doesn't allow directly interrupt injection.  Using posted interrupts
for the PIC is not a viable option as the guest BIOS/kernel will not
do EOI for PIC IRQs, i.e. will leave the vIRR bit set.

Make PIC the property of common x86 machine type. Hence all x86
machines, including microvm, can disable it.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Sergio Lopez <slp@redhat.com>
Message-Id: <20220310122811.807794-3-xiaoyao.li@intel.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/microvm.c         | 27 +--------------------------
 hw/i386/pc_piix.c         |  4 +++-
 hw/i386/pc_q35.c          |  4 +++-
 hw/i386/x86.c             | 25 +++++++++++++++++++++++++
 include/hw/i386/microvm.h |  2 --
 include/hw/i386/x86.h     |  2 ++
 6 files changed, 34 insertions(+), 30 deletions(-)

diff --git a/hw/i386/microvm.c b/hw/i386/microvm.c
index 89b555a2f5..754f1d0593 100644
--- a/hw/i386/microvm.c
+++ b/hw/i386/microvm.c
@@ -247,7 +247,7 @@ static void microvm_devices_init(MicrovmMachineState *mms)
         x86ms->pci_irq_mask = 0;
     }
 
-    if (mms->pic == ON_OFF_AUTO_ON || mms->pic == ON_OFF_AUTO_AUTO) {
+    if (x86ms->pic == ON_OFF_AUTO_ON || x86ms->pic == ON_OFF_AUTO_AUTO) {
         qemu_irq *i8259;
 
         i8259 = i8259_init(isa_bus, x86_allocate_cpu_irq());
@@ -491,23 +491,6 @@ static void microvm_machine_reset(MachineState *machine)
     }
 }
 
-static void microvm_machine_get_pic(Object *obj, Visitor *v, const char *name,
-                                    void *opaque, Error **errp)
-{
-    MicrovmMachineState *mms = MICROVM_MACHINE(obj);
-    OnOffAuto pic = mms->pic;
-
-    visit_type_OnOffAuto(v, name, &pic, errp);
-}
-
-static void microvm_machine_set_pic(Object *obj, Visitor *v, const char *name,
-                                    void *opaque, Error **errp)
-{
-    MicrovmMachineState *mms = MICROVM_MACHINE(obj);
-
-    visit_type_OnOffAuto(v, name, &mms->pic, errp);
-}
-
 static void microvm_machine_get_rtc(Object *obj, Visitor *v, const char *name,
                                     void *opaque, Error **errp)
 {
@@ -632,7 +615,6 @@ static void microvm_machine_initfn(Object *obj)
     MicrovmMachineState *mms = MICROVM_MACHINE(obj);
 
     /* Configuration */
-    mms->pic = ON_OFF_AUTO_AUTO;
     mms->rtc = ON_OFF_AUTO_AUTO;
     mms->pcie = ON_OFF_AUTO_AUTO;
     mms->ioapic2 = ON_OFF_AUTO_AUTO;
@@ -684,13 +666,6 @@ static void microvm_class_init(ObjectClass *oc, void *data)
 
     x86mc->fwcfg_dma_enabled = true;
 
-    object_class_property_add(oc, MICROVM_MACHINE_PIC, "OnOffAuto",
-                              microvm_machine_get_pic,
-                              microvm_machine_set_pic,
-                              NULL, NULL);
-    object_class_property_set_description(oc, MICROVM_MACHINE_PIC,
-        "Enable i8259 PIC");
-
     object_class_property_add(oc, MICROVM_MACHINE_RTC, "OnOffAuto",
                               microvm_machine_get_rtc,
                               microvm_machine_set_rtc,
diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index f843dd906f..578e537b35 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -218,7 +218,9 @@ static void pc_init1(MachineState *machine,
     }
     isa_bus_irqs(isa_bus, x86ms->gsi);
 
-    pc_i8259_create(isa_bus, gsi_state->i8259_irq);
+    if (x86ms->pic == ON_OFF_AUTO_ON || x86ms->pic == ON_OFF_AUTO_AUTO) {
+        pc_i8259_create(isa_bus, gsi_state->i8259_irq);
+    }
 
     if (pcmc->pci_enabled) {
         ioapic_init_gsi(gsi_state, "i440fx");
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index 302288342a..42eb8b9707 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -265,7 +265,9 @@ static void pc_q35_init(MachineState *machine)
     pci_bus_set_route_irq_fn(host_bus, ich9_route_intx_pin_to_irq);
     isa_bus = ich9_lpc->isa_bus;
 
-    pc_i8259_create(isa_bus, gsi_state->i8259_irq);
+    if (x86ms->pic == ON_OFF_AUTO_ON || x86ms->pic == ON_OFF_AUTO_AUTO) {
+        pc_i8259_create(isa_bus, gsi_state->i8259_irq);
+    }
 
     if (pcmc->pci_enabled) {
         ioapic_init_gsi(gsi_state, "q35");
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 784d54efa9..78b05ab7a2 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -1261,6 +1261,23 @@ static void x86_machine_set_pit(Object *obj, Visitor *v, const char *name,
     visit_type_OnOffAuto(v, name, &x86ms->pit, errp);
 }
 
+static void x86_machine_get_pic(Object *obj, Visitor *v, const char *name,
+                                void *opaque, Error **errp)
+{
+    X86MachineState *x86ms = X86_MACHINE(obj);
+    OnOffAuto pic = x86ms->pic;
+
+    visit_type_OnOffAuto(v, name, &pic, errp);
+}
+
+static void x86_machine_set_pic(Object *obj, Visitor *v, const char *name,
+                                void *opaque, Error **errp)
+{
+    X86MachineState *x86ms = X86_MACHINE(obj);
+
+    visit_type_OnOffAuto(v, name, &x86ms->pic, errp);
+}
+
 static char *x86_machine_get_oem_id(Object *obj, Error **errp)
 {
     X86MachineState *x86ms = X86_MACHINE(obj);
@@ -1351,6 +1368,7 @@ static void x86_machine_initfn(Object *obj)
     x86ms->smm = ON_OFF_AUTO_AUTO;
     x86ms->acpi = ON_OFF_AUTO_AUTO;
     x86ms->pit = ON_OFF_AUTO_AUTO;
+    x86ms->pic = ON_OFF_AUTO_AUTO;
     x86ms->pci_irq_mask = ACPI_BUILD_PCI_IRQS;
     x86ms->oem_id = g_strndup(ACPI_BUILD_APPNAME6, 6);
     x86ms->oem_table_id = g_strndup(ACPI_BUILD_APPNAME8, 8);
@@ -1389,6 +1407,13 @@ static void x86_machine_class_init(ObjectClass *oc, void *data)
     object_class_property_set_description(oc, X86_MACHINE_PIT,
         "Enable i8254 PIT");
 
+    object_class_property_add(oc, X86_MACHINE_PIC, "OnOffAuto",
+                              x86_machine_get_pic,
+                              x86_machine_set_pic,
+                              NULL, NULL);
+    object_class_property_set_description(oc, X86_MACHINE_PIC,
+        "Enable i8259 PIC");
+
     object_class_property_add_str(oc, X86_MACHINE_OEM_ID,
                                   x86_machine_get_oem_id,
                                   x86_machine_set_oem_id);
diff --git a/include/hw/i386/microvm.h b/include/hw/i386/microvm.h
index 83f9ac0b2a..fad97a891d 100644
--- a/include/hw/i386/microvm.h
+++ b/include/hw/i386/microvm.h
@@ -67,7 +67,6 @@
 #define PCIE_ECAM_SIZE        0x10000000
 
 /* Machine type options */
-#define MICROVM_MACHINE_PIC                 "pic"
 #define MICROVM_MACHINE_RTC                 "rtc"
 #define MICROVM_MACHINE_PCIE                "pcie"
 #define MICROVM_MACHINE_IOAPIC2             "ioapic2"
@@ -85,7 +84,6 @@ struct MicrovmMachineState {
     X86MachineState parent;
 
     /* Machine type options */
-    OnOffAuto pic;
     OnOffAuto rtc;
     OnOffAuto pcie;
     OnOffAuto ioapic2;
diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
index b81fce54a0..9089bdd99c 100644
--- a/include/hw/i386/x86.h
+++ b/include/hw/i386/x86.h
@@ -66,6 +66,7 @@ struct X86MachineState {
     OnOffAuto smm;
     OnOffAuto acpi;
     OnOffAuto pit;
+    OnOffAuto pic;
 
     char *oem_id;
     char *oem_table_id;
@@ -86,6 +87,7 @@ struct X86MachineState {
 #define X86_MACHINE_SMM              "smm"
 #define X86_MACHINE_ACPI             "acpi"
 #define X86_MACHINE_PIT              "pit"
+#define X86_MACHINE_PIC              "pic"
 #define X86_MACHINE_OEM_ID           "x-oem-id"
 #define X86_MACHINE_OEM_TABLE_ID     "x-oem-table-id"
 #define X86_MACHINE_BUS_LOCK_RATELIMIT  "bus-lock-ratelimit"

From 18aa91cddd8caa14aae094f9b59c0381959cb320 Mon Sep 17 00:00:00 2001
From: Wei Huang <wei.huang2@amd.com>
Date: Fri, 22 Apr 2022 00:51:46 -0500
Subject: [PATCH 407/935] hw/i386/amd_iommu: Fix IOMMU event log encoding
 errors

Coverity issues several UNINIT warnings against amd_iommu.c [1]. This
patch fixes them by clearing evt before encoding. On top of it, this
patch changes the event log size to 16 bytes per IOMMU specification,
and fixes the event log entry format in amdvi_encode_event().

[1] CID 1487116/1487200/1487190/1487232/1487115/1487258

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Wei Huang <wei.huang2@amd.com>
Message-Id: <20220422055146.3312226-1-wei.huang2@amd.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Acked-by: Jason Wang <jasowang@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/amd_iommu.c | 24 ++++++++++++++----------
 1 file changed, 14 insertions(+), 10 deletions(-)

diff --git a/hw/i386/amd_iommu.c b/hw/i386/amd_iommu.c
index ea8eaeb330..725f69095b 100644
--- a/hw/i386/amd_iommu.c
+++ b/hw/i386/amd_iommu.c
@@ -201,15 +201,18 @@ static void amdvi_setevent_bits(uint64_t *buffer, uint64_t value, int start,
 /*
  * AMDVi event structure
  *    0:15   -> DeviceID
- *    55:63  -> event type + miscellaneous info
- *    63:127 -> related address
+ *    48:63  -> event type + miscellaneous info
+ *    64:127 -> related address
  */
 static void amdvi_encode_event(uint64_t *evt, uint16_t devid, uint64_t addr,
                                uint16_t info)
 {
+    evt[0] = 0;
+    evt[1] = 0;
+
     amdvi_setevent_bits(evt, devid, 0, 16);
-    amdvi_setevent_bits(evt, info, 55, 8);
-    amdvi_setevent_bits(evt, addr, 63, 64);
+    amdvi_setevent_bits(evt, info, 48, 16);
+    amdvi_setevent_bits(evt, addr, 64, 64);
 }
 /* log an error encountered during a page walk
  *
@@ -218,7 +221,7 @@ static void amdvi_encode_event(uint64_t *evt, uint16_t devid, uint64_t addr,
 static void amdvi_page_fault(AMDVIState *s, uint16_t devid,
                              hwaddr addr, uint16_t info)
 {
-    uint64_t evt[4];
+    uint64_t evt[2];
 
     info |= AMDVI_EVENT_IOPF_I | AMDVI_EVENT_IOPF;
     amdvi_encode_event(evt, devid, addr, info);
@@ -234,7 +237,7 @@ static void amdvi_page_fault(AMDVIState *s, uint16_t devid,
 static void amdvi_log_devtab_error(AMDVIState *s, uint16_t devid,
                                    hwaddr devtab, uint16_t info)
 {
-    uint64_t evt[4];
+    uint64_t evt[2];
 
     info |= AMDVI_EVENT_DEV_TAB_HW_ERROR;
 
@@ -248,7 +251,8 @@ static void amdvi_log_devtab_error(AMDVIState *s, uint16_t devid,
  */
 static void amdvi_log_command_error(AMDVIState *s, hwaddr addr)
 {
-    uint64_t evt[4], info = AMDVI_EVENT_COMMAND_HW_ERROR;
+    uint64_t evt[2];
+    uint16_t info = AMDVI_EVENT_COMMAND_HW_ERROR;
 
     amdvi_encode_event(evt, 0, addr, info);
     amdvi_log_event(s, evt);
@@ -261,7 +265,7 @@ static void amdvi_log_command_error(AMDVIState *s, hwaddr addr)
 static void amdvi_log_illegalcom_error(AMDVIState *s, uint16_t info,
                                        hwaddr addr)
 {
-    uint64_t evt[4];
+    uint64_t evt[2];
 
     info |= AMDVI_EVENT_ILLEGAL_COMMAND_ERROR;
     amdvi_encode_event(evt, 0, addr, info);
@@ -276,7 +280,7 @@ static void amdvi_log_illegalcom_error(AMDVIState *s, uint16_t info,
 static void amdvi_log_illegaldevtab_error(AMDVIState *s, uint16_t devid,
                                           hwaddr addr, uint16_t info)
 {
-    uint64_t evt[4];
+    uint64_t evt[2];
 
     info |= AMDVI_EVENT_ILLEGAL_DEVTAB_ENTRY;
     amdvi_encode_event(evt, devid, addr, info);
@@ -288,7 +292,7 @@ static void amdvi_log_illegaldevtab_error(AMDVIState *s, uint16_t devid,
 static void amdvi_log_pagetab_error(AMDVIState *s, uint16_t devid,
                                     hwaddr addr, uint16_t info)
 {
-    uint64_t evt[4];
+    uint64_t evt[2];
 
     info |= AMDVI_EVENT_PAGE_TAB_HW_ERROR;
     amdvi_encode_event(evt, devid, addr, info);

From aa8581945a13712ff3eed0ad3ba7a9664fc1604b Mon Sep 17 00:00:00 2001
From: Si-Wei Liu <si-wei.liu@oracle.com>
Date: Fri, 6 May 2022 19:28:12 -0700
Subject: [PATCH 408/935] virtio-net: setup vhost_dev and notifiers for cvq
 only when feature is negotiated

When the control virtqueue feature is absent or not negotiated,
vhost_net_start() still tries to set up vhost_dev and install
vhost notifiers for the control virtqueue, which results in
erroneous ioctl calls with incorrect queue index sending down
to driver. Do that only when needed.

Fixes: 22288fe ("virtio-net: vhost control virtqueue support")
Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <1651890498-24478-2-git-send-email-si-wei.liu@oracle.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/net/virtio-net.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index a788f1c5c7..7545a16ed7 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -245,7 +245,8 @@ static void virtio_net_vhost_status(VirtIONet *n, uint8_t status)
     VirtIODevice *vdev = VIRTIO_DEVICE(n);
     NetClientState *nc = qemu_get_queue(n->nic);
     int queue_pairs = n->multiqueue ? n->max_queue_pairs : 1;
-    int cvq = n->max_ncs - n->max_queue_pairs;
+    int cvq = virtio_vdev_has_feature(vdev, VIRTIO_NET_F_CTRL_VQ) ?
+              n->max_ncs - n->max_queue_pairs : 0;
 
     if (!get_vhost_net(nc->peer)) {
         return;

From 68b0a6395f36a8f48f56f46d05f30be2067598b0 Mon Sep 17 00:00:00 2001
From: Si-Wei Liu <si-wei.liu@oracle.com>
Date: Fri, 6 May 2022 19:28:13 -0700
Subject: [PATCH 409/935] virtio-net: align ctrl_vq index for non-mq guest for
 vhost_vdpa

With MQ enabled vdpa device and non-MQ supporting guest e.g.
booting vdpa with mq=on over OVMF of single vqp, below assert
failure is seen:

../hw/virtio/vhost-vdpa.c:560: vhost_vdpa_get_vq_index: Assertion `idx >= dev->vq_index && idx < dev->vq_index + dev->nvqs' failed.

0  0x00007f8ce3ff3387 in raise () at /lib64/libc.so.6
1  0x00007f8ce3ff4a78 in abort () at /lib64/libc.so.6
2  0x00007f8ce3fec1a6 in __assert_fail_base () at /lib64/libc.so.6
3  0x00007f8ce3fec252 in  () at /lib64/libc.so.6
4  0x0000558f52d79421 in vhost_vdpa_get_vq_index (dev=<optimized out>, idx=<optimized out>) at ../hw/virtio/vhost-vdpa.c:563
5  0x0000558f52d79421 in vhost_vdpa_get_vq_index (dev=<optimized out>, idx=<optimized out>) at ../hw/virtio/vhost-vdpa.c:558
6  0x0000558f52d7329a in vhost_virtqueue_mask (hdev=0x558f55c01800, vdev=0x558f568f91f0, n=2, mask=<optimized out>) at ../hw/virtio/vhost.c:1557
7  0x0000558f52c6b89a in virtio_pci_set_guest_notifier (d=d@entry=0x558f568f0f60, n=n@entry=2, assign=assign@entry=true, with_irqfd=with_irqfd@entry=false)
   at ../hw/virtio/virtio-pci.c:974
8  0x0000558f52c6c0d8 in virtio_pci_set_guest_notifiers (d=0x558f568f0f60, nvqs=3, assign=true) at ../hw/virtio/virtio-pci.c:1019
9  0x0000558f52bf091d in vhost_net_start (dev=dev@entry=0x558f568f91f0, ncs=0x558f56937cd0, data_queue_pairs=data_queue_pairs@entry=1, cvq=cvq@entry=1)
   at ../hw/net/vhost_net.c:361
10 0x0000558f52d4e5e7 in virtio_net_set_status (status=<optimized out>, n=0x558f568f91f0) at ../hw/net/virtio-net.c:289
11 0x0000558f52d4e5e7 in virtio_net_set_status (vdev=0x558f568f91f0, status=15 '\017') at ../hw/net/virtio-net.c:370
12 0x0000558f52d6c4b2 in virtio_set_status (vdev=vdev@entry=0x558f568f91f0, val=val@entry=15 '\017') at ../hw/virtio/virtio.c:1945
13 0x0000558f52c69eff in virtio_pci_common_write (opaque=0x558f568f0f60, addr=<optimized out>, val=<optimized out>, size=<optimized out>) at ../hw/virtio/virtio-pci.c:1292
14 0x0000558f52d15d6e in memory_region_write_accessor (mr=0x558f568f19d0, addr=20, value=<optimized out>, size=1, shift=<optimized out>, mask=<optimized out>, attrs=...)
   at ../softmmu/memory.c:492
15 0x0000558f52d127de in access_with_adjusted_size (addr=addr@entry=20, value=value@entry=0x7f8cdbffe748, size=size@entry=1, access_size_min=<optimized out>, access_size_max=<optimized out>, access_fn=0x558f52d15cf0 <memory_region_write_accessor>, mr=0x558f568f19d0, attrs=...) at ../softmmu/memory.c:554
16 0x0000558f52d157ef in memory_region_dispatch_write (mr=mr@entry=0x558f568f19d0, addr=20, data=<optimized out>, op=<optimized out>, attrs=attrs@entry=...)
   at ../softmmu/memory.c:1504
17 0x0000558f52d078e7 in flatview_write_continue (fv=fv@entry=0x7f8accbc3b90, addr=addr@entry=103079215124, attrs=..., ptr=ptr@entry=0x7f8ce6300028, len=len@entry=1, addr1=<optimized out>, l=<optimized out>, mr=0x558f568f19d0) at /home/opc/qemu-upstream/include/qemu/host-utils.h:165
18 0x0000558f52d07b06 in flatview_write (fv=0x7f8accbc3b90, addr=103079215124, attrs=..., buf=0x7f8ce6300028, len=1) at ../softmmu/physmem.c:2822
19 0x0000558f52d0b36b in address_space_write (as=<optimized out>, addr=<optimized out>, attrs=..., buf=buf@entry=0x7f8ce6300028, len=<optimized out>)
   at ../softmmu/physmem.c:2914
20 0x0000558f52d0b3da in address_space_rw (as=<optimized out>, addr=<optimized out>, attrs=...,
   attrs@entry=..., buf=buf@entry=0x7f8ce6300028, len=<optimized out>, is_write=<optimized out>) at ../softmmu/physmem.c:2924
21 0x0000558f52dced09 in kvm_cpu_exec (cpu=cpu@entry=0x558f55c2da60) at ../accel/kvm/kvm-all.c:2903
22 0x0000558f52dcfabd in kvm_vcpu_thread_fn (arg=arg@entry=0x558f55c2da60) at ../accel/kvm/kvm-accel-ops.c:49
23 0x0000558f52f9f04a in qemu_thread_start (args=<optimized out>) at ../util/qemu-thread-posix.c:556
24 0x00007f8ce4392ea5 in start_thread () at /lib64/libpthread.so.0
25 0x00007f8ce40bb9fd in clone () at /lib64/libc.so.6

The cause for the assert failure is due to that the vhost_dev index
for the ctrl vq was not aligned with actual one in use by the guest.
Upon multiqueue feature negotiation in virtio_net_set_multiqueue(),
if guest doesn't support multiqueue, the guest vq layout would shrink
to a single queue pair, consisting of 3 vqs in total (rx, tx and ctrl).
This results in ctrl_vq taking a different vhost_dev group index than
the default. We can map vq to the correct vhost_dev group by checking
if MQ is supported by guest and successfully negotiated. Since the
MQ feature is only present along with CTRL_VQ, we ensure the index
2 is only meant for the control vq while MQ is not supported by guest.

Fixes: 22288fe ("virtio-net: vhost control virtqueue support")
Suggested-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <1651890498-24478-3-git-send-email-si-wei.liu@oracle.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/net/virtio-net.c | 33 +++++++++++++++++++++++++++++++--
 1 file changed, 31 insertions(+), 2 deletions(-)

diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 7545a16ed7..1ea524ff09 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -14,6 +14,7 @@
 #include "qemu/osdep.h"
 #include "qemu/atomic.h"
 #include "qemu/iov.h"
+#include "qemu/log.h"
 #include "qemu/main-loop.h"
 #include "qemu/module.h"
 #include "hw/virtio/virtio.h"
@@ -3172,8 +3173,22 @@ static NetClientInfo net_virtio_info = {
 static bool virtio_net_guest_notifier_pending(VirtIODevice *vdev, int idx)
 {
     VirtIONet *n = VIRTIO_NET(vdev);
-    NetClientState *nc = qemu_get_subqueue(n->nic, vq2q(idx));
+    NetClientState *nc;
     assert(n->vhost_started);
+    if (!virtio_vdev_has_feature(vdev, VIRTIO_NET_F_MQ) && idx == 2) {
+        /* Must guard against invalid features and bogus queue index
+         * from being set by malicious guest, or penetrated through
+         * buggy migration stream.
+         */
+        if (!virtio_vdev_has_feature(vdev, VIRTIO_NET_F_CTRL_VQ)) {
+            qemu_log_mask(LOG_GUEST_ERROR,
+                          "%s: bogus vq index ignored\n", __func__);
+            return false;
+        }
+        nc = qemu_get_subqueue(n->nic, n->max_queue_pairs);
+    } else {
+        nc = qemu_get_subqueue(n->nic, vq2q(idx));
+    }
     return vhost_net_virtqueue_pending(get_vhost_net(nc->peer), idx);
 }
 
@@ -3181,8 +3196,22 @@ static void virtio_net_guest_notifier_mask(VirtIODevice *vdev, int idx,
                                            bool mask)
 {
     VirtIONet *n = VIRTIO_NET(vdev);
-    NetClientState *nc = qemu_get_subqueue(n->nic, vq2q(idx));
+    NetClientState *nc;
     assert(n->vhost_started);
+    if (!virtio_vdev_has_feature(vdev, VIRTIO_NET_F_MQ) && idx == 2) {
+        /* Must guard against invalid features and bogus queue index
+         * from being set by malicious guest, or penetrated through
+         * buggy migration stream.
+         */
+        if (!virtio_vdev_has_feature(vdev, VIRTIO_NET_F_CTRL_VQ)) {
+            qemu_log_mask(LOG_GUEST_ERROR,
+                          "%s: bogus vq index ignored\n", __func__);
+            return;
+        }
+        nc = qemu_get_subqueue(n->nic, n->max_queue_pairs);
+    } else {
+        nc = qemu_get_subqueue(n->nic, vq2q(idx));
+    }
     vhost_net_virtqueue_mask(get_vhost_net(nc->peer),
                              vdev, idx, mask);
 }

From 9bd055073e375c8a0d7ebce925e05d914d69fc7f Mon Sep 17 00:00:00 2001
From: Si-Wei Liu <si-wei.liu@oracle.com>
Date: Fri, 6 May 2022 19:28:14 -0700
Subject: [PATCH 410/935] vhost-vdpa: fix improper cleanup in
 net_init_vhost_vdpa

... such that no memory leaks on dangling net clients in case of
error.

Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <1651890498-24478-4-git-send-email-si-wei.liu@oracle.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 net/vhost-vdpa.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c
index 1e9fe47c03..df1e69ee72 100644
--- a/net/vhost-vdpa.c
+++ b/net/vhost-vdpa.c
@@ -306,7 +306,9 @@ int net_init_vhost_vdpa(const Netdev *netdev, const char *name,
 
 err:
     if (i) {
-        qemu_del_net_client(ncs[0]);
+        for (i--; i >= 0; i--) {
+            qemu_del_net_client(ncs[i]);
+        }
     }
     qemu_close(vdpa_device_fd);
 

From 6f3910b5eee00b8cc959e94659c0d524c482a418 Mon Sep 17 00:00:00 2001
From: Si-Wei Liu <si-wei.liu@oracle.com>
Date: Fri, 6 May 2022 19:28:15 -0700
Subject: [PATCH 411/935] vhost-net: fix improper cleanup in vhost_net_start

vhost_net_start() missed a corresponding stop_one() upon error from
vhost_set_vring_enable(). While at it, make the error handling for
err_start more robust. No real issue was found due to this though.

Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <1651890498-24478-5-git-send-email-si-wei.liu@oracle.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/net/vhost_net.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index df0f050548..ccac5b7a64 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -381,6 +381,7 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs,
             r = vhost_set_vring_enable(peer, peer->vring_enable);
 
             if (r < 0) {
+                vhost_net_stop_one(get_vhost_net(peer), dev);
                 goto err_start;
             }
         }
@@ -390,7 +391,8 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs,
 
 err_start:
     while (--i >= 0) {
-        peer = qemu_get_peer(ncs , i);
+        peer = qemu_get_peer(ncs, i < data_queue_pairs ?
+                                  i : n->max_queue_pairs);
         vhost_net_stop_one(get_vhost_net(peer), dev);
     }
     e = k->set_guest_notifiers(qbus->parent, total_notifiers, false);

From 6aee7e4233f6467f69531fcd352adff028f3f5ea Mon Sep 17 00:00:00 2001
From: Si-Wei Liu <si-wei.liu@oracle.com>
Date: Fri, 6 May 2022 19:28:16 -0700
Subject: [PATCH 412/935] vhost-vdpa: backend feature should set only once
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The vhost_vdpa_one_time_request() branch in
vhost_vdpa_set_backend_cap() incorrectly sends down
ioctls on vhost_dev with non-zero index. This may
end up with multiple VHOST_SET_BACKEND_FEATURES
ioctl calls sent down on the vhost-vdpa fd that is
shared between all these vhost_dev's.

To fix it, send down ioctl only once via the first
vhost_dev with index 0. Toggle the polarity of the
vhost_vdpa_one_time_request() test should do the
trick.

Fixes: 4d191cfdc7de ("vhost-vdpa: classify one time request")
Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Acked-by: Eugenio Pérez <eperezma@redhat.com>
Message-Id: <1651890498-24478-6-git-send-email-si-wei.liu@oracle.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/vhost-vdpa.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c
index ed106bff47..870ab74b04 100644
--- a/hw/virtio/vhost-vdpa.c
+++ b/hw/virtio/vhost-vdpa.c
@@ -667,7 +667,7 @@ static int vhost_vdpa_set_backend_cap(struct vhost_dev *dev)
 
     features &= f;
 
-    if (vhost_vdpa_one_time_request(dev)) {
+    if (!vhost_vdpa_one_time_request(dev)) {
         r = vhost_vdpa_call(dev, VHOST_SET_BACKEND_FEATURES, &features);
         if (r) {
             return -EFAULT;

From d71b0609fc04217e28d17009f04d74b08be6f466 Mon Sep 17 00:00:00 2001
From: Si-Wei Liu <si-wei.liu@oracle.com>
Date: Fri, 6 May 2022 19:28:17 -0700
Subject: [PATCH 413/935] vhost-vdpa: change name and polarity for
 vhost_vdpa_one_time_request()

The name vhost_vdpa_one_time_request() was confusing. No
matter whatever it returns, its typical occurrence had
always been at requests that only need to be applied once.
And the name didn't suggest what it actually checks for.
Change it to vhost_vdpa_first_dev() with polarity flipped
for better readibility of code. That way it is able to
reflect what the check is really about.

This call is applicable to request which performs operation
only once, before queues are set up, and usually at the beginning
of the caller function. Document the requirement for it in place.

Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com>
Message-Id: <1651890498-24478-7-git-send-email-si-wei.liu@oracle.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Acked-by: Jason Wang <jasowang@redhat.com>
---
 hw/virtio/vhost-vdpa.c | 23 +++++++++++++++--------
 1 file changed, 15 insertions(+), 8 deletions(-)

diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c
index 870ab74b04..66f054a12c 100644
--- a/hw/virtio/vhost-vdpa.c
+++ b/hw/virtio/vhost-vdpa.c
@@ -368,11 +368,18 @@ static void vhost_vdpa_get_iova_range(struct vhost_vdpa *v)
                                     v->iova_range.last);
 }
 
-static bool vhost_vdpa_one_time_request(struct vhost_dev *dev)
+/*
+ * The use of this function is for requests that only need to be
+ * applied once. Typically such request occurs at the beginning
+ * of operation, and before setting up queues. It should not be
+ * used for request that performs operation until all queues are
+ * set, which would need to check dev->vq_index_end instead.
+ */
+static bool vhost_vdpa_first_dev(struct vhost_dev *dev)
 {
     struct vhost_vdpa *v = dev->opaque;
 
-    return v->index != 0;
+    return v->index == 0;
 }
 
 static int vhost_vdpa_get_dev_features(struct vhost_dev *dev,
@@ -453,7 +460,7 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void *opaque, Error **errp)
 
     vhost_vdpa_get_iova_range(v);
 
-    if (vhost_vdpa_one_time_request(dev)) {
+    if (!vhost_vdpa_first_dev(dev)) {
         return 0;
     }
 
@@ -596,7 +603,7 @@ static int vhost_vdpa_memslots_limit(struct vhost_dev *dev)
 static int vhost_vdpa_set_mem_table(struct vhost_dev *dev,
                                     struct vhost_memory *mem)
 {
-    if (vhost_vdpa_one_time_request(dev)) {
+    if (!vhost_vdpa_first_dev(dev)) {
         return 0;
     }
 
@@ -625,7 +632,7 @@ static int vhost_vdpa_set_features(struct vhost_dev *dev,
     struct vhost_vdpa *v = dev->opaque;
     int ret;
 
-    if (vhost_vdpa_one_time_request(dev)) {
+    if (!vhost_vdpa_first_dev(dev)) {
         return 0;
     }
 
@@ -667,7 +674,7 @@ static int vhost_vdpa_set_backend_cap(struct vhost_dev *dev)
 
     features &= f;
 
-    if (!vhost_vdpa_one_time_request(dev)) {
+    if (vhost_vdpa_first_dev(dev)) {
         r = vhost_vdpa_call(dev, VHOST_SET_BACKEND_FEATURES, &features);
         if (r) {
             return -EFAULT;
@@ -1120,7 +1127,7 @@ static int vhost_vdpa_set_log_base(struct vhost_dev *dev, uint64_t base,
                                      struct vhost_log *log)
 {
     struct vhost_vdpa *v = dev->opaque;
-    if (v->shadow_vqs_enabled || vhost_vdpa_one_time_request(dev)) {
+    if (v->shadow_vqs_enabled || !vhost_vdpa_first_dev(dev)) {
         return 0;
     }
 
@@ -1242,7 +1249,7 @@ static int vhost_vdpa_get_features(struct vhost_dev *dev,
 
 static int vhost_vdpa_set_owner(struct vhost_dev *dev)
 {
-    if (vhost_vdpa_one_time_request(dev)) {
+    if (!vhost_vdpa_first_dev(dev)) {
         return 0;
     }
 

From 2a7888cc3aa31faee839fa5dddad354ff8941f4c Mon Sep 17 00:00:00 2001
From: Si-Wei Liu <si-wei.liu@oracle.com>
Date: Fri, 6 May 2022 19:28:18 -0700
Subject: [PATCH 414/935] virtio-net: don't handle mq request in userspace
 handler for vhost-vdpa

virtio_queue_host_notifier_read() tends to read pending event
left behind on ioeventfd in the vhost_net_stop() path, and
attempts to handle outstanding kicks from userspace vq handler.
However, in the ctrl_vq handler, virtio_net_handle_mq() has a
recursive call into virtio_net_set_status(), which may lead to
segmentation fault as shown in below stack trace:

0  0x000055f800df1780 in qdev_get_parent_bus (dev=0x0) at ../hw/core/qdev.c:376
1  0x000055f800c68ad8 in virtio_bus_device_iommu_enabled (vdev=vdev@entry=0x0) at ../hw/virtio/virtio-bus.c:331
2  0x000055f800d70d7f in vhost_memory_unmap (dev=<optimized out>) at ../hw/virtio/vhost.c:318
3  0x000055f800d70d7f in vhost_memory_unmap (dev=<optimized out>, buffer=0x7fc19bec5240, len=2052, is_write=1, access_len=2052) at ../hw/virtio/vhost.c:336
4  0x000055f800d71867 in vhost_virtqueue_stop (dev=dev@entry=0x55f8037ccc30, vdev=vdev@entry=0x55f8044ec590, vq=0x55f8037cceb0, idx=0) at ../hw/virtio/vhost.c:1241
5  0x000055f800d7406c in vhost_dev_stop (hdev=hdev@entry=0x55f8037ccc30, vdev=vdev@entry=0x55f8044ec590) at ../hw/virtio/vhost.c:1839
6  0x000055f800bf00a7 in vhost_net_stop_one (net=0x55f8037ccc30, dev=0x55f8044ec590) at ../hw/net/vhost_net.c:315
7  0x000055f800bf0678 in vhost_net_stop (dev=dev@entry=0x55f8044ec590, ncs=0x55f80452bae0, data_queue_pairs=data_queue_pairs@entry=7, cvq=cvq@entry=1)
   at ../hw/net/vhost_net.c:423
8  0x000055f800d4e628 in virtio_net_set_status (status=<optimized out>, n=0x55f8044ec590) at ../hw/net/virtio-net.c:296
9  0x000055f800d4e628 in virtio_net_set_status (vdev=vdev@entry=0x55f8044ec590, status=15 '\017') at ../hw/net/virtio-net.c:370
10 0x000055f800d534d8 in virtio_net_handle_ctrl (iov_cnt=<optimized out>, iov=<optimized out>, cmd=0 '\000', n=0x55f8044ec590) at ../hw/net/virtio-net.c:1408
11 0x000055f800d534d8 in virtio_net_handle_ctrl (vdev=0x55f8044ec590, vq=0x7fc1a7e888d0) at ../hw/net/virtio-net.c:1452
12 0x000055f800d69f37 in virtio_queue_host_notifier_read (vq=0x7fc1a7e888d0) at ../hw/virtio/virtio.c:2331
13 0x000055f800d69f37 in virtio_queue_host_notifier_read (n=n@entry=0x7fc1a7e8894c) at ../hw/virtio/virtio.c:3575
14 0x000055f800c688e6 in virtio_bus_cleanup_host_notifier (bus=<optimized out>, n=n@entry=14) at ../hw/virtio/virtio-bus.c:312
15 0x000055f800d73106 in vhost_dev_disable_notifiers (hdev=hdev@entry=0x55f8035b51b0, vdev=vdev@entry=0x55f8044ec590)
   at ../../../include/hw/virtio/virtio-bus.h:35
16 0x000055f800bf00b2 in vhost_net_stop_one (net=0x55f8035b51b0, dev=0x55f8044ec590) at ../hw/net/vhost_net.c:316
17 0x000055f800bf0678 in vhost_net_stop (dev=dev@entry=0x55f8044ec590, ncs=0x55f80452bae0, data_queue_pairs=data_queue_pairs@entry=7, cvq=cvq@entry=1)
   at ../hw/net/vhost_net.c:423
18 0x000055f800d4e628 in virtio_net_set_status (status=<optimized out>, n=0x55f8044ec590) at ../hw/net/virtio-net.c:296
19 0x000055f800d4e628 in virtio_net_set_status (vdev=0x55f8044ec590, status=15 '\017') at ../hw/net/virtio-net.c:370
20 0x000055f800d6c4b2 in virtio_set_status (vdev=0x55f8044ec590, val=<optimized out>) at ../hw/virtio/virtio.c:1945
21 0x000055f800d11d9d in vm_state_notify (running=running@entry=false, state=state@entry=RUN_STATE_SHUTDOWN) at ../softmmu/runstate.c:333
22 0x000055f800d04e7a in do_vm_stop (state=state@entry=RUN_STATE_SHUTDOWN, send_stop=send_stop@entry=false) at ../softmmu/cpus.c:262
23 0x000055f800d04e99 in vm_shutdown () at ../softmmu/cpus.c:280
24 0x000055f800d126af in qemu_cleanup () at ../softmmu/runstate.c:812
25 0x000055f800ad5b13 in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at ../softmmu/main.c:51

For now, temporarily disable handling MQ request from the ctrl_vq
userspace hanlder to avoid the recursive virtio_net_set_status()
call. Some rework is needed to allow changing the number of
queues without going through a full virtio_net_set_status cycle,
particularly for vhost-vdpa backend.

This patch will need to be reverted as soon as future patches of
having the change of #queues handled in userspace is merged.

Fixes: 402378407db ("vhost-vdpa: multiqueue support")
Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <1651890498-24478-8-git-send-email-si-wei.liu@oracle.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/net/virtio-net.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 1ea524ff09..7ad948ee7c 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -1381,6 +1381,7 @@ static int virtio_net_handle_mq(VirtIONet *n, uint8_t cmd,
 {
     VirtIODevice *vdev = VIRTIO_DEVICE(n);
     uint16_t queue_pairs;
+    NetClientState *nc = qemu_get_queue(n->nic);
 
     virtio_net_disable_rss(n);
     if (cmd == VIRTIO_NET_CTRL_MQ_HASH_CONFIG) {
@@ -1412,6 +1413,18 @@ static int virtio_net_handle_mq(VirtIONet *n, uint8_t cmd,
         return VIRTIO_NET_ERR;
     }
 
+    /* Avoid changing the number of queue_pairs for vdpa device in
+     * userspace handler. A future fix is needed to handle the mq
+     * change in userspace handler with vhost-vdpa. Let's disable
+     * the mq handling from userspace for now and only allow get
+     * done through the kernel. Ripples may be seen when falling
+     * back to userspace, but without doing it qemu process would
+     * crash on a recursive entry to virtio_net_set_status().
+     */
+    if (nc->peer && nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_VDPA) {
+        return VIRTIO_NET_ERR;
+    }
+
     n->curr_queue_pairs = queue_pairs;
     /* stop the backend before changing the number of queue_pairs to avoid handling a
      * disabled queue */

From 6852c21db229c4bf4c1db772444bdfbbd027e5b8 Mon Sep 17 00:00:00 2001
From: Stefan Hajnoczi <stefanha@redhat.com>
Date: Mon, 16 May 2022 16:57:01 +0100
Subject: [PATCH 415/935] vhost-user-scsi: avoid unlink(NULL) with fd passing

Commit 747421e949fc1eb3ba66b5fcccdb7ba051918241 ("Implements Backend
Program conventions for vhost-user-scsi") introduced fd-passing support
as part of implementing the vhost-user backend program conventions.

When fd passing is used the UNIX domain socket path is NULL and we must
not call unlink(2).

The unlink(2) call is necessary when the listen socket, lsock, was
created successfully since that means the UNIX domain socket is visible
in the file system.

Fixes: Coverity CID 1488353
Fixes: 747421e949fc1eb3ba66b5fcccdb7ba051918241 ("Implements Backend Program conventions for vhost-user-scsi")
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220516155701.1789638-1-stefanha@redhat.com>
Reviewed-by: Raphael Norwitz <raphael.norwitz@nutanix.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 contrib/vhost-user-scsi/vhost-user-scsi.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/contrib/vhost-user-scsi/vhost-user-scsi.c b/contrib/vhost-user-scsi/vhost-user-scsi.c
index b2c0f98253..9ef61cf5a7 100644
--- a/contrib/vhost-user-scsi/vhost-user-scsi.c
+++ b/contrib/vhost-user-scsi/vhost-user-scsi.c
@@ -433,13 +433,16 @@ out:
     if (vdev_scsi) {
         g_main_loop_unref(vdev_scsi->loop);
         g_free(vdev_scsi);
-        unlink(opt_socket_path);
     }
     if (csock >= 0) {
         close(csock);
     }
     if (lsock >= 0) {
         close(lsock);
+
+        if (opt_socket_path) {
+            unlink(opt_socket_path);
+        }
     }
     g_free(opt_socket_path);
     g_free(iscsi_uri);

From e2c1d78da3223e8b39c14cce3da9e62655c1ddbb Mon Sep 17 00:00:00 2001
From: Vladislav Yaroshchuk <vladislav.yaroshchuk@jetbrains.com>
Date: Thu, 17 Mar 2022 20:28:33 +0300
Subject: [PATCH 416/935] net/vmnet: add vmnet dependency and customizable
 option

vmnet.framework dependency is added with 'vmnet' option
to enable or disable it. Default value is 'auto'.

used vmnet features are available since macOS 11.0,
but new backend can be built and work properly with
subset of them on 10.15 too.

Reviewed-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Tested-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Signed-off-by: Vladislav Yaroshchuk <Vladislav.Yaroshchuk@jetbrains.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 meson.build                   | 16 +++++++++++++++-
 meson_options.txt             |  2 ++
 scripts/meson-buildoptions.sh |  1 +
 3 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/meson.build b/meson.build
index 53a4728250..52f6befc0f 100644
--- a/meson.build
+++ b/meson.build
@@ -580,6 +580,18 @@ if cocoa.found() and get_option('gtk').enabled()
   error('Cocoa and GTK+ cannot be enabled at the same time')
 endif
 
+vmnet = dependency('appleframeworks', modules: 'vmnet', required: get_option('vmnet'))
+if vmnet.found() and not cc.has_header_symbol('vmnet/vmnet.h',
+                                              'VMNET_BRIDGED_MODE',
+                                              dependencies: vmnet)
+  vmnet = not_found
+  if get_option('vmnet').enabled()
+    error('vmnet.framework API is outdated')
+  else
+    warning('vmnet.framework API is outdated, disabling')
+  endif
+endif
+
 seccomp = not_found
 if not get_option('seccomp').auto() or have_system or have_tools
   seccomp = dependency('libseccomp', version: '>=2.3.0',
@@ -1741,6 +1753,7 @@ config_host_data.set('CONFIG_VHOST_KERNEL', have_vhost_kernel)
 config_host_data.set('CONFIG_VHOST_USER', have_vhost_user)
 config_host_data.set('CONFIG_VHOST_CRYPTO', have_vhost_user_crypto)
 config_host_data.set('CONFIG_VHOST_VDPA', have_vhost_vdpa)
+config_host_data.set('CONFIG_VMNET', vmnet.found())
 config_host_data.set('CONFIG_VHOST_USER_BLK_SERVER', have_vhost_user_blk_server)
 config_host_data.set('CONFIG_PNG', png.found())
 config_host_data.set('CONFIG_VNC', vnc.found())
@@ -3897,7 +3910,8 @@ summary(summary_info, bool_yn: true, section: 'Crypto')
 # Libraries
 summary_info = {}
 if targetos == 'darwin'
-  summary_info += {'Cocoa support':   cocoa}
+  summary_info += {'Cocoa support':           cocoa}
+  summary_info += {'vmnet.framework support': vmnet}
 endif
 summary_info += {'SDL support':       sdl}
 summary_info += {'SDL image support': sdl_image}
diff --git a/meson_options.txt b/meson_options.txt
index 29c6b90cec..7325cdad45 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -197,6 +197,8 @@ option('netmap', type : 'feature', value : 'auto',
        description: 'netmap network backend support')
 option('vde', type : 'feature', value : 'auto',
        description: 'vde network backend support')
+option('vmnet', type : 'feature', value : 'auto',
+       description: 'vmnet.framework network backend support')
 option('virglrenderer', type : 'feature', value : 'auto',
        description: 'virgl rendering support')
 option('png', type : 'feature', value : 'auto',
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index 5d2172bfb4..9b0d151dad 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -158,6 +158,7 @@ meson_options_help() {
   printf "%s\n" '  vhost-kernel    vhost kernel backend support'
   printf "%s\n" '  vhost-net       vhost-net kernel acceleration support'
   printf "%s\n" '  vhost-user      vhost-user backend support'
+  printf "%s\n" '  vmnet           vmnet.framework network backend support'
   printf "%s\n" '  vhost-user-blk-server'
   printf "%s\n" '                  build vhost-user-blk server'
   printf "%s\n" '  vhost-vdpa      vhost-vdpa kernel backend support'

From 81ad2964e93848fc0998145043c395dea67f00da Mon Sep 17 00:00:00 2001
From: Vladislav Yaroshchuk <vladislav.yaroshchuk@jetbrains.com>
Date: Thu, 17 Mar 2022 20:28:34 +0300
Subject: [PATCH 417/935] net/vmnet: add vmnet backends to qapi/net

Create separate netdevs for each vmnet operating mode:
- vmnet-host
- vmnet-shared
- vmnet-bridged

Reviewed-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Tested-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Vladislav Yaroshchuk <Vladislav.Yaroshchuk@jetbrains.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 net/clients.h       |  11 ++++
 net/meson.build     |   7 +++
 net/net.c           |  10 ++++
 net/vmnet-bridged.m |  25 +++++++++
 net/vmnet-common.m  |  19 +++++++
 net/vmnet-host.c    |  24 ++++++++
 net/vmnet-shared.c  |  25 +++++++++
 net/vmnet_int.h     |  25 +++++++++
 qapi/net.json       | 133 +++++++++++++++++++++++++++++++++++++++++++-
 9 files changed, 277 insertions(+), 2 deletions(-)
 create mode 100644 net/vmnet-bridged.m
 create mode 100644 net/vmnet-common.m
 create mode 100644 net/vmnet-host.c
 create mode 100644 net/vmnet-shared.c
 create mode 100644 net/vmnet_int.h

diff --git a/net/clients.h b/net/clients.h
index 92f9b59aed..c9157789f2 100644
--- a/net/clients.h
+++ b/net/clients.h
@@ -63,4 +63,15 @@ int net_init_vhost_user(const Netdev *netdev, const char *name,
 
 int net_init_vhost_vdpa(const Netdev *netdev, const char *name,
                         NetClientState *peer, Error **errp);
+#ifdef CONFIG_VMNET
+int net_init_vmnet_host(const Netdev *netdev, const char *name,
+                          NetClientState *peer, Error **errp);
+
+int net_init_vmnet_shared(const Netdev *netdev, const char *name,
+                          NetClientState *peer, Error **errp);
+
+int net_init_vmnet_bridged(const Netdev *netdev, const char *name,
+                          NetClientState *peer, Error **errp);
+#endif /* CONFIG_VMNET */
+
 #endif /* QEMU_NET_CLIENTS_H */
diff --git a/net/meson.build b/net/meson.build
index c965e83b26..754e2d1d40 100644
--- a/net/meson.build
+++ b/net/meson.build
@@ -44,4 +44,11 @@ if have_vhost_net_vdpa
   softmmu_ss.add(files('vhost-vdpa.c'))
 endif
 
+vmnet_files = files(
+  'vmnet-common.m',
+  'vmnet-bridged.m',
+  'vmnet-host.c',
+  'vmnet-shared.c'
+)
+softmmu_ss.add(when: vmnet, if_true: vmnet_files)
 subdir('can')
diff --git a/net/net.c b/net/net.c
index a094cf1d29..2db160e063 100644
--- a/net/net.c
+++ b/net/net.c
@@ -1020,6 +1020,11 @@ static int (* const net_client_init_fun[NET_CLIENT_DRIVER__MAX])(
 #ifdef CONFIG_L2TPV3
         [NET_CLIENT_DRIVER_L2TPV3]    = net_init_l2tpv3,
 #endif
+#ifdef CONFIG_VMNET
+        [NET_CLIENT_DRIVER_VMNET_HOST] = net_init_vmnet_host,
+        [NET_CLIENT_DRIVER_VMNET_SHARED] = net_init_vmnet_shared,
+        [NET_CLIENT_DRIVER_VMNET_BRIDGED] = net_init_vmnet_bridged,
+#endif /* CONFIG_VMNET */
 };
 
 
@@ -1105,6 +1110,11 @@ void show_netdevs(void)
 #endif
 #ifdef CONFIG_VHOST_VDPA
         "vhost-vdpa",
+#endif
+#ifdef CONFIG_VMNET
+        "vmnet-host",
+        "vmnet-shared",
+        "vmnet-bridged",
 #endif
     };
 
diff --git a/net/vmnet-bridged.m b/net/vmnet-bridged.m
new file mode 100644
index 0000000000..91c1a2f2c7
--- /dev/null
+++ b/net/vmnet-bridged.m
@@ -0,0 +1,25 @@
+/*
+ * vmnet-bridged.m
+ *
+ * Copyright(c) 2022 Vladislav Yaroshchuk <vladislav.yaroshchuk@jetbrains.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/qapi-types-net.h"
+#include "vmnet_int.h"
+#include "clients.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
+
+#include <vmnet/vmnet.h>
+
+int net_init_vmnet_bridged(const Netdev *netdev, const char *name,
+                           NetClientState *peer, Error **errp)
+{
+  error_setg(errp, "vmnet-bridged is not implemented yet");
+  return -1;
+}
diff --git a/net/vmnet-common.m b/net/vmnet-common.m
new file mode 100644
index 0000000000..3bf42fc643
--- /dev/null
+++ b/net/vmnet-common.m
@@ -0,0 +1,19 @@
+/*
+ * vmnet-common.m - network client wrapper for Apple vmnet.framework
+ *
+ * Copyright(c) 2022 Vladislav Yaroshchuk <vladislav.yaroshchuk@jetbrains.com>
+ * Copyright(c) 2021 Phillip Tennen <phillip@axleos.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/qapi-types-net.h"
+#include "vmnet_int.h"
+#include "clients.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
+
+#include <vmnet/vmnet.h>
diff --git a/net/vmnet-host.c b/net/vmnet-host.c
new file mode 100644
index 0000000000..a461d507c5
--- /dev/null
+++ b/net/vmnet-host.c
@@ -0,0 +1,24 @@
+/*
+ * vmnet-host.c
+ *
+ * Copyright(c) 2022 Vladislav Yaroshchuk <vladislav.yaroshchuk@jetbrains.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/qapi-types-net.h"
+#include "vmnet_int.h"
+#include "clients.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
+
+#include <vmnet/vmnet.h>
+
+int net_init_vmnet_host(const Netdev *netdev, const char *name,
+                        NetClientState *peer, Error **errp) {
+  error_setg(errp, "vmnet-host is not implemented yet");
+  return -1;
+}
diff --git a/net/vmnet-shared.c b/net/vmnet-shared.c
new file mode 100644
index 0000000000..6dfb133a18
--- /dev/null
+++ b/net/vmnet-shared.c
@@ -0,0 +1,25 @@
+/*
+ * vmnet-shared.c
+ *
+ * Copyright(c) 2022 Vladislav Yaroshchuk <vladislav.yaroshchuk@jetbrains.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/qapi-types-net.h"
+#include "vmnet_int.h"
+#include "clients.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
+
+#include <vmnet/vmnet.h>
+
+int net_init_vmnet_shared(const Netdev *netdev, const char *name,
+                          NetClientState *peer, Error **errp)
+{
+  error_setg(errp, "vmnet-shared is not implemented yet");
+  return -1;
+}
diff --git a/net/vmnet_int.h b/net/vmnet_int.h
new file mode 100644
index 0000000000..c383038a1d
--- /dev/null
+++ b/net/vmnet_int.h
@@ -0,0 +1,25 @@
+/*
+ * vmnet_int.h
+ *
+ * Copyright(c) 2022 Vladislav Yaroshchuk <vladislav.yaroshchuk@jetbrains.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+#ifndef VMNET_INT_H
+#define VMNET_INT_H
+
+#include "qemu/osdep.h"
+#include "vmnet_int.h"
+#include "clients.h"
+
+#include <vmnet/vmnet.h>
+
+typedef struct VmnetState {
+  NetClientState nc;
+
+} VmnetState;
+
+
+#endif /* VMNET_INT_H */
diff --git a/qapi/net.json b/qapi/net.json
index b92f3f5fb4..d6f7cfd4d6 100644
--- a/qapi/net.json
+++ b/qapi/net.json
@@ -452,6 +452,120 @@
     '*vhostdev':     'str',
     '*queues':       'int' } }
 
+##
+# @NetdevVmnetHostOptions:
+#
+# vmnet (host mode) network backend.
+#
+# Allows the vmnet interface to communicate with other vmnet
+# interfaces that are in host mode and also with the host.
+#
+# @start-address: The starting IPv4 address to use for the interface.
+#                 Must be in the private IP range (RFC 1918). Must be
+#                 specified along with @end-address and @subnet-mask.
+#                 This address is used as the gateway address. The
+#                 subsequent address up to and including end-address are
+#                 placed in the DHCP pool.
+#
+# @end-address: The DHCP IPv4 range end address to use for the
+#               interface. Must be in the private IP range (RFC 1918).
+#               Must be specified along with @start-address and
+#               @subnet-mask.
+#
+# @subnet-mask: The IPv4 subnet mask to use on the interface. Must
+#               be specified along with @start-address and @subnet-mask.
+#
+# @isolated: Enable isolation for this interface. Interface isolation
+#            ensures that vmnet interface is not able to communicate
+#            with any other vmnet interfaces. Only communication with
+#            host is allowed. Requires at least macOS Big Sur 11.0.
+#
+# @net-uuid: The identifier (UUID) to uniquely identify the isolated
+#            network vmnet interface should be added to. If
+#            set, no DHCP service is provided for this interface and
+#            network communication is allowed only with other interfaces
+#            added to this network identified by the UUID. Requires
+#            at least macOS Big Sur 11.0.
+#
+# Since: 7.1
+##
+{ 'struct': 'NetdevVmnetHostOptions',
+  'data': {
+    '*start-address': 'str',
+    '*end-address':   'str',
+    '*subnet-mask':   'str',
+    '*isolated':      'bool',
+    '*net-uuid':      'str' },
+  'if': 'CONFIG_VMNET' }
+
+##
+# @NetdevVmnetSharedOptions:
+#
+# vmnet (shared mode) network backend.
+#
+# Allows traffic originating from the vmnet interface to reach the
+# Internet through a network address translator (NAT).
+# The vmnet interface can communicate with the host and with
+# other shared mode interfaces on the same subnet. If no DHCP
+# settings, subnet mask and IPv6 prefix specified, the interface can
+# communicate with any of other interfaces in shared mode.
+#
+# @start-address: The starting IPv4 address to use for the interface.
+#                 Must be in the private IP range (RFC 1918). Must be
+#                 specified along with @end-address and @subnet-mask.
+#                 This address is used as the gateway address. The
+#                 subsequent address up to and including end-address are
+#                 placed in the DHCP pool.
+#
+# @end-address: The DHCP IPv4 range end address to use for the
+#               interface. Must be in the private IP range (RFC 1918).
+#               Must be specified along with @start-address and @subnet-mask.
+#
+# @subnet-mask: The IPv4 subnet mask to use on the interface. Must
+#                be specified along with @start-address and @subnet-mask.
+#
+# @isolated: Enable isolation for this interface. Interface isolation
+#            ensures that vmnet interface is not able to communicate
+#            with any other vmnet interfaces. Only communication with
+#            host is allowed. Requires at least macOS Big Sur 11.0.
+#
+# @nat66-prefix: The IPv6 prefix to use into guest network. Must be a
+#                unique local address i.e. start with fd00::/8 and have
+#                length of 64.
+#
+# Since: 7.1
+##
+{ 'struct': 'NetdevVmnetSharedOptions',
+  'data': {
+    '*start-address': 'str',
+    '*end-address':   'str',
+    '*subnet-mask':   'str',
+    '*isolated':      'bool',
+    '*nat66-prefix':  'str' },
+  'if': 'CONFIG_VMNET' }
+
+##
+# @NetdevVmnetBridgedOptions:
+#
+# vmnet (bridged mode) network backend.
+#
+# Bridges the vmnet interface with a physical network interface.
+#
+# @ifname: The name of the physical interface to be bridged.
+#
+# @isolated: Enable isolation for this interface. Interface isolation
+#            ensures that vmnet interface is not able to communicate
+#            with any other vmnet interfaces. Only communication with
+#            host is allowed. Requires at least macOS Big Sur 11.0.
+#
+# Since: 7.1
+##
+{ 'struct': 'NetdevVmnetBridgedOptions',
+  'data': {
+    'ifname':     'str',
+    '*isolated':  'bool' },
+  'if': 'CONFIG_VMNET' }
+
 ##
 # @NetClientDriver:
 #
@@ -460,10 +574,16 @@
 # Since: 2.7
 #
 #        @vhost-vdpa since 5.1
+#        @vmnet-host since 7.1
+#        @vmnet-shared since 7.1
+#        @vmnet-bridged since 7.1
 ##
 { 'enum': 'NetClientDriver',
   'data': [ 'none', 'nic', 'user', 'tap', 'l2tpv3', 'socket', 'vde',
-            'bridge', 'hubport', 'netmap', 'vhost-user', 'vhost-vdpa' ] }
+            'bridge', 'hubport', 'netmap', 'vhost-user', 'vhost-vdpa',
+            { 'name': 'vmnet-host', 'if': 'CONFIG_VMNET' },
+            { 'name': 'vmnet-shared', 'if': 'CONFIG_VMNET' },
+            { 'name': 'vmnet-bridged', 'if': 'CONFIG_VMNET' }] }
 
 ##
 # @Netdev:
@@ -477,6 +597,9 @@
 # Since: 1.2
 #
 #        'l2tpv3' - since 2.1
+#        'vmnet-host' - since 7.1
+#        'vmnet-shared' - since 7.1
+#        'vmnet-bridged' - since 7.1
 ##
 { 'union': 'Netdev',
   'base': { 'id': 'str', 'type': 'NetClientDriver' },
@@ -492,7 +615,13 @@
     'hubport':  'NetdevHubPortOptions',
     'netmap':   'NetdevNetmapOptions',
     'vhost-user': 'NetdevVhostUserOptions',
-    'vhost-vdpa': 'NetdevVhostVDPAOptions' } }
+    'vhost-vdpa': 'NetdevVhostVDPAOptions',
+    'vmnet-host': { 'type': 'NetdevVmnetHostOptions',
+                    'if': 'CONFIG_VMNET' },
+    'vmnet-shared': { 'type': 'NetdevVmnetSharedOptions',
+                      'if': 'CONFIG_VMNET' },
+    'vmnet-bridged': { 'type': 'NetdevVmnetBridgedOptions',
+                       'if': 'CONFIG_VMNET' } } }
 
 ##
 # @RxState:

From 73f99db534e36cdf7dd045b3fde2af29132c0a35 Mon Sep 17 00:00:00 2001
From: Vladislav Yaroshchuk <vladislav.yaroshchuk@jetbrains.com>
Date: Thu, 17 Mar 2022 20:28:35 +0300
Subject: [PATCH 418/935] net/vmnet: implement shared mode (vmnet-shared)

Interaction with vmnet.framework in different modes
differs only on configuration stage, so we can create
common `send`, `receive`, etc. procedures and reuse them.

Reviewed-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Tested-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Signed-off-by: Phillip Tennen <phillip@axleos.com>
Signed-off-by: Vladislav Yaroshchuk <Vladislav.Yaroshchuk@jetbrains.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 net/vmnet-common.m | 359 +++++++++++++++++++++++++++++++++++++++++++++
 net/vmnet-shared.c |  97 +++++++++++-
 net/vmnet_int.h    |  40 ++++-
 3 files changed, 491 insertions(+), 5 deletions(-)

diff --git a/net/vmnet-common.m b/net/vmnet-common.m
index 3bf42fc643..2cb60b9ddd 100644
--- a/net/vmnet-common.m
+++ b/net/vmnet-common.m
@@ -10,6 +10,8 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/main-loop.h"
+#include "qemu/log.h"
 #include "qapi/qapi-types-net.h"
 #include "vmnet_int.h"
 #include "clients.h"
@@ -17,3 +19,360 @@
 #include "qapi/error.h"
 
 #include <vmnet/vmnet.h>
+#include <dispatch/dispatch.h>
+
+
+static void vmnet_send_completed(NetClientState *nc, ssize_t len);
+
+
+const char *vmnet_status_map_str(vmnet_return_t status)
+{
+    switch (status) {
+    case VMNET_SUCCESS:
+        return "success";
+    case VMNET_FAILURE:
+        return "general failure (possibly not enough privileges)";
+    case VMNET_MEM_FAILURE:
+        return "memory allocation failure";
+    case VMNET_INVALID_ARGUMENT:
+        return "invalid argument specified";
+    case VMNET_SETUP_INCOMPLETE:
+        return "interface setup is not complete";
+    case VMNET_INVALID_ACCESS:
+        return "invalid access, permission denied";
+    case VMNET_PACKET_TOO_BIG:
+        return "packet size is larger than MTU";
+    case VMNET_BUFFER_EXHAUSTED:
+        return "buffers exhausted in kernel";
+    case VMNET_TOO_MANY_PACKETS:
+        return "packet count exceeds limit";
+#if defined(MAC_OS_VERSION_11_0) && \
+    MAC_OS_X_VERSION_MIN_REQUIRED >= MAC_OS_VERSION_11_0
+    case VMNET_SHARING_SERVICE_BUSY:
+        return "conflict, sharing service is in use";
+#endif
+    default:
+        return "unknown vmnet error";
+    }
+}
+
+
+/**
+ * Write packets from QEMU to vmnet interface.
+ *
+ * vmnet.framework supports iov, but writing more than
+ * one iov into vmnet interface fails with
+ * 'VMNET_INVALID_ARGUMENT'. Collecting provided iovs into
+ * one and passing it to vmnet works fine. That's the
+ * reason why receive_iov() left unimplemented. But it still
+ * works with good performance having .receive() only.
+ */
+ssize_t vmnet_receive_common(NetClientState *nc,
+                             const uint8_t *buf,
+                             size_t size)
+{
+    VmnetState *s = DO_UPCAST(VmnetState, nc, nc);
+    struct vmpktdesc packet;
+    struct iovec iov;
+    int pkt_cnt;
+    vmnet_return_t if_status;
+
+    if (size > s->max_packet_size) {
+        warn_report("vmnet: packet is too big, %zu > %" PRIu64,
+            packet.vm_pkt_size,
+            s->max_packet_size);
+        return -1;
+    }
+
+    iov.iov_base = (char *) buf;
+    iov.iov_len = size;
+
+    packet.vm_pkt_iovcnt = 1;
+    packet.vm_flags = 0;
+    packet.vm_pkt_size = size;
+    packet.vm_pkt_iov = &iov;
+    pkt_cnt = 1;
+
+    if_status = vmnet_write(s->vmnet_if, &packet, &pkt_cnt);
+    if (if_status != VMNET_SUCCESS) {
+        error_report("vmnet: write error: %s\n",
+                     vmnet_status_map_str(if_status));
+        return -1;
+    }
+
+    if (pkt_cnt) {
+        return size;
+    }
+    return 0;
+}
+
+
+/**
+ * Read packets from vmnet interface and write them
+ * to temporary buffers in VmnetState.
+ *
+ * Returns read packets number (may be 0) on success,
+ * -1 on error
+ */
+static int vmnet_read_packets(VmnetState *s)
+{
+    assert(s->packets_send_current_pos == s->packets_send_end_pos);
+
+    struct vmpktdesc *packets = s->packets_buf;
+    vmnet_return_t status;
+    int i;
+
+    /* Read as many packets as present */
+    s->packets_send_current_pos = 0;
+    s->packets_send_end_pos = VMNET_PACKETS_LIMIT;
+    for (i = 0; i < s->packets_send_end_pos; ++i) {
+        packets[i].vm_pkt_size = s->max_packet_size;
+        packets[i].vm_pkt_iovcnt = 1;
+        packets[i].vm_flags = 0;
+    }
+
+    status = vmnet_read(s->vmnet_if, packets, &s->packets_send_end_pos);
+    if (status != VMNET_SUCCESS) {
+        error_printf("vmnet: read failed: %s\n",
+                     vmnet_status_map_str(status));
+        s->packets_send_current_pos = 0;
+        s->packets_send_end_pos = 0;
+        return -1;
+    }
+    return s->packets_send_end_pos;
+}
+
+
+/**
+ * Write packets from temporary buffers in VmnetState
+ * to QEMU.
+ */
+static void vmnet_write_packets_to_qemu(VmnetState *s)
+{
+    while (s->packets_send_current_pos < s->packets_send_end_pos) {
+        ssize_t size = qemu_send_packet_async(&s->nc,
+                                      s->iov_buf[s->packets_send_current_pos].iov_base,
+                                      s->packets_buf[s->packets_send_current_pos].vm_pkt_size,
+                                      vmnet_send_completed);
+
+        if (size == 0) {
+            /* QEMU is not ready to consume more packets -
+             * stop and wait for completion callback call */
+            return;
+        }
+        ++s->packets_send_current_pos;
+    }
+}
+
+
+/**
+ * Bottom half callback that transfers packets from vmnet interface
+ * to QEMU.
+ *
+ * The process of transferring packets is three-staged:
+ * 1. Handle vmnet event;
+ * 2. Read packets from vmnet interface into temporary buffer;
+ * 3. Write packets from temporary buffer to QEMU.
+ *
+ * QEMU may suspend this process on the last stage, returning 0 from
+ * qemu_send_packet_async function. If this happens, we should
+ * respectfully wait until it is ready to consume more packets,
+ * write left ones in temporary buffer and only after this
+ * continue reading more packets from vmnet interface.
+ *
+ * Packets to be transferred are stored into packets_buf,
+ * in the window [packets_send_current_pos..packets_send_end_pos)
+ * including current_pos, excluding end_pos.
+ *
+ * Thus, if QEMU is not ready, buffer is not read and
+ * packets_send_current_pos < packets_send_end_pos.
+ */
+static void vmnet_send_bh(void *opaque)
+{
+    NetClientState *nc = (NetClientState *) opaque;
+    VmnetState *s = DO_UPCAST(VmnetState, nc, nc);
+
+    /*
+     * Do nothing if QEMU is not ready - wait
+     * for completion callback invocation
+     */
+    if (s->packets_send_current_pos < s->packets_send_end_pos) {
+        return;
+    }
+
+    /* Read packets from vmnet interface */
+    if (vmnet_read_packets(s) > 0) {
+        /* Send them to QEMU */
+        vmnet_write_packets_to_qemu(s);
+    }
+}
+
+
+/**
+ * Completion callback to be invoked by QEMU when it becomes
+ * ready to consume more packets.
+ */
+static void vmnet_send_completed(NetClientState *nc, ssize_t len)
+{
+    VmnetState *s = DO_UPCAST(VmnetState, nc, nc);
+
+    /* Callback is invoked eq queued packet is sent */
+    ++s->packets_send_current_pos;
+
+    /* Complete sending packets left in VmnetState buffers */
+    vmnet_write_packets_to_qemu(s);
+
+    /* And read new ones from vmnet if VmnetState buffer is ready */
+    if (s->packets_send_current_pos < s->packets_send_end_pos) {
+        qemu_bh_schedule(s->send_bh);
+    }
+}
+
+
+static void vmnet_bufs_init(VmnetState *s)
+{
+    struct vmpktdesc *packets = s->packets_buf;
+    struct iovec *iov = s->iov_buf;
+    int i;
+
+    for (i = 0; i < VMNET_PACKETS_LIMIT; ++i) {
+        iov[i].iov_len = s->max_packet_size;
+        iov[i].iov_base = g_malloc0(iov[i].iov_len);
+        packets[i].vm_pkt_iov = iov + i;
+    }
+}
+
+
+int vmnet_if_create(NetClientState *nc,
+                    xpc_object_t if_desc,
+                    Error **errp)
+{
+    VmnetState *s = DO_UPCAST(VmnetState, nc, nc);
+    dispatch_semaphore_t if_created_sem = dispatch_semaphore_create(0);
+    __block vmnet_return_t if_status;
+
+    s->if_queue = dispatch_queue_create(
+        "org.qemu.vmnet.if_queue",
+        DISPATCH_QUEUE_SERIAL
+    );
+
+    xpc_dictionary_set_bool(
+        if_desc,
+        vmnet_allocate_mac_address_key,
+        false
+    );
+
+#ifdef DEBUG
+    qemu_log("vmnet.start.interface_desc:\n");
+    xpc_dictionary_apply(if_desc,
+                         ^bool(const char *k, xpc_object_t v) {
+                             char *desc = xpc_copy_description(v);
+                             qemu_log("  %s=%s\n", k, desc);
+                             free(desc);
+                             return true;
+                         });
+#endif /* DEBUG */
+
+    s->vmnet_if = vmnet_start_interface(
+        if_desc,
+        s->if_queue,
+        ^(vmnet_return_t status, xpc_object_t interface_param) {
+            if_status = status;
+            if (status != VMNET_SUCCESS || !interface_param) {
+                dispatch_semaphore_signal(if_created_sem);
+                return;
+            }
+
+#ifdef DEBUG
+            qemu_log("vmnet.start.interface_param:\n");
+            xpc_dictionary_apply(interface_param,
+                                 ^bool(const char *k, xpc_object_t v) {
+                                     char *desc = xpc_copy_description(v);
+                                     qemu_log("  %s=%s\n", k, desc);
+                                     free(desc);
+                                     return true;
+                                 });
+#endif /* DEBUG */
+
+            s->mtu = xpc_dictionary_get_uint64(
+                interface_param,
+                vmnet_mtu_key);
+            s->max_packet_size = xpc_dictionary_get_uint64(
+                interface_param,
+                vmnet_max_packet_size_key);
+
+            dispatch_semaphore_signal(if_created_sem);
+        });
+
+    if (s->vmnet_if == NULL) {
+        dispatch_release(s->if_queue);
+        dispatch_release(if_created_sem);
+        error_setg(errp,
+                   "unable to create interface with requested params");
+        return -1;
+    }
+
+    dispatch_semaphore_wait(if_created_sem, DISPATCH_TIME_FOREVER);
+    dispatch_release(if_created_sem);
+
+    if (if_status != VMNET_SUCCESS) {
+        dispatch_release(s->if_queue);
+        error_setg(errp,
+                   "cannot create vmnet interface: %s",
+                   vmnet_status_map_str(if_status));
+        return -1;
+    }
+
+    s->send_bh = aio_bh_new(qemu_get_aio_context(), vmnet_send_bh, nc);
+    vmnet_bufs_init(s);
+
+    s->packets_send_current_pos = 0;
+    s->packets_send_end_pos = 0;
+
+    vmnet_interface_set_event_callback(
+        s->vmnet_if,
+        VMNET_INTERFACE_PACKETS_AVAILABLE,
+        s->if_queue,
+        ^(interface_event_t event_id, xpc_object_t event) {
+            assert(event_id == VMNET_INTERFACE_PACKETS_AVAILABLE);
+            /*
+             * This function is being called from a non qemu thread, so
+             * we only schedule a BH, and do the rest of the io completion
+             * handling from vmnet_send_bh() which runs in a qemu context.
+             */
+            qemu_bh_schedule(s->send_bh);
+        });
+
+    return 0;
+}
+
+
+void vmnet_cleanup_common(NetClientState *nc)
+{
+    VmnetState *s = DO_UPCAST(VmnetState, nc, nc);
+    dispatch_semaphore_t if_stopped_sem;
+
+    if (s->vmnet_if == NULL) {
+        return;
+    }
+
+    if_stopped_sem = dispatch_semaphore_create(0);
+    vmnet_stop_interface(
+        s->vmnet_if,
+        s->if_queue,
+        ^(vmnet_return_t status) {
+            assert(status == VMNET_SUCCESS);
+            dispatch_semaphore_signal(if_stopped_sem);
+        });
+    dispatch_semaphore_wait(if_stopped_sem, DISPATCH_TIME_FOREVER);
+
+    qemu_purge_queued_packets(nc);
+
+    qemu_bh_delete(s->send_bh);
+    dispatch_release(if_stopped_sem);
+    dispatch_release(s->if_queue);
+
+    for (int i = 0; i < VMNET_PACKETS_LIMIT; ++i) {
+        g_free(s->iov_buf[i].iov_base);
+    }
+}
diff --git a/net/vmnet-shared.c b/net/vmnet-shared.c
index 6dfb133a18..18cadc72bd 100644
--- a/net/vmnet-shared.c
+++ b/net/vmnet-shared.c
@@ -10,16 +10,105 @@
 
 #include "qemu/osdep.h"
 #include "qapi/qapi-types-net.h"
+#include "qapi/error.h"
 #include "vmnet_int.h"
 #include "clients.h"
-#include "qemu/error-report.h"
-#include "qapi/error.h"
 
 #include <vmnet/vmnet.h>
 
+
+static bool validate_options(const Netdev *netdev, Error **errp)
+{
+    const NetdevVmnetSharedOptions *options = &(netdev->u.vmnet_shared);
+
+#if !defined(MAC_OS_VERSION_11_0) || \
+    MAC_OS_X_VERSION_MIN_REQUIRED < MAC_OS_VERSION_11_0
+    if (options->has_isolated) {
+        error_setg(errp,
+                   "vmnet-shared.isolated feature is "
+                   "unavailable: outdated vmnet.framework API");
+        return false;
+    }
+#endif
+
+    if ((options->has_start_address ||
+         options->has_end_address ||
+         options->has_subnet_mask) &&
+        !(options->has_start_address &&
+          options->has_end_address &&
+          options->has_subnet_mask)) {
+        error_setg(errp,
+                   "'start-address', 'end-address', 'subnet-mask' "
+                   "should be provided together"
+        );
+        return false;
+    }
+
+    return true;
+}
+
+static xpc_object_t build_if_desc(const Netdev *netdev)
+{
+    const NetdevVmnetSharedOptions *options = &(netdev->u.vmnet_shared);
+    xpc_object_t if_desc = xpc_dictionary_create(NULL, NULL, 0);
+
+    xpc_dictionary_set_uint64(
+        if_desc,
+        vmnet_operation_mode_key,
+        VMNET_SHARED_MODE
+    );
+
+    if (options->has_nat66_prefix) {
+        xpc_dictionary_set_string(if_desc,
+                                  vmnet_nat66_prefix_key,
+                                  options->nat66_prefix);
+    }
+
+    if (options->has_start_address) {
+        xpc_dictionary_set_string(if_desc,
+                                  vmnet_start_address_key,
+                                  options->start_address);
+        xpc_dictionary_set_string(if_desc,
+                                  vmnet_end_address_key,
+                                  options->end_address);
+        xpc_dictionary_set_string(if_desc,
+                                  vmnet_subnet_mask_key,
+                                  options->subnet_mask);
+    }
+
+#if defined(MAC_OS_VERSION_11_0) && \
+    MAC_OS_X_VERSION_MIN_REQUIRED >= MAC_OS_VERSION_11_0
+    xpc_dictionary_set_bool(
+        if_desc,
+        vmnet_enable_isolation_key,
+        options->isolated
+    );
+#endif
+
+    return if_desc;
+}
+
+static NetClientInfo net_vmnet_shared_info = {
+    .type = NET_CLIENT_DRIVER_VMNET_SHARED,
+    .size = sizeof(VmnetState),
+    .receive = vmnet_receive_common,
+    .cleanup = vmnet_cleanup_common,
+};
+
 int net_init_vmnet_shared(const Netdev *netdev, const char *name,
                           NetClientState *peer, Error **errp)
 {
-  error_setg(errp, "vmnet-shared is not implemented yet");
-  return -1;
+    NetClientState *nc = qemu_new_net_client(&net_vmnet_shared_info,
+                                             peer, "vmnet-shared", name);
+    xpc_object_t if_desc;
+    int result = -1;
+
+    if (!validate_options(netdev, errp)) {
+        return result;
+    }
+
+    if_desc = build_if_desc(netdev);
+    result = vmnet_if_create(nc, if_desc, errp);
+    xpc_release(if_desc);
+    return result;
 }
diff --git a/net/vmnet_int.h b/net/vmnet_int.h
index c383038a1d..adf6e8c20d 100644
--- a/net/vmnet_int.h
+++ b/net/vmnet_int.h
@@ -15,11 +15,49 @@
 #include "clients.h"
 
 #include <vmnet/vmnet.h>
+#include <dispatch/dispatch.h>
+
+/**
+ *  From vmnet.framework documentation
+ *
+ *  Each read/write call allows up to 200 packets to be
+ *  read or written for a maximum of 256KB.
+ *
+ *  Each packet written should be a complete
+ *  ethernet frame.
+ *
+ *  https://developer.apple.com/documentation/vmnet
+ */
+#define VMNET_PACKETS_LIMIT 200
 
 typedef struct VmnetState {
-  NetClientState nc;
+    NetClientState nc;
+    interface_ref vmnet_if;
 
+    uint64_t mtu;
+    uint64_t max_packet_size;
+
+    dispatch_queue_t if_queue;
+
+    QEMUBH *send_bh;
+
+    struct vmpktdesc packets_buf[VMNET_PACKETS_LIMIT];
+    int packets_send_current_pos;
+    int packets_send_end_pos;
+
+    struct iovec iov_buf[VMNET_PACKETS_LIMIT];
 } VmnetState;
 
+const char *vmnet_status_map_str(vmnet_return_t status);
+
+int vmnet_if_create(NetClientState *nc,
+                    xpc_object_t if_desc,
+                    Error **errp);
+
+ssize_t vmnet_receive_common(NetClientState *nc,
+                             const uint8_t *buf,
+                             size_t size);
+
+void vmnet_cleanup_common(NetClientState *nc);
 
 #endif /* VMNET_INT_H */

From 5cfa6437309fbf6c52a941898ca6b94dcddb5584 Mon Sep 17 00:00:00 2001
From: Vladislav Yaroshchuk <vladislav.yaroshchuk@jetbrains.com>
Date: Thu, 17 Mar 2022 20:28:36 +0300
Subject: [PATCH 419/935] net/vmnet: implement host mode (vmnet-host)

Reviewed-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Tested-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Signed-off-by: Vladislav Yaroshchuk <Vladislav.Yaroshchuk@jetbrains.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 net/vmnet-host.c | 118 ++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 111 insertions(+), 7 deletions(-)

diff --git a/net/vmnet-host.c b/net/vmnet-host.c
index a461d507c5..05f8d78864 100644
--- a/net/vmnet-host.c
+++ b/net/vmnet-host.c
@@ -9,16 +9,120 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/uuid.h"
 #include "qapi/qapi-types-net.h"
-#include "vmnet_int.h"
-#include "clients.h"
-#include "qemu/error-report.h"
 #include "qapi/error.h"
+#include "clients.h"
+#include "vmnet_int.h"
 
 #include <vmnet/vmnet.h>
 
-int net_init_vmnet_host(const Netdev *netdev, const char *name,
-                        NetClientState *peer, Error **errp) {
-  error_setg(errp, "vmnet-host is not implemented yet");
-  return -1;
+
+static bool validate_options(const Netdev *netdev, Error **errp)
+{
+    const NetdevVmnetHostOptions *options = &(netdev->u.vmnet_host);
+
+#if defined(MAC_OS_VERSION_11_0) && \
+    MAC_OS_X_VERSION_MIN_REQUIRED >= MAC_OS_VERSION_11_0
+
+    QemuUUID net_uuid;
+    if (options->has_net_uuid &&
+        qemu_uuid_parse(options->net_uuid, &net_uuid) < 0) {
+        error_setg(errp, "Invalid UUID provided in 'net-uuid'");
+        return false;
+    }
+#else
+    if (options->has_isolated) {
+        error_setg(errp,
+                   "vmnet-host.isolated feature is "
+                   "unavailable: outdated vmnet.framework API");
+        return false;
+    }
+
+    if (options->has_net_uuid) {
+        error_setg(errp,
+                   "vmnet-host.net-uuid feature is "
+                   "unavailable: outdated vmnet.framework API");
+        return false;
+    }
+#endif
+
+    if ((options->has_start_address ||
+         options->has_end_address ||
+         options->has_subnet_mask) &&
+        !(options->has_start_address &&
+          options->has_end_address &&
+          options->has_subnet_mask)) {
+        error_setg(errp,
+                   "'start-address', 'end-address', 'subnet-mask' "
+                   "should be provided together");
+        return false;
+    }
+
+    return true;
+}
+
+static xpc_object_t build_if_desc(const Netdev *netdev)
+{
+    const NetdevVmnetHostOptions *options = &(netdev->u.vmnet_host);
+    xpc_object_t if_desc = xpc_dictionary_create(NULL, NULL, 0);
+
+    xpc_dictionary_set_uint64(if_desc,
+                              vmnet_operation_mode_key,
+                              VMNET_HOST_MODE);
+
+#if defined(MAC_OS_VERSION_11_0) && \
+    MAC_OS_X_VERSION_MIN_REQUIRED >= MAC_OS_VERSION_11_0
+
+    xpc_dictionary_set_bool(if_desc,
+                            vmnet_enable_isolation_key,
+                            options->isolated);
+
+    QemuUUID net_uuid;
+    if (options->has_net_uuid) {
+        qemu_uuid_parse(options->net_uuid, &net_uuid);
+        xpc_dictionary_set_uuid(if_desc,
+                                vmnet_network_identifier_key,
+                                net_uuid.data);
+    }
+#endif
+
+    if (options->has_start_address) {
+        xpc_dictionary_set_string(if_desc,
+                                  vmnet_start_address_key,
+                                  options->start_address);
+        xpc_dictionary_set_string(if_desc,
+                                  vmnet_end_address_key,
+                                  options->end_address);
+        xpc_dictionary_set_string(if_desc,
+                                  vmnet_subnet_mask_key,
+                                  options->subnet_mask);
+    }
+
+    return if_desc;
+}
+
+static NetClientInfo net_vmnet_host_info = {
+    .type = NET_CLIENT_DRIVER_VMNET_HOST,
+    .size = sizeof(VmnetState),
+    .receive = vmnet_receive_common,
+    .cleanup = vmnet_cleanup_common,
+};
+
+int net_init_vmnet_host(const Netdev *netdev, const char *name,
+                        NetClientState *peer, Error **errp)
+{
+    NetClientState *nc = qemu_new_net_client(&net_vmnet_host_info,
+                                             peer, "vmnet-host", name);
+    xpc_object_t if_desc;
+    int result = -1;
+
+    if (!validate_options(netdev, errp)) {
+        return result;
+    }
+
+    if_desc = build_if_desc(netdev);
+    result = vmnet_if_create(nc, if_desc, errp);
+    xpc_release(if_desc);
+    return result;
 }

From 2c313ae2a0676d79eaa260a2cba64d93feb0683f Mon Sep 17 00:00:00 2001
From: Vladislav Yaroshchuk <vladislav.yaroshchuk@jetbrains.com>
Date: Thu, 17 Mar 2022 20:28:37 +0300
Subject: [PATCH 420/935] net/vmnet: implement bridged mode (vmnet-bridged)

Reviewed-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Tested-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Signed-off-by: Vladislav Yaroshchuk <Vladislav.Yaroshchuk@jetbrains.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 net/vmnet-bridged.m | 137 ++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 132 insertions(+), 5 deletions(-)

diff --git a/net/vmnet-bridged.m b/net/vmnet-bridged.m
index 91c1a2f2c7..46d2282863 100644
--- a/net/vmnet-bridged.m
+++ b/net/vmnet-bridged.m
@@ -10,16 +10,143 @@
 
 #include "qemu/osdep.h"
 #include "qapi/qapi-types-net.h"
-#include "vmnet_int.h"
-#include "clients.h"
-#include "qemu/error-report.h"
 #include "qapi/error.h"
+#include "clients.h"
+#include "vmnet_int.h"
 
 #include <vmnet/vmnet.h>
 
+
+static bool validate_ifname(const char *ifname)
+{
+    xpc_object_t shared_if_list = vmnet_copy_shared_interface_list();
+    bool match = false;
+    if (!xpc_array_get_count(shared_if_list)) {
+        goto done;
+    }
+
+    match = !xpc_array_apply(
+        shared_if_list,
+        ^bool(size_t index, xpc_object_t value) {
+            return strcmp(xpc_string_get_string_ptr(value), ifname) != 0;
+        });
+
+done:
+    xpc_release(shared_if_list);
+    return match;
+}
+
+
+static char* get_valid_ifnames()
+{
+    xpc_object_t shared_if_list = vmnet_copy_shared_interface_list();
+    __block char *if_list = NULL;
+    __block char *if_list_prev = NULL;
+
+    if (!xpc_array_get_count(shared_if_list)) {
+        goto done;
+    }
+
+    xpc_array_apply(
+        shared_if_list,
+        ^bool(size_t index, xpc_object_t value) {
+            /* build list of strings like "en0 en1 en2 " */
+            if_list = g_strconcat(xpc_string_get_string_ptr(value),
+                                  " ",
+                                  if_list_prev,
+                                  NULL);
+            g_free(if_list_prev);
+            if_list_prev = if_list;
+            return true;
+        });
+
+done:
+    xpc_release(shared_if_list);
+    return if_list;
+}
+
+
+static bool validate_options(const Netdev *netdev, Error **errp)
+{
+    const NetdevVmnetBridgedOptions *options = &(netdev->u.vmnet_bridged);
+    char* if_list;
+
+    if (!validate_ifname(options->ifname)) {
+        if_list = get_valid_ifnames();
+        if (if_list) {
+            error_setg(errp,
+                       "unsupported ifname '%s', expected one of [ %s]",
+                       options->ifname,
+                       if_list);
+            g_free(if_list);
+        } else {
+            error_setg(errp,
+                       "unsupported ifname '%s', no supported "
+                       "interfaces available",
+                       options->ifname);
+        }
+        return false;
+    }
+
+#if !defined(MAC_OS_VERSION_11_0) || \
+    MAC_OS_X_VERSION_MIN_REQUIRED < MAC_OS_VERSION_11_0
+    if (options->has_isolated) {
+        error_setg(errp,
+                   "vmnet-bridged.isolated feature is "
+                   "unavailable: outdated vmnet.framework API");
+        return false;
+    }
+#endif
+    return true;
+}
+
+
+static xpc_object_t build_if_desc(const Netdev *netdev)
+{
+    const NetdevVmnetBridgedOptions *options = &(netdev->u.vmnet_bridged);
+    xpc_object_t if_desc = xpc_dictionary_create(NULL, NULL, 0);
+
+    xpc_dictionary_set_uint64(if_desc,
+                              vmnet_operation_mode_key,
+                              VMNET_BRIDGED_MODE
+    );
+
+    xpc_dictionary_set_string(if_desc,
+                              vmnet_shared_interface_name_key,
+                              options->ifname);
+
+#if defined(MAC_OS_VERSION_11_0) && \
+    MAC_OS_X_VERSION_MIN_REQUIRED >= MAC_OS_VERSION_11_0
+    xpc_dictionary_set_bool(if_desc,
+                            vmnet_enable_isolation_key,
+                            options->isolated);
+#endif
+    return if_desc;
+}
+
+
+static NetClientInfo net_vmnet_bridged_info = {
+    .type = NET_CLIENT_DRIVER_VMNET_BRIDGED,
+    .size = sizeof(VmnetState),
+    .receive = vmnet_receive_common,
+    .cleanup = vmnet_cleanup_common,
+};
+
+
 int net_init_vmnet_bridged(const Netdev *netdev, const char *name,
                            NetClientState *peer, Error **errp)
 {
-  error_setg(errp, "vmnet-bridged is not implemented yet");
-  return -1;
+    NetClientState *nc = qemu_new_net_client(&net_vmnet_bridged_info,
+                                             peer, "vmnet-bridged", name);
+    xpc_object_t if_desc;
+    int result = -1;
+
+    if (!validate_options(netdev, errp)) {
+        return result;
+    }
+
+    if_desc = build_if_desc(netdev);
+    result = vmnet_if_create(nc, if_desc, errp);
+    xpc_release(if_desc);
+    return result;
 }

From b0290db12d7b6567916ee3cba7bf2f08183121e8 Mon Sep 17 00:00:00 2001
From: Vladislav Yaroshchuk <vladislav.yaroshchuk@jetbrains.com>
Date: Thu, 17 Mar 2022 20:28:38 +0300
Subject: [PATCH 421/935] net/vmnet: update qemu-options.hx

Update qemu-options.hx to support vmnet networking backend.

Reviewed-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Tested-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Signed-off-by: Vladislav Yaroshchuk <Vladislav.Yaroshchuk@jetbrains.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 qemu-options.hx | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/qemu-options.hx b/qemu-options.hx
index b484640067..a664baaa18 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -2795,6 +2795,25 @@ DEF("netdev", HAS_ARG, QEMU_OPTION_netdev,
 #ifdef __linux__
     "-netdev vhost-vdpa,id=str,vhostdev=/path/to/dev\n"
     "                configure a vhost-vdpa network,Establish a vhost-vdpa netdev\n"
+#endif
+#ifdef CONFIG_VMNET
+    "-netdev vmnet-host,id=str[,isolated=on|off][,net-uuid=uuid]\n"
+    "         [,start-address=addr,end-address=addr,subnet-mask=mask]\n"
+    "                configure a vmnet network backend in host mode with ID 'str',\n"
+    "                isolate this interface from others with 'isolated',\n"
+    "                configure the address range and choose a subnet mask,\n"
+    "                specify network UUID 'uuid' to disable DHCP and interact with\n"
+    "                vmnet-host interfaces within this isolated network\n"
+    "-netdev vmnet-shared,id=str[,isolated=on|off][,nat66-prefix=addr]\n"
+    "         [,start-address=addr,end-address=addr,subnet-mask=mask]\n"
+    "                configure a vmnet network backend in shared mode with ID 'str',\n"
+    "                configure the address range and choose a subnet mask,\n"
+    "                set IPv6 ULA prefix (of length 64) to use for internal network,\n"
+    "                isolate this interface from others with 'isolated'\n"
+    "-netdev vmnet-bridged,id=str,ifname=name[,isolated=on|off]\n"
+    "                configure a vmnet network backend in bridged mode with ID 'str',\n"
+    "                use 'ifname=name' to select a physical network interface to be bridged,\n"
+    "                isolate this interface from others with 'isolated'\n"
 #endif
     "-netdev hubport,id=str,hubid=n[,netdev=nd]\n"
     "                configure a hub port on the hub with ID 'n'\n", QEMU_ARCH_ALL)
@@ -2814,6 +2833,9 @@ DEF("nic", HAS_ARG, QEMU_OPTION_nic,
 #endif
 #ifdef CONFIG_POSIX
     "vhost-user|"
+#endif
+#ifdef CONFIG_VMNET
+    "vmnet-host|vmnet-shared|vmnet-bridged|"
 #endif
     "socket][,option][,...][mac=macaddr]\n"
     "                initialize an on-board / default host NIC (using MAC address\n"
@@ -2836,6 +2858,9 @@ DEF("net", HAS_ARG, QEMU_OPTION_net,
 #endif
 #ifdef CONFIG_NETMAP
     "netmap|"
+#endif
+#ifdef CONFIG_VMNET
+    "vmnet-host|vmnet-shared|vmnet-bridged|"
 #endif
     "socket][,option][,option][,...]\n"
     "                old way to initialize a host network interface\n"

From fd8c8c056d17e78efdcfe680b4830cd646d5d5a0 Mon Sep 17 00:00:00 2001
From: Vladislav Yaroshchuk <vladislav.yaroshchuk@jetbrains.com>
Date: Thu, 17 Mar 2022 20:28:39 +0300
Subject: [PATCH 422/935] net/vmnet: update hmp-commands.hx

Update HMP for supporting vmnet.

Reviewed-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Tested-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Signed-off-by: Vladislav Yaroshchuk <Vladislav.Yaroshchuk@jetbrains.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 hmp-commands.hx | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/hmp-commands.hx b/hmp-commands.hx
index 03e6a73d1f..564f1de364 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1269,7 +1269,11 @@ ERST
     {
         .name       = "netdev_add",
         .args_type  = "netdev:O",
-        .params     = "[user|tap|socket|vde|bridge|hubport|netmap|vhost-user],id=str[,prop=value][,...]",
+        .params     = "[user|tap|socket|vde|bridge|hubport|netmap|vhost-user"
+#ifdef CONFIG_VMNET
+                      "|vmnet-host|vmnet-shared|vmnet-bridged"
+#endif
+                      "],id=str[,prop=value][,...]",
         .help       = "add host network device",
         .cmd        = hmp_netdev_add,
         .command_completion = netdev_add_completion,

From 052c2579b89b0d87debe8b05594b5180f0fde87d Mon Sep 17 00:00:00 2001
From: Helge Deller <deller@gmx.de>
Date: Thu, 10 Mar 2022 17:55:50 +0100
Subject: [PATCH 423/935] tulip: Assign default MAC address if not specified
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The MAC of the tulip card is stored in the EEPROM and at startup
tulip_fill_eeprom() is called to initialize the EEPROM with the MAC
address given on the command line, e.g.:
    -device tulip,mac=00:11:22:33:44:55

In case the mac address was not given on the command line,
tulip_fill_eeprom() initializes the MAC in EEPROM with 00:00:00:00:00:00
which breaks e.g. a HP-UX guest.

Fix this problem by moving qemu_macaddr_default_if_unset() a few lines
up, so that a default mac address is assigned before tulip_fill_eeprom()
initializes the EEPROM.

Signed-off-by: Helge Deller <deller@gmx.de>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 hw/net/tulip.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/net/tulip.c b/hw/net/tulip.c
index d5b6cc5ee6..097e905bec 100644
--- a/hw/net/tulip.c
+++ b/hw/net/tulip.c
@@ -967,6 +967,8 @@ static void pci_tulip_realize(PCIDevice *pci_dev, Error **errp)
     pci_conf = s->dev.config;
     pci_conf[PCI_INTERRUPT_PIN] = 1; /* interrupt pin A */
 
+    qemu_macaddr_default_if_unset(&s->c.macaddr);
+
     s->eeprom = eeprom93xx_new(&pci_dev->qdev, 64);
     tulip_fill_eeprom(s);
 
@@ -981,8 +983,6 @@ static void pci_tulip_realize(PCIDevice *pci_dev, Error **errp)
 
     s->irq = pci_allocate_irq(&s->dev);
 
-    qemu_macaddr_default_if_unset(&s->c.macaddr);
-
     s->nic = qemu_new_nic(&net_tulip_info, &s->c,
                           object_get_typename(OBJECT(pci_dev)),
                           pci_dev->qdev.id, s);

From e4e68c31db5472111fbff528d7348a7e6b2ae07d Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Mon, 16 May 2022 10:23:10 +0200
Subject: [PATCH 424/935] gitlab-ci: Switch the container of the 'check-patch'
 & 'check-dco' jobs
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The 'check-patch' and 'check-dco' jobs only need Python and git for
checking the patches, so it's not really necessary to use a container
here that has all the other build dependencies installed. By using a
lightweight Alpine container, we can improve the runtime here quite a
bit, cutting it down from ca. 1:30 minutes to ca. 45 seconds.

Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220516082310.33876-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/static_checks.yml | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/.gitlab-ci.d/static_checks.yml b/.gitlab-ci.d/static_checks.yml
index 5e955540d3..94858e3272 100644
--- a/.gitlab-ci.d/static_checks.yml
+++ b/.gitlab-ci.d/static_checks.yml
@@ -1,12 +1,13 @@
 check-patch:
   stage: build
-  image: $CI_REGISTRY_IMAGE/qemu/centos8:latest
-  needs:
-    job: amd64-centos8-container
+  image: python:3.10-alpine
+  needs: []
   script:
     - .gitlab-ci.d/check-patch.py
   variables:
     GIT_DEPTH: 1000
+  before_script:
+    - apk -U add git perl
   rules:
     - if: '$CI_PROJECT_NAMESPACE == "qemu-project"'
       when: never
@@ -15,12 +16,13 @@ check-patch:
 
 check-dco:
   stage: build
-  image: $CI_REGISTRY_IMAGE/qemu/centos8:latest
-  needs:
-    job: amd64-centos8-container
+  image: python:3.10-alpine
+  needs: []
   script: .gitlab-ci.d/check-dco.py
   variables:
     GIT_DEPTH: 1000
+  before_script:
+    - apk -U add git
   rules:
     - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
       when: never

From f3f0cbaf97c78b1fc1f3855f576e46ea6ad4f34c Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Mon, 16 May 2022 13:59:12 +0200
Subject: [PATCH 425/935] Remove Ubuntu 18.04 container support from the
 repository
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

According to our "Supported build platforms" policy, we now do not support
Ubuntu 18.04 anymore. Remove the related container files and entries from
our CI.

Message-Id: <20220516115912.120951-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/containers.yml                   |   5 -
 .../custom-runners/ubuntu-20.04-aarch32.yml   |   2 +-
 .../custom-runners/ubuntu-20.04-aarch64.yml   |   2 +-
 scripts/ci/setup/build-environment.yml        |  14 +-
 tests/docker/dockerfiles/ubuntu1804.docker    | 144 ------------------
 tests/lcitool/refresh                         |   7 -
 6 files changed, 4 insertions(+), 170 deletions(-)
 delete mode 100644 tests/docker/dockerfiles/ubuntu1804.docker

diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
index b9b675fdcb..e9df90bbdd 100644
--- a/.gitlab-ci.d/containers.yml
+++ b/.gitlab-ci.d/containers.yml
@@ -19,11 +19,6 @@ amd64-debian-container:
   variables:
     NAME: debian-amd64
 
-amd64-ubuntu1804-container:
-  extends: .container_job_template
-  variables:
-    NAME: ubuntu1804
-
 amd64-ubuntu2004-container:
   extends: .container_job_template
   variables:
diff --git a/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch32.yml b/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch32.yml
index 9c589bc4cf..47856ac53c 100644
--- a/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch32.yml
+++ b/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch32.yml
@@ -1,6 +1,6 @@
 # All ubuntu-20.04 jobs should run successfully in an environment
 # setup by the scripts/ci/setup/qemu/build-environment.yml task
-# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
+# "Install basic packages to build QEMU on Ubuntu 20.04"
 
 ubuntu-20.04-aarch32-all:
  needs: []
diff --git a/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml b/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml
index 920e388bd0..951e490db1 100644
--- a/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml
+++ b/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml
@@ -1,6 +1,6 @@
 # All ubuntu-20.04 jobs should run successfully in an environment
 # setup by the scripts/ci/setup/qemu/build-environment.yml task
-# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
+# "Install basic packages to build QEMU on Ubuntu 20.04"
 
 ubuntu-20.04-aarch64-all-linux-static:
  needs: []
diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/build-environment.yml
index 9182e0c253..232525b91d 100644
--- a/scripts/ci/setup/build-environment.yml
+++ b/scripts/ci/setup/build-environment.yml
@@ -33,10 +33,9 @@
       when:
         - ansible_facts['distribution'] == 'Ubuntu'
 
-    - name: Install basic packages to build QEMU on Ubuntu 18.04/20.04
+    - name: Install basic packages to build QEMU on Ubuntu 20.04
       package:
         name:
-        # Originally from tests/docker/dockerfiles/ubuntu1804.docker
           - ccache
           - gcc
           - gettext
@@ -90,7 +89,7 @@
       when:
         - ansible_facts['distribution'] == 'Ubuntu'
 
-    - name: Install packages to build QEMU on Ubuntu 18.04/20.04 on non-s390x
+    - name: Install packages to build QEMU on Ubuntu 20.04 on non-s390x
       package:
         name:
           - libspice-server-dev
@@ -100,15 +99,6 @@
         - ansible_facts['distribution'] == 'Ubuntu'
         - ansible_facts['architecture'] != 's390x'
 
-    - name: Install basic packages to build QEMU on Ubuntu 18.04
-      package:
-        name:
-        # Originally from tests/docker/dockerfiles/ubuntu1804.docker
-          - clang
-      when:
-        - ansible_facts['distribution'] == 'Ubuntu'
-        - ansible_facts['distribution_version'] == '18.04'
-
     - name: Install basic packages to build QEMU on Ubuntu 20.04
       package:
         name:
diff --git a/tests/docker/dockerfiles/ubuntu1804.docker b/tests/docker/dockerfiles/ubuntu1804.docker
deleted file mode 100644
index b3f2156580..0000000000
--- a/tests/docker/dockerfiles/ubuntu1804.docker
+++ /dev/null
@@ -1,144 +0,0 @@
-# THIS FILE WAS AUTO-GENERATED
-#
-#  $ lcitool dockerfile --layers all ubuntu-1804 qemu
-#
-# https://gitlab.com/libvirt/libvirt-ci
-
-FROM docker.io/library/ubuntu:18.04
-
-RUN export DEBIAN_FRONTEND=noninteractive && \
-    apt-get update && \
-    apt-get install -y eatmydata && \
-    eatmydata apt-get dist-upgrade -y && \
-    eatmydata apt-get install --no-install-recommends -y \
-            bash \
-            bc \
-            bsdmainutils \
-            bzip2 \
-            ca-certificates \
-            ccache \
-            clang \
-            dbus \
-            debianutils \
-            diffutils \
-            exuberant-ctags \
-            findutils \
-            g++ \
-            gcc \
-            gcovr \
-            genisoimage \
-            gettext \
-            git \
-            glusterfs-common \
-            hostname \
-            libaio-dev \
-            libasan5 \
-            libasound2-dev \
-            libattr1-dev \
-            libbrlapi-dev \
-            libbz2-dev \
-            libc6-dev \
-            libcacard-dev \
-            libcap-ng-dev \
-            libcapstone-dev \
-            libcurl4-gnutls-dev \
-            libdaxctl-dev \
-            libdrm-dev \
-            libepoxy-dev \
-            libfdt-dev \
-            libffi-dev \
-            libgbm-dev \
-            libgcrypt20-dev \
-            libglib2.0-dev \
-            libgnutls28-dev \
-            libgtk-3-dev \
-            libibumad-dev \
-            libibverbs-dev \
-            libiscsi-dev \
-            libjemalloc-dev \
-            libjpeg-turbo8-dev \
-            liblttng-ust-dev \
-            liblzo2-dev \
-            libncursesw5-dev \
-            libnfs-dev \
-            libnuma-dev \
-            libpam0g-dev \
-            libpcre2-dev \
-            libpixman-1-dev \
-            libpmem-dev \
-            libpng-dev \
-            libpulse-dev \
-            librbd-dev \
-            librdmacm-dev \
-            libsasl2-dev \
-            libsdl2-dev \
-            libsdl2-image-dev \
-            libseccomp-dev \
-            libselinux1-dev \
-            libsnappy-dev \
-            libspice-protocol-dev \
-            libspice-server-dev \
-            libssh-dev \
-            libsystemd-dev \
-            libtasn1-6-dev \
-            libubsan1 \
-            libudev-dev \
-            libusb-1.0-0-dev \
-            libusbredirhost-dev \
-            libvdeplug-dev \
-            libvirglrenderer-dev \
-            libvte-2.91-dev \
-            libxen-dev \
-            libzstd-dev \
-            llvm \
-            locales \
-            make \
-            multipath-tools \
-            netcat-openbsd \
-            nettle-dev \
-            ninja-build \
-            openssh-client \
-            perl-base \
-            pkgconf \
-            python3 \
-            python3-numpy \
-            python3-opencv \
-            python3-pillow \
-            python3-pip \
-            python3-setuptools \
-            python3-sphinx \
-            python3-sphinx-rtd-theme \
-            python3-venv \
-            python3-wheel \
-            python3-yaml \
-            rpm2cpio \
-            sed \
-            sparse \
-            systemtap-sdt-dev \
-            tar \
-            tesseract-ocr \
-            tesseract-ocr-eng \
-            texinfo \
-            xfslibs-dev \
-            zlib1g-dev && \
-    eatmydata apt-get autoremove -y && \
-    eatmydata apt-get autoclean -y && \
-    sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
-    dpkg-reconfigure locales && \
-    dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \
-    mkdir -p /usr/libexec/ccache-wrappers && \
-    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/c++ && \
-    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
-    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/clang && \
-    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/g++ && \
-    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc
-
-RUN pip3 install meson==0.56.0
-
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-# https://bugs.launchpad.net/qemu/+bug/1838763
-ENV QEMU_CONFIGURE_OPTS --disable-libssh
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 2d198ad281..fb49bbc441 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -65,11 +65,6 @@ def generate_cirrus(target, trailer=None):
    cmd = lcitool_cmd + ["variables", target, "qemu"]
    generate(filename, cmd, trailer)
 
-ubuntu1804_skipssh = [
-   "# https://bugs.launchpad.net/qemu/+bug/1838763\n",
-   "ENV QEMU_CONFIGURE_OPTS --disable-libssh\n"
-]
-
 ubuntu2004_tsanhack = [
    "# Apply patch https://reviews.llvm.org/D75820\n",
    "# This is required for TSan in clang-10 to compile with QEMU.\n",
@@ -85,8 +80,6 @@ def debian_cross_build(prefix, targets):
 try:
    generate_dockerfile("centos8", "centos-stream-8")
    generate_dockerfile("fedora", "fedora-35")
-   generate_dockerfile("ubuntu1804", "ubuntu-1804",
-                       trailer="".join(ubuntu1804_skipssh))
    generate_dockerfile("ubuntu2004", "ubuntu-2004",
                        trailer="".join(ubuntu2004_tsanhack))
    generate_dockerfile("opensuse-leap", "opensuse-leap-152")

From 83f79d4efc8dac7e511bc4375d0f9d90ff9db731 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Fri, 13 May 2022 16:49:05 +0100
Subject: [PATCH 426/935] tests/qtest: fix registration of ABRT handler for
 QEMU cleanup
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

qtest_init registers a hook to cleanup the running QEMU process
should g_assert() fire before qtest_quit is called. When the first
hook is registered, it is supposed to triggere registration of the
SIGABRT handler. Unfortunately the logic in hook_list_is_empty is
inverted, so the SIGABRT handler never gets registered, unless
2 or more QEMU processes are run concurrently. This caused qtest
to leak QEMU processes anytime g_assert triggers.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220513154906.206715-2-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qtest/libqtest.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c
index 228357f1ea..4a4697c0d1 100644
--- a/tests/qtest/libqtest.c
+++ b/tests/qtest/libqtest.c
@@ -197,11 +197,11 @@ static bool hook_list_is_empty(GHookList *hook_list)
     GHook *hook = g_hook_first_valid(hook_list, TRUE);
 
     if (!hook) {
-        return false;
+        return true;
     }
 
     g_hook_unref(hook_list, hook);
-    return true;
+    return false;
 }
 
 void qtest_add_abrt_handler(GHookFunc fn, const void *data)

From 6cbde91a27587ca27a3c1979fe7f5f0d28eb6db9 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Fri, 13 May 2022 16:49:06 +0100
Subject: [PATCH 427/935] tests/qtest: use prctl(PR_SET_PDEATHSIG) as fallback
 to kill QEMU
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Although we register a ABRT handler to kill off QEMU when g_assert()
triggers, we want an extra safety net. The QEMU process might be
non-functional and thus not have responded to SIGTERM. The test script
might also have crashed with SEGV, in which case the cleanup handlers
won't ever run.

Using the Linux specific prctl(PR_SET_PDEATHSIG) syscall, we
can ensure that QEMU gets sent SIGKILL as soon as the controlling
qtest exits, if nothing else has correctly told it to quit.

Note, technically the death signal is sent when the *thread* that
called fork() exits. IOW, if you are calling qtest_init() in one
thread, letting that thread exit, and then expecting to run
qtest_quit() in a different thread, things are not going to work
out. Fortunately that is not a scenario that exists in qtests,
as pairs of qtest_init and qtest_quit are always called from the
same thread.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220513154906.206715-3-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qtest/libqtest.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c
index 4a4697c0d1..2e49618454 100644
--- a/tests/qtest/libqtest.c
+++ b/tests/qtest/libqtest.c
@@ -19,6 +19,9 @@
 #include <sys/socket.h>
 #include <sys/wait.h>
 #include <sys/un.h>
+#ifdef __linux__
+#include <sys/prctl.h>
+#endif /* __linux__ */
 
 #include "libqtest.h"
 #include "libqmp.h"
@@ -301,6 +304,20 @@ QTestState *qtest_init_without_qmp_handshake(const char *extra_args)
     s->expected_status = 0;
     s->qemu_pid = fork();
     if (s->qemu_pid == 0) {
+#ifdef __linux__
+        /*
+         * Although we register a ABRT handler to kill off QEMU
+         * when g_assert() triggers, we want an extra safety
+         * net. The QEMU process might be non-functional and
+         * thus not have responded to SIGTERM. The test script
+         * might also have crashed with SEGV, in which case the
+         * cleanup handlers won't ever run.
+         *
+         * This PR_SET_PDEATHSIG setup will ensure any remaining
+         * QEMU will get terminated with SIGKILL in these cases.
+         */
+        prctl(PR_SET_PDEATHSIG, SIGKILL, 0, 0, 0);
+#endif /* __linux__ */
         if (!g_setenv("QEMU_AUDIO_DRV", "none", true)) {
             exit(1);
         }

From 0ce9b08c10d043307d125709032a897d05c80bdd Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Fri, 13 May 2022 08:39:58 +0200
Subject: [PATCH 428/935] docs/about: Update the support statement for Windows
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Our support statement for Windows currently talks about "Vista / Server
2008" - which is related to the API of Windows, and this is not easy
to understand for the non-technical users. Additionally, glib sets the
_WIN32_WINNT macro to 0x0601 already, which indicates the Windows 7 API,
so QEMU effectively depends on the Windows 7 API, too.

Thus let's bump the _WIN32_WINNT setting in QEMU to the same level as
glib uses and adjust our support statement in the documentation to
something similar that we're using for Linux and the *BSD systems
(i.e. only the two most recent versions), which should hopefully be
easier to understand for the users now.

And since we're nowadays also compile-testing QEMU with MSYS2 on Windows
itself, I think we could mention this build environment here, too.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/880
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Stefan Weil <sw@weilnetz.de>
Message-Id: <20220513063958.1181443-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 configure                      |  2 ++
 docs/about/build-platforms.rst | 12 ++++++++----
 include/qemu/osdep.h           |  2 +-
 3 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/configure b/configure
index 0cc8c876f7..103a7b02e8 100755
--- a/configure
+++ b/configure
@@ -1486,6 +1486,8 @@ fi
 ##########################################
 # glib support probe
 
+# When bumping glib_req_ver, please check also whether we should increase
+# the _WIN32_WINNT setting in osdep.h according to the value from glib
 glib_req_ver=2.56
 glib_modules=gthread-2.0
 if test "$modules" = yes; then
diff --git a/docs/about/build-platforms.rst b/docs/about/build-platforms.rst
index e9163ba556..1958edb430 100644
--- a/docs/about/build-platforms.rst
+++ b/docs/about/build-platforms.rst
@@ -86,11 +86,15 @@ similar versions.
 Windows
 -------
 
-The project supports building with current versions of the MinGW toolchain,
-hosted on Linux (Debian/Fedora).
+The project aims to support the two most recent versions of Windows that are
+still supported by the vendor. The minimum Windows API that is currently
+targeted is "Windows 7", so theoretically the QEMU binaries can still be run
+on older versions of Windows, too. However, such old versions of Windows are
+not tested anymore, so it is recommended to use one of the latest versions of
+Windows instead.
 
-The version of the Windows API that's currently targeted is Vista / Server
-2008.
+The project supports building QEMU with current versions of the MinGW
+toolchain, either hosted on Linux (Debian/Fedora) or via MSYS2 on Windows.
 
 .. _Homebrew: https://brew.sh/
 .. _MacPorts: https://www.macports.org/
diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index 1c1e7eca98..a72e99db85 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -75,7 +75,7 @@ QEMU_EXTERN_C int daemon(int, int);
 #ifdef _WIN32
 /* as defined in sdkddkver.h */
 #ifndef _WIN32_WINNT
-#define _WIN32_WINNT 0x0600 /* Vista */
+#define _WIN32_WINNT 0x0601 /* Windows 7 API (should be in sync with glib) */
 #endif
 /* reduces the number of implicitly included headers */
 #ifndef WIN32_LEAN_AND_MEAN

From 28e7e95e6bc374262a3f03cf7b6fb7a1b3c3ebbe Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Mon, 16 May 2022 16:58:21 +0200
Subject: [PATCH 429/935] tests/vm: Add capstone to the NetBSD and OpenBSD VMs

The Capstone library that is shipped with NetBSD and OpenBSD works
fine when compiling QEMU, so let's enable this in our build-test
VMs to get a little bit more build-test coverage.

Message-Id: <20220516145823.148450-2-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/vm/netbsd  | 3 ++-
 tests/vm/openbsd | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index 4cc58df130..45aa9a7fda 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -46,7 +46,8 @@ class NetBSDVM(basevm.BaseVM):
         "jpeg",
         "png",
 
-	# libs: ui
+        # libs: ui
+        "capstone",
         "SDL2",
         "gtk3+",
         "libxkbcommon",
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index dc34b2718b..13c8254214 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -48,7 +48,8 @@ class OpenBSDVM(basevm.BaseVM):
         "jpeg",
         "png",
 
-	# libs: ui
+        # libs: ui
+        "capstone",
         "sdl2",
         "gtk+3",
         "libxkbcommon",

From 43ecd16678b74b281c3cc7de0dd62a36279b6025 Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Mon, 16 May 2022 16:58:22 +0200
Subject: [PATCH 430/935] capstone: Allow version 3.0.5 again

According to

 https://lore.kernel.org/qemu-devel/20200921174118.39352-1-richard.henderson@linaro.org/

there was an issue with Capstone 3.0.4 from Ubuntu 18, which was the reason
for bumping our minimum Capstone requirement to version 4.0. And indeed,
compiling with that version 3.0.4 from Ubuntu 18.04 still fails (after
allowing it with a hack in meson.build). But now that we've dropped support
for Ubuntu 18.04, that issue is not relevant anymore. Compiling with Capstone
version 3.0.5 (e.g. used in Ubuntu 20.04) seems to work fine, so let's allow
that version again.

Message-Id: <20220516145823.148450-3-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/buildtest.yml | 1 +
 meson.build                | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 0aea7ab84c..a4d43d743b 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -42,6 +42,7 @@ build-system-ubuntu:
   variables:
     IMAGE: ubuntu2004
     CONFIGURE_ARGS: --enable-docs --enable-fdt=system --enable-slirp=system
+        --enable-capstone=system
     TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu
       microblazeel-softmmu mips64el-softmmu
     MAKE_CHECK_ARGS: check-build
diff --git a/meson.build b/meson.build
index 53a4728250..526ff29a86 100644
--- a/meson.build
+++ b/meson.build
@@ -2525,7 +2525,7 @@ capstone = not_found
 capstone_opt = get_option('capstone')
 if capstone_opt in ['enabled', 'auto', 'system']
   have_internal = fs.exists(meson.current_source_dir() / 'capstone/Makefile')
-  capstone = dependency('capstone', version: '>=4.0',
+  capstone = dependency('capstone', version: '>=3.0.5',
                         kwargs: static_kwargs, method: 'pkg-config',
                         required: capstone_opt == 'system' or
                                   capstone_opt == 'enabled' and not have_internal)

From 83602083b4ada6ceb86bfb327e83556ebab120fc Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Mon, 16 May 2022 16:58:23 +0200
Subject: [PATCH 431/935] capstone: Remove the capstone submodule

Now that we allow compiling with Capstone v3.0.5 again, all our supported
build hosts should provide at least this version of the disassembler
library, so we do not need to ship this as a submodule anymore.

Message-Id: <20220516145823.148450-4-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/buildtest.yml    |   4 +-
 .gitlab-ci.d/windows.yml      |   5 +-
 .gitmodules                   |   3 -
 capstone                      |   1 -
 configure                     |  21 -------
 meson.build                   | 113 ++--------------------------------
 meson_options.txt             |   3 +-
 scripts/meson-buildoptions.sh |   5 +-
 8 files changed, 11 insertions(+), 144 deletions(-)
 delete mode 160000 capstone

diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index a4d43d743b..e9620c3074 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -42,7 +42,7 @@ build-system-ubuntu:
   variables:
     IMAGE: ubuntu2004
     CONFIGURE_ARGS: --enable-docs --enable-fdt=system --enable-slirp=system
-        --enable-capstone=system
+        --enable-capstone
     TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu
       microblazeel-softmmu mips64el-softmmu
     MAKE_CHECK_ARGS: check-build
@@ -119,7 +119,7 @@ build-system-fedora:
   variables:
     IMAGE: fedora
     CONFIGURE_ARGS: --disable-gcrypt --enable-nettle --enable-docs
-             --enable-fdt=system --enable-slirp=system --enable-capstone=system
+             --enable-fdt=system --enable-slirp=system --enable-capstone
     TARGETS: tricore-softmmu microblaze-softmmu mips-softmmu
       xtensa-softmmu m68k-softmmu riscv32-softmmu ppc-softmmu sparc64-softmmu
     MAKE_CHECK_ARGS: check-build
diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
index 47f6073773..cf7724b8e5 100644
--- a/.gitlab-ci.d/windows.yml
+++ b/.gitlab-ci.d/windows.yml
@@ -57,7 +57,7 @@ msys2-64bit:
   - $env:CHERE_INVOKING = 'yes'  # Preserve the current working directory
   - $env:MSYSTEM = 'MINGW64'     # Start a 64 bit Mingw environment
   - .\msys64\usr\bin\bash -lc './configure --target-list=x86_64-softmmu
-      --enable-capstone=system --without-default-devices'
+      --enable-capstone --without-default-devices'
   - .\msys64\usr\bin\bash -lc "sed -i '/^ROMS=/d' build/config-host.mak"
   - .\msys64\usr\bin\bash -lc 'make -j2'
   - .\msys64\usr\bin\bash -lc 'make check'
@@ -90,7 +90,6 @@ msys2-32bit:
   - $env:MSYSTEM = 'MINGW32'     # Start a 32-bit MinG environment
   - mkdir output
   - cd output
-  - ..\msys64\usr\bin\bash -lc "../configure --target-list=ppc64-softmmu
-      --enable-capstone=system"
+  - ..\msys64\usr\bin\bash -lc "../configure --target-list=ppc64-softmmu"
   - ..\msys64\usr\bin\bash -lc 'make -j2'
   - ..\msys64\usr\bin\bash -lc 'make check'
diff --git a/.gitmodules b/.gitmodules
index f4b6a9b401..b8bff47df8 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -31,9 +31,6 @@
 [submodule "ui/keycodemapdb"]
 	path = ui/keycodemapdb
 	url = https://gitlab.com/qemu-project/keycodemapdb.git
-[submodule "capstone"]
-	path = capstone
-	url = https://gitlab.com/qemu-project/capstone.git
 [submodule "roms/seabios-hppa"]
 	path = roms/seabios-hppa
 	url = https://gitlab.com/qemu-project/seabios-hppa.git
diff --git a/capstone b/capstone
deleted file mode 160000
index f8b1b83301..0000000000
--- a/capstone
+++ /dev/null
@@ -1 +0,0 @@
-Subproject commit f8b1b833015a4ae47110ed068e0deb7106ced66d
diff --git a/configure b/configure
index 103a7b02e8..180ee688dc 100755
--- a/configure
+++ b/configure
@@ -307,10 +307,8 @@ skip_meson=no
 
 # 1. Track which submodules are needed
 if test "$default_feature" = no ; then
-  capstone="disabled"
   slirp="disabled"
 else
-  capstone="auto"
   slirp="auto"
 fi
 fdt="auto"
@@ -869,14 +867,6 @@ for opt do
   --enable-uuid|--disable-uuid)
       echo "$0: $opt is obsolete, UUID support is always built" >&2
   ;;
-  --disable-capstone) capstone="disabled"
-  ;;
-  --enable-capstone) capstone="enabled"
-  ;;
-  --enable-capstone=git) capstone="internal"
-  ;;
-  --enable-capstone=*) capstone="$optarg"
-  ;;
   --with-git=*) git="$optarg"
   ;;
   --with-git-submodules=*)
@@ -1566,16 +1556,6 @@ case "$fdt" in
     ;;
 esac
 
-##########################################
-# capstone
-
-case "$capstone" in
-  auto | enabled | internal)
-    # Simpler to always update submodule, even if not needed.
-    git_submodules="${git_submodules} capstone"
-    ;;
-esac
-
 ##########################################
 # check and set a backend for coroutine
 
@@ -2220,7 +2200,6 @@ if test "$skip_meson" = no; then
   test "$werror" = yes && meson_option_add -Dwerror=true
 
   # QEMU options
-  test "$capstone" != auto && meson_option_add "-Dcapstone=$capstone"
   test "$cfi" != false && meson_option_add "-Dcfi=$cfi"
   test "$fdt" != auto && meson_option_add "-Dfdt=$fdt"
   test -n "${LIB_FUZZING_ENGINE+xxx}" && meson_option_add "-Dfuzzing_engine=$LIB_FUZZING_ENGINE"
diff --git a/meson.build b/meson.build
index 526ff29a86..c4fb8fa5d0 100644
--- a/meson.build
+++ b/meson.build
@@ -2522,13 +2522,10 @@ genh += custom_target('config-poison.h',
 ##############
 
 capstone = not_found
-capstone_opt = get_option('capstone')
-if capstone_opt in ['enabled', 'auto', 'system']
-  have_internal = fs.exists(meson.current_source_dir() / 'capstone/Makefile')
+if not get_option('capstone').auto() or have_system or have_user
   capstone = dependency('capstone', version: '>=3.0.5',
                         kwargs: static_kwargs, method: 'pkg-config',
-                        required: capstone_opt == 'system' or
-                                  capstone_opt == 'enabled' and not have_internal)
+                        required: get_option('capstone'))
 
   # Some versions of capstone have broken pkg-config file
   # that reports a wrong -I path, causing the #include to
@@ -2537,110 +2534,10 @@ if capstone_opt in ['enabled', 'auto', 'system']
   if capstone.found() and not cc.compiles('#include <capstone.h>',
                                           dependencies: [capstone])
     capstone = not_found
-    if capstone_opt == 'system'
-      error('system capstone requested, it does not appear to work')
+    if get_option('capstone').enabled()
+      error('capstone requested, but it does not appear to work')
     endif
   endif
-
-  if capstone.found()
-    capstone_opt = 'system'
-  elif have_internal
-    capstone_opt = 'internal'
-  else
-    capstone_opt = 'disabled'
-  endif
-endif
-if capstone_opt == 'internal'
-  capstone_data = configuration_data()
-  capstone_data.set('CAPSTONE_USE_SYS_DYN_MEM', '1')
-
-  capstone_files = files(
-    'capstone/cs.c',
-    'capstone/MCInst.c',
-    'capstone/MCInstrDesc.c',
-    'capstone/MCRegisterInfo.c',
-    'capstone/SStream.c',
-    'capstone/utils.c'
-  )
-
-  if 'CONFIG_ARM_DIS' in config_all_disas
-    capstone_data.set('CAPSTONE_HAS_ARM', '1')
-    capstone_files += files(
-      'capstone/arch/ARM/ARMDisassembler.c',
-      'capstone/arch/ARM/ARMInstPrinter.c',
-      'capstone/arch/ARM/ARMMapping.c',
-      'capstone/arch/ARM/ARMModule.c'
-    )
-  endif
-
-  # FIXME: This config entry currently depends on a c++ compiler.
-  # Which is needed for building libvixl, but not for capstone.
-  if 'CONFIG_ARM_A64_DIS' in config_all_disas
-    capstone_data.set('CAPSTONE_HAS_ARM64', '1')
-    capstone_files += files(
-      'capstone/arch/AArch64/AArch64BaseInfo.c',
-      'capstone/arch/AArch64/AArch64Disassembler.c',
-      'capstone/arch/AArch64/AArch64InstPrinter.c',
-      'capstone/arch/AArch64/AArch64Mapping.c',
-      'capstone/arch/AArch64/AArch64Module.c'
-    )
-  endif
-
-  if 'CONFIG_PPC_DIS' in config_all_disas
-    capstone_data.set('CAPSTONE_HAS_POWERPC', '1')
-    capstone_files += files(
-      'capstone/arch/PowerPC/PPCDisassembler.c',
-      'capstone/arch/PowerPC/PPCInstPrinter.c',
-      'capstone/arch/PowerPC/PPCMapping.c',
-      'capstone/arch/PowerPC/PPCModule.c'
-    )
-  endif
-
-  if 'CONFIG_S390_DIS' in config_all_disas
-    capstone_data.set('CAPSTONE_HAS_SYSZ', '1')
-    capstone_files += files(
-      'capstone/arch/SystemZ/SystemZDisassembler.c',
-      'capstone/arch/SystemZ/SystemZInstPrinter.c',
-      'capstone/arch/SystemZ/SystemZMapping.c',
-      'capstone/arch/SystemZ/SystemZModule.c',
-      'capstone/arch/SystemZ/SystemZMCTargetDesc.c'
-    )
-  endif
-
-  if 'CONFIG_I386_DIS' in config_all_disas
-    capstone_data.set('CAPSTONE_HAS_X86', 1)
-    capstone_files += files(
-      'capstone/arch/X86/X86Disassembler.c',
-      'capstone/arch/X86/X86DisassemblerDecoder.c',
-      'capstone/arch/X86/X86ATTInstPrinter.c',
-      'capstone/arch/X86/X86IntelInstPrinter.c',
-      'capstone/arch/X86/X86InstPrinterCommon.c',
-      'capstone/arch/X86/X86Mapping.c',
-      'capstone/arch/X86/X86Module.c'
-    )
-  endif
-
-  configure_file(output: 'capstone-defs.h', configuration: capstone_data)
-
-  capstone_cargs = [
-    # FIXME: There does not seem to be a way to completely replace the c_args
-    # that come from add_project_arguments() -- we can only add to them.
-    # So: disable all warnings with a big hammer.
-    '-Wno-error', '-w',
-
-    # Include all configuration defines via a header file, which will wind up
-    # as a dependency on the object file, and thus changes here will result
-    # in a rebuild.
-    '-include', 'capstone-defs.h'
-  ]
-
-  libcapstone = static_library('capstone',
-                               build_by_default: false,
-                               sources: capstone_files,
-                               c_args: capstone_cargs,
-                               include_directories: 'capstone/include')
-  capstone = declare_dependency(link_with: libcapstone,
-                                include_directories: 'capstone/include/capstone')
 endif
 
 slirp = not_found
@@ -3970,7 +3867,7 @@ summary_info += {'bzip2 support':     libbzip2}
 summary_info += {'lzfse support':     liblzfse}
 summary_info += {'zstd support':      zstd}
 summary_info += {'NUMA host support': numa}
-summary_info += {'capstone':          capstone_opt == 'internal' ? capstone_opt : capstone}
+summary_info += {'capstone':          capstone}
 summary_info += {'libpmem support':   libpmem}
 summary_info += {'libdaxctl support': libdaxctl}
 summary_info += {'libudev':           libudev}
diff --git a/meson_options.txt b/meson_options.txt
index 29c6b90cec..fcdc43f7db 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -252,8 +252,7 @@ option('virtfs', type: 'feature', value: 'auto',
 option('virtiofsd', type: 'feature', value: 'auto',
        description: 'build virtiofs daemon (virtiofsd)')
 
-option('capstone', type: 'combo', value: 'auto',
-       choices: ['disabled', 'enabled', 'auto', 'system', 'internal'],
+option('capstone', type: 'feature', value: 'auto',
        description: 'Whether and how to find the capstone library')
 option('slirp', type: 'combo', value: 'auto',
        choices: ['disabled', 'enabled', 'auto', 'system', 'internal'],
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index 5d2172bfb4..7fc56d3f4a 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -16,9 +16,6 @@ meson_options_help() {
   printf "%s\n" '  --enable-block-drv-whitelist-in-tools'
   printf "%s\n" '                           use block whitelist also in tools instead of only'
   printf "%s\n" '                           QEMU'
-  printf "%s\n" '  --enable-capstone[=CHOICE]'
-  printf "%s\n" '                           Whether and how to find the capstone library'
-  printf "%s\n" '                           (choices: auto/disabled/enabled/internal/system)'
   printf "%s\n" '  --enable-cfi             Control-Flow Integrity (CFI)'
   printf "%s\n" '  --enable-cfi-debug       Verbose errors in case of CFI violation'
   printf "%s\n" '  --enable-debug-mutex     mutex debugging support'
@@ -77,6 +74,7 @@ meson_options_help() {
   printf "%s\n" '  brlapi          brlapi character device driver'
   printf "%s\n" '  bzip2           bzip2 support for DMG images'
   printf "%s\n" '  cap-ng          cap_ng support'
+  printf "%s\n" '  capstone        Whether and how to find the capstone library'
   printf "%s\n" '  cloop           cloop image format support'
   printf "%s\n" '  cocoa           Cocoa user interface (macOS only)'
   printf "%s\n" '  coreaudio       CoreAudio sound support'
@@ -209,7 +207,6 @@ _meson_option_parse() {
     --disable-cap-ng) printf "%s" -Dcap_ng=disabled ;;
     --enable-capstone) printf "%s" -Dcapstone=enabled ;;
     --disable-capstone) printf "%s" -Dcapstone=disabled ;;
-    --enable-capstone=*) quote_sh "-Dcapstone=$2" ;;
     --enable-cfi) printf "%s" -Dcfi=true ;;
     --disable-cfi) printf "%s" -Dcfi=false ;;
     --enable-cfi-debug) printf "%s" -Dcfi_debug=true ;;

From 9f225e607f215003de1e4157255bb0199adff9aa Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Thu, 5 May 2022 19:39:47 +0100
Subject: [PATCH 432/935] target/arm: Postpone interpretation of stage 2
 descriptor attribute bits

In the original Arm v8 two-stage translation, both stage 1 and stage
2 specify memory attributes (memory type, cacheability,
shareability); these are then combined to produce the overall memory
attributes for the whole stage 1+2 access.  In QEMU we implement this
by having get_phys_addr() fill in an ARMCacheAttrs struct, and we
convert both the stage 1 and stage 2 attribute bit formats to the
same encoding (an 8-bit attribute value matching the MAIR_EL1 fields,
plus a 2-bit shareability value).

The new FEAT_S2FWB feature allows the guest to enable a different
interpretation of the attribute bits in the stage 2 descriptors.
These bits can now be used to control details of how the stage 1 and
2 attributes should be combined (for instance they can say "always
use the stage 1 attributes" or "ignore the stage 1 attributes and
always be Device memory").  This means we need to pass the raw bit
information for stage 2 down to the function which combines the stage
1 and stage 2 information.

Add a field to ARMCacheAttrs that indicates whether the attrs field
should be interpreted as MAIR format, or as the raw stage 2 attribute
bits from the descriptor, and store the appropriate values when
filling in cacheattrs.

We only need to interpret the attrs field in a few places:
 * in do_ats_write(), where we know to expect a MAIR value
   (there is no ATS instruction to do a stage-2-only walk)
 * in S1_ptw_translate(), where we want to know whether the
   combined S1 + S2 attributes indicate Device memory that
   should provoke a fault
 * in combine_cacheattrs(), which does the S1 + S2 combining
Update those places accordingly.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220505183950.2781801-2-peter.maydell@linaro.org
---
 target/arm/helper.c    | 42 ++++++++++++++++++++++++++++++++++++------
 target/arm/internals.h |  7 ++++++-
 2 files changed, 42 insertions(+), 7 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 432bd81919..93c58ad29a 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -3187,6 +3187,12 @@ static uint64_t do_ats_write(CPUARMState *env, uint64_t value,
     ret = get_phys_addr(env, value, access_type, mmu_idx, &phys_addr, &attrs,
                         &prot, &page_size, &fi, &cacheattrs);
 
+    /*
+     * ATS operations only do S1 or S1+S2 translations, so we never
+     * have to deal with the ARMCacheAttrs format for S2 only.
+     */
+    assert(!cacheattrs.is_s2_format);
+
     if (ret) {
         /*
          * Some kinds of translation fault must cause exceptions rather
@@ -10717,6 +10723,19 @@ static bool get_level1_table_address(CPUARMState *env, ARMMMUIdx mmu_idx,
     return true;
 }
 
+static bool ptw_attrs_are_device(CPUARMState *env, ARMCacheAttrs cacheattrs)
+{
+    /*
+     * For an S1 page table walk, the stage 1 attributes are always
+     * some form of "this is Normal memory". The combined S1+S2
+     * attributes are therefore only Device if stage 2 specifies Device.
+     * With HCR_EL2.FWB == 0 this is when descriptor bits [5:4] are 0b00,
+     * ie when cacheattrs.attrs bits [3:2] are 0b00.
+     */
+    assert(cacheattrs.is_s2_format);
+    return (cacheattrs.attrs & 0xc) == 0;
+}
+
 /* Translate a S1 pagetable walk through S2 if needed.  */
 static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
                                hwaddr addr, bool *is_secure,
@@ -10745,7 +10764,7 @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
             return ~0;
         }
         if ((arm_hcr_el2_eff(env) & HCR_PTW) &&
-            (cacheattrs.attrs & 0xf0) == 0) {
+            ptw_attrs_are_device(env, cacheattrs)) {
             /*
              * PTW set and S1 walk touched S2 Device memory:
              * generate Permission fault.
@@ -11817,12 +11836,14 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
     }
 
     if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
-        cacheattrs->attrs = convert_stage2_attrs(env, extract32(attrs, 0, 4));
+        cacheattrs->is_s2_format = true;
+        cacheattrs->attrs = extract32(attrs, 0, 4);
     } else {
         /* Index into MAIR registers for cache attributes */
         uint8_t attrindx = extract32(attrs, 0, 3);
         uint64_t mair = env->cp15.mair_el[regime_el(env, mmu_idx)];
         assert(attrindx <= 7);
+        cacheattrs->is_s2_format = false;
         cacheattrs->attrs = extract64(mair, attrindx * 8, 8);
     }
 
@@ -12560,14 +12581,22 @@ static uint8_t combine_cacheattr_nibble(uint8_t s1, uint8_t s2)
 /* Combine S1 and S2 cacheability/shareability attributes, per D4.5.4
  * and CombineS1S2Desc()
  *
+ * @env:     CPUARMState
  * @s1:      Attributes from stage 1 walk
  * @s2:      Attributes from stage 2 walk
  */
-static ARMCacheAttrs combine_cacheattrs(ARMCacheAttrs s1, ARMCacheAttrs s2)
+static ARMCacheAttrs combine_cacheattrs(CPUARMState *env,
+                                        ARMCacheAttrs s1, ARMCacheAttrs s2)
 {
     uint8_t s1lo, s2lo, s1hi, s2hi;
     ARMCacheAttrs ret;
     bool tagged = false;
+    uint8_t s2_mair_attrs;
+
+    assert(s2.is_s2_format && !s1.is_s2_format);
+    ret.is_s2_format = false;
+
+    s2_mair_attrs = convert_stage2_attrs(env, s2.attrs);
 
     if (s1.attrs == 0xf0) {
         tagged = true;
@@ -12575,9 +12604,9 @@ static ARMCacheAttrs combine_cacheattrs(ARMCacheAttrs s1, ARMCacheAttrs s2)
     }
 
     s1lo = extract32(s1.attrs, 0, 4);
-    s2lo = extract32(s2.attrs, 0, 4);
+    s2lo = extract32(s2_mair_attrs, 0, 4);
     s1hi = extract32(s1.attrs, 4, 4);
-    s2hi = extract32(s2.attrs, 4, 4);
+    s2hi = extract32(s2_mair_attrs, 4, 4);
 
     /* Combine shareability attributes (table D4-43) */
     if (s1.shareability == 2 || s2.shareability == 2) {
@@ -12731,7 +12760,7 @@ bool get_phys_addr(CPUARMState *env, target_ulong address,
                 }
                 cacheattrs->shareability = 0;
             }
-            *cacheattrs = combine_cacheattrs(*cacheattrs, cacheattrs2);
+            *cacheattrs = combine_cacheattrs(env, *cacheattrs, cacheattrs2);
 
             /* Check if IPA translates to secure or non-secure PA space. */
             if (arm_is_secure_below_el3(env)) {
@@ -12849,6 +12878,7 @@ bool get_phys_addr(CPUARMState *env, target_ulong address,
         /* Fill in cacheattr a-la AArch64.TranslateAddressS1Off. */
         hcr = arm_hcr_el2_eff(env);
         cacheattrs->shareability = 0;
+        cacheattrs->is_s2_format = false;
         if (hcr & HCR_DC) {
             if (hcr & HCR_DCT) {
                 memattr = 0xf0;  /* Tagged, Normal, WB, RWA */
diff --git a/target/arm/internals.h b/target/arm/internals.h
index 6ca0e95746..9b354eea7e 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -1149,8 +1149,13 @@ bool pmsav8_mpu_lookup(CPUARMState *env, uint32_t address,
 
 /* Cacheability and shareability attributes for a memory access */
 typedef struct ARMCacheAttrs {
-    unsigned int attrs:8; /* as in the MAIR register encoding */
+    /*
+     * If is_s2_format is true, attrs is the S2 descriptor bits [5:2]
+     * Otherwise, attrs is the same as the MAIR_EL1 8-bit format
+     */
+    unsigned int attrs:8;
     unsigned int shareability:2; /* as in the SH field of the VMSAv8-64 PTEs */
+    bool is_s2_format:1;
 } ARMCacheAttrs;
 
 bool get_phys_addr(CPUARMState *env, target_ulong address,

From 4a0b47c8150d3ac0a90f470191d64a3b199e6269 Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Thu, 5 May 2022 19:39:48 +0100
Subject: [PATCH 433/935] target/arm: Factor out FWB=0 specific part of
 combine_cacheattrs()

Factor out the part of combine_cacheattrs() that is specific to
handling HCR_EL2.FWB == 0.  This is the part where we combine the
memory type and cacheability attributes.

The "force Outer Shareable for Device or Normal Inner-NC Outer-NC"
logic remains in combine_cacheattrs() because it holds regardless
(this is the equivalent of the pseudocode EffectiveShareability()
function).

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220505183950.2781801-3-peter.maydell@linaro.org
---
 target/arm/helper.c | 88 +++++++++++++++++++++++++--------------------
 1 file changed, 50 insertions(+), 38 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 93c58ad29a..a2a9635841 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -12578,6 +12578,46 @@ static uint8_t combine_cacheattr_nibble(uint8_t s1, uint8_t s2)
     }
 }
 
+/*
+ * Combine the memory type and cacheability attributes of
+ * s1 and s2 for the HCR_EL2.FWB == 0 case, returning the
+ * combined attributes in MAIR_EL1 format.
+ */
+static uint8_t combined_attrs_nofwb(CPUARMState *env,
+                                    ARMCacheAttrs s1, ARMCacheAttrs s2)
+{
+    uint8_t s1lo, s2lo, s1hi, s2hi, s2_mair_attrs, ret_attrs;
+
+    s2_mair_attrs = convert_stage2_attrs(env, s2.attrs);
+
+    s1lo = extract32(s1.attrs, 0, 4);
+    s2lo = extract32(s2_mair_attrs, 0, 4);
+    s1hi = extract32(s1.attrs, 4, 4);
+    s2hi = extract32(s2_mair_attrs, 4, 4);
+
+    /* Combine memory type and cacheability attributes */
+    if (s1hi == 0 || s2hi == 0) {
+        /* Device has precedence over normal */
+        if (s1lo == 0 || s2lo == 0) {
+            /* nGnRnE has precedence over anything */
+            ret_attrs = 0;
+        } else if (s1lo == 4 || s2lo == 4) {
+            /* non-Reordering has precedence over Reordering */
+            ret_attrs = 4;  /* nGnRE */
+        } else if (s1lo == 8 || s2lo == 8) {
+            /* non-Gathering has precedence over Gathering */
+            ret_attrs = 8;  /* nGRE */
+        } else {
+            ret_attrs = 0xc; /* GRE */
+        }
+    } else { /* Normal memory */
+        /* Outer/inner cacheability combine independently */
+        ret_attrs = combine_cacheattr_nibble(s1hi, s2hi) << 4
+                  | combine_cacheattr_nibble(s1lo, s2lo);
+    }
+    return ret_attrs;
+}
+
 /* Combine S1 and S2 cacheability/shareability attributes, per D4.5.4
  * and CombineS1S2Desc()
  *
@@ -12588,26 +12628,17 @@ static uint8_t combine_cacheattr_nibble(uint8_t s1, uint8_t s2)
 static ARMCacheAttrs combine_cacheattrs(CPUARMState *env,
                                         ARMCacheAttrs s1, ARMCacheAttrs s2)
 {
-    uint8_t s1lo, s2lo, s1hi, s2hi;
     ARMCacheAttrs ret;
     bool tagged = false;
-    uint8_t s2_mair_attrs;
 
     assert(s2.is_s2_format && !s1.is_s2_format);
     ret.is_s2_format = false;
 
-    s2_mair_attrs = convert_stage2_attrs(env, s2.attrs);
-
     if (s1.attrs == 0xf0) {
         tagged = true;
         s1.attrs = 0xff;
     }
 
-    s1lo = extract32(s1.attrs, 0, 4);
-    s2lo = extract32(s2_mair_attrs, 0, 4);
-    s1hi = extract32(s1.attrs, 4, 4);
-    s2hi = extract32(s2_mair_attrs, 4, 4);
-
     /* Combine shareability attributes (table D4-43) */
     if (s1.shareability == 2 || s2.shareability == 2) {
         /* if either are outer-shareable, the result is outer-shareable */
@@ -12621,37 +12652,18 @@ static ARMCacheAttrs combine_cacheattrs(CPUARMState *env,
     }
 
     /* Combine memory type and cacheability attributes */
-    if (s1hi == 0 || s2hi == 0) {
-        /* Device has precedence over normal */
-        if (s1lo == 0 || s2lo == 0) {
-            /* nGnRnE has precedence over anything */
-            ret.attrs = 0;
-        } else if (s1lo == 4 || s2lo == 4) {
-            /* non-Reordering has precedence over Reordering */
-            ret.attrs = 4;  /* nGnRE */
-        } else if (s1lo == 8 || s2lo == 8) {
-            /* non-Gathering has precedence over Gathering */
-            ret.attrs = 8;  /* nGRE */
-        } else {
-            ret.attrs = 0xc; /* GRE */
-        }
+    ret.attrs = combined_attrs_nofwb(env, s1, s2);
 
-        /* Any location for which the resultant memory type is any
-         * type of Device memory is always treated as Outer Shareable.
-         */
+    /*
+     * Any location for which the resultant memory type is any
+     * type of Device memory is always treated as Outer Shareable.
+     * Any location for which the resultant memory type is Normal
+     * Inner Non-cacheable, Outer Non-cacheable is always treated
+     * as Outer Shareable.
+     * TODO: FEAT_XS adds another value (0x40) also meaning iNCoNC
+     */
+    if ((ret.attrs & 0xf0) == 0 || ret.attrs == 0x44) {
         ret.shareability = 2;
-    } else { /* Normal memory */
-        /* Outer/inner cacheability combine independently */
-        ret.attrs = combine_cacheattr_nibble(s1hi, s2hi) << 4
-                  | combine_cacheattr_nibble(s1lo, s2lo);
-
-        if (ret.attrs == 0x44) {
-            /* Any location for which the resultant memory type is Normal
-             * Inner Non-cacheable, Outer Non-cacheable is always treated
-             * as Outer Shareable.
-             */
-            ret.shareability = 2;
-        }
     }
 
     /* TODO: CombineS1S2Desc does not consider transient, only WB, RWA. */

From 8c7e17ef389699fd392e8613e144816b90af3a3a Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Thu, 5 May 2022 19:39:49 +0100
Subject: [PATCH 434/935] target/arm: Implement FEAT_S2FWB

Implement the handling of FEAT_S2FWB; the meat of this is in the new
combined_attrs_fwb() function which combines S1 and S2 attributes
when HCR_EL2.FWB is set.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220505183950.2781801-4-peter.maydell@linaro.org
---
 target/arm/cpu.h    |  5 +++
 target/arm/helper.c | 84 +++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 86 insertions(+), 3 deletions(-)

diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 18ca61e8e2..98efc638bb 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -3941,6 +3941,11 @@ static inline bool isar_feature_aa64_st(const ARMISARegisters *id)
     return FIELD_EX64(id->id_aa64mmfr2, ID_AA64MMFR2, ST) != 0;
 }
 
+static inline bool isar_feature_aa64_fwb(const ARMISARegisters *id)
+{
+    return FIELD_EX64(id->id_aa64mmfr2, ID_AA64MMFR2, FWB) != 0;
+}
+
 static inline bool isar_feature_aa64_bti(const ARMISARegisters *id)
 {
     return FIELD_EX64(id->id_aa64pfr1, ID_AA64PFR1, BT) != 0;
diff --git a/target/arm/helper.c b/target/arm/helper.c
index a2a9635841..073d6509c8 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -5161,6 +5161,9 @@ static void do_hcr_write(CPUARMState *env, uint64_t value, uint64_t valid_mask)
         if (cpu_isar_feature(aa64_scxtnum, cpu)) {
             valid_mask |= HCR_ENSCXT;
         }
+        if (cpu_isar_feature(aa64_fwb, cpu)) {
+            valid_mask |= HCR_FWB;
+        }
     }
 
     /* Clear RES0 bits.  */
@@ -5172,8 +5175,10 @@ static void do_hcr_write(CPUARMState *env, uint64_t value, uint64_t valid_mask)
      * HCR_PTW forbids certain page-table setups
      * HCR_DC disables stage1 and enables stage2 translation
      * HCR_DCT enables tagging on (disabled) stage1 translation
+     * HCR_FWB changes the interpretation of stage2 descriptor bits
      */
-    if ((env->cp15.hcr_el2 ^ value) & (HCR_VM | HCR_PTW | HCR_DC | HCR_DCT)) {
+    if ((env->cp15.hcr_el2 ^ value) &
+        (HCR_VM | HCR_PTW | HCR_DC | HCR_DCT | HCR_FWB)) {
         tlb_flush(CPU(cpu));
     }
     env->cp15.hcr_el2 = value;
@@ -10731,9 +10736,15 @@ static bool ptw_attrs_are_device(CPUARMState *env, ARMCacheAttrs cacheattrs)
      * attributes are therefore only Device if stage 2 specifies Device.
      * With HCR_EL2.FWB == 0 this is when descriptor bits [5:4] are 0b00,
      * ie when cacheattrs.attrs bits [3:2] are 0b00.
+     * With HCR_EL2.FWB == 1 this is when descriptor bit [4] is 0, ie
+     * when cacheattrs.attrs bit [2] is 0.
      */
     assert(cacheattrs.is_s2_format);
-    return (cacheattrs.attrs & 0xc) == 0;
+    if (arm_hcr_el2_eff(env) & HCR_FWB) {
+        return (cacheattrs.attrs & 0x4) == 0;
+    } else {
+        return (cacheattrs.attrs & 0xc) == 0;
+    }
 }
 
 /* Translate a S1 pagetable walk through S2 if needed.  */
@@ -12618,6 +12629,69 @@ static uint8_t combined_attrs_nofwb(CPUARMState *env,
     return ret_attrs;
 }
 
+static uint8_t force_cacheattr_nibble_wb(uint8_t attr)
+{
+    /*
+     * Given the 4 bits specifying the outer or inner cacheability
+     * in MAIR format, return a value specifying Normal Write-Back,
+     * with the allocation and transient hints taken from the input
+     * if the input specified some kind of cacheable attribute.
+     */
+    if (attr == 0 || attr == 4) {
+        /*
+         * 0 == an UNPREDICTABLE encoding
+         * 4 == Non-cacheable
+         * Either way, force Write-Back RW allocate non-transient
+         */
+        return 0xf;
+    }
+    /* Change WriteThrough to WriteBack, keep allocation and transient hints */
+    return attr | 4;
+}
+
+/*
+ * Combine the memory type and cacheability attributes of
+ * s1 and s2 for the HCR_EL2.FWB == 1 case, returning the
+ * combined attributes in MAIR_EL1 format.
+ */
+static uint8_t combined_attrs_fwb(CPUARMState *env,
+                                  ARMCacheAttrs s1, ARMCacheAttrs s2)
+{
+    switch (s2.attrs) {
+    case 7:
+        /* Use stage 1 attributes */
+        return s1.attrs;
+    case 6:
+        /*
+         * Force Normal Write-Back. Note that if S1 is Normal cacheable
+         * then we take the allocation hints from it; otherwise it is
+         * RW allocate, non-transient.
+         */
+        if ((s1.attrs & 0xf0) == 0) {
+            /* S1 is Device */
+            return 0xff;
+        }
+        /* Need to check the Inner and Outer nibbles separately */
+        return force_cacheattr_nibble_wb(s1.attrs & 0xf) |
+            force_cacheattr_nibble_wb(s1.attrs >> 4) << 4;
+    case 5:
+        /* If S1 attrs are Device, use them; otherwise Normal Non-cacheable */
+        if ((s1.attrs & 0xf0) == 0) {
+            return s1.attrs;
+        }
+        return 0x44;
+    case 0 ... 3:
+        /* Force Device, of subtype specified by S2 */
+        return s2.attrs << 2;
+    default:
+        /*
+         * RESERVED values (including RES0 descriptor bit [5] being nonzero);
+         * arbitrarily force Device.
+         */
+        return 0;
+    }
+}
+
 /* Combine S1 and S2 cacheability/shareability attributes, per D4.5.4
  * and CombineS1S2Desc()
  *
@@ -12652,7 +12726,11 @@ static ARMCacheAttrs combine_cacheattrs(CPUARMState *env,
     }
 
     /* Combine memory type and cacheability attributes */
-    ret.attrs = combined_attrs_nofwb(env, s1, s2);
+    if (arm_hcr_el2_eff(env) & HCR_FWB) {
+        ret.attrs = combined_attrs_fwb(env, s1, s2);
+    } else {
+        ret.attrs = combined_attrs_nofwb(env, s1, s2);
+    }
 
     /*
      * Any location for which the resultant memory type is any

From e04bf5a793ee5e7afdda8a1d1af08888fd67a989 Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Thu, 5 May 2022 19:39:50 +0100
Subject: [PATCH 435/935] target/arm: Enable FEAT_S2FWB for -cpu max

Enable the FEAT_S2FWB for -cpu max. Since FEAT_S2FWB requires that
CLIDR_EL1.{LoUU,LoUIS} are zero, we explicitly squash these (the
inherited CLIDR_EL1 value from the Cortex-A57 has them as 1).

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220505183950.2781801-5-peter.maydell@linaro.org
---
 docs/system/arm/emulation.rst |  1 +
 target/arm/cpu64.c            | 11 +++++++++++
 2 files changed, 12 insertions(+)

diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst
index 8ed466bf68..8f25502ced 100644
--- a/docs/system/arm/emulation.rst
+++ b/docs/system/arm/emulation.rst
@@ -52,6 +52,7 @@ the following architecture extensions:
 - FEAT_RAS (Reliability, availability, and serviceability)
 - FEAT_RDM (Advanced SIMD rounding double multiply accumulate instructions)
 - FEAT_RNG (Random number generator)
+- FEAT_S2FWB (Stage 2 forced Write-Back)
 - FEAT_SB (Speculation Barrier)
 - FEAT_SEL2 (Secure EL2)
 - FEAT_SHA1 (SHA1 instructions)
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 04427e073f..e83c013e1f 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -812,6 +812,7 @@ static void aarch64_max_initfn(Object *obj)
 {
     ARMCPU *cpu = ARM_CPU(obj);
     uint64_t t;
+    uint32_t u;
 
     if (kvm_enabled() || hvf_enabled()) {
         /* With KVM or HVF, '-cpu max' is identical to '-cpu host' */
@@ -842,6 +843,15 @@ static void aarch64_max_initfn(Object *obj)
     t = FIELD_DP64(t, MIDR_EL1, REVISION, 0);
     cpu->midr = t;
 
+    /*
+     * We're going to set FEAT_S2FWB, which mandates that CLIDR_EL1.{LoUU,LoUIS}
+     * are zero.
+     */
+    u = cpu->clidr;
+    u = FIELD_DP32(u, CLIDR_EL1, LOUIS, 0);
+    u = FIELD_DP32(u, CLIDR_EL1, LOUU, 0);
+    cpu->clidr = u;
+
     t = cpu->isar.id_aa64isar0;
     t = FIELD_DP64(t, ID_AA64ISAR0, AES, 2);      /* FEAT_PMULL */
     t = FIELD_DP64(t, ID_AA64ISAR0, SHA1, 1);     /* FEAT_SHA1 */
@@ -918,6 +928,7 @@ static void aarch64_max_initfn(Object *obj)
     t = FIELD_DP64(t, ID_AA64MMFR2, IESB, 1);     /* FEAT_IESB */
     t = FIELD_DP64(t, ID_AA64MMFR2, VARANGE, 1);  /* FEAT_LVA */
     t = FIELD_DP64(t, ID_AA64MMFR2, ST, 1);       /* FEAT_TTST */
+    t = FIELD_DP64(t, ID_AA64MMFR2, FWB, 1);      /* FEAT_S2FWB */
     t = FIELD_DP64(t, ID_AA64MMFR2, TTL, 1);      /* FEAT_TTL */
     t = FIELD_DP64(t, ID_AA64MMFR2, BBM, 2);      /* FEAT_BBM at level 2 */
     cpu->isar.id_aa64mmfr2 = t;

From 75662f36e3d11b986456210670719a7837111cd7 Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Mon, 9 May 2022 16:54:57 +0100
Subject: [PATCH 436/935] target/arm: Implement FEAT_IDST

The Armv8.4 feature FEAT_IDST specifies that exceptions generated by
read accesses to the feature ID space should report a syndrome code
of 0x18 (EC_SYSTEMREGISTERTRAP) rather than 0x00 (EC_UNCATEGORIZED).
The feature ID space is defined to be:
 op0 == 3, op1 == {0,1,3}, CRn == 0, CRm == {0-7}, op2 == {0-7}

In our implementation we might return the EC_UNCATEGORIZED syndrome
value for a system register access in four cases:
 * no reginfo struct in the hashtable
 * cp_access_ok() fails (ie ri->access doesn't permit the access)
 * ri->accessfn returns CP_ACCESS_TRAP_UNCATEGORIZED at runtime
 * ri->type includes ARM_CP_RAISES_EXC, and the readfn raises
   an UNDEF exception at runtime

We have very few regdefs that set ARM_CP_RAISES_EXC, and none of
them are in the feature ID space. (In the unlikely event that any
are added in future they would need to take care of setting the
correct syndrome themselves.) This patch deals with the other
three cases, and enables FEAT_IDST for AArch64 -cpu max.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220509155457.3560724-1-peter.maydell@linaro.org
---
 docs/system/arm/emulation.rst |  1 +
 target/arm/cpregs.h           | 24 ++++++++++++++++++++++++
 target/arm/cpu.h              |  5 +++++
 target/arm/cpu64.c            |  1 +
 target/arm/op_helper.c        |  9 +++++++++
 target/arm/translate-a64.c    | 28 ++++++++++++++++++++++++++--
 6 files changed, 66 insertions(+), 2 deletions(-)

diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst
index 8f25502ced..3e95bba0d2 100644
--- a/docs/system/arm/emulation.rst
+++ b/docs/system/arm/emulation.rst
@@ -31,6 +31,7 @@ the following architecture extensions:
 - FEAT_FlagM2 (Enhancements to flag manipulation instructions)
 - FEAT_HPDS (Hierarchical permission disables)
 - FEAT_I8MM (AArch64 Int8 matrix multiplication instructions)
+- FEAT_IDST (ID space trap handling)
 - FEAT_IESB (Implicit error synchronization event)
 - FEAT_JSCVT (JavaScript conversion instructions)
 - FEAT_LOR (Limited ordering regions)
diff --git a/target/arm/cpregs.h b/target/arm/cpregs.h
index db03d6a7e1..d9b678c2f1 100644
--- a/target/arm/cpregs.h
+++ b/target/arm/cpregs.h
@@ -461,4 +461,28 @@ static inline bool cp_access_ok(int current_el,
 /* Raw read of a coprocessor register (as needed for migration, etc) */
 uint64_t read_raw_cp_reg(CPUARMState *env, const ARMCPRegInfo *ri);
 
+/*
+ * Return true if the cp register encoding is in the "feature ID space" as
+ * defined by FEAT_IDST (and thus should be reported with ER_ELx.EC
+ * as EC_SYSTEMREGISTERTRAP rather than EC_UNCATEGORIZED).
+ */
+static inline bool arm_cpreg_encoding_in_idspace(uint8_t opc0, uint8_t opc1,
+                                                 uint8_t opc2,
+                                                 uint8_t crn, uint8_t crm)
+{
+    return opc0 == 3 && (opc1 == 0 || opc1 == 1 || opc1 == 3) &&
+        crn == 0 && crm < 8;
+}
+
+/*
+ * As arm_cpreg_encoding_in_idspace(), but take the encoding from an
+ * ARMCPRegInfo.
+ */
+static inline bool arm_cpreg_in_idspace(const ARMCPRegInfo *ri)
+{
+    return ri->state == ARM_CP_STATE_AA64 &&
+        arm_cpreg_encoding_in_idspace(ri->opc0, ri->opc1, ri->opc2,
+                                      ri->crn, ri->crm);
+}
+
 #endif /* TARGET_ARM_CPREGS_H */
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 98efc638bb..a99b430e54 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -3946,6 +3946,11 @@ static inline bool isar_feature_aa64_fwb(const ARMISARegisters *id)
     return FIELD_EX64(id->id_aa64mmfr2, ID_AA64MMFR2, FWB) != 0;
 }
 
+static inline bool isar_feature_aa64_ids(const ARMISARegisters *id)
+{
+    return FIELD_EX64(id->id_aa64mmfr2, ID_AA64MMFR2, IDS) != 0;
+}
+
 static inline bool isar_feature_aa64_bti(const ARMISARegisters *id)
 {
     return FIELD_EX64(id->id_aa64pfr1, ID_AA64PFR1, BT) != 0;
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index e83c013e1f..804a54922c 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -928,6 +928,7 @@ static void aarch64_max_initfn(Object *obj)
     t = FIELD_DP64(t, ID_AA64MMFR2, IESB, 1);     /* FEAT_IESB */
     t = FIELD_DP64(t, ID_AA64MMFR2, VARANGE, 1);  /* FEAT_LVA */
     t = FIELD_DP64(t, ID_AA64MMFR2, ST, 1);       /* FEAT_TTST */
+    t = FIELD_DP64(t, ID_AA64MMFR2, IDS, 1);      /* FEAT_IDST */
     t = FIELD_DP64(t, ID_AA64MMFR2, FWB, 1);      /* FEAT_S2FWB */
     t = FIELD_DP64(t, ID_AA64MMFR2, TTL, 1);      /* FEAT_TTL */
     t = FIELD_DP64(t, ID_AA64MMFR2, BBM, 2);      /* FEAT_BBM at level 2 */
diff --git a/target/arm/op_helper.c b/target/arm/op_helper.c
index 390b6578a8..c4bd668870 100644
--- a/target/arm/op_helper.c
+++ b/target/arm/op_helper.c
@@ -631,6 +631,7 @@ uint32_t HELPER(mrs_banked)(CPUARMState *env, uint32_t tgtmode, uint32_t regno)
 void HELPER(access_check_cp_reg)(CPUARMState *env, void *rip, uint32_t syndrome,
                                  uint32_t isread)
 {
+    ARMCPU *cpu = env_archcpu(env);
     const ARMCPRegInfo *ri = rip;
     CPAccessResult res = CP_ACCESS_OK;
     int target_el;
@@ -674,6 +675,14 @@ void HELPER(access_check_cp_reg)(CPUARMState *env, void *rip, uint32_t syndrome,
     case CP_ACCESS_TRAP:
         break;
     case CP_ACCESS_TRAP_UNCATEGORIZED:
+        if (cpu_isar_feature(aa64_ids, cpu) && isread &&
+            arm_cpreg_in_idspace(ri)) {
+            /*
+             * FEAT_IDST says this should be reported as EC_SYSTEMREGISTERTRAP,
+             * not EC_UNCATEGORIZED
+             */
+            break;
+        }
         syndrome = syn_uncategorized();
         break;
     default:
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 6a27234a5c..176a3c83ba 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -1795,6 +1795,30 @@ static void gen_set_nzcv(TCGv_i64 tcg_rt)
     tcg_temp_free_i32(nzcv);
 }
 
+static void gen_sysreg_undef(DisasContext *s, bool isread,
+                             uint8_t op0, uint8_t op1, uint8_t op2,
+                             uint8_t crn, uint8_t crm, uint8_t rt)
+{
+    /*
+     * Generate code to emit an UNDEF with correct syndrome
+     * information for a failed system register access.
+     * This is EC_UNCATEGORIZED (ie a standard UNDEF) in most cases,
+     * but if FEAT_IDST is implemented then read accesses to registers
+     * in the feature ID space are reported with the EC_SYSTEMREGISTERTRAP
+     * syndrome.
+     */
+    uint32_t syndrome;
+
+    if (isread && dc_isar_feature(aa64_ids, s) &&
+        arm_cpreg_encoding_in_idspace(op0, op1, op2, crn, crm)) {
+        syndrome = syn_aa64_sysregtrap(op0, op1, op2, crn, crm, rt, isread);
+    } else {
+        syndrome = syn_uncategorized();
+    }
+    gen_exception_insn(s, s->pc_curr, EXCP_UDEF, syndrome,
+                       default_exception_el(s));
+}
+
 /* MRS - move from system register
  * MSR (register) - move to system register
  * SYS
@@ -1820,13 +1844,13 @@ static void handle_sys(DisasContext *s, uint32_t insn, bool isread,
         qemu_log_mask(LOG_UNIMP, "%s access to unsupported AArch64 "
                       "system register op0:%d op1:%d crn:%d crm:%d op2:%d\n",
                       isread ? "read" : "write", op0, op1, crn, crm, op2);
-        unallocated_encoding(s);
+        gen_sysreg_undef(s, isread, op0, op1, op2, crn, crm, rt);
         return;
     }
 
     /* Check access permissions */
     if (!cp_access_ok(s->current_el, ri, isread)) {
-        unallocated_encoding(s);
+        gen_sysreg_undef(s, isread, op0, op1, op2, crn, crm, rt);
         return;
     }
 

From 3d52472f81f0e0684fcab238ab816faeee6b8bcd Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Mon, 9 May 2022 17:04:43 +0100
Subject: [PATCH 437/935] target/arm: Drop unsupported_encoding() macro
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The unsupported_encoding() macro logs a LOG_UNIMP message and then
generates code to raise the usual exception for an unallocated
encoding.  Back when we were still implementing the A64 decoder this
was helpful for flagging up when guest code was using something we
hadn't yet implemented.  Now we completely cover the A64 instruction
set it is barely used.  The only remaining uses are for five
instructions whose semantics are "UNDEF, unless being run under
external halting debug":
 * HLT (when not being used for semihosting)
 * DCPSR1, DCPS2, DCPS3
 * DRPS

QEMU doesn't implement external halting debug, so for us the UNDEF is
the architecturally correct behaviour (because it's not possible to
execute these instructions with halting debug enabled).  The
LOG_UNIMP doesn't serve a useful purpose; replace these uses of
unsupported_encoding() with unallocated_encoding(), and delete the
macro.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220509160443.3561604-1-peter.maydell@linaro.org
---
 target/arm/translate-a64.c | 8 ++++----
 target/arm/translate-a64.h | 9 ---------
 2 files changed, 4 insertions(+), 13 deletions(-)

diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 176a3c83ba..f502545307 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -2127,13 +2127,13 @@ static void disas_exc(DisasContext *s, uint32_t insn)
              * with our 32-bit semihosting).
              */
             if (s->current_el == 0) {
-                unsupported_encoding(s, insn);
+                unallocated_encoding(s);
                 break;
             }
 #endif
             gen_exception_internal_insn(s, s->pc_curr, EXCP_SEMIHOST);
         } else {
-            unsupported_encoding(s, insn);
+            unallocated_encoding(s);
         }
         break;
     case 5:
@@ -2142,7 +2142,7 @@ static void disas_exc(DisasContext *s, uint32_t insn)
             break;
         }
         /* DCPS1, DCPS2, DCPS3 */
-        unsupported_encoding(s, insn);
+        unallocated_encoding(s);
         break;
     default:
         unallocated_encoding(s);
@@ -2307,7 +2307,7 @@ static void disas_uncond_b_reg(DisasContext *s, uint32_t insn)
         if (op3 != 0 || op4 != 0 || rn != 0x1f) {
             goto do_unallocated;
         } else {
-            unsupported_encoding(s, insn);
+            unallocated_encoding(s);
         }
         return;
 
diff --git a/target/arm/translate-a64.h b/target/arm/translate-a64.h
index 38884158aa..f2e8ee0ee1 100644
--- a/target/arm/translate-a64.h
+++ b/target/arm/translate-a64.h
@@ -18,15 +18,6 @@
 #ifndef TARGET_ARM_TRANSLATE_A64_H
 #define TARGET_ARM_TRANSLATE_A64_H
 
-#define unsupported_encoding(s, insn)                                    \
-    do {                                                                 \
-        qemu_log_mask(LOG_UNIMP,                                         \
-                      "%s:%d: unsupported instruction encoding 0x%08x "  \
-                      "at pc=%016" PRIx64 "\n",                          \
-                      __FILE__, __LINE__, insn, s->pc_curr);             \
-        unallocated_encoding(s);                                         \
-    } while (0)
-
 TCGv_i64 new_tmp_a64(DisasContext *s);
 TCGv_i64 new_tmp_a64_local(DisasContext *s);
 TCGv_i64 new_tmp_a64_zero(DisasContext *s);

From 272f75e89088c968c861fef516a4ebc70846dcd5 Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Thu, 12 May 2022 16:14:52 +0100
Subject: [PATCH 438/935] hw/intc/arm_gicv3_cpuif: Handle CPUs that don't
 specify GICv3 parameters

We allow a GICv3 to be connected to any CPU, but we don't do anything
to handle the case where the CPU type doesn't in hardware have a
GICv3 CPU interface and so the various GIC configuration fields
(gic_num_lrs, vprebits, vpribits) are not specified.

The current behaviour is that we will add the EL1 CPU interface
registers, but will not put in the EL2 CPU interface registers, even
if the CPU has EL2, which will leave the GIC in a broken state and
probably result in the guest crashing as it tries to set it up.  This
only affects the virt board when using the cortex-a15 or cortex-a7
CPU types (both 32-bit) with -machine gic-version=3 (or 'max')
and -machine virtualization=on.

Instead of failing to set up the EL2 registers, if the CPU doesn't
define the GIC configuration set it to a reasonable default, matching
the standard configuration for most Arm CPUs.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220512151457.3899052-2-peter.maydell@linaro.org
---
 hw/intc/arm_gicv3_cpuif.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/hw/intc/arm_gicv3_cpuif.c b/hw/intc/arm_gicv3_cpuif.c
index 9efba798f8..df2f858356 100644
--- a/hw/intc/arm_gicv3_cpuif.c
+++ b/hw/intc/arm_gicv3_cpuif.c
@@ -2755,6 +2755,15 @@ void gicv3_init_cpuif(GICv3State *s)
         ARMCPU *cpu = ARM_CPU(qemu_get_cpu(i));
         GICv3CPUState *cs = &s->cpu[i];
 
+        /*
+         * If the CPU doesn't define a GICv3 configuration, probably because
+         * in real hardware it doesn't have one, then we use default values
+         * matching the one used by most Arm CPUs. This applies to:
+         *  cpu->gic_num_lrs
+         *  cpu->gic_vpribits
+         *  cpu->gic_vprebits
+         */
+
         /* Note that we can't just use the GICv3CPUState as an opaque pointer
          * in define_arm_cp_regs_with_opaque(), because when we're called back
          * it might be with code translated by CPU 0 but run by CPU 1, in
@@ -2763,13 +2772,12 @@ void gicv3_init_cpuif(GICv3State *s)
          * get back to the GICv3CPUState from the CPUARMState.
          */
         define_arm_cp_regs(cpu, gicv3_cpuif_reginfo);
-        if (arm_feature(&cpu->env, ARM_FEATURE_EL2)
-            && cpu->gic_num_lrs) {
+        if (arm_feature(&cpu->env, ARM_FEATURE_EL2)) {
             int j;
 
-            cs->num_list_regs = cpu->gic_num_lrs;
-            cs->vpribits = cpu->gic_vpribits;
-            cs->vprebits = cpu->gic_vprebits;
+            cs->num_list_regs = cpu->gic_num_lrs ?: 4;
+            cs->vpribits = cpu->gic_vpribits ?: 5;
+            cs->vprebits = cpu->gic_vprebits ?: 5;
 
             /* Check against architectural constraints: getting these
              * wrong would be a bug in the CPU code defining these,

From 9c6f933e71ccfde036d7e19c1ddc2b1a82cc45c0 Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Thu, 12 May 2022 16:14:53 +0100
Subject: [PATCH 439/935] hw/intc/arm_gicv3: report correct PRIbits field in
 ICV_CTLR_EL1

As noted in the comment, the PRIbits field in ICV_CTLR_EL1 is
supposed to match the ICH_VTR_EL2 PRIbits setting; that is, it is the
virtual priority bit setting, not the physical priority bit setting.
(For QEMU currently we always implement 8 bits of physical priority,
so the PRIbits field was previously 7, since it is defined to be
"priority bits - 1".)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220512151457.3899052-3-peter.maydell@linaro.org
Message-id: 20220506162129.2896966-2-peter.maydell@linaro.org
---
 hw/intc/arm_gicv3_cpuif.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/intc/arm_gicv3_cpuif.c b/hw/intc/arm_gicv3_cpuif.c
index df2f858356..ebf269b73a 100644
--- a/hw/intc/arm_gicv3_cpuif.c
+++ b/hw/intc/arm_gicv3_cpuif.c
@@ -657,7 +657,7 @@ static uint64_t icv_ctlr_read(CPUARMState *env, const ARMCPRegInfo *ri)
      * should match the ones reported in ich_vtr_read().
      */
     value = ICC_CTLR_EL1_A3V | (1 << ICC_CTLR_EL1_IDBITS_SHIFT) |
-        (7 << ICC_CTLR_EL1_PRIBITS_SHIFT);
+        ((cs->vpribits - 1) << ICC_CTLR_EL1_PRIBITS_SHIFT);
 
     if (cs->ich_vmcr_el2 & ICH_VMCR_EL2_VEOIM) {
         value |= ICC_CTLR_EL1_EOIMODE;

From 9774c0f7ba6ae2980a291cb53a13661ddaa2f5de Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Thu, 12 May 2022 16:14:54 +0100
Subject: [PATCH 440/935] hw/intc/arm_gicv3_kvm.c: Stop using GIC_MIN_BPR
 constant

The GIC_MIN_BPR constant defines the minimum BPR value that the TCG
emulated GICv3 supports.  We're currently using this also as the
value we reset the KVM GICv3 ICC_BPR registers to, but this is only
right by accident.

We want to make the emulated GICv3 use a configurable number of
priority bits, which means that GIC_MIN_BPR will no longer be a
constant.  Replace the uses in the KVM reset code with literal 0,
plus a constant explaining why this is reasonable.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220512151457.3899052-4-peter.maydell@linaro.org
Message-id: 20220506162129.2896966-3-peter.maydell@linaro.org
---
 hw/intc/arm_gicv3_kvm.c | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/hw/intc/arm_gicv3_kvm.c b/hw/intc/arm_gicv3_kvm.c
index 2922c516e5..3ca643ecba 100644
--- a/hw/intc/arm_gicv3_kvm.c
+++ b/hw/intc/arm_gicv3_kvm.c
@@ -673,9 +673,19 @@ static void arm_gicv3_icc_reset(CPUARMState *env, const ARMCPRegInfo *ri)
     s = c->gic;
 
     c->icc_pmr_el1 = 0;
-    c->icc_bpr[GICV3_G0] = GIC_MIN_BPR;
-    c->icc_bpr[GICV3_G1] = GIC_MIN_BPR;
-    c->icc_bpr[GICV3_G1NS] = GIC_MIN_BPR;
+    /*
+     * Architecturally the reset value of the ICC_BPR registers
+     * is UNKNOWN. We set them all to 0 here; when the kernel
+     * uses these values to program the ICH_VMCR_EL2 fields that
+     * determine the guest-visible ICC_BPR register values, the
+     * hardware's "writing a value less than the minimum sets
+     * the field to the minimum value" behaviour will result in
+     * them effectively resetting to the correct minimum value
+     * for the host GIC.
+     */
+    c->icc_bpr[GICV3_G0] = 0;
+    c->icc_bpr[GICV3_G1] = 0;
+    c->icc_bpr[GICV3_G1NS] = 0;
 
     c->icc_sre_el1 = 0x7;
     memset(c->icc_apr, 0, sizeof(c->icc_apr));

From 84597ff39484ec171567c7c80061100eb4a6c331 Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Thu, 12 May 2022 16:14:55 +0100
Subject: [PATCH 441/935] hw/intc/arm_gicv3: Support configurable number of
 physical priority bits

The GICv3 code has always supported a configurable number of virtual
priority and preemption bits, but our implementation currently
hardcodes the number of physical priority bits at 8.  This is not
what most hardware implementations provide; for instance the
Cortex-A53 provides only 5 bits of physical priority.

Make the number of physical priority/preemption bits driven by fields
in the GICv3CPUState, the way that we already do for virtual
priority/preemption bits.  We set cs->pribits to 8, so there is no
behavioural change in this commit.  A following commit will add the
machinery for CPUs to set this to the correct value for their
implementation.

Note that changing the number of priority bits would be a migration
compatibility break, because the semantics of the icc_apr[][] array
changes.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220512151457.3899052-5-peter.maydell@linaro.org
Message-id: 20220506162129.2896966-4-peter.maydell@linaro.org
---
 hw/intc/arm_gicv3_cpuif.c          | 182 ++++++++++++++++++++---------
 include/hw/intc/arm_gicv3_common.h |   7 +-
 2 files changed, 130 insertions(+), 59 deletions(-)

diff --git a/hw/intc/arm_gicv3_cpuif.c b/hw/intc/arm_gicv3_cpuif.c
index ebf269b73a..69a15f7a44 100644
--- a/hw/intc/arm_gicv3_cpuif.c
+++ b/hw/intc/arm_gicv3_cpuif.c
@@ -787,6 +787,36 @@ static uint64_t icv_iar_read(CPUARMState *env, const ARMCPRegInfo *ri)
     return intid;
 }
 
+static uint32_t icc_fullprio_mask(GICv3CPUState *cs)
+{
+    /*
+     * Return a mask word which clears the unimplemented priority bits
+     * from a priority value for a physical interrupt. (Not to be confused
+     * with the group priority, whose mask depends on the value of BPR
+     * for the interrupt group.)
+     */
+    return ~0U << (8 - cs->pribits);
+}
+
+static inline int icc_min_bpr(GICv3CPUState *cs)
+{
+    /* The minimum BPR for the physical interface. */
+    return 7 - cs->prebits;
+}
+
+static inline int icc_min_bpr_ns(GICv3CPUState *cs)
+{
+    return icc_min_bpr(cs) + 1;
+}
+
+static inline int icc_num_aprs(GICv3CPUState *cs)
+{
+    /* Return the number of APR registers (1, 2, or 4) */
+    int aprmax = 1 << MAX(cs->prebits - 5, 0);
+    assert(aprmax <= ARRAY_SIZE(cs->icc_apr[0]));
+    return aprmax;
+}
+
 static int icc_highest_active_prio(GICv3CPUState *cs)
 {
     /* Calculate the current running priority based on the set bits
@@ -794,14 +824,14 @@ static int icc_highest_active_prio(GICv3CPUState *cs)
      */
     int i;
 
-    for (i = 0; i < ARRAY_SIZE(cs->icc_apr[0]); i++) {
+    for (i = 0; i < icc_num_aprs(cs); i++) {
         uint32_t apr = cs->icc_apr[GICV3_G0][i] |
             cs->icc_apr[GICV3_G1][i] | cs->icc_apr[GICV3_G1NS][i];
 
         if (!apr) {
             continue;
         }
-        return (i * 32 + ctz32(apr)) << (GIC_MIN_BPR + 1);
+        return (i * 32 + ctz32(apr)) << (icc_min_bpr(cs) + 1);
     }
     /* No current active interrupts: return idle priority */
     return 0xff;
@@ -980,7 +1010,7 @@ static void icc_pmr_write(CPUARMState *env, const ARMCPRegInfo *ri,
 
     trace_gicv3_icc_pmr_write(gicv3_redist_affid(cs), value);
 
-    value &= 0xff;
+    value &= icc_fullprio_mask(cs);
 
     if (arm_feature(env, ARM_FEATURE_EL3) && !arm_is_secure(env) &&
         (env->cp15.scr_el3 & SCR_FIQ)) {
@@ -1004,7 +1034,7 @@ static void icc_activate_irq(GICv3CPUState *cs, int irq)
      */
     uint32_t mask = icc_gprio_mask(cs, cs->hppi.grp);
     int prio = cs->hppi.prio & mask;
-    int aprbit = prio >> 1;
+    int aprbit = prio >> (8 - cs->prebits);
     int regno = aprbit / 32;
     int regbit = aprbit % 32;
 
@@ -1162,7 +1192,7 @@ static void icc_drop_prio(GICv3CPUState *cs, int grp)
      */
     int i;
 
-    for (i = 0; i < ARRAY_SIZE(cs->icc_apr[grp]); i++) {
+    for (i = 0; i < icc_num_aprs(cs); i++) {
         uint64_t *papr = &cs->icc_apr[grp][i];
 
         if (!*papr) {
@@ -1590,7 +1620,7 @@ static void icc_bpr_write(CPUARMState *env, const ARMCPRegInfo *ri,
         return;
     }
 
-    minval = (grp == GICV3_G1NS) ? GIC_MIN_BPR_NS : GIC_MIN_BPR;
+    minval = (grp == GICV3_G1NS) ? icc_min_bpr_ns(cs) : icc_min_bpr(cs);
     if (value < minval) {
         value = minval;
     }
@@ -2171,19 +2201,19 @@ static void icc_reset(CPUARMState *env, const ARMCPRegInfo *ri)
 
     cs->icc_ctlr_el1[GICV3_S] = ICC_CTLR_EL1_A3V |
         (1 << ICC_CTLR_EL1_IDBITS_SHIFT) |
-        (7 << ICC_CTLR_EL1_PRIBITS_SHIFT);
+        ((cs->pribits - 1) << ICC_CTLR_EL1_PRIBITS_SHIFT);
     cs->icc_ctlr_el1[GICV3_NS] = ICC_CTLR_EL1_A3V |
         (1 << ICC_CTLR_EL1_IDBITS_SHIFT) |
-        (7 << ICC_CTLR_EL1_PRIBITS_SHIFT);
+        ((cs->pribits - 1) << ICC_CTLR_EL1_PRIBITS_SHIFT);
     cs->icc_pmr_el1 = 0;
-    cs->icc_bpr[GICV3_G0] = GIC_MIN_BPR;
-    cs->icc_bpr[GICV3_G1] = GIC_MIN_BPR;
-    cs->icc_bpr[GICV3_G1NS] = GIC_MIN_BPR_NS;
+    cs->icc_bpr[GICV3_G0] = icc_min_bpr(cs);
+    cs->icc_bpr[GICV3_G1] = icc_min_bpr(cs);
+    cs->icc_bpr[GICV3_G1NS] = icc_min_bpr_ns(cs);
     memset(cs->icc_apr, 0, sizeof(cs->icc_apr));
     memset(cs->icc_igrpen, 0, sizeof(cs->icc_igrpen));
     cs->icc_ctlr_el3 = ICC_CTLR_EL3_NDS | ICC_CTLR_EL3_A3V |
         (1 << ICC_CTLR_EL3_IDBITS_SHIFT) |
-        (7 << ICC_CTLR_EL3_PRIBITS_SHIFT);
+        ((cs->pribits - 1) << ICC_CTLR_EL3_PRIBITS_SHIFT);
 
     memset(cs->ich_apr, 0, sizeof(cs->ich_apr));
     cs->ich_hcr_el2 = 0;
@@ -2238,27 +2268,6 @@ static const ARMCPRegInfo gicv3_cpuif_reginfo[] = {
       .readfn = icc_ap_read,
       .writefn = icc_ap_write,
     },
-    { .name = "ICC_AP0R1_EL1", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 8, .opc2 = 5,
-      .type = ARM_CP_IO | ARM_CP_NO_RAW,
-      .access = PL1_RW, .accessfn = gicv3_fiq_access,
-      .readfn = icc_ap_read,
-      .writefn = icc_ap_write,
-    },
-    { .name = "ICC_AP0R2_EL1", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 8, .opc2 = 6,
-      .type = ARM_CP_IO | ARM_CP_NO_RAW,
-      .access = PL1_RW, .accessfn = gicv3_fiq_access,
-      .readfn = icc_ap_read,
-      .writefn = icc_ap_write,
-    },
-    { .name = "ICC_AP0R3_EL1", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 8, .opc2 = 7,
-      .type = ARM_CP_IO | ARM_CP_NO_RAW,
-      .access = PL1_RW, .accessfn = gicv3_fiq_access,
-      .readfn = icc_ap_read,
-      .writefn = icc_ap_write,
-    },
     /* All the ICC_AP1R*_EL1 registers are banked */
     { .name = "ICC_AP1R0_EL1", .state = ARM_CP_STATE_BOTH,
       .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 9, .opc2 = 0,
@@ -2267,27 +2276,6 @@ static const ARMCPRegInfo gicv3_cpuif_reginfo[] = {
       .readfn = icc_ap_read,
       .writefn = icc_ap_write,
     },
-    { .name = "ICC_AP1R1_EL1", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 9, .opc2 = 1,
-      .type = ARM_CP_IO | ARM_CP_NO_RAW,
-      .access = PL1_RW, .accessfn = gicv3_irq_access,
-      .readfn = icc_ap_read,
-      .writefn = icc_ap_write,
-    },
-    { .name = "ICC_AP1R2_EL1", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 9, .opc2 = 2,
-      .type = ARM_CP_IO | ARM_CP_NO_RAW,
-      .access = PL1_RW, .accessfn = gicv3_irq_access,
-      .readfn = icc_ap_read,
-      .writefn = icc_ap_write,
-    },
-    { .name = "ICC_AP1R3_EL1", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 9, .opc2 = 3,
-      .type = ARM_CP_IO | ARM_CP_NO_RAW,
-      .access = PL1_RW, .accessfn = gicv3_irq_access,
-      .readfn = icc_ap_read,
-      .writefn = icc_ap_write,
-    },
     { .name = "ICC_DIR_EL1", .state = ARM_CP_STATE_BOTH,
       .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 11, .opc2 = 1,
       .type = ARM_CP_IO | ARM_CP_NO_RAW,
@@ -2430,6 +2418,54 @@ static const ARMCPRegInfo gicv3_cpuif_reginfo[] = {
     },
 };
 
+static const ARMCPRegInfo gicv3_cpuif_icc_apxr1_reginfo[] = {
+    { .name = "ICC_AP0R1_EL1", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 8, .opc2 = 5,
+      .type = ARM_CP_IO | ARM_CP_NO_RAW,
+      .access = PL1_RW, .accessfn = gicv3_fiq_access,
+      .readfn = icc_ap_read,
+      .writefn = icc_ap_write,
+    },
+    { .name = "ICC_AP1R1_EL1", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 9, .opc2 = 1,
+      .type = ARM_CP_IO | ARM_CP_NO_RAW,
+      .access = PL1_RW, .accessfn = gicv3_irq_access,
+      .readfn = icc_ap_read,
+      .writefn = icc_ap_write,
+    },
+};
+
+static const ARMCPRegInfo gicv3_cpuif_icc_apxr23_reginfo[] = {
+    { .name = "ICC_AP0R2_EL1", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 8, .opc2 = 6,
+      .type = ARM_CP_IO | ARM_CP_NO_RAW,
+      .access = PL1_RW, .accessfn = gicv3_fiq_access,
+      .readfn = icc_ap_read,
+      .writefn = icc_ap_write,
+    },
+    { .name = "ICC_AP0R3_EL1", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 8, .opc2 = 7,
+      .type = ARM_CP_IO | ARM_CP_NO_RAW,
+      .access = PL1_RW, .accessfn = gicv3_fiq_access,
+      .readfn = icc_ap_read,
+      .writefn = icc_ap_write,
+    },
+    { .name = "ICC_AP1R2_EL1", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 9, .opc2 = 2,
+      .type = ARM_CP_IO | ARM_CP_NO_RAW,
+      .access = PL1_RW, .accessfn = gicv3_irq_access,
+      .readfn = icc_ap_read,
+      .writefn = icc_ap_write,
+    },
+    { .name = "ICC_AP1R3_EL1", .state = ARM_CP_STATE_BOTH,
+      .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 9, .opc2 = 3,
+      .type = ARM_CP_IO | ARM_CP_NO_RAW,
+      .access = PL1_RW, .accessfn = gicv3_irq_access,
+      .readfn = icc_ap_read,
+      .writefn = icc_ap_write,
+    },
+};
+
 static uint64_t ich_ap_read(CPUARMState *env, const ARMCPRegInfo *ri)
 {
     GICv3CPUState *cs = icc_cs_from_env(env);
@@ -2772,6 +2808,44 @@ void gicv3_init_cpuif(GICv3State *s)
          * get back to the GICv3CPUState from the CPUARMState.
          */
         define_arm_cp_regs(cpu, gicv3_cpuif_reginfo);
+
+        /*
+         * For the moment, retain the existing behaviour of 8 priority bits;
+         * in a following commit we will take this from the CPU state,
+         * as we do for the virtual priority bits.
+         */
+        cs->pribits = 8;
+        /*
+         * The GICv3 has separate ID register fields for virtual priority
+         * and preemption bit values, but only a single ID register field
+         * for the physical priority bits. The preemption bit count is
+         * always the same as the priority bit count, except that 8 bits
+         * of priority means 7 preemption bits. We precalculate the
+         * preemption bits because it simplifies the code and makes the
+         * parallels between the virtual and physical bits of the GIC
+         * a bit clearer.
+         */
+        cs->prebits = cs->pribits;
+        if (cs->prebits == 8) {
+            cs->prebits--;
+        }
+        /*
+         * Check that CPU code defining pribits didn't violate
+         * architectural constraints our implementation relies on.
+         */
+        g_assert(cs->pribits >= 4 && cs->pribits <= 8);
+
+        /*
+         * gicv3_cpuif_reginfo[] defines ICC_AP*R0_EL1; add definitions
+         * for ICC_AP*R{1,2,3}_EL1 if the prebits value requires them.
+         */
+        if (cs->prebits >= 6) {
+            define_arm_cp_regs(cpu, gicv3_cpuif_icc_apxr1_reginfo);
+        }
+        if (cs->prebits == 7) {
+            define_arm_cp_regs(cpu, gicv3_cpuif_icc_apxr23_reginfo);
+        }
+
         if (arm_feature(&cpu->env, ARM_FEATURE_EL2)) {
             int j;
 
diff --git a/include/hw/intc/arm_gicv3_common.h b/include/hw/intc/arm_gicv3_common.h
index 4e41610055..46677ec345 100644
--- a/include/hw/intc/arm_gicv3_common.h
+++ b/include/hw/intc/arm_gicv3_common.h
@@ -51,11 +51,6 @@
 /* Maximum number of list registers (architectural limit) */
 #define GICV3_LR_MAX 16
 
-/* Minimum BPR for Secure, or when security not enabled */
-#define GIC_MIN_BPR 0
-/* Minimum BPR for Nonsecure when security is enabled */
-#define GIC_MIN_BPR_NS (GIC_MIN_BPR + 1)
-
 /* For some distributor fields we want to model the array of 32-bit
  * register values which hold various bitmaps corresponding to enabled,
  * pending, etc bits. These macros and functions facilitate that; the
@@ -206,6 +201,8 @@ struct GICv3CPUState {
     int num_list_regs;
     int vpribits; /* number of virtual priority bits */
     int vprebits; /* number of virtual preemption bits */
+    int pribits; /* number of physical priority bits */
+    int prebits; /* number of physical preemption bits */
 
     /* Current highest priority pending interrupt for this CPU.
      * This is cached information that can be recalculated from the

From 39f29e599355f9512482b67624e7a6c9000c5ddd Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Thu, 12 May 2022 16:14:56 +0100
Subject: [PATCH 442/935] hw/intc/arm_gicv3: Use correct number of priority
 bits for the CPU

Make the GICv3 set its number of bits of physical priority from the
implementation-specific value provided in the CPU state struct, in
the same way we already do for virtual priority bits.  Because this
would be a migration compatibility break, we provide a property
force-8-bit-prio which is enabled for 7.0 and earlier versioned board
models to retain the legacy "always use 8 bits" behaviour.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220512151457.3899052-6-peter.maydell@linaro.org
Message-id: 20220506162129.2896966-5-peter.maydell@linaro.org
---
 hw/core/machine.c                  |  4 +++-
 hw/intc/arm_gicv3_common.c         |  5 +++++
 hw/intc/arm_gicv3_cpuif.c          | 15 +++++++++++----
 include/hw/intc/arm_gicv3_common.h |  1 +
 target/arm/cpu.h                   |  1 +
 target/arm/cpu64.c                 |  6 ++++++
 6 files changed, 27 insertions(+), 5 deletions(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index b03d9192ba..bb0dc8f6a9 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -41,7 +41,9 @@
 #include "hw/virtio/virtio-pci.h"
 #include "qom/object_interfaces.h"
 
-GlobalProperty hw_compat_7_0[] = {};
+GlobalProperty hw_compat_7_0[] = {
+    { "arm-gicv3-common", "force-8-bit-prio", "on" },
+};
 const size_t hw_compat_7_0_len = G_N_ELEMENTS(hw_compat_7_0);
 
 GlobalProperty hw_compat_6_2[] = {
diff --git a/hw/intc/arm_gicv3_common.c b/hw/intc/arm_gicv3_common.c
index 5634c6fc78..351843db4a 100644
--- a/hw/intc/arm_gicv3_common.c
+++ b/hw/intc/arm_gicv3_common.c
@@ -563,6 +563,11 @@ static Property arm_gicv3_common_properties[] = {
     DEFINE_PROP_UINT32("revision", GICv3State, revision, 3),
     DEFINE_PROP_BOOL("has-lpi", GICv3State, lpi_enable, 0),
     DEFINE_PROP_BOOL("has-security-extensions", GICv3State, security_extn, 0),
+    /*
+     * Compatibility property: force 8 bits of physical priority, even
+     * if the CPU being emulated should have fewer.
+     */
+    DEFINE_PROP_BOOL("force-8-bit-prio", GICv3State, force_8bit_prio, 0),
     DEFINE_PROP_ARRAY("redist-region-count", GICv3State, nb_redist_regions,
                       redist_region_count, qdev_prop_uint32, uint32_t),
     DEFINE_PROP_LINK("sysmem", GICv3State, dma, TYPE_MEMORY_REGION,
diff --git a/hw/intc/arm_gicv3_cpuif.c b/hw/intc/arm_gicv3_cpuif.c
index 69a15f7a44..66e06b787c 100644
--- a/hw/intc/arm_gicv3_cpuif.c
+++ b/hw/intc/arm_gicv3_cpuif.c
@@ -2798,6 +2798,7 @@ void gicv3_init_cpuif(GICv3State *s)
          *  cpu->gic_num_lrs
          *  cpu->gic_vpribits
          *  cpu->gic_vprebits
+         *  cpu->gic_pribits
          */
 
         /* Note that we can't just use the GICv3CPUState as an opaque pointer
@@ -2810,11 +2811,17 @@ void gicv3_init_cpuif(GICv3State *s)
         define_arm_cp_regs(cpu, gicv3_cpuif_reginfo);
 
         /*
-         * For the moment, retain the existing behaviour of 8 priority bits;
-         * in a following commit we will take this from the CPU state,
-         * as we do for the virtual priority bits.
+         * The CPU implementation specifies the number of supported
+         * bits of physical priority. For backwards compatibility
+         * of migration, we have a compat property that forces use
+         * of 8 priority bits regardless of what the CPU really has.
          */
-        cs->pribits = 8;
+        if (s->force_8bit_prio) {
+            cs->pribits = 8;
+        } else {
+            cs->pribits = cpu->gic_pribits ?: 5;
+        }
+
         /*
          * The GICv3 has separate ID register fields for virtual priority
          * and preemption bit values, but only a single ID register field
diff --git a/include/hw/intc/arm_gicv3_common.h b/include/hw/intc/arm_gicv3_common.h
index 46677ec345..ab5182a28a 100644
--- a/include/hw/intc/arm_gicv3_common.h
+++ b/include/hw/intc/arm_gicv3_common.h
@@ -248,6 +248,7 @@ struct GICv3State {
     uint32_t revision;
     bool lpi_enable;
     bool security_extn;
+    bool force_8bit_prio;
     bool irq_reset_nonsecure;
     bool gicd_no_migration_shift_bug;
 
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index a99b430e54..a42464eb57 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -1002,6 +1002,7 @@ struct ArchCPU {
     int gic_num_lrs; /* number of list registers */
     int gic_vpribits; /* number of virtual priority bits */
     int gic_vprebits; /* number of virtual preemption bits */
+    int gic_pribits; /* number of physical priority bits */
 
     /* Whether the cfgend input is high (i.e. this CPU should reset into
      * big-endian mode).  This setting isn't used directly: instead it modifies
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 804a54922c..7628f4fa39 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -87,6 +87,7 @@ static void aarch64_a57_initfn(Object *obj)
     cpu->gic_num_lrs = 4;
     cpu->gic_vpribits = 5;
     cpu->gic_vprebits = 5;
+    cpu->gic_pribits = 5;
     define_cortex_a72_a57_a53_cp_reginfo(cpu);
 }
 
@@ -140,6 +141,7 @@ static void aarch64_a53_initfn(Object *obj)
     cpu->gic_num_lrs = 4;
     cpu->gic_vpribits = 5;
     cpu->gic_vprebits = 5;
+    cpu->gic_pribits = 5;
     define_cortex_a72_a57_a53_cp_reginfo(cpu);
 }
 
@@ -191,6 +193,7 @@ static void aarch64_a72_initfn(Object *obj)
     cpu->gic_num_lrs = 4;
     cpu->gic_vpribits = 5;
     cpu->gic_vprebits = 5;
+    cpu->gic_pribits = 5;
     define_cortex_a72_a57_a53_cp_reginfo(cpu);
 }
 
@@ -252,6 +255,7 @@ static void aarch64_a76_initfn(Object *obj)
     cpu->gic_num_lrs = 4;
     cpu->gic_vpribits = 5;
     cpu->gic_vprebits = 5;
+    cpu->gic_pribits = 5;
 
     /* From B5.1 AdvSIMD AArch64 register summary */
     cpu->isar.mvfr0 = 0x10110222;
@@ -317,6 +321,7 @@ static void aarch64_neoverse_n1_initfn(Object *obj)
     cpu->gic_num_lrs = 4;
     cpu->gic_vpribits = 5;
     cpu->gic_vprebits = 5;
+    cpu->gic_pribits = 5;
 
     /* From B5.1 AdvSIMD AArch64 register summary */
     cpu->isar.mvfr0 = 0x10110222;
@@ -1008,6 +1013,7 @@ static void aarch64_a64fx_initfn(Object *obj)
     cpu->gic_num_lrs = 4;
     cpu->gic_vpribits = 5;
     cpu->gic_vprebits = 5;
+    cpu->gic_pribits = 5;
 
     /* Suppport of A64FX's vector length are 128,256 and 512bit only */
     aarch64_add_sve_properties(obj);

From 5d55f827677a521feaab6dc651168e6136954e88 Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Thu, 12 May 2022 16:14:57 +0100
Subject: [PATCH 443/935] hw/intc/arm_gicv3: Provide ich_num_aprs()

We previously open-coded the expression for the number of virtual APR
registers and the assertion that it was not going to cause us to
overflow the cs->ich_apr[] array.  Factor this out into a new
ich_num_aprs() function, for consistency with the icc_num_aprs()
function we just added for the physical APR handling.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220512151457.3899052-7-peter.maydell@linaro.org
Message-id: 20220506162129.2896966-6-peter.maydell@linaro.org
---
 hw/intc/arm_gicv3_cpuif.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/hw/intc/arm_gicv3_cpuif.c b/hw/intc/arm_gicv3_cpuif.c
index 66e06b787c..8867e2e496 100644
--- a/hw/intc/arm_gicv3_cpuif.c
+++ b/hw/intc/arm_gicv3_cpuif.c
@@ -49,6 +49,14 @@ static inline int icv_min_vbpr(GICv3CPUState *cs)
     return 7 - cs->vprebits;
 }
 
+static inline int ich_num_aprs(GICv3CPUState *cs)
+{
+    /* Return the number of virtual APR registers (1, 2, or 4) */
+    int aprmax = 1 << (cs->vprebits - 5);
+    assert(aprmax <= ARRAY_SIZE(cs->ich_apr[0]));
+    return aprmax;
+}
+
 /* Simple accessor functions for LR fields */
 static uint32_t ich_lr_vintid(uint64_t lr)
 {
@@ -145,9 +153,7 @@ static int ich_highest_active_virt_prio(GICv3CPUState *cs)
      * in the ICH Active Priority Registers.
      */
     int i;
-    int aprmax = 1 << (cs->vprebits - 5);
-
-    assert(aprmax <= ARRAY_SIZE(cs->ich_apr[0]));
+    int aprmax = ich_num_aprs(cs);
 
     for (i = 0; i < aprmax; i++) {
         uint32_t apr = cs->ich_apr[GICV3_G0][i] |
@@ -1333,9 +1339,7 @@ static int icv_drop_prio(GICv3CPUState *cs)
      * 32 bits are actually relevant.
      */
     int i;
-    int aprmax = 1 << (cs->vprebits - 5);
-
-    assert(aprmax <= ARRAY_SIZE(cs->ich_apr[0]));
+    int aprmax = ich_num_aprs(cs);
 
     for (i = 0; i < aprmax; i++) {
         uint64_t *papr0 = &cs->ich_apr[GICV3_G0][i];

From e1be11a5a440bfa552e0a8bb109d72294054e3f0 Mon Sep 17 00:00:00 2001
From: Chris Howard <cvz185@web.de>
Date: Thu, 12 May 2022 11:42:02 +0200
Subject: [PATCH 444/935] Fix aarch64 debug register names.

Give all the debug registers their correct names including the
index, rather than having multiple registers all with the
same name string, which is confusing when viewed over the
gdbstub interface.

Signed-off-by: CHRIS HOWARD <cvz185@web.de>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 4127D8CA-D54A-47C7-A039-0DB7361E30C0@web.de
[PMM: expanded commit message]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 073d6509c8..91f78c91ce 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -6554,14 +6554,16 @@ static void define_debug_regs(ARMCPU *cpu)
     }
 
     for (i = 0; i < brps; i++) {
+        char *dbgbvr_el1_name = g_strdup_printf("DBGBVR%d_EL1", i);
+        char *dbgbcr_el1_name = g_strdup_printf("DBGBCR%d_EL1", i);
         ARMCPRegInfo dbgregs[] = {
-            { .name = "DBGBVR", .state = ARM_CP_STATE_BOTH,
+            { .name = dbgbvr_el1_name, .state = ARM_CP_STATE_BOTH,
               .cp = 14, .opc0 = 2, .opc1 = 0, .crn = 0, .crm = i, .opc2 = 4,
               .access = PL1_RW, .accessfn = access_tda,
               .fieldoffset = offsetof(CPUARMState, cp15.dbgbvr[i]),
               .writefn = dbgbvr_write, .raw_writefn = raw_write
             },
-            { .name = "DBGBCR", .state = ARM_CP_STATE_BOTH,
+            { .name = dbgbcr_el1_name, .state = ARM_CP_STATE_BOTH,
               .cp = 14, .opc0 = 2, .opc1 = 0, .crn = 0, .crm = i, .opc2 = 5,
               .access = PL1_RW, .accessfn = access_tda,
               .fieldoffset = offsetof(CPUARMState, cp15.dbgbcr[i]),
@@ -6569,17 +6571,21 @@ static void define_debug_regs(ARMCPU *cpu)
             },
         };
         define_arm_cp_regs(cpu, dbgregs);
+        g_free(dbgbvr_el1_name);
+        g_free(dbgbcr_el1_name);
     }
 
     for (i = 0; i < wrps; i++) {
+        char *dbgwvr_el1_name = g_strdup_printf("DBGWVR%d_EL1", i);
+        char *dbgwcr_el1_name = g_strdup_printf("DBGWCR%d_EL1", i);
         ARMCPRegInfo dbgregs[] = {
-            { .name = "DBGWVR", .state = ARM_CP_STATE_BOTH,
+            { .name = dbgwvr_el1_name, .state = ARM_CP_STATE_BOTH,
               .cp = 14, .opc0 = 2, .opc1 = 0, .crn = 0, .crm = i, .opc2 = 6,
               .access = PL1_RW, .accessfn = access_tda,
               .fieldoffset = offsetof(CPUARMState, cp15.dbgwvr[i]),
               .writefn = dbgwvr_write, .raw_writefn = raw_write
             },
-            { .name = "DBGWCR", .state = ARM_CP_STATE_BOTH,
+            { .name = dbgwcr_el1_name, .state = ARM_CP_STATE_BOTH,
               .cp = 14, .opc0 = 2, .opc1 = 0, .crn = 0, .crm = i, .opc2 = 7,
               .access = PL1_RW, .accessfn = access_tda,
               .fieldoffset = offsetof(CPUARMState, cp15.dbgwcr[i]),
@@ -6587,6 +6593,8 @@ static void define_debug_regs(ARMCPU *cpu)
             },
         };
         define_arm_cp_regs(cpu, dbgregs);
+        g_free(dbgwvr_el1_name);
+        g_free(dbgwcr_el1_name);
     }
 }
 

From 6e76d35f2375c3ef58aaaccbe5cee54b20a1f74a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
Date: Mon, 9 May 2022 22:20:35 +0200
Subject: [PATCH 445/935] hw/adc/zynq-xadc: Use qemu_irq typedef
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Except hw/core/irq.c which implements the forward-declared opaque
qemu_irq structure, hw/adc/zynq-xadc.{c,h} are the only files not
using the typedef. Fix this single exception.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Bernhard Beschow <shentey@gmail.com>
Message-id: 20220509202035.50335-1-philippe.mathieu.daude@gmail.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/adc/zynq-xadc.c         | 4 ++--
 include/hw/adc/zynq-xadc.h | 3 +--
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/hw/adc/zynq-xadc.c b/hw/adc/zynq-xadc.c
index cfc7bab065..032e19cbd0 100644
--- a/hw/adc/zynq-xadc.c
+++ b/hw/adc/zynq-xadc.c
@@ -86,7 +86,7 @@ static void zynq_xadc_update_ints(ZynqXADCState *s)
         s->regs[INT_STS] |= INT_DFIFO_GTH;
     }
 
-    qemu_set_irq(s->qemu_irq, !!(s->regs[INT_STS] & ~s->regs[INT_MASK]));
+    qemu_set_irq(s->irq, !!(s->regs[INT_STS] & ~s->regs[INT_MASK]));
 }
 
 static void zynq_xadc_reset(DeviceState *d)
@@ -262,7 +262,7 @@ static void zynq_xadc_init(Object *obj)
     memory_region_init_io(&s->iomem, obj, &xadc_ops, s, "zynq-xadc",
                           ZYNQ_XADC_MMIO_SIZE);
     sysbus_init_mmio(sbd, &s->iomem);
-    sysbus_init_irq(sbd, &s->qemu_irq);
+    sysbus_init_irq(sbd, &s->irq);
 }
 
 static const VMStateDescription vmstate_zynq_xadc = {
diff --git a/include/hw/adc/zynq-xadc.h b/include/hw/adc/zynq-xadc.h
index 2017b7a803..c10cc4c379 100644
--- a/include/hw/adc/zynq-xadc.h
+++ b/include/hw/adc/zynq-xadc.h
@@ -39,8 +39,7 @@ struct ZynqXADCState {
     uint16_t xadc_dfifo[ZYNQ_XADC_FIFO_DEPTH];
     uint16_t xadc_dfifo_entries;
 
-    struct IRQState *qemu_irq;
-
+    qemu_irq irq;
 };
 
 #endif /* ZYNQ_XADC_H */

From 1a13efcc56f0a50149da25ecca1f81f346496f86 Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Fri, 13 May 2022 14:18:01 +0100
Subject: [PATCH 446/935] target/arm/helper.c: Delete stray obsolete comment
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

In commit 88ce6c6ee85d we switched from directly fishing the number
of breakpoints and watchpoints out of the ID register fields to
abstracting out functions to do this job, but we forgot to delete the
now-obsolete comment in define_debug_regs() about the relation
between the ID field value and the actual number of breakpoints and
watchpoints.  Delete the obsolete comment.

Reported-by: CHRIS HOWARD <cvz185@web.de>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220513131801.4082712-1-peter.maydell@linaro.org
---
 target/arm/helper.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 91f78c91ce..d4db21dc92 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -6540,7 +6540,6 @@ static void define_debug_regs(ARMCPU *cpu)
         define_one_arm_cp_reg(cpu, &dbgdidr);
     }
 
-    /* Note that all these register fields hold "number of Xs minus 1". */
     brps = arm_num_brps(cpu);
     wrps = arm_num_wrps(cpu);
     ctx_cmps = arm_num_ctx_cmps(cpu);

From 24526bb92f69a323a9339dd94e823ad1d680f483 Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Fri, 13 May 2022 13:28:52 +0100
Subject: [PATCH 447/935] target/arm: Make number of counters in PMCR follow
 the CPU

Currently we give all the v7-and-up CPUs a PMU with 4 counters.  This
means that we don't provide the 6 counters that are required by the
Arm BSA (Base System Architecture) specification if the CPU supports
the Virtualization extensions.

Instead of having a single PMCR_NUM_COUNTERS, make each CPU type
specify the PMCR reset value (obtained from the appropriate TRM), and
use the 'N' field of that value to define the number of counters
provided.

This means that we now supply 6 counters instead of 4 for:
 Cortex-A9, Cortex-A15, Cortex-A53, Cortex-A57, Cortex-A72,
 Cortex-A76, Neoverse-N1, '-cpu max'
This CPU goes from 4 to 8 counters:
 A64FX
These CPUs remain with 4 counters:
 Cortex-A7, Cortex-A8
This CPU goes down from 4 to 3 counters:
 Cortex-R5

Note that because we now use the PMCR reset value of the specific
implementation, we no longer set the LC bit out of reset.  This has
an UNKNOWN value out of reset for all cores with any AArch32 support,
so guest software should be setting it anyway if it wants it.

This change was originally landed in commit f7fb73b8cdd3f7 (during
the 6.0 release cycle) but was then reverted by commit
21c2dd77a6aa517 before that release because it did not work with KVM.
This version fixes that by creating the scratch vCPU in
kvm_arm_get_host_cpu_features() with the KVM_ARM_VCPU_PMU_V3 feature
if KVM supports it, and then only asking KVM for the PMCR_EL0 value
if the vCPU has a PMU.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
[PMM: Added the correct value for a64fx]
Message-id: 20220513122852.4063586-1-peter.maydell@linaro.org
---
 target/arm/cpu.h       |  1 +
 target/arm/cpu64.c     | 11 +++++++++++
 target/arm/cpu_tcg.c   |  6 ++++++
 target/arm/helper.c    | 25 ++++++++++++++-----------
 target/arm/internals.h |  4 +++-
 target/arm/kvm64.c     | 12 ++++++++++++
 6 files changed, 47 insertions(+), 12 deletions(-)

diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index a42464eb57..3dc79f121b 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -965,6 +965,7 @@ struct ArchCPU {
         uint64_t id_aa64dfr0;
         uint64_t id_aa64dfr1;
         uint64_t id_aa64zfr0;
+        uint64_t reset_pmcr_el0;
     } isar;
     uint64_t midr;
     uint32_t revidr;
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 7628f4fa39..a752b64856 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -79,6 +79,7 @@ static void aarch64_a57_initfn(Object *obj)
     cpu->isar.id_aa64isar0 = 0x00011120;
     cpu->isar.id_aa64mmfr0 = 0x00001124;
     cpu->isar.dbgdidr = 0x3516d000;
+    cpu->isar.reset_pmcr_el0 = 0x41013000;
     cpu->clidr = 0x0a200023;
     cpu->ccsidr[0] = 0x701fe00a; /* 32KB L1 dcache */
     cpu->ccsidr[1] = 0x201fe012; /* 48KB L1 icache */
@@ -133,6 +134,7 @@ static void aarch64_a53_initfn(Object *obj)
     cpu->isar.id_aa64isar0 = 0x00011120;
     cpu->isar.id_aa64mmfr0 = 0x00001122; /* 40 bit physical addr */
     cpu->isar.dbgdidr = 0x3516d000;
+    cpu->isar.reset_pmcr_el0 = 0x41033000;
     cpu->clidr = 0x0a200023;
     cpu->ccsidr[0] = 0x700fe01a; /* 32KB L1 dcache */
     cpu->ccsidr[1] = 0x201fe00a; /* 32KB L1 icache */
@@ -185,6 +187,7 @@ static void aarch64_a72_initfn(Object *obj)
     cpu->isar.id_aa64isar0 = 0x00011120;
     cpu->isar.id_aa64mmfr0 = 0x00001124;
     cpu->isar.dbgdidr = 0x3516d000;
+    cpu->isar.reset_pmcr_el0 = 0x41023000;
     cpu->clidr = 0x0a200023;
     cpu->ccsidr[0] = 0x701fe00a; /* 32KB L1 dcache */
     cpu->ccsidr[1] = 0x201fe012; /* 48KB L1 icache */
@@ -261,6 +264,9 @@ static void aarch64_a76_initfn(Object *obj)
     cpu->isar.mvfr0 = 0x10110222;
     cpu->isar.mvfr1 = 0x13211111;
     cpu->isar.mvfr2 = 0x00000043;
+
+    /* From D5.1 AArch64 PMU register summary */
+    cpu->isar.reset_pmcr_el0 = 0x410b3000;
 }
 
 static void aarch64_neoverse_n1_initfn(Object *obj)
@@ -327,6 +333,9 @@ static void aarch64_neoverse_n1_initfn(Object *obj)
     cpu->isar.mvfr0 = 0x10110222;
     cpu->isar.mvfr1 = 0x13211111;
     cpu->isar.mvfr2 = 0x00000043;
+
+    /* From D5.1 AArch64 PMU register summary */
+    cpu->isar.reset_pmcr_el0 = 0x410c3000;
 }
 
 void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp)
@@ -1022,6 +1031,8 @@ static void aarch64_a64fx_initfn(Object *obj)
     set_bit(1, cpu->sve_vq_supported); /* 256bit */
     set_bit(3, cpu->sve_vq_supported); /* 512bit */
 
+    cpu->isar.reset_pmcr_el0 = 0x46014040;
+
     /* TODO:  Add A64FX specific HPC extension registers */
 }
 
diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
index ea4eccddc3..b751a19c8a 100644
--- a/target/arm/cpu_tcg.c
+++ b/target/arm/cpu_tcg.c
@@ -425,6 +425,7 @@ static void cortex_a8_initfn(Object *obj)
     cpu->ccsidr[1] = 0x2007e01a; /* 16k L1 icache. */
     cpu->ccsidr[2] = 0xf0000000; /* No L2 icache. */
     cpu->reset_auxcr = 2;
+    cpu->isar.reset_pmcr_el0 = 0x41002000;
     define_arm_cp_regs(cpu, cortexa8_cp_reginfo);
 }
 
@@ -496,6 +497,7 @@ static void cortex_a9_initfn(Object *obj)
     cpu->clidr = (1 << 27) | (1 << 24) | 3;
     cpu->ccsidr[0] = 0xe00fe019; /* 16k L1 dcache. */
     cpu->ccsidr[1] = 0x200fe019; /* 16k L1 icache. */
+    cpu->isar.reset_pmcr_el0 = 0x41093000;
     define_arm_cp_regs(cpu, cortexa9_cp_reginfo);
 }
 
@@ -565,6 +567,7 @@ static void cortex_a7_initfn(Object *obj)
     cpu->ccsidr[0] = 0x701fe00a; /* 32K L1 dcache */
     cpu->ccsidr[1] = 0x201fe00a; /* 32K L1 icache */
     cpu->ccsidr[2] = 0x711fe07a; /* 4096K L2 unified cache */
+    cpu->isar.reset_pmcr_el0 = 0x41072000;
     define_arm_cp_regs(cpu, cortexa15_cp_reginfo); /* Same as A15 */
 }
 
@@ -607,6 +610,7 @@ static void cortex_a15_initfn(Object *obj)
     cpu->ccsidr[0] = 0x701fe00a; /* 32K L1 dcache */
     cpu->ccsidr[1] = 0x201fe00a; /* 32K L1 icache */
     cpu->ccsidr[2] = 0x711fe07a; /* 4096K L2 unified cache */
+    cpu->isar.reset_pmcr_el0 = 0x410F3000;
     define_arm_cp_regs(cpu, cortexa15_cp_reginfo);
 }
 
@@ -835,6 +839,7 @@ static void cortex_r5_initfn(Object *obj)
     cpu->isar.id_isar6 = 0x0;
     cpu->mp_is_up = true;
     cpu->pmsav7_dregion = 16;
+    cpu->isar.reset_pmcr_el0 = 0x41151800;
     define_arm_cp_regs(cpu, cortexr5_cp_reginfo);
 }
 
@@ -1093,6 +1098,7 @@ static void arm_max_initfn(Object *obj)
     cpu->isar.id_isar5 = 0x00011121;
     cpu->isar.id_isar6 = 0;
     cpu->isar.dbgdidr = 0x3516d000;
+    cpu->isar.reset_pmcr_el0 = 0x41013000;
     cpu->clidr = 0x0a200023;
     cpu->ccsidr[0] = 0x701fe00a; /* 32KB L1 dcache */
     cpu->ccsidr[1] = 0x201fe012; /* 48KB L1 icache */
diff --git a/target/arm/helper.c b/target/arm/helper.c
index d4db21dc92..aa7a8e0572 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -39,7 +39,6 @@
 #include "cpregs.h"
 
 #define ARM_CPU_FREQ 1000000000 /* FIXME: 1 GHz, should be configurable */
-#define PMCR_NUM_COUNTERS 4 /* QEMU IMPDEF choice */
 
 #ifndef CONFIG_USER_ONLY
 
@@ -5544,13 +5543,6 @@ static const ARMCPRegInfo el2_cp_reginfo[] = {
       .resetvalue = 0,
       .writefn = gt_hyp_ctl_write, .raw_writefn = raw_write },
 #endif
-    /* The only field of MDCR_EL2 that has a defined architectural reset value
-     * is MDCR_EL2.HPMN which should reset to the value of PMCR_EL0.N.
-     */
-    { .name = "MDCR_EL2", .state = ARM_CP_STATE_BOTH,
-      .opc0 = 3, .opc1 = 4, .crn = 1, .crm = 1, .opc2 = 1,
-      .access = PL2_RW, .resetvalue = PMCR_NUM_COUNTERS,
-      .fieldoffset = offsetof(CPUARMState, cp15.mdcr_el2), },
     { .name = "HPFAR", .state = ARM_CP_STATE_AA32,
       .cp = 15, .opc1 = 4, .crn = 6, .crm = 0, .opc2 = 4,
       .access = PL2_RW, .accessfn = access_el3_aa32ns,
@@ -6604,7 +6596,7 @@ static void define_pmu_regs(ARMCPU *cpu)
      * field as main ID register, and we implement four counters in
      * addition to the cycle count register.
      */
-    unsigned int i, pmcrn = PMCR_NUM_COUNTERS;
+    unsigned int i, pmcrn = pmu_num_counters(&cpu->env);
     ARMCPRegInfo pmcr = {
         .name = "PMCR", .cp = 15, .crn = 9, .crm = 12, .opc1 = 0, .opc2 = 0,
         .access = PL0_RW,
@@ -6619,10 +6611,10 @@ static void define_pmu_regs(ARMCPU *cpu)
         .access = PL0_RW, .accessfn = pmreg_access,
         .type = ARM_CP_IO,
         .fieldoffset = offsetof(CPUARMState, cp15.c9_pmcr),
-        .resetvalue = (cpu->midr & 0xff000000) | (pmcrn << PMCRN_SHIFT) |
-                      PMCRLC,
+        .resetvalue = cpu->isar.reset_pmcr_el0,
         .writefn = pmcr_write, .raw_writefn = raw_write,
     };
+
     define_one_arm_cp_reg(cpu, &pmcr);
     define_one_arm_cp_reg(cpu, &pmcr64);
     for (i = 0; i < pmcrn; i++) {
@@ -7979,6 +7971,17 @@ void register_cp_regs_for_features(ARMCPU *cpu)
               .type = ARM_CP_EL3_NO_EL2_C_NZ,
               .fieldoffset = offsetof(CPUARMState, cp15.vmpidr_el2) },
         };
+        /*
+         * The only field of MDCR_EL2 that has a defined architectural reset
+         * value is MDCR_EL2.HPMN which should reset to the value of PMCR_EL0.N.
+         */
+        ARMCPRegInfo mdcr_el2 = {
+            .name = "MDCR_EL2", .state = ARM_CP_STATE_BOTH,
+            .opc0 = 3, .opc1 = 4, .crn = 1, .crm = 1, .opc2 = 1,
+            .access = PL2_RW, .resetvalue = pmu_num_counters(env),
+            .fieldoffset = offsetof(CPUARMState, cp15.mdcr_el2),
+        };
+        define_one_arm_cp_reg(cpu, &mdcr_el2);
         define_arm_cp_regs(cpu, vpidr_regs);
         define_arm_cp_regs(cpu, el2_cp_reginfo);
         if (arm_feature(env, ARM_FEATURE_V8)) {
diff --git a/target/arm/internals.h b/target/arm/internals.h
index 9b354eea7e..b654bee468 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -1304,7 +1304,9 @@ enum MVEECIState {
 
 static inline uint32_t pmu_num_counters(CPUARMState *env)
 {
-  return (env->cp15.c9_pmcr & PMCRN_MASK) >> PMCRN_SHIFT;
+    ARMCPU *cpu = env_archcpu(env);
+
+    return (cpu->isar.reset_pmcr_el0 & PMCRN_MASK) >> PMCRN_SHIFT;
 }
 
 /* Bits allowed to be set/cleared for PMCNTEN* and PMINTEN* */
diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c
index b8cfaf5782..363032da90 100644
--- a/target/arm/kvm64.c
+++ b/target/arm/kvm64.c
@@ -505,6 +505,7 @@ bool kvm_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
      */
     int fdarray[3];
     bool sve_supported;
+    bool pmu_supported = false;
     uint64_t features = 0;
     uint64_t t;
     int err;
@@ -537,6 +538,11 @@ bool kvm_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
                              1 << KVM_ARM_VCPU_PTRAUTH_GENERIC);
     }
 
+    if (kvm_arm_pmu_supported()) {
+        init.features[0] |= 1 << KVM_ARM_VCPU_PMU_V3;
+        pmu_supported = true;
+    }
+
     if (!kvm_arm_create_scratch_host_vcpu(cpus_to_try, fdarray, &init)) {
         return false;
     }
@@ -659,6 +665,12 @@ bool kvm_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
             dbgdidr |= (1 << 15); /* RES1 bit */
             ahcf->isar.dbgdidr = dbgdidr;
         }
+
+        if (pmu_supported) {
+            /* PMCR_EL0 is only accessible if the vCPU has feature PMU_V3 */
+            err |= read_sys_reg64(fdarray[2], &ahcf->isar.reset_pmcr_el0,
+                                  ARM64_SYS_REG(3, 3, 9, 12, 0));
+        }
     }
 
     sve_supported = ioctl(fdarray[0], KVM_CHECK_EXTENSION, KVM_CAP_ARM_SVE) > 0;

From e8ca920f3d1cedf7f2ebb41d13cd4d5946d3d882 Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Fri, 13 May 2022 14:13:15 +0100
Subject: [PATCH 448/935] hw/arm/virt: Fix incorrect non-secure flash dtb node
 name

In the virt board with secure=on we put two nodes in the dtb
for flash devices: one for the secure-only flash, and one
for the non-secure flash. We get the reg properties for these
correct, but in the DT node name, which by convention includes
the base address of devices, we used the wrong address. Fix it.

Spotted by dtc, which will complain
Warning (unique_unit_address): /flash@0: duplicate unit-address (also used in node /secflash@0)
if you dump the dtb from QEMU with -machine dumpdtb=file.dtb
and then decompile it with dtc.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220513131316.4081539-2-peter.maydell@linaro.org
---
 hw/arm/virt.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 1a45f44435..587e885a98 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1195,7 +1195,7 @@ static void virt_flash_fdt(VirtMachineState *vms,
         qemu_fdt_setprop_string(ms->fdt, nodename, "secure-status", "okay");
         g_free(nodename);
 
-        nodename = g_strdup_printf("/flash@%" PRIx64, flashbase);
+        nodename = g_strdup_printf("/flash@%" PRIx64, flashbase + flashsize);
         qemu_fdt_add_subnode(ms->fdt, nodename);
         qemu_fdt_setprop_string(ms->fdt, nodename, "compatible", "cfi-flash");
         qemu_fdt_setprop_sized_cells(ms->fdt, nodename, "reg",

From afdcbddcc92ef75ed1905e6ae7aa00db06e86dfc Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Fri, 13 May 2022 14:13:16 +0100
Subject: [PATCH 449/935] hw/arm/virt: Drop #size-cells and #address-cells from
 gpio-keys dtb node

The virt board generates a gpio-keys node in the dtb, but it
incorrectly gives this node #size-cells and #address-cells
properties. If you dump the dtb with 'machine dumpdtb=file.dtb'
and run it through dtc, dtc will warn about this:

Warning (avoid_unnecessary_addr_size): /gpio-keys: unnecessary #address-cells/#size-cells without "ranges" or child "reg" property

Remove the bogus properties.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220513131316.4081539-3-peter.maydell@linaro.org
---
 hw/arm/virt.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 587e885a98..097238faa7 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -925,8 +925,6 @@ static void create_gpio_keys(char *fdt, DeviceState *pl061_dev,
 
     qemu_fdt_add_subnode(fdt, "/gpio-keys");
     qemu_fdt_setprop_string(fdt, "/gpio-keys", "compatible", "gpio-keys");
-    qemu_fdt_setprop_cell(fdt, "/gpio-keys", "#size-cells", 0);
-    qemu_fdt_setprop_cell(fdt, "/gpio-keys", "#address-cells", 1);
 
     qemu_fdt_add_subnode(fdt, "/gpio-keys/poweroff");
     qemu_fdt_setprop_string(fdt, "/gpio-keys/poweroff",

From 9598c1bb39b2d4f0d3a55072cc70251c452132cd Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Mon, 16 May 2022 11:30:58 +0100
Subject: [PATCH 450/935] ptimer: Rename PTIMER_POLICY_DEFAULT to
 PTIMER_POLICY_LEGACY

The traditional ptimer behaviour includes a collection of weird edge
case behaviours.  In 2016 we improved the ptimer implementation to
fix these and generally make the behaviour more flexible, with
ptimers opting in to the new behaviour by passing an appropriate set
of policy flags to ptimer_init().  For backwards-compatibility, we
defined PTIMER_POLICY_DEFAULT (which sets no flags) to give the old
weird behaviour.

This turns out to be a poor choice of name, because people writing
new devices which use ptimers are misled into thinking that the
default is probably a sensible choice of flags, when in fact it is
almost always not what you want.  Rename PTIMER_POLICY_DEFAULT to
PTIMER_POLICY_LEGACY and beef up the comment to more clearly say that
new devices should not be using it.

The code-change part of this commit was produced by
  sed -i -e 's/PTIMER_POLICY_DEFAULT/PTIMER_POLICY_LEGACY/g' $(git grep -l PTIMER_POLICY_DEFAULT)
with the exception of a test name string change in
tests/unit/ptimer-test.c which was added manually.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Francisco Iglesias <francisco.iglesias@amd.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220516103058.162280-1-peter.maydell@linaro.org
---
 hw/arm/musicpal.c            |  2 +-
 hw/dma/xilinx_axidma.c       |  2 +-
 hw/dma/xlnx_csu_dma.c        |  2 +-
 hw/m68k/mcf5206.c            |  2 +-
 hw/m68k/mcf5208.c            |  2 +-
 hw/net/can/xlnx-zynqmp-can.c |  2 +-
 hw/net/fsl_etsec/etsec.c     |  2 +-
 hw/net/lan9118.c             |  2 +-
 hw/rtc/exynos4210_rtc.c      |  4 ++--
 hw/timer/allwinner-a10-pit.c |  2 +-
 hw/timer/altera_timer.c      |  2 +-
 hw/timer/arm_timer.c         |  2 +-
 hw/timer/digic-timer.c       |  2 +-
 hw/timer/etraxfs_timer.c     |  6 +++---
 hw/timer/exynos4210_mct.c    |  6 +++---
 hw/timer/exynos4210_pwm.c    |  2 +-
 hw/timer/grlib_gptimer.c     |  2 +-
 hw/timer/imx_epit.c          |  4 ++--
 hw/timer/imx_gpt.c           |  2 +-
 hw/timer/mss-timer.c         |  2 +-
 hw/timer/sh_timer.c          |  2 +-
 hw/timer/slavio_timer.c      |  2 +-
 hw/timer/xilinx_timer.c      |  2 +-
 include/hw/ptimer.h          | 16 ++++++++++++----
 tests/unit/ptimer-test.c     |  6 +++---
 25 files changed, 44 insertions(+), 36 deletions(-)

diff --git a/hw/arm/musicpal.c b/hw/arm/musicpal.c
index 7c840fb428..b65c020115 100644
--- a/hw/arm/musicpal.c
+++ b/hw/arm/musicpal.c
@@ -464,7 +464,7 @@ static void mv88w8618_timer_init(SysBusDevice *dev, mv88w8618_timer_state *s,
     sysbus_init_irq(dev, &s->irq);
     s->freq = freq;
 
-    s->ptimer = ptimer_init(mv88w8618_timer_tick, s, PTIMER_POLICY_DEFAULT);
+    s->ptimer = ptimer_init(mv88w8618_timer_tick, s, PTIMER_POLICY_LEGACY);
 }
 
 static uint64_t mv88w8618_pit_read(void *opaque, hwaddr offset,
diff --git a/hw/dma/xilinx_axidma.c b/hw/dma/xilinx_axidma.c
index bc383f53cc..cbb8f0f169 100644
--- a/hw/dma/xilinx_axidma.c
+++ b/hw/dma/xilinx_axidma.c
@@ -552,7 +552,7 @@ static void xilinx_axidma_realize(DeviceState *dev, Error **errp)
 
         st->dma = s;
         st->nr = i;
-        st->ptimer = ptimer_init(timer_hit, st, PTIMER_POLICY_DEFAULT);
+        st->ptimer = ptimer_init(timer_hit, st, PTIMER_POLICY_LEGACY);
         ptimer_transaction_begin(st->ptimer);
         ptimer_set_freq(st->ptimer, s->freqhz);
         ptimer_transaction_commit(st->ptimer);
diff --git a/hw/dma/xlnx_csu_dma.c b/hw/dma/xlnx_csu_dma.c
index 60ada3286b..1ce52ea5a2 100644
--- a/hw/dma/xlnx_csu_dma.c
+++ b/hw/dma/xlnx_csu_dma.c
@@ -666,7 +666,7 @@ static void xlnx_csu_dma_realize(DeviceState *dev, Error **errp)
     sysbus_init_irq(SYS_BUS_DEVICE(dev), &s->irq);
 
     s->src_timer = ptimer_init(xlnx_csu_dma_src_timeout_hit,
-                               s, PTIMER_POLICY_DEFAULT);
+                               s, PTIMER_POLICY_LEGACY);
 
     s->attr = MEMTXATTRS_UNSPECIFIED;
 
diff --git a/hw/m68k/mcf5206.c b/hw/m68k/mcf5206.c
index 6d93d761a5..2ab1b4f059 100644
--- a/hw/m68k/mcf5206.c
+++ b/hw/m68k/mcf5206.c
@@ -152,7 +152,7 @@ static m5206_timer_state *m5206_timer_init(qemu_irq irq)
     m5206_timer_state *s;
 
     s = g_new0(m5206_timer_state, 1);
-    s->timer = ptimer_init(m5206_timer_trigger, s, PTIMER_POLICY_DEFAULT);
+    s->timer = ptimer_init(m5206_timer_trigger, s, PTIMER_POLICY_LEGACY);
     s->irq = irq;
     m5206_timer_reset(s);
     return s;
diff --git a/hw/m68k/mcf5208.c b/hw/m68k/mcf5208.c
index 655207e393..be1033f84f 100644
--- a/hw/m68k/mcf5208.c
+++ b/hw/m68k/mcf5208.c
@@ -197,7 +197,7 @@ static void mcf5208_sys_init(MemoryRegion *address_space, qemu_irq *pic)
     /* Timers.  */
     for (i = 0; i < 2; i++) {
         s = g_new0(m5208_timer_state, 1);
-        s->timer = ptimer_init(m5208_timer_trigger, s, PTIMER_POLICY_DEFAULT);
+        s->timer = ptimer_init(m5208_timer_trigger, s, PTIMER_POLICY_LEGACY);
         memory_region_init_io(&s->iomem, NULL, &m5208_timer_ops, s,
                               "m5208-timer", 0x00004000);
         memory_region_add_subregion(address_space, 0xfc080000 + 0x4000 * i,
diff --git a/hw/net/can/xlnx-zynqmp-can.c b/hw/net/can/xlnx-zynqmp-can.c
index 22bb8910fa..82ac48cee2 100644
--- a/hw/net/can/xlnx-zynqmp-can.c
+++ b/hw/net/can/xlnx-zynqmp-can.c
@@ -1079,7 +1079,7 @@ static void xlnx_zynqmp_can_realize(DeviceState *dev, Error **errp)
 
     /* Allocate a new timer. */
     s->can_timer = ptimer_init(xlnx_zynqmp_can_ptimer_cb, s,
-                               PTIMER_POLICY_DEFAULT);
+                               PTIMER_POLICY_LEGACY);
 
     ptimer_transaction_begin(s->can_timer);
 
diff --git a/hw/net/fsl_etsec/etsec.c b/hw/net/fsl_etsec/etsec.c
index 6d50c39543..4e6cc708de 100644
--- a/hw/net/fsl_etsec/etsec.c
+++ b/hw/net/fsl_etsec/etsec.c
@@ -393,7 +393,7 @@ static void etsec_realize(DeviceState *dev, Error **errp)
                               object_get_typename(OBJECT(dev)), dev->id, etsec);
     qemu_format_nic_info_str(qemu_get_queue(etsec->nic), etsec->conf.macaddr.a);
 
-    etsec->ptimer = ptimer_init(etsec_timer_hit, etsec, PTIMER_POLICY_DEFAULT);
+    etsec->ptimer = ptimer_init(etsec_timer_hit, etsec, PTIMER_POLICY_LEGACY);
     ptimer_transaction_begin(etsec->ptimer);
     ptimer_set_freq(etsec->ptimer, 100);
     ptimer_transaction_commit(etsec->ptimer);
diff --git a/hw/net/lan9118.c b/hw/net/lan9118.c
index 6aff424cbe..456ae38107 100644
--- a/hw/net/lan9118.c
+++ b/hw/net/lan9118.c
@@ -1363,7 +1363,7 @@ static void lan9118_realize(DeviceState *dev, Error **errp)
     s->pmt_ctrl = 1;
     s->txp = &s->tx_packet;
 
-    s->timer = ptimer_init(lan9118_tick, s, PTIMER_POLICY_DEFAULT);
+    s->timer = ptimer_init(lan9118_tick, s, PTIMER_POLICY_LEGACY);
     ptimer_transaction_begin(s->timer);
     ptimer_set_freq(s->timer, 10000);
     ptimer_set_limit(s->timer, 0xffff, 1);
diff --git a/hw/rtc/exynos4210_rtc.c b/hw/rtc/exynos4210_rtc.c
index ae67641de6..d1620c7a2a 100644
--- a/hw/rtc/exynos4210_rtc.c
+++ b/hw/rtc/exynos4210_rtc.c
@@ -564,14 +564,14 @@ static void exynos4210_rtc_init(Object *obj)
     Exynos4210RTCState *s = EXYNOS4210_RTC(obj);
     SysBusDevice *dev = SYS_BUS_DEVICE(obj);
 
-    s->ptimer = ptimer_init(exynos4210_rtc_tick, s, PTIMER_POLICY_DEFAULT);
+    s->ptimer = ptimer_init(exynos4210_rtc_tick, s, PTIMER_POLICY_LEGACY);
     ptimer_transaction_begin(s->ptimer);
     ptimer_set_freq(s->ptimer, RTC_BASE_FREQ);
     exynos4210_rtc_update_freq(s, 0);
     ptimer_transaction_commit(s->ptimer);
 
     s->ptimer_1Hz = ptimer_init(exynos4210_rtc_1Hz_tick,
-                                s, PTIMER_POLICY_DEFAULT);
+                                s, PTIMER_POLICY_LEGACY);
     ptimer_transaction_begin(s->ptimer_1Hz);
     ptimer_set_freq(s->ptimer_1Hz, RTC_BASE_FREQ);
     ptimer_transaction_commit(s->ptimer_1Hz);
diff --git a/hw/timer/allwinner-a10-pit.c b/hw/timer/allwinner-a10-pit.c
index c3fc2a4daa..971f78462a 100644
--- a/hw/timer/allwinner-a10-pit.c
+++ b/hw/timer/allwinner-a10-pit.c
@@ -275,7 +275,7 @@ static void a10_pit_init(Object *obj)
 
         tc->container = s;
         tc->index = i;
-        s->timer[i] = ptimer_init(a10_pit_timer_cb, tc, PTIMER_POLICY_DEFAULT);
+        s->timer[i] = ptimer_init(a10_pit_timer_cb, tc, PTIMER_POLICY_LEGACY);
     }
 }
 
diff --git a/hw/timer/altera_timer.c b/hw/timer/altera_timer.c
index c6e02d2b5a..0f1f54206a 100644
--- a/hw/timer/altera_timer.c
+++ b/hw/timer/altera_timer.c
@@ -185,7 +185,7 @@ static void altera_timer_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    t->ptimer = ptimer_init(timer_hit, t, PTIMER_POLICY_DEFAULT);
+    t->ptimer = ptimer_init(timer_hit, t, PTIMER_POLICY_LEGACY);
     ptimer_transaction_begin(t->ptimer);
     ptimer_set_freq(t->ptimer, t->freq_hz);
     ptimer_transaction_commit(t->ptimer);
diff --git a/hw/timer/arm_timer.c b/hw/timer/arm_timer.c
index 84cf2726bb..69c8863472 100644
--- a/hw/timer/arm_timer.c
+++ b/hw/timer/arm_timer.c
@@ -180,7 +180,7 @@ static arm_timer_state *arm_timer_init(uint32_t freq)
     s->freq = freq;
     s->control = TIMER_CTRL_IE;
 
-    s->timer = ptimer_init(arm_timer_tick, s, PTIMER_POLICY_DEFAULT);
+    s->timer = ptimer_init(arm_timer_tick, s, PTIMER_POLICY_LEGACY);
     vmstate_register(NULL, VMSTATE_INSTANCE_ID_ANY, &vmstate_arm_timer, s);
     return s;
 }
diff --git a/hw/timer/digic-timer.c b/hw/timer/digic-timer.c
index e3aae4a45a..d5186f4454 100644
--- a/hw/timer/digic-timer.c
+++ b/hw/timer/digic-timer.c
@@ -139,7 +139,7 @@ static void digic_timer_init(Object *obj)
 {
     DigicTimerState *s = DIGIC_TIMER(obj);
 
-    s->ptimer = ptimer_init(digic_timer_tick, NULL, PTIMER_POLICY_DEFAULT);
+    s->ptimer = ptimer_init(digic_timer_tick, NULL, PTIMER_POLICY_LEGACY);
 
     /*
      * FIXME: there is no documentation on Digic timer
diff --git a/hw/timer/etraxfs_timer.c b/hw/timer/etraxfs_timer.c
index 139e5b86a4..ecc2831baf 100644
--- a/hw/timer/etraxfs_timer.c
+++ b/hw/timer/etraxfs_timer.c
@@ -370,9 +370,9 @@ static void etraxfs_timer_realize(DeviceState *dev, Error **errp)
     ETRAXTimerState *t = ETRAX_TIMER(dev);
     SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
 
-    t->ptimer_t0 = ptimer_init(timer0_hit, t, PTIMER_POLICY_DEFAULT);
-    t->ptimer_t1 = ptimer_init(timer1_hit, t, PTIMER_POLICY_DEFAULT);
-    t->ptimer_wd = ptimer_init(watchdog_hit, t, PTIMER_POLICY_DEFAULT);
+    t->ptimer_t0 = ptimer_init(timer0_hit, t, PTIMER_POLICY_LEGACY);
+    t->ptimer_t1 = ptimer_init(timer1_hit, t, PTIMER_POLICY_LEGACY);
+    t->ptimer_wd = ptimer_init(watchdog_hit, t, PTIMER_POLICY_LEGACY);
 
     sysbus_init_irq(sbd, &t->irq);
     sysbus_init_irq(sbd, &t->nmi);
diff --git a/hw/timer/exynos4210_mct.c b/hw/timer/exynos4210_mct.c
index d0e5343996..e175a9f5b9 100644
--- a/hw/timer/exynos4210_mct.c
+++ b/hw/timer/exynos4210_mct.c
@@ -1503,17 +1503,17 @@ static void exynos4210_mct_init(Object *obj)
 
     /* Global timer */
     s->g_timer.ptimer_frc = ptimer_init(exynos4210_gfrc_event, s,
-                                        PTIMER_POLICY_DEFAULT);
+                                        PTIMER_POLICY_LEGACY);
     memset(&s->g_timer.reg, 0, sizeof(struct gregs));
 
     /* Local timers */
     for (i = 0; i < 2; i++) {
         s->l_timer[i].tick_timer.ptimer_tick =
             ptimer_init(exynos4210_ltick_event, &s->l_timer[i],
-                        PTIMER_POLICY_DEFAULT);
+                        PTIMER_POLICY_LEGACY);
         s->l_timer[i].ptimer_frc =
             ptimer_init(exynos4210_lfrc_event, &s->l_timer[i],
-                        PTIMER_POLICY_DEFAULT);
+                        PTIMER_POLICY_LEGACY);
         s->l_timer[i].id = i;
     }
 
diff --git a/hw/timer/exynos4210_pwm.c b/hw/timer/exynos4210_pwm.c
index 220088120e..02924a9e5b 100644
--- a/hw/timer/exynos4210_pwm.c
+++ b/hw/timer/exynos4210_pwm.c
@@ -400,7 +400,7 @@ static void exynos4210_pwm_init(Object *obj)
         sysbus_init_irq(dev, &s->timer[i].irq);
         s->timer[i].ptimer = ptimer_init(exynos4210_pwm_tick,
                                          &s->timer[i],
-                                         PTIMER_POLICY_DEFAULT);
+                                         PTIMER_POLICY_LEGACY);
         s->timer[i].id = i;
         s->timer[i].parent = s;
     }
diff --git a/hw/timer/grlib_gptimer.c b/hw/timer/grlib_gptimer.c
index d511890109..5c4923c1e0 100644
--- a/hw/timer/grlib_gptimer.c
+++ b/hw/timer/grlib_gptimer.c
@@ -383,7 +383,7 @@ static void grlib_gptimer_realize(DeviceState *dev, Error **errp)
 
         timer->unit   = unit;
         timer->ptimer = ptimer_init(grlib_gptimer_hit, timer,
-                                    PTIMER_POLICY_DEFAULT);
+                                    PTIMER_POLICY_LEGACY);
         timer->id     = i;
 
         /* One IRQ line for each timer */
diff --git a/hw/timer/imx_epit.c b/hw/timer/imx_epit.c
index ebd58254d1..2bf8c754b2 100644
--- a/hw/timer/imx_epit.c
+++ b/hw/timer/imx_epit.c
@@ -347,9 +347,9 @@ static void imx_epit_realize(DeviceState *dev, Error **errp)
                           0x00001000);
     sysbus_init_mmio(sbd, &s->iomem);
 
-    s->timer_reload = ptimer_init(imx_epit_reload, s, PTIMER_POLICY_DEFAULT);
+    s->timer_reload = ptimer_init(imx_epit_reload, s, PTIMER_POLICY_LEGACY);
 
-    s->timer_cmp = ptimer_init(imx_epit_cmp, s, PTIMER_POLICY_DEFAULT);
+    s->timer_cmp = ptimer_init(imx_epit_cmp, s, PTIMER_POLICY_LEGACY);
 }
 
 static void imx_epit_class_init(ObjectClass *klass, void *data)
diff --git a/hw/timer/imx_gpt.c b/hw/timer/imx_gpt.c
index 5c0d9a269c..80b8302639 100644
--- a/hw/timer/imx_gpt.c
+++ b/hw/timer/imx_gpt.c
@@ -505,7 +505,7 @@ static void imx_gpt_realize(DeviceState *dev, Error **errp)
                           0x00001000);
     sysbus_init_mmio(sbd, &s->iomem);
 
-    s->timer = ptimer_init(imx_gpt_timeout, s, PTIMER_POLICY_DEFAULT);
+    s->timer = ptimer_init(imx_gpt_timeout, s, PTIMER_POLICY_LEGACY);
 }
 
 static void imx_gpt_class_init(ObjectClass *klass, void *data)
diff --git a/hw/timer/mss-timer.c b/hw/timer/mss-timer.c
index fe0ca905f3..ee7438f168 100644
--- a/hw/timer/mss-timer.c
+++ b/hw/timer/mss-timer.c
@@ -232,7 +232,7 @@ static void mss_timer_init(Object *obj)
     for (i = 0; i < NUM_TIMERS; i++) {
         struct Msf2Timer *st = &t->timers[i];
 
-        st->ptimer = ptimer_init(timer_hit, st, PTIMER_POLICY_DEFAULT);
+        st->ptimer = ptimer_init(timer_hit, st, PTIMER_POLICY_LEGACY);
         ptimer_transaction_begin(st->ptimer);
         ptimer_set_freq(st->ptimer, t->freq_hz);
         ptimer_transaction_commit(st->ptimer);
diff --git a/hw/timer/sh_timer.c b/hw/timer/sh_timer.c
index c72c327bfa..7788939766 100644
--- a/hw/timer/sh_timer.c
+++ b/hw/timer/sh_timer.c
@@ -239,7 +239,7 @@ static void *sh_timer_init(uint32_t freq, int feat, qemu_irq irq)
     s->enabled = 0;
     s->irq = irq;
 
-    s->timer = ptimer_init(sh_timer_tick, s, PTIMER_POLICY_DEFAULT);
+    s->timer = ptimer_init(sh_timer_tick, s, PTIMER_POLICY_LEGACY);
 
     sh_timer_write(s, OFFSET_TCOR >> 2, s->tcor);
     sh_timer_write(s, OFFSET_TCNT >> 2, s->tcnt);
diff --git a/hw/timer/slavio_timer.c b/hw/timer/slavio_timer.c
index 90fdce4c44..8c4f6eb06b 100644
--- a/hw/timer/slavio_timer.c
+++ b/hw/timer/slavio_timer.c
@@ -405,7 +405,7 @@ static void slavio_timer_init(Object *obj)
         tc->timer_index = i;
 
         s->cputimer[i].timer = ptimer_init(slavio_timer_irq, tc,
-                                           PTIMER_POLICY_DEFAULT);
+                                           PTIMER_POLICY_LEGACY);
         ptimer_transaction_begin(s->cputimer[i].timer);
         ptimer_set_period(s->cputimer[i].timer, TIMER_PERIOD);
         ptimer_transaction_commit(s->cputimer[i].timer);
diff --git a/hw/timer/xilinx_timer.c b/hw/timer/xilinx_timer.c
index 1eb927eb84..c7f17cd646 100644
--- a/hw/timer/xilinx_timer.c
+++ b/hw/timer/xilinx_timer.c
@@ -223,7 +223,7 @@ static void xilinx_timer_realize(DeviceState *dev, Error **errp)
 
         xt->parent = t;
         xt->nr = i;
-        xt->ptimer = ptimer_init(timer_hit, xt, PTIMER_POLICY_DEFAULT);
+        xt->ptimer = ptimer_init(timer_hit, xt, PTIMER_POLICY_LEGACY);
         ptimer_transaction_begin(xt->ptimer);
         ptimer_set_freq(xt->ptimer, t->freq_hz);
         ptimer_transaction_commit(xt->ptimer);
diff --git a/include/hw/ptimer.h b/include/hw/ptimer.h
index c443218475..4dc02b0de4 100644
--- a/include/hw/ptimer.h
+++ b/include/hw/ptimer.h
@@ -33,9 +33,17 @@
  * to stderr when the guest attempts to enable the timer.
  */
 
-/* The default ptimer policy retains backward compatibility with the legacy
- * timers. Custom policies are adjusting the default one. Consider providing
- * a correct policy for your timer.
+/*
+ * The 'legacy' ptimer policy retains backward compatibility with the
+ * traditional ptimer behaviour from before policy flags were introduced.
+ * It has several weird behaviours which don't match typical hardware
+ * timer behaviour. For a new device using ptimers, you should not
+ * use PTIMER_POLICY_LEGACY, but instead check the actual behaviour
+ * that you need and specify the right set of policy flags to get that.
+ *
+ * If you are overhauling an existing device that uses PTIMER_POLICY_LEGACY
+ * and are in a position to check or test the real hardware behaviour,
+ * consider updating it to specify the right policy flags.
  *
  * The rough edges of the default policy:
  *  - Starting to run with a period = 0 emits error message and stops the
@@ -54,7 +62,7 @@
  *    since the last period, effectively restarting the timer with a
  *    counter = counter value at the moment of change (.i.e. one less).
  */
-#define PTIMER_POLICY_DEFAULT               0
+#define PTIMER_POLICY_LEGACY                0
 
 /* Periodic timer counter stays with "0" for a one period before wrapping
  * around.  */
diff --git a/tests/unit/ptimer-test.c b/tests/unit/ptimer-test.c
index 9176b96c1c..a80ef5aff4 100644
--- a/tests/unit/ptimer-test.c
+++ b/tests/unit/ptimer-test.c
@@ -768,8 +768,8 @@ static void add_ptimer_tests(uint8_t policy)
     char policy_name[256] = "";
     char *tmp;
 
-    if (policy == PTIMER_POLICY_DEFAULT) {
-        g_sprintf(policy_name, "default");
+    if (policy == PTIMER_POLICY_LEGACY) {
+        g_sprintf(policy_name, "legacy");
     }
 
     if (policy & PTIMER_POLICY_WRAP_AFTER_ONE_PERIOD) {
@@ -862,7 +862,7 @@ static void add_ptimer_tests(uint8_t policy)
 static void add_all_ptimer_policies_comb_tests(void)
 {
     int last_policy = PTIMER_POLICY_TRIGGER_ONLY_ON_DECREMENT;
-    int policy = PTIMER_POLICY_DEFAULT;
+    int policy = PTIMER_POLICY_LEGACY;
 
     for (; policy < (last_policy << 1); policy++) {
         if ((policy & PTIMER_POLICY_TRIGGER_ONLY_ON_DECREMENT) &&

From 07b034ea828eb089de1a88e0043a8f3065f2d205 Mon Sep 17 00:00:00 2001
From: Florian Lugou <florian.lugou@provenrun.com>
Date: Tue, 17 May 2022 16:52:42 +0200
Subject: [PATCH 451/935] target/arm: Fix PAuth keys access checks for disabled
 SEL2

As per the description of the HCR_EL2.APK field in the ARMv8 ARM,
Pointer Authentication keys accesses should only be trapped to Secure
EL2 if it is enabled.

Signed-off-by: Florian Lugou <florian.lugou@provenrun.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220517145242.1215271-1-florian.lugou@provenrun.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index aa7a8e0572..fdd51e5e75 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -6768,7 +6768,7 @@ static CPAccessResult access_pauth(CPUARMState *env, const ARMCPRegInfo *ri,
     int el = arm_current_el(env);
 
     if (el < 2 &&
-        arm_feature(env, ARM_FEATURE_EL2) &&
+        arm_is_el2_enabled(env) &&
         !(arm_hcr_el2_eff(env) & HCR_APK)) {
         return CP_ACCESS_TRAP_EL2;
     }

From 5814d587fe861fe9cab651638959f6db843df54e Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Mon, 16 May 2022 22:48:44 -0700
Subject: [PATCH 452/935] target/arm: Enable FEAT_HCX for -cpu max

This feature adds a new register, HCRX_EL2, which controls
many of the newer AArch64 features.  So far the register is
effectively RES0, because none of the new features are done.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220517054850.177016-2-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu.h    | 20 ++++++++++++++++++
 target/arm/cpu64.c  |  1 +
 target/arm/helper.c | 50 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 71 insertions(+)

diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 3dc79f121b..fac526a490 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -362,6 +362,7 @@ typedef struct CPUArchState {
         uint32_t pmsav5_data_ap; /* PMSAv5 MPU data access permissions */
         uint32_t pmsav5_insn_ap; /* PMSAv5 MPU insn access permissions */
         uint64_t hcr_el2; /* Hypervisor configuration register */
+        uint64_t hcrx_el2; /* Extended Hypervisor configuration register */
         uint64_t scr_el3; /* Secure configuration register.  */
         union { /* Fault status registers.  */
             struct {
@@ -1545,6 +1546,19 @@ static inline void xpsr_write(CPUARMState *env, uint32_t val, uint32_t mask)
 #define HCR_TWEDEN    (1ULL << 59)
 #define HCR_TWEDEL    MAKE_64BIT_MASK(60, 4)
 
+#define HCRX_ENAS0    (1ULL << 0)
+#define HCRX_ENALS    (1ULL << 1)
+#define HCRX_ENASR    (1ULL << 2)
+#define HCRX_FNXS     (1ULL << 3)
+#define HCRX_FGTNXS   (1ULL << 4)
+#define HCRX_SMPME    (1ULL << 5)
+#define HCRX_TALLINT  (1ULL << 6)
+#define HCRX_VINMI    (1ULL << 7)
+#define HCRX_VFNMI    (1ULL << 8)
+#define HCRX_CMOW     (1ULL << 9)
+#define HCRX_MCE2     (1ULL << 10)
+#define HCRX_MSCEN    (1ULL << 11)
+
 #define HPFAR_NS      (1ULL << 63)
 
 #define SCR_NS                (1U << 0)
@@ -2312,6 +2326,7 @@ static inline bool arm_is_el2_enabled(CPUARMState *env)
  * Not included here is HCR_RW.
  */
 uint64_t arm_hcr_el2_eff(CPUARMState *env);
+uint64_t arm_hcrx_el2_eff(CPUARMState *env);
 
 /* Return true if the specified exception level is running in AArch64 state. */
 static inline bool arm_el_is_aa64(CPUARMState *env, int el)
@@ -3933,6 +3948,11 @@ static inline bool isar_feature_aa64_ats1e1(const ARMISARegisters *id)
     return FIELD_EX64(id->id_aa64mmfr1, ID_AA64MMFR1, PAN) >= 2;
 }
 
+static inline bool isar_feature_aa64_hcx(const ARMISARegisters *id)
+{
+    return FIELD_EX64(id->id_aa64mmfr1, ID_AA64MMFR1, HCX) != 0;
+}
+
 static inline bool isar_feature_aa64_uao(const ARMISARegisters *id)
 {
     return FIELD_EX64(id->id_aa64mmfr2, ID_AA64MMFR2, UAO) != 0;
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index a752b64856..3ff9219ca3 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -934,6 +934,7 @@ static void aarch64_max_initfn(Object *obj)
     t = FIELD_DP64(t, ID_AA64MMFR1, LO, 1);       /* FEAT_LOR */
     t = FIELD_DP64(t, ID_AA64MMFR1, PAN, 2);      /* FEAT_PAN2 */
     t = FIELD_DP64(t, ID_AA64MMFR1, XNX, 1);      /* FEAT_XNX */
+    t = FIELD_DP64(t, ID_AA64MMFR1, HCX, 1);      /* FEAT_HCX */
     cpu->isar.id_aa64mmfr1 = t;
 
     t = cpu->isar.id_aa64mmfr2;
diff --git a/target/arm/helper.c b/target/arm/helper.c
index fdd51e5e75..7d983d7fff 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -5288,6 +5288,52 @@ uint64_t arm_hcr_el2_eff(CPUARMState *env)
     return ret;
 }
 
+static void hcrx_write(CPUARMState *env, const ARMCPRegInfo *ri,
+                       uint64_t value)
+{
+    uint64_t valid_mask = 0;
+
+    /* No features adding bits to HCRX are implemented. */
+
+    /* Clear RES0 bits.  */
+    env->cp15.hcrx_el2 = value & valid_mask;
+}
+
+static CPAccessResult access_hxen(CPUARMState *env, const ARMCPRegInfo *ri,
+                                  bool isread)
+{
+    if (arm_current_el(env) < 3
+        && arm_feature(env, ARM_FEATURE_EL3)
+        && !(env->cp15.scr_el3 & SCR_HXEN)) {
+        return CP_ACCESS_TRAP_EL3;
+    }
+    return CP_ACCESS_OK;
+}
+
+static const ARMCPRegInfo hcrx_el2_reginfo = {
+    .name = "HCRX_EL2", .state = ARM_CP_STATE_AA64,
+    .opc0 = 3, .opc1 = 4, .crn = 1, .crm = 2, .opc2 = 2,
+    .access = PL2_RW, .writefn = hcrx_write, .accessfn = access_hxen,
+    .fieldoffset = offsetof(CPUARMState, cp15.hcrx_el2),
+};
+
+/* Return the effective value of HCRX_EL2.  */
+uint64_t arm_hcrx_el2_eff(CPUARMState *env)
+{
+    /*
+     * The bits in this register behave as 0 for all purposes other than
+     * direct reads of the register if:
+     *   - EL2 is not enabled in the current security state,
+     *   - SCR_EL3.HXEn is 0.
+     */
+    if (!arm_is_el2_enabled(env)
+        || (arm_feature(env, ARM_FEATURE_EL3)
+            && !(env->cp15.scr_el3 & SCR_HXEN))) {
+        return 0;
+    }
+    return env->cp15.hcrx_el2;
+}
+
 static void cptr_el2_write(CPUARMState *env, const ARMCPRegInfo *ri,
                            uint64_t value)
 {
@@ -8405,6 +8451,10 @@ void register_cp_regs_for_features(ARMCPU *cpu)
         define_arm_cp_regs(cpu, zcr_reginfo);
     }
 
+    if (cpu_isar_feature(aa64_hcx, cpu)) {
+        define_one_arm_cp_reg(cpu, &hcrx_el2_reginfo);
+    }
+
 #ifdef TARGET_AARCH64
     if (cpu_isar_feature(aa64_pauth, cpu)) {
         define_arm_cp_regs(cpu, pauth_reginfo);

From fab8ad39fb75a0d9f097db67b2a334444754e88e Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Mon, 16 May 2022 22:48:45 -0700
Subject: [PATCH 453/935] target/arm: Use FIELD definitions for CPACR, CPTR_ELx

We had a few CPTR_* bits defined, but missed quite a few.
Complete all of the fields up to ARMv9.2.
Use FIELD_EX64 instead of manual extract32.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220517054850.177016-3-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/arm/boot.c       |  2 +-
 target/arm/cpu.c    | 11 ++++++---
 target/arm/cpu.h    | 44 +++++++++++++++++++++++++++++++-----
 target/arm/helper.c | 54 ++++++++++++++++++++++-----------------------
 4 files changed, 75 insertions(+), 36 deletions(-)

diff --git a/hw/arm/boot.c b/hw/arm/boot.c
index a47f38dfc9..a8de33fd64 100644
--- a/hw/arm/boot.c
+++ b/hw/arm/boot.c
@@ -761,7 +761,7 @@ static void do_cpu_reset(void *opaque)
                         env->cp15.scr_el3 |= SCR_ATA;
                     }
                     if (cpu_isar_feature(aa64_sve, cpu)) {
-                        env->cp15.cptr_el[3] |= CPTR_EZ;
+                        env->cp15.cptr_el[3] |= R_CPTR_EL3_EZ_MASK;
                     }
                     /* AArch64 kernels never boot in secure mode */
                     assert(!info->secure_boot);
diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index 029f644768..d2bd74c2ed 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -201,9 +201,11 @@ static void arm_cpu_reset(DeviceState *dev)
         /* Trap on btype=3 for PACIxSP. */
         env->cp15.sctlr_el[1] |= SCTLR_BT0;
         /* and to the FP/Neon instructions */
-        env->cp15.cpacr_el1 = deposit64(env->cp15.cpacr_el1, 20, 2, 3);
+        env->cp15.cpacr_el1 = FIELD_DP64(env->cp15.cpacr_el1,
+                                         CPACR_EL1, FPEN, 3);
         /* and to the SVE instructions */
-        env->cp15.cpacr_el1 = deposit64(env->cp15.cpacr_el1, 16, 2, 3);
+        env->cp15.cpacr_el1 = FIELD_DP64(env->cp15.cpacr_el1,
+                                         CPACR_EL1, ZEN, 3);
         /* with reasonable vector length */
         if (cpu_isar_feature(aa64_sve, cpu)) {
             env->vfp.zcr_el[1] =
@@ -252,7 +254,10 @@ static void arm_cpu_reset(DeviceState *dev)
     } else {
 #if defined(CONFIG_USER_ONLY)
         /* Userspace expects access to cp10 and cp11 for FP/Neon */
-        env->cp15.cpacr_el1 = deposit64(env->cp15.cpacr_el1, 20, 4, 0xf);
+        env->cp15.cpacr_el1 = FIELD_DP64(env->cp15.cpacr_el1,
+                                         CPACR, CP10, 3);
+        env->cp15.cpacr_el1 = FIELD_DP64(env->cp15.cpacr_el1,
+                                         CPACR, CP11, 3);
 #endif
     }
 
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index fac526a490..c1865ad5da 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -1261,11 +1261,45 @@ void pmu_init(ARMCPU *cpu);
 #define SCTLR_SPINTMASK (1ULL << 62) /* FEAT_NMI */
 #define SCTLR_TIDCP   (1ULL << 63) /* FEAT_TIDCP1 */
 
-#define CPTR_TCPAC    (1U << 31)
-#define CPTR_TTA      (1U << 20)
-#define CPTR_TFP      (1U << 10)
-#define CPTR_TZ       (1U << 8)   /* CPTR_EL2 */
-#define CPTR_EZ       (1U << 8)   /* CPTR_EL3 */
+/* Bit definitions for CPACR (AArch32 only) */
+FIELD(CPACR, CP10, 20, 2)
+FIELD(CPACR, CP11, 22, 2)
+FIELD(CPACR, TRCDIS, 28, 1)    /* matches CPACR_EL1.TTA */
+FIELD(CPACR, D32DIS, 30, 1)    /* up to v7; RAZ in v8 */
+FIELD(CPACR, ASEDIS, 31, 1)
+
+/* Bit definitions for CPACR_EL1 (AArch64 only) */
+FIELD(CPACR_EL1, ZEN, 16, 2)
+FIELD(CPACR_EL1, FPEN, 20, 2)
+FIELD(CPACR_EL1, SMEN, 24, 2)
+FIELD(CPACR_EL1, TTA, 28, 1)   /* matches CPACR.TRCDIS */
+
+/* Bit definitions for HCPTR (AArch32 only) */
+FIELD(HCPTR, TCP10, 10, 1)
+FIELD(HCPTR, TCP11, 11, 1)
+FIELD(HCPTR, TASE, 15, 1)
+FIELD(HCPTR, TTA, 20, 1)
+FIELD(HCPTR, TAM, 30, 1)       /* matches CPTR_EL2.TAM */
+FIELD(HCPTR, TCPAC, 31, 1)     /* matches CPTR_EL2.TCPAC */
+
+/* Bit definitions for CPTR_EL2 (AArch64 only) */
+FIELD(CPTR_EL2, TZ, 8, 1)      /* !E2H */
+FIELD(CPTR_EL2, TFP, 10, 1)    /* !E2H, matches HCPTR.TCP10 */
+FIELD(CPTR_EL2, TSM, 12, 1)    /* !E2H */
+FIELD(CPTR_EL2, ZEN, 16, 2)    /* E2H */
+FIELD(CPTR_EL2, FPEN, 20, 2)   /* E2H */
+FIELD(CPTR_EL2, SMEN, 24, 2)   /* E2H */
+FIELD(CPTR_EL2, TTA, 28, 1)
+FIELD(CPTR_EL2, TAM, 30, 1)    /* matches HCPTR.TAM */
+FIELD(CPTR_EL2, TCPAC, 31, 1)  /* matches HCPTR.TCPAC */
+
+/* Bit definitions for CPTR_EL3 (AArch64 only) */
+FIELD(CPTR_EL3, EZ, 8, 1)
+FIELD(CPTR_EL3, TFP, 10, 1)
+FIELD(CPTR_EL3, ESM, 12, 1)
+FIELD(CPTR_EL3, TTA, 20, 1)
+FIELD(CPTR_EL3, TAM, 30, 1)
+FIELD(CPTR_EL3, TCPAC, 31, 1)
 
 #define MDCR_EPMAD    (1U << 21)
 #define MDCR_EDAD     (1U << 20)
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 7d983d7fff..40da63913c 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -766,11 +766,14 @@ static void cpacr_write(CPUARMState *env, const ARMCPRegInfo *ri,
          */
         if (cpu_isar_feature(aa32_vfp_simd, env_archcpu(env))) {
             /* VFP coprocessor: cp10 & cp11 [23:20] */
-            mask |= (1 << 31) | (1 << 30) | (0xf << 20);
+            mask |= R_CPACR_ASEDIS_MASK |
+                    R_CPACR_D32DIS_MASK |
+                    R_CPACR_CP11_MASK |
+                    R_CPACR_CP10_MASK;
 
             if (!arm_feature(env, ARM_FEATURE_NEON)) {
                 /* ASEDIS [31] bit is RAO/WI */
-                value |= (1 << 31);
+                value |= R_CPACR_ASEDIS_MASK;
             }
 
             /* VFPv3 and upwards with NEON implement 32 double precision
@@ -778,7 +781,7 @@ static void cpacr_write(CPUARMState *env, const ARMCPRegInfo *ri,
              */
             if (!cpu_isar_feature(aa32_simd_r32, env_archcpu(env))) {
                 /* D32DIS [30] is RAO/WI if D16-31 are not implemented. */
-                value |= (1 << 30);
+                value |= R_CPACR_D32DIS_MASK;
             }
         }
         value &= mask;
@@ -790,8 +793,8 @@ static void cpacr_write(CPUARMState *env, const ARMCPRegInfo *ri,
      */
     if (arm_feature(env, ARM_FEATURE_EL3) && !arm_el_is_aa64(env, 3) &&
         !arm_is_secure(env) && !extract32(env->cp15.nsacr, 10, 1)) {
-        value &= ~(0xf << 20);
-        value |= env->cp15.cpacr_el1 & (0xf << 20);
+        mask = R_CPACR_CP11_MASK | R_CPACR_CP10_MASK;
+        value = (value & ~mask) | (env->cp15.cpacr_el1 & mask);
     }
 
     env->cp15.cpacr_el1 = value;
@@ -807,7 +810,7 @@ static uint64_t cpacr_read(CPUARMState *env, const ARMCPRegInfo *ri)
 
     if (arm_feature(env, ARM_FEATURE_EL3) && !arm_el_is_aa64(env, 3) &&
         !arm_is_secure(env) && !extract32(env->cp15.nsacr, 10, 1)) {
-        value &= ~(0xf << 20);
+        value = ~(R_CPACR_CP11_MASK | R_CPACR_CP10_MASK);
     }
     return value;
 }
@@ -827,11 +830,11 @@ static CPAccessResult cpacr_access(CPUARMState *env, const ARMCPRegInfo *ri,
     if (arm_feature(env, ARM_FEATURE_V8)) {
         /* Check if CPACR accesses are to be trapped to EL2 */
         if (arm_current_el(env) == 1 && arm_is_el2_enabled(env) &&
-            (env->cp15.cptr_el[2] & CPTR_TCPAC)) {
+            FIELD_EX64(env->cp15.cptr_el[2], CPTR_EL2, TCPAC)) {
             return CP_ACCESS_TRAP_EL2;
         /* Check if CPACR accesses are to be trapped to EL3 */
         } else if (arm_current_el(env) < 3 &&
-                   (env->cp15.cptr_el[3] & CPTR_TCPAC)) {
+                   FIELD_EX64(env->cp15.cptr_el[3], CPTR_EL3, TCPAC)) {
             return CP_ACCESS_TRAP_EL3;
         }
     }
@@ -843,7 +846,8 @@ static CPAccessResult cptr_access(CPUARMState *env, const ARMCPRegInfo *ri,
                                   bool isread)
 {
     /* Check if CPTR accesses are set to trap to EL3 */
-    if (arm_current_el(env) == 2 && (env->cp15.cptr_el[3] & CPTR_TCPAC)) {
+    if (arm_current_el(env) == 2 &&
+        FIELD_EX64(env->cp15.cptr_el[3], CPTR_EL3, TCPAC)) {
         return CP_ACCESS_TRAP_EL3;
     }
 
@@ -5343,8 +5347,8 @@ static void cptr_el2_write(CPUARMState *env, const ARMCPRegInfo *ri,
      */
     if (arm_feature(env, ARM_FEATURE_EL3) && !arm_el_is_aa64(env, 3) &&
         !arm_is_secure(env) && !extract32(env->cp15.nsacr, 10, 1)) {
-        value &= ~(0x3 << 10);
-        value |= env->cp15.cptr_el[2] & (0x3 << 10);
+        uint64_t mask = R_HCPTR_TCP11_MASK | R_HCPTR_TCP10_MASK;
+        value = (value & ~mask) | (env->cp15.cptr_el[2] & mask);
     }
     env->cp15.cptr_el[2] = value;
 }
@@ -5359,7 +5363,7 @@ static uint64_t cptr_el2_read(CPUARMState *env, const ARMCPRegInfo *ri)
 
     if (arm_feature(env, ARM_FEATURE_EL3) && !arm_el_is_aa64(env, 3) &&
         !arm_is_secure(env) && !extract32(env->cp15.nsacr, 10, 1)) {
-        value |= 0x3 << 10;
+        value |= R_HCPTR_TCP11_MASK | R_HCPTR_TCP10_MASK;
     }
     return value;
 }
@@ -6147,8 +6151,7 @@ int sve_exception_el(CPUARMState *env, int el)
     uint64_t hcr_el2 = arm_hcr_el2_eff(env);
 
     if (el <= 1 && (hcr_el2 & (HCR_E2H | HCR_TGE)) != (HCR_E2H | HCR_TGE)) {
-        /* Check CPACR.ZEN.  */
-        switch (extract32(env->cp15.cpacr_el1, 16, 2)) {
+        switch (FIELD_EX64(env->cp15.cpacr_el1, CPACR_EL1, ZEN)) {
         case 1:
             if (el != 0) {
                 break;
@@ -6161,7 +6164,7 @@ int sve_exception_el(CPUARMState *env, int el)
         }
 
         /* Check CPACR.FPEN.  */
-        switch (extract32(env->cp15.cpacr_el1, 20, 2)) {
+        switch (FIELD_EX64(env->cp15.cpacr_el1, CPACR_EL1, FPEN)) {
         case 1:
             if (el != 0) {
                 break;
@@ -6178,8 +6181,7 @@ int sve_exception_el(CPUARMState *env, int el)
      */
     if (el <= 2) {
         if (hcr_el2 & HCR_E2H) {
-            /* Check CPTR_EL2.ZEN.  */
-            switch (extract32(env->cp15.cptr_el[2], 16, 2)) {
+            switch (FIELD_EX64(env->cp15.cptr_el[2], CPTR_EL2, ZEN)) {
             case 1:
                 if (el != 0 || !(hcr_el2 & HCR_TGE)) {
                     break;
@@ -6190,8 +6192,7 @@ int sve_exception_el(CPUARMState *env, int el)
                 return 2;
             }
 
-            /* Check CPTR_EL2.FPEN.  */
-            switch (extract32(env->cp15.cptr_el[2], 20, 2)) {
+            switch (FIELD_EX32(env->cp15.cptr_el[2], CPTR_EL2, FPEN)) {
             case 1:
                 if (el == 2 || !(hcr_el2 & HCR_TGE)) {
                     break;
@@ -6202,10 +6203,10 @@ int sve_exception_el(CPUARMState *env, int el)
                 return 0;
             }
         } else if (arm_is_el2_enabled(env)) {
-            if (env->cp15.cptr_el[2] & CPTR_TZ) {
+            if (FIELD_EX64(env->cp15.cptr_el[2], CPTR_EL2, TZ)) {
                 return 2;
             }
-            if (env->cp15.cptr_el[2] & CPTR_TFP) {
+            if (FIELD_EX64(env->cp15.cptr_el[2], CPTR_EL2, TFP)) {
                 return 0;
             }
         }
@@ -6213,7 +6214,7 @@ int sve_exception_el(CPUARMState *env, int el)
 
     /* CPTR_EL3.  Since EZ is negative we must check for EL3.  */
     if (arm_feature(env, ARM_FEATURE_EL3)
-        && !(env->cp15.cptr_el[3] & CPTR_EZ)) {
+        && !FIELD_EX64(env->cp15.cptr_el[3], CPTR_EL3, EZ)) {
         return 3;
     }
 #endif
@@ -13396,7 +13397,7 @@ int fp_exception_el(CPUARMState *env, int cur_el)
      * This register is ignored if E2H+TGE are both set.
      */
     if ((hcr_el2 & (HCR_E2H | HCR_TGE)) != (HCR_E2H | HCR_TGE)) {
-        int fpen = extract32(env->cp15.cpacr_el1, 20, 2);
+        int fpen = FIELD_EX64(env->cp15.cpacr_el1, CPACR_EL1, FPEN);
 
         switch (fpen) {
         case 0:
@@ -13442,8 +13443,7 @@ int fp_exception_el(CPUARMState *env, int cur_el)
      */
     if (cur_el <= 2) {
         if (hcr_el2 & HCR_E2H) {
-            /* Check CPTR_EL2.FPEN.  */
-            switch (extract32(env->cp15.cptr_el[2], 20, 2)) {
+            switch (FIELD_EX64(env->cp15.cptr_el[2], CPTR_EL2, FPEN)) {
             case 1:
                 if (cur_el != 0 || !(hcr_el2 & HCR_TGE)) {
                     break;
@@ -13454,14 +13454,14 @@ int fp_exception_el(CPUARMState *env, int cur_el)
                 return 2;
             }
         } else if (arm_is_el2_enabled(env)) {
-            if (env->cp15.cptr_el[2] & CPTR_TFP) {
+            if (FIELD_EX64(env->cp15.cptr_el[2], CPTR_EL2, TFP)) {
                 return 2;
             }
         }
     }
 
     /* CPTR_EL3 : present in v8 */
-    if (env->cp15.cptr_el[3] & CPTR_TFP) {
+    if (FIELD_EX64(env->cp15.cptr_el[3], CPTR_EL3, TFP)) {
         /* Trap all FP ops to EL3 */
         return 3;
     }

From 60f1c8017a2b137013a8ae83911d74700f692425 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Tue, 26 Apr 2022 19:51:29 -0700
Subject: [PATCH 454/935] linux-user: Clean up arg_start/arg_end confusion
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

We had two sets of variables: arg_start/arg_end, and
arg_strings/env_strings.  In linuxload.c, we set the
first pair to the bounds of the argv strings, but in
elfload.c, we set the first pair to the bounds of the
argv pointers and the second pair to the bounds of
the argv strings.

Remove arg_start/arg_end, replacing them with the standard
argc/argv/envc/envp values.  Retain arg_strings/env_strings
with the meaning we were using in elfload.c.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/714
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220427025129.160184-1-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/elfload.c          | 10 ++++++----
 linux-user/linuxload.c        | 12 ++++++++++--
 linux-user/main.c             |  4 ++--
 linux-user/qemu.h             | 12 ++++++++----
 semihosting/arm-compat-semi.c |  4 ++--
 5 files changed, 28 insertions(+), 14 deletions(-)

diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index 61063fd974..8c0765dd4b 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -1516,8 +1516,8 @@ static inline void init_thread(struct target_pt_regs *regs,
     regs->iaoq[0] = infop->entry;
     regs->iaoq[1] = infop->entry + 4;
     regs->gr[23] = 0;
-    regs->gr[24] = infop->arg_start;
-    regs->gr[25] = (infop->arg_end - infop->arg_start) / sizeof(abi_ulong);
+    regs->gr[24] = infop->argv;
+    regs->gr[25] = infop->argc;
     /* The top-of-stack contains a linkage buffer.  */
     regs->gr[30] = infop->start_stack + 64;
     regs->gr[31] = infop->entry;
@@ -2120,8 +2120,10 @@ static abi_ulong create_elf_tables(abi_ulong p, int argc, int envc,
     u_envp = u_argv + (argc + 1) * n;
     u_auxv = u_envp + (envc + 1) * n;
     info->saved_auxv = u_auxv;
-    info->arg_start = u_argv;
-    info->arg_end = u_argv + argc * n;
+    info->argc = argc;
+    info->envc = envc;
+    info->argv = u_argv;
+    info->envp = u_envp;
 
     /* This is correct because Linux defines
      * elf_addr_t as Elf32_Off / Elf64_Off
diff --git a/linux-user/linuxload.c b/linux-user/linuxload.c
index 2ed5fc45ed..745cce70ab 100644
--- a/linux-user/linuxload.c
+++ b/linux-user/linuxload.c
@@ -92,6 +92,11 @@ abi_ulong loader_build_argptr(int envc, int argc, abi_ulong sp,
     envp = sp;
     sp -= (argc + 1) * n;
     argv = sp;
+    ts->info->envp = envp;
+    ts->info->envc = envc;
+    ts->info->argv = argv;
+    ts->info->argc = argc;
+
     if (push_ptr) {
         /* FIXME - handle put_user() failures */
         sp -= n;
@@ -99,19 +104,22 @@ abi_ulong loader_build_argptr(int envc, int argc, abi_ulong sp,
         sp -= n;
         put_user_ual(argv, sp);
     }
+
     sp -= n;
     /* FIXME - handle put_user() failures */
     put_user_ual(argc, sp);
-    ts->info->arg_start = stringp;
+
+    ts->info->arg_strings = stringp;
     while (argc-- > 0) {
         /* FIXME - handle put_user() failures */
         put_user_ual(stringp, argv);
         argv += n;
         stringp += target_strlen(stringp) + 1;
     }
-    ts->info->arg_end = stringp;
     /* FIXME - handle put_user() failures */
     put_user_ual(0, argv);
+
+    ts->info->env_strings = stringp;
     while (envc-- > 0) {
         /* FIXME - handle put_user() failures */
         put_user_ual(stringp, envp);
diff --git a/linux-user/main.c b/linux-user/main.c
index 7ca48664e4..651e32f5f2 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -878,9 +878,9 @@ int main(int argc, char **argv, char **envp)
             fprintf(f, "entry       0x" TARGET_ABI_FMT_lx "\n",
                     info->entry);
             fprintf(f, "argv_start  0x" TARGET_ABI_FMT_lx "\n",
-                    info->arg_start);
+                    info->argv);
             fprintf(f, "env_start   0x" TARGET_ABI_FMT_lx "\n",
-                    info->arg_end + (abi_ulong)sizeof(abi_ulong));
+                    info->envp);
             fprintf(f, "auxv_start  0x" TARGET_ABI_FMT_lx "\n",
                     info->saved_auxv);
             qemu_log_unlock(f);
diff --git a/linux-user/qemu.h b/linux-user/qemu.h
index 46550f5e21..7d90de1b15 100644
--- a/linux-user/qemu.h
+++ b/linux-user/qemu.h
@@ -40,15 +40,19 @@ struct image_info {
         abi_ulong       data_offset;
         abi_ulong       saved_auxv;
         abi_ulong       auxv_len;
-        abi_ulong       arg_start;
-        abi_ulong       arg_end;
-        abi_ulong       arg_strings;
-        abi_ulong       env_strings;
+        abi_ulong       argc;
+        abi_ulong       argv;
+        abi_ulong       envc;
+        abi_ulong       envp;
         abi_ulong       file_string;
         uint32_t        elf_flags;
         int             personality;
         abi_ulong       alignment;
 
+        /* Generic semihosting knows about these pointers. */
+        abi_ulong       arg_strings;   /* strings for argv */
+        abi_ulong       env_strings;   /* strings for envp; ends arg_strings */
+
         /* The fields below are used in FDPIC mode.  */
         abi_ulong       loadmap_addr;
         uint16_t        nsegs;
diff --git a/semihosting/arm-compat-semi.c b/semihosting/arm-compat-semi.c
index 7a51fd0737..b6ddaf863a 100644
--- a/semihosting/arm-compat-semi.c
+++ b/semihosting/arm-compat-semi.c
@@ -1106,7 +1106,7 @@ target_ulong do_common_semihosting(CPUState *cs)
 #else
             unsigned int i;
 
-            output_size = ts->info->arg_end - ts->info->arg_start;
+            output_size = ts->info->env_strings - ts->info->arg_strings;
             if (!output_size) {
                 /*
                  * We special-case the "empty command line" case (argc==0).
@@ -1146,7 +1146,7 @@ target_ulong do_common_semihosting(CPUState *cs)
                 goto out;
             }
 
-            if (copy_from_user(output_buffer, ts->info->arg_start,
+            if (copy_from_user(output_buffer, ts->info->arg_strings,
                                output_size)) {
                 errno = EFAULT;
                 status = set_swi_errno(cs, -1);

From 3a7a27cffb5ed50840e26a2567e187b17958cc64 Mon Sep 17 00:00:00 2001
From: Yang Weijiang <weijiang.yang@intel.com>
Date: Tue, 17 May 2022 11:50:24 -0400
Subject: [PATCH 455/935] target/i386: Remove LBREn bit check when access Arch
 LBR MSRs

Live migration can happen when Arch LBR LBREn bit is cleared,
e.g., when migration happens after guest entered SMM mode.
In this case, we still need to migrate Arch LBR MSRs.

Signed-off-by: Yang Weijiang <weijiang.yang@intel.com>
Message-Id: <20220517155024.33270-1-weijiang.yang@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/kvm/kvm.c | 21 +++++++++------------
 1 file changed, 9 insertions(+), 12 deletions(-)

diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index a9ee8eebd7..e2d675115b 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -3373,15 +3373,14 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
             int i, ret;
 
             /*
-             * Only migrate Arch LBR states when: 1) Arch LBR is enabled
-             * for migrated vcpu. 2) the host Arch LBR depth equals that
-             * of source guest's, this is to avoid mismatch of guest/host
-             * config for the msr hence avoid unexpected misbehavior.
+             * Only migrate Arch LBR states when the host Arch LBR depth
+             * equals that of source guest's, this is to avoid mismatch
+             * of guest/host config for the msr hence avoid unexpected
+             * misbehavior.
              */
             ret = kvm_get_one_msr(cpu, MSR_ARCH_LBR_DEPTH, &depth);
 
-            if (ret == 1 && (env->msr_lbr_ctl & 0x1) && !!depth &&
-                depth == env->msr_lbr_depth) {
+            if (ret == 1 && !!depth && depth == env->msr_lbr_depth) {
                 kvm_msr_entry_add(cpu, MSR_ARCH_LBR_CTL, env->msr_lbr_ctl);
                 kvm_msr_entry_add(cpu, MSR_ARCH_LBR_DEPTH, env->msr_lbr_depth);
 
@@ -3801,13 +3800,11 @@ static int kvm_get_msrs(X86CPU *cpu)
 
     if (kvm_enabled() && cpu->enable_pmu &&
         (env->features[FEAT_7_0_EDX] & CPUID_7_0_EDX_ARCH_LBR)) {
-        uint64_t ctl, depth;
-        int i, ret2;
+        uint64_t depth;
+        int i, ret;
 
-        ret = kvm_get_one_msr(cpu, MSR_ARCH_LBR_CTL, &ctl);
-        ret2 = kvm_get_one_msr(cpu, MSR_ARCH_LBR_DEPTH, &depth);
-        if (ret == 1 && ret2 == 1 && (ctl & 0x1) &&
-            depth == ARCH_LBR_NR_ENTRIES) {
+        ret = kvm_get_one_msr(cpu, MSR_ARCH_LBR_DEPTH, &depth);
+        if (ret == 1 && depth == ARCH_LBR_NR_ENTRIES) {
             kvm_msr_entry_add(cpu, MSR_ARCH_LBR_CTL, 0);
             kvm_msr_entry_add(cpu, MSR_ARCH_LBR_DEPTH, 0);
 

From f8d426a6852c560fdd8648ae961c8189909a4b82 Mon Sep 17 00:00:00 2001
From: Jaroslav Jindrak <dzejrou@gmail.com>
Date: Tue, 17 May 2022 14:38:58 +0200
Subject: [PATCH 456/935] hostmem: default the amount of prealloc-threads to
 smp-cpus

Prior to the introduction of the prealloc-threads property, the amount
of threads used to preallocate memory was derived from the value of
smp-cpus passed to qemu, the amount of physical cpus of the host
and a hardcoded maximum value. When the prealloc-threads property
was introduced, it included a default of 1 in backends/hostmem.c and
a default of smp-cpus using the sugar API for the property itself. The
latter default is not used when the property is not specified on qemu's
command line, so guests that were not adjusted for this change suddenly
started to use the default of 1 thread to preallocate memory, which
resulted in observable slowdowns in guest boots for guests with large
memory (e.g. when using libvirt <8.2.0 or managing guests manually).

This commit restores the original behavior for these cases while not
impacting guests started with the prealloc-threads property in any way.

Fixes: 220c1fd864e9d ("hostmem: introduce "prealloc-threads" property")
Signed-off-by: Jaroslav Jindrak <dzejrou@gmail.com>
Message-Id: <20220517123858.7933-1-dzejrou@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 backends/hostmem.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/backends/hostmem.c b/backends/hostmem.c
index a7bae3d713..624bb7ecd3 100644
--- a/backends/hostmem.c
+++ b/backends/hostmem.c
@@ -274,7 +274,7 @@ static void host_memory_backend_init(Object *obj)
     backend->merge = machine_mem_merge(machine);
     backend->dump = machine_dump_guest_core(machine);
     backend->reserve = true;
-    backend->prealloc_threads = 1;
+    backend->prealloc_threads = machine->smp.cpus;
 }
 
 static void host_memory_backend_post_init(Object *obj)

From c3a28d71221d30b1f58bf5efc88add20635ce46d Mon Sep 17 00:00:00 2001
From: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Date: Mon, 23 May 2022 12:52:39 +0200
Subject: [PATCH 457/935] linux-user/syscall.c: fix build without RLIMIT_RTTIME

RLIMIT_RTTIME is not provided by uclibc-ng or by musl prior to version
1.2.0 and
https://github.com/bminor/musl/commit/2507e7f5312e79620f6337935d0a6c9045ccba09
resulting in the following build failure since
https://git.qemu.org/?p=qemu.git;a=commit;h=244fd08323088db73590ff2317dfe86f810b51d7:

../linux-user/syscall.c: In function 'target_to_host_resource':
../linux-user/syscall.c:1057:16: error: 'RLIMIT_RTTIME' undeclared (first use in this function); did you mean 'RLIMIT_NOFILE'?
 1057 |         return RLIMIT_RTTIME;
      |                ^~~~~~~~~~~~~
      |                RLIMIT_NOFILE

Fixes:
 - http://autobuild.buildroot.org/results/22d3b584b704613d030e1ea9e6b709b713e4cc26

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20220523105239.1499162-1-fontaine.fabrice@gmail.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/syscall.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index dd0d92ba4e..488facb356 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -1053,8 +1053,10 @@ static inline int target_to_host_resource(int code)
         return RLIMIT_RSS;
     case TARGET_RLIMIT_RTPRIO:
         return RLIMIT_RTPRIO;
+#ifdef RLIMIT_RTTIME
     case TARGET_RLIMIT_RTTIME:
         return RLIMIT_RTTIME;
+#endif
     case TARGET_RLIMIT_SIGPENDING:
         return RLIMIT_SIGPENDING;
     case TARGET_RLIMIT_STACK:

From 2f6f4290e0f0f3a8ea49a81afebd2d41c327477f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
Date: Mon, 9 May 2022 22:57:26 +0200
Subject: [PATCH 458/935] linux-user/elfload: Remove pointless non-const
 CPUArchState cast
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

fill_thread_info() takes a pointer to const.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220509205728.51912-2-philippe.mathieu.daude@gmail.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/elfload.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index 8c0765dd4b..acc21748f9 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -3973,7 +3973,7 @@ static int fill_note_info(struct elf_note_info *info,
         if (cpu == thread_cpu) {
             continue;
         }
-        fill_thread_info(info, (CPUArchState *)cpu->env_ptr);
+        fill_thread_info(info, cpu->env_ptr);
     }
     cpu_list_unlock();
 

From a0939b89161a756a7f58cab2bec0a2410c5d0b5f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
Date: Mon, 9 May 2022 22:57:27 +0200
Subject: [PATCH 459/935] linux-user: Have do_syscall() use CPUArchState*
 instead of void*
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220509205728.51912-3-philippe.mathieu.daude@gmail.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/strace.c         | 202 ++++++++++++++++++------------------
 linux-user/strace.h         |   4 +-
 linux-user/syscall.c        |  32 +++---
 linux-user/uname.c          |   2 +-
 linux-user/uname.h          |   2 +-
 linux-user/user-internals.h |  16 +--
 6 files changed, 129 insertions(+), 129 deletions(-)

diff --git a/linux-user/strace.c b/linux-user/strace.c
index 2cdbf030ba..9fa681dea9 100644
--- a/linux-user/strace.c
+++ b/linux-user/strace.c
@@ -22,10 +22,10 @@ struct syscallname {
     int nr;
     const char *name;
     const char *format;
-    void (*call)(void *, const struct syscallname *,
+    void (*call)(CPUArchState *, const struct syscallname *,
                  abi_long, abi_long, abi_long,
                  abi_long, abi_long, abi_long);
-    void (*result)(void *, const struct syscallname *, abi_long,
+    void (*result)(CPUArchState *, const struct syscallname *, abi_long,
                    abi_long, abi_long, abi_long,
                    abi_long, abi_long, abi_long);
 };
@@ -593,7 +593,7 @@ print_fdset(int n, abi_ulong target_fds_addr)
 /* select */
 #ifdef TARGET_NR__newselect
 static void
-print_newselect(void *cpu_env, const struct syscallname *name,
+print_newselect(CPUArchState *cpu_env, const struct syscallname *name,
                 abi_long arg1, abi_long arg2, abi_long arg3,
                 abi_long arg4, abi_long arg5, abi_long arg6)
 {
@@ -611,7 +611,7 @@ print_newselect(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_semctl
 static void
-print_semctl(void *cpu_env, const struct syscallname *name,
+print_semctl(CPUArchState *cpu_env, const struct syscallname *name,
              abi_long arg1, abi_long arg2, abi_long arg3,
              abi_long arg4, abi_long arg5, abi_long arg6)
 {
@@ -623,7 +623,7 @@ print_semctl(void *cpu_env, const struct syscallname *name,
 #endif
 
 static void
-print_execve(void *cpu_env, const struct syscallname *name,
+print_execve(CPUArchState *cpu_env, const struct syscallname *name,
              abi_long arg1, abi_long arg2, abi_long arg3,
              abi_long arg4, abi_long arg5, abi_long arg6)
 {
@@ -656,7 +656,7 @@ print_execve(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_ipc
 static void
-print_ipc(void *cpu_env, const struct syscallname *name,
+print_ipc(CPUArchState *cpu_env, const struct syscallname *name,
           abi_long arg1, abi_long arg2, abi_long arg3,
           abi_long arg4, abi_long arg5, abi_long arg6)
 {
@@ -700,7 +700,7 @@ print_syscall_err(abi_long ret)
 }
 
 static void
-print_syscall_ret_addr(void *cpu_env, const struct syscallname *name,
+print_syscall_ret_addr(CPUArchState *cpu_env, const struct syscallname *name,
                        abi_long ret, abi_long arg0, abi_long arg1,
                        abi_long arg2, abi_long arg3, abi_long arg4,
                        abi_long arg5)
@@ -721,7 +721,7 @@ print_syscall_ret_raw(struct syscallname *name, abi_long ret)
 
 #ifdef TARGET_NR__newselect
 static void
-print_syscall_ret_newselect(void *cpu_env, const struct syscallname *name,
+print_syscall_ret_newselect(CPUArchState *cpu_env, const struct syscallname *name,
                             abi_long ret, abi_long arg0, abi_long arg1,
                             abi_long arg2, abi_long arg3, abi_long arg4,
                             abi_long arg5)
@@ -751,7 +751,7 @@ print_syscall_ret_newselect(void *cpu_env, const struct syscallname *name,
 #define TARGET_TIME_ERROR    5   /* clock not synchronized */
 #ifdef TARGET_NR_adjtimex
 static void
-print_syscall_ret_adjtimex(void *cpu_env, const struct syscallname *name,
+print_syscall_ret_adjtimex(CPUArchState *cpu_env, const struct syscallname *name,
                            abi_long ret, abi_long arg0, abi_long arg1,
                            abi_long arg2, abi_long arg3, abi_long arg4,
                            abi_long arg5)
@@ -786,7 +786,7 @@ print_syscall_ret_adjtimex(void *cpu_env, const struct syscallname *name,
 
 #if defined(TARGET_NR_clock_gettime) || defined(TARGET_NR_clock_getres)
 static void
-print_syscall_ret_clock_gettime(void *cpu_env, const struct syscallname *name,
+print_syscall_ret_clock_gettime(CPUArchState *cpu_env, const struct syscallname *name,
                                 abi_long ret, abi_long arg0, abi_long arg1,
                                 abi_long arg2, abi_long arg3, abi_long arg4,
                                 abi_long arg5)
@@ -805,7 +805,7 @@ print_syscall_ret_clock_gettime(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_gettimeofday
 static void
-print_syscall_ret_gettimeofday(void *cpu_env, const struct syscallname *name,
+print_syscall_ret_gettimeofday(CPUArchState *cpu_env, const struct syscallname *name,
                                abi_long ret, abi_long arg0, abi_long arg1,
                                abi_long arg2, abi_long arg3, abi_long arg4,
                                abi_long arg5)
@@ -824,7 +824,7 @@ print_syscall_ret_gettimeofday(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_getitimer
 static void
-print_syscall_ret_getitimer(void *cpu_env, const struct syscallname *name,
+print_syscall_ret_getitimer(CPUArchState *cpu_env, const struct syscallname *name,
                             abi_long ret, abi_long arg0, abi_long arg1,
                             abi_long arg2, abi_long arg3, abi_long arg4,
                             abi_long arg5)
@@ -843,7 +843,7 @@ print_syscall_ret_getitimer(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_getitimer
 static void
-print_syscall_ret_setitimer(void *cpu_env, const struct syscallname *name,
+print_syscall_ret_setitimer(CPUArchState *cpu_env, const struct syscallname *name,
                             abi_long ret, abi_long arg0, abi_long arg1,
                             abi_long arg2, abi_long arg3, abi_long arg4,
                             abi_long arg5)
@@ -862,7 +862,7 @@ print_syscall_ret_setitimer(void *cpu_env, const struct syscallname *name,
 #if defined(TARGET_NR_listxattr) || defined(TARGET_NR_llistxattr) \
  || defined(TARGGET_NR_flistxattr)
 static void
-print_syscall_ret_listxattr(void *cpu_env, const struct syscallname *name,
+print_syscall_ret_listxattr(CPUArchState *cpu_env, const struct syscallname *name,
                             abi_long ret, abi_long arg0, abi_long arg1,
                             abi_long arg2, abi_long arg3, abi_long arg4,
                             abi_long arg5)
@@ -894,7 +894,7 @@ print_syscall_ret_listxattr(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_ioctl
 static void
-print_syscall_ret_ioctl(void *cpu_env, const struct syscallname *name,
+print_syscall_ret_ioctl(CPUArchState *cpu_env, const struct syscallname *name,
                         abi_long ret, abi_long arg0, abi_long arg1,
                         abi_long arg2, abi_long arg3, abi_long arg4,
                         abi_long arg5)
@@ -1760,7 +1760,7 @@ print_termios(void *arg)
 
 #ifdef TARGET_NR_accept
 static void
-print_accept(void *cpu_env, const struct syscallname *name,
+print_accept(CPUArchState *cpu_env, const struct syscallname *name,
              abi_long arg0, abi_long arg1, abi_long arg2,
              abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -1774,7 +1774,7 @@ print_accept(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_access
 static void
-print_access(void *cpu_env, const struct syscallname *name,
+print_access(CPUArchState *cpu_env, const struct syscallname *name,
              abi_long arg0, abi_long arg1, abi_long arg2,
              abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -1787,7 +1787,7 @@ print_access(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_acct
 static void
-print_acct(void *cpu_env, const struct syscallname *name,
+print_acct(CPUArchState *cpu_env, const struct syscallname *name,
            abi_long arg0, abi_long arg1, abi_long arg2,
            abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -1799,7 +1799,7 @@ print_acct(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_brk
 static void
-print_brk(void *cpu_env, const struct syscallname *name,
+print_brk(CPUArchState *cpu_env, const struct syscallname *name,
           abi_long arg0, abi_long arg1, abi_long arg2,
           abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -1811,7 +1811,7 @@ print_brk(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_chdir
 static void
-print_chdir(void *cpu_env, const struct syscallname *name,
+print_chdir(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg0, abi_long arg1, abi_long arg2,
             abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -1823,7 +1823,7 @@ print_chdir(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_chroot
 static void
-print_chroot(void *cpu_env, const struct syscallname *name,
+print_chroot(CPUArchState *cpu_env, const struct syscallname *name,
              abi_long arg0, abi_long arg1, abi_long arg2,
              abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -1835,7 +1835,7 @@ print_chroot(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_chmod
 static void
-print_chmod(void *cpu_env, const struct syscallname *name,
+print_chmod(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg0, abi_long arg1, abi_long arg2,
             abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -1848,7 +1848,7 @@ print_chmod(void *cpu_env, const struct syscallname *name,
 
 #if defined(TARGET_NR_chown) || defined(TARGET_NR_lchown)
 static void
-print_chown(void *cpu_env, const struct syscallname *name,
+print_chown(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg0, abi_long arg1, abi_long arg2,
             abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -1863,7 +1863,7 @@ print_chown(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_clock_adjtime
 static void
-print_clock_adjtime(void *cpu_env, const struct syscallname *name,
+print_clock_adjtime(CPUArchState *cpu_env, const struct syscallname *name,
                     abi_long arg0, abi_long arg1, abi_long arg2,
                     abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -1887,7 +1887,7 @@ static void do_print_clone(unsigned int flags, abi_ulong newsp,
 }
 
 static void
-print_clone(void *cpu_env, const struct syscallname *name,
+print_clone(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg1, abi_long arg2, abi_long arg3,
             abi_long arg4, abi_long arg5, abi_long arg6)
 {
@@ -1907,7 +1907,7 @@ print_clone(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_creat
 static void
-print_creat(void *cpu_env, const struct syscallname *name,
+print_creat(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg0, abi_long arg1, abi_long arg2,
             abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -1920,7 +1920,7 @@ print_creat(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_execv
 static void
-print_execv(void *cpu_env, const struct syscallname *name,
+print_execv(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg0, abi_long arg1, abi_long arg2,
             abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -1933,7 +1933,7 @@ print_execv(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_faccessat
 static void
-print_faccessat(void *cpu_env, const struct syscallname *name,
+print_faccessat(CPUArchState *cpu_env, const struct syscallname *name,
                 abi_long arg0, abi_long arg1, abi_long arg2,
                 abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -1948,7 +1948,7 @@ print_faccessat(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_fallocate
 static void
-print_fallocate(void *cpu_env, const struct syscallname *name,
+print_fallocate(CPUArchState *cpu_env, const struct syscallname *name,
                 abi_long arg0, abi_long arg1, abi_long arg2,
                 abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -1968,7 +1968,7 @@ print_fallocate(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_fchmodat
 static void
-print_fchmodat(void *cpu_env, const struct syscallname *name,
+print_fchmodat(CPUArchState *cpu_env, const struct syscallname *name,
                abi_long arg0, abi_long arg1, abi_long arg2,
                abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -1983,7 +1983,7 @@ print_fchmodat(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_fchownat
 static void
-print_fchownat(void *cpu_env, const struct syscallname *name,
+print_fchownat(CPUArchState *cpu_env, const struct syscallname *name,
                abi_long arg0, abi_long arg1, abi_long arg2,
                abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -1999,7 +1999,7 @@ print_fchownat(void *cpu_env, const struct syscallname *name,
 
 #if defined(TARGET_NR_fcntl) || defined(TARGET_NR_fcntl64)
 static void
-print_fcntl(void *cpu_env, const struct syscallname *name,
+print_fcntl(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg0, abi_long arg1, abi_long arg2,
             abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2135,7 +2135,7 @@ print_fcntl(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_fgetxattr
 static void
-print_fgetxattr(void *cpu_env, const struct syscallname *name,
+print_fgetxattr(CPUArchState *cpu_env, const struct syscallname *name,
                 abi_long arg0, abi_long arg1, abi_long arg2,
                 abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2150,7 +2150,7 @@ print_fgetxattr(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_flistxattr
 static void
-print_flistxattr(void *cpu_env, const struct syscallname *name,
+print_flistxattr(CPUArchState *cpu_env, const struct syscallname *name,
                  abi_long arg0, abi_long arg1, abi_long arg2,
                  abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2164,7 +2164,7 @@ print_flistxattr(void *cpu_env, const struct syscallname *name,
 
 #if defined(TARGET_NR_getxattr) || defined(TARGET_NR_lgetxattr)
 static void
-print_getxattr(void *cpu_env, const struct syscallname *name,
+print_getxattr(CPUArchState *cpu_env, const struct syscallname *name,
                abi_long arg0, abi_long arg1, abi_long arg2,
                abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2180,7 +2180,7 @@ print_getxattr(void *cpu_env, const struct syscallname *name,
 
 #if defined(TARGET_NR_listxattr) || defined(TARGET_NR_llistxattr)
 static void
-print_listxattr(void *cpu_env, const struct syscallname *name,
+print_listxattr(CPUArchState *cpu_env, const struct syscallname *name,
                 abi_long arg0, abi_long arg1, abi_long arg2,
                 abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2195,7 +2195,7 @@ print_listxattr(void *cpu_env, const struct syscallname *name,
 
 #if defined(TARGET_NR_fremovexattr)
 static void
-print_fremovexattr(void *cpu_env, const struct syscallname *name,
+print_fremovexattr(CPUArchState *cpu_env, const struct syscallname *name,
                    abi_long arg0, abi_long arg1, abi_long arg2,
                    abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2208,7 +2208,7 @@ print_fremovexattr(void *cpu_env, const struct syscallname *name,
 
 #if defined(TARGET_NR_removexattr) || defined(TARGET_NR_lremovexattr)
 static void
-print_removexattr(void *cpu_env, const struct syscallname *name,
+print_removexattr(CPUArchState *cpu_env, const struct syscallname *name,
                   abi_long arg0, abi_long arg1, abi_long arg2,
                   abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2222,7 +2222,7 @@ print_removexattr(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_futimesat
 static void
-print_futimesat(void *cpu_env, const struct syscallname *name,
+print_futimesat(CPUArchState *cpu_env, const struct syscallname *name,
                 abi_long arg0, abi_long arg1, abi_long arg2,
                 abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2237,7 +2237,7 @@ print_futimesat(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_gettimeofday
 static void
-print_gettimeofday(void *cpu_env, const struct syscallname *name,
+print_gettimeofday(CPUArchState *cpu_env, const struct syscallname *name,
                    abi_long arg0, abi_long arg1, abi_long arg2,
                    abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2250,7 +2250,7 @@ print_gettimeofday(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_settimeofday
 static void
-print_settimeofday(void *cpu_env, const struct syscallname *name,
+print_settimeofday(CPUArchState *cpu_env, const struct syscallname *name,
                    abi_long arg0, abi_long arg1, abi_long arg2,
                    abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2263,7 +2263,7 @@ print_settimeofday(void *cpu_env, const struct syscallname *name,
 
 #if defined(TARGET_NR_clock_gettime) || defined(TARGET_NR_clock_getres)
 static void
-print_clock_gettime(void *cpu_env, const struct syscallname *name,
+print_clock_gettime(CPUArchState *cpu_env, const struct syscallname *name,
                     abi_long arg0, abi_long arg1, abi_long arg2,
                     abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2277,7 +2277,7 @@ print_clock_gettime(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_clock_settime
 static void
-print_clock_settime(void *cpu_env, const struct syscallname *name,
+print_clock_settime(CPUArchState *cpu_env, const struct syscallname *name,
                     abi_long arg0, abi_long arg1, abi_long arg2,
                     abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2290,7 +2290,7 @@ print_clock_settime(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_getitimer
 static void
-print_getitimer(void *cpu_env, const struct syscallname *name,
+print_getitimer(CPUArchState *cpu_env, const struct syscallname *name,
                 abi_long arg0, abi_long arg1, abi_long arg2,
                 abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2303,7 +2303,7 @@ print_getitimer(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_setitimer
 static void
-print_setitimer(void *cpu_env, const struct syscallname *name,
+print_setitimer(CPUArchState *cpu_env, const struct syscallname *name,
                 abi_long arg0, abi_long arg1, abi_long arg2,
                 abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2317,7 +2317,7 @@ print_setitimer(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_link
 static void
-print_link(void *cpu_env, const struct syscallname *name,
+print_link(CPUArchState *cpu_env, const struct syscallname *name,
            abi_long arg0, abi_long arg1, abi_long arg2,
            abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2330,7 +2330,7 @@ print_link(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_linkat
 static void
-print_linkat(void *cpu_env, const struct syscallname *name,
+print_linkat(CPUArchState *cpu_env, const struct syscallname *name,
              abi_long arg0, abi_long arg1, abi_long arg2,
              abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2346,7 +2346,7 @@ print_linkat(void *cpu_env, const struct syscallname *name,
 
 #if defined(TARGET_NR__llseek) || defined(TARGET_NR_llseek)
 static void
-print__llseek(void *cpu_env, const struct syscallname *name,
+print__llseek(CPUArchState *cpu_env, const struct syscallname *name,
               abi_long arg0, abi_long arg1, abi_long arg2,
               abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2369,7 +2369,7 @@ print__llseek(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_lseek
 static void
-print_lseek(void *cpu_env, const struct syscallname *name,
+print_lseek(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg0, abi_long arg1, abi_long arg2,
             abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2400,7 +2400,7 @@ print_lseek(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_truncate
 static void
-print_truncate(void *cpu_env, const struct syscallname *name,
+print_truncate(CPUArchState *cpu_env, const struct syscallname *name,
                abi_long arg0, abi_long arg1, abi_long arg2,
                abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2413,7 +2413,7 @@ print_truncate(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_truncate64
 static void
-print_truncate64(void *cpu_env, const struct syscallname *name,
+print_truncate64(CPUArchState *cpu_env, const struct syscallname *name,
                  abi_long arg0, abi_long arg1, abi_long arg2,
                  abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2430,7 +2430,7 @@ print_truncate64(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_ftruncate64
 static void
-print_ftruncate64(void *cpu_env, const struct syscallname *name,
+print_ftruncate64(CPUArchState *cpu_env, const struct syscallname *name,
                   abi_long arg0, abi_long arg1, abi_long arg2,
                   abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2447,7 +2447,7 @@ print_ftruncate64(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_mlockall
 static void
-print_mlockall(void *cpu_env, const struct syscallname *name,
+print_mlockall(CPUArchState *cpu_env, const struct syscallname *name,
                abi_long arg0, abi_long arg1, abi_long arg2,
                abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2459,7 +2459,7 @@ print_mlockall(void *cpu_env, const struct syscallname *name,
 
 #if defined(TARGET_NR_socket)
 static void
-print_socket(void *cpu_env, const struct syscallname *name,
+print_socket(CPUArchState *cpu_env, const struct syscallname *name,
              abi_long arg0, abi_long arg1, abi_long arg2,
              abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2920,7 +2920,7 @@ static struct {
 };
 
 static void
-print_socketcall(void *cpu_env, const struct syscallname *name,
+print_socketcall(CPUArchState *cpu_env, const struct syscallname *name,
                  abi_long arg0, abi_long arg1, abi_long arg2,
                  abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2941,7 +2941,7 @@ print_socketcall(void *cpu_env, const struct syscallname *name,
 
 #if defined(TARGET_NR_bind)
 static void
-print_bind(void *cpu_env, const struct syscallname *name,
+print_bind(CPUArchState *cpu_env, const struct syscallname *name,
            abi_long arg0, abi_long arg1, abi_long arg2,
            abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2955,7 +2955,7 @@ print_bind(void *cpu_env, const struct syscallname *name,
 #if defined(TARGET_NR_stat) || defined(TARGET_NR_stat64) || \
     defined(TARGET_NR_lstat) || defined(TARGET_NR_lstat64)
 static void
-print_stat(void *cpu_env, const struct syscallname *name,
+print_stat(CPUArchState *cpu_env, const struct syscallname *name,
            abi_long arg0, abi_long arg1, abi_long arg2,
            abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2971,7 +2971,7 @@ print_stat(void *cpu_env, const struct syscallname *name,
 
 #if defined(TARGET_NR_fstat) || defined(TARGET_NR_fstat64)
 static void
-print_fstat(void *cpu_env, const struct syscallname *name,
+print_fstat(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg0, abi_long arg1, abi_long arg2,
             abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2985,7 +2985,7 @@ print_fstat(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_mkdir
 static void
-print_mkdir(void *cpu_env, const struct syscallname *name,
+print_mkdir(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg0, abi_long arg1, abi_long arg2,
             abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -2998,7 +2998,7 @@ print_mkdir(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_mkdirat
 static void
-print_mkdirat(void *cpu_env, const struct syscallname *name,
+print_mkdirat(CPUArchState *cpu_env, const struct syscallname *name,
               abi_long arg0, abi_long arg1, abi_long arg2,
               abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3012,7 +3012,7 @@ print_mkdirat(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_rmdir
 static void
-print_rmdir(void *cpu_env, const struct syscallname *name,
+print_rmdir(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg0, abi_long arg1, abi_long arg2,
             abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3024,7 +3024,7 @@ print_rmdir(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_rt_sigaction
 static void
-print_rt_sigaction(void *cpu_env, const struct syscallname *name,
+print_rt_sigaction(CPUArchState *cpu_env, const struct syscallname *name,
                    abi_long arg0, abi_long arg1, abi_long arg2,
                    abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3038,7 +3038,7 @@ print_rt_sigaction(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_rt_sigprocmask
 static void
-print_rt_sigprocmask(void *cpu_env, const struct syscallname *name,
+print_rt_sigprocmask(CPUArchState *cpu_env, const struct syscallname *name,
                      abi_long arg0, abi_long arg1, abi_long arg2,
                      abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3058,7 +3058,7 @@ print_rt_sigprocmask(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_rt_sigqueueinfo
 static void
-print_rt_sigqueueinfo(void *cpu_env, const struct syscallname *name,
+print_rt_sigqueueinfo(CPUArchState *cpu_env, const struct syscallname *name,
                       abi_long arg0, abi_long arg1, abi_long arg2,
                       abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3083,7 +3083,7 @@ print_rt_sigqueueinfo(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_rt_tgsigqueueinfo
 static void
-print_rt_tgsigqueueinfo(void *cpu_env, const struct syscallname *name,
+print_rt_tgsigqueueinfo(CPUArchState *cpu_env, const struct syscallname *name,
                         abi_long arg0, abi_long arg1, abi_long arg2,
                         abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3167,7 +3167,7 @@ print_syslog_action(abi_ulong arg, int last)
 }
 
 static void
-print_syslog(void *cpu_env, const struct syscallname *name,
+print_syslog(CPUArchState *cpu_env, const struct syscallname *name,
              abi_long arg0, abi_long arg1, abi_long arg2,
              abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3181,7 +3181,7 @@ print_syslog(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_mknod
 static void
-print_mknod(void *cpu_env, const struct syscallname *name,
+print_mknod(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg0, abi_long arg1, abi_long arg2,
             abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3200,7 +3200,7 @@ print_mknod(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_mknodat
 static void
-print_mknodat(void *cpu_env, const struct syscallname *name,
+print_mknodat(CPUArchState *cpu_env, const struct syscallname *name,
               abi_long arg0, abi_long arg1, abi_long arg2,
               abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3220,7 +3220,7 @@ print_mknodat(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_mq_open
 static void
-print_mq_open(void *cpu_env, const struct syscallname *name,
+print_mq_open(CPUArchState *cpu_env, const struct syscallname *name,
               abi_long arg0, abi_long arg1, abi_long arg2,
               abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3239,7 +3239,7 @@ print_mq_open(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_open
 static void
-print_open(void *cpu_env, const struct syscallname *name,
+print_open(CPUArchState *cpu_env, const struct syscallname *name,
            abi_long arg0, abi_long arg1, abi_long arg2,
            abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3256,7 +3256,7 @@ print_open(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_openat
 static void
-print_openat(void *cpu_env, const struct syscallname *name,
+print_openat(CPUArchState *cpu_env, const struct syscallname *name,
              abi_long arg0, abi_long arg1, abi_long arg2,
              abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3274,7 +3274,7 @@ print_openat(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_mq_unlink
 static void
-print_mq_unlink(void *cpu_env, const struct syscallname *name,
+print_mq_unlink(CPUArchState *cpu_env, const struct syscallname *name,
                 abi_long arg0, abi_long arg1, abi_long arg2,
                 abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3286,7 +3286,7 @@ print_mq_unlink(void *cpu_env, const struct syscallname *name,
 
 #if defined(TARGET_NR_fstatat64) || defined(TARGET_NR_newfstatat)
 static void
-print_fstatat64(void *cpu_env, const struct syscallname *name,
+print_fstatat64(CPUArchState *cpu_env, const struct syscallname *name,
                 abi_long arg0, abi_long arg1, abi_long arg2,
                 abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3302,7 +3302,7 @@ print_fstatat64(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_readlink
 static void
-print_readlink(void *cpu_env, const struct syscallname *name,
+print_readlink(CPUArchState *cpu_env, const struct syscallname *name,
                abi_long arg0, abi_long arg1, abi_long arg2,
                abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3316,7 +3316,7 @@ print_readlink(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_readlinkat
 static void
-print_readlinkat(void *cpu_env, const struct syscallname *name,
+print_readlinkat(CPUArchState *cpu_env, const struct syscallname *name,
                  abi_long arg0, abi_long arg1, abi_long arg2,
                  abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3331,7 +3331,7 @@ print_readlinkat(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_rename
 static void
-print_rename(void *cpu_env, const struct syscallname *name,
+print_rename(CPUArchState *cpu_env, const struct syscallname *name,
              abi_long arg0, abi_long arg1, abi_long arg2,
              abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3344,7 +3344,7 @@ print_rename(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_renameat
 static void
-print_renameat(void *cpu_env, const struct syscallname *name,
+print_renameat(CPUArchState *cpu_env, const struct syscallname *name,
                abi_long arg0, abi_long arg1, abi_long arg2,
                abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3359,7 +3359,7 @@ print_renameat(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_statfs
 static void
-print_statfs(void *cpu_env, const struct syscallname *name,
+print_statfs(CPUArchState *cpu_env, const struct syscallname *name,
              abi_long arg0, abi_long arg1, abi_long arg2,
              abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3372,7 +3372,7 @@ print_statfs(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_statfs64
 static void
-print_statfs64(void *cpu_env, const struct syscallname *name,
+print_statfs64(CPUArchState *cpu_env, const struct syscallname *name,
                abi_long arg0, abi_long arg1, abi_long arg2,
                abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3385,7 +3385,7 @@ print_statfs64(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_symlink
 static void
-print_symlink(void *cpu_env, const struct syscallname *name,
+print_symlink(CPUArchState *cpu_env, const struct syscallname *name,
               abi_long arg0, abi_long arg1, abi_long arg2,
               abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3398,7 +3398,7 @@ print_symlink(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_symlinkat
 static void
-print_symlinkat(void *cpu_env, const struct syscallname *name,
+print_symlinkat(CPUArchState *cpu_env, const struct syscallname *name,
                 abi_long arg0, abi_long arg1, abi_long arg2,
                 abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3412,7 +3412,7 @@ print_symlinkat(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_mount
 static void
-print_mount(void *cpu_env, const struct syscallname *name,
+print_mount(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg0, abi_long arg1, abi_long arg2,
             abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3428,7 +3428,7 @@ print_mount(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_umount
 static void
-print_umount(void *cpu_env, const struct syscallname *name,
+print_umount(CPUArchState *cpu_env, const struct syscallname *name,
              abi_long arg0, abi_long arg1, abi_long arg2,
              abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3440,7 +3440,7 @@ print_umount(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_umount2
 static void
-print_umount2(void *cpu_env, const struct syscallname *name,
+print_umount2(CPUArchState *cpu_env, const struct syscallname *name,
               abi_long arg0, abi_long arg1, abi_long arg2,
               abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3453,7 +3453,7 @@ print_umount2(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_unlink
 static void
-print_unlink(void *cpu_env, const struct syscallname *name,
+print_unlink(CPUArchState *cpu_env, const struct syscallname *name,
              abi_long arg0, abi_long arg1, abi_long arg2,
              abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3465,7 +3465,7 @@ print_unlink(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_unlinkat
 static void
-print_unlinkat(void *cpu_env, const struct syscallname *name,
+print_unlinkat(CPUArchState *cpu_env, const struct syscallname *name,
                abi_long arg0, abi_long arg1, abi_long arg2,
                abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3479,7 +3479,7 @@ print_unlinkat(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_unshare
 static void
-print_unshare(void *cpu_env, const struct syscallname *name,
+print_unshare(CPUArchState *cpu_env, const struct syscallname *name,
               abi_long arg0, abi_long arg1, abi_long arg2,
               abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3491,7 +3491,7 @@ print_unshare(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_utime
 static void
-print_utime(void *cpu_env, const struct syscallname *name,
+print_utime(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg0, abi_long arg1, abi_long arg2,
             abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3504,7 +3504,7 @@ print_utime(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_utimes
 static void
-print_utimes(void *cpu_env, const struct syscallname *name,
+print_utimes(CPUArchState *cpu_env, const struct syscallname *name,
              abi_long arg0, abi_long arg1, abi_long arg2,
              abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3517,7 +3517,7 @@ print_utimes(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_utimensat
 static void
-print_utimensat(void *cpu_env, const struct syscallname *name,
+print_utimensat(CPUArchState *cpu_env, const struct syscallname *name,
                 abi_long arg0, abi_long arg1, abi_long arg2,
                 abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3532,7 +3532,7 @@ print_utimensat(void *cpu_env, const struct syscallname *name,
 
 #if defined(TARGET_NR_mmap) || defined(TARGET_NR_mmap2)
 static void
-print_mmap(void *cpu_env, const struct syscallname *name,
+print_mmap(CPUArchState *cpu_env, const struct syscallname *name,
            abi_long arg0, abi_long arg1, abi_long arg2,
            abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3550,7 +3550,7 @@ print_mmap(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_mprotect
 static void
-print_mprotect(void *cpu_env, const struct syscallname *name,
+print_mprotect(CPUArchState *cpu_env, const struct syscallname *name,
                abi_long arg0, abi_long arg1, abi_long arg2,
                abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3564,7 +3564,7 @@ print_mprotect(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_munmap
 static void
-print_munmap(void *cpu_env, const struct syscallname *name,
+print_munmap(CPUArchState *cpu_env, const struct syscallname *name,
              abi_long arg0, abi_long arg1, abi_long arg2,
              abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3617,7 +3617,7 @@ if( cmd == val ) { \
 }
 
 static void
-print_futex(void *cpu_env, const struct syscallname *name,
+print_futex(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg0, abi_long arg1, abi_long arg2,
             abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3634,7 +3634,7 @@ print_futex(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_kill
 static void
-print_kill(void *cpu_env, const struct syscallname *name,
+print_kill(CPUArchState *cpu_env, const struct syscallname *name,
            abi_long arg0, abi_long arg1, abi_long arg2,
            abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3647,7 +3647,7 @@ print_kill(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_tkill
 static void
-print_tkill(void *cpu_env, const struct syscallname *name,
+print_tkill(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg0, abi_long arg1, abi_long arg2,
             abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3660,7 +3660,7 @@ print_tkill(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_tgkill
 static void
-print_tgkill(void *cpu_env, const struct syscallname *name,
+print_tgkill(CPUArchState *cpu_env, const struct syscallname *name,
              abi_long arg0, abi_long arg1, abi_long arg2,
              abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3674,7 +3674,7 @@ print_tgkill(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_statx
 static void
-print_statx(void *cpu_env, const struct syscallname *name,
+print_statx(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg0, abi_long arg1, abi_long arg2,
             abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3690,7 +3690,7 @@ print_statx(void *cpu_env, const struct syscallname *name,
 
 #ifdef TARGET_NR_ioctl
 static void
-print_ioctl(void *cpu_env, const struct syscallname *name,
+print_ioctl(CPUArchState *cpu_env, const struct syscallname *name,
             abi_long arg0, abi_long arg1, abi_long arg2,
             abi_long arg3, abi_long arg4, abi_long arg5)
 {
@@ -3775,7 +3775,7 @@ static int nsyscalls = ARRAY_SIZE(scnames);
  * The public interface to this module.
  */
 void
-print_syscall(void *cpu_env, int num,
+print_syscall(CPUArchState *cpu_env, int num,
               abi_long arg1, abi_long arg2, abi_long arg3,
               abi_long arg4, abi_long arg5, abi_long arg6)
 {
@@ -3804,7 +3804,7 @@ print_syscall(void *cpu_env, int num,
 
 
 void
-print_syscall_ret(void *cpu_env, int num, abi_long ret,
+print_syscall_ret(CPUArchState *cpu_env, int num, abi_long ret,
                   abi_long arg1, abi_long arg2, abi_long arg3,
                   abi_long arg4, abi_long arg5, abi_long arg6)
 {
diff --git a/linux-user/strace.h b/linux-user/strace.h
index 1e232d07fc..d5e7f26bcb 100644
--- a/linux-user/strace.h
+++ b/linux-user/strace.h
@@ -18,10 +18,10 @@
 #ifndef LINUX_USER_STRACE_H
 #define LINUX_USER_STRACE_H
 
-void print_syscall(void *cpu_env, int num,
+void print_syscall(CPUArchState *cpu_env, int num,
                    abi_long arg1, abi_long arg2, abi_long arg3,
                    abi_long arg4, abi_long arg5, abi_long arg6);
-void print_syscall_ret(void *cpu_env, int num, abi_long ret,
+void print_syscall_ret(CPUArchState *cpu_env, int num, abi_long ret,
                        abi_long arg1, abi_long arg2, abi_long arg3,
                        abi_long arg4, abi_long arg5, abi_long arg6);
 /**
diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 488facb356..2395988e66 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -1598,7 +1598,7 @@ static abi_long do_pipe2(int host_pipe[], int flags)
 #endif
 }
 
-static abi_long do_pipe(void *cpu_env, abi_ulong pipedes,
+static abi_long do_pipe(CPUArchState *cpu_env, abi_ulong pipedes,
                         int flags, int is_pipe2)
 {
     int host_pipe[2];
@@ -7252,7 +7252,7 @@ void syscall_init(void)
 }
 
 #ifdef TARGET_NR_truncate64
-static inline abi_long target_truncate64(void *cpu_env, const char *arg1,
+static inline abi_long target_truncate64(CPUArchState *cpu_env, const char *arg1,
                                          abi_long arg2,
                                          abi_long arg3,
                                          abi_long arg4)
@@ -7266,7 +7266,7 @@ static inline abi_long target_truncate64(void *cpu_env, const char *arg1,
 #endif
 
 #ifdef TARGET_NR_ftruncate64
-static inline abi_long target_ftruncate64(void *cpu_env, abi_long arg1,
+static inline abi_long target_ftruncate64(CPUArchState *cpu_env, abi_long arg1,
                                           abi_long arg2,
                                           abi_long arg3,
                                           abi_long arg4)
@@ -7561,7 +7561,7 @@ static inline int target_to_host_mlockall_arg(int arg)
 #if (defined(TARGET_NR_stat64) || defined(TARGET_NR_lstat64) ||     \
      defined(TARGET_NR_fstat64) || defined(TARGET_NR_fstatat64) ||  \
      defined(TARGET_NR_newfstatat))
-static inline abi_long host_to_target_stat64(void *cpu_env,
+static inline abi_long host_to_target_stat64(CPUArchState *cpu_env,
                                              abi_ulong target_addr,
                                              struct stat *host_st)
 {
@@ -7963,7 +7963,7 @@ int host_to_target_waitstatus(int status)
     return status;
 }
 
-static int open_self_cmdline(void *cpu_env, int fd)
+static int open_self_cmdline(CPUArchState *cpu_env, int fd)
 {
     CPUState *cpu = env_cpu((CPUArchState *)cpu_env);
     struct linux_binprm *bprm = ((TaskState *)cpu->opaque)->bprm;
@@ -7980,7 +7980,7 @@ static int open_self_cmdline(void *cpu_env, int fd)
     return 0;
 }
 
-static int open_self_maps(void *cpu_env, int fd)
+static int open_self_maps(CPUArchState *cpu_env, int fd)
 {
     CPUState *cpu = env_cpu((CPUArchState *)cpu_env);
     TaskState *ts = cpu->opaque;
@@ -8042,7 +8042,7 @@ static int open_self_maps(void *cpu_env, int fd)
     return 0;
 }
 
-static int open_self_stat(void *cpu_env, int fd)
+static int open_self_stat(CPUArchState *cpu_env, int fd)
 {
     CPUState *cpu = env_cpu((CPUArchState *)cpu_env);
     TaskState *ts = cpu->opaque;
@@ -8080,7 +8080,7 @@ static int open_self_stat(void *cpu_env, int fd)
     return 0;
 }
 
-static int open_self_auxv(void *cpu_env, int fd)
+static int open_self_auxv(CPUArchState *cpu_env, int fd)
 {
     CPUState *cpu = env_cpu((CPUArchState *)cpu_env);
     TaskState *ts = cpu->opaque;
@@ -8143,7 +8143,7 @@ static int is_proc(const char *filename, const char *entry)
 #endif
 
 #if HOST_BIG_ENDIAN != TARGET_BIG_ENDIAN
-static int open_net_route(void *cpu_env, int fd)
+static int open_net_route(CPUArchState *cpu_env, int fd)
 {
     FILE *fp;
     char *line = NULL;
@@ -8188,7 +8188,7 @@ static int open_net_route(void *cpu_env, int fd)
 #endif
 
 #if defined(TARGET_SPARC)
-static int open_cpuinfo(void *cpu_env, int fd)
+static int open_cpuinfo(CPUArchState *cpu_env, int fd)
 {
     dprintf(fd, "type\t\t: sun4u\n");
     return 0;
@@ -8196,7 +8196,7 @@ static int open_cpuinfo(void *cpu_env, int fd)
 #endif
 
 #if defined(TARGET_HPPA)
-static int open_cpuinfo(void *cpu_env, int fd)
+static int open_cpuinfo(CPUArchState *cpu_env, int fd)
 {
     dprintf(fd, "cpu family\t: PA-RISC 1.1e\n");
     dprintf(fd, "cpu\t\t: PA7300LC (PCX-L2)\n");
@@ -8208,18 +8208,18 @@ static int open_cpuinfo(void *cpu_env, int fd)
 #endif
 
 #if defined(TARGET_M68K)
-static int open_hardware(void *cpu_env, int fd)
+static int open_hardware(CPUArchState *cpu_env, int fd)
 {
     dprintf(fd, "Model:\t\tqemu-m68k\n");
     return 0;
 }
 #endif
 
-static int do_openat(void *cpu_env, int dirfd, const char *pathname, int flags, mode_t mode)
+static int do_openat(CPUArchState *cpu_env, int dirfd, const char *pathname, int flags, mode_t mode)
 {
     struct fake_open {
         const char *filename;
-        int (*fill)(void *cpu_env, int fd);
+        int (*fill)(CPUArchState *cpu_env, int fd);
         int (*cmp)(const char *s1, const char *s2);
     };
     const struct fake_open *fake_open;
@@ -8531,7 +8531,7 @@ _syscall2(int, pivot_root, const char *, new_root, const char *, put_old)
  * of syscall results, can be performed.
  * All errnos that do_syscall() returns must be -TARGET_<errcode>.
  */
-static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1,
+static abi_long do_syscall1(CPUArchState *cpu_env, int num, abi_long arg1,
                             abi_long arg2, abi_long arg3, abi_long arg4,
                             abi_long arg5, abi_long arg6, abi_long arg7,
                             abi_long arg8)
@@ -13155,7 +13155,7 @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1,
     return ret;
 }
 
-abi_long do_syscall(void *cpu_env, int num, abi_long arg1,
+abi_long do_syscall(CPUArchState *cpu_env, int num, abi_long arg1,
                     abi_long arg2, abi_long arg3, abi_long arg4,
                     abi_long arg5, abi_long arg6, abi_long arg7,
                     abi_long arg8)
diff --git a/linux-user/uname.c b/linux-user/uname.c
index 415fdf31b6..1e3e98156d 100644
--- a/linux-user/uname.c
+++ b/linux-user/uname.c
@@ -28,7 +28,7 @@
  * NB: the default emulated CPU ("any") might not match any existing CPU, e.g.
  * on ARM it has all features turned on, so there is no perfect arch string to
  * return here */
-const char *cpu_to_uname_machine(void *cpu_env)
+const char *cpu_to_uname_machine(CPUArchState *cpu_env)
 {
 #if defined(TARGET_ARM) && !defined(TARGET_AARCH64)
 
diff --git a/linux-user/uname.h b/linux-user/uname.h
index 4503094211..4ae563f46c 100644
--- a/linux-user/uname.h
+++ b/linux-user/uname.h
@@ -4,7 +4,7 @@
 #include <sys/utsname.h>
 #include <linux/utsname.h>
 
-const char *cpu_to_uname_machine(void *cpu_env);
+const char *cpu_to_uname_machine(CPUArchState *cpu_env);
 int sys_uname(struct new_utsname *buf);
 
 #endif /* UNAME_H */
diff --git a/linux-user/user-internals.h b/linux-user/user-internals.h
index ddc260e465..0bb26331c1 100644
--- a/linux-user/user-internals.h
+++ b/linux-user/user-internals.h
@@ -59,7 +59,7 @@ int info_is_fdpic(struct image_info *info);
 
 void target_set_brk(abi_ulong new_brk);
 void syscall_init(void);
-abi_long do_syscall(void *cpu_env, int num, abi_long arg1,
+abi_long do_syscall(CPUArchState *cpu_env, int num, abi_long arg1,
                     abi_long arg2, abi_long arg3, abi_long arg4,
                     abi_long arg5, abi_long arg6, abi_long arg7,
                     abi_long arg8);
@@ -132,22 +132,22 @@ void print_termios(void *arg);
 
 /* ARM EABI and MIPS expect 64bit types aligned even on pairs or registers */
 #ifdef TARGET_ARM
-static inline int regpairs_aligned(void *cpu_env, int num)
+static inline int regpairs_aligned(CPUArchState *cpu_env, int num)
 {
     return ((((CPUARMState *)cpu_env)->eabi) == 1) ;
 }
 #elif defined(TARGET_MIPS) && defined(TARGET_ABI_MIPSO32)
-static inline int regpairs_aligned(void *cpu_env, int num) { return 1; }
+static inline int regpairs_aligned(CPUArchState *cpu_env, int num) { return 1; }
 #elif defined(TARGET_PPC) && !defined(TARGET_PPC64)
 /*
  * SysV AVI for PPC32 expects 64bit parameters to be passed on odd/even pairs
  * of registers which translates to the same as ARM/MIPS, because we start with
  * r3 as arg1
  */
-static inline int regpairs_aligned(void *cpu_env, int num) { return 1; }
+static inline int regpairs_aligned(CPUArchState *cpu_env, int num) { return 1; }
 #elif defined(TARGET_SH4)
 /* SH4 doesn't align register pairs, except for p{read,write}64 */
-static inline int regpairs_aligned(void *cpu_env, int num)
+static inline int regpairs_aligned(CPUArchState *cpu_env, int num)
 {
     switch (num) {
     case TARGET_NR_pread64:
@@ -159,11 +159,11 @@ static inline int regpairs_aligned(void *cpu_env, int num)
     }
 }
 #elif defined(TARGET_XTENSA)
-static inline int regpairs_aligned(void *cpu_env, int num) { return 1; }
+static inline int regpairs_aligned(CPUArchState *cpu_env, int num) { return 1; }
 #elif defined(TARGET_HEXAGON)
-static inline int regpairs_aligned(void *cpu_env, int num) { return 1; }
+static inline int regpairs_aligned(CPUArchState *cpu_env, int num) { return 1; }
 #else
-static inline int regpairs_aligned(void *cpu_env, int num) { return 0; }
+static inline int regpairs_aligned(CPUArchState *cpu_env, int num) { return 0; }
 #endif
 
 /**

From 0effdc29b53240b1b2f25e48a61a287558e8ad6f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
Date: Mon, 9 May 2022 22:57:28 +0200
Subject: [PATCH 460/935] linux-user: Remove pointless CPU{ARCH}State casts
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220509205728.51912-4-philippe.mathieu.daude@gmail.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/syscall.c        | 49 +++++++++++++++++--------------------
 linux-user/uname.c          |  2 +-
 linux-user/user-internals.h |  2 +-
 3 files changed, 25 insertions(+), 28 deletions(-)

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 2395988e66..f55cdebee5 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -1612,16 +1612,16 @@ static abi_long do_pipe(CPUArchState *cpu_env, abi_ulong pipedes,
        pipe syscall, but didn't replicate this into the pipe2 syscall.  */
     if (!is_pipe2) {
 #if defined(TARGET_ALPHA)
-        ((CPUAlphaState *)cpu_env)->ir[IR_A4] = host_pipe[1];
+        cpu_env->ir[IR_A4] = host_pipe[1];
         return host_pipe[0];
 #elif defined(TARGET_MIPS)
-        ((CPUMIPSState*)cpu_env)->active_tc.gpr[3] = host_pipe[1];
+        cpu_env->active_tc.gpr[3] = host_pipe[1];
         return host_pipe[0];
 #elif defined(TARGET_SH4)
-        ((CPUSH4State*)cpu_env)->gregs[1] = host_pipe[1];
+        cpu_env->gregs[1] = host_pipe[1];
         return host_pipe[0];
 #elif defined(TARGET_SPARC)
-        ((CPUSPARCState*)cpu_env)->regwptr[1] = host_pipe[1];
+        cpu_env->regwptr[1] = host_pipe[1];
         return host_pipe[0];
 #endif
     }
@@ -7566,7 +7566,7 @@ static inline abi_long host_to_target_stat64(CPUArchState *cpu_env,
                                              struct stat *host_st)
 {
 #if defined(TARGET_ARM) && defined(TARGET_ABI32)
-    if (((CPUARMState *)cpu_env)->eabi) {
+    if (cpu_env->eabi) {
         struct target_eabi_stat64 *target_st;
 
         if (!lock_user_struct(VERIFY_WRITE, target_st, target_addr, 0))
@@ -7965,7 +7965,7 @@ int host_to_target_waitstatus(int status)
 
 static int open_self_cmdline(CPUArchState *cpu_env, int fd)
 {
-    CPUState *cpu = env_cpu((CPUArchState *)cpu_env);
+    CPUState *cpu = env_cpu(cpu_env);
     struct linux_binprm *bprm = ((TaskState *)cpu->opaque)->bprm;
     int i;
 
@@ -7982,7 +7982,7 @@ static int open_self_cmdline(CPUArchState *cpu_env, int fd)
 
 static int open_self_maps(CPUArchState *cpu_env, int fd)
 {
-    CPUState *cpu = env_cpu((CPUArchState *)cpu_env);
+    CPUState *cpu = env_cpu(cpu_env);
     TaskState *ts = cpu->opaque;
     GSList *map_info = read_self_maps();
     GSList *s;
@@ -8044,7 +8044,7 @@ static int open_self_maps(CPUArchState *cpu_env, int fd)
 
 static int open_self_stat(CPUArchState *cpu_env, int fd)
 {
-    CPUState *cpu = env_cpu((CPUArchState *)cpu_env);
+    CPUState *cpu = env_cpu(cpu_env);
     TaskState *ts = cpu->opaque;
     g_autoptr(GString) buf = g_string_new(NULL);
     int i;
@@ -8082,7 +8082,7 @@ static int open_self_stat(CPUArchState *cpu_env, int fd)
 
 static int open_self_auxv(CPUArchState *cpu_env, int fd)
 {
-    CPUState *cpu = env_cpu((CPUArchState *)cpu_env);
+    CPUState *cpu = env_cpu(cpu_env);
     TaskState *ts = cpu->opaque;
     abi_ulong auxv = ts->info->saved_auxv;
     abi_ulong len = ts->info->auxv_len;
@@ -8892,7 +8892,7 @@ static abi_long do_syscall1(CPUArchState *cpu_env, int num, abi_long arg1,
 #if defined(TARGET_NR_getxpid) && defined(TARGET_ALPHA)
     /* Alpha specific */
     case TARGET_NR_getxpid:
-        ((CPUAlphaState *)cpu_env)->ir[IR_A4] = getppid();
+        cpu_env->ir[IR_A4] = getppid();
         return get_errno(getpid());
 #endif
 #ifdef TARGET_NR_getpid
@@ -9415,7 +9415,7 @@ static abi_long do_syscall1(CPUArchState *cpu_env, int num, abi_long arg1,
             if (!is_error(ret)) {
                 host_to_target_old_sigset(&mask, &oldset);
                 ret = mask;
-                ((CPUAlphaState *)cpu_env)->ir[IR_V0] = 0; /* force no error */
+                cpu_env->ir[IR_V0] = 0; /* force no error */
             }
 #else
             sigset_t set, oldset, *set_ptr;
@@ -10035,7 +10035,7 @@ static abi_long do_syscall1(CPUArchState *cpu_env, int num, abi_long arg1,
         }
 #ifdef TARGET_ALPHA
         /* Return value is the unbiased priority.  Signal no error.  */
-        ((CPUAlphaState *)cpu_env)->ir[IR_V0] = 0;
+        cpu_env->ir[IR_V0] = 0;
 #else
         /* Return value is a biased priority to avoid negative numbers.  */
         ret = 20 - ret;
@@ -11415,7 +11415,7 @@ static abi_long do_syscall1(CPUArchState *cpu_env, int num, abi_long arg1,
          {
             uid_t euid;
             euid=geteuid();
-            ((CPUAlphaState *)cpu_env)->ir[IR_A4]=euid;
+            cpu_env->ir[IR_A4]=euid;
          }
         return get_errno(getuid());
 #endif
@@ -11425,7 +11425,7 @@ static abi_long do_syscall1(CPUArchState *cpu_env, int num, abi_long arg1,
          {
             uid_t egid;
             egid=getegid();
-            ((CPUAlphaState *)cpu_env)->ir[IR_A4]=egid;
+            cpu_env->ir[IR_A4]=egid;
          }
         return get_errno(getgid());
 #endif
@@ -11437,7 +11437,7 @@ static abi_long do_syscall1(CPUArchState *cpu_env, int num, abi_long arg1,
           case TARGET_GSI_IEEE_FP_CONTROL:
             {
                 uint64_t fpcr = cpu_alpha_load_fpcr(cpu_env);
-                uint64_t swcr = ((CPUAlphaState *)cpu_env)->swcr;
+                uint64_t swcr = cpu_env->swcr;
 
                 swcr &= ~SWCR_STATUS_MASK;
                 swcr |= (fpcr >> 35) & SWCR_STATUS_MASK;
@@ -11479,8 +11479,7 @@ static abi_long do_syscall1(CPUArchState *cpu_env, int num, abi_long arg1,
                  * could be queried.  Therefore, we store the status
                  * bits only in FPCR.
                  */
-                ((CPUAlphaState *)cpu_env)->swcr
-                    = swcr & (SWCR_TRAP_ENABLE_MASK | SWCR_MAP_MASK);
+                cpu_env->swcr = swcr & (SWCR_TRAP_ENABLE_MASK | SWCR_MAP_MASK);
 
                 fpcr = cpu_alpha_load_fpcr(cpu_env);
                 fpcr &= ((uint64_t)FPCR_DYN_MASK << 32);
@@ -11504,7 +11503,7 @@ static abi_long do_syscall1(CPUArchState *cpu_env, int num, abi_long arg1,
                 fex = alpha_ieee_fpcr_to_swcr(fpcr);
                 fex = exc & ~fex;
                 fex >>= SWCR_STATUS_TO_EXCSUM_SHIFT;
-                fex &= ((CPUArchState *)cpu_env)->swcr;
+                fex &= (cpu_env)->swcr;
 
                 /* Update the hardware fpcr.  */
                 fpcr |= alpha_ieee_swcr_to_fpcr(exc);
@@ -11536,9 +11535,8 @@ static abi_long do_syscall1(CPUArchState *cpu_env, int num, abi_long arg1,
                     info.si_signo = SIGFPE;
                     info.si_errno = 0;
                     info.si_code = si_code;
-                    info._sifields._sigfault._addr
-                        = ((CPUArchState *)cpu_env)->pc;
-                    queue_signal((CPUArchState *)cpu_env, info.si_signo,
+                    info._sifields._sigfault._addr = (cpu_env)->pc;
+                    queue_signal(cpu_env, info.si_signo,
                                  QEMU_SI_FAULT, &info);
                 }
                 ret = 0;
@@ -11824,7 +11822,7 @@ static abi_long do_syscall1(CPUArchState *cpu_env, int num, abi_long arg1,
         to_flock64_fn *copyto = copy_to_user_flock64;
 
 #ifdef TARGET_ARM
-        if (!((CPUARMState *)cpu_env)->eabi) {
+        if (!cpu_env->eabi) {
             copyfrom = copy_from_user_oabi_flock64;
             copyto = copy_to_user_oabi_flock64;
         }
@@ -12052,13 +12050,13 @@ static abi_long do_syscall1(CPUArchState *cpu_env, int num, abi_long arg1,
 #ifdef TARGET_NR_set_thread_area
     case TARGET_NR_set_thread_area:
 #if defined(TARGET_MIPS)
-      ((CPUMIPSState *) cpu_env)->active_tc.CP0_UserLocal = arg1;
+      cpu_env->active_tc.CP0_UserLocal = arg1;
       return 0;
 #elif defined(TARGET_CRIS)
       if (arg1 & 0xff)
           ret = -TARGET_EINVAL;
       else {
-          ((CPUCRISState *) cpu_env)->pregs[PR_PID] = arg1;
+          cpu_env->pregs[PR_PID] = arg1;
           ret = 0;
       }
       return ret;
@@ -12785,8 +12783,7 @@ static abi_long do_syscall1(CPUArchState *cpu_env, int num, abi_long arg1,
             info.si_errno = 0;
             info.si_code = TARGET_SEGV_MAPERR;
             info._sifields._sigfault._addr = arg6;
-            queue_signal((CPUArchState *)cpu_env, info.si_signo,
-                         QEMU_SI_FAULT, &info);
+            queue_signal(cpu_env, info.si_signo, QEMU_SI_FAULT, &info);
             ret = 0xdeadbeef;
 
         }
diff --git a/linux-user/uname.c b/linux-user/uname.c
index 1e3e98156d..32f71f2492 100644
--- a/linux-user/uname.c
+++ b/linux-user/uname.c
@@ -54,7 +54,7 @@ const char *cpu_to_uname_machine(CPUArchState *cpu_env)
     return "armv5te" utsname_suffix;
 #elif defined(TARGET_I386) && !defined(TARGET_X86_64)
     /* see arch/x86/kernel/cpu/bugs.c: check_bugs(), 386, 486, 586, 686 */
-    CPUState *cpu = env_cpu((CPUX86State *)cpu_env);
+    CPUState *cpu = env_cpu(cpu_env);
     int family = object_property_get_int(OBJECT(cpu), "family", NULL);
     if (family == 4) {
         return "i486";
diff --git a/linux-user/user-internals.h b/linux-user/user-internals.h
index 0bb26331c1..6175ce53db 100644
--- a/linux-user/user-internals.h
+++ b/linux-user/user-internals.h
@@ -134,7 +134,7 @@ void print_termios(void *arg);
 #ifdef TARGET_ARM
 static inline int regpairs_aligned(CPUArchState *cpu_env, int num)
 {
-    return ((((CPUARMState *)cpu_env)->eabi) == 1) ;
+    return cpu_env->eabi == 1;
 }
 #elif defined(TARGET_MIPS) && defined(TARGET_ABI_MIPSO32)
 static inline int regpairs_aligned(CPUArchState *cpu_env, int num) { return 1; }

From 9a12adc704f977b31d6cde6cb26e5424c128907b Mon Sep 17 00:00:00 2001
From: Ilya Leoshkevich <iii@linux.ibm.com>
Date: Wed, 4 May 2022 00:51:56 +0200
Subject: [PATCH 461/935] linux-user/s390x: Fix unwinding from signal handlers

Commit 31330e6cecfd ("linux-user/s390x: Implement setup_sigtramp")
removed an unused field from rt_sigframe, disturbing offsets of other
fields and breaking unwinding from signal handlers (e.g. libgcc's
s390_fallback_frame() relies on this struct having a specific layout).
Restore the field and add a comment.

Reported-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Fixes: 31330e6cecfd ("linux-user/s390x: Implement setup_sigtramp")
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220503225157.1696774-2-iii@linux.ibm.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/s390x/signal.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/linux-user/s390x/signal.c b/linux-user/s390x/signal.c
index f47713e04a..4979c4b017 100644
--- a/linux-user/s390x/signal.c
+++ b/linux-user/s390x/signal.c
@@ -84,6 +84,11 @@ struct target_ucontext {
 
 typedef struct {
     uint8_t callee_used_stack[__SIGNAL_FRAMESIZE];
+    /*
+     * This field is no longer initialized by the kernel, but it's still a part
+     * of the ABI.
+     */
+    uint16_t svc_insn;
     struct target_siginfo info;
     struct target_ucontext uc;
 } rt_sigframe;

From 1a75b14038bd77b0af7d2ddb0cf230b31f75623e Mon Sep 17 00:00:00 2001
From: Ilya Leoshkevich <iii@linux.ibm.com>
Date: Wed, 4 May 2022 00:51:57 +0200
Subject: [PATCH 462/935] tests/tcg/s390x: Test unwinding from signal handlers

Add a small test to prevent regressions.

Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20220503225157.1696774-3-iii@linux.ibm.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 tests/tcg/s390x/signals-s390x.c | 69 ++++++++++++++++++++++++++-------
 1 file changed, 55 insertions(+), 14 deletions(-)

diff --git a/tests/tcg/s390x/signals-s390x.c b/tests/tcg/s390x/signals-s390x.c
index dc2f8ee59a..48c3b6cdfd 100644
--- a/tests/tcg/s390x/signals-s390x.c
+++ b/tests/tcg/s390x/signals-s390x.c
@@ -1,4 +1,5 @@
 #include <assert.h>
+#include <execinfo.h>
 #include <signal.h>
 #include <string.h>
 #include <sys/mman.h>
@@ -11,22 +12,28 @@
  * inline asm is used instead.
  */
 
+#define DEFINE_ASM_FUNCTION(name, body) \
+    asm(".globl " #name "\n" \
+        #name ":\n" \
+        ".cfi_startproc\n" \
+        body "\n" \
+        "br %r14\n" \
+        ".cfi_endproc");
+
 void illegal_op(void);
-void after_illegal_op(void);
-asm(".globl\tillegal_op\n"
-    "illegal_op:\t.byte\t0x00,0x00\n"
-    "\t.globl\tafter_illegal_op\n"
-    "after_illegal_op:\tbr\t%r14");
+extern const char after_illegal_op;
+DEFINE_ASM_FUNCTION(illegal_op,
+    ".byte 0x00,0x00\n"
+    ".globl after_illegal_op\n"
+    "after_illegal_op:")
 
 void stg(void *dst, unsigned long src);
-asm(".globl\tstg\n"
-    "stg:\tstg\t%r3,0(%r2)\n"
-    "\tbr\t%r14");
+DEFINE_ASM_FUNCTION(stg, "stg %r3,0(%r2)")
 
 void mvc_8(void *dst, void *src);
-asm(".globl\tmvc_8\n"
-    "mvc_8:\tmvc\t0(8,%r2),0(%r3)\n"
-    "\tbr\t%r14");
+DEFINE_ASM_FUNCTION(mvc_8, "mvc 0(8,%r2),0(%r3)")
+
+extern const char return_from_main_1;
 
 static void safe_puts(const char *s)
 {
@@ -49,8 +56,9 @@ static struct {
 
 static void handle_signal(int sig, siginfo_t *info, void *ucontext)
 {
+    int err, i, n_frames;
+    void *frames[16];
     void *page;
-    int err;
 
     if (sig != expected.sig) {
         safe_puts("[  FAILED  ] wrong signal");
@@ -86,6 +94,17 @@ static void handle_signal(int sig, siginfo_t *info, void *ucontext)
     default:
         break;
     }
+
+    n_frames = backtrace(frames, sizeof(frames) / sizeof(frames[0]));
+    for (i = 0; i < n_frames; i++) {
+        if (frames[i] == &return_from_main_1) {
+            break;
+        }
+    }
+    if (i == n_frames) {
+        safe_puts("[  FAILED  ] backtrace() is broken");
+        _exit(1);
+    }
 }
 
 static void check_sigsegv(void *func, enum exception exception,
@@ -122,7 +141,7 @@ static void check_sigsegv(void *func, enum exception exception,
     assert(err == 0);
 }
 
-int main(void)
+int main_1(void)
 {
     struct sigaction act;
     int err;
@@ -138,7 +157,7 @@ int main(void)
     safe_puts("[ RUN      ] Operation exception");
     expected.sig = SIGILL;
     expected.addr = illegal_op;
-    expected.psw_addr = (unsigned long)after_illegal_op;
+    expected.psw_addr = (unsigned long)&after_illegal_op;
     expected.exception = exception_operation;
     illegal_op();
     safe_puts("[       OK ]");
@@ -163,3 +182,25 @@ int main(void)
 
     _exit(0);
 }
+
+/*
+ * Define main() in assembly in order to test that unwinding from signal
+ * handlers until main() works. This way we can define a specific point that
+ * the unwinder should reach. This is also better than defining main() in C
+ * and using inline assembly to call main_1(), since it's not easy to get all
+ * the clobbers right.
+ */
+
+DEFINE_ASM_FUNCTION(main,
+    "stmg %r14,%r15,112(%r15)\n"
+    ".cfi_offset 14,-48\n"
+    ".cfi_offset 15,-40\n"
+    "lay %r15,-160(%r15)\n"
+    ".cfi_def_cfa_offset 320\n"
+    "brasl %r14,main_1\n"
+    ".globl return_from_main_1\n"
+    "return_from_main_1:\n"
+    "lmg %r14,%r15,272(%r15)\n"
+    ".cfi_restore 15\n"
+    ".cfi_restore 14\n"
+    ".cfi_def_cfa_offset 160");

From 565a84c1e61acb6e2bce03e5ca88b5ce400231ca Mon Sep 17 00:00:00 2001
From: Ilya Leoshkevich <iii@linux.ibm.com>
Date: Wed, 4 May 2022 13:48:19 +0200
Subject: [PATCH 463/935] linux-user/host/s390: Treat EX and EXRL as writes

clang-built s390x branch-relative-long test fails on clang-built s390x
QEMU due to the following sequence of events:

- The test zeroes out a code page, clang generates exrl+xc for this.

- do_helper_xc() is called. Clang generates exrl+xc there as well.

- Since there already exists a TB for the code in question, its page is
  read-only and SIGSEGV is raised.

- host_signal_handler() calls host_signal_write() and the latter does
  not recognize exrl as a write. Therefore page_unprotect() is not
  called and the signal is forwarded to the test.

Fix by treating EXRL (and EX, just in case) as writes. There may be
false positives, but they will lead only to an extra page_unprotect()
call.

Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220504114819.1729737-1-iii@linux.ibm.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/include/host/s390/host-signal.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/linux-user/include/host/s390/host-signal.h b/linux-user/include/host/s390/host-signal.h
index 6f191e64d7..25fefa00bd 100644
--- a/linux-user/include/host/s390/host-signal.h
+++ b/linux-user/include/host/s390/host-signal.h
@@ -50,6 +50,7 @@ static inline bool host_signal_write(siginfo_t *info, host_sigcontext *uc)
     case 0x50: /* ST */
     case 0x42: /* STC */
     case 0x40: /* STH */
+    case 0x44: /* EX */
     case 0xba: /* CS */
     case 0xbb: /* CDS */
         return true;
@@ -61,6 +62,12 @@ static inline bool host_signal_write(siginfo_t *info, host_sigcontext *uc)
             return true;
         }
         break;
+    case 0xc6: /* RIL-b format insns */
+        switch (pinsn[0] & 0xf) {
+        case 0x0: /* EXRL */
+            return true;
+        }
+        break;
     case 0xc8: /* SSF format insns */
         switch (pinsn[0] & 0xf) {
         case 0x2: /* CSST */

From d6cd3ae0ebdfab9922f932dc303e1faa618ea547 Mon Sep 17 00:00:00 2001
From: Dylan Reid <dylan@rivosinc.com>
Date: Fri, 6 May 2022 16:54:57 +0000
Subject: [PATCH 464/935] target/riscv: Fix VS mode hypervisor CSR access

VS mode access to hypervisor CSRs should generate virtual, not illegal,
instruction exceptions.

Don't return early and indicate an illegal instruction exception when
accessing a hypervisor CSR from VS mode. Instead, fall through to the
`hmode` predicate to return the correct virtual instruction exception.

Signed-off-by: Dylan Reid <dgreid@rivosinc.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20220506165456.297058-1-dgreid@rivosinc.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/csr.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/target/riscv/csr.c b/target/riscv/csr.c
index 3500e07f92..4ea7df02c9 100644
--- a/target/riscv/csr.c
+++ b/target/riscv/csr.c
@@ -3141,13 +3141,13 @@ static inline RISCVException riscv_csrrw_check(CPURISCVState *env,
 #if !defined(CONFIG_USER_ONLY)
     int effective_priv = env->priv;
 
-    if (riscv_has_ext(env, RVH) &&
-        env->priv == PRV_S &&
-        !riscv_cpu_virt_enabled(env)) {
+    if (riscv_has_ext(env, RVH) && env->priv == PRV_S) {
         /*
-         * We are in S mode without virtualisation, therefore we are in HS Mode.
+         * We are in either HS or VS mode.
          * Add 1 to the effective privledge level to allow us to access the
-         * Hypervisor CSRs.
+         * Hypervisor CSRs. The `hmode` predicate will determine if access
+         * should be allowed(HS) or if a virtual instruction exception should be
+         * raised(VS).
          */
         effective_priv++;
     }

From 02b511985e33d71859943682860f629ead5bd20a Mon Sep 17 00:00:00 2001
From: eopXD <eop.chen@sifive.com>
Date: Thu, 5 May 2022 02:42:17 -0700
Subject: [PATCH 465/935] target/riscv: rvv: Fix early exit condition for whole
 register load/store

Vector whole register load instructions have EEW encoded in the opcode,
so we shouldn't take SEW here. Vector whole register store instructions
are always EEW=8.

Signed-off-by: eop Chen <eop.chen@sifive.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <165181414065.18540.14828125053334599921-0@git.sr.ht>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/insn_trans/trans_rvv.c.inc | 56 +++++++++++++------------
 1 file changed, 30 insertions(+), 26 deletions(-)

diff --git a/target/riscv/insn_trans/trans_rvv.c.inc b/target/riscv/insn_trans/trans_rvv.c.inc
index 90327509f7..391c61fe93 100644
--- a/target/riscv/insn_trans/trans_rvv.c.inc
+++ b/target/riscv/insn_trans/trans_rvv.c.inc
@@ -1118,10 +1118,10 @@ GEN_VEXT_TRANS(vle64ff_v, MO_64, r2nfvm, ldff_op, ld_us_check)
 typedef void gen_helper_ldst_whole(TCGv_ptr, TCGv, TCGv_env, TCGv_i32);
 
 static bool ldst_whole_trans(uint32_t vd, uint32_t rs1, uint32_t nf,
-                             gen_helper_ldst_whole *fn, DisasContext *s,
-                             bool is_store)
+                             uint32_t width, gen_helper_ldst_whole *fn,
+                             DisasContext *s, bool is_store)
 {
-    uint32_t evl = (s->cfg_ptr->vlen / 8) * nf / (1 << s->sew);
+    uint32_t evl = (s->cfg_ptr->vlen / 8) * nf / width;
     TCGLabel *over = gen_new_label();
     tcg_gen_brcondi_tl(TCG_COND_GEU, cpu_vstart, evl, over);
 
@@ -1153,38 +1153,42 @@ static bool ldst_whole_trans(uint32_t vd, uint32_t rs1, uint32_t nf,
  * load and store whole register instructions ignore vtype and vl setting.
  * Thus, we don't need to check vill bit. (Section 7.9)
  */
-#define GEN_LDST_WHOLE_TRANS(NAME, ARG_NF, IS_STORE)                      \
+#define GEN_LDST_WHOLE_TRANS(NAME, ARG_NF, WIDTH, IS_STORE)               \
 static bool trans_##NAME(DisasContext *s, arg_##NAME * a)                 \
 {                                                                         \
     if (require_rvv(s) &&                                                 \
         QEMU_IS_ALIGNED(a->rd, ARG_NF)) {                                 \
-        return ldst_whole_trans(a->rd, a->rs1, ARG_NF, gen_helper_##NAME, \
-                                s, IS_STORE);                             \
+        return ldst_whole_trans(a->rd, a->rs1, ARG_NF, WIDTH,             \
+                                gen_helper_##NAME, s, IS_STORE);          \
     }                                                                     \
     return false;                                                         \
 }
 
-GEN_LDST_WHOLE_TRANS(vl1re8_v,  1, false)
-GEN_LDST_WHOLE_TRANS(vl1re16_v, 1, false)
-GEN_LDST_WHOLE_TRANS(vl1re32_v, 1, false)
-GEN_LDST_WHOLE_TRANS(vl1re64_v, 1, false)
-GEN_LDST_WHOLE_TRANS(vl2re8_v,  2, false)
-GEN_LDST_WHOLE_TRANS(vl2re16_v, 2, false)
-GEN_LDST_WHOLE_TRANS(vl2re32_v, 2, false)
-GEN_LDST_WHOLE_TRANS(vl2re64_v, 2, false)
-GEN_LDST_WHOLE_TRANS(vl4re8_v,  4, false)
-GEN_LDST_WHOLE_TRANS(vl4re16_v, 4, false)
-GEN_LDST_WHOLE_TRANS(vl4re32_v, 4, false)
-GEN_LDST_WHOLE_TRANS(vl4re64_v, 4, false)
-GEN_LDST_WHOLE_TRANS(vl8re8_v,  8, false)
-GEN_LDST_WHOLE_TRANS(vl8re16_v, 8, false)
-GEN_LDST_WHOLE_TRANS(vl8re32_v, 8, false)
-GEN_LDST_WHOLE_TRANS(vl8re64_v, 8, false)
+GEN_LDST_WHOLE_TRANS(vl1re8_v,  1, 1, false)
+GEN_LDST_WHOLE_TRANS(vl1re16_v, 1, 2, false)
+GEN_LDST_WHOLE_TRANS(vl1re32_v, 1, 4, false)
+GEN_LDST_WHOLE_TRANS(vl1re64_v, 1, 8, false)
+GEN_LDST_WHOLE_TRANS(vl2re8_v,  2, 1, false)
+GEN_LDST_WHOLE_TRANS(vl2re16_v, 2, 2, false)
+GEN_LDST_WHOLE_TRANS(vl2re32_v, 2, 4, false)
+GEN_LDST_WHOLE_TRANS(vl2re64_v, 2, 8, false)
+GEN_LDST_WHOLE_TRANS(vl4re8_v,  4, 1, false)
+GEN_LDST_WHOLE_TRANS(vl4re16_v, 4, 2, false)
+GEN_LDST_WHOLE_TRANS(vl4re32_v, 4, 4, false)
+GEN_LDST_WHOLE_TRANS(vl4re64_v, 4, 8, false)
+GEN_LDST_WHOLE_TRANS(vl8re8_v,  8, 1, false)
+GEN_LDST_WHOLE_TRANS(vl8re16_v, 8, 2, false)
+GEN_LDST_WHOLE_TRANS(vl8re32_v, 8, 4, false)
+GEN_LDST_WHOLE_TRANS(vl8re64_v, 8, 8, false)
 
-GEN_LDST_WHOLE_TRANS(vs1r_v, 1, true)
-GEN_LDST_WHOLE_TRANS(vs2r_v, 2, true)
-GEN_LDST_WHOLE_TRANS(vs4r_v, 4, true)
-GEN_LDST_WHOLE_TRANS(vs8r_v, 8, true)
+/*
+ * The vector whole register store instructions are encoded similar to
+ * unmasked unit-stride store of elements with EEW=8.
+ */
+GEN_LDST_WHOLE_TRANS(vs1r_v, 1, 1, true)
+GEN_LDST_WHOLE_TRANS(vs2r_v, 2, 1, true)
+GEN_LDST_WHOLE_TRANS(vs4r_v, 4, 1, true)
+GEN_LDST_WHOLE_TRANS(vs8r_v, 8, 1, true)
 
 /*
  *** Vector Integer Arithmetic Instructions

From 77046729f943ce2055648e8339ddd688dd67dd83 Mon Sep 17 00:00:00 2001
From: Atish Patra <atishp@rivosinc.com>
Date: Fri, 13 May 2022 15:14:58 -0700
Subject: [PATCH 466/935] hw/intc: Pass correct hartid while updating mtimecmp

timecmp update function should be invoked with hartid for which
timecmp is being updated. The following patch passes the incorrect
hartid to the update function.

Fixes: e2f01f3c2e13 ("hw/intc: Make RISC-V ACLINT mtime MMIO register writable")

Signed-off-by: Atish Patra <atishp@rivosinc.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20220513221458.1192933-1-atishp@rivosinc.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 hw/intc/riscv_aclint.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/hw/intc/riscv_aclint.c b/hw/intc/riscv_aclint.c
index 0412edc982..e6bceceefd 100644
--- a/hw/intc/riscv_aclint.c
+++ b/hw/intc/riscv_aclint.c
@@ -233,7 +233,8 @@ static void riscv_aclint_mtimer_write(void *opaque, hwaddr addr,
                 continue;
             }
             riscv_aclint_mtimer_write_timecmp(mtimer, RISCV_CPU(cpu),
-                                              i, env->timecmp);
+                                              mtimer->hartid_base + i,
+                                              env->timecmp);
         }
         return;
     }

From 6047dcc2459fc6d1c49c4aa02e2e902dd3113856 Mon Sep 17 00:00:00 2001
From: Tsukasa OI <research_trasio@irq.a4lg.com>
Date: Tue, 10 May 2022 20:29:07 +0900
Subject: [PATCH 467/935] target/riscv: Move Zhinx* extensions on ISA string

This commit moves ISA string conversion for Zhinx and Zhinxmin extensions.
Because extension category ordering of "H" is going to be after "V",
their ordering is going to be valid (on canonical order).

Signed-off-by: Tsukasa OI <research_trasio@irq.a4lg.com>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <7a988aedb249b6709f9ce5464ff359b60958ca54.1652181972.git.research_trasio@irq.a4lg.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index ccacdee215..9f38e56316 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -999,8 +999,6 @@ static void riscv_isa_string_ext(RISCVCPU *cpu, char **isa_str, int max_str_len)
         ISA_EDATA_ENTRY(zfh, ext_zfh),
         ISA_EDATA_ENTRY(zfhmin, ext_zfhmin),
         ISA_EDATA_ENTRY(zfinx, ext_zfinx),
-        ISA_EDATA_ENTRY(zhinx, ext_zhinx),
-        ISA_EDATA_ENTRY(zhinxmin, ext_zhinxmin),
         ISA_EDATA_ENTRY(zdinx, ext_zdinx),
         ISA_EDATA_ENTRY(zba, ext_zba),
         ISA_EDATA_ENTRY(zbb, ext_zbb),
@@ -1021,6 +1019,8 @@ static void riscv_isa_string_ext(RISCVCPU *cpu, char **isa_str, int max_str_len)
         ISA_EDATA_ENTRY(zkt, ext_zkt),
         ISA_EDATA_ENTRY(zve32f, ext_zve32f),
         ISA_EDATA_ENTRY(zve64f, ext_zve64f),
+        ISA_EDATA_ENTRY(zhinx, ext_zhinx),
+        ISA_EDATA_ENTRY(zhinxmin, ext_zhinxmin),
         ISA_EDATA_ENTRY(svinval, ext_svinval),
         ISA_EDATA_ENTRY(svnapot, ext_svnapot),
         ISA_EDATA_ENTRY(svpbmt, ext_svpbmt),

From a4a9a4432e2bf280a989ca344466d7375db7993f Mon Sep 17 00:00:00 2001
From: Tsukasa OI <research_trasio@irq.a4lg.com>
Date: Tue, 10 May 2022 20:29:08 +0900
Subject: [PATCH 468/935] target/riscv: Add short-isa-string option

Because some operating systems don't correctly parse long ISA extension
string, this commit adds short-isa-string boolean option to disable
generating long ISA extension strings on Device Tree.

For instance, enabling Zfinx and Zdinx extensions and booting Linux (5.17 or
earlier) with FPU support caused a kernel panic.

Operating Systems which short-isa-string might be helpful:

1.  Linux (5.17 or earlier)
2.  FreeBSD (at least 14.0-CURRENT)
3.  OpenBSD (at least current development version)

Signed-off-by: Tsukasa OI <research_trasio@irq.a4lg.com>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <7c1fe5f06b0a7646a47e9bcdddb1042bb60c69c8.1652181972.git.research_trasio@irq.a4lg.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.c | 6 +++++-
 target/riscv/cpu.h | 2 ++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index 9f38e56316..dc93412395 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -879,6 +879,8 @@ static Property riscv_cpu_properties[] = {
     DEFINE_PROP_BOOL("x-aia", RISCVCPU, cfg.aia, false),
 
     DEFINE_PROP_UINT64("resetvec", RISCVCPU, cfg.resetvec, DEFAULT_RSTVEC),
+
+    DEFINE_PROP_BOOL("short-isa-string", RISCVCPU, cfg.short_isa_string, false),
     DEFINE_PROP_END_OF_LIST(),
 };
 
@@ -1049,7 +1051,9 @@ char *riscv_isa_string(RISCVCPU *cpu)
         }
     }
     *p = '\0';
-    riscv_isa_string_ext(cpu, &isa_str, maxlen);
+    if (!cpu->cfg.short_isa_string) {
+        riscv_isa_string_ext(cpu, &isa_str, maxlen);
+    }
     return isa_str;
 }
 
diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
index fe6c9a2c92..f5ff7294c6 100644
--- a/target/riscv/cpu.h
+++ b/target/riscv/cpu.h
@@ -425,6 +425,8 @@ struct RISCVCPUConfig {
     bool aia;
     bool debug;
     uint64_t resetvec;
+
+    bool short_isa_string;
 };
 
 typedef struct RISCVCPUConfig RISCVCPUConfig;

From 4bcfc391ac627155448951b45a8432eab91c2db9 Mon Sep 17 00:00:00 2001
From: Tsukasa OI <research_trasio@irq.a4lg.com>
Date: Sat, 14 May 2022 15:29:40 +0900
Subject: [PATCH 469/935] hw/riscv: Make CPU config error handling generous
 (virt/spike)

If specified CPU configuration is not valid, not just it prints error
message, it aborts and generates core dumps (depends on the operating
system).  This kind of error handling should be used only when a serious
runtime error occurs.

This commit makes error handling on CPU configuration more generous on
virt/spike machines.  It now just prints error message and quits (without
coredumps and aborts).

Signed-off-by: Tsukasa OI <research_trasio@irq.a4lg.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <d17381d3ea4992808cf1894f379ca67220f61b45.1652509778.git.research_trasio@irq.a4lg.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 hw/riscv/spike.c | 2 +-
 hw/riscv/virt.c  | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/riscv/spike.c b/hw/riscv/spike.c
index 068ba3493e..e41b6aa9f0 100644
--- a/hw/riscv/spike.c
+++ b/hw/riscv/spike.c
@@ -230,7 +230,7 @@ static void spike_board_init(MachineState *machine)
                                 base_hartid, &error_abort);
         object_property_set_int(OBJECT(&s->soc[i]), "num-harts",
                                 hart_count, &error_abort);
-        sysbus_realize(SYS_BUS_DEVICE(&s->soc[i]), &error_abort);
+        sysbus_realize(SYS_BUS_DEVICE(&s->soc[i]), &error_fatal);
 
         /* Core Local Interruptor (timer and IPI) for each socket */
         riscv_aclint_swi_create(
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 3326f4db96..244d6408b5 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -1351,7 +1351,7 @@ static void virt_machine_init(MachineState *machine)
                                 base_hartid, &error_abort);
         object_property_set_int(OBJECT(&s->soc[i]), "num-harts",
                                 hart_count, &error_abort);
-        sysbus_realize(SYS_BUS_DEVICE(&s->soc[i]), &error_abort);
+        sysbus_realize(SYS_BUS_DEVICE(&s->soc[i]), &error_fatal);
 
         if (!kvm_enabled()) {
             if (s->have_aclint) {

From 91a3387dc42b261e95eb402bf7d043b3a043209c Mon Sep 17 00:00:00 2001
From: Tsukasa OI <research_trasio@irq.a4lg.com>
Date: Sat, 14 May 2022 15:29:41 +0900
Subject: [PATCH 470/935] hw/riscv: Make CPU config error handling generous
 (sifive_e/u/opentitan)

If specified CPU configuration is not valid, not just it prints error
message, it aborts and generates core dumps (depends on the operating
system).  This kind of error handling should be used only when a serious
runtime error occurs.

This commit makes error handling on CPU configuration more generous on
sifive_e/u and opentitan machines.  It now just prints error message and
quits (without coredumps and aborts).

This is separate from spike/virt because it involves different type
(TYPE_RISCV_HART_ARRAY) on sifive_e/u and opentitan machines.

Signed-off-by: Tsukasa OI <research_trasio@irq.a4lg.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <09e61e58a7543da44bdb0e0f5368afc8903b4aa6.1652509778.git.research_trasio@irq.a4lg.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 hw/riscv/opentitan.c | 2 +-
 hw/riscv/sifive_e.c  | 2 +-
 hw/riscv/sifive_u.c  | 4 ++--
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/hw/riscv/opentitan.c b/hw/riscv/opentitan.c
index 2d401dcb23..4495a2c039 100644
--- a/hw/riscv/opentitan.c
+++ b/hw/riscv/opentitan.c
@@ -142,7 +142,7 @@ static void lowrisc_ibex_soc_realize(DeviceState *dev_soc, Error **errp)
     object_property_set_int(OBJECT(&s->cpus), "num-harts", ms->smp.cpus,
                             &error_abort);
     object_property_set_int(OBJECT(&s->cpus), "resetvec", 0x8080, &error_abort);
-    sysbus_realize(SYS_BUS_DEVICE(&s->cpus), &error_abort);
+    sysbus_realize(SYS_BUS_DEVICE(&s->cpus), &error_fatal);
 
     /* Boot ROM */
     memory_region_init_rom(&s->rom, OBJECT(dev_soc), "riscv.lowrisc.ibex.rom",
diff --git a/hw/riscv/sifive_e.c b/hw/riscv/sifive_e.c
index dcb87b6cfd..d65d2fd869 100644
--- a/hw/riscv/sifive_e.c
+++ b/hw/riscv/sifive_e.c
@@ -195,7 +195,7 @@ static void sifive_e_soc_realize(DeviceState *dev, Error **errp)
 
     object_property_set_str(OBJECT(&s->cpus), "cpu-type", ms->cpu_type,
                             &error_abort);
-    sysbus_realize(SYS_BUS_DEVICE(&s->cpus), &error_abort);
+    sysbus_realize(SYS_BUS_DEVICE(&s->cpus), &error_fatal);
 
     /* Mask ROM */
     memory_region_init_rom(&s->mask_rom, OBJECT(dev), "riscv.sifive.e.mrom",
diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
index cc8c7637cb..a2495b5ae7 100644
--- a/hw/riscv/sifive_u.c
+++ b/hw/riscv/sifive_u.c
@@ -830,8 +830,8 @@ static void sifive_u_soc_realize(DeviceState *dev, Error **errp)
     qdev_prop_set_string(DEVICE(&s->u_cpus), "cpu-type", s->cpu_type);
     qdev_prop_set_uint64(DEVICE(&s->u_cpus), "resetvec", 0x1004);
 
-    sysbus_realize(SYS_BUS_DEVICE(&s->e_cpus), &error_abort);
-    sysbus_realize(SYS_BUS_DEVICE(&s->u_cpus), &error_abort);
+    sysbus_realize(SYS_BUS_DEVICE(&s->e_cpus), &error_fatal);
+    sysbus_realize(SYS_BUS_DEVICE(&s->u_cpus), &error_fatal);
     /*
      * The cluster must be realized after the RISC-V hart array container,
      * as the container's CPU object is only created on realize, and the

From 61cdf4593e4e1bf10cb58a5b8939414f4cd50834 Mon Sep 17 00:00:00 2001
From: Tsukasa OI <research_trasio@irq.a4lg.com>
Date: Sun, 15 May 2022 11:56:07 +0900
Subject: [PATCH 471/935] target/riscv: Fix coding style on "G" expansion
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Because ext_? members are boolean variables, operator `&&' should be
used instead of `&'.

Signed-off-by: Tsukasa OI <research_trasio@irq.a4lg.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Message-Id: <91633f8349253656dd08bc8dc36498a9c7538b10.1652583332.git.research_trasio@irq.a4lg.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index dc93412395..e439716337 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -596,8 +596,8 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)
             return;
         }
 
-        if (cpu->cfg.ext_g && !(cpu->cfg.ext_i & cpu->cfg.ext_m &
-                                cpu->cfg.ext_a & cpu->cfg.ext_f &
+        if (cpu->cfg.ext_g && !(cpu->cfg.ext_i && cpu->cfg.ext_m &&
+                                cpu->cfg.ext_a && cpu->cfg.ext_f &&
                                 cpu->cfg.ext_d)) {
             warn_report("Setting G will also set IMAFD");
             cpu->cfg.ext_i = true;

From 1d398ab9dcfb8f5fb4b9a285ea3167f7ba85976d Mon Sep 17 00:00:00 2001
From: Tsukasa OI <research_trasio@irq.a4lg.com>
Date: Sun, 15 May 2022 11:56:08 +0900
Subject: [PATCH 472/935] target/riscv: Disable "G" by default

Because "G" virtual extension expands to "IMAFD", we cannot separately
disable extensions like "F" or "D" without disabling "G".  Because all
"IMAFD" are enabled by default, it's harmless to disable "G" by default.

Signed-off-by: Tsukasa OI <research_trasio@irq.a4lg.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <cab7205f1d7668f642fa242386543334af6bc1bd.1652583332.git.research_trasio@irq.a4lg.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index e439716337..1fb76b4295 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -812,7 +812,7 @@ static Property riscv_cpu_properties[] = {
     /* Defaults for standard extensions */
     DEFINE_PROP_BOOL("i", RISCVCPU, cfg.ext_i, true),
     DEFINE_PROP_BOOL("e", RISCVCPU, cfg.ext_e, false),
-    DEFINE_PROP_BOOL("g", RISCVCPU, cfg.ext_g, true),
+    DEFINE_PROP_BOOL("g", RISCVCPU, cfg.ext_g, false),
     DEFINE_PROP_BOOL("m", RISCVCPU, cfg.ext_m, true),
     DEFINE_PROP_BOOL("a", RISCVCPU, cfg.ext_a, true),
     DEFINE_PROP_BOOL("f", RISCVCPU, cfg.ext_f, true),

From 9f6b7da5d27b744ddbdba98ba8b89c710dd3c1b7 Mon Sep 17 00:00:00 2001
From: Tsukasa OI <research_trasio@irq.a4lg.com>
Date: Sun, 15 May 2022 11:56:09 +0900
Subject: [PATCH 473/935] target/riscv: Change "G" expansion

On ISA version 20190608 or later, "G" expands to "IMAFD_Zicsr_Zifencei".
Both "Zicsr" and "Zifencei" are enabled by default and "G" is supposed to
be (virtually) enabled as well, it should be safe to change its expansion.

Signed-off-by: Tsukasa OI <research_trasio@irq.a4lg.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <d1b5be550a2893a0fd32c928f832d2ff7bfafe35.1652583332.git.research_trasio@irq.a4lg.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index 1fb76b4295..4ca6a8623f 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -598,13 +598,16 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)
 
         if (cpu->cfg.ext_g && !(cpu->cfg.ext_i && cpu->cfg.ext_m &&
                                 cpu->cfg.ext_a && cpu->cfg.ext_f &&
-                                cpu->cfg.ext_d)) {
-            warn_report("Setting G will also set IMAFD");
+                                cpu->cfg.ext_d &&
+                                cpu->cfg.ext_icsr && cpu->cfg.ext_ifencei)) {
+            warn_report("Setting G will also set IMAFD_Zicsr_Zifencei");
             cpu->cfg.ext_i = true;
             cpu->cfg.ext_m = true;
             cpu->cfg.ext_a = true;
             cpu->cfg.ext_f = true;
             cpu->cfg.ext_d = true;
+            cpu->cfg.ext_icsr = true;
+            cpu->cfg.ext_ifencei = true;
         }
 
         if (cpu->cfg.ext_zdinx || cpu->cfg.ext_zhinx ||

From 1086504c6f46a2b3be90e887dddb4741bf8c500d Mon Sep 17 00:00:00 2001
From: Tsukasa OI <research_trasio@irq.a4lg.com>
Date: Sun, 15 May 2022 11:56:10 +0900
Subject: [PATCH 474/935] target/riscv: FP extension requirements

QEMU allowed inconsistent configurations that made floating point
arithmetic effectively unusable.

This commit adds certain checks for consistent FP arithmetic:

-   F requires Zicsr
-   Zfinx requires Zicsr
-   Zfh/Zfhmin require F
-   D requires F
-   V requires D

Because F/D/Zicsr are enabled by default (and an error will not occur unless
we manually disable one or more of prerequisites), this commit just enforces
the user to give consistent combinations.

Signed-off-by: Tsukasa OI <research_trasio@irq.a4lg.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <00e7b1c6060dab32ac7d49813b1ca84d3eb63298.1652583332.git.research_trasio@irq.a4lg.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index 4ca6a8623f..b960473f7d 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -610,11 +610,36 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)
             cpu->cfg.ext_ifencei = true;
         }
 
+        if (cpu->cfg.ext_f && !cpu->cfg.ext_icsr) {
+            error_setg(errp, "F extension requires Zicsr");
+            return;
+        }
+
+        if ((cpu->cfg.ext_zfh || cpu->cfg.ext_zfhmin) && !cpu->cfg.ext_f) {
+            error_setg(errp, "Zfh/Zfhmin extensions require F extension");
+            return;
+        }
+
+        if (cpu->cfg.ext_d && !cpu->cfg.ext_f) {
+            error_setg(errp, "D extension requires F extension");
+            return;
+        }
+
+        if (cpu->cfg.ext_v && !cpu->cfg.ext_d) {
+            error_setg(errp, "V extension requires D extension");
+            return;
+        }
+
         if (cpu->cfg.ext_zdinx || cpu->cfg.ext_zhinx ||
             cpu->cfg.ext_zhinxmin) {
             cpu->cfg.ext_zfinx = true;
         }
 
+        if (cpu->cfg.ext_zfinx && !cpu->cfg.ext_icsr) {
+            error_setg(errp, "Zfinx extension requires Zicsr");
+            return;
+        }
+
         if (cpu->cfg.ext_zk) {
             cpu->cfg.ext_zkn = true;
             cpu->cfg.ext_zkr = true;

From bc573816692357e4c824d5c1df73987d3f0ea0c4 Mon Sep 17 00:00:00 2001
From: Tsukasa OI <research_trasio@irq.a4lg.com>
Date: Sun, 15 May 2022 11:56:11 +0900
Subject: [PATCH 475/935] target/riscv: Move/refactor ISA extension checks

We should separate "check" and "configure" steps as possible.
This commit separates both steps except vector/Zfinx-related checks.

Signed-off-by: Tsukasa OI <research_trasio@irq.a4lg.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <c3145fa37a529484cf3047c8cb9841e9effad4b0.1652583332.git.research_trasio@irq.a4lg.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.c | 31 ++++++++++++++++---------------
 1 file changed, 16 insertions(+), 15 deletions(-)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index b960473f7d..00a068668f 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -630,14 +630,27 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)
             return;
         }
 
+        if ((cpu->cfg.ext_zve32f || cpu->cfg.ext_zve64f) && !cpu->cfg.ext_f) {
+            error_setg(errp, "Zve32f/Zve64f extensions require F extension");
+            return;
+        }
+
+        /* Set the ISA extensions, checks should have happened above */
         if (cpu->cfg.ext_zdinx || cpu->cfg.ext_zhinx ||
             cpu->cfg.ext_zhinxmin) {
             cpu->cfg.ext_zfinx = true;
         }
 
-        if (cpu->cfg.ext_zfinx && !cpu->cfg.ext_icsr) {
-            error_setg(errp, "Zfinx extension requires Zicsr");
-            return;
+        if (cpu->cfg.ext_zfinx) {
+            if (!cpu->cfg.ext_icsr) {
+                error_setg(errp, "Zfinx extension requires Zicsr");
+                return;
+            }
+            if (cpu->cfg.ext_f) {
+                error_setg(errp,
+                    "Zfinx cannot be supported together with F extension");
+                return;
+            }
         }
 
         if (cpu->cfg.ext_zk) {
@@ -663,7 +676,6 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)
             cpu->cfg.ext_zksh = true;
         }
 
-        /* Set the ISA extensions, checks should have happened above */
         if (cpu->cfg.ext_i) {
             ext |= RVI;
         }
@@ -734,20 +746,9 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)
             }
             set_vext_version(env, vext_version);
         }
-        if ((cpu->cfg.ext_zve32f || cpu->cfg.ext_zve64f) && !cpu->cfg.ext_f) {
-            error_setg(errp, "Zve32f/Zve64f extension depends upon RVF.");
-            return;
-        }
         if (cpu->cfg.ext_j) {
             ext |= RVJ;
         }
-        if (cpu->cfg.ext_zfinx && ((ext & (RVF | RVD)) || cpu->cfg.ext_zfh ||
-                                   cpu->cfg.ext_zfhmin)) {
-            error_setg(errp,
-                    "'Zfinx' cannot be supported together with 'F', 'D', 'Zfh',"
-                    " 'Zfhmin'");
-            return;
-        }
 
         set_misa(env, env->misa_mxl, ext);
     }

From 8f1b6087983ed417ca142c0a7fd7080cc279e0f5 Mon Sep 17 00:00:00 2001
From: Bernhard Beschow <shentey@gmail.com>
Date: Tue, 1 Mar 2022 23:52:19 +0100
Subject: [PATCH 476/935] hw/vfio/pci-quirks: Resolve redundant property
 getters
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The QOM API already provides getters for uint64 and uint32 values, so reuse
them.

Signed-off-by: Bernhard Beschow <shentey@gmail.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220301225220.239065-2-shentey@gmail.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 hw/vfio/pci-quirks.c | 34 +++++++++-------------------------
 1 file changed, 9 insertions(+), 25 deletions(-)

diff --git a/hw/vfio/pci-quirks.c b/hw/vfio/pci-quirks.c
index 0cf69a8c6d..f0147a050a 100644
--- a/hw/vfio/pci-quirks.c
+++ b/hw/vfio/pci-quirks.c
@@ -1565,22 +1565,6 @@ static int vfio_add_nv_gpudirect_cap(VFIOPCIDevice *vdev, Error **errp)
     return 0;
 }
 
-static void vfio_pci_nvlink2_get_tgt(Object *obj, Visitor *v,
-                                     const char *name,
-                                     void *opaque, Error **errp)
-{
-    uint64_t tgt = (uintptr_t) opaque;
-    visit_type_uint64(v, name, &tgt, errp);
-}
-
-static void vfio_pci_nvlink2_get_link_speed(Object *obj, Visitor *v,
-                                                 const char *name,
-                                                 void *opaque, Error **errp)
-{
-    uint32_t link_speed = (uint32_t)(uintptr_t) opaque;
-    visit_type_uint32(v, name, &link_speed, errp);
-}
-
 int vfio_pci_nvidia_v100_ram_init(VFIOPCIDevice *vdev, Error **errp)
 {
     int ret;
@@ -1618,9 +1602,9 @@ int vfio_pci_nvidia_v100_ram_init(VFIOPCIDevice *vdev, Error **errp)
                                nv2reg->size, p);
     QLIST_INSERT_HEAD(&vdev->bars[0].quirks, quirk, next);
 
-    object_property_add(OBJECT(vdev), "nvlink2-tgt", "uint64",
-                        vfio_pci_nvlink2_get_tgt, NULL, NULL,
-                        (void *) (uintptr_t) cap->tgt);
+    object_property_add_uint64_ptr(OBJECT(vdev), "nvlink2-tgt",
+                                   (uint64_t *) &cap->tgt,
+                                   OBJ_PROP_FLAG_READ);
     trace_vfio_pci_nvidia_gpu_setup_quirk(vdev->vbasedev.name, cap->tgt,
                                           nv2reg->size);
 free_exit:
@@ -1679,15 +1663,15 @@ int vfio_pci_nvlink2_init(VFIOPCIDevice *vdev, Error **errp)
         QLIST_INSERT_HEAD(&vdev->bars[0].quirks, quirk, next);
     }
 
-    object_property_add(OBJECT(vdev), "nvlink2-tgt", "uint64",
-                        vfio_pci_nvlink2_get_tgt, NULL, NULL,
-                        (void *) (uintptr_t) captgt->tgt);
+    object_property_add_uint64_ptr(OBJECT(vdev), "nvlink2-tgt",
+                                   (uint64_t *) &captgt->tgt,
+                                   OBJ_PROP_FLAG_READ);
     trace_vfio_pci_nvlink2_setup_quirk_ssatgt(vdev->vbasedev.name, captgt->tgt,
                                               atsdreg->size);
 
-    object_property_add(OBJECT(vdev), "nvlink2-link-speed", "uint32",
-                        vfio_pci_nvlink2_get_link_speed, NULL, NULL,
-                        (void *) (uintptr_t) capspeed->link_speed);
+    object_property_add_uint32_ptr(OBJECT(vdev), "nvlink2-link-speed",
+                                   &capspeed->link_speed,
+                                   OBJ_PROP_FLAG_READ);
     trace_vfio_pci_nvlink2_setup_quirk_lnkspd(vdev->vbasedev.name,
                                               capspeed->link_speed);
 free_exit:

From 96c7fff703d56798bd5dcb1ef6d42ead144580a3 Mon Sep 17 00:00:00 2001
From: Bernhard Beschow <shentey@gmail.com>
Date: Tue, 1 Mar 2022 23:52:20 +0100
Subject: [PATCH 477/935] hw/riscv/sifive_u: Resolve redundant property
 accessors
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The QOM API already provides accessors for uint32 values, so reuse them.

Signed-off-by: Bernhard Beschow <shentey@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20220301225220.239065-3-shentey@gmail.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 hw/riscv/sifive_u.c | 24 ++++--------------------
 1 file changed, 4 insertions(+), 20 deletions(-)

diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
index a2495b5ae7..e4c814a3ea 100644
--- a/hw/riscv/sifive_u.c
+++ b/hw/riscv/sifive_u.c
@@ -713,36 +713,20 @@ static void sifive_u_machine_set_start_in_flash(Object *obj, bool value, Error *
     s->start_in_flash = value;
 }
 
-static void sifive_u_machine_get_uint32_prop(Object *obj, Visitor *v,
-                                             const char *name, void *opaque,
-                                             Error **errp)
-{
-    visit_type_uint32(v, name, (uint32_t *)opaque, errp);
-}
-
-static void sifive_u_machine_set_uint32_prop(Object *obj, Visitor *v,
-                                             const char *name, void *opaque,
-                                             Error **errp)
-{
-    visit_type_uint32(v, name, (uint32_t *)opaque, errp);
-}
-
 static void sifive_u_machine_instance_init(Object *obj)
 {
     SiFiveUState *s = RISCV_U_MACHINE(obj);
 
     s->start_in_flash = false;
     s->msel = 0;
-    object_property_add(obj, "msel", "uint32",
-                        sifive_u_machine_get_uint32_prop,
-                        sifive_u_machine_set_uint32_prop, NULL, &s->msel);
+    object_property_add_uint32_ptr(obj, "msel", &s->msel,
+                                   OBJ_PROP_FLAG_READWRITE);
     object_property_set_description(obj, "msel",
                                     "Mode Select (MSEL[3:0]) pin state");
 
     s->serial = OTP_SERIAL;
-    object_property_add(obj, "serial", "uint32",
-                        sifive_u_machine_get_uint32_prop,
-                        sifive_u_machine_set_uint32_prop, NULL, &s->serial);
+    object_property_add_uint32_ptr(obj, "serial", &s->serial,
+                                   OBJ_PROP_FLAG_READWRITE);
     object_property_set_description(obj, "serial", "Board serial number");
 }
 

From bb06941f95edd8a231bee0ac52a8a1dbf6b08e6a Mon Sep 17 00:00:00 2001
From: Weiwei Li <liweiwei@iscas.ac.cn>
Date: Wed, 18 May 2022 09:26:11 +0800
Subject: [PATCH 478/935] target/riscv: check 'I' and 'E' after checking 'G' in
 riscv_cpu_realize

 - setting ext_g will implicitly set ext_i

Signed-off-by: Weiwei Li <liweiwei@iscas.ac.cn>
Signed-off-by: Junqiang Wang <wangjunqiang@iscas.ac.cn>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20220518012611.6772-1-liweiwei@iscas.ac.cn>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index 00a068668f..87e1eddce6 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -584,18 +584,6 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)
         uint32_t ext = 0;
 
         /* Do some ISA extension error checking */
-        if (cpu->cfg.ext_i && cpu->cfg.ext_e) {
-            error_setg(errp,
-                       "I and E extensions are incompatible");
-            return;
-        }
-
-        if (!cpu->cfg.ext_i && !cpu->cfg.ext_e) {
-            error_setg(errp,
-                       "Either I or E extension must be set");
-            return;
-        }
-
         if (cpu->cfg.ext_g && !(cpu->cfg.ext_i && cpu->cfg.ext_m &&
                                 cpu->cfg.ext_a && cpu->cfg.ext_f &&
                                 cpu->cfg.ext_d &&
@@ -610,6 +598,18 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)
             cpu->cfg.ext_ifencei = true;
         }
 
+        if (cpu->cfg.ext_i && cpu->cfg.ext_e) {
+            error_setg(errp,
+                       "I and E extensions are incompatible");
+            return;
+        }
+
+        if (!cpu->cfg.ext_i && !cpu->cfg.ext_e) {
+            error_setg(errp,
+                       "Either I or E extension must be set");
+            return;
+        }
+
         if (cpu->cfg.ext_f && !cpu->cfg.ext_icsr) {
             error_setg(errp, "F extension requires Zicsr");
             return;

From 075eeda93166f1914097ffb84b33df4ecc50d63c Mon Sep 17 00:00:00 2001
From: Frank Chang <frank.chang@sifive.com>
Date: Mon, 23 May 2022 23:31:46 +0800
Subject: [PATCH 479/935] target/riscv: Fix typo of mimpid cpu option

"mimpid" cpu option was mistyped to "mipid".

Fixes: 9951ba94 ("target/riscv: Support configuarable marchid, mvendorid, mipid CSR values")
Signed-off-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20220523153147.15371-1-frank.chang@sifive.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.c | 4 ++--
 target/riscv/cpu.h | 2 +-
 target/riscv/csr.c | 8 ++++----
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index 87e1eddce6..fe8ceb4133 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -37,7 +37,7 @@
 #define RISCV_CPU_MARCHID   ((QEMU_VERSION_MAJOR << 16) | \
                              (QEMU_VERSION_MINOR << 8)  | \
                              (QEMU_VERSION_MICRO))
-#define RISCV_CPU_MIPID     RISCV_CPU_MARCHID
+#define RISCV_CPU_MIMPID    RISCV_CPU_MARCHID
 
 static const char riscv_single_letter_exts[] = "IEMAFDQCPVH";
 
@@ -869,7 +869,7 @@ static Property riscv_cpu_properties[] = {
 
     DEFINE_PROP_UINT32("mvendorid", RISCVCPU, cfg.mvendorid, 0),
     DEFINE_PROP_UINT64("marchid", RISCVCPU, cfg.marchid, RISCV_CPU_MARCHID),
-    DEFINE_PROP_UINT64("mipid", RISCVCPU, cfg.mipid, RISCV_CPU_MIPID),
+    DEFINE_PROP_UINT64("mimpid", RISCVCPU, cfg.mimpid, RISCV_CPU_MIMPID),
 
     DEFINE_PROP_BOOL("svinval", RISCVCPU, cfg.ext_svinval, false),
     DEFINE_PROP_BOOL("svnapot", RISCVCPU, cfg.ext_svnapot, false),
diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
index f5ff7294c6..44975e3e5a 100644
--- a/target/riscv/cpu.h
+++ b/target/riscv/cpu.h
@@ -408,7 +408,7 @@ struct RISCVCPUConfig {
 
     uint32_t mvendorid;
     uint64_t marchid;
-    uint64_t mipid;
+    uint64_t mimpid;
 
     /* Vendor-specific custom extensions */
     bool ext_XVentanaCondOps;
diff --git a/target/riscv/csr.c b/target/riscv/csr.c
index 4ea7df02c9..0d5bc2f41d 100644
--- a/target/riscv/csr.c
+++ b/target/riscv/csr.c
@@ -674,13 +674,13 @@ static RISCVException read_marchid(CPURISCVState *env, int csrno,
     return RISCV_EXCP_NONE;
 }
 
-static RISCVException read_mipid(CPURISCVState *env, int csrno,
-                                 target_ulong *val)
+static RISCVException read_mimpid(CPURISCVState *env, int csrno,
+                                  target_ulong *val)
 {
     CPUState *cs = env_cpu(env);
     RISCVCPU *cpu = RISCV_CPU(cs);
 
-    *val = cpu->cfg.mipid;
+    *val = cpu->cfg.mimpid;
     return RISCV_EXCP_NONE;
 }
 
@@ -3372,7 +3372,7 @@ riscv_csr_operations csr_ops[CSR_TABLE_SIZE] = {
     /* Machine Information Registers */
     [CSR_MVENDORID] = { "mvendorid", any,   read_mvendorid },
     [CSR_MARCHID]   = { "marchid",   any,   read_marchid   },
-    [CSR_MIMPID]    = { "mimpid",    any,   read_mipid     },
+    [CSR_MIMPID]    = { "mimpid",    any,   read_mimpid    },
     [CSR_MHARTID]   = { "mhartid",   any,   read_mhartid   },
 
     [CSR_MCONFIGPTR]  = { "mconfigptr", any,   read_zero,

From c1fbcecb3a97ecce2cde5052319df34ca6bcc988 Mon Sep 17 00:00:00 2001
From: Anup Patel <apatel@ventanamicro.com>
Date: Wed, 11 May 2022 20:15:21 +0530
Subject: [PATCH 480/935] target/riscv: Fix csr number based privilege checking

When hypervisor and VS CSRs are accessed from VS-mode or VU-mode,
the riscv_csrrw_check() function should generate virtual instruction
trap instead illegal instruction trap.

Fixes: 0a42f4c44088 (" target/riscv: Fix CSR perm checking for HS mode")
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Message-Id: <20220511144528.393530-2-apatel@ventanamicro.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/csr.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/target/riscv/csr.c b/target/riscv/csr.c
index 0d5bc2f41d..6dbe9b541f 100644
--- a/target/riscv/csr.c
+++ b/target/riscv/csr.c
@@ -3139,7 +3139,7 @@ static inline RISCVException riscv_csrrw_check(CPURISCVState *env,
     int read_only = get_field(csrno, 0xC00) == 3;
     int csr_min_priv = csr_ops[csrno].min_priv_ver;
 #if !defined(CONFIG_USER_ONLY)
-    int effective_priv = env->priv;
+    int csr_priv, effective_priv = env->priv;
 
     if (riscv_has_ext(env, RVH) && env->priv == PRV_S) {
         /*
@@ -3152,7 +3152,11 @@ static inline RISCVException riscv_csrrw_check(CPURISCVState *env,
         effective_priv++;
     }
 
-    if (!env->debugger && (effective_priv < get_field(csrno, 0x300))) {
+    csr_priv = get_field(csrno, 0x300);
+    if (!env->debugger && (effective_priv < csr_priv)) {
+        if (csr_priv == (PRV_S + 1) && riscv_cpu_virt_enabled(env)) {
+            return RISCV_EXCP_VIRT_INSTRUCTION_FAULT;
+        }
         return RISCV_EXCP_ILLEGAL_INST;
     }
 #endif

From 24826da0eeacb27a5da6be764c8e853b2cede25b Mon Sep 17 00:00:00 2001
From: Anup Patel <apatel@ventanamicro.com>
Date: Wed, 11 May 2022 20:15:22 +0530
Subject: [PATCH 481/935] target/riscv: Fix hstatus.GVA bit setting for traps
 taken from HS-mode

Currently, QEMU does not set hstatus.GVA bit for traps taken from
HS-mode into HS-mode which breaks the Xvisor nested MMU test suite
on QEMU. This was working previously.

This patch updates riscv_cpu_do_interrupt() to fix the above issue.

Fixes: 86d0c457396b ("target/riscv: Fixup setting GVA")
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20220511144528.393530-3-apatel@ventanamicro.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu_helper.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index e1aa4f2097..b16bfe0182 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -1367,7 +1367,7 @@ void riscv_cpu_do_interrupt(CPUState *cs)
         case RISCV_EXCP_INST_PAGE_FAULT:
         case RISCV_EXCP_LOAD_PAGE_FAULT:
         case RISCV_EXCP_STORE_PAGE_FAULT:
-            write_gva = true;
+            write_gva = env->two_stage_lookup;
             tval = env->badaddr;
             break;
         case RISCV_EXCP_ILLEGAL_INST:
@@ -1434,7 +1434,6 @@ void riscv_cpu_do_interrupt(CPUState *cs)
                 /* Trap into HS mode */
                 env->hstatus = set_field(env->hstatus, HSTATUS_SPV, false);
                 htval = env->guest_phys_fault_addr;
-                write_gva = false;
             }
             env->hstatus = set_field(env->hstatus, HSTATUS_GVA, write_gva);
         }

From 62cf02451edb1d23bc44a35aca56a8347dfebff7 Mon Sep 17 00:00:00 2001
From: Anup Patel <apatel@ventanamicro.com>
Date: Wed, 11 May 2022 20:15:23 +0530
Subject: [PATCH 482/935] target/riscv: Set [m|s]tval for both illegal and
 virtual instruction traps

Currently, the [m|s]tval CSRs are set with trapping instruction encoding
only for illegal instruction traps taken at the time of instruction
decoding.

In RISC-V world, a valid instructions might also trap as illegal or
virtual instruction based to trapping bits in various CSRs (such as
mstatus.TVM or hstatus.VTVM).

We improve setting of [m|s]tval CSRs for all types of illegal and
virtual instruction traps.

Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20220511144528.393530-4-apatel@ventanamicro.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.c        |  2 ++
 target/riscv/cpu.h        |  8 +++++++-
 target/riscv/cpu_helper.c |  1 +
 target/riscv/translate.c  | 17 +++++++++++++----
 4 files changed, 23 insertions(+), 5 deletions(-)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index fe8ceb4133..ce1c257eef 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -406,6 +406,7 @@ void restore_state_to_opc(CPURISCVState *env, TranslationBlock *tb,
     } else {
         env->pc = data[0];
     }
+    env->bins = data[1];
 }
 
 static void riscv_cpu_reset(DeviceState *dev)
@@ -445,6 +446,7 @@ static void riscv_cpu_reset(DeviceState *dev)
     env->mcause = 0;
     env->miclaim = MIP_SGEIP;
     env->pc = env->resetvec;
+    env->bins = 0;
     env->two_stage_lookup = false;
 
     /* Initialized default priorities of local interrupts. */
diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
index 44975e3e5a..f08c3e8813 100644
--- a/target/riscv/cpu.h
+++ b/target/riscv/cpu.h
@@ -30,6 +30,12 @@
 
 #define TCG_GUEST_DEFAULT_MO 0
 
+/*
+ * RISC-V-specific extra insn start words:
+ * 1: Original instruction opcode
+ */
+#define TARGET_INSN_START_EXTRA_WORDS 1
+
 #define TYPE_RISCV_CPU "riscv-cpu"
 
 #define RISCV_CPU_TYPE_SUFFIX "-" TYPE_RISCV_CPU
@@ -140,7 +146,7 @@ struct CPUArchState {
     target_ulong frm;
 
     target_ulong badaddr;
-    uint32_t bins;
+    target_ulong bins;
 
     target_ulong guest_phys_fault_addr;
 
diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index b16bfe0182..d99fac9d2d 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -1371,6 +1371,7 @@ void riscv_cpu_do_interrupt(CPUState *cs)
             tval = env->badaddr;
             break;
         case RISCV_EXCP_ILLEGAL_INST:
+        case RISCV_EXCP_VIRT_INSTRUCTION_FAULT:
             tval = env->bins;
             break;
         default:
diff --git a/target/riscv/translate.c b/target/riscv/translate.c
index 0cd1d9ee94..55a4713af2 100644
--- a/target/riscv/translate.c
+++ b/target/riscv/translate.c
@@ -107,6 +107,8 @@ typedef struct DisasContext {
     /* PointerMasking extension */
     bool pm_mask_enabled;
     bool pm_base_enabled;
+    /* TCG of the current insn_start */
+    TCGOp *insn_start;
 } DisasContext;
 
 static inline bool has_ext(DisasContext *ctx, uint32_t ext)
@@ -236,9 +238,6 @@ static void generate_exception_mtval(DisasContext *ctx, int excp)
 
 static void gen_exception_illegal(DisasContext *ctx)
 {
-    tcg_gen_st_i32(tcg_constant_i32(ctx->opcode), cpu_env,
-                   offsetof(CPURISCVState, bins));
-
     generate_exception(ctx, RISCV_EXCP_ILLEGAL_INST);
 }
 
@@ -1017,6 +1016,13 @@ static uint32_t opcode_at(DisasContextBase *dcbase, target_ulong pc)
 /* Include decoders for factored-out extensions */
 #include "decode-XVentanaCondOps.c.inc"
 
+static inline void decode_save_opc(DisasContext *ctx, target_ulong opc)
+{
+    assert(ctx->insn_start != NULL);
+    tcg_set_insn_start_param(ctx->insn_start, 1, opc);
+    ctx->insn_start = NULL;
+}
+
 static void decode_opc(CPURISCVState *env, DisasContext *ctx, uint16_t opcode)
 {
     /*
@@ -1033,6 +1039,7 @@ static void decode_opc(CPURISCVState *env, DisasContext *ctx, uint16_t opcode)
 
     /* Check for compressed insn */
     if (extract16(opcode, 0, 2) != 3) {
+        decode_save_opc(ctx, opcode);
         if (!has_ext(ctx, RVC)) {
             gen_exception_illegal(ctx);
         } else {
@@ -1047,6 +1054,7 @@ static void decode_opc(CPURISCVState *env, DisasContext *ctx, uint16_t opcode)
         opcode32 = deposit32(opcode32, 16, 16,
                              translator_lduw(env, &ctx->base,
                                              ctx->base.pc_next + 2));
+        decode_save_opc(ctx, opcode32);
         ctx->opcode = opcode32;
         ctx->pc_succ_insn = ctx->base.pc_next + 4;
 
@@ -1113,7 +1121,8 @@ static void riscv_tr_insn_start(DisasContextBase *dcbase, CPUState *cpu)
 {
     DisasContext *ctx = container_of(dcbase, DisasContext, base);
 
-    tcg_gen_insn_start(ctx->base.pc_next);
+    tcg_gen_insn_start(ctx->base.pc_next, 0);
+    ctx->insn_start = tcg_last_op();
 }
 
 static void riscv_tr_translate_insn(DisasContextBase *dcbase, CPUState *cpu)

From d644e5e44ff627d6b4da73a65795f60335ba4cb9 Mon Sep 17 00:00:00 2001
From: Anup Patel <apatel@ventanamicro.com>
Date: Wed, 11 May 2022 20:15:28 +0530
Subject: [PATCH 483/935] hw/riscv: virt: Fix interrupt parent for dynamic
 platform devices

When both APLIC and IMSIC are present in virt machine, the APLIC should
be used as parent interrupt controller for dynamic platform devices.

In case of  multiple sockets, we should prefer interrupt controller of
socket0 for dynamic platform devices.

Fixes: 3029fab64309 ("hw/riscv: virt: Add support for generating
platform FDT entries")
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20220511144528.393530-9-apatel@ventanamicro.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 hw/riscv/virt.c | 25 ++++++++++++-------------
 1 file changed, 12 insertions(+), 13 deletions(-)

diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 244d6408b5..293e9c95b7 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -478,10 +478,12 @@ static void create_fdt_socket_plic(RISCVVirtState *s,
     qemu_fdt_setprop_cell(mc->fdt, plic_name, "phandle",
         plic_phandles[socket]);
 
-    platform_bus_add_all_fdt_nodes(mc->fdt, plic_name,
-                                   memmap[VIRT_PLATFORM_BUS].base,
-                                   memmap[VIRT_PLATFORM_BUS].size,
-                                   VIRT_PLATFORM_BUS_IRQ);
+    if (!socket) {
+        platform_bus_add_all_fdt_nodes(mc->fdt, plic_name,
+                                       memmap[VIRT_PLATFORM_BUS].base,
+                                       memmap[VIRT_PLATFORM_BUS].size,
+                                       VIRT_PLATFORM_BUS_IRQ);
+    }
 
     g_free(plic_name);
 
@@ -561,11 +563,6 @@ static void create_fdt_imsic(RISCVVirtState *s, const MemMapEntry *memmap,
     }
     qemu_fdt_setprop_cell(mc->fdt, imsic_name, "phandle", *msi_m_phandle);
 
-    platform_bus_add_all_fdt_nodes(mc->fdt, imsic_name,
-                                   memmap[VIRT_PLATFORM_BUS].base,
-                                   memmap[VIRT_PLATFORM_BUS].size,
-                                   VIRT_PLATFORM_BUS_IRQ);
-
     g_free(imsic_name);
 
     /* S-level IMSIC node */
@@ -704,10 +701,12 @@ static void create_fdt_socket_aplic(RISCVVirtState *s,
     riscv_socket_fdt_write_id(mc, mc->fdt, aplic_name, socket);
     qemu_fdt_setprop_cell(mc->fdt, aplic_name, "phandle", aplic_s_phandle);
 
-    platform_bus_add_all_fdt_nodes(mc->fdt, aplic_name,
-                                   memmap[VIRT_PLATFORM_BUS].base,
-                                   memmap[VIRT_PLATFORM_BUS].size,
-                                   VIRT_PLATFORM_BUS_IRQ);
+    if (!socket) {
+        platform_bus_add_all_fdt_nodes(mc->fdt, aplic_name,
+                                       memmap[VIRT_PLATFORM_BUS].base,
+                                       memmap[VIRT_PLATFORM_BUS].size,
+                                       VIRT_PLATFORM_BUS_IRQ);
+    }
 
     g_free(aplic_name);
 

From 5160bacc0638088a7cb0180d2be3d8c2c8a21831 Mon Sep 17 00:00:00 2001
From: "Hongren (Zenithal) Zheng" <i@zenithal.me>
Date: Wed, 18 May 2022 20:46:58 +0800
Subject: [PATCH 484/935] target/riscv: add zicsr/zifencei to isa_string

Zicsr/Zifencei is not in 'I' since ISA version 20190608,
thus to fully express the capability of the CPU,
they should be exposed in isa_string.

Signed-off-by: Hongren (Zenithal) Zheng <i@zenithal.me>
Tested-by: Jiatai He <jiatai2021@iscas.ac.cn>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <YoTqwpfrodveJ7CR@Sun>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index ce1c257eef..a91253d4bd 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -1029,6 +1029,8 @@ static void riscv_isa_string_ext(RISCVCPU *cpu, char **isa_str, int max_str_len)
      *    extensions by an underscore.
      */
     struct isa_ext_data isa_edata_arr[] = {
+        ISA_EDATA_ENTRY(zicsr, ext_icsr),
+        ISA_EDATA_ENTRY(zifencei, ext_ifencei),
         ISA_EDATA_ENTRY(zfh, ext_zfh),
         ISA_EDATA_ENTRY(zfhmin, ext_zfhmin),
         ISA_EDATA_ENTRY(zfinx, ext_zfinx),

From d616889ece83d006ff630c72eb5bb38ad3b86645 Mon Sep 17 00:00:00 2001
From: Bin Meng <bin.meng@windriver.com>
Date: Thu, 24 Mar 2022 21:48:11 +0800
Subject: [PATCH 485/935] hw/core: Sync uboot_image.h from U-Boot v2022.01

Sync uboot_image.h from upstream U-Boot v2022.01 release [1].

[1] https://source.denx.de/u-boot/u-boot/-/blob/v2022.01/include/image.h

Signed-off-by: Bin Meng <bin.meng@windriver.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20220324134812.541274-1-bmeng.cn@gmail.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 hw/core/uboot_image.h | 213 ++++++++++++++++++++++++++++--------------
 1 file changed, 142 insertions(+), 71 deletions(-)

diff --git a/hw/core/uboot_image.h b/hw/core/uboot_image.h
index 608022de6e..18ac293359 100644
--- a/hw/core/uboot_image.h
+++ b/hw/core/uboot_image.h
@@ -1,23 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
 /*
+ * (C) Copyright 2008 Semihalf
+ *
  * (C) Copyright 2000-2005
  * Wolfgang Denk, DENX Software Engineering, wd@denx.de.
- *
- * See file CREDITS for list of people who contributed to this
- * project.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; either version 2 of
- * the License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, see <http://www.gnu.org/licenses/>.
- *
  ********************************************************************
  * NOTE: This header file defines an interface to U-Boot. Including
  * this (unmodified) header file in another file is considered normal
@@ -31,50 +17,83 @@
 
 /*
  * Operating System Codes
+ *
+ * The following are exposed to uImage header.
+ * New IDs *MUST* be appended at the end of the list and *NEVER*
+ * inserted for backward compatibility.
  */
-#define IH_OS_INVALID		0	/* Invalid OS	*/
-#define IH_OS_OPENBSD		1	/* OpenBSD	*/
-#define IH_OS_NETBSD		2	/* NetBSD	*/
-#define IH_OS_FREEBSD		3	/* FreeBSD	*/
-#define IH_OS_4_4BSD		4	/* 4.4BSD	*/
-#define IH_OS_LINUX		5	/* Linux	*/
-#define IH_OS_SVR4		6	/* SVR4		*/
-#define IH_OS_ESIX		7	/* Esix		*/
-#define IH_OS_SOLARIS		8	/* Solaris	*/
-#define IH_OS_IRIX		9	/* Irix		*/
-#define IH_OS_SCO		10	/* SCO		*/
-#define IH_OS_DELL		11	/* Dell		*/
-#define IH_OS_NCR		12	/* NCR		*/
-#define IH_OS_LYNXOS		13	/* LynxOS	*/
-#define IH_OS_VXWORKS		14	/* VxWorks	*/
-#define IH_OS_PSOS		15	/* pSOS		*/
-#define IH_OS_QNX		16	/* QNX		*/
-#define IH_OS_U_BOOT		17	/* Firmware	*/
-#define IH_OS_RTEMS		18	/* RTEMS	*/
-#define IH_OS_ARTOS		19	/* ARTOS	*/
-#define IH_OS_UNITY		20	/* Unity OS	*/
+enum {
+	IH_OS_INVALID		= 0,	/* Invalid OS	*/
+	IH_OS_OPENBSD,			/* OpenBSD	*/
+	IH_OS_NETBSD,			/* NetBSD	*/
+	IH_OS_FREEBSD,			/* FreeBSD	*/
+	IH_OS_4_4BSD,			/* 4.4BSD	*/
+	IH_OS_LINUX,			/* Linux	*/
+	IH_OS_SVR4,			/* SVR4		*/
+	IH_OS_ESIX,			/* Esix		*/
+	IH_OS_SOLARIS,			/* Solaris	*/
+	IH_OS_IRIX,			/* Irix		*/
+	IH_OS_SCO,			/* SCO		*/
+	IH_OS_DELL,			/* Dell		*/
+	IH_OS_NCR,			/* NCR		*/
+	IH_OS_LYNXOS,			/* LynxOS	*/
+	IH_OS_VXWORKS,			/* VxWorks	*/
+	IH_OS_PSOS,			/* pSOS		*/
+	IH_OS_QNX,			/* QNX		*/
+	IH_OS_U_BOOT,			/* Firmware	*/
+	IH_OS_RTEMS,			/* RTEMS	*/
+	IH_OS_ARTOS,			/* ARTOS	*/
+	IH_OS_UNITY,			/* Unity OS	*/
+	IH_OS_INTEGRITY,		/* INTEGRITY	*/
+	IH_OS_OSE,			/* OSE		*/
+	IH_OS_PLAN9,			/* Plan 9	*/
+	IH_OS_OPENRTOS,		/* OpenRTOS	*/
+	IH_OS_ARM_TRUSTED_FIRMWARE,     /* ARM Trusted Firmware */
+	IH_OS_TEE,			/* Trusted Execution Environment */
+	IH_OS_OPENSBI,			/* RISC-V OpenSBI */
+	IH_OS_EFI,			/* EFI Firmware (e.g. GRUB2) */
+
+	IH_OS_COUNT,
+};
 
 /*
  * CPU Architecture Codes (supported by Linux)
+ *
+ * The following are exposed to uImage header.
+ * New IDs *MUST* be appended at the end of the list and *NEVER*
+ * inserted for backward compatibility.
  */
-#define IH_CPU_INVALID		0	/* Invalid CPU	*/
-#define IH_CPU_ALPHA		1	/* Alpha	*/
-#define IH_CPU_ARM		2	/* ARM		*/
-#define IH_CPU_I386		3	/* Intel x86	*/
-#define IH_CPU_IA64		4	/* IA64		*/
-#define IH_CPU_MIPS		5	/* MIPS		*/
-#define IH_CPU_MIPS64		6	/* MIPS	 64 Bit */
-#define IH_CPU_PPC		7	/* PowerPC	*/
-#define IH_CPU_S390		8	/* IBM S390	*/
-#define IH_CPU_SH		9	/* SuperH	*/
-#define IH_CPU_SPARC		10	/* Sparc	*/
-#define IH_CPU_SPARC64		11	/* Sparc 64 Bit */
-#define IH_CPU_M68K		12	/* M68K		*/
-#define IH_CPU_NIOS		13	/* Nios-32	*/
-#define IH_CPU_MICROBLAZE	14	/* MicroBlaze   */
-#define IH_CPU_NIOS2		15	/* Nios-II	*/
-#define IH_CPU_BLACKFIN		16	/* Blackfin	*/
-#define IH_CPU_AVR32		17	/* AVR32	*/
+enum {
+	IH_ARCH_INVALID		= 0,	/* Invalid CPU	*/
+	IH_ARCH_ALPHA,			/* Alpha	*/
+	IH_ARCH_ARM,			/* ARM		*/
+	IH_ARCH_I386,			/* Intel x86	*/
+	IH_ARCH_IA64,			/* IA64		*/
+	IH_ARCH_MIPS,			/* MIPS		*/
+	IH_ARCH_MIPS64,			/* MIPS	 64 Bit */
+	IH_ARCH_PPC,			/* PowerPC	*/
+	IH_ARCH_S390,			/* IBM S390	*/
+	IH_ARCH_SH,			/* SuperH	*/
+	IH_ARCH_SPARC,			/* Sparc	*/
+	IH_ARCH_SPARC64,		/* Sparc 64 Bit */
+	IH_ARCH_M68K,			/* M68K		*/
+	IH_ARCH_NIOS,			/* Nios-32	*/
+	IH_ARCH_MICROBLAZE,		/* MicroBlaze   */
+	IH_ARCH_NIOS2,			/* Nios-II	*/
+	IH_ARCH_BLACKFIN,		/* Blackfin	*/
+	IH_ARCH_AVR32,			/* AVR32	*/
+	IH_ARCH_ST200,			/* STMicroelectronics ST200  */
+	IH_ARCH_SANDBOX,		/* Sandbox architecture (test only) */
+	IH_ARCH_NDS32,			/* ANDES Technology - NDS32  */
+	IH_ARCH_OPENRISC,		/* OpenRISC 1000  */
+	IH_ARCH_ARM64,			/* ARM64	*/
+	IH_ARCH_ARC,			/* Synopsys DesignWare ARC */
+	IH_ARCH_X86_64,			/* AMD x86_64, Intel and Via */
+	IH_ARCH_XTENSA,			/* Xtensa	*/
+	IH_ARCH_RISCV,			/* RISC-V */
+
+	IH_ARCH_COUNT,
+};
 
 /*
  * Image Types
@@ -113,33 +132,85 @@
  *	U-Boot's command interpreter; this feature is especially
  *	useful when you configure U-Boot to use a real shell (hush)
  *	as command interpreter (=> Shell Scripts).
+ *
+ * The following are exposed to uImage header.
+ * New IDs *MUST* be appended at the end of the list and *NEVER*
+ * inserted for backward compatibility.
  */
 
-#define IH_TYPE_INVALID		0	/* Invalid Image		*/
-#define IH_TYPE_STANDALONE	1	/* Standalone Program		*/
-#define IH_TYPE_KERNEL		2	/* OS Kernel Image		*/
-#define IH_TYPE_RAMDISK		3	/* RAMDisk Image		*/
-#define IH_TYPE_MULTI		4	/* Multi-File Image		*/
-#define IH_TYPE_FIRMWARE	5	/* Firmware Image		*/
-#define IH_TYPE_SCRIPT		6	/* Script file			*/
-#define IH_TYPE_FILESYSTEM	7	/* Filesystem Image (any type)	*/
-#define IH_TYPE_FLATDT		8	/* Binary Flat Device Tree Blob	*/
-#define IH_TYPE_KERNEL_NOLOAD  14	/* OS Kernel Image (noload)	*/
+enum {
+	IH_TYPE_INVALID		= 0,	/* Invalid Image		*/
+	IH_TYPE_STANDALONE,		/* Standalone Program		*/
+	IH_TYPE_KERNEL,			/* OS Kernel Image		*/
+	IH_TYPE_RAMDISK,		/* RAMDisk Image		*/
+	IH_TYPE_MULTI,			/* Multi-File Image		*/
+	IH_TYPE_FIRMWARE,		/* Firmware Image		*/
+	IH_TYPE_SCRIPT,			/* Script file			*/
+	IH_TYPE_FILESYSTEM,		/* Filesystem Image (any type)	*/
+	IH_TYPE_FLATDT,			/* Binary Flat Device Tree Blob	*/
+	IH_TYPE_KWBIMAGE,		/* Kirkwood Boot Image		*/
+	IH_TYPE_IMXIMAGE,		/* Freescale IMXBoot Image	*/
+	IH_TYPE_UBLIMAGE,		/* Davinci UBL Image		*/
+	IH_TYPE_OMAPIMAGE,		/* TI OMAP Config Header Image	*/
+	IH_TYPE_AISIMAGE,		/* TI Davinci AIS Image		*/
+	/* OS Kernel Image, can run from any load address */
+	IH_TYPE_KERNEL_NOLOAD,
+	IH_TYPE_PBLIMAGE,		/* Freescale PBL Boot Image	*/
+	IH_TYPE_MXSIMAGE,		/* Freescale MXSBoot Image	*/
+	IH_TYPE_GPIMAGE,		/* TI Keystone GPHeader Image	*/
+	IH_TYPE_ATMELIMAGE,		/* ATMEL ROM bootable Image	*/
+	IH_TYPE_SOCFPGAIMAGE,		/* Altera SOCFPGA CV/AV Preloader */
+	IH_TYPE_X86_SETUP,		/* x86 setup.bin Image		*/
+	IH_TYPE_LPC32XXIMAGE,		/* x86 setup.bin Image		*/
+	IH_TYPE_LOADABLE,		/* A list of typeless images	*/
+	IH_TYPE_RKIMAGE,		/* Rockchip Boot Image		*/
+	IH_TYPE_RKSD,			/* Rockchip SD card		*/
+	IH_TYPE_RKSPI,			/* Rockchip SPI image		*/
+	IH_TYPE_ZYNQIMAGE,		/* Xilinx Zynq Boot Image */
+	IH_TYPE_ZYNQMPIMAGE,		/* Xilinx ZynqMP Boot Image */
+	IH_TYPE_ZYNQMPBIF,		/* Xilinx ZynqMP Boot Image (bif) */
+	IH_TYPE_FPGA,			/* FPGA Image */
+	IH_TYPE_VYBRIDIMAGE,	/* VYBRID .vyb Image */
+	IH_TYPE_TEE,            /* Trusted Execution Environment OS Image */
+	IH_TYPE_FIRMWARE_IVT,		/* Firmware Image with HABv4 IVT */
+	IH_TYPE_PMMC,            /* TI Power Management Micro-Controller Firmware */
+	IH_TYPE_STM32IMAGE,		/* STMicroelectronics STM32 Image */
+	IH_TYPE_SOCFPGAIMAGE_V1,	/* Altera SOCFPGA A10 Preloader	*/
+	IH_TYPE_MTKIMAGE,		/* MediaTek BootROM loadable Image */
+	IH_TYPE_IMX8MIMAGE,		/* Freescale IMX8MBoot Image	*/
+	IH_TYPE_IMX8IMAGE,		/* Freescale IMX8Boot Image	*/
+	IH_TYPE_COPRO,			/* Coprocessor Image for remoteproc*/
+	IH_TYPE_SUNXI_EGON,		/* Allwinner eGON Boot Image */
+
+	IH_TYPE_COUNT,			/* Number of image types */
+};
 
 /*
  * Compression Types
+ *
+ * The following are exposed to uImage header.
+ * New IDs *MUST* be appended at the end of the list and *NEVER*
+ * inserted for backward compatibility.
  */
-#define IH_COMP_NONE		0	/*  No	 Compression Used	*/
-#define IH_COMP_GZIP		1	/* gzip	 Compression Used	*/
-#define IH_COMP_BZIP2		2	/* bzip2 Compression Used	*/
+enum {
+	IH_COMP_NONE		= 0,	/*  No	 Compression Used	*/
+	IH_COMP_GZIP,			/* gzip	 Compression Used	*/
+	IH_COMP_BZIP2,			/* bzip2 Compression Used	*/
+	IH_COMP_LZMA,			/* lzma  Compression Used	*/
+	IH_COMP_LZO,			/* lzo   Compression Used	*/
+	IH_COMP_LZ4,			/* lz4   Compression Used	*/
+	IH_COMP_ZSTD,			/* zstd   Compression Used	*/
+
+	IH_COMP_COUNT,
+};
 
 #define IH_MAGIC	0x27051956	/* Image Magic Number		*/
 #define IH_NMLEN		32	/* Image Name Length		*/
 
 /*
- * all data in network byte order (aka natural aka bigendian)
+ * Legacy format image header,
+ * all data in network byte order (aka natural aka bigendian).
  */
-
 typedef struct uboot_image_header {
 	uint32_t	ih_magic;	/* Image Header Magic Number	*/
 	uint32_t	ih_hcrc;	/* Image Header CRC Checksum	*/

From 8fe63fe8e512d77583d6798acd2164f1fa1e40ab Mon Sep 17 00:00:00 2001
From: Bin Meng <bin.meng@windriver.com>
Date: Thu, 24 Mar 2022 21:48:12 +0800
Subject: [PATCH 486/935] hw/core: loader: Set is_linux to true for VxWorks
 uImage
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

VxWorks 7 uses the same boot interface as the Linux kernel on Arm
(64-bit only), PowerPC and RISC-V architectures. Add logic to set
is_linux to true for VxWorks uImage for these architectures in
load_uboot_image().

Signed-off-by: Bin Meng <bin.meng@windriver.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20220324134812.541274-2-bmeng.cn@gmail.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 hw/core/loader.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/hw/core/loader.c b/hw/core/loader.c
index 8167301f04..edde657ac3 100644
--- a/hw/core/loader.c
+++ b/hw/core/loader.c
@@ -696,6 +696,21 @@ static int load_uboot_image(const char *filename, hwaddr *ep, hwaddr *loadaddr,
         if (is_linux) {
             if (hdr->ih_os == IH_OS_LINUX) {
                 *is_linux = 1;
+            } else if (hdr->ih_os == IH_OS_VXWORKS) {
+                /*
+                 * VxWorks 7 uses the same boot interface as the Linux kernel
+                 * on Arm (64-bit only), PowerPC and RISC-V architectures.
+                 */
+                switch (hdr->ih_arch) {
+                case IH_ARCH_ARM64:
+                case IH_ARCH_PPC:
+                case IH_ARCH_RISCV:
+                    *is_linux = 1;
+                    break;
+                default:
+                    *is_linux = 0;
+                    break;
+                }
             } else {
                 *is_linux = 0;
             }

From 3d393bb7d3348142868d2af651500949881d41f0 Mon Sep 17 00:00:00 2001
From: Jamin Lin <jamin_lin@aspeedtech.com>
Date: Wed, 25 May 2022 10:31:33 +0200
Subject: [PATCH 487/935] docs: add minibmc section in aspeed document
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220506031521.13254-2-jamin_lin@aspeedtech.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
 docs/system/arm/aspeed.rst | 61 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 61 insertions(+)

diff --git a/docs/system/arm/aspeed.rst b/docs/system/arm/aspeed.rst
index 60ed94f187..6f2e4fb53d 100644
--- a/docs/system/arm/aspeed.rst
+++ b/docs/system/arm/aspeed.rst
@@ -120,3 +120,64 @@ FMC chip and a bigger (64M) SPI chip, use :
 .. code-block:: bash
 
   -M ast2500-evb,fmc-model=mx25l25635e,spi-model=mx66u51235f
+
+
+Aspeed minibmc family boards (``ast1030-evb``)
+==================================================================
+
+The QEMU Aspeed machines model mini BMCs of various Aspeed evaluation
+boards. They are based on different releases of the
+Aspeed SoC : the AST1030 integrating an ARM Cortex M4F CPU (200MHz).
+
+The SoC comes with SRAM, SPI, I2C, etc.
+
+AST1030 SoC based machines :
+
+- ``ast1030-evb``          Aspeed AST1030 Evaluation board (Cortex-M4F)
+
+Supported devices
+-----------------
+
+ * SMP (for the AST1030 Cortex-M4F)
+ * Interrupt Controller (VIC)
+ * Timer Controller
+ * I2C Controller
+ * System Control Unit (SCU)
+ * SRAM mapping
+ * Static Memory Controller (SMC or FMC) - Only SPI Flash support
+ * SPI Memory Controller
+ * USB 2.0 Controller
+ * Watchdog Controller
+ * GPIO Controller (Master only)
+ * UART
+ * LPC Peripheral Controller (a subset of subdevices are supported)
+ * Hash/Crypto Engine (HACE) - Hash support only. TODO: HMAC and RSA
+ * ADC
+
+
+Missing devices
+---------------
+
+ * PWM and Fan Controller
+ * Slave GPIO Controller
+ * PECI Controller
+ * Mailbox Controller
+ * Virtual UART
+ * eSPI Controller
+ * I3C Controller
+
+Boot options
+------------
+
+The Aspeed machines can be started using the ``-kernel`` to load a
+Zephyr OS or from a firmware. Images can be downloaded from the
+ASPEED GitHub release repository :
+
+   https://github.com/AspeedTech-BMC/zephyr/releases
+
+To boot a kernel directly from a Zephyr build tree:
+
+.. code-block:: bash
+
+  $ qemu-system-arm -M ast1030-evb -nographic \
+        -kernel zephyr.elf

From fa699e80a80950ed31661dc8ea36a8b9aee9b094 Mon Sep 17 00:00:00 2001
From: Peter Delevoryas <pdel@fb.com>
Date: Wed, 25 May 2022 10:31:33 +0200
Subject: [PATCH 488/935] hw/arm/aspeed: Add fby35 machine type
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add the 'fby35-bmc' machine type based on the kernel DTS[1] and userspace
i2c setup scripts[2]. Undefined values are inherited from the AST2600-EVB.

Reference images can be found in Facebook OpenBMC Github Release assets
as "fby35.mtd". [3]

You can boot the reference images as follows (fby35 uses dual-flash):

qemu-system-arm -machine fby35-bmc \
    -drive file=fby35.mtd,format=raw,if=mtd \
    -drive file=fby35.mtd,format=raw,if=mtd \
    -nographic

[1] https://github.com/facebook/openbmc-linux/blob/412d5053258007117e94b1e36015aefc1301474b/arch/arm/boot/dts/aspeed-bmc-facebook-fby35.dts
[2] https://github.com/facebook/openbmc/blob/e2294ff5d31dd65c248fe396a385286d6d5c463d/meta-facebook/meta-fby35/recipes-fby35/plat-utils/files/setup-dev.sh
[3] https://github.com/facebook/openbmc/releases

Signed-off-by: Peter Delevoryas <pdel@fb.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220503225925.1798324-2-pdel@fb.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
 hw/arm/aspeed.c | 63 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 63 insertions(+)

diff --git a/hw/arm/aspeed.c b/hw/arm/aspeed.c
index a74c13ab0f..725c169488 100644
--- a/hw/arm/aspeed.c
+++ b/hw/arm/aspeed.c
@@ -21,6 +21,7 @@
 #include "hw/misc/led.h"
 #include "hw/qdev-properties.h"
 #include "sysemu/block-backend.h"
+#include "sysemu/reset.h"
 #include "hw/loader.h"
 #include "qemu/error-report.h"
 #include "qemu/units.h"
@@ -951,6 +952,35 @@ static void bletchley_bmc_i2c_init(AspeedMachineState *bmc)
     i2c_slave_create_simple(i2c[12], TYPE_PCA9552, 0x67);
 }
 
+static void fby35_i2c_init(AspeedMachineState *bmc)
+{
+    AspeedSoCState *soc = &bmc->soc;
+    I2CBus *i2c[16];
+
+    for (int i = 0; i < 16; i++) {
+        i2c[i] = aspeed_i2c_get_bus(&soc->i2c, i);
+    }
+
+    i2c_slave_create_simple(i2c[2], TYPE_LM75, 0x4f);
+    i2c_slave_create_simple(i2c[8], TYPE_TMP421, 0x1f);
+    /* Hotswap controller is actually supposed to be mp5920 or ltc4282. */
+    i2c_slave_create_simple(i2c[11], "adm1272", 0x44);
+    i2c_slave_create_simple(i2c[12], TYPE_LM75, 0x4e);
+    i2c_slave_create_simple(i2c[12], TYPE_LM75, 0x4f);
+
+    aspeed_eeprom_init(i2c[4], 0x51, 128 * KiB);
+    aspeed_eeprom_init(i2c[6], 0x51, 128 * KiB);
+    aspeed_eeprom_init(i2c[8], 0x50, 32 * KiB);
+    aspeed_eeprom_init(i2c[11], 0x51, 128 * KiB);
+    aspeed_eeprom_init(i2c[11], 0x54, 128 * KiB);
+
+    /*
+     * TODO: There is a multi-master i2c connection to an AST1030 MiniBMC on
+     * buses 0, 1, 2, 3, and 9. Source address 0x10, target address 0x20 on
+     * each.
+     */
+}
+
 static bool aspeed_get_mmio_exec(Object *obj, Error **errp)
 {
     return ASPEED_MACHINE(obj)->mmio_exec;
@@ -1293,6 +1323,35 @@ static void aspeed_machine_bletchley_class_init(ObjectClass *oc, void *data)
         aspeed_soc_num_cpus(amc->soc_name);
 }
 
+static void fby35_reset(MachineState *state)
+{
+    AspeedMachineState *bmc = ASPEED_MACHINE(state);
+    AspeedGPIOState *gpio = &bmc->soc.gpio;
+
+    qemu_devices_reset();
+
+    /* Board ID */
+    object_property_set_bool(OBJECT(gpio), "gpioV4", true, &error_fatal);
+    object_property_set_bool(OBJECT(gpio), "gpioV5", true, &error_fatal);
+    object_property_set_bool(OBJECT(gpio), "gpioV6", true, &error_fatal);
+    object_property_set_bool(OBJECT(gpio), "gpioV7", false, &error_fatal);
+}
+
+static void aspeed_machine_fby35_class_init(ObjectClass *oc, void *data)
+{
+    MachineClass *mc = MACHINE_CLASS(oc);
+    AspeedMachineClass *amc = ASPEED_MACHINE_CLASS(oc);
+
+    mc->desc       = "Facebook fby35 BMC (Cortex-A7)";
+    mc->reset      = fby35_reset;
+    amc->fmc_model = "mx66l1g45g";
+    amc->num_cs    = 2;
+    amc->macs_mask = ASPEED_MAC3_ON;
+    amc->i2c_init  = fby35_i2c_init;
+    /* FIXME: Replace this macro with something more general */
+    mc->default_ram_size = FUJI_BMC_RAM_SIZE;
+}
+
 #define AST1030_INTERNAL_FLASH_SIZE (1024 * 1024)
 /* Main SYSCLK frequency in Hz (200MHz) */
 #define SYSCLK_FRQ 200000000ULL
@@ -1411,6 +1470,10 @@ static const TypeInfo aspeed_machine_types[] = {
         .name          = MACHINE_TYPE_NAME("bletchley-bmc"),
         .parent        = TYPE_ASPEED_MACHINE,
         .class_init    = aspeed_machine_bletchley_class_init,
+    }, {
+        .name          = MACHINE_TYPE_NAME("fby35-bmc"),
+        .parent        = MACHINE_TYPE_NAME("ast2600-evb"),
+        .class_init    = aspeed_machine_fby35_class_init,
     }, {
         .name           = MACHINE_TYPE_NAME("ast1030-evb"),
         .parent         = TYPE_ASPEED_MACHINE,

From 264a360ae0981a22e35b3e68b3eded1cac666afd Mon Sep 17 00:00:00 2001
From: Peter Delevoryas <pdel@fb.com>
Date: Wed, 25 May 2022 10:31:33 +0200
Subject: [PATCH 489/935] docs: aspeed: Add fby35 board
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add fby35 to the list of Aspeed boards.

Signed-off-by: Peter Delevoryas <pdel@fb.com>
Message-Id: <20220506193354.990532-2-pdel@fb.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
 docs/system/arm/aspeed.rst | 1 +
 1 file changed, 1 insertion(+)

diff --git a/docs/system/arm/aspeed.rst b/docs/system/arm/aspeed.rst
index 6f2e4fb53d..5d0a7865d3 100644
--- a/docs/system/arm/aspeed.rst
+++ b/docs/system/arm/aspeed.rst
@@ -31,6 +31,7 @@ AST2600 SoC based machines :
 - ``tacoma-bmc``           OpenPOWER Witherspoon POWER9 AST2600 BMC
 - ``rainier-bmc``          IBM Rainier POWER10 BMC
 - ``fuji-bmc``             Facebook Fuji BMC
+- ``fby35-bmc``            Facebook fby35 BMC
 
 Supported devices
 -----------------

From 188052a13377ff1f88b433255005eb9082d048e5 Mon Sep 17 00:00:00 2001
From: Iris Chen <irischenlj@fb.com>
Date: Wed, 25 May 2022 10:31:33 +0200
Subject: [PATCH 490/935] hw: m25p80: allow write_enable latch get/set
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The write_enable latch property is not currently exposed.
This commit makes it a modifiable property.

Signed-off-by: Iris Chen <irischenlj@fb.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Francisco Iglesias <frasse.iglesias@gmail.com>
Message-Id: <20220513055022.951759-1-irischenlj@fb.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
 hw/block/m25p80.c              |  1 +
 tests/qtest/aspeed_gpio-test.c | 40 +++++++------------------------
 tests/qtest/aspeed_smc-test.c  | 43 ++++++++++++++++++++++++++++++++++
 tests/qtest/libqtest.c         | 24 +++++++++++++++++++
 tests/qtest/libqtest.h         | 22 +++++++++++++++++
 5 files changed, 98 insertions(+), 32 deletions(-)

diff --git a/hw/block/m25p80.c b/hw/block/m25p80.c
index 7d3d8b12e0..81ba3da4df 100644
--- a/hw/block/m25p80.c
+++ b/hw/block/m25p80.c
@@ -1533,6 +1533,7 @@ static int m25p80_pre_save(void *opaque)
 
 static Property m25p80_properties[] = {
     /* This is default value for Micron flash */
+    DEFINE_PROP_BOOL("write-enable", Flash, write_enable, false),
     DEFINE_PROP_UINT32("nonvolatile-cfg", Flash, nonvolatile_cfg, 0x8FFF),
     DEFINE_PROP_UINT8("spansion-cr1nv", Flash, spansion_cr1nv, 0x0),
     DEFINE_PROP_UINT8("spansion-cr2nv", Flash, spansion_cr2nv, 0x8),
diff --git a/tests/qtest/aspeed_gpio-test.c b/tests/qtest/aspeed_gpio-test.c
index c1003f2d1b..bac63e8742 100644
--- a/tests/qtest/aspeed_gpio-test.c
+++ b/tests/qtest/aspeed_gpio-test.c
@@ -28,30 +28,6 @@
 #include "qapi/qmp/qdict.h"
 #include "libqtest-single.h"
 
-static bool qom_get_bool(QTestState *s, const char *path, const char *property)
-{
-    QDict *r;
-    bool b;
-
-    r = qtest_qmp(s, "{ 'execute': 'qom-get', 'arguments': "
-                     "{ 'path': %s, 'property': %s } }", path, property);
-    b = qdict_get_bool(r, "return");
-    qobject_unref(r);
-
-    return b;
-}
-
-static void qom_set_bool(QTestState *s, const char *path, const char *property,
-                         bool value)
-{
-    QDict *r;
-
-    r = qtest_qmp(s, "{ 'execute': 'qom-set', 'arguments': "
-                     "{ 'path': %s, 'property': %s, 'value': %i } }",
-                     path, property, value);
-    qobject_unref(r);
-}
-
 static void test_set_colocated_pins(const void *data)
 {
     QTestState *s = (QTestState *)data;
@@ -60,14 +36,14 @@ static void test_set_colocated_pins(const void *data)
      * gpioV4-7 occupy bits within a single 32-bit value, so we want to make
      * sure that modifying one doesn't affect the other.
      */
-    qom_set_bool(s, "/machine/soc/gpio", "gpioV4", true);
-    qom_set_bool(s, "/machine/soc/gpio", "gpioV5", false);
-    qom_set_bool(s, "/machine/soc/gpio", "gpioV6", true);
-    qom_set_bool(s, "/machine/soc/gpio", "gpioV7", false);
-    g_assert(qom_get_bool(s, "/machine/soc/gpio", "gpioV4"));
-    g_assert(!qom_get_bool(s, "/machine/soc/gpio", "gpioV5"));
-    g_assert(qom_get_bool(s, "/machine/soc/gpio", "gpioV6"));
-    g_assert(!qom_get_bool(s, "/machine/soc/gpio", "gpioV7"));
+    qtest_qom_set_bool(s, "/machine/soc/gpio", "gpioV4", true);
+    qtest_qom_set_bool(s, "/machine/soc/gpio", "gpioV5", false);
+    qtest_qom_set_bool(s, "/machine/soc/gpio", "gpioV6", true);
+    qtest_qom_set_bool(s, "/machine/soc/gpio", "gpioV7", false);
+    g_assert(qtest_qom_get_bool(s, "/machine/soc/gpio", "gpioV4"));
+    g_assert(!qtest_qom_get_bool(s, "/machine/soc/gpio", "gpioV5"));
+    g_assert(qtest_qom_get_bool(s, "/machine/soc/gpio", "gpioV6"));
+    g_assert(!qtest_qom_get_bool(s, "/machine/soc/gpio", "gpioV7"));
 }
 
 int main(int argc, char **argv)
diff --git a/tests/qtest/aspeed_smc-test.c b/tests/qtest/aspeed_smc-test.c
index 87b40a0ef1..ec233315e6 100644
--- a/tests/qtest/aspeed_smc-test.c
+++ b/tests/qtest/aspeed_smc-test.c
@@ -26,6 +26,7 @@
 #include "qemu/osdep.h"
 #include "qemu/bswap.h"
 #include "libqtest-single.h"
+#include "qemu/bitops.h"
 
 /*
  * ASPEED SPI Controller registers
@@ -40,6 +41,7 @@
 #define   CTRL_FREADMODE       0x1
 #define   CTRL_WRITEMODE       0x2
 #define   CTRL_USERMODE        0x3
+#define SR_WEL BIT(1)
 
 #define ASPEED_FMC_BASE    0x1E620000
 #define ASPEED_FLASH_BASE  0x20000000
@@ -49,6 +51,8 @@
  */
 enum {
     JEDEC_READ = 0x9f,
+    RDSR = 0x5,
+    WRDI = 0x4,
     BULK_ERASE = 0xc7,
     READ = 0x03,
     PP = 0x02,
@@ -348,6 +352,44 @@ static void test_write_page_mem(void)
     flash_reset();
 }
 
+static void test_read_status_reg(void)
+{
+    uint8_t r;
+
+    spi_conf(CONF_ENABLE_W0);
+
+    spi_ctrl_start_user();
+    writeb(ASPEED_FLASH_BASE, RDSR);
+    r = readb(ASPEED_FLASH_BASE);
+    spi_ctrl_stop_user();
+
+    g_assert_cmphex(r & SR_WEL, ==, 0);
+    g_assert(!qtest_qom_get_bool
+            (global_qtest, "/machine/soc/fmc/ssi.0/child[0]", "write-enable"));
+
+    spi_ctrl_start_user();
+    writeb(ASPEED_FLASH_BASE, WREN);
+    writeb(ASPEED_FLASH_BASE, RDSR);
+    r = readb(ASPEED_FLASH_BASE);
+    spi_ctrl_stop_user();
+
+    g_assert_cmphex(r & SR_WEL, ==, SR_WEL);
+    g_assert(qtest_qom_get_bool
+            (global_qtest, "/machine/soc/fmc/ssi.0/child[0]", "write-enable"));
+
+    spi_ctrl_start_user();
+    writeb(ASPEED_FLASH_BASE, WRDI);
+    writeb(ASPEED_FLASH_BASE, RDSR);
+    r = readb(ASPEED_FLASH_BASE);
+    spi_ctrl_stop_user();
+
+    g_assert_cmphex(r & SR_WEL, ==, 0);
+    g_assert(!qtest_qom_get_bool
+            (global_qtest, "/machine/soc/fmc/ssi.0/child[0]", "write-enable"));
+
+    flash_reset();
+}
+
 static char tmp_path[] = "/tmp/qtest.m25p80.XXXXXX";
 
 int main(int argc, char **argv)
@@ -373,6 +415,7 @@ int main(int argc, char **argv)
     qtest_add_func("/ast2400/smc/write_page", test_write_page);
     qtest_add_func("/ast2400/smc/read_page_mem", test_read_page_mem);
     qtest_add_func("/ast2400/smc/write_page_mem", test_write_page_mem);
+    qtest_add_func("/ast2400/smc/read_status_reg", test_read_status_reg);
 
     ret = g_test_run();
 
diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c
index 2e49618454..8c159eacf5 100644
--- a/tests/qtest/libqtest.c
+++ b/tests/qtest/libqtest.c
@@ -1440,3 +1440,27 @@ void qtest_client_inproc_recv(void *opaque, const char *str)
     g_string_append(qts->rx, str);
     return;
 }
+
+void qtest_qom_set_bool(QTestState *s, const char *path, const char *property,
+                         bool value)
+{
+    QDict *r;
+
+    r = qtest_qmp(s, "{ 'execute': 'qom-set', 'arguments': "
+                     "{ 'path': %s, 'property': %s, 'value': %i } }",
+                     path, property, value);
+    qobject_unref(r);
+}
+
+bool qtest_qom_get_bool(QTestState *s, const char *path, const char *property)
+{
+    QDict *r;
+    bool b;
+
+    r = qtest_qmp(s, "{ 'execute': 'qom-get', 'arguments': "
+                     "{ 'path': %s, 'property': %s } }", path, property);
+    b = qdict_get_bool(r, "return");
+    qobject_unref(r);
+
+    return b;
+}
diff --git a/tests/qtest/libqtest.h b/tests/qtest/libqtest.h
index 4ab0cad326..94b187837d 100644
--- a/tests/qtest/libqtest.h
+++ b/tests/qtest/libqtest.h
@@ -783,4 +783,26 @@ QTestState *qtest_inproc_init(QTestState **s, bool log, const char* arch,
                     void (*send)(void*, const char*));
 
 void qtest_client_inproc_recv(void *opaque, const char *str);
+
+/**
+ * qtest_qom_set_bool:
+ * @s: QTestState instance to operate on.
+ * @path: Path to the property being set.
+ * @property: Property being set.
+ * @value: Value to set the property.
+ *
+ * Set the property with passed in value.
+ */
+void qtest_qom_set_bool(QTestState *s, const char *path, const char *property,
+                         bool value);
+
+/**
+ * qtest_qom_get_bool:
+ * @s: QTestState instance to operate on.
+ * @path: Path to the property being retrieved.
+ * @property: Property from where the value is being retrieved.
+ *
+ * Returns: Value retrieved from property.
+ */
+bool qtest_qom_get_bool(QTestState *s, const char *path, const char *property);
 #endif

From 699db71520502836efc0e9102b0ffa6b0e5d0758 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= <clg@kaod.org>
Date: Wed, 25 May 2022 10:31:33 +0200
Subject: [PATCH 491/935] aspeed: Introduce a get_irq AspeedSoCClass method
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

and make routine aspeed_soc_get_irq() common to all SoCs. This will be
useful to share code.

Cc: Jamin Lin <jamin_lin@aspeedtech.com>
Cc: Peter Delevoryas <pdel@fb.com>
Reviewed-by: Peter Delevoryas <pdel@fb.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220516055620.2380197-1-clg@kaod.org>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
 hw/arm/aspeed_ast10x0.c     |  5 +++--
 hw/arm/aspeed_ast2600.c     |  5 +++--
 hw/arm/aspeed_soc.c         | 13 ++++++++++---
 include/hw/arm/aspeed_soc.h |  3 +++
 4 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/hw/arm/aspeed_ast10x0.c b/hw/arm/aspeed_ast10x0.c
index 4271549282..9ae9efaac1 100644
--- a/hw/arm/aspeed_ast10x0.c
+++ b/hw/arm/aspeed_ast10x0.c
@@ -61,11 +61,11 @@ static const int aspeed_soc_ast1030_irqmap[] = {
     [ASPEED_DEV_KCS]       = 138, /* 138 -> 142 */
 };
 
-static qemu_irq aspeed_soc_get_irq(AspeedSoCState *s, int ctrl)
+static qemu_irq aspeed_soc_ast1030_get_irq(AspeedSoCState *s, int dev)
 {
     AspeedSoCClass *sc = ASPEED_SOC_GET_CLASS(s);
 
-    return qdev_get_gpio_in(DEVICE(&s->armv7m), sc->irqmap[ctrl]);
+    return qdev_get_gpio_in(DEVICE(&s->armv7m), sc->irqmap[dev]);
 }
 
 static void aspeed_soc_ast1030_init(Object *obj)
@@ -280,6 +280,7 @@ static void aspeed_soc_ast1030_class_init(ObjectClass *klass, void *data)
     sc->irqmap = aspeed_soc_ast1030_irqmap;
     sc->memmap = aspeed_soc_ast1030_memmap;
     sc->num_cpus = 1;
+    sc->get_irq = aspeed_soc_ast1030_get_irq;
 }
 
 static const TypeInfo aspeed_soc_ast1030_type_info = {
diff --git a/hw/arm/aspeed_ast2600.c b/hw/arm/aspeed_ast2600.c
index eedda7badc..4161a0cc4b 100644
--- a/hw/arm/aspeed_ast2600.c
+++ b/hw/arm/aspeed_ast2600.c
@@ -114,11 +114,11 @@ static const int aspeed_soc_ast2600_irqmap[] = {
     [ASPEED_DEV_I3C]       = 102,   /* 102 -> 107 */
 };
 
-static qemu_irq aspeed_soc_get_irq(AspeedSoCState *s, int ctrl)
+static qemu_irq aspeed_soc_ast2600_get_irq(AspeedSoCState *s, int dev)
 {
     AspeedSoCClass *sc = ASPEED_SOC_GET_CLASS(s);
 
-    return qdev_get_gpio_in(DEVICE(&s->a7mpcore), sc->irqmap[ctrl]);
+    return qdev_get_gpio_in(DEVICE(&s->a7mpcore), sc->irqmap[dev]);
 }
 
 static void aspeed_soc_ast2600_init(Object *obj)
@@ -572,6 +572,7 @@ static void aspeed_soc_ast2600_class_init(ObjectClass *oc, void *data)
     sc->irqmap       = aspeed_soc_ast2600_irqmap;
     sc->memmap       = aspeed_soc_ast2600_memmap;
     sc->num_cpus     = 2;
+    sc->get_irq      = aspeed_soc_ast2600_get_irq;
 }
 
 static const TypeInfo aspeed_soc_ast2600_type_info = {
diff --git a/hw/arm/aspeed_soc.c b/hw/arm/aspeed_soc.c
index 58714cb2a0..c339b5c74d 100644
--- a/hw/arm/aspeed_soc.c
+++ b/hw/arm/aspeed_soc.c
@@ -121,11 +121,11 @@ static const int aspeed_soc_ast2400_irqmap[] = {
 
 #define aspeed_soc_ast2500_irqmap aspeed_soc_ast2400_irqmap
 
-static qemu_irq aspeed_soc_get_irq(AspeedSoCState *s, int ctrl)
+static qemu_irq aspeed_soc_ast2400_get_irq(AspeedSoCState *s, int dev)
 {
     AspeedSoCClass *sc = ASPEED_SOC_GET_CLASS(s);
 
-    return qdev_get_gpio_in(DEVICE(&s->vic), sc->irqmap[ctrl]);
+    return qdev_get_gpio_in(DEVICE(&s->vic), sc->irqmap[dev]);
 }
 
 static void aspeed_soc_init(Object *obj)
@@ -487,6 +487,7 @@ static void aspeed_soc_ast2400_class_init(ObjectClass *oc, void *data)
     sc->irqmap       = aspeed_soc_ast2400_irqmap;
     sc->memmap       = aspeed_soc_ast2400_memmap;
     sc->num_cpus     = 1;
+    sc->get_irq      = aspeed_soc_ast2400_get_irq;
 }
 
 static const TypeInfo aspeed_soc_ast2400_type_info = {
@@ -512,6 +513,7 @@ static void aspeed_soc_ast2500_class_init(ObjectClass *oc, void *data)
     sc->irqmap       = aspeed_soc_ast2500_irqmap;
     sc->memmap       = aspeed_soc_ast2500_memmap;
     sc->num_cpus     = 1;
+    sc->get_irq      = aspeed_soc_ast2400_get_irq;
 }
 
 static const TypeInfo aspeed_soc_ast2500_type_info = {
@@ -528,4 +530,9 @@ static void aspeed_soc_register_types(void)
     type_register_static(&aspeed_soc_ast2500_type_info);
 };
 
-type_init(aspeed_soc_register_types)
+type_init(aspeed_soc_register_types);
+
+qemu_irq aspeed_soc_get_irq(AspeedSoCState *s, int dev)
+{
+    return ASPEED_SOC_GET_CLASS(s)->get_irq(s, dev);
+}
diff --git a/include/hw/arm/aspeed_soc.h b/include/hw/arm/aspeed_soc.h
index e13af374b9..3789f38603 100644
--- a/include/hw/arm/aspeed_soc.h
+++ b/include/hw/arm/aspeed_soc.h
@@ -94,6 +94,7 @@ struct AspeedSoCClass {
     const int *irqmap;
     const hwaddr *memmap;
     uint32_t num_cpus;
+    qemu_irq (*get_irq)(AspeedSoCState *s, int dev);
 };
 
 
@@ -153,4 +154,6 @@ enum {
     ASPEED_DEV_I3C,
 };
 
+qemu_irq aspeed_soc_get_irq(AspeedSoCState *s, int dev);
+
 #endif /* ASPEED_SOC_H */

From ab5e86053d16721b5b92780e23bc3104fdcb1f05 Mon Sep 17 00:00:00 2001
From: Peter Delevoryas <pdel@fb.com>
Date: Wed, 25 May 2022 10:31:33 +0200
Subject: [PATCH 492/935] hw: aspeed: Add missing UART's
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This adds the missing UART memory and IRQ mappings for the AST2400, AST2500,
AST2600, and AST1030.

This also includes the new UART interfaces added in the AST2600 and AST1030
from UART6 to UART13. The addresses and interrupt numbers for these two
later chips are identical.

Signed-off-by: Peter Delevoryas <pdel@fb.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220516062328.298336-2-pdel@fb.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
 hw/arm/aspeed_ast10x0.c     | 24 ++++++++++++++++++++++++
 hw/arm/aspeed_ast2600.c     | 19 +++++++++++++++++++
 hw/arm/aspeed_soc.c         |  6 ++++++
 include/hw/arm/aspeed_soc.h |  8 ++++++++
 4 files changed, 57 insertions(+)

diff --git a/hw/arm/aspeed_ast10x0.c b/hw/arm/aspeed_ast10x0.c
index 9ae9efaac1..fa2cc4406c 100644
--- a/hw/arm/aspeed_ast10x0.c
+++ b/hw/arm/aspeed_ast10x0.c
@@ -33,14 +33,38 @@ static const hwaddr aspeed_soc_ast1030_memmap[] = {
     [ASPEED_DEV_SBC]       = 0x7E6F2000,
     [ASPEED_DEV_GPIO]      = 0x7E780000,
     [ASPEED_DEV_TIMER1]    = 0x7E782000,
+    [ASPEED_DEV_UART1]     = 0x7E783000,
+    [ASPEED_DEV_UART2]     = 0x7E78D000,
+    [ASPEED_DEV_UART3]     = 0x7E78E000,
+    [ASPEED_DEV_UART4]     = 0x7E78F000,
     [ASPEED_DEV_UART5]     = 0x7E784000,
+    [ASPEED_DEV_UART6]     = 0x7E790000,
+    [ASPEED_DEV_UART7]     = 0x7E790100,
+    [ASPEED_DEV_UART8]     = 0x7E790200,
+    [ASPEED_DEV_UART9]     = 0x7E790300,
+    [ASPEED_DEV_UART10]    = 0x7E790400,
+    [ASPEED_DEV_UART11]    = 0x7E790500,
+    [ASPEED_DEV_UART12]    = 0x7E790600,
+    [ASPEED_DEV_UART13]    = 0x7E790700,
     [ASPEED_DEV_WDT]       = 0x7E785000,
     [ASPEED_DEV_LPC]       = 0x7E789000,
     [ASPEED_DEV_I2C]       = 0x7E7B0000,
 };
 
 static const int aspeed_soc_ast1030_irqmap[] = {
+    [ASPEED_DEV_UART1]     = 47,
+    [ASPEED_DEV_UART2]     = 48,
+    [ASPEED_DEV_UART3]     = 49,
+    [ASPEED_DEV_UART4]     = 50,
     [ASPEED_DEV_UART5]     = 8,
+    [ASPEED_DEV_UART6]     = 57,
+    [ASPEED_DEV_UART7]     = 58,
+    [ASPEED_DEV_UART8]     = 59,
+    [ASPEED_DEV_UART9]     = 60,
+    [ASPEED_DEV_UART10]    = 61,
+    [ASPEED_DEV_UART11]    = 62,
+    [ASPEED_DEV_UART12]    = 63,
+    [ASPEED_DEV_UART13]    = 64,
     [ASPEED_DEV_GPIO]      = 11,
     [ASPEED_DEV_TIMER1]    = 16,
     [ASPEED_DEV_TIMER2]    = 17,
diff --git a/hw/arm/aspeed_ast2600.c b/hw/arm/aspeed_ast2600.c
index 4161a0cc4b..f3ecc0f3b7 100644
--- a/hw/arm/aspeed_ast2600.c
+++ b/hw/arm/aspeed_ast2600.c
@@ -61,7 +61,18 @@ static const hwaddr aspeed_soc_ast2600_memmap[] = {
     [ASPEED_DEV_IBT]       = 0x1E789140,
     [ASPEED_DEV_I2C]       = 0x1E78A000,
     [ASPEED_DEV_UART1]     = 0x1E783000,
+    [ASPEED_DEV_UART2]     = 0x1E78D000,
+    [ASPEED_DEV_UART3]     = 0x1E78E000,
+    [ASPEED_DEV_UART4]     = 0x1E78F000,
     [ASPEED_DEV_UART5]     = 0x1E784000,
+    [ASPEED_DEV_UART6]     = 0x1E790000,
+    [ASPEED_DEV_UART7]     = 0x1E790100,
+    [ASPEED_DEV_UART8]     = 0x1E790200,
+    [ASPEED_DEV_UART9]     = 0x1E790300,
+    [ASPEED_DEV_UART10]    = 0x1E790400,
+    [ASPEED_DEV_UART11]    = 0x1E790500,
+    [ASPEED_DEV_UART12]    = 0x1E790600,
+    [ASPEED_DEV_UART13]    = 0x1E790700,
     [ASPEED_DEV_VUART]     = 0x1E787000,
     [ASPEED_DEV_I3C]       = 0x1E7A0000,
     [ASPEED_DEV_SDRAM]     = 0x80000000,
@@ -78,6 +89,14 @@ static const int aspeed_soc_ast2600_irqmap[] = {
     [ASPEED_DEV_UART3]     = 49,
     [ASPEED_DEV_UART4]     = 50,
     [ASPEED_DEV_UART5]     = 8,
+    [ASPEED_DEV_UART6]     = 57,
+    [ASPEED_DEV_UART7]     = 58,
+    [ASPEED_DEV_UART8]     = 59,
+    [ASPEED_DEV_UART9]     = 60,
+    [ASPEED_DEV_UART10]    = 61,
+    [ASPEED_DEV_UART11]    = 62,
+    [ASPEED_DEV_UART12]    = 63,
+    [ASPEED_DEV_UART13]    = 64,
     [ASPEED_DEV_VUART]     = 8,
     [ASPEED_DEV_FMC]       = 39,
     [ASPEED_DEV_SDMC]      = 0,
diff --git a/hw/arm/aspeed_soc.c b/hw/arm/aspeed_soc.c
index c339b5c74d..96bc060680 100644
--- a/hw/arm/aspeed_soc.c
+++ b/hw/arm/aspeed_soc.c
@@ -48,6 +48,9 @@ static const hwaddr aspeed_soc_ast2400_memmap[] = {
     [ASPEED_DEV_ETH1]   = 0x1E660000,
     [ASPEED_DEV_ETH2]   = 0x1E680000,
     [ASPEED_DEV_UART1]  = 0x1E783000,
+    [ASPEED_DEV_UART2]  = 0x1E78D000,
+    [ASPEED_DEV_UART3]  = 0x1E78E000,
+    [ASPEED_DEV_UART4]  = 0x1E78F000,
     [ASPEED_DEV_UART5]  = 0x1E784000,
     [ASPEED_DEV_VUART]  = 0x1E787000,
     [ASPEED_DEV_SDRAM]  = 0x40000000,
@@ -80,6 +83,9 @@ static const hwaddr aspeed_soc_ast2500_memmap[] = {
     [ASPEED_DEV_ETH1]   = 0x1E660000,
     [ASPEED_DEV_ETH2]   = 0x1E680000,
     [ASPEED_DEV_UART1]  = 0x1E783000,
+    [ASPEED_DEV_UART2]  = 0x1E78D000,
+    [ASPEED_DEV_UART3]  = 0x1E78E000,
+    [ASPEED_DEV_UART4]  = 0x1E78F000,
     [ASPEED_DEV_UART5]  = 0x1E784000,
     [ASPEED_DEV_VUART]  = 0x1E787000,
     [ASPEED_DEV_SDRAM]  = 0x80000000,
diff --git a/include/hw/arm/aspeed_soc.h b/include/hw/arm/aspeed_soc.h
index 3789f38603..709a78285b 100644
--- a/include/hw/arm/aspeed_soc.h
+++ b/include/hw/arm/aspeed_soc.h
@@ -105,6 +105,14 @@ enum {
     ASPEED_DEV_UART3,
     ASPEED_DEV_UART4,
     ASPEED_DEV_UART5,
+    ASPEED_DEV_UART6,
+    ASPEED_DEV_UART7,
+    ASPEED_DEV_UART8,
+    ASPEED_DEV_UART9,
+    ASPEED_DEV_UART10,
+    ASPEED_DEV_UART11,
+    ASPEED_DEV_UART12,
+    ASPEED_DEV_UART13,
     ASPEED_DEV_VUART,
     ASPEED_DEV_FMC,
     ASPEED_DEV_SPI1,

From c5e1bdb9e243ad5563196175415f42e459040367 Mon Sep 17 00:00:00 2001
From: Peter Delevoryas <pdel@fb.com>
Date: Wed, 25 May 2022 10:31:33 +0200
Subject: [PATCH 493/935] hw: aspeed: Add uarts_num SoC attribute
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

AST2400 and AST2500 have 5 UART's, while the AST2600 and AST1030 have 13.

Signed-off-by: Peter Delevoryas <pdel@fb.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220516062328.298336-3-pdel@fb.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
 hw/arm/aspeed_ast10x0.c     | 1 +
 hw/arm/aspeed_ast2600.c     | 1 +
 hw/arm/aspeed_soc.c         | 2 ++
 include/hw/arm/aspeed_soc.h | 1 +
 4 files changed, 5 insertions(+)

diff --git a/hw/arm/aspeed_ast10x0.c b/hw/arm/aspeed_ast10x0.c
index fa2cc4406c..bb8177e86c 100644
--- a/hw/arm/aspeed_ast10x0.c
+++ b/hw/arm/aspeed_ast10x0.c
@@ -301,6 +301,7 @@ static void aspeed_soc_ast1030_class_init(ObjectClass *klass, void *data)
     sc->ehcis_num = 0;
     sc->wdts_num = 4;
     sc->macs_num = 1;
+    sc->uarts_num = 13;
     sc->irqmap = aspeed_soc_ast1030_irqmap;
     sc->memmap = aspeed_soc_ast1030_memmap;
     sc->num_cpus = 1;
diff --git a/hw/arm/aspeed_ast2600.c b/hw/arm/aspeed_ast2600.c
index f3ecc0f3b7..a9523074a0 100644
--- a/hw/arm/aspeed_ast2600.c
+++ b/hw/arm/aspeed_ast2600.c
@@ -588,6 +588,7 @@ static void aspeed_soc_ast2600_class_init(ObjectClass *oc, void *data)
     sc->ehcis_num    = 2;
     sc->wdts_num     = 4;
     sc->macs_num     = 4;
+    sc->uarts_num    = 13;
     sc->irqmap       = aspeed_soc_ast2600_irqmap;
     sc->memmap       = aspeed_soc_ast2600_memmap;
     sc->num_cpus     = 2;
diff --git a/hw/arm/aspeed_soc.c b/hw/arm/aspeed_soc.c
index 96bc060680..7008cd1af7 100644
--- a/hw/arm/aspeed_soc.c
+++ b/hw/arm/aspeed_soc.c
@@ -490,6 +490,7 @@ static void aspeed_soc_ast2400_class_init(ObjectClass *oc, void *data)
     sc->ehcis_num    = 1;
     sc->wdts_num     = 2;
     sc->macs_num     = 2;
+    sc->uarts_num    = 5;
     sc->irqmap       = aspeed_soc_ast2400_irqmap;
     sc->memmap       = aspeed_soc_ast2400_memmap;
     sc->num_cpus     = 1;
@@ -516,6 +517,7 @@ static void aspeed_soc_ast2500_class_init(ObjectClass *oc, void *data)
     sc->ehcis_num    = 2;
     sc->wdts_num     = 3;
     sc->macs_num     = 2;
+    sc->uarts_num    = 5;
     sc->irqmap       = aspeed_soc_ast2500_irqmap;
     sc->memmap       = aspeed_soc_ast2500_memmap;
     sc->num_cpus     = 1;
diff --git a/include/hw/arm/aspeed_soc.h b/include/hw/arm/aspeed_soc.h
index 709a78285b..669bc49855 100644
--- a/include/hw/arm/aspeed_soc.h
+++ b/include/hw/arm/aspeed_soc.h
@@ -91,6 +91,7 @@ struct AspeedSoCClass {
     int ehcis_num;
     int wdts_num;
     int macs_num;
+    int uarts_num;
     const int *irqmap;
     const hwaddr *memmap;
     uint32_t num_cpus;

From 94d10f42109a5ba23e26012722b16f0f479990cb Mon Sep 17 00:00:00 2001
From: Peter Delevoryas <pdel@fb.com>
Date: Wed, 25 May 2022 10:31:33 +0200
Subject: [PATCH 494/935] hw: aspeed: Ensure AST1030 respects uart-default
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The AST1030 machine initialization was not respecting the Aspeed SoC
property "uart-default", which specifies which UART should be connected to
the first serial device, it was just always connecting UART5. This doesn't
change any behavior, because the default value for "uart-default" is UART5,
but it makes it possible to override this in new machine definitions using
the AST1030.

Signed-off-by: Peter Delevoryas <pdel@fb.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220516062328.298336-4-pdel@fb.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
 hw/arm/aspeed_ast10x0.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/hw/arm/aspeed_ast10x0.c b/hw/arm/aspeed_ast10x0.c
index bb8177e86c..faafb800f3 100644
--- a/hw/arm/aspeed_ast10x0.c
+++ b/hw/arm/aspeed_ast10x0.c
@@ -215,9 +215,9 @@ static void aspeed_soc_ast1030_realize(DeviceState *dev_soc, Error **errp)
                        qdev_get_gpio_in(DEVICE(&s->armv7m),
                                 sc->irqmap[ASPEED_DEV_KCS] + aspeed_lpc_kcs_4));
 
-    /* UART5 - attach an 8250 to the IO space as our UART */
-    serial_mm_init(get_system_memory(), sc->memmap[ASPEED_DEV_UART5], 2,
-                   aspeed_soc_get_irq(s, ASPEED_DEV_UART5),
+    /* UART - attach an 8250 to the IO space as our UART */
+    serial_mm_init(get_system_memory(), sc->memmap[s->uart_default], 2,
+                   aspeed_soc_get_irq(s, s->uart_default),
                    38400, serial_hd(0), DEVICE_LITTLE_ENDIAN);
 
     /* Timer */

From 470253b6d05ecec4bf7c8f8616db779454414292 Mon Sep 17 00:00:00 2001
From: Peter Delevoryas <pdel@fb.com>
Date: Wed, 25 May 2022 10:31:33 +0200
Subject: [PATCH 495/935] hw: aspeed: Introduce common UART init function
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Peter Delevoryas <pdel@fb.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220516062328.298336-5-pdel@fb.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
 hw/arm/aspeed_ast10x0.c     |  7 ++-----
 hw/arm/aspeed_ast2600.c     |  7 ++-----
 hw/arm/aspeed_soc.c         | 16 ++++++++++++----
 include/hw/arm/aspeed_soc.h |  1 +
 4 files changed, 17 insertions(+), 14 deletions(-)

diff --git a/hw/arm/aspeed_ast10x0.c b/hw/arm/aspeed_ast10x0.c
index faafb800f3..938615d55f 100644
--- a/hw/arm/aspeed_ast10x0.c
+++ b/hw/arm/aspeed_ast10x0.c
@@ -15,7 +15,6 @@
 #include "sysemu/sysemu.h"
 #include "hw/qdev-clock.h"
 #include "hw/misc/unimp.h"
-#include "hw/char/serial.h"
 #include "hw/arm/aspeed_soc.h"
 
 #define ASPEED_SOC_IOMEM_SIZE 0x00200000
@@ -215,10 +214,8 @@ static void aspeed_soc_ast1030_realize(DeviceState *dev_soc, Error **errp)
                        qdev_get_gpio_in(DEVICE(&s->armv7m),
                                 sc->irqmap[ASPEED_DEV_KCS] + aspeed_lpc_kcs_4));
 
-    /* UART - attach an 8250 to the IO space as our UART */
-    serial_mm_init(get_system_memory(), sc->memmap[s->uart_default], 2,
-                   aspeed_soc_get_irq(s, s->uart_default),
-                   38400, serial_hd(0), DEVICE_LITTLE_ENDIAN);
+    /* UART */
+    aspeed_soc_uart_init(s);
 
     /* Timer */
     object_property_set_link(OBJECT(&s->timerctrl), "scu", OBJECT(&s->scu),
diff --git a/hw/arm/aspeed_ast2600.c b/hw/arm/aspeed_ast2600.c
index a9523074a0..b0a4199b69 100644
--- a/hw/arm/aspeed_ast2600.c
+++ b/hw/arm/aspeed_ast2600.c
@@ -11,7 +11,6 @@
 #include "qapi/error.h"
 #include "hw/misc/unimp.h"
 #include "hw/arm/aspeed_soc.h"
-#include "hw/char/serial.h"
 #include "qemu/module.h"
 #include "qemu/error-report.h"
 #include "hw/i2c/aspeed_i2c.h"
@@ -372,10 +371,8 @@ static void aspeed_soc_ast2600_realize(DeviceState *dev, Error **errp)
     sysbus_connect_irq(SYS_BUS_DEVICE(&s->adc), 0,
                        aspeed_soc_get_irq(s, ASPEED_DEV_ADC));
 
-    /* UART - attach an 8250 to the IO space as our UART */
-    serial_mm_init(get_system_memory(), sc->memmap[s->uart_default], 2,
-                   aspeed_soc_get_irq(s, s->uart_default), 38400,
-                   serial_hd(0), DEVICE_LITTLE_ENDIAN);
+    /* UART */
+    aspeed_soc_uart_init(s);
 
     /* I2C */
     object_property_set_link(OBJECT(&s->i2c), "dram", OBJECT(s->dram_mr),
diff --git a/hw/arm/aspeed_soc.c b/hw/arm/aspeed_soc.c
index 7008cd1af7..912798a9c9 100644
--- a/hw/arm/aspeed_soc.c
+++ b/hw/arm/aspeed_soc.c
@@ -303,10 +303,8 @@ static void aspeed_soc_realize(DeviceState *dev, Error **errp)
     sysbus_connect_irq(SYS_BUS_DEVICE(&s->adc), 0,
                        aspeed_soc_get_irq(s, ASPEED_DEV_ADC));
 
-    /* UART - attach an 8250 to the IO space as our UART */
-    serial_mm_init(get_system_memory(), sc->memmap[s->uart_default], 2,
-                   aspeed_soc_get_irq(s, s->uart_default), 38400,
-                   serial_hd(0), DEVICE_LITTLE_ENDIAN);
+    /* UART */
+    aspeed_soc_uart_init(s);
 
     /* I2C */
     object_property_set_link(OBJECT(&s->i2c), "dram", OBJECT(s->dram_mr),
@@ -544,3 +542,13 @@ qemu_irq aspeed_soc_get_irq(AspeedSoCState *s, int dev)
 {
     return ASPEED_SOC_GET_CLASS(s)->get_irq(s, dev);
 }
+
+void aspeed_soc_uart_init(AspeedSoCState *s)
+{
+    AspeedSoCClass *sc = ASPEED_SOC_GET_CLASS(s);
+
+    /* Attach an 8250 to the IO space as our UART */
+    serial_mm_init(get_system_memory(), sc->memmap[s->uart_default], 2,
+                   aspeed_soc_get_irq(s, s->uart_default), 38400,
+                   serial_hd(0), DEVICE_LITTLE_ENDIAN);
+}
diff --git a/include/hw/arm/aspeed_soc.h b/include/hw/arm/aspeed_soc.h
index 669bc49855..02a5a9ffcb 100644
--- a/include/hw/arm/aspeed_soc.h
+++ b/include/hw/arm/aspeed_soc.h
@@ -164,5 +164,6 @@ enum {
 };
 
 qemu_irq aspeed_soc_get_irq(AspeedSoCState *s, int dev);
+void aspeed_soc_uart_init(AspeedSoCState *s);
 
 #endif /* ASPEED_SOC_H */

From 6827ff20b2975e84045ba356ba3e6aadc686a53c Mon Sep 17 00:00:00 2001
From: Peter Delevoryas <pdel@fb.com>
Date: Wed, 25 May 2022 10:31:33 +0200
Subject: [PATCH 496/935] hw: aspeed: Init all UART's with serial devices
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Background:

AspeedMachineClass.uart_default specifies the serial console UART, which
usually corresponds to the "stdout-path" in the device tree.

The default value is UART5, since most boards use UART5 for this:

    amc->uart_default = ASPEED_DEV_UART5;

Users can override AspeedMachineClass.uart_default in their board's machine
class init to specify something besides UART5. For example, for fuji-bmc:

    amc->uart_default = ASPEED_DEV_UART1;

We only connect this one UART, of the 5 UART's on the AST2400 and AST2500
and the 13 UART's on the AST2600 and AST1030, to a serial device that QEMU
users can use. None of the other UART's are initialized, and the only way
to override this attribute is by creating a specialized board definition,
requiring QEMU source code changes and rebuilding.

The result of this is that if you want to get serial console output on a
board that uses UART3, you need to add a board definition. This was
encountered by Zev in OpenBMC. [1]

Changes:

This commit initializes all of the UART's present on each Aspeed chip with
serial devices and allows the QEMU user to connect as many or few as they
like to serial devices. For example, you can still run QEMU and just connect
stdout to the machine's default UART, without specifying any additional
serial devices:

    qemu-system-arm -machine fuji-bmc \
        -drive file=fuji.mtd,format=raw,if=mtd \
        -nographic

However, if you don't want to add a special machine definition, you can now
manually configure UART1 to connect to stdout and get serial console output,
even if the machine's default is UART5:

    qemu-system-arm -machine ast2600-evb \
        -drive file=fuji.mtd,format=raw,if=mtd \
        -serial null -serial mon:stdio -display none

In the example above, the first "-serial null" argument is connected to
UART5, and "-serial mon:stdio" is connected to UART1.

Another example: you can get serial console output from Wedge100, which uses
UART3, by reusing the palmetto AST2400 machine and rewiring the serial
device arguments:

    qemu-system-arm -machine palmetto-bmc \
        -drive file=wedge100.mtd,format=raw,if=mtd \
        -serial null -serial null -serial null \
        -serial mon:stdio -display none

There is a slight change in behavior introduced with this change: now, each
UART's memory-mapped IO region will have a serial device model connected to
it. Previously, all reads and writes to those regions would be ineffective
and return zero values, but now some values will be nonzero, even when the
user doesn't connect a serial device backend (like a socket, file, etc). For
example, the line status register might indicate that the transmit buffer is
empty now, whereas previously it might have always indicated it was full.

[1] https://lore.kernel.org/openbmc/YnzGnWjkYdMUUNyM@hatter.bewilderbeest.net/
[2] https://github.com/facebook/openbmc/releases/download/v2021.49.0/fuji.mtd
[3] https://github.com/facebook/openbmc/releases/download/v2021.49.0/wedge100.mtd

Signed-off-by: Peter Delevoryas <pdel@fb.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220516062328.298336-6-pdel@fb.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
 hw/arm/aspeed_soc.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/hw/arm/aspeed_soc.c b/hw/arm/aspeed_soc.c
index 912798a9c9..30574d4276 100644
--- a/hw/arm/aspeed_soc.c
+++ b/hw/arm/aspeed_soc.c
@@ -546,9 +546,18 @@ qemu_irq aspeed_soc_get_irq(AspeedSoCState *s, int dev)
 void aspeed_soc_uart_init(AspeedSoCState *s)
 {
     AspeedSoCClass *sc = ASPEED_SOC_GET_CLASS(s);
+    int i, uart;
 
     /* Attach an 8250 to the IO space as our UART */
     serial_mm_init(get_system_memory(), sc->memmap[s->uart_default], 2,
                    aspeed_soc_get_irq(s, s->uart_default), 38400,
                    serial_hd(0), DEVICE_LITTLE_ENDIAN);
+    for (i = 1, uart = ASPEED_DEV_UART1; i < sc->uarts_num; i++, uart++) {
+        if (uart == s->uart_default) {
+            uart++;
+        }
+        serial_mm_init(get_system_memory(), sc->memmap[uart], 2,
+                       aspeed_soc_get_irq(s, uart), 38400,
+                       serial_hd(i), DEVICE_LITTLE_ENDIAN);
+    }
 }

From 7b1d21a8ba1e6091f1e54c74e7c664e26300accc Mon Sep 17 00:00:00 2001
From: Jamin Lin <jamin_lin@aspeedtech.com>
Date: Wed, 25 May 2022 10:31:33 +0200
Subject: [PATCH 497/935] hw/gpio Add GPIO read/write trace event.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add GPIO read/write trace event for aspeed model.

Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220525053444.27228-2-jamin_lin@aspeedtech.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
 hw/gpio/aspeed_gpio.c | 54 +++++++++++++++++++++++++++++++------------
 hw/gpio/trace-events  |  4 ++++
 2 files changed, 43 insertions(+), 15 deletions(-)

diff --git a/hw/gpio/aspeed_gpio.c b/hw/gpio/aspeed_gpio.c
index 9b736e7a9f..4620ea8e8b 100644
--- a/hw/gpio/aspeed_gpio.c
+++ b/hw/gpio/aspeed_gpio.c
@@ -15,6 +15,7 @@
 #include "qapi/visitor.h"
 #include "hw/irq.h"
 #include "migration/vmstate.h"
+#include "trace.h"
 
 #define GPIOS_PER_GROUP 8
 
@@ -523,11 +524,15 @@ static uint64_t aspeed_gpio_read(void *opaque, hwaddr offset, uint32_t size)
     uint64_t idx = -1;
     const AspeedGPIOReg *reg;
     GPIOSets *set;
+    uint32_t value = 0;
+    uint64_t debounce_value;
 
     idx = offset >> 2;
     if (idx >= GPIO_DEBOUNCE_TIME_1 && idx <= GPIO_DEBOUNCE_TIME_3) {
         idx -= GPIO_DEBOUNCE_TIME_1;
-        return (uint64_t) s->debounce_regs[idx];
+        debounce_value = (uint64_t) s->debounce_regs[idx];
+        trace_aspeed_gpio_read(offset, debounce_value);
+        return debounce_value;
     }
 
     reg = &agc->reg_table[idx];
@@ -540,38 +545,55 @@ static uint64_t aspeed_gpio_read(void *opaque, hwaddr offset, uint32_t size)
     set = &s->sets[reg->set_idx];
     switch (reg->type) {
     case gpio_reg_data_value:
-        return set->data_value;
+        value = set->data_value;
+        break;
     case gpio_reg_direction:
-        return set->direction;
+        value = set->direction;
+        break;
     case gpio_reg_int_enable:
-        return set->int_enable;
+        value = set->int_enable;
+        break;
     case gpio_reg_int_sens_0:
-        return set->int_sens_0;
+        value = set->int_sens_0;
+        break;
     case gpio_reg_int_sens_1:
-        return set->int_sens_1;
+        value = set->int_sens_1;
+        break;
     case gpio_reg_int_sens_2:
-        return set->int_sens_2;
+        value = set->int_sens_2;
+        break;
     case gpio_reg_int_status:
-        return set->int_status;
+        value = set->int_status;
+        break;
     case gpio_reg_reset_tolerant:
-        return set->reset_tol;
+        value = set->reset_tol;
+        break;
     case gpio_reg_debounce_1:
-        return set->debounce_1;
+        value = set->debounce_1;
+        break;
     case gpio_reg_debounce_2:
-        return set->debounce_2;
+        value = set->debounce_2;
+        break;
     case gpio_reg_cmd_source_0:
-        return set->cmd_source_0;
+        value = set->cmd_source_0;
+        break;
     case gpio_reg_cmd_source_1:
-        return set->cmd_source_1;
+        value = set->cmd_source_1;
+        break;
     case gpio_reg_data_read:
-        return set->data_read;
+        value = set->data_read;
+        break;
     case gpio_reg_input_mask:
-        return set->input_mask;
+        value = set->input_mask;
+        break;
     default:
         qemu_log_mask(LOG_GUEST_ERROR, "%s: no getter for offset 0x%"
                       HWADDR_PRIx"\n", __func__, offset);
         return 0;
     }
+
+    trace_aspeed_gpio_read(offset, value);
+    return value;
 }
 
 static void aspeed_gpio_write(void *opaque, hwaddr offset, uint64_t data,
@@ -585,6 +607,8 @@ static void aspeed_gpio_write(void *opaque, hwaddr offset, uint64_t data,
     GPIOSets *set;
     uint32_t cleared;
 
+    trace_aspeed_gpio_write(offset, data);
+
     idx = offset >> 2;
     if (idx >= GPIO_DEBOUNCE_TIME_1 && idx <= GPIO_DEBOUNCE_TIME_3) {
         idx -= GPIO_DEBOUNCE_TIME_1;
diff --git a/hw/gpio/trace-events b/hw/gpio/trace-events
index 1dab99c560..9736b362ac 100644
--- a/hw/gpio/trace-events
+++ b/hw/gpio/trace-events
@@ -27,3 +27,7 @@ sifive_gpio_read(uint64_t offset, uint64_t r) "offset 0x%" PRIx64 " value 0x%" P
 sifive_gpio_write(uint64_t offset, uint64_t value) "offset 0x%" PRIx64 " value 0x%" PRIx64
 sifive_gpio_set(int64_t line, int64_t value) "line %" PRIi64 " value %" PRIi64
 sifive_gpio_update_output_irq(int64_t line, int64_t value) "line %" PRIi64 " value %" PRIi64
+
+# aspeed_gpio.c
+aspeed_gpio_read(uint64_t offset, uint64_t value) "offset: 0x%" PRIx64 " value 0x%" PRIx64
+aspeed_gpio_write(uint64_t offset, uint64_t value) "offset: 0x%" PRIx64 " value 0x%" PRIx64

From 17075ef244d4ca52f7f097927c72b0e09f8d8a5c Mon Sep 17 00:00:00 2001
From: Jamin Lin <jamin_lin@aspeedtech.com>
Date: Wed, 25 May 2022 10:31:33 +0200
Subject: [PATCH 498/935] hw/gpio: Add ASPEED GPIO model for AST1030
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

AST1030 integrates one set of Parallel GPIO Controller
with maximum 151 control pins, which are 21 groups
(A~U, exclude pin: M6 M7 Q5 Q6 Q7 R0 R1 R4 R5 R6 R7 S0 S3 S4
S5 S6 S7 ) and the group T and U are input only.

Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220525053444.27228-3-jamin_lin@aspeedtech.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
 hw/arm/aspeed_ast10x0.c | 11 +++++++++++
 hw/gpio/aspeed_gpio.c   | 27 +++++++++++++++++++++++++++
 2 files changed, 38 insertions(+)

diff --git a/hw/arm/aspeed_ast10x0.c b/hw/arm/aspeed_ast10x0.c
index 938615d55f..d534541684 100644
--- a/hw/arm/aspeed_ast10x0.c
+++ b/hw/arm/aspeed_ast10x0.c
@@ -136,6 +136,9 @@ static void aspeed_soc_ast1030_init(Object *obj)
         snprintf(typename, sizeof(typename), "aspeed.wdt-%s", socname);
         object_initialize_child(obj, "wdt[*]", &s->wdt[i], typename);
     }
+
+    snprintf(typename, sizeof(typename), "aspeed.gpio-%s", socname);
+    object_initialize_child(obj, "gpio", &s->gpio, typename);
 }
 
 static void aspeed_soc_ast1030_realize(DeviceState *dev_soc, Error **errp)
@@ -281,6 +284,14 @@ static void aspeed_soc_ast1030_realize(DeviceState *dev_soc, Error **errp)
         sysbus_mmio_map(SYS_BUS_DEVICE(&s->wdt[i]), 0,
                         sc->memmap[ASPEED_DEV_WDT] + i * awc->offset);
     }
+
+    /* GPIO */
+    if (!sysbus_realize(SYS_BUS_DEVICE(&s->gpio), errp)) {
+        return;
+    }
+    sysbus_mmio_map(SYS_BUS_DEVICE(&s->gpio), 0, sc->memmap[ASPEED_DEV_GPIO]);
+    sysbus_connect_irq(SYS_BUS_DEVICE(&s->gpio), 0,
+                       aspeed_soc_get_irq(s, ASPEED_DEV_GPIO));
 }
 
 static void aspeed_soc_ast1030_class_init(ObjectClass *klass, void *data)
diff --git a/hw/gpio/aspeed_gpio.c b/hw/gpio/aspeed_gpio.c
index 4620ea8e8b..5138fe812b 100644
--- a/hw/gpio/aspeed_gpio.c
+++ b/hw/gpio/aspeed_gpio.c
@@ -819,6 +819,15 @@ static GPIOSetProperties ast2600_1_8v_set_props[ASPEED_GPIO_MAX_NR_SETS] = {
     [1] = {0x0000000f,  0x0000000f,  {"18E"} },
 };
 
+static GPIOSetProperties ast1030_set_props[ASPEED_GPIO_MAX_NR_SETS] = {
+    [0] = {0xffffffff,  0xffffffff,  {"A", "B", "C", "D"} },
+    [1] = {0xffffffff,  0xffffffff,  {"E", "F", "G", "H"} },
+    [2] = {0xffffffff,  0xffffffff,  {"I", "J", "K", "L"} },
+    [3] = {0xffffff3f,  0xffffff3f,  {"M", "N", "O", "P"} },
+    [4] = {0xff060c1f,  0x00060c1f,  {"Q", "R", "S", "T"} },
+    [5] = {0x000000ff,  0x00000000,  {"U"} },
+};
+
 static const MemoryRegionOps aspeed_gpio_ops = {
     .read       = aspeed_gpio_read,
     .write      = aspeed_gpio_write,
@@ -971,6 +980,16 @@ static void aspeed_gpio_ast2600_1_8v_class_init(ObjectClass *klass, void *data)
     agc->reg_table = aspeed_1_8v_gpios;
 }
 
+static void aspeed_gpio_1030_class_init(ObjectClass *klass, void *data)
+{
+    AspeedGPIOClass *agc = ASPEED_GPIO_CLASS(klass);
+
+    agc->props = ast1030_set_props;
+    agc->nr_gpio_pins = 151;
+    agc->nr_gpio_sets = 6;
+    agc->reg_table = aspeed_3_3v_gpios;
+}
+
 static const TypeInfo aspeed_gpio_info = {
     .name           = TYPE_ASPEED_GPIO,
     .parent         = TYPE_SYS_BUS_DEVICE,
@@ -1008,6 +1027,13 @@ static const TypeInfo aspeed_gpio_ast2600_1_8v_info = {
     .instance_init  = aspeed_gpio_init,
 };
 
+static const TypeInfo aspeed_gpio_ast1030_info = {
+    .name           = TYPE_ASPEED_GPIO "-ast1030",
+    .parent         = TYPE_ASPEED_GPIO,
+    .class_init     = aspeed_gpio_1030_class_init,
+    .instance_init  = aspeed_gpio_init,
+};
+
 static void aspeed_gpio_register_types(void)
 {
     type_register_static(&aspeed_gpio_info);
@@ -1015,6 +1041,7 @@ static void aspeed_gpio_register_types(void)
     type_register_static(&aspeed_gpio_ast2500_info);
     type_register_static(&aspeed_gpio_ast2600_3_3v_info);
     type_register_static(&aspeed_gpio_ast2600_1_8v_info);
+    type_register_static(&aspeed_gpio_ast1030_info);
 }
 
 type_init(aspeed_gpio_register_types);

From 247c00294a4b3cc694f24811eef07e57eb67aa82 Mon Sep 17 00:00:00 2001
From: Jamin Lin <jamin_lin@aspeedtech.com>
Date: Wed, 25 May 2022 10:31:33 +0200
Subject: [PATCH 499/935] hw/gpio support GPIO index mode for write operation.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

It did not support GPIO index mode for read operation.

Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220525053444.27228-4-jamin_lin@aspeedtech.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
 hw/gpio/aspeed_gpio.c         | 168 ++++++++++++++++++++++++++++++++++
 include/hw/gpio/aspeed_gpio.h |  14 +++
 2 files changed, 182 insertions(+)

diff --git a/hw/gpio/aspeed_gpio.c b/hw/gpio/aspeed_gpio.c
index 5138fe812b..c834bf19f5 100644
--- a/hw/gpio/aspeed_gpio.c
+++ b/hw/gpio/aspeed_gpio.c
@@ -16,6 +16,7 @@
 #include "hw/irq.h"
 #include "migration/vmstate.h"
 #include "trace.h"
+#include "hw/registerfields.h"
 
 #define GPIOS_PER_GROUP 8
 
@@ -204,6 +205,28 @@
 #define GPIO_1_8V_MEM_SIZE            0x1D8
 #define GPIO_1_8V_REG_ARRAY_SIZE      (GPIO_1_8V_MEM_SIZE >> 2)
 
+/*
+ * GPIO index mode support
+ * It only supports write operation
+ */
+REG32(GPIO_INDEX_REG, 0x2AC)
+    FIELD(GPIO_INDEX_REG, NUMBER, 0, 8)
+    FIELD(GPIO_INDEX_REG, COMMAND, 12, 1)
+    FIELD(GPIO_INDEX_REG, TYPE, 16, 4)
+    FIELD(GPIO_INDEX_REG, DATA_VALUE, 20, 1)
+    FIELD(GPIO_INDEX_REG, DIRECTION, 20, 1)
+    FIELD(GPIO_INDEX_REG, INT_ENABLE, 20, 1)
+    FIELD(GPIO_INDEX_REG, INT_SENS_0, 21, 1)
+    FIELD(GPIO_INDEX_REG, INT_SENS_1, 22, 1)
+    FIELD(GPIO_INDEX_REG, INT_SENS_2, 23, 1)
+    FIELD(GPIO_INDEX_REG, INT_STATUS, 24, 1)
+    FIELD(GPIO_INDEX_REG, DEBOUNCE_1, 20, 1)
+    FIELD(GPIO_INDEX_REG, DEBOUNCE_2, 21, 1)
+    FIELD(GPIO_INDEX_REG, RESET_TOLERANT, 20, 1)
+    FIELD(GPIO_INDEX_REG, COMMAND_SRC_0, 20, 1)
+    FIELD(GPIO_INDEX_REG, COMMAND_SRC_1, 21, 1)
+    FIELD(GPIO_INDEX_REG, INPUT_MASK, 20, 1)
+
 static int aspeed_evaluate_irq(GPIOSets *regs, int gpio_prev_high, int gpio)
 {
     uint32_t falling_edge = 0, rising_edge = 0;
@@ -596,6 +619,144 @@ static uint64_t aspeed_gpio_read(void *opaque, hwaddr offset, uint32_t size)
     return value;
 }
 
+static void aspeed_gpio_write_index_mode(void *opaque, hwaddr offset,
+                                                uint64_t data, uint32_t size)
+{
+
+    AspeedGPIOState *s = ASPEED_GPIO(opaque);
+    AspeedGPIOClass *agc = ASPEED_GPIO_GET_CLASS(s);
+    const GPIOSetProperties *props;
+    GPIOSets *set;
+    uint32_t reg_idx_number = FIELD_EX32(data, GPIO_INDEX_REG, NUMBER);
+    uint32_t reg_idx_type = FIELD_EX32(data, GPIO_INDEX_REG, TYPE);
+    uint32_t reg_idx_command = FIELD_EX32(data, GPIO_INDEX_REG, COMMAND);
+    uint32_t set_idx = reg_idx_number / ASPEED_GPIOS_PER_SET;
+    uint32_t pin_idx = reg_idx_number % ASPEED_GPIOS_PER_SET;
+    uint32_t group_idx = pin_idx / GPIOS_PER_GROUP;
+    uint32_t reg_value = 0;
+    uint32_t cleared;
+
+    set = &s->sets[set_idx];
+    props = &agc->props[set_idx];
+
+    if (reg_idx_command)
+        qemu_log_mask(LOG_GUEST_ERROR, "%s: offset 0x%" PRIx64 "data 0x%"
+            PRIx64 "index mode wrong command 0x%x\n",
+            __func__, offset, data, reg_idx_command);
+
+    switch (reg_idx_type) {
+    case gpio_reg_idx_data:
+        reg_value = set->data_read;
+        reg_value = deposit32(reg_value, pin_idx, 1,
+                              FIELD_EX32(data, GPIO_INDEX_REG, DATA_VALUE));
+        reg_value &= props->output;
+        reg_value = update_value_control_source(set, set->data_value,
+                                                reg_value);
+        set->data_read = reg_value;
+        aspeed_gpio_update(s, set, reg_value);
+        return;
+    case gpio_reg_idx_direction:
+        reg_value = set->direction;
+        reg_value = deposit32(reg_value, pin_idx, 1,
+                              FIELD_EX32(data, GPIO_INDEX_REG, DIRECTION));
+        /*
+         *   where data is the value attempted to be written to the pin:
+         *    pin type      | input mask | output mask | expected value
+         *    ------------------------------------------------------------
+         *   bidirectional  |   1       |   1        |  data
+         *   input only     |   1       |   0        |   0
+         *   output only    |   0       |   1        |   1
+         *   no pin         |   0       |   0        |   0
+         *
+         *  which is captured by:
+         *  data = ( data | ~input) & output;
+         */
+        reg_value = (reg_value | ~props->input) & props->output;
+        set->direction = update_value_control_source(set, set->direction,
+                                                     reg_value);
+        break;
+    case gpio_reg_idx_interrupt:
+        reg_value = set->int_enable;
+        reg_value = deposit32(reg_value, pin_idx, 1,
+                              FIELD_EX32(data, GPIO_INDEX_REG, INT_ENABLE));
+        set->int_enable = update_value_control_source(set, set->int_enable,
+                                                      reg_value);
+        reg_value = set->int_sens_0;
+        reg_value = deposit32(reg_value, pin_idx, 1,
+                              FIELD_EX32(data, GPIO_INDEX_REG, INT_SENS_0));
+        set->int_sens_0 = update_value_control_source(set, set->int_sens_0,
+                                                      reg_value);
+        reg_value = set->int_sens_1;
+        reg_value = deposit32(reg_value, pin_idx, 1,
+                              FIELD_EX32(data, GPIO_INDEX_REG, INT_SENS_1));
+        set->int_sens_1 = update_value_control_source(set, set->int_sens_1,
+                                                      reg_value);
+        reg_value = set->int_sens_2;
+        reg_value = deposit32(reg_value, pin_idx, 1,
+                              FIELD_EX32(data, GPIO_INDEX_REG, INT_SENS_2));
+        set->int_sens_2 = update_value_control_source(set, set->int_sens_2,
+                                                      reg_value);
+        /* set interrupt status */
+        reg_value = set->int_status;
+        reg_value = deposit32(reg_value, pin_idx, 1,
+                              FIELD_EX32(data, GPIO_INDEX_REG, INT_STATUS));
+        cleared = ctpop32(reg_value & set->int_status);
+        if (s->pending && cleared) {
+            assert(s->pending >= cleared);
+            s->pending -= cleared;
+        }
+        set->int_status &= ~reg_value;
+        break;
+    case gpio_reg_idx_debounce:
+        reg_value = set->debounce_1;
+        reg_value = deposit32(reg_value, pin_idx, 1,
+                              FIELD_EX32(data, GPIO_INDEX_REG, DEBOUNCE_1));
+        set->debounce_1 = update_value_control_source(set, set->debounce_1,
+                                                      reg_value);
+        reg_value = set->debounce_2;
+        reg_value = deposit32(reg_value, pin_idx, 1,
+                              FIELD_EX32(data, GPIO_INDEX_REG, DEBOUNCE_2));
+        set->debounce_2 = update_value_control_source(set, set->debounce_2,
+                                                      reg_value);
+        return;
+    case gpio_reg_idx_tolerance:
+        reg_value = set->reset_tol;
+        reg_value = deposit32(reg_value, pin_idx, 1,
+                              FIELD_EX32(data, GPIO_INDEX_REG, RESET_TOLERANT));
+        set->reset_tol = update_value_control_source(set, set->reset_tol,
+                                                     reg_value);
+        return;
+    case gpio_reg_idx_cmd_src:
+        reg_value = set->cmd_source_0;
+        reg_value = deposit32(reg_value, GPIOS_PER_GROUP * group_idx, 1,
+                              FIELD_EX32(data, GPIO_INDEX_REG, COMMAND_SRC_0));
+        set->cmd_source_0 = reg_value & ASPEED_CMD_SRC_MASK;
+        reg_value = set->cmd_source_1;
+        reg_value = deposit32(reg_value, GPIOS_PER_GROUP * group_idx, 1,
+                              FIELD_EX32(data, GPIO_INDEX_REG, COMMAND_SRC_1));
+        set->cmd_source_1 = reg_value & ASPEED_CMD_SRC_MASK;
+        return;
+    case gpio_reg_idx_input_mask:
+        reg_value = set->input_mask;
+        reg_value = deposit32(reg_value, pin_idx, 1,
+                              FIELD_EX32(data, GPIO_INDEX_REG, INPUT_MASK));
+        /*
+         * feeds into interrupt generation
+         * 0: read from data value reg will be updated
+         * 1: read from data value reg will not be updated
+         */
+        set->input_mask = reg_value & props->input;
+        break;
+    default:
+        qemu_log_mask(LOG_GUEST_ERROR, "%s: offset 0x%" PRIx64 "data 0x%"
+            PRIx64 "index mode wrong type 0x%x\n",
+            __func__, offset, data, reg_idx_type);
+        return;
+    }
+    aspeed_gpio_update(s, set, set->data_value);
+    return;
+}
+
 static void aspeed_gpio_write(void *opaque, hwaddr offset, uint64_t data,
                               uint32_t size)
 {
@@ -610,6 +771,13 @@ static void aspeed_gpio_write(void *opaque, hwaddr offset, uint64_t data,
     trace_aspeed_gpio_write(offset, data);
 
     idx = offset >> 2;
+
+    /* check gpio index mode */
+    if (idx == R_GPIO_INDEX_REG) {
+        aspeed_gpio_write_index_mode(opaque, offset, data, size);
+        return;
+    }
+
     if (idx >= GPIO_DEBOUNCE_TIME_1 && idx <= GPIO_DEBOUNCE_TIME_3) {
         idx -= GPIO_DEBOUNCE_TIME_1;
         s->debounce_regs[idx] = (uint32_t) data;
diff --git a/include/hw/gpio/aspeed_gpio.h b/include/hw/gpio/aspeed_gpio.h
index 6dee3cd438..41b36524d0 100644
--- a/include/hw/gpio/aspeed_gpio.h
+++ b/include/hw/gpio/aspeed_gpio.h
@@ -50,6 +50,20 @@ enum GPIORegType {
     gpio_reg_input_mask,
 };
 
+/* GPIO index mode */
+enum GPIORegIndexType {
+    gpio_reg_idx_data = 0,
+    gpio_reg_idx_direction,
+    gpio_reg_idx_interrupt,
+    gpio_reg_idx_debounce,
+    gpio_reg_idx_tolerance,
+    gpio_reg_idx_cmd_src,
+    gpio_reg_idx_input_mask,
+    gpio_reg_idx_reserved,
+    gpio_reg_idx_new_w_cmd_src,
+    gpio_reg_idx_new_r_cmd_src,
+};
+
 typedef struct AspeedGPIOReg {
     uint16_t set_idx;
     enum GPIORegType type;

From 554c294514dc7445c5442266a50012ed774d63fe Mon Sep 17 00:00:00 2001
From: Jamin Lin <jamin_lin@aspeedtech.com>
Date: Wed, 25 May 2022 10:31:33 +0200
Subject: [PATCH 500/935] hw/gpio: replace HWADDR_PRIx with PRIx64
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

1. replace HWADDR_PRIx with PRIx64
2. fix indent issue

Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220525053444.27228-5-jamin_lin@aspeedtech.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
 hw/gpio/aspeed_gpio.c         | 8 ++++----
 include/hw/gpio/aspeed_gpio.h | 2 +-
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/hw/gpio/aspeed_gpio.c b/hw/gpio/aspeed_gpio.c
index c834bf19f5..a62a673857 100644
--- a/hw/gpio/aspeed_gpio.c
+++ b/hw/gpio/aspeed_gpio.c
@@ -561,7 +561,7 @@ static uint64_t aspeed_gpio_read(void *opaque, hwaddr offset, uint32_t size)
     reg = &agc->reg_table[idx];
     if (reg->set_idx >= agc->nr_gpio_sets) {
         qemu_log_mask(LOG_GUEST_ERROR, "%s: no getter for offset 0x%"
-                      HWADDR_PRIx"\n", __func__, offset);
+                      PRIx64"\n", __func__, offset);
         return 0;
     }
 
@@ -611,7 +611,7 @@ static uint64_t aspeed_gpio_read(void *opaque, hwaddr offset, uint32_t size)
         break;
     default:
         qemu_log_mask(LOG_GUEST_ERROR, "%s: no getter for offset 0x%"
-                      HWADDR_PRIx"\n", __func__, offset);
+                      PRIx64"\n", __func__, offset);
         return 0;
     }
 
@@ -787,7 +787,7 @@ static void aspeed_gpio_write(void *opaque, hwaddr offset, uint64_t data,
     reg = &agc->reg_table[idx];
     if (reg->set_idx >= agc->nr_gpio_sets) {
         qemu_log_mask(LOG_GUEST_ERROR, "%s: no setter for offset 0x%"
-                      HWADDR_PRIx"\n", __func__, offset);
+                      PRIx64"\n", __func__, offset);
         return;
     }
 
@@ -872,7 +872,7 @@ static void aspeed_gpio_write(void *opaque, hwaddr offset, uint64_t data,
         break;
     default:
         qemu_log_mask(LOG_GUEST_ERROR, "%s: no setter for offset 0x%"
-                      HWADDR_PRIx"\n", __func__, offset);
+                      PRIx64"\n", __func__, offset);
         return;
     }
     aspeed_gpio_update(s, set, set->data_value);
diff --git a/include/hw/gpio/aspeed_gpio.h b/include/hw/gpio/aspeed_gpio.h
index 41b36524d0..904eecf62c 100644
--- a/include/hw/gpio/aspeed_gpio.h
+++ b/include/hw/gpio/aspeed_gpio.h
@@ -67,7 +67,7 @@ enum GPIORegIndexType {
 typedef struct AspeedGPIOReg {
     uint16_t set_idx;
     enum GPIORegType type;
- } AspeedGPIOReg;
+} AspeedGPIOReg;
 
 struct AspeedGPIOClass {
     SysBusDevice parent_obj;

From 3569664ee9f08c94a5a2c69f778f0d023e5e882f Mon Sep 17 00:00:00 2001
From: luzhipeng <luzhipeng@cestc.cn>
Date: Fri, 20 May 2022 10:19:35 +0800
Subject: [PATCH 501/935] qga: add guest-get-diskstats command for Linux guests
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add a new 'guest-get-diskstats' command for report disk io statistics
for Linux guests. This can be useful for getting io flow or handling
IO fault, no need to enter guests.

Signed-off-by: luzhipeng <luzhipeng@cestc.cn>
Message-Id: <20220520021935.676-1-luzhipeng@cestc.cn>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Konstantin Kostiuk <kkostiuk@redhat.com>
Signed-off-by: Konstantin Kostiuk <kkostiuk@redhat.com>
---
 qga/commands-posix.c | 123 +++++++++++++++++++++++++++++++++++++++++++
 qga/commands-win32.c |   6 +++
 qga/qapi-schema.json |  86 ++++++++++++++++++++++++++++++
 3 files changed, 215 insertions(+)

diff --git a/qga/commands-posix.c b/qga/commands-posix.c
index 69f209af87..12b50b7124 100644
--- a/qga/commands-posix.c
+++ b/qga/commands-posix.c
@@ -2783,6 +2783,122 @@ GuestMemoryBlockInfo *qmp_guest_get_memory_block_info(Error **errp)
     return info;
 }
 
+#define MAX_NAME_LEN 128
+static GuestDiskStatsInfoList *guest_get_diskstats(Error **errp)
+{
+#ifdef CONFIG_LINUX
+    GuestDiskStatsInfoList *head = NULL, **tail = &head;
+    const char *diskstats = "/proc/diskstats";
+    FILE *fp;
+    size_t n;
+    char *line = NULL;
+
+    fp = fopen(diskstats, "r");
+    if (fp  == NULL) {
+        error_setg_errno(errp, errno, "open(\"%s\")", diskstats);
+        return NULL;
+    }
+
+    while (getline(&line, &n, fp) != -1) {
+        g_autofree GuestDiskStatsInfo *diskstatinfo = NULL;
+        g_autofree GuestDiskStats *diskstat = NULL;
+        char dev_name[MAX_NAME_LEN];
+        unsigned int ios_pgr, tot_ticks, rq_ticks, wr_ticks, dc_ticks, fl_ticks;
+        unsigned long rd_ios, rd_merges_or_rd_sec, rd_ticks_or_wr_sec, wr_ios;
+        unsigned long wr_merges, rd_sec_or_wr_ios, wr_sec;
+        unsigned long dc_ios, dc_merges, dc_sec, fl_ios;
+        unsigned int major, minor;
+        int i;
+
+        i = sscanf(line, "%u %u %s %lu %lu %lu"
+                   "%lu %lu %lu %lu %u %u %u %u"
+                   "%lu %lu %lu %u %lu %u",
+                   &major, &minor, dev_name,
+                   &rd_ios, &rd_merges_or_rd_sec, &rd_sec_or_wr_ios,
+                   &rd_ticks_or_wr_sec, &wr_ios, &wr_merges, &wr_sec,
+                   &wr_ticks, &ios_pgr, &tot_ticks, &rq_ticks,
+                   &dc_ios, &dc_merges, &dc_sec, &dc_ticks,
+                   &fl_ios, &fl_ticks);
+
+        if (i < 7) {
+            continue;
+        }
+
+        diskstatinfo = g_new0(GuestDiskStatsInfo, 1);
+        diskstatinfo->name = g_strdup(dev_name);
+        diskstatinfo->major = major;
+        diskstatinfo->minor = minor;
+
+        diskstat = g_new0(GuestDiskStats, 1);
+        if (i == 7) {
+            diskstat->has_read_ios = true;
+            diskstat->read_ios = rd_ios;
+            diskstat->has_read_sectors = true;
+            diskstat->read_sectors = rd_merges_or_rd_sec;
+            diskstat->has_write_ios = true;
+            diskstat->write_ios = rd_sec_or_wr_ios;
+            diskstat->has_write_sectors = true;
+            diskstat->write_sectors = rd_ticks_or_wr_sec;
+        }
+        if (i >= 14) {
+            diskstat->has_read_ios = true;
+            diskstat->read_ios = rd_ios;
+            diskstat->has_read_sectors = true;
+            diskstat->read_sectors = rd_sec_or_wr_ios;
+            diskstat->has_read_merges = true;
+            diskstat->read_merges = rd_merges_or_rd_sec;
+            diskstat->has_read_ticks = true;
+            diskstat->read_ticks = rd_ticks_or_wr_sec;
+            diskstat->has_write_ios = true;
+            diskstat->write_ios = wr_ios;
+            diskstat->has_write_sectors = true;
+            diskstat->write_sectors = wr_sec;
+            diskstat->has_write_merges = true;
+            diskstat->write_merges = wr_merges;
+            diskstat->has_write_ticks = true;
+            diskstat->write_ticks = wr_ticks;
+            diskstat->has_ios_pgr = true;
+            diskstat->ios_pgr = ios_pgr;
+            diskstat->has_total_ticks = true;
+            diskstat->total_ticks = tot_ticks;
+            diskstat->has_weight_ticks = true;
+            diskstat->weight_ticks = rq_ticks;
+        }
+        if (i >= 18) {
+            diskstat->has_discard_ios = true;
+            diskstat->discard_ios = dc_ios;
+            diskstat->has_discard_merges = true;
+            diskstat->discard_merges = dc_merges;
+            diskstat->has_discard_sectors = true;
+            diskstat->discard_sectors = dc_sec;
+            diskstat->has_discard_ticks = true;
+            diskstat->discard_ticks = dc_ticks;
+        }
+        if (i >= 20) {
+            diskstat->has_flush_ios = true;
+            diskstat->flush_ios = fl_ios;
+            diskstat->has_flush_ticks = true;
+            diskstat->flush_ticks = fl_ticks;
+        }
+
+        diskstatinfo->stats = g_steal_pointer(&diskstat);
+        QAPI_LIST_APPEND(tail, diskstatinfo);
+        diskstatinfo = NULL;
+    }
+    free(line);
+    fclose(fp);
+    return head;
+#else
+    g_debug("disk stats reporting available only for Linux");
+    return NULL;
+#endif
+}
+
+GuestDiskStatsInfoList *qmp_guest_get_diskstats(Error **errp)
+{
+    return guest_get_diskstats(errp);
+}
+
 #else /* defined(__linux__) */
 
 void qmp_guest_suspend_disk(Error **errp)
@@ -3131,6 +3247,13 @@ GuestDiskInfoList *qmp_guest_get_disks(Error **errp)
     return NULL;
 }
 
+GuestDiskStatsInfoList *qmp_guest_get_diskstats(Error **errp)
+{
+    error_setg(errp, QERR_UNSUPPORTED);
+    return NULL;
+}
+
+
 #endif /* CONFIG_FSFREEZE */
 
 #if !defined(CONFIG_FSTRIM)
diff --git a/qga/commands-win32.c b/qga/commands-win32.c
index d56b5fd2a7..dcdeb76a68 100644
--- a/qga/commands-win32.c
+++ b/qga/commands-win32.c
@@ -2532,3 +2532,9 @@ char *qga_get_host_name(Error **errp)
 
     return g_utf16_to_utf8(tmp, size, NULL, NULL, NULL);
 }
+
+GuestDiskStatsInfoList *qmp_guest_get_diskstats(Error **errp)
+{
+    error_setg(errp, QERR_UNSUPPORTED);
+    return NULL;
+}
diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json
index f989597b0c..9fa20e791b 100644
--- a/qga/qapi-schema.json
+++ b/qga/qapi-schema.json
@@ -1490,3 +1490,89 @@
 { 'command': 'guest-ssh-remove-authorized-keys',
   'data': { 'username': 'str', 'keys': ['str'] },
   'if': 'CONFIG_POSIX' }
+
+##
+# @GuestDiskStats:
+#
+# @read-sectors: sectors read
+#
+# @read-ios: reads completed successfully
+#
+# @read-merges: read requests merged
+#
+# @write-sectors: sectors written
+#
+# @write-ios: writes completed
+#
+# @write-merges: write requests merged
+#
+# @discard-sectors: sectors discarded
+#
+# @discard-ios: discards completed successfully
+#
+# @discard-merges: discard requests merged
+#
+# @flush-ios: flush requests completed successfully
+#
+# @read-ticks: time spent reading(ms)
+#
+# @write-ticks: time spent writing(ms)
+#
+# @discard-ticks: time spent discarding(ms)
+#
+# @flush-ticks: time spent flushing(ms)
+#
+# @ios-pgr: number of I/Os currently in flight
+#
+# @total-ticks: time spent doing I/Os (ms)
+#
+# @weight-ticks: weighted time spent doing I/Os since the last update of this field(ms)
+#
+# Since: 7.1
+##
+{ 'struct': 'GuestDiskStats',
+  'data': {'*read-sectors': 'uint64',
+           '*read-ios': 'uint64',
+           '*read-merges': 'uint64',
+           '*write-sectors': 'uint64',
+           '*write-ios': 'uint64',
+           '*write-merges': 'uint64',
+           '*discard-sectors': 'uint64',
+           '*discard-ios': 'uint64',
+           '*discard-merges': 'uint64',
+           '*flush-ios': 'uint64',
+           '*read-ticks': 'uint64',
+           '*write-ticks': 'uint64',
+           '*discard-ticks': 'uint64',
+           '*flush-ticks': 'uint64',
+           '*ios-pgr': 'uint64',
+           '*total-ticks': 'uint64',
+           '*weight-ticks': 'uint64'
+           } }
+
+##
+# @GuestDiskStatsInfo:
+#
+# @name disk name
+#
+# @major major device number of disk
+#
+# @minor minor device number of disk
+##
+{ 'struct': 'GuestDiskStatsInfo',
+  'data': {'name': 'str',
+           'major': 'uint64',
+           'minor': 'uint64',
+           'stats': 'GuestDiskStats' } }
+
+##
+# @guest-get-diskstats:
+#
+# Retrieve information about disk stats.
+# Returns: List of disk stats of guest.
+#
+# Since: 7.1
+##
+{ 'command': 'guest-get-diskstats',
+  'returns': ['GuestDiskStatsInfo']
+}

From 323f3a8f22ee4eecf427a9322bd0fb449c33abd7 Mon Sep 17 00:00:00 2001
From: Konstantin Kostiuk <kkostiuk@redhat.com>
Date: Mon, 23 May 2022 22:16:44 +0300
Subject: [PATCH 502/935] trivial: qga: Log version on start
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Konstantin Kostiuk <kkostiuk@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20220523191644.823726-2-kkostiuk@redhat.com>
Signed-off-by: Konstantin Kostiuk <kkostiuk@redhat.com>
---
 qga/main.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/qga/main.c b/qga/main.c
index 3b9546c185..c373fec3ee 100644
--- a/qga/main.c
+++ b/qga/main.c
@@ -1271,6 +1271,8 @@ static GAState *initialize_agent(GAConfig *config, int socket_activation)
     g_log_set_fatal_mask(NULL, G_LOG_LEVEL_ERROR);
     ga_enable_logging(s);
 
+    g_debug("Guest agent version %s started", QEMU_FULL_VERSION);
+
 #ifdef _WIN32
     /* On win32 the state directory is application specific (be it the default
      * or a user override). We got past the command line parsing; let's create

From 2e7b2189582787ae4bedf698983b5a81f4ac4656 Mon Sep 17 00:00:00 2001
From: Konstantin Kostiuk <kkostiuk@redhat.com>
Date: Wed, 25 May 2022 11:59:53 +0300
Subject: [PATCH 503/935] tests: Bump Fedora image version for
 cross-compilation
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

There are 2 reason for the bump:
 - Fedora 33 is not supported anymore
 - Some changes in the guest agent required updates of
   mingw-headers

Signed-off-by: Konstantin Kostiuk <kkostiuk@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220525085953.940116-2-kkostiuk@redhat.com>
Signed-off-by: Konstantin Kostiuk <kkostiuk@redhat.com>
---
 tests/docker/dockerfiles/fedora-win32-cross.docker | 2 +-
 tests/docker/dockerfiles/fedora-win64-cross.docker | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/docker/dockerfiles/fedora-win32-cross.docker b/tests/docker/dockerfiles/fedora-win32-cross.docker
index 84a8f5524d..a06bd29e8e 100644
--- a/tests/docker/dockerfiles/fedora-win32-cross.docker
+++ b/tests/docker/dockerfiles/fedora-win32-cross.docker
@@ -1,4 +1,4 @@
-FROM registry.fedoraproject.org/fedora:33
+FROM registry.fedoraproject.org/fedora:35
 
 # Please keep this list sorted alphabetically
 ENV PACKAGES \
diff --git a/tests/docker/dockerfiles/fedora-win64-cross.docker b/tests/docker/dockerfiles/fedora-win64-cross.docker
index d7ed8eb1cf..b71624330f 100644
--- a/tests/docker/dockerfiles/fedora-win64-cross.docker
+++ b/tests/docker/dockerfiles/fedora-win64-cross.docker
@@ -1,4 +1,4 @@
-FROM registry.fedoraproject.org/fedora:33
+FROM registry.fedoraproject.org/fedora:35
 
 # Please keep this list sorted alphabetically
 ENV PACKAGES \

From b9a002609fd887447eca8ee10777690d691f91d4 Mon Sep 17 00:00:00 2001
From: Konstantin Kostiuk <kkostiuk@redhat.com>
Date: Tue, 24 May 2022 18:43:44 +0300
Subject: [PATCH 504/935] qga-win32: Add support for NVME bus type
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Bus type spaces (Indicates a storage spaces bus) is not
supported, so return it as unknown.

Signed-off-by: Konstantin Kostiuk <kkostiuk@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20220524154344.869638-2-kkostiuk@redhat.com>
Signed-off-by: Konstantin Kostiuk <kkostiuk@redhat.com>
---
 qga/commands-win32.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/qga/commands-win32.c b/qga/commands-win32.c
index dcdeb76a68..36f94c0f9c 100644
--- a/qga/commands-win32.c
+++ b/qga/commands-win32.c
@@ -490,6 +490,11 @@ static GuestDiskBusType win2qemu[] = {
 #if (_WIN32_WINNT >= 0x0601)
     [BusTypeVirtual] = GUEST_DISK_BUS_TYPE_VIRTUAL,
     [BusTypeFileBackedVirtual] = GUEST_DISK_BUS_TYPE_FILE_BACKED_VIRTUAL,
+    /*
+     * BusTypeSpaces currently is not suported
+     */
+    [BusTypeSpaces] = GUEST_DISK_BUS_TYPE_UNKNOWN,
+    [BusTypeNvme] = GUEST_DISK_BUS_TYPE_NVME,
 #endif
 };
 

From 29320530cf6684646b3a642fdbb5bc77ee8039de Mon Sep 17 00:00:00 2001
From: Liu Yiding <liuyd.fnst@fujitsu.com>
Date: Wed, 13 Apr 2022 12:20:54 +0800
Subject: [PATCH 505/935] docs: Correct the default thread-pool-size

Refer to 26ec190964 virtiofsd: Do not use a thread pool by default

Signed-off-by: Liu Yiding <liuyd.fnst@fujitsu.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Message-id: 20220413042054.1484640-1-liuyd.fnst@fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 docs/tools/virtiofsd.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/tools/virtiofsd.rst b/docs/tools/virtiofsd.rst
index e457b13d56..5f5ac9dd56 100644
--- a/docs/tools/virtiofsd.rst
+++ b/docs/tools/virtiofsd.rst
@@ -132,7 +132,7 @@ Options
 .. option:: --thread-pool-size=NUM
 
   Restrict the number of worker threads per request queue to NUM.  The default
-  is 64.
+  is 0.
 
 .. option:: --cache=none|auto|always
 

From 52bcd997800fab67d57bea6d93e368f6f7a93b24 Mon Sep 17 00:00:00 2001
From: Howard Chiu <howard_chiu@aspeedtech.com>
Date: Wed, 25 May 2022 10:00:01 +0000
Subject: [PATCH 506/935] hw/arm/aspeed: Add i2c devices for AST2600 EVB
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add EEPROM and LM75 temperature sensor according to hardware schematic

Signed-off-by: Howard Chiu <howard_chiu@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
 hw/arm/aspeed.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/hw/arm/aspeed.c b/hw/arm/aspeed.c
index 725c169488..98dc185acd 100644
--- a/hw/arm/aspeed.c
+++ b/hw/arm/aspeed.c
@@ -527,8 +527,15 @@ static void ast2500_evb_i2c_init(AspeedMachineState *bmc)
 
 static void ast2600_evb_i2c_init(AspeedMachineState *bmc)
 {
-    /* Start with some devices on our I2C busses */
-    ast2500_evb_i2c_init(bmc);
+    AspeedSoCState *soc = &bmc->soc;
+    uint8_t *eeprom_buf = g_malloc0(8 * 1024);
+
+    smbus_eeprom_init_one(aspeed_i2c_get_bus(&soc->i2c, 7), 0x50,
+                          eeprom_buf);
+
+    /* LM75 is compatible with TMP105 driver */
+    i2c_slave_create_simple(aspeed_i2c_get_bus(&soc->i2c, 8),
+                     TYPE_TMP105, 0x4d);
 }
 
 static void romulus_bmc_i2c_init(AspeedMachineState *bmc)

From 3c7b72ddca9ce85a9d1e8a98fd0996b74597b5ae Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Sat, 14 May 2022 08:50:10 +0200
Subject: [PATCH 507/935] thread-pool: optimize scheduling of completion bottom
 half

The completion bottom half was scheduled within the pool->lock
critical section.  That actually results in worse performance,
because the worker thread can run its own small critical section
and go to sleep before the bottom half starts running.

Note that this simple change does not produce an improvement without
changing the thread pool QemuSemaphore to a condition variable.

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Message-Id: <20220514065012.1149539-2-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 util/thread-pool.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/util/thread-pool.c b/util/thread-pool.c
index 196835b4d3..4979f30ca3 100644
--- a/util/thread-pool.c
+++ b/util/thread-pool.c
@@ -127,9 +127,8 @@ static void *worker_thread(void *opaque)
         smp_wmb();
         req->state = THREAD_DONE;
 
-        qemu_mutex_lock(&pool->lock);
-
         qemu_bh_schedule(pool->completion_bh);
+        qemu_mutex_lock(&pool->lock);
     }
 
     pool->cur_threads--;

From 900fa208f50623672a6f879374222a7fd4717791 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Sat, 14 May 2022 08:50:11 +0200
Subject: [PATCH 508/935] thread-pool: replace semaphore with condition
 variable
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Since commit f9fc8932b1 ("thread-posix: remove the posix semaphore
support", 2022-04-06) QemuSemaphore has its own mutex and condition
variable; this adds unnecessary overhead on I/O with small block sizes.

Check the QTAILQ directly instead of adding the indirection of a
semaphore's count.  Using a semaphore has not been necessary since
qemu_cond_timedwait was introduced; the new code has to be careful about
spurious wakeups but it is simpler, for example thread_pool_cancel does
not have to worry about synchronizing the semaphore count with the number
of elements of pool->request_list.

Note that the return value of qemu_cond_timedwait (0 for timeout, 1 for
signal or spurious wakeup) is different from that of qemu_sem_timedwait
(-1 for timeout, 0 for success).

Reported-by: Lukáš Doktor <ldoktor@redhat.com>
Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Message-Id: <20220514065012.1149539-3-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 util/thread-pool.c | 68 +++++++++++++++++++---------------------------
 1 file changed, 28 insertions(+), 40 deletions(-)

diff --git a/util/thread-pool.c b/util/thread-pool.c
index 4979f30ca3..6e3d4e4a2f 100644
--- a/util/thread-pool.c
+++ b/util/thread-pool.c
@@ -57,7 +57,7 @@ struct ThreadPool {
     QEMUBH *completion_bh;
     QemuMutex lock;
     QemuCond worker_stopped;
-    QemuSemaphore sem;
+    QemuCond request_cond;
     QEMUBH *new_thread_bh;
 
     /* The following variables are only accessed from one AioContext. */
@@ -74,23 +74,6 @@ struct ThreadPool {
     int max_threads;
 };
 
-static inline bool back_to_sleep(ThreadPool *pool, int ret)
-{
-    /*
-     * The semaphore timed out, we should exit the loop except when:
-     *  - There is work to do, we raced with the signal.
-     *  - The max threads threshold just changed, we raced with the signal.
-     *  - The thread pool forces a minimum number of readily available threads.
-     */
-    if (ret == -1 && (!QTAILQ_EMPTY(&pool->request_list) ||
-            pool->cur_threads > pool->max_threads ||
-            pool->cur_threads <= pool->min_threads)) {
-            return true;
-    }
-
-    return false;
-}
-
 static void *worker_thread(void *opaque)
 {
     ThreadPool *pool = opaque;
@@ -99,20 +82,25 @@ static void *worker_thread(void *opaque)
     pool->pending_threads--;
     do_spawn_thread(pool);
 
-    while (!pool->stopping) {
+    while (!pool->stopping && pool->cur_threads <= pool->max_threads) {
         ThreadPoolElement *req;
         int ret;
 
-        do {
+        if (QTAILQ_EMPTY(&pool->request_list)) {
             pool->idle_threads++;
-            qemu_mutex_unlock(&pool->lock);
-            ret = qemu_sem_timedwait(&pool->sem, 10000);
-            qemu_mutex_lock(&pool->lock);
+            ret = qemu_cond_timedwait(&pool->request_cond, &pool->lock, 10000);
             pool->idle_threads--;
-        } while (back_to_sleep(pool, ret));
-        if (ret == -1 || pool->stopping ||
-            pool->cur_threads > pool->max_threads) {
-            break;
+            if (ret == 0 &&
+                QTAILQ_EMPTY(&pool->request_list) &&
+                pool->cur_threads > pool->min_threads) {
+                /* Timed out + no work to do + no need for warm threads = exit.  */
+                break;
+            }
+            /*
+             * Even if there was some work to do, check if there aren't
+             * too many worker threads before picking it up.
+             */
+            continue;
         }
 
         req = QTAILQ_FIRST(&pool->request_list);
@@ -134,6 +122,12 @@ static void *worker_thread(void *opaque)
     pool->cur_threads--;
     qemu_cond_signal(&pool->worker_stopped);
     qemu_mutex_unlock(&pool->lock);
+
+    /*
+     * Wake up another thread, in case we got a wakeup but decided
+     * to exit due to pool->cur_threads > pool->max_threads.
+     */
+    qemu_cond_signal(&pool->request_cond);
     return NULL;
 }
 
@@ -229,13 +223,7 @@ static void thread_pool_cancel(BlockAIOCB *acb)
     trace_thread_pool_cancel(elem, elem->common.opaque);
 
     QEMU_LOCK_GUARD(&pool->lock);
-    if (elem->state == THREAD_QUEUED &&
-        /* No thread has yet started working on elem. we can try to "steal"
-         * the item from the worker if we can get a signal from the
-         * semaphore.  Because this is non-blocking, we can do it with
-         * the lock taken and ensure that elem will remain THREAD_QUEUED.
-         */
-        qemu_sem_timedwait(&pool->sem, 0) == 0) {
+    if (elem->state == THREAD_QUEUED) {
         QTAILQ_REMOVE(&pool->request_list, elem, reqs);
         qemu_bh_schedule(pool->completion_bh);
 
@@ -280,7 +268,7 @@ BlockAIOCB *thread_pool_submit_aio(ThreadPool *pool,
     }
     QTAILQ_INSERT_TAIL(&pool->request_list, req, reqs);
     qemu_mutex_unlock(&pool->lock);
-    qemu_sem_post(&pool->sem);
+    qemu_cond_signal(&pool->request_cond);
     return &req->common;
 }
 
@@ -323,7 +311,7 @@ void thread_pool_update_params(ThreadPool *pool, AioContext *ctx)
      * We either have to:
      *  - Increase the number available of threads until over the min_threads
      *    threshold.
-     *  - Decrease the number of available threads until under the max_threads
+     *  - Bump the worker threads so that they exit, until under the max_threads
      *    threshold.
      *  - Do nothing. The current number of threads fall in between the min and
      *    max thresholds. We'll let the pool manage itself.
@@ -333,7 +321,7 @@ void thread_pool_update_params(ThreadPool *pool, AioContext *ctx)
     }
 
     for (int i = pool->cur_threads; i > pool->max_threads; i--) {
-        qemu_sem_post(&pool->sem);
+        qemu_cond_signal(&pool->request_cond);
     }
 
     qemu_mutex_unlock(&pool->lock);
@@ -350,7 +338,7 @@ static void thread_pool_init_one(ThreadPool *pool, AioContext *ctx)
     pool->completion_bh = aio_bh_new(ctx, thread_pool_completion_bh, pool);
     qemu_mutex_init(&pool->lock);
     qemu_cond_init(&pool->worker_stopped);
-    qemu_sem_init(&pool->sem, 0);
+    qemu_cond_init(&pool->request_cond);
     pool->new_thread_bh = aio_bh_new(ctx, spawn_thread_bh_fn, pool);
 
     QLIST_INIT(&pool->head);
@@ -383,15 +371,15 @@ void thread_pool_free(ThreadPool *pool)
 
     /* Wait for worker threads to terminate */
     pool->stopping = true;
+    qemu_cond_broadcast(&pool->request_cond);
     while (pool->cur_threads > 0) {
-        qemu_sem_post(&pool->sem);
         qemu_cond_wait(&pool->worker_stopped, &pool->lock);
     }
 
     qemu_mutex_unlock(&pool->lock);
 
     qemu_bh_delete(pool->completion_bh);
-    qemu_sem_destroy(&pool->sem);
+    qemu_cond_destroy(&pool->request_cond);
     qemu_cond_destroy(&pool->worker_stopped);
     qemu_mutex_destroy(&pool->lock);
     g_free(pool);

From 232e9255478f3849957d2f4b083d6e5d4736ab04 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Sat, 14 May 2022 08:50:12 +0200
Subject: [PATCH 509/935] thread-pool: remove stopping variable

Just setting the max threads to 0 is enough to stop all workers.

Message-Id: <20220514065012.1149539-4-pbonzini@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 util/thread-pool.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/util/thread-pool.c b/util/thread-pool.c
index 6e3d4e4a2f..31113b5860 100644
--- a/util/thread-pool.c
+++ b/util/thread-pool.c
@@ -69,7 +69,6 @@ struct ThreadPool {
     int idle_threads;
     int new_threads;     /* backlog of threads we need to create */
     int pending_threads; /* threads created but not running yet */
-    bool stopping;
     int min_threads;
     int max_threads;
 };
@@ -82,7 +81,7 @@ static void *worker_thread(void *opaque)
     pool->pending_threads--;
     do_spawn_thread(pool);
 
-    while (!pool->stopping && pool->cur_threads <= pool->max_threads) {
+    while (pool->cur_threads <= pool->max_threads) {
         ThreadPoolElement *req;
         int ret;
 
@@ -370,7 +369,7 @@ void thread_pool_free(ThreadPool *pool)
     pool->new_threads = 0;
 
     /* Wait for worker threads to terminate */
-    pool->stopping = true;
+    pool->max_threads = 0;
     qemu_cond_broadcast(&pool->request_cond);
     while (pool->cur_threads > 0) {
         qemu_cond_wait(&pool->worker_stopped, &pool->lock);

From c06ebc0f1b13b9b3802b35969c00e19a31cde484 Mon Sep 17 00:00:00 2001
From: Viktor Prutyanov <viktor.prutyanov@redhat.com>
Date: Fri, 20 May 2022 11:43:39 +0300
Subject: [PATCH 510/935] contrib/elf2dmp: add ELF dump header checking

Add ELF header checking to prevent processing input file which is not
QEMU x86_64 guest memory dump or even not ELF.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1013

Signed-off-by: Viktor Prutyanov <viktor.prutyanov@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220520084339.171684-1-viktor.prutyanov@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 contrib/elf2dmp/qemu_elf.c | 53 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/contrib/elf2dmp/qemu_elf.c b/contrib/elf2dmp/qemu_elf.c
index b601b6d7ba..ebda60dcb8 100644
--- a/contrib/elf2dmp/qemu_elf.c
+++ b/contrib/elf2dmp/qemu_elf.c
@@ -118,6 +118,53 @@ static void exit_states(QEMU_Elf *qe)
     free(qe->state);
 }
 
+static bool check_ehdr(QEMU_Elf *qe)
+{
+    Elf64_Ehdr *ehdr = qe->map;
+
+    if (sizeof(Elf64_Ehdr) > qe->size) {
+        eprintf("Invalid input dump file size\n");
+        return false;
+    }
+
+    if (memcmp(ehdr->e_ident, ELFMAG, SELFMAG)) {
+        eprintf("Invalid ELF signature, input file is not ELF\n");
+        return false;
+    }
+
+    if (ehdr->e_ident[EI_CLASS] != ELFCLASS64 ||
+            ehdr->e_ident[EI_DATA] != ELFDATA2LSB) {
+        eprintf("Invalid ELF class or byte order, must be 64-bit LE\n");
+        return false;
+    }
+
+    if (ehdr->e_ident[EI_VERSION] != EV_CURRENT) {
+        eprintf("Invalid ELF version\n");
+        return false;
+    }
+
+    if (ehdr->e_machine != EM_X86_64) {
+        eprintf("Invalid input dump architecture, only x86_64 is supported\n");
+        return false;
+    }
+
+    if (ehdr->e_type != ET_CORE) {
+        eprintf("Invalid ELF type, must be core file\n");
+        return false;
+    }
+
+    /*
+     * ELF dump file must contain one PT_NOTE and at least one PT_LOAD to
+     * restore physical address space.
+     */
+    if (ehdr->e_phnum < 2) {
+        eprintf("Invalid number of ELF program headers\n");
+        return false;
+    }
+
+    return true;
+}
+
 int QEMU_Elf_init(QEMU_Elf *qe, const char *filename)
 {
     GError *gerr = NULL;
@@ -133,6 +180,12 @@ int QEMU_Elf_init(QEMU_Elf *qe, const char *filename)
     qe->map = g_mapped_file_get_contents(qe->gmf);
     qe->size = g_mapped_file_get_length(qe->gmf);
 
+    if (!check_ehdr(qe)) {
+        eprintf("Input file has the wrong format\n");
+        err = 1;
+        goto out_unmap;
+    }
+
     if (init_states(qe)) {
         eprintf("Failed to extract QEMU CPU states\n");
         err = 1;

From ab9f0f7d44d6795050187b3bb0fc28fe804a062b Mon Sep 17 00:00:00 2001
From: BALATON Zoltan <balaton@eik.bme.hu>
Date: Sat, 23 Apr 2022 11:36:57 +0200
Subject: [PATCH 511/935] hw/audio/ac97: Coding style fixes to avoid checkpatch
 errors
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Message-Id: <62862a057e9c9ec0bb45248b2b9a3a1babb346a6.1650706617.git.balaton@eik.bme.hu>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/audio/ac97.c | 727 ++++++++++++++++++++++++------------------------
 1 file changed, 357 insertions(+), 370 deletions(-)

diff --git a/hw/audio/ac97.c b/hw/audio/ac97.c
index 3cb8131060..6b1c12bece 100644
--- a/hw/audio/ac97.c
+++ b/hw/audio/ac97.c
@@ -87,39 +87,39 @@ enum {
 #define GC_CR    2              /* rw */
 #define GC_VALID_MASK ((1 << 6) - 1)
 
-#define GS_MD3   (1<<17)        /* rw */
-#define GS_AD3   (1<<16)        /* rw */
-#define GS_RCS   (1<<15)        /* rwc */
-#define GS_B3S12 (1<<14)        /* ro */
-#define GS_B2S12 (1<<13)        /* ro */
-#define GS_B1S12 (1<<12)        /* ro */
-#define GS_S1R1  (1<<11)        /* rwc */
-#define GS_S0R1  (1<<10)        /* rwc */
-#define GS_S1CR  (1<<9)         /* ro */
-#define GS_S0CR  (1<<8)         /* ro */
-#define GS_MINT  (1<<7)         /* ro */
-#define GS_POINT (1<<6)         /* ro */
-#define GS_PIINT (1<<5)         /* ro */
-#define GS_RSRVD ((1<<4)|(1<<3))
-#define GS_MOINT (1<<2)         /* ro */
-#define GS_MIINT (1<<1)         /* ro */
+#define GS_MD3   (1 << 17)      /* rw */
+#define GS_AD3   (1 << 16)      /* rw */
+#define GS_RCS   (1 << 15)      /* rwc */
+#define GS_B3S12 (1 << 14)      /* ro */
+#define GS_B2S12 (1 << 13)      /* ro */
+#define GS_B1S12 (1 << 12)      /* ro */
+#define GS_S1R1  (1 << 11)      /* rwc */
+#define GS_S0R1  (1 << 10)      /* rwc */
+#define GS_S1CR  (1 << 9)       /* ro */
+#define GS_S0CR  (1 << 8)       /* ro */
+#define GS_MINT  (1 << 7)       /* ro */
+#define GS_POINT (1 << 6)       /* ro */
+#define GS_PIINT (1 << 5)       /* ro */
+#define GS_RSRVD ((1 << 4) | (1 << 3))
+#define GS_MOINT (1 << 2)       /* ro */
+#define GS_MIINT (1 << 1)       /* ro */
 #define GS_GSCI  1              /* rwc */
-#define GS_RO_MASK (GS_B3S12|                   \
-                    GS_B2S12|                   \
-                    GS_B1S12|                   \
-                    GS_S1CR|                    \
-                    GS_S0CR|                    \
-                    GS_MINT|                    \
-                    GS_POINT|                   \
-                    GS_PIINT|                   \
-                    GS_RSRVD|                   \
-                    GS_MOINT|                   \
+#define GS_RO_MASK (GS_B3S12 | \
+                    GS_B2S12 | \
+                    GS_B1S12 | \
+                    GS_S1CR  | \
+                    GS_S0CR  | \
+                    GS_MINT  | \
+                    GS_POINT | \
+                    GS_PIINT | \
+                    GS_RSRVD | \
+                    GS_MOINT | \
                     GS_MIINT)
 #define GS_VALID_MASK ((1 << 18) - 1)
-#define GS_WCLEAR_MASK (GS_RCS|GS_S1R1|GS_S0R1|GS_GSCI)
+#define GS_WCLEAR_MASK (GS_RCS | GS_S1R1 | GS_S0R1 | GS_GSCI)
 
-#define BD_IOC (1<<31)
-#define BD_BUP (1<<30)
+#define BD_IOC (1 << 31)
+#define BD_BUP (1 << 30)
 
 #define EACS_VRA 1
 #define EACS_VRM 8
@@ -183,7 +183,7 @@ enum {
 };
 
 #ifdef DEBUG_AC97
-#define dolog(...) AUD_log ("ac97", __VA_ARGS__)
+#define dolog(...) AUD_log("ac97", __VA_ARGS__)
 #else
 #define dolog(...)
 #endif
@@ -206,9 +206,9 @@ enum {
     LAST_INDEX
 };
 
-MKREGS (PI, PI_INDEX * 16);
-MKREGS (PO, PO_INDEX * 16);
-MKREGS (MC, MC_INDEX * 16);
+MKREGS(PI, PI_INDEX * 16);
+MKREGS(PO, PO_INDEX * 16);
+MKREGS(MC, MC_INDEX * 16);
 
 enum {
     GLOB_CNT = 0x2c,
@@ -218,36 +218,35 @@ enum {
 
 #define GET_BM(index) (((index) >> 4) & 3)
 
-static void po_callback (void *opaque, int free);
-static void pi_callback (void *opaque, int avail);
-static void mc_callback (void *opaque, int avail);
+static void po_callback(void *opaque, int free);
+static void pi_callback(void *opaque, int avail);
+static void mc_callback(void *opaque, int avail);
 
-static void warm_reset (AC97LinkState *s)
+static void warm_reset(AC97LinkState *s)
 {
-    (void) s;
+    (void)s;
 }
 
-static void cold_reset (AC97LinkState * s)
+static void cold_reset(AC97LinkState *s)
 {
-    (void) s;
+    (void)s;
 }
 
-static void fetch_bd (AC97LinkState *s, AC97BusMasterRegs *r)
+static void fetch_bd(AC97LinkState *s, AC97BusMasterRegs *r)
 {
     uint8_t b[8];
 
-    pci_dma_read (&s->dev, r->bdbar + r->civ * 8, b, 8);
+    pci_dma_read(&s->dev, r->bdbar + r->civ * 8, b, 8);
     r->bd_valid = 1;
-    r->bd.addr = le32_to_cpu (*(uint32_t *) &b[0]) & ~3;
-    r->bd.ctl_len = le32_to_cpu (*(uint32_t *) &b[4]);
+    r->bd.addr = le32_to_cpu(*(uint32_t *) &b[0]) & ~3;
+    r->bd.ctl_len = le32_to_cpu(*(uint32_t *) &b[4]);
     r->picb = r->bd.ctl_len & 0xffff;
-    dolog ("bd %2d addr=%#x ctl=%#06x len=%#x(%d bytes)\n",
-           r->civ, r->bd.addr, r->bd.ctl_len >> 16,
-           r->bd.ctl_len & 0xffff,
-           (r->bd.ctl_len & 0xffff) << 1);
+    dolog("bd %2d addr=0x%x ctl=0x%06x len=0x%x(%d bytes)\n",
+          r->civ, r->bd.addr, r->bd.ctl_len >> 16,
+          r->bd.ctl_len & 0xffff, (r->bd.ctl_len & 0xffff) << 1);
 }
 
-static void update_sr (AC97LinkState *s, AC97BusMasterRegs *r, uint32_t new_sr)
+static void update_sr(AC97LinkState *s, AC97BusMasterRegs *r, uint32_t new_sr)
 {
     int event = 0;
     int level = 0;
@@ -260,8 +259,7 @@ static void update_sr (AC97LinkState *s, AC97BusMasterRegs *r, uint32_t new_sr)
         if (!new_mask) {
             event = 1;
             level = 0;
-        }
-        else {
+        } else {
             if ((new_mask & SR_LVBCI) && (r->cr & CR_LVBIE)) {
                 event = 1;
                 level = 1;
@@ -275,69 +273,67 @@ static void update_sr (AC97LinkState *s, AC97BusMasterRegs *r, uint32_t new_sr)
 
     r->sr = new_sr;
 
-    dolog ("IOC%d LVB%d sr=%#x event=%d level=%d\n",
-           r->sr & SR_BCIS, r->sr & SR_LVBCI,
-           r->sr,
-           event, level);
+    dolog("IOC%d LVB%d sr=0x%x event=%d level=%d\n",
+          r->sr & SR_BCIS, r->sr & SR_LVBCI, r->sr, event, level);
 
-    if (!event)
+    if (!event) {
         return;
+    }
 
     if (level) {
         s->glob_sta |= masks[r - s->bm_regs];
-        dolog ("set irq level=1\n");
+        dolog("set irq level=1\n");
         pci_irq_assert(&s->dev);
-    }
-    else {
+    } else {
         s->glob_sta &= ~masks[r - s->bm_regs];
-        dolog ("set irq level=0\n");
+        dolog("set irq level=0\n");
         pci_irq_deassert(&s->dev);
     }
 }
 
-static void voice_set_active (AC97LinkState *s, int bm_index, int on)
+static void voice_set_active(AC97LinkState *s, int bm_index, int on)
 {
     switch (bm_index) {
     case PI_INDEX:
-        AUD_set_active_in (s->voice_pi, on);
+        AUD_set_active_in(s->voice_pi, on);
         break;
 
     case PO_INDEX:
-        AUD_set_active_out (s->voice_po, on);
+        AUD_set_active_out(s->voice_po, on);
         break;
 
     case MC_INDEX:
-        AUD_set_active_in (s->voice_mc, on);
+        AUD_set_active_in(s->voice_mc, on);
         break;
 
     default:
-        AUD_log ("ac97", "invalid bm_index(%d) in voice_set_active", bm_index);
+        AUD_log("ac97", "invalid bm_index(%d) in voice_set_active", bm_index);
         break;
     }
 }
 
-static void reset_bm_regs (AC97LinkState *s, AC97BusMasterRegs *r)
+static void reset_bm_regs(AC97LinkState *s, AC97BusMasterRegs *r)
 {
-    dolog ("reset_bm_regs\n");
+    dolog("reset_bm_regs\n");
     r->bdbar = 0;
     r->civ = 0;
     r->lvi = 0;
     /** todo do we need to do that? */
-    update_sr (s, r, SR_DCH);
+    update_sr(s, r, SR_DCH);
     r->picb = 0;
     r->piv = 0;
     r->cr = r->cr & CR_DONT_CLEAR_MASK;
     r->bd_valid = 0;
 
-    voice_set_active (s, r - s->bm_regs, 0);
-    memset (s->silence, 0, sizeof (s->silence));
+    voice_set_active(s, r - s->bm_regs, 0);
+    memset(s->silence, 0, sizeof(s->silence));
 }
 
-static void mixer_store (AC97LinkState *s, uint32_t i, uint16_t v)
+static void mixer_store(AC97LinkState *s, uint32_t i, uint16_t v)
 {
-    if (i + 2 > sizeof (s->mixer_data)) {
-        dolog ("mixer_store: index %d out of bounds %zd\n",
-               i, sizeof (s->mixer_data));
+    if (i + 2 > sizeof(s->mixer_data)) {
+        dolog("mixer_store: index %d out of bounds %zd\n",
+              i, sizeof(s->mixer_data));
         return;
     }
 
@@ -345,22 +341,21 @@ static void mixer_store (AC97LinkState *s, uint32_t i, uint16_t v)
     s->mixer_data[i + 1] = v >> 8;
 }
 
-static uint16_t mixer_load (AC97LinkState *s, uint32_t i)
+static uint16_t mixer_load(AC97LinkState *s, uint32_t i)
 {
     uint16_t val = 0xffff;
 
-    if (i + 2 > sizeof (s->mixer_data)) {
-        dolog ("mixer_load: index %d out of bounds %zd\n",
-               i, sizeof (s->mixer_data));
-    }
-    else {
+    if (i + 2 > sizeof(s->mixer_data)) {
+        dolog("mixer_load: index %d out of bounds %zd\n",
+              i, sizeof(s->mixer_data));
+    } else {
         val = s->mixer_data[i + 0] | (s->mixer_data[i + 1] << 8);
     }
 
     return val;
 }
 
-static void open_voice (AC97LinkState *s, int index, int freq)
+static void open_voice(AC97LinkState *s, int index, int freq)
 {
     struct audsettings as;
 
@@ -373,7 +368,7 @@ static void open_voice (AC97LinkState *s, int index, int freq)
         s->invalid_freq[index] = 0;
         switch (index) {
         case PI_INDEX:
-            s->voice_pi = AUD_open_in (
+            s->voice_pi = AUD_open_in(
                 &s->card,
                 s->voice_pi,
                 "ac97.pi",
@@ -384,7 +379,7 @@ static void open_voice (AC97LinkState *s, int index, int freq)
             break;
 
         case PO_INDEX:
-            s->voice_po = AUD_open_out (
+            s->voice_po = AUD_open_out(
                 &s->card,
                 s->voice_po,
                 "ac97.po",
@@ -395,7 +390,7 @@ static void open_voice (AC97LinkState *s, int index, int freq)
             break;
 
         case MC_INDEX:
-            s->voice_mc = AUD_open_in (
+            s->voice_mc = AUD_open_in(
                 &s->card,
                 s->voice_mc,
                 "ac97.mc",
@@ -405,47 +400,46 @@ static void open_voice (AC97LinkState *s, int index, int freq)
                 );
             break;
         }
-    }
-    else {
+    } else {
         s->invalid_freq[index] = freq;
         switch (index) {
         case PI_INDEX:
-            AUD_close_in (&s->card, s->voice_pi);
+            AUD_close_in(&s->card, s->voice_pi);
             s->voice_pi = NULL;
             break;
 
         case PO_INDEX:
-            AUD_close_out (&s->card, s->voice_po);
+            AUD_close_out(&s->card, s->voice_po);
             s->voice_po = NULL;
             break;
 
         case MC_INDEX:
-            AUD_close_in (&s->card, s->voice_mc);
+            AUD_close_in(&s->card, s->voice_mc);
             s->voice_mc = NULL;
             break;
         }
     }
 }
 
-static void reset_voices (AC97LinkState *s, uint8_t active[LAST_INDEX])
+static void reset_voices(AC97LinkState *s, uint8_t active[LAST_INDEX])
 {
     uint16_t freq;
 
-    freq = mixer_load (s, AC97_PCM_LR_ADC_Rate);
-    open_voice (s, PI_INDEX, freq);
-    AUD_set_active_in (s->voice_pi, active[PI_INDEX]);
+    freq = mixer_load(s, AC97_PCM_LR_ADC_Rate);
+    open_voice(s, PI_INDEX, freq);
+    AUD_set_active_in(s->voice_pi, active[PI_INDEX]);
 
-    freq = mixer_load (s, AC97_PCM_Front_DAC_Rate);
-    open_voice (s, PO_INDEX, freq);
-    AUD_set_active_out (s->voice_po, active[PO_INDEX]);
+    freq = mixer_load(s, AC97_PCM_Front_DAC_Rate);
+    open_voice(s, PO_INDEX, freq);
+    AUD_set_active_out(s->voice_po, active[PO_INDEX]);
 
-    freq = mixer_load (s, AC97_MIC_ADC_Rate);
-    open_voice (s, MC_INDEX, freq);
-    AUD_set_active_in (s->voice_mc, active[MC_INDEX]);
+    freq = mixer_load(s, AC97_MIC_ADC_Rate);
+    open_voice(s, MC_INDEX, freq);
+    AUD_set_active_in(s->voice_mc, active[MC_INDEX]);
 }
 
-static void get_volume (uint16_t vol, uint16_t mask, int inverse,
-                        int *mute, uint8_t *lvol, uint8_t *rvol)
+static void get_volume(uint16_t vol, uint16_t mask, int inverse,
+                       int *mute, uint8_t *lvol, uint8_t *rvol)
 {
     *mute = (vol >> MUTE_SHIFT) & 1;
     *rvol = (255 * (vol & mask)) / mask;
@@ -457,120 +451,120 @@ static void get_volume (uint16_t vol, uint16_t mask, int inverse,
     }
 }
 
-static void update_combined_volume_out (AC97LinkState *s)
+static void update_combined_volume_out(AC97LinkState *s)
 {
     uint8_t lvol, rvol, plvol, prvol;
     int mute, pmute;
 
-    get_volume (mixer_load (s, AC97_Master_Volume_Mute), 0x3f, 1,
-                &mute, &lvol, &rvol);
-    get_volume (mixer_load (s, AC97_PCM_Out_Volume_Mute), 0x1f, 1,
-                &pmute, &plvol, &prvol);
+    get_volume(mixer_load(s, AC97_Master_Volume_Mute), 0x3f, 1,
+               &mute, &lvol, &rvol);
+    get_volume(mixer_load(s, AC97_PCM_Out_Volume_Mute), 0x1f, 1,
+               &pmute, &plvol, &prvol);
 
     mute = mute | pmute;
     lvol = (lvol * plvol) / 255;
     rvol = (rvol * prvol) / 255;
 
-    AUD_set_volume_out (s->voice_po, mute, lvol, rvol);
+    AUD_set_volume_out(s->voice_po, mute, lvol, rvol);
 }
 
-static void update_volume_in (AC97LinkState *s)
+static void update_volume_in(AC97LinkState *s)
 {
     uint8_t lvol, rvol;
     int mute;
 
-    get_volume (mixer_load (s, AC97_Record_Gain_Mute), 0x0f, 0,
-                &mute, &lvol, &rvol);
+    get_volume(mixer_load(s, AC97_Record_Gain_Mute), 0x0f, 0,
+               &mute, &lvol, &rvol);
 
-    AUD_set_volume_in (s->voice_pi, mute, lvol, rvol);
+    AUD_set_volume_in(s->voice_pi, mute, lvol, rvol);
 }
 
-static void set_volume (AC97LinkState *s, int index, uint32_t val)
+static void set_volume(AC97LinkState *s, int index, uint32_t val)
 {
     switch (index) {
     case AC97_Master_Volume_Mute:
         val &= 0xbf3f;
-        mixer_store (s, index, val);
-        update_combined_volume_out (s);
+        mixer_store(s, index, val);
+        update_combined_volume_out(s);
         break;
     case AC97_PCM_Out_Volume_Mute:
         val &= 0x9f1f;
-        mixer_store (s, index, val);
-        update_combined_volume_out (s);
+        mixer_store(s, index, val);
+        update_combined_volume_out(s);
         break;
     case AC97_Record_Gain_Mute:
         val &= 0x8f0f;
-        mixer_store (s, index, val);
-        update_volume_in (s);
+        mixer_store(s, index, val);
+        update_volume_in(s);
         break;
     }
 }
 
-static void record_select (AC97LinkState *s, uint32_t val)
+static void record_select(AC97LinkState *s, uint32_t val)
 {
     uint8_t rs = val & REC_MASK;
     uint8_t ls = (val >> 8) & REC_MASK;
-    mixer_store (s, AC97_Record_Select, rs | (ls << 8));
+    mixer_store(s, AC97_Record_Select, rs | (ls << 8));
 }
 
-static void mixer_reset (AC97LinkState *s)
+static void mixer_reset(AC97LinkState *s)
 {
     uint8_t active[LAST_INDEX];
 
-    dolog ("mixer_reset\n");
-    memset (s->mixer_data, 0, sizeof (s->mixer_data));
-    memset (active, 0, sizeof (active));
-    mixer_store (s, AC97_Reset                   , 0x0000); /* 6940 */
-    mixer_store (s, AC97_Headphone_Volume_Mute   , 0x0000);
-    mixer_store (s, AC97_Master_Volume_Mono_Mute , 0x0000);
-    mixer_store (s, AC97_Master_Tone_RL,           0x0000);
-    mixer_store (s, AC97_PC_BEEP_Volume_Mute     , 0x0000);
-    mixer_store (s, AC97_Phone_Volume_Mute       , 0x0000);
-    mixer_store (s, AC97_Mic_Volume_Mute         , 0x0000);
-    mixer_store (s, AC97_Line_In_Volume_Mute     , 0x0000);
-    mixer_store (s, AC97_CD_Volume_Mute          , 0x0000);
-    mixer_store (s, AC97_Video_Volume_Mute       , 0x0000);
-    mixer_store (s, AC97_Aux_Volume_Mute         , 0x0000);
-    mixer_store (s, AC97_Record_Gain_Mic_Mute    , 0x0000);
-    mixer_store (s, AC97_General_Purpose         , 0x0000);
-    mixer_store (s, AC97_3D_Control              , 0x0000);
-    mixer_store (s, AC97_Powerdown_Ctrl_Stat     , 0x000f);
+    dolog("mixer_reset\n");
+    memset(s->mixer_data, 0, sizeof(s->mixer_data));
+    memset(active, 0, sizeof(active));
+    mixer_store(s, AC97_Reset, 0x0000); /* 6940 */
+    mixer_store(s, AC97_Headphone_Volume_Mute, 0x0000);
+    mixer_store(s, AC97_Master_Volume_Mono_Mute, 0x0000);
+    mixer_store(s, AC97_Master_Tone_RL, 0x0000);
+    mixer_store(s, AC97_PC_BEEP_Volume_Mute, 0x0000);
+    mixer_store(s, AC97_Phone_Volume_Mute, 0x0000);
+    mixer_store(s, AC97_Mic_Volume_Mute, 0x0000);
+    mixer_store(s, AC97_Line_In_Volume_Mute, 0x0000);
+    mixer_store(s, AC97_CD_Volume_Mute, 0x0000);
+    mixer_store(s, AC97_Video_Volume_Mute, 0x0000);
+    mixer_store(s, AC97_Aux_Volume_Mute, 0x0000);
+    mixer_store(s, AC97_Record_Gain_Mic_Mute, 0x0000);
+    mixer_store(s, AC97_General_Purpose, 0x0000);
+    mixer_store(s, AC97_3D_Control, 0x0000);
+    mixer_store(s, AC97_Powerdown_Ctrl_Stat, 0x000f);
 
     /*
      * Sigmatel 9700 (STAC9700)
      */
-    mixer_store (s, AC97_Vendor_ID1              , 0x8384);
-    mixer_store (s, AC97_Vendor_ID2              , 0x7600); /* 7608 */
+    mixer_store(s, AC97_Vendor_ID1, 0x8384);
+    mixer_store(s, AC97_Vendor_ID2, 0x7600); /* 7608 */
 
-    mixer_store (s, AC97_Extended_Audio_ID       , 0x0809);
-    mixer_store (s, AC97_Extended_Audio_Ctrl_Stat, 0x0009);
-    mixer_store (s, AC97_PCM_Front_DAC_Rate      , 0xbb80);
-    mixer_store (s, AC97_PCM_Surround_DAC_Rate   , 0xbb80);
-    mixer_store (s, AC97_PCM_LFE_DAC_Rate        , 0xbb80);
-    mixer_store (s, AC97_PCM_LR_ADC_Rate         , 0xbb80);
-    mixer_store (s, AC97_MIC_ADC_Rate            , 0xbb80);
+    mixer_store(s, AC97_Extended_Audio_ID, 0x0809);
+    mixer_store(s, AC97_Extended_Audio_Ctrl_Stat, 0x0009);
+    mixer_store(s, AC97_PCM_Front_DAC_Rate, 0xbb80);
+    mixer_store(s, AC97_PCM_Surround_DAC_Rate, 0xbb80);
+    mixer_store(s, AC97_PCM_LFE_DAC_Rate, 0xbb80);
+    mixer_store(s, AC97_PCM_LR_ADC_Rate, 0xbb80);
+    mixer_store(s, AC97_MIC_ADC_Rate, 0xbb80);
 
-    record_select (s, 0);
-    set_volume (s, AC97_Master_Volume_Mute, 0x8000);
-    set_volume (s, AC97_PCM_Out_Volume_Mute, 0x8808);
-    set_volume (s, AC97_Record_Gain_Mute, 0x8808);
+    record_select(s, 0);
+    set_volume(s, AC97_Master_Volume_Mute, 0x8000);
+    set_volume(s, AC97_PCM_Out_Volume_Mute, 0x8808);
+    set_volume(s, AC97_Record_Gain_Mute, 0x8808);
 
-    reset_voices (s, active);
+    reset_voices(s, active);
 }
 
 /**
  * Native audio mixer
  * I/O Reads
  */
-static uint32_t nam_readb (void *opaque, uint32_t addr)
+static uint32_t nam_readb(void *opaque, uint32_t addr)
 {
     AC97LinkState *s = opaque;
-    dolog ("U nam readb %#x\n", addr);
+    dolog("U nam readb 0x%x\n", addr);
     s->cas = 0;
     return ~0U;
 }
 
-static uint32_t nam_readw (void *opaque, uint32_t addr)
+static uint32_t nam_readw(void *opaque, uint32_t addr)
 {
     AC97LinkState *s = opaque;
     uint32_t index = addr;
@@ -578,10 +572,10 @@ static uint32_t nam_readw (void *opaque, uint32_t addr)
     return mixer_load(s, index);
 }
 
-static uint32_t nam_readl (void *opaque, uint32_t addr)
+static uint32_t nam_readl(void *opaque, uint32_t addr)
 {
     AC97LinkState *s = opaque;
-    dolog ("U nam readl %#x\n", addr);
+    dolog("U nam readl 0x%x\n", addr);
     s->cas = 0;
     return ~0U;
 }
@@ -590,89 +584,84 @@ static uint32_t nam_readl (void *opaque, uint32_t addr)
  * Native audio mixer
  * I/O Writes
  */
-static void nam_writeb (void *opaque, uint32_t addr, uint32_t val)
+static void nam_writeb(void *opaque, uint32_t addr, uint32_t val)
 {
     AC97LinkState *s = opaque;
-    dolog ("U nam writeb %#x <- %#x\n", addr, val);
+    dolog("U nam writeb 0x%x <- 0x%x\n", addr, val);
     s->cas = 0;
 }
 
-static void nam_writew (void *opaque, uint32_t addr, uint32_t val)
+static void nam_writew(void *opaque, uint32_t addr, uint32_t val)
 {
     AC97LinkState *s = opaque;
     uint32_t index = addr;
     s->cas = 0;
     switch (index) {
     case AC97_Reset:
-        mixer_reset (s);
+        mixer_reset(s);
         break;
     case AC97_Powerdown_Ctrl_Stat:
         val &= ~0x800f;
-        val |= mixer_load (s, index) & 0xf;
-        mixer_store (s, index, val);
+        val |= mixer_load(s, index) & 0xf;
+        mixer_store(s, index, val);
         break;
     case AC97_PCM_Out_Volume_Mute:
     case AC97_Master_Volume_Mute:
     case AC97_Record_Gain_Mute:
-        set_volume (s, index, val);
+        set_volume(s, index, val);
         break;
     case AC97_Record_Select:
-        record_select (s, val);
+        record_select(s, val);
         break;
     case AC97_Vendor_ID1:
     case AC97_Vendor_ID2:
-        dolog ("Attempt to write vendor ID to %#x\n", val);
+        dolog("Attempt to write vendor ID to 0x%x\n", val);
         break;
     case AC97_Extended_Audio_ID:
-        dolog ("Attempt to write extended audio ID to %#x\n", val);
+        dolog("Attempt to write extended audio ID to 0x%x\n", val);
         break;
     case AC97_Extended_Audio_Ctrl_Stat:
         if (!(val & EACS_VRA)) {
-            mixer_store (s, AC97_PCM_Front_DAC_Rate, 0xbb80);
-            mixer_store (s, AC97_PCM_LR_ADC_Rate,    0xbb80);
-            open_voice (s, PI_INDEX, 48000);
-            open_voice (s, PO_INDEX, 48000);
+            mixer_store(s, AC97_PCM_Front_DAC_Rate, 0xbb80);
+            mixer_store(s, AC97_PCM_LR_ADC_Rate,    0xbb80);
+            open_voice(s, PI_INDEX, 48000);
+            open_voice(s, PO_INDEX, 48000);
         }
         if (!(val & EACS_VRM)) {
-            mixer_store (s, AC97_MIC_ADC_Rate, 0xbb80);
-            open_voice (s, MC_INDEX, 48000);
+            mixer_store(s, AC97_MIC_ADC_Rate, 0xbb80);
+            open_voice(s, MC_INDEX, 48000);
         }
-        dolog ("Setting extended audio control to %#x\n", val);
-        mixer_store (s, AC97_Extended_Audio_Ctrl_Stat, val);
+        dolog("Setting extended audio control to 0x%x\n", val);
+        mixer_store(s, AC97_Extended_Audio_Ctrl_Stat, val);
         break;
     case AC97_PCM_Front_DAC_Rate:
-        if (mixer_load (s, AC97_Extended_Audio_Ctrl_Stat) & EACS_VRA) {
-            mixer_store (s, index, val);
-            dolog ("Set front DAC rate to %d\n", val);
-            open_voice (s, PO_INDEX, val);
-        }
-        else {
-            dolog ("Attempt to set front DAC rate to %d, "
-                   "but VRA is not set\n",
-                   val);
+        if (mixer_load(s, AC97_Extended_Audio_Ctrl_Stat) & EACS_VRA) {
+            mixer_store(s, index, val);
+            dolog("Set front DAC rate to %d\n", val);
+            open_voice(s, PO_INDEX, val);
+        } else {
+            dolog("Attempt to set front DAC rate to %d, but VRA is not set\n",
+                  val);
         }
         break;
     case AC97_MIC_ADC_Rate:
-        if (mixer_load (s, AC97_Extended_Audio_Ctrl_Stat) & EACS_VRM) {
-            mixer_store (s, index, val);
-            dolog ("Set MIC ADC rate to %d\n", val);
-            open_voice (s, MC_INDEX, val);
-        }
-        else {
-            dolog ("Attempt to set MIC ADC rate to %d, "
-                   "but VRM is not set\n",
-                   val);
+        if (mixer_load(s, AC97_Extended_Audio_Ctrl_Stat) & EACS_VRM) {
+            mixer_store(s, index, val);
+            dolog("Set MIC ADC rate to %d\n", val);
+            open_voice(s, MC_INDEX, val);
+        } else {
+            dolog("Attempt to set MIC ADC rate to %d, but VRM is not set\n",
+                  val);
         }
         break;
     case AC97_PCM_LR_ADC_Rate:
-        if (mixer_load (s, AC97_Extended_Audio_Ctrl_Stat) & EACS_VRA) {
-            mixer_store (s, index, val);
-            dolog ("Set front LR ADC rate to %d\n", val);
-            open_voice (s, PI_INDEX, val);
-        }
-        else {
-            dolog ("Attempt to set LR ADC rate to %d, but VRA is not set\n",
-                    val);
+        if (mixer_load(s, AC97_Extended_Audio_Ctrl_Stat) & EACS_VRA) {
+            mixer_store(s, index, val);
+            dolog("Set front LR ADC rate to %d\n", val);
+            open_voice(s, PI_INDEX, val);
+        } else {
+            dolog("Attempt to set LR ADC rate to %d, but VRA is not set\n",
+                  val);
         }
         break;
     case AC97_Headphone_Volume_Mute:
@@ -693,16 +682,16 @@ static void nam_writew (void *opaque, uint32_t addr, uint32_t val)
         /* None of the features in these regs are emulated, so they are RO */
         break;
     default:
-        dolog ("U nam writew %#x <- %#x\n", addr, val);
-        mixer_store (s, index, val);
+        dolog("U nam writew 0x%x <- 0x%x\n", addr, val);
+        mixer_store(s, index, val);
         break;
     }
 }
 
-static void nam_writel (void *opaque, uint32_t addr, uint32_t val)
+static void nam_writel(void *opaque, uint32_t addr, uint32_t val)
 {
     AC97LinkState *s = opaque;
-    dolog ("U nam writel %#x <- %#x\n", addr, val);
+    dolog("U nam writel 0x%x <- 0x%x\n", addr, val);
     s->cas = 0;
 }
 
@@ -710,7 +699,7 @@ static void nam_writel (void *opaque, uint32_t addr, uint32_t val)
  * Native audio bus master
  * I/O Reads
  */
-static uint32_t nabm_readb (void *opaque, uint32_t addr)
+static uint32_t nabm_readb(void *opaque, uint32_t addr)
 {
     AC97LinkState *s = opaque;
     AC97BusMasterRegs *r = NULL;
@@ -719,53 +708,53 @@ static uint32_t nabm_readb (void *opaque, uint32_t addr)
 
     switch (index) {
     case CAS:
-        dolog ("CAS %d\n", s->cas);
+        dolog("CAS %d\n", s->cas);
         val = s->cas;
         s->cas = 1;
         break;
     case PI_CIV:
     case PO_CIV:
     case MC_CIV:
-        r = &s->bm_regs[GET_BM (index)];
+        r = &s->bm_regs[GET_BM(index)];
         val = r->civ;
-        dolog ("CIV[%d] -> %#x\n", GET_BM (index), val);
+        dolog("CIV[%d] -> 0x%x\n", GET_BM(index), val);
         break;
     case PI_LVI:
     case PO_LVI:
     case MC_LVI:
-        r = &s->bm_regs[GET_BM (index)];
+        r = &s->bm_regs[GET_BM(index)];
         val = r->lvi;
-        dolog ("LVI[%d] -> %#x\n", GET_BM (index), val);
+        dolog("LVI[%d] -> 0x%x\n", GET_BM(index), val);
         break;
     case PI_PIV:
     case PO_PIV:
     case MC_PIV:
-        r = &s->bm_regs[GET_BM (index)];
+        r = &s->bm_regs[GET_BM(index)];
         val = r->piv;
-        dolog ("PIV[%d] -> %#x\n", GET_BM (index), val);
+        dolog("PIV[%d] -> 0x%x\n", GET_BM(index), val);
         break;
     case PI_CR:
     case PO_CR:
     case MC_CR:
-        r = &s->bm_regs[GET_BM (index)];
+        r = &s->bm_regs[GET_BM(index)];
         val = r->cr;
-        dolog ("CR[%d] -> %#x\n", GET_BM (index), val);
+        dolog("CR[%d] -> 0x%x\n", GET_BM(index), val);
         break;
     case PI_SR:
     case PO_SR:
     case MC_SR:
-        r = &s->bm_regs[GET_BM (index)];
+        r = &s->bm_regs[GET_BM(index)];
         val = r->sr & 0xff;
-        dolog ("SRb[%d] -> %#x\n", GET_BM (index), val);
+        dolog("SRb[%d] -> 0x%x\n", GET_BM(index), val);
         break;
     default:
-        dolog ("U nabm readb %#x -> %#x\n", addr, val);
+        dolog("U nabm readb 0x%x -> 0x%x\n", addr, val);
         break;
     }
     return val;
 }
 
-static uint32_t nabm_readw (void *opaque, uint32_t addr)
+static uint32_t nabm_readw(void *opaque, uint32_t addr)
 {
     AC97LinkState *s = opaque;
     AC97BusMasterRegs *r = NULL;
@@ -776,25 +765,25 @@ static uint32_t nabm_readw (void *opaque, uint32_t addr)
     case PI_SR:
     case PO_SR:
     case MC_SR:
-        r = &s->bm_regs[GET_BM (index)];
+        r = &s->bm_regs[GET_BM(index)];
         val = r->sr;
-        dolog ("SR[%d] -> %#x\n", GET_BM (index), val);
+        dolog("SR[%d] -> 0x%x\n", GET_BM(index), val);
         break;
     case PI_PICB:
     case PO_PICB:
     case MC_PICB:
-        r = &s->bm_regs[GET_BM (index)];
+        r = &s->bm_regs[GET_BM(index)];
         val = r->picb;
-        dolog ("PICB[%d] -> %#x\n", GET_BM (index), val);
+        dolog("PICB[%d] -> 0x%x\n", GET_BM(index), val);
         break;
     default:
-        dolog ("U nabm readw %#x -> %#x\n", addr, val);
+        dolog("U nabm readw 0x%x -> 0x%x\n", addr, val);
         break;
     }
     return val;
 }
 
-static uint32_t nabm_readl (void *opaque, uint32_t addr)
+static uint32_t nabm_readl(void *opaque, uint32_t addr)
 {
     AC97LinkState *s = opaque;
     AC97BusMasterRegs *r = NULL;
@@ -805,36 +794,36 @@ static uint32_t nabm_readl (void *opaque, uint32_t addr)
     case PI_BDBAR:
     case PO_BDBAR:
     case MC_BDBAR:
-        r = &s->bm_regs[GET_BM (index)];
+        r = &s->bm_regs[GET_BM(index)];
         val = r->bdbar;
-        dolog ("BMADDR[%d] -> %#x\n", GET_BM (index), val);
+        dolog("BMADDR[%d] -> 0x%x\n", GET_BM(index), val);
         break;
     case PI_CIV:
     case PO_CIV:
     case MC_CIV:
-        r = &s->bm_regs[GET_BM (index)];
+        r = &s->bm_regs[GET_BM(index)];
         val = r->civ | (r->lvi << 8) | (r->sr << 16);
-        dolog ("CIV LVI SR[%d] -> %#x, %#x, %#x\n", GET_BM (index),
+        dolog("CIV LVI SR[%d] -> 0x%x, 0x%x, 0x%x\n", GET_BM(index),
                r->civ, r->lvi, r->sr);
         break;
     case PI_PICB:
     case PO_PICB:
     case MC_PICB:
-        r = &s->bm_regs[GET_BM (index)];
+        r = &s->bm_regs[GET_BM(index)];
         val = r->picb | (r->piv << 16) | (r->cr << 24);
-        dolog ("PICB PIV CR[%d] -> %#x %#x %#x %#x\n", GET_BM (index),
+        dolog("PICB PIV CR[%d] -> 0x%x 0x%x 0x%x 0x%x\n", GET_BM(index),
                val, r->picb, r->piv, r->cr);
         break;
     case GLOB_CNT:
         val = s->glob_cnt;
-        dolog ("glob_cnt -> %#x\n", val);
+        dolog("glob_cnt -> 0x%x\n", val);
         break;
     case GLOB_STA:
         val = s->glob_sta | GS_S0CR;
-        dolog ("glob_sta -> %#x\n", val);
+        dolog("glob_sta -> 0x%x\n", val);
         break;
     default:
-        dolog ("U nabm readl %#x -> %#x\n", addr, val);
+        dolog("U nabm readl 0x%x -> 0x%x\n", addr, val);
         break;
     }
     return val;
@@ -844,7 +833,7 @@ static uint32_t nabm_readl (void *opaque, uint32_t addr)
  * Native audio bus master
  * I/O Writes
  */
-static void nabm_writeb (void *opaque, uint32_t addr, uint32_t val)
+static void nabm_writeb(void *opaque, uint32_t addr, uint32_t val)
 {
     AC97LinkState *s = opaque;
     AC97BusMasterRegs *r = NULL;
@@ -853,54 +842,52 @@ static void nabm_writeb (void *opaque, uint32_t addr, uint32_t val)
     case PI_LVI:
     case PO_LVI:
     case MC_LVI:
-        r = &s->bm_regs[GET_BM (index)];
+        r = &s->bm_regs[GET_BM(index)];
         if ((r->cr & CR_RPBM) && (r->sr & SR_DCH)) {
             r->sr &= ~(SR_DCH | SR_CELV);
             r->civ = r->piv;
             r->piv = (r->piv + 1) % 32;
-            fetch_bd (s, r);
+            fetch_bd(s, r);
         }
         r->lvi = val % 32;
-        dolog ("LVI[%d] <- %#x\n", GET_BM (index), val);
+        dolog("LVI[%d] <- 0x%x\n", GET_BM(index), val);
         break;
     case PI_CR:
     case PO_CR:
     case MC_CR:
-        r = &s->bm_regs[GET_BM (index)];
+        r = &s->bm_regs[GET_BM(index)];
         if (val & CR_RR) {
-            reset_bm_regs (s, r);
-        }
-        else {
+            reset_bm_regs(s, r);
+        } else {
             r->cr = val & CR_VALID_MASK;
             if (!(r->cr & CR_RPBM)) {
-                voice_set_active (s, r - s->bm_regs, 0);
+                voice_set_active(s, r - s->bm_regs, 0);
                 r->sr |= SR_DCH;
-            }
-            else {
+            } else {
                 r->civ = r->piv;
                 r->piv = (r->piv + 1) % 32;
-                fetch_bd (s, r);
+                fetch_bd(s, r);
                 r->sr &= ~SR_DCH;
-                voice_set_active (s, r - s->bm_regs, 1);
+                voice_set_active(s, r - s->bm_regs, 1);
             }
         }
-        dolog ("CR[%d] <- %#x (cr %#x)\n", GET_BM (index), val, r->cr);
+        dolog("CR[%d] <- 0x%x (cr 0x%x)\n", GET_BM(index), val, r->cr);
         break;
     case PI_SR:
     case PO_SR:
     case MC_SR:
-        r = &s->bm_regs[GET_BM (index)];
+        r = &s->bm_regs[GET_BM(index)];
         r->sr |= val & ~(SR_RO_MASK | SR_WCLEAR_MASK);
-        update_sr (s, r, r->sr & ~(val & SR_WCLEAR_MASK));
-        dolog ("SR[%d] <- %#x (sr %#x)\n", GET_BM (index), val, r->sr);
+        update_sr(s, r, r->sr & ~(val & SR_WCLEAR_MASK));
+        dolog("SR[%d] <- 0x%x (sr 0x%x)\n", GET_BM(index), val, r->sr);
         break;
     default:
-        dolog ("U nabm writeb %#x <- %#x\n", addr, val);
+        dolog("U nabm writeb 0x%x <- 0x%x\n", addr, val);
         break;
     }
 }
 
-static void nabm_writew (void *opaque, uint32_t addr, uint32_t val)
+static void nabm_writew(void *opaque, uint32_t addr, uint32_t val)
 {
     AC97LinkState *s = opaque;
     AC97BusMasterRegs *r = NULL;
@@ -909,18 +896,18 @@ static void nabm_writew (void *opaque, uint32_t addr, uint32_t val)
     case PI_SR:
     case PO_SR:
     case MC_SR:
-        r = &s->bm_regs[GET_BM (index)];
+        r = &s->bm_regs[GET_BM(index)];
         r->sr |= val & ~(SR_RO_MASK | SR_WCLEAR_MASK);
-        update_sr (s, r, r->sr & ~(val & SR_WCLEAR_MASK));
-        dolog ("SR[%d] <- %#x (sr %#x)\n", GET_BM (index), val, r->sr);
+        update_sr(s, r, r->sr & ~(val & SR_WCLEAR_MASK));
+        dolog("SR[%d] <- 0x%x (sr 0x%x)\n", GET_BM(index), val, r->sr);
         break;
     default:
-        dolog ("U nabm writew %#x <- %#x\n", addr, val);
+        dolog("U nabm writew 0x%x <- 0x%x\n", addr, val);
         break;
     }
 }
 
-static void nabm_writel (void *opaque, uint32_t addr, uint32_t val)
+static void nabm_writel(void *opaque, uint32_t addr, uint32_t val)
 {
     AC97LinkState *s = opaque;
     AC97BusMasterRegs *r = NULL;
@@ -929,40 +916,42 @@ static void nabm_writel (void *opaque, uint32_t addr, uint32_t val)
     case PI_BDBAR:
     case PO_BDBAR:
     case MC_BDBAR:
-        r = &s->bm_regs[GET_BM (index)];
+        r = &s->bm_regs[GET_BM(index)];
         r->bdbar = val & ~3;
-        dolog ("BDBAR[%d] <- %#x (bdbar %#x)\n",
-               GET_BM (index), val, r->bdbar);
+        dolog("BDBAR[%d] <- 0x%x (bdbar 0x%x)\n", GET_BM(index), val, r->bdbar);
         break;
     case GLOB_CNT:
-        if (val & GC_WR)
-            warm_reset (s);
-        if (val & GC_CR)
-            cold_reset (s);
-        if (!(val & (GC_WR | GC_CR)))
+        if (val & GC_WR) {
+            warm_reset(s);
+        }
+        if (val & GC_CR) {
+            cold_reset(s);
+        }
+        if (!(val & (GC_WR | GC_CR))) {
             s->glob_cnt = val & GC_VALID_MASK;
-        dolog ("glob_cnt <- %#x (glob_cnt %#x)\n", val, s->glob_cnt);
+        }
+        dolog("glob_cnt <- 0x%x (glob_cnt 0x%x)\n", val, s->glob_cnt);
         break;
     case GLOB_STA:
         s->glob_sta &= ~(val & GS_WCLEAR_MASK);
         s->glob_sta |= (val & ~(GS_WCLEAR_MASK | GS_RO_MASK)) & GS_VALID_MASK;
-        dolog ("glob_sta <- %#x (glob_sta %#x)\n", val, s->glob_sta);
+        dolog("glob_sta <- 0x%x (glob_sta 0x%x)\n", val, s->glob_sta);
         break;
     default:
-        dolog ("U nabm writel %#x <- %#x\n", addr, val);
+        dolog("U nabm writel 0x%x <- 0x%x\n", addr, val);
         break;
     }
 }
 
-static int write_audio (AC97LinkState *s, AC97BusMasterRegs *r,
-                        int max, int *stop)
+static int write_audio(AC97LinkState *s, AC97BusMasterRegs *r,
+                       int max, int *stop)
 {
     uint8_t tmpbuf[4096];
     uint32_t addr = r->bd.addr;
     uint32_t temp = r->picb << 1;
     uint32_t written = 0;
     int to_copy = 0;
-    temp = MIN (temp, max);
+    temp = MIN(temp, max);
 
     if (!temp) {
         *stop = 1;
@@ -971,11 +960,11 @@ static int write_audio (AC97LinkState *s, AC97BusMasterRegs *r,
 
     while (temp) {
         int copied;
-        to_copy = MIN (temp, sizeof (tmpbuf));
-        pci_dma_read (&s->dev, addr, tmpbuf, to_copy);
-        copied = AUD_write (s->voice_po, tmpbuf, to_copy);
-        dolog ("write_audio max=%x to_copy=%x copied=%x\n",
-               max, to_copy, copied);
+        to_copy = MIN(temp, sizeof(tmpbuf));
+        pci_dma_read(&s->dev, addr, tmpbuf, to_copy);
+        copied = AUD_write(s->voice_po, tmpbuf, to_copy);
+        dolog("write_audio max=%x to_copy=%x copied=%x\n",
+              max, to_copy, copied);
         if (!copied) {
             *stop = 1;
             break;
@@ -987,11 +976,10 @@ static int write_audio (AC97LinkState *s, AC97BusMasterRegs *r,
 
     if (!temp) {
         if (to_copy < 4) {
-            dolog ("whoops\n");
+            dolog("whoops\n");
             s->last_samp = 0;
-        }
-        else {
-            s->last_samp = *(uint32_t *) &tmpbuf[to_copy - 4];
+        } else {
+            s->last_samp = *(uint32_t *)&tmpbuf[to_copy - 4];
         }
     }
 
@@ -999,37 +987,37 @@ static int write_audio (AC97LinkState *s, AC97BusMasterRegs *r,
     return written;
 }
 
-static void write_bup (AC97LinkState *s, int elapsed)
+static void write_bup(AC97LinkState *s, int elapsed)
 {
-    dolog ("write_bup\n");
+    dolog("write_bup\n");
     if (!(s->bup_flag & BUP_SET)) {
         if (s->bup_flag & BUP_LAST) {
             int i;
             uint8_t *p = s->silence;
-            for (i = 0; i < sizeof (s->silence) / 4; i++, p += 4) {
+            for (i = 0; i < sizeof(s->silence) / 4; i++, p += 4) {
                 *(uint32_t *) p = s->last_samp;
             }
-        }
-        else {
-            memset (s->silence, 0, sizeof (s->silence));
+        } else {
+            memset(s->silence, 0, sizeof(s->silence));
         }
         s->bup_flag |= BUP_SET;
     }
 
     while (elapsed) {
-        int temp = MIN (elapsed, sizeof (s->silence));
+        int temp = MIN(elapsed, sizeof(s->silence));
         while (temp) {
-            int copied = AUD_write (s->voice_po, s->silence, temp);
-            if (!copied)
+            int copied = AUD_write(s->voice_po, s->silence, temp);
+            if (!copied) {
                 return;
+            }
             temp -= copied;
             elapsed -= copied;
         }
     }
 }
 
-static int read_audio (AC97LinkState *s, AC97BusMasterRegs *r,
-                       int max, int *stop)
+static int read_audio(AC97LinkState *s, AC97BusMasterRegs *r,
+                      int max, int *stop)
 {
     uint8_t tmpbuf[4096];
     uint32_t addr = r->bd.addr;
@@ -1038,7 +1026,7 @@ static int read_audio (AC97LinkState *s, AC97BusMasterRegs *r,
     int to_copy = 0;
     SWVoiceIn *voice = (r - s->bm_regs) == MC_INDEX ? s->voice_mc : s->voice_pi;
 
-    temp = MIN (temp, max);
+    temp = MIN(temp, max);
 
     if (!temp) {
         *stop = 1;
@@ -1047,13 +1035,13 @@ static int read_audio (AC97LinkState *s, AC97BusMasterRegs *r,
 
     while (temp) {
         int acquired;
-        to_copy = MIN (temp, sizeof (tmpbuf));
-        acquired = AUD_read (voice, tmpbuf, to_copy);
+        to_copy = MIN(temp, sizeof(tmpbuf));
+        acquired = AUD_read(voice, tmpbuf, to_copy);
         if (!acquired) {
             *stop = 1;
             break;
         }
-        pci_dma_write (&s->dev, addr, tmpbuf, acquired);
+        pci_dma_write(&s->dev, addr, tmpbuf, acquired);
         temp -= acquired;
         addr += acquired;
         nread += acquired;
@@ -1063,14 +1051,14 @@ static int read_audio (AC97LinkState *s, AC97BusMasterRegs *r,
     return nread;
 }
 
-static void transfer_audio (AC97LinkState *s, int index, int elapsed)
+static void transfer_audio(AC97LinkState *s, int index, int elapsed)
 {
     AC97BusMasterRegs *r = &s->bm_regs[index];
     int stop = 0;
 
     if (s->invalid_freq[index]) {
-        AUD_log ("ac97", "attempt to use voice %d with invalid frequency %d\n",
-                 index, s->invalid_freq[index]);
+        AUD_log("ac97", "attempt to use voice %d with invalid frequency %d\n",
+                index, s->invalid_freq[index]);
         return;
     }
 
@@ -1078,7 +1066,7 @@ static void transfer_audio (AC97LinkState *s, int index, int elapsed)
         if (r->cr & CR_RPBM) {
             switch (index) {
             case PO_INDEX:
-                write_bup (s, elapsed);
+                write_bup(s, elapsed);
                 break;
             }
         }
@@ -1089,13 +1077,13 @@ static void transfer_audio (AC97LinkState *s, int index, int elapsed)
         int temp;
 
         if (!r->bd_valid) {
-            dolog ("invalid bd\n");
-            fetch_bd (s, r);
+            dolog("invalid bd\n");
+            fetch_bd(s, r);
         }
 
         if (!r->picb) {
-            dolog ("fresh bd %d is empty %#x %#x\n",
-                   r->civ, r->bd.addr, r->bd.ctl_len);
+            dolog("fresh bd %d is empty 0x%x 0x%x\n",
+                  r->civ, r->bd.addr, r->bd.ctl_len);
             if (r->civ == r->lvi) {
                 r->sr |= SR_DCH; /* CELV? */
                 s->bup_flag = 0;
@@ -1104,20 +1092,20 @@ static void transfer_audio (AC97LinkState *s, int index, int elapsed)
             r->sr &= ~SR_CELV;
             r->civ = r->piv;
             r->piv = (r->piv + 1) % 32;
-            fetch_bd (s, r);
+            fetch_bd(s, r);
             return;
         }
 
         switch (index) {
         case PO_INDEX:
-            temp = write_audio (s, r, elapsed, &stop);
+            temp = write_audio(s, r, elapsed, &stop);
             elapsed -= temp;
             r->picb -= (temp >> 1);
             break;
 
         case PI_INDEX:
         case MC_INDEX:
-            temp = read_audio (s, r, elapsed, &stop);
+            temp = read_audio(s, r, elapsed, &stop);
             elapsed -= temp;
             r->picb -= (temp >> 1);
             break;
@@ -1131,36 +1119,35 @@ static void transfer_audio (AC97LinkState *s, int index, int elapsed)
             }
 
             if (r->civ == r->lvi) {
-                dolog ("Underrun civ (%d) == lvi (%d)\n", r->civ, r->lvi);
+                dolog("Underrun civ (%d) == lvi (%d)\n", r->civ, r->lvi);
 
                 new_sr |= SR_LVBCI | SR_DCH | SR_CELV;
                 stop = 1;
                 s->bup_flag = (r->bd.ctl_len & BD_BUP) ? BUP_LAST : 0;
-            }
-            else {
+            } else {
                 r->civ = r->piv;
                 r->piv = (r->piv + 1) % 32;
-                fetch_bd (s, r);
+                fetch_bd(s, r);
             }
 
-            update_sr (s, r, new_sr);
+            update_sr(s, r, new_sr);
         }
     }
 }
 
-static void pi_callback (void *opaque, int avail)
+static void pi_callback(void *opaque, int avail)
 {
-    transfer_audio (opaque, PI_INDEX, avail);
+    transfer_audio(opaque, PI_INDEX, avail);
 }
 
-static void mc_callback (void *opaque, int avail)
+static void mc_callback(void *opaque, int avail)
 {
-    transfer_audio (opaque, MC_INDEX, avail);
+    transfer_audio(opaque, MC_INDEX, avail);
 }
 
-static void po_callback (void *opaque, int free)
+static void po_callback(void *opaque, int free)
 {
-    transfer_audio (opaque, PO_INDEX, free);
+    transfer_audio(opaque, PO_INDEX, free);
 }
 
 static const VMStateDescription vmstate_ac97_bm_regs = {
@@ -1168,44 +1155,44 @@ static const VMStateDescription vmstate_ac97_bm_regs = {
     .version_id = 1,
     .minimum_version_id = 1,
     .fields = (VMStateField[]) {
-        VMSTATE_UINT32 (bdbar, AC97BusMasterRegs),
-        VMSTATE_UINT8 (civ, AC97BusMasterRegs),
-        VMSTATE_UINT8 (lvi, AC97BusMasterRegs),
-        VMSTATE_UINT16 (sr, AC97BusMasterRegs),
-        VMSTATE_UINT16 (picb, AC97BusMasterRegs),
-        VMSTATE_UINT8 (piv, AC97BusMasterRegs),
-        VMSTATE_UINT8 (cr, AC97BusMasterRegs),
-        VMSTATE_UINT32 (bd_valid, AC97BusMasterRegs),
-        VMSTATE_UINT32 (bd.addr, AC97BusMasterRegs),
-        VMSTATE_UINT32 (bd.ctl_len, AC97BusMasterRegs),
-        VMSTATE_END_OF_LIST ()
+        VMSTATE_UINT32(bdbar, AC97BusMasterRegs),
+        VMSTATE_UINT8(civ, AC97BusMasterRegs),
+        VMSTATE_UINT8(lvi, AC97BusMasterRegs),
+        VMSTATE_UINT16(sr, AC97BusMasterRegs),
+        VMSTATE_UINT16(picb, AC97BusMasterRegs),
+        VMSTATE_UINT8(piv, AC97BusMasterRegs),
+        VMSTATE_UINT8(cr, AC97BusMasterRegs),
+        VMSTATE_UINT32(bd_valid, AC97BusMasterRegs),
+        VMSTATE_UINT32(bd.addr, AC97BusMasterRegs),
+        VMSTATE_UINT32(bd.ctl_len, AC97BusMasterRegs),
+        VMSTATE_END_OF_LIST()
     }
 };
 
-static int ac97_post_load (void *opaque, int version_id)
+static int ac97_post_load(void *opaque, int version_id)
 {
     uint8_t active[LAST_INDEX];
     AC97LinkState *s = opaque;
 
-    record_select (s, mixer_load (s, AC97_Record_Select));
-    set_volume (s, AC97_Master_Volume_Mute,
-                mixer_load (s, AC97_Master_Volume_Mute));
-    set_volume (s, AC97_PCM_Out_Volume_Mute,
-                mixer_load (s, AC97_PCM_Out_Volume_Mute));
-    set_volume (s, AC97_Record_Gain_Mute,
-                mixer_load (s, AC97_Record_Gain_Mute));
+    record_select(s, mixer_load(s, AC97_Record_Select));
+    set_volume(s, AC97_Master_Volume_Mute,
+               mixer_load(s, AC97_Master_Volume_Mute));
+    set_volume(s, AC97_PCM_Out_Volume_Mute,
+               mixer_load(s, AC97_PCM_Out_Volume_Mute));
+    set_volume(s, AC97_Record_Gain_Mute,
+               mixer_load(s, AC97_Record_Gain_Mute));
 
     active[PI_INDEX] = !!(s->bm_regs[PI_INDEX].cr & CR_RPBM);
     active[PO_INDEX] = !!(s->bm_regs[PO_INDEX].cr & CR_RPBM);
     active[MC_INDEX] = !!(s->bm_regs[MC_INDEX].cr & CR_RPBM);
-    reset_voices (s, active);
+    reset_voices(s, active);
 
     s->bup_flag = 0;
     s->last_samp = 0;
     return 0;
 }
 
-static bool is_version_2 (void *opaque, int version_id)
+static bool is_version_2(void *opaque, int version_id)
 {
     return version_id == 2;
 }
@@ -1216,15 +1203,15 @@ static const VMStateDescription vmstate_ac97 = {
     .minimum_version_id = 2,
     .post_load = ac97_post_load,
     .fields = (VMStateField[]) {
-        VMSTATE_PCI_DEVICE (dev, AC97LinkState),
-        VMSTATE_UINT32 (glob_cnt, AC97LinkState),
-        VMSTATE_UINT32 (glob_sta, AC97LinkState),
-        VMSTATE_UINT32 (cas, AC97LinkState),
-        VMSTATE_STRUCT_ARRAY (bm_regs, AC97LinkState, 3, 1,
-                              vmstate_ac97_bm_regs, AC97BusMasterRegs),
-        VMSTATE_BUFFER (mixer_data, AC97LinkState),
-        VMSTATE_UNUSED_TEST (is_version_2, 3),
-        VMSTATE_END_OF_LIST ()
+        VMSTATE_PCI_DEVICE(dev, AC97LinkState),
+        VMSTATE_UINT32(glob_cnt, AC97LinkState),
+        VMSTATE_UINT32(glob_sta, AC97LinkState),
+        VMSTATE_UINT32(cas, AC97LinkState),
+        VMSTATE_STRUCT_ARRAY(bm_regs, AC97LinkState, 3, 1,
+                             vmstate_ac97_bm_regs, AC97BusMasterRegs),
+        VMSTATE_BUFFER(mixer_data, AC97LinkState),
+        VMSTATE_UNUSED_TEST(is_version_2, 3),
+        VMSTATE_END_OF_LIST()
     }
 };
 
@@ -1295,7 +1282,7 @@ static uint64_t nabm_read(void *opaque, hwaddr addr, unsigned size)
 }
 
 static void nabm_write(void *opaque, hwaddr addr, uint64_t val,
-                      unsigned size)
+                       unsigned size)
 {
     if ((addr / size) > 64) {
         return;
@@ -1325,20 +1312,20 @@ static const MemoryRegionOps ac97_io_nabm_ops = {
     .endianness = DEVICE_LITTLE_ENDIAN,
 };
 
-static void ac97_on_reset (DeviceState *dev)
+static void ac97_on_reset(DeviceState *dev)
 {
     AC97LinkState *s = container_of(dev, AC97LinkState, dev.qdev);
 
-    reset_bm_regs (s, &s->bm_regs[0]);
-    reset_bm_regs (s, &s->bm_regs[1]);
-    reset_bm_regs (s, &s->bm_regs[2]);
+    reset_bm_regs(s, &s->bm_regs[0]);
+    reset_bm_regs(s, &s->bm_regs[1]);
+    reset_bm_regs(s, &s->bm_regs[2]);
 
     /*
      * Reset the mixer too. The Windows XP driver seems to rely on
      * this. At least it wants to read the vendor id before it resets
      * the codec manually.
      */
-    mixer_reset (s);
+    mixer_reset(s);
 }
 
 static void ac97_realize(PCIDevice *dev, Error **errp)
@@ -1373,13 +1360,13 @@ static void ac97_realize(PCIDevice *dev, Error **errp)
     c[PCI_INTERRUPT_LINE] = 0x00;      /* intr_ln interrupt line rw */
     c[PCI_INTERRUPT_PIN] = 0x01;      /* intr_pn interrupt pin ro */
 
-    memory_region_init_io (&s->io_nam, OBJECT(s), &ac97_io_nam_ops, s,
-                           "ac97-nam", 1024);
-    memory_region_init_io (&s->io_nabm, OBJECT(s), &ac97_io_nabm_ops, s,
-                           "ac97-nabm", 256);
-    pci_register_bar (&s->dev, 0, PCI_BASE_ADDRESS_SPACE_IO, &s->io_nam);
-    pci_register_bar (&s->dev, 1, PCI_BASE_ADDRESS_SPACE_IO, &s->io_nabm);
-    AUD_register_card ("ac97", &s->card);
+    memory_region_init_io(&s->io_nam, OBJECT(s), &ac97_io_nam_ops, s,
+                          "ac97-nam", 1024);
+    memory_region_init_io(&s->io_nabm, OBJECT(s), &ac97_io_nabm_ops, s,
+                          "ac97-nabm", 256);
+    pci_register_bar(&s->dev, 0, PCI_BASE_ADDRESS_SPACE_IO, &s->io_nam);
+    pci_register_bar(&s->dev, 1, PCI_BASE_ADDRESS_SPACE_IO, &s->io_nabm);
+    AUD_register_card("ac97", &s->card);
     ac97_on_reset(DEVICE(s));
 }
 
@@ -1395,13 +1382,13 @@ static void ac97_exit(PCIDevice *dev)
 
 static Property ac97_properties[] = {
     DEFINE_AUDIO_PROPERTIES(AC97LinkState, card),
-    DEFINE_PROP_END_OF_LIST (),
+    DEFINE_PROP_END_OF_LIST(),
 };
 
-static void ac97_class_init (ObjectClass *klass, void *data)
+static void ac97_class_init(ObjectClass *klass, void *data)
 {
-    DeviceClass *dc = DEVICE_CLASS (klass);
-    PCIDeviceClass *k = PCI_DEVICE_CLASS (klass);
+    DeviceClass *dc = DEVICE_CLASS(klass);
+    PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
 
     k->realize = ac97_realize;
     k->exit = ac97_exit;
@@ -1419,7 +1406,7 @@ static void ac97_class_init (ObjectClass *klass, void *data)
 static const TypeInfo ac97_info = {
     .name          = TYPE_AC97,
     .parent        = TYPE_PCI_DEVICE,
-    .instance_size = sizeof (AC97LinkState),
+    .instance_size = sizeof(AC97LinkState),
     .class_init    = ac97_class_init,
     .interfaces = (InterfaceInfo[]) {
         { INTERFACE_CONVENTIONAL_PCI_DEVICE },
@@ -1427,11 +1414,11 @@ static const TypeInfo ac97_info = {
     },
 };
 
-static void ac97_register_types (void)
+static void ac97_register_types(void)
 {
-    type_register_static (&ac97_info);
+    type_register_static(&ac97_info);
     deprecated_register_soundhw("ac97", "Intel 82801AA AC97 Audio",
                                 0, TYPE_AC97);
 }
 
-type_init (ac97_register_types)
+type_init(ac97_register_types)

From dafea9e2868a5349bb0f1650dd18fa1aac806ee4 Mon Sep 17 00:00:00 2001
From: BALATON Zoltan <balaton@eik.bme.hu>
Date: Sat, 23 Apr 2022 11:36:57 +0200
Subject: [PATCH 512/935] hw/audio/ac97: Remove unimplemented reset functions
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The warm_reset() and cold_reset() functions are not implemented and do
nothing so no point in calling them or keep around as dead code.
Therefore remove them for now.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Message-Id: <cc6e99fd498a9ae358ebce787fc04ab6e8201879.1650706617.git.balaton@eik.bme.hu>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/audio/ac97.c | 17 +----------------
 1 file changed, 1 insertion(+), 16 deletions(-)

diff --git a/hw/audio/ac97.c b/hw/audio/ac97.c
index 6b1c12bece..6584aa749e 100644
--- a/hw/audio/ac97.c
+++ b/hw/audio/ac97.c
@@ -222,16 +222,6 @@ static void po_callback(void *opaque, int free);
 static void pi_callback(void *opaque, int avail);
 static void mc_callback(void *opaque, int avail);
 
-static void warm_reset(AC97LinkState *s)
-{
-    (void)s;
-}
-
-static void cold_reset(AC97LinkState *s)
-{
-    (void)s;
-}
-
 static void fetch_bd(AC97LinkState *s, AC97BusMasterRegs *r)
 {
     uint8_t b[8];
@@ -921,12 +911,7 @@ static void nabm_writel(void *opaque, uint32_t addr, uint32_t val)
         dolog("BDBAR[%d] <- 0x%x (bdbar 0x%x)\n", GET_BM(index), val, r->bdbar);
         break;
     case GLOB_CNT:
-        if (val & GC_WR) {
-            warm_reset(s);
-        }
-        if (val & GC_CR) {
-            cold_reset(s);
-        }
+        /* TODO: Handle WR or CR being set (warm/cold reset requests) */
         if (!(val & (GC_WR | GC_CR))) {
             s->glob_cnt = val & GC_VALID_MASK;
         }

From dba2b2941ca2ebd5275b3b2c6e72678f98f83a8a Mon Sep 17 00:00:00 2001
From: BALATON Zoltan <balaton@eik.bme.hu>
Date: Sat, 23 Apr 2022 11:36:57 +0200
Subject: [PATCH 513/935] hw/audio/ac97: Remove unneeded local variables

Several functions have a local variable that is just a copy of one of
the function parameters. This is unneeded complication so just get rid
of these.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <d959aa0b267eb139a994e41ca0b7ba87d9cef7a9.1650706617.git.balaton@eik.bme.hu>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/audio/ac97.c | 102 +++++++++++++++++++++++-------------------------
 1 file changed, 49 insertions(+), 53 deletions(-)

diff --git a/hw/audio/ac97.c b/hw/audio/ac97.c
index 6584aa749e..be2dd701a4 100644
--- a/hw/audio/ac97.c
+++ b/hw/audio/ac97.c
@@ -557,9 +557,8 @@ static uint32_t nam_readb(void *opaque, uint32_t addr)
 static uint32_t nam_readw(void *opaque, uint32_t addr)
 {
     AC97LinkState *s = opaque;
-    uint32_t index = addr;
     s->cas = 0;
-    return mixer_load(s, index);
+    return mixer_load(s, addr);
 }
 
 static uint32_t nam_readl(void *opaque, uint32_t addr)
@@ -584,21 +583,21 @@ static void nam_writeb(void *opaque, uint32_t addr, uint32_t val)
 static void nam_writew(void *opaque, uint32_t addr, uint32_t val)
 {
     AC97LinkState *s = opaque;
-    uint32_t index = addr;
+
     s->cas = 0;
-    switch (index) {
+    switch (addr) {
     case AC97_Reset:
         mixer_reset(s);
         break;
     case AC97_Powerdown_Ctrl_Stat:
         val &= ~0x800f;
-        val |= mixer_load(s, index) & 0xf;
-        mixer_store(s, index, val);
+        val |= mixer_load(s, addr) & 0xf;
+        mixer_store(s, addr, val);
         break;
     case AC97_PCM_Out_Volume_Mute:
     case AC97_Master_Volume_Mute:
     case AC97_Record_Gain_Mute:
-        set_volume(s, index, val);
+        set_volume(s, addr, val);
         break;
     case AC97_Record_Select:
         record_select(s, val);
@@ -626,7 +625,7 @@ static void nam_writew(void *opaque, uint32_t addr, uint32_t val)
         break;
     case AC97_PCM_Front_DAC_Rate:
         if (mixer_load(s, AC97_Extended_Audio_Ctrl_Stat) & EACS_VRA) {
-            mixer_store(s, index, val);
+            mixer_store(s, addr, val);
             dolog("Set front DAC rate to %d\n", val);
             open_voice(s, PO_INDEX, val);
         } else {
@@ -636,7 +635,7 @@ static void nam_writew(void *opaque, uint32_t addr, uint32_t val)
         break;
     case AC97_MIC_ADC_Rate:
         if (mixer_load(s, AC97_Extended_Audio_Ctrl_Stat) & EACS_VRM) {
-            mixer_store(s, index, val);
+            mixer_store(s, addr, val);
             dolog("Set MIC ADC rate to %d\n", val);
             open_voice(s, MC_INDEX, val);
         } else {
@@ -646,7 +645,7 @@ static void nam_writew(void *opaque, uint32_t addr, uint32_t val)
         break;
     case AC97_PCM_LR_ADC_Rate:
         if (mixer_load(s, AC97_Extended_Audio_Ctrl_Stat) & EACS_VRA) {
-            mixer_store(s, index, val);
+            mixer_store(s, addr, val);
             dolog("Set front LR ADC rate to %d\n", val);
             open_voice(s, PI_INDEX, val);
         } else {
@@ -673,7 +672,7 @@ static void nam_writew(void *opaque, uint32_t addr, uint32_t val)
         break;
     default:
         dolog("U nam writew 0x%x <- 0x%x\n", addr, val);
-        mixer_store(s, index, val);
+        mixer_store(s, addr, val);
         break;
     }
 }
@@ -693,10 +692,9 @@ static uint32_t nabm_readb(void *opaque, uint32_t addr)
 {
     AC97LinkState *s = opaque;
     AC97BusMasterRegs *r = NULL;
-    uint32_t index = addr;
     uint32_t val = ~0U;
 
-    switch (index) {
+    switch (addr) {
     case CAS:
         dolog("CAS %d\n", s->cas);
         val = s->cas;
@@ -705,37 +703,37 @@ static uint32_t nabm_readb(void *opaque, uint32_t addr)
     case PI_CIV:
     case PO_CIV:
     case MC_CIV:
-        r = &s->bm_regs[GET_BM(index)];
+        r = &s->bm_regs[GET_BM(addr)];
         val = r->civ;
-        dolog("CIV[%d] -> 0x%x\n", GET_BM(index), val);
+        dolog("CIV[%d] -> 0x%x\n", GET_BM(addr), val);
         break;
     case PI_LVI:
     case PO_LVI:
     case MC_LVI:
-        r = &s->bm_regs[GET_BM(index)];
+        r = &s->bm_regs[GET_BM(addr)];
         val = r->lvi;
-        dolog("LVI[%d] -> 0x%x\n", GET_BM(index), val);
+        dolog("LVI[%d] -> 0x%x\n", GET_BM(addr), val);
         break;
     case PI_PIV:
     case PO_PIV:
     case MC_PIV:
-        r = &s->bm_regs[GET_BM(index)];
+        r = &s->bm_regs[GET_BM(addr)];
         val = r->piv;
-        dolog("PIV[%d] -> 0x%x\n", GET_BM(index), val);
+        dolog("PIV[%d] -> 0x%x\n", GET_BM(addr), val);
         break;
     case PI_CR:
     case PO_CR:
     case MC_CR:
-        r = &s->bm_regs[GET_BM(index)];
+        r = &s->bm_regs[GET_BM(addr)];
         val = r->cr;
-        dolog("CR[%d] -> 0x%x\n", GET_BM(index), val);
+        dolog("CR[%d] -> 0x%x\n", GET_BM(addr), val);
         break;
     case PI_SR:
     case PO_SR:
     case MC_SR:
-        r = &s->bm_regs[GET_BM(index)];
+        r = &s->bm_regs[GET_BM(addr)];
         val = r->sr & 0xff;
-        dolog("SRb[%d] -> 0x%x\n", GET_BM(index), val);
+        dolog("SRb[%d] -> 0x%x\n", GET_BM(addr), val);
         break;
     default:
         dolog("U nabm readb 0x%x -> 0x%x\n", addr, val);
@@ -748,23 +746,22 @@ static uint32_t nabm_readw(void *opaque, uint32_t addr)
 {
     AC97LinkState *s = opaque;
     AC97BusMasterRegs *r = NULL;
-    uint32_t index = addr;
     uint32_t val = ~0U;
 
-    switch (index) {
+    switch (addr) {
     case PI_SR:
     case PO_SR:
     case MC_SR:
-        r = &s->bm_regs[GET_BM(index)];
+        r = &s->bm_regs[GET_BM(addr)];
         val = r->sr;
-        dolog("SR[%d] -> 0x%x\n", GET_BM(index), val);
+        dolog("SR[%d] -> 0x%x\n", GET_BM(addr), val);
         break;
     case PI_PICB:
     case PO_PICB:
     case MC_PICB:
-        r = &s->bm_regs[GET_BM(index)];
+        r = &s->bm_regs[GET_BM(addr)];
         val = r->picb;
-        dolog("PICB[%d] -> 0x%x\n", GET_BM(index), val);
+        dolog("PICB[%d] -> 0x%x\n", GET_BM(addr), val);
         break;
     default:
         dolog("U nabm readw 0x%x -> 0x%x\n", addr, val);
@@ -777,31 +774,30 @@ static uint32_t nabm_readl(void *opaque, uint32_t addr)
 {
     AC97LinkState *s = opaque;
     AC97BusMasterRegs *r = NULL;
-    uint32_t index = addr;
     uint32_t val = ~0U;
 
-    switch (index) {
+    switch (addr) {
     case PI_BDBAR:
     case PO_BDBAR:
     case MC_BDBAR:
-        r = &s->bm_regs[GET_BM(index)];
+        r = &s->bm_regs[GET_BM(addr)];
         val = r->bdbar;
-        dolog("BMADDR[%d] -> 0x%x\n", GET_BM(index), val);
+        dolog("BMADDR[%d] -> 0x%x\n", GET_BM(addr), val);
         break;
     case PI_CIV:
     case PO_CIV:
     case MC_CIV:
-        r = &s->bm_regs[GET_BM(index)];
+        r = &s->bm_regs[GET_BM(addr)];
         val = r->civ | (r->lvi << 8) | (r->sr << 16);
-        dolog("CIV LVI SR[%d] -> 0x%x, 0x%x, 0x%x\n", GET_BM(index),
+        dolog("CIV LVI SR[%d] -> 0x%x, 0x%x, 0x%x\n", GET_BM(addr),
                r->civ, r->lvi, r->sr);
         break;
     case PI_PICB:
     case PO_PICB:
     case MC_PICB:
-        r = &s->bm_regs[GET_BM(index)];
+        r = &s->bm_regs[GET_BM(addr)];
         val = r->picb | (r->piv << 16) | (r->cr << 24);
-        dolog("PICB PIV CR[%d] -> 0x%x 0x%x 0x%x 0x%x\n", GET_BM(index),
+        dolog("PICB PIV CR[%d] -> 0x%x 0x%x 0x%x 0x%x\n", GET_BM(addr),
                val, r->picb, r->piv, r->cr);
         break;
     case GLOB_CNT:
@@ -827,12 +823,12 @@ static void nabm_writeb(void *opaque, uint32_t addr, uint32_t val)
 {
     AC97LinkState *s = opaque;
     AC97BusMasterRegs *r = NULL;
-    uint32_t index = addr;
-    switch (index) {
+
+    switch (addr) {
     case PI_LVI:
     case PO_LVI:
     case MC_LVI:
-        r = &s->bm_regs[GET_BM(index)];
+        r = &s->bm_regs[GET_BM(addr)];
         if ((r->cr & CR_RPBM) && (r->sr & SR_DCH)) {
             r->sr &= ~(SR_DCH | SR_CELV);
             r->civ = r->piv;
@@ -840,12 +836,12 @@ static void nabm_writeb(void *opaque, uint32_t addr, uint32_t val)
             fetch_bd(s, r);
         }
         r->lvi = val % 32;
-        dolog("LVI[%d] <- 0x%x\n", GET_BM(index), val);
+        dolog("LVI[%d] <- 0x%x\n", GET_BM(addr), val);
         break;
     case PI_CR:
     case PO_CR:
     case MC_CR:
-        r = &s->bm_regs[GET_BM(index)];
+        r = &s->bm_regs[GET_BM(addr)];
         if (val & CR_RR) {
             reset_bm_regs(s, r);
         } else {
@@ -861,15 +857,15 @@ static void nabm_writeb(void *opaque, uint32_t addr, uint32_t val)
                 voice_set_active(s, r - s->bm_regs, 1);
             }
         }
-        dolog("CR[%d] <- 0x%x (cr 0x%x)\n", GET_BM(index), val, r->cr);
+        dolog("CR[%d] <- 0x%x (cr 0x%x)\n", GET_BM(addr), val, r->cr);
         break;
     case PI_SR:
     case PO_SR:
     case MC_SR:
-        r = &s->bm_regs[GET_BM(index)];
+        r = &s->bm_regs[GET_BM(addr)];
         r->sr |= val & ~(SR_RO_MASK | SR_WCLEAR_MASK);
         update_sr(s, r, r->sr & ~(val & SR_WCLEAR_MASK));
-        dolog("SR[%d] <- 0x%x (sr 0x%x)\n", GET_BM(index), val, r->sr);
+        dolog("SR[%d] <- 0x%x (sr 0x%x)\n", GET_BM(addr), val, r->sr);
         break;
     default:
         dolog("U nabm writeb 0x%x <- 0x%x\n", addr, val);
@@ -881,15 +877,15 @@ static void nabm_writew(void *opaque, uint32_t addr, uint32_t val)
 {
     AC97LinkState *s = opaque;
     AC97BusMasterRegs *r = NULL;
-    uint32_t index = addr;
-    switch (index) {
+
+    switch (addr) {
     case PI_SR:
     case PO_SR:
     case MC_SR:
-        r = &s->bm_regs[GET_BM(index)];
+        r = &s->bm_regs[GET_BM(addr)];
         r->sr |= val & ~(SR_RO_MASK | SR_WCLEAR_MASK);
         update_sr(s, r, r->sr & ~(val & SR_WCLEAR_MASK));
-        dolog("SR[%d] <- 0x%x (sr 0x%x)\n", GET_BM(index), val, r->sr);
+        dolog("SR[%d] <- 0x%x (sr 0x%x)\n", GET_BM(addr), val, r->sr);
         break;
     default:
         dolog("U nabm writew 0x%x <- 0x%x\n", addr, val);
@@ -901,14 +897,14 @@ static void nabm_writel(void *opaque, uint32_t addr, uint32_t val)
 {
     AC97LinkState *s = opaque;
     AC97BusMasterRegs *r = NULL;
-    uint32_t index = addr;
-    switch (index) {
+
+    switch (addr) {
     case PI_BDBAR:
     case PO_BDBAR:
     case MC_BDBAR:
-        r = &s->bm_regs[GET_BM(index)];
+        r = &s->bm_regs[GET_BM(addr)];
         r->bdbar = val & ~3;
-        dolog("BDBAR[%d] <- 0x%x (bdbar 0x%x)\n", GET_BM(index), val, r->bdbar);
+        dolog("BDBAR[%d] <- 0x%x (bdbar 0x%x)\n", GET_BM(addr), val, r->bdbar);
         break;
     case GLOB_CNT:
         /* TODO: Handle WR or CR being set (warm/cold reset requests) */

From 267b5e7e378afd260004cb37a66a6fcd641e3b53 Mon Sep 17 00:00:00 2001
From: "Maciej S. Szmigiero" <maciej.szmigiero@oracle.com>
Date: Mon, 23 May 2022 18:26:58 +0200
Subject: [PATCH 514/935] target/i386/kvm: Fix disabling MPX on "-cpu host"
 with MPX-capable host

Since KVM commit 5f76f6f5ff96 ("KVM: nVMX: Do not expose MPX VMX controls when guest MPX disabled")
it is not possible to disable MPX on a "-cpu host" just by adding "-mpx"
there if the host CPU does indeed support MPX.
QEMU will fail to set MSR_IA32_VMX_TRUE_{EXIT,ENTRY}_CTLS MSRs in this case
and so trigger an assertion failure.

Instead, besides "-mpx" one has to explicitly add also
"-vmx-exit-clear-bndcfgs" and "-vmx-entry-load-bndcfgs" to QEMU command
line to make it work, which is a bit convoluted.

Make the MPX-related bits in FEAT_VMX_{EXIT,ENTRY}_CTLS dependent on MPX
being actually enabled so such workarounds are no longer necessary.

Signed-off-by: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
Message-Id: <51aa2125c76363204cc23c27165e778097c33f0b.1653323077.git.maciej.szmigiero@oracle.com>
Cc: qemu-stable@nongnu.org
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/cpu.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 35c3475e6c..385691458f 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1355,6 +1355,14 @@ static FeatureDep feature_dependencies[] = {
         .from = { FEAT_7_0_EBX,             CPUID_7_0_EBX_INVPCID },
         .to = { FEAT_VMX_SECONDARY_CTLS,    VMX_SECONDARY_EXEC_ENABLE_INVPCID },
     },
+    {
+        .from = { FEAT_7_0_EBX,             CPUID_7_0_EBX_MPX },
+        .to = { FEAT_VMX_EXIT_CTLS,         VMX_VM_EXIT_CLEAR_BNDCFGS },
+    },
+    {
+        .from = { FEAT_7_0_EBX,             CPUID_7_0_EBX_MPX },
+        .to = { FEAT_VMX_ENTRY_CTLS,        VMX_VM_ENTRY_LOAD_BNDCFGS },
+    },
     {
         .from = { FEAT_7_0_EBX,             CPUID_7_0_EBX_RDSEED },
         .to = { FEAT_VMX_SECONDARY_CTLS,    VMX_SECONDARY_EXEC_RDSEED_EXITING },

From 758c925ee04ecd9d2d543a60f3c389e4e814c6aa Mon Sep 17 00:00:00 2001
From: Lev Kujawski <lkujaw@member.fsf.org>
Date: Fri, 20 May 2022 23:52:00 +0000
Subject: [PATCH 515/935] ide_ioport_read: Return lower octet of data register
 instead of 0xFF

Prior to this patch, the pre-GRUB Solaris x86 bootloader would fail to
load on QEMU with the following screen output:

SunOS Secondary Boot version 3.00

prom_panic: Could not mount filesystem.
Entering boot debugger:
[136419]: _

This occurs because the bootloader issues an ATA IDENTIFY DEVICE
command, and then reads the resulting 256 words of parameter
information using inb rather than the correct inw. As the previous
behavior of QEMU was to return 0xFF and not advance the drive's sector
buffer, DRQ would never be cleared and the bootloader would be blocked
from selecting a secondary ATA device, such as an optical drive.

Resolves:
* [Bug 1639394] Unable to boot Solaris 8/9 x86 under Fedora 24

Signed-off-by: Lev Kujawski <lkujaw@member.fsf.org>
Message-Id: <20220520235200.1138450-1-lkujaw@member.fsf.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/ide/core.c  | 6 +++++-
 hw/ide/macio.c | 4 +++-
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/hw/ide/core.c b/hw/ide/core.c
index 3a5afff5d7..c2caa54285 100644
--- a/hw/ide/core.c
+++ b/hw/ide/core.c
@@ -2166,7 +2166,11 @@ uint32_t ide_ioport_read(void *opaque, uint32_t addr)
     hob = bus->cmd & (IDE_CTRL_HOB);
     switch (reg_num) {
     case ATA_IOPORT_RR_DATA:
-        ret = 0xff;
+        /*
+         * The pre-GRUB Solaris x86 bootloader relies upon inb
+         * consuming a word from the drive's sector buffer.
+         */
+        ret = ide_data_readw(bus, addr) & 0xff;
         break;
     case ATA_IOPORT_RR_ERROR:
         if ((!bus->ifs[0].blk && !bus->ifs[1].blk) ||
diff --git a/hw/ide/macio.c b/hw/ide/macio.c
index f08318cf97..1c15c37ec5 100644
--- a/hw/ide/macio.c
+++ b/hw/ide/macio.c
@@ -267,7 +267,9 @@ static uint64_t pmac_ide_read(void *opaque, hwaddr addr, unsigned size)
 
     switch (reg) {
     case 0x0:
-        if (size == 2) {
+        if (size == 1) {
+            retval = ide_data_readw(&d->bus, 0) & 0xFF;
+        } else if (size == 2) {
             retval = ide_data_readw(&d->bus, 0);
         } else if (size == 4) {
             retval = ide_data_readl(&d->bus, 0);

From 7110fe56c1f69b9e1bdf838558fdcb75d1568964 Mon Sep 17 00:00:00 2001
From: Vitaly Kuznetsov <vkuznets@redhat.com>
Date: Wed, 25 May 2022 13:59:44 +0200
Subject: [PATCH 516/935] i386: Use hv_build_cpuid_leaf() for
 HV_CPUID_NESTED_FEATURES

Previously, HV_CPUID_NESTED_FEATURES.EAX CPUID leaf was handled differently
as it was only used to encode the supported eVMCS version range. In fact,
there are also feature (e.g. Enlightened MSR-Bitmap) bits there. In
preparation to adding these features, move HV_CPUID_NESTED_FEATURES leaf
handling to hv_build_cpuid_leaf() and drop now-unneeded 'hyperv_nested'.

No functional change intended.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20220525115949.1294004-2-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/cpu.h     |  1 -
 target/i386/kvm/kvm.c | 25 +++++++++++++++----------
 2 files changed, 15 insertions(+), 11 deletions(-)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 0d528ac58f..2e918daf6b 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1804,7 +1804,6 @@ struct ArchCPU {
     uint32_t hyperv_vendor_id[3];
     uint32_t hyperv_interface_id[4];
     uint32_t hyperv_limits[3];
-    uint32_t hyperv_nested[4];
     bool hyperv_enforce_cpuid;
     uint32_t hyperv_ver_id_build;
     uint16_t hyperv_ver_id_major;
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index e2d675115b..38af0e4f04 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -831,6 +831,8 @@ static bool tsc_is_stable_and_known(CPUX86State *env)
         || env->user_tsc_khz;
 }
 
+#define DEFAULT_EVMCS_VERSION ((1 << 8) | 1)
+
 static struct {
     const char *desc;
     struct {
@@ -1254,6 +1256,13 @@ static uint32_t hv_build_cpuid_leaf(CPUState *cs, uint32_t func, int reg)
         }
     }
 
+    /* HV_CPUID_NESTED_FEATURES.EAX also encodes the supported eVMCS range */
+    if (func == HV_CPUID_NESTED_FEATURES && reg == R_EAX) {
+        if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS)) {
+            r |= DEFAULT_EVMCS_VERSION;
+        }
+    }
+
     return r;
 }
 
@@ -1384,11 +1393,11 @@ static int hyperv_fill_cpuids(CPUState *cs,
     struct kvm_cpuid_entry2 *c;
     uint32_t signature[3];
     uint32_t cpuid_i = 0, max_cpuid_leaf = 0;
+    uint32_t nested_eax =
+        hv_build_cpuid_leaf(cs, HV_CPUID_NESTED_FEATURES, R_EAX);
 
-    max_cpuid_leaf = HV_CPUID_IMPLEMENT_LIMITS;
-    if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS)) {
-        max_cpuid_leaf = MAX(max_cpuid_leaf, HV_CPUID_NESTED_FEATURES);
-    }
+    max_cpuid_leaf = nested_eax ? HV_CPUID_NESTED_FEATURES :
+        HV_CPUID_IMPLEMENT_LIMITS;
 
     if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNDBG)) {
         max_cpuid_leaf =
@@ -1461,7 +1470,7 @@ static int hyperv_fill_cpuids(CPUState *cs,
     c->ecx = cpu->hyperv_limits[1];
     c->edx = cpu->hyperv_limits[2];
 
-    if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS)) {
+    if (nested_eax) {
         uint32_t function;
 
         /* Create zeroed 0x40000006..0x40000009 leaves */
@@ -1473,7 +1482,7 @@ static int hyperv_fill_cpuids(CPUState *cs,
 
         c = &cpuid_ent[cpuid_i++];
         c->function = HV_CPUID_NESTED_FEATURES;
-        c->eax = cpu->hyperv_nested[0];
+        c->eax = nested_eax;
     }
 
     if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNDBG)) {
@@ -1522,8 +1531,6 @@ static bool evmcs_version_supported(uint16_t evmcs_version,
         (max_version <= max_supported_version);
 }
 
-#define DEFAULT_EVMCS_VERSION ((1 << 8) | 1)
-
 static int hyperv_init_vcpu(X86CPU *cpu)
 {
     CPUState *cs = CPU(cpu);
@@ -1620,8 +1627,6 @@ static int hyperv_init_vcpu(X86CPU *cpu)
                          supported_evmcs_version >> 8);
             return -ENOTSUP;
         }
-
-        cpu->hyperv_nested[0] = evmcs_version;
     }
 
     if (cpu->hyperv_enforce_cpuid) {

From 869840d26c929b99694e31b1a18e83bdea6e97ca Mon Sep 17 00:00:00 2001
From: Vitaly Kuznetsov <vkuznets@redhat.com>
Date: Wed, 25 May 2022 13:59:45 +0200
Subject: [PATCH 517/935] i386: Hyper-V Enlightened MSR bitmap feature

The newly introduced enlightenment allow L0 (KVM) and L1 (Hyper-V)
hypervisors to collaborate to avoid unnecessary updates to L2
MSR-Bitmap upon vmexits.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20220525115949.1294004-3-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/hyperv.txt                | 9 +++++++++
 target/i386/cpu.c              | 2 ++
 target/i386/cpu.h              | 1 +
 target/i386/kvm/hyperv-proto.h | 5 +++++
 target/i386/kvm/kvm.c          | 7 +++++++
 5 files changed, 24 insertions(+)

diff --git a/docs/hyperv.txt b/docs/hyperv.txt
index 33588a0396..5d85569b99 100644
--- a/docs/hyperv.txt
+++ b/docs/hyperv.txt
@@ -239,6 +239,15 @@ This enlightenment requires a VMBus device (-device vmbus-bridge,irq=15)
 and the follow enlightenments to work:
 hv-relaxed,hv_time,hv-vapic,hv-vpindex,hv-synic,hv-runtime,hv-stimer
 
+3.22. hv-emsr-bitmap
+=====================
+The enlightenment is nested specific, it targets Hyper-V on KVM guests. When
+enabled, it allows L0 (KVM) and L1 (Hyper-V) hypervisors to collaborate to
+avoid unnecessary updates to L2 MSR-Bitmap upon vmexits. While the protocol is
+supported for both VMX (Intel) and SVM (AMD), the VMX implementation requires
+Enlightened VMCS ('hv-evmcs') feature to also be enabled.
+
+Recommended: hv-evmcs (Intel)
 
 4. Supplementary features
 =========================
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 385691458f..474e9b582e 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6968,6 +6968,8 @@ static Property x86_cpu_properties[] = {
                       HYPERV_FEAT_STIMER_DIRECT, 0),
     DEFINE_PROP_BIT64("hv-avic", X86CPU, hyperv_features,
                       HYPERV_FEAT_AVIC, 0),
+    DEFINE_PROP_BIT64("hv-emsr-bitmap", X86CPU, hyperv_features,
+                      HYPERV_FEAT_MSR_BITMAP, 0),
     DEFINE_PROP_ON_OFF_AUTO("hv-no-nonarch-coresharing", X86CPU,
                             hyperv_no_nonarch_cs, ON_OFF_AUTO_OFF),
     DEFINE_PROP_BIT64("hv-syndbg", X86CPU, hyperv_features,
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 2e918daf6b..c788285736 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1106,6 +1106,7 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
 #define HYPERV_FEAT_STIMER_DIRECT       14
 #define HYPERV_FEAT_AVIC                15
 #define HYPERV_FEAT_SYNDBG              16
+#define HYPERV_FEAT_MSR_BITMAP          17
 
 #ifndef HYPERV_SPINLOCK_NEVER_NOTIFY
 #define HYPERV_SPINLOCK_NEVER_NOTIFY             0xFFFFFFFF
diff --git a/target/i386/kvm/hyperv-proto.h b/target/i386/kvm/hyperv-proto.h
index e40e59411c..cea18dbc0e 100644
--- a/target/i386/kvm/hyperv-proto.h
+++ b/target/i386/kvm/hyperv-proto.h
@@ -86,6 +86,11 @@
  */
 #define HV_SYNDBG_CAP_ALLOW_KERNEL_DEBUGGING    (1u << 1)
 
+/*
+ * HV_CPUID_NESTED_FEATURES.EAX bits
+ */
+#define HV_NESTED_MSR_BITMAP                (1u << 19)
+
 /*
  * Basic virtualized MSRs
  */
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 38af0e4f04..f389bbedf2 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -973,6 +973,13 @@ static struct {
         .dependencies = BIT(HYPERV_FEAT_SYNIC) | BIT(HYPERV_FEAT_RELAXED)
     },
 #endif
+    [HYPERV_FEAT_MSR_BITMAP] = {
+        .desc = "enlightened MSR-Bitmap (hv-emsr-bitmap)",
+        .flags = {
+            {.func = HV_CPUID_NESTED_FEATURES, .reg = R_EAX,
+             .bits = HV_NESTED_MSR_BITMAP}
+        }
+    },
 };
 
 static struct kvm_cpuid2 *try_get_hv_cpuid(CPUState *cs, int max,

From 9411e8b6faeb1d88d4441c63c5ec072a01b2914e Mon Sep 17 00:00:00 2001
From: Vitaly Kuznetsov <vkuznets@redhat.com>
Date: Wed, 25 May 2022 13:59:46 +0200
Subject: [PATCH 518/935] i386: Hyper-V XMM fast hypercall input feature

Hyper-V specification allows to pass parameters for certain hypercalls
using XMM registers ("XMM Fast Hypercall Input"). When the feature is
in use, it allows for faster hypercalls processing as KVM can avoid
reading guest's memory.

KVM supports the feature since v5.14.

Rename HV_HYPERCALL_{PARAMS_XMM_AVAILABLE -> XMM_INPUT_AVAILABLE} to
comply with KVM.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20220525115949.1294004-4-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/hyperv.txt                | 6 ++++++
 target/i386/cpu.c              | 2 ++
 target/i386/cpu.h              | 1 +
 target/i386/kvm/hyperv-proto.h | 2 +-
 target/i386/kvm/kvm.c          | 7 +++++++
 5 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/docs/hyperv.txt b/docs/hyperv.txt
index 5d85569b99..af1b10c0b3 100644
--- a/docs/hyperv.txt
+++ b/docs/hyperv.txt
@@ -249,6 +249,12 @@ Enlightened VMCS ('hv-evmcs') feature to also be enabled.
 
 Recommended: hv-evmcs (Intel)
 
+3.23. hv-xmm-input
+===================
+Hyper-V specification allows to pass parameters for certain hypercalls using XMM
+registers ("XMM Fast Hypercall Input"). When the feature is in use, it allows
+for faster hypercalls processing as KVM can avoid reading guest's memory.
+
 4. Supplementary features
 =========================
 
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 474e9b582e..63cec0ea68 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6970,6 +6970,8 @@ static Property x86_cpu_properties[] = {
                       HYPERV_FEAT_AVIC, 0),
     DEFINE_PROP_BIT64("hv-emsr-bitmap", X86CPU, hyperv_features,
                       HYPERV_FEAT_MSR_BITMAP, 0),
+    DEFINE_PROP_BIT64("hv-xmm-input", X86CPU, hyperv_features,
+                      HYPERV_FEAT_XMM_INPUT, 0),
     DEFINE_PROP_ON_OFF_AUTO("hv-no-nonarch-coresharing", X86CPU,
                             hyperv_no_nonarch_cs, ON_OFF_AUTO_OFF),
     DEFINE_PROP_BIT64("hv-syndbg", X86CPU, hyperv_features,
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index c788285736..37e9553584 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1107,6 +1107,7 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
 #define HYPERV_FEAT_AVIC                15
 #define HYPERV_FEAT_SYNDBG              16
 #define HYPERV_FEAT_MSR_BITMAP          17
+#define HYPERV_FEAT_XMM_INPUT           18
 
 #ifndef HYPERV_SPINLOCK_NEVER_NOTIFY
 #define HYPERV_SPINLOCK_NEVER_NOTIFY             0xFFFFFFFF
diff --git a/target/i386/kvm/hyperv-proto.h b/target/i386/kvm/hyperv-proto.h
index cea18dbc0e..f5f16474fa 100644
--- a/target/i386/kvm/hyperv-proto.h
+++ b/target/i386/kvm/hyperv-proto.h
@@ -54,7 +54,7 @@
 #define HV_GUEST_DEBUGGING_AVAILABLE            (1u << 1)
 #define HV_PERF_MONITOR_AVAILABLE               (1u << 2)
 #define HV_CPU_DYNAMIC_PARTITIONING_AVAILABLE   (1u << 3)
-#define HV_HYPERCALL_PARAMS_XMM_AVAILABLE       (1u << 4)
+#define HV_HYPERCALL_XMM_INPUT_AVAILABLE        (1u << 4)
 #define HV_GUEST_IDLE_STATE_AVAILABLE           (1u << 5)
 #define HV_FREQUENCY_MSRS_AVAILABLE             (1u << 8)
 #define HV_GUEST_CRASH_MSR_AVAILABLE            (1u << 10)
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index f389bbedf2..7e6f934eda 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -980,6 +980,13 @@ static struct {
              .bits = HV_NESTED_MSR_BITMAP}
         }
     },
+    [HYPERV_FEAT_XMM_INPUT] = {
+        .desc = "XMM fast hypercall input (hv-xmm-input)",
+        .flags = {
+            {.func = HV_CPUID_FEATURES, .reg = R_EDX,
+             .bits = HV_HYPERCALL_XMM_INPUT_AVAILABLE}
+        }
+    },
 };
 
 static struct kvm_cpuid2 *try_get_hv_cpuid(CPUState *cs, int max,

From aa6bb5fad58d049c6ea97448d4caba4499d60634 Mon Sep 17 00:00:00 2001
From: Vitaly Kuznetsov <vkuznets@redhat.com>
Date: Wed, 25 May 2022 13:59:47 +0200
Subject: [PATCH 519/935] i386: Hyper-V Support extended GVA ranges for TLB
 flush hypercalls

KVM kind of supported "extended GVA ranges" (up to 4095 additional GFNs
per hypercall) since the implementation of Hyper-V PV TLB flush feature
(Linux-4.18) as regardless of the request, full TLB flush was always
performed. "Extended GVA ranges for TLB flush hypercalls" feature bit
wasn't exposed then. Now, as KVM gains support for fine-grained TLB
flush handling, exposing this feature starts making sense.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20220525115949.1294004-5-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/hyperv.txt                | 7 +++++++
 target/i386/cpu.c              | 2 ++
 target/i386/cpu.h              | 1 +
 target/i386/kvm/hyperv-proto.h | 1 +
 target/i386/kvm/kvm.c          | 8 ++++++++
 5 files changed, 19 insertions(+)

diff --git a/docs/hyperv.txt b/docs/hyperv.txt
index af1b10c0b3..4b132b1c94 100644
--- a/docs/hyperv.txt
+++ b/docs/hyperv.txt
@@ -255,6 +255,13 @@ Hyper-V specification allows to pass parameters for certain hypercalls using XMM
 registers ("XMM Fast Hypercall Input"). When the feature is in use, it allows
 for faster hypercalls processing as KVM can avoid reading guest's memory.
 
+3.24. hv-tlbflush-ext
+=====================
+Allow for extended GVA ranges to be passed to Hyper-V TLB flush hypercalls
+(HvFlushVirtualAddressList/HvFlushVirtualAddressListEx).
+
+Requires: hv-tlbflush
+
 4. Supplementary features
 =========================
 
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 63cec0ea68..3429a4e455 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6972,6 +6972,8 @@ static Property x86_cpu_properties[] = {
                       HYPERV_FEAT_MSR_BITMAP, 0),
     DEFINE_PROP_BIT64("hv-xmm-input", X86CPU, hyperv_features,
                       HYPERV_FEAT_XMM_INPUT, 0),
+    DEFINE_PROP_BIT64("hv-tlbflush-ext", X86CPU, hyperv_features,
+                      HYPERV_FEAT_TLBFLUSH_EXT, 0),
     DEFINE_PROP_ON_OFF_AUTO("hv-no-nonarch-coresharing", X86CPU,
                             hyperv_no_nonarch_cs, ON_OFF_AUTO_OFF),
     DEFINE_PROP_BIT64("hv-syndbg", X86CPU, hyperv_features,
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 37e9553584..5ff48257e5 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1108,6 +1108,7 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
 #define HYPERV_FEAT_SYNDBG              16
 #define HYPERV_FEAT_MSR_BITMAP          17
 #define HYPERV_FEAT_XMM_INPUT           18
+#define HYPERV_FEAT_TLBFLUSH_EXT        19
 
 #ifndef HYPERV_SPINLOCK_NEVER_NOTIFY
 #define HYPERV_SPINLOCK_NEVER_NOTIFY             0xFFFFFFFF
diff --git a/target/i386/kvm/hyperv-proto.h b/target/i386/kvm/hyperv-proto.h
index f5f16474fa..c7854ed6d3 100644
--- a/target/i386/kvm/hyperv-proto.h
+++ b/target/i386/kvm/hyperv-proto.h
@@ -59,6 +59,7 @@
 #define HV_FREQUENCY_MSRS_AVAILABLE             (1u << 8)
 #define HV_GUEST_CRASH_MSR_AVAILABLE            (1u << 10)
 #define HV_FEATURE_DEBUG_MSRS_AVAILABLE         (1u << 11)
+#define HV_EXT_GVA_RANGES_FLUSH_AVAILABLE       (1u << 14)
 #define HV_STIMER_DIRECT_MODE_AVAILABLE         (1u << 19)
 
 /*
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 7e6f934eda..a11c8e88f6 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -987,6 +987,14 @@ static struct {
              .bits = HV_HYPERCALL_XMM_INPUT_AVAILABLE}
         }
     },
+    [HYPERV_FEAT_TLBFLUSH_EXT] = {
+        .desc = "Extended gva ranges for TLB flush hypercalls (hv-tlbflush-ext)",
+        .flags = {
+            {.func = HV_CPUID_FEATURES, .reg = R_EDX,
+             .bits = HV_EXT_GVA_RANGES_FLUSH_AVAILABLE}
+        },
+        .dependencies = BIT(HYPERV_FEAT_TLBFLUSH)
+    },
 };
 
 static struct kvm_cpuid2 *try_get_hv_cpuid(CPUState *cs, int max,

From 3aae0854b26aff303202c6f9542445f58b2539fe Mon Sep 17 00:00:00 2001
From: Vitaly Kuznetsov <vkuznets@redhat.com>
Date: Wed, 25 May 2022 13:59:48 +0200
Subject: [PATCH 520/935] i386: Hyper-V Direct TLB flush hypercall

Hyper-V TLFS allows for L0 and L1 hypervisors to collaborate on L2's
TLB flush hypercalls handling. With the correct setup, L2's TLB flush
hypercalls can be handled by L0 directly, without the need to exit to
L1.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20220525115949.1294004-6-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/hyperv.txt                | 11 +++++++++++
 target/i386/cpu.c              |  2 ++
 target/i386/cpu.h              |  1 +
 target/i386/kvm/hyperv-proto.h |  1 +
 target/i386/kvm/kvm.c          |  8 ++++++++
 5 files changed, 23 insertions(+)

diff --git a/docs/hyperv.txt b/docs/hyperv.txt
index 4b132b1c94..14a7f449ea 100644
--- a/docs/hyperv.txt
+++ b/docs/hyperv.txt
@@ -262,6 +262,17 @@ Allow for extended GVA ranges to be passed to Hyper-V TLB flush hypercalls
 
 Requires: hv-tlbflush
 
+3.25. hv-tlbflush-direct
+=========================
+The enlightenment is nested specific, it targets Hyper-V on KVM guests. When
+enabled, it allows L0 (KVM) to directly handle TLB flush hypercalls from L2
+guest without the need to exit to L1 (Hyper-V) hypervisor. While the feature is
+supported for both VMX (Intel) and SVM (AMD), the VMX implementation requires
+Enlightened VMCS ('hv-evmcs') feature to also be enabled.
+
+Requires: hv-vapic
+Recommended: hv-evmcs (Intel)
+
 4. Supplementary features
 =========================
 
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 3429a4e455..bb6a5dd498 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6974,6 +6974,8 @@ static Property x86_cpu_properties[] = {
                       HYPERV_FEAT_XMM_INPUT, 0),
     DEFINE_PROP_BIT64("hv-tlbflush-ext", X86CPU, hyperv_features,
                       HYPERV_FEAT_TLBFLUSH_EXT, 0),
+    DEFINE_PROP_BIT64("hv-tlbflush-direct", X86CPU, hyperv_features,
+                      HYPERV_FEAT_TLBFLUSH_DIRECT, 0),
     DEFINE_PROP_ON_OFF_AUTO("hv-no-nonarch-coresharing", X86CPU,
                             hyperv_no_nonarch_cs, ON_OFF_AUTO_OFF),
     DEFINE_PROP_BIT64("hv-syndbg", X86CPU, hyperv_features,
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 5ff48257e5..82004b65b9 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1109,6 +1109,7 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
 #define HYPERV_FEAT_MSR_BITMAP          17
 #define HYPERV_FEAT_XMM_INPUT           18
 #define HYPERV_FEAT_TLBFLUSH_EXT        19
+#define HYPERV_FEAT_TLBFLUSH_DIRECT     20
 
 #ifndef HYPERV_SPINLOCK_NEVER_NOTIFY
 #define HYPERV_SPINLOCK_NEVER_NOTIFY             0xFFFFFFFF
diff --git a/target/i386/kvm/hyperv-proto.h b/target/i386/kvm/hyperv-proto.h
index c7854ed6d3..464fbf09e3 100644
--- a/target/i386/kvm/hyperv-proto.h
+++ b/target/i386/kvm/hyperv-proto.h
@@ -90,6 +90,7 @@
 /*
  * HV_CPUID_NESTED_FEATURES.EAX bits
  */
+#define HV_NESTED_DIRECT_FLUSH              (1u << 17)
 #define HV_NESTED_MSR_BITMAP                (1u << 19)
 
 /*
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index a11c8e88f6..f148a6d52f 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -995,6 +995,14 @@ static struct {
         },
         .dependencies = BIT(HYPERV_FEAT_TLBFLUSH)
     },
+    [HYPERV_FEAT_TLBFLUSH_DIRECT] = {
+        .desc = "direct TLB flush (hv-tlbflush-direct)",
+        .flags = {
+            {.func = HV_CPUID_NESTED_FEATURES, .reg = R_EAX,
+             .bits = HV_NESTED_DIRECT_FLUSH}
+        },
+        .dependencies = BIT(HYPERV_FEAT_VAPIC)
+    },
 };
 
 static struct kvm_cpuid2 *try_get_hv_cpuid(CPUState *cs, int max,

From 9ad6634ec956bcf3558059aae8c6b2b5ee985307 Mon Sep 17 00:00:00 2001
From: Vitaly Kuznetsov <vkuznets@redhat.com>
Date: Wed, 25 May 2022 13:59:49 +0200
Subject: [PATCH 521/935] i386: docs: Convert hyperv.txt to rST

rSTify docs/hyperv.txt and link it from docs/system/target-i386.rst.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Message-Id: <20220525115949.1294004-7-vkuznets@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/hyperv.txt             | 303 ------------------------------------
 docs/system/i386/hyperv.rst | 288 ++++++++++++++++++++++++++++++++++
 docs/system/target-i386.rst |   1 +
 3 files changed, 289 insertions(+), 303 deletions(-)
 delete mode 100644 docs/hyperv.txt
 create mode 100644 docs/system/i386/hyperv.rst

diff --git a/docs/hyperv.txt b/docs/hyperv.txt
deleted file mode 100644
index 14a7f449ea..0000000000
--- a/docs/hyperv.txt
+++ /dev/null
@@ -1,303 +0,0 @@
-Hyper-V Enlightenments
-======================
-
-
-1. Description
-===============
-In some cases when implementing a hardware interface in software is slow, KVM
-implements its own paravirtualized interfaces. This works well for Linux as
-guest support for such features is added simultaneously with the feature itself.
-It may, however, be hard-to-impossible to add support for these interfaces to
-proprietary OSes, namely, Microsoft Windows.
-
-KVM on x86 implements Hyper-V Enlightenments for Windows guests. These features
-make Windows and Hyper-V guests think they're running on top of a Hyper-V
-compatible hypervisor and use Hyper-V specific features.
-
-
-2. Setup
-=========
-No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In
-QEMU, individual enlightenments can be enabled through CPU flags, e.g:
-
-  qemu-system-x86_64 --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ...
-
-Sometimes there are dependencies between enlightenments, QEMU is supposed to
-check that the supplied configuration is sane.
-
-When any set of the Hyper-V enlightenments is enabled, QEMU changes hypervisor
-identification (CPUID 0x40000000..0x4000000A) to Hyper-V. KVM identification
-and features are kept in leaves 0x40000100..0x40000101.
-
-
-3. Existing enlightenments
-===========================
-
-3.1. hv-relaxed
-================
-This feature tells guest OS to disable watchdog timeouts as it is running on a
-hypervisor. It is known that some Windows versions will do this even when they
-see 'hypervisor' CPU flag.
-
-3.2. hv-vapic
-==============
-Provides so-called VP Assist page MSR to guest allowing it to work with APIC
-more efficiently. In particular, this enlightenment allows paravirtualized
-(exit-less) EOI processing.
-
-3.3. hv-spinlocks=xxx
-======================
-Enables paravirtualized spinlocks. The parameter indicates how many times
-spinlock acquisition should be attempted before indicating the situation to the
-hypervisor. A special value 0xffffffff indicates "never notify".
-
-3.4. hv-vpindex
-================
-Provides HV_X64_MSR_VP_INDEX (0x40000002) MSR to the guest which has Virtual
-processor index information. This enlightenment makes sense in conjunction with
-hv-synic, hv-stimer and other enlightenments which require the guest to know its
-Virtual Processor indices (e.g. when VP index needs to be passed in a
-hypercall).
-
-3.5. hv-runtime
-================
-Provides HV_X64_MSR_VP_RUNTIME (0x40000010) MSR to the guest. The MSR keeps the
-virtual processor run time in 100ns units. This gives guest operating system an
-idea of how much time was 'stolen' from it (when the virtual CPU was preempted
-to perform some other work).
-
-3.6. hv-crash
-==============
-Provides HV_X64_MSR_CRASH_P0..HV_X64_MSR_CRASH_P5 (0x40000100..0x40000105) and
-HV_X64_MSR_CRASH_CTL (0x40000105) MSRs to the guest. These MSRs are written to
-by the guest when it crashes, HV_X64_MSR_CRASH_P0..HV_X64_MSR_CRASH_P5 MSRs
-contain additional crash information. This information is outputted in QEMU log
-and through QAPI.
-Note: unlike under genuine Hyper-V, write to HV_X64_MSR_CRASH_CTL causes guest
-to shutdown. This effectively blocks crash dump generation by Windows.
-
-3.7. hv-time
-=============
-Enables two Hyper-V-specific clocksources available to the guest: MSR-based
-Hyper-V clocksource (HV_X64_MSR_TIME_REF_COUNT, 0x40000020) and Reference TSC
-page (enabled via MSR HV_X64_MSR_REFERENCE_TSC, 0x40000021). Both clocksources
-are per-guest, Reference TSC page clocksource allows for exit-less time stamp
-readings. Using this enlightenment leads to significant speedup of all timestamp
-related operations.
-
-3.8. hv-synic
-==============
-Enables Hyper-V Synthetic interrupt controller - an extension of a local APIC.
-When enabled, this enlightenment provides additional communication facilities
-to the guest: SynIC messages and Events. This is a pre-requisite for
-implementing VMBus devices (not yet in QEMU). Additionally, this enlightenment
-is needed to enable Hyper-V synthetic timers. SynIC is controlled through MSRs
-HV_X64_MSR_SCONTROL..HV_X64_MSR_EOM (0x40000080..0x40000084) and
-HV_X64_MSR_SINT0..HV_X64_MSR_SINT15 (0x40000090..0x4000009F)
-
-Requires: hv-vpindex
-
-3.9. hv-stimer
-===============
-Enables Hyper-V synthetic timers. There are four synthetic timers per virtual
-CPU controlled through HV_X64_MSR_STIMER0_CONFIG..HV_X64_MSR_STIMER3_COUNT
-(0x400000B0..0x400000B7) MSRs. These timers can work either in single-shot or
-periodic mode. It is known that certain Windows versions revert to using HPET
-(or even RTC when HPET is unavailable) extensively when this enlightenment is
-not provided; this can lead to significant CPU consumption, even when virtual
-CPU is idle.
-
-Requires: hv-vpindex, hv-synic, hv-time
-
-3.10. hv-tlbflush
-==================
-Enables paravirtualized TLB shoot-down mechanism. On x86 architecture, remote
-TLB flush procedure requires sending IPIs and waiting for other CPUs to perform
-local TLB flush. In virtualized environment some virtual CPUs may not even be
-scheduled at the time of the call and may not require flushing (or, flushing
-may be postponed until the virtual CPU is scheduled). hv-tlbflush enlightenment
-implements TLB shoot-down through hypervisor enabling the optimization.
-
-Requires: hv-vpindex
-
-3.11. hv-ipi
-=============
-Enables paravirtualized IPI send mechanism. HvCallSendSyntheticClusterIpi
-hypercall may target more than 64 virtual CPUs simultaneously, doing the same
-through APIC requires more than one access (and thus exit to the hypervisor).
-
-Requires: hv-vpindex
-
-3.12. hv-vendor-id=xxx
-=======================
-This changes Hyper-V identification in CPUID 0x40000000.EBX-EDX from the default
-"Microsoft Hv". The parameter should be no longer than 12 characters. According
-to the specification, guests shouldn't use this information and it is unknown
-if there is a Windows version which acts differently.
-Note: hv-vendor-id is not an enlightenment and thus doesn't enable Hyper-V
-identification when specified without some other enlightenment.
-
-3.13. hv-reset
-===============
-Provides HV_X64_MSR_RESET (0x40000003) MSR to the guest allowing it to reset
-itself by writing to it. Even when this MSR is enabled, it is not a recommended
-way for Windows to perform system reboot and thus it may not be used.
-
-3.14. hv-frequencies
-============================================
-Provides HV_X64_MSR_TSC_FREQUENCY (0x40000022) and HV_X64_MSR_APIC_FREQUENCY
-(0x40000023) allowing the guest to get its TSC/APIC frequencies without doing
-measurements.
-
-3.15 hv-reenlightenment
-========================
-The enlightenment is nested specific, it targets Hyper-V on KVM guests. When
-enabled, it provides HV_X64_MSR_REENLIGHTENMENT_CONTROL (0x40000106),
-HV_X64_MSR_TSC_EMULATION_CONTROL (0x40000107)and HV_X64_MSR_TSC_EMULATION_STATUS
-(0x40000108) MSRs allowing the guest to get notified when TSC frequency changes
-(only happens on migration) and keep using old frequency (through emulation in
-the hypervisor) until it is ready to switch to the new one. This, in conjunction
-with hv-frequencies, allows Hyper-V on KVM to pass stable clocksource (Reference
-TSC page) to its own guests.
-
-Note, KVM doesn't fully support re-enlightenment notifications and doesn't
-emulate TSC accesses after migration so 'tsc-frequency=' CPU option also has to
-be specified to make migration succeed. The destination host has to either have
-the same TSC frequency or support TSC scaling CPU feature.
-
-Recommended: hv-frequencies
-
-3.16. hv-evmcs
-===============
-The enlightenment is nested specific, it targets Hyper-V on KVM guests. When
-enabled, it provides Enlightened VMCS version 1 feature to the guest. The feature
-implements paravirtualized protocol between L0 (KVM) and L1 (Hyper-V)
-hypervisors making L2 exits to the hypervisor faster. The feature is Intel-only.
-Note: some virtualization features (e.g. Posted Interrupts) are disabled when
-hv-evmcs is enabled. It may make sense to measure your nested workload with and
-without the feature to find out if enabling it is beneficial.
-
-Requires: hv-vapic
-
-3.17. hv-stimer-direct
-=======================
-Hyper-V specification allows synthetic timer operation in two modes: "classic",
-when expiration event is delivered as SynIC message and "direct", when the event
-is delivered via normal interrupt. It is known that nested Hyper-V can only
-use synthetic timers in direct mode and thus 'hv-stimer-direct' needs to be
-enabled.
-
-Requires: hv-vpindex, hv-synic, hv-time, hv-stimer
-
-3.18. hv-avic (hv-apicv)
-=======================
-The enlightenment allows to use Hyper-V SynIC with hardware APICv/AVIC enabled.
-Normally, Hyper-V SynIC disables these hardware feature and suggests the guest
-to use paravirtualized AutoEOI feature.
-Note: enabling this feature on old hardware (without APICv/AVIC support) may
-have negative effect on guest's performance.
-
-3.19. hv-no-nonarch-coresharing=on/off/auto
-===========================================
-This enlightenment tells guest OS that virtual processors will never share a
-physical core unless they are reported as sibling SMT threads. This information
-is required by Windows and Hyper-V guests to properly mitigate SMT related CPU
-vulnerabilities.
-When the option is set to 'auto' QEMU will enable the feature only when KVM
-reports that non-architectural coresharing is impossible, this means that
-hyper-threading is not supported or completely disabled on the host. This
-setting also prevents migration as SMT settings on the destination may differ.
-When the option is set to 'on' QEMU will always enable the feature, regardless
-of host setup. To keep guests secure, this can only be used in conjunction with
-exposing correct vCPU topology and vCPU pinning.
-
-3.20. hv-version-id-{build,major,minor,spack,sbranch,snumber}
-=============================================================
-This changes Hyper-V version identification in CPUID 0x40000002.EAX-EDX from the
-default (WS2016).
-- hv-version-id-build sets 'Build Number' (32 bits)
-- hv-version-id-major sets 'Major Version' (16 bits)
-- hv-version-id-minor sets 'Minor Version' (16 bits)
-- hv-version-id-spack sets 'Service Pack' (32 bits)
-- hv-version-id-sbranch sets 'Service Branch' (8 bits)
-- hv-version-id-snumber sets 'Service Number' (24 bits)
-
-Note: hv-version-id-* are not enlightenments and thus don't enable Hyper-V
-identification when specified without any other enlightenments.
-
-3.21. hv-syndbg
-===============
-Enables Hyper-V synthetic debugger interface, this is a special interface used
-by Windows Kernel debugger to send the packets through, rather than sending
-them via serial/network .
-When enabled, this enlightenment provides additional communication facilities
-to the guest: SynDbg messages.
-This new communication is used by Windows Kernel debugger rather than sending
-packets via serial/network, adding significant performance boost over the other
-comm channels.
-This enlightenment requires a VMBus device (-device vmbus-bridge,irq=15)
-and the follow enlightenments to work:
-hv-relaxed,hv_time,hv-vapic,hv-vpindex,hv-synic,hv-runtime,hv-stimer
-
-3.22. hv-emsr-bitmap
-=====================
-The enlightenment is nested specific, it targets Hyper-V on KVM guests. When
-enabled, it allows L0 (KVM) and L1 (Hyper-V) hypervisors to collaborate to
-avoid unnecessary updates to L2 MSR-Bitmap upon vmexits. While the protocol is
-supported for both VMX (Intel) and SVM (AMD), the VMX implementation requires
-Enlightened VMCS ('hv-evmcs') feature to also be enabled.
-
-Recommended: hv-evmcs (Intel)
-
-3.23. hv-xmm-input
-===================
-Hyper-V specification allows to pass parameters for certain hypercalls using XMM
-registers ("XMM Fast Hypercall Input"). When the feature is in use, it allows
-for faster hypercalls processing as KVM can avoid reading guest's memory.
-
-3.24. hv-tlbflush-ext
-=====================
-Allow for extended GVA ranges to be passed to Hyper-V TLB flush hypercalls
-(HvFlushVirtualAddressList/HvFlushVirtualAddressListEx).
-
-Requires: hv-tlbflush
-
-3.25. hv-tlbflush-direct
-=========================
-The enlightenment is nested specific, it targets Hyper-V on KVM guests. When
-enabled, it allows L0 (KVM) to directly handle TLB flush hypercalls from L2
-guest without the need to exit to L1 (Hyper-V) hypervisor. While the feature is
-supported for both VMX (Intel) and SVM (AMD), the VMX implementation requires
-Enlightened VMCS ('hv-evmcs') feature to also be enabled.
-
-Requires: hv-vapic
-Recommended: hv-evmcs (Intel)
-
-4. Supplementary features
-=========================
-
-4.1. hv-passthrough
-===================
-In some cases (e.g. during development) it may make sense to use QEMU in
-'pass-through' mode and give Windows guests all enlightenments currently
-supported by KVM. This pass-through mode is enabled by "hv-passthrough" CPU
-flag.
-Note: "hv-passthrough" flag only enables enlightenments which are known to QEMU
-(have corresponding "hv-*" flag) and copies "hv-spinlocks="/"hv-vendor-id="
-values from KVM to QEMU. "hv-passthrough" overrides all other "hv-*" settings on
-the command line. Also, enabling this flag effectively prevents migration as the
-list of enabled enlightenments may differ between target and destination hosts.
-
-4.2. hv-enforce-cpuid
-=====================
-By default, KVM allows the guest to use all currently supported Hyper-V
-enlightenments when Hyper-V CPUID interface was exposed, regardless of if
-some features were not announced in guest visible CPUIDs. 'hv-enforce-cpuid'
-feature alters this behavior and only allows the guest to use exposed Hyper-V
-enlightenments.
-
-
-5. Useful links
-================
-Hyper-V Top Level Functional specification and other information:
-https://github.com/MicrosoftDocs/Virtualization-Documentation
diff --git a/docs/system/i386/hyperv.rst b/docs/system/i386/hyperv.rst
new file mode 100644
index 0000000000..2505dc4c86
--- /dev/null
+++ b/docs/system/i386/hyperv.rst
@@ -0,0 +1,288 @@
+Hyper-V Enlightenments
+======================
+
+
+Description
+-----------
+
+In some cases when implementing a hardware interface in software is slow, KVM
+implements its own paravirtualized interfaces. This works well for Linux as
+guest support for such features is added simultaneously with the feature itself.
+It may, however, be hard-to-impossible to add support for these interfaces to
+proprietary OSes, namely, Microsoft Windows.
+
+KVM on x86 implements Hyper-V Enlightenments for Windows guests. These features
+make Windows and Hyper-V guests think they're running on top of a Hyper-V
+compatible hypervisor and use Hyper-V specific features.
+
+
+Setup
+-----
+
+No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In
+QEMU, individual enlightenments can be enabled through CPU flags, e.g:
+
+.. parsed-literal::
+
+  |qemu_system| --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ...
+
+Sometimes there are dependencies between enlightenments, QEMU is supposed to
+check that the supplied configuration is sane.
+
+When any set of the Hyper-V enlightenments is enabled, QEMU changes hypervisor
+identification (CPUID 0x40000000..0x4000000A) to Hyper-V. KVM identification
+and features are kept in leaves 0x40000100..0x40000101.
+
+
+Existing enlightenments
+-----------------------
+
+``hv-relaxed``
+  This feature tells guest OS to disable watchdog timeouts as it is running on a
+  hypervisor. It is known that some Windows versions will do this even when they
+  see 'hypervisor' CPU flag.
+
+``hv-vapic``
+  Provides so-called VP Assist page MSR to guest allowing it to work with APIC
+  more efficiently. In particular, this enlightenment allows paravirtualized
+  (exit-less) EOI processing.
+
+``hv-spinlocks`` = xxx
+  Enables paravirtualized spinlocks. The parameter indicates how many times
+  spinlock acquisition should be attempted before indicating the situation to the
+  hypervisor. A special value 0xffffffff indicates "never notify".
+
+``hv-vpindex``
+  Provides HV_X64_MSR_VP_INDEX (0x40000002) MSR to the guest which has Virtual
+  processor index information. This enlightenment makes sense in conjunction with
+  hv-synic, hv-stimer and other enlightenments which require the guest to know its
+  Virtual Processor indices (e.g. when VP index needs to be passed in a
+  hypercall).
+
+``hv-runtime``
+  Provides HV_X64_MSR_VP_RUNTIME (0x40000010) MSR to the guest. The MSR keeps the
+  virtual processor run time in 100ns units. This gives guest operating system an
+  idea of how much time was 'stolen' from it (when the virtual CPU was preempted
+  to perform some other work).
+
+``hv-crash``
+  Provides HV_X64_MSR_CRASH_P0..HV_X64_MSR_CRASH_P5 (0x40000100..0x40000105) and
+  HV_X64_MSR_CRASH_CTL (0x40000105) MSRs to the guest. These MSRs are written to
+  by the guest when it crashes, HV_X64_MSR_CRASH_P0..HV_X64_MSR_CRASH_P5 MSRs
+  contain additional crash information. This information is outputted in QEMU log
+  and through QAPI.
+  Note: unlike under genuine Hyper-V, write to HV_X64_MSR_CRASH_CTL causes guest
+  to shutdown. This effectively blocks crash dump generation by Windows.
+
+``hv-time``
+  Enables two Hyper-V-specific clocksources available to the guest: MSR-based
+  Hyper-V clocksource (HV_X64_MSR_TIME_REF_COUNT, 0x40000020) and Reference TSC
+  page (enabled via MSR HV_X64_MSR_REFERENCE_TSC, 0x40000021). Both clocksources
+  are per-guest, Reference TSC page clocksource allows for exit-less time stamp
+  readings. Using this enlightenment leads to significant speedup of all timestamp
+  related operations.
+
+``hv-synic``
+  Enables Hyper-V Synthetic interrupt controller - an extension of a local APIC.
+  When enabled, this enlightenment provides additional communication facilities
+  to the guest: SynIC messages and Events. This is a pre-requisite for
+  implementing VMBus devices (not yet in QEMU). Additionally, this enlightenment
+  is needed to enable Hyper-V synthetic timers. SynIC is controlled through MSRs
+  HV_X64_MSR_SCONTROL..HV_X64_MSR_EOM (0x40000080..0x40000084) and
+  HV_X64_MSR_SINT0..HV_X64_MSR_SINT15 (0x40000090..0x4000009F)
+
+  Requires: ``hv-vpindex``
+
+``hv-stimer``
+  Enables Hyper-V synthetic timers. There are four synthetic timers per virtual
+  CPU controlled through HV_X64_MSR_STIMER0_CONFIG..HV_X64_MSR_STIMER3_COUNT
+  (0x400000B0..0x400000B7) MSRs. These timers can work either in single-shot or
+  periodic mode. It is known that certain Windows versions revert to using HPET
+  (or even RTC when HPET is unavailable) extensively when this enlightenment is
+  not provided; this can lead to significant CPU consumption, even when virtual
+  CPU is idle.
+
+  Requires: ``hv-vpindex``, ``hv-synic``, ``hv-time``
+
+``hv-tlbflush``
+  Enables paravirtualized TLB shoot-down mechanism. On x86 architecture, remote
+  TLB flush procedure requires sending IPIs and waiting for other CPUs to perform
+  local TLB flush. In virtualized environment some virtual CPUs may not even be
+  scheduled at the time of the call and may not require flushing (or, flushing
+  may be postponed until the virtual CPU is scheduled). hv-tlbflush enlightenment
+  implements TLB shoot-down through hypervisor enabling the optimization.
+
+  Requires: ``hv-vpindex``
+
+``hv-ipi``
+  Enables paravirtualized IPI send mechanism. HvCallSendSyntheticClusterIpi
+  hypercall may target more than 64 virtual CPUs simultaneously, doing the same
+  through APIC requires more than one access (and thus exit to the hypervisor).
+
+  Requires: ``hv-vpindex``
+
+``hv-vendor-id`` = xxx
+  This changes Hyper-V identification in CPUID 0x40000000.EBX-EDX from the default
+  "Microsoft Hv". The parameter should be no longer than 12 characters. According
+  to the specification, guests shouldn't use this information and it is unknown
+  if there is a Windows version which acts differently.
+  Note: hv-vendor-id is not an enlightenment and thus doesn't enable Hyper-V
+  identification when specified without some other enlightenment.
+
+``hv-reset``
+  Provides HV_X64_MSR_RESET (0x40000003) MSR to the guest allowing it to reset
+  itself by writing to it. Even when this MSR is enabled, it is not a recommended
+  way for Windows to perform system reboot and thus it may not be used.
+
+``hv-frequencies``
+  Provides HV_X64_MSR_TSC_FREQUENCY (0x40000022) and HV_X64_MSR_APIC_FREQUENCY
+  (0x40000023) allowing the guest to get its TSC/APIC frequencies without doing
+  measurements.
+
+``hv-reenlightenment``
+  The enlightenment is nested specific, it targets Hyper-V on KVM guests. When
+  enabled, it provides HV_X64_MSR_REENLIGHTENMENT_CONTROL (0x40000106),
+  HV_X64_MSR_TSC_EMULATION_CONTROL (0x40000107)and HV_X64_MSR_TSC_EMULATION_STATUS
+  (0x40000108) MSRs allowing the guest to get notified when TSC frequency changes
+  (only happens on migration) and keep using old frequency (through emulation in
+  the hypervisor) until it is ready to switch to the new one. This, in conjunction
+  with ``hv-frequencies``, allows Hyper-V on KVM to pass stable clocksource
+  (Reference TSC page) to its own guests.
+
+  Note, KVM doesn't fully support re-enlightenment notifications and doesn't
+  emulate TSC accesses after migration so 'tsc-frequency=' CPU option also has to
+  be specified to make migration succeed. The destination host has to either have
+  the same TSC frequency or support TSC scaling CPU feature.
+
+  Recommended: ``hv-frequencies``
+
+``hv-evmcs``
+  The enlightenment is nested specific, it targets Hyper-V on KVM guests. When
+  enabled, it provides Enlightened VMCS version 1 feature to the guest. The feature
+  implements paravirtualized protocol between L0 (KVM) and L1 (Hyper-V)
+  hypervisors making L2 exits to the hypervisor faster. The feature is Intel-only.
+
+  Note: some virtualization features (e.g. Posted Interrupts) are disabled when
+  hv-evmcs is enabled. It may make sense to measure your nested workload with and
+  without the feature to find out if enabling it is beneficial.
+
+  Requires: ``hv-vapic``
+
+``hv-stimer-direct``
+  Hyper-V specification allows synthetic timer operation in two modes: "classic",
+  when expiration event is delivered as SynIC message and "direct", when the event
+  is delivered via normal interrupt. It is known that nested Hyper-V can only
+  use synthetic timers in direct mode and thus ``hv-stimer-direct`` needs to be
+  enabled.
+
+  Requires: ``hv-vpindex``, ``hv-synic``, ``hv-time``, ``hv-stimer``
+
+``hv-avic`` (``hv-apicv``)
+  The enlightenment allows to use Hyper-V SynIC with hardware APICv/AVIC enabled.
+  Normally, Hyper-V SynIC disables these hardware feature and suggests the guest
+  to use paravirtualized AutoEOI feature.
+  Note: enabling this feature on old hardware (without APICv/AVIC support) may
+  have negative effect on guest's performance.
+
+``hv-no-nonarch-coresharing`` = on/off/auto
+  This enlightenment tells guest OS that virtual processors will never share a
+  physical core unless they are reported as sibling SMT threads. This information
+  is required by Windows and Hyper-V guests to properly mitigate SMT related CPU
+  vulnerabilities.
+
+  When the option is set to 'auto' QEMU will enable the feature only when KVM
+  reports that non-architectural coresharing is impossible, this means that
+  hyper-threading is not supported or completely disabled on the host. This
+  setting also prevents migration as SMT settings on the destination may differ.
+  When the option is set to 'on' QEMU will always enable the feature, regardless
+  of host setup. To keep guests secure, this can only be used in conjunction with
+  exposing correct vCPU topology and vCPU pinning.
+
+``hv-version-id-build``, ``hv-version-id-major``, ``hv-version-id-minor``, ``hv-version-id-spack``, ``hv-version-id-sbranch``, ``hv-version-id-snumber``
+  This changes Hyper-V version identification in CPUID 0x40000002.EAX-EDX from the
+  default (WS2016).
+
+  - ``hv-version-id-build`` sets 'Build Number' (32 bits)
+  - ``hv-version-id-major`` sets 'Major Version' (16 bits)
+  - ``hv-version-id-minor`` sets 'Minor Version' (16 bits)
+  - ``hv-version-id-spack`` sets 'Service Pack' (32 bits)
+  - ``hv-version-id-sbranch`` sets 'Service Branch' (8 bits)
+  - ``hv-version-id-snumber`` sets 'Service Number' (24 bits)
+
+  Note: hv-version-id-* are not enlightenments and thus don't enable Hyper-V
+  identification when specified without any other enlightenments.
+
+``hv-syndbg``
+  Enables Hyper-V synthetic debugger interface, this is a special interface used
+  by Windows Kernel debugger to send the packets through, rather than sending
+  them via serial/network .
+  When enabled, this enlightenment provides additional communication facilities
+  to the guest: SynDbg messages.
+  This new communication is used by Windows Kernel debugger rather than sending
+  packets via serial/network, adding significant performance boost over the other
+  comm channels.
+  This enlightenment requires a VMBus device (-device vmbus-bridge,irq=15).
+
+  Requires: ``hv-relaxed``, ``hv_time``, ``hv-vapic``, ``hv-vpindex``, ``hv-synic``, ``hv-runtime``, ``hv-stimer``
+
+``hv-emsr-bitmap``
+  The enlightenment is nested specific, it targets Hyper-V on KVM guests. When
+  enabled, it allows L0 (KVM) and L1 (Hyper-V) hypervisors to collaborate to
+  avoid unnecessary updates to L2 MSR-Bitmap upon vmexits. While the protocol is
+  supported for both VMX (Intel) and SVM (AMD), the VMX implementation requires
+  Enlightened VMCS (``hv-evmcs``) feature to also be enabled.
+
+  Recommended: ``hv-evmcs`` (Intel)
+
+``hv-xmm-input``
+  Hyper-V specification allows to pass parameters for certain hypercalls using XMM
+  registers ("XMM Fast Hypercall Input"). When the feature is in use, it allows
+  for faster hypercalls processing as KVM can avoid reading guest's memory.
+
+``hv-tlbflush-ext``
+  Allow for extended GVA ranges to be passed to Hyper-V TLB flush hypercalls
+  (HvFlushVirtualAddressList/HvFlushVirtualAddressListEx).
+
+  Requires: ``hv-tlbflush``
+
+``hv-tlbflush-direct``
+  The enlightenment is nested specific, it targets Hyper-V on KVM guests. When
+  enabled, it allows L0 (KVM) to directly handle TLB flush hypercalls from L2
+  guest without the need to exit to L1 (Hyper-V) hypervisor. While the feature is
+  supported for both VMX (Intel) and SVM (AMD), the VMX implementation requires
+  Enlightened VMCS (``hv-evmcs``) feature to also be enabled.
+
+  Requires: ``hv-vapic``
+
+  Recommended: ``hv-evmcs`` (Intel)
+
+Supplementary features
+----------------------
+
+``hv-passthrough``
+  In some cases (e.g. during development) it may make sense to use QEMU in
+  'pass-through' mode and give Windows guests all enlightenments currently
+  supported by KVM. This pass-through mode is enabled by "hv-passthrough" CPU
+  flag.
+
+  Note: ``hv-passthrough`` flag only enables enlightenments which are known to QEMU
+  (have corresponding 'hv-' flag) and copies ``hv-spinlocks`` and ``hv-vendor-id``
+  values from KVM to QEMU. ``hv-passthrough`` overrides all other 'hv-' settings on
+  the command line. Also, enabling this flag effectively prevents migration as the
+  list of enabled enlightenments may differ between target and destination hosts.
+
+``hv-enforce-cpuid``
+  By default, KVM allows the guest to use all currently supported Hyper-V
+  enlightenments when Hyper-V CPUID interface was exposed, regardless of if
+  some features were not announced in guest visible CPUIDs. ``hv-enforce-cpuid``
+  feature alters this behavior and only allows the guest to use exposed Hyper-V
+  enlightenments.
+
+
+Useful links
+------------
+Hyper-V Top Level Functional specification and other information:
+
+- https://github.com/MicrosoftDocs/Virtualization-Documentation
+- https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/tlfs/tlfs
+
diff --git a/docs/system/target-i386.rst b/docs/system/target-i386.rst
index 96bf54889a..e64c013077 100644
--- a/docs/system/target-i386.rst
+++ b/docs/system/target-i386.rst
@@ -26,6 +26,7 @@ Architectural features
    :maxdepth: 1
 
    i386/cpu
+   i386/hyperv
    i386/kvm-pv
    i386/sgx
    i386/amd-memory-encryption

From daa55f3ecf2d413295eb2d539e7629fc05d7bc92 Mon Sep 17 00:00:00 2001
From: Lei He <helei.sig11@bytedance.com>
Date: Wed, 25 May 2022 17:01:11 +0800
Subject: [PATCH 522/935] qapi: crypto-akcipher: Introduce akcipher types to
 qapi
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Introduce akcipher types, also include RSA related types.

Signed-off-by: Lei He <helei.sig11@bytedance.com>
Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 qapi/crypto.json | 64 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)

diff --git a/qapi/crypto.json b/qapi/crypto.json
index 15c24f0078..653e6e3f3d 100644
--- a/qapi/crypto.json
+++ b/qapi/crypto.json
@@ -534,3 +534,67 @@
   'data': { '*loaded': { 'type': 'bool', 'features': ['deprecated'] },
             '*sanity-check': 'bool',
             '*passwordid': 'str' } }
+##
+# @QCryptoAkCipherAlgorithm:
+#
+# The supported algorithms for asymmetric encryption ciphers
+#
+# @rsa: RSA algorithm
+#
+# Since: 7.1
+##
+{ 'enum': 'QCryptoAkCipherAlgorithm',
+  'prefix': 'QCRYPTO_AKCIPHER_ALG',
+  'data': ['rsa']}
+
+##
+# @QCryptoAkCipherKeyType:
+#
+# The type of asymmetric keys.
+#
+# Since: 7.1
+##
+{ 'enum': 'QCryptoAkCipherKeyType',
+  'prefix': 'QCRYPTO_AKCIPHER_KEY_TYPE',
+  'data': ['public', 'private']}
+
+##
+# @QCryptoRSAPaddingAlgorithm:
+#
+# The padding algorithm for RSA.
+#
+# @raw: no padding used
+# @pkcs1: pkcs1#v1.5
+#
+# Since: 7.1
+##
+{ 'enum': 'QCryptoRSAPaddingAlgorithm',
+  'prefix': 'QCRYPTO_RSA_PADDING_ALG',
+  'data': ['raw', 'pkcs1']}
+
+##
+# @QCryptoAkCipherOptionsRSA:
+#
+# Specific parameters for RSA algorithm.
+#
+# @hash-alg: QCryptoHashAlgorithm
+# @padding-alg: QCryptoRSAPaddingAlgorithm
+#
+# Since: 7.1
+##
+{ 'struct': 'QCryptoAkCipherOptionsRSA',
+  'data': { 'hash-alg':'QCryptoHashAlgorithm',
+            'padding-alg': 'QCryptoRSAPaddingAlgorithm'}}
+
+##
+# @QCryptoAkCipherOptions:
+#
+# The options that are available for all asymmetric key algorithms
+# when creating a new QCryptoAkCipher.
+#
+# Since: 7.1
+##
+{ 'union': 'QCryptoAkCipherOptions',
+  'base': { 'alg': 'QCryptoAkCipherAlgorithm' },
+  'discriminator': 'alg',
+  'data': { 'rsa': 'QCryptoAkCipherOptionsRSA' }}

From db5ca5fbfa6597ac9dd1ae40f986696db9c8b9dd Mon Sep 17 00:00:00 2001
From: zhenwei pi <pizhenwei@bytedance.com>
Date: Wed, 25 May 2022 17:01:12 +0800
Subject: [PATCH 523/935] crypto: Introduce akcipher crypto class
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Introduce new akcipher crypto class 'QCryptoAkCIpher', which supports
basic asymmetric operations: encrypt, decrypt, sign and verify.

Suggested by Daniel P. Berrangé, also add autoptr cleanup for the new
class. Thanks to Daniel!

Co-developed-by: lei he <helei.sig11@bytedance.com>
Signed-off-by: lei he <helei.sig11@bytedance.com>
Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 crypto/akcipher.c         | 102 ++++++++++++++++++++++++
 crypto/akcipherpriv.h     |  55 +++++++++++++
 crypto/meson.build        |   1 +
 include/crypto/akcipher.h | 158 ++++++++++++++++++++++++++++++++++++++
 4 files changed, 316 insertions(+)
 create mode 100644 crypto/akcipher.c
 create mode 100644 crypto/akcipherpriv.h
 create mode 100644 include/crypto/akcipher.h

diff --git a/crypto/akcipher.c b/crypto/akcipher.c
new file mode 100644
index 0000000000..ab28bf415b
--- /dev/null
+++ b/crypto/akcipher.c
@@ -0,0 +1,102 @@
+/*
+ * QEMU Crypto akcipher algorithms
+ *
+ * Copyright (c) 2022 Bytedance
+ * Author: zhenwei pi <pizhenwei@bytedance.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "crypto/akcipher.h"
+#include "akcipherpriv.h"
+
+QCryptoAkCipher *qcrypto_akcipher_new(const QCryptoAkCipherOptions *opts,
+                                      QCryptoAkCipherKeyType type,
+                                      const uint8_t *key, size_t keylen,
+                                      Error **errp)
+{
+    QCryptoAkCipher *akcipher = NULL;
+
+    return akcipher;
+}
+
+bool qcrypto_akcipher_supports(QCryptoAkCipherOptions *opts)
+{
+    return false;
+}
+
+int qcrypto_akcipher_encrypt(QCryptoAkCipher *akcipher,
+                             const void *in, size_t in_len,
+                             void *out, size_t out_len, Error **errp)
+{
+    const QCryptoAkCipherDriver *drv = akcipher->driver;
+
+    return drv->encrypt(akcipher, in, in_len, out, out_len, errp);
+}
+
+int qcrypto_akcipher_decrypt(QCryptoAkCipher *akcipher,
+                             const void *in, size_t in_len,
+                             void *out, size_t out_len, Error **errp)
+{
+    const QCryptoAkCipherDriver *drv = akcipher->driver;
+
+    return drv->decrypt(akcipher, in, in_len, out, out_len, errp);
+}
+
+int qcrypto_akcipher_sign(QCryptoAkCipher *akcipher,
+                          const void *in, size_t in_len,
+                          void *out, size_t out_len, Error **errp)
+{
+    const QCryptoAkCipherDriver *drv = akcipher->driver;
+
+    return drv->sign(akcipher, in, in_len, out, out_len, errp);
+}
+
+int qcrypto_akcipher_verify(QCryptoAkCipher *akcipher,
+                            const void *in, size_t in_len,
+                            const void *in2, size_t in2_len, Error **errp)
+{
+    const QCryptoAkCipherDriver *drv = akcipher->driver;
+
+    return drv->verify(akcipher, in, in_len, in2, in2_len, errp);
+}
+
+int qcrypto_akcipher_max_plaintext_len(QCryptoAkCipher *akcipher)
+{
+    return akcipher->max_plaintext_len;
+}
+
+int qcrypto_akcipher_max_ciphertext_len(QCryptoAkCipher *akcipher)
+{
+    return akcipher->max_ciphertext_len;
+}
+
+int qcrypto_akcipher_max_signature_len(QCryptoAkCipher *akcipher)
+{
+    return akcipher->max_signature_len;
+}
+
+int qcrypto_akcipher_max_dgst_len(QCryptoAkCipher *akcipher)
+{
+    return akcipher->max_dgst_len;
+}
+
+void qcrypto_akcipher_free(QCryptoAkCipher *akcipher)
+{
+    const QCryptoAkCipherDriver *drv = akcipher->driver;
+
+    drv->free(akcipher);
+}
diff --git a/crypto/akcipherpriv.h b/crypto/akcipherpriv.h
new file mode 100644
index 0000000000..739f639bcf
--- /dev/null
+++ b/crypto/akcipherpriv.h
@@ -0,0 +1,55 @@
+/*
+ * QEMU Crypto asymmetric algorithms
+ *
+ * Copyright (c) 2022 Bytedance
+ * Author: zhenwei pi <pizhenwei@bytedance.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QCRYPTO_AKCIPHERPRIV_H
+#define QCRYPTO_AKCIPHERPRIV_H
+
+#include "qapi/qapi-types-crypto.h"
+
+typedef struct QCryptoAkCipherDriver QCryptoAkCipherDriver;
+
+struct QCryptoAkCipher {
+    QCryptoAkCipherAlgorithm alg;
+    QCryptoAkCipherKeyType type;
+    int max_plaintext_len;
+    int max_ciphertext_len;
+    int max_signature_len;
+    int max_dgst_len;
+    QCryptoAkCipherDriver *driver;
+};
+
+struct QCryptoAkCipherDriver {
+    int (*encrypt)(QCryptoAkCipher *akcipher,
+                   const void *in, size_t in_len,
+                   void *out, size_t out_len, Error **errp);
+    int (*decrypt)(QCryptoAkCipher *akcipher,
+                   const void *out, size_t out_len,
+                   void *in, size_t in_len, Error **errp);
+    int (*sign)(QCryptoAkCipher *akcipher,
+                const void *in, size_t in_len,
+                void *out, size_t out_len, Error **errp);
+    int (*verify)(QCryptoAkCipher *akcipher,
+                  const void *in, size_t in_len,
+                  const void *in2, size_t in2_len, Error **errp);
+    void (*free)(QCryptoAkCipher *akcipher);
+};
+
+#endif /* QCRYPTO_AKCIPHER_H */
diff --git a/crypto/meson.build b/crypto/meson.build
index 685fb37097..313f935f27 100644
--- a/crypto/meson.build
+++ b/crypto/meson.build
@@ -1,6 +1,7 @@
 crypto_ss.add(genh)
 crypto_ss.add(files(
   'afsplit.c',
+  'akcipher.c',
   'block-luks.c',
   'block-qcow.c',
   'block.c',
diff --git a/include/crypto/akcipher.h b/include/crypto/akcipher.h
new file mode 100644
index 0000000000..51f5fa2774
--- /dev/null
+++ b/include/crypto/akcipher.h
@@ -0,0 +1,158 @@
+/*
+ * QEMU Crypto asymmetric algorithms
+ *
+ * Copyright (c) 2022 Bytedance
+ * Author: zhenwei pi <pizhenwei@bytedance.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QCRYPTO_AKCIPHER_H
+#define QCRYPTO_AKCIPHER_H
+
+#include "qapi/qapi-types-crypto.h"
+
+typedef struct QCryptoAkCipher QCryptoAkCipher;
+
+/**
+ * qcrypto_akcipher_supports:
+ * @opts: the asymmetric key algorithm and related options
+ *
+ * Determine if asymmetric key cipher decribed with @opts is
+ * supported by the current configured build
+ *
+ * Returns: true if it is supported, false otherwise.
+ */
+bool qcrypto_akcipher_supports(QCryptoAkCipherOptions *opts);
+
+/**
+ * qcrypto_akcipher_new:
+ * @opts: specify the algorithm and the related arguments
+ * @type: private or public key type
+ * @key: buffer to store the key
+ * @key_len: the length of key buffer
+ * @errp: error pointer
+ *
+ * Create akcipher context
+ *
+ * Returns: On success, a new QCryptoAkCipher initialized with @opt
+ * is created and returned, otherwise NULL is returned.
+ */
+
+QCryptoAkCipher *qcrypto_akcipher_new(const QCryptoAkCipherOptions *opts,
+                                      QCryptoAkCipherKeyType type,
+                                      const uint8_t *key, size_t key_len,
+                                      Error **errp);
+
+/**
+ * qcrypto_akcipher_encrypt:
+ * @akcipher: akcipher context
+ * @in: plaintext pending to be encrypted
+ * @in_len: length of plaintext, less or equal to the size reported
+ *          by a call to qcrypto_akcipher_max_plaintext_len()
+ * @out: buffer to store the ciphertext
+ * @out_len: length of ciphertext, less or equal to the size reported
+ *           by a call to qcrypto_akcipher_max_ciphertext_len()
+ * @errp: error pointer
+ *
+ * Encrypt @in and write ciphertext into @out
+ *
+ * Returns: length of ciphertext if encrypt succeed,
+ *          otherwise -1 is returned
+ */
+int qcrypto_akcipher_encrypt(QCryptoAkCipher *akcipher,
+                             const void *in, size_t in_len,
+                             void *out, size_t out_len, Error **errp);
+
+/**
+ * qcrypto_akcipher_decrypt:
+ * @akcipher: akcipher context
+ * @in: ciphertext to be decrypted
+ * @in_len: the length of ciphertext, less or equal to the size reported
+ *          by a call to qcrypto_akcipher_max_ciphertext_len()
+ * @out: buffer to store the plaintext
+ * @out_len: length of the plaintext buffer, less or equal to the size
+ *           reported by a call to qcrypto_akcipher_max_plaintext_len()
+ * @errp: error pointer
+ *
+ * Decrypt @in and write plaintext into @out
+ *
+ * Returns: length of plaintext if decrypt succeed,
+ *          otherwise -1 is returned
+ */
+int qcrypto_akcipher_decrypt(QCryptoAkCipher *akcipher,
+                             const void *in, size_t in_len,
+                             void *out, size_t out_len, Error **errp);
+
+/**
+ * qcrypto_akcipher_sign:
+ * @akcipher: akcipher context
+ * @in: data to be signed
+ * @in_len: the length of data, less or equal to the size reported
+ *          by a call to qcrypto_akcipher_max_dgst_len()
+ * @out: buffer to store the signature
+ * @out_len: length of the signature buffer, less or equal to the size
+ *           by a call to qcrypto_akcipher_max_signature_len()
+ * @errp: error pointer
+ *
+ * Generate signature for @in, write into @out
+ *
+ * Returns: length of signature if succeed,
+ *          otherwise -1 is returned
+ */
+int qcrypto_akcipher_sign(QCryptoAkCipher *akcipher,
+                          const void *in, size_t in_len,
+                          void *out, size_t out_len, Error **errp);
+
+/**
+ * qcrypto_akcipher_verify:
+ * @akcipher: akcipher context
+ * @in: pointer to the signature
+ * @in_len: length of signature, ess or equal to the size reported
+ *          by a call to qcrypto_akcipher_max_signature_len()
+ * @in2: pointer to original data
+ * @in2_len: the length of original data, less or equal to the size
+ *           by a call to qcrypto_akcipher_max_dgst_len()
+ * @errp: error pointer
+ *
+ * Verify @in and @in2 match or not
+ *
+ * Returns: 0 for succeed,
+ *          otherwise -1 is returned
+ */
+int qcrypto_akcipher_verify(QCryptoAkCipher *akcipher,
+                            const void *in, size_t in_len,
+                            const void *in2, size_t in2_len, Error **errp);
+
+int qcrypto_akcipher_max_plaintext_len(QCryptoAkCipher *akcipher);
+
+int qcrypto_akcipher_max_ciphertext_len(QCryptoAkCipher *akcipher);
+
+int qcrypto_akcipher_max_signature_len(QCryptoAkCipher *akcipher);
+
+int qcrypto_akcipher_max_dgst_len(QCryptoAkCipher *akcipher);
+
+/**
+ * qcrypto_akcipher_free:
+ * @akcipher: akcipher context
+ *
+ * Free the akcipher context
+ *
+ */
+void qcrypto_akcipher_free(QCryptoAkCipher *akcipher);
+
+G_DEFINE_AUTOPTR_CLEANUP_FUNC(QCryptoAkCipher, qcrypto_akcipher_free)
+
+#endif /* QCRYPTO_AKCIPHER_H */

From 99d423f10c636c39405924e68584f50f78a0bb8c Mon Sep 17 00:00:00 2001
From: Lei He <helei.sig11@bytedance.com>
Date: Wed, 25 May 2022 17:01:13 +0800
Subject: [PATCH 524/935] crypto: add ASN.1 DER decoder
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add an ANS.1 DER decoder which is used to parse asymmetric
cipher keys

Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
Signed-off-by: lei he <helei.sig11@bytedance.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 crypto/der.c                 | 189 +++++++++++++++++++++++
 crypto/der.h                 |  81 ++++++++++
 crypto/meson.build           |   1 +
 tests/unit/meson.build       |   1 +
 tests/unit/test-crypto-der.c | 290 +++++++++++++++++++++++++++++++++++
 5 files changed, 562 insertions(+)
 create mode 100644 crypto/der.c
 create mode 100644 crypto/der.h
 create mode 100644 tests/unit/test-crypto-der.c

diff --git a/crypto/der.c b/crypto/der.c
new file mode 100644
index 0000000000..f877390bbb
--- /dev/null
+++ b/crypto/der.c
@@ -0,0 +1,189 @@
+/*
+ * QEMU Crypto ASN.1 DER decoder
+ *
+ * Copyright (c) 2022 Bytedance
+ * Author: lei he <helei.sig11@bytedance.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "crypto/der.h"
+
+enum QCryptoDERTypeTag {
+    QCRYPTO_DER_TYPE_TAG_BOOL = 0x1,
+    QCRYPTO_DER_TYPE_TAG_INT = 0x2,
+    QCRYPTO_DER_TYPE_TAG_BIT_STR = 0x3,
+    QCRYPTO_DER_TYPE_TAG_OCT_STR = 0x4,
+    QCRYPTO_DER_TYPE_TAG_OCT_NULL = 0x5,
+    QCRYPTO_DER_TYPE_TAG_OCT_OID = 0x6,
+    QCRYPTO_DER_TYPE_TAG_SEQ = 0x10,
+    QCRYPTO_DER_TYPE_TAG_SET = 0x11,
+};
+
+#define QCRYPTO_DER_CONSTRUCTED_MASK 0x20
+#define QCRYPTO_DER_SHORT_LEN_MASK 0x80
+
+static uint8_t qcrypto_der_peek_byte(const uint8_t **data, size_t *dlen)
+{
+    return **data;
+}
+
+static void qcrypto_der_cut_nbytes(const uint8_t **data,
+                                   size_t *dlen,
+                                   size_t nbytes)
+{
+    *data += nbytes;
+    *dlen -= nbytes;
+}
+
+static uint8_t qcrypto_der_cut_byte(const uint8_t **data, size_t *dlen)
+{
+    uint8_t val = qcrypto_der_peek_byte(data, dlen);
+
+    qcrypto_der_cut_nbytes(data, dlen, 1);
+
+    return val;
+}
+
+static int qcrypto_der_invoke_callback(QCryptoDERDecodeCb cb, void *ctx,
+                                       const uint8_t *value, size_t vlen,
+                                       Error **errp)
+{
+    if (!cb) {
+        return 0;
+    }
+
+    return cb(ctx, value, vlen, errp);
+}
+
+static int qcrypto_der_extract_definite_data(const uint8_t **data, size_t *dlen,
+                                             QCryptoDERDecodeCb cb, void *ctx,
+                                             Error **errp)
+{
+    const uint8_t *value;
+    size_t vlen = 0;
+    uint8_t byte_count = qcrypto_der_cut_byte(data, dlen);
+
+    /* short format of definite-length */
+    if (!(byte_count & QCRYPTO_DER_SHORT_LEN_MASK)) {
+        if (byte_count > *dlen) {
+            error_setg(errp, "Invalid content length: %u", byte_count);
+            return -1;
+        }
+
+        value = *data;
+        vlen = byte_count;
+        qcrypto_der_cut_nbytes(data, dlen, vlen);
+
+        if (qcrypto_der_invoke_callback(cb, ctx, value, vlen, errp) != 0) {
+            return -1;
+        }
+        return vlen;
+    }
+
+    /* Ignore highest bit */
+    byte_count &= ~QCRYPTO_DER_SHORT_LEN_MASK;
+
+    /*
+     * size_t is enough to store the value of length, although the DER
+     * encoding standard supports larger length.
+     */
+    if (byte_count > sizeof(size_t)) {
+        error_setg(errp, "Invalid byte count of content length: %u",
+                   byte_count);
+        return -1;
+    }
+
+    if (byte_count > *dlen) {
+        error_setg(errp, "Invalid content length: %u", byte_count);
+        return -1;
+    }
+    while (byte_count--) {
+        vlen <<= 8;
+        vlen += qcrypto_der_cut_byte(data, dlen);
+    }
+
+    if (vlen > *dlen) {
+        error_setg(errp, "Invalid content length: %zu", vlen);
+        return -1;
+    }
+
+    value = *data;
+    qcrypto_der_cut_nbytes(data, dlen, vlen);
+
+    if (qcrypto_der_invoke_callback(cb, ctx, value, vlen, errp) != 0) {
+        return -1;
+    }
+    return vlen;
+}
+
+static int qcrypto_der_extract_data(const uint8_t **data, size_t *dlen,
+                                    QCryptoDERDecodeCb cb, void *ctx,
+                                    Error **errp)
+{
+    uint8_t val;
+    if (*dlen < 1) {
+        error_setg(errp, "Need more data");
+        return -1;
+    }
+    val = qcrypto_der_peek_byte(data, dlen);
+
+    /* must use definite length format */
+    if (val == QCRYPTO_DER_SHORT_LEN_MASK) {
+        error_setg(errp, "Only definite length format is allowed");
+        return -1;
+    }
+
+    return qcrypto_der_extract_definite_data(data, dlen, cb, ctx, errp);
+}
+
+int qcrypto_der_decode_int(const uint8_t **data, size_t *dlen,
+                           QCryptoDERDecodeCb cb, void *ctx, Error **errp)
+{
+    uint8_t tag;
+    if (*dlen < 1) {
+        error_setg(errp, "Need more data");
+        return -1;
+    }
+    tag = qcrypto_der_cut_byte(data, dlen);
+
+    /* INTEGER must encoded in primitive-form */
+    if (tag != QCRYPTO_DER_TYPE_TAG_INT) {
+        error_setg(errp, "Invalid integer type tag: %u", tag);
+        return -1;
+    }
+
+    return qcrypto_der_extract_data(data, dlen, cb, ctx, errp);
+}
+
+int qcrypto_der_decode_seq(const uint8_t **data, size_t *dlen,
+                           QCryptoDERDecodeCb cb, void *ctx, Error **errp)
+{
+    uint8_t tag;
+    if (*dlen < 1) {
+        error_setg(errp, "Need more data");
+        return -1;
+    }
+    tag = qcrypto_der_cut_byte(data, dlen);
+
+    /* SEQUENCE must use constructed form */
+    if (tag != (QCRYPTO_DER_TYPE_TAG_SEQ | QCRYPTO_DER_CONSTRUCTED_MASK)) {
+        error_setg(errp, "Invalid type sequence tag: %u", tag);
+        return -1;
+    }
+
+    return qcrypto_der_extract_data(data, dlen, cb, ctx, errp);
+}
diff --git a/crypto/der.h b/crypto/der.h
new file mode 100644
index 0000000000..e3d3aeacdc
--- /dev/null
+++ b/crypto/der.h
@@ -0,0 +1,81 @@
+/*
+ * Copyright (c) 2022 Bytedance
+ * Author: lei he <helei.sig11@bytedance.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QCRYPTO_ASN1_DECODER_H
+#define QCRYPTO_ASN1_DECODER_H
+
+#include "qapi/error.h"
+
+/* Simple decoder used to parse DER encoded rsa keys. */
+
+/**
+ *  @opaque: user context.
+ *  @value: the starting address of |value| part of 'Tag-Length-Value' pattern.
+ *  @vlen: length of the |value|.
+ *  Returns: 0 for success, any other value is considered an error.
+ */
+typedef int (*QCryptoDERDecodeCb) (void *opaque, const uint8_t *value,
+                                   size_t vlen, Error **errp);
+
+/**
+ * qcrypto_der_decode_int:
+ * @data: pointer to address of input data
+ * @dlen: pointer to length of input data
+ * @cb: callback invoked when decode succeed, if cb equals NULL, no
+ * callback will be invoked
+ * @opaque: parameter passed to cb
+ *
+ * Decode integer from DER-encoded data.
+ *
+ * Returns: On success, *data points to rest data, and *dlen
+ * will be set to the rest length of data, if cb is not NULL, must
+ * return 0 to make decode success, at last, the length of the data
+ * part of the decoded INTEGER will be returned. Otherwise, -1 is
+ * returned.
+ */
+int qcrypto_der_decode_int(const uint8_t **data,
+                           size_t *dlen,
+                           QCryptoDERDecodeCb cb,
+                           void *opaque,
+                           Error **errp);
+
+/**
+ * qcrypto_der_decode_seq:
+ *
+ * Decode sequence from DER-encoded data, similar with der_decode_int.
+ *
+ * @data: pointer to address of input data
+ * @dlen: pointer to length of input data
+ * @cb: callback invoked when decode succeed, if cb equals NULL, no
+ * callback will be invoked
+ * @opaque: parameter passed to cb
+ *
+ * Returns: On success, *data points to rest data, and *dlen
+ * will be set to the rest length of data, if cb is not NULL, must
+ * return 0 to make decode success, at last, the length of the data
+ * part of the decoded SEQUENCE will be returned. Otherwise, -1 is
+ * returned.
+ */
+int qcrypto_der_decode_seq(const uint8_t **data,
+                           size_t *dlen,
+                           QCryptoDERDecodeCb cb,
+                           void *opaque,
+                           Error **errp);
+
+#endif  /* QCRYPTO_ASN1_DECODER_H */
diff --git a/crypto/meson.build b/crypto/meson.build
index 313f935f27..b8152ae7cb 100644
--- a/crypto/meson.build
+++ b/crypto/meson.build
@@ -6,6 +6,7 @@ crypto_ss.add(files(
   'block-qcow.c',
   'block.c',
   'cipher.c',
+  'der.c',
   'hash.c',
   'hmac.c',
   'ivgen-essiv.c',
diff --git a/tests/unit/meson.build b/tests/unit/meson.build
index 264f2bc0c8..5a7993e56c 100644
--- a/tests/unit/meson.build
+++ b/tests/unit/meson.build
@@ -78,6 +78,7 @@ if have_block
     'test-crypto-hmac': [crypto],
     'test-crypto-cipher': [crypto],
     'test-crypto-secret': [crypto, keyutils],
+    'test-crypto-der': [crypto],
     'test-authz-simple': [authz],
     'test-authz-list': [authz],
     'test-authz-listfile': [authz],
diff --git a/tests/unit/test-crypto-der.c b/tests/unit/test-crypto-der.c
new file mode 100644
index 0000000000..aed0f28d68
--- /dev/null
+++ b/tests/unit/test-crypto-der.c
@@ -0,0 +1,290 @@
+/*
+ * QEMU Crypto akcipher algorithms
+ *
+ * Copyright (c) 2022 Bytedance
+ * Author: lei he <helei@bytedance.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "crypto/der.h"
+
+/* rsa(512) private key, generated by openssl */
+static const uint8_t test_rsa512_priv_key[] =
+    "\x30\x82\x01\x39"      /* SEQUENCE, offset: 0, length: 313 */
+    "\x02\x01\x00"          /* INTEGER, offset: 4, length: 1 */
+    "\x02\x41"              /* INTEGER, offset: 7, length: 65 */
+    "\x00\xb9\xe1\x22\xdb\x56\x2f\xb6\xf7\xf0\x0a\x87\x43\x07\x12\xdb"
+    "\x6d\xb6\x2b\x41\x8d\x2c\x3c\xa5\xdd\x78\x9a\x8f\xab\x8e\xf2\x4a"
+    "\xc8\x34\x0c\x12\x4f\x11\x90\xc6\xc2\xa5\xd0\xcd\xfb\xfc\x2c\x95"
+    "\x56\x82\xdf\x39\xf3\x3b\x1d\x62\x26\x97\xb7\x93\x25\xc7\xec\x7e"
+    "\xf7"
+    "\x02\x03\x01\x00\x01"  /* INTEGER, offset: 74, length: 3 */
+    "\x02\x40"              /* INTEGER, offset: 79, length: 64 */
+    "\x1e\x80\xfe\xda\x65\xdb\x70\xb8\x61\x91\x28\xbf\x6c\x32\xc1\x05"
+    "\xd1\x26\x6a\x1c\x83\xcc\xf4\x1f\x53\x42\x72\x1f\x62\x57\x0a\xc4"
+    "\x66\x76\x30\x87\xb9\xb1\xb9\x6a\x63\xfd\x8f\x3e\xfc\x35\x3f\xd6"
+    "\x2e\x6c\xc8\x70\x8a\x17\xc1\x28\x6a\xfe\x51\x56\xb3\x92\x6f\x09"
+    "\x02\x21"              /* INTEGER, offset: 145, length: 33 */
+    "\x00\xe3\x2e\x2d\x8d\xba\x1c\x34\x4c\x49\x9f\xc1\xa6\xdd\xd7\x13"
+    "\x8d\x05\x48\xdd\xff\x5c\x30\xbc\x6b\xc4\x18\x9d\xfc\xa2\xd0\x9b"
+    "\x4d"
+    "\x02\x21"             /* INTEGER, offset: 180, length: 33 */
+    "\x00\xd1\x75\xaf\x4b\xc6\x1a\xb0\x98\x14\x42\xae\x33\xf3\x44\xde"
+    "\x21\xcb\x04\xda\xfb\x1e\x35\x92\xcd\x69\xc0\x83\x06\x83\x8e\x39"
+    "\x53"
+    "\x02\x20"             /* INTEGER, offset: 215, length: 32 */
+    "\x68\x8d\x2a\xf7\xcb\xcc\x09\x21\x86\xcc\x98\x21\xc4\x7c\xa4\x09"
+    "\xc5\x81\xd8\x71\x1a\x2b\x6f\xbb\xa4\xde\xb3\x6e\xbe\x3b\x85\x0d"
+    "\x02\x20"             /* INTEGER, offset: 249, length: 32 */
+    "\x64\x06\x0e\xef\xe0\x6a\x5e\x6a\x41\x42\x96\x6d\xb8\x7d\xea\x95"
+    "\xb8\x9d\x58\xf5\x12\x38\x03\x22\x94\x9d\x99\xf4\x42\x5e\x68\x81"
+    "\x02\x20"             /* INTEGER, offset: 283, length: 32 */
+    "\x7f\x1d\x87\xe8\x55\x30\x75\xc7\x29\xec\xc9\x65\x76\x5a\x6a\xa3"
+    "\x4a\x6e\xe1\x26\x65\xd1\x76\xd5\xb9\xd1\x8b\xa8\x73\xe2\x6a\x9e";
+
+static const uint8_t test_rsa2048_priv_key[] =
+    "\x30\x82\x04\xa6"          /* SEQUENCE, offset: 0, length 1190 */
+    "\x02\x01\x00"              /* INTEGER, offset: 4, length: 1 */
+    "\x02\x82\x01\x01"          /* INTEGER, offset: 7, length: 257 */
+    "\x00\xd1\x48\xc2\xc1\x1d\x4f\x94\xf2\xbb\x9b\xe2\x2d\xe1\xea\x4c"
+    "\xce\x41\x72\xe3\x41\x7e\x9d\x91\x85\xa3\x4e\xe1\x2c\xf6\x52\x6d"
+    "\xf9\x84\x64\xdf\x87\x28\x4a\xc9\x9d\x78\x93\x47\xc8\xd9\x66\x2e"
+    "\xf4\xc6\xf0\x32\x15\x1a\xe8\xaf\x5a\xca\x3a\xd3\x3e\xf6\xde\x86"
+    "\xdd\x9b\xa6\x4d\x74\x58\xf0\x11\x7f\x66\xd5\x1c\xd8\xde\xa3\xf8"
+    "\xa3\xfc\x33\x55\x89\xa9\xc3\xea\x5b\x2e\x31\x06\xf8\xcb\x9e\x6e"
+    "\xb2\x68\x0d\xe6\xc3\x5c\x2d\xf8\xa2\xbd\x00\x1a\xf6\xb6\xdd\x14"
+    "\x8d\x11\x6d\x2d\xc6\x0c\x09\xe6\xf6\xb9\x8b\x87\x4c\x9f\x4d\x63"
+    "\xd3\x94\xf4\x32\xca\xcf\x5e\xbf\xe2\x7f\x73\x5a\x65\xec\x82\x0d"
+    "\x7f\x30\x25\x03\xd4\x3a\xff\xa2\xe8\xd6\xb5\x1f\x4f\x36\x64\x61"
+    "\xc3\x5f\xb2\x9e\x0c\x53\x04\x19\x34\x99\xe8\xe3\xe6\xd3\x2f\x45"
+    "\x58\x8e\x5d\x54\x5a\xa0\xc0\x5e\x51\x9b\x22\x15\xec\x26\x6f\x72"
+    "\x68\xe9\xbf\x5d\x1d\xb5\xd9\xe4\x81\x1a\x92\x66\xa8\xcb\x73\x46"
+    "\xab\x96\x7b\xf8\x9c\xf5\xb5\x9e\x2b\x13\x71\xe0\x01\x0c\x59\x1b"
+    "\x63\x9f\xb7\xd1\xcd\x47\x8e\xc7\x3a\xbe\xcb\x47\xa7\x23\x43\xa7"
+    "\x7d\xbd\x2c\x4e\x22\x37\xcc\xf9\x1b\x1b\xbb\xed\xec\xf0\x47\x92"
+    "\x43"
+    "\x02\x03\x01\x00\x01"      /* INTEGER, offset 268, length 3 */
+    "\x02\x82\x01\x01"          /* INTEGER, offset 273, length 257 */
+    "\x00\x8d\x21\x97\x0c\x29\x9a\xf8\x23\xf4\x76\x3b\xc1\x9b\x3e\xa8"
+    "\x8a\xd2\xc2\x0a\x14\xa9\xb0\xd2\x68\x9f\x67\x5b\x1c\x3a\x03\xfe"
+    "\x5b\xac\x77\x65\xf1\xbc\x2f\x2a\xe5\x01\x61\xb8\x9f\xee\x53\x25"
+    "\x49\x36\x3a\xd6\x5b\x3b\x29\x3c\xcf\x69\xde\xdf\x83\xef\x70\xc2"
+    "\xdc\x00\xd1\xd6\x1b\xa6\xba\x45\xe2\x77\x53\x31\xbf\xe1\xec\x0b"
+    "\x89\x72\x52\x9f\xd5\x54\xe1\x64\x52\x16\xc5\x43\x21\x56\x16\xc2"
+    "\x29\x97\x58\x00\x8d\x2f\xc5\x64\x8d\x42\x0d\x27\x21\xc6\xd1\x31"
+    "\xc1\xab\xc5\xc7\x7f\x6d\xb0\xe3\xca\xef\xf6\xf2\xc7\xae\x09\xbf"
+    "\x4d\xc0\x4e\x90\x2c\x28\xb9\xcc\x22\x74\xf2\xd5\xff\x4d\x86\xf6"
+    "\xec\x45\x1f\xbf\x25\x4c\x30\x26\x76\x4f\x09\x13\x83\xef\x35\x73"
+    "\xa3\xa2\xb1\x40\xcf\x07\x7a\x83\xae\xea\x00\xea\x74\xc7\x54\x6a"
+    "\x88\x19\xed\x35\xd3\x7e\x5e\xac\x51\xc1\x1e\x5e\x2c\x57\x72\x20"
+    "\x10\x6a\x0c\x47\xe1\xf0\x36\x70\xd2\xa7\x57\x64\x47\x46\x9f\xca"
+    "\x23\x8a\x48\x50\x1d\x33\x6a\x86\x46\x69\xed\x54\x65\x6b\x9e\xab"
+    "\x1f\x84\x87\xf4\x92\x8a\x6c\x44\x20\xaa\x8d\xd8\x50\xde\x45\x74"
+    "\xe0\xa8\xc7\xb9\x38\x74\x24\x51\x33\xf0\x39\x54\x6c\x11\xae\xc2"
+    "\x29"
+    "\x02\x81\x81"              /* INTEGER, offset 534, length 129 */
+    "\x00\xe8\x26\xd1\xf9\xa0\xd3\x0e\x3f\x2f\x89\x9b\x94\x16\x12\xd1"
+    "\xae\x3c\x53\x9c\xcf\xc6\xf7\x03\xf5\xdf\x39\xdc\x25\x5d\xcb\xb8"
+    "\xb9\x74\x3e\x3b\x36\xf6\xa0\x8d\xb1\x0e\xd8\xfe\x8c\xcd\x01\x13"
+    "\x77\x73\x08\x0f\x32\xbd\xe6\x95\xdc\xd0\x14\x7d\x44\xdc\x3e\xd9"
+    "\xaa\x8a\x32\xe6\x0e\x76\xb6\x05\xc5\x6b\x87\x78\x9a\x32\xe2\xf8"
+    "\x78\xba\x58\x75\x58\xd5\x26\x9d\x9a\x0f\xb6\xca\xb5\x27\xd8\x58"
+    "\xae\x3f\x49\x54\xd2\x2b\xac\x28\x39\x88\x31\x42\x12\x08\xea\x0b"
+    "\x39\x58\xae\xf3\x82\xa0\xe2\x75\x7c\x96\xa9\xb8\x57\x29\x6d\xd7"
+    "\x37"
+    "\x02\x81\x81"              /* INTEGER, offset 666, length 129 */
+    "\x00\xe6\xc8\x91\x50\x49\x97\x56\x70\x6e\x25\xf5\x77\x25\xa5\x41"
+    "\xfe\xd7\x25\x1b\xc1\x4a\xff\x37\x44\x2b\x46\xa0\xdf\xe8\x02\x09"
+    "\xdd\xa8\x41\xa1\x12\x84\x3c\xf8\xc2\x13\x3e\xb8\x4b\x22\x01\xac"
+    "\xa6\x09\xb2\xe9\xcd\xc8\x51\xee\xde\xa3\x1e\x6b\xfe\xb1\xf8\xb6"
+    "\x9e\x48\x36\x62\x0b\x05\xfa\x38\xc1\x06\x04\x58\x95\x4d\x25\x13"
+    "\x6d\x0b\x12\x0b\xc9\x6d\x59\xfc\x33\x03\x36\x01\x12\x09\x72\x74"
+    "\x5e\x98\x65\x66\x2f\x3a\xde\xd8\xd4\xee\x6f\x82\xe6\x36\x49\x12"
+    "\x6a\x94\x28\xe9\x28\x9e\xef\x29\xdc\xdf\xab\x94\x65\x02\x4e\x4b"
+    "\x55"
+    "\x02\x81\x81"              /* INTEGER, offset 798, length 129 */
+    "\x00\xc9\xda\xb7\x48\x6e\x66\x15\x45\x2b\x78\x63\x26\x67\xeb\x05"
+    "\x16\x92\xad\xc0\xf3\x88\xf4\xcf\x24\xc2\x6b\xf4\xd7\x28\xaf\x32"
+    "\x77\x4e\x73\xad\xd9\x24\xa8\x85\x8b\x26\x75\xd7\x1f\x66\x41\x41"
+    "\x43\xe3\x69\x66\x8d\xa0\x41\x16\x9d\x60\xef\xef\xdc\x28\x05\x1e"
+    "\x0e\x03\x0c\x2e\xac\xf4\xdb\x60\x39\x40\x3e\x12\xc7\x40\xe7\xc9"
+    "\x54\x6f\xf2\xea\x55\xcb\x40\x40\x58\xec\xc0\xeb\x90\x88\x8c\xbc"
+    "\xcf\x05\x88\x25\x90\x79\x18\xc0\x01\x06\x42\x8e\x48\x50\x27\xf0"
+    "\x8a\x74\x69\xea\xa1\xf2\x71\xf5\xe5\xd6\xba\xcb\xe6\x3d\xc7\x9c"
+    "\x11"
+     "\x02\x81\x81"              /* INTEGER, offset 930, length 129 */
+    "\x00\xc9\xf5\x04\xad\x34\xe9\x39\xdc\x83\x97\xb6\x3a\x40\xf8\x60"
+    "\x4b\x69\xec\xf0\x5f\xf3\x88\x69\xcd\xbe\xed\x3c\xc5\x14\x5c\x0c"
+    "\x54\x2b\xf4\xda\xc6\xc0\x70\x36\xe4\x67\x41\x00\xb7\xc7\x17\x9e"
+    "\x05\x63\x01\x6d\x77\x06\x71\x24\xcf\x32\x01\xe2\x51\xed\x5e\x90"
+    "\x38\xed\x4a\xa1\xfb\xb1\x8c\x69\xf4\x08\x96\xef\x0a\x20\x8b\x6c"
+    "\x77\x85\x33\x92\x9a\xff\x95\xba\x8c\xcd\xa7\x89\xc2\x46\x00\x21"
+    "\xf3\xd1\xfb\x12\x34\x0c\x99\x8d\x38\xb1\x3b\x66\x5a\x9d\x70\xce"
+    "\xab\xf3\xe1\xe5\x40\x05\xed\x97\x3d\xd1\x82\x6e\x07\x02\xc0\x8f"
+    "\x4d"
+    "\x02\x81\x81"              /* INTEGER, offset 1062, length 129 */
+    "\x00\xe4\x96\x79\xa8\x6a\x70\xdd\x67\x42\xff\x15\x11\x9e\x01\x71"
+    "\xac\xf1\x70\x7d\x87\xe2\x6e\x0c\x4d\xbb\x21\x15\xbb\xa7\x4e\x0c"
+    "\x09\x7e\x82\xca\x91\xbe\xd0\xdd\x9c\x8c\xb0\x77\x64\x30\x1b\x7e"
+    "\xbb\x69\xcb\x4c\xde\xd6\x6a\xb9\x72\x15\x79\xdc\x05\x99\x69\x8b"
+    "\x24\xa1\xad\x13\x35\x31\xc0\x0b\xf1\xd2\x06\x7c\x94\x1a\x21\x2f"
+    "\x02\xb9\xf0\xd0\xbb\xf7\xb7\x78\xf9\x3d\x76\x60\xd6\x6b\x5f\x35"
+    "\x88\x14\x33\xe6\xbc\xca\x6b\x88\x90\x57\x3b\x0c\xa3\x6e\x47\xdf"
+    "\x4e\x2f\x4c\xf9\xab\x97\x38\xe4\x20\x32\x32\x96\xc8\x9e\x79\xd3"
+    "\x12";
+
+#define MAX_CHECKER_COUNT 32
+
+typedef struct QCryptoAns1DecoderResultChecker QCryptoAns1DecoderResultChecker;
+struct QCryptoAns1DecoderResultChecker {
+    int (*action) (const uint8_t **data, size_t *dlen,
+                   QCryptoDERDecodeCb cb, void *opaque, Error **errp);
+    QCryptoDERDecodeCb cb;
+    const uint8_t *exp_value;
+    size_t exp_vlen;
+};
+
+typedef struct QCryptoAns1DecoderTestData QCryptoAns1DecoderTestData;
+struct QCryptoAns1DecoderTestData {
+    const char *path;
+    const uint8_t *test_data;
+    size_t test_data_len;
+    QCryptoAns1DecoderResultChecker checker[MAX_CHECKER_COUNT];
+};
+
+typedef struct QCryptoAns1DecoderTestContext QCryptoAns1DecoderTestContext;
+struct QCryptoAns1DecoderTestContext {
+    const uint8_t *data;
+    size_t dlen;
+};
+
+static int checker_callback(void *opaque, const uint8_t *value,
+                            size_t vlen, Error **errp)
+{
+    QCryptoAns1DecoderResultChecker *checker =
+        (QCryptoAns1DecoderResultChecker *)opaque;
+
+    g_assert(value == checker->exp_value);
+    g_assert(vlen == checker->exp_vlen);
+    return 0;
+}
+
+static void test_ans1(const void *opaque)
+{
+    const QCryptoAns1DecoderTestData *test_data =
+        (QCryptoAns1DecoderTestData *)opaque;
+    QCryptoAns1DecoderTestContext ctx[MAX_CHECKER_COUNT];
+    int seq_depth = 0, checker_idx = 0;
+    ctx[seq_depth].data = test_data->test_data;
+    ctx[seq_depth].dlen = test_data->test_data_len;
+    bool all_checker_completed = false;
+
+    do {
+        const QCryptoAns1DecoderResultChecker *checker =
+            &test_data->checker[checker_idx++];
+        QCryptoAns1DecoderTestContext *c = &ctx[seq_depth];
+        if (!checker->action) {
+            all_checker_completed = true;
+            break;
+        }
+        g_assert(checker->action(&c->data, &c->dlen, checker_callback,
+                                 (void *)checker, &error_abort)
+            == checker->exp_vlen);
+        if (checker->action == qcrypto_der_decode_seq) {
+            ++seq_depth;
+            ctx[seq_depth].data = checker->exp_value;
+            ctx[seq_depth].dlen = checker->exp_vlen;
+        }
+        while (seq_depth != 0 && ctx[seq_depth].dlen == 0) {
+            --seq_depth;
+        }
+
+    } while (true);
+    g_assert(seq_depth == 0);
+    g_assert(ctx[seq_depth].dlen == 0);
+    g_assert(all_checker_completed);
+}
+
+static QCryptoAns1DecoderTestData test_data[] = {
+{
+    .path = "/crypto/der/parse-rsa512-priv-key",
+    .test_data = test_rsa512_priv_key,
+    .test_data_len = sizeof(test_rsa512_priv_key) - 1,
+    .checker = {
+        { qcrypto_der_decode_seq, checker_callback,
+          test_rsa512_priv_key + 4, 313 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa512_priv_key + 4 + 2, 1 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa512_priv_key + 7 + 2, 65 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa512_priv_key + 74 + 2, 3 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa512_priv_key + 79 + 2, 64 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa512_priv_key + 145 + 2, 33 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa512_priv_key + 180 + 2, 33 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa512_priv_key + 215 + 2, 32 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa512_priv_key + 249 + 2, 32 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa512_priv_key + 283 + 2, 32 },
+    },
+},
+{
+    .path = "/crypto/der/parse-rsa2048-priv-key",
+    .test_data = test_rsa2048_priv_key,
+    .test_data_len = sizeof(test_rsa2048_priv_key) - 1,
+    .checker = {
+        { qcrypto_der_decode_seq, checker_callback,
+          test_rsa2048_priv_key + 4, 1190 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa2048_priv_key + 4 + 2, 1 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa2048_priv_key + 7 + 4, 257 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa2048_priv_key + 268 + 2, 3 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa2048_priv_key + 273 + 4, 257 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa2048_priv_key + 534 + 3, 129 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa2048_priv_key + 666 + 3, 129 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa2048_priv_key + 798 + 3, 129 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa2048_priv_key + 930 + 3, 129 },
+        { qcrypto_der_decode_int, checker_callback,
+          test_rsa2048_priv_key + 1062 + 3, 129 },
+    },
+},
+
+};
+
+int main(int argc, char **argv)
+{
+    size_t i;
+    g_test_init(&argc, &argv, NULL);
+
+    for (i = 0; i < G_N_ELEMENTS(test_data); i++) {
+        g_test_add_data_func(test_data[i].path, &test_data[i], test_ans1);
+    }
+
+    return g_test_run();
+}

From 4c5e512ee0c49efb42286600aef31739c0dcee5d Mon Sep 17 00:00:00 2001
From: Lei He <helei.sig11@bytedance.com>
Date: Wed, 25 May 2022 17:01:14 +0800
Subject: [PATCH 525/935] crypto: Implement RSA algorithm by hogweed
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Implement RSA algorithm by hogweed from nettle. Thus QEMU supports
a 'real' RSA backend to handle request from guest side. It's
important to test RSA offload case without OS & hardware requirement.

Signed-off-by: lei he <helei.sig11@bytedance.com>
Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 crypto/akcipher-nettle.c.inc | 451 +++++++++++++++++++++++++++++++++++
 crypto/akcipher.c            |   4 +
 crypto/meson.build           |   4 +
 crypto/rsakey-builtin.c.inc  | 200 ++++++++++++++++
 crypto/rsakey-nettle.c.inc   | 158 ++++++++++++
 crypto/rsakey.c              |  44 ++++
 crypto/rsakey.h              |  92 +++++++
 meson.build                  |  11 +
 8 files changed, 964 insertions(+)
 create mode 100644 crypto/akcipher-nettle.c.inc
 create mode 100644 crypto/rsakey-builtin.c.inc
 create mode 100644 crypto/rsakey-nettle.c.inc
 create mode 100644 crypto/rsakey.c
 create mode 100644 crypto/rsakey.h

diff --git a/crypto/akcipher-nettle.c.inc b/crypto/akcipher-nettle.c.inc
new file mode 100644
index 0000000000..a7c0c6a1ee
--- /dev/null
+++ b/crypto/akcipher-nettle.c.inc
@@ -0,0 +1,451 @@
+/*
+ * QEMU Crypto akcipher algorithms
+ *
+ * Copyright (c) 2022 Bytedance
+ * Author: lei he <helei.sig11@bytedance.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include <nettle/rsa.h>
+
+#include "qemu/osdep.h"
+#include "qemu/host-utils.h"
+#include "crypto/akcipher.h"
+#include "crypto/random.h"
+#include "qapi/error.h"
+#include "sysemu/cryptodev.h"
+#include "rsakey.h"
+
+typedef struct QCryptoNettleRSA {
+    QCryptoAkCipher akcipher;
+    struct rsa_public_key pub;
+    struct rsa_private_key priv;
+    QCryptoRSAPaddingAlgorithm padding_alg;
+    QCryptoHashAlgorithm hash_alg;
+} QCryptoNettleRSA;
+
+static void qcrypto_nettle_rsa_free(QCryptoAkCipher *akcipher)
+{
+    QCryptoNettleRSA *rsa = (QCryptoNettleRSA *)akcipher;
+    if (!rsa) {
+        return;
+    }
+
+    rsa_public_key_clear(&rsa->pub);
+    rsa_private_key_clear(&rsa->priv);
+    g_free(rsa);
+}
+
+static QCryptoAkCipher *qcrypto_nettle_rsa_new(
+    const QCryptoAkCipherOptionsRSA *opt,
+    QCryptoAkCipherKeyType type,
+    const uint8_t *key,  size_t keylen,
+    Error **errp);
+
+QCryptoAkCipher *qcrypto_akcipher_new(const QCryptoAkCipherOptions *opts,
+                                      QCryptoAkCipherKeyType type,
+                                      const uint8_t *key, size_t keylen,
+                                      Error **errp)
+{
+    switch (opts->alg) {
+    case QCRYPTO_AKCIPHER_ALG_RSA:
+        return qcrypto_nettle_rsa_new(&opts->u.rsa, type, key, keylen, errp);
+
+    default:
+        error_setg(errp, "Unsupported algorithm: %u", opts->alg);
+        return NULL;
+    }
+
+    return NULL;
+}
+
+static void qcrypto_nettle_rsa_set_akcipher_size(QCryptoAkCipher *akcipher,
+                                                 int key_size)
+{
+    akcipher->max_plaintext_len = key_size;
+    akcipher->max_ciphertext_len = key_size;
+    akcipher->max_signature_len = key_size;
+    akcipher->max_dgst_len = key_size;
+}
+
+static int qcrypt_nettle_parse_rsa_private_key(QCryptoNettleRSA *rsa,
+                                               const uint8_t *key,
+                                               size_t keylen,
+                                               Error **errp)
+{
+    g_autoptr(QCryptoAkCipherRSAKey) rsa_key = qcrypto_akcipher_rsakey_parse(
+        QCRYPTO_AKCIPHER_KEY_TYPE_PRIVATE, key, keylen, errp);
+
+    if (!rsa_key) {
+        return -1;
+    }
+
+    nettle_mpz_init_set_str_256_u(rsa->pub.n, rsa_key->n.len, rsa_key->n.data);
+    nettle_mpz_init_set_str_256_u(rsa->pub.e, rsa_key->e.len, rsa_key->e.data);
+    nettle_mpz_init_set_str_256_u(rsa->priv.d, rsa_key->d.len, rsa_key->d.data);
+    nettle_mpz_init_set_str_256_u(rsa->priv.p, rsa_key->p.len, rsa_key->p.data);
+    nettle_mpz_init_set_str_256_u(rsa->priv.q, rsa_key->q.len, rsa_key->q.data);
+    nettle_mpz_init_set_str_256_u(rsa->priv.a, rsa_key->dp.len,
+                                  rsa_key->dp.data);
+    nettle_mpz_init_set_str_256_u(rsa->priv.b, rsa_key->dq.len,
+                                  rsa_key->dq.data);
+    nettle_mpz_init_set_str_256_u(rsa->priv.c, rsa_key->u.len, rsa_key->u.data);
+
+    if (!rsa_public_key_prepare(&rsa->pub)) {
+        error_setg(errp, "Failed to check RSA key");
+        return -1;
+    }
+
+    /**
+     * Since in the kernel's unit test, the p, q, a, b, c of some
+     * private keys is 0, only the simplest length check is done here
+     */
+    if (rsa_key->p.len > 1 &&
+        rsa_key->q.len > 1 &&
+        rsa_key->dp.len > 1 &&
+        rsa_key->dq.len > 1 &&
+        rsa_key->u.len > 1) {
+        if (!rsa_private_key_prepare(&rsa->priv)) {
+            error_setg(errp, "Failed to check RSA key");
+            return -1;
+        }
+    } else {
+        rsa->priv.size = rsa->pub.size;
+    }
+    qcrypto_nettle_rsa_set_akcipher_size(
+        (QCryptoAkCipher *)rsa, rsa->priv.size);
+
+    return 0;
+}
+
+static int qcrypt_nettle_parse_rsa_public_key(QCryptoNettleRSA *rsa,
+                                              const uint8_t *key,
+                                              size_t keylen,
+                                              Error **errp)
+{
+    g_autoptr(QCryptoAkCipherRSAKey) rsa_key = qcrypto_akcipher_rsakey_parse(
+        QCRYPTO_AKCIPHER_KEY_TYPE_PUBLIC, key, keylen, errp);
+
+    if (!rsa_key) {
+        return -1;
+    }
+    nettle_mpz_init_set_str_256_u(rsa->pub.n, rsa_key->n.len, rsa_key->n.data);
+    nettle_mpz_init_set_str_256_u(rsa->pub.e, rsa_key->e.len, rsa_key->e.data);
+
+    if (!rsa_public_key_prepare(&rsa->pub)) {
+        error_setg(errp, "Failed to check RSA key");
+        return -1;
+    }
+    qcrypto_nettle_rsa_set_akcipher_size(
+        (QCryptoAkCipher *)rsa, rsa->pub.size);
+
+    return 0;
+}
+
+static void wrap_nettle_random_func(void *ctx, size_t len, uint8_t *out)
+{
+    qcrypto_random_bytes(out, len, &error_abort);
+}
+
+static int qcrypto_nettle_rsa_encrypt(QCryptoAkCipher *akcipher,
+                                      const void *data, size_t data_len,
+                                      void *enc, size_t enc_len,
+                                      Error **errp)
+{
+
+    QCryptoNettleRSA *rsa = (QCryptoNettleRSA *)akcipher;
+    mpz_t c;
+    int ret = -1;
+
+    if (data_len > rsa->pub.size) {
+        error_setg(errp, "Plaintext length %zu is greater than key size: %zu"
+                   data_len, rsa->pub.size);
+        return ret;
+    }
+
+    if (enc_len < rsa->pub.size) {
+        error_setg(errp, "Ciphertext buffer length %zu is less than "
+                         "key size: %zu", rsa->pub.size);
+        return ret;
+    }
+
+    /* Nettle do not support RSA encryption without any padding */
+    switch (rsa->padding_alg) {
+    case QCRYPTO_RSA_PADDING_ALG_RAW:
+        error_setg(errp, "RSA with raw padding is not supported");
+        break;
+
+    case QCRYPTO_RSA_PADDING_ALG_PKCS1:
+        mpz_init(c);
+        if (rsa_encrypt(&rsa->pub, NULL, wrap_nettle_random_func,
+                        data_len, (uint8_t *)data, c) != 1) {
+            error_setg(errp, "Failed to encrypt");
+        } else {
+            nettle_mpz_get_str_256(enc_len, (uint8_t *)enc, c);
+            ret = nettle_mpz_sizeinbase_256_u(c);
+        }
+        mpz_clear(c);
+        break;
+
+    default:
+        error_setg(errp, "Unknown padding");
+    }
+
+    return ret;
+}
+
+static int qcrypto_nettle_rsa_decrypt(QCryptoAkCipher *akcipher,
+                                      const void *enc, size_t enc_len,
+                                      void *data, size_t data_len,
+                                      Error **errp)
+{
+    QCryptoNettleRSA *rsa = (QCryptoNettleRSA *)akcipher;
+    mpz_t c;
+    int ret = -1;
+
+    if (enc_len > rsa->priv.size) {
+        error_setg(errp, "Ciphertext length %zu is greater than key size: %zu",
+                   rsa->priv.size);
+        return ret;
+    }
+
+    switch (rsa->padding_alg) {
+    case QCRYPTO_RSA_PADDING_ALG_RAW:
+        error_setg(errp, "RSA with raw padding is not supported");
+        break;
+
+    case QCRYPTO_RSA_PADDING_ALG_PKCS1:
+        nettle_mpz_init_set_str_256_u(c, enc_len, enc);
+        if (!rsa_decrypt(&rsa->priv, &data_len, (uint8_t *)data, c)) {
+            error_setg(errp, "Failed to decrypt");
+        } else {
+            ret = data_len;
+        }
+
+        mpz_clear(c);
+        break;
+
+    default:
+        error_setg(errp, "Unknown padding algorithm: %d", rsa->padding_alg);
+    }
+
+    return ret;
+}
+
+static int qcrypto_nettle_rsa_sign(QCryptoAkCipher *akcipher,
+                                   const void *data, size_t data_len,
+                                   void *sig, size_t sig_len, Error **errp)
+{
+    QCryptoNettleRSA *rsa = (QCryptoNettleRSA *)akcipher;
+    int ret = -1, rv;
+    mpz_t s;
+
+    /**
+     * The RSA algorithm cannot be used for signature/verification
+     * without padding.
+     */
+    if (rsa->padding_alg == QCRYPTO_RSA_PADDING_ALG_RAW) {
+        error_setg(errp, "Try to make signature without padding");
+        return ret;
+    }
+
+    if (data_len > rsa->priv.size) {
+        error_setg(errp, "Data length %zu is greater than key size: %zu",
+                   rsa->priv.size);
+        return ret;
+    }
+
+    if (sig_len < rsa->priv.size) {
+        error_setg(errp, "Signature buffer length %zu is less than "
+                         "key size: %zu", rsa->priv.size);
+        return ret;
+    }
+
+    mpz_init(s);
+    switch (rsa->hash_alg) {
+    case QCRYPTO_HASH_ALG_MD5:
+        rv = rsa_md5_sign_digest(&rsa->priv, data, s);
+        break;
+
+    case QCRYPTO_HASH_ALG_SHA1:
+        rv = rsa_sha1_sign_digest(&rsa->priv, data, s);
+        break;
+
+    case QCRYPTO_HASH_ALG_SHA256:
+        rv = rsa_sha256_sign_digest(&rsa->priv, data, s);
+        break;
+
+    case QCRYPTO_HASH_ALG_SHA512:
+        rv = rsa_sha512_sign_digest(&rsa->priv, data, s);
+        break;
+
+    default:
+        error_setg(errp, "Unknown hash algorithm: %d", rsa->hash_alg);
+        goto cleanup;
+    }
+
+    if (rv != 1) {
+        error_setg(errp, "Failed to make signature");
+        goto cleanup;
+    }
+    nettle_mpz_get_str_256(sig_len, (uint8_t *)sig, s);
+    ret = nettle_mpz_sizeinbase_256_u(s);
+
+cleanup:
+    mpz_clear(s);
+
+    return ret;
+}
+
+static int qcrypto_nettle_rsa_verify(QCryptoAkCipher *akcipher,
+                                     const void *sig, size_t sig_len,
+                                     const void *data, size_t data_len,
+                                     Error **errp)
+{
+    QCryptoNettleRSA *rsa = (QCryptoNettleRSA *)akcipher;
+
+    int ret = -1, rv;
+    mpz_t s;
+
+    /**
+     * The RSA algorithm cannot be used for signature/verification
+     * without padding.
+     */
+    if (rsa->padding_alg == QCRYPTO_RSA_PADDING_ALG_RAW) {
+        error_setg(errp, "Try to verify signature without padding");
+        return ret;
+    }
+    if (data_len > rsa->pub.size) {
+        error_setg(errp, "Data length %zu is greater than key size: %zu",
+                   rsa->pub.size);
+        return ret;
+    }
+    if (sig_len < rsa->pub.size) {
+        error_setg(errp, "Signature length %zu is greater than key size: %zu",
+                   rsa->pub.size);
+        return ret;
+    }
+
+    nettle_mpz_init_set_str_256_u(s, sig_len, sig);
+    switch (rsa->hash_alg) {
+    case QCRYPTO_HASH_ALG_MD5:
+        rv = rsa_md5_verify_digest(&rsa->pub, data, s);
+        break;
+
+    case QCRYPTO_HASH_ALG_SHA1:
+        rv = rsa_sha1_verify_digest(&rsa->pub, data, s);
+        break;
+
+    case QCRYPTO_HASH_ALG_SHA256:
+        rv = rsa_sha256_verify_digest(&rsa->pub, data, s);
+        break;
+
+    case QCRYPTO_HASH_ALG_SHA512:
+        rv = rsa_sha512_verify_digest(&rsa->pub, data, s);
+        break;
+
+    default:
+        error_setg(errp, "Unsupported hash algorithm: %d", rsa->hash_alg);
+        goto cleanup;
+    }
+
+    if (rv != 1) {
+        error_setg(errp, "Failed to verify signature");
+        goto cleanup;
+    }
+    ret = 0;
+
+cleanup:
+    mpz_clear(s);
+
+    return ret;
+}
+
+QCryptoAkCipherDriver nettle_rsa = {
+    .encrypt = qcrypto_nettle_rsa_encrypt,
+    .decrypt = qcrypto_nettle_rsa_decrypt,
+    .sign = qcrypto_nettle_rsa_sign,
+    .verify = qcrypto_nettle_rsa_verify,
+    .free = qcrypto_nettle_rsa_free,
+};
+
+static QCryptoAkCipher *qcrypto_nettle_rsa_new(
+    const QCryptoAkCipherOptionsRSA *opt,
+    QCryptoAkCipherKeyType type,
+    const uint8_t *key, size_t keylen,
+    Error **errp)
+{
+    QCryptoNettleRSA *rsa = g_new0(QCryptoNettleRSA, 1);
+
+    rsa->padding_alg = opt->padding_alg;
+    rsa->hash_alg = opt->hash_alg;
+    rsa->akcipher.driver = &nettle_rsa;
+    rsa_public_key_init(&rsa->pub);
+    rsa_private_key_init(&rsa->priv);
+
+    switch (type) {
+    case QCRYPTO_AKCIPHER_KEY_TYPE_PRIVATE:
+        if (qcrypt_nettle_parse_rsa_private_key(rsa, key, keylen, errp) != 0) {
+            goto error;
+        }
+        break;
+
+    case QCRYPTO_AKCIPHER_KEY_TYPE_PUBLIC:
+        if (qcrypt_nettle_parse_rsa_public_key(rsa, key, keylen, errp) != 0) {
+            goto error;
+        }
+        break;
+
+    default:
+        error_setg(errp, "Unknown akcipher key type %d", type);
+        goto error;
+    }
+
+    return (QCryptoAkCipher *)rsa;
+
+error:
+    qcrypto_nettle_rsa_free((QCryptoAkCipher *)rsa);
+    return NULL;
+}
+
+
+bool qcrypto_akcipher_supports(QCryptoAkCipherOptions *opts)
+{
+    switch (opts->alg) {
+    case QCRYPTO_AKCIPHER_ALG_RSA:
+        switch (opts->u.rsa.padding_alg) {
+        case QCRYPTO_RSA_PADDING_ALG_PKCS1:
+            switch (opts->u.rsa.hash_alg) {
+            case QCRYPTO_HASH_ALG_MD5:
+            case QCRYPTO_HASH_ALG_SHA1:
+            case QCRYPTO_HASH_ALG_SHA256:
+            case QCRYPTO_HASH_ALG_SHA512:
+                return true;
+
+            default:
+                return false;
+            }
+
+        case QCRYPTO_RSA_PADDING_ALG_RAW:
+        default:
+            return false;
+        }
+        break;
+
+    default:
+        return false;
+    }
+}
diff --git a/crypto/akcipher.c b/crypto/akcipher.c
index ab28bf415b..f287083f92 100644
--- a/crypto/akcipher.c
+++ b/crypto/akcipher.c
@@ -23,6 +23,9 @@
 #include "crypto/akcipher.h"
 #include "akcipherpriv.h"
 
+#if defined(CONFIG_NETTLE) && defined(CONFIG_HOGWEED)
+#include "akcipher-nettle.c.inc"
+#else
 QCryptoAkCipher *qcrypto_akcipher_new(const QCryptoAkCipherOptions *opts,
                                       QCryptoAkCipherKeyType type,
                                       const uint8_t *key, size_t keylen,
@@ -37,6 +40,7 @@ bool qcrypto_akcipher_supports(QCryptoAkCipherOptions *opts)
 {
     return false;
 }
+#endif
 
 int qcrypto_akcipher_encrypt(QCryptoAkCipher *akcipher,
                              const void *in, size_t in_len,
diff --git a/crypto/meson.build b/crypto/meson.build
index b8152ae7cb..5f03a30d34 100644
--- a/crypto/meson.build
+++ b/crypto/meson.build
@@ -21,10 +21,14 @@ crypto_ss.add(files(
   'tlscredspsk.c',
   'tlscredsx509.c',
   'tlssession.c',
+  'rsakey.c',
 ))
 
 if nettle.found()
   crypto_ss.add(nettle, files('hash-nettle.c', 'hmac-nettle.c', 'pbkdf-nettle.c'))
+  if hogweed.found()
+    crypto_ss.add(gmp, hogweed)
+  endif
   if xts == 'private'
     crypto_ss.add(files('xts.c'))
   endif
diff --git a/crypto/rsakey-builtin.c.inc b/crypto/rsakey-builtin.c.inc
new file mode 100644
index 0000000000..aeeacc8f9b
--- /dev/null
+++ b/crypto/rsakey-builtin.c.inc
@@ -0,0 +1,200 @@
+/*
+ * QEMU Crypto akcipher algorithms
+ *
+ * Copyright (c) 2022 Bytedance
+ * Author: lei he <helei.sig11@bytedance.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "der.h"
+#include "rsakey.h"
+
+static int extract_mpi(void *ctx, const uint8_t *value,
+                       size_t vlen, Error **errp)
+{
+    QCryptoAkCipherMPI *mpi = (QCryptoAkCipherMPI *)ctx;
+    if (vlen == 0) {
+        error_setg(errp, "Empty mpi field");
+        return -1;
+    }
+    mpi->data = g_memdup2(value, vlen);
+    mpi->len = vlen;
+    return 0;
+}
+
+static int extract_version(void *ctx, const uint8_t *value,
+                           size_t vlen, Error **errp)
+{
+    uint8_t *version = (uint8_t *)ctx;
+    if (vlen != 1 || *value > 1) {
+        error_setg(errp, "Invalid rsakey version");
+        return -1;
+    }
+    *version = *value;
+    return 0;
+}
+
+static int extract_seq_content(void *ctx, const uint8_t *value,
+                               size_t vlen, Error **errp)
+{
+    const uint8_t **content = (const uint8_t **)ctx;
+    if (vlen == 0) {
+        error_setg(errp, "Empty sequence");
+        return -1;
+    }
+    *content = value;
+    return 0;
+}
+
+/**
+ *
+ *        RsaPubKey ::= SEQUENCE {
+ *             n           INTEGER
+ *             e           INTEGER
+ *         }
+ */
+static QCryptoAkCipherRSAKey *qcrypto_builtin_rsa_public_key_parse(
+    const uint8_t *key, size_t keylen, Error **errp)
+{
+    QCryptoAkCipherRSAKey *rsa = g_new0(QCryptoAkCipherRSAKey, 1);
+    const uint8_t *seq;
+    size_t seq_length;
+    int decode_ret;
+
+    decode_ret = qcrypto_der_decode_seq(&key, &keylen,
+                                        extract_seq_content, &seq, errp);
+    if (decode_ret < 0 || keylen != 0) {
+        goto error;
+    }
+    seq_length = decode_ret;
+
+    if (qcrypto_der_decode_int(&seq, &seq_length, extract_mpi,
+                               &rsa->n, errp) < 0 ||
+        qcrypto_der_decode_int(&seq, &seq_length, extract_mpi,
+                               &rsa->e, errp) < 0) {
+        goto error;
+    }
+    if (seq_length != 0) {
+        goto error;
+    }
+
+    return rsa;
+
+error:
+    if (errp && !*errp) {
+        error_setg(errp, "Invalid RSA public key");
+    }
+    qcrypto_akcipher_rsakey_free(rsa);
+    return NULL;
+}
+
+/**
+ *        RsaPrivKey ::= SEQUENCE {
+ *             version     INTEGER
+ *             n           INTEGER
+ *             e           INTEGER
+ *             d           INTEGER
+ *             p           INTEGER
+ *             q           INTEGER
+ *             dp          INTEGER
+ *             dq          INTEGER
+ *             u           INTEGER
+ *       otherPrimeInfos   OtherPrimeInfos OPTIONAL
+ *         }
+ */
+static QCryptoAkCipherRSAKey *qcrypto_builtin_rsa_private_key_parse(
+    const uint8_t *key, size_t keylen, Error **errp)
+{
+    QCryptoAkCipherRSAKey *rsa = g_new0(QCryptoAkCipherRSAKey, 1);
+    uint8_t version;
+    const uint8_t *seq;
+    int decode_ret;
+    size_t seq_length;
+
+    decode_ret = qcrypto_der_decode_seq(&key, &keylen, extract_seq_content,
+                                        &seq, errp);
+    if (decode_ret < 0 || keylen != 0) {
+        goto error;
+    }
+    seq_length = decode_ret;
+
+    decode_ret = qcrypto_der_decode_int(&seq, &seq_length, extract_version,
+                                        &version, errp);
+
+    if (qcrypto_der_decode_int(&seq, &seq_length, extract_mpi,
+                               &rsa->n, errp) < 0 ||
+        qcrypto_der_decode_int(&seq, &seq_length, extract_mpi,
+                               &rsa->e, errp) < 0 ||
+        qcrypto_der_decode_int(&seq, &seq_length, extract_mpi,
+                               &rsa->d, errp) < 0 ||
+        qcrypto_der_decode_int(&seq, &seq_length, extract_mpi, &rsa->p,
+                               errp) < 0 ||
+        qcrypto_der_decode_int(&seq, &seq_length, extract_mpi, &rsa->q,
+                               errp) < 0 ||
+        qcrypto_der_decode_int(&seq, &seq_length, extract_mpi, &rsa->dp,
+                               errp) < 0 ||
+        qcrypto_der_decode_int(&seq, &seq_length, extract_mpi, &rsa->dq,
+                               errp) < 0 ||
+        qcrypto_der_decode_int(&seq, &seq_length, extract_mpi, &rsa->u,
+                               errp) < 0) {
+        goto error;
+    }
+
+    /**
+     * According to the standard, otherPrimeInfos must be present for version 1.
+     * There is no strict verification here, this is to be compatible with
+     * the unit test of the kernel. TODO: remove this until linux kernel's
+     * unit-test is fixed.
+     */
+    if (version == 1 && seq_length != 0) {
+        if (qcrypto_der_decode_seq(&seq, &seq_length, NULL, NULL, errp) < 0) {
+            goto error;
+        }
+        if (seq_length != 0) {
+            goto error;
+        }
+        return rsa;
+    }
+    if (seq_length != 0) {
+        goto error;
+    }
+
+    return rsa;
+
+error:
+    if (errp && !*errp) {
+        error_setg(errp, "Invalid RSA private key");
+    }
+    qcrypto_akcipher_rsakey_free(rsa);
+    return NULL;
+}
+
+QCryptoAkCipherRSAKey *qcrypto_akcipher_rsakey_parse(
+    QCryptoAkCipherKeyType type, const uint8_t *key,
+    size_t keylen, Error **errp)
+{
+    switch (type) {
+    case QCRYPTO_AKCIPHER_KEY_TYPE_PRIVATE:
+        return qcrypto_builtin_rsa_private_key_parse(key, keylen, errp);
+
+    case QCRYPTO_AKCIPHER_KEY_TYPE_PUBLIC:
+        return qcrypto_builtin_rsa_public_key_parse(key, keylen, errp);
+
+    default:
+        error_setg(errp, "Unknown key type: %d", type);
+        return NULL;
+    }
+}
diff --git a/crypto/rsakey-nettle.c.inc b/crypto/rsakey-nettle.c.inc
new file mode 100644
index 0000000000..cc49872e78
--- /dev/null
+++ b/crypto/rsakey-nettle.c.inc
@@ -0,0 +1,158 @@
+/*
+ * QEMU Crypto akcipher algorithms
+ *
+ * Copyright (c) 2022 Bytedance
+ * Author: lei he <helei.sig11@bytedance.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include <nettle/asn1.h>
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "rsakey.h"
+
+static bool DumpMPI(struct asn1_der_iterator *i, QCryptoAkCipherMPI *mpi)
+{
+    mpi->data = g_memdup2(i->data, i->length);
+    mpi->len = i->length;
+    return true;
+}
+
+static bool GetMPI(struct asn1_der_iterator *i, QCryptoAkCipherMPI *mpi)
+{
+    if (asn1_der_iterator_next(i) != ASN1_ITERATOR_PRIMITIVE ||
+        i->type != ASN1_INTEGER) {
+        return false;
+    }
+    return DumpMPI(i, mpi);
+}
+
+/**
+ *        RsaPrivKey ::= SEQUENCE {
+ *             version     INTEGER
+ *             n           INTEGER
+ *             e           INTEGER
+ *             d           INTEGER
+ *             p           INTEGER
+ *             q           INTEGER
+ *             dp          INTEGER
+ *             dq          INTEGER
+ *             u           INTEGER
+ *       otherPrimeInfos   OtherPrimeInfos OPTIONAL
+ *         }
+ */
+static QCryptoAkCipherRSAKey *qcrypto_nettle_rsa_private_key_parse(
+    const uint8_t *key, size_t keylen, Error **errp)
+{
+    QCryptoAkCipherRSAKey *rsa = g_new0(QCryptoAkCipherRSAKey, 1);
+    struct asn1_der_iterator i;
+    uint32_t version;
+    int tag;
+
+    /* Parse entire struct */
+    if (asn1_der_iterator_first(&i, keylen, key) != ASN1_ITERATOR_CONSTRUCTED ||
+        i.type != ASN1_SEQUENCE ||
+        asn1_der_decode_constructed_last(&i) != ASN1_ITERATOR_PRIMITIVE ||
+        i.type != ASN1_INTEGER ||
+        !asn1_der_get_uint32(&i, &version) ||
+        version > 1 ||
+        !GetMPI(&i, &rsa->n) ||
+        !GetMPI(&i, &rsa->e) ||
+        !GetMPI(&i, &rsa->d) ||
+        !GetMPI(&i, &rsa->p) ||
+        !GetMPI(&i, &rsa->q) ||
+        !GetMPI(&i, &rsa->dp) ||
+        !GetMPI(&i, &rsa->dq) ||
+        !GetMPI(&i, &rsa->u)) {
+        goto error;
+    }
+
+    if (version == 1) {
+        tag = asn1_der_iterator_next(&i);
+        /**
+         * According to the standard otherPrimeInfos must be present for
+         * version 1. There is no strict verification here, this is to be
+         * compatible with the unit test of the kernel. TODO: remove this
+         * until linux-kernel's unit-test is fixed;
+         */
+        if (tag == ASN1_ITERATOR_END) {
+            return rsa;
+        }
+        if (tag != ASN1_ITERATOR_CONSTRUCTED ||
+            i.type != ASN1_SEQUENCE) {
+            goto error;
+        }
+    }
+
+    if (asn1_der_iterator_next(&i) != ASN1_ITERATOR_END) {
+        goto error;
+    }
+
+    return rsa;
+
+error:
+    error_setg(errp, "Failed to parse RSA private key");
+    qcrypto_akcipher_rsakey_free(rsa);
+    return NULL;
+}
+
+/**
+ *        RsaPubKey ::= SEQUENCE {
+ *             n           INTEGER
+ *             e           INTEGER
+ *         }
+ */
+static QCryptoAkCipherRSAKey *qcrypto_nettle_rsa_public_key_parse(
+    const uint8_t *key, size_t keylen, Error **errp)
+{
+
+    QCryptoAkCipherRSAKey *rsa = g_new0(QCryptoAkCipherRSAKey, 1);
+    struct asn1_der_iterator i;
+
+    if (asn1_der_iterator_first(&i, keylen, key) != ASN1_ITERATOR_CONSTRUCTED ||
+        i.type != ASN1_SEQUENCE ||
+        asn1_der_decode_constructed_last(&i) != ASN1_ITERATOR_PRIMITIVE ||
+        !DumpMPI(&i, &rsa->n) ||
+        !GetMPI(&i, &rsa->e) ||
+        asn1_der_iterator_next(&i) != ASN1_ITERATOR_END) {
+        goto error;
+    }
+
+    return rsa;
+
+error:
+    error_setg(errp, "Failed to parse RSA public key");
+    qcrypto_akcipher_rsakey_free(rsa);
+    return NULL;
+}
+
+QCryptoAkCipherRSAKey *qcrypto_akcipher_rsakey_parse(
+    QCryptoAkCipherKeyType type, const uint8_t *key,
+    size_t keylen, Error **errp)
+{
+    switch (type) {
+    case QCRYPTO_AKCIPHER_KEY_TYPE_PRIVATE:
+        return qcrypto_nettle_rsa_private_key_parse(key, keylen, errp);
+
+    case QCRYPTO_AKCIPHER_KEY_TYPE_PUBLIC:
+        return qcrypto_nettle_rsa_public_key_parse(key, keylen, errp);
+
+    default:
+        error_setg(errp, "Unknown key type: %d", type);
+        return NULL;
+    }
+}
diff --git a/crypto/rsakey.c b/crypto/rsakey.c
new file mode 100644
index 0000000000..cc40e072f0
--- /dev/null
+++ b/crypto/rsakey.c
@@ -0,0 +1,44 @@
+/*
+ * QEMU Crypto RSA key parser
+ *
+ * Copyright (c) 2022 Bytedance
+ * Author: lei he <helei.sig11@bytedance.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "rsakey.h"
+
+void qcrypto_akcipher_rsakey_free(QCryptoAkCipherRSAKey *rsa_key)
+{
+    if (!rsa_key) {
+        return;
+    }
+    g_free(rsa_key->n.data);
+    g_free(rsa_key->e.data);
+    g_free(rsa_key->d.data);
+    g_free(rsa_key->p.data);
+    g_free(rsa_key->q.data);
+    g_free(rsa_key->dp.data);
+    g_free(rsa_key->dq.data);
+    g_free(rsa_key->u.data);
+    g_free(rsa_key);
+}
+
+#if defined(CONFIG_NETTLE) && defined(CONFIG_HOGWEED)
+#include "rsakey-nettle.c.inc"
+#else
+#include "rsakey-builtin.c.inc"
+#endif
diff --git a/crypto/rsakey.h b/crypto/rsakey.h
new file mode 100644
index 0000000000..974b76f659
--- /dev/null
+++ b/crypto/rsakey.h
@@ -0,0 +1,92 @@
+/*
+ * QEMU Crypto RSA key parser
+ *
+ * Copyright (c) 2022 Bytedance
+ * Author: lei he <helei.sig11@bytedance.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QCRYPTO_RSAKEY_H
+#define QCRYPTO_RSAKEY_H
+
+#include "qemu/osdep.h"
+#include "qemu/host-utils.h"
+#include "crypto/akcipher.h"
+
+typedef struct QCryptoAkCipherRSAKey QCryptoAkCipherRSAKey;
+typedef struct QCryptoAkCipherMPI QCryptoAkCipherMPI;
+
+/**
+ * Multiple precious integer, encoded as two' complement,
+ * copied directly from DER encoded ASN.1 structures.
+ */
+struct QCryptoAkCipherMPI {
+    uint8_t *data;
+    size_t len;
+};
+
+/* See rfc2437: https://datatracker.ietf.org/doc/html/rfc2437 */
+struct QCryptoAkCipherRSAKey {
+    /* The modulus */
+    QCryptoAkCipherMPI n;
+    /* The public exponent */
+    QCryptoAkCipherMPI e;
+    /* The private exponent */
+    QCryptoAkCipherMPI d;
+    /* The first factor */
+    QCryptoAkCipherMPI p;
+    /* The second factor */
+    QCryptoAkCipherMPI q;
+    /* The first factor's exponent */
+    QCryptoAkCipherMPI dp;
+    /* The second factor's exponent */
+    QCryptoAkCipherMPI dq;
+    /* The CRT coefficient */
+    QCryptoAkCipherMPI u;
+};
+
+/**
+ * Parse DER encoded ASN.1 RSA keys, expected ASN.1 schemas:
+ *        RsaPrivKey ::= SEQUENCE {
+ *             version     INTEGER
+ *             n           INTEGER
+ *             e           INTEGER
+ *             d           INTEGER
+ *             p           INTEGER
+ *             q           INTEGER
+ *             dp          INTEGER
+ *             dq          INTEGER
+ *             u           INTEGER
+ *       otherPrimeInfos   OtherPrimeInfos OPTIONAL
+ *         }
+ *
+ *        RsaPubKey ::= SEQUENCE {
+ *             n           INTEGER
+ *             e           INTEGER
+ *         }
+ *
+ * Returns: On success QCryptoAkCipherRSAKey is returned, otherwise returns NULL
+ */
+QCryptoAkCipherRSAKey *qcrypto_akcipher_rsakey_parse(
+    QCryptoAkCipherKeyType type,
+    const uint8_t *key, size_t keylen, Error **errp);
+
+void qcrypto_akcipher_rsakey_free(QCryptoAkCipherRSAKey *key);
+
+G_DEFINE_AUTOPTR_CLEANUP_FUNC(QCryptoAkCipherRSAKey,
+                              qcrypto_akcipher_rsakey_free);
+
+#endif
diff --git a/meson.build b/meson.build
index 9ebc00f032..df7c34b076 100644
--- a/meson.build
+++ b/meson.build
@@ -1121,6 +1121,7 @@ endif
 # gcrypt over nettle for performance reasons.
 gcrypt = not_found
 nettle = not_found
+hogweed = not_found
 xts = 'none'
 
 if get_option('nettle').enabled() and get_option('gcrypt').enabled()
@@ -1158,6 +1159,15 @@ if not gnutls_crypto.found()
   endif
 endif
 
+gmp = dependency('gmp', required: false, method: 'pkg-config', kwargs: static_kwargs)
+if nettle.found() and gmp.found()
+  hogweed = dependency('hogweed', version: '>=3.4',
+                       method: 'pkg-config',
+                       required: get_option('nettle'),
+                       kwargs: static_kwargs)
+endif
+
+
 gtk = not_found
 gtkx11 = not_found
 vte = not_found
@@ -1769,6 +1779,7 @@ config_host_data.set('CONFIG_GNUTLS_CRYPTO', gnutls_crypto.found())
 config_host_data.set('CONFIG_TASN1', tasn1.found())
 config_host_data.set('CONFIG_GCRYPT', gcrypt.found())
 config_host_data.set('CONFIG_NETTLE', nettle.found())
+config_host_data.set('CONFIG_HOGWEED', hogweed.found())
 config_host_data.set('CONFIG_QEMU_PRIVATE_XTS', xts == 'private')
 config_host_data.set('CONFIG_MALLOC_TRIM', has_malloc_trim)
 config_host_data.set('CONFIG_STATX', has_statx)

From e09d1c274782e30dcc0ab5e4d62b624d0028d718 Mon Sep 17 00:00:00 2001
From: Lei He <helei.sig11@bytedance.com>
Date: Wed, 25 May 2022 17:01:15 +0800
Subject: [PATCH 526/935] crypto: Implement RSA algorithm by gcrypt
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Added gcryt implementation of RSA algorithm, RSA algorithm
implemented by gcrypt has a higher priority than nettle because
it supports raw padding.

Signed-off-by: lei he <helei.sig11@bytedance.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 crypto/akcipher-gcrypt.c.inc | 595 +++++++++++++++++++++++++++++++++++
 crypto/akcipher.c            |   4 +-
 2 files changed, 598 insertions(+), 1 deletion(-)
 create mode 100644 crypto/akcipher-gcrypt.c.inc

diff --git a/crypto/akcipher-gcrypt.c.inc b/crypto/akcipher-gcrypt.c.inc
new file mode 100644
index 0000000000..abb1fb272e
--- /dev/null
+++ b/crypto/akcipher-gcrypt.c.inc
@@ -0,0 +1,595 @@
+/*
+ * QEMU Crypto akcipher algorithms
+ *
+ * Copyright (c) 2022 Bytedance
+ * Author: lei he <helei.sig11@bytedance.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include <gcrypt.h>
+
+#include "qemu/osdep.h"
+#include "qemu/host-utils.h"
+#include "crypto/akcipher.h"
+#include "crypto/random.h"
+#include "qapi/error.h"
+#include "sysemu/cryptodev.h"
+#include "rsakey.h"
+
+typedef struct QCryptoGcryptRSA {
+    QCryptoAkCipher akcipher;
+    gcry_sexp_t key;
+    QCryptoRSAPaddingAlgorithm padding_alg;
+    QCryptoHashAlgorithm hash_alg;
+} QCryptoGcryptRSA;
+
+static void qcrypto_gcrypt_rsa_free(QCryptoAkCipher *akcipher)
+{
+    QCryptoGcryptRSA *rsa = (QCryptoGcryptRSA *)akcipher;
+    if (!rsa) {
+        return;
+    }
+
+    gcry_sexp_release(rsa->key);
+    g_free(rsa);
+}
+
+static QCryptoGcryptRSA *qcrypto_gcrypt_rsa_new(
+    const QCryptoAkCipherOptionsRSA *opt,
+    QCryptoAkCipherKeyType type,
+    const uint8_t *key,  size_t keylen,
+    Error **errp);
+
+QCryptoAkCipher *qcrypto_akcipher_new(const QCryptoAkCipherOptions *opts,
+                                      QCryptoAkCipherKeyType type,
+                                      const uint8_t *key, size_t keylen,
+                                      Error **errp)
+{
+    switch (opts->alg) {
+    case QCRYPTO_AKCIPHER_ALG_RSA:
+        return (QCryptoAkCipher *)qcrypto_gcrypt_rsa_new(
+            &opts->u.rsa, type, key, keylen, errp);
+
+    default:
+        error_setg(errp, "Unsupported algorithm: %u", opts->alg);
+        return NULL;
+    }
+
+    return NULL;
+}
+
+static void qcrypto_gcrypt_set_rsa_size(QCryptoAkCipher *akcipher, gcry_mpi_t n)
+{
+    size_t key_size = (gcry_mpi_get_nbits(n) + 7) / 8;
+    akcipher->max_plaintext_len = key_size;
+    akcipher->max_ciphertext_len = key_size;
+    akcipher->max_dgst_len = key_size;
+    akcipher->max_signature_len = key_size;
+}
+
+static int qcrypto_gcrypt_parse_rsa_private_key(
+    QCryptoGcryptRSA *rsa,
+    const uint8_t *key, size_t keylen, Error **errp)
+{
+    g_autoptr(QCryptoAkCipherRSAKey) rsa_key = qcrypto_akcipher_rsakey_parse(
+        QCRYPTO_AKCIPHER_KEY_TYPE_PRIVATE, key, keylen, errp);
+    gcry_mpi_t n = NULL, e = NULL, d = NULL, p = NULL, q = NULL, u = NULL;
+    bool compute_mul_inv = false;
+    int ret = -1;
+    gcry_error_t err;
+
+    if (!rsa_key) {
+        return ret;
+    }
+
+    err = gcry_mpi_scan(&n, GCRYMPI_FMT_STD,
+                        rsa_key->n.data, rsa_key->n.len, NULL);
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to parse RSA parameter n: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+
+    err = gcry_mpi_scan(&e, GCRYMPI_FMT_STD,
+                        rsa_key->e.data, rsa_key->e.len, NULL);
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to parse RSA parameter e: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+
+    err = gcry_mpi_scan(&d, GCRYMPI_FMT_STD,
+                        rsa_key->d.data, rsa_key->d.len, NULL);
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to parse RSA parameter d: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+
+    err = gcry_mpi_scan(&p, GCRYMPI_FMT_STD,
+                        rsa_key->p.data, rsa_key->p.len, NULL);
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to parse RSA parameter p: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+
+    err = gcry_mpi_scan(&q, GCRYMPI_FMT_STD,
+                        rsa_key->q.data, rsa_key->q.len, NULL);
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to parse RSA parameter q: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+
+    if (gcry_mpi_cmp_ui(p, 0) > 0 && gcry_mpi_cmp_ui(q, 0) > 0) {
+        compute_mul_inv = true;
+
+        u = gcry_mpi_new(0);
+        if (gcry_mpi_cmp(p, q) > 0) {
+            gcry_mpi_swap(p, q);
+        }
+        gcry_mpi_invm(u, p, q);
+    }
+
+    if (compute_mul_inv) {
+        err = gcry_sexp_build(&rsa->key, NULL,
+            "(private-key (rsa (n %m) (e %m) (d %m) (p %m) (q %m) (u %m)))",
+            n, e, d, p, q, u);
+    } else {
+        err = gcry_sexp_build(&rsa->key, NULL,
+            "(private-key (rsa (n %m) (e %m) (d %m)))", n, e, d);
+    }
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to build RSA private key: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+    qcrypto_gcrypt_set_rsa_size((QCryptoAkCipher *)rsa,  n);
+    ret = 0;
+
+cleanup:
+    gcry_mpi_release(n);
+    gcry_mpi_release(e);
+    gcry_mpi_release(d);
+    gcry_mpi_release(p);
+    gcry_mpi_release(q);
+    gcry_mpi_release(u);
+    return ret;
+}
+
+static int qcrypto_gcrypt_parse_rsa_public_key(QCryptoGcryptRSA *rsa,
+                                               const uint8_t *key,
+                                               size_t keylen,
+                                               Error **errp)
+{
+
+    g_autoptr(QCryptoAkCipherRSAKey) rsa_key = qcrypto_akcipher_rsakey_parse(
+        QCRYPTO_AKCIPHER_KEY_TYPE_PUBLIC, key, keylen, errp);
+    gcry_mpi_t n = NULL, e = NULL;
+    int ret = -1;
+    gcry_error_t err;
+
+    if (!rsa_key) {
+        return ret;
+    }
+
+    err = gcry_mpi_scan(&n, GCRYMPI_FMT_STD,
+                        rsa_key->n.data, rsa_key->n.len, NULL);
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to parse RSA parameter n: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+
+    err = gcry_mpi_scan(&e, GCRYMPI_FMT_STD,
+                        rsa_key->e.data, rsa_key->e.len, NULL);
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to parse RSA parameter e: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+
+    err = gcry_sexp_build(&rsa->key, NULL,
+                          "(public-key (rsa (n %m) (e %m)))", n, e);
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to build RSA public key: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+    qcrypto_gcrypt_set_rsa_size((QCryptoAkCipher *)rsa, n);
+    ret = 0;
+
+cleanup:
+    gcry_mpi_release(n);
+    gcry_mpi_release(e);
+    return ret;
+}
+
+static int qcrypto_gcrypt_rsa_encrypt(QCryptoAkCipher *akcipher,
+                                      const void *in, size_t in_len,
+                                      void *out, size_t out_len,
+                                      Error **errp)
+{
+    QCryptoGcryptRSA *rsa = (QCryptoGcryptRSA *)akcipher;
+    int ret = -1;
+    gcry_sexp_t data_sexp = NULL, cipher_sexp = NULL;
+    gcry_sexp_t cipher_sexp_item = NULL;
+    gcry_mpi_t cipher_mpi = NULL;
+    const char *result;
+    gcry_error_t err;
+    size_t actual_len;
+
+    if (in_len > akcipher->max_plaintext_len) {
+        error_setg(errp, "Plaintext length is greater than key size: %d",
+                   akcipher->max_plaintext_len);
+        return ret;
+    }
+
+    err = gcry_sexp_build(&data_sexp, NULL,
+                          "(data (flags %s) (value %b))",
+                          QCryptoRSAPaddingAlgorithm_str(rsa->padding_alg),
+                          in_len, in);
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to build plaintext: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+
+    err = gcry_pk_encrypt(&cipher_sexp, data_sexp, rsa->key);
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to encrypt: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+
+    /* S-expression of cipher: (enc-val (rsa (a a-mpi))) */
+    cipher_sexp_item = gcry_sexp_find_token(cipher_sexp, "a", 0);
+    if (!cipher_sexp_item || gcry_sexp_length(cipher_sexp_item) != 2) {
+        error_setg(errp, "Invalid ciphertext result");
+        goto cleanup;
+    }
+
+    if (rsa->padding_alg == QCRYPTO_RSA_PADDING_ALG_RAW) {
+        cipher_mpi = gcry_sexp_nth_mpi(cipher_sexp_item, 1, GCRYMPI_FMT_USG);
+        if (!cipher_mpi) {
+            error_setg(errp, "Invalid ciphertext result");
+            goto cleanup;
+        }
+        err = gcry_mpi_print(GCRYMPI_FMT_USG, out, out_len,
+                             &actual_len, cipher_mpi);
+        if (gcry_err_code(err) != 0) {
+            error_setg(errp, "Failed to print MPI: %s/%s",
+                       gcry_strsource(err), gcry_strerror(err));
+            goto cleanup;
+        }
+
+        if (actual_len > out_len) {
+            error_setg(errp, "Ciphertext buffer length is too small");
+            goto cleanup;
+        }
+
+        /* We always padding leading-zeros for RSA-RAW */
+        if (actual_len < out_len) {
+            memmove((uint8_t *)out + (out_len - actual_len), out, actual_len);
+            memset(out, 0, out_len - actual_len);
+        }
+        ret = out_len;
+
+    } else {
+        result = gcry_sexp_nth_data(cipher_sexp_item, 1, &actual_len);
+        if (!result) {
+            error_setg(errp, "Invalid ciphertext result");
+            goto cleanup;
+        }
+        if (actual_len > out_len) {
+            error_setg(errp, "Ciphertext buffer length is too small");
+            goto cleanup;
+        }
+        memcpy(out, result, actual_len);
+        ret = actual_len;
+    }
+
+cleanup:
+    gcry_sexp_release(data_sexp);
+    gcry_sexp_release(cipher_sexp);
+    gcry_sexp_release(cipher_sexp_item);
+    gcry_mpi_release(cipher_mpi);
+    return ret;
+}
+
+static int qcrypto_gcrypt_rsa_decrypt(QCryptoAkCipher *akcipher,
+                                      const void *in, size_t in_len,
+                                      void *out, size_t out_len,
+                                      Error **errp)
+{
+    QCryptoGcryptRSA *rsa = (QCryptoGcryptRSA *)akcipher;
+    int ret = -1;
+    gcry_sexp_t data_sexp = NULL, cipher_sexp = NULL;
+    gcry_mpi_t data_mpi = NULL;
+    gcry_error_t err;
+    size_t actual_len;
+    const char *result;
+
+    if (in_len > akcipher->max_ciphertext_len) {
+        error_setg(errp, "Ciphertext length is greater than key size: %d",
+                   akcipher->max_ciphertext_len);
+        return ret;
+    }
+
+    err = gcry_sexp_build(&cipher_sexp, NULL,
+                          "(enc-val (flags %s) (rsa (a %b) ))",
+                          QCryptoRSAPaddingAlgorithm_str(rsa->padding_alg),
+                          in_len, in);
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to build ciphertext: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+
+    err = gcry_pk_decrypt(&data_sexp, cipher_sexp, rsa->key);
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to decrypt: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+
+    /* S-expression of plaintext: (value plaintext) */
+    if (rsa->padding_alg == QCRYPTO_RSA_PADDING_ALG_RAW) {
+        data_mpi = gcry_sexp_nth_mpi(data_sexp, 1, GCRYMPI_FMT_USG);
+        if (!data_mpi) {
+            error_setg(errp, "Invalid plaintext result");
+            goto cleanup;
+        }
+        err = gcry_mpi_print(GCRYMPI_FMT_USG, out, out_len,
+                             &actual_len, data_mpi);
+        if (gcry_err_code(err) != 0) {
+            error_setg(errp, "Failed to print MPI: %s/%s",
+                       gcry_strsource(err), gcry_strerror(err));
+            goto cleanup;
+        }
+        if (actual_len > out_len) {
+            error_setg(errp, "Plaintext buffer length is too small");
+            goto cleanup;
+        }
+        /* We always padding leading-zeros for RSA-RAW */
+        if (actual_len < out_len) {
+            memmove((uint8_t *)out + (out_len - actual_len), out, actual_len);
+            memset(out, 0, out_len - actual_len);
+        }
+        ret = out_len;
+    } else {
+        result = gcry_sexp_nth_data(data_sexp, 1, &actual_len);
+        if (!result) {
+            error_setg(errp, "Invalid plaintext result");
+            goto cleanup;
+        }
+        if (actual_len > out_len) {
+            error_setg(errp, "Plaintext buffer length is too small");
+            goto cleanup;
+        }
+        memcpy(out, result, actual_len);
+        ret = actual_len;
+    }
+
+cleanup:
+    gcry_sexp_release(cipher_sexp);
+    gcry_sexp_release(data_sexp);
+    gcry_mpi_release(data_mpi);
+    return ret;
+}
+
+static int qcrypto_gcrypt_rsa_sign(QCryptoAkCipher *akcipher,
+                                   const void *in, size_t in_len,
+                                   void *out, size_t out_len, Error **errp)
+{
+    QCryptoGcryptRSA *rsa = (QCryptoGcryptRSA *)akcipher;
+    int ret = -1;
+    gcry_sexp_t dgst_sexp = NULL, sig_sexp = NULL;
+    gcry_sexp_t sig_sexp_item = NULL;
+    const char *result;
+    gcry_error_t err;
+    size_t actual_len;
+
+    if (in_len > akcipher->max_dgst_len) {
+        error_setg(errp, "Data length is greater than key size: %d",
+                   akcipher->max_dgst_len);
+        return ret;
+    }
+
+    if (rsa->padding_alg != QCRYPTO_RSA_PADDING_ALG_PKCS1) {
+        error_setg(errp, "Invalid padding %u", rsa->padding_alg);
+        return ret;
+    }
+
+    err = gcry_sexp_build(&dgst_sexp, NULL,
+                          "(data (flags pkcs1) (hash %s %b))",
+                          QCryptoHashAlgorithm_str(rsa->hash_alg),
+                          in_len, in);
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to build dgst: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+
+    err = gcry_pk_sign(&sig_sexp, dgst_sexp, rsa->key);
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to make signature: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+
+    /* S-expression of signature: (sig-val (rsa (s s-mpi))) */
+    sig_sexp_item = gcry_sexp_find_token(sig_sexp, "s", 0);
+    if (!sig_sexp_item || gcry_sexp_length(sig_sexp_item) != 2) {
+        error_setg(errp, "Invalid signature result");
+        goto cleanup;
+    }
+
+    result = gcry_sexp_nth_data(sig_sexp_item, 1, &actual_len);
+    if (!result) {
+        error_setg(errp, "Invalid signature result");
+        goto cleanup;
+    }
+
+    if (actual_len > out_len) {
+        error_setg(errp, "Signature buffer length is too small");
+        goto cleanup;
+    }
+    memcpy(out, result, actual_len);
+    ret = actual_len;
+
+cleanup:
+    gcry_sexp_release(dgst_sexp);
+    gcry_sexp_release(sig_sexp);
+    gcry_sexp_release(sig_sexp_item);
+
+    return ret;
+}
+
+static int qcrypto_gcrypt_rsa_verify(QCryptoAkCipher *akcipher,
+                                     const void *in, size_t in_len,
+                                     const void *in2, size_t in2_len,
+                                     Error **errp)
+{
+    QCryptoGcryptRSA *rsa = (QCryptoGcryptRSA *)akcipher;
+    int ret = -1;
+    gcry_sexp_t sig_sexp = NULL, dgst_sexp = NULL;
+    gcry_error_t err;
+
+    if (in_len > akcipher->max_signature_len) {
+        error_setg(errp, "Signature length is greater than key size: %d",
+                   akcipher->max_signature_len);
+        return ret;
+    }
+
+    if (in2_len > akcipher->max_dgst_len) {
+        error_setg(errp, "Data length is greater than key size: %d",
+                   akcipher->max_dgst_len);
+        return ret;
+    }
+
+    if (rsa->padding_alg != QCRYPTO_RSA_PADDING_ALG_PKCS1) {
+        error_setg(errp, "Invalid padding %u", rsa->padding_alg);
+        return ret;
+    }
+
+    err = gcry_sexp_build(&sig_sexp, NULL,
+                          "(sig-val (rsa (s %b)))", in_len, in);
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to build signature: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+
+    err = gcry_sexp_build(&dgst_sexp, NULL,
+                          "(data (flags pkcs1) (hash %s %b))",
+                          QCryptoHashAlgorithm_str(rsa->hash_alg),
+                          in2_len, in2);
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to build dgst: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+
+    err = gcry_pk_verify(sig_sexp, dgst_sexp, rsa->key);
+    if (gcry_err_code(err) != 0) {
+        error_setg(errp, "Failed to verify signature: %s/%s",
+                   gcry_strsource(err), gcry_strerror(err));
+        goto cleanup;
+    }
+    ret = 0;
+
+cleanup:
+    gcry_sexp_release(dgst_sexp);
+    gcry_sexp_release(sig_sexp);
+
+    return ret;
+}
+
+QCryptoAkCipherDriver gcrypt_rsa = {
+    .encrypt = qcrypto_gcrypt_rsa_encrypt,
+    .decrypt = qcrypto_gcrypt_rsa_decrypt,
+    .sign = qcrypto_gcrypt_rsa_sign,
+    .verify = qcrypto_gcrypt_rsa_verify,
+    .free = qcrypto_gcrypt_rsa_free,
+};
+
+static QCryptoGcryptRSA *qcrypto_gcrypt_rsa_new(
+    const QCryptoAkCipherOptionsRSA *opt,
+    QCryptoAkCipherKeyType type,
+    const uint8_t *key, size_t keylen,
+    Error **errp)
+{
+    QCryptoGcryptRSA *rsa = g_new0(QCryptoGcryptRSA, 1);
+    rsa->padding_alg = opt->padding_alg;
+    rsa->hash_alg = opt->hash_alg;
+    rsa->akcipher.driver = &gcrypt_rsa;
+
+    switch (type) {
+    case QCRYPTO_AKCIPHER_KEY_TYPE_PRIVATE:
+        if (qcrypto_gcrypt_parse_rsa_private_key(rsa, key, keylen, errp) != 0) {
+            goto error;
+        }
+        break;
+
+    case QCRYPTO_AKCIPHER_KEY_TYPE_PUBLIC:
+        if (qcrypto_gcrypt_parse_rsa_public_key(rsa, key, keylen, errp) != 0) {
+            goto error;
+        }
+        break;
+
+    default:
+        error_setg(errp, "Unknown akcipher key type %d", type);
+        goto error;
+    }
+
+    return rsa;
+
+error:
+    qcrypto_gcrypt_rsa_free((QCryptoAkCipher *)rsa);
+    return NULL;
+}
+
+
+bool qcrypto_akcipher_supports(QCryptoAkCipherOptions *opts)
+{
+    switch (opts->alg) {
+    case QCRYPTO_AKCIPHER_ALG_RSA:
+        switch (opts->u.rsa.padding_alg) {
+        case QCRYPTO_RSA_PADDING_ALG_RAW:
+            return true;
+
+        case QCRYPTO_RSA_PADDING_ALG_PKCS1:
+            switch (opts->u.rsa.hash_alg) {
+            case QCRYPTO_HASH_ALG_MD5:
+            case QCRYPTO_HASH_ALG_SHA1:
+            case QCRYPTO_HASH_ALG_SHA256:
+            case QCRYPTO_HASH_ALG_SHA512:
+                return true;
+
+            default:
+                return false;
+            }
+
+        default:
+            return false;
+        }
+
+    default:
+        return true;
+    }
+}
diff --git a/crypto/akcipher.c b/crypto/akcipher.c
index f287083f92..ad88379c1e 100644
--- a/crypto/akcipher.c
+++ b/crypto/akcipher.c
@@ -23,7 +23,9 @@
 #include "crypto/akcipher.h"
 #include "akcipherpriv.h"
 
-#if defined(CONFIG_NETTLE) && defined(CONFIG_HOGWEED)
+#if defined(CONFIG_GCRYPT)
+#include "akcipher-gcrypt.c.inc"
+#elif defined(CONFIG_NETTLE) && defined(CONFIG_HOGWEED)
 #include "akcipher-nettle.c.inc"
 #else
 QCryptoAkCipher *qcrypto_akcipher_new(const QCryptoAkCipherOptions *opts,

From 689309c4ac40f01b93f29d66282d044c10d8e2e1 Mon Sep 17 00:00:00 2001
From: Lei He <helei.sig11@bytedance.com>
Date: Wed, 25 May 2022 17:01:16 +0800
Subject: [PATCH 527/935] test/crypto: Add test suite for crypto akcipher
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add unit test and benchmark test for crypto akcipher.

Signed-off-by: lei he <helei.sig11@bytedance.com>
Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 crypto/akcipher-nettle.c.inc            |  14 +-
 tests/bench/benchmark-crypto-akcipher.c | 137 +++++
 tests/bench/meson.build                 |   1 +
 tests/bench/test_akcipher_keys.inc      | 537 ++++++++++++++++++
 tests/unit/meson.build                  |   1 +
 tests/unit/test-crypto-akcipher.c       | 711 ++++++++++++++++++++++++
 6 files changed, 1394 insertions(+), 7 deletions(-)
 create mode 100644 tests/bench/benchmark-crypto-akcipher.c
 create mode 100644 tests/bench/test_akcipher_keys.inc
 create mode 100644 tests/unit/test-crypto-akcipher.c

diff --git a/crypto/akcipher-nettle.c.inc b/crypto/akcipher-nettle.c.inc
index a7c0c6a1ee..02699e6e6d 100644
--- a/crypto/akcipher-nettle.c.inc
+++ b/crypto/akcipher-nettle.c.inc
@@ -171,14 +171,14 @@ static int qcrypto_nettle_rsa_encrypt(QCryptoAkCipher *akcipher,
     int ret = -1;
 
     if (data_len > rsa->pub.size) {
-        error_setg(errp, "Plaintext length %zu is greater than key size: %zu"
+        error_setg(errp, "Plaintext length %zu is greater than key size: %zu",
                    data_len, rsa->pub.size);
         return ret;
     }
 
     if (enc_len < rsa->pub.size) {
         error_setg(errp, "Ciphertext buffer length %zu is less than "
-                         "key size: %zu", rsa->pub.size);
+                         "key size: %zu", enc_len, rsa->pub.size);
         return ret;
     }
 
@@ -218,7 +218,7 @@ static int qcrypto_nettle_rsa_decrypt(QCryptoAkCipher *akcipher,
 
     if (enc_len > rsa->priv.size) {
         error_setg(errp, "Ciphertext length %zu is greater than key size: %zu",
-                   rsa->priv.size);
+                   enc_len, rsa->priv.size);
         return ret;
     }
 
@@ -264,13 +264,13 @@ static int qcrypto_nettle_rsa_sign(QCryptoAkCipher *akcipher,
 
     if (data_len > rsa->priv.size) {
         error_setg(errp, "Data length %zu is greater than key size: %zu",
-                   rsa->priv.size);
+                   data_len, rsa->priv.size);
         return ret;
     }
 
     if (sig_len < rsa->priv.size) {
         error_setg(errp, "Signature buffer length %zu is less than "
-                         "key size: %zu", rsa->priv.size);
+                         "key size: %zu", sig_len, rsa->priv.size);
         return ret;
     }
 
@@ -330,12 +330,12 @@ static int qcrypto_nettle_rsa_verify(QCryptoAkCipher *akcipher,
     }
     if (data_len > rsa->pub.size) {
         error_setg(errp, "Data length %zu is greater than key size: %zu",
-                   rsa->pub.size);
+                   data_len, rsa->pub.size);
         return ret;
     }
     if (sig_len < rsa->pub.size) {
         error_setg(errp, "Signature length %zu is greater than key size: %zu",
-                   rsa->pub.size);
+                   sig_len, rsa->pub.size);
         return ret;
     }
 
diff --git a/tests/bench/benchmark-crypto-akcipher.c b/tests/bench/benchmark-crypto-akcipher.c
new file mode 100644
index 0000000000..15e69557ed
--- /dev/null
+++ b/tests/bench/benchmark-crypto-akcipher.c
@@ -0,0 +1,137 @@
+/*
+ * QEMU Crypto akcipher speed benchmark
+ *
+ * Copyright (c) 2022 Bytedance
+ *
+ * Authors:
+ *    lei he <helei.sig11@bytedance.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * (at your option) any later version.  See the COPYING file in the
+ * top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "crypto/init.h"
+#include "crypto/akcipher.h"
+#include "standard-headers/linux/virtio_crypto.h"
+
+#include "test_akcipher_keys.inc"
+
+static QCryptoAkCipher *create_rsa_akcipher(const uint8_t *priv_key,
+                                            size_t keylen,
+                                            QCryptoRSAPaddingAlgorithm padding,
+                                            QCryptoHashAlgorithm hash)
+{
+    QCryptoAkCipherOptions opt;
+    QCryptoAkCipher *rsa;
+
+    opt.alg = QCRYPTO_AKCIPHER_ALG_RSA;
+    opt.u.rsa.padding_alg = padding;
+    opt.u.rsa.hash_alg = hash;
+    rsa = qcrypto_akcipher_new(&opt, QCRYPTO_AKCIPHER_KEY_TYPE_PRIVATE,
+                               priv_key, keylen, &error_abort);
+    return rsa;
+}
+
+static void test_rsa_speed(const uint8_t *priv_key, size_t keylen,
+                           size_t key_size)
+{
+#define BYTE 8
+#define SHA1_DGST_LEN 20
+#define SIGN_TIMES 10000
+#define VERIFY_TIMES 100000
+#define PADDING QCRYPTO_RSA_PADDING_ALG_PKCS1
+#define HASH QCRYPTO_HASH_ALG_SHA1
+
+    g_autoptr(QCryptoAkCipher) rsa =
+        create_rsa_akcipher(priv_key, keylen, PADDING, HASH);
+    g_autofree uint8_t *dgst = NULL;
+    g_autofree uint8_t *signature = NULL;
+    size_t count;
+
+    dgst = g_new0(uint8_t, SHA1_DGST_LEN);
+    memset(dgst, g_test_rand_int(), SHA1_DGST_LEN);
+    signature = g_new0(uint8_t, key_size / BYTE);
+
+    g_test_message("benchmark rsa%zu (%s-%s) sign...", key_size,
+                   QCryptoRSAPaddingAlgorithm_str(PADDING),
+                   QCryptoHashAlgorithm_str(HASH));
+    g_test_timer_start();
+    for (count = 0; count < SIGN_TIMES; ++count) {
+        g_assert(qcrypto_akcipher_sign(rsa, dgst, SHA1_DGST_LEN,
+                                       signature, key_size / BYTE,
+                                       &error_abort) > 0);
+    }
+    g_test_timer_elapsed();
+    g_test_message("rsa%zu (%s-%s) sign %zu times in %.2f seconds,"
+                   " %.2f times/sec ",
+                   key_size,  QCryptoRSAPaddingAlgorithm_str(PADDING),
+                   QCryptoHashAlgorithm_str(HASH),
+                   count, g_test_timer_last(),
+                   (double)count / g_test_timer_last());
+
+    g_test_message("benchmark rsa%zu (%s-%s) verification...", key_size,
+                   QCryptoRSAPaddingAlgorithm_str(PADDING),
+                   QCryptoHashAlgorithm_str(HASH));
+    g_test_timer_start();
+    for (count = 0; count < VERIFY_TIMES; ++count) {
+        g_assert(qcrypto_akcipher_verify(rsa, signature, key_size / BYTE,
+                                         dgst, SHA1_DGST_LEN,
+                                         &error_abort) == 0);
+    }
+    g_test_timer_elapsed();
+    g_test_message("rsa%zu (%s-%s) verify %zu times in %.2f seconds,"
+                   " %.2f times/sec ",
+                   key_size, QCryptoRSAPaddingAlgorithm_str(PADDING),
+                   QCryptoHashAlgorithm_str(HASH),
+                   count, g_test_timer_last(),
+                   (double)count / g_test_timer_last());
+}
+
+static void test_rsa_1024_speed(const void *opaque)
+{
+    size_t key_size = (size_t)opaque;
+    test_rsa_speed(rsa1024_priv_key, sizeof(rsa1024_priv_key), key_size);
+}
+
+static void test_rsa_2048_speed(const void *opaque)
+{
+    size_t key_size = (size_t)opaque;
+    test_rsa_speed(rsa2048_priv_key, sizeof(rsa2048_priv_key), key_size);
+}
+
+static void test_rsa_4096_speed(const void *opaque)
+{
+    size_t key_size = (size_t)opaque;
+    test_rsa_speed(rsa4096_priv_key, sizeof(rsa4096_priv_key), key_size);
+}
+
+int main(int argc, char **argv)
+{
+    char *alg = NULL;
+    char *size = NULL;
+    g_test_init(&argc, &argv, NULL);
+    g_assert(qcrypto_init(NULL) == 0);
+
+#define ADD_TEST(asym_alg, keysize)                    \
+    if ((!alg || g_str_equal(alg, #asym_alg)) &&       \
+        (!size || g_str_equal(size, #keysize)))        \
+        g_test_add_data_func(                          \
+        "/crypto/akcipher/" #asym_alg "-" #keysize,    \
+        (void *)keysize,                               \
+        test_ ## asym_alg ## _ ## keysize ## _speed)
+
+    if (argc >= 2) {
+        alg = argv[1];
+    }
+    if (argc >= 3) {
+        size = argv[2];
+    }
+
+    ADD_TEST(rsa, 1024);
+    ADD_TEST(rsa, 2048);
+    ADD_TEST(rsa, 4096);
+
+    return g_test_run();
+}
diff --git a/tests/bench/meson.build b/tests/bench/meson.build
index 00b3c209dc..279a8fcc33 100644
--- a/tests/bench/meson.build
+++ b/tests/bench/meson.build
@@ -20,6 +20,7 @@ if have_block
      'benchmark-crypto-hash': [crypto],
      'benchmark-crypto-hmac': [crypto],
      'benchmark-crypto-cipher': [crypto],
+     'benchmark-crypto-akcipher': [crypto],
   }
 endif
 
diff --git a/tests/bench/test_akcipher_keys.inc b/tests/bench/test_akcipher_keys.inc
new file mode 100644
index 0000000000..df3eccb45e
--- /dev/null
+++ b/tests/bench/test_akcipher_keys.inc
@@ -0,0 +1,537 @@
+/*
+ * Copyright (c) 2022 Bytedance, and/or its affiliates
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ * Author: lei he <helei.sig11@bytedance.com>
+ */
+
+/* RSA test keys, generated by OpenSSL */
+static const uint8_t rsa1024_priv_key[] = {
+    0x30, 0x82, 0x02, 0x5c, 0x02, 0x01, 0x00, 0x02,
+    0x81, 0x81, 0x00, 0xe6, 0x4d, 0x76, 0x4f, 0xb2,
+    0x97, 0x09, 0xad, 0x9d, 0x17, 0x33, 0xf2, 0x30,
+    0x42, 0x83, 0xa9, 0xcb, 0x49, 0xa4, 0x2e, 0x59,
+    0x5e, 0x75, 0x51, 0xd1, 0xac, 0xc8, 0x86, 0x3e,
+    0xdb, 0x72, 0x2e, 0xb2, 0xf7, 0xc3, 0x5b, 0xc7,
+    0xea, 0xed, 0x30, 0xd1, 0xf7, 0x37, 0xee, 0x9d,
+    0x36, 0x59, 0x6f, 0xf8, 0xce, 0xc0, 0x5c, 0x82,
+    0x80, 0x37, 0x83, 0xd7, 0x45, 0x6a, 0xe9, 0xea,
+    0xc5, 0x3a, 0x59, 0x6b, 0x34, 0x31, 0x44, 0x00,
+    0x74, 0xa7, 0x29, 0xab, 0x79, 0x4a, 0xbd, 0xe8,
+    0x25, 0x35, 0x01, 0x11, 0x40, 0xbf, 0x31, 0xbd,
+    0xd3, 0xe0, 0x68, 0x1e, 0xd5, 0x5b, 0x2f, 0xe9,
+    0x20, 0xf2, 0x9f, 0x46, 0x35, 0x30, 0xa8, 0xf1,
+    0xfe, 0xef, 0xd8, 0x76, 0x23, 0x46, 0x34, 0x70,
+    0xa1, 0xce, 0xc6, 0x65, 0x6d, 0xb0, 0x94, 0x7e,
+    0xe5, 0x92, 0x45, 0x7b, 0xaa, 0xbb, 0x95, 0x97,
+    0x77, 0xcd, 0xd3, 0x02, 0x03, 0x01, 0x00, 0x01,
+    0x02, 0x81, 0x80, 0x30, 0x6a, 0xc4, 0x9e, 0xc8,
+    0xba, 0xfc, 0x2b, 0xe5, 0xc4, 0xc5, 0x04, 0xfb,
+    0xa4, 0x60, 0x2d, 0xc8, 0x31, 0x39, 0x35, 0x0d,
+    0x50, 0xd0, 0x75, 0x5d, 0x11, 0x68, 0x2e, 0xe0,
+    0xf4, 0x1d, 0xb3, 0x37, 0xa8, 0xe3, 0x07, 0x5e,
+    0xa6, 0x43, 0x2b, 0x6a, 0x59, 0x01, 0x07, 0x47,
+    0x41, 0xef, 0xd7, 0x9c, 0x85, 0x4a, 0xe7, 0xa7,
+    0xff, 0xf0, 0xab, 0xe5, 0x0c, 0x11, 0x08, 0x10,
+    0x75, 0x5a, 0x68, 0xa0, 0x08, 0x03, 0xc9, 0x40,
+    0x79, 0x67, 0x1d, 0x65, 0x89, 0x2d, 0x08, 0xf9,
+    0xb5, 0x1b, 0x7d, 0xd2, 0x41, 0x3b, 0x33, 0xf2,
+    0x47, 0x2f, 0x9c, 0x0b, 0xd5, 0xaf, 0xcb, 0xdb,
+    0xbb, 0x37, 0x63, 0x03, 0xf8, 0xe7, 0x2e, 0xc7,
+    0x3c, 0x86, 0x9f, 0xc2, 0x9b, 0xb4, 0x70, 0x6a,
+    0x4d, 0x7c, 0xe4, 0x1b, 0x3a, 0xa9, 0xae, 0xd7,
+    0xce, 0x7f, 0x56, 0xc2, 0x73, 0x5e, 0x58, 0x63,
+    0xd5, 0x86, 0x41, 0x02, 0x41, 0x00, 0xf6, 0x56,
+    0x69, 0xec, 0xef, 0x65, 0x95, 0xdc, 0x25, 0x47,
+    0xe0, 0x6f, 0xb0, 0x4f, 0x79, 0x77, 0x0a, 0x5e,
+    0x46, 0xcb, 0xbd, 0x0b, 0x71, 0x51, 0x2a, 0xa4,
+    0x65, 0x29, 0x18, 0xc6, 0x30, 0xa0, 0x95, 0x4c,
+    0x4b, 0xbe, 0x8c, 0x40, 0xe3, 0x9c, 0x23, 0x02,
+    0x14, 0x43, 0xe9, 0x64, 0xea, 0xe3, 0xa8, 0xe2,
+    0x1a, 0xd5, 0xf9, 0x5c, 0xe0, 0x36, 0x2c, 0x97,
+    0xda, 0xd5, 0xc7, 0x46, 0xce, 0x11, 0x02, 0x41,
+    0x00, 0xef, 0x56, 0x08, 0xb8, 0x29, 0xa5, 0xa6,
+    0x7c, 0xf7, 0x5f, 0xb4, 0xf5, 0x63, 0xe7, 0xeb,
+    0x45, 0xfd, 0x89, 0xaa, 0x94, 0xa6, 0x3d, 0x0b,
+    0xd9, 0x04, 0x6f, 0x78, 0xe0, 0xbb, 0xa2, 0xd4,
+    0x29, 0x83, 0x17, 0x95, 0x6f, 0x50, 0x3d, 0x40,
+    0x5d, 0xe5, 0x24, 0xda, 0xc2, 0x23, 0x50, 0x86,
+    0xa8, 0x34, 0xc8, 0x6f, 0xec, 0x7f, 0xb6, 0x45,
+    0x3a, 0xdd, 0x78, 0x9b, 0xee, 0xa1, 0xe4, 0x09,
+    0xa3, 0x02, 0x40, 0x5c, 0xd6, 0x66, 0x67, 0x58,
+    0x35, 0xc5, 0xcb, 0xc8, 0xf5, 0x14, 0xbd, 0xa3,
+    0x09, 0xe0, 0xb2, 0x1f, 0x63, 0x36, 0x75, 0x34,
+    0x52, 0xea, 0xaa, 0xf7, 0x52, 0x2b, 0x99, 0xd8,
+    0x6f, 0x61, 0x06, 0x34, 0x1e, 0x23, 0xf1, 0xb5,
+    0x34, 0x03, 0x53, 0xe5, 0xd1, 0xb3, 0xc7, 0x80,
+    0x5f, 0x7b, 0x32, 0xbf, 0x84, 0x2f, 0x2e, 0xf3,
+    0x22, 0xb0, 0x91, 0x5a, 0x2f, 0x04, 0xd7, 0x4a,
+    0x9a, 0x01, 0xb1, 0x02, 0x40, 0x34, 0x0b, 0x26,
+    0x4c, 0x3d, 0xaa, 0x2a, 0xc0, 0xe3, 0xdd, 0xe8,
+    0xf0, 0xaf, 0x6f, 0xe0, 0x06, 0x51, 0x32, 0x9d,
+    0x68, 0x43, 0x99, 0xe4, 0xb8, 0xa5, 0x31, 0x44,
+    0x3c, 0xc2, 0x30, 0x8f, 0x28, 0x13, 0xbc, 0x8e,
+    0x1f, 0x2d, 0x78, 0x94, 0x45, 0x96, 0xad, 0x63,
+    0xf0, 0x71, 0x53, 0x72, 0x64, 0xa3, 0x4d, 0xae,
+    0xa0, 0xe3, 0xc8, 0x93, 0xd7, 0x50, 0x0f, 0x89,
+    0x00, 0xe4, 0x2d, 0x3d, 0x37, 0x02, 0x41, 0x00,
+    0xbe, 0xa6, 0x08, 0xe0, 0xc8, 0x15, 0x2a, 0x47,
+    0xcb, 0xd5, 0xec, 0x93, 0xd3, 0xaa, 0x12, 0x82,
+    0xaf, 0xac, 0x51, 0x5a, 0x5b, 0xa7, 0x93, 0x4b,
+    0xb9, 0xab, 0x00, 0xfa, 0x5a, 0xea, 0x34, 0xe4,
+    0x80, 0xf1, 0x44, 0x6a, 0x65, 0xe4, 0x33, 0x99,
+    0xfb, 0x54, 0xd7, 0x89, 0x5a, 0x1b, 0xd6, 0x2b,
+    0xcc, 0x6e, 0x4b, 0x19, 0xa0, 0x6d, 0x93, 0x9f,
+    0xc3, 0x91, 0x7a, 0xa5, 0xd8, 0x59, 0x0e, 0x9e,
+};
+
+static const uint8_t rsa2048_priv_key[] = {
+    0x30, 0x82, 0x04, 0xa4, 0x02, 0x01, 0x00, 0x02,
+    0x82, 0x01, 0x01, 0x00, 0xbd, 0x9c, 0x83, 0x6b,
+    0x0e, 0x8e, 0xcf, 0xfa, 0xaa, 0x4f, 0x6a, 0xf4,
+    0xe3, 0x52, 0x0f, 0xa5, 0xd0, 0xbe, 0x5e, 0x7f,
+    0x08, 0x24, 0xba, 0x87, 0x46, 0xfb, 0x28, 0x93,
+    0xe5, 0xe5, 0x81, 0x42, 0xc0, 0xf9, 0x17, 0xc7,
+    0x81, 0x01, 0xf4, 0x18, 0x6a, 0x17, 0xf5, 0x57,
+    0x20, 0x37, 0xcf, 0xf9, 0x74, 0x5e, 0xe1, 0x48,
+    0x6a, 0x71, 0x0a, 0x0f, 0x79, 0x72, 0x2b, 0x46,
+    0x10, 0x53, 0xdc, 0x14, 0x43, 0xbd, 0xbc, 0x6d,
+    0x15, 0x6f, 0x15, 0x4e, 0xf0, 0x0d, 0x89, 0x39,
+    0x02, 0xc3, 0x68, 0x5c, 0xa8, 0xfc, 0xed, 0x64,
+    0x9d, 0x98, 0xb7, 0xcd, 0x83, 0x66, 0x93, 0xc3,
+    0xd9, 0x57, 0xa0, 0x21, 0x93, 0xad, 0x5c, 0x75,
+    0x69, 0x88, 0x9e, 0x81, 0xdc, 0x7f, 0x1d, 0xd5,
+    0xbd, 0x1c, 0xc1, 0x30, 0x56, 0xa5, 0xda, 0x99,
+    0x46, 0xa6, 0x6d, 0x0e, 0x6f, 0x5e, 0x51, 0x34,
+    0x49, 0x73, 0xc3, 0x67, 0x49, 0x7e, 0x21, 0x2a,
+    0x20, 0xa7, 0x2b, 0x92, 0x73, 0x1d, 0xa5, 0x25,
+    0x2a, 0xd0, 0x3a, 0x89, 0x75, 0xb2, 0xbb, 0x19,
+    0x37, 0x78, 0x48, 0xd2, 0xf2, 0x2a, 0x6d, 0x9e,
+    0xc6, 0x26, 0xca, 0x46, 0x8c, 0xf1, 0x42, 0x2a,
+    0x31, 0xb2, 0xfc, 0xe7, 0x55, 0x51, 0xff, 0x07,
+    0x13, 0x5b, 0x36, 0x59, 0x2b, 0x43, 0x30, 0x4b,
+    0x05, 0x5c, 0xd2, 0x45, 0xa0, 0xa0, 0x7c, 0x17,
+    0x5b, 0x07, 0xbb, 0x5d, 0x83, 0x80, 0x92, 0x6d,
+    0x87, 0x1a, 0x43, 0xac, 0xc7, 0x6b, 0x8d, 0x11,
+    0x60, 0x27, 0xd2, 0xdf, 0xdb, 0x71, 0x02, 0x55,
+    0x6e, 0xb5, 0xca, 0x4d, 0xda, 0x59, 0x0d, 0xb8,
+    0x8c, 0xcd, 0xd3, 0x0e, 0x55, 0xa0, 0xa4, 0x8d,
+    0xa0, 0x14, 0x10, 0x48, 0x42, 0x35, 0x56, 0x08,
+    0xf7, 0x29, 0x5f, 0xa2, 0xea, 0xa4, 0x5e, 0x8e,
+    0x99, 0x56, 0xaa, 0x5a, 0x8c, 0x23, 0x8f, 0x35,
+    0x22, 0x8a, 0xff, 0xed, 0x02, 0x03, 0x01, 0x00,
+    0x01, 0x02, 0x82, 0x01, 0x00, 0x4e, 0x4a, 0xf3,
+    0x44, 0xe0, 0x64, 0xfd, 0xe1, 0xde, 0x33, 0x1e,
+    0xd1, 0xf1, 0x8f, 0x6f, 0xe0, 0xa2, 0xfa, 0x08,
+    0x60, 0xe1, 0xc6, 0xf0, 0xb2, 0x6d, 0x0f, 0xc6,
+    0x28, 0x93, 0xb4, 0x19, 0x94, 0xab, 0xc3, 0xef,
+    0x1a, 0xb4, 0xdd, 0x4e, 0xa2, 0x4a, 0x24, 0x8c,
+    0x6c, 0xa6, 0x64, 0x05, 0x5f, 0x56, 0xba, 0xda,
+    0xc1, 0x21, 0x1a, 0x7d, 0xf1, 0xf7, 0xce, 0xb9,
+    0xa9, 0x9b, 0x92, 0x54, 0xfc, 0x95, 0x20, 0x22,
+    0x4e, 0xd4, 0x9b, 0xe2, 0xab, 0x8e, 0x99, 0xb8,
+    0x40, 0xaf, 0x30, 0x6a, 0xc6, 0x60, 0x0c, 0xd8,
+    0x25, 0x44, 0xa1, 0xcb, 0xbb, 0x73, 0x77, 0x86,
+    0xaa, 0x46, 0xf3, 0x54, 0xae, 0xa8, 0xa0, 0xdb,
+    0xdd, 0xab, 0x6e, 0xfb, 0x2c, 0x5a, 0x14, 0xaf,
+    0x08, 0x13, 0xa7, 0x6c, 0xe9, 0xfd, 0xcd, 0x4c,
+    0x1f, 0x20, 0x3a, 0x16, 0x2b, 0xf0, 0xb6, 0x7c,
+    0x47, 0x5f, 0xd1, 0x0a, 0x2c, 0xc4, 0xa5, 0x68,
+    0xd0, 0x43, 0x75, 0x6b, 0x65, 0xaa, 0x32, 0xc6,
+    0x99, 0x06, 0xcb, 0x8f, 0xe6, 0x8d, 0xce, 0xbf,
+    0x4d, 0x0d, 0x7b, 0x22, 0x2a, 0x8a, 0xcb, 0x7d,
+    0x7f, 0x16, 0x48, 0x85, 0xf1, 0x86, 0xcb, 0x54,
+    0xb9, 0x39, 0xd4, 0xbc, 0xe3, 0x2d, 0x27, 0x59,
+    0xf6, 0x81, 0x5e, 0x94, 0x45, 0xdf, 0xb9, 0x22,
+    0xaf, 0x64, 0x0d, 0x14, 0xec, 0x8c, 0xeb, 0x71,
+    0xac, 0xee, 0x09, 0x4c, 0xbf, 0x34, 0xf9, 0xf4,
+    0x66, 0x77, 0x36, 0x3b, 0x41, 0x74, 0x01, 0x4f,
+    0xfc, 0x56, 0x83, 0xba, 0x14, 0xb0, 0x2f, 0xdd,
+    0x4d, 0xb9, 0x3f, 0xdf, 0x71, 0xbe, 0x7b, 0xba,
+    0x66, 0xc8, 0xc5, 0x42, 0xc9, 0xba, 0x18, 0x63,
+    0x45, 0x07, 0x2f, 0x84, 0x3e, 0xc3, 0xfb, 0x47,
+    0xda, 0xd4, 0x1d, 0x0e, 0x9d, 0x96, 0xc0, 0xea,
+    0xee, 0x45, 0x2f, 0xe1, 0x62, 0x23, 0xee, 0xef,
+    0x3d, 0x5e, 0x55, 0xa1, 0x0d, 0x02, 0x81, 0x81,
+    0x00, 0xeb, 0x76, 0x88, 0xd3, 0xae, 0x3f, 0x1d,
+    0xf2, 0x49, 0xe0, 0x37, 0x49, 0x83, 0x82, 0x6c,
+    0xf7, 0xf1, 0x17, 0x30, 0x75, 0x2e, 0x89, 0x06,
+    0x88, 0x56, 0x32, 0xf6, 0xfa, 0x58, 0xcb, 0x3c,
+    0x98, 0x67, 0xc3, 0xde, 0x10, 0x82, 0xe5, 0xfa,
+    0xfa, 0x52, 0x47, 0x8d, 0xd7, 0x00, 0xc6, 0xcb,
+    0xf7, 0xf6, 0x57, 0x9b, 0x6e, 0x0c, 0xac, 0xe8,
+    0x3b, 0xd1, 0xde, 0xb5, 0x34, 0xaf, 0x8b, 0x2a,
+    0xb0, 0x2d, 0x01, 0xeb, 0x7c, 0xa0, 0x42, 0x26,
+    0xbb, 0x2b, 0x43, 0x0e, 0x1d, 0xe2, 0x4e, 0xc9,
+    0xc1, 0x0a, 0x67, 0x1d, 0xfc, 0x83, 0x25, 0xce,
+    0xb2, 0x18, 0xd9, 0x0d, 0x70, 0xf5, 0xa3, 0x5a,
+    0x9c, 0x99, 0xdd, 0x47, 0xa1, 0x57, 0xe7, 0x20,
+    0xde, 0xa1, 0x29, 0x8d, 0x96, 0x62, 0xf9, 0x26,
+    0x95, 0x51, 0xa6, 0xe7, 0x09, 0x8b, 0xba, 0x16,
+    0x8b, 0x19, 0x5b, 0xf9, 0x27, 0x0d, 0xc5, 0xd6,
+    0x5f, 0x02, 0x81, 0x81, 0x00, 0xce, 0x26, 0x31,
+    0xb5, 0x43, 0x53, 0x95, 0x39, 0xdd, 0x01, 0x98,
+    0x8b, 0x3d, 0x27, 0xeb, 0x0b, 0x87, 0x1c, 0x95,
+    0xfc, 0x3e, 0x36, 0x51, 0x31, 0xb5, 0xea, 0x59,
+    0x56, 0xc0, 0x97, 0x62, 0xf0, 0x63, 0x2b, 0xb6,
+    0x30, 0x9b, 0xdf, 0x19, 0x10, 0xe9, 0xa0, 0x3d,
+    0xea, 0x54, 0x5a, 0xe6, 0xc6, 0x9e, 0x7e, 0xb5,
+    0xf0, 0xb0, 0x54, 0xef, 0xc3, 0xe1, 0x47, 0xa6,
+    0x95, 0xc7, 0xe4, 0xa3, 0x4a, 0x30, 0x68, 0x24,
+    0x98, 0x7d, 0xc1, 0x34, 0xa9, 0xcb, 0xbc, 0x3c,
+    0x08, 0x9c, 0x7d, 0x0c, 0xa2, 0xb7, 0x60, 0xaa,
+    0x38, 0x08, 0x16, 0xa6, 0x7f, 0xdb, 0xd2, 0xb1,
+    0x67, 0xe7, 0x93, 0x8e, 0xbb, 0x7e, 0xb9, 0xb5,
+    0xd0, 0xd0, 0x9f, 0x7b, 0xcc, 0x46, 0xe6, 0x74,
+    0x78, 0x1a, 0x96, 0xd6, 0xd7, 0x74, 0x34, 0x54,
+    0x3b, 0x54, 0x55, 0x7f, 0x89, 0x81, 0xbc, 0x40,
+    0x55, 0x87, 0x24, 0x95, 0x33, 0x02, 0x81, 0x81,
+    0x00, 0xb0, 0x18, 0x5d, 0x2a, 0x1a, 0x95, 0x9f,
+    0x9a, 0xd5, 0x3f, 0x37, 0x79, 0xe6, 0x3d, 0x83,
+    0xab, 0x46, 0x86, 0x36, 0x3a, 0x5d, 0x0c, 0x23,
+    0x73, 0x91, 0x2b, 0xda, 0x63, 0xce, 0x46, 0x68,
+    0xd1, 0xfe, 0x40, 0x90, 0xf2, 0x3e, 0x43, 0x2b,
+    0x19, 0x4c, 0xb1, 0xb0, 0xd5, 0x8c, 0x02, 0x21,
+    0x07, 0x18, 0x17, 0xda, 0xe9, 0x49, 0xd7, 0x82,
+    0x73, 0x42, 0x78, 0xd1, 0x82, 0x4e, 0x8a, 0xc0,
+    0xe9, 0x33, 0x2f, 0xcd, 0x62, 0xce, 0x23, 0xca,
+    0xfd, 0x8d, 0xd4, 0x3f, 0x59, 0x80, 0x27, 0xb6,
+    0x61, 0x85, 0x9b, 0x2a, 0xe4, 0xef, 0x5c, 0x36,
+    0x22, 0x21, 0xcd, 0x2a, 0x6d, 0x41, 0x77, 0xe2,
+    0xcb, 0x5d, 0x93, 0x0d, 0x00, 0x10, 0x52, 0x8d,
+    0xd5, 0x92, 0x28, 0x16, 0x78, 0xd3, 0x1a, 0x4c,
+    0x8d, 0xbd, 0x9c, 0x1a, 0x0b, 0x9c, 0x91, 0x16,
+    0x4c, 0xff, 0x31, 0x36, 0xbb, 0xcb, 0x64, 0x1a,
+    0xf7, 0x02, 0x81, 0x80, 0x32, 0x65, 0x09, 0xdf,
+    0xca, 0xee, 0xa2, 0xdb, 0x3b, 0x58, 0xc9, 0x86,
+    0xb8, 0x53, 0x8a, 0xd5, 0x0d, 0x99, 0x82, 0x5c,
+    0xe0, 0x84, 0x7c, 0xc2, 0xcf, 0x3a, 0xd3, 0xce,
+    0x2e, 0x54, 0x93, 0xbe, 0x3a, 0x30, 0x14, 0x60,
+    0xbb, 0xaa, 0x05, 0x41, 0xaa, 0x2b, 0x1f, 0x17,
+    0xaa, 0xb9, 0x72, 0x12, 0xf9, 0xe9, 0xf5, 0xe6,
+    0x39, 0xe4, 0xf9, 0x9c, 0x03, 0xf5, 0x75, 0x16,
+    0xc6, 0x7f, 0xf1, 0x1f, 0x10, 0xc8, 0x54, 0xb1,
+    0xe6, 0x84, 0x15, 0xb0, 0xb0, 0x7a, 0x7a, 0x9e,
+    0x8c, 0x4a, 0xd1, 0x8c, 0xf1, 0x91, 0x32, 0xeb,
+    0x71, 0xa6, 0xbf, 0xdb, 0x1f, 0xcc, 0xd8, 0xcb,
+    0x92, 0xc3, 0xf2, 0xaf, 0x89, 0x22, 0x32, 0xfd,
+    0x32, 0x12, 0xda, 0xbb, 0xac, 0x55, 0x68, 0x01,
+    0x78, 0x56, 0x89, 0x7c, 0xb0, 0x0e, 0x9e, 0xcc,
+    0xc6, 0x28, 0x04, 0x7e, 0x83, 0xf5, 0x96, 0x30,
+    0x92, 0x51, 0xf2, 0x1b, 0x02, 0x81, 0x81, 0x00,
+    0x83, 0x6d, 0xd1, 0x98, 0x90, 0x41, 0x8c, 0xa7,
+    0x92, 0x83, 0xac, 0x89, 0x05, 0x0c, 0x79, 0x67,
+    0x90, 0xb6, 0xa1, 0xf3, 0x2f, 0xca, 0xf0, 0x15,
+    0xe0, 0x30, 0x58, 0xe9, 0x4f, 0xcb, 0x4c, 0x56,
+    0x56, 0x56, 0x14, 0x3f, 0x1b, 0x79, 0xb6, 0xef,
+    0x57, 0x4b, 0x28, 0xbd, 0xb0, 0xe6, 0x0c, 0x49,
+    0x4b, 0xbe, 0xe1, 0x57, 0x28, 0x2a, 0x23, 0x5e,
+    0xc4, 0xa2, 0x19, 0x4b, 0x00, 0x67, 0x78, 0xd9,
+    0x26, 0x6e, 0x17, 0x25, 0xce, 0xe4, 0xfd, 0xde,
+    0x86, 0xa8, 0x5a, 0x67, 0x47, 0x6b, 0x15, 0x09,
+    0xe1, 0xec, 0x8e, 0x62, 0x98, 0x91, 0x6f, 0xc0,
+    0x98, 0x0c, 0x70, 0x0e, 0x7d, 0xbe, 0x63, 0xbd,
+    0x12, 0x5a, 0x98, 0x1c, 0xe3, 0x0c, 0xfb, 0xc7,
+    0xfb, 0x1b, 0xbd, 0x02, 0x87, 0xcc, 0x0c, 0xbb,
+    0xc2, 0xd4, 0xb6, 0xc1, 0xa1, 0x23, 0xd3, 0x1e,
+    0x21, 0x6f, 0x48, 0xba, 0x0e, 0x2e, 0xc7, 0x42
+};
+
+static const uint8_t rsa4096_priv_key[] = {
+    0x30, 0x82, 0x09, 0x29, 0x02, 0x01, 0x00, 0x02,
+    0x82, 0x02, 0x01, 0x00, 0xcc, 0x30, 0xc6, 0x90,
+    0x49, 0x2b, 0x86, 0xe7, 0x7a, 0xa5, 0x7a, 0x9a,
+    0x4f, 0xee, 0x0e, 0xa1, 0x5c, 0x43, 0x64, 0xd0,
+    0x76, 0xe1, 0xfd, 0x0b, 0xfd, 0x43, 0x7a, 0x65,
+    0xe6, 0x20, 0xbd, 0xf2, 0x0e, 0xbe, 0x76, 0x54,
+    0xae, 0x37, 0xbe, 0xa0, 0x02, 0x96, 0xae, 0x8d,
+    0x8a, 0xae, 0x3b, 0x88, 0xbb, 0x67, 0xce, 0x7c,
+    0x20, 0xbf, 0x14, 0xc3, 0x71, 0x51, 0x87, 0x03,
+    0x34, 0xaa, 0x3c, 0x09, 0xff, 0xe9, 0xeb, 0xb7,
+    0x85, 0x5c, 0xbb, 0x8d, 0xce, 0x8e, 0x3f, 0xd1,
+    0x16, 0x30, 0x00, 0x32, 0x2f, 0x25, 0x8d, 0xef,
+    0x71, 0xd9, 0xea, 0x6b, 0x45, 0x53, 0x49, 0xc3,
+    0x09, 0x4f, 0xb0, 0xa8, 0xa5, 0x89, 0x76, 0x59,
+    0x31, 0xa5, 0xf1, 0x5c, 0x42, 0x54, 0x57, 0x70,
+    0x57, 0xad, 0xd8, 0xeb, 0x89, 0xa6, 0x87, 0xa2,
+    0x6c, 0x95, 0x58, 0x8f, 0xb6, 0x82, 0xc7, 0xde,
+    0xc2, 0x3a, 0xdc, 0x5b, 0xe8, 0x02, 0xcc, 0x26,
+    0x4b, 0x01, 0xaa, 0xe6, 0xf3, 0x66, 0x4d, 0x90,
+    0x85, 0xde, 0xf4, 0x5d, 0x80, 0x98, 0xc6, 0x65,
+    0xcf, 0x44, 0x4c, 0xde, 0xb5, 0x4a, 0xfc, 0xda,
+    0x0a, 0x0a, 0x10, 0x26, 0xa3, 0xcb, 0x9d, 0xe4,
+    0x8d, 0xab, 0x2c, 0x04, 0xfd, 0xaa, 0xfc, 0x3b,
+    0xac, 0x4e, 0x56, 0xb8, 0x4c, 0x9f, 0x22, 0x49,
+    0xcb, 0x76, 0x45, 0x24, 0x36, 0x2d, 0xbb, 0xe6,
+    0x7e, 0xa9, 0x93, 0x13, 0x96, 0x1e, 0xfc, 0x4b,
+    0x75, 0xd4, 0x54, 0xc8, 0x8c, 0x55, 0xe6, 0x3f,
+    0x09, 0x5a, 0x03, 0x74, 0x7c, 0x8a, 0xc8, 0xe7,
+    0x49, 0x0b, 0x86, 0x7c, 0x97, 0xa0, 0xf2, 0x0d,
+    0xf1, 0x5c, 0x0e, 0x7a, 0xc0, 0x3f, 0x78, 0x2d,
+    0x9b, 0xe2, 0x26, 0xa0, 0x89, 0x49, 0x0c, 0xad,
+    0x79, 0xa6, 0x82, 0x98, 0xa6, 0xb7, 0x74, 0xb4,
+    0x45, 0xc8, 0xed, 0xea, 0x81, 0xcd, 0xf0, 0x3b,
+    0x8e, 0x24, 0xfb, 0x0c, 0xd0, 0x3a, 0x14, 0xb9,
+    0xb4, 0x3b, 0x69, 0xd9, 0xf2, 0x42, 0x6e, 0x7f,
+    0x6f, 0x5e, 0xb1, 0x52, 0x5b, 0xaa, 0xef, 0xae,
+    0x1e, 0x34, 0xca, 0xed, 0x0a, 0x8d, 0x56, 0xd6,
+    0xdd, 0xd4, 0x2c, 0x54, 0x7a, 0x57, 0xca, 0x7e,
+    0x4a, 0x11, 0xde, 0x48, 0xdf, 0x2b, 0x09, 0x97,
+    0x39, 0x24, 0xce, 0x45, 0xe0, 0x75, 0xb1, 0x19,
+    0x42, 0xdb, 0x63, 0x40, 0x9b, 0xb9, 0x95, 0x96,
+    0x78, 0x91, 0xd5, 0x19, 0x12, 0xab, 0xef, 0x55,
+    0x6f, 0x0d, 0x65, 0xc0, 0x8f, 0x62, 0x99, 0x78,
+    0xc0, 0xe0, 0xe1, 0x33, 0xc7, 0x68, 0xff, 0x29,
+    0x66, 0x22, 0x3a, 0x6f, 0xa0, 0xf8, 0x5c, 0x68,
+    0x9b, 0xa9, 0x05, 0xad, 0x6b, 0x1d, 0xae, 0xc1,
+    0x30, 0xbb, 0xfe, 0xb7, 0x31, 0x85, 0x0d, 0xd1,
+    0xd5, 0xfc, 0x43, 0x1e, 0xb3, 0x61, 0x6f, 0xc4,
+    0x75, 0xed, 0x76, 0x9d, 0x13, 0xb3, 0x61, 0x57,
+    0xc8, 0x33, 0x0d, 0x77, 0x84, 0xf0, 0xc7, 0x62,
+    0xb9, 0x9e, 0xd5, 0x01, 0xfa, 0x87, 0x4a, 0xf5,
+    0xd7, 0x4f, 0x5d, 0xae, 0xe7, 0x08, 0xd2, 0x5a,
+    0x65, 0x30, 0xc9, 0xf0, 0x0a, 0x11, 0xf1, 0x2a,
+    0xd3, 0x43, 0x43, 0xca, 0x05, 0x90, 0x85, 0xf4,
+    0xbc, 0x37, 0x49, 0x40, 0x45, 0x35, 0xd3, 0x56,
+    0x06, 0x4c, 0x63, 0x93, 0x07, 0x14, 0x8b, 0xd3,
+    0x12, 0xd0, 0xe5, 0x00, 0x48, 0x76, 0xd2, 0xdf,
+    0x7c, 0xea, 0xc7, 0xff, 0xf0, 0x88, 0xd5, 0xa4,
+    0x61, 0x7d, 0x79, 0xc2, 0xda, 0x53, 0x24, 0xdc,
+    0x20, 0xae, 0xe6, 0x08, 0x65, 0xef, 0xc9, 0x0d,
+    0x7d, 0x66, 0x6d, 0x1b, 0x1c, 0x5d, 0x46, 0xe1,
+    0x26, 0x8a, 0x29, 0x77, 0x76, 0x19, 0xe5, 0x19,
+    0x2a, 0x75, 0x21, 0xf1, 0x92, 0x8a, 0x9c, 0x7b,
+    0xe8, 0x0b, 0x38, 0xc1, 0xbf, 0x76, 0x22, 0x45,
+    0x4a, 0xd3, 0x43, 0xc3, 0x8c, 0x74, 0xd8, 0xd8,
+    0xec, 0x3e, 0x14, 0xdf, 0x02, 0x03, 0x01, 0x00,
+    0x01, 0x02, 0x82, 0x02, 0x01, 0x00, 0x9e, 0x13,
+    0x64, 0xa5, 0x6e, 0xff, 0xf3, 0x80, 0x60, 0xc2,
+    0x9b, 0x17, 0xbb, 0xa9, 0x60, 0x4a, 0x2b, 0x53,
+    0x41, 0x48, 0xe1, 0xc0, 0x32, 0x56, 0x85, 0xcb,
+    0x27, 0x86, 0x9b, 0x91, 0xdd, 0x7a, 0xf7, 0x4f,
+    0x1b, 0xec, 0x92, 0xb3, 0x35, 0x30, 0x4a, 0xd0,
+    0xbc, 0x71, 0x77, 0x5b, 0x4b, 0x5b, 0x9f, 0x39,
+    0xcd, 0xf0, 0xea, 0xa9, 0x03, 0x3a, 0x0b, 0x10,
+    0x42, 0xa5, 0x88, 0xb0, 0x01, 0xaa, 0xfc, 0x23,
+    0xec, 0x08, 0x37, 0x86, 0x82, 0xec, 0x55, 0x6c,
+    0x6a, 0x9b, 0x43, 0xc2, 0x05, 0x64, 0xd4, 0x7b,
+    0x0e, 0x56, 0xc0, 0x9d, 0x23, 0x8d, 0xc8, 0x2d,
+    0xa2, 0x7d, 0x0b, 0x48, 0x56, 0x4b, 0x39, 0x5c,
+    0x21, 0xf3, 0x0b, 0x2c, 0x9c, 0x9d, 0xff, 0xfb,
+    0xab, 0x75, 0x9d, 0x6b, 0x48, 0xf3, 0x8f, 0xad,
+    0x0c, 0x74, 0x01, 0xfb, 0xdc, 0x83, 0xe5, 0x97,
+    0x79, 0x84, 0x4a, 0x79, 0xa6, 0xfe, 0xbf, 0xae,
+    0xea, 0xbc, 0xfa, 0x74, 0x60, 0x0a, 0x4b, 0x84,
+    0x77, 0xa7, 0xda, 0xfb, 0xaf, 0xd2, 0x73, 0x2b,
+    0xd2, 0xec, 0x1e, 0x79, 0x91, 0xc9, 0x18, 0x30,
+    0xe5, 0x6f, 0x27, 0x36, 0x83, 0x2a, 0x66, 0xc3,
+    0xcb, 0x88, 0x94, 0xe4, 0x5f, 0x3f, 0xbd, 0xe2,
+    0x11, 0x43, 0x61, 0x31, 0x84, 0x91, 0x49, 0x40,
+    0x29, 0x1b, 0x58, 0x18, 0x47, 0x8e, 0xb1, 0x22,
+    0xd6, 0xc4, 0xaa, 0x6a, 0x3d, 0x22, 0x7c, 0xa5,
+    0xa0, 0x4c, 0x0a, 0xfc, 0x46, 0x66, 0xbb, 0xbe,
+    0x04, 0x71, 0xe8, 0x9b, 0x76, 0xf1, 0x47, 0x39,
+    0x6a, 0x2f, 0x23, 0xad, 0x78, 0x80, 0x1c, 0x22,
+    0xcd, 0x41, 0x5e, 0x09, 0x16, 0x6c, 0x91, 0x48,
+    0x91, 0x91, 0x3d, 0x8c, 0xe6, 0xba, 0x81, 0x8d,
+    0xbb, 0xf2, 0xd0, 0xaa, 0xc7, 0x8f, 0xc6, 0x01,
+    0x60, 0xa7, 0xef, 0x1e, 0x8e, 0x91, 0x6d, 0xcc,
+    0x30, 0x9e, 0xea, 0x7c, 0x56, 0x9d, 0x42, 0xcf,
+    0x44, 0x85, 0x52, 0xa8, 0xf2, 0x36, 0x9c, 0x46,
+    0xfa, 0x9d, 0xd3, 0x4e, 0x13, 0x46, 0x81, 0xce,
+    0x99, 0xc9, 0x58, 0x47, 0xe4, 0xeb, 0x27, 0x56,
+    0x29, 0x61, 0x0f, 0xb5, 0xcb, 0xf3, 0x48, 0x58,
+    0x8f, 0xbc, 0xaf, 0x0a, 0xbf, 0x40, 0xd1, 0xf6,
+    0x4f, 0xd2, 0x89, 0x4a, 0xff, 0x6f, 0x54, 0x70,
+    0x49, 0x42, 0xf6, 0xf8, 0x0e, 0x4f, 0xa5, 0xf6,
+    0x8b, 0x49, 0x80, 0xd4, 0xf5, 0x03, 0xf8, 0x65,
+    0xe7, 0x1f, 0x0a, 0xc0, 0x8f, 0xd3, 0x7a, 0x70,
+    0xca, 0x67, 0xaf, 0x71, 0xfd, 0x4b, 0xe1, 0x17,
+    0x76, 0x74, 0x2e, 0x12, 0x7b, 0xad, 0x4b, 0xbb,
+    0xd2, 0x64, 0xd0, 0xa9, 0xf9, 0x79, 0xa9, 0xa6,
+    0x03, 0xd2, 0xc2, 0x8f, 0x47, 0x59, 0x1b, 0x7c,
+    0xe3, 0xce, 0x92, 0xb2, 0xac, 0x3e, 0xee, 0x12,
+    0x43, 0x5f, 0x23, 0xec, 0xf1, 0xd3, 0xf2, 0x21,
+    0x22, 0xe8, 0x7e, 0x7f, 0xa4, 0x93, 0x8e, 0x78,
+    0x69, 0x69, 0xa0, 0xc9, 0xce, 0x86, 0x36, 0x13,
+    0x10, 0x21, 0xc4, 0x7a, 0x52, 0xcf, 0x53, 0xd9,
+    0x9b, 0x58, 0xe6, 0x2d, 0xeb, 0x60, 0xe3, 0x75,
+    0x1a, 0x22, 0xf6, 0x3c, 0x54, 0x6b, 0xfa, 0xa1,
+    0x5d, 0xf6, 0x38, 0xf0, 0xd4, 0x26, 0x2d, 0x7d,
+    0x74, 0x99, 0x6a, 0x13, 0x8a, 0x07, 0x9f, 0x07,
+    0xc5, 0xf4, 0xa8, 0x20, 0x11, 0xa9, 0x76, 0x11,
+    0xe4, 0x48, 0xae, 0xa4, 0x8a, 0xa1, 0xbf, 0x1f,
+    0xba, 0x37, 0x50, 0x53, 0x43, 0x91, 0x45, 0x88,
+    0x03, 0x52, 0xba, 0xac, 0xc8, 0xe3, 0xe1, 0xba,
+    0x63, 0x24, 0x72, 0xbe, 0x1d, 0x01, 0x1f, 0x6c,
+    0x34, 0x10, 0xb8, 0x56, 0x4a, 0x67, 0x28, 0x4b,
+    0x7a, 0x2b, 0x31, 0x29, 0x47, 0xda, 0xdf, 0x53,
+    0x88, 0x79, 0x22, 0x31, 0x15, 0x56, 0xe3, 0xa0,
+    0x79, 0x75, 0x94, 0x90, 0xb2, 0xe8, 0x4b, 0xca,
+    0x82, 0x6d, 0x3c, 0x69, 0x43, 0x01, 0x02, 0x82,
+    0x01, 0x01, 0x00, 0xe7, 0x8b, 0xd6, 0x1a, 0xe8,
+    0x00, 0xed, 0x9d, 0x7c, 0x5a, 0x32, 0x10, 0xc1,
+    0x53, 0x50, 0xbe, 0x27, 0x1d, 0xef, 0x69, 0x73,
+    0xa2, 0x8f, 0x95, 0x96, 0x86, 0xfe, 0xfb, 0x82,
+    0xdb, 0xea, 0x7d, 0x73, 0x5a, 0x2b, 0xe7, 0x4b,
+    0xd5, 0x8f, 0x4f, 0xaf, 0x85, 0x1d, 0x15, 0x1a,
+    0x58, 0x5f, 0x41, 0x79, 0x70, 0x5c, 0x8f, 0xa9,
+    0x8e, 0x23, 0x31, 0xa7, 0x6d, 0x99, 0x0c, 0xf0,
+    0x51, 0xbf, 0xbb, 0xd3, 0xe3, 0xa3, 0x34, 0xf0,
+    0x1d, 0x7f, 0x4a, 0xb7, 0x8f, 0xf6, 0x0a, 0x49,
+    0x65, 0xaf, 0x35, 0x7b, 0x02, 0x2e, 0x69, 0x49,
+    0x95, 0xb5, 0x20, 0x70, 0xb2, 0x98, 0x54, 0x9b,
+    0x8e, 0x4f, 0x48, 0xa8, 0xfa, 0x7e, 0xc7, 0x0a,
+    0xae, 0x84, 0xe1, 0xba, 0x85, 0x98, 0x96, 0x8a,
+    0x7c, 0xdd, 0xcc, 0xcd, 0xd8, 0x5b, 0x50, 0x60,
+    0x88, 0x2d, 0xb6, 0x3e, 0xb8, 0xc2, 0xae, 0xa5,
+    0x62, 0x10, 0xcd, 0xdc, 0xae, 0x86, 0xfe, 0x31,
+    0x8b, 0xf7, 0xee, 0x1a, 0x35, 0x46, 0x83, 0xee,
+    0x5f, 0x55, 0x9a, 0xc2, 0xca, 0x53, 0xb7, 0x2c,
+    0xbf, 0x03, 0x8a, 0x78, 0xcc, 0x1d, 0x96, 0x7b,
+    0xac, 0x00, 0x62, 0x1e, 0xbd, 0x6f, 0x0b, 0xa5,
+    0xec, 0xf3, 0x02, 0x47, 0x47, 0x1e, 0x3d, 0xf6,
+    0x78, 0x42, 0xe4, 0xcd, 0xf8, 0x14, 0xa3, 0x7d,
+    0xd5, 0x2f, 0x6e, 0xcc, 0x1a, 0x9e, 0xe7, 0xcf,
+    0x48, 0xb9, 0x80, 0xb8, 0xba, 0xaa, 0x7b, 0xae,
+    0x65, 0x74, 0x09, 0x7b, 0x43, 0x26, 0x31, 0xa2,
+    0x95, 0x43, 0x69, 0xd0, 0xb7, 0x95, 0xe4, 0x76,
+    0x2c, 0x42, 0x19, 0x47, 0x4f, 0x63, 0x35, 0x9c,
+    0xa2, 0x1a, 0xce, 0x28, 0xdf, 0x76, 0x98, 0x1d,
+    0xd4, 0x2e, 0xf6, 0x3a, 0xc8, 0x3e, 0xc7, 0xaf,
+    0xf7, 0x38, 0x3f, 0x83, 0x3a, 0xcb, 0xae, 0x41,
+    0x75, 0x46, 0x63, 0xaa, 0x45, 0xb1, 0x2c, 0xd9,
+    0x9f, 0x17, 0x37, 0x02, 0x82, 0x01, 0x01, 0x00,
+    0xe1, 0xc1, 0x57, 0x4d, 0x0f, 0xa5, 0xea, 0x1d,
+    0x39, 0x9c, 0xe0, 0xf0, 0x6d, 0x13, 0x7f, 0x79,
+    0xdc, 0x72, 0x61, 0xc0, 0x7f, 0x88, 0xf6, 0x38,
+    0x4f, 0x49, 0x06, 0x1e, 0xb8, 0x6c, 0x21, 0x04,
+    0x60, 0x76, 0x5a, 0x6d, 0x04, 0xd1, 0x6d, 0xac,
+    0x7c, 0x25, 0x4f, 0x32, 0xcb, 0xbc, 0xf8, 0x4a,
+    0x22, 0x8f, 0xf5, 0x41, 0xfd, 0x1c, 0x76, 0x30,
+    0xc2, 0x5f, 0x99, 0x13, 0x5c, 0x57, 0x0f, 0xfd,
+    0xac, 0x0b, 0x10, 0x9a, 0x4f, 0x78, 0x0a, 0x86,
+    0xe8, 0x07, 0x40, 0x40, 0x13, 0xba, 0x96, 0x07,
+    0xd5, 0x39, 0x91, 0x51, 0x3e, 0x80, 0xd8, 0xa0,
+    0x1f, 0xff, 0xdc, 0x9e, 0x09, 0x3b, 0xae, 0x38,
+    0xa9, 0xc2, 0x14, 0x7b, 0xee, 0xd2, 0x69, 0x3d,
+    0xd6, 0x26, 0x74, 0x72, 0x7b, 0x86, 0xd4, 0x13,
+    0x5b, 0xb8, 0x76, 0x4b, 0x08, 0xfb, 0x93, 0xfa,
+    0x44, 0xaf, 0x98, 0x3b, 0xfa, 0xd0, 0x2a, 0x04,
+    0x8b, 0xb3, 0x3c, 0x6d, 0x32, 0xf7, 0x18, 0x6a,
+    0x51, 0x0e, 0x40, 0x90, 0xce, 0x8e, 0xdf, 0xe8,
+    0x07, 0x4c, 0x0f, 0xc7, 0xc8, 0xc2, 0x18, 0x58,
+    0x6a, 0x01, 0xc8, 0x27, 0xd6, 0x43, 0x2a, 0xfb,
+    0xa5, 0x34, 0x01, 0x3c, 0x72, 0xb1, 0x48, 0xce,
+    0x2b, 0x9b, 0xb4, 0x69, 0xd9, 0x82, 0xf8, 0xbe,
+    0x29, 0x88, 0x75, 0x96, 0xd8, 0xef, 0x78, 0x2a,
+    0x07, 0x90, 0xa0, 0x56, 0x33, 0x42, 0x05, 0x19,
+    0xb0, 0x69, 0x34, 0xf9, 0x03, 0xc5, 0xa8, 0x0d,
+    0x72, 0xa2, 0x27, 0xb4, 0x45, 0x6d, 0xd2, 0x01,
+    0x6c, 0xf1, 0x74, 0x51, 0x0a, 0x9a, 0xe2, 0xc1,
+    0x96, 0x80, 0x30, 0x0e, 0xc6, 0xa9, 0x79, 0xf7,
+    0x6f, 0xaf, 0xf6, 0xe8, 0x2a, 0xcc, 0xbd, 0xad,
+    0x8f, 0xe0, 0x32, 0x87, 0x85, 0x49, 0x68, 0x88,
+    0x15, 0x5c, 0xdb, 0x48, 0x40, 0xa2, 0xfa, 0x42,
+    0xe8, 0x4e, 0x3e, 0xe2, 0x3f, 0xe0, 0xf3, 0x99,
+    0x02, 0x82, 0x01, 0x00, 0x08, 0x39, 0x97, 0x69,
+    0x6d, 0x44, 0x5b, 0x2c, 0x74, 0xf6, 0x5f, 0x40,
+    0xe9, 0x1d, 0x24, 0x89, 0x1c, 0xaa, 0x9b, 0x8e,
+    0x8b, 0x65, 0x02, 0xe4, 0xb5, 0x6c, 0x26, 0x32,
+    0x98, 0xfb, 0x66, 0xe0, 0xfd, 0xef, 0xfe, 0x0f,
+    0x41, 0x4a, 0x5c, 0xc4, 0xdf, 0xdf, 0x42, 0xa1,
+    0x35, 0x46, 0x5e, 0x5b, 0xdd, 0x0c, 0x78, 0xbd,
+    0x41, 0xb0, 0xa2, 0xdf, 0x68, 0xab, 0x23, 0xfc,
+    0xa9, 0xac, 0xbd, 0xba, 0xd6, 0x54, 0x07, 0xc0,
+    0x21, 0xa7, 0x6a, 0x96, 0x24, 0xdf, 0x20, 0x46,
+    0x4d, 0x45, 0x27, 0x6c, 0x26, 0xea, 0x74, 0xeb,
+    0x98, 0x89, 0x90, 0xdd, 0x8e, 0x23, 0x49, 0xf5,
+    0xf7, 0x70, 0x9e, 0xb0, 0x5e, 0x10, 0x47, 0xe0,
+    0x9a, 0x28, 0x88, 0xdf, 0xdb, 0xd8, 0x53, 0x0b,
+    0x45, 0xf0, 0x19, 0x90, 0xe4, 0xdf, 0x02, 0x9f,
+    0x60, 0x4e, 0x76, 0x11, 0x3b, 0x39, 0x24, 0xf1,
+    0x3f, 0x3e, 0xb4, 0x8a, 0x1b, 0x84, 0xb7, 0x96,
+    0xdf, 0xfb, 0xb0, 0xda, 0xec, 0x63, 0x68, 0x15,
+    0xd7, 0xa9, 0xdb, 0x48, 0x9c, 0x12, 0xc3, 0xd6,
+    0x85, 0xe8, 0x63, 0x1f, 0xd0, 0x1a, 0xb0, 0x12,
+    0x60, 0x62, 0x43, 0xc1, 0x38, 0x86, 0x52, 0x23,
+    0x7f, 0xc9, 0x62, 0xf8, 0x79, 0xbf, 0xb4, 0xfb,
+    0x4e, 0x7e, 0x07, 0x22, 0x49, 0x8e, 0xbe, 0x6c,
+    0xf0, 0x53, 0x5a, 0x53, 0xfd, 0x3c, 0x14, 0xd8,
+    0xf7, 0x2c, 0x06, 0x2a, 0xe4, 0x64, 0xfd, 0x19,
+    0x57, 0xa0, 0x92, 0xf6, 0xa3, 0x42, 0x47, 0x61,
+    0x0b, 0xfd, 0x71, 0x5f, 0x98, 0xe2, 0x6c, 0x98,
+    0xa8, 0xf9, 0xf9, 0x7f, 0x1c, 0x61, 0x5d, 0x8c,
+    0xd1, 0xfb, 0x90, 0x28, 0x32, 0x9b, 0x7d, 0x82,
+    0xf9, 0xcc, 0x47, 0xbe, 0xc7, 0x67, 0xc5, 0x93,
+    0x22, 0x55, 0x0d, 0xd2, 0x73, 0xbe, 0xea, 0xed,
+    0x4d, 0xb5, 0xf4, 0xc2, 0x25, 0x92, 0x44, 0x30,
+    0xeb, 0xaa, 0x13, 0x11, 0x02, 0x82, 0x01, 0x01,
+    0x00, 0x82, 0x42, 0x02, 0x53, 0x4e, 0x72, 0x16,
+    0xf1, 0x21, 0xea, 0xe8, 0xc7, 0x10, 0xc8, 0xad,
+    0x46, 0xec, 0xf1, 0x7a, 0x81, 0x8d, 0x94, 0xc3,
+    0x2c, 0x9e, 0x62, 0xae, 0x0b, 0x4f, 0xb1, 0xe4,
+    0x23, 0x18, 0x5d, 0x71, 0xb3, 0x71, 0x92, 0x3d,
+    0x4b, 0xc6, 0x9d, 0xe8, 0x62, 0x90, 0xb7, 0xca,
+    0x33, 0x4c, 0x59, 0xef, 0xd3, 0x51, 0x6d, 0xf8,
+    0xac, 0x0d, 0x9b, 0x07, 0x41, 0xea, 0x87, 0xb9,
+    0x8c, 0x4e, 0x96, 0x5b, 0xd0, 0x0d, 0x86, 0x5f,
+    0xdc, 0x93, 0x48, 0x8b, 0xc3, 0xed, 0x1e, 0x3d,
+    0xae, 0xeb, 0x52, 0xba, 0x0c, 0x3c, 0x9a, 0x2f,
+    0x63, 0xc4, 0xd2, 0xe6, 0xc2, 0xb0, 0xe5, 0x24,
+    0x93, 0x41, 0x2f, 0xe0, 0x8d, 0xd9, 0xb0, 0xc2,
+    0x54, 0x91, 0x99, 0xc2, 0x9a, 0xc3, 0xb7, 0x79,
+    0xea, 0x69, 0x83, 0xb7, 0x8d, 0x77, 0xf3, 0x60,
+    0xe0, 0x88, 0x7d, 0x20, 0xc3, 0x8a, 0xe6, 0x4d,
+    0x38, 0x2e, 0x3b, 0x0e, 0xe4, 0x9b, 0x01, 0x83,
+    0xae, 0xe4, 0x71, 0xea, 0xc3, 0x22, 0xcb, 0xc1,
+    0x59, 0xa9, 0xcc, 0x33, 0x56, 0xbc, 0xf9, 0x70,
+    0xfe, 0xa2, 0xbb, 0xc0, 0x77, 0x6b, 0xe3, 0x79,
+    0x8b, 0x95, 0x38, 0xba, 0x75, 0xdc, 0x5f, 0x7a,
+    0x78, 0xab, 0x24, 0xbe, 0x26, 0x4d, 0x00, 0x8a,
+    0xf1, 0x7e, 0x19, 0x64, 0x6f, 0xd3, 0x5f, 0xe8,
+    0xdf, 0xa7, 0x59, 0xc5, 0x89, 0xb7, 0x2d, 0xa2,
+    0xaf, 0xbd, 0xe0, 0x16, 0x56, 0x8f, 0xdc, 0x9e,
+    0x28, 0x94, 0x3a, 0x07, 0xda, 0xb6, 0x2c, 0xb5,
+    0x7d, 0x69, 0x14, 0xb0, 0x5e, 0x8a, 0x55, 0xef,
+    0xfc, 0x6f, 0x10, 0x2b, 0xaa, 0x7a, 0xea, 0x12,
+    0x9b, 0xb8, 0x6f, 0xb9, 0x71, 0x20, 0x30, 0xde,
+    0x48, 0xa4, 0xb9, 0x61, 0xae, 0x5c, 0x33, 0x8d,
+    0x02, 0xe8, 0x00, 0x99, 0xed, 0xc8, 0x8d, 0xc1,
+    0x04, 0x95, 0xf1, 0x7f, 0xcb, 0x1f, 0xbc, 0x76,
+    0x11, 0x02, 0x82, 0x01, 0x00, 0x2d, 0x0c, 0xa9,
+    0x8f, 0x11, 0xc2, 0xf3, 0x02, 0xc8, 0xf2, 0x55,
+    0xc5, 0x6d, 0x25, 0x88, 0xba, 0x59, 0xf6, 0xd1,
+    0xdb, 0x94, 0x2f, 0x0b, 0x65, 0x2c, 0xad, 0x54,
+    0xe0, 0x2b, 0xe6, 0xa3, 0x49, 0xa2, 0xb3, 0xca,
+    0xd7, 0xec, 0x27, 0x32, 0xbb, 0xa4, 0x16, 0x90,
+    0xbb, 0x67, 0xad, 0x1b, 0xb9, 0x0f, 0x78, 0xcb,
+    0xad, 0x5c, 0xc3, 0x66, 0xd6, 0xbb, 0x97, 0x28,
+    0x01, 0x31, 0xf9, 0x0f, 0x71, 0x2a, 0xb9, 0x5b,
+    0xea, 0x34, 0x49, 0x9c, 0x6b, 0x13, 0x40, 0x65,
+    0xbd, 0x18, 0x0a, 0x14, 0xf9, 0x33, 0x47, 0xe8,
+    0x9f, 0x64, 0x0e, 0x24, 0xf6, 0xbb, 0x90, 0x23,
+    0x66, 0x01, 0xa6, 0xa4, 0xa9, 0x7f, 0x64, 0x51,
+    0xa3, 0x8a, 0x73, 0xc1, 0x80, 0xaf, 0x7a, 0x49,
+    0x75, 0x5d, 0x56, 0x1c, 0xaa, 0x3f, 0x64, 0xa9,
+    0x96, 0xfd, 0xb0, 0x90, 0xc5, 0xe0, 0x3d, 0x36,
+    0x05, 0xad, 0xad, 0x84, 0x93, 0x84, 0xab, 0x1b,
+    0x34, 0x57, 0x39, 0xae, 0x0e, 0x80, 0x0f, 0x4a,
+    0x9b, 0x32, 0x56, 0xbd, 0x30, 0xeb, 0xd1, 0xc8,
+    0xc4, 0x9f, 0x9c, 0x07, 0xb6, 0x05, 0xb1, 0x21,
+    0x7f, 0x69, 0x92, 0x9f, 0xb7, 0x68, 0xe7, 0xde,
+    0xb7, 0xbc, 0xb4, 0x89, 0x5b, 0x1c, 0x1b, 0x48,
+    0xd1, 0x44, 0x6e, 0xd7, 0x6b, 0xe2, 0xa1, 0xf4,
+    0xbf, 0x17, 0xb4, 0x43, 0x70, 0x26, 0xd4, 0xb9,
+    0xf5, 0x19, 0x09, 0x08, 0xe9, 0xa3, 0x49, 0x7d,
+    0x2f, 0xdc, 0xe8, 0x75, 0x79, 0xa1, 0xc1, 0x70,
+    0x1b, 0x60, 0x97, 0xaf, 0x0c, 0x56, 0x68, 0xac,
+    0x0e, 0x53, 0xbe, 0x56, 0xf4, 0xc3, 0xb1, 0xfb,
+    0xfb, 0xff, 0x73, 0x5b, 0xa7, 0xf6, 0x99, 0x0e,
+    0x14, 0x5a, 0x5f, 0x9d, 0xbd, 0x8e, 0x94, 0xec,
+    0x8b, 0x38, 0x72, 0xbc, 0x8b, 0xca, 0x32, 0xa8,
+    0x39, 0x43, 0xb1, 0x1d, 0x43, 0x29, 0xbe, 0x60,
+    0xdb, 0x91, 0x6c, 0x9c, 0x06,
+};
diff --git a/tests/unit/meson.build b/tests/unit/meson.build
index 5a7993e56c..287b367ec3 100644
--- a/tests/unit/meson.build
+++ b/tests/unit/meson.build
@@ -77,6 +77,7 @@ if have_block
     'test-crypto-hash': [crypto],
     'test-crypto-hmac': [crypto],
     'test-crypto-cipher': [crypto],
+    'test-crypto-akcipher': [crypto],
     'test-crypto-secret': [crypto, keyutils],
     'test-crypto-der': [crypto],
     'test-authz-simple': [authz],
diff --git a/tests/unit/test-crypto-akcipher.c b/tests/unit/test-crypto-akcipher.c
new file mode 100644
index 0000000000..b5be563884
--- /dev/null
+++ b/tests/unit/test-crypto-akcipher.c
@@ -0,0 +1,711 @@
+/*
+ * QEMU Crypto cipher algorithms
+ *
+ * Copyright (c) 2022 Bytedance
+ * Author: lei he <helei.sig11@bytedance.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "qemu/osdep.h"
+
+#include "crypto/init.h"
+#include "crypto/akcipher.h"
+#include "qapi/error.h"
+
+static const uint8_t rsa1024_private_key[] = {
+    0x30, 0x82, 0x02, 0x5c, 0x02, 0x01, 0x00, 0x02,
+    0x81, 0x81, 0x00, 0xe6, 0x4d, 0x76, 0x4f, 0xb2,
+    0x97, 0x09, 0xad, 0x9d, 0x17, 0x33, 0xf2, 0x30,
+    0x42, 0x83, 0xa9, 0xcb, 0x49, 0xa4, 0x2e, 0x59,
+    0x5e, 0x75, 0x51, 0xd1, 0xac, 0xc8, 0x86, 0x3e,
+    0xdb, 0x72, 0x2e, 0xb2, 0xf7, 0xc3, 0x5b, 0xc7,
+    0xea, 0xed, 0x30, 0xd1, 0xf7, 0x37, 0xee, 0x9d,
+    0x36, 0x59, 0x6f, 0xf8, 0xce, 0xc0, 0x5c, 0x82,
+    0x80, 0x37, 0x83, 0xd7, 0x45, 0x6a, 0xe9, 0xea,
+    0xc5, 0x3a, 0x59, 0x6b, 0x34, 0x31, 0x44, 0x00,
+    0x74, 0xa7, 0x29, 0xab, 0x79, 0x4a, 0xbd, 0xe8,
+    0x25, 0x35, 0x01, 0x11, 0x40, 0xbf, 0x31, 0xbd,
+    0xd3, 0xe0, 0x68, 0x1e, 0xd5, 0x5b, 0x2f, 0xe9,
+    0x20, 0xf2, 0x9f, 0x46, 0x35, 0x30, 0xa8, 0xf1,
+    0xfe, 0xef, 0xd8, 0x76, 0x23, 0x46, 0x34, 0x70,
+    0xa1, 0xce, 0xc6, 0x65, 0x6d, 0xb0, 0x94, 0x7e,
+    0xe5, 0x92, 0x45, 0x7b, 0xaa, 0xbb, 0x95, 0x97,
+    0x77, 0xcd, 0xd3, 0x02, 0x03, 0x01, 0x00, 0x01,
+    0x02, 0x81, 0x80, 0x30, 0x6a, 0xc4, 0x9e, 0xc8,
+    0xba, 0xfc, 0x2b, 0xe5, 0xc4, 0xc5, 0x04, 0xfb,
+    0xa4, 0x60, 0x2d, 0xc8, 0x31, 0x39, 0x35, 0x0d,
+    0x50, 0xd0, 0x75, 0x5d, 0x11, 0x68, 0x2e, 0xe0,
+    0xf4, 0x1d, 0xb3, 0x37, 0xa8, 0xe3, 0x07, 0x5e,
+    0xa6, 0x43, 0x2b, 0x6a, 0x59, 0x01, 0x07, 0x47,
+    0x41, 0xef, 0xd7, 0x9c, 0x85, 0x4a, 0xe7, 0xa7,
+    0xff, 0xf0, 0xab, 0xe5, 0x0c, 0x11, 0x08, 0x10,
+    0x75, 0x5a, 0x68, 0xa0, 0x08, 0x03, 0xc9, 0x40,
+    0x79, 0x67, 0x1d, 0x65, 0x89, 0x2d, 0x08, 0xf9,
+    0xb5, 0x1b, 0x7d, 0xd2, 0x41, 0x3b, 0x33, 0xf2,
+    0x47, 0x2f, 0x9c, 0x0b, 0xd5, 0xaf, 0xcb, 0xdb,
+    0xbb, 0x37, 0x63, 0x03, 0xf8, 0xe7, 0x2e, 0xc7,
+    0x3c, 0x86, 0x9f, 0xc2, 0x9b, 0xb4, 0x70, 0x6a,
+    0x4d, 0x7c, 0xe4, 0x1b, 0x3a, 0xa9, 0xae, 0xd7,
+    0xce, 0x7f, 0x56, 0xc2, 0x73, 0x5e, 0x58, 0x63,
+    0xd5, 0x86, 0x41, 0x02, 0x41, 0x00, 0xf6, 0x56,
+    0x69, 0xec, 0xef, 0x65, 0x95, 0xdc, 0x25, 0x47,
+    0xe0, 0x6f, 0xb0, 0x4f, 0x79, 0x77, 0x0a, 0x5e,
+    0x46, 0xcb, 0xbd, 0x0b, 0x71, 0x51, 0x2a, 0xa4,
+    0x65, 0x29, 0x18, 0xc6, 0x30, 0xa0, 0x95, 0x4c,
+    0x4b, 0xbe, 0x8c, 0x40, 0xe3, 0x9c, 0x23, 0x02,
+    0x14, 0x43, 0xe9, 0x64, 0xea, 0xe3, 0xa8, 0xe2,
+    0x1a, 0xd5, 0xf9, 0x5c, 0xe0, 0x36, 0x2c, 0x97,
+    0xda, 0xd5, 0xc7, 0x46, 0xce, 0x11, 0x02, 0x41,
+    0x00, 0xef, 0x56, 0x08, 0xb8, 0x29, 0xa5, 0xa6,
+    0x7c, 0xf7, 0x5f, 0xb4, 0xf5, 0x63, 0xe7, 0xeb,
+    0x45, 0xfd, 0x89, 0xaa, 0x94, 0xa6, 0x3d, 0x0b,
+    0xd9, 0x04, 0x6f, 0x78, 0xe0, 0xbb, 0xa2, 0xd4,
+    0x29, 0x83, 0x17, 0x95, 0x6f, 0x50, 0x3d, 0x40,
+    0x5d, 0xe5, 0x24, 0xda, 0xc2, 0x23, 0x50, 0x86,
+    0xa8, 0x34, 0xc8, 0x6f, 0xec, 0x7f, 0xb6, 0x45,
+    0x3a, 0xdd, 0x78, 0x9b, 0xee, 0xa1, 0xe4, 0x09,
+    0xa3, 0x02, 0x40, 0x5c, 0xd6, 0x66, 0x67, 0x58,
+    0x35, 0xc5, 0xcb, 0xc8, 0xf5, 0x14, 0xbd, 0xa3,
+    0x09, 0xe0, 0xb2, 0x1f, 0x63, 0x36, 0x75, 0x34,
+    0x52, 0xea, 0xaa, 0xf7, 0x52, 0x2b, 0x99, 0xd8,
+    0x6f, 0x61, 0x06, 0x34, 0x1e, 0x23, 0xf1, 0xb5,
+    0x34, 0x03, 0x53, 0xe5, 0xd1, 0xb3, 0xc7, 0x80,
+    0x5f, 0x7b, 0x32, 0xbf, 0x84, 0x2f, 0x2e, 0xf3,
+    0x22, 0xb0, 0x91, 0x5a, 0x2f, 0x04, 0xd7, 0x4a,
+    0x9a, 0x01, 0xb1, 0x02, 0x40, 0x34, 0x0b, 0x26,
+    0x4c, 0x3d, 0xaa, 0x2a, 0xc0, 0xe3, 0xdd, 0xe8,
+    0xf0, 0xaf, 0x6f, 0xe0, 0x06, 0x51, 0x32, 0x9d,
+    0x68, 0x43, 0x99, 0xe4, 0xb8, 0xa5, 0x31, 0x44,
+    0x3c, 0xc2, 0x30, 0x8f, 0x28, 0x13, 0xbc, 0x8e,
+    0x1f, 0x2d, 0x78, 0x94, 0x45, 0x96, 0xad, 0x63,
+    0xf0, 0x71, 0x53, 0x72, 0x64, 0xa3, 0x4d, 0xae,
+    0xa0, 0xe3, 0xc8, 0x93, 0xd7, 0x50, 0x0f, 0x89,
+    0x00, 0xe4, 0x2d, 0x3d, 0x37, 0x02, 0x41, 0x00,
+    0xbe, 0xa6, 0x08, 0xe0, 0xc8, 0x15, 0x2a, 0x47,
+    0xcb, 0xd5, 0xec, 0x93, 0xd3, 0xaa, 0x12, 0x82,
+    0xaf, 0xac, 0x51, 0x5a, 0x5b, 0xa7, 0x93, 0x4b,
+    0xb9, 0xab, 0x00, 0xfa, 0x5a, 0xea, 0x34, 0xe4,
+    0x80, 0xf1, 0x44, 0x6a, 0x65, 0xe4, 0x33, 0x99,
+    0xfb, 0x54, 0xd7, 0x89, 0x5a, 0x1b, 0xd6, 0x2b,
+    0xcc, 0x6e, 0x4b, 0x19, 0xa0, 0x6d, 0x93, 0x9f,
+    0xc3, 0x91, 0x7a, 0xa5, 0xd8, 0x59, 0x0e, 0x9e,
+};
+
+static const uint8_t rsa1024_public_key[] = {
+    0x30, 0x81, 0x89, 0x02, 0x81, 0x81, 0x00, 0xe6,
+    0x4d, 0x76, 0x4f, 0xb2, 0x97, 0x09, 0xad, 0x9d,
+    0x17, 0x33, 0xf2, 0x30, 0x42, 0x83, 0xa9, 0xcb,
+    0x49, 0xa4, 0x2e, 0x59, 0x5e, 0x75, 0x51, 0xd1,
+    0xac, 0xc8, 0x86, 0x3e, 0xdb, 0x72, 0x2e, 0xb2,
+    0xf7, 0xc3, 0x5b, 0xc7, 0xea, 0xed, 0x30, 0xd1,
+    0xf7, 0x37, 0xee, 0x9d, 0x36, 0x59, 0x6f, 0xf8,
+    0xce, 0xc0, 0x5c, 0x82, 0x80, 0x37, 0x83, 0xd7,
+    0x45, 0x6a, 0xe9, 0xea, 0xc5, 0x3a, 0x59, 0x6b,
+    0x34, 0x31, 0x44, 0x00, 0x74, 0xa7, 0x29, 0xab,
+    0x79, 0x4a, 0xbd, 0xe8, 0x25, 0x35, 0x01, 0x11,
+    0x40, 0xbf, 0x31, 0xbd, 0xd3, 0xe0, 0x68, 0x1e,
+    0xd5, 0x5b, 0x2f, 0xe9, 0x20, 0xf2, 0x9f, 0x46,
+    0x35, 0x30, 0xa8, 0xf1, 0xfe, 0xef, 0xd8, 0x76,
+    0x23, 0x46, 0x34, 0x70, 0xa1, 0xce, 0xc6, 0x65,
+    0x6d, 0xb0, 0x94, 0x7e, 0xe5, 0x92, 0x45, 0x7b,
+    0xaa, 0xbb, 0x95, 0x97, 0x77, 0xcd, 0xd3, 0x02,
+    0x03, 0x01, 0x00, 0x01,
+};
+
+static const uint8_t rsa2048_private_key[] = {
+    0x30, 0x82, 0x04, 0xa4, 0x02, 0x01, 0x00, 0x02,
+    0x82, 0x01, 0x01, 0x00, 0xbd, 0x9c, 0x83, 0x6b,
+    0x0e, 0x8e, 0xcf, 0xfa, 0xaa, 0x4f, 0x6a, 0xf4,
+    0xe3, 0x52, 0x0f, 0xa5, 0xd0, 0xbe, 0x5e, 0x7f,
+    0x08, 0x24, 0xba, 0x87, 0x46, 0xfb, 0x28, 0x93,
+    0xe5, 0xe5, 0x81, 0x42, 0xc0, 0xf9, 0x17, 0xc7,
+    0x81, 0x01, 0xf4, 0x18, 0x6a, 0x17, 0xf5, 0x57,
+    0x20, 0x37, 0xcf, 0xf9, 0x74, 0x5e, 0xe1, 0x48,
+    0x6a, 0x71, 0x0a, 0x0f, 0x79, 0x72, 0x2b, 0x46,
+    0x10, 0x53, 0xdc, 0x14, 0x43, 0xbd, 0xbc, 0x6d,
+    0x15, 0x6f, 0x15, 0x4e, 0xf0, 0x0d, 0x89, 0x39,
+    0x02, 0xc3, 0x68, 0x5c, 0xa8, 0xfc, 0xed, 0x64,
+    0x9d, 0x98, 0xb7, 0xcd, 0x83, 0x66, 0x93, 0xc3,
+    0xd9, 0x57, 0xa0, 0x21, 0x93, 0xad, 0x5c, 0x75,
+    0x69, 0x88, 0x9e, 0x81, 0xdc, 0x7f, 0x1d, 0xd5,
+    0xbd, 0x1c, 0xc1, 0x30, 0x56, 0xa5, 0xda, 0x99,
+    0x46, 0xa6, 0x6d, 0x0e, 0x6f, 0x5e, 0x51, 0x34,
+    0x49, 0x73, 0xc3, 0x67, 0x49, 0x7e, 0x21, 0x2a,
+    0x20, 0xa7, 0x2b, 0x92, 0x73, 0x1d, 0xa5, 0x25,
+    0x2a, 0xd0, 0x3a, 0x89, 0x75, 0xb2, 0xbb, 0x19,
+    0x37, 0x78, 0x48, 0xd2, 0xf2, 0x2a, 0x6d, 0x9e,
+    0xc6, 0x26, 0xca, 0x46, 0x8c, 0xf1, 0x42, 0x2a,
+    0x31, 0xb2, 0xfc, 0xe7, 0x55, 0x51, 0xff, 0x07,
+    0x13, 0x5b, 0x36, 0x59, 0x2b, 0x43, 0x30, 0x4b,
+    0x05, 0x5c, 0xd2, 0x45, 0xa0, 0xa0, 0x7c, 0x17,
+    0x5b, 0x07, 0xbb, 0x5d, 0x83, 0x80, 0x92, 0x6d,
+    0x87, 0x1a, 0x43, 0xac, 0xc7, 0x6b, 0x8d, 0x11,
+    0x60, 0x27, 0xd2, 0xdf, 0xdb, 0x71, 0x02, 0x55,
+    0x6e, 0xb5, 0xca, 0x4d, 0xda, 0x59, 0x0d, 0xb8,
+    0x8c, 0xcd, 0xd3, 0x0e, 0x55, 0xa0, 0xa4, 0x8d,
+    0xa0, 0x14, 0x10, 0x48, 0x42, 0x35, 0x56, 0x08,
+    0xf7, 0x29, 0x5f, 0xa2, 0xea, 0xa4, 0x5e, 0x8e,
+    0x99, 0x56, 0xaa, 0x5a, 0x8c, 0x23, 0x8f, 0x35,
+    0x22, 0x8a, 0xff, 0xed, 0x02, 0x03, 0x01, 0x00,
+    0x01, 0x02, 0x82, 0x01, 0x00, 0x4e, 0x4a, 0xf3,
+    0x44, 0xe0, 0x64, 0xfd, 0xe1, 0xde, 0x33, 0x1e,
+    0xd1, 0xf1, 0x8f, 0x6f, 0xe0, 0xa2, 0xfa, 0x08,
+    0x60, 0xe1, 0xc6, 0xf0, 0xb2, 0x6d, 0x0f, 0xc6,
+    0x28, 0x93, 0xb4, 0x19, 0x94, 0xab, 0xc3, 0xef,
+    0x1a, 0xb4, 0xdd, 0x4e, 0xa2, 0x4a, 0x24, 0x8c,
+    0x6c, 0xa6, 0x64, 0x05, 0x5f, 0x56, 0xba, 0xda,
+    0xc1, 0x21, 0x1a, 0x7d, 0xf1, 0xf7, 0xce, 0xb9,
+    0xa9, 0x9b, 0x92, 0x54, 0xfc, 0x95, 0x20, 0x22,
+    0x4e, 0xd4, 0x9b, 0xe2, 0xab, 0x8e, 0x99, 0xb8,
+    0x40, 0xaf, 0x30, 0x6a, 0xc6, 0x60, 0x0c, 0xd8,
+    0x25, 0x44, 0xa1, 0xcb, 0xbb, 0x73, 0x77, 0x86,
+    0xaa, 0x46, 0xf3, 0x54, 0xae, 0xa8, 0xa0, 0xdb,
+    0xdd, 0xab, 0x6e, 0xfb, 0x2c, 0x5a, 0x14, 0xaf,
+    0x08, 0x13, 0xa7, 0x6c, 0xe9, 0xfd, 0xcd, 0x4c,
+    0x1f, 0x20, 0x3a, 0x16, 0x2b, 0xf0, 0xb6, 0x7c,
+    0x47, 0x5f, 0xd1, 0x0a, 0x2c, 0xc4, 0xa5, 0x68,
+    0xd0, 0x43, 0x75, 0x6b, 0x65, 0xaa, 0x32, 0xc6,
+    0x99, 0x06, 0xcb, 0x8f, 0xe6, 0x8d, 0xce, 0xbf,
+    0x4d, 0x0d, 0x7b, 0x22, 0x2a, 0x8a, 0xcb, 0x7d,
+    0x7f, 0x16, 0x48, 0x85, 0xf1, 0x86, 0xcb, 0x54,
+    0xb9, 0x39, 0xd4, 0xbc, 0xe3, 0x2d, 0x27, 0x59,
+    0xf6, 0x81, 0x5e, 0x94, 0x45, 0xdf, 0xb9, 0x22,
+    0xaf, 0x64, 0x0d, 0x14, 0xec, 0x8c, 0xeb, 0x71,
+    0xac, 0xee, 0x09, 0x4c, 0xbf, 0x34, 0xf9, 0xf4,
+    0x66, 0x77, 0x36, 0x3b, 0x41, 0x74, 0x01, 0x4f,
+    0xfc, 0x56, 0x83, 0xba, 0x14, 0xb0, 0x2f, 0xdd,
+    0x4d, 0xb9, 0x3f, 0xdf, 0x71, 0xbe, 0x7b, 0xba,
+    0x66, 0xc8, 0xc5, 0x42, 0xc9, 0xba, 0x18, 0x63,
+    0x45, 0x07, 0x2f, 0x84, 0x3e, 0xc3, 0xfb, 0x47,
+    0xda, 0xd4, 0x1d, 0x0e, 0x9d, 0x96, 0xc0, 0xea,
+    0xee, 0x45, 0x2f, 0xe1, 0x62, 0x23, 0xee, 0xef,
+    0x3d, 0x5e, 0x55, 0xa1, 0x0d, 0x02, 0x81, 0x81,
+    0x00, 0xeb, 0x76, 0x88, 0xd3, 0xae, 0x3f, 0x1d,
+    0xf2, 0x49, 0xe0, 0x37, 0x49, 0x83, 0x82, 0x6c,
+    0xf7, 0xf1, 0x17, 0x30, 0x75, 0x2e, 0x89, 0x06,
+    0x88, 0x56, 0x32, 0xf6, 0xfa, 0x58, 0xcb, 0x3c,
+    0x98, 0x67, 0xc3, 0xde, 0x10, 0x82, 0xe5, 0xfa,
+    0xfa, 0x52, 0x47, 0x8d, 0xd7, 0x00, 0xc6, 0xcb,
+    0xf7, 0xf6, 0x57, 0x9b, 0x6e, 0x0c, 0xac, 0xe8,
+    0x3b, 0xd1, 0xde, 0xb5, 0x34, 0xaf, 0x8b, 0x2a,
+    0xb0, 0x2d, 0x01, 0xeb, 0x7c, 0xa0, 0x42, 0x26,
+    0xbb, 0x2b, 0x43, 0x0e, 0x1d, 0xe2, 0x4e, 0xc9,
+    0xc1, 0x0a, 0x67, 0x1d, 0xfc, 0x83, 0x25, 0xce,
+    0xb2, 0x18, 0xd9, 0x0d, 0x70, 0xf5, 0xa3, 0x5a,
+    0x9c, 0x99, 0xdd, 0x47, 0xa1, 0x57, 0xe7, 0x20,
+    0xde, 0xa1, 0x29, 0x8d, 0x96, 0x62, 0xf9, 0x26,
+    0x95, 0x51, 0xa6, 0xe7, 0x09, 0x8b, 0xba, 0x16,
+    0x8b, 0x19, 0x5b, 0xf9, 0x27, 0x0d, 0xc5, 0xd6,
+    0x5f, 0x02, 0x81, 0x81, 0x00, 0xce, 0x26, 0x31,
+    0xb5, 0x43, 0x53, 0x95, 0x39, 0xdd, 0x01, 0x98,
+    0x8b, 0x3d, 0x27, 0xeb, 0x0b, 0x87, 0x1c, 0x95,
+    0xfc, 0x3e, 0x36, 0x51, 0x31, 0xb5, 0xea, 0x59,
+    0x56, 0xc0, 0x97, 0x62, 0xf0, 0x63, 0x2b, 0xb6,
+    0x30, 0x9b, 0xdf, 0x19, 0x10, 0xe9, 0xa0, 0x3d,
+    0xea, 0x54, 0x5a, 0xe6, 0xc6, 0x9e, 0x7e, 0xb5,
+    0xf0, 0xb0, 0x54, 0xef, 0xc3, 0xe1, 0x47, 0xa6,
+    0x95, 0xc7, 0xe4, 0xa3, 0x4a, 0x30, 0x68, 0x24,
+    0x98, 0x7d, 0xc1, 0x34, 0xa9, 0xcb, 0xbc, 0x3c,
+    0x08, 0x9c, 0x7d, 0x0c, 0xa2, 0xb7, 0x60, 0xaa,
+    0x38, 0x08, 0x16, 0xa6, 0x7f, 0xdb, 0xd2, 0xb1,
+    0x67, 0xe7, 0x93, 0x8e, 0xbb, 0x7e, 0xb9, 0xb5,
+    0xd0, 0xd0, 0x9f, 0x7b, 0xcc, 0x46, 0xe6, 0x74,
+    0x78, 0x1a, 0x96, 0xd6, 0xd7, 0x74, 0x34, 0x54,
+    0x3b, 0x54, 0x55, 0x7f, 0x89, 0x81, 0xbc, 0x40,
+    0x55, 0x87, 0x24, 0x95, 0x33, 0x02, 0x81, 0x81,
+    0x00, 0xb0, 0x18, 0x5d, 0x2a, 0x1a, 0x95, 0x9f,
+    0x9a, 0xd5, 0x3f, 0x37, 0x79, 0xe6, 0x3d, 0x83,
+    0xab, 0x46, 0x86, 0x36, 0x3a, 0x5d, 0x0c, 0x23,
+    0x73, 0x91, 0x2b, 0xda, 0x63, 0xce, 0x46, 0x68,
+    0xd1, 0xfe, 0x40, 0x90, 0xf2, 0x3e, 0x43, 0x2b,
+    0x19, 0x4c, 0xb1, 0xb0, 0xd5, 0x8c, 0x02, 0x21,
+    0x07, 0x18, 0x17, 0xda, 0xe9, 0x49, 0xd7, 0x82,
+    0x73, 0x42, 0x78, 0xd1, 0x82, 0x4e, 0x8a, 0xc0,
+    0xe9, 0x33, 0x2f, 0xcd, 0x62, 0xce, 0x23, 0xca,
+    0xfd, 0x8d, 0xd4, 0x3f, 0x59, 0x80, 0x27, 0xb6,
+    0x61, 0x85, 0x9b, 0x2a, 0xe4, 0xef, 0x5c, 0x36,
+    0x22, 0x21, 0xcd, 0x2a, 0x6d, 0x41, 0x77, 0xe2,
+    0xcb, 0x5d, 0x93, 0x0d, 0x00, 0x10, 0x52, 0x8d,
+    0xd5, 0x92, 0x28, 0x16, 0x78, 0xd3, 0x1a, 0x4c,
+    0x8d, 0xbd, 0x9c, 0x1a, 0x0b, 0x9c, 0x91, 0x16,
+    0x4c, 0xff, 0x31, 0x36, 0xbb, 0xcb, 0x64, 0x1a,
+    0xf7, 0x02, 0x81, 0x80, 0x32, 0x65, 0x09, 0xdf,
+    0xca, 0xee, 0xa2, 0xdb, 0x3b, 0x58, 0xc9, 0x86,
+    0xb8, 0x53, 0x8a, 0xd5, 0x0d, 0x99, 0x82, 0x5c,
+    0xe0, 0x84, 0x7c, 0xc2, 0xcf, 0x3a, 0xd3, 0xce,
+    0x2e, 0x54, 0x93, 0xbe, 0x3a, 0x30, 0x14, 0x60,
+    0xbb, 0xaa, 0x05, 0x41, 0xaa, 0x2b, 0x1f, 0x17,
+    0xaa, 0xb9, 0x72, 0x12, 0xf9, 0xe9, 0xf5, 0xe6,
+    0x39, 0xe4, 0xf9, 0x9c, 0x03, 0xf5, 0x75, 0x16,
+    0xc6, 0x7f, 0xf1, 0x1f, 0x10, 0xc8, 0x54, 0xb1,
+    0xe6, 0x84, 0x15, 0xb0, 0xb0, 0x7a, 0x7a, 0x9e,
+    0x8c, 0x4a, 0xd1, 0x8c, 0xf1, 0x91, 0x32, 0xeb,
+    0x71, 0xa6, 0xbf, 0xdb, 0x1f, 0xcc, 0xd8, 0xcb,
+    0x92, 0xc3, 0xf2, 0xaf, 0x89, 0x22, 0x32, 0xfd,
+    0x32, 0x12, 0xda, 0xbb, 0xac, 0x55, 0x68, 0x01,
+    0x78, 0x56, 0x89, 0x7c, 0xb0, 0x0e, 0x9e, 0xcc,
+    0xc6, 0x28, 0x04, 0x7e, 0x83, 0xf5, 0x96, 0x30,
+    0x92, 0x51, 0xf2, 0x1b, 0x02, 0x81, 0x81, 0x00,
+    0x83, 0x6d, 0xd1, 0x98, 0x90, 0x41, 0x8c, 0xa7,
+    0x92, 0x83, 0xac, 0x89, 0x05, 0x0c, 0x79, 0x67,
+    0x90, 0xb6, 0xa1, 0xf3, 0x2f, 0xca, 0xf0, 0x15,
+    0xe0, 0x30, 0x58, 0xe9, 0x4f, 0xcb, 0x4c, 0x56,
+    0x56, 0x56, 0x14, 0x3f, 0x1b, 0x79, 0xb6, 0xef,
+    0x57, 0x4b, 0x28, 0xbd, 0xb0, 0xe6, 0x0c, 0x49,
+    0x4b, 0xbe, 0xe1, 0x57, 0x28, 0x2a, 0x23, 0x5e,
+    0xc4, 0xa2, 0x19, 0x4b, 0x00, 0x67, 0x78, 0xd9,
+    0x26, 0x6e, 0x17, 0x25, 0xce, 0xe4, 0xfd, 0xde,
+    0x86, 0xa8, 0x5a, 0x67, 0x47, 0x6b, 0x15, 0x09,
+    0xe1, 0xec, 0x8e, 0x62, 0x98, 0x91, 0x6f, 0xc0,
+    0x98, 0x0c, 0x70, 0x0e, 0x7d, 0xbe, 0x63, 0xbd,
+    0x12, 0x5a, 0x98, 0x1c, 0xe3, 0x0c, 0xfb, 0xc7,
+    0xfb, 0x1b, 0xbd, 0x02, 0x87, 0xcc, 0x0c, 0xbb,
+    0xc2, 0xd4, 0xb6, 0xc1, 0xa1, 0x23, 0xd3, 0x1e,
+    0x21, 0x6f, 0x48, 0xba, 0x0e, 0x2e, 0xc7, 0x42
+};
+
+static const uint8_t rsa2048_public_key[] = {
+    0x30, 0x82, 0x01, 0x0a, 0x02, 0x82, 0x01, 0x01,
+    0x00, 0xbd, 0x9c, 0x83, 0x6b, 0x0e, 0x8e, 0xcf,
+    0xfa, 0xaa, 0x4f, 0x6a, 0xf4, 0xe3, 0x52, 0x0f,
+    0xa5, 0xd0, 0xbe, 0x5e, 0x7f, 0x08, 0x24, 0xba,
+    0x87, 0x46, 0xfb, 0x28, 0x93, 0xe5, 0xe5, 0x81,
+    0x42, 0xc0, 0xf9, 0x17, 0xc7, 0x81, 0x01, 0xf4,
+    0x18, 0x6a, 0x17, 0xf5, 0x57, 0x20, 0x37, 0xcf,
+    0xf9, 0x74, 0x5e, 0xe1, 0x48, 0x6a, 0x71, 0x0a,
+    0x0f, 0x79, 0x72, 0x2b, 0x46, 0x10, 0x53, 0xdc,
+    0x14, 0x43, 0xbd, 0xbc, 0x6d, 0x15, 0x6f, 0x15,
+    0x4e, 0xf0, 0x0d, 0x89, 0x39, 0x02, 0xc3, 0x68,
+    0x5c, 0xa8, 0xfc, 0xed, 0x64, 0x9d, 0x98, 0xb7,
+    0xcd, 0x83, 0x66, 0x93, 0xc3, 0xd9, 0x57, 0xa0,
+    0x21, 0x93, 0xad, 0x5c, 0x75, 0x69, 0x88, 0x9e,
+    0x81, 0xdc, 0x7f, 0x1d, 0xd5, 0xbd, 0x1c, 0xc1,
+    0x30, 0x56, 0xa5, 0xda, 0x99, 0x46, 0xa6, 0x6d,
+    0x0e, 0x6f, 0x5e, 0x51, 0x34, 0x49, 0x73, 0xc3,
+    0x67, 0x49, 0x7e, 0x21, 0x2a, 0x20, 0xa7, 0x2b,
+    0x92, 0x73, 0x1d, 0xa5, 0x25, 0x2a, 0xd0, 0x3a,
+    0x89, 0x75, 0xb2, 0xbb, 0x19, 0x37, 0x78, 0x48,
+    0xd2, 0xf2, 0x2a, 0x6d, 0x9e, 0xc6, 0x26, 0xca,
+    0x46, 0x8c, 0xf1, 0x42, 0x2a, 0x31, 0xb2, 0xfc,
+    0xe7, 0x55, 0x51, 0xff, 0x07, 0x13, 0x5b, 0x36,
+    0x59, 0x2b, 0x43, 0x30, 0x4b, 0x05, 0x5c, 0xd2,
+    0x45, 0xa0, 0xa0, 0x7c, 0x17, 0x5b, 0x07, 0xbb,
+    0x5d, 0x83, 0x80, 0x92, 0x6d, 0x87, 0x1a, 0x43,
+    0xac, 0xc7, 0x6b, 0x8d, 0x11, 0x60, 0x27, 0xd2,
+    0xdf, 0xdb, 0x71, 0x02, 0x55, 0x6e, 0xb5, 0xca,
+    0x4d, 0xda, 0x59, 0x0d, 0xb8, 0x8c, 0xcd, 0xd3,
+    0x0e, 0x55, 0xa0, 0xa4, 0x8d, 0xa0, 0x14, 0x10,
+    0x48, 0x42, 0x35, 0x56, 0x08, 0xf7, 0x29, 0x5f,
+    0xa2, 0xea, 0xa4, 0x5e, 0x8e, 0x99, 0x56, 0xaa,
+    0x5a, 0x8c, 0x23, 0x8f, 0x35, 0x22, 0x8a, 0xff,
+    0xed, 0x02, 0x03, 0x01, 0x00, 0x01
+};
+
+static const uint8_t test_sha1_dgst[] = {
+    0x3c, 0x05, 0x19, 0x34, 0x29, 0x19, 0xc7, 0xe0,
+    0x87, 0xb6, 0x24, 0xf9, 0x58, 0xac, 0xa4, 0xd4,
+    0xb2, 0xd9, 0x03, 0x9e,
+};
+
+static const uint8_t exp_signature_rsa2048_pkcs1[] = {
+    0x4e, 0x82, 0x56, 0x4c, 0x84, 0x66, 0xca, 0x1e,
+    0xc6, 0x92, 0x46, 0x20, 0x02, 0x6b, 0x64, 0x46,
+    0x15, 0x6b, 0x24, 0xf2, 0xbb, 0xfa, 0x44, 0x3c,
+    0xaf, 0x42, 0xc8, 0x41, 0xfd, 0xce, 0xed, 0x95,
+    0x34, 0xaf, 0x25, 0x09, 0xd1, 0x06, 0x94, 0xaa,
+    0x52, 0xd4, 0x29, 0xc8, 0x52, 0x34, 0x67, 0x59,
+    0x4f, 0x5a, 0xfd, 0x23, 0x30, 0x5e, 0xc7, 0x1e,
+    0xa6, 0xe0, 0x1b, 0x23, 0xca, 0x82, 0x47, 0x9a,
+    0x2e, 0x2c, 0x66, 0x45, 0x5a, 0x12, 0xa9, 0x15,
+    0xbf, 0xd6, 0xd6, 0xfa, 0x8d, 0x60, 0x99, 0x89,
+    0x91, 0x39, 0x06, 0xb7, 0xd3, 0x9a, 0xef, 0x15,
+    0x7b, 0x95, 0x87, 0x77, 0x2c, 0x41, 0xd4, 0x71,
+    0xd5, 0xdf, 0x22, 0x7b, 0x01, 0xe2, 0xc1, 0xfb,
+    0xb9, 0x4e, 0x0c, 0x9b, 0xd5, 0x04, 0xed, 0x2b,
+    0x7e, 0x73, 0x53, 0xaa, 0x33, 0x89, 0x9d, 0x95,
+    0x28, 0x8f, 0x8b, 0x80, 0x34, 0x7a, 0xea, 0xe3,
+    0x66, 0x8a, 0xa8, 0xad, 0xed, 0x91, 0x43, 0xdd,
+    0x77, 0xe5, 0xd7, 0x16, 0xda, 0xa8, 0x00, 0x29,
+    0x3f, 0x9f, 0xe0, 0x1d, 0x42, 0x9d, 0x35, 0x5d,
+    0x0f, 0xf3, 0x90, 0x27, 0x3a, 0x8c, 0x46, 0x13,
+    0x53, 0x3e, 0x3b, 0x38, 0x77, 0xf8, 0x57, 0x61,
+    0xbc, 0xc4, 0x54, 0x68, 0x48, 0xae, 0x58, 0x03,
+    0x33, 0x94, 0x3f, 0x18, 0x1e, 0xb3, 0x3f, 0x79,
+    0xa7, 0x26, 0x92, 0x5d, 0x32, 0x2a, 0xdb, 0xe6,
+    0x3a, 0xe8, 0xd7, 0xaa, 0x91, 0xfe, 0x9f, 0x06,
+    0x26, 0x68, 0x8c, 0x27, 0x31, 0xb0, 0x04, 0x9e,
+    0x94, 0x79, 0x63, 0xa1, 0xc7, 0xe8, 0x5b, 0x8c,
+    0xd3, 0xf1, 0x88, 0x58, 0x31, 0x2f, 0x4e, 0x11,
+    0x00, 0xfe, 0x29, 0xad, 0x2c, 0xa9, 0x8e, 0x63,
+    0xd8, 0x7d, 0xc5, 0xa1, 0x71, 0xfa, 0x08, 0x29,
+    0xea, 0xd6, 0x6c, 0x53, 0x00, 0x52, 0xa0, 0xed,
+    0x6b, 0x7c, 0x67, 0x50, 0x71, 0x2d, 0x96, 0x7a,
+};
+
+static const uint8_t exp_signature_rsa1024_pkcs1[] = {
+    0x6b, 0x5b, 0xbb, 0x3b, 0x1f, 0x08, 0xd8, 0xc0,
+    0x4a, 0xf1, 0x5a, 0x12, 0xc2, 0x39, 0x14, 0x65,
+    0x4f, 0xda, 0x79, 0x67, 0xf2, 0x89, 0x25, 0xad,
+    0x9e, 0x7e, 0xba, 0xa8, 0x34, 0x15, 0x03, 0xdd,
+    0x80, 0x6b, 0x01, 0xd7, 0x4a, 0xf3, 0xd6, 0xef,
+    0x1e, 0x48, 0xf3, 0xbc, 0x75, 0x1a, 0xc4, 0x2c,
+    0x90, 0x15, 0x9f, 0x21, 0x24, 0x98, 0x21, 0xef,
+    0x6d, 0x3b, 0xf3, 0x82, 0x8f, 0x8d, 0xd8, 0x48,
+    0x37, 0x16, 0x19, 0x8e, 0x3c, 0x64, 0xa0, 0x9e,
+    0xf7, 0x0c, 0xd9, 0x5c, 0xc6, 0x13, 0xc4, 0x5f,
+    0xf8, 0xf3, 0x59, 0x5b, 0xd0, 0x33, 0x95, 0x98,
+    0xde, 0x67, 0x25, 0x58, 0x46, 0xba, 0xee, 0x0f,
+    0x47, 0x7a, 0x7f, 0xd0, 0xe4, 0x77, 0x09, 0x17,
+    0xe9, 0x81, 0x6e, 0x2d, 0x33, 0x9b, 0x13, 0x0b,
+    0xc9, 0xb2, 0x0c, 0x2c, 0xb5, 0xdf, 0x52, 0x8f,
+    0xab, 0x0d, 0xc6, 0x59, 0x1d, 0xc7, 0x33, 0x7b,
+};
+
+static const uint8_t test_plaintext[] = {
+    0x00, 0x44, 0xbc, 0x6f, 0x77, 0xfb, 0xe2, 0xa4,
+    0x98, 0x9e, 0xf5, 0x33, 0xa0, 0xbd, 0x81, 0xb9,
+    0xf1, 0x44, 0x7f, 0x79, 0x89, 0x23, 0xe5, 0x46,
+    0x66, 0x9f, 0x98, 0x95, 0x6f, 0x56, 0x78, 0xf6,
+    0xf5, 0xac, 0x9c, 0xda, 0xc2, 0x79, 0x59, 0xf0,
+    0x1b, 0x03, 0xfa, 0x46, 0x1c, 0x1f, 0x18, 0x07,
+    0xce, 0xad, 0xed, 0x3d, 0x11, 0xf9, 0x1b, 0x26,
+    0x4a, 0x97, 0x28, 0x71, 0x5f, 0x2c, 0x5e, 0x58,
+    0xf0, 0xd6, 0xbf, 0xa4, 0x12, 0xd0, 0x1d, 0x07,
+    0xcb, 0x73, 0x66, 0xb6, 0xa4, 0x09, 0xaf, 0x5d,
+    0xe9, 0x14, 0x14, 0xaf, 0x69, 0xd6, 0xee, 0x0a,
+    0xfc, 0xca, 0xac, 0x94, 0x47, 0xd5, 0x9d, 0x5b,
+    0x2b, 0xfb, 0xce, 0x9d, 0x04, 0xc1, 0xaf, 0xa5,
+    0xa1, 0x8d, 0xa9, 0x48, 0xa8, 0x65, 0xe6, 0x9f,
+    0x74, 0x78, 0x16, 0x32, 0x93, 0xb5, 0x21, 0xb9,
+    0x9f, 0x3f, 0xc1, 0xe5, 0xa2, 0x50, 0x8b, 0x12,
+    0xfb, 0x3e, 0xb0, 0x8a, 0x00, 0xc7, 0x20, 0x56,
+    0xb3, 0xb1, 0x29, 0x95, 0x89, 0xd6, 0x50, 0xf5,
+    0x37, 0x38, 0x8e, 0x12, 0xf1, 0xba, 0x82, 0x37,
+    0x34, 0x68, 0x4b, 0xe8, 0xe3, 0x11, 0x1c, 0x46,
+    0xf9, 0x63, 0x3a, 0xd6, 0xf3, 0x3f, 0x55, 0xa6,
+    0xbd, 0x89, 0xf1, 0x2d, 0x38, 0x91, 0x7c, 0xc2,
+    0x4d, 0xf1, 0x69, 0x82, 0x6d, 0x71, 0x77, 0xf4,
+    0xfc, 0x43, 0x20, 0x6f, 0x43, 0xb9, 0x43, 0xd1,
+    0x65, 0xbd, 0xca, 0xb1, 0x43, 0x87, 0xf8, 0xc8,
+    0x76, 0x21, 0xa9, 0xeb, 0x3e, 0x9a, 0xef, 0xc9,
+    0x0e, 0x79, 0xbc, 0xf0, 0xf8, 0xc8, 0xe2, 0xbc,
+    0x33, 0x35, 0x3e, 0xfc, 0xf9, 0x44, 0x69, 0x06,
+    0x7c, 0x7f, 0x5d, 0xa2, 0x9e, 0xab, 0xc2, 0x82,
+    0xa0, 0xfb, 0xc5, 0x79, 0x57, 0x8c, 0xf1, 0x1c,
+    0x51, 0x64, 0x4c, 0x56, 0x08, 0x80, 0x32, 0xf4,
+    0x97, 0x8f, 0x6f, 0xb2, 0x16, 0xa6, 0x9d, 0x71,
+};
+
+static const uint8_t exp_ciphertext_rsa1024_raw[] = {
+    0x01, 0xa0, 0xc2, 0x94, 0x9f, 0xd6, 0xbe, 0x8d,
+    0xe9, 0x24, 0xaa, 0x9c, 0x67, 0xd7, 0xe3, 0x04,
+    0x34, 0xbf, 0xd3, 0x27, 0xa1, 0x43, 0xeb, 0x60,
+    0x6b, 0x5b, 0x64, 0x15, 0x55, 0x16, 0x98, 0x35,
+    0xc2, 0x59, 0xa7, 0xf7, 0x24, 0xf7, 0x05, 0xb9,
+    0xe8, 0x56, 0x6f, 0xf2, 0x7d, 0x8b, 0x3c, 0xcb,
+    0xa6, 0xc2, 0xac, 0x0c, 0x37, 0x8c, 0x70, 0x70,
+    0x55, 0x05, 0x07, 0x0d, 0x63, 0x6b, 0x7d, 0x5f,
+    0xae, 0x03, 0x1e, 0x55, 0x05, 0xbb, 0xa8, 0xe7,
+    0xff, 0xa0, 0x8c, 0x5b, 0x6b, 0x01, 0x48, 0x2e,
+    0x4f, 0x7f, 0xe2, 0x74, 0xc6, 0x32, 0xa7, 0x2d,
+    0xdb, 0x91, 0x9b, 0x67, 0x4d, 0x71, 0xf9, 0x8c,
+    0x42, 0x43, 0x75, 0x4e, 0xd0, 0x0e, 0x7c, 0xa0,
+    0x97, 0x1a, 0x5f, 0x8e, 0x6f, 0xe4, 0xfa, 0x16,
+    0x1d, 0x59, 0x0e, 0x0b, 0x11, 0x12, 0xa3, 0x0c,
+    0xa6, 0x55, 0xe6, 0xdb, 0xa7, 0x71, 0xa6, 0xff,
+};
+
+static const uint8_t exp_ciphertext_rsa1024_pkcs1[] = {
+    0x93, 0x78, 0x6a, 0x76, 0xb8, 0x94, 0xea, 0xe4,
+    0x32, 0x79, 0x01, 0x8b, 0xc1, 0xcb, 0x2e, 0x2d,
+    0xfe, 0xdc, 0x9b, 0xe3, 0xe9, 0x23, 0xe4, 0x0a,
+    0xb0, 0x6b, 0x9f, 0x6b, 0x62, 0xf5, 0x3d, 0xf0,
+    0x78, 0x84, 0x77, 0x21, 0xad, 0x0b, 0x30, 0x30,
+    0x94, 0xe2, 0x18, 0xc4, 0x9b, 0x12, 0x06, 0xc8,
+    0xaa, 0xf7, 0x30, 0xe4, 0xc8, 0x64, 0xe7, 0x51,
+    0xf1, 0x6a, 0xe1, 0xa2, 0x58, 0x7a, 0x02, 0x9c,
+    0x8e, 0xf0, 0x2d, 0x25, 0x6b, 0xb7, 0x25, 0x5e,
+    0x05, 0xaf, 0x38, 0xb2, 0x69, 0x5e, 0x6c, 0x75,
+    0x6e, 0x27, 0xba, 0x5d, 0x7d, 0x35, 0x72, 0xb7,
+    0x25, 0xd4, 0xaa, 0xb2, 0x4b, 0x9e, 0x6b, 0x82,
+    0xb2, 0x32, 0xe2, 0x13, 0x1d, 0x00, 0x21, 0x08,
+    0xae, 0x14, 0xbb, 0xc0, 0x40, 0xb7, 0x0d, 0xd5,
+    0x0e, 0x4d, 0x6d, 0x9a, 0x70, 0x86, 0xe9, 0xfc,
+    0x67, 0x2b, 0xa4, 0x11, 0x45, 0xb6, 0xc4, 0x2f,
+};
+
+static const uint8_t exp_ciphertext_rsa2048_raw[] = {
+    0x09, 0x7b, 0x9e, 0x7c, 0x10, 0x1f, 0x73, 0xb4,
+    0x5f, 0xdb, 0x4f, 0x05, 0xe7, 0xfc, 0x9e, 0x35,
+    0x48, 0xd8, 0xc8, 0xf5, 0xac, 0x6d, 0xb4, 0xb0,
+    0xd4, 0xf7, 0x69, 0x0f, 0x30, 0x78, 0xbb, 0x55,
+    0x67, 0x66, 0x66, 0x05, 0xf4, 0x77, 0xe2, 0x30,
+    0xa5, 0x94, 0x10, 0xa3, 0xcb, 0xee, 0x13, 0x9f,
+    0x47, 0x1b, 0x2e, 0xf9, 0xfd, 0x94, 0x09, 0xbd,
+    0x26, 0x6e, 0x84, 0xc7, 0x5c, 0x42, 0x20, 0x76,
+    0x72, 0x83, 0x75, 0x68, 0xa4, 0x18, 0x2d, 0x76,
+    0x62, 0xc3, 0xab, 0xc0, 0xc9, 0x36, 0x59, 0xe0,
+    0xa9, 0x70, 0x1f, 0xff, 0x97, 0x07, 0x0d, 0x88,
+    0xc2, 0xd8, 0x51, 0x35, 0xf7, 0xb0, 0x50, 0xe4,
+    0x9f, 0x3d, 0xd4, 0x71, 0x8b, 0x40, 0x89, 0x71,
+    0x6c, 0xd8, 0xc2, 0x63, 0xb6, 0x3a, 0xce, 0xb1,
+    0x32, 0xf1, 0xc6, 0x11, 0x31, 0x25, 0x48, 0xcf,
+    0xeb, 0xbc, 0xd3, 0x9b, 0xc5, 0xbd, 0xd2, 0x57,
+    0x73, 0x9b, 0x20, 0xb8, 0xdf, 0xbe, 0xb8, 0x40,
+    0xb6, 0xac, 0x24, 0xdb, 0x94, 0x6a, 0x93, 0x43,
+    0x4a, 0xa8, 0xa3, 0xcf, 0xd5, 0x61, 0x1b, 0x46,
+    0x1d, 0x6f, 0x57, 0xec, 0xa6, 0xd0, 0x44, 0x05,
+    0x48, 0xb8, 0x90, 0x80, 0x23, 0x8e, 0x5f, 0xb0,
+    0x4b, 0x6f, 0xe3, 0xf9, 0xb0, 0x04, 0x60, 0xae,
+    0x80, 0xcf, 0xa5, 0x5c, 0x11, 0xe4, 0xce, 0x57,
+    0x5b, 0xbb, 0xde, 0x92, 0xfc, 0xe7, 0x3f, 0xe0,
+    0xfc, 0x06, 0xc8, 0xf3, 0x8c, 0xac, 0x86, 0x09,
+    0x31, 0xe5, 0x7e, 0xfb, 0x5d, 0xa7, 0x57, 0xf8,
+    0x1d, 0x23, 0x9d, 0xa3, 0xeb, 0x53, 0x28, 0xde,
+    0xbf, 0x53, 0xef, 0x35, 0x3c, 0x7e, 0x3c, 0x1b,
+    0x76, 0x9d, 0x09, 0x25, 0x43, 0xd4, 0x8b, 0xca,
+    0xda, 0x45, 0x5b, 0xdc, 0x9f, 0x57, 0x5a, 0x30,
+    0x2e, 0xe9, 0x73, 0x68, 0x28, 0xfa, 0x40, 0xb0,
+    0x7c, 0x31, 0xd7, 0x8b, 0x4e, 0x99, 0x94, 0xf1,
+};
+
+static const uint8_t exp_ciphertext_rsa2048_pkcs1[] = {
+    0xa5, 0x19, 0x19, 0x34, 0xad, 0xf6, 0xd2, 0xbe,
+    0xed, 0x8f, 0xe5, 0xfe, 0xa2, 0xa5, 0x20, 0x08,
+    0x15, 0x53, 0x7c, 0x68, 0x28, 0xae, 0x07, 0xb2,
+    0x4c, 0x5d, 0xee, 0xc1, 0xc6, 0xdc, 0xd6, 0x8b,
+    0xc6, 0xba, 0x46, 0xe1, 0x16, 0xa9, 0x04, 0x72,
+    0xdf, 0x8f, 0x1e, 0x97, 0x2a, 0x55, 0xe7, 0xac,
+    0x08, 0x0d, 0x61, 0xe8, 0x64, 0x8b, 0x6f, 0x96,
+    0x0e, 0xbb, 0x8a, 0x30, 0xb3, 0x73, 0x28, 0x61,
+    0x16, 0x89, 0x90, 0x88, 0x8e, 0xda, 0x22, 0xe6,
+    0x42, 0x16, 0xc7, 0xe8, 0x30, 0x0d, 0x7f, 0x44,
+    0x1e, 0xef, 0xe6, 0xdb, 0x78, 0x54, 0x89, 0xa5,
+    0x60, 0x67, 0xb3, 0x35, 0x2d, 0x79, 0x49, 0xcf,
+    0xe6, 0x8f, 0xf3, 0x64, 0x52, 0x1c, 0x6c, 0x43,
+    0x7e, 0xb0, 0xde, 0x55, 0xdf, 0xbe, 0xb7, 0xb1,
+    0xdb, 0x02, 0xee, 0x76, 0x96, 0xcc, 0x0b, 0x97,
+    0x8c, 0x23, 0xaa, 0x7d, 0x4c, 0x47, 0x28, 0x41,
+    0x7a, 0x20, 0x39, 0x1f, 0x64, 0x0b, 0xf1, 0x74,
+    0xf1, 0x29, 0xda, 0xe9, 0x3a, 0x36, 0xa6, 0x88,
+    0xb8, 0xc0, 0x21, 0xb8, 0x9b, 0x5d, 0x90, 0x85,
+    0xa3, 0x30, 0x61, 0x17, 0x8c, 0x74, 0x63, 0xd5,
+    0x0f, 0x95, 0xdc, 0xc8, 0x4f, 0xa7, 0x24, 0x55,
+    0x40, 0xe2, 0x84, 0x57, 0x65, 0x06, 0x11, 0x30,
+    0x2b, 0x9e, 0x32, 0x95, 0x39, 0xf2, 0x1a, 0x3f,
+    0xab, 0xcd, 0x7b, 0x7f, 0x9c, 0xf0, 0x00, 0x50,
+    0x7c, 0xf4, 0xbe, 0xcb, 0x80, 0xea, 0x66, 0xba,
+    0x0e, 0x7b, 0x46, 0x0b, 0x25, 0xe0, 0xc1, 0x03,
+    0x29, 0x11, 0x2d, 0x69, 0x4f, 0x21, 0xa2, 0x58,
+    0x37, 0x4b, 0x84, 0x15, 0xb3, 0x65, 0x3a, 0xac,
+    0xd4, 0xd0, 0xf6, 0xdf, 0x4b, 0x82, 0xca, 0x9e,
+    0xbb, 0xbe, 0x3c, 0x4d, 0xd5, 0xbf, 0x00, 0xd6,
+    0x12, 0x48, 0x72, 0x0b, 0xc7, 0xf8, 0xe1, 0xcd,
+    0xd0, 0x28, 0x03, 0x19, 0xa6, 0x06, 0x13, 0x45,
+};
+
+typedef struct QCryptoAkCipherTestData QCryptoAkCipherTestData;
+struct QCryptoAkCipherTestData {
+    const char *path;
+    QCryptoAkCipherOptions opt;
+
+    const uint8_t *priv_key;
+    size_t priv_key_len;
+    const uint8_t *pub_key;
+    size_t pub_key_len;
+
+    const uint8_t *plaintext;
+    size_t plen;
+    const uint8_t *ciphertext;
+    size_t clen;
+    const uint8_t *dgst;
+    size_t dlen;
+    const uint8_t *signature;
+    size_t slen;
+};
+
+static QCryptoAkCipherTestData test_data[] = {
+    /* rsa1024 with raw padding */
+    {
+        .path = "/crypto/akcipher/rsa1024-raw",
+        .opt = {
+            .alg = QCRYPTO_AKCIPHER_ALG_RSA,
+            .u.rsa = {
+                .padding_alg = QCRYPTO_RSA_PADDING_ALG_RAW,
+            },
+        },
+        .pub_key = rsa1024_public_key,
+        .pub_key_len = sizeof(rsa1024_public_key),
+        .priv_key = rsa1024_private_key,
+        .priv_key_len = sizeof(rsa1024_private_key),
+
+        .plaintext = test_plaintext,
+        .plen = 128,
+        .ciphertext = exp_ciphertext_rsa1024_raw,
+        .clen = sizeof(exp_ciphertext_rsa1024_raw),
+    },
+
+    /* rsa1024 with pkcs1 padding */
+    {
+        .path = "/crypto/akcipher/rsa1024-pkcs1",
+        .opt = {
+            .alg = QCRYPTO_AKCIPHER_ALG_RSA,
+            .u.rsa = {
+                .padding_alg = QCRYPTO_RSA_PADDING_ALG_PKCS1,
+                .hash_alg = QCRYPTO_HASH_ALG_SHA1,
+            },
+        },
+        .pub_key = rsa1024_public_key,
+        .pub_key_len = sizeof(rsa1024_public_key),
+        .priv_key = rsa1024_private_key,
+        .priv_key_len = sizeof(rsa1024_private_key),
+
+        .plaintext = test_plaintext,
+        .plen = 64,
+        .ciphertext = exp_ciphertext_rsa1024_pkcs1,
+        .clen = sizeof(exp_ciphertext_rsa1024_pkcs1),
+        .dgst = test_sha1_dgst,
+        .dlen = sizeof(test_sha1_dgst),
+        .signature = exp_signature_rsa1024_pkcs1,
+        .slen = sizeof(exp_signature_rsa1024_pkcs1),
+    },
+
+    /* rsa2048 with raw padding */
+    {
+        .path = "/crypto/akcipher/rsa2048-raw",
+        .opt = {
+            .alg = QCRYPTO_AKCIPHER_ALG_RSA,
+            .u.rsa = {
+                .padding_alg = QCRYPTO_RSA_PADDING_ALG_RAW,
+            },
+        },
+        .pub_key = rsa2048_public_key,
+        .pub_key_len = sizeof(rsa2048_public_key),
+        .priv_key = rsa2048_private_key,
+        .priv_key_len = sizeof(rsa2048_private_key),
+
+        .plaintext = test_plaintext,
+        .plen = 256,
+        .ciphertext = exp_ciphertext_rsa2048_raw,
+        .clen = sizeof(exp_ciphertext_rsa2048_raw),
+    },
+
+    /* rsa2048 with pkcs1 padding */
+    {
+        .path = "/crypto/akcipher/rsa2048-pkcs1",
+        .opt = {
+            .alg = QCRYPTO_AKCIPHER_ALG_RSA,
+            .u.rsa = {
+                .padding_alg = QCRYPTO_RSA_PADDING_ALG_PKCS1,
+                .hash_alg = QCRYPTO_HASH_ALG_SHA1,
+            },
+        },
+        .pub_key = rsa2048_public_key,
+        .pub_key_len = sizeof(rsa2048_public_key),
+        .priv_key = rsa2048_private_key,
+        .priv_key_len = sizeof(rsa2048_private_key),
+
+        .plaintext = test_plaintext,
+        .plen = 128,
+        .ciphertext = exp_ciphertext_rsa2048_pkcs1,
+        .clen = sizeof(exp_ciphertext_rsa2048_pkcs1),
+        .dgst = test_sha1_dgst,
+        .dlen = sizeof(test_sha1_dgst),
+        .signature = exp_signature_rsa2048_pkcs1,
+        .slen = sizeof(exp_signature_rsa2048_pkcs1),
+    },
+
+};
+
+static void test_akcipher(const void *opaque)
+{
+    const QCryptoAkCipherTestData *data = opaque;
+    g_autofree uint8_t *plaintext = NULL;
+    g_autofree uint8_t *ciphertext = NULL;
+    g_autofree uint8_t *signature = NULL;
+    QCryptoAkCipher *pub_key, *priv_key;
+
+    if (!qcrypto_akcipher_supports((QCryptoAkCipherOptions *)&data->opt)) {
+        return;
+    }
+    pub_key = qcrypto_akcipher_new(&data->opt,
+                                   QCRYPTO_AKCIPHER_KEY_TYPE_PUBLIC,
+                                   data->pub_key, data->pub_key_len,
+                                   &error_abort);
+    g_assert(pub_key != NULL);
+    priv_key = qcrypto_akcipher_new(&data->opt,
+                                    QCRYPTO_AKCIPHER_KEY_TYPE_PRIVATE,
+                                    data->priv_key, data->priv_key_len,
+                                    &error_abort);
+    g_assert(priv_key != NULL);
+
+    if (data->plaintext != NULL) {
+
+        ciphertext = g_new0(uint8_t, data->clen);
+        g_assert(qcrypto_akcipher_encrypt(pub_key, data->plaintext, data->plen,
+                                          ciphertext, data->clen,
+                                          &error_abort) > 0);
+
+        /**
+         * In the asymmetric encryption algorithms, the ciphertext generated
+         * each time may be different, here only compare the decrypted
+         * plaintext
+         */
+        plaintext = g_new0(uint8_t, data->clen);
+        g_assert(qcrypto_akcipher_decrypt(priv_key, ciphertext,
+                                          data->clen, plaintext,
+                                          data->plen,
+                                          &error_abort) == data->plen);
+        g_assert(!memcmp(plaintext, data->plaintext, data->plen));
+    }
+
+    if (data->signature != NULL) {
+        signature = g_new(uint8_t, data->slen);
+        g_assert(qcrypto_akcipher_sign(priv_key, data->dgst, data->dlen,
+                                       signature, data->slen,
+                                       &error_abort) > 0);
+        /**
+         * The signature generated each time may be different, here only check
+         * the verification.
+         */
+        g_assert(qcrypto_akcipher_verify(pub_key, data->signature, data->slen,
+                                         data->dgst, data->dlen,
+                                         &error_abort) == 0);
+        g_assert(qcrypto_akcipher_verify(pub_key, signature, data->slen,
+                                         data->dgst, data->dlen,
+                                         &error_abort) == 0);
+        ++signature[0];
+        /* Here error should be ignored */
+        g_assert(qcrypto_akcipher_verify(pub_key, signature, data->slen,
+                                         data->dgst, data->dlen, NULL) != 0);
+    }
+
+    qcrypto_akcipher_free(pub_key);
+    qcrypto_akcipher_free(priv_key);
+}
+
+int main(int argc, char **argv)
+{
+    size_t i;
+    g_test_init(&argc, &argv, NULL);
+    g_assert(qcrypto_init(NULL) == 0);
+
+    for (i = 0; i < G_N_ELEMENTS(test_data); i++) {
+        g_test_add_data_func(test_data[i].path, &test_data[i], test_akcipher);
+    }
+
+    return g_test_run();
+}

From f0cfb761bc6e590d648b759e6bdb8c946062b5f5 Mon Sep 17 00:00:00 2001
From: Lei He <helei.sig11@bytedance.com>
Date: Wed, 25 May 2022 17:01:17 +0800
Subject: [PATCH 528/935] tests/crypto: Add test suite for RSA keys
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

As Daniel suggested, Add tests suite for rsakey, as a way to prove
that we can handle DER errors correctly.

Signed-off-by: lei he <helei.sig11@bytedance.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 tests/unit/test-crypto-akcipher.c | 285 +++++++++++++++++++++++++++++-
 1 file changed, 282 insertions(+), 3 deletions(-)

diff --git a/tests/unit/test-crypto-akcipher.c b/tests/unit/test-crypto-akcipher.c
index b5be563884..4f1f4214dd 100644
--- a/tests/unit/test-crypto-akcipher.c
+++ b/tests/unit/test-crypto-akcipher.c
@@ -517,6 +517,158 @@ static const uint8_t exp_ciphertext_rsa2048_pkcs1[] = {
     0xd0, 0x28, 0x03, 0x19, 0xa6, 0x06, 0x13, 0x45,
 };
 
+static const uint8_t rsa_private_key_lack_element[] = {
+    /* RSAPrivateKey, offset: 0, length: 176 */
+    0x30, 0x81, 0xb0,
+    /* version, offset: 4, length: 1 */
+    0x02, 0x01, 0x00,
+    /* n, offset: 7, length: 65 */
+    0x02, 0x41,
+    0x00, 0xb9, 0xe1, 0x22, 0xdb, 0x56, 0x2f, 0xb6,
+    0xf7, 0xf0, 0x0a, 0x87, 0x43, 0x07, 0x12, 0xdb,
+    0x6d, 0xb6, 0x2b, 0x41, 0x8d, 0x2c, 0x3c, 0xa5,
+    0xdd, 0x78, 0x9a, 0x8f, 0xab, 0x8e, 0xf2, 0x4a,
+    0xc8, 0x34, 0x0c, 0x12, 0x4f, 0x11, 0x90, 0xc6,
+    0xc2, 0xa5, 0xd0, 0xcd, 0xfb, 0xfc, 0x2c, 0x95,
+    0x56, 0x82, 0xdf, 0x39, 0xf3, 0x3b, 0x1d, 0x62,
+    0x26, 0x97, 0xb7, 0x93, 0x25, 0xc7, 0xec, 0x7e,
+    0xf7,
+    /* e, offset: 74, length: 3 */
+    0x02, 0x03, 0x01, 0x00, 0x01,
+    /* d, offset: 79, length: 64 */
+    0x02, 0x40,
+    0x1e, 0x80, 0xfe, 0xda, 0x65, 0xdb, 0x70, 0xb8,
+    0x61, 0x91, 0x28, 0xbf, 0x6c, 0x32, 0xc1, 0x05,
+    0xd1, 0x26, 0x6a, 0x1c, 0x83, 0xcc, 0xf4, 0x1f,
+    0x53, 0x42, 0x72, 0x1f, 0x62, 0x57, 0x0a, 0xc4,
+    0x66, 0x76, 0x30, 0x87, 0xb9, 0xb1, 0xb9, 0x6a,
+    0x63, 0xfd, 0x8f, 0x3e, 0xfc, 0x35, 0x3f, 0xd6,
+    0x2e, 0x6c, 0xc8, 0x70, 0x8a, 0x17, 0xc1, 0x28,
+    0x6a, 0xfe, 0x51, 0x56, 0xb3, 0x92, 0x6f, 0x09,
+    /* p, offset: 145, length: 33 */
+    0x02, 0x21,
+    0x00, 0xe3, 0x2e, 0x2d, 0x8d, 0xba, 0x1c, 0x34,
+    0x4c, 0x49, 0x9f, 0xc1, 0xa6, 0xdd, 0xd7, 0x13,
+    0x8d, 0x05, 0x48, 0xdd, 0xff, 0x5c, 0x30, 0xbc,
+    0x6b, 0xc4, 0x18, 0x9d, 0xfc, 0xa2, 0xd0, 0x9b,
+    0x4d,
+    /* q, offset: 180, length: 33 */
+    0x02, 0x21,
+    0x00, 0xd1, 0x75, 0xaf, 0x4b, 0xc6, 0x1a, 0xb0,
+    0x98, 0x14, 0x42, 0xae, 0x33, 0xf3, 0x44, 0xde,
+    0x21, 0xcb, 0x04, 0xda, 0xfb, 0x1e, 0x35, 0x92,
+    0xcd, 0x69, 0xc0, 0x83, 0x06, 0x83, 0x8e, 0x39,
+    0x53,
+    /* lack element: dp, dq, u */
+};
+
+static const uint8_t rsa_public_key_lack_element[] = {
+    /* RSAPublicKey, offset: 0, length: 67 */
+    0x30, 0x81, 0x43,
+    /* n, offset: 7, length: 65 */
+    0x02, 0x41,
+    0x00, 0xb9, 0xe1, 0x22, 0xdb, 0x56, 0x2f, 0xb6,
+    0xf7, 0xf0, 0x0a, 0x87, 0x43, 0x07, 0x12, 0xdb,
+    0x6d, 0xb6, 0x2b, 0x41, 0x8d, 0x2c, 0x3c, 0xa5,
+    0xdd, 0x78, 0x9a, 0x8f, 0xab, 0x8e, 0xf2, 0x4a,
+    0xc8, 0x34, 0x0c, 0x12, 0x4f, 0x11, 0x90, 0xc6,
+    0xc2, 0xa5, 0xd0, 0xcd, 0xfb, 0xfc, 0x2c, 0x95,
+    0x56, 0x82, 0xdf, 0x39, 0xf3, 0x3b, 0x1d, 0x62,
+    0x26, 0x97, 0xb7, 0x93, 0x25, 0xc7, 0xec, 0x7e,
+    0xf7,
+    /* lack element: e */
+};
+
+static const uint8_t rsa_public_key_empty_element[] = {
+    /* RSAPublicKey, offset: 0, length: 69 */
+    0x30, 0x81, 0x45,
+    /* n, offset: 7, length: 65 */
+    0x02, 0x41,
+    0x00, 0xb9, 0xe1, 0x22, 0xdb, 0x56, 0x2f, 0xb6,
+    0xf7, 0xf0, 0x0a, 0x87, 0x43, 0x07, 0x12, 0xdb,
+    0x6d, 0xb6, 0x2b, 0x41, 0x8d, 0x2c, 0x3c, 0xa5,
+    0xdd, 0x78, 0x9a, 0x8f, 0xab, 0x8e, 0xf2, 0x4a,
+    0xc8, 0x34, 0x0c, 0x12, 0x4f, 0x11, 0x90, 0xc6,
+    0xc2, 0xa5, 0xd0, 0xcd, 0xfb, 0xfc, 0x2c, 0x95,
+    0x56, 0x82, 0xdf, 0x39, 0xf3, 0x3b, 0x1d, 0x62,
+    0x26, 0x97, 0xb7, 0x93, 0x25, 0xc7, 0xec, 0x7e,
+    0xf7,
+    /* e: empty element */
+    0x02, 0x00,
+};
+
+static const uint8_t rsa_private_key_empty_element[] = {
+    /* RSAPrivateKey, offset: 0, length: 19 */
+    0x30, 0x81, 0x13,
+    /* version, offset: 4, length: 1 */
+    0x02, 0x01, 0x00,
+    /* n: empty element */
+    0x02, 0x00,
+    /* e: empty element */
+    0x02, 0x00,
+    /* d: empty element */
+    0x02, 0x00,
+    /* p: empty element */
+    0x02, 0x00,
+    /* q: empty element */
+    0x02, 0x00,
+    /* dp: empty element */
+    0x02, 0x00,
+    /* dq: empty element */
+    0x02, 0x00,
+    /* u: empty element */
+    0x02, 0x00,
+};
+
+static const uint8_t rsa_public_key_invalid_length_val[] = {
+    /* RSAPublicKey, INVALID length: 313 */
+    0x30, 0x82, 0x01, 0x39,
+    /* n, offset: 7, length: 65 */
+    0x02, 0x41,
+    0x00, 0xb9, 0xe1, 0x22, 0xdb, 0x56, 0x2f, 0xb6,
+    0xf7, 0xf0, 0x0a, 0x87, 0x43, 0x07, 0x12, 0xdb,
+    0x6d, 0xb6, 0x2b, 0x41, 0x8d, 0x2c, 0x3c, 0xa5,
+    0xdd, 0x78, 0x9a, 0x8f, 0xab, 0x8e, 0xf2, 0x4a,
+    0xc8, 0x34, 0x0c, 0x12, 0x4f, 0x11, 0x90, 0xc6,
+    0xc2, 0xa5, 0xd0, 0xcd, 0xfb, 0xfc, 0x2c, 0x95,
+    0x56, 0x82, 0xdf, 0x39, 0xf3, 0x3b, 0x1d, 0x62,
+    0x26, 0x97, 0xb7, 0x93, 0x25, 0xc7, 0xec, 0x7e,
+    0xf7,
+    /* e, */
+    0x02, 0x03, 0x01, 0x00, 0x01,  /* INTEGER, offset: 74, length: 3 */
+};
+
+static const uint8_t rsa_public_key_extra_elem[] = {
+    /* RSAPublicKey, length: 80 */
+    0x30, 0x81, 0x50,
+    /* n, offset: 7, length: 65 */
+    0x02, 0x41,
+    0x00, 0xb9, 0xe1, 0x22, 0xdb, 0x56, 0x2f, 0xb6,
+    0xf7, 0xf0, 0x0a, 0x87, 0x43, 0x07, 0x12, 0xdb,
+    0x6d, 0xb6, 0x2b, 0x41, 0x8d, 0x2c, 0x3c, 0xa5,
+    0xdd, 0x78, 0x9a, 0x8f, 0xab, 0x8e, 0xf2, 0x4a,
+    0xc8, 0x34, 0x0c, 0x12, 0x4f, 0x11, 0x90, 0xc6,
+    0xc2, 0xa5, 0xd0, 0xcd, 0xfb, 0xfc, 0x2c, 0x95,
+    0x56, 0x82, 0xdf, 0x39, 0xf3, 0x3b, 0x1d, 0x62,
+    0x26, 0x97, 0xb7, 0x93, 0x25, 0xc7, 0xec, 0x7e,
+    0xf7,
+    /* e, offset: 74, length: 3 */
+    0x02, 0x03, 0x01, 0x00, 0x01,
+    /* Additional integer field, length 3 */
+    0x02, 0x06, 0xe1, 0x22, 0xdb, 0xe1, 0x22, 0xdb,
+};
+
+typedef struct QCryptoRSAKeyTestData QCryptoRSAKeyTestData;
+struct QCryptoRSAKeyTestData {
+    const char *path;
+    QCryptoAkCipherKeyType key_type;
+    QCryptoAkCipherOptions opt;
+    const uint8_t *key;
+    size_t keylen;
+    bool is_valid_key;
+    size_t exp_key_len;
+};
+
 typedef struct QCryptoAkCipherTestData QCryptoAkCipherTestData;
 struct QCryptoAkCipherTestData {
     const char *path;
@@ -537,7 +689,98 @@ struct QCryptoAkCipherTestData {
     size_t slen;
 };
 
-static QCryptoAkCipherTestData test_data[] = {
+static QCryptoRSAKeyTestData rsakey_test_data[] = {
+    {
+        .path = "/crypto/akcipher/rsakey-1024-public",
+        .key_type = QCRYPTO_AKCIPHER_KEY_TYPE_PUBLIC,
+        .key = rsa1024_public_key,
+        .keylen = sizeof(rsa1024_public_key),
+        .is_valid_key = true,
+        .exp_key_len = 128,
+    },
+    {
+        .path = "/crypto/akcipher/rsakey-1024-private",
+        .key_type = QCRYPTO_AKCIPHER_KEY_TYPE_PRIVATE,
+        .key = rsa1024_private_key,
+        .keylen = sizeof(rsa1024_private_key),
+        .is_valid_key = true,
+        .exp_key_len = 128,
+    },
+    {
+        .path = "/crypto/akcipher/rsakey-2048-public",
+        .key_type = QCRYPTO_AKCIPHER_KEY_TYPE_PUBLIC,
+        .key = rsa2048_public_key,
+        .keylen = sizeof(rsa2048_public_key),
+        .is_valid_key = true,
+        .exp_key_len = 256,
+    },
+    {
+        .path = "/crypto/akcipher/rsakey-2048-private",
+        .key_type = QCRYPTO_AKCIPHER_KEY_TYPE_PRIVATE,
+        .key = rsa2048_private_key,
+        .keylen = sizeof(rsa2048_private_key),
+        .is_valid_key = true,
+        .exp_key_len = 256,
+    },
+    {
+        .path = "/crypto/akcipher/rsakey-public-lack-elem",
+        .key_type = QCRYPTO_AKCIPHER_KEY_TYPE_PUBLIC,
+        .key = rsa_public_key_lack_element,
+        .keylen = sizeof(rsa_public_key_lack_element),
+        .is_valid_key = false,
+    },
+    {
+        .path = "/crypto/akcipher/rsakey-private-lack-elem",
+        .key_type = QCRYPTO_AKCIPHER_KEY_TYPE_PRIVATE,
+        .key = rsa_private_key_lack_element,
+        .keylen = sizeof(rsa_private_key_lack_element),
+        .is_valid_key = false,
+    },
+    {
+        .path = "/crypto/akcipher/rsakey-public-empty-elem",
+        .key_type = QCRYPTO_AKCIPHER_KEY_TYPE_PUBLIC,
+        .key = rsa_public_key_empty_element,
+        .keylen = sizeof(rsa_public_key_empty_element),
+        .is_valid_key = false,
+    },
+    {
+        .path = "/crypto/akcipher/rsakey-private-empty-elem",
+        .key_type = QCRYPTO_AKCIPHER_KEY_TYPE_PRIVATE,
+        .key = rsa_private_key_empty_element,
+        .keylen = sizeof(rsa_private_key_empty_element),
+        .is_valid_key = false,
+    },
+    {
+        .path = "/crypto/akcipher/rsakey-public-empty-key",
+        .key_type = QCRYPTO_AKCIPHER_KEY_TYPE_PUBLIC,
+        .key = NULL,
+        .keylen = 0,
+        .is_valid_key = false,
+    },
+    {
+        .path = "/crypto/akcipher/rsakey-private-empty-key",
+        .key_type = QCRYPTO_AKCIPHER_KEY_TYPE_PRIVATE,
+        .key = NULL,
+        .keylen = 0,
+        .is_valid_key = false,
+    },
+    {
+        .path = "/crypto/akcipher/rsakey-public-invalid-length-val",
+        .key_type = QCRYPTO_AKCIPHER_KEY_TYPE_PUBLIC,
+        .key = rsa_public_key_invalid_length_val,
+        .keylen = sizeof(rsa_public_key_invalid_length_val),
+        .is_valid_key = false,
+    },
+    {
+        .path = "/crypto/akcipher/rsakey-public-extra-elem",
+        .key_type = QCRYPTO_AKCIPHER_KEY_TYPE_PUBLIC,
+        .key = rsa_public_key_extra_elem,
+        .keylen = sizeof(rsa_public_key_extra_elem),
+        .is_valid_key = false,
+    },
+};
+
+static QCryptoAkCipherTestData akcipher_test_data[] = {
     /* rsa1024 with raw padding */
     {
         .path = "/crypto/akcipher/rsa1024-raw",
@@ -697,14 +940,50 @@ static void test_akcipher(const void *opaque)
     qcrypto_akcipher_free(priv_key);
 }
 
+static void test_rsakey(const void *opaque)
+{
+    const QCryptoRSAKeyTestData *data = (const QCryptoRSAKeyTestData *)opaque;
+    QCryptoAkCipherOptions opt = {
+        .alg = QCRYPTO_AKCIPHER_ALG_RSA,
+        .u.rsa = {
+            .padding_alg = QCRYPTO_RSA_PADDING_ALG_PKCS1,
+            .hash_alg = QCRYPTO_HASH_ALG_SHA1,
+        }
+    };
+    g_autoptr(QCryptoAkCipher) key = qcrypto_akcipher_new(
+        &opt, data->key_type, data->key, data->keylen, NULL);
+
+    if (!qcrypto_akcipher_supports(&opt)) {
+        return;
+    }
+
+    if (!data->is_valid_key) {
+        g_assert(key == NULL);
+        return;
+    }
+
+    g_assert(key != NULL);
+    g_assert(qcrypto_akcipher_max_ciphertext_len(key) == data->exp_key_len);
+    g_assert(qcrypto_akcipher_max_plaintext_len(key) == data->exp_key_len);
+    g_assert(qcrypto_akcipher_max_signature_len(key) == data->exp_key_len);
+    g_assert(qcrypto_akcipher_max_dgst_len(key) == data->exp_key_len);
+}
+
 int main(int argc, char **argv)
 {
     size_t i;
     g_test_init(&argc, &argv, NULL);
     g_assert(qcrypto_init(NULL) == 0);
 
-    for (i = 0; i < G_N_ELEMENTS(test_data); i++) {
-        g_test_add_data_func(test_data[i].path, &test_data[i], test_akcipher);
+    for (i = 0; i < G_N_ELEMENTS(akcipher_test_data); i++) {
+        g_test_add_data_func(akcipher_test_data[i].path,
+                             &akcipher_test_data[i],
+                             test_akcipher);
+    }
+    for (i = 0; i < G_N_ELEMENTS(rsakey_test_data); i++) {
+        g_test_add_data_func(rsakey_test_data[i].path,
+                             &rsakey_test_data[i],
+                             test_rsakey);
     }
 
     return g_test_run();

From 64ee6989c9476857ecaec5496cdc188b324af5de Mon Sep 17 00:00:00 2001
From: Helge Deller <deller@gmx.de>
Date: Thu, 26 May 2022 12:54:29 +0200
Subject: [PATCH 529/935] New SeaBIOS-hppa version 6

Staring with SEABIOS_HPPA_VERSION 6 the serial ports are now emulated as
on physical hardware, with LASI UART being serial port #0 and DINO UART
as serial port #1. On older versions those ports were swapped.

This SeaBIOS-hppa fix is needed to allow fixing the qemu serial
pass-through from host to guest.

Signed-off-by: Helge Deller <deller@gmx.de>
---
 pc-bios/hppa-firmware.img | Bin 719040 -> 719368 bytes
 roms/seabios-hppa         |   2 +-
 2 files changed, 1 insertion(+), 1 deletion(-)

diff --git a/pc-bios/hppa-firmware.img b/pc-bios/hppa-firmware.img
index 392bce072bad607b1baaff0136f35844fa5d439e..b2cbb71ee0f91623d038db04d6900a5ed17472a6 100644
GIT binary patch
delta 316773
zcmce;3sh7`);3&q`od{w=;q$so15YdQ5pe7Xc4f*8;Hn7koKBZOhO_i#$ad_6%&l;
zK@TcMOuS%X5)zVNX3PX9nvl^X7<JS{6P(vfPNI3!W}G)>qA&kbryJY%`)2+Bzt*?b
z$FexJt9I?$wQJX|UAqc<ZjWfZGom?$yVJitE>_h?6w6m_pT0LCWmM9rN5-TsT9CRR
zJ$=;TWlz>VGiq$o*s)0|V^gZej5Cc(N*z@*CMikW-bq&q>*;|}c2g#?$JS=j)MpjM
zxGIsF7HNrbZzMG}D1WqX*#E26>epd3Wl~eLf(QlNkVK6^vi1<3aM{s;J~V|_cX*S)
zPd6H~enOd<^z%p0C4|-)$N8X%hq3D*=g)<Z*h`T#^&8GE<$GyUzae&`1wCYIj9q=o
zSfz?c`Rab*E<ZHA*EpQD)9H=Ffe@|gbbap{hcXt_jZRo?c78ny2JwYHK9-WMI?b0H
z7u+BeXhS)#jII@0wIY)KE4z*)W!KVi=i9>8ez`O3{)U~#gb`LFv83oEp;;G7Y1eDI
z{mLN2;%R@*y23|r<*_k@q^A2s-@K)~Cs+f9dfytXxp~X;LRU+G2j_i?jh7+cyrn)S
zbOike?dfU?j^*&_4B?c8pr1i(sRH3_$;7tJZj2eR)JRPJWC+S@UJ&eNRwtQzxNX^4
zpRqcYP-m6PqoQ`AofWA@ZWm1iMdTDUpJt2W(}aS^anT?ao;6@eCYEZjy~!F~sx$h@
zM8?$--eMxck;suWsp~>yDy3;%TH}9HjMOVer%<bFED7q>f-!!FkJ|oaTj*VFouAe;
zPHh)@;y>n8%|hdla!$3Q>&B2jae+qi$knXC^BEzI^OM8wA|GMSAYI+V43yp#`i5t7
z8mka6qQAcSlPpd3$63C72$xrVDl3`~<#J4+B%%7FtYIbvA=Rg|LM%&ovs4xEsi{Lx
z_;7BD>5$&jzO>!OE4T`2e)wr##g${41~%J`MxN)&QMS*th;Z6AKVoVi97p&Q{5@s(
zHha}p)@ItDGp|fTztcX>63!e6j!z;?7N+su?T>#%>?VVf@RXY@RVj{|LWnFc(*kcH
zcvOq`bzTkfQBfsQmG`H@@llbS_ZH#us60xAsKjYp*bQMtV*YTx-K*TQBGr&JaH)-7
zWJ}}QnId?VDh;W-NfJ{cFN0^PX*}O4^e2v>c0oIOJr~j@G>=Xk5}C`ld4`(8$zzRm
z_9Tp~OsewPlKRy|Uz35XX8cI=Bkl_~M#uR1M6+Us!h3+L97B^}NE+*Bh#M?sUoiTN
zP?5BO-V^$h632%f@~V)@e<PDuGV;2&QokAx@{~BqhgCAT`jZK?x_5$xcj7D)EMpQQ
z1JAN5R>Z#G><7~!N~EaBc>Wb-xS$-HCww(V&js8NZjBkj`Iv=(v3Xp8Qm7a^p7U=K
zTE`}G(8J!bG3;{-AHOz1pPb0~WeDcvZ#aFOU^ETqRPn-e(|erHJAyK0>^T2o7t4<|
z-m{xAf8|b=0zOUsDr37_D*h@4vwCU1XP~emC6o&&6B<(z6TPR)H8R&3+odZ1E2cxl
zKFBqj`w?ru7)bpF0*LnM5pJZ+jMvTq;e=_YveKu0lr@Pd4AvY;{U(db=vZ@5XQ5<V
z2{j6Dj(fzHSQ>h=;-NNEUF%U8nmUvcVODA*O%g7r9*)$W=6z+|SqqgkFeNZ2t9JZ#
zrI*2!q1qv|rOo72J;JTDd`{INq@~a0=x0K6dO1fcg<I)k)k(yXP6~v$@x}^SS;Gl=
z{1Y(jK|-;7G*~9SCKc(y5(>f6i9SC$>h82X9`@8cJxR^cdp28*0TLmeQdQJiSzG3l
zpCHC^{sb5P!p5u*px0%6Yr^gd?GpmMq9Q(+^+^_Grb<XEiV_x$52xvZeS9Pr{!gKO
zd}`np({9oW9aWn4kUp;soI;vUyoFoii|G4;c|tHP>nfcvA9h3N&Pa>CliEf&+2q{}
zxy)$B8tt6dKF`q9bZWi%gEBzSPn_*T{{^<7Raw5$eEO46GjRgFDjb_QpXvm!%rI}A
zo!LAlTPFA9%-9%y?E1Q_i!N!Kg*Il`U>T#L-bA`)&fv!it1}~q>F#c_8k0sWz&bV%
zCDvgXCUpxYb-NU$PM>u;Yn|5~Po*WE3Lj?1(MiIM%owzR;kB|huu79LsX3NCwAZvK
zYe1SGbTYM^@J-gNZ&4<U&Wh!fzX@})<}!PBEbGrfVLz}%l**>VK#a^XN%O-_3JvC$
zFp}6wRUx|L%*ed3yWI@)h7rM}g>q$<@j7Aeq|q>+!r4igfvS#a1F8;IY0J{}`}FGf
zn@l<(D!W>(>lwEadeWRdJ3`kvO(2~zQnKzi@_j5nt&Q~Bl6AsBc5QUwrK+d_smk*w
zwkj}YHgA2fpyf;sQlz)9&@y=}r*0Q|C(jO5cQ6d5=Ce*^tz!zx6fGwW)ux=P5MP<r
zeD8IpHKdQVXLO_}eyeaS=TF|sfDsGyB-W@KFr_KGp3e2+I6k4PVajG2_?B$agNY^e
zGK3Ps>81IRTLt6PZFH;9Id!P72KBkDyHuEJODx|iNK=Q>PX&GcCNBJn(3(Fwel2zc
zCFH6!h0;GsRpINrD!5SDSXG5DL)jduGAc#5l^+R}^O`ozGi)7eAXP@r5UkT;<5XZ-
zi4Edk+ZwXEp(Vs=RzrUS7GZpwqN#9XTC&QiUEfYSMqd}2re|~71w!BS<$n4xXi#c0
zaQ#jL$&C@F&v=jH3kBuOzjHo63Abiu$H;9<+1zekZ%i@;6SBF%SkA`9gk?2C9}8v$
z(+I&fE0Y7zF)K4N{5~^G)gNW-uKpONEEwyA=StZ^kgB}43(A6-L7ul|+oj@4uv|+q
zkEl{?5o`sq^iWq*fr7$5G#BQD=-S7v%o^}Y_Vo8MDnq=IIe*VO&v09YPWZX-Dm^5e
zoxPVn+GQ)6MKu>OY^_wOT_W_%DWUHQhT>S7D`XVMKzdchBcP`*6dUQ(u4Bc1lxl<z
z=T5<IkCIf(6n4;!scl#XQBkXG85&_$N!Sn#u`SW?<5m)GS+a&`&+04Aw?#Rac6fX0
z1NZ>0?B<Ulmxbn%NRWB6WEy_|UXqI6$)%%b7?PMpNd173j2N|SA_(;A%e{Gh$ocXp
z$ITyu9jx9`r{NjSh7)Fk<rxhR6&Qs<1hD)FE0$XZ{G~cjFO7=UnNmtTtUhb~3Z|xO
zi0*lP=uJtd^*rSnG{TxhOlcb7=h6&%S{PXtp@<^4h3RDrHBfaJ1WPOty368dpm4P;
z6S`nYAwpDn1KlVbEFaGKd?56dkETg)&-*LoBCZRUD@N0G0$(}XFZ>Ey=DY?|<jIvv
zl@U}}QCU@Acs@N;k_|v**mWpQt=6t*J9P;&M9%U^&H41Oy)K;Hz^Pz3rjMZq9$YEv
z?t|JgxjriFkm(f4y`TV<VUt)H@EynxV|f={y}UJ6MwIa?A%Ff_Ola@?d334}SCz;a
zeh}tVJxe8_uPQboEfyvhn?Z4ExiTr%<U<+E6g4$n*(~U-<2hO&Ot%i_6bZs=YYc<h
ztdX2TFPyLr(A~m`1zEIJxVK<D7j6~O7G{m<m#V^YnZt&8`~YfNfl;PG9_3PH@PU-c
z#2~9LG+W-FE5ky%Iu>4{fxgV4>a8;-3|0eulF+gkj)GpQ3N2)_4`u6S)Y#&ajZURM
zHNE1dC{+fn6YeZN%mrQ%+LmO|%fi(qskB1~SUSOX80(HLC`KQM=en#*2WVj7|K6?N
zQK8m08sa%<8|piT_2XvA_lj`UR)t|_EX$&8Lfx|QoVQgtzHIVD2N*k!MT<${-<SK+
zW0gAL-Mlx5=pMk&$P>j@F<ao3zM8JI<s)dwR2MHvrVvWpefq2u>Z&6~U}{XEia$$H
zA@@_$iIH{3<RDe57W_{hy>APrs=vmt+gC(k#(P%O(^A3wXqYc-E;he4gl8U}R0-p?
z;?bu<`Bv)^8fjfjhgetA0C@6t=H%Fz`R$ov5bViHEwf!Xd1P8L_;w-gu_D?ojI0k4
zwm%lg!E^0+EOK0AyjQGe2*23#<W%5j6a4IKuh$OsYt+X3HEW0Xwc3UgBO|;F_0xK@
ze5ZYyr4f?q!{wtC+g~gxeEZ{{Q&N34%eVSWRtR?6Y4YClm#k=0G~d}}e*9zV89IZ>
zc4{3M(La&Rg}f`wdEy1STIheGD}-rVxhZt0UaIss0GZ?><wzBJY)ns%plQP8C+q1L
zp>$=HCJE9llPbb?3Oy^cIpq~WSv!GR1aoZ_ez(_7@E(R!gO9Vh*F-_89Rc61uFfc!
zR!z~g$*Ncpp%NNab%pHnYT$DE8p7@ZI}c=K9h2s(whIMMjjLBhYai2x%4#rQrP0<P
z&!&97^0xL#<l$=}e??o1JmYA-@}%}D<TY~ssMd-+Q=$3F?b?ONJKH{^U5vc5?E>vm
z<ehD&YHj*Z)BD5+_+zvy^r4cVdirC!N;vy;6}H%st6%2GcA;-|Hb*o<z%%8X=BKWj
zXVlcs;~JA3bGe{ZNm7N!2SVeT(b!LpuUQ@$#(V|{%l9W3T2DD&8Frc(U8%xvqF{bD
zc36aor&f%A3Yf(*51Sz~SLJ_*>G?Lhm7UK(A>y{s^z3j>XAq7*8{-qYc2+N3OE!$k
z&?SQOYzf^WnCoIfX`g4Ed_>z9n~Y<7sJa<@;0HKQ_hJg(OxfaN_iO#jJ);4<8s(8!
zddSW`+ci;E<0@Om#&l_3NC?=B7p!YTL)Aa9qTUlR4)9e1UQ3;*>K}xbwOOOmO=^<F
z1b~sSJ+|zx{ozrpg;c2?V>}3xqaJLby1wJNuq8)XB^t1y%MGv=-*T0?IK$d>;us&o
zkvT&<v$K0uh>YV;^`3QHIF7E3sB^KVye`<E3(ZqrVJK3C*DA=JW86Q4a)8w_QZi1Z
zvJiFua~uQVQ4-39{^zo=bdB}Z^sTO@`ZW~Byo+Bqf{uY(rI5g3E;P*9b45h4O!g}0
z54YK>3Fm?zs+%D!ULPAR`!?u|nQTmN_;;)YVCUE?!gmTC>qm1-W(k#l3={f>D22Q0
zhXjQ-v*p7gB<v(0M@fG0NDxdLl3}rI8{SZ>c9?Mt>I!e*IOYT!H_c_A)ti>G8XS~_
z8=D>*%$9C`j8*n-evy@!kgaGO5!iF7P1E{6dXT~^gto?KX`^7+GMW|&mMxJS)eE*Q
zQ)r*ix#hi3kKepDb5pcUSU=%IFAHY)2jO(2R1ti=%l^DTqmOG_^^+m@ic9wJa9yK*
zvotSar#xIZl*q3YOijPiCxnq(H_%?8b89SqU)cIbdQxb9@mYGjOS^3-rAq`;vxpAQ
zHYd}*9ox^qIG^1<nRa&RcN9@9)5XGgAMJfvJQf)C3e^_jm;gsZxGJpiq`k}-!YghT
ztSzZDL}+cvj-fqr;Sj#j{)(Ax`Q<p;C72Xh1KN*_yQK<Up>VI|h4>GkH8rld2;3wi
zu^D;0G%sK)%Ga<u<W~vDb`J63YfJ&~gc(XQ539N})02ISWcg{nMo8E-1dhb4UBUSM
z$gZJ<w8!!X+JnvXgG=Rnw%jU%Z-gG@Y_mtV!+=k5u3#=wn(ucqHC;_?2WgM&aLf;c
z!@-+-PZ)dU_>@13b5Y^jT|<LtFFRyOm4TmuYJ-bvvk)$ZDKFbLP+@|&h~pcDb`dd`
z`$CV15JIDHL!8M$G^W<%kYzYl8%@s;y;Pwo1T*Ex8<5ArwRH%Px;k3(DQCDZ+}X`0
zYxSO?bgj_7=kMNd{yxr%h69++96(`jvKnm6i+DmfyEin1_B#3YSe9C@v9l8!#9eCp
zhzJ9Fhf-bF(6$^3m0Gdy6E@E;rGn;)m%@BcRbRz9{2IbpYyr>nj1XF1`WNS86I%Aq
z<-%_ZxAqs&bAq|O3dXO!{c+kU=nstJBF%#3z!Z8^Xg*K^qSAqv+1ae=<)wb=b|w&P
zuDrZUXQ(@brd)#%{z^7nvM%c@V=4Q#9Q=iH%KJj&;R&=y=y^6&_~vjNr?@64k31gH
z!|Vsx+Rf~l>_W`<KOpN)1$SC#J~G?W83wbAN0{N(ks*G{>r5w2{`5;PA;E65vgkv{
zQcrd3rTyViwhVQv(9%&&%XS<Mb(U4hWpPJmK}~9pPVmOTjM$KtOY^ja!il4i@o*<_
zM7O2ER3N@4yCpb)F`lIPJ}23c$?^;Ydvs`gNRCN^7~r+6bXiT}h+h7F%=MZ)vIa~%
z9GZC2HSN{u)O$C&up%v#=JD%<gRfy@|4lUSrUAme*N#982aj!lQ<w1iBeY*=emxa|
zjNaEDV+XLbH->PL_XX=4Lvh+`d}Dyl5E|dCrYhmioB!erSGpXXBe;<}LHRwT<5GFZ
zT4>c|<o|&D5~l8qCwxn6sYGu=SYk@oXcf_0Rukdbu2?p8?OjWB>$84$E(aQkrC{j_
zcn6aO;aT3rk(&hw%A|7NjZPbic+M5R6YwG7op)o@d@J+Ex~{&9`0)EXjwjQ?F4OVf
zDSYvyKO89>gNbBzmu*~A7#ob%UkAeXL*Bz4)UoIw!wV*H(mb!<Jiq5kr1BtU8Uol5
z<n560Fc0U=gg=&#_q@W;T^p@cW^bOS9QKqq)$+i7CNE+(c~U)^PsMSh)P=K5IPu3s
zPN^4e{V_2}bA@$ec?1zl1F>TZDVNHXj|xdAzM%Vdd~k-Q2xmXo!$tkpW&0COu&0F;
zA9mv~71t9J!^@r_^8!snd84*p-?m}9SwHQ3R><>9&9<qld81(K8O!0**0Y9F>4nao
zETQD2jtc>?lZz;3`Rt!3(D%FekE%I(T&O)g)`zz>?>DPWQE(KZd7JRg=~p?JHS5QZ
zQ$ttZ$48YO*_fkA9}WoiGl@{%H_uda3Rs@A`zZ!>_Hkz5KTF1e)%w|nh!7?ooN-=f
z(qj%RJ5{EB<BJ#qSGwc#A{s5s`FuGSc}?i~d_96x*x2pz!84Tbv14(ek*;f$D!5_B
zw<D=khIrcIFNSjn6gGc>y@~JY{6a_dcQ?b1vbe;?JbMp|O?=UJ2PgDX=hkq+*97yI
z(`5An7m@p*H*!3XnFkA7lLx<qMO-c1`LYY9sN;Mh=ebU}d_GEZP<CI-iDt)z;7OF%
z!*}q(K+3eIP1|rB)j>_dxtUOlSEYHHlcw!Njc_H@LUUA(uas$50jFFJVU}qth0+Vt
zFl`4f?5BZUITs&;**PG*^VcHyoBXBcIqrR-<x*ABT?E*Dry+okkXtl@OT(ma{V?YK
zX&><mwPa}Y2#0exdW5qp30u0REB5jUO1*{tE0d|EEAA^V8u)1HSDD+ZPi5|AI>ej|
zsZ5<K<a|AIyz0BukF(+#2oF|)s+iO(nPa3f)w^hN66JyHi*iqt7ovQlJac8L?}W2o
z|4t_gy;tAk$WMa(S{U6av|ZbjsBB^~;sY_v2(AdJ?B)$BU)XGGHx6IiMsnmjf0X}S
z?+^;UnFr6}*f;q>v|lRo`3a#|=N5Vb`_Kqozo7m0Y2Q8^UR9>=p?U1kRu*wUX!$mu
z<L?Xo-%gHL#SReQ(8pOBz5pB_FtQV0w5C%q_iv`7x_bNJBy@Cz|E-*c2C}tTR{c?C
z2weU>I<vlwO*?cN>hm98aZ%TW6*syAcROtfJS{9xP*zH1!A}Tj-~CLV5fc9X7tZH;
z*PXwQrZ~;UNh4qy3#3=@`;PRps)o&j5dL(Wp#3q7VgL9VE+_hLMho5!y`oJ0zQEtS
zgi~ez&18C@E8vz7<$QRd?ms8ShN)Jielu<z8%OF_<F-p>^m^*4aV$7r;~X1YqK2RT
z1O3?kb1xJ}`SbTM%}&aGP#UA-<xuLtWgjv58!7IDq7leWA7L9kfP1|T;i_Zuumg_>
zRALs_mO=Yub&ul{n6uG<i$T+dtlvh|e>-4`Q>O`*+Y42Ef;EE^-n<>>U64|(kWxPP
zSe^2-vPS5?9g4uDaPRhT_DT2!mLx%__+_}7PcS9;92c5?iK0^1v0t90SUw4Np03}@
zmI^|DL_U#~`afz}gE6_`8Gu(?o+dWglCon>nPC~*R#65#qGwZvvPrBA?qu6$yVU|*
z=F0P~I(ElW?kY1l%Yv`VG}Z`_|NJaHJs(?Kez22Xeh98E%7T1Z8E7Oq;lPuFuxAEW
z;wt-8i2JV{e#(<948sCPP%mGpR0q}nuebda=~(Pa*?_ZiXQ}tQT}{72EpmkuckAh+
zLdI_h{`@4^ek;O!o%k&ZVX&*eg#~^FbMLxr%W~dmlm;9UwEqg_qJ9t({<Xe7@_MY9
zCkQm#q|(Sw(TATrqSDB>kS7=w`ik5gTVOG3JIEdv{0uH+_{P_=zGY>&#~@NExyU4t
zdLeVW=SEL|)GHg0WqsyT=Vx}&^JVlHF8XT3*|}UQjrf5NQ*i|*pDYd7ER6moky$#|
zxS8Gf{R~u4fA49{uvjypXs}yozV}$Y+HRYuZe)?+*mm7*c~Px_qmrSnLHX)f;}-)`
zX~=EFU(9eSYmmPJqezhNLH^{WDnkrw6T2gkPHWS(qFt#B&rYN=%|irAtG0rcO`4~C
zBDLJJm(^S1cq6-><b}2Ohf}j~?|!E6_xo!fRj~yei^kGC4sNHHg&{-J3oOZe!nh~1
zetWUZc-Q2w<z*tHc%$ZZ&jzOb2*jK+C8`V?+j9$??dcDx<+=i?%tJl{!ywF&X$5c<
zmSnZnVp7|>`u=;Hl5P~O3K~T#yX^}4n4%sxy=>fI0nXWWxM92Hu2R)@OPab7{>iA+
za_wcQG*phgY-orrfQ>zh7?S~}`hBU?uoK?{fRm74i@NQ=UBZTi?a)tVElP)zfOmmZ
z8gvM1c@$|o*vkdHk&lYt@>n4XrbwlMGhm7>E`(`-M~O`yba)`QkC|(i4_1U=t==+@
zbf58{I-18D8&8<~nR4YT_yptIiBz@@i?V(*9r9t9W^3R>t~L!pL|?<HO`ZsjdEs_J
zOX`}QG4_E`xt(a#&>ft*wEG<m{Snt$;(jl>n4T7uS~@gd%e*=^6sUy-VYNa?`mR(J
zc~0I$7I|)f5x0s3TDp*Wi(Oip9Hg)^DPltiG{r(PY-5O3Bnh3rhl)YoGzV8;%u#^x
z3o#8<=tY}19pa<FmVz=IP)x%(huG##hl7t#yy+aey*u28dQtkRn4qKopgYBIJ^c$8
zeod70bTnPtZSbY}9H;-Td$B(aV%$F)K*N0Laj8r%A4`~n0p~(I5<n+R3|PU~j<xr&
zL&h7;)}aG9Qs#rlUriw%eK&7J%5nJE*_o8>i-@irN-f8>7Zv$W%OWa91=67`V4gw5
zX@N8fSyMC-9}lESbgS4NNb@+}E=qxPekkR&Y5H*Oc)b#*by<N~AdTJDiPj)!;0K~T
zh+d)-#nNCJ3sKhv)8%0O`(XNQ2>sXM9*<UeKPmBwH!Xs^<RK`<Zw)k!?&~&$&|sQw
zm&(25NsvmsieV!Ha2;`1uv>{l{C-3kyE!08O5_8X&LPiBC7$<1TPQ70^qQV}YND^#
z2cCOyi7zLdM;Ac?23bP5yMzP~!NA?+-}pJABb3I5AF|ew8*HLt^FOv&GYC99vOTrj
z*a7VnM~6Xadc>A6=wJ^+ns#+!h<An`vo{=&zhq^I->Gdqlq~L{{Ni|W0qdm_e5p~c
zM4s`;G3iU?9+OhbJvw3e=Qt&yJjCqeU<f}oU6ls+axDw%TmFuq0Cr^9OV~!uIg^K&
z8BWa{_mgN3ryu9h{^*__=ocm~LEcJdu*DSp3!-ns=;i1ZAIj=d_^jVjK3AJfzC@k!
zGvAxigi~FT&apKly>seeK0$1Xps`a+$LYD%O;=*#rXJxFqK!Qy8q7x5`q1e0%qzkG
zzXzFNYIrw9nUb%uq*PN|N|R5tX%x{n_YeI|A|6j+W7sa<jiAp(8i~+uR7%8qh*TOi
zg9(0{eUu!KMljkYwnWk)zRn9b!JfoPlyfjipNXU+X{mTOlFsFTRS-pkQm7-PT4_(=
zl{G2Dl+FCv1iM64olen($1}0I^4`cRnE=cgRNNayGwK~|__0Mk)RNA#u!cn{i&_Q2
zwz6{L+YnPM-)Z~@#Pr<2fJq6ZI;1>q8scp<eXs1D8jdz?HuUDx%C2Qmue8PK?6C>h
z+{(CXv853G0zDo9C=FhM(%*`&JQcC(=4NXRk*GM=NJ}~26=J6mOW}Zc!AM8Pzm;09
zvc=Bvp-|i$sU!qirovrLCiQCYR3g6`Wy{2bXgZJD#fE5Fg2i(+nx;l<Pg$c%OWi`a
ztuf{_wkxrH9J^*|$}_4oF)4<Q0A@uDoj`Ystugd@*K!^ji<LD@w8o;MQ*4W+LxP}G
zp(Ii&QO&^SD<2}I5<`f1IhGy{^1JW3hQ)qIEs4$MhqLjR;#E7umN+_=8pTiI=q$Vw
z5)JWm93|qccp8j@xM+)~<5}^+co+yZuFAN^GZlv%EU;VMi+DvaZf;7|Z;9bU=pvW|
z`w*HJZN!dBESv3db_k*zW5MN)rzZ8v#L;XDZSh3>eh96gtHhk4G!K2fFq9hUcJbIy
z8m28vDI)>ch*$_pJU^6Xz=Y}(Xz;Mz@WH^m(TJ`y7#+4k(T|zISY-&CaKqu53=<0y
zXrQhn>OLgl`Dyrpgh|QZzcv9<pDOkx&@k#N-b$dU)GEdeqlxIhXc(Qq1#c0XhtbI}
z1{a3W4RoJaI-C}TG~%|iQHoOSV=E2o+Tv75k4fUG;q)e5Cw7jYMf9Yo97*4#3&eLu
z(hq&lq2mNME<M@ZFbXq7)5V@d`UqVsCXA-jXqi|$nhLO~d=kxyn5ezy7lbxxK2!Xd
zzf|Tk%g@e6EX@s9iHnoyd_VnBY-Wa>BZ%#?o4dJwJH-o0bZCg|idCPU7!83a*=nx7
zI&msYe3BS2hJH;OMaLM}?$<^ASi0GFA7~aZn}fcL(%j&+V#ipz65|L@rlFiioS2pj
z85D?h$+Rlume(dO@6rdJdj1+$xqL0Ld9`_db@M;GZi#o2X_4=+zK`V-l7+c((%gWE
z?oty}3A*$`3LP5B+nAYQOVe}+54b$A1hx2jlbn=cyhH3up*RdXMB_NRkn<fWHjJaQ
zq1WaSMErUj^cw7^(kdD+)~3=Y)x0gm16Jc!Dg^1>osdR@6kOO9(L4c*=zXzu0{sLl
z#F{~$r5f>S2AxHp=^i<ezDCjhawhG@3T@89#=A`H$)ZaUXVCWG`s)*t1Wn^@S4o0S
z@N1MJ!+%=l&)e8eImJxlpeHNLkj8eg(~L#n`Ga`N43370(<h<Vb>hKE^l7?949}+9
z#yr97ysR69^e|(^wBD4?UB>wFegY<TJjFr@+se$P`2=@ayqZn3xqzQU<76o2WwCBD
zokF2ilj+leCvZ1e3Za%@`INB@O_~(iA{OL8P{B6wogA7)JH$IVG=@`HMMEx4rgyui
z=Q1O4TwI+;UmlLLnA{~>n||NP3ot5ZCob=IW7@5@SaMfpb*|4OF=q;0&++YI?-crM
z`gW!nFxuFSNq%#Ck{aKao9*z62nm%+l!Xt~d!inj09`1RD9(v>7U)>1*k-|MNEaOz
z3?^Mnn@VS5S~gFmLx+T#uuchkaB%*3n1eUqk>z!73zG&+NZI*=VGe1Mu1UNwm5xsg
zsg=tycvjB)1NUzs(;qpNc?dpFC9{tW9TP=n)Kk*AQZXZ+Msi$&Sdk9_DQd)?e3)dF
zcq1Q*eOIiTMnk;tKvkNjMz|Zx8wFvsS!|s~H^T%Nr_;T_Ix!v4T`^|{CL>L3pFw|x
z1!$d#*?(QUI+I3HrTF_yD&lTcY?(!iA|`^A0UlN^6Qo5T#t~NNH_K=6lBg_ToV1I!
z0=g}56;q8fZ9U`VNTY%0(3BUoh0x@QqNR{-!s{fluMpcJZSGE(O`k-RMci9NNBDTR
zOHsjF@dPSVn(I4Jyii2*q25t*Xl#t8-Q=%?yBaVNnroM58!san1E%qscBwc3Mq>`$
z94+e`+a{SRN#**Bh?HNJiX-1;w@XrS$S_e`OkZLg94n?Tcvte0C$C-NXP4&kJH?#2
z7zV03=7LuUbS|xrC}y|O0#w3y0DEPW_fNoXl%x4`)!&Ks64@iXS%Mj$qOKI4aD=$H
zlztD{PA|jaStTATgQ7;TPnh_RG8#@LQD080u{7%tMS+~fa+*N<x_ip$do)7PF=C;f
zSdw){_JRgar;IN3T#qtQsK5m2#7h-)3=cOv9g8hb)K}7!`qNpHaA{DeZPd33ZAl#4
zE40n}jR;R%M&5`SOL)Ex1{NU-TN_ERAhC|{B*$pEfRm=(ho%Zy1D-;w9p-^BCjqz>
z^1c&If2>}oYe$>35euNxl*<R*Ua4ID9n)K>OgB+HRY`;E?`}J0Ps92xjV+3#m>Vr~
z{G{^8>oQtJ*b?^Le9R~-Pwcuk2W%O9GgBmdT^D&Ad>q6gOM-8sZQlqh*no>~o2Y1)
z%S*XW;a3p0w6L=EF5KNHONuS3kCe{?0IX6e_Y8agA@i$1d?O8aK2E~$SXHgAauBA+
zF`j8o;_7Rt>xF#eCwi>R7E2}6TclDIyjYcl$05sOlhF^mNhyw$?~|}c^Gx2$>M^$u
zRc=9qUGJ(?E<+vDR#!*fspaYfmoO(onEgx&tj;uy??jy$9iE<wCl<>x2)ni|bG9Sr
zP$%D)m<)OX2FBn`V0jECGJ!-=(Og9*(hK5qRWxIWuaYf4?4a$=6K;4QI}$QIV2ZWr
zx&0rQ48Cfyzlug0X>WBKX~wM^S1pj%sVnJmpISe=9Q+bIjF$6aq?Ja%1ZP<3_z>DZ
z?%k|`7Z2O@vI#&VTP^LktfJyJD;>dk6Y+$VW|cL1?NkQF2#M6TiZtf$R5lO7`MZ>@
zDC_jvr3_;AsB5=9MeM9T^^Vr;)uIfr)si&1u0`45f}34%r#NQ;oiOIOtGL5e+~b0e
zyWn0I+~b1##GVB-%y9g#EzU<;rc^x3s`Ja1ElHY2{Ko=1v|iK7Y;E-hEicUtqD)z*
zV%oOZ6K2QSw|U^oRfui&K8`b~_W8P@Y+bQ+CQq$=pf_MAOs7dwr5~<=Jj~8AYBr^l
zK9oIS3grnCnI%Kjv&}xV*Rqu)NOS$S%X*6KE7k+IGjQ+<#P)?WHf%NPU-l;v<HaH<
zx%oGbPF7~QDt@z&hSVeMp>DQZA>`721OGxs6T96o!sA{Q?T^<~<(jf+D5^zX%(8=O
z#QTZU#`9QR^Ymk)p=(k_=*j4nk%$|xqg;Rnp&&h5wX&|2`y*Q8aoLrJ-to;W51bmQ
z-2W}p$HNs#_|u8Llwpdrlzat_P+lF(sx;QU)`-<1n*bm#Ky1b=wlAV#_0h=6(Iadb
zHnymk{+j|J5EJbHJOgj}uE=X5keh2srtFlHX9|W3g}V|j5oaah#qDS}l4bAEjvYN1
z(|OuH^#tvhyoD4@exA&>T&3-kUqHTz%w`igZG#8wx^^8~tlRAMBly?`-S76g5&GEm
zquDpqh{2l{DLQ227xAorF7B#u5wXoa{BEoO!2o~V)#1M|e0=}K%2L}D9f(71L99}K
zVaML;G5?hb2hWCJinOqdzEk_&h`I~9A8)y8<QZdA>%pl!Vt+Opv=gJ7Ji=;Z8)yPX
z{@do=#^F<1i1B%0c(%^2uVA(h_vLNY48;mIN_cL=Ckv#2l}~QN2%{BNZ89D<W%#wa
z)Q>>bYNSf<5U5|ftl81t?WPRMOm>x+u!I`I7%QL@$|zZrX%M3XH_~9KqvE_JG%8>V
zk0&TjGgYozhNxJd<ty>UC0PGc#A8cnQRUWg@8V_iT>iIMvyS5XLu`)jcb1TcnsC@8
zga66a%`zm}M9AxcFTj}y5m>AXtO30KGD}g0QKobc2YUz0%FynjSh19b#fQXZk7ndF
zv6DyRE{u`(xXn~wZt98VBd|-iErnNlSG=<ndvB>|c!Yi%sA6YGev#)MoEXY9YdpV_
zinR+wtqs0fNcTt^Ra1kOy<Dqdvvoc_q*jX3sG#Oayjsj(Mk8>)gum2)mk}t|ECbm?
zV#_kv1#c)xDN`2I{1f?hSN<CEHBwaIC@4iKQx4<<k#BeA1MVYV1GCYF@~v!+EuVUj
z)T^2A8&6>j{=}fO`*pnWMAc91%MujEhyT=5x#()<Ndzt5Vr)jQ9LbiQk69{L-DW3>
zsh+$Ir;0r(`B_&|s(n^(nQmOo9O$XxC_g05T#lXiP}tp+Ygx(9uQJ+A0a;gGXo<;Z
zk18ZW8ey$Fy)F8-sb$27IHs>u?6F|-Hd5fabt^?T6v|ybH6P|FDk_3?v*oYR%YCf+
zmKQfguko?zTNn+?J$e~OUNh5a<3AE<bWtKTcHz4+k>;4br=8$&Y)U@1y%oqG;u=$`
zxiqH<@oC2eY`SdUoR_iDi+VEI%fLjb(uYM~eWjAHb7Dd@otP*;+T?aK<SpMC{TLUU
z8Zx&rE*g&6L)kZ5Bn2C~Gam<6qEIb+V%^m=KLSl5<c1f7tZS36X8I%l5{tgQAWZy>
z_Y@;n(7YH~W)KiFJzg!$#K7+_r-dvYfqT_J-b389f*R||Z%iaEE;*^v>lr7Ph66bJ
z1c16PsBdD_@i^zh4MN7_?1~UnXJHB7-Mk*RLUpuLUjFhap7t@Ao$jll4zUoOR8GdE
zmMeHEicXoln-sXpV3ZYROAwIJ;agI9#3<GP4ezrRoBAk?st>=&)P>PAh3at_&U}B$
zZjyo1Mhim8%15qco@VyV6oH)$Dz)e@P_3((=c=#auSm+n@z*t(Z}pyN9~IjGL)?gs
z`~@m$KP6=wGK^-+X6lgnD2x0IeAuB5U!x7X27QaE2J6#Ok6)}qc6eu<-@NXd0b%$o
zI-hxxA@VmoJy>UtoX$Kgm4~gZKAm}vHDm&nnQcQ`csMt&>(4R<8N-a0tL&i6FxoiB
zA9|nBV(pm)OIbtI^_R@yk+v?6PG-t@{P~3KgvW8q36BH_$;#X@mQvFdl5-?D$&@Rd
zejx(#mhrR?Gd1_`*iQH;E*mGDFpTUF@GoofJq685&+|B#mSR^ZW9KqFws<!spKuyo
z8jJNzmZJ1@RriI*=yZzH86n-_HFSy}Jtr<+59i;v`_y`bR&a)~Gyt3`b~Moa4BCj3
zTC{j#Bi+U6Z;Nv_(KVdzx9+}8^a!O##np{C8+ie4q>ob?BATCvp<5=lJ`Y+8#PAnz
z3V%Xud4X=Dg<@P2patD+O>{Dcvw#1KbcZ_fV(K>&*NG#y(Og9^`A+QFMu&$+ZAbLh
z-di_+3ETubb46txFA5(eDw}b@)`&UH^e`8)TfEf_?m|TCb_7U#*NS!9aqjlLEw*o`
z!#S+VvkW#Ai;nFyvflfdMOP>8L!@iFRHDAZ4t}53wYP$D8Q@9m^bB}4gVC!NcQ2Eq
zxkNs7OGUbCG6yAidnn_Xjpyp*Kby0dzJd`h(&2GO)_s{)u@)EqM8DBG;SkiOLW&C8
ziYpRBgSd4EjplenrFPJmppcX7jdSnS(w*#U*?>#GIpn%1?Vvx_YYu4J^?S4@;MTIs
zy(3u{afU0EiZv(u_8_S941qhZIoh`;l;tas-`=+;gyrY=?J?XnJ%ZO;#hPU%8)?7}
zu?g>i*f!}Y6=@^bLYD(qMOqb9B3C{d6nXuGe9qvz+$Mi}Mx`d6Q#G|44U^AjZjp*2
zzY`a?;Fvi~Y;U2VK76!kDXwRkIV|QiVs8r_!g;(eI#9=XtP{06X`-LzKI>;|&rqJx
zmWnjj#OWxn5BmV_VGjD0Urk8G>UC^P%$_5^jEzYSSQV=a*_ftX#QO#InxcrGC_f?a
zlk@R%Kqr){xA!1;gJ8c@jP2$b`MpdTE*On~&mmrp-7!)`+nsFESdtV){95tePT1XH
zqHz~O6S{Xr^DbNhRf;B&#zgAhm#5{#tgW#YLd?&T8uJ$NWNLZ9NvSAeiC8Pr7rCI<
zMP)1WN>MB(w9?3I)lMb~y!Z@v9e?*oMXE=cXr1C!6*Cdw?&5&GZS=d)1=3)*!Zk7?
z9N;5KMchuYk<kk(7LT>kaa`bc;*C~1YV@3_Rd5kz!u3dH&t+nj?RYMO%Tp`|$n#?C
zWDpp<%qY@e9J^`g5dJN;`$(nXWwH<QZ;r;&ys(X;eK*ba@j75?pjw$;vEsVex0?p}
zd0s~l!h)@3ecw`qeToC7ig$O@p<@=v;w$zDk;#I08^Z0lv>V0X=a(4uY^>YtS%>fz
zAL8e(iv@cyCO%25-9w*8kb2i%EJ;4Cd*ogebCFMo1^b{be44moAI2PZz5C!kysy<5
ziDw&EVKx!*&6ntC&S#m}{}O!+bh-=n(=Tag(5GIzxw12D?Qg?w>zNR>f**UGl-8Yb
z00B-;l_J)>Om7Cte-U@4?S$^UV7SZha0hG=YhIyQV{ppEA)+`q7yCqqD~|(;qKBce
zKjpBD=dJi-HO_{w6vjn}{jbpAQH@U2pm*KPg7ytIgBG$^qE^;i!n>I6xP$bbSGT>B
z{@P3};_6A<Ky;deR9Ky9{we3TQtK|Ol*HC-ZXR9HeJ7jq@}y1g&EWQPzHJWLVsp@)
z4hMZv@1SBrcjqkbSA}nqgM4p6Dn+s(t?oWv#4Vzl1P8fka*&_RVq7sdI-<`(_kzT}
zyAHZP0V&5p4>UOF%PWvNx@(HLBZ^6v9aPtYWN=U))cYKBeOdnhBk3~GuAsZLj2qC@
z*E(oAX#Bf0e)0caKEA?1$AgdYTBJIpR-_K36G-4YwZNGspzQ?E&cQb?+(C0Cd@JG?
z5#pfv79`;2gJ(7SmEYTabrJW7qW-Ld&NMsdEFjH7gM!sa7-XT+L1$Z$?l@=>h|U2#
z9px;|)B{GR@)xw&SI+l3Xfbin5-=>|A<xPO+x@$5{l6>xzfo2K#*yl2$<=?)Vd-=S
zErlpc4?62hU88@Pxw40(!ph4qt?bK2JquGX>o`&x66nlobfpe_JCRu3@p@-w<{XGQ
z04W3MG6+C)MQBvig7nTqWjD}?3H5XS-&0u=@X&S;ugnVA8q0o+lM9NC@r#X?f=R~s
z6=K#h?l&=GIcLxo6pbZCxwEHEnqD=wJ7+mp=~bUPDmiiKqU02~K`WOmSh(!*m7{Dm
zH48>P_5b((bb#)0Pp&|!Luy27MruRqK<Y#~fz*rChjbO`mV;YV;Y?578|vUz@=jo7
z0P=B2Bauu<8A#Jzb=|X`;sh$re~RPhu0(GuS0gpJDw~{NMs;Ny(m_}Gapcds^8Ls=
z99%89t<@tLkP?uRob|P4vEV6gTx4y53$~&T{nXYV)ghs;+Q#mt)!cU)@!Wc@G2{P4
zaN^Dl+_?H1gxEI`@`Qc>Kaf9hkTzG@FoV17fYpgtqbGz?R_-80CRbbEhs#(U`*D}e
zHV~r65t8J=b0n<JLCRcZ^$hXhLYWzVhU8>sa+0%?VMz~_odCgV)FYNV;38;Q>?(`-
zpJnz<&UOkH-l4OM?t>p8l6r`J4V=f47Ra_0sf{Qpl%V#YgP3k1Y-B|Kn1dwhiB^eu
zs(r^nl8zB?-i-VS2N@Gi{PYaxluUD|k>#CzB$yfgWx0Hqo#EdQGd6M)sIhzXMsB~d
zzJN!31=UC+=OA6+l@DMv8@0k+KdJ_B4-j4WqtIQV)&+wE*dwFscn`TAA{%WdB&mZ}
z%di#sV}y*sg^COV7i^RIF^vxuG@-D^-QgH?Ft&wP4K^6tN=Whv7rq(!9vloMR|jdx
z-@+A>)lDx2^O4%lBM470F~}bzB&~q~&Y!eKQcu!aA1Y`A!ZG)V(mDxAPv8|Y0Tic?
zL@IUHr&j>J;ASu#5*wf9#vh*nxXr~7nvHKqay$gzsV8JYyPLoSaF~HVY8vb?BL^^J
z$n}!}h9=gyMvM_ntVL>ZF@(=VNNC~-H~z#PLNa+boT&wDcQcq--++Q!ZjLe?gk&YS
z8)RX`S*!!sPZl~b*SH9P0W%nwq;yZ&B%n@uNT()2r?M;D?X#_bI{>pHr26b*D7fG*
z$c6|fk90G{*52gf55Xq@-*a~e0g;@XhhPgKx%lHmxdSkmYeH&qx6j1@^5#6KuP1q>
zh#cT?{$K;Ph%I&(Lm+N}ge;Pa00>%c5i)g-t3$w3(SGVXZjnwsj>}k!8$W+K;8Sh}
z^Lr86wGQHA{->eC={8pb1M({fnQ_5|kcRwaLS`O!F@z3hLISfA+yrNhB&0y;UIhg}
zEtu{eaX|s#8*cmpiI774xuHDZdLS2~lfsAQs1S2h*oX?(Phk__cEGNp!h?X%x(OEc
z5i&c#-66BJvkmU{v%>+ec&NSxaF25e>KW15rwA!Z8Waign6o0p*yRpUUj&I1JtX3y
z<A5I;QISN*oB$x=AN!dD2Ie%lJDk%9xZh20&J99}88VOc&wh%PNDp;bj1K4KxCzX)
z5K?-O$FUj6n1a#{q=y(Nl?W*daCg9VmNFBN89si>(g-Pcxaw`l-yvikTb8`De=<)?
zNL8B4QlW7b3`bS1ivT`VtC4VhA`@irJ4k&l7#&nyAjJC6s=<8>v955};}I08KEeA8
z4xqXR@MYAq`DZ`XSCQ_z8&uyTWJQB}N>(%y^5{YLDtWYnkjFYO1lArBdF&V=HMe-}
zAY9|XelCwtCQqKg0H0`c(|e*Fu*6e&{#m0ZZV~e2Lk*rpgC`$i;7KsBvcb*3O2D-_
zZjsko0K1fm(PMjIZ9g*j$9`%t=c`7#@mE1Yt87gE3;}7?3PPSjG);yvqNgCz)#2_D
ztu_Ka>o&cs`v`gF>H|YUp6MrK&1%;O(SA)GA<rIk<3HO$NFDypbFja<k%T;Vwcahl
z=Rm-YC+>p`*aHYz*W`xRHRA~0=!Vxf5wc-WVi>>%NNhtJGWf^RhIWJ%7Q0{+HQ11j
zxh2rhNyx_f1h<xN1V<YWx*FiKu><Lz3kCtWXk=6ALvRJ)vo08)O_0cDvm4%=1Nely
z!_7U2-=B5&*9iRjElPKTEy!-Ec?hm0q^ZueDzJo_?1XGXI8GiR;B64`jwIKVS&%m&
z*<J0y?2ZPc9yh%m&<UZ_&4BO@)_+TyTd7(y2x)o8Jx49a0pD?tpye(htxawnX~mT6
z#`ToUkREv@A$yv5uR(YZ>h~($^?Sj~UM+sPe)j62e|!7g4D7u@NZSebGHL4p-0vO%
zbBp)2x|MEU8zK7(-0=Q6gdC`F!^~xVxx-EW<zs{#>UG<JL-l6~Ir5N0aRefNt-v)x
z1M(n<8&RiHVT7+W5pwLI-SXICz<1peJ9dwd*9R>hn!S!`dSlS`qx~B-guF@ItEB!-
zi0sW%t~mk1o4tf|THGS*oDTS*b=`T2kaufc3>uMNO~`xrgX=+&zgGcxkX{n<px2$`
zq7S$mI&yr_KS;v-A7=%4rY;iz!to1)yg%sS0DixgkUtpR9sD7VkPp*bdqFAk(3z9B
zTuR!Eyn~QaJv^SlW0LjAgW=N;dDW*c6Y}xkvSs_v$67-Ef*7aVz<@kNc&65^bZ0Qa
zGp&r8^XE((;0taoJ_CZE*?I55_MbHnaxTfqAO!>GFu*TePAr4J9Eo(yHDG4{zwE>%
zd=jsd30RRwgUjJ=Lvz_k$Q6TI0$0Kb`RXAZ_-ZjB-^95_4BT%L2>D^8Ye0a1NFwCN
zIQOdg5%o9m-?6d&>Y0dcqQhGmZY{lKCgkS~7lBsf0Xr=2dIxmkceA^L-*W)31~vR+
z>GwLM;}5|n2=y|$@D0evAu)$X-v4<3RzkH+4?3XQX26#p5TM$tg!)-rBLo4z>4XN5
z2YUkzPy%*)L^Pm}&>;6-K!fTC4K_Z|dKwG{rZg~y*#4gcppnq2{STHMoqB`Ne7Dlj
z{L6svx*5v9N9eS0H$2S<_~3(ONT+oG9&}nUz-hM#E$e--*U+-FgjO^{XXO1~51@(A
z$Bwyl0K~R}<JLi?Oh6tT?jGst5cs>n^zK0i10C$HA#_jL1D&CJG60)_@BE>AatM88
z#RDCpuK@oQI5?Su2(PS00h>ctgIB=7D|HXScEG1x44}hTdI>$qi^p5JSMv^np~G`5
z5858abaq(X@ME2XKJIYC9q9bE8*X@A5~1rF#g5(FkRS=GgH*Uk8Sx!C@ZVCccw;vg
zGwwf22~kNT$G^sxdv+6<vXM4$8x&{@;caeqOMi0<1h{$-7JTW*ZtEUyvw};Y0K!D`
zdQK_wFL5!-_cHFKp=h{%pszsg(?R$?WOx)B|JOnf(ex5mr9lYkSQ?T6I0pT|T^4tG
z&4Pp|&ugu$P^>`*S_k(kP+tS;YlEmgKz$XcuLAW|85Mo7_D6yFDllI)h#C92C=GCq
z0_W%;juki^z&Q$>qb_QFHLMVr9l-1u#5AJt2rxT<*<lc`x~Lri&Jh_$>~k`A7)VEe
zbYzg+I<!0ta)*I>*h%hR5QT#~1kA(0JnY20<17T`Az&VoF?Vf{2Ye8ihk$v=Me}#o
z{2(w70`s7g=D_a`6utt?gTOrK!n~_zg{2O~vK)Lnx)<!>I8HF))yURYxS<}L(VkDp
zE(kYY?0cW8x_u6Ehxp1=1ZXynbdXGkgJdH>v!UHVHdQ#t<{k%W=tRB`c^D_>j{^Y+
z;AkBr{uJ;L_m4jd97MY&@eZ7%0fWxQ4mLj0)+6d)=Vsw;H~y<wyWT<eG&^Mff>pLQ
zGyUk+zrlU0h-hTg9mLn@Aafd#3{JR6B1FjnZ)A7c+gun`-Ekm((OufbS(Nl<_knJ%
zI)1o{s(n=<PpN36Dn43;zc#|}kZ6@kou~>?Q+%WSReoxJquN&-dyTX5iN-`DRV#k(
z{`DHC)+`89EB>Z1s#N|mogp#>?>H3=S7}Wus#s%8RH0CBRQbh$R)k99uhN37w|cZn
zGa7`cN`owlj3`S2k+{SRmA6l#5|q@6-F3$IzUMCBnNIh`|KdCi^pe=`qe!Ehx<hmd
zodyFNW>Ab*a~rhXOClBPed#;h(rCp~74#jks!6e#`|}f`?*>IkH{GhZmKE|nmjOoo
z{R33t=$D46!e`09<HcXkDsl~*YcSjZO8xQAU*)5w$}q63_*I<qnIbMWJ%eycs!miC
zB&v89+hMM6D84H77;(dAiq~krIQnx%5Zxu_eXjWLAsgyZg_c}^O2<CH#O0WhQ~XYh
z7<!p90$qIHz5ENsJJi^|iYkW#?th$d5*(mjs?JjRs%Zu|(Wn(W#msYxu=xLaj@7Ca
zjg-l;%r#A8A=)yatJGsv-ioip=gujD>3(t7IYmYNhqY)Ch;gactHLXQg7GKjD=~K;
zdQnxhI!L7oS4AePRAH)-CPw;pO(MFGXDTfb1tV2y>VSNsYJ}oN9c2iPe}}MCrBZCx
z(e-MDW>7{GflPf>TE(wxK~b%EkxK-3KZ#F#sYt|=?)EPg&(Skt)Oke}rmX(FB9{K#
z{mOX-{(NMePlh5gUtOy*sz<5(F&VxpPObP{{M!Y^ga~6|A`^(9V_n|Cf6PYJMz!K?
z5Z5aHET&#m3=RDBX(lBIE=a9-gJ;cEiZ8{L7Zo$`xGE7K+V_cnxu{6c9p-_mj+LiX
zrTAkv{j1_}8s){NDq7_gg}P{!jzQ3K^%z&SyWtYJEgz!}P(`uwC{>i&U*(^uiiid@
z%3l>BE5$?=qoYz!hDwc9E3R`?rC02SFoqzXs2Up$alEva5r_sJc>0E;0sg8XYAW8l
ztQg4+)N~KOqDauty6&RCDSr2$_rzI0Dw4R}N#eR66_3yw@rNH3XSh8X;(Iq0(cGS#
z?khJHp{nQ_#cmDy^=AtX3n`Rx93tk*3>76R&VhYa47ja`p><;FZABFRxT)l}0{@ST
z`1NhYo7_RY*z}9ys>-T3qNL)<Ulal2x_>C}Kt<8+L8GI^?X0BWjv_^DxTDCYHQis_
zQ4FJ84e9>ZKcU9lx+JmlS4A6b>Yi~|;isT2-PYd~^@{q7SU-d;S5la#@%T$)QmDkQ
zM3ut9t;*q$orJU$kDf_#Az?^!8PXK?Ybis5c_fMf6H&=eRVpt6W%?1k3YvWx82D?I
z(S`WphicYQI8x-TGwZ05eJ`$~oCk>|RhRxsm8xZr)e`Ldhel3L2m^M}B}VS`cUV!f
zxNblZ!7ck;Y#&eram((D#|I#uaPjhh!cOfw%aoD6`*Om9&}?)bpA)S?XX|!uRmLW@
zEk(8x-a$!tPP{kj-y;Ep_`JpEMDeKUe1u^l?x-9eit*6%JNuO}^}ZPNu|skRSxNxB
z9_#`rfKR`00hl`Sk-`NiAV4|Gjaj*5@iO8pevftE$$p(@eT1P8Zcgw@RI@DE#fUe|
zaRHnQrJis*P+s?ovs~t2ZMF+w9K86o3t${Hdb$9a1G$H{S1x(b1>=Al!+y<S<ii;U
z7lRkp5IO7QK(o>X$Q*?4;*?72FRD1@#QNlwOCI3Lq~2zvE;3RPAf=cayl^#<vrbY+
zrnmr^l#H??c;N~c%2E=%&^DN7BY8E)iN2JL<mH1dKpu(QHX1{D&;jdTk<5P0W&KC8
z{x6Du<JgP^FGe4%1QQm#7=1YNtdGq}E`ar6-{Jz~J{Xdc%@@WXH;}tvLsZB^jAHe3
zf)}BhWt~IpHMszJh%!ojbMPYcFXN#9Md;s|XZ^SCbOEgY?FU?d+`rtobLEl;9kBkD
zY_^p0Y#CYq=YtoZf0lLjzj?n4ko#xls(M}U0`xEAp#KHv-<fCqpJ;Xgtp9hPcL8$$
za^umsR2}R<?w{$a@;qB7?X!lD55_uC?G9eNl8iTCU4e(-CCG=!c@FD`^HtWf&ki2H
z75-ZGdxsvsmA(P&ce4S%Rl0EW?muIEG~Q*?HY0#%myVB-i6pSD0=)uqBUo1ws<y1p
zjbv4_>d*fyH%bQBV4iHuH9C6+FzWMMC%IiJWf<KdhO3k#=rPfvQkt_?<c^l{Jb?!a
zNiv`X&{3Q_MkeP2Ao?G<V`V@`^7Oe=kX3(#;`)2yC!pnhN6zQYz!j$YjyRxF#!Xc+
zCvG$)%MH1C5WU9`*p|%;a;M>?kb1|;+A+x_w}9n8VuKoHnUgyXgYvi!LKsx4%mrKY
zYj%n&)XESVBW_YFhXf|);{Sw&Z1F#Ckcgz*Oj$_cNwqSo-s64(3Oy=wbAZ8nj|9Na
z&CO-#-{AQe@la+EKUr3&m6=mKfFoqsgRx?X(dQO}75)<t@Yd$eb+ZzaTY?HAufaK4
zxur6o09wby+_EegQUW^lYHqpQNCf~QohR3L0O)!tmz~+kQiMK;w?}TJjHUswCPMs^
zhcaYbcCJ-MaLIgu4CokN-P^o!@#i~4vl!^g#@wZH6%$eU0@0gShNO_(WpbI!QFiWf
z8DI?UjmfQ+0mjg&>ACFCOO~nuY`!DTLsP01@i${Jmczk|7uJ&8+7LE;M%sHr?o%?d
z7l1Pxa-Ws~rXAYlx$;qWsW)S8#e0feJL)uFqiRiz_$`PAFVEc|qjL;nX_h{B6F|>Y
zQKwNFN)9t6y|6Kt1x(3O#$}$6`#d_<999LdSh#@XzQBa@9+MPwxk(1J<Wa1diyX<_
z#z?(@I{z8D&2pa1Mr^rBd`<%^*GB&Y=!urxR$0LGzXvDUv&{;kqiHuQ(0mOV$6T5K
z7MCyg&wUxRyfQ!+O8AP*8WTgyceyO6OqMdHhwRNgBm=UU%*;J315D_1w&osT9g0z&
zO0`cyZin1E8^M7yVzQ@lRQ+K+>zE?^<uzUIcz}&4@G8~j1%ZsuUf_DF@|gYb*~@zF
zTam{Mh|g<`%DYi{p)yIP9n1am!eu}YVBx|%W)sL#KLCIB%#)46E`I=X@8_`%oGc9l
zpj+@wo>7K^03F^X{--DO=8L>onNTRoG9J&1lL2Ohb{)za!npclB!HMbc|+x<nE=jQ
z6f?Y(A%XMrM#yE8P`2?u^H_An=Zv`AOF1(8i#(?(nTe`{bMlz=@cFs`fR{ZFuQx&6
z3SirnycC(jLIB0*^2W(EivS!h6fb)zjR9?s0O&F1jhE3L2k>sWsMRV%!Zze(%4Mri
zHvVv4mfZ9y07njqX038aeNf(H8R1!!&DoKc1Hj{c9e_tZ%*&MlJAjg+JZ8_xt_A>g
zBl9dWunEAQx8+Th2{Zzz{4y_JuGs=0+nP5`u3;+pI-fUP2ATk@em`%93~c4n;JNL}
zn<+ytqUy}kd9!3-8-N8qc?B}i4B!u^^9p5PJAjV2#0YN;@9R8fhCJ@Kpsea}-dq4a
zH(LQ5ZWjx^mB#wEHUO1B<jKb7y@LSOZO(Jrmkt2qCg)YMHfN3kC>h9`FWbsa0I9F$
zRmr4}14z^6F)QQq`w4lG>_YZ6*P4K;Vi&{w%P=>%h<wh-aDCK+g0JNQ^#cMoW!U3E
z{qHhN2VtF;49}8v|2&P)dl3xim{hi`&togwxu1<2$YV=ew~ej3_=ddg@&IB0H0I{*
zkmHU^hcl=!Pmmjp0FWP)*8;$4|2sD1?P3)6n^04@C~vnvfH#@4c08Zg#%exX@PMiP
zGE>F}@S8HMWiSsXUH9StGKl&w<bsdLSVBHom}CUxl>swy>{UL*3$*pjB#gZiqD6%s
zXJ#nZ+aX#b0B<Ep0l}|zu3vf{&A$9t8KPs&MgcQyPKe&wjA82iP;Y$NnC-#puLdt%
zyp&|~@_sBk#-Cow)}Y3@F&*p7_QZo8-Pg>fT2iu^to2vr4gFPdzfKvF@MX46R^$L6
zd>)Vu-q4g8gpT5DJ$7)iG#J34t71Q}>+`b%8Fp4U0lWIK7;K0pI~Wf`^?t}N*JK+I
z{3lu9uy{d0b_|I5TF`vvC)u)9d5;bJu#_!ZmG>A=6<=n@Gm5flub-EV|1ll_W3ad)
zdnkzcF3`)P{;dHAq<Y`^6W9Ro%XdDxOqJlmcLsXiy?9Ctrjr)70c+DVl!W7*OB8CK
zM1JFA3H&fue$9^wddA~>e)t8K&9~!u$|mhuV)2Iu;@gJ&C)X_g7?W=g$dmtvtM7n|
z>e&9jclRnqdRq{Y6#+pyBBFp`L!_)Iq6;c3bzxmPB1m-Yy^u(_#$FR+>;*N}XlyZ9
zq9&%9<mLTd+Ixx5<RzLs<NtfkMa=u>^9kPX%$YN1&di)Sb7t<{Qes)Y4NdagYmsno
z78(D5d?T->4xz16JNgwfZ{(Q4C}O)8bP4w&(;`9ULWB73qO>U1DK)+5Zdx=08pNyJ
z(*{6@zOJ-%>>i%RcJKii@cvF|16dE`*c?8V7RzN|gyjZj$<cd2+7QZZBSTzoNQ-AC
zT7ZkPywzFiX?X;J&RUHbg3fl?N-BB;sZMXvrn`cX>u^xkw5Gdp4J6gg>FM07ZPX#l
z=BIlwck&2qb?M~DDlW>RoODkH<f1tKkgoW$<SD-Xc{(pe+;3z4DGzp$>_&HRJYR05
zfz{{dbY(^I1z47l-W!uw-l-+UZ9YCG3maEg9prS`1v#Bp1!b_DAA+`~RbJ@=^LHhK
zgbIE}RqxKp(2#_sjsUqM<?|pgT*zPv9+1XX1!j0N0qG>aBZF2(pEZ4`wy|H28Lvfc
ze?x7qxHBmvygMeFhXbD989inKJ-0y$yWPH-G;iH5f(N9CbCsekHGgC(^Ml;Cs}nPw
zsg^ccjN9MIbYb1F<7}5n4M^!pev*7E(+#i4Rji_pU1G^}XEIuUUy(BvP19_%x{}F_
zYNrP5ug+994Ycm=PRqoNbTqfOg<6nzVN70M$aRxi@C*sNcaN!nz;0h6!R-+Ygxi<%
z5J95Q1cC2Ca>->(1?Ga=mw1l19kY}g`2f$A_r@&6pylVoSH>)Ro_?fe%rcyb2@hQG
zrv@&`2#h2l!lI^`=so}?3$~6~4&&fH0?$<qV>UlmVcGpLZ)(XcMS35XG50Aw>0?YF
z=U>Ns0+!ZSMC5c-kNFgHO(?~4zrry;FqI3550-l{AE}zn1)vx=MzqDecHcu_*l3{M
zgi<6O1s**5jm65gP3O%a3I_pbZoRG-Bhdj!mUTV~as)b<DhIhs11vf{8c?t6V+B^%
z@Emzt*J+(nvG4p?=Z~O`M?c8mp@U-G$2;lB@$oo?&hY7{Q~Vy9H*0jdAnqBO%{j+)
z!89DU(`;UKQrDlr6`E0NpXfpeu!*}S>)b;b*^whWb4`LS0z5Q&`6qWN#Nm++Ka&9_
zG^T3*l!H8^K<}YC+5l<vG=QuQ>EejsN+iJMbh+F^>OHcIdx!Tc1;1JAlEKe&CE8o^
zST~HT5=k?&Kh_Ob5H5@hvrBX%6r=)r>tp%4hZH>6S;x*#yVxCx^PlP1`)L=gfNmYp
zjZq3%1Fc@CWB;dJv;peaD2HiK{Hji;kk})!{+e#Af~Y4}{#}=)AnJ*dEZsN-IRPDT
zme*)twq)G|PLxP9Z?s$MXoKZ>gH*9vr>89J(yI%K1zj$H=M7T3>v0`>JZ+?QD0gxv
z2aG1N?w$`=HyBz7A`GzB<%6RaS+?{8Is=jN#wL8dZki2Ud~N5dFW<A%6%%1R2q)Ug
zNuF5wWSx;&TNA6+Mb<jnH+j`iV;<`&I7<{(h$LPm__q6!bP%YjQy`SO_`9U5rBdV?
zBCC$GVD^X}$s1W#-XP7gAE~3m!k*bIf^N3_k*DO3;W3xWJRcr5UQ&SS%wsvyO9~4-
zMaE*`E!|S)ngD=iEn|=@q*&`{)8uU{FYuBA5h&f`g}wh#`LdTZtvpco8k1SaQ6zC*
znr=NNK+kMU0B>7}?VWL6NA8i21w8LA(rv=C)l-@P8}H~g11LMT1EX|X7|^~l##*-(
zz~Kj8MHcF|p#mR^yr!q*Yw4rg#nq9eTd+j;Is+2^jR!hqlSFpA@9(-dDW}a3VCToW
zJxt~gU~^LU7K0#w?H#(k3_=0UR_gXMpxJ>5d4ND0xqI?9-9Z*ER)z4Wyh)DoMxS2M
z9YLbcn!%vBkgGe&Wl3`%?A5h1NCuedsyohLBtW6F?gRsJYtB8<on(*!u-&YCn?V*p
zk7(T~1`_}lrt3~K&;vwA=*}>hqLQz9OM}D!S?q;<*8|-pB>PwzK~pqKcZF%@0<@Ru
zt}<8%kaj_LjloiYb!qb0Uf9_T)^%{&I;73Iue-ruBf#Dsx|<BP0$dH$-2(90vh(?X
z^I5|KuFDhmQJh}Iwa0+;QJmiHG~zGzchcR3vU(?hmHZ#wJ>tv$=2Xjg-G_{53NCm}
z_Yo_aEZ>I5y88?`)R3e5m;numD`wp%3}_{_%GQ0#Vv^O{@T2ZC&LJQ0WrOZ>2KE4(
zy6YYQIQ-y10-b#1r27i6x3O4LR$MFS{)TXY&jA61Cm-p)#-h>N$op%d0gp3RVhs`^
zsGz0YN%t)k{1T*|8+G4t)l^O&sQaG4c9Q-0Hr)?gHHo`pfc%k<6xd7F{lwI4yT|5R
z>mGr`*Po4m?A2Qenjq_bWePSVDch`dPl$rX$3$D*@07(B2ebc=PP$m!czWMj$9JxL
zkLcK*;2l*^wu=Jb@qY5Fy`>?cvQF&&qR1^*H=Rg+zBU;P9E25ZSy}-I(*Ds~8s+nv
zPIRIY+ij7Ez7W}8*=`4PO_rznN*c>_JVP+i1OGxY&e4e)La>DpZO$x+hA(9Z+KZi{
zdA|^CYKKnbutU3uw_b#4Q98V{&w{J=u_Y9-S_|zy>Q27CaD=oMCBcy@1M__)Ux7RA
z60|>qWOKSs?8_w21@Z4J0NS`^Af>gpPj8Tt%t2IDm(0O0NWI)ZO3{ZtdX`z%Yo%T=
z;9|rJn$hd+bRzlUeO?B;5Tj1yg_qe)zoiq|8E=<51s})-tw|x<)_{2&6+GQ3$Kd~w
zV@PKma9D2i(uu>cs(5?D!du+FN11zYZjh4>#SKUCKGy)&UdMNy7+`d^d@QNS-SZ<y
zGG~gqCwwEn0K5hO;-6>><fp~vhyQ>f6xrt|{PA-P#T3oQu`eai>C)j}kYa37o1dqQ
zRULi^@nibhsl}C4!4gziDXYa*?o>uduB)#V)Z%LRWqwPf<dVu_Lh$kYmRM6gZr`U^
zr_r9l?;s?*`GfpUAIa5{F9{~=G$;6Ce%I?LP?zQcw!#A}y?!fcw$1#~GmZRc>~df?
z+|%r6kCjie&AKej9#y++rP-F2m*xN}WusBMG|iE8+GdMDt=~Z>;$uEzofb@dazGv3
zn#S%`o1E<@g@rk#RX+D7>sF>!l2IeiQg%qIg7%!xqxj)9o@42bTHl}KeSXp?i*Bp~
z)lf>YvxvK6xw<4L0KXMud9VGM8`YZdLEKn!<^BDUwej#+H>5c~BI);^8S9R1i@!fD
zHMcUyQov3qjUi3-Z35MzRmoYjK@V`#)Rq{9EUH`yWK_=2vXN{0N*agXvPd&HBLoY}
zEV-?(G$1H0OW8+|DJ=OSiz*jNY1eahYL*)V8W_7><$w2u<uA)pwg%LUz$sZYPnB&^
zab;F7fB-ke1fn|v{Mp#E3A8=#mPK*EfI5`(TaiW43a8HmO?hpNkVVIU%8I_KILnW_
zi|ul3mf|FnT{g|lqAk3O(C&m2fUFSof50K6PwXb|_J>@hSrIf0+SyJnyD&G4CT_sb
zn$q$*Au9-m+CP|-m^vqm`qbf6GL(dz{tTueXZ`D0k#Nrgex6H<RX`n@r+#6=;|^%E
zVn7=>9;D5sS;}UaI9}hL6>B3vG!8C6oQVO5jg42d(2f#?tOYd%o2Jbag{%Z(^AD2c
zUREMM6Px0)EQ%&tJteoKzin0$<@~CpoXRZnulsc;uVT@gS;IIdnR4p0hI1KO^0$o7
zqG+SFJuUf`UEQ-N+}O`!8qv+lqHtrsz^%LvqiX_xWud&E<9^*4yLMp@#yz@lAmgMi
z9Lv~1Blb1xtgKXGe-c$~Je)NKS+UBH)qI?VZ+|0)hV0C&taJirW0X$GAaF5GQ&dqb
zWMvY#D(?xDe625MWdZpuk#7e|t`Yti-+^lSQ;0tShfrhvT<tgsH3pf$_R#EcyhuGD
zEndl$UHf4)_>EK6CJIOI9zISP^kff?pB-mMqpjTy;O)cX?7>DT^#oWyW1M0Sya3w%
z7^m0+ib=n|XB;nT?Y*eEDTfA(r>PfoNn@T-DU7FR1#SUq%8T*F2C;r%t&&gllcJI?
zu{y~J2i;?J`BM3kc=LFQfd@x{^{(deJoq_Em76!7&)Ik&ZON5`g3yw6<7p?@FVIdX
zBvaJQS7KZHPeKLVw1m{AdW=|?t>9V2^{Du-qcBPkmM`qq2NpFrlS_qCsfuoXxXJ;0
z9W)yj>)meRcIdU+x18h&0qp{uvPogV&mf><T3)stfMQ6ROXbgk;Ki=YrbwZ3gj2gT
zo3+zMepRe(wlnA0@UAJFEKBe++vmfkzavYA@3f-d>YcJlcm2nq9BNSlJ=BTKrP+#&
zq5aIIx!Gi6@BsmZqc=c0u>^~E{$ripn^-h~aQE<RK7C|WzB?eBPal=tRrzhFY%&M^
z?<rLU;km0p=Ypjn+Ivdu7&D{)|FMWR#Q%f2E_kN~Yjv6I)nDozRGK}2OOn@FDQ7DS
zHks1ad)dk<0_ot4w`}N-Gp#JSsXxww9%!nI>xFEZ!b+Wok7d&oRt|Fp9?hmH?C=Be
z1RLeRfYzsh_p)gQ`>Y{PcK4p_QCQ_eFt)IHJ)NCG;72Feum&M}G=V?0q+}XwAwO~p
zIQs4O{SeCQ91ooYp$?4I)Wkh&oU-Y>Aao`eqZgFZ0;Lz^2_e|HEtQ)?B;UTBDi?Qq
zXVV@ztPkkR|DCO85sALWET0LHhFI3nYB9?%J0CehDS6T9bF(Sj@3V&1j3L9br!jB_
zXm}&LfTm0v&Bn=r*@c|rp{cE|5VDIn$4f}XcIIhz1qNwY82yQEMu{Xmq`)l^+VKZN
zzYweBKVrOGyG^8#En(yz!KbD$@?Ey_z$OBh));SRLWlkfO`tP`u!WtP!AzL$HbFd&
zomyBull&8&%UhdpQO$+wd?3Fcic1zU9u@84tGhS}EbK07uyGB0Kt+yNgUg%Y-Niao
z8zys-fTZxhcNZJFk`6zH)?;t1%%P<^yd3S0{w2p+gXc=pV$XXyHt5dqIvN7Y=I7Xw
z651C69DOIp(gSN__{OE=nuYg+T;csXyEJ?OH^hRNGV3OMBh(UJi9Z~^#$KPO7eFoJ
zdn8A}MG@M#mER-f&gA51c4j!`JEUB^l04u=%C|@fl9Ta83kyyTndnO-`S~U%zaaVy
zDJe^nhrLMo6e(v)lZU@Z`2;EU-z1NCk@7K8(%O?pzDT)`C3ufpaticiZ4c$eJxNye
zLpJw{MRn*Z(t4~+Rt3E%a%qlrvZ}uWysACPbtN@IvMPipgph15sRRlIf_DM$;C=sO
zRT!Z*p6(GdQ08q`oV(B+amtsHRpIE)h&>R?#ow<t-LB$`Q-|I~wVDQtpP1MUY3;0p
z^Psw%Gtq-mk~rlwQkKo081y3L1X2Q*PYj|;6q=(**>qvzkr%uVBV~Hp#3QtEQEEGc
z+WM@TD0Y8AbdW}@Pa7hK<t*?sy04`&OzMI3$Xq<XyHMU1A$f+;X~2=u+6daHMUfYG
zb9w}?g=`mBZjRu!ke8UryX1dFNP$r|BiNcRAkU+@G(w3t`>?r<AZzaMLvJz_t9v>{
zcsYUVP&qMD3bpi%@PRI)Y@#vUA4d4nplZkS*}V~3dZuWv<K>7x^qhp}@TC#a9-yDq
zd8h1FAqHouw#<%Ix7#G?ZJ@WP!+M-^%-7QqQnV|c7w^~iK)^V94vFwarkmam5=Fz3
zScKPq)%OKQWI^y_KkNNL;e1`vR2P*C`T(fO=`%8y!)NFNF<PAS$&I|bNKY$)P)eS6
zYlmK03Y=+1MIFW*i8>4>x@C5H@}8rcj7VuA_9&;g3X-OzTF}E&z{D&i(>y6ku8xud
zji2ciH`<nXMSFr*G_9ts6MN~&U5I|7DVtF!=rxddKr^wNd0+3z;?q=*8Kw8)sZJ8V
zcTL~3FUWo+iJO1Xdk2DG4oQ4@g#0uL3viU4);*!ri?otu<FBW^=YVES6=Vy8^AvN5
zYG|m}hZA^@YJgHB5Tb~7r5fy3=_6TNRKs42o-f+8Qw<P=5BAtWnp3OiJt`j&ltk&3
z6|a}71iSQYdc_CUl4*&40&Y%sZ1j6bj$grN$PaoBneok_jhT85neqLF$$5GXnQ{Cy
z<f#4twVQ8Xl!oXJG9bNf|4RNnTIx4sE;#QwW}|Nh=a@MPXAX`%=gh$|=6qs>p5tMB
zL1xD>dGY|L>hF3EesN#zd|iK*E1(EV_m%o{4ESi}RsDGeyz?CJ2P#c=cG3@~N-u$U
z#YO!P45FB?Nw2d%*2gm-y@pKIComwr4n3hy#DI@^Vn<quS>Pz-3(*TYjdIM_u*U<M
z@kg=8F*j*<9rNV>OZ^sgpY*iW&qmz?Gr_dpte=Bt#d{d=OmAU8&QZmG<#92RSNIkE
zd=egJ4P!6nPcYEuyXaSgE|yMu&=+gCaU}f4OnF_5G(cP-zaN88f02G27o}A@ZK<Ba
zvTZaf^p3LqKq)Zzw0;wpb*7=_vB64zk=WKMY~TGW+bKj|pyy~IAM7=MtbdOQIG(p5
z$}N{>LhKEc?U*gUHBj<0HRSTJ=dn<DBDW`zvvE48%~iraq}D?tayjVpfOmI`zaE*}
z7mNnwBg?-mS2^0Hww*QRDo4B2G4q>qm7`tauza~39t(H+t6bj3^MKCp$fZ+~LHT6(
z3jUcJ%9SSrY`vWuMmd!u*>i2DM>f0GU3hjGB;~EG;pWI(a%tj7g6Lmz*`r}=+qe%u
z=KZ`ujYo2eY~(w!QeVrRTqUqiUe8h~w~T2#HH9-uh1_z0xNaoT{_D9W29eaOgJ!!B
zCuF*_B$99jgXj9O<mB8HRCy^?{xEkX^CC&A@8qsxK)OIJs~K3MmPx3F#zR~;>St8b
z!az$q)FCP-A$K1!oR7R&ALP<hjhm14u(S5H)9?_Xenz9AxUE+nx$uK&kfBH4AB3oK
zLmqb`_vrqIc}gde4Ez7cQzCcN9|yYUDRDaz;h<LT87KAg4oJxBPwb8$&A%*9Il@x<
zRh|+jX+?`XI}WF5<Fz2To{^{6CfZx>Tb!rZrWk<tuH`AFDUO8PKfo!Ert#qaAZzRG
zydkUy;=1EXUOWSmuFXF$fdNT3aCcrJ10HYdHc5Hpu@52pXuOkWQ4Y2EglFr!+Prki
zTa3KR-gy~a+-)!BWp;5lFU-^ZmHSwQJIYLYH7|<+G2fq-ryNqz$XMr^H=c87cWT^|
zm(2is&=9!PZQ48<b3^WFis~D20}pvVxQ>j)x-L%{bJUJ4cJjxAC9V5P4lNb!wazQ2
zLUhToEJt=2BK5SHr_HNET70w|HAD&sKzy>Zr{jn7IODxq&09`+4Meq2DcL+kig9@0
zlt*SM{<4yGW(bV!{JeQAWv3F^Dk&kEmp-zZObcX!AIOJfPj)4`P7^+vf_%z;=w0{8
z?hI%@bn5D44+3qp$2xd*G9Op7A)jJDnfKbIY+0_>Nt0;-Ot7SZaKG1Nn!L(ULDS>O
z;SBhU%XHOg3Y~^0tR#kQQ<ytX)qz{5Fn7iF)K6jVe0<tqJ4IQU?PwY%q+v)cvUQpg
zKy2<PY!ovC?fVER!BhHCeaxptJB1_0%x7wn{O@?lqm1Ut#)ngq;X5UI@Gz{JGK`$k
z_PNM;?aY+n3`o=Y9aBaSS>V#=osy`;VY+TUB`R^4nKaN}{i9_{3ZxoZh@!i5<f;V8
z%i+Y7G0cuSq;!VdmLMgIedLc4umJ3ldn96i_qiOMC{3^wPIz9&&|2<z2QmllMNY!h
zF~oE=7|i-$Y8o0x<^c719C4ajNm;j$wRYRoDsDHmXwZjKtGkr2RGM1DfJ9ime`+lj
zOZZ~M!TGrL)EUI#vC6_M8oF5Y{=ca+6&9rRp|@yI-hm_e{NgMnJ`Khywbn+YSa*^=
zxseO+Bgr((M*NVyQ6#BN)u?FY&)Q&GdXi>oX4Dl5!5Wl^I!vOjDh}qZw(u5o!Lwj5
zcOs)HH^eHqC+-g<S@`DB&|6s0ht`7GjlI2seTdlynwl~pxHpo9-XdP-5`%rI^as>Z
z>v6$afM`!LaHypZwT7<^zv~-JKCDpMlYE4st3z?v7-tFAdy~V!&hR?0+||d^mO>)Q
z&MdO6SC_~;7ufcMcbx3(M&%P@L0oe*)Rsg_uF;ef3$`Ru0=`j7QrMD6(Vk>oP*BSS
zX>|#^VcUm$LJLs4)MD#L0w(!u>T!YGwl4}L*91`o2`@uHo2$0DFXUSa{;R(C$In%P
z>LZq~hewxu8)AECsG5(+i0xd^!;`>gq|y3MdysmQzajyKMfKo?B8eSHOKdIbioJn9
zgJy$$VfB<A)S_WAq~spdqCv9-6g!vL!xvuju$Yl7X%Am5fmGy#-RPm^7G(ffkL%Hg
z8=ggNh;v69M!nO+pN0kP!_+1-Qu_S2M@46=EQks;Vk5RL&&m-kiJrq9QIC{S6;@6!
zQfM@Fi?+6+aL=%IR5aPuDi+U*Da-G$8pME{vuWR1#Q{(l4TUDusjbLj5BmmLdwr}5
zltP+D!Kx6C^0Q=|js3<muZZ?`x9hClfV9KD!E;A<s}p#3_%y}Z>ObhF;mJzHzhXD0
zRBY91^&5lEit#QcS8TGj`UShH;gb>U81ASiC0YHBov`qLWYT$BTm6j6hgV^rGaN_u
z6fv}vT0O-+VfbX$KpDu+znmOKrvk$-qS}pz!(JRo+zt(M=PgVma<Z$!+;Qb6dXD_*
zFv;6B6iHLXu+D2P0B`*&+YFZiv+BZ>9T<g%-VO>=c3{}h36&mU6q^sL=lkZ9uiA%&
z1CFRf*2)jVB4~WFt)AR1ERq4O+iz}{EyJbWE}JO;8{QuS;CWZb2Zl>MBkzQVA#Fru
zO0RI@Fp_R?`7a5N>>_L(5+22XRCjHv{A9SK9c?dZii!(`@PWt)tEVZPkQ5#ZFwzkp
z3gbG2k-nYB1F4{y7(NKOBle{94j)X_G@uAVxYI^RUb>EO+8{Vwr&UbO4NnBXK@&jZ
zrSPG|vw<&pLT+K?Vjcm`0(jm@2~VQjkQ8<JFycwW?Z6)S$_S}X@GB{H;VDQQMRQ`9
zHheTO<>fTLBRrJ>wWp+)>^u^OX1MVQj`L21)9Vi-FPM6VkHxSUm4XqGmKUCd4faT1
zJnvZ-K8~Ij<9VVkd_0~<33yIbg=bTGyO81+PF~HZB0d2Rr`t*+HVYU&qwKhf37O$J
zsA5zKG6v2}4X0S)sQVP=+ITozk3{8QprJlImv^Q#a3Db*k0Ns327MTwr=?L82jIId
zJTI6)0>F^8@Vo&8k~#1mJ{fWd?IUR*ufF7DcL6n}^px#JNuCzGt%pc%tR3BN5Zb$Q
zD8ueD2>ud_PU-K7%m_X;pWegT?i!UJ#jNdU*Mj@+#M*1VwOt2g%~Dw2V3wry{0_UD
z3}_gxzGZg{_9kT(`A_&FR)0J%?;Ib6SVyM~9OT6lY65u1?6w`4Mm#(422!0cFdZYR
zFMfL-ds6Md3}h&WlGhguq|hu5<7mjGti%7Gv&_~sPwV#&)KiTe#P;CtNLf@o@B&AS
zqB)5IjT2AU*}n!Jqu&Da)4$oTXMR+Z?J#@#l#1eY;V5uIij-_|wcpNZv?I2fZNGy7
zt!Ab;`<)CZqEzyQz48JAjp4IX?O*4d9&}WF<v-T;BAL0-AHvGW?)D;irkJ)swM*?q
z#Wj_LV7x*y)LtD+_L!n+YEv1SXmL(O$tT=K6|MWDzXRc^Vte(^2#1V*Ld{yZ5zPuI
z^311GiPRKG5UJ0%r%Lz$XxaRH2L>ed-9`D11aM>s?x~0=rjA#x-p)+V*I@jmrbI)o
z1^Jy<h~p5kOiqrb+i|JNVOBIpNgm-27-CA%QIO2(Ga9n1ildb%2@P4?fqieZ)N^2M
z@_<AX9Ya-bJUn111jT6Qq63otQglEv3)NZFD^>E#%ZP+k`a!_oWxt5{JLD4;{`P#R
z#^3mf68<*7-wl5^|JfaXvz@H)H~*-$kkTuJb{u15RCzrxgx-HrY~-1rLv&6!H|j3G
znkq&0`y29)y%I8^D}Vo`kQ@ezp$nM^;;?%8U#XJ6<q6N1kjY3I^F-PC!8E1WaKE?<
zCT&PMj7QoaJqBTO$aqZmF&TJvSrU><&vKm+ao3P3^!zuPLJ`SvA+!Ta`-RW?LWba3
zu~oa8L*g0Gd~N(9gpNi9xn+zrdi=eRk*p<BX2{neqZp7OZY~WOO`wfj8so)~R0ed=
zcX~j`7$&2)*+|?Jr;s8@nobS@Iw?3!3LLa6#E4m=#G6`&gp>g&M=Z<chm=#!18Z`8
zy;n=tAvBQF=c91#Qh9Njq|KeryQ8#Uko*d3BZ=ClI+Ywkd#LmuLAvp92>A$#nTyB>
zslk$%{-ZKxLTXt&>iA=4L+V&?GINmaSsHF54no1BQgRe6i5bJ0SyZsXj7L!vnXKmn
zFiP1TE2Aiiwm(8?9GkDGFb0E#wTKXGt%{0-QLuW-=Y|J=0Xx4(o>2qg6=l*2yuu@D
z5Y0lY9r%GT$;sijsKJa?$TV+|+ta0F?}(@*o~hIsJ`19f32>08Ekt(CkiwHEvc<s-
z24T@Hw<sOz&%CBEAKS?s^E4qUive}^%j&3c3_7c^WWfAHL`|YJ-Y&I;MCl1|HO<jc
zxeUlnFkg(yV=}&_VOtm(H5DUOw+dOuUy8!cf*PY4KBG>EGBBX}UivL+8Uv-ir~(GG
zBA?HSDr6wh7`j;(Rm6xwkniWnS((_?MnsiTTD!F-BS$Ah83C;AX)M+@N0o7on`Xqw
z;X+h7Kv+Gwx~)G%nK&m1V8we;(;0-)Dtu{Olo?)U<~3+*onReB-(uCh1G3ZoqLiHy
z+2^HGqm<Di2^JIzERL!}A>C^S(5IIFYQ;L-ET@Yv&7}sh9vpDzEV-~q!0(L#jCB!{
zE2@ft8aT@egs&Vmh%+hAPub!(&?67xZtT%S07rLXs^m?(NO2a@Uj4*QEUpHtX3P#<
zhg_$VhK8CA;sP=sgTl#7kBwfB0%M~;#ox<VVyq3~Vhn+C4XCzyp8UB^3Z6dEAg;og
z6C&s?nA;<RxQ3AvkYR~IZ0U}?BsZXy>kZ<|K;s%bf!<Csh_5iEKjrN(h-)ctaS)-+
z1`#1oVO(2s7QKRI5MLwox?D3>a!=G4#7%%gl6nGi&0ibD%?i{5y7n}PTbQkS3gDX8
z4dM>Kaqa49fYW}IFOHQ$eOnFUn+nlNq+Z!&5cest1+elB*&z$B)CG!1sO{>l2hn=z
z9q}lmTZESSiN^>@0u`TU5KoYNm!(KR_1_HQDXQ|r?v%S?s6jl<%)3i~668BWx&P>{
z^cK!L$0@#{a#xaY8}*hYLg_wAc%8eiio26=JLBx<-8=3wW979d%KwmYU041;7%LV(
z0{Oo(o=pq;$Y1>p;``{-@e82&836{d!wGxnS%L*{mShlbFr-s9@s>fn$q3gXgjtUb
z;w?th@vlS~#M_Lh<M%Ezh#xSbj!&3s5brRej-Q90%ec#kF4w>BVi4~!(vmd&oKfR_
zNR#bl!F{{Fu?F#T9Nb0ClIZ&4B42~}K!FmV-6@0k1w*wppvMq{_$5PC55ODf<gXZt
z_JCKK4dUOFTnE5YzZ=A_m9kENg$9H8cZRmEfX8ACqUHaTY#)-VG7EBjOLD!8Y}Imu
z_#L)VQM2qwtQlzSzZBMXfJK80;*ShPs>vGD^Cty50tQ3qhYDw^SbE7I{+nSps(8i*
zgZPM{wZ)SvPXDCQApQpx=Zu8527h1>X*-DS0ZRMDAU^h>o}=DTeQOY(a31x}7EG_-
z8By<C4>O2=FrwZ$gtq^e5%rGyb%XdPBkG+umKwxo%+o^Mlb>Qx37n|~l7BF$L`F1c
zXSW&fv)?2P&Dp`{3@SCFP@tPO230pkkwB|M465!hirHTH;UJ-3vq5FWd2v7yH3k(v
z_605pL|JH1*)U3`@wdL*pt5C1t-pQIpz6UW6{ycU29+J744~dO3@Up@SwItg4Jt=Q
z6M)?M7*tMNy&h<DPlL*t^QHj#nhh!!M$>?LeqvC$GAg#$<8VCQpu(*;$X15TVSgG_
z?u^Vp`z{+)9*nAinsysh8b<X3+1O2cvQ)6@6X|V|mW2jYFSw~N<Ulh5<P{Sn@32J%
zl`k3E1~s~U93<4Tk&v{2x276YeHgm>10INz?@W;TJ8m$j0#Jw0KzDEp3-NQ8(ttkR
z%6&A<>WLvds~u?Aczq@rROBs8td*zbNa0>;gDQ&4swvy@t&>4Tx85e6l;6m~tu}ju
zYLE}Hps`~R4XQXsc0_#pEjxp10uet#W+gb~a7(E*zE=z?@-Q($fy#mms!5Fadh3Q0
z)&^B63Qp>eOsr=rBLHr)@Oq{y;~ZYkEP_F0;=InwEhs~8oC&ajkf8dZ)}X2;@)9tL
zIBZbWAoMix5lMj6Pel<XTKzzp{C}vEBHEK$lFO<Kz?YUrWxm{4Z%{QV!alc*leTg#
zVM^S!ABr91gctZcV=LjT2#p@c`mNW=k0wg>1)qrx_~L^>b%8sI>K-!yB<-kMUj#pO
z(s>L6425^DiIE1?Wu)FuHrArR6%?4%A|w|PxXNJeX0eXYHK1|0s+|RQ^Mrh5k`$Vb
z)DG;>^|*!1>@Jq6^aj-}ysb7?VRsv=_oS`DW|8@Qz(@pShIa@aRCO{Os>kU*7=FYI
zRTF5pnPcbBAFi}=_2J-sOp}fQT1OhoxwAB7f+Gb{HG1rukP+kP1>2Nc0HUJ~$xbx<
zbGLxVi6ZQ|Ta@+X0Unjh{#I1zsH2l76<Ul9nZvYcHWm-+4rJ7PJ8hN&wg5k3YS0cY
z_v=p0AA-_S$M<$YdhQ)^Klj})L`r^dSH9i0?^Y<rx7ODIjR>#g{P!WyKx2z9ji@|&
zi~Nq-w;c6^_alcZ?1rxiTNda1mE_Y~YQO1KU`>SUkyq1NV8h-z5iXlwV2cdpZP#Hx
z7tn<YtEc2j=64j(t~75w?Vm$?7Vx%|W4=pXE#PfwTTdQRr6AhF?O-3)rNgs;0&J5D
zl)7*ng4#INwR?~JFc<I48VdM$qK!Q81G@@1X3a6Aaft<9EDiaic@N~MJUFav1$@%R
zTO|Fm0zPSLBlq_1qJrK`?FO*^tpd7|tGt>I*|pq64{HA2>DmH1Vx5u>I<)@95$o=p
z0y_G@21-NMYYGAxkhi?Hq=1h;+Pcfv@+ABeo{Zmk3v{?xplnB|`s%OcUX$^u46gz{
zx@)62@JhP^KDuk`O<~X8!-mrRb0G=OuMZsB6ZVClx4$;DCt@l>(m8ay`zu4mJ>AaZ
zNj+?+_!h)mj2V$f8}7p0ZiUosQz*<@(^|;g#+tl2y-?{kNhN<W8Qa{A)Jyptg$^K3
z>B%w2fcKo7r(?KI4MtYad(LJI2)`xN=~ceY6{H?M*J*g-T*-iX<JM5;Dh75mjsxCv
zuEsc?@(o7c=C7RTC}rws9(^>0rv@Y0TkTwjCR#cJ(Zxt|JL&v{va6Z$T|D|NL2{D4
z^Y5VSB%BtE>F)0=UVo7^1xW#uoyGT`C#m+Ghl{y%uCsWHlaT+x|2d1dU*w-g(#&Jd
z;s<?Sq>)>ib;DV_!%1gFYiIE(CVReC_MM8+wDB-Bz45S0K*YplE19s7n|e8me?V{f
z6|k*$7deanwIMkv5IFT;WaF1NJF8TjMvg^oma|HY$?NtdITm{lI;$jDX2(z6C=irk
z@2u*MS?c7%tv`whO>br9x9|k%BUlCk%mrcpb<V0o@Tc-yh;U&e1c}eGc2?8n!u&gm
zAnIv~AV5K8XLSJ~Tml8p{N4CpF&?hMjNzju3eM^x*u8u*9_n(^%PWJN)y2=%qAovy
zq-mcxTht{l(vBkSP@J>6goNXhgz|&9treW@ER7=j$d|nK9jt((^@`|LWCPI*Ut*Nh
zv=-U2EHs#IPA}>~pp7DjV^fRlAcnZHJAfgxh>b=&ZFKwYEOJ1OMchbpXPiwD2eaDf
zO7i@`A}8iRGiY-_ku!6kx3jCFi(D840^I(#$dy4jKz>J&8;cYB+y)sQvWh>afJaRa
zQ7WRt{K8bwPEr-|p=TS!?Jq36Uepsgip7gMSLDrrEZ%z#MT(`NmTdk}&NoP5RvXC_
zy<#oDVvq)S&n;3cgg4UCj}|GRf07z65q@dFHQ6EZ@1S#dtBAsGLTO)Q-TFw5pC%;-
z^(dla7NL}8<2xZm$~jAafcNr>lyesP;=<Nv@{VaZ$Q0%G@Q=gJqGT>Kgc#kD{{*8n
zsVIfhl8KRj(P#$40Lrcxar~&AqAgI+7~~YDraV_rp`I0VrVE$JOADmlAsw1(p*WGu
zO$YU-&ynJx7-0w|18g2t%wGaYBI97nm2Vdy{{9sOtHN#+)7?ksd_K4n8k$iA+pxl~
z;wh2NsR}>XfT?!fPAX1={VO8p((rvT-I^~Nhv#hvis{mP(G=>i!|~1wi)rW=(FN+h
zPQ^6xi}+QqnkB_F@QXegiJYUOiwh@V))alSa6D@0UR(rGi+&RqQiVnK^r?Xj>*;YK
z+7r>USMgZlc|}uPBoy<_gd}R>qM+jO)ZTwk69e0dvl+;57fJ~!AF+}Y2NUfn`i8M1
zE3E-oIn{c`I0Cmutf!H#lq)3Z3+#$Fpu*yKIiX0378lD^MN+zjUW1j^7r%v8l*9`;
zRWs^^;=Keix+kL*P9uu<5t?d8s9R|90qzS&LX~pyL7?Jz$B{tcCB=tecuMLV^*}j~
ziVqW7;J6gXX=L#cD5hks<6|Hw`X~f1dDH1}j-|X(C_Y9VQa{NWc^e)VA4h#c(j+a=
zg4c?VgA0&9w+U@7$>27LYq}DqDhZaZggVxWivK22YB8E!qTJi%4?z_bkSv^!*WN)u
zuMw8K$qDKg#fo6sN=_=NT~yaP^I^z8&=;$&dk{^JS>;BdSWTA<OHL9$67*-Lv=Tr$
zbAPeAFLC=C`ciFhE3t<}rDLJ`;5%g{bV<DQW$OEGp(T#2U+Vh>ua&Un;o-gEu{^B=
zcidk_hIe?0GJ~mz)ryixFjGz!&MhlZzDGf^xA(`F;EoA$)HMD|$F^ept-64Oh*B%a
zRQh%#Vo38T5?<0kk?>vnRkW_Y`=r#KtZ*sOZ#u&CMxuWRnz3~QsX$#S6{wp`q4T;*
zzTQ5#$}WR!urUc$XVqu04d!FAEAM5n4d#o3OGahbQUCJA!Tb(6s}#}L8TA=-v#0zF
ziq^DdWU~89jq4xV2}mFKGz91HC$;+g+zIHI<h3gdaO4EA>stoa_GAd=2P$NI3(Yr;
zf%4){;VYdvS`|jv9qSAoiPi3pLMz^ue=miZOPBi?ak1m)j0r3;#eXl|ms5?>fPr=y
zxlB%{WjjY?<S|GF=r=f{^Rx`0Fgs%k=a8YdsN{oS7vz}1@20m;K-yb9GI$j&)dQ3S
zWbh(dnhP*xul#RNi;Z%RG87Ura5M$2(LKCDt{Hr(4cj+Z(r+`$D2G3>bb6SaQ3k^t
zlVN7+G7yxe%X7=5yx7u=dKRUcoX(zg{uwi&aTAFQjW_*O;|fqTE<2UOExE!3UFs!*
zF7+zZ6o`d-33RDf$<b1M6Yu+H%%%EnDfP|&Ykdmn>|qAadsqQB78F7k6Y7$IK$i?m
z_q>Y<bjhIPXu;sHPsRc$z;sKoB<*<oq6~`Ti-qAg{WIup-}EX)zEyvfZ<U2nr+}RG
zTtEeM4nYE4@)78gkIA0PN1#hSC8txq)sSy`l_DQDZPVv;$`^vAZhLY@8wsd5+ZpS)
zF9k@8De@Wvl5ca4eA^^>4ceTsk<(}jG?!*<Vn7>_!;Kl60cf?2#U8dkIzyy|-@F;d
z2_Hh~GaVle$dMEA4?0~Oq_hZmm{g3=Bp}QxMnXG9sL=0>&?q3xtIj+v&}?;v7)RHl
z+WdhwuFVh!^ChW3pgo&2M0%lCc+j7O$@k945QjsUidD$0v9xB0BT>I{qKQ^^B64|Q
z2^3UH(Gi$}PGkdwfx-wq0Yq{Wmltv?JfF)==!M)$o+7s;L!|eB6v6SG8IfKD652@c
zLBC{(^d=B~I{+x}n+!3PP<tfM`L{Dfx(9}L2!W`<X`Gh~L=8^oawBOup*M&sHt{0C
zN8<?(rA)erQ}F>h>aMs*{?IH1jyV0yGt(cbl`kRn{ll37{#1&3ex^B7armg`uk_1Q
z96s6~&UMZVqAqVENB5m0a#96Fl*l+P|9d8#Upf3he(tN2GnEY^jf^9GGU*P8vPIpM
zmZ_{T9>}Q}pQ)@c8k)*y?7cH%NS<fNf;<De<Y~E@srZc~PwBQy#cw2eu7qaB$qy=U
z?dD!49Z81OlgE~PKQo?dB;#>nY$hK`meM<cTMlK?@h;ulqD5%dB0NVJhhq_{8ip08
zbuX<!w{O!L<nWZ%pdN{8{55?{$G1CD>G-zULdUn$UgHm^<mGr8$=9l~>;d=ni;d&p
z94faK^E-@mtSay&+N#%$*;ov!4p0k13yc#ehtJ3s+ZpMyh4S&LnK8zROh(fy`iXH8
z=eW`NOw}+fwN*zy#}6(Do`zn=Jk(kps#J<&TFZ-Cr~b9pd;sVC&RPx3jx>Z?r!gQm
ze~^u_fC1GC5em5sbsa<~;v7%vpyB`OWh|v?E0GltV&r2p+Fiiw9&V&_Go`UZ3`XVL
zjH>m0YNT_sz+d?bC@q!N#!6yA3o_JI&4m;e^3f_>?*7<VkF@GD%8^Ma9^WqmV>YHO
zb0i}zE;V!-2mzY@Qrd_BeP{hmXN=7NHTa}4c}(C@<J85Y^>i;IU3slptVl5TuRP}2
zVa#>)*nE`k>M{CRq31m&&#sod;$w`9nV?fqOPIiu`l^OP(5+uu8<$f>=M`aB{-q-M
zcD3ZkAEJT|*8U~K*vf#MEDV`f|0+Nm^QX?+JJ={Q;M=S<=eLOU<$|?wJ@)8@N~1RZ
zg{7QZBL!HB|1fSvVr^lUUTbkDkMxStRrrDV9mw3JNV@AUrH!vMAc4B=GtzCg+Ff00
ztv%YM*4|sR#yx0l9qlMFy7p2rj@iV_HSV*MpVmlDmS0Kw>009f&ZG@w;5_3&fI7AI
zc~{lBbn)Erf40UCiRWldeSuLheniE1yj?qJ<Znc^lU(oJGJebq`5tjd4_o7BROlt8
z(C04-i3jCXweWNg@x3SG*NEEJxzN~yynkm7WXLdJ{|_MWSN>i|(W||U-x9lzl|tW9
zq0v+bbl)=_zs6dh<zf66=<2-{x*xlk-|;v8)Wv+=9^*q9F9vEYBNXwTQWm$VxzzY4
z^HT}<Eo(vdiWQFhVVwA*T=lDAbFBX$>$LDIvQB4?l69(ZA?q}{J6R{~7_v@Hv&cHF
z?M9bNj(^N?^0p)XMv=A!^;eSVlP5;e!CM$kt-#8$Y`#%+RKx>nyI>TZwA2bdS~$JO
z(J0aqF+=Dq8K5+u3(Ai5Sj<;F?QInM6V2dGnvlQH7{$;o=Jw-^Vi>h>JTt$&qNh<L
z^FO1uvtZQ!6pZdFczeB39026Ck_*PUyBWoosNh@4WzO87*^0ImvEHu7XFEpO8O8ZT
z^I<3R1zjXlij3kyUc_w4J9zV&hf!Pt(gv|pA4@rtw6l7@QCx<sz+Y8F+A&pY<gZ{f
zgm!XWrHGtdi|<|-#np(@%=nPF#tXBz*cio^iESMiNjHq*E5w#>QO%oV6xVVk<a78Q
zGK#Gr#H-jqx0cH-Gq9dr?X2OoF1e-mjN*FcLH@_hZ;avwMh-x;cN)cwplk@G`vh9a
zC~jg|yh-d~BfO2`R@5-lr<3WnE~Xgz;`aYxx`UDBIoF+CTruE9x-&M@r|XjgGegU0
zj8Y}9`u8%5yD58KXC-fTRYH>Q`JYNCC?K>yuVZgl9hW{giu=Gskasn}U_4jCq5olZ
z_<vX(`5#tC6;>qCs1EejaWHEP>6GZi|FAjvKWyIqA2z3$&16z$%PFIHnltHyuV$A~
z#M*?;@})XX?!(|YPjx&W#P4*~G5r;zc%iF~y&H|<#jZMb_mbDlL@%H0l<X>%;t5$0
zU=*)$1=P#LgDx@(?LC0PFp92IWha%u3|6+rt7XXL9`G*iYZPxFyXkp*Z+6K-J#ec_
z78*OZ3B`AH!3W%6YH#gQqj-l&slAH>jpAMKw0g?z4fzj#Zs8-UqgbBSC<R%HyS{(j
zRM#a*!$%l^pA!{%|HSYCGo&fC7#8gdU&`WTq$vcC`?^X_e^+G`|4k+DgLd$7qxgtQ
zwo%E~BaPzEoYnOe*`}V~bQPVr3zfpl5N0W@koz=A?g1`JE&Gt%r;Bv#@76|@ib(TF
zB0G6<lhij49#yBiFsr<a^!QN8sG`G&S*;+&+NJ6|jR3;prLyI$t`%|Cj;@m9Ls8Nl
zW^2|B`F@ks$EUfMQAK9BITK}TT8%0PfUtUM+O_Sn$1EvWTrMZif=x{`s&IUSPN&Vp
z%p{}A6OqK`Oun@|3k!Ag)XrO}7=hgufo&gR+XmvkV~ncK+uA@Ru5#+bcIK&3MQ_%T
zD~5ve-bZc_75t&IVE@ief)=P`2tz7~Xv#ll;hnc%?SvFJ7eA~{FU4X7+_=`oAF(%O
zncuV7r5}Lvd2;%~kGu5e99r;i?sf^|9P%~&-*t&(;3SN~rgo@H6i!;2f8bMtjiDem
zy0jv{^lc94??uW+lgq2nB2FzW6yx8GB)<tRZG9mGUGP_z?}TIAR9AufXsKeg%SN>5
zrM<ieyKKUsp4CVB_@B#W&gkmnm(ENB(+JT>p6v!3JC|KBn6pX5)p;(jQ`31>KCsT^
z4FVKZvIrkIy6k2|UQW+@E^oqFcl(lBoTGQy11V=CNCpS+r8D$a=<I&+oqzZX2hQgy
zS|-#We>T1NKie6Ko=t^PI^%hi6N~n3uyJ_{YMGrvYFX$RjQcMMF6wtIpzS=b*oyNC
z^%fVka#xtO_ZpHepLOZBnA`?_2YuhV23WyHm+n`oU@6);=^t*cbjxA(X-%2nN-t@}
zVVjzYjBi{$sKy<I2&sM*=;X>QTM7LukbX4=N3$)V;1H0cEz5=`=Om-io9~rPAl7_q
z_thC?ISi=JuX)LXW=mlSxY>e2@z++CO`$>*q&c>?Y%0;QbGUL%8DI9}BRSp2WqjF>
z_eGV*<gMUfxkY=pu>;HK!k+SaRbgnEa$#>Fa%yjs@y4Tl>GR1u=S$Wm_v?)7bIZ)A
z*s>F(khY>r+K{%g$}VXkdKClO03F;^#=8>!ChWBYud*7hS&OXZ(lYuzR>bo7&4MyI
z0t~AULQYA38Esph-`*^n0bp$}FP(!A7s_RgoHYV;;>%^cX(=5EuySu1ZCWflM?II_
zLeXm>VhAZs*WIpcHp%`SNd4xP&FM|g<arJaEW>xR1<Q{#@P_|fHkSY_BgWL6Wcu?Z
zjjq3g$NF2zWdRGoWbSB?ZaiGJ5Q)mQ)>Xx2i+G<Gi5y&BO}D@)9UVtZEJtZ?ym~Q-
z62BD(_wyp*O%A<C-e>kx@;+T+$opJ&fV|JQ+522!L*8d<HhG_WZjkpmKEoP+-!HYn
z-+kq__-onN1AqH%vBTf<^X&0AbcqB07S4CX-;$?Jeeut$&d&HdUgLtl1DCnt@22@~
z`1|H4cl>Q1BHPcEf{i<U0qkyS@%OFeeehSa#1DVN{!J`;E%&Fxm4$htHlDTu^C+~v
z&=CJQ4k+inEaa3l)C=(s0OE&>!g%Oqb^I4V_;8*RQ2MF(FBRmLBX63EMf$qPmN$Ml
zif`i%;)CXmQ&sW*N9?bGeWX7AA9z--MAt5j|Av7`9ohP8M*R1H^LO%FR`EZ8yYk{%
zWOMwFTor90O>^UaVn73aSwj3n2G}H7goM`ke-kRDL3sFJ{3C#Q+xh^k>lObq13Gfs
z5F7t1EZ+Q`LG%rQ_}}RH%}`Bstq}j1p7%%MQ(Z#*6M8-pgP&;-;(w>-6LBc)_DTFx
z0%s`j9MvoSPXZTcmRo$I;-4`ZiM3!{ZoDWEx{^v`C2lSL4ECTuNAPEe6FvQhp6=n#
zyh!|sUr2Xjj!zI1;zih)1+m~VG$dY>NbpD@3o~V0yhx|V(Vn6fuq--Wv;stjgaTT&
zJ&hNw6&$%)q&KYNMH`6VoG->8zpW!)w59AMF^(pZ+n1<BF{zGpMB=H57imMjAQqYQ
z0^)+A&Q65AU66eQ!d*AUi=N0`P?Y=(|ACDDLk3MZ`QUtMKw4nD*as9zbo1<aj)3L}
z5~_H=Tf8VkqL>Atn5$xF8$YwvoSTZa&n)-CS-H?a75+yKTOjpvFD&l`P77Ti*z`ub
z@?Zi}<opFvu%)S-j(QjNN7kOlPUW$b)q<4ibIRkeHZGh(QmK2DQ^0d!3xB0xVRuxs
z=Sg`2Bw9%Cd%b_aGmg%C7t(veHLY>pv~01-J)R%O$GvQFbNu6cIERe(^^I}#n-3Nx
z7Ih#vPWcuS8AW4e9N#Qy=a-O2Jn)R83nYu8krk<r3nUhuuWrQg{Sp4g(cMLH94u_-
z3o@(ntmDF{kdb%3ap7GOEt?+~!9dbf6jTeAxJbORxyZ;%^}?rkY$yYNjsWeqvWue^
z@)liEm<;|alOYNd$aM8lTs#4erswP85*Sc4J^x7@y}PyO5`WK%0;P-Yc5!ze>l8QK
z+5;}3jlzD!U)ak%7fF7uLzRs5zh;a@hNTY7(U9z}jI*I}e7eJW?w$~rgPg^6UH-@7
z`AW6#V3frljLU^}U)(}7YVlAwd*1GG+cBgUZ$rYO=o$>_yKy`0>EGYtcKE=1n>$+@
zCysz7mh{7P8r|AIPE_8tAb0ZS^f*y@*@C>cJ@~D_6c&qO+NSU2o{Oa@XXVc6s8ybE
zA_Xv)ESIM&#@qRe<k`rzEcfJ@1obR=O_39I#$MYPC+dm#M=-hrwz<$1Ze9V||1D0;
z!x&%k<4T~g8*$=fUR_#%mZ6R*ECBh?&80vz6WiBObC>i3XQ#8h;>54PdFez@++E}l
zC;nYgB;aeG$?;3z6-UR3-%#QMJN_t8ocJx1xPioSDNg*35uZMcl)&y0v7^1Wr6x}N
zxeo!pwYKDEJSs=F?D|B1Xv!yh(2xq{EH)W%I4=ys^V0sNX*6jg@O=8BsQ|AKWAVb&
zS~S?nL<{p$`WQjuA`@M6TgpGQV%yU+mjNxzHWy9v7|_(p?=a10z!!`*^)@Ypp;&qZ
z-0~cpOcZ@sHbPT5qe3t(>*BU-zG?YixLHgqy4bnCW?K1zU5M7SdH^_xCMVMl7=UFp
zdRWCrrd@qWNIW0<!Ss56ssbN$Hg&+`TlOksiW+CSi7}>#zSGTgi@+L5Na=3+fN1^&
znxLnqyHLo|1p>X@y)w&mj}gUR=lyK@kO9TuTz@is#DE-QpA)A03`h-$>r5Xr=!Q>z
znyy2<<s$a2%WCM)tF%5I3T`yDKm#jkLEc<yTHB?8*KJL0T^g|1d7I=e4b0G))+-u7
zp^h`AO$^9X#r(^(l>sfP_Jd5@7<g)C2&OlvSz{^gIC_q052sTE{h){GEe7NYsRx+$
zGl&6b$TA&ZfECgr^b0i|W<*Qma$nOi2DC($O*NfhKyF=?gXt85RDhZurqc{E033Fk
z&Jb9W1)vKuon<f{An=*#9D@k}lfzBt8B7G|zQuHbz#A4l&;qmRBB2e_01h28z005&
zpxJD?#Gnj7_r!GBox05oQ2C|lJq8uH*|qFnd`xTex|8V_()m#kbh9%3#zkpenev5+
zE}AdhK%EE0{=vXnQyq;@l$-u!#4rBm>};9tNGv`B;T`pKCuBXpg*n}sC%-JNkX#3E
z9EEZ8_4EW-IORRT@^z^=$X{gc8-SEjJm0A_`}1>3kvS66cr}fpsomtv75EK@)8-`3
zqvM{bf0#!ypyQ{~iSj%U1R2fQoJQvT{T#EN0AH)Tyu&<|0qyFl9pnouaDr_x*HapQ
z8tK*-@~=n}m&raWarAK5yqU9T^)-HG-o=2HxbhP79tO1QUGcs70D(5ToYcC-e3StV
ziNZh3CmGPa)H$W*vkYi_?CWK|$Y3~?dB=Q}0S%Tb=gb`pQdIJVmD0%aMDs(!?URt1
zyW9L929p6)Zsy+^7-+*b<s0)Km`f`?=LDHW3Rtc_3RCxftXW*iL}ZhX#GA#J8PPHj
zR&5sB7|}9N=Ob6GlDr+`%%ajxI(@yNmS0;X4YK5$MV7tYpRTXGHpncVLtU#K=SXJp
zU8?I>5PJrg#Y@DmO-LStFTk3`%Y;f*$h!o>D?lsZA%Vjz2y&G;yfdepS^Nkb)}(ZD
zxc@&KK4uQ`*wvD&A6R@!EbvY;SY*#Mi=TnT8a1)_o>;WW3s+0wmLJUGk6euDa)z1t
z6^u36B<wFl_bKkx?}7x1D*5Bsx3bORuT)6+0yRdB_!~pqzC&eiJT!}ssj?k&g3aPn
zFliYKc|X`@7XJiU%V5^-A7=4c0L((m7}8zU&zNh-T+`+Yl<Z?xiNvwgA86oXvr0vI
zXb@Ju)xUzr81_LqdbTD7IF~V3zip&hrKak?-3mi%R#}q(^ODCG%mlCjpU7O8CYXkg
z%qm;jwzrgS6(x7GitZM^?1c)O2g;Y%AfB{Aeu#f`-<VYcNSHQ?j9ndKRt;p)Xyv^0
z(5xB)^fEr`0MT0J;W3y+5vhv&M6)WMYFIr-WmeH^sxNPljV+R2&ULeDEb|oUg`Dco
z%&IH}+5o;AZ&u|oRM`Tyg_u>78A^7<bz!ktH3eK>j-4~gtfH$aYt<z6*YblFDY$5Z
zS*6TTih%AsVpcU#*;0xY?>uN$&0*vSw11*mwSf6j52)WVtCljNW>#gIRjU|LGc$iO
zt6l+md4mVgjK{L+WmNQzS+$!<=-c2g1MMT!t_3>#wpqn9yPY~PH_WU$#d&lt|H_YM
z)dfbOK>2lM)ip+uK<{2KtKK728UwVhkNoS)Qk><AS#^`zyPNzTEPQLVtYV9>Rn$H;
ztG7V|tu#jidYjeTsWb;X-|#T2cQB$k)p5zJ-pTb4<*5L(TA8>sn|<@l>eo4sZtA|V
z$*d-O)sjnU8TzNWSp62Wp+Vf*YF6)M6c2PI&a74%od{&(Y*rseUaK@L3srP8tB(+6
zyY@T4K}XDLMI+jO0WRKURv%Mx{|yM!;k;JrZ3#wcH*TC2_4L;JX7wqwe65z=V_oBr
zKtGb`n(f3}qby46?wH_<-O_3`r|tc+5)vB|f^iPpngjQ6^-&}Q-A(9E38hGAb^8LP
z7LQ2agQd1-^p&@jiB1(1qiucY0z+BTTJhq>_^j>n`L(!c)T4rK^()`R?(=g6#bJd~
zI_}$TTS0MHp|l49*9uzFgi=~D@;fS=0DRW)uM-x2A6nrK`05)<6)!$J)IYXDgAP<~
z0w0U0@MI3OLZKq%UcXf+(7vLIp4hP|-+%S|XN5Pwt8b9Y;{8*FvLz$e<opnMVXHJ?
za9{;Rbp`%)GKk{GAmq7Q{_1g7au0H#+Pb+_gdn?3i{f_|$q!p4uPmV=oJ){*w!U{o
z1Or-ETmDu-Ypik$e(!%Pq8KPnYDF}F58{mA`pO#X3cA75mZNaZeH9;}>d)UnIhABx
zK^H9Ab}Ct`UzNP33Kc`UDr_#TNb0Ka(C&(4r9xus5n)|H`xcpAT)w-gVkFa%=TURL
zViW`NE+hRac+bMOCu+Wur8X%h>I-#xcPcs%1Z%gYTXhw^=#frFQfT9V%aV7T(t8ly
zLb<#RWAcM^2M<D|imFlRN)I~#<aeY~598bCq*fU-KyGEVb{%TSI~nt4QO53FK2iqy
z2%!+8c(}WV8;#?=_pvW?{m{M43mdlpNB4{U!0{oTN6GH*!Vj_tr9}RX`z4N}?!8Os
zXZOnrx^)4=+3ljc=nr1%^41DWnj}%E?CykqI}9O5w^mxQ`bbANr&n@AmCbuer83Z|
z^Kw^K+A<jp-O~>%`7R`X4YJpWpi1RJI(fwAos|wuM`5Ol|H#{9%#JmcE}Z6%wB0XP
zx-tj?s5)G!oJ7z7_OYo{Mm_n-a%m-x`gZb@Z^cyd+br!ANxFHp(u*08PqcP)r4o8b
z2ACpND&w3ypIWC%CG?O=zpYZ|Bqy$uJbTqWuI$5w$q}v(sPtn%uI~(Mxn><c99K~p
z#A#-voxNSD99mQZ%=D~Ojx6d41XPAHStCHBXJr_JmjKSSR4UPixd2-ODwWBwkjUC9
zmC3Lapy_fYO$MQTCBU2ul`#xjRPbUiy;V7oQ7f{-wp7M4SO?I&MIQDVoXqK!9DZou
zjkKb7<QYh_<W(xO-~`et<0^-7q0<26awWwX)*n6(aOEGB%Fg{Uz=)xh%EtY_G(Hi&
z5!PSe@n}yj;MSzd(J-y+>{1e|D3rgR?oHiMS9yST{WBUZn(8VIPACNG7P;qo$;p5X
z{RZXx-GhXb8C9g}4Vj>HGFSO^75m^>RbK`uhTqAm@@F2T;$`!z__f5+dLd<G6{&H<
z3M8Y_NG>2RTaVzuhYG`_zcNhjVi;CZHLQza?53*W45-h*eFR3mP}(S%UaGR|hRAF$
zJ^at6Dlbf<^*8XWIaWpAf!#o#LF%1e6+_RNc-}XviatQRVMSeafu&fe8tp)ZpBu}K
zp`KtH(M<N|#{IcfbZEDcnw^$c)dP)yX0bKe@S!B^$7B>Ds?xiIpe_=-JROC*KbT74
z?%{T7{9TdVMKzwNRv|6>EK%t(TMMzRwxoGzX>O~_TL@0{;7_%y%a;-=^zc{a{rjV^
z9(H&HqLrOzI&+b7YnDgXmxtBmFJbLlH_T&_7WI6CwJ&mQEG*27Ru2(zBw-W%#QGhz
zhgc8mrbJ;z4Dt|X@cCa7g<;Ok0VMS%wa|%QFUieA{Dey8%Xu3mt=A$C@pEpz6J@_T
zySoPmVK(iO*KU-;d%Jol0hc!ZNtem@lRZ=!RQjJ_y==bx#YO~WIy_Wkv5w(FAQF4t
z^H600t$$`M&`S_;dDi}Pbh(*60^Do3KP`s%Z3F22YyY087+*U?7{oNqpHEfzwVS<H
z>FmLO4_HQM{Gs5@W1(7uJDoi=_>r3H?Wr?n9Ag`8ar+V^2JT)JF}oA02|+J!9!r*a
zp&eB2uGT~;DnC!iQ4_786E#5IngQMD-+r2yZcu;aWX(Y85dIeUUU~%~?Y}{qK@fY3
zr=}R+Y1hP|D*DZf!h%{sGuQ?}5g|!Zkv=pWe&oQXrdq@EpMRC{ytjtKk4c^&ySqq}
zh%n3M09xty9@Gp)gvs$!Ee*`b4m-^-s);P+#BLh02})3PW|n3o=$u^m3SCBckY+T|
zm1^)FsU{UUTlhBoh#xhysc`ztpT_+bJZ@>|R7@e7n5rqYWu;g*lhDXr7f;mq0|eE@
zm^C$+He2=()!w}rZ(A11A8wZVTT=0;d{W<@PB*vQ3x;iaOCx6Wr3@mU{}5ee>!1<&
zs4EFzy}d@&8~LN3Ton7jSpA4rO?r^%=S0tBoZk6oVqc#6Wa*6IiHeo^?|Ag^$waa;
zTmKYsR#VU@SP}!7P`s5xKi-}gi1Ch{+ua=cDfz@ec#lB$bLhA069cu3KFcYos|E^S
z^d<5L^=I^V<dp+OF!~l;2*oh^XHH2uQ4VJG5W;w=66HU)NOqQCiTxlHK8^I4jd3Xn
z$m;#=Io?LETL=+1g^MB-v^_;>aetKBo<c1~BSW|$)JQZxRB06*LDk<$3?sUUq68Fh
zF)^I-9*EUBwImO|0F9CMKy1vZjT#6@p_B*WT%Z^rXc=fJ&;XzyMlC?mKw*s50Yw2t
zGujFi2{ee&?ws1e00|5Z=G4XkBr`aXQ%kCeU{NNC=W}XFZxN*Z2jbP7TGD9*Y5xIw
zrk3;_p=ER*dFYD>(*6VSORynC+J7K^n^Q}D7(v>9AU@2I4cjCS%S6`PAI~+py_E5E
z;a&KGiCRp_?aS-R@e7iP^v(FKn!55>G@QQwwv9h=>DK@<UrdVaTVTkyqi?>16|tL0
z2Q`T?NZ&ylgWB_n10nqO<)rcL#}Z>1sDLa&t5f12ns9i}5Sdmp6X}#;hl@nrv29Tz
zpEEp|OL=emCJv#zzH_Z}imGP_iSf+lPYy2A=59ydX<!u5+|`;LF`O{_H{lZGxw+^&
zJLD(P^X%~N{OCChhJJf^wL96K_8K^Ir>Z^Zb)AdfX{%{Fv4f5nYg((lU|e@xlecb{
zQU-ZdD}GSdD!1cfR+>F4u>pkNbG6Pf!evx-prHmlc6@x?XNMH;JX}{>z?0JjHG=k-
z5{|$NO}O{}w>lM?bZoE~^gP$doM%#Q`nJ06lsidYe^i=nIbP?b(8eJzZC0I+f*OIY
zCD!#(kUwz^sqf^9Z`uegZ`L!H!QvXAo*nhJ3gt^ck(u@6ucQ75pd~K#E(#jaDNTC?
zw@WNQyP%J;|D^S=eai8z$D}gR;!&TXkl#Y#{K)z=1uX?SnN_b-P$rP;?)vcx$_F~y
zvwot21_Ld1nb9fi0HAkrXLJfX5~yFxj80)EbqaK1DywHAsmEsVqZuUqiA=fWxKtE2
zSd2vKy5DDL74C^Zy+54cub?zKe6*~tRi$!g$l4Rqu;deS8)%DiGG9}T<Jtx%7<iXm
zWIvWAG|)SuC-b$`#k3tcseIA88`T`1+Teyj`^i{&+X-n^L0=TOqn>Gp(#o$=gx#2F
zj|qdz;I!JfUXo@~EpMlTZr6yJPBi%WEkW12Go7`R!Izp&{(;BIFMdPm%qN<ev<-fH
zkG$rj6xa9QO!|%~{R}nM`K9M)_QLAMKQ*m6<J71n!pHLWC-EkEsciGM<kvk6)!`Rt
zqTZIg0u7Dy#=xmC<@Qt?9uvwS1t0Wjf3$9lpxh$4`fVw&`#PoEMwGJ{8VAGboLa}_
zPQh@U+WkU6=OIpw2~_ZVK2K>(B<mGH>Z`fjNXzZ1?`geG$ZSkPubukI8lZ1oV>0FZ
z!e^;(w`1vh`!W8UDoVv3slJic*wfuQd;7G-V;ZpxQqv`aL4WVBZA?Rbr|CZV-loQM
zSCo4~hQfJBV+JZewL6H6i1)0<Oppm}L2f`3XEy2-<N?%jp>ZrwWNwfrP~Ult<0vnw
zKcT#i#_<Z`UoNV_51=*XAmQ|W)Ms<CaUwcSdA+@VY$JV<qw{l!7k0Zf(RWeKM1ijS
z>n84CJ^-F+)8wp}I^?X?H7Q#^(!;HoCf@q-_qyghmc36)LsDUfQt>;NyPN2P5U2NJ
z?6lcCH3dV2Ge=NnsC!d?l8;@G?R%Rj5TU#>IHb9W`d4`&Z$pK==`_w4*T`q_kHyig
ziOlkuPc=n2@ogH2{ydXNowRpUld_%SGvn)z+?oc1?yOp&8`4EL^Li6SAgrD;ov|pc
zX($?dj#@Dys7cvr5x>(9o0Odvb$ZE~rs3Rj{{4i}g-z^ZwbPDS4wtLX;H|jZ^71oM
z;P8V@e8S7$eOUfp69qm6erdbqSW_kga^Uhhn)uy5{z>g~Elp#YxfVHF1LQ|%q+yn#
zrW|PPtQuPLxi2<R|D21^l;eXPO>~e-%S9?>HRXcroSR8%D#RLau3V4mdNmbc3_9oI
zx!A9%82+`;j^|TTo9NRh*t3A?>K(RC1z>dUEY`hq@$gIWv+hksmJnYBln<Yk22BZy
zYpMjz`4|W`U`10EM(DY6QjV)_Qx!CNuH04;sR~x;T)7)^oTfBY1reav(ccPfs){5K
z1b_~z=4M6%Z2C$bbq<H$Mm%pk9NP;L;dYUxp+<=H@j;EoNKHev5ZfEp?*Fm%C2&<0
z@B3%&2FfCc3UUBJkWCTXaRt{jO>sxuP!z%3w-m*TS1(scLR-9=nwDBtmgbsThD&MY
zR=HJbwpeCXWLjpL|L2)=&V|0e|L4P8=6&bgX6BuDma~0uP=4j^y?UT|ZTX-!DlNJu
z`PvwKd_iLyo)Bv9$Llh?O@xeCG(7Zt8-53}@vo^A-zHLu?@94(qI@tuE3Wkc6>enu
z{R2Grc@SC-Jf9NEP<5yr1%CZQ>w*1HIqUuDi=n|1pi@M>5*mUD`Os$iPW#AE?lli>
zr%$}UDKxY;fCD_EOWqi&$`UruFu%|+0%y3UfB(hMa9Ki^SezOPo9)V>8+3zdCqf&b
z=&*<Zu(Dey-)($oJDV|dd?;Vte7J5CUIfS2fiMvq+T0#L<?@HvL(d-rR$(|g^aiuJ
z`<dN6U-mw<gAyJ@T!n$V0bF4dJ4@vEf)1V2OAZU*0xR%?HXzF{MnPTRa#s>}*oNv>
zTt_BWd&Oo4b7%|#`e8PMau|PI&^dVQo4bI+&9P^gJu9>c$_{stn}BNunLS_9$@hOA
z9tOG<lS5mOE??4}119&6&~|9{`pbYZ=T8G>b$n=hU}6VIOg1pPp9t*;jDFQ{1TbEq
zp`EZt2^=M9GBM{ItQVRlX;8XpUue24?Inu`L8GjV_6{_me1kST9BT^2066>uNK9LM
zCE)h>ZJ{Q93PihUa%)exPE3o`f*`ogk-hGW!<KMyz&+)#rI9%Ae*UnfUqV=q_yXQY
zd$cY(a?g7O+U|sowf}{JIL<fnc<!0=Zr>xAt_lm}s$bqu_|g3Wx$3_fsZ2~Sh{jky
znu0O?a*YCBW;mLHcyL5}Q^6j9>9Q5&&1Mw|=><FjJ~{{k{U}xeJpu}N2z7L;W<zVK
zmNq<dzMzfAhW$SmxHlcKwCQn7)541qu>+vx=pcrVn!^GLnp5gml)4oMkr?2!Yz5qh
zerU!WVw02tzRF%3HWU7EM;*1)wR2<B;^~6NFzSwer3B|pAFmYj#1Q|mhYcmopD)m2
zXU%?CveYi<haSOww%K&BqsO4&#oe_FIHi5KT9Y$Smot-c22svx8-ywaLs88SU*aJ>
zWM(nwtmLV!d-qXGEqgk|eaN`_=Fz?d*%0?peffUPf-z_`?$G)D%!08ESioVvYfiy9
zijX&#Hh-;vx5Iweo5MU<DVRXO%vkryPX+mstTBM5?-V>K$&vwl{B^-3N!FP)i9fOz
zz=~(j7EGn?10YztqhK0v2!H^BzbN2~t3Ep5e)2=SiRtr#xl(F36rEgGFi!$K0X*|Y
z!2<YRLl>jhtAz!2S&{)@x}%^7C4r*`0odNPU=d0_s>r1B4>xk4T?KqrdWb*QgS?mN
zqiYh5lkjc8+RH&&+c{uAy2jTn)!jB97vWGs|97k<BM6U$NXGu8JLWCv-kWa)-LVi!
z*At_w^9ylYi8m{dgDYb!Y+D}pgm#$HtvT)tv@Xa5hx~xl=w1tkdGhfK>{$kIqUM6(
z%&&xE&VI+VH^e+rAfUsQ1$^P`u_ELrm>2K?#lW@5@9VLE&nIg6F$?75iNvpYw17`1
zN_=s_f^kNE$fpbB-NO80mCLpOHw$C)oYn-FJ_8DC$6@icPmcXmU|P&B42FvgZ6`4y
zz|7xWsG=<fcO&M_<G`%_vJj_hI@g=;0yA_&VG9q}o2!5s(XFse6&G8p?XbefJzVxT
zsci1l!hTh3RJI_XaG;0n4JvzicVVuF?F}mPk0^Z7!}SK0CHzu2wTg?%+Vn1TRB=()
z?9@V66_=gL3i1k{_OQK9WxIYWe96Q1I+Z1CF5Kv0d!5RnzbkyLii^r3tcCllxTtK;
zGld^iaZ%TMqVR}^Ys+yvP@Pf=FL>CtP?_JR!mm7RTc~VdY2nvZY*hB#tA)3#xTvf=
zxbUwkE-LFCQTUIC>$T$vxZqyDQ22Y;UVE39C9@V4iXac;Ypa0D4k#4CRh&J6GJRYq
z600~<fl6CbD3YrfLxAe07K#=g##gCw&Wb|O!^8OMyFl%{UMPA?MmwG9RjPdV<wDV~
zijykCHW!MqRh(40?pC3g;8FQ1RfhL06i<2>H&bO=SfN<xVcblW6`KmhVh`hHYIJNY
z6n0k?Csm$^E)*|TaZ+W=#|y>!Do(0w7g;DadKfoRrFUSV*yCZ`M3vo#7K#HN#!Xb&
zb8Dgapo+0)f*k`fzfgQu#YvURpDPp>t2n9hg>Hr7D-Y*Js@#6DQ2c5zn)AB>rM4~<
ze|Q)-ZUHJ{Z=v|Biji-r_kOwN!e|8Nlj*#7ZOHM3F#xoE%Hq`vIf3JPA!6&AdkbSF
zU<R<T&cZlJ#@mfn4p_(sWVCOjHs8LGn;s?C3nfE-Stu_&t(1>@d~y-$EIq9Y@$`ps
zvH<1ta~E=6)wKC<aX<ZuCA8s%06Wj9PV$!ASF`PrR4R956I$D&P!L}3KK2PFi!pZ1
zIL>@|b~_)%_y3tQ)yg03nsJ;4^W1jLxLS;zlTI%j5=G@>5#lGQ@eIr6{Pfz=2IW+I
zHZt5#AIB6BxJ5;aoeG|g>@GD?Zfp^XPdy=t?Dj|UrPECYsf{sppX!iK=lv<YJKVmx
zn#9@slEef6<6ELO#e}52GHc@?9M@&4$xw0X^-N3%d1?x}Br^y3fp4pP^LpkbPEDtr
zNZRXfYDry5AF0i;0Xp@%HzeClM|k0+Zn4^0zVG_TvMJc<&QnuuoaB&?_(e^{L?rVm
z+ts!Pex6Bfi;k#s6NaU{p|*`=o~9mo8eA>?)pn8+r9FOA+snKxEd?g^qS_wzkshBB
zIZ^_K>7&&S219n9+6kK@pFU(bTnL8jacUaw2tC<dV&<YP>&{iv>w<67VzskB?y5^h
ze%xiX3)-hy0rFiYs$FFsm8*3|?Iu5Cx_>U(1MjR<P9M>h+@%$Ebv$Z!dM}i$T%pdi
z8qK?(UotWER^{7I<i(Qx(|{=%uP%T(ygw7B?Vkk9ut>GA3X=zQ&8DdKTC#i`%1fuJ
zMV|7F88EGMin>T*;7QHw6V+lY6JK}@mT^&WCOezrg`A^jE>igg_%r=LIpZaDDe^Bq
z+Jzn;y;^-$t`3mrSy0(lbqm&SN-`9>+o{{E$ZJMt1C#oTx*ctMM$W|hM@Sb!ByYA$
zy%U1Ls(tWve<o0~R;zc>F*;@tFp=j~F6ebk1~6S7s`skGLCV5-RR|2Ov*YD^K~-VG
zf?s<t=(@L6VTPG!$MYtdPxq_B(g@H*-bAz6M-^6VEA3~e@-CX7ajLL!h^Fx_nm1;v
zf|udX&XO<w8r%&g55Z%YSURAYcsb?l5*Wq9mpQn}VY({3;m~Kfp*p(+>1A2iK*7L!
z(X9%eAf8>p9(jpZPU!$=k4h3ORzUN@HB~%Gnq8z>_kcPe_w0>q>Ofb3Cj6FadO(_s
zq<J?3s0C@NiLdn4P<t}KR_8kE86ib`adr2pe+7DI>p0bn!@^G^z|`%&YF-DR|DRIV
zCF+0j4b|&oG-<yVh;uMByXUK3XFRAyH-K6?L-pbf!TsdND%LR$cT82izN$ja1#072
z)$5xo#&^+S9WxSZ`=Tb&A@=kEX3SrSb#PT9I8Tnhb(4TfZ;==oiMAX~6ZUif=Be_;
zXpFgf2c)4Xz`V96F}@-0MdU8hv@RZ)6)h5*Kv0Ohgtsmdn2*jUHiP>5HYtZ({YE9W
z1Wm*>$;BmveNSR4F;4+wH?8vqs>jQT9l_Rs_hp#&Q1tqfiRr}1Ah1pW=4?=6H)3{5
znmWMLYMS^MjE<B+bB`}O;5IHGuazC}h#m0Rvfb!_lHt*Cwa@q|Ua^N^x?UDQu9@&Q
zaM@9x?Ihje#f`9y`0R+JizMdL;jCYTY`P{J@CI3BOjGXdHB5sdV*Ho$mBbPw{>mn1
z!#kF@49s5Y`@A`mG)un4D<TbDwJJ5-7w4NOX~-I_tpuS;FiR(d{hvP$Lg!QAzDF=}
zKF^Yd;e-PRDer~*o*=kgK}f0naZv=ep;>PrKWR?UU@L0268Y_8i=G5ttVP5;cB$w^
zUVc6HebE{#BKL_~>2_~Ee{n9=goVX^dkxH3+8nySmHVI1FgJQV<itF4z1?3H@t)rn
z&)RDNIG4)#($>VzTY6&u<oWleh4w&}^lZafhue*FJ!M7Y(+v{VUJfWPtWK(*V&{3$
zxm{%a!(I>JORVK)ZsZ|*umrsM!l+kc>>(23u~4`1c76^^tmR2izsdGc2}A-Ib;`~&
zE^ohh?y6Vq?q=uF==t?1nsnOUMAA|CXNmSCS-|tBeYNaO0fd*w0(fVool~~<?!=j=
z?J3#^YjF>jve4>odJZ?09I*2kRmtTBbygR9D+#0k*l^$8S^_-&eenl7uVE{>JhR()
z$Ife8{y(<>aO5RB&sUXP-cp6-1+Q!=xxCFe@S2?mtV(Vh0PS1ad8Qs`OEntu{5CdZ
z!{l&QZre8v7Fkh#fwx2-zqrT-m%Y%AmD^m;efONDz5n_}yyMOrZwbXcMM>^X=PmWa
z)-K}xs>;f6ZfWHbYV(JdMZuI)=$?7r(ki~2Wf7b3!Us70ouMoWLtgWS!h}V9Bb0W|
zT-eBc_B@tXFST@h4vLFh9DSDPIEQ@f4>V$5EysBP36I>q7jPc4mWOezf*hA*7c^1$
zHrkU99TtCi$^|F6n2-(G$zG4-YU$Y8#gxj&!{{*=d&ok+N6O3KJGMn&*Lks(QJ5nO
zYw3l%Oqf(J4v>TuW}N+ooPJq!2iHlyNHyd%T|Z+<h#UeVmeiO3^Ol6l|Ff2;^8fNB
zVV2f7NnG8-Df+^YOS!1{o@Dn=7c6S{pH3k!708>e-TRzEUMk4t&9v|R<&L^&slx#S
zXl0I53<fuDfCSXyS*I8R2tx*tt%*|%1%xpJXwyQce8vZZ22ggEQw)>6ln!Y76Hbu@
zO0S1q<O`|`|212jVm*v{AEi;nPWhGzv=X2lA2`L!-rRZT^1T|zesPLdqyeV%abt26
zr`QPS();ENkX~?#O|lGkJ?kxb>9lM~llI0*wELy2a^!x%wMBn-aXmEqrPIuBC{FO?
zUoY<9$?quchrIpL`yiV0PVox>ydEMyscG>_<j<udpSP=c4f3&nBLBj!;`PM$B>leP
zmp$~Yi(fJ5=Nu}2&4ce$yv4xhY$|>Wc}>oH#qSU=<(#n>@ANPXF5cx~*jK#I!*I0t
zeS=}r$l?zSnpua6dHbLy^kngIgJ!OG@d@N*WhU<_{=}dkx1so~LBBDe_?)D-qc(E`
ziZ6H=4i;ZD7$$#IeA&p4+fw`u@={qrS@CrvKPA5SClAff#lIT(xoO3>k(WkHNh$ur
zL-R-RJ%grJTJeK)7O*LvYEt~KB!V-GM=`~McXEE!3a-&Tp;#`hw40u%xQhj+fKM*+
zrsre7xGTTFkz<7)fM-7|7V@^nm4Sf#J}nl!tue472++*+#X_4Bf&sO7wpcK@1Xl13
z3brQ2f|tJnD;P93Ei4wi{uNjeNoV`=MVQfD`j&iJ^wkl?q7J6!uUgq~6QEzQ7!w08
zI+qF3&vygOlNTj9OhM`Czluc;Nk`LZS0b}a77F_4CkI`;tnk$Wc0dDHedPVuApdT#
zD^%tUo(jq1OP+rFT;b%|Zj@b?Wq}HNF?^znuY3OLq)~WJ76y}XY?v#NWjBy-=5X<4
zIbWSbe$sds57{NYMK@Ph1OLKC*W(6$SdMF;f#3g?izjuGzTmKn-_((E?7ObI@{NK8
z`L5|k!JeV6xduM7i;FMQk_=rsy7)dUDQI54tK7gxop$j(RuVto$MuqdZ}f_bZ=I6(
zXAimFGVr4=xOO3Lmj&^MUGEtMMJrrK42INH*GU7v`eoPW5|3$V|Bx`(SMahcvk~yS
zC%7)7*U?P?CUkRM=AJ{|T|C9@x*{Ep$HblHxW1NT{=qZHPgh*maQb~^ww+V`XYaVa
zkr1x{1>JOgD}fNsUh+8e3iMvVgOhL%c{SW|%XJ;zaisvRvapj&_!01>U4Qs4wa~6l
zAgFu7Qme<VDek4b(*V9t*SxY3=_0p<m^xk^zO+UZd79&zN1?mc6-(lPi|FD~ODQk=
zf5QCcI;C}C1Rid}gt50#>1$AO^?`h1wR8)W$kT=rN9pSlV10-6Dt(iHyl?HfO!w1Q
zERDk6zgM~qtV%AgXm#0Hx?KX{0G5q#pS*%~$MdDTBrXcLj+;t%OQ0cu`5j94NPz3F
z`F%_GN+3pg1(W6LJ6z(C-wbS=?i9TjNX*l7Rm;i1y~riGru0^nuamKmt+<9Cyh|B`
zFSP4e;>Y5m%3cRh+!rN$mj>P{agF62cJ<1!vHb1NdVPcPJa@@eOYDGefjKm{r$5)6
zdw_mj3e<Ut`VIsEFLX=wON9-Om0knLE!?$G-<+{|O6iYSzg!DNgStfNEo?ZhHE@fs
zEn(ueJK}3g{lWE0g;{G&bo~-OqI<P*GmPHXN)JHLH7BZE<4EZN2vM&3kDWG6DLoL0
zd@k^#ZkHZNK%Q;X{7mVAR(c-mq-9@Q!h1NegJ8$Qu9}-lHyC!YBadG!-4H9+RVXZ<
zRJzD0ysH&1YRE$5aeKvYUt1D`wqcu%ok)|?c{=4ciS8!X@EVs<#MhZtI@7=p)FjM|
zqXd6`>!NgKOIVgh!g1Y8CmV#*G{VV}kcDksr4x+8)mq_%#40Hh+89!j8<dXG3GLso
z_NdMnNl7u50j0wYF<)zi!{e*O3}-V*V@lhW4mJoMXoQ0$A*D2nD($Bie#?_MsOX2_
zEgNN{qqHCTNJ;abV`u&Pbu>h7oLkz<;Ns#2T)nDs^^#l)erIeeO$6b$@B7chrKQqD
z3{+($81l1A6C=@Svyt!dTxnus=HWaWPL|4B(9;5a^OaK0bV^zv@a~h}SiB>h$aPOB
zO%0?JB-kHRQW}ht3ap%!Z*RNrd}C=?2mXNjTi$M8B3kM;es6Inert(P+mwi&(g}Td
z$-hMOBETCj7rgGy`PPye;za(SqXe(SK|wOsy5DUB2H&FC`K=``@+agMuPzZYXcj-f
zwdP)l;M39)SXjq>|65DboC;}ILRiT?L;GCWLM8Va>Fqi{x8z=Pa8dJ+<0V3%9OtE)
zaBtdV2(KXlpRqJnK90bG>@4@w-&vx?hwg3o*JYINjPbLSk{@JU&_Gk%B{yZ1RUQMa
zqvWh^#&xsogOaltoU#u<cs7>p?ymE_rBVGSwBmyZj7IT69N(dM!1tEekclPoK5vZp
z>x;Ddye(bPz5IKN+WFN-C7TUBuWQAdWw9^5l~E!?=Sp^$*;#_;O;B}+xuwyqOk%UR
zZ+&kGZ&R;ijqGySgs1A4AdXAgA6jX&rIOZ{@u7t8=arwxKpaTFj{9R)me>vQNgBBw
zZlw>-5<9eOzO-+B$>J(4nU_nRGT2Hqwx=W;&8@S!WTH`cP%E4mr}yGUN6ADv;N{>i
zndtudx~1k5w@R`Nh95PC>}m|zT7*Ey`0p=~gECi9el*KQ(zXl=BGLj&3BND<W4ycd
z50>~Ry8H98Y)LPa!0R!n2ESj@0!n^-QR}f5MvvuZm$Z;Q2E{#|D{0Y`z2`ewDQVS!
zA=Zn&?SAtIOQ>%|Nvu9-emv#=^ao4bo?2;1f^>X(&gj&V2+oV>sUQE;Y7&vms!K5?
zks2We+9q$8X@Fkm8y(zDf3!4Cecff6B3)N`3X8B-E>jN@psqK@`OHyVrd|+;r^n_Z
z%lp7k@-YZp(e4#LV*6DF)=i6Dri4&1lrgwllW}~r{ea694YC{i^>UtT-6(@mufvQR
zCrH6101uCDoCtx~Wcjks8~ee0*5)#WlbMgfQh2yzR#0^EH<u|GML%&(fg-yp*i&};
z6PGEFWxVh}w!o^QgD#Udix^T^<SmOZKr5QNOf@l+At*uo(^p*LZ_?#xbmDK1h^2qI
z#2+kLs2BagB6}IxmY#QsTP$0xmvK@>YoI7C*CnpAXqR4e-BYx5r%Rk?(U*GBd0E6p
zm#?5hOLxHXWVK6h+GXe7<kLdOF9IRwLIC#pkbeNpDgzWMBMz&i3q;kL4giEWb~8?!
zbbZk|C(~imj_+K80pjLpNLcOzfKxfc)r_Ss@eYtmnomBg*depZM$lH=bBQei%y0H`
zi83;;Lt%)&Hy{L$n+?G|<hXm=PnIbEb0|qV?GlTkWJtOSoZqi5F$-L*#m|0j-<y`$
zQKMMB!XI72m|=hJ0Y+9&n_>evQ3H^3EW~GLxkNrKT1mzaV0V@wahuMVCF7W+buYhZ
zdECCwB{B@68#+-&JiA+$o*}1RUHSlz^v+jZ;xV1^R#TnvG0CXY%K1pP*>RW91_nx|
ze&7<lwNU~a!`#Duw)o0-&a+yb<g^Q68yx`TysKN$Nt>;~GOLM8q#BB!(-o&yF}mOR
z*-|(D+W=0{F40KN(V)uD1`x^S2GHX<0CJXg|N65f&W`!^7k}WfMG!pM^Y}`r^cA+|
zXtXWI4*TbqBwcNw^fE@Vkgv+<FuzQcQLOB3V`;FqP1)NGxYb0JQ!>lmZUPBh2F_So
z_O|RcGTWUMW$VfPn#R4JCU9W^?)C5}gL^$pHn`WfmbQR<&EhgQnW^H*vt@h>5e9|}
zs6{QymgAq23#!8n%bdvn%A1-pW6PXWk?TJ-H&-chx`TePG;iNcE16HzX(D9Jk7OP(
z(q~N>UzrTKm`CQcE}JUFLfli5$zAk|rD6Ry%W|pVeW0?l%5se+$c%L#_{GvP;#Uoy
zEr*h1E6c{<({;K1ezgQQ=Gh;3`BoX(0I+lzx)yp-g$X~F(N%xrfft1K`_<C2=NwHC
z@9{Q_emqh#z|hdQ%DU=?{`Q<k*j3h>(z<4pwWG9CnzT<Kt?O-#sAV;2Ej`js%r8sQ
zrB(8{8MY;Pq{a0pi$!5&3wM{_EYa=ypd9VTTfX(aczbJEsN{y~!KGzEB%YyB1#!yL
zdhG0a_xj&(+R?CB93eHF?f0S5*?_=oc%dDD`MCgaJEoGR&lbN=z%GRGCG>}4@iF@U
z_elUYEiM+{fIP4Qp{iJXi>&l(9+dn6&2o{JomDK~ksyQHkmJSjb(Tt8E-LnWB!n~_
z0Eq~X2$77KJ4Y3ZPqEwxtN@VxmpiM{64|31vh<nickMZWTD;8h`%r|_FJFZJnpzhN
zetqiqWqMV1M9TT33s$^e7K_(dbj4j!iKAJ{J$|uRybdq9z^k>a(pFg|En)-G%u&ES
z=0LHa&)q%`%C3`(#bl_weO}ggmQmkVvx@~ov9br1y3J-vK2t36tnzH1D4z2WHPNc{
z<9R=E9|Ff?sX20O1>p7R#bO>t^6l#YXkh`2k`^vrT`UR>vNiXL#X<u>xPR?;OH{*2
zl(-vwE76+HK$wyyTa%6ZG{*GJIQPxpEumT36INkdBYeIa)jM&&Sa31&M-Lgz0LZa}
zy5yry1EM_93;+VlmD#f5gJi|oT5P)^u=)2%?oqcbsuf}G54(HeZA+uhlW-n~b8(zs
z;ZbAhXMel|K2|JE3u-5TfAdwRxWZx>v!<I<T%a+Zg6_jB?(c6~VnndJ#veEg809l<
z>P(#I@)Q&Mc69l{pPXVRsPCjn(*XdftOV;6H-NS*VLdP90-p@lEhL_#6K`P*Gl;je
zmc*bdPjZSv(rwl03UQzUx*?j>LWJo%ThRtiA#bANj)ce8PBw_Vz4hWYNIBlwDMnGs
z@4A#xSebdWjiPB=C8yhR2OpT@x#97PPSG81hcys%!~!x}?Jl3}V{uwMi{)$upcN#h
zk#^~~PSF<1IFj!U*J;~IS{8r2*C}|nge-nuFHVL_cto{HMj(^pmQ`s65^Z*01;LCs
zlJk_#87nz8<FDSag!aG@e1t*tw@$<>z8>)rXk5+GqJB=ng(C(2$=gp@t6)j%?sV4@
z5j0B21<IWtfQ=6K)Vr40=B0XpPgQ|a_%t^Jha7eaLBT5__=slQvAcM0%Vh+ij!x%&
zHopw;xjxR{LHHN9kZtYG-{}QuUa}MafDQ;i@SN{*{zRf!5cw22f08bs5&f(Yp~1>h
zonPv8`Kv1kQI{{Jb#h)jl;`|Z@apYK$lLdW^C0CNlXI_=Z)B2FPt#YN2cziDV8@r^
zH~finlkv_E7<V}%PU+!1EY%@DI^TIjdOo)KPQD%omSPZ{xa53?lCS#CQJi}<G#7^J
zFFQ92XinVgd=B6}1sW1Raz4j)NMjY>pX21?Ecf`d?CxaeGGhi*m6;cuGg&egB`KLs
zp4UiYVc^KWEa92uu%=Ipa{(g|nj5m#M>})Ku^t?ab~$sR7`%~hpYF^-3j|hxV^a&~
zc%Wpx>>Zt){Iw35y4X38%xB%7{bfn9|Ad0Hznypuv0iTL?CgT_zeBXP>k^3qZQbJR
zl0e~XkZ!EGtXKKd&Nd`}Oe1eojl7MAysd|P$2up+#@`d6cGgeMWcQ+bmZ;bgXEc$^
zf!rJCj7G0&w(ibz)?)#8wtM5;$3bgPS|Z*h?hDA=v={(9O51fzS-V8AbMEst5f;6J
z*7t0`?gyfS*BX=B{lE|?Nv_FB?LIrM_01&@$(s#T;C(1rN8nn;h9#md9Q^pjCBlkO
zZNf>w#J}-$#HeS6gRR*0Ob@d0#9`CoXL`VN7z0OXc=DMz7Mw@H@&nJrVKS9|KR@@G
z1fXQU9KZNXlAIHLbEYZJq`*rraA=Iqf2NakJ>&}$o=KDBQs(hzYC+~dLx70{b1k_-
z))dr&F*paKpjwYpkh}4HOH;AUopm3J;T`3srQp0!2=3JZAOg%Of#LM{#d1@rGy}ls
z{BqNZnsVNm1EOcyaVt43@7h>ynnC3^H1-)>xugZ;i~-e*))cLanpIs?p{z%s9c9bZ
zauYuZO&^B5j9<%5Lm=-#Lcmk_rW*iEY8TR=Z`n=&=oSfdbe5Zj;-~@l>qu2Yxz{P=
zY_uY(+(buxutXQog{@%-=+cgW#?v*%PA1*H?uLB_%F0cBSwA*HVM4j7pH^AOZk}0g
zqMJVW1*rUN0J63KtX}|-vH{Sz<es_&He<8uvbn62vX{>qT`r$R2pk1XMcx27O65-Q
zaOZLpLp~toVI!?+V`WQ#lorMtQd*rTHzg3*!%60qj^!r$0>+8*FxNfufh9B^1_Vk2
zeDI>!a#LL#pq$^vwq7^9T>J&%hwp<Q<7oW@`0vGX@pl!qcqpl%bp6C~lSL}!YGT0X
za=~X>={f&$n}uqM52Ju+Cyf;!wr4|w4~-#?!55Qe0cgIxTpX7|<Pa(sze}6MDXGN=
z7yjMp4=tg>$36TZY#9$0--G4iny;)8*E#!cmy26iL}Ts7-j3c@E>^&fN7^<G0Nq!&
z;aS=9va`jrZD_16z>P(kSgWaCWmJ&1t&z5YDiOds_k)L)8ZCX0!>E@_Xf%T0+DRS>
zl#A^$OS7gYm5UwHN{D_G;cojcMi)$cG%g@VDHn@4%z02c^0RWW*igc{ERm+LDwdzh
z#S~WMAj;XCQ{*N<^I|cjBij=t7zP3#_cFX#OhvC>DAejQy?R|{+oc|g4s2X5MhZnJ
z^>xuBWh()I6N{0QK+$kwG0O00_S+~KJT%$a)n$*<M8k)+eOb=40im=Apm&MKBADlL
z@BDymD6ikITy$jlXsx_s7|V4tI%46Voi!B?C>OpYyQY!(swDHlyL8HhzYG>=JRDSS
ze`Kj2mtQV|3|!Lba#0uV6;}_7!E#Yg8pa-hKUY4o)ai)<^cbK9<?>WJR}o?d$QRZr
zzlRDb45EAE%I~3(ls$;6`2pqkr3mB;?B(}yEl!5#ry9J_$2#0z4#D$&Ef>6>5c6DJ
zNP9&`@x}|FsG1afLk2+V9$LHnJM=3Cs4x|1y1zsFNZp7p-^neZru%v|-PfyhmtV(-
zRMLFivBKKNJ`A)_QHk44G#gA$YM9?O3@<yN{I&*hg4q6AIhOOnbi{w$%vnnL1x$^B
zqx@&ifNNfeWgb>_ca~pB)~!N-xd5k-7QzoM^pu+54DPqe4~E+mVdkyY@STI0Ee%Nr
z;go@+-2NtOuq_%U;L22ElQquHTj%+`ci6G-XgO|=5$5s!^p%%oAA=H;%SLTN0MPW6
z@^{cW41C}i_HQ|sG{O=sIdVei74j6le7w<A;Li1M!yT5%VW>%R*TOw1fuqn*=1+h@
z>14Jl7j>`nET0l6>&4Qaw9*;Gtkg<pc}gu_T4`Z5-i2Bz+Zs|ASL0ivl}Pi~Jyjmw
zoc6yd?QdSK#;v8z?#X6rNGlp>Dbz&q$yUR_v?>#LHvtqqR#ocWZ?=YcFDb_~A*_n~
zs@WPmWRyJ8FON-NHPA(MJD11G#X0im)Ywj}F!JcsSgZ!%K4{~VQRNLNW3XJRl{etH
zSgCnR1B?swX>W@)Oq99vEY=3C-_WRnIQi;Se9*%1f*?%9hCc<h(A6C%bH8t~M%quK
z&o(V~SRRMV-zY;5<NRA#uj-30<fN<Az~W0-Peb+LdJaohm}c|Q8N13w9o&?RHUNj~
z8O%05$fG3tklE$~X8S3JC56mObmkPR<aUp-TI-8Z?)g@0gnfg<(#eOHp&XV}kfA}q
zxdSF*dn9a}s%USpWc1-Nqr<`&Dvwa6xE&UJ?i^$s^vP!&mT;>!F~NV&?Si1dHysuW
z1>MjES)?H9sK41^{#V!);l)iSl$ig;?#5V^3a?0#7<KcCly&3#SU?Z%vE3c!KOxwQ
z&pwm+PsvQdW7{~)zp;orP!|2>DO&iM!+eoNNA;qMDO$y_@N<~Ikn2p;WX4j5nWylw
zlKloc%wGYJr$B`X?qfD<l)X^Sz1yLwg5ti`#Z`D}m$lVl-j5=0-bGC#_Wy6?vbNIL
zC$k-9932SnWSws3|I$5~O}cSVwUu-{Alqbzz^(sFIIfK(OmLWSJx_Sg&;@RmbW&Fb
zhZ)~a7T%lmqP4OJR~Q`TmoO%9S!Ygj<hgso@`+mOILz3!3GWYdj^{LCc9z*3W?bYG
z-naF#(kf}oqy)6&gh+?E%p)%6p2Iu|EH&bEmPwLDr<-Cht?lVBPxUZm_i>nUEnU>$
zDwHibCLAH{Mj5!Wvm9n#n!(B$1})s<Fn1^6M4hm^r#DiH9A+G1h#FkBQ%0IptPdlv
zNQ8hQia7ke!^}yz2G1Bs=PT(DoN65OnK>106ChV!2&o!auWBK7sGh@YBd_l2W}D=t
z*mr+*m=U){O|vdm@f20ubeQh5C_*bT-LERT>@eM75jTl2#&kz3>Vlgk0>d4quUPhq
zdvXnHxCvi1URJ{zBVKU7Q^Q)9>n^CRIPEauijt^#QWtPu3ed-@>7qVX*>rC>ObbZ)
zqfWWN({w|l9i}-b@~Nd4&5=T?HQmr?+4@-y6Gyra9u%@mbE}5>!oCh(M-x6HH5r26
zWVlg=)vn?Y4-rSPeB%fdeBbf~8G(F{IK*wT@Ig{kUflLrw6M8D{LCU=4`I>IRYkE5
zaarK=^FD9uWtTl=y&pMvq$hkX=tUR)w<5hC$%=$J#0Q}B<w}a;Kd5Hxp=!n=^1@iS
zA;N|Btt)By*4+-f*i8XLbOF1o3E*S2Ms4{HZI6I`Al8EKf^`n@7WoTx{<o^}^Q+<7
zHgCZS2M?2l?<$?_HBVKWFK{{p*BQP$^)jq9(BQ0|^O{4rP~`WNUgVO^;J)Ky?Jj1x
z+xc31+F7Y(I~`(;VPIi`L#*{^-n!f&7Ey#gn#3ZH^ab-AqJTyFb?H1f_QZ6~R}1FR
zinj3%F^zOQvu7hulXRLF*k^j&V8KX-$OoOjw@#N|jgC)@Xx_4*ze8}9<R7h*<w!DW
z^8UiXYktB%Q!mPpMW|tSRB8Z_aw|GN*C7T<K<R)@arqR7z+h&Nh+3Q%NzA8P6mAnp
zYzxylk6(0%8rptQPNUl09&EzJs~w`Yp<vBDhX^u&A;%q}juGyfZ+8g3&`@rv9HMy*
z_ai^6xBWLrSUJEU_`w61ff!WF1Qcdj(N-I|MDYZMsE!aIPxD2scpb%=5h#7Qi>?|d
z&jj1{bUZ@sF0@O<<@IO>hKyFz4gNWk$0?4>Xjpf#zqP)76c$~W<?)3WQELO2dj4A+
zztJEr5m9rVTJi=S%=Z^Qz)H&cl7p^W>mUDV9MEqylzEQxw_1aQE4&di97kB&m4UuI
z-*6nm)>5ACz~hb^c{tq1nDOo0T~juX>36;ESP$6&F`8^#E2JLEUe8@rAy;zmrZ}dO
zhKE(&dmU5Zv-*LOW47!g9w<2qq~pMrwsyyS+9GeZ+LYnQWuu_W1D0z1<8rwa%=Mi+
zPjPTo!9)WkxwRd*0Yn6#wIFP8l;i<^{xgmzh~EchpCZQ-&;kZD@rX4Jc`UwC5rH<S
zMPYj>4f%sHZV_Njv2S%WW-pM04<f)r8cW^C!;cz6iCoXZjT+;OM3cKM!VyL$SR6PG
zgy@hk*&&d-u#O`JDDAi~Vs_C>uq^OfU^Xo-dI?SI8I*W{Ux}D)m$xDnF>ESnGxU2F
z(X0ns6n%saz_S#%lGmW<BifPcI}@L7C^|01!wCaF_vKAlMXyT7Q0OMfXNum}r<dCD
zwhj?^8%p_1W-Hpkli}!dMWq7G);z3e7T1Bb2Sab?1x0fM@xNB>qIneb-m0Pkc|d^t
zi(QN6%RF?}?xAU!k0DMc*Pucu<GIKth|jNAGzHSV@`^H1_1eo&!I?_Y2v%?p6lAO@
z8WX6qjkRLRA|Lu{-u5CMaB;61#0#_~j-q5pmA5dSd7%imLZg9z@BU9wasu<v+<H-w
zyjHO%(0AHov`h=^a)N?U(0D>os$3x;Kd^662V>+kxLnjhZZd$MpId~NQRC2IrDh>N
zmj~^1`w>s~!A<*WxZi#3ID?kBGiqC7)#e!9tLudOakcz80qbA@Fy-^+?iIDI!C6Zr
zZ@s>4aVz+NaKEd5sD9A>fEiOKD}dq&_~ama4NV~zwbSbZfK~|vJrOR0vRwN{Mgfq9
zLeI`Xw<*XPT~GZU?K9px-0#caN>hglSZfh$KXJDYf)}sMXNAK3j&)N+ot0wZFpThU
zzvG0~i2PwwXU_uoad&_lMBcER+z9|r5Zo*#prPJCT~FeW6))hx9ji$44OLMQXp5*!
z{;a@Hy#t@X0H3CQhtUE`TBX=cN^h~Y7kY8YW8r>_(d`mXkK2lV6?H>&yB7B#VA!>U
zP`Jk~=VJgHdh9Bu)v${_QNKNZrt=4Clvnb?{pO&{q!tVQ`${bf?Yf#dJt1kKq2?(<
zi#=-Q_5!%nqh=nAFw_L+K|{z@s<A=MXIsMk(s8IjHQOuT+TEz8ev+;w{qgErI!7wP
zUQJDxsQ;+xif~!2mHA0_*~*O|J_=3$TvoUrZ@8iB@f=l24O2w)50&DbmWq(SXyb4{
ztl$I|x{8S5;YtQlUjiI4BN4Ba;eev=0D2NBzqexE@klb`IXMWT2;Xy7vtr(fNY6(+
zm01Mn@{5rcp?7ty9eCrqB{|%06gB>4Xj}oRazo=f0Ba0|Zvn8sIa1e%fdlsT)wR6~
z%pODAQ2>Xlv}H<d@SV}n7MyKpI|HgyhPJN&d|_z&8Ng3Q^|En$gLntOcEwU%<DVe9
zqiYnX4qM4+)p3dmJmY{tt7cQ{d0i`?9}+&gR>ANQtZNl<02&!u^Q2b%Rw)fftBU4G
zDTdZg0MMIKt6KzPZ}7OG22CU+0JFqUB)q#z_MI3HAjeR%M5@7Wl^SZ9hBU>{G9SRa
zs=7TdJ?43^1cdwLp>81`7)nd9xDqZx)VdnUZD@TBz-B`$f-#y0zyI+Vu>&;Q4Xx-u
zQ9-}c{PcUauVrAk-}D-Wwo{<MH+VE<R{(rvMxdvEieHd^HniQ6+VJ~dZTCR)r>@O}
zn9uc>f^j+4FWirpm^dzXAoz%L9GBq_;<ez1%S-@3j=nPmCdv>P030t)o7~TZSOXi_
zJj1^>f`dLT_uUTntAn`)f&55_H7a6txSw{K3$_=Lo{i#xWw>8(9hh*+ZK`i=9y&MN
zF9hKNO8~_@3~6vTv;kIKp>MnU*2e-419;~-b_af?tF<*6$6OBbGQDxkkZ)e4_k?*1
zfLHal*?Ea=!{7h(zj-HU-qG8}d=S8Upupgb@IjlzC}=73`EWmbIC?+y8U#n+Sj>DH
z=>!2pZu4!VUybpw13JNoUyg@HRz;|EM@AWban52P!2ZR7prsZ7cA^ExZkBM}&>dT)
z1pHPrGz$O}Tk!Tm8D%W30JQLo`yCaaR?I(y`^8h^M~23(pz3UB><gfep>Y&|5mKZ3
zgHVhn#xV=xv1NjxVHSWHp0Tjwq_hgZ9198164u_(f}vn3G_;fgSZZiNZ&+3t^^zU1
z<9|EAvKcfR?S|IvfZndsdYRE>$7N^@3HM8+R*V|0H4lJtpP}^=03RD#5#TIe7+P;g
zyYZ{|RtssCA3*cHq4ZA%JoJMevv*oRjeZUHYYdIyQMyJeyX}##kuO=Y*3>mx(RkJn
zT_d<*vy@Rih*~j0S)+B0{OG2&X_Z26JL5JE4*i-?<1>avL_I6M4JbXvivF`2{bxo0
zSx4xT1iwcH(`r_$pA}tcHM-J@kY;_#6JT~m%1NSLxL*=xFxi+^JK~~MPKT(ZbvY7k
zpl@0?AiZSNEuO8p6TklnFxEFg^SV*Dy#RKD0>e4t8h@U`PlFiL|AU+1ehQaiX%HTf
zD(|}irXhA|{lcw@_UW)1t1jQ~u#||Nt}i=$zb7=uSa$ec$Lc0}z7cnPf9R^k9eqC9
z`MqrY@1Z0l-1i1rvjLt?)hdZNYpo!_I-Wt|lql=?HGnI<+h}v{PShHe#P5GAc@so8
zj7t6m;I1(rB}Z86Ce78=@MUT?MOVYEwk<$cV?#x4k+Lb>)4?JKoh=E-#=2NrM*!_T
zezB{Lbc;pdzBgHs>xPD&pz2|0$OJIN(2xfpS9gnDk*EPeV>P$1%|!WhL)qh4s@N7)
zX=@_2tq%A78QLN#lVQvD4AL?~+e-k}8rt3luvM#1zXsO2BN#+%`;hh;Dn15qtV#n6
zheeyieQ#02F+;;SP<>`-KwsFd7#e;9@QbFQ!lU6nkoeA%41``*Y>muS8b(MB?}YpQ
z0u52Ax&|*yZeISn1}_x=-ij)_-m5Wy1WiM;NWD+IS_9eA(9jt`nkOXfDx!Drm*Ke*
z(?Zl+hL%2{G*%*BSc!NUD-o~p0F0Fgco85B37%D?*EG;fF#@62d_Wktx?AsBCAH$0
zK6x8j8#XkwmVnY_)a`iy&l+`m1;7SbH|Sd{CE&N(w1V=tklr+^`Yr;Y7Z%QXAly{}
zYQ_A1xbIzRl<QS^gx5z%M-7di0ytx6MCW;3G5oJW`qRhN{Lc$x$jcZ*UKl@Ke^gZw
zGef1AybJ$W$3LMlTF#cx=#3us##d<N(Df$5SJ%iV9=t=1I$oA_#P5IV=#Ay3cO$)y
z-pv7|;O%1&|ARZFp|zf!^3sty;kF}a@ZsQPSn%$R)XR{4Ps+yce`OB=%^*Ye7yzR|
z!GPC?Km53EWO^^e-df?l7h_Pwr55KGxzmcih=`ZNkav;xTr@)7B|?`yA@7Sm0Kf5s
zyf5{o7S%*P$)Dr=`J*i^6SfTZ{ghX{V!{ICuGII_wv3PN_-Jb*dlR<pDz5Pw;RW#K
z9dA`CKp|e4v%=CbUSe`kksZfTiK$4F2^>HMuIN1v9}m_T?kEVK=KQcOWO&|Scp5Y%
z27?>G3uG8Qq*6(WmkhJRdfLhHF5<s=Httn2kIsou6bnqaqnO^nQAo@wAtst@nr{%9
z<V_=i{8l3I>Kr0{b>yEwn!N8yTpbP9$VU<eR7xa<4Y?ad0EwHwYJxXLF$3rB8)FUj
z8H9FN!gh$wa!-q~hS&$fjJeXzE@7>Fz$hz*QR@v~_dZISQD5_p8GtSzA!5bq?XeWA
zQl-gKNVJ5tK@A&W9ilbjHz54lXp>t2ZpucaF6=BIvTqnZ?Ij?{4h|}NfOMbA`s&JR
zV3|>)rd0OM-{9LMm2HyB@YYok+lumiKopFG7txz{gdhWluhEDN(d4Mswz0j~VsmiC
z1-P@s5uOD({qdaQwVq*35l4Cc7Q5IRFNDtu!>0=XBR}SJSkqXZsVD>S+LUL)0wrOk
z@Wm*zPGcK3uHQ6A2@6E*Z|HKLZiK7Di^GC2?^%<Z@R~vxw*tx~(~6dGh^&N4;kO9`
zOldGGECfS5p(!em)f)D$M4DR<zyOfE$J;9#yk#>DL5s#eBuzGqABofm?bF~b)HG`{
zPEQshjUZ6KLlyuJa0X`1asfhQ_r<GP%~{VQlZ7;Zz{|)WF8aRM6~Nn2bsSu{8zHWg
zr8`k_6sZCs&^#Pl0RIi_sRsYrO#ZMo4l4%pa5RlCHetSB(GnbAnTN-kR(RK}6t%W1
z<`=&bb~Nn@*pJ|C9^L}zZ1fs`ahs>jgl(~i$F|r!yfb7}BK-=@!~5dZ2xt~RL3zW;
z5ALk*z;|47)64*y3GV?B@mLa?vwjCLB9|X7?U}RwB?D`8s8R&j>rfNu@>r$AplO8E
zfWR4K1|apJ^idBhMc_3`$1mPGU^8_=9pf<*n@9VBXd)6t@Y}k6vyt#pGxKQ5DneRF
zpcyjdNM!`Dc=20{G<X#u1U<jEk+#AQg@2x6-iX8Z_@Cf+d`D5{I3*P53RE@^nE*u*
zd1!bC;zCweMTwNP>ZUcTpKMllTD&zB16BBSW5|kZ5EhI9kT4$Ba_so+L)wcTiDX4N
zQ2jne!r}vNH=-U?Y*7$*BO8asppuDL)SE{{g8(Dh?=!MBUJ2bJQJKh2Hi6H0iNty>
zfLMDS=uwL$dWY2x)Tk}|S<9d0Wb6ceqjo{ek4WDUs6gf+(mw=F;$j2($-gE61YmzO
zpg(OK^*u5fNF3~=;0XT1k%pls*dVC$h_+S6&y~jaw}&;L^*`#?`=g8e^JqkW`0L2o
z;KSkq;Ak6eF96|X5RP<!2*bmF?&Hw2Skka1M%(ZjH(<7)s1ymiHdG54j6{FNzfsGP
zc?D?$HH@Z}cGS~<2hw&zsHguSqz@tioaE0X{(R3L)JN8%NUKLQcG-WU?=YO?FQa8h
zT_|Dj@;{4o8WI`2{I4Nh#WQDk0Y_QlVq2v$q^F{ovKeZU?8^~r5*H)Z@DWS@zmR^4
z;fZh}o{hqRFs9yCNPKbFzT1h#@VF#Yq80{MtsYO#nEI5G=n8;g+6&B-Cv&Dw0;aA9
zgG;?4>1w4iv;$Cz*H4LSG2EJP6`2H8wH7Orm0F7c@#TqtbO4LPTJW1%Wi3$Bg2jlx
zwbmf5CZHO{1k3=oBE111>1Sm2BJD=|di{h**R(ev41-z~3t+@5cjv~|Ap0I)Ato?(
z(;GDN&<o(WoA?UN`<bF>-d9LG*pt?I09psnti|0!(-z45f%H28Sc$EP62OHOtqiDx
zRGR>?Nl1+WMA6IuxMDyXnt4|@GoS}BU1;Ww|DTyn=T|D3-Dsw_CGj2bBo$FGsvdwK
z9>C+(rW=rf0|(?1cpjN0NZf=rU5d<eNGqVx+d_?d0Fizzfy2n4-2!$)edY!&qTuy!
zAwtL(m9O&_cv48d{tYe1%(@g9f~hYvh0_%nL_0JBo(tZF)_HSY;w;|qkaWObdeW>o
zdXg7{(T9n1uog)ALYWVCOpZxkwC71Re-`s+DSql~h9L0PZ;Acw-Ncc+O3^e^lQgmk
zCAlM#tS$XkCyq|06mLGW?;f3G4eow7aV*3)onwe^ItTKir}IbV=6l39&4l!in<YLj
zrJvEHKb|0^A4tM|Jhx%)?xMs#l=lnraQ^<9^XsNJ{RP6pAK)iK@CUn0-Snnse!vWu
z*+lhD3aAt#mf^9m7>>1$kx}`I>MbuC!SVo%(SVBjc#+*EAUR}!3`83S`td7ApLbNf
zQ_x#{B_fvsfgvD`{1M1-9f`0P7#%fp)(i-3=EKDT5Y3Ue)F_^&dbg;FQGFAL$C0|g
z{Z6cg6)Eh$xfw$$#dr=%v}p1X8=^y|TB8mkT*nN|yW{QM6!R+w^r>aa4J=e(UKd5A
z)PU?UMQV)(B%K1elaZdpgfwLYIUwww&V>nMA41cVxu&NS_>o*2g(+fE=Snd$o*Op&
z!amg-;a;RP=B-JS`eGIpdCyh~`)Eu>l>)z5OQ_yYkz@)SIqyxVn~OAuz<OkgksK(_
zgZ~D;gtS({y0cR^Fw5gj{30}}-p|&+9xNyf=vxZ9CuzFsjoU;-v!~I<Nz)-9C;`m7
zklp}b59*3;gOk_Bhv`I61~R!wSp+cLYQKi`G64*?+SstwK1Tb4T6e_{CXMVmP>LEf
z!){sxA=1=FAgz6mC(=PBm14{+X+mkZe>^4(5#+`bfT(GZH*LYPNd(z90c?sCPiPx5
zh`2$I6L=4qJfv&_n302?Msg9jfXq&$jRd|!=1(N7g+$P=Xy|ApxQ^V5CcOY7R&ioW
z+NFBqi7V0U7Z{R+xv5S^r1o}NvJ0$9Zv*Uy)O!I~-qZVb-6SJ2L>iGHrFvx0dteMs
z-of=fwW{9h@k$>t{y?P|D>pDn6{<H@oub7sXdaJ>)R~7glZq>>IE)|v0N@6sXF-%(
z0~yR=buJO8hkY4D*7=h_6aaKq-HrsBATtwb_^arSTr_nqMj4z;Ft9f-$9$5zCK5ke
zls5Ps23_9*f0nFSV1uvJJq!PzV7-CE!-eiE&8$(Kb6cWx7s!5O?E{N899qZ~SZ)U$
z{jY|;1~k?gxqXO6`Gn(KyP|&0+-c3NQP#Cm;`rv)XuGRY<V=z-a7XpN3FR%$nH5vY
z^U!_^3Hy0d%FE~|_(VPF6L+k@peO2~e)Zzp063s1O})Ulmn9Xzc0uZ-;M*}Nrz=H%
zPt+P=-jwn=(Ah{MfI#!-{tDIh%K((7L8>=TjasIo{U`XL_$j2B1W5ZF($fY9tO7%0
zRpitGLo%9i0@l_V;XV-V#9Sb9t^tEL&Q3I;2MlGr`v9z+bX{A?#b;XT!CC9^h)ODh
zf7ZLHD;tL;3po1g%GOj@7Hr{X3aD%xMwg~+A*jKcwjD@0u9}Why|GXftvK1H9Rvr$
zXK+0NpCHp3!8N#f5LAo<QQ9S-h9C{-3gEhwfw&BAf0iP822VoDL(7V^duUli2)3+9
zdkAzX5&{tQr$?f&f}PrQwps%>?TNOJL1lU$kf1Mv*Ac+v7`&VMcM?LZ4*m#<yWys@
zAu}Bg8GHrlJOKop;D3>D<DW>MX<l(jA+`tsu|85=0%gcFMM@;_A~M~P(&{Ou<U&|;
zOIq`e>NTmhB3l0iuSzaQdA1!tBd8P02vhP}K#P#3!>y9v1NJP4TZRKTgxvxhc_OwN
zt3CiSoPE23YXMWC?1YJP^;yC+Uxs)H#<(&izYZzifuSGe!)-(MAnlSxcTr^j4P_Hq
zb{}|*{E#CoOM$C2{Rc%(Ixc)91a1*>3%f8=@)?-1fGv;WP2|K#4MF3C+-Gr$S}C4H
zOq9h($Q_9b<p>e=17-0b6fe_@zr}Xe#1F^UZ-Nw0UqTG8n(U^6mETVm={qpx)lj{r
zp(|61(Ok(j*vb{)dJG9Sii-48XbvL31Y#c2lLW3K123t+9YHMpHpHKuhs=G<Y+dpQ
zDRwq0<5Ga?#p$KZG{lXG*W~<(zDxPDQcOx^a88a>z53P!0am1@W@w)JXsr4l5@6+i
zMEZ_~#i7d08Up$U=`S>8vt(dfgAscGQ?r)XIMhP&VR&fPrP3}Y!KZPa00>nHZHyEL
zm}VjFaTag$h<%NKK-?WgbPhrzU^%Guf4+E37CcbISPYfa#wfr75Vd%+8)o98V49Sn
z2}<orrrA=^qy-wzo;rd^ETl-dOqV{pN;=P%#H8mbU6ZU)YAqyE*8#~z%=9Hvz#U!F
zE(5)R=*bqvJZ&9}`xs~r9*ImL68(~TwK{f^PWnf6<nx-?wlxj06;i?m#q=#s0NXaG
zPJ3As(6*(4g#A;(h%^vrVyt)uDPmV{s2#ZxiYc`=%p5sMG4)1IwOydA$$ubh6P~l=
zJ!dheY>AUx%V-JN(IWZbWO`3$+Nm>5%rls*HH88;5}ngd2-==7h>9fo2K--Y{Kdep
zGMLS&I_m?CwVlNv*-q9`xOzg)b`0|DfQct1It5F?nLwu+=<m7Sn<-m-R3F8BE)Ru6
zjKVveac!<o9g!k=vla82l_;KxVzB7TiYGN*_=_xD{@+4x$+=3`x>XnaWF0W;Qs2|d
zju>T;lI5&ct@`c0H1JI%9*)JCy4~OIciQ6k`jUdvUeuSq-D!X#+Q)(%=cpTW>l_#e
zc_!~{@{WzsY1&H~%und3u~-@mtz;=q=6Fieez>8qr!0R@F>m!I;Za5GyT%N3E}zNF
zHJr9(u3-*CaUqA&Slz+fuh)f)lnsyr2j@_2f54D^iw)44R^<ffWuNI~lX^;Yq>eTY
zjV7l`bWaTptJ$x3@Bb9pUO5Q-)X;%}pZz^0Mwp)IBhtw<ZjfF$$S52K^(KTmv`qni
zCvZ{#xoH9a_0;`g9A+1`X@{r4hBiH`x9OsH!Kd|}eLPx)3$6}jps}Ps7Sn(V$Dh=(
zo2W3bsOYd<hK)zdNn+Uyfkh4-eNWf(wWi1DyLDUizQYL3k6{&1u1#Nc=V0eB)1K}=
z(%#zGej6^tuQu?Nakw+FW8@Od2^}-qbMy5-Kisc&Y$eNc+as&}D2oKv35r<H(!qmp
z<K<P=yAk4Wvw})x`UXhu!(Cyg;CT4Vc@(w6LiHN*SjmNUvIG0Z6kvO4yS>oizz)`T
zdvQM6ZZ8yZHFPGIx}Aa{vjQ^1RPO}1ZO4DH!042K;w4BpSrnaGBZGxV=o$o9W%^bQ
zOE%3$F|o5g9+mNmV*W|>=44@S#=EsUbqmn;m!V<Q=)*THsor6jbFeLa25X(8H&t)>
zs?D@4MH&8^>UGDjQut(}+9|AD+6q)7KO5+F1fcye(taS9)lt3fMgh4J2&}R~zXx(x
zvFi0_J=7u@Yb3QMHVwQ}CbOaHb*~1nSUss&GpS9Y@}`YZRIk4^I~27aki9{PFBf?6
zF^RKNff$E0Hc8v`roAaQy)kejAIvx7mqRh*x<14Ctiob47dT$s&~SKC0MUZppVl2n
zvJZWr`|8{n_>j4#w~b@j=dvD@eN(x>2mKTZnT-*^7%06hlJl8w2Fp?AJxhVKcQYFP
zhk~ak{a)nH3$@`0^uM$>rT+!e|I*g9#g6h>)$0aNEYel9$75)M8`z>^VWKEKRsrZJ
zJBeQpRK0G3H{deP@Y3$tu&x@QdU3Izu43mT(tZAm;+^2tC}@g3z}yILUVlRM%JyZ`
zQ|T7y!Lfu_b`u@IafDZPcOAfqgVz{;n0`|}HKuyy$UcMUBW~4e92OAiZvm)<_2_u?
zal=mlY#*k2y@={6$pBUiQN7j}fc?Wcs@GZx(VS)l_)(J42KD|_y&~8Koz38R1yqrS
z9>=GuR}_{@oy|TuXn}@ulz`~iQUKDj5sET(pX${J|L5FRz2fClWH-40ku4p}d$=C=
zQEfN;v6O9yZCdR<aOHRD%3bmjXU(0ZpsWFl;9MAd>STCrKn{@jFa2`^kk3a!y^Swy
z<mrq04btaPd^L<}lket$ABROD)vc$9G1zj{#!Wej5;<A5-Ni1o^G&#Qq<u1aYs_Du
z>W}mU7Mo)&SZso0f*TyypiYTgsG8bgb=vs?2K$89L6?IxhF)3w0dCamj!e)bUV>5>
zKx@CNIdo@kSa}=+To62;(_EL@#Ldxg8-TMD$$8H-7lRXo+et`{vpPrDY8;5cQp!-B
z0q!1jjX0e~`4O=q;=XD-=O+*e_n;-_V!R>}ieV>l7G1#>?shqGTK;4fb-)2kx67!@
z45UfgXtxz$<Hy6WFocn?GH~yHsM-n;mn3o{kS|B^JJhOeKK2gmjIixUTcdEL)i$4=
zMAR2ZN1~<ywIGC!tKe0JNO6#d&-B;|BWVW^cJy$#9awy@$F`7IbYu7(q)H&CZ&qz~
zbYW&+R14!IqJbW0Y^n@2dEIcbFkTT|XGP`Uc+R#+x-jT^B6VZKXQ6%bM}uN1n`oYF
z_^eE=gSu`L!u~%Y2eBF<$F_`pku^>aIaye^GDLMf10htbtqii3>S!zoBsvdkq<lTr
zWu4RtyV<CCfecdxxEKPlxAc-wl-NynX%nU$siIr6|EL(AuA_TX3a5J<@JT7dv-OhE
zEYT`9e437%^Isvybu{OBnx7v}JBHr?+Ky^<lQTW>84`a_CpQ*6`QwNlfmI?Kp_{SR
z!H43}NF#6=85h9N@R>RMxdgu(KVG#~GLYb&f!?<i#qS#ZF5_V0m>Clk6g6dxLJv(+
zlui+uiDOcTY_54u#K$N&lsKkYb=;r8{hBzYc{N;vXhmtDx@&c_+U<U-^|n4oH7EwU
zCtT;lPgUz5Ix=!O263_OIvJ;>>#PmK+VRy&(I=H>XZXc|y=vvn8$FUS%!j@UmiLg}
z<x#05Jj(kh`rsEQrM5{38$B{LLc0?A87NQlcp~HTO7R3Liv!;psx7|`s9ptC#&r<f
zK>8jgHQ@#~<2E4dtRwFec%&#%2-Z;nXwd=i4=iRWZD<yxon+K%5~kYl=?n1~&x3}d
z;i6h2@!+~im>m*^!ULiPBlRYLMu=K~G>5=*$heVKAX@ZoA}{p9W8@wH3qh%;KTfGL
zD->GYShZzQ&lu?Gp9i+Lk+xC~?<wq$U>vm%Y41r)4LtbE#6lVY*t)%T3n12D1mn<8
z3LSC{`qHJ)wyMnxp*=$&bO<UQg-(t-LZNt)N(^D`e?YoMbA1u^VMr8@mWHAu8=@&1
zh7gEF2K(oR(FB?y(-sMLF4)D;PRL=*H|)|CMZKXHmh+V2vA;;$2PG4c^60RAXE7qM
zEh@Ezg0Xx+RMzZXSSk7`((Vk^W~1G+X*Zmx;ZsO6X?F(X4nf>&xC(Q4!=;4Y2lNHf
z8OZL3!LsTDj8^Gu%MesVnl9aj(i^f%F%i|E^plhx0#39;L%!{#2`8l?h<DMLTBGSL
zLohm`(HPNnQG<RdG@O}h(o^homBEfmkug_V?Nn`e?Yroe1-XNt2NQZZI+1d5;v)vX
z3<#55G(BZ7dN=w>qzMELAcM|~o=@O7GS4BEqe%yU3*k$&ChgUerQf0CHKdJLa@j=&
z7JK<J5Y5sXu|L+Q!w~tQbcJdgOLP8)IsF@eXCD$TcsE%AN&TAuI)#MuB+(x|8vQ%c
zZwNRUTeN`F%Q$ct`hfkHu}hE2!w?&}o$O`5%n+zMv|I=laUnQt(*O9nUWAzm6=)Ah
z@Ob&FC>>NAzB?`+AI~Ox0tTQfqUnKyvlJx;BQvHpfeFYYAmJcL3`SLBS|T;C0c9DN
zsm$%)3-{#U5UoFa4;9?0+G;~=?`{w~=o`>>MM`7UcS6FTUjdCk;=Lk+{z0YyiDP!K
zH!_%iV{m(d7+e>bSCL*J&=46kV9Z_u$;coe#T-VBcsXZ4Bx00Z{!^N$mJBR&dN+V>
zUe1X*i*%a6dSo#7#(W78yqpvBBhq!YZT}1|+L^(I`rn19SAR?rTuozAL==+aV-PNC
z49J%C`%tyjW&LneCI-PBVlcL2?3JwGhY&t!7eK_g7~I<;1|3qAMp&yg@`jfVg+mQk
zz~;v<MyhJbtKS>wkAPSu(HK`mPm=NO9N5qajq2zY3<nu*zJi+bsx1T-^a-K`lR?r5
z;iwT`Akl<}yMxf8jnL?g+7oagGahL)YB&%JwxN|;g`xn?lH#tbHkINS*$1@%=|ZFe
zO1n-0>3|SQ8<i7)n>4~K(Fm2q_#5zXr5J=eUJwqFuo7uODLjiF@D)f-ARUL4Bpxjf
zKu<Tig7i6nf#_+(HKh@n2~q}5gOsf_B`HX?B~l2lk`Hu%3b7!z289HXuN)AXB(@HL
zwa7F?ie$y?59GL@mk+!Haq_lq{9+@l+9DwG3EV>>2H8NLgv78u2x4PXky=ZxZQ#n_
zr9=GUAvo0*LoQySABawj?S|BaYA+#!NE!P?R{&UG51I>d3y%Gs;3@mSA5b(LX&8i>
zUVk3t&$Das%emHa5#IU4HXO3P{xXPik;cK{2H|qq>)&8Nn`??Ya!EXB34;p)!l30^
zKzPDfhK;*LryaBjpIq>leK4$23`YN>6Oy7-TWeU|_Y$lgfCv{m2WbYIFABB|fTgi7
zAe9hsBJ&~AAp*}La}|kKzy@Hb#6CpgG%{eTqQoJ7#RcPLA29%fFp1Z~2J8o_Gg2B%
z9DGtS+ZmMl&ilVHiRG8DgHTb-r5HA^3UEW_YH_SDR*(N1##nq$XQ^N@>_#Kuy<=h=
zW))0bzMNZ{p+(tq0~(;rK7y@06oEy-{M8nu{bJY%Ad16G7{{xd13Doy3~4Zd-pEWv
z%13JsU5?fcqOktB(lhk3l!a)aOjB)-Q`UOO>VFv27)^1UQ2L)i27Ma0oWNzYe;h(V
z+-5>QA+rMscio8ocagy)8Fv7?tN|wMvRD(;POFJ_!DA*gVI1_w-9EU}iXAyjwGE)&
zo79Ugi^Iee_YpgC7=^w9=pxdm5SF=$^I;O&iPP@*Jjg-=_JN!Udm%yY$L49crJ5^c
zv~nNsYi(-Rr<#mDI&Y3Vi0z}Vgfnt<G_FNL;1eM_bAf?gPV~^TayAwj#<}@;oL?G?
z3oIN!lDSJKF{ZTf8|cBAcFaUrMAybb3g;0Sr$b<QasbaGG7e%Hu<fL3z2FNRj|cku
z1!&9Y3%I;1xB6V_UP6O3<CkBePpQ@~&|cVbfzuwLTK9Q4hv=O9s&gIyCr+Vs*7qbU
z;sWns@QRenmsyl#zOP!hcqA>>C2dKnX2ToO1`?<KpjuaZh_~y+tD~wBukjF{J*wKR
zPJ?<tfKKg<tw!xKs^|7ntxM_Y0Y2lfyR??nN4~DlxbbrUl*0G{p{Pk_tZL;A2iUUK
z!rQOL<EQT_{G43^<FX&&OJ^Q+!>A6YVAkmg)%0<V=V_B!p61MZPqnu8uzsvt+PXSt
zYIRNyzVt0H2q$LNcHuB!s*gNVv$i+LaO!65V6VZ>4b-bKp7jgGoB-1=sMcsh;lO&j
z!e~5Smi`tNBK3!>)<&R|-$*yD9S2QxhFC*5ogvO+{X3IXtCvSuhAzwt!t5SD;soQV
zcuf!MtZ}O4u7`D+#%j4M?Vz36Ce`w_r~D1Q{Of9dX8FdDJjJbAzC{ci_y#*?$PBRC
zC)ZFdCp`Rjbp8|3tmi9;0@{arwCY*-s=j6gfj->wTh1pEkd5_e3)QmI5HcuA7lMa=
zJj2Ao8`eM#mBSXQ7PmocA2eL2c2}ofZJ3O9w7lqPjFDGWOM$^U=mnj#piwnF3#xM#
zdN|AOs+KVx&NDjan1-0Lj2^J$RA(LMVO{rzYKbyf2e;Q*qht%|?Gjy`lT&v37R}o%
zy!BZ2s`nVx!ezr*zSj`0XNi-ZXXinlCBBY=#~Rr<t<ar59%T+)nNRf^`WYhGHvUo>
zL{9iawbYU&6!ZB&)%>SN%<q~Q^PkddG@H$Q1zxiPJ0Qq0|6R>!Gw&yFR^Wp}R`Wm7
zN<AtK5i%+*(Hy}1NK<D7&mlqvPkz!=4E`NPG+wWoPZ~xH$=8iI8B^8d=F_|l$MAad
z8EFJYSMRH;`7Ev~4UxOJwRte|-3O}qQ?&Vzd84!tX_c=IgG&J=70a(n4qO4n-TqX~
z>phA;(G{<kiXpw|qH2CQ7<^yJOF?s&=`Hd~bz3${TRe5+r^=ONf7WftCsgsNYMx}M
z7@DQ4m{h%ir&Je**Kv59!aPlu$o37HE*lsY%vh?LXJ8(ZC&ee0tsG*lW7kI4^J7(W
zACIn=bzOa`MQL;2>bm+#T@bl+t!mEj#cKOyj(6p4*uD0!YQ{<w*I_WB4gDx!3SMUc
zOk?(f(iWFHCaUHj9zGBKHeinXxgpm2i3M3W6Z}y%3qL#vfJL)*RVOo!&jsN667F8t
zX8m_F+!uyeBSvfiiF{gr*jVk@$#eq;6g+l{`yJ^QdFo`k(FSB`iV_cr@fLaNRE5J@
zFJ6_WPF1+pz%_Sg48^I_r>coZ$P7O5SV6`2sxO_`^t0AyO8gKY2kB0$-wwDPtCZeH
zapL?w%;hy9<A`9pq6|_L|6Z!;1RIG5XOZ~7lm};75A_<HXsOj8egY(S9IaL1M5<ne
z5+I#g6+F0?fm^3l!Gqfe+-~>NnN}BjHvUtjQ!pJ9he?a9KwL-qMz7ClqdtEFc^62;
z4bvIvFp9!&VJ6hjD>>Xxj?CjYP3k=jR-x0g&VV7^OErBFLe~Q^qEA8{E{1ydLD(Df
zK4@;CF>p|Bw8e2QaeEKqR+I&EAW2^t4fm4+pdJT6)<ZPd<i)D#sNP_YKLDY;19U{R
z-e5;tYYmnV1ErB#dmU}3wO2xWrLmIGMr$w+asUYXY7OSW<pVd~-98I_jz!Uk7P6Ap
zXd^24e|qwo$J!CvdnZF^?TBbt``k3ubkMN&l40%0Ub?jhbq7r-g2DxwwFh+vO>hJE
zT)bwj2Z__0gf}#6J-82nJLDcST(eg9y9coL@f-giYtd#f%v7nGR_TWI`W}Yeq2G;8
z(+yjt`&|O+mT*ZkY?bbIi5TIDCe1DnE&{mF#Go-aur+y*7&?i~-7`jLewWx6Am74B
zztb0kO5%7R#{h}Rz=Zc*Nsmcf03<JC(C^miewVlc$nyXA-Ml*3IPj{?<GASqasQ!e
zdIPJ&QSV~oz^gWiYmr{$&c-gOu+0YTR@By_ns$=%8-o)2m&DEgkFWQDi}H8^{-5WL
zQw#(F4^9tbJy8*{A)umSLp1hoG<FfJ*t=)J8Vgb6j3xFKYt-0HFfq1h)M%<kO*fq+
zCNcf}&hB%lzxVzB{kX@ov%9miv$M0aU1YR1%LgK&ebm@tykA5oD*<keNXrsuU$v++
zPFeV^d&qnVusBPaH?vzK!%P#AZaoP?(h+H|tO^O=yLFDTb+Gu}asl<wTln68hsE~}
zwYB)h9EG0!n(rN8qiC!c#;7qLX}%Z2egpQimIptqS_rIoBD8-UszSp;`}-#JaZHxR
zUo9F}!K#GLmNKi>5MVv461rHGP^~k-_T}Y*tteYps}ibZ089@W&O2oQ$;fCdOLDeu
zC<R%%t2P~nT`h}|w(gMt-(u8fRhqZ|QE5n%W35_id5OspquRGoHd(XjciXCqYTE#A
zk-BL40qnzW?Dw;!6$JJ`!}_aLmI<l~(nv0=%>8A3#ae4ii`DhmKT%+6s?e)*7tCps
z^nk9vT4folv1D7A)wCv<8K<RPvR3=jXiCv@fV9e>x)*?=)lAuiL?%lm1I{#2->o26
zsRx~*m7iaiQGTV&oT483?`Xq{=#D5`vm$uD(4DKTnd-=4cGup7)RF#D%&HoHMcL$e
zKj$~}kOM+#fb`n7L{%wb$k~#ZW@AYd6+G~6l&v}^QF`2?a@1CV0Rv+sN_Dqa#WAWK
zfxr7Wao3Y6?*RjGZ7G4-&}ak*P7aW<Z&7-xG*24R@(lZCBcp5{df^^ub0b9rcEB-(
z(}O84E3wsX;cmEzK~Fa%iftG@ZpiTkBC_J<N9K(-h7)pEk_+1Nhofu}psm*%w9^Fb
zz!nN^1j<{X{SUN%AeY77h_bmtA-5y*a#1)}399Vl^25h6Ah1P=jgk%&Ig71E!Rjdr
zL8q2kCra%a^|rA6=g?Vz^`S_bBec|G0RIXy*ou4ct47)E2(0WOsWu#hYlWaeixok3
zFpGG{GKmtKP(`gw+w6G66o~DQlaqErrNd9<41m+h4<0!P0*gQ`F^3qj3xUojs#IT5
z4K-rXQnAaMGvKE08Z^x+lpLp2g~})NRi@K+WkDG7rG8~P%~qW0v_ajOPTMt;IB(%D
zRK&ByS1<q@?r?1t(_zIA2&oy|(6AqF=J#`k=`fkbzK6R(Uc#hmM-9cJLBoVcOjcoQ
zRbztd&{^?MB9Z{usJF8;0F&f!u6;z5`6Fg+C+a_D0dYPT&Uptvoylbh;|NQWQLs4_
zFQas`AW4+GnTdD#UcAiJGRu)sUJdBYpV2;@+>rDI?oN4z0_HEpBnGoTc0m6<iw1rg
zB7niMs!`^jA?S_4RI)r@P@QvH=HQ`xc$pooUQT?^@<IJom&@JkRhdB=oQ*FTmjDV{
zSBy8x{6N?(lP!aD$gai#Aar+EN8yn&eCYQG$sc(nu9(rLdhoZgAnEZ`lvy~Q9eg6x
zt496i#i~;EDz01!N;&Frt6EN!IbV$3ba_p!&R~2s36smsYE6kU7gm!HQ66hH07{67
zG8a_@G+96|Y@qe0xueYI6sPoX&s?lzNoBJ{WfPP)*iowGKT&250;vy?t?dUF9yzFz
z6lNtp0OLfEF}b=OlYEqxDlY!+>@G@Q=<7F_MUF$Ex=53ef2&ASRbJATSKf#+N6?mI
zkO^&tr~lAX#~)Ek;+sQNWeP)8Wim}%y{j;8+dQ=&U5^ZgL|K^(tSyO%+<Yg&4M^ns
zeXyf+NUDYOSu{_rDZcXQNP4=JM&4tf9Q3leo+y(ewB%G^RQ;R3RYW9*_+m6govNyc
zMs*~-`e@mlwHUUyRSBq-AxA8PZB<Ey$WzZonWdd*B^kbCJZpuQMhY3q2C02PW*OKP
zdP~&-Cm+c>f~ZBws^Q5fGeJDhN(HXHD;tF}WYUVAL8hG1(R+t7>IaP&%&5q0tm=dC
zrsMIHR<ufzwvQ*V_7<653x(=t*m9FGLe*a}d7b5@<ft#{Uow`5DAZq41TM{1-S?N3
z6=E8eZ*kQ7a;hxjOdwrGkEr*VwyH3n@0f(B_bH+&Pi~^eJ`7S?@E)@>(?5)Qlda*w
zIh50@cH3o^pid0mue;R@>9D1+J%XjT4L+&CnZ^9zD$=<IpEqRUc^UZTd=_=`N%Lzn
zV>s5vfv5uGFOj1psWn*$F#Z~3VUYe>5O*mMRrFUDVDp5pc80?fzq+rg-F2!`%%L9I
z=ZmTysN|)%1tNEs8%On$vS#K}J|1~nl;J4r9r}dwM0Jz4ayVpg|4BwvH~4;fS;*i4
z@*S<e2g)}sRp5i<yS{uI+bO`nW>H<V{0xM}DN&t^@HhQRRHq_j8uM>dN6B<(Q}Wq;
zI;tbBd1zA^8;89`aK_n*Q5{rxe;wYTyb34iz_6(HDr71l3Hed&%d1Oe32FILR67;2
zQirrFqe93vaC%f*$-Tzfs5bI<d|cGi@^|MuzMM=)M6ESZY2tk&D5|CWwRxjbi{wb1
z_MFiSHDWYw37f9u<fw1M9QKs&y-beQ^kG|^QR-mM1V22LjL*NHGXYI%JsX=dp_QbU
zh{YG=4EIMIs3Q(n5hbv0lbjy@z?nL*2dsRrz67>wm($iC_=*l}i=r-SJ+ka$|4vEo
zupCV)<lWYBni0o}lGCi2B0~yw>g}9b3LBW8pOaJ4!aUrmQ<aRjQcX?IDcQtN{I1NL
zU>FTO=5RS2(4uNia9tG$zb37#+}&&P(R_d94zk8S?{Z}}G7_~HI;(rj%B{q_PZwe9
z_j6^84MN6)r25;H>(j~b#sGcEq?pR_P)8(l_;MW>kGf_5R%hl*=2oswU|635qhmjS
zvC<4ixW~!On3bzb34AMuIl{d!fpiO^G=h$oh&C&FmC7}o^m`-FhSksE-V{$$KHkM$
z6A!Ga{tfOPx@SZ?K7PghqApnIUazv7yiud(h+YuZZb)S}*BGjAa(7n|B7wWID*K{`
zJHi-Kh!~GqNTSUzJ#P<0nHIhCyzMttI?6s`vplW=KnUF&G;%E8f0gEYht^;CkoFOK
z5E&UQ@N-t~o(x~s#xPQwxNVaK$CtF@Cp+Rnvz=as=h|vKRq>Qs7Br6Y`L?MgC7R&K
z{{kb~IF#$VGMqbd&OvawXtzy<I$}=6F;1-hZ_Jp%qe;}cmSK*BMRBy9xJY6iyW+NW
ziBy$j#G!%Wk^x3~0ajKm-H00sZ~z3H?&7v}rDI6XE3fi@M(1zDEhfT(@|j}>g1@)i
zZb6+Iac6PI=U}@<|4o*gP-0?%3ZQZpfJ44TBOGOI;m3WuMsS1NamqZZ2tA40mLgJ7
zBTN^vsqpPfLyRtWf45D|*B&k~=I$U)4-t}4tn(<+g6arqWcnNKXP*f<Hl^Gy!<%x^
zwlLuacIc2d+%|PzGDPx>0UKViuNqDBDx^i2&w0jeb3W0_FsJ1h@_WdX-+TipnntcQ
z%HhfgXl)xt4|6Hp^p3t+BOQ^=-X^2G6k8FotDvz`0e`tL6qaS3b(`NSFvZMr!OW$5
zfzauM<%dewga_F@z0CK~NR)HD+q_*EyCboyzYU}+`YP6neVf7>hoP2=HopwSlP)Ye
zmFG6!wz8iG&`?9|(UlZ8%<GaybHltLN#I7#A8zwqa*!&kL4&VRr!<2GuG2K;VYI4W
zSmHL{r`=?+u|i#`StEsrjl}aLA8l~0<ut1Bkn2nlx8PB3^Mn5%9`+{5Gm3{P@Nixa
z;i2?$Vfiz!PB4AHkK)OQVs7)ORg_AXAW%(ujn1RCrOZd^t3<eKWa0+PEkJwV_+55}
zs9$a!GEX_=0)z8}Gcc(!O*|*~ScF@k1ZqM{x4F~GqG><ysa}1IkINbkbfzr8?1oXK
zxZC9IS#$K)f`PVPV<m0{jjKp;xfEdxzOA^;vN%mb(oGW1@yh_iZSKQp9mD<<opwJl
z_xh9N<9W3(6=QKyW$RS?y$mnLboD2Di<lSu$!_BLkOgF&S+0TD6)EF3U&de>!&Or{
zqc4dh`|yfz3nE0C!_`2h=_<pVDwc|@ri;9u+F))bR{C-Hc809Aymmh1e#)GoScAZh
z`6@`-wwbNu1fe?@;M+Ia5zV=7YY;p3YpMnPU=FDRE*ha6#;b9^yh=9Q5-)jRt!6kf
zC-pZXkhIiaO>B#rak^QWBmFTkerwi~0#d2-AsLC<;Wo37!Lh1<RBFvs*5GTh=SbCR
z&CWnN%A)$T%Ou&Ngz%8t9K%+}v3F>?(~-)WeQ>>{9sl;Ihi;)JI<960Gn3P9V`9zU
zq2)2dOyT5-nIh97#{3S3Iq3swWc}*9O^iIOKpOeHt9*<TMKo@&E-rW1s@7DK@mbBW
zR&o3Ii~BslWmdcE>yNn>;A)HZd3P1<Yu#plXdk!5Py1ft$bnZyyY?`%JWYs4s%svT
zpxb<0#=RwlU@}M=6=#cP%K4eVnW`9|w~8@2)onIbRa{_XBbItPVmvE}N;%2f6=C@x
zyM=o%Mk}etF`}dZP`x$p`>u`U&X-zl`p$3;lZ-_~QmrD@ss}L9Dtt?;@U>uGt(G(r
zxhyIBzKA=&s*$vI8|HY93#$u9YIPx=Tn8i<j@0Up>nn{!Qq7fpKJu$lafz4d9lz6`
zY&bE8`jf%^TH~eSvYDUJ6<2Ac+YGl#JfEVOp;B*?3`#5tHoU4RSduNk&9q7!jzEiU
ztaS*tTkBEAD#l^Tmax%nmQ<_S5&=HddJpKezul&KyGDS2;C=@f*wk&hg*SQJL>`I@
z04VJs%HmD!W&oQ2EK|X4mbSQ98$+db8#SZiHo4<eQBr#Vz&_R0+dMpDuV?t0bJTMO
zN_)P#Ff2Hq=<sF&2X|~cF%W8x$Bk7pCB(TaT2g+EfRKJk6%D)0o8X7D7PN5QMEl{X
z%$wAnkDE&}rLt(63wqNC^dHf5wV0fl7%#PPo`}nFIpRS)Yj4D@_iMTZGo|YJRh3x2
zB&mi<M09IyB)j%*OVQOnhUW;W)I`vN9F;1ER4Dg~D7%T=pJVgx0#EHTxYw=n5z!8M
zi5L))7;cphKl4Hsw6J{I?=RmKu+V;Vd%q!KLql4?wwV{#qB7a5#aSISNii92v@(pK
zXWXXKW=YgMn4vYnqz`<5OmIYbKLXtcRB6tSZnK!b_Pzl4nWft1?N_R81`qL}+9u8R
zv#x+RKloYq51zmLnr+@mr5zTz&0x-^OnRF>bh<4?deYxgL<67>>L5{dilEE9({4-W
z%81j_1ES4rfhk%K442{=Z5hUKcF-z9b_PXAlSVPQO<5(Mzgps1hho$T<1RGSbZ3eH
zW%Em|KL{Z<m(8k{Tc<wJlB_1^t<xHxED9Le3Z<BUV@X7&C)T^KS((n+0O-7AI<+RY
zBaPx(X(03sb-Lp^d!!S2&thMl5rBqy`VR#5zMr-!z$STy8}1Jz7K4yFsHi$~ycFT7
zzuX`ClfDY@xK)HX_oN8&OA&5zFU#b={YAJ=v};xo>Y!2U+%mYMN_wI>e=Ft(yYC?A
zQ<$4mf?nq*;(TS*R=meB;@xa^BmpfWyPQlh@+&g?Idwc`fR!qqz9^o)NHvf6p?ESR
zm13`4XkuhnQM7#tZBu*_4^uUsW*k4(qAh+Vo*91HveSre1ebT;qt}>nUJ~><PlV@i
z3;o%w!josTMo#D?v%9F$RCj+Zvx(9_#$&?7Z?yWDhaO}qem`!n>K_rQ>_H+@zV9b<
zvN3TB*{a#ksXl6t5z8B8bRQ8<yN^ExjDgD=H7wXUU~hWHaeT>r{s{~AF0gB!LF0j4
zuwZo&x4JG7l-KsU4_kuLbT?V4f#Fo1=}|S()Vo`mXff^+Fn8*wQcy}=w-Qcn^A+7w
z7fYvZ4Uxj8GzD$^)Fev=Tua<rpTxeICQI$7+Pk+xxthY7_Eb?t_KpxmqP*Wg!S-pN
z%)>tSz`d-5SWOird260j7-G1e*Mdgy)IZ&sif9S-`sz(}M7aapYUc2;ByU>NJ+LtR
zvTx*6N0sPb+ym>8eLAGPamYQeseDV^9Y47H_~TA5?mIcvQJT1YD1kq&Jg=?e9v$oM
zSs1sY?_c7!|Ha)?x-biG&t!kx3tzaK7REgnqDs(|u3WujoYK?Xv?4KO*gNDGchlHH
z48F{1jtbsngGc0KPHJ)&Va~9tZdQqf2%`;V=J`aEgX885PZ7g-D<aVh1`Dr#$&_KD
z>7-9rM+s+`1jltsG$W`wJsmj1`8Y@qmkVRa?B3MUiC;1aZ_Zc*(x&4RzZ5g2Pw<4%
zMxs1hoj)XaLY9&E4PA8pU4(8Yls1G{Q%gf;;(Lh&iWD^UxedN{(-~z%C(6z%nUt9E
zJukjM;U)?jdP`&9lqlFk1m&jof-*!h+6*1}Lx}N)FLARlU!JK+jY`~x)aDOI7{(ci
zFKJcOlF6@oi45pf7T)n8!^e*_5+@T;@ix%AD{*ohlDGj3uu7)9^rM9B5~pDjdB+2H
z^-Y|iu<^aAbK+tJx60;SBpK)|eJavS@;bL6ajnz~+?|d(ux1+axw%7az4R`k4vZQl
zSd;Ub-YFFfV|DAqLDZP{2l6M2LCnTOt76CCst?&Qm_}o<BfUx?JJRV4{p=Wm1o2#<
zD&>%9`IajzCw5J&Y_VXbdZ96~atvf`E^4GncGXmgRK>}HM*dw@opHK4W!fUKMmgrQ
zhf<;~rxI(H=lk8@u|teR-upFY?gD(}TVE}&qk(s=E_AFew6I{%nZ&dwp@Uw5(x041
zf)lFrHWAhB6BA5%iEUO%C8`O$07YCastIPKUsPxQs^l_(mks=)I*S9l@V5}8M782W
zzsPFuc?o9)`3oRzIzHhn-I_eyIn&|tzF^f~|DzI3$zy`)5EdkSs8|5$r`IHWBvpEw
z?|DNKZu-BEyqciaU<5w;Y{I`mYKpozjEjfY?zst_End$$F0WxGbgnAAmWPO)e=j$8
zGdDiF^RQ5|!c$q;9V!)#r;(b>AJ|_)_9{kD#-*E=t0C;{VR%|pU2g6VlQjW62^*K2
zFAGsJ57Eux`+cS(I%Mv0^PozKeKxz7ui7l^Ai6rJdo2^)^V-nmCia194YS$yBv~aP
zfcbYpRB;xb7zxBPaQt$!9+c1iG)Oj1EjJqo+Pokd;BvDOtjr&xd1%PX;;#)=JX~&W
z3z{&72$M^Y@+&nrHYA43Pwb%h>qaa$BPsrz@;c*4m2s%T$aP0x^!*6EP3JVw;ekk6
z{#_#pTkn(Q{{)bII*GfBI#~XX4x^S2&RAX`NT(`zLA0dtwVCZGUDc-`Utkh|g_y3o
z%dbhyYYKj?iohw<={3IRXFFn_%(g4|DOD%Fi?bb3y>2WYPlD_*A*xcwm!(j{NCD@^
zlUk{eg|nBBR|=jdNh$gGhQM#b{OpcJWqEeFdbj?ttiNOr)Q~0c1365Y<8W4;xBMxo
zjoXBNp)+~vsfU?7MPaN=eADM3wr4a3kF+R_^F0lzr*NtVlE|~24wo`l+qq&PMVMPn
z%C=%5or+eXD;ClsmDZS2WyQjl;A<RA&8=WVGZhEY<hCmoqb5)MC|HgUS}_nTrRLrg
z1-xP)Dq5wT*kQ$>Xt3Y(#b!IA%0{n9m$ogp_<=QDdN7SOLt*u$XLHT+!4;X25<niK
z&#o9sf)n59{D!Igv{bPzT(T7jI>}6!@!akePg`<3uc;#9X*%cpA!=c1#nTPZ3wu-n
zn8?=!DEG&TcJOjuQ;=62yP~~hbU!%L_rzR>yFoIAvY%YR!%F77#R^?*g$~MV=dP$t
zOEGFFbn$+4&{|i~>gx*vZ@x-YvAWQpGpM98;JeuM717cx)uXe+%}Kq`LG%6$fxDA>
zp=oD-6(S94QZF);m$HoAlPC3}G6q}@&NL<s1wcwleD&vvSFIAym&tthyA!Wgk)}@;
z)x1L{zT@GE73rTA&YpN(<;Qoz#);Qqnd0u&W)rVRS>GjYOuWH9kNH!GQ~m9U@1nE9
z`dFOqg|uA#d51Z8%ot-Ls*Ch%WBEDG4she``V-e+Z7(pPpxTg$Yg$RBfkp)bjB9yo
zKw%6WKJi7gX52`yzw_V3_2ngj?}d4enuGgJ%tt^L6tVJJ06}lpmn@F`C0X>cy2J%h
zaL?e`t-3@GBH!|NCuU2&c{*RrYn88LJ2%zJ_PTxIe4SO?d`As${!Uc{pXtwQo1jrk
zl$>{1uubI<rE<t42iK;FT$E-mjL<m@k-`X{$swb~!-5S}Ip~57^MAwU;aH<a4dAX<
zCXNvH`)<v5JUIk=XJI8l?$39j$c0hEE%`2tAm5B36O$z09Xj76j46M<^_p7wwi`0B
z0n?TvdV!x(Cj0ZH>Kdx5^L1F@@Q4|>@Tg!Eqho!miTw*R)w_OND$^avdcPwRW0Cbm
zfgq$BYpFUxCoPnw9@lAN?ZS?A;UAAdv>D?Mm&LNvvnG}`;io*nxkx$%1hXvK*eSXc
z=9ObV+59QU^27?d70}ObD#$heBhhnD6k+13N3N|1rKm8Uj!6te?;@a=ALQDelmyb7
zo$}yOMq1ElBiAM`qjO<NmV6>Le9Gc*Zm_Mm3LwGu_k1%JI+CCLmhY~A<l17i1{ABn
zS1H$)puFOfz@#wA1~<?!HhwnYSy@qgw)e_NSvOmZQDj`au(Av*P>VNP1Qsm^aP(52
zYY~f*D_2&dM^Ue+muk7Pw!h`Qd17Tf7@(}j<_%XS!`#I?c!zMju`;=)`pz^~HuT4S
zs`ScM{_pl1SGJMZdx8fr23y$%(`@k`GOU%nvb#Ud3-?#{)7ZVJv1Tj#GikH<0FlCD
zR}Pc!BYa2JSvidTJd2M6%f^<KYTj4W>@&w#j&MR+Vz3w*D@P%ap|^mKskCyDKZBTr
zm52S`#h+PuvMf6qc#_Xt+=Ga5omR@+uNoxS&Be2Q4W4sU_2xbnq6NhtPg;1coX%f)
zgrB@}^3zp*wjd1$YL2mWgx5=ksxiqJTTew1wbP-(7#p*)gy1=H4G#@QNkq*FV{92{
zuqASa=(>H3Z8Q_~XdGQb*dT!OMvbwpmN>H0e@&?|wl%@BlK{{5>SJu%uvtoUrbtVk
z0a7w>jBP?iJlX1klQA~yo(aM8-0?BCB`QoAt%wTQs#aV=$~B}hKSNWz{AQk_ZFRk!
zwScy@<P0*ORmjkhzO3hvp(WB~x7?Gp7*l!4g|OhkM%MBu`BZv#b3c%zg@GhxRMu2M
z@(<BTI^CbNLxn8X0e}l~HP_i#Xb9^gQ^#}GXAO`A`K8@_-!I1eJ|~gV85pycT8$>P
zJ*zjd)E=Xy2h^jVS-mR@AVdy#W%aHth@|2Uoz0R#ZY}8^UYsZ2Tztg$<j#+L53s(a
ze=?fAla(l0FB6T@&5>0P;qh%>;z($in^m8wv1Nfu7k$H5NyWNg`V3=vR8}=~-?ADZ
z<BY6o@s#U~3<H8}4NaRMYwdF{b-1f#f%CaC4S(_rYSa9NKcQhlZ;37P#QlbE!>MH(
zX%AOV`T8!!F1p(Anh5V<$>XIq4c7{;vmjZU(r_oOW7*l@vAo#W@Fn#v{M)pin1>EE
zWFv^VtU$+H{z%LfI_4;5<&QRO%t^=1)4(4fvt&7A`wkpyG;ACSfVU5Wc}vyz>r!r_
zKFct0g3+*rqNGqR5xAUAWKiqZhTY2vDPjud3<9!cPQw9cB3BP(w&5F(ujhI-hWC8n
z3Xn@^bJih`?c-bl$0XG{%nebP*4Tf8lw^HMJ#<roE5JKk=2D)IBYR0O95tM?E=W|0
zD?krTLGgSU@Ex7w3K&aAyk16KwUhCn(3B^8tt()L=*ruKy#6*w{hn*BTmduvp7u~T
zOL^40vp43t0$z9$#Uz69471Z`TmikGEHKU@Qf$S6XSsh=(Ny)O&U4l=A=CobIA#pA
z^=>cWc!%KnmmdkJk5oL>TXz92n|`3$5LZB7nndou0M_DGH%D~$P@Y4C#&vQ9^pjg=
zQBX$R9k^|VgdZotS;C9G;R@)l$u;V-j!^diYdtHOqduQ)&~b+S>k3Fm>EzC1>TP8i
z!Y|<75Dyhn_W@2W{s{XQ6}XaSl>l`Ka#DX~5+7Z0ox!)y+q%F|#~+OFI<A1`AY9=B
z$G)jV$iNK}4+52dBqdA|58Y3~23($ax}%?|&cq<H3a<n9ee>}l^@(NC7yEJ*avNcd
zUC*$zuv{)W*!MEQG~&b<JkRdi%GX5PSV7SkVcU1AqLnmPrdu!aYm|1i3HB#lNMzRY
zOqFsao)wbsDLx=8@kL@YYj=^&=BW#Ocb{jM0JD9mn4K$<Wh!}`!EC(AVb+YmNk*~z
zo??HiM>Wo~^D4C)k$M+KvIW+Zg)lh5Z0vb9cPb>a3d2oy17`$IcofMk3(3a9NVdS*
zTngyN!n?sbmp6Qimpe)ot73V-@a+{Zd$)v!+@ij#c%SByF6FKI5ZiR9&j^@cq$r{<
zWD{cn30ggjF+*b5U?@aiVE=*K{g*q{7r%nOPY@!Utb6a935z(_uYR$DTmeIhG4r`^
z2^>6t+s}0IJ|7>i;ZBKXGanam=TV%b0*zcElKXX`Bx^xVWL`G$x@urrz%^K!e93bB
zzPPU*rqGA304f7nl$AeeCdqZc^{?;m6^<s}yv8xQ{g-W^n_c1;;JiadZ+_znkcA&<
zcuBPNBm-d24X%I+ejx22sdRaH0@xK$72J$bjigvy6=J%wTmdnDV0(Zq;gJW;w6+SQ
z+!h{=Wob%)Z7iVLPd`pE(=*91*TC#NwV&RC;g`g0pwJstB<*)rJdQaigh*p0Shco_
z<(F-e`+su<*iba8m&=<RstatZPl3InJh##Ki>pEf>B59+x=A|Zfy+M4pLA7G;3!V}
zG#H5D6jRA%pCy{;7H9(gaM`y=O4(wW_a+c3#Xg(%qNh%MrL?NG&tnHV$t7|0Vo^Gc
z#iT$*+Mh?8s8`7Iqwu&a&%)WhyDJ^emjANnoo6z!(*GB;JTm@BW??~SbUSmMWJ5?=
zU6*~Q-*aQGuR)%pR8(`9{beawIuV{D!U4Z$=>*@XJcql=Y?uAGkaJrz#ePS5h<%#a
zcOMe!dpVEYuK!9A;u^c`CnHFsrkk3&?5D(oRHzNjuQ*)x^g>=XQBzt#>I(Z{n`f`f
zJ|q#LtpuiS#LWyOUAk1@(vdEEi^7yI>k72DP#py+uQqr2n)}%75bM$>1Eoyy^<DOM
z{xXgH$7Sy+oMvse6&+-x%g!U)G=c)-bOD<U)w7+e>*%InuU1tUhse&u&4>|8nfOhV
zbzvm!2e>0<05dr*d+bx-q(@8o2=|TyTBxfZe5gSxGq!;Q0+Y&&Z6O}o1r5;7&MwtY
z)e9im-_>=Gy-*)R$s*3Q11@_5{)R$wJ@k9Mvg$fHmp$2^um-?t2!q;pEj8aziy5ny
z`#KaDXS__P(;uyL*_#mX!WW`$=sfFn#I+UAvwRG~4WJ_kTPZq+)>v;8ZiH%puQxaB
zdoayj_(K{}pj^~tyxyB-*8dE1wF?+`d7`oPQfroU0<W*NPCv>i6QAP+N2qt9%U+63
zzM|Mce@iF7W<e$srQZW?B8tTv8Z&MbY;-Fhd1D~dn=Jjiy|lC{1b{;mM!nVWB9*SP
zH0)M9*$q&p@`agC`FU~y;5j;VJL_?)FA(Y##+1>FG3uc!>s=Rh>Jipp^{@VpIFCj7
zgeZ5UjjjGqr1@d$((nf*UhxBiRq={9NFD03d+5K{q*;o*-gl&xW0-bk>y$RH_q&wl
zS4*AOr(dp5I_VCmpo*UxbO-!0%y8K&sqTRKsZWL153=f~J}jsoX;n>qX3FX#3at!d
zLsyr*xJdTx0;2-uH7*Hj*H-B%5?P!(&4jU{mbrJ_9)|o$3#^Yo)Q`1T7H+YuerJH~
zK>;o18adJ;WiDZHtbPWe>0&0?sPwgR3;AQg;|U(??|ceVL15*tU3Pii(<>B~KMDbh
z6@|MLg${<gLZL-VQ9muSK<A};R07oe+67jIfqr_H5@WGc&)PLs^;khl_dvNnCMqL8
zz-6nV7R}}O#Jbv{FvnIvAD8=wRy?+!{E7#ScytX<fcy*Qb;UJin^<yMZLh2FILOM(
zA45(~%(?8caTbN-e&lB%A@o+L(h|0_EFURFFz;iR?HpSD$}wu?N`E{UU+ezBlefoJ
z)OL<medU<1?^;K-geaHol2n~&c6tWeB}7YkhWz5PU6zpB>ULI_?T+#YRbAKn^4B`5
z6n(KWo^7RkKdxmRkfvfUC+y$!tX6NfKA6CwwCb<NanFF!CeCH+<3SJP1Gs=YU4Foz
zbokXzQ2^jq+_zLm?{{1__B<&)+JMH`pp>P&^zqKg3lNs#Oo4AqC35~Cmu<XP1T~$Q
zb#T>z);{mDP4@%40Bi$rSWlO2ksmk-;21%~RM+|~gl<l8+0=m%!Sf<+d3l*^ciEm7
z#UjA7xHFQZGf8Aa*XCrGEjNPDzD(jYxQ4re0^9PpVQ=vlQQC=8o1hfk)|%RETR%Wx
zGn75MwTr;SKOGd3@SgCGg!8S0=)0{Lkel7UkuS3G+X>!x2$s{-;1#2{zDgHsA-^jD
zlO;ec$#My#-)!8HMB1ICRf<-pmCKurEgf{)4NN+z!v(1iBo(br0{dZIVrFlsD~NrL
z^=!Nzt`Tjp3MWLgQgL%FL~?vevTw;zP`M9A1+sNUAzCDeboLmy8UEly7ShcRgIlX~
z=dE<F`vZJ=>m4E98;1EJ&;M+=C15%NGNL}Ac<Ur9!C(FanPzLrNE>%s={>}H-pE}w
zn#j_UR+o0`ek-tsKd`s-I9u}tEa;}FjKCyU(AoE0D=;>*Lj!pz;j*REk~cD#vhVu=
zF}Q{MNIakNaS!*U5VzT5W$>bvLF!d<e^^zTjLnEjv+iI!DtW50jYZ&Q>Pm%p-}%3S
z7|V!kY*dHK6@#q-!{*O5M>YjZ8XYo~g2;8+im3Zmc<jUQ87ds96d9mpOQxGrC0z5`
z3#G!DYzY$ZM8z4P(sOL==d$rQBW<R@So1l5Z;4{qJUK=;_qy?bu#g)3j{AvL)OD~+
zALj&^9El6J8eR)}W@&iIXtv~Nl=j*z0I|%|ZZ1l?`cqxDwpMdaZcUUl$X$y7h@fwh
zTlX#QjkO82YgbwIC=73rqzXs3uh_P^Tw!c3i4-&?t$=zD18U1A!nH|Y32{r5E{45%
ze+o9hSACPi<)ERq*y(Gv$q_aDqBh!XRZ5C}M&WPXUPN_kwrcd|TH$X#{y505@U0l4
z@ZY)SvK5uNl+C|{D)Z1*6yBlmWzviwv@6SZSIIGCU{5tnu$grwg71$_j&gOAyTX&s
z&l>jq-2GTyU9)EU+-0+eBAEO85;hE%$wouAExqJ2WiQi~t-f}f*~}B-GCvaqd|1eE
zyzVmh3mIE})8#T{JEkhvmg~9@=ApulYzt?uqzCtdtaACif^hXOE^~(<+}hGY$b6Va
zxK*}_#HEpZS0To&i55l+<i}`v1*2!B%UrH7PS^dexm+_MWee|I{^~NP7GXMQU~iXM
z8B1~NCG4^tB9z;|)ONR78N-EHJtNtR8?O$e6BpB(BP&MoKHRQSj7v|u%&PRPaU%ho
z!@U78VTa4~4?&X!#Rs~#GJaBTnoaH}yQ7N_GR$q)h|Ls`diYJ@{fpSYWA*LHD=@Yt
ziPgt1o41G(N&Mkd)SiQ!p$ow=S7P6@oez*SbRo~%QVh<|h<GxJH8br`NH-;?Dz+^q
z?ZQI2wbhr_nPDl&ZM1?TsLyS3ZI`7YwuyFT*t_jzQHH#3D%j6RYw@kx$|;0R7TryX
zYkoAW3JH~j?xsZiT+uD<9J-r8_x5BV6#;7q{~Fd4PiH<-am@uYgK15-QbzWa6puQO
zyltG={WPrY?DK4U1Ux;}y8yLoOM~3*(R6Pgp$n#9@tW@KOyEGb($?Emf9A{B<|ysG
zZ`j)O1J5y%NTW}adkl<i!TP<4?b3NDaFxQBD)ieJ#<p@~)=*{;->Ks=%ONt&PEx++
zA~HL&ERiW!ON&gyZWOGOB{JobEfz7p@ixq)GvC3{<z`s)Y``DxGChtufM!5Ov%8x3
z9X+K$TCWC{lXK|J9*9O(MM|OK)OC0D45QZyml-ZHzGHEy(xfIkT1AsW=DF4`vzU;H
z7;7_l=iHcfN+p=Zu=hl?8)oBXT67kxZ_&9H;3~-6IhWLFip+j4Gg!#nxm?I>BgEhd
z@Lk>Rh%A<&soojw`*AzdTa&EN7QV0@gf6y1Gki67ur|FxLUnzq;?Qknm=wi8jlGrN
z`b$c)bA5;^&0m#OVY(YeRvXu^lqGEt0A*9YR4+~y<yU4HRY?v(^!_-d1%GX5733{o
z=UOvs*jgo}TQ2JD>3ZPk1!$^qg`D<{nmH3T@-fhmxkj>TNdMgMb_wG4WEqi}&QSQD
zHd9Ig!KSd^5%`}x6~RB^SIl$$2x+oRWRZ45llCKBlp^g95dNx3`>{Jkm3H2Umf0xS
zqK&lB78vUK`Ek;yoySOP1V0+d9an*&4@v8(B;L1vr^D6js_P!4be7&!ofX~HSCewj
zs_#aPKu}+ka?h&nMsx{{I(ihtzYzsbm)mG4E`zG>ELGjjFn6`|@mgI;FCAJc@2k+V
zn!*SB6Elflf%U9yu8-&{c4ddCO8H1tidx5UeWWV`M%91Qbp+}*_CwRE!!&zN#i=gr
zh|<gCI0-jSQ+FiA;!_T=#lecYW5s+sUgC(|Za`Z+nPWy9uH(|zH^z_pZgepgLHX8o
z0uv$RTW+9WALuI%GQ3Y5BjkbW65Ypc*@FHG-CLs%a9n3^e<hrcFL9q6R)S3dlAw5i
z>#`&m>`(ABaSHHwB?WmJjs7)Qs@`*|z}V!JUFH0;R_1y~QgH^$+;b5QvSSd*0U`#T
zWqbsth|<}VObE?r_p6VDG>5IWz?q_MH=(vf<_c345atM*)pO>#2?jZ%5Q)S_UG>Yp
zH`fhv-ow6OTU*+;tid&YLOfUbKsp=qFqy6G89v}s<Ct=I?#a)6$!nTnf9D@x=x%O;
zyrjw6Grcft``EjxyBMzwq>ZAbpnK2564}Ro<&Pl9LLafJQuiY+MJF1$X5fEr)yhYF
z=2AOr_q?xL`Bts6kdVYBuH{d%GkWdkFjBpRu=l<e;N=OrEj4Zq>5a6AEN`Mkr13yt
zeL3Fj;(0+n`R~O|)Dn^l8x(94f7ViRBxI#uLfBL-60)whtU$=G5$Ni>0WR+KTD9Kz
z9g<zO#H8_EJh%N~vTiu>jgXD5jgnx$Kf!mz`Np5%PdpE#ve!vNf@yo-^{v_CDDB8q
zUFpd^4)?HnuAz!X%AF5Fb(C#iYhCW4R=HE4I3*x1E0Y<jp(1;@l(M)e_`UY(Bm8cG
z_NM(WyzJ^B^8)*Fb(8E-O&I8Fve!}CjPmu_>nPbepT!n19hO)Zb*wfh7`44Mudb_&
z^c%85&m@4{(Tmkh@3!_&&r2lOUzLDM$2|Lkqz-)fdmU8<?JrfE$+90kuST-c@Qr#P
zH0gbh+{z?(mk7n1OL}}l)c$6q4(;vPq}2@}&kpr)V?QBtR8#lp#bjCiU3}i7R`U+?
zz2hZcn|&B>t3a189q-Gh;zToe<gmBstrT=xO6|&qVSm=sjz6-IB4q%scPzIJ&GmGN
zV58r6s0L3~{}F~qb})qH6Fdhu^$`8>{J%ZV2vLV+^-=)4e)K%!@dJ=CPv2q;^%VOh
z;Za5)Eglda2r>0Na;bnvt>#I_pW*HpS9yf`a7UtXtuGz^5l=@4^3Z}-CKCcL4=HSA
z)`+gf=T^wWB`AhyB15TFkRen_$(PqWmWci-Rp|=f?EP%Kq4teI!kq&%G<TT0RorPp
zc~YcLg*(X>cbYI=)TES0c6fUlS<R;jrdbo2oGY>mB+q!{K@m@LCJ2R{O-A8HAbMSU
zds>7Nz6?JcX)>KRSdA9r9A5j61<lihKyAQ`Bb^kO1|Wm51LGbG;?kKGIY4s_dDZB^
zGR#9GPky8ZC;?g_ri-U6sGFw&{Z7htsHiSi*>Vz@tXJTc=yH{<3N4DxLjd>cYOnBk
zwO6vLy~%Z=T$W0$VwGIB<2Ct@!i_BLsqTkWVS}!WG@D!Xe6X&T^8~tZB%7h!!Im15
z7!JhyjZF2*q&xr2gZ&Aj86-)NjyzUJPNGEufi=ysm_!y1E_tk4C1kM`vi)I*mWAfG
zv@9Gv{V+tUXK(ZGtRT4%(CTqWgraCcH>Dm15)=0g=3SU*`0;1?$Osb1LIo~u$jV3P
z+(_FhrI)CeL$a<>GBDD%`Y9TaER$U59%)-s63<pV%@QMRYlU61`ci!1Y<zHB`Nk~p
z)jWvda6QsCl8q6Eb`WFCf=HWOWnvr-8f-+`*oCg+j(sB1wnRL6GMtkfywI|4tD<Li
z|43WsLVE55jm;cF+Go%|?6}J0vzMgg7MekP4^k*)6u7Hmq%GVu*|YviZ;<YMEz+jW
z%E=yrymgVbvMLoG_q&m{%1<zk8|(Y|prdN_HtJHZNL#GPeV&?ZzH}hcR*%kVT_2{`
z*(&x~r1`BVtyk9j(hfN)o0WW7ha69NE`x1S=SWk{m8znwI2mcaqwyFQXoMNXeJ>wk
zXUO4-<bK^AX&y($;^2=I+B{j>Fb`M6Vw2qz{g1-<p?Y{5X`aT&I!s)Y)7tO&iOYnJ
zr|U+VJB5zJi$O;@OtzK46+BdoxwC@s1<z|U@L-cF50zwkcP5F?%66B^wz|$%KM^u{
zw~=fZms>@eYoHpXXCA&2I+i0>0HQ?kTjZ}nqT}RzZo)=FNMamTN73;}qdjbuMb0B!
zZY3yRs}N}}F-1ned8Cqsa|x8vncRHEk;IguHAi&hHsuu_O^%jKXp4<-2{D7lj)RQJ
z<SgcN(+pfcVvec?Q0ot)>2iRJN@=c6SH%Nx3U`D?dqmH_M4Id=5vJ1Iy58dnrs@Qp
z7_6I)%<gmx;N`sj^1<@K(JY!#H|pi6)$Y_(NvpPz3}PfL3xw&Am4V!b^$A6~%?#R-
z(b!@~2KvZoTv)Zf-w!*=hh$ladMTFrqK`OYGsZ{C-UTo&wo<mi>W5in7#62SnvEpr
zw3V(=q{>VV;Sz%KD$JqVO^GhD`fRO8-bs^G-&m<?!hM*1{0Kcpe;8NK8)<SHTyyh7
zcyt|LOaj0dFdhc*0QWP%=-vPmG_V<1nuS$moH8@gEC(UA8qvGLqw+bJR&s1#Zi18H
z=%!PVW=Yg|7qRWs5boH<7~xgP2+`Pi;DI^Ks?eErrH6W@8foRywI}`)v#~KtlW#;`
zkP;j_OMUB|nPMNbI~n=4G%aG#@l}}~d0Ix0$7Cb&Q=22-Kqy&+8pSOdX^}6I{dT$|
z`(5cpUNz|9ihpdF$Gv%;F=LI$y@KX2G@!eiah1KXv$<h!dV&biQjWlQ5s{sAv<k*W
zCcx?{*kY_`CMS+RV;0F=5inFgsHr9@YN_zGVxJK{nlz$%TjdIPSFDA~J6GYVDJptb
zk^q$-Yb#2~R77Js6rU3*rs;{5h(hr>e7y?6%?9B5h{wgpWZZc1l;>j^?m6)YUpM16
zGQMg2i$doB#hOTYHs=ImazMM~QZu1ydp=zbv&JW$f(VhBOg;O#UCa`J%e=pa>q(gV
z7F;I(72FmmZ(SNY0;~YBaIAnh70^br<AhB73p>1NpOcO1snqO0ZEXj03b0JZSFf&C
zUcs0_1$BrNbNgh56l^<%+Kb!$6rO|j=kFTw@tXblTk7L2`}613$NOq9$Q?%t2l7~a
zPh=y(lE|1p^B?4|*>Rj)lbQJwALnsr#q$Lp@8Nhf+svd<G(&isagfl=yv)aa+|Q~+
zSo3T0z}JNmk(cUNspe3r(H&bJvCdne7^u`kMS&kRq!vy_?)E-mn5X7St)7JbL!~UT
z4@H9tDi76Rnkg7ZeDhEWAMJ3d61A-qRiHF5tO%p|aN%7(fE~gw=JZ6Jos<Oti7AyS
zODv`m<sA=1iI8=a+Ulanwk*w^ffVR{f}(vtouH#)q$&b@s-ab5t&&7D?%XY!k&Ir7
zw=nfr8J(AmY{(*UUdoWRAs?!dz>k-9Sp1N@m_CtkaACBlUp9e7p5$^4HHK}zjWTVq
z$KqoU+x!5Iah4jyHb0B!%wvPtmykFHu?J5&sy&G=aD5#)S#-gxNxp9_(6G43o(i<u
zSMii1rNY(7p`suT2e(a|Vnk-CZ&4$G1(Eesyw81ePdVbfxsl1#5!*)PYOAZ`BWq9{
zMq~SYe9O*9#wcG?mM$Jd*qAOeUvr0w&5S2pDO9-(s84=``Jj?vEZP-e?#8dS1idB`
z{*6*2%)PW;WdL+27h&#Go}j^5MufS)EE|Kp{{@Y!iRWcRM<to`O@w(ssLdmNovU~v
zEzi7&2=gEzVfln~sDKB#3d^tVw<e^$P0km4;@Gwb^NOgvJOqz>gx`U<YS|H{7vjSD
zP>Ot6gss^Fh_{r0@a=dh!kh_3&bxT748wzLD$nb8@T7@{o$wLnY(=Ts2_Iq3QAQeB
zrrIO)Jt4ABDZe3*g|j2fMKlA{w}3iWAk>zCgiMLy<q;t98*L1u-{lBX7S{8(1dSUo
zoUEFVRsI%YHlH71Hia&|sj1()5oU`pixIA0Bg|CH_3CJU$~tlyh&BgAm>fJb^S{-V
zWmXnW^-<(kD@RGLfb-I%2s6GM3c`8CVxfL_Z}JCntzJlcgo&7tsz5!JX6%-Z<@DT4
zuook!ilj4#@@LvXya^H$I&b6T=)dqbObws`{EPcd-R2NshFicE>71$?P(H&uXbc7A
z`9E+2nSzsB$7~;z&rl(*s#Zu2AsAUwo$Kg6Hdc!GkrJLhPiI(jLc~uM*wd_bQU4xs
zUl=80>f{R%zpC#rRdPuyL}H4CejV|-r2amnstgY!KBtxAk(XH`J{Kb|T_UF!kkR;M
z5pNOo^&sD^e9rW}9<MVz;?J<g{D|XxzaB3shRx{A(DL7iMHc7Q1cHmJWe8`lqA@x*
zVv$&ehw1+j&QTG|1dEq%$GC_b!EzB%d403GgD=9T=w)kp#43^H#$mzZ29$)rwJZfM
z$)+qi+Ex|KxAZiRE&~-ZhTg&Y-Ci+b8ki}u`G!Y`?h!Eq&eoculX>;GYRog$H(9?C
zfzerPa>NYmH3o2D_{w}AA#DVo%#mEE5HSpHyfGj|B^*ZE4C@2(n1qN?diDShk~KzE
za}C`z#a6peLC&z8>>SZWDF{$()F`?rdlEj;8oG!*RwI~J)(vy5#HSi@PgRGKd-3%V
zH6-_6b<!G6$z7t?DkspQzD=)FMUx_$DzDC+t3o89uqAg2I4UlQC<Tr)2^vQ!nvTXH
zC!tET)8u;|B|q$ENhP-Zh?r!Px74J)nrB9+r8fXoWnw94z<{wvvN<f2Ws@@_7_}vv
z`W%R4yuwc6t#gx2tQnp7sFk>nN=yOcTM!Dn)d(hkns4SC4riMYl4;9x$!2FO-J3yH
zI?lF|sif;is6XA+ws;x|5{sPf%wn7wL1X9o!@F-dN)~-nmhOgL_5Jq-7pBpx8ft6(
zn;kP{TYRz^ja_nYi8)jh(kpKcYb-|ToBM-D49GNwH7TvW`DzOe=3qyR8e|M>2dm$F
zg;phIYqy(XwvMVatb;(0^L^;_u#O0CSbnhB@WVQZARksTy2zA+9M6rBpF@ai$(DKZ
ztx}$E6f$9~kvG4hVV-Y<WK`dlw}3ACd^5g1GxHW8CD~>8J~=UOK|SDUd<O^QEkqm!
zT;@BuRNlf$R-7?I^A^^XIEYE!b58)xfvo7Od5epUk|=mqP0m|FsyV;%z3p(`Qi&AK
z_a<B3GW8uie4vph$@JmLt$8c_-|koQa{bZWZ{+1dflBH=pO;(4`cAx%ms>@?eY4)g
z-e^nOPG%mB+dMDdO@`;K@}pR~DDMUJO;bGk7cD<;J^io3XfKtwL48YSaxO^1w%Uav
zb%<fFGw&q{6(V~*nYYWIs_EIh-Kw%Ak!XV#z0LCuA>3*^?YR*^j*Q6@V_VeAfXs15
z-s@C|D(AJ%d2fUZPO6W1crVjd#4E^_Aynsr^Gq=VvcnbVeLw*(<(cIaIy?y*@t`5p
zI6QxCo>@NB=mdr^nn#|QtO(<s4=P4^1hwiH7c|T>n<^yabUr}EZuw|z@hG~;<WPq1
z-dRU=uO#Ll)Mu}hBFvC4xn3~aM7q+l^G5omYXz14_U62iOq`sT?+b17MzUI^#B1T~
zypcUgEtA6o4&-Hp@jXEy?cIj+X)Tj%F*J$ua@_5@J;@fQ%o*v+y`w56+3FznDy;FP
zBwJlHy*e*G`fQRd0YWOw*K-|7w)ikZ&2<Rk`7dByt$Q-b{2jy>WWwmlPw{vN`5`E)
z|72FX{zmvOOb;4MPnBf;LpO0@Hk*-|t4K2c14CFJ$v?ASl1=a_UtD66%~WlTpp73T
z*=))KJ1-0e(zSPzO`iX`u+Ruo$FP!YF6ozO^@3tSH=*m7Z%t(M!hFg-{q-bs7vx>E
z1xc@*WbS9<^F<q&Ua}{duc@k$`LWloCz<);6OR<*H2j(WgQXoVx}^LqbCb+Bg&LWr
zf@$V?#Wdfw^Bm<lj5Nt3Sr=U>W1~j6QI~Jo_%V1((m$e%F7^%@IA8=6*ugw+BchRH
zKB4jl+n5D-5Pf7hNa6&SmjqfS{sBqRf0Ay>*{O@qhrqj=Wu;f6o?nek`a}_3)XRk#
z=}Dg|k1%6<=Okt)4f8EW@W25XMv_>scil7T!Om?+x&e$_4k|)uk4Z-q9Uz2`BZ>^Z
zCtOH6L-oDo;CuO^q_^bziLhW29#hl-zA-oHxT+pz-(mc`V*JdVAS%&FT0vfSB~6m1
zV5Opf?{p(+RY`$MlDRjCUFU-&=9#H~pm`yR-fX1kt~+F0e*`DHy3`9M#BjD?qX8<+
z*Oba!8gS@eP>cyc(F3Yni!cG^ZKS^F&tFO!>8IlhU-m^u%@J-Am;F3xyrx?sU09mb
zn@E`4<|R*v%oQc|MqjAQg<(i<YzX>{EF-B88t&uq!Ez`dsjp@=BDSly&nB13Tv|#+
zUbp*fRVb+%Htdh_m4A!P@UpS!l3d0R*U4u~U}Tu0o?*W<<8hE~V}&u~#^<zeeP&6a
zO^oYHd*qT*0GOQI4$+bQmt|sKRH24>*@-@wFfR5*I1zu5GLAkQy9GIEa5mU0*PFy{
zRdpPkF@As%yOSnUTTbo9w}^dJ!9`EbaK^qYToDbDRyXzlhD`iL!)QxB39-UPxv2Vu
z0D~i*Vi#Un8mEwrEJqDa#m3!*+ge#69a)vw^p-5B7%jYKS;>F`HIhR%TYQiEiUliV
zF*c{bSRFh}uA++_Uz!fE0=qapa#lBXg66Q13YSuC)pb9upMIPCIm3%<4-_>vEdGA6
zSv2^!SxPnEDNk%{fz*=Xp`58zXd9^>@qsWEvm`YMqcNl(L-lv6>%sgx^?)*dPi@0c
z_ys2ZYxk4EvU>W@tVvNMyY2*SU_XKpGAVYFWH%R_*JhIHSzLGVU?`_f!c7oQ1|M^9
zv&3EvnM8Ki7VGS;tpdChw^*_}^H_Fz6Kj+q4^tYencO68cI+H>qrJ_Af$B}l)E7wV
zBkAgR0H4V02ARs*v)sP_LV;}a;TQ8gmcw}F6)$JK)b0DJTS>UNBAz8G24i~_?~yy#
z@U!%4K@0C((|)#|%bjwm`*C|gFOw`UEPw>et1B`I!7V?4Rx?fUqR7WHLRsKUvxc>G
z@^ckhQqNjknMkONdz()ZR*zRpA*)%7y|UET=WR|*<dA{7rSWQ}mb1YaMj~gar*Lvh
z6~a3>j#$W9>Mh(=EoMWMK-2tbtu2d4&An7$tjdvfzL1WwL#Vl{?WBIL6O6>Qbc^R4
zA2GOS@tokpi!s`=zSLYtsku8=&3y~BGp-}-Uv=ZLnp1+pMCmH#-qbc!RXQf0GI>xb
zQ?;Xaj>Pt`+R<t$-y~^A*ShQS^+0l^d@cLodV6&Ndq9*Fu%!fy)$QaNsMhU7H7mWu
zLhxGLDoavV<AL_&vArnowT4ovnFJri?GX=>+VU#yl6a8RmY?J9N^zHKLG`LxE|Juh
zf8c%>abrdMk%plqRp9p?E$}L-&aq@(4RgIpY?>-B+UwfZ5T!2CDo8O=R#aaaaX4D6
zT7OzAiNk@|(MD|R(ndEU{$F)TQfx<EpL7<<6^LJUZfw0moDVe4dP+1j&Uzvea-Yb_
z-(%|~$+yPY0ItRV>$`B(@su}JCyPN>`bmk2eV8mJ;bF2^l}uAoM^ge3GlFBw_=$LD
z8kq%lj4e|Rr8<Le^r)Ji&(j%~Qy7FDqhF0JE~&ht&EGkold(TuCqrQuPZ79I7N(O?
z>;`AMaNv1#gMb%QmYgik0*M-%U+>j3%OIe<;82fORDT;ULc?THCW<q#V}k%L+Ay#8
z!t>Uc1_A5odcD{Csdcml0UM=6hdGCq4wg+pmJOD7xk12YRo24M5k1)Tr&V_azA|_Q
zD|<Jt6Byaxe|@O3Rc2p;+*ud+sXbYKLg>N{^=uF@&CjbFl|rF6E?}A>ICN0v81kJ~
z1tJ=Uaw=LpsNzxk$Xp6Cy`)g&d;T5tfxiH=DFE~j97yD4{}+)r#IzHtW@#d!`$mKX
znym>HyKyv$=P?;^1HpmNx@{r!;E@Iai4Y1S3HS=s6UG6Nc6hNEGuB;4;Y*Qlf~tPm
zARq}7@5WakpLffbdYuy^eiBm^i5Bk~?+RjnNP~b50B?LJ^M?%ry2$rWx(wY^8H7$5
zg$U^_9_MfBL;fa2LTvqi<Yyk1N9N)Nwp9@OZa=j<*W19h4u+}2hx_g|u&FA{Q;&Bx
zuyJt0d^eqJ-jx|Xva!8bXo$QkGklMLv>{R$gGXkVMhHNWyxWglw_o7Vc**sI$~D7m
zsWOe=a*uacDMY+0^$_>2$3Yr*VGOZi=I+h7t^~v|-)k@TUu9fZ7EV!!nv?N9HspKl
z`EKLQ_>i_Tvt{sDBjckoG^4M9u2`O-uGwPN%xhe^nISU^7==>B86Okf`$O=Uv7o#y
z-@lL`@x6>YDh}UQzRkF&*e~W!T&E0fqx4_B=6xXZ!?Wu+Z3mIaDjl7+tzu8Vw^tR~
zY1?5H+I;UhRY0e0XUD=qzxSdara`*UMA2SZ`<v3qX;YhP-mk1CM)1*URo%t6aN2lR
z%zVGH@6fyK@gC%~&G0u(*N0Bq>{6VmI%XL}`lQViWlUF20P#{t?j@wYFDG-Vzl~3|
zA}A8X&nm=nJG9d#lgA7eS~34!+-b@%Tr9lf9wzllO3!qsBrP-E+k!9*k@qWO7CwQq
zkw0{pyA>S(YT{zcXN??VIL$qRlMXAbC9YXj=1>20ntPe2zf;ow{DJ@gOZPfWY2Y<`
z(x;Y}R_`?T`%|IW(w+hD#bW?N6e_Qgb{*)o@^ae{P-ZGr2Dbs;DnBk=V5Cs6Z-IR!
zmpVK3J!g##qZI{D`&Z53I?WSSKCK|Rl{_=f+>f>WdrBa-4TEO$qj{v*(bQ=^coM9)
zi+B;ezE1OB<s+kBZ#&KZAWh~KThYi{bptr;J8qQ~24eBGT7r|A$2N_fwkQ3-O90OR
z%pK{ph4_L00{j8<>AQOc#woPC{xwQEZB-D(4@b~DfA2*OTT_tM(u;rhD<5rett)c8
za;J*VRzpc^ggVQxL}Oq5Ho@I-auxaN*ZCNP8z3I)r8OLGJ(6DgR6hV}Gs<MO1jV8v
zdmK~1gAI3bfnmRt0|hyzS(jC+1p(R~ZerNps$d=eJei9pjW}Dx&SODY(w#Ky8%_Yb
z;am2BqjJ5ghOJRgyx(gZPgc3qsf|-CM9bRMN8nXfVB3C@{M3HK{2~=_b>EL4u%+z0
z6|1=wtDVZ^RVY`95p$h*fh2x4+X9dAgAd6{->Yh!*AKddK*Fi-4fAFVs(HmnPFpLf
zjC4WpxrM;L4ZUqUj@rl)s6vlfCoK6$cBB8lIr18_|H6@v3W#EH<h9b8BR5O=ZhlCS
zY6?dl<%VHhmVEq@*zNq%X*<o=G2Cx*v8%G%TVwpSo^09e#K3M>@gA|;@ng4J&_cUi
z`z^b@^*pesd3Jx6C;Ca2{HiS9CCghS{8>Iv<YiWt#2`x*?~yF=lcfbM%u@UPS*``k
zYS5FTlJOJXqax%d_(jOgPtb?AHj9vpjBm#=GpJ?!5S{~;j1wIhSBV~#aoFAZlwZc-
zTkDH{8UFy!d({02##UO!C#u!YafWZwO~;d7X>AW=Xcgvkn)LMM$JP8qd`8kw{6rMs
z`Jpv9q@g{Kv0qrZM;h5*1aql|YGhAi%{^VjBH(E!o^WYofBOlTWDy{(tcSmu9UpF(
z-`pg7PQLk%e2pifJj{qd9$#RXU!NjawkWVL`%rAy$u}fmqLETwMcZISTjlG0iwjbB
z=pot1yGW(a9Uebv<@476AhOPyoWLGYbg*;kv5dHeLvlC~t8=&=^#2i_q_ezTj&QI2
ziRr4voamsj-cI@7RLHYnou&thxBC#zRo(AVJ{dDjQLX%hr6O)mB~-v|gN1jKbW#+B
zaOU==|3Pi0nZ8LMJA#Yd)0*V=72lGN9Ujju!`8G3@qc*8LloW+n?7;&D&qM?s>Ebf
zQ6}=fp}&$dVMdKAvh8lue5WZ}mEM=7$vdOzMCeM*=}iR^G;Tx|-3|I)Ji0rZsk#~6
zy_k)63H6ijORqn@rPCZIYuI;|SV+dPC7qs?Oa|X`nq5q<KbQB2AiZoCf+`;i)xB$%
z8qg$Dl#)OF4oP(`{!Xll?CM$8+i51zvS;NGTOG9jB#)k4uq7#MNM}{tQ!yHAlEMlg
z0~_h--sNR6(P`FKg#fx4x5nx~)pcZaBTrwzz4>_FB`r-h>u*sj{3^B50=07YbxW;0
zO{e|zElaHs1GS>!Jya`R0iahbaG_q&eoL=B{WWO50>6mStn*s<?!K?Z=)MBp(z18=
zcM%Z(o>vi|i8>L@Uj*nA*Z+jqup%(bP_r1H$j%yslS7qSI7<rp*;a%2q^GakT}PBB
zS?Bf%a~Y52W~RtFq1iO^ueo>XF2k58r&$%nqbBq|c||7l08~^HdaVp&&vGZiA;AU4
zOq7T@P3*AGvSU;{gAof8f5y|W_G#JlwW$ZurSdQj+Qj3cwe3?>s!m{Z_HBk&W$`4#
zX$D3zr|FfKPO(D5);ZM$_hI?O$i~eCSha*xO-U=@8h}3f${@P)Pv0kW{aL3ew=Kxl
zgf<6oJh5V59Y+NO3&}y}oSb?hZoVkN&V--AQStWGn1XHI!LeiCYhT@fua9vz(T}s7
zhTi-7sCJVYke+*1IVrkuRPTbJ>B}bGnRE>Xs^v4e{8NZ>`-#FJYxAW=)XWbQQTv=`
zS%~_qi$zq`Ht?>^pB7bZpI}-hP&^RTHUd|Id}m?@{5!)}=2LdaUV5Aap^OThnPGvg
zr?Lwtcv^hrl&(ywZ^m7!FJ7we^Y7!N`o6EBYp-oI`IM=wYVZ3*iHyGkF0Cx}GSvB_
zYS#EC&>Gv0@hFud7n@Z+4{!-?p<J-<{MoAHwwnNLkeV7nUS&WM*U9;#h}0ee_ThHP
z{N;D+3k-Jd+K4yVd7qB=^J~)0y+Qa{92I4Lca)DixDUm%n-A|_{QS<eP|##je0PPN
zeW=HFA>z5oM=cyL38@Ye<S3?~4bCclE3{oQ!S#i`SfT9@?{+Q4gZgOK71tSFd&>b(
zR;cuQ>}r!Hs=7y!0I!!?{6d{)o}_v|>Bhx`lwo7{eCK-(#Xh3Qu+jNmb%XX%RB`d9
z;-Xx&W;j1q%maSqTc=ttVM+)FexVqshknkReulB6mF&zAn~+NMPK(8+J0!LEwcPSc
z^<<EV429Ya!1dvSMwn4wS7W;=z$O5z8RN`oX~A-V<p5*clPNXa2S2=RC%1V{n(xeH
zV#qk`Q95^+P$cZRihG^N-o*Eu!}Ku7Xm1B7+u$4ZaLVoliP+wSizq6KKh9wU8Hf3a
zF6JDDjH&QCgjWL=_O^4FrRCexHruxcMt?AzCRF(t{t38oTGU1e3{h)8oBZ3)^rUBw
zHJl>_h^(|<hI@|06FWFZ3G7w;TX35^sxPs2YZ>j4yY_q3M80#rRjT$UfLY^K_BN=l
zzJoPuJtJ5841T50Sj8gaSIpSkcJ-S{l5@B>tv-Vo^cgDN!#+d&bQu=3u*=YXtIKGQ
z0%{LimG+;Oul4m;&3whb<TT_B-5Go}Kzp$rq;L3YzOU_<j?!f~ohnn@U#(R=fwpa?
zhguj;`bK=^aB-QKPV%X5`<ITVIDDz>se3kGyL*nvC#GlX;CmG;Kcy9W>t0jKk2$0K
zoKP2RXRPS6wHNYK=q8<)Mc%#FwU?eug?{SWBWa!5`?bw?S-dqhs;^^w-+$#OTUrdY
zPXMXudbJOBMjI=}Kwp)6klx!$KSxM&6&q)TZuX7D>)kKCqNwD+7yKO}s;j$UyczB6
z&pE<!8{lwhX)oX09H&G~j#^YcW)6*0^3flapak;{)p0XUwlF#Sx3FNp0sFbHY6<<@
z3CtXVr1?ra2U-%;K{j4=D26us_EVJ?+RS#`IZ(<x3Z3=se*`zi$x!**U-)Q^OGAIz
zj*H4-7NtW^94&xl^Y2P39N@NHa1KIgncp=eZ~<<fcu;N~e7NVugWc5O98*jkkVN&^
zS~<EQ((`KsI6H{SJLo13V%9-UrpY^N9q8sdpeAJrEwi?>mnfCPq#*}GO|X3Xspe5H
zG}=QR^%8AD<L*Fz*5OOdqh8Gv%xiE9s?p&e&6Z~^wsa)pj!r*USdpPbYn{)+m2c+=
zBj8F$7|}6JJaDBW+P7n8;mTTzD;);`K`{weS}O{LE4!S1ILvN-H-tbK(eZgZ-R^h2
z_@YU6RE*&5v~sJY>GCU#cv?=s%d<coZ{V)d_~f;<jz8gkBm%nbl^afVVnj~3eOJ<V
z?Hfnw3Z<Ru>Gs32%hNN;*-k9v?^>fDQBm!b{P>E0%e?(~ry8uQq~mmOE5$nX7>2Wz
zW*kV?PjEI?NLGg^_0_mG2w9kt9-wgp2+5OaUr$w{mTv2;prYQ>Q7a@!RMnqU#5NzX
znN+Cwilp-G`j$;vq-hxDEG=oi57B8#6H%dW{fx7;pih?<GSAP6-a&22KEz#%0e&FO
z;s+YSz)T~02O3(=A$Ov(I{sN%?i+~SQG-}ie5V*(1&`bo6urx;<4!b)POYn>l4ZM+
z9#c%|hdH!Mlv-(Mk^Z83jcB!mD@gj|<W<^TH`wz17ZLVKyH(w+Z9;%_ue<IU4_B&M
z-DfGAQ*rAD5P%_I1Vry4(I<5PalxH^{V1#mLcImXZnPcq<^|Em89M(EM~$zaMrOlt
zL&Y<S4*^UekKM_S=5fZaH2Nt$yz7uA?FV%-%_2?Ne!Dvu_P5orSK9^g((l)9LUi`!
zQ{}Mtpb?!58AcwNAil;bVegOAG<$PtuZq3PpkuyfZ*Cm1gflPW_GtF1Yfc4r0cVY8
zOnuH~$~Zqeb}t}dK=ca~_J=xNDXr!Q2UUezyLU?i;Fne%{UR*+v9X^eH^GWArCGw{
zWT&r%CHenPmW)#@*`2Lf@}pWkRbzf-T}nEt>+R9LCO@#HKrsP6m-I$IN9LWS!fGl8
zysjDWTvdw!f0OHPr0#rGb2Jm96kf&)FBO&~j$wQeEwhX;%q*xHk~wIs5$&bL@GVF8
zqrIp_*=Z<pJFBF`^F|OV_xQ$jihg*bVS((ecnHy!l|Nh>Tv!@*+w<3={TrGJ2I|a4
z)>4_N3`QaRp|^C#qv}e=V(j(45Y2NZn!;BD(ZdTnM-OsR=sZnUx*|)cTBm7Xv!k`m
zqF~=+5Q{1++@SPP*P~@sR!aX<C0cvCbJ0B*3m9^d{I+}1vM%e-<{2v+ulZAe#@eee
zpPwQhgIM`k;d0oeZ$Wefb@f^@s#Tek_k&{%<IM5sk|hoEelVjPTJW?3(QfQ28EADb
zhjRkkRw_D5)IekWeQ}*VUb(X}y0jnoBEUQ%PWUFeG-_Q%ya?=7VC~mMmj;o*y6~s-
z17KqoM3=5%!C-fna_E&u)XIY9uszl=e>tQsAc$rJrPI^Tcj+jB-tL5YJ#tY%bcJ$?
zcSe`7gpLCHh4-VYhgm6TIbE`WrPYm&^T%5aY!$G@=c5}*JepsZt+>s=l3GPK@x%55
z+v_2R)KFrj<2!*nrt+U}m_KhMe(m$oTo$53-y-w^l71Q^r3Gj2{K{@7;WSmbXYVv)
zGW9D>waSwq`Ha>nS0Q$J566iTbroWlPw?EK^V%n6`VOa#gyt_c)Fp^N|6WV5qvEC%
z82Ox$kXsOcaVoRO<JWL%PC8%B{}CAP`|l^rr$M@L|02h7ELA*RHT%=dg^7~>CF5bp
z45bNAlj)<S@lqPJdVa8>ZhQJ=9SMreGLk&CN|Uv1mQKVFR$us}r#^s6q9<NED#I<Z
zdDpl8XGh6mmkis}^mo5J@a6x^kt1|Z;m(D2IqE8NX%+Y7_C=${e<H-~DwdcH3*%vm
z-o9}EL}giKDlwZ%n$BmJzi_|u5tC42Hin5*Z&xI`YYAZ6YL=Ky#}}55KM_}J*uZg<
zj1u!1;!HnjW6uFZ)=ehI6S(!l3hYm{fnq31%dZIQw(-vzJTkGv<`c0)r{5{j9s25!
zgBdFd@wb#Xl}Gmy@=cyZ>e?P!T7Ooedj(B%SNPL)a%D_Ii0*E66J2)!6q!ueSi;)K
zZFGGX*jpZXjiZD*lAeeEKb%-(PaQ0w_F_=rZWO3n;IN5CiQe=FzqRnyx{obgTF5rc
zUl;p2-pA6Wi<7$e^`LL!eWq<0GyK*S?`0{rp79u`YAt?}6;VzM=@4t2oGgcV;;FTx
zDKIX6gU0Y%@xpkpPwG7$-7h)+R@EP(CQ@lD@pmPSKw-Yf4+%t5Pd^#X$6*Onb>Cii
zyZ&?uH@q!#72X~d-ae?_mm<1pu&0-qL;ET|%R}71rI?wXOrwNSP{NJ`T&$<RQ9>Q_
z6-Xwozh?lDAeE8f?UoB@foC8f(Kix_7TxW%XAlrbQjp(){OB1B#L^Po%Teu4PdX6M
ze5C5$5!cEi)3PPXh=vr{MBHdk<{*TvtRE)u-Du$Kl_knY(xtrKR1Hbj%7r5r3XHO$
z%8;H^B8IWv?@Lh)uY5s@PjPp}^C2HU;JzbY+a#35D?hPY{RvzV@%+Qb)3_G0drTu#
zZ`|6lOEhIE{db<YRS0`f>>kTyW=t^l*noRcFlPHP?g7HWvBvlTp0~wL*BDtKBWg~G
zR!>^kdQxpYLoM=pmcb((f-T2F*0UOr$`-PoP4T3_AYSq|n)^b3ca%+$9rtb4g$G_$
zT??M&Rl@_XfpmJF%u#3j5FVIAY&CuPhdj-(;kod@<#d$l;^0w#`bPckaGR@qi+*=_
zLdJ#%9#CBe{qlo9d?$Z*lxlld?Hei5J3R1^)L>&=J@i6qcwmYwTK_Q^oD$m9Pr#z(
zcxbySM5t+|s3F?ut-cBm993(MR}xx8@D!~OUR3S2{bMKTM{mVfN^-#io}&MF47pKr
z>I279Ek9Kl!i#>cG78E{H^Pg4QI0)lQ3TBXG`#4SDnLp-r%8CxOa4;-DMAe$MK2L~
zdY<pj14nGRfkETA_$It4XTgk#vw~#DL3q(u@r-Q=wheWB)&6j}OH>Ol`g|dpGoX3%
zV_%Oy9OX;IS}8aST9{%nDWdL#7adTT;$JJ{0XpN;e>lp9)Ce!iUJP9ye|7Zz@Q0&}
z`Kj;8KON;#qJ;Eyt-_0zDvTg|K)0m>aY+(9FJ|GXV|m=6;YHmlL~+13;!lpx-|?;d
z(-B)DJiJJOAI;yjN%?=sIuH0LitmB%?p=;_a!XAJX%`@Y6q3*hkkF)erArGCdVquu
z7kcPDM^iyjupqstC@83?h^VNjsMvlgSYr7ppuqq8X7>XA{`q{~X6MbDH#2YEym?c0
zu78;C;?026+BMWEY}55SI9ih|pFU)|J}#%70e<ZZzB+%iwc@9}Y|(Z^&L5esw-jxe
zmUIRV9mK5s^GlLy0@qu}<DKWotS`_#8QuTGbiGX;g|A}=<K_69={j$5)A~<?s1Tc$
z#AUkS3$STAYxi8Yrn?zCp!dD@H`ijy?*IhJ$YALIA)8$bO_!|LeZPv-O93wn$Ol%J
z>Cn-)YCLSZX5-@B>PnwFu$%a&fhQ%fjhAD<5skIUiRVah0p~)V*j&@KD4fMs=e06@
z8#Pi-+$7WG-INTsRV;2fu@L=V0ZwsArQv)F8YU8Cx!-h^dLTL|M4v(MHt?1ZISxR?
z;t)h9P1i>Jm)nyea*$>JPXKOCG90IQ`5pL)j$h+s02p`-pj`)q5b!#F;H?|L86Cs%
zCof|GZpAd5)OUavm@5H>^8qV>WfG{#%NAhMOagWJg@%XfYYuL9(r<IV(eh+QbNyK`
z5sXIaHVdOrjVDDs^JQ$7O=ra6)gNNP3cBl{kjJ)$FxhI0W5B!Q@s6TI91jNyAx}lP
zT?qM!1tpRJmm&N_i6j#e{AcQlF3As1-QnDW;b0ILAT|9~0%SkXN#LJKTme$!BdIux
z9RpK=I&lQXQZ2`HK~w#K4*;EN60nZz8V+hc@F1|C`aOJ#HGyls$lEMOI1<bBN0<w%
zCkvORYeyBk(I086&{A(IxeFib&L+E(17C#I3;s5Lq>n25HPiLD5It{KaNt>pUK1kY
zdDxFvL^t77g+VD5BI9F5tbw>A16js5jB~u;k`4S`8vQ0UG5*7(sREcg68MckI#5pn
zcX(+Jw3S+iQm36xX-u|5WoB)+lfIKROQIH8HF&BH`95IRU{Eho_t5AmN^N&;l+s04
zG+j@}Lp9s3!l1!W4OCSqfe~NVLsZjrJrxGgQ7JUtX;3~eN&?IZ2I27zniW;1Kgv4O
zAJKjhJ*sLSonpH7CzJG3KY#Bo$RAZ1^d?QQDn%31wGSET42Xs?8G?PlUI~ok<$2(^
z;!DSTbdO5W)^r`oB1NX~y#)c?ZIJ42`hPfU8K;OIfG7l_L&6E4VGskcchHBVGhzkB
zm^A1+fa!!`1n_bj_=kXwo(~QM0%@6OJ`OaDrr08B2ijn;2Z*N$9%GqAn!xh!q%X0>
zXBnc$`f>U}=gFMKbiF8dgNq>=oQAFrPO&Y~;ARAxXkvdG95G#L&|sO&FnVJhS3e7;
zj=0?fCi+a*2m;zs0(Bdit~`HWNvP=>=?~1gYPv>YA77TwrD7g2UHMdT%3(P;X1XSj
zR6FXG+}bxaUHxf(xv|PH=0y~aA-;cQ>wS{Edw5?cKIN3k@|aAbr*E6CF6D`iMC1>X
zR9)zb#t~RQQJ+AvnkGn!ilj*W+;k=SQ#h(XwDE)&6l!UOt`KNOAj&FXZB;^wzHEW%
zsw06?yB<lEpo$&DitJv~)mq9md5Jt$NLGhelSiuqrmLpPi;N#O%z$g<8SjSE+KFH!
z-*6S0M<|=nGrnxYRo(Tv;_Mq>xYGMggk`}wm^66N4A}2y;*?{=sr$e*MByaR`X@8s
z0IKb5q6!~yNXjd_{4o>EfWy^^kTaXp8<+t{+;TeB_a!;?bHswA%>s@qa;ruGUrLQ8
zJtaA($lFn}-}i?ZAfMx?HgdiNISm@HAyx!IAov=AO#9gF+Gf6<4p-$UIU6|ZfElnc
zk#wy%F07dW+oD*XSm^UQTxs4#X22>IN!$;j<IZak?^*~3`@uj;mJ@$914{hy0U>sL
zNxF*1@7xh$2Jk(t5s+s24*_cC)XJ{JY^@6p<!5F<J0X$P|De+Z;$^l*0If|3XooAS
zgw(x@8PFBERK6mq_C8?-G?G+Ykn40sCfoD@jl{HxOod#Q9W3&lO-%e%$iMTgg}jlU
zJSy!T@&t>VVz>U>3<!eUB~GA}i8-A#$QPvTSgOg9(QVUA=dVrKOfNK@ALG&*t_Q3t
zIzJ_iW>?X9m7;9;!*qVZ@K*azDf#ACP3JM8XNfPsRVj75>0|+t6o_;EXwx}Y1%zO?
z&2au*hQ9d$t}4mz*b&y5VoR9LB59B2>r7|Eat43<P6oKDduxcxeB5a|W2KxG<VN3s
z#ilbJOEr5iAMN49I%~xts&|BNSZ~wGrA-j_PG*kS#dKDtfvdDplcr}#8~uXml%C0i
zYXHaZtDDXW%ocl4LFLvN6|P`Ugs4JgMbu3j+!3qQ1}u%@tiVbXjiL=64GfEyuWn6e
zki}0MJRhX|I!q5XjJJC!#EW7qr>fPC9`wHoSA4r%E`IX^)|bV*3mOVvx`*TEIF*|=
zcs(qZgY0k%q-tLfj0l4dgRDK@bcoxjO7JPj_X!*{KxSFV<pb!!w<r$X#Zgw<R>if=
z2s99iBIzQIzW-Hy1PdR@JZ8V?klXxamh&La#)EY%m(}r)AK4yc>lqaG=Q6E37iy>{
z2~5Xt!Bn<8t3tO=YDVA(z8w`@)q*x_Mx}!UZ~9JEa7B0EpjSwL604ObYx<5rSQ6-d
zLh3IJzPC~Y$$lS&k{Q0*LpNq9$#wrnGoq4wiGfJ&1o;*y$^A^qZp4_QVY7pjvfRM!
zqEp3{lVejnjMZK679@)Z&GFWU5Wb)qBqUcef+yKgoS{)soa~~lbbk~+dUu-fEejoL
zp}W&T#rj4}GaaugO+{@td|^7QTdvT%;2wyLKI7QNyjESMQDd*^cqo~5&#Be~%CYaB
z2~_nUSDj`DNb)XCJDYy0Mm*+(j@f7$KN(#(B_Bc0qbaAKwbQS<C;iMISM7=+rn;HQ
zbUY}=?3C}>AXj819c4dV%XEy4<&p(C-_8W@lOR`A>T=W3*5AoO`V-+?Xgb=*K{dtB
zpvOHK#0R^gy#F=%4wd3?*r0PB>nieDhY{MnuF@ucXPI|&WH|5u3Mm?8{oXQ&M}g94
z%?O)F+Ff{S^5laH(~-^6MQA%LbJ$8~J^^HA5LV2~AHdIeAz@okhnm`bM%WX^y1Ihk
zs-CkNeo5FVma4IMLn_j`U2qLqIWj`kz545_q6Y!o|Kzc~dRmfETO49N1Qp7Iu#XHt
za~5`F_;}iMR1LN2mbsa_B}v`F`dM|Wnr_u?NE}&3*>$VhSja(;d`K(0P}m!$gL7rt
zw6h>^4~G;cD}TnpLOragLww;B>3FrNf;Vi8rVV`t*j1#a;Eq%!ZpXS64wCK2G976_
zWN_56ZnMMgis^zN$D!{7?^5HeCRUBZEN{$nZ;g$@k+N@=l_ynJIxVY{s$b=kiLX|u
zE3)61p>Rys7l<6DokfZFCYuh!uNGnFxjT&V8i`VUGM6S5kCH*ws)<v|Jjd3TPzfBc
zRID`A<+=Ya-rwD78glw4awUPVo~H2^f!e*q&WGii#+&}U!n5u8a}44|nl(6BzZN3*
z%LyB~U#6(3FFeduv(5rb)eSBMu`a3$*G%Ji8D?9l<q_kh+RVIH`+9}BlDysOju^)H
zU_61nlNGk1UjdhC#+C1Y#I04xJ0SnAtJ=bNqP1pJWa2rDW*t@u%J5JqJ6n{)nt^PP
z%wpUr)7T@G{ur?meksZ{_ESCWsIe~2YBPgQ%?LuH4P#wk*!?o)FHlMt7Vh#?zmF=#
zEijF(O7*ZC73DbQjupNR;jTK#W7WH945}_Xk;0Mxn8t(Dc*faA<UZeJ8e>CA^Q~`l
zxGO1cjYA75ESO3~`K4kbVl1C+o5mz5k4SLeX5W=?SM~mrZkonq{9}xYRoAsj@1Q{E
zjCHE`w)8iswDLj7>H(%PoqlFrtT_62)0idW;wZqR0MC%!EML!xu6VC^3}&%`X*5-K
zPkrVk#b}Dp=02s5iJv-hJi%PtXSWxDFrObbRGrr}jn*OJqk$*|HdbOOum|FA)!#JQ
ziFn9t82x3~4v_INrqLdqRpg(6<Or22q_<NTq{84D@EzEFn@yv$KRs<c?C*FfP<IhQ
zI@b?W1tJD(M&u<{i~Gq0${0<G8W^F-ABe-$4^NT8unT8I{+CdDAcxe_X=c`=1+!i>
zjjEO+Gupf5Gb^SsM)g33fCzfZX!47s54pdy1uGtzYLwG8O(Q`9(z>|+hBX&p{FN=P
zH>)XsWxr%JV+t7^>}%?`{5{4TO~P^O!B9&zUOcnijMr}^+o35XM2U?$8lf6mzs;as
zv70Z#?TU2OBfpoTeCck?`rD>{Surisl0{G_#Pn4<h}Or|CW^X??1y9VhYtez<^_<n
zu5_B=D}f~-YYs8>&k&_(`tZHLZji&ygRBjb_85L1I7^9N?gO$lEKBEwnmQX-RgFr)
zG+M%GQ@;{RXEHTH!_NCYce|?A{aOq?HHxTYvWd&dld5zz>bv=IQ~#CmL^|nkX_Oy)
zF{Uf7(g=F<h<Hq6a-Q0VFe)wf^)y{k`ZeEp6H_qA)X&;=ZAgWT9I0#7NM+{rvq_ej
zAHn2wL<$waebS|?$zIWjslYgpXF7mvYt?_mLm)SStku)h`O!vI{}FG2JW1iIwqu7#
z7TV-V9ZcQ(Hltl-FS=E=03y_J#4QPo;KJQtfb*S3RSZQQBY2*i?`(KkRu{y4vn7K0
z=EybcgHw?NDqD44nbwGfEynKYrv45~=|)u=B@YXl7bAfud1(){mcVviysUHN^_S3E
zUZw#BG_O;Zld7I3v<2880m_>vtPV<m4wgrq@?Mv~RbB)>m%v{f1;-wZj3!Xc$x9B<
zERKMC0oJS<qoH$&#MJ~ph-HY8{UlVE5cMC)-nvl@6&<-6C}D{vvT>Q#xIVJdoZC!&
zN04Sz;R6RFay;>9)kqdIjmROqybSO?g%R0{mk$70S&D4O%MZY}QmJh)kDN*9Z{UXf
zhDGB+!(hlAkpGtEeC1v?-Yg0uar2d==SV3Kaug@08sUk+@mW(}RfT+H6KNExJSsr)
zd6s;lsNbk~poZ_)%C2hOc52|%Pi5ExXE%?jpQ3A^NOa@Ob|!mLaAY~0Mzm;;*R^Ry
z>7Jv1sp+FA+jA+99+iZ#znXfHKVh}DR>Got6HZeJ11Q{?4!)ID#30ktF_-!rWnI;U
zY6P%;>I#!FLrs0Iih)clhL@zu8qI7;AVLBm(==0mRvNXfj0G5EonzEgmcY|BkqP~k
zVOOX&d7$cOS;p_gJnr=pdggUgk5<LzR}Sb~y0S2lwRh<HDpQXaC0L&$SDu)9LNdPC
zc`c`4SRNq{qg!O{XfXiHd35gAGOCFSV~rwuDrJ&UWE7Ge)fB5B^-NQ>0fD-W>KE^o
z`x(Hh3WJ#p$f;4OYyBXp9*AYYLXcAD`l`+#(6>=LK^~}V>ivXLa-l1XlCfs?6Q(Yk
zs_s)DegS?`b1MBoHKlT_+a&9_oRKiR)n9>JWR|I9#_w`EAio1rWY<E;4kWf?bX7aI
zwtgfHKf1Q9g9Fx51TC;3Y5Y1kx-0P=(ZRrmwhoTQkc=J-9Tt$Ut(lNHHKRJGEvqpj
z6K9bLTBO7Gb~RUZep*PTQ&+2@Baw}Vk|td*khyb{WK$2}#F!Dem<l#MMktQhQUa+M
z^pGsSYIY&ef*4txV;M{j=`8XPuR{+D7k;9$3&Tvd%4Du89K}0gK-RTb;zub`BU~;I
zNtfextJ|7?h?<lN=_PC&q<>&uv{9ij{nO$J-P2j-_d+)cx+pGAmqLDz7sJUH=&qL2
zy%%6w1Dnt#`F^SHs^!f%1KmZj`ZY|fdcDzR0mpi4pdO>qq|r0M;pG-&Kt+3xei+yc
z4)vQ%AD^_3@XLT4Bp7_moQ&yLbYFqHOr;pZXHrMes?m1<*=z6pnK|3W<$F<rd^tNq
z$6H`)YJP2SiHYnFEviXTsu4oQ`RTM_W|HOLNBxcQw+`^DpHX9E$m}OkGA3A)<f-IJ
zkDC=Q4>_payqeidq$EbBMzacRiJvh0c$uZuXeNZ?ABHCR6|+q_&0uKW{>p48B!h)y
z+;=G0u~ailG?z(V&5isYuVnW!^?&MNGb`Te0{IZ;w*$9*gV_L_)IWa;FqV?m<P4er
z+`Lx`EGHq<kEdxEgC)mCjtLO{nh_lpKAfHKkz*(U?*(H<@ZN~Q{cYNK;g*qOS_vKR
z+pdosON*)Zd6h?w6>miFV|tDpo2+<Idi84q?JA5M=7QTGi}Nt9s(`h{ZWxXH&T|&K
zVJPxD&xH>!Ap2pR)J`5{iwzjr)nB$F7Kg6*af(6e`H@{AQDu8+>d3CRH0m~*mA6Os
zlGZL2iLV-YziN5jTW%eR74fa9;R^KD8QGq7|L7>-y--!<4v{L1vd<41*_>U6ni=78
z_tePdNg`pX+t88CDX@A+7R{U6bw(E1uvhi>wI7*{Iq=IOn-=Cui80#9mQo6J`SX!S
z@|?8-C4;LGlZpzpJf~>mJAAZ-v<aYj!{3H_;TB)>Xjf|e&-0uixcbB8PV&l`1n6ff
z5SgFn3=Ni~#5_@%fLj8-&C#v~Nlo$`zmP}FV^$uHU+&36ty1~(_`}M>@!LIlIDWe)
z56AEK<l*?;%A;pZ@;H>|_>eqe2ibXih_m6ZhvP$>2!9@XYpQxUKE5}PkMGUnlY8^{
z#L6Q$#+BWyC}Ogf=jca6#V(eQ?L<ps7h|in^j7o`Ben?DTX-_h(NIeAyrOPMBO-_5
z7+19COrE2j#9WV<%*6@>u!yQy=lc?%IUzh5<EoLqKhKcL#Qhv0bjF7m0qUX}$oMc$
z0<=j)TVfa!NE8(7S~s!?$-fzsXGq1~c@AruXMCa>I%KT8s$L7p!rpntr>gF9`}fX0
zdBy~?ik}maUj*5NG@Q_$BwM_gKw~?r!)@}6iOEt}-<??3kc2vUMpqIggj(g3Uch_a
z2o%WZnyQLa9OtT1Bf$-)W83o#`9TfJq_U7{+=!mO{c*0A*@g7X5yPglXI?YUh*Cv_
z)ohDx6azYJRiY?5mnzo_S0T(*S=5bpWwq{^r{D3X+ojU!cQD}zp?p)6XXr`*8X1IU
z<>_Dh3Ck?PuT=)VN8sZ<pQnFHl{{6fs88cn-Ua;f#itE4DlWG|j+>aLueL3K=VMr}
z91DWVdp@>0jJ}!?(^fQMlo5bz+x0ws4L-a&xG=9W0WpelieU`JUz0*0rj?XeR}*fT
zpD(yUKoyt3)M~y-32Y)|`ZgxG;zutLYn-<P?p0Ui=_MGn#7cHyO4y50C0n?K7z(a5
zMe;Ank$}{q<!5<1yQxZh6W7}b>mf60A=jx8+gxhli}twsH7(53YvGD}J|-7g&5<hC
zp$C9Ks-4^C>D6tiCdw{>L|4bp^?dBx>v2`ft|cuX$Z$R%_k1i`sYigK>)glTv!cA$
z?bNl7W7QyhooT7(W8OzEDou1v^DfT&0m+T{D0G~b_XB-NSylA$DmG-qcbaf{(wTSC
z&*hrM<)o5B=!gt0?`4%+=!i^S!~7Y~9FX_Couqc4mE?JqM8jpOTRr!N&fqd*aPDwE
z8`g|Eax>+{#ih0I<p<`<I*j`inBlW>r~2Plj?FDpQc(sxw@4XqXxh4Sry)MYVRFUX
z>0FOhr`m)5Gpofq%Tg_Je-0f!MawP5q6DOqX3tZ(D{%(YWpVpX<gQZh>K>om_3c${
zAX>O3cfGhaa^=~!>$&TddhwpOJl9&N;r-cBxkvo;LsD{2z(`#jJFl|uaxHdIIL*b;
zUTWI&i`*nt(T1ud^;AjZlZKgZ=dyfCnY)Ec5u-4w?2;8|TE~yv&fe^@KX#Tf$dA?;
zN#S&SS*Hm2T&+`_;WWyQWJSSLFz&qDc|UGxo!e~t?qC$pU&kqcd={x3JK7L>2-wCc
z!yV&E@r~Lx{szvW4qZ+G)M47^*f9&5y};A5>&)5J@&lj!ugIKDf9%S=xQqA}Rzc*$
zM}3!5Gm<`ILMJB$sK6_W-Ck!$sX;OM$l)EL|6PL%Oe`jAZf6%ILo$92IOeZbv78Es
zJLz_IrhrNB`>S;Z8oo%hYIT9Yd4H|Qf!wy=t(7;4vjkSPs6hVbQiWajRspQwn2O%5
zYI(KbEwVaA=AX%~Riz1TXHDu;nFwwKC%1aNlhB9#pe?=E@_!?ntMOyF0m)grlIDd1
z6I2bTR7CT-ovl>EGo^~wjFb*#dX-&#DWsm-9=9`t33%N-%y?5CAPH(ShTm#8Qs^;b
zx&U%2Ck4rl83GKJz+7GgF<t`8c_{&w&>aFsz_pCk3naMB?YK&O-5xLwkQ6>ZPMict
zipPMR5)eKIfPHv0aWa_?5W3fZGuAE?4EexLp8S~sP1uFfS;J?~59adqzkEHQG@|<%
zCPaEwK08wTadi`j6iw;>jEK7`SJSa27nJNu_L4>1Zjz5+ph(#)sj57qIUaoh)GU9Z
zp$g6CAmfjMUJcsX+taI@6H-5!x+`ufMWgyDlKoIAXWSvTv$2TvFA*!!AM+t_QR;Z8
zCJmGlOJi`Xq-r0{@nsb$UH-jHPr!oM%Ky;V{iJjatYxlxY>WQ;ylq`o%JEj=>oxM1
zDn*i4Sw!BuVP0kJz0BLvK`{^y?m@X;-{y7(iE^Ry>Q~ySgIcF`J7tI$2Y<|$z!&t6
zuG6493{s0*<#txWtk&DfS0`QNSh-#~(JsPqy9oU*DRg}ZN})U_1xHLmx`z)vMAr4b
z_S2RrJQE0ozUi+;oD#x0P0v3gQ`VGk+>Se9wNg@iWp!QEn{F3u?i9D<zY5!d62I>t
zR8uZWmAJUeZU>7!A_n5NA0&jOGk5rM>$zgQC)|$jrIMu#e?>dpj-OOO7C+M8aXZSe
zy!1Bdf^RQ%I}Wq5v=?WEYYgauX>xl(^^Ks#R*ynLsOOlgAP4pYiPy<MsEwryjtz;_
zv~gr5^JiHi9oqxwM8EWEZpU-}jtP+z;e9_5q8dB7Ept6oAhmb=1nI_(H>yd_#LNf0
zeD%FoiOd#Pr+!#$+((9})G6?jN_=QnV(c1HtPmXyc${QsIJBgBBSPE`zK1i?CP+s-
z3B?iMfCSF*@;2}W$B6<CqP7uoQZL|RP`?1*+8e|$V*1LHKQka&Hi-AKzbjoc^&5(1
z{eZ;YKdcU!%Jf!s$WC)4S@5&0l2*7v{>i@oq_}EDV^C!uJuSyiRxDdv&t|KBTPIc4
zb**lYwh{_D5dB^@fB8JSE;dtpcBphF(e2>IO}TU82e(6R$THF{Ak{<F2*pd25|EaA
zJe@+13Q>aHqnbgEJVgKKc1$NtdMauVI?U~uBQ3&i)1;}4v5xr&H$9bEUAmk!PV41%
zlz(evq}Tj6N)+iH3f1T0(56>Om5(;nOo4O0XjA&!G2(~#Qqowu=#9G61`=h#B;yIU
zqaiaDa|dOrKb^8HDF8#!D_^zB<Y-o2q!V_A&HWz$8R_qUn*5sEksU6{xmnssms7@4
ztb3^bdk@OXDP73w2imyS<;?2KG2|D1aXUIuAk<r^(*8eH?*B*iu~EJ5@*4eblwVf8
z0vJ3ffy}}xUnPU3Y8KW-nr6jHw<8&+xc;w*q4046jqSkRXWfpvF%qz4lH!+1{quG{
zsduyf^4}<b);5K*jphzyf0|w&qx@-PpD~h5GY*4EwKF~KBhccmaya7rnP#-Mb70NW
z`r^yTxc}cMf2KBtRIkb!xAB6mB^a5Dp<Oo8ZJZ=awSUyEyW2RgyhUP`Jmi~HpPg;P
zZG5Q+tU4QCsq7(Wk>fVLiWNO2kkiF&sNZnlr!<U=n_y^r<LmN*Wrl%5LyT|AH6%0c
z|5IiC8%6Pbl`~w?URGU{GI1!zu4!iBzfu00+7x_=!?yy%-NvK%2(sGLqLBacTb4$q
z?A1=)N)+}+%;z|yJ5CmSd#-XDyHs64^a6Td7g{j7c{49#31r1g@!i|QZKzXa@Xg*2
zlQY6MYpcz7gXog*9RdurhB;%C@MV}AR|HJOKxREh;_I3Nr`_i^e9RrQMo^#tAFFm*
za#xAu%^3t)>++>#GIf{+B2dFG4|u|DY>@;d_<xLx+PGq1o>;MLzLe}*T3*;%%rY*y
zjipKms8`l=$`BdLEJH+e&xLMdc6szAD|)UK-I~h}d%BHDBy12Cgn&}d6T;N1A77Ey
zoK#2FVtt=wvhgQoW|0V@VI2g)AZQF`fHZedv~|ETkdkC@eMtg>L15fIkky8OY(`=c
z*#+QDzU_UFK}Jg?-%rTa`={F&fw@gsO6)b@hiFY7KM<-pvK_{gzj|(C6f$Y>L>6)R
z8ixNp?DV)*pq>{qj@*&Ydxdz+FZgz5xoUWK6GV8{lS&z^92yN}->cDUWb!BQhXkI}
zSf(U!M=n5lZx0xp=&^bhBZ8$81th-eHXfiXuU?`ka`%bR@LgL9^0OEsh4B$Uuq}lL
z>f;@W7{;?;GCs)`!vnS$#?w#7w~r<#%J}LZQbu`78D+<-p`68)GL9bw<oU}unZP6|
z<7=D{vC5cfmywf8>Ju`JCM_*=8{KKr#*JL)q_k7_cqt(PX`k*5q<siTyL4|!rmZB^
z%U`&S`$*O30pG6;I3)U-uVO=2LIjQrk*{`j8!aSqw=cJ$D~8CSM0$xt!&ZOoHX5t0
zCI!6+=jF4(#0qBu?`<N$#EEPev>w{-Hr&YZ3}d}sS>)KnL^u~0gH}AK;}T$&EipI3
z6~!5U5;z1>RMd@D($~D&`0HSQ1S=JDH?<UR0;Af5TJgh~#TZqTF(cd`xSvwU1k}1|
zd)8$%Zw}H);J<v7I$<I(HeQX;hR2_<1mq%YUgJkxmd#7FY#xrRSH8P^btq-g+&kSy
z7A@EK8+xpKbU9%+@R$S!^YT1!oa(uE@_~|kc{$-#;I!I!Im1kKFGEygxz9=OBJKNX
zW%=+@I(e|cEl$=n^|Gd+hezIYbDLP1wvn299r9tg(L4rkE$lHQd61f%`^T++L2bkq
zO?V%8hXV|5{cAxX#|d<=3ExVBBP5uqF2r){-^xf#f{B5EQ~K5>oG5;G&S;{3BQ&sy
z!ZCYLxSi790$X^c!)?kDq(U@Nlb6=vAivS*P+_mR_1_RmQy*0idyTi&jK~wQBNL+}
zP@9+ez+_~(nQz^FIV#ncZatEY-ZTp7Z9YW&+rS$V*u%@Wz!$1C-}Alt#e4{xkh{#!
z+WeEN8C@zff#;xLX^Q4}wih)i?V-M6HJT1l0@eA1PziLllD?1m1~)DmO{dvR+6GE$
zUmvooNA-M6iPYp8`$9tX5uv!h?R;FJQ_7|aHT%h}|4x0H@eKw6-rDWbv`IMh6aSIG
zvjheKeQ-0{z0Tnvj*vHdOk&Q0MZYG=o;SaBr%ieoSkJUmN#h&Zgr9tJKaQPfR?8N~
zq&FaW)e?qRoAebS^#jo=1&0Sn(a!xS;%mky4N82I{(|OrTYLrBsDfY-pYI>?#A$BE
z=cnwks7Bw6T~Ro-QN*zU=r&VYg<G%aLZ!_&GaExFUbJ8uWfUP}ilr@@$<I_uTeOY4
zw=Gz;u-byJ`uL*QX!adsMxf7bidjDjSKnb<)GB>_7>jycl<#<Lrft60U86TR^EWsH
zg0BwU<Gaz+RY@+bV9StxkX@3baj~&mf2tlP`b{bdkAkGS<BKNJ0iJ3MLpneKMyQ}y
zvN9R^fnZ6-SR7vf$$_fY5LB=Lk~!)`mRo-b6Q<H_1G$B&9xKosR}oZmsj9KArOY0r
z#9b~@<MucCC`%nwJ%(G6y;<ltYDSB0(0?xUQ<b=m?bjSTpMoIJ_eL{Ue8zFj$bE>Q
zI=W%qH+t->l`NZyNqMp>cCy~S8qHl%k>A_NMyrwpioA}a&pV&;b!(1Og;B}o_b8ab
z-O)N?$KP@5J1AVj(lTv71CaX2nWEK|MO~u&wSS6=?y5Us*bG6&+O<bj3tp)!*PMfF
zXkF)C@E?#@l;|E;UL`^`K;5R+`)6pi$(=f=r3Tr<8zag>-`bC4T6CkV`**0aR?KF&
z`O%k}jgL`mH{oWCgf!t~EZXEsvMRmqNw>Z-K$XU5Azv6qeE@=XRAbqRr3A7h7AwtQ
zps!Oc8zRlR&Z-zZqN}*|Vw*?HKy^7W2nJ~Nt4;9m>cvV;Y$l8EGFe>Cr0^HFUf^fa
z-)15^z}8PcwIf!I@)YsZ90J(d@>0#Z%W#&(un*}vx%J_SAuC=-5wx{w+G<5zxix|~
z#hYqTw8<F=sKHQ>Di09%1KmMh=nAsEl8-hSEt@<6*@z^@sXWnf%t-sxGVLf@qn=!=
z;nrp8t<@|B?)IG^@4br03JEb5PDbM=Z;`-oUiJXHB=7()F9F9Tfcr3+foU?=l^N}4
z@q)3L{9oGt=pR-uY1QAirj;wIqSY^G^kbD(qvy1AMc1c9xewX8D{Z`Lm0QREk^W}1
zdKUq<E&@@Paq~d}r+vPaEnSgb>NH5zNu~hHX!tIc@U)|Y{Ibh+*s|-ADec1f4O(=H
zhM1Ub>0O#zVw#c&GRC$KUGB4ytwCnnvg>`X>=sk2Ihrv}{Zqp0L#=au6FEFeB66QH
zKmxDvG8V{}!1KIJ2c}AZp<oIkp0ZQ|rMx^1JT9`s`cIK`XGC@-mh5ug{U6yK-J@i8
zycL~$|NqFY#W}a$#+F@fx-GjlwM2H*T@-;|jlkH6oa4s0^~QeMfwl@aR%&L`wk#L$
z+?j4Y&53MA;dC75zd8e#Tk|1AboLpybpWQ77{Kj6(!3ilNx6IlIr!h?q|Fh_+&Ub|
zS3d976RTM++LRBds^yr-XDMP|r~DWAiP~?>lVJcZx6lmX+vb$mjzTpYY+S1u_sP1B
z8i+CdRR_7DBa7S+F=B?YL+m2Cb(cDu)GGB}pN}!WH{I(O_Z3@Zj$b7A9a6?N8`SMG
z?kl#&OpVewTv$i-wkx*6Ol<?MIXF7o6<c4XjsZ6;ZA3onI$x#=AHr*Zr7~ec?mzf-
zh${iy*Q#(3gp^bF0ejg~lOFjTL#%p=Qps9(n+2jDbjYb_&s5dX=#Nv+0<TF7{c-C1
zfSh8J{y6mta9INM$Eh~}@kgpjhb*iNn27bcI7_M84ErCYdM-yPRfpCnRR_j;`yvH(
zJj7E8arMo2pYw|giCf}od({%xxh$)L6{2Z{H4zOR?3^?_O<LFuXrHFW-*X*oTp_se
zmbl(dv&2=n72JBH<waZ{+$*k+Y;hGLhr;6`E;`}yLJ=2F5tkUAw%kLcx?tg}kjaO%
z(gh2VP@$5LB>o&w5~2$hUIV@pGP+<9x?OnN>VlQEBF4L-nuzOou_dmy$NxuM$9E`k
zJ<<ko?YUQ6c)y2}-7hLh2|ip|?-!Lig)8-Vv7OYng3^kYRW(CViPKgcyV9M`x(_iv
z55BKV+i;lLm55%p^C49F&Y@_l>lESC6eT61;YTe6C)fw*Mxi%cvARyXCH}6n{hqGF
z#_7Mh&cmO%p(BSYp>&<)s^@U}!@XMdNW0(I7e@W)4$afta+Z4VeN^DRSU10L=~WYU
z3jW4*_cYtFh?asY6VX03ZA1~P&C{4$$i$v^t!V5Je6eXYG`Uq{OyUI1Eg#X#%%f-{
zuv(6FrSEYMRLm$^`^D-<tK0*t3XgE*lMGZYiAcAD<{sn`^$S$JbPyV(&a%)u2hkep
zm#y4GRlS69hc<4e#qbfvpIMCMmdS*rzT$SSYA#{J81qm&S9I5<k|6(acXNM&`*N%V
z&F?8mb5#;jWq$8&PP0_LVzqhmE>fsae2<nYbDyid%j3oCik6NsLt6E|g;uM^BhkuN
zs5)v{^i<l^LIf61)(Uq9+)J|IyyR5V-Lm1lU}4j3(qi}RmMfUj-)Z_N#fP03wZ=J3
zmrz)4;nTNDHi0Ue?+Z0;`KwKrxV47-zL)xi)9sl}SBlW(H(Uy}rfaMQ*hJcsms{#T
zG6U6|VJ-F3<kO+M@3Z!dxcthO(IJJQA+1+S{i6W?5?u#zw(db$OLXNFvKV<_NlTrz
z6{ExNkUmzgr9OqtEEVY3wxwRE@>V}P(NdpYU8JTKAzSK8RE%W%m<v6VOt^={UK;Em
z$KU@4^~S$Z{1VEf30h0NgkCdfNl3xO0s^ZURXRE#Jh8N;UK)lBkg3VP&jnBRTKW)K
zT^~5CCkJ1CYk5)Ri7OJ`v6X7rmKSO8h|4~sgR5>>VYteneY|AgYudq8Gw4doR?<2h
zxA=y1aCx#`vM8FUB7_gm8>O|BJF3~cX}~J&%Psl-Ml(89v8X&TiqDY_e1Mv6wQkG$
zlto%sTE1oM3Pt_^>K64u><X4e)UUXWyAZnqRZ>5#n|eNWRj_m|yYE^*pfNj9@=FPf
zovj$irGGG(jp8W=B$`dB+z*0Zwjg$PI^tG`1}4>toy$^Xr^jq=*2R3YBSb<2s?zIE
zJ{&tyNvKTASV<AohF0fBl;(X$>_j?d8iQ8R7r@79c~c9u*oo<q58+$DU(ije7dufp
zD*b;N<(XCm`A(|kjh&+M1wl`kmW&p3<}5EQ9y^sw5RN9vJST%0l)8+;TK1v?)Hoe`
zzZB>5GOgor{%@2H(D^#qP~H2Hm-H$51;1$xz&$lF_J01+^xCu@Ko_#6*nR0xF;ebn
zlYns`d%PMuP?WJ1;e~+LH?I?Wem-6|VRAcOV98-98C}BUs?|a{I^KyLsm|&`cG+CY
z=f^JDR_erx#IY7~<=f``*hUl;S&-Ihk4ChA<tx0uGc2}IGXI{V#uTUt1xn)s9Vth)
zDhKbW=e@Bx%Cy*RV_E6h*{Vy&-e;8;nY_L_HjRwZ_>8V&KPoX)WYRT&iX)S>C|f3b
zsLKuxo^(xvYzAanh5Xq1Y>jvMV<6bREOHoKJ45saz}kb+wH2*9?F-=JN<z%&*%dQ2
zjdq<TKd04^`Tn+#d@pv!cCdc3-75}T5qyz;c1moLDiUeZsBdbe*rYhS?sE>sCN)!f
zL&Im+jpg%Y%1$roT3}H|CD@dQK8;o1G0QPdQbkd^w27fI-o!>LiAj5<gvYvB-5~jL
z$PlKR_M~wsYWuL5O>z^#gMDLG3&Al6>i3IT%~+*<&ik?jF{|mfs0Vy%Js;CU*mTRX
zsCtCorIN#HDh1naC%d@fyxpQgM`|&HYVdQ=Z_reWUX2+XrWo<d&|1utd+^gNy!6h2
zHDi?N=r)1@IHt8~KSkg69(t~uSNe)<PIr&VKsM@>bhr7G>f+*<jLJkG4HtGXDKT-X
zz#sa?cV$++ASR9mvx^xcSK?_$dB|F*_4DGhdr0BxR+B=@by)X35mOB{PGtnrx`@tn
z&q||U)kHi9WIBW3bd!b3v{22lMUF?f**eKc<dKk-h|9kB638)rq}PuvqxlM0cy8y>
zb2%)qg?H<)Ko5pheSaHJxKwoIe2>maTB8Ry8W6zn@28JWr^wUUtJC)n0=tkx@i~yb
ze=yh_prr)H@PaN(?<fH@ZaP{&-P?!L0$VO=at)o)T`k*8Q|l4aWidwIij_U^A~7F`
znogr{fJ6IxCQIxV3{9K9#NLqDpgZDFS?RX0m?aaH9udUIM5WIzER)VJ8~*Epy6Iw2
zh36J}#WQkPV{0^tnu{dDKJ~EiOjLbHazxNQliB)sa~-G-Y<<X+Kab8&YZ^V)#S!)J
zOde0k+d+A}B#*Q5xG0Y+a$oBBT$M1uPF~sTk=dTn<Iv+Ql#TBu*P!&|Zka8ER1OGJ
z3-svCpy*kZOPq|JIWEr0R7-q>&fl{WKM!W~e89%U@jgNx1@f3HkLB{<CmZ!GLp7rp
z{-!!WCDqL#B3ZpqmFKRgD+KDD_vi^fp}JLrJ&2MNi^E50o;y`Etf9IY({l$)sOGHq
z4`PI*9eVkExjitsGd+4>h#ePF=Fx-vfmP!?)|tWA87g`n8^&cyH$A$G{=n9+pxy)(
zw-po`WTQR<HF%RpuRzW=s?4SeDo6ElfU1<`(M@cD*COjx72E?6;2Qf=_DTlTY?()|
z44F-p4XSt~sJm3%LAC$IqgTb{u&IWF8gmp>dBJ)WfqJX2N3UiT-%U_zQ@tiyL@IXR
zkjOeFq>LV0l>j+d+-u#>I=eJMo>$SCYKhpgJW(d!eysZD6osN&#eL>EODp&3CD`p?
z3$g54+%KNVV0-;eUyJ*Nx6Mz|XT}5hbTfht%Jk|(HJ$w9U7jhzpa!JzE)-*PE{b8i
zGWU@ZO0v1DE~)#IatTnZa3eMLq8<CcRXn(83r?cY-4feCI+-vT@n`hTLphDj(<H~P
z$H1S`s8M`X&k%U_mYG|2*UNC-3p`Dz>jhq(1CB`mJ2m}n;4LXwRW!2e_wx7ywXA+Y
zbG&!U=j-LF!##v@Euqo-AL+_!Nm3pAV98dNwe{-HSi7<I)r^K*(3nr8IBhJ-4|xER
z%W|qOKEcC}1{i%>Np=a;(=lWOT8pkDQvL8KULa&=OUaK=B>n&z6rbjKi*&;a;g-8o
zp@mz2&nmF@w*h;EP8&)81Us@-uH><gmHQ(wpZ+DFZq`G6rvD85D1}l9A~c-Z_{pv0
zxLuzs{sezRb3+0on88pq!y8bZqPogrZYzwYalhM6F@x&NPytC12gGO|I}5gIDH)oj
z1-9ZcL90wNQlP2p&!Q;-w4D*_IjLKUD%3L1CbAgNRV3wIR-3rjh-)C0m8YgvuKKwY
z;USFIbMMmzk!|su{OkP#^8D-lj{19WT3~%rq}f3=!tSd75!S5-gY_t^NMUys{brEl
zwwoc~WJAi-4q#<WKk04FTQ8tnO+wvS5*6$~5Tk%R35@1tDlka`g}iJ47E540FYf^?
zh3F?&^YRPu1H;y-C(HDN!%ACUt)e!9`gV7K#;Ki&C7D@43iB$dPI*rgWVo8yo!Xu{
zZ>N=!8H!6)^g8r_zE%BcAA)d$q`f56r-i|JP_F^6h!D=nRtjSDIPtL9KzQJF(du>8
z?+#ENmRg@(Nem|pM5nzXbx*c6YCUipq>NevRSM;&LL>da)5Z3!RS%Fa=qMmb6E0gh
z^-Whfsbf<)k|2S_-fbV1XRo+2DCX;kUt_>N`4rW-#&N~IN_tOI1^V*L58=t>>gupo
z-(tHs0}e}8ajbO88Lqx;5vfux*sjngY_weXE!hk>A^FFBBX#nl&LRO<@SQue80cg;
zJbAKAkMF`*=aqk&9#=)b#%JjL`Qa4(<mZ&*ci>kET+_5!bjVqT1g`T!`^~B(fp>YS
z4any2sQ_Lwf%+1N;H5o~ErFW66aZtX^r^a3`i5ro;|p2+6jPsBwU+ZJp>}MUsM_`8
z+7YW7@_TN2{92K_Z$MvMwo@$#??8M_;PAb0KzV0v0yYQ(TRi=!t6x0A0FOjXig30O
zf(U2<BXYhm(`tuPJ(%=y*H{0h!muaI$YIvQ!m!LLQU6GbVVEu11RT8E^Gf^SCZ>pS
z^u_gaRrU6;<~;q!`HL^-@bt)`HhO^{y;);2q(^QhM}AYme=V@u-_j$Wkv&0T_F6HD
zG;R=xHw7WKcMMm#Na>4|_QmB>=4-@%K2QkYLNAEd;PRxr<cO|Qii8<bV(fx=W&dI`
z%TnS+)RgWrp!~wvF}kJ1xM$ms`y%?gDzm4kubrxle)o4(SEI=js(H3zZw7UsPbO4j
zAHO@gGYegw^`rzCnrFQZ(9<2=|K#Os;7bBMFy^!F0JkvLt+N%oP*~JU;I>j@-~?e;
z54zUq=IlN|ui7X+zjv*B=7@vF6Qdj0ggxDX4H@j7E{0|Zzzu8ibJs$Ezoy9U{&0wd
z8B1)e|A#~DFuBRkUC$m1Fvu&pL|&i>FOWwEUH4i)uBLlv(I**=28zU4fI`JQ<AqE_
zORH!yp7hA2N<T^<p=p9#K{jw5BSH`C>}*J8Pc5gRLqXFmAz5gS_-Trvndhg8qW3}L
z9gtAh4H4_Q7NZpqErZB-dRv(e5o8IY66y(!d=URMtuVV3;3hr_gEf$v3=$2{<t86J
zvdl)|QI6F!73v=wC4&v;OQ~BFy-L;1YXs82wN6|}#h4#;T;(^#^N@VNogf~NYXq5n
z1ANX=fEv+?h7mTQ#B`p(9UF8^2umQWfnztv0ijsxs1ut0N>ierA<KZ!%&tB9hz=*R
z{9j`@E;0XFL3Iv8N%AeG^Cfr<HiXv};YIJA6A4riUKed%e<@zMJ=!Q<wJ-U3=@UPJ
zHA4rfcbm@B2z_unVI||@MA~ak7Eq7x?)8a(fj%l|nrIFKly^>Fs!w@bcxZ`#h_8@v
z)^uLLbPW4W(B3k=mQ%65Y&s7_XvUD;!uqm^W*t1%0E^LL*TZFc(snc`?7T{Wo{~<}
zicKfZi1A=We8{lN{N$uAFF(+!8O$df6!vsMI9@Z2+jtxLgmZ9w0YGo`2`72E1bir&
zUZda>J|;wmoAc!yk}g6dR?ipCWG7W|FCvBpAjM77xrM3>=`6`_lC!&1`2vRfF-@CG
zYkB9~p>*#3gsKBoCGa9IxFB=mCGaLM82!1eshayo#hA_pdx~Hh(KOo-9ldnlhGqcJ
zkANA%%P2t3bD6?-3Q!;clx^-j08KH>TI4?$g`c~Ol9{b+;h0?<>voBMZT2T>Gq4ej
ztzDVAU&B%gm6*yx4CZK|RDHU3AF+-Uriw_!GNZK%Ec-ZwL7btznr&cVmULJ=3Md`X
z!3NjBtMOwNLXhrK3nMO<W(}F<YxJNiA!xrH-0mCppsOa=s~gq2$-RzT;Z~;C;g$*Y
z)btsqb6*6-c0EUt4*^e#THpgJwQ#)ynGDVHWFN+K4x?d)u9E~`LX3r-`w3!j{Y(Q<
ziaLxBQi`FnykN`<rtmsP;70pyJ?N^Ch-rHLA0k^=?Vroa?LfV~l)la`JKY#xk0Gwg
zzU(2ciUB7CCt;8GFG%Q6>WN!tI!B7CPax;;AWJpEt5TFd0J&i*+#~X4h7rydHu_3!
z0<|FZ<mkL<$R6IXTDIEGtH|%BbG4;X+j;c~)U`EsUS|Rw(b$-zE>dK52bE)8snvLO
zzJdP&7<KYR40S~ZP_ys_zKo%+n4lc1Hp6_renVaHL3wuYv~R&sDpPC+FT+y~F(RlC
zu`@{sA!w+kMI+C@2~vOcWTVM+UZlvw`%||Hc7-cY;dv~l%o~mTk`pz#wbU41Pt;{y
zgjmaDN>Sm8lv_C)I(Ag)pD%Hkt6BgqLWSPGoMCti?Xgx@#Dsc+Jnxguh%WRyPf(oZ
z=-Hc*s3%35Hx0nBI(kmvWfve$31>y*4u50#yW}FyOqYKDnh!uyBhd0@hqKtKNCpm)
z3R|jPqk}}EZzBTyu`@&)Y81<)26PN)OF~5s6|vHeXe<%Z1&ks?)ph7S4(34V5{yN4
zu=_REigEN*_lW%Rilt)J5n`QWw1(Nrsi9@Mx5g>4I-Zo=20|m319?)=fKyV{Q@{xc
z;GE3+2zXyS$EA=U-4TCFObJ*NZ{CkgT25^e#iI(*xc1vVf<zspDhi~iwh7RcGZT%Y
z{HP4m0gi#V`2VOrT;l>9u2F`{M6l3BB8Eq4${!`79BB>hA}E}5S>eiSqd37xeT-9s
zaZ={ddBc5aBWU2ed6gNhWj0|so|cq0CsNu|ve4k^?zzP7&%9!sh8PA0u((y86`uLX
zW$v@s=v#-tURDx)JxHha#)R%7F1bnO7NX)>N6pI(Fvcw=`yTp7UKugoTknC!!S<$#
zp})-_Q#V41{Pz&OJ8shidY^II4M7edpS*j1C*AEBLJV}?o)W0X%Vc1J1e)+dBY5Wv
zzx&~*29Cr^A$nV)$Gs!`h7<n?@Q@dViIpVTM1oHPIE2P+TGV?C5KX(in3vPQ%MxIO
z^u7zcJ(IvQf;mj+d*B-hNJVb}HzdHgS6l@MlK=`*+!)B>z@+|jnV!OZ(-0=6h3G8J
z8l!{pm-IUjbOky}vC`;Mawej91Tdr~fhuTbWB$hz%c-<cj|xQsu`_`p38e8t#fl%|
zmqiY8{ctMHt)7}zKV&u%8O5NbehI7iDaiIPlB=J>Dn3c*MOiU8cnA9L#h&tH9X@0p
zcNVK35i4foEIuoNFkU_Y-siRqyv&Wz;!&t4?rw<a#3=kf<H?;6T?M{JQBwouN8V6I
zxpo@`O|6b54o4M7T^H?@(a--SPogMxYJF5~w-rS}P`tgRsx`4jiGxBui$`d^Db3zP
zWbh}zd2q+x-TnX^>YicAbiFTr6te<JdO@@^F+b$<SdwnFlg@|Td?t-5>3kGzenm;j
zh%%m2lN@wxOSz1|;=3idqS#Ci`63QBU;5dgkn^McY*5Jg$-?Fvn+;xRIUl=e>Ltqe
zm-5t{AwS@D<g<LBnr418lCx^8T4p}_Fu$`@Us|Rf+D#o6hR86Le;*0t=iTP_1LVx8
zw9I_eW&SW4MkzqO9x4~6eUi;jk`11Ul?^ICf4n~%RA~NmX-=8F)Eg?z$y1uMjBc3!
z6U`~Jm-&l;1s(#-ULZib6xltq7s5DHHrAfK;1ya3vXkk{e7wy0t6*PVu@VU+t_hKz
zGlnaztcq=gY_q>&PY~F}5U3=P(m*OEPwJ&RB^8STRw^`N{sDigmk6BjmM1$aHA^{b
zC6k^xX1PDvTTsebWhKQA30y2sbxCE$Q!*2uXv`^ps;?mwEl{a`Ch#Mc%qrp)=(WUc
zA$le8n8$G1%qrgvsQ4&UC0T$S<;aj2t8Y>ICQ7+>hv?NwF_shQ#_iuAz>F-AGxmwh
z0F2wW387*OWD<HOn3wy27HtSrg$3f(a4W{HmsqUs0x@k1@+DM<&;ektM{{KS3GE#5
zBY5%`^15{NOq1wqzz-5|@j_u2{H1Bay>*$MCi=>xXY6~zuMOX-I{|~PaHqQmWgiF$
z9alX?dEO7o0-Ntui&JSrRv^7<y5wk!DXoJ95|4!FG4LEGy+D+DVMRdM9+Y~aAgYU$
z#M-3wP*Qq1M30peosg24nuWOE3q5qZw0?e5gL+_U>V@_Y_ZZSV)upjQV+<-SQ5>6+
zY4W%kD(0rbu?+F>c5s#f`1KxY22~k7#~t*|9m`igI4S<?Zthwyk-sP@=C>|$IQ3L|
zlS=W&%lhlVkrXroNVj|6!j=TGl|Urbco`ZJ-wx56R3}9Xe~OL}-6u-$ww=tous^82
zD!T(WNQe<tPnBI$q&=Rai;jaKP~`!`C0%u13V?}H)w-26el8F?^);m|*^JS1R78@s
z#4~<}l1rk(Wm{4l{}MRX)-$}2i@#9ZRz>%-gfsp#NizLk_FS^sZ#g96W!7u(i63n+
zL#ajG;~Kw^Cs@L&GW$YsY#1QbQ{%%^y_@&~+=cJh7fsn`12Z)~VlZx9fpZS2!~)#<
z7CeSz;UW-dSLA19dQGH&$UY9z<v{<G{vyARkwYZo(n5yHh1(cWBT?|gU&&#E#QXym
z6SDAt@Q6x9pAZZLx03a06k`gjijJse5O~k<I3*l!kuHi^<-$*a4<yiw7q~9`Rd`G+
z)6<ir9JQ(^>!FmR;BVnUH5Q@ei^5=0iw^II!f{zdmt7PofepOSV;9wu07YGt1!PFz
zDPCI9s~53r;D|a(s58)UCV_Va^D&{p!2J@yKrNaJ%(jQn^d?f9T3N}@GlWjuLqTK+
zU9=3~Y^xdvtpGhKJ=-3@9KpC{^0Oroh?cRX##d!}eWs2yQGJnigJix#F(y@jL5<(x
z<O4S0v(&g_&Hn1IwUlGxpVH3%Z?LbAa@-y4g+G?8{&BHRJT<K#S@sJi{UYo=uznQS
z!FQsL8tJ_41)gq0pgH8<L!Joj<J7%I4lySHR!AH*n5Hhe2w=J#HTv=L1@M{2GKKY7
z2PgF(q}Gzx`%U2@v}Dou<SM3c5$(C?zhVj<HKvy78I`@zVhbmID+~*{SBDl}ti=&1
zwrybs0Vv}j3`~C5vW1#V=rSUp+9_{otX-<jfb<k=#$OPEh^$_ij)9RyfC4q!=_0)>
z-Ch?cV2Z<9pM~3Q3Adn!2v_wymW^eCt@?8MWjlJ3L@!W1&x*EEU$>)IOEjuZ$Gj)H
zKHXaU;DSdbI#6+}pAW7lYh48=1UXBknJ(X)==BSUst393$jcTTTkb9uQEW&2p!mEj
zK3+x>!AW>J5)!K!Rfa_Hn$(jCd9of%-jg@d1T}(?E@Q#s5CTzn*^9lAJgSjt8jZ(c
zT4Qo&$uIB}Rd)oh!6+av&B}K2gHj%~e3*_W`V@Q1lP3$Rp`xm8Q_1?1HL*%hw97_N
zNYdtL{7fpTWdFjNf#KD~sVL^2vK~J96^XKOCoHK>7MItaW+M&0Q@4^_vaI10mVV?n
z`=s15tIHQj$PjkLZxfkbm)pd&q0%FGViS|~s$Q{1*+@f-;^IUgL2H3*B>gUEu!)W+
z49sH2n#HLgutlMtV-wZ*S*ul$j@>fmuBwDNXa#L31OH-1<Hc>UiJ{lQ4kCkxBu2zN
z2pAwd!qRBEL26sMR(`OaNipV%FB+B&#ROoCaKz(qgmol@eONqC0uS)A1K27&qz5q#
zgs>!77f)u+2w_o7JUxE#QDMO{tV1pfUwj65MFN|7xd5D(fR{=xz6^YV<qJD0sMiVo
z1W1<*dza&v;Xp8kB8;&h3(w6F_8%~K3QIiL!|=#5JrgTNUVDT8#yg(=Gld9mM06IA
zE`hqdbOPE-iT1)FyfvW_fOoKjy75L8mlR82054mCQVA6CvKM%o6^A`|@0o{HeNP9=
zLUUnraa(AeB~*9GQIF+PXA*#rr{)YF+Qfsq%yAFGw7ZQh-w??3Dl&2smwE{@TJj3%
z{H}i{BO!@+YR=#Td=pvdpZhI&2U5Qiu>=7w`J_D2T9t?=i4L;WR`@tF5Qk;ScYugh
zemZ#xc5=yYUX@BJwnL@jNvdPCW#K2J5-)Da9pIJ(!nlwl7zixSYL809lSIc!R1{=q
zwUmCnv<jog!72K9$d%KV`HXZmo|w|$=Y%|0Zr2sZZ)qk_AFa>a4?=Y~*#wX&kQDBc
z%7-TjUzCK??7}T&3|%Vjl`0%V=+cg2+{24WbOrMInlX*<L>%GsiRuSDV9%;Dzfml`
z{G;qA@#~7-@RvOgI^}2~rlpgCNm2uuRb`fmUE?VQd4*0htr-<~x)_={z%0KCl*yQ(
z$nKd{5oT6pWADzYEQGBBOK~5TE{45W5Uz6c|0w!ttc&u~khLw8EV?Q*JgLAN2xZzb
zzfji0WUXH)4-?p4o~W%##FIpC24MfEz2r~zG=xw3Qyn94q&yYlE?My;m7JlN_PamP
z8R%Z|CqgNgzFnROiI9jViQXa6^dx_x%g`zH3S<0TidHWDwmcQKf>b<7B{P)igZ-&c
z*rjR$CVGfcF1@7^;cjPbREc<!=zU4Fl0>5NCG^gckh^-Q)G2G1BIl`joxzuc{$)E8
z`G~Qkwgkwe1Orl%p?P2>nLMLX@g$Q^{gZ$aEJ;aAe+FtYToE58Yrj?jTXQk^4nEb)
zWEfWL0^vYF{MTA6SUM`AkKU4eP<f=Un1`EXD`(bWl@(928jYoyF%%A}KE;rWC6+E+
zvKm-fF1Zsb5l?h;43(O(N$NolF4+S3Bp{XA4Lt6jw0pD8La){OHh8S0;QJOU#Sw@O
z`BTuwC9jZ;>cm+W#gOsTw9w#jl8AY*>cb`AN-p3JWnGp&EPXxeM{5Y3$wr5NJGfYl
zk%4N3sKI@V?RULKDU~b{F{%Vms9%GB<~op>Qz(b{w<SLUS0x}-Pysqg1}j}G5#`Ka
ziRgE7j8OSTP`spI-9`CkiFsfQ(MoOt*Toowa>SJ8S?2J|yRs4;PvreICUfRx(ulm5
zMFZm3i@cXL1hUHIov0G=B+)mLkylRdh`^WS04*hez?Wf_mUS%Gq7;>iC#k;mYtga+
z&~TH1WiDla4qsEIH^9q547d2E74~z2q#oNLoC@$+CNWYOEW$Fn(XzSXOot!lZ3Vz#
zAEmVos4JX!lHbM%N-2B4s%oW>ZS+^|F#<dNRcoLoy*#Cjxl7oX^%Yk{9g6bT=vj!L
zQFiteyt5(8(Tv$cSR!$Rzf9ao0Q>F;f14NPQ_DV}G_qRJfVsMQk|cNs_H%Y?c*=^!
zvflxYfhae9Inm2*BQmub(V$QzDg+rA{Aie3jVQN*{75YpnJ4x02Jb;mv&GYtR*-LM
zmW$A&6_!^6s+3o7rIHFy%^57~#k1e=C!#%<C-@W9Cy-X2XroHRlSEHRqTBvNEud@e
z;6f~wRR==tWn$2vf#&$Ur!`x^!BlgE3>;mV5X`AXm$1D1%oMAa%xUk7n9XI+*%I`n
z^6F*q%eWxf%9HBriJ}Qx{Q2HbPQCqw9Zq1V=#C7T8JD5gY6o+EPEx$(PcaFi3H}sw
z2+WfHk}*_MiwJYRk$!4T{BwxN)-wt$$9Rs+`E`zdLCkbVmQd!}`<e_*>L8yTd&+W)
z=Qb>z!}<z~b=JZFPI0*toY7VqjVDepixA>`ien=_@bYEAQt^SqSV2%<Hi!=#Mu%RG
z!Y*G=8hU;7g?kRF4eH_Fm~sZ@Str#T53s3NbF=2fFRWeFvj`c?%cINGW;fZ<*9oHo
zz?0aU{f8htD;=$NLak4Q&P3PupQAkcPeF4YI43C+NVy*uWI2*u{s}rjy6u|d4N5<c
zeLm(Kw13|O>t7m6<U+tlY8FNJ|B0jyHP48oNGSvRpfgkqm8rQ_(-WiRWwYj(cSc^Y
zq5ZaD8o9lQ5KaVzN%LHxsH@VXVC*s&8JMT#*MJ`+fH_$HC-4U*!4MhpQL(oCjzlpO
ztf&qk2P5MNUXp=$2^`{OFwj*3)O>}+ZkGVPcm<C53j9qYgPN}(qZQvW)!Vn9-BUV@
z_0*hh<5i^LZ7OYv9pFPJueP#+#+FfL76sj=8Oy7{BeMZ<tdXqb9FLKi%L|5eC2MQ?
zb%xGHOiMJaj)^ga*|G_sZy>G4*$TbX47`IuT9io5>nvr5?gq74iKSl2M<)7>S`4Cs
z%mncQ;N2;qR(#yt2tZQ$jV`?O2l`5&FE68k5t2(0xs-_})VXK|$_={nS+3G=tR=n}
zVAWl}v5*%ZAZG*XFpBFp9wNjby6P#~yYY4F^1EVD2z~`Z^U{D#LIo)=tjJj}ZRG{}
zCOvHWk+vE<cwN+{$sijoOKPGWLsZ9WGTugS@T2n-^&%Vnlpn1Qgf&4qAN9VV8O1OA
z@#=t1lM^=iB^y07RJb%PvC+TS=m!;-ru!}Qe3zfe2}O<WJ__&oHT~!-(k+`_5wst5
zo%&;s^c$SfY}`Wxn-pjUcEQ)4za8-<sg|OZR(A*5N%NhRZB%)=D5oeypna@U1@o7|
zR3iPy3PdAL@Qc2(=xd$0keqIwDbt%JiJ~M$nSo2H;NLEMn`XQDH95=%rc2;MUUmSh
zr4UiH;mtDcnPZSx{S!R??g@_zibqqk!fa0N@LOA;M?g~oERsM0FF4j~wn|{GKSua9
zrTBj@m!iV>Z~0Pb%~4=~G@-}gd|d`Jo)|;3LLGd&1z!n|W6;pg)?ldhzp({tJ_bG#
z7WBL|-vVFT9GVA8hm)rkYF4PI!$Bny4pI-MzZQpo2zYNYl>hxM(rO-|Ic{}BT6#t<
zato2~Z$LNuM{}I@m7{wK8o4)P_w1h;uokl~l@Z(2jG^n%7lc=e#^DXMP=m!#^u@vk
zQKkHgmD5Wq^iV~$mne*+{*UA$nV?&@*aclE+OIbJ@0LxX(J>Wmt?0-wQ(knUo1+?{
zf(y5iS~|&4)pwe+u=N%5u!Tp2saI0`U6cHz*fS)`UkZg)g>T%&<|Lz^a2mWwiq{xU
zDucgm^n*@8H~HQ|FEVZPQ02-tZGbIX49Z#v;~0s4UGC^i>R<*LSgd7(L_0drqSt$A
z+S*V6?^M4&gqKvHmUNV_C?t79IL{3Hg;a}o)0#K_Aig)ylfX@sd+ku*LE*<#K);D|
zyLJLFPI|xy#RBa}T4PrDpbzT1I8KrkK`;xLA%O+{fbdv@#@<|m;gtW|?H+jZN$bRu
zeiQ9o%b>h=u}JU_Iq~GXywFvv!%YrgWJhZi)KgAgRrshnnJpDAX*Rd!er_H7-0)b|
zVlmflgIjii;`Zf2S7fNAjoEudw!VprxW8Zm1cy~O++ha(AVWd8ly(RFuX`Qqj)|SP
zzS6PosC2aJjB9IgOV>WZ36ATGTuJY0M#&!%!(?IYM*ve${rX{EZUZU?H+5YSP+MZo
z@zNB?5(acWFF&tjEO}X?J_r2(&`Sc0ee1>nqcLNVizy=_VZAxc4BWw*^^yflh9c?o
z>llOAVLXUokc-?!XcMqTDO`&}ONEwf7k!bHTD6gaa<wS2WA?K%Bg?X)w=LERQo_^d
zZBo7)@NGr~S=H6A--Pc0;Au)!k%?-{>Z(M;%s}}fr(_K!s>lGj4)e0^4JpwuN>mYt
zVcn0w_mZkurNT)_o@W}{F-j!^Wo4$~dVV34EZ6-b$>wpirs5;2W%(I1%fcBIe}kkJ
zP(zsB;-xpx8PbSg$GRglk`YnOvF@;XNfa#*Oon6KewxjQXbJHQpb#G+qBEJ4YR1yP
zB*x1)y&iF`Uq)Br$4H3V&F`AkApX-V5HCxSc0(CSz1ANDX#N%??2o`MQRpP$G64Kp
z0E>!rTaE`ci`rE}m{KSNFNIWbW!A~Ok~9(Yx;kzZhvB0ES|ut)t+LdW6kaIz*b1(X
z*g#+BrPj|AW-f;9E@2z&_+F(^qq4uz$UIw(jZnQINCqQm?gmzKms|lSmS@;n)!=RI
z23M$4N(r)0N!cX>G+vdB-*i<>8hH;}T$XVzPgbIZb0d3{U&g_(u?8cQ;4r9S(3W))
z+&sm$(K)}_nhH0`yb&wZcMVyzrfv*wh|^o9%TwcC%D-yCq!cf&YYDRb6jfjUvDSuU
z-LRT8r$*OVCOZz9q6}Ao|ETC;MAMM`8OzSa5j{$Z71O~k*4GqEnP0UH(GaIRla+*J
za=1iEl_9860~9`0O4;wOh`h#N>|L~G)Q+Hx5gKy~p8UBDJTznZCbFtsQy$6kXef_d
zd32RWe|e0Q$7D_f)m|cRtL3pp9*@i8fIME5#~bqaKptPp<0pCC@bYB&8x_v@4y8Oo
zmix&!gYLD#1vpg{-8A_OQGe0_GVDFI#(~@tb`s<l(jur9y9ZXgG>)_DXK&4*FG_@A
zh%k|Nr)I2BzO;#{+E9&LD#_GuA3eF%GsJ7jm)s5(E{qie#2a4)K?+b;dJQXHj(ymu
z4UK_@5_pW4)<BNhVA#jK@U2cMBf?$HI7r3{nGWqEc0*U7vjiIP!r;GQU?*<E<0?K$
zAqM1|$P8K4kX!gR<O8E5AcW|}hG~*LYbELyz6}e(%%`ZL1DbW%aE0Id>|E%d(>XGj
z($VxW_&?WEg*IDL=i*jYwDqMmKbj7{!G08Gv1Y84A>KZO*nmP^%m<fi(+&>BgH*TF
zNs3EWt}F9=DJxu2wQcWx<$RmC3-;N6oUtJ~Eyp2OKH(d`g4;(bOJ?wBm}KXA`fjcb
zn^+pX@)LiqU8DvL=h!u9Xj}P}H~sPBr2-A{l;oQ2Rq`1~Sa87Q`qQ+GWZc=Rm6Syx
zy4<-QE-EjR!p$ZU=^~TD<T4MUHGIv%x)_|nR^aCn+Ie<?T|1QiQ6Uj*vGfm~n#S*b
z8$_)-#G^tx8lx0gN#}~$BZo_UV^+HAd0n;2^qf#h!&B3*n9f`HC2PgpyRx8v9yrc4
z)0I=E=Uzs)U5xc5I=O{VGSaXTE-5vBcKTCBmeub<6Ow2RBOwd~>hyU?OUpqp<aeny
zyaBw5<zeH*tLHQk(<4vKX*xQyp1OJz%L7N-_>riS?4E4+4ER*7G028X4qBp~Dp@xc
z^;SRRPnIpb{K>v2*?-jBI|s9SJcs({AS3H{Sc`n?SGnSc9{{&UL*jM&na(SmPF{0A
z!(;FTlKu{`Ze;|Y=F$WBZn%w(*X>6ogFmw>8GO|q_JaSihrJLdw8&&*uw-_{8eCSp
z68isx3>gw&Fw%5>Lq2Pn1slOQcpK^B8><NGk;1x$%{s}?x}nWFm~On$1Ms_wdM<i+
z8}qW}jBo8~SJj}6))2Sm8{fXwoD$xzGJb2dE2-W$>TohiV&=%vVKQ!v4wm1^KkTw7
zU=E#>Aunc)E75xa(!(;Pt2@ObqwuDmq-;}oOQRh&rU5Au!1!-$4m8F(ru_xUF?fr0
zO#25c6TgjJ#cpj{S*Eu^y&yR4cC5nN+b|2mv1tngLjiG~>+%(zV-vpJ#v=UcjZ-Cb
zh|m&X5j;wF!bAB!b$<&`o6hUk!lM+wU?VNFajo$9jZ~%R?8fH+3hpR<P182vz-+1@
zf%ka9E!_0D1U~1skZwRT2~ZJ9SNf;~{$$AWO$7R|g5-0nd}+0M@d4bXX{>Kcw!Q?S
z%YYxrmIyEiWwQ&Q(>Z+2c&Q1<IT2rbUa|l=P2}svOGluc1RmrC>$!QL1jh1$@40z2
z-zYhJIG<EEKs`Ay5x_558S4WjYx?o#J-`zT84lkxD7v7T7^?O4h0S!LjsOR+No^QM
zTfZRIsm%qnqeNc5kr#0^9X=YkUOMT9NIGh``E}q`EjX}DZ^sDrSd_2*I*ddqbJ}((
zqPX?C2Iyfl?*=})aFo`7`XX>a0u_1r0r*Y?G6;sHbqM_ls79gw+`6o736hvrydc0W
zaT35JY)J>?JFU{bywKaX<VauyFZ~$+x7;tGNrVajzW;NSGGNv#pd9y6%+lfuy5`vb
zIE7@SkcfaEgG8g-;W=$Tj*-{VZ9%lDj8eU9Tjhh}#n0kKFBz7LAP^s;V_})zI!?nL
z@<i-z$6|J|jE%Ip(UH+?%Nk&n1Q>I-Yy-A<2)v}|u}D^`mbna)M#r~^-VHo1fv@ik
zd_>?7Z~$3${Dl{^a?6W!{f>Xr_35as>szDX#yVVJ{quT05zJA`#^D=SHhzHk9@3lY
z&2c-H(VI3^qy;vOfXCaw8}t@g_iW8k5oneL=!xjg#=X$&KS%r(;ByI_;^jx+sx;X$
zWGoAyTW$b<DNjK$Ij5Lx3eb~;$#10d3YV>z%dIY(ixm(qn`|yys}KzWp|@sT>#opQ
zK~P(r2g$gN#ufb#97oc4Jey=Lm0ppAwSfeh!?B5Pq}uP%Xbzs57Uy=X3)A`<n`V;n
zrg>0de6}`)NBa}^X7Y@ciDPV`ziq|#ozM)q2z1k0TLIc!QNGDVjf<5*AN0-M$ek+8
za?>*j%9oPXszrSvi%qogUkN@_l&Ze;D+(|4vbQzqC}ea)#IVyNI|!xYMI<{B$)wYi
zvlGykB(XH5?=j)Y%7lDFTnammrKo(ovH8{^zS8arS!By={tS_2`%9t)5^?tndr7E#
zg|5qCIbM8)diy+k$={d2S9aa96t;q3V|?<JUA=w8b<3eN6M$iu?rs#V-YX;sl_lkF
z^vJD`0<t{Z?S5WR*R5>w7~Mvc>GU`TY<X(V7=97SDE%+$-K`Lk?|^TmnD_M%?UiDx
z%?6_og9<vf*hb4VV0afeDC}9QmyUIGYn8~oxBti1d4N|{Jb`}CxsY;iZW3zf5JJ18
zkpvJG2qG#d`Uk;|(wkHv6al#b8wyCVT%?9-s8U0R&_hRR=%Dl>L9l@B{btWO3BLF8
zeS33uc6N7mc6MgAoITxLjp|pC1{q87R4bm*@^?1osx)YxkbjnG@zT1~^$Q_S`1pr4
zSOPME(q)g5E_<$%a&hyo8TcxnVL0KMK3uh>A6ig|;8Q-8nt-=d_2KK6y~AQwBXE<j
zVAf^ilZy7>0~9eEiz-z((HfrLMp~OBf!_j~zX>W8^et+LO`Vn(f{O+6z6mPi9}qLs
z7{}=Avm@qn!(e1~jE?ck|L?oTbTMix_58#b=4m*16T>=mYR{z&?1fI6NH2|eE2d>A
zRm-=fR#l8a?PHdBj(9HSExwpIDZs3M(uYV~tpzcyi*vs1hAKkry|ytO;ECK~!3gqu
zn)4gsp`xgkG0{@d<RGrUiK(WGmLhggX5zsZMz!37C6E&qB(4B*-GU?(K>oBKNeG#h
z>szk{NiKj?490*{bV>mz$wI1K0D0PiFh6k$dc}g&Er4{jAdeM5-nSt23Lv8`$m0c&
zX+Zo`k6m|){@y}*vH-Hhg48d79I+q`3Luv)NW%ii?-r!7#SiNyv6yGAbYw?;*g5ii
z5cg-5L*7+i7-UYKS6y+XROI<6UT@=Z3NkVIPabO4jXWO>Ru-^|gwnv~)rveH&%8F9
zcmu*?JvychR*@GYm3rfQEqE%alQO3~OS!UQZXMH<i?L91`mQdyyekd+ZVV6f{T54c
zAu<PgYAtfenewbfPL4y)TT~-=bwN&!L(coa`dj4WIOGI?O|Zz>^9VV$?px$c@%?|v
z2^D$@LC^3&<YdUH7&+OSrxMfYm^k6i)SU^sg_C3CnJQKjzNhNve~O%pP-~7|1a{IQ
zadJb8V^cj$I9#x;?#xK}RE%(O>SKiXD!m8vQjyaH_6_k4gm&PSUlBPyR6VY*c$!K_
zP3_TRkcymv_L?eRi>Q*uw@xE7B-c3-ClJOJf>88|yzV}=Y~aFV?x3q-4f?5#13^=`
zGp9Z&rwMJO?fOGEqF%^X3%xE|s8fgLr|2zIc}qo-+u0ev&QEqSj<+lP-bv|Z4i=52
zYGp_s2dZyu)gM85<9oD{+_5+HbSWMZ|0g$0WnGJG0G~?%DWt#Jk+H8W&23Oo8PgsZ
z@3z%EZ@>kiQ+IMvWSWXhb$B?Nmd~itUPU%?c(Dmc7PqTL>RFy}=TE}jDjoSWQ6qGW
zFl<^QLftBzK(eS&(>??;v`S|nI071nf~Kvh(glbRD!J|wu2$&^q^t#*UQem%wW@Rj
zQo)7{0Me^U>m(KVluglegn9acDy@_Kz`Qovc3|t`IQ{!|<g>Ww8E3=_3~k3$IpZRX
z=nI-M21WLWROa*^n5&Fml$sGmC;`gMWf}#$ZF~~>1PcMv+2S;3?jsqon^9FFTX{$!
z$ax_<gCFPIDmo<ciLy?Pr$Ktk$<cxcQw$;FZ*cLtsbgYfk9g|1ElA-Yv?ILZkTFW!
z2V+j8tVc}W>=X+9GkQCP4kI#z_vif<9iznH8!IB~aDKxgdJG7kJJ^$mFcPGEUeqtW
zT^wX%b7VR*<{24+R1*Yzs~Pib5<m$8<v^rdQfD<qII)J1<rE6LXKeNBLg5`b=o{ly
zWG#xA(a#~{C`gB$92bb3lb8O5*<=VK!<EQdH67$U(%IUw82vX9q)Z4Jp|O!O<Cn<v
ziWHCIYR;UeRE8Wb%kW5SDi7rex(_DufYQo{)!FOhsWSRy4ddU)5xho`F+nmsOFH^B
zLzYs69#Ck2Sv)efCMd~*LX$IIB(NB!Y07w$$m>pv$>74qnK6;EFe#&>ljB{Gx;Z%p
z5=nEKOb)neCP&6Pl9KTW>FhRI#eYczwId8_sR@EX=2MX)G`5t)$jq22*gbhzL0HCi
zkvS6um$8koMJXA2H=yQ>j4u>H9iI2aa`WYy$a=*93Fk8Y4#%L>6HR|kZegAHLS%}Q
zHnVi#){LMsPt2?%I0<M?3#aw;NMAlqOG)$Pr*$71DO>fr?3p71jWU7~jb(wp^6x|-
zBZDjZBI!D931`n{FvTB;*u=0Cy`FKB@PpRt5u0q&LjzTYSjTy_A~xBkXD0t=R@J5_
zBAe3Q7{a2GNzcum%^i6xs-ooDM*M(xIQtV?DBr}Ki4L6kKZ$H;%hLx0c81N_OL_Q~
z@HHeCV$N&MMn-3@BrKB11s-w<$0c%$hbx31nUu}`1NF}TIi_nurilhm$W+=e%sLgA
zK9e`}6SP|nqJf4Eo}kiH<rsNwSZn>lA*ubD%Sg6cl)*e=cG&_&j)IevzbuO}`-%J%
z+hWw0<itvL#@6}C)(WUf#K--|kByq$+ey+l(qJ+Y(<GAv)eD{4lq52>v@Wu-oO5}@
zT%USZwUyeuI%1-HaDGlixLApZi81mVA{IGfVzTrc&o%2tOl(N6X9bG|j`%9Db{5}6
zn^QbY%xJ_UOHngs-xwk?nI&XsnM@VqbZBD4;~1xsRAyWfCm^!~#wp@)WQu!>E#nmN
zIL2vCgf>nQPfB@NO3gwu&in)86ru0x6i5mo9^({IKaAn1ky-(I0qYb|UyPF=4I%Wy
zI7QUAs8O@#0{Ir>6j5J{li-{p9L6|B)E7d5%towcJ-|3c)EDC<kQ#(a7^jH(Vw?m*
zU9+DP;}p?A&cbL|I<VnbCtzM1Z8V_I;>0*bG{!h(Ru$v)!39J(vnGt_3mVs3M|8wE
z&0UF{W<E)V0O4yUm8&u|3T;>0BI=@T=8k0UyJ`c;vk0hg*DB<C_7du!y$I{%<iI%T
zq6<aTE$!r33(jgM#||Q>4k6=Na4{?V9gOIR4xCq4$e^=kA0ixZ$QUNZNu7&Gf{b~x
z6s5^H4a!N>p+(14Vwcfl78%@=Lyj8}Nu?a@Ur5h$Nce}yUls`+@>CRiJ7E77F&N`C
z@4ArCU#U5C|D0f(1W-^BM&XD!Aps+&Ntja{6n{~>)H#)iRiIs(1ahEwjUo|oOzh{a
zcXFWn=J=cpj}fWk&`btuG|ETBVfd}$8<XD1LB^iXX>QTHThkl{wl89^znn6^t1d*N
zI_A7VcugWd^3aLUK_Y0EIeiGdG}nidy{R<Ws1xx%#_79Lf-{bEM0U<#rvXiMlbb|T
zr^(+9b!hq+ln))6upD#z<MNx_R5!UrM0J|{-6AIkW^&GCCr2icdHGFls+-&<qI!Ol
zSCGEULEcPcqc%>NlQqrK<Zcn~)2U|WCKC4${5cXsna(*zI3ke+JX|20vy2l944GAJ
zMSyY2EI}u%8yQi%7@aPzMdq^@mx$Wd=#sRd5sCR}9V9I=Kkb7R5wh83<r^+(rD~Rw
zxLCChmPAy|Pn;a+^<7XIV|QTGcR}&S?}4oEg6jC^mqU$x(>?qhO98Y}s&bF;H$?if
z7z(5*JwKWM6}G5+$?!KKb<t|h9a?ieM!yN7N+NY2^Q#Jlzm*unK1H&QoB2@ZO%NN4
zg&^6=m>bGior~^dlCRY6qT#KA(75xLP;I|a;jN13x=3l*F1&?K$+1#eSSd-Y&Al7`
zVvwA|wh~^n5|FIN&W1N*!dL7*W#|v7Ehs??D7Be|qUOT?xkX!WV=^CuH5YW0S?i2X
z!k?8)YsgezH1dLGR;Fj2Om)dri#)~hbRK9MG9~<Jp>07&O+w)D`Iy1JWUKL3_>)ey
zm&x{0MOG?a4@W^t=cu_|37xBS?%hd+H*{eAfc0g0iC5NDG3wX9!qo<B2Q4&h&XR&D
z<X2MiIcvo=XZ;j|t)Dv-4C$l3V9(K|%>fb;p6W%&7N&zSitvfuji5-ef>Q7{hs1>?
z7PO!;9e29%OG=qQHYL|iM-2`4*AW3qm3D;d2Fgc4Y4k+E^A3k6Xz?<@JyJ5flITS&
z<HFV9z51*0N>MsW+K7|kY{^<_N8xSVD&b`;A%*KRW`}bsms~%S((pjI>=-f6?MTYp
z@JPR0FF$Bd5}t!H7d@-5eorzUr^h4ZlKDk3D8hu7ljJs9`?B{%b^i;pE}75AkO^~W
zZj#}WIv35PyW$TJgPbDo*rgZNrBh1k*`z0oAh+Tsd}3oeB^klGdx?B}D9La$IA5&D
zw?mSQngUN1XQS)hq`wXNWYLkpn1yV*c1rpi38F!y?P#C$i&VR~h`t^^=@*|ggJx6o
zFUi6np0@>(e!-DfU4pT$r<3lN<d&p+3cuYSmt<WqpQr9LLRp20STDJuKbMho6f`Z1
z^yb{D)EKfb=_qQ$Uo->2KEhUncH*g|<57}{_+Nxy1a>|t*M`ljL8n&6H<gQ?tt-NZ
zUjU`dv7{e8sFP1grsjDKfkm}Vx@0w4&3ge@vnuj+fTYU~>{Vc`)8!J_T^UKY=&HpR
zXb@d>f-iRVS}ew5$sJmPo#(T-Rgzc$HJ5NO-26U{ob3s1B~popo`iQLf?Aq4h%i7R
zuxj2&LOPKpG-lpd!f1TNJM56;uO|npE)I%(0*6x8q?d32(0-d#-5F*rNiUJ3dS=Ke
zdr5lu1sOSat|{~BNn}_Y=(9MeqCYQ0@sUX4vVd-ew6Kv|rL$VD@-ULAoL5wN^^|$)
zmSE_GY+3)1b;}p-(e<x=n6yi}{`<$F8<y{?N@+JNG1%3xKn!FN%MpW|$~<=o?>WHV
zTP4Y=mFgm+^8HguEA2t~{lf4*eO1!R>acV+?5!{<X)Wc)r}2F7X3{#{89cXanZy)@
z3VBApvIYK4n0#j?X^Vaajtp*<=PX$Qq@Olh{Uo->-*`yUKwTv*SlXK!Kpt3Bkahh7
z<D~xZj-5KFq(Oj-WP?II9DSl58bCdRs>?HQ=o(Z@p2I|+B)zZeq{=pNNkd6)GH@d`
zb<e3+C}|kZ!O~|;>YwvZ(g)E>y);%u_3zqQB@LIzmRgGI>)%sJe%}0DT2{!;IiED!
zD`ZJz&c>v<ma<X`CDQtzSYw#bzex}CSmPr{i7cI>HAbxQalR5+y5vz1*9ci!Bp*ez
z$e8u9%!Vj&nL!)k4-i8@Z7mXO^w7n7I<)VnVvU}<m0F3!8hvy`RaL6(iddsm7zC$E
zF~jwTm4M(Y>u-Z~s!XH_8?vEGgP!vri8UIPmc|iY<4Wm|WA8ENxYbDEONlC2*(*W5
zAQLO%CZDjHH;W(}mSa1`-V=>N#(5VB#~5L62F3nb2H9-RERfNh*gW<({=M+CQr|)M
zcg1Zb;Qw6wdVe8^@JC%y=8Boo)L?jo$*JIh_MPu-sXI^o+zFtHdJt<Aks|;rss20i
zeb-PT-|*0$@RmeA=V2IOutbLQFo7U11V=qU3#?p1ER(>#wYhQ~57~rGysBfa+{x2k
z!fuHi<{_7G6t8k|HUwmgFJG!i>^Y8*F8={fZ}C}@Zxe1x<Od!e68>PU{=D73W57&j
zs@UB`PszalxtpaamS2`(42_KMpFh&?!Nl=2$(qrw2rn4qALyATn3DUJr8ZX74m?;E
zRL*^eWnxhqE7}JtF6RUSu4^iXJXgqJ9p<`HuS{2+_0T)DAJrdSCC&Vzxh$6Q$LmOw
z=jW;0a;`;w>EqbH;lhf4XxiihN@X&5GG+5}@>U+OXPLz$LZ@dkS~4rT^V+K0rGWfD
zTW$3;@waH)qPJkH9OdDn4NV^3@f?DT{bA|KmHK2}2&dchcBfws7pvRoMp+nM`D!3~
z1xGY;l)0*7puviu3Kf3V`Ym>Um3;jcyB~?M^;_V5aQr_rQp+5R<*G>R3(rw)W936`
ze2YiyN$d|OP-TCUOWo}%(#YjzqE#oN3X7kQooNVy)G4Ycb|yDQ_nI9hswj3Y9I;hl
z>^xm2<u^PQ%R4mYN~ssA_bk+6Ph{1mK=eu+zQsz6?c4KW2kU{Er%z(CJIDyj`lY(y
z#SVs{vJ{`0K&T0ffjU?>jQI0}#=t22{l}~!m5C+D>`6tj+hT`WgI{G%2lmYtsDI+0
z*kPJF1B}=FS+O5LH>zG`ZYN}GzcQ8=Q}h7NJO%7HbrqOse<^V@aSXp~EwaHc)<1S4
zZe|vfDeWA}^eyv73=zCh5L`CeBhDdEX4hY~)&rry71gCV2ahs;r<h+UUiy6^Lcw(m
zX4W+1MHY5Acd%U!-K#I_NeU=mHo~9BI%cicB6Xvk<ol9!M!D!FD_2HA2rq_M!c<DO
z1is8<$Z$ks^Qxc{{+tv}w<~jX!!XfnDYsBe+UZ+ep6*I`Ipub~x?Y|5ABbhaoOJQL
zS`<zBL6<c|Wa2=|58B_e$~~gXrGT-WQsiUADvM3UlyfqnL9odDW*|k|?$v9wJg4mP
z<;!!*t^&6$WmkdQmcn@j)pV^$(r1lR7HApGQ#UEha@*F9({5YJ0?<S+&&SD}uaKIM
zg(>ni(xMqCT)J4zPev#vo=RDS`CdDYcuztHft^oT;=m>WoA_Uuhy$g}v6L)5tVqVq
zn!f~CRNEBGD<Hm}uvWWmDeI-(sl-vR^S7r%@2(7g$`;(VH8Ln+_TqfEZS7aIWDfW$
z>OhJd#9Q+aF<XmvoqvRIP$D~cxJI}lk+nSBBixn9Vjlh>{3Vf@d{>8>7X;JwH#7e)
z-4yUG4)Q#C^L0xjuhlv?q>+fw&AygCTB~pI2wCcgP@GP@W+~mZZv{%O;iNO7uYI|-
zeYp42>~D&M-eOrwyKJ}O9#y&4*LqPdP1n*>(ch;a+E6LqcY)p0s`5O}5`k$2E?dg9
z=U|29xu;B*{uGz3w@R5FEzg1VYl0%nPZZ^p($ewMTDDAiMRy*&<qKlh)&y0kVfkz+
zZ|a@^jx@e4&srMgbC!Dd{Jpt>gtb9Y4F*`f@~YJ<BJg|Gro1N_tBXYXjY|0d5!b!~
zP5I*26kcUf=DI^PWo)Ow$hBPixi{rm@xxN3-$#Cxd{hwDm(yb+S=QV2SX(EX;|VV(
z%lU*xWo1i!y=;ybK&-Lmq|0zAwKAFGNFcth$67K{gjZyhyRBfX`IEOh%}YL@n<ts$
z)+cjToXp>7Mki<a^2b_oR>4?H&MFvd$!lb+y(?qwqX)^~+GFiT4QH%v?4!q8^0)R_
zTY%rQplEf!47TKN?ZLL7F~E8Zkcp>~r_ePU`w;I(=q9l9$<rJdYJ0(?|Bkgrpp-e5
z{GC147BHR{tOOR-HhI3)FtuO{u+4g`C5yzvbH7@!58xhauq7{rwHrEHtY!Pxo+52b
z%TqTP-W0YrS^S|5xs02Q%gA_wa7-d|dH9iVK_cJsaGQX`V{RPF!ykm-Br-y&g{(6!
zG#G0)rgCi~XFDn^4l4QN4g18-20eALM%WEzOmcVG#!#)KXE-BF55F6StP!;_*_s{Q
z$g@V=M(cD>{_Loc47Hnu?4fp(_5T0Q?5OFx^!n!4lgCJ}Z``DZTJji%+QvSX1dJ&d
zYRO}sgA>+JOCIYCwYt4k@>o6AWOj5QJ-MD10cCE|vy<d{1*0yxzV1f2%&5!R5LBU=
zHR_V}My#Ap8QUqjiS$oaSf^g?Rq`|X8TjJ5$<IMs_)VT4JD&Uk&w7U5ZESLL4G$!4
z42p^;TmQZ)*>d;op8b@b-87DT3>ub<q-XVbT@&cPG05lN$&gTqB~(2;(@vAxI<)^N
z8e>vh?as&uU4LehUMAW61|=PFO)48jfxUT7-8ZR15nLgeUVk=nQbmX270V_?Ye@@}
z57bYpfpivm=N&XJbNk>*+Pj)mOE2+DyQeOnREKd|G=t~&c20W2lI7+Yza(#=NV;b8
zK$!w19`qI}599HMWD1m6((C$5I&SkWJ*n~*3Z*q#acrIgdcw=zLhgLcv3d1>L3&Q*
zFF=tJKT7txZt4>A^lXTjPfCnz-><78g0S!%n>RTL`;6Ch8FweQ672g+yt6Is^*4M0
zGqaKj(8A(`u$orR%ji0Cs)cn4$xK<cv?p^T+2T+o%6nZ`3o}Tz;Kwg)O?X}+sDOp(
z1o_NKiSi7-EuRsaO_;_@Hs+RZ*a$GUNPIJ4y+kmx3y%{HNn|4rd4yY-tlTD?+nTS;
z?Bxt4Geqxz<+fAC&;EA-?v+tyoqp3JEVnNyl7Iu`Bs3A@mb*r|Zl+VVyC4DAtE-u&
zo>JykALsW3i}Q_=eM?LMmb{LuayfXW6YA$Dyutz-8sQ|f$eZc;P&p*{XrVBP&Pj;%
zx^U$v_>|o_>njPX5wiouNvZ)_7y9_Myg<+Fpwi(RysjJ(LHYdR!L5NWvx9t3zu<Lc
zND*7R*hTOTnpH&Yb|j%$Rcd7Se(%xQ8C0p{7{@^hnu)xtEQxFl{7d2fm%Ogok%Ay|
zf~*j#ShCUUn!}Pql`58gw&x4}VInW1H+x;Jke9=BQK64SUUFtYZ~OtkMJ>@-#6Vu8
zN?QfJq6-3m(OFc}$8^7}*Yzqy#N1)8woqy0W@`tp%QDxXv?H`8@1YmHuD2*i9QZ|3
z2ostxIA>Gc2vTd0^Sa(aEM>)X(ILWifR7#by4rfBDd-tmxVSLDlBK+^4%(grK%f_w
zj_%YAru493{>1|U_Kxn~5nyizaR#tJbpPJK`ZzGAgNw6iz!yurE<e@OdV@JXHT)3i
zb*1B-=SU_zx5aw_#^iWiUkNSW0Kj%E{!Pe#pv)boqWL7F*Y%<BoVGPz0``(9{n2}`
z>oYWvkiUe^TvC}rb8qIUqjG10{L;aopnA7gVqE3^1x_M?m!j0s7Rq(}g}}wFK_&eY
zUG5LSWjitFHYsgn8>vLF*YyRE9lOc7n<g!JiBOM7&c8gYCd`-N8O(s&EfoDpkhk%6
zSL0kFDc^gsJG)clr;&hP1e8RGW$%0Uo307jOm58GU4i%|=>>_U@$fp~Rff#bPRiUn
z4bo+dO3%yqwLX3ma!bb4v0QRsDz{(x4}#df<5<?!4Zt}*Ub*&-rxistPPSbLLw0Xv
zGo*$Tb9^dX+uIbJYBCD-H&4{@?UrTT(J4@5J6k-tGC2Z877#@2Tf_-9kSmq3_|WhT
zN;(mu%*GVGW4i6|3XG+PI-IoJA-8DeFHg$S!*HGXL=7k%skhRPGJ9LOQ_#dV!us7K
z;X*kUnfRjFenh6KGIz-e?CzPcXaJ$TM84vIRfzAm!t>($g^Np+T1uwx|CPvkA`G6T
z#Uzr$Lrp>r>4RfVAK=3*#j9KT<kLV|8a~GGRE?CmlQ$bowk6fl*rxnEOIwez(yH5f
zthLqFoullwB2vfbvD)51pB+J^INhc@Z|Bayc%pBOq9D3z=S7-V(3isgqx-BL+C|rp
z^#ALjqja=%(5_O#9IJzlvG^e?JiFcuxON6TUKlnpdwpsr(%dXh9YKx5jq6@lCr;(<
znhl>8yh;;V5n9rWBVXyQC#pZv9D!zEXV+@YcvKWHCd#X5`YvX!@0#g)J%-+r6wzLS
ztyih&et|8rvH!KOc^S96bgm>}G*~$^bL1UT2S$6P@SJ#og}frzehV8R(<#sClm(J9
z!b-Vdm-CjS*x0vh7_+N)1y9B_!gkC*7J%?rgO1d<CwC6l#n^E`1E(iKR4NLt2U_q6
z*rnG(b8e#tUo8xzvM9?Sj!Qo2b=B8;m56A}209XNIW*RnYyF*MQT5esUYE(n1Bwkh
zvwI{ZiSB004G0m@gC_dt>;R7}F?n5u0V?@4v1mj$#OKT)lG)hnDlRGKh%|5Ob(L`<
zZ(%day6M;(q%?WxbyadJ<9sP1@9p%uqP6Q31bd>$<kMvfSxx(U&65yhQS~oys4`Dh
zEuhLgiDGi7GR3MoRGAkYs=hqsHLp5spl<WpBXyhCb=~4uMfux$&3mHAQ<Z8)(?Y#w
zj;=jV^*{luF=7Jux@uTtu+;!zh$V_}X<L9W$;@GS>nsC&=~%+Ymc=mlXi5OAAS|@m
zA6!W<6SqeES5M1RJ$XsPum4ez*E~{DnR`~T7`a+h!a>6R80A{~qQ+n;3UvN><qnj!
zxM)i+63%Op>sd<aotr(owJ_Ebaqd~mnw#voDiY>4_aUKH^&)&dS>2K)&cLDFEFrz(
zHTj$%D~mLJZ&NrbEL%@gDt}7kvynuwSlX|U7O%I&qB}%y((G5Qk*^_MbCDKIHcvZ&
ztv8|fPU~)QX;B&BlJPEu?oRWX-%CD>=hAxw|6QO5dw9*Iy36>yHC?mxAL)=p`e%||
z>iZYSGgUfd-#B=W7@Ct<(~}Na7E7pVcZj@jB^|P?IpJA=&%X&K9W0%^tQVo{fKE&s
zyyn!BG;y!__JoiD9eZ^GFuR;m&l%=~N-%Ipn6QY-<b^rg?Zk=X_EJ+beF)vC)6oj{
zAz*!Q%g{dLkmh!3|3IU?K_%X8r_2K=iANEsMybw9?K|Z)U(juY{rg{x)TF(@+o8Q?
z!-oo!yynaBrFsyNgEzfqD`B0C!_^4uYmUU@)tPT-XDqOJZ&0;5hBa!<E*iUytW<f;
z&&Y*RF#BpC6R!OYfH3_qUy~Q%7YEAh3##lNWl<1U*``41Nz(P`sR-Cf8e5N;>0fz`
zKkW`0jH_luN(UWiQBzuB{247}!13{fQFO(u*8%$Mu2=_fd34VX?ZNz8NID1LLScT+
z+zDjw<Go$FBE<5Y%u1SO(*7m<P5-=K$l~yUlyCM2l_;Dk3=Wjp&vCT5x&g**p>lSl
z?pfnDCi1{si)+SR-5QHS#ywrI-8AE_-Lwc1gl0<BdDUwicG^dOsio4sLnW>D9oAJS
zwLF{<Vz<wKIL>O|a>mJWUrmdE!!HT}09p{9N8n+rAvQqdcX<y&cYt)Gaa?yJfPsWG
zfVjiPiAd4#07ekxa^2biuaWBjzX15T#VF&f-{#cvg+OQi0P*y_aXC*7&cMm^v{&ZA
zNfbc08%OQdF5d_09#I>EJ6U$^;4*~O;7p=+oFt*IGLLj4ubzcj=Lh@ifiO6ol;Wi5
zcijgwR}}O~aOoxi3+C+N3>0gohYuQPFbkQh3QHyR>`Fxak2AeSSIut5!l5fxM|S1(
zdav2EN~EW2bzv?M#)8qcwnXUF?(Mw!yN7<w+P<2#BneCTJ~Z<kB6SkIMt{vUB70|f
zje)|fERQT>cn!YZPNk$A$nqL|6U>UFz3(-?WCW@%j2QH##k#&0>qKt)jq(~J#E4{h
zdbJ;{0_zS2h5HvaV@YYa*C=ia&%mQ}U~>_k!z(P|;TVn{9Lvwbd~xHXdBttnS%Fu%
z{9l05%;JwrGnHC_<y+yS??iS=+p@Fb6@aY(N<+)qva`b99%x&jB0S}6Ia$#gpr~LG
zp7IXxV}K%2(u@ki4ykDlAp>Bbgx9Euy_7|P6>ABr0HQ+-XFRSr4sg$_c}g0{pZUW>
zFcMg>knftxGLBjqMsSOhoVz7Y4QeC%j~>!fIP7=01@2xh1Ak>HLP?1n<^fSy8G{%N
zYG*l4hf_t2hG;Ko5K_pU@x%vF-#lIbU2zi34MZw=jYM{I4u9hk_Z-fzL_k(1U@=$R
z=b;;+om_Z$6#*QS4a%oZF#^CQuILx|;D?}6wR(83>wXi-rt9JN^(rz{!}Vak{+x_u
zdI*-^w3P0SAA)N58D@oLf%T>Dyv`Yv0kt4YAFVpl4auc9j$#RAB(I!Dm{n18^SI6S
zmFt16g_Z%Rt6VwqE3fz{DYMfX-i5X(4d_V;-9(g*y+D}|w~#49&I~`9a|qiLG?fdr
z$|L?G;S3c)?P6O#R{jg@Z(vNsm(W0&tFH1ALV-c#_qK4XssyY8FuHuXP2{QsU^Rg;
z`CjGZMYF8(%M$O}M&8v9klEoXJ`%15WJr%G^Qhh##WV{H*IL}|qtDx2Ah1i8@;S<d
z;P&Uh)j-UwINC@13G98HcOdgH4f{l3_(kL$SX<7a3gYBPGB>gjmk11fDu`HSjmM4P
zj&d9Lx(z!nFuM|46CS-GFuRhzHr_u1L;R$?VHWIIQ9YTkD;Z@2nWpAf@|6wKT9vH{
z`gtJp6Wz~ps9O$^?0d)ineJxv-?6bUlHUESob;|GZgRa|l`1^bNn>><@l5A@0ji%~
ze2p`Z$GojsQ5Z4|DbZ&cb6Z<d(z=3WZC3T8U*4s!5QtZ8fmrnou&;n25U)A?_Z_eq
zz^LnWM<8|s+aa)kSI$O?B(J(eK*0P$NJm>rvPjHw%bs3Z18;hs8rnSGnuHWSr`^yo
zlyUT9?=ZVA*}=%Fjv5(~2P04sCtlQBy54R`rEJh-)gh$Vd$>b6N<)#ND?_~RG2)MN
zudC@_m<~=OLPKUTvWW`@p0mb#WD}P-o47VK|Ae0vw@>(Gy+A3=C|-Ja$RE}Q?eV)Z
zAZ7IhAg6G&7@hEl{Nl<R-MMqwN@CwEmnrv&ijriB`_M|3xSvQ77<hF4(5ATP+(g36
zQiR)Li=1JQ`*_t5OsD>pHnu~sPsl4H^JhZXtAy4RFytrznWY;nzg6S4AX!fuI$rZ;
zJhYec4oAhF;Eab}PRi;)zIIaaRK;?VuDFrNX9&OYBK%VfQ+s(yn!UzuAp_uDqoE*L
z27gF46LM_fiN7U5F6mc87|pBX>`87(foTc1k<VCTKewjT2Q%p4q4qG!Z6L_?G%xfh
z5|oe(sR<5=)*Ju4x4EtL(vUKx4QTEb<QbW49&O2A7Byw{D63xL_-dX4PwY!0?@L0?
z_q1JB6S5`}#<Q9*0JF*A*M>aRu<eMYV;Ovf+j$%7+Js>@14~Z?)d)TriiEyeIB@<%
zP*jcWkWfh!@5#3zfth!0TbXCgq4?<(ns!0SN?nEAf|WW74*AR*;}m>S&h&P@=Z(Q<
zR9$!;+Ql0aWj|A73?sp4DLC7_F)6^UBVgXDGO)7#JBTL9772rn1L9hRH&$xXggWL4
zl*h@R{J17<a_g99DGSX&S~FiAJ-s~@j5U`Xj>6|WUHbJ`b<BenPJO_4>)7)+HlnJE
z14g{6_EhTeF7v@U%XLFYl#R>JM%GzgQt09^K%JFBVQP`Ej;c-;%b2N8f2n<mvG3L4
z_yQS3W=;));>3e>0)=U%e);&+XF!gATxW*W2Em!4A%XW!a`p36rKopNe4XCZTNFcb
z>K64T6|2*`h}6dOie+_rqfGS1@6_dW29yvyTyJS$5bAAFyv$Dz_`-~6O?sX>)k!JW
zcIn6Szt)gl9g-?r#|g6N;;aod`zt_boZ|iq#D6AS0WqpAP<SLB(&|!#NPyb{0G|Pf
z30>Wg@I<6+Yy)f|Kt0tAGpfO*%6nZCaunSge;~~8L0K#xeYEB`Vb0lDwHn%1zYJ35
zf-obV&m%yiYr+Tv$oCt7_>_5}uxr8vHK1STfz*<~5=*bKt_c&3=)RrO0MyaFKp_hV
z^YF!%uL?8jI>2mz8z`&ydtt^)(Y(;p(lz081FZLUh8gdBWXcZ){@Cg>DC5%^HE9E@
zZf7RgD5*@0G|F{Qt}^mZ=UQufXNCmgP6yTEdwVP^$~d+zy&6n@3OZYf|NE4AwgGil
zDivmkGZZ(-oDe68f|N!jKTwG`EX?R3U7JM(4++1>=T9&FtjpkhxGt$hn9)6gL3)|S
zuX%zXOWl|L<-y;KpQmw$E|+voxGSpZEDIqrPd}>_&@g4`F%+KBNEFZ6p32o%4(CYU
z@Q0Urc?Ti_u`HM;(=ps_be2~s%*z)k8rN%0Cy5;AVFF=vF+r0GZ_^s?#Z;%@h^|n7
zoLkS7tc3wx-|4Xb)>n?4HS*M%7K+C4i@1jw|Ko<BbMjW_<t9|Il%NTCoQK_nZBkte
zsg5jdtlG};))^kL_3<XXZfLdd%t!yJj~u7A+c_S$TMb7ZS-Cikb0t@ooqIzkvxts5
zo)9SvdtCT=8h%dVEa|!4cDe@QXz5IKztGSNn#~hf1MbGnu96E|r*rY59IU>e4drRv
zE}d&@;ES_CCCj(7aqz;ljg)*iO|qDOzGPs_*`UW>ucgc%A14~Ci=oHBIn%_(X$nxR
zXp2i$C+oxwc4B8cv85)>vdYZ7tkOpYTAT|i9$H-K&wrku8|ZZ|sI>bWeI@JG=g$Sk
zpF_6kt2)(%wgTBMDhr%4BvA({F+^U))bSEJ9x3ZPy|crN;;1{lWP71|s8ZQ)0%2qo
z%^+evO@yA+%WEH-VMYmtMtrnhod`3cz)-0ktx*E0=ec;h9E!84=&p<Mx+F!NLkXc6
z)tXy`OTZ}PTq7w1Z@eTST%V{2yP%5_51k-~IH>t2ficq7b|-WOR<=f%KCmt@9G$g8
zQS4!tA_c3592X4pUxt8Za?J^qdSDyExMfn7(Dd=xIUB=%t|WAkZxrDZ+gR<1(yq;%
zOry^L(NN)x`BTE5CLm9-+Gd1`pLdSgocSZ~*68FaVMZutR4;zcwU9p+!J6|%rt+_y
zLeUc>MpNYo)!NN!EuPY;)^Zfob{&w4lWDx{h#X&BD?KIxY)iS`RX&Lew3v>&xL;EH
zYdJc((poaTC=QPq@;f;!!bMM<#KRFiQA2NP$a?&<&tTc(pAS1ChU=UVUOR*ZdQaF9
zbo7t41AkuNveTNalp1k4Y`krG_U6)ii=_81r6mae#@j1o>(FHDOgdm!lQ8`nWh!yJ
zx^;2D4n80Dt=)O+UIAvg#$l7~&RaJW*dR28yx4Q8gVozJg`cT@QpOX;#IYJNDQsR<
zH2!6Jt1Jp+be`IGOSD|^7s9?n$^G=CD4g%Wo<+!z$W$H{6BeT6ioYOQXN5VT49wkk
z-_jvJ`Gi$ksXn6P_E)lXj_HT9`i;5E9mupsZ}hYcn@O+!G({>PdEFX<d{U@5A5t|h
z4~IIbAlt6sGpRtk*N;@N?}Af-F5jLN?0a2wD8D#dVN<yr=~9@VN*1P5!8XE{7;9Dl
zhSVT<kIV{)^WKjQ&7T!maeG#<?f|9iBb&~T6DykOk{llpIr*PHRj}VyMwi65HT~8d
zu$H#j@8oDHNrUtpVE<s5TS$B$#Uf|>Fa#Rlau;I|wPK#$JdA@BbS4Yy_4!{zi_g@y
zB0i%kiD$+V0n@O4QDMC_9yI+d=VHQoMPm+n%|<FKObY9Tv=(`X=d_7oy&jYD@YGoK
z4(nfmD+a%zYAUc_4^AUlW}{iZhP|qbM7l4>=rUi`=Y%K*C+5}X1#Fe_dRYv5xQvUq
zLF8PP7WN!D{aWc=ep_dKPP>s-*5{rj>)Te=W)OgtC2P%=WZix|%sQJSm5$aqt;0zc
z_7j5rzEF3dUs#QN>asOr4LzK!Drz)VrvGxbm8+(eOX|2x)>4m!mCDchP-iU_WAloJ
zm1;=U53QVK@^i*^gUfSzB;@Lx%DlqvrW%-*kQ>8vWKS@j{QZP;)p#Br*14}rxJq+d
z9lZ+t`8N|pPbLrN`LW{(*PZ8zlM-$?&rdv`aMO8yZ)d_S#&(hHHnMh`V3fs^|L9DD
zQ6vi0+b7WeG6%40(L;ZTBwXvE5AG#o+fLKEpAga$)foc|6SAR3Ov8F~_WGh|lFeqq
zRwh8#dJun!&;;1j<q6vz*brc8w%X?AF{IkR29&6_3HISbwVsL8`teo#vMHEw(AG-p
zrvaRT>wdKfO-y(>VO##$ylWh}zODy;)SLu)A>rx+#^g2h_WA{cc@kO6!v?}SiJ))R
z?;`Aw$V?s%6Anq_YaX!J>rWH$U;B)wYXmtwdks^y{#U|XUI0j$NR8Pt3Feo-gcU43
zT|LVpU<!(D0|ag;EF<kYp27cn%)hA~+}GP!J8IXoD;uJ>bgpAi<h0*4aU_$>5O)79
z`>;mT!Sz3s`6LT0*PgKkGl_a$NkunU96xm{5O*~w+TW)(bJO_=i)BJ|?J)8K_ZLU=
z+{2Qk#f<RUdw51>mcaSs;XETVOYji%GcxnN14m}Q*KmE5Heo5w2gV$gku^BMIwF^+
zW-#}`sX$O}RZQsPM=smyjUBjvU~h<|tQCh7I@toS0g2o2YL5ZkRYF(H+A|$3%YN`?
zf<CO5N-83<Ax+zugg$onZ=fePj8A7+Ag!6Q<}-O?!t6Yik&aOcXOZIiP^v*JI(3#o
zMh|bq+-)qxnuRO5A~fboZ}H2(Oz49tx-M^`$Q{EQs}L$n<T)Pd5E3QwBo8eJFG(bk
zhj$2X!s;2f=;a>+ORoi$Yb_;b)RU6=<%++A!Fbg--gZiUk1Tzhl1CEZGZ0qE2=hj)
zlU4E}BAJw&(L$G8={j#?=#nRiF$=$t@FDcxSmTts1@w(hsTYW_FlCi`j|gYGb*Y=M
zx*PwIQdwh3;a<7?&sgiCS^1&pTM45n`ljsn3M=IHj7{>gO0pkbY^qD(z7k#bre;K*
zbIN{=NGn!4GEnC!KRP^~$%bN{;#W8^2~OPnz$qAIu&KL5q}*YIflj#rB40b@&Lc7l
znr9Ynq*?LQ4Oo$GndzH)8B5=Uq|LM$QdtJYj78WYMDM1&+JKiO4Yei(jbt5Y#!)_n
zGMVH<B<s;4WV5g~TYQX++IHCwB(rtPQpmruQbpk)&MeM_gtPggqMBL78Oi?_mYBt5
z@5C>Koy^(ITg&wzi(guxu;a6|!Vc!O(>LR{I0|@HyZ9}@wZe{V5x*5pt`+%#JMr7~
zvnA{CCkvBt_osmlH-j3yc)L2KQ_2^}tRAYA@<jvtd0<-nxau;r^-zr;mm(CAdFt}`
ziHyD5)orDlx=<_K?SWf2QSu`&RlEK1!!`YAjN84bBBW*f@MtMkzt<Pf;WSnIA5BO6
zC&egspgy%4FI)FC8p&DHDBfDu%2U&(;(x9YZIE|A0T&^@v8@a?;UH~V%_#liW_&YE
z5OM!~e)4*B>@Y~hzld^ayv<Pu@&B`j{$~DfE=<0?v*PuIby>urXmc5@BI4h2@DhMk
z6V<RQBYsexnx4_4V}FY6MzK%FaV@)Oip9$OIYn!V_-3+qBySmQev!~jUWCoNPqsFC
zXsnHYi=pxJAE^GjKGNF}-jPUo9(ohxMcBI-#LW!s&FK<h0=D@J!WdW-&Nr;gyJ(Zm
z-x2~`Ss}xYvV$ZozJW93>TaGx`Yd~BY+g=ei9Iwn?<BI_9vYhu6FGz+PVc5i2I{AH
z;Lz<Lw_iveCSEDO47~X647@wp4%uhPCdN#YeVfQto9t}5Ih(iCG}+lDh{%aBp*XuT
zp+byJ@h}yfUvxgz*@Bj$iT3mK3?bE~IvYl1!zitUvilI}Ayj`=poC<?&MAkk^oYDx
zXCS}%yx+jH3eSn}0PpjL*p0}B!`Yg{@IHGSVYJhTS-5}M8BQzc%WRoPgy%R-SqCJ`
zY03uC7t^JR;sqU$eG-%tx+y$dA^hmH<N=Z2?Urnz-?o_25+-S=Q+Xe7fDo&<qNn9p
zdL(aBz8<lr691Dbr{!8IDo>v&vsF}-u%W)FsOid5QF)x7u~d|h8Al!#tnw~MiT$<T
zK0Q;^7I7KB&*W+Mxq0dF@pYZCd$%#2%IK<#wZC0GaOX}?RB3U?88~$jeto45L%W{V
zL4k^SL1kMGX7D`!X0S0G)vx!w+aDl;dMT2v_E7G@UxSUWB?(LZ)V5$_Br4+W2AC1j
zGT8V;7bN@B@#GqfH4pIyMpJC~Ly8qx8kF397+9GXRHJrQXExu51{-f{a_O^Q;_S-b
zF7Zg^ZzFzGz7DLh-0XLk1={!Q`kwhuea};0e41Xw2sU2uN)gOLcuAyDFbhf)f6gYt
zVAA)?IZvOSom8+9L_-_DMS&Y%4L00hu=LR#xJN(YU;c8y)1&kIK)jT%x_I7If!TM1
z%2yZxU9bNTe2tRt6@{rgqJytR2nNq%4+qYH<Nsf{&U!X7A0B+Rl>H3;*&6bU9t~;}
zd{OHAMp0<H%fVZu&~yqSVN19q=t6_HFyqk`ka-J=ngw9YO}~_2j!{W-&3m#tIxsEx
zQ*CY<Q}odL!DFk*1c7JYmEbS6Es?3rPET+*olEMvX=UqH)piBJ?4BYPD6rud?n<~H
zZKboxWx0x^rzT0dR{X)XYNpgro(QhylzRVFav>MhpgMWDTs;w7jSBP|r0=QV8s)@_
z2q`052ba#zG{(wQ8l_C8f$e*$;J|D5f+C*>w?1Td^O&+OFRlGGIkK-J8ZD^0bBWZH
zIj0s7kJ+C+fnSq7X`R4ILs*`wO~R-`fo1o2|Lh5mIox9-C*Sv&BQ*j&{%d8h5?&^I
z1!WxRtctW(Ds!O6^wWC#%F{+y@Nu`y@tD>gOP(4x44+q9xd+#43Q;kdD3EC1mYABI
zy7v=c4NbV(@&<}x%ZvJ)4$#NBy7ha|z9EbOCH{cNluN9N;A3t%K-fy-dQ=67c&OCU
z!c~u1LpP4o?j&xN#rhiSJ!X7}G9O&WW=5xV>Q9Xcx=|qDziv$dD6po&2^95K9P6#!
zC@!ps$4sUeIUl!mD`6AB?G-&{iU6|!TqK+T*u9;{Y-j_wVePhs0_+<EP(%~VZS>K$
zTJhFi$g6zYl=<AXXt`0qV|H*djs(X)EV?@|XY+R3G+<NEN?u%n-{w=Q^gs=(2h=v$
zv+X-p@06F7x!>!OJOb`NV~_G}$4Hi?{G=-$^H~RoM%?!E3-IG%o*Ms`Y|HYCN%WX6
zhH{0J`H(K!_9sE6hYyR>VnlCyA&JOC8A2%x$HN33e1sT@z{l;435{eeQF)tEd{uo6
z%42r6E8fnS-2Ory<LY5EP&$)R45}Zd$xk==c?+ZD_jZ=@Mu3Yt-`<B^fQQ$3dzTI*
z(+3{2D09f)GliPpNFPP`M3rMXiP`Ohbn&eDMG`!w!Mf8QvUI-?!1gZ*6Jo4oCopKW
z#(QMhi8!0lejA*>?qtR7b*JrEq3u&CUp);0937o7p6AzTZKzN!sq+cnNn{}pD+$Xb
zvWADvgpCr}!NVRxj@>x+JEU==J!Xiv4MczBwC)&4M|4$(sp^8%OY&?f;|NIrwx1Kz
zIUz&#uSwb>w$W{Jn#-CHa~C_Q+m&7)`{UohqThq+M6fOUZ(o^rPb?$}fxO>Y%tjc-
zi5Mtv6_5D@Jbs|5Ko+*&CR`VikcAz{>h^ynf-LNaC-~?zZxta4f!Og3p@BpYh#j2?
z?PVj!RSN??p|x&h0D)(*?g6(=xoUj_#vsCgvJ(58rw<7ugz9m&v+&?}VCEk|QK?g9
z&R>RN+R3t<zG<-w#XM{W&56b)^|(^YpYxcd#il)M7O97H>B=&sA-Q|I!0&$qMH}M+
zr5^-Ud|DRU(G|wSN?KbAuJQ0O!Hv{I(s+oC)UN?OcHCq91-7%_tFG<QV>dT2^g&Q<
z|4(AoWkiOu7<c4d@UV4_MnTK?NRPoO1!ewO$gb>ej8rE35Np<WjID-fXGs^weP9}p
z>S9HRywZnABU^d<u6T@Xb;PDn(<f&<hAc_wx0afW^aK(f1_k+lB4yb&kMToE$wNe%
zc3P`JI!HEOt+`BkBxz}<J;wDC5P1=m{$~ad$T#+Lun+R3HIkVdDj?Nvn{kwx3Hn~i
zvA{xIJ>uM0r2i#vOAU;H_K}zuO}mi$Aya)#E)*IcqqiRZ6gd7csHQP0VEh?W%&$!k
zTG*HX!P@qq*^P<X=-33eU!%GFM;n=?c#L)U9v))`wc&<z>B;i~hWuYSG~qA#{*N@_
zW2*@QnyXElF%wnAw>|{22BfEa9%C`e(BJ5yZm4^T$5`%_B6Y9T>++^j=@&glM|4Qh
z*1)gH?CkFaH7uTV<j6VQr*?cr5OXAb*0H{{8^9()7C>o#r(|hAfNO*c)Hq|R$LP&}
z!Q5Gt5wX)vOgi)Z5bEgy0NuIs2{fldy8j+SYR`opqp$7;QhF1*-4(hvfFiGaNsIfk
zgd_d3a|+>W?fVu@<14@w1O5Nv^e|j%ihTX!(yuytjOLh5{i@^N7i^ZjSXkLFN&kKC
zzaXdUZ44P)J09~GHAJBQc}2eoM~qI7JVVAQbv;H+9L7c`h=g|Y81eL!UymOVfkZ6=
zcF$UGKCm2Bo4YMWcye`#mtg}kkc&JNc?XY-T<kj`H(}D*XLVl#rU%by(d8LAKChqc
zaxI&hUfY(KN@0O7|Hhjc<*_&Jq`r6Ncs59V|H={{b!=dl=HX8iR_k?-%<URo)bE~p
zw&-W%=3j9Yae}tN@kSSUW^nAZ#s(4kU?)QHZx5&@$`Cw&#Ob99@D2-}KUq>hJnW1i
z^g}&48`iIl=Sw=Y|0w!18$oj!KUdd8eMylIf1*dX5LQ=_jRDUTk^C&;Hwo9P^iN~-
zObNGPe*n9$k7#+O+IYM0$#;do8M%|eu-<k^ufiH1Us+fEGZp)K@~aAYrbj53!bs}x
zRqu)>8!wLPPjT;hf=E3Ut+Z+FzeY?ezwp2_$4e%eT#9Ag^$g)@bekOW0s+ga(X?gd
zBPwzs9{^0N4bHc$R@}C%yIN4pOB655WSY!NnoNEnGou(aW;<lQ4eA>XnSF`$)|vjL
z3YcO2l4)UK4kD^w$+d!h*)^2#zC;-FyFMX&<W!YbNOH+9xt72Yqoz|8Zo{r|PF0JD
zEO0oIR$Ms3uW)1;95Gm_v^cUFyj7YbevJW3G~Of525~rIL-RRe#T|}prI>7rcN&*g
zPm{?nWUc|rU@_im+kQ~@I&C{eL>vD8ll<CNy13+(>2XLWzOtbDpV7WTtWxg|@C-00
zER}=1ro5}fkQh)^4?EPYOnI4;bQ{n?c|K%7BtH;Qo?hQG1i{-P?}umn;28=pMKBTQ
z^?f~^VMvi%K;%fQ%k^a2Zhpz#xtdZbLydQl@7m9JIwx9Lc3p?e%N2F4UG-om{tw|V
zU{ur1E!A*G7P6cpcRo>_Y}re7Id~U2kF$BLYhz)+JPZQEOb)1v8P0i@&_tC*{zR5Q
zgI?9z)4eEk;pJ*^&uI-(3)~mg+V0^TypNm_a&w`4;ae50A`F*b73X0&!D4pK7a)zX
znVs`3k${pDa&vontd~c)*Rcj!<Rx#vtI&=F&B7NLTPcWOj*!3HV#Nz6P~DsZ!1h?I
zc+qA>&Lv<!TC8}<VFj*x&d(Mr^xTrNcGDZXgJ6ZO6IQrRTda6dSi#%cYIhPLk+mbu
z3ebcVJw5+}6@2Z>V#RJ&mv%oXtmtX8V)rXRTFH`=W<~LyvfQTzds>lR=t-LuyE}mN
zj-2E6%G#71mMo6V1o0z}@U&1&Z^p#@nP3ee494dyJ_o=>#L+q_2~B~PuHfSSDU`At
z*I(<N-3-s&A35Dn{F-!w{L&3=r1xYz@18(l^;VB(*gy7pPc0!aHBXh09+P{GT|qGx
z_iI(QxW9WAnI>D@uf-4$?(b&s?Dp@r*q>~(e>WO<_eG2SDK`6e-vM^pV!t?*l6jAZ
zP#E^>I$?j|&KCQ*2S-cdo){3K9qxlA+#m0$4fhLAwYk415o@_8Uf4h0=D?ok$cAPW
zWr!q}_()hFzp$WwD0z3=Y}oTMh%KBhu#j_fUrqHip|HZgIEZM?J#Sfv>=sL3mH0#V
zRi@{e!YE4JPkXw7)rEd4$xK7{lNo~3BTEcL2D{CYn#1t*E=vqVqs89AiueOWkY;;{
z!EKg{5+wYv{-Bi?d-`4wxL>)=3QnZ{Ah%i3iJ1LH1v(ZEj@FA<VMW}g^&%p9*7b9n
zHQ}-@I((?xjMb46lz)-rgqNx8(FF;&83)Uq1&NZ3@eyCUjs3Q0j7g_mGBRE63M@#F
zpqoIW8Jm0h*YwMZ1kf6^ITgrU!Zf`g0T3@p79{o@B<uo%RvQOO326Wx5bnZ?9#sJ%
zZuWwN+c>IO!SW(x?Jet*1qrutB19-~79`w8u4V)Vd+#e?y(lXZ0G}1U5&w#ijt>H3
zj58v6-vB^C>=o~FdquZ#R)C1(KAO4r0l<di0aB(0#*t7NVE1-z<C=pB&HLm6fWAQh
z%i5R|{Xjp8e{`85y_25DyUlRP2~YN2AaI$fD@4|l;J<Oy;(rqeyasq8rH+{6Htu-s
z48M`#7l27u+(w=a++S3w{gKSvwa9*flFw88Qhy=08HV77<k*6{zdV>SKNQ0)9kaih
zGo}23^W9YXFT`QBLmq-8Q_B5y2(s=jGs^u>5mF_GJCQ0%$=L9EtQ?^?L2|zc!b=1$
z(Qx_Qf{stIzbC`d>wysWo5VgNe2O0`%M4$9B$l6FEWg{>jOh>k-ZuUF>8bq_wdvOw
zSbmN7h~+1a<+q{vmfwnt!;%FX_s<|qqkP$2V;74FFEipVkXmN>u24CZ$BlQ{_AEcW
zjZEgpHJ8aeDKc|vj2|SE=rP%P#oguBXXE9toOSey+?Ay4u^fcav+8j@mFrwfRWyZs
zU4&Kbct|u7WX{+AbGNY;u4zZXJt(rX$mZJ8U=2bfx^4^93Josp$I*P%kh&o!0)_E>
zXGY;WZsT)u%0$=Yc?|4<X+T8(S`%F&jcgU-LpVkkOmy7_OFqzoh&37MHoh(;1rS-b
z&27ldNcKe6ZOqccD;~jlWTu-2Cd$V6-Yrf9vg(o7okdo^w)C<!({&qDv2Q_lt?3T2
zN4)gZ>ok35g4=lBX+W5)1xf>+uW2_xXnw((?IOTJC*~VE1FhV_HT|R9#%tuzuf>J5
zbQ`Zfq7#ib^niybQ6g_@m6mCA?kCD;q(^hO6J)ywG4pjB-JD`tz3w*NqnNN{62l_w
zHfdA;E)!i>n5@z6+vPTRThT@Cz`yR`68<Ad@Sq!Rqk;Hm?VxCkC_sH&QRr$=S$KfN
zEg@twz@&{0?3wL;l->T_0Hv)B?V0WVp8;NDL`##K=q7Urs0bkx=~YcBd<&_<Z_c%p
z(1!2;F2;d6BB@2v^&~aZ6I{{nP0+Jbw@ecC?9}D;*sZGN%uZe23jYN;-DP9Q?DW}w
zZi5pd%H{2^XQ#yIBt1Kw^P$^_h(eP}HX*o}COBRMrj0V}3F&s3Al8U;U)P1)L-yzx
zIUi(CNZmJeF040w@U!j$o?-PJ-A%eQMu1+~O<&0@wB0X(Sz*D|s*iI2t9wt%uYJOe
z>!V!Zr70g$E>;j}p`5Hw19!rLOVu1_!8um~oM-C#{xP_j49;`4-OHpT`QiikN#5Wn
z2Az6XL=U>mur5f7J{zQOlYQ*C8_lHkKNT09>>jTxzHU`8o_%^<asOHFuXLe$=IfrU
z$BQ`n$@kq;^$_P7A(>%~6C!l#3<M_pI)(dXppT*a^-1pGQhvmfln=vYTfC7JLjy)Q
z%o@|jJ<_U67TMe%>SvpA_8fJT?0dY`$lcyiHxchy1h=m$1c&wO)?T^0T7}?N`<1&l
z<0$!o_}_ybxEtzhdAvZ$=DHFgFY=^z0No8CNZjuO-3jfh%S^=G5D5|cc;HLI=!&{W
z_%IUn5MN7JrDxji#=1V@hY0)hOxyjmhM{y0Tq0cfZ}(4P5O``RS7bQ~g<rglXAOY|
z;Qs2k)cuxHz4Fv{KZTKrd=cX#ZtX$*`Gd0UE!OT}IYKGz1Tu^1cVCR8{Hi$jGZ?%`
z|40y*qi7F4O-QAj<^6c*M(9E=YvvxOXYTj_SgU3jtH^t{u{zivy!UKl)jv+p+y(6s
zV+B+u?iMuPSXpsu^NX=MIGiwyd<uipFHejWd}nh}&p3hgM<0mYDpEzmc-=`aPNz>2
zWzfI1bdn@kJHBa@!)$W(0(V`}R+Wf0(j9rat<0m`b*nuxne@BsCW1g89vn^h)at`}
zb{`&`0c@Jg5~UP;^CI)D{@}>9?vH~DL74AMGm6v@_R6oS?S9;u?HybVj$8(z`(rne
zZF;u%RHQrJ@Po^IBk}_OHI+rjT2#jC`G!5^b;l#rLgXRD^Ps%z$<RmNX+b-L{2nT1
z2^!a%S$GKRp^6Z$3x@Eb?JdF+ZNd-HA%|+J=k4ij<Sp_s65X{T^ehU`Me$WVEewZP
zE;v+I0Qi56eP%^HM3k#oH9UBtRAqbcL>LgZ2k$ehGHi=gla*ATSq&mn52HRGJMJ^9
zYbuG1D&+T>HJsS0Cwyk2$1WiGzR#?!Goc-el>|BAWr}>%PxqO%3jFlck6JU=+2S*H
zYVz{bXUKs|4$HZsI^d_1pb4cd%?MBFjYFT|-zmM73S<DGm-f>^#U6;CerN_E07m=)
zpRvpK(+}+exCLO3ssOd04&XB3VrkKIK4Y(LIM-TG^r1fhh86J{&Y{a6YLO5J@bg+e
z<Dg#}PxU`MM|c|OYe_!ikOOQ3@J)c*EBcJX4zLfv&OVt0_>5yA(mZDp;4_YEj)8#8
z{UA%#nT32tu9Nd2z@PP&q0cxiz%0Dw!)TtvKI>`$pBaV6*9o~1?(yd;{xyZbd1J4=
zVR*PH=#At(pwESKjJ@LfWUk>eegrf1FsZ!>J&A0|^%*~*gG6u-e?S-{J9}XVm8(Rm
zmF13HDyH*HBKaFVP4MMA_=jf$oJm1RSA52ODF{X#UP@SGm;Sp``c{D1=z`A|!H@o;
z3#`kgMX5mQ=qmijyDc$B*K5jUV7l7l)PMM>Rj;XETaYR)63$cF=vq!`mS&OC&`h!w
z^*NJ3{fo|mnPn(@D??o74u=^_hi^y(q{Dv@ev@2L>nU)U<l<MUVLr2D82T@&9GFq3
z6gM0a42kUL0UdrMjBOHE)Niot3_~&McR<Xtj+C}15u?W_^c%kjozIwtUM`t#>*XV;
zxFc0;y$lAE2#xoMUM7xS9_^Rxk9>&3LZdp-%SVz3q8=3=WiN#u9zB)kHl(tBtBy1z
z$Y-}pF%diR3gLf(GRIQYC3nyZF1In=(bZ=}xsTq!tFxa%=}AjptMUTOv4OWW&=jRU
z`kLNYJX!)T_%CY)5>?;I^T-TD&u%{FNbdY{Ohrb6Sav3v$Mk%3B69)f1+tlwMD=gS
zL!uEbbCR0B`HUr6R|uEa%XS|CEf{(Wj|d8bEr`GeC4(a$nVejc-1BmLM!?BkDw@G0
z13yq+t3`26h~OhQn|{2VseP<7;z52n-DgbJk}I*XoqWa&9AEW~)|7^P1W#mBw)w_A
z@aZcwl85^tveY`N8JGr0ybo*oL!^<de|?OSZUxgHpV23TQmpBZ&*+D>D7B9ImTmJH
z14~LCB0}_Vt*zo8o&AiE%LPCD*Jq3seKG_xU(iaRpKU!NfYyTxMeq2GXt<zfXQgEc
zLCA~72-sVBKBIEMR-Vt`>;*&wjz<OSt-Mmf#a`A1qCf0P(yEJkMb{>4GhowXZRMrs
zwP9&D!)LSwM|9`oqQO#$h{|NY*@|qh80b<exI#5QW}~#USCm&3i*5n!2lSNBXS9%>
z9uinkD!81#OuU{#_>AJHc-i1`m2ulSOXN=>%9Q^vNcv97wlTyxuU5qOoA|J0dh3f^
zeZRqo$TPvZkN*$?zu~#bNZ((Y4R-4q)bfjHjHoXqtUTyjsY}SC?_k7AI?A3t__A~^
zN_hKu-&)Nq+1v81D@Q~8fD85Wt=C=2^XD^t8y@XeYgRELE}hPThquGJE1z}gJjnij
zqkNY&|Ad776MVea$4Dv(2@pG*nMA2|;T<}4QNGzeX@p?C@w4w!jWyN6`jqKFsdah^
z;2W)l1}>KEPw8LI^o`ZR&ND`RyyiR4h~5OI1jVWqd6O8NoWPu*EDA=PeuVT}Tv>Z3
z%s0(KwD6`qGR1EA2efqQ>KmwO`4L(eP6Nez5mt}s6PQ*;&v6WEb}*ux%nm9)?Q7=@
z-Ev_T{q3r1`o%%?aW{-i2wr1lBFxtk9oy;}1=bQ*qTC!k8Su5NW=#f;3`XsxRhP+t
zucc_pEaKw`)?~ofO8U&d4!|73tcvozA75)N3dAvGN0wlobom(Da&l<{oM>eavm?7f
z*d_0`w=T{*`SPA5*PY5tW#1RRRNVxUkXikeVpsXe7ICT3HgPA&exy1!9_mt&MNJ(O
z?;|&;hpv4}OB8QxV|tDn7IAv2@(ppu_)#w~9Jc3df{)e!Rt<u6oe*4J-Z_Ebr)+|c
zrht$nioSIdAsCFe4dV&xvV1vQDJjkMjRUUoeWf{dH3%<4tU~#ZEDbLPpf&4Ct6~E^
z$}v7iL3$fue}(B*3%Soq&&7^DMSTq{_Q%;S^3m6UwYD0V;54uoupU+e6P?Dq4{VUt
zK<ze5qbCx^O9Q1!8d%{85W3RB1gC|psU4jnGb364ZoN-TFu!7gAIB6{=!XTC-HN00
z3ExRO4grwe3cU(0?O&vHV=HjsiF4qzH5QP@_9{<fBSMXn6e@3mdP1bp@po#9SOTtO
zw|WC*uVfr`P6~DWICb<#fajvSgXg>)dh`LHd$yXf2S7K>dWmN!^fLJeriB{v9;Tcs
zI98kxS(4?$cy6LRUt1PAORNYrZm?Al|1EAw{NEtJ>SOXzwi4We?~>QmkJ01Dk|Ywv
z1Kn|qvpcQ?{H<fJ5Lz+6NJz?47n%z6aUfj@ZQm#ItUz8Oh6i$N6pO0~J-{2!F<;L6
zlnXU>6;ZC(DpZ{C0SK`4*fc6GSq>AM@VRTkAtm_(fO83)ka8t3#Ev6l$GZ#4GI?cO
zpqqeX6P8M3uTm#4Ehh|#knuz+K{_tsG7min{`NxLFPy8oRWVOp;yf6YejI8{ET&vB
zFVnlX80IIwCVVO6e9TFuTTc>$z!Nhh@*EG#2<#o3w_5SQEa1dmiL~Y64&kOms4F*^
zzzop5#Uc93qXg<_u;#+N+)5G~%>!lU){zKJ%|*0xUw(`To^i=Z%2ioTP2Hm1xwJdC
z51TT#mQ&R0$|QYZ{Gnn(?^Tk<5<Zj2X&z<~rc2}?5330)B(jYM1lgaPBkj`E-vjg$
z;ViY>#v{%Blkh+y=;o8yzmtX7U6OaSE@#U?;FrJnNr+6eyn&NKO($_}PDaQmzuZR2
z&4#XM){*3PPYu(n$G{fo5%QH`H1BQ>vywJgZeR2sNg62&&*lhzyE$2}DS1iD6|Hxs
zEeCyw&UvMfyrTto!UG2@u_1&K$Y0iOxkgB2_}=F0HkXlk>sQr|XFI`Fc?6ue-2jR)
zswc~EocVS$CRg(R71x9xktg$Zds3<ssxr7Q*XQ_sJE2V9k1Hsef=d}vCFRL6b#2~T
z4wmL#TEna$8hF70bt7571(U9m_u0wEq{+HTIvJ%enJ>Z7WHR5$Pc`4Ss=IHc#Do^4
z4Av=^$CDzI#8e_l{EU{`55J6FNj?sk+g+S>6Uve%WlBmQx^i$ae|ny}I!6zHZK1|D
zEEiSfZ6fpDcv@eNP)h`58$7v()jrvV@Tx>Id0@z%?9YDgy;XVYTBa`ITJW?;<*LGK
zUgkaA+mj5KlQSf;lZV9wd7+@>H7Vg7v9*Ze$<=h!ebn)_OVUyDt11P*4<S`h4Cej*
z;B6-`^D^&u<>3(F0KIkJPlwQ$>!@(2fAn18S~=+$<_gysJ^Fp4>kySfA2?iqZ~85X
zdpGm^>goj{Jt`!43RqPeOEn=>ah7mWNdJq6UkE=#dQ=Fc-$$ZO1ruBj<*0FL;}^>R
zCzPMHDL+*jv?!bMQ#jM7VjiLV8uqhY0qIg&H089`rSVH?tvt|FQwm@rb_&1XRDwj3
zc|d|sJt?J$^umX0eIMn60ES7c14XLP%eG8bZJ*BsgNu!LGb>%>;VPynfAqZk5jDfY
z#4Dnzl~@1b8ig8fh;+#Lky<Y5ZcccXNaF82bSHF0CD)3uh`F$#fEeoWi!P_<!Tu~y
zU3^`0yhNzcD^9tpjh8|uQ@}t%KjCr-N=lqZN%DWWH7H$)DEU)p%Tu4C$`i3>r;x5w
zQy4TCM@eOr)JR$<w-$DnJkmbi_!R^{lCb1uFyx9hxt30VD`is)*n0vC`I}tYM{g}0
zxp>B+|NkmZ3y!qne|hT1cC<)ERSq>ip_8gzmR9tJqBR6LwVK$Thi!x{#ZdTBm8C61
zE$;kC&p7=KmE!Ng8)M&MN?oIQlMrf*gWEND!N}ORkwReKseQtmgz`XkyC&Q~Q5pM=
z0lrQ6NuU=gU4Q^igL=9MpIJCOj50U)MWlY*KsEK_=2C9;v6OrGTW~59Se-BqV?9n|
zC{H(_q{DM<I?*|g&?%MTD^kM)2zd_jGz|~Qg^fSml)7c+b-Ej&6Q>RjKbxn1YAr^?
z&p*UDZVi{f7>7HOP9K~eERolE_?Yma)Z0h;4fg-!$g-tvSqnFQ_yh0;&74+o*}|Lm
zBzrVj#}mGg2tNI3)YR$862S#L{T*R0tnf#*lH%hnQ8Erc4tynHxzu&RZlohKFiUF(
zasGb#L)X|gZU;hFLky}oey6t(HlkrrRycklO+STMmaN7mktXZ4Mty&#8B#^bEi;!w
z@EpIRki!;&H1kraN5);t7F3H%a+!l1m*5&K?h$Hyg-a6aq24R_>ZdOfq-&#kpogxM
zCH4>Dp+u@Hb*3aCQX=SwGZGRcg68m_X-4cBi9OFlI|3^_=9O1@7)0nVkq$h3LHHbR
z<x1~7b>$k`<;p-UROXcrNtsQU35oKK_?2{VK%zcOFp9#OnvEb)UJ^gEgs?~=Yk62l
z;8j~*VnF$yk>qWX1Pjj~&}R-v1UKsp9?Ka~w(^qrnJa`#62ZWnLAji{V>=*M^kNvl
z{K-q=u9`87vy@p}9~?y}AuPRuF0ft_xB39z4mIHCjc{#V(}dw<e2q?!(f+dj*xQxu
z%Dm7Wa9V!nk*KcMlFjcc*CY>Jj6T%%v}QoC{^N}NWv47FNO@mZc$Hqq2K!w#_ZB3f
zMJVoSaXZuHf)q3$HC}BYid!B~YSshe!FouS2W`YJHdU_LUpslbi7U+C(>S(oY2RKi
zhQ|IYes3E@`fL?KC3{)%Jcb-3epOhgu>>)UmCu<kOb4efp>_-r4r+qITBgQ(WLc3o
z^xM$<?H(&mb+W2>wlSd*#bZgBMIry^`-&pqm*tXBgDpx|>@BL4;dJ&z!V7fXdEU{p
zPJPv3@sxOTX`Z#pi<pZ2Nca=*R4sa6hZ>vFyzx(qvh`BLTZGqSbbJjJiDiiOBlJe`
z`V*^*1rilpTg$U6@hOt9CTBm^1XAcqO&?l8g@hXMln^)Cp-&_%L+D%M(D#JhFpML{
zO#kY93^7-n$hdA7W8v%~@?9tmAM=m13GZ_BT{o_{9|KEbgG;_>dwp>jAzQMv!36U{
zIiv(Jh%cG%fm)v81x%-Sckyz&flRxl9;Bo8*MTB&?B;(((wqNb--kjcLKCkIUOrc)
zlB0H6AKPWY`wRE~lUa-GIpkc-X#675_`-^`1Su9P8ZRC^OTjqm+Kh&c=q41MuOY)R
zuq-aP<h$eHK{?rlFBBbW%#y{NgdQmA_yfe~EWM>9v(-7JWwz?E=EHN*X(>aoDz%>p
zHT35Wvyj@f2BF42{jy@Hv4%2KD!^@E`~W6r#FFBJ%hjOVmkmjin*Ib!3V*q#6b@F>
zuu$Xm%G@yga$rDwa5cZ|KJD3!OA<qczY!Ul7i#p^WfR$VFw{u%*hCJ!A8HJ!MmfW&
zbyr3x6L?&^uZ>Wns#9O09K4kJs*<AXYcev_sDW@M7o#}el~AJ=1F~*O8u1>}`NxyX
zf%lKyq*Pt46xx1%sNu;k^fTZgErF0Vd3d0CLU1WRFZRVk`KHj@pr!mqTNAH@Xf*EA
z1}m&%m#!-G4lPYd*Ds)l-qls<cMn5<*EQkH()2%d%j5sEM%-Eo@6$Q-qC-Qia#rCN
zL6(>1n+YQ?;tSL&XO;ewE?v&{)OooTSC&={J;)yW(W`7zj-oSOm(`A1L((*N4pK<%
zJ;uAlj(!}7OAM~;zsiuoYMfo8S}OJJflzq?hAkmiEx8qkwr{Ou)vJ>K3f)Ryl142X
zx`nRRh|~gtb+S0AqE*Xk_+Zxp-`D8FDMt5-<mwwLtL+q_&h8~_i{Ug(mOF4Lk&~ub
zSl6>ypR@NwrO_4gP63}_mCvw;G!324V2WTWty)WilG;%f)Xk@I*2$$obpCv_1QmBD
zM#T$~gbhYFecwlgP`M>Pc{qh+P)M9EWG^-xF_|QV_^Ig}op!DQxj#7*I>`rG7ftry
zq|nK%`zAs1xhDyAKvlJ94+KnQApIY@&I7!vV(a^RpPiGFMj#M6XcCBo7FrZRAfXpQ
zipNMtq=hC(6Qvvk6ciDu2N18QsDLOznlwd3MFGXm_4TC*UIdg-qzm8gKYJ(S`aU1e
zv%<`*HEY(aS+l0>nK^YQlNbb;P|yr@CsV(-{*0cj9#`j*tH#-;`Dk~_n@j$>C8Y09
zo``<d(>HHsKneC)7}qV~=6aT~0q#&X?itP33ez%m9|5`XbaY5!Pmb5g$mkGOWZrtD
zbBFF#w}ED8YILZX$@3tq^ZTR6c`Y_?I*I1Hc-<e-0JRub-D>1kV+hq4m~Q&WxQ^&g
z?Jy0P1R=biKtnoKdFVCZW&F*&4UCV+!mrc`ly80{dN_?V^cT1%Nx${#l8I(f57%zk
zp^tzqZi&MinVMOLSoR(IiOtxd7mX{uUMEntb*bn9h^odKx)Wv<46}JIvjN7q$aEF>
zi!@q#pm+Z1aB<5z94BT64YM9~IRF=m?&^mLCS7l>tGKRI3az515&s+A+S6*iV=l6F
zl8}U{ZR=X9Z32T@Z2#y7i&@$vUY!(OH(!q#@a-hwG8Q5TO;XonMhJKxu9mK*=@ebp
zv<(SNLk<ryPfthJWsazC;S6xr;nifOw0g{`q}#(+fKzV!)k~J>Pod|Y?CP`in=v)q
zydK*x!z?HA64wUiP2@UDeF-)@Haf}E+9M45u|}}WuU)Mr$$QcSM=+=()kve!lQ2`6
z?vWY5P}0z3GFz12|Hrb9CIAi}2+AbqS&w|L$#K-#=(~zp&b@aq`j^p<93AZj=zL6>
zECS3bkFpu&l&MGHJK&2FTsq=Aj#H+)WgVj$M{g)Hh)5!!No*Oq+A%EZ*d2;IO#tzZ
z)m7vb0&RiT73f0;j0_BZNW$sGs`juDG<UC{xH{O+u^`Zelsdl?=mYf9%%#jnc~;*x
z5<NPrs{J^!*MC6*9)bV`9m`SV8v;uKru|m5`3QTO*fwCZ#GomT{Rn(Tq%3CitAKsZ
zkW&_0`!x+<x6mnj2Z5%*eTv}0f5mD2%9_C`doKa{+pkj;X-QxSurNfVFZIb&>R?is
zx%|3Xk#PjR0=`s)p7-ltz*$9J;_y7)=6F1jP}!FWJpi<0dGO9=YC6-@bV5}-oju-r
z)-fe|=X3Cn0*|nU<hva;d>+(3F{3LUe+(F=32fQ@c~-y0sFqppJ#3^pWs%_c1mFox
zIr?_eaDe)_zp7{1m~n)BG@Vdun4o+Hiia${2}<AXUQ#+q5fiKomSYey<1vISN%fDl
zCh*n)6YfW8m4Dp=|EQU_z-YM2cuHmXLRw0=zbp4>BU;zI7VRzr>Pd<mUc|{kYO?od
zR#b;sTm83`-RwyC#=968*SQnSXW_Z`1WISJ1*92OeSIs3H!}3^FVVzgyt|~l7vA11
z*nJ1X{Ip&FnPMa+B|_%@MGTX+{x^))7e%L0k^2T?Q86n~8=c|!Tt(DM?*m+nc4A&6
zZ~{0gTU(2oV>S}I30zTRmt~z`EI+|904L^a0`&mRi(~MpC)oWt!Ghn3!6ZW``V;Hv
zbq!>Y@Hf;oVr<=tx>qlPsArtO@=i=8MLpxh9ALJ3Mr@-zYe1Tje1D9+0MQ$ANy>@s
z0l_k0u_74PiS>Yf{Ve(@JYv!H34F+j&(!;3#}cA8C)me!Vy9ZxZ;1elVJDVyeyb1E
zQv?nC2CLuNDS~c(8vyiGgv$Rm5g4xsmH%x%FozEXv0p3YAhFfJTZ$Yf!0`Lq4n@wQ
zf7W@{e|(<pb1qlC{0awqvHb?0dA9$bF3@z6fyWiivj%QJL0v(gfJYp{{--z4)1XuG
ztU+4kVhIf+gl5G0&ro2HR6C%yYyS=JkX>MTk71my{STP#nsr07*%@ViIe5TgFRhix
zCmSmqkn5%K<!*?7BKuQ!7|zFCi)%PZq^@bZ3uss|x(;AajH_>Ku8mo0#+J#mXa-su
zN&k{E$|B_q;AyHEdzXi`z#bH~3YZnZa&gOvGTc6D5Gd0F#_BKk6;;7*f1yFCY))k)
zrGwn3R~vnZK@Z94B<Y1D3orLN_i<Rzlh=FA%7~jYvOrv#hz}>%m1mh6Cnh4gS~1$?
zdTpfH1ZerI2{2os)_-;?Qi{M?;13KgCW#G_>y3&14e0C0^>*C+Ee>eE<@&<}l7I?|
z&>2qF1!^lYg+LR!$jL@t_n>@N3X~RWe~wlfX$VvIIN26xO^UjQwt-J}(T31<29BIP
zM(U=q_P%14)8umrcnE^NKrclYI8NeWPL7eZ-Bt2wVwnEPS&D?{rYAAqldltF;ImHt
z8~8yHRU!{KqsSeW^?Lwt6sb-?3FQ>Im%!Zs-vNC$S`wfa{odwDBJ>{Ljlsm80=O9F
zyMZzOz8+ZXb%?<c_)g>+7i-U<M>MvOSfj!3p8>n&0^Z%3XAQ1wh`0*;5}$k;w?QlK
zqLtsj1r8`e9e+Oz{4d5cnDeT`<{6vlDd!x8qVxIOtL=f9x!fE_=Z#-d){q#T0d!`|
zFk))|`(I+zD4&rmU2p4ai<1lEuy}>+@2EnHE$bA^wo^WBjO(i4kO*y{=r5Nn?bSH-
z#z*0-L$;?X0_AB%9kM-D4M?-<v!!<9O*n3X5cT_R&=#jq=c#&hcrKp93*Tem@tcs<
zcVi2vW<V2TEC<{U-{fx9*9>@Ge;<qk(J<DetuPolrZsq(+bNn1F6b1#@trk3`J&9m
z&0q#e^_nKniC~hc^>~9vhhnhZ!><zF-6wGI6d;Z5L%{vFy#Vh6FvXQB%2-Zj3gTaX
z4bgi8U8&sl_dG0KAQNxqyS^68B;ZML8(Pe8JKdPMF-9=7f<)jiGrU;)Fvgzwn3|0C
zJB2l$V&mGhAHVf%om#5=N%`#;RbUA65n0|yKIDq)S~BOVHiccw&^B&8GjlyfC3@Fh
zq$hFa6RTIx@aiUkqM-}%{j8)X@e7Hk=pz{|lO{zY<|3<~Wp(*9X{eoPSwpTQb=5j6
zt0WYIK1u2+icG5x3*q@m9fhbNCLPO&iQX%!+&QV4k~=E78J*YgCTA-*Cp=?nBr$if
zoQJYz!}QTXN%a_P%*#jXbCUQ?9yk1L=OkSPFzK}yCe<&dur-fv*Yn9*)nII8I*JW%
zS*yzJSl1Qd*_nX~$)P1ogb?E@ES5aGq@}p(t^S94E@@fSvg~EyFEay4fu?#gZ&Ks%
zxy-<w_JnXkR-n8+FI+P#P^s(YC6(?mMN-$1X-l{Q=rmmb^`Hw&O0kkaO*1Hc;i)C@
z{AVpfXG`wLY{-vESa)V-x^KH$PBR}lqCd$jBk4C0W-n(Y+(XHk6{L?%C}4GZW)c`G
zM@joyMRlj7X5~SQni<u*K;2*UjOrjnhKuOzA}DoP9KW8?*|LJ?qS7P@3$bS1;&hrX
z%Db%z+ac?t?o{rD6x8*us5`^$ng(jdkBH)(F;cgg)bKM+12sa;wWx%oXb~flb?P1L
z@@+P(H*)f_=ydH3@)u){(h8%wT>VCT>Zu*THbv0%sr|rS9RVsz-!;Ed>Q6W~Wl9@%
zsaTZ%keo}Y&D+L}vy$I;YyLKp51f+xrhBy9e$cX3UQc#^hbGUuZw2Ko*E+C8-*g@6
zN`5ZhRMa~4GX?#a{^;1zq^VJ`u2biL(=GTh4MxircUGp5u<EkNvQCrjbR2cbWo38Y
zNu4m|FaMJHEnTPiFNhz?r6Zqa&_7+85AIHG?L4bdoGQ&<5%Q|D(;@=G+@{b}22#|u
z?xRz-)Q6BrE`Hv+#eKBFI2wJkH!P?D+s>vQR`M*iid(eKXHg3lH4O`#V#`(hgij$J
zb`wjxD?^`49uWbzfq7QLi~O&jfOA^j=jvjFT#G6P?P=aAwmMEs9)V!klXbcm@Njy+
zz9X&V5tL-o@XM#iRUT|5KknL)b(+!O^t?NDYAQK2N{M2(9$1U(x-UspL*z^HDr4}g
zgKcvF>ON+3Emr|CE<%3pKIUsC&D?n~#NJl!eT#)DVBs50i(?GCpG+QvF|^d4OY|qn
zgYHE2)*&M0en=j~c%67eZ5mjf+((w$a#i^L7J>WiZ^Pjhfr@eeNoLk7X`PnW!#kC7
zJ~>M%j7qI4hkqv}x@7V_k>qTZ9WL53P&NA9<QgVAGu*UgppLyYJet%ZN0Te6c{hph
zcz9LIK*b_S$z`yXgx(~zjt_s=GLYtLT{HZ5OT2nV#9J|K)_3KYWXxt%Sk=hlI+AMB
zsqQSc?VQ%Sy#vMO^0z7UOCY$%;<2;Vi6ZP$tg9#rS_U3=AW3*pJ@?wIRiesusa6>(
zgGitCI{Q>k>n~i)>$fpUvz7>P%tb6IKy>_a7H-mMeceUOD?mIKF>4x#Hi<5R@0Ez6
zLPp=3HBks&4^u5B79eK)HjBmzIc-|G7(S3I3l$&w$}G99Hhu6;jlP&=MhWwL-Hd~?
zf<laUa|Cbm`f^r#A^4!JLfRJ~N~F$eDa2|Q(Xs$BWBshVg-GA)V(vB=<|K7)%&MOt
z!DqZV3m53LIpY$f8G`V~tyxjF^}A`w1?=vhIjaQ3ZSQaqC0s-^UN)<jt$lW<Z3A<q
zjtH!a{i5HmdGPgYTccaszVwn6(}<WBW+wH`@NB!S+n=E8HY%H+W`|B2+@cHBDNJ~;
zONiUtwKZFN>`vPiURnALQOW|<;A*yAUZ{^e9_1Nh(Ke{EOSA3bLjCNaiW^fDYHiPK
zw(UXv<)NYtDkSyPX_<W!RJ(X{&^P-g=0;H<UM-oeefD-$T*M`kT*TXrvd;*S<s#1D
zBV0uJU$cJ|qMM8OwSd(VZ)ATZ#3-ZHY`qL4qY7D8<rdkW2{X&Ze1_X{&27k%><@)l
z;UYdPK=fFjtsdEKhl|)$fLJjzn`7)wyCZJtVd`D%m$?2`_RFXU0b5&Em227ZFd7Zp
z{T(+l)NinrJ+=UQ*Np7p9`^onE)#8{x>+|rliepD+0jMzaZ4CAV33vlDAJfo$wTY2
zbBZz$7Lf{fFU)=rk?v;=s0QZ<tyjA}|H)?D4mtNf?P5B38nLV%$Fnn)<s%o7X%O_E
z>VIZ8H3mYs@r~@JRaFhwMaa#NPX+FJAUi|354ySaJ5a7VySK}(F2p$(QQaW=Da+dD
zvo#55U)sgoX)L~w?^*l2VZ1b(=NFy!ja{m;|D@WrR6QXgBgN(gP^bOz-mdfV9}3sG
zKX6B=af*G7{*cuz#k+IQk?QsfVjmr4rPzAt$X`nrcH6%V&69to*qmfiBPr>7k{+&*
zVr%rZml4@LJH=e4PuNLhOM?_UPfmKEdBsY3FQE*L4v6#MxfI>icG`z-8M`NDI-I&h
zxz;K6``|iM@lfyovr6S!i$^daMYpz{4&yw!=M4)_LAY-yBvqdjTTaVdQW!Ne#hwQ_
zzr3mU=u_-)_x6F<2b-qwoR)+WyG7<^1}TZ6vsh83<fPcROLxHOu!;N?OOa9{AAb)q
ztjWXw2%c%<2kirQRC~ZFP`9hnyP0mGQmVP@rxd#!<vpNV!y6x2)PZlNIv2fST8bS&
zrTIe@w>m_gwl;;&Sdb6#XgHGsO`+PRt(7@!Y?|_y$YvXJNcqc9nJxRBDc2cmA?fv8
z$`O%hE=Orc3<<`&BJ^a6Tro{1s!1syiHdV0qWY+S>d6#2bZcOtqI*qJj*;)AEGI`t
zo#`jX@_!0A@*NfOWQyk1ZW$wIr_51~MsAMKod1)>3~<xBr93XOzAoA0g+zJgHm@Nm
zeMB_FCF=8kSPd~U_D)g1@3_XLdN7~WJ(W`)ipJ~F;M8PVT1qQXedAKKDnw-ljZHG+
z+=(g4n&fn9O0{YXPjRmp5+2ker7DeN77Gt|NU4?|ri|()Oz-KG1Yz`RDfQegqLjLU
z8Wz8FmeR1K`jfe#K&+AJU3eMpTl|tjPj{`XQ=f3-j)Af@w_!ZdpJe&mX&{k(m9u>2
ztMMWtF;%jBV~mD~V3X522CCE;mgRd?rQ&Kk{Tw@*&%ar|{*o)xdV;E7$?`qqp*xod
z|JX55BmP{Lj~98J&RP=o?inuDDNr-?3LLA#D#A!hq{d<*7^-mv63H)~i>O)W*`RBE
zo#jg?>ekGQR>T~ssQPkdgDm?Hy6miv=_{rYaff&K_3M;nA5`f_Jk-G|7=Ea*<@f8P
z9+Lh^mc3tu`qVBS`$cF5;FJ<s_Gh5Fl=4uY6`;yo%Cd8Xs^g(@3sCXXv+Vgob@Nd3
z4GQT;%?xGP3rS@zOUZ%OS@sJe9ODtbP{1SA$+DjmYL16`vH%r`&$1sEYPE-YyZ{wn
zJj)&`l-{vaRrEQ^txDwiEW3XZ2B<Fi`sjZf_gj{IH`yN4^#f(Q+n^A*V|147uFi?H
zteR!tn{O4n7G~KE@*^>eh-j@gP}O?;{w(`0vFl%~U(P7LkltmMDe#_DHQb;JAIn3r
z{YDG}QQk06p6q&Ac8u~YjU6@O2~cq^>J1FHVM3PeFGBZP`DxZw(d>vb*)`L3{lSzK
zS?7f~7H=}2XY<M$7;lKq*Ucwd^h4GWh=PH*%0?{Bpv?e3scP0ScNI(-W%IIji>gNW
zW|u(4niI0NDc0F#yzQ>rjJKudGkzgC>m3n|3%7YNP`TV3_`-5|Qa--Z{aMco|6%yq
z2LtI%e|33HO1sT#Qf)&!o64>}k~OA?0={Zl!%SfG+gSru`_R9$I>6?k#^Jvn4BVTT
zkk!U?H!$O-W>pbpLbz!#P&IU4R&uQ2=RCC7Tzbl?gzho9*0k=IS>^MQ8(m~Y7x@xp
z#=MkO%3u}QH76^<M4AyEnrbGx4_z=8LgfFBG<QCkYFDE#hO*0hMAiQ3OYeMAZQboy
zs+|O3wq6-nF^!0!!@{CJNwrg4aYXlHs$Eoc<2|~fh3OIu9hp{8Zpi{u==Noa)IXHz
zW0GM0hshJ3@DR6Go=nv#0&8FbI})ikq`iT>JHehq|J1`&SPRVT%Wk&AW;*2#Rc1D}
zynn#RXYQgck{@3XnfkT-_`|vrHfm<-53b<VEo<?!sXGlnVOzs6qh;#OL_SuS7rh64
zlDZQwWnT0q+s>*+4i;B!Urp6&@ZrI(2zm{<@L*0um#&)nx+r8~)a~_u>bG=NjY2E>
zq-xsy@CKJEY^abi1@$g*g@mf=&#99{rME!fHObY}=IlWA#Pz9<g6Ue<#XRa_E@cPm
zh6cDv?ekf}@d3k8I@nYuHQU>PY0Ye9ergj>sa;=ovo-mrQvLJE29Fl5R3j{_{`S-w
z?81C`1EoyV_D46p_lBD4iX@~Bi!Xpr_@v(Le+(TQir=4FFP?<zY_@b~1LgEl;4qPH
zR|xz8oL1xvfvdn}CSb+yXS<_&G5x8)zRun_%lTp^yB7Y0DY+l>)(7thQJz=I-ZI*l
zFWxIPl%ZlcUT2CqgGgrrUNL8w^_|gsb1LRc5^#r#>Gj{mRPc(4NSKoMh+9DJdWvsk
z3K*Q)&<}&pdF3E?AFOhB!Uh{WqqozF5B6Ifo!o;&8w2;5E$zMcTDK3tcckiA-&g%p
z*~G-wXv6MR(LB~y3e9`J^_p6W6_96q;fq}Zm7^C#`ZR0l-X)yejcpGKY`_7od6D)W
znRoX?R5P+CF{YqqGMqan(srkE^uF2OMtZmZi98mL^re<k9wNCjBi-i{G^5y8InsUe
zOapuOk4bsg7ioXx_CZazv#Le<YAG7t(k+mf7)FHBhazo$&&6_j=vjVV*FV~g6@D#A
zw`D=3z1E=W85A+Iz|dql(th8aEJL^CW~9B$(CLjK$+}Flb6w2s_Hv~C8i`zOHknqs
znYz;(ev7ncY5!BhmO1y;<IG?;_pIV{Vy62;(s;i<5i;o#qdfBf&<>(dGZKwAw{wUt
ztwpqnXzW2{jY{m<rcB5|pXsYa)Tv+9mNU|=sx#9#L}cwNA89|uL`P1kr=}lOXOmFY
z3ZL&DC?C?uyJA}8C9)TG5f3x%IU9K#+2gT7>t1!Bo+s%*p{bENTxW_{M-lImR-{y<
zzW$oD??~HE+Sdt@C%v>ku%<sO^W!R!vUIt;Kd9~>$(1;5*1KDHP7hWd<4C|@{-}w$
z_8vk&RE@Xs3hntCXpnAY{*pcNF_>g<pyo>CA+<=aqHY}y8D9bqZH;O56?;Ww4VOmh
z@m!~uESRbuG-Sf;Ssr=7V46qaQb8SXQTVhW+ao_$o}O-=&kGP+--^`UL$9$e;==-F
zTbD+1-3g5AQX}21JQb6@B$DqdU{<+#a&I$h7s<55>GiW${u>5^AL{#6<eL!3pEhR{
zBh9JLO!3;17Re_+vRrg|%`4#b;EBjtAbJ;Z5wi*e``}pQG-c_nFBepDT0xc%evH(%
zUGFRqAutnfW4?^!#Z%hG%`(<S;1b?D9yv@dLF?4s)4;at9y!dcstF_YFh<*iosx9z
z<jDG_*uPyK^$icgL*9z4pDGrFzc?FNp9bT1g|aNG`^89ezg>5lcSlFwCsR?F9&gq>
z3S+zN$Y*9-^t(s%X+Av>KGKUj(AglNPnJlchsK?<5dQQ;WaWH%%DO8*Oj%m!8~Wk*
zMVgh98JFj@j4Yul^m#09ycMagLmz6!VZ>fRM|;A3p6Q*xm7(Oe3xgrw38jAz>56GV
zpKhNP>Q(rPXDbGMZoiPLdT~zBmnU~KP_cE#g1!sM_~+%Mz^~KwpznXh?uN(ie+@g&
zz*&?-p;^$%YC+!)=pHfaN0qw+3qC3LpyGDWYz1rOFH4KQrw;7~eR_pDDktdM&kCye
za*3eWXOtol;Jl<;(6>^YU+^SY`Ok58Nw<W)a2yx(twyfI&?%Fz-N>MCzVdDG^36Ar
z6Hx*4p`-x#*L^kU8!ozEJi6hCX9O4B@S6E_<<|#&+Vt<MshTP%7g#_y$nZ02;Ul~B
zQhvR-*SaU@>!Dl?yj(pB=ju@)#XWO_K5aSom0uC}pfOb<k|J0~QW);8SGi}dn73>g
z^gW<VZ+Mv=P^OTSu3QiNGgr%oMxiHzzBbBr(96}Pa8YgknQ6&MW%^`W(5F4*M~irw
z+&yHkq-KRPh2-`=*=BOB4Eh==m%F7yq;OG<3g<GJRw`57DM4Q?WzvI%Qbn!8Dywx{
z(XO8BPEn?eD?wisW!j_t3+1X(I9C;u%j|Xq-I3edn>0<8aYc4l>uXT#fNC)g$J%v+
zKF#YN&GW=aD$Fpcpyg_A585}O>vz9PXV<-1h|s?IPkC$Vmxf+_Jm`yr-i+Nt%Y~C4
zHFE(KTPA30_qyK{kKLKV+%$hGFa%jy?J|<JYnPy{PTy~%m+3?yJ=-V#spqa;l&N%d
z(Ef!?{Uh8=A^VrYxqdOZTv=_+nmk#9_F?1vkR*SMD;+I<(@O>Iz2f(vC&b>u?DiIB
zx4$sEuRM040)Ep=8GbQACQR^q-Q%~dFu!eu`Rypo@56tJH$6t|>J|^$`iRy4OOM^U
z!irs|SAAWFQ0rXKUM<4&9^vZ3gmN)1T}JJot&wv;Igjp*!gOyG(AD`mXzL@_fQ}yB
zlEQRL3R+a}-l$3Id@XU)S_f?w&dz}8Uar}Nb7|k_zw98bwKApN9kicNrfXiNCkkhJ
z;-8sP?~d)4Gn%vWQ<*JRvP*<>`v<Z@1KoX-pgpp1=8^x*d}lEuW{DuH3(FbU-z#x&
z;Y@@7nTe|xsvDLkr#NuEm#MH*4B4(%jP1`4?FH>VrC93i#SH8J7_|FApZ`_C=!)G&
z_$6fqZ8^_@Cp~_8DmULX+rfVdk(8-SRUQx89mq6DI|eumyMvnv_kr`YJNz?OmB*E<
z%r`+>%Zou>yj-mc7uD*YnK(&IrsRi%b_r$r$;(utaHbL_lh+cZiZfBIR3iNNz(DnW
zGlO=y!p1D?NPgHGwC%d?azkt6<odzO@Xv9-R|YQ|_T;1@m;WhJa{V~d=-1WeCBnA`
z2GaPgkF-IYqR3aXd^{6mF|HZR6XRE4hK}+IG0roLMO0x*@OKgIa*2K~MD%+BQJI>-
zV<OTd9Zq_|?4)+*5%HKugyvIk1;2x8u)BF4{H_p{UdJlFoV||V4`gGWPsvF)gPI`^
z)^~WAeqM<DbC=v0(HF*sn0;YGuIm9Q=Ou+E-4v(cZw23|C{|u2-njF&d~e*XlEWi%
znA`1z%cK{AZ<P_L%f&i3Ncmm%TU^x@wJiS<*!FlTI3BJ;Cb<HRuW*~|_`2dsdb{qy
z@mbVpQJfL>seEBa%?wU9HHCBL&0t8J!~Vg6iX~Z4o%~YF-IjdbZOQtB16g-3(T_3*
zhbCG4fVDP*diDrnt~4}49Dp0@mc^pTQ3L4tZQQJD2D4HgYP&+#HEx!0X5C=j{7G^<
zf3R)?bFhJ0EngheSUa?do1sdYSF4hL1glUhb7Y7NRoW;6(wFEE)VjxQvv<A<>Py|w
z32x@HW;RDS++qm#eLQk+D-{Q54P3mmhCHD)aL&?7c93QHtPERU08(b;S>G_pH^1lX
z4BM2(siOBmAs`Og`r{$s{;|?&hF#Gp)tNi9n0bIM!%lYF$@*g&R1>N)4Sp}fzAK8r
zPi458h>G5OGsDh|RWVS!2E58ik#D-Xht-BvSu`|IiWA1ZV+|~4n4U`$a~&r6<^zjg
zyeF}9;l%GP=VjKu!yeOFlDW}r$>`o|#}y{W4KD{vqyjeIa+Y`bHgdcfo-mZwVdrJB
zUM14O6rHZAxgg^il%N+a_fNOjUD4`z#HK1n3D00>Mh6>^d-YJG1KxeXkxEKT`_;!Y
zp3B$3acGX1m2t)#FYQl<w97S&Ru<4UKb6rd50!^K+SkfhA(~B=^(QU;CvWvw3r=M0
zghngxKiN3=vlVG$%Vg{l-6G;+fRUuVP%mRoe%c(;p5e8Lui4VFI$6%}_aGQGGvi0?
zxMhOe0c>O|>i|a&N4{d)Ay+YaVer6iXgKOQ{F~(A>~&<=vHW^cx=K0<9M(2h_-nMX
z5;7}sWX5x_c&r&K`tV`!?4SLq{j>0C0#^a;pCyj;lXBUz&N9%P_3_K5i#hqUf47B(
z6KDGC6mdc%;mOZp7-xA&!dXnmIa?XP+Bh%e^QY%oUwv<}SHWM)YLl@Yj*ngAgsr~~
zRP3RgO;_YT0!@HMoWS&Bp=a9yt;buw=s!(XHW&;8tGK^)4A%4&PqYRe<m}{Qn`nGH
zBj0d#%W%HBVmZr@?6Ljej6EIEnXIqb;0qiHe>^;p)`9J~139J|;SA<8&WI)mpSzDj
zy8#a?vNJy-MT8MsyK-k~&9nV2m9n2BXF8}lq8Bz_GW3<@8vTe-g{<|f+O7$Y;H9zx
z6aFgb8vTeBB>tDtBji^L+=lhql{SxfKVyhix+4zbGU&;p&5Drqmprxis@ix<_t;9^
zk<rcAKE&7M@G`n_XxVjmZLekA;o(NUVsLlxoaXR1;JN<Wj3mR;INFTlQtA+-V|Qj9
zt{)bz*E54Tg_W?AjlGfm&CW+gx-m+SGv{U0xyK!Kg`Vo7GYg@cyVwL5!v_wm*<=id
zM$m~_Sy+wKQ6CRfLu|iqq>&9(KM9OBzRB+!r(S|W&dvwsVsTSGBr)Dg{1=JKw8@(*
zHqz{9FhR?Dqp#nWWvZTM&z{K6(-f67d4yY1R<dP>P?2@^JK$ZmH7AeAvtMi@+bBNP
z?|YbDGVucTF!^cDRO1!%MsRF=ZG${JoMyIj*ZF-J(2Tx_8L!1^^Du-wy@gTya~>#I
z8+7KLw=n9pkC6Nez)b_++F=C#1azu>?f4L(tH5Q&rd!sz7=X7jd~07KkPMXNA)B@D
zP-Z7n=CL}Tu;(^<mx`o!o@)=ZQH0KWt~bzA5xVuc5x{UosNK1#z+^@C5LgOub;Y-K
zJ2U)qYly9o(8nk?7dLnA2jDwukXzK5dxX2`zTB4hI?kTwc82UC#ZM7w7s!qRhlQbN
z?qWKlxQ^w^?L+Dr;8aAOy|@u4M_Ynno7Oap(%N`wxjH849xSvM=Wys$BZRtbE|<Iq
zn`YGSvb;x@nQE32iFMY}Wk=ma`c>N~K`P*|yuJ7Z%eT6un2n|B@jm>}fxVcwIn7@N
z`8u!i0c|ut({A1pI3HnI=Y5qdUx>#9L?@vV$$9%R0mUN=Qu7}Z@ThPJdQ9LvlN!ED
z@=qe9sZtW*2{EF@f(TiGQuP*s6LabjlX%?HFaA2C%@N8;w7;^VIa77iOue1aJH_*<
zJbMoQg5NcFM*k!hZ`eab9ljS9Mw^9>$lF=uz&!VN$g^KEKl@%H+3!omJjbl2()($W
z^GQG%TDpW@-`GEtSPRNO--L$-e&B~K=eEaL>o05QBECk-I*qJ9^n|7t(1Tn*<PZ=H
zRD^NtykL|fQwY#e&WFmuq^FJxs|ItkC~79G^cYos4dzaPo@>9{3?RFad*2*rX+)%J
z;`iy6*f_2&`u4pB#}|R;C3gpGao@W#N($V!mE^a9*A)4jz$d^*ihNH1kA40}MKICx
zCjhQy`S#KCE>Ph4E8=wBH#eldwm*tg#-j@<+=kd+GS8mZNvV9pyU-8lg?<ixOlq3+
z!(YoPR>U{5xaEv}9T5*6g61iJS7Uq!za=mOcm@NQSHoz*8Ea1XW0*AFybR0tZzelB
z@y|^qel~P}V^Z4?!3f5Fkwiih(Y04zLx<hNZ0;JhIwZe`KOPe(9euzZg2n~HN5?P*
z<zYcs<+voW*u1M{jx8(O4a@gk4hZws))-3|PCZGwI53G|9`K}58kd^Zk0Q-X%FLYv
zX<OF|byV1%$7IhGqe0ES*DD5bL-Tk<;lOEeGDhj%b{b8s^v=eP56QaBXb*pH@ALc8
zY2fkrR^JYM=7o8{OKRXg=zYh(iQ&>OEK`J*yRaHqsmM73wEBe)XuTbOdv@?4W5b2e
ze*Sy`&4(ok90Lw1l1$(lpyvLNM|P&#3N&!$K4Stc<)M|**_k`tr1U~L$~f4vw3UC6
z`$4`B$L84!ayS9LiQ!>>u6zoAS@_l+fySR$$q2D2G!x=VfPUy@6GP3#u0W8kFn@#P
zZ2n$1cRpl_S7I3TZmSaT?-z#vIf|f_i`d6SJ$LirN&@)ni<3Rqw;;>7z6o{N|8m9`
zLuqn-7Z(8YQ2l}yxW10Y_sOJar3H)-^gd_2IlPTGQd$`=jYb=yE1$|qYr8h4hEnrx
zaBXaSPPpk4fl>|mU03<4@e|#470LX4t_6%=LYjY!73H3hAFUgG_WXk$a!Yvb6WHi+
zFZ%EBI-*?l!g$CQF~(|>D%%-di*j*VTMi*ij_Ld}{Jz@Q#)LeoxrN?(aRu-%*~ZK0
zW(#BG#ht*1M7H_~U_KW$=4?$Ma0<}dKwB%i&A7FyW&K4F7q5sSbT_doz@3UUCGaqC
zzasY&SPV>6M5%uOr)jgTv`UHhk>?3v(C;Nij=wGmLlc$QZ8=Z<EX*`8Ok^)Lpv|_<
z%d;2eP^?wrn|hPER`!&BUT5nk_}WXYfEJ2vCctQVsUsd>>pMz$l^CPwrS2$lA!ed}
z_LQ#j7Sgh5rG?`?WG2X{F;?`&%F#EBJ}E(&r+avg+8g{C7vE^GZ;DH>0D{hN=dE%*
zYiKep_tK18cKrl3#7Y)oz#IMHr=R42#loJhMLhKhtm)|nj}lC?`l4PEKkQ8duo<*>
zs?RUpNlIM3d06j+1Zpbly)+Vd4DO3}djq#k>qvM8Oxf-PDrD?bX*ImG2OB7n-laqw
zpdsv_G+IV!zj$fpH968$z(r3yyJ^03P%dBuk~f{OC_Lh+Kq;o7M&A=wi-z7Il9gWC
zZWZ8OZgY!SK+$TQ34FjDtfJMI&UnT6qMGHG!@JeK4HpoL6^r?6+4uT=b=8A6I@F(v
zTzX2TeKds8Zj5)0c4Ik`p9h}Bur}UB;1xjDzp*ZXRlwVdG$9}>TCYeu0=U3SU$E3$
zx+>3Jgx7`fpVf7JdZO<0ETyMhIszP0WETN!{F2svOZ5={C3@DS%T#0$%WC(XDs#=&
zyS93-QOP39)!M|Lr4o1cp+O?Y&XNwo%(pi7%hy(pnZBui+ea5MufAv`y5{$_rhiXL
z<|VYFc)H7RfG;$Rxz2J@HZok;$NwO<MyA7GcG3O5wq%*iG}k`<H{|sJy)BX$>$mI^
zQN)@9O%+KbK)1TAAAUWNL|_omPmvk~o&=uY$+k0NV0}<tHL*<MaWoW~90&6=v%x<L
zOjU$_aT%X*c~K=4f=bEYn7<7FiB0wU8dB-yH-OhXm0o68xcr{0QtR?|B3n_*9NeI*
zQFCR{(`Xho!^-CDlS<L~<QfG!b;i<fP--J7m7lt?PqYdrO$wB5v(xgO<Oa`VU5++$
zsV3lI@P~$LlVGKC60@J*mqgvn*Wk%Zi<xCrQY9_<5;$FCy%7wDCIym0ySo`oQ6A$P
zuBjzm^1dB@`vxoaJSKrLSF~7~cL@Ldb61muMU`pbw{J>e&Qy*7{5*h^2bi+_JjD3%
zkGesZ%OzT?On1M13%V(s7V&*Fj%2Dt^2j0)JP+YaDWRta%dp4b`)E39V)l0V(+ZZe
zHrG(H<cA)oQN+1fhQfxS(zPE$!|dwv0joK=RzPw_Dg-@CZ0Hx4j{`^0s+D)ra`v?a
zrT3tGt~VQPHN~W|Irjs>xkM}AoSp*3(eR4NflBeas)Aq*d9JoadiaOQoMJR->fKfz
z1LLX+QgRGVPbU3i5ZV@|suTS_=A+JxIlV~kWs>8l>cm7wD6F*1M9l9;X4{vq0~oLU
zBMzp_YiPk>V0eYW>q-&B_Pspo{7R#$+)$$LJ$&X2Jtz1h#&m_z>`JOQ=8EGN#P9)E
z8Y=Q5fd_%kiu_985um3cc?6hbUtx}J|48q-@(d6n@>2|f*MOI)&SBnEyD-s|`$M8{
z8QpE_>-?soeRvH7`0Fd16=8_Fasbd8x6BXlIJ||}3E(I?y?{F3J!Lshx0BN8Yut9_
zaaQ-;iVNG-2|zODXvOFHS>64GP?--{&YB+J&G>^P_TZYXr?)Q9e+)`XCSLLqPXC!~
z%qa}+)9`Ck0wtm|yi!htcT8c%K?AAKske*E&v-HDp~@FXQh>^_?B92g%lxd5Ydprw
zc`s#LL^^}-uBsHmb7z#ATwLs-Lw4vNc=oCWxpOb;BL4079sgv|$RjW&_7eweF1o$x
zuSDOgn8Gu4Xoa5|qVFrfCAGp|YRT4?b(Pk<>Qf|0pc;_IbK3UdG<fV&yOu4P=o?78
zPUkwfeHb2B8vym-QMRO7Qam04Iw~R_BLJ>D+lL>A#Ywu3{?aEC?Pm~WT6eKw!f_RQ
zx;l-fzL-bCkN9ur-9>rUUs_X#^q<Kxm(|j=x9G<2u7~_3;6+8=&X3T?{-P}R;=E~n
zH9kGUWrhz#&bt%sc}%CKF*kJi{EO_%^7)vE_|*3JI5DBywo)KhBgSIuuSTv|^I<o=
zNj^Jl{VI!PS1c0|yA}oP3UO0AW3hTsdU_YNqHDz$F;zV}H+{G<)sQ`1)-}PH>R%B&
ztv4B*vG6IzB-Ia04?ur^6_fs8{1-Uc*K{T$IiAh^wZKcmWO7_P`)h?|pTfAOb66_J
zZX-AK*9WdqDXQ)lvM!Qhm@;d&f9ebMO-Vc2m1vI+7h+b$l{VQ;V}F9v;m$L6DQ$}B
z#D6h#XB5#&&OXzQu9_rc;4e$oMBhuibodOrN%omRsBbIqfg(K!d=|~0-Ieq(LAcH{
zEK(URu6_sT-4Z=QaP<su%4J*zmNhKP{VT?EA$ryf%Z;JB4E!a=bu8-|1JN~HuYD$!
zKs*qqLT-__w&B2ED<ZLz#%TQ}*|o9&Tht-*BUsle6HB>Wn+zGmg$%bg^Rd%sM|-uo
zrm|{zwYi2A*X}cwft>k}`<HU=M@aL)!nJmg=w*4;?OHDYE#xaLi_J!|WVz+&_0(s7
z&Mzki(t-Kqj3F{Aznm7P9RB`SIZs0}DZiYhM3|>wLFaK6;Wx0L^V=<(C)1s0|5hWq
z4<+#0Ye=YluGP*c2Fh+BhjsplYqT@;F~y1T8Jjgr3G4g~%UlDV!BT@4C>j};o9J52
z3@tCbNuY`_^F5640yREsR*bBjEUPiY&MXRHZ02=+oXd~&{%dsnYs{RCdZuh+%$z9J
z{KcmHJ|6jSW_*a5P1&PAKTx-GOxT<DkTxfMPHZ}RbNXw@;p)uz1IkI?>(5b<&otyr
zKDHhp_JS3O{|Ghp`~1*8ujdZSGGFv<{a@tx5ZI~}-S4D!vz(cFNo4Dvq<#w=;OmKR
zTZKI9QW@if2Gl)@`+Gi?N&L2|;Ew`_#cu$)w%tvPdG)o^iZmr~9nj|Pwhq+)dNIqo
z9w&ZGs4jKVj}4TaQ1<}*o_kaLCV|GJuQNIJZF`&mF8cc2iqLbe!{K^cl{A6X!nWsF
zEx>8V7{e(*^4z+th@IzfXTBZlpzQ<b36>pi5@7yuJ;b|9z8y;ljRVFgGMfO_be*x@
zw?lz>!1J`?rJj~IX?35gonTt{DHQ$=6Tc%Rp;%6&LR}oybI-GgJ(8h6P4%Y=Gr}Ve
zQ0XJs|Md@mTye~0FJ;>z8O^wm%!zF%Iju|6Asnn^Q<C1dJ(UJ)?BNp4{pS!0_(*7X
z(twEwgOfiy5zKck#-wGrhWXqDr9D~5!P2EvP^f+W8qIw-Ym5e`#+GmEb&zo?QVqrb
z*AD{U6Tp5poyUH*Ywc3@ZND!P4Zl!Iji5Eyrd!aR1+-PLIhNF|n1)`O*-T5{U><zK
zmicVv1%*q$FdReiS-o?#_pGZZd^6h)dgbOuF){p1*&Zf#2T(zgegtrdH_{YgIKF`&
zxY0=bp31W>Gr*GTr+QrXab_(LyH`PDC^z~lGM@m0(v2q+nL%JW{qM$9mGl$#v-us@
zezq(ytYklb)Z0t#W<5!UE#Ht~4lrAhPYJvNEKy_&0j%qW_B6J<OJF;&Rgu>S><2zY
z*O!z2+kDg-IpIFf2cl_&e>V5&4Cx*f9dVm`)_b>`8;cX>qTAfFcHM67%Z#TQy=EO#
z+W7y|+?Sc`n0*QTiUIFXsex(i(B_KpJD?J!RtAC_wBU`?Zfoa0Mr*TN$DiVd0_bB-
z9aKid^5ssjtiSOifBR_d+?k{{!5F#=LpS=Hf#&ZrvY6ZtsXtYLpj~xaFH+1&L_fLv
zAgE3w{9Q#HdGWxP`)`t(0ZkP7jQ}0y?}rq*Kwv5`5r*p+Zm-lZLOogC<_w_o;=f?H
zE(!c{V5uVT{2Nd9_XlGBBrMj|Bz6qYF8I2Jmc>%ax*5;Z-?y$cp*BDhYO*d!2KG8$
zWJK|;dx+GLz+-G6=*Ob39Mtg0U#4-@?T45z&7Mzj?@WOPcXe|@B_b>yprD-$-R*BE
zisPrfRQDT7sCj+@%GF1do6iGiQ}1^9);*eMT~(wq0!iuX=ma;zz<fm>Bd{4*k92wk
z{pvDJMfl4&P~GMjhco*!(yhW2Z(@o!_hO1ymy>W*Hu5xNF$}_2nYFoJYQ~Zxu@6AP
z#P!9BkU5i%1T~soVeH$`671D9*U(-xTj&^p(mG*WY@xmQfEZ74(?)t}gGjSEWo`E0
zCBb$fyu=ZtCZTZ(fX9ZG;IGV70b=C(rO`;GLN1?V^u10Y>)^$dPLf(~7kTkhk&khu
zPjoxSi+ZXrKP_M88{uze2g-%mBv{9-m;B|5>aEU@`!c1_flXh?U5>{72l$DEUNq|^
zhHVCWg#F#`i`H6}faP05%FXk@pIFbDVx*#}Igbld(z0&RLvBT=AFOGWXI+a@KPVrH
ztz+|1*qOJJ)>(tW-6{>R31F3H568ErFR^r>nj$pvEe!OQ2Iw`Y`_>b{<M^yKGo876
zETt|aMKFg2!g?O*yJj|z%U7;rm&2-Fx<eDchrw?E)^OV<w7$QRn4jJH12~P+uZ?jV
z#k^2w#Qly$^n^_J*0F2Q%jRmQb`2`Obq%<r$Qh~=55y><KAr;TB2f8z5%Apz*QvgD
z*0A!E+^NnbBnrKxAKqA71L6#zt|CbUvH*?OYiUAu3vH{dBGOql5G3;64FZFJK9ZR&
zsQ>$hJl(KhP`;i-q(Ysm2w%0(pv=hoU!$^Z&4%SOu)JOyu@ht&|1~IIxBicpKAR=h
zRQojq;S;3!i&v;)Cs4z=@<?kwgX=rMDn+&u*aU1~=IvYa9-&>pPSt~v+H?pfs~z7c
z+$P9p-BVrqT2Bgtbt$pFme2ZY^6$@Q{WFn+iiq=Xz;SWz?{W6WHNz0|c{il%T#IwN
z&b$(;t-HCIr9O3yS%&5k^QyQAla}H3`FX9wcg|%aK{E%s`@BaWvDt-2<P1aC)k8d`
zyzWe!$(6X$sW-GF%ZtLeJ@wj3kNRz~sVz1TOYD3P;Z9qeF}tb0I_R28G?BSksL^Uo
z8rA#@I45)?ZlYxQ?ceBNoQ-hi$MDPc{Vft0iR@*py`pl6)Z&x<a$VrAU(av9<<Tyv
z>e0T%WW&&|{}yaL(W17^(SCakM`rfCO5{EE1Y_STk4vsh>_CTb=Da`$dsuk(yg<Fs
zagx%zK+Ot$_`4QoK;~a2DfO`5{y+onKqW1Rp?|L&=eK`?3$Kg0N6Y>8kuoaWy{X{F
zKw6UZ)0=*-brIe~?uUx$ttb+hDp`xDSLAdeRMHw4&YT}eNb2lWrcp|viZ`mS%0T`0
z$?(Yefz+y@8-BZSbMfSE`ylvbg0ysZZPE*VJBxkBg`J5!H#Pj>{6Jaznef5+9Q{RU
zI;3pqs($0QpGu$#N1zMz^4rgth~Bae+|{1(YrNmQ^s?}Aax8nF2m+yeP|M!&o0nb|
z>dJ;t#ZkPAdMLwh>yXdFk3E!aP_d8p?`!!(Hdp6;i|0{bo7#TcT%9*Jj;22D|4X?R
z`Jy<t-rvveiLdrK!lP#T^^q$<2hb?!uO=#osepU9Q)erlo{1B(I`{UQ>wk-uxrBRY
z8*})Qa`xOsjWgpO8`dveYGI&b=VdhFBE9ie>ubNRim@@Q`=G?mx-qFBsv=xd|Lq?k
zvBC*1*AbP(6<RFfagxjpp{G*9n-&Htb#CU@eFp0g=*K_vk2lH`-(l<gGm3!NWUf2T
zO7(Y=DGU_bXI!?Ol0CNHKjRM;Vq0xdpi+@${=WHam%I?}zKG%PC4XD7$P|k=To!FB
zi-n#Hf3TPT9$dBhQ26V(N5^7oy5y9w)6-h!J4R~@M4LX;U&EtaEX}Xj<*$MFH`guE
zc8wZh8aJ4yJS0M<vgH2Z52aE%g>hcd&6BF?4J16Sgg>>c!aA<><y#TvO?@5Xj3;gl
z(+>RNMb#t|C<r-vY4Z!RwjGr`Y`|kA7IDk5?NkygnV(mVu;W>Z7)svzw$o$S2a12h
z;#{0O(P;P%;8xRx2w5Fu=w$In#YV%+PLypI>!%>rfcOt`VQhEAB}0cy6-YF94ws~i
z;pey?iMT|&!(mAJ=uzall&t&|*$!xyZUnr95^e1y^CFyWkFDI_inx+5-8NFztBi*)
zM;xb&gx+~pO(vi2)AdUhq(gk15_s&}w$A}a(-HGH-Ge8=ZC_iUv0KXVG)kcm-%Ma4
z4z(uU_ogO1`YK}AV#D|irck!`5^^1mXqJ!olA161IG#ejWD=9mJ09j=-YMqcR~PdG
z21v7aJknHFD=ba#)1Tj#d^=v0_K-fK6)}?7gIyWkuIkCK2N`y%qWlPmX$IdlQqEU~
z{Z22%sfoC$DA<~JJmO^}ca}baH8k%V@p6I+AUrT?SH#Qo9;9iq1Zh^0R_nJ2PSoOz
z2!Av!VrE5;_JKALGt<2EevKn$-luHwM*lP08mTC0&)|s3kyLr<|HO|RGs=pXT*_2>
zT;E}Ysfn@2=cYzXu4Z!3Eal&d;OYe8<XJ^Z-hd)Wns_;>4>fthC?b4nNuYc+Mr^06
zUJrZsT7-J(%RO9K2O5X%$~q9mCiD*1S{g_xEAgqy!-cByu&GLT$kIUhM&lydQL2vd
z7SS+h4DRKh(xvfs#zHA>3@|KxbX}6@$1*zSZQ#}L$4hmnu$rN61hp*1VXEMKYp5GR
zt)z$|fIpoc)Tox*Y!S^!X=vTcz2B`7=|%u`8AQ$cJA&gXXgyvcq?<M&Ir5%}^y+Rh
zVx;3|(K!s$+lIt3fp@GT>ZqTidzAUavWQA>wb|Y%Lg%r{NSDh#kS;5kTG>M?kF+AH
z(Eh4U#N)tlWlX*h!7+TAC(r7PbXJ-&zS6djaS9cU<zkPfu`9}zY_daxIj;LkTSL!u
zHt<Bmo!X*&r5h}t0iISw{9gvbiVPqit-Mo0<jlEV;p1+^=fFpb;B+GX1N;j6&X%=z
zWkhi!5h$I7{_weo;`vQIqgzCAf3j)op34GNLM<F~iSLzP@lz|N5lPd@cp@eLAW~QI
z5^3ir(nd9BpexbA@nyr-+(pl<>-Zjqg2A8}S6Q<hAI9PQtFFhc9V2W3yLNcr0(R{n
zPdpO#y%MO>e2_<-N!uA}am~atS(*c=@XV%0?c~|n%<<J%5t>0s;`*58$=TsyuLR1r
zKVprvtOtfUzS_)DA3p;F(<T0Oi&}|iZCaA{wj=%qE|VzUwfSF7>MskFY?5d3rW)(L
z@HejnQk!hZ=ke;J9*;DShaczdui^1X%jfX`iS6Lg=YZjn@vlJYJ<lVlN>9?9D8EKY
z>84cX9>pJVe5I(-s~>qaDn%!tMsxl}S2-fJP8#C);zV)Lqlm*F6euc=vJ2S8G3%r<
zWO_Ww@kNknxo+l4Q4tIe1)2D5&fA&jSIUGe+mAVXU?x*%H&e*IVbn_uRoOS_j4t;T
zOC9@y2#0y73)nWO2Dqi^j{U198Ovw7OzdC5@d^$?Wo>k9y<4|@TQODw_E#)ON31QT
z&Ot=OmtF=KHPf-#bR}Yig3O-wIC@tMSDj~V8AGzUIPltPs{PzlQ1nQRE&icE-)aMg
zvv9QE$C8l7_AdeEF1C|L<5C@4dw?2v{T~6FF)+=#63(0X!^*0bHMf~#Z%5da2UsW)
zf4=9Zj=e)pqPuTyq8Ys%`{R5j{{xeMvy|AvvOynKUt?;Ozm(Xr6BFf2(`)*g#Fxu0
zEGFDyw4T_Cg{`O`etLPJa%d0Sr^{!&-mRJ|<*PUiBjqUy(9o%BzKV8K&B$25@v8Z%
zYeaS=+pUq+7TaDG^g~DmRiFlM{-Xw~yc$HZq>YVii$J+aMxa`+1*(VE4c0>vj;%}b
zDM|K_ZvD{QJnSX$l&!mRQqlr8_Fi8{p{#==&2w$(T}b6ex?g8js9DDY`V&KU9R8|f
zPoz7pxj{wEQSvv+dE!r`E2)(t@o$lRjQG!g8glHAC$D~;^4&B^h7zT{_w!JrnKJM>
zV{T7N-F^5n#~uj(K8*_azu@s7SlZ)14gQln{*FB+pFPv&H-^Uc8!OWoQjy3DG)~Cs
zlT)}zopdf-B=t)8^gQ<XbdqB~M8Vf5QgD`9a7BHu;D<_h1;0kYFE8dhj#c4N_+OJv
z(I8#1S(%h7cpdAxUBQcdl>V>cDzFRJ1FAK8$$`HtULE@py#9)sx@x*<fJx=eU>E1B
z9ihFe>wH%nTv5MLzO<op3L$&Mk+HUfT)DIg{XEGzWttcH=nIis(oMfPMgI<&7CA)X
z&k!Q*DJizQWd}I<AE{}Qa|ZE66*vrhU3n}9@<%j8c-ANu>rv&Xe$<1NNj}Roghitr
z)daY!GTr<VU0!*k*oTOk&8%bEeCM(ngv`-*16A)Naouv~ikCHdI*C(B+gaYZN-da+
z+tCYwIpLdc1S*CQKYA@*W6p8ME@qf>6m;cMs+_Bx3jRuE?BhFc4spOxZVap8m=rdZ
zoRypMtX|7lv0G8g9lJW)r*G{Q(bo{rr((DP<E;FIz<mIR2hEGdWN^r^#`z_Yt)n+L
zvw_8H#zH<|xUocv`oK8^^3Csr>tk@W4UV1{wc9x&96O<U-Pn$41+?Hyp0ld4p@}zt
zImYB1p`G92#(=ZxF=)B~ofLVL0CVmbVM43uf-ysY9HE%n$4mw^$?k<^tt!~~$nnm-
zTfJCotZEG7336B36UosQ{2<}+X1#`4&g%bw*0K2fkCP1zX1bSgLq3L#ELu!XmG_cz
z=HvJtiFcBYf622nW2WW@9J?XgjBoQX)mg=G8#4=-fsR%cQHo9MZ9qTbv66k1m>+@f
z75SaON#M9F>i}9=MH8Y+J0`^0QD+rx70b?DY`QA&3klTkZL{|~)@<PP?)l%MO_ewm
zNc#lYUa+te{&KF~v0F(#woskbzre2(&`yy91O@|xsAumFZZqS1f!NF>6%zB;vMM`v
zJJ_r&PR&-ortH{jEOX*mPWd>ivA|e7QtTvzT#W&e6JdneOe)qGtM}=9pCrN4CU`2=
z(`%WP=G#9NQ<{!Os<~F0-ES{3&r1syYK)qu2`zN<1JpS##-wfcsHfhho?x`rYtL;F
zv>6tq<yLje+V_#8^&k(!V}C0*#rLzE<8IH);Nh4}&d(bCSN=kq<7a+O)-Yn;cXfWo
zl@$9$+Z*4uaQ2C6Rb!WG9~~Sjt#+J!6k*<`p0>o<2d#PY1dp&UOU@)-?q**HCC7~!
zJ~CuEU&;4qmx77I*RCHTSI=F}c2mMrZV6^w_e$8FN{&sWQ^Izv)XFqmw>MBwUUT<d
z&RSYC?hz8qZ>+8Q!S$MJjhr;mD&WoZaeZqd@LCDXR%+}{V0*d*Tx;Ye`aN)97_4SW
zm*J0Aa+-Cf%j*^9gDFYi&jSB}7pz{<yzG!=$CU;WhH4SvERV6Q&kitJuxNE&!|F9T
zh+{nz$Hzuv!g0mq!mzv5vZApxi>x;ePZ=j4A?uBM4w!*m#RXBBYglY<YOsi9u6nDI
z0%r)+RP)^PoWsNgx=&o?%{HR40(WMkuEZm8;|R_%q9nvo-MR6`&ns&WP5_y#x;IVx
zPh$|KWuc#jnsEIOceJecmOEq7Z|ozITBFiCom`!aHPnc@2$+`+>#@v+HH5^y3#>x@
z5A}2=VQnewsKo68c7{_{Gmp9mE3;^OXP3*WACqLMH%_(RCszF=fLO6uiu;vC*gL!O
zct1xUYNJ{>%+E-{^T*u)t}%&cooqj<gBSsJC)!kQqqd=j!HP~t)$pC4W%(W5s)y##
z0z+;&oZpA!y`>JPH7sXMl&gg5%%V{O<keZeBf|*GdZxHj2_5uNr!w=RH9hFieQ=`s
zY`Ufi<!a8;tc*8xoVrXe{r6cTd_GQl)FPVNCS>Wy)5fk$Yt9&`ErHJRDs80OcblP*
z(j(KFOMc9>NCMy|?YXIG&Fgq+tx3Dj+^J1#LAz>CrwC(1k;fUA-fafE<Cf+Awc^rR
zFi@`H`?>ROA4ssQBF}VF>_I}W086m5&U}k;&x^SS1IADgDu0*0(D{nYH83r$wbs4w
zD*t-mT}4>f7WoF)OI^F5|6%jO71nUHy|<9V&?XhhLWidNQh}_yO$BJ#FNdZzHgy-}
zBA2o;6Rv#9#<UC7mkq?G>9N)X!&4iv@N9@a@+ljdb+0IEbxEslDA^)oj_;DzH+m#Y
z@%3H(q58%x)9M&f>bC-3#%;XU#Z_}1jM%82QLV`DuA1w5YL2gD+3|PM^qq4p^OHsI
zWyukbw$$96caz4tg4@J81FQ@3d+fAyQ;;yO6zOS(J;*M@)2inqUvQDtP0f^Z%g<?P
z`AF?oA%B|jghGzLl$M&0{LDqBx=1SbK<KNq6oW-1jg+RRt&N2>Oj5ks#dCNzzMiCB
z?zO!;aUN`ES<>bw^_yvGXl+87Ypa?ep;<Xv#T^prK3}$$EzJFF)0C}MiS(%W975bz
zUmLn#tBPX9p)TIGY-K|?MRb+v<;B#?Cca;`^4)}s1b8{+V%fXODBX0|VA*npK3DYR
z482QJ!O##Uul8n&6_J$9OO;KOH`VC2cAq3jyvr&_&`3}dif>AnG4)s^itFM1@A0Dr
zxwXTW`9JhonMCciXoi#fK)k+;lu49&7K!zTuarqBzz%b<2~x7bKC`q;Q~~yR7aQeb
z^&aTn*|jG~dyD1~DfLTjE)@F~hs|%ZY%E_}OAc$1Qv?OYjji2+6Ae!8iSRpX1GQVt
zBfQSRrXR_vT@P7e+pE3fuDu*%8Uy4V@#CzQV=!yFeTrqZAJkC0IP03D))mtlwyR?H
zBa(C3=IrSvwSbBhJ%5{YY}i<|?#pfs8z<!>rf+DdeVTQ8S`Q}mS#FR6@#;O+u)5k~
z-3Av^osCZOu*SqH4Xcx#5g757Ke_CnXO8O*#E%|fEjw7BYU9rsr2Bp0vq!0l>#j21
zcfBk8RW858`@Zf(M}Ok8dr-*{YhyK_`fN$engKoTKku`%T{UqAP;dT-I2%j>vDwO6
zB;s9v`|Qp}KNR!+6py%bVd5?%(y2*W^`g&iKUma<ME$Nu-M%n&2T>oA^q(jB>^74A
z{U1HTHpx<bF<0O=@@@kS;Rin3wB-A@Ji-=+F#OB9K)Fimb?sX<$SRyW3o}C~z1#b*
z7s6%V=dr?3&}Fpr+3AR5Yh$P8S43)C*2i;*jG9S=4Inagr=!2-vr`jsy(396KT*5>
z-)U}kc>{g`$Bt(m65ozQu}Ou^|1wU*KLLy&QS4G(c6R*xB)-LUNvjKU%#DnQ=E#Mr
zwk5-77ncq;6!oMlPNNk_SG<<k8tKC4-Vc;YYUxqM7p975E|Rd*rK+-?Ilv50R0qQ}
zqFzRT3zQ#wufFW6e9d{jwFf`;gmu-JLY@^flO*#gXM>)XMpIY4>_z#KQ#6L`G5h|R
z{ZZ>r(^bJ$$j`npq(PPZf6^c-z9>_aqRGn7lIwQy^nq!UeZQ*q8}-3y=mFoaVxkXD
z<o=afksGSUvwQ~)RcDv#AhQVbaRI7>Xb39is&6&*?We_}Hz<;$0e|Ci<bbQL$ZFIn
zng!@5Ysk0X&1V;7<SyDnvK1>V+Zub+h*6gBGn3KT_yq;O7872+A&}zZ2ab1d2xR#-
z)eT2(<od-^$iJyG(?94ZGa;+<6pL`oE`7DHQ=+5L(-VB@UTCDROrk@TUfgK+RFyU@
zgmarlcF(FZL&AGsw|iz9CK&MiXYH=}y=l|Bcn*(sWjLVsfZP6~ZG2s%Mv+bX!|!gy
z#D5n}7tt6`+Vl~-V}Ab4e&IS6<R9PKZZDj&Z%!7jy>Mn?FgC~52~PYdxW>Y@b$K!b
z_LyO}G(7XHdzV3zC{1sknhsIRQZ9G9=s1)*iUIImW@#T}g?nuZ+-biOp0o)Ow}_o~
zxDm0nWxbwaH!|#W{Zyz%E)Pmtve0f|B!th*{E|MX%tX;{Afiq7kV>#;Sq++V9)_rx
z-B6RM5B@-c1;9&5M6Q})_vP>Hd-Q&jsn!QsD(oIt4hs8pPq_BxK%)IixaDRx|6m&Z
zk|xAfoLwyr7P<LMS@0BnQ)0z=oSDo?hUgXA4Pio1y`cQ<kZu)e8IM%$_S_g-m)nir
zw?vwio(-Oa0KdzA+^(ShRHBwhTRmcz6ser)mdat@2RQc*edHbO;@FE6AhPu@*hNLA
z*4i@E<ychsMOH(tELrPls<K-RwRIOn+8rsnANW*}YywQjik?-3EtO)W051QT&6Z*f
zh}2<H7RMTR)l=4;&X%um9}%Oi_S}5T@r^O7@3?~o>1;U{{_KOm9sZ{%C^7tdez1Nx
zZi|=NJzR@mNCcBKyTfHNb05u*eR9AipIwnOm)+Lo|3z{mp9I4y(nchpy_uDtl}RSc
z0ev|g#NylXeZ=|e5{s#KgsJy$bD1W5^n*P_|A#(0W)GHf^dn>U=0arNc;3c40{THQ
zn{Uf@u^x)tL0~2@g;qAic@SF>yF9lg^n1X{{ERG{ihY5)8bm<PNt|uHk&$Ow2S?vP
zy1~xa<C@NftOk|E&&3UKFDJ)BRj*u}6}YuPUJ&WtIS!*5gw2NKwks6!&N&J=4g5zD
zKHnrn5V?tTnO{SUXF<AeoLXALWwQ27NC7Hf9TDHKB+lggQ36gdl+X|zHt2d=hQY|7
zyp-CiGN^-u`%7`#RZy!s<m&}!b<?Ub0XA_GMk>;lz#L!(BAA?4#3tfAKZBKz-T+!V
zNhaeDM80XRn0T@_N;t{lz13FAykhxye5*(@ry88?3q<l0hR$YK!YwwCTJ6;)O+$7J
zrPuAd!Y^zKr1~>dh47|r+!Op3Y@>X!cWfko%VXqc>?+PNIH%>G1gZeZNZL}w?Io$d
z0zc}Nk8TNC(_NFXFOgoJY|u)!I>{Z8Z3j2qT-k7Q#m6YJ*Rm6fKTV9g18^SZ3c2H7
z;pF{D#^};;3Pb!1+B-1X;&)pr<xn)bZn+B7a>8bc@!?FBy#+Qe0?#Y*DS<bE*Ce!v
zWvrIJd2%o!AB~qO&RrHa<99^9&KEk@2;I9@ayO*@P!%&8WdpJJZ;Bvw@e9B?BlP#K
z&>s$WHG2kuKMVKW!9L9*evYmM#hS^%M;f*|th8*5$}WE6ej>t;*RoiDEq<w|yXLiq
zrZjXVIAr+_Cpf`MiJ=s`Hy-uFa~z6$hdsa&HGpcI<a9c)3D{_g*2Ml<A`+85SZj&;
zP_U*!W4_K+^Q^{;37hZBA06cGvPoL0@*DHlN~~n3*of6^p-Et1Fv)S1>@+oWGx}?Z
zjz9-RUMGNIm*~dwUi}+V_b*!^Q97jg(F(mFB|lJN6d;?IhbS=>m||7(a-nJRs%;{o
z%BvKUfqqe9c7BFeiM#><m9VeWgnB38_Ud9g>s9cQj+jdD*D6yyI*L`p&O6XqNH0t^
zK4OXx&MF?wMi;N`nWv|k-~dqA2gRe~i;4FKALm(3n1)#6#rxyp4oP^W66*lYJK7)M
z6-xXY_=fTCLp)3qwvf$VgfC^^6OH>fJlyWFyObTOElDos=e<SVs~*?HBn-bCO}C*a
zEV@cK|AA6=gJ^2KfmTaw4&0|mH3E+T`u4D)A^{YbIGLW&WIPGx1(C7Jj$dlh1_@5x
z_U$2jx(>zIEZtN=j(azm4yiH6k6(}|lVP?Tn3rFt#imZO6V^&yOr4bNdnNdPDP^~(
zN*_<n&-x=-zg5X}4O6nY{HzTub_1J)K9GbedLmaHA@laBb~W&mgf~pYAt(Nh1?iV<
z!D9=CDyrPvq7-wKD!)Gfl1sovY>InVcF9N}f}J44G*dGT%YU-m933|HpY#})On|6p
zsFKSWA~+k;h;oaIdR$9lJSAnRhAVxkq}EN6cEC$eYxppV+BKPx&mQo)J%lzJuy&s!
zBG8c0)BZB9o{*F~U@a_Yx{`D_`rGkkj8>Ua<S(+PPadY%9$y2RMk<-5$YugV0G9f`
z<DU><SGMGY=0q5gGEb5dpS|mAF54VGK!*`WEAxihhrgCJGHy8D!p({6D~aQFd9qzu
za-SkuD((dED<@42Q<IN_%*VAGjKj@hwABULPa0x1%t;t&?b>|9v~*l@>Hmb&cd=wY
zKy(ZRE>JyoC(-sJ7#~Vr0sd6vtYw$NN=v28L^UIBSg2_vhGQC)jI@ydVmbQ)uJn^V
zp6*L^gua7XEWcSxv79K?%hDm~5h`$vdbIvpp`PXJKX7SoLBk-+@kA)~43QAUjktmS
zM@AES19(;P-|)_dF~1p7DH1lc>&Vi1R?{fL#(V7k1vVqAl3^>bL6Lh190a~q6`CW8
za)g>zmMr>fC6%)eNfbkgRhG*0c$7}G?9%a)qB((dpoTPXi+uOW%|troY>d;{t!X58
z7VXl=Q~F*-IH^>cFL<R}NbgXab8Y#baOuIJ!LIZZxTw@LXtA}dcK9?~RypC8J%j=$
z#)6WmwE)49YI`F4`K7agAckWrXR|V@g#KDB%Dq_xu_xe<O0&aXx<5T}V#R!<a^tj1
zKMssg$$0Tre4c7{kVsb*{h7pWwGjL?KpUH_UL&v+&`i1&`|PFJjxGJJW;V(xtFs~-
zB&V2}jnLl!do?$CEuYmNr2OuUzhq#%i|m}t`|!@EWsiyQr-GRbn@VOT^I0*Cx5>Kn
z*qkLOZ}2#uG11N6_-xKx*esbN5>8He1OYT_xSofK_-D4YKWE+Hw2HOtGK>V+JCV`^
z$^)e}YY=fUQAbYa*Gg4>>HG|Jp-Fe8tt}#nyCj)C2IsT)jkF@%3lY2GWm;;M(87oV
z{)nWm7#e+?QG?{#xX6|ybJ%x?Gn!7%_I8UbEewsx4%(b&wd%@YebeZlvt@1%`U0L^
zflgTIpda<;S4Lt(xyOk6NqM^hlMR=0I+PRXX_2}~KIlhctANlOiv3FHGhpXMY-!M+
zY^hhnf#tBC3fyBkpT7uAP8|6E1^&kpin(q8aX&vK)SV=s2TmbFP7P|)dW_Tt`g65|
zS)zCc;9JiW=633kCy*4o%V|ii7YK9w-OkCh?6NdLS;STCeesR*tTxjLn_Y=LVVqk0
z1K?4A*=o44cn)#s5>Y&!tL#HS#~ws_ltPp?%U#d5=b><|&jTgd-0;OuwVXW@!W}*j
zRP^srHw%yWJW#6Vjk-{up%CNkzfjlUBA#7VJUP^1-{@@$v2Wn3%f1RM=|N&=n?l-J
z)D=&&=MU9vZd6m1ZnsiLZrj506(1Bn`Z=eQA5O@#+O?+yvsn5<lhudm{#+)s%WmjF
zBt21oQ)CeBdYUoQ`Jz268Mu|CSuwuji#~GCZqj%!X}k)K(yKPE-)^VrvL*#YW%2rr
zolrZO#WHo>FKbYCw=<M`2>6x=KeuC-!&{V#p@i-V)C8(3!uVK@7Ae;l;ksSX)12*?
z+Zsv!G{R9Tjci1`=t}*iPx82F_tT~HfX|nqQc4`{lUnY_-rY-k=5~Z;F0^w*vplQ)
zFkG8yg1tY|&JjUM?gX?^q%VQ-z$itS4V4Q6a}}9M-~%8RbBI{znnMKcs2mk6_cOK;
z@m`1!v-NVCH%Ck$a1*$}dHPai^Q;byG@$aA-TUJHS8>{3UVyF?#$LWCfI0X|RVRQp
z%9o=C9ol#!i*GJp_M`yKu<$DR3=teE9JLs7i_{72)jcD_k<q4nm3)qjHsv$K@sa-z
z$L7ep4g5}cq0GJGRh{kRY7R72&fS(TwwttTuGUHlTB-d3%f46v`lVnPK*Hm`43tPc
zLLncKxb-$UO_0OUvXN?-4vQ^cxwP<mUj|Bd>O@rD{<utLUt8pjd6W{n0PRt8%tVUQ
z|6EZ{)BlTnb2AVv<_!pj14HSBF(btFfhk-YIUO#!m+ij1^gOFmC)h4FY+s_E<u!qN
zA}~%7rgY_L*Yb1eUwPTqM96u7Ni|>+>pm8Vt?YG3$4^?F44b1!x3?s@=uG7q#C>~f
z5%>Y%5dt5-%Vd|Q?aQC-!4;{!DQU>mnH_}N?+VjLHdbeUxdI%`T5`Wxcra*Jhyt`&
z+=w@*kP4Kq<PqbL$--Z&n76?y60Eg?&U$QAW}HO@y(wpy_oqT3t4n_}MEDN-ENB1d
ze7=}Z1rt%G>A+O!tqUgMUZ8R-?~FN4UUleJ*<g1Q9;xboWmeSv$Tk;{<gG>!SPiUD
z1Z%Ib2iVol8t!H_d;)X$b?tooZ{Ux3IV(mGxv2;|E8^BG{y*y8Kd!1Od;C7<-UHkN
zq*o;YMMMQbMFl}6Lq$bJqr$|*qNJokBgc};in=snQNuJkksFKBl8PEDYAC7fGg!{#
zlrv*xlQqs{Gfr7qF^yABp*`=l&p8*I&wQWP_w{=IdeD9E{<-$rYpuQZ+H3D~ZqiT^
zI2kTDB8yy9`-p57x!;eGD(_J|nUE7+nlI=Pv?;$(_V1~oOc_7C6A#TK?V)KFI?GC2
z(QPbd#yh6i@Ban7xroj%4548&Bykgws|hP4!Qh77Lug<&&21rqVLR+G)Y9AyBJhhF
ziM>s?A}*YJinL&`&InP|4x$&5%?bP}_XpoapacoClilPJ3ncXN@`r;6VdvREW|17W
zPZF5w$*fDmzL&&e61jx2Y#{aikETMBUkx@d(fcHUmH{*)>dnFWW&aEQ@QT-PISugv
zIUwOm@{N+9v*e!>?%`!iW@iO0(O70D+AjGi!lP`4mK2cjnQHt!?vP@`eDN~LL!LT!
zJ5ytoCa2oY!TtfdE*GiTKL<LgUjpGn!k>^rDX(Cp&ZiNS2V(sDvs0uR;#PtE^w1AN
zyufsbr|+3I=^H%DPjE;q>xn-)zzqRm-F>@h!0KWY=)T^ALHGThe!eA$EA+k(43nQM
zHiwI~bRW_+oM8^1j76ne7SPfN9OmgJx8d-m6fgD~qTGkeR#!LNK;m|cfxux#T@fk^
zV|l{EvO%E1$`>GhH3>6Cka~Ln)O|s^eT2{~30OJ&0O1WuETLW`2{ET$K((bX`BEl8
z`WKizDOVCI29pq&T?T*AuMRe!-~ku^TP-f&lsl*-+h75l@*rW02e^Yoy9XE>Fy(bd
z5dA8X>4hILjOruqDA2r5xx=_=hrxE+@SVG++qkAqq$>KY!R8C_=&jwTv~zC-%3)qF
zkn$H1_v#M0N<i7Wnqy5muiG(q-r}~YI|bbLkTr=uxyF!(fOHYh{>VH0Q2o9@ozTvA
z-tAOqgLc~C`}RqCJdfFy0z*?il|-{)jyMZ<(uiyxIX$mcK6pcw?>q)KVhv$6uPo3&
zP8j?}f0CSnli=+=E}RifRJq54(@tWCBwC5EP>p~gfFsgPI3}d}gdOjBa(!bpHnS_1
zqriHg$mK1vf-s`h<viRoQgyx+l^eoFUJV6(g~Iy-c#(W8jONHQ2nd<)T)G)~6=4~e
zpL+<*&pk@&Rl@&B;y97N5&q2M&gY&2`ZtFEouA48;o>MX;>hnL0k~0@5SG#V8mznX
zZg)w2dE{01T2=R2{zv8b4j|`YCC5Yo<b7#3ls2q!qh$3yFOA6KgookdniewtNHw0*
z)4LV{d{pw>w2bpd1!UZ=e)o1%`YgtYfM99pak2~6;~e!K@ZNIAnTjGA^}QZvii}e}
zjzdbG&GWx~QcZq`h4=)?SitE0{JSG)SsXCTRBV&fG;l0VAu@+BT@sl@E+Ujkf@zWp
zAyZdK0)3c@mQKBq)xJ1Rgz(x_Vim9;{5E6BWY#`?0hyo6sA~9$b;(>KoHhLK@;YRc
z&TMxc+>$S={<Rw=lZ8<)`XZ<(bJ09QALiK#!Cc$Xe>?{@G4yFLXdwKU$s#sD>SF}l
zG5(>yAtD(sIB$zKu0ws&zF5>)V9agYuY4CJFK;?9Q}+?x7)9czQbTS^^J(;XTePI{
zZ@RhO+oIhxy}eKf1Q?O}F3se+uEw=o5;2KmZeTwAaRAHY(5L0Scrl47f~<V}2Hsr9
zuIZr`O$aW-I@(v3N$7MPGoa&68gpi(MMUy=p^?1I;E&?7@kbA85HY!~Kq|VGWKzF>
zNw-l`1QboH_jWPT_tCAXAEQQ6S?+!B-)xv^2zM&Wz3+Y0S=vZ~yqDzthlpUArR8_x
zDt-Uww;FT!dJ%Z*fQR+;-ZZHNZPOMI=DYN*X%zbM33<;L3whteO3>s@E2Y{6qa5<q
zgjlWJdVSs`K|AQHTU-6Y^mWoMebZP(({K^#`Cn5jy9;l?pr8lbB1>EipIpV^X)OPH
zPcjwWUo9V;Hao)%VVb4M<awW^Hw{ggc0IG?{fDOWA%oPdC5LxA8a1t9-iRS18YtB4
zVz5T|pm}gBInaD>YqxO=iqv@=<KS=TWA6*Vzl*RQlI?ws$de&HS|x>AN<#^yb+L1h
zS8?2e2GpO1yclsl6g((sih!p5j_~Fvhl13DfP5k%{iY5ah4it7B@F@H+%N&1v`4^0
zK>0xAUp6a(ZLPKm<P(DVr$MoU;zrb1e?sNILx;Vu$;W<(Q5)+L!yoN0@~Ba%*>TGO
zj~&ijHE&(iZTwhdp-z<52P|Xet*o?ZCy>9iuO)>vj*chDT74_RI69Y*B?*L)$F0rL
zMXboT!lcnO99<y^7TVF$u9_964d>dA+YHW*W8N83gunRxyFmR3Hf^JCA>6pop%zsl
zsPgxI6?vE-P%0#?aB+~86^r5Je_@U!TpEquNZWgaM04n`lP>lXXR3a{3th}xakP$p
zfUw1NT2l%ybRBERIdLFy2(2Ve>(MBp(ePQjmq$NK;u%92xCX2A$A^W}d@@yrl*krS
zm^^wH;U%eq7NSr9OO-xMCJmF_+%S`!v^Rj)f%0pb8Fc5c0ZE~e$V+g@>{uLjniZoD
zP;tK`grc7kKFJ|*vwZMsT8HxAlKrm~5)X=rdTS??`iCT6L>zvMZRqMPxfvsK<EB#S
zcbZ{d_jI?hUR1SSWDbe9>AEh%Oy@h4ZUW)Dee&@>W>zf(p{nrl<Pfu-*P8ta1JJCM
z>IsCwlIS3kMi?cDHX@S<A}qCR2hs}&>>A8knCB!wZ+!-Mf|&dyaNYhJz|bFkJzHM2
z;De}ONboBZy2z)2i}|)#L8wa*Wi?1g{?Zp}6j8NuJV8AU5p->9h<Vo_(EPp2>hz^l
z=B+-O)#)sd=~pnpwD)&?rEr>0=JXKrUiMChWJ~$W&=6`}M_3~%OpEk7!foMZ=oXr-
zp;=&v{NTL*&@`zitz?XtQ%);Xb$W|#B`9*{r9XA6Ez(NBi!`P`P1xa~_&-d-^j-Mc
zxtK-+)=&MZo3#X`39~N!w*-FZ!vEaX$m)BoYwoqc2G-Ut>^3%J64xusUlu{J+KZ{m
zUL*ZIC{}x=d|Zdwb1P?C)P%FKGl%>L#{RhgCf&+XnLfFPkMAUfO_3f>WEv)y(#(t~
zDFIAIA|W1TZYUySk%~FQu2TAkpw1`W?3J-D3fx6CGe$|`N+L)|hS>DAkU3)>q0pVP
zG=@w9!mx&1&vQM0e$J&*#v-bn?@}p)ZCHkW^hVgfVV$_~W6v9MO9<Bg8M|21HKUqJ
ztFT&Q&-0qql5?{mkK>5@e?yZHmrv>JPFV7nWSR;iGwvj8kOb_{xR<br8S6B{8ABsk
zXo5rTvb_e|sq?U$s_GV~l#FMnfnn5-bIykS#PdYiewYU;{i7(}O^^qCx#ePJyiJfN
zU-}Ox@(JPZ!0q=vk*^8e?ke7(^$wqg5s2n34gL@5UaO}uK7;22NAa!~(NEMs6X|+=
z^b_f*aBf&XIko6otfVoFc1#T5#lC4c<k&JFChK(+TLydRE+ed08eF|n$JUN<#;>JR
z9@%z$wTI7T1o0bymrOuU_~Y>uj6qb!%%Hb;CdU}e$T8Qr3+!EJ1OQ~n*v6R8%ixZ9
zgF~s0+2ZP6YM^^{y*^_j&4};HkoB{ViQ-iA28Yrg(@F)Ik{a7Q5!Ac*G=gKyr&(<d
zpXDL_OX@!3A^ir4->}TYKZLa3RS;fi%@;fp9Muc$T~+YE_x?U|#gy!5>oy(;(#HYe
z|3_yc^&Kw(kSRIl&yskZ$Ug{QFsC}+PI6n+3T_YZX%r0hEk*?&{#ME{OldJpYU1dF
zOApjbmlVGy+BAfoFYFTTJ<x(5Vm$eOBj0gBYVV6S*crwH?R`)!JfJ!L03z_>Wgr<N
zZPx08;s+|kcwivD2@iA|zbFMEEyy3;BzKM`JmK3^hMRH1Yjgp-#!kSd8fy`J;P8uT
z2e*$d5!^s+!49#9e^IN?FBcu=-9E2$8&8WS)l1P|Wd*qmYL8`Y89PuC?-5~pJ2vMW
z5>gqWI-f@QP~RLJWlo(k>Xdq|nfGI>=>KVGaMe!6qqjGX=lkh>JjVL6;s0wtk6sI#
z;3C@?+00M}1fM=8bzGbmLL1@Zcc;3IXGG2J5kCHH44r)+q@#L5Ee!d7tZ!2a{QQ0@
z8Eu3|VAc19Xy5OLnJlHcy^zlcrVWgex1szk!M{>UsrxH}_@gvLw(etpi`ojN2?Sot
z<-j~d_5tar5L$DD|4hDM$uOT`7x#J(Lz1l>W!_1L6mf@e;8k?R+f_b_8Wq~nXJF?P
zgA1d529fL0XBC@L9Es&^2km>1hz=R)X8{qpwenFXA3tF%9a7_j9!u_E-|9%i|LG|~
zI9CL7t$))-Qv|!tFvl4n{f*@Q1jGi1h<vD<;zkLU5BACH4nCeFP0OXy7yKTaZu%Wf
z#}aU1o6kqlRKDZ<n>I`7^CKvkOURJ~%hEXX__zk>{`^@$x}FEtR!IFBf(S(TC6w#j
zP41wf5J)nG;T&>F8TR_aPf~W4iuqeqQglZ^d`qO4sqDXTOQ-`Dcdn7+wdWyMa6Ki$
zu;<TpVRxXmG;D+@4EiPwx}(Ge-QloBI%T0k%ABq{s@$%)d)PZe<#I*`)jHR}nvMt<
z@bHuB&A&w@XFUrlg4*9lP+w-#IW>SZd6cM1+(}nCr%%s94_&KBB_4@NwhP?kYE^I~
zYS=lYKo%JK$#sY=5O6EHuQh&_)%P4u7ysVwb<tva4wS66bRpQYa8<*9PqhI*w9QS*
zdp|W&T^gjW+TnEeaU5mBJE4HM4!pP3K6_Lu^ev={z7Zn??`VeLTab@+5WHgxLO}N&
z0ux?-d>PHil1({tSLC>3pP-_%gVG0c8NZ$Re0jI=T)r3x!N&*tHXv|E&lZ7uT-Yc-
z_ohYaMkJ_<@IQ0Nt*3=3B_`O%$EDf^dJ<XtIW-Oteh((k)e0sk{Ey{U4;5TY;NNpC
zcy@Lg>kNMZt8vFcS=snRDh8WB1X=LT#aq;IL}wo%J7ciVkM;4%-DuEr5!P{E5I&d0
zej@)Sd@G4PhMCC%of*TP&-ETf2fB^t9rw{WA4g5#o$K1k^U1@Yq!q6GG>q|Fg~NzX
zUL{*b466R<I=>J*S65&$n2l2WM0>Xa7wf!TZk`@qpj_web@NWikOP-U-Y*TXLCGE+
zZAPjSA9H-pPAj|!;=R(|(}ldW_Lq&C3oi`j$M?_}6^w~jh?tq92*V{Yfe7@^oGJXl
z!|Gdz-p#y%u$(>RLQYqnXAV&I+u^*C(*NoC(r5+6w-IV3aSM^qJ$y7u3Wss#{e)(L
z_gjI7S(o`ZVLR}|q5HgkcVY0O;dhE${DVf~&&^~fnYmMD;x8f0Lq9Oe=NXjvbw8D5
zgCARwk#m-$FX((aLGsoj*jl_p$Q~7t$mfer^Xv*kipxYa*FhAi@XL(?mDX}AOA|QN
z;nb(dwWm(Z1z|q@63!R1Ab-kS()`5*C@jDv5gXuB(KOlyC@#<}=5CD?yOaLYw<&f6
zRerfqCYqG;9F2zgsRy%6qgE%`URD+8L5}urW@4Kd1M~jO@MIYJ9P;xzC^zp*sKy)&
zWpa<kH$RfpDZ)3B7-*Q|(Ujv^Abs;O2*-occw7j+`I5VVAg@N9pG#yH;U#fy&HK(2
zLtUP`f4)Ojefi{>komY1KEtCg&)%Ek>2UlTx?Yy9;qN}*sb~$DKY@z7rQ$3qF$>54
zQ3m{cn=EPs{rq)}uuXHOj&*b1cis;XJb%#9-k%RrAATA&%0A&HXG${u%x&^yykk+k
zkmVL#siuC0Ml6w}P+BCSN`u}e>#A?}2i?Yd!suhNt{z(mVRoYd#~&m7RTBR)%n1;O
z$0`ipt620CN(l2Lf#YPtj|uDWmc1%|z*iTLY9~C2hxgSnGOqWkaOO4|x&F_;rgyW*
zxS2dU5?}5yG$Q^;|6}f8L;989?;UsK=2(FWd>)m~1tj0ASBkB(5nKNIUA*f<AHJRR
zr;t_0@%x+w-9}HGoPi`=8*lL>l>b;8J#${Kj~0(@R@Z*cujb;0)USB;%S9T|!2+*G
zMs6g9;;l!qeR{rPPA0*005gi;6ZTT^L5wUaN(bM@TO4HgzbKK4-%|0RSSo!^_ym>w
zP$GgXT^AU7#ri^=utMweYEAXV*AcNQcr41|RLXoP9Uv@{?mHhh1w^_}qI@4pmkCQ+
zqqzMhfYnRS!y@7>Io|o=AM`vdg0XME^AbMZ&=r_cRPbeUl3{K~rY4*u$Qk7Jzk3Nm
zt5Z`X<o`k+>8}-yYYo^|$4jOt+~bhzivwWAL64xZHc!y%*)JtBlGxA>KcHm!Il{w2
zHX&l`qupkpU3Tol2E<#aGyhk61j7N)rj`B5#4`y|ES#-f<W4vIUrrYKw*H#z^9V%%
z@hu2+n>^Hk5*@Z6UT?F$Y(u~nBv9d6!ZiYM0RqM=`pk*z3Ab}-vLK7x%YnFAO0vnl
zhtS9`<k(D;xlhOlL>Wj1BcF?+(FPXVvCSsS+zOhni8o)40mPR%WUQ5L)_@%zKXc+X
zLd$d#4|bc;WinhTd4${-2+xt8!#Lgn_+H``13yCs2K6WYj__N#mvs`-Z6Csq9!8^d
zMHzmDDG<^kv9tlIo6$9H{z6w$mgZ~1i;6U(8#$rAmPV@_%tbrq%#t%~?&?T6(_aIv
z7nW(AA^RgrV$dYI<)hN_2AT5c1-EHPC=!z(HKcT%F1^t!Wh}6DGlF-rM|0%1c#GFW
z`A>+uZ1YbT8B#j&u#oCD#=E(h6o&D{ZzS=kVP*vhv5;o-bKS-%k>gnQ#-pWlCnYd2
zlr>T;DU}F_7b%-XnL@*4!Go-9T{M%T*A4%_gygD#Y~W>ax@0PLTv^C{7C${{sxl&1
z6RwiP6-1b|S%Q_ijtJ6_b+;toX4ZDX!;)ykla+;FW<4c1Y@wrn8vcLEbx_5soOOV(
zPZBK1*$^T-QW70Renz;9d8c+u4!R+m@nmzk$y9$L@+zT=2F+33rZ0nG8_B8h{!W}l
z|F{~;HIJtJ4}>>y$C%A{sj~k@klzVNP6b9YgU?3QX8#lR`D(d?fWG<%H+5F}Zq^J6
z{2*-eHF%PZ!$^(}O?4CqS%&}X*@DTZhB?Us$|*^FLjt-^8X_R~((rpg?$EzMgGmD9
zTK%(+zC&)2J{)mNVnvLsI=AGo2$)+E)FsVs$#;E9<Ver7+MJTF#lH61B<PY|PD$u%
zZJM~?l4MvBeXX-4omvE2_=#KpJ{e{5jClVS$m>6gA!N(DjA#Y{_omshmq;n0L=vwM
z;izWPl|oefIcCdWNwGprs+Gi7L>?qut{_`W4|OhtY%fvtAnITeJf8HN%;^XFgU|{3
z&7hbAS@Xd-igpuzCBulwcL%6{Q~>A@lAqF$l2X_PgZ_D@`urbJiB;G{Lh^r36iS3E
z%5a(W^DuaOb7g9y&IF3@IM~tkKP#jH%8EQc7(5`5Yx?9NBr^N|*{Z6)j!L(gt<BGp
zJfvE_j+z*;MoQKGuXzrTpHefCNavGR5BQ%LYxs|+i4Afqqe5FvdQ&ES^DL^=%cM`f
zJl@|ioJyP$o6Qwe_<-<7Nh~JPO^}24<jbK>^EHN<gUsdd5`426u}r4x<3iSESe(<J
z`IaM^vAKcN5JG|^?jtghFkTXm5W#fFnGbDD)*(k#-^8*fB41|s3)xt$G*ONEr*Sk%
z%xcM|r>}%0TA0iNhl~ORt@emGcqoB{V78ePj$}x7VhlZQ#+{RM5rIQ$bMt#dstA=y
zPz0@g3iC~W(owws4wmYFAzzwUIknWg*^r}+&3^^pebWAK<jLUV3}iF3$+?%XNfIZC
zY-7IXw2)HKME-}c3tB%25r4>(bu;<dc>ld;82)eBTbU2mQ2r+2bxB-B1UoF}s3br)
zmyK2~-c|F#bBRnKjFkimGIuUvmaybmdV277Qso30-h;CB$i1HMBS}3#gdJn9T-ANB
zjmR^EM<ww*kt2kIl6ciHCnpf%B!S+V%#@qV8<fol57Ei*4F5NOk&-`?dp_Y@Nqj~G
zEGL&?PW;g72>&;Cnu&1XCOu6Y7sC&o&NTVvaTWR(-Zo)4)8w1o!me;d438fUdeg+q
z6#<YX9RfbX%l;u?1las3-uGL~?{DS0)HBb50CbzoH8As;M~NWflW!8r{0Zg#%qydG
zQ`4xC2_c{S>Rfzrtk-lCwSVR{eUJn{vq&o<&}#BydWsr96gen=iwWhSe2Yk>@k6CE
zwr}t4TOw$4;w$Cr+?G#;%f}<4x`@?Yw8yu<3m0jzu3|Nid0_04a<%)LsKkh3Ce5?z
z!*8PULZ>X5=rj>Kb4ja1oQ;nlzeQp<CVv`ly@plx9k#r=v7Ip>fs;Fh9-lI@jWV(i
z3GYGD%`<f?%P@vV##^gp(f^WGn->}86zWbfX*FEd!_AkH8cK+d7(<FBcnZtcl<^Gk
z6sU=&I`O8AVy(tJl0l=3e)TQ;kT(E<vj47h7@hxtR~*XsAq-wYhSh5dyqj``B)%cS
zES_=$iA|s}<tGGo#O9_TkzWuVA3}dEz}|$4jND_G_j6(5-<Ru-V?`}*(koKrez&An
zY%ky11YM+;zQ}`ydEY!5ZFI2}K+Iqn!M|H2&jG>VuanoQ^K{!N&=5WSZ{8{uu7`-}
zyu*F!<vW{!@9=D_3JI4k+;^J)ik}3ID&hYAbUF_d3aOyaC@Sl^M@0VzN*Fd9$M5ip
zd7FALLn?5AP|_ziptaqmWay0I`tC&`*l>u&9G>#J6`}<iMZ?ky&{{n)nFi9!3Af~t
zUU>XS3cc8|r}t8WBT7is>SDX!!?s##NA3`+Ixrl0Z2bo4oHa7phUdgvzeXT?E)Z!F
zXX&FX9#j57_yYCv=m2stRKFLC@lk|wD&JH6GUFee4<hI!Dx4c{?m#?yCL<S*E};VJ
z<J5ryP!!Rlmy;SvNS4Hph)gErfUqsQK%wvV>jgcOB2t2oN>MH|^9f2GloHf1CFra5
z{N|_)cfNnx9gkEkXC?yZ`@OwY(MnQMCD)%R5x0f5y9N9#mJ{b!Ce3J@xgeH#Pf&gI
z9j;t4<%Pgajydl2%rb+{xTTL#KAoYYQn6W7kc^9&H+42)hNF{u^dtlDO1jp%ph%n}
zJ#~ipk2EIcNy8a=&s|<QZzE=-bCN|*d`La+xMhQWvewz4_sDUBKicBUIjrAC6XWCw
z$HUa~feca$khpriGn`XegxyY-wt=1OA%oJp*E7b*G~O0DsM`$1q*5VYTn{JD8|CE1
z6pCt=Pa|&Vt}`HOOtEu3a&nFbc0Xe?W6Ja^Ck58`A9kCA(UV4-fI4*>a`8PPHT5#W
z5=opQvYBuX`lctr(KkK(0Gc`VC31PD!t9ALlUzV7ae(8~C>R)t=JlVt5*R%()O?$;
z-vjs;65QT!y84_17ZHIDcK~hXp){tR!m-Nl(wOngApOZn;=rg#*85YbY;i;UFpZf9
z;5-{Ci08SWCq80~{^+RvTA&6WbB;<*%@uhJTnCxWY`i?Y6(CzY-)4NkA<e0Bmo&?c
z3TfgQO4S~btijOZ^&Sctvv&r=sh?q>7L7>XZ+5AglTlO51L_wiqo#!>kIx!^Rrb^s
zmrcFwth2^fuDq`1rty=qCQZtkJZbXs>?yfZvZjt-lbw}i>;l5(WfCNI{hys+7$b}b
zlLaLBx@nkapXfGQgL_H718d&71A30HqjtE`pQx1oZnybZMZHt$-2YbVD{4)DxZ7Oa
zzJ~?vzk?d!r_zkGyUn&WCvb*1mDIDRm|<6Tn`?}=kzS^Ug!!q1>40%)%y&K>#!=`V
zB6oC~w~HZ;S}pju89?zr`=)J0tXz#ouq5>^ZwZI7TTNCh))g}V<_B07fU@#?)1>oN
zmQxd@Na~W6Cruw!f0|_aXuq7>&&E3+*7X-WB>I=zc^MJ@F`%rkm)zMB{qY9L9c@|y
z{pTj9n8ty-oxWZ%xo=?{cp{p{@2c%TM4e;atiJjoDru;LxY^Plz2*xJ1i(fFE1fXv
zzR`gx13&jIUw)(EivzUT4_qdGe&6aqYVw1o&N8?e58P%iWf?#?@W_4bfia<BeaX{^
z+yo4B)@I8g&mf09mO~!PA&=#d$8yMHIpnb%@;Ej3>(t7VmO~xOp$=$y)RCw|oxEJ)
z9(g3{lE-q$V>#q;`g+B3$P=f@v(XHk6S~=Fo`7V=7x6Oi1QAs;g0~$Q#shF$4+pJb
zSd;p%HfUyGBIlJ7iuzX?!`<{Y(k*x_tVuKx9c)ZI#kMs%hAgSuMR-9H#YC_m@{UTP
zkoTwLV*j5dFXEoarrfYDk&i0*SPKT$Z4}b>tR0fL%P^<0z)j<&{D##;gtcqhQb}wj
z@(JNJNj%BaT1mJ<63-K<C#;hMkj~yi=#s>nMBXQa-j&q5#J(gPm97uR^6@z%>0%)H
zoRx4tQ&c-b%!|s6th345PAh)L*vz*HOfMs%ACWPHQIZ%!<Q&2*N#NYbN9gi@B#B8x
z*a7ALL=rRb_vCkwYM0bPBA}K38crc2a#**yp3XG_>=%v5k&+#;l<Zw(7l6cDWJi<>
zUJ<W>R|$(4N6$vYUilC$7I7GG|0bX%jEG}sicy4NlE9joK8p}Ki&UgV47*_ZrINsO
zo6eqM`n4MJN+4GN`Ce(Yn(XZad;$h<S23smitwT&?jUl6a8MF#p=Kl!`b**=BJ&B;
zB>{J4TuZoG5-$)5{fv)0jUfF_q^ElFBeTeQnjmbCJex=-;RS{nIgct$uwl5AK;RjB
z3Bu3Fr9}Qk_(BpZ409$kd*)C{ASN?WSTiq>#4SYbCWxkuypzZ{$Ul>dBt|4$nR$Tl
zhDK%^J#`A%k3npME6B!yp3M<~SI&Rk8iFQ3AE3S>mii3i-2`j2&XM?_KJk8i;&yNR
zz_)$kU#r;2z%YQ~nDT(W<^Vm10UH|}puO>aR&U(yjRQ2Cf8cBN@EL&wwJ|afadCoG
z(g(TTc)vdRc5fUw;W`JtcH{o#%dauqxC}dTqG0v9>W&O#Oi!>{`XJgH@7E{a?v2wy
zxDJTk)-RBxu8j%|L(y6fYCwh!t((1+Wr%itMD2(QOgv4~-iaJU95u|sC;oq{X*Vx3
zBSf2trges+HSIE%C|A=K3}vN_;WbvSrY%5S7Tl6}qhUq_P@$rX3nbi!2DO#}WgD4B
z)*sA>Xsvv2p^Pa~&_(4CW|27yomC*eC}Uks;zGg<L_6ybNV-)oA3u|i`$5WjltOgk
ztdAwpZkV%CYO|vy@iLJ$1esCRZ;1RrcwZ848Rnb@f=pxU5RokeC5cb4Cg-$~dPGuR
z6M3DmTM|7)&`opxO4kul$UwwEMuGvRbsu{cBO+d_A}ss{QHfpXzL#hz`)gi(Y8Gle
zGnokXYhj|4pxO$v3F9S!Kd5k?k!u#tmeev*%L%e{L|j2+1L1Z_pqL8pCp1ZKk%isn
zT^Z`0Xzsvt(rE#m%JLAgUhCn{pxXDS#YNKaD;m8>*eQwci7>XpUz3Q8BJw^#R+-2_
zM4)KlXObAsd&;nd3xANrI1-Yf6&4%4xw(W1x}%M7E}Kc`*CW1<{KW*_?ZZSaAzTcI
zh^L6GBJhhcMg+*r4H0gU#IK29-Ot@92{<$tX3iDe8u4c$Qu-^c@-8%t>=d#^m(Kl|
z@K;H&EQIF%o7BH#Rl0^cS@3M06kTVS^RV3K@rD>9lC^r?2!b90Ds~>~c^-GBj7a*P
zS4ZHn4I}aqBIx#ct&)()LBiXTc!|h2gfBHRuhY>o$Q{MLRn$U~C}1@v_CGc7ZK8_)
zKWpHG(v1S=G&TI0139c#(e1tQ|7$fo3<Vk1sDS8=|6gff_5B%vVWw8Kqh`PEU=-G$
zfa{I_U+dem{?DowzsP|{cD~y$5b_U>FtQ<i?gZ1{Z(7XRF=xf3C#J-hJ68=16!;R^
zp#_}W1%vY7(uFbR0CQlZIcVo=ae?b1LlcadiMn0HStHE!#07&=P&qG~!+?2#**QBx
zDwoA9i&;wcE>mjfC$8@2?`IgVn1kpsCZ@rRaXK);;(yF|Ge*)8XT?bRS7xp;e~^<#
z9gy-mi(4>gW+IN1-CiDuFBp_WH?Q`#8Ik0}&V7xZVn)<i(?(2Zcq429yhl2U=<WE|
z4j5fS;EV|I>hE6ABL<U)@%*}ZrXi!0mjMfAIU2$UOLBh8@LcR5q2#=w2kc?2$;0~=
z)w-lW(wKebFf)6QCeMhW%OZq6zjJXI!stf?1PP|Y@6{_wfg#x_dPtgtdf4Af+$HfI
zY2PxVy}n1lAj5cD1%?HZE_vraD(ij@^x?y_5f7SYOF!?@EUn+57=OQ^{g%WJFoWz8
z{%EEeF)=e@{LIVu%>KlMaR+tGFy2?U4?|^sV7f8_%N7iRo9yF_KRL{eaG2{d^U!HE
zdC#ahnvVa%<CFe=Gh$+B-OKn7ducE1K=KYdAR~Ku_K~pQ>_H+c3kC_J|7uP!U2)W6
zykJm%qA>e!y?oI5IOFr##2itJk^_kY{;oOEOUsYdEy;nQgFZQpog&sp)f35q<iMv+
zF9Fp~N(~quh_^r2V!`DRb;j_(&^cfH2Ojjti~^A_DbV7e(GcbrB-CmYzw*Yb2a|Nh
z&Sk>`_eJ=w$Wq;@fyG=2F=Fbv&E_5mmfLM^K{Fc#e6QhqU$=R`ESB6JxWCXa?96U+
zyX2GJzK(C)Bim`Ozw7ueBE6Sy>W2St&+rEN_r-t|r%!8;HLxgL$~1E-;AM9zJXS#k
zC*jnRa?9YMPp5o!yTOX*6zNZ=yvQqW13|wa=lwb5*LdZE|9}jyP=4*psqk~J!tGfU
zmWM02<-Sn3ype?lu?_#}_V(>w`JGKWn??s#FWDJg8%V#-x?)ZBO*<!k8Ccg(WuFXW
zt9~Z~Q`H$K1J_1OFpby=YU|0sH1*2Kz%XClpq+<K237|`nFr&1Sz8%4*Zhb_^03US
zw{0~{ojxAx%kGUw4rtJ2AY+<FB=t)z-oC%eO3gJFTe<b7#Y6T~_X((VR^)(2r@Btp
z8l3StP#Xd!CV}ptAru=qpx)_GS8Mu@X3e^pE^4L7scPsn##yz-1QK<dv4<?4<6mV}
zb(rQ7YmeC-Z7y+3`;@P=s@hHSLaW`}BiOFos*0lQv1Y4U8)g5-+_iIBz)mp}c3J$K
z2Cn_r;;p1qv&u@Xa8l|z+g@dMt3TWJ7+=s*foOZYd{2$G=lO!MDim#x!1wlWw4JbE
zTdd97IqQb-en^KTX8(%fK3Osbvwv-t$=*+7y#L-{pHeRKKxKv%j8ncCJJXC)6JzZ7
z(BTT|>p3x?yWG6eX*!_0SmP&N5g-lZr3zON)$IocEhEmXwBqtz@|y#L`6hR8rTa4q
zPouVZMv;)z5+K30Ix}*cVWwANb~&Je`>p9lf%NpMvI@$m8L4aTFVlHkzHsZoi3DGx
zVI>sn!G*JxFV-Ft8epZ;zaBGD7jdW0by;4QB93OTF2hSZ<#ib%j^Q|B>&3l#F>X5t
zOYBhxjEkU1=2-KLZKny6c}A;&Jc>y7{vq5K7O4ZV_Vgh*GxN++tFYBFOT(3<yk3nv
z!_J+ZUoP`UC~qu1Eq-y-dZQq<V05_fV#e*&1D5;&XK4YuUz=;-xxH|RGf<~)M^5B=
zLramDj;cQmZbwdkc=F(sCZF`S-bf0!JPl%>9```D$Qt7gW8_vVFbXjOa;V!VD=asc
z3RZ$vS)t7Bl~(G-Hqc+NBC|cXWrY_DJztQD3|d-27`IiKWfXXz3+6kZsZ%yz<|=<-
zB5Ms|Ub05SlNokCS6Z!(m_0EN!P*mZEEbK@6yF}tS9|=<cnWvMcgFkIhPGOr@pTEp
zFyZaxRVe`0(y}_G6702HNz|jSc_|j`AhX&URH_?kEM#VtcU!y^30=`@b__9>Tw)y^
z(ma%8)2Oads165EMUn?|d2E6&pPB67y*w5YTHFJJfC+pDw;F!Ihgzg!g#+FnT~)-Y
z`}^DH`VU2SsL%S_<K|XdydL=qD>lKr#!AR!dS>qM2~As0SUvWEXea6cuteDl6gZMX
z-IUBTsQS_xby3hB5^CwOTBCcMCbF!`pHwclj4Zmy+T{T5wi=_4IJLTE5c22N6l8?z
z3^04Em5||^wAISk;q&Ee)yVMe5Thb4Hqzg1zK|lTh3@3C8Y^U^N@hERI3Bex;Km__
z{DH@1>wxanh)pqnWOYaF4VXXbTb^RBaB@WzI)7eEjTbDYoK>UR+7+O&tRo)q7~X3V
zQy<tW0MxAm?4gCyq1&vI+~ZM@tVdc`S`C3F0n!?1lhlsDt}sYZNY3P@=PT;ep#gU0
z_&$)Xa1gx0;qqn8S>8Zh-FR8Enmo|X7<74tWdzMjgcfJ;bts`BQLPzh$6c-q1TCSG
zeD4cvv=Q!&cAG89Hv4Ea$+nnXXYeKM!qN&xSGEb*GT37Bd8mH_36EBY2Q{v$QXdbr
zX9sI6qtGn3$~hw{w>AcMsk}j~Xf<ltAbVD5W3V-7USzcfJA(e3nMobNU8Fr~UbWvc
z>iE3_tIpTtlZzQHmscPlC5+eiQmIVMdSR!;|95#wpYjrC4uIN<MiE8G9*Q{<1M^&x
zX|N*cPjzUhPSgXs5*y3%HO5(1tiLf%Bv5mF1#3&Hh-p(yXN;pG!vZ5^DG;wIra6YM
zZ85F#wIimTFI_3*fH)?!gz_;>Qr;4?P0C&Pl>y5r5`Hws>=0F?+c5^6Z(gdi%}c|%
z39NMF*2lEOkn3}FLnS=Ow4#GdiLU5<(K@EfLIOw%g@D_qIK^&UOP`e23wG63d-Seo
zp;aG{nm|n&rW4h(j5aGfG_5#4fmU+30Dh9)W$`NNC04V^$u<hj456S_cJ%~gv7C5R
zdhDC=4z0}a9JmHT$n2<E$H%%RM^7FUI8sg89X${*ONp$`HzIE`ZeT5uBI)>=T46+)
zmBmjvl!Y_v&=pDM#a3oLKd_;3bSl?jY?(`~%o09Imiuc?fP@2QC7jA*0M(+VH5~Ut
z%B7=KL81ReR)UCERvv03uZURDI`Y<WA!3Eq;XC5v>xhq^1g%!*4YnggtE^zJr~ygf
zBw0qjlWA4gx{)?tk5-*eb$}1!yt+Krx@}7+EcUAyrRBR>OX$r=sHi+(U0o&F>vi^e
zvfW8^HIqQgvwCN+J<wMjr#=~MXZfx+RARh63CYxrt6QnX2&`1OH13BRK&;MVjns3l
zY^#-8;Kl&lTPo{xfxOkq%`0@6mynAHMfR!Z4lPw%aSY6FY2{I!P|X@6v&{(T+2){Q
zT)R#EF5XTUyUjBC(?x%lU5q+yH&cVP|JrtBRVr&SHLmSa{S)jYvrA<rpmi$Lxe0bU
zU)Lwt!}+>TQcda=s-!nq?1bU?Q4=8%qXP=Z2Ft=qPm^O{@#aLV^I|nF(VpoW6sxNF
zCb=Wgo;b2c*3fEeO^sk6*6hH;dPCrM8L@1iI+<t>8*x#r#VehcS?jPcoPNp!l@8Xz
zuZt4Y*+c9Mvr$zJu`gyP&^5%Kl(G@aXpjG>-{wibZr>h~GD`g?ZSvK?p?1NgQfr5`
z-x_kGF)8&%V`MHLI+B!=O0(CbnJdc|<y%IRb;gZGw684LEz@mJ&*H@tz38_OwMY0a
zidF9nwMUu9)VD+Ja=uEF*bW?1w>w`?C)t;qSvv;|vu)GNQbUvNaTg;mNpPyiWTrWq
z51fnl$XuzidJ-FkKnNGDb6Z3=ms$-8tqHv*)44D_6Nc9(Y)e3@^mG$iox4pvoD3`X
zsolwTvbkR!PPRv1eLyN#Te<8otip?-thOw4eO<)XFduXZtF@YiQDt3)^rh`3qQ060
z>4E?Ba9)AfgQZqd6(7xJm+2zq6c(y$hucHULe)6jPMBC7#m_i+?P?r}55{TE(#d(p
z;>-&Xey>8IdS|#j!fE6V{(QsYZRn*|9(v6&7d;-!H`ttSsZl9N9^aKIcG{$)8=Z=P
zKmS;x*}H6<kEW_&^I1ctLtEXxWSx3Bg^93E9ZIniS71+Mpw9A(w0Y^&U>-ntE0ZmT
zCtW2Qoh2X@trpJfTTiQ_WeM~pROC++9DJ80sIn3EsKrtgUeZdsgfyjApwQW#yYi*8
zp*{J!YsOTvr@&)T$>CJL{X<xK7g_4}BiMc}3RuQ`1Y?KUVM^OQzJoqjK<Hjdxn{?j
zYmyz7yhVuqAc(WBj(aN_UHV$IG2UD$-@CZ=L#9Magn4>!i`pz-jkrC?z91#jw^o`3
zx{D>@G5CUvRK2~FDiu64A@q}YCEbY3dU4^0#G5TxBidLK{(|GGeWX3iJg$B}l1=*w
z)jiTqs6dL?Iaq4|QHpRle~fG~1qCwTf+ALvqDqNMr4_n{RDib?$bWUMH-yr?nx(=T
zk3FUPRMjYZWN5!IR_|F#cO>bKz>g?UfhY0y1SsXyu!_SqtYWF*WS7XMD?(Q;$?0ua
zD|14Rx3r1<iFrKC!X=ES)6d|#l$~my>04r{lvI0ye3zu!3CQTmROWQGYDk41De6hS
zLJ6tfaF&*r(4qV>+NBkik%JiI7x&c;k_!i|xJ+M7oRqoDTzY5?O<l2DddM<{3;Bkx
z(GtRFmmbDSaR$EhNW2yi$?eH7LoS4}$^y4@-F;b=Nc3VWAQZEbP%a)~=d=MGQ5){c
zx{I%52f$ZPft6u9#i?E8)UNuE?5oIDzfH4aLtUwKplzjdi1sQ92ysq|WlW^MiPaFc
zdYztN+VTjKN^0n?8ytUA7*pM{%9^P2`*g1JuX6LR0x3Bxldf3Pi}75QR-)FtaTQ^$
zLWq(I!H`U;M_vJ1hCG+3=agZ{xdT`e>&z=%bzzTsX0$zcUXP1ns<*Pcr7~@+tU;39
zi{L_P58}$2&KCTfZWkWZuqGb-@36@JZ?F!kl5`aM5p{LCoyOOubUQxrh|m<u92bi9
zcD&$-+MR9>PFxtUc!ezVZZ)}+1)&}KER0p%>Gqj37P8z^jQwx!6D_DG)WReofzLv&
zc)>D_oahdTdVkPveuJ8mVW%ZFSjJ++W$_{FG>TAsdj<&V;;Y=^UJ}gTrJl{O2hZ9k
z_Qn$HxQWIQKZ2&_d~y$o!K1TXgJ(g#vs^hSxHXH4RbY&rF>s$DjCR&i7;BUrRkO#~
zsb-tHY7A5Jta4d)S6P|NBWq19h~j#cKdPS9XmNUcp_PlSCI>_9tTxO_UF=pFS1mUB
zDoZ>iR%Nxn#@uREyUSajDpMiDR4nNWtp>BjlVHq$<AQOm<=VI>KCZ1udr3gnRdr;E
zxywGvk{ww}rgq$9V%}M0s7VK>ak~U3nOMfSgO<5em5#OlgqfH%&Q6+Gks-S;$9Ogt
zRmdv265j-us)kE8Z5CBzxVp|*RI6?oXSXIUz#g;a#9?0TC~7kCnFX@M*4nW^<z?D)
zLf}-ykc-M3Vu|)NHndqWcVo0kwOhHeBreVWm8nU1N8WIET`}&+>-CEhj~vj&3#MGO
z?kF6jiLM!u2G?WhGFIl0`YzK>xw0Y{IpC14uD#f^xtD<31N+sOEiUteeTs0IdqwpW
z9qm)ppo=1;Rg)1p;D}SC#RdM~S*Ys9+e0ywTE^RnMWq=Y$?t3y<Yav??re_r2XN(h
z33tIU?raVC3f!^W*&0Q4M{^r@>{lm&w(9Wt!ZiVL$Kmq>n!J&t0uE2^I9%el>pV*Q
zmttwlA7TQU_3f5{zPI9QkinWhBCKgw->&K=U~IOlZ4>Nai#7zT$PIn`p!IuY>!Ar(
zf86pniqB1`Sbu`wgV646V}qrIUUD}C_<;`TS^o)%x>Uc3_BggG(<a*K$z7JQ(G+>z
zC^l!MRp;C2!&|FGgsP#5_F$@Rp2$}1V5v9$I}fh&#vkj!Z78yg6?pYz`+1E+lsgZu
z=~GR_{Z1~PT}2C`4dqtETBBZm3DZsLq3Cfn)P?bHsMidqahcJu(E_w=!i|O|$#%rE
zI?Uxe+J|~vzM~y3!%l}^?Z~3bs9kL63@jYe^!YR|s?^pjJ7rd#P<g%H%xh}WueZ7y
z>$kB@hF|%6MH8TI#Dphzk5JBmTc5c(+m17P)S21#gxtdxH}09Y^(NFn6A8riM=jxm
zjvh1QBmy*?f3RhJkGe72o&<m=v+c7&DFK|3l+vy~B|tklwlL~5!XWB1^`1{xC~}6X
zsZn1P9%_AQP?JC_jrvj<x2`g^6}N)eqd=V6dRQY!3M8gtqf{aUl`D6clSIIIeGkf{
zKr4v_I$o4ILJH2&i99D!tP>+d3j;FGz$RVckZAS}ea5EAMO@C&L|^ddOoZRWsy8Ru
zDL%fx;CptGsJo?BLX*6f#sT-6_C0cHrUmjhv6k;HH~;nAp2*0h_gZDVp3QTd7<=p0
z`8oCkvtHen!wlM}9?G%vGn(sskxTBiE~KXXG0->iWSv-Im9t*;<|$t;`^`KxI@eyE
zN{vytk*OimhjG18BsP#^N4r2&XRckioSH$erWUt1nv#4|Txt*(?r8u`Nr4Rs&?>~i
zn3Gcz=U<B|a^SCXTnf@vZ7dYj+<&7%T|C)t=j<(Ziamy}Db81gd=;s?r`SpTo7!>p
zc-rc<J?i->_K4wGH21AdwE_Z4&zU`0vEDh%p1hs^nqu=n=MI&Y2cRA5{5*6*p1LK^
zPLJO&+6iycaz|%5Bx}uAFXpkOt5+Z7L8UHrDo<N3sb|^gJVmLig8HM_$Bw~aY|k+C
z^uYL#xJzkGx(AM^ThFqSC&O<soKu~>jpl@~rv+USWXTK;r*L>NvFdfMMPszosr_f!
zxopKFrr8ritTW@7Xye-a%*?QbYdmIIywx7%dfeF`>0Td8l`|Sg60Fi)|0xuY6*|~(
zp6i}{7?0(}YJW2$AnG_Y5nXw>>alK-)ljvSfFa?ina7UAV!3D$AZis9q-s%@5zX1<
zS{;Rs0+FG**+I|`O15B+4@a_L9vdNx?d<8*$&aOatj?GEG8t%4RH;H|v$x@U(b;Hh
zog8c?NUnU5d8Us|vQ)}noGas8WyR&423)c=aLER?mQXT%chiTXD<7^<r_N?KTp{ks
zl~&$9pHo#-sXL<|Zf26ndV)6|*GI9oe~R_CTub@GEqK};``mc=xMf9K`(61bB~L2p
zT90rEKH7TAEg>auwb2MYQkvnLf@<^<(GLj;_$#1WnPpUP^G?0hMZTaWBx*UL-pFU4
zcSL=fkA;w+BBt9T%_NmE-A<VoHBzfEx4#F=rFRc6Qal31AHW~1MhL=qkrKS*PdeyQ
z_e{43Cv}O%ztY(zxB}_cnA@znrsD~0R!8ZSt+zeH&gX013^3_YD`&7R&sBHMV0z`M
zU(B$_n<eV~8TQb$@{CCTSKWn0?y%?frgi7idm*o+nhDS%HEkxNY*ou<+Btmvl+5_O
zm62;*3!>MoUQ(_0%w+CYt1q3egaTZSjcQJTy?V$I%NUA?3{B}35k1H!D%6_=c9Kjx
z7LihGzweMwwh|B-Bpzo@Fj1`?K7N0h)zissLz7jZY+HP7Vb-?=5=3LjQr=dtPiq50
z-?pkGYtvq1D{}3n4xN@d?4(L`)yQpzxvNAbfm^v?Av4Zl2(TTtv~{wki9oKYk<Xgi
zsJjjMJp>NU%x_IEp`M%t@jKPtSy1Gl`g)c<e#EgPS%dZ3^~A9xPtaryar0EcZ2ND%
zEtX1~V-J__d2{TPP_6iktF8WQo-N);B%{>C+0rD$xzK9i1_o+EFX&sWb-rBTs^Not
zTToj0h)%w&N`3Oyz=<{MG?^Lb7OZ0)8W+y%M%$RS{!KxSG1J*RreBmH3Bxb;!j^5g
z-Fl<KC&}F_0JNIAKP%{)jFBXK*wSt3eV0U!YvaP1^vxS(U%kpIs8O@xqtEm;Z&Vi+
z+T)iV6puqSTO7eg;5Kh8l7lH1acP#zU~<`~GgzPC8U|J5ij{75(6MARk?!B9J}6{k
zyixtD&`wD>z&Zrv2cV%gpPeD!-=fCPwUf@K2XWSBqG^O4(!>4aX-m!1v(m%;2XymD
ze4_kO>!nmxKi5u6sKPZZTf8>&fUPyRQ(C2V&&4smPJK4lzC5?f*+2s4<s@gB)3shM
z)jq#6t0|&6LN<gVZw~mEkEyzO$Wgm`avqboQXQO!O>$60&9`UpHE+J1&DXm5c0s5#
z!Eb-%s-0I#6U=3hkDgxXYr2up>!@2m9nG(o_E{nl!n&9Cdq%SHa)m_&o%?cS*hYT2
zlYSm{`ys*lruzxK+}X!QBoXe2xVH$taw6!rkGZBa2?nbtz$+&L+P?G>($<R$LHhhb
z^V_vNtg>?PIWM*jnjd0=TIz)&%#MxfuSNDW^RODWz)s8EFFO*{#Wr(?oPO!C-rge~
z1wE=C^MyeZ{sVMfoVsCwJ(a6MPcN{~J`(_9zThtsDc8)ZYgMNfaMh&MGVqNou@0HX
zypuISEmr)~i>(4I4QF259*ds$p=<OkAti_i{Us`1D7}rVV^Nn`UR2_>t2-F%(~gvu
zD%weOiE)NuhnI`n4y$bo5t75|jfHlqFBDYW3+=(vu!tbg5{`#XI2h76bSType~=9Y
zE#7g(A>(m5<tUTsCL$e5QU&MW%}i3u&asCk07hR2bHRwxaw;{RV<%T`6q`_Q_(Mq!
z550u&(2EPjop!aEt^=VXqIW$2p5$XJ$v#JyxXYa`QT>YTcr#OtE@nBaRtt;mNoVGX
z;jzLxAd3bYd?Dzqg=$kV>+v4-R52ENp88#}J@S&u;}#^<OKxqIoUHICMptdMWk8n;
zttM-))>Z<dcFkh!S`Dps%@I8*G-PA1Uqo0Y$ZbuySseMft$Anhx%NaKD_!mREXIwh
z^?ZBOY*?usifx|j%=pDYne7_vjtq;n#?OxmhtvMROER6DKQP--sq7MaL~*5T8&+DQ
zcr2!`J-VaO!il-qDi!Ci19OK1bD;-whvg4Eq+vRHqs5)-h7x;JG??jg^QW5B_7ZzC
z*X-UeL600z35%FJ2UOuAoq6*jgz124U1Vos+`P5O9va$c8JO@^#bKYx`=vk?Ja=b_
zX?dZQG*wd-+U!o5?`?5*SBbY!lYMs`i<pDyZmuRaa!-m~i=KhIJ5s|%z;b8CZlV{t
zZ7uEHo2Q9GpSj0m@T;t1?OSx5#kx#m<dm+I(khGNgfQ9MQuXWwcCs(0OTBf0ogv@f
zyWeSxo%H#O?J>dJH6S88LES2+YiGk^`$oU-{z7FgwMQ<zzfdkWX!XP;TbyBfXyyYg
zY%?^{<^wwtB+7KcANs%^%h)4l21ooIjz9^%x<FmC6oKBOHZ8TM$kAdF0-g7jK5Lh{
zs3yJK^Zo=5da-w`9sd1(a1*MUEv1bnwDTd8W=n0OV-9J+EwyS;nLRZZ_39$@P(Vi=
zXb&ZTkQiJCO#Cya`EDOdP`8!Y={~+6FVp9_`^)T$Q}<{uk#xGXFbs1mOAO%IFEf^{
z)$06Z7_7{x2mFlUgujO`Ehz37sB!x;JH15L@U1s=6~3LpY;F6{=MnPZdfkVg?jFW9
zBkQq_{>rr12AX`@;BrNf3hKl%JFT!fPUAwoN9s@uTDT-?ueH1uGE>Y)+Pux5xjiM%
zzm~fN^2Z3ar>M&=vd_GFd&+R%T4x`ia}n{#yNnIS2yS(W?^4r#dx4movh11L%Yu%j
z;>0z@9gs#zK(?M-mEhooL$6+SCmm9%KDx+G9GKN#yRoE=&Obj>#a@gu$yB))V}xg_
zB^NUrGgXMpAuX;!C=OwL7VI_Jp}H=%<0F83PNn+G#r9xzQkqq&^m10SR<*p`PUP#R
zayu!Q$z?g+o9=H?vwE}~p7Z_tayy6bo&HOh_`XR;RQ9FzXmicZvP<nsJbCJs3YOMN
zb)rH;9(kEINJ=iVC#6r`A2gRc`ca~n_C}pE72VU6O!dTN_K56E%h+a`%Q2Ns_@tFH
z?kP_8rA{^wG>Qy}X5U}NuBK2WUv3YLpMsGZ;96KNOV%afI>oBwa`q~v>PMF|-ct4W
z<@T^)m{nqhSV1{Ts5AG=9PxT<Rv%q%4+_Ovyel|g9NhIoy^Wi!a2nyMv0@cgTbY&q
z+i%+ngl1DB;bQb(o_U#7USnQ%dfsx$TYh@prIP2}dNQW!wq-*l)3t%7X33p4nNcDh
zQ;w^9R@j55PdP5m7Y4yDX9Wic&EgP~L1>TTGPFho`8(u#(h6jKn@?XC)gZX_@X-o8
zE|EGn;#AsU;@i0b#U_9BuxyX&SE=>Sq)K~YNlGxVDJ61J>L{+LYJ={gl#;+EV=ccI
z&rZlC+eMa;biL22ZZP=*3$*@4?HQ3jb@qg^GHB!ZqIUIcrJZsC>xSG{#DMydqYb3z
zi^z4xBpQ!0CanIWEL)Fgl~r*SOW#!o`Zz5#(%E~|s4KY9!S}o??BSVG{VGn<uEG{u
zZ0$331AUwXVv_<jy{7o2fZBA0o!Bp7klP)dP0CW8SJ;<azBfZC&!WkE-YfT|{XgK;
z^y~Byxd;h%Qq~!Rl8VsTT%!&*(O0bfKiuRsX>pr)gEHgxs~fMhQ$szBo0Af`*y3Ng
zOD+!A`D5_{hcldh&TQrCiAxak%udG!jc0q|toP=b2dBy<r03X#XeW<(X1ACerB+dS
zRE-hSZMaTBb8uc9FILNie_B^vd}eo%V679x9sANv4Uf?vUFU_?mV<*h3%Y&FcDUmo
zT;bpaDdO0Rx!Wk==u>&>`m4|td8*|qJCEB%?_On(99EEOMb~<k?^BI=(syK3lr(s{
zK*d$rX=lT}TC%l;F2y210#EWadb-3zl*yr;4pQTQs;RPPT?Q3hN*U8@L_4lxyX>l3
zG1sQoJ9dZ$o8N93XH(;B{osN=>!V8k@oK~*%9h#zePAg4IEcKRsRplPl4PoBE4e6)
zj>^Hb$l);FT2n8=sn1qv`E~^x1!q3I-Jq{y$kdRix59$-uMfJeJAF2r`yVTUlqMnd
zNfQz1iBcN&`M7P>J1gz@5$vd0hXnkuoz1l-fb&7cJ$qElDwwcG6*^zHuCk|)>R4ru
z$wE-l(P$Y*#deTQtU%6VQVXM>>Ofbbn)&rfx1>a!akZU1w?gJ?BiCGllxdsp*A+p}
z0sQMK%Xe+K_xs8$epsT~YVo&mTM$u`KRUUuMy<J;Yi$kcr&mMIW9r$f?NV-zL{{79
zBo>HPnYPs`DB)A<8r?<}s8!XNA(iUhYWrL)|BtHeaq;^)9lKgX_V}cZt5MhBb2_5V
zyT(o!d<1~-c}<<1PPnX8`&BJv+_<{$8ny?y>iKI}$8yz&*VyS});Xe+Ag5>@=C1Ie
z#!k+7`p9{Ev6{FV?@qBg*ZI17wLR3X;{p`iXm3+LTg}S3SM8F*z3KzLR_zs*S6L|)
zK(45EB*rh2`NGjGE|)Mp#DtR+k^;G8Z}P%X2#zEKvY7IkgkG-2sI30L>!0OFPoGkC
z*^lhR0Xx~T%888p38-=hn{o%6^8X_?wWndjGc~=~)c#j&$o+r8rdF^S5^OpRnWlVo
z(Hdx9rEXqhr?SP^x(4<hP<z+l|39F<biRgPXD8>pSS%)%Ce({@eNiLDGOlSR(J+=T
z#;G;e+3`NUg${#W%<S!2Q=&81=`PplF4yTUSJB<@Nm%S|Z=E%Ldn4ubCed(jorv>w
z2vd#IS!d&abarotdj2}({D^x0I{WU68Xb8I!cXql{>Qw}rZfd^r=-K&bSag29}yw!
zld>e(os`b11slj|b4+kOrUjkqm9=*KvQB)g?hgQ1Lz~RV?fysO(*fda{}vn&23A`E
zbe}gH790}5)mBoD|L%I3A%I>eha@65-mF?srpDLchh%mnAii9W6)%xHpB9jtgkhEB
zs9Py@cKHrwhzA=4D<OqWQhKi!8RJpyxePQ{?!bx)M%Q3zZ(#`K)UU(|;O<(s91_*d
zN))14#|1a3dsueHE3Ev*nkaJ4YsS~e@gR|eJRVc7kVTF^i`sH8H_ms~Rx6IZrj=S1
zMAN<Ylb9Z3NHhZ|NE3x4?YiQ`N9)BsA8j;HLjRd5p!KquC_axrx+|(A>@LUYVI~Fx
zzUh79-m;sRfSK`u*CIjgcetQZ0d=4^AqCp%NGzY=QI^ppR*J>PZt{$-e!>kwJ*9dg
zD}uuoGo?YDxSlyysS<9mXC$-Y$+1{m5^kR6Guj4)y=lN2VW^rLn0X!Q{u}K1ED?XD
zg>RIhM&4*=#3wjQMY?9Xj95sYvR_?xqdmB^c$-gFjEg#4ghxomHtePbmohp-YV2}p
zI-*$bHuYxCjccI#zKbAJs@}ZO9+QvJUclI`f<k#jp;)W~>M_a@>r%xXsiW<^(Gw}$
zgm;*6Ol7UZ|8qhWud|1w46)({t~YYiICY+uyxy28PZ4Tk0a!y~W6x2;ZnDo-yVu$A
zbE;6_$TbC{YCTV=IjT;32SvCNkLe>v9p!2SWS1zfgmEV&Q8n}?mZ$`^@+LbuzDth3
zs;&J4-W%vlC?VaDgX*E1*oCF2&YSEpIVqTILc$#jwpx36drxn?F;d5Gl5rcDOH4Uw
zpVp${ZpKZMs|s$mlS*<+z(Z3rE%*HYa$Sd^I<7i1ZI^acmRiRm>LO_*YQ!APO&lhd
zJDJVV1!~95c5<l5Im|w6b&HoYQZ9R??Wz^WemD+4!tuP`xIa+0)rfE#2#J_|(oX`E
z(EWw|?ouQJN-va?p=zs^DPL_h_*#8Fwcr*zQO*SA*dr)9+4+;&tcl`_b)HkGM2<cG
zr<oA|Yl|N)*=ps^L6OcW)QQt8<oP!Z-fHE|LG{cjl(?avbIb_*ue(k~@Wd08jH5Gq
zIXE-}y%Q_HQp>YDJ@@+V^;zvnFDIRr(~Izs6KYH?{?mh&fwjdWBty5MF1=|w^q?xQ
z#U47YZmPA@W(>1Lnc|5}JT#J_MG1?N4mr<LG-;I!3C0HHqIsE>zdztiW2U@hC#XZU
zkS$d}TxeyKYqzp=B!!uBhRBn9aa@K-=?J)<cysJ7?j<dC1jvo+sWo>0#waF~6F=PH
zb6{ym#}Ff=!rB$l9x00=|L9;GL=j=n9K9`4d+l_$<BG&muD0?EAfIESXd%uFs{3!X
zN6g8y47_aC5mTSzxvDGkgd7howc5>Ad7i-)U&$TB{#ED7?G@ea_%?OwRy!?pu#h<@
zXGZ4uwyc;9K^`4Qh#Wb}V)E!rPK0E?EagWt`$SKx>XdUdT<AUxJ9bx``^b^B9J`;B
z4T`ju(FsJ|NrVn_VJ2K835xjR0LdH&Lo8@uaTv{|WI<<KhBFH!n&)gx+~^9oLW%bK
z>AI*So+}frG_C?Tr~!$FPh$8frC49c(~czkjR99xX69NR4~gohQS=Zq@pL;rR+FfI
zeS%k?tAijQW6dO5=|&F+oTtMi`*6To-(|?KHGjuz1}?RNN6b3s*%K!_U$uu&!TD-m
z$R2I8ArmKkT#}kmXOC5Bb#~CKRa5G?mBbHQtluy2L`jV29MbWrn-em^WSLMDaNWSB
zL|up5gp#0~bLvHZYzl*uK?}Xcrpn%pcO(&IZ*yZ@q)36^P1D?IQUf>QjF8i0ov_1{
z4gDDOdYLD3V+-J%Z=HS^Y#l)44~7oQb!OO(ksC8)$0+m~mXe_#2ntWq3G2jZbRjny
zBHX#7+fJx+U3c=FR#vCF=61WEc-3X<#KtidQaU2^qypx|j27XO*3)iW^JZcmvjsXy
zzY`1VERQ-9OOE@sLXiA2jvX%VJLR$xWOv1i8cgB|o}^U&O+2-jy2EEKo9}?i>co@m
zKJ`1%yPBfqC99sBO^&2zb&6M9cb#=~oR&s+)@B`3vwqA?j^jaONY9Ea9o3Z2I&S#a
zbFI)z2n-S9>;%`g>ozAG86=k?bcYj;6iSC86B8<govsd;&?*eMK{MorJ`AZ6%Xz7F
zD7|aKB!=K{q$KNfj%1gM+FWYo?~BybV-&*8Y>rvhbF_hw;@_6lW{H*B;b=C&@W>uv
zh-QKt*Q3ww?u^jBJ*jhI4LJ5uZ<8vhFSHKCp{L`L{Oh1~QZCFf{rV}f98|a=Ple#u
zX!eV3lc}NR2DWh!CyHK7)=Bl20Z*2*6@^dBd$ZLa*0bqoR7climA6<tFH>f<$kSiW
z8L2cqATEw;tV-Z<!BIpdr!3c-Jd-+7WQVO;Flm?8MKX0JH878uRC-cOYScauoxhQM
zA-7H-;?j-O-)5O*@!_~rhg#Bqn|i3;P6+M5n#G)UsvqoCB$E#EbR1Jw3(BOUU4FB|
z1%IT^T%L5)AYbdgNmwGzO-*ys@v!#H$=8;%GroLUrnr6O6zYQIPDvpp8D2@T<?$`2
zn0!z9<zVu8l&VE33LP}r8hi5_El2xNB7@9nQV-u@51OZ|wL%NGMJr$(>D0A$AheQ!
z*hXIOs3R?*IqhXy${mU5R9|dh`gB-Uqg+?fw9e^t{7}NZT>+~gKh;BuRHNuk2gSXy
z^25Bfy-BNapD0y|1x;kZin_rlO3O|K3GKWTl(V@}racc@xO0MXvxB!lIc-v0os{5P
zFO$z1Z=8%bVRS+-{ZsXdqzRA;(fW)!Q+;@c9lt6!AV=>GpL2t_tvTx!G0M#^moq{g
zjdkgz6G5%M1!8VWtOt?bmtr8f1#+s6+bY+`D_Qf)L}`o0kU{1asfs(HUy<Q&47%${
zZjspQTBGFFYkBbE{yRKM<~H_95OM$dOqYteP2O>^Nz_F|sJ5T&9glNq%1c6?Omz~6
z0{*n#B2uFA^s<s&C@Y(*#j+dK&<4DS&5&4>M67I|Q=z_<AsaY7C5G)yvSMVR)x04k
zy6n}TIvpij@t&yW?w8|I`4bu((l+LWR<m^=jQr$al&-8RNv+8&1hU+yjs{L{^3`7(
zY@Ehw++FtAq4}0^KsM0Ufw-14`>tD)Th*Go>|s2qBAXICgdW`|?^oOIVmp(iUb>6x
zpLy!DyX+<S^$PB`ZwC0wciSU^(}J4Dt~ll$Q^)SM6B4>DL#|R<P5ye>Y`LXVyH(Op
z>@2ROp8FH~4z8;m_z9bxjLnTWiZWDMqg~C{&l~NVL;Kyw*E81WlQ%~Urs-S;K(5$C
zvN*}0ryYcl3_5=)A7xcYj@~m(JCP*S!g2RaA?Dd)EIX2Oc0jxFwd6=#R}r8BcFKOW
z>8ExIPQK@UYNwUgxBFy|BsYtB_bdru%iRYOyas@TElb+j^}I@0kaM<1XP>|!L3Tdx
zBhtANufZuJ735D&R`d6&oO^JRv0b<a2m3*_{vLbGK;B^8$Wzp^Fz8WEZ&CkqkDXL-
zT;@roRn3j@N(}-C)8m8!RaW^5_pwTWO2rhZSW7)U&1F#e$JLOZ*+cVLSeWQmH75?$
z);|B|;>Z<>3b4JH=<W?btEAcQT?TaVn3=Dx{h2)ovTXPneA}y@{28LQR~;mkkb&?q
z#F_cdy5sVF`au=9kq7z!EX(j7j&L*woN>Xu6o_M`@E#6FX-qz3_&4!}ZQMqJ(A0Gx
zq}Tfnt2y|%_gK0n{^Wee66o+I;8d~5eb?*f%CGlk@5~)?h2=^;`y}p4h(xiqg^4q}
z5<KDe(zMXB)2GNwQ>5j9B49;mHVC6YHn~m>-DHmnwRLErph(1`px%*#f;LAE#6Bv3
zNY33Wv!KAhZD$K21#OL{>`~;FqE2`Yk%A+rM+c)>xfp=1pJY~tBTdv!?$YNInrP)_
zGW?f^n@-F<a6YqLqn>=wqn69eIgm|n_>M+2N1<#qP3IVGQ9_JYZxuNP@SMQ5D7{b7
z?3k0F5`PZoYt)RN+gD@kKlyWe3m3f$@3rs2_y5klWHzdqP4+MO`e&1UdkCdejs|Qr
z_sQ|BD-?6kN5g&B^IB+S@}>Wv-<&qCX#1{DvBu${$1KvL@p4D%T;3pYT5hT}j&<k~
zXNYs!Xw9!R^{Ju$c^|tGp3&6GRA$_qa@S?Xi)D~{hFG{!mMmA`=CYxt_lvA!yxV|*
zYue4tOaXw4&)i&nW9ky#m~dRqs9l8I>`R^OOWlUK#SA1z21wU)8FtWj{YWb)ipt9@
zkmZ6w$KHr&d6`A(FPpiAR%033qfujBzngbuwE8-HI!31;K05`|I^YPgjz=06KSfG@
zs%RXIx|<&n+neu>V|IzG7fY<zp;+E7kad}&2lXtNy)M&V9+>5US(oCjR8HZ(RDW;T
zKK+U+Rxy``vpZ!T-k|5<4a`H`WL}%lkUzSI7Q;)_9~8&__3CS7=Ug`5VxGs;7)5ec
z|I|UwP7BVDxvNj|{IZyuP`DH@jAwq*)X3_4t!wnt8j?<tJ&IFQ<;@4_&B#tthnqRH
zNm8erucR$@qFkrQ+93rj7CWRwe^WHvtaI)VO{|kUL@DAW)^JE~+`_t5r*>_z$M%aA
zLr-W2B#Mr2u}7FCI|tuy&$rB=y7VCqpn|Hw`Fc*iYSf{J?6hI9l}T?I?Q%iwsLAaf
zcDBe7U?$Y4<gHw5tx;!fwMPagwAP8kJc5VEbm7jcx7x4x%>uRcaeK7ys{*xXJF9b<
zx_vuXRjH@8+wm*n^0a>rkAHUx4meE^qdT8vQp>yXRl98aMD%r3eEx3Kz4#^-Fz38n
z=d4<jz@Y*^RGHq&#j*l*Wve}9JnM<RJa(k_fp+6d$in?ctF+5?F*&3AvQquF)lN#P
zw2Ua8vbCaQdzX1g_NOwB7$P;yNA0)}HINrwrlrO$tq$+z7F($-CC*ub@l`<9b>Y16
zRg#PuFOr5v{RCNHQup;ffc0e;n-^T0GKUNjj<GC4YJ3rkD~KWt1t%r*$ScvEX|O-v
zBU{c@Ru;eh=A3f4)L{TW-=qHUD6HP2{w-fe)u_kpwE4xj7JW57D{ncUx-wPoEtE%6
zsyP=0V}1MZ-crvKWZcVk(a{GEZ0diWtA6$vH-dB3?#JMIq58*TaJ^g&d)$t{`nWs<
zuPJaWF3j~~S(@wQOY>H_gevDe9@32Q^VR}AE#0W&e$cscdDYGBl>oqemLbBIj=6^H
zP4InIAi_5`^cm_z9Jujv2$pvw%$m=7!s7UOAdJ}OsbO*atT`->5_QE<qOLef)D=gG
zf{bw28PmZOkrSTLRomK_H7V-lHl}%jdcO_I6sU+N?2(*A=~v{GaIn0@>XIj-wL}R?
z4vKt0l6*EW+NbS8tx*?0VJ9ZC9?2uwMHgdQd4IHeyIuYC32qd=W2v1_U;q`Wzdym%
zsX~?VB##>vsU=TxK3}5lc@kGyrF!y7`~T_ddjR4puD$Qw9m4KqlPn2=O<0mmAjuk0
zi9tn0gOMm&G-yzXqCv$<6e}9ERH?frL@}ZviWyQ-v8FArq9uxkT3WHvKB{PGr4=o$
zsOX25*IpGZRjS|boSD09^nD*~+&S~-%$YN1&YU^(=axVkzW6E3ya}20xOc?@o6HdW
zY5UF?&Xr}4V_L12-+$a2IzC_NE--(%)Uj8MVVuo_%TSTQ1vN#iPmSSn`P$>&Do8_q
zr}uQIf@miW)sD*6PH#rI0<Cm=&cPYCpxbjMToydP@&Nzy5X%jX^*>L^zjnfB9h9GU
zddu*$`3Z0CRoTkGF2fLKg3<orCKczwz|{kAc%aUE8{d{V?g(W$7=KE^wJOjE%zr8f
z)7wuJe61#o@z%^%GB*JDuAPySPhg{JMlN{LyYiTz79=`o9nOJB<?yp8bd$Hb&hXr1
zw75J`oPi;y6Jq4IpY)!E!_i$&!eAehe^5UkKZ(g@NG|#r#^8`#|1)padEbLh8ZhT<
zzF(ByFT{{J=7bfI-|KY73XhgLNEVOrb`qO*v+J;-4yx-yvioP=P&x9Q2_ZR?wl~W;
zSC!Tob;b1D9&@0MJCZqGS5ZZ-AjA_AbWNfQ{)fa$cdb5q#%$rlTA9=34J}Y;+%&s#
zp__;06<ui8uxwI4&vtn;*AC<U73LLqJB{z=z}9whf46|HYF;Gj$Jh<XXi)Qa3#4_>
zyEGN4RZ!HFN|qv5fV5K&LK3RwB?rC79icFD_kFkA@4f|eIsmti<d`t_%AX&^Ria+`
zKL@?1WajAOI{8}Lkciz>z=Wy%H&x4Xp2Efk#vtFJAH)11##|Hnsu)*sQ#D$J2VixH
zRx&Jh=L%6k8iC~LrZzZDX*OfRZ-R<LFNy{i=F}JmZW^Y^s8smy#EQd}n?|U6FBAi&
zeD5i5GECWjJ>|_@(5^HT=b|><azJK3jaj`zp8GWPONZR{wD-u$F}1UW-Kc8Of-mEs
zDxM^6jpE%7l%=NLii6DhO!=+Zu`}hj<~SP`#w|m9b5|8?c9#28GwQ7!m(x#smo6O7
ziytkN&@YeZ=KbHZx*-CUa=ZF@s2c;QQoh)Y8v=jv%MZG}8_gPd%ro9aC&SvPn^Vsx
z_(YdEZt5^90-1mFs}nE?{$-ScO0_Z(f31^|XT0nEp%SztrgHxplCM64!8Is9dImac
zP^LcXUAhF?-y`<!#vYv~&n%zKJz65qeHOX#MTloXZ%{s{etz?;H@prby&49msMb0e
zM7%LZx{;ak#t0w#b-5Do#G7fl(fT(7GIR)N0`lxb-a;&w_Z-4ZG$IcjLKP$O%|l>9
z*}ficO6j=T7b_^{CIRu_WFCec3l#<R+4{krYy|nZ$zYkYH*5@+D^D18hWbO@ALPiY
z9w_!4`DhO&;R5+u59XK(`Be|xQ!}xd?L&5bqZAozAHWjMf#l`jvKFsz5tC2i3YQoM
z$?nlXcp04_)^|>j#5{%yFV~1fI&4ACC*y-Y-c}J9^mwaV|Dn=HO66le_nrcge*5Q`
z47=o4KZmto%VVDNZdH*B1I*w@MldUcNLkSzl^l`JJ?C9==?K|O9L<rmK&j0YJ~{-$
z1>5s5k6;fqR<Jz}iz@eCyfF#e<7$x}q;Oo`Xbf9>o?QD2Tz_wsZNKm?&BK5W;OQa}
z$iy*eg}W6?i@5xpqc)#%kX$gG?L%Dp%%N|V9^$1>EFVpLwSA0VOVy*{uhIT!Sem^!
z%S7?B^&<AjUZ}t}S=8&ziS(;op8~82xkrdw4~^>6c9ZIyJ-X!Ka2snQW+<QadKlV`
zPH1FfqgfQeN5e;8Vyfw(!w@qA9xnI0HpTe$a5<^b$kk%Lf0n0|#W73~21$JB{o!()
zkK}c{p^@k#fC@<r2d(ru2XN>x738t;aD9*tw!}vJ!}aKWcj?~M@Mk(*yISji<YksQ
z1wn^XAximOH|GdH?uPQdjiC|Kc?a&phwJ4_zw|D-MB|`%@RX{XEyx<`;a=>Y#Y**(
zFhXFU%dbWxHKaH8V8_F7h{PtbhkNDvUwM}v&Bc{Z0D0q!LuEO|W9d{TFqf-S6z!>(
zUB7~nS1<qeE8M}Ym+8;r2ztHT{5(v)1M=SIp$mp&-}6`x4au*b$3dGRd1fCDs}IR<
z_rX4H_Q`#HIJy{=FZ6jg9?g^8m7>9XkWO^>@WF1y`MsFx&>?D;XJ+halDRKHqYcT6
zU+^wDX($&SD_YbW%`<qM)tRiDXJ}1yHJyLD+DV2f$#W*7*=xurU+|_S?-fCmayf&j
zxlR801-z0|M^2aTdPLnG6F-WU`#i)*UXtv^MkLlDG^OorpvNdwu%(o?ML4##1P&>c
z7eV#pma%t?p4no>x_1EAhzw@7iY4x%ulIm(pDU2}h^VY{VbMjCy!zurdGD{inTsh8
z6li4~5JM<Ai%Nd!*WRU}X2Ofp0~NJ!;n2pSu^0LF7CYfXo)@7B;hH+5-+RQ03AKmG
zXY03j!j?c6lvl*uF2?pYxwjvSPuTsuGeA2#+XKAO%nEsXC(qv0>g_$TmD+afw1GoP
zO>f7<Luhd-V0%ER@$C~Tok~r6II9kVnW|JRl=y6oC&yKZC*Dwkxr-OWxLZ+}-UFm6
zDVT>7XHeoAIJEUQ-gcK6-8X<~GAh@opHlU68~t?1jseJQw|sK|GTSW||IWJ!>yvH2
zgSP9Jcl-{r+Ak0O4q0a8yT9`u>$=Ay(_ZqPlUAzN-XXnD!ku_eo4oZUY_qn7aRq>B
zxYy@wtQiQZ-9YM?d-1JDr(+Q2l>?)=H-J8zZ!Mhk?AvYR933{y)P*o3$QaSX@?Jmg
zG`ZZ{YaW4jZIT1JuJCthRxmyDfgjrqB=uBPqs)5Q`#W^|KVC+Q8)aY+!>Ci9I_OP3
zCq&J-k~Ryt(<uLq_9c9Kczc~Q??c3Hgn(e3bOEj%V4Xu6h;nx2xeC<cg|c+XM+f1y
zpOOQE-mJ7xtgE)e;?uNj&yns|yf49_^Xe<G7i#6_uYjalne{3xms(l;DyHZbdCjZr
z^V#*P_m~1mP_Muhm`Tx=Fnb5)t_flE;oOIq5V%E*BRpgw+%+9$BQtK`+%+wyUWFE#
zmifQ;o`xqGWaRhWV}n6qGzvUi+%3lO!;D3XXc*qblr7^^kjCyXnMN_~zdKBFx!sTL
zMKY^BxKriVqPDxE+F{D>u6*hRg>H8*jo2~)-`Ka+jJ|uI*oxwSUTzj{o9NPa^Hpr_
zth;k*-G%Qtpgp_MEJsEVrZG>=yt}@hY)Tz;*VbYJVxXZGH1OcKyNQ~m5H{;xBn#|r
z^x{HStxqgeS9U?96YZIEli{^-y`P*%LVh^pT@V42`8f@J{93bUGosajZLDQ*dY5$>
zB3-*ZY63-g{$0k}tlgeDnJF!<6`9}?YigZI<0*xtGx{3G1H(N)(P12H3)d`;!;jb)
z{?dT_@(<p?`Q`B6D8xF|LENa_AmxW6M)YbRUQN*s-KC9jcpDk-kzLdUOt4O6OQdwr
z+2yBdnAE?8<!OJ!efS1?wc}Dz+qp=4z*TCSyzP(P#TUE<4zVw&i!6A?^;Yi8LWH>v
z9Tw^g^jo<Y<us)m$Z>(PWH<*copD6m&8XjMk&}Pq<wo*rFdv)cs@HJsp-*1=ns>kJ
zZ6T*$^A_V(hK<ACvs~{OvUwPa3%~ut-ZidwgtUjfM`T31+#rEFZ=AyP0t@{?OvXFq
z@qh9zI|0&Cj9FL|<L+9Sn07VrAr2=~KUslWuttTellEYo_#xT!CvPxNMCaSKifX8f
zt*Az)zE>pskP5%hU;pG?axqq8>V!xKTnem7UD)jc3GbEB@uQ1HIc{%WjF~eo5{?B#
z(mEwBvWz3Kq;VqUd4Kk%URa?PJ~$9~1zxV9!$PdM-kXfAP~K-}DK8=3>*IYW(roXg
z9Ek~=2he*3Bo1&$vmO9x|L1`G<)3lf{eXP?&v3HzO4sYO%@yS{EmU}d!rB$|0WL6*
zHlXNn+TBwGOpN2zNoF`99HR}YGEoQ*c+3}~B60H(k-(*v{1ad*s~)(wHPWy`dKA|X
zR#>UB(vaKZJS{a*O7Y5$Lw#e-@(-_jgPF~EiINagm%j~uSHJ^MhYhuuP5*Y2^pAKK
z(VM^E7Zx(GJ#y(ZDl_q&ie+e8EEsFfJ3V|Ce=}~{LvDx<>d!^@I^LO<H;&*aV=vuS
zQ-2f-HJL5;eu!P2?31x7I60pIq@yYC^g2iRnbLO)$WVe*`qqIaem|A)0L#1J4j3aU
zYaQ;qi-v3NQxGJQqI|bEoI@s-1CZ$JcwSZTWb*Df=}oQ1-^F7yZ+Mp-k4c{%E*N#=
zxYTAbm@qVl+6Tcv?c?BJ$gB`ub8z&kf=E)FeLqJ&_y&%~!UHrxTWL1Eb&DI{vD8=J
zz}}WhEmNsvOES*)RrG}_8Voae8Kt8u5RH4DR2N$H{s7$`XB+eVXaPx@6Gr=DK^h^H
zIa+`*8w*~%csA@zc&etTtbfzHY-5u;4#h;hRjd|qxp~z6gzc?zY&83IxI*!bqHFwg
zPyj=c`14kq9DCE7y5s;K_qq_ry(YD&sr0ds^#8@1I=9=S?nME?Seab+7w<2yE%wj9
z;3!yw^o)8lwm{ovp_jAP!OUMr-g|f$)gNlPc9fJYo(K%E?x!mm%FD$xsMKvLbsIdu
zxQfC>sbrtLZ4|ZiNjVDpC|CY+6mN&;%KxaJQ~t`(_x}nDx<>xuuW)16NZ;SQcj0H>
z-(X@4%fJ22doq5OyoD^ovg|Eyb_B9YC!L29dJ{2hSl?IropuvfDg#<W!C~r9Qdvk7
zBTmbZj~<cWL}hz9n9a(%VmO5_Kp?h)AGiSfoRXJg1VA^^b$6S7q<air_aeD$42r5q
z7L38JDwmgx;aQbhxqA$5qDJ||7#iIue>>(qssbao6e3!QbFuU$0rTwgA$l6-GEoeB
zD|V&#a<DlEQ(nO!?KkV3S3od|<}@hZT2Z;@mW8p`p%~K@6>{_2Fc53yEpKD;s+Ett
z?Ok^?*ZFNRM#BO&68Um2Z#3w73b*`<$&fNliCb45*!K>O;mpV-?_ds@kte+4JvMI!
z79tKiiorS9oabp$5q?#OA9lR}+;7E6N6|57h7LSxEBb0%KKc&Caa!{qo-ctnl4G4v
zl_mf14mJ&H<p=M0mo1)&F&$rFnN<u|h2`>hy{UL-)ELLB8Di2skN_I!AOy%o>ZW(W
z@_c#CyI^^#-2bk3#Tk{t=x~F}UGAoYSvFtQB#fvV6K10uHtAMkDWhnHB|xs=2KShH
z+e1ZnN^>0bf_cO6v4~L@R$$Jiyz%WydCWL^yh@ggW0l@1?-}==wWLzSnELHe>?Al<
zT-_?|aqkjWl_7oadDB-_fkpW2hDeH;D^+a==q?2dng>RnZB(_%bKdi=ico@LIYsvI
zW%mnUR^#pkaT<Mj%@jl=HltjF&SF~#+Zm`I>DOkp#U{&Zt4kdzuWE}iyy`$4!>a~_
z(LgUuG`Ty4d5R*6>Q?a=srXnA=;#Yov@_!{3#&%uG4Feij!a>;f<lHJ6OEJlZ35dP
zuxs#Am+jW(4XW9eh0dc-o^-i}++<?okD~MNA`<mdo2!?sDkr)GoreJe5BveQ9w#b)
zg%-_LQ;xomVQ9;*-uIpmxyD15<n^K!Dgs}1CjNjB=c*QJPjP$7$s0xq1i(ed{-$kt
z0n7tVke?T#-U4aYV9pA;cNXASQCRGZ(m^Vk>u|G@kIE~+Lr<U@22Ly)*A&nRA>Dv$
z8tJYz9x$&?T)XKuBZ%{aFhKDK#MgpDIG@>tL%&#^;wExHYzIKIp_2^Lhra=#B;i`R
zKL+YqEUpd8<rC18c&%YVJut*JKTcucten|S+e>sYkQ#dO3@smdf9vE4S}C)R$oEl@
zog610phH9I4}_hB9_3w7xL-ucH+HHh8L}l(ieT;(aq_mgR2BVkacO=4tyv>8KJYF{
zLx+>am5tpE_UCb)muBR-A9&L$XZT<popUu_LKvrwoiIL&*Y$$4HS{vP;54lw8-kS0
z3FAboUEzTFxQsc^&<uK6o#*=XJ@R)Sc+d5}m?r#L=!vXsI(hP9nq2*%H_!EAKwhqY
zAN&yeAh!I)hj6y#$~QlRldej7{_f343F<qPF6tR3|CI)L;@{x~AC=qw&e<FP?mapK
zGI@e~@9zzK0|m#E;AU~r%*_RUpCw=WyLV+UOs3b~HF%zw(w&Kixc2)4GG!8VPRk9G
zu!cvNF)XLAbmYeL4#H%FX(~D}Ig%lVGvjcX(Tew#ezL%wAQfXqy*iBR`yOzH5dtD1
zAa1@o#h3Osjp?MJsxY6#S;8|dpxA*@-H?~t<d>7)#f$hU9=2Ux?g--@0e)!Z{$z7C
zj=`As)XTE}@jmIg*N~onc+WajhT+q8W;s*lYiqa|Dvs(gWnm4pBgv{Miw0eDZqaWB
zazwG*^A9{CDNE$qe|lGJQ=`L_&<u$XtHeAYp*A9>--XZO;NG5Uw2IKE;z~8r8&-{G
zzf0`>7#xX}jjcSX!%&#23l~@DOHHVtRe52kuGYdFR_7GNRT?_GPNtyQ3ZEJonG}Ri
zO_{%nb7u39{Lw$XnU$~xsxYpqwqt)))M80RDi{U!786e9$KGQ27;<==BYL5`HisPH
zl&6B0Qo1pFG2xAnavY<lc@%q@_Pt#)+xDhsVs7gsb<mAnI)}e|dog<uMbyZ>gYp_1
z)9awzWy808pCO;Iy=x=)K^gNR$q)ZfZXvuwOyGTCzxyVPNCzORpNP7zZIE;`ANaek
zt(&VN4%~Mj1eXZ+2NQo^M}s*BJMPp$nt=O2B6&*|!U}h$I5SM=VR~sY=!Mo7+kZ(a
z7s74cPbZwNL#0go{Wv!95s2@S9Upmf&*_3$g0_;~r#heo9&Ba&nJpnWeVsa6Fb1)x
zp`+B)Kfv2k;A1m8mZM4@@v%4kNcwQ)Y_S@6Rp#8x<l_YPoZU`C2tlp9{$uYV*w?#0
zhI!U3J3oeU=$E4(dkdgaGCuK^21andnJb9>2aY52fsnlG6Yp`62g>l408)VLfvOTV
z6Ll~~zynnz@!LdVi5|RWYoDf@vf4e43;^vuOfA&LWw;ur$Jjm}_L6iJ`~0oWx{@*n
zMJb)1$`gs$=ElsA_X;Vx1so;qg1LZX4wdYq2}mDxB7oOOI)U}G(>4^pcS`P^!a<(S
z1~PD2IQCE&RUsEL?CIn~O!*=cvD8s27Q=(>n+j$ZDs2vv%<E3yGomj><v2{(Gm_&p
zanA^jTNNl*nF)Z%JA(%l04)oKJsLz&R&REnhfnIZkQSiY4SNy}O4oL{+kNis>XrgU
ze^f4`pL!QX3e~bAaQKxW<74PyY{QD85i;7ehv}0x%|1M{LlzC|;ZMRe6Kmu4lM>pT
z0)zcsboq;C^-oH=5@@>)e`qnP{<u#!7|Ujv@|ia|9Sq_jlLBcE;cjKjg8QUi7JTMS
zIk}#wEERdsT#oP<pEM|Mdrl{HJhy(TtBLH!1c(Y%hPtM@<etyG%l!Sy;{vBBIUJ@2
z<*z=2OMW<3Zw0>yQh`!j>{B!!x(>++O~z?(neg;8HQ@txZq%nK^mwf^`+S;(4>s~k
z7LXj{(|moqgdGT<ggnl5Jp*AyKRH-G$-)PEVQ>k!j>Vi>_(iFA$PW7?OnR0^j`2x0
zNs1FBL{69yQH~sTq;uvRbz6fyzn|nPS!RTAAT!^%UZ_G8L8TG?j0b1nBk79X0uO(p
zlA{aa%Ry8?t}jL|u6QVNi-=reId$$hiQ7#qZa2k=TdVStOn5r+g+e2l$o*(dQ|#&>
z!Jh@7+I3IrFmMvm&q6Xfja?^5mm0q0)YcV#hDaRH6D8k>gP}(n=&*cP&()%iMDkF4
zR+2-*$q55xEQk@Ub5LfY0^%@7P#HP-9~>b0?0`Jw-`=Gce&(a*esgpUs^(66>GOc_
zvQcw;*-U$)g*CMC*(h{`(^?g#TBx<-^5DO*%1n{J`!{a6rI70zH)QhNH(}Ob>H8Nx
z#7hkWRtoC&{40eY{{^`u<NtEX<#IW04|1SfEEv}`mEXe$6qMy2;&2Gh9MQhFA`Ij7
ze`_q(Z!|i`EgX+2)s40J-VQgG%l-?;&pyY}dW?^Tmx^9<8Y?#L94CG3bMH#mw2+Bk
zcr%t|lk4#UJYYjB*+y3zJ^Ke^6_lWD)7kR$FW_y>mREm)2OF#8LtkKW%9gKu;awdb
z$#?G^g`7|aPLBu%*s7Zz<zeM^(<u0NqYTb?m#u6@lQ72DVv|Ea2dTdKcbkW98R%yJ
zt5;Ucz;DbX;X&s4tqZ{NdBpvXQ(}5kvFLLTJXti)x4LM$hgxESzaEWPozNf$XJDs%
z?w2z&*yrk#Oa6mb2rw@iq{q8PF;{ui_v5-o>Aft!2Iv5|KRBDBl=0_P^eHG!_va8o
z8X2W^#5-Ld$CXmt*Ye>#%{maAKx{ki^FbaO3?Pj1u^@J@DGSBYmjw8a<n8l88W}6i
zEE*V#H&GTm_)e9mpfj;jv|fu+LOJCjlZArx(!UdSdp4+rC2)U{B5(Q9`vsO?H-F{5
z2tTiWg^k&o2j*Jq&AJE9vDTXP4@9hE&H87avMw^|Zaq&zDPgWu%{u2%EG_Y{rgYqo
zDh{u=(??)XGybvqKd(R*yR2n!B(l-y3$YHSrh=FoCJ7N<HB5)YG5%3+PLmJ2tiVa#
z|L-hYk%lj8sFJ*#brwx1|MOaDyR4(&U|Q<7R%Ul<!vU<OFH=^FJnABz0iN@dXIK59
z;Lir+^=^x<V>P?2tl)sa*}5uGd<iPLq=6RB><<N(yTWqFZIQd|L${RyFBE=)=X`+)
z7Chi<c;XoHpX%=baf>hU;d~rY>)JgKr*==5XEp=sR>|8$9mj8bWS_?yl=lcL#dV2K
zJ|?U_oRX|eu<~IxKAK=H$Gw*q6Rc|NvggdP?!wP=bF6b0j6*1RBScw6M!82WNwki@
zdF8@HtFo%gpr<lfHZH|GqgrGrc!7~L7wx*V&7gyYtZ5NV9Ulwh!iEmdw`$E6^91!h
zbtfGt=npNq%p?6CYl#cL*&get?91@Em@DrWSIvIx0x<@L1HC<|#uN&Y^rOTsk991S
z{JF<UKL^ttZH(Zuc`aQw&vq!jtcezP+>XnJ$e4$MfLbFAP_AVE;LbD^!6)ODWoukE
za%j2NT4x?LMbdZ;eJVQEJt%iEqWO({igtD|FP@UOTUH9r#q6`J<M1<RSx2q6)gm=W
zs|4NR^Juz;kidVbeEh(89w^~G*br0g<eZjyNsyRnd0~=88(p^~Sw}5`Fq7q$qOMsn
z6VGdsy-C&*jMCSWtStP@Bv~in=LDbiYgb;I{K98tcpEW-&}(_Ea`{~AH?F)cIWgDD
zq~E#otgQ8UAk`0{_ou-<6?#OZ(GgiSSU?VFqzqwjD|`Yta)NV!R;8VoEo<gk0n7(?
z%>%)?@+bHy$ydYQpH9pFgFh9)I5+=^Bf7<vhq0ET`P&Q>9jl$|KM|Mq=A(7f^2GU8
zC@?|PV0*fBHY12UaYEiO-%43M&F|GI;!Xk`%-p!D7~@I#vU5JjEtW6Mx6;dtb!XFS
zSQv}!(>ke`5XFMZy@h&CYS%cadOdJ;bPlM>Ih?AKy5wnoD>c%kYbcDbp=-7V1Y<Si
z!}co#rwi%*w%Ho;>$IMuxB7@m(&PDckWw_3f>F=B-Ns$U(vD+J>NqC7L;EI-Q#yRE
z3XD*fpc~)-$*x<Esm|y2?oqc$5$TzeY)7L-voI%e9jA2RQ&t~wq65>2=RY-^PW8#-
zldV-Q{9cx9tpr;elA#}q<&TmrJTM~%lC3q1%7_J<MG$^|E(%&sEt820&<zc8<pS$C
zS|Q*QlOin`*T7S!$>L@s`&1i5cFlfL>r`~LT38zEJ+WNtVc#dW%;s7T*Qz$^*B6%P
z!AJz4{`IZ2gN`q<!K@Sc@{I-7BAf*M=K?E;AKyYubu;qRg;we%FuJNS0IKPr1j{D)
zy?&-fw}hKR_zG%C6J3KgatlOuEr8m)P516w9uap#8LEb<{7gYXtvs;M$~vJI-x%wk
zCAy$NA4no~SAdzR7<?&?6D`(*1)cJ%g;qu>Mh<-^HV|~}uGhFx>@;Z4*=fdk|BTah
zY8H@Y@#Bz`>Wn!u*iOrmH7V$rEO~E=g|~iYor$OA@#_k#R1uz&JjhflxU~>=lzUqZ
zgoDhM4FND$*@g&mQiYNmRI!1n#;0P+W|^X+J)p}&(V8R}*_<MeS_GPA<hK@C>2x%P
z6?EK7$H-I&BbP_~H=`LVMQf~-+yB1|{6ZWCQ)7{Mg<NMqatG+q{{gxhw^=mdW9ve`
zw+LEqKz_N%%32yyvq>@RJkdk8s9HC3C!Hx|L8`SV?Mz<^DMJ|PaVE%_b+S4YnypFh
zO@$V3ke#U*Fg^13sn$~5ZvRKBbwtKl7`IrL;*&I}yO{Q8)E_82yIAI?VSX5pWoZ!B
z0l7QP3T92K{6RXQ%$ng_K^h&-AIcS1%a_uu)uC$IT-YXxu14dop41l|6nQFFasb1)
zS{@s)Rz{#C%VDU~7jwy8XUylcDc=e@ySRGNHRp3^nOk92QtzLW?{{yfW7iJAXRsN-
zM<QlZ08om;H=tJFL@Ovb2QtDR#i#`le-2pwBb##Q$S40Hy`+Q=chSfCz#j}Yr;(f_
zlNVbnf)JBD<iJ<P`Ev*HIVR+$#a3|Hc@P=3(8XJ&4(9WNdJCge-hQOD&~<)LKC;+a
z6M-o3435_=M3UbTCbahaGEAY$);PaXBqUEpA?iFT0M2T>^ZT`Y#vQrhrcy#K`U37!
z965>U8R`tQhEnNRM&RU1*h7_+%r>TzJT5DiU}y}>+m=}AtD#4EXjCAZGo773?#zqS
zM4;SUjqhed^s421OCWRA(vuDhILMT4h4~Y+>DE#78_~Z{rdw-p$?lDGE1fo5AyGn*
zqMjX17lfrR2q_NB)j?}59hD_VTz0P0Eo!c)$dNY$F-PP`8MHDoa_C4HzWx&CCW&xC
z?AZXLqCoy3Xr%@UG|#oorEONEeiekG#?mqavfn7rRX<Vn^Hc^b`yM%#VJ%7;L`QI&
ztNZpX1;Re+gA3@}luUjsQ67!dqmYUt?7~l}SL<o^x+wMT;eX)absh4CrPgUMtDjzK
z9lx?MD-{gSH2FVkvsUmY#h>VlznO5|w4ArhqE`dgEVGv3=WO+J%`!BlQ{K7E+Jc|A
z@#DH)$o!DCjD9Z-S)q&TVGJj2kGq4Mlb5u8_71Yht350=YBrmPMKt5BgZKj68Z_ei
z7WsO}TAGcqRB~8>iuS`0=+>h|O;j#lZqXf%^Oj@PV9Q&VTdN{B_^88i@Z<1l4*74$
zQW^=TxN)YUo|XXQ@VcS6SSw1(Q?A4%4f2ngD7o41NDj&CjX<w#IrnzVrj_;R9x{&5
zdz6|3DWUZ@K4}tLVceKc(>tzH7LaXCn&-yi*rl2qOVpWVY8DXG;$fE+7yw1`HuclC
z0+UP~m1Jsaqof@eNo`XMNo{lNjBRZjT)uckQDtd^SMvL?z|<e`-`b&P<2rZk_Z|^-
zyiQ~*PPc$Fz^;Pt2^yfFn%Hm&@!{4HviBM7t+v1%;=TsgKdCjhPLXNDDfMtT1Mh5o
zJDRk$cs=$kua6#Pd4QZ9m~k>8a!vBdOe-yd%-7>EkILr={f>?@^$ZEB+A(w_c22Z|
zWe3{Ai*h}Hn3x?fVYM~31JA->znJ=X#}H1RtM1-09HVx}nDfe#u>*RkIzl|4Lx($T
zI{9#=Nb;%P<&EYYHbKxc3YszW<0!OWxszPB^jk>C*)P4*hZEsZec+XW=2<{kH^Fpu
z^0CJPJdPMUi$X*_k-W1I24tL>xsyGvtjc#*;^TOyWNF0?$jQid@y;5&owOdQ4irme
zv~@6+#%4{tolzDYPD*P;n*LbqY!~h-mRnX@!AN#odv;}q=J<@H8@hBO04rM@gzu^g
za}TgQ@2b)|+V5^MwN+0U=t)D$TZu+SALegbGF-)-LswXL)#3BBs{Op>A3+3)F#~qB
z@TCwZ?BwGH?`n;2KA^)=8%GioKR4X-zQ_zH_9G%mN6g8Lk`6}QjbrLs6$OX#U{0U_
zp3LQknHwQAn4d}T8zD51s##Xu<&mwcFmK`a*Q=~-I5PgZ3MLD7OJFTfiMz6y+W;VC
zcjXapZ~()Rvf1b)JX6EMN8&~lJy=vrzQMvCVmNd5!6HaM4OBllygAe3r9`Xi=x5^w
z;FC4tT4jIJHi>BZn)MGdkOQC1-bNn|hmvYk8D^)`CtGMNY!gZ4vs#kknDX+t=?JKP
zCghi^t<2*>TG8U*5L*(8yq}fQwMI~n_8?5?^w9w-S}rdP!-fpYTf){6!K!qObNDYY
zS@R!dR1^2X9{GIOIv11byd$h57fm=%KvOnlr94$D&ppBloG>1a9hm7JcRp%p7zM~j
zB+SNr^+g}Bt0R?9H_3;Nu$E;)6=mtO8zSl+cRS}es21+Mr(5J-kFe58p7B>u&lHzo
zz)=?i=Di;sR!pPrekLFch?U4J1JhL2Dmo4_`$w7TWZ@dv5q0wBHJH=tWcwQHNc;@0
zv2t)L<v(kz<Jb0-#`dFW%skU=&}%U){Lge_A7&OI3(D)cvhqlv%$57p&w%>*{79H4
z)iRuIWfY;qwt}=Fb2JrQHjp-*rU?T{GsV=Ko9S@Rb?R8lOhcU-?XxwJd1!ii<vrO}
zp4Elrr&?sl-)CEk&o}^HLbP&<cuvG#s24Xm$i}7MsF;I~BgJi|d)RK=jAQ0ayy1we
z*}!d;$)k^g2V($|=c?tcrdmORgWLEYSb1czwws-fg#^mK1jj=-@ZDezC74$*u@oDo
z8T?YwmjG+(QUi@coX4kEMZ`uE2cTR#1S2Q(rZGWQG0r^a%NLKbk}t}g^5PUHi6#=w
zanwQsvD+M^yX1_)tm^a9brhYYRv6g%*@DluO~>9TG8+zf^~o`fM;>`J3@t3`ijRgh
zmnZK%+N#41xx{0v)#r|=RRM?-ScN+a1k;7a_;KD4pJZ~5-|4t56sMm=a@F7nJrlz<
zi3VAJ4E(`k_%5}zY~vvH??t#nG|CsU70GUny<Ej!&SlY76}wf%Zk@%LP#`})##$2T
zQ3KzPL#m>x24vC{XC^eZo37fB;!FY_A!`ajUXfsi4L+5j+TTFiKU=W8ogf~mo;6L5
zEmg6~2r^yeYI#N&2Gp5A6^tY`@d=*XRqLJ8Gsxg~B2OV7aPARm&Z44;NHOWdA*_?W
z9A%5j#R6x1mEx?&hTS;Gz?1%$gNAE|9%^6BRGOEoIfR1DUOI4Kd^IBjYpsk(3E7SG
zt^mbl3uDNf({8vw^B7_PhXZx^f61^YF2f?&BVWhx=@+vZ5I#Hu&gT+#5Pafc1u+UZ
zJ~j%zf>GLM;$X}W9Z96e#A#w7QuF2itc6KdEUjbtXZ5jGE}-&b;g2qr&Bt0R*NpfR
zFTP*w4eKX{jIX+ioy!1U4P`^9_0fQ@@GRo7)}m~*-p>?_(U;Un`<l3{=X~Rt!iZKQ
zuU3ZE!TnVvk6(v1f0evwowf3qA~m|l=6BQR#vvkd4t;retP)0wFPD$6v({~#8ISuw
zF=|Zo4ZEBVUQ=d>b7D{!&E}j-?ib~F`=S|#sS<vaw0fDEJ}Es{X69m2nviGaTFcLG
zB-?p2^5F;ykpZNhy4lS@fZjlqHh8sW@M3Vzr#n3JYj_%<40&{J37n?ty{+=OTzFXO
z<r}%+alM>(oRxWgyBb3tR+)U#gc+7PTHwr{Cc>mjdAb8Y-adFVV_P_=C)K8lEnG0o
zcWCEv)(f7MqXtgD+9Y|xLby?MCOP5$qzN59wCz-D^&ImrhxQg)$GZzUpl*e)z(}x-
z>}n4CIow2HkB`G67$^t}e+q+Bvq?vVWw6NlfF9|l+#)+iltf+dO%1y!>+z6_0T-RD
zRbV+<F7k^-9UNQju09o6L1l;V+$+9{+pD5f4FR0mhq=;f5-Qpo2#Ihuw#*1TF&8|+
zehP=H=*)%i`EgWAgiAyvPOcyasP>4IfCy!Au_94YD#|LpmP8<V?>)lrr_4cQF8W4f
zh-0YV;<DxAF@G!rDJeqE`dAw0K6#8{Jo!-4l!Kh=0D~Nk5<gS2Iozu`mtV{vx>|tG
z@Ue>QDvCK}nhN^3BTg-b{||4&Q2eQ76t5U?DtWX~eSi|+mqw;gn8F3MR1-cG?c+{8
zEsu#{TpKD5OWL6N+e4)`bRO2<<UnOc216nXjV^+7kh3}QS>Xi)pN=j;z8Vhq$}i8h
zQco{5m@u$5SLEf3{6b2cjRe1$SsP2J**@;gjjdFBh{I<L2tzh~3od{GdFQvRbyuGW
zd_v?h)6eu#I3|{7`ZZWy4^248f>cz=jYRd#__Jy_ONz)YAUb%XSfnro=S>r#J{Iuv
z`{nXtD@2bUXaS<!7l5yZ9s(C&+=JgK8t>P@i*l$MA9vqHXbn0P4P~0k@;Hfkv?XvD
zaa$%S3BIBRT$d5kxWwF7Y|UBLNU@%<D6b?UXkU<EP3a{8`ChSATv7~^L6lTbj!T)9
z#Vk#ia(lD0IlhGvaR*;I0(C8N%0z7&kFAOVZoFShP{pvk{Tz7C_-C=~JI6}Su&H#}
z0UDz%LbQqY1kr)Z7KFLNmj5`%T6$bPBk>6zi{TYrRKm}aQaK}x#stxv&^rfrH<}Vf
zOJZvx6|J0+r=5$L6zm*#<4VP-z%|{01WLLJap>%CM3-|jxQZ$;(=RHATw=u4D@4DT
zNe~TlqKx?)X}t+};XgjZ08NjuIvU(U{`*{O9z<;NTq`{cTESqhU|W#GG#ZJdZ8;43
zIXo<n*#w6Mo#RJevs_pAX{k_y?>avvRaS|b8c|y(>Z8o->!x+4>|DZgeLcKwaH)wX
zY)g({PFLWxdLQ1Pr<fbL$<S^}h;M8p#;Eae;~C&GMr9u5s~u<AfSL}95XXEM3=*XO
zvbZjK+R1<}q|CRCQ8+t?!{aik1dcV7XWf&>qa)hGg*gX(|F;-l2B+K<_K7+U?_i-K
z(!is9N+p*Ku&|hslLg|y{Mw096fb-{+;)P?P=_ROq~y6Z5u>4qs#KFs17sRu6mZ0H
zSN35jc2!dbr5JZ{t<e6Me+r7Ve!8QO^Me8&)pvkZ4)Qy?zmEKlF%bcQIVIGHJ6mV7
z-r2@sZp57<)S&!w?9WlQ-RL{N2&nn#ZUATzv!w~66jgyOVZ6|QHOl~ZEnVlQcq&U_
z1pB#%_cT#Lb_xsK9#uF+;L$H+Y(_>fA<Q3U>%lC^k!|wL^Wd=oZ7>xfIha2OUHEKp
z)UA>!eo{kw2WUbFF!H^ctCXu{nsSBuO)=w+3W|9EZ2(bOSeeamJr9g>L_J>M?4nt{
z(~VKz$>UQN2*dJYHlTw0%4FnxYZaF4KR(}D7U`u@;XKi5wsXVf5HU)L4}IYr-4@1T
zkxK4|Rs{$d=FY<$QNhjL50(g#!ZNiV%@nvRa)6Q12h6!dm<QcMECxzu9zsS#WL*~~
z%nyS=<V(RODTAuZgA{(Gl=E@6cJwvWdKSgEgH9*>=p+?acpshNFv%iid(2OuV(4Q<
z^5AA`MFbdq0=Fib-M~kA%Bb65O|)s={y2|DYC$u%;m0+UTH9=bMdKXvlUh|B*&sPn
zrs!~Ya#tP}&kgRVA|fGNN-1^pV7pAn^GmH@8CqN+I?YZ~Ot`^6knuQLfv3e;3qOG<
z=CiV(p6C}wJ!DgH^+}X<+V`_u3WxGU0ZVh&wES(UwHkW}|0uNrv;y>rb}`2I58?;i
z6o&F`rx`)%?x(_ppfRC?CFQAnVT_u1)M1Fhr$I)Rh`QU{;(&XS8~HSbuLn!c3743F
z(Bfj;5XV0QCPE{z#&~X&!b6ECHhDygXT(FT!{<zl9yeyA4k3Cly-?gQ(PA`&`&oy1
z_&;AJe{}(NmdfM@7g)#SGUg!><ud)C7k$lTe#0qB|NquU+^bU@`fZAAzR(KFstdsj
z4(HODv|Lfn0{areekTlhA%rA-s+V47k@2wjybOLIo>a1lizkT}StmvIc!+|c64CCT
zULc|in-)@PTx_pj^u{O2Vk&?OJh*!>Kx5TxH7Eu(rqr^JkK@n)p$tj0l}W&U03|dJ
zVWlIg@!&KAu%A&n7NIU|;JOBD8<W6nkv(F@V(x3>0)x~`=KVHT3`Jf}Aj|{QnNOqJ
zY^f5s`9a0pqTAErb>dLjVbKxF3p6F74DOH4#3t%q^8qjg8m6C#R6tp|z|_cz$c_a-
z54U}vpF5p88t3_my_J=s0}l;Q1fAbUg{t9uMyOaQ2MNZ@<4iczwER_>wRDREmg3rt
zUYxu<O<7?0_Q5ElGOsjJ1RI@XqlZFzaK+;D^C{a98jiIHYo0&$%j34d)z>e#Zn4rY
z1$m61UWwH(23vt>cTck<4wn=0&;iq~z64l8YRHUaF%=b}ZVW>k7k;UVH)}N_P*<kx
zxWDDe*SA>L!LhXQVyg;2PvQrk-ot<x@Qu$Glk@HQRP=8g)jF?to|u_8uA}5Bmsn|d
zlI4O+uvyz8Z@I);wt9@@yO7za-VLBv0|>)AWn4LH(06+9m1ZljvH*MNqPmun->oMl
zQBcL<X&wwd3XhlK1I|`jaTE?Z(w3O6bJ^H<3j&4hyk2vP(SOiE=xJ8S75P|l6jR|3
zo4HO}Gtg|2!;JoWo2)2@9X5>Zci2!^dp6>w9LoMbh|0!N9+p(^Veu2hk3P)FaMq1+
zb^qk*a0D+Nv&|z+^)&Q&MI|AnNJFo>Z0>;kdKm0@=?Tx}3^6jhPg!)7{iA$2?@}wv
zpU;$IK44Y$QK8&`Shzs0z7%`2#qv(|^Ar5gN*V&mll4a!cR6Cp)8G|N-X=@5SX~xl
zMLdZRhnXq9EYav~v_!Mjsd8{^AWQUlqu+>yFndS5V$5sPc*F^I>f}bN)e`Mi+et}I
z9NrpZb@dV2gSAdrv{|hxv7f3>$r3}>lqIIECLJpymtj-hmOr@6I&Q8_e9|P^a`ZCm
zXrv@<#m|&HX{%L`nL-`bKfhstXj;(0>&K4~MeWk9|2RcHwABi1L@bXNn6bltn@Rc@
zq9#N%xhYvRB@ZTxvE-R#BHM{?r1%5LqBFUd;?4ba^7F0Ml2ht5Dq4>5pb-bmZt67i
z2ZdBO#$C8b6p^&yYxl9t40g%$D&U=dI6(LAaqE7<#G4FCKax4YB=Q2>d_;{6pGdLY
zqDdT}iPC(ug(~PYd$>A+FiZECO1J&6nuZH|SIi$OB55u%e+aoGBb}DcwKU6^NB(d|
z&?HWp*7%}<WQ7{T<8TI?3c>1|V)38JmC2V|-%Za|>ZO|})EShh6jeN7{bx`<cDZ%r
zhAu@sVfS>$8iK~n`e;l~KgmEdzGcVRh6$oaVu*_W%t-$g)=BeGyAUJzA4PtdEw@~O
zeT{6n>k2C;Jxqia(g-qLsILH@UzlXJxCiCX6;|5nMxGK$D=U96dxIH5O18-5_I}wW
z=Ur(9E&vN)evP|8LcbfRhTV9lW1NZklGAda2L4}ZZF*JpRW{Ky;2L9YeOV-Lz0wM9
zDS{tTkQb60(L(J!;2-jf5&sa+$eszFK8WA&43Y@9o0=)yZ#-m7Pdyd%Ws5dA5-F>1
zRN7Z!OKViFue3@c?bLuKS1YH@^-$(Evf8?|G|pvS=CFk@x0Esu&lye>!-*rbwl!7M
z=wVT1l&gKt5Ut3~xeRq^XhAPYSEvLNq7wE9#jwC;Wl?tcW{e_Io-CV-&_pDbP)ceZ
z37~3JYB6gj?xV+|6i)^YbK<dq!$7)8ZHOZD&44oS@EKDgA!XPUqfoi>o~tbDh$7|@
z4522jHqgy5DM&Lg%Hx6CADEDRS6OS;GF5Eyn{+T2P9W0%_#mmO+6hy$IGiohzs<)i
zPQ;IL<=DyN3&i9CWm_#~k?vg(7Zqw22Hm*jMT@z-LNm#ihl3r2bNM)t=*eUJgYZqz
zWISn#riJ}19E%Z`G7C*I7vPsE6%n#qSo{}HQ*1WVzC=)%O-+1ilW-T;5bh;vNajGV
zO$EeEpd%wLHi*~>(wF8EuC6z&nI;P}E2R7x2~idS0^yw`4+ml*oMQ@etXuSmu0+wD
zIKZN`64k;1av-rsfsh0!B4`xu{*{muV3>s8gex4Qr~Acpu#pK$2fMH;97`Vxijm-0
zkg}xr2?K+#3m=6P9l<UYHznp7gIHxQnM|iV!9Mx+)z-<l6_ZhgpHaD`$_o0?<>*v$
z;c}iyxvR=r76i9E<RIuzZ^#f$8Eq@XffW-gILD0aLk>EgDKMLN@>p5}f*nP=W`O%o
zPp0OY5eC^rJd(wl7vlKbAa?=_O*R@xVomt@Q-JuAW0VmeT{N3QVPs;iuUT8k02j#s
z&vQiIipEUQoH?E)CbQbOF%+*ohh|2W+_R0(uKg50b8{4pRAEMm{KGaYIJbl{vjJdc
zl*-Sy!9pmLORj;vP%hVBV=byKr!ttoM%GNP;er)9Vko=gD2iC8TFy;c3%-I_KgH*I
zgi$N%n|ZuZg45QFR&wtYbDw9RFzjj`hZ(l)y9Sp!3gl7OS~r^q<bi8()#89$Tx}KO
z=iAlRx`iOuFRI#2%yXTr(oYJ@&T4C^e~6ZO;VM0G9Uql%R$Ir>t~)09pqTcyGkwR;
zMl_QhVIJQhkG&3C;T`hI>u|#W-(9=TT0R$|8&bBVd1AF3yv|y+w3;xDCQc@bQO~$X
zOk#N<I=xej<dh=m|BjV*3EBeFB@gm3&W%2WBl_^LAysu;KJ$hIp1}eQ;#76FnorF2
z?Sz3nn&$c*PG`<6EtWgJW0f9Vj63q8>GYn{Mc?V|XA+iz4w_=Cl~Oy+N5#Wz7@AAF
zrMTWIoT~(phsM%TdBOFtiAH4ue(<gamhw&5q#{IxmGXt_p}Mkh#~fo&P3z`H2gE7b
zSX@7AR~GFJ=nwdH*-AGz)NvnV6S@t96wc!CvM?!!_D#c^C}mkS^?vK7o=u{E(}ap@
zWL+N;Gv`LyHi-^O>)kY{(jYh?5#4lvp);E`)3Tu@S*x90PBCiEp!dXb#DFzoiN-)1
zbNL+50i(W96jxK_=NiPJzC|;O7(y4oVS{}h?3+}<InrZhQv4?53X8mAQCco4J7}VW
z9_leixEnWx<ySRW0-&*Bt*J}UZP3Jc4P8w8dy++8@_4eCOrA^@Q_55{&x7KFT>)G@
zCht|5=U34BrLbDmHZa=rA#?en2<}@oFK*7Eb`_Gj4{7h9`MO)%Z=1mq<0LUjCR3YB
zP5GNq+z;e~jEpX}qPkYv_JYTv58ntAP57!AHSU6-{QV79`mxX?*sS82W($}Ew+)B4
zPU5W{_}$<T5;N9dk{I_z=Tf<fCK<dDX9q{UotEfN8e_@7yom*td*TXovtLXlc6da;
zr_n2#z1?#~@7(^mVl)t4Oen6drQGH4pbx?MLNQlX(<+>>iHW2cMoCeAwdOIeL`$<o
zOHyl+Xiqwjgbl{7^He5KnIf8#T9QOtQaeXgdZ<HUc?o+ZBs?G*k{XjZD@Or4Ce>U>
z$sny=%x!)0^&72~Y2%4vGI7i!COkHbP0tLg*lm;2yWLuP5}1o62%GVhjm%-3vLbcH
zO68hhTsWV??JhMzT+<;h-)^PmLOxJcbsiB-YfR=efewoAgorGf3Gvxt+wexaUXpFw
zt&Iis(X>Q3TV`_c-~BlS^xv966F)!5_t1X}I~U;Ru^Iv%s}uC!dj4;;h{6Mhme*Rl
z%|rinleK2f;yYX$8pP>46E^Ik4P(Pdq5ohb7B|-Fu^2Cv;d*O@D|<v1*IS2<szGAu
zhJ`iBu3IUefp^#__fh@$Dc+!B!z%7*zs&hQ?C+>7|GsrJeded>`&RmKL8R84zOVv$
zH#98RuyY|`UFX^+g!4{k>^libi2j352;p^z?}LbATld|DOLXh{Wbig?&C=XSq}Dm8
z5;yGh0M1iIj;oZFx524(!U4eJ_~%pcd5!p`>NMebZ3LgD2q)4|U0xS{>I<2SUB-q6
zbHh&BYmb#VF&p6=T_!{c*y%0DPrah$PUG}D%w4Y28{9kRV+`R>BgWk>PsCcbA|Fwe
z8ySCtv0*12LNNmLAG&dcpVEzT@9owi*GW^dB4TB(r8cxHw%lQy*5EpQr+XI_(rv<y
zCSFs*zbyO%j{Gj!9kEtNYAGtNuXOfPS`klu<iLFh7vrC%@>GhiQ{~Y7oi6Ai{?jC$
z+5o6pr8gL-De^V0Q*ESC^8lw3E$g!>GyZAh>sw__6y`*&Y>rx~YfhssD^{%F9@ltI
zE5*SB3Tw&OP$7q+R@&lDL}kY{Nt3i;WZ&&LXr3d3_*vYGOoeecHI70<mfQ}BE|OKZ
z<G4kxY@#x)vV)*9If$RNl_(n0J*#mSQFE(Q&z|0pw38`x(k5kS2auG>{2gdik1PkY
zxEKZM6_MN~-So3@g_d$v$}TDc69UlM5o8`O<l%r&tik6}BWjckG7{=A@N-kP4DQ6;
zj1pp8k<$sMt4`1jEh#}-Yg}0;egg3&aq-*~+Ji<s;dkH)gtPHakCF455DsXrC|v2r
zc?T#{Tq`w8&Kr_rJFWC&0tJ=BKhTXkCB6o<x<-Z@!0vuo++Z!cDi>*aaRg~jZLUUq
zvBG`W0MrfH+=KL-xN@4%&D20=VCstTeD4q{zN!&P<#FWdBBkSq|5}e<Ky+8Byc>49
z0pL%gyMRa=Rbw$Wc8Par<Ap`CWS4d1#tuLQaav4MUB=XXlY}jV`fDU?$&-D%tn@`w
zNOCw|fG}#o)n|K`b$TQpP`<)O3C>8_T1WA!c8=E#-P#T4Yqe}0MqIh#V{9br{1rJ!
zE6|-zVN_jFV;D&#8(3K7Q+HTLoYjZ;pw6TVUOS37ry|{<*UliWNfCQkhlUUj#MO8-
z{7#5Phb+1iGe)PZz0*2jk&QG5xkhw@Pxjr3G1M-{?zGYtO(MtFs=Co9eT~S`D6<=Z
zu~C*Zsxgnqjs0=?b+6Qt02If`l8(QL+8oc{EWjWYU(Ok9#O}+moNB~bwU7+o1!zc?
z++}4hA48Tl#h=r5C2eT%oxYRb!3`lEWpAmMEy#vu_RvqZ9Hp{jQtamKob26*Y?dW}
z7SoW6Djh~;o!;H)lg+!Ww6od~HE`I7G#aW9R}|L}EwK$<h_6??k4da<<gRRK1KklM
zv1NyM;~j`H$`7yNj&hlEH@G=OKdG)e3uOhsbNZmG{r`IfjYGha_f09i%Y*ppxc21(
z28Zw<djH*cdm$=C6L?ZDvzx3V&&oww$>F5j*MK-Z$f&3@KJTM0wjGhydND*9@Xfbw
z+1_Mjo=A0JswNxe4r9YElP6TfX{NOFAY2f~NZs=-({ctxWi%tXR?P=R>O;;@I*3CJ
z`*HvA9&6oEltmjRXYABZS@=|z2dU-sFQj3=Y`+IYjRVwrWf$i7n9ZSyYU@LKR@|@-
z0Y;5&<8lVY@vW55y^#7+SwKG(vf^GucFKl(!OTI~Pf)F#xEDMvmHs{G!zLNtgUCu*
zyvJI09F5O-Vb`+rL>{^p1{oP^r!~w!eWxG!^iX`VMz-&<mb!2pzi$sT`+yv!pJF);
zgi!E-W>i)u3z`9y%L@7#kWu={mF>+|=H=9@?Hd-hM+o_@4I3Ix->|cA7Jtw1|3Cfb
zwMg$$gRSt6IlB@YJUhLJG6*j?^IVq9+H0MFhbYSTTASec@7#;=+AqiUTA7PRD4#Z0
zwdnoUCj<9c>yIE-Xz7TzNM5c(JV}B=PtW#6@3T(GXhW3dTx@iKYed1zBXamYc$XTa
z|9(`Ri%+$J<Jq$6elVd+Hr@}ia^=YV;8SQ{3*v$@)B>PV7PTOMSk||obdKzy$gmu1
zLFrs69>7n53_oCf3qOcVmwO(tJYan518DAy9D2Y?U9z7fvLJ5YDIvx}Vc$M$)q)>T
zqe%*b_-D|PLFVnVmYr3jqm{5?@c>xhp99xxI0e6G;TP0jkdN4Mz=^My?fW2Py|QN?
zXv&f{K^4*?t<1$VIEv%S5Tzgi{^>%AW*5sk915n#i40>fZWL-tQ-&atovPb6?wa4=
z-<eG1ozZJP7Dc!y4!t4?s~*G`#>H!VsnnznM`wFB?6fvCK#Y=N9i5gXb6e4fMp@m8
zL#yM+K2<2s+k;~W=Gi1i@uN~#cJ|EI;&o-;3L(BxHA-3XyG%AP0*D6>zSV)BLPvY)
zl|fjiP2v%fF-7Qx#)J)bB+8uq7;uBKc)ztSxydjVfRX|XrVV$lx?}YQd@a>V6T8AY
z*WjheMY3-{nA$KwxM0x`a}C?o+NxpJQGls_N~b3jDZNL-xRuWEfQCI9ruI{Q9(c&_
z;I~xz{!*O|cB1@d*ZxW!p#;jcKdND#CT-VK(;DWlYT2%*38V_<(=TAI7_eW#&l-pN
ze_;LHG(SAl)rMz2Xa+X%KPuw7$To05#t>Opy&=j~X}fN%kW&v^o7Q%txIi3a$MqO)
z@l!h;xD#+!9DE3HudIE<I_9(kanb)qbjK|0USPW#P#bO+aP96Dwy~6C6{iE3prdTp
zEp2lA5e)iDncHq{O{Pjc$c?s(xwfmbPabHuHYFD!nr<RdbPb}<lHyTl@lILzsI_T1
zNmi~(O@M{8rV;SCsv_ZN+x2jc?0(eBj1=pr&9<wj1dSvT(Ab14u(GF4Y0&960v-X3
zeIUw`5Vg&xNH~0e@&AN9w)ylpl0q8Zk96>!{8gOZVVhljfD1Kz$Tpur`A7`GZxBIj
zf$xJ0{1Gq^7#<D34Y(gLbrSxc@G;<txb&}VSAHL0q0;9h!r4#*xD+r5o`bK?DCdKQ
z)A1c&Kk7GYbOie0m?-jQYdD*}JPZU*`a1eVaU6USU<@WBPv<|I{11Ss#Z+OLZ5-Q*
zU|7Rf*~YrQIQ-YyM(!le@fmeG16X`qB^p<t;hnajyb>C2wp}~MklzpN5ZgIF0z3@Z
zr{Q0~BSF2KqVxav*WlLxx5cHu1vt`$h<aUM9PWv1z&RTJr{k_raH0vAWS~;R3*m;y
zkAt%uw}ehV4)7GxyEJ?jUQIp#xLw2N<1o7NN$7|Q@<|{=zlOKtNO>(_>T@(Q@piyq
zsi9e#xDP|C0I(jBi9bYoA7CxiiQPzVk4t|QN6~$NHOCW2?TC9@eO!h~WM}|ftt<Q-
zunjn0!Jc`Jqe8<OfQg_{4IgRa7(3t=4d;P?Qox!(&uR1m31Cg2=iCS~l*eVb1aMg#
zd?gA1f$7r-YmiP-O?{4G>WKob08BDL@E)9qC)lZ=l}^Y5&er801x#I7rs2rbI18VH
zh$bEJYbcT-z%C$!V)6VQ>Aippb^4f%Bk}kv((p%sivUk5*z2=hFYX3RiU|aHGXUQ;
z3wu7Xjg7g0wG2hP7-k!Zg?$>qMtWNW_4_n@u6iDXSqOr>TWnX`81mz)6cOmHw2dN+
z8eDo~8S&l>n7Xh^!w&(T1YE7*=WOFF;&27S5ySi6s9+Eg)JKHyHDu@ktozXWE<Gs%
zSdS9#mw*QVw<(0yGTZnT2rxYwUSs3g55T%Y>kN9b1Np~w`Zn88P8)(T{;ltkw+0!s
z5LqpF!-IrGv)Fo!-t++6su4U7cmi;VhX3k#W;Fah;0mN?Yj_GWLglp(TK_?M6W{>S
z@kJ>k30Dqd5d^1w4PN;*c<tBVQve^BtuUzwFv*Z^K+*+p>ja1ZMNrb!<mLff>LUG%
z3ab%OqaqlFCOp3%aE&g|fR{N)X!10CKfTlexLCtofT_!JH2fRD#32WM!!}BZ00-X$
z;6}jZF7TfSm`21nBD4_sz!Bx#Q3!oOz*M2`BHuCO#DS11Au@b<w(Hqqq-z%Y3IU5a
z_*;Mh#?sHXT@U2|&L{pu=zLp|Aq5dBfH7=**CPWJ(9^yzf^?sTDIIgcLmt31?y3AE
zNDs%Q{|xEWVyFDCY@@6LaDi^np8-z*j%XJ7-m?wmU(y*cPPgQNIlT&=n?m<FP=JJn
z>I{*C2@TTa=N@4jTl0~guhZATMx*?CRLzBqD36m)KO2)*8_oYG02^krxnRkTA{01Z
z!{sQ@4On-{+-m_311{GHzHhs3ZvnhO=id#t6R}jcaBd6I8v&D^ApFpL6_Bx{5?%f&
z+ZDz9V;GBchUZbh6NhjR8EWI;H*8mA8nBc9eWd&2(!sH7`v3=31@l~hM*-^w%<}`D
z0o<h1mjYiY&HwWh0rQ|ZszI^o6gb&-eXkSg$vXW^dRqnQdKk@v$#!cA()Ij5Z>w#5
z2l-uvx`ONRD2a`9hoCw<A3`#u72CWAP@cVmP6c4m4eiLFht)h76gNx)b_jZj9vlH&
zuPYb<$AW;J20(^xqy`u|eF_|EMf!Zzfcd6vY)87I>gUe~2clWC2<Bs$)fNHP70f>t
z83=*y!}%MLPI^HTG#_)ba?7a-=WmAFEeGlO8or#|a3Dy}1@o`5jhi7urf%T;Tk*(6
zT?hrbbb%&(?Ry9r{2Km&ZG3MG>1`VBB;Op;vo-uI2y8;SL*TFJO%uQkI(<aFOX7_C
z`R^cqCDKzhNB$A9jrwf#S+_3m1&n{{)43{xKap;~pu$|0?$5N1XbND>A^!=u<I;@$
zGdlfDln(&bHliOK*+B$Gb^4`{sZzk2V1ER&5${xJq3~}9+>Q(bx&X$Aau{lOKj1E;
zI|Td~Z)2bVF&L_N7jd9W=LbiWhfu?VfXk6ytKf(qn&=L&+==i*H{V$phY%*%oqe-i
zlAMSZP`c(&@<MtQ1F)VOlCgY>l8|YkOg;(DLJ#0-Rblc5q)z}Q&FIGXPu@h{M1*}h
z!)3@YiVT`X$xu8^QFM`Jaq=xlr#{qDoqQ)?YLP-{BxCyC0~vAkXnc>M&#6Kue;42p
zz}x_g|KwlV#(lYn(3&avWxx%93p9dp!2N)24Sxy*)W>?*F2Jz5zZ-C)N?+iI&=G-p
zGFq?#2F`$nF;^_W<kgZ7n7WYWp9LqPKt)`EA{3wsweDU3J6-uYbp=<!1e=UY$8_B?
z4Gy;I^m>%<i%Y*7a8De(A8@aNBQ$$0_z4Ig7CIC^1I+}`>B9vtK&q)ir^T<>Mk`6Z
z7NP|(O7{!MNRuYuACQspxbl#pA5i`}mA=r6CrmtmJ$n9MxEK+BL=5W;N5Eu?0@egA
z#7gEtLO80^FNQ<44{(--Zy*ONVBO~nArlQ$0hPz(v#=RoqHY6$PWmHsuLm;Xi)aK-
z+Qy?5$WW}|7j5H*ohTd9FeabxH3Fel91Edpe>8~v9XcH{@jaAx#(fH8;wL23V=8^c
z1R6vf&<#qN2bl0hNNQnFq(JpP4vL&%l#&Oy3Gj$Qkb?R8mI}anx=p#zHaaT-x9aq(
zk&X(TE}EN-^cleUI(<9r0UIz8fG$tD%{Its95M5BhDLh(1sSv=O4)~3SOnmHo&Fr)
z0l-?drhr3T#enN{`X2%J1An)M#~?#lNY@-onFLH->@)x--!pYI|BtHzi=arJB_Y&`
zV$o56sX_-n!!{1(g5`$JUuGLUfSuLsB4|$Kiq`4(&|}I-*K^CFeV8pf0DF~8EP`48
zTo&q22<QS&gM}Vs(7Jok^S1GP{_Nba=nqJ5M7nOlqQBb43t*{B3+*D30peglRgel9
z=m$s46oo$()9o#zNY|P%H66n+(v1k+C#hMs@tYCAxw=4(ZT!}UWUY8o&xBM1b_|YG
zaNs5CV^Sojb0IX}%WZ&lpQm0$u5BcBYJA|(D-94z&5=~dK;#d^!fBmhk8S)h8?f%<
z)CU1~fdB`_Wb-=ZpVaw(4Xc+prlmUdWoX8Fz?y@puiM5*2=I)~kGbWIAP6eb@E3N(
z6*+(m3l+=KVE4aS2@3mFhBWAf4|0&M2~1mw`zT=;ksh7C4vLFdt_esx9Rz^oP8X$>
zpwFuSYnG>B`fVT%YXZ~01H1nA(%Aw_&__>1bp{NRv|E9&2e7vB(lAOs1ObjeAPuTC
zQh;==8Pk4b8-E9fV(Da{5J#PKOvjVR@76+`7Wp$W2xOpz2Q*dMJGSe#DZmbe|FDgJ
zqCgDcS4ba2x<(jCqI)KoEH#G$X}0URI%rlB2<Vl-a<rIbLg5dPcM}b8@J9kCLTaf(
zE!BaI$dCtEkK4d`^aed(l4@`yaHVagWnpPosk$Ig4Tr@5(zOr=ZpLl{$v}ZluLs<V
z*|$o;?!Cz08Ot9r0(T$+Lgp-$0uO+ocEEc64?K#6i46|uVHN1I%_I=uNO|BlXmL4U
zEdzl+AiV<lb2I_(+OEd~NDt`-{M&ZbQF(4K#{Xik?Rt!aOlMe(`TD<F<KR5n^MIbH
zQ3w}f7?OM2H2{Qul)p&3ze$zD{k<5|GrPwLhI@SRcH3NlE^;QH#kb)UJ$3mY@X`Fg
zxDgRU$e_pRVk|Ci0qk&KaU0<NI2gWMwHu-d`fvDn?nb)y`7C}D9P9(ES4xZD$7It2
zSaWFcM;JBL3Wog!0{#ON3R3HsNJ}idj}0MWAwi!nS%guN9hVLc-JTCv%gB;rP`*1Z
zJs;&8<I00$w>W(cAzxAk0&0<NlK%w+EU5rOs!-c#ORhs7fwj)4Sn@rzu!#$({E(UN
zjR4lea>;$DpbN0pBTF6zL4|-F0v`uK)TK14s6iN(QQ~MS@t-Jq5fL4TNYd~Sh7A$4
zLc_#T;(%u9k`EztWq>tHmyqsH0i34trzfBRGk`S#>EK}97~sV^eI?)lz!9gwafqN6
zI0ZJ^uAAy1WG%V?hGAVB()GAczZ?z*;y@PC(W3OLkxm@a989l;P?rEE0x3NTo3RRT
zordp4{sz?Vn9b>6*-wj*p+skR2D%*xoDGR|2w7WY9KzRufcj9Y*7W}am@3d+lKwB-
z^?_gKM;8YZ!LjN%I28@(p!q*XRT#`d1hL$K*P=j0T!B*nQ-zx4!LtC@#KCZ2)KQBy
z2ZLAG=CdKdG^{aDf;XZ8Hef9S!CSE-LiOX;g35ql$W9`sz&^kQ5K{b)5wwFNImk~f
zq!wWz@lXgbjdQR#2nWVP)ItY-6^bhWm>L8b3&JA0eIgG22n0nc5TW}h!-W==Y8Vw{
zkS?bd=@w@k0R%mOHNuP&Y}bSBfT;l>AY+5=+7Fn!jNo&?kvybpj$~l=dycxu!FP@A
zk}G<x(;{I+YN|8P&(G!qCN>cfa56kT0Jua~@=vI1LP}gfB}*acoz%*ZhGEq_UNoD3
zDSQZz6VXn7EN&iejmwYaUgW276)e388R9Fz>gA{L2rwV~G#<ersO$LhVEa$w%Z~#N
z#8vn?^2ZUlEEz_~f5la>Y_;v`h(oySR1}CqxU3X#9KvN_V@Eu~+X2VplYm=i30U@o
z?fOYP{(cxC9q|=F*>uELV52}>1tAQjpTt!V$^aZ!K?uITAIDV?IveS6_(RaPKZ(O1
zirkC>@fF+yIKBcbp?@4-!Ov~ikK!wUw)}B?1#r#%IKF}z<d4I@d?DaC{E_95>>tHd
zuzbDkIuKXE@(Yn3SHW@^9Y2bzVEGQ@kFQ`q;J6Bw{|s<E{@*}5&f-7QSU!pj@fCaw
zIIe;f9uz1;AL>!CVi~4XnqagpSON9@SjV9Q&sz~QLh*SENO2e|9<>dV@RFfGFtq9u
ztpLsCJbG5Z$$4-j4#sGEq8YHj(8K6j@eW28;d5ZyHvcOM*vQ~fz5>F0uxB<s6KjRY
z6I~RcD#%PmhQ9#T6=p7X4lgJIGf%M1$GefP5oQ+HW(SpbU})KctaTMZnHSmSS>s5a
z(&fK{^j^T4z|7n2h<N~wbrmTJGw-4=XJTTF>H^X>%P~<c$kOl=AYeGILTpSwZW9b7
z;8^Cb&|>DG#`oX0KJB8*W1qSPxs$^>{SAz!E@G+Xz<k(?zGlG1x&Z9KC)xm~Xb$}k
zx(J}@RQM@YO?8NH2%Mo4{D5;+`K&oG?*KaZvjVp3*OWg29K!#zB1hVWX9y9+ss&lc
zgGC<#cBu-ppr$*=0AHzCnzfm(1ppq@6>LQpO``k9G+cx7G%d{2`GctN`ykk-E4;%t
z52PSncUdIs9-I@(2kg-ZTG6NNvl+5}2w^3Jy3ew}Z&86c%GLxwZJT$tpg|K0Sje(|
zX`4Uoj4O{WxE)<M$D`B#00OI!?%@VTj4X_%r?L^zrzoBa1#}Zh@d<!26J|lG4|V`{
z2*6o_jwoQ=f-HDp`EbKR#j%wp?8TcXUA1^+B1Ru|A(hAfS1y7~)Duh%hOn#*K?bHM
zLL*oMAqoN36|7tbAtQ<%I3EP{%i(_Om<V;H6AiU<@B@H)@T|NNbcF$Hx>w$S^ge=h
zH9N?ppg%AYCdbOX5Va9%GKj=zUil#Mk0IHkVKDmfC}5|)UW{HoX%hkhD)3tvQ3dcG
zI}LvglQZRa;1N0z2A4**>gko3DGrVzSrfbpE(lL6;5^l{s}_J!1Aq%v{#8NS{BZ?n
z6>9LT%EVmQIgAXtg{#)0#R0&6)xuS$*yfpyC{V5u!00+Sfpn)rOf3iBjDu?+I@N%6
z3m4bf1mI5-bPG(DD8ayo2G2boqqmmwoDUX3R95Xo3m|KTPZzi!6;KNtf_{JsNS1X4
zs}7*&KL>cRV(}_y{)4{(>=4*vy9xk0U9<|vUeqBl-Jr<dP~afaO^x7xAe%K*K(T1G
z3!`@spytr(6tFZ3xJV&by;S9Q*XjJQCSGhtS4M#k|6iR8nJ7X2u)@C@9POYk)A%E+
z&j0~4fSn31Kn9X!hk&CUmid`wt6@MLd;`h4g{!XtM}l$rYa#npaq#z%|0&9^3Ah(<
zCtxMS5o0y%l_!QMLTC6PSOh6{6yoY05QG-R2zmh(QkOUcz66<>1n5-oXH-D2?y}Vw
z{p$FbZomh0IST0o#6cM3t4WHx5#bOp1GsAzVK@mauK}#}TX>0WZb6s0`cxN$S3y4!
z!g8gb!dRYmQUf%;Fzn_h_7grO)Zx>?p?<*B09p`+&q0d{bOgZ{*)*IDSE(O`0AFRB
z-vL2pq3YvsHOBE!T>hJ(!9#!>H9_}4U(qlm0`UJZ>46#zL+Zmn#%gl{Fh+$%QXhuY
zc1<5@_?>lO!lB7mtsk2H%WvGa{mQM|uex#D<<(oSzp3_6(~$Muh31J8A6<MMueXej
zTfbd$-Bs6Zz4H3+Y~Omt_1D)L1YLVYB?E^ZeGf-Z5B0xqEu4!Zwj-Zd=ei0fWO&M2
zItNv|c2&vZDJ#?4YPk0Ec$Xzp)-rDgz%bFJz#f2kv)};0q8OMk0<cV`d}^(jgY$*1
z{Z%smQ!6t89{}7xEZcxRp%TGy*@Rec)NuU(MHkn}o=>f13Dbt_={h+|WpR|y^)$|v
ze@0;s!nh$oVJJSVNM-3~IAmKQ>+zG&ZJ1|{%Z|@*(6&_$Qp`KJJN;8Rjr^|pxbpZf
zeE(@>o-D-goYkQGPp`<De<AHZ39^OK{*y0zDecP|U|J$4kTz!{j?b*El>TYtyV;hx
g)5!OODOpaxFAd8E{HA_#4|@MR(C_>~Dh%WQ0<2iV{{R30

delta 315404
zcmce;3sh7`)<0Tx`gLd;x_Ni=Y#u7U5Ty}N1O&kbe4v0f2--e_ViJs)U_ypgQPF5b
z4|>pG#0f@BbOr|vIG>rpiDt-XW{8>zW=uk6%;Y4RZ#s#Y(Zm_=uTD3%_x`_i?_KM^
z)}Li@YFF*rwQJX|UAuM_Hr*ZD_wCr`81DPly>YRsE}~eIym!_s@sq}<j(=j=sw7LI
zB{6aQiq+55ZX7=;aT5L|T9!<jn3S5BHoj(JVxoBZExKOVOplHymJH&Et<9h*FDM-5
z4GGk;+}mM(C6Zbim1`VZj{Hk!_wO`YGN>h5;Sh?rkqNw0viA|5aM{s8zBHLv+kHsT
zdt1$!-=Hju4DlyEh!3qdPxeI<b$fRQ7r>buu^&a!lutP#6=-jzd`cW<D|*Q0?{yC-
zW0fk0RG{t^?(w72+RbBFJ738>1_;rrCii#1Jc_ZPw!2`p(e>5D59131d@OZz*Lz*%
zxZsOIkuH?e)_1SfsTGmr$LxBNm|aUJUuz3{@#_Ate{b1ujvs3`6Kk?QUTD^bQqujV
z{)jR-q@(b6nK$?dt~@q|kd!pv=-YRc>4H6Ql+U=~n%j4@f!!^E8qP<_#><dz-%;HW
zI)nd4o4a>~#B%s_nK<P<1F@#-t5O8QRVC@G_BhNjV^<l7C4h`XdCg0LBa78ZS$*7|
z?9BIB9ZRUQ%jHo~`_ayhR3o>G)(eWrnQBd4Rh*_yD2kjM4PxP$53NahYc<&3W{<Y%
z&Hgfx$@PS{TJ*y4$Z<5Q`+8&wrF^%}{2z*uddKJ$YE_ptUezm@N9^-e^<8ZXy{D`9
z*I6d3`h>m_?{dlkp=o3}r@Y#IYvgaZAOm^gW@gZ}bd&R1PS`!<BeK%<-94j?lqL!T
zW3oBERR|nA)DTU7`>^`=nO^3<?s;hWGSR#G-AupgOPMC##N}3B%#5}u38{W3b2J~y
z<w#2cFIYlJJRi>GIkvV}@k*{jS{#0f*Kp;Sria@dW;4%o<tRIBX(Sx@BbMca;|PDA
zzo`u0<5;qrwJH2v#*ISsS@>?I@UP9LSx>)992PYRMZY^ND$<93_nJ3@pGNTWPUgXD
zIZ^kT?dCv<5UsQ%YCo@LeT1C?t#bZ67yi=L%(r7V{q~`Li$4*ZTH~m(_C&(OYnh^J
zfBGE|H!2BF*#Nj3788-Dz_L^){Bm52kDp8}Dr%v$M0Z~(8Xw8&&IvW+^C%H|#usv-
zcLZfZfxl+RyIfn5V$6K_Q5(O!DpfNi*b{;|jZ>&g7z@;ngw33>Lx`V{Ffw8T-=+<<
zgp;S5>K%!^iYt?r_?%1mc$%NZNa|Sk(&C8mLj8mof1i1*n4vHcN;5)Qgsusb{6oeK
z7qc(;Iw9Phu!Smxy2OMjp&xoz$mBnj$txLo-_KJ%o&xfexX6d@W^ncIr_$>Fsa_Bh
z!#OAnB_{X>UT0OTh<#CP63i3BXqJ#ZF^>zlBQ#DNDNLD|>K~pgOBZ;&g0U8$6Yfr&
z!uhudiIWmID6@T14Er?V<JTc{qR?-KFgWQ`PTwf>CXL~grNXVGw>Y2kLYrk0=f71L
zumn#DxS#TI`d*J5{F50h_psP2moZ;`GgIcS`jbo(tE;{_jY>-bb_nL=1b<zLj4m^u
zz8A$_f<2irU!OdO^R5uCBuA<Y1d1;FBY7b;2s0-?;itDY_GOO1?B}~zs_@&%qo`i^
za&i+I)}*{1>HP!mhapBqy$Gc>G8>bXq`dm)_1;EHy7H=ENu9%~i-gA10#2PO45TjL
z$iId7v~rH@78=tgsbP}R^v%NgG_zovGIjLQf#qTUu^c1=nAUO_pt8b?nLE5=wWh*%
zGWSYR5#P+cl(~snA;CO#oKadFc^b(5P#2};5E<~^!YQQ1BzfP|V$<g`B@{9$-Z|P(
zP}m_wMsDg}J9ROvf^a!KHM%0Djc~F-yKTy1WMj<^&ik-7G$oDNZ-1rOCB#je?@NA!
zf?>up{iMZQkkB}7DlHNErY)wvLR3bWkFSGSG$tP=yO|lWF`8#K*Jpm{mROe1X2zO<
z-HVF)CDJEZ>6&MRT^W&R{$@tFZYdj*_aUv)I)Vt-GUDiJ;a*0}Xy1FVpZ2j!34t-m
z!3@l>9>PeMXFik`2Y;1PPWWB+%+F9J<YdNjs%W7qa{)6;eVM-t4)<fqka8)LEgCZu
zB^LZwLTlEm=rd*d5|eK)Ti7ohY|pYNh|D@HRGD>(ukg$16JP~}q3Ibx>eRxA>Qpw0
zs!ZP@-+sdpi^W$+%C1)X_Dx=oN$tp<AK}|oD3C50Dap4N`2m(MY$N?uNxnirPOaC*
zkCsF|l$LmZAvEWN&@`blXA-9x5^m<q=TwrAI3t)r))`AoX)@wspA(knU9$#D3B~f-
zD)i0xjgKO5>{0`XHR~T*QWf1V<$80RrmDMj=5`u%1I*8`gb-`WAVzHB4A6_Sf+c?s
zJu6(yALTa@^|?$lQ(#uDv6?~QLH;QEtq@nRjf?m|I9V`Z#4%WAC4{E5gwo$gOCmOU
zS8$=SIam_06lEn+WmM(9!bs>%RAHeud?Uk`DkC=twS}>9<H4~KHvGpujhQ`|YT~k6
zVc!CaFt#nxMEI}5B=_?AzVIadP-vf(&H0oGw`Q&J_Zy7{Hj9xPlDk~Yl?nFQZ*gAv
zf^p8@IepN+x!F@>?Xt0H+rHVHh|!YmjplMTDyHCev*sCSA({J5=BDbt%!7Og=$Xjh
zrNx1{LjBwf4k~vZpNJ8{gSnxcZiiqjniK5(m244I+(MWu8w?37opO^<UldE<?`|(r
zP`Xv<n3rerZJ)e8^PzW=Ho)7gG<hd+0or<PcvZTupqPJ?zAp^Te}(Svt}mWTz5b0M
z>!eEEZsB^#Li&j?t~8cDDHN5)KyvF#$3p*(l$z<f?!Hog?9jrs1vBwGY+*_R&trqz
zn$iaSj*8k)mCo~QACfJP9Rj6X!LGY8M$1TzJwJ&jRV#UZ@_NFpMqYQtP;sp-%E{El
zM{5Y=1G%!>U%S@x&KH071=}zgEp3}JTsLSawjLm@mRKJC5Fxr+f2(f2e+e&quy7hU
z(c4n+d!}tdLr5#L2Ptneej>)No`!z<4dp((!E~)W%6a?i5F4wv*7H2W*>XP1Xx+&3
z(vtA;kU${IGx{m%@P&Y5NzxcFh7xNVF{A%x??!F7zR3`}rcGnGE%~-3FVxt5U-U1^
zPvePxry=yVujE^+y{HXlj9XH9K~a`YzY(&^A{3G84T8OFnU{fi7JIF+df{?e9GxaO
z%QB!}aPWkr@<w_}=q(?^>CXzc$|t}f3S9JO%0+q!cPb{(6GC|9eE$f_>;mtFxz}<w
zsWM`;;HX?OapSeLP)W8JmEm5{pjw^7!1mXLup~-XdE~@vX<@Ip@?jSQ!{U(Sy)qOP
zcFb}S_S74CR2jCKm4Sj2`FSkwrectH)XIoWLdD_+I$F57coAJEq%KL|OnySuk{4*a
zaBE3yM1SmjU+i5`r77jg#8`_jWiV6qlr)uDh_g@ONV8zKkKwp`!Y+FZgSzaI95*Bk
z*dNjh!obo@sGMQh6fUe^n6)gEb_u(dg$2EroI{MVw!@^$TW4k1^zQS^KB7Ts%$Mn}
zH^&cGOO+w3h0Ya|I0|#MBFt~J<uy-RsWSM4z^{Cr3#LNX%1rEr&Xp<HHRD%J^_#~C
zvps`R0^<Gd+Eou}(8m9@1OE?%rm6|FSLm%8<yXcu*weW`6`WN|FqERznQ&j4S5M(|
z{lb;iIn(+u8NKioA!hz_xi15juM57w`+$i45qwk@JzL)3N2>Ij*gb2_SZZ45<|WZ$
zqQuju?+Kx~I$~@Rjvk?k-$_xXf2XABovb_AGF7T}{NH(m12kCu7Yc_a(#X3|6zhcR
zYa7PL7h?Ot?#*`YXeml{c<%2rDV(f$sgg@$n6Q~VbGiVR363Y94b=?VSJFuP3Odrh
zo(97Gb+DCN#q71=sgWF<#?+^ZX^oK8U=of!6~x@f^G`+kMny3HfM21#ke?15>8VJD
z%0HbDsTq3ukCaqjflq!JJ~`zInYpa~1N`!6jkCMt>369%ESJeIzaCBE)@5^{r-Z6?
zFHx&-cU`yX3sz+b9c7RzHSb~Exkxjl3d32U@0qbM6L+3zpk>0^^-Fx11k0p~u(w%@
zeNMA*Z+#@EqJpt@Dm4lvwM+2(Ozl*kd2r}`T~xeR3J+?T^V?i+7OWd)ddY_?sRGVz
z>xORACGSRV#z14(Jz(d7x~xvpV)YJT#j}$e)F$0ihERFN7ptkR26;BSi&bCgo<UwM
z=ReTZBF`9GtU9NA7I`l@e^h5jo@vWs)ehY<<Xvso=~f``YFnmTg}kfnbX}Dp)N+pa
z0)Mn_tsztrOwYYb4+}%jEx`_(Rrhm_?htO(Wpk7Y@f*uIZBTdPMm6>4RZJYrZ-N3P
zN)?(fgtq4=U{AU7{F=ycCSAzPKM3|xE9Wc2-)6QGWh;e}7h*@}T6k*5_-BIIa;Y*r
zS7vTWz%i!MdmQ#E7llZZ(Eh?0&Q}tyyb$9Xwqb66CY3QO!zzUbFD#^|g_8Q1P%@yc
zmrqfLV`cSLH^V7;8%N$&Ou;Ybo%VJ3xBlUQ*$7_E^2jSS=NSLkYB`kExXWaJ`=doB
z6WE+>;fdLzg<%Af7B4I$9_x%rty1GBG{1-=j-Sx^VrIiq9MBS(5P-<`+Oj_ng-0>s
zQl+}g+=2Z~J={?3#md>iY>y-JLspJ<b{A3A2QotpcGHn6RX5yaZl<w98PjYKk8m1P
zGS?`~aub>^BLz@{ww?>e$+HP{Zq8LQ12XJVc^AVE!=um{W$mO2@57iV4%5K41$R5;
zd#sMpmvJhU8&T&Pw0?NhG7;rQ;qK2ey=vG|o~>}fJTgwOH)PTq-R%v}Q&{ru@J(ZB
z5PUL)ghAv&KV5xPM7$_eY!0Ivgu2bK(S$ic=!4m7C{l!vJf`8<s|bHvIKO!Uw|lOz
z?!Ur>fssm~XrodvY>5dr-IHB?1U#7QF9$uoQa20MElDtB^;>?SR$k4**`+(Nk>i*%
zY}vMeeRgeI#cFW65$<h!YB;-g`%|p)=JuCai3Qo3rm;csAGLY4#y;MI6n0VQYI=d%
zgmF72(2YXHj!2FS3H3W>(stqEj<-VjXz%UZOx-T5weV4cLKgP@a5_$^FnV>j>=bBp
zldjc}1CdvJ<OpZJ!**#=glun^^NIX1!Mf{T^q`Qndkbw9F7A%SZ)x{`(XWJ#mtUYw
z-R3=`DBUesn?-au)SQIhfqO4w>mAyggE&&$zG8|sT_;TO^%)_{$BMIdh3d4>Crp5u
zathCDNk227@Sq2T+LjbLT{zj29Ygx$!jYPO$7@+^b1z4vA>N|Me5iZJd{C;;Zxjss
zUmEcRG^WNK=YVTuCRJulpR_3OEXto}b;utU`u30X)pS_`;WjgrBu%eycYlT!pCL0@
zb4k-Bq#qb*BE2{R<))Nt?qR#`AG^wRzN$(m!V3pR%_Du*Uz0v;yKjF~&S%SX(lz~1
zq#1i0(e2m<X1WXj7bz|FzW{e)psIuP$?nGDAh;WRR{wc7)c_XBBEt6vMg^08cK(to
zgWd<#MmN;~AyEudc2;d6!d!8AjHdt39elR7QM*m|j=uwDHXME_JI5`9VaSmd`Hm2-
zi#YiA3-`o19HeM%O)~Apx!G*lNDNYi*G6zzj=T~1r9w+9cD;e_^Q{GxGmQ}VLu_Jq
z9U4WC31<%d-3JcfyP45&0@Iiic>I+lHTYW;fk?$Gp(f(;pB8IYr<8j+*eMS)W2<^c
zF9fuWqQ2cz+h$N`faCD*nNGZ#0-BCj!~Dh}f?z`QAX;9-i?q4I$ya~md<KNhBMZ0)
z)4uj%dS57MUjmzYru}K`YH>#=a}k|F#nGAc1EJ&SLePA0^ylnI*Z%WW{;DA+C>*kR
zc`HxHfv-K+C?vj?4M(lJ_O(freLFk8qnzpoq3!jls%F_DI5!6g-@ZPAQ+_BIk3SvJ
z%q$H!J1Cn&IR+GPRMxZ##oI#1@%dV8>xfucr|6sab&m9h34?EL37}WKg?LAm9RY^a
z&Q)5~;73QoqpH$XgF<I#^@vT&Z)Tjp<Rr$<kEBwi&-j#bE|u|X1qLve$e5`o=E6yB
zIx*E}FSN7@X^pf<w^0~45jg?}JDlsQQuStNRgUb`;K0Wil@|M6VCKoX5dwc>)Cl8A
zix-K9VAI&~*b=8V$lrIlev2maA+`}>8KLi9^2RLc!-9l%q-D|~{)Eu`W;PD@V*5cF
zC>Ty2r_Dm|$t~~~(@#A?%|gei6uMNndFm;4zMJ)nkzCXWq4pP}aJp>!#Y33MwqI7$
z@dDrVBWI%B%F|=HacqZu3#nHsHywkv<skoS<ae|6!Fa-VVO5IWU_mTnQTGIu-e9fM
z3wyc|V;B<7bg$GSROnhiG%_VkZS0OegIR<~E6&ajRXH(JWm37{DVHrqNbLsS1^9iT
z|7?s}GswKTZs%Eqj6W9wdz0wKZfoyP)XNNXIj|`tCI<`4e!T;J)tz65CC|fzv%O3X
z1j_jk5(n`|Yro_cO69@Ks01?3<&Bqhy~z8Vkp5rgQ@oiAQ10H(?J`%3yuDbNymyxe
zU9xzSEQ^*J&^#YU7Mlx4yH^<auLMppwC{XEu$HoZtWO~DXe177L*-Jra=$R+{GaG0
zA@J?Xv{D#)`w&M)bl3j|hvjbs$8URZyh`nhiP2OsO=3Q$Wt7IQ8#1(Q*_&l3yq0O&
zxhktFzfNNp>iZ^f_;mF>&nbt5aTn%rq0!y-7nV~TRfc{ym44nG{!TSVn}nuIlYBLu
z%}27-mMHil(V9-7|I!;AjA8A&Pt*ABTkoDwYNlgavNL}YS}rHRTEBC-njMT1uN<Km
z(3PjzzVLn$oha14za?Ti69Z1Qr<ml}x?zXRlu!K-rhtJ7{9`fA7pnfahKo`O*Z;Vg
zsg~_ekz7QCQ1_=XF(hl|wF#v!+s%B=A0`xA0`!inwV~3?OA{*PqiQU%c3=FHo*GiO
z!`O~pY9L$l9DOYQ@wb7yhy)CN@H}T!2_;txVRzcEBKR{X^j?KA>=pQH-LOLEuO)Ea
z8-+X9qL%8|DwfL0M7B*!l?E2tgx|NcmG@;ct-a41;Bfe&u1xohWe?miJ)DnlZVt5K
z4b-2r>?LYMCmHThIfzrHI}Dt1IbvC++bXQRUWi%iy?%sF>$ZLP6!iF@(EsORc(vgl
z?c@}vh0c$b(0+j*jP&CTDF@}Cd^q!4s?zxGl)>|qCJT3O<WOUG>c`$RXn)Gb8GEZQ
zW*lUy0+lgKW$Gsd+h68PQTwF4n>m7kaEBGBib=VVF;Oa0eS#(zP(F=)QLaV#Jd~f3
z=cr8WBMklJC%RI&dGoDE8pIm2O)TYR#)XtC8GmDa@H!>{`nJ&Z$+iT=T_!U=2xE-k
zijm50-=gWPY!5D)R<w~Ba$Nw*|8B4eD?VKWhotY*f?zTvmFa^l8u!L~9$V5_y;(4S
z_MBfk0x~Mgm(V<BdCMZ+6FNUD;Jii%cR$OCILyu{V8ho{8NLJKIBLdm-4d<M5=w@)
z)6(vnLvStXx)c9ePD6ub*Sz|j3=>WjhxA#7HnvWn(^*2t-#+GukKnk~6Li7t$}ww{
z56ViZEaadt>x+NTqeA-M|G??Jy7@0BP@IiZrLhPYtdL&AZ~m))sIu7X2#L?d3FfcE
z81~n1(xQFNiHb7y=R)}Hk7)cpXS5J>$Dh8}9e>A{az2AX^M6khHhkmf&v~U>oXq0u
zHLk(Hb!z?QZ!os?|NRQo#Q2XdY4bj3612nO{COyGvg<udfSKY3C>pmeX=AG<JY+X{
zg43BZ`n@LvDlyjtZpnbWtHf!lnDfzyt3b<^%==>-K6_}1Q(J|KZ<nbwRrYi$yz_0`
zXmfJ4qBr@En!4nFD6^W|N8OhQHwMRq)zJi(i8wAqPfDQHziFxj!*^p40u|D~gBht3
zYQ7tz)>K*IeeMbE-$lVH^nLdN#d=D=`&|B6wonjWWapDI$riBR`aH(yfj0oIvpz?v
zWHZW6fMv$LY_XvXc*Fo@WM!*`2Y17QCbAk}GT-0muJIDa{ctU9X#uvbf)H1m0uyma
zWx;8z448>7IOwZkSewo*beDZAr2ccCzv>(!QROUp1Re5|Y<g(@KY!(~oQUPDlr1zn
zzuJ60>2CiQ=Ke`x;9djWFBIL!bxDv=f4>-WG;lu(p)cqCu%I7c&)xTGnU~}RZwq{1
zF#j0Jksu-c$IS&%Ua<)I<NBaVvPH5;t-m}LTjUMo3C4)NA}_=iS+jJV<d7RaYB@&u
zrZ+P`V`aE)Ba#jG8T?OCL20ZND*?qOoV3f)+D}OTDGZF-e!2|xjC=5$*Sy#)LM2<o
z1)<}?Qw^%Vs%a{AdqHB`_20@1;d%HI>8dW2_r{w4^iZ;eno`oRQEh~SYzswH23zVz
zc(*pwS061g#;`WA`x0njo4yt8T4i`X?z9=rV+6{lY6q=OX_0bWO1ZY5)m!5<c6PzY
z3w!@Ph8hIJe=>w0|NUaac(&+b(O6o<!E5xkGGu6Mku^zEHTjv$`!AQ7?^yzLyi8=g
z#_n}W+sL#Sv6qXM1XcXj_S_;@dx{GReUVhg`!SkW#3eGVK(4}?q-wWXRGpR}xIMvU
zsy(0%)Narqpxph>|GcP-J|){{tokM|vk~^IVCOW4miNaN?Z*{Bq+5Gh6!a-Y1FmzK
zh`=q}t-Wxp4$54msQRp_s(yGJ@aA-b5Sbjr+0qzW1jEX17L2gZ=Oml)ZG4XcPDK6~
z>h=P6H{%wkVSjlPQF<H(e2OGn@cU566G(e8l3c*2_^1djj}@X|req7+08XuLghIfj
zV!MWp2~zB5me=i76(cljuuj(Ze5Rp#>Z`FgozEI#YE_`nREZCG8W!QScyVgP6L|P0
zN{f!v@7ib{08cIy1Q_;ls@9%<FZwkuu*7a}x`KWq8g+Ek2%foQjB6;15fQa{kpDet
zN#qCecCcK##S+c8iYs(<8P$ou*U_Y44!U9>FzrE>SbdFb;$yiQ(cnWD<HC#W<uGd|
z%P6i$toNZKeK8JJh7*TnG?yrL`Oq=o;V(Y4gr4h3^rhaEek-Qy>G$+)G0{N(z(uIU
z2L?KU9_tzBM+-R4&!?v@fCe+p_XN@~Ke(V}2Kk)9yaad%;(rCwsa(J<@m?T}3YWv{
zvGX%1+r$ubJC;&Thoq>;AFRuXm=r`uv0!()UR)AHqmZ>k>%}cWG?AVa&jistPSYno
z2%?Lp5UVcL5U!hIP~t=`Yc30{u`4*tvjgI<z`EsQC*iS1n!ZUY3;4nOs|+eF^!-9?
z38o*>mEzhE8VfNuhtM@({ihK6nTh<kqEFK*ZxRc=XCtQfqiYBX@p~gprI&ifnJBKh
zillOHc><(`-Y=q?Kz14w9CjiR|F2P2Dw_;;k+jfz8dELC3i2bw`cPV)pgogvahjj^
z+uB38J(m+sp386Fig2(=WjB=&4Z;_=sr;+vil_{uvElFA>&Y#)5@HM9wc6A5{Za+l
zn^JD>gc6E5VNe(%c81ZhX@t#Y;mv6#pLG8wc8am56Mt5QK%cs*kCGLAlwUCd(KuTe
z<5P`tCGyimT{sOJ{XQgDJ{*<D5gPq)FS1Cz_3!$4CK%a$v3cDWi^6F(rw9^T!s)wt
zWGK3?4_%vyHD2BgXR{qL`a6W1N0Y(m7GKKhlKIU0$$wO5S^S7P`5%0L@-C8UNz^Cq
zHl=mtzmCPy9zkRC<0l(9>#iFyarwtJRng`?5)EaEuJxtSn_W(f<x7wmt%fr~l*t8N
z*5qoMn!L+5+A>~W)I2l_8>{A7$riOk)J4*tQG@tWBn{&<onmJs9qH$~ZWA1doJ2V%
zlip{MbR3Nrby0MINz)napFzhiB_w(hcz+j<C3Hxmi1<Pj4L8n)z+B$a!ti9|m28w*
z=|ucx6isjFljo<{ms-;_L$I$_sVr&(WG5SMrhS%Jjnn)+Wbw0y57`S6%Q0oIWu&gg
z@};srKOAk^svHvaZDoc9^~$O^Uq@^_^Ub(VVr`Hw)>BrJ!ssn11MK+9@{UyvG}~*4
zM8!2`YU2#o#fxSr_IsjarV~coNGVrU#g_O|sNoE0q3Il8L=ZA4E~=Es*P(2$m>x|R
z(IT-mnl7ZHMQ1cki8z=1ysAHC2jO<dWTmoQhi$vq{%pw`RsG_O7&;c1H8FH5y&#^9
zp*!8Hc4{ov$vm+(78MS$E0&H7hTepdNNJ&J19nmQP$w-kP8aXQ($|9nMrfaBuK)Oz
zvDy3>HXh3e<yEmWj!weF{Uwgh#S0#B+z2{3RBzkv82b%|A#Yzyv$t|_b~_flSU-YJ
zVR*eG=;BZfZnwCm%N56*IHz;f{jNJx$)>&`CXS@bvEo}s(mb$R1tas>NE*VjBlO}=
zBWVRaEZRoVJgOIujG|_GPV5^+!*rXH%Sa%s3Ib9F@yk&(edV|ClFAurbYXxGq+}S9
zjl00g5qU79JYAke7Dh<dXE7Kn3s<-B`D|n!B%+1CZ$r5yWiW$XZfBu9nw7s7el&i1
z5{A<hPZve3V<i9ZpxqX0=1KKOuF+sK2$pO#?Z9Z7LKDT*(KG=(KQS6+*0@RR7)^6v
z4y4g^3%w++9Yc$$P3#*3%e7h@97Au@6XM0ODEvw^j-$V%JH-BR^tXQZ(RVz?Vs)86
z`c+TscuXQ)Dqc^ZPtapx`UF}?H;GLXs6f9E!xL#HjA%t74Wr}5x<tCz-|r~fq-GpP
z;1>R#w7~yuQA(tvO#fy!rTWseXiS=t%`slD<ioDli18EYU!rYNS@=9=`E(LupHU$n
zu=UNfjX9Un0^^5b+$6f)?-B%2#H<gB&C-I9W8(QqbUntBm_$Q4^)7K%5{7LS5dd96
zOGLQyA^7kXTI@G(;9dDFV}-k1CM^ie?OAJqf<eQMB-2rmSoByBY{6QNVX<qlFxjH_
zAv2OkdpX2g$#f2fuVpe_#u+-q*2#1}bS7)8Ui`;oXnU3zpMoxDi%lu?8MUS}S)=JZ
zkP6Wy_oS!NU<DUW#geI5S)Yq1r_$eJ+0>@f7wAOMnNH`@qdi&E=$jPH?_|&(EZdGu
z?5%sn>zQ;FGg|3cG*qSOWIN)bEE-4CL`N3Jc}~2T1*P})J1`x6&lBy_(ZLC^cRGEJ
zo)#0c=^lDe?8~MhCL;!hM-tcorjS+``rS;_S?4*U=*)%%4+;`3IavKhu{np%WEzq~
zp9_k^Wupz!T8Q;l#&#oVy6J(qVg@7-JRtVZz}y=|K9|ODT)#LjmnKnrk3APQ0TZw*
zkN$iNkz8F;bm{*EX457`1@FgA`$0^fy($)lpB*|E_^uXhGbv&bL*mVukjgpHW~EUk
z`9>8+8apb{Hn%4#B|?`;3l;apW>mvkcUhspOGRZqcwQ>b%BORn@vr35Q6o)lSX6|)
z6}Z+L=H!id)cCosOLmt+P}ULVl&1UM6{UPSCBb-8F2@*IIUfMrzZ%axaWUf<{H97~
z^%^^;iOf_dr*ZLOQ2~wQ=smHf0CM0u#p?wywByBl1<=EOvAK|%yzwAZTBJtsnKrY6
zFlrZ17Siolq?TFq6>1d+W&yH_w%L$5FP@oA|AjSrat@~SL(w^hMpA{SpG!qfBZ-}J
zX|ZZ0)|7Oh2+C|0jYX{AL9xDw?g=`~R@LRUzA19h&d75>92Cv-pm<o8^XN7_8WL~K
zqj;D#(33u&K11Ol{<4^k_0{3vAF>-SlR~jZlSQeR7J!wc5*i!Bw_5^~a2^Av=>>m>
zJcW29$fn9Ng>RQi1F_UgFo3fMN@<AKC+s3eDmBd$&875J)?Z&KeaUAlFKKz*N`Hs6
zfPY)GEdbM~I==v%L0AiDL&S^h5?FAUiTNm$Aj;<(U^n6TW0V$Xe8iT8vPbrCA*4&h
zSQ|XiT(QnZzl0R+Wl;CSVqY0_Eth@5L`^vjr@dlaIjzRJX+|spGJLa~#?$tm>*e$<
z8o>>XT?VHz$=A%DyWstj*{xceQ6?U*fbf0A+ZA*okF_!r?=7c`ag{W=;oZzMMB(S@
znhb41TO!By99^?v8xH=1$eXPKiQif8V2yIC+DJU!fW)(o@P5W@y^C|2!<VKAnGdx>
ztE1`xd>0mlrCf!4uMthYuHNL^jy9=dmqK6h4hD2_z)^p}bP_?EY2sid4Qc4#^M)f8
zOVAcu97!<~T6h#hDv$hJMym*0$-YeG*}0$LV0VoMPCGjWFdw^W8aF7H+qnBEA8_Fk
zg9|t0J$M0fJBEuq&Q072+$+G_09%R%gacA>iy{*a!C6yP<*HzW@mK+YP#1BH59(SW
zheTG-%CO2M)LSK+>J;jFai_B;HYw6|)mR#Z^RH?En<dZWx;mUlp1R1-Q8y&lEyA9`
zDwz(tIzmXTTy@Vav>XU+Xsq2;XBn-LQp(i^bofJl8Zoa<C+xDc%+-!k)X6@q#fSq^
zY2*&kzJ%t}yW$&5X!=OQf3t;zMO5QDB!=T2N`H^dF;kjtKRfa^nmI+Cokp5Te{~ya
z#%&f?Es)my^`zIg*54tAj0BC@`n_0SrxCPCtg_Q7CNebnZ05t4Uw0T7pR7?;Eg7<I
zz`4jy$8tJ{DA{Rdnaz8@GAKq!pj8`4Q^9_veHbn{psYe!m-hi>FsnyhP1Uo+!Rk})
z>ax6Bl!2g_D%Z6r>)dd&8+M3}rF81VCU<eCySUj6_qyR$H{9oj+r>LeX_&F~&n@h9
z1-?`8kg47u*KXKJQh70I86DNYw=x@9y+y}kLmtf(F&|U4#}Pj%*0D!}yG$Xr*{2uR
zM7o{zqk7pqBgxY#2g!`2&yq&7q)PwyETNh#R~gN+q#<Zo>5s@RPgvL1bXDIT$EZ&0
zZW52(`<$$tb~B09;K~BtRkL_y8I5g#f5c^By31k5*`Bo!rza~bvtsRCYctidMa1rK
zm^xSwg4RM)B~-$0y-&zTM~wVS>ATpSycy-kq66?qsoZNvH1w6VXKNQ-nWYN9AI#rl
zku5STjRqa5BJ|tn^^rKDFUIMzJTMKx7z10Svd)zUnsAxf>CQv{cst7j2jTC4&n;gn
z67g3D1IeQmsmTQj9Av!fm=S5Jf0I3dkXJnLaL&^i#4F2bSObp#I+xp@uIa?un#q=d
zB?vOI5C-Sto!t$2X#{Z#tVyJutv^c$yct|Dc#H84iD>`2uiZ>m1Akv1z5%2?Kb>^u
z>?B1wyU2XZUwh7O<X<N9+1wUx(ZG!B<hKZW91UZn*hc+Nj`}g*nK5X~az&kt^b(&1
zq`ipc!TTLE5Gz0+z}r+m<_|2N(x#|`;CH~OScs^$jlFM!izu(l>OW>4#GqOc**Q3H
zIqDwhy}P5{%riz9WAXq4%n&~5;#c7|ZY*Zrny#tBNbheyXdW}Og_w5=WBm0FLj^N=
zxQcGGr*j>wF`V`>IRYtS<vDE_Q8d@8OY(2lrTe#zZTQne%r>#4>2as*vl<!*lL_UH
z)`^Q&(r5)Z5F1uv<0YcFl12qK4YKPz*Ieq)A%fJNA1eM~B^J6UN-Jq`!>P$<2`4S^
zS{Iw8r}!?7ouU7UCFJvQ2Z^1|7zC=)Y9!ABfX7hjZF+nG0B<%HftkHJWq8=Ge_-D#
zv#KxFq5r86j}38pFfbWokKG%aJptcHep77Dgr8!k%Z00%=F1CLi!qMH20XY5E@GmX
z_yo39vsm&3{VZq%JJ9jVwTIYjcs;NESSr=s5Uo{ku~zp~RZ%s~@QHv)%wnUymS(EO
zt6jv-G=#k@LfeIG#+&WU{_xGqnjt*U!S$WyK0+sW79E%@o?MOIE{OxH>8Pm@&=(ui
z8!0MCg*<{&LAlTk8`BZw$0J_@qtS-^+iZICwHksOk#o^;V!;|3TJAMQ%j0hMLUn(}
zX~t1B+NG5H8n9`uv-see_?D&<L4iY-#>^WC2w0nTo9l3mWV1d%`Ma4(Y;hnyc>_kG
z*n*3HEQ)v8(IELn@i%Lzv4M24tsbVqjF6w7RLXnh>>)+&%O{&uuG+-zWUP%a9Z^vc
ztd||nMz8U;8(Lo87X7?$m7xW8xXXH;3~qPC$=kZQJAs<7A;E4(uSfXy7<J%aG0E~S
z?X|2WU6w{1sH-_%whE)leQR(Jc}})tIR7Ajba@{;9Fhd7Qa@T|YGL?1vA&v4OJG-h
zl?oF>-qB$8XIyM+%-F-YXuRSGW#4R(6m99rcp6-ZUslWBl=oU%5Wx;n0i^LIA@h@*
zn;8Mfzse$5FA3A$=hfmfYiVALEV97N8%$*0EzE9^P^p}Lz?@FpFb44|@#D4B+(0)n
z6>xLOL6PyN3=rKy6FbTVg1R55Z^O%$-?ARI2<cDDC%XXhtCs|`*;<b$=FEc0$7C|V
zV0L`2#-zs<BVb)lu+X?cDT@9n=O8I^m%$(^3=B8L|Kp6)QhCH#)&MJFD+<LuPtvG{
zh)Sj?4960R_Y|m}d61;v{@kb0f)MNzpJZHOhRYIx-3!XJ{4$j4X2u8ApJd#S%EQ0F
z7*6-MwV#b`BqU!$hw@ca@)<+QwxpZ0^3~KS^HIj0uqukADDni_{J`2UwP1Z(8}N&D
z$nFbS=eIxi8wWx7F29ERJs|S^S~cqoM~zEptU}EC1J;lUhvDNkLOrt$?cu#J25*0E
zIC;H2Jd!caXtHvYHOh>mEu$qfP#*TAmrAQ@V8oc^s}eL_%<@&WX?m?~ns`Xp&ipKv
zQqmN?u8~As!bk?>eFJyx)+zWi4K=sm@7N#sC@vep+1D_#W5Dn9D$ojE>$TT#Hng$Z
zT6SiV%0s6m7ZA=%pXvZ(IZ~8?p6VI*G@V6z95pn@A1B|7o8h-d_rz^MOoE!k&PIS`
zBEOX$VbE3_peBmp+vow#kBVou(dRjXS5M-0dYs}!IM{?kj1KS)`ZUGyxqT-L*+%i+
zPS9E>*1v=!?<Vo?OE~f_6&rT}TG#Vn7tP@iu+G{;_o<`yr+hkXlelXS%~gcZ7e#Y3
z9TQ5%Bk<|yuV1_pn}CCPmrV9jariglie?;dkBS}5^mQ(DqBv_WxH~QO?M0Zuf0}r6
zFHXJwI}YrlW4Q3u;<$YPO+jMOJ{sAe_ga2)+ToNN)AmXWHS5?~d9S{`6@<&Ah3XqD
zcmcSM!RS<n%ZcgI0y<g_sTLa!%M2`3op#}6nLntP|9H%5`54Bu*idQl&iuDbtJIfW
zN*T7=ryj%VtB|6?c0;R;jp8@^Xf)?#5)A^42@Wk|FNXVX+V-=rbqnspN-!*|K)-JA
zo~Ua#9MYYK)5`9>j%R*|)0a&u^`19y2%(aVq#XIQfkUAzUx9q|z#$XMFCI8#yk~g=
zC$&=TeM=Q-gr(nxConu7LTaUA-6yhkB4ma9B`8BKyLmxgev&Z!^mSK&BfZjV1gF~7
zZZ_sz%h(|mM;#L{w%~wx;lO?x>g#pdvI>`z>|I8w*FiCUKOKobnh=Xn$9egQYxmOx
zf9=~WGPb*K6pwCZNX6R2;z^V@gue(sZw9+uLHQgQaX&UBX2hjZ&3!f`ISN#&`J4@@
z@IzecvzG+L{J-TJbbmSj1)DqE$*K4D!6QJF9BnsY`0TAn8OqIomtw2ICKrhVv{xyc
zGkG2r#a`3IiUTmV7evPa#0v}?#P$QYJ^4Xw5ot^$-jK6dIX`!Itd)?gounpf2YDu?
zJn({4963U~F4C8|kSE0ztxzjvkl55pBeT_|OcY+>T)6w(dPpi(%eQ(i>8gX62oUr?
z>S&vA7OFrR9d<Z9W}Mo6<E3InsVFgeAwi<<Af3zw9}{OBq~j-);x9G4AfqT_3VSUQ
zt8B+>5!^#!F+iUCV{<^jNJ`u=#&M8_j`YfBTaIK4|5o-e{)aQIv?x4696Cs|eRXM;
zMyiwBmnt`kiHB&gzxPHwT(e?7**vfcv6#}p&qdoII%?v5S$w6M(=u7`UWLduZo0-Z
zc;`y9fsGYUXOH1wJH)TpD0Usfm^3xw^+R+g3;)_*!IIR}_3U~D<y@4P*mW5CqNx+F
z9LAu-H})uBr5wfIS7_U~iY$v>)E}V}IQ?ib>j=d|tD2s!BlIc_4O!}akSn{~*8VG4
zYy*>`PVi^1bLx89jv{Qxso~H3oZb%lmc4+!+;(1nO)%bLc(_k(5(i$RnG<n_#BrcB
z<P+G{PIsQ!*gl5F{@B4fMbnPI_2R1VmBP5QVpazY8E<o=&VH;}@^CY#A$wcd&YGji
z_@2fN`oLQ-BlPCzqW@~97O`$R_b@unNh)T!(&8?6{+_3-^19fX&Ea`S&)sa!TTAc0
zHJdxa`K3E)3-6@+o1Jubmy;go@9CP${Y&8&?j&DYkY*s2Ag$=>E#{U}uRtfcZFZ7>
zj1%KZxd{<9PWlQ+9By^eSNoA}IqA_jC;hn&$=Xv>${klszv!g;lSoP@^-V+quKZ>B
z|Bs}PN4xYMTN(GztHJK1X`t~xr71=KYxxwblTNXr{2snlNR3D>NS#RMoiqh}vosZL
zr-JqjeDioG&ApDVb;NQ)oV37#1l$7ftY*Io&h*?|&OM=M=ylRL<B;w+>0C4@T7iT?
z7CD`C{w$;+CoKliV!*Rd&eH4$PFjKv<u7Qlube-F@;gqt5Dd$B$g}d{cK_42;eRUp
zKT)<2j3YJBh5i3ChqhEFwLz4&W>>w<J^II)vppsiR$hi_WnVVxxtM~v9Y}FVpfk7D
zom%j1Lt=Fu4X(_bEQlEMR2+|V5d<K*Vl*mtAhkbMHh@mdn69k<_f%E~JhUCgE1Ly)
zRxJB5=M<HiN31Ygi>8}LtQ9j?bN6*c#gj;J?)?1evzAQksaV5RdN(8|Od3BiVb$^^
z3t7K%>9W;NuODAkQ?qpZv;TkpV*{ui_e=@W3ZxpOI;2LVW~5f64x}!mUZjgi*PYz*
z3RimORy+!vZs38FTdzY3L<&cWL$bK)*3am%Kg+#BOVQDKJJMQrd9CZq2(Cw`>vy@!
z+mS!x&i5lfh$JE1b8@v{xYmGVbk)})U|z?iM5MYwE2_{<tqo}f61u9b>FKEBzVPDi
z;koszT!Q{yB4IyH!Vlo15&Upc&1&bA;;zly*7W}!nToTwaFZKA)v=9`b%uxdarM;Z
zE*ow1lpUSr!mBY5LMbbEl47&Ft>5Eitd9M7%I4#?z<_t|Bo8hnVRcSY<}Pbsh>sV_
z%mgqb7c)6Hl(4eViI0_af?zf35#xR6CTJ~km*FoF9xrojbG1{r@s9N}x{rQvUe+{d
zwcg$keLa!`sgWpYAiletB)Nomb7tgQoFt)+=#+`bw>ingK;pw^A>V;~Bk?ydoRc!u
zQD&AuBj@9@7=E9eAMIfHS6F`Hq$Er7<doF3rOTGDSh-5nY~|vqt|xITcSPB+mRCX`
zOekbr4H5#%kKpr!jK{6%Fg!jB@O4l9_(4Jv;?dCEK4Bc-S~m=m7y+4Zi$|~mKqB%I
zc)^P|8MY&jNottr;PHA62$*RMjZAbt1|x<)Dc;lJBy=$80k0ayPeL<GoEtw2`FPwD
z;fhkOPer~2soz5{8RAIgc|_%1{iW(i14+HZAlFZ-laMsI)&-|QZPEf!??y>80&eCN
zGK{jc7Now%>Ms&9CC*bnB_42-n*pFr*@blTF?gtfkg2;oBbwSwNcuRB$kQRV^j5TV
z|D?AOGOfZrLP%_y9m(Nl0H0});IvK;{<M>T@p5(eXWBhNGS+$+%xI`V!JvnujGKgH
zvJty~GBM&z*1<504zeoT1i(O+9m(mLqAX0=^mGqAJ&TZR1c8TtvS$Hq0nCb!8nRna
z(B~=0h6r<TnKq2T7GzGxV{j*6+-f|kzeC83L=QZ}LP&17TY_LP*Nil*W9TLq1IWW4
zYsu{mnE$*P2;R=}OaWWj*0pYiz@!xtvR-#H0D{&*ge0=u9Rkir`}}r~Nb@@gDX@6(
z3sM1hc^E7>gP`uLVSLPgAv&B@;%)$O&$1CRyU&e)#<MRHGN;4M5IUR#3Cs=j5S)uP
zbDbWopNlCdO7%#rC>`*C2fye#A@j^`xB+zM#UVX5NAoa8^J-Dy{+U+?c$b?2l+9~K
zddyDDJ5R`b;^~kX;Q9EY%i*8-$j-MtR$l@5q-zQq*gVYdBBVHcSR}}skxD!q7egY&
zkBPV#A})SxM8z0U2_xk0vjhy3)Ob29sRew+L$722FkALK=AZq*uO_9Bby$iH7bJQJ
zEU*w_Yvysv2C@P97No}*uz`UxTp`N!z+u}>nOW{Yre79ENcl~7eHHRUNNid1uKr2+
zJwlemxh<0s`FJF|n*csbRv@)K2Dbw~?}pLEl0HK0kF6RzR*fAG{bYtvZ->aMJ9%Fj
z#sI2e7pgC!p3Oh|sqRO*<!MkY5wf<%GbL*w!YA?9Lc;`}gan>y;rU_ssa8U226^2u
zTyqn!JVF=!8Vqn<lZW2AU4XHd$n(z{ts6votid`oc;+z%o&f{vYdj3B2V9%z5qYfz
zuv@7Zz1l9ME1m(>4iK`z=)vCr32i80@&^t}8*GF;d)~cUK}65?5mLu{MpUN*-0LyD
zb>|7$*#F3okd0Red47d^glPYK6(KJ)d+=X?#Oedx_~@_RNXXCn@xL?344}c!K%n7)
zr-1`}ZL0IYo9c07ul2y2>i`c+3<NhrVq2P!!9SL^K*WtjZWu+4B}lCv2{g75vb7=5
zqvcz{(bi@Ufvw<ptK=cD^$sE1W_aLjR=~X;{B4lP_HiC~dm><5iVZWg{Ujkxy`KJ>
zfZx#M^fYL?OURB254^)p$gV2)sz^nCEg^eWxOD*Vo+?82g}bNBio6+Vt-C!&ysrl7
zq=(+VE<%Jh4+BCw)_+T!N2yxk32ABf%u!1R;33ZlT5b{2TIbP`R!m9j9d|vZu+>S(
zp*r4scuEhU{*cpCe+UCSbPo)<e-1r>{=IU=!@w&8gtT>fmPs2#)^^1+0_H0pZuBVK
z;U+?k47+Gx;7ArBN3EWE=12d$#Y6w+t%Mvq<FNzB8hQyi{+L5?+)l`wh?>hIFd`3v
zZ`QgQKnHKu5pwde-ST7+;9DMvos<YUW%e+9Dvpp}4BLLR|3wAT9nUIh_=S^@Uv|0Y
z1PH%8LvUr_5m{F%;K$Z=R~I2??QRCm$gd#etr?yXzhwnHOfM06(CZ0z(+AvxHoe3C
zK_ceAmld3Ma|DFmK0?k7dpLm4*$Mfz&eOrK4TStQ&b=4dkdG(i!k}A8n~}ds$i<U9
z-kKA6uejJn$fZ7ygf3kKJiKh#{&VRbA%D2#vHgFL2)S(cDBWd@@Ny%g=K8tZ1h~(m
z#g{?w{k1&)s|3^=k*^`-gK!sv6wH2LCgf_MYk&;C3Y@F0?g2CVf3*$Q@8P^&CSXS%
z4F-9Sp&8T>azp8nzzwwd_%R*$xQLKX4ek-6?Wci+d}VYG2=G_ognW%)@~}j{M*Zz@
zmr^#c?esQ~?!<ev^v*a!{t@pc(26`@r^Qq6OeN%}ah?u-N(8(DTrmkC{Zxh2@fh4m
zsJHHsgsHayuvM;??LYONMW_zf0&Whet{(8kM+B&@pHP2`dxS>hQwa^Y^Js6N0Zu{#
zJsuGaJWpt_XD^_^RfLA<9%(%d0RuB@7(?hEM<Y73meBkwkCq+HA0V{AqcpVOBH&vd
zh6?Z;uMk^@d&IO*2e|o>lF`Bzz{5@}kNhB^WoOU;O(1bv)=OwbEp$fS|4~p;N9a=}
zZXGZax;vcE)>gO3;*oDd8a710Zv}$~bsh!|qU_+{Bb}iKZvq}-y~#iH;4MO5^Q<EJ
z8t`8W<VnUb!fQqp7(ESM0|T#xKL&%T*F1AfUxP?r>lABRxi|7&>mu~nU8|cEp|2wt
z&`CV-Q#L}MZuP*O=={w_59}BqbW^-&Kgf*?{<0BdV5H<9Gk%YI_@6JYh;0YC819=c
zvF{*)_~QuOp6C(wb_?WX6z?45#`TyFaoZJK6-D4AOuTrQ8>v0}-~rT?{fM?#xg~t3
zlRJq3&70!7&#B4q@E!{@y%~?xid2Vo#sfz<4MTeqXm1Ro^#Sb-puGXKH)J%i2zXw=
zdjohU?uqsz2)P5}1TapxFxuUeP5`6RO=+M8lsbXe3A|1hp3_aK6L`n(9cXtEIu3;6
zE(C;tWR{Kt;q_rcn?UGw5PBVWue<P?-FUA9@0bhkE=CRjc*lTu%!N1P#ybYQ4mZ7@
z7+wePI)K;V!b?2r>bC=UuetH=8G!d%s#CFAiEmTSylGrMC!T(d8-;rpLO8RdfRF<a
zWPvzvj;eYRI=EweWvb3ewi=xz<EE2jW1rrFaLYEUlWaffB#j8MY&(xU?1<~fNf`d-
zdrmUqBJgM7dj;+9I7!xBBs*Xy;1*yzN!v*=@f0@~PdiS(MI8nwIaKeIT>)0vTF+u=
zJ&C{IK2t>0GU`s^XLgd3S|p_lX6F|r2fT4TwqJ2!R5j#8XrZUBo3kqE8$B0#xatuj
z15{LPPz6p_QAM3u<v&72N2>y6t7wE-Wr|5qsnpQ{Dt{_2_=K~sOgEd&>WONq@|g$(
zm2M;&J@0O0W~eH!0F{o_t0$<uCZHkmUdW@&jKV}ErwUYDdXcJB0V*y)WkRts3>AuB
za}<086{r<MJ-_*cQ+p|*)v-MzzT~d!>Ghr;|H)~Mct@+!D^h7wPljHh_oBw0IYz}4
zHMjX*&(282W<R_N^+{AbTS0$?%VNcLda*~hTk)?<(-&Mix(f&hRE2{Z8m0=LF8@vv
z!{1lr8h6xyn?Onf@Gn5+tEON`rB?hPZhT)6=be~NI3-mlC`uE=zV{Vx8gI+!fhw=S
z=x9}704Qgu&1%I<;+8)u{xfn{1Bjso7pT&+4?4sxmy%O_N6ZjJnKA;Tzw0^vC&d|R
z-oJq=#{j<lG~*y7P`yf>sq$0Pba3FMR_qjC_&^b+efTq0t5!TDV*3Y*P}3Ld7>w}*
z1O)&M9hfmbwW3A5_JN{;{y|K<sxYlQ<4sl3>R^>BTosw5QiZ9;Ss0NAHJH&rd5TgK
z3e2jpiY<D|Dt>yrLZwnX@Ljc8t?)`v@$w8#0}fT`6po)UjZkdm5>!5le~Q1osz{*Q
z#s9vl_!<4P_~bRk5<FFZ|C%C}{@C;FHAOrP`KM31BC<eTt1_#{s{$ZgKNY7|;4d~l
zR7?$JqioXK(A~dZq?9r_o&aT?;<DKEp<-0f<>%N?A&p?Q;v~=7sucJum=6`RwXJ*t
zK(xOqhW%L)ukYZ2s*aV}RVm)?ng3_S(=-tJ1@Wj%W);u)PGr?RLmz>^K!aN0Lsb(M
z8_cRGvnntW)zPBQ4TW)ZgxXK#9St}KGL}cL_>iN4s!7O?OhBK}uB=(CyrCG!?OfOM
z(hWtt7Y*+@^H;@B8rmjyeXU63S_j0xeyw-{&*=+pD=u>fZ;Byz6w%zlTRl_mC_+^z
zFa1^#h3EMjzE#A~aIuv^X0i8M#THs6rhljSCHGpF_|12Un<}C>uB2l0_liLAuiq=W
zy=)5nv0^lSMEm!OXfg47#VGObcNO{6(3ALsVl?HRZtq$2PpBB@7!dvLDcWEFesNFX
zub`Hm3qL6u6b+w1rwLi3qc9Xx@F%cj9y1vUDut8VFpfiZI?^fyekBzXhO~eo&1An;
zF{Ii-LKOe`H&Id6uJX@7nROg*Zswz$;O{IZ@c80~dTpZcKFH~7o2Zg~KiNb%4ap`;
zKKqC&RjZ$>W!vd7OHO<ku!{#7xl`v^(bSyfMikB5MmMAqFGR@6v&s0iY7?%u!MS2T
zRR(iwRQpZJC>pmvP3fRb`+u#B3_da=EEsqbfK2D*@iU^mklD8XD`jlr(N)MUvk|f|
zZpH{7AiYHb3GqG6&xis^diDv1i71|ON+<?O2gP)%jA;mlI8L6Di^wVh;8c_wz<+#1
z-uu7}Z~)%%b^{8s#zlmiynf}1)x=f&7VG~k`!&cq2}37bS;%@+vn)Blh&NWa0nUxG
zlyD-->wk2W%N)E|;0726FaOOAFb<mh-GI!2+(Xa$m5;h$9B`A_uQEnHoN@56n87Jy
zpodF2<+O&>^(&Xv5V?lP*gXvUV}>08Y{jCGWpzZ(y7)al*A2-0$|z#UvbAoMRU~9t
z)o`9M`^GF6`YOik&riAmnOV7QG`M}#0qb9p%6=_k{YSF?!L0(^28&A-%JAi>av_na
zIrh2%nHob@GU0<;mX`;kV1rS}gNb7GWg*K^&9bh+^iOsJ@?d0?{o)Yo=&-9o?r84`
zHz0Q;W1U^U@)0G*wUWtLDa+W*IvNaFijG*;)zS9jZb0scm8$~9B}!%Zj8<2L+|k*6
zZb0rx#<JkbahQ_a5mP7SAk*Xag=40KK#x_gioXS=hF~T1nG<>ZRv2~cw`jm`WpE(-
zz2Au6s=#neL%{4Q(RgS~kD39z@YIwTtQc}2p3w~Q4$O^UG`~UBj-9!YtV&kF#UJKI
z$pD+xXPR@(aw9gY<Nuh;$Z>d)RH4G3xQb$(N;#IE5j#}MtgzbL2{NV@n4pm;13Cbm
zmEuE{GE&`-yhAC*s+D7X?#uby*|?Zf-xn9Dm2nMT%qZZ0w_0P&&4bu9W3imKug)!m
zi>4OW*G^0#xkW7B&)6SrEy<k>_Ibt^*iVsrW6SLM!?~%CIQG%yrZM0xC94^eJ4FWg
zko7gSBzLL|Fb=iTbJNBDsFm8FskzhS#`wDo5|NmjAxlq;&?qyP^ISX%HI=zDfWiAS
zTgorU&1LAH;&m0#c;T;hWg((g9*g1;93#UT#<?}dkXy<)zXAf_Z-LyO%=sAcw;H7>
zJ~y`v^<)*}`Qoo~%Vh)=07!L_T%!Td{rUl3X=)geyI3yrM$w!lxl3eZS?UG3cDcqE
zHPPSaE|md2W1&a%&c&ag5U<s!Qa0tTlB*aC<*Rd_kO4OLIseS9k^v^;=lEPaPX<>C
z0G$Q7Yh*x~2I!SBxz#enc)VDWyH*C&0JcBKeNqM(!`hhKr(^(e2=Z2-h<B+m);B{|
zEW=dPnwUTs2R>VJpOvxLfG%&%eNG10fOKne>tw)(jbZI?6u6_o7=}>w{1|bnm(o=E
zOzsvLgJT%0W*Tz00mT1pK}gD<bGOS?tdFpg+$MDB^(GtT_3gPk<QgXKJRx_dT%!tE
zyKE`ReTj+lEynOO&Ejb<WpF?<@&U7RS<F-0j5`dnxkdcgOBpl&DE$qfb#1w=@)Xej
z3{SrHs2xNm&>mLcHH3Dj-P!>r*E|!D`*YCpo({Uu<JTTB+47AzM+<Gp%Y9ugV{$Lq
zoqJs7i1B(hF}G9him`X}nz%=+9N+M!f%QU<g5o^6mxts@EcTmJmlq5&zU{!(s`89z
z=-bX@G4NELi2*GBMb3`O3zbPSommr>7cK(^0L#|oF>66q`2+Z!f1YeE4g>&LsLEq|
zELjx<pke9X^UN~D_SV;5&0~8sIS>LszdSESt}y{P_EBD}ToVc){pq|o8DJ*sz^S~E
zK=bXx|HwyT4&{xK+hhQ^{83)K3}gaOSo6lnKo)=#tMbOmz;vc3TmP9i4v_DF_^!8d
zT=qwKE=x2ARUH+1lhBjzP!RxcM;;zQqDMP`J)h?#%OsWoD7}$4S*}?Q;Po=mqEni!
zN1p)D7oRspMtd5-*=2by6H)`9Br$KATvH36?WMd78Q1_|%IkTVGPP#`96u$t>69bq
zM&{+n2rsav`|@T0;JJDLPy9YFR|Xsa7B0%0DK~8dP;bez%D^@NzZ3KDrVmnY0#Ny}
z=;s4lV91*#m+eAX-G#i_GO!!K=~wgS$iT}0E^o}6D+7A~EDg?M@qV(3>GiMQ&zmO$
zdjWKw%VW22WYs<ZrYG}?Wk6sxf6J2%7}o+|$uII2pvKqP%E7^Vy;D5pqcjH|Z9`S%
zw|TPddaDD#raj{4KFWZGffE31n!LrbW$glx@<!egP~f><0I9k>W~lV!yu5r4fO6Mk
zwv@qrGRzGxFW&(fj(P+S$*}rSd#4O*T=fZzwp>7m8|V!(JXc;Q*XW46ml*?WY&&*}
zM}5I_3$qg=8uRwbJjVcNnw_@~SY*{029)IqGB6fEL2zCR0N4J|`BL5iMy$hvnt99f
z4h8@?!<KsIzC3m}r@y%L5gQ#c8|Fvw85!0Qe-?0zVYvSfzOT#hJ7f|ezh9GR2IM^*
zCgtR>c#}718`z4Tbl0Rqg~5ld_4;3%bY=klN|FqELwffwgAbZr?N^%gtl1=BhLxC@
zRL~5V`NR+c1oLy|Yz;&BJY?C5RU{kXVk{m;_q`k0UZ`+6YbSfNweacnL(E25le4`U
zozLYR=~v=UdRVlNv-NV*Kp^}vINKmMW%i=8GTQ)$fUIKn;@B-Q-2kPYpB==oGs7`)
z#E^o)n!K_@@UGGjg8Ul4Y$IaLcz}RqyE-tNnHay>Xg=rn*|LRsi!u1RGh4PWvKvtG
zarOvCQ8w!h%d<zy0AsMUB6}2w`K>ZA`56Awh_g?F-}0$!0Qlv%oZO&FaN#!}Js(^<
zGX~R1=d}TA8y=I9@U(b73boH6zxCO8ezZIP{CDvN#^YQ5_ywoV@6=ApChY~@8UTmJ
z?<n%W|I8YIG5K8rd9sR;TzvvVGM-$dv@(n1cN_f8=(kQ`C!n`5uUKBQ^3%{ndm4D@
zA0`$g0?nVP?e*;i6J?pQp)bE#Fi8fO4!;~xkOU@TjciBnED(R;r!*zMT`*ZjVAglU
zX9a085CP!g)B@RGI>1bE?Gpu4<)&r;?;H^y_$fne{{kDo(dSRW6#E}zg83JCIe*+-
z7yu#xGl99TtuRoQFXQD;PZr8^d4Q?Lj-7?UtmXi76tr^+nS)DKF`eC7DL&<ong60t
z-i+R2Q<HzEFifUn0I>MOLKfx~4>Apk{=U$)GerT|F{3aNi&Q*eW)gb!xA{dld<GPQ
zHUBrjn&0mDSu$&X0q(lCSu$&%GS;HMpCz*<GjQ-vv%p#~8Kd<2zqtAixTucp|9h9K
ztcVB*EJa{NRFIAcC?IP?q^u~43o0vhVO<skq}bNpYb0C?c1?^W8l%{IjEOy#m+mt?
zdFkc7n3(eao^t{7`+q)qx!;*HXU?3NIdi7mSw!udlA)ojOS=Q)X2?BzNM6S487$i)
z@)K8wXRvIgq?-JP88mA8HuWM2#(Xk*oE8#%0SS5-5RmxV=xlBac)v7e^mux2hGdTW
z{4=T3y1fA&(_7@G9#XWe+AdS+9OOQ|VaRl0eUXY<-^_GneX+CcmPv(5@h6u_ekaon
zUyoHhM~YgyG}E2QXl#CNd#0jr>T34aGPzK#RDhQoGRYwjnyCrw8<*+DY5Z(dg1qaa
z^Lj(B2PlD?gn)fVMpr;!x6dfxSM(FN&oFWCjIQWN;Cql<^&VY;j^Oqg-V+=~FQY;}
z!n^gu(aX?c`Tgjf(aYPzUu_({92-8uBUe18!(B;5gAyXr?5G>M$0KFIzR@dS8QfFx
zUO9L4wssYkJs$m%mK<4x_i`Og+fG9Ad+0qb{~P@tSgLP|h`CiQJGo05<FDXw7=4ep
z?k)i7*wLZ``mp<90;2%@2&71QS|#uZ9U};_JM})ip5P<^&6}<7$w+hplDF%971SN*
zNS3_FT^eA^HbC6j`d;>6bsO)IZ|U0%Q7U$OKhg(c>BS=yB6ws&jqc-|_2ln(T!iX<
z`{@;Thg#&?9KGW1P{*BcK_5&tw2M0K>P!0m1g=pxY7+G!1lXFrxk>LH%E*yC;CeYx
zPs=r$DRR1p6yo%|egHF}7F5$&UgRP5wdwRUb<#|s=3@Vfo~KUNA_2B#>y@b!ja}iJ
z^=t}S<9T?30n$`(9r2W=;BU2FhkQKOLl32bekh9|lE&tIq#vdrkx+$xxPnwb2R_n|
zkb^Z+zd>Glc5_<A&In#;uV+7}RkR1Xdt5(SNumZ?vqPVzpe{hSw#myiQt+U=dc6|Z
z4S}_9>BlID>S)zB`YZ)e9hDgLV-@5Kbl6M&S|fS)+@v4RffDKL#q(<YL@LXD6uY`v
z?&T@<7<`}VpvMV)9?>;Z9YE8Q+7h`U3|8Iq6IKYCy&$YlQ0r-0=G9fUdP@FIA3Ex3
zAIeLIlsT^Q7Ej3wxtf@Pni4dvR_kfn<W&c`N(K3@r{q0qlfLqqT(7AleKk>#Z-(4!
zI0|-+>K?&6Id`7;%yb{2pAAxPgOXj8m(&+^KaW!EYODJaFR72}ydbaglEUmSQompH
zj(pKe^241p{|3G9BK<nfLG*`2v32_O3ep0Z&dLGa($w;B{U#1n$J0vR{BiotXxiQe
zG;N>mD3eddMLqdAzKg*9!BYKJyd!sjmiP7B0F)W-;Y|H@1~hw&QR{aAIQ_tb!Xo`n
zWZ=6<{@Ppe9~i9P%h{2|TDVsK0t1r##n1K1L`VzJ`*;0I6w^#@+MbW(WFHK15A_F`
zIuv2&>+~-(phk{<b(pECk;^aXkFX@M1oq0Sk+l3x{i_J`ZK8?p<pTXNPMQdC?`3@}
z1FF<1KKc_3MgSCg=}$691-K~cPcg^<*j1}P%^(Y)Yqb6hgYf{1vgPhQrA#qQp3+nD
z^7&kU6;Zy68W2^qK!1(t<^i-;$?HKNj*}1cMDI<|-{Po^h?@CWf15!Iz`-8+I}COJ
zyb+|o3*ftFPkVFmZQ|zA;RE|B&Zpv3qdEC1&Sz(8n^&Tp^$(x}KPsF2r~V=FWxsEV
zEl>Y1M${!1?$p1{>Lat%BIw^?z-t0i^mG6~D5YAxR;z!H0S#*Q+4}ccOlp=}{-^(d
zV`#u%Z_<Crpc}weC;cM;ryrb1pi{p)>pueYv)D9c#npoTV@%n7j|m_=^}GHP7<4}i
zPlJUzyv|;Q;XsI>gtqg}`Y$QrXCU=#(SOBRQ#wPq{yzkEk?bd4(tpialeoJR^xrTb
zan&32-!d7AtKO`a^?p)U^yKfE5!>7``D*=7NZ=o=^yC?SQqcHK`u{NnTbPucYW;6S
zF&0d6T=l<G)HRX<-TG-8T`UgfJW%6;93niaC!?SjMUE5&z!MSjkABkN&`o-=^Rpy(
z-QDyex#HT97?@BF(E$Kyfp#az6aA%;ww-#>nNsX>Kp?6~WUpeEBM|ZzU74G+00Fv(
zLjdw0v{i_Nq@oA@Lew@=4WU+vNarp9sW*|n(I%_UGg+rR)Qh~5&?<_0OcJLKlKr(<
z#zb!Za8<N#NaWfJ*TCn{SKvcFy!Hv0V+ay^Gevurfo=34jb9E@8ZLYF1}RAwL|Jx7
z7yOLW%MGNVL^`ESx-gL1gjP{FBvus^3t~8x#9wSHSj4lWqc%J0#b`>}jK~m^UgRN^
zbD8#zUX0;lN^J=TazZs}WT%Wg<0;{#Ht~2g?GSIU1EzIUbhsY3{q*8c%$a<&u;y;x
zBh7p8QV@}TOD_&X@?N(AHr&UJLk!TA*L@_>cjQ6<Ii(XO+j+zD3FuF=HKErpG;ay$
zgBc4yz!2)~^$Q*WZlUPUrPvxpfbAFB?m7AZL8ke3779{A?n3;S-i~(SD#~CLvb1iu
z6IZ(vfuO6YtrqOWHJG>rtdevk)?z~N%>qD;EZn|NQKyk#9dI0y-T5}non~}wF8ns8
zd7yK;6iI?vo#t$Vj}>sYhSu5AT#<?J2%~7gdg`e4UwWpIH;jo2Opb4wBTbWVJpnph
zSz0#;;<}eQYFb{J6R4C~L-o3}?o=7g4p@r~IPOe*ELYWO!Ndm_ISA2BY3w$!EXjYR
z$*loW-!RWKE0_zVa1(2q(yU|_>0S0ntAt8ju7b-^*>jAVxcnk_=`D>EbL26-CGF&o
z+$f90d+}q)`wk377A>d8xFO8tR}%H*t7F_T`v?rCf#&X*F|__Bl*W*fdd~&b;^%c)
zG@tI{rKv74;m!h%r<B$e@{Qh-m($-_qy%Nj)wV$XskbyBNSCEJ{$znlzR04ig;JWP
zoG;13;TQm#672JlGXpU`RAni?Jgw*Tot#BIRGG&V*JaTZs*jgqD53d2!8|0ep>#Om
zmPLzteQ5j%SeZp@3(g-2n)2%DLKZLS@#6GqOI85a3|sFZS&E-a*4w-+izeKzLaVco
z;+z$Ns_xT@@SJY4)CY1|vm&U*TiIkTzdkREI&z;Un$q$bAu9;xJTROTkvcbvs?X`7
z4k|(nE%`ZJoQjyuFK0!<!|wBB9t~-I8c;m-GeaMLSeq3C+P)J(I?I|hkb5a{ys;-M
zwhJAh<E4O{Zdv60D$a9%DT}?|R!1c>>aTq#D*9=~tBLhvN6SQ0ZK9BsNEy(yJoRA~
zE&cWVnldP^%A%z|wTe8G{tj8m6!TO|F?CrwfPP(LRLFnvD_KK1N=H$%vxadZas{^M
zWzpK8+SLHC*C~tE2K)I;C9(xsv^Ll;WCxGqsI<OMSty@&+>cgUgoqv;*oSe~4jjVR
znA{PNpaf7;-sGH>l}h~2Bd?azS)&o9G-b`ZS!t-0e&?wv*Bi3Z30#R$sw;!Q8}XW=
ziee!vlfW&xb3e&H?RHicP{68GB+?kOcs)8fgW7+eU_AQb=oqw0uob!2F4K;ckb4j<
zt-i8&EDviuH@{XOSN21r=|5H(&1eaFU)oruZ_pC-iPy(Eay7aEoIX9a8`ub?WIHy`
z8LL<bT6C7}#wr$q7M))>I+lmQ)}B=Klvf6fqy8CmU1OPH6~@tGgxVDizE~K?Q{zW!
zmFyNIMI~Qn-I9?GdXH7-Pw7jBTgGXTCYW5uy$i>2AK|5^+`@6Z>&ESEJ1#p2LP>Uv
zqd8x{5J#bqj8i9niB0iMLZzLwgzU^QXxOfM!Ly9>QSr08Fj5d!tm@PYRyBAWr*foJ
z)tv%xf&%7TJqHE7*-6|5y>|MRf?jz70Ui4|XOqJEliiF>%gc5IfMWnO%bKkyOyoU}
zY$sMFd30BsvS~$7@!PNgVhoo2ZT%&O_QuoSjb*|Op{TLh&e^23{y9hp{(LU4oxfs)
zXcBdGSvDCVTq3QZfj8TSl7J&PT>4X;?MHAf==P;$^Cl$g@U?_&-h_m&;f5J{`Mb{9
zWH0)^r(_j`_x7U5KK-S^rSB=Lzvv+S|HC47p$zUVaLuMgvXBa-i~KSB*&><qjcgj@
z1>9A{`WV1~RAAkntqk$B{nGSdwz5e;zTmmu@|phd#0~Pp{y0nUiKeQ!R>-CvuE=uq
zTsC!Zh3CLy+0?<EejuJ;qb&YYT?~AfO&#2~i5%s9N3%!5g$VhC4C>+6vQr5B><oKU
zCuENT2({Ca@~QcS{LCfeb#1rr2UA>I3uz+=?ZMcNioRcV&Zg~v(Ai*&+EI1`l-ie%
zgh+jH_2Jtgl7H{Ex)AsIWYa7-tS{)xf6F$oh(tfVR`v>&2HTown3>r>J0CGZDLLus
z^Rj6j-?xbdo591fr!sH>s5_ipK#i)I`sbwZ>_U$5&{S7d2-!s(<0YiRCU(lMK(h^t
zrbpZ?q)5g~Ivg3H6%TZVLRh4)W4wBF%Atn+Df+^zZ^d|{FSzdIrv4ZNuA5?fTo`=-
zO`tM_u;p!<LEo6xbG&#0PH0#oliY-_b6_vd4!KgEpU6RBI4RMBS4F!9AsBm2g`GtW
zrnX_9P?Doxg3C*>oy8ht8`i=>j}a99X=kzSS<pMsdhG306KI4Fx1!w9zf4fW#|y^~
zWnz~6a6%VUXLtiOf#oYEIB=g^1aR!l3AU~nRKs^JBWEr=6mo@!wpD5PGA@WsXwyyj
zPN*fk1`l4P#zFv5FN0dfb=5_|e-WCwl-Ch*e~K=eJsl2t6Cqdb=mtCsc>^IqvJMwN
z*sy=dL>Cbh;IGp?BRY$alx4c1&qB^1<eXJE>{-ZZgmn8tH~d-1DTJi8>P9>ZIf=1*
ze@|Ts^rd!%^5TEf@mWl+6`SgneF*EiN~a2XmSpc-wNBOF38UL#T}{b!L8l7g4k73)
zC03wNAgn0Bt+?4&rwSv~%w`~BCemDB#kmWeu_8HJrwT`PMjVD%u7Lr5XbsBIr&snM
zTTPuUKuqd{uvS*W4p6;5CC7t8k~yRWA<Gx%1U(Dch>*T3a)Kxmg{BE1Td(K5`i%E;
z2$?o6=T#W6w%pbuw_dAr#Lmx%)=`V~ZH9SbPc-6hI&q~kPV9>C$b7uNwo3jpLh=k-
zh356@7;OYi_M)i6-KmJ+(UA@2suv=7bmT#2(jj?Rq|`U+?FcsJ3&}B=WsOi)mVMa_
zN4S8E(+_@RD%J!zN5Cr-qK3<xBBfBTfCyjcGRi3$-Tg^~KQ*dWyq`ZNKZ=z6Y__)3
zU+)y65l+*VIkJkDnI%IP=qKug0oxV}474K@?TPm#t%j~x0*+otS-m*H%@BZ`qiGO-
z;V(mP4~_vi{?rf%0++jzrmCo1F!X_Doj)S0IBbTYFB*kQ5xJQUmX0;hNFbCt12sJ~
zC_{k@^_-{^=nGLNz(v2@(Lm01^js4{o~QozV5vbF^CV44m0(bO1hT*r2gx_1FhYN7
zP~2<>VioNRR?)s-)s)}UKz>5>Uz)NRg@Qo?i3iLhj`fcXo-97~=$MfPFYeJK@%0-9
z@7^GLN)pfd!{E~w1Pe*v*V7F=FHR=AnDn<n*~cKiC2Qg!1C513sTV0Kt4pAP=C1?h
zX(}OZ7~HCu>y&HVY(qGKJCrN*49^dGrd%CY8zNbQl<UEz@}+3WYr;$e&t7@azhsa>
z8QG{k%zeqA_{dtaAklvzrSttRhW(W1Q^XJc+Himh!~=TE1OqRp@n|`zP>vXYxyk^;
z5hfrd?)pm386fp5SPVA%&vh}hf=$dqg$*y0wX@-6GG=pfgMnAZ_;l3na|T`+<Fh7_
z4u-SLnN8(4hI1X+?0H#!gsg3{;Ub6e{4<91rE_sM45Cc0Bh!^P41>{PV!kAW&H2cX
zz<{`gOfn=gAcYM%X-Gojj`_=xR1mX_^subW&c=KR6FXoY9*T*Lc|cR_n7=qQ?pg4+
z$orI5ZJ2|+2aW^N+FHY0yeoPe@VCLnfV>Q=Bp-^Ayu$Ao7Lf2TRcL4f?ZCiL>S|a6
zx>z5OLGf$3a3p-o1o@X3X@Iyv4jPCddzw6EARG{lTu$gUdEG$Cy+^&0=(T?)I*E8&
zx#3ODhzdUIBg1tj;03-dgWPhdQODAf_bpp4xp9ib1NY@BejG_$H8+<VHLrgcUd;6-
z4s2nL1m!BrJ*3B1(sQ{{KjOLFlCMYP_6EPWup-0-mgOq@uvD7!mAT43EJ?IrVXm?d
zODtAAC$ETw$Y161gq@pf{=;0_=ZGt!=34MeuCmRc1K9CyE-mN8)r?@jw3S}T5M$?q
ztSR9IWrV3^7~dX|OO8oAwbSTda@j9oo7r$2zvJ9qqJ}fMMP1~GIH|Yo{aj_8p8DuA
z$J{ce@zfN~C>3(c0pcA<qL=UIni)h=J;p6)%Q@bgC6R>lXfHQT=ybU&Df4NR`473P
zm>21y>iyi+3`iHqWeo#0a<L&BY60;M)RB?RItE&rh7MLa3%Q4g;ZnrSd@q+4w&Ry#
zR8z_wzi}#+8|TYW^5Shh^QdMAQM0;b%j0Pccf^7F^0;ETBUa7L<BH{4e)&mWU!`VA
zuETbD$}%3+(P5`NUlx+YIuaz;#Y_Es!V>dniazKh!UD_kl#MB+uH}>Ql2%+KzZ;J|
zvWZ#{+|cJK)`%vRht}jN)+h$x`i(rr8pV^~FDE$XQD+|XJEC^Hn>U!VC9b>g<Rvg5
z>6%0H5*d(mC=#FJQx+ID?v?V$7avU9pr-fpVw5c@-m2MgQ=6AgaVrt`dhfi94(<+D
z@-jQPTUN;z2I1_&*1RkZBlG?8xIASyiJHbnpS*D#L+0Cbw0W@Pu6<8aRNElrQ5*Bc
zZ~S10t;ti`7|FQZO&&N{(z>tbb)=$W>b!Eukw7zqvMI88u;krkxi+to!s6uBgQY%w
z;yPM%!YFQ6JoB3Vs&if)Q7u<ezCBoqv3>5GM}{Zi28ZR@STqvB`Hwu_2;z>rd_~@T
zj%m{wn<7PLv;S>BiH5+$en>Ppdr}XgYqQ&vXnjwa*u3UD$(;d-edg{Y4+7106M&A~
zouuI?>S~kSCn*h%EzTP=q)9ZMC5}PV%e^L1|5erqXJUT9fOoRYcbzBG0(2s6dL7v&
z|2qNZel~uE7393(S?4-g8E74;+a-=cb6M=_Jh>00d56<XR?G?&JtcT@KTb)_WnIu@
zUN2@gQ$|e=<`_rnR@vF2FqzhElBlb+JejP61C->$Ev|m@_@U%lw$4M;rmK^OF<1nU
z|8Vkfq6=Bp-u;r4#g>kfNlD6DOFcE}X1jHhQ;@|F6O!V9?2SZ>p%*8QW_DB?rE}z;
z5+xn3P8^T~lYdk;CBevlCa+17#@kHe+w&Mw&keze<aRG|7N(3QrW?Ru=6h4pP$n`5
z$j{?d=P6c-dWfhEFHNcB;!=^~{xzklLo3T1r%Y!+BCKhhQmwdJ#KGl+dddvq@Fy)_
zfkj;h3%}2%)GI8g7KS_|^K&1baQi7(1SYG3F|uvxBKjaQ*_E8c#aaZJM|Tnb#Xcp1
z?9Ak{ta_>qrr{{rrkPPwC<JSeBI*Q*y4n)VRd3^o<RbfEFLxrN9yi!N*c<!($&2`y
z(2$21y@%9;+3lBm29sNr{I+I#nGo!Updk;5*M*V6{*?M{Dye#GuofWNmy8*5=|!b!
zCcdxr3nmX$DD@_<V9170Y%1z(!3H045!lsj0L#6>o(?on>O5Fv2d@s1_pEgAhVQHM
zaHI4|2_T++Hq?Pc($#57iUkJ}DRB&mq#o-)B1QX>aX~^YC#2CO?6yNMt_dwb^*WnF
z0122JsHrWk!L`*$q^k>}3=*3mV3#)>a-Ydp2L7vm3hYY#NcRED*VUs#zAZz#YABlz
z$cF7%+0~Q4M^y6KPr8zNb)S;}L!-L#sF2M5p)Ix^dBxtxBW?lM7dA}pN+lXPh?Lxw
zN))$1K(Y%XyZXZ^9y(|QOWM_6OCS|7VYj<#xkMQN>aksWalx~w4DsG5!^lfr1F2ci
zY|73YijZD^bggJBl?`)$1Ey0O3+=n3B+(1GBE}<RWQD!+vydzZ(kWVPPm4A~&mf~o
zKK8MASL|5+Lwgw1Haj-;NBejHS{_59@ilh#<RuLK28j;#u`f^(X&MB3TDy=Z4V5(E
ze{jz$q8Zv-Gwff4v_rqa`z^KoNxVBfPEp(c7u7UuBxfw|7%F*r{%DU+5jhvL+Wr?z
zkcLgg`ou6ChDNNd)<one1vaOw_J6{cA2yYXmI|3JZqkL(YX7h|z^~<W*t3m>x5C5R
zdFnv}X?A6pyFck9hF3Df*dQnKKpO7kA4ZLNcogWSbO}?I>#3d&{1pbjOfDZL1!T<%
zQ)X1Oq;onlOqo&94AbftMvL)bjeLfD(%o)h;l!*4QL7$>MNo@mM`}{Hut)}EW?$Mb
zzdua!bKOl#sp0)m<?YIsyAGGUBi|1XL)h?|l%C<lVFWe9z@_1l9fVE8!lM|F3;L!}
zo-ka}j&hYWMa6|e_&~&jHBzTdOb(9)7~$5FEhCgVBCu^gAQdz@;c<u^emKQ1d=O<b
z3rVmTdv>_wrGFSslK`i?b_kOT!jk~7i3HGaD|`s?oP{%(EVnRXCD*x&0G{_#!;>jC
zGQ}=@DDk8QcKE2=dxX@>CZ{-tryy`7jeSG4;iHHtkDmDt!&4bhc}fDqM-ymPab<<4
z0jP006YS?-38ya~MqD%dg^xi)8JTXNmAmjP%$`RC;(h;)@Uis167M-V;p6Z=vJ>7@
zRpHq-iZ~;r1cZ|ZGqQ}g$;0(%MZ<RsXk;T@IFIp};S-R@$aF*utWOP}i25A)A>?W~
z9d1CFviM&&J3N=?nAE%=K_0g?^3&q}6`rT1wiXZIzcD;7m_VWm%XWj)!t(|Y(kUgI
z1UYc&0(Gu6x11d>Bc~KPu$PwScm?l@ZU*k-FX&VJ<@qBe&ws8or$l&yD1!ITr}a=f
zzDeoRn6)F#WSn;JDCf06?Rbl#Y+&6#*6}vyN-aG9q2nC})F{`y<9HYLCIz!GRBTEF
z{;zFoEQJ_GO9sYyq1#~@3P1cH#O!kzm_|JBX^P5fgn{Wu(%X}GcGeEeVA*L1xv_X)
zCW9`_b1M>j%rdK~JJ+@jG*FKBs5>8d4<U;!121>1`y8Rw!kmka-8O;6s3Tx^_U~?9
zo0%Ww<S?|`76#O4uoHMB1)hPA{9}sbHB#4YkCHE<O1|i}mjN~3^CjJ0V9=HJqp$s{
z?k18=8}&Yx$YkGcVu7L@AS~^V8HEk`AQa6d6C{7J<SH7?M|~Fz$IjBt?q{rFjQWe@
zU-T)Y4Jq=>r#*z!bYvB2$nQqnc)Mo#ihL&qq?QLu^Sg6CypNWOrjt5RIsICnov%T&
zN==W304wuxiWhDL9pgQhs*A?lO-NO?p`v*i<5z(ff}*1!ne#`~5?5QI=`25fKpWu7
z_Go3zKAzh|G;RI~rGu!s8E5JSBms^lgVu6-zz~F?VKch{bXQqBy8({6fAEuMq)GvK
zWAS-+;McfrL0JBWi09z<RCs#-V25Wyj)do|2c7WT-nlcL+0OQO=ATsyDLq4IA~CuJ
zdEN*Lq0gKY<9P0Ah~61nKV9V?Q>CbWUm*VY=8*9n@h{&BnZQ7?a3MJ$4r`Q0kH!a5
zfALyf$Rq@f{!5wU!Oo>Qab>v+W^G713`JU^0qt#J$T;-u(PQ!Mx-=x0-dk!+SSSsd
zOz&TiW+HU)AvCc{`;GVRLI$(asJ-o77?Qw%+FQdHA+*&g$lr{XMveP0WCUx8DmUbl
zkdX|?0?)FBj3UtN3}CtulFESg>dq#FjAk<WJdMOnaSkbhr0HW3iKE1e(=gXR6k<XL
zQI3Q)4GSp)Q1(QYuLvopm`7^zaedZE>JVzi=}VEgdR<5X+OMM8UIihA3?z*OZc&IC
zgK_#&?&4{`;Z<fmB~!^&XLKPnu}c3LOj=HdRJld9Tf2yikm(pd(|=alSV%QXPHpV?
z>mfB<CbD)Ads4dOJvb5EjydY0Xqe0x#oVHT|Ixyt`Xkr&78a#Uh?NE_A5NEideuio
z!V0KWya9OR0a6D1<QX*(ZcrwvZl$06bvm5{h#ExR5U-9|cFV3A_}(rqDw)D~0^_?d
zN=KlXYNC07T#+G#>n5?WISZbP-*Jo5L&D74%rj~XGozZ0OpMB6KsCL-Yt&c<G?_y-
z?<1SxaZwW~jI(JT5M?01*~}UomCJyf1Ivx5JSO9F7Y>DyQB%-N_0J*d#3DIA6Nm8r
zj+#moZDoimU_gWJMMG2}1G3F`W<(V+ppo<DMEM|++Tx;0DXdkk$(W!QqD%m4S1M}t
z!l*KianlSRF-(Xm2MB8<_qFNgC^N?d0j#_iHH|?i4XsyqL|NcTX5NNoHmcQ8bYrOg
zb&#Enh*D-hq?u(UQA*uQf(?oK7Dv?}k$&4@tmMqYQbr9<chd=|SyrQ13l2DJrYkHG
zjN%M{F&a@<QCSRB$5HkmeD17Ktfx5mS1`16IBiTnZj+rDV;C9}*M=#Rm)s)7nF!nb
zzN1)N1y-}r4fOZqJ9=qIXsuCPNOmGFoQ(9CxXnl~Chh|~U*Cl*Ta4loRQK3f$hK;^
zJZOv*Y%>_e)o4~i1fBG9`^hM-WkjEN7}pxbb)6BH>;|-Imr+~~G<KFJ(CJY|@j0dp
zq__h{aRbGz3?lS`QN+TfFm{VBi#`!Eikk=>PziwUsYbB{P)N4R0vvL{A3p-7;PJqg
zuZ`k11sVW*cpJs-oQ>UNz_l+M#od5o&)7``occf6KT8VrZ^0!53ehTrUVG6f9#Y^s
zK<i=IoF#>&hKsL~F6?$3fi|Q|;xR^d2`%d{9w#J`fY+8A#gioO4Ji`P?qj2PhO&IW
zGsUh<k^h|~dDy=0tkhOAVifOBIV4GV%Rg$5@Bzk^T$_Z?FwSnT-LW?qD_<f}{QHb+
zI^w@(tXTF4#6R_<F)?Bejo%~w7ibi3qEg2#gXU+18O2-9m>1jVWZX<gqj;MUZGVY(
zjp7|fxU2^|g+}o%BdYi31{uY-7*V|+TyGTbF`{};oMII3GopH*-^D0Cpvx|!Y;@%P
zrkhcG$dOu-rk{&s6yJd~*?u@t*E`lIeuy2lD4RrQ6BqY0ijNd10Xm*BivMP4rv~&G
zY!n|eRCNWs-E0&;VkmY4ytdFNeyqeg0iLy~jN&IsVrSq&lTrMXp+gVA<B3M`GbPrS
z#IhP7)|Vt!Gh$UMjN(_AL`B&gNvs(t?Y9bRN5G<Fqxc;|k#bTWGm76UusdKdg#H20
z7G<-ejHTH|@kfsCL>bT6Z4`fEsP?3c)8A__ivLB%6GlT>gYFr{Us+jHJ88cd#U~zA
zaa231FOA}F97naY9lh&!MpQdDB8=iwMpQdTQS?7-45@nDZy3eD7*X}SxXvj4%}l9w
z^7TfQz;Rk2`DdewZtaS)QJ0;w&8Si_qAok=f>CA1C=}>USEC9)iwVj|pfv-Gs?M;9
z*?#!B8=>DUqsrbH?Jvp}4;V4ssKVV&K#4?ZHmYzbKadV+GvXW=jR1P<s!`RIQ7TZc
zOGcF=qYNOw+eTG4Mp-~P{fw&ajK%}G^)jlQIeP=pwg98bMQMMN0sSpTl`E4@1@eB+
zsOrI}7$_*gsB&Xe1~l|9qspC;1?Uhm^k7s4)Of_G(lDwO$i!|vnx%qK&mpa?TV+)B
zRO~VkI+)5A26mWGC+cdd3IwCdkL+!o3Xu<%8dd&`?9dj*LT)XaAxR5(*JxDrV%Q@P
z@UTvfm>~7<zT2qkgPesrI(A!Vmdhqc1A1wd(`D%EIg#A=ov1N+eIOZCVQ5S__43&X
zQn(GZ6vb)nDBAXevr$D3Zq9kI68~jXk-LEH4WQo0JXVnf$mVXuUEai;+M2P8Q5Da~
zk=VR-$kC`8Pi%fgq!oEgV9gL4|7%7Sd7kLd#HK9LsKP-wKt9^K<)YfCDn-hPbche*
zpUMP)Gbud&smeH}ZTwT2IjZfH3X+h=Ektx8396oGjjAf5HiJ*ZQKM=)R-bZyB?&NC
zs;W^C8Z1jetNRRj(kl1F=XGUV9W<0G=I?fkQPrSG+iprH?&WNhrPqE)b^QOsR~g$2
z7ewg#1crM<wyd8h)fRjx))iP#+m{ucBJ0ruK+=k=4Q23OCtgLfLDRYPmI$r9j?fQv
zrfMX(h6EFz7j#7g-e53qn^;5WO`x%7aAXk9=0*HCrWBfv(%ix%-H<JYGP^5PDuYpV
z7ay37SJ=IU!G7XiVVlVO?lBSpnc;nc$9b3q!_8o51H-qOp=vx$I43wd^@oeCoMAX}
z65Z<7Ky7zw`?)wYFLbAMRLvAjvJes5-3z8Hw-iLjJao>~U2^w;$eEV!bN49Y(IdPn
z9|hP`qGK9oPfD~Bvp1)yQ@hwas5THW{fDVDo#=EDx)M$Ia=&cT{3}pe>O?<Rgy+6P
zp6Q{Jg$OC~!$nJkj)bD+YyF*2i0~Q?IS7FUn%4PKgUTC@I=bJ<zZ@)aX*fXGZGSVS
zGA>1{$fLJaf8Vn}O@!MKH@&5x3%lw>xO_!{10od1bLh_nbTUG%BF8iTVF68H^S0Xq
zgnAe79F~`Tmu@ZKIc&2xx2RGO9p-Ydd+U13vw)W3CY36AVLJr5@#5FMqjE+b_K8dd
zy!X*e)1<?P3V7L?7nQ~)6?m~U<i6&8F0Tf)ZCe3vtnt*zu)csd)|$!xf3UPbIR@Yc
zu=&*je@^NS0NJ%%L=P(d!5VD=ZL?0!LKI5>Y?pQ4lmglf!Nf{K=VS`{G9c%ALq!2^
zhctJV`%J=zKR?JLCgDS&8wF%~ln?!@zLpy%Ny)Ze1-w(o-v_R8E8v~FW<Of_^cy;a
zj-Ct2cz+>jh&SvDzwg>P#JfMeUqrRL%R|Kdoi5?cZs-v40K{C09+5X3f)^ffE2L_h
z{0$3U$koQ0yi-xARGXxdV<%(!yOU}u|6!pM$W#1z5wg#9XP0Scu2aGh<^9mbf(8+=
ziYmRbpGyU)hqt-vMHedrs*Srkmr4eX)Q<aHcd0@<o=o>gZu`om8kIaHgInJ|h&TmJ
zP4KaEsX-BKZEMki2y#2+@*73ZV9Gss4OoTX<Zdp%gR+f~wl`WrUBnyDf_fsT&m<S|
zW_yt8&<=Q@%a*x_cR2|0_r7ou-+C6`f}r|yF5<o3&%!n$P1bD}@jeG#5Y;Z?pXluQ
zedJkF(3*B0g{E6hs{|~s;LKbsoX4#_UBsu*TYfcc>w~2(;vZc|PFf3`@)Ozk6$@Qd
zDh?w@q}t%3vcmx9_8B=M2T!`FBv|I|kKF`FlhMsZ)fv6iS;M7oLWg$y6|A4<4$?~q
z83?c#g#9<Vs9u5pmH#{uF1iRo60+1Tc19B99Tr3$l8<$#L%@oe=poD)HZn(Wu`7aw
z%O^8oS3Vo8o=bMID{fbSU3mk7rhe>VSMn^V7D2D*5JXb&u0eU#9QbkBF49OcgnTUP
z(7FnEV9$weMO`3+@iR1s=`BSzhb|y!Yjj0j2{hAGbWCcIBg7DQb_Os`C}Q)_O0(VG
zdy1SmhWhB(E=9b=)lA2V7lao%GXvT+*%nsh!esOTb>-k9R|b6n-uh2b4+hk53-ccq
zxv@O4?bgQlUsmRawB#|phbR?k5MP)H;)xDLyyMwSt$fk_BIQUwP1vI@6wzr~<#U1S
z&P9qzp@MAtpM2Vg@BDU>&3VpQ{>~^3@L5)*m;xV!r5`K8IE$ntHNFZ=oQiWL!{nT)
z(g3Gdi)b-UDD91?yYI^KR7n@qqlmUtgi`wE;nIL2WlyC)!1c*R%AQIHz)pMl@2S|d
z>>~FqfMGdUq~k<`iP1eduRzk-9E(yoOh=4Di$*aR3Q%^xh!>DrhXFu8qY+b>nbNMG
z!YQnua~)Xu8nqvpByazrMs63|#d53QS)7QJ_cT>PaT3{^d!#qtSBr<B*$TD@R!?#<
zU(k_E=D}Q0%*%kuq^k*&i|Jrv;peob6?VIr&NR9d@m8KtHwz3fi!1y*fdXmgs_^YC
z=y^Asq~bKlQRHesv-zo*4$Bwi;C<(bVmd2dR6w<Qbhyi+VjFdZB04eM+qrlu+D#GP
zfirz=F)hv(eJ}zs$Ho*Fq7xN;vuGT0a4If>;6;Ckizve)SGp-+3!U{YnuO9tc=s$G
zLo9D<ii?C|KB|yR1zsFkJdR5LHI=t-b1{8-T==zG))YwzDIc&d6;~7mD*A?Tch+qk
zu(HwhH{%E#B<VI4C2u=blD^Qfn3krCb@Ij{DOy}9-zbvOZB(Dq=Hde=MTt(xshm+O
z6dxoowzCe!a2{TKh)|&;p-!R2hq-FH6S8hEJ_1y%>plV~yrlRQ7^0Gf?gpR<zZV}R
zw5<CwAm<UquR=2=8@oRNf~LtoDS6rXNsbK_UwoWuq;aAeaV<je3FIdvPt*b}+*y2r
zy!YgZfm|k(xnwMtNnG0zP^bjhIszJ4E6OSSWD2FJN69h{<%^z*;3br7WNcGzeuDB!
zURDVB$;qhxO3pt^M(aQ&x7*@j*ze^Pt8RD@P1l*_CZX7ljun=iM=EM<B~O^rUI1l%
zYq4E#65>nfOSQ$Vq#G)tbUaiaeBWF`%fh8>B|3$c(D%57_o(t0?kr&g#f`m1kk6Lj
z&b4Ow-z7N9xxPd>5>M9T=J=9mu%nv-p~uqkO=xJVuOlF$)E;@4UWmj}(+XM{FPTd#
z<9l0GG@f1+OS@6$D<%4^XPDkZ^#27-O%l=9WD$K$5v`=x)ba83L6wdfUDUuPYVldM
z8ElbxzwFx047SL8IB98Sh9lJ@uOa3?ln<6l-a&I}Gw1+M`DG-<oer7o_EMwoAKM0~
z+&E+LU8&?X<_?G9m^;Gu{=h)(O5J-wcpEjX9{Q&#G5i%z;pVm&En@CFWavqR)<A%j
zm*pH2EL^rc$Aned|7MJ5>1cuX>SKA2Ng6QFEhCr7u`4F*Nzce*K;5I?pp3RnF@VBJ
z8Iw6C6~N{oJC?zR)MOYrjB4~ij|{R8La70uBrJnBZ%T6kCLfbCLG3dyqnHUMBh2Hg
zj1mS@0CLpwd=NPOkiloy_`0LBqvd^Nu*r!T7N#o$U1_%bL79{nYt5*ou+}PaEWK+2
zGwPvT^GHxYyN&<Ut^(S$Yblq!%y*ceL%T%Kp<RW#O}iY^u3aTYOZnwI4$PQG`O!HQ
z<hS6T`6-|+zweQs?G6)k<VOS@`6<+G`Eg8peo740=FvVG3sG3}L&Xxb;`P%q6rRrt
zhTjRypyPVeY86>l|5FwQ_UvAwX{|A{FcjD!41o?|l(cQaa7?=}N=%zDYaq<DT16Pl
zxTY=a5N6wcT}Cqrqd3YL8@Y}INQP#<iGU)^wkdMhG|4OOg^U)apr$>`nz5CE8x?t8
z#x?*NL1QriogJMa(qL}c4NHQHN9Ih!#eP%dP59$`Cqs-=N`yFc7Dgx%(1LLJt!a|m
zG<@d<gdW0iLaP>NPS*@Eo(?iK2LiQh$q)zev8KL2`(MZq2NQbKpR&*Iosl69L-rNV
zArkd1j!={ggmSeZa!{e<EQ{nb0WuK@w2VNKiBP*t?Fndw3|&}>>yS`k-7X=0Xro97
zLvj(UCw`wHrm~=sNPh8hhDfKe@VOok6(Wt}bU;*ybV85#$SHk|Q?ZNfGH*~P{vc)2
zahQs?q3#Du<>(5j@9=Z>o|%CNw9ZH1&10E;0x1#IUwv(+;)zkiz7~<Gcw#g|o9C6u
zXVmzLoJ(iqEftd1=1P`p#W$IuVCM7#IiSxMWhzrPs@_+FGwE1{avj^=ahZzq=7E@s
zyiEG~3AZazM?2^4lNke8Nx4FvfgSR!yPK&vbQDwiQl{e2kv!L8GUKrXgi!K`JAIr<
zyN5z4&C8~Km`U4*VU6TN>E6jq;23J}Cnsj|_F*Y~g17xtChf)2aVi=hX0FD2glQNC
zh{_b0|E6O!K)iLI1_-A=X@KaOWQV8uBid@+orSGdeEYJL_FAXz<f}*Wrg)mj<EcCh
z?wi+`#==8WP7>xnG|?8Qa$QH|%cg7$`IX10OrZs)@f5>bT1(tabOr*Sq9Ud~(Uikv
z)N7)DGfm_eH`-#UOu;Z%dFs(95z6y4_B7=o*D5+mgIsA(wIkOl3}_5RuK57?;28k9
z8krqwXo;I_tCYOdiKap(katu{K0W9tO`i}G?`ZMJjh1=4Qql%I<x4`=)u~NZ%9Tbs
z<XS}oZ9AQF6*f^nGSwoi>awzBQHs~M%0R=`Y~2#cM3#Iy)g5wa{6}ijOa>%#<5d&R
zyMRh!&j&oFyLa$tdab94j+IVdsYo#IpFHL}+DM+Zf-iu>(xL8!MAxqFMGQ#YF>_3d
z8IZc6`z4%)S}}CLlw&+e_tV{>^QJG=rWKU;Rm4O7RTSS=dedqKa?Ete$42R)oelqt
zZ(`6U+VlSut(jt2w1Y!UG6OzKI{oT)v9?@Ln>J%QY*JXZ{FCL@4wevxFC^kgW1M_@
zy5wjpu`}%eLA9x)Dyr)gHG*v=e$9M0BKIpL+51nnFLbc&bjU=9%BuHw)I{~Ej+*e>
zt~Kq4HfogCWI9O6TFIQm%rhNwlyj>kXTwv9oEBs{%#k#=>pS0c1fWK(ZLi%LEcCUt
zklo)pnEpjPGc>gYCc*SJC1VZ0dDO&LE47jauiuj|RpU#8`3lvC|Dck;s>Y{~t$f_b
z^a)nhYcwPh`q-x|0+}xVKTV$j^nJ>Ac@%B+Gkr;UK2j2WMTs&f5$OKIbo_B>t--_e
zE$C{s3f*@d%<uO#ec!=+<5ANO@`@Um$y7zUKPigKFw1Jfrya;wCHxzvNB^8XHr`=;
z`0lFO4KOL{r(}H=eMQ#i+*z_d72afhMs+6Zqa95LJsKB~@!8Og4tkvUh?m2g&jgx8
za=mMB>gYNWlj!6l45LzDs93(jBz9NC18TmGOT)EP3SQ_xd$hYrq(Naur?!-Rl;U$j
zd0H)6^O{b6Cb2)!3~8eY`3H?j4DDdvZLCQQqY_SJ=5MX`Hi=~LXVkYPjQT$bqdOA5
z)nXC{0C}zFgz@ffCUG_;JfbVJae?M2+HNb)T=FHFY5bKInnXTYj*mSsnqSCtG>Ho+
z<@;?}F6>A+d74RFq$DKLm)`O)iAzCR*SU@BGL9sjt!Xuh%MnFYl>ypYMy-kOqNs~%
zW4l@@iLM&APng6tSV@`jz5}tH^I{j1xSrTHka)LE;&a5dmDtWV%6Df-K7Ci2#OFCL
zdrF}`8)XtVQi@w`%r~*jvP&Iqe*DTLZea#aU@+&PNo+x6T@+ncp_NSHR?gXn_zWHH
zV-k0OPklfepPe0LNBb9dF&k3a9WdSff0*v+V2VaB(h08m0RE)khI4U3eN-7WRLbP}
z(4Ho7AH^?e%jBhwOeD0e{r@Kuntcnc?UEnt$mHs0Ch-uM2&F7}#6?JcgqXH#;FWf!
zHi5Vv{Xbk^{XblfDO^dok+)DyC%~s6vQ4;??R=QcssF>~^#5UVrbD=OS4`qrj>Elc
zHevc9lX#A0^`|^e9Y?ddNO`n3>PsDYOxtV{FL&f|aF0p6(vinLbkWxsxe}X@3Us_{
z#O8dPsBaJ(?y&`7Ch<+q(V-g#G~!Ap429ONsJk$<s2iM4Ssp?4G~6m{uO6S`-X`%j
zq8r;aaHm5$QsmtZ>8K67)gc`<ta}|As9t9h?{l0FT`#vJ+$25#Pqm6Q5b|F;llV5}
zQ7)ftkb-RG9k)z3Hgrf*_iwb$4~dH0dgAzqIZ{7Z5@!<s?N3oWur!iYJKB+Q+D(;7
z{E<?A2--m>OyW<JvYAr85N{HH<|yS11WMmH{+o`ZIfszx{}4Zu&c#rszjkCw4f}~7
zMJbsc#cSV=lw*EZn^Y=FX(R#N<P(ij?><-@!$?hCV5YT$^hAneQqhjV%uOJj79{`O
zh%aq6b&QcS_jQnt3k9h=4BO26a?DJrm+!)!CKVa#S>up&dW%Wr1Q6CpMSF9PTs>0?
z7N3(_W=cM3<4h`S>Y&nTCQ(1ir1Hdq-mGzaxOnD1ycV{d7e!<0v>0rA5!)>w?mgP1
zYCBjB1o^~Sl6y29I8e@ipf#z2i1LRv%KjY+!Z580VMMbXIDV<K@bTGGM<K<{H2~w)
z>;#O7En8dzvEro+^!s0M?FZm;mAuRF6R!O^hKBq*`(490hMcUx*IXkRI13}OGBm_B
z3i}<ie&DtH*`XjVcWpv^=>=YFw;-gY+Vy#84SR<+T7NG?P{4TC=H3v3PTJd*7sC~9
zY^cD2u`IF5wFL#5{R$7uu3OQHXZFS6O5E+>x{V_`DslGZsbCr*nrQm5#l_KeFRbSr
z7l^o~(Del>I*-$bceuVtfYx|y!aWz)eT-<o&ikS3OK{KKKBE#(Fu3lAlyfjwfr~i%
zGJTObCp4k$!50R&T&1-uVLIaH(0BB6=(~qG;4mk>Z554T5>UP^F0KclmO1I9mPJ#8
zacU*e)$S4wj$qY|%RCjE33l6E?VcUlJ=BPx*I#$-w1gZ9{_TuI4Rx@C4X&NvpoFxc
zI`KQV9&|Qg&L#L+J?K*@ni;1e;`<&Rl;gfaEM$J(*SQDfe5jVtPkrgvP=KmJ!J$vG
zwk#W(oJ%V(+deGgD?|C9?elZWCQv@On*<HmN8T_;3QJt%N$O2_b5+@7N<_;j$B&gw
zAv$&sSIK4h3`mFicguLGoc&+xIr$H8usx(%+n9l6bjnWo&Z#h_OgUw@2r<?FDx*_&
zLhG{j-Z}R()+X2MjQjJ-EXdgQ5=bF!1(C9}A<bo01|%&+uVg@;|B=0AJeA?gRNu7u
zluc*;T13sVmQ^zd<OBa@v=JB97=)OT(lVOs2&Kx2-?AA1YS%&R^_A6e%uswjStFFy
zGZ=<6SGJcmFrdw3aeWz2fl5aJtU6XkQy|+*BL!QV&9R;0QVk|`>N`7@&7o3#2ikyn
zWpiQ4Y~PWWI;3xz4Yt(wGpTdf&t>xnJmD{q=Z-{^oBJ8HyRX1w^E<k-h2SwS1C%YN
z%N8L_IbgrqQnr|9fsu&0B5s#ov`Jd8jCg9iwo^1uUb6-zjY!ymjrI8h<;OOhOFKcH
z>1ikOOjo*)XPTN#p6UL7k!LzCgBDY7mUY4N&~yhpZF{=n*>Afeo)?#Q!!vZL6P|@D
zy5m{e$r;b*y<G4dr*Xw|;PM`LZe8Jq=S!LHc(xAnz_YVOgJ)cUJY$~Z+wGt~fa4u4
zo(G<jx6hM2`xlDZgbz^6`OPTIB2&VL*eRUfEaa5b)d~rZ01`%t!Z>K4S^j9Aq_O1`
z)d&e6Asn}{c_Q3#b;8F$E_X#Xwh5z<`~|KF-tlcIQ6+pv>~Dj8q#@yRykmp_s9u-w
z1p|?4tm&_eg#Q38c!|GwO86Q)m5<0GXC-{c+0i`FY+IJ_JtJ!L%SR;qz<{PoiA@PV
z65v~XkDg5U31I#Uy%4i8FyUtgv?aD>NW%YM^cK7nM4#p-{7UaHhia;-g@h;c{%Ry{
z%@PuRqxX|B__YHe;dgpJ7mvhl@7WUmBy^dUhogEX{6*lJ4!}Pu;co^bFdU4{O%Me_
zH&dxm#Bao7kPkgh;xX8r-hQXI_wblc$AJ=-(;<};lf}dY5oTs#0=NtbNf0Fxd?1}f
zQIAa!J9`60`-)oNvcU<WJs|2K6mVB3dF=v8lU*dn05v~M5FIEoS>zjF-97_P#fWyN
ztq4y|f=Fk#7SbKj^zGlmvbH*dg<WWP6|&qYPY^v3yRc0DZGn`M7M>vX0!*e;T>|2%
z4h4uRNZ@zVK0%aG;Fx8h=x}0aGrzM@oSKSq*~-1JA<e%VwD5O%`9i6uyScn4I4z=^
z%BRh9EDt77AfH}{qp$PJX`6LX1fmWI&gHQb^*lnREh&%3(6<Ob#DlDRmeVrjA`E@d
z-=eO_=73n92#FTaeaAOn@r<Xv*2Odhp57AgL!%N~)Dxxgyw%EU!4m@GeL04#+>LGV
z^g9W-Xpjct!%^}697Q(IG$EdkZ{X%DFdzQNGoDUqERI7|q#>TaS#A56BAyRvu#JDP
zG@jQ3Tlo~q>OysV7$vf>(Tfl7kZAde_y`7)rlO!qu*FB>)5ygz%h2~?<gj>F8HDFm
zp_jWm#>W$r>k5-W|74<kX-$^xjbGvuD2A`c&d-TYWI)%>8O3<|Vrubqz7>j=LKnZ=
z!QEwybNn#12V6QQh5hh<u$Su=O94HGD-r4cj5vS@TLYM*AlV%e=VRh|dxiDfIWc|$
zVwN;?xED*7D%oB}TS_<*p9>Sc<aw&fCBxyn`MAgLLi=6v0s<DtO-I*zH-2|F`uj)x
zZeRFj^A>C4#o^GzQaTtps;PgxI6}!9@J>a%sC?-_&e{I1@gjZPg=Hfk^G|Z!5-G|B
z125Q&T<sYz(t_mDwera&I3~VSeiyN}wVtGWQqR(Diku*heX}KAG!XI6V06DzyqF7J
zg@=+u+U?_bF%NBg>CdZx!fwZllX#3+2ecfbPG$j`fM!{Nz+7nENX1<m3eL{wd&Y~O
zfb+6Rpm?w}C0_irqcod7jTb*-M5T${883d`QJNPA$BSPOdgRFW>cop*a-16|ZMWn*
zOC^mLO|5N>@#4?Dz`ucyhb?_Vr&#2_5pBcSQ0#IGaj6K3HczE~8iDt-H_QcShC)BI
z_r*h<%`~1aqfh!9R-5Vc*s>ke3mv@8^B7R8>vF|Bp8<8I{D<ZR4EWs6*1qONFb&J-
zWLutxvze9>mZxg0Gb(I?d3guB<txl9{>g4-2fH4d%&VTU3(=a_3;>7DW@qzmbou3V
z1{l5H&3k*3ka&OPXY&jFDGMP*ZN7yqhvl0gQ`A`V9keP%^gTVzcL8k6nj|5mv-uto
z{RE<*Kg|!IkY&rLe5(xRhYaX@i}_E@|6)KkrN{T?w;7P9>wD7t4g*p{(nj;U3_1yB
zwfP3bThSSlqUCk;*i820m8b^uI%r^JJ^9SDtmX|J8hF9g+}xo7M<27?p@A7W=FN%*
zkjJeH=B*6KIK_Nx-ob!|$Zm1woeVrRGX(RCRI2ebK8~7e-p}Eo07v}H2N*=EFlV+K
zV1AiV45I1`=EDqV!0Q)kKFWZ`yA}P+#~J7V%1X>98H@m^^e~@ckP0xphxsf6a-5v@
zna>eu$^y^_na?vA2hjI#^92I7rtv_NV$ByhiVjzF-fq53;Gh9uVXgTJfgMu;UO8oc
zjX^QMtXlI`24w*H-^{POgKU`v!1|^6I)e%mRVwd{c6q_s{0r&)6bL%mn}6k`0+oOA
z7iK!}y=({79yI%uftqJU=D!&5<tP&luAA1KSbPM!`<<paBMQAAAnf)u7w+z>Xwuft
zZ(1UHHXr7+E1!&zpiibH!oanK7*ermzSz>c4?;@y0QajbfeccLERpEHYp4TF>1G*7
zfjnr9>T4Oy6eQx>*DT2lXp<-3&N6}l?Gu#d%O9@9R&=E$o5RTB-<)VM5a8pCuOF~X
zVL&5Xm51!N3cJcCOD%=9QuDd{kvtAz;(ECee{82M+c=5_WYedXy$qrO$}24U8N>pt
z{K;~dKr@{cYI@Ofi~+ToB0I||215YmS}o@pPy;#?Xt~0GJpHCOEpITO)^zQX<rae!
zmF%}#8sRk3@&n=4iHOWSB3G@Jy!(7_d5Vs;%6D#%MWm(3HK$<ZZpK^0RZL1I`PJd_
z;nmn3?IYj7pYA$~sEAMdp|?B9|6MJ`1(aGu7O6Fm&UI``wul$Nc8%Ly$-)azYo5yb
zHQ3Q{w20ThcvUAmqMJF~BECU%ug~pd5#L7kYt!X*You_Zd!Oj=!7k{s>n-93pj)dZ
zy8jT}7Wpyg#GmEvYcVS@P(Hs<_+ve`N5S^LEI!{EeRqOI{GS3v#G)R>Um4<f8koQM
z#3DW+=KJObTf{%XWZe*y?A~sR_!r354Iv#Iv$KeQ6Vg+uE1#lw0Il8P52W*z_pFuN
zZIAoVShwaWQF%ho5xhdRHXS&hF=w|k!=kdI?7rOrTWC?KDZ?eYaRv1NUBEOlAI1Zn
z_ic;H0d(u8?GPn*i;B+Lt@lHAvyv<-chF(kKsO8%J&VelP^%ojPU<1<mNV8#!TRql
zssT(xOSW$$T2$NxN@+N|`oyB5KCm9Y7y^0MEx~Iz%O_I4`6Df=1j={AT$M#dUvsYC
zAzxW11x&baQH^1qq8e(f`qZMzQeYRr*M?hEc??w!fXxFes!0qbN8-B3Vo^;7*Yyc=
z!JkfXY#=9M*4Ogj^-^%iZi`CkI<!!{=Zr<wNNG!H4R_B;i)t>T?m#c+%WKwSi++Md
zwVK&ei84{z=YZDlpo_Ze1o?IFw7qFj?PC&B+IpZvgm|a^{AG)Z`&Mfp;&KOCRA)FY
z2<W;0SyY!9kwcU}%c6RdQ6$i7*Db2+gi2$8HU?W%H#lw}klPxI>L#N>K!c~r*5{<2
z<L_8hcc|Eh$Wg(#wV@Ad+9vD}wSQXdc0v(NT|nQ*&tkWWQuBi2i$J>>(PtO8u37B%
zFlPzqOql%WImu_#UW*+Wj&=FeAcy>Au{*#CDD$Qki`_v+H2t}zv)Cy`PXg-VVzE1l
zxTY>cv%svItlJ>@Iv=vw9j64X+8+^$PF=A9=c2+v=Ed^bQyuPjY_U6o!ffcX0lwy1
zk3?U@^)NW|^dyqPI(JX>$Bbo-n!^rWv_djdVlcMYoAO}D*PKE?(1XPO6fg||O<s?Y
zYDsz`@8&fB{S;&Rs*%nWw9eV|U)tlJ-cs@GX!Okea?d85w&_vf2m<B$=UzWo(9&kx
z_0O&qw6rOdc16;CJ{2^`2&H7A^B-0?1Nb)akLVTt6k6d9`20a73%)FZ$A|v06&lou
zauD`-oV*?R1x>5)#tcchoz(N23Lk*y50b;+^L>Rf1tQ1b;xPGflQdvZcm*wrDnF5h
zYy%jCw7bF2pJj!1CHJMdn_EQ)qMQ35ISx!@JTG}=sVl-c1^Gss`&L9SAm3=+e=2B9
zQ;v2Yk}9GYC@x|}G=ML5n8EcqSzSRVTblC}uE(FpwqOL`?{Q|7x`Iw9G}9{8gQXRT
z%$x>Z{qc$<2IQaZQCAFMKr+m-RwOeZ8D2S3!KXy&Mp_)VicnY3)Ip}20=p4RN9?BG
zuNcXI*hL0ZIP$!K_sFMzB~NX}O-*0erFW*ZG}LW)Dc!y!?G?XtvWY@7FK8@%(>c8>
z(Jkjyt#qa41x+!ohD}#$p7NOaUb>S9QIW`%ndwT+I|1NFlc?r#{Wn#CO#e5b3T$oW
znL4HX64wdO<mAD?C(Do^LMTM@9_8-gMh*PnNlco0yzgG-g;`af?(SFmf#rYkK2mmn
z4K9mNO4Q%DU*#ptgU1LxaerMwN3NimyS?Eq27;Ac`S=RV!FOOi$l4iG^35k8<)|jB
zJ=Z5`?@oo4HK5F_E38TnpqkEYvN|vswdu1@tb8i4m73!4Aggjto172JL8}v!c>+}Y
zDgP<spd5Z&io*gCwr`WQ2ZJDh%F}YxMx5iaTDj@-#R7L@t^DmqD|xSXu3NpBnmmyW
zW30*=fDT}C7pu|%$ls{;v?^-=sQ@*eR$2oPS~CE8NLFRmPu^>-VC}^*<geC-S@}+%
zR)Y$w$upegTN`mt+6-$DQ&>Q8{#~oGTTlg1ueB<h1+@gitf5TS0MMYdhB258aABiW
zSqPX1up`W>^qWOQw#};an`Hovx2@D~gw|C6bFW)t7_3vl!@T;cbs(cAM1^g)#xmFl
zFzZEm>n1pRHCA33Xx)dfqBrHY5EfKuReHxsgjsdgp`7R}K>2nnEdgvkaS`C!m$Lt6
z$uIaCx4Le$D!yZsbrdY?MwgVNN?I)6B7yFHr%GD5-y+GSn<cIBd%U`Fk~kq{MkVQR
z3rzr=EtLUGMos6wSlOEa)m!Y&%0R68<47aG@)ec*Azo>%kTRl@)U#zBCFoQcsbuom
zW+}@1eFTAF@;@2sl=#RkxCMJf<#04vp|n9T&#rXrg!Rl`2KamPE4|PMHs8m)=6EGt
zRJz3t?|$i(G4wtT?}sug=_1@M>+n@!u~0e6iPE-P#TL5XLWr0}R^|44^C}(CV_T?j
zX?c}hAu61A%s{rhj}L8MMpqIdD$_fIq9GEqsT?}DaL-8R7KS;}>hQ`*9aQ6p>N$jE
zpC>BAX%%(qs4*T|>X>%rS%fBe@EziI<*9^B9)U`yJvb7B+(VDPC}i7Sh;HOqIQFwW
zI&Q$VD^J9TwsEw_L@o091|wSJ##oq#8BHEy4T`X}KStd9ojk-^*d}FtV|bj0ID<-*
zOzRsLmdM*$B#l@kx3*yId#Q)`A(xhahvfOiojrIlW9xqT`xYtO&&NYqqiE*e9+~u5
z=b_4=Tz&*6{AOdqR-Bu7=%E^e@d+1eBG4Ns3uud^7U+n6e4aXxcJ#K<m9#yF1=1H;
zxNje7{v?p@xx#IISX?ld1@h(te>!q-A1%ZO?gz^VO(4{{Z9Eid^ra&sb^S@HweF<)
z8RyvBwb2UcwqmYL6*0RXt_eY<ZW~WVbkTTd*j=rOQZ&4Skeeo2K^tLU-L;wlo#=0<
zCZ-egZ!OXcB>g{P6TA+L(*6k2(8;{*zB~kI;*k~oL_=XgwV)Y9mrD!D)Onzf{B1C5
zy*0dMm+XnK2TL_cSgF|7zbC-KlbRt|qv`%wON}n_mZN4Uu^ox1oNgLku;Z1;dV^*J
z=$tit7A+$@NHdD)rfHC0Dk8Q|qkcF1JI!d8B#@f)c3=FTMM_A-z*~s&WNJzsSOZzJ
zNJ2z!O(4PX0D_)kESl-)659`RNmrpMZC@@&Z<qSpGV!Y1=H88V{I<UrjO-3*#H`*F
zLF5bLQ7;Zo8j*K-k^wfmYgB%SANAL3VlS8%y6<D+=Scz1^iJ|#`X#A1n}#O<rr0FK
zPW&+r)p;q2?8Htx5t|nU4T3GHFB6J)bLbbrllr15W3Bu_4*d{$QeU`xK=0(xZ=5If
z)iU}Zr=+F|s1KvZh$GaW(Wi(j2Z~_yCAbiZVf0N-NjXstV)O%q@lqwpc{^~fY<N;X
z2&HxvpOAiO(B7$B`~GVOR<3=&BfKU;LA%nG68A@{UFlR}6f%ShLWM-}LzPm|o=olg
zNnu1cNtA&4TuBP2xJP1DPBqDcn})Ht@kngQsg4>5NDB~;#Cbq5K+rPKGN1uKL5$V`
zMFWK~+6WW{6wPP{P$W<sqkTEmg8=ZY4wxLtsg4KGF*uo1O{$4tQEcMHoNCfr1Zn?~
z_(o1O=`@10{|GfxP5O?|GI|GbsEY{F{v+`**bpM^KN7#psir!NAniXAf5@pu1t&$2
z_8&dd?5;`NeV?{#cGomdlwXLLhOZn{LXs98a@X3L@&PCz-BZ4!XH9u5G(mTl?d(C{
z_yp8}%pbjE*B)5yUDE{NBiInf9NO+kib42p4JxGijiiAn&91dn1-s5A#WGM4YVu5q
zV>>7jvae6F(OUg(jYRdcb9EB0)<2p@k*E754W_uDdFq^^su{RyklFmDe7%cyvO8!h
z2|uf&IJ*aNz?@%&*CEfHrC&KBKAGO<#C{b(@385+VyoQA@U%{c({-`R$Ai8kx$>j7
z%9ApqO}ObTRbH?*yKl>X?2=OAdQ~Yd3-8|1diU-}ti^7u#X8{cIa`+);Tp0!Ja;;H
z?0$3tr%@7IhM^+3x4I%r&>mOT5AdxGPJDk{qe7964-x5_DvqV!ED|E>DK>p)O=pUo
zEdTGAG{<(K#!I1%M_k&>8eau90KJ)9(@Q~t#5JV0jVo?z5!Ss@%Up(tYk|CP)jBAY
zvw<QrYss@l{t-Y+y=z?+G`vlk)@m-7Sb%atA7j3uv97h?#2?3{GSTK&o1&24MdJMU
z+B5|%13G1>)hj3ysK>tAaSF-@Iu=lyqo6@R%e-f_2|EDjQo)QiVMhS<TQ{Rk*okcd
zoh)Sa<dAy0Ectx~Nq=&j{LKleC~SxriO@}|8Cr#V5|H1&W&|oIjrQwo&DE+@t_*qh
zNolC=;=DSVOq?pxRAEE2&KV}(bw3%D<s<6oE5}nse5RD<4X2d*mpf6;;i+|QSiC-!
zApdz%T3rx?1ou1DJ0i96O9)}N>${=D(2qi)W%al!)l)8~$wA(mUhhn;pTA)1@u1#C
zOA&n1=QM&(Pkr`-Kj%Ky)YC-v^kMnyQ&N2Ill64#X7|V4>4U&!Z`AknrjZL7XwEq|
zXo>JoIps7ydbY|Xr=@_-(a3I=RbG8s^6G192nE5JXyr&zGhT<5Av@kK(|)IJh@jXq
z`Q~XXXKv+m4KxsA#R7L6HV6&WMU`XFi%boJV42QrWmac~;`PuonOz1uH_*iM%=ffy
zAv7eBg^M7KPQTqiL-LvLX#h{mY@j<6&itYV=v~vGqnO`#lk&8dF048ICmv_YQZd7s
z-9RJpS%<b3aMnj&e@1fmUWt?k=QpHduk*}bWGq|;H)J5UGl%4hXCyb9twGN+2YDcF
z-SvhsK#}=Dv>o4je#2ObOYTo7?_t9@1@W)uOjoNLCIFv(6&ZB7(vXANQa-=#AKNey
zBT3theV328HPSsF=V)!W{Od-pRbFPOcWHF#5(q+`;%&%jRHkNB)Vr~bJT+^kDV|M`
z>(5Gq!*S_lDt?FYNF&`?aQ0PHXtTR>V=!bmM{|uK?u~R*7Q-UIu4D4Q&*D4Q8S-yu
zu@ToKd!NGsfX1znEbqC;nj&nCHV#DnoEt{u2QwR$$qjF=-uT6>k-ly?uU6;=chJ?}
zZ=|IIwTkIX)8ZS4plBE9cDdm}jmo@-_?>;ysLXq)5=-R9VQk2IP|-#eHYyvTR5UqO
zzIhH}zAJxzPU<`CWFv2d@+|}_?lsc7qrhJluRGV6$$*@;yoZha6&n9E^M#F#W0<)X
zF+0L!{dsAqZCc|5sOY>JD)N0tY^2(_5Thx_^%0G<<4B%EDn&Ksg6x8qS!yhVoxfl;
zAiKcEA~Y$NBD@#(Z!CrbEwtkOOi5!2zf(_LbHJgo0E{kN!O(U=2geh?3EpU832}Fw
z>~ukjn;aS6Xa&v1K@e=f%En4GzzbGVP7jC1N@(<g)j<)d5{Bl2m8R^@lN&382+%jv
z2Vxp4BMFe%Km}ECF{!g|{Yu_>0lUOBGj2H@+Y=JerG9nOg;-xWofj>Unz||>)(-~l
z!WI6_wX;5;d2!?TI3JSeGfo%Ro{oNGB94~vl&ji?&4}}3qZ9)&?}u^pGld~P#}oTF
zEwfK0_Hn(sqPt!EECW=y5H9ng<ZZixT+XyadXqJ}M3d%jW4%3){8AH@`>hsl4FhTv
zy>@wfV$8j?pV~x%)|;k|myT1LICs$7+ZEuVGeE>{Zy$~!_2jyH_ayLu{P=UbynQ)_
zn#4k#H)OU6m%gQDP<7ASA5oV(2Ld!_c++LOmyVMfy-U35F5%0#Wfzq!w9>h`OZyY>
zXyONjUfT2qLWNd(Jxs5q7wGl$EDFTS15ly`QP|b?ux}AIC(1@K{4+F{-ffGy+!<jm
zSMh)jD9KOl5#9(Z-9llP=?6|O{s%81%I%PecHV)o*_ZthE?maHQK%eRs)FZ#IW83Q
ztl8f5t2mcOvE$D!0g>6POvhJOUd{*I`WfEzljoPMOxF`3rBA&FL+xJc5n@*!iI7c^
z-U$flJCQ@Q2su8%I|(6e`)icS-rIW!<|QsgOhc2$^M2mROoP}xm%McxJDQ{2+Dn5Q
zydbqGd<-4_KeoOEuBzgF|IA$gWf8;$J%Av{rXcRP;=ZT2?+T)b`<e@a2X9`kkcw7%
ztu!??EzMlW)XK~ym(+02%Cgk5vLxHgEdS3l=bQ_D|35yTxy<{{yUoly`<!!**0<C~
z133C4NGv<!<8f#Ft=blTjY7XXxHAr}6Wt^whp!en_MUs^QCpZe=00=OR!`h;mmRhB
z)grsb<<vtPI9?kSx&L^Mz9OMx9a^6g%ke|@qbqLY#1W8}Mv?~R#7ls_G}D<QC;#%Q
z-APN1ocw<op-gI@6NTn~yxCC9D}!=)8gRTBx`<;(<+KfTe!NSF(gAsqx3G>hYM)aV
zRvsUQ#(y03)HN`N+eF9r=yo-irqC{!(!yicp<mrkj@eSQv$__Z9mEWQmgB?Nqf{Oi
znA4a-Z&K(UY!sr+=X&LEIeNkeH>aj3Iegu^-jXK#og@2D>;ZtT?MnZguB78~r^&N9
z-Ow(N-&AU5&K$4g@HI3idhs~3@e4WVU3;QMRPKw3OYNM#s5I11I(_i?SqOUR{i-<}
z;!eDx3mRYunoU6iDd-h1NL6x%V9Il1^O_SFQh3kr1UuRtXZz%&gZZOy`F7Eq4Aef?
z<ovv2&e%FkV7o6_lrv6tWNi11x8?Aj=!rD8`<hD51Oh(n{679MXQCvl4<Knz&SXi3
zb+mH&>zpZ)tOM<iJH}T$u6n67XF7Es0l~UMIWvIMuNT(5p2PPleRRZq@&sNK^m)#F
zDYY}`&ThzAAc5`xR_x8mg)`P&#_sGjXO4WZV?QTiGmCQakrI?S5WwzUIg7#b(TSl{
zequWt=$C-6NLLS_zi`_Bk$x#OR>FS**57E+Tw@FV=nmfoRJ~*&4#kkd2I06(Mi8E4
zH>@-qK)RFulJ3KWcF>&+lXNT5+&Zj|<?OqWCR<zJSlG5Y_9=b)B)T!~%h_{>f<u0&
zXiUFc`I<_8v}jCnt~T6EIOCT)g7Gy_%;E5){sNOH{OWz18@YU@@1zs)3BI}M%>5?B
z5BAB;@WdzP%7^!eU+0~hY2phUx#P_E;On{a8f_|QTmiYbMT-?=DsphDYF^b??64e`
zok0jNO`guH2^Se6cbj%R1ZLs;c`C|c5c?F%j_ts#`z9~R!?j~AFhjQIH7Vzs56sBk
zd3=kc!4(Hg%VBvvJzUxjDw{tqudj#g4JykG$s6EddxOe0y`PumVS9th0>bkqmvd2B
z{7-q)%eknmMXx+ZITv+3-9FFd;d))8vYZ)t&wAKir?Qei^Iq|=y-sEETk|%Tvr$>p
zt-NjJTvQfr&pTAkMP>Wf<bB}b+D={XRmwZ&;o81k1FBte-c=9Vb}IAVlJ|Kz8<j13
zF7K;yHY$65SKfniE-EXollP~GYa5mI3eRJguNVHd?eVxsTPsflc-XeB)qopbm?whE
z8Rr9)9+D?&mUG4dWjUQE63RJ4fJ%KiPc-x}zV;BP&T5`$;$eJ^D(AkKC%SqVUt0^*
zzMt|$FUiOzV|k4#54@fy`j&H2W$4yCF}9qOD%by(Cnk7QZl%hw?s;OehjA-ariSK;
zMIOejRC#1)o_MC5ks2L4@`UCp=cLLr_433^<(yR6v}c~!;Njdtm8~N3#AXlU7OM0K
z$`kuNj9aL(%aA<rZaE`Wc83)olrzS~YiNiw^Tfq+PO4mqg1zS9+)S0Ld*_MIJ)E1V
za`#tx;%Af5I?)D{+B{GETF$r>sPK34#GmDiDd>oOHdS5}h0ghG7fT8ned(fT0QxHB
znN5p0qT`eged{`9k-Yl3#tL9j=%QFjW(TllxO?Nrwpx+DFOr8qYve-@AAbSaw(j;t
zjVPP%JjWXLvyW}Db-xVMxbJk1r+}|zYZ25S7hDtCX*`t;d%+!b7K6b=O;^qbGqYV9
zpF<BQ<4CmTzNRbZ(P8!mjZdxc=n(zbl&gydM^fKdbm24AICi_%iR|aui=InNM}%wS
z)9B)Z-c(T{=R#*9dQeTUn?(I3k)}OScwF9KpjsaT@wpD|>73syUEtP@)kKcX@=bAf
zOM&q#RFg2^=r3~GoQy5Fp(<up%DG)b(cx#Se1YQ7Y{UobQhACp^af2jSA?LyZKaCT
zmH3g`7%QH0yL_NavmE2eNA+jb=JEy1$9hl0LUw`5edKEQ5fA@KO~F7T<H@_#mL`5d
zWwj;BzS?~lmi&g=GJ<g$f2<d{ng*z?BqdV2-d9`8xJ>N{Ov*L2HS8ljK4fvM2N-5f
zP}`Uc=`+-JSPl91Bg4_QU`U^$rsC%0*`5-Ug0ifhr?#&SzAewH9RhIstSRE-ZmS(p
zK8-ZQJ5EwN$v84s<+9pYeh%|c3d#d-OjORF(&yMkr!;jua(DhcC|UEOI@@j*@1dr^
zSbC}Q#T;@#I+P4d;Z!vj>hStTn0BZEFvH^1ymCw&)Zs13S{0cdjr5{<YQ86Za|4)G
zG*?|LF^y@=B((sOz|}ip87B>&j8CU{A;;j^g{ljQp9}=$tk=|~h`($rL5+{uq`oFc
zf5f@Jv%Eyzj@g*f5DMKL)wk@3>qbWalk$_g8)f^69BB`QN*6*;-Ka$UjoZ2UheHps
zJD;;j<s`+xR0AgBiuy-6CJ>m8|ELeko4enlI8_KVuF^^Jt(2;;V1}>1m2&-lRajwW
z=_KA(x$vGUZ1n(5=XIa0!K$!h9jTSh@dFq<P8D8kqN%*Y@<yI2cm`g&K)z^dP!E_q
z_%MEk@HCQVCZ&ZiihC<_ara=RDtzG3rCdRk79!|ROVm-~F`M0>3hoM)%Flql;s+8s
zK<R18kI4vVR^L^{WYQdDt*!r*Iv}_7r*!H-Re&b!o@#kanlDImAP}hBZmNYZpw&_P
z1Ho1e>$eM0QnWW`Zx@33%D<gcR4X<BFGPW<^CQ)|9zegprLGd{pS)M~K8Ygj`x0@E
zbfDbNsNUzwQOQ6p%~8ECm7{1>+Z5INx(Btm6sXOgs@{CGQJ1xNBPy(IT0&Jn<U~5e
zem`I`o+R)q!ZoML_P2f~Q0<!}L`0x0$I*nh9|JSboe+gKSL28@bOtcn4kyIb#gh(P
zHd@wa1GB1iLIVg2k)MTKpAO7NHxe2_eJx)prwo|BqZ68fCj5@%8VZbdI3b0YTwpZI
z`tCq=-IUN4Y<1!#+g)HbTux|DjO+x~w+E)QdO~Mn4oI5Iz*I>}=mw)Bx=74fRsg2Y
z7dOcYc)|*}xaw_GKw)MST<s!1=_>ZO2U(3V08f!jYbYz~;sMe<^K3mV6E2>TbP>c{
z7{&aBccy{{8}|c~(Oh87u4C%t6BDqKZ_t$(@n<?Q8xPvvG%@c`-=!7|(k%HhsqoII
zs<rLI{NCfe?@O^1zn+KhDkNPRV-UjrOMOA;)WZCZq2*i}BMrj?Qhcd!nBN(KyA^aP
zRZr*hRlURCLVU`i{DEx$YY^W(DSs01ViS7IlQ;5T<bmqR@AF@>qvt;Jb9>E)PbOS$
zYr(|fkXA_!aB?Z#);;7B->7Qgz&m@J79f*&^|i@!S``47+j6}0YM}8#P0Zg>0Jaop
zK}>n7g*gyjeoAIU-uvgkco{F;L4%T~r&A)6HEs}H=H1~fe`__+eThv}KI*hqQvyC5
z&R=Vwg-D2N_s-)revnCQ;?_^!sakCbL;x6lMdPNHPt`cCh1b5{S>tW0D_f8><*L>|
z(oy)u1{xn2xw3@|&x19!BmiMA!~odyipC*Zf1}^*t6H*;QVCE3k{0>9i!R$@`X1A`
zZ>nT*0XnCr)=UD)05(}PJcEf8`7)HBHMtp{#Z9`+kL7t)Kv@$2$6nL)*Tpmgu;#w|
z$z@waV_*3+5sT*vU*4uP3}X)7`o7L$J49Yh1?j2li@o5NSLG?d)^K;fE4J31wk_tB
zC!eZ(dUSfT#qyHR8opqr%hJUG?Bds`01j+f%-f>MnlLU^<!ou|Ioo2KU@KQ0?oY1R
zn#J|DEoRYOJ&tYOS<2#2#PxYuUZcf)vyuLlySzkqgR9uwd#$PC5-6?(L-@r8j?0M0
z{K%3$SkrL@K!S}saRJ*MSEbsz0GjzbuE`4EMqIOC$EOl#pd5i4pR+h@0o>?g0bEN;
zN2H<m6FG4@x#<}y3NsUXF;Q#i|3ujY#*=RftT3;&GE=i9YZW8$pd|o_LnPq|D>i;Z
zF3I=>zS1+~5?jHbB_XmkWGtyA|4&^~TmE0LM3w(nFA23Z$1dWgu1?VhhI~31WwBpt
z;grv|z+8aQ51k?nX(%E<lmB%Jc~X$YtD5c)okE@zWLW{d_rxg%NJ<Wc{ui8LARx3a
zKx-yD#UPo+>z_@YbBe)$(82(DHE@a{fY8JMp@})=(=upe0Mo}h#V}dS?E&o`<P^iD
z(2fe9d|TvYwK>HGSbh|#kp)iqZU)`(y(gVwlT=B=Pu_QmS0!X=KQ1Fvaf;1=K0Rtp
z!@IJ+bc!u94L6mma_x?a`t*`4ZHxXwL?UiU*ZqKVk$&!i8mO92FEPHcAl?&yy`YUJ
zzPF$+<3~X=_h7+l#C<G?PibDT2Jy>n5zl_F;AO;P{ziQDdj%Vae@Y|6hXtEF4DAYD
zH5ulfF4*S5cP-d%;xl&^yotCj=y<^%;-8|NYX$o}^rH$&JoFzH9Q4qiEqLFgpEADS
z1I;9wbGm?c{dA$13r?9t^8*XcATDz=^>D$*CjGeW1*Ina=8%HR9{P}is~-A~3a**-
zQ_mNCVaCVpF8CU8>e7^))dhD9K$(_Y@V$rV*Mgr-{QPbO4-l6|OiL;F)kE`l!9$a#
zN@~Gl;#m^&nil*kY2fhUNpykWb<EFO!8N+X7s#2Iei?b%2CYDF$oTjRy7<Za1%lVJ
zaIaSSO@ZJAt)LVBNPF(H0wJ#?t_cF<f1yC|PGZoBU_i6C6$pJq;ad`#JXavtM+KeW
zn+CiZ6bK%U1)X5uv!$Rw1W8DqtbVou{`YxL>1*<#%h$#gh-w(xKj&g&LG=QWfs?e$
zqtGc_!ujaub0lS`g2eNG6^Kkyj$@Cp29Z@VQP5>SKH}mr#OI3<pC9Y0g?PY5#1GVQ
z)s}IS=Y%Gi{K(Yzpeu|_Z<~o<%d{Y-g-vqtWyPOgFw_2#X*HQPHrkcIw4V@f<Z$t|
zGoN2Te9BZ8H{~V1NoQ9l6Tf<gtEY({rcHJYFcS{la&ae5GUS|b@dGkaP|3Hhi6%aG
zrfa5&-#^C1TS1b3Xipd4TqWr{wsrAMQxd;mrmNV*N1k`_)ld?@Fxd49;~Fik_o|C8
zY?2Ajop!xxGK~JxRbt}fj=J76@%b;hj+ywB6xUf3|KjVeOA?O(@6hN_*XQu18#x#x
zy2QJ_K&_*i08HrZ`a+IpT*+(GHn?s`2jn(#ySc8fBwaww+2dy_uAA5!zmdb4!o_{A
zuO+|}qu?K0-$)>YgP`2)yn%7x2A)X7a3c@M9lyEmz&mbeaFs<}UBaJ05bgTgZz-<K
zDK|<Xs7rj4rFM^D(>5&S?E|z0PIzu?NBBZtR=$?Gv{EE_8sm(_>7IPUmeBtT6kL&C
z5pM!~%=p&OqUy*z9udMI^Fh6$ZBX)+kdJ~EZKn>oSy5P2^tuF?-C_NTb`X$<8qcS@
zzq?_pXTiIQ1HQ!J={R@Xmw3o%ox3j}t#i?S$kSh0yl`;QI}(UiUd6cZ`XQHi;y(*3
zs>k{71`+dYzG^!g^bTT)u4%nsMpMjikKDlzUg`_N)g9Xw`ZGEIVUKr_JOC;3?Gm`*
zhvYUTJu<zRyz8?bUn6;HYsLNDm$vBsw}AUFzk2{@qx+GtOG-4Efw~QXz}1~o{8L~=
z-=doUxwyMo$1ihiwo>#RCOtRnpsZY#itb@4ax>aJ>ML95kVi#w#i{H^Wi|e!NLXYE
z0oWQ|B&>Sbp`w@YN!Krv8=<LgE5f%2l$%8;hDyhZ-i2J{%Yd;nW++APMj)OA{OCvS
z(yws;GTeRlD_dCCBCIZ0sj#f_mZFWOWjjH6>Uz<}7&#q7V)2}!#b)9oJ#leeCL)fl
zs9ra13Be^;f@77^plE?X`E^70(wnv@yr3BPYBP#vn|Pi}L;UPmN(kU*7>Z^$m4yPr
zaZeRZH3^^A38zXzN@?jTnqVgKts{^!A%RldRldgi#oyIY84^W$TmPaFCjD(aaYS6X
z))CE2twkM*1{s9k2%T_{B&3u^kwtyYM9zSrwJ&;LnfuK}MSW5B3iIK9<7-=hmS5Dv
zBp#y^_ozVJLlP_ao%K#p0tml38ZaBjMnwr|WXc*aOw2Ayh(M*}Al~))qJ;X4!#y@$
zE|RyuQ-k~_Dn)z(M@bC=K6pb>3lz}JBE-AI7r9$}W2<I|e|__(JM9}=-D;>~+_u}*
zsZcaE?7CgaUGR-9Tum($-K8J-@=U)_^dP|7l(~D{AADm=2`NJSa8V)N{epysn9$xT
z0S2EwsB+5|8}T#Z&ul6bvuFxG548?5W}CpGQ1{?lwxpgXq+7&?7Ctn!^MT+~*9#wF
zt!Fq;;lsv;=E0?fg5{4lPwK*bXp=77eg2j$M2vNRcgq$f&bWPV+oC#V`pp_YM=6xg
zHK(Egrfn$vp)w0h>4uqOQDLc}^p1}#<-$@lFj>kVER7+dd&X^By;}YC<iqH<%;dw^
zoI~=i+qRgHS%vZzHcItQfo}aS-R<t(wyCwYc^7Uq_3YA<x9Z6ecWkQvFFI;XqIA^m
zcWhxT!V6!P^(V{boLYzuRrU$rw$f>fB&{FYZs7`9E65wxMC<P081j`u%_N_xlWXu1
zqa_t;a9G_j4sI)ard-R=+lBK?wiP<tJjo_o=hnhWX5ukDaZ;?u!wV<L)+t+R;UstK
zZ*7&I`mHeCWVoj@q*q`_*PAoyAmC7;Y*<-}@|}+#K1@-l&*c?#AGv;o{8Z_8iSFmV
z#S5-_2Jrl}um@5wCZdH5I#JjJO1|5mS56bNawcXMHj$MB#a*8-Y?8!^@|&s@HmhT_
z12}DS>9cGgUE&K{1xcCA>Zq8)80_6)nW=nt5dx4ZZ|bJ@qlCB$65^!<I?|Z-h2b2<
z$n&?J+wg{z0qIZ`p%bE&Z3%Q)`WxkPw}ZRrJ6nBqkIOPmI<YbjQ>JDvOIH%0Jnl|(
zmwt!m=+6Ll)puD^C1Xcy3H!M$@c`~V2fi)O0>H`p-RIa<+qucK`<TlT#q<x1bnal?
zeGV4A-WickNWsCH+eCLi34ykT@>Q32KLB&7kINE9=KBV-7ACnBq@Df6WvPj@@A=FP
z71#76oj>QY1Tl%{2uunpPdehV_%MmrmYC!tlhA;Vv~pQ0V;DkVL;ACKT;ea%W$Sd}
zFG)9rX-l8D#IH<y#z_0sBYx=(m$=8ISBxYMk|+&urDnOr9VQ(#lJ0ntmcHu}SD3`9
z6O0m9WRknz_qNcCSl~`wcZql97zSYBGyo_++55q2aa2{5A7X=$hot_1&{h4=Kp!iE
zPzz_fK>6MuU4p&B4?N>v84Q2}F1spxX7&8{wvc)sDlV~IfOJPcmsn2HS3w%$0|4E_
z53wK}T<Z4y!4?^C9SMo&U1D*h>~vZI=l_RG%mEX<?#HU`>>q3~qcho+=iPS+a}@lM
zm&lkEeHaYnU<#mxk(KnZE-?`lXis4L07ecsC6*YB!(~qeQ_{z}@BLuwseR}YX(rK6
z22olZE76diCPz)l;S&8l(mQ<V5_rT*+Iz2w!PreQ8nkkJkp)!h5<St5WG_Gcq)YUY
z0BsD_GF+maDQ1cxrkxx?0c<WpsvJTMW7@Tq8o@cIsY|4ojC^Z0izTHZV~S){(83aH
zyF@+E-uo_)gD(I%`hsa<jch;>tOY<_p9dfZV0X`Zwpa~S@e?mAV(KoqVdwFe+R|T`
zfiWm@HWXIdPkh>nW&}w`W1klC`EokVPt#?ewtSa4>aR;(zN-!wgeZyRq04u{+vJ38
z*2?9(WNngJbDmnhf!sTF?hQ17a|Up4fS;M%8(^}@y`j0Z1>7$`yWCA?s+j!ga=vy4
z-H;2Y#ch_a#6Rrp;o4Hdawp<Hhk$Bm(sCzNWCcvm%2Jj)-P3=vHE!KQPgzLQIj4lI
zg%OOSH}rjZIo}%$x#()=wp%`3iiNmb$>jdxCtKZG`<7=>!%?8p$1cw@i(qJyyXMcf
zrs==xc=<vRW)4=QqXC+_(%DX>{Ls&r4>RQ7=Y}4H@x3atM8Gr*bq_-1X~O;Gbm{wp
zr8M^rXsNnTm(2UwhGh3IKig{7S*zna$$XQ!Q`+)YWaeN26Jzh&ns)kACu&;Gyu4|I
z)P`)I$y=Uim{nFyCrtE6i|x8R28m@U?pN;Py?@+EL;3M`wow<mN|x7_+)zE}x#htm
zo~Kg<a{$z9taQ_Ie@V3(ziB%PEWrZ4w<MuJaNC@|_sdA>dq7~QyqXGN;WPlaWvQg1
z1UXXtGDRZ~T4(KV3&d&o!!I)dY<adod=2`b6TTCb0`U!^($Be}!Mc&HHhpY?eE)##
z@CKI_SoAFt<nxz$kdf3M0%K*b$xd`%R)M&Hxk%6n0O?N(#6?a_*7&+VD6`e<`U2v{
zaP^CR)xQc)HGUb1uJ$wfwWWE1;J1!`S!LvDZ-g9F(2|3(1G9}u-?&46!IF>)PF*h$
zughTv0JFAJW=-$S0qJ(JAILaXAn1z^J_BW^sRd#xR6h7j=6jBr@7J;m#Cihzk-G=%
zwHtm?AhPXpE1xKC%MdlmuJq;hK5=J&<F?XVIRgXm`ho(n04@B%4*+Ok4vf0W+4eJ=
z3Pheswhkq}$OJGtSRdFTeP>eQA&}yBK*I;N8Y9s=FgH}NL6)Q}K==m4%ke&xlZ8u%
zki|2P3k0VmzxI+{9RN8@AwLt5ul|vqo(=%SV#d4a0jAkA-G?67RIVd_UE==ifvsML
znb<+Y1~@8DpQs>Zvr!$XdiVA*)a=;h6gN;-sZjlLZ>P9Q)!ZdNamU^MS6j3Qb&vlQ
z7wR*8XH1`sJy`BLV*QOuKKzSQ>;v_0ou#1wfK=9ib=n31Et$exeL5NVhG5-J;+Y2V
zcJ^i_@%HAD7<9$WoFb2OdknffY<Ylgur4(ZjpnyKsBWkJdSg}x_nV#2iZK(~u9ZN_
zsZLHYno|BSq>RRd%%g2IP1_?m-RZyK17>`n_0)Bz=mJN>)CW3Zt~ouW_In2-IW>;S
zavVcyGf0%QOMh^RmQcn<{Cm1V+fvd})aipx!Rrt*`9&kSp`6JZqFOXWCnQ@ev(gA8
z`tZLNg4y>-&Wi?TjO5gfZ}q#ac313~hnqzI8bsmo6)X)$;p&#=4{-`k7b)-$URPnJ
zYf4)8OTXK~gEI|WkX-5k*j(iP_;*`O<6<Mhw>-fqd>fmBLykIypx`wSd`dSi>JNM%
z^J{cJZJo|XEPmOuX9YWd0pXusNNA}!f1wv}R*szi3LOxDZggR>^LrBUu*^5#`Mq=j
zo#;oM2nDvhz4J38FMoCiA@cH>v`&tYAI@-I5cG#NkazHB=V8h_>;CEwTTQP_P-DvX
z_z(2wQ=R-?jvNZ7^>H4R(xGY0Oy@D_=vcZtkLx<oX3l)(+(SXP{N^gocXTx8i5qS^
zw+bjqc*prXz=u2|OZdq7JhPq39KFBD$>&QRh9lwa7S3hnNT=fc>dx6rnShk!p-%3x
z$ng*c9{kf5HuMEp(>vOkYxEkBwV}Q<lN>z%s8{05jAS>Bc<c7gO!hM;z_F#Zb39No
zUs@L@M}58gr$6f)K;|p%q=&X-@w0o(LtDtWFgZPOc8oBTuS<1yjHf7OzcceGb5Z=f
zvjv#{>aCNvs6gJrL*CLuzIUsWj{nzmXrJ?o`}2pksF+nwE`<JC4b(dgol&ST-LAK1
zIBPJ0i`jP?x}*QHMQY8Khy%oZ35i>t1psH!*AdgUED@}TN4%WDq&+BU&w}bv4RjV;
z%n|BQ4cWm~<_Pu33+31Rhb8}zmjwjge<bTjoEyBlL{x|Gp1Qt7*wN=&uuJvPUwD)u
zYsCn#6%?=NO11*YwxTOchcR%9x@T9!GT}2MtUR_NmWxZe^ulQ?`235kgHzX6B+3EK
zFLQ>nA{ho=?F^DJGgq{eZiaYXqZQnpLvc{%sTEZq^Y0PBM1Z*p97gWFKtUB4gY7E{
zs?w8!+^$D>(NKwd|0B$J4;5RMg7c~a+^+<J2rvX*?JRRvY$=jv02ni~*s`iJ2N2)6
zAmY*$t*;|)eC{Z=%%bw2boN=Ck8}>q%mCG_<`iv+np06!p3Fy(9cjy=+!=r4okvN5
z^CnIM(_oCESLsE|-UWbakU(2!v1JH$6L7aiiXOssje`Trs`z3{M;P$9&=Am(xi<xL
zY(+rl>BI!noGzw0@2)Pk^r^%{K@JajjfyRO^~^zb<DtbCI_2XBKuydBAae)6ejR|6
z4S>cq>`vp1yf&&RT)uvhDJye}<#Pf-nE>(w0k8qeMc>gLTCs(_H$df|1idI@WJ!RQ
zCdSbyN@;ec*b+}*KgWw#yA)ek_izN|pUHvqQGiw8FUp$01&qqR5KO0FZjdy}7q2%h
zwp7O+$(0h;-}=mA@h1d$|HJMKjp^^laN_I5;;(XY@sA{joDH*zEjDQhXAAwu6pIt!
zrThHL1r^kY6OllmmY_!PX%Xp5C?WZx_QVAc(s*~VI3<P1c2q2Wkw%GgQj0H6@ZIbF
zvDFs-?sxuyEtA0Fceq&G^pn}-%;?~wVsQ`CWz5Ft0#R=ji&e1WiM}8MKp!@2cuu;5
z^Z@Z(3mR((aAVpfHtDL@ni-^RFH74%l>lIUP1(!<o%RQW<}WA68)mtu|7!~gXjUxt
zO1mKJNtAo_zi3o2>dC~wOyqwtTQ~PQM}1Z-xc4Dzkk&7e%4xjq`(iPT#vd~Za2kCq
zRd;4Ftu0FsDQFF1dWEb^N9A92=+@7yX#LZg)I-rh^^3(Qp$H}15Istk3jp}C7)1#b
z4PO?cO~+<Ujh20eE<3%V>~XqixXoMN6mvU3C@Fz-DA6sPaSowZ`j;TRR^MXLmg(d4
z^tPc)H_T{@sery|R4}|)_>t_6PUfeQ%r|Q+L<Yz{0)@9IbhV$omNv6k1e>VD^Tnb%
zDkfIN?5<eUkap2C-f37Y_}Gg)S4n!cSn#qb8&x-?#TLt5>MTXb4;(BCEq;jHDeOqW
z$M@PR`_Z>f3@Lsj1tOl~D&~jX(M8)ZL5^L$-9AEl0eTkvQ7m{10z+MOh}&kMcohOD
zT0#oGv;rWN52;;z3$=_^D=ZomLisJ2CY7Vpxg{5Wy7D^}l;6>nQ+M$lv`8h@&+QQQ
z-r8u;LPZ(wi0C$0X6Tq-bPTsfk^Vr3I7;k(s~FRDVL25rZuT6d_$tQ7pv-{Tv*4&#
zV;F~3U7W>N8yZ%j*SQK8kru)$u6CE2;2Q3?iw|Q_B&@tm3{T>{TvO6vxG3WMMse}s
zL~!i)n=lUAj#MH&Y{Gex%VLk!_@WDbKpci0JX?&5Ey6k}fUbn^ZNLRUi6LjRK1={W
zRqior5*)+5iZQ7Xwm8X=8A1-ZonAcNEIV*#dAQ*&%VaCnCAq7hyAQ&*X@#{E7^F^R
zF>{CG)wJSiK{BIE{aH_)Ma&vKb&e<1=BKCTRp4Eur?T)N^_dEMOY{^B*64G!Vy!Xl
z-!1KLTp`EJrOjYZ_^4REGe8~!A@@UM?qQV8p$q5Zs;0@Q<wkAPQ@d42?fxI5`pJ~e
zV1^~{E-S{UBkXoLM=Op&ucWNe{Uatq)+Dzai(}%6M{QT{P#h!Y@rb{>+CAN7uOo)L
zi*5FrTBbZeD6Runk~=OSty963>R>1^U8znt#;L!FtvHza-_rF5bA&ha^T`U+tAa7w
zoBkQx#1I7k3~q14+dAF#?DlHfB~<^GXC1blXij&Z8!b0Ng#DH=yF*NRT_w!!g#8lQ
z`_XWRtrHCJ;&lx&c9OG3*o_?&;T1s&li4dmGW(L*D-g`u6$d88016G}WV_^czi+qK
z!aH!T+3n%ls}5T`U!Gz)Z0!D!ZE)^|CM>Ol*Exf;reyTxZl%Lki*(#bowmVY!>71G
z#tx<7Ck|VfT_5z|43`f=P|yyC%|=1o%BB4_DTq31ZFN}x6`CTvxnM<_^<T^e%xSOi
z#*DPqRx@wj`g{ENMi$V6d+bvV>mLy8-Nj)3Lo!qF*i?u0K9jh5rQG|Tq(z@Otk;-y
z+DN*VtY-|f0f+UvoX8?4vsOB+pMgSVvhM(g^>YAn^C+*8JIdQ0sX64hz8jj3P~2@p
z+!0UihL<?3hmhpM`#Ch?(Eny`cnOW2ob9mgBOPCcMY?_eOE)>2bmO3EC+WDawxu@+
zcm6NoI9YY$8ab?6NjT3CxYd*U$(<cme8^Y$yly0|lS%F>KK4*8)M0&|X~&JU=RIuW
z0vuMH5DK3^jie%(gn72Zx(w}1P9P@4IjqYiAgx8x6tGm{kuh^MMY5o1NjJ@8T9@Xq
zayf;`p@uxB4|G^F$TiU5$}qXo$2zQB&0|Urx<!W_)-I%*ZqRk{6nk>M!;1YOd<)u0
zN|hpv_ClXZ&Zmf@_Z(J^>y@|>K?5~ENr?Wd(lOuJ(_w$0K7R|+p&I40xevn~Rxk1z
zK5g}qtQ33T4~G?fJuYk+Viix)kp~XTBPK=ZNtQ?DNw*!A$lsXO&Pe-BPwNP;3v*aL
zXVNzJ=}Pu63qGZLw~{?tTko*kkR@ldJj+)`%Yzk1t~x9@d=ix}7~-x-aVF)pa>_+}
z9hO{D-ZLn3JtaA$zQZyXNxs#Lq`6XvQIZ;qa!7qy8)F?7HgVsE24Q&<Uo_akBRAnY
zR!<WAJi*=PFoRYc;ve*-m>{y{3BIiOYG+FGJLV7%$dYHUJn&ewsFg$f$RwPuXsk5x
zV|iMVLwv!seFobXo}^w+9NY#JzHG$gx>_+K%tx;$G9Psv;sen6sRrE#6^#9`g0XzZ
zvnE#!=qChc#d@c?^L>YSn*v4{0^Y75;GGHr_yTGV|3MFb?pB9*ll%^Y|IG^g{BpIv
zp3HsGA+~iV9e0J9p>3YL<}P;#PF4I47)ks%n7+y2yz3A+Y!UvsMv_YwgFDU7-bLiN
z*ZA4HYg|<?d)FaeHVw>c<PhsTns=^sh{Y6PG%m5&BRzMqL*y{&14BBumpy$s$En=K
zEaaAP4l#ptyfQ)CXGl8T3$)oDH^?3D5EDTc;BU}PtU$-dNOW(>9p(_6Xa#WfNpYDT
zi+rv-c#tRpMj1(IG6^~C67SH@4Y^ufIL#pjfC5$62D9+uxekH0O;A*63p|Ov<Q@nB
z5_^R*6`ZG_ppw1@mBXjJdnr?%+2jyarA&~mgL?;?0Cv<<1JHVZL&BQj4iW2_p;z$+
z3L~g!skcsaL{(}VDB6LQBvFY5s=bafnzZPogL1F1Wp~FD<lt&*3QpVWInXNf+-wcV
zoH|Z%e1XDqe-nW9N~X6Q!Ee@v+^w7u;zId-8pN3-a>w0F-c!I>e)Tx!LiSf2bk9H~
zU<MoQn{{QJZTZdSAmOxcWR63&hc!Wd+$?dN#Bx+_4Z*8UnQ=H39}LpCDyQtukX^FJ
zv4OJVb=f#|p&rWKz_plzbFl*{j_IV~PM6O+j_L4AV>in2w5$j2Mmch%bHJ9Cnq#3X
zcb)}q8S2Plp<vP!$YBcn<FdH6%JQ4PKygeW;TkH*s_hs=B`7Tj8x$#dfM2-A@f7ig
z!R(vwcnVs;0LddaJLGox8bt*8LM;+oOKFH7Zr~mjXiwHk9Q9cXBxw)+!KIG+Qa9r8
zmHJSkAEPRc`q**N<-Qf;2qhC7F=!$PQ6ZtSLLhfhs3RIE44aKG4|zI&JuD0Q4wx;^
z=C4Q5dKw;{%tyK=Zz<)!DNAFPaVHVQdboA|2~>cRh4`R4`6p;cmfvi=Bq;x+6c3LJ
z`qYoNv+}n}$57}d4L`|$&lp0g%3B^Hh|i(&nUiL;gImZ^m-7n+imh>I{%p?ist$+V
z4!QaBg7Cje)%^Js^xnq&1#+7K@t1n#=g2s8R_&#0$w7w!zc+N|LKz@s3&c+h&z}nE
z-r4y>koBsokipqXKKor|KNO^`%1;k6*fQ)GEXaqtT5upg3EW)c2J^(WuqZzXQt^Qe
zJh<o1Pm)a=@h*SoC&dHLeb45L^OL3AAio(?Q8HW%1&1P`{)GG%a#Dc!fIj(r^|Nd_
zb-u`N&Dv!vS~xAgjlY7ei#581y-BoRy8XzNy>b0MD$M^8R;s~;?wwWbF=`7m?-xVE
z{5cc8M8G}>08F{m!u?ZKd(GB`lDEcymbk?IZkYd<{>XmtlfYTirz(Kr2>50qdJ`tg
z3G2*A0MHtA!krpy4;QuEql4{HH6rey9L5KR`F~ck%+lrvtg?x9XWi~#d-8;rC(`gR
z|C4Ae)m|2phM^^g`JW=RQA`{*{pmRXPj>;hRb&s#%$fl348b?V1Qf^{$kSQu#Nz!K
zxOj>n-yszl1y73{#h*3!sd412JE@wz_JqD+{?DK$mDDovHl1jvSf_~UVTK9MbR}S#
zu!K-uj|t9h0AoES6wpAJz&@o`d;ZMj&!!$&G|zCqRn1;qd&Uquw>vnWF~!a!wA3SZ
zeh+}lJz^I?sVTN5&u2rpEmF%DVlTcC=HDLs2{eB95qM{38eglip{0G#ids5EC~YgM
z=@|JRHJ#AoRVYAyHCz@TrYA}yO7;1%?)>UlK>k=J_B2(5{6#kw=8xe`)No<eDZkj4
z2J$`x;5*7Fgb@Iu{z7Cj!o*&Rb?+(3i-+1EhZ6iQ<KqC<y|)0&MVLe2Dk9GyEJjU%
zpGw>Cnz&8t9Ogfo8h<l2t^!%Hsc}7kmradt0@yLi&<JM-ds;<p2Y}gcYC8_#Xt}nb
zQrpNd|1nTjbG)hS6AfhNOl_Y7xNd6u5y1Cm{?c)4LF~b=Vymd}4<KN;%%4DZ*cV{m
zPv&o&VgbuIU{I)7If|N}8Cv=Dj_}o$qANjVh?<605euN6sWn?_#qWQ#ipHQxHnp|`
zfK4k?YrbT`FSTNpEJFI4T2W%6r><2ILjd4KTcQf<QjAB)lv>>zYQnEsd13~_G*it&
z01L|1ydXVBHpnplY^phKswo7ebRT$(coD&k6sct!fURcU)=5M0`yY=HdqJ}s4yP9>
zDo`Au-|0nqn+0sk4)dQ`$<%fZ6!@}?^iLkti_cAMKLPkr&(q=%GfxkJ{KJrLLC0*N
zTchQTtsdsjLnOAmz35}aWwyL<5vTWDI7pJG$QIO<B`87mUxCDd-O?SQz0r#W^#an@
z)7Gn^Comde<%5sH{P}(*^t^Ry*(1d!cZXW`P;t>cx|Y4M$ak-*h1S?Y5w-V4atkWV
zzcJi|eY<rQ!XyZ$TkX5dlE<&C;Fe}NSc^fk+$aa@O90jy<zU?o;5DPv_kGGz=kI?!
z%DN9Udr&MI8>;m%!h0Y=OAQZ1c|`M39P1BZ{$Z%-+IJwBw?(bz5zY{(i^v0npG_~^
z3!UJ@FTJpyT@mVV+J;fg22nN}0Y<P-XR88$?qS3Bmu3qyEZw_PiooxGEVW@@%ob-<
zpsg8zCZ0CF_Xvm;>)&Dic)znyrB_g6Cxi~B#y$Xgn;J(07-=d*hfy|7>}9`Ev5ni%
z`PwF!8s`8&br`K}?^$Uieie*-@z5IDo{Fdy9g8i`)LH~!si_sUV_T~`=1SFYOxso<
z@c}34_O{&sc9m=Sf<4LJFQ6qP%s+uz-Zr(o3(AA0mX86PHnpHHvt2jyc30YoU&XIN
z_htJQG`BTVD>^gVZ=TZJX9Klj^A7W`4~=0lhDJME;1fe5-)>~bH%?^z*)birhZq{c
z4V$HmiiK&%=wpvE6!PPb_9RTEj1gg<kL;s^!~7di<MXCQ^l)~3&Q5xa9rb57>(7q*
zvyU_e1in)drq!vCKRc?<ZdRQgy_bESr!UzTA(hn+^G`~|a3$-^z62!PZ=rA6S0bzc
zK;N`)M0iE7v$QaK_15el?K==&H#74NfVW`*#`17}J2N~NKXu}f;SU~!`73Y<B^A=b
z+qyS~*_&x|VH@T|ez!0ih?;B69Q<w*T4>H3{O({v5|wZECVt;`(t8ub`}W<IWqccw
zL&E&-qRi^}(0Hh_qvNz6A;6q`g1|9O=Hx2?H+pIMD7p{zjcUU0e|lm24<Nd0<_uGQ
z`|loa-1k^&eXMK!o?4}wLo44@=A{@~xyJPhG_-o5i|~pte51q%4ZTe8-+$cED-lHX
z4Xs{n0krmbV@Wkumvv2;-w)LI1C>x?cZ9B{#-RWPn;NqLWEs9u5&>0;_0@9U&=jxP
zAed=t!`#HHd~Q-Azh!0J73TLNv_){rBeksn)iP7tD*)D++R$3PcAEKX%0ex{?|(WH
zuY;g@$55zwod$HWJb%atEIbtEcaIv+ni?;I>Y}L;Rp)iX)Oa7jPo~E1tdkPTsMy-P
z9)Sj5Mv|S1w;jtHtEo^EB_pNAkHY+Zg2u?UhDL7;e%=9wMsF1W-dHL<$Gbj&cvIs@
zsgW`&YV>Xnnx>}44ggX;j#H8^iv_=6aeoqF_tN&7N_&ISoSJxJYU0hQ3EkCuJc2$m
zK?d`sQe8s%EYo`iXv~R<_d)<@>P8zWSu2&|mky3g4We$Wsk9JXw6}{88j1G{2+x^X
zUj?wy%<4KR0>A&w>YJe1VP^FJ01VYePgZgS#ESJ|nBVWz_=2hNBT&hC5*1!R&*%M#
zsS$PSeZzFuBhuNwEO%GM8!gh?Y?0n*jo!bO=kZf%Aby$0KcF$noED%zP<-$mRhdPf
z004f5LO#==`P4S^_=OaK-~Z&%2eVP1dPW|78Uski<|gE0B2x03WWg`SKEH<^jqO3z
z4)>Cv(wBV!Ytg3{LJw2dL&=C=?Z3(fgJ__sECaw8$Yv+nk3aliY($qH=)h}+`CW@f
z5~pe$r2IHli~7RoP_NOxFU$`0Q$k;RI@IgE0e)+Cs7?JoGv;Sh6ETB7=lJumB@X_Y
zhWTCKkyUhLpj?vrU1-Vf*S#dlUQbIzHkGwhT(7B$1~#`$0g7GJ^w9QbPtn=T2$nKF
z(-Ec;z~JHo=k!^im<%Ho45rpap&iNaqRH?qXbMdRH-Ob-7*hutR!WAV(C!*J-bV*$
zEy8Uk>lnW7#0CPi%GWO->7o#mtd*^IC2|Ih1oAtH#EWrAImbZ$4Wz~AcZti@arFWv
zVPKg=E-{c%K;jNxW$-Rj>OrDpMD-8Ai^}_lHbM7-8c`a&hyP1hFfn7?XQS;QS}2e=
zrQt(DoB2YT%xMxbhUVaNoS7J6NBN+$_qmF2Sy5v25^Ye1xvqj=WG=J?vRv=HSx8@l
z@GG;B?g98gmMV2Umbx-S@gXe{W4Cgoqq4^ckEkruP*w?(7k+L6%Jv*ZuKcJBzf^{|
zs*0FakiQ4ka8I@pUM6qd1Fe<NBr4UDC4)hNg&os}B{3IIoPs@tIKp!vrym|uyxToA
zNfr$Vm-?WnhKJ&F1Sl%RGp>dv#c(HuW!QN|Xpmlp!ta!#*k`Z|>xU<0DxpD`S=C+U
zPKdEbXwQZQV_>y6ZNNjtP_8GGPc5tHgTo$_3ICO7a!Q@ip&{r6;+r4?!=Aw2H3;)-
zDAwUBNVY3bUFS_N%V3mfyahDF<6%73_Lb|QeCoW3oDOe}eaA%zBMEfkmJ5K#*!i*!
zPXmOZzz^?TwGIcD-*ANf1STPZp40E8P5|aXRSCFoRUx*Rskulwj&KCMq;>e^G7+$h
zHC5+dFAH9SB4RPjS%-fE0<7ZvKBpzvw6YGrZ&~G2xlB}9r&wRwA~Y23D%g+y-8%d+
z&}h)|7q>vYELd2JILtY$BT%n?@EgCMp?L)A&mYC&FCcG3H2KNB6MQ9-n`Q=jS@0eY
z5r>6>b@(9=qqFqK5uSDUNir};MP(uoLk2W~ZYr~crs`i0p$>tyi1bJ3P3faQDic9x
zDILFfOMsW9Bk~w`4bn%$$^B7O{@LIfjSTxgjevKUSx5hjNIt?M0vNpgixHLs7=r=B
ze;vZ0wS-XX{<{!%!Vg8jK+J4zLHN(`JHC===s2Y|&^wXYMu=3or-*Dwb)ci*n+GCf
zuKe|4g-{f$`{g)$Z8TKjpUgfiqE2W{KlsKZSj)EKe-Pmv)JOy~%7*HH8UeExxZQ}a
zP%JMf=!hfghejil30NvvN8AGe+K&H4vejP$-J^yg*3Jtz2|%MZ1!DZSK#%S%(R=M0
zP@`w^XFPv~k+B^MA`fD|L%2nt5Rrcn{wA;*M+~T^fXW0mA%e;XppBz3I0wK{1Nsnv
z`2iykhM^`<Ac4s9NMzGAew;MEpB7q&*55O%4?q<KWYdU#@Yhl8z&Aw}&KNI&Ns>tT
z8ib>IfhY%IeoyG>EooR1W4!QEGGJyRsR&^yf$@l-B?e$Rfb1diD#Au;7(**H<TGF|
z!fryyXTXODA4C9H&7V#Dd6PfLkIcsuJs(lB0~7ws=x+FB2NqHtO4xY?lp>smM0Q>Q
zHxa(Xn}_hijMWLxc$F!Gn<|PWon1|$wi<m+!ZWq?=PU#M1nKwD+y_p;GgH_G##r18
zf$s@B_$Z+OhdhZW|0;bDx=x-oeV&rw3WRR?zOuKEOwOD>1(@m{%yDdUC*CSkhQtA-
zcz>U;4$Z9rr=5xDnyWm6*+7-WfRd_7$N)M6KqIKK90OQV3X>ZEdKm$uh)9YslL??(
zsIn8`4FHK~{8ipTcpK&G%~eVg+@T8kpDIUkVZ;*l(K(6`6D_mxn~ip(S>8S?#02)_
z^bOtEjRd@ZPk5EamQgT`{T$&sRnk@uKwIIbRqik)1(9D7ejxyx(JLu|oQcuqK=1`t
zB><ZPvFr)NvPY9f2et&%f=2&t7#-LZn2t31^8au2|Ja<AU#1LArp-RKggp?D=%kiC
zkT42?3z?*?h`_f4vk1J1$PxssdNq->9I@vSRzbgyjr!jOv=Lz)0W|nPlxg7GFlOj7
zqqkW<6otrcXMK0Q94&9(MKMau3>bn@acC$<GSG`wC=om~T#|~7+{FoVc<CYWNPu**
zIelX3W!~th&LzynJR$K*Wg+-+T$T7`Yi@k=XDNS{<EPpikObx?SYQ8>Fp9?(NuzX0
zqZ&|<J15cJRNIs=rXhu(oYyG{W1wHg{pTgf$L@F@1NN~{m$cASm$VS_qvrBQ#_~Pt
zl14$@=_Uz1rMk<yx}Nb=hd5e5&&Gy=7Ml}#w#K<=Vv^lg`x9E;UYgLGS{@*d=F?Aa
zK1K$U9zaF7;1AZk*g!1Tc=<bwkwcrRKJ~E^H*!8^3pd$X+nkD=sHi@P7zyy~kP=vm
zaHN)^TzCygHaZ~rWN(l^zkPHmRrN_io$?)uoJ<6TfHdMeAj1q~ERa!=v**l$(1yO8
zZUBJ{1o1?wV7}_ps51J6`#=mp=mnQO^D?ZEZ#kWxGN_C@$clo3Z^*}6rl+fwDxr^T
z^AIEVcwGLp;@JUxaGC~7K|b{|$a^ayxhmvmELAI219{sTkd5XSGzY`lG#j}<fhT9=
zur~9qau6Ndv<S3Nw%ey60gg5KpE5BDy94qUHmW{t;H$|^d1YxzWm0I_eaggWOhM%@
zW+bZ56q3wC1IV5Lb&C<O3nsEhAX1F*ERwTlBeD%)t1=&^-76Dgwo?Rt(ZQ-du1Z+!
z1xEsLL_rlL&QyJtV*F_I0?IjYCIkdy<P3TrLEA^DJ059kiGUlRB6tuY(-ANi6~SW>
z*^Tf90b)NxD5d?utvlf-2RiOTsUM^Vk_KKAjP3;Y2(d99yrN8GU>O6gMPUJP7<NQ3
zW(vV@g5V(9QlydRb$~4p8WAW#WC+3l0v{rRaVmHsf%AyGfUuIl=ZL(Au#*5R;2046
zCxKs4(CE~vVfHE)y@@Zu2su0^mZ(0s9Vi-Mmt7>H{Hk?@(^czCOG?0+xDOC!0@ViQ
zf@P)IiR?5xk#w^YDXZ9t7&VY_fHi<${HZ=K;5|NKJm#4hk7f3cs6MYkLzA)4Jl+Wv
z4usrb_<!ON;Kzfl8j7LX%LGm!@*Tq01TJE^R|&uq0yhBkM(9caxvrjzFzz)}M^=|I
zk%iKMlL-z8jn`sS${G@ZpK%Iy?z2$liQfQ!9jtr|xH@HBuY^W00kg4xySbY+wnugt
z{}8D&K!z+%vIpgF{BRNHaaoFvZsM(@Zvu_kN){SM)(oUiSjMRu^4H4kh8TNiDe?Wr
z_9(4WnaFG}%iuTF=PM|0f^8y8avHS%j({=9k~|6(1^a49pZLuV3<iN3%@7*305DHc
zlDxr)lCmTt$2BmssnJbo3kdVxiGIj61_Vnonn#U^2yE{t{;W5kT7&n3Jy`W&X*R8c
z^3OVr<a~r&0&t=lFCn~ulE}oIJL?*<h=CU<Lj=16B`L;`kYP%|Xj5aKDd7_UA5+4o
zrUWzuO3+e=mWgpGEPy2Rlw~yHCb&{jq&4m!V5%lkp$^M=r3N4ENNNh2)O=tf5yCnF
zST02<IA9h{k1_Q*q@cyr#8=?}ZA2lYLmfxO)U7}dLcjt~*N-F6iBqKhE@~xQ&Jij1
zsq#0_OhLeAfJlY6)uhHn1kl85u0klLF<l_DJxr^)9bqE@OloVML^y%Fu<cdrs!tVS
z<AAw>aDf17uI4?2Z?Py7?K`4d^FhF8wk!=>!kV?xnt7`Cq^gQ&z7rnM5H%VSg;0k&
z=b<tiq8)}{))LYb9?)<;uxk|2438698s=en0H2#w4F^d}f#ik)P-vVMEDKQylcJ4I
z-I5Fa=#J6K(FQCH;TTD;LnXbXAzUnE0m3|)glS09+eoUzr2RlILnx3*n3yGfh@@6b
zg71WYHsmcVyDSZ#g)-zElXTI17?W2b=|hC~W%9K$q45h7GJQVNzd+ij2$z|jjA3(f
zijnL_a&mPf-ADM5*_eoS=^2~k%~32>MEl_|=M&W%?>P}E95|CcVUcI!m>$DmEemVq
zX%q*M=w!57ZGcU}QEq#9(WKl3h^)rI(-CLllX4Z7BU+RasCvhMqeTc>^&~WAjBco{
z<h?KuJNZ!Zlj_~4G6-<;VM#`b)Iy`IHJiW%L{=kkVwQycT1zr!@wHw@*ordOl79pm
zonVW(OezB!<4&!Evi1I3Cgd~d@E}Vgj2gABBfyIkn)QM{#<tkT>U~<rZG`o^AnS-(
z<r76WNsQqyV+Rs0AptWv=Aw*COB*)xCqFhMu}|aX2c%Dl({U*_;?P!!Lo229l(2H?
z+*A@%(0^s@&?W1XOAYQWKyo%YbBYvz#T%yuDI-nvQk)us9~C(x3;3D9v%yH>PYq;2
z1>{utw4{8wBC?GkcCRURI|WR|#`@R4L+Z(jv`Y*DUl>S@8JJp1tVt#pXrqv)lDa6T
zQ4rJOJmg^wZ0Y$x5iO$)H4|?OuL;lFvY)qElefpROh?V9C5+^waIceTrNK19V49R>
zGTAE&WqU04Q9-pGe77KamPwT@(eD||t4(yi$!tx5MsmV($%#Q&mh_~x5`EX;JZ7Rx
zh#rl0*j9y}n<o0KZ2z-miI2wm^l~Gl{%fXw(Lu5FDiz6+7YqBB$0(u|UOvwL-x%~K
zai3C*PBRiy{+q~cNHH3{E48*7Xv%#IW<#pB(ny?aBu@5`JX`d9;g+wSFLV<WRiclh
ztu_J8=_SOVU$gRs4=DDGoeH*mL6>nGQXfI(dC*}qb)#XeBb~jVPda5~JR}VoeydxO
z$R<UM#TYW=StO5TU7}7}TTKg+sbVa4>2_RT1ikdZb&Npi%7%=<xS``1c@I0o%PHd&
zWWeEDcQa&+l9?L|*Our^Av9TQld(LUD4Nl%vA2!H9D`?yEdH^-8R%68dTIozGeZsZ
zb_0#;^h6s?qV)+A?I$cNdW-f<#0P?P^{9qP8k^^|-cT=>)Y%vkxRB0MNRROX|GIOu
zC*KQtp|+_5J``%HQK*Yiy4iYlj>mYz=~0_zAhEUY7F~yo<59;F3pJcH<sX%krt#~5
z?nY{q1Mv<6|DK`8rR(uj9xRiUhZdWCmw7-s`X-oBeV%)68+#+|PYC-o1RW$^5Z4Ux
zQ%f)+w0YQ?E2aPW;S#jX4W_pZXzlq?77Gl1irB!^e+J@q<+rL2Z>u%Z%9NQH^OW9P
z!nOSXqgw4NNNR=|C8jC0*m2*}fPG^wus!spS?x-ovKmXXdcOM7tTx8++L4@-w*3Gy
zcS2^U>JtmUYx^3e7i}?K)NYK>0OL(t3>CHeBJ@BvrOaH*2FgO&EGC&vQ^BBZ{a*D6
zfstAxjbiTw>WjgUP-^t$jTF^~p8>-fas}op#~-LZ!G6%2im`IUpQ`r*|1#m*8`(}~
z=2BZC8xymEZbbmvXCPz)xhz!m{xuTFc0jB}cn-*t6{`1dm@s46swuS(0K2;umc0q8
zH;%KQ1BjalpG9WR$WT=8KXp5l+W!Fg7bx*v0`G@mic*S^N3Dv$O%<+CQwPvD=lj8f
zeCK06-&+yw*u`5gv=WDgH38R-+kNE0wHCf;dTd(legcy0L)h2oa`cYE##Zcm%IKf{
zFctb2C-tc{++ViCb75#V|FW_A%u&F6Ah0eQhTLVc_(aGJkiKvYyz3u^;+1BwQF>bt
zy)Cs9%F6FK)%#m+F{JYDb2nu8+hmpvfUau+w3SuDPyeajcflL@b#>5g#KJqNzv`{=
zggo__Lfk$BXP|hyn<NIvy_i6sdU%WaAk}-ctV}3vp9VeHG4RG$=yU)(1>U2(7yvc{
zyvJaETJ64k$V>Ik0J*%Ser$v4Jr>I?(S8v0RN13?XQE2$mIBzFp?a@Hc9n(zR*hD@
zUo?Rei&SqG1Zp1EG}7>+G(;KH_(Sy$XBo7A1uU<ED#Fy`xUPCfVxH9g72iyhKwa4;
zMk>nm4^{7aBjG@EAF1AP3QPAssCLQHt;t(nJ$+TLyRg0PvNBOM1-^QKzS_~6FlYW8
z1xa<71n)uXQdPKRU?!0GPX|={Aasm`MlZgRk$WL(F~Z_VzW7D;l5gH1o%RWSj$v7n
zkq<4W<eu?V)$4aG7&|!OxDhxy5E;*b=n2B#m`i4Cz+4gx69$0cEYv9xi&RT1Og=l{
zf`XWU))^XvP=#(+^)9aJ*%1jI$|EJ|##K4e!DBnT2)EOW=~plioUe6wlfFGy5(Y0b
za32ykAFT&+Py`t}&=|1R*BQR7z;N8aJun#1Zw8+w&Y)55qB978q<UTU7wGEt`>2}N
ze6&dPbz;~7Y%U*R5qF-KFk|9Wl+ye*1dx{qgxY#@_R6WLcXgpCzR=cS-fvgEa?tT`
zzbdpJLf=S!8d~*Q=&x5s=x88^N8%{eYayM7j4p(kkuwxc^~w#QvjV<_a0LSKnH{gZ
z2wim;y4$csAo0B&uSG=W0`V%sD?rYCQ}xm!(f14h;u69+qbKo-kUc;e)=P=w7160m
zWF|JtycWY@X=d0T2oKn&Xu}>W0#BESPEjnQ1+w6Wch+mDQ%~Uj+oz<-KE-Pp>jH;s
z`kZh0P=kJwL9dVc!_lR%qC0U|qnK=NoEV+i@B=bY6P-A-k%KS><oUE|_@@Tn+f0GE
z#N9J+XaA#OL<0l;ElKq)@;Oq-h~7rZBc|y28ZlDGbq?|fk+aben8(o2&T$5QVjOjj
zxNM+PO>~+>8}pUUgH1HfWU$?tZBS1&(KA>`Bj=!y^4VPs&F0Sy_}%!as=W*Vj-W@2
zUY{z8zuYEG8<vnUYl4EbBzVLicxr6~uY`<b;u`Co6Aph2pPrD>s3L9)aO)B>8dt<&
zR1H7vzS7yQY1LKx17i@X(-i1G;5sL+tM*?FWCTXHh^B_?r0Ijll2A+wJCupu(cHzt
zFE-g#JMU|BIfG_C1RfWG1#?6eH!UUMZr+sWjbH3pdU4OP%da}2rbOg}GMBp#Y17KY
zQ<yVh`?ZqlHL;o^x^@6n8VS}QJO`5+R04C_azHx~wi9>>k>d!5QKJ3n9%w6&Hu!f)
zJHSq>L8$75k6MVX+o{tVq_+{iq_j|ODGsToD3Mm6{vv=T6PbVzO&|%8E(odUE&7DW
zgIjovobO|fC-tQLQYMD(RA_a5)oVEQJcgcr5EzNRJaRbo@a7?(-j1ArF#a4y18%_$
zJ&QhGsb=3@wJ8uo(1Nl3rW?Ny`Y>vNwWaE1h0t!?JsOORN1~D=XHqDhhZ2L4`^Xm%
zRuI^R$Xf`n6DUFC6v9UYK1AdS0(V>nqi2r%4&e@WTn1l93~fHLtP_&Hh2DX%!3K@b
zkrs4yLlA1vVf&=BLtt5ap-&CW>A+mD$U?7$w0qL-G}X%sc6aB^xxw`z0mF0MNZOr7
zxxD~&#U!n6b3#J_xe)RoyDu8c+NI`1We%s)sk-z*n~-{0Rw>4zN|cUEkz&w3aK4GK
zgVH%D4LSnoAi_HY&LMIV;TUqzH<X4ma#nhZyt6qdtW2bhlU6%bFF(rtopKw333JZ6
zpHi-qqS^p@f<R9hgmE|uttARKD#W0{h$JG!6UaoQ143IA>7YCapMeaZNW0%+>LR3|
z1xNM6G*%O7n9WU;%g8w(lI>tSMjTK>=_=K0EX?V_i&trfz%v11Jk41JNomMc6t*Lx
zW)irJ$OeS<=y1|r&^w$SeT@!h$Ya)D+D4;S9)`Y=Td+N}c_1(yTCOPaxd$+8=>Pb-
zk%Uo55d*J65<Fi1DoPvGs|swyxiOON!T?l76g_a@?}(sfMqw*f3_?SUx`uFt06Ms+
zZxC*3P?m;;A#OyNc3{*uwV@dgoCJb0Ss43Nuc}n^6%}Eqi~0rOCl&_JB?cBMN;Kv(
z(O9yGfol-Kd?%W%c;IG4;6l-@37`{-hKEPf3kSZhA%-%I9#7~DBC`-?Ad@_5>F>vG
zi@!<()s}{NPtQZp&7+oRRAMx@gn85wU4*a{5_r@Sy&B<pmUh23oToE_W%9q>kkft`
zH8`Wj8Xd*K%GSidWT@yZGk8Mvs*ViyN@oT;Ld-^lSD3*QG_Nlp^m@@eUKxn4B^tAm
z=>2e?A#kz&*{l%!!qHS)cCB7OX8^H8qL=9CDY!i%hD-+=swMg;kof+S81M(^ZpvnL
zMfD1y1-od04`?nS@Z~-YSZxEV145xke@~z;-b#j)dj81a08HVAJS)9-z<U7GrL;S$
z7e4tdda<t`a2})$5aK8eJ&zdhIUo(aYrVFF;5qfO5XMR={l}Gwfh8=JM9fW^KuT{O
zn)gRXQ*S=PTu4deUUGlbbUkRUx0nEGy524X6caP%3n6Q1N@B3;l|Uf{GzIIcdgvYM
z;U)-<RD!7;J$^luNxh>4S|C!2aGp84D_aWven2t6G9J+37i(YDE1V+VqsUDlXMV1e
zpB8+Ja9eWqgb{7Xg<m`rsd`0&>!~^9;vB!;15k3Of52=+{z3S=697!P2aaQ{fi{q-
zi*dmV3x^nQ2(`SPhU9!!Eq*!r+OEMnpTf;k%j=^+R1Kjj9B$weAl9R4Su2a*<Rp0@
zwiqU==pxX9FtNdo&r{~iY}`G1?ZB1zR6;O|uitNgWM`fjsd_bs)qSvsEBa#@6BCUP
z!8(Y9ZT%66>4eaNKm;PlO3ZWu=n`X=Arug3g~$$stpvIuat`^AIf45;qCc8sBK@HM
zAmHyK`~({Z9nia}KGi(EgWTe>_~a{0K8_Ld8qDI{%<?n}t4s_YXRvVL`ac~TlW!X=
z+=9fqLGE}CQ4}$*yIkD*p>gsBf3l<Ncc@GZ>CRHdFP@!Lz3|CN(dPpcLcg;x@e#s9
z*xV08XskDal>ll!79JK`6NNb>7oEZ<bc24q0G*PuPC?cT)vG6D_0@1UPxM2T#Wq7|
zLjFmJbVR_FA<++meJn~Ob^w7Th@>NoCV*;+os2LM>#lz5vF<{34?*tqjOZsEs0LY)
z;h1Y+LXILHrh4_KUMJNi<B7Ofw5ZrQsKCTwl-d@M6M_Bm&?OuqQBItnkI#lI6ku=2
znScpUU#`%MxpVJ1`e@U))V-pwJ*kf|`1JkG;LY4EIgPn-Uvoz_s}0dl*>vW#>IQYz
zN}`9nN9Br0OL13D!=0{;251gw<2TY-Iq}NQmn&rKKcr2>OhsuJ$o+@3VVG~cby>Av
z^#d*yE|#`EP@h6yjpGy=Ud(we$M!}VfT(k-{W^*U%PnwfF{=Hbhx3;}O=mq=k@ek*
ztoQ;rSmEy6tnscVmsley$@)mOZ#O0N3pZ@oo>;+#H>4mEr+lm0U-S_7GKgP{H1{HC
z@yj0S(vMWT(?h+(pmxSo5bZK;bXQUBOX-I@e8<heTA;8N;=6svjh_pk2*&q&8-*}5
zNwq(#%f}EsWD<V*T*ObQRtW1dyzy-^kG|gweKX@L=$mcoV{y*$6leBv)!y9X>_5)n
zY+jKwr6MQWS^G2?g#9pkE3_tjr_)z%nb}*Lbl6n0w?U7HZz>qs7|;B@jUE)Hf2rD|
zJPP+13Zt+?)_zbtb|*)u_IkBh^Q@?KTMTE2F=f;GSdaC4=BS$8+av8qLz=ggX0TSO
z$XeOMI%kS%``zRm;Hz`mewUoIGrh8E`^uBv#Yq3Ef}7dCmXhhB(>AEKZ?GO8(1mNp
z!SldBHAuCc@$eTL{AZ$=&sWjl^>6FwHrhClH`0Q9x!$*3!H~!GK7?ForrP#-guG`6
z*;mmBjn~gW4(eeAs?F`8{==YlSEPQiLSfim@)X9XZ&h24$vH6D;LNE<V+|8=%K>Tt
zpayJt9?s?3KdLRm!#c@e&8ScVw#<sG<2<bE_o}u?59d{bGg6k2p){%@Cr9V@X}Yu7
zIKh$TXg(RLEv5!q0^ex}*R%0c1xhM6?rd=wZpU6^*)+4LHeXZOpfE$3Z^az?n<AOR
z0I3WTC;YA2s>l?Ixim<%{^1d`3}Q5_{jGmg$gY*|zcWk9`d0;`t-KuGNb|*}s`YPa
zq|qx4S|@v@DY^q#pXln$&U4T@*?IDFq+-xA7*T(NYUQ`GWap_3w&|I%o{cWAF6;S<
zMtmZTK<n!DrD`q3y|BUZlg{h5BYy{Ns`UcOd~mhVdKYPzZ}x&m0VNg7FGLPl1I3;H
zP^}wG#e=h?UJWRz7~=E4Qmvb6f^n|Q&m6rLt*=(JWwW%!%-i6(QbhpsHaLqaE~wTi
z9u>bES({R^e5X|uH(je(z%yi$EZ~{4fMLO`m8x|Xcks9|d}ajzEDe>0Mp-YB9I~;!
zFiExcHdPO4W2o+3q0_eZsi?ZI)XXNabc<?D^TR~D4V%BR3;9@gMzvxRic=R1a6>Wz
zr{N71z|^M$C@pcgF-x@ubMtvfo_q&2pa`(;mnR>6TB+6mxO5|JIJW%0Q?0@uPaVua
zx;)kyaula54VLL^GjBNwI1uPwH^d%3@&(YyC;Er}qi;)D?lz%UVB&xkXza^!Tgq}5
z^Js2p#_mVhE4QV}ap!<LEw`o0akqiH>8>#p+fvt6%MV7G#r_5KpD0Z<7|V}(y()1w
zMTxT*j&0oDTZ=hMuie<v#YIkeDrD{<tW^|DF9Uk0mNP6kJZ{sA6|j^Wy2IBRIXKf)
z&p}*uNN($|XW>kWo`pDMC9a8{1rM$-aJ}>_cyL%6#$~$u4YRv6^sR9W3t>9OZHwOf
z0`WXTv5}wiW`4E-xfw`wEtXHDQvq=h;T<EB!?#Os-;Fzmy|DP>CA#60z|h`XwOlv0
ziQ|rg;i!V?Q?C@XtPHHmhI4~eE~)jsyWL#g_iE3TDHAaJGaJe9?Xr>7U?Um676mr-
zS=DmfD6pPbo+;5Run~ui0z2N^D6orAdR{ND<E@PH`d+E8#NE*g%tWH;DshkX0`uT%
zDoT8?TZ}-RW8yU8yv*b&*x2*o|51~eI-`y8tZBq~X>ACt9dQ`eK0jZz95$^*Iq24o
zylq%}*l^HzZ0W@(=++)K95lWQa2?`wYduI5a{N%;S`ThEaMRtfBXw&Hzq?IqFaICb
z!k=N7rA)P~H4N)f3d4S5F^>MsFl?>ickwGAyFfQ=t>Jd@CBW_0?egHx0(UAQI0HMn
z77r4t;%~ZBM(KVR{}|vyI0yaCm@g^`Xio`MfMh$utE)KERT3}@NQeQFez)H6y96}a
zgtq_nJA8ax9_#eP4G@eC|EQKXFoPUDz^WhXB=iJJcP?}^9{%*is$2eI&7fNLk&>Gw
zJOD~SFOe{SKHB54?xLgRQsQas`sgnDn29}Y`XT3NdFV4ou048o=Jp9>oi<1}V_ldn
zy3?h5j)Qm{Q0k{fEcd}9R7;vs?%g23GLq$<(a9+Hw8pyoCS<{$(R#V3HP_uYVHt2s
z^>X*%wg9(L_XiK|J>U)mW;8)n=(xZPcbhTx1>Mo`6D}i^8kI1>=*$v+1oAth5(XNT
zz*it8{1eG{WvP}yMkUlo&s@Kn`=2p5h=n=z(Wlq1gC>OINXrnU)h)%E&#0pec{seF
zno&pH4q#_QssE#D86K#J^aV!A*GHk$@6W>Rc3fWtr!P190OD>NIC<uRet|gD5Kfgf
zkuHy0bYypQHtbo_FOpgAg|pEvkatqLkJDFk>GIG8%>`1r&oCs*;}!_P!maxO<M6LL
z8cxhoV0sKh<@N)4k12}0)|3%X`Kx@C$~b}R0gGk7{vU;AXON*EJXd39kkP3(&PTVa
zmQG#(^7Zl6(n${ODB3|Ik<x@CCMQ}A|5h#hU|%)96*FUpK8{;j$SUB}&XQ6YiQ0Vy
zqhjV=)sl!OO@^Xg6?r!x<KG0*Bw``KQ`GwK?)sx_puna{#=i)6kpt0IX$KU2oeN)Y
z+*arw-8}|x!o`Kw8mc84gYM85AhJE(bL>DVG8)ZNiY$UihgKUT7}&C|<sfo|&me%3
z2XG!~Q;=|$>=b+IfNH7EvD^n@vne*yF2z=lHpIROvD@I1)-HGxC%#FkT52N`xT#f^
z9yw(;mIoHq54>m^^-p5)RsRIG%}dTh;y~<R@K?6$aO^pa#cr&%<U0^z7^q)L0Q+F|
zzee~9m9I2tBdro+e`Rf=@;RwMJ1!|EJO_e!p1rb>YO&VfRHRI)KLxmT|BtTo0FSEp
z0{y+aSqLUT$|j2mB|uClA+(SL2!s|0y$Yd8hlCnxD1oJfUc?Zx^j-x75kcui1OyQT
zQ7i}wR@B)3qM*LtnYkOl_g=nlHh0d<oH=vm%$YM?s2-TAsJ6Qib6L)=`U~#6N@_XV
zW^bsB977&gk16Rbh65zJ7=uO*q!>l4b{A8NST$&O*lD(+USJBq`psYiUD?1ytK`M3
zlE-8aI)ur#UG5mD`bbL7ugtdDN>cKH!OXVpN+J@ah#6B6PfHcUZpHH{kr2d@rq>@T
zzoQUZ!@q%HKj`oEbEN4YA{c|ph?y$<fNNEm_&V?+c|;4B+*yikRavsoY@0|d_*iv|
zVcYQ$F-*z;7u$z~n?GR~cYM77^+lY!!o@*FDTpVVpj3u0<IJIW8Rwe?iP8m_8F+IK
z;$;F4Q$M}Dni4R7K{v@*I)-k^9W$>yqY3lZViE<|ZrqOEf*K-#!NJPm=3jE!eh_Ud
z<|OX8co6fLi?|Eof%`ExaMwlfWSwMCBYgL8pM!zcSj8P~ek9zN>6$@p$gTQ*Ac@rJ
zm2vo_5I5->l_&S0=ih;jQgxmSHwy>AgR%nDB!T%(Wf2as(9KJl=(=2_wj$h|E5`8P
z=R9o&Ii8JEvox!36F0s}xVfMrA$3HRt-OsQNDT`&7b+&`v7os~NsVXaAWlvH8*VNQ
z5JDyTnhnV36Q@Wb0<9GLF>YjtjmgsWxF9b@5+3};-z^z`ish&1Eq8{Sle|SAoM{zf
z60&X;V~Q>Yt$X>W;hoJ9H18_Zx;DcHzuaAydjwOLY<n82s%w-^VpQ{i#s3I{w=D@b
z2RP9IGF%w^p_R)3+D1*cXX#)^CAZB&)EfmPh>Ss4nqP*S-5gf645?tHl4l$!9I7BM
zCfwxMEZitC#PTpIiP#P4Tt(B=3H}Nfs+Gls3BpA)!_BsZ2*+9o+frEeYzdvm28ElY
zy>zBOYeguHsKE`jUC1m8?*i_q8s@Ykd54hY8M10{G~A5Glm4vK&DsdrJ~ZS870P}g
zQ|46lj-ny|^BXaU!IGJ*Dv9tW<HOm2Us@3>>5Dy`#@ce^MWMPLcIBj3R+U&x?i+l$
zS@>6UI_d37DAZq30_#Hc@UNm3UK)=lFZ@F}E|xysmu@62{6l89D#&HQRpP@x^wddr
z1kl~ft2k5dgrDV@vwt?FQv<T}_vjt{*XRy5{T151lrYU-`dI%R7TCMTF|Cqxv;GIj
z=sm94=hCw8!;cm>zcbT^V`c0QFEAd~0CA#Jm8=IC581UU8yzx=23wfB6o`^}*h}D^
z57Qlyab5eUDqX88f}QiUuXSD_!<MdAQ?$dojl+9N0W++Z<*H{m%DFS%ri5snuF`4_
zmKfA;q7mK|UZ2V<F^J8j;a#Kj_W=2ZFo6%0?|SlWY^MMNT8DShLNfrqHVW@tg#Uv-
z4DVEgOso78-ce=x5BcoA5Z;l-obiv0lLN01?0(xF-a*2LmeJuI%BygK_Kyv3uR;<C
zi9Z?MzP#KLDFtcqb9g%y(ou)BLx2e(SKq1OZ6){WYr@;e|MBtRt>ypDkHcFP5n78c
z4{wS182Kj97dD4iPnF;^!Qn0Bzs()qyvS0bP3<?WDO$xyZUNh_E?ug=A@ZhY%1}qe
zvIE#MXFU7bQd#VRAaj^p^mYE`(g{i2y_xJeI+V+)%W2SX1(Nu68ZS-r#!u3LX)uxN
zN(pS&ZfRR@;AkD#HeLex-n?LG+oqC{x($A5JK8sylIY~vrJ718Sfvv+^(JzYs;Nq4
zNP&*sSXxuzC)rC2mzMJ8bwwvD6=&r&W#rOQjlKAHjb7?sUQuWc`+<llm6!V0QGxJk
z;@0SH!o+6#qC3bwfH@yTvya{|hr7^P-P*gOTZ)N4Sn8|qU(xkcP)(3jyBu8)eHy6V
zxSm)m+InKy9Nta`#-W6n2kFc_%Hrs1kRCXo!06ZpV2m`0;Wx&~C799Gqyz`eq0xrn
z%8af?TP2ZEjbzo@B4WccZm3i?D{ZCd>Q1`4;a?G+iQ_UCbND}eOv6nU&o6u|!^!&n
zh-dg%hg+krSm@R`It)Tkf#wJoRJI!u9fqN&zUMFS93Re|^qeh*)U5f$+r0lJyD!?b
z=;trl{)b=nnj`wiYa9R|ZbToykz@IOSeoyFT7Tg~@<(K--J2Dzfb4DA&>NIT$(uJ0
zvrU4vBR0{di+99>YCDyO=js|fmGQXq{Kj!8-!`R`L=#$0e#gbr#sMO9Rx}hC&pklG
zeT0c4XH_zch$w=Di-RfXF5LDiV+M@|M`w1|kDLYI98R_nzxiR9tqXlUb!Jri0e25z
zs2d>Mmn$D*<+x$29|WE16=r*$j$-iY@`|1wnjRw-xrwb=K4Z)PGLZY=rM6;6;RejY
ze2aQSmM@4&MePb8nxV<aOi%TZj&e5F5l@Gaj%SOUfCVE@dq$09;I+vxS~6bg7iLql
zx(5r4+0bF6N$Ez7hdd#~ueK0E9_w+dJtmpkl$yH?M<SwZfl_#kpCK2+Z0ho3q3Hl_
zKPFUmeT5Tc;ol?7=6t3npf-m5W|}>bgzRhsvB@4t^b5Fi5|Y-25yk9p3?3&GRULR~
z`jrW4-IC4fYFpBKWSpOBD<TF3G(T6se_wMA49mP6X8xeSlycNHIJD#-5ac@WY=Aud
zfro~FN*+pyi1Or&a)gB^g_*a!NiyZu>1rUkRT=eF)AQyil>53c^RkuI?E7S;2HKf-
z6erAULWAand7W?J#7kzF`2mPj?cAbRpEADS`b}d_q*&Sf!j3TWL)u8@0)`ZFNm=Z-
zxUa=?gOA^Fzsjjm_*dVUqi%cf9~EYP^fUtrFXSKRRgLO)#lJ@IPcB8BTHKSHiPaMr
zX1-w+;JHiWttPrgchb^pzCk}F(pw#sQN4s*{xbod-ezxz`ZavR%wrBA>?Ohx)#}fQ
zCx;INqdIm$OT(DZKFr){WznP!_*9QRrZYD(>@Urr2bkS3ij)j9d3e?wy;U$UQ&qho
zE{WDtq$I-xxv-*oI~)jvVWmkoP#DH9!-g<(FZSx_PrPZn6BD~lC(FPyq%aj?PbxH&
zq}u0Z%rGX<n~bJWeT+Amlwu}p$U3v>vW8rwY?%2f2GW?O-V`qqX|WJlmTN}#*Eff&
z(Mc1Gi78^G$a=a+<gp#*W?~IK0pH#tYb}wT4}_I5rzzGTl4CaeN!vAZlyo43=lnfg
z#yFzfHAs*x&2H>g-R#UEa95gL_071o<yErbDjEJ0xFbj!+Q%H>OIoSFo45}F(hyBf
z#3}Kseiz^uQa~zoE+iv)JHpINL~pDrpj7Hl>=UT|gA`Zz@d)>qtgC0wCkbboNOB;|
z9K-&{v8$!A9>=gNibE*O?EO!Aw&8&VH3H)ZS%q~qH&3_XbhD8(-bOMxV$4l4%!zp-
zHhr6fnHYE&Ff`*iNBI~hJQz1r7nd7uRUv9bfU~q!+!w6k)}Ys~kqA&)TOV)C6o5@F
z+UHzSw66&>+4^pd>+7Yx8*${`GDW-gh$a%^2jcom&?-JA;KoTI(7vLIIFB-2&dc~t
zSH%FmRg6jP!pvk<#RWzVot>`9@uR3oQ9tq$Mc`SG&BnQ_Xf>3q)DQrwx5iqxRwm7)
zyc8*ZW0)z=n}`CbaRzsUb0uSuo915N7XWWt9BJX@2%T?@0$Pq-lazB?+5o?*4YXpp
zYJ7ES_*HaNjlXenFUzZU=#J^6Yx=sS>8NC~0m#S^jY`vL!^xQOgREpVLx{<4BAqM(
zPg$wW9A<8GZ9ca=%nY*FFn2c-6#-J2Sw~@msMMg!qEbmFuCm33Af%j97By=22Ku~b
z%6Lb(8$Q)^Q@E@DgqiBe8Q4&hzN+R1fWF9#oV79X0lbOZW3eIJVnfaQ0KWrRwnCU$
z+G1GEKLGw-US>_gOs+>&tgYo|7`2Kry5&h79;}z)#-tcFt=V=Qq1NA0(sPpNaA5*F
z<7_)F62C02v^5(t_mXHy`89k>^i8a&XG8GALkn6s8>0QNPi8}E(cWvJ0JWl{Y0gU(
zO!CW(q+w=FCSWEMNQ<O>)<Q4Vsw*CvXRTH^nTb{^Yu<3FWqwt4l}Avs<W?Wit+kNs
zTF+aGu2z3MeMx28#bnRCY?UgTR4Dg~D7&l`6i+65YK@>V)*5D&kBH3n@GIpjobAES
zY>x#kET8s!%Qq2Jv>)BxPl|-lkmj*-X5AoECfl_*SgEGNC&j1^#IHw~Y4ayOiok!4
zut8~piFuxNSxCez(9NJqm;MoE7W3BL5`c>=)iwu7Lr+Z$GyPC)6B~M2w~{y>FY9RG
zwYGRQ+ngP$eiww9{+v;nxIlW&<BjBZe<RTV68Tl!UUZo|dzEyl2MYCoXfvB{BdrJ0
zR0C+s5H29Bz$!yd7mAQ7@;}M9oYK?3TjE*kG)NCymzm{Ib2K4-X>Jcnfanj1l}1|Y
zOWYl+wYh8kitlF`ujb4Xlw#wJB@vk(*&cS)%5>HuK<6dX$@l4~=^U<>1~S+%q?Ofr
z6e*p^4U2uXLjgM@`wam0p+#HmionW84mZL+l2{BnYA51i+!UeA!?2s)q(cA>u!=AX
zVharUr3jyJb;_hStRmD#ZfZ}qicotlp4r^jqk5uQoLQy1{KGy+&?kLs)mCk^ZtbO3
zZPk7k&l!Ei92P{5%mc`;$n0I}cyd`QRUCa$9DR{$9&y2V0=;*|<;Z;_2ERhvSI{<@
z1s%oSIGS->X^XbF%6Ou^wB^JR-S96T_6@zp<Qzf#0<AC(a9k~KHjVKl8?BHNI?0?y
zsx;NYzLhCL=^x|T;96OI%wrES71tftP4$n6R89wNrA>BJbtg*M)Rg9w&eR?wlrs=1
z1A{oarMUjU7^s}7Zo#l_;zmY}<4jZ7d33mnHx1b2$bsX5UGTyXx44rcC?^hv9h59+
zVNF)>UkwDOykgZz6ZZ<kj*U|#><G-AvQ7#rrM`^2<Sq5bcs>#-Y(i7e#t(`N;R;$D
zwpFGcr+kAX9qSgh70T6|)s&mm4rK2rp-7ac87SCZ?UTt>j?>893WHBnBzblbUP~Ba
zguSE%jo@rsScW26LhE~OO>u;W`G=|5!h@2$NsX`ph2ir(!P$;VQ4hlg)Fu1DkaFf|
z*nlSTEpd1JFYJZFxEDSBvYB6a81@1s@Wz!FwRPM#ri69(#+_OmOf;tMv;q~gQTMR!
zRro##e8|JF?$Wxf+`6MfHHHgchczk0Fsp<rSCcTY1V{yjCbTsp$U@O1rVxdv#8gLx
zK&*Gim#RZb5}pnF$kTW#*N#t>soRUL1k)cR#$YvBm|)5d6Lq$5>M??2J0+O1XJ1bQ
zPL*SQPk{6|xiE&z>rIJF_zLz;YYxsP;}gCTb7g@4gwaNVyjXp5xc`JqBjGz2pYD4|
zpS(Yp^pPZku!I{41&Wkj^6+qiY4^`C68=)k!++eUaeNC8)I7_i*aX?Kg~yz>lh<B|
zxQSv1$lKavyD34kEg^YrYENGGN(cp;q3`5yVw_o@u(@tElFREF&BGJ6Jxjo7#DzRw
zR{6<X&hCN)hWIKEcTLaqX^e5V5+9CDQB?B1scS+u){wg<->yCh(^Oi4_e@x%;8qdc
z%>|v#K%PA^lBa{2GOm#dfumDV57tb>$<d6RWZB-`oT@OY2g^#GJHCQptZLCIVIa(P
zzfAsQF|ev+txG?s^5fDELc3U{A6%)h^n<ZWtkQdWyB($a^-hSkiZ^|TywI2sT?Hcg
zX!5Hn@>OlgBh4r8sXCOaqo_lflt`!^!R+-YVsAN{P@_EGAN!9TVkGb=usMAQ;P-y;
z+;BVUC#UHW#^@4Cc?VufNG+}yD>K7K7m^@L71d_6F^ui*cvBu#n|@u1r>DboWc3bq
z#q?K|^u?R5LV=wA*3%#@kcBZsAbTx|e;aHwo`+{m#>c-+Cl)Ag7A4$>Kku!^hCAcm
z370rlb@l5k>68@4n+}-ZHsWt8E|Bol=J=1L3UBc}XH@(r-tR+i$A9Ym9(_6fA3wDa
zR}zNR?MUA~J6=!vkj(_)c6{f`!fgYM;(U~A?q)K4W^@1&%XhReJV2xxPeUrU-W|UI
z{UqoWMKXn6vMW~&VP{6;NvWJ`?hBN)3Ow-}bIn(Ukb`)-7Wb@~;fN|RH`nB*by_)m
zoayphnSmWNKuv6&7>SlSF(%hcbf_vZXRgy!BqDJq<&oBkGYAwFm5gV=)LgSJl+Rq}
zCmX18&H92i-;ef{Yc_<rCx>e$8uH@!i7|?ax#l*%31f&bDVUU-)m+$cIrV#F55-?Q
zKGzJD;(w+y4pkWkD2$<${^W3}Rw-7vNX<d&(J(qtWM4`2hfd^+1fINSBw_&G$^F}h
zc!LddR<Njm+`n}g!tmOh+yWBKN&xO_<Q7CpBG1n=9i=OGR#ERoDl|lO)ycgoQ5P%t
z)k*@VsHZl2;%7NxiqATs;Kx)|c}C50gg^Ib?syX9#FkK1HNG6gMUw);W%f%cc%~-z
z&O(nAM|O&x$yE>7ACy&>oN^kn7^X0%C#{6=byj&PRo0T+GEzaeh`UkeP=?*?-W<wM
z5-W$?GEmP8OcVOSakbuaOEQ?QQL0yi-8#=)4XXQ8p6#qWK~i<+viVY?ucfHV=F`V$
zMY?Q0#+*_mlWQ!S--6td{i)t%?1rXxL7LQd*&<Zxk>8{N9v;4I09a;wL<cV$fR;X4
zT$FHD+OmOB0{2|aafFwPT{c*BfXuFgb+B}38f&`3>hYV+TH=vq8Az3SQEBw$WkX4D
z<R_iqFqNMcJ@xA6;o%}?M;F4og$I_kwuF23KZ=Uh^x!9ltM_}CwQgXr9Zwa2`F>r1
zh(DLLgS)f;0eQu-%i2pufBI*5_Re;Mm1<0(>_?Z?qEK^E6}nmqo#)PMN2puM4{fr3
zdE_TLux0V+I1ai)N5%AI6)igEWUGX74DRG`or%0ndgM>Oi(OkLr&s9PXb8iLP3(#K
znscxO9G=({tvqvc32Au~dy=WVmSyaoF|j8V(*J#Oo9G@2grsD5<+rRWR(5k`nm_D=
ztSgm7Nsvd?AzAN7@??v2T=N%X$>gSfr)p!?HQ1)Ux3<o@7H)k9U(dSEwvf5;#IJTa
z>jRWq-~fx;J$6W5JIq02#u!;G=~6(jrY1j!D^Y;UNm;A0%jX`00&-p5QgY=RjBD7R
zq%Z~y&sv9ijU5a2Yi8E^@)F<Ea4wIY4aqu*kjy=8<u&(1!6-Q#RzH2MzH;t|aL=e-
zTKdYl=g7AF?W`QKo!3cco0AIM%64{hzNH}799eT!R(!{A%$kQGX;kOCMq1W<_3c00
z^TS+6Nc!2Gs&F3DqGDcu$v`b@(ofIpuksF;yytZyZ`Y=*A(Ho1o%awarj>Vkij_A-
z9_r1TA`jC=9y*U_{V<8<-4QH>8Qf7BgsBX!l0o{YtVA-HKTv0oND6NTb(>fjv>TFD
z-$@VS(gjWOW=8EbP_?J5k*tQ^ns*J&N>+vT6rb<#7b515lbpm*S>G|MpO={V;}kJ2
zA!hzSnnK?bSur$)`EQXC6<kwQ7aC{2w1n7BS+xp#>-n!g1<|IDH(V^PQ}eURnb3cX
zZB+CBpm2z6Ih40kG%o@Z!9KLP@rZd=1>Ha}7dAb}Gyf&gg16};YNX}axPhKU;uj2M
zTe_PAefO_Co6N<l9`RTV9+ZV~&~LPnXOrjGx#}cSK9MRuWm+fC-&Rrukb3(qPsIg}
zq~6>4?rP@Qs%Ql%eUhhIo-JN^#V3J@fr1%F=eqIx3B6?*uJ^#@p|aMtNKC7DE-o+2
zLe-)^7J)^d12{V0b98~DqPf;{dx66hab|g4%Bx-_Y}R0T61-eAkXH%E8_Sbw$hXI}
zkjH~F{QEODTiynvZBYhDY9=l3=B><?kCylKqBzoec|YcI7L6oMP|WgS@+}Q5wD$60
z?B!WB)?aq9ELYRPay4p?)62On3j@ht43OocsP2F}z{gZuzQBvWO8j!qp@ojXQf?v6
zNoZ`R<uVVf#|euXdTuXtRCafGDnQq_H<~oxyx;j*ew3g5=j5k{{OqJs_a~0Ab%bS$
zqtvKmjIFz(0w#B;Fvi9NEg^Ug-NZwCP!(Ha!WdgRI%;t|A=eyZY@?a0N2h2;V~zmM
z&l+P}C2{KGSzUgNZM8pBi*`KQ>x{8&!+Z%&qeP1(ZYkd}wh0ySG*Wk-jImkwP)JoT
zJUzy?ScSn;$zQTn?IKvpHKZ9oLsP`U$v90DdH$=+c{Gv5(-GawLMDz}pSciGSS&qr
zi#?f(Fq{|94)h;nWaeUwE}pFv>E=NoNeu-_qfwbt1j(C3C+Spg(he0eSrPy)$lbbK
zkS1_|%p%X)p4ng4<(C9`au+-5yDv$k!HkhvIju^|*qPajSZX<R$w>9IXJ)Tx0hEwK
zT$#OU2_mVu1D7*pyj#=F_qrwW&GkooPwM%^_fYGb^oG&&{mcZFwJ3(J#WU+dny1YY
zM|^`dne~_@Tk@7tDeMbba$T{`U;4Y;ikVeWZA-o{G0w=W%GAc<=`twrvo$bnl;g!B
zo)b$r;uZtWg;5QD_6lc{lMQ}GQ3Tu(i{qK!8eE1`FLHf`apgpV%OW9SS2SwY;Bu^d
zlX%!i4Xz5W4oapkcWJOjkk2GwO=^RkG?5o)`j6$I#s)8|Z>iKxTZnn!XahEgm^EMA
zt)u2X5p|i4I*N&glMRwN3Aq`0k{1yTuKDuJe-YI_6Qw)BXpo|EE)+~8ZmICH=I;%<
zMM&0S5iX4YvSnd|{%90eZ)FkU8;`I17M8?Vc>BoZBlpEF?T$dUk9GOHDXF?+QOKa$
z8t~5(%GamVMaMLB`Scks^D0ZfBYR0O9Cf<1yPv2LmyaHn3W|@x_r^k(&saMD^=0tX
zO2UI-tUNheTt3r8=iMUY^bMDf+v~a7+Ou#e<BGX1pOpeb26`|j{g%t87dx=dK7bbm
z!i>DvE9v}uQN2sMGY#}3?wcxOprTiM31??U0PdOcBLTf2bqu~b)NCEO!lmjEN`my~
z9R`qsYuXGc-D8+9WfB_e?&R|6t5uOvrz64b4GBL?f|-ODJLB@{r${D<b%HuQS>efJ
zo$WjeeZ>36<ue%llGlU@w>;*k>Wsk+7tgzVY{bbX@xXt`A@6H^r*JaMmG>JT|KQxR
z{d+H)U*CacsE67vpJpJ;y#|iG30N9bVZ0nJ-%A6Ge-6im8s^?|e6++Ri-#QI-@qN<
zBS=oV?rn<i9o%_T!(I;ggWN_~vJ0JQ=Dse8MiBf7Gw$&!r+#Sg2wZ7Fk&IaFomA0E
zI$b7N-{IFN?P}w1B~75IC)5y+q-=<jE0y<7;sdf0S`eGLx_8))+Ps;}7*8;mbZ2jX
zEM-k3P-axmGwqks>NW&DkSG$EVRCjN-FKo<?6=5bkJK9)=h=VNJjgJYl`4#6^Of89
z^mUPJ$ps5YmNoF)&vle8cHWBK+Y`9V;VR~1dEK6x%Xoj-X?fLFY?Y&Jh7jd5!Dyti
zyTE2GmED=g*=4I36TLC|S~2cDjxqIV9GS$RRpmJz$CxHDYynxS<QI0c^Xp3@BzSHv
zqvsQhNF58{d)esvPG5tmV}3DZTs}jJF$KD}D@^T)<0!FxFChu(xWVEX&c|5XXgUs3
zVTH-b_Pnhk+)v1M!ZhM!8`2#;36}O=CW7B}2kK(a+;sUQiSe=AR!p~?q}t#*?Ae&-
zXzX78u!?T{<?8E}n6Lz#^T_DzFD{?*=rd`H343un0e0Wv@~Pkjo&$Kcygc*k@~I4N
zMy5u>2e_-mbY;4Hs(67k{DcCeBQnV_SKoz{Rl-C{$Q7POWE%@8tKUeBh*9K3U~qf+
z=_g<;Rj9-_o`BuA;(cc!L_nW|Rjcn>e%T+nj|Oe?hd$XTn_nkD7uZ&h0=q?XZoBC6
zDX9wOrwbFH=_YBq;j(9YlX~P@HngTa8wR2^#W-@=XNYpT0~(({UG^=KQZ`rS&(M`(
zpGgzaCQ<$eTF=_&aKMA)lGwdS^iZ<Y;PtsK`%5SWb*I|Nig>VHlxO||mpxZPMBk$N
z>?<*_MfFKO-~8YCJbW4y=!bH*Gr>uwgrwGS*{MqHv0b0*xt;HLHom#b{;HI7FfkT3
zaM|~JJxiv!><0>oq*m<*y`(UeQQbrPBl_(p3ID<C5n2lrtx80bUG}3a`s!J!RxbN7
z@gOsXRMP9kUG~9+oc>!)eF3Q>%!R3uuej_(P@U_lKwwwmX3(26!9&XOOD4GNDc+<j
zM(Fyrr>L%lq*t1IzVSGs%~(&s3f?6e<#5@%OM&v$O5x73E;}bm;Aeqx>QnY&sTV+3
z{-m3Qy=rCQ6gxkw<D#pGolN}3%FZwnC7@%K{!FvD>@iG?N{5#?3nwcIQf)C&0B_+D
z#>L3efLnZ0*D>`aJ|3y-m=y8QY=mX8R?n8W2TWV5>E3+)I5mBrMx5-!F1uWp7J%KE
zNRN<sUfo6KvL|^HegyD3(y_7^+(I?j<*o>%kkw81UhbQGOUx1@aJkFgn1GcVMH@j}
z;$Ju}aJTP;Y~E2a>iW@j1TGhql!OmSuN$h`=jj@Tea~woRVQ71<&rPsbSlfS=Yh#Z
z!QAnQKG#dKY0_<+o~MsSx}Uyg0z04w*78d(`?FXG73uShM`9tYZkEB^>hFLXiwZJ_
zRv9-6CWk2>WRR|I1u+inrKSA|t+23eQ`pjc89*vS*i-j;TziYJWrRI$+D%<L^tzcS
zXFF?nt7a1{JSwYsWYk5g*3A~ht9oyPC_DZeWdTv<N&8!M#Nx1(nuT;x_Z2@fSV*sU
zo#F(z?2&Z-t9`V}uV&O;PU2-S*3NXEQv7u{09j`#{<^#H?9?5{vz{xf9HCXHnz{#Z
z2fS5-#ZmW!w^lCUK?+(L#)jUWfYnTuufE$|B(fxzp9$-<ina$LYtr26-o(l6cp_^-
zmdMn71n>bYp_ejOziyS6?HM9AMm<_|y^>-?+5RQ?`hSDV5?tEb57~?8WS*U?SrHqe
zNO3S~6;iBRi(0wXt3tv`aJu{(&{oMX*EF+G`+9Sx&DX0VoL<w-;<OcH^?G)1Of<ns
zf0s?In2Vxc^J4%^c-F>-H?r3H#P+jSpP=`8uVDy%STJAIm~CRwX<fZWR4qc`1zDMS
zy>eK1)(0*d*M}PBS}*dqAtB&SfKmsxx79EMme0B8vYkVfzjqwZdjs%bgspuYPyXw!
zqPBA=miLZ(eAYUuhF5gi^!W=iM@85!Atb5?81m55YON#8-0m5))={bG+AoQ)t-L31
zt)l|#tcE>8n#`tWwa&7=fYmOVs9q{gZjWyh>$1IoFbAFmfb7>BTE73l!G_D$S42W$
zuEXWQ`d%NqZ0cf9lw-ZOai=ZyqYrORl15XKY3+?RBA*=YvW<7Ej)It<;%)$~b>3y0
z>IMD{@OOa2`nhZiEMWcehEYFU5HV4;zAvGhQ(ZQ7Kx8l>n3?tKmzN22m+d9d2Lc?0
z8zD(Llcc&NY38!!g%HX_wo!jFZXzmZ%Qx7KhluiwbpI2Ss@h7E6uWGFS%J~EQAS%E
z2~7M`K_LmR3;&4J)JiCkuo`uwDcE@#w&}{+Q??ih;Cjju61XSn6k_MrQ>aQ=`K^(F
zNs?4)?o6>2vvh8eBpyU!>ztsZO;u@=th91iP*OF=icYtU*Qwa_C`R%6pC9L#%FAel
z;b|J{2CMLbwWY#3U1QxK$D;&mi$uc2{F0KhiAJz)X#N;$OUcv<-e4i^^EkMbN_Rk~
zb8ndTM2IBx{a1ybM9N|Nh7}S#R&j8m%5m#JE5}3sNvKV~t#lMpUmLjBT2otEsMS2!
zy2c9p-W%xdr7Bbz--2(l%1LU8&Tj9j0)x8^7$%hrcG)nk&5d!QLk}S@vvIS;gTAf*
z67EGIdvk!!U}Fy}gXUMr{c#0qG8op|+2n6-9H(BU(_;=Rh$>V;ON5%ILS#^b$ZXsp
zA$stn<ze#`&5@1gABTv{rBfP3)NLzVPDolQr>XF%Y_$}s@9RuA*(8EyHspv9XG(}b
zvlL^1N>8yd-DTrZMjBUvvHA-CKNStKxlR?W7wW_TLIu>{fLld#dwzsVpYs&p3EUB@
zg{HWD1q+Z5aqpqZ*YpAqLvuPqx_VPwwzl4O|0^+mhT1hE0Eqvly+W5}(O8>MyCz4t
z4^$YQlBf#5nz9MoHho$cn_b(2#zz9sA)9`G98yd65Uz<7G!mteVQ<DN&Sus2y_I1}
zx8z3?PZ{L-aT9YEbF@8gt5iy~INieL0Y!8lP?hdmcMF>*JPq=83swy27T&++vK5s%
zmd$$ul(lFp3Nz^zWHyZ;v@NT9R6*Pjm(3K`B-8!vMq2fQrd!H&SK|tsSx1t2)@^n~
z)T#dutb5eku<vuJ$OyFUuFGZ*Kz{e3;FFv<%$>6Ao_X12$}Xra{XD;Jc2qS(J>|Br
zCbeIe&fM=U-If=1QOpB{o!*vBPlK$|d1FZFRy}l?JEU}5?G{32$27aP+_qBhdW`T_
zFQ}%5c`;g11Y_iKmzk?D*42HqnX3zaZwt>}{_QfS6v03oFwkX2V<2vQ!!XPpBCFds
zX=B`s#yVos&qxwJlvT&oiHmU!n~kJ2To(kf`vsR-*+qS#){-)D!vV(caGBmQY7+V&
zX@>6YjGu?A7N6Qnx}1TAxot7AnKm+GOImo*|4AFr0(<%t7~3Mn*yEQfBnops+bU5p
zd$Ngxrb%LI#=b}LJdHbv*lnYE3~%STz187t$v}5oN}>F!R;p%Gklt;yhC^!!?=In5
zRcu3D%QVEc5uy}*A4#j(wok;}(_kBi3tq4YZR{oVee%34g!&QjiXv3nJA^hCLLUfi
z!nwP+FT_)tkDqWq2xiRK;q4@MrLOFa8d1pM!261+4~g4-4O?6E&Gr!WMnJ7v(lECN
zXu7wnBk8DOfkkM#w>MC9E2X_1RSDh2w__&WHf(K5gJ(RM3GY+oe7v#sfuY{b*e;!j
z0#_=0#X`TEVQlj!vm!Exc)yOzj6g=3?x1|lL`HVRSTYh((~^;YDcD0xMk11QUCQ%g
z?gl}<B|fR&k~zR4!(C>iBMwkK$Y^>=6Tc%wSfSOcZ-g9FZ+aI6a$6Ot5o$%<g;&=w
zdaibvK_b*UQUjDmH96WW8Wl3nb#j@-giIK1ED|zzHm>4UZQd+aUDGMINi?Wz(OImX
zMP~}Y#*n$QF{z;qGW)tre<5>cJ0UYsh#}LHLZ;SszD)h3n{|bmO}kj7_~MCRq1tCG
zCrQhzC<8Tyo#qo!rt5+UhKU-(PIlWCX0ViC$JpsMFd#8a7ioqR24l%c{%Tv_#YERb
z;mXdQB@|a4Mk}Utga4UbT))GV)DA$DHM{foSjCgyYgsJmP3gPGDlPhZ1B)ZbR)b7-
zP0Cy=Nl}YMySpAZdICu>u8`B-s?(=a9ek8yJEEFZZLp9;bAu)7Jc-dXfO;&FI$Vn8
zd3mR!jN8Xc!^=xG4L@Pai;8Km3Aawu@DpVc4b$LF-0Kz%+(Qcuz5%XZo~9w^zckPU
z8;Eu98}OKh?y8CKtVf0^wNiw~E|%s@gcbt^P5j994gBdGfmfYE-4(8B`^GCWUxV^X
zP1`qKiJ{wQ@K>bbwaoZSm)?+E8wOIDoskxq6y0T)%Cv-9GIraanNA#H&^tB{Qo`K?
zxB?p$ST%da^)Wrnu10F#nd@U!Lu$3i^|7uQ2(EY5bqIntwn1U5(>r@6#j5`BkW%kt
zP##xC6MU$VW@bZ}+R$CcUs3#~qL@>&uEX+0$Jy|CT-(TuF{6R0t`q-w+}Iey$o$oH
z#Fu-E3T!X=vI$l0U)Xq?bVi8>u1j=3yJY{nT)oh62F@*>PxyEdw^;V-ssshZA7+&F
zalK25-OS4C6sw7}(wkx@p6v$HT6-oH7)S(L+W9q%My~fI(JGqbo;fghQz_kAc8e+L
zZH|qknQf}1o7wKkp4i<EmjgYB?6unM2=6eK&_*%{=a&(&>pCd+lwvo1+ZAN{I_@>`
zEaKx7j>|P|ZKw0`0q$A^o-OjTkCwP8-LSv^w&(TTj@V*v==}E7_1xd>C{+w~mV%zx
z)5BAIkHb}Y<D)06uIBrw@S<5pp6UB<m}<2nzIUn3xO<j)+U{WjO}Y>DfWj;Lv<T+L
z>*mq$2Z;Du*Jy5IU8AxBnd~c-C?_MO5|im|k|T5_=2?}P+!|PluEgcuN@PR0)XCaZ
zS*ws=!&hEO>t()D->R47DI}X@)l2exJafG@xi*dXMv0BCjWF;P7H?EVCNC$>GH-&-
zcs5FXtfeUkLtc5w)94jPX~%TcFZFwcRmR4yp^8Rq#oZuON8R=o(d8a$l{@)W5bcT0
z%wRfdC?!;-JkP(ksD?F1pI~mv8pYB1AGp$F=3s9J-Gb7p3Ij{1iGwt(MYU9kxsleu
zjpr5&VU94Zg#`GI8q92N58<WSu-x-J>o?*NbaOPiHmFs#4K}}lt4$3_Rn(CAgw7zs
zRHrXoYm0nY639NFOGhI2F%n>lZu*!^BU0}hl=a@ywIYvuChc`N2k!lVCf@v~$m8^s
z+_MshFY&`jx#mf3U6JB$j)?TA68%fUfL@WCgur|W={+Wr)0T$0FIlzfNWDBQ_!nJ?
zoKc);2l?Ky-}Ccc>^&5vaZ>ukS0}}a5{PVvNh@kA`Agg<(X_VSk-Qiyr8gQub|VK;
z(`9yKmk>(zGfh7-v)>3KQno?JY)2zP8$}*44P%~9q{X$8h9Q7me?<0(REPi|ppon;
z@GRZuDB{i_O+Q95DR&9>h4lYL$_)jPYRypcU4)xwT;&z(gN!y(ats??@uA3$4rD1s
z+8R?3uU35A>c=Y+DCLS3f(hi+2NP6CbEno6uXdA=Al-A52p!y9>xSr8Svrn@oPole
z{q;0+nD150Y4kD#ZPCn0@-pW#u+x#UPdrlZ@WKBGCzE_dc7Y@`QeH5LY^LhKFdCP_
zg_dKi*DEq5fN<Jm<4U+Fo?kUm`a0b9p9-4A35B||p!1c+!!S*a=JwZkDhMkvB^xXq
z40$JMe`^eKBVT?FvK~zd=pw3gi7W@|W+L(TrChI_(d8-^A(6=%3$1ayE?2qA(iY%8
z7Ld`no38Z=PuKc$R;@SACdve<(@G+TNMYlBxE&O3WMpJDFYGEX8L>9A>UsaqTFm1E
zlwdOie9f*Q2|+-(*GMAY2P1vjYh`pRNR*(TWP<Xw>N;{Ftr-ZcX(r|rvhW%cD&88`
zve0az6*Ay)h?a$B=d~=nHuZ6cPUCL&krhN!^P(dWN+yMNq23-68~F@Z9Wi6^(=YOo
z?kA9z3S8Qd<&uEep|%zN!WQb~Km|r<rF=tetIE)RWNqa__fXqvL}BgAc$y}J+SUlW
z4&rgoUw{veE8m!<q51~2gDhn63D2mVHtaI^B-A#t2&e~=Mq*7?2>b>Si#_f-_1K_L
z+hXzL%aBm=bVJ=*<QNKPW`x>07gD&|Z*0a8()MuB+$${dv^wA@U9)Pa4a3ARUoS=S
zo##Sr>I|Mt+U9Q!wUrah$#}v(2(?8&!>Dhf$8*3@+4S|CI^YO?_CTnuF826Z%r^{<
z8efK*OecY?r{{+Qj%arkDUtb?v<pNB8B9D9YF<QTAd}|n?1yFNWvKaXA^FSR3N_!C
z*of>qL`!#tnr9XLq%Acx)I97>`#EW@IRMbEV#k7rlAA|MgZ1<3dYOlP=Wf#IfpVyM
z3Onm{#1j>@=4XClM?!GxI-w>Ox?vtn1s`QJ*{%X>@K9mq&I&>}o)dHMc&DMMJacEF
zgjm_`QrTA1+3KfCCgn+KE4|w~)LbnjbF|{%>VUBv^8yeqeS}5-Y9u{&tf$^VN7+gY
zM&)e7yR9Ydi7KJyVsFwrR?@`~6ga>${h%Y!od#Mp*K}wVJP=+?iV|Ej#jkOnR~a~V
z9Jx&*1F^1~1mcPjb5w1BT74Kz_}`@VvzmKTRWSez#`UEc9@5h`p(a~Ygm<*V=dT0)
z7MG>=Eugqi+q7l#%`$v&v5NUd46Udu3lMKu%}&jwv}_y7;6>81rZ^a~(o3t7OgV&+
zK)L&M_N+PNC|@GR3ht?R<hgjr5tBYORJJ{UFjZX?fDgkPj-s-Mw+l5JO3+j*UBgg8
zYsG7bmNJqpap+3>9^Ou*w0GT5xy+qZ+pJXbqA6_c^9Ef*8CX=;9cpr<T=Q)v+{gx4
zB_1HFETTD^Y{abw7}X14yahZC@Ni`YDDy(ih)}j5HKiAY8|9aB{4`87%}c&&k#5u3
zP_q>Jy2}+(sxjJ|y{iaUN`<(A<b&q}+*Nul?!mVo_>Hu3<JvR-iXll$%DxzSK}zuE
z%onuOxtog}(C%pHw^m!Ljr2_oJtbqrH`fqUW^<^lxwC3DimNv|gsvm|?R`;px23x{
zc9gEH<Y$I?*xe~|%vdAz6+v?l8h*rmL&fi`Vc456@%g42^7_V!Fzlp9Rj~g_*?i^U
zL}TYfjAnYZ!=Yx8jAcGU^|PF6N~5L<x8|e|S0jz+*p_($-c?GYI_z$&aMci9yem?G
z%8#WLE@UdAu@Oqn3dMjmk90&7O3vcz*aey_YOQHe@m%ABrqWcF5|4ex2hFu<oOr(B
z1J&BJ72}lTy$YQJ6stq!L7u~>!Z^^leFZF6Q`p;g)3-2TdiW+ph|FZ@`O@uTe+XRG
z4>Vj)*WA|dBeHIw;5OeFDc<%1D?m&cD<D<^b|i{TOF?AKK-l3<Wxs`YE6`Zgymm0#
zKpYt-9q&q#E-GOcDyT!K*yKm$t&!t2@upY;O`k7=XO#V=+lG8hvA^`G`j~5f>Ad<_
zrUrFflB6&pk0tknHUz<uUm4YByovlZ6;%Diw}+3>xKZLk<eN(9RPik5V=-=_c;@r5
z7PndvHeEkLdNhC%k(cU0spbHw(H*pe!ddB-r~oSUwKKrC7*Y#ILwCE`%y_J^)ap^_
zf9(T{?AN{k6;!_V6ZyQ3lXqTTd&I~0xG#x%pqLaD=mA<@Ga7WWVrZltyQTXSMxWG*
z>!>UlNKC0rjEQtpiSmxSqD0`O`*b1M%+#p@{YHT{5TqpkC5k0WD5SLJ5qB~Kl`1^?
zr&Wb56GbWRTp>!4oL`R6sE?IVIkT(}f=Q}oU+yA}g~%_fRs(}xej(4|kmPO{9QSU^
zKz1kgiuz?N5qi{N4Pu)$rId}WL2R@3c-mTn*k)8=vwlwvVs|2f3}VxcIjR<~%`S>>
zLnn!HIQE-osRbGqAKG0&$GdvYA9FOS@KNYc(HIB)+on!7LNnF3K~mp>(7Gz#22Yja
zjyU(4&?LPBRW6{qk`-EAb=dZ~_+GpcT1EMqFzfyx#KugUdHhCzSj%{@_vo4AN}(6d
zhnNqrR~Eb$V(!MTEUFW-M9{E#i1~`k>f<{^gqVAk$8S)k5n}EuhkY32<2SAbo>!42
zm1N>Co}Z4hnWjvL`5HtAo&`;Z3V4vJz_Zo7)`7IPNjt<mc=MGI^F2{``G_6Y4}S<H
zRdYgEkP-qR`DAJE)JOwDSE;)09s5Jf>5}_BLf#vJ2gWGR>96sm;&I<&BYcQCQ<0@M
z!iSi%l;K5|&Fvxjsu9`YbmC7S^B07e3upwWcR(loEzm*@5;7&mV(k^{u<r*UrWl?l
z`}&RRKb)Kzlhet*L~J%U#B5TGwyL*7_5CcwOriHzjBx!PVm8OZtcC<q){!&7wAnwz
z<n*F>a=We>Gg>$`K#^ZHf}+Ub!Ap}v%sAz-IH;fVn{-lc;wupsVj>bGDlj6BlJRbH
zjoUC!HO>HgAYqjRi+Y7T`v~56i3*`paO2^r@HP<MDBx;<%hZJsA!d*TOtAy}w|x4*
zF%-1ybAp<adWb?wFtpk}vO-+!F;XGW7?l#Ot8){b$Hr<QKT*O{Idp|JCWQQ4fxXaj
zYGvKOo+_stW$Qu={YAjHA$KL#PIV|a<Sz41e9OzNA$OA~)nH-S;NQVF-V=fuYo0pi
znQ@90uTupY?;$VV>T^R5^L?s7QVg5bnStUzAqy-Ht@;gITtZ`*r{`cnjb0P70DZ;8
z!%(o~XN9~dSnBib7#FgX@6&UTh|_)i<uFBv2ZYQpWEdeUL}Zf#{m1opL*SZd5;Y=B
zN84H$ZChb9V)0(JiX~(Xy-TmSy=q7{Il*}I^ld3bTF5jwTGOr3xb?Scu+!DIka889
zve=}MX&7m05(7~y`$LE{4w=P(Jr^QFhQXdQWlN}(!$Ktouvdu>8AXFp(<lhk7*)H~
zb&C`O?LwrSTshh^qzkMPqxnp4jiC#DMc@E>D;hx;F~X|*)3~}~qXmCm6gwZp-39SM
z$-U(Ikm{2At2%9Ur=*qmHBrT9Ue$CYs_1;TSP&w+>DBRNXlkP3XqvcsM6e07^4uV<
zxH#llLHvtG{4DKAA?7i|Qc%(vY+#6dvQ&Y8NHQ_#N7PC(hv?>OzExSR&;h6{6HI>n
z`;Rq}%wfzGyj7VYQ<A9<kVwYMd`Y`?agzChH|;np?F%X`q{q2a2&PW-Cb%3tBCW|5
z+WrcykkH~>lG)jd=A4CwqqC48X!;WB#d4)Ro`z3g>87xt2-0Gc%rv4?Jz3oLO`;}h
z=L)s;)>puXegq_v@O&c^xq$L0*}e~h47Gj#Z1D`)iw_uE_M2R;`b=ie;=99m(9$r^
z4)-6?Kf@T-n2`kEe6>|s`HvVi&=}T^x;Z=AGI_gB5R-RQwP771P&H>K;5l$%SVyEi
z@T|XF<uk04Nd7^k1Qry5l?$`v=S`%(REzw%79Y<2Rbs+eBY$p1!#wvF0#<EX{yYu_
zp8J>Y$m#j>5Vo9(d>?h^&#P;g=Y9D0_sO4+Q1ySG@1$q*=ewV?B8?f7KfjJdLfY~d
zJ_B?)WJO)cml0)7MZQ<e$X`sVS%2}p?RfqYi6f`VHa(O7qWbn9KETLd3Pd%M(R=cj
zdB4Ms=jVB&hn>mKg94Q_?7TZaudL-sxR9S$Nxl^e@+EtP$o@%wKC?yVivj3l=A%XQ
zcjMvtE4)aT<mRtb-?Z_!55fHW^>p4U5BszE8`QUSPH&%+u&t<Ms!l2)>6Oy?FH5jw
z*E1x4mp56HxAS+a`Xkl3L;}~f$v;5zP#b&CRR(Zqvgh*Kj#4$B&p(4Xkh7fH1A6uO
z{C8<w;+2%G5xaBFd{Zoj9G3zu^SI7A$`;#_Z<bdPJq^z}O1X%y_Rh^q$v2xQy(h^}
zdPbaMue;y|(Sc0Q;&YB_Zi&kuB9XI-A7h$)$(@8bMUarxocxiNgq&|rg<am7Ke8%3
zlkW@d^GC8wr9@=@g8Y%)Ni7q@{SW772J$^YA???OIgFNxwknAAc{%TP&5>w}Ro07;
z;T}~j(N-IQR$<AP5^Z%t(4Y4R8hts@77rm67D>d|PKh=yPEnH=f_UyajH$IDiRK?5
zz96cr_!oF+adUt5%j`FaMXg5&@P)d5W0|o`H2<atxzLcU%S=8bn*V|!P{#G~8R>~O
z!KZw&35hmSH5Y<5evxRiDG%JcPzlKMy%KHDIPs)-emu_}h@L{ngZm~@bU}twQ{PE6
zcL_Pi{iKUdH21Ls?80$UzU-H1o)8g`d9M?nB$_9^9-;aa?3nwj)u1n2lj63>OEk|)
z<}w8ZkIeImM}E?x6U}R?Qt59XV9pW5LGE_Aa7~m8x8&8~`wZG-(C);)MVq`+%5Omb
z5hSyd&zx;&)<p9e$zNt}#w^8ypwfl<NgV(3l0adeZ39WszY{-^BUkUVEdk>`DJPv9
zm3%xd@s3-S0ne0?iC?I1(P7(rCT>@`p7I~iKix<aOZ9fS>GWjhHYHvMMlP=@LTEmT
zhZG&)fR00o48A8^O?->Gd*>A2xw(lK<@@bGe-gf_r~`a`ZsK8eN=PfVIdaf)_7u@+
zM&dH^x?PSmziA4VD+>4?Y$UEIC2--}Y`Mz1_D6}#CR6!_d9hOoMQ;uwbh})Mas7~*
zoN6%UAg0A|BXJH5w!$J!>0DLX{ELpz;BC|u{oU@wkyfr3H+m{x<dktaqL#axI9^GR
zigaOVVlN`mBbXO&myijd#9p*ob^9>I?}g#OdQ7H~_yX$Zv&sH)b|A5jW-QXRYoNy_
zm(W~lMYUaXcx;s@mzwn3SIc8l=OvQW9vY90lj3fAGvxMCG?8OFd2I5|tKM^Tsov8d
z-MR{6$PLk{-+RnbLaJESmonr|QvleTBl;K}+0+>ovrdH)cm+)`Av<PWMS9NnDDvpb
zF<TIl`tFPBFmTLPRl)x0<NF&iJ89ar<otVFN{nn?Mdt8KD;@Kyl4rhC>%{EGnu&YQ
zFxrw&!x-VJY-n0hU~no_jK%j_#VTYYBdEUVn7*mFCeaG%$jZb<B2qFaf64$BE5`An
zj>-w(0$dKuR0v|s(gI`UgJE)MUCj8>sP1UChDM5I88bnm3y#gW^$36*ebX<>O}fO`
z;_3lKjSX|ZZ%igl|K0cPK$%`)X5L7VV^deJqS#W-;yWci`M8Pu0L!sNUk2${ztzL|
ztG@&N9``L{-j&Sl*HSwZ)#HIov8Z{0tHA^<Vy}TwVq(lhvb)Ss*Sv}ulkyktH}Md)
zIjn6SEFQ$Sc?9`4FDuzil<ZQi?79$C6<1lZ+xJv<dS`67Ax~Qxt1vrAn-epOjcS+2
zGn$rHEt|*V;z+uR5w&{N(l!wqqx79^D?gR3K>T9U$Ivp}=H;(QD0j9sM^iU%R1psw
z*@?jzUd4Oj$~OF5*=9ituWZwPwxu6LKAIyC&0B+BW+PUPGq`#m;$*hLY^Fd=P^=q-
zd~h(DX*l<EYszKPB?(>Sqo*g{t8GnBG@GbrFl0Vv6-M2AtZjJLK*p0vHwNANVg^yB
z71?;zZEshe)1o)%IW2n4dq7fGEqa4wUL3J$J_$D>a*SI<Z-^y&&9?*GEW*R~Nbrg9
zbc`87ZC-gs>g+{=-^3jf&n!O9;m(LBi;qumA4+9*l*(LfRi>b%Q8fRN0<1XsRAq|n
zxMi_#1v7Dl6HU#!P-6RLGefnl_uq_3v)a}wDd0qDTUUd10n^G`1^k-~{?r9btD+0o
zg3h5ufNnZHN_(4*YH2;K;<l)Z70{xNlo#n;HIIZwiQE{vOY_Q~QYwU^MF*VRKqk`L
zVgxQjJV<Yg8MvuZ-rWBPZ;O0F<f6V+tbx}m?RoWzBiMcY$s)3?OYK9JgUCZ`#H6Yc
zqQkECE1{G|Y6U46DhmyiN*oR)D;<#9QsSsfh+<k%s5*A7y*la=(Oo1(6TRH*n7V~1
z4=9vXW9lkt&^YU=)e*jr{u@)5{snz57aPUYS2#T*ud*A+rb(%S#`Ka>B}Ss^WL0Qp
z-efUPkWyQd0u|GoF=f3}yq`!a-;Ob5BT%ig_(q$m>G^z}afHGk>=^xaOi4-Q7M=e7
zNS%zW_c|GByJRDQ>tumC8Rf2j5v}1jy1vg!l_f)%0FbC*`!!ehOoM>(f<wJ)QSCBb
z<b>&>3_2O#j`e-Ggu}e%!gFz4eV_I8zP4+nJqzD=JUdLxulx_{`)pPPD;&<yX<aL-
zdMFZetm9gF>l_Hl>C~@9DcfY`RcM=WnV(u>vDOM6gh|`4zE8H7G1u+{2yxkpI8VuI
z%!C~vBcX6A60D}LYCXb(L?}Z@h@J5yu`?m|T4}!l13>fQ|4pNqb+MgDqiGbfuUep)
zn$AED`Hv>?9A+D?-Qlncgx2W<p|72&?~?$baLq8UOZg{c1Ce(29-im3@HCQOJe9wy
z?~{l{cYQO+=iK#te~m*XUVd_$7bIG&OYZE)hLQR{9ROb6Ayb3(eY(i^9$kj6stnM1
zeVIjP%G2y^{+PY-;m}&&#!tP4m>wB<^=&Jl^@GyNz;M^Mt%Zl`kmBBg`ZiUA`RWzW
z`nE0DydSWRZeEx9JUC=qS7>^CAoF=ofVAmR7{lPiyiw5yHSs}da@~HFr{^WtER}1z
zDXK@T8n^jD7lnw&ryk=T@-#@}E{q{|%j|>c*OYu1<_$l&Ff09<vT2G!)U5OmF*9%Y
z@!iIqev@`Fy^a4^BmLvDG?^cPE?brUiExnJC2r%r&(c3t9@SjZKO?&PSN}0%L3vBQ
zACMs7M*8Pg0xm8~|3<N244&9d>0Dpwy_?Pbw@m%ctnakFhT5#u(P`T%=JSoAs?bi`
z4y(}SjTWkaPTS6og@wM+!wb_OU1*|cudMS;?%}kl4Kp{sR8t-JXrZd^;!>QpiB#8(
zFFoCFaKqzpr)`>_H578)blT)>d-U;Y+62-Eb|y!N6i3$w@e)Yp8E3=%P}a(;UB;&x
z78D8M-U_i?C+)PopvrEvWNO>xG-Zq|Hr-(~iKCSCJGZV^B9S|Vuo^-RYJNLzJ^4e2
zxm(cz;9cB>@|h#Y7*2DK;H1NCaTj;HG83bJIn7s?g1>d${?a@j080)!&Ar5`@d90D
z`2iyal4_roN-m3QA+H*~a|B=sh01NDW@3VlEHAeX0cB=EGDw|*n^b<>V1bcB!R7(W
zkz1gh`eL+X{F`yB)r_svJmSp<l2gxU7J7Gq7c_&%n6n=@fmJ(NIn4(U!$8lC==E`$
z|0o|B^}OLU|AjP}O-y}=`xW4@-<&poVIXEgODeLZ+&kSS*=Z~81!e&32QY7f(^kR@
zTn%s`X5IZQ1;(*PvH?zhjZ#iqB}DONW%}Oxy^ay-Q{2bmdzFuGabH*DoaGo5pIwA1
zRe-ZBi!b(79)f?v{UV-)Jo|+pxBOQ;6r~l#Xyr#|eXI?DEuK$5a+GQ`&ahwVPJZ(E
zq12Le*Dv`tHf-<Qvgt#gN2eRMi#|l@EA|*G#Tai#cNq2!Ilz{B;%+*k>mD_14N)*R
z*|T6RLg@v`W7*osyLr+w^4j*5<j0V(dp6>K>&d&xp0EQt*2gSCsl`T~q#)LD$lUa$
zVSeoLCZ4CjnHD(K3qBww<MDX{_JUv<bsF#kslTUAmwn;1wG_?|7Q9z11m2zKZQI2N
z-Q$!YdmIZ*IP$S<NdJFx<oJEfk&ky1#p1{bU(Jz^Kk!Wdm?Cj;azpl#Kjwz_D&@m(
zh*{1ry|L3g%?^f7dRu0BD_U!-P|GYQ24=a6_k>xFAG6$o7MkVSZ<*z-Fk@P|%Y)yW
z<&id$CBG`m4<yU?yjfx%wTiK_BnDZkcu!=BpDZnCVV2tO&9W|75_FDAppUq>h(sUZ
z7l}6Sp!`1VBQlIYx1wTN$;BHY(5>hMTD7+Xn&=3$O7w(4x9UZd9$tZ_Ewvir6=;S9
zt;SOKM;JP-+9H*~P5@Pto-&_0io3TUYAm4AxSdY(Yns)kUwes|PSR;!A{OIW*a{rd
zs_x3rE->P8i{kA@qgB3_fUS5odkK&d?3EVvjhBFl76GyjThq4@<wxQUry1sV9>e^s
zh|s7GTIi)(Q=JJs%yz(UClOrDV}9mv1vFAS^$#zypj`x7)}5yZSf336hdzgRc!z~{
z?tjqAFpzAI&}vn-O#eKUEjMwfaBLBk@45d^IM<KJnC>$<mlb4xW~#=TWMO^e*$1`F
z()B3smTV;9n(iA^fLg*VP+3?C;8p^m0&eRsoU!Ob4M_gCUeM)ks>*$YtvsR}ZY}go
zzU7E0PC<^HH*Agn2K2J0&n-u&gB_dqY!dn$JNC?5cWF09^-8j_Y}1QQQ}zqp_>yk$
z^J?gPdOmY%6M^`R8<9!xz;cjybni2r7{<YBx<gIYuSR|<o%Gc9PIH{BCV$@2LNczl
z>IlhXP=V8AAxJV|{`2!(B1os$g`nu;p6Z{ogZjpo8mHC6Ob(E9AukSenu)Z;8QEm2
z&cNT1Cp9E(qDqT2y^T9uMR%xjBZ2g7a<5^lKg-K{pVO?TC<fRFmuPiv4ZOg?0Q*<N
z9%7Efo3~dD%=kvs0KZBNq@V_F(*o3lKx=x`){`tXKn&D?iuZ&Xz>gZRpoMBc`z<xl
zdJb4}K(8+Sm~lW$&+TPeFn$wxn87v!5y{)H0knB{nLggMFoWhQB60iL|2uppvj#G!
za9eh`AZ=_<QPMV9iKYw?B=sHl6UTUHT<oS`TbMir0|_%LX%g-{wb51LG?EeeOxo(V
zlLR{(o4_vPa~&x2aPO6Inw3!~Y98*+XqkrtP*KgpwKR-9tDMLM<riyeI?>E*v8KLk
zR7FKJ7<VvXXS804dy%T&)R*{D%gF$);?8Ibrl`~>ur_qrZHiT9eInCo`i3$!DUX~!
zi+)<`RM*<e4(K-M`Zi?%Rt|QmY2?7OBY-60;*bKrBZm3KGJ@A%cA9dHf@~*f(;nBF
z^Rp}efY1j*v)?%<Cz^=M!?boLoQyX8aA<8ivoS`SQMeJ}@!<n@x0#6En_(F2O7&F#
z0+IKZa!ReKy)|kkiVb+vH;`cmf}$-7!n_pB1;;Ee1#c+|_BzdSQ1GQ3dQ=pwB#y^R
z0S0Ir7I!oS`|-#bi0O3ZYF}(krBO4KcGZ^^N$49-6UF3>)baE#0`~mz1;^<wQAu%9
zNq2X~N+tbJUDr{YGvsh2TGi1H(zi)<{D8Yx4)zaq{-o51VQ`(O(Y9oyQYUg#S@gU3
zF$3F1MT#-{i&c|tN#3@BRL2PN{umswot!^aWaK0_Lbs)FZksB*j_&s?Fx(uuBjP0I
zZ}gjY7fUbYCVT)+b)pz5ZBZd@CrC&-A1iQq(BU_U1oyG~+U~{45zPBogl#Y5E?M0h
zxUq-)HaIJ3gOPcNwl{Gf3S%YZSGc?4kt~0~6~IR-S1iWlyN+iaR29m%_g#l8?!5B`
zstL{jk<pv;Ml}NgFRB>yiDHnyV$5fXF~GSmddFQlGbuBxMa`V)VwKTYvWa0kAocL=
zB$y6^)4?SJkGB8f9_ornucs@bor_I{!N6)%ai+JhU^RhN1I8#L1OBpvV6<zAOYvng
z?L}t>MKumaDlJ-nm{bd@wcQ9@A7Urma1O(=SFjDh)&Ogm=9C=*3U&b4{-Uz7;+cNW
z5#nxl1;Ba9=CX5`v|vG2fV&S2jV^~zqmOf$5Y^H?P>aV1fgv94iPFAI<lv04hI6FA
z&f|~4RU#lO%{fY7h+q50xa3IH8Cct#jP^)M`&N-|nHhKLRSy7ZxUSVyFJ?VUGC9$j
z2A(!uv_8M0^;gheeRbPX`t1jjs=uZ5iGb2q(VkHH_)+>6v{32eccX7Dt=}Ha&>l7_
z4V=xsHqq~2{reyNZ8j3!n|)F%=&#jj?I+Z~6-?JJ`uppq|4D80O%(mDc4}tdreCS|
z5wX%lOIub+_9LC~*AxGnbfO|`s?$Qf&eP^==KRnALg&}V{~LXRj-F{nzxDqKhpQ_6
zKmVU_1X;qr`ENlj)5|?|pU&W$hW|yNZL_IxN+EdRzu{JSELMFp?f(-_N2=1V`){~!
z1T`_(m}XcJj{jc~)Yi~xr7Z%#`oF`e3!m`@$3dymX&8w+*HTh7@vmxVTB1(+Z8^`o
z-@xlsozu7Mq<Ff|*SwzA*NUz#pm;(kWUC>{XWAO+^ChkCN^f_csMZ6{#yb0P0x@C(
z!dF^!$9EodBdxDrEopt`&{%0ZZfCzFBG5qFqcz)4jFl5J&VDHtY#Fc@`zW>0ubsfm
zAv8lzq;r6!BiipK#x`{G_v=(%=w{nt=KuuiyQ*m5?~f4tDNY9I-=lcje~r6GySE)i
z1Al)m74QFWXa`p3zrU`KfZKM#IS?gme*Y&u<a4-+;`xD(I9#lF?(jh??LaH#IXF*^
z@3{JB|Ca!0a~-5Nmbc$JOvWJ<qER#IIeUsmJxC_=_^&2&zF()g(zC`BTygfSC-F!+
z7dJ~YrDszMwhq`T&4%6<8#){U_G-8rvV;%wq>r+#arTA}-zN*P@Sy`m?{H5%@Sy{}
zM~4F8!y3g0-gGp9JXCDBUqs;$Htce~AommAuSf(+-cd?^9|hl$l6O=k=Uumm6{5!Y
z70u&pO#=01QOD858G$B}r{Fp+!7ZW$X!_Sbb*cj<a%%4T>m`&9?OZ`hM{f{y-`QF9
z1}JnY`gvW#+H9;D<!mQL|NSDwLXO%gIa0gsob8klm2fLbX{V(JB&Vl3)y#rqbiam9
z)>1KACu^zM4D$8UoXHA#*AhzgCv!9B{jQ{V?+0fx<YSkWgoIe4mTv2;prUTkQ7gnt
zRMo}$Rb(78q7TV7_4TyA&r}2P8w5H_tN8Ee_@#-XP`B#gEG?)9%fp`MXGiUz24pkj
zE|Hgyq{;jkrH(pA?TC^&NEo;t=ZTif5TbTe7lz?KhdU7|rx&AkS>n*~dthJF=eT7#
zRu7#fr(*H#I!Ypx#}CtxpK8^k)Kan_`3Lt1jj`(_o6Zjr0rPsO*J|^TmcHxBr6ZZh
zV(*51eN_UsfYT>x4~f>s0pjL2`}#!eB|_Z=#%^SgMTml^!;JBN_zu}vPyKX)(oRLi
zgSO~IpW4Yq9=j{a><>B1Xtd>9UWJmX)c&pwnzSF82P-5^DWTm-hW&EALMSF91BP0S
ziO$IYsvGvc<{Om<8Ad*tbm0FAVee0gn!R~6aA9w!7+j=gZ(c01;7ljfPN!Cyz3PTm
zfepqnl2eP_`s~i+US4+Wrai-smDJwLac(KC<_G6#g<HE-Tj7`XAGHpa{P@tz5~{ru
zW}{}wJUnxRCA<HhEa|0KvYY8PsjZ*XDz2KlP)nAQj_SHx&n-+*aLtsO!W6|5lvF8q
z)I!>AXQ{s$iXYQ7KNeQD_^}cqmVvDEQO)2CjU}!S3zhfw4^eJ73;)cY?v=<GIM#@A
z(|GvivH&-p1|rq!D(k2jr6eMgkw3|m#Ic>C9$(M+(~YMfdgk97E^RNcZAV@Qycy-)
zVO3C0XEw5?%1mx6h#IBl>eX5Js8MxD!xI~%>UA}0phaTAWE$uEo>2px6gpp1CC?rx
zstUSm8rX4Z?fxhj_7udT$_h6qefYH~Ss0em7u*4LyK_<8Ukz%$w-rRWWo_7-5AVKO
z`JB{Pdp^$R{ih&SK32G#zUlKIDulW^Q3;)_Eb(8@Bdl+ojw)5kFn>MIxQ+&$eK;zN
zrX)A-bz(BF)2roB$lJ<Cg^OB92Ji@15GfaMM#=7Ufpta$b*@d~3BN?iO|=R(1XxdC
z?bk+?j<8^Bfh`9%W@%JuQ9Y9O9PXIa4lFH`*$lSFKvSk%d_es(Xw;EW8J&NW&~Aj*
zkCfX8qAGx0T3(j`oIltv+>ELgs1wMPO_%b(QtL#;dgCPks{?G&m8gaikJi|w39b>a
z#FkNwy|A{xT1QSG|K^}Z#Ts4Ga9t(;xrX^mZ{pWFAH_W<IxwBU!ASZk?3om0&x}`-
zh0_VA<;ty%C+|xy$FH>9O2t94p2oTt{nBMDZnSvN(_Jv$x@6Pq?WIq0cbSiGuGnHH
zCmQO`#$VLzN;XSPz&N5IcQ*cVMcS!vq*vi(ssw7@lE4B9oT<o}D)S{a5`*wm2i^F8
zm4ic;s-CJtnD!I3@M~nv$3c)K<%5bR%jc#v;5zZZP}gSt+TBy~S4L0_-1W6p+TxpG
z#o8hxP4K=XXeTO)#tB9CXM8l8Hw<-Q%&!NCAUeEV_EQ(eFuXsB$|_Xx?Ur&ivTi-U
zenjgqs{bp(-fn2c;F<8cP8UEEMKk@D6)lTq!857MsR(|T6}VhrRpNtZh7n8TK-@lH
z+Y*CkQV}x!(&akt@~{EpCK|zWOOW;yErs&dp%lMm;d*+C|0avw)<ngVvtDeJk$o*w
zzYgvOEpfCwGe&6n{ZXvK*DYAS!H-~fkr(<}eHq*hD@N#hzAP?O)7Q=FNuEyyCUP3T
zG1%I7ZahB*SbC&e-V+H{2juestig$aSLU@~wMm46J--XLby$`W+>1Wre<sBkCR3)t
z$^yK_i6<Myip!C5Kvt?$keci99cvUj88*SFAcDi3yu^A#SL5%ah*k6@V?1`&?`+V#
z0%FFXzt1mBY%ROQ-L?|QIVg=*&ZBYHT#}qxio0*v-zX+r^|dfuER1JisFP?BcF!B4
zrhyR1It)Pm7bsA-o|_s+ZI+apLNKUS0+nF2m%3PgAvlb>D0_vvcth&qk8k_Hk}z8R
zsXTLNA5|~%4EgVm%#6qkBUouUVc#{}g~)zJu&lm{v~(>3<e$j?K>Res-L)p*=OPCH
z5}hTHPzzmOh#Uw6k`yEl$fC$WKrB_$6;0dq)yTm>M14xCA8}trs_H3gW!KG#FqMeR
z7>FR2^TIj<Yl*bvtPL*DL#4*)7Sd)W=U_U$o?Bp)3s7eK#Na9cl$6~+_Sat_1x>aa
z=jrUPZ{lMiZa(?iCZcg(Ka6iXZnJpa=HnaO7qT->BV-MSs5!w+*fj75um389X%st+
z*JRxaV_E}z^#x;&g)yxakmeS~G(;$En2ZuM#`CfuV_O>BvbY!9ULx$V$V)qd=S@Mj
z)Qjvqkhi?ZKF9MJOglms+RXF%AC7W8D^o9R)&}`r5hlr&+uWK#zE_cjsjFoTA^m?r
zzDtR%=6D~-iz6GB1^MRs0+G9uGi`W6El;hCAYbW+jlr_ezjuSD`U6K~i77$8`&E~O
z8S>zDPrnC_XWQOWn`Vmi3i3T5wVI5piymng<l9J=?EfrJdI@dfC2GMcJhXomA`~`N
z6cTOp9?!W4oOQ41`T2pv89s;<qxS?A{hNMSowOij(Z3@NxpXw*Pe+*+>r|ScqIVT)
zzs$i#P|<r4Y@v%FVCHv0Mdf$|6jAnBErW_)@|OKiHbn?;ie4h})YYDxKOHd<<^0BR
zIZsegj<p$CbNpn7MNrY>c*eE?+lKm{D}Oq|f~y4;eaVaFFIi+cd(ZRtpN{guHLVmI
zpGZ;jFCJC@I;d#>!W7u?D&zh-<H3J1QJWA{l)WdqRvs27LgiUN$_NBcSOA_%;<3uq
z7$8SYWE)&iQO#6c^41Yjaw+WEjzLAAEkxOm1lvXeVd$w)uA7Lbw&e*cK?Y$eNM+y&
zddP*-t3CA|I%0yuf{GLr;`tFg-9tQ+9y+Q;Md<vC{HlnNJa-)lDss~s_fZ`1rDr^s
z9y%hN-ngIYxGkxg&+{KSLYnQOSkzLHi?Hmy<A^hzLE8Pw{5%RO@-8^XT|!}&S0ycS
zS*JF-(;~VDqWmh|HEbmsdb&U24Ef5t4C|zWv6-7dx@BQdkqLAl-{cY(dFfu7>G)T>
z6_~`M$%U^ADl!H0@o%}x{?$yvQ*q73GmwuixK0K=^vHPy<|^jP)GyjUs7Ou;mcM<r
z7nrM)1kA5kj78kbD}${i(~mxHlDH49H@70%eC86TjSS(!_XQQnuSlek5@{t7hT_sC
zl6?e`2C7IWf{Nr}8~sy_NcKfU$ij^k4|=OR{eAZZls=s6lWlgvaR9di&mj9&zP`uZ
zlc4>ac`u5yi{}I%rEy{6xx@$j=@BKKn|#FMYE8oPmAr0&<#OS7^5@8{Hi~3s@|gm2
zZKU)-p4)%38baaM$}miRRpD3HAw^fK>^(+;aJVeYN!i!#P!V_Nh-)OG+bVyLNkq(&
z^4oqS;v1^`9|jeX*S7u*lp>>Ndd$Ji6b~4BY{ac>Aq@CX$j5T&@eb~-2s}|z2A-O*
z+3N%P758-+JbF8_&Hg-FT-xK>QkZqj%8l6WIsT7qki3l_8&dr;j~rBFLx@$w>+UKr
zf+aP)g?Wm9J6eV8*#lH*-U?p#o1*>updwo(5`$1x!952NiQ6YA;DiEmy}yd|7no3;
z|9(p(+j4rXo@0n5GGUv?$9&vuDd1m}#6}m@a|@11CYz1Ut0%SB^SpSD@bM+?4yCr8
zr%W3#E-(D9R|3`U2$9i^kx~MFRSEtNS!V)QWzjwU=XvhsifnQpQ2|kwiwdX+iaUb1
zrG;B&X>O>Xpr|N_=62o1UED63m9|=GTA5l|YU-Pnnw44o>MeWIUfXFmbNPSIJkLe@
zzkELD;h8gM&di)SbLPzQ%zdUA<Cp2L!B3?ALG#)=x+6nM7gNV{JsS^Idw&(KLUlz|
zp$ClkZX2RTrt6tdh^9%Q$1*<*02~symlt~S@CMNkSL5k^>=)6as`fE0OxNCIlCtro
zL^Rx+19`T}piZ^^Xq+lV2h+8u9#n4$&z2D60u11W5zh;b%<#dAFExA=|A!*=FkLUT
zA;piv_hks^Tf<e~(jSHBh$6ZdqG0R;pVk}tWe6CIy~9g02z@H3GoTg&^CfVUmj{6@
z5}=n2e;Rm#mWkmDNJIY}vn1_+LooaZ@H$QK=x0_FSU#R~9Cke|tMe#VK|*i^Or+_0
z&2M#vztf(yR8yIq;U5wBP!m&{@sjCMLk3H4hEWH@w`M;W44V4{fys~fR_m@PPlV|z
z0H^H-H|H1AHIANrbR$N*@gt_IkTG@I%f7dCS7UFd@UkM!bPbJw8}C!(-NSpf_=?k_
z<WWwxyIiI#7kRQ!Oytz?!qJi{m+oauC5tC%nA~JFHINh$q-goM=}N3g@g77QkAp$e
z3>COcpc6kLZ=9FId8Vr=6zaRA<tt2AGYM?Ai<?xJG@lM<otL_IlX6X6DUWrM)yr$h
zBl~&N)kx(<#t#RX0pIyEcF216x^ggNYz3AQ$RtoG`T8yyt_as6#oTw(aJ6b(3fGeN
zMAEOB0efrs`~;ttd%(0MsRX+HU<N#o#yWoU<vCmpybd{MIwR8zcnMEu%02?E|1twa
z{;F2UJ>Y<<2r)||%z%R`#;R4omr|=Kdn9Z5dBG{KlK-A-W`KM{q(ttNT`%FWAy$Mz
zAm}>+ttBf0>$;^9>MF#9nl3N{<koX_x_H*h1f-v5^VODS20VyuoKxXTbh=W#tIUAa
z^@I*pc0NT)2FHNaX=-q!1e2*iMNRyhc6>!E72gfNr>+^mkr2cAtu^8Ys7Y0}U8mXd
zUHrI9W<XC!0>n43eT+c7Oymfpv@?D0IbC&=FA3f5=gfe1LMIMM<VFH?C{-Cr)UK%{
z@`VPtqAc<qT}`fyhg{D53c1WwYRIFhF68|o_g<2pRPFY;8Bklu#kUiNwPQqZ>Sh`O
za93><u*3`qLvFJlRQDh}IL~zc+!2?n%yfQ&yK1;htDa7gzh>9d`7I5);hO3EA`r>{
zEtRP{Z8{GKeam34a!{OyR02(Ob;skTlO;{kBjNQ^Oy^t`5c07$`vteNe@uFP>My2q
zPz~Kmo6b5)ZHb$$Dd;*^n$EU0bO(af_aCOSy<(_5L1$*1@&u{Z!4T6KtLk-uDo}Z6
zJVKv6hEEl7Z=Fpbj5sI$`siV%Q?8{lYSl?*4w-8@BhZR^z15^jj7rR_rn8>pv0qO7
z9EvlYwU|xz$7RdcdW+{(Ya8}QsA@$hDsA}JShNcu`@^kP3OQZkE+8k!8}agSq3Nt`
z@zX{y;~x>jBzOSZ@!NXCOU|jVo*=6(1CoTduTL#kQ?JZs<q9UFb_{&!VvZlFfy^66
zps^#Gf$V)9WU7s1v=}iEB)!G)6aQ5SW`Zms@InyCHZ~=L?TD=uhb||-`(o+ux^4tA
zgkrk%D~I=Y)v>VtXpn2K>G(@3z>H@Ej>w4PmZ$2t=0`F_kNAR%MruEUpd#f6AW!~G
z$1g!twm&OPH`r|oR4YPi0<X);0H=S{8dm?sno)N$@%#Pr{;oyp9H319Yj%zMb9^qC
z@iOqNpGtj}!qP(jogyd6=$y^yFW-BuU3Gm&0$t94%bF2&(f3xMt7+s;%2cAH`V7?l
zh%p%+Y7h6-s_lyP_PeGz-g$(ux)`W5I7c&rnp-E1`aA8Y^{Tmae-y3OpO$>*5rq!%
zqgCGsFE<@0{C#7?e@%yVT^E`c6oTOBWsV0?2X%Kw!`-GM@}XqbS7%zYE5{xj4F7!W
zwc4&`ohFgyy+x*DHoaXL2PTY;*(e)7312p?5LwToHD{c*)33fQed8ck)6PL>)Rk1G
zW26{C8D7IkGm;YUv(Rb|=*Wwu04}<~v6iNzfWY*<1eg$dgIv+x^`@hTzc&WgBf?2(
zdc{FC!%jcoR-UF~AUW{koyuf5WFrUl>BD^kXl)0@rdqpe04p@|*^3bpoTBuOAG7A2
zTU($Q@hhq;x@Y~UwHNH2Kvs-qgr<;q9ZsJ-`AEfdWU>kpqGS0(dqcAjkb`ldnY?TR
zHsj2M4o02j{}`h(MgDV@(Mb*OCUg!IF9Qcqio2@S8k7;tDM&mVwoHlwI7a3Ptb@X`
z*hh<Vm#6sEPX^OZxz9KxT*kXO6;y84i>9M~h*Y&&lm4bBN##N#tjg7IWmoPbdA(s*
zu0F1jbgYp-1OKJBg}#L<(8lGbS<k0MN5v#7x5&{XP*}-4v22?3(WcZPE)+YNcMp&|
z-D<;@stp{KD;*@G=_ZD_V$x?r@Bo!-!{!((e@4h<RXMJVvMVRcbY%rnxi+U%<qn3p
znzv|@N7-8G3#ESLGELo`ZaNIV4uq~?$e|nt#|J`!eRV=zQFRyCnh+wT*l#IM=TMi&
z`y%h}Z8HryOchm0AatN<{7IncePZfEb4=sy8pc=b_)`ph(*rCOGEPIRmIj2)`=ptk
z#aGv+A)}zR@)6)R4RwnjiwsPdEp<1zEQMvwRld(dU6J|*-=CqbByUj4*wKs%#*<h%
zv8j10fW@@Ps*gZ=Y~)iQA4^eUHJ<FI8Fhj{90gv5GAsnjyKKt;fcz|(CAyQQu~Vx4
zIn6})^#s${E9Kj7RK{6G(|EqN#;`$)=iLOZBl09>-ssZ{VJ=T(E)|PgXc}9U7EzBn
znWW#o!S_R$t68$Rm8@hL!^OIacwhR_G)7XvnWx*4`*N3QOb8*(4qul#RCA+43oa`u
zrK0?(tP$Q+$!m&~N0wOjY&VUPKmxKP)^(g|O!Ws?A2X)nR%2PL+OJbe1=;q_rctRX
zWVgx&#Z6Yu){HTY8REF8+akyRZ5rk*86U?1CS@w51nqo()N#f4>0HD_EyFZAD)Xnl
zaw`fXhy&<8p%=xM78FzF9Q<*2X9jFaGCEKllQoTQ7za5PHaZK)^yP)S>TwWA8{M0p
z6W`c0dZDT+ehJ7$Af;IH*@PUK9K8Wp3sS0nmq?hxji#ALKNc^l)L-h%@H+Z+;NS?x
zjg2fL50P1wF`g6+F)2|`5ce@~uC8D)AKs|v3H}INCb{&L2hwQ8Hm{jReZNY(9}@}f
zH;qVSyrCWjdp!9?(R197*^-R$x6836rjejzOd947hBcXBf+*YcZ+)N~l`6?+<_z+A
zq2(x3zwXb-^pJAmRxngk4HLT_Cy;ChrmgdBce^4w%T)zvuzsVhMA*gAQ2d!`WGZTF
zc`SC>gxoUU`5*dzawGG}N$6=)Q~wMph!n>l%`r}pZ@&*x1{J#1m<%8lWFw0B@A`^|
zHfAu${vb!62iX)PEi<MZC?S`l!$Ed~<%9Dm-WQ11{1>g4aLm*%#L}@EXoN<e_nP{b
zs!@Zq>GDCKssASo(nrLiQ`hjird}P$QJIF82UE4%)PH8&kRCkdA)peXj4w@Ho$cK3
zs63}xy;J|)s;M^S9jM;IWG3gUT@9nIx;@bEyVQ|wjjm!Zljp0>>#j?G%Qy9tsv+30
zIE8j}73*K`LuLK-lS!8KA9DfD@@*Cln0~UIIAHvA1|4i{U698Ifb4-#_JimNbOG69
zpsBwlT^_43wiFmoUFtLXwd){bAWvEN(A%A65RB-IlGYcN==E3*+Zu9R@YolCy%M0k
z#=Zu;O4-|?PrrWb`?0Po(%XsPM$vTVDpP-#<GV)vX^>HkvHy}l0WawISdL~G^@rdK
z2gp6g63FF+2G5U#wR06MR^Lk~2k0z;RlJM@?w7z8UZ_C+6bU@Z%X(lb0dM^md3zgR
z+25#tgqO>}f3N}dPcOs{)~C-oAC$OHIKGc>U7$(eYhI8{L467Q$_oN5NMp4p^4Dsu
zeHDVyoO?}udu<xxAxayCI1A9g0$JjULh1!YKmp5+MpPJYbAbmW3P}`H0;{D~d*L2M
zS1EW5*s%ziw?7WUB{GW6!$U1HI8%j6EKq)Y2TfE);ufel+zY%JIB>^g`{AlcsPpQ2
zlFextVm=DIyCt7vl8@ByE#R0?s%4VeUz_?73MAVZw7VL1^&{z`PpE4@53y(ldenq@
zdQX#0FF$Q3i}qlOO)EP0GL5Ast)go$qbcZG6RckB&!#@zpR$2wrJR0y%JN$&ed+aG
z@!rlZWp(vA$||dy=uCYMR#e^mQdDT_b5#sv)7F`~S2~vrTMzJy1>SH81eeoypQDu<
zsLhq^p{Bkdga{dvo^67Zq?czas!blK&^j@&-^=3#Zy(lj-Zk|YRqATB0<IsDP8a!~
z?Yh3&)Z-CogMMA*<``2?NXA3^K<ipEI-ih-fim(xF=8&5H0g<qkPVuMccn%jy(MLm
z(Wl_Qz?T?PMi)KR9(~54f8(@xb%YnJsxTO+#?|8?bxl)s1(6Ok2Pt(<Q*{P`29E0k
z^7$Ckf8&)lZV5;kvSvSL>axr0J^{i9Y*LdleWaR`IVv|fFc6&NFualTAQzdnRx*S1
zS>P#<BD*e1b|8)buiLrx@FUNGJY#F+{Ypgx+mgnwmE(RV{%4dju&u3><EcF39cqfC
zhs%sE2MMafseZGlrgW=?fLq(ulrQoy{XmLLC%HA}!}i>d=ghQhZ6s<G`D-3*TQM2z
z$oY(9QxE1OoDr3S3_2bl6i4i40xi!FXye!3dkAzPMiz-!veSd_5|N3IqleO!8OWQk
z8&=O`Yfk3F!m$`q#(rJHB;wo%sS&36T1B}6+vb|Ti5hJdx{ZN~?l(LuZCnUse^?Bg
zBiPR7WuY4<bccM!QLcuewam|y1`6t;Z)23JiMO@%t8=1?(Px^;uQ-||oaT-`g9r(Z
zkLSVErUc~>qdvYD&=nk-IJJ?i<`6Cd#)2Dh*qn;hQgkc8EvE3e1rJac870O)1v~+6
z(7(;uejG;?wDEF>C4F!q?IteKc**=!BAaQ8nK?v%K<HE8BU-EVI%{4KeSl`NzM4_T
z0BEPyM<hy290wsVai!!+=bP139^6}{b#7qZ?XP|GAGO$iKW^URWgZv(hY%J&22JuC
zW_LeL!zgG@Uo?9{(lAO$CVt<S+q&ACZA6KgIW^kI|6-9{!@1P|nJ3LQ@m6mbk5Z2R
z0o=9+%?xl-|3Z32p-D*_QBuq7a#W#RLzn42Tu>zXnhm4#VPiN4QBWkAo)0T28p}IF
zg}P^L(T;*58bMtau}v$OKzCN}^I{4nVDZ#@(a?ek$%+>R9v48n^{iDe+NBvWkrwCC
zEXg)JkKVv-G|}(A9_Xc<TES>^^xfCP#*~o#=sVT^9|J^eKtW%-Y%x<U4t?<k6^E8*
z3i?8#%J%x~g1)#z>JqTkt^zfW-Yi1iRA8NX<vnX_!4RwANtM^CfO~6Tq;XsOnh;fM
zoMS<t%?sQGo!Qff`5{d1>?-JtUb#<D=%T`c&J<d`Q-RJHKEF6J$w9sQ`W0ki5B%cD
zq+PjFqev^rlFZem*)Khn?+k>0qbI|sOfPGw5kd#weLY+RPJq5U=6$Fa?eJZQakWhQ
zI^P)_AvmmkL~8;K?LbsvzB439k`nV|3<0+Ud|ev3GLpLHJANXQ#-k(?jlgYrsC6!X
z9$OlzJRHB=mWSh)+wySydRrcjU#&d;kUU<^cbq4W*ho8%^UTQo^>Ca|yH$_78(VpN
za(f=1+@8n(-JZw)S$XVf?8@|}hfk&SL!!whM-DiNqQ>S#^3qB+ER5K6v~SUy`Hr?y
zl5$1emPVu;rHIqfHbv54#qu52nyB1czUWlGqq)%U3!lo>6$G$?N+4&35}=XaeO)DK
z*+Vs3@C0aDs-7pwXB^Boq~dYUgiC7Ud;^uA60~tXP68wg??DU}juv??)>RpnPAk0~
zmv4v=!1axDMRysNZ+xm`B5Zf;%{L~&Hoil6;dIzeO2w6w72hSR39tmB*zUV4-<X^%
z#q~WN=Nff)t9+xcBsyu8MLG!Y`D1Bxqi;(|6fR}#OGnnSl}wDjy`Yh$f&-7|8}hp(
z)K=v<fbnnay}rTmuB^-l=)+^j&0tTyNxl(n)!j&FZ{-rrz)vnJQ8XP&74y|!Ak0yf
zdpF+ICTeKDe$$^WPo>juV&CIW`Why<VnTOWsIL?&UylS=8}Ipi{Zs0a@SYX*X}rp%
z1Rwg-45_hPqB*fVUteQe1<xibv1%F!YVFx%^&ou>1Ej6p#3;i6**3VWttUbtFfWFH
z*hUEyQP8zCl;-;*!BwZuB0XQPROP0@N@INy9(J`l`8s-B@#9yDDb8OB&-$zL^$JXz
z=Uux973>VD@+{gx48>KgCAn5~m4K8z>)-i$bGz(`{p^IzkqKp&YuSkHtXlS{$2G(|
zCtq)ZlkVB18Htl9i&V539RUQ*a#yc>J<=9$qR-|PiNEXFB&w@NgR%@x+O~W>46!D@
zW>bZw)lh|7RCjQKYi|CVHoGQeSi4oB)AHYJA%q0Rukt;S=qmTF&i@7xj{PiTqL%*+
zy-b;0H1b=l%h>NV4L(nY<RA9)c}n<9%Rj6{6EZek%Rj2}3mMy*SIfyF$pd5Z?H|E1
znjE*1yrPn5jL0dGIe+QdOu7ql#&8B(Gn$^k*q&RKQwXItj`sm4a*7mN-gk}7nZUdI
z1n*-O=9K#1S53<)Q=(JGK8H(>G4{~(aOadGW5si7gPa*$MAr0!k=80g_MrJXJ10GJ
zehe8iP0N{YYK2-r28njQkh2PZL0!tX=XlO)_0EMXWm?X9e?b=Q$XPG`kX%f;4ZPAk
z-t*VzSj$1YKQ}q&B|rVBjGRO8QP=d&i}5XJ%5?J%lx@`S)P3L=IZ3MOKdX{7#|{Q$
z$l~b(ALj5qy2ej~gh>&jF~FR1Dbch(AGz5+;?kr-${;@>Xf%5pV<3wo0bi-bkzE{u
zd4vP8P~fP?opCz{<HI(4g$?LOZsIL;<UB$ld;bo(c(kx5!0-Gywznn4V%fmv$xW;d
z{idX1*67&Y9h$+wAojl>TXoyn)S(w-%BMdz|90Ft+$F0Z@~x`=*yEa!#CK0z&#y@Z
z-dOGSI{Qcs;>bsBDJ5<BXAKTA3z@39ow-VeWIP%e<*!zps@7q*lh4YHq_zHP6+<(@
zU#pn}%Kf#<RJEd)`fKG);#7lGtqKVFgg}3+$L&?HtHpuLnq#;_YpQh+_lyL2WWGiA
z$m&jVI~!4-h(j!S?~)d2UMC5CI0)M630mP+vRNNrR6wPg)@7%>ISNpC7<hF$&Fgk%
ztA=O(6{8s~IHFT;9oJ8&r?%7WOlSU{lFx*>1!RT)1?2osi&$QM1F9v^0&|Dv6$MBD
zJ6Xg?S`;CHUc4asB7XIgy<50`#&jLUc+l<mmiQDtMKS^;1^QUjMFNuIE})MDgz->-
zqr9XcLBJqFlYwGu&kKfpdMQu-%zzH;dFk~y3Q>zeJo!y&_zYVk`X6U<q}P|Dma_Yo
z{5g@LDMNRO7S(6GU<2^EWLL76EaIM`Y_Z^4$nP@f^{QwL^fRTMKT)1SZ`T~fEMKIk
z%l1l@I@UWVq<%8Kr?f6`qx#K{kWH?XGwwyVY^540FH*<m%ngd>0du5|FE-LT8qGhY
zF*x8-zpLiB@VS)kp4*wy)ST+oBQBFTLP{5Bt&^qr7WiCsy@DBHm-iQcL0U+PdL>k7
zEd1)txt$+VS~OzhZq(_`huqHEs1x_xY1%2JPDj-8uiNRC{(#e8v>aGQf9RV{8McGe
z;#Rwzbuq2YM`UOYFQ0)>gotpf2+hmvA`JOhq3IWWM?qQvT4ppty3r<;tvqRxA#(eL
z9<j?t%hsf7sW??_j&T*vm+5MYi*CnFu~W@2Q?{gHLfFP$tZbA{L9GppqrO83bEdf+
z99*LEP6$5;AjC{>*{>28_nF)Asfrs*+_qN;#h^(~_`Yt=sg~nzhwxTMNTxsScKo0M
zvgY3EUALp!%0c?!>E&+6J{G5TCz5Fg5nQU=P7%2gv>5Dh2(1cO4|3R0khrOgqgoOD
zu;|4oO`AwoGM$zs*`h1JFLX|?=61a7@0}1y5k9bqCct4rl(gM7&1hLCev<TM$6E~~
zXJV3pW*q0=t)ys6@XjHaZG1?^sg|kmlS-VoD=`6Aa6%h7^%O9gWP3U5q<O=G-Hx+#
zyVPXqjkBQ`0SuJ@#%scKU@E7Z0w$xo;la{BH-Yj2tL;5$7%|J`$)6bzBYV=j*(#R)
z*^-4K`lpJ&VD-wD%z{;0-#JInLA$A2Eri+fl5b^8SCg1Po9vpOY`!c6Kgaqmo%d8r
zS5k3rt1qPXhJs#1|5IxpMqrBkFz<7@(xXJ(gEDFSwLiNZPe|jZu0pCW#uCDP({~V%
zmV2TVg&r5I1iMo;gPh8U`QGiAA!)wx<)ykJho<7Z7^(k4CsOZrJ8C|dGEzVJCrY&D
zHVW0}nxHx-?sq#1QJqE@JFQSur&Y%y@kM;^rLvqsStnp3CY)l~RZD-;?P$w9#(aVT
zq_v@(OJ{<ioDn|-%dPxwM<*Ir6{*!sJ44ojt*@nwRx3eGJ>zy{vSgpuhC8>7RuBI#
z)$_NZyxGST*>M}!?H03-K)w2>+i@oaLYIY^oMDkV-bQu(e^6=RHtOygUTL0xqWrSD
zJAmPW6385ld&7*h?u;y2t6XV^m9!9t(T%j-h@osZf%bM__kOn{rLhETnY8fBByEMA
zPs>|bKlD$OKWm%9EA3=Hf$Wd1bvqg>Hyhcf$CGLLJ}@oqOpiT6fCUGpg;qnv)nt0Z
z&VeOSyCKs3x!9=xFDTWkx7KZN>^{M0okQ~FlibE(vQ*oLJ^Q<jGs-<AX6X~Yh;+8Y
zoo?exMPOxb{6}REL6@#><3F*Y6a>2Fx{coy8O@b03&J#)@zt$@y#)#lF)r3<Ncw00
zOZC$~Q54_za=I(V+sMK@*ut`F+FEwCL?vvLzovGOFLU@-VT{{&6t6&5pt|JoUw%f^
zXkCS^EPa$HjEmUH373E`F<S#yyN$<HB|uySu3!YZFt~YJSJCZd6;1KoJ;ZIOlW*|N
z+y;~5!ne)6Hs1}ROYoIX2q!wMVa|}bpA2&o86hS%#2U7lPU4H21LxT1Hhj!@+Qd?z
z03R!PZI)qT$UBph3$p&@`=B)^_$0UAG?)fF={B}Vf*yE$@^$1ya-P^!OjhPO83$_$
z+k|<=1-G$G2?6zLGhJC7W4UE@h#t7eZOpET-e^V7wW66gdIq|UDU$Fl`H7g8G~g8M
zx;nTgtvRKctiw{Z2BQe1t9qAkgl&`C7=h_|UnH?ugp=_VMPw8-Mo@qA35tuEoR|($
z*o?rHljBVj83iWZ2U6IL=wyp=Cdg@ntG(L$j4>K>=ZwuXmiLC+7)u{YSVi;(pfX0&
ziyM$>q3odX<j-x4Lrxj&TI$7#8kYZk4E4l&L6wUgXPiN1@8{t+i)&>B4)8m5)&7Oi
z7|m$Mr`hk9X>GYXjem+Oi;zafDTb?d3(5ImV4DQsH}PrUDY?()eGWIZTj$%B!9knj
zq>}q=jJB(7ImoY(C^;O2{smhO_xj~<9^_eD4)^-y@Egb<?ed(sAXmA`Qz|)x;<S~+
z!BU)J{DNZff2BBb<4JLYASus@-&A?B+q$BL&v6_5Y1OuG&~(x&{TZIjCkRMu^v}4h
zQTu0+L$(}JJo;a^aR-Uonc?zV0R+$zZF6{0iM)&7Aq)Cb&-nJWbtQ!F;8Y`H&|tUG
zMbeD*ecjg8SU=+nXy;1s5{c@q`Oa;$SA9?lyN<*wP{#JLk|BZj=@#voLy<e9*+Y-J
z4L9w2oDpA-5K*>2gMiL018w)?;#BIIU`x%7e5Df>j|YZ<6y0^Bu5?1LR=geTMzB&}
zH>xPwSNsuhI(`fj8iPF_44bA+qH7diw`X$J_2#pVBhxtuErI{?$?c^2fEll>v*Gb4
zq$wx0z!<h4X4|wx%NXL}dgZIq6WC(fZim}wL+iEQPFI#sN+<OP?vg-LUd8}<RL{MG
zFQ??Y(n*tmiE6{<_{ZwIQlq_mSE}c#Cf4M$QtACcMgv=wXQ6u~94mR>%`I@%+Qy4?
z9fy1cZ*=-U`cvp*n$bb-yE%2kt$!g?JB-vMJhDkMI6&dnzY-L3oJ1F!v|19pM1r>J
z-ZQs;Nrq@iK&wpJB^`|Q7x|e#qXR#eMu+O4a7-i=ZiK?|>&o=HO)Ugz6N!76^ESw{
z8Uhdfms|e@p>!NX)kFWqTMkV#=^XHe1pcFGvZgpW3RzZ-s@8lhRH`rCdKA6A!+E4v
zHHUcgfAUlbOyy-YpuWf5bdHbn&$0B>;Wu)Z2|)L2Iht|TS?2PbJlq_qIi71sO*Ut$
zZ*+~04I$%PBd5izC4kFPC!fqizM_n+Z6-YeCAF`s=J5SS^?Wi_k{b;$G9;AGAz6mg
zjGlwIxM?#Jb87a3TmO~%bUG;2{DHTgc@z?HPJUbh3^r4o1g_zK^jyU0jtFQ+Nz7tm
z(3L5v5?I4aFQ5}MP$iACl2urS`w47C$1iPROhE=yCRxJpYExDaS}Jo^mEw@dFRZ8g
zQN-8D^rv_P8u?aG#5V;qHRT0Wai)hoMSGe`wMv}|hiOqb?qUEJD7Y%k#`ROU5NY!&
zW@!k;i&V-fqX-#$EN#(gu3wCCx3|SQe_QOc+M?5ABGEt}ET!RQofEFU?Y5`|)dRUG
z^*W;QMY7Gb`(^*v$N@p$=I!)7+==;Y9EL=bevsJ%Lr^})tv}NoEB!7N{TMi#QapV!
z9pD+}FVX=@43Jk*J4}ZDe2^q#kdDXr>Ce}<hM|%a$P(4D6Sppl>MC7#kU3QKz;Mm+
z&3l^BS*mJOX2~2vN}PL+8u!|dhN++VJAl8Dd5_S~*NiUY(|N1Vmnv}`*rqvll!80%
zJJQ(|k#$jiD^p}C^sGtc3?=Sk#a}hTfqgcQ?{DLA%Hok(M|ebg&soVbIXWma8xh_?
zlzT_16{eI@G{!?!SVV)#6Q4PgTeI<*Z)$xo_3yg%?UX-ZRkgOa18Dhpo6@JqIx$gx
z0o{!(a#JRao+-#!B_OZHHkwv4Aw_PU2iw-VQ@;eYE~!xBeL}gU1fKz(q^5iGC3k=c
zw*T%TWI*@Zi}1PxQG>m!{52?(?#IW_Wm=5&F|<_Uu3}C5f$}!L&+XN%UuCY=Vi;ig
zX%{JYrmYcGEi8>FIZD>A%A$DX^KP9JG19a~7x{)V`a=+?g>r)qttHS#Vln!nD3wZ4
zf<;RzEg`@ork-1$@8{t?uI`%#!|<hkT1%MBS6XK?S#pcXk{Tvuf4cP&n@LtZn}_T^
zTR+d$lUU_AP{bpP31GTwN;T^i!&!VsB|g+WB)rqDk5L?}wf*}bR~xq3ccjP_8$>DI
z$l(j%69klz2hw&<O0I+a6Xe<cATdXx`crWVrq)9?B8iDAPZTT$yYLM;pm*|QeAd$3
zdNHOfyB&@|FR3(2Ftv>Y=uuPYKvQ!h(3qD2K&}MP!l~ncJOVwh@`6#EIu9=L0~}Ky
z0UknC_Fu3%U3NX+p)6PQFsqxPAO|!{K`gC4!jW{knCeYpsf8d=`i0#mq~%v{a_cx+
zs^zm+Bi3%yK~O;D4Ig{Nm!8e{9Mp8Us;NkHIW3v%muPR2Dv5SIY)dq^wX_x`eGb@(
z@R;lAxt&!O+SF4Z|79DY+&gUKw;(Ut5*>7#M0@2>*Fw$ks((tN_aKX|&xxo;a(;z@
zWE!6$8od_sg6yWrd9YsPywDA&$uY=Y6L>-J)A%IZ=tWPRHVEh|5~VLrn+z0-M9VCR
zc3oX#ACzI>+T}vWW4^OlE?1ozDeq?qZbzwaL|0c#0R1XwsBd0ZW-?vBv&GlFiHOfu
z^E)iLA?mmyx8B|sbxs3Yx7#cIvuU$xWIT6)TTi8S8owE`@p1mEBYHVQ5z@)$+}8P=
zY$@SG@>|mzk5h`_(yW5&eu2FQW*hgr^+c?J%oD{a)Z~6p{1ILwXwSq3mVY>HE+VkJ
zBK7?j8b0kIU=vc=IKXlTa$fe8<hUZdbC8-WCaG~BgHV6C`?@R$<y;RIgJ>v|#?F^p
zcOeQbyQQ_;VSAXD<>sQ|KiwB>-8py!n(b1k?)}wGJ?;y(>P&ls0=$A-OJ}@bYtFRq
z@NlPn1(MEq!PcBox2BcWP90mwTF#d(g%M$lOld1j9R2efzcO(%g8RFAoKeXo@;+cN
zJATq(UuHm8Pth(}Wbd9Oib8{wjsc_}N_XYcbFH+Sd=@JGwG>aXR8HSXe=V&5788*E
zTDk*}rFZGCrH6o5kmJiCmUeY-r26YEP2yCWXwnbeSPjU~j2`xV5^9Or6N$L)D{-Il
zi!07At{zWX;yOhWNKY$04V*wS^t4kbnpmmQKY$-o)o6dJw}mT<(X_G%OIoK>Eoqf?
z1J?;@c@fu#w~Ol|TU=$xp)3z^NpC!uC*tBM;^Ib5qsJx@1AVb<5-?E$3wc36WlBKw
z#j>S<2uS*3*#>}}Ak`Pkb`f~e>WdLt*_(t;Ag+VOmbiM9DRFsgI^IEh{Iks4UNLVB
zgyJkrr$BUsgBx!bB187RboXm^m+NsUOqu-Gl#+!hoq4UNNTHU}m7}WiAt=f*tLA;_
zYj3+>)V-oM)!K&b<WV8&+H(-HrVqV{JY|YihWk)<HW9s7Ytl%+>>BU~RoQUL>Onob
z`8($}25#v;(rx(y=^y>+;Y;qN(w};63!y))P~9n{h;{8Xw|6J|imr3+kbKR(ok`A!
zTx#%syjy<oR2B{>_*+-q<@9~=Tw-L<Q|6=RH>qQ%v+i9^uMxWG(SSJ$f3UovCRdIW
zB~H@ZGbOM6#Gpyj+i}5NL#@?L_b|ncvh~VQzh34ZR$rL-PWQy}$yHHw)#0d-I(0)2
z9WMTo{GgdT&vJB{vxC^(&HS8H!e)!braxvVVO5xNp8Cg08mpBCB#Lgs7o8r^QsI>@
z59sLUdruAD=)&FZ&NYm$S&Tbd<(u2f)u4gh@||;qr<5s6rS(1Ai!VMzL{!ys?(WF8
znl>J>O4Cli7q~mMv<$JuQw4=91~*Rj#h0G!_>}C6FI?I2L20);#>!P!t={eUD79iY
zMGRj1t{s<BHm*t3)lXUliU!^BI2y3xbjPJ`jM~3vsb9U_)~Tbct8=}U>|%FZYc<0r
z(jLB;rGL*{SaXJE>Bq>YcQE#;XRj<>zK)V5R9u-X{d}ZUkIDHRmx)fjJ4^qlW~K6u
zzyFD{78Zpf&RQ~t`}1qE^kU(@2inKlWa-n`b?d#y_x&As0AH$w(ky+cDi)da9%XZ!
zS(8ccY5$8_@=p{$i!)^sWv!s=3||>sGP#7n8cqfDJ`CZZwORV+P^psg$F<J|Pp!08
zo>8*0JZ#)Rj_6#?Iwu0fRqg$^QkJZ9u~Kv2Q@vd&p=n_%hhFiLf$#I)u12*lXJt$4
z^cn1P_HlXI%(5sts3L@Mt(TT1ccQbGggvcyG>a(%#P3;D9?U&_?(f5gyBSvNW~EUS
zX<cdgUa>2M{7w^9js7rpB?~m_H~l7l9J>-#l3$ErktTL^kaRHgSa$!dUsIVKBKfrl
zjGe6*$Tf>Fn2q8o1|*tIsqA0;jh)>JO;INqrnHQm%WCSKqik;WO=VX^K=(K7i@H1+
zJ6Q>+T1)?pA}Hto&R;0a2LZ8@J*Y-03cL(hkeXjwro~QfCHWA30(b<vDQ#jWOaB!s
za1?kA`A%t>5<5-h3*rOdJ(S?i3sf)l?$}c9h<NFu*tFIxG;lG+-a9!vA%4q7QTMy{
z!Ia7kMmnkoT_F8u;GQXu9n3!(TAO|a_>GJx<b8A>DP%eS%j<$0a3Xe?Qgy=g!t!Pl
z<!a2>Ji*B)OzOw}Vn_Q)vLWf>EUec09MX*37z(fv^Q%_6<xG5^Yha!cJ1kpXg?b0s
z?BW+O90y!-zg|^IY&+zKL`ZA3N;|Y5;u7!gj*V@X%)i&EHwEpW3hLW)7e;WE-5!>C
zo}H+=aqJydnFg_2ye>8swyC8Qpa<p76UpQrwIq{@DHh3;GZQH9%?X&?Zz0QvOsiEG
zo5m6F+`Ypzqi<c7Sd6}8UOp9I<-^GRo0nyb-{lMH3NfQ;UnEveTb4gUgY;RaIUd|D
zPMfu4Q|gpOV9bvig8Hzav#qxki>7E=W{cP)RZ1xW6>eEKHYv`o{G3-~lRBxEp$#%q
zV)+)F>Y)97TLhw=kVYriq%VFI8=XoP*{~p4^j$(3BBM`ijMccbQH!uxuF4{LFQ|~F
zn{w0aXkxF>#+&56hLNKhuMq+kU-((>#%tWL{G9jYD;lq%`=U3nY4S>AR=DFK`^h33
z5O#}9j?<|m>=5Snb;Wu6o)0O|8gng%zIq9ATztCmh)~6eUk%q9PrD6YZsDbu4r|g_
znUH?5Aw^o_ZmLZceUIDd`I!SLkz5IT?7qh72uB^}?$?D<om<v8J%Z@{VZyF)i^g$M
z@qX)l?tUy*tY{oZx3r7dI1WXs`Jo8wC+25&kiw-jkV4DNVRt;)xB-fs+5*a4(W8Dp
zrBbj4A|E7D{yOjqtK+FmYpTAZdEM;tq$l!7Xp_hkVfR5UP%H8yy?*Rjnh!^}K~ImK
z!?}hn;jmNZ8v6fPU(@&81l3!1x*HaHbeTf+zlh2;MCJ6oA>{ZW@LqcYnCym&h{h3C
zznWBjnV7F7fJH0+75G^KH1G^^DED$ztG;EXCb#?<{nbKJxvXy(#7fL|O@ICYV$@gJ
z<yaDMyEJ3KB^ApYr8`$V^nXdTxHd-|YSQsFw=8D%KShj8jP7|H3#D&4`_J0|XNX}{
zJXybd_B~;Kh(v#iBtk#+upwTpSDl4K`A`Ecu_5zzUr?9XkdY^U9=#pX7|@ro^lh%u
z(cc~?Z<FOQOCF2mu}&V_<ht64*Hyv*J9&iHBlALIz!dhW_2MA8t7iaL@Kjw>Ib0)y
zK#$%@DBkC$D`UVl7l(Kngg=5o80gZl$pcLr6UTe3JX*-3qda=aLym)O(KO@kA&5dw
zcTvl#3$VH*FI}n1%iGlL3(e1X^n@Q#-TFZuL`jOAuyLB_CLXbSth$NRa}!&rX19Zf
zGg4xi?jGd(dH}~i7I^f)U^_0j+N0O@2UeGQbalw_O@@qtrK2@%C_iTFqCc?WCa9&L
z;<kc9^KH}?P$M4l=(Wh%M(whxYTrT~235C>M>jDBUYqQFaQ8-nYhRP{5~xngJ$eLW
zHq{kS^Cy9-p}IR5RIi^rdVMB!7S-L2K^46Ys-|Ffr-OQDs7G&L72i!zY7^NAEg}^g
zgGi)rXOV-pRSA&88+Z5pNoU{4?Ky$YG@x6h<x8*i6=UgrmlcX`758tCTqbl+7}%;I
zVDqr{THH^bsbKHkLtl&giMOf(`VG0yfRCiFuGS-SHJPTwyFAk%81xaO@h%j@cOAtr
z-stj31tqDftxIa8Hv`i=BT3=nwL@Hdy!Sc9gR8~hBnsUxu{)$K2@jDlJmcgLMOACb
zv2Q8(T^cotukVqi`$1I>)-C&1!F4b&h)~~Eydawyqa-k&m+3&M6s$fP+4pgI@T(en
z<RQ)R{=a;ggIvw{Kvpi~GzJ}%uB`6rRL4<Rsg;FxJ+jK$IQIWkt1N=b-3%BNS@a*p
ziMhy=nv*ONJp4R`G3al}E`fSFMzug|(RD<s?@QnXLiXB}{LT})CT7hWpXzysbYs%s
zmNP=3h1*ciYOwd*1iSA{wI0P98rXvD9KkZ>b?o~ubz2TBlfb9EYz8(;p(Fv^ac<+M
zW_OVU?5SD_9)o6wKfxXX-rfEbQBIY`-0Un#aoV5aRft~pr#MOAUz*3tA}XADxy}<?
zk=-r8pWqxc@B0&cN#F~Sl>Rbxi;7f*TJG6I7WdZ{NqLtyC2k>cabj6{YFb47&xs#A
z6yx>sTG}8g4X@9?195-8e+R;mwzHXH{W{W&Quk8DKUE)-Zw$RLLepRWLSaP;Pu0_}
zs@$Gp2zdQx%Jc*9odmo$cw-S|#vhF|{ctee8Q8-On3^%5&kT_O!{p3{Km!S2$7a&e
zX10_-7hYxoWfHiP7azdtw0@+ZT2I)nwDm+i{a1=~Uoh9cADKq{Yk>5t_!Fc$;-w3r
zH`>fUsqK*!{<PmC1}*;MdU`W@z`cl5rG20$#B#G_{KrD=6$UFnO$H{QU{MEUa|bad
zRO4bTAmt*_>dMGl0~D2UJU5sac78;s!$j(yBx}^V_Z8p7dtK3dGpjyp^6NZX8EQ~W
zIB9d0(OvnmMS551BncP9-}AkCudAVV9Hsk?`c{Zc##6Axz&tXETWlQ-yDyW}GBn=5
zPx6jqVN;HK-Z#ibFTAMGJ8iVwqOPb9NltMLq&0_b?`MOy$Q5++<}8LU8RQPnuJ-Ej
z^4+gIH9fAL{vDs!4`qc`KfIN4JP2%&z*D^J0-lk;K3-^*2VRxHYF^F&EP?5VkMi<=
zz{e6e#mkStMG1VOX|o~$EWUpDYwG-$W(?^cpf%SIe+QcNo>^(sE}pF_sojuP{%Xjt
z`RVa<M8dwC_u+^g{);e{3s2Y>!gi&87?qrbYR&2_3~ccX@t;^lJi=g~zloHGEdpA2
zQJm}tU__CQgfOM!qO&2t!|+Hb%z6X8gyCm)i9WPS<TdVBl8wLeKOFBrBpheMQQA1N
z#t?E_vUO&US&ZtmnK<q^5JQ{!D=B9K^{8AMo#jVYX-tLmsKt;D24~F<WJaXOt7MCk
zD1MJh9VijnIAk}J^(nt9&eWtxn05Ha-9yzl20T=4>%oK`5cfSyrtuQrLi8M}gnX$N
zRHuCNhcaa<q5AZUp*7d&`gM4nL`Y6G-Gz!vr=qK^W44N6?Y~v^F}iC)1J71^^>8Ph
zF#*Hi=#NIt8UzfKz>B<00VYY{FfS{C<r46|!<*1PA@=@?;wNp@JfRu<h9wKX<`8zH
z&&|ROn{|~{&}UoFN98b%Jh8Tc9oSyJ|7S)y`MLAiwSWK*fj)`=M@^C4`k@;M<M|2J
zt2x8`zjR}hk=uSWWHv6y>~Pqt#$bf(#^Hlu%-pP><I_DfCqI{G4C6e+1XiHX3C|Fr
z5}DB=N<ss`Mtv-SgpLVvo!hX!3<U!)pR*eQ4K**-Rzw#SQNILgYz&jL1&SyMqNXBh
zMN_S4?oUW@L&Ku4MbieFbZCrcxyzID9}JfX;}V(+4Fi?&?0Dj{fJ{}9vvx(qA;$}q
z!`D%C0adv#1lL-TUA9p&%y8nBYDClX>|+)5X0GeyRxq%StNf;U9tx1A64%ESOJ;Q7
z*-JT<5dE>Dp-B=dOy?o|u;Jo>Bw!AW=x>n!>}R2Tic&=H)byjj6JJP{0U@pX4!B3u
zEfL43A&!Ia`Wvc)GAbu*F`cgquh)gw7I-y8X=Y=dX1^w^j@Yc^082nf&VcTURa3TT
zELQsD&2SE3K=Nt{PnphR2z`Vuyktb2yc0g30v~XhfIj&}(EA0AU7Uli&%R0ZDUJ)*
zEb)RW&PmgG7V9$lF+q0*T`gl>!e^%Q`Ebn`HC8x(CZg#J3%uPqExiQpuhx^itil<g
zD^wB`m4uo$-*hsR86)1odkn4OM@e;gc>=q@Fg|H=wcdm-2*>YC;|6YqKB)qZV}TJ8
zz&_2H56qKHWlW#6i4Yxb&hj}VEkPuVbC@h)tohcCaMkj5B?9r2;Hv4|LQO{1lGInp
z*L}D00SxyjIM7yekdWan=H&?Rx&*MXbKV8slE6}47!~GxOTFAX6<@j<+`*Yf42?GO
zHMG+Gil)t_CFjaw%24;4yoCXBy3CEoG&dTElmNWuHU$zT@Fg#8fOIL?-?n7TKu6`{
z;#QmWi0TStqOPQoi*hE9bVU#9Dn^iu-;=sq#<J;f%M6ZW?8f5KhwYD;^`z~$jyk0i
zDPzg3DvFO{HNjMy9>mky_vuKEn{Bg$OzGq-LwC67q~V@RwIec98O)|7+!nyv<6SiQ
z*lN95q!=!qnm*HX?g^*hF3L1_7%))O0_QJDw8FI#GBtSfWG};Xj;3YuWYX_?2;!wI
zl+9g;BwWwXM3k->gM%s^Uy2kNvly26%@Qe%oR!j>VVvGPLD~PZG$2Pa!W!tkRMa{v
zVO|s8=}|6Ez)lJJe&^NuA_?hD>WNooItxV4lSLs<Sc(z)K4sYksC%=%kVau%5&Av;
zhv;gv-?ZB5$w7Y8klnq!FKwNjdw^8W+d4b<V*>Bn>N@vV0zad!*-0IwwCZXtN9ATT
zRBCdTqGwSI7jb%9OZN4o1x2sN>;1%BHM=ZCzVmsm#<g2m)f+v?cRkM)UweQZ^!l2O
zri#UOuu8bKQKAbL%aepu<hn#@iUS0Zih8mGWjfDMkumi|__i2ah+&=^s6z>p!!^0+
z)foL>DOF{-6eyWe)MDq!9_Dqv%uxmQy*1j^U}PqdoCCxBQUFo8VLfPbvHSzM1UCAV
zhE?~}_IyCLV*}}%p4xFl#V}S4OeRwd<h*DJwBscg5Z5O9Gnpq)&eguGF|Mef=-({2
zcFZ%r(PP+3&ai_=$ya4LQ>5FlSf(?i`o9`^On0b=nRdjl5+Pl~m~N<=<=*Cm4%@4W
zZ8XL3H=HBZ(G%$rY30iY;hsoD*7OpZ!_cQ6iGfB<Vm&Q#GNfvkg{B2sa71cV2;@sZ
zYUKrHi!0Y1lCIDmkQk&i59OP;k?G2jfuej=fkwmshU76yYD{N=Ss6R&56VA}!I^$^
zvDaViYf7sc9)bhz)7Z0;NG*ObK1#!VED<$GYit)m;lRt<)>sy4EcvMqh8i+X$_zTc
zu_RHJR?VR0gATYgRC!AZFRk1y(r#|tSZQs;$KA^@yJXG5)8F$kn?Lh1(SgPc6<`^B
zJS#m55Y*fWj5OD$fIX>1{AO+aZ_ty1MSgGcc6|;)IR+f&jNf(U#h!~4X_v&T0s9f)
z{ZK}+>xZc!t6Wh08m#xnkt#lcsl9$0ipzj3FM4Fgb{)rm9z)-}za;QqhVCe!o&<j5
z1&`3148I%hYCT1bC5d%|^&Uh|TnWD$XsGuNpeOti>w>=FO(8f4z(q7}(8OMb81D!P
z<nS^PkkimN7&E;LG2ZE{Ki()5ObMZNz$yt)QSTGL4hdB7!qmn4k_0yJ@&)iIr#ba!
zs`VBZrJ9MU!8(hy#yBr`6*=@*A^07T>q`<j_UO<Vi{=LdwHgt44^?f?|9=zPk3=Jd
z;&Wo-fkqPemKQ2Ezb8NH@&XFolJ-|mO-l=&jYP(^AfFdfA-NkEKpOe=vibaykv6|T
z77t!H3H>=SvOHOm51tn)DZV0p8c-sE^SsOj7~OGzpB_MM@G;a=(*lD<C&tM!fTu@5
zv>sT8uC@3L#6#$s{O|}G+tO*N=D2;Ln%>)WGf{N2WsK%%Jl0ZM%lPabrCJl?lQ=y1
z-|<{xR)^B;M&|P$24o#&H?3g>;E=MH)l<%8a;y~-D1&17ZHx|yt|k@+e;x~?d%5)7
z;<O6}Ogt@4FB3Q*oEUW!CnK<jn3}-(${MNyol<;e1b@LmGhV*ARR#JNDUbWvd_dqm
zVRQC>+1L%`ZBE&$b$M#e;BW9u#>+a5YN`K_oJAL_rBILgzltfAhH9P&xt6#nScbAf
zha{k(7J!?7Ljp9^0<>pAZQ4ocL-S0P3r}+Sfm{kZSh;w$1@$3}u(Mf!MlFby)|Kf^
zb5;}8Q(Cv0URn4wtt-=;1ucLS4*{k(kfCfmVIJ0=!Qb%`PBLM9E*o1jyRpLLCbOFb
z?Qwk<v?X0l#mXd*xHee&cOk!SXjQBmv|S}273&A&G9W5Rw1|_6$&-rdPEr-IuwbRa
znOQK@pK2U|{F+qzl-lm2%Sp0ONhUusprlv|rObU)X7dQlsY!NRWyVudiPKax$De8?
zgv<S@HWS#0VY7;O3VJQ^!C+l1P7!C;tbBGr^oZ!6%4j#CXT=uY*g+piRLb>KupUX0
z3ERbXJqH0+Wr6(eNTT?^H}GB;pkfQ&l>myj;3wcxcLMLjB8^P*N!(9jF|`Y6=Y>+{
z8>q^{p}=5|=J2M!0Bxao6+HP1epC8=`WfOk0F@GWgBMD>@DWWD?l-FSRMFQ6xUUh0
zH;LN?JVRHw8SFvX8$dz}si!E<2emW6qP6sf)DWmzNC%%G^7o4=HAAvbd?{FO49^Lw
z7f4C*&lbL9OKG8$<cLT~j87{53Mq~T>#?Mm=t4?jc@~}q-jR%Y_$?2tu{<ePtYg*_
z<0SLIFQrGzMN&jH=g^hRqbFvlsH+OcBE`c6;KY;|fj<kmP0gSyqwB=U5@T&q$q!j^
z#a<7oVGEoZe!4hoB75H$;}qt2>Z#PGC8eKuR4RluUib;{k=^+geof$_l82-kq6Ui7
z!Fq>CQvBsl!Aj}EpF{^v+sPK`U}ys5{rnrG%Tu}KAxxCi+mSXY=FFqx`E%U05&{H~
z?)fiy!TaaOuthLT1HOM7I<)~jQ*5T*RYgQt%Rj}#l?db;BRwU_l49{JaLhOS)06q~
z6Rvtn4@)$~a(w^|?7OGf70whzIVr_56*l-b5lPNex{&g?#?KWBmawYAbLVgc7$enF
z<MUX(pST13fPmw<OBC1x#A{s`@VAj$3#lcIe;xH)gvKut1pO{*OSRqzJwr^N)YiS%
zAemNAg!T!Qk9rb9M$1Lr8C0WQ1p6~y3+sdq)@rpyn2kk46_an!p?uW}*OT=I6l3bU
z6eH>n2#SG1VR)Tn(d4se5inN*VZ6{&7Hy#%(J9q>s|Y4k>`q4}>mih*<N!RP+e3h6
zFM0_cP3Yu~=<bA$0Y@d!pBHAci=>UCr9htoQub)<*rNXeTqod&W@KLUEAaCJ1Xc@X
z6B|!>Xp2;_D8=F?fY-ygA%j+{M5#@aHp#CricZd_ATnqzP6tFqWE8amC~>PKd(isF
zD9U)!@ZoAbtwdUgC+EGA`Ao)`e2nxB_mDmt=z`bM@KtL<m^Ozma-_FRenHy#f5!VX
zl;hTT5C3SaR~q!{sc9w2vU@P)S&`Zo@V*o1Emi%Fm%+fG?gajX8U>f{=cbI7*lWb(
z1L7-r8wT<wHJwGMVFWMBfF&Mm8|E+Vf><h^{3WmVo4>_4FpJlbtC+vVBFcxv{5cx7
zs+JQ8%KS~i-*YrX?-oA;>|{37Fw-`F=^?_DC+2T#^22`n$CNJZ8e}76a+kVJx)Qyn
zRiw2zFh$+Hpt@cwY>2EEOi?qST<Ke_y4t885G2RI1XW>mw~`Rw(J8L_wNKi?=DrIO
zyl4mS^xY)rrEkils$`@zM4-wh?H)#Jj$M_^6x=M;HD{=$-?Tia@!d))CAT!`vol<h
zXIilx!8hQrIVu(}JyCEHo*|;a6h-f%e)v7=$+SCJ50dqysRNK`^a-T(67Zq~aJCj7
z1rEbIm8Re<tueKh<QI4Zo9YN!OCKlddn?<i(NZ3@K$t47<$F-K6jWO!5T{MmRZwYC
zwpQVG*+$BSw{R-d_z7B)$u@><z60Y_46{pF?w&eNVr(39B~jn0;=?lU661(8u=Q4w
z<Cblk`kHU*RF<+&`3+jREeJ7askx%gJTYj=dVR4ZSE#xew8f`@w=rmrsDDAS4iXHS
zBkCQ{3=WGw1c4z5*@{6^<6V>N+B&`4nA=B8M>ez}PeE4=e2+in2-ydEIQDL+L?PTI
zPQXx1zNMvlD-%!{q_?IRbH#TIxdK5wKt45W!sNz*L!>Qf0W_6>$xCmb8|4TU?Puu&
z!ipfh8ReK03yaW9;`0FSNa4Z4sv}gA6ay0^a1Sq-_$B2M5W>a4Ld;g^Y{6hWmuv#W
zhJ~)?Wk2u&U$TW_$XjE+9ifkcVPs#zPf|HTKLm>jBCFj&f8);15Z68Q7yOAOSjr_I
zNr=&O37926N{Mz0iyNHO3<kJx#Ss?73sqd2BmsP*rQLum3DCQj4h9CH2|HgUCuZ@m
zI2I%;$EAPbX6&SOmQstQBR!T2-1>3x40(EKzOcLw9?J&?LAy)j?JN9Nn4H81UW$m8
zPJqDgs<$pvtaxh9py#7Bqs%{bTT0(rI-PVfbz6!6moBVHgmEJyOs%APfgV&g8X1U-
zv6Sw!ROG5sZ6mN%rD7V<dX-AWlSHg!7-gF!(O1x*QcItZz<FL?0QS~owNWMFNuq<2
zh^e}j6TR5G^fhe%3!U|1=#`h&x=MN$PYhwu%R+zMcGQ+KnP2)bJ+$>U+fg$DTU&CF
z!W~!n@Fd-9q?_K_UpU6UrI-AL`<1}YVwl4+NksqCv^32q$J2F$b>Y4e!b@-1)1cO8
z6;Ga&{V<N$^d)xLmxVwoo{>mtSrpJfDj>6<)}M(L<0%DsgHAL34=NzDpk;|bf?owb
z6NB$nbhk}|2r~_`u{F~mD@=Bx0?S$fEnzR#s<k+&1<L#!!_>%<*nEtFQ~;s0Ns<c4
zlM2XEcX>}+D9bv*q=SPlBTcrf7to_7Rhml0lT>d@s#*S2gP<EI0ZB9p7*Ug`gG$7c
zL~<IUe2+g-F?19BiBQUA(`yoW3sowfq>_nA`9*&!6n5D{e=3x6+3K29SO#jvlSC}P
z8Z)B&iO|<&Tm6Yp%4JWeMEJ*T=Bq?JNhC{>GkQbk6_sCh0N_p_zaCnmLyDy02|atq
z#*DfCOwK}n#-9oHW7#KaMMN^GQi*ty=+gk95&qu-U;Fv1$!i!6NSke1E&5-Rb3r%p
zh-Qk0)(K>x)&a=AP7_uj9d#lIg##g^uM>xlWb0;|?J6stWHla3Gc(4{u7c*S5KAn&
zS<w#2sFB=Gm53+0DGOEJnW{DE!d!N%Rb<<>s_0LkuYZ=^W*_uglS@IEW@Dym6G_0_
zsbZKv0c~6{foxPCZgWKJ7*9<L37RO07`fGCt6~mxGx25Gum<?_LuxCp&BxY=Ir9Ti
zW99G}^BFf8GSv!C8%FY59-kCSd{aghMx7=<2mQz$BoCxg4)IMZurn3wB~aoI&`HwI
z)%d3J?WX#KQ+XYs-aT}oYs%tWqqIpux{KmIAhV~?LvUaeso0JvLr?Ou2YA*meyFJh
zPsINf2K0d{5e2PXfeETO;urtF34B^3ej1WQJV|s>GV-35R3gpGz}FI>XH{UNDt@j}
zsz{ZJC#f#^m1;R-OvQ~FrDCiNy9?g9I7sC>->AZV_6Zn<4dL8tLm*0YN*w*Ld_v8D
zCS3gD2t!iKsmStlf7Pz2{CJYz#&FG;HPx=#@+`<Q?W!%mi$EWL)iTtim8Ud0S05O&
z9>ybm;VZjF%kPEw9%XjtRO=Z@kaeYe4e{|DVRMNa4~Qckww#wm02^q^+s<IBO^OUs
zXhGXCr?X_qN){ZJKL~7*z#F`v)61VkWNI-XBTpsDBTZn?qoMX<g47SAn4u;kJgJ{I
zXs5_Yyi93@<u3xy`^$Woz-u)XoGDVt;E7b8mPAYZiB3WHwm;Fw1kTqany(V^B+)aH
zNbcxUIpNDK|H^NFmsb<|QGEOiCIz2It39}Myi4Wor{m>{Z$FzQhzWXErLB=xpP6Gw
zpRuOMs?RXPw#Egf-Evuq-(gjeq*ZD!Qb<R=8Gqy-GSU+%+?>Jw!d<60f7>-$f#<U#
zQ2arajE=zY62$2_Ng)T<q&h1a0QLMS80S_bNHNn{5tNmJIoqWqGn6x|%AY>m@|?YM
zWR9=*R>Vwi#B-kWo)F*iobA|Mmg78!g_W9_hq4mos>12}{GRj-DIrq|@ubt!WH9E5
zC(YV}xUk|Li6^}RzimZQD{-Ym-h@WAEE*U>bFM(qS1>}#2pe-qV@JvtHJ5Mgj^I@<
z7C*9Sea~X*KQE3x)wDjT?#0(!F%%execFqTH`*sXv1vk+PobN8hR(2e6FKiizgJM=
z6=jk#fs}i95Mr=fu@EI89aFTE=SlLD2qrBD*56tG{?wU~3B^W>Vje%>bX%laN@}!M
zq(n+_1cqaYM5h|f(e%W4d0DJE=B<_&Ozx0;OsSC`PY9ob!lZgGP}Ei5O2ODvFw!w*
zD>eX?5@0M@;eD9LHVO6S<yk-$7SnNlR~!Lem%tca-UGzjNS~r<D`CBY6HG?>90L7-
z9uhzdD<1&xPmT1=ypYk#)wJ542deeP;i8-J)VOfz1g$pjcVxH6y8@!;fjt^KMCm_K
z&<8bR`Fp~H9<=fpa99H9?@BE1$`3h<_a{SbJ2i`GX1X03%$Bp~qhDeA)Sfj9O>4Hy
z47`cOT6BS$H(SOI-Ia54hbrl>vqN{~O9ov*{)o4{YAi8*B=k2gp9AL!{1wOt(QsOI
zQvz^VT?Yszm%q}<<ulQQX6MX6x$Jj7OH=w^xx^;{i4w@>ML<pkHe=A&|3Z;gmjHLu
z-0d&Yq>CA<e^OUP%u`L;{)!-J$qo_zmo#~SzC)%(KR@19gOOK6YdZ9{(XxCd+A&IX
zx(-8ZbbmiOUs0FX=y86u90&929nelG%zQa*uefYf<U4J&d@m#DybwWmEVI$i+USw$
zP+7-qHu?=elS7JHbXRzP;zwVQ{@3x8pcTqXsjr|EBG>{XlPQ8t3N!<A@y!<uL3~Nl
zQh#9qR{sh7K=b{z*gA5d|FuqzT!=usT89c2^n|HI`cD;*S^J%ueoYp2h1SaH>ilZG
zQ-ml=k~i86T*|@oMXcQGS0B-|H6DNot$uYqFLa7E@=J(G>}u;*cM*CC*hFdn4uuDM
zwZwHaYt825&O#hW{qI<A$4djSN&4R{c)<@}+f4!;{V~F?BlY;Zm!r~Kp~Ly_LBx#&
zhQtsU4d;t8+VRBfnYEhX3od+Ec$7jj9VnH+d|nm+jJW#Wt9cPv>F4m9bhu7DwGfj%
zeK=s*Ee=)CG0zhYUTy8;gdU9(7D%hpHO+Bd9VJW80rjRVbP4)01}YCB>Vm!&Drn^1
z>4?Y{)?)HS?qZudF}6Lrm_(gfwKhF!8D|8=5c*<a1N~(`E3e`uQK+FrEt26+tfQ5S
z#GqT(8xSMfuNIqCAynBU8XZ&7T~$I^oT(`~(VbBZQNcy`5v}xC^_|Z5XpS$xrwcBc
zs<@Kh)laCE6gyv{6f-|o6~283o6~li(}=}Vr}l7C862?DBb|coaLz`bvC(<T^X(X?
z8Le|bS<7Uby*h>m;Ux7oYa3Xs<&mNty=l=aVYJIW;5iB4?XCR~I4vE8<=4(7niIl#
zX5dexS~8Z_{A(}q*s!&@y82(QY1%q$*gBW+n+J=(-X>HJs0)j$v5G|?H%}$4HEVr{
z`Y!&AWLIg|bqPS61hV`A;jtEty?PghQ~nRJd*Iaq>(G;absDsixTQ$&f;#lHV6m%7
z@4q;Jkr}AfQcpR2RqG?vVo7)D$KP7q{`%9;&FSanhEq?VJKQpdD{en7=98_BB2>RX
zuSi>S?xv7$eGnnlx{?g}cGW4jn}Oe8n=9UyvTrB1E1eyc2ROo}U+L|r+%LWE%3yN1
z7wE@%jw_6ENvtGR?3EbCgmqqEh6FI%>z)L*;alrhX7hFyI3rONynG3KDjYVkc+8H&
z(zz0|73>Y*ssxy}R#J}2IvBO69Lk7*AWk&{w<nSyi>Xx9AmZ`hD?Jb<%U~UqN2m+X
zUa4P~Jm$8dXiJBP`tWU$wTFVPbt$uSZE3N@%u+qJ%N$!&EN^Ozt@@Rh;M<7`PL&ev
zhc81?<seE_r%tt=<y0jaZ3fDhITd$NqB`-!PY0$-iSXo&I$r#T%8kG}NtUUSIf$HR
z8r!f@6%8aA4ngHk;BiSdkApUKhN*To>g<qq7Cz4rjdxXfOad?Sg4$O8Dy08%RK7$b
z8FfBzRPIwR7eprn^8?m$uS$NEA8w5UVsIJ4Ymv)l%~<v#yu#7=^@wYII%l-_?G&P(
z;wRp4ikA5>Q&DrJNaEl`;jFB`AE5cWkggs=6xS+AnA8VPo3AH1!p#~2svR8O)%A3Y
z+&&$_ODR?4XV&?>W#5YAE1Rh(%phu;9j?^&m1VA^uspeXS8!d$BKoSAxtdNGAcg>$
ziB)w3IWDU-YK#_E(r9F!sYXhumIxBnlh%&2TDw9{afwA6y-w10NxQ9Yj6R~2Aaj?L
z-8a92pFi7fr{YxJ1bY{=ojt}^$njcZuHxR#9<rCqsWyHoN?5l$=r>xW6iZ=P&c0>F
zVl;KBaodwJ<j7O&>Nfstf;GDvAFuB~WVwjmw0;4zMWZzg+vh_=S6PlO8m;lH)-%|y
zU!$TM5xq^aA-Y1WrJ8m$&ajHV{1Lm-Bh(f}TU>G$7LDb!iX}w`pmuS%h~G%Ld)yL<
z*B}LR2&2g<rexPN<`6vja~t?)#)^St)#MX-T$0Dn^0>~UX>EDblSjNfT5|lSX*YSh
zQyzomF<Ksz<ne$!mdK-09uLXmX?eWlmFH3A<UWQHzC@PA<eR>eiqjp|?^028%jDxk
zy@0}}+uLXDe&fr{3XohZU@fK;+U?Rl#IB#cZG(P@2-Dv}gyFn9Nu)gKGc3!lKTIyK
z%A{`(CpNOHh}V)oc01&J!OHsLi_;g@p99{PUc&mAV^1ETFMxkbfbOuKWv}&@)po>l
zzrbXdG8$YbjeBIQWSPscr#2M71HTe@?k2w+$Yf!I<4!Ht!KHwbLhQxGsB~HM*po&a
z1>F!Pfu_8~0gWYlmQoyhG6^*Yk|`>R-{^Zy9VA@&tZ(>A?m7yTW(MCl2{C*{0p787
zW4Jm2m$41qpoguL^W;oEg+&JW%XMrkuiK;}LZG<oEc-$di|FN~;;J~`dt}hmcGFjN
zv59l(ahx`^I9;)6$N1c<m`7@o89drXSh=o}bA2{#+eAxwS8ev^nk$Y{+d{hnZEgF#
z%70_`h+-*DTU;W!^Ly2YBnu4+_$;klOHXEk!!P2Y-PuIf42EMwyJhz0<14eq9c1>G
zT<t-M#^yi41~3YP#a5>D^gIrBT?}2pbpdR#bOoN8mSF}MM6I5}qZZ@ShAbe1oEmSG
zGo!w+)vo5=fG^4UilpJG=@(4rbsUhjoM12ln7wUa7~PP^c6Y$FYCY#9YV2ZAuiz}C
z5k$#I!xFiq)V7(<PZ=}TtcE7&f;B<}VdK>?^PsDiCtw6q<PB4SNf;Rp6?yfpiDG5s
zsX0x@U97XNNnpaiKg5qjb7W0@!xCVTS~ifwNL^79^;F6FvAnlts6Sbf@bV{HN3u0)
zM%ERJdayS2?~05jsiT+EeeKt{;>Sz^w^2jlDML)>1&$(f#WA&5MbZZWR<Mj9G-AU}
z;7QauWr$RAi(Sbl?GY{LMMtF?(1MOYw-0zhGCO6B8*4b&eGnO9?O{-0Ixk8-EG`<g
z@#Hof23`}^1+cFDk;S_9MVoc)pKaE)Z$S4Bzz;*}Ip`s06OFa=MK>!s87kV017+=;
z=FGPSv$fk*bQ8TE@#1F`%5X%{a<-|Wm8^9oHD?EqQ^_!G%+Vpam3-~5q^kMVfYF~R
z6{W{A-W^-(O5}1Cz43OL-K9+P$Pm2gX(_#Ysgb&c@|_375U1j4ZTJfKg26rYMMw^y
ztzs%u4}xV5xZ!s(Vw(n6>)pN=`;&6a?O2Vgx1k+YWE1+gk%rmm6z=#Jj!jbtHDaQ(
zv7v<K5K0AFz+>}-YQ4L1d{TZ1(AoPj*5|@w^9<s#JsUd+k6%c&89mxK8jvF*o7eJ!
z_p))11ZeL~<AJ^sc#4-lfd5E<vTvp|o9~psVbsA_pU^cHlQ#dK%9nO+9v?6jS86?N
z?AZJrB$#F2MzSRW47K$!IQUp+boe0modAwXfHBkeDL}t-_!{!^Gw_21nsIFfy}!yK
zfi}Frt*TB_mW6%&RJLqJQE3iJ1l&!}tvpWFzERK&0Qxb0IDFYqbbum>=<kJ1SE5D$
zL$O-j8C1K?5c}2r5c)DuUe3ykc$tpP=vwnu)JTNA`4njof7K){D6m@ZDZ|xQzV9kA
zBb(kK94V^Zyl;Ryezzflf^517HBD4CQvx6JQVFaTfeeS?rfNbD16w6<jhB~z=OruJ
zyo!ENby5NmynF<(DdgB3$IJJ?WeK#<v@Ph2w@OycH&e+ikpQ0yIyUzplM>X(L66(q
z=RH0Z-phvrJ&(&ot9ePl_q9c%d*ViITR_Lrk%=I>Ym{UKsxO|{3%TL|mj0#s4C91y
zIQpcasUc!9c_MbVV@W6mawC;D`rzMhX%DoOKp$S<w<X6zV3wlCW?8ASau_j<KFf&i
z5A>D5Bew@O5Eurq>2LITmKU^gix`JK2QUtF)NWW%874QdnCRQEfsaC4_!pGRD6s!b
zLkeY-{>2QjTc-4{=5ySR<@B$O??{`70IAs))K)EucFR{Z%LDX8v}t1zG<!D@UjZzY
zz;a$T0_&v_mm_sq9o-_}rTl&_2Njo10eX^fdCt#eFJvkpTvR}~Z1Qt?4eYC`8M?9P
zb*mXdsE%rnQ_3MqWnr+}<J1d>XydnHO3>0RXXuh!-o_<#Z0c04_u8n@LOeAs&h4nA
zzME(h$EG_V`U?1xw(hm__Doh;nK&k}xTL<?QPyy$X2>O_o6rQM65TH-^bw64ph2ti
zg7gJ!Zkn(8CRdo3GNt}VjC@XMt!^k^BsSc}KP-6p90MPPzD7NSUiQqUc&qhT+zlkM
z!-I)1HYbU6b|9T8t0?j>!1pAHrJa3TZK%hxf+1fSZ~76QMxe^a8+(s+ns0NkLQc0u
zw;)fnyVr5Qiro4JUlJ-;qi-)*4iQ(Q`5q6)D~!#b^EqDMpA@#1U>p17bG+slN~15m
zb!$DKHg>(A3+x*tILVaS=m+iAJAoe1^>gz=%WP%0$moX>-a)6ao|?09FSKJbyE#Tb
z%CHqfymhq{^NvhWZ7ILnkTA+%06MnVMvE?t$)(f^yU+fl)X^_Hkp<WyN7u|msgS<s
z5@}8<P5ig#zZAwdMv9O3tM5UWTekvLBFBqXPxA5cjB4aD71GFe4A30hR&4kNNJ*5y
z*!xuF!_(WFZDA=yakU<o23@f>g>h+egy#UNSGA28b%(m|%tU@V!}FS{X#p8l->qU+
zCL1X}IYH+LV!O-kug<{b6rQ?P9r3r+KYzSK8_Thvxt<;RTDtO-(;iF}n6b$oul(OL
z#p@X-d!Bc$_uMI$d{p%d8IwPnTOZhht&)aWFxE4$HYJl^&U&etharPOjQ2Wsc!o8D
zW)RB-RoSgMa`BAk9_%g`U$_ZP9q$>_PKD$$s^Eb<>V%U|gMl8F@ib%08(g2`X<D25
zYuUnL0X$p1>!JH=T&N|O4Hl-wEzA=ZCiNEPfQ3m%2klgES(w(hFrQkOHn%W8SeOjK
za96uzcGE(&ts&ZC-SFjUcMIdOXxiVxw6!oDZei}QFdc7UhFX|Tw=nX%2~@SS!gwY7
zEpsgt8?kmhwyd=<U2b7^SQri{`DtFXFmeeIwYLl0%Zf*}v_+G`zi@H&*J9eNL~a?`
zI;NS%9c1BOG~mOw7HIR+{6^f?$3UjE3;!Y>vi%_61df7xvP1Y6Etw#nAbbh<V&d4c
zncS`xu4zN}DZSLvPie-328?_9LVpX0^rnCI|LA%T@TiI};Ct>}NZHLMp@t5TP?nH{
zBvb{00#a3)fQkarLQ|xJ=+abBKtN>qQ$r6ODUnW)7FuWlLI>$Z41$V^`u)z_y8++#
z{qj7sxpU^s%$YN%OuKVs2>HDDi){?5hv+HQ|8{tWkb{(1<jgWHax!dkW;v+kZ;P4?
zo0?htpH<wVCc~y?RuZsU7BvT-rzW_AP0kIQ91Ot!gB)x}J1)7a5I)nLqvCGEy~@I!
z*`s51180_%XR7E(Xh+peuZ7P<riEj(#uG+aB+g`hBn+Fip0J{{Qq|{&&!#Ul>@qO#
zsi+SI^;O|>1UQ{3kCPyu7cIFye6GMQ5vT353-lhM!slVzX3K{$q6YTur@~YG2(>_{
zktau0=_HMzfoFujTFZk5sNj*stZSdGxu&Wz*|HR~{=v(yn#If%voYI8D3Y5^tdbOn
zArzxByGCw`eXen&PR_71w#-fTXJi&9l+5n!Pf4~;bF26u$>WFW+b#8nVBXq*F{8O8
zGTu^gZ%7BE<%HLv1Bw92qrY5|GOi<hs8d=Qb2b{iI_l1-6HBJi+($W3g(um}o!tzU
zG>Yo1!t42&)E&rsT<scO4|k#Yx)8|!qPp-crg|734Wg%KUyAAql0}W06U~Pg%0_hq
zf@>0-ITSRfQ&e{#La0EPoy_?<ss|9p*&1>MNOn{*kWv<8Zb_x)7K&;UufkupDVo~`
zP^+jm30`1KM<wIjK7{v?qOo(rUuD>zGD95G$8yPA${ZNc4>V;A3Ga>XpIZShm%`wG
z?oPsXQ0Dj5D7bs$^Y9m0J)6T0syUy5*4(26nHkRil!x;KIlg3P@a>%wkk2bv_Xv0V
zSWk%jrrmhTX38a<a;Jv(j-l#hAf+5sYF-c_z+d$^8EF`c!mCmB+>fOkGR=ca^H@F7
zRnNnX&U-=L$=9@wlc9mJKD=5n4n<hlRY6GbW7i{6*COFmjw-4L+Q!cCk<88Kbr2G6
zg3yBSx=jKo3>=KiaJdC<t{fuNBy<AhJ%6FFcwR4A2oVy<KzA7D!oA$%W6q29=U|30
zj{|TP(b9^~{X~-k0WxleSE=Mjo<TahWft;cB1o2$Uq_dZY#G0Vk1S2`oKiFAzYoGj
zf*g;U-<}7o=e*rsiM`I#DZ)v~p7Jn96=Pk2?Ei+33I!lT`uVVU9=3MgHM<3%&;qky
zcyuLdcpxZ3@2><_*fdQk<VmqeqRC{yikfA^qv25s8<{$THwB4L3HRqfY*R}3n@kS4
zY9@q77q_ufVo7HM(<=TYBG@45RkRgs7-YU2K1yR_7E{p1lqP~*k%xB(Z%IVz=uPOU
zRDMNQ--VnRNQ)?cHJ-<zdmYllWsa>bks)PsD9)v(nEgv~ao*I{;fc9vr+iaV0*b*j
z->Q^=;w9aJ>cNm&s6EGoNBL3b*YVv*35YZL`U=hG-#5O7^Vv5V3Y)bDT-XuY9g`Wh
z!*)3-qabmZw&!6xq;Bka$_j!w#U(YvcGxZ_g^o)(qFqi{dN>Zy%Sx3=CoSyA-Fg?K
zAM*?hlf~ABuh5u<xZIR~3366=;ZQGkm6jt&cPvCg=f6y-1Ic*?N{-3IIuP1Ogkj+P
zkp%Ya%!R2uOd@>AJZ_=*T=~DlF(uW+e#^0TF~bXFS<!L`!3&L^!RdaFV43;2dMZ%c
z`32j3y%q$N^v1QJkSpYS=16Sk5!co72n&A8Ez$M}wZ$Xwu7S?DFg!QepYgCo%EC(i
z6n*CnmrllAkUV~<tT#1;%aSldvPgn8X4eZ*a-?1*a&hU-u&Hi_i3<;fiWdl*T2`Ke
z#1)54Mdm)K>}$3#pr&`>u@G^VVKYz?WC3M-%PY1$O)AyGmL#6$*tD=($ZlcGk8xdG
z{nF!jny^|(5O*D0o+hjoo@U`8?P<bZlJXFR`O66@c$%=<ehB7u{wX|7SnUuQyb97^
zgh#lVu-f8jfGl`{P%Nq|5Q`eM;58sG<7vWb7qW3i0~wB|39BuHO0H#u`FNTzUbeO%
z$ADzuX~JrwP8x^07W^!pCajJgJd<xBdbhAVt_GNR8Uew>h4Gce(}Zzo-&}fCJk2K=
z2<+0EFrpu5T!|gl1y8fMA5yvWcQPaq>M*g~*i)lmSJdvX>e!LR>`*#3enxUrf}FD5
zIE;t3w0~h6In-$9$ZHp!C#-r=e~!-JboA%wO9Z0;8L1|?*kS&ThIPTXE&c^EQqhTp
zDC<J5p|%PgAznva3gbN*bFr*dX)?xuGRncb4w{ayiA}^n`E%qY$K9~_B7SV<Z42cr
zr$$~vWRXQesT>u-E*;pvg$>2)ELkih#DVY~VU0}!C|Fb@Z&+nWSRyZpY7(}A;+3<1
zy3`C}`)HRYfgIwe@`Y6{LXIK+94Em!?$2<U$VETRWI()(l3|t0`H}CEe%p`y8<EEr
z&7WzS#YcI<hI&g-J_j<*R1C!;8nW<TiOeMe#v-RgrtyHWTU1DM{YA1j)?TMt*f6}#
zQt@o5%}7UN7nSihps{XpgRlxTd1(!srbRKJMB6ki!X+&7*2rygW8LHyVcaNUE=~65
zXaExSQ1fU}3nH)QHo38Ga)+=A{w6Pahjd(rh1`QkH|=#&+iIGn$;n~E=u|UxIEnoU
zUJ>0?gn7{?1R0a0cIROn;d602O0meW^r~$L@H$J6<5{+R8CDfJQ)z;4;!47*TG}OP
zW5VKc(@IHNTyEMYTf-{lroAX>iIqx7IpVH9Nee5No7h$o%UNkFjn_K&QekB@qM7=c
zFD5l0##rxrCpDm2U}|;{GlP$OzSMw7Zwp*b$-<#;g=<u`$VW4#l+~}7sKNzV1<}7M
zPv|>wW#Q6aIxcEN8c_eM1$LZOAlU|4geWgs1A;=k4-^b-9e`O~){AQUPY!L(VhiMx
zQnyoRxfVKUtd-QlN{VNRu2<-50dllkC-^!n3n<Zj*>Py4fF>4}T9i&LTl(avf>Fv`
zURq{*^CpG9%51N|CaS0{Hg|bmE6=O`JeSCmO`ZZdIuBF~Ul>|nh*)0Qs_W7++*M_=
zReUG(C4aU*$@W`m7984#Vgj|flk4smyQ8{&&{c)j^}~t*E5afYFQ}^m)USVqQFYie
zTE4?(At`t^e*eQlE3R4SrI>PrvS3JOR0f+#lm6wLZD^7kF_L#d7FQv}+WiHJ6pNvu
zLjN1_5*L<O(A@sA;&y*6eu+}hE&f`vT=H?Kx0)EJBxy&eZeTTw5mNy#ITso$sc8nd
zCyRuZ5zB|~TQMNC?*J8A2E8Xr+Ni6cY|&b2axAENRA@2F55e_$D?+)phFr@@shb%p
zugNl9?Lx|`&~PvB8FC?IJOcq`t~B)x^YKP4bV}bazv42cF1A&?+)8VIx^l%Y{{>k$
z%;#b#S{IxhZ#bmRmG9DBF`2|5Ctnx4^t`%sN=dpH|DqA#RNRbDY+|2yBT#oQk<Tc?
ziHf}z#b-w1jY<MflCeXNlktBW^vTMveeo-q|MrQOB?aqF_q{#iIqN{$5q({J{4XA9
z2F<4EUlN2tJnx<t{|oE#`fK)Muf;zu%%w?B<WmaXs`1u+^EvA0%9x`N5$n}U^rtoA
zvp|E{YBB6u+<+Q`R>Wsv7`*ux0~ko?h0sp@G5&OfWFn3jES@Q_Tk+X`*fC)KPs3Z+
zhku#@O0iS%Ke`mx!IONOa1&TW$N1}3qt)U^z#i(W&*SB5m7>v${~<gbDVM?SUl4zv
zRi;(519exO;UlAcdx|?)y@HmA=&K@8@ho$itHWU2YHC<QDN6z+a)Ad7%#!>PfxIPD
zy`-c>_V7@NP*EbN`I2gcM22=hvq@r|RA&*iY3|NKJ#>M@7S<y4YEA~)@1UxmldU!A
z)%7%t7hd)9{ihi^LzXh@Gh|)e)Azq^0j0e^KURFslDMpxKl`L*>ya#vs~i>}k~qky
zaxg^aQUxP-Ay$Tm&7L43?Y@fNFH*4jS9%-n?=Q#m5=$cX*L@}tvM%M9hz$Ir>K@u>
z(CZ(lcv*<z)6^)|upi?$*!sDqN9c$B__l2YxZ4p<mzoj3net<X@qF~B_$|6$cy8G;
zp4kT#^1OTA^fdty$$jE~kkm9;!b6vKS{)>A<(1*_gLV0kyS5U=A*h4(m{M0ZxUy6h
z+NJCJDt-vye0xDbeuu~aayryP?2rob4E*R%J}<Q)#UB%E@bL}`?s=?cp7;@X`Lzb)
zti=KGpF}G4`cxG$phq_q|EWawRZ(1E|B;IKMk{m88KE)bYWzZci$(CFz441JlcW@~
z-{zlaV}wvUTq|?5G1hM?*0w`!V_SKnjn8sz#oAsPmJ70MMQ&7V#f06_h>(=H9VW$O
z5<?zsTM=#a*2TNt@7$kTKKp1kQsl%w(Z+{5qA=baw?-R9LLfMaVhVouqW<C%AZ)Gv
znpoFLzSOk=uX)c$8}*7xy9n*LJ9=XDBc=`aD=UUWqU#$xvGRe9=tos0BhucKP#6AE
z-6Js>WLz?yAWMaBherR34{FwusTn#G*F0LhRx?y}NjhO0G9;gX@V+WT^H1+Lz*I!s
zROb43BdNh~34fEoJ4HDDuGF2Q9<%_cBL0Xr^2vdN4eyb5pTJv2=7v^EEv3av9TIt!
zhbTfBiM+@Ij9gl+fMiC^H}oP#SxecQHa86BA(_yfgT&^B(L4<xd?XR(o=Zm(Mldj3
zH68+1%5c0$zUWJw7+p7!!P=@NB!5fzMj|tLSU^ZY10SrlU(qs0%65h6g>>})0sBp|
z3AJv%DCY<{X`vqU)9=NE@HEz%fv!8~`*dAEL~ypH`|EOibJqowa6ZVfbbozG-=1{=
zQNejc+u>Oq-$StTmJyxzxz`62f3-jTCdX9dm(Gg*o93*S;rdJ;^sObxzT(WUc-T(Z
zDiJ7LdVsJW>VL{3g!n#KkHSFHO3cZ6Ifo-}<49}x^5<|0zit_>_4))|keo=hdQ)0@
zA44^B5nRkKEq%7f>Tu@KUm^TygEBXkmcpz<7I0luzCr)!BWR|U)eXzBO2v?eBXp+i
zsG^UE^|2h4uhfQsQtsubWU*7x-2I4$@HN?h(?gmb_R*wV&}PqvttxmqdcH1LsulAQ
zJ)c{i`*sWw^AWumn_%0jXtDl!$EWV8=w<kc4N?OFEQ{RD9`U?!U-U}Bna(aw^eT;Q
z*{<m2+IGRo*SNKPb@WiJbvbG(9A_I39ZFk`zL0+?wnY}jm!2k^0M@lq^ibV0;=d5?
z0;A4hH7un`#n-16gn;PX(H~2V=vpdbo?7<@8uj8o(IYe&1{m#jc%wg|Ihbgb+J}&=
zhlbG;{MnH4)K95P^w09LUf%E0W4mwaiO}$i1CO4{NHHx56_W9t%%oCfAwmX}AUJHa
z=f-zHnF8}m`=1+->(cxOj8f-O%p8gr5t)ko$mI)Z^2Ih;YT@+29yvv?zKN0)P`;_m
zECn@48=|G}ZYRn24cpywy8)agq80RwEq7Fl=zcSeiE-{LL-j|9gbk#=B&9|kOS~?o
zH0{q2F=UDN{kA#nEWH}rDv={3_9!mx-hUug_xjUioYf+KBCmBR^ZV)|Ihl!k1qQ2Z
zmFqpJ3#NpLeG;=Y+mV#SOS+3B^V{<hwF~}!pmw{7`#rhxowz>%VlDYj-2ZGKmUy7B
zQjK?rV2y8@xLiwej=ERIKM>p8Ob^5omxJcj(y<Os%Vp$2D-z|?r!o*ry-qk+S_Wc?
zEAhgcn;`|blVwhU-AY{Tht&i2;(tX%gs<4C#5ApDB)2ED1s2gU(Q@v@88k2J9M#|3
zZJ<irYPD7^`v72{kr2FpLE<h3Vw>t)tVMC`f!O92v}7{PRuP$ra*}zIOinjXB;&^f
zmJrO%qj&&i*(8Y!;$ap+j+$-m$pdD1*&>O&%foj#hGlCIg?rP)1c{71zkg5LkfD1G
zEJM8Mpq8O}8rH$GXPa(m*jm3_71W7Ymhs&j;oG_?puG3qeakp+j+W1_4KPJ)?>!ZZ
zOwCBv|7-l!cn6&yxh9dTG@x}Rk^|G{l;gRXB?xn#jlUA-G=d?vAk3v#_4w-pl{hz2
zo_(D*2ZWb=T?|=bOaGv&Wy{1jbr-^6K3O?&Ghd^y24IPA>+S%KNXo23rHM$s=ZAi@
zD)BvStiY&oI<cd+A9U81cmQZdTdT7~4-Rfk{76u{OQiqg#7~fXJ!YV7lm1S8Rob>?
z0tHahtNDGEwy*+yGV#?aU?qtbj-H*6rE#*gwj^BeYvC5z&yH=Cz);-Q!Y#6&{S0Dh
zp+8*|smO+eT~gN8f?5j`kT1U$ChUHugx-Wb{w616>LyF(%3BjSa!Te_n)wN7o?I<V
zNPDJ*32D!?FkzEu;fJDypBV{qmP)=Vc^4J3rbFAKwH79D_mSqwvL6t|Efu_?g$e(&
zwQ$*8fLBnRsXr#nqPw;OU(Q%*dAPuCCCu@|u<y(3{Z|Vsf>P{M!ctocmm})SI{=I5
zn6S)hm|ETwSPwlNO28ip*S%^vCUALwOA8a$(y`kjE!MJSYir^5Dzs!Q_$p#^!UD8#
z+cY$4dtWko3EUQ7Ztu*)mxOT=d6S2435<Qr?M-+{A<UHs3|qc}unbkWJ5D)T|Nm*>
zUHeSVHa%Af(pq?zDbm~ws^6=%aGqZaD~cB0-DYXswg)mDD&u9xU1meut>Zf!J<%hO
z*7Zai@3OL@wecP!{Qt*nsPTsg!Ol(z9A%<};aVFLCZLVmqb-4$@Jt&MCNzREmNq6#
z%+*F;&Fum1ilY;1YDrM$j{c&M2{oT7WI}D-n=l)N?7KanRDMe#6ZBST67Y$Ad~>!3
zM0Bm2&<GMja{$#iozRSDJqJ(zHlev6Ug<)@Ykqj++6iy4XTGC<NLN0}nP83m?GFBe
z4&Kp>g3ujSiKK(I=5Fy7*%9FJjz%+7Tp?AHaYmmR9sRV=720QX)T1I%)UDTN=#`P3
zg(&G{-WkQ25XKJRIqC3>Qu!EB-QaormorL>kfrgwe)Eh-EpQ?IJ{+KCR76PgbrECC
zKqIi6{tnoTD%PPFuxD?XQ4Mv?FXvi5%AE0{CDP6Dj<wu*!s(Ko2{IpwJL=9;5{7fe
zL*_$qh24&)(qTKt=oywfPq5#9?Cb=3Y%6yjXRZO+IpDt_J;CxmLy;24zUX$`(<QFd
zGb3UiF$CeA`}a^ps!A((RZLBL&F#3s@F-iX-%}aQ?JB9i^$VD3mQ0D3ZzF7|WaYep
z$s?y)eu;37NzAU2WUeg3BUPxR+i@!|y0VK-SpJkC8vwham0D4eAg^H;Dv1v6YDlaZ
zp&{?pn7djjfpsL_gV04HeR%kkFjOMLc$h($hN@&&;Xqb5Wu|lLjuE@w!^&>1j0au@
zd+A?1bBkK*vOAC>2`SGK8i>YaFH(+sO-R`HEP<h#d~x(;YP_Le9#GjFVbs|#Kf`LQ
zvvZ<MCons({qUQtsbLth?@PhCP&xhge4#Lj&WVk7JGM&a?3!+O&ek%*YQ!uPN!SXr
zRCoHg-4kEFT>(WyceouHS{#8L-Q%mdE5H-c+U=lG%KYI4yNDD$o$w9ag?l8!eB@fm
z7fX-sA@YJ9sTNNJ<Ez9ieBajXSQt)3e`|PS2Bojw>vk++p&`m+S!jF4;q5A-@)fs8
zwnkJ=%x17LRzxM^9Y~GI1f1WJiWO_=r&PJWS4w(QmjvV-;RH2KDC>5#g|M=Z*=o%L
z)(^eh4$FxvwGtz+;wivmZ@3-22?7rQ)+f|r>Qitb)zu-j>ep_^d*Yi~k}{Og2VjlU
zZbwJAGzGwS$l}UXKnoXfJKon`9l#O7uE?&*NPlk|wyKm;s|rUB=mP8mKdce3q{smu
z0Q=An>jf+s`X{Bi9bT%b@*WESRB$}M+cC0)@BxI62?GF@&G5P%(}bAmKvD>Eh5kR3
zxp#UbUu1MU#t74C+^SuKossfIP`6_|#z)9sb(nCFLbI3VsH`e7b>SEB00pbNn|C@*
z1XHa#LpX)VWpPsSbU_ihpn6&<>UrgamX_qRuRENdkeU6$jAfweeWF5vZpS1bdq1P7
zj0}qTjc`RGTj@9yadk^k)j*Ul;}o%lgn1JAkq1H9$gDi$XZd*w_yAJ2`YhojTiFK+
z^G;tnGb8grSwO!M9!Z2Itbvf#|Db(YEtGkP6SaFpUqupxZ2h%HGw8nP*Qq_^*b2ge
z{~(C}JB@SwJ^-B4Bb4K?6q2cN((OV}--G2ZnL*mIo@RIZP#$o86!z$^pQzJoE$_Ow
zj4y3ZK(+dNtb?C>{ci-ds4UblvXIJ{q}44Xl|5E`2OE~XZI6qAouS~W*MJT8#qMQE
zp&*1|)TBq4=xz8~svL+LfYS%+0t}K3xPnKJm~zV89}l|^FtA@!k`OGB7Ca0l^n&*V
zj|k5YyEQ8aOC>Udhn<A25@AHM<^<uGaD0km3Jm8w&VjjrZV;~3C$a)Qq9>}JGWT_s
zOe;yrAv~aEr=skZog!IUwy%NRGUz`=|0Es|`uz*7B2d_4>6@&AR_E;-t@)nYdEMkx
z;CziSkjm)2ear}+>t3C-$?CLy={o88PV@I!7JV;$wvX|&zt6H*c99X${=&Z0eF3$+
zurhZtoZDZT)61ExpTz3J!aHt9SH=?i*{L>{W1rUiMfiiZpB$+-r)bN`nuu^Ga0W<S
z^tr&8hs%YE`}N1-M=Fs9nl33wlg9SybuJryhK-Iv?`8f69UrsdYYO5mQ~lbd6hZrS
z;dcZU^rjS!wUQL%P*O73Lm_3kq>QpsIEW$TWJE}cV8?=<Z3CI8^%6u8JGgdAe2=J<
zhMcS@b48y!BcN!x$9VJJm)wpQ;f-F1%2<Zpwsix+I~Xmo{<G@)nt|Gmk&X2gTW@n&
zYHi!k?J#jQ8GI%hQ6`tzakH#Ih=@@%<77X_Wm&Sk096v@iFtG~Y0P+sNNQ8xRtooG
zsf#=5o}Ym=_|xqu<5b3_EFvG_M<QLa-bEy$qub%p<p>3h2e{3%xfIAG916^{<(^Ss
zo{fJ-fqCIM3QYEjEegK*(QV#Bb}Z*g-RAA*>NfA_x@9m$<>n*Xn>3*rF1MMXYtK==
zZ<Bq3_`ZWXD_Uf*jX?QItz|D_Z6H7xVrIk&IRL5=Vk}o-9)uFFMiundG5$fMEb4h{
zm%x5>&ryB4ipl4fKeu_3Q)mZ2VkPr?^na~{L1mR=b776aa+QT=)C&P+a|M#e6TZ-L
z(<hJ8tFj06i^eQ&B4vGet5MHG4lWcqahivbGpl-$pgt^t{ok`126Nq*!@cV^w<0_G
z1K)@8z+vIwR+{#LEW(Zdl1LOZ%a}tru+<WYo-|?yAY5A;+B@2J>>zJ0ztz)1T3Jlk
zWb~qx1B2b>8bQQsuAN2rAHbu1-R4@|V*pkYmZ&BSm+7AwT8wk}d>C0~ONSh82JaC$
za|R<m&E=D1IAV9mj6}Ob9s~3~0NV6zP?I!e>O02l--Qk8%3Q&1&Mr)wj^Qo_b?Muc
z_`(uOH3~4Nl!1Z6o(PAqQ1WWrb1N{gQZt<imrg&NPtu>A+t=44so^!P)Dnl|baENE
zuTNtcwT1n|n=3Q3yYJsa0THiXaGTBintCKSToXnES=&WGOU-bbtzb(9`BeDPpWJ3^
z&0!+n=X0Cw`~uGy%zUf3jOKh@4+oU57GNo;*<E9Iki{#vIi6e$pUi$5$Q<kNA3&IK
znC+rulI7cZnAa8?SQJ!#!KOg!iP!b$*$3F%7AsRS>7Ti?-Ns*b{|w!RaS50H$qcC^
ztuX$IlrrEqa`jymI$}X5fF8Re-Us-0WS{ptgZa0RbP2#XK_9z?BI|c|8~+M8$!qrx
z@1g4639`mfW*MCd{!+qXx@;&L4ANyse*QnYMDu*uG|ls!M*_-K=&T!Q{3I>u7_K|t
z_zC}*+1cW(@ldze=gQ=3ay@zdL27QIRJFEl<AlGxGG<&W?LB(JYVQeM9K@s%*4pho
zQQ2<q8Gy$tS#_OwP3i*hgzyXE50{NBU8hp(7#6I{2asMhP8Y`yM0tReB9s8iu-G^g
zE`}U{BvcTf&uwJ;fz<&fSWGf523ysyYYni8(2gh=H(=I;2E5zR8S2}RA1N>x%;<hy
zhre6bUyR65)-pCU$5$Qg3rCF?#Z_6(aA*f*o{S<vPsFT4fCIE%4DF-L6URu=Z@dq+
zU}KeeY$>p9z7LObt2zst8)PtN)U-<!)Bm;8&4+h{S}gFf1y&FB;5>QXj<i8Urw3+F
zB@sapj2=}bLU;8{c0b!X6h8KQPAd$)1P%K@s}|=r251fiz|ljC+{R#GO`5A~Uy(pQ
z*iNP3Wo~jCBm9wpquj<fsGsVNx*YqAWMNc4i&3Hv{U^JPQQ|<-TzxwarJT_^VoA)}
zCQOV!cN+z5i5Z+uuQfMjGk`nsp(Qc}ZIM~`89e{Q7MX&!$gEofa4A4(Z^2sBk_6x^
z;W%9;P37yK+5oQm8{l66MP!QWCbOPF`_=~n6p1Ne3&(m)o_BqIfg&;`{ZKsAdXXb(
zPbuLGHLY(>XbjL-(ruK+XUb~8dZc#!y8y8yhF=5MGi+Ml|IHjFE#%KUo(6`ns;Dhf
z>o<V0PDaStpX7MekjAnV=>j2<qkAXtX=e6JFvz}sheRgvaD;FODIC(=8UY>uOVk=y
zivb<d1|+@0hNo0Zc;9pV0-&ix%D9a<wth~u!cne2NEsIhrzNtTQ<F4!Lvm16Vr!+C
zl@!ggn{}g)0Pu_JU$Ouqo7UzYGB2x5n+Le>=&lqAM=I!nJLo{hJE2;u?{J(qvvZ*C
zgnO1T?sX!dq8EM3D@(4gFLnD*pbWy4SqH05Mj_O6P!|4CG<w79gr=o6e^1*Tu>rns
z=nQp(D%0w4sB`cMa3y7S?d)D*OV^<Gl#nd;=u{wOLfi`G3>ohr?Hl?L`owA~S7<Xw
z{7b@EDnevd*%Gs1DX_)Bn3%7ofiiX7u#xZ`Fo;}Zi_3-#VEcd}1nc}no(6UT7<2KB
z{=D~rd2a*U+}QoS4WvA^flr02K@+48u_Jo_b&$3cgJjsu5I8Hq?i9g0CguWIG3gQd
z8`xmHuZG21y`*8x%QWm2fmxPka2dAKG_dV+nJe3fy#zMcDiDXQ@jexpW#k4ouwhdL
zHq62klPJV25tv;`2OG9UU^aotHY}4ldTu2HZ6LGM+)75;Fph0=!a7i-d+=ntpXG$M
zY%3q`<Q}iP8Pj;G35=w7$Cr@awZv_L?m(XDq=~wdc&2kEf$DRg;`4kL-EA0b4o^l#
z^!rlT;T3c#ZFpBuhEQx|s*U;SDa4_zEe;#&0;>%SacJl7y*Ge00!B^kZE@K65wN}j
z^SR{!q{#8c2?P-@sOVxXngZBFSSP!G0~@+W=BSTr##kfcyw~*5bRhUyU${s3>pBkl
zQOwG4R9NA0&IIedV0W^ccp8Zp3FjFE%E7*mi^y=AUv?|pA7KGbb8J{|{%*X@qUFY$
zOk-sC7Yv5e8t=K?U*b$CZD{W4J}YjY?%Vi?+8$E82;1=M*3RzfRiciv^di8=R$rdp
zFVZ_y&nPYN@(zD0T?$d>CCL)@k3&F5*iY;63W#+5$CxJj0_pw;gvAyvR^$w43o6(q
zZRA(l*omqBnR>KQKAW=fAHq`#7@jW2J*mWC4~T5Cxb*P|F29n0<SXL-6q9>~;~#o)
zDUBIgT98ycPW>N?OR^L&NLESxYfI8Xnd&0`XAu8hMqLqE%k2+eNwvpG*enThnZFt?
zx-7US48v=W9&BXxM?~|_+_af7oce7gI`fAVP*d|l>yV%XrASS1NNnEZR_^Au%^P?O
z5FVA{K0%(5$>x!k{G}noX$3g`AnODJkMU*{{-&P$b$FKDo3_h3LE4LinyeEH%A?l_
zSi2d-nn(6t-%Hs6MUADtH?uL{*>Wt4+4D<<b;%Hrx4XNlpMbRxP(}ziTNDC7t7;qa
zG`wfps}wo#u<|o@B(2c10I}cCF%}IE8t*Qv%Y?SGa*VgfU+%J@6z=U#<l_(AWh02l
zGj){p2q}Ve$XzxO`1BAtUFt3;wm9v-IyF+ZPEbNlgKK;Kic&lNSIs<wB{@5f5ln|G
z)y#_&g{>#4iKm*L0iWIdo~@a$Tyu5rKR{J8k6K8zf&Wp>T&anrjH@zGBVJAhYT_lk
zoT;|XPCO_3n&ZFpZ9mI}1sIMbwu^>U<G=$OS~8+Kmk4cfN2~er(jNUT@41FRvc9M`
z&!Sdv+F6jr7DQS@j#Y(p4ykWtwQ3(w9|r{3C8NIh0@XgqM|+ZZUcb582iO!j4&!CV
zrrIE~<zF0UH7%fwWR}TlW^{-VscFtp=UOP`*dsl9`MZj;%|p@)*7<|9K5(~dr2z_1
zH5N8*5S>n#1YSf(fbd4zn3hi11aS8}fDHlSM+4Gs0{zi<;T%gN#~f2K6rC888De;l
z)abs<AoYkn#D5r8eNTha{sbv?U5F9m6nY#}Mi~Is>;d2bu*)&!iW=0v+h8b)MUneX
zbxfIRME2`C5I{8%2B`VI72$QpW9znu7}Yfpd+~iTp(}-bFf7D)J(8D*+d8IvX@K?-
zG!4@$Bw&0_7>flyKY_xUEAzr@HbzpIEC-M}h43}nex7A5S-HDV-FN(aKoxJ6e`Qh5
zPU%-n73Ex{QR*;Qe^7OqA|Xa{1$MGWm{TfCqAZ0BByl2@xI;pW-qMe0RIq?BQ@(zB
zeYq}!6WqF_79mDYIrVyd9|%7ZWNrKURvx?<>hJF{0J?s{G3B9{ql+xN$n5ZfHaa7q
zN^BRj8TG{2Tx_o#{kmgpq-*$NsCr`*;;>09N1+7zB}|zYlPUkk9HllDB*?J!##A0^
z5#)WhLJ3rOV;QZ<<0UH`kqR%e_-DT;Wi0{d`p!N1Z+#^o^?HuFP+QSBei?Cw7;kWG
z&_%|sCZeV`wI{TZ2L8&!M}*!|U5Hdimd18%dvt9W{QtJ^LcjmiM~-uA?Hs?_t@c|d
zYYF2#!zsDC>{20}Jj##)r!e|VHfjk!&%;mOW76fdF_BCSWN4)`8BKlvTnvb4i96Fs
z9W~Mw?IzDNuF^=1wfq-iPpK#cYrk>6xt)h$m%dC-Dr1zi;2M>k@O8fwP($9Gx%w;7
zTe?^}OgK=*!nw=_fxg+H#dV7ZIx*EwjI<M5Y6cCma^ANSacDYc%QBMFO1y|Yn3aOI
zb=GxhJ`@$d++b5K$rtHN5&p|Febp`pROu5W8wQ{=kOW7DiSj}V*DIj2MTo=&<;IWe
zc(^RQeUK4i6wFJq%z&?mF;9OVNF}j8M9lg`5H-E<_L(!pD1`3Bl=mIK98gIXJB`L-
zwXS{-lzf*eT1IQrGz4V=Fmha~CmD$EAf)Lt79m$CS<J+y+XOk*K{eL|hTd&P|2DgU
z6|Wd_-481VtSqN*Lav7kR&P0g7~;KALaBxq%_(Ji;|@Y@7q%DH$h&B7$b&LM7x}6X
zVr~C);G`bhOh`3893=KC6bo=J4@EWrxt&XnKZh8>ZUC2<-<emp*~w8rIly}L)vEl+
z?#b1&c*)K6DVm!b%&XUU=u7CKHZ$fqSI>S$aWDlTBxgG*eT~-DLDD@U&bjMylFA@x
z)p`mUFR9jG<!p+zka|^yJZ8}E<gf@At8tbwj#!O~dILk&iC%4rH;-u=a#AeaB_W(+
z?W%Wykds*Kt2w@utN6mW)=G`~DP*$ka}MpH_f|^pU9S=={F`jAoNZo5LzctD{mn!4
zYm!OC&k?eK9c>=+KfCib{|)R{Jva`TX?N6?vb+&hnnfPg7|iRXtlpk4{7mwaQj<`%
zG7A(bAxr4eN_X)<43};+%29{2#pV@k7P1tR_v7zke!76&l+air?Rj{U&>E9hu$kDP
z56vlZ-0|=g%cA^vLs+Fv>~j|HNQiBLOfQ_(Z^YdwMW#i16Q^Uye0ud-JE;H%y`>XD
zzD88Aqg3!B)JX*^tqQI+lnV3!`?(4ZANE(E%eUtRhmj|8!Uz=<GTTiZ8D1)pQ=waW
z6MB}l<^^C#4TAUFynr}w{n*gld4UzT=LK63tu4$LwR|0B7%PVAnw%sMIe9~F>QP5*
z+jLDvw<h1BeP&;d*g3AdC5hQ1Vs)ge%q}D@Vhi9fZb1Va&I03<YRwe7O$cWy=uF;8
zqA9AP7N7PVMSRAUk-->C1T9(7DIL;R;~@mg<%mp3-$<N8-;PLSsTm=Ck=A@&cpf-2
zq;Cx=FJFkP6NC&X#Vv%hs5<Ea8q-K<(QO1j?eCDbx=5t^`ej{aTYY4RVi?i1<!$rp
zm#tFTGq|Z-K=h*W2y%WmBBT*Hz1qUvD5|qI(j!MJYok}`>!McHCJ<1mpk%GomaKa&
zg;>XwsPsl7ozps}WMRK3*q_Q=Idfo0ML%^nMrg!}TAi#aD!!sj?~M^wu1Z!espBqL
zi_{1ylAAS6XDw3J<`oSqQkSaJtenMib4K@t%ZmoYX6u~FyeVg#29Jo%M*pfF4P@~C
zH1-lkx9XUXZv9m3Et>1iS_r)9+p%IN6UOjd<8<sD|8waXv3LE?FE)+6=YRevGxk2Z
zoo{akS;UPsiU%k)_Ij+5kC{QkfqKP1)+k_Egx45dt%yz6B;}|(CFs~EA|SUb$O%8M
zicQC~i;3Uz6~T*p2m&1{<(%yn%+Z!DSeTv8*dJJ7yiM|!-w5}C&E68b$A)cvfm2o`
zV*I0NZ52~MiRc(>A5&CYnPP3Nsclc}QHwOSwITLrYkkJ{w-il}Z56va_Y~i4Ch~W*
zL5NrsE8i@*wGmytJ&FtrW4FF85!~|Du7vj`(vyb|34JBPxOVGs!cd9OD_cJ!jFCt~
z9wrmy{O#>pQa^*Zt^eZ_B#Ez6<4Tzx^UGf>%ckZnj^;WN-veb0VU=j$9R?8p_l{X(
zt@L*od&z4NxAazekUlwd2S+340B`BxX3##r+epD2yrAw}SLTc7poLMDJp|HgdQ@~5
z7tD(g$8Ygg>A*y0t;=FrV*phabKw4}NS-TLGPVlMuPPHWWN9^=PZ+~9vb36Ef__Gp
z*7)Jb(i#oduNK9wWkiAmU`o<P##*Q7a@4$f)E**|a_@ZXhhF5hquvOD3;PJ0C~N(>
z*sit!Y|8`epWcIds@NX5yM}zHUfz~D`ctev&zGdsHbi7w39J7<wEKS>J-MysNMsW!
zPL<W5nR{aw=BSiD*xpd)4R>l#2x8H-n<yVWybU+FZ6&K2j`;J?xJP=6Ulca>LtNCI
zrSz8EM7-@V;h;nuJX|E4Ark*P4^IfcN#veV+fkhDCaj*9O)t;&b$-B??9HGuvR<NV
z;Di-5$T=%jB6cWa@ViCr8nzcDYlL0Hc1C8~qwE^CBVya@`)g=Jqy;sk)Rr2=*awZ2
zk>Fqqe-F51n3I~DK`@7RbEHivyr%*qLa)S*p+R>C`m3Yy+q?Oz8%JcUzq(mOL`rpo
zmJ?Yb)v-L|ukKmdEE-Yt{n)Rh=nQ|+JINz&&IqrzU-I&E&R_OJB0u}f{+GyKERv+)
z2NLJ{Sv0?(u!vt_(F|C0ua;f#jsmPO@P?Bvcn3V(QOYiNM-q`bcDXxP=-$y3n&(&6
zopAFm$JzQ=?)_eS(TZn#s=_Ip=)4&?miAO7Z&T>G8xxWuG6=6H@AJ${=W{AuNajSW
zwL%Wn^H`EwaE;6<W$dyy=9X;+{VRP!;ULae%HdjYO$Z7XOy8t?0Yy4@k68=v7$2MW
zKhP={v-X);k4eK|Ru%KT_x+e%elxuwIc67dZPue(#QcE0*XBO+Vay)=Y)O91S@Ini
z?+d&iQ0KM#*C?DqCc!SPw-hp|j{Q7%M9kM%;HpQp-o|{5P0;W6%-#|+6-B&%&9=Zv
zt+fSS>6>ytpk&<V__G6}Vm{U6V?KVWL={0TV?K?PV)aXbF`Q^q)f^Tb7w@w%*Er@C
zji00DAZ*4aF$6hlV;MEXykgsd9lhzdj-{2F^eCo@uAca8!i?Ts-XEf3Uc;=slHoT5
zF>lyPzvDYlR*~({q8NP-9lmD=D!k(dZ5(3ce2MakfSe{|idERZAZAF8nmeI)mjRTU
zOu6;rxu(4Vvb>7+{7hl7tzw#>t70Q|{7QHzFUIDqC6_+`7P>j+9c2E&2CRP04g47_
z*cl*^vpmpYJM&BA2oI?8&WayMY!^?}33AOq=w@6{4(hP;WkM3SREV-tI2MG$>&RgY
zF+4k)6PnsGzw>P(ZEcy~`2mqWw#@IuIqe*Zh;hM`ZFov~h1z6lkzbT4rYFku^9Vo1
zc(9$~!8FC+68YLs@meCQ{RD3(BIm_4tp|zhD{Ir5tO9e3env00ap!qWFJiLuC&De8
z;B<JD4v(~9NoQ!0UO)&QsR{P{jFJk$sgSBdGh*I{y*brwV$)$_dPSSqbOw{@i8it6
zO^GzNiA`tjkS=qFQ0{xx6sNxr<UK#dsiNz;hJw#HoIVN^#_f&-epLD-!k2!E=M$Od
zr+77y<w7wtGfbh_Sq2RJQRtivmJP^hlWPO4>BAd3b&q9Ua`YK7>p<KTj$$QqW&`sw
z=e}iLD&&mw+ZUk|6}Vo7W^lc@o-s<*)5PZZ`s8qUAm<<WiK<k`Yu~>X+O=nY6;qvQ
zuu2Mje^7UxU$V#F=tXrzTc1QnO=}{Z31if^21C=sfyORqP^A_J8gl`NIeJ(Xg=zXW
z0J=zTig!Lu1pAS1uQ~+M(cc4&Z<R0a!+@|7LlM`YmVw6Sx=`7-j>!x(zQSb(<@Z&2
z7*ISkjnV`*5HcU8`QCcS!uQs0Y|4)bG~U%!(s>VW+jYKM=(#%I#fwp=oEVz;d7y#k
zQ%Md+$1y6<cp1a<ON?E6!~fLYl-gs4=}~K-(afz|rN}n5aUf$NvXzmEtk7gI0sLjA
z>qAaJ1sVY~_mxhRxcxm}nO_1Tj4ypPe+h_r>vO1Re>U*8<jjQSdp&`-QB1vJKJi@O
zZCPdM&Z<S=T}p+4phr~*ya@UuS7`5U!&KnK@Mq{3HM%awx8oNK<4=LR1hqDxfDVDX
zLP4F*GbQh0hNEktMmYe_S}!Bd{x1c7VfWOdk3bweBJhi6jw5hldE0S#ZU%m%or26<
z_7w<B)|sTLrB<%wa$@;Ih=AHNQGAK7^P_+g4Ias&J;m15#aih}@sh6JR0y;UFU5ZT
zN?>_^v5)hEi71qZ>Vz@AU5_}T#Z|drjqVUwG#9I}g;f+2n=mGHaOXZMu&77cto=q{
zaqTx0BDt@h%aql35y{67!uu(rk&;RG9T7}IzAjn-T;>2yH9vmf8rT)Afe6FPwMZCe
zg!tM%=8d#hT;``P8$02#%N(sSm?b>E1X^q>@|RM^$w-y2v*Ha<m+7VDEUC#1vEDtz
zZr4heY3-rpsIQY5$7=WW$KpytRas3Hw&5h<P$khrfE6|6ioR9e^k4Pi9H0!6Kioih
zbqQ5LiOF=CQAk9-E&v7-deOSxF#wSwmBf%RsHMxSs9Q%Wo^IDNxK(kh%Zy<q@Q=lK
z!^nYM2T)_IZWIXD2<HI`tg*1dtJDt^_d__vh2(RY3DhAc<bLQy=nimC1(%s9z%&5k
z33B+pXR^zbbDA2s86ck=R{cT&idYKfbwaj1LTEc!j}VSU%5IU%eBaKvn@-y8jf(6E
zjJ;avOtrfqp*{x5je+~E8>LDQRJ3|P?QTcv8?4kRcOuDhf@u`oZ)mIW?oUV_1~C4n
z%Y4-joB?opGx(8~qbA>$O;~>McrNobjzxRTKX6LB7ZT=UYyLRK11@g&N{Q^@;RixG
z4&#rzJRB#;S8f0Jod*F>l2FrFnNP0jjTD#J&0p~!q(90*ss79hN;gspuy*)o75Ryh
zp9q|h*WHcs{;UI|Vcnhx*8l%pl%u9RmbEbVCoVHT^TOY|Ld~DdM)qLB_r$6aEFCdp
zozh1J!~7x%F4JI*>G#th$`aF_mk71WT1!k|&}xnM+!7OU_9(qJICqW7irZ^UdtRj&
z8mO<t{_`yg8NqXVTDw=MX2h`vtFwnI?#(}Y@X&$qwnPT<AOPLCQ9Se~^z%1vgKpec
zE;Gp8fwE*XS+|b<+w-xiPizuZ9hQ1Y?kEL*P0~2R=i)M_Opx7bl0Ffa=rr^CXSwM*
zl??`23wyHF*YZg~H7}>a{;e!?=_xsqKo$|l`W<9?Wcg_566yF_<U+ZlT;_`~{SP5a
zgk#UQgeek1IQF1sdzkoX;n;JE$Z>kn9R-adA$xuz+>i(o!Y2#W-e5G08_TOyr~Z1n
z-9A)+_qYG(&)A_#D^M_Zd$}FOQ3Y=A#U}3cNHe~+M-6}I{fOtU0x(9(%waN%#&a`x
zn8F4bZ+kOP#R<@mPCUdJJLE;6b#J)LB1Ofo{h2phYkg6a{pDI{ThC<Q>^}k`jcL9e
ze*}~+HV+Wy#Q3v}wy1(@{8>YAeZT(^P|Af6+&-$+INev~&w#4ldDgj1ml@0&+{yCP
zDF$&Ol};|=G9KzW)9peZmX$(fuVGWV%lHAamoNsJ{`#2o!6Sg^c?OX;hY+b}8*<N0
zm$AE=xE2iY=ht1v!60gp_m&!rc6p5>ScpGmEBU)aF5@@`E@2E2X~TJK5@W0dB71uB
zgtURzT*jS3P<a#c{#OSe7#HJ6X-&U0M>1zIlz;rdZk&<D^aI<k<fLGp9xh|FpLWmF
zu{QCeF=JJa5)pdG)U@e^MB_6}q3`3r*eIFoGQI{!A03oxCi|Td=GmA6_1Y_8@{Ot5
z8`;!%wkB6sP^|q6T*ej`#?p|!Qc14veRy7uEqFSX_B?&VZqH{{dj!;3J3V85F_u1G
zuB8Mt5UiIyE<;{ls&`XKJDsy!#yUh*>@R&-vo~rz@{Y^sf^o?&<Eq;-AN#v|MT<dQ
zIF3$ds=b(*z2c{&2fGlddK5r+!g~OP8C?^E82~1;hq`whHO^b)GCttHM%_ZnIwI1w
z!*tgIK!^7JC>Ey^Rmv&AKI>gZKiwy|efYF5-1p#VK;eRP%D(ypEGMd+e<12}-dE&r
zjuHpC*pedmPW`H*%V>@d)%Q;QJ=<o{YkB?eO8>p)zaW3V+Zduldy`xSyM)U8C!c;3
zju<^2ewB=~>bi_d5is%sk>F&P5kt@THvi2P{qyV&wBB<VSy?4=Pu3_Gi|Z6LpVrdV
z*)%6#mq0Dg=d`SZNT0G}OqT+tpDt-}<r&$%te@?2EuT8Fge@3l^zyLljz~rtr!su6
z{=@JQp^MP6BDKG_(zQ)$|5q0A5YKH<qCXT}rM)X%7p-4Fb?x$N$Upy($dG6^y-@Ea
z&&cgQOFM|5I{T1>e=E=iq9|czG0{N_?}XraA4m#_C;VC@;+3}@<J7LMZ;-A5xCxbx
zbUBRWlmk)UP^5fPb6-zF$0*qnaLv+<BK{3wO4NXXsLU)G3a1fIA<Wi0P%d7|wqOjR
z_N|3aUh5TFv05WTSYumPf-0wRr-=Sld0cauM*fS-)L-o0cYtiTHT}W#eLoVp!m5*Y
zv;!82qvaPKxE8sULp8z!h@ai}6X71_O^$ehfS=W9+RySTgxt6X07q+sbN#FpxBcwC
zClvD=#mhRECKFdf9)2M+r2sWDHP>YBXE3p!lbxE({fPGd!a7q1=+|{7e#x{VFGmp7
zcarNGNM#9;5;@I7ETM94RXI8rzvNmSMpZZcRWXRzU&CJ&mSBG?KS$^|IKr=R<U2TG
z$PSutWPeA(yP6{$MgRkrXuRi!9>nE%l;nQy$`dPYb7X%nib)m>;RuZz7$s!NFJx{4
z%b4JA+aOQ}_}ey`$Vl;@1IBXT8NcW-Qe=)C)Rk{2r~w&z<bady^@(c`#$2|99L5t%
zB?grf>0`R)8Y;4s6p}pXedQX1LL~enqCB^OYdC_pYZ}ifXI&q=K|%--=&eIt-C#(*
z`#|I{ti$}hW3pFrcPr2Bv?GmP$amF-u5NKwmi<#8b0Ur(neU-hpZF5OB4AXL?38MV
zZy>A%Ms+=H`Q9)6d6jKzT^kHGCxL*P$*Fbm#QU+%`|qpbFTpuP9yI7xZCpL`gS*$?
zxlciQQa}>y9?pO(86LT@5Qh7vn1KboB03q>2~}<OX1q)!Ny&M*#RFZ|3nbj&Sdn7n
z-EK#oQL&n5%`nf2Bf(`eBjatW?aeTo!mf<Lg#M6B+h4QWo-qN~m%wQI>vr2SW_STk
z2T0@fyizhtSWV+~oiyIz1A)fBW;Z_LEC^?0fm_-RmbAT(>kZoO$h6y@aRZ!d(sn))
zr4=dTZy<l^wQCW!f{d(5^$vBl2Ep;W<bxFlpzJ_ERU)Tl#ZQh#7Q{7zIo2gy%u@*^
zkYkYt7-bzOgI_AR=s(9IEt-Jo6y|EKwxMKeCq?r8PEE=IwDUk^-3wkJ<+jMZ{Hn^X
z4$N3AN^ndEYFLy=XH0D6s-mSiM->tkksFI0c`sX?Q3YzGGY&K*ykd1m6*^lw;{eKY
z;3HT_XC&C2ao{Up<AHhUjYPXQ4)_3mYjuV!0_n`_fvuu5be(iY-bkx6670@6a0rA0
z{>}hPI%Be{DxHzHz1<lH&Va+$WJ&OI;AbEh*76uo>5W2Tg$42p3u*_G_cK2m{si-h
zze6nK4Bd6JT@5HLFDsRrUk9<82OSn7`^3_7g`Vi1OLaBOYxmm0aF9ajwZcq0<S?sm
zUf#fnR;yTFaGHgYXnTEuNF{s4AgHC&ED@m`0jn5l4FIjAc+bHM2NI?^%~JkI?IBLH
zv_H~+vM(qgFw!enA^Du9^%^3PEyJB=C6`@t=*LboT1N^|{2Y>FTT$2ZYY|Sfa@l8V
z5yjgu8=m7dj@WWAfq9{+l`{AoYY`;q7SKd89V}kSD~l08MR${cAVmio>a_@fh?lHI
z92`RU5R8~ir;&-Xk+q0}^9gbUq<0KJWXoQQa2i>f6Qt7Z2X|viDsFWeXM&U?_?&+&
z!f9k{J|J8N(Yb@a%3=h-SA}gX)8j!7Rb;>cV+{S4<#YfTheI-a+*846y;+3d9$H42
z53ufJfRrf$c_@o;5Ma+_ry<LR0z3@w4#|7}{Xzj2w}CHuNvRxTO%m+URC7IrcbcJ+
z6P_F%M__R62r6dr-^jA~-v9zxN(hqkjiXjNji23ihPh;z2{8Vq)5!4y8R#C~#H3xz
z=R{n6j^da4^Ek~Aq%=r&*TlXY-b>gmk<&cj{10dP=Z+JfinNCO3vsyUpfr$V?s)hj
zK~~gd?s)hP;g;kGzL%pW<q<`2yR00+%vl{y+I9Fh!Y@R;!SJ6+lsZxn<#W3r#95eF
zJRy!DsH`wd>L+fVU);RY*omJHmQ&2)=Z{EuQTus~ft%NO&$)TxxOp3zJMgjMGCGmF
zhL1ENG^Bi)(oW)}dTKVldy>p;yHT6WeYrj_-Jal&Y#@XEuWLRxkADSzio2h%QZKSQ
zg7=HJ|Ic)}jFdgGgFRjTy1jB-DyH(of9A`usuK@!MyyQnI!|>Po3;KhDsU!-cN5v%
zTvVQ6U5`UVP$K4DvXbRAwu+qi4?e#+TMslF$Q-Kn8{JUM_ngL;6ee?Bhw~A*gGT@n
z`+Jm#_st>1>e*JsgTQ?CY_jV#rUp^AHQ99<-xiS)kiqXDL9R)%=ekZ~fmS(NjIFuu
zNSWDUevCERbR)h_dHoslVPL;ygrMiT?OQsH*|^DoRo0}3*mJ}5qzABLUv(#UvDz(g
zJ+NEQ6ys-4c%4SGXA@qhfj5Ut-xFsb6JDp$PS_*w%3&pq_Rm>K<1N1}6ua`awsNUP
zxBg;}M!PiAy9(KfOn03|vcH(t?VZL)6yth8qU(^|Cfo4~NLFkQA8;B2iy~dlM0<AY
zG=|xlC_R(?Hdhn>;xy{WaIF&rP7qfaGP9yZW}O#=BgkZ1LJNS>XLXQunX(?i93J@y
zp!8N<dv<$dI>5>3e1p+Wqk-lCfSAqve21M-jVb*Tl7;^bY`=Kq2B{ZCmh+87chCD?
z3kfXk4Y{jltWF~it&|z7BV@nb$SVFBt0Uyxe?k7vvoU1G`f7it!Kp6gaL4HxD>1rE
z&sZ0I<ut+~u(gFLMF=jS3D)w{LD_HRk6bD9#fp`kcXT0ZkVN`P4hq?`Rp&jO3ujK3
zJkUMDGo1ff_nIw@wo#qkL8(rE`Wm?d%Oe#3>dut%tDbRQ)#aa}d`P)kN`!}U(!O@u
zCqbjRb^0ym8oxq@%JoiQa<l|Yo~eCJ4Uq?OHLB|TPRf=ml_+7Ie%9|NIJfC%sFDex
z`m?as)5#&-5w1*~BxCy;r=8e0@-kN9Jkm_(WFaneu~p^dD4Vze%bnA#qUE+G=S;0)
zl0V_GbG9}VJR|S(EX^W9zw?0`N}_&k8>UeH)>=+ZAp`xL^63=W9<NszY??I6&(H}&
zoTHy>hHP%+TARRfx)of=snk1-ot^zW3S;SCc+|O^5FFA!xwCTiunI{Y<b7W`KS1>o
z{*f``P{UbQXUkFj2Qr^iq6p!?>oKFVE(9fx<XBri0!w^MUYvE2BH>Ra^TN!;(z>2{
zw!q74#hLHw$+`0tT_5qGgn@c;?yN5`uVluVXO8=?h+x*0tD<s*Wm72ps%NW7=6rDH
z=;0&7!2Yu-Lb<9sU&d91`$xi=QYJg?+Gl3&A(HN&+4jF8u2z0kWoJX&rZ>E<ATq$t
zyhpf2IqPW4QO12oL;X|R0Y&s=9~H#sHNoYDud!X;QN|8Oi`p&^G@0xR+H)?CxXkq}
zXs*k%;(nKRv^>Q`kx$%K{|nY6HGH={NtKx^<5cl};a4=Qncw-16oWCK9toTV_Mb$d
z=VSu<{W@oLv1nz8){~CR=j$jlF=usZ^Upt<HFQ>w0|AyDMF@|^S$$a3?!%*vfHjnf
zrIf;OK3_MhKWf_jakMoEE$s<ne7?Yf5i)8y+L7?C-2+EIBGTK}Azxr|Zxf4x7(M;4
zXTZ)F1XdHU48+A20oXf)mH2kTR!fMu^31|J0Bk=*g<k;xQTa<+L_xfs?H*;!cl3nf
zkc3S5!tZOW+RiFrUK#Gk79h?y@~n=&0_r6JDAlv6$1JVIpCW%`h))!mw}<#d5O#YD
z(PNfFCG-C%tEnD7?2c1AgZZp++GAGGR1z5%;xQ}wBO71wm~k$zT|mNPk6Be`!aC+Z
z0dj0BihO<~?=h<|D7SYNlg77U?z78d?9=4ssPUa>ww#W0M3iDAPl6_twmc-<(YuNO
zE7GterMR+39V^5@-V3$HKKy}<<d4y}$C5ya$@CbqY$PN3V;>TF0_+_FP><vROeBoQ
zVFk7H7>9Jj0i+UA0EXoA7}{t`P1%GjfM3@17&4^M)&C6e9>8|qS{~z=4LzQRt20cX
zdn$O06Mi6$=(yV>(*%!!7od6mX@bW%tvLolN5Y%rObziE+5Vgh08ZEYi5}y;01@fq
zrwGSSz_k4fJZ1#%drxSA+z9vha};bzA#mO}WbY;(&jJ6oydLyrK@i%2GEQbA9^*$a
zlg=o0qA;N#kzH3j#x>hJoM2$Y3&)tOkfUVz%gXYzKTADQ>*{%g*ONO1IMEDf6HwxB
zdW^?X5bQhAp3ufF{da%qy#OYo=U>i%ANm##`N~mGj<vHrlB2%DR`EuZqfldAuNma8
z_fztX(Diyu{rZ9&w?8qS@CBuPRl;AIb-+YQ10=gr<L}FI1i#oLm|2XnKSh~kc5z}F
zVX8!4<spSIS8_#gKI$udc8Fi4MtID^A<9uW8^nm|Agv%QlP~Q=NcHs0i4E+VI3m`;
zvJ0qU#Ckxx3A^lR##bDvr^ulc7UVJJU~>!Cwr%c-Oz;ldHWv(RuEu-L<`T!|+R$8^
z`;~k_OL0$_o@SglOEL7QQhdNYA|3Q`Ja-Ucc9C+EARpk4klL6Qo%kIy{MBU33>O}S
zwPg|_=P9+*<*Rd~rt+F;3n=~8vc(#<!G<jpSP0kK3hT3Ew&ELBIIvQ)+twVV@RMBo
ztq%gp1>rJB`H~rXYBmpFfmr-qGEZp#{8Z-(0~5$*))Eotdr&yAsCS6$>UoUSS_;Bt
zRzd{ppwcii7${?{R*o2Jnrlx{j>%MK$uW=NqguTH8X@;}h`#$uYHMf}<iqXf4mhlL
zrAO|O@#Z&6JjP5BSZ|s{C-(6e^B5<n>DodXDOey;AKCDmY<Tqb8hR2`WH8kX9sxuK
z6k;*n973d?ZFW5<NAj~dkjMBih(fG6kjLnclPDr<Cf^<M7=zJF=0J3@Q2nX4Qg$Ql
zNl*@$V*@?LM6n}YYm`hOw$lg$SO+fT|Jh?i!UauUQCZXw@**qQ<OxMDJe%Qo49;U%
zM3lnhl@IofEfN^neX_@hC8s^r@fcP7-eLf~5wGnS#E9WdaEaly3AOh2MskOw7%ZbB
zIATa;b5<AC4*SVQv=!M?!RIO(SgO3NMidn;h$V}%S|cF2W#?s&$7msv;Pt&xG_ZuX
z=p8+u@E8TL_j*22bf2BGQ0{!9=-K~*{G75eWH4MlpNCrll%rTBeZ#BgH`o(?HBfi_
z6CrRq&kaU<o@(aUt*f2w6=fL}B_(V)<Jq7~SV@n<hz)d~J(uvL>0FfXZY$4b%`Djk
z^K2<WLu3~$Pk+x=%}}1dT;|#Kd|z8rj8O$g(uMHwr;wh?b6m3wWN-h;o*OzMBpmt1
zvrv<80ums0A(MzAgG1l%+Ff}TdZZD8_115mF9fT2dkgCe#{ESG>y0tbS6Y%_cJVk$
z|7MwIqNsq(H}LQ>0Zj7pj66<Z?4SR~d=jIVrZD&~j;WVP0;2oBXS-ycIToUYH|M$8
zciq3CrTd4T!J3x2(1MN)E?~2I!Vu3e&2~M#@eG$||0;wvFBp}Fd4X5GP~X$Zuj0is
ztQK`Dr&}arLr-^F#9W{qOB)3}Eiu8ZvnZ;Hj5JDg((?dM%ktJd;A9!4PL`-3^8imv
zdklH91~6+L;AyRUk$7{0TqIIA)9Y!YsRTgtPqxK->GJW#C0c+1Z(7^xc#=`$$qyB;
zt+yd{!(+^&QAO`E9G>i9hX&q~#27+<1lUostwmf?q)i-l;N-^^aY+cLj0R3l7vNOt
zp|fB1Q-^ChxeQp{Opl(WOeg+5VIwfiWg|bq$AD!*u&xu5N-7W__+>xA=Rr6tHor|3
zAs7sfSy<Qcn*;R7zD?zttRf#Lg$o~h8Pzp%J<CCqtY<j@@ege}NCCd^l4$HeXzw8G
zFZH(7Lhi%T)3B3wsPBf=!phNB3#G44KD8PcYd0{99?dFXH89R@Ocq?qDs44TkE&^O
z){BIi(m<(F8Ymwc@zTOryM<W|!Kg2jB3UeNvq?NOzv7{J-_cPj76&a`9a$|2Z%8|~
zS&QauwIx^#n4z?XEXBnr{xhm=#6wE=%6(=stbtcb?ieY(J=l2U?Z=Uoh+yL^bzLgR
zTrWs?5woI_NGsqn7BjNCQe_7eu3N3(+|gea&NJ)t$e!T&-!09W4``Nck8H(g!MLqZ
zLkg9p<G~|>jURDVvbK}8jrM18Q#tkJsLQ;{2IBf)gM*unm^T^P$cj#uARa*v3$2F-
zG5d%i2w7((a*_vnDT~uTj+iSvFpkLji!F?ppK{cdssjC0sZ)5KQ-Q;jBh~~WZ_l5q
zN~l7;u@%@by)r@}=7mcH8~gJqN2M!N99s*7CWMAmT(|@tI`&n^lw(Trw+1|tz@aNg
zY$v5oGetd}Pf+^G`{x234kVe-ULs%cu%56?BGY)dMYt*vcy-201!sbVxYZnqy2qLG
z>v|)s@MposR5qc@hSKTx?txT?@RE@88E2jD{Z0%5&om-(-@(IM1a{8M`?U7VFv0+d
zK-(D~;Twq{%V*XQma$8FKQTw$7|8tCy#ErQ!vr~_eZLV8*9aFSLQ}I95l$tU_d8NG
z;uU#CQfPNJWoJjQvvdCgifSNh<%L(o94i35_ZN~>jSw#p>d$UUcvT`_@jyx0??}W;
z|7W8O+5MzlY@eI=p)Xr-q>=Xz@*qXcm&j=zRufjTJ1wtr-B>C5!7qO?GZ2}`n)IB!
z%9V}SWPdNpf1|OI*R>px%|#zZY8|<O^7FcuBa)YSdzmeQrrAS%K~J>?i;<s8(rB$M
zqxensXU)>@aycSXB;6X;)7O$SM_%Qk(1X5#k!&>Od2S5ZQdC&9`u=SYd9;32)tDn}
z%DqrWIPnJ*?;-ra3Fx2lG9{D$`5aSzM4rr_(3k9^gd^zujf$L%FCjGR+jRv+Q*fOF
zdK|%XywC>P^f;j9P@(KPCnVx70)KYa-AK?c!;I9)o9yH%xlPi^_yWm%jLII6c|~rj
ziEK&A6+30`vy`D0D@L&Dqba#Sk_<>%UXUC&#*t$xWihbvGmo;QNz)|d8I*Dm2RVeD
zqi#(Wt5A4%urZz0q;d%`>k<Ag`wAgjq-Hk^demO2v*>X4KN9JRRB?}}I?MHF=A(~t
z)a_}yjN5^8!j&V+DMd^o8TC0Uhe{re=HX3(ys1$5wiGdwSZ4xv%$tw%Qv%M(`c-8D
zhe1eLtc3YE4wOEGUJ@zG1C*Z~OxJlIzeM-YwA=S-n%!G^zHvK9x{3M5?XmJ8C{Klx
z|3~YE@};!?uuvWm6gW#L|3`>LO3r>p7$uQSJP57Rp*$i;D2LFqYZ!-|UEwDkqfRCK
zLi!t!9>u$47U{dd-RUR&2$4gOo=Z7S_8H|A_UIzT)YFCWOJS{D6vn|7^U<H=%_hiE
z*hdd|@FM1CuSsbl%hE`c?pbL9NdAAc8Y#JDi)*=(xg0SPZE+>rEu2~IdD2#6p`!%y
zyFeRpz5#|QXD}6Pc;6CX`2u?ow-ZVp6COxpy;A3xHJuB?T*n=<NV}5yj5I0|GcVHT
zh-;Xmt~Sv8FBEL_t*jj7n@bsO$zO_4On6>M3VNS{<o^ejzB=NDGlV#oNQlGC$9>KN
zvUaXM&AU2KDucGCzBT0ni&RGlRx=_kFIT<Eg?ECIV%ewiF#-$v+nxfC(pOtv&5-=Q
zzsm&{^SW}>kEjAvybx@BPG^*#FZ5NSs!oLWB!cBQ*PGB&h<!nbtYNkKN8$f7Ga^Gj
z8f_fzNui5$qho`Oui;|Fg?v=)Fp7E(;XXG|%84xr<g?z6DR;3>#^EWzza@Mn;5ig8
zz-0io5;pUdh7&a@G*u)=epT(tiz&B46Use-+MUD7pW7o5VG_>q+zkpk(b=X`>`V!t
zQ6aE^cO!K-BIh=#APvOd2ssk@L8<fMgpji2WNGDEJ@F0vBBzx%ac_*l@{u}Ex128{
z5n6GcLCX1PsVQ2z4cWTpmr={6vIcG(-vBsOv!iuoZ@RD}hWy7blD8(Ix<roi(15@f
zbdBSCd3b~HnlP$0#UH<8NriFzSHMg>&%Z5o9k$soi6Y*(LVZtyjQp-$*EM#mEJx!i
zzKv4JsPBA<p8RyfbX;TkXxSW*l^>s24yz*fl~)Wb?7jAwVPY3DO%mCeqQ75uJ<oG0
z)3OECU|Mf%#e~Boi0mC~Ok<!?X+QPcI7AiW31fu(-WZe{XNWB!$cI*LT;pK}fx}?t
z4HW(SIl^g)U;@rRq!H(TB6c%CsSC_~F36qnH(5Wq0QWBxmk643p*kUfG04qIIqGH>
zR^n!YmKyVB5-H6HO(0QT5Wm?+h7}QGVvYQ;!t)do<qh!*=-`Dm5*ft9`-Dys>DiYj
z39?}>uZUkjWG@VsNJ}2Z5X5TAE8-V$02esT#w+4H%p>^hVZ}|oFvc%`@`kv>a~VaF
zVa0`=gzX|JH_>9=8(rfGEaL4X+=YDw;J7%uR(YgjZmw0_>jJon)e)8w%j@~{-_0z^
zBUl%Z0$gIH)`Io?c;e=5JEiZl6m(GX(8W3rlV<XG^&kamNkVt5r1{S>4c1=gR&7$`
zj`&;6{SPAS8q!A|v>(1&RXM7*^ykGAJ`^U@XKWgwhv#|~G@6$W9n5{yg~Nmlxt~Xl
zEnIFa`YpdIB-mJ8L^-1S!CpDGaN!c+LRlgl*aQRp)OgRWClZH#8=AYGXvL{cPAy!x
zOSmlrQKM+dWj0TFnSy^TjRYI)L^`5VsZw<10=jnL5j}O8GtJf=dDU1oC0<jSXD#mz
zA*0Lzz6U%`%h|WV#!l>L<$vk%*aKWH?Vy-{MHRl4MxO>!j8GUm8hcF~MM+j%V{^3d
zI%5AODUlH4Con|_yqrQcLBU20B~+H-hR}DhDWS2@mwQ<4#oUHn!Rg@2Fe#F+q2%LO
zsg)1Gzu1O+S74)Vm)EdnewR0JcgHwTSu*90reh)&)*FvCG;8gFT;<LF?(L}^9bCT>
zgaDKv=ZV}wr8W4wLMMu(RlaHE5y`)TwERazc-4cyG1AbHOt%XEhrI&6f>6b1SC-3l
zsWjaxJLYA(Z2IQPmH%YcB7F%x5C<8POd6kK5h+2+W&9o^gLH@2X~^AAdflsqP<XkZ
z=%!U_E)Q^%2>+!KdgBj`&l4GJERcnsSlLaA$s|U{>b)nKpe`yZ6I7Qq1zucOrodRV
zsttn;{mI2NL~bCF!@Se@&Om0rn<zvj0p0ytuyGvJ{*#$ZYG6M7-ea(_91)fsh;rQB
zb~@N-h2_=Dz_H(hCUv&LZ&y7^-gqMI@l@5?`HID|fiIVT96ZDbgaKyaUqn8B5^N06
zg%de^JlGiMvT6VLX|OS<Jmrjmmi-HYnaWe&x52?iIe&d|F1x;Rr0Dt@j1D#`BI5}~
zD9&>;*r*c48Gz!@{Si&8g=p~RxZR2*U9A+_`MY4lm0M^-;6W{ckd>DWAI4%BtLc{w
z52gqI1X?0@hdbgH;1fXOGH$R!x^(ZMf`6u^iJ$404TB%*s`N{Z!N2R80(H~>(k+j9
z{^b%zL(%NA^$ot7OGCC*_*IbQ#rh_~$g2z&;<ByMuj$g|I6$>_t-1X)Cip1(?^z4k
zz#K<sw3l^{D%F{d(i=x9r0P+l7qP4_e8=MgU+^wO^YAnmJIPC*|2rKluhh_?jw*7!
z4vqSuj8(5ncpCfzeMuU%Z0i^Ny+))K5UhjBRnA+rd=DS&THyO8eYnNwS(;o!gSS|m
zRu>0QXYaBq*0J#<WmFX_E5E8<@U|$-)W++i&m+vjPHe=EU))aE>Px8-7~w5JJ&?AH
znptg^*3h<PSQ={WNsL;T>laClWPLkG`QRB4n=mFvg{4qMWnoGD7ZnuidBHQJ5X}C?
zgM@u#{``9I3=de{HOWJ31<z!eIQ|P#uMsXHQC?Mr#z4Sqlq+Ej(5Hka`jwZ2F(|{5
z--0KK7RIl$csQ}F;L+*SH-!SI4`%Q727hidJN`U~Bz{i02}P_-pI6s~gIK3Su-8UR
zC_!TI#z(<kA$yE(M?CD1%a3};1bbhBYTY~Kn+02+ZRQi7{Q&A+{Di4A7@`LwU_$gI
z#xIvbW3@UAeuEA0tHepWR1Q2$#}Fh~c6Ki{A=JkLO-*6Epv^FMf69zvA*R^q;b^`P
zHmeVLhvjx~Uo2zq45}<6JWhB(QFM){xGaoa>Otsik<qudO~z-yye!xDz9J^rmy!@z
zv{9+xZgz+DUP)mkq_D)-t-`u#=_1p7!W>}qc6a*#)uk=KHb`N6q_F9U>}!n*X1Pj>
zbYgE91@GFU=Ww7VMBfN*WOFuAmc~hJR7Mhch6Rn{ga!HfvwYY!xHSXu*SKx0VsL`a
z0z*JmErhnJEun+V2|6=0Tp^r@m1D6jf)mhiF_D)^yqsU(VI7=M*V;?J+zD7KGE-Y(
zgvId7DZu2+9(|e^DS3xeS7SB;Zc$U+>HiBX<oi1@FvQzlIRe)8A&m{x$B%@#i3Nho
zK%7W=rw&l9iA(y*Vq6)qp!CA!ON0}^5QZ|)CBkrpmR>0U3}!}FwcfS5vKiP4oKyre
z<WA`_s1Eu$xI8oV8t2f^B3@Kk%+!@%iAA6SSL+a18goSa%)`foj}Z1?wtpN&a5h&{
z35z6xI9%OE*dmcCJfM45_en%bfV`{cB=Qyy4+%e)K$RwF8t{`)!ubY|(Z>r;H3rE4
zA6@4GURAOE?Y+;=$w^2E5CVjr1R|kG4Iqdl^xi`N>Agz?5tMRJQA(5!2M`4m6_KVQ
zQWX^uDFPO*dbxrE3Wyqdz52bs+2@2@{?C`^StomDm07c9P2DrIof^v^Qu?=vsquzo
z{o(_>8U)2p%CY*BAJj;%Xg@C9xx~SVUk(A4{+EIZyu<@_=a&Z3Rxfhn-y($T|H64u
zCw>bLQ-LQH_=JZI0IQyUC;ls*7%2R5NT?rqI1ijw;5W-UO?jR!tiV+sssJejbVU7h
zBj6F%bBpsqx_&1Vi%Xtn5Olh;0t|glKM9OgfD$_W3J_3$4*&G0z*YkO;@x<<4qRqh
z@y<AM%3f8nFo~CltoYU$&FJbK3inJtAi}B(KT;!SgPJK}l+c-?K%B<H#b@PM1E*u!
zEYNQWJsT7yb*8LFC;c0!O91tWh9Ks136HWh)}WR$mYECc+uX5C|FyxW3q*xYvQo<%
zLIy`7tZNhN9dAwIwgiSR{qJhvLmC6j5C?q4z)y9U^`(%cxop~<K`2o*I>S&B>OqW`
zYTy}l2S_7bOBA}zd=PS~{~OG+!sKBNDm<vVJNT-r_w5Y6&|y$VsV1p|+?i+HU3PJp
z&DhQzZ<=rBaWAd*SiV@7hK1SxpmWCHOCnS7tcR0vrhKS%1}{FN%Y^b#KxZBWIx5hb
z2VCvUEV<bxES&FgLT>`A6u`yK>;?8Hkj=xtf$tPp$^$NRh9v<fAKrL28VF;CVrm6C
zNbhs175VJB&^)w|<Wwon;&f*#5u-|Rwk}Xxl_KBG9BXi#(fn{edl9PF=g5^4g}<I{
z3ZyH5&z$WEbd$C}BST}LvowWgA5)+Pz2I4LbCv~dC#oS2ROqwJpPi^SJe&mntpFA{
zi;JGUtN^z8H9ruk0Ga<4k-sJ=K<0ma7^uq|gQ#r9EF#ny;J#}oY84OTfWZp9!^38J
z;a}GU-F|RKlb4%p6UOc9JsWiUUvEQrQ@$N_Fvl8#433*S^;Sy;mG-xAAk3hD%CUw{
z1Wno3llh}i|2OjeTLGDNFou@v*ZWcKWL#g4)K;i0*Q4v1b<T(?hJ3mkSKlLYGA6ne
z!8pCy^G?QgWlZ!@*V*dV3;a=!j+`Da>WaAT^I0=l&G#)@UPMd0yZ*aqneRI^q}~3l
zB3aD$OAt$~sJ}?q(_m;Weru`O!_cd_qO(*{OkGkgq{dLvRZa5OKd8jrs>BKes9`vV
z<|*}c8Kj2{QdfpGP-cW=(!G2Qj_=FW*;EzLk>;>oxoMipc`r?N*m-X6fnuf#E?-z=
zCE8)nZ9lpDtq;&kA(cTEFie53c?bZDDV2P9jOX?(LU{FW6sJeChu?MpTEDrYg4+-5
zRUm<fAA#=`Xu!j-z!@^D4u$lUjhQlPl>LP|l!w)|sqXwn&-2?~#HjB4M(^_5ZR!pO
z&0*ML)NT}IAHqBhb-B@<<(74h{_vbv0qVs$5<XW-+VVclbB7M)9QE&9Ed{8;=Wy+F
ztrhr$hi8B(dS`Wq4&@xx{TwCexr5&4HUWBpbcYV*+&<t_1+Mb&AAq-ooXg>XDt0dC
zDFUSxLuYcH(cAer1*-EvMLypxSX+kW810|r6gjn}!7db5CvpBUyz=}gwGSKU3WgUn
zR2)5@$7bqB!m!GQH=%kOn5Y05K0gC^o{}COZ`q^T8xQFv?~0gaRv?U=@>+q&H=mQ~
z82yp9Avud+)pjr=XC;AG^0CWqcuGjnG~+T6>AXuP4;>Kk3hYCbp$w=u0_!X!YNIA=
zyUZT+%Xv<F9o@zwuOM2RbI*Sc@G8aZ**uRIoj+;>_DA4W=;SbWhY*c5&-?#PYrwho
z9BYL7n2F&3B|KH(`~~1w<1M?~*5DEDCCJkn;77xox|O^kcSyJ&zaws@TSXcs=T;F9
z3vs%$t(T+?r|(o1YS;|C-!moHi-`amih!Qf5Db<8cd5Y@;3o^<x1r#yd&2WnGV*e#
z7kQCg;z{Bs@^YuQWnBmdl+!z)43;*dA?;of&z+H=@SY2al0D+CWC!Loq_e{%7(Jtq
z#?TzpwNK*uX&h2CZgBzMzQDxT6z2u{kPD6B=i8<K$1T);R0;O?BhI+(PFg$ZpSRf=
znd;^`ZB#I4fs`~~0WU9O(WgNg)7DKf1FeLo7`?>(<zraeEMW?@kBv72EvxhA2_x+^
z%Nl7Vbk{5_?HCk84<_^yMVeKEIr7U19fUY1Asyl+#9XJ;DWNgZ=~0SqOx-oI;cV`5
zA~Ro=gsPHRHoX->2aHLmC36hn`TeW}z69hxf88Z;H_cc4(PyevLJF5jEufB$Zda|s
zy-@e{)ylb{HQ+kGuxyO}$5mW5j8vA*ScX5K$p=VZHfLEg2~Oj=Yp-R^Dq5DkIxr^P
zm*7i3P27}bfhFm_vZd?Bj2SY-T2>aDSW9BYJjP>LJ)+y1<$<r$edW4;ysTUuqK3!@
zS=E;nC5c8I5i#`6vZBRAYt2B3#g~`G@XuO;EtfrzR-aE<SY^}F+<Rr=_y~^3&(h)x
zLbH`;4k5N{<SXZ|m{v-5(k`ZTb<z^RxG5cK8S$u6`WJ|?^CJ4>7Q<J4BH9U&<03k_
z2vS`U$LC=?SyslS2=^jf%V}K1>`;mbo);wpBX&lx>Dp=BiG;dWjVK%VtC8=)m<bWw
zcSh_46C0@1*jJ@wV@*RLDZtnRm8=V0fDWvSZ}}?lcw=6`G*+X(<u_sn0|OOc@O6QD
ze_@<<5*1iM%v@a2Tj<iXX)RgHV)p)@L@rizns}q1v=TMXNAg>kZOF{TH)Opun^fL*
z65X%f<X8{SBXX4{i%m;c)mE{@Sur9b4ga(JOszU#{5T?24vM4;y0DO%<jo?Hf0rkb
zfYe_?^j2UC>Tj-%*l)Fum0$i6`I4{GWGU)zZVt&Oz{k9wcQ)OE;-(f~`4t^^NIOmT
zh>lU-g+u6Z;eaaq!&Jy-a{}|4_=@-q(>yI|8_li6Y!;%MIw7Wi!SkKrp*?J-#wnaL
zEdTtYFC5tn2g&@%`H5q~NNz}uRsR(K^`$wd=?ypYV@eUD%=`+RqI^0`P8`E<Q04Qx
zm(ksYf2t1bpJgSEVT5BMLH=H*{BSGrao10*-_wbykB`?qo;X%<i2I$M?)PpqQx8Wg
zE2Q_HS`g+%sAcv|J={_Hg4pLG)TTX*jZNICT;g_&!A$n12eaqO@QkBuDgC|ABo3wY
zo0;X9&k~0gC;6Xnf)d{)4y{Z*KC3eHtxDvh0pbtjH1)Ny=L8xw^OdfBK9P=J+B(fX
zG@Aw!TbEE6_Q=l?TbJW`tJa4;y`7jYLi(v@rvh_ejVzs5CmeyzV*^{7`6@<kN~~gn
zO#{C+^VP7Q48%A0Rm^`rv2?W9M7S!@zPYb-{*=Uc3c1igMI8uCZ0<|(H$RWb9Dg@k
zUsA)sWUAg5+226XA_`5`WM_YaR_7Kl9&?X6iK1i8Y?l3;7CIRO#fK&DQP|-LYC~hA
zvfmX&7nkDQ(xRXUkzxCig%{PVxBb~~i)@rj_O>B2m^#_73Guv(c<mk{C3~3=TBafC
zWx0qB2eN5iofaRuhy}TbS<%_g331d#JeP}j!k0an1vRI|c^5M|7c=9}Y!z!ukBb=U
zBFOm2H?q}+wJho5J@1#<rYl(@eP^7?&JZHi<;b`v>g(*bLbP)cZF3QYle3%IEFQGf
zr*Ozgvs}!Ko!Pa;GQ&mGHi)tK+uiKCg-l`_vS|;UmTO&tln_DH9@&))LG1DDL=k-A
z5+vq|ZZk8xun<4Gh{C1)Di?h(=hKHXwQBFQyk-vg5g=Cue-^6CYniq>#8#zol{MJ}
zOnHZM)?rMh-AYw1O>{k;&gA3q&}q3`bT#tQJ>=4rA*oL@wP5eG>Q^w!KXAB3D8-}&
zuHLOoyQFZFf-*|dEu$f*;>$DbSfQ2#QL)A`h1xVI(~cC1w`3;eNEbyGYBbCITbKi8
zqjBcn2__{FD@$iym;GCvb}`q9BftE<)im>>Snj$A-4*U4N}kR<4We}s7jZgQ)`WL5
zzY?OVF>9u-twCo*m2aBKksGIV3m37E*3LzYSe^Ni5dMBH=A&Fp&z+g7vaNN05k}ma
zi&!%+Ga$qYH}wGdF8FiwAS?4_OoW0hEUUupOf^gthtqm{be3#8J{S4GjLgv?<Z)NV
z=yH_5%lhF<nSF!UHc>9NubaZy!9%Ug0cc~!H(hsT4lBUOFIL&x`$}dPRBBV+)s-E)
z!R+lnpQ$dr%_A<NU9LfTUd&7rVv>tU^BPQ`R2#OcHnliO$=Sn;68&jrCZDEq+AMMj
z^)V{fa*uY-tSrO^7g0GEv1vi31_EuqaS>&W)8}b&HklIn^tDWu-kdhKT&m*tsMw{d
zmg&>i<jY~HN%jHg+HUCMHd@}(f#Yp_W&F*OY+d}7-Xke^8vwhv+jfi^FxE=4b^DSx
zl}oXlwgaJg>UxrWUZbNl6Qv6mx|L6|b&|<mO`vQ4B>ST2%n9sWoMit&pwQ<8wl_+$
z|5W2W#2h%5psR(9Ait@Qmy&d*+-bYtxH|z;(Nq&kv`Dhw6DlW&dhcEa-TakkArTpc
zl5~>XX{T3SYqAL#5kYg&hC?B(h9=o;vp^97W9KE=4=jM5Z-nZz*-7?71=4gVV{EG=
zZue41Li+}y+WE@b{y=5^g|CE)!C}XCn6KTTTzk4UtIe1eo1yxrCzI?F;?|1H2g8zV
zcNi&cTVF`BeW2Rw?VhytVZ%_LWP4AGJ!Ml8@4KL>6{KN<3p9Chl)}a_JnMs2N!LV|
zX*?k5njtfb`+Jh^(9uHDdw$X>k+@!(bjpy><ga-q>7ucQA$mUPV-dNInDlWTqJxGg
zPtv`g7f06n{-jl6Iq4?7>V6Hg-ggtxaGJ6g%i^TDaCGpx9CQB<$6`@E*FWiTQR!xE
z%=362Rn&n7MU74B>lc-V`cU=#KeF&)QXG<`I^AKsOVuSLt4{f(t}5=WCc)}S%|*1_
zC2F3B$n*zW@d|76^rS?MdphPLS5-zQahNd<egZ=WCRH@Dc<$CUFtvlP=;#jT2}G_=
zs^$7Um8ioVVyYcUs$WDE#+-H_)WB39OYcW#pGnd(x*6hhOfk2+CKY$t8?*B8Wx9t3
z&*?ZFe~SDp-Qx}p2^@MLV0Yw_*m;4H9eow5JeKYmATC<Xj;}|J<GpaYXOPrOvnFuj
zhfvoL`b6Ndj=m}}x6(aa^y_pg09Btsfwdid5BgO@DvnC`gc)rKR9Q*@$24_a>3bz}
z2^s5@3A);mbWfoIZq9;17HyPKd3;*Ebo(ci@AM2zYqANreK_(4c2BpzSL$yEQQud<
z(fxTnePDOhh^n8Z+lNKCH%NF`gr+Y|Dx7Zb6Y69TwJ#SHe?8saEY$5FYI80sW_r54
z5LD-+AZnpOq5ar-{&agWu?%NP+1Dc7eo=&v1PS%oCO73|Cta5(IGwu(QBUQfd@<?v
z<3j1}zw-QeE-EHA-5x2_oFHmsE-L(Tx;-erRoUqrvd5r%xL?!l+G4vIWUFmZfhnDR
zW$af1D?0m1l<bmjKLAOW2MZ1yHkMbU5182p^iygCj&=4Wm1-F^aLgzWCEc1t_A(z%
zxATc-V4y%3U#Y~y>9-VmA=(h$ap|Z^)7Gb77UG>i+b+J+o&S@5N};3CCZS&pnNR7Y
z|KiR_VJV)IzE?!oqQ{PzV5RRfM6TQ~a^*hxT>4IkGD-w?A~&L9`a24x1x|JGl`qj9
zGDNOQpfF4m(y>kYY~g1F%4GPe+Uo=DGJIu{&ZR$*|0ODqr&9VT^Du5x`jGr+;Qu+j
z9pqiZ121Lx9*QfT-qJWfm?vkaR}iLY;D-!f#fnYR6QfKTPS^hCI8%B#tcC-$p!B{@
zFBw8k4OH&RRTP2rLMEm$E&aG;Gf?ch%M=TN|J$6GeJa_mL<Q)2CP-81{^qLeQ*yN)
z7n1D+=rW^&bP0wIpN#x0*_NZaE;+QvPsw%x5jF}E7RXCj$PmJ{=9y%>NUo)Oe4YG<
zcxDr`COi3$P&<e0zYK?2jYz(tXEkhVo(WXXWOK-<<dbTcnA>IUaam8Aaf;g|m|@PU
zQP3mjtdXPka#2k35w(|@KY|%MFZnwcLqoT8PVycj4{W!B2uy93yoa{bOcsY8Ox{D|
zV6N_lYfnYWkdkM%(rRV0W>?*6x+=VVpOq=nVil7$&FUtnA%CykZ{uPW)!MHanyis$
zw<#`FAdeMvzL9J~RPo2;DI(ehQ6#da7^*;rZobNKJCg?prJ4a|fQy;a%~!LmJ7P|5
zn@41OBT{GEwk8BlcJsv-9pv&g{9h9W@^@#@c#iZZYek`l?ztoXHC>IY-ydfQU?lG>
zdaTK@4xF=)7uzPe7MG)SzryN8kHes60op~~;}aeh1A1fD;{zU611k%QC4rb9^mCS#
zkpnj^=SyuN3@bD(ryq+6ezUVUn99L0Epo^9PEIA29&F%rQ`wBHjiFR_64)L}<qHCz
zE0x~=lM2HJHx+N8X`4TpN)Ab2Gh4_GrkDhVC)Z~wyT?3MM+a|SjIhnh2um@)>(aB>
z;UG6F180HLX8Gb^dF$T3@(y99_xx%|GAn<S9!1mRGA72Uielp6_gIsQY6U`*Q^syD
z_VAVWuL$?3(|@#NL0vOXKx5k>g*1)}_q21zzPc=?|MGBqza06|w<wG7UWCkwD79;I
zAl!BbM+9E_F+6y1pTJ`)!#&AG#UrwL;qKK3O9&i#AlzJN7H#^@M`shYAz$F(M}3tV
ztRS^wKZV;nO={ngnq=-YV{W6Cy~iD1LbrWIxV_2H-8FQ9)sM2|mnwyqhlkrMgz8x%
zC}M>wm+l$r@p`zunn(^JE1ULOSKgyI@}J>$Hfw1G)L)q0RXsl#?kbS2?55cxQn-x1
z6jU;#7NxEOm!a{uCbBw_L)zKH2rsWoxS@z_tss$m`z4d6WnE<0e=$;#SZ)r5LCv-<
z)&eS94X{n=aQkUS4Qgq6o(xRu=}YGOKdNQn*I~*#MDE7O;YRowG>*9l!$Ybwz22qH
z_-BXz8WP+C!Ci^VUo`x;P~0Tq#uIm>a5$$sTm{w>x7K{`BwSzeQtNf`An;{5X4HL>
zunKGb=9m4l;TH?wqkTk!k6!#v0p;K~;Os8$kG;gfAsAS|<d?tj$B+p{)XngpR4l!|
zceC`9X(jRutD|m;tp4(FU4rZMk`u4-XNT+P7NZSN`(Fz`Dpc=iSHe*jLvuA^Z}=BN
zB)f<&auGW=hHGZtyP=EtC|A~w*TO$EvfRcp++FojDw(UpHyg}Yc>LIY^F3*u!(W%Q
z*Ij9^8w5>J|Ks6r5EYYSwgZNneV1vHwY_>cM}^?n?Q$&071iZzc(xG7T|{=SV%si+
zKWA8SgdzXW<??L%DSR?KeIi|+$@dUnhmRK`$wiEJ5p*>hFNTj&tB@vaTd=Jk4IgFZ
zo;;)VDEiz&pG(tCFNCv$11b$t$aGyJBf^huH-^_umJASIUJ9>EiJ9;GJbE|WoIcmP
z2u}S!Y<!$&YPgv-D8AxL;jLjaYYkMs0U?^t{|Fywp0y<EeJVUrN)EA{zGb0CK%#V%
z06$#rK!gBX<wFuk{ejRksaFU+k8TofCL>Ga#dF(*v(p{1b)zR*;YHmP8?zg5aE7Bb
z*1qlggw_KzOmDR~!*f>AuRysbn?P~$pQqZ<guIZP;c+XDX4G>68J--s;Ux6#g$&P?
zMB3~(&>ndm?__xRl!w#z^Ps%{m|7fkc_zA0Xw<W|W`<`sbp6T%>2~AN=QKBe!B29#
zDM)Kbkn78DMBht0W-~mxWhP=+hUYM|l-M_<kV5-NMJgb9S^o^rT1nPqPdQrqzrC&~
z3iU_g)C|u$Qs9-D*ji_0covFninfD^Z=ungfD%{;B?&-Sb7h8^mGyfgNH@AzP;=3Z
zelSE=^8E~t)?oYn7^G9nXmTjJp++7*T2Gc)O1>}Q)#_w;{5|37?+J4C%*)j?SBpCH
zGdx<a?O!)YmtmZjfTU;CkQPDd;R%*iXTGF2tDoU%FD88xPFZZ9m#KY_$!~NLSF`%!
zT9%XHX(=vsBjReAm#gLdNiE9}(`TP$c(ml&|9FteU2+ZT*Z6*>&ptCun=?EOY`^6U
zaF<C4<xQ%=|KZv!uA0*_Jk>~QKzcB#YI#jo?S4booF=B!n;D)8VwxXhs*ua%$AA?K
zmsupr2wp}Sw5G}`^``u;+7VDJ7;3_c#Oipo#_<ET2PG!tN<?6S5h%I`cV*aG&>a}(
z(%FCKp|k(4#@IKv=#<!lyGU#0QyHFc=uMYCGAYC3Gfj-T^bAiSw@a4}@fo&Oeg~>G
zqNwbPd1YU`-xSs3rBL@rGi)^h1E;!7e*0`5tJ`Pqw|e(S#Z@da!#*Ld!@;CZ<YhWx
zm|Vr}pYkYfYvm}nX3NQnMM~WCG8y(k#0@GHN&6$!59XD3Ft0qN7&3(Y)&26Omof5U
zGZ<7MZ)8xtoq5IW%qwnpUU9q=CjC&$BmJhwN?uLm>3z^3tr$u@ecdHgo9%aVOSjsk
z411mEb_VIz<)u@5<F+EH)iZ4MtAkDl>0ZyJ!~g8pa|vr4$*}dlXmEUxa9J)Pbj$89
z(i%skZAyy_o4K(wxLuG*pK;IC_NDuoQd)>9xpsy<0j4^GKMQhA$jde1ey-%&Q3HpK
zvoh>w@xOBMftG`P>Hfj)LPdt1m6tc`e%`VLjhclsc$;ZCLmmt!I6N=Y@cWrK1fjg)
zcxoYsOb#;TZ6W=(+eAhU8k2=)eHqgy9mNgn9?!5%@0*6uP?O7Y9`X{>GHkV^L*5O_
z)Aduiw#v9)iG(yURd_tZZYQQ+gG}vQCVYqX*KT(|SB1yL6@M(l)&yi|$skwryh$~`
zpDF&Bm=e2X*oDQkD#%ngFH>Q|6fB6Mu}q98iO8Y8%BALI*d_AT4YehS-P|5L&~d1*
zNX<r|apfCE9?xig<9;0y>$2EGA>VP!I55<g!m8t*p}s2i(ZC-=*-)bi7E*HZNXao$
zG^Z{#E#o}856gCm&gUUIpGy?~V8$;Z+UyePrm=GiTz!7YB})D$;~NpVE6*9<goq;1
z`Wr(Pm@|yMwiR6=Uxe_WzcA=%;}TyQkMJ3faN7}(A%{r#TRM;4n4y|EJgD>QW$&r;
zdTrHTQfU2X#_LUJg_gKd-v~*a(q>V{o9s~<p6!aYE{&2vH$8O>4ABZO{CkyjdOn*m
zk#t5hcQu}9-jli6o>)`q1RfjiD;9nfW+t%N!+q6?G)0vo;&cmTwp%DihEwqd=o`ry
zBNHrpbYCrC^%|IgtKki#u*{VzZ;O3{@jB(%(y(fc!n-r0y(TR~%!`9U-)7X5QU_e6
zYMNG)=d_v`HA5rf*4~Vo^~~M?a<OVvhWb(eNcBkgL4}l{xr+ReQGsljoi%V&XekcT
zE8H$aWkh;#zjHi8FSABAb(xEsp&HLH7uU2@?-5`wUjH6*a1~Qae?wL+ucCI&8WQj?
zFK1^k3167Xc4u<JrpRLn2)L<P;wXO#dvSJvB1sJsFa8&}GF~2$YL_;J=ggm3&|DRl
zYA3qoWc|S)^pA2C8S-vTwI4u}f0gDWAx7M|A=OTcQYsL%0$PM|ZOpNz?q08vIV`V8
zoVIFIX9t{Z4o@EIXnB6VhBDqDS{pWwt+DuMJ&|lTF+3)6kLA3~KHX7u13!=S#Wgq=
zq|qH2k}I08Ez4QOmX1-65oxn!(VVbr%OU_@{;jSJA73*c4O|V>7{z|vEU3+{l`;Mb
z#uCrtY%;_jjH>_WJkl6qX09FZ_?1tk&I*}9xtwX{%BgchaqkefvWhW{UT_-Alpi&q
zzm>X1BwIoM2pmV$qCfnpT0@kF{c#=mlgRN2sh^5)iDmtXE`P=o_u|8;dLL-w>Js-L
zMm=(d2zI)Ro&dqvd8x;>(3}QxFreGuj@*@XLYC2WfJZk&k^;-}#L<k8Qtc=|9~P?G
z8HQQF46Q{4mhxryMbKGsS*f!MTFzrpOzHz`!Ly3?r&iGdTX<LvXca9kizTVR9zq*|
zbzH0&IEaMJ7QPlY(_16I7=8ow3BZ*NPT=1>90CrIhM(IR!Y<@kU+vMF0>75kGL=K}
z&SRgkY1K=k{pa_<aROmCc=#1K!zM>>As#LPe@(PJk?RaAYYaw!73*!04@dawgyr#Z
z?)@>p93eI62p!CQd|#choE0s(yzsFlAaw+@!zsDO5tjLUD*`<pWBUw?az`^vHUg=a
z*%>v)hxVH>!e3<VPaU+?+MFAZAp*g*xxd(l^H;PbUp&m#Ep6=_^FXi|{@3vja~fWb
zKjz_3DM{^8u2WYZ(=l&sy6gF4%&F@6k)hnIk=9>L6HB1_7&djSr6Hk0k28}%rd(>l
zgBPFm(|M?(Wm0>fKc3=@*{hZ@wMY3-%h>96YMCG|Yly*>;r7eXeG&QokEsbpq-oYu
z6HSdE9>+5??=+rk^-4{thnp7qoEKqP51QqV)T+iU!OmQsS_7-#>%z8iv1#|Pf&Y%7
z&s?6`(Iv8Iv(n~yJy}K})<<W;+dyqr*3n9+4aVq(trLNgriJo)o>VoV7X0-vP>&XA
z+CCyF5fj;*$fg+Vpb>+4J>$TT((A*$o^(?3b9I>g64fqhnv$COkegI`qA`*6*CYgV
zWhry&Ly(!1A+h7Vo^Di=$?URrrnY9AGHqjyzRb7rT8_PBzgg9N*Xv1zX54Q4m}4PU
z&Xa&Hq=@~3I|Mcrbmm{S@aj#a{KQWOrYV%d1Fb-gc8YIG<zY3jQh{_H-UBu((3XcU
z0Y38S*))_ixzGp6zts4Qc5Ym@l6)WZRUk)!w=C;YIKYN)&!(4oV3vNVhyruT|D{TV
z9#9B9E;R-?E8^KSkq7K_sjZae;FV|7$AqQ;lazyl&iqpxDfeu)=?B=(&09t*_A1G@
zf^;+HzeH8JG(#9VvcfEak6kECNn#fP3&L{jzzI6bEiA8(H5;KR8AZ~>o4BWfrIjzo
zUJCmM>K0UQH*+M(y%>z_zdZsh8_Y=}uEr+n>{wk5@%98#SP8HJ-CkOSh+`5qo}#Du
z;R*no7R_URv42F3mAnZX_YjP_0^rhmU|l%^tqg^9+5+djD*zM@&y5XT0T863P0$qp
zmp%kOfQ{;E2lgvZQUS%3$t71UZt^72K5D9*d>8tetWUS})m~@ZSdn6Cv91kf9W#?}
zXWSmid^*RTgR!R$gm{%CaC%eJOTI?}8>Gc8E16y8b~+rm=i$g4dtR2YRif9EjC(#Y
z&f#i{!zz>gz#fXaFyEQ+=-u+Ai@>j35AdB2wu`jZ`ek07-_t;>?E3e7$CRH#K-Ogk
zrtix0z+%|t7zL{HK&`r5LIJFLxgt=Z1VZj=->q^m?-VJqQOJ3sf1WJNPk^D7vhW0{
zN$*2#a_spOBFZ)NdUPu5$<ZW!s5cVp05zp{JACmF_XM0$wL0`9(H#Ie<Dutx7zya0
z!J&CPOarDW@Cx<+@=JtfDYTA<wZJNg`p{gI;5oFH2s)(82Qbv3V<^IbEzhAJEb9t`
zo-1DL^ZgWJ_sc%~T2{gQo-D4)8b1;hzh4dw<E$$Q3e4ewq41Rlae(=Uj1`>m{zsKn
zJd@I!&y`)@f37h4TTMvk7Ln(8#{b7Jz%Ykyi|nF!4H<qfVQp3FkeGe3rsQP^4YwK?
ze3siOjXx@;dmpeDwhQd@koOhr=Rw=`<u~XfxhpKsH{~(6BR;d=7H)%CZ+X5x3Wkl@
zMq5$uwVd}KCa!Mar}4}_=HGP1KB->K<}M2(7X24gw>(GR5~ESoj0;V(%-IjmZX9Dl
z16h+_%Rc1wRHeWt>O{-#*%*(}@0EvC;2h)h>|RWWCjLrO1!yd<FtWMQQGpFq>?;F+
zew60!?Lo)b{W&pY?#kn$ImUxzzM#N=cz6v^>F?$qoM&x4Tyy3>uZNFpwb**d@64ZW
zVv4&S@=-EOn}PLs$VWAD>_ri5&))iwKgV9kFxUJt-K{x>hGD|vbQxPaL3|Kkz~gLf
zW?5G$;462j8VfTl=fhWYmgS>1<Zu-EjzzVPXz#Da0tFT5&O>RSqymF@p!vS~K(HxX
zNU6g)spd46j+_ZUU^TUcS25+)`j~#<nNV9O52Mf)u6O-#!XGLRNENrujhm<=oirhG
z&4m=M>vQ8=#swL!&rPh6Tad8J<<cc%!7!|$wrpZE7jFcUBRSeco$mBLVMUmo<(_dm
zVrMU$7)0uTX9PD@TrD4YH(6dY?m>f}9>l*E#9z$CS2Y?g6&_sJ6IM(3{4i46)PQVX
zu2L}#$PBNiI{q|?x3-?`RQRj1ZFBk4%h+@K2ZV+JLlpRg2h4Pp6T6=6hj~~4%)<w^
z|KJw=_OnFLVq9G%3I?ZFnUG%nLV^F%v0nX$&=o>E7z|xY2ROFr*+F90@Y`z(Dd!zE
zNclbVNf)LjvFm|Xg=q)|qd)z+Floe|27acTceKy37e|mYtMIW}Q#tPSG-tOwJMi~w
ze*za3ppY70yG7_mDxsl@>S<Zm83bJSVC%)vt}UNB_rC>UieDd-Q7Jtg5!m#UuUKUH
zU@#?cgkY^y^+#YEsA!&XIT)&%xJ<+a+lgn5+bqVTIgnBOX<qIxy6bwx(=Uo@ii;m4
zyuuOlBo7Bv4{dt&erg1k_9G^`7B54br#aAJy!kr4&h?@=<<dbxZ?<b2Jr?ak8fTH8
z{!<chT(Y|8kQ+D2?@{DrSK+K+WKqRY#<I!eU|gkOoDx_>oYb=$g|w4M8e%oKlQ;+v
z>S&+3=lY8phzayK@dUMK$*WWo$sH1*6J~C@i=>OlSBsoD(p~goFkMeX<50qUBh|BI
z6MD{5qr$IcKjih))FI0)A7c=D-|I<o<e%}-EeBi=-Ey2*-0FH&FZFTDFFeuUTyKI;
zZNXWt(-d9rpulY&`T{)(d>}U&53u#nSw5D;cA2{KXJtmR&gAJdTgwBWX96!Mkj(=g
zeO*(<<@n)sTH@;~$;aY7L<H}*VW8tK0eB)Fat*%9HRI&h@0n5O#nghpyJQQRlnq&H
znA<^6o9&-FGx%9#G`j8ew4gRmzQAo}KhqXoXPkC@-KZRUZaNW<)1TO9-X_!*SMuv0
zOW0mdD_Eu$g5}J1$WH>g03<HTOAOup_1}T>3LNEuN_7320w*l%MghQ(o#P=9D9gQT
z7ps#311&_v$1zoPrtnhSzSs(MRe<Ak_QgjSXxvCA)UX^D!K5%a=9d9OTq6#c8k63j
z*}uVEbH=1Mx)aC@n)C*_yD<Q>%%vH0O?vMbyC(wC^5MCoT$4_@mTOXH{2q58;FnS7
z?zQZ*rXrZ$++C8P4Kf4De%vrN2h*6Ts|J(}8KWWb57h`W!nEk?`i%5?63DE1sXTRH
zK{GQ;D5vRCA+{e`ulMqL>J>C|tky@p9(HL#%07j_gHwI^{4ez|^aZ$SZ**Uh$g-@5
zcYE!-tOexY7WvE?)q=NZw112<Q69^RujjS7EYq2K=vnp`{Fq2gd#d1%517XN<08<l
zupD9^_1gbH=l>J-k82XWUZoPBl|QVwqE8g2QI5~T!#_^Pnqz>`r8r6Tl%ZrX0eTvn
z{LbIBfVvXM*~IAJ#?!z=jmAFi2g&!4gn`bGn8Bc41k91+oNq1X5F<bTcp2XvEse7p
z;d-&|9aVG?KIivVP^=3y@cYWed~y~9lhaFR@q4SlXg_BlO`IA5D~I~Amc0;64`?jq
z`Ns=FTis-Jve&~v*O}41H_^pSbTnC=9Ov2L#hK$L<FdyY#ou@xSXq}qDzU3AXIe91
z>Jf|c-}u0Y9g$;QUSv$Q*&pZGNLxFjK3;o#A{55}-Dn)YS$xkE`W0Y4XdkCLym1G(
zrNBzo7wMF4hADs@Z<YcW{M*M#=jMYz1qJr;K+U+>gzTJbm1A8QX43sG&a;AA`Rqt8
z53^4WgJ2-gR{{F1n<#QqljnG@POwkXG~Hy-esdOUCs!C!Y<S;tp5aCR2wnHeS=1{t
zE%)w;E6km=81oE$Rp|V^d(IX@G4HaR_2t2verq!IWVqk$AuR4Dby1p%>H7=d%x4(H
zb|#3M^M7$y6M{)@RFc!g=1#(ba<5W+Mx`%vF(I*6=|z|KmAbkl%;n0(Wmxuku2Fb4
zOk!Q%GHs>Xzi80n;>+EK=bR@RU!5LAw+vi(##hR_RKY;hv%b>)<80Tt>Q<j?aUOQ}
zJJWBg?5|_sRlut<yBffMQNwP&2dq<o3V!oz;3(zz(-}l;Q#lqd66YC0IZj`Ph@T|l
zXW-w6C|*P*ClUVzZYY3=TgB)tZWX4PKKURrzNN0{mr59CKaVQUm6HcN3=Ix+t1@MK
z^#l<MXxN<%eRHg9rWpA(9P_Ds&yAtlZlIUHMQd`ajsi3>Zh$&<Eh2P!*_5_tGU-zV
zTww)G7HY@Y3m8;Ar+s}!P;>a3h6I^_1fjDz9iP`Xa@AsIA0#QRS__djT`P%3S_aZO
zhoog#(S#(fy2J(EeU6Xn;*`1udU{C}rR%6y@y1v86IX{gQk*UanPf75HHW+OxH)XP
z@nP5DbkoGJU}d;^R|dleu8neiU^>H+VL`XK2EE*B`U~oIjjLSq&$H|c0pgY^&RjHc
zZH>s>ps8V3i7-O1(!&0W8wRI;A!Wz8#j!=(6Nxj|yi9fDj#{2n#r-9^X-3GkQ<kUJ
zm%JCFe+;KqMXU6$M5{z1;ym-X+wl4R6#vC?SZn~$Pk~}QJO+#^=Ux+V1uue&Un?xG
zO*Dqp4FR{X&8-QvkNSMqtwjKB73uu`Ewu55gI^^xAEM{gU;lm!DjKm{ey$m?fB%Cg
zrU$pyDu5H;+6rt@qWcj++vfg~Xh(J4@0|wy3GlH3$9Xse9JrS``YSjR+(Z}Rs4p~7
zc9gdt04kT8x3hFnw@xdaQqVJ|bASIQbC+TGGnl#CUdy7s<{GhzF=7=r9bJ|B!t23w
zZeyd{T+nXPxlImkmk1?O-6X@WlJSR(c$*cQ+sUDH8WTu2<#w6YA}|V%y1W))2S{R{
z3e~;FAi`V<ba|WW!!P^*#%`)6=rZGQx$p~DiQ;yD<HD4+Q0F)dJD5&K3tXOp5n8jb
zu2`-Y&0yN`0<UeTxPP&R;4b1>kI+K3fb*zyrZG9*7WFy+44{lLrcIzv940~hQqo=z
zOH|H`83~p>ZMuHu)0pr=($lDzY&YdOLRl>8>DQ1W)tRvs$|>{o<9|i;PZ)Ye7dv>T
zc8NV@G;EE$9xt@BE~%Ak#He7$GV~Y%4A3<04PtvBih}a&z@u(6oVY!a_b#3t-{)A@
z!%UkqxaQBacC)_KJnby_S-=d*8!Rry4Y&P^`OyOE4%*M#uLCOyXk*3gZNL`f?TkYn
zwu5a_&7+d{81i=31AQ3yQURLL+j#Ts|0s~i18wJRqq!D!cQ#|zfTZ2kjU;BR#pl=D
zq0ISO!?1g_W$6^p9X#H%yFU-8c!z<sXLnZ~ct3xqlme}Iz<2IcP~Z_BShKlPlUD9}
zON-7guwf>f>QC}{t?MtUuxjomfiBEbPv+?7zxp{%eL0nn9G;@W-{}W*7eDJ!J3Gq>
zp44mtWz9h8dKPrd_=8JZtPxum;*d<?U&n3!WI1NmOrhNyc<3C$xR&HP#s1erpm<Lh
zYPzmhah-0~F2!k1r-f#<mruAULGMmipEkHBl*}WRXUB)MuQ!#XN@y4AP6J4YzQwb3
zBfhd$^OfRnc|GBnGy7y=Jf9HAoe(max}(*AtzQtk!*XWV7KRq&&O5-{a+a;P$;^J^
zEc<F*qHwc6#kwD=G<E>n<ShH(`cOr(57?_fSsuOxjw(=@hl{`&#C_<mlVjaLU)cUt
z>y+g6mb{*Pr#(LqsQ~rqE<SR%tO8AVr~>Hni{do-o)3q*&a%CuktApNqt-#?t}*o6
zUxKzSP+Ng#cxVPRQeZp}J;?uE?x*r>AI<}Fo4W%P=*@%Bqbckg(Q=mDQZ=P9Or4&_
z#y9wO%6&!tdjcpG`60K+vnSjua_XEgr`!_H&SrHex5()~)mvqouLfvzcwdp<4Hmh+
zG%(!rL^P>T;#+?}9tCYmeCto>Od;M~NCYLmxe_IQLM2{;hR26Z=G?|?1i;BC8)@b5
zuA{^^Hzm5E<-Ax<C@Rj~Pk}u~$kSw#yg<;pvaMSn=BU<go(aJ>z!6F0(gBZNP~N=&
z{Hnlfmi1rygS-D#U=x8#KzS*(JIA^iZ<KnfvhANhna|O}(B=~mv;<Uan~(57+x6dm
zlKzxroF_CJ(1Q79hIIek18`NH=UtNdZw~M)8F{xL^lK5>T$n<Z#T?4|yJ#uPT)2F5
zuev{e8K+gYyD}j1ztoH5^t$EmWT0F*0!#-Wp`Z-mz1J-Y(=rERL$6zcVT`NSEq~Vp
zsw3ZOLi%NLtXtGri?RJn$D|My;qNryVFj?r-~PV*^h7<~&U|a8dQE<5`77IO4RK!D
zgLb<7_;1|t?{RqCtrv;lo9`pE<|P-rH4(>%Q3NDL>s{_UotJ(k(wTB9LvxXv^le*W
z$kVMj*U#pdwe(5&zaksB3ukl8GWs+(u2(RwI%4d)ayk-Ff|%F51PewcU~v*a#1?&&
z{^k=(K>E0C(>SzJ8aJ4hHv5p0c^Xmf_KO=AEx&0N=V%6_woOL2+}bfm>#L!-kkX$M
zr+04|@um`(CNZehHZ00JgV3x*#!v;>+>NByN(D^5xffP^4j|iBZ;JXoKb<!{JXtvJ
z^+amQ$^(dxAzBVB!KdD#1uQ}Jn_FL)W5jL*)~j-?$DM9nRplrdRm0|;va?_oCAJ<f
z{97V+BCsSYHJ<gQ2{9h~`)dVM@_zy6wHmgbn)pvyfMesH^-Z1m`z_IQB8E=upSny7
z-uLEMw=+~{N|vu-m%!KN{>A|B{RHr2>7R^L0uL&N^!`~4ETklE_j3!1)`ILx+qpzy
zC&WBd!>&TjoA1vrb+^Dlh5hGM1u)h>j57auPk|LY902wyBM(WzJ#KTm@f{<}OLWH?
z*U@O+o6v7O2ki;qM+Nkt2z~Ej<53>20GAY?2LAImaJQ&JnzI!Ed{TO^>6_cm3----
z)X<fxaf<{l|Nr}D+o}!!gYe&>F0DT-Klq=%`S!*C(@_6KH^P|9v%WW}mRG~TFAYNt
zyAau#KUA+I*FTHcE<h*rT+d9`>I10BSU-XXfj)wHoBrBV2`9aEdDk=ViEgP%eY>SA
zx~N2Jhl!EQiwbLeNV1Ze7Lq)Nfd55_?idv9jc&{^FC<)*_NLpx%s-*rnlXi)t{QZk
z!G=Z@^K)E;iA!*+{epsub1(7|%g|sAyA8dT&KFb!Z`U))aE?jRng%gV#OaQ%pENb~
zHZ@&u$QN`I#;v`#7X_)aB!{LWxRh+&4z(KYaK?E;jj$@{J0~eLJ*$GgbB27s0x))^
zZ)(6{mUOTEEA>m;MizX-m+l8yM3lf1+WH<bj6gNs_`ji3)pcum?Kgw83oi$0-();!
zXy5-4Y%S4Z#}5m6?e+9R`+E}D__^2SB0{TtMFK}Vcx`=3oDFOz7x<dlPXv+|`f3f{
z0IF(dn98$aI9xkP7VaS``J~t0CKolEOBR09UxygH_P>zEAqIC#y4OBMCuFa7PdK<C
zop6xK`@+4P661NR<t!=_*uBtKIiW;Q^9D(Ia@l}ZB38#6-pUC?Eb=9LriBHnE%Fsl
zS>(0T<1J@#3AC9t+iN$1MqjwBQ42#gHUTi)$ep#wmu^2Ccy|$V`?Ey(dVB5X=^^9J
zYWt(Fa$EYWiC%M2%HsO)t=K^T6|gO+74Lh^MJbDQghQxU?CPSrrh0iphYGWUDBGZ-
z1`O(t3Y=E=ENxGEEvtKNuk=Z?9;Fzbeag$~FkCviS8R}%l?+dHU3)Ngp7&Y>?d9Q_
zgswGIIuC|+gsSy_Cd)!Vr#{~O60l^jOSm7KvqAIK#lF6s22(yuG8qe2JL1(@vfPvY
z-oM<CvK*0wl;j`0V~iBtUPE$Y%1H{Gl<ea~nKMIAR|>RW;w#swwU^-vmob5U{44K7
zV?6EG9JSp$gU(-D>=@l<Kj`hKA{~Nqf%dN0j)_6B-_G!6<PrNRV*RbX{X=4xE%FX9
z&k~CY-CW<2*u_h|tt3{5r~<m%2yI)@Vo?b66<vCJ>(Cghngo_c4;Vkh@;;29LZ1gM
z=YDSVnaQH9I?`JuM2q$P)|vy}Dut!7E44}$NsS(Ed8<j6H1TY{=1nGb5@QD=WUwSF
z%VK%5q&K;^p0%Orz}hf#$-TCF#t^nfX+P7_zLlgCC<xi3X=5wf+MGCQFjqr{+4<dc
zZ0j(w2h9hQ!|a%-AnA`p9;Ypgp+j5IQjN_x<PPSf)15GR9s;q+(wPOvk$@d1mo7L^
zk9Y*yci~vsp|I<Q4k4w8GzS%z{!W`Ud{~(OI?jNI5mm>Hh3$HxsS`E>1KYY#RT_p}
z$BH&<7Ip=obooJ6*v+7F_CR8KDvfUcgq<OcLIZQG2frtc4CQ-S@2U`=A%Q|7Tn5al
zikfGrB4f4?x@6g9?XzwwXHwi$JgmZb%9CLhBUx1AQtp!ea$yI^<FeT?h&#aWqR_w?
z#2jGUFlI}P(ec0oB)V0agdYuiE2zoAK}41}n%w9+sD>5xR*ZP<5q-xgV6?D@yUD**
zF_?T7$q!Q|`LKs6EYE9vGR3pJJSpx8MSVxiVXSK{xES^_wsDqg3vrYFVJ}lH3Jv6W
z$bqnzsZAmMUS?PfT@9?iCTu2|Sw0q`YQKifBs=EN`N!E|GZ`lt=JqYaW?~K#Kd@Pt
zf9AvYB+YDPB$4+1kzrHAC2CDf))Ql`u&G6jWuELmif1xs+;i6Ku&Io=NE_Rfd^3y#
z6{L}4<-ZF-f^4(=d|$FQA!HhyU_udaX;_BB*EZK`Dt(7l5{8Q8V528Ywfkk=QA7@g
z#0E7b2ScDPG%)bRa$chJb<y4OWU!mbU<^rgt04W{vg?LSSHH)nhP6gNy{gKuF4B1F
z%d4uYxLO<6hL9x#C$8$annck6(16<xm+U(&l)+|JSc9MdfBqM9C0sEQ8X!S#=P=lC
zo;DI1Ai+d^K-321w?Q?djO|<?<`?TB+Uifksv4Q9f8;6szhUf#p>SfPR#j8TM2FW2
zt6JHOMzyL9!z4NI`^&zPJ=cVlLx9baNPa4v6)&r=ct%z)XEJ4vD4%78Rls$Wo3P?Q
z5#=WFN?2*7JS4~JgrZi87+;C)YZ{Z%=zyvFub@YZH3ST8eG|gUYUxnlsSSG&sHi}F
z9-07Y3RLBx8=!C3`6e(Td!;#{F~Cp-I`S|V$R@o`mUVDlSgcV2RBv+m(fqL3P<hYj
z9~PTvl(@RWSD|QI#~c8BWj)Q=nrs3o+ISzR@e23CoDH;j#aFgyOUFa`I<MRblC-9W
zg=Csbviucag&2MORNQIU71R+>{<AvBxu><`sSER~gM$=xaVB#tCaoqI`BNI0)?NZs
z7c7}&S?xzTp6ZN1A6J`Xy#3Yvs3nf-n4TJJhrI!;BvPE!`ESbL6C!tWLFMCDEyu%`
z!H2)<OK!L-BxGgfppcZ15L)T5Z-YWoLPD++c^M&ncWL|?ID`y;7j#te35?=$6;5Ee
zoMahk#P)MM+#%qsoEgkzQ3^gcmqnq=Wh#nMVB&YmSjQ7BCY>CU@uI2yxlGZhlgkvy
zTIutaGF*?RIG!+ZvCG<Uh0!@+^7$)`$x&w41&5cy?CUS#QrLGxCdXFxU22M({+ibu
z`-<plLnqV~`Z-YbXmO@HHuGU{oq{q>gX1VUYNvnT*k1`Xz2G2=M_QdRTMDW<cl`EO
z>Zx>Jz}R_?%|a>xb+E<uJGzaFhB?RD-j7&wieU9P@;GZYDC&?JE9SBcw<--8%_!Y|
z4_87P+uI1Jx71D?M{DWW9|jw9?@*%q21g2oZkRDv(RmduYkq6T-lZP93$s4LvkwP8
zTIDO&ln&hc9PkVigzXIZ_3CrASr&i7ObxcL$4mDUr?)}8DB8|Mz(}2^6@=}0)EX;L
ze6_E9|1D%-x+?W+NtIlsR#K}qLhvpCUJT}HCAKiRdY{O5g1K7h`i$-Up2)A4+Chc&
zC89hEzq;C2ELQTa0GCL9Gb?%||HW!wW&hK|bxVX}>qLA~vOS{5K+Lj$o;2<+TPNRS
zfJMxUUSl!C`!{e`WBRI{iRF9Qbc-5DCUwzI0V><*w;X#im2LewvSs#+zjivP>}2YR
ziXc2c;4{k3KJ0gFzeZ7yVqP@c5GTzrnQZmmQH-YT4qEq}-^)^E9=*n~hakVN9NNf#
zy=zeZkYYjkm64wul<(M2gv2u<emym6;CPk$2ws)F-j$;8TYWR~CaEpWd6Ohxg$`WC
z*`7&p?5-qyXBY{mtGw1&!GycgikghIBH<=WdCg*#8i4%IjK4T`cN{X5!8`cMy$pV9
z`~Nq&QY@mCCy}MPwt-)K$+7#<Ccl1t7qw(-RTIl%h>P=757W}q9V8C>L7xX--q^W-
zl8@emvAUF8JCH_kSBi7N6fgSW=;8gVno4nj%Ii0Zzd+=(5Y9gATxcL=3GV|m;Bj<W
zhNyfqfC=Tt;~;-TLPWdi+Y5Y3_9MEKCtAlZnAh39UxBa6Q-`lppT!%|1*pxy;JKHb
z8!8Z(BmM*UzaaA6wa(2TYh)G6j;uu7o^sBuXb6W8<|R{P-6)M|96LtbI!G7U$+69y
z!z36<t|GhPmvb*Vc0rRS?;6%J#Ob+0ix2$Dl6QIC_eL!`z_m|6e+XbRF{?{!Sp@6N
z(j)m;<=B;36@7D&sFp%O&5As(02L_mG{DY3b2l&y422@rJ16kjH>RZ$+uYnJQ_KkS
zf!pT%B-{$&QiRjBzwpuMMC^4=3Fl{hQx^r>k$(agb)oRvEU`iqV}99{<eZ}X-%N(?
zZMKg@-Ue<efSczF2RKaNyj_-uSb$w<&f7G6`5pi?g6_SDh}@NnVP<0=ZC3BLM2FMJ
zzS&gM39V`j)57{t5eVPia@K7o-98~S7thx>m@Jp`%_$+wJw~NBS&ShROf*q4@f)D7
zQVLeYQOB;&I^kQ@P<5^U7xb(JRKrMXjiC<^>Hstm{v#fy0+SVZj|X0g=X+5e$Ey@)
z?H`2J0;?3b#ls2Uxb&y(RsP;G3%cXYx^179|Et=wXya!A%vncA(r2QDqwvedddF@q
z1zDJK*1dtio4{oSn7u?50t%4PK5Ont(^{9at`m{FjFeqSDeu^=CFS2_Y~2WPvvL;2
zz%@$Cm+SDrC`!;DmHY@H91vE-2sI5<Q$V-8_xYBHpgF;lvtGfHdCR=PCufSo7gbto
z9oO(C?nE%od=uAQwu;aOUaQwdZ1XDA>=@?tiz}k{+8}wGdu2Dtn63Agdn#x-D?&D6
zlzZqCN7FqnU`Ldn0ljm^t+2JYJZ7u&qk8<cZ$y~pM|;JRlFbFE&X2S(1%K88#@EfA
zx`#yecDPG+h>8!Hsp*GE!rc1&+-m1gs?vdmmf}#l8kxX~waf|UIY-=<2w}YrIJ_)G
zRufm2>9vEhb|u5FW1Gm@g+p3tF>1zk)zhQe&AnBfO%#80KO)Qr%-(+0Z8tX=jVO#x
zK)b3&vrW>B*?W|K7(aumN|X)>sZooX3``t_D7Q$E{ZMD;6=X+EhvJ0V<aM1~*{?AC
zOG<`50L*n|ze3Mz{Y3a<VEago@|{)rSSI_94w4zEvzo?3gM%pMCjKa1AsX+EE~wTE
zpJPvuv#uc}BFBx60n|LmaiePh)o`ik3?ht6v5KkgBkCxV?2CH9G5)@;lBqMxVSg-i
zFS^Ej$*9VN+j$A|#hs0QlE=Azqf4P^@;^VuH2vc38R0x<csIMe&lNL;3d3_4j@|&9
zY+UP%mreRfY1!nR;;v1`8)7tV2-L5N<nauC)u}{x1v*h2U3)oGaI~aKmd}nJLFBN&
zj&)q5bqZN#TJ=u0D{CMlVA+>DXYA-FBx|4)kgT~tHU}o($)>YBrMIsUO&tbk#4H6x
ze;WN7gZ4+ST6TZEkM!|1wwC2vs&L3*Mro&ma`?{c^7;;L*8OuSfssCkK_Da>UvnxO
zl4GtBDl=-vvmvj{OdK7?Sl06;oN^eU?;xa_q0IX7)Xu(>HBqpf_4@9MhCz+>sz(pc
zp3!yreHk0P#&FujRS)6j`7M2;Z2abwCiH)r-FH%h(o)9V5QRaCWu-Kc{y1**c3=Z>
zvu39>sS%8mSx%b+wJA+0SFP6MFAWr@OWF{HxT6+(VDxK6r!=LHT>m-d+K>VXo|XSW
zzd|%``I`Za@Uu?ehqic)qjz!N!C>+YwLzVcyZW9>X`zYghR$&G1iC8FoQJ1?apbiV
z_IEdTnXloQks0Yzijmz?(oH_aQX3l=tDBXQj%iIbCtvBcO}-#KFgoRtTv9DaV2MW<
znguBXg*I?w*+9ve<s$22nUI+JHA7;mbxx_9E9QNdvaT^G&m>saH6&&5NVAk0hLQ}-
zgWWVJ8%w(Ou7NwN4Dz$cpMSn<@0w8*o-z1ABEP^1I#Fu+GSJ3IwZ`Kt6?`We_>G!k
zxI2JbSk`_grK(9t2v>`$DMmcV&ZAQ*hmc)eWMz{v9mL=MOG-)zsYR%=WFW;flEO~>
zJ0&@UooiSFjW)_iD51vR)Uu7=A`P~rP=-#T`@f|pty8Gb7`hL)(@lwemeyDwKex7G
z>5Z37SJ1cwt)|-*)SO*hD@@L&xR}SR;?<a~70e$qjHh~v(;TB<F+cSxwpH=+hOU+9
z%2UG&=10THJBpXD&2y}eTT!kSe;{7*rmAKXFJb7P5`76n@6wbqG(5vwn(<`8Sou|x
zqQ&FXlB!eNv{)L%;s2W!OM}7%YeI3Xc)ZES5>f084Bq5R@oc&kn9G0vSMhOL7#RX;
zb20Pvf%rI?XNg3A^hUfc#c{E<U2GxQ++d%7EuP!IgV-)EHp0cuL!pCnt51^kmMkDp
z^h9+I5_^_*F`qv9pj>s$K&&NBehd^nzIxL*rN1%IWV5fjzrCIh;l=&3s@FmnI<gi}
z-F`WrczEfu`L~!St(O@u$^(-utL@PGTAz708ndp+uHU+Xrs_NwZdbo`btx#Khzj+a
ziYVD7YDyIId43yqsQ-u%4PC?|{)7-_`mXv~x_P&oi*fHYQxVmAtbSz`(YsSzOl6iZ
zO^TDN)vpXY-6|TddS=D<L7#kgS<JXG){5`zG9zBC1%`pgJoZ>+`duC0?_QM<1#N9D
z_+<aU<L@#~cvmkyB0uxkJ+Z);EoLg=v3rpo*}=$%uX^lE*M{2g7bMOElS|x<NJ@&R
z&%7SHlQAACzNbr#B)L;w;?5%8D&n`7c<i=*QU4@roskxG+q~56ME#R8@<ocrZmEpC
z$NpbK*fLS&LPppjXSY=QHpCF_@Ytr9-_zv;lGrpaVN=`<Wsz9?JCB`S!H8tDlA%n`
zOPMaptrmBSSg$S$y!)Q7ByY$LzsKs&BFM+i^Vn7E8;VC+NL%f(liAzWxfd~JasU0-
zQgp#3(d373%Gmjto#ekv<bz)kiM#xVWrG+dCo#o{l(=M9oE^g)Gp0S=L}vz`TiA<e
zc2`K{?F~G3Y}~_G;r;K?LE>X6Hd~8F5*u4hVuRYn!hrt#rq1@*G0Nzcn82#{eZ|8+
zHza|N-)C&lF{r15i%QR!mB7oS9`%s>cG<k8wZ5^3KQ@7V(8yepW96I4fAjKY%Uo&!
zirfmZ(}2gk2YbFl_p=YM{xsDXT&Z04K3}=~e`0!*Ja2hoz7&7%4h4<HYB4~4!E@SV
zLa&%ce(%xuhs_#3sh?)(1vgZWWP83B)d#vvXXN*RJzIS7{aSerQ#g@ZU>K%e|AXNq
ziipDuJus0Sa}LnQU=Vc}8%mJ^KERP81@mTPy*+l!Sj)4|FghQo`mc@+JoJGt$@9TC
zfnFc@(mfyC3A{?!uWo;93C4iXPi0tEgRV>?p{naV9pfB5J@cHWYVc{ICqB+W|2ezu
zUebSSA0)SIVE3wMS)VN8`QRG6SDF!mUoM<ucUMuU=4^dBhEZI1`UPqVxLrTlrqN4M
zcd~VfkX?n`DrDzHc4slFiQf8Np#N513IEowMAIQxz{C!=4%D0OXzK;x+UD|)&$3&E
z_&*F6u9a|0G{EXP*KTH%%CYK&5g8{_f7qrfM9o~`c7ZY&wFEt3y*SI+7HNkMUl<m+
zEhUCXR^!}s9{$lz%T4E5;k4pSIxQ^gwMupala5ZJ3e~`sjta|OvFi!7HOG1=Cs&0(
z`a)FC$hJq6BVlITpxM=7hzi>EquD#^Y{L@@GyooAL&UaEP1pSL3A;{6k8P1ktd6S(
ziG6<5u3+@QwKSM9qxEfX1pf0Oi-#mZq|6X)SBgg7Q=t^(TkXUb9ZZIipEQbO)=X@?
zU0EYPqs$;k2Z%I2NU9>7HQA0=oQikb5i=vS!4u)*GwK)ZQmSNu8QXk?Jlp;VEZ^pP
zz%Cm2WE+E&MRox(s_eI?x{L*IFRWT$6HVqnFB00TV%xzD4Q92Z0Mp|FqZOd3D!|CB
zKtKWJ=LJ3jwkg2cOM#z(9~hoR?<b-O2?S`j!p2!`+4O3*riSH2S<&AIkvjr?w)@I>
zMfab;Gd!m7xI?R(X1!&&cbpZOQ$Cl|VjyFr?kE@dBFMO-dEi3mad;sA4lY=45~#9+
z<08-HScR98kuZ~yP4gKKd_3PCVgA2r4_9Wle1!`#%5iUU-r1I1Z27lk7mNZN1%Bh9
zI*>x)7~<{_TVekSHbFvX0v$t)69_y`_UnzZd}g!E&I*AeJA6gGFN!~KVTUilzX9|j
zq?pymou9z^4p}a^5SXjLJ|5NsZ<1yjv#)w9Oq#zs)inD`TMCK=pn`iT!!Y(yI%%&F
zqU9?1iz?I}C4>RF{&hHqmc>d&o1Fr+m*+vYpJLs$8A;7p8e$uhT}VTVX191?Zc-?o
z02?FiLX>8qYN%jX&lnXh^ZBO`x)<n-5aSdAa%>}V5rKJ(%$q++njGThuB!ZrY&39o
zv2WLDUW$wsdK-8vmCy_=x6t7kW9`Zg9NFzl_TuFjtNkvFH3w{p(HpgQPgNSP!b=q`
z^qvCicsK%lDNRNEkTms{;u}{Xw6dwKaAJ&&1iV4zpp|kp5<8&W?)s24M>jUD*s*L5
zmu_}kY(+v9jFPM>Y5UXep#gh09p)&M?1-U#<19W&Qf?T@%Fu(+LDk3f{vWX?%xOl|
ziLC?FRNxsN+5xSku!v3RdYdF>Xc@wwsqVbs3NlV2Fd?My3ZrnJ#2%4+ci^K4XpF|J
z_{Gjq0F7f`29_CxaRn*-Q5|DdKb6iLi;u!HYL$q#eE%#}*NM{Y`sP>-G;%dv`A6*C
zb=p0Sn6<!~HbML#O>P_TYw?MiN98ky=3{7fvT^VoHc^6=LH$W~ADX(4cCb_JouS~r
z1-|yPo6>2=GG2pGn%a8#no1wFaP8)UJqi5BDC6mPHphBoI?v`k?#HV+OKhU3G9XlO
z;>y`cHfl9yIgtq3ZxbC|&Q4LGH)h2n_6BfGfj*X97<VpgbDNW?!q9qy91#&(X#7~s
znreFC5`g?)Z9w5lK#El^$b{u#QdxyXlvFB)fjUySc8H+`fo2e3QTxVusMqt{R*lcc
zcd1!^6L-kuf?q2>HZp?I{>Q&fbT5f|<d8{1C@VIS<*ILKuylLWBRR%i+hQYQ3QG32
zDa74`YNBNC`8O}}wq=uc4*-9&x6wir#?1;pMK|@)svN5!OJ6ybpMPjy)UFd*)D_au
ztt)$rvI!4CyF<Kh!Mi#nbw7bmDY^H*M^IBK=RI1~uE(x>hjEl$_;=tmf%mUaTM!%<
ztH7^3V8FNs7&tX#O%BH^WEyq&X{8#bqQRSiX?qw&S0R{1wDG-9Sm3a?k;%B0K>bh#
zrYi#t<rG#?yR*?>Y<m>pJyFzdi~gUq39(Lv^=YMo-)(!7D(apcPBlw;4e4Gvcux|O
zI8NB`1cNbmY3WlBZZe5xFpfqbZa&_kb4?9Ts8sl6Mpt4!7j}KJj{I?O%Z{TtiCcmH
zaB$0x+X$?;S_h@(7{$(&m|K`e+Iub}csoQNmJ13t1CNHhUZYS-EG-a{x)+*HLsH2=
z+}DzYx`W&1SZVP*8$*5Cf_}^%+{Ye~z`v1)6%auJ1ikD7(HMAeAJQE>5)ss{IBHjY
zi@N}zksZx4>t_iLAN@Tu3^YC#caHjcW}Iag;WbH-7zO-1R0GO2AwXA@_ARUo**;y$
zp_el+QafCU=Ttsv=RjsfkG4ac{o$m^cV$wPMV?TA)QY?WWV5jonOtgR_}Djm=0#R;
z=?>Czv|Gn=J~M0ePiSe(J$#%VCPeP&z%ABvQk8wiyqII9_v3kys>NsL@rE%JQ4s+{
z9ed0>JbVFs%*G;n%r*<djrqJNcaNx?;S8gwu9mQ`&>4I-KUf?^F2Zm@g?KesCPnTN
zbKByI12@H`s?h2DTD?lxf6$<rgo@J76s4IkDimd#NzqbDj?ev)CZTko0d*t&9O?hv
zf-HG?i(4){n3Zq9T;<uc9(&)AoU?Fr160GuT;_pZrRW$5{YR?6(TMu({#pqo?4OJ(
zBE+4Gz7!JiE`fKX#Xme81pHjBX-*0@Dz9cH+$mw>ug+c#{N6ha!M}j-6gbX<B7Wwj
zum!czJM=1;6}RE8@{=}#O}AJpRXKcKR(?8Zr#+$x|Goev?`ui|C(&ZkA$Io{rR6Ak
z3170s+L&T+f32n^-pKEW*|LlAznJ|90S>c>#A+(`ABklJzgV<M+G*Yt<X)b;E9reW
zqSKsVb}_6}%=GBJd7#ZMrZ=3;hx5?U8ekV|O;>DK<%m{zy~HFl$`SrCV3@`!X6PbW
zvw;_a{c+=S>hHokC32^`^Z7fXTbMhjX?)e~en0;~-EX2F$8LOAQx!<f4L0LUQ`jU<
zWEvzFIybE!>1z<d8LcK4c{Xd)`@TfmW_WI~)xawXe8a;wV2cJ3BEAc;6+WF0c9MRx
zZ$b==jEZp~vxZ{TMI>#PB{oCbeUqf<LfK&kL}!3sG-zmQ)B*ptv@RX!yU!QlFGZ8d
zChmP7;it|xtTYP3Dw@=pnm2S@j@7)WWwE;kvw!h9Wi|A91YQIF!1hBI=w~IyH<Vk9
zuAh~(OEJMUl<|p{9bcSU7OvCVLl+QYjXOSFftB=}@dF9<zKSCaUB{w$C5mw9XFSv)
zwQeHX5B_Ojq9xREP|ZMbXw4o<R~|nXm?0g`k;4}KWHZq7avKu&Dif>46T;k!oq(da
z+))2@xF!-{`?)>z4#Ktoo0Sck)ciMdtd{D)%#y->25z31eBiNtye`m-DNn1$zV(>u
zL{J~&FZLu*tSG9qWLBVtZhsji8f@tFnz^}(qU9>+IB>P!uNTnfj!3r2aPX9^9{djV
zaDbJX*m6s-1#ty<<`;D(s7<`{C;0Et&GZ@jk7I8<#V%e5@RcS|!suq-eSnAtK%JgM
z=C_slMHMBfZ)FFDjw?2j)@VkOt*Tp|*m6>emn*~cGn60CoVGO6^54uKbF9`6k?dqS
z#FsQ#4hQJ$;@yBwJ;C3I)9>%fW$TuvKX$$>B6;iyB#M!?u9^8VMJ<0#HvbYgC>YmB
zrixgX+b6UhC4QreNn2mtX)PC9VbZ=v+I(2YE<OnuqrhPvHUM}BAK2mH0I*Mivpk#t
zenhz*pXip()*3T^MHBs{+`>IpHy%F#b4tmAI4FJGxHe(IxRZ(-=aH$}sP1S!CM&rM
zc=(L-caGJzJ0)St<={$uCX7K^@jrmG3Q#sBiU14?JYoJyJkg<-sH#v?9=Za$MJ}w9
z>sw*etrFz01UnTyVIz2$2QVe@gi#?%ybLTWWqFExmt(a{QD4ok*7Ln9Y5WiDg02XS
za0zCeB^cg#iu}d{gOw5=l7n`sp)SfZzo4536*~1jBr)AcOm`*9tOsefU5U(POY99v
zq}3{MOcL9MJoY|;P0)7&`1Od2M@%E-W+GigiBqutB-YKACrX}Ts=#n*L1DDrW!YCK
zs)GZ;s3$^A{tZ)Aaf_zyA(EU4#dG&~<-1g<EEQ9`>6WMXnR_^D9M||HSgPRpIz~gV
z7oE<?r4BK`_2esqLVp40DTRCwp^g4?COJj_JA3A*qI$l*mR*vjpkxF!K3`>})1JA{
z=(hqJ4ze!z*M%IbV*|u4HDY^EL;kuAb#b6Dft+w2DE5+7sgyaf)?~={-pine{zK{=
z_?Y-_dG2V$sQeBbgNw>mlA7l^@H-D<fl;V*;3g0FM#&d@a&+myPbuisiG|I8j=BdW
zvewL+a|9C29QJS~wdMob-jZ(ss};!4!>7QuayYk$DF*k~Di~aL6$##4QWI}6m+}E#
zO~FLpkqp(;KZ?(D(qlP?8wBODYGs$w79GPxzfh{I?A95V`JhDLpF><C%QoK5YJ9Vl
z->=NHy)1ML$xBs&v3X@4S^$j{!2L@N2KuUx7w1tpE#F+eTpdC`2YO0~aW#RL6_Ci?
z0G~KDD;Y4-d-^sfK2U@1^YCTSECl~Ga8!M%S;^>pPI{tNSuGMI_?d<9aawovUi0pn
z|1<Ay#=x@+&44@3`|2ZPHe<b^)M<ceoztwgWtWZsSP^uZwc~-jmDW86&ARi@3TQ%G
z)vS>!L6?a<%^~U4Zc(bL`kydkj4l|ec`VVqq#qIdNV+><qmTHIscJ?JN|^rZ2nB3^
zcmv7ffZ+tTh4H`=Q|YA&6yO2xD*XksQ!}}QRd)HfsNGxhGD+(iXKZ}1sNGIC{vUZE
z#K_RO^!1>~GVzvOMgssPR;C(InJbg*ZKV+QBmL|)n!qxxfyUUdYZW5i43s~DNguN8
z?WGlm6K&HDD{?YIHGvZ^y!+on+P&l%N29S*U@{T}6LCF*C#O&;zXf#~#yN7E)uzB?
ze5~(XZAg{zA|AnbW-sbwv%jbB8D7Y$k8$$pGeb&i@$4L{AG%uUCaqNz(Bc5;ECOCs
z;B_8$09zE;#KZT%*Lb1ftOBPMaf;(jA;znqbApUzqb$3uFUVMy*+bbf42~jBk?VfC
zgfyt-&5)K^l15Q}fbmCB<aUCTI`$*h{Q&EhO#^BZ*h*2BeH<970E2+CXjyid0<>pk
zzXLcGZ*RrRm>Jq-|4V@ynQD-&e4iGz-_~bdjtvGW!3pJnvSLSY0tzJ5Q2@aSoq%=<
zAUJ_3U&0`Aknht3Nhceljs|l<1H2)aji(k_+6d=qS0%?9&C5LKj}^7IQbxyS(~h;P
zhm<MUJ7JRCebg$pBfJmpRa0V%Oxy`~Jaufh;&5P19MzIXEK(yf>Y`2n1*RS2+$f7G
zqe;&;i^E^<Lw<XlErH@oz+FA<Gk9RalkgP>O569-kDAZ+jFl5<+7hF<n4rC6oU!=J
z_gk3+9Q|u%Sw_tNqwf9Vs;aUE?sM)vz&(O`6%<qyP*hYD1XN5^R7^}POjJ}#N-8uo
zN-Qc0N-AocqDN|@X<E_5OjK4>RyNT@lg&6Y*3e`#HV;iUSy?fYPgzmD-?h&<fHU*F
zpXc*=|9a7V@BX>=+H0-7_S$Rjb8f;^s!VdhBrtCi7E7X-$R+~Rp9wr-KPB8Qg!zd5
z^trb|n9bPVE_*Ssv=Ck|mK5l3#!_nhROP=FmQjtxy}1#KJ4YD($KL~eX$`P`LAYNM
z+YNIxJW2Qp%$GKT`O+qG))C~bd90%5XzGu?jz^c5z5w*EfU;6bUM3eEI9f^K4I<wX
zzM}VXY`$}u7n-gYhhDbBD&1lE9}eNPOn_i>hlc^=L237nVUA%H8^eo~d`sUal0%pS
z)5_}r@?TWD+W{oQD({dyH!T@gBuh`~l0SxxpT{`Mk1_mM8E3~-cbsD?fVbHl=NLBl
zV;c21<LRzRKK4V&@>6i+*XnnFWT|Z>c^WX<HPG=1Etj;@3hQJHz82q-V?;U$C?4OE
z6GRa1G3-TrOPH345Hisqk%#h2WJ)Arz513M5sAApNOml!?PCC15lJRfRIa9Uw92R|
zOk=cU771g?)US8e?@AO6bA6($>sMB(`VOS>0-}PawwwQicJz<fNPlAqam<4eR})!4
zm@A2!iL4?lcit0iT!m)rl225{gd4e7`Sv{wl=l{xiEL#PFB?N*v992EtMz$r0nI}2
z#pu@7fSwl@IB9z0#RX;}WKX;b@N!Xi1h`(6D~a%76Rt-3-~KbJ;E0V#bNH}$!BswP
zh7ng4GxAG>6aje^iq@+Z?I6lkNPBBOZ9muiN4H8Q9Bjs#`;%X2+Ap$BgyO1Xsi|wx
z6tm~uVO>T=260WRcWz;*?@k22Z3I?x-@DU^AjyfW=Dv656G5~SpNEa_t|9Ul!n+6X
zu)cfl4aNe#wgRrY1bB&t{~f<n6Jr0JaNMPDMWxV}Pw3l!BJ}+eMu0=#e^Bj<F%EsH
z0DX0}KJ+Dx*V)bOA+MA6$lF~F7+gzwwpWxgU|DmtRR#)rz;&{ERq)B>9iEo*zq19w
ze|MsMpnCN<$6{%X4UxpTM4||TA?UknX7eF~)Qu(QYRFnq-@_X&<5g?sRM)r|6bT<R
z4{jg_njb3eGOiabT`SY^D5q7vLs*t$#}VQo*&zln7Sn62819GAL1WhuE?}R)d(4gN
znPvKuBhUW+MG)|)OlT3%u}JyY(lMB%BB4}-f_xf%USuw5DCp+)P|!)c6dZdkU}V=W
zTN90}UhS{ft9^g}SrF_fM@SCRwbZ>CIvnBx4#+}`>#I+@)bRK7=enX=ou}DxeVfOQ
z3c$0}-O#_wxLIVOMriS8Rw(lZ*1ECx5Fm_s1Hw4AmGHPE5XP}@5`HHMgmLW0gb!J3
zZxAN^NZ=(XX7vrL4Raie|2UiVrTImAXu$L{!v8ot7uWN<=YxCmZz+x;M4jVMiz*>h
z`EWpSCf3C>-|0pi0h5y&fl1|mHbN3EiN;N)DW}KW<D&BlDjy`yRQ)rjB-Q2(TPa45
zjGONIzbS?nx{@_~3Dfo6htW#n{~iZ`aWGo@zsE6*ae2lVP=eDp9~MsY$xIoZFPlMO
z@;DjXDyagGDHOO;rFWK8O9o7KvwN8Aq`eW80On%a8Fc5c0ZE~e$V+heeDQk<C&pb#
z*dz%!G457EZ3c<y@-g3~{I~|P|0|xH)na^B-vD#N{~_tE$OE@YJ;k-mm4<7xq+i&7
z)kexs6W2@1F^mKAsw#jzNVp#WSJlbKJwpw%YL>1kZ2W$>S&JX-FSsAgDyjVp;b}>f
z6Zw^qZjNh{l#K8X1d*1iLL%=G*btaid0qnawnc$wfAxnyv%rDPwp5{le2ORfMs5{=
zk)Tya8^moKHU=2Q!Z5ZS)Z<)3*S3b6*dYGD$br6Q>?tb$(`D><%P`0L<nq3}O>UcP
z+~kufJlx#DM(8i#=zpsE18T84j)&LUQ9K^&ef)@?W@r+c711m(Tz;<Ie@sr-pORL{
z<9JLZr`0SHGj%ILk<&!|sax%oRx_m{vNwJSVX=qeS_F6eYCQ7XHKRd=#!;pzo$U@v
z6E@J}FC~<)lX|K&v}}j9X@@L9FtUpK4!6V**Xzh%IS@3fqNxgd##cbGs?qW>Wdt9;
z)iq&c#N6Rmz{mG&Sb}Oa$4}eI2lO+mpxyYM1eV13pAnj*<QS1>39Yd1mJH3hhmLWM
zEd4`J=M$gz2KFru3csRun<PdPK_147ab6|E7s9>o&QKadCIP`Y(pH||dGC6UN}p5h
zq(`M6NOa4}E$qJq5n~+Z4!@q8cmDUDa$E4zYGzUxOL)Y&ZnLCe<U||t)Qt$hEj2=1
zKBcqUV99&3!52m*B@#wU;!i|U2~(J#P9vNvG^(XhaQN*I&1sNLojJTg#c6>8NkY}a
z690f-bQyIAiD!yx{cjUg2E0n~g@htWK#Qa+2%99qN}g0pxEZ(uo+EM>;ivAJU8mIy
zpN0|W$NLxje|^i<MoD|Awuhe$*6N5FA#{4CI+Ao)Pi~Jka%$1F*hERcAT$GBESI`N
zj^*;#k37o-^QVqohim@Sxmu*AT#qw4h0^L%wAF6F&Y_GTdLHmzB%l`j(MySZNcaoA
zMPs&1z&kR5TXR~=-X0~q;M2g~GjYoJ|NAj_a1&7B6Q;YWmKvy9U9VTwN}3Vh9;MG8
zdksFPnmf1&3ji}u2FLstY;R8#L6uJ<IKh0J#ii5dB0T}sFd@%HdctKS*0ZWauR+>x
z$9B^cV6*qEM}i}IS-rgg{`cP1Z_rb+wWQ0~)lDBA!vD^{BK56T1CS{>;W|ldBXTF<
zj(8Gx#_~an`$`2j`82YJ`j$t~vQ5gdLur9NYP}OT9bCGv+<APUQ(o>KQjHOw(<a=z
z4@LRh$24mZt4sg%gx*7XZ`8XFWO`7|>Cha%4?%w(rP9e$k=jxl4xBtO6%b_KlvQ2E
zquC&&1^NB?a`kA+CBC~0aS~3crVH3rJp~JD0*2~@z2NX@p@ZA|;{-QQTihbn@1q!l
zeO@r<Zfd#fiUPZcBi+YEwd$4V{gZyOE&&{C%Y-K+A@G@#6ON!Xb>$i%)Z=^_MI(H(
z7leA1{)bBcaGOs=6HE*NqsO7jWjvYkzTV6@-qgnrfMqRt@2KnF=hEI$NxBH117QTM
zZ(pEo7x8vs;@>cp&2KRaCMFPgi<SB9-+gyuGn(IiNyb6~N7v?e2AcZYcg>-E(*B8T
z+8e>KgTv&7CVzWhV7yD!BElLEV5za(oOmfZD>xPi-*O|$fq5;JuB4;y3<wQd_{Acy
zWJFJ}J$wB%;K`niGVfqRiYg2aHYhr*F8dpAU1{tyuqC3M-~})KF}pK#(|AW*NAtY!
ze#D_wM*8uaB5x`3ktH9C7)z_vIIhQ%G1Rv?lv=L~!Wkl!DgL|bQE^lU;XQ;MlDi1R
zIGZ+G@6%0jh6Ia-`f%_2Ka^Kl28*Q9&-qz7-Sk<Sy-s*da$lvXd>`@OeXlNkkCML;
zcplJfWlfqil`wV&iT?r8R-Q^rmipfcA`s!}L*iI7{2v@+yT&iq)tk&&_o2t9kMVpJ
z(r{eAnBAEpmH1goIYN^osR}(So@Nz%hM9DHv_oNOk6|e;b$tip>27*Rmb1EN0${L>
zn(biljO}93ie?iGs*hjL2ZL)hY{=wba6>O_2LlX$ux~AJ`{v?d?<|&wxLR5B4kW|D
z);a;vTii1MPOxMI0%kubJ`*Bi_(c8Wc-Tlg#sT}`Aob+&uu<|5u8i=*8S1E%W;?3m
z!<FiX<6%Q>mgIB(N$Is}<cDFy?J~FWK9&7p*r@s2-K4y^Q|9=RHVDMwHr+nC&js`z
zWx~HRNC)muc1Q)a??Z0R6hOP3hX3#LgnKLCx(puxLb|`pi}1;k_6$izeL%X^xj|tC
zQyFiY`Akfg@l=vn1Hngz`ffqMPFNyfj|fA?xdQe~IudgMVeJBPvuT0Ck#}$T|NfHH
zn@3Lw*E1~QlQ6p{T@EHsmAaTbeFi2l;wij9OSc%6hCds7t+id2F+QRDQ1izitJ_lC
zKCMS|Hp_&OG}Py}4gW{WQI|^_0e=Uf&X1c&{9KdnB{WD;4H1^(NzH8NTtASE`;_(r
zeen0NDe)M+;;s3Jhi)A0DnI99JeB7V{iC0#sUL;KEZohtv1_QcR}kmyMyfQsd3rE`
zBAwUj=IIwo2QJWgC+V8`@zEFR&X2<4SX_ITj67sr{F{q;o$S-d97KM0sPAq@_VEPJ
zU;I~4ZzITY+~QA&bQ1U>0IIQ_Ui_6|CL82@%hvE5PBL1NP|Dtj+X2r$=GvO?oH(jr
zuO&xIg4vZ^J%Nw$lA1(h8X;Zay#c&);AHZA!W`g<%k`N8ak&QHkGoA|-~}3q$2FPh
zlzhHSpQj<q{fD5%GYm>RuP5Ho^x<|hd3a6aB>TkIlBYedGUL^wC&K!LWN5eR3n#*2
zhgV5cs+bwVZs;0xNdIxJ>N>#>F+~`fK_9Q9j!ca_&7h|-ARhdHQOcax+1bdopJ%%T
z#?#p@jQgZ-E%I_$&NJ7#w9p0ll-IUUX2a8ElI>-Q(ZCS*S07_0mB`Y4_D%*>LiTY{
z>Ys}L_H1M;8N(*|0!d(;C+{TOEs5U~d6n=YggyJuM7|(=Ac^CKIXRxdzQT8Q7m-zj
z72<e0`&m=0>iyAlv`<!KK9|zrphax;Ke-r7iYMaD$#gjR5~(Op##04TPO2N$phZ)t
zhu=Xz96uSFPrg#Kq5(p^Nzs0_?31v_$Y-&-G@YJlcQofG$J8~SgpIL}JIPN2>LE$S
zy2%vvhfl&rRcCoRQkYq)iB7jr=9PQ9jCbO};G|IIWF+H%rHRgOBZ%Yel_!ZjPr&%_
zz4A*UpAuNReXso1FsGm{rbObedgUk-!TD-V!7Q1wa4k6>3i|I9;5=PnWcWXQ5$>3|
z{M?7}4hma*r=Q9@y7A3$r_b?J@%|}FJ00mbX{&vohK;YbfIC`Dka{fE@0xl2hCbQq
zJ`5k~vpDcxfZ>aIiL}RF8Erkr1UxBswz~t)+1*JIJ}Y}kVw?!s7S?U#r8|gGH$b|V
z3{ji{y<6yD#8yXJ4`ZNoA7FCDR#!8q`YZ;ufv}#MdtT5rrN6JEt^G`|&*ZX8?AKJ>
z!(uxH=fsqoA<Q0mgjp~_waNN%Qrx0K`g5yzErsK8Pi@w>k-KxK&Xo8}o`H4l7Tg^W
zM8B^_aqjs5)Pdz-`p`5$mdkiYhD2L(pz_%Z^gl8Lf;_YUwf@kpx&qUQ3cjg056y=f
zzzJPD<z7i(o;nFZj%hAOoS*5Vbe2)My0ovRTv*Y)-|)xk!abnn3)Bc2TQW5P4@xst
z{26ZUi@MAJvO{-+qAiq>|BIgrK5`koiGAIa{e%b6Urkly&Zes$q~s=YKPG%2N;J2<
z%e3o71CCrAtv5fPUx|q2vOK0l5CQ@a4}mYYlhi0eEXNdFYB5G$%$DQxbyD;h*=Wd=
z@%-G3&Fol+ms<Y}5(t~}9LB;sB}01@?=@ME;%HtTZN3-|gy%BINRe*J!LDl-a3su_
zO=5MI*)K;#LrU1TrYs}K<VNW>-NnzX*{vijp!`Wj2CY+AEmKP5V%7H$>cJ2AX}Nyr
zP0@Ovf8|QSD*5devtO~3{Y9jgou#QPbKLC5qWXKb&|joGh>IFbpC?DeT+Na4<21%w
z<Nso*FsOpLE$@)lMW@`Rn<ejRnf0h}uPp=#2jAcG`dsf<Laym4)brt9r0n|>nj<$x
zTQ4)KPKO~o_by^!7_uqyaOTJx8ScFpE@sL-gu5km5s^m;k3gGyHv{7oq+$2=#TQbF
z_DOk}@RC@I_r48?=TU*YZr!{WJ)45!O*yQKzN6@M!+#2KGVf(+Okp`qLC2UXjL2!i
zSCWV_%+%^oK75i&ATol$Z|<0C3K66sb*v;{W$GM4mLzhCVA`bSf`eK?M_(HLFJF_A
zmE<C^sh3G&JrR^&>I0G}H_WLM35iTHb%W&GLF#-$z9<N#h%xo1u0=tpSG!DK6vH+~
zB}V%X;KTVx2BcFM?Nc`rE|tXFL~bNp2OUQx3J4aNsZ4>Xcfwd-DNIHY{sWVHgYP=c
zy}*wm$1vFy?CYclhET1V0Fi3=|H*5IP4zG!_7ZkV;wd61@2SrS%vM@*Q|=%7SAwqB
zvQnjgQ_+Vq)+PFY!YzpqDYDAklAnuwxg|keQtOtyTHU88N*6V_MPl`Ojbe03vs<F=
z3Kv|g3=5xkuW7a7fEK~smvxzbqKs}%wEwdhxHJUP?Y^4<y-Rpo618{>rwZvlmDII{
znZ~%&Orh!=V!Q7rg)UBumIQ*CmPts1Y`cLxY9&<LCM7SCdmdq_%<0{~1tD1%k|=&q
zijPoKL|7xE2q9a@^-r<*eK#{3;1Rd?q$r!K895aFRTsG>%sp#R*N+Qy4crO3F~|Kh
z_Sy;w&;)05Q>iysrh6)A2=mZ;_c7bQoFEl2M9D+rn)l@x(f%)I_sJ}E^vRc-l=WrU
zc$+EQxQTWTtBfzhQmTvkka=ZN&oZPBeq#-J&yRkcrt`_01N=XqNaN4NEIFI;qt?<c
zmE}@>PZcHwG`u+4-x|kM-30?0W2u0pn1=Q?8wV2k3E?*3#>LRFQ7qUrq%&;~9B7<Q
zHr|_4!huFI(lE)=nkB(<nDz|eX-S}M(|$uZB#8@&94Gt@ikYlWj!wUp)len}q|X@%
zJ4}?Q{v8@kD6>$q>F9vE=oD9uN(}SCVL~$A_oB7UO(0Tb_`k+YZr;oCo%TP3FD0?V
zFw+@I+K&!#j2{lL1jr|Vqy4vzH~jxI%q32GG<Am<a`JHR{Q$g2+B4SlF@yw3v=Bip
zRi`IQig}qni!ehHzb3MpunOAl=9=e^NINi|jP~!q6Zmy2IP6KK{89qOq=~E3Ovgq`
z-zEtfr8g7AGrh+q@&@5G61!2I>4;9cY&=FiNl$xbGhx!dCQHV*=N!Y#2ogjE?^#7;
zGGT%wAX0{uW=rBqBFK5h7D=GCGM*+pAqn`K!DP&M2mbA8q?6wn{;#+-Xztlb?k9v3
zlK2@Bh??;wX2qHx>zz69RlYFcT0IAx?f#EV&iwhRyl;u{Qj5Itqi~$5_thPJ%VE9o
zW4p9eOTWS($=CElrHuMZwC@e<@2^KgizhZ2<}@ET{2&R|scA8UC?U*W(CSZIJBE+^
zF4HK+eAk~wOmv={Kk1gqRG#RyM2SqB&bXcc`DqIX^Yk<|e#`*B)gbsa0_Y75DPz<>
z<~u|Cx~Ol742?4fDKBw`_Vv|$%XJA>fGCx(?^UzE4vQXx5_R;1Lpf2Snm&38@bXS3
zyt?M=u*~Z3yZX%b6|554(7h1(#bRgJA4glSVy=DrD8s5RV-%|i@`}t+A2YUk85>K_
zw5t%Wd%n}HEW?NkjkY$6Iq)2<?(I+cKEiHV#f8%9-r=O4B|H_vT)%fbF;?|yveWzl
z)X-P{(+x7hEgIcWM$KCySDj{$aw{OBKxZWSy8b7=yQAd05xZBAdDt+g!@TKYgVnRF
z@%*kiT};gSmx+ucjF!Y<B4-ok52wF6UL&^#MH#x^Fn4}o_`l&$oQ@+zU*D~ls?dXO
z$wKFcIbw8?UKB&y88vqqzgg{KD}atUBlu>M<hdYDx_LM0Jf|(wPPFwm_w?2ai*WPw
zQi?GTys#u^XYdU#h1#gLe8aB|WpNKbl%hbYqsPBFNs(+CRvwoY{6vvi{|Q72708%V
z&QQ>O0nD2EPx|DtU0_bu{~4;HroO#d$nzISvypJIfI8}e;xyC)RpG2G0W_@c<?=p7
z5u#xOr=_G%Zn@U5q4f^7-|nPHV_PP*LmP#LPHji}nw|m%CxIZ99&KTC`MXDm=%j;P
zQyaQ*`gYK(1M#Nck&V6jtr(6?Z;^|fO}}46_+fs)(zayriCY?N^6FLpcQU6R9z=yF
z2#*QGr8IkZ6e*VW>0A^sAD%?y2;uh(-VQnlf0rlNv0_0-B+?RNKAwqq2{12}BFCmi
zJ8k1PVQskc-8Jrb1WGPdJIT`*pq#3enjm(8^OKC<iJwB7-vWOgi-+@*jHZp?i)sIz
zY=Yfhm~7R0+3I7a+cl1v?$xQ%v;F!$#`<i?(+MuMqJo50A2P=G2=6*NsasDm2%Sbx
z!kZPj;vMNO1DW@kmfa^EQgokBd9|*fH{#z19LG?%O9$Dye_!fs-Mi&{z~9eeQ?Iw~
z*Vc)m`W}%Nw^8>)GA1pY&e=wV%ys#dlwKQe6+3y!{I0uC=87cb?<~VC{R8`xAQiI2
zDe-;u7&i-Jhyp&1s1XPI!+`KiTGvvAIjzUG>Ih@gZ<8x!_<;KEzAkfUDJ_}>)DMf0
zlkXmbILy%LT+lT65hIf~8JROCpl!bEkYgNZ_+2MCD+uSUCGol$%clT3BnZf-kv%w+
zMZ*6>iokfCnyfA}F82VUI%iztb~WQx5;w<Ft;2(SA31V^pPzs+hcct|r(pyJhgI{k
zQ2+OzvG?USV9XgTn=_i2NJG2HkWIM1U)T{XU_*~OXB$7@#lZT!iZ#HTiqGEKVoVj6
zf;se*Lz*9WR9};3^$8(ObO2h`$mAS~dVKvg;Z*OTi4p!bvnrG>LVx{|iufUHy7{(R
z@I%;H)oGJ6Ctr5)rRf=|8L6q0OEz3paqZ-^)HMF5W~@t_nx2_Db8`9A)Kudo0I&-r
zNNnqePB4rF`DFmZ2)>SeH0!x8^P&BRNdECgLLKy+Ttn@iN`IkJRzsKhGo#k2<foSU
z@rSUXtBbqL&4ukKkpJ9tke{ZrzwI)cgQxL;{4bTFsWf+NmwCm$=FwgywfRR}mM2DX
zQ2A>?s>`f_4n~n-W*;yET)}zrkG2)EVKdX*lGJ-9v&}H~nh1<x?lJ>l+VO&E1`J~(
zcirfGV}w(aTUU|_F@4nhX`<<)`MUqczWd|*HN0lYm@LuGn}!n2WT$9!+)^vq^Cen)
zi)4>8ErH%(a>i(MY<K#4nYS8{dbA&{-??X2U`b|`&pb_*@mYT`8o+DX68~oUf<oAO
zZ#3p^dG~{lmp_q-bp49(%X@wqm_GPH-@0|z7(Sl*r7akyJg7p=K;o#qY-R<ATkvog
z9lN}gWdNe1N#&Y>3Dt3ZsCE;6ZC-C^&RbzQREu<|W;s-|9I9Ck)hvf<mP0kmp_)_k
zsFMn@9I9Ck)j-UnnnXRS5%;JjQI~3#Lp95xn$y?ImP0i*=zn}iO|%0`s(Z*5EklnH
zi8Vt83)z_S&Tvp3hLtvOvmyI>nZ9f}XM8ZQ#E5g!nX?QtbH*^KM3YQ1jg)81kf?s)
zWED_`9?4uLi98}0&6(RIkwfGU7}l9D4wnXqhjoE`l*-2zFt9MTXVP}&B1zm%1j~Eo
z1CnSk%(MEDkeBsZ>@&_{GkMm9lK2HPjO7uONIXL%ju0t{HX`c?T;w;b-x5KXXH}O=
z>TP1cpT%a(usS37_>_?pGmtEtt7l1Wmw{|s--GEt$&yzn@ms*=tR}+!k{Cdwo$yOZ
z3@7q;!ZAsVV>+|V$P(XmNE#8gL0RJ@F^4rTi=B7Y5=osy<Wj<gxQvX@kzJ;~1||r+
z00N<-B|BtQm$|J7yt2uDlkAWp!7Jod!$^hq{)v)xNIrO|!3gOD9Hh#+LlP&6{E~pj
z-UvbQW*s9`OTADCmi0a18%coc>?lHnhP(mDh=zZMwAxHINYBoc#C1g05DFx5D-n1(
zTlglgvYy>a_=P0yC-OJKQAxm^IT3_VNnk0@kq5%&j5dPw%Ug<4XOSP8O4ee6ussw;
z&RI@a#xO(Arpgl75GN%|$X!nmeulDro5Pfz!voPqsN^!Y=Wr}xgkC}9Yr<!exL#Br
zVT2^uLe8B8`RB?HjfCzXLfyHSXk_-$(-lJYlMvhBu94C4gK7*3yma0h)^KzI8UeKy
zvdU)|Zx6S|=^Tj<=@Sq3iHG;bJHF`?Kc#-xKM+%}LE64yrS(F7)q%W(VH>wNkbC1n
zt2Z9r8wYYve#a>l3=52$J>1IggIsSs*e5@{Hx7uNI)K;+2=fi~<bc4K`THD<cK5-k
zHy*Tl<Kew=!1d$<?z6DKOmwQXN5e2|sJhEbS%&D*CY2Kiq@1BsZ_9wtCk!*E^#8Xy
z_1=rj5D|INsm^e;PF>A%<m%M9Ooq8X8+f&+Q?pSa*+Yk2W0)ZUl%=TM?CFG5bftw)
z{<dr~jnrGskbYXtBG}o8efC-^$6^PW*?R~(M9*GM<R6505#rQaRj^+mx*BCS?<Nqk
z_7g!%&g0O>uwLYSRpBILs#w1!v5UZ^f5UpyFc*v=$i%bI8w)ZBX_EMe$N~Z{0yM0D
z61kXAB#CY!OtA%cjE#^mI36+>yvsp53rp9>9>oZW);cDU{A1*|iLRrm?4B325VQiB
zM&tmYRZ8X&IZQYtiKRr|C%h*K_U{Y6CCHKxatY!RPl%Pol|%rNlOmmk=5(2L)raUU
z+f|{Vi!~BAcA2*qy4fcLw{2SCege^+LjNrjNM8V@fN;Jfz9Yh5bBalXh7qYC$Z`@I
zNdyY#+#(6qpPYS!U6PnY<OQ^Tj#f?V^m6_{ctiJ8Pfst=lcvBuWPeULNvn_th@h}@
zzLEq6{6cJLHti&yCxWCb94d*|h+xexoFoZ2wh#@uP?TuMdqj!}YqV}#$$f{^cgPp5
zx$q{!4U%9i3mXXavixiUnU7%N*-~;9xsMQJ)eF6m$U%Z0!5u{YPWZEw+-sO;k01<^
z#Dhdo;%9T+$q0R%2r_y0CP^^<xU=sjb-Tvqb;Fn%gx+CnT=|RDn)a25f7P@%I@;&|
zqoy5+`tGA-KXHKeC{5INZ~XtPWMhhR95_9i6F9x`|C6S5>-*QO+hn+LSztm}3S_N-
zlThV7x|9xj<Ns&f8>`X=21fqx3V4(#;O90C3{?AvPBHxh&I+FuJ~Mp$u=sEj=I|Ww
z)aQJWfo$Kfa9)8kI8-uMvSkZ6^%h5FhCH`+NZ_i_YCM6`IAmt{;>cL+@fXc0#@u0U
zUV=G(*y6}|RMSBo2b~&uzIkANh`=ihUmd=R?3Z-ytYMo6_%Y*u$qSGC1H!}W%y6e4
z>Gow_p%E@=x{!36nPDu7bkewfULooA7PUBXF1G2fyevipQgribZ<~ZzAC_;sx-vEp
zS^b*84(aXd*G|iX*+LAOCZN4u?}e5ylsq;bhs+^{49qaz@BoL;Ot2*5Hw?%lh*Raa
z|4tCzv92?fe<TX}cjicQYNV!0!id!&LYCjVcnoLI2?0Tb>GB5$bb@5Usl~cUcFc8*
z_y`9}QzK)MqBnc#ydv78@>^y<52*y$$&SE;=Yg{le2i1=92poiwBujeX*AweZ6gCw
z)qkSR*a4B@{sAKftcV_D1~C%fF%ymO@Hydr^z2b{AaP;fyE<kVf9|CjWSlz;hGv20
zUrbkow6u_!$ykp$Ob>CG?lSqkJ~cN9UH@v1g9QK0laKxZbHc-c+A9&AXY>|}f_dM8
z8QRO)ziCD}Vk!dg0Tgh>RLlM1$gE)^UB^}1sKC%kANFM$#1`&2r2f0+RxfQoQf6FW
zMC1vFIa*3(78%CJDkUy3%KSu~%a`v{&Q=ouT%~(CsRf7)tnk5|KD``u>U^%AhzrET
zeBo8mXgYXEh3Ebe7q}<HS8`O<jSVcXMwf=ybeWC2p;89lvZc@NGIwHa8;ALpH8;Tb
zal^2a@k&TO>4zYNcu^jrz5b5pJC}4R-#G;D4=;}4|M5YwFU2?X(0*3bT(8jixdo~4
zx>w<6l=+;5Q_BHx(ktK0`YZ22==qa!%NXR9H-n(RvZquQ3n`!Om0z*XFa{;{lrQ%x
zto16mPK}-#4gg=SSAJ76Di`7H`O_WP=Xm9}WjvQWKCpR3XvJ0Qwp?=AbB$Gj30FP$
z`sabG2Ppp!fk^e&?*p-F^Y?*fYm{k3s5`$8M60Dg1QONE9|Ey{bH?M(Rs0ax9GEjX
zJIa^37wiN4>|J(L=%AXKU){KuJ*%XPB7Aa+?Z!g~)#);DV#yEfGLjFf_%M5Tb*B|N
zXuDJ7qgnEC-ofFtO704n%e`ns=%88$xUSWGW~<3SrIi#-j-be?YUnf@!7|+`MCmdk
za9Nddo2ZoA(<`iYQ#~JM4>Av_--g+9&3Y9cuzzhfKlfI^jyGePEq>erZ|kq#v0@WB
z08T7+QmS7+d!u<wZS7}I@D1!z`}*0F<@*o)?6Z9XPpjx~JK^Fs%dohJV?_kE7=g|S
zV@!um=78-HHZRev8P2;PArB~;1M~9wWC_;hz|vF`Ls;Z^U~8~XDYttdAp;2-YFD_O
zT#e*pn(M91Y+tQuW#@UV1|2RYswcvrV@2kr4y-}P^7IhcXbb2;C-clph-!d?x-H%e
zTVh3JxePD|b!VA4`URi<sb>(}?2qNl&-EFB6=2d(V}|bIVHP&oC@78GLCeV2J*~2`
z3#l0@-31R8>b%unmqC~j<J)dnF%I4?Cc#5$SA;#GI>?(wOB}?Bx`@*!kV;a$9!2=f
zU|EuvcFN0=MEH8*2&otM>V>=Q94s+i9WX9}!k39=|LteUlG&fVInz<IK#xI>)>xUL
z%KF>0hnEX03ap$a%f!4EKDp)P>ec>sMs-q=%od@%vFeQY+OTa#c4GFpp2D?^+p7mG
z`D4ngO16yD&<f*LWAt7tb(PChW7WQlUbw^=xYKrDdgwMoi<wSWuiBR$R(nQA`_cy@
z-+QiZ@JaXEjM$#0XMpU3!hY!Jp%54Y3L11_YLU50uoN^4Q!QS(yTM9aYXkh5_Ry&g
z*cno^RtWq|8#SSVG=oxZvm9i`1D(C72Xu}~9%#o@7m0K-2WRlHJG>>FAzLls9TDbo
z>*Ub7Xuj&A?~11IuITz`|CZ{#R(<qQvUN}9#aZ!4p_Y|R9fjsf!}R+_P&aj;6wN}5
z7nE_0NF!oim}nW1W|<XPpj+!~gQvpGqGJ{>F)~+LP3Ea#d^Hb0I-IW~qq|27M3JD)
zfmknkVP}jliy5xfoq1s=gtPkP8hfok4Bx@MhF=;`i&TWr-TR}bb<|E`E+#em25d9V
z8`p23ni;fb`w#bPQr8CUN!8gFuP?sDiij~cSux4fOm6WBW%p!SC++rrPSgV^E>sz8
zag1feQ#U@jKUH5yg)wP~2$PkZ37E`min2Q#Fj-bZzmpD_EEkLbfU~J|n_7L)sg`PW
z+Z|4|R2ixKxqS(S!lQ^71k#F0@}=#yl3IMe^t~EGz8#9KI}s7eVc@gzq8~My7qO5f
zO5T2mbv*1~z>PyN`2&xOHs%4{s}d1!USV~H9SE3L^i-fc9<8WzMUOgv(o|Y%ybuV=
zPlxf<5}>iHIu3NI6kFl7fxXgRMGUq_<n(D(LhkV}XmwH=msqF6YH4gW1R4XRb_ZG{
zr6G#$a`XTlUy`FPA8aR2?gQx(2R{xfMG`J5OZ5ioYSD|zRQq5%srn)`bI`m%Xwjdq
z{V~U4%nP)tb;sr6E!AXbLNob37^tzC7d3XHEy+gvXg`vT;Vu37(oke}$RGG!S|*(o
z!XuN<wt*)Ha_2@9j>8Egh$@#QtF4jt{9vYK<d{WPQMbwGse${{>ya#%nd(HOJ+J!I
zz`CHhhTn&64EnEQ7B&X=k@jeNS%YQN@G}NhjjzGy!w7PDz8-POXZXGsqV`&?;itmg
z_`jD2`;;?@T<q5ui)6xu>hO+mSm{zwgO$nfSc8V@L_MGzB3Pq++oM>d{oA8N9yKM_
zqvK*lXluiFhdbKT>xGgSXjdCv7tU9GcmrQO%}^!TzG?`4y5DwDPd&Ivfw@*^^Q*%g
zrVBAkpklJsFNlmc_iOE^V`r2J1s)pgl)7;(P12Sb5@fWE{hIp8VEaJiRk)0{Lc?^T
zdfL!vL$UgCh&{9#K9``i8cp7;gVwSTy$xk20<tcqd`1svWE^D>W{(V)F@BNm(n>xm
zxdT#wD&%O46-u{iXbT`TF0nSDM&r#@R%)HkT^lrYb2cSGUTm-Aaz4uC6k1bI7dmAt
zt>k<#%wOlPJWc1o6Z9KpbzNzdi3V1kL+z*$8x9IP3al96e5$sBa*5?ungwcLG`8i2
z{lQ+214qDMa%4G~R#}xBY4&w%efAiRBbkbq2O_NNcUJdkq>Z3#`6{%aR3Ip#h=6r@
zlw@zy+1to=W%+VAtHnw+N85vam$#{d(RQkDV~hGc+D?-LW>Xp;XF=0a9<$d<%ywhI
z=q;5sG~2|ga`U<z>c?cj^Uyx^+)*TIi;FS%o#|Q@qxQwvLx*p-jDc9Z1M@82%j7LI
ztkYL+S4U#(VZq&qlJHDRk#S{<`X<JXHCxosVa)10HDj1Pp09Po>^Q!vN#)d7ydZ12
zb<FHE`8r{8!39m?5IDTTGRR&<w##@38s1&lXDwy)gEQ49!|aJDjUmJBxxT@t)k?le
z-aFh*8GS-lw=%1|LJ$-4b1=?QqO94P@T(7~kA~YL6IP$Lc<1tJtCBkxlu&W)sS;-}
z!k;yk8au*HGAq@&BkZ+U_rDlnr^V-EJhl5z_-&rE>+-del)>viX_KV7N7&hBzM36t
z$IX#aR{+-_1PxaIYm9!r!cnw*RRFbENB$+t;H5hN0~A>9inSAbt52(!V(l^JQT2YT
zUBuU%k#;g)mpfmLBkhaL*yp|*Y1^h5t3u-JNoJy&8fOn#sMSrW)jh0kI0JO?SXw8l
zuE45`X+rotyqCh|WCW%*W?u{<BXu<rOSh|^#KE}(>d`oRlzCA7Ce9vbwyA%_*~8P@
zq|I_GBa4rmGOfy>VPQ)SaP^u*)K?KB(D=`g&P`Tx;t`HiwKm=!o|`IaXoVG1!bgL7
zkR28EfN*YMn7jeZZG&nSr4fwMoRQ+AQD`1KI&Otj^-{c@FhDmVOLufqos75ReCKwl
z{s{;O-{&USV@Gt<Yi%Q)oOg7)h@IQuyrb&Q1be7iq8?7L<Eu+#DXfUGf=LW5E0=n%
z4U|tw66lgA$m-7W5n0hAm_a@C%&&4(Y$CcvNGV92QBMzawFM4BEB-voXEn1=<&3ry
z%A~Ld!Tc5>LxB~@ai*awBsyC#4G7V-4Cn99_E?dBIMHwa*dv1ZXn<zXy8F#zJ{MO9
zvs3EL(RQpa|CIV-G&&wpjbzGK)`)f$+@RO8s%U<IfbV8Q>lBG*2~YG$mNlT@#AiVE
z`4P#!Ez-=1du%&DD$(0ro}Y)_cF{dQpI$`bq_)IMc_p|`)LO(vDQX<g?^L_S*dxtO
zbzqF0kbYd$d6|{Ne6`B9(1)u%mm@AE46z(Y1)6Ja`{l(dBGHZ?2Lp?xO_hgwL96DQ
z(BI=h!2xxCqCL9$pm0xH{RNG&8XQOx6`|I{E7%<ag>;SOU3p#&qEf?Ym?e9Z5E)QG
zR(fy4QW0@I&Vm{Yd-GHemkLf=#sNP=Xj5M%+CzK=U8>($drDCO+-vnCj|cr+jBrdZ
zZ3?VlNV%BRxvP8?L94;P*Uz_%nm@#DMIr6PThVPE_F<cK`1bJwy-u;#+2)D})iBoP
zjmC){O;+XRgS87_Ri4G0Og30qd3`m5<idU{Dp^)G%3MiUb*P-Cu6V9GWEpWHo^j<G
zW2ms|Fw-5#5<L>F<x8R`lFS$vLScTk+quTSut4RFv-x!<Kp~WI?gVg=TS9gK<n+AQ
z&|@@7b{xAoUAy#OvM+P8c@?onV@XS*Oc)Vzi8|x<z(-g{3caS3spBAFMyMag+0nyv
zZmE@0XEGc6fR9j9#@o@CquBDWH3~441decFNro_Kxs_k6CudJyHjE+9rO1*TRGo_o
z4lq5Vbk%cCI8!n*v3C^)>NzLW{_*zE7+qRw>C#?wDE+IHw*Tk0?M_==`tNPq)mcgQ
zaPx>-kz|jpKH^gOlnl6+%!|oSfy^m7zqe;6|1vjU6Iu^r@hOY9qaxc)<`Jy{m~;$d
zF$*!-E3JLzVQp19G;_0;Tl>^ENp?h1K#1cLQ<0Yp!zU-&C{gbZV2di$<O!f$Veuk?
z4c78w))^=-()p!selN;S{$*}{FUpIWExfwmR_klFdilBNfY=TztW##A)w>i(?lG}m
zPMHbhi%$b=7neJ0mxH9cYUF0Be@?KI1|Kj4sMb6}3!|_@B~P>y%?7n>B6Bi1APde$
zD+#%=ic3B2i>ou&a_5|pTwsL^s_7~g9U>(Ko_>%XfjOznGBN;?Q38;XGXEBHuT_Tq
z-MfF&Rc1l1Su8p$Sb26kj9#ioU`%Ss1xwS%=EK#IWU(a5z7T7?z=}=sSFl@4LTy<2
zMc^d^Hzpm_C^EB*N$9MVYQZG?FIa(b$#!f?UXpBb9JOsM%ab*90~>v=IK@deZI<OR
za%V`(O4Y_>yJ^@2j3#SA6!ud~5_jQ#93Ds&W2$9>8b8@ya9M*)vW-@5p+hH8yT&aI
zR`?x8TB20DfD1(8OZ>M?jqa^HA?mC|#;u9H?q}kmgW772T<xNDYpQy4vYmNpUNCgf
zF`cbF#q)~{S>LcXjrk=w{*Lx3!us-x$|*Y0r>H^~g-EL!BXrPlr$|c#oVzVmT`|QT
zfyMNbDfY13>?Du0x79{^xZYM9;Sb>d(7Y$%u1Lmh^#NbDi}P*uVN`dtvT^G{brNWK
zhtKl_=+?vM1vD8$#{?YO-+DOTZ`XJu{tv~{mOs6M7`40AvJ@8Q-Rg=Iti|2xE>g>C
zPg$Y2^zk6pw#rsPQ>eDn@^2Tnm5{3zr~MV$QEb$9iNrX$r}(uC=|cYrEn3v~DOk5H
zYC<Y|iWapjRc|ylr((FZs9RF)wCeT(Z%DVbS9(K=@F3J>SjKvMRh7KBXOlC|+uF<f
zR1-<P?FcVOp#{;pTq|UYQEQCuNou6^2<pmuFs&=sJfv}<QCDRFS}Z1`u12yQ$tvt&
z+^xGuc#OMsH;jWrnsFpN#@*UHkSgL?WX8$-tu0V@PPOA#mI*z#;l6b#M8D1yrp~tQ
z>`amUtV1$$*{F#HPHwyDOfwR-F`_^rwD;$ib|;jTW>3jDY;nPz8CYvV95#)@m~9=F
zus}zT8sf|cE9aln^3pUr%{-xYrr9&ABLev7D3y*P0<;s)zfl|21EQ9l0Nv{f8P3SP
zgg5fq?4YKIp8mDjLIzzWGlFfqn4&<;jOf$|k^+g$9I^LGtdi;$GfhNzqE-vpdb&^0
z@!Z4&DVU@anNA{EClVYBz{o7c9$oKHY<{CYl+x5AE(cFC^Z9c`{WaFACXitXJ7N3o
zF0xM8UAFJeB6TF)?&o{$v^th<$InM=$FK~<G=w=%ZG-o(cel%dl-5ZA@ySo`%6MLn
z?+y)JwZkgp4Q8IZ!Z=i}=49AY%yP9Ug9%loewtxtCDqpWLRajtR_b@1>EZt-Q8fDs
ztBAEoa1@gKCQ<z(gAHS%8Zyn^oJfr^SclY**+hrG$q*yQF|54?k4&?3)=@L))zmWi
zo0?ePbeBfNg{K-oO>E$n7${iH5tNft9Od7F3Uc6=I35A%sx$(+YMJ;=g*s=t&CWx8
zH{G7V*Vq~Q>pc0&P}k0|V+Ypk_h~k0tX|!(nr7Guaj`V_ZAr8O0!mM;_SgvTfTKO}
zx#KhJ6w_=}Q8NM5sOHSn8{myI?eWn#c%m^;bJuCPbIH?`q?%{i@sRe7nNX=keKyk`
zi=7jBmOVc0D4u&~6ip}7iHeC~JQO7hT~FMlv_|s2BdY8yd(<@eEyi$Sa!*gXcsB|s
z$eJ6BqcBdK9%Yq!$^vzFnR@vwJA<wDsk7{qDOe7Zn0}L*{Y>8;3(I&ItG|Fnwc?bs
zW71%M7F5e-*`unlGo{xIq?b!f!kV8YMrgmQ6Lq|T2reEY<7dZ2Cd*{IUEo$e5+SR)
zZg(WY%Fa&IaxOBo0IMe+A&)=9+Sb`YunbD}hA~vv=OD%cn8Z%VirMZeAD7k(V%4B5
zJ31y`$gt50#yhIXEzQkW*;(lBe6?A=EboiVA<JbS{2$sZkC!oTw4yT4043QGAQmCn
zP)|!GnZ7&d%Tb~a<f+fVWh9nHCN(n;`<$wxW8Jy^K&?ucZO2_yhmYB@$c+a&Evuh(
z#1(B)zH0<~==(qy`!?%MucRv=uYS?}KA0`X6+Q8O_sdM@FB?>(<%8Mkf!TI6GWYy!
zd(=p5=nN3bIN;jWo|y$3_E_rFY<sjBP|<U6OPn`at1Y+7J-LE_hZ-rKGUAU8_7q0+
zL5-B4CVzn3-6E#^Mk|nb1_31-i^ADLxZ>xwSXe85o{d(rdKM(iTJ?^69apF4*jao{
zn+t*`RN-9K|9ExvT;^1~+BMgnJTD6lhHlpefR(YpjXCnRFgGdG|3z0$hD)YCX^p60
znv<ad+4Pj5CS>D#s#m$$b~;~GWJb4^gmN%0XW+s+7t<Am>gjB>K%sii`3jqdldw{y
z&9gT{?FZ)BF<753%(G*wnQkm81=bOt;|f6r=<*Tf2ouz5@g4BVJauzV%Ax-XYqzht
zM=l;Y#awdKQFFOI?F<Ozn+syCyAK(Ajo#F;Gg6&Ss#;eK-Dj8wBVd`=H2WN;pu;L)
zGb(UuZeoQKL%X~}J}atVMVBeh&s*Dltv<;+?d!5cL($SR&;Pwu0j(Ou?Zco9)j8je
zFx%B<^X<tAM`J}?^@{iNqp_a2ipR^%Q?nM>ANU%(RD2FjHNNNNAdRKs7%sC0vU|4j
z3N<w}!aHk(h%2q#X1i?Wv>DLYRq4~bHhc`J5v7)euw}`r1v*|{Ib$lX1VwFf>?{>~
z!7sd(3Olzm#r<~&ITIYumUH}?BuN;4u`+gU$0^qv^#HUA0IlD4#Rh%Tu(ZT3+j&gy
z<8-uBn<S2cXsnWh7Ty$7aqk8AT^g&@vW52K>UQxYlyT}Q$Vg|6Rk?Dk<Kay!J$=PX
zD2A1c&x4ff&N1Dr-7$YOjdoE(TWgiQvrq;~V<{3>Y;q}dEiG5yFSH|L+E~Zvde=!0
zh9=3bI+b*`9Xrcwkpl|QOHk<Pz*FFY!mc*mJQUxl)7J|@ReiQSHl_gIwCr)4%??{H
z2^y0Z3e>@~ae$Yo56`wQ9@e4_UJ=z71I{w3YrUAQiWlK2II6B+#N6Ah9$sWmMyLOA
z5%x>F^5xod_?nk%PvvV%uAN<-9pksZaFy6g*)is7XbIGp`Wk;EM54U{>S*4*aKIAr
z5Z=AO{aZFw5`8hxqJqwSF~7&?ezA#u9&r01!5XQ-mA%;1$I2wp(-CoRA${q1&~Kk~
zZF3T=s(LcKbUdIfR4*ZIy}0L9)6~_pxv5IsznJk?srMG!XPJjp%sKYhjDwbeA9@Ae
zju!3nmLc49Lfi;?2sgpx^)3vTPc??R@*I1HX{eu{W6w$kfY>$omqfgEPc`)xmoH)A
zF*#0Q)_K_~xWpb(U4zw{ggkM>M_kHuSaX^+!iP5UQgq)F6}OeXW>te7AoCl4px3TF
z*tcDK&@&-wh%y4ZXQjFEuq)&BKGn1YMSfVlw!}`XzOmclT~l}>PnoqiBxQ&q_tn@?
z90so7<ma@IMXzMlhd9acYW!<McZ|{y2d5kv(GVtU6kLb~6Ix1FI;9Xs7Gqtg%JZ-~
z0_wIrd(=5qVqI$2Ky|?3u$K@HdvT4HZeA7Rx*A$TdS@_Tt76m_dG@ej6w7^+C|%s+
zWs>5ArL2&JYT;5lZAhZjT8~SXMF?3IzTA+iwl8JJzh5;k#qv*7hnCu-FUUV-!2nH?
zs)Dr+NyMnDDzpp+z0zv14msW_KvWeicP~B>J+3E=K(0D@$S>k8;&x+=_D<<&?Q@IH
zwNreolh@?4udY;k^X)P7VLTem%ICnq?fB7lne`gUmMDw0%l|{Shza0G`vWh?RCE5o
ztR-Jr%j|@_e9IVvHXGCES~pTbiYqOB2v{JFVFyN|hWTTp2eT2F_iLEW&TDy-x^kI4
zrXQH`V?T0Ct?plDPvbPIV;OU-O+}uM>EEX2p06{vo{u(aQ+v<1lQAt1pKp(-Mo+R8
zu!=i<=EZtp0g+c?#dKY1#m>-FWiYQqEb_KWuN3GVvkdW-G8Q`r(^qP&P|O<H{pbn%
z%DzOesO@BoH;*K`hQZRdtxG!q`)T6P=d=P<zMS(GzVCFuTbA3Sd>KjV@NzpzzQ1z6
z;|rYhIR*BF;O=sO%PvuqFeB-?Z3Xr<e&5dHs(Y0^dg{*Oxt==WIxCJajjnlL7F!UF
zh<RTg_YYVgq`i4xi8`+ki>O4E79x@*YI~tQJ+DL@dRz$n0*SPa`&%v1Mf4~Q4skY%
z1!T4OkNDjgYTkDWk0Bc+M9JJ8yWKI6boJe_YQSoH#<0eo?)Nn7r~_@!0d?VOdzg>!
zYggOj<@>?atTxH&x2x^7vr4qD2<n;lAMXKhKUNp`vrPt_y%9KD<)GxWzupmY-R<58
zb<rBuqX<>C#vY$frThH8Z3eZv<=Y{|-rWE8dGvW8QN#Ds)dLygQ4xJk^kviD5@_(r
z?<<Lp*T5ghP+zXG$8Ly-(s)qs!FXn{mIH~}TP&}I2(|ejw+UR@KiHP(-@-A!{DJ+0
zZOr}9I%B($u-A&>tb%N5&=%+0;_Mf6C3VwUdq~oQonmdu>SaE7I_Ov*PFzEGKpN+r
zQome_Sn+*qtvzgT>_F|vl2Yl?I4?$p6*0qOR7w%*I7a0cF-2n3b!4urbFH~}CcQq)
z^%^xg1sQYu0Mif!<PxU0-Wz0VEofz9Mr^bi%!f?>?Uj42ht$a;Ebx34eF3Xiy*lp#
z&QR*rh70W2pxiri;0T~}L)~)$Qo#2wFR;`3e(uW)n7O|6Je9DHGv}kv<*u_!d@Lf5
zU8I-N6BmK}VKw+-tc}Ae`(it7{PfJAxz15=61}iDDmum;(DVki^I|(;D!$@<rnwHA
z=d@2+Q4Ai_o$L#pY#{V$_UXm!bXwGa_4bJ983EZwUBKSk$7L!HbgRl<uaC*st!KOk
zm0E9)9LZ{}?^Olmyr9NBB2&ceZHA%VTyIBmRuSB0WQixccD=X96Z>1EG^0XjTxKPg
z_;0##FAy3{iS!i1piJ{3tEj@f=*+xzlDF>6ybC4o!ZY&<M@Xh?;LK>0%U?3bgd8)9
z)Qu(f(AhJJ#0$e#BM^gHr~?JPUWqE9*6RJ1QA~b|Tnbvx4A|$>_b)XF?k~JqVn+?5
z&NX<KT1=cgm!QSuPq$S+l%Rh0tI?O(Dftn>z}@koX^CUF^r@}2H4*uNyNxaUB0F0n
zmu#!MgrwViR#~0N7g(TV*X&LT{i(Ahl*y{C;5EC|1DDwG=cD-LmLCSw6?!J?p4X7;
zj7bz8WlUrJ2UcmxGNO%E@nx)Cmv!{<YUrkE$tv<v?q={k{ZczFS*l;gs~0Xi%xY;J
zHgyAih6G}1TtpE!i_@}P-E^rvY(Pz<+nsdQsG2XeFSxigNhr^%*JVCbDmRk-Kc<U^
zMbfo$>ktqUFAPU@h3M?~K274CCiA@}S#A?=P-fIYb=hSYU?&(iM<VI?Nc|hCI8h1I
z_#^Nc_hdNz45^QGwF{bPNRwl8Ml->k1=}3bK0~e!J^6R(5LTK)j)~1sVC5EtRT$x2
z25M8Qtf7ffAr(dj{L{&`LyqOjV041GV@|rMxNr^9^>b()I;0z4L6>iJi#z_Iyyw-`
zsZsH{=tXaU#$$=<l2UX<qS{_+XL23rwNiWZ$mB$;UzKO*K3172eTRmHiSRw1tgH?8
z*jdPC71`P%mtqkhiXW4Fjh-&?5M^?B+w8~MRLKT=-bGMRV?m?Y#qNbE&%q)_*6eb}
zkkDYWc3Z|QglkrNk6ckDtFsI-39}{2(TAAe^y47%R*VYW$RvqT6E|`(Ek+LHORaQH
z+T{uys#jZ>w0tX%je#>C-m1_SHDqc?)H@>u!`Iqf=bk>g&FHmXHFi=Oi`2)aAkZmN
z+T#v#+p1SL+R+J}J>Xv>>Ty9+L1@>$UwyX`ChS)km+P-hm)p}xHC=8`NJUV_qtTL1
zi0L4^QGuK&?jjidSi@c`h-zNuvEj8W_5J1csD*hlU$=ADB}kbztX|6tdY<6d3M}81
zJ-y$PWAP&lW!7$gv!5<d5&Y4~J;kcHj8mfubwe5SJgOcjvkSOW@>Q9=WEhh;=sRn#
zm7UM0p0K)&N>*z&VVLBrZJX?+nC@?GvL{6!IN%t@8nVYH)u|#kv)~<3b2i&CLyrIu
zK9|?X;f2debx@U2#^P|xX6*ZT_3&oav3T`|&Gz^SC64ICRKVC)xe?`_c|6C7kVj!g
zu|HGAU4gqTQ_Xh1)?Q(cu*;In0+_wOK~-PD%Gs(~q_9=}hOdpS!tzopz8J{GWsb!7
zMKYi5Nagwo(?cvcNjV-O*<3u^0l|@kKq~W1lhDhx7?m{;cmq=%>FHCdmX+JX20eSy
zx!d8={n=axn_LH*-2WAu(lfB(F&6Ew(*2hHM{LObf5E2I#ir&AY#Nf(Iafmb0=4l<
zj@sMQU01^2Hg(`iP6FE0U!AXkTkKKk&u5C|r788i(HA#TtmB$!6750rc|#R%v7>!_
z3n3z(kLm4Ni_?L4r@MHkyLhL&ctv+{X^lO-9pJWnFJDM`y-BpEw*$!Y7Nn`z>Fhw|
zzjU@^pL%!;f__B3vBkb)%_&D9gD{jEum3Xfr&1aZyW_{h-SMs?-3SShc_U@9@H=)q
z%NBfK(F!_NxgOKvCiPf_9lg5AGDf&BHnv7In4u5(AC4Z6W^^`ii!pgogH{0L=S_#j
zhXin$6`StAqgJK}pci-Ja1+7trq$vcHFB$SEiVT7<yNcsfZPeSm`NiSf@2+hE2ZK*
zkUtP#jC`<XuwvrrB%Yg!UcnS*yS@|Q-D^eXhw>O;p~cUQa!Z0U6?c)@StYugrHEI9
zIdKQJDUQxDm=0OXHBsbP*o-cg<3l1bi98crEUO%U7La3r9OavdvCnSNN-Pbc>Rw$E
z-fax;#{ja&ipG(4Em7i)_2Qm4R`!n7|C%YF^}0Dsd=`IlmsCmEZI08zjOh;eX7`DE
z>+UcMJLAt@3ro9?9Z*3)b$A8AfGmgF>=>3i#iJ~v8I~^wQh{~6f7<}zhM*o_n4*ui
zH6i0G)QPK^WBDp@jXh@+OP(B6MaAOxY3$z|+2bn%R{Tj-at$+YpSt53dl75ITeR?v
zKB)#*+DXxtvsR4POqUT0o8k|uWtH~Og3NtBSu)nNx(Fvoh8RvP5PCQ|Luwp!X_}B(
zFHUg1$8+NvsJ__(<b-VXVx>JHD_gGPV-jTN$X)(CF%PK6C=*H&#o4H%yL+R@<K=R`
z<PJNk;;+Snb6jOzYY&ef))h5)n~^b=!-}&;Z8PS|lY81)kgBI6mZ<*M*|XG>*V@qw
z3ee!lH3efzJwK;82H9J$mvkT-BHJQo!?-g6*>$u_D#)ZXhjpp&>sX^KwfZ`HRCJ4T
z5&FoWcLux52Ll}5uI{`}UuA2)&YqASfz2i)Y~lB64)NBU-uU)V9bYZuHn5kNazgsC
zb;`OPM@zg)z21(^kI(n0IX3>h|8Oyfp^B|94Vx{V?nY)#hSY@8NMk&XJ2wo_oy_LA
zWVQQxPBt@~W9?3>OWdNNa_4JobCtO8d*bk;C!X0G_XleB8X;~2A(8WN!8i*?B2dO9
zo@uLq*Z5Sl7P48zlrOXDd`&)|npwq-!#ro_8x)=F{7LN&#}Ia&8Yn@IJ%^_`9wnJ!
z@gpC5IqpJ{F38b|Gb?2J?;g6>%3OesU63PjLqAG5{?xzSYRZKto}eUk%<bjih$!?<
zg#27954;@Q;oIS}_Q$>$dqz$#!o!cNVK>+#<Q5?2)(Wd}#6Hv|ywpq?(XR4tz#!^W
z8*Z@2&WY_3Wr}Nd*s;+JElh+e_K@?$14bl`Ai;!gxk6rOWgT&!*Ko2Y;$^!5vIPai
zN-L?zwPAIeiP)^T53R^39JTSS0oTuNPHb)pBd({T8`o27;z4dKtp!IAn0UC==fINQ
zCmuzNkP7Qy$o^0U;^QCLlOPH#wvxx~3)Ox%-R-0dF_p`#tZc~V7%5taCv~ekZnP5?
zBw7YOHS4&kPxf4zBp;W9pjFm>vq_#kam811BiX>}T)Ew%yPe#iKEILMhV41bK{*~W
zCpV;q-x5qqye1}e^caiDqcb`DS??(I$sNgkqGwdq`km*NeL2oW8mJTHKG~z`aFBBg
zinNxQC<#Q}NmL)^LQPMRBq-vKqabq<46&dAO8y$|Eekr6qMTVE(L`rw;znIfGbu~^
z{dC=ILmt2qtu%>q7N&xZ_MEerlPV(Qxh02JB>dY0ZX!9}@_0zpeMky8$#@l=(*}w9
zx5aq%+jXxp)?}h34#?!g0p~F)$vzygwzU~DY|Y=vnt`jV;Bm9YdrpWUPfk+%YEZ#R
z>R^pM&Mub4XFZFLnsSppQ4P7t4w|KE+)dnCDmAQaM+BZ|jLDoha%WS<HzlfvEK@Q9
zt|Ql!sOyKDk`<H_O}*$(j9_pwXrb4{plsfFV-e9pBlMc6P>}+^o2I$bq!?~Q8P!gc
z5@84LonUUE*9$$7n~05K`6}UDgufCX^2g{##yc~tWpsTKUFk_WG9pPo$I~-Or<91-
zXeIX?LfpBd+fFHSl-j^p1U&VCy5MGefcV8_>fqE|!Q2edlL}l?qIL_Pw4Qe3dOoHk
zGF#lcttqKBmPehGtW(a-a%r2x!NJ9Sr(CKf+bdSoU@Xtv#3uT$<-%xUi_cuWh+)wr
zwFytMbC1PLN#t~jhufSflUmLlCr8p#o5Tn1)bE(2rO}<WsYlh+ZFY2ZXOQWoXGN-x
zYRac}8vbqEH}nz$L)bjka&0>}ek4*ZQ0O^0<w%ZnC^9i6PuS_|fGPFDkgGLAuI|H-
z0<oM|S;xk=O-W-24o9+5&*aF|Tv3~=tgOSKntCobr{=3~wsAO7EVen%xp07BCAK=6
zO)xytE)3C3aN~OPQ;!`8(at@obG#fJ`>3}xmM*c}tUwfcIx5y*39Vx@kSx=$pT$f=
zg&Xp02#$?Lzt}d(8fyBfeVojRq8F1jt=ux;$vXC;@M&FdwtBgiO-H3VT8mX)XBl`8
zt=;m#mUo(GrnQNWW0O?^JnlmZf75d^yvdWcFJ89TngwaiS{KRGNvmKUF{$*VNUPLN
z5S?E|zK~ldYGj6Jh-up`voN|R?$n`{px&VFyoKk!8?k0Fr=9BUy^18QJ)ozo7L>G(
zHosZyf<Mw{E~j-E<ZIoR#;pP8VyBtb*`qzvleFdRj4z3n@orz~sk&gDQ<6$al2?*x
zd3;OHB;Qkh>6m;TrAkqXLI+K@%HDibf!2PM$RN{e)ZMq)k!R~_oPoGxNv{X2Bb~Zd
zBSI?~h;8UrM;&PiP2XLprCduydXswZR;JHB%i1m%STwEEn;b`!aIZOFWoIRNNRiqu
zdecF%HG=!+HC4SyYx`jeT_r6KidkV-8@XerCW3_aUJA-nxpbztxLUZ=y5$ZBZ(q`S
zO6*dtlVW_^Wb!%VHDtUoy#7iiz8-Z@uSjmxzpc-xW7HpRv!gerpOUk7M<Jwl<G04K
zCSsI<eZqxEiKd@+>7^6hT73(|jED#iBERnhy))3qNzBot3?J`8%_<b7jSy>uGcr_u
z9rVjE{M&=>dXkYL_PW+68Rc3Yytw~XkCGXcy%I#+zb)CNVg`y;oPdHz4f%R#B&Hr@
zd&fgvx<nH4?5LAC6!4GjEg~f<PcJJ|Q)Ojywb;~36@EKz#9Bx!N+LqG&xugqN|Ftn
zo)RNlW36ymXf<z0i7xxqQm3PlUD2MXW*n3QPx%x29MU!>PX}8ay~uZm>B_p2)SAXZ
zAgonQw{vupq~5yS&NZ{uh&$|wBa$qmLpIP>N7U{idiXBrY4xi34tr!|(2`9FE<%s)
z(+;ZoJJ`;|s$bl}MbJd`=R52bIQCM1VqXXFM}J}`1ZQ?@8oT0{c~l+yi5(Mj%rZ{N
zcBH}2lV;AfD6pGxOhs(BQ@Nlzd%JzBnWfsdv)PGKUEA>#MX4b_wafUr`KR`E)d#s7
zA)CXba(yD_$iZ1Uml2RFMv<&eGU~J1p(LZuTE$0UDWaoo-m{L!O0}N2`}Pg<tdrhh
zyLr|r?a0@nBXM0tfC|{@2h~k?a+ueu9=y{YTU5T^Cz~YwyxA;C09!6En6t`xV4j36
zYuc>xMtv(BAUb&*j|ACS-2E4nrG8d1UKyhLPflC2TGg1l@RG4#xC<9MS9$KTCk*Bd
z(%X5^Syl!;%Gq`5@w@EU>`s{|B~}?%%1bl|Aj}q7!iQ9}p2JSJeijwurD7HJ^gNeA
zWp%24_4bG?Ru*QuepIE*I_&>cT)9F~0oIC*?p^`3@*Dl$Z9o_9+$42Ty&VZzw${V9
zR<)}hS!-2?NX0}UeGGAKlC$u*Vllg2S$Ff$AAn^YZs!okncuoQxEBv`tax5v;b@I%
zhYbJS2YCayGx8ZFAfy-k23}h5i1*~UCjPV}#}sJLLj?O7Vv_r|>PNq~dee7qqufEc
zR8K#NyAmQ%>_cJ7+?E(m`n@zQwCwaL^3oJ(IiLtwA({=sDEc2&ro#Wr9#h?*?+;3e
z$U%0wBL~?HjvR=Alns%bt5s%pGM8!C%t~&<c7AHbx#g%6o<}762>Q{%XkI)PpzA1^
zx6hF#nvQDGCls1!Mdm2@pMjNndSS<TOm~g?sCJK9E;Hw!ba#9wLK?%+HkzjMPd0}M
zF@A-2OH1thQ~Sd7Mnwza0!!ID;C!)~vctX{Yk$`cdnb3kGaBqW@&3QsKxU;%xyOE*
zuMh6AZ>mNs6`=yR%TtoFUN~S-NOAl+fK{qYzVsjTThPF5ZQs^-YZ5+s>>{_alRH}H
zUf2V|$xXB-u@GJ03~@mNt@#zDKJC;8_p%+~5lpR3WyUSYb$w>Mr3I;nb~#nDX1M~l
zkR3I>udz<@`T+*6X}1tRXV2b#VZ6Tj#G5~Sr{t8{9g~}Vp_6@~+b}+lfpFZYhrN(t
z2Yp*dTR~A&UM8zwgbX_38br&>%uv5qJThEt8QP^$VQqbdcTzNQGgZf&)ntAIwspV}
zVjT}PtbPn=eng^Z9F4l09}>}<@9{rN7K{~E#0X4p7f4C6=s`UT=9eV<ivsgJFeUNs
zQsopLNc8uX9dKR)wbY~G{3e-)SL=CrHS<t6IlDn<$RFLqhPW*Caw7--<?3&ZcKStU
zcQMbyD~wz@t$*wQr>EKHh2P#Md0t_7MRiXpU>MJ$z>MA>sh4zwY*L(}0&hM@Z^qPG
zbz~=ZIQahD{f^jW4-1i^D7PreEs9bU#c}sKc^5}WC)7&kt7aDq)(O?J%bqx(LM%L?
z8SPU}sN=ir1oOaiq4(K~Ead*&y|k-TmCo0L@^wrd-fNE?$=axI&h2+@&T*|r9{<xA
zPC2Fq?BjmTF*RYIJvx{YQ6rx65Y`U*m~!m7wfpRs{M_QXtJxms`yyG*Yhqc>QCBvB
zRe{>mWJj;(zJlX2Ho7A4ylFxhT}dpDTF#9xcFTTGL|#X!>qVX2xz5El@^T%gK4A18
zXq-$|(GPPcELp95*q%O_bwuC$I<7BuQ;8~{p;UPM!76BTeM`>hKF?RrKWxXw=37P>
z&)-^Mvad@%CL2?kKQyGqKOVNDs;PmT=rS!WZfRl9Hg36<$P(flA{d{alJ#6TZhR4t
zG2=ec(5N3H3rphR-UqclZ(-MhPg9lxgM?!xi*OnzPviEXh<Cd?DVaZBN$=^#f9;pO
z=0+=(UvSe;gE^$pUH;d8_43bP^?voSe08YEN9?hSGVv+;DtuPvIzBx*?nsrVQpz|H
z1!H}2a7VW10y1{wXvee%Sw{0|ysCbLi@x#d$w%ONs(SwsxSp%}H}h0<r#v99DR5H!
zt!^|{bDe}Vug9X&2Rt6qjPcX@Y&{*_D14BSORi%b>u9e80Ope@5x()5X~^Cb-zUi;
zd=slbL5(oitY|s?$~@k~nomyjh~p=xdJy|G*dvag)b@y@L|t)|s4I>Vb;VJlAS2v$
z#&k47<b)?_RQ=EOt+Ypf&Z%s&dgJF%CRzQ@&+X9}&^{AU%I8RVh1DjHMr(-@k{lKJ
zfFxP$T!eb^C)ven$uGEO%Gwf%NtL@+-yiEKervb7;TK#Se5*@6@(T=~RQ2{RxHOfj
zf{*gVQHEOhC`a;H>c&U$k>#shkJ|Z2!>5m8-gT&`7JKyIV`6^r=h}5fN4&~yVL2^S
zn_BGn=}C^xf)5?po}FrRqy|Azbhg*fi8^}J=*U&Cw%DaeL-J$x0#rfyV>|<MNYy=N
zk4(yAl>Wd{-t-mg4=lo0q1RPC!T)qn%Z<$SKXt0#J%+P7pgwubF63*$<95a+F^*wf
z$Q0KIL!ORGCojOnbthm)Yo&doA(#24O>is*<867e3LJs_c5)K8qV#<GV^I?GtzPOl
zf55x^ggW*(CsZet?+JTsO28E)xlx^zib$pEx1Y#Eyk#XuV5`yK3)FD?MZ8VqsFhFH
zi@Bq=?+H7euUDL}zdgaCa!`frXAT}zDf{ijWtHfpLuTmm9ag~(%R0jQaojkF0O?v@
z5vI995~M1TEIr5fiP-Ej_n$G<bp_S5-;OV$-Xer#k(_H5=~`SD(K@3tvqZb10A8sX
zsVQEWS1LXrw-;Cw=G)FMa+LTh+*@Zobg3*>@lV?EgB>vX{W==8tJ~E1Pcp7HRposB
z>`8mf)HWW+VqL+_X<QSFjjh3e=QYXl4CgAVs#E*_+}HGwt-gKI9yOHGQWUk6iiM~<
zplr%hNJ61n@{~Pof&*Ez@0#48|2pvW5_)_h$I?-=`uS5lL)5I^d&-_YCf2>EljUj~
z5wWew8ZoE-)<QM=X-+hlgEKhKnbFDmVRiaWxKEDhF18jjs#H(#w$_S4aYu$_^{A(<
zb$Cl<HPeW1Ma98ItJcSw+Czb@ZL$~@SP{H-#iPpBc46;I>kvN``ZO;YEma>qZI2ne
z-O*4zEV!;-#kR7lH>f$S=$8hyy49Xka@aZ2;uwmTn(+%Ybfp=7eHpLgpsft`I$q!D
zUI)KErsq2N^|9Vbg>fD4fpd?djq9tteZKSzEh~<w6Rq~B$RnA3XNwf<SHqss2fs6)
zK?L&Edgtq>&oF`V)zi<|iM~H~t9PHV7n(&X?pb@tER4s>&?@IavHj-p|EI3E0f?)(
z`iJjb2H3sXBulb^1(sx!Y?4hvlo(=AQ8A)oMS}(vBR1GzB@q?PYSB``geXQdL@|RF
z6_vKMqQ#00T5Qo`i;9&#<*Bp}RaCTSX}cO!s%YQeIWu?J=>J^}cg}pAIdkUBnKR#}
zb|WK@`L`Lh_XW=X8z84rtxUw<%H_4cbe-yP7oj9kmHRih{Pi!<xO(K<zl6@}kqL)g
zi_@U}o#NnrEY3M|%+lG|qq*{w!-(yammdbbPI;&LJagC;+JKf`NkjZft&>5-Uj|7x
zGB^Ly&)a|<j(FUmW}0-I`maKIdVogA6MJ0w7$(>DpeO2=Kj}dh{qmI_Fd<(?dR*S(
zA+;iwSIA`o=E0F1v^$n6a_W8YgYB7c@<x-ve0x>c7%Wh8U{o9Ghs;0l%aUiI+5Ph1
zv*?90<!_%wCsQc@{VXP?CZb*2hdk+xRAjMz024S1qL+mOSv;_XPcF$TCd1fChL3h)
ze$jSfU}rn28;+vFwKgJ=wpq}$lkq_(FQ^E#dfb7nf5`OVJbCYPu9G3te|!$TVXOT2
zb1)Z1<&s~ywkywtKIZVl{pc0~q^;;jDf{Kme&tHLte=b~_GV8SnS`^2o3_7j!nPnf
z60Dv^6Sf60nu<G!TN%CS*w~itCwENTx+u1`pj_JP^5URlW3Ov*PCXfDyK%|va%_NB
zxa%>dh>6cJ!nqWK1cT{p>*dsE^?oyTFOPYm@u=qu_c4AgR*zu9{wUfX@yd^SU1=Wj
zpUoG(>CZz4HprakU0JnVYP}~9gF&tl;?{!$dZXQ>y6*{9@{qTc)e$q4_R$*;LA}x5
zj7%&sivmo}@aC7OZhG)2#LR$)3OtT2QGPvCKzg+HN-^Iv%TtPCTc&V>BtH25Pyz0e
z;{m*(k?7rj3P}_P4e>b!aA-Fr<hJopg`c*xM4S6V6{!8V;)AO&lWAA&D%}AfZgiE@
z0027d43Wy?xG_t3(t()f!zr8)(@qDj!-p#5bH8?_U8-@AKe6kmgsh?->cGlbG*t%)
zBLoJ@{7QHdV(fR(7mGZEnUH8N_E3jB_BXD@$8d5r8GyL4$)U6C<k4_S<D1LbDT)qM
z$hP0W%Bzrn_zjNaR>(Ph*nM6hPw#`x*CcQ0gD&WmU40l3^~!(uVT(<#-1GwPQ@`j1
z7{~{v<!vuu*P>HC^@8h+W4O25A?nQ^(q`@+-qWo(e-M2gDunuA`hgh-s%7v+sI*@B
ztruNsC-!Dzs)|PQM$-hYGqro`rU@DpT}eBiuC#+O)x@19qiLFy554G0NotDtkqVa>
zD(p=S@^>%d5}b0HZTX&4R39+$(X>nvBY8}6u#%UJsZTptWdH$pZwKp&X+ea2>x(cG
zrQ*V=?&C5J4$>W5tXL0r;|LJO$%KJ@?c#xOx5Jn7uqdx~VAMr+g!)n0C*&=^buCY&
zI8dPF)j$lPv`uCNKlfYL;=qAP>hZ4-Wns+S@O=UY^9;wqLfd_K`nS-8n3FpGCD-a@
z!)o=Cx7F`%fiZz9C@qW5w-~z{<gG7Z^a;bC*92%yXSa`+m{}q3ZsG2mO1-;1I#k=;
zp@(Xu@pq%+A+*>Ju-m89_-?Fosrd+F_qa3G31%wgzB>||<4LhuPDQf}12I)Aat~wH
zL8A-@vDXYroZCV;czDb2Tx~FZWMlvxWu;7enLqjJa}9l(Wy{MD+BW&h%MjW&>3YSr
z1tXM8UV*ynk~h8rVeOKSyn-kb@=ve0j&n3l$;4M(=ceT8(RV;Ek+5{#)gZ5W6)UU_
zA->`l#N9P*uc?t!Ed*i))VOPAwr1ewu>+&HTc8r>TMH+hhxQv;$Ak<sc_9o5GDmc~
zylX}n{f@ZZRk%RcG06^9;Jdxwqi(@;&W{*(S4H`AcUXD{T`!~F-yB4V!_xU0+EI%<
z<~3LHxo#@P4q7H47BWAtAH9^10PilhJAPE<^^g#Zlr9v5CPq4>f>4O-TXP(FYWPA?
zTIIplFkz2-^ImfWQ{2(I+6|je*VS&nJo1|B6-<`?`VTM|%H)TC07+%i`#NlwGMW22
z`sg~j?RD6fW%7pCUB~7@gn9*zzf6kO_}TL>cMJ=o4_iLOglLJUogZW(+%X<vGc%^;
z+%Yc4UxykRmzhJZ(;Rn9%4>&Q$N8PYXb@(p=oaJnFk`9oD8wC1*%ICWY3%osZ4_1i
z`@JNXyFFN0B*WT?Q&t`=Yx|okZKmvR&81pU==OI|lPwYM5*$<E9&LYjp;d>ydD%gn
zG|{Q=^wD{0X8xT48hJTtNk;CNoRtx{snJt0@2sdHt5Q4T!eR744B+$!W@M>J+=-Rk
zQjM@i*CI8@J5h@Z9bvavs7~jCNZZ?)aD(BBq(Rg$%og&me{?OV1(W%%4ZZbRvuG>4
z)n;vsX0T(Ql^Hx8`<<!>g?s)T#`@rXXToxCvt!wEaEVp5j->GvL(&<2l|2W;H9+1W
zjDbQ`sWJHB8^vEN<iGyM<-4E&GdK#d4z&@7^+ZVN(TL%_3W!&cw@r6(I0kQ+@vd#9
zDqw<jC|6u@kG;T8*)XYZdgb~*;ZS=O-7|fe2zM^hbKfeoL0<DGSL%gtLM`)bP#4+o
zjO(qy%tE+15A7Ce`}11?v~udx4aB%mnKB%MW}~n>96ciH@vS;J`X?T6;26CZhrW}C
zapa*>o;&Qi-!T@E6T_}Tvt4fdU)R}=w<l%Y|3Y!$+x@?;HIBC<a_oOyt1~LQaK)pl
zym4}?gF}pdyhX13vunvFNJ}BQVNr;4YbCh*p`Q0|=)#Y8kr{XiMyZ(Wq}3POzgO1$
z+2!|TWA+*wIx3+qwj&!I`d+r|LMUd2Ui-5vZCe=vcymYxCKXtdI<U@#626yDJCC-B
zQrzsY4P9r9C#DwQNduJ_&l2{;n8x;$r@i4yzNk=*e6SbrJE9o+>pLI=?c=?X=n&;S
zo-XAv#Cx5*0>u$98Hw>RwWaqmNgOaG&3XW&{k=&(@dkFfH_1P~foYZw`7xl{<~rWa
zLWw8HttW#fBBZ&w(kF&!T~84(F@zf?Z6)#dL0XzB5&4({kIsfDPn?j1C+P;BR+s=&
zS@o>tg-y!KtB|hajB&bCk4h%;0&!0J)~Sg+@>hl&3NTh9Ul?)um)GFFNkU8#^)~oj
zhB<(0ETu(_`nRhO{l&G2?(qe?Fpz=mkzG!v4rTUE;S!WA>Wr4=opwH5zZHk>AvVMY
z^>fe&;GJ>#oxflcV+S2nQ$O;Bmdq3fKg4oQ=1CY8oRrG|!YQ8)dlNqs`fer}N)Sr-
z2GGQps}deyc^Aw9UqpG>X3o1Pw`M*$K_cnNcRNB^WMbI>Nxt^yQ3dxU?+%gL)LQ)A
zae4e-T}zJ7qe)|2Cmn~4Qd`Af{7?dw4|0Lr$HBmWSthy?u<5FdNK%Y_-!Jd{D>lVq
z3TT4X&?4ptt)b0^{`#+2)KZ}(DwJ$V#`(VTzDRk4VI~iww09Z2am17ILaE;Grjy@n
zW4>RRM}lU%QNE~?8VE(M%tMonI=7|Hy6yf><y0vv|IfAL3~Z31T4$o(%29*3>>R3o
z!uD1HR+v57U8wj*-lZPeA%GT1{CTTE4*ef4`EKHkUKe4b*QAy-6+Y%ZH0ny8+vZd!
zqL6+pUoIbY{R+!rZ;fIPSe2X}b!A)vT^mFt2RFdV-;f7Qju}SvLnc=ZkgDA!QhcoW
z=`4nt<zf<4=uQ>76Ena#gThHEWv9I6Z^)%n-u^dOM*;c7-*C-$K>kyG*1gHl)o;Rp
zE|rhJiD~RoIrFCL4t#EV3kF7?{KH$Wlkl;|5T#ERjN!=}h$?My9*XZx1pipu@0hWR
zO-v~lT0ddJ)TX37i@HT@lc70!c!Cq<jiul<_p@bvUVDI7#y3`gdv?mp(Ey+si^VWr
z`Nl^c8N;l5ww(Vq)Ks?2dK*?%fjsAJ+?G-%Z+;uoL}B^h+bDEc{_<_tx-vA}Vn}HD
zh}vh#JiEM?uGaXrD1@~YJ<NMK*qnt9FK>|6ndyhJd<Sw-kWjDY)(T3UH!qA{A*I3#
z<>~LhJS>yD-a*$@CJ(&h+Hef#`CTzatpXMjP32r3HfS=COa7gafHF)ehaF8a@(woO
zOi1fpbRiRR)w{0aawcFPVw<BFOu(u<_mYa9E8Kk5>xE!{3-&mQj)WQ7>ZpzAD@WzQ
zcOj1Bx&kosMc0W&YoWp;Uw9X*24(Uu@4A+xPDHt09#NBWT&63$GHD$5-g<>GjvFt;
zWZXbJXrLVspckR1j)UhRxosRg&y#nIyOy0<gsZXRz~-*F#&}W$I&QTv>f+E}HpIas
z-5&L>fgwP1!S!)t>Rt|oxkY|N`GR}Hm}3#6jvlqonM1!@B$vF0Dle9K?_reQB7g9n
z>+G~55#{Q42e6c2XK`h{9DC1|=D2cF&b;SJUwP%EFz_;km=m>DDjM496a`C~6E<GN
zsA!NUzwcUEO96`I<k`vR+%JSxjZ+uIX;kK}F~~@?L)nGOVp9m~8OR^uS9w~aedSdp
z#kQDNG(;I*(G<h*if&=l(@hMGah(EhAu~~wxAMPO`A2I&dtaoy?H-4jR52iz{M~g7
zp7k<FA;XNRixK;Z2-Zhn*5KBzNSwBAkj+jHDvw?T>2eIkk%f&Pc?WT;5!F(wqnC^-
z+dCJPhZce<`1Uy6P*nUvEt)N+9Q-@l;i&xg-(8z(cTSNZd94UTN8oK_;tvRk&W!K$
zc;lR<G#iEh1~5g(6IDBdIG7i8>Z?9Et6h2Mmg4s2VLwqw?5(36RMgk)j9@HE+kqIs
zu$o?^!B!<>S0-%@(goNRreoGSL_uZZ`YpE@e(WTK{fQsSybBz{&de@s_r>58uQvF^
zZUEF9+QG1V;0=DTBwR%&$3VRXWYPyvl(?wj19X2)+T_QPZ|ssY+h~1>&jN#@lO|~N
z$O~L24bwoGmBdNY<m5??lbUF^kotkJ6H%kQ42tO&QS^<aDhh^d2`}y$76lx<b1rEc
z`f+mk(FahQrE>0vuCx?XI2l}-Sl;0IJdX3qggoU#S4#N=@2KM$y;tDIK4aUhFG=Rg
zc@?8Y>Qzw|I4PFx#yfOfFW4(cuZD08bDp6Y1W0Wzk<WhUI?scb-8@0mL@<+m-;gQ)
zaOF5&9Fgbi@16g^Lddi7@qb{lEg)a{2PWx?<@7&X8D5-WW{I3#D1NO<uAam!_<+1*
zlA~8ox{jgS3OB_aeBBV;4(v<95fa?Nta<XfNB(xwwZiWu)9c_G;RY#eI5N|SYp*v+
z@lWJAE|2>sjNyJ}49n?j&Dl}C12-9AiqWq%qjw}j4!gxMWkv(u*Sg36x1E%XIrWD=
zoZNSUGmH=r2?4S54`X~@f6JH-8Y&L)7MwKhY60o}IZ-lSZIJ)^rz>?4U!%je%d5?y
z#4s<`K74PIxeB{q%)9)u;9ssM9e1_K>3_M-KIPB==Cti@&O9`#t>HqbILgO7ggx<U
zao0RFG3ZFRS>G1OVUx1<-?-xKP(&_`xR%ko<6xJ02#tr{b4tWwJeLtZ{f>m2R7xte
z^3bSaQZ>+ZRt;vqLmd1V9EnDa4m>q}$}UuB=;KVlCkJ^FhEU7#ELP`mwuEqihxiOT
z&XhPTLq;Y!du;QUcn6A`s^onU*YffzS|TkL6}z#(D#931kqSnFrb)ud{AilQ6hjub
zb9gU|3un<pIN}_nrUa51;lWHECR~QDAIBsWdsRS@^nc_^UtS`NPErTmSf;c2d$1I}
z2T?>M9juhwK0^0eDR1})bIS*D0Qn==`r3mds>wp`FBRGOm>gmP4-R<ZZa`bJ0kZmu
zsC#k;NjLLOzk71KIV<A8J^2C5h^TuXIbK1%nNV$eFah^~M4BvF2rJy~;>^%J<uFL7
zGw6lZ7rUP)l?&lE?`gwG?rLPpY40tPKlm8L7s-~7UD@Xr!7M>p$?#JZaBsKTpNE9^
z_F>Z3F7&-Hy)dYuP1I^+W8ORFW;5H4(d?9|M_lP^oyabltQmSNlyD=HkG<Ck`|W}N
zf3sI^KjK=14b3+kfqj-GTaQ3Fl*_>*cwVMS&Yf}<`>IKeAN9k$k=fiQZ<=y#tZkmg
zH3A3)viqDxY$s}Glz{u3B=I{%Vv%mVW=ol-o1!-L*fIdL_ra{8GA_ZPINiqX>%>};
zokdrRJ+7q4y`q>lQ00h3taGF1$DM=lK9miPl6FB~Kr%;}U<bRs*NFfgBk2GTMRKWr
zNPc&-y!8|8<SDEt3zvoC0J)JB;vm8Ss7vf($`#AuYp=+d2hcp&zA0mNA=3i`B=fq`
z55UOdPBF`7!hz~6yNCy>J<<FAOaMgQncScNa2C*bkpVxFvU-bjI(dsOP5&`XMg)?N
znRZOKyW@8ACVnE-_^3cueTtj;vedZ3r}n&t?UDb-(8E}U6$K+?wCM?^BMq8;xM7DZ
z8rH){yre?3aeD;DjXGr2MaRClS3i>56;I1`_@Tte`r}SrU<{k3_{^1*-W#2eDHZ{o
zswDNnQIAx}tj}EDlPZYHVvz&QWebmS1Z>4-36a%F70;!gYHcLDF<y{8&vPSFt+Mtr
z*Ah>cnsC9KC`~v_^~fhb!;E}iG~Y7545SRHIN2vCK6D*|;~S0BFk`~q&lJ7as0#Io
zL08<`z0W5eJcG!WSU_@&PeS@!2~Qv#!Nug+KtMJX+=_mhUp?ZfrI}-WFeZ#?QK}t|
z<|RGLmhcgEXF8#UlMqF{mEapk{I&`wC!f>ciQgjuCCiL35SuUTQY3*u)ba}-904l|
z5J+}rI{D#8?T(=rq5^W=e2$06#o-P`ZZ;(*mQ&>pk+|K!;&wxfxYes!k_l%gUhy-M
zs6ZdBsg52RB=}QY&Scl5b^|9N{nRb1{)1H~NSA89G^edI{1l$pp(l#I;Ri#HG|*A;
zFrEt|k3^cG_%t_*nv?AY%BT}9TF0QsL<PhlK~New`W1GNeA*<}eePO((I?YX+;6t7
z9#!0F7r(JLmy6rMX4(^IV_ez+=m;&hpDH)yLZ!u3!JlK4X~<_k$N4rCfX~L{#@&Ek
zgPYW6Xq!|p6oQ<(Jf9iaTpl8o0(81;PRHdz45W)XV~VEYJ9&eGvfKmg4rJ3kep;1$
zigx<{D(r-6o#WQ%e4tl_9pw`{Tv#sDXV`!C1%}pxyeGU^^qSKcv4KWiXoL@a;acJN
zEFwSq!j-YaM>CIjVXuxxvJH+_y6_LiDkwqOrhRh53?^HB^5Pj>wpc9hnnCB}lh4n%
zR)zX=;|`8OPN)K>`vn7R)lCm@vx<wO*7u*VbbskuvZ4k>LK|O?H4cGpTh+~f8u$#f
zF7{^~vgk`p8#76`oOyoB0<e4@asOi@sy9{L{bvd@S=7(BIH<daT4I8~?#-p9l6+nz
zd%uL8@}C(w`6ZUQI;Hg$?jI-yUr3L4^`ftGs<+;{dg-bxj|OPxx5qylqLlIH#h8w@
ztLt+JA+?O+YT}(DMEMny-`C=8B|qp+V%x^gd$?&Z05{S{omjo5DCA483Gk2P?eiXL
z87oW=HH_3P6a|;TQzpu3cdX>C$D$NaKykQ1z)83L+wM5$W;L(`?k^0v>%Xoq%#r)P
zYh8@LzqU@n-!H9GVUV0`TI;dd;9Ao<&a626sEG)4te$%zZ-{eM-Ol;sOJh8&BiZwh
z$`9AMhiI@!osQD~=V!{?IBUs<OmH8n5^uCB1CuNBx)DNz>*Uih<(MAeNkwJ@D7_93
zxy4z&6Wjj(qijbQ-r!Ivc|2>6nvnk4GC3A!t-}n{JYlWKY|)kjcuh}4R*D>|Bkls8
zouR2+^+Up+cgyX<qJvm<!V3Dkg;5CV3NJ-Qm)6tBnP)`7@h-3I6Bf;v{Z&{Qn1;f~
zf9@CPVZj4FfhUg5n=KQUc-u5~$RV`4%?WvGb9OmrBOq@Tyi-)O|IUYHi__(oKZv)y
zj&0NO-gv7I8zqYqtXxcK989p5V$c86305VR*#n8z9r*k#(K>Iz5X6F4LUiAM>6Emb
z)@od&neDX7D_RY@J(H#5vIdVWG~~R{m@^jzyR5;Wy@jlP;7v6jb>pmtcF(uMX0y3T
zy<cvJ0|ot1l*^_LxvVtDWmD4cveso@h9_ip+$*k_eenW52HFGNQK=dg(vkQh#SJd&
zI7<0bmz91l`a6D@D98^J;b8_X<=?KR(H$4#+r4DXLqkBX0mHYUUA-`Ju*-_xGQykf
zx`+RE|KTLd+F-65BWc9d`Dz~9CV3mY`RaV~wpTE>jmhigSYB+5xowWM5ue^U*1A(i
z)hN|Z!vy*v*Yl~rhLGTX5@qd@A1L89Sj-l3qfjx&WylSQ8JB0eEn4W><+j!>f-uu)
z$7>(gqvX}H(`}_;vgG$}D~Qje+d2`SRdcNu9VZC+kGWQcD}ok;N;|=jN%O3i94GkX
z$UJK~eZQP%1y4Ppgc=ZDu1LZBRN!HeLYrh&V^MTdgb=tD+60blvbWIcux42{-||7Q
zZki8*TcyOOs9iOG55F<{)PCWQndm>EUzfQ2D8VLgeyoAKqq#FTPQY<J4@x&KS9`30
zZ<wgT@^tZRL~!DSVR^a7@~#>;X4e!>41F^z#&}{#wt7Hrj(pBzrI+UD%BG93@SF}$
zYMq!vh@wt4zlD5GY}7a@d;LhN_Ra!TSw~ZKVyj%AWF^<O>KyW8b7-B-0nTU+xiJ3n
z!RdUuyKOdy+;Xkw=$<~Jl9YXJIiwUdLC#9&y#2-<#^R1+PwY50y+cn<7$-No9c5^s
z4nc>%0iwTioQl5A9p^fr&X2;=nUEBzt1Fm=Igs->xfSnNeZ+w_ObwoYsyUrJEmtnE
zRyy!KZ-KP}Yz;4fe$0{gEwGkh4{Xl@Yt5p3V!>A7$3#CT1uduK%g+{|8mi=?h1NzI
zBH-akk>Zaj;3=4l$MBIX<WojLWY_F_TBo3@)yUE~WokB71k^Ui#F}DDggA6+R#8-q
zvlyIOPb=tn?gDn5$dxZIv=(72=vxadKRz=H(b-MNW4u;!5v;CCG=NIlD8aHx*e0rU
zNvJp|O<o!4p<O5;(=Rf^0K&~jZPnKi<-=YpxVQ|jiuKP@npdTFA(4*DYmskz(XLwL
z=Uyv=ZuIe?kMa2(`)inc3el82c9G?G<W0$Q7lDr+c@;i%D-Rx+qD_f-<ilnraqu+H
z?6lx%LB1h@wIkdKNrOy@g6s2P5ykDKS|$TzvjCTrl?NvW1(X1Q(#=d89)u}T)rKc7
zn7GaTa(*&O&@V&DR>nDCA>Sg1;WH*25mhf65(czrR<`yV3JSsLle(pS-CH&#TZ=-c
z&-_1H?djObr5X}(3AoS9m#-yTYtdK#CmAZlEf=R)>BrE?KDr4uD<6L<XzLYA_L<XO
zny`c=9b+b(RW7egLDFj3m;y~(C4Z6vp>CJ2q*#k_`22$uYjs8e@R9M7Ux?Bb(jtob
zfq-x2$YXub;oWkp&su+4H<byG-{nD^^7PAl9G9YwD=dE6T+D6y94E=O{X>+(68VzP
zT7^FTpFU(?BHgKIXeDx8s<omPYOEATbwya6*kugoHmK<j)M8=Dq$A;T=z&eJ{HR*b
z4SC{rJ7WOeCe0yupJ6rx0EH-g-Gop5K+n0Te16;_Y*q$DcswfXKz|;lR?26i7Ujy1
zQo+Dn=}rR!hvl(pfQIF_(ky>U5$akEH*r}grvdT{obm?<#Aek8(yTR?=Ih=Qmr95^
zo|u++K|Z=K)z4p0gy#__>&mok3vgE3T+pTEDQ3G9%g6;?a!ERxPnSHIK11?S`e14;
z-AZ2trNpvXrlgxYgbT1YK`mSpsi1UgiTqVMWTZrnrCZCEpub#>*1DWV1bKAnAj!JE
zb<WsYBbWNE5Iy#Kj^A2G-<$RKr+#ZqhL<v|5I8vqyOmn+Y(u#acN6%nWLgHmPG%L#
zJ%Qpf2tkZx8P<B*s7ljdnb~$_DMel8mz5dTiY0!%1QP;!tiy-$qQwCYMC!`0(vM|>
zeAWcgreYbq*@)35P}Ey~o`Jdw%ap~?n_+pj`g~t~o>*+H`))hEAc{#OY{Q_pz#}F%
zwoW#CkZ)xt%@yLxFro^@(wNB`v$FJFBrD$-Z?^i#8!U=?%e*C)*H=j*ar6&aT-_`y
zmsqF4T6uDbb^MBOFc~>5#}(%Ivp%?tZbf!H(H47_;OcSd3RrZ*-->{>1fMhTS&W97
zi6zU-Y{%XClnL$O;UDt2rbX@vSeHO^kK*IFCL%X2wU*GgWU0kZ>155>9dn2{D`(E`
z*+ax4Cwx?Bo!Mj_<<W%83Stx3*<M?R)~`<<i;5Q0K7_2d7ggm7qX>vrM2{<rvbLiz
z&z7L7Qu>!!bV#Fc83qNT^2TN8+^eSLL(8nxGts+{s&n#9A~j^rQ`d4`P9fXS+#Iee
zP~w-8&y_&IHJCn}&EdLf`Q<Vz<21-2wGliy5*-y>A0po36eKjHQd8IGM32H;pR0Bn
z6DF`aoGmY1j;4w?<?yM<R@;x(hU0eMM6V|)!C<u&E+=%#SxG8}6<`><G~r7cFI*Gd
zEgNpY?CnijEdmhM4`55FA2R)Zv;1l~ly0+J60}k_^^-Nu*uFn1AkcBULJH&e$H?$d
zArW_67#O2C_EgKOgV5jTQ-YR1wOSb0qRC%d7acru@eu2UptUHixkOzA0y6jXuEi3D
zp6S@rEF(e7TMOe++d+H!sK1D#3fY5AyE|0f?CFbAvImP$>V^Sh50n__!}dQEYR@R`
zIoKiQxOLm3eIR>A34*=D3%+|M41FI7trYI1$x`}e;jU7>S#!@c_F~uRt)>h#%>u%@
z0p<ix9U~|@L$NnIK*STpd$VA<#n^XyvlW39RD?I{PR8mtn<s$mh}3P@8hcA|VbQ4w
z=3X37G??+010I{Tv-YCYw7N=oF}G?z(K=f@`*=J65yx#D!dHYpMa2#jl!@Qxizy_4
z1b3}5=ejOkG<Yc67|8dPhp1@43kL5iCbfx~U{745sjX~^K(`B0+#-~{?je4Dl3~a`
z$fQH1`^xcFtYQnV!2?DrX2iZaI;2q>1?+gthJE$1OaXLMXhVNu;^#)(yu-_Vs$B4x
zqdgLq*O5L$-VKB5oD(^Fb6`9m0ZK{@rU8<K77eG@Xn-g|*7OghZJ3hxu0kJ&@AIpy
zOqg9Cu7ZJr<qH@Dl===I*8u>;z9UGy!8Oyi@XdP90rOcVNeYRgA7+PXvM#@e7|xvi
zA(}nUKzV2enj<})N3=SQxjv@JJy|N`-65#(Qu$=aTD%DAat?afIi;)@7}Jw=^6ik7
z?07PQN2smk8{JydVIK~yV`>xQ=XrEY5P5?dxLJWH^^c<byg;768rGgy-mn^~vsga7
z8b(gLd~UUM9zL!$*4px6``%>Ymzo3n1%?h-;1j;gxZRkK$jq@b>mK6w1-Vpk+gAh|
zc}PhF%=-QJi$1h=lNtJSwQN~qEvbcG395|{qAqScPVb4JTvasi025m@p-_y?#q_|6
zGV;4;%81E@C1_DpTY!08MY|QpsOFv#Wa(6j<s~4Pqdrqido9$DZ%d_E<?^((Fbc}$
z?zLb_xqNUfEb?;s@>(kkCq6!3YaPG7y*RqqOY;9rn?ZN5upmFvhD8pIl0QKCMnGPk
z36ueOull^CKA&a6&Vc!~0YqhYke$(C{#@OyPDzpSOVsLigbtab$p{`unNHCJBXEXW
z{#M#hbG6z^GE-F@9qeivXw}MecF6tftQ@Nq0|_-|kb~>2RN6>`$t00gz&#q)A6*^F
zgeT{KNKkJi>SCDAJ`%C5V16Sn@~Ouatb8(tvCv^Sc^Od&y6NghGjW7@xhIU}9uhh#
z5NtgNr@#|q9&=7~Z2gtMbJEtLQZGr?Wuh;>&Vz+iphSu(w;tvYe@&yr>DVI}9U(N0
zVcHFioyNIw;n2e(x6k4VM@%y?<-$HnrP53o%45e^-t6K&7hnC!iLn9Kz+%fKVuW7k
z(su%xRqZalSxZG4lA&W^6PXct_OUR>g7VH|t!mi$BFkD;+pmTI$S)4ZmhI7i=|D+5
zFc0$!qC4KS-zDUyvvQPT)4HxGltonOHjHD5B7ln()INEHI0k`EKe3C)k9vju-0nt7
zjR<i)l=Na8@xgU~_8vR!d%5zxoHOFM@~tc@<&sQg_XIaVz;b6N8xSMDT@m8JMp8WW
zhhkB0V#bHtOh-7tf^Np+g0RK_s-9TwHF)EM;#Za2vff%$i^7zlVg^iD(8`)I9R+GU
zMoGHWK0W1})5x1@vR9sp+X=Xy5$>@)Po_!P?A5B*qLhgUG3gl()@xr4G?HcHV0kqD
z3$dZW8P-sT<FL7a&4I6a496aLlR*7<qTIY(btB}A?y3AQM6+B!c#Mm8Cr$hDW6H!d
za>*v7D*zzH|MdxD$V_N6;y(GKAqKFGPrG9xTx-X8GbYBHFkrrcqCX}=|JNd5FD;=*
zIgUz)^9k(z9bKo!e?#(*$6@{=BtJaP@-K#FfLxb~K3X3u)vbl%67e}QxWUTCgx4h-
zFnOFO@7jRguHTcm?Ot&(#1q=YrPe~;#7)lLOyJN9`BZegV1(?)9H8nbUm^;=-$8!F
zu`-|iNHe+Ntr~Ed&$(yzk!DTQ@~^UKJ27rCm`~?kMBAdFMYd{2$sv=@Vp*JRtw=#U
zQXpgV@o4*2+GIzIc3)-7rfe$(`;|Mgtqm(C#$(>21}O>C!;ab7WGPK_9@>chH6Q~U
z(E|?4{EgPq3&J#6yHzxq*gitIk<?i}yO0OaCF*E#Rx1@3TKRlBelx$y7I7S9<L1Jd
zxa#HMjj)_6<exW!<2V3&JThyP#~yE`qz&ZRJ+qlSfPsbXUIECm<1Mdyln$LRCzGcS
z*Bx)Y;9T(|16xxgBy7ThyFql$vEBW1CbawTcTcreC7AadzWWU8__zyECLBQX;Pp@=
zK4N5g7<A)0R}m;+pJDQGd)PgtLdm@@5A6t6rSNzeER6}m_Pw*M4^WVQoo%ID*-V>X
zY3r+lA|Brl-63)dMRl`7v^n~e=K!T1!cDAr!nT*aHYtQY?5h{_ZWE=Yx%+?!WnxK-
zLF%ut-ofsEdF!{Vv|7gQ1@Y`lbLLLr@d)_forMsW=qS$sC8lZc*l4-9Js;z^Zv=7V
zlb~yz!kG&_O%Z4&8=#-vmC!1N@Mbd5p*VuFGRQ^rcropXCsN7h<Z(*rc9Z)Qv`8=R
zWMOwc`Kw}`(noHOkNqPQClDexP0v!j>54fm$V9k}T7+#bJSuE+HPhjwWSd)#sx+GI
zbTV%OB8Wm4V7iZ^+5SP!?DPOftD+!!V4LNIIGQDJ1yD{N`%f~VxnOm+$jKGC`7yqV
z|IEEJaw)x=Ylj|aHgrv$*)0rtN1=6$<IHYZUubQ(@=Rb9BAaXB%xTWm7Pd2IG+0^#
zl`_ckQc%RDMpj%EXDjjXicBVt*K38BNjQI;NOiN=VzzAYxtPf4(u`1IQw&ZUTCgwl
za>2YB@7KVK{ahTb&x=tU42i%Ornw}@LCm#F#8JduGD1O^1FA<iBBsr{wOoX4Ve)o{
zL}@uOkJjf1)}&qv9Dv-i>0-+(XU?^bE6Nc@p(rY&$d@r+3t7}I<FaS66kb-xskrtp
z>xXU@StTOe%FT&pzes|#e3kXdTh7CR2Y*uK^XFMfm@aw!JWLpk%72_^E#6qc@!i7B
z^7x%rLhWIRDW4EVL%e8;?@hqbiN-|HoY;~`bd_Us^A;;(5x6)WhZ7B>0*7P=;wk6~
zudLc)C9f`E3K?Yuc;*r1Lk=<G=oO-0%*2cOggVAl5tKjQf~ksbIkd&{;aGrSu&WeV
zgD;$Zr<2{($S8-MTsuw=yZdxup$F(GI+&Pk%mvye4~b-XrRg~|H@lTp&d9PNm@4Di
zH#3_sU0Z>bTuf~N%^+jAc9bxbVfS~RS;sYbJ(mfbRl;~ZD%KYC>%ld&wj9C~9^x8?
zSo=kQeZB_<2s8;1<gDnbB<Pb6SLH2(1Y~AW%3Fq{^L%S%EfAmzyt)zAwR38YK`-YP
zV%~^iliMw-*}aDaf=C0Gaw&9AGgHALLQ^NGU*^~zqylzG2sfL(;4>5*Nek&@ZYi}!
zeKuw3WF=M)d4xMgr1traLb0!eA}Ecwk8=%%P#~ICR*L7gERGKfxOLtJUfIZRYx_F#
z+Xh7~2+S&?Lfl?I8};@Ec5@+a@23LgmSVY!qU}fBd4x}OCwBrsiI^>oV=4`pjT;Xj
z90OdnbUL0$ptR(Mv!832HeVW|*2`jdKqdBy%4&*u01?525WjKOgHDY+o0zBphz$C<
zK?mN>t&79)oQ60%5LVt%fwTUTh)U4GZt4JhoaI60^#wRUgMj97A|}vpikUc6FMRGp
z(LieugRvRT{=NbBsKE`FT@KOX=!_E`@txcbCDZ`={{l*PC|}lWwN{=BK|&?eh(_#r
z5Upmri7j?LB;Wna7TH1ALOG&EHzL_fIFz)t)KDObpT#Kg?E1gAS}8Ps7sAUDcR%x@
z8E7fb2UL7??A!()EMZDnq8>y<1G|$c+Cv}{XuVjHWKhY2pWF}U%|>g3AV7Bq$-fO$
z+3wYkjLhPFWP;r!sTA+g83Gk!AIp|c7F)||fzmDfA<-1q!W?}ppDGo~qgAu`$3bey
zc}-k~AD2>SxXA=Z#@Xj5Why)BUbCi5(Gl0lm3Wjt7r3LCs6@k3N~;6B$MnvY3oU;M
zid+WHcbZ}X`#p)&$59Gg;mu0-iE5$%O-PwT>eatC&DAH7)-K=A1LO|mh&-0{u5tO>
z3o$kw7e=c?jK#Mnh|YwDMA4Mk#hUghP~aAAVvKWqYLE(oX3;O&;vqCjdH=#oC>!EC
zScHBN!jrZTl=vYAp9Wz;%+j>RiT1cjF80%C?oJFeCmdozN&e6U9Pms&q4%(gd2WE*
zLy3(}(d-;y(e5>b(Gw?HMTZbQ=wFBj&w~dj8dtq`v%2_AzWmii)*7tQzkiW+Y&Ihv
z5_O!WAM~O@aGIAmMD-XiO%p$>+ququXUMxQw*2yni?J%n?(EZpm_HDu+^%1N+he;S
zL4=S5Qq|ilEJUiac@_MCKk0B22R}rKb)w_$4w+Yi3GTZ)h!U79ZJu6_==Dr55OoV1
z7gE$ilN5v_GuZ7vK+V>aZRCSIykS_u%|xXsJIRl2NK<x^ySknOnyN595tX>x8L<ce
z@1n3eAtA<Ul|C{F&J>v=CXP(7hQiqePBBQ;V&2=t8AH{VGT2UFE;VSg*-2+DC|R87
zb~d|gKjiieZ(u1BC78PCM5S`X%Tau2n|>y;nSqj0>O3j*KE#Jb-@|3UuZydhsunxw
ziJ|4?m;qBUX~#S=R0Stxj2<)#F^yM;<kOc}D_0IN{T@+_11@@$@#;AF`2$d6la!tD
zT3G(~5-Y_YCVvcNa&yUdsF>KM`rQBNlE-bsG+&o2-Dag<Hl`!iD0vwUV97_c#Z9w_
z4HpoNXztUFzIfO;sx6FoIMXswJ%(0{bwbMGuVoq$h$&IV*qcE)yv@2A(?^>xwJPv=
z9G_DmaZWMd9-l8J=SSvK(l^<wWnS+*F*9%cD6h78qU|XEI(b1U7IN$4_e!lLs|HDu
z^O=3>fIc11CzS8xGc~Mb-)Wc6ms&nLY@919!xa2(1*wC)3U-5q;1w(9@jPnfqj7cd
z&;$MPIKe)UKylu$#4H`DPW0^AdF1m}15PxW*t$E#`F()O1P-XKY_J@?KgId5hKWEz
z_=jA$*NuGt(I79o44p?G_EW*$s&k++C%HrWC%iH-6o=K-dzAk$v8)gMH)id|IKO{$
ze%SrSs5RzDj;;b&PtfXR<&+(H8hYpz;X2B#fssDoGjUVR5F?9pD(jD;e-x5)zHJ3P
zAx4GXfwkC2S@Kl)UY$+8$g)8+qfO&w0o*z>8ZXA<CzwqiAusMwnkalcp(9arCAK?7
zr?ZoqltJN~p2O=BXTu}Zl8jFvSg_*Q8u+MM4t?87IiXsWiIX!w-9gddYOq9;)oIQ8
z`hud*RTt|k1KX`dXLXQ-G*~T`XtP@9aC~^-MlkLrKSIf29EU`!)uIB?K+!V<#gH{+
ziD|1*`^uW_SSJ{ja=W#0?kFKp_TfjP@-N%1V-Vsj!)HuxDzhL~RJHx{>lcW|1sw}0
z(~sc=#56LsK|rXcEHO2|?&x^jZUaFv?1?auA4713s7q>05{*fNNn$K%CW(<`%D2j_
zloFrf5D(J=A`6>V+VNf~Vwm?I9fW3ypl%h!QjeA$Di8%Et$2_=iig1>dEVvN8TVk1
zFkpoZV@?y-DJZr2(9}^;ADY&nHxefrMLYGe<|B|5gH0?OLYLp8j4ks~H?=>WW-%XC
zL{c|qK3YJD(8w^{4pRqZu6?vskVQZ$+xVi2<ikTPk-!-+HGn(Ur?~aZ1mwK$Sl`1Y
zWL99^+!fCtMY*WpzV6E&+4dc4?Pee(787=7TeKi3-0bX)>4Y3OMH3#N<7j;Z(e*Br
z?UxgB?hfn3`N&;}5&TD<U-{%kJFsfvlY4epS@bNNN90ojG#$t<4^JjcGF#$$<ZC;u
zl+rNwrlj6Y6Ev3bq%lJX$rRaK-me-k=^7_S;@XAiM0*pxuvj>^3Ci0y?f_B!aX>yC
z2dy>E<b5?E^UAIC435&OM{hib$#a#QJX=<mTmDN>Mfp6f$ewjn$ac?=M~rxexbt=n
zb0<X1#;hO-bsOGcvmxj^l26lFK?&WNFsfTQ&^;jkS&sFz0hxP+RaDzZv^F|gRM-?n
zZYBG#ON-)MW@Hvy7IX6`@^Hd%q8LsbA=AxNUZY2O<x{M-gdx&8=3It4G_;_DL@Q8)
zZdFNoE%|Vp3I-{<Hz14xQoP&_7NLj;ETWL)AW5GpRB{fhBCepvkb)v%06@_mLwRFB
zIst8n0#w6*G7Qa>G9tk{Y>H8+Yq|fsmbDtJR)$q0XX|TYn3SvO8{o#k<@XKC=f7*M
zS<h7Q;K8K*w3sxaz5`E)s;q6dEyt-o>AMm;3z18`DeC48%ojuRC+3UD{IP{%VqpUt
z9Q5qFQ$&)+7l_FP$~seCy$fQz0yV;*1GGGoU@o<lf;$I0J}BFzLxjbdL+;cbOo5OI
zFlma$h5ZZ9wLQHIGKA&<?qE)*SddY~O(J!iJTtjSsS$Fs<%%~m33p)?B}{9TW(6iH
zRBQD^>rId*HPeu%SM;Vdks>vNP$YdOER#AG1p+b68S68KemV|zc2}b4P8?u~3L=|&
z*zt)y46Z=+LK#7;uoSF-xB$Uq{0x~uXPb0P2R-t%zafKiOb353+#E|E@{1Awn4kSR
zg@Jy&3(q-<4u6;OL&k#`y{1j3Qxt!ve6zwj35QzJc3LM@pvv<_YuZGbh@`b;h>nb&
zWuk9c{c?_qK-9QL4H8X1I7lNz(VjM(CdSg5GDJ&8$1>5iY-$-t2Q!^CebJv@pCKAE
zT9=9TWfLlBLO#C}>&z@ceRw#G;;%vZY~8!2o26g(x#pUF23cCx2AK-#m+`D1*Ac7C
zOt6EbpZwP$P8Al8c!mfs^38^j8<Dt<)~qjLfRkjPhO$u9hUKDZ`FK!F23xsNk-wgw
zG&4N1ahH`A@=`L)El(|*Stja&qd_qqZ06X<`Wd5#h{(v5f7oUD=jIXu?q1D|Jo(u!
zEOq3|l&h>@Z2?7w)zY!1d#xB<JDkbMY~PVJ(`(qHP)8fe>{v%08@f3YF7<lU3ChD6
zZkQ5ASkyFeL#6-<VGQ*&KUK(5l!3%B`>BD)nHi(<`KxfqBU7%av~I+4fk!HF^rA^7
zUu_lQv*T)O!$Oen5fyFdp`q#t+lgNJ)2pq;o?bFZLlwF|Jib!CakaH^c{NqK*Dt1B
zZA{?tvmQ-kvzW&>%VV#>`aSmlUxOnD&GLuWSWD+ZC<Dp{Hcu##FJEJ=#Gc5=HP{%K
zE$3cqrCbWKVG!lO+8pQd(c?a3dyX=+F{D_6)<St~%+w&Hvs-mP=BbU8fTvT<Q`<S5
zwcO$yyoGKRACrUg@uKncp3_C&>1}5bmb_+xKDtsV#=L1t9%94PT-+wjDl30(KcnGh
zwRk{YPz6hCK;DYad_fZnjX3R3i1Lf%udAR1eK=o^2CVD?^9&oDbj<p7v4&<r-we)U
z{+P!N=H_zdHcQCn9&!iSy~Ilmwr$Js7Rrb=kQR!TEj?RA|CR~m73TgTAZE^!+pe>c
zY9SW^QMaX?L(i(wjLL)>WwrOM0`gIP4c#A>B?hb!OEmaenX~62Z;T}JMPVgnc<v<E
z!lIc)1f2)ALnB~z2`e@gn4an3$~g~_LL#S76qkx}`5}l{0jfRdXQ9se%r=MU&_nPd
zlrp6Cc~PSbT#w~~A}B@J9UWw(4Kq70D5L=sPC10@8UF<ko?KB-frE$)Z}n3V@@X^y
zvF)HPy<6LNTTyyLOp-;_>QF|(R;2a-_aLV&2Cc}gh1Sb37t$9ee{;R%#twtQ>#g+T
zpf9l8#of&%U<4+I*uA|E_vc`e57SO!((0WfhTRbs{<1Qey4y;x9dxx?qI=E|H-O8_
zSP+@pm(Nh<<B2U!4DY9%qR!PmS9H$po-4Y1!_4q2b0~Hxrs;<;p+j8Wkw*i4!X`%M
zOfpLH@~E+txdS5P7xi-*=7^>_%^bXR$1qp=oO%Z7o+|W|*`mwprjd%dvr>w%wPNmA
zqL@gWV3o16MdpRA#kFn3N?Lyki1G&J&`#ioUg7fX1UsYd339hNlrrB{EDTunWSGNR
z?LZ%-?7>!6Ay<u)pwN(%`9~n5NsJh-x{;UT6C>s?gd2b5WY{_>&tI1UcS~zF{T<4~
z-}^m-CVfVo^mn{-0X~m;2zbmV=&zsu1`5bsarnv`to`QUBR5*1gbVX;bL@-TTrW27
zRR<#jwCs)5SUP=Bv3X%tlH>cg8JqW+o9i8?@5KfI`g!RO;#A>Nn9p(cl2?HKz`qv{
zBwNcI>+)pP53IxMM&MhlP*DT}H>^>#V>0n6+>8wH#b!_TtCq#Lz&xLjb+=$a4)6DL
z-GW7+dO3(s>J;MSoc@jcj>|k$Yb{&60ikAS^TN7Xpgg1g%Dolv(0H>$GOyNJ<Jed%
zD{FypP&U+BCmx^UMwD^<qc2FxiQq$}(<Gh%p*FmyDH8o~BMy#xP1jmW7BvIP`dU!G
zEUH6)7LrwU$fQv=(q~L|)M0JCNe&_`DGSMHFHR8ufGxLG&eU0}PADU<7#?YYPOgVP
zmBEHP;V#5KO$<&7mEMXRt7OfsR{GLWgyhb~KGg?z<!tOz>tzpOr<TF1IfjqAK5_gd
za;ufHIuCrL{IvIG%CAsGhilfqMP}{6$iE+6C1-H0)Ek@kniPdHM(Iu`G8>hI)ElR%
zFpc)~4q3OyN=`d{5^)Mt`dBH-#|4{vtmO#-1eb4?2kWdwjx*|I1b#3WynC(Xi;55w
z&{d#wEU1!&d#&_KNaiXxFDygVownBr06)#Mv-9CDQ@q}1Y_2!=(w+(YG>Ws^;cnJx
z?S`R=#wnD(z1YpsEl2jkU;_cBbMwL?<dR>%a4$#FxfGTl3=}kxG;ZEoR#3lv-?u3j
zjp<xs+ju@xNZz*pc^z<9#WX4{!CQ#BQH}5PhHr1aefws4W4+~bY?+cx^;Y^7RPik_
z33WzA!|*SP@u#Xq-JBo9Cq}f+r)ti`KP~&`SHtbo+|{_wZ<3yUR?2a`2r5t<j8?fu
zdx1b0{nOmMfS6yLuj=oDQdzanT7E(o{Hj!y?=#MrtpT0Q)&RnDVo26J#qp5AePDY#
zg0o_RHN{)0q;`|k*+cIlE1T4#C9pV*uu>JqlBe+$55fOya#b9GpVo(R*KJszE|E32
zp>=1=mfNhgwap05`(~3WVUoU?#U(`6*Rr@I2#M+4ys+yQmY5g0A!(AfjUm2W8F_G{
z0=G?r(OF9R<mTJ06V9O8%vG6D6kDXWmy>@?1J+2lx558wL~QSqeYacbi-_ZSy7IH&
zMt)_bGIG1MVRa`UzYfxvt{Q-!jY(G)9snwAfNBcM$_6yqX4%vL#oi)&8>~%>Mq_a3
z_&68gyu(^@ZX<#Uzsa%dM!uQFb>M(eqvGy^dU2pJ!w00$A6(=emf<_B<*8K9eK9C?
zjyU$vg&fc5La<G|SEURO;TzkvY5yB2X)KFTXOt73z47kH%1xil-H)AXxch^kUfHrA
zMIV&C`@x6`IZ9CTp*!){Cxdq)#-J>uk0Hb4StDBrs+N6sTFGbbq0(tJO}W?&b1!jd
z;OK@~Ukty3m?q()1QDaYRr(vP^wS8e`KAQyYleTx(RI0R5`Lveqra^{HZ@{L_icDN
zw$VySC5g~{=Z2$myS-EPA-DCt2+sMY95zhBzfv`OxKvjKB5_r@`z~CJP$>)VLh&nP
z_%3Vhx&R^+9ZkkPRq)fT46dk-!Sif`ccCq|$%(tb#*lR1jb5&uBFC^qS5@=4%)cAi
z)X2)aacmxI#0__&%4%dUzy%Kwjij;y_y^h^n2{5A1Lu%*9{|oycm`wmsAIJb$@~L|
z)gUVn3mX?3=u;>=$g@Qb9zf}Oq`L`FnapakR;A_v5lk8CpWupS_jt9eLac-$c(rVn
zZ{KhE(v8juK(Run+5JRN4mP3KrE;RlT72S@5k%{^`z_bzy%t~{_2=Di*W3LeU1Nry
zZU$XBGVdVB?UH2&!IyH`KoIsT(g*Lq(kCD%4ubfAbl(H0PiE4mOcv6|BP;K*Hi7Hy
z_gGsn!#;Hn*x4mR_o9dHM>g5o#LJ~TBgRY9vg}^)V?wswYo#QW!;@+zn?vc&x`BIv
zt>I8JO4cv^&6td-k|oW+8IX<5Ai^iRnvrd*9BM{>*d}=&sCLVO`{3!66$BN@ru&e<
zEBo(5a{r-2@bk*vLsq&g!1>U#dxeL9EK^n;vd+N=fy=S+bnK9|DYcd8iK!f~qOD~}
zW=VAZ9$6r*l?(14g*S<c;v?*CStqR}>pj}rtx`QkRUq)shABG13ajx&mK*}s)Czdz
z#WWhl5aWxELoFbQkjAR;0AM3x&MA~dEzoc^fMQLwEES9y-o_R%0#-|lbzIs)l90ie
zCZQ?!_Dc8tpge%!f*9%6<$jDv%a8Hb$&xE%DUyL7HTR>+!m|5*@Oem%60UxviESww
z*S99V=C5&`<HnPmz||}ZA3(*A$chKhTlC202doWistqG)?DU0sXe_7QzVh^aq59S5
z9rQP499Vq!lDh(pOYdB^fB7B3h84H1k{>-_on5<+iyx=hY(yL_W5^Wystq3E046>V
z%nc#pxSv3R`PwAF9t~6NBK`dV4f9D1z&3trVHDn~!{;!()p0-R2|}pq+;M-OhWYC9
zh~pO{8s;aS0FP;SK79cbmnc0G5^+4!ci7!({lhe84o^RbD<?3m;%@`D3S@4ZwPk%6
zkp+6bBy2f;4Cl?kuPEFJxGDxd3>dnu%{q2zd5rge;7!%6(Znr?IO?I6A3-MdGV>8M
zw4khc#M++Jf=Eu}g_4N55l3g6oJ43+2;Ou=lDw<ny;>GN$|}9_QKZH5;g157Th2Uc
zEw8PBy^W~xW%dwK!&gKcJt(k)0#TIsFbySvw_b<W)7D?WSfSwb_t3Byup2P`kAEa$
zK0S`0fQBDOIPlRl35P!yF}wN<oU_Z<9xq4CXJQe&0oVtG7_)PNalljqP7NOc+z&XY
zU|g<cK0`I7!*TV4BexH*(BY{dm}<HhFv^s$ln1@oIaa7W*1-dnWmai8o1UBnoT=ec
z=m}dOu*1)w*KT9rLcmjib9DUDh;dvdU@9?XxGQ3)QL%=<7csI!$UmAvEiO(fMuKV`
z0W97)h{ES-_&$t@NruNY{Ak3nm#W$W?9_e^1Ev=2*6?fg7+S~w%h%xdFn*?u%7#}*
z_=pD7fNM1TDd1+nSqe@x0Tav1H4GJYA`xK2LE1tMShF~BBj7Q>T{`~RxOcY+aGQoN
zh#0lGS@5XT9uUG@YQFs%-i_VSWq^Ykz7;UBJfz`6Xea>fhLZRw!aD)$DoyN0cv(#N
z9}qqbSaUdWG*TP4GdCu}B(`8z0k#SHoNnj<oU1Z)&chWM#33E-%m7RTjq32V5p2l@
z+^pdo5I{1cTe9;sgp-VD0-fj8A_CQ!9pO^Ilz|QJzy%k1kP){=ScP!n7}YVFrLzuj
z05Hh_!3SvHH()yhXxGgxfHQUaM*ve5mT0*4X?tQod%OsJ(S-;OAcX#KzD|3+0q5)R
zF~F063p5Ow$<GEnsbH5o;`r^i0F(X!L9PtIXV1dUBN5|_0AMXMwXRHf5DWV>0yL{L
z^O3(UvFki_cLj3<1i3DWI36SfIN3u4y2>L)K@s49PJbg{s=^8lKL~gPaHWQyix_Is
zf#F)i^?PJMLPIPkgnvSWcEGw0UGLI;7l3uEaD4^18*r;aXf25t=YRl{6c12XYa+On
z0#2Qwbtc^?f%xM(d?#*^AdXfMjP`Hc6fx9<f{xG(n1n>L*m^8t6p@g&=<weF9tK>b
z;lJ7Q2O54KFv&oshNmDS9>7|rtS{+`CBQy}<4Gc84i4``JqS+u8oc6b@cOU8Cj)Mp
z&2UZu;9kJG0CO(HwHJa2Pz24nlCHr33_U^Te2qDk@F-Ot3||kp3ox??;n0L?8bTwu
zmu{y(c%g>708^D^Y4|0;#337o=2LSII(!0f7~!Q3(!a=H8XiON&_aYa+zmC!pc1(K
zxP^i;)a~4TY{a-I8$w#9!*e2zU%C;lS?tb-nLu@J!{?CM5CdNTA+7?POZ<n>xwj(%
zQS1ebX5+pV5%K`*-rZe`@IehzcoV`408_iC^baF^A}0Lj2q%K=^uLZ6CCz~IG{Juc
zO!?Jn7P-+GsCf__0qs=Hfhc&cmkuHz0SOJ|8L;O(G`u=uY!4wkSHl}%I8g?=Rn3Kr
zsHqPf{;h~(&p7hO%e35k%>_&LkPsDX7~Q9u6VVB-0^Eo2QVstg;<&X2@B$rwKV9+x
zSeI~aGs20*q$kij&V}YHgN)57(&>K@any|<e`Ar3@EdqUVh|1jCZV)5_)El5I|kT}
z|32X9m~e3HD$*l<mBBm*y&M8KpkWVSs-i{>FGl)Y<ZsMV1k8itz)m~Ev=f{Zaoj|8
zmZZbaqPsp2uA9-ktr5rfa}cik`+3{xo(>3QzRuv<h*33)aGRiNz(lZCZ1e6z`r2Sj
z24K;3jevEtn)fTZj}Wj;&?|rmL5<E}1S}SS?E*lCuBQSRI(!NotV8&GRe<?s#Mq5+
zTh-5>4-V8(v(X6VqnU-X0qYFrA4d@Y>pGl&2Ew}lYl7y(BnuA#)&$Sripd|6fm}tv
z{L5)F2n6ZAVE(R%abq7~JO1}^&q6g3(4`YJ;vwl?MDS?%0lZp0i11bocjDRM3WR5B
z_?IA%N@NrGB3-2cxL${ksH-z<Juv?r#4kj6vgXLYVDr`ZP-opb!50z7%{hoLS4Hq7
z;*19fFtb&-XL-b^L$Iki<k=K4xE0Lk@UxH};kHfa0Y~;!1FqBImqDg-0c(Oiwdjr1
z+?I~8J7TCgEe)edsQD}n-%s;cKwuN_V_fEd0z_e`Vl|Pa<AWn=5=+B_5#zQ3#1AXD
z)&os+8(408c%Yka&x%0^6YTcR*(ym&L<uqxt~r#nkgjn6oTCsXt%J~20oF2=bRyl{
z0Ju_zZ$>!D3~9zVwEv_nH0cGmTSxddBJ?AIW>FFpPh%yjNV7QUW`q|aTuXJ*?SS*6
z2#q9k-v=NgjvkHgG1NI_XvgmY+y|HofcBsCe8jjX01vI1l3oQ&CCJkV#sPN$j%fH3
zAOL9Vg#~C<_qG9UP~i(a5V{~>-T5q71_P%<!{{p(p!33x5LIQ=|13BG2?8+*3Xq^Z
zCIRg9W`Z?A3otf(kg7oI_66v!o5#Vy7EM46(znHg-wC)S2EHFK$q>@lQt!3kCm_JD
zBcOL%@C>AQ0<f;b1up;|2COyV0*q{0Y5;2?S^%RIhOS{YY6AWR85xX84;gxZ;#aHi
zg)YD(6HeX#FHEJ$MR*MB2&-W-<pI_#T!@j(52+4Eb@;Z3(b@?(sNw4XQ@F14g^-E*
zBAp(c&%!3WW!wM)?eK@`unJ_vU8@m188IG#P}!Pk;csyo1uTmT=x}sCH&p?lRvZhV
zYJb#&_#HYNGI3J@!u43t3z_&y8Pbob@MRMykWCOc?41Xg;@46$AP=bC$MfKkqY`*?
z09OMZ(J=b!n<;_r4ZRmdj7~z>qQkF5cqw3AMRPL|J^?sahwp|xFa%i3jQ5s^@kEg9
zPdmegi18#LXx0f10geFf*YI<6M=fBjTD{;<R}SE69sWOnyMVu2!(-^(+z8hk^G<@m
zX~1>?VDddvPW}J5O0Wou<d;-Jttb{@KH!&>p$)^1KO6wd4IRHEV)OvEN3)BdIeYqI
z;JYHYjTf-)TNWKcZ`lOcrBwSOnDx(jkbk^SBYYZlNF~;FxCnOsZ$h(u!=gW;1XT#v
zC0O*gi1ETG!nM#YA{i(_xKCw}3>oMLM@+B6pN#JI=6;0hR+XHNX2@%0x=xaVm=LB2
z**ZZMO%Q{nTJa>G1*tAZxHdSF!GTw(j&)zK5Ss7R2Ee+`ldk{_e6)uSY%`L<q1RA=
zIL(n{$UyBM14uBfBOHhr|KkI!>p1y`fLlR;4WqMpgW^x>_%Fih9RRGGY4WSkjOBne
z2b1537$YPDGde!{mcKYbiyoLHe-Wv5)HWi*LdCKa*!_Pk0)_o5K??N32Y!TW0#jDt
z;Dr}Pq*I4)fa1zRxF#UwbP&)9SXWU>5$YWHY?h~>``s4+tO-oH26p|exw8q<P)ARc
z=?Kt_Dc=XeHo%(YDQG1hf&hCyAO)(lHiU59>{5P&vl4-ra57L3KN^nicrp_O)Iyw6
z`{#(`mPtgQfd@2I$~%A|R8b256*2ye1knuti*RDO&d@i9PD`M()Ex4qL>$inc2b2y
zulSZiX2uaeSK;@ed%oE-i@(-)0;HBQ)GYIzfd~P>y50KDr(4VclT?Evz8w)W1qiar
zRTcOuF=5exa4p2X8?oiR0&t!VuK|qk`4tL|JBav=(fGB7?>2Ze!9x#~eD{H%2Ee-i
z_dSAXHI&HI&C1t>nSLsPmU7=qD6wB7q!Ry;Rss+|OB3*J#PPFfga>p1{u6Pq&B+Bu
z`%iU69FMtYBc!6g{%Lg#oD(tK1JDyy3SlamA<aoU27nNRq+Seo4DqP|n6pkr_snzJ
z1Y=G+b$7&EKn3X0>2Hac&~;dR08IUVY6CWSgT;0`O~v5y<{m`Q97t^i+!+IRVB2>E
zVBPto{vLBY=K$7oKB<2N2Ri`kky7gW=xj)D=vI>Y5n4^Af?<DwfG=S}5ev16lx9U7
zPn7_sW`a6TTZC3ZQf!BVL$?M2YZ*y97U^4K!gG<H2(;6KV>dSgCITSjX(b>a8?Z_9
zFCZYT3<%5MVP|l4#PPjpz`9kW-GmaNPVEw=L1u311FV~6+C9ji1+dm5X%B%QDxpo_
z;~=OSuuTw}WgT%enfOl>{T3c1gmW}Jgl5wRc$tQYrNjZv(zFjDbUwhErD@Rp)iZ$c
z>I30Rj|WW6P7{z04pt8UPSxQn0CxedwG(WF2bI81a7M&&Lpg-3Stmd<tgc76ZujYz
z!v^dI97H&(DE&%=hXHF2riUTax{f)#F5+k<nW)zB??(Je<ge{;BONUJSqKq|bcAOx
zr2^@;7a-ChWUcK8rxH_!Zva6lVBMRg{~d4<U|l8YpG6!KLpnaH*q;cF*#r@s3^+jj
zzh<EyLiHR8g$*AAI4cI;gbZ>3YYzF(ikQC)0H$`0mgGk>e30VnI`)@gg##e=#jEhE
z5uac?{`UdrA%7G9F@k;Q%_4|EB_Im#1Hv-ES~2<C0h7?!Fzkgr6rP3nsB`~IfQbMb
zhH7tZ1FSjb2Zx>`8L|sF1Gv^R>ya@Jt$-3}1Q~uH1Q><aAbcnW21kB42DlFdph_|}
zWAzO%R~f<Qf@1-MYmQ~0_j``2SQC)3E8>v5p0`e`t$?R4RR-$$8IojTBe;-(NruNs
zaEf#$|Ax9I$tNx$lg04}p9UPzFsz!#W8`Qt<`5ny!M5XLaKnbJ9UsHJ+MmT_u=oZ<
zh|K__m!HKVK!5PF>X-}<LtV$F{{zBf(~l#3Iwt<-fMW<;k_02<r!g5USru_~#2{R9
z3KGO1TvCki7=%l}#*SEow*rpECjnQ_60qcnh~p=*`1@gmbi`%=W%HBR3?hJIG6<k4
z{Uj!XKnCEL3<4V>jvvQl5cn3tWAF!{ZGRktKTvxkJYqAr3vg@(7()LzHiKs)jvvKl
z0B!l>*bFc=_oLVhAn8Af!M}7N;28Y1OIO1qCWED?MjY)i87#dB;V~I3h0)O-lflwG
zfMYYbA8<?tOMebH7C)NmW3%|rGM0`aLTm;fBSK6D%bb97P=~q|EL(yuRRGqyVj0x;
zV-1HV`>a|MZ>?3nNEam{*j&nuWEti%pX!N$(WsuN0xZzN(1@13gGLpMfg=&~rvbo5
z1~=wqklCkNX2X|bNbp1x!3zKKB!vG8u+DG!(ui4mPBT367r@gnmT!ufj~4^h2$$zY
z%#NZM_-r7ca81ziizDXQq-v&g`fCv08WVnN#C)s^s-{2@xcm-!VG-SDosKUfwPq<e
zkTj+}o&W*im<$gCrV@|^MFjpD1dYUm{~j6m01KV|e<J3!ZNL}O8U6)kX%B@f4$OyD
z=&k`=sN=s&EUJ<2er&%ehksD}&(LOZz}X58CZJ$rfHj_=FXDJHN0uU4?JMwhDD*+7
z*3Le_J5+@Qw_<BKC^rT*n(Yzu9FRX(7dKc1wAl!sr{hDFzj!iA>IRJegSVl0-3Zt9
z5xfidMgclCz82Vw9eFVk9)*BqN%u>j3wYxv7O-FhpO2V7D~yQ-w%v+eHo>XG{|Gt?
z2(K;&<op+Iz<r9Yxlj!^Ail8)F#50{1o5c`z*-K1AER0VfNeT4KU(|LIN*f};R+L`
z+(wF^3brB<4WkqwA;AAD7NMYif>ps*1W<3o1ncx`kU<|{oxzF?kPBxFoC|{50c*)#
zkspz(e`hUQPqJaBfp&Q6<(N1-P)Nd|3$fxlgty6>-+^}!nK+^gjc&yc;oXO5P7Obe
ztOI~;w7uv~`v3+|(G>r85yuAr>+GWb2|Xo++wchL5Ug=!>Y_qzKh=w1P5MgA2gtPF
z<CskaJaxld>5rH{2Ivq<u2(LHppjr`+E=azl4-ym#jlkoN6fRrh+nEQS$P4%2M}&&
z0^R=9b1`rg1cWMF7b>+HMEndyEb1dqAVJA!?I#TBHHziX@3CdE15P`?2cU_jD5|Cp
zMo`yZ0d7-FTM5nj)Q_Zp0KA=mw{B)D--P!;z^0D&PgFCt7ma3>1I6qFsEfJE3o!`*
zE>Oi<wOGZEt5(^rLZAHGBGf@0@ZtYe*=XHah#yk;SAi|H9X0T<GdL3ji~_VXxDW`N
zW5Quwa^;f#0Si{a40-BVglj&o+J$mc<=ObcfSob$4-o$zimwT{8&<?qjquQdy$Zel
z6I}%B2#-e0Z}kJ#99q=_f~W*`1}~tZDq`SQAjE@!?F?W|K1Hyum{n-0YUh+Lzz4N-
z00t5AGz&@msKhn_Gk}|B5r*b~<w$F*oKRZCyadg|(WlafRz}P^!dI$PNeF|yPK4X|
zLa;ucxQXy7$qX$z9V{w`hb~d*T)<<p@D*#x<t2!guc|UsiDp(66YWN*PB-8No$p=H
zAc)WTfOjEkrbQY?O@)4pIh-s??_re{f&_Mr93Fhdx+p%W>hkM%?%uI|_Z8Rgyu5Pz
zwKs$hFCDTTGY|jykJb$fvGcF+J?jl~lMKFZ{qAu5->pPb{^NZs?eLrl%R3i`QUae?
z=Q+;El2xBri!Eg0*w+P3*CqQtv6jS-8IJqQWbY@|^7sgxRkHI_IKi;{>y*<2XOnUU
z;B1qnpIXZj?gyItyJS5@#v51<_*INfI5Xuid|iET=3!{)JM<Z{88IABm&o8}NQzkn
z$J5wY%5L10{xo(2lN(yc@k_ku`Wbc@X32hh;+qZgtjA>JGwj>kB)!w{xfLf<cgwtK
zoIRR1CCjIA+kUW3HsYJGvdu95_jTDrVKdjtQ3{)Bm*PJN`*OrE&wNH^{RjEv-(?u<
m(`5<VjvG2;4Z^XTPqx$d<wiM(Z}P`yplZ*j>d-%vzW*P9>kv2q

diff --git a/roms/seabios-hppa b/roms/seabios-hppa
index 17ca7a9998..458626c4c6 160000
--- a/roms/seabios-hppa
+++ b/roms/seabios-hppa
@@ -1 +1 @@
-Subproject commit 17ca7a9998c1755d42321cfc0afb5f480f5a58ff
+Subproject commit 458626c4c6441045c0612f24313c7cf1f95e71c6

From 12629fcf4df783f67e84173a18c9a511d37eb62d Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 10:02:24 -0700
Subject: [PATCH 530/935] target/m68k: Clear mach in m68k_cpu_disas_set_info

Zero selects all cpu features in disas/m68k.c,
which is really what we want -- not limited to 68040.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20220430170225.326447-2-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 target/m68k/cpu.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/target/m68k/cpu.c b/target/m68k/cpu.c
index c7aeb7da9c..5671067923 100644
--- a/target/m68k/cpu.c
+++ b/target/m68k/cpu.c
@@ -75,12 +75,8 @@ static void m68k_cpu_reset(DeviceState *dev)
 
 static void m68k_cpu_disas_set_info(CPUState *s, disassemble_info *info)
 {
-    M68kCPU *cpu = M68K_CPU(s);
-    CPUM68KState *env = &cpu->env;
     info->print_insn = print_insn_m68k;
-    if (m68k_feature(env, M68K_FEATURE_M68000)) {
-        info->mach = bfd_mach_m68040;
-    }
+    info->mach = 0;
 }
 
 /* CPU models */

From abc098351e533de5ca0ed9c90901f9f4dac348fc Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Sat, 30 Apr 2022 10:02:25 -0700
Subject: [PATCH 531/935] target/m68k: Enable halt insn for 68060

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20220430170225.326447-3-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 target/m68k/translate.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/target/m68k/translate.c b/target/m68k/translate.c
index 4026572ed8..e4efd988d2 100644
--- a/target/m68k/translate.c
+++ b/target/m68k/translate.c
@@ -6003,6 +6003,7 @@ void register_m68k_insns (CPUM68KState *env)
     INSN(tas,       4ac0, ffc0, M68000);
 #if defined(CONFIG_SOFTMMU)
     INSN(halt,      4ac8, ffff, CF_ISA_A);
+    INSN(halt,      4ac8, ffff, M68060);
 #endif
     INSN(pulse,     4acc, ffff, CF_ISA_A);
     BASE(illegal,   4afc, ffff);

From f73eb9484bf07812db21de985cb9c1e4ad3c82a9 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Sat, 7 May 2022 07:48:26 +0200
Subject: [PATCH 532/935] pseries: allow setting stdout-path even on machines
 with a VGA

-machine graphics=off is the usual way to tell the firmware or the OS that the
user wants a serial console.  The pseries machine however does not support
this, and never adds the stdout-path node to the device tree if a VGA device
is provided.  This is in addition to the other magic behavior of VGA devices,
which is to add a keyboard and mouse to the default USB bus.

Split spapr->has_graphics in two variables so that the two behaviors can be
separated: the USB devices remains the same, but the stdout-path is added
even with "-device VGA -machine graphics=off".

Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220507054826.124936-1-pbonzini@redhat.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/ppc/spapr.c         | 12 ++++++++----
 include/hw/ppc/spapr.h |  2 +-
 2 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 6de800524a..d112b85b4f 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1066,7 +1066,7 @@ static void spapr_dt_chosen(SpaprMachineState *spapr, void *fdt, bool reset)
             _FDT(fdt_setprop_string(fdt, chosen, "qemu,boot-device", boot_device));
         }
 
-        if (!spapr->has_graphics && stdout_path) {
+        if (spapr->want_stdout_path && stdout_path) {
             /*
              * "linux,stdout-path" and "stdout" properties are
              * deprecated by linux kernel. New platforms should only
@@ -2712,6 +2712,7 @@ static void spapr_machine_init(MachineState *machine)
     const char *kernel_filename = machine->kernel_filename;
     const char *initrd_filename = machine->initrd_filename;
     PCIHostState *phb;
+    bool has_vga;
     int i;
     MemoryRegion *sysmem = get_system_memory();
     long load_limit, fw_size;
@@ -2950,9 +2951,12 @@ static void spapr_machine_init(MachineState *machine)
     }
 
     /* Graphics */
-    if (spapr_vga_init(phb->bus, &error_fatal)) {
-        spapr->has_graphics = true;
+    has_vga = spapr_vga_init(phb->bus, &error_fatal);
+    if (has_vga) {
+        spapr->want_stdout_path = !machine->enable_graphics;
         machine->usb |= defaults_enabled() && !machine->usb_disabled;
+    } else {
+        spapr->want_stdout_path = true;
     }
 
     if (machine->usb) {
@@ -2962,7 +2966,7 @@ static void spapr_machine_init(MachineState *machine)
             pci_create_simple(phb->bus, -1, "nec-usb-xhci");
         }
 
-        if (spapr->has_graphics) {
+        if (has_vga) {
             USBBus *usb_bus = usb_bus_find(-1);
 
             usb_create_simple(usb_bus, "usb-kbd");
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index 14b01c3f59..072dda2c72 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -194,7 +194,7 @@ struct SpaprMachineState {
     Vof *vof;
     uint64_t rtc_offset; /* Now used only during incoming migration */
     struct PPCTimebase tb;
-    bool has_graphics;
+    bool want_stdout_path;
     uint32_t vsmt;       /* Virtual SMT mode (KVM's "core stride") */
 
     /* Nested HV support (TCG only) */

From 1e665723e92cd3dae4d8943bf7bd1799a3b4a82a Mon Sep 17 00:00:00 2001
From: Bernhard Beschow <shentey@gmail.com>
Date: Thu, 5 May 2022 18:18:01 +0200
Subject: [PATCH 533/935] hw/ppc/e500: Remove unused BINARY_DEVICE_TREE_FILE

Commit 28290f37e20cda27574f15be9e9499493e3d0fe8 'PPC: E500: Generate
device tree on reset' improved device tree generation and made
BINARY_DEVICE_TREE_FILE obsolete.

Signed-off-by: Bernhard Beschow <shentey@gmail.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Message-Id: <20220505161805.11116-8-shentey@gmail.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/ppc/e500.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/hw/ppc/e500.c b/hw/ppc/e500.c
index 2bc3dce1fb..7f7f5b3452 100644
--- a/hw/ppc/e500.c
+++ b/hw/ppc/e500.c
@@ -47,7 +47,6 @@
 #include "hw/irq.h"
 
 #define EPAPR_MAGIC                (0x45504150)
-#define BINARY_DEVICE_TREE_FILE    "mpc8544ds.dtb"
 #define DTC_LOAD_PAD               0x1800000
 #define DTC_PAD_MASK               0xFFFFF
 #define DTB_MAX_SIZE               (8 * MiB)

From 5bb55f3e3b00679519a83ffe688eae0e68e305a7 Mon Sep 17 00:00:00 2001
From: Alexey Kardashevskiy <aik@ozlabs.ru>
Date: Wed, 4 May 2022 16:55:36 +1000
Subject: [PATCH 534/935] spapr: Use address from elf parser for kernel address

tl;dr: This allows Big Endian zImage booting via -kernel + x-vof=on.

QEMU loads the kernel at 0x400000 by default which works most of
the time as Linux kernels are relocatable, 64bit and compiled with "-pie"
(position independent code). This works for a little endian zImage too.

However a big endian zImage is compiled without -pie, is 32bit, linked to
0x4000000 so current QEMU ends up loading it at
0x4400000 but keeps spapr->kernel_addr unchanged so booting fails.

This uses the kernel address returned from load_elf().
If the default kernel_addr is used, there is no change in behavior (as
translate_kernel_address() takes care of this), which is:
LE/BE vmlinux and LE zImage boot, BE zImage does not.
If the VM created with "-machine kernel-addr=0,x-vof=on", then QEMU
prints a warning and BE zImage boots.

Note #1: SLOF (x-vof=off) still cannot boot a big endian zImage as
SLOF enables MSR_SF for everything loaded by QEMU and this leads to early
crash of 32bit zImage.

Note #2: BE/LE vmlinux images set MSR_SF in early boot so these just work;
a LE zImage restores MSR_SF after every CI call and we are lucky enough
not to crash before the first CI call.

Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Tested-by: Joel Stanley <joel@jms.id.au>
Reviewed-by: Fabiano Rosas <farosas@linux.ibm.com>
Message-Id: <20220504065536.3534488-1-aik@ozlabs.ru>
[danielhb: use PRIx64 instead of lx in warn_report]
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/ppc/spapr.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index d112b85b4f..fd4942e881 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2975,14 +2975,16 @@ static void spapr_machine_init(MachineState *machine)
     }
 
     if (kernel_filename) {
+        uint64_t loaded_addr = 0;
+
         spapr->kernel_size = load_elf(kernel_filename, NULL,
                                       translate_kernel_address, spapr,
-                                      NULL, NULL, NULL, NULL, 1,
+                                      NULL, &loaded_addr, NULL, NULL, 1,
                                       PPC_ELF_MACHINE, 0, 0);
         if (spapr->kernel_size == ELF_LOAD_WRONG_ENDIAN) {
             spapr->kernel_size = load_elf(kernel_filename, NULL,
                                           translate_kernel_address, spapr,
-                                          NULL, NULL, NULL, NULL, 0,
+                                          NULL, &loaded_addr, NULL, NULL, 0,
                                           PPC_ELF_MACHINE, 0, 0);
             spapr->kernel_le = spapr->kernel_size > 0;
         }
@@ -2992,6 +2994,13 @@ static void spapr_machine_init(MachineState *machine)
             exit(1);
         }
 
+        if (spapr->kernel_addr != loaded_addr) {
+            warn_report("spapr: kernel_addr changed from 0x%"PRIx64
+                        " to 0x%"PRIx64,
+                        spapr->kernel_addr, loaded_addr);
+            spapr->kernel_addr = loaded_addr;
+        }
+
         /* load initrd */
         if (initrd_filename) {
             /* Try to locate the initrd in the gap between the kernel

From 162eec18c0aed28eda472bacf207b7b222894169 Mon Sep 17 00:00:00 2001
From: Alexey Kardashevskiy <aik@ozlabs.ru>
Date: Fri, 6 May 2022 15:51:24 +1000
Subject: [PATCH 535/935] spapr/docs: Add a few words about x-vof

The alternative small firmware needs a few words of what it can and
absolutely cannot do; this adds those words.

Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Message-Id: <20220506055124.3822112-1-aik@ozlabs.ru>
[danielhb: added linebreaks before and after table]
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 docs/system/ppc/pseries.rst | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/docs/system/ppc/pseries.rst b/docs/system/ppc/pseries.rst
index d9b65ad4e8..3e1bbe6726 100644
--- a/docs/system/ppc/pseries.rst
+++ b/docs/system/ppc/pseries.rst
@@ -32,14 +32,43 @@ Missing devices
 Firmware
 ========
 
+The pSeries platform in QEMU comes with 2 firmwares:
+
 `SLOF <https://github.com/aik/SLOF>`_ (Slimline Open Firmware) is an
 implementation of the `IEEE 1275-1994, Standard for Boot (Initialization
 Configuration) Firmware: Core Requirements and Practices
 <https://standards.ieee.org/standard/1275-1994.html>`_.
 
+SLOF performs bus scanning, PCI resource allocation, provides the client
+interface to boot from block devices and network.
+
 QEMU includes a prebuilt image of SLOF which is updated when a more recent
 version is required.
 
+VOF (Virtual Open Firmware) is a minimalistic firmware to work with
+``-machine pseries,x-vof=on``. When enabled, the firmware acts as a slim
+shim and QEMU implements parts of the IEEE 1275 Open Firmware interface.
+
+VOF does not have device drivers, does not do PCI resource allocation and
+relies on ``-kernel`` used with Linux kernels recent enough (v5.4+)
+to PCI resource assignment. It is ideal to use with petitboot.
+
+Booting via ``-kernel`` supports the following:
+
++-------------------+-------------------+------------------+
+| kernel            | pseries,x-vof=off | pseries,x-vof=on |
++===================+===================+==================+
+| vmlinux BE        |     ✓             |     ✓            |
++-------------------+-------------------+------------------+
+| vmlinux LE        |     ✓             |     ✓            |
++-------------------+-------------------+------------------+
+| zImage.pseries BE |     x             |     ✓¹           |
++-------------------+-------------------+------------------+
+| zImage.pseries LE |     ✓             |     ✓            |
++-------------------+-------------------+------------------+
+
+¹ must set kernel-addr=0
+
 Build directions
 ================
 

From c9f8004b6adf7020ba742d16b132e84ff6e57863 Mon Sep 17 00:00:00 2001
From: Murilo Opsfelder Araujo <muriloo@linux.ibm.com>
Date: Tue, 10 May 2022 20:54:39 -0300
Subject: [PATCH 536/935] mos6522: fix linking error when CONFIG_MOS6522 is not
 set

When CONFIG_MOS6522 is not set, building ppc64-softmmu target fails:

/usr/bin/ld: libqemu-ppc64-softmmu.fa.p/monitor_misc.c.o:(.data+0x1158): undefined reference to `hmp_info_via'

Make devices configuration available in hmp-commands*.hx and check for
CONFIG_MOS6522.

Fixes: 409e9f7131e5 (mos6522: add "info via" HMP command for debugging)
Signed-off-by: Murilo Opsfelder Araujo <muriloo@linux.ibm.com>
Cc: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Cc: Fabiano Rosas <farosas@linux.ibm.com>
Cc: Thomas Huth <thuth@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220510235439.54775-1-muriloo@linux.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hmp-commands-info.hx | 2 +-
 monitor/misc.c       | 3 +++
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index adfa085a9b..834bed089e 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -880,7 +880,7 @@ SRST
     Show intel SGX information.
 ERST
 
-#if defined(TARGET_M68K) || defined(TARGET_PPC)
+#if defined(CONFIG_MOS6522)
     {
         .name         = "via",
         .args_type    = "",
diff --git a/monitor/misc.c b/monitor/misc.c
index 6c5bb82d3b..3d2312ba8d 100644
--- a/monitor/misc.c
+++ b/monitor/misc.c
@@ -84,6 +84,9 @@
 #include "hw/s390x/storage-attributes.h"
 #endif
 
+/* Make devices configuration available for use in hmp-commands*.hx templates */
+#include CONFIG_DEVICES
+
 /* file descriptors passed via SCM_RIGHTS */
 typedef struct mon_fd_t mon_fd_t;
 struct mon_fd_t {

From 4ddc104689b186c4e4ed30be59a54463501761cf Mon Sep 17 00:00:00 2001
From: Leandro Lupori <leandro.lupori@eldorado.org.br>
Date: Tue, 3 May 2022 13:39:04 -0300
Subject: [PATCH 537/935] target/ppc: Fix tlbie

Commit 74c4912f097bab98 changed check_tlb_flush() to use
tlb_flush_all_cpus_synced() instead of calling tlb_flush() on each
CPU. However, as side effect of this, a CPU executing a ptesync
after a tlbie will have its TLB flushed only after exiting its
current Translation Block (TB).

This causes memory accesses to invalid pages to succeed, if they
happen to be on the same TB as the ptesync.

To fix this, use tlb_flush_all_cpus() instead, that immediately
flushes the TLB of the CPU executing the ptesync instruction.

Fixes: 74c4912f097bab98 ("target/ppc: Fix synchronization of mttcg with broadcast TLB flushes")
Signed-off-by: Leandro Lupori <leandro.lupori@eldorado.org.br>
Reviewed-by: Fabiano Rosas <farosas@linux.ibm.com>
Message-Id: <20220503163904.22575-1-leandro.lupori@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/helper_regs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/ppc/helper_regs.c b/target/ppc/helper_regs.c
index 6159a15b7b..12235ea2e9 100644
--- a/target/ppc/helper_regs.c
+++ b/target/ppc/helper_regs.c
@@ -292,7 +292,7 @@ void check_tlb_flush(CPUPPCState *env, bool global)
     if (global && (env->tlb_need_flush & TLB_NEED_GLOBAL_FLUSH)) {
         env->tlb_need_flush &= ~TLB_NEED_GLOBAL_FLUSH;
         env->tlb_need_flush &= ~TLB_NEED_LOCAL_FLUSH;
-        tlb_flush_all_cpus_synced(cs);
+        tlb_flush_all_cpus(cs);
         return;
     }
 

From 3278677f6a5e048b17ea7623c107786448fdf6f4 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Tue, 17 May 2022 13:15:20 -0300
Subject: [PATCH 538/935] target/ppc: Fix FPSCR.FI bit being cleared when it
 shouldn't
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

According to Power ISA, the FI bit in FPSCR is non-sticky.
This means that if an instruction is said to modify the FI bit, then
it should be set or cleared depending on the result of the
instruction. Otherwise, it should be kept as was before.

However, the following inconsistency was found when comparing results
from the hardware (tested on both a Power 9 processor and in
Power 10 Mambo):

(FI bit is set before the execution of the instruction)
Hardware: xscmpeqdp(0xff..ff, 0xff..ff) = FI: SET -> SET
QEMU: xscmpeqdp(0xff..ff, 0xff..ff) = FI: SET -> CLEARED

As the FI bit is non-sticky, and xscmpeqdp does not list it as a field
that is changed by the instruction, it should not be changed after its
execution.
This is happening to multiple instructions in the vsx implementations.

If the ISA does not list the FI bit as altered for a particular
instruction, then it should be kept as it was before the instruction.

QEMU is not following this behavior. Affected instructions include:
- xv* (all vsx-vector instructions);
- xscmp*, xsmax*, xsmin*;
- xstdivdp and similars;
(to identify the affected instructions, just search in the ISA for
 the instructions that does not list FI in "Special Registers Altered")

Most instructions use the function do_float_check_status() to commit
changes in the inexact flag. So the fix is to add a parameter to it
that will control if the bit FI should be changed or not.
All users of do_float_check_status() are then modified to provide this
argument, controlling if that specific instruction changes bit FI or
not.
Some macro helpers are responsible for both instructions that change
and instructions that aren't suposed to change FI. This seems to always
overlap with the sfprf flag. So, reuse this flag for this purpose when
applicable.

Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517161522.36132-2-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h        |   2 +
 target/ppc/fpu_helper.c | 122 +++++++++++++++++++++-------------------
 2 files changed, 66 insertions(+), 58 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 48596cfb25..901ded79e9 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -735,6 +735,8 @@ enum {
                       (1 << FPSCR_VXSOFT) | (1 << FPSCR_VXSQRT) | \
                       (1 << FPSCR_VXCVI))
 
+FIELD(FPSCR, FI, FPSCR_FI, 1)
+
 #define FP_DRN2         (1ull << FPSCR_DRN2)
 #define FP_DRN1         (1ull << FPSCR_DRN1)
 #define FP_DRN0         (1ull << FPSCR_DRN0)
diff --git a/target/ppc/fpu_helper.c b/target/ppc/fpu_helper.c
index f6c8318a71..f1ea4aa10e 100644
--- a/target/ppc/fpu_helper.c
+++ b/target/ppc/fpu_helper.c
@@ -370,7 +370,6 @@ static inline void float_inexact_excp(CPUPPCState *env)
 {
     CPUState *cs = env_cpu(env);
 
-    env->fpscr |= FP_FI;
     env->fpscr |= FP_XX;
     /* Update the floating-point exception summary */
     env->fpscr |= FP_FX;
@@ -462,7 +461,8 @@ void helper_fpscr_check_status(CPUPPCState *env)
     }
 }
 
-static void do_float_check_status(CPUPPCState *env, uintptr_t raddr)
+static void do_float_check_status(CPUPPCState *env, bool change_fi,
+                                  uintptr_t raddr)
 {
     CPUState *cs = env_cpu(env);
     int status = get_float_exception_flags(&env->fp_status);
@@ -474,8 +474,10 @@ static void do_float_check_status(CPUPPCState *env, uintptr_t raddr)
     }
     if (status & float_flag_inexact) {
         float_inexact_excp(env);
-    } else {
-        env->fpscr &= ~FP_FI; /* clear the FPSCR[FI] bit */
+    }
+    if (change_fi) {
+        env->fpscr = FIELD_DP64(env->fpscr, FPSCR, FI,
+                                !!(status & float_flag_inexact));
     }
 
     if (cs->exception_index == POWERPC_EXCP_PROGRAM &&
@@ -490,7 +492,7 @@ static void do_float_check_status(CPUPPCState *env, uintptr_t raddr)
 
 void helper_float_check_status(CPUPPCState *env)
 {
-    do_float_check_status(env, GETPC());
+    do_float_check_status(env, true, GETPC());
 }
 
 void helper_reset_fpstatus(CPUPPCState *env)
@@ -684,7 +686,7 @@ uint64_t helper_##op(CPUPPCState *env, uint64_t arg)       \
     } else {                                               \
         farg.d = cvtr(arg, &env->fp_status);               \
     }                                                      \
-    do_float_check_status(env, GETPC());                   \
+    do_float_check_status(env, true, GETPC());             \
     return farg.ll;                                        \
 }
 
@@ -710,7 +712,7 @@ static uint64_t do_fri(CPUPPCState *env, uint64_t arg,
 
     /* fri* does not set FPSCR[XX] */
     set_float_exception_flags(flags & ~float_flag_inexact, &env->fp_status);
-    do_float_check_status(env, GETPC());
+    do_float_check_status(env, true, GETPC());
 
     return arg;
 }
@@ -1721,7 +1723,7 @@ void helper_##name(CPUPPCState *env, ppc_vsr_t *xt,                          \
         }                                                                    \
     }                                                                        \
     *xt = t;                                                                 \
-    do_float_check_status(env, GETPC());                                     \
+    do_float_check_status(env, sfprf, GETPC());                              \
 }
 
 VSX_ADD_SUB(xsadddp, add, 1, float64, VsrD(0), 1, 0)
@@ -1757,7 +1759,7 @@ void helper_xsaddqp(CPUPPCState *env, uint32_t opcode,
     helper_compute_fprf_float128(env, t.f128);
 
     *xt = t;
-    do_float_check_status(env, GETPC());
+    do_float_check_status(env, true, GETPC());
 }
 
 /*
@@ -1798,7 +1800,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt,                            \
     }                                                                        \
                                                                              \
     *xt = t;                                                                 \
-    do_float_check_status(env, GETPC());                                     \
+    do_float_check_status(env, sfprf, GETPC());                              \
 }
 
 VSX_MUL(xsmuldp, 1, float64, VsrD(0), 1, 0)
@@ -1828,7 +1830,7 @@ void helper_xsmulqp(CPUPPCState *env, uint32_t opcode,
     helper_compute_fprf_float128(env, t.f128);
 
     *xt = t;
-    do_float_check_status(env, GETPC());
+    do_float_check_status(env, true, GETPC());
 }
 
 /*
@@ -1872,7 +1874,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt,                             \
     }                                                                         \
                                                                               \
     *xt = t;                                                                  \
-    do_float_check_status(env, GETPC());                                      \
+    do_float_check_status(env, sfprf, GETPC());                               \
 }
 
 VSX_DIV(xsdivdp, 1, float64, VsrD(0), 1, 0)
@@ -1905,7 +1907,7 @@ void helper_xsdivqp(CPUPPCState *env, uint32_t opcode,
 
     helper_compute_fprf_float128(env, t.f128);
     *xt = t;
-    do_float_check_status(env, GETPC());
+    do_float_check_status(env, true, GETPC());
 }
 
 /*
@@ -1940,7 +1942,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)              \
     }                                                                         \
                                                                               \
     *xt = t;                                                                  \
-    do_float_check_status(env, GETPC());                                      \
+    do_float_check_status(env, sfprf, GETPC());                               \
 }
 
 VSX_RE(xsredp, 1, float64, VsrD(0), 1, 0)
@@ -1985,7 +1987,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)             \
     }                                                                        \
                                                                              \
     *xt = t;                                                                 \
-    do_float_check_status(env, GETPC());                                     \
+    do_float_check_status(env, sfprf, GETPC());                              \
 }
 
 VSX_SQRT(xssqrtdp, 1, float64, VsrD(0), 1, 0)
@@ -2029,7 +2031,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)             \
     }                                                                        \
                                                                              \
     *xt = t;                                                                 \
-    do_float_check_status(env, GETPC());                                     \
+    do_float_check_status(env, sfprf, GETPC());                              \
 }
 
 VSX_RSQRTE(xsrsqrtedp, 1, float64, VsrD(0), 1, 0)
@@ -2182,7 +2184,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt,                             \
         }                                                                     \
     }                                                                         \
     *xt = t;                                                                  \
-    do_float_check_status(env, GETPC());                                      \
+    do_float_check_status(env, sfprf, GETPC());                               \
 }
 
 VSX_MADD(XSMADDDP, 1, float64, VsrD(0), MADD_FLGS, 1)
@@ -2234,7 +2236,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *s1, ppc_vsr_t *s2,\
                                                                                \
     helper_compute_fprf_float128(env, t.f128);                                 \
     *xt = t;                                                                   \
-    do_float_check_status(env, GETPC());                                       \
+    do_float_check_status(env, true, GETPC());                                 \
 }
 
 VSX_MADDQ(XSMADDQP, MADD_FLGS, 0)
@@ -2283,7 +2285,7 @@ VSX_MADDQ(XSNMSUBQPO, NMSUB_FLGS, 0)
                                                                               \
     memset(xt, 0, sizeof(*xt));                                               \
     memset(&xt->fld, -r, sizeof(xt->fld));                                    \
-    do_float_check_status(env, GETPC());                                      \
+    do_float_check_status(env, false, GETPC());                               \
 }
 
 VSX_SCALAR_CMP(XSCMPEQDP, float64, eq, VsrD(0), 0)
@@ -2319,7 +2321,7 @@ void helper_xscmpexpdp(CPUPPCState *env, uint32_t opcode,
     env->fpscr |= cc << FPSCR_FPCC;
     env->crf[BF(opcode)] = cc;
 
-    do_float_check_status(env, GETPC());
+    do_float_check_status(env, false, GETPC());
 }
 
 void helper_xscmpexpqp(CPUPPCState *env, uint32_t opcode,
@@ -2348,7 +2350,7 @@ void helper_xscmpexpqp(CPUPPCState *env, uint32_t opcode,
     env->fpscr |= cc << FPSCR_FPCC;
     env->crf[BF(opcode)] = cc;
 
-    do_float_check_status(env, GETPC());
+    do_float_check_status(env, false, GETPC());
 }
 
 static inline void do_scalar_cmp(CPUPPCState *env, ppc_vsr_t *xa, ppc_vsr_t *xb,
@@ -2401,7 +2403,7 @@ static inline void do_scalar_cmp(CPUPPCState *env, ppc_vsr_t *xa, ppc_vsr_t *xb,
         float_invalid_op_vxvc(env, 0, GETPC());
     }
 
-    do_float_check_status(env, GETPC());
+    do_float_check_status(env, false, GETPC());
 }
 
 void helper_xscmpodp(CPUPPCState *env, uint32_t opcode, ppc_vsr_t *xa,
@@ -2466,7 +2468,7 @@ static inline void do_scalar_cmpq(CPUPPCState *env, ppc_vsr_t *xa,
         float_invalid_op_vxvc(env, 0, GETPC());
     }
 
-    do_float_check_status(env, GETPC());
+    do_float_check_status(env, false, GETPC());
 }
 
 void helper_xscmpoqp(CPUPPCState *env, uint32_t opcode, ppc_vsr_t *xa,
@@ -2505,7 +2507,7 @@ void helper_##name(CPUPPCState *env, ppc_vsr_t *xt,                           \
     }                                                                         \
                                                                               \
     *xt = t;                                                                  \
-    do_float_check_status(env, GETPC());                                      \
+    do_float_check_status(env, false, GETPC());                               \
 }
 
 VSX_MAX_MIN(xsmaxdp, maxnum, 1, float64, VsrD(0))
@@ -2688,7 +2690,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)   \
     }                                                              \
                                                                    \
     *xt = t;                                                       \
-    do_float_check_status(env, GETPC());                           \
+    do_float_check_status(env, sfprf, GETPC());                    \
 }
 
 VSX_CVT_FP_TO_FP(xscvspdp, 1, float32, float64, VsrW(0), VsrD(0), 1)
@@ -2714,7 +2716,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)      \
     }                                                                 \
                                                                       \
     *xt = t;                                                          \
-    do_float_check_status(env, GETPC());                              \
+    do_float_check_status(env, sfprf, GETPC());                       \
 }
 
 VSX_CVT_FP_TO_FP2(xvcvdpsp, 2, float64, float32, 0)
@@ -2750,7 +2752,7 @@ void helper_##op(CPUPPCState *env, uint32_t opcode,                       \
     }                                                                   \
                                                                         \
     *xt = t;                                                            \
-    do_float_check_status(env, GETPC());                                \
+    do_float_check_status(env, true, GETPC());                          \
 }
 
 VSX_CVT_FP_TO_FP_VECTOR(xscvdpqp, 1, float64, float128, VsrD(0), f128, 1)
@@ -2785,7 +2787,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)   \
     }                                                              \
                                                                    \
     *xt = t;                                                       \
-    do_float_check_status(env, GETPC());                           \
+    do_float_check_status(env, sfprf, GETPC());                    \
 }
 
 VSX_CVT_FP_TO_FP_HP(xscvdphp, 1, float64, float16, VsrD(0), VsrH(3), 1)
@@ -2810,7 +2812,7 @@ void helper_XVCVSPBF16(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)
     }
 
     *xt = t;
-    do_float_check_status(env, GETPC());
+    do_float_check_status(env, false, GETPC());
 }
 
 void helper_XSCVQPDP(CPUPPCState *env, uint32_t ro, ppc_vsr_t *xt,
@@ -2833,7 +2835,7 @@ void helper_XSCVQPDP(CPUPPCState *env, uint32_t ro, ppc_vsr_t *xt,
     helper_compute_fprf_float64(env, t.VsrD(0));
 
     *xt = t;
-    do_float_check_status(env, GETPC());
+    do_float_check_status(env, true, GETPC());
 }
 
 uint64_t helper_xscvdpspn(CPUPPCState *env, uint64_t xb)
@@ -2889,9 +2891,10 @@ uint64_t helper_xscvspdpn(CPUPPCState *env, uint64_t xb)
  *   ttp   - target type (int32, uint32, int64 or uint64)
  *   sfld  - source vsr_t field
  *   tfld  - target vsr_t field
+ *   sfi   - set FI
  *   rnan  - resulting NaN
  */
-#define VSX_CVT_FP_TO_INT(op, nels, stp, ttp, sfld, tfld, rnan)              \
+#define VSX_CVT_FP_TO_INT(op, nels, stp, ttp, sfld, tfld, sfi, rnan)         \
 void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)             \
 {                                                                            \
     int all_flags = env->fp_status.float_exception_flags, flags;             \
@@ -2910,20 +2913,23 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)             \
                                                                              \
     *xt = t;                                                                 \
     env->fp_status.float_exception_flags = all_flags;                        \
-    do_float_check_status(env, GETPC());                                     \
+    do_float_check_status(env, sfi, GETPC());                                \
 }
 
-VSX_CVT_FP_TO_INT(xscvdpsxds, 1, float64, int64, VsrD(0), VsrD(0), \
+VSX_CVT_FP_TO_INT(xscvdpsxds, 1, float64, int64, VsrD(0), VsrD(0), true, \
                   0x8000000000000000ULL)
-VSX_CVT_FP_TO_INT(xscvdpuxds, 1, float64, uint64, VsrD(0), VsrD(0), 0ULL)
-VSX_CVT_FP_TO_INT(xvcvdpsxds, 2, float64, int64, VsrD(i), VsrD(i), \
+VSX_CVT_FP_TO_INT(xscvdpuxds, 1, float64, uint64, VsrD(0), VsrD(0), true, 0ULL)
+VSX_CVT_FP_TO_INT(xvcvdpsxds, 2, float64, int64, VsrD(i), VsrD(i), false, \
                   0x8000000000000000ULL)
-VSX_CVT_FP_TO_INT(xvcvdpuxds, 2, float64, uint64, VsrD(i), VsrD(i), 0ULL)
-VSX_CVT_FP_TO_INT(xvcvspsxds, 2, float32, int64, VsrW(2 * i), VsrD(i), \
+VSX_CVT_FP_TO_INT(xvcvdpuxds, 2, float64, uint64, VsrD(i), VsrD(i), false, \
+                  0ULL)
+VSX_CVT_FP_TO_INT(xvcvspsxds, 2, float32, int64, VsrW(2 * i), VsrD(i), false, \
                   0x8000000000000000ULL)
-VSX_CVT_FP_TO_INT(xvcvspsxws, 4, float32, int32, VsrW(i), VsrW(i), 0x80000000U)
-VSX_CVT_FP_TO_INT(xvcvspuxds, 2, float32, uint64, VsrW(2 * i), VsrD(i), 0ULL)
-VSX_CVT_FP_TO_INT(xvcvspuxws, 4, float32, uint32, VsrW(i), VsrW(i), 0U)
+VSX_CVT_FP_TO_INT(xvcvspsxws, 4, float32, int32, VsrW(i), VsrW(i), false, \
+                  0x80000000ULL)
+VSX_CVT_FP_TO_INT(xvcvspuxds, 2, float32, uint64, VsrW(2 * i), VsrD(i), \
+                  false, 0ULL)
+VSX_CVT_FP_TO_INT(xvcvspuxws, 4, float32, uint32, VsrW(i), VsrW(i), false, 0U)
 
 #define VSX_CVT_FP_TO_INT128(op, tp, rnan)                                     \
 void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)               \
@@ -2940,7 +2946,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)               \
     }                                                                          \
                                                                                \
     *xt = t;                                                                   \
-    do_float_check_status(env, GETPC());                                       \
+    do_float_check_status(env, true, GETPC());                                 \
 }
 
 VSX_CVT_FP_TO_INT128(XSCVQPUQZ, uint128, 0)
@@ -2955,7 +2961,7 @@ VSX_CVT_FP_TO_INT128(XSCVQPSQZ, int128, 0x8000000000000000ULL);
  *     words 0 and 1 (and words 2 and 3) of the result register, as
  *     is required by this version of the architecture.
  */
-#define VSX_CVT_FP_TO_INT2(op, nels, stp, ttp, rnan)                         \
+#define VSX_CVT_FP_TO_INT2(op, nels, stp, ttp, sfi, rnan)                    \
 void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)             \
 {                                                                            \
     int all_flags = env->fp_status.float_exception_flags, flags;             \
@@ -2977,13 +2983,13 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)             \
                                                                              \
     *xt = t;                                                                 \
     env->fp_status.float_exception_flags = all_flags;                        \
-    do_float_check_status(env, GETPC());                                     \
+    do_float_check_status(env, sfi, GETPC());                                \
 }
 
-VSX_CVT_FP_TO_INT2(xscvdpsxws, 1, float64, int32, 0x80000000U)
-VSX_CVT_FP_TO_INT2(xscvdpuxws, 1, float64, uint32, 0U)
-VSX_CVT_FP_TO_INT2(xvcvdpsxws, 2, float64, int32, 0x80000000U)
-VSX_CVT_FP_TO_INT2(xvcvdpuxws, 2, float64, uint32, 0U)
+VSX_CVT_FP_TO_INT2(xscvdpsxws, 1, float64, int32, true, 0x80000000U)
+VSX_CVT_FP_TO_INT2(xscvdpuxws, 1, float64, uint32, true, 0U)
+VSX_CVT_FP_TO_INT2(xvcvdpsxws, 2, float64, int32, false, 0x80000000U)
+VSX_CVT_FP_TO_INT2(xvcvdpuxws, 2, float64, uint32, false, 0U)
 
 /*
  * VSX_CVT_FP_TO_INT_VECTOR - VSX floating point to integer conversion
@@ -3008,7 +3014,7 @@ void helper_##op(CPUPPCState *env, uint32_t opcode,                          \
     }                                                                        \
                                                                              \
     *xt = t;                                                                 \
-    do_float_check_status(env, GETPC());                                     \
+    do_float_check_status(env, true, GETPC());                               \
 }
 
 VSX_CVT_FP_TO_INT_VECTOR(xscvqpsdz, float128, int64, f128, VsrD(0),          \
@@ -3047,7 +3053,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)        \
     }                                                                   \
                                                                         \
     *xt = t;                                                            \
-    do_float_check_status(env, GETPC());                                \
+    do_float_check_status(env, sfprf, GETPC());                         \
 }
 
 VSX_CVT_INT_TO_FP(xscvsxddp, 1, int64, float64, VsrD(0), VsrD(0), 1, 0)
@@ -3073,7 +3079,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)        \
     }                                                                   \
                                                                         \
     *xt = t;                                                            \
-    do_float_check_status(env, GETPC());                                \
+    do_float_check_status(env, false, GETPC());                         \
 }
 
 VSX_CVT_INT_TO_FP2(xvcvsxdsp, int64, float32)
@@ -3085,7 +3091,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)\
     helper_reset_fpstatus(env);                                 \
     xt->f128 = tp##_to_float128(xb->s128, &env->fp_status);     \
     helper_compute_fprf_float128(env, xt->f128);                \
-    do_float_check_status(env, GETPC());                        \
+    do_float_check_status(env, true, GETPC());                  \
 }
 
 VSX_CVT_INT128_TO_FP(XSCVUQQP, uint128);
@@ -3109,7 +3115,7 @@ void helper_##op(CPUPPCState *env, uint32_t opcode,                     \
     helper_compute_fprf_##ttp(env, t.tfld);                             \
                                                                         \
     *xt = t;                                                            \
-    do_float_check_status(env, GETPC());                                \
+    do_float_check_status(env, true, GETPC());                          \
 }
 
 VSX_CVT_INT_TO_FP_VECTOR(xscvsdqp, int64, float128, VsrD(0), f128)
@@ -3167,7 +3173,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)       \
     }                                                                  \
                                                                        \
     *xt = t;                                                           \
-    do_float_check_status(env, GETPC());                               \
+    do_float_check_status(env, sfprf, GETPC());                        \
 }
 
 VSX_ROUND(xsrdpi, 1, float64, VsrD(0), float_round_ties_away, 1)
@@ -3195,7 +3201,7 @@ uint64_t helper_xsrsp(CPUPPCState *env, uint64_t xb)
     uint64_t xt = do_frsp(env, xb, GETPC());
 
     helper_compute_fprf_float64(env, xt);
-    do_float_check_status(env, GETPC());
+    do_float_check_status(env, true, GETPC());
     return xt;
 }
 
@@ -3355,7 +3361,7 @@ void helper_xsrqpi(CPUPPCState *env, uint32_t opcode,
     }
 
     helper_compute_fprf_float128(env, t.f128);
-    do_float_check_status(env, GETPC());
+    do_float_check_status(env, true, GETPC());
     *xt = t;
 }
 
@@ -3408,7 +3414,7 @@ void helper_xsrqpxp(CPUPPCState *env, uint32_t opcode,
 
     helper_compute_fprf_float128(env, t.f128);
     *xt = t;
-    do_float_check_status(env, GETPC());
+    do_float_check_status(env, true, GETPC());
 }
 
 void helper_xssqrtqp(CPUPPCState *env, uint32_t opcode,
@@ -3434,7 +3440,7 @@ void helper_xssqrtqp(CPUPPCState *env, uint32_t opcode,
 
     helper_compute_fprf_float128(env, t.f128);
     *xt = t;
-    do_float_check_status(env, GETPC());
+    do_float_check_status(env, true, GETPC());
 }
 
 void helper_xssubqp(CPUPPCState *env, uint32_t opcode,
@@ -3460,5 +3466,5 @@ void helper_xssubqp(CPUPPCState *env, uint32_t opcode,
 
     helper_compute_fprf_float128(env, t.f128);
     *xt = t;
-    do_float_check_status(env, GETPC());
+    do_float_check_status(env, true, GETPC());
 }

From c582a1dbc8e44f5e976ce9c2ac4ce0bc38a33cae Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Tue, 17 May 2022 13:15:21 -0300
Subject: [PATCH 539/935] target/ppc: Fix FPSCR.FI changing in
 float_overflow_excp()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This patch fixes another not-so-clear situation in Power ISA
regarding the inexact bits in FPSCR. The ISA states that:

"""
When Overflow Exception is disabled (OE=0) and an
Overflow Exception occurs, the following actions are
taken:
...
2. Inexact Exception is set
XX <- 1
...
FI is set to 1
...
"""

However, when tested on a Power 9 hardware, some instructions that
trigger an OX don't set the FI bit:

xvcvdpsp(0x4050533fcdb7b95ff8d561c40bf90996) = FI: CLEARED -> CLEARED
xvnmsubmsp(0xf3c0c1fc8f3230, 0xbeaab9c5) = FI: CLEARED -> CLEARED
(just a few examples. Other instructions are also affected)

The root cause for this seems to be that only instructions that list
the bit FI in the "Special Registers Altered" should modify it.

QEMU is, today, not working like the hardware:

xvcvdpsp(0x4050533fcdb7b95ff8d561c40bf90996) = FI: CLEARED -> SET
xvnmsubmsp(0xf3c0c1fc8f3230, 0xbeaab9c5) = FI: CLEARED -> SET

(all tests assume FI is cleared beforehand)

Fix this by making float_overflow_excp() return float_flag_inexact
if it should update the inexact flags.

Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Rashmica Gupta <rashmica.g@gmail.com>
Message-Id: <20220517161522.36132-3-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/fpu_helper.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/target/ppc/fpu_helper.c b/target/ppc/fpu_helper.c
index f1ea4aa10e..88f9e756a5 100644
--- a/target/ppc/fpu_helper.c
+++ b/target/ppc/fpu_helper.c
@@ -329,24 +329,25 @@ static inline void float_zero_divide_excp(CPUPPCState *env, uintptr_t raddr)
     }
 }
 
-static inline void float_overflow_excp(CPUPPCState *env)
+static inline int float_overflow_excp(CPUPPCState *env)
 {
     CPUState *cs = env_cpu(env);
 
     env->fpscr |= FP_OX;
     /* Update the floating-point exception summary */
     env->fpscr |= FP_FX;
-    if (env->fpscr & FP_OE) {
+
+    bool overflow_enabled = !!(env->fpscr & FP_OE);
+    if (overflow_enabled) {
         /* XXX: should adjust the result */
         /* Update the floating-point enabled exception summary */
         env->fpscr |= FP_FEX;
         /* We must update the target FPR before raising the exception */
         cs->exception_index = POWERPC_EXCP_PROGRAM;
         env->error_code = POWERPC_EXCP_FP | POWERPC_EXCP_FP_OX;
-    } else {
-        env->fpscr |= FP_XX;
-        env->fpscr |= FP_FI;
     }
+
+    return overflow_enabled ? 0 : float_flag_inexact;
 }
 
 static inline void float_underflow_excp(CPUPPCState *env)
@@ -468,7 +469,7 @@ static void do_float_check_status(CPUPPCState *env, bool change_fi,
     int status = get_float_exception_flags(&env->fp_status);
 
     if (status & float_flag_overflow) {
-        float_overflow_excp(env);
+        status |= float_overflow_excp(env);
     } else if (status & float_flag_underflow) {
         float_underflow_excp(env);
     }

From dd657a35b429e9affa373a0e6162dd930854ba78 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?V=C3=ADctor=20Colombo?= <victor.colombo@eldorado.org.br>
Date: Tue, 17 May 2022 13:15:22 -0300
Subject: [PATCH 540/935] target/ppc: Rename sfprf to sfifprf where it's also
 used as set fi flag
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The bit FI fix used the sfprf flag as a flag for the set_fi parameter
in do_float_check_status where applicable. Now, this patch rename this
flag to sfifprf to state this dual usage.

Signed-off-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Rashmica Gupta <rashmica.g@gmail.com>
Message-Id: <20220517161522.36132-4-victor.colombo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/fpu_helper.c | 112 ++++++++++++++++++++--------------------
 1 file changed, 56 insertions(+), 56 deletions(-)

diff --git a/target/ppc/fpu_helper.c b/target/ppc/fpu_helper.c
index 88f9e756a5..8592727792 100644
--- a/target/ppc/fpu_helper.c
+++ b/target/ppc/fpu_helper.c
@@ -1693,9 +1693,9 @@ uint32_t helper_efdcmpeq(CPUPPCState *env, uint64_t op1, uint64_t op2)
  *   nels  - number of elements (1, 2 or 4)
  *   tp    - type (float32 or float64)
  *   fld   - vsr_t field (VsrD(*) or VsrW(*))
- *   sfprf - set FPRF
+ *   sfifprf - set FI and FPRF
  */
-#define VSX_ADD_SUB(name, op, nels, tp, fld, sfprf, r2sp)                    \
+#define VSX_ADD_SUB(name, op, nels, tp, fld, sfifprf, r2sp)                  \
 void helper_##name(CPUPPCState *env, ppc_vsr_t *xt,                          \
                    ppc_vsr_t *xa, ppc_vsr_t *xb)                             \
 {                                                                            \
@@ -1712,19 +1712,19 @@ void helper_##name(CPUPPCState *env, ppc_vsr_t *xt,                          \
                                                                              \
         if (unlikely(tstat.float_exception_flags & float_flag_invalid)) {    \
             float_invalid_op_addsub(env, tstat.float_exception_flags,        \
-                                    sfprf, GETPC());                         \
+                                    sfifprf, GETPC());                       \
         }                                                                    \
                                                                              \
         if (r2sp) {                                                          \
             t.fld = do_frsp(env, t.fld, GETPC());                            \
         }                                                                    \
                                                                              \
-        if (sfprf) {                                                         \
+        if (sfifprf) {                                                       \
             helper_compute_fprf_float64(env, t.fld);                         \
         }                                                                    \
     }                                                                        \
     *xt = t;                                                                 \
-    do_float_check_status(env, sfprf, GETPC());                              \
+    do_float_check_status(env, sfifprf, GETPC());                            \
 }
 
 VSX_ADD_SUB(xsadddp, add, 1, float64, VsrD(0), 1, 0)
@@ -1769,9 +1769,9 @@ void helper_xsaddqp(CPUPPCState *env, uint32_t opcode,
  *   nels  - number of elements (1, 2 or 4)
  *   tp    - type (float32 or float64)
  *   fld   - vsr_t field (VsrD(*) or VsrW(*))
- *   sfprf - set FPRF
+ *   sfifprf - set FI and FPRF
  */
-#define VSX_MUL(op, nels, tp, fld, sfprf, r2sp)                              \
+#define VSX_MUL(op, nels, tp, fld, sfifprf, r2sp)                            \
 void helper_##op(CPUPPCState *env, ppc_vsr_t *xt,                            \
                  ppc_vsr_t *xa, ppc_vsr_t *xb)                               \
 {                                                                            \
@@ -1788,20 +1788,20 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt,                            \
                                                                              \
         if (unlikely(tstat.float_exception_flags & float_flag_invalid)) {    \
             float_invalid_op_mul(env, tstat.float_exception_flags,           \
-                                 sfprf, GETPC());                            \
+                                 sfifprf, GETPC());                          \
         }                                                                    \
                                                                              \
         if (r2sp) {                                                          \
             t.fld = do_frsp(env, t.fld, GETPC());                            \
         }                                                                    \
                                                                              \
-        if (sfprf) {                                                         \
+        if (sfifprf) {                                                       \
             helper_compute_fprf_float64(env, t.fld);                         \
         }                                                                    \
     }                                                                        \
                                                                              \
     *xt = t;                                                                 \
-    do_float_check_status(env, sfprf, GETPC());                              \
+    do_float_check_status(env, sfifprf, GETPC());                            \
 }
 
 VSX_MUL(xsmuldp, 1, float64, VsrD(0), 1, 0)
@@ -1840,9 +1840,9 @@ void helper_xsmulqp(CPUPPCState *env, uint32_t opcode,
  *   nels  - number of elements (1, 2 or 4)
  *   tp    - type (float32 or float64)
  *   fld   - vsr_t field (VsrD(*) or VsrW(*))
- *   sfprf - set FPRF
+ *   sfifprf - set FI and FPRF
  */
-#define VSX_DIV(op, nels, tp, fld, sfprf, r2sp)                               \
+#define VSX_DIV(op, nels, tp, fld, sfifprf, r2sp)                             \
 void helper_##op(CPUPPCState *env, ppc_vsr_t *xt,                             \
                  ppc_vsr_t *xa, ppc_vsr_t *xb)                                \
 {                                                                             \
@@ -1859,7 +1859,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt,                             \
                                                                               \
         if (unlikely(tstat.float_exception_flags & float_flag_invalid)) {     \
             float_invalid_op_div(env, tstat.float_exception_flags,            \
-                                 sfprf, GETPC());                             \
+                                 sfifprf, GETPC());                           \
         }                                                                     \
         if (unlikely(tstat.float_exception_flags & float_flag_divbyzero)) {   \
             float_zero_divide_excp(env, GETPC());                             \
@@ -1869,13 +1869,13 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt,                             \
             t.fld = do_frsp(env, t.fld, GETPC());                             \
         }                                                                     \
                                                                               \
-        if (sfprf) {                                                          \
+        if (sfifprf) {                                                        \
             helper_compute_fprf_float64(env, t.fld);                          \
         }                                                                     \
     }                                                                         \
                                                                               \
     *xt = t;                                                                  \
-    do_float_check_status(env, sfprf, GETPC());                               \
+    do_float_check_status(env, sfifprf, GETPC());                             \
 }
 
 VSX_DIV(xsdivdp, 1, float64, VsrD(0), 1, 0)
@@ -1917,9 +1917,9 @@ void helper_xsdivqp(CPUPPCState *env, uint32_t opcode,
  *   nels  - number of elements (1, 2 or 4)
  *   tp    - type (float32 or float64)
  *   fld   - vsr_t field (VsrD(*) or VsrW(*))
- *   sfprf - set FPRF
+ *   sfifprf - set FI and FPRF
  */
-#define VSX_RE(op, nels, tp, fld, sfprf, r2sp)                                \
+#define VSX_RE(op, nels, tp, fld, sfifprf, r2sp)                              \
 void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)              \
 {                                                                             \
     ppc_vsr_t t = { };                                                        \
@@ -1937,13 +1937,13 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)              \
             t.fld = do_frsp(env, t.fld, GETPC());                             \
         }                                                                     \
                                                                               \
-        if (sfprf) {                                                          \
+        if (sfifprf) {                                                        \
             helper_compute_fprf_float64(env, t.fld);                          \
         }                                                                     \
     }                                                                         \
                                                                               \
     *xt = t;                                                                  \
-    do_float_check_status(env, sfprf, GETPC());                               \
+    do_float_check_status(env, sfifprf, GETPC());                             \
 }
 
 VSX_RE(xsredp, 1, float64, VsrD(0), 1, 0)
@@ -1957,9 +1957,9 @@ VSX_RE(xvresp, 4, float32, VsrW(i), 0, 0)
  *   nels  - number of elements (1, 2 or 4)
  *   tp    - type (float32 or float64)
  *   fld   - vsr_t field (VsrD(*) or VsrW(*))
- *   sfprf - set FPRF
+ *   sfifprf - set FI and FPRF
  */
-#define VSX_SQRT(op, nels, tp, fld, sfprf, r2sp)                             \
+#define VSX_SQRT(op, nels, tp, fld, sfifprf, r2sp)                           \
 void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)             \
 {                                                                            \
     ppc_vsr_t t = { };                                                       \
@@ -1975,20 +1975,20 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)             \
                                                                              \
         if (unlikely(tstat.float_exception_flags & float_flag_invalid)) {    \
             float_invalid_op_sqrt(env, tstat.float_exception_flags,          \
-                                  sfprf, GETPC());                           \
+                                  sfifprf, GETPC());                         \
         }                                                                    \
                                                                              \
         if (r2sp) {                                                          \
             t.fld = do_frsp(env, t.fld, GETPC());                            \
         }                                                                    \
                                                                              \
-        if (sfprf) {                                                         \
+        if (sfifprf) {                                                       \
             helper_compute_fprf_float64(env, t.fld);                         \
         }                                                                    \
     }                                                                        \
                                                                              \
     *xt = t;                                                                 \
-    do_float_check_status(env, sfprf, GETPC());                              \
+    do_float_check_status(env, sfifprf, GETPC());                            \
 }
 
 VSX_SQRT(xssqrtdp, 1, float64, VsrD(0), 1, 0)
@@ -2002,9 +2002,9 @@ VSX_SQRT(xvsqrtsp, 4, float32, VsrW(i), 0, 0)
  *   nels  - number of elements (1, 2 or 4)
  *   tp    - type (float32 or float64)
  *   fld   - vsr_t field (VsrD(*) or VsrW(*))
- *   sfprf - set FPRF
+ *   sfifprf - set FI and FPRF
  */
-#define VSX_RSQRTE(op, nels, tp, fld, sfprf, r2sp)                           \
+#define VSX_RSQRTE(op, nels, tp, fld, sfifprf, r2sp)                         \
 void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)             \
 {                                                                            \
     ppc_vsr_t t = { };                                                       \
@@ -2020,19 +2020,19 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)             \
         env->fp_status.float_exception_flags |= tstat.float_exception_flags; \
         if (unlikely(tstat.float_exception_flags & float_flag_invalid)) {    \
             float_invalid_op_sqrt(env, tstat.float_exception_flags,          \
-                                  sfprf, GETPC());                           \
+                                  sfifprf, GETPC());                         \
         }                                                                    \
         if (r2sp) {                                                          \
             t.fld = do_frsp(env, t.fld, GETPC());                            \
         }                                                                    \
                                                                              \
-        if (sfprf) {                                                         \
+        if (sfifprf) {                                                       \
             helper_compute_fprf_float64(env, t.fld);                         \
         }                                                                    \
     }                                                                        \
                                                                              \
     *xt = t;                                                                 \
-    do_float_check_status(env, sfprf, GETPC());                              \
+    do_float_check_status(env, sfifprf, GETPC());                            \
 }
 
 VSX_RSQRTE(xsrsqrtedp, 1, float64, VsrD(0), 1, 0)
@@ -2158,9 +2158,9 @@ VSX_TSQRT(xvtsqrtsp, 4, float32, VsrW(i), -126, 23)
  *   fld   - vsr_t field (VsrD(*) or VsrW(*))
  *   maddflgs - flags for the float*muladd routine that control the
  *           various forms (madd, msub, nmadd, nmsub)
- *   sfprf - set FPRF
+ *   sfifprf - set FI and FPRF
  */
-#define VSX_MADD(op, nels, tp, fld, maddflgs, sfprf)                          \
+#define VSX_MADD(op, nels, tp, fld, maddflgs, sfifprf)                        \
 void helper_##op(CPUPPCState *env, ppc_vsr_t *xt,                             \
                  ppc_vsr_t *s1, ppc_vsr_t *s2, ppc_vsr_t *s3)                 \
 {                                                                             \
@@ -2177,15 +2177,15 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt,                             \
                                                                               \
         if (unlikely(tstat.float_exception_flags & float_flag_invalid)) {     \
             float_invalid_op_madd(env, tstat.float_exception_flags,           \
-                                  sfprf, GETPC());                            \
+                                  sfifprf, GETPC());                          \
         }                                                                     \
                                                                               \
-        if (sfprf) {                                                          \
+        if (sfifprf) {                                                        \
             helper_compute_fprf_float64(env, t.fld);                          \
         }                                                                     \
     }                                                                         \
     *xt = t;                                                                  \
-    do_float_check_status(env, sfprf, GETPC());                               \
+    do_float_check_status(env, sfifprf, GETPC());                             \
 }
 
 VSX_MADD(XSMADDDP, 1, float64, VsrD(0), MADD_FLGS, 1)
@@ -2670,9 +2670,9 @@ VSX_CMP(xvcmpnesp, 4, float32, VsrW(i), eq, 0, 0)
  *   ttp   - target type (float32 or float64)
  *   sfld  - source vsr_t field
  *   tfld  - target vsr_t field (f32 or f64)
- *   sfprf - set FPRF
+ *   sfifprf - set FI and FPRF
  */
-#define VSX_CVT_FP_TO_FP(op, nels, stp, ttp, sfld, tfld, sfprf)    \
+#define VSX_CVT_FP_TO_FP(op, nels, stp, ttp, sfld, tfld, sfifprf)  \
 void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)   \
 {                                                                  \
     ppc_vsr_t t = { };                                             \
@@ -2685,19 +2685,19 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)   \
             float_invalid_op_vxsnan(env, GETPC());                 \
             t.tfld = ttp##_snan_to_qnan(t.tfld);                   \
         }                                                          \
-        if (sfprf) {                                               \
+        if (sfifprf) {                                             \
             helper_compute_fprf_##ttp(env, t.tfld);                \
         }                                                          \
     }                                                              \
                                                                    \
     *xt = t;                                                       \
-    do_float_check_status(env, sfprf, GETPC());                    \
+    do_float_check_status(env, sfifprf, GETPC());                  \
 }
 
 VSX_CVT_FP_TO_FP(xscvspdp, 1, float32, float64, VsrW(0), VsrD(0), 1)
 VSX_CVT_FP_TO_FP(xvcvspdp, 2, float32, float64, VsrW(2 * i), VsrD(i), 0)
 
-#define VSX_CVT_FP_TO_FP2(op, nels, stp, ttp, sfprf)                  \
+#define VSX_CVT_FP_TO_FP2(op, nels, stp, ttp, sfifprf)                \
 void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)      \
 {                                                                     \
     ppc_vsr_t t = { };                                                \
@@ -2710,14 +2710,14 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)      \
             float_invalid_op_vxsnan(env, GETPC());                    \
             t.VsrW(2 * i) = ttp##_snan_to_qnan(t.VsrW(2 * i));        \
         }                                                             \
-        if (sfprf) {                                                  \
+        if (sfifprf) {                                                \
             helper_compute_fprf_##ttp(env, t.VsrW(2 * i));            \
         }                                                             \
         t.VsrW(2 * i + 1) = t.VsrW(2 * i);                            \
     }                                                                 \
                                                                       \
     *xt = t;                                                          \
-    do_float_check_status(env, sfprf, GETPC());                       \
+    do_float_check_status(env, sfifprf, GETPC());                     \
 }
 
 VSX_CVT_FP_TO_FP2(xvcvdpsp, 2, float64, float32, 0)
@@ -2733,9 +2733,9 @@ VSX_CVT_FP_TO_FP2(xscvdpsp, 1, float64, float32, 1)
  *   tfld  - target vsr_t field (f32 or f64)
  *   sfprf - set FPRF
  */
-#define VSX_CVT_FP_TO_FP_VECTOR(op, nels, stp, ttp, sfld, tfld, sfprf)    \
-void helper_##op(CPUPPCState *env, uint32_t opcode,                       \
-                 ppc_vsr_t *xt, ppc_vsr_t *xb)                            \
+#define VSX_CVT_FP_TO_FP_VECTOR(op, nels, stp, ttp, sfld, tfld, sfprf)  \
+void helper_##op(CPUPPCState *env, uint32_t opcode,                     \
+                 ppc_vsr_t *xt, ppc_vsr_t *xb)                          \
 {                                                                       \
     ppc_vsr_t t = *xt;                                                  \
     int i;                                                              \
@@ -2767,9 +2767,9 @@ VSX_CVT_FP_TO_FP_VECTOR(xscvdpqp, 1, float64, float128, VsrD(0), f128, 1)
  *   ttp   - target type
  *   sfld  - source vsr_t field
  *   tfld  - target vsr_t field
- *   sfprf - set FPRF
+ *   sfifprf - set FI and FPRF
  */
-#define VSX_CVT_FP_TO_FP_HP(op, nels, stp, ttp, sfld, tfld, sfprf) \
+#define VSX_CVT_FP_TO_FP_HP(op, nels, stp, ttp, sfld, tfld, sfifprf) \
 void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)   \
 {                                                                  \
     ppc_vsr_t t = { };                                             \
@@ -2782,13 +2782,13 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)   \
             float_invalid_op_vxsnan(env, GETPC());                 \
             t.tfld = ttp##_snan_to_qnan(t.tfld);                   \
         }                                                          \
-        if (sfprf) {                                               \
+        if (sfifprf) {                                             \
             helper_compute_fprf_##ttp(env, t.tfld);                \
         }                                                          \
     }                                                              \
                                                                    \
     *xt = t;                                                       \
-    do_float_check_status(env, sfprf, GETPC());                    \
+    do_float_check_status(env, sfifprf, GETPC());                  \
 }
 
 VSX_CVT_FP_TO_FP_HP(xscvdphp, 1, float64, float16, VsrD(0), VsrH(3), 1)
@@ -3035,9 +3035,9 @@ VSX_CVT_FP_TO_INT_VECTOR(xscvqpuwz, float128, uint32, f128, VsrD(0), 0x0ULL)
  *   sfld  - source vsr_t field
  *   tfld  - target vsr_t field
  *   jdef  - definition of the j index (i or 2*i)
- *   sfprf - set FPRF
+ *   sfifprf - set FI and FPRF
  */
-#define VSX_CVT_INT_TO_FP(op, nels, stp, ttp, sfld, tfld, sfprf, r2sp)  \
+#define VSX_CVT_INT_TO_FP(op, nels, stp, ttp, sfld, tfld, sfifprf, r2sp)\
 void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)        \
 {                                                                       \
     ppc_vsr_t t = { };                                                  \
@@ -3048,13 +3048,13 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)        \
         if (r2sp) {                                                     \
             t.tfld = do_frsp(env, t.tfld, GETPC());                     \
         }                                                               \
-        if (sfprf) {                                                    \
+        if (sfifprf) {                                                  \
             helper_compute_fprf_float64(env, t.tfld);                   \
         }                                                               \
     }                                                                   \
                                                                         \
     *xt = t;                                                            \
-    do_float_check_status(env, sfprf, GETPC());                         \
+    do_float_check_status(env, sfifprf, GETPC());                       \
 }
 
 VSX_CVT_INT_TO_FP(xscvsxddp, 1, int64, float64, VsrD(0), VsrD(0), 1, 0)
@@ -3136,9 +3136,9 @@ VSX_CVT_INT_TO_FP_VECTOR(xscvudqp, uint64, float128, VsrD(0), f128)
  *   tp    - type (float32 or float64)
  *   fld   - vsr_t field (VsrD(*) or VsrW(*))
  *   rmode - rounding mode
- *   sfprf - set FPRF
+ *   sfifprf - set FI and FPRF
  */
-#define VSX_ROUND(op, nels, tp, fld, rmode, sfprf)                     \
+#define VSX_ROUND(op, nels, tp, fld, rmode, sfifprf)                   \
 void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)       \
 {                                                                      \
     ppc_vsr_t t = { };                                                 \
@@ -3158,7 +3158,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)       \
         } else {                                                       \
             t.fld = tp##_round_to_int(xb->fld, &env->fp_status);       \
         }                                                              \
-        if (sfprf) {                                                   \
+        if (sfifprf) {                                                 \
             helper_compute_fprf_float64(env, t.fld);                   \
         }                                                              \
     }                                                                  \
@@ -3174,7 +3174,7 @@ void helper_##op(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)       \
     }                                                                  \
                                                                        \
     *xt = t;                                                           \
-    do_float_check_status(env, sfprf, GETPC());                        \
+    do_float_check_status(env, sfifprf, GETPC());                      \
 }
 
 VSX_ROUND(xsrdpi, 1, float64, VsrD(0), float_round_ties_away, 1)

From d5aa9e79048bfc42616949391591fe7947574e5d Mon Sep 17 00:00:00 2001
From: Frederic Barrat <fbarrat@linux.ibm.com>
Date: Mon, 23 May 2022 17:18:59 +0200
Subject: [PATCH 541/935] pnv/xive2: Don't overwrite PC registers when writing
 TCTXT registers
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

When writing a register from the TCTXT memory region (4th page within
the IC BAR), we were overwriting the Presentation Controller (PC)
register at the same offset. It looks like a silly cut and paste
error.

We were somehow lucky: the TCTXT registers being touched are
TCTXT_ENx/_SET/_RESET to enable physical threads and the PC registers
at the same offset are either not used by our model or the update was
harmless.

Found through code inspection.

Signed-off-by: Frederic Barrat <fbarrat@linux.ibm.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220523151859.72283-1-fbarrat@linux.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/intc/pnv_xive2.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/hw/intc/pnv_xive2.c b/hw/intc/pnv_xive2.c
index 87303b4064..a39e070e82 100644
--- a/hw/intc/pnv_xive2.c
+++ b/hw/intc/pnv_xive2.c
@@ -1295,7 +1295,6 @@ static void pnv_xive2_ic_tctxt_write(void *opaque, hwaddr offset,
                                      uint64_t val, unsigned size)
 {
     PnvXive2 *xive = PNV_XIVE2(opaque);
-    uint32_t reg = offset >> 3;
 
     switch (offset) {
     /*
@@ -1322,8 +1321,6 @@ static void pnv_xive2_ic_tctxt_write(void *opaque, hwaddr offset,
         xive2_error(xive, "TCTXT: invalid write @%"HWADDR_PRIx, offset);
         return;
     }
-
-    xive->pc_regs[reg] = val;
 }
 
 static const MemoryRegionOps pnv_xive2_ic_tctxt_ops = {

From 8f6086044b4de2907ee9660690b709b488b3d55b Mon Sep 17 00:00:00 2001
From: Matheus Ferst <matheus.ferst@eldorado.org.br>
Date: Tue, 17 May 2022 09:39:18 -0300
Subject: [PATCH 542/935] target/ppc: declare darn32/darn64 helpers with
 TCG_CALL_NO_RWG

Signed-off-by: Matheus Ferst <matheus.ferst@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517123929.284511-2-matheus.ferst@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/helper.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/ppc/helper.h b/target/ppc/helper.h
index aa6773c4a5..44eb6b7b7c 100644
--- a/target/ppc/helper.h
+++ b/target/ppc/helper.h
@@ -59,8 +59,8 @@ DEF_HELPER_FLAGS_2(cmpeqb, TCG_CALL_NO_RWG_SE, i32, tl, tl)
 DEF_HELPER_FLAGS_1(popcntw, TCG_CALL_NO_RWG_SE, tl, tl)
 DEF_HELPER_FLAGS_2(bpermd, TCG_CALL_NO_RWG_SE, i64, i64, i64)
 DEF_HELPER_3(srad, tl, env, tl, tl)
-DEF_HELPER_0(darn32, tl)
-DEF_HELPER_0(darn64, tl)
+DEF_HELPER_FLAGS_0(darn32, TCG_CALL_NO_RWG, tl)
+DEF_HELPER_FLAGS_0(darn64, TCG_CALL_NO_RWG, tl)
 #endif
 
 DEF_HELPER_FLAGS_1(cntlsw32, TCG_CALL_NO_RWG_SE, i32, i32)

From 9aa898b8977abca0af6b2ed9ec0f7cfa20b9d7f5 Mon Sep 17 00:00:00 2001
From: Matheus Ferst <matheus.ferst@eldorado.org.br>
Date: Tue, 17 May 2022 09:39:19 -0300
Subject: [PATCH 543/935] target/ppc: use TCG_CALL_NO_RWG in vector helpers
 without env

Helpers of vector instructions without cpu_env as an argument do not
access globals.

Signed-off-by: Matheus Ferst <matheus.ferst@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517123929.284511-3-matheus.ferst@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/helper.h | 162 ++++++++++++++++++++++----------------------
 1 file changed, 81 insertions(+), 81 deletions(-)

diff --git a/target/ppc/helper.h b/target/ppc/helper.h
index 44eb6b7b7c..da740ad9af 100644
--- a/target/ppc/helper.h
+++ b/target/ppc/helper.h
@@ -133,15 +133,15 @@ DEF_HELPER_FLAGS_1(ftsqrt, TCG_CALL_NO_RWG_SE, i32, i64)
 #define dh_ctype_vsr ppc_vsr_t *
 #define dh_typecode_vsr dh_typecode_ptr
 
-DEF_HELPER_3(vavgub, void, avr, avr, avr)
-DEF_HELPER_3(vavguh, void, avr, avr, avr)
-DEF_HELPER_3(vavguw, void, avr, avr, avr)
-DEF_HELPER_3(vabsdub, void, avr, avr, avr)
-DEF_HELPER_3(vabsduh, void, avr, avr, avr)
-DEF_HELPER_3(vabsduw, void, avr, avr, avr)
-DEF_HELPER_3(vavgsb, void, avr, avr, avr)
-DEF_HELPER_3(vavgsh, void, avr, avr, avr)
-DEF_HELPER_3(vavgsw, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vavgub, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vavguh, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vavguw, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vabsdub, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vabsduh, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vabsduw, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vavgsb, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vavgsh, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vavgsw, TCG_CALL_NO_RWG, void, avr, avr, avr)
 DEF_HELPER_4(vcmpeqfp, void, env, avr, avr, avr)
 DEF_HELPER_4(vcmpgefp, void, env, avr, avr, avr)
 DEF_HELPER_4(vcmpgtfp, void, env, avr, avr, avr)
@@ -153,12 +153,12 @@ DEF_HELPER_4(vcmpeqfp_dot, void, env, avr, avr, avr)
 DEF_HELPER_4(vcmpgefp_dot, void, env, avr, avr, avr)
 DEF_HELPER_4(vcmpgtfp_dot, void, env, avr, avr, avr)
 DEF_HELPER_4(vcmpbfp_dot, void, env, avr, avr, avr)
-DEF_HELPER_3(vmrglb, void, avr, avr, avr)
-DEF_HELPER_3(vmrglh, void, avr, avr, avr)
-DEF_HELPER_3(vmrglw, void, avr, avr, avr)
-DEF_HELPER_3(vmrghb, void, avr, avr, avr)
-DEF_HELPER_3(vmrghh, void, avr, avr, avr)
-DEF_HELPER_3(vmrghw, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vmrglb, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vmrglh, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vmrglw, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vmrghb, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vmrghh, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vmrghw, TCG_CALL_NO_RWG, void, avr, avr, avr)
 DEF_HELPER_FLAGS_3(VMULESB, TCG_CALL_NO_RWG, void, avr, avr, avr)
 DEF_HELPER_FLAGS_3(VMULESH, TCG_CALL_NO_RWG, void, avr, avr, avr)
 DEF_HELPER_FLAGS_3(VMULESW, TCG_CALL_NO_RWG, void, avr, avr, avr)
@@ -171,15 +171,15 @@ DEF_HELPER_FLAGS_3(VMULOSW, TCG_CALL_NO_RWG, void, avr, avr, avr)
 DEF_HELPER_FLAGS_3(VMULOUB, TCG_CALL_NO_RWG, void, avr, avr, avr)
 DEF_HELPER_FLAGS_3(VMULOUH, TCG_CALL_NO_RWG, void, avr, avr, avr)
 DEF_HELPER_FLAGS_3(VMULOUW, TCG_CALL_NO_RWG, void, avr, avr, avr)
-DEF_HELPER_3(vslo, void, avr, avr, avr)
-DEF_HELPER_3(vsro, void, avr, avr, avr)
-DEF_HELPER_3(vsrv, void, avr, avr, avr)
-DEF_HELPER_3(vslv, void, avr, avr, avr)
-DEF_HELPER_3(vaddcuw, void, avr, avr, avr)
-DEF_HELPER_2(vprtybw, void, avr, avr)
-DEF_HELPER_2(vprtybd, void, avr, avr)
-DEF_HELPER_2(vprtybq, void, avr, avr)
-DEF_HELPER_3(vsubcuw, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vslo, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vsro, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vsrv, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vslv, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vaddcuw, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_2(vprtybw, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vprtybd, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vprtybq, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_3(vsubcuw, TCG_CALL_NO_RWG, void, avr, avr, avr)
 DEF_HELPER_FLAGS_5(vaddsbs, TCG_CALL_NO_RWG, void, avr, avr, avr, avr, i32)
 DEF_HELPER_FLAGS_5(vaddshs, TCG_CALL_NO_RWG, void, avr, avr, avr, avr, i32)
 DEF_HELPER_FLAGS_5(vaddsws, TCG_CALL_NO_RWG, void, avr, avr, avr, avr, i32)
@@ -192,19 +192,19 @@ DEF_HELPER_FLAGS_5(vadduws, TCG_CALL_NO_RWG, void, avr, avr, avr, avr, i32)
 DEF_HELPER_FLAGS_5(vsububs, TCG_CALL_NO_RWG, void, avr, avr, avr, avr, i32)
 DEF_HELPER_FLAGS_5(vsubuhs, TCG_CALL_NO_RWG, void, avr, avr, avr, avr, i32)
 DEF_HELPER_FLAGS_5(vsubuws, TCG_CALL_NO_RWG, void, avr, avr, avr, avr, i32)
-DEF_HELPER_3(vadduqm, void, avr, avr, avr)
-DEF_HELPER_4(vaddecuq, void, avr, avr, avr, avr)
-DEF_HELPER_4(vaddeuqm, void, avr, avr, avr, avr)
-DEF_HELPER_3(vaddcuq, void, avr, avr, avr)
-DEF_HELPER_3(vsubuqm, void, avr, avr, avr)
-DEF_HELPER_4(vsubecuq, void, avr, avr, avr, avr)
-DEF_HELPER_4(vsubeuqm, void, avr, avr, avr, avr)
-DEF_HELPER_3(vsubcuq, void, avr, avr, avr)
-DEF_HELPER_4(vsldoi, void, avr, avr, avr, i32)
-DEF_HELPER_3(vextractub, void, avr, avr, i32)
-DEF_HELPER_3(vextractuh, void, avr, avr, i32)
-DEF_HELPER_3(vextractuw, void, avr, avr, i32)
-DEF_HELPER_3(vextractd, void, avr, avr, i32)
+DEF_HELPER_FLAGS_3(vadduqm, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_4(vaddecuq, TCG_CALL_NO_RWG, void, avr, avr, avr, avr)
+DEF_HELPER_FLAGS_4(vaddeuqm, TCG_CALL_NO_RWG, void, avr, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vaddcuq, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vsubuqm, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_4(vsubecuq, TCG_CALL_NO_RWG, void, avr, avr, avr, avr)
+DEF_HELPER_FLAGS_4(vsubeuqm, TCG_CALL_NO_RWG, void, avr, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vsubcuq, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_4(vsldoi, TCG_CALL_NO_RWG, void, avr, avr, avr, i32)
+DEF_HELPER_FLAGS_3(vextractub, TCG_CALL_NO_RWG, void, avr, avr, i32)
+DEF_HELPER_FLAGS_3(vextractuh, TCG_CALL_NO_RWG, void, avr, avr, i32)
+DEF_HELPER_FLAGS_3(vextractuw, TCG_CALL_NO_RWG, void, avr, avr, i32)
+DEF_HELPER_FLAGS_3(vextractd, TCG_CALL_NO_RWG, void, avr, avr, i32)
 DEF_HELPER_4(VINSBLX, void, env, avr, i64, tl)
 DEF_HELPER_4(VINSHLX, void, env, avr, i64, tl)
 DEF_HELPER_4(VINSWLX, void, env, avr, i64, tl)
@@ -213,16 +213,16 @@ DEF_HELPER_FLAGS_2(VSTRIBL, TCG_CALL_NO_RWG, i32, avr, avr)
 DEF_HELPER_FLAGS_2(VSTRIBR, TCG_CALL_NO_RWG, i32, avr, avr)
 DEF_HELPER_FLAGS_2(VSTRIHL, TCG_CALL_NO_RWG, i32, avr, avr)
 DEF_HELPER_FLAGS_2(VSTRIHR, TCG_CALL_NO_RWG, i32, avr, avr)
-DEF_HELPER_2(vnegw, void, avr, avr)
-DEF_HELPER_2(vnegd, void, avr, avr)
-DEF_HELPER_2(vupkhpx, void, avr, avr)
-DEF_HELPER_2(vupklpx, void, avr, avr)
-DEF_HELPER_2(vupkhsb, void, avr, avr)
-DEF_HELPER_2(vupkhsh, void, avr, avr)
-DEF_HELPER_2(vupkhsw, void, avr, avr)
-DEF_HELPER_2(vupklsb, void, avr, avr)
-DEF_HELPER_2(vupklsh, void, avr, avr)
-DEF_HELPER_2(vupklsw, void, avr, avr)
+DEF_HELPER_FLAGS_2(vnegw, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vnegd, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vupkhpx, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vupklpx, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vupkhsb, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vupkhsh, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vupkhsw, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vupklsb, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vupklsh, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vupklsw, TCG_CALL_NO_RWG, void, avr, avr)
 DEF_HELPER_5(vmsumubm, void, env, avr, avr, avr, avr)
 DEF_HELPER_5(vmsummbm, void, env, avr, avr, avr, avr)
 DEF_HELPER_FLAGS_4(VPERM, TCG_CALL_NO_RWG, void, avr, avr, avr, avr)
@@ -239,14 +239,14 @@ DEF_HELPER_4(vpkudus, void, env, avr, avr, avr)
 DEF_HELPER_4(vpkuhum, void, env, avr, avr, avr)
 DEF_HELPER_4(vpkuwum, void, env, avr, avr, avr)
 DEF_HELPER_4(vpkudum, void, env, avr, avr, avr)
-DEF_HELPER_3(vpkpx, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vpkpx, TCG_CALL_NO_RWG, void, avr, avr, avr)
 DEF_HELPER_5(vmhaddshs, void, env, avr, avr, avr, avr)
 DEF_HELPER_5(vmhraddshs, void, env, avr, avr, avr, avr)
 DEF_HELPER_5(vmsumuhm, void, env, avr, avr, avr, avr)
 DEF_HELPER_5(vmsumuhs, void, env, avr, avr, avr, avr)
 DEF_HELPER_5(vmsumshm, void, env, avr, avr, avr, avr)
 DEF_HELPER_5(vmsumshs, void, env, avr, avr, avr, avr)
-DEF_HELPER_4(vmladduhm, void, avr, avr, avr, avr)
+DEF_HELPER_FLAGS_4(vmladduhm, TCG_CALL_NO_RWG, void, avr, avr, avr, avr)
 DEF_HELPER_FLAGS_2(mtvscr, TCG_CALL_NO_RWG, void, env, i32)
 DEF_HELPER_FLAGS_1(mfvscr, TCG_CALL_NO_RWG, i32, env)
 DEF_HELPER_3(lvebx, void, env, avr, tl)
@@ -289,43 +289,43 @@ DEF_HELPER_4(vcfsx, void, env, avr, avr, i32)
 DEF_HELPER_4(vctuxs, void, env, avr, avr, i32)
 DEF_HELPER_4(vctsxs, void, env, avr, avr, i32)
 
-DEF_HELPER_2(vclzb, void, avr, avr)
-DEF_HELPER_2(vclzh, void, avr, avr)
-DEF_HELPER_2(vctzb, void, avr, avr)
-DEF_HELPER_2(vctzh, void, avr, avr)
-DEF_HELPER_2(vctzw, void, avr, avr)
-DEF_HELPER_2(vctzd, void, avr, avr)
-DEF_HELPER_2(vpopcntb, void, avr, avr)
-DEF_HELPER_2(vpopcnth, void, avr, avr)
-DEF_HELPER_2(vpopcntw, void, avr, avr)
-DEF_HELPER_2(vpopcntd, void, avr, avr)
-DEF_HELPER_1(vclzlsbb, tl, avr)
-DEF_HELPER_1(vctzlsbb, tl, avr)
-DEF_HELPER_3(vbpermd, void, avr, avr, avr)
-DEF_HELPER_3(vbpermq, void, avr, avr, avr)
-DEF_HELPER_3(vpmsumb, void, avr, avr, avr)
-DEF_HELPER_3(vpmsumh, void, avr, avr, avr)
-DEF_HELPER_3(vpmsumw, void, avr, avr, avr)
-DEF_HELPER_3(vpmsumd, void, avr, avr, avr)
-DEF_HELPER_2(vextublx, tl, tl, avr)
-DEF_HELPER_2(vextuhlx, tl, tl, avr)
-DEF_HELPER_2(vextuwlx, tl, tl, avr)
-DEF_HELPER_2(vextubrx, tl, tl, avr)
-DEF_HELPER_2(vextuhrx, tl, tl, avr)
-DEF_HELPER_2(vextuwrx, tl, tl, avr)
+DEF_HELPER_FLAGS_2(vclzb, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vclzh, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vctzb, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vctzh, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vctzw, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vctzd, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vpopcntb, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vpopcnth, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vpopcntw, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_2(vpopcntd, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_1(vclzlsbb, TCG_CALL_NO_RWG, tl, avr)
+DEF_HELPER_FLAGS_1(vctzlsbb, TCG_CALL_NO_RWG, tl, avr)
+DEF_HELPER_FLAGS_3(vbpermd, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vbpermq, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vpmsumb, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vpmsumh, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vpmsumw, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vpmsumd, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_2(vextublx, TCG_CALL_NO_RWG, tl, tl, avr)
+DEF_HELPER_FLAGS_2(vextuhlx, TCG_CALL_NO_RWG, tl, tl, avr)
+DEF_HELPER_FLAGS_2(vextuwlx, TCG_CALL_NO_RWG, tl, tl, avr)
+DEF_HELPER_FLAGS_2(vextubrx, TCG_CALL_NO_RWG, tl, tl, avr)
+DEF_HELPER_FLAGS_2(vextuhrx, TCG_CALL_NO_RWG, tl, tl, avr)
+DEF_HELPER_FLAGS_2(vextuwrx, TCG_CALL_NO_RWG, tl, tl, avr)
 DEF_HELPER_5(VEXTDUBVLX, void, env, avr, avr, avr, tl)
 DEF_HELPER_5(VEXTDUHVLX, void, env, avr, avr, avr, tl)
 DEF_HELPER_5(VEXTDUWVLX, void, env, avr, avr, avr, tl)
 DEF_HELPER_5(VEXTDDVLX, void, env, avr, avr, avr, tl)
 
-DEF_HELPER_2(vsbox, void, avr, avr)
-DEF_HELPER_3(vcipher, void, avr, avr, avr)
-DEF_HELPER_3(vcipherlast, void, avr, avr, avr)
-DEF_HELPER_3(vncipher, void, avr, avr, avr)
-DEF_HELPER_3(vncipherlast, void, avr, avr, avr)
-DEF_HELPER_3(vshasigmaw, void, avr, avr, i32)
-DEF_HELPER_3(vshasigmad, void, avr, avr, i32)
-DEF_HELPER_4(vpermxor, void, avr, avr, avr, avr)
+DEF_HELPER_FLAGS_2(vsbox, TCG_CALL_NO_RWG, void, avr, avr)
+DEF_HELPER_FLAGS_3(vcipher, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vcipherlast, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vncipher, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vncipherlast, TCG_CALL_NO_RWG, void, avr, avr, avr)
+DEF_HELPER_FLAGS_3(vshasigmaw, TCG_CALL_NO_RWG, void, avr, avr, i32)
+DEF_HELPER_FLAGS_3(vshasigmad, TCG_CALL_NO_RWG, void, avr, avr, i32)
+DEF_HELPER_FLAGS_4(vpermxor, TCG_CALL_NO_RWG, void, avr, avr, avr, avr)
 
 DEF_HELPER_4(bcdadd, i32, avr, avr, avr, i32)
 DEF_HELPER_4(bcdsub, i32, avr, avr, avr, i32)

From 491bcaaa35a68157c0546c65fe6b91e65357b6c9 Mon Sep 17 00:00:00 2001
From: Matheus Ferst <matheus.ferst@eldorado.org.br>
Date: Tue, 17 May 2022 09:39:20 -0300
Subject: [PATCH 544/935] target/ppc: use TCG_CALL_NO_RWG in BCD helpers

Helpers of BCD instructions only access the VSRs supplied by the
TCGv_ptr arguments, no globals are accessed.

Signed-off-by: Matheus Ferst <matheus.ferst@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517123929.284511-4-matheus.ferst@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/helper.h | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/target/ppc/helper.h b/target/ppc/helper.h
index da740ad9af..a02c4be906 100644
--- a/target/ppc/helper.h
+++ b/target/ppc/helper.h
@@ -327,21 +327,21 @@ DEF_HELPER_FLAGS_3(vshasigmaw, TCG_CALL_NO_RWG, void, avr, avr, i32)
 DEF_HELPER_FLAGS_3(vshasigmad, TCG_CALL_NO_RWG, void, avr, avr, i32)
 DEF_HELPER_FLAGS_4(vpermxor, TCG_CALL_NO_RWG, void, avr, avr, avr, avr)
 
-DEF_HELPER_4(bcdadd, i32, avr, avr, avr, i32)
-DEF_HELPER_4(bcdsub, i32, avr, avr, avr, i32)
-DEF_HELPER_3(bcdcfn, i32, avr, avr, i32)
-DEF_HELPER_3(bcdctn, i32, avr, avr, i32)
-DEF_HELPER_3(bcdcfz, i32, avr, avr, i32)
-DEF_HELPER_3(bcdctz, i32, avr, avr, i32)
-DEF_HELPER_3(bcdcfsq, i32, avr, avr, i32)
-DEF_HELPER_3(bcdctsq, i32, avr, avr, i32)
-DEF_HELPER_4(bcdcpsgn, i32, avr, avr, avr, i32)
-DEF_HELPER_3(bcdsetsgn, i32, avr, avr, i32)
-DEF_HELPER_4(bcds, i32, avr, avr, avr, i32)
-DEF_HELPER_4(bcdus, i32, avr, avr, avr, i32)
-DEF_HELPER_4(bcdsr, i32, avr, avr, avr, i32)
-DEF_HELPER_4(bcdtrunc, i32, avr, avr, avr, i32)
-DEF_HELPER_4(bcdutrunc, i32, avr, avr, avr, i32)
+DEF_HELPER_FLAGS_4(bcdadd, TCG_CALL_NO_RWG, i32, avr, avr, avr, i32)
+DEF_HELPER_FLAGS_4(bcdsub, TCG_CALL_NO_RWG, i32, avr, avr, avr, i32)
+DEF_HELPER_FLAGS_3(bcdcfn, TCG_CALL_NO_RWG, i32, avr, avr, i32)
+DEF_HELPER_FLAGS_3(bcdctn, TCG_CALL_NO_RWG, i32, avr, avr, i32)
+DEF_HELPER_FLAGS_3(bcdcfz, TCG_CALL_NO_RWG, i32, avr, avr, i32)
+DEF_HELPER_FLAGS_3(bcdctz, TCG_CALL_NO_RWG, i32, avr, avr, i32)
+DEF_HELPER_FLAGS_3(bcdcfsq, TCG_CALL_NO_RWG, i32, avr, avr, i32)
+DEF_HELPER_FLAGS_3(bcdctsq, TCG_CALL_NO_RWG, i32, avr, avr, i32)
+DEF_HELPER_FLAGS_4(bcdcpsgn, TCG_CALL_NO_RWG, i32, avr, avr, avr, i32)
+DEF_HELPER_FLAGS_3(bcdsetsgn, TCG_CALL_NO_RWG, i32, avr, avr, i32)
+DEF_HELPER_FLAGS_4(bcds, TCG_CALL_NO_RWG, i32, avr, avr, avr, i32)
+DEF_HELPER_FLAGS_4(bcdus, TCG_CALL_NO_RWG, i32, avr, avr, avr, i32)
+DEF_HELPER_FLAGS_4(bcdsr, TCG_CALL_NO_RWG, i32, avr, avr, avr, i32)
+DEF_HELPER_FLAGS_4(bcdtrunc, TCG_CALL_NO_RWG, i32, avr, avr, avr, i32)
+DEF_HELPER_FLAGS_4(bcdutrunc, TCG_CALL_NO_RWG, i32, avr, avr, avr, i32)
 
 DEF_HELPER_4(xsadddp, void, env, vsr, vsr, vsr)
 DEF_HELPER_5(xsaddqp, void, env, i32, vsr, vsr, vsr)

From f2454bfe73d1ae9e7561fc3c29e0d2729381d27c Mon Sep 17 00:00:00 2001
From: Matheus Ferst <matheus.ferst@eldorado.org.br>
Date: Tue, 17 May 2022 09:39:21 -0300
Subject: [PATCH 545/935] target/ppc: use TCG_CALL_NO_RWG in VSX helpers
 without env

Helpers of VSX instructions without cpu_env as an argument do not access
globals.

Signed-off-by: Matheus Ferst <matheus.ferst@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517123929.284511-5-matheus.ferst@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/helper.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/target/ppc/helper.h b/target/ppc/helper.h
index a02c4be906..f82c5bd0db 100644
--- a/target/ppc/helper.h
+++ b/target/ppc/helper.h
@@ -533,10 +533,10 @@ DEF_HELPER_FLAGS_5(XXPERMX, TCG_CALL_NO_RWG, void, vsr, vsr, vsr, vsr, tl)
 DEF_HELPER_4(xxinsertw, void, env, vsr, vsr, i32)
 DEF_HELPER_3(xvxsigsp, void, env, vsr, vsr)
 DEF_HELPER_FLAGS_5(XXEVAL, TCG_CALL_NO_RWG, void, vsr, vsr, vsr, vsr, i32)
-DEF_HELPER_5(XXBLENDVB, void, vsr, vsr, vsr, vsr, i32)
-DEF_HELPER_5(XXBLENDVH, void, vsr, vsr, vsr, vsr, i32)
-DEF_HELPER_5(XXBLENDVW, void, vsr, vsr, vsr, vsr, i32)
-DEF_HELPER_5(XXBLENDVD, void, vsr, vsr, vsr, vsr, i32)
+DEF_HELPER_FLAGS_5(XXBLENDVB, TCG_CALL_NO_RWG, void, vsr, vsr, vsr, vsr, i32)
+DEF_HELPER_FLAGS_5(XXBLENDVH, TCG_CALL_NO_RWG, void, vsr, vsr, vsr, vsr, i32)
+DEF_HELPER_FLAGS_5(XXBLENDVW, TCG_CALL_NO_RWG, void, vsr, vsr, vsr, vsr, i32)
+DEF_HELPER_FLAGS_5(XXBLENDVD, TCG_CALL_NO_RWG, void, vsr, vsr, vsr, vsr, i32)
 
 DEF_HELPER_2(efscfsi, i32, env, i32)
 DEF_HELPER_2(efscfui, i32, env, i32)

From eb69a84bb07be94179eb2275cd9d4bed12afc66f Mon Sep 17 00:00:00 2001
From: Matheus Ferst <matheus.ferst@eldorado.org.br>
Date: Tue, 17 May 2022 09:39:22 -0300
Subject: [PATCH 546/935] target/ppc: Use TCG_CALL_NO_RWG_SE in fsel helper

fsel doesn't change FPSCR and CR1 is handled by gen_set_cr1_from_fpscr,
so helper_fsel doesn't need the env argument and can be declared with
TCG_CALL_NO_RWG_SE. We also take this opportunity to move the insn to
decodetree.

Signed-off-by: Matheus Ferst <matheus.ferst@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517123929.284511-6-matheus.ferst@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/fpu_helper.c            | 15 +++++++--------
 target/ppc/helper.h                |  2 +-
 target/ppc/insn32.decode           |  7 +++++++
 target/ppc/translate/fp-impl.c.inc | 30 ++++++++++++++++++++++++++++--
 target/ppc/translate/fp-ops.c.inc  |  1 -
 5 files changed, 43 insertions(+), 12 deletions(-)

diff --git a/target/ppc/fpu_helper.c b/target/ppc/fpu_helper.c
index 8592727792..588448702f 100644
--- a/target/ppc/fpu_helper.c
+++ b/target/ppc/fpu_helper.c
@@ -919,18 +919,17 @@ float64 helper_frsqrtes(CPUPPCState *env, float64 arg)
 }
 
 /* fsel - fsel. */
-uint64_t helper_fsel(CPUPPCState *env, uint64_t arg1, uint64_t arg2,
-                     uint64_t arg3)
+uint64_t helper_FSEL(uint64_t a, uint64_t b, uint64_t c)
 {
-    CPU_DoubleU farg1;
+    CPU_DoubleU fa;
 
-    farg1.ll = arg1;
+    fa.ll = a;
 
-    if ((!float64_is_neg(farg1.d) || float64_is_zero(farg1.d)) &&
-        !float64_is_any_nan(farg1.d)) {
-        return arg2;
+    if ((!float64_is_neg(fa.d) || float64_is_zero(fa.d)) &&
+        !float64_is_any_nan(fa.d)) {
+        return c;
     } else {
-        return arg3;
+        return b;
     }
 }
 
diff --git a/target/ppc/helper.h b/target/ppc/helper.h
index f82c5bd0db..ddfa0308bc 100644
--- a/target/ppc/helper.h
+++ b/target/ppc/helper.h
@@ -120,7 +120,7 @@ DEF_HELPER_2(fre, i64, env, i64)
 DEF_HELPER_2(fres, i64, env, i64)
 DEF_HELPER_2(frsqrte, i64, env, i64)
 DEF_HELPER_2(frsqrtes, i64, env, i64)
-DEF_HELPER_4(fsel, i64, env, i64, i64, i64)
+DEF_HELPER_FLAGS_3(FSEL, TCG_CALL_NO_RWG_SE, i64, i64, i64, i64)
 
 DEF_HELPER_FLAGS_2(ftdiv, TCG_CALL_NO_RWG_SE, i32, i64, i64)
 DEF_HELPER_FLAGS_1(ftsqrt, TCG_CALL_NO_RWG_SE, i32, i64)
diff --git a/target/ppc/insn32.decode b/target/ppc/insn32.decode
index 39372fe673..1d0b55bde3 100644
--- a/target/ppc/insn32.decode
+++ b/target/ppc/insn32.decode
@@ -17,6 +17,9 @@
 # License along with this library; if not, see <http://www.gnu.org/licenses/>.
 #
 
+&A              frt fra frb frc rc:bool
+@A              ...... frt:5 fra:5 frb:5 frc:5 ..... rc:1       &A
+
 &D              rt ra si:int64_t
 @D              ...... rt:5 ra:5 si:s16                 &D
 
@@ -308,6 +311,10 @@ STFDU           110111 ..... ...... ...............     @D
 STFDX           011111 ..... ...... .... 1011010111 -   @X
 STFDUX          011111 ..... ...... .... 1011110111 -   @X
 
+### Floating-Point Select Instruction
+
+FSEL            111111 ..... ..... ..... ..... 10111 .  @A
+
 ### Move To/From System Register Instructions
 
 SETBC           011111 ..... ..... ----- 0110000000 -   @X_bi
diff --git a/target/ppc/translate/fp-impl.c.inc b/target/ppc/translate/fp-impl.c.inc
index cfb27bd020..f9b58b844e 100644
--- a/target/ppc/translate/fp-impl.c.inc
+++ b/target/ppc/translate/fp-impl.c.inc
@@ -222,8 +222,34 @@ static void gen_frsqrtes(DisasContext *ctx)
     tcg_temp_free_i64(t1);
 }
 
-/* fsel */
-_GEN_FLOAT_ACB(sel, 0x3F, 0x17, 0, PPC_FLOAT_FSEL);
+static bool trans_FSEL(DisasContext *ctx, arg_A *a)
+{
+    TCGv_i64 t0, t1, t2;
+
+    REQUIRE_INSNS_FLAGS(ctx, FLOAT_FSEL);
+    REQUIRE_FPU(ctx);
+
+    t0 = tcg_temp_new_i64();
+    t1 = tcg_temp_new_i64();
+    t2 = tcg_temp_new_i64();
+
+    get_fpr(t0, a->fra);
+    get_fpr(t1, a->frb);
+    get_fpr(t2, a->frc);
+
+    gen_helper_FSEL(t0, t0, t1, t2);
+    set_fpr(a->frt, t0);
+    if (a->rc) {
+        gen_set_cr1_from_fpscr(ctx);
+    }
+
+    tcg_temp_free_i64(t0);
+    tcg_temp_free_i64(t1);
+    tcg_temp_free_i64(t2);
+
+    return true;
+}
+
 /* fsub - fsubs */
 GEN_FLOAT_AB(sub, 0x14, 0x000007C0, 1, PPC_FLOAT);
 /* Optional: */
diff --git a/target/ppc/translate/fp-ops.c.inc b/target/ppc/translate/fp-ops.c.inc
index 4260635a12..0538ab2d2d 100644
--- a/target/ppc/translate/fp-ops.c.inc
+++ b/target/ppc/translate/fp-ops.c.inc
@@ -24,7 +24,6 @@ GEN_FLOAT_AC(mul, 0x19, 0x0000F800, 1, PPC_FLOAT),
 GEN_FLOAT_BS(re, 0x3F, 0x18, 1, PPC_FLOAT_EXT),
 GEN_FLOAT_BS(res, 0x3B, 0x18, 1, PPC_FLOAT_FRES),
 GEN_FLOAT_BS(rsqrte, 0x3F, 0x1A, 1, PPC_FLOAT_FRSQRTE),
-_GEN_FLOAT_ACB(sel, sel, 0x3F, 0x17, 0, 0, PPC_FLOAT_FSEL),
 GEN_FLOAT_AB(sub, 0x14, 0x000007C0, 1, PPC_FLOAT),
 GEN_FLOAT_ACB(madd, 0x1D, 1, PPC_FLOAT),
 GEN_FLOAT_ACB(msub, 0x1C, 1, PPC_FLOAT),

From cf862bee0e0d31de4809ac7bacb1d65ff7e8b9ce Mon Sep 17 00:00:00 2001
From: Matheus Ferst <matheus.ferst@eldorado.org.br>
Date: Tue, 17 May 2022 09:39:23 -0300
Subject: [PATCH 547/935] target/ppc: declare xscvspdpn helper with call flags

Move xscvspdpn to decodetree, declare helper_xscvspdpn with
TCG_CALL_NO_RWG_SE and drop the unused env argument.

Signed-off-by: Matheus Ferst <matheus.ferst@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517123929.284511-7-matheus.ferst@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/fpu_helper.c             |  2 +-
 target/ppc/helper.h                 |  2 +-
 target/ppc/insn32.decode            |  1 +
 target/ppc/translate/vsx-impl.c.inc | 22 +++++++++++++++++++++-
 target/ppc/translate/vsx-ops.c.inc  |  1 -
 5 files changed, 24 insertions(+), 4 deletions(-)

diff --git a/target/ppc/fpu_helper.c b/target/ppc/fpu_helper.c
index 588448702f..55ef4b5d48 100644
--- a/target/ppc/fpu_helper.c
+++ b/target/ppc/fpu_helper.c
@@ -2878,7 +2878,7 @@ uint64_t helper_xscvdpspn(CPUPPCState *env, uint64_t xb)
     return (result << 32) | result;
 }
 
-uint64_t helper_xscvspdpn(CPUPPCState *env, uint64_t xb)
+uint64_t helper_XSCVSPDPN(uint64_t xb)
 {
     return helper_todouble(xb >> 32);
 }
diff --git a/target/ppc/helper.h b/target/ppc/helper.h
index ddfa0308bc..9be69fa91e 100644
--- a/target/ppc/helper.h
+++ b/target/ppc/helper.h
@@ -395,7 +395,7 @@ DEF_HELPER_3(XSCVSQQP, void, env, vsr, vsr)
 DEF_HELPER_3(xscvhpdp, void, env, vsr, vsr)
 DEF_HELPER_4(xscvsdqp, void, env, i32, vsr, vsr)
 DEF_HELPER_3(xscvspdp, void, env, vsr, vsr)
-DEF_HELPER_2(xscvspdpn, i64, env, i64)
+DEF_HELPER_FLAGS_1(XSCVSPDPN, TCG_CALL_NO_RWG_SE, i64, i64)
 DEF_HELPER_3(xscvdpsxds, void, env, vsr, vsr)
 DEF_HELPER_3(xscvdpsxws, void, env, vsr, vsr)
 DEF_HELPER_3(xscvdpuxds, void, env, vsr, vsr)
diff --git a/target/ppc/insn32.decode b/target/ppc/insn32.decode
index 1d0b55bde3..d4c2615b1a 100644
--- a/target/ppc/insn32.decode
+++ b/target/ppc/insn32.decode
@@ -708,6 +708,7 @@ XSCVUQQP        111111 ..... 00011 ..... 1101000100 -   @X_tb
 XSCVSQQP        111111 ..... 01011 ..... 1101000100 -   @X_tb
 XVCVBF16SPN     111100 ..... 10000 ..... 111011011 ..   @XX2
 XVCVSPBF16      111100 ..... 10001 ..... 111011011 ..   @XX2
+XSCVSPDPN       111100 ..... ----- ..... 101001011 ..   @XX2
 
 ## VSX Vector Test Least-Significant Bit by Byte Instruction
 
diff --git a/target/ppc/translate/vsx-impl.c.inc b/target/ppc/translate/vsx-impl.c.inc
index 3692740736..9b4f309d9d 100644
--- a/target/ppc/translate/vsx-impl.c.inc
+++ b/target/ppc/translate/vsx-impl.c.inc
@@ -1045,7 +1045,27 @@ GEN_VSX_HELPER_R2(xscvqpuwz, 0x04, 0x1A, 0x01, PPC2_ISA300)
 GEN_VSX_HELPER_X2(xscvhpdp, 0x16, 0x15, 0x10, PPC2_ISA300)
 GEN_VSX_HELPER_R2(xscvsdqp, 0x04, 0x1A, 0x0A, PPC2_ISA300)
 GEN_VSX_HELPER_X2(xscvspdp, 0x12, 0x14, 0, PPC2_VSX)
-GEN_VSX_HELPER_XT_XB_ENV(xscvspdpn, 0x16, 0x14, 0, PPC2_VSX207)
+
+bool trans_XSCVSPDPN(DisasContext *ctx, arg_XX2 *a)
+{
+    TCGv_i64 tmp;
+
+    REQUIRE_INSNS_FLAGS2(ctx, VSX207);
+    REQUIRE_VSX(ctx);
+
+    tmp = tcg_temp_new_i64();
+    get_cpu_vsr(tmp, a->xb, true);
+
+    gen_helper_XSCVSPDPN(tmp, tmp);
+
+    set_cpu_vsr(a->xt, tmp, true);
+    set_cpu_vsr(a->xt, tcg_constant_i64(0), false);
+
+    tcg_temp_free_i64(tmp);
+
+    return true;
+}
+
 GEN_VSX_HELPER_X2(xscvdpsxds, 0x10, 0x15, 0, PPC2_VSX)
 GEN_VSX_HELPER_X2(xscvdpsxws, 0x10, 0x05, 0, PPC2_VSX)
 GEN_VSX_HELPER_X2(xscvdpuxds, 0x10, 0x14, 0, PPC2_VSX)
diff --git a/target/ppc/translate/vsx-ops.c.inc b/target/ppc/translate/vsx-ops.c.inc
index b8fd116728..52d7ab30cd 100644
--- a/target/ppc/translate/vsx-ops.c.inc
+++ b/target/ppc/translate/vsx-ops.c.inc
@@ -200,7 +200,6 @@ GEN_XX2FORM(xscvdpspn, 0x16, 0x10, PPC2_VSX207),
 GEN_XX2FORM_EO(xscvhpdp, 0x16, 0x15, 0x10, PPC2_ISA300),
 GEN_VSX_XFORM_300_EO(xscvsdqp, 0x04, 0x1A, 0x0A, 0x00000001),
 GEN_XX2FORM(xscvspdp, 0x12, 0x14, PPC2_VSX),
-GEN_XX2FORM(xscvspdpn, 0x16, 0x14, PPC2_VSX207),
 GEN_XX2FORM(xscvdpsxds, 0x10, 0x15, PPC2_VSX),
 GEN_XX2FORM(xscvdpsxws, 0x10, 0x05, PPC2_VSX),
 GEN_XX2FORM(xscvdpuxds, 0x10, 0x14, PPC2_VSX),

From c36ab970ac0ce257a6badb30f6a485a81c2289d2 Mon Sep 17 00:00:00 2001
From: Matheus Ferst <matheus.ferst@eldorado.org.br>
Date: Tue, 17 May 2022 09:39:24 -0300
Subject: [PATCH 548/935] target/ppc: declare xvxsigsp helper with call flags

Move xvxsigsp to decodetree, declare helper_xvxsigsp with
TCG_CALL_NO_RWG, and drop the unused env argument.

Signed-off-by: Matheus Ferst <matheus.ferst@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517123929.284511-8-matheus.ferst@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/fpu_helper.c             |  2 +-
 target/ppc/helper.h                 |  2 +-
 target/ppc/insn32.decode            |  4 ++++
 target/ppc/translate/vsx-impl.c.inc | 18 +++++++++++++++++-
 target/ppc/translate/vsx-ops.c.inc  |  1 -
 5 files changed, 23 insertions(+), 4 deletions(-)

diff --git a/target/ppc/fpu_helper.c b/target/ppc/fpu_helper.c
index 55ef4b5d48..9489e06504 100644
--- a/target/ppc/fpu_helper.c
+++ b/target/ppc/fpu_helper.c
@@ -3205,7 +3205,7 @@ uint64_t helper_xsrsp(CPUPPCState *env, uint64_t xb)
     return xt;
 }
 
-void helper_xvxsigsp(CPUPPCState *env, ppc_vsr_t *xt, ppc_vsr_t *xb)
+void helper_XVXSIGSP(ppc_vsr_t *xt, ppc_vsr_t *xb)
 {
     ppc_vsr_t t = { };
     uint32_t exp, i, fraction;
diff --git a/target/ppc/helper.h b/target/ppc/helper.h
index 9be69fa91e..aed1b24fdb 100644
--- a/target/ppc/helper.h
+++ b/target/ppc/helper.h
@@ -531,7 +531,7 @@ DEF_HELPER_FLAGS_2(XXGENPCVDM_le_comp, TCG_CALL_NO_RWG, void, vsr, avr)
 DEF_HELPER_4(xxextractuw, void, env, vsr, vsr, i32)
 DEF_HELPER_FLAGS_5(XXPERMX, TCG_CALL_NO_RWG, void, vsr, vsr, vsr, vsr, tl)
 DEF_HELPER_4(xxinsertw, void, env, vsr, vsr, i32)
-DEF_HELPER_3(xvxsigsp, void, env, vsr, vsr)
+DEF_HELPER_FLAGS_2(XVXSIGSP, TCG_CALL_NO_RWG, void, vsr, vsr)
 DEF_HELPER_FLAGS_5(XXEVAL, TCG_CALL_NO_RWG, void, vsr, vsr, vsr, vsr, i32)
 DEF_HELPER_FLAGS_5(XXBLENDVB, TCG_CALL_NO_RWG, void, vsr, vsr, vsr, vsr, i32)
 DEF_HELPER_FLAGS_5(XXBLENDVH, TCG_CALL_NO_RWG, void, vsr, vsr, vsr, vsr, i32)
diff --git a/target/ppc/insn32.decode b/target/ppc/insn32.decode
index d4c2615b1a..483349ff6d 100644
--- a/target/ppc/insn32.decode
+++ b/target/ppc/insn32.decode
@@ -710,6 +710,10 @@ XVCVBF16SPN     111100 ..... 10000 ..... 111011011 ..   @XX2
 XVCVSPBF16      111100 ..... 10001 ..... 111011011 ..   @XX2
 XSCVSPDPN       111100 ..... ----- ..... 101001011 ..   @XX2
 
+## VSX Binary Floating-Point Math Support Instructions
+
+XVXSIGSP        111100 ..... 01001 ..... 111011011 ..   @XX2
+
 ## VSX Vector Test Least-Significant Bit by Byte Instruction
 
 XVTLSBB         111100 ... -- 00010 ..... 111011011 . - @XX2_bf_xb
diff --git a/target/ppc/translate/vsx-impl.c.inc b/target/ppc/translate/vsx-impl.c.inc
index 9b4f309d9d..ca11e2c4b8 100644
--- a/target/ppc/translate/vsx-impl.c.inc
+++ b/target/ppc/translate/vsx-impl.c.inc
@@ -2151,7 +2151,23 @@ static void gen_xvxexpdp(DisasContext *ctx)
     tcg_temp_free_i64(xbl);
 }
 
-GEN_VSX_HELPER_X2(xvxsigsp, 0x00, 0x04, 0, PPC2_ISA300)
+static bool trans_XVXSIGSP(DisasContext *ctx, arg_XX2 *a)
+{
+    TCGv_ptr t, b;
+
+    REQUIRE_INSNS_FLAGS2(ctx, ISA300);
+    REQUIRE_VSX(ctx);
+
+    t = gen_vsr_ptr(a->xt);
+    b = gen_vsr_ptr(a->xb);
+
+    gen_helper_XVXSIGSP(t, b);
+
+    tcg_temp_free_ptr(t);
+    tcg_temp_free_ptr(b);
+
+    return true;
+}
 
 static void gen_xvxsigdp(DisasContext *ctx)
 {
diff --git a/target/ppc/translate/vsx-ops.c.inc b/target/ppc/translate/vsx-ops.c.inc
index 52d7ab30cd..4524c5b02a 100644
--- a/target/ppc/translate/vsx-ops.c.inc
+++ b/target/ppc/translate/vsx-ops.c.inc
@@ -156,7 +156,6 @@ GEN_XX3FORM(xviexpdp, 0x00, 0x1F, PPC2_ISA300),
 GEN_XX2FORM_EO(xvxexpdp, 0x16, 0x1D, 0x00, PPC2_ISA300),
 GEN_XX2FORM_EO(xvxsigdp, 0x16, 0x1D, 0x01, PPC2_ISA300),
 GEN_XX2FORM_EO(xvxexpsp, 0x16, 0x1D, 0x08, PPC2_ISA300),
-GEN_XX2FORM_EO(xvxsigsp, 0x16, 0x1D, 0x09, PPC2_ISA300),
 
 /* DCMX  =  bit[25] << 6 | bit[29] << 5 | bit[11:15] */
 #define GEN_XX2FORM_DCMX(name, opc2, opc3, fl2) \

From 8f5eeee3f1f1e7da4a1bf1ecb5527071fde1b2d5 Mon Sep 17 00:00:00 2001
From: Matheus Ferst <matheus.ferst@eldorado.org.br>
Date: Tue, 17 May 2022 09:39:25 -0300
Subject: [PATCH 549/935] target/ppc: declare xxextractuw and xxinsertw helpers
 with call flags

Move xxextractuw and xxinsertw to decodetree, declare both helpers with
TCG_CALL_NO_RWG, and drop the unused env argument.

Signed-off-by: Matheus Ferst <matheus.ferst@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517123929.284511-9-matheus.ferst@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/helper.h                 |  4 +-
 target/ppc/insn32.decode            |  9 ++++-
 target/ppc/int_helper.c             |  6 +--
 target/ppc/translate/vsx-impl.c.inc | 63 +++++++++++++----------------
 target/ppc/translate/vsx-ops.c.inc  |  2 -
 5 files changed, 39 insertions(+), 45 deletions(-)

diff --git a/target/ppc/helper.h b/target/ppc/helper.h
index aed1b24fdb..640a70cd5c 100644
--- a/target/ppc/helper.h
+++ b/target/ppc/helper.h
@@ -528,9 +528,9 @@ DEF_HELPER_FLAGS_2(XXGENPCVDM_be_exp, TCG_CALL_NO_RWG, void, vsr, avr)
 DEF_HELPER_FLAGS_2(XXGENPCVDM_be_comp, TCG_CALL_NO_RWG, void, vsr, avr)
 DEF_HELPER_FLAGS_2(XXGENPCVDM_le_exp, TCG_CALL_NO_RWG, void, vsr, avr)
 DEF_HELPER_FLAGS_2(XXGENPCVDM_le_comp, TCG_CALL_NO_RWG, void, vsr, avr)
-DEF_HELPER_4(xxextractuw, void, env, vsr, vsr, i32)
+DEF_HELPER_FLAGS_3(XXEXTRACTUW, TCG_CALL_NO_RWG, void, vsr, vsr, i32)
 DEF_HELPER_FLAGS_5(XXPERMX, TCG_CALL_NO_RWG, void, vsr, vsr, vsr, vsr, tl)
-DEF_HELPER_4(xxinsertw, void, env, vsr, vsr, i32)
+DEF_HELPER_FLAGS_3(XXINSERTW, TCG_CALL_NO_RWG, void, vsr, vsr, i32)
 DEF_HELPER_FLAGS_2(XVXSIGSP, TCG_CALL_NO_RWG, void, vsr, vsr)
 DEF_HELPER_FLAGS_5(XXEVAL, TCG_CALL_NO_RWG, void, vsr, vsr, vsr, vsr, i32)
 DEF_HELPER_FLAGS_5(XXBLENDVB, TCG_CALL_NO_RWG, void, vsr, vsr, vsr, vsr, i32)
diff --git a/target/ppc/insn32.decode b/target/ppc/insn32.decode
index 483349ff6d..435cf1320c 100644
--- a/target/ppc/insn32.decode
+++ b/target/ppc/insn32.decode
@@ -161,8 +161,10 @@
 &XX2            xt xb
 @XX2            ...... ..... ..... ..... ......... ..           &XX2 xt=%xx_xt xb=%xx_xb
 
-&XX2_uim2       xt xb uim:uint8_t
-@XX2_uim2       ...... ..... ... uim:2 ..... ......... ..       &XX2_uim2 xt=%xx_xt xb=%xx_xb
+&XX2_uim        xt xb uim:uint8_t
+@XX2_uim2       ...... ..... ... uim:2 ..... ......... ..       &XX2_uim xt=%xx_xt xb=%xx_xb
+
+@XX2_uim4       ...... ..... . uim:4 ..... ......... ..         &XX2_uim xt=%xx_xt xb=%xx_xb
 
 &XX2_bf_xb      bf xb
 @XX2_bf_xb      ...... bf:3 .. ..... ..... ......... . .        &XX2_bf_xb xb=%xx_xb
@@ -666,6 +668,9 @@ XXSPLTW         111100 ..... ---.. ..... 010100100 . .  @XX2_uim2
 
 ## VSX Permute Instructions
 
+XXEXTRACTUW     111100 ..... - .... ..... 010100101 ..  @XX2_uim4
+XXINSERTW       111100 ..... - .... ..... 010110101 ..  @XX2_uim4
+
 XXPERM          111100 ..... ..... ..... 00011010 ...   @XX3
 XXPERMR         111100 ..... ..... ..... 00111010 ...   @XX3
 XXPERMDI        111100 ..... ..... ..... 0 .. 01010 ... @XX3_dm
diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c
index 8c1674510b..9a361ad241 100644
--- a/target/ppc/int_helper.c
+++ b/target/ppc/int_helper.c
@@ -1647,8 +1647,7 @@ VSTRI(VSTRIHL, H, 8, true)
 VSTRI(VSTRIHR, H, 8, false)
 #undef VSTRI
 
-void helper_xxextractuw(CPUPPCState *env, ppc_vsr_t *xt,
-                        ppc_vsr_t *xb, uint32_t index)
+void helper_XXEXTRACTUW(ppc_vsr_t *xt, ppc_vsr_t *xb, uint32_t index)
 {
     ppc_vsr_t t = { };
     size_t es = sizeof(uint32_t);
@@ -1663,8 +1662,7 @@ void helper_xxextractuw(CPUPPCState *env, ppc_vsr_t *xt,
     *xt = t;
 }
 
-void helper_xxinsertw(CPUPPCState *env, ppc_vsr_t *xt,
-                      ppc_vsr_t *xb, uint32_t index)
+void helper_XXINSERTW(ppc_vsr_t *xt, ppc_vsr_t *xb, uint32_t index)
 {
     ppc_vsr_t t = *xt;
     size_t es = sizeof(uint32_t);
diff --git a/target/ppc/translate/vsx-impl.c.inc b/target/ppc/translate/vsx-impl.c.inc
index ca11e2c4b8..900c1a1ab2 100644
--- a/target/ppc/translate/vsx-impl.c.inc
+++ b/target/ppc/translate/vsx-impl.c.inc
@@ -1585,7 +1585,7 @@ static bool trans_XXSEL(DisasContext *ctx, arg_XX4 *a)
     return true;
 }
 
-static bool trans_XXSPLTW(DisasContext *ctx, arg_XX2_uim2 *a)
+static bool trans_XXSPLTW(DisasContext *ctx, arg_XX2_uim *a)
 {
     int tofs, bofs;
 
@@ -1795,42 +1795,35 @@ static void gen_xxsldwi(DisasContext *ctx)
     tcg_temp_free_i64(xtl);
 }
 
-#define VSX_EXTRACT_INSERT(name)                                \
-static void gen_##name(DisasContext *ctx)                       \
-{                                                               \
-    TCGv_ptr xt, xb;                                            \
-    TCGv_i32 t0;                                                \
-    TCGv_i64 t1;                                                \
-    uint8_t uimm = UIMM4(ctx->opcode);                          \
-                                                                \
-    if (unlikely(!ctx->vsx_enabled)) {                          \
-        gen_exception(ctx, POWERPC_EXCP_VSXU);                  \
-        return;                                                 \
-    }                                                           \
-    xt = gen_vsr_ptr(xT(ctx->opcode));                          \
-    xb = gen_vsr_ptr(xB(ctx->opcode));                          \
-    t0 = tcg_temp_new_i32();                                    \
-    t1 = tcg_temp_new_i64();                                    \
-    /*                                                          \
-     * uimm > 15 out of bound and for                           \
-     * uimm > 12 handle as per hardware in helper               \
-     */                                                         \
-    if (uimm > 15) {                                            \
-        tcg_gen_movi_i64(t1, 0);                                \
-        set_cpu_vsr(xT(ctx->opcode), t1, true);                 \
-        set_cpu_vsr(xT(ctx->opcode), t1, false);                \
-        return;                                                 \
-    }                                                           \
-    tcg_gen_movi_i32(t0, uimm);                                 \
-    gen_helper_##name(cpu_env, xt, xb, t0);                     \
-    tcg_temp_free_ptr(xb);                                      \
-    tcg_temp_free_ptr(xt);                                      \
-    tcg_temp_free_i32(t0);                                      \
-    tcg_temp_free_i64(t1);                                      \
+static bool do_vsx_extract_insert(DisasContext *ctx, arg_XX2_uim *a,
+    void (*gen_helper)(TCGv_ptr, TCGv_ptr, TCGv_i32))
+{
+    TCGv_i64 zero = tcg_constant_i64(0);
+    TCGv_ptr xt, xb;
+
+    REQUIRE_INSNS_FLAGS2(ctx, ISA300);
+    REQUIRE_VSX(ctx);
+
+    /*
+     * uim > 15 out of bound and for
+     * uim > 12 handle as per hardware in helper
+     */
+    if (a->uim > 15) {
+        set_cpu_vsr(a->xt, zero, true);
+        set_cpu_vsr(a->xt, zero, false);
+    } else {
+        xt = gen_vsr_ptr(a->xt);
+        xb = gen_vsr_ptr(a->xb);
+        gen_helper(xt, xb, tcg_constant_i32(a->uim));
+        tcg_temp_free_ptr(xb);
+        tcg_temp_free_ptr(xt);
+    }
+
+    return true;
 }
 
-VSX_EXTRACT_INSERT(xxextractuw)
-VSX_EXTRACT_INSERT(xxinsertw)
+TRANS(XXEXTRACTUW, do_vsx_extract_insert, gen_helper_XXEXTRACTUW)
+TRANS(XXINSERTW, do_vsx_extract_insert, gen_helper_XXINSERTW)
 
 #ifdef TARGET_PPC64
 static void gen_xsxexpdp(DisasContext *ctx)
diff --git a/target/ppc/translate/vsx-ops.c.inc b/target/ppc/translate/vsx-ops.c.inc
index 4524c5b02a..bff14bbece 100644
--- a/target/ppc/translate/vsx-ops.c.inc
+++ b/target/ppc/translate/vsx-ops.c.inc
@@ -320,5 +320,3 @@ VSX_LOGICAL(xxlorc, 0x8, 0x15, PPC2_VSX207),
 GEN_XX3FORM(xxmrghw, 0x08, 0x02, PPC2_VSX),
 GEN_XX3FORM(xxmrglw, 0x08, 0x06, PPC2_VSX),
 GEN_XX3FORM_DM(xxsldwi, 0x08, 0x00),
-GEN_XX2FORM_EXT(xxextractuw, 0x0A, 0x0A, PPC2_ISA300),
-GEN_XX2FORM_EXT(xxinsertw, 0x0A, 0x0B, PPC2_ISA300),

From ffc2a2818a6ae321830de2a8c93c78d437c64171 Mon Sep 17 00:00:00 2001
From: Matheus Ferst <matheus.ferst@eldorado.org.br>
Date: Tue, 17 May 2022 09:39:26 -0300
Subject: [PATCH 550/935] target/ppc: introduce do_va_helper

Signed-off-by: Matheus Ferst <matheus.ferst@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517123929.284511-10-matheus.ferst@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/translate/vmx-impl.c.inc | 32 +++++------------------------
 1 file changed, 5 insertions(+), 27 deletions(-)

diff --git a/target/ppc/translate/vmx-impl.c.inc b/target/ppc/translate/vmx-impl.c.inc
index 764ac45409..e66301c007 100644
--- a/target/ppc/translate/vmx-impl.c.inc
+++ b/target/ppc/translate/vmx-impl.c.inc
@@ -2553,20 +2553,17 @@ static void gen_vmladduhm(DisasContext *ctx)
     tcg_temp_free_ptr(rd);
 }
 
-static bool trans_VPERM(DisasContext *ctx, arg_VA *a)
+static bool do_va_helper(DisasContext *ctx, arg_VA *a,
+    void (*gen_helper)(TCGv_ptr, TCGv_ptr, TCGv_ptr, TCGv_ptr))
 {
     TCGv_ptr vrt, vra, vrb, vrc;
-
-    REQUIRE_INSNS_FLAGS(ctx, ALTIVEC);
     REQUIRE_VECTOR(ctx);
 
     vrt = gen_avr_ptr(a->vrt);
     vra = gen_avr_ptr(a->vra);
     vrb = gen_avr_ptr(a->vrb);
     vrc = gen_avr_ptr(a->rc);
-
-    gen_helper_VPERM(vrt, vra, vrb, vrc);
-
+    gen_helper(vrt, vra, vrb, vrc);
     tcg_temp_free_ptr(vrt);
     tcg_temp_free_ptr(vra);
     tcg_temp_free_ptr(vrb);
@@ -2575,27 +2572,8 @@ static bool trans_VPERM(DisasContext *ctx, arg_VA *a)
     return true;
 }
 
-static bool trans_VPERMR(DisasContext *ctx, arg_VA *a)
-{
-    TCGv_ptr vrt, vra, vrb, vrc;
-
-    REQUIRE_INSNS_FLAGS2(ctx, ISA300);
-    REQUIRE_VECTOR(ctx);
-
-    vrt = gen_avr_ptr(a->vrt);
-    vra = gen_avr_ptr(a->vra);
-    vrb = gen_avr_ptr(a->vrb);
-    vrc = gen_avr_ptr(a->rc);
-
-    gen_helper_VPERMR(vrt, vra, vrb, vrc);
-
-    tcg_temp_free_ptr(vrt);
-    tcg_temp_free_ptr(vra);
-    tcg_temp_free_ptr(vrb);
-    tcg_temp_free_ptr(vrc);
-
-    return true;
-}
+TRANS_FLAGS(ALTIVEC, VPERM, do_va_helper, gen_helper_VPERM)
+TRANS_FLAGS2(ISA300, VPERMR, do_va_helper, gen_helper_VPERMR)
 
 static bool trans_VSEL(DisasContext *ctx, arg_VA *a)
 {

From b2dc03a5c3a2bbfdf74121cd10d007803ea61e34 Mon Sep 17 00:00:00 2001
From: Matheus Ferst <matheus.ferst@eldorado.org.br>
Date: Tue, 17 May 2022 09:39:27 -0300
Subject: [PATCH 551/935] target/ppc: declare vmsum[um]bm helpers with call
 flags

Move vmsumubm and vmsummbm to decodetree, declare both helpers with
TCG_CALL_NO_RWG, and drop the unused env argument.

Signed-off-by: Matheus Ferst <matheus.ferst@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517123929.284511-11-matheus.ferst@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/helper.h                 | 4 ++--
 target/ppc/insn32.decode            | 3 +++
 target/ppc/int_helper.c             | 6 ++----
 target/ppc/translate/vmx-impl.c.inc | 5 ++++-
 target/ppc/translate/vmx-ops.c.inc  | 2 --
 5 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/target/ppc/helper.h b/target/ppc/helper.h
index 640a70cd5c..f0761fe38d 100644
--- a/target/ppc/helper.h
+++ b/target/ppc/helper.h
@@ -223,8 +223,8 @@ DEF_HELPER_FLAGS_2(vupkhsw, TCG_CALL_NO_RWG, void, avr, avr)
 DEF_HELPER_FLAGS_2(vupklsb, TCG_CALL_NO_RWG, void, avr, avr)
 DEF_HELPER_FLAGS_2(vupklsh, TCG_CALL_NO_RWG, void, avr, avr)
 DEF_HELPER_FLAGS_2(vupklsw, TCG_CALL_NO_RWG, void, avr, avr)
-DEF_HELPER_5(vmsumubm, void, env, avr, avr, avr, avr)
-DEF_HELPER_5(vmsummbm, void, env, avr, avr, avr, avr)
+DEF_HELPER_FLAGS_4(VMSUMUBM, TCG_CALL_NO_RWG, void, avr, avr, avr, avr)
+DEF_HELPER_FLAGS_4(VMSUMMBM, TCG_CALL_NO_RWG, void, avr, avr, avr, avr)
 DEF_HELPER_FLAGS_4(VPERM, TCG_CALL_NO_RWG, void, avr, avr, avr, avr)
 DEF_HELPER_FLAGS_4(VPERMR, TCG_CALL_NO_RWG, void, avr, avr, avr, avr)
 DEF_HELPER_4(vpkshss, void, env, avr, avr, avr)
diff --git a/target/ppc/insn32.decode b/target/ppc/insn32.decode
index 435cf1320c..fdb8d76456 100644
--- a/target/ppc/insn32.decode
+++ b/target/ppc/insn32.decode
@@ -599,6 +599,9 @@ VMULLD          000100 ..... ..... ..... 00111001001    @VX
 
 ## Vector Multiply-Sum Instructions
 
+VMSUMUBM        000100 ..... ..... ..... ..... 100100   @VA
+VMSUMMBM        000100 ..... ..... ..... ..... 100101   @VA
+
 VMSUMCUD        000100 ..... ..... ..... ..... 010111   @VA
 VMSUMUDM        000100 ..... ..... ..... ..... 100011   @VA
 
diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c
index 9a361ad241..85a7442103 100644
--- a/target/ppc/int_helper.c
+++ b/target/ppc/int_helper.c
@@ -875,8 +875,7 @@ VMRG(w, u32, VsrW)
 #undef VMRG_DO
 #undef VMRG
 
-void helper_vmsummbm(CPUPPCState *env, ppc_avr_t *r, ppc_avr_t *a,
-                     ppc_avr_t *b, ppc_avr_t *c)
+void helper_VMSUMMBM(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b, ppc_avr_t *c)
 {
     int32_t prod[16];
     int i;
@@ -928,8 +927,7 @@ void helper_vmsumshs(CPUPPCState *env, ppc_avr_t *r, ppc_avr_t *a,
     }
 }
 
-void helper_vmsumubm(CPUPPCState *env, ppc_avr_t *r, ppc_avr_t *a,
-                     ppc_avr_t *b, ppc_avr_t *c)
+void helper_VMSUMUBM(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b, ppc_avr_t *c)
 {
     uint16_t prod[16];
     int i;
diff --git a/target/ppc/translate/vmx-impl.c.inc b/target/ppc/translate/vmx-impl.c.inc
index e66301c007..4cbd724641 100644
--- a/target/ppc/translate/vmx-impl.c.inc
+++ b/target/ppc/translate/vmx-impl.c.inc
@@ -2587,9 +2587,12 @@ static bool trans_VSEL(DisasContext *ctx, arg_VA *a)
     return true;
 }
 
-GEN_VAFORM_PAIRED(vmsumubm, vmsummbm, 18)
 GEN_VAFORM_PAIRED(vmsumuhm, vmsumuhs, 19)
 GEN_VAFORM_PAIRED(vmsumshm, vmsumshs, 20)
+
+TRANS_FLAGS(ALTIVEC, VMSUMUBM, do_va_helper, gen_helper_VMSUMUBM)
+TRANS_FLAGS(ALTIVEC, VMSUMMBM, do_va_helper, gen_helper_VMSUMMBM)
+
 GEN_VAFORM_PAIRED(vmaddfp, vnmsubfp, 23)
 
 GEN_VXFORM_NOA(vclzb, 1, 28)
diff --git a/target/ppc/translate/vmx-ops.c.inc b/target/ppc/translate/vmx-ops.c.inc
index d960648d52..5b85322c06 100644
--- a/target/ppc/translate/vmx-ops.c.inc
+++ b/target/ppc/translate/vmx-ops.c.inc
@@ -221,11 +221,9 @@ GEN_VXFORM_UIMM(vcfsx, 5, 13),
 GEN_VXFORM_UIMM(vctuxs, 5, 14),
 GEN_VXFORM_UIMM(vctsxs, 5, 15),
 
-
 #define GEN_VAFORM_PAIRED(name0, name1, opc2)                           \
     GEN_HANDLER(name0##_##name1, 0x04, opc2, 0xFF, 0x00000000, PPC_ALTIVEC)
 GEN_VAFORM_PAIRED(vmhaddshs, vmhraddshs, 16),
-GEN_VAFORM_PAIRED(vmsumubm, vmsummbm, 18),
 GEN_VAFORM_PAIRED(vmsumuhm, vmsumuhs, 19),
 GEN_VAFORM_PAIRED(vmsumshm, vmsumshs, 20),
 GEN_VAFORM_PAIRED(vmaddfp, vnmsubfp, 23),

From 89a5a1aee2df29c311d11386923b750bf1ea6bc2 Mon Sep 17 00:00:00 2001
From: Matheus Ferst <matheus.ferst@eldorado.org.br>
Date: Tue, 17 May 2022 09:39:28 -0300
Subject: [PATCH 552/935] target/ppc: declare vmsumuh[ms] helper with call
 flags

Move vmsumuhm and vmsumuhs to decodetree, declare vmsumuhm helper with
TCG_CALL_NO_RWG, and drop the unused env argument.

Signed-off-by: Matheus Ferst <matheus.ferst@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517123929.284511-12-matheus.ferst@eldorado.org.br>
[danielhb: added #undef VMSUMUHM to fix ppc64 build]
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/helper.h                 |  4 ++--
 target/ppc/insn32.decode            |  2 ++
 target/ppc/int_helper.c             |  5 ++---
 target/ppc/translate/vmx-impl.c.inc | 24 ++++++++++++++++++++++--
 target/ppc/translate/vmx-ops.c.inc  |  1 -
 tcg/ppc/tcg-target.c.inc            |  1 +
 6 files changed, 29 insertions(+), 8 deletions(-)

diff --git a/target/ppc/helper.h b/target/ppc/helper.h
index f0761fe38d..5127851f2c 100644
--- a/target/ppc/helper.h
+++ b/target/ppc/helper.h
@@ -242,8 +242,8 @@ DEF_HELPER_4(vpkudum, void, env, avr, avr, avr)
 DEF_HELPER_FLAGS_3(vpkpx, TCG_CALL_NO_RWG, void, avr, avr, avr)
 DEF_HELPER_5(vmhaddshs, void, env, avr, avr, avr, avr)
 DEF_HELPER_5(vmhraddshs, void, env, avr, avr, avr, avr)
-DEF_HELPER_5(vmsumuhm, void, env, avr, avr, avr, avr)
-DEF_HELPER_5(vmsumuhs, void, env, avr, avr, avr, avr)
+DEF_HELPER_FLAGS_4(VMSUMUHM, TCG_CALL_NO_RWG, void, avr, avr, avr, avr)
+DEF_HELPER_5(VMSUMUHS, void, env, avr, avr, avr, avr)
 DEF_HELPER_5(vmsumshm, void, env, avr, avr, avr, avr)
 DEF_HELPER_5(vmsumshs, void, env, avr, avr, avr, avr)
 DEF_HELPER_FLAGS_4(vmladduhm, TCG_CALL_NO_RWG, void, avr, avr, avr, avr)
diff --git a/target/ppc/insn32.decode b/target/ppc/insn32.decode
index fdb8d76456..43ea03c3e7 100644
--- a/target/ppc/insn32.decode
+++ b/target/ppc/insn32.decode
@@ -601,6 +601,8 @@ VMULLD          000100 ..... ..... ..... 00111001001    @VX
 
 VMSUMUBM        000100 ..... ..... ..... ..... 100100   @VA
 VMSUMMBM        000100 ..... ..... ..... ..... 100101   @VA
+VMSUMUHM        000100 ..... ..... ..... ..... 100110   @VA
+VMSUMUHS        000100 ..... ..... ..... ..... 100111   @VA
 
 VMSUMCUD        000100 ..... ..... ..... ..... 010111   @VA
 VMSUMUDM        000100 ..... ..... ..... ..... 100011   @VA
diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c
index 85a7442103..9285a1c2a1 100644
--- a/target/ppc/int_helper.c
+++ b/target/ppc/int_helper.c
@@ -942,8 +942,7 @@ void helper_VMSUMUBM(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b, ppc_avr_t *c)
     }
 }
 
-void helper_vmsumuhm(CPUPPCState *env, ppc_avr_t *r, ppc_avr_t *a,
-                     ppc_avr_t *b, ppc_avr_t *c)
+void helper_VMSUMUHM(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b, ppc_avr_t *c)
 {
     uint32_t prod[8];
     int i;
@@ -957,7 +956,7 @@ void helper_vmsumuhm(CPUPPCState *env, ppc_avr_t *r, ppc_avr_t *a,
     }
 }
 
-void helper_vmsumuhs(CPUPPCState *env, ppc_avr_t *r, ppc_avr_t *a,
+void helper_VMSUMUHS(CPUPPCState *env, ppc_avr_t *r, ppc_avr_t *a,
                      ppc_avr_t *b, ppc_avr_t *c)
 {
     uint32_t prod[8];
diff --git a/target/ppc/translate/vmx-impl.c.inc b/target/ppc/translate/vmx-impl.c.inc
index 4cbd724641..da81296b96 100644
--- a/target/ppc/translate/vmx-impl.c.inc
+++ b/target/ppc/translate/vmx-impl.c.inc
@@ -2587,11 +2587,31 @@ static bool trans_VSEL(DisasContext *ctx, arg_VA *a)
     return true;
 }
 
-GEN_VAFORM_PAIRED(vmsumuhm, vmsumuhs, 19)
 GEN_VAFORM_PAIRED(vmsumshm, vmsumshs, 20)
-
 TRANS_FLAGS(ALTIVEC, VMSUMUBM, do_va_helper, gen_helper_VMSUMUBM)
 TRANS_FLAGS(ALTIVEC, VMSUMMBM, do_va_helper, gen_helper_VMSUMMBM)
+TRANS_FLAGS(ALTIVEC, VMSUMUHM, do_va_helper, gen_helper_VMSUMUHM)
+
+static bool do_va_env_helper(DisasContext *ctx, arg_VA *a,
+    void (*gen_helper)(TCGv_ptr, TCGv_ptr, TCGv_ptr, TCGv_ptr, TCGv_ptr))
+{
+    TCGv_ptr vrt, vra, vrb, vrc;
+    REQUIRE_VECTOR(ctx);
+
+    vrt = gen_avr_ptr(a->vrt);
+    vra = gen_avr_ptr(a->vra);
+    vrb = gen_avr_ptr(a->vrb);
+    vrc = gen_avr_ptr(a->rc);
+    gen_helper(cpu_env, vrt, vra, vrb, vrc);
+    tcg_temp_free_ptr(vrt);
+    tcg_temp_free_ptr(vra);
+    tcg_temp_free_ptr(vrb);
+    tcg_temp_free_ptr(vrc);
+
+    return true;
+}
+
+TRANS_FLAGS(ALTIVEC, VMSUMUHS, do_va_env_helper, gen_helper_VMSUMUHS)
 
 GEN_VAFORM_PAIRED(vmaddfp, vnmsubfp, 23)
 
diff --git a/target/ppc/translate/vmx-ops.c.inc b/target/ppc/translate/vmx-ops.c.inc
index 5b85322c06..15b3e06410 100644
--- a/target/ppc/translate/vmx-ops.c.inc
+++ b/target/ppc/translate/vmx-ops.c.inc
@@ -224,7 +224,6 @@ GEN_VXFORM_UIMM(vctsxs, 5, 15),
 #define GEN_VAFORM_PAIRED(name0, name1, opc2)                           \
     GEN_HANDLER(name0##_##name1, 0x04, opc2, 0xFF, 0x00000000, PPC_ALTIVEC)
 GEN_VAFORM_PAIRED(vmhaddshs, vmhraddshs, 16),
-GEN_VAFORM_PAIRED(vmsumuhm, vmsumuhs, 19),
 GEN_VAFORM_PAIRED(vmsumshm, vmsumshs, 20),
 GEN_VAFORM_PAIRED(vmaddfp, vnmsubfp, 23),
 
diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc
index cfcd121f9c..fc8ae47293 100644
--- a/tcg/ppc/tcg-target.c.inc
+++ b/tcg/ppc/tcg-target.c.inc
@@ -4008,3 +4008,4 @@ void tcg_register_jit(const void *buf, size_t buf_size)
 #undef VMULOUB
 #undef VMULOUH
 #undef VMULOUW
+#undef VMSUMUHM

From 6f52f731a666d8aeaac5a10b208e9ca6773fb1a6 Mon Sep 17 00:00:00 2001
From: Matheus Ferst <matheus.ferst@eldorado.org.br>
Date: Tue, 17 May 2022 09:39:29 -0300
Subject: [PATCH 553/935] target/ppc: declare vmsumsh[ms] helper with call
 flags

Move vmsumshm and vmsumshs to decodetree, declare vmsumshm helper with
TCG_CALL_NO_RWG, and drop the unused env argument.

Signed-off-by: Matheus Ferst <matheus.ferst@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517123929.284511-13-matheus.ferst@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/helper.h                 | 4 ++--
 target/ppc/insn32.decode            | 2 ++
 target/ppc/int_helper.c             | 5 ++---
 target/ppc/translate/vmx-impl.c.inc | 3 ++-
 target/ppc/translate/vmx-ops.c.inc  | 1 -
 5 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/target/ppc/helper.h b/target/ppc/helper.h
index 5127851f2c..5e43920b9e 100644
--- a/target/ppc/helper.h
+++ b/target/ppc/helper.h
@@ -244,8 +244,8 @@ DEF_HELPER_5(vmhaddshs, void, env, avr, avr, avr, avr)
 DEF_HELPER_5(vmhraddshs, void, env, avr, avr, avr, avr)
 DEF_HELPER_FLAGS_4(VMSUMUHM, TCG_CALL_NO_RWG, void, avr, avr, avr, avr)
 DEF_HELPER_5(VMSUMUHS, void, env, avr, avr, avr, avr)
-DEF_HELPER_5(vmsumshm, void, env, avr, avr, avr, avr)
-DEF_HELPER_5(vmsumshs, void, env, avr, avr, avr, avr)
+DEF_HELPER_FLAGS_4(VMSUMSHM, TCG_CALL_NO_RWG, void, avr, avr, avr, avr)
+DEF_HELPER_5(VMSUMSHS, void, env, avr, avr, avr, avr)
 DEF_HELPER_FLAGS_4(vmladduhm, TCG_CALL_NO_RWG, void, avr, avr, avr, avr)
 DEF_HELPER_FLAGS_2(mtvscr, TCG_CALL_NO_RWG, void, env, i32)
 DEF_HELPER_FLAGS_1(mfvscr, TCG_CALL_NO_RWG, i32, env)
diff --git a/target/ppc/insn32.decode b/target/ppc/insn32.decode
index 43ea03c3e7..f001c02a8c 100644
--- a/target/ppc/insn32.decode
+++ b/target/ppc/insn32.decode
@@ -601,6 +601,8 @@ VMULLD          000100 ..... ..... ..... 00111001001    @VX
 
 VMSUMUBM        000100 ..... ..... ..... ..... 100100   @VA
 VMSUMMBM        000100 ..... ..... ..... ..... 100101   @VA
+VMSUMSHM        000100 ..... ..... ..... ..... 101000   @VA
+VMSUMSHS        000100 ..... ..... ..... ..... 101001   @VA
 VMSUMUHM        000100 ..... ..... ..... ..... 100110   @VA
 VMSUMUHS        000100 ..... ..... ..... ..... 100111   @VA
 
diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c
index 9285a1c2a1..b9dd15d607 100644
--- a/target/ppc/int_helper.c
+++ b/target/ppc/int_helper.c
@@ -890,8 +890,7 @@ void helper_VMSUMMBM(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b, ppc_avr_t *c)
     }
 }
 
-void helper_vmsumshm(CPUPPCState *env, ppc_avr_t *r, ppc_avr_t *a,
-                     ppc_avr_t *b, ppc_avr_t *c)
+void helper_VMSUMSHM(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b, ppc_avr_t *c)
 {
     int32_t prod[8];
     int i;
@@ -905,7 +904,7 @@ void helper_vmsumshm(CPUPPCState *env, ppc_avr_t *r, ppc_avr_t *a,
     }
 }
 
-void helper_vmsumshs(CPUPPCState *env, ppc_avr_t *r, ppc_avr_t *a,
+void helper_VMSUMSHS(CPUPPCState *env, ppc_avr_t *r, ppc_avr_t *a,
                      ppc_avr_t *b, ppc_avr_t *c)
 {
     int32_t prod[8];
diff --git a/target/ppc/translate/vmx-impl.c.inc b/target/ppc/translate/vmx-impl.c.inc
index da81296b96..d7524c3204 100644
--- a/target/ppc/translate/vmx-impl.c.inc
+++ b/target/ppc/translate/vmx-impl.c.inc
@@ -2587,9 +2587,9 @@ static bool trans_VSEL(DisasContext *ctx, arg_VA *a)
     return true;
 }
 
-GEN_VAFORM_PAIRED(vmsumshm, vmsumshs, 20)
 TRANS_FLAGS(ALTIVEC, VMSUMUBM, do_va_helper, gen_helper_VMSUMUBM)
 TRANS_FLAGS(ALTIVEC, VMSUMMBM, do_va_helper, gen_helper_VMSUMMBM)
+TRANS_FLAGS(ALTIVEC, VMSUMSHM, do_va_helper, gen_helper_VMSUMSHM)
 TRANS_FLAGS(ALTIVEC, VMSUMUHM, do_va_helper, gen_helper_VMSUMUHM)
 
 static bool do_va_env_helper(DisasContext *ctx, arg_VA *a,
@@ -2612,6 +2612,7 @@ static bool do_va_env_helper(DisasContext *ctx, arg_VA *a,
 }
 
 TRANS_FLAGS(ALTIVEC, VMSUMUHS, do_va_env_helper, gen_helper_VMSUMUHS)
+TRANS_FLAGS(ALTIVEC, VMSUMSHS, do_va_env_helper, gen_helper_VMSUMSHS)
 
 GEN_VAFORM_PAIRED(vmaddfp, vnmsubfp, 23)
 
diff --git a/target/ppc/translate/vmx-ops.c.inc b/target/ppc/translate/vmx-ops.c.inc
index 15b3e06410..d7cc57868e 100644
--- a/target/ppc/translate/vmx-ops.c.inc
+++ b/target/ppc/translate/vmx-ops.c.inc
@@ -224,7 +224,6 @@ GEN_VXFORM_UIMM(vctsxs, 5, 15),
 #define GEN_VAFORM_PAIRED(name0, name1, opc2)                           \
     GEN_HANDLER(name0##_##name1, 0x04, opc2, 0xFF, 0x00000000, PPC_ALTIVEC)
 GEN_VAFORM_PAIRED(vmhaddshs, vmhraddshs, 16),
-GEN_VAFORM_PAIRED(vmsumshm, vmsumshs, 20),
 GEN_VAFORM_PAIRED(vmaddfp, vnmsubfp, 23),
 
 GEN_VXFORM_DUAL(vclzb, vpopcntb, 1, 28, PPC_NONE, PPC2_ALTIVEC_207),

From fcb830af30adaef6219f6b712f24ec3e794cf2c8 Mon Sep 17 00:00:00 2001
From: Nicholas Piggin <npiggin@gmail.com>
Date: Thu, 19 May 2022 23:59:05 +1000
Subject: [PATCH 554/935] target/ppc: Fix eieio memory ordering semantics

The generated eieio memory ordering semantics do not match the
instruction definition in the architecture. Add a big comment to
explain this strange instruction and correct the memory ordering
behaviour.

Signed-off: Nicholas Piggin <npiggin@gmail.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220519135908.21282-2-npiggin@gmail.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/translate.c | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)

diff --git a/target/ppc/translate.c b/target/ppc/translate.c
index fa34f81c30..eb42f7e459 100644
--- a/target/ppc/translate.c
+++ b/target/ppc/translate.c
@@ -3513,7 +3513,32 @@ static void gen_stswx(DisasContext *ctx)
 /* eieio */
 static void gen_eieio(DisasContext *ctx)
 {
-    TCGBar bar = TCG_MO_LD_ST;
+    TCGBar bar = TCG_MO_ALL;
+
+    /*
+     * eieio has complex semanitcs. It provides memory ordering between
+     * operations in the set:
+     * - loads from CI memory.
+     * - stores to CI memory.
+     * - stores to WT memory.
+     *
+     * It separately also orders memory for operations in the set:
+     * - stores to cacheble memory.
+     *
+     * It also serializes instructions:
+     * - dcbt and dcbst.
+     *
+     * It separately serializes:
+     * - tlbie and tlbsync.
+     *
+     * And separately serializes:
+     * - slbieg, slbiag, and slbsync.
+     *
+     * The end result is that CI memory ordering requires TCG_MO_ALL
+     * and it is not possible to special-case more relaxed ordering for
+     * cacheable accesses. TCG_BAR_SC is required to provide this
+     * serialization.
+     */
 
     /*
      * POWER9 has a eieio instruction variant using bit 6 as a hint to

From 9d82353826422ed0e34ad76961fe0cae5f67e58e Mon Sep 17 00:00:00 2001
From: Nicholas Piggin <npiggin@gmail.com>
Date: Thu, 19 May 2022 23:59:06 +1000
Subject: [PATCH 555/935] tcg/ppc: ST_ST memory ordering is not provided with
 eieio

eieio does not provide ordering between stores to CI memory and stores
to cacheable memory so it can't be used as a general ST_ST barrier.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-of-by: Nicholas Piggin <npiggin@gmail.com>
Message-Id: <20220519135908.21282-3-npiggin@gmail.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 tcg/ppc/tcg-target.c.inc | 2 --
 1 file changed, 2 deletions(-)

diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc
index fc8ae47293..4750091c9c 100644
--- a/tcg/ppc/tcg-target.c.inc
+++ b/tcg/ppc/tcg-target.c.inc
@@ -1836,8 +1836,6 @@ static void tcg_out_mb(TCGContext *s, TCGArg a0)
     a0 &= TCG_MO_ALL;
     if (a0 == TCG_MO_LD_LD) {
         insn = LWSYNC;
-    } else if (a0 == TCG_MO_ST_ST) {
-        insn = EIEIO;
     }
     tcg_out32(s, insn);
 }

From fc879703f74851e3e861894a0c4a6902877d0c2c Mon Sep 17 00:00:00 2001
From: Nicholas Piggin <npiggin@gmail.com>
Date: Thu, 19 May 2022 23:59:07 +1000
Subject: [PATCH 556/935] tcg/ppc: Optimize memory ordering generation with
 lwsync

lwsync orders more than just LD_LD, importantly it matches x86 and
s390 default memory ordering.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220519135908.21282-4-npiggin@gmail.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 tcg/ppc/tcg-target.c.inc | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc
index 4750091c9c..de4483e43b 100644
--- a/tcg/ppc/tcg-target.c.inc
+++ b/tcg/ppc/tcg-target.c.inc
@@ -1832,11 +1832,14 @@ static void tcg_out_brcond2 (TCGContext *s, const TCGArg *args,
 
 static void tcg_out_mb(TCGContext *s, TCGArg a0)
 {
-    uint32_t insn = HWSYNC;
-    a0 &= TCG_MO_ALL;
-    if (a0 == TCG_MO_LD_LD) {
+    uint32_t insn;
+
+    if (a0 & TCG_MO_ST_LD) {
+        insn = HWSYNC;
+    } else {
         insn = LWSYNC;
     }
+
     tcg_out32(s, insn);
 }
 

From 03abfd90cfb02aa08f44bbb7141b0aaaf69042ef Mon Sep 17 00:00:00 2001
From: Nicholas Piggin <npiggin@gmail.com>
Date: Thu, 19 May 2022 23:59:08 +1000
Subject: [PATCH 557/935] target/ppc: Implement lwsync with weaker memory
 ordering

This allows an x86 host to no-op lwsyncs, and ppc host can use lwsync
rather than sync.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220519135908.21282-5-npiggin@gmail.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h       |  4 +++-
 target/ppc/cpu_init.c  | 13 +++++++------
 target/ppc/machine.c   |  3 ++-
 target/ppc/translate.c |  8 +++++++-
 4 files changed, 19 insertions(+), 9 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 901ded79e9..bf8f8aad2c 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -2273,6 +2273,8 @@ enum {
     PPC2_ISA300        = 0x0000000000080000ULL,
     /* POWER ISA 3.1                                                         */
     PPC2_ISA310        = 0x0000000000100000ULL,
+    /*   lwsync instruction                                                  */
+    PPC2_MEM_LWSYNC    = 0x0000000000200000ULL,
 
 #define PPC_TCG_INSNS2 (PPC2_BOOKE206 | PPC2_VSX | PPC2_PRCNTL | PPC2_DBRX | \
                         PPC2_ISA205 | PPC2_VSX207 | PPC2_PERM_ISA206 | \
@@ -2281,7 +2283,7 @@ enum {
                         PPC2_BCTAR_ISA207 | PPC2_LSQ_ISA207 | \
                         PPC2_ALTIVEC_207 | PPC2_ISA207S | PPC2_DFP | \
                         PPC2_FP_CVT_S64 | PPC2_TM | PPC2_PM_ISA206 | \
-                        PPC2_ISA300 | PPC2_ISA310)
+                        PPC2_ISA300 | PPC2_ISA310 | PPC2_MEM_LWSYNC)
 };
 
 /*****************************************************************************/
diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
index 527ad40fcb..0f891afa04 100644
--- a/target/ppc/cpu_init.c
+++ b/target/ppc/cpu_init.c
@@ -5769,7 +5769,7 @@ POWERPC_FAMILY(970)(ObjectClass *oc, void *data)
                        PPC_MEM_TLBIE | PPC_MEM_TLBSYNC |
                        PPC_64B | PPC_ALTIVEC |
                        PPC_SEGMENT_64B | PPC_SLBI;
-    pcc->insns_flags2 = PPC2_FP_CVT_S64;
+    pcc->insns_flags2 = PPC2_FP_CVT_S64 | PPC2_MEM_LWSYNC;
     pcc->msr_mask = (1ull << MSR_SF) |
                     (1ull << MSR_VR) |
                     (1ull << MSR_POW) |
@@ -5846,7 +5846,7 @@ POWERPC_FAMILY(POWER5P)(ObjectClass *oc, void *data)
                        PPC_64B |
                        PPC_POPCNTB |
                        PPC_SEGMENT_64B | PPC_SLBI;
-    pcc->insns_flags2 = PPC2_FP_CVT_S64;
+    pcc->insns_flags2 = PPC2_FP_CVT_S64 | PPC2_MEM_LWSYNC;
     pcc->msr_mask = (1ull << MSR_SF) |
                     (1ull << MSR_VR) |
                     (1ull << MSR_POW) |
@@ -5985,7 +5985,7 @@ POWERPC_FAMILY(POWER7)(ObjectClass *oc, void *data)
                         PPC2_PERM_ISA206 | PPC2_DIVE_ISA206 |
                         PPC2_ATOMIC_ISA206 | PPC2_FP_CVT_ISA206 |
                         PPC2_FP_TST_ISA206 | PPC2_FP_CVT_S64 |
-                        PPC2_PM_ISA206;
+                        PPC2_PM_ISA206 | PPC2_MEM_LWSYNC;
     pcc->msr_mask = (1ull << MSR_SF) |
                     (1ull << MSR_VR) |
                     (1ull << MSR_VSX) |
@@ -6159,7 +6159,7 @@ POWERPC_FAMILY(POWER8)(ObjectClass *oc, void *data)
                         PPC2_FP_TST_ISA206 | PPC2_BCTAR_ISA207 |
                         PPC2_LSQ_ISA207 | PPC2_ALTIVEC_207 |
                         PPC2_ISA205 | PPC2_ISA207S | PPC2_FP_CVT_S64 |
-                        PPC2_TM | PPC2_PM_ISA206;
+                        PPC2_TM | PPC2_PM_ISA206 | PPC2_MEM_LWSYNC;
     pcc->msr_mask = (1ull << MSR_SF) |
                     (1ull << MSR_HV) |
                     (1ull << MSR_TM) |
@@ -6379,7 +6379,7 @@ POWERPC_FAMILY(POWER9)(ObjectClass *oc, void *data)
                         PPC2_FP_TST_ISA206 | PPC2_BCTAR_ISA207 |
                         PPC2_LSQ_ISA207 | PPC2_ALTIVEC_207 |
                         PPC2_ISA205 | PPC2_ISA207S | PPC2_FP_CVT_S64 |
-                        PPC2_TM | PPC2_ISA300 | PPC2_PRCNTL;
+                        PPC2_TM | PPC2_ISA300 | PPC2_PRCNTL | PPC2_MEM_LWSYNC;
     pcc->msr_mask = (1ull << MSR_SF) |
                     (1ull << MSR_HV) |
                     (1ull << MSR_TM) |
@@ -6596,7 +6596,8 @@ POWERPC_FAMILY(POWER10)(ObjectClass *oc, void *data)
                         PPC2_FP_TST_ISA206 | PPC2_BCTAR_ISA207 |
                         PPC2_LSQ_ISA207 | PPC2_ALTIVEC_207 |
                         PPC2_ISA205 | PPC2_ISA207S | PPC2_FP_CVT_S64 |
-                        PPC2_TM | PPC2_ISA300 | PPC2_PRCNTL | PPC2_ISA310;
+                        PPC2_TM | PPC2_ISA300 | PPC2_PRCNTL | PPC2_ISA310 |
+                        PPC2_MEM_LWSYNC;
     pcc->msr_mask = (1ull << MSR_SF) |
                     (1ull << MSR_HV) |
                     (1ull << MSR_TM) |
diff --git a/target/ppc/machine.c b/target/ppc/machine.c
index 7104a5c67e..a7d9036c09 100644
--- a/target/ppc/machine.c
+++ b/target/ppc/machine.c
@@ -157,7 +157,8 @@ static int cpu_pre_save(void *opaque)
         | PPC2_ATOMIC_ISA206 | PPC2_FP_CVT_ISA206
         | PPC2_FP_TST_ISA206 | PPC2_BCTAR_ISA207
         | PPC2_LSQ_ISA207 | PPC2_ALTIVEC_207
-        | PPC2_ISA205 | PPC2_ISA207S | PPC2_FP_CVT_S64 | PPC2_TM;
+        | PPC2_ISA205 | PPC2_ISA207S | PPC2_FP_CVT_S64 | PPC2_TM
+        | PPC2_MEM_LWSYNC;
 
     env->spr[SPR_LR] = env->lr;
     env->spr[SPR_CTR] = env->ctr;
diff --git a/target/ppc/translate.c b/target/ppc/translate.c
index eb42f7e459..1d6daa4608 100644
--- a/target/ppc/translate.c
+++ b/target/ppc/translate.c
@@ -4041,8 +4041,13 @@ static void gen_stqcx_(DisasContext *ctx)
 /* sync */
 static void gen_sync(DisasContext *ctx)
 {
+    TCGBar bar = TCG_MO_ALL;
     uint32_t l = (ctx->opcode >> 21) & 3;
 
+    if ((l == 1) && (ctx->insns_flags2 & PPC2_MEM_LWSYNC)) {
+        bar = TCG_MO_LD_LD | TCG_MO_LD_ST | TCG_MO_ST_ST;
+    }
+
     /*
      * We may need to check for a pending TLB flush.
      *
@@ -4054,7 +4059,8 @@ static void gen_sync(DisasContext *ctx)
     if (((l == 2) || !(ctx->insns_flags & PPC_64B)) && !ctx->pr) {
         gen_check_tlb_flush(ctx, true);
     }
-    tcg_gen_mb(TCG_MO_ALL | TCG_BAR_SC);
+
+    tcg_gen_mb(bar | TCG_BAR_SC);
 }
 
 /* wait */

From a702c5339eda791b969ed531ce99456df7ca8451 Mon Sep 17 00:00:00 2001
From: "Lucas Mateus Castro (alqotel)" <lucas.araujo@eldorado.org.br>
Date: Tue, 24 May 2022 11:05:30 -0300
Subject: [PATCH 558/935] target/ppc: Implement xxm[tf]acc and xxsetaccz

Implement the following PowerISA v3.1 instructions:
xxmfacc: VSX Move From Accumulator
xxmtacc: VSX Move To Accumulator
xxsetaccz: VSX Set Accumulator to Zero

The PowerISA 3.1 mentions that for the current version of the
architecture, "the hardware implementation provides the effect of ACC[i]
and VSRs 4*i to 4*i + 3 logically containing the same data" and "The
Accumulators introduce no new logical state at this time" (page 501).
For now it seems unnecessary to create new structures, so this patch
just uses ACC[i] as VSRs 4*i to 4*i+3 and therefore move to and from
accumulators are no-ops.

Signed-off-by: Lucas Mateus Castro (alqotel) <lucas.araujo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220524140537.27451-2-lucas.araujo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h                    |  5 +++++
 target/ppc/insn32.decode            |  9 +++++++++
 target/ppc/translate/vsx-impl.c.inc | 31 +++++++++++++++++++++++++++++
 3 files changed, 45 insertions(+)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index bf8f8aad2c..c865206827 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -2663,6 +2663,11 @@ static inline int vsr_full_offset(int i)
     return offsetof(CPUPPCState, vsr[i].u64[0]);
 }
 
+static inline int acc_full_offset(int i)
+{
+    return vsr_full_offset(i * 4);
+}
+
 static inline int fpr_offset(int i)
 {
     return vsr64_offset(i, true);
diff --git a/target/ppc/insn32.decode b/target/ppc/insn32.decode
index f001c02a8c..c0f545ca38 100644
--- a/target/ppc/insn32.decode
+++ b/target/ppc/insn32.decode
@@ -154,6 +154,9 @@
 &X_vrt_frbp     vrt frbp
 @X_vrt_frbp     ...... vrt:5 ..... ....0 .......... .           &X_vrt_frbp frbp=%x_frbp
 
+&X_a            ra
+@X_a            ...... ra:3 .. ..... ..... .......... .         &X_a
+
 %xx_xt          0:1 21:5
 %xx_xb          1:1 11:5
 %xx_xa          2:1 16:5
@@ -734,3 +737,9 @@ XVTLSBB         111100 ... -- 00010 ..... 111011011 . - @XX2_bf_xb
 &XL_s           s:uint8_t
 @XL_s           ......-------------- s:1 .......... -   &XL_s
 RFEBB           010011-------------- .   0010010010 -   @XL_s
+
+## Accumulator Instructions
+
+XXMFACC         011111 ... -- 00000 ----- 0010110001 -   @X_a
+XXMTACC         011111 ... -- 00001 ----- 0010110001 -   @X_a
+XXSETACCZ       011111 ... -- 00011 ----- 0010110001 -   @X_a
diff --git a/target/ppc/translate/vsx-impl.c.inc b/target/ppc/translate/vsx-impl.c.inc
index 900c1a1ab2..235be360e2 100644
--- a/target/ppc/translate/vsx-impl.c.inc
+++ b/target/ppc/translate/vsx-impl.c.inc
@@ -2816,6 +2816,37 @@ static bool trans_XVCVBF16SPN(DisasContext *ctx, arg_XX2 *a)
     return true;
 }
 
+    /*
+     *  The PowerISA 3.1 mentions that for the current version of the
+     *  architecture, "the hardware implementation provides the effect of
+     *  ACC[i] and VSRs 4*i to 4*i + 3 logically containing the same data"
+     *  and "The Accumulators introduce no new logical state at this time"
+     *  (page 501). For now it seems unnecessary to create new structures,
+     *  so ACC[i] is the same as VSRs 4*i to 4*i+3 and therefore
+     *  move to and from accumulators are no-ops.
+     */
+static bool trans_XXMFACC(DisasContext *ctx, arg_X_a *a)
+{
+    REQUIRE_INSNS_FLAGS2(ctx, ISA310);
+    REQUIRE_VSX(ctx);
+    return true;
+}
+
+static bool trans_XXMTACC(DisasContext *ctx, arg_X_a *a)
+{
+    REQUIRE_INSNS_FLAGS2(ctx, ISA310);
+    REQUIRE_VSX(ctx);
+    return true;
+}
+
+static bool trans_XXSETACCZ(DisasContext *ctx, arg_X_a *a)
+{
+    REQUIRE_INSNS_FLAGS2(ctx, ISA310);
+    REQUIRE_VSX(ctx);
+    tcg_gen_gvec_dup_imm(MO_64, acc_full_offset(a->ra), 64, 64, 0);
+    return true;
+}
+
 #undef GEN_XX2FORM
 #undef GEN_XX3FORM
 #undef GEN_XX2IFORM

From 345531533f26df49e74f16dafc88408408173ece Mon Sep 17 00:00:00 2001
From: "Lucas Mateus Castro (alqotel)" <lucas.araujo@eldorado.org.br>
Date: Tue, 24 May 2022 11:05:31 -0300
Subject: [PATCH 559/935] target/ppc: Implemented xvi*ger* instructions

Implement the following PowerISA v3.1 instructions:
xvi4ger8:     VSX Vector 8-bit Signed/Unsigned Integer GER (rank-4 update)
xvi4ger8pp:   VSX Vector 8-bit Signed/Unsigned Integer GER (rank-4 update)
Positive multiply, Positive accumulate
xvi8ger4:     VSX Vector 4-bit Signed Integer GER (rank-8 update)
xvi8ger4pp:   VSX Vector 4-bit Signed Integer GER (rank-8 update)
Positive multiply, Positive accumulate
xvi8ger4spp:  VSX Vector 8-bit Signed/Unsigned Integer GER (rank-4 update)
with Saturate Positive multiply, Positive accumulate
xvi16ger2:    VSX Vector 16-bit Signed Integer GER (rank-2 update)
xvi16ger2pp:  VSX Vector 16-bit Signed Integer GER (rank-2 update)
Positive multiply, Positive accumulate
xvi16ger2s:   VSX Vector 16-bit Signed Integer GER (rank-2 update)
with Saturation
xvi16ger2spp: VSX Vector 16-bit Signed Integer GER (rank-2 update)
with Saturation Positive multiply, Positive accumulate

Signed-off-by: Lucas Mateus Castro (alqotel) <lucas.araujo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220524140537.27451-3-lucas.araujo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h                    |   1 +
 target/ppc/helper.h                 |  13 +++
 target/ppc/insn32.decode            |  18 ++++
 target/ppc/int_helper.c             | 130 ++++++++++++++++++++++++++++
 target/ppc/internal.h               |  15 ++++
 target/ppc/translate/vsx-impl.c.inc |  41 +++++++++
 6 files changed, 218 insertions(+)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index c865206827..dff3ca8222 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -238,6 +238,7 @@ typedef union _ppc_vsr_t {
 
 typedef ppc_vsr_t ppc_avr_t;
 typedef ppc_vsr_t ppc_fprp_t;
+typedef ppc_vsr_t ppc_acc_t;
 
 #if !defined(CONFIG_USER_ONLY)
 /* Software TLB cache */
diff --git a/target/ppc/helper.h b/target/ppc/helper.h
index 5e43920b9e..1666797edf 100644
--- a/target/ppc/helper.h
+++ b/target/ppc/helper.h
@@ -133,6 +133,10 @@ DEF_HELPER_FLAGS_1(ftsqrt, TCG_CALL_NO_RWG_SE, i32, i64)
 #define dh_ctype_vsr ppc_vsr_t *
 #define dh_typecode_vsr dh_typecode_ptr
 
+#define dh_alias_acc ptr
+#define dh_ctype_acc ppc_acc_t *
+#define dh_typecode_acc dh_typecode_ptr
+
 DEF_HELPER_FLAGS_3(vavgub, TCG_CALL_NO_RWG, void, avr, avr, avr)
 DEF_HELPER_FLAGS_3(vavguh, TCG_CALL_NO_RWG, void, avr, avr, avr)
 DEF_HELPER_FLAGS_3(vavguw, TCG_CALL_NO_RWG, void, avr, avr, avr)
@@ -537,6 +541,15 @@ DEF_HELPER_FLAGS_5(XXBLENDVB, TCG_CALL_NO_RWG, void, vsr, vsr, vsr, vsr, i32)
 DEF_HELPER_FLAGS_5(XXBLENDVH, TCG_CALL_NO_RWG, void, vsr, vsr, vsr, vsr, i32)
 DEF_HELPER_FLAGS_5(XXBLENDVW, TCG_CALL_NO_RWG, void, vsr, vsr, vsr, vsr, i32)
 DEF_HELPER_FLAGS_5(XXBLENDVD, TCG_CALL_NO_RWG, void, vsr, vsr, vsr, vsr, i32)
+DEF_HELPER_5(XVI4GER8, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVI4GER8PP, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVI8GER4, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVI8GER4PP, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVI8GER4SPP, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVI16GER2, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVI16GER2S, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVI16GER2PP, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVI16GER2SPP, void, env, vsr, vsr, acc, i32)
 
 DEF_HELPER_2(efscfsi, i32, env, i32)
 DEF_HELPER_2(efscfui, i32, env, i32)
diff --git a/target/ppc/insn32.decode b/target/ppc/insn32.decode
index c0f545ca38..0e189fe2da 100644
--- a/target/ppc/insn32.decode
+++ b/target/ppc/insn32.decode
@@ -175,6 +175,12 @@
 &XX3            xt xa xb
 @XX3            ...... ..... ..... ..... ........ ...           &XX3 xt=%xx_xt xa=%xx_xa xb=%xx_xb
 
+# 32 bit GER instructions have all mask bits considered 1
+&MMIRR_XX3      xa xb xt pmsk xmsk ymsk
+%xx_at          23:3
+@XX3_at         ...... ... .. ..... ..... ........ ...          &MMIRR_XX3 xt=%xx_at xb=%xx_xb \
+                                                                pmsk=255 xmsk=15 ymsk=15
+
 &XX3_dm         xt xa xb dm
 @XX3_dm         ...... ..... ..... ..... . dm:2 ..... ...       &XX3_dm xt=%xx_xt xa=%xx_xa xb=%xx_xb
 
@@ -743,3 +749,15 @@ RFEBB           010011-------------- .   0010010010 -   @XL_s
 XXMFACC         011111 ... -- 00000 ----- 0010110001 -   @X_a
 XXMTACC         011111 ... -- 00001 ----- 0010110001 -   @X_a
 XXSETACCZ       011111 ... -- 00011 ----- 0010110001 -   @X_a
+
+## VSX GER instruction
+
+XVI4GER8        111011 ... -- ..... ..... 00100011 ..-  @XX3_at xa=%xx_xa
+XVI4GER8PP      111011 ... -- ..... ..... 00100010 ..-  @XX3_at xa=%xx_xa
+XVI8GER4        111011 ... -- ..... ..... 00000011 ..-  @XX3_at xa=%xx_xa
+XVI8GER4PP      111011 ... -- ..... ..... 00000010 ..-  @XX3_at xa=%xx_xa
+XVI16GER2       111011 ... -- ..... ..... 01001011 ..-  @XX3_at xa=%xx_xa
+XVI16GER2PP     111011 ... -- ..... ..... 01101011 ..-  @XX3_at xa=%xx_xa
+XVI8GER4SPP     111011 ... -- ..... ..... 01100011 ..-  @XX3_at xa=%xx_xa
+XVI16GER2S      111011 ... -- ..... ..... 00101011 ..-  @XX3_at xa=%xx_xa
+XVI16GER2SPP    111011 ... -- ..... ..... 00101010 ..-  @XX3_at xa=%xx_xa
diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c
index b9dd15d607..105b626d1b 100644
--- a/target/ppc/int_helper.c
+++ b/target/ppc/int_helper.c
@@ -782,6 +782,136 @@ VCT(uxs, cvtsduw, u32)
 VCT(sxs, cvtsdsw, s32)
 #undef VCT
 
+typedef int64_t do_ger(uint32_t, uint32_t, uint32_t);
+
+static int64_t ger_rank8(uint32_t a, uint32_t b, uint32_t mask)
+{
+    int64_t psum = 0;
+    for (int i = 0; i < 8; i++, mask >>= 1) {
+        if (mask & 1) {
+            psum += sextract32(a, 4 * i, 4) * sextract32(b, 4 * i, 4);
+        }
+    }
+    return psum;
+}
+
+static int64_t ger_rank4(uint32_t a, uint32_t b, uint32_t mask)
+{
+    int64_t psum = 0;
+    for (int i = 0; i < 4; i++, mask >>= 1) {
+        if (mask & 1) {
+            psum += sextract32(a, 8 * i, 8) * (int64_t)extract32(b, 8 * i, 8);
+        }
+    }
+    return psum;
+}
+
+static int64_t ger_rank2(uint32_t a, uint32_t b, uint32_t mask)
+{
+    int64_t psum = 0;
+    for (int i = 0; i < 2; i++, mask >>= 1) {
+        if (mask & 1) {
+            psum += sextract32(a, 16 * i, 16) * sextract32(b, 16 * i, 16);
+        }
+    }
+    return psum;
+}
+
+static void xviger(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b, ppc_acc_t  *at,
+                   uint32_t mask, bool sat, bool acc, do_ger ger)
+{
+    uint8_t pmsk = FIELD_EX32(mask, GER_MSK, PMSK),
+            xmsk = FIELD_EX32(mask, GER_MSK, XMSK),
+            ymsk = FIELD_EX32(mask, GER_MSK, YMSK);
+    uint8_t xmsk_bit, ymsk_bit;
+    int64_t psum;
+    int i, j;
+    for (i = 0, xmsk_bit = 1 << 3; i < 4; i++, xmsk_bit >>= 1) {
+        for (j = 0, ymsk_bit = 1 << 3; j < 4; j++, ymsk_bit >>= 1) {
+            if ((xmsk_bit & xmsk) && (ymsk_bit & ymsk)) {
+                psum = ger(a->VsrW(i), b->VsrW(j), pmsk);
+                if (acc) {
+                    psum += at[i].VsrSW(j);
+                }
+                if (sat && psum > INT32_MAX) {
+                    set_vscr_sat(env);
+                    at[i].VsrSW(j) = INT32_MAX;
+                } else if (sat && psum < INT32_MIN) {
+                    set_vscr_sat(env);
+                    at[i].VsrSW(j) = INT32_MIN;
+                } else {
+                    at[i].VsrSW(j) = (int32_t) psum;
+                }
+            } else {
+                at[i].VsrSW(j) = 0;
+            }
+        }
+    }
+}
+
+QEMU_FLATTEN
+void helper_XVI4GER8(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                     ppc_acc_t *at, uint32_t mask)
+{
+    xviger(env, a, b, at, mask, false, false, ger_rank8);
+}
+
+QEMU_FLATTEN
+void helper_XVI4GER8PP(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                       ppc_acc_t *at, uint32_t mask)
+{
+    xviger(env, a, b, at, mask, false, true, ger_rank8);
+}
+
+QEMU_FLATTEN
+void helper_XVI8GER4(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                     ppc_acc_t *at, uint32_t mask)
+{
+    xviger(env, a, b, at, mask, false, false, ger_rank4);
+}
+
+QEMU_FLATTEN
+void helper_XVI8GER4PP(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                       ppc_acc_t *at, uint32_t mask)
+{
+    xviger(env, a, b, at, mask, false, true, ger_rank4);
+}
+
+QEMU_FLATTEN
+void helper_XVI8GER4SPP(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                        ppc_acc_t *at, uint32_t mask)
+{
+    xviger(env, a, b, at, mask, true, true, ger_rank4);
+}
+
+QEMU_FLATTEN
+void helper_XVI16GER2(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                      ppc_acc_t *at, uint32_t mask)
+{
+    xviger(env, a, b, at, mask, false, false, ger_rank2);
+}
+
+QEMU_FLATTEN
+void helper_XVI16GER2S(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                       ppc_acc_t *at, uint32_t mask)
+{
+    xviger(env, a, b, at, mask, true, false, ger_rank2);
+}
+
+QEMU_FLATTEN
+void helper_XVI16GER2PP(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                        ppc_acc_t *at, uint32_t mask)
+{
+    xviger(env, a, b, at, mask, false, true, ger_rank2);
+}
+
+QEMU_FLATTEN
+void helper_XVI16GER2SPP(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                         ppc_acc_t *at, uint32_t mask)
+{
+    xviger(env, a, b, at, mask, true, true, ger_rank2);
+}
+
 target_ulong helper_vclzlsbb(ppc_avr_t *r)
 {
     target_ulong count = 0;
diff --git a/target/ppc/internal.h b/target/ppc/internal.h
index 8094e0b033..2add128cd1 100644
--- a/target/ppc/internal.h
+++ b/target/ppc/internal.h
@@ -18,6 +18,8 @@
 #ifndef PPC_INTERNAL_H
 #define PPC_INTERNAL_H
 
+#include "hw/registerfields.h"
+
 #define FUNC_MASK(name, ret_type, size, max_val)                  \
 static inline ret_type name(uint##size##_t start,                 \
                               uint##size##_t end)                 \
@@ -291,4 +293,17 @@ G_NORETURN void ppc_cpu_do_unaligned_access(CPUState *cs, vaddr addr,
                                             uintptr_t retaddr);
 #endif
 
+FIELD(GER_MSK, XMSK, 0, 4)
+FIELD(GER_MSK, YMSK, 4, 4)
+FIELD(GER_MSK, PMSK, 8, 8)
+
+static inline int ger_pack_masks(int pmsk, int ymsk, int xmsk)
+{
+    int msk = 0;
+    msk = FIELD_DP32(msk, GER_MSK, XMSK, xmsk);
+    msk = FIELD_DP32(msk, GER_MSK, YMSK, ymsk);
+    msk = FIELD_DP32(msk, GER_MSK, PMSK, pmsk);
+    return msk;
+}
+
 #endif /* PPC_INTERNAL_H */
diff --git a/target/ppc/translate/vsx-impl.c.inc b/target/ppc/translate/vsx-impl.c.inc
index 235be360e2..6026b203e0 100644
--- a/target/ppc/translate/vsx-impl.c.inc
+++ b/target/ppc/translate/vsx-impl.c.inc
@@ -17,6 +17,13 @@ static inline TCGv_ptr gen_vsr_ptr(int reg)
     return r;
 }
 
+static inline TCGv_ptr gen_acc_ptr(int reg)
+{
+    TCGv_ptr r = tcg_temp_new_ptr();
+    tcg_gen_addi_ptr(r, cpu_env, acc_full_offset(reg));
+    return r;
+}
+
 #define VSX_LOAD_SCALAR(name, operation)                      \
 static void gen_##name(DisasContext *ctx)                     \
 {                                                             \
@@ -2847,6 +2854,40 @@ static bool trans_XXSETACCZ(DisasContext *ctx, arg_X_a *a)
     return true;
 }
 
+static bool do_ger(DisasContext *ctx, arg_MMIRR_XX3 *a,
+    void (*helper)(TCGv_env, TCGv_ptr, TCGv_ptr, TCGv_ptr, TCGv_i32))
+{
+    uint32_t mask;
+    TCGv_ptr xt, xa, xb;
+    REQUIRE_INSNS_FLAGS2(ctx, ISA310);
+    REQUIRE_VSX(ctx);
+    if (unlikely((a->xa / 4 == a->xt) || (a->xb / 4 == a->xt))) {
+        gen_invalid(ctx);
+        return true;
+    }
+
+    xt = gen_acc_ptr(a->xt);
+    xa = gen_vsr_ptr(a->xa);
+    xb = gen_vsr_ptr(a->xb);
+
+    mask = ger_pack_masks(a->pmsk, a->ymsk, a->xmsk);
+    helper(cpu_env, xa, xb, xt, tcg_constant_i32(mask));
+    tcg_temp_free_ptr(xt);
+    tcg_temp_free_ptr(xa);
+    tcg_temp_free_ptr(xb);
+    return true;
+}
+
+TRANS(XVI4GER8, do_ger, gen_helper_XVI4GER8)
+TRANS(XVI4GER8PP, do_ger,  gen_helper_XVI4GER8PP)
+TRANS(XVI8GER4, do_ger, gen_helper_XVI8GER4)
+TRANS(XVI8GER4PP, do_ger,  gen_helper_XVI8GER4PP)
+TRANS(XVI8GER4SPP, do_ger, gen_helper_XVI8GER4SPP)
+TRANS(XVI16GER2, do_ger, gen_helper_XVI16GER2)
+TRANS(XVI16GER2PP, do_ger, gen_helper_XVI16GER2PP)
+TRANS(XVI16GER2S, do_ger, gen_helper_XVI16GER2S)
+TRANS(XVI16GER2SPP, do_ger, gen_helper_XVI16GER2SPP)
+
 #undef GEN_XX2FORM
 #undef GEN_XX3FORM
 #undef GEN_XX2IFORM

From 6d525ca972085fb82fe2dec04fdf257318f9e5a5 Mon Sep 17 00:00:00 2001
From: "Lucas Mateus Castro (alqotel)" <lucas.araujo@eldorado.org.br>
Date: Tue, 24 May 2022 11:05:32 -0300
Subject: [PATCH 560/935] target/ppc: Implemented pmxvi*ger* instructions

Implement the following PowerISA v3.1 instructions:
pmxvi4ger8:     Prefixed Masked VSX Vector 8-bit Signed/Unsigned Integer
GER (rank-4 update)
pmxvi4ger8pp:   Prefixed Masked VSX Vector 8-bit Signed/Unsigned Integer
GER (rank-4 update) Positive multiply, Positive accumulate
pmxvi8ger4:     Prefixed Masked VSX Vector 4-bit Signed Integer GER
(rank-8 update)
pmxvi8ger4pp:   Prefixed Masked VSX Vector 4-bit Signed Integer GER
(rank-8 update) Positive multiply, Positive accumulate
pmxvi8ger4spp:  Prefixed Masked VSX Vector 8-bit Signed/Unsigned Integer
GER (rank-4 update) with Saturate Positive multiply, Positive accumulate
pmxvi16ger2:    Prefixed Masked VSX Vector 16-bit Signed Integer GER
(rank-2 update)
pmxvi16ger2pp:  Prefixed Masked VSX Vector 16-bit Signed Integer GER
(rank-2 update) Positive multiply, Positive accumulate
pmxvi16ger2s:   Prefixed Masked VSX Vector 16-bit Signed Integer GER
(rank-2 update) with Saturation
pmxvi16ger2spp: Prefixed Masked VSX Vector 16-bit Signed Integer GER
(rank-2 update) with Saturation Positive multiply, Positive accumulate

Signed-off-by: Lucas Mateus Castro (alqotel) <lucas.araujo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220524140537.27451-4-lucas.araujo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/insn64.decode            | 30 +++++++++++++++++++++++++++++
 target/ppc/translate/vsx-impl.c.inc | 10 ++++++++++
 2 files changed, 40 insertions(+)

diff --git a/target/ppc/insn64.decode b/target/ppc/insn64.decode
index 691e8fe6c0..0eed35c8cd 100644
--- a/target/ppc/insn64.decode
+++ b/target/ppc/insn64.decode
@@ -68,6 +68,15 @@
                 ...... ..... ..... ..... ..... .. ....   \
                 &8RR_XX4_uim3 xt=%8rr_xx_xt xa=%8rr_xx_xa xb=%8rr_xx_xb xc=%8rr_xx_xc
 
+# Format MMIRR:XX3
+&MMIRR_XX3      !extern xa xb xt pmsk xmsk ymsk
+%xx3_xa         2:1 16:5
+%xx3_xb         1:1 11:5
+%xx3_at         23:3
+@MMIRR_XX3      ...... .. .... .. . . ........ xmsk:4 ymsk:4  \
+                ...... ... .. ..... ..... ........ ...  \
+                &MMIRR_XX3 xa=%xx3_xa xb=%xx3_xb xt=%xx3_at
+
 ### Fixed-Point Load Instructions
 
 PLBZ            000001 10 0--.-- .................. \
@@ -115,6 +124,27 @@ PSTFS           000001 10 0--.-- .................. \
 PSTFD           000001 10 0--.-- .................. \
                 110110 ..... ..... ................     @PLS_D
 
+## VSX GER instruction
+
+PMXVI4GER8      000001 11 1001 -- - - pmsk:8 ........              \
+                111011 ... -- ..... ..... 00100011 ..-  @MMIRR_XX3
+PMXVI4GER8PP    000001 11 1001 -- - - pmsk:8 ........              \
+                111011 ... -- ..... ..... 00100010 ..-  @MMIRR_XX3
+PMXVI8GER4      000001 11 1001 -- - - pmsk:4 ---- ........         \
+                111011 ... -- ..... ..... 00000011 ..-  @MMIRR_XX3
+PMXVI8GER4PP    000001 11 1001 -- - - pmsk:4 ---- ........         \
+                111011 ... -- ..... ..... 00000010 ..-  @MMIRR_XX3
+PMXVI16GER2     000001 11 1001 -- - - pmsk:2 ------ ........       \
+                111011 ... -- ..... ..... 01001011 ..-  @MMIRR_XX3
+PMXVI16GER2PP   000001 11 1001 -- - - pmsk:2 ------ ........       \
+                111011 ... -- ..... ..... 01101011 ..-  @MMIRR_XX3
+PMXVI8GER4SPP   000001 11 1001 -- - - pmsk:4 ---- ........         \
+                111011 ... -- ..... ..... 01100011 ..-  @MMIRR_XX3
+PMXVI16GER2S    000001 11 1001 -- - - pmsk:2 ------ ........       \
+                111011 ... -- ..... ..... 00101011 ..-  @MMIRR_XX3
+PMXVI16GER2SPP  000001 11 1001 -- - - pmsk:2 ------ ........       \
+                111011 ... -- ..... ..... 00101010 ..-  @MMIRR_XX3
+
 ### Prefixed No-operation Instruction
 
 @PNOP           000001 11 0000-- 000000000000000000     \
diff --git a/target/ppc/translate/vsx-impl.c.inc b/target/ppc/translate/vsx-impl.c.inc
index 6026b203e0..b10eded1da 100644
--- a/target/ppc/translate/vsx-impl.c.inc
+++ b/target/ppc/translate/vsx-impl.c.inc
@@ -2888,6 +2888,16 @@ TRANS(XVI16GER2PP, do_ger, gen_helper_XVI16GER2PP)
 TRANS(XVI16GER2S, do_ger, gen_helper_XVI16GER2S)
 TRANS(XVI16GER2SPP, do_ger, gen_helper_XVI16GER2SPP)
 
+TRANS64(PMXVI4GER8, do_ger, gen_helper_XVI4GER8)
+TRANS64(PMXVI4GER8PP, do_ger, gen_helper_XVI4GER8PP)
+TRANS64(PMXVI8GER4, do_ger, gen_helper_XVI8GER4)
+TRANS64(PMXVI8GER4PP, do_ger, gen_helper_XVI8GER4PP)
+TRANS64(PMXVI8GER4SPP, do_ger, gen_helper_XVI8GER4SPP)
+TRANS64(PMXVI16GER2, do_ger, gen_helper_XVI16GER2)
+TRANS64(PMXVI16GER2PP, do_ger, gen_helper_XVI16GER2PP)
+TRANS64(PMXVI16GER2S, do_ger, gen_helper_XVI16GER2S)
+TRANS64(PMXVI16GER2SPP, do_ger, gen_helper_XVI16GER2SPP)
+
 #undef GEN_XX2FORM
 #undef GEN_XX3FORM
 #undef GEN_XX2IFORM

From c29018cc7395fdb7efd5e21c6f6c4d9cfbcf9a3c Mon Sep 17 00:00:00 2001
From: "Lucas Mateus Castro (alqotel)" <lucas.araujo@eldorado.org.br>
Date: Tue, 24 May 2022 11:05:33 -0300
Subject: [PATCH 561/935] target/ppc: Implemented xvf*ger*

Implement the following PowerISA v3.1 instructions:
xvf32ger:   VSX Vector 32-bit Floating-Point GER (rank-1 update)
xvf32gernn: VSX Vector 32-bit Floating-Point GER (rank-1 update) Negative
multiply, Negative accumulate
xvf32gernp: VSX Vector 32-bit Floating-Point GER (rank-1 update) Negative
multiply, Positive accumulate
xvf32gerpn: VSX Vector 32-bit Floating-Point GER (rank-1 update) Positive
multiply, Negative accumulate
xvf32gerpp: VSX Vector 32-bit Floating-Point GER (rank-1 update) Positive
multiply, Positive accumulate
xvf64ger:   VSX Vector 64-bit Floating-Point GER (rank-1 update)
xvf64gernn: VSX Vector 64-bit Floating-Point GER (rank-1 update) Negative
multiply, Negative accumulate
xvf64gernp: VSX Vector 64-bit Floating-Point GER (rank-1 update) Negative
multiply, Positive accumulate
xvf64gerpn: VSX Vector 64-bit Floating-Point GER (rank-1 update) Positive
multiply, Negative accumulate
xvf64gerpp: VSX Vector 64-bit Floating-Point GER (rank-1 update) Positive
multiply, Positive accumulate

Signed-off-by: Lucas Mateus Castro (alqotel) <lucas.araujo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220524140537.27451-5-lucas.araujo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h                    |   4 +
 target/ppc/fpu_helper.c             | 194 +++++++++++++++++++++++++++-
 target/ppc/helper.h                 |  10 ++
 target/ppc/insn32.decode            |  13 ++
 target/ppc/translate/vsx-impl.c.inc |  12 ++
 5 files changed, 231 insertions(+), 2 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index dff3ca8222..40c779f246 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -2643,6 +2643,8 @@ static inline bool lsw_reg_in_range(int start, int nregs, int rx)
 #define VsrSW(i) s32[i]
 #define VsrD(i) u64[i]
 #define VsrSD(i) s64[i]
+#define VsrSF(i) f32[i]
+#define VsrDF(i) f64[i]
 #else
 #define VsrB(i) u8[15 - (i)]
 #define VsrSB(i) s8[15 - (i)]
@@ -2652,6 +2654,8 @@ static inline bool lsw_reg_in_range(int start, int nregs, int rx)
 #define VsrSW(i) s32[3 - (i)]
 #define VsrD(i) u64[1 - (i)]
 #define VsrSD(i) s64[1 - (i)]
+#define VsrSF(i) f32[3 - (i)]
+#define VsrDF(i) f64[1 - (i)]
 #endif
 
 static inline int vsr64_offset(int i, bool high)
diff --git a/target/ppc/fpu_helper.c b/target/ppc/fpu_helper.c
index 9489e06504..712c71162c 100644
--- a/target/ppc/fpu_helper.c
+++ b/target/ppc/fpu_helper.c
@@ -414,7 +414,7 @@ void helper_store_fpscr(CPUPPCState *env, uint64_t val, uint32_t nibbles)
     ppc_store_fpscr(env, val);
 }
 
-void helper_fpscr_check_status(CPUPPCState *env)
+static void do_fpscr_check_status(CPUPPCState *env, uintptr_t raddr)
 {
     CPUState *cs = env_cpu(env);
     target_ulong fpscr = env->fpscr;
@@ -455,13 +455,19 @@ void helper_fpscr_check_status(CPUPPCState *env)
     }
     cs->exception_index = POWERPC_EXCP_PROGRAM;
     env->error_code = error | POWERPC_EXCP_FP;
+    env->fpscr |= error ? FP_FEX : 0;
     /* Deferred floating-point exception after target FPSCR update */
     if (fp_exceptions_enabled(env)) {
         raise_exception_err_ra(env, cs->exception_index,
-                               env->error_code, GETPC());
+                               env->error_code, raddr);
     }
 }
 
+void helper_fpscr_check_status(CPUPPCState *env)
+{
+    do_fpscr_check_status(env, GETPC());
+}
+
 static void do_float_check_status(CPUPPCState *env, bool change_fi,
                                   uintptr_t raddr)
 {
@@ -3468,3 +3474,187 @@ void helper_xssubqp(CPUPPCState *env, uint32_t opcode,
     *xt = t;
     do_float_check_status(env, true, GETPC());
 }
+
+static inline void vsxger_excp(CPUPPCState *env, uintptr_t retaddr)
+{
+    /*
+     * XV*GER instructions execute and set the FPSCR as if exceptions
+     * are disabled and only at the end throw an exception
+     */
+    target_ulong enable;
+    enable = env->fpscr & (FP_ENABLES | FP_FI | FP_FR);
+    env->fpscr &= ~(FP_ENABLES | FP_FI | FP_FR);
+    int status = get_float_exception_flags(&env->fp_status);
+    if (unlikely(status & float_flag_invalid)) {
+        if (status & float_flag_invalid_snan) {
+            float_invalid_op_vxsnan(env, 0);
+        }
+        if (status & float_flag_invalid_imz) {
+            float_invalid_op_vximz(env, false, 0);
+        }
+        if (status & float_flag_invalid_isi) {
+            float_invalid_op_vxisi(env, false, 0);
+        }
+    }
+    do_float_check_status(env, false, retaddr);
+    env->fpscr |= enable;
+    do_fpscr_check_status(env, retaddr);
+}
+
+typedef void vsxger_zero(ppc_vsr_t *at, int, int);
+
+typedef void vsxger_muladd_f(ppc_vsr_t *, ppc_vsr_t *, ppc_vsr_t *, int, int,
+                             int flags, float_status *s);
+
+static void vsxger_muladd32(ppc_vsr_t *at, ppc_vsr_t *a, ppc_vsr_t *b, int i,
+                            int j, int flags, float_status *s)
+{
+    at[i].VsrSF(j) = float32_muladd(a->VsrSF(i), b->VsrSF(j),
+                                    at[i].VsrSF(j), flags, s);
+}
+
+static void vsxger_mul32(ppc_vsr_t *at, ppc_vsr_t *a, ppc_vsr_t *b, int i,
+                         int j, int flags, float_status *s)
+{
+    at[i].VsrSF(j) = float32_mul(a->VsrSF(i), b->VsrSF(j), s);
+}
+
+static void vsxger_zero32(ppc_vsr_t *at, int i, int j)
+{
+    at[i].VsrSF(j) = float32_zero;
+}
+
+static void vsxger_muladd64(ppc_vsr_t *at, ppc_vsr_t *a, ppc_vsr_t *b, int i,
+                            int j, int flags, float_status *s)
+{
+    if (j >= 2) {
+        j -= 2;
+        at[i].VsrDF(j) = float64_muladd(a[i / 2].VsrDF(i % 2), b->VsrDF(j),
+                                        at[i].VsrDF(j), flags, s);
+    }
+}
+
+static void vsxger_mul64(ppc_vsr_t *at, ppc_vsr_t *a, ppc_vsr_t *b, int i,
+                         int j, int flags, float_status *s)
+{
+    if (j >= 2) {
+        j -= 2;
+        at[i].VsrDF(j) = float64_mul(a[i / 2].VsrDF(i % 2), b->VsrDF(j), s);
+    }
+}
+
+static void vsxger_zero64(ppc_vsr_t *at, int i, int j)
+{
+    if (j >= 2) {
+        j -= 2;
+        at[i].VsrDF(j) = float64_zero;
+    }
+}
+
+static void vsxger(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                   ppc_acc_t  *at, uint32_t mask, bool acc, bool neg_mul,
+                   bool neg_acc, vsxger_muladd_f mul, vsxger_muladd_f muladd,
+                   vsxger_zero zero)
+{
+    int i, j, xmsk_bit, ymsk_bit, op_flags;
+    uint8_t xmsk = mask & 0x0F;
+    uint8_t ymsk = (mask >> 4) & 0x0F;
+    float_status *excp_ptr = &env->fp_status;
+    op_flags = (neg_acc ^ neg_mul) ? float_muladd_negate_c : 0;
+    op_flags |= (neg_mul) ? float_muladd_negate_result : 0;
+    helper_reset_fpstatus(env);
+    for (i = 0, xmsk_bit = 1 << 3; i < 4; i++, xmsk_bit >>= 1) {
+        for (j = 0, ymsk_bit = 1 << 3; j < 4; j++, ymsk_bit >>= 1) {
+            if ((xmsk_bit & xmsk) && (ymsk_bit & ymsk)) {
+                if (acc) {
+                    muladd(at, a, b, i, j, op_flags, excp_ptr);
+                } else {
+                    mul(at, a, b, i, j, op_flags, excp_ptr);
+                }
+            } else {
+                zero(at, i, j);
+            }
+        }
+    }
+    vsxger_excp(env, GETPC());
+}
+
+QEMU_FLATTEN
+void helper_XVF32GER(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                     ppc_acc_t *at, uint32_t mask)
+{
+    vsxger(env, a, b, at, mask, false, false, false, vsxger_mul32,
+           vsxger_muladd32, vsxger_zero32);
+}
+
+QEMU_FLATTEN
+void helper_XVF32GERPP(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                       ppc_acc_t *at, uint32_t mask)
+{
+    vsxger(env, a, b, at, mask, true, false, false, vsxger_mul32,
+           vsxger_muladd32, vsxger_zero32);
+}
+
+QEMU_FLATTEN
+void helper_XVF32GERPN(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                       ppc_acc_t *at, uint32_t mask)
+{
+    vsxger(env, a, b, at, mask, true, false, true, vsxger_mul32,
+           vsxger_muladd32, vsxger_zero32);
+}
+
+QEMU_FLATTEN
+void helper_XVF32GERNP(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                       ppc_acc_t *at, uint32_t mask)
+{
+    vsxger(env, a, b, at, mask, true, true, false, vsxger_mul32,
+           vsxger_muladd32, vsxger_zero32);
+}
+
+QEMU_FLATTEN
+void helper_XVF32GERNN(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                       ppc_acc_t *at, uint32_t mask)
+{
+    vsxger(env, a, b, at, mask, true, true, true, vsxger_mul32,
+           vsxger_muladd32, vsxger_zero32);
+}
+
+QEMU_FLATTEN
+void helper_XVF64GER(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                     ppc_acc_t *at, uint32_t mask)
+{
+    vsxger(env, a, b, at, mask, false, false, false, vsxger_mul64,
+           vsxger_muladd64, vsxger_zero64);
+}
+
+QEMU_FLATTEN
+void helper_XVF64GERPP(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                       ppc_acc_t *at, uint32_t mask)
+{
+    vsxger(env, a, b, at, mask, true, false, false, vsxger_mul64,
+           vsxger_muladd64, vsxger_zero64);
+}
+
+QEMU_FLATTEN
+void helper_XVF64GERPN(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                       ppc_acc_t *at, uint32_t mask)
+{
+    vsxger(env, a, b, at, mask, true, false, true, vsxger_mul64,
+           vsxger_muladd64, vsxger_zero64);
+}
+
+QEMU_FLATTEN
+void helper_XVF64GERNP(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                       ppc_acc_t *at, uint32_t mask)
+{
+    vsxger(env, a, b, at, mask, true, true, false, vsxger_mul64,
+           vsxger_muladd64, vsxger_zero64);
+}
+
+QEMU_FLATTEN
+void helper_XVF64GERNN(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                       ppc_acc_t *at, uint32_t mask)
+{
+    vsxger(env, a, b, at, mask, true, true, true, vsxger_mul64,
+           vsxger_muladd64, vsxger_zero64);
+}
diff --git a/target/ppc/helper.h b/target/ppc/helper.h
index 1666797edf..f38cdbe1d8 100644
--- a/target/ppc/helper.h
+++ b/target/ppc/helper.h
@@ -550,6 +550,16 @@ DEF_HELPER_5(XVI16GER2, void, env, vsr, vsr, acc, i32)
 DEF_HELPER_5(XVI16GER2S, void, env, vsr, vsr, acc, i32)
 DEF_HELPER_5(XVI16GER2PP, void, env, vsr, vsr, acc, i32)
 DEF_HELPER_5(XVI16GER2SPP, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVF32GER, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVF32GERPP, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVF32GERPN, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVF32GERNP, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVF32GERNN, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVF64GER, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVF64GERPP, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVF64GERPN, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVF64GERNP, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVF64GERNN, void, env, vsr, vsr, acc, i32)
 
 DEF_HELPER_2(efscfsi, i32, env, i32)
 DEF_HELPER_2(efscfui, i32, env, i32)
diff --git a/target/ppc/insn32.decode b/target/ppc/insn32.decode
index 0e189fe2da..6b644155ec 100644
--- a/target/ppc/insn32.decode
+++ b/target/ppc/insn32.decode
@@ -178,6 +178,7 @@
 # 32 bit GER instructions have all mask bits considered 1
 &MMIRR_XX3      xa xb xt pmsk xmsk ymsk
 %xx_at          23:3
+%xx_xa_pair     2:1 17:4 !function=times_2
 @XX3_at         ...... ... .. ..... ..... ........ ...          &MMIRR_XX3 xt=%xx_at xb=%xx_xb \
                                                                 pmsk=255 xmsk=15 ymsk=15
 
@@ -761,3 +762,15 @@ XVI16GER2PP     111011 ... -- ..... ..... 01101011 ..-  @XX3_at xa=%xx_xa
 XVI8GER4SPP     111011 ... -- ..... ..... 01100011 ..-  @XX3_at xa=%xx_xa
 XVI16GER2S      111011 ... -- ..... ..... 00101011 ..-  @XX3_at xa=%xx_xa
 XVI16GER2SPP    111011 ... -- ..... ..... 00101010 ..-  @XX3_at xa=%xx_xa
+
+XVF32GER        111011 ... -- ..... ..... 00011011 ..-  @XX3_at xa=%xx_xa
+XVF32GERPP      111011 ... -- ..... ..... 00011010 ..-  @XX3_at xa=%xx_xa
+XVF32GERPN      111011 ... -- ..... ..... 10011010 ..-  @XX3_at xa=%xx_xa
+XVF32GERNP      111011 ... -- ..... ..... 01011010 ..-  @XX3_at xa=%xx_xa
+XVF32GERNN      111011 ... -- ..... ..... 11011010 ..-  @XX3_at xa=%xx_xa
+
+XVF64GER        111011 ... -- .... 0 ..... 00111011 ..-  @XX3_at xa=%xx_xa_pair
+XVF64GERPP      111011 ... -- .... 0 ..... 00111010 ..-  @XX3_at xa=%xx_xa_pair
+XVF64GERPN      111011 ... -- .... 0 ..... 10111010 ..-  @XX3_at xa=%xx_xa_pair
+XVF64GERNP      111011 ... -- .... 0 ..... 01111010 ..-  @XX3_at xa=%xx_xa_pair
+XVF64GERNN      111011 ... -- .... 0 ..... 11111010 ..-  @XX3_at xa=%xx_xa_pair
diff --git a/target/ppc/translate/vsx-impl.c.inc b/target/ppc/translate/vsx-impl.c.inc
index b10eded1da..cc754ab175 100644
--- a/target/ppc/translate/vsx-impl.c.inc
+++ b/target/ppc/translate/vsx-impl.c.inc
@@ -2898,6 +2898,18 @@ TRANS64(PMXVI16GER2PP, do_ger, gen_helper_XVI16GER2PP)
 TRANS64(PMXVI16GER2S, do_ger, gen_helper_XVI16GER2S)
 TRANS64(PMXVI16GER2SPP, do_ger, gen_helper_XVI16GER2SPP)
 
+TRANS(XVF32GER, do_ger, gen_helper_XVF32GER)
+TRANS(XVF32GERPP, do_ger, gen_helper_XVF32GERPP)
+TRANS(XVF32GERPN, do_ger, gen_helper_XVF32GERPN)
+TRANS(XVF32GERNP, do_ger, gen_helper_XVF32GERNP)
+TRANS(XVF32GERNN, do_ger, gen_helper_XVF32GERNN)
+
+TRANS(XVF64GER, do_ger, gen_helper_XVF64GER)
+TRANS(XVF64GERPP, do_ger, gen_helper_XVF64GERPP)
+TRANS(XVF64GERPN, do_ger, gen_helper_XVF64GERPN)
+TRANS(XVF64GERNP, do_ger, gen_helper_XVF64GERNP)
+TRANS(XVF64GERNN, do_ger, gen_helper_XVF64GERNN)
+
 #undef GEN_XX2FORM
 #undef GEN_XX3FORM
 #undef GEN_XX2IFORM

From 2d9cba74ef72232189ac5611968c1b9045a3e1a7 Mon Sep 17 00:00:00 2001
From: "Lucas Mateus Castro (alqotel)" <lucas.araujo@eldorado.org.br>
Date: Tue, 24 May 2022 11:05:34 -0300
Subject: [PATCH 562/935] target/ppc: Implemented xvf16ger*

Implement the following PowerISA v3.1 instructions:
xvf16ger2:   VSX Vector 16-bit Floating-Point GER (rank-2 update)
xvf16ger2nn: VSX Vector 16-bit Floating-Point GER (rank-2 update) Negative
multiply, Negative accumulate
xvf16ger2np: VSX Vector 16-bit Floating-Point GER (rank-2 update) Negative
multiply, Positive accumulate
xvf16ger2pn: VSX Vector 16-bit Floating-Point GER (rank-2 update) Positive
multiply, Negative accumulate
xvf16ger2pp: VSX Vector 16-bit Floating-Point GER (rank-2 update) Positive
multiply, Positive accumulate

Signed-off-by: Lucas Mateus Castro (alqotel) <lucas.araujo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220524140537.27451-6-lucas.araujo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/cpu.h                    |  3 +
 target/ppc/fpu_helper.c             | 95 +++++++++++++++++++++++++++++
 target/ppc/helper.h                 |  5 ++
 target/ppc/insn32.decode            |  6 ++
 target/ppc/translate/vsx-impl.c.inc |  6 ++
 5 files changed, 115 insertions(+)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 40c779f246..6d78078f37 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -227,6 +227,7 @@ typedef union _ppc_vsr_t {
     int16_t s16[8];
     int32_t s32[4];
     int64_t s64[2];
+    float16 f16[8];
     float32 f32[4];
     float64 f64[2];
     float128 f128;
@@ -2643,6 +2644,7 @@ static inline bool lsw_reg_in_range(int start, int nregs, int rx)
 #define VsrSW(i) s32[i]
 #define VsrD(i) u64[i]
 #define VsrSD(i) s64[i]
+#define VsrHF(i) f16[i]
 #define VsrSF(i) f32[i]
 #define VsrDF(i) f64[i]
 #else
@@ -2654,6 +2656,7 @@ static inline bool lsw_reg_in_range(int start, int nregs, int rx)
 #define VsrSW(i) s32[3 - (i)]
 #define VsrD(i) u64[1 - (i)]
 #define VsrSD(i) s64[1 - (i)]
+#define VsrHF(i) f16[7 - (i)]
 #define VsrSF(i) f32[3 - (i)]
 #define VsrDF(i) f64[1 - (i)]
 #endif
diff --git a/target/ppc/fpu_helper.c b/target/ppc/fpu_helper.c
index 712c71162c..a9b2ef370f 100644
--- a/target/ppc/fpu_helper.c
+++ b/target/ppc/fpu_helper.c
@@ -36,6 +36,15 @@ static inline float128 float128_snan_to_qnan(float128 x)
 #define float32_snan_to_qnan(x) ((x) | 0x00400000)
 #define float16_snan_to_qnan(x) ((x) | 0x0200)
 
+static inline float32 bfp32_neg(float32 a)
+{
+    if (unlikely(float32_is_any_nan(a))) {
+        return a;
+    } else {
+        return float32_chs(a);
+    }
+}
+
 static inline bool fp_exceptions_enabled(CPUPPCState *env)
 {
 #ifdef CONFIG_USER_ONLY
@@ -3501,6 +3510,57 @@ static inline void vsxger_excp(CPUPPCState *env, uintptr_t retaddr)
     do_fpscr_check_status(env, retaddr);
 }
 
+typedef float64 extract_f16(float16, float_status *);
+
+static float64 extract_hf16(float16 in, float_status *fp_status)
+{
+    return float16_to_float64(in, true, fp_status);
+}
+
+static void vsxger16(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                     ppc_acc_t  *at, uint32_t mask, bool acc,
+                     bool neg_mul, bool neg_acc, extract_f16 extract)
+{
+    float32 r, aux_acc;
+    float64 psum, va, vb, vc, vd;
+    int i, j, xmsk_bit, ymsk_bit;
+    uint8_t pmsk = FIELD_EX32(mask, GER_MSK, PMSK),
+            xmsk = FIELD_EX32(mask, GER_MSK, XMSK),
+            ymsk = FIELD_EX32(mask, GER_MSK, YMSK);
+    float_status *excp_ptr = &env->fp_status;
+    for (i = 0, xmsk_bit = 1 << 3; i < 4; i++, xmsk_bit >>= 1) {
+        for (j = 0, ymsk_bit = 1 << 3; j < 4; j++, ymsk_bit >>= 1) {
+            if ((xmsk_bit & xmsk) && (ymsk_bit & ymsk)) {
+                va = !(pmsk & 2) ? float64_zero :
+                                   extract(a->VsrHF(2 * i), excp_ptr);
+                vb = !(pmsk & 2) ? float64_zero :
+                                   extract(b->VsrHF(2 * j), excp_ptr);
+                vc = !(pmsk & 1) ? float64_zero :
+                                   extract(a->VsrHF(2 * i + 1), excp_ptr);
+                vd = !(pmsk & 1) ? float64_zero :
+                                   extract(b->VsrHF(2 * j + 1), excp_ptr);
+                psum = float64_mul(va, vb, excp_ptr);
+                psum = float64r32_muladd(vc, vd, psum, 0, excp_ptr);
+                r = float64_to_float32(psum, excp_ptr);
+                if (acc) {
+                    aux_acc = at[i].VsrSF(j);
+                    if (neg_mul) {
+                        r = bfp32_neg(r);
+                    }
+                    if (neg_acc) {
+                        aux_acc = bfp32_neg(aux_acc);
+                    }
+                    r = float32_add(r, aux_acc, excp_ptr);
+                }
+                at[i].VsrSF(j) = r;
+            } else {
+                at[i].VsrSF(j) = float32_zero;
+            }
+        }
+    }
+    vsxger_excp(env, GETPC());
+}
+
 typedef void vsxger_zero(ppc_vsr_t *at, int, int);
 
 typedef void vsxger_muladd_f(ppc_vsr_t *, ppc_vsr_t *, ppc_vsr_t *, int, int,
@@ -3579,6 +3639,41 @@ static void vsxger(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
     vsxger_excp(env, GETPC());
 }
 
+QEMU_FLATTEN
+void helper_XVF16GER2(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                     ppc_acc_t *at, uint32_t mask)
+{
+    vsxger16(env, a, b, at, mask, false, false, false, extract_hf16);
+}
+
+QEMU_FLATTEN
+void helper_XVF16GER2PP(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                        ppc_acc_t *at, uint32_t mask)
+{
+    vsxger16(env, a, b, at, mask, true, false, false, extract_hf16);
+}
+
+QEMU_FLATTEN
+void helper_XVF16GER2PN(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                        ppc_acc_t *at, uint32_t mask)
+{
+    vsxger16(env, a, b, at, mask, true, false, true, extract_hf16);
+}
+
+QEMU_FLATTEN
+void helper_XVF16GER2NP(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                        ppc_acc_t *at, uint32_t mask)
+{
+    vsxger16(env, a, b, at, mask, true, true, false, extract_hf16);
+}
+
+QEMU_FLATTEN
+void helper_XVF16GER2NN(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                        ppc_acc_t *at, uint32_t mask)
+{
+    vsxger16(env, a, b, at, mask, true, true, true, extract_hf16);
+}
+
 QEMU_FLATTEN
 void helper_XVF32GER(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
                      ppc_acc_t *at, uint32_t mask)
diff --git a/target/ppc/helper.h b/target/ppc/helper.h
index f38cdbe1d8..4070c0891c 100644
--- a/target/ppc/helper.h
+++ b/target/ppc/helper.h
@@ -550,6 +550,11 @@ DEF_HELPER_5(XVI16GER2, void, env, vsr, vsr, acc, i32)
 DEF_HELPER_5(XVI16GER2S, void, env, vsr, vsr, acc, i32)
 DEF_HELPER_5(XVI16GER2PP, void, env, vsr, vsr, acc, i32)
 DEF_HELPER_5(XVI16GER2SPP, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVF16GER2, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVF16GER2PP, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVF16GER2PN, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVF16GER2NP, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVF16GER2NN, void, env, vsr, vsr, acc, i32)
 DEF_HELPER_5(XVF32GER, void, env, vsr, vsr, acc, i32)
 DEF_HELPER_5(XVF32GERPP, void, env, vsr, vsr, acc, i32)
 DEF_HELPER_5(XVF32GERPN, void, env, vsr, vsr, acc, i32)
diff --git a/target/ppc/insn32.decode b/target/ppc/insn32.decode
index 6b644155ec..b8e317159c 100644
--- a/target/ppc/insn32.decode
+++ b/target/ppc/insn32.decode
@@ -763,6 +763,12 @@ XVI8GER4SPP     111011 ... -- ..... ..... 01100011 ..-  @XX3_at xa=%xx_xa
 XVI16GER2S      111011 ... -- ..... ..... 00101011 ..-  @XX3_at xa=%xx_xa
 XVI16GER2SPP    111011 ... -- ..... ..... 00101010 ..-  @XX3_at xa=%xx_xa
 
+XVF16GER2       111011 ... -- ..... ..... 00010011 ..-  @XX3_at xa=%xx_xa
+XVF16GER2PP     111011 ... -- ..... ..... 00010010 ..-  @XX3_at xa=%xx_xa
+XVF16GER2PN     111011 ... -- ..... ..... 10010010 ..-  @XX3_at xa=%xx_xa
+XVF16GER2NP     111011 ... -- ..... ..... 01010010 ..-  @XX3_at xa=%xx_xa
+XVF16GER2NN     111011 ... -- ..... ..... 11010010 ..-  @XX3_at xa=%xx_xa
+
 XVF32GER        111011 ... -- ..... ..... 00011011 ..-  @XX3_at xa=%xx_xa
 XVF32GERPP      111011 ... -- ..... ..... 00011010 ..-  @XX3_at xa=%xx_xa
 XVF32GERPN      111011 ... -- ..... ..... 10011010 ..-  @XX3_at xa=%xx_xa
diff --git a/target/ppc/translate/vsx-impl.c.inc b/target/ppc/translate/vsx-impl.c.inc
index cc754ab175..01978a585a 100644
--- a/target/ppc/translate/vsx-impl.c.inc
+++ b/target/ppc/translate/vsx-impl.c.inc
@@ -2898,6 +2898,12 @@ TRANS64(PMXVI16GER2PP, do_ger, gen_helper_XVI16GER2PP)
 TRANS64(PMXVI16GER2S, do_ger, gen_helper_XVI16GER2S)
 TRANS64(PMXVI16GER2SPP, do_ger, gen_helper_XVI16GER2SPP)
 
+TRANS(XVF16GER2, do_ger, gen_helper_XVF16GER2)
+TRANS(XVF16GER2PP, do_ger, gen_helper_XVF16GER2PP)
+TRANS(XVF16GER2PN, do_ger, gen_helper_XVF16GER2PN)
+TRANS(XVF16GER2NP, do_ger, gen_helper_XVF16GER2NP)
+TRANS(XVF16GER2NN, do_ger, gen_helper_XVF16GER2NN)
+
 TRANS(XVF32GER, do_ger, gen_helper_XVF32GER)
 TRANS(XVF32GERPP, do_ger, gen_helper_XVF32GERPP)
 TRANS(XVF32GERPN, do_ger, gen_helper_XVF32GERPN)

From 6f642338dc929a513ba5ba8f07a23894ba235bd0 Mon Sep 17 00:00:00 2001
From: "Lucas Mateus Castro (alqotel)" <lucas.araujo@eldorado.org.br>
Date: Tue, 24 May 2022 11:05:35 -0300
Subject: [PATCH 563/935] target/ppc: Implemented pmxvf*ger*

Implement the following PowerISA v3.1 instructions:
pmxvf16ger2:   Prefixed Masked VSX Vector 16-bit Floating-Point GER
(rank-2 update)
pmxvf16ger2nn: Prefixed Masked VSX Vector 16-bit Floating-Point GER
(rank-2 update) Negative multiply, Negative accumulate
pmxvf16ger2np: Prefixed Masked VSX Vector 16-bit Floating-Point GER
(rank-2 update) Negative multiply, Positive accumulate
pmxvf16ger2pn: Prefixed Masked VSX Vector 16-bit Floating-Point GER
(rank-2 update) Positive multiply, Negative accumulate
pmxvf16ger2pp: Prefixed Masked VSX Vector 16-bit Floating-Point GER
(rank-2 update) Positive multiply, Positive accumulate
pmxvf32ger:    Prefixed Masked VSX Vector 32-bit Floating-Point GER
(rank-1 update)
pmxvf32gernn:  Prefixed Masked VSX Vector 32-bit Floating-Point GER
(rank-1 update) Negative multiply, Negative accumulate
pmxvf32gernp:  Prefixed Masked VSX Vector 32-bit Floating-Point GER
(rank-1 update) Negative multiply, Positive accumulate
pmxvf32gerpn:  Prefixed Masked VSX Vector 32-bit Floating-Point GER
(rank-1 update) Positive multiply, Negative accumulate
pmxvf32gerpp:  Prefixed Masked VSX Vector 32-bit Floating-Point GER
(rank-1 update) Positive multiply, Positive accumulate
pmxvf64ger:    Prefixed Masked VSX Vector 64-bit Floating-Point GER
(rank-1 update)
pmxvf64gernn:  Prefixed Masked VSX Vector 64-bit Floating-Point GER
(rank-1 update) Negative multiply, Negative accumulate
pmxvf64gernp:  Prefixed Masked VSX Vector 64-bit Floating-Point GER
(rank-1 update) Negative multiply, Positive accumulate
pmxvf64gerpn:  Prefixed Masked VSX Vector 64-bit Floating-Point GER
(rank-1 update) Positive multiply, Negative accumulate
pmxvf64gerpp:  Prefixed Masked VSX Vector 64-bit Floating-Point GER
(rank-1 update) Positive multiply, Positive accumulate

Signed-off-by: Lucas Mateus Castro (alqotel) <lucas.araujo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220524140537.27451-7-lucas.araujo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/insn64.decode            | 38 +++++++++++++++++++++++++++++
 target/ppc/translate/vsx-impl.c.inc | 18 ++++++++++++++
 2 files changed, 56 insertions(+)

diff --git a/target/ppc/insn64.decode b/target/ppc/insn64.decode
index 0eed35c8cd..5ecc5c85bf 100644
--- a/target/ppc/insn64.decode
+++ b/target/ppc/insn64.decode
@@ -73,10 +73,15 @@
 %xx3_xa         2:1 16:5
 %xx3_xb         1:1 11:5
 %xx3_at         23:3
+%xx3_xa_pair    2:1 17:4 !function=times_2
 @MMIRR_XX3      ...... .. .... .. . . ........ xmsk:4 ymsk:4  \
                 ...... ... .. ..... ..... ........ ...  \
                 &MMIRR_XX3 xa=%xx3_xa xb=%xx3_xb xt=%xx3_at
 
+@MMIRR_XX3_NO_P ...... .. .... .. . . ........ xmsk:4 .... \
+                ...... ... .. ..... ..... ........ ... \
+                &MMIRR_XX3 xb=%xx3_xb xt=%xx3_at pmsk=1
+
 ### Fixed-Point Load Instructions
 
 PLBZ            000001 10 0--.-- .................. \
@@ -145,6 +150,39 @@ PMXVI16GER2S    000001 11 1001 -- - - pmsk:2 ------ ........       \
 PMXVI16GER2SPP  000001 11 1001 -- - - pmsk:2 ------ ........       \
                 111011 ... -- ..... ..... 00101010 ..-  @MMIRR_XX3
 
+PMXVF16GER2     000001 11 1001 -- - - pmsk:2 ------ ........ \
+                111011 ... -- ..... ..... 00010011 ..-  @MMIRR_XX3
+PMXVF16GER2PP   000001 11 1001 -- - - pmsk:2 ------ ........ \
+                111011 ... -- ..... ..... 00010010 ..-  @MMIRR_XX3
+PMXVF16GER2PN   000001 11 1001 -- - - pmsk:2 ------ ........ \
+                111011 ... -- ..... ..... 10010010 ..-  @MMIRR_XX3
+PMXVF16GER2NP   000001 11 1001 -- - - pmsk:2 ------ ........ \
+                111011 ... -- ..... ..... 01010010 ..-  @MMIRR_XX3
+PMXVF16GER2NN   000001 11 1001 -- - - pmsk:2 ------ ........ \
+                111011 ... -- ..... ..... 11010010 ..-  @MMIRR_XX3
+
+PMXVF32GER      000001 11 1001 -- - - -------- .... ymsk:4 \
+                111011 ... -- ..... ..... 00011011 ..-  @MMIRR_XX3_NO_P xa=%xx3_xa
+PMXVF32GERPP    000001 11 1001 -- - - -------- .... ymsk:4 \
+                111011 ... -- ..... ..... 00011010 ..-  @MMIRR_XX3_NO_P xa=%xx3_xa
+PMXVF32GERPN    000001 11 1001 -- - - -------- .... ymsk:4 \
+                111011 ... -- ..... ..... 10011010 ..-  @MMIRR_XX3_NO_P xa=%xx3_xa
+PMXVF32GERNP    000001 11 1001 -- - - -------- .... ymsk:4 \
+                111011 ... -- ..... ..... 01011010 ..-  @MMIRR_XX3_NO_P xa=%xx3_xa
+PMXVF32GERNN    000001 11 1001 -- - - -------- .... ymsk:4 \
+                111011 ... -- ..... ..... 11011010 ..-  @MMIRR_XX3_NO_P xa=%xx3_xa
+
+PMXVF64GER      000001 11 1001 -- - - -------- .... ymsk:2 -- \
+                111011 ... -- ....0 ..... 00111011 ..-  @MMIRR_XX3_NO_P xa=%xx3_xa_pair
+PMXVF64GERPP    000001 11 1001 -- - - -------- .... ymsk:2 -- \
+                111011 ... -- ....0 ..... 00111010 ..-  @MMIRR_XX3_NO_P xa=%xx3_xa_pair
+PMXVF64GERPN    000001 11 1001 -- - - -------- .... ymsk:2 -- \
+                111011 ... -- ....0 ..... 10111010 ..-  @MMIRR_XX3_NO_P xa=%xx3_xa_pair
+PMXVF64GERNP    000001 11 1001 -- - - -------- .... ymsk:2 -- \
+                111011 ... -- ....0 ..... 01111010 ..-  @MMIRR_XX3_NO_P xa=%xx3_xa_pair
+PMXVF64GERNN    000001 11 1001 -- - - -------- .... ymsk:2 -- \
+                111011 ... -- ....0 ..... 11111010 ..-  @MMIRR_XX3_NO_P xa=%xx3_xa_pair
+
 ### Prefixed No-operation Instruction
 
 @PNOP           000001 11 0000-- 000000000000000000     \
diff --git a/target/ppc/translate/vsx-impl.c.inc b/target/ppc/translate/vsx-impl.c.inc
index 01978a585a..e79752899a 100644
--- a/target/ppc/translate/vsx-impl.c.inc
+++ b/target/ppc/translate/vsx-impl.c.inc
@@ -2916,6 +2916,24 @@ TRANS(XVF64GERPN, do_ger, gen_helper_XVF64GERPN)
 TRANS(XVF64GERNP, do_ger, gen_helper_XVF64GERNP)
 TRANS(XVF64GERNN, do_ger, gen_helper_XVF64GERNN)
 
+TRANS64(PMXVF16GER2, do_ger, gen_helper_XVF16GER2)
+TRANS64(PMXVF16GER2PP, do_ger, gen_helper_XVF16GER2PP)
+TRANS64(PMXVF16GER2PN, do_ger, gen_helper_XVF16GER2PN)
+TRANS64(PMXVF16GER2NP, do_ger, gen_helper_XVF16GER2NP)
+TRANS64(PMXVF16GER2NN, do_ger, gen_helper_XVF16GER2NN)
+
+TRANS64(PMXVF32GER, do_ger, gen_helper_XVF32GER)
+TRANS64(PMXVF32GERPP, do_ger, gen_helper_XVF32GERPP)
+TRANS64(PMXVF32GERPN, do_ger, gen_helper_XVF32GERPN)
+TRANS64(PMXVF32GERNP, do_ger, gen_helper_XVF32GERNP)
+TRANS64(PMXVF32GERNN, do_ger, gen_helper_XVF32GERNN)
+
+TRANS64(PMXVF64GER, do_ger, gen_helper_XVF64GER)
+TRANS64(PMXVF64GERPP, do_ger, gen_helper_XVF64GERPP)
+TRANS64(PMXVF64GERPN, do_ger, gen_helper_XVF64GERPN)
+TRANS64(PMXVF64GERNP, do_ger, gen_helper_XVF64GERNP)
+TRANS64(PMXVF64GERNN, do_ger, gen_helper_XVF64GERNN)
+
 #undef GEN_XX2FORM
 #undef GEN_XX3FORM
 #undef GEN_XX2IFORM

From 5724e131ca6cd970555e66fb47b90b287ffa8ea0 Mon Sep 17 00:00:00 2001
From: "Lucas Mateus Castro (alqotel)" <lucas.araujo@eldorado.org.br>
Date: Tue, 24 May 2022 11:05:36 -0300
Subject: [PATCH 564/935] target/ppc: Implemented [pm]xvbf16ger2*

Implement the following PowerISA v3.1 instructions:
xvbf16ger2:   VSX Vector bfloat16 GER (rank-2 update)
xvbf16ger2nn: VSX Vector bfloat16 GER (rank-2 update) Negative multiply,
Negative accumulate
xvbf16ger2np: VSX Vector bfloat16 GER (rank-2 update) Negative multiply,
Positive accumulate
xvbf16ger2pn: VSX Vector bfloat16 GER (rank-2 update) Positive multiply,
Negative accumulate
xvbf16ger2pp: VSX Vector bfloat16 GER (rank-2 update) Positive multiply,
Positive accumulate
pmxvbf16ger2:   Prefixed Masked VSX Vector bfloat16 GER (rank-2 update)
pmxvbf16ger2nn: Prefixed Masked VSX Vector bfloat16 GER (rank-2 update)
Negative multiply, Negative accumulate
pmxvbf16ger2np: Prefixed Masked VSX Vector bfloat16 GER (rank-2 update)
Negative multiply, Positive accumulate
pmxvbf16ger2pn: Prefixed Masked VSX Vector bfloat16 GER (rank-2 update)
Positive multiply, Negative accumulate
pmxvbf16ger2pp: Prefixed Masked VSX Vector bfloat16 GER (rank-2 update)
Positive multiply, Positive accumulate

Signed-off-by: Lucas Mateus Castro (alqotel) <lucas.araujo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220524140537.27451-8-lucas.araujo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 target/ppc/fpu_helper.c             | 40 +++++++++++++++++++++++++++++
 target/ppc/helper.h                 |  5 ++++
 target/ppc/insn32.decode            |  6 +++++
 target/ppc/insn64.decode            | 11 ++++++++
 target/ppc/translate/vsx-impl.c.inc | 12 +++++++++
 5 files changed, 74 insertions(+)

diff --git a/target/ppc/fpu_helper.c b/target/ppc/fpu_helper.c
index a9b2ef370f..fed0ce420a 100644
--- a/target/ppc/fpu_helper.c
+++ b/target/ppc/fpu_helper.c
@@ -3517,6 +3517,11 @@ static float64 extract_hf16(float16 in, float_status *fp_status)
     return float16_to_float64(in, true, fp_status);
 }
 
+static float64 extract_bf16(bfloat16 in, float_status *fp_status)
+{
+    return bfloat16_to_float64(in, fp_status);
+}
+
 static void vsxger16(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
                      ppc_acc_t  *at, uint32_t mask, bool acc,
                      bool neg_mul, bool neg_acc, extract_f16 extract)
@@ -3639,6 +3644,41 @@ static void vsxger(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
     vsxger_excp(env, GETPC());
 }
 
+QEMU_FLATTEN
+void helper_XVBF16GER2(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                       ppc_acc_t *at, uint32_t mask)
+{
+    vsxger16(env, a, b, at, mask, false, false, false, extract_bf16);
+}
+
+QEMU_FLATTEN
+void helper_XVBF16GER2PP(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                         ppc_acc_t *at, uint32_t mask)
+{
+    vsxger16(env, a, b, at, mask, true, false, false, extract_bf16);
+}
+
+QEMU_FLATTEN
+void helper_XVBF16GER2PN(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                         ppc_acc_t *at, uint32_t mask)
+{
+    vsxger16(env, a, b, at, mask, true, false, true, extract_bf16);
+}
+
+QEMU_FLATTEN
+void helper_XVBF16GER2NP(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                         ppc_acc_t *at, uint32_t mask)
+{
+    vsxger16(env, a, b, at, mask, true, true, false, extract_bf16);
+}
+
+QEMU_FLATTEN
+void helper_XVBF16GER2NN(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
+                         ppc_acc_t *at, uint32_t mask)
+{
+    vsxger16(env, a, b, at, mask, true, true, true, extract_bf16);
+}
+
 QEMU_FLATTEN
 void helper_XVF16GER2(CPUPPCState *env, ppc_vsr_t *a, ppc_vsr_t *b,
                      ppc_acc_t *at, uint32_t mask)
diff --git a/target/ppc/helper.h b/target/ppc/helper.h
index 4070c0891c..6233e28d85 100644
--- a/target/ppc/helper.h
+++ b/target/ppc/helper.h
@@ -555,6 +555,11 @@ DEF_HELPER_5(XVF16GER2PP, void, env, vsr, vsr, acc, i32)
 DEF_HELPER_5(XVF16GER2PN, void, env, vsr, vsr, acc, i32)
 DEF_HELPER_5(XVF16GER2NP, void, env, vsr, vsr, acc, i32)
 DEF_HELPER_5(XVF16GER2NN, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVBF16GER2, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVBF16GER2PP, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVBF16GER2PN, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVBF16GER2NP, void, env, vsr, vsr, acc, i32)
+DEF_HELPER_5(XVBF16GER2NN, void, env, vsr, vsr, acc, i32)
 DEF_HELPER_5(XVF32GER, void, env, vsr, vsr, acc, i32)
 DEF_HELPER_5(XVF32GERPP, void, env, vsr, vsr, acc, i32)
 DEF_HELPER_5(XVF32GERPN, void, env, vsr, vsr, acc, i32)
diff --git a/target/ppc/insn32.decode b/target/ppc/insn32.decode
index b8e317159c..18a94fa3b5 100644
--- a/target/ppc/insn32.decode
+++ b/target/ppc/insn32.decode
@@ -763,6 +763,12 @@ XVI8GER4SPP     111011 ... -- ..... ..... 01100011 ..-  @XX3_at xa=%xx_xa
 XVI16GER2S      111011 ... -- ..... ..... 00101011 ..-  @XX3_at xa=%xx_xa
 XVI16GER2SPP    111011 ... -- ..... ..... 00101010 ..-  @XX3_at xa=%xx_xa
 
+XVBF16GER2      111011 ... -- ..... ..... 00110011 ..-  @XX3_at xa=%xx_xa
+XVBF16GER2PP    111011 ... -- ..... ..... 00110010 ..-  @XX3_at xa=%xx_xa
+XVBF16GER2PN    111011 ... -- ..... ..... 10110010 ..-  @XX3_at xa=%xx_xa
+XVBF16GER2NP    111011 ... -- ..... ..... 01110010 ..-  @XX3_at xa=%xx_xa
+XVBF16GER2NN    111011 ... -- ..... ..... 11110010 ..-  @XX3_at xa=%xx_xa
+
 XVF16GER2       111011 ... -- ..... ..... 00010011 ..-  @XX3_at xa=%xx_xa
 XVF16GER2PP     111011 ... -- ..... ..... 00010010 ..-  @XX3_at xa=%xx_xa
 XVF16GER2PN     111011 ... -- ..... ..... 10010010 ..-  @XX3_at xa=%xx_xa
diff --git a/target/ppc/insn64.decode b/target/ppc/insn64.decode
index 5ecc5c85bf..de115c1943 100644
--- a/target/ppc/insn64.decode
+++ b/target/ppc/insn64.decode
@@ -150,6 +150,17 @@ PMXVI16GER2S    000001 11 1001 -- - - pmsk:2 ------ ........       \
 PMXVI16GER2SPP  000001 11 1001 -- - - pmsk:2 ------ ........       \
                 111011 ... -- ..... ..... 00101010 ..-  @MMIRR_XX3
 
+PMXVBF16GER2    000001 11 1001 -- - - pmsk:2 ------ ........ \
+                111011 ... -- ..... ..... 00110011 ..-  @MMIRR_XX3
+PMXVBF16GER2PP  000001 11 1001 -- - - pmsk:2 ------ ........ \
+                111011 ... -- ..... ..... 00110010 ..-  @MMIRR_XX3
+PMXVBF16GER2PN  000001 11 1001 -- - - pmsk:2 ------ ........ \
+                111011 ... -- ..... ..... 10110010 ..-  @MMIRR_XX3
+PMXVBF16GER2NP  000001 11 1001 -- - - pmsk:2 ------ ........ \
+                111011 ... -- ..... ..... 01110010 ..-  @MMIRR_XX3
+PMXVBF16GER2NN  000001 11 1001 -- - - pmsk:2 ------ ........ \
+                111011 ... -- ..... ..... 11110010 ..-  @MMIRR_XX3
+
 PMXVF16GER2     000001 11 1001 -- - - pmsk:2 ------ ........ \
                 111011 ... -- ..... ..... 00010011 ..-  @MMIRR_XX3
 PMXVF16GER2PP   000001 11 1001 -- - - pmsk:2 ------ ........ \
diff --git a/target/ppc/translate/vsx-impl.c.inc b/target/ppc/translate/vsx-impl.c.inc
index e79752899a..7acdbceec4 100644
--- a/target/ppc/translate/vsx-impl.c.inc
+++ b/target/ppc/translate/vsx-impl.c.inc
@@ -2898,6 +2898,12 @@ TRANS64(PMXVI16GER2PP, do_ger, gen_helper_XVI16GER2PP)
 TRANS64(PMXVI16GER2S, do_ger, gen_helper_XVI16GER2S)
 TRANS64(PMXVI16GER2SPP, do_ger, gen_helper_XVI16GER2SPP)
 
+TRANS(XVBF16GER2, do_ger, gen_helper_XVBF16GER2)
+TRANS(XVBF16GER2PP, do_ger, gen_helper_XVBF16GER2PP)
+TRANS(XVBF16GER2PN, do_ger, gen_helper_XVBF16GER2PN)
+TRANS(XVBF16GER2NP, do_ger, gen_helper_XVBF16GER2NP)
+TRANS(XVBF16GER2NN, do_ger, gen_helper_XVBF16GER2NN)
+
 TRANS(XVF16GER2, do_ger, gen_helper_XVF16GER2)
 TRANS(XVF16GER2PP, do_ger, gen_helper_XVF16GER2PP)
 TRANS(XVF16GER2PN, do_ger, gen_helper_XVF16GER2PN)
@@ -2916,6 +2922,12 @@ TRANS(XVF64GERPN, do_ger, gen_helper_XVF64GERPN)
 TRANS(XVF64GERNP, do_ger, gen_helper_XVF64GERNP)
 TRANS(XVF64GERNN, do_ger, gen_helper_XVF64GERNN)
 
+TRANS64(PMXVBF16GER2, do_ger, gen_helper_XVBF16GER2)
+TRANS64(PMXVBF16GER2PP, do_ger, gen_helper_XVBF16GER2PP)
+TRANS64(PMXVBF16GER2PN, do_ger, gen_helper_XVBF16GER2PN)
+TRANS64(PMXVBF16GER2NP, do_ger, gen_helper_XVBF16GER2NP)
+TRANS64(PMXVBF16GER2NN, do_ger, gen_helper_XVBF16GER2NN)
+
 TRANS64(PMXVF16GER2, do_ger, gen_helper_XVF16GER2)
 TRANS64(PMXVF16GER2PP, do_ger, gen_helper_XVF16GER2PP)
 TRANS64(PMXVF16GER2PN, do_ger, gen_helper_XVF16GER2PN)

From 96c343cc774b52b010e464a219d13f8e55e1003f Mon Sep 17 00:00:00 2001
From: Joel Stanley <joel@jms.id.au>
Date: Tue, 24 May 2022 11:05:37 -0300
Subject: [PATCH 565/935] linux-user: Add PowerPC ISA 3.1 and MMA to hwcap

These are new hwcap bits added for power10.

Signed-off-by: Joel Stanley <joel@jms.id.au>
Signed-off-by: Lucas Mateus Castro (alqotel) <lucas.araujo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220524140537.27451-9-lucas.araujo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 linux-user/elfload.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index acc21748f9..f7eae357f4 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -779,6 +779,8 @@ enum {
     QEMU_PPC_FEATURE2_DARN = 0x00200000, /* darn random number insn */
     QEMU_PPC_FEATURE2_SCV = 0x00100000, /* scv syscall */
     QEMU_PPC_FEATURE2_HTM_NO_SUSPEND = 0x00080000, /* TM w/o suspended state */
+    QEMU_PPC_FEATURE2_ARCH_3_1 = 0x00040000, /* ISA 3.1 */
+    QEMU_PPC_FEATURE2_MMA = 0x00020000, /* Matrix-Multiply Assist */
 };
 
 #define ELF_HWCAP get_elf_hwcap()
@@ -836,6 +838,8 @@ static uint32_t get_elf_hwcap2(void)
                   QEMU_PPC_FEATURE2_VEC_CRYPTO);
     GET_FEATURE2(PPC2_ISA300, QEMU_PPC_FEATURE2_ARCH_3_00 |
                  QEMU_PPC_FEATURE2_DARN | QEMU_PPC_FEATURE2_HAS_IEEE128);
+    GET_FEATURE2(PPC2_ISA310, QEMU_PPC_FEATURE2_ARCH_3_1 |
+                 QEMU_PPC_FEATURE2_MMA);
 
 #undef GET_FEATURE
 #undef GET_FEATURE2

From 06680b15b4ee3184b57c9c910886a9ff8a7e58c0 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Wed, 25 May 2022 16:41:26 +0200
Subject: [PATCH 566/935] include: move qemu_*_exec_dir() to cutils
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The function is required by get_relocated_path() (already in cutils),
and used by qemu-ga and may be generally useful.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220525144140.591926-2-marcandre.lureau@redhat.com>
---
 include/qemu/cutils.h                |   7 ++
 include/qemu/osdep.h                 |   8 --
 qemu-io.c                            |   1 +
 storage-daemon/qemu-storage-daemon.c |   1 +
 tests/qtest/fuzz/fuzz.c              |   1 +
 util/cutils.c                        | 117 +++++++++++++++++++++++++++
 util/oslib-posix.c                   |  86 +-------------------
 util/oslib-win32.c                   |  36 ---------
 8 files changed, 129 insertions(+), 128 deletions(-)

diff --git a/include/qemu/cutils.h b/include/qemu/cutils.h
index 5c6572d444..40e10e19a7 100644
--- a/include/qemu/cutils.h
+++ b/include/qemu/cutils.h
@@ -193,6 +193,13 @@ int uleb128_decode_small(const uint8_t *in, uint32_t *n);
  */
 int qemu_pstrcmp0(const char **str1, const char **str2);
 
+/* Find program directory, and save it for later usage with
+ * qemu_get_exec_dir().
+ * Try OS specific API first, if not working, parse from argv0. */
+void qemu_init_exec_dir(const char *argv0);
+
+/* Get the saved exec dir.  */
+const char *qemu_get_exec_dir(void);
 
 /**
  * get_relocated_path:
diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index a72e99db85..b1c161c035 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -557,14 +557,6 @@ void qemu_set_cloexec(int fd);
  */
 char *qemu_get_local_state_dir(void);
 
-/* Find program directory, and save it for later usage with
- * qemu_get_exec_dir().
- * Try OS specific API first, if not working, parse from argv0. */
-void qemu_init_exec_dir(const char *argv0);
-
-/* Get the saved exec dir.  */
-const char *qemu_get_exec_dir(void);
-
 /**
  * qemu_getauxval:
  * @type: the auxiliary vector key to lookup
diff --git a/qemu-io.c b/qemu-io.c
index d70d3dd4fd..2bd7bfb650 100644
--- a/qemu-io.c
+++ b/qemu-io.c
@@ -16,6 +16,7 @@
 #endif
 
 #include "qemu/help-texts.h"
+#include "qemu/cutils.h"
 #include "qapi/error.h"
 #include "qemu-io.h"
 #include "qemu/error-report.h"
diff --git a/storage-daemon/qemu-storage-daemon.c b/storage-daemon/qemu-storage-daemon.c
index 9b8b17f52e..c104817cdd 100644
--- a/storage-daemon/qemu-storage-daemon.c
+++ b/storage-daemon/qemu-storage-daemon.c
@@ -44,6 +44,7 @@
 
 #include "qemu/help-texts.h"
 #include "qemu-version.h"
+#include "qemu/cutils.h"
 #include "qemu/config-file.h"
 #include "qemu/error-report.h"
 #include "qemu/help_option.h"
diff --git a/tests/qtest/fuzz/fuzz.c b/tests/qtest/fuzz/fuzz.c
index a7a5e14fa3..0ad4ba9e94 100644
--- a/tests/qtest/fuzz/fuzz.c
+++ b/tests/qtest/fuzz/fuzz.c
@@ -15,6 +15,7 @@
 
 #include <wordexp.h>
 
+#include "qemu/cutils.h"
 #include "qemu/datadir.h"
 #include "sysemu/sysemu.h"
 #include "sysemu/qtest.h"
diff --git a/util/cutils.c b/util/cutils.c
index b2777210e7..a58bcfd80e 100644
--- a/util/cutils.c
+++ b/util/cutils.c
@@ -26,6 +26,15 @@
 #include "qemu/host-utils.h"
 #include <math.h>
 
+#ifdef __FreeBSD__
+#include <sys/sysctl.h>
+#include <sys/user.h>
+#endif
+
+#ifdef __NetBSD__
+#include <sys/sysctl.h>
+#endif
+
 #include "qemu/ctype.h"
 #include "qemu/cutils.h"
 #include "qemu/error-report.h"
@@ -931,6 +940,114 @@ static inline const char *next_component(const char *dir, int *p_len)
     return dir;
 }
 
+static const char *exec_dir;
+
+void qemu_init_exec_dir(const char *argv0)
+{
+#ifdef G_OS_WIN32
+    char *p;
+    char buf[MAX_PATH];
+    DWORD len;
+
+    if (exec_dir) {
+        return;
+    }
+
+    len = GetModuleFileName(NULL, buf, sizeof(buf) - 1);
+    if (len == 0) {
+        return;
+    }
+
+    buf[len] = 0;
+    p = buf + len - 1;
+    while (p != buf && *p != '\\') {
+        p--;
+    }
+    *p = 0;
+    if (access(buf, R_OK) == 0) {
+        exec_dir = g_strdup(buf);
+    } else {
+        exec_dir = CONFIG_BINDIR;
+    }
+#else
+    char *p = NULL;
+    char buf[PATH_MAX];
+
+    if (exec_dir) {
+        return;
+    }
+
+#if defined(__linux__)
+    {
+        int len;
+        len = readlink("/proc/self/exe", buf, sizeof(buf) - 1);
+        if (len > 0) {
+            buf[len] = 0;
+            p = buf;
+        }
+    }
+#elif defined(__FreeBSD__) \
+      || (defined(__NetBSD__) && defined(KERN_PROC_PATHNAME))
+    {
+#if defined(__FreeBSD__)
+        static int mib[4] = {CTL_KERN, KERN_PROC, KERN_PROC_PATHNAME, -1};
+#else
+        static int mib[4] = {CTL_KERN, KERN_PROC_ARGS, -1, KERN_PROC_PATHNAME};
+#endif
+        size_t len = sizeof(buf) - 1;
+
+        *buf = '\0';
+        if (!sysctl(mib, ARRAY_SIZE(mib), buf, &len, NULL, 0) &&
+            *buf) {
+            buf[sizeof(buf) - 1] = '\0';
+            p = buf;
+        }
+    }
+#elif defined(__APPLE__)
+    {
+        char fpath[PATH_MAX];
+        uint32_t len = sizeof(fpath);
+        if (_NSGetExecutablePath(fpath, &len) == 0) {
+            p = realpath(fpath, buf);
+            if (!p) {
+                return;
+            }
+        }
+    }
+#elif defined(__HAIKU__)
+    {
+        image_info ii;
+        int32_t c = 0;
+
+        *buf = '\0';
+        while (get_next_image_info(0, &c, &ii) == B_OK) {
+            if (ii.type == B_APP_IMAGE) {
+                strncpy(buf, ii.name, sizeof(buf));
+                buf[sizeof(buf) - 1] = 0;
+                p = buf;
+                break;
+            }
+        }
+    }
+#endif
+    /* If we don't have any way of figuring out the actual executable
+       location then try argv[0].  */
+    if (!p && argv0) {
+        p = realpath(argv0, buf);
+    }
+    if (p) {
+        exec_dir = g_path_get_dirname(p);
+    } else {
+        exec_dir = CONFIG_BINDIR;
+    }
+#endif
+}
+
+const char *qemu_get_exec_dir(void)
+{
+    return exec_dir;
+}
+
 char *get_relocated_path(const char *dir)
 {
     size_t prefix_len = strlen(CONFIG_PREFIX);
diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index 477990f39b..7a34c1657c 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -48,14 +48,13 @@
 #endif
 
 #ifdef __FreeBSD__
-#include <sys/sysctl.h>
-#include <sys/user.h>
 #include <sys/thr.h>
+#include <sys/types.h>
+#include <sys/user.h>
 #include <libutil.h>
 #endif
 
 #ifdef __NetBSD__
-#include <sys/sysctl.h>
 #include <lwp.h>
 #endif
 
@@ -283,87 +282,6 @@ void qemu_set_tty_echo(int fd, bool echo)
     tcsetattr(fd, TCSANOW, &tty);
 }
 
-static const char *exec_dir;
-
-void qemu_init_exec_dir(const char *argv0)
-{
-    char *p = NULL;
-    char buf[PATH_MAX];
-
-    if (exec_dir) {
-        return;
-    }
-
-#if defined(__linux__)
-    {
-        int len;
-        len = readlink("/proc/self/exe", buf, sizeof(buf) - 1);
-        if (len > 0) {
-            buf[len] = 0;
-            p = buf;
-        }
-    }
-#elif defined(__FreeBSD__) \
-      || (defined(__NetBSD__) && defined(KERN_PROC_PATHNAME))
-    {
-#if defined(__FreeBSD__)
-        static int mib[4] = {CTL_KERN, KERN_PROC, KERN_PROC_PATHNAME, -1};
-#else
-        static int mib[4] = {CTL_KERN, KERN_PROC_ARGS, -1, KERN_PROC_PATHNAME};
-#endif
-        size_t len = sizeof(buf) - 1;
-
-        *buf = '\0';
-        if (!sysctl(mib, ARRAY_SIZE(mib), buf, &len, NULL, 0) &&
-            *buf) {
-            buf[sizeof(buf) - 1] = '\0';
-            p = buf;
-        }
-    }
-#elif defined(__APPLE__)
-    {
-        char fpath[PATH_MAX];
-        uint32_t len = sizeof(fpath);
-        if (_NSGetExecutablePath(fpath, &len) == 0) {
-            p = realpath(fpath, buf);
-            if (!p) {
-                return;
-            }
-        }
-    }
-#elif defined(__HAIKU__)
-    {
-        image_info ii;
-        int32_t c = 0;
-
-        *buf = '\0';
-        while (get_next_image_info(0, &c, &ii) == B_OK) {
-            if (ii.type == B_APP_IMAGE) {
-                strncpy(buf, ii.name, sizeof(buf));
-                buf[sizeof(buf) - 1] = 0;
-                p = buf;
-                break;
-            }
-        }
-    }
-#endif
-    /* If we don't have any way of figuring out the actual executable
-       location then try argv[0].  */
-    if (!p && argv0) {
-        p = realpath(argv0, buf);
-    }
-    if (p) {
-        exec_dir = g_path_get_dirname(p);
-    } else {
-        exec_dir = CONFIG_BINDIR;
-    }
-}
-
-const char *qemu_get_exec_dir(void)
-{
-    return exec_dir;
-}
-
 #ifdef CONFIG_LINUX
 static void sigbus_handler(int signal, siginfo_t *siginfo, void *ctx)
 #else /* CONFIG_LINUX */
diff --git a/util/oslib-win32.c b/util/oslib-win32.c
index dafef4f157..6c818749d2 100644
--- a/util/oslib-win32.c
+++ b/util/oslib-win32.c
@@ -269,42 +269,6 @@ void qemu_set_tty_echo(int fd, bool echo)
     }
 }
 
-static const char *exec_dir;
-
-void qemu_init_exec_dir(const char *argv0)
-{
-
-    char *p;
-    char buf[MAX_PATH];
-    DWORD len;
-
-    if (exec_dir) {
-        return;
-    }
-
-    len = GetModuleFileName(NULL, buf, sizeof(buf) - 1);
-    if (len == 0) {
-        return;
-    }
-
-    buf[len] = 0;
-    p = buf + len - 1;
-    while (p != buf && *p != '\\') {
-        p--;
-    }
-    *p = 0;
-    if (access(buf, R_OK) == 0) {
-        exec_dir = g_strdup(buf);
-    } else {
-        exec_dir = CONFIG_BINDIR;
-    }
-}
-
-const char *qemu_get_exec_dir(void)
-{
-    return exec_dir;
-}
-
 int getpagesize(void)
 {
     SYSTEM_INFO system_info;

From 49e0128c483aa4a49860210ea51e00523ec26aaa Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Wed, 25 May 2022 16:41:27 +0200
Subject: [PATCH 567/935] util/win32: simplify qemu_get_local_state_dir()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

SHGetFolderPath() is a deprecated API:
https://docs.microsoft.com/en-us/windows/win32/api/shlobj_core/nf-shlobj_core-shgetfolderpatha

It is a wrapper for SHGetKnownFolderPath() and CSIDL_COMMON_PATH is
mapped to FOLDERID_ProgramData:
https://docs.microsoft.com/en-us/windows/win32/shell/csidl

g_get_system_data_dirs() is a suitable replacement, as it will have
FOLDERID_ProgramData in the returned list. However, it follows the XDG
Base Directory Specification, if `XDG_DATA_DIRS` is defined, it will be
returned instead.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Stefan Weil <sw@weilnetz.de>
Message-Id: <20220525144140.591926-3-marcandre.lureau@redhat.com>
---
 util/oslib-win32.c | 17 ++++-------------
 1 file changed, 4 insertions(+), 13 deletions(-)

diff --git a/util/oslib-win32.c b/util/oslib-win32.c
index 6c818749d2..5723d3eb4c 100644
--- a/util/oslib-win32.c
+++ b/util/oslib-win32.c
@@ -40,9 +40,6 @@
 #include "qemu/error-report.h"
 #include <malloc.h>
 
-/* this must come after including "trace.h" */
-#include <shlobj.h>
-
 static int get_allocation_granularity(void)
 {
     SYSTEM_INFO system_info;
@@ -237,17 +234,11 @@ int qemu_get_thread_id(void)
 char *
 qemu_get_local_state_dir(void)
 {
-    HRESULT result;
-    char base_path[MAX_PATH+1] = "";
+    const char * const *data_dirs = g_get_system_data_dirs();
 
-    result = SHGetFolderPath(NULL, CSIDL_COMMON_APPDATA, NULL,
-                             /* SHGFP_TYPE_CURRENT */ 0, base_path);
-    if (result != S_OK) {
-        /* misconfigured environment */
-        g_critical("CSIDL_COMMON_APPDATA unavailable: %ld", (long)result);
-        abort();
-    }
-    return g_strdup(base_path);
+    g_assert(data_dirs && data_dirs[0]);
+
+    return g_strdup(data_dirs[0]);
 }
 
 void qemu_set_tty_echo(int fd, bool echo)

From 56b6dab2743b276a50036cd0a115e98cb1134f38 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Wed, 25 May 2022 16:41:28 +0200
Subject: [PATCH 568/935] tests: make libqmp buildable for win32
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220525144140.591926-4-marcandre.lureau@redhat.com>
---
 tests/qtest/libqmp.c | 34 +++++++++++++++++++++++++++++-----
 tests/qtest/libqmp.h |  2 ++
 2 files changed, 31 insertions(+), 5 deletions(-)

diff --git a/tests/qtest/libqmp.c b/tests/qtest/libqmp.c
index 0358b8313d..ade26c15f0 100644
--- a/tests/qtest/libqmp.c
+++ b/tests/qtest/libqmp.c
@@ -18,6 +18,11 @@
 
 #include "libqmp.h"
 
+#ifndef _WIN32
+#include <sys/socket.h>
+#endif
+
+#include "qemu/cutils.h"
 #include "qapi/error.h"
 #include "qapi/qmp/json-parser.h"
 #include "qapi/qmp/qjson.h"
@@ -87,6 +92,7 @@ QDict *qmp_fd_receive(int fd)
     return qmp.response;
 }
 
+#ifndef _WIN32
 /* Sends a message and file descriptors to the socket.
  * It's needed for qmp-commands like getfd/add-fd */
 static void socket_send_fds(int socket_fd, int *fds, size_t fds_num,
@@ -120,17 +126,23 @@ static void socket_send_fds(int socket_fd, int *fds, size_t fds_num,
     } while (ret < 0 && errno == EINTR);
     g_assert_cmpint(ret, >, 0);
 }
+#endif
 
 /**
  * Allow users to send a message without waiting for the reply,
  * in the case that they choose to discard all replies up until
  * a particular EVENT is received.
  */
-void qmp_fd_vsend_fds(int fd, int *fds, size_t fds_num,
-                      const char *fmt, va_list ap)
+static void
+_qmp_fd_vsend_fds(int fd, int *fds, size_t fds_num,
+                  const char *fmt, va_list ap)
 {
     QObject *qobj;
 
+#ifdef _WIN32
+    assert(fds_num == 0);
+#endif
+
     /* Going through qobject ensures we escape strings properly */
     qobj = qobject_from_vjsonf_nofail(fmt, ap);
 
@@ -148,10 +160,14 @@ void qmp_fd_vsend_fds(int fd, int *fds, size_t fds_num,
         if (log) {
             fprintf(stderr, "%s", str->str);
         }
+
+#ifndef _WIN32
         /* Send QMP request */
         if (fds && fds_num > 0) {
             socket_send_fds(fd, fds, fds_num, str->str, str->len);
-        } else {
+        } else
+#endif
+        {
             socket_send(fd, str->str, str->len);
         }
 
@@ -160,15 +176,23 @@ void qmp_fd_vsend_fds(int fd, int *fds, size_t fds_num,
     }
 }
 
+#ifndef _WIN32
+void qmp_fd_vsend_fds(int fd, int *fds, size_t fds_num,
+                      const char *fmt, va_list ap)
+{
+    _qmp_fd_vsend_fds(fd, fds, fds_num, fmt, ap);
+}
+#endif
+
 void qmp_fd_vsend(int fd, const char *fmt, va_list ap)
 {
-    qmp_fd_vsend_fds(fd, NULL, 0, fmt, ap);
+    _qmp_fd_vsend_fds(fd, NULL, 0, fmt, ap);
 }
 
 
 QDict *qmp_fdv(int fd, const char *fmt, va_list ap)
 {
-    qmp_fd_vsend_fds(fd, NULL, 0, fmt, ap);
+    _qmp_fd_vsend_fds(fd, NULL, 0, fmt, ap);
 
     return qmp_fd_receive(fd);
 }
diff --git a/tests/qtest/libqmp.h b/tests/qtest/libqmp.h
index 5cb7eeaa18..3445b753ff 100644
--- a/tests/qtest/libqmp.h
+++ b/tests/qtest/libqmp.h
@@ -21,8 +21,10 @@
 #include "qapi/qmp/qdict.h"
 
 QDict *qmp_fd_receive(int fd);
+#ifndef _WIN32
 void qmp_fd_vsend_fds(int fd, int *fds, size_t fds_num,
                       const char *fmt, va_list ap) G_GNUC_PRINTF(4, 0);
+#endif
 void qmp_fd_vsend(int fd, const char *fmt, va_list ap) G_GNUC_PRINTF(2, 0);
 void qmp_fd_send(int fd, const char *fmt, ...) G_GNUC_PRINTF(2, 3);
 void qmp_fd_send_raw(int fd, const char *fmt, ...) G_GNUC_PRINTF(2, 3);

From 69f56c140de06fee0f266b93634475e1d968e430 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Wed, 25 May 2022 16:41:29 +0200
Subject: [PATCH 569/935] qga: flatten safe_open_or_create()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

There is a bit too much nesting in the function, this can be simplified
a bit to improve readability.

This also helps with the following error handling changes.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220525144140.591926-5-marcandre.lureau@redhat.com>
---
 qga/commands-posix.c | 126 +++++++++++++++++++++----------------------
 1 file changed, 63 insertions(+), 63 deletions(-)

diff --git a/qga/commands-posix.c b/qga/commands-posix.c
index 12b50b7124..3b2392398e 100644
--- a/qga/commands-posix.c
+++ b/qga/commands-posix.c
@@ -339,73 +339,73 @@ find_open_flag(const char *mode_str, Error **errp)
 static FILE *
 safe_open_or_create(const char *path, const char *mode, Error **errp)
 {
-    Error *local_err = NULL;
     int oflag;
+    int fd = -1;
+    FILE *f = NULL;
 
-    oflag = find_open_flag(mode, &local_err);
-    if (local_err == NULL) {
-        int fd;
-
-        /* If the caller wants / allows creation of a new file, we implement it
-         * with a two step process: open() + (open() / fchmod()).
-         *
-         * First we insist on creating the file exclusively as a new file. If
-         * that succeeds, we're free to set any file-mode bits on it. (The
-         * motivation is that we want to set those file-mode bits independently
-         * of the current umask.)
-         *
-         * If the exclusive creation fails because the file already exists
-         * (EEXIST is not possible for any other reason), we just attempt to
-         * open the file, but in this case we won't be allowed to change the
-         * file-mode bits on the preexistent file.
-         *
-         * The pathname should never disappear between the two open()s in
-         * practice. If it happens, then someone very likely tried to race us.
-         * In this case just go ahead and report the ENOENT from the second
-         * open() to the caller.
-         *
-         * If the caller wants to open a preexistent file, then the first
-         * open() is decisive and its third argument is ignored, and the second
-         * open() and the fchmod() are never called.
-         */
-        fd = open(path, oflag | ((oflag & O_CREAT) ? O_EXCL : 0), 0);
-        if (fd == -1 && errno == EEXIST) {
-            oflag &= ~(unsigned)O_CREAT;
-            fd = open(path, oflag);
-        }
-
-        if (fd == -1) {
-            error_setg_errno(&local_err, errno, "failed to open file '%s' "
-                             "(mode: '%s')", path, mode);
-        } else {
-            qemu_set_cloexec(fd);
-
-            if ((oflag & O_CREAT) && fchmod(fd, DEFAULT_NEW_FILE_MODE) == -1) {
-                error_setg_errno(&local_err, errno, "failed to set permission "
-                                 "0%03o on new file '%s' (mode: '%s')",
-                                 (unsigned)DEFAULT_NEW_FILE_MODE, path, mode);
-            } else {
-                FILE *f;
-
-                f = fdopen(fd, mode);
-                if (f == NULL) {
-                    error_setg_errno(&local_err, errno, "failed to associate "
-                                     "stdio stream with file descriptor %d, "
-                                     "file '%s' (mode: '%s')", fd, path, mode);
-                } else {
-                    return f;
-                }
-            }
-
-            close(fd);
-            if (oflag & O_CREAT) {
-                unlink(path);
-            }
-        }
+    oflag = find_open_flag(mode, errp);
+    if (oflag < 0) {
+        goto end;
     }
 
-    error_propagate(errp, local_err);
-    return NULL;
+    /* If the caller wants / allows creation of a new file, we implement it
+     * with a two step process: open() + (open() / fchmod()).
+     *
+     * First we insist on creating the file exclusively as a new file. If
+     * that succeeds, we're free to set any file-mode bits on it. (The
+     * motivation is that we want to set those file-mode bits independently
+     * of the current umask.)
+     *
+     * If the exclusive creation fails because the file already exists
+     * (EEXIST is not possible for any other reason), we just attempt to
+     * open the file, but in this case we won't be allowed to change the
+     * file-mode bits on the preexistent file.
+     *
+     * The pathname should never disappear between the two open()s in
+     * practice. If it happens, then someone very likely tried to race us.
+     * In this case just go ahead and report the ENOENT from the second
+     * open() to the caller.
+     *
+     * If the caller wants to open a preexistent file, then the first
+     * open() is decisive and its third argument is ignored, and the second
+     * open() and the fchmod() are never called.
+     */
+    fd = open(path, oflag | ((oflag & O_CREAT) ? O_EXCL : 0), 0);
+    if (fd == -1 && errno == EEXIST) {
+        oflag &= ~(unsigned)O_CREAT;
+        fd = open(path, oflag);
+    }
+    if (fd == -1) {
+        error_setg_errno(errp, errno,
+                         "failed to open file '%s' (mode: '%s')",
+                         path, mode);
+        goto end;
+    }
+
+    qemu_set_cloexec(fd);
+
+    if ((oflag & O_CREAT) && fchmod(fd, DEFAULT_NEW_FILE_MODE) == -1) {
+        error_setg_errno(errp, errno, "failed to set permission "
+                         "0%03o on new file '%s' (mode: '%s')",
+                         (unsigned)DEFAULT_NEW_FILE_MODE, path, mode);
+        goto end;
+    }
+
+    f = fdopen(fd, mode);
+    if (f == NULL) {
+        error_setg_errno(errp, errno, "failed to associate stdio stream with "
+                         "file descriptor %d, file '%s' (mode: '%s')",
+                         fd, path, mode);
+    }
+
+end:
+    if (f == NULL && fd != -1) {
+        close(fd);
+        if (oflag & O_CREAT) {
+            unlink(path);
+        }
+    }
+    return f;
 }
 
 int64_t qmp_guest_file_open(const char *path, bool has_mode, const char *mode,

From 0edbfbe31c532a98a8674b68c94a97537f4e4230 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Wed, 25 May 2022 16:41:30 +0200
Subject: [PATCH 570/935] qga: add qga_open_cloexec() helper
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

QGA calls qemu_open_old() in various places. Calling qemu_open() instead
isn't a great alternative, as it has special "/dev/fdset" handling and
depends on QEMU internal monitor data structures.

Instead, provide a simple helper for QGA needs, with Error* support. The
following patches will make use of it.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220525144140.591926-6-marcandre.lureau@redhat.com>
---
 qga/cutils.c    | 33 +++++++++++++++++++++++++++++++++
 qga/cutils.h    |  8 ++++++++
 qga/meson.build |  1 +
 3 files changed, 42 insertions(+)
 create mode 100644 qga/cutils.c
 create mode 100644 qga/cutils.h

diff --git a/qga/cutils.c b/qga/cutils.c
new file mode 100644
index 0000000000..b8e142ef64
--- /dev/null
+++ b/qga/cutils.c
@@ -0,0 +1,33 @@
+/*
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#include "cutils.h"
+
+#include "qapi/error.h"
+
+/**
+ * qga_open_cloexec:
+ * @name: the pathname to open
+ * @flags: as in open()
+ * @mode: as in open()
+ *
+ * A wrapper for open() function which sets O_CLOEXEC.
+ *
+ * On error, -1 is returned.
+ */
+int qga_open_cloexec(const char *name, int flags, mode_t mode)
+{
+    int ret;
+
+#ifdef O_CLOEXEC
+    ret = open(name, flags | O_CLOEXEC, mode);
+#else
+    ret = open(name, flags, mode);
+    if (ret >= 0) {
+        qemu_set_cloexec(ret);
+    }
+#endif
+
+    return ret;
+}
diff --git a/qga/cutils.h b/qga/cutils.h
new file mode 100644
index 0000000000..f0f30a7d28
--- /dev/null
+++ b/qga/cutils.h
@@ -0,0 +1,8 @@
+#ifndef CUTILS_H_
+#define CUTILS_H_
+
+#include "qemu/osdep.h"
+
+int qga_open_cloexec(const char *name, int flags, mode_t mode);
+
+#endif /* CUTILS_H_ */
diff --git a/qga/meson.build b/qga/meson.build
index 6d9f39bb32..35fe2229e9 100644
--- a/qga/meson.build
+++ b/qga/meson.build
@@ -65,6 +65,7 @@ qga_ss.add(files(
   'commands.c',
   'guest-agent-command-state.c',
   'main.c',
+  'cutils.c',
 ))
 qga_ss.add(when: 'CONFIG_POSIX', if_true: files(
   'channel-posix.c',

From 1a89a17b769352dbced5c6d8b0935e4d1f306c6e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Wed, 25 May 2022 16:41:31 +0200
Subject: [PATCH 571/935] qga: use qga_open_cloexec() for safe_open_or_create()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The function takes care of setting CLOEXEC.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220525144140.591926-7-marcandre.lureau@redhat.com>
---
 qga/commands-posix.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/qga/commands-posix.c b/qga/commands-posix.c
index 3b2392398e..2ecc43eca9 100644
--- a/qga/commands-posix.c
+++ b/qga/commands-posix.c
@@ -27,6 +27,7 @@
 #include "qemu/cutils.h"
 #include "commands-common.h"
 #include "block/nvme.h"
+#include "cutils.h"
 
 #ifdef HAVE_UTMPX
 #include <utmpx.h>
@@ -370,10 +371,10 @@ safe_open_or_create(const char *path, const char *mode, Error **errp)
      * open() is decisive and its third argument is ignored, and the second
      * open() and the fchmod() are never called.
      */
-    fd = open(path, oflag | ((oflag & O_CREAT) ? O_EXCL : 0), 0);
+    fd = qga_open_cloexec(path, oflag | ((oflag & O_CREAT) ? O_EXCL : 0), 0);
     if (fd == -1 && errno == EEXIST) {
         oflag &= ~(unsigned)O_CREAT;
-        fd = open(path, oflag);
+        fd = qga_open_cloexec(path, oflag, 0);
     }
     if (fd == -1) {
         error_setg_errno(errp, errno,
@@ -382,8 +383,6 @@ safe_open_or_create(const char *path, const char *mode, Error **errp)
         goto end;
     }
 
-    qemu_set_cloexec(fd);
-
     if ((oflag & O_CREAT) && fchmod(fd, DEFAULT_NEW_FILE_MODE) == -1) {
         error_setg_errno(errp, errno, "failed to set permission "
                          "0%03o on new file '%s' (mode: '%s')",

From 87ed8b2c2cd03d2596c11559f3e64ba15f58a99e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Wed, 25 May 2022 16:41:32 +0200
Subject: [PATCH 572/935] qga: throw an Error in ga_channel_open()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Allow for a single point of error reporting, and further refactoring.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220525144140.591926-8-marcandre.lureau@redhat.com>
---
 qga/channel-posix.c | 42 +++++++++++++++++-------------------------
 1 file changed, 17 insertions(+), 25 deletions(-)

diff --git a/qga/channel-posix.c b/qga/channel-posix.c
index a996858e24..25fcc5cb1a 100644
--- a/qga/channel-posix.c
+++ b/qga/channel-posix.c
@@ -119,7 +119,7 @@ static int ga_channel_client_add(GAChannel *c, int fd)
 }
 
 static gboolean ga_channel_open(GAChannel *c, const gchar *path,
-                                GAChannelMethod method, int fd)
+                                GAChannelMethod method, int fd, Error **errp)
 {
     int ret;
     c->method = method;
@@ -133,21 +133,20 @@ static gboolean ga_channel_open(GAChannel *c, const gchar *path,
 #endif
                            );
         if (fd == -1) {
-            g_critical("error opening channel: %s", strerror(errno));
+            error_setg_errno(errp, errno, "error opening channel");
             return false;
         }
 #ifdef CONFIG_SOLARIS
         ret = ioctl(fd, I_SETSIG, S_OUTPUT | S_INPUT | S_HIPRI);
         if (ret == -1) {
-            g_critical("error setting event mask for channel: %s",
-                       strerror(errno));
+            error_setg_errno(errp, errno, "error setting event mask for channel");
             close(fd);
             return false;
         }
 #endif
         ret = ga_channel_client_add(c, fd);
         if (ret) {
-            g_critical("error adding channel to main loop");
+            error_setg(errp, "error adding channel to main loop");
             close(fd);
             return false;
         }
@@ -159,7 +158,7 @@ static gboolean ga_channel_open(GAChannel *c, const gchar *path,
         assert(fd < 0);
         fd = qemu_open_old(path, O_RDWR | O_NOCTTY | O_NONBLOCK);
         if (fd == -1) {
-            g_critical("error opening channel: %s", strerror(errno));
+            error_setg_errno(errp, errno, "error opening channel");
             return false;
         }
         tcgetattr(fd, &tio);
@@ -180,7 +179,7 @@ static gboolean ga_channel_open(GAChannel *c, const gchar *path,
         tcsetattr(fd, TCSANOW, &tio);
         ret = ga_channel_client_add(c, fd);
         if (ret) {
-            g_critical("error adding channel to main loop");
+            error_setg(errp, "error adding channel to main loop");
             close(fd);
             return false;
         }
@@ -188,12 +187,8 @@ static gboolean ga_channel_open(GAChannel *c, const gchar *path,
     }
     case GA_CHANNEL_UNIX_LISTEN: {
         if (fd < 0) {
-            Error *local_err = NULL;
-
-            fd = unix_listen(path, &local_err);
-            if (local_err != NULL) {
-                g_critical("%s", error_get_pretty(local_err));
-                error_free(local_err);
+            fd = unix_listen(path, errp);
+            if (fd < 0) {
                 return false;
             }
         }
@@ -202,24 +197,19 @@ static gboolean ga_channel_open(GAChannel *c, const gchar *path,
     }
     case GA_CHANNEL_VSOCK_LISTEN: {
         if (fd < 0) {
-            Error *local_err = NULL;
             SocketAddress *addr;
             char *addr_str;
 
             addr_str = g_strdup_printf("vsock:%s", path);
-            addr = socket_parse(addr_str, &local_err);
+            addr = socket_parse(addr_str, errp);
             g_free(addr_str);
-            if (local_err != NULL) {
-                g_critical("%s", error_get_pretty(local_err));
-                error_free(local_err);
+            if (!addr) {
                 return false;
             }
 
-            fd = socket_listen(addr, 1, &local_err);
+            fd = socket_listen(addr, 1, errp);
             qapi_free_SocketAddress(addr);
-            if (local_err != NULL) {
-                g_critical("%s", error_get_pretty(local_err));
-                error_free(local_err);
+            if (fd < 0) {
                 return false;
             }
         }
@@ -227,7 +217,7 @@ static gboolean ga_channel_open(GAChannel *c, const gchar *path,
         break;
     }
     default:
-        g_critical("error binding/listening to specified socket");
+        error_setg(errp, "error binding/listening to specified socket");
         return false;
     }
 
@@ -272,12 +262,14 @@ GIOStatus ga_channel_read(GAChannel *c, gchar *buf, gsize size, gsize *count)
 GAChannel *ga_channel_new(GAChannelMethod method, const gchar *path,
                           int listen_fd, GAChannelCallback cb, gpointer opaque)
 {
+    Error *err = NULL;
     GAChannel *c = g_new0(GAChannel, 1);
     c->event_cb = cb;
     c->user_data = opaque;
 
-    if (!ga_channel_open(c, path, method, listen_fd)) {
-        g_critical("error opening channel");
+    if (!ga_channel_open(c, path, method, listen_fd, &err)) {
+        g_critical("%s", error_get_pretty(err));
+        error_free(err);
         ga_channel_free(c);
         return NULL;
     }

From b9947c9ce7eec9f31c9adfea2cd1f42e0bfe76b8 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Wed, 25 May 2022 16:41:33 +0200
Subject: [PATCH 573/935] qga: replace qemu_open_old() with qga_open_cloexec()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

qemu_open_old() uses qemu_open_internal() which handles special
"/dev/fdset/" path for monitor fd sets, set CLOEXEC, and uses Error
reporting (and some O_DIRECT special error casing).

The monitor fdset handling is unnecessary for qga, use
qga_open_cloexec() instead.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Konstantin Kostiuk <kkostiuk@redhat.com>
Message-Id: <20220525144140.591926-9-marcandre.lureau@redhat.com>
---
 qga/channel-posix.c  | 13 +++++++++----
 qga/commands-posix.c |  8 ++++----
 2 files changed, 13 insertions(+), 8 deletions(-)

diff --git a/qga/channel-posix.c b/qga/channel-posix.c
index 25fcc5cb1a..6796a02cff 100644
--- a/qga/channel-posix.c
+++ b/qga/channel-posix.c
@@ -1,8 +1,10 @@
 #include "qemu/osdep.h"
+#include "qemu/cutils.h"
 #include <termios.h>
 #include "qapi/error.h"
 #include "qemu/sockets.h"
 #include "channel.h"
+#include "cutils.h"
 
 #ifdef CONFIG_SOLARIS
 #include <stropts.h>
@@ -127,11 +129,14 @@ static gboolean ga_channel_open(GAChannel *c, const gchar *path,
     switch (c->method) {
     case GA_CHANNEL_VIRTIO_SERIAL: {
         assert(fd < 0);
-        fd = qemu_open_old(path, O_RDWR | O_NONBLOCK
+        fd = qga_open_cloexec(
+            path,
 #ifndef CONFIG_SOLARIS
-                           | O_ASYNC
+            O_ASYNC |
 #endif
-                           );
+            O_RDWR | O_NONBLOCK,
+            0
+        );
         if (fd == -1) {
             error_setg_errno(errp, errno, "error opening channel");
             return false;
@@ -156,7 +161,7 @@ static gboolean ga_channel_open(GAChannel *c, const gchar *path,
         struct termios tio;
 
         assert(fd < 0);
-        fd = qemu_open_old(path, O_RDWR | O_NOCTTY | O_NONBLOCK);
+        fd = qga_open_cloexec(path, O_RDWR | O_NOCTTY | O_NONBLOCK, 0);
         if (fd == -1) {
             error_setg_errno(errp, errno, "error opening channel");
             return false;
diff --git a/qga/commands-posix.c b/qga/commands-posix.c
index 2ecc43eca9..0047245273 100644
--- a/qga/commands-posix.c
+++ b/qga/commands-posix.c
@@ -1406,7 +1406,7 @@ static void get_nvme_smart(GuestDiskInfo *disk)
                  | (((sizeof(log) >> 2) - 1) << 16)
     };
 
-    fd = qemu_open_old(disk->name, O_RDONLY);
+    fd = qga_open_cloexec(disk->name, O_RDONLY, 0);
     if (fd == -1) {
         g_debug("Failed to open device: %s: %s", disk->name, g_strerror(errno));
         return;
@@ -1739,7 +1739,7 @@ int64_t qmp_guest_fsfreeze_freeze_list(bool has_mountpoints,
             }
         }
 
-        fd = qemu_open_old(mount->dirname, O_RDONLY);
+        fd = qga_open_cloexec(mount->dirname, O_RDONLY, 0);
         if (fd == -1) {
             error_setg_errno(errp, errno, "failed to open %s", mount->dirname);
             goto error;
@@ -1806,7 +1806,7 @@ int64_t qmp_guest_fsfreeze_thaw(Error **errp)
 
     QTAILQ_FOREACH(mount, &mounts, next) {
         logged = false;
-        fd = qemu_open_old(mount->dirname, O_RDONLY);
+        fd = qga_open_cloexec(mount->dirname, O_RDONLY, 0);
         if (fd == -1) {
             continue;
         }
@@ -1892,7 +1892,7 @@ qmp_guest_fstrim(bool has_minimum, int64_t minimum, Error **errp)
 
         QAPI_LIST_PREPEND(response->paths, result);
 
-        fd = qemu_open_old(mount->dirname, O_RDONLY);
+        fd = qga_open_cloexec(mount->dirname, O_RDONLY, 0);
         if (fd == -1) {
             result->error = g_strdup_printf("failed to open: %s",
                                             strerror(errno));

From 561bfcb69dbd3ad055f9f7ff53f04213c09b51e3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Wed, 25 May 2022 16:41:34 +0200
Subject: [PATCH 574/935] qga: make build_fs_mount_list() return a bool
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Change build_fs_mount_list() to return bool, in accordance
with the guidance under = Rules = in include/qapi/error.h

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Suggested-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220525144140.591926-10-marcandre.lureau@redhat.com>
---
 qga/commands-posix.c | 25 ++++++++++---------------
 1 file changed, 10 insertions(+), 15 deletions(-)

diff --git a/qga/commands-posix.c b/qga/commands-posix.c
index 0047245273..0469dc409d 100644
--- a/qga/commands-posix.c
+++ b/qga/commands-posix.c
@@ -673,7 +673,7 @@ static int dev_major_minor(const char *devpath,
 /*
  * Walk the mount table and build a list of local file systems
  */
-static void build_fs_mount_list_from_mtab(FsMountList *mounts, Error **errp)
+static bool build_fs_mount_list_from_mtab(FsMountList *mounts, Error **errp)
 {
     struct mntent *ment;
     FsMount *mount;
@@ -684,7 +684,7 @@ static void build_fs_mount_list_from_mtab(FsMountList *mounts, Error **errp)
     fp = setmntent(mtab, "r");
     if (!fp) {
         error_setg(errp, "failed to open mtab file: '%s'", mtab);
-        return;
+        return false;
     }
 
     while ((ment = getmntent(fp))) {
@@ -714,6 +714,7 @@ static void build_fs_mount_list_from_mtab(FsMountList *mounts, Error **errp)
     }
 
     endmntent(fp);
+    return true;
 }
 
 static void decode_mntname(char *name, int len)
@@ -738,7 +739,7 @@ static void decode_mntname(char *name, int len)
     }
 }
 
-static void build_fs_mount_list(FsMountList *mounts, Error **errp)
+static bool build_fs_mount_list(FsMountList *mounts, Error **errp)
 {
     FsMount *mount;
     char const *mountinfo = "/proc/self/mountinfo";
@@ -751,8 +752,7 @@ static void build_fs_mount_list(FsMountList *mounts, Error **errp)
 
     fp = fopen(mountinfo, "r");
     if (!fp) {
-        build_fs_mount_list_from_mtab(mounts, errp);
-        return;
+        return build_fs_mount_list_from_mtab(mounts, errp);
     }
 
     while (getline(&line, &n, fp) != -1) {
@@ -794,6 +794,7 @@ static void build_fs_mount_list(FsMountList *mounts, Error **errp)
     free(line);
 
     fclose(fp);
+    return true;
 }
 #endif
 
@@ -1592,8 +1593,7 @@ GuestFilesystemInfoList *qmp_guest_get_fsinfo(Error **errp)
     Error *local_err = NULL;
 
     QTAILQ_INIT(&mounts);
-    build_fs_mount_list(&mounts, &local_err);
-    if (local_err) {
+    if (!build_fs_mount_list(&mounts, &local_err)) {
         error_propagate(errp, local_err);
         return NULL;
     }
@@ -1716,8 +1716,7 @@ int64_t qmp_guest_fsfreeze_freeze_list(bool has_mountpoints,
     }
 
     QTAILQ_INIT(&mounts);
-    build_fs_mount_list(&mounts, &local_err);
-    if (local_err) {
+    if (!build_fs_mount_list(&mounts, &local_err)) {
         error_propagate(errp, local_err);
         return -1;
     }
@@ -1798,8 +1797,7 @@ int64_t qmp_guest_fsfreeze_thaw(Error **errp)
     Error *local_err = NULL;
 
     QTAILQ_INIT(&mounts);
-    build_fs_mount_list(&mounts, &local_err);
-    if (local_err) {
+    if (!build_fs_mount_list(&mounts, &local_err)) {
         error_propagate(errp, local_err);
         return 0;
     }
@@ -1872,15 +1870,12 @@ qmp_guest_fstrim(bool has_minimum, int64_t minimum, Error **errp)
     FsMountList mounts;
     struct FsMount *mount;
     int fd;
-    Error *local_err = NULL;
     struct fstrim_range r;
 
     slog("guest-fstrim called");
 
     QTAILQ_INIT(&mounts);
-    build_fs_mount_list(&mounts, &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
+    if (!build_fs_mount_list(&mounts, errp)) {
         return NULL;
     }
 

From a85d09269bb1a7071d3ce0f2957e3ca9dba7c047 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Wed, 25 May 2022 16:41:35 +0200
Subject: [PATCH 575/935] test/qga: use G_TEST_DIR to locate os-release test
 file
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This a more accurate way to lookup the test data, and will allow to move
the test in a subproject.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Konstantin Kostiuk <kkostiuk@redhat.com>
Message-Id: <20220525144140.591926-11-marcandre.lureau@redhat.com>
---
 tests/unit/test-qga.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/tests/unit/test-qga.c b/tests/unit/test-qga.c
index d6df1ee92e..ab0b12a2dd 100644
--- a/tests/unit/test-qga.c
+++ b/tests/unit/test-qga.c
@@ -914,15 +914,14 @@ static void test_qga_guest_get_osinfo(gconstpointer data)
 {
     TestFixture fixture;
     const gchar *str;
-    gchar *cwd, *env[2];
-    QDict *ret, *val;
+    QDict *ret = NULL;
+    char *env[2];
+    QDict *val;
 
-    cwd = g_get_current_dir();
     env[0] = g_strdup_printf(
-        "QGA_OS_RELEASE=%s%ctests%cdata%ctest-qga-os-release",
-        cwd, G_DIR_SEPARATOR, G_DIR_SEPARATOR, G_DIR_SEPARATOR);
+        "QGA_OS_RELEASE=%s%c..%cdata%ctest-qga-os-release",
+        g_test_get_dir(G_TEST_DIST), G_DIR_SEPARATOR, G_DIR_SEPARATOR, G_DIR_SEPARATOR);
     env[1] = NULL;
-    g_free(cwd);
     fixture_setup(&fixture, NULL, env);
 
     ret = qmp_fd(fixture.fd, "{'execute': 'guest-get-osinfo'}");

From 4f3c5778a9643c4026be47f161e63112a83db265 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Wed, 25 May 2022 16:41:36 +0200
Subject: [PATCH 576/935] qga/wixl: prefer variables over environment
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

No need to setup an environment or to check if the variable is undefined
manually.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Konstantin Kostiuk <kkostiuk@redhat.com>
Message-Id: <20220525144140.591926-12-marcandre.lureau@redhat.com>
---
 qga/installer/qemu-ga.wxs | 30 +++++++++---------------------
 qga/meson.build           |  9 ++++-----
 2 files changed, 13 insertions(+), 26 deletions(-)

diff --git a/qga/installer/qemu-ga.wxs b/qga/installer/qemu-ga.wxs
index 0950e8c6be..8a19aa1656 100644
--- a/qga/installer/qemu-ga.wxs
+++ b/qga/installer/qemu-ga.wxs
@@ -1,17 +1,5 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <Wix xmlns="http://schemas.microsoft.com/wix/2006/wi">
-  <?ifndef env.QEMU_GA_VERSION ?>
-    <?error Environment variable QEMU_GA_VERSION undefined?>
-  <?endif?>
-
-  <?ifndef env.QEMU_GA_DISTRO ?>
-    <?error Environment variable QEMU_GA_DISTRO undefined?>
-  <?endif?>
-
-  <?ifndef env.QEMU_GA_MANUFACTURER ?>
-    <?error Environment variable QEMU_GA_MANUFACTURER undefined?>
-  <?endif?>
-
   <?ifndef var.Arch?>
     <?error Define Arch to 32 or 64?>
   <?endif?>
@@ -43,20 +31,20 @@
     Name="QEMU guest agent"
     Id="*"
     UpgradeCode="{EB6B8302-C06E-4BEC-ADAC-932C68A3A98D}"
-    Manufacturer="$(env.QEMU_GA_MANUFACTURER)"
-    Version="$(env.QEMU_GA_VERSION)"
+    Manufacturer="$(var.QEMU_GA_MANUFACTURER)"
+    Version="$(var.QEMU_GA_VERSION)"
     Language="1033">
     <?if $(var.Arch) = 32 ?>
     <Condition Message="Error: 32-bit version of Qemu GA can not be installed on 64-bit Windows.">NOT VersionNT64</Condition>
     <?endif?>
     <Package
-      Manufacturer="$(env.QEMU_GA_MANUFACTURER)"
+      Manufacturer="$(var.QEMU_GA_MANUFACTURER)"
       InstallerVersion="200"
       Languages="1033"
       Compressed="yes"
       InstallScope="perMachine"
       />
-    <Media Id="1" Cabinet="qemu_ga.$(env.QEMU_GA_VERSION).cab" EmbedCab="yes" />
+    <Media Id="1" Cabinet="qemu_ga.$(var.QEMU_GA_VERSION).cab" EmbedCab="yes" />
     <Property Id="WHSLogo">1</Property>
     <MajorUpgrade
       DowngradeErrorMessage="Error: A newer version of QEMU guest agent is already installed."
@@ -66,7 +54,7 @@
       <Directory Id="$(var.GaProgramFilesFolder)" Name="QEMU Guest Agent">
         <Directory Id="qemu_ga_directory" Name="Qemu-ga">
           <Component Id="qemu_ga" Guid="{908B7199-DE2A-4DC6-A8D0-27A5AE444FEA}">
-            <File Id="qemu_ga.exe" Name="qemu-ga.exe" Source="$(env.BUILD_DIR)/qga/qemu-ga.exe" KeyPath="yes" DiskId="1"/>
+            <File Id="qemu_ga.exe" Name="qemu-ga.exe" Source="$(var.BUILD_DIR)/qga/qemu-ga.exe" KeyPath="yes" DiskId="1"/>
             <ServiceInstall
               Id="ServiceInstaller"
               Type="ownProcess"
@@ -88,10 +76,10 @@
             <File Id="libstdc++-6.lib" Name="libstdc++-6.dll" Source="$(var.Mingw_bin)/libstdc++-6.dll" KeyPath="yes" DiskId="1"/>
           </Component>
           <Component Id="qga_vss_dll" Guid="{CB19C453-FABB-4BB1-ABAB-6B74F687BFBB}">
-            <File Id="qga_vss.dll" Name="qga-vss.dll" Source="$(env.BUILD_DIR)/qga/vss-win32/qga-vss.dll" KeyPath="yes" DiskId="1"/>
+            <File Id="qga_vss.dll" Name="qga-vss.dll" Source="$(var.BUILD_DIR)/qga/vss-win32/qga-vss.dll" KeyPath="yes" DiskId="1"/>
           </Component>
           <Component Id="qga_vss_tlb" Guid="{D8D584B1-59C2-4FB7-A91F-636FF7BFA66E}">
-            <File Id="qga_vss.tlb" Name="qga-vss.tlb" Source="$(env.BUILD_DIR)/qga/vss-win32/qga-vss.tlb" KeyPath="yes" DiskId="1"/>
+            <File Id="qga_vss.tlb" Name="qga-vss.tlb" Source="$(var.BUILD_DIR)/qga/vss-win32/qga-vss.tlb" KeyPath="yes" DiskId="1"/>
           </Component>
           <?endif?>
           <?if $(var.Arch) = "32"?>
@@ -133,9 +121,9 @@
           </Component>
           <Component Id="registry_entries" Guid="{D075D109-51CA-11E3-9F8B-000C29858960}">
             <RegistryKey Root="HKLM"
-                         Key="Software\$(env.QEMU_GA_MANUFACTURER)\$(env.QEMU_GA_DISTRO)\Tools\QemuGA">
+                         Key="Software\$(var.QEMU_GA_MANUFACTURER)\$(var.QEMU_GA_DISTRO)\Tools\QemuGA">
               <RegistryValue Type="string" Name="ProductID" Value="fb0a0d66-c7fb-4e2e-a16b-c4a3bfe8d13b" />
-              <RegistryValue Type="string" Name="Version" Value="$(env.QEMU_GA_VERSION)" />
+              <RegistryValue Type="string" Name="Version" Value="$(var.QEMU_GA_VERSION)" />
             </RegistryKey>
           </Component>
         </Directory>
diff --git a/qga/meson.build b/qga/meson.build
index 35fe2229e9..31370405f9 100644
--- a/qga/meson.build
+++ b/qga/meson.build
@@ -122,15 +122,14 @@ if targetos == 'windows'
                             output: 'qemu-ga-@0@.msi'.format(host_arch),
                             depends: deps,
                             command: [
-                              find_program('env'),
-                              'QEMU_GA_VERSION=' + config_host['QEMU_GA_VERSION'],
-                              'QEMU_GA_MANUFACTURER=' + config_host['QEMU_GA_MANUFACTURER'],
-                              'QEMU_GA_DISTRO=' + config_host['QEMU_GA_DISTRO'],
-                              'BUILD_DIR=' + meson.build_root(),
                               wixl, '-o', '@OUTPUT0@', '@INPUT0@',
                               qemu_ga_msi_arch[cpu],
                               qemu_ga_msi_vss,
+                              '-D', 'BUILD_DIR=' + meson.build_root(),
                               '-D', 'Mingw_bin=' + config_host['QEMU_GA_MSI_MINGW_BIN_PATH'],
+                              '-D', 'QEMU_GA_VERSION=' + config_host['QEMU_GA_VERSION'],
+                              '-D', 'QEMU_GA_MANUFACTURER=' + config_host['QEMU_GA_MANUFACTURER'],
+                              '-D', 'QEMU_GA_DISTRO=' + config_host['QEMU_GA_DISTRO'],
                             ])
     all_qga += [qga_msi]
     alias_target('msi', qga_msi)

From be2db8368f84d9053baa8388f037b7ff11035787 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Wed, 25 May 2022 16:41:37 +0200
Subject: [PATCH 577/935] qga/wixl: require Mingw_bin
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

No clear reason to make guesses here.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Konstantin Kostiuk <kkostiuk@redhat.com>
Message-Id: <20220525144140.591926-13-marcandre.lureau@redhat.com>
---
 qga/installer/qemu-ga.wxs | 9 ---------
 1 file changed, 9 deletions(-)

diff --git a/qga/installer/qemu-ga.wxs b/qga/installer/qemu-ga.wxs
index 8a19aa1656..651db6e51c 100644
--- a/qga/installer/qemu-ga.wxs
+++ b/qga/installer/qemu-ga.wxs
@@ -4,15 +4,6 @@
     <?error Define Arch to 32 or 64?>
   <?endif?>
 
-  <?ifndef var.Mingw_bin?>
-    <?if $(var.Arch) = "64"?>
-      <?define Mingw_bin=/usr/x86_64-w64-mingw32/sys-root/mingw/bin ?>
-    <?endif?>
-    <?if $(var.Arch) = "32"?>
-      <?define Mingw_bin=/usr/i686-w64-mingw32/sys-root/mingw/bin ?>
-    <?endif?>
-  <?endif?>
-
   <?if $(var.Arch) = "64"?>
     <?define ArchLib=libgcc_s_seh-1.dll?>
     <?define GaProgramFilesFolder="ProgramFiles64Folder" ?>

From 0480a1b67b22218637ab299fb0ea7df2ff8287ce Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Wed, 25 May 2022 16:41:38 +0200
Subject: [PATCH 578/935] qga/wixl: simplify some pre-processing
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Sadly, wixl doesn't have 'elif'.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Konstantin Kostiuk <kkostiuk@redhat.com>
Message-Id: <20220525144140.591926-14-marcandre.lureau@redhat.com>
---
 qga/installer/qemu-ga.wxs | 20 +++++++-------------
 1 file changed, 7 insertions(+), 13 deletions(-)

diff --git a/qga/installer/qemu-ga.wxs b/qga/installer/qemu-ga.wxs
index 651db6e51c..e5b0958e18 100644
--- a/qga/installer/qemu-ga.wxs
+++ b/qga/installer/qemu-ga.wxs
@@ -1,21 +1,15 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <Wix xmlns="http://schemas.microsoft.com/wix/2006/wi">
-  <?ifndef var.Arch?>
-    <?error Define Arch to 32 or 64?>
-  <?endif?>
-
   <?if $(var.Arch) = "64"?>
     <?define ArchLib=libgcc_s_seh-1.dll?>
     <?define GaProgramFilesFolder="ProgramFiles64Folder" ?>
-  <?endif?>
-
-  <?if $(var.Arch) = "32"?>
-    <?define ArchLib=libgcc_s_dw2-1.dll?>
-    <?define GaProgramFilesFolder="ProgramFilesFolder" ?>
-  <?endif?>
-
-  <?ifndef var.ArchLib ?>
-    <?error Unexpected Arch value $(var.Arch)?>
+  <?else?>
+    <?if $(var.Arch) = "32"?>
+      <?define ArchLib=libgcc_s_dw2-1.dll?>
+      <?define GaProgramFilesFolder="ProgramFilesFolder" ?>
+    <?else?>
+      <?error Unexpected Arch value $(var.Arch)?>
+    <?endif?>
   <?endif?>
 
   <Product

From 5b9e7d05d63372540139ba70a8f7c3699d7751b6 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Wed, 25 May 2022 16:41:39 +0200
Subject: [PATCH 579/935] qga/wixl: replace QEMU_GA_MSI_MINGW_BIN_PATH with
 glib bindir
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Use more conventional variables to set the location of pre-built
DLL/bin.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Konstantin Kostiuk <kkostiuk@redhat.com>
Message-Id: <20220525144140.591926-15-marcandre.lureau@redhat.com>
---
 configure                 |  9 ++++++---
 meson.build               |  5 ++++-
 qga/installer/qemu-ga.wxs | 24 ++++++++++++------------
 qga/meson.build           |  2 +-
 4 files changed, 23 insertions(+), 17 deletions(-)

diff --git a/configure b/configure
index 180ee688dc..f2baf2f526 100755
--- a/configure
+++ b/configure
@@ -1495,6 +1495,11 @@ for i in $glib_modules; do
     fi
 done
 
+glib_bindir="$($pkg_config --variable=bindir glib-2.0)"
+if test -z "$glib_bindir" ; then
+	glib_bindir="$($pkg_config --variable=prefix glib-2.0)"/bin
+fi
+
 # This workaround is required due to a bug in pkg-config file for glib as it
 # doesn't define GLIB_STATIC_COMPILATION for pkg-config --static
 
@@ -1860,8 +1865,6 @@ if test "$QEMU_GA_VERSION" = ""; then
     QEMU_GA_VERSION=$(cat $source_path/VERSION)
 fi
 
-QEMU_GA_MSI_MINGW_BIN_PATH="$($pkg_config --variable=prefix glib-2.0)/bin"
-
 # Mac OS X ships with a broken assembler
 roms=
 if { test "$cpu" = "i386" || test "$cpu" = "x86_64"; } && \
@@ -1948,7 +1951,6 @@ if test "$debug_tcg" = "yes" ; then
 fi
 if test "$mingw32" = "yes" ; then
   echo "CONFIG_WIN32=y" >> $config_host_mak
-  echo "QEMU_GA_MSI_MINGW_BIN_PATH=${QEMU_GA_MSI_MINGW_BIN_PATH}" >> $config_host_mak
   echo "QEMU_GA_MANUFACTURER=${QEMU_GA_MANUFACTURER}" >> $config_host_mak
   echo "QEMU_GA_DISTRO=${QEMU_GA_DISTRO}" >> $config_host_mak
   echo "QEMU_GA_VERSION=${QEMU_GA_VERSION}" >> $config_host_mak
@@ -2020,6 +2022,7 @@ echo "QEMU_CXXFLAGS=$QEMU_CXXFLAGS" >> $config_host_mak
 echo "QEMU_OBJCFLAGS=$QEMU_OBJCFLAGS" >> $config_host_mak
 echo "GLIB_CFLAGS=$glib_cflags" >> $config_host_mak
 echo "GLIB_LIBS=$glib_libs" >> $config_host_mak
+echo "GLIB_BINDIR=$glib_bindir" >> $config_host_mak
 echo "GLIB_VERSION=$(pkg-config --modversion glib-2.0)" >> $config_host_mak
 echo "QEMU_LDFLAGS=$QEMU_LDFLAGS" >> $config_host_mak
 echo "LD_I386_EMULATION=$ld_i386_emulation" >> $config_host_mak
diff --git a/meson.build b/meson.build
index df7c34b076..bf318d9cbb 100644
--- a/meson.build
+++ b/meson.build
@@ -466,7 +466,10 @@ add_project_arguments(config_host['GLIB_CFLAGS'].split(),
                       native: false, language: ['c', 'cpp', 'objc'])
 glib = declare_dependency(compile_args: config_host['GLIB_CFLAGS'].split(),
                           link_args: config_host['GLIB_LIBS'].split(),
-                          version: config_host['GLIB_VERSION'])
+                          version: config_host['GLIB_VERSION'],
+                          variables: {
+                            'bindir': config_host['GLIB_BINDIR'],
+                          })
 # override glib dep with the configure results (for subprojects)
 meson.override_dependency('glib-2.0', glib)
 
diff --git a/qga/installer/qemu-ga.wxs b/qga/installer/qemu-ga.wxs
index e5b0958e18..813d1c6ca6 100644
--- a/qga/installer/qemu-ga.wxs
+++ b/qga/installer/qemu-ga.wxs
@@ -58,7 +58,7 @@
           </Component>
           <?ifdef var.InstallVss?>
           <Component Id="libstdc++_6_lib" Guid="{55E737B5-9127-4A11-9FC3-A29367714574}">
-            <File Id="libstdc++-6.lib" Name="libstdc++-6.dll" Source="$(var.Mingw_bin)/libstdc++-6.dll" KeyPath="yes" DiskId="1"/>
+            <File Id="libstdc++-6.lib" Name="libstdc++-6.dll" Source="$(var.BIN_DIR)/libstdc++-6.dll" KeyPath="yes" DiskId="1"/>
           </Component>
           <Component Id="qga_vss_dll" Guid="{CB19C453-FABB-4BB1-ABAB-6B74F687BFBB}">
             <File Id="qga_vss.dll" Name="qga-vss.dll" Source="$(var.BUILD_DIR)/qga/vss-win32/qga-vss.dll" KeyPath="yes" DiskId="1"/>
@@ -69,40 +69,40 @@
           <?endif?>
           <?if $(var.Arch) = "32"?>
           <Component Id="gspawn-helper-console" Guid="{446185B3-87BE-43D2-96B8-0FEFD9E8696D}">
-            <File Id="gspawn-win32-helper-console.exe" Name="gspawn-win32-helper-console.exe" Source="$(var.Mingw_bin)/gspawn-win32-helper-console.exe" KeyPath="yes" DiskId="1"/>
+            <File Id="gspawn-win32-helper-console.exe" Name="gspawn-win32-helper-console.exe" Source="$(var.BIN_DIR)/gspawn-win32-helper-console.exe" KeyPath="yes" DiskId="1"/>
           </Component>
           <Component Id="gspawn-helper" Guid="{CD67A5A3-2DB1-4DA1-A67A-8D71E797B466}">
-            <File Id="gspawn-win32-helper.exe" Name="gspawn-win32-helper.exe" Source="$(var.Mingw_bin)/gspawn-win32-helper-console.exe" KeyPath="yes" DiskId="1"/>
+            <File Id="gspawn-win32-helper.exe" Name="gspawn-win32-helper.exe" Source="$(var.BIN_DIR)/gspawn-win32-helper-console.exe" KeyPath="yes" DiskId="1"/>
           </Component>
           <?endif?>
           <?if $(var.Arch) = "64"?>
           <Component Id="gspawn-helper-console" Guid="{9E615A9F-349A-4992-A5C2-C10BAD173660}">
-            <File Id="gspawn-win64-helper-console.exe" Name="gspawn-win64-helper-console.exe" Source="$(var.Mingw_bin)/gspawn-win64-helper-console.exe" KeyPath="yes" DiskId="1"/>
+            <File Id="gspawn-win64-helper-console.exe" Name="gspawn-win64-helper-console.exe" Source="$(var.BIN_DIR)/gspawn-win64-helper-console.exe" KeyPath="yes" DiskId="1"/>
           </Component>
           <Component Id="gspawn-helper" Guid="{D201AD22-1846-4E4F-B6E1-C7A908ED2457}">
-            <File Id="gspawn-win64-helper.exe" Name="gspawn-win64-helper.exe" Source="$(var.Mingw_bin)/gspawn-win64-helper-console.exe" KeyPath="yes" DiskId="1"/>
+            <File Id="gspawn-win64-helper.exe" Name="gspawn-win64-helper.exe" Source="$(var.BIN_DIR)/gspawn-win64-helper-console.exe" KeyPath="yes" DiskId="1"/>
           </Component>
           <?endif?>
           <Component Id="iconv" Guid="{35EE3558-D34B-4F0A-B8BD-430FF0775246}">
-            <File Id="iconv.dll" Name="iconv.dll" Source="$(var.Mingw_bin)/iconv.dll" KeyPath="yes" DiskId="1"/>
+            <File Id="iconv.dll" Name="iconv.dll" Source="$(var.BIN_DIR)/iconv.dll" KeyPath="yes" DiskId="1"/>
           </Component>
           <Component Id="libgcc_arch_lib" Guid="{ADD4D07D-4515-4AB6-AF3E-C904961B4BB0}">
-            <File Id="libgcc_arch_lib" Name="$(var.ArchLib)" Source="$(var.Mingw_bin)/$(var.ArchLib)" KeyPath="yes" DiskId="1"/>
+            <File Id="libgcc_arch_lib" Name="$(var.ArchLib)" Source="$(var.BIN_DIR)/$(var.ArchLib)" KeyPath="yes" DiskId="1"/>
           </Component>
           <Component Id="libglib" Guid="{D31BFD83-2773-4B65-B45A-E0D2ADA58679}">
-            <File Id="libglib_2.0_0.dll" Name="libglib-2.0-0.dll" Source="$(var.Mingw_bin)/libglib-2.0-0.dll" KeyPath="yes" DiskId="1"/>
+            <File Id="libglib_2.0_0.dll" Name="libglib-2.0-0.dll" Source="$(var.BIN_DIR)/libglib-2.0-0.dll" KeyPath="yes" DiskId="1"/>
           </Component>
           <Component Id="libintl" Guid="{A641BC2D-A907-4A94-9149-F30ED430878F}">
-            <File Id="libintl_8.dll" Name="libintl-8.dll" Source="$(var.Mingw_bin)/libintl-8.dll" KeyPath="yes" DiskId="1"/>
+            <File Id="libintl_8.dll" Name="libintl-8.dll" Source="$(var.BIN_DIR)/libintl-8.dll" KeyPath="yes" DiskId="1"/>
           </Component>
           <Component Id="libssp" Guid="{7880087B-02B4-4EF6-A5D3-D18F8E3D90E1}">
-            <File Id="libssp_0.dll" Name="libssp-0.dll" Source="$(var.Mingw_bin)/libssp-0.dll" KeyPath="yes" DiskId="1"/>
+            <File Id="libssp_0.dll" Name="libssp-0.dll" Source="$(var.BIN_DIR)/libssp-0.dll" KeyPath="yes" DiskId="1"/>
           </Component>
           <Component Id="libwinpthread" Guid="{6C117C78-0F47-4B07-8F34-6BEE11643829}">
-            <File Id="libwinpthread_1.dll" Name="libwinpthread-1.dll" Source="$(var.Mingw_bin)/libwinpthread-1.dll" KeyPath="yes" DiskId="1"/>
+            <File Id="libwinpthread_1.dll" Name="libwinpthread-1.dll" Source="$(var.BIN_DIR)/libwinpthread-1.dll" KeyPath="yes" DiskId="1"/>
           </Component>
           <Component Id="libpcre" Guid="{7A86B45E-A009-489A-A849-CE3BACF03CD0}">
-            <File Id="libpcre_1.dll" Name="libpcre-1.dll" Source="$(var.Mingw_bin)/libpcre-1.dll" KeyPath="yes" DiskId="1"/>
+            <File Id="libpcre_1.dll" Name="libpcre-1.dll" Source="$(var.BIN_DIR)/libpcre-1.dll" KeyPath="yes" DiskId="1"/>
           </Component>
           <Component Id="registry_entries" Guid="{D075D109-51CA-11E3-9F8B-000C29858960}">
             <RegistryKey Root="HKLM"
diff --git a/qga/meson.build b/qga/meson.build
index 31370405f9..619ff095bc 100644
--- a/qga/meson.build
+++ b/qga/meson.build
@@ -126,7 +126,7 @@ if targetos == 'windows'
                               qemu_ga_msi_arch[cpu],
                               qemu_ga_msi_vss,
                               '-D', 'BUILD_DIR=' + meson.build_root(),
-                              '-D', 'Mingw_bin=' + config_host['QEMU_GA_MSI_MINGW_BIN_PATH'],
+                              '-D', 'BIN_DIR=' + glib.get_variable('bindir'),
                               '-D', 'QEMU_GA_VERSION=' + config_host['QEMU_GA_VERSION'],
                               '-D', 'QEMU_GA_MANUFACTURER=' + config_host['QEMU_GA_MANUFACTURER'],
                               '-D', 'QEMU_GA_DISTRO=' + config_host['QEMU_GA_DISTRO'],

From bb6960a11ae9c9d727bab45c22cb4a9eb12cde55 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Wed, 25 May 2022 16:41:40 +0200
Subject: [PATCH 580/935] test/qga: use g_auto wherever sensible
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Konstantin Kostiuk <kkostiuk@redhat.com>
Message-Id: <20220525144140.591926-16-marcandre.lureau@redhat.com>
---
 tests/unit/test-qga.c | 121 +++++++++++++++---------------------------
 1 file changed, 43 insertions(+), 78 deletions(-)

diff --git a/tests/unit/test-qga.c b/tests/unit/test-qga.c
index ab0b12a2dd..530317044b 100644
--- a/tests/unit/test-qga.c
+++ b/tests/unit/test-qga.c
@@ -52,7 +52,10 @@ fixture_setup(TestFixture *fixture, gconstpointer data, gchar **envp)
 {
     const gchar *extra_arg = data;
     GError *error = NULL;
-    gchar *cwd, *path, *cmd, **argv = NULL;
+    g_autofree char *cwd = NULL;
+    g_autofree char *path = NULL;
+    g_autofree char *cmd = NULL;
+    g_auto(GStrv) argv = NULL;
 
     fixture->loop = g_main_loop_new(NULL, FALSE);
 
@@ -78,17 +81,12 @@ fixture_setup(TestFixture *fixture, gconstpointer data, gchar **envp)
 
     fixture->fd = connect_qga(path);
     g_assert_cmpint(fixture->fd, !=, -1);
-
-    g_strfreev(argv);
-    g_free(cmd);
-    g_free(cwd);
-    g_free(path);
 }
 
 static void
 fixture_tear_down(TestFixture *fixture, gconstpointer data)
 {
-    gchar *tmp;
+    g_autofree char *tmp = NULL;
 
     kill(fixture->pid, SIGTERM);
 
@@ -107,7 +105,6 @@ fixture_tear_down(TestFixture *fixture, gconstpointer data)
 
     tmp = g_build_filename(fixture->test_dir, "sock", NULL);
     g_unlink(tmp);
-    g_free(tmp);
 
     g_rmdir(fixture->test_dir);
     g_free(fixture->test_dir);
@@ -122,7 +119,7 @@ static void qmp_assertion_message_error(const char     *domain,
                                         QDict          *dict)
 {
     const char *class, *desc;
-    char *s;
+    g_autofree char *s = NULL;
     QDict *error;
 
     error = qdict_get_qdict(dict, "error");
@@ -131,7 +128,6 @@ static void qmp_assertion_message_error(const char     *domain,
 
     s = g_strdup_printf("assertion failed %s: %s %s", expr, class, desc);
     g_assertion_message(domain, file, line, func, s);
-    g_free(s);
 }
 
 #define qmp_assert_no_error(err) do {                                   \
@@ -146,7 +142,7 @@ static void test_qga_sync_delimited(gconstpointer fix)
     const TestFixture *fixture = fix;
     guint32 v, r = g_test_rand_int();
     unsigned char c;
-    QDict *ret;
+    g_autoptr(QDict) ret = NULL;
 
     qmp_fd_send_raw(fixture->fd, "\xff");
     qmp_fd_send(fixture->fd,
@@ -180,15 +176,13 @@ static void test_qga_sync_delimited(gconstpointer fix)
 
     v = qdict_get_int(ret, "return");
     g_assert_cmpint(r, ==, v);
-
-    qobject_unref(ret);
 }
 
 static void test_qga_sync(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
     guint32 v, r = g_test_rand_int();
-    QDict *ret;
+    g_autoptr(QDict) ret = NULL;
 
     /*
      * TODO guest-sync is inherently limited: we cannot distinguish
@@ -210,33 +204,27 @@ static void test_qga_sync(gconstpointer fix)
 
     v = qdict_get_int(ret, "return");
     g_assert_cmpint(r, ==, v);
-
-    qobject_unref(ret);
 }
 
 static void test_qga_ping(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
-    QDict *ret;
+    g_autoptr(QDict) ret = NULL;
 
     ret = qmp_fd(fixture->fd, "{'execute': 'guest-ping'}");
     g_assert_nonnull(ret);
     qmp_assert_no_error(ret);
-
-    qobject_unref(ret);
 }
 
 static void test_qga_id(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
-    QDict *ret;
+    g_autoptr(QDict) ret = NULL;
 
     ret = qmp_fd(fixture->fd, "{'execute': 'guest-ping', 'id': 1}");
     g_assert_nonnull(ret);
     qmp_assert_no_error(ret);
     g_assert_cmpint(qdict_get_int(ret, "id"), ==, 1);
-
-    qobject_unref(ret);
 }
 
 static void test_qga_invalid_oob(gconstpointer fix)
@@ -253,7 +241,8 @@ static void test_qga_invalid_oob(gconstpointer fix)
 static void test_qga_invalid_args(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
-    QDict *ret, *error;
+    g_autoptr(QDict) ret = NULL;
+    QDict *error;
     const gchar *class, *desc;
 
     ret = qmp_fd(fixture->fd, "{'execute': 'guest-ping', "
@@ -266,14 +255,13 @@ static void test_qga_invalid_args(gconstpointer fix)
 
     g_assert_cmpstr(class, ==, "GenericError");
     g_assert_cmpstr(desc, ==, "Parameter 'foo' is unexpected");
-
-    qobject_unref(ret);
 }
 
 static void test_qga_invalid_cmd(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
-    QDict *ret, *error;
+    g_autoptr(QDict) ret = NULL;
+    QDict *error;
     const gchar *class, *desc;
 
     ret = qmp_fd(fixture->fd, "{'execute': 'guest-invalid-cmd'}");
@@ -285,14 +273,13 @@ static void test_qga_invalid_cmd(gconstpointer fix)
 
     g_assert_cmpstr(class, ==, "CommandNotFound");
     g_assert_cmpint(strlen(desc), >, 0);
-
-    qobject_unref(ret);
 }
 
 static void test_qga_info(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
-    QDict *ret, *val;
+    g_autoptr(QDict) ret = NULL;
+    QDict *val;
     const gchar *version;
 
     ret = qmp_fd(fixture->fd, "{'execute': 'guest-info'}");
@@ -302,14 +289,12 @@ static void test_qga_info(gconstpointer fix)
     val = qdict_get_qdict(ret, "return");
     version = qdict_get_try_str(val, "version");
     g_assert_cmpstr(version, ==, QEMU_VERSION);
-
-    qobject_unref(ret);
 }
 
 static void test_qga_get_vcpus(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
-    QDict *ret;
+    g_autoptr(QDict) ret = NULL;
     QList *list;
     const QListEntry *entry;
 
@@ -322,14 +307,12 @@ static void test_qga_get_vcpus(gconstpointer fix)
     entry = qlist_first(list);
     g_assert(qdict_haskey(qobject_to(QDict, entry->value), "online"));
     g_assert(qdict_haskey(qobject_to(QDict, entry->value), "logical-id"));
-
-    qobject_unref(ret);
 }
 
 static void test_qga_get_fsinfo(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
-    QDict *ret;
+    g_autoptr(QDict) ret = NULL;
     QList *list;
     const QListEntry *entry;
 
@@ -346,14 +329,13 @@ static void test_qga_get_fsinfo(gconstpointer fix)
         g_assert(qdict_haskey(qobject_to(QDict, entry->value), "type"));
         g_assert(qdict_haskey(qobject_to(QDict, entry->value), "disk"));
     }
-
-    qobject_unref(ret);
 }
 
 static void test_qga_get_memory_block_info(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
-    QDict *ret, *val;
+    g_autoptr(QDict) ret = NULL;
+    QDict *val;
     int64_t size;
 
     ret = qmp_fd(fixture->fd, "{'execute': 'guest-get-memory-block-info'}");
@@ -366,14 +348,12 @@ static void test_qga_get_memory_block_info(gconstpointer fix)
         size = qdict_get_int(val, "size");
         g_assert_cmpint(size, >, 0);
     }
-
-    qobject_unref(ret);
 }
 
 static void test_qga_get_memory_blocks(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
-    QDict *ret;
+    g_autoptr(QDict) ret = NULL;
     QList *list;
     const QListEntry *entry;
 
@@ -391,14 +371,12 @@ static void test_qga_get_memory_blocks(gconstpointer fix)
             g_assert(qdict_haskey(qobject_to(QDict, entry->value), "online"));
         }
     }
-
-    qobject_unref(ret);
 }
 
 static void test_qga_network_get_interfaces(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
-    QDict *ret;
+    g_autoptr(QDict) ret = NULL;
     QList *list;
     const QListEntry *entry;
 
@@ -410,8 +388,6 @@ static void test_qga_network_get_interfaces(gconstpointer fix)
     list = qdict_get_qlist(ret, "return");
     entry = qlist_first(list);
     g_assert(qdict_haskey(qobject_to(QDict, entry->value), "name"));
-
-    qobject_unref(ret);
 }
 
 static void test_qga_file_ops(gconstpointer fix)
@@ -642,7 +618,7 @@ static void test_qga_file_write_read(gconstpointer fix)
 static void test_qga_get_time(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
-    QDict *ret;
+    g_autoptr(QDict) ret = NULL;
     int64_t time;
 
     ret = qmp_fd(fixture->fd, "{'execute': 'guest-get-time'}");
@@ -651,8 +627,6 @@ static void test_qga_get_time(gconstpointer fix)
 
     time = qdict_get_int(ret, "return");
     g_assert_cmpint(time, >, 0);
-
-    qobject_unref(ret);
 }
 
 static void test_qga_blacklist(gconstpointer data)
@@ -693,18 +667,22 @@ static void test_qga_blacklist(gconstpointer data)
 static void test_qga_config(gconstpointer data)
 {
     GError *error = NULL;
-    char *cwd, *cmd, *out, *err, *str, **strv, **argv = NULL;
+    g_autofree char *out = NULL;
+    g_autofree char *err = NULL;
+    g_autofree char *cwd = NULL;
+    g_autofree char *cmd = NULL;
+    g_auto(GStrv) argv = NULL;
+    g_auto(GStrv) strv = NULL;
+    g_autoptr(GKeyFile) kf = NULL;
+    char *str;
     char *env[2];
     int status;
     gsize n;
-    GKeyFile *kf;
 
     cwd = g_get_current_dir();
     cmd = g_strdup_printf("%s%cqga%cqemu-ga -D",
                           cwd, G_DIR_SEPARATOR, G_DIR_SEPARATOR);
-    g_free(cwd);
     g_shell_parse_argv(cmd, NULL, &argv, &error);
-    g_free(cmd);
     g_assert_no_error(error);
 
     env[0] = g_strdup_printf("QGA_CONF=tests%cdata%ctest-qga-config",
@@ -712,7 +690,6 @@ static void test_qga_config(gconstpointer data)
     env[1] = NULL;
     g_spawn_sync(NULL, argv, env, 0,
                  NULL, NULL, &out, &err, &status, &error);
-    g_strfreev(argv);
 
     g_assert_no_error(error);
     g_assert_cmpstr(err, ==, "");
@@ -759,18 +736,14 @@ static void test_qga_config(gconstpointer data)
     g_assert_true(g_strv_contains((const char * const *)strv,
                                   "guest-get-time"));
     g_assert_no_error(error);
-    g_strfreev(strv);
 
-    g_free(out);
-    g_free(err);
     g_free(env[0]);
-    g_key_file_free(kf);
 }
 
 static void test_qga_fsfreeze_status(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
-    QDict *ret;
+    g_autoptr(QDict) ret = NULL;
     const gchar *status;
 
     ret = qmp_fd(fixture->fd, "{'execute': 'guest-fsfreeze-status'}");
@@ -779,16 +752,15 @@ static void test_qga_fsfreeze_status(gconstpointer fix)
 
     status = qdict_get_try_str(ret, "return");
     g_assert_cmpstr(status, ==, "thawed");
-
-    qobject_unref(ret);
 }
 
 static void test_qga_guest_exec(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
-    QDict *ret, *val;
+    g_autoptr(QDict) ret = NULL;
+    QDict *val;
     const gchar *out;
-    guchar *decoded;
+    g_autofree guchar *decoded = NULL;
     int64_t pid, now, exitcode;
     gsize len;
     bool exited;
@@ -827,14 +799,13 @@ static void test_qga_guest_exec(gconstpointer fix)
     decoded = g_base64_decode(out, &len);
     g_assert_cmpint(len, ==, 12);
     g_assert_cmpstr((char *)decoded, ==, "\" test_str \"");
-    g_free(decoded);
-    qobject_unref(ret);
 }
 
 static void test_qga_guest_exec_invalid(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
-    QDict *ret, *error;
+    g_autoptr(QDict) ret = NULL;
+    QDict *error;
     const gchar *class, *desc;
 
     /* invalid command */
@@ -859,13 +830,13 @@ static void test_qga_guest_exec_invalid(gconstpointer fix)
     desc = qdict_get_str(error, "desc");
     g_assert_cmpstr(class, ==, "GenericError");
     g_assert_cmpint(strlen(desc), >, 0);
-    qobject_unref(ret);
 }
 
 static void test_qga_guest_get_host_name(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
-    QDict *ret, *val;
+    g_autoptr(QDict) ret = NULL;
+    QDict *val;
 
     ret = qmp_fd(fixture->fd, "{'execute': 'guest-get-host-name'}");
     g_assert_nonnull(ret);
@@ -873,14 +844,13 @@ static void test_qga_guest_get_host_name(gconstpointer fix)
 
     val = qdict_get_qdict(ret, "return");
     g_assert(qdict_haskey(val, "host-name"));
-
-    qobject_unref(ret);
 }
 
 static void test_qga_guest_get_timezone(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
-    QDict *ret, *val;
+    g_autoptr(QDict) ret = NULL;
+    QDict *val;
 
     ret = qmp_fd(fixture->fd, "{'execute': 'guest-get-timezone'}");
     g_assert_nonnull(ret);
@@ -889,14 +859,12 @@ static void test_qga_guest_get_timezone(gconstpointer fix)
     /* Make sure there's at least offset */
     val = qdict_get_qdict(ret, "return");
     g_assert(qdict_haskey(val, "offset"));
-
-    qobject_unref(ret);
 }
 
 static void test_qga_guest_get_users(gconstpointer fix)
 {
     const TestFixture *fixture = fix;
-    QDict *ret;
+    g_autoptr(QDict) ret = NULL;
     QList *val;
 
     ret = qmp_fd(fixture->fd, "{'execute': 'guest-get-users'}");
@@ -906,15 +874,13 @@ static void test_qga_guest_get_users(gconstpointer fix)
     /* There is not much to test here */
     val = qdict_get_qlist(ret, "return");
     g_assert_nonnull(val);
-
-    qobject_unref(ret);
 }
 
 static void test_qga_guest_get_osinfo(gconstpointer data)
 {
     TestFixture fixture;
     const gchar *str;
-    QDict *ret = NULL;
+    g_autoptr(QDict) ret = NULL;
     char *env[2];
     QDict *val;
 
@@ -958,7 +924,6 @@ static void test_qga_guest_get_osinfo(gconstpointer data)
     g_assert_nonnull(str);
     g_assert_cmpstr(str, ==, "unit-test");
 
-    qobject_unref(ret);
     g_free(env[0]);
     fixture_tear_down(&fixture, NULL);
 }

From 4de43540644962687fcd2df2a2454c1e46c2f92a Mon Sep 17 00:00:00 2001
From: Helge Deller <deller@gmx.de>
Date: Sat, 28 May 2022 11:15:13 +0200
Subject: [PATCH 581/935] hppa: Sync contents of hppa_hardware.h header file
 with SeaBIOS-hppa

The hppa_hardware.h header file holds many constants for addresses and
offsets which are needed while building the firmware (SeaBIOS-hppa) and
while setting up the virtual machine in QEMU.

That's why this header file needs to be in sync between both source code
repositories. This patch adds a comment mentioning this dependency at
the top of this file and restores some DINO relevant offsets.

Signed-off-by: Helge Deller <deller@gmx.de>
Reviewed-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/hppa_hardware.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/hw/hppa/hppa_hardware.h b/hw/hppa/hppa_hardware.h
index 8b6b9222cb..3f7627b98f 100644
--- a/hw/hppa/hppa_hardware.h
+++ b/hw/hppa/hppa_hardware.h
@@ -1,4 +1,5 @@
 /* HPPA cores and system support chips.  */
+/* Be aware: QEMU and seabios-hppa repositories share this file as-is. */
 
 #ifndef HW_HPPA_HPPA_HARDWARE_H
 #define HW_HPPA_HPPA_HARDWARE_H
@@ -30,6 +31,11 @@
 #define PCI_HPA         DINO_HPA        /* PCI bus */
 #define IDE_HPA         0xf9000000      /* Boot disc controller */
 
+/* offsets to DINO HPA: */
+#define DINO_PCI_ADDR           0x064
+#define DINO_CONFIG_DATA        0x068
+#define DINO_IO_DATA            0x06c
+
 #define PORT_PCI_CMD    (PCI_HPA + DINO_PCI_ADDR)
 #define PORT_PCI_DATA   (PCI_HPA + DINO_CONFIG_DATA)
 

From 5079892df5f113c7f2b77f53bf7663f6c7bc6be9 Mon Sep 17 00:00:00 2001
From: Helge Deller <deller@gmx.de>
Date: Sat, 28 May 2022 11:26:29 +0200
Subject: [PATCH 582/935] hppa: Fix serial port assignments and pass-through

This fixes the serial ports in the emulation to behave as on original
hardware.

On the real hardware, the LASI UART is serial port #0 and the DINO UART
is serial port #1. This is fixed in SeaBIOS-hppa firmware v6, which is
why at least this firmware version is required.

The serial port addresses in hppa/hppa_hardware.h have to be swapped,
and when creating the virtual serial ports the correct port addresses
are used.

This patch now for example allows to specify on the qemu command line:
     -serial mon:stdio -serial /dev/ttyS4
to use the emulated ttyS0 in the guest for console output, and pass
ttyS4 from the host to ttyS1 in the guest.

Signed-off-by: Helge Deller <deller@gmx.de>
Reviewed-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
---
 hw/hppa/hppa_hardware.h |  4 ++--
 hw/hppa/machine.c       | 22 ++++++++--------------
 2 files changed, 10 insertions(+), 16 deletions(-)

diff --git a/hw/hppa/hppa_hardware.h b/hw/hppa/hppa_hardware.h
index 3f7627b98f..a5ac3dd0fd 100644
--- a/hw/hppa/hppa_hardware.h
+++ b/hw/hppa/hppa_hardware.h
@@ -41,8 +41,8 @@
 
 #define FW_CFG_IO_BASE  0xfffa0000
 
-#define PORT_SERIAL1    (DINO_UART_HPA + 0x800)
-#define PORT_SERIAL2    (LASI_UART_HPA + 0x800)
+#define PORT_SERIAL1    (LASI_UART_HPA + 0x800)
+#define PORT_SERIAL2    (DINO_UART_HPA + 0x800)
 
 #define HPPA_MAX_CPUS   16      /* max. number of SMP CPUs */
 #define CPU_CLOCK_MHZ   250     /* emulate a 250 MHz CPU */
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index d1e174b1f4..63b9dd2396 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -32,7 +32,7 @@
 
 #define MAX_IDE_BUS 2
 
-#define MIN_SEABIOS_HPPA_VERSION 1 /* require at least this fw version */
+#define MIN_SEABIOS_HPPA_VERSION 6 /* require at least this fw version */
 
 #define HPA_POWER_BUTTON (FIRMWARE_END - 0x10)
 
@@ -236,20 +236,14 @@ static void machine_hppa_init(MachineState *machine)
     /* Realtime clock, used by firmware for PDC_TOD call. */
     mc146818_rtc_init(isa_bus, 2000, NULL);
 
-    /* Serial code setup.  */
-    if (serial_hd(0)) {
-        uint32_t addr = DINO_UART_HPA + 0x800;
-        serial_mm_init(addr_space, addr, 0,
-                       qdev_get_gpio_in(dino_dev, DINO_IRQ_RS232INT),
-                       115200, serial_hd(0), DEVICE_BIG_ENDIAN);
-    }
+    /* Serial ports: Lasi and Dino use a 7.272727 MHz clock. */
+    serial_mm_init(addr_space, LASI_UART_HPA + 0x800, 0,
+        qdev_get_gpio_in(lasi_dev, LASI_IRQ_UART_HPA), 7272727 / 16,
+        serial_hd(0), DEVICE_BIG_ENDIAN);
 
-    if (serial_hd(1)) {
-        /* Serial port */
-        serial_mm_init(addr_space, LASI_UART_HPA + 0x800, 0,
-                qdev_get_gpio_in(lasi_dev, LASI_IRQ_UART_HPA), 8000000 / 16,
-                serial_hd(1), DEVICE_BIG_ENDIAN);
-    }
+    serial_mm_init(addr_space, DINO_UART_HPA + 0x800, 0,
+        qdev_get_gpio_in(dino_dev, DINO_IRQ_RS232INT), 7272727 / 16,
+        serial_hd(1), DEVICE_BIG_ENDIAN);
 
     /* Parallel port */
     parallel_mm_init(addr_space, LASI_LPT_HPA + 0x800, 0,

From 95d0f1d80eba918b6f1d7201994a468a38134ec1 Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Mon, 30 May 2022 10:26:52 +0100
Subject: [PATCH 583/935] docs/system/arm: Add FEAT_HCX to list of emulated
 features

In commit 5814d587fe861fe9 we added support for emulating
FEAT_HCX (Support for the HCRX_EL2 register). However we
forgot to add it to the list in emulated.rst. Correct the
omission.

Fixes: 5814d587fe861fe9 ("target/arm: Enable FEAT_HCX for -cpu max")
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220520084320.424166-1-peter.maydell@linaro.org
---
 docs/system/arm/emulation.rst | 1 +
 1 file changed, 1 insertion(+)

diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst
index 3e95bba0d2..49cc3e8340 100644
--- a/docs/system/arm/emulation.rst
+++ b/docs/system/arm/emulation.rst
@@ -29,6 +29,7 @@ the following architecture extensions:
 - FEAT_FRINTTS (Floating-point to integer instructions)
 - FEAT_FlagM (Flag manipulation instructions v2)
 - FEAT_FlagM2 (Enhancements to flag manipulation instructions)
+- FEAT_HCX (Support for the HCRX_EL2 register)
 - FEAT_HPDS (Hierarchical permission disables)
 - FEAT_I8MM (AArch64 Int8 matrix multiplication instructions)
 - FEAT_IDST (ID space trap handling)

From b5fb359cb5b9bf5d85932b869a707550a4e18ad1 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <f4bug@amsat.org>
Date: Wed, 25 May 2022 18:19:26 +0200
Subject: [PATCH 584/935] target/arm/hvf: Include missing "cpregs.h"
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Fix when building HVF on macOS Aarch64:

  target/arm/hvf/hvf.c:586:15: error: unknown type name 'ARMCPRegInfo'; did you mean 'ARMCPUInfo'?
          const ARMCPRegInfo *ri;
                ^~~~~~~~~~~~
                ARMCPUInfo
  target/arm/cpu-qom.h:38:3: note: 'ARMCPUInfo' declared here
  } ARMCPUInfo;
    ^
  target/arm/hvf/hvf.c:589:14: error: implicit declaration of function 'get_arm_cp_reginfo' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
          ri = get_arm_cp_reginfo(arm_cpu->cp_regs, key);
               ^
  target/arm/hvf/hvf.c:589:12: warning: incompatible integer to pointer conversion assigning to 'const ARMCPUInfo *' (aka 'const struct ARMCPUInfo *') from 'int' [-Wint-conversion]
          ri = get_arm_cp_reginfo(arm_cpu->cp_regs, key);
             ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  target/arm/hvf/hvf.c:591:26: error: no member named 'type' in 'struct ARMCPUInfo'
              assert(!(ri->type & ARM_CP_NO_RAW));
                       ~~  ^
  /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/assert.h:99:25: note: expanded from macro 'assert'
      (__builtin_expect(!(e), 0) ? __assert_rtn(__func__, __ASSERT_FILE_NAME, __LINE__, #e) : (void)0)
                          ^
  target/arm/hvf/hvf.c:591:33: error: use of undeclared identifier 'ARM_CP_NO_RAW'
              assert(!(ri->type & ARM_CP_NO_RAW));
                                  ^
  1 warning and 4 errors generated.

Fixes: cf7c6d1004 ("target/arm: Split out cpregs.h")
Reported-by: Duncan Bayne <duncan@bayne.id.au>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220525161926.34233-1-philmd@fungible.com
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1029
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/hvf/hvf.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 86710509d2..1fdc5eef92 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -17,6 +17,7 @@
 #include "sysemu/hvf_int.h"
 #include "sysemu/hw_accel.h"
 #include "hvf_arm.h"
+#include "cpregs.h"
 
 #include <mach/mach_time.h>
 

From fd71f258bc0d245b2223bf3032c0ebc82f8a3463 Mon Sep 17 00:00:00 2001
From: Icenowy Zheng <uwu@icenowy.me>
Date: Fri, 20 May 2022 20:42:00 +0800
Subject: [PATCH 585/935] hw/sd/allwinner-sdhost: report FIFO water level as 1
 when data ready

U-Boot queries the FIFO water level to reduce checking status register
when doing PIO SD card operation.

Report a FIFO water level of 1 when data is ready, to prevent the code
from trying to read 0 words from the FIFO each time.

Signed-off-by: Icenowy Zheng <uwu@icenowy.me>
Message-id: 20220520124200.2112699-1-uwu@icenowy.me
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/sd/allwinner-sdhost.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/hw/sd/allwinner-sdhost.c b/hw/sd/allwinner-sdhost.c
index 041e45c680..455d6eabf6 100644
--- a/hw/sd/allwinner-sdhost.c
+++ b/hw/sd/allwinner-sdhost.c
@@ -114,7 +114,9 @@ enum {
 };
 
 enum {
+    SD_STAR_FIFO_EMPTY      = (1 << 2),
     SD_STAR_CARD_PRESENT    = (1 << 8),
+    SD_STAR_FIFO_LEVEL_1    = (1 << 17),
 };
 
 enum {
@@ -467,6 +469,11 @@ static uint64_t allwinner_sdhost_read(void *opaque, hwaddr offset,
         break;
     case REG_SD_STAR:      /* Status */
         res = s->status;
+        if (sdbus_data_ready(&s->sdbus)) {
+            res |= SD_STAR_FIFO_LEVEL_1;
+        } else {
+            res |= SD_STAR_FIFO_EMPTY;
+        }
         break;
     case REG_SD_FWLR:      /* FIFO Water Level */
         res = s->fifo_wlevel;

From 05a546429f23841988c353d6ed7a1908cd209292 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:14 -0700
Subject: [PATCH 586/935] target/arm: Introduce TRANS, TRANS_FEAT

Steal the idea for these leaf function expanders from PowerPC.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-2-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate.h | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/target/arm/translate.h b/target/arm/translate.h
index 6f0ebdc88e..9f0bb270c5 100644
--- a/target/arm/translate.h
+++ b/target/arm/translate.h
@@ -576,4 +576,15 @@ static inline MemOp finalize_memop(DisasContext *s, MemOp opc)
  */
 uint64_t asimd_imm_const(uint32_t imm, int cmode, int op);
 
+/*
+ * Helpers for implementing sets of trans_* functions.
+ * Defer the implementation of NAME to FUNC, with optional extra arguments.
+ */
+#define TRANS(NAME, FUNC, ...) \
+    static bool trans_##NAME(DisasContext *s, arg_##NAME *a) \
+    { return FUNC(s, __VA_ARGS__); }
+#define TRANS_FEAT(NAME, FEAT, FUNC, ...) \
+    static bool trans_##NAME(DisasContext *s, arg_##NAME *a) \
+    { return dc_isar_feature(FEAT, s) && FUNC(s, __VA_ARGS__); }
+
 #endif /* TARGET_ARM_TRANSLATE_H */

From c5edf07d33558ed71932de221906ed47a52f3bb2 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:15 -0700
Subject: [PATCH 587/935] target/arm: Move null function and sve check into
 gen_gvec_ool_zz

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-3-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 36 +++++++++++++++---------------------
 1 file changed, 15 insertions(+), 21 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index dd4a5b23ab..903514cb6a 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -137,13 +137,19 @@ static int pred_gvec_reg_size(DisasContext *s)
 }
 
 /* Invoke an out-of-line helper on 2 Zregs. */
-static void gen_gvec_ool_zz(DisasContext *s, gen_helper_gvec_2 *fn,
+static bool gen_gvec_ool_zz(DisasContext *s, gen_helper_gvec_2 *fn,
                             int rd, int rn, int data)
 {
-    unsigned vsz = vec_full_reg_size(s);
-    tcg_gen_gvec_2_ool(vec_full_reg_offset(s, rd),
-                       vec_full_reg_offset(s, rn),
-                       vsz, vsz, data, fn);
+    if (fn == NULL) {
+        return false;
+    }
+    if (sve_access_check(s)) {
+        unsigned vsz = vec_full_reg_size(s);
+        tcg_gen_gvec_2_ool(vec_full_reg_offset(s, rd),
+                           vec_full_reg_offset(s, rn),
+                           vsz, vsz, data, fn);
+    }
+    return true;
 }
 
 /* Invoke an out-of-line helper on 3 Zregs. */
@@ -1377,13 +1383,7 @@ static bool trans_FEXPA(DisasContext *s, arg_rr_esz *a)
         gen_helper_sve_fexpa_s,
         gen_helper_sve_fexpa_d,
     };
-    if (a->esz == 0) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zz(s, fns[a->esz], a->rd, a->rn, 0);
-    }
-    return true;
+    return gen_gvec_ool_zz(s, fns[a->esz], a->rd, a->rn, 0);
 }
 
 static bool trans_FTSSEL(DisasContext *s, arg_rrr_esz *a)
@@ -2424,11 +2424,7 @@ static bool trans_REV_v(DisasContext *s, arg_rr_esz *a)
         gen_helper_sve_rev_b, gen_helper_sve_rev_h,
         gen_helper_sve_rev_s, gen_helper_sve_rev_d
     };
-
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zz(s, fns[a->esz], a->rd, a->rn, 0);
-    }
-    return true;
+    return gen_gvec_ool_zz(s, fns[a->esz], a->rd, a->rn, 0);
 }
 
 static bool trans_TBL(DisasContext *s, arg_rrr_esz *a)
@@ -8385,10 +8381,8 @@ static bool trans_AESMC(DisasContext *s, arg_AESMC *a)
     if (!dc_isar_feature(aa64_sve2_aes, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zz(s, gen_helper_crypto_aesmc, a->rd, a->rd, a->decrypt);
-    }
-    return true;
+    return gen_gvec_ool_zz(s, gen_helper_crypto_aesmc,
+                           a->rd, a->rd, a->decrypt);
 }
 
 static bool do_aese(DisasContext *s, arg_rrr_esz *a, bool decrypt)

From 0ea3cdbf7fc0d4ff143b0af6dc5d45354eb942f2 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:16 -0700
Subject: [PATCH 588/935] target/arm: Use TRANS_FEAT for gen_gvec_ool_zz

Convert SVE translation functions using gen_gvec_ool_zz to TRANS_FEAT.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-4-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 39 +++++++++++++-------------------------
 1 file changed, 13 insertions(+), 26 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 903514cb6a..f7e7a569b7 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -1375,16 +1375,12 @@ static bool trans_ADR_u32(DisasContext *s, arg_rrri *a)
  *** SVE Integer Misc - Unpredicated Group
  */
 
-static bool trans_FEXPA(DisasContext *s, arg_rr_esz *a)
-{
-    static gen_helper_gvec_2 * const fns[4] = {
-        NULL,
-        gen_helper_sve_fexpa_h,
-        gen_helper_sve_fexpa_s,
-        gen_helper_sve_fexpa_d,
-    };
-    return gen_gvec_ool_zz(s, fns[a->esz], a->rd, a->rn, 0);
-}
+static gen_helper_gvec_2 * const fexpa_fns[4] = {
+    NULL,                   gen_helper_sve_fexpa_h,
+    gen_helper_sve_fexpa_s, gen_helper_sve_fexpa_d,
+};
+TRANS_FEAT(FEXPA, aa64_sve, gen_gvec_ool_zz,
+           fexpa_fns[a->esz], a->rd, a->rn, 0)
 
 static bool trans_FTSSEL(DisasContext *s, arg_rrr_esz *a)
 {
@@ -2418,14 +2414,11 @@ static bool trans_INSR_r(DisasContext *s, arg_rrr_esz *a)
     return true;
 }
 
-static bool trans_REV_v(DisasContext *s, arg_rr_esz *a)
-{
-    static gen_helper_gvec_2 * const fns[4] = {
-        gen_helper_sve_rev_b, gen_helper_sve_rev_h,
-        gen_helper_sve_rev_s, gen_helper_sve_rev_d
-    };
-    return gen_gvec_ool_zz(s, fns[a->esz], a->rd, a->rn, 0);
-}
+static gen_helper_gvec_2 * const rev_fns[4] = {
+    gen_helper_sve_rev_b, gen_helper_sve_rev_h,
+    gen_helper_sve_rev_s, gen_helper_sve_rev_d
+};
+TRANS_FEAT(REV_v, aa64_sve, gen_gvec_ool_zz, rev_fns[a->esz], a->rd, a->rn, 0)
 
 static bool trans_TBL(DisasContext *s, arg_rrr_esz *a)
 {
@@ -8376,14 +8369,8 @@ static bool trans_USDOT_zzzz(DisasContext *s, arg_USDOT_zzzz *a)
     return true;
 }
 
-static bool trans_AESMC(DisasContext *s, arg_AESMC *a)
-{
-    if (!dc_isar_feature(aa64_sve2_aes, s)) {
-        return false;
-    }
-    return gen_gvec_ool_zz(s, gen_helper_crypto_aesmc,
-                           a->rd, a->rd, a->decrypt);
-}
+TRANS_FEAT(AESMC, aa64_sve2_aes, gen_gvec_ool_zz,
+           gen_helper_crypto_aesmc, a->rd, a->rd, a->decrypt)
 
 static bool do_aese(DisasContext *s, arg_rrr_esz *a, bool decrypt)
 {

From 913a8a0023a480121d17cd3a0fa505688a1f3b0a Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:17 -0700
Subject: [PATCH 589/935] target/arm: Move null function and sve check into
 gen_gvec_ool_zzz

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-5-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 74 ++++++++++++--------------------------
 1 file changed, 23 insertions(+), 51 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index f7e7a569b7..fd1d749c0e 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -153,14 +153,20 @@ static bool gen_gvec_ool_zz(DisasContext *s, gen_helper_gvec_2 *fn,
 }
 
 /* Invoke an out-of-line helper on 3 Zregs. */
-static void gen_gvec_ool_zzz(DisasContext *s, gen_helper_gvec_3 *fn,
+static bool gen_gvec_ool_zzz(DisasContext *s, gen_helper_gvec_3 *fn,
                              int rd, int rn, int rm, int data)
 {
-    unsigned vsz = vec_full_reg_size(s);
-    tcg_gen_gvec_3_ool(vec_full_reg_offset(s, rd),
-                       vec_full_reg_offset(s, rn),
-                       vec_full_reg_offset(s, rm),
-                       vsz, vsz, data, fn);
+    if (fn == NULL) {
+        return false;
+    }
+    if (sve_access_check(s)) {
+        unsigned vsz = vec_full_reg_size(s);
+        tcg_gen_gvec_3_ool(vec_full_reg_offset(s, rd),
+                           vec_full_reg_offset(s, rn),
+                           vec_full_reg_offset(s, rm),
+                           vsz, vsz, data, fn);
+    }
+    return true;
 }
 
 /* Invoke an out-of-line helper on 4 Zregs. */
@@ -1173,13 +1179,7 @@ static bool trans_LSL_zzi(DisasContext *s, arg_rri_esz *a)
 
 static bool do_zzw_ool(DisasContext *s, arg_rrr_esz *a, gen_helper_gvec_3 *fn)
 {
-    if (fn == NULL) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0);
-    }
-    return true;
+    return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0);
 }
 
 #define DO_ZZW(NAME, name) \
@@ -1345,10 +1345,7 @@ static bool trans_RDVL(DisasContext *s, arg_RDVL *a)
 
 static bool do_adr(DisasContext *s, arg_rrri *a, gen_helper_gvec_3 *fn)
 {
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, a->imm);
-    }
-    return true;
+    return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, a->imm);
 }
 
 static bool trans_ADR_p32(DisasContext *s, arg_rrri *a)
@@ -1390,13 +1387,7 @@ static bool trans_FTSSEL(DisasContext *s, arg_rrr_esz *a)
         gen_helper_sve_ftssel_s,
         gen_helper_sve_ftssel_d,
     };
-    if (a->esz == 0) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0);
-    }
-    return true;
+    return gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0);
 }
 
 /*
@@ -2426,11 +2417,7 @@ static bool trans_TBL(DisasContext *s, arg_rrr_esz *a)
         gen_helper_sve_tbl_b, gen_helper_sve_tbl_h,
         gen_helper_sve_tbl_s, gen_helper_sve_tbl_d
     };
-
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0);
-    }
-    return true;
+    return gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0);
 }
 
 static bool trans_TBL_sve2(DisasContext *s, arg_rrr_esz *a)
@@ -2460,10 +2447,7 @@ static bool trans_TBX(DisasContext *s, arg_rrr_esz *a)
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0);
-    }
-    return true;
+    return gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0);
 }
 
 static bool trans_UNPK(DisasContext *s, arg_UNPK *a)
@@ -2618,10 +2602,7 @@ static bool do_zip(DisasContext *s, arg_rrr_esz *a, bool high)
 static bool do_zzz_data_ool(DisasContext *s, arg_rrr_esz *a, int data,
                             gen_helper_gvec_3 *fn)
 {
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, data);
-    }
-    return true;
+    return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, data);
 }
 
 static bool trans_ZIP1_z(DisasContext *s, arg_rrr_esz *a)
@@ -6693,13 +6674,10 @@ static bool trans_MUL_zzz(DisasContext *s, arg_rrr_esz *a)
 static bool do_sve2_zzz_ool(DisasContext *s, arg_rrr_esz *a,
                             gen_helper_gvec_3 *fn)
 {
-    if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) {
+    if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0);
-    }
-    return true;
+    return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0);
 }
 
 static bool trans_SMULH_zzz(DisasContext *s, arg_rrr_esz *a)
@@ -8377,11 +8355,8 @@ static bool do_aese(DisasContext *s, arg_rrr_esz *a, bool decrypt)
     if (!dc_isar_feature(aa64_sve2_aes, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzz(s, gen_helper_crypto_aese,
-                         a->rd, a->rn, a->rm, decrypt);
-    }
-    return true;
+    return gen_gvec_ool_zzz(s, gen_helper_crypto_aese,
+                            a->rd, a->rn, a->rm, decrypt);
 }
 
 static bool trans_AESE(DisasContext *s, arg_rrr_esz *a)
@@ -8399,10 +8374,7 @@ static bool do_sm4(DisasContext *s, arg_rrr_esz *a, gen_helper_gvec_3 *fn)
     if (!dc_isar_feature(aa64_sve2_sm4, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0);
-    }
-    return true;
+    return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0);
 }
 
 static bool trans_SM4E(DisasContext *s, arg_rrr_esz *a)

From 84a272f565caa871a2762f74e718440bf9bbafc3 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:18 -0700
Subject: [PATCH 590/935] target/arm: Introduce gen_gvec_ool_arg_zzz

Use gen_gvec_ool_arg_zzz instead of gen_gvec_ool_zzz
when the arguments come from arg_rrr_esz.
Replaces do_zzw_ool and do_zzz_data_ool.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-6-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 48 +++++++++++++++++---------------------
 1 file changed, 21 insertions(+), 27 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index fd1d749c0e..37343fb2f0 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -169,6 +169,12 @@ static bool gen_gvec_ool_zzz(DisasContext *s, gen_helper_gvec_3 *fn,
     return true;
 }
 
+static bool gen_gvec_ool_arg_zzz(DisasContext *s, gen_helper_gvec_3 *fn,
+                                 arg_rrr_esz *a, int data)
+{
+    return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, data);
+}
+
 /* Invoke an out-of-line helper on 4 Zregs. */
 static void gen_gvec_ool_zzzz(DisasContext *s, gen_helper_gvec_4 *fn,
                               int rd, int rn, int rm, int ra, int data)
@@ -1177,11 +1183,6 @@ static bool trans_LSL_zzi(DisasContext *s, arg_rri_esz *a)
     return do_shift_imm(s, a, false, tcg_gen_gvec_shli);
 }
 
-static bool do_zzw_ool(DisasContext *s, arg_rrr_esz *a, gen_helper_gvec_3 *fn)
-{
-    return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0);
-}
-
 #define DO_ZZW(NAME, name) \
 static bool trans_##NAME##_zzw(DisasContext *s, arg_rrr_esz *a)           \
 {                                                                         \
@@ -1189,7 +1190,7 @@ static bool trans_##NAME##_zzw(DisasContext *s, arg_rrr_esz *a)           \
         gen_helper_sve_##name##_zzw_b, gen_helper_sve_##name##_zzw_h,     \
         gen_helper_sve_##name##_zzw_s, NULL                               \
     };                                                                    \
-    return do_zzw_ool(s, a, fns[a->esz]);                                 \
+    return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, 0);                    \
 }
 
 DO_ZZW(ASR, asr)
@@ -1387,7 +1388,7 @@ static bool trans_FTSSEL(DisasContext *s, arg_rrr_esz *a)
         gen_helper_sve_ftssel_s,
         gen_helper_sve_ftssel_d,
     };
-    return gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0);
+    return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, 0);
 }
 
 /*
@@ -2417,7 +2418,7 @@ static bool trans_TBL(DisasContext *s, arg_rrr_esz *a)
         gen_helper_sve_tbl_b, gen_helper_sve_tbl_h,
         gen_helper_sve_tbl_s, gen_helper_sve_tbl_d
     };
-    return gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0);
+    return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, 0);
 }
 
 static bool trans_TBL_sve2(DisasContext *s, arg_rrr_esz *a)
@@ -2447,7 +2448,7 @@ static bool trans_TBX(DisasContext *s, arg_rrr_esz *a)
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    return gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0);
+    return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, 0);
 }
 
 static bool trans_UNPK(DisasContext *s, arg_UNPK *a)
@@ -2599,12 +2600,6 @@ static bool do_zip(DisasContext *s, arg_rrr_esz *a, bool high)
     return true;
 }
 
-static bool do_zzz_data_ool(DisasContext *s, arg_rrr_esz *a, int data,
-                            gen_helper_gvec_3 *fn)
-{
-    return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, data);
-}
-
 static bool trans_ZIP1_z(DisasContext *s, arg_rrr_esz *a)
 {
     return do_zip(s, a, false);
@@ -2648,12 +2643,12 @@ static gen_helper_gvec_3 * const uzp_fns[4] = {
 
 static bool trans_UZP1_z(DisasContext *s, arg_rrr_esz *a)
 {
-    return do_zzz_data_ool(s, a, 0, uzp_fns[a->esz]);
+    return gen_gvec_ool_arg_zzz(s, uzp_fns[a->esz], a, 0);
 }
 
 static bool trans_UZP2_z(DisasContext *s, arg_rrr_esz *a)
 {
-    return do_zzz_data_ool(s, a, 1 << a->esz, uzp_fns[a->esz]);
+    return gen_gvec_ool_arg_zzz(s, uzp_fns[a->esz], a, 1 << a->esz);
 }
 
 static bool trans_UZP1_q(DisasContext *s, arg_rrr_esz *a)
@@ -2661,7 +2656,7 @@ static bool trans_UZP1_q(DisasContext *s, arg_rrr_esz *a)
     if (!dc_isar_feature(aa64_sve_f64mm, s)) {
         return false;
     }
-    return do_zzz_data_ool(s, a, 0, gen_helper_sve2_uzp_q);
+    return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_uzp_q, a, 0);
 }
 
 static bool trans_UZP2_q(DisasContext *s, arg_rrr_esz *a)
@@ -2669,7 +2664,7 @@ static bool trans_UZP2_q(DisasContext *s, arg_rrr_esz *a)
     if (!dc_isar_feature(aa64_sve_f64mm, s)) {
         return false;
     }
-    return do_zzz_data_ool(s, a, 16, gen_helper_sve2_uzp_q);
+    return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_uzp_q, a, 16);
 }
 
 static gen_helper_gvec_3 * const trn_fns[4] = {
@@ -2679,12 +2674,12 @@ static gen_helper_gvec_3 * const trn_fns[4] = {
 
 static bool trans_TRN1_z(DisasContext *s, arg_rrr_esz *a)
 {
-    return do_zzz_data_ool(s, a, 0, trn_fns[a->esz]);
+    return gen_gvec_ool_arg_zzz(s, trn_fns[a->esz], a, 0);
 }
 
 static bool trans_TRN2_z(DisasContext *s, arg_rrr_esz *a)
 {
-    return do_zzz_data_ool(s, a, 1 << a->esz, trn_fns[a->esz]);
+    return gen_gvec_ool_arg_zzz(s, trn_fns[a->esz], a, 1 << a->esz);
 }
 
 static bool trans_TRN1_q(DisasContext *s, arg_rrr_esz *a)
@@ -2692,7 +2687,7 @@ static bool trans_TRN1_q(DisasContext *s, arg_rrr_esz *a)
     if (!dc_isar_feature(aa64_sve_f64mm, s)) {
         return false;
     }
-    return do_zzz_data_ool(s, a, 0, gen_helper_sve2_trn_q);
+    return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_trn_q, a, 0);
 }
 
 static bool trans_TRN2_q(DisasContext *s, arg_rrr_esz *a)
@@ -2700,7 +2695,7 @@ static bool trans_TRN2_q(DisasContext *s, arg_rrr_esz *a)
     if (!dc_isar_feature(aa64_sve_f64mm, s)) {
         return false;
     }
-    return do_zzz_data_ool(s, a, 16, gen_helper_sve2_trn_q);
+    return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_trn_q, a, 16);
 }
 
 /*
@@ -6677,7 +6672,7 @@ static bool do_sve2_zzz_ool(DisasContext *s, arg_rrr_esz *a,
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0);
+    return gen_gvec_ool_arg_zzz(s, fn, a, 0);
 }
 
 static bool trans_SMULH_zzz(DisasContext *s, arg_rrr_esz *a)
@@ -8355,8 +8350,7 @@ static bool do_aese(DisasContext *s, arg_rrr_esz *a, bool decrypt)
     if (!dc_isar_feature(aa64_sve2_aes, s)) {
         return false;
     }
-    return gen_gvec_ool_zzz(s, gen_helper_crypto_aese,
-                            a->rd, a->rn, a->rm, decrypt);
+    return gen_gvec_ool_arg_zzz(s, gen_helper_crypto_aese, a, decrypt);
 }
 
 static bool trans_AESE(DisasContext *s, arg_rrr_esz *a)
@@ -8374,7 +8368,7 @@ static bool do_sm4(DisasContext *s, arg_rrr_esz *a, gen_helper_gvec_3 *fn)
     if (!dc_isar_feature(aa64_sve2_sm4, s)) {
         return false;
     }
-    return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0);
+    return gen_gvec_ool_arg_zzz(s, fn, a, 0);
 }
 
 static bool trans_SM4E(DisasContext *s, arg_rrr_esz *a)

From 32e2ad6589070d8b12209969514ac46ff9e435ab Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:19 -0700
Subject: [PATCH 591/935] target/arm: Use TRANS_FEAT for gen_gvec_ool_arg_zzz

Convert SVE translation functions using
gen_gvec_ool_arg_zzz to TRANS_FEAT.

Remove trivial wrappers do_aese, do_sm4.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-7-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 165 ++++++++++---------------------------
 1 file changed, 45 insertions(+), 120 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 37343fb2f0..c89c25166c 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -1184,18 +1184,16 @@ static bool trans_LSL_zzi(DisasContext *s, arg_rri_esz *a)
 }
 
 #define DO_ZZW(NAME, name) \
-static bool trans_##NAME##_zzw(DisasContext *s, arg_rrr_esz *a)           \
-{                                                                         \
-    static gen_helper_gvec_3 * const fns[4] = {                           \
+    static gen_helper_gvec_3 * const name##_zzw_fns[4] = {                \
         gen_helper_sve_##name##_zzw_b, gen_helper_sve_##name##_zzw_h,     \
         gen_helper_sve_##name##_zzw_s, NULL                               \
     };                                                                    \
-    return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, 0);                    \
-}
+    TRANS_FEAT(NAME, aa64_sve, gen_gvec_ool_arg_zzz,                      \
+               name##_zzw_fns[a->esz], a, 0)
 
-DO_ZZW(ASR, asr)
-DO_ZZW(LSR, lsr)
-DO_ZZW(LSL, lsl)
+DO_ZZW(ASR_zzw, asr)
+DO_ZZW(LSR_zzw, lsr)
+DO_ZZW(LSL_zzw, lsl)
 
 #undef DO_ZZW
 
@@ -1380,16 +1378,11 @@ static gen_helper_gvec_2 * const fexpa_fns[4] = {
 TRANS_FEAT(FEXPA, aa64_sve, gen_gvec_ool_zz,
            fexpa_fns[a->esz], a->rd, a->rn, 0)
 
-static bool trans_FTSSEL(DisasContext *s, arg_rrr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        NULL,
-        gen_helper_sve_ftssel_h,
-        gen_helper_sve_ftssel_s,
-        gen_helper_sve_ftssel_d,
-    };
-    return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, 0);
-}
+static gen_helper_gvec_3 * const ftssel_fns[4] = {
+    NULL,                    gen_helper_sve_ftssel_h,
+    gen_helper_sve_ftssel_s, gen_helper_sve_ftssel_d,
+};
+TRANS_FEAT(FTSSEL, aa64_sve, gen_gvec_ool_arg_zzz, ftssel_fns[a->esz], a, 0)
 
 /*
  *** SVE Predicate Logical Operations Group
@@ -2412,14 +2405,11 @@ static gen_helper_gvec_2 * const rev_fns[4] = {
 };
 TRANS_FEAT(REV_v, aa64_sve, gen_gvec_ool_zz, rev_fns[a->esz], a->rd, a->rn, 0)
 
-static bool trans_TBL(DisasContext *s, arg_rrr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve_tbl_b, gen_helper_sve_tbl_h,
-        gen_helper_sve_tbl_s, gen_helper_sve_tbl_d
-    };
-    return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, 0);
-}
+static gen_helper_gvec_3 * const sve_tbl_fns[4] = {
+    gen_helper_sve_tbl_b, gen_helper_sve_tbl_h,
+    gen_helper_sve_tbl_s, gen_helper_sve_tbl_d
+};
+TRANS_FEAT(TBL, aa64_sve, gen_gvec_ool_arg_zzz, sve_tbl_fns[a->esz], a, 0)
 
 static bool trans_TBL_sve2(DisasContext *s, arg_rrr_esz *a)
 {
@@ -2438,18 +2428,11 @@ static bool trans_TBL_sve2(DisasContext *s, arg_rrr_esz *a)
     return true;
 }
 
-static bool trans_TBX(DisasContext *s, arg_rrr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve2_tbx_b, gen_helper_sve2_tbx_h,
-        gen_helper_sve2_tbx_s, gen_helper_sve2_tbx_d
-    };
-
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, 0);
-}
+static gen_helper_gvec_3 * const tbx_fns[4] = {
+    gen_helper_sve2_tbx_b, gen_helper_sve2_tbx_h,
+    gen_helper_sve2_tbx_s, gen_helper_sve2_tbx_d
+};
+TRANS_FEAT(TBX, aa64_sve2, gen_gvec_ool_arg_zzz, tbx_fns[a->esz], a, 0)
 
 static bool trans_UNPK(DisasContext *s, arg_UNPK *a)
 {
@@ -2641,62 +2624,30 @@ static gen_helper_gvec_3 * const uzp_fns[4] = {
     gen_helper_sve_uzp_s, gen_helper_sve_uzp_d,
 };
 
-static bool trans_UZP1_z(DisasContext *s, arg_rrr_esz *a)
-{
-    return gen_gvec_ool_arg_zzz(s, uzp_fns[a->esz], a, 0);
-}
+TRANS_FEAT(UZP1_z, aa64_sve, gen_gvec_ool_arg_zzz,
+           uzp_fns[a->esz], a, 0)
+TRANS_FEAT(UZP2_z, aa64_sve, gen_gvec_ool_arg_zzz,
+           uzp_fns[a->esz], a, 1 << a->esz)
 
-static bool trans_UZP2_z(DisasContext *s, arg_rrr_esz *a)
-{
-    return gen_gvec_ool_arg_zzz(s, uzp_fns[a->esz], a, 1 << a->esz);
-}
-
-static bool trans_UZP1_q(DisasContext *s, arg_rrr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve_f64mm, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_uzp_q, a, 0);
-}
-
-static bool trans_UZP2_q(DisasContext *s, arg_rrr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve_f64mm, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_uzp_q, a, 16);
-}
+TRANS_FEAT(UZP1_q, aa64_sve_f64mm, gen_gvec_ool_arg_zzz,
+           gen_helper_sve2_uzp_q, a, 0)
+TRANS_FEAT(UZP2_q, aa64_sve_f64mm, gen_gvec_ool_arg_zzz,
+           gen_helper_sve2_uzp_q, a, 16)
 
 static gen_helper_gvec_3 * const trn_fns[4] = {
     gen_helper_sve_trn_b, gen_helper_sve_trn_h,
     gen_helper_sve_trn_s, gen_helper_sve_trn_d,
 };
 
-static bool trans_TRN1_z(DisasContext *s, arg_rrr_esz *a)
-{
-    return gen_gvec_ool_arg_zzz(s, trn_fns[a->esz], a, 0);
-}
+TRANS_FEAT(TRN1_z, aa64_sve, gen_gvec_ool_arg_zzz,
+           trn_fns[a->esz], a, 0)
+TRANS_FEAT(TRN2_z, aa64_sve, gen_gvec_ool_arg_zzz,
+           trn_fns[a->esz], a, 1 << a->esz)
 
-static bool trans_TRN2_z(DisasContext *s, arg_rrr_esz *a)
-{
-    return gen_gvec_ool_arg_zzz(s, trn_fns[a->esz], a, 1 << a->esz);
-}
-
-static bool trans_TRN1_q(DisasContext *s, arg_rrr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve_f64mm, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_trn_q, a, 0);
-}
-
-static bool trans_TRN2_q(DisasContext *s, arg_rrr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve_f64mm, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_trn_q, a, 16);
-}
+TRANS_FEAT(TRN1_q, aa64_sve_f64mm, gen_gvec_ool_arg_zzz,
+           gen_helper_sve2_trn_q, a, 0)
+TRANS_FEAT(TRN2_q, aa64_sve_f64mm, gen_gvec_ool_arg_zzz,
+           gen_helper_sve2_trn_q, a, 16)
 
 /*
  *** SVE Permute Vector - Predicated Group
@@ -8345,41 +8296,15 @@ static bool trans_USDOT_zzzz(DisasContext *s, arg_USDOT_zzzz *a)
 TRANS_FEAT(AESMC, aa64_sve2_aes, gen_gvec_ool_zz,
            gen_helper_crypto_aesmc, a->rd, a->rd, a->decrypt)
 
-static bool do_aese(DisasContext *s, arg_rrr_esz *a, bool decrypt)
-{
-    if (!dc_isar_feature(aa64_sve2_aes, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zzz(s, gen_helper_crypto_aese, a, decrypt);
-}
+TRANS_FEAT(AESE, aa64_sve2_aes, gen_gvec_ool_arg_zzz,
+           gen_helper_crypto_aese, a, false)
+TRANS_FEAT(AESD, aa64_sve2_aes, gen_gvec_ool_arg_zzz,
+           gen_helper_crypto_aese, a, true)
 
-static bool trans_AESE(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_aese(s, a, false);
-}
-
-static bool trans_AESD(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_aese(s, a, true);
-}
-
-static bool do_sm4(DisasContext *s, arg_rrr_esz *a, gen_helper_gvec_3 *fn)
-{
-    if (!dc_isar_feature(aa64_sve2_sm4, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zzz(s, fn, a, 0);
-}
-
-static bool trans_SM4E(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_sm4(s, a, gen_helper_crypto_sm4e);
-}
-
-static bool trans_SM4EKEY(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_sm4(s, a, gen_helper_crypto_sm4ekey);
-}
+TRANS_FEAT(SM4E, aa64_sve2_sm4, gen_gvec_ool_arg_zzz,
+           gen_helper_crypto_sm4e, a, 0)
+TRANS_FEAT(SM4EKEY, aa64_sve2_sm4, gen_gvec_ool_arg_zzz,
+           gen_helper_crypto_sm4ekey, a, 0)
 
 static bool trans_RAX1(DisasContext *s, arg_rrr_esz *a)
 {

From bd394cf52f8201da609bca3dc31b48783ea69d1e Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:20 -0700
Subject: [PATCH 592/935] target/arm: Use TRANS_FEAT for do_sve2_zzz_ool

Convert SVE translation functions using do_sve2_zzz_ool
to use TRANS_FEAT and gen_gvec_ool_arg_zzz.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-8-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 88 ++++++++++++++------------------------
 1 file changed, 31 insertions(+), 57 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index c89c25166c..6ec996e7f2 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -6617,55 +6617,36 @@ static bool trans_MUL_zzz(DisasContext *s, arg_rrr_esz *a)
     return true;
 }
 
-static bool do_sve2_zzz_ool(DisasContext *s, arg_rrr_esz *a,
-                            gen_helper_gvec_3 *fn)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zzz(s, fn, a, 0);
-}
+static gen_helper_gvec_3 * const smulh_zzz_fns[4] = {
+    gen_helper_gvec_smulh_b, gen_helper_gvec_smulh_h,
+    gen_helper_gvec_smulh_s, gen_helper_gvec_smulh_d,
+};
+TRANS_FEAT(SMULH_zzz, aa64_sve2, gen_gvec_ool_arg_zzz,
+           smulh_zzz_fns[a->esz], a, 0)
 
-static bool trans_SMULH_zzz(DisasContext *s, arg_rrr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_gvec_smulh_b, gen_helper_gvec_smulh_h,
-        gen_helper_gvec_smulh_s, gen_helper_gvec_smulh_d,
-    };
-    return do_sve2_zzz_ool(s, a, fns[a->esz]);
-}
+static gen_helper_gvec_3 * const umulh_zzz_fns[4] = {
+    gen_helper_gvec_umulh_b, gen_helper_gvec_umulh_h,
+    gen_helper_gvec_umulh_s, gen_helper_gvec_umulh_d,
+};
+TRANS_FEAT(UMULH_zzz, aa64_sve2, gen_gvec_ool_arg_zzz,
+           umulh_zzz_fns[a->esz], a, 0)
 
-static bool trans_UMULH_zzz(DisasContext *s, arg_rrr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_gvec_umulh_b, gen_helper_gvec_umulh_h,
-        gen_helper_gvec_umulh_s, gen_helper_gvec_umulh_d,
-    };
-    return do_sve2_zzz_ool(s, a, fns[a->esz]);
-}
+TRANS_FEAT(PMUL_zzz, aa64_sve2, gen_gvec_ool_arg_zzz,
+           gen_helper_gvec_pmul_b, a, 0)
 
-static bool trans_PMUL_zzz(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_sve2_zzz_ool(s, a, gen_helper_gvec_pmul_b);
-}
+static gen_helper_gvec_3 * const sqdmulh_zzz_fns[4] = {
+    gen_helper_sve2_sqdmulh_b, gen_helper_sve2_sqdmulh_h,
+    gen_helper_sve2_sqdmulh_s, gen_helper_sve2_sqdmulh_d,
+};
+TRANS_FEAT(SQDMULH_zzz, aa64_sve2, gen_gvec_ool_arg_zzz,
+           sqdmulh_zzz_fns[a->esz], a, 0)
 
-static bool trans_SQDMULH_zzz(DisasContext *s, arg_rrr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve2_sqdmulh_b, gen_helper_sve2_sqdmulh_h,
-        gen_helper_sve2_sqdmulh_s, gen_helper_sve2_sqdmulh_d,
-    };
-    return do_sve2_zzz_ool(s, a, fns[a->esz]);
-}
-
-static bool trans_SQRDMULH_zzz(DisasContext *s, arg_rrr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve2_sqrdmulh_b, gen_helper_sve2_sqrdmulh_h,
-        gen_helper_sve2_sqrdmulh_s, gen_helper_sve2_sqrdmulh_d,
-    };
-    return do_sve2_zzz_ool(s, a, fns[a->esz]);
-}
+static gen_helper_gvec_3 * const sqrdmulh_zzz_fns[4] = {
+    gen_helper_sve2_sqrdmulh_b, gen_helper_sve2_sqrdmulh_h,
+    gen_helper_sve2_sqrdmulh_s, gen_helper_sve2_sqrdmulh_d,
+};
+TRANS_FEAT(SQRDMULH_zzz, aa64_sve2, gen_gvec_ool_arg_zzz,
+           sqrdmulh_zzz_fns[a->esz], a, 0)
 
 /*
  * SVE2 Integer - Predicated
@@ -7964,14 +7945,12 @@ static bool trans_UQRSHRNT(DisasContext *s, arg_rri_esz *a)
 }
 
 #define DO_SVE2_ZZZ_NARROW(NAME, name)                                    \
-static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a)                 \
-{                                                                         \
-    static gen_helper_gvec_3 * const fns[4] = {                           \
+    static gen_helper_gvec_3 * const name##_fns[4] = {                    \
         NULL,                       gen_helper_sve2_##name##_h,           \
         gen_helper_sve2_##name##_s, gen_helper_sve2_##name##_d,           \
     };                                                                    \
-    return do_sve2_zzz_ool(s, a, fns[a->esz]);                            \
-}
+    TRANS_FEAT(NAME, aa64_sve2, gen_gvec_ool_arg_zzz,                     \
+               name##_fns[a->esz], a, 0)
 
 DO_SVE2_ZZZ_NARROW(ADDHNB, addhnb)
 DO_SVE2_ZZZ_NARROW(ADDHNT, addhnt)
@@ -8016,13 +7995,8 @@ static bool trans_HISTCNT(DisasContext *s, arg_rprr_esz *a)
     return do_sve2_zpzz_ool(s, a, fns[a->esz - 2]);
 }
 
-static bool trans_HISTSEG(DisasContext *s, arg_rrr_esz *a)
-{
-    if (a->esz != 0) {
-        return false;
-    }
-    return do_sve2_zzz_ool(s, a, gen_helper_sve2_histseg);
-}
+TRANS_FEAT(HISTSEG, aa64_sve2, gen_gvec_ool_arg_zzz,
+           a->esz == 0 ? gen_helper_sve2_histseg : NULL, a, 0)
 
 static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a,
                             gen_helper_gvec_4_ptr *fn)

From 7ad416b143ff4a7d3eeb0ffbfeabaf0d25f76bdc Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:21 -0700
Subject: [PATCH 593/935] target/arm: Move null function and sve check into
 gen_gvec_ool_zzzz

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-9-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 102 ++++++++++++++-----------------------
 1 file changed, 38 insertions(+), 64 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 6ec996e7f2..5aaef5b18f 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -176,15 +176,21 @@ static bool gen_gvec_ool_arg_zzz(DisasContext *s, gen_helper_gvec_3 *fn,
 }
 
 /* Invoke an out-of-line helper on 4 Zregs. */
-static void gen_gvec_ool_zzzz(DisasContext *s, gen_helper_gvec_4 *fn,
+static bool gen_gvec_ool_zzzz(DisasContext *s, gen_helper_gvec_4 *fn,
                               int rd, int rn, int rm, int ra, int data)
 {
-    unsigned vsz = vec_full_reg_size(s);
-    tcg_gen_gvec_4_ool(vec_full_reg_offset(s, rd),
-                       vec_full_reg_offset(s, rn),
-                       vec_full_reg_offset(s, rm),
-                       vec_full_reg_offset(s, ra),
-                       vsz, vsz, data, fn);
+    if (fn == NULL) {
+        return false;
+    }
+    if (sve_access_check(s)) {
+        unsigned vsz = vec_full_reg_size(s);
+        tcg_gen_gvec_4_ool(vec_full_reg_offset(s, rd),
+                           vec_full_reg_offset(s, rn),
+                           vec_full_reg_offset(s, rm),
+                           vec_full_reg_offset(s, ra),
+                           vsz, vsz, data, fn);
+    }
+    return true;
 }
 
 /* Invoke an out-of-line helper on 2 Zregs and a predicate. */
@@ -2421,11 +2427,8 @@ static bool trans_TBL_sve2(DisasContext *s, arg_rrr_esz *a)
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn,
-                          (a->rn + 1) % 32, a->rm, 0);
-    }
-    return true;
+    return gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn,
+                             (a->rn + 1) % 32, a->rm, 0);
 }
 
 static gen_helper_gvec_3 * const tbx_fns[4] = {
@@ -3813,11 +3816,8 @@ static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a)
         { gen_helper_gvec_sdot_b, gen_helper_gvec_sdot_h },
         { gen_helper_gvec_udot_b, gen_helper_gvec_udot_h }
     };
-
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzzz(s, fns[a->u][a->sz], a->rd, a->rn, a->rm, a->ra, 0);
-    }
-    return true;
+    return gen_gvec_ool_zzzz(s, fns[a->u][a->sz],
+                             a->rd, a->rn, a->rm, a->ra, 0);
 }
 
 /*
@@ -3827,13 +3827,7 @@ static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a)
 static bool do_zzxz_ool(DisasContext *s, arg_rrxr_esz *a,
                         gen_helper_gvec_4 *fn)
 {
-    if (fn == NULL) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->index);
-    }
-    return true;
+    return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->index);
 }
 
 #define DO_RRXR(NAME, FUNC) \
@@ -7122,13 +7116,10 @@ static bool trans_SQCADD_rot270(DisasContext *s, arg_rrr_esz *a)
 static bool do_sve2_zzzz_ool(DisasContext *s, arg_rrrr_esz *a,
                              gen_helper_gvec_4 *fn, int data)
 {
-    if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) {
+    if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data);
-    }
-    return true;
+    return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data);
 }
 
 static bool do_abal(DisasContext *s, arg_rrrr_esz *a, bool uns, bool sel)
@@ -8215,24 +8206,21 @@ static bool trans_CMLA_zzzz(DisasContext *s, arg_CMLA_zzzz *a)
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot);
-    }
-    return true;
+    return gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn,
+                             a->rm, a->ra, a->rot);
 }
 
 static bool trans_CDOT_zzzz(DisasContext *s, arg_CMLA_zzzz *a)
 {
-    if (!dc_isar_feature(aa64_sve2, s) || a->esz < MO_32) {
+    static gen_helper_gvec_4 * const fns[] = {
+        NULL, NULL, gen_helper_sve2_cdot_zzzz_s, gen_helper_sve2_cdot_zzzz_d
+    };
+
+    if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_helper_gvec_4 *fn = (a->esz == MO_32
-                                 ? gen_helper_sve2_cdot_zzzz_s
-                                 : gen_helper_sve2_cdot_zzzz_d);
-        gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->rot);
-    }
-    return true;
+    return gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn,
+                             a->rm, a->ra, a->rot);
 }
 
 static bool trans_SQRDCMLAH_zzzz(DisasContext *s, arg_SQRDCMLAH_zzzz *a)
@@ -8245,10 +8233,8 @@ static bool trans_SQRDCMLAH_zzzz(DisasContext *s, arg_SQRDCMLAH_zzzz *a)
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot);
-    }
-    return true;
+    return gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn,
+                             a->rm, a->ra, a->rot);
 }
 
 static bool trans_USDOT_zzzz(DisasContext *s, arg_USDOT_zzzz *a)
@@ -8452,10 +8438,7 @@ static bool do_i8mm_zzzz_ool(DisasContext *s, arg_rrrr_esz *a,
     if (!dc_isar_feature(aa64_sve_i8mm, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data);
-    }
-    return true;
+    return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data);
 }
 
 static bool trans_SMMLA(DisasContext *s, arg_rrrr_esz *a)
@@ -8478,11 +8461,8 @@ static bool trans_BFDOT_zzzz(DisasContext *s, arg_rrrr_esz *a)
     if (!dc_isar_feature(aa64_sve_bf16, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzzz(s, gen_helper_gvec_bfdot,
-                          a->rd, a->rn, a->rm, a->ra, 0);
-    }
-    return true;
+    return gen_gvec_ool_zzzz(s, gen_helper_gvec_bfdot,
+                             a->rd, a->rn, a->rm, a->ra, 0);
 }
 
 static bool trans_BFDOT_zzxz(DisasContext *s, arg_rrxr_esz *a)
@@ -8490,11 +8470,8 @@ static bool trans_BFDOT_zzxz(DisasContext *s, arg_rrxr_esz *a)
     if (!dc_isar_feature(aa64_sve_bf16, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzzz(s, gen_helper_gvec_bfdot_idx,
-                          a->rd, a->rn, a->rm, a->ra, a->index);
-    }
-    return true;
+    return gen_gvec_ool_zzzz(s, gen_helper_gvec_bfdot_idx,
+                             a->rd, a->rn, a->rm, a->ra, a->index);
 }
 
 static bool trans_BFMMLA(DisasContext *s, arg_rrrr_esz *a)
@@ -8502,11 +8479,8 @@ static bool trans_BFMMLA(DisasContext *s, arg_rrrr_esz *a)
     if (!dc_isar_feature(aa64_sve_bf16, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzzz(s, gen_helper_gvec_bfmmla,
-                          a->rd, a->rn, a->rm, a->ra, 0);
-    }
-    return true;
+    return gen_gvec_ool_zzzz(s, gen_helper_gvec_bfmmla,
+                             a->rd, a->rn, a->rm, a->ra, 0);
 }
 
 static bool do_BFMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel)

From 5f425b927b7578cbd5dfaed5d22b92ea331d1822 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:22 -0700
Subject: [PATCH 594/935] target/arm: Use TRANS_FEAT for gen_gvec_ool_zzzz

Convert SVE translation functions directly using
gen_gvec_ool_zzzz to TRANS_FEAT.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-10-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 89 +++++++++++++-------------------------
 1 file changed, 29 insertions(+), 60 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 5aaef5b18f..823c1d0ae3 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2417,19 +2417,12 @@ static gen_helper_gvec_3 * const sve_tbl_fns[4] = {
 };
 TRANS_FEAT(TBL, aa64_sve, gen_gvec_ool_arg_zzz, sve_tbl_fns[a->esz], a, 0)
 
-static bool trans_TBL_sve2(DisasContext *s, arg_rrr_esz *a)
-{
-    static gen_helper_gvec_4 * const fns[4] = {
-        gen_helper_sve2_tbl_b, gen_helper_sve2_tbl_h,
-        gen_helper_sve2_tbl_s, gen_helper_sve2_tbl_d
-    };
-
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn,
-                             (a->rn + 1) % 32, a->rm, 0);
-}
+static gen_helper_gvec_4 * const sve2_tbl_fns[4] = {
+    gen_helper_sve2_tbl_b, gen_helper_sve2_tbl_h,
+    gen_helper_sve2_tbl_s, gen_helper_sve2_tbl_d
+};
+TRANS_FEAT(TBL_sve2, aa64_sve2, gen_gvec_ool_zzzz, sve2_tbl_fns[a->esz],
+           a->rd, a->rn, (a->rn + 1) % 32, a->rm, 0)
 
 static gen_helper_gvec_3 * const tbx_fns[4] = {
     gen_helper_sve2_tbx_b, gen_helper_sve2_tbx_h,
@@ -3810,15 +3803,12 @@ DO_ZZI(UMIN, umin)
 
 #undef DO_ZZI
 
-static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a)
-{
-    static gen_helper_gvec_4 * const fns[2][2] = {
-        { gen_helper_gvec_sdot_b, gen_helper_gvec_sdot_h },
-        { gen_helper_gvec_udot_b, gen_helper_gvec_udot_h }
-    };
-    return gen_gvec_ool_zzzz(s, fns[a->u][a->sz],
-                             a->rd, a->rn, a->rm, a->ra, 0);
-}
+static gen_helper_gvec_4 * const dot_fns[2][2] = {
+    { gen_helper_gvec_sdot_b, gen_helper_gvec_sdot_h },
+    { gen_helper_gvec_udot_b, gen_helper_gvec_udot_h }
+};
+TRANS_FEAT(DOT_zzzz, aa64_sve, gen_gvec_ool_zzzz,
+           dot_fns[a->u][a->sz], a->rd, a->rn, a->rm, a->ra, 0)
 
 /*
  * SVE Multiply - Indexed
@@ -8196,46 +8186,25 @@ static bool trans_UMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a)
     return do_umlsl_zzzw(s, a, true);
 }
 
-static bool trans_CMLA_zzzz(DisasContext *s, arg_CMLA_zzzz *a)
-{
-    static gen_helper_gvec_4 * const fns[] = {
-        gen_helper_sve2_cmla_zzzz_b, gen_helper_sve2_cmla_zzzz_h,
-        gen_helper_sve2_cmla_zzzz_s, gen_helper_sve2_cmla_zzzz_d,
-    };
+static gen_helper_gvec_4 * const cmla_fns[] = {
+    gen_helper_sve2_cmla_zzzz_b, gen_helper_sve2_cmla_zzzz_h,
+    gen_helper_sve2_cmla_zzzz_s, gen_helper_sve2_cmla_zzzz_d,
+};
+TRANS_FEAT(CMLA_zzzz, aa64_sve2, gen_gvec_ool_zzzz,
+           cmla_fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot)
 
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn,
-                             a->rm, a->ra, a->rot);
-}
+static gen_helper_gvec_4 * const cdot_fns[] = {
+    NULL, NULL, gen_helper_sve2_cdot_zzzz_s, gen_helper_sve2_cdot_zzzz_d
+};
+TRANS_FEAT(CDOT_zzzz, aa64_sve2, gen_gvec_ool_zzzz,
+           cdot_fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot)
 
-static bool trans_CDOT_zzzz(DisasContext *s, arg_CMLA_zzzz *a)
-{
-    static gen_helper_gvec_4 * const fns[] = {
-        NULL, NULL, gen_helper_sve2_cdot_zzzz_s, gen_helper_sve2_cdot_zzzz_d
-    };
-
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn,
-                             a->rm, a->ra, a->rot);
-}
-
-static bool trans_SQRDCMLAH_zzzz(DisasContext *s, arg_SQRDCMLAH_zzzz *a)
-{
-    static gen_helper_gvec_4 * const fns[] = {
-        gen_helper_sve2_sqrdcmlah_zzzz_b, gen_helper_sve2_sqrdcmlah_zzzz_h,
-        gen_helper_sve2_sqrdcmlah_zzzz_s, gen_helper_sve2_sqrdcmlah_zzzz_d,
-    };
-
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn,
-                             a->rm, a->ra, a->rot);
-}
+static gen_helper_gvec_4 * const sqrdcmlah_fns[] = {
+    gen_helper_sve2_sqrdcmlah_zzzz_b, gen_helper_sve2_sqrdcmlah_zzzz_h,
+    gen_helper_sve2_sqrdcmlah_zzzz_s, gen_helper_sve2_sqrdcmlah_zzzz_d,
+};
+TRANS_FEAT(SQRDCMLAH_zzzz, aa64_sve2, gen_gvec_ool_zzzz,
+           sqrdcmlah_fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot)
 
 static bool trans_USDOT_zzzz(DisasContext *s, arg_USDOT_zzzz *a)
 {

From cab79ac93459567a6b0a8f437d116cba4118c52e Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:23 -0700
Subject: [PATCH 595/935] target/arm: Introduce gen_gvec_ool_arg_zzzz

Use gen_gvec_ool_arg_zzzz instead of gen_gvec_ool_zzzz
when the arguments come from arg_rrrr_esz.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-11-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 823c1d0ae3..e795baf6f9 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -193,6 +193,12 @@ static bool gen_gvec_ool_zzzz(DisasContext *s, gen_helper_gvec_4 *fn,
     return true;
 }
 
+static bool gen_gvec_ool_arg_zzzz(DisasContext *s, gen_helper_gvec_4 *fn,
+                                  arg_rrrr_esz *a, int data)
+{
+    return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data);
+}
+
 /* Invoke an out-of-line helper on 2 Zregs and a predicate. */
 static void gen_gvec_ool_zzp(DisasContext *s, gen_helper_gvec_3 *fn,
                              int rd, int rn, int pg, int data)
@@ -7109,7 +7115,7 @@ static bool do_sve2_zzzz_ool(DisasContext *s, arg_rrrr_esz *a,
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data);
+    return gen_gvec_ool_arg_zzzz(s, fn, a, data);
 }
 
 static bool do_abal(DisasContext *s, arg_rrrr_esz *a, bool uns, bool sel)
@@ -8407,7 +8413,7 @@ static bool do_i8mm_zzzz_ool(DisasContext *s, arg_rrrr_esz *a,
     if (!dc_isar_feature(aa64_sve_i8mm, s)) {
         return false;
     }
-    return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data);
+    return gen_gvec_ool_arg_zzzz(s, fn, a, data);
 }
 
 static bool trans_SMMLA(DisasContext *s, arg_rrrr_esz *a)
@@ -8430,8 +8436,7 @@ static bool trans_BFDOT_zzzz(DisasContext *s, arg_rrrr_esz *a)
     if (!dc_isar_feature(aa64_sve_bf16, s)) {
         return false;
     }
-    return gen_gvec_ool_zzzz(s, gen_helper_gvec_bfdot,
-                             a->rd, a->rn, a->rm, a->ra, 0);
+    return gen_gvec_ool_arg_zzzz(s, gen_helper_gvec_bfdot, a, 0);
 }
 
 static bool trans_BFDOT_zzxz(DisasContext *s, arg_rrxr_esz *a)
@@ -8448,8 +8453,7 @@ static bool trans_BFMMLA(DisasContext *s, arg_rrrr_esz *a)
     if (!dc_isar_feature(aa64_sve_bf16, s)) {
         return false;
     }
-    return gen_gvec_ool_zzzz(s, gen_helper_gvec_bfmmla,
-                             a->rd, a->rn, a->rm, a->ra, 0);
+    return gen_gvec_ool_arg_zzzz(s, gen_helper_gvec_bfmmla, a, 0);
 }
 
 static bool do_BFMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel)

From eeb4e84d384104a6872b06005a4636d60cb4f55e Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:24 -0700
Subject: [PATCH 596/935] target/arm: Use TRANS_FEAT for do_sve2_zzzz_ool

Convert SVE translation functions using do_sve2_zzzz_ool
to use TRANS_FEAT and gen_gvec_ool_arg_zzzz.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-12-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 263 +++++++++++--------------------------
 1 file changed, 79 insertions(+), 184 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index e795baf6f9..cae6df705a 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -7109,45 +7109,19 @@ static bool trans_SQCADD_rot270(DisasContext *s, arg_rrr_esz *a)
     return do_cadd(s, a, true, true);
 }
 
-static bool do_sve2_zzzz_ool(DisasContext *s, arg_rrrr_esz *a,
-                             gen_helper_gvec_4 *fn, int data)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zzzz(s, fn, a, data);
-}
+static gen_helper_gvec_4 * const sabal_fns[4] = {
+    NULL,                    gen_helper_sve2_sabal_h,
+    gen_helper_sve2_sabal_s, gen_helper_sve2_sabal_d,
+};
+TRANS_FEAT(SABALB, aa64_sve2, gen_gvec_ool_arg_zzzz, sabal_fns[a->esz], a, 0)
+TRANS_FEAT(SABALT, aa64_sve2, gen_gvec_ool_arg_zzzz, sabal_fns[a->esz], a, 1)
 
-static bool do_abal(DisasContext *s, arg_rrrr_esz *a, bool uns, bool sel)
-{
-    static gen_helper_gvec_4 * const fns[2][4] = {
-        { NULL,                    gen_helper_sve2_sabal_h,
-          gen_helper_sve2_sabal_s, gen_helper_sve2_sabal_d },
-        { NULL,                    gen_helper_sve2_uabal_h,
-          gen_helper_sve2_uabal_s, gen_helper_sve2_uabal_d },
-    };
-    return do_sve2_zzzz_ool(s, a, fns[uns][a->esz], sel);
-}
-
-static bool trans_SABALB(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_abal(s, a, false, false);
-}
-
-static bool trans_SABALT(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_abal(s, a, false, true);
-}
-
-static bool trans_UABALB(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_abal(s, a, true, false);
-}
-
-static bool trans_UABALT(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_abal(s, a, true, true);
-}
+static gen_helper_gvec_4 * const uabal_fns[4] = {
+    NULL,                    gen_helper_sve2_uabal_h,
+    gen_helper_sve2_uabal_s, gen_helper_sve2_uabal_d,
+};
+TRANS_FEAT(UABALB, aa64_sve2, gen_gvec_ool_arg_zzzz, uabal_fns[a->esz], a, 0)
+TRANS_FEAT(UABALT, aa64_sve2, gen_gvec_ool_arg_zzzz, uabal_fns[a->esz], a, 1)
 
 static bool do_adcl(DisasContext *s, arg_rrrr_esz *a, bool sel)
 {
@@ -7159,18 +7133,11 @@ static bool do_adcl(DisasContext *s, arg_rrrr_esz *a, bool sel)
      * Note that in this case the ESZ field encodes both size and sign.
      * Split out 'subtract' into bit 1 of the data field for the helper.
      */
-    return do_sve2_zzzz_ool(s, a, fns[a->esz & 1], (a->esz & 2) | sel);
+    return gen_gvec_ool_arg_zzzz(s, fns[a->esz & 1], a, (a->esz & 2) | sel);
 }
 
-static bool trans_ADCLB(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_adcl(s, a, false);
-}
-
-static bool trans_ADCLT(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_adcl(s, a, true);
-}
+TRANS_FEAT(ADCLB, aa64_sve2, do_adcl, a, false)
+TRANS_FEAT(ADCLT, aa64_sve2, do_adcl, a, true)
 
 static bool do_sve2_fn2i(DisasContext *s, arg_rri_esz *a, GVecGen2iFn *fn)
 {
@@ -8048,149 +8015,77 @@ static bool trans_FMMLA(DisasContext *s, arg_rrrr_esz *a)
     return true;
 }
 
-static bool do_sqdmlal_zzzw(DisasContext *s, arg_rrrr_esz *a,
-                            bool sel1, bool sel2)
-{
-    static gen_helper_gvec_4 * const fns[] = {
-        NULL,                           gen_helper_sve2_sqdmlal_zzzw_h,
-        gen_helper_sve2_sqdmlal_zzzw_s, gen_helper_sve2_sqdmlal_zzzw_d,
-    };
-    return do_sve2_zzzz_ool(s, a, fns[a->esz], (sel2 << 1) | sel1);
-}
+static gen_helper_gvec_4 * const sqdmlal_zzzw_fns[] = {
+    NULL,                           gen_helper_sve2_sqdmlal_zzzw_h,
+    gen_helper_sve2_sqdmlal_zzzw_s, gen_helper_sve2_sqdmlal_zzzw_d,
+};
+TRANS_FEAT(SQDMLALB_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz,
+           sqdmlal_zzzw_fns[a->esz], a, 0)
+TRANS_FEAT(SQDMLALT_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz,
+           sqdmlal_zzzw_fns[a->esz], a, 3)
+TRANS_FEAT(SQDMLALBT, aa64_sve2, gen_gvec_ool_arg_zzzz,
+           sqdmlal_zzzw_fns[a->esz], a, 2)
 
-static bool do_sqdmlsl_zzzw(DisasContext *s, arg_rrrr_esz *a,
-                            bool sel1, bool sel2)
-{
-    static gen_helper_gvec_4 * const fns[] = {
-        NULL,                           gen_helper_sve2_sqdmlsl_zzzw_h,
-        gen_helper_sve2_sqdmlsl_zzzw_s, gen_helper_sve2_sqdmlsl_zzzw_d,
-    };
-    return do_sve2_zzzz_ool(s, a, fns[a->esz], (sel2 << 1) | sel1);
-}
+static gen_helper_gvec_4 * const sqdmlsl_zzzw_fns[] = {
+    NULL,                           gen_helper_sve2_sqdmlsl_zzzw_h,
+    gen_helper_sve2_sqdmlsl_zzzw_s, gen_helper_sve2_sqdmlsl_zzzw_d,
+};
+TRANS_FEAT(SQDMLSLB_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz,
+           sqdmlsl_zzzw_fns[a->esz], a, 0)
+TRANS_FEAT(SQDMLSLT_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz,
+           sqdmlsl_zzzw_fns[a->esz], a, 3)
+TRANS_FEAT(SQDMLSLBT, aa64_sve2, gen_gvec_ool_arg_zzzz,
+           sqdmlsl_zzzw_fns[a->esz], a, 2)
 
-static bool trans_SQDMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_sqdmlal_zzzw(s, a, false, false);
-}
+static gen_helper_gvec_4 * const sqrdmlah_fns[] = {
+    gen_helper_sve2_sqrdmlah_b, gen_helper_sve2_sqrdmlah_h,
+    gen_helper_sve2_sqrdmlah_s, gen_helper_sve2_sqrdmlah_d,
+};
+TRANS_FEAT(SQRDMLAH_zzzz, aa64_sve2, gen_gvec_ool_arg_zzzz,
+           sqrdmlah_fns[a->esz], a, 0)
 
-static bool trans_SQDMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_sqdmlal_zzzw(s, a, true, true);
-}
+static gen_helper_gvec_4 * const sqrdmlsh_fns[] = {
+    gen_helper_sve2_sqrdmlsh_b, gen_helper_sve2_sqrdmlsh_h,
+    gen_helper_sve2_sqrdmlsh_s, gen_helper_sve2_sqrdmlsh_d,
+};
+TRANS_FEAT(SQRDMLSH_zzzz, aa64_sve2, gen_gvec_ool_arg_zzzz,
+           sqrdmlsh_fns[a->esz], a, 0)
 
-static bool trans_SQDMLALBT(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_sqdmlal_zzzw(s, a, false, true);
-}
+static gen_helper_gvec_4 * const smlal_zzzw_fns[] = {
+    NULL,                         gen_helper_sve2_smlal_zzzw_h,
+    gen_helper_sve2_smlal_zzzw_s, gen_helper_sve2_smlal_zzzw_d,
+};
+TRANS_FEAT(SMLALB_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz,
+           smlal_zzzw_fns[a->esz], a, 0)
+TRANS_FEAT(SMLALT_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz,
+           smlal_zzzw_fns[a->esz], a, 1)
 
-static bool trans_SQDMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_sqdmlsl_zzzw(s, a, false, false);
-}
+static gen_helper_gvec_4 * const umlal_zzzw_fns[] = {
+    NULL,                         gen_helper_sve2_umlal_zzzw_h,
+    gen_helper_sve2_umlal_zzzw_s, gen_helper_sve2_umlal_zzzw_d,
+};
+TRANS_FEAT(UMLALB_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz,
+           umlal_zzzw_fns[a->esz], a, 0)
+TRANS_FEAT(UMLALT_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz,
+           umlal_zzzw_fns[a->esz], a, 1)
 
-static bool trans_SQDMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_sqdmlsl_zzzw(s, a, true, true);
-}
+static gen_helper_gvec_4 * const smlsl_zzzw_fns[] = {
+    NULL,                         gen_helper_sve2_smlsl_zzzw_h,
+    gen_helper_sve2_smlsl_zzzw_s, gen_helper_sve2_smlsl_zzzw_d,
+};
+TRANS_FEAT(SMLSLB_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz,
+           smlsl_zzzw_fns[a->esz], a, 0)
+TRANS_FEAT(SMLSLT_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz,
+           smlsl_zzzw_fns[a->esz], a, 1)
 
-static bool trans_SQDMLSLBT(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_sqdmlsl_zzzw(s, a, false, true);
-}
-
-static bool trans_SQRDMLAH_zzzz(DisasContext *s, arg_rrrr_esz *a)
-{
-    static gen_helper_gvec_4 * const fns[] = {
-        gen_helper_sve2_sqrdmlah_b, gen_helper_sve2_sqrdmlah_h,
-        gen_helper_sve2_sqrdmlah_s, gen_helper_sve2_sqrdmlah_d,
-    };
-    return do_sve2_zzzz_ool(s, a, fns[a->esz], 0);
-}
-
-static bool trans_SQRDMLSH_zzzz(DisasContext *s, arg_rrrr_esz *a)
-{
-    static gen_helper_gvec_4 * const fns[] = {
-        gen_helper_sve2_sqrdmlsh_b, gen_helper_sve2_sqrdmlsh_h,
-        gen_helper_sve2_sqrdmlsh_s, gen_helper_sve2_sqrdmlsh_d,
-    };
-    return do_sve2_zzzz_ool(s, a, fns[a->esz], 0);
-}
-
-static bool do_smlal_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel)
-{
-    static gen_helper_gvec_4 * const fns[] = {
-        NULL,                         gen_helper_sve2_smlal_zzzw_h,
-        gen_helper_sve2_smlal_zzzw_s, gen_helper_sve2_smlal_zzzw_d,
-    };
-    return do_sve2_zzzz_ool(s, a, fns[a->esz], sel);
-}
-
-static bool trans_SMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_smlal_zzzw(s, a, false);
-}
-
-static bool trans_SMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_smlal_zzzw(s, a, true);
-}
-
-static bool do_umlal_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel)
-{
-    static gen_helper_gvec_4 * const fns[] = {
-        NULL,                         gen_helper_sve2_umlal_zzzw_h,
-        gen_helper_sve2_umlal_zzzw_s, gen_helper_sve2_umlal_zzzw_d,
-    };
-    return do_sve2_zzzz_ool(s, a, fns[a->esz], sel);
-}
-
-static bool trans_UMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_umlal_zzzw(s, a, false);
-}
-
-static bool trans_UMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_umlal_zzzw(s, a, true);
-}
-
-static bool do_smlsl_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel)
-{
-    static gen_helper_gvec_4 * const fns[] = {
-        NULL,                         gen_helper_sve2_smlsl_zzzw_h,
-        gen_helper_sve2_smlsl_zzzw_s, gen_helper_sve2_smlsl_zzzw_d,
-    };
-    return do_sve2_zzzz_ool(s, a, fns[a->esz], sel);
-}
-
-static bool trans_SMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_smlsl_zzzw(s, a, false);
-}
-
-static bool trans_SMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_smlsl_zzzw(s, a, true);
-}
-
-static bool do_umlsl_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel)
-{
-    static gen_helper_gvec_4 * const fns[] = {
-        NULL,                         gen_helper_sve2_umlsl_zzzw_h,
-        gen_helper_sve2_umlsl_zzzw_s, gen_helper_sve2_umlsl_zzzw_d,
-    };
-    return do_sve2_zzzz_ool(s, a, fns[a->esz], sel);
-}
-
-static bool trans_UMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_umlsl_zzzw(s, a, false);
-}
-
-static bool trans_UMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_umlsl_zzzw(s, a, true);
-}
+static gen_helper_gvec_4 * const umlsl_zzzw_fns[] = {
+    NULL,                         gen_helper_sve2_umlsl_zzzw_h,
+    gen_helper_sve2_umlsl_zzzw_s, gen_helper_sve2_umlsl_zzzw_d,
+};
+TRANS_FEAT(UMLSLB_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz,
+           umlsl_zzzw_fns[a->esz], a, 0)
+TRANS_FEAT(UMLSLT_zzzw, aa64_sve2, gen_gvec_ool_arg_zzzz,
+           umlsl_zzzw_fns[a->esz], a, 1)
 
 static gen_helper_gvec_4 * const cmla_fns[] = {
     gen_helper_sve2_cmla_zzzz_b, gen_helper_sve2_cmla_zzzz_h,

From eec05e4e1766713a29821e6205feddc88496eea6 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:25 -0700
Subject: [PATCH 597/935] target/arm: Use TRANS_FEAT for gen_gvec_ool_arg_zzzz

Convert SVE translation functions directly using
gen_gvec_ool_arg_zzzz to TRANS_FEAT.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-13-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 47 ++++++++------------------------------
 1 file changed, 10 insertions(+), 37 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index cae6df705a..f7367a4f62 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -8302,37 +8302,15 @@ static bool trans_FMLSLT_zzxw(DisasContext *s, arg_rrxr_esz *a)
     return do_FMLAL_zzxw(s, a, true, true);
 }
 
-static bool do_i8mm_zzzz_ool(DisasContext *s, arg_rrrr_esz *a,
-                             gen_helper_gvec_4 *fn, int data)
-{
-    if (!dc_isar_feature(aa64_sve_i8mm, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zzzz(s, fn, a, data);
-}
+TRANS_FEAT(SMMLA, aa64_sve_i8mm, gen_gvec_ool_arg_zzzz,
+           gen_helper_gvec_smmla_b, a, 0)
+TRANS_FEAT(USMMLA, aa64_sve_i8mm, gen_gvec_ool_arg_zzzz,
+           gen_helper_gvec_usmmla_b, a, 0)
+TRANS_FEAT(UMMLA, aa64_sve_i8mm, gen_gvec_ool_arg_zzzz,
+           gen_helper_gvec_ummla_b, a, 0)
 
-static bool trans_SMMLA(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_i8mm_zzzz_ool(s, a, gen_helper_gvec_smmla_b, 0);
-}
-
-static bool trans_USMMLA(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_i8mm_zzzz_ool(s, a, gen_helper_gvec_usmmla_b, 0);
-}
-
-static bool trans_UMMLA(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_i8mm_zzzz_ool(s, a, gen_helper_gvec_ummla_b, 0);
-}
-
-static bool trans_BFDOT_zzzz(DisasContext *s, arg_rrrr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve_bf16, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zzzz(s, gen_helper_gvec_bfdot, a, 0);
-}
+TRANS_FEAT(BFDOT_zzzz, aa64_sve_bf16, gen_gvec_ool_arg_zzzz,
+           gen_helper_gvec_bfdot, a, 0)
 
 static bool trans_BFDOT_zzxz(DisasContext *s, arg_rrxr_esz *a)
 {
@@ -8343,13 +8321,8 @@ static bool trans_BFDOT_zzxz(DisasContext *s, arg_rrxr_esz *a)
                              a->rd, a->rn, a->rm, a->ra, a->index);
 }
 
-static bool trans_BFMMLA(DisasContext *s, arg_rrrr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve_bf16, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zzzz(s, gen_helper_gvec_bfmmla, a, 0);
-}
+TRANS_FEAT(BFMMLA, aa64_sve_bf16, gen_gvec_ool_arg_zzzz,
+           gen_helper_gvec_bfmmla, a, 0)
 
 static bool do_BFMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel)
 {

From e82d3536cd1af0c5bde907269f3dc394bdf3912b Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:26 -0700
Subject: [PATCH 598/935] target/arm: Rename do_zzxz_ool to
 gen_gvec_ool_arg_zzxz

Rename the function to match gen_gvec_ool_arg_zzzz,
and move to be adjacent.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-14-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index f7367a4f62..b42df76c69 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -199,6 +199,12 @@ static bool gen_gvec_ool_arg_zzzz(DisasContext *s, gen_helper_gvec_4 *fn,
     return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data);
 }
 
+static bool gen_gvec_ool_arg_zzxz(DisasContext *s, gen_helper_gvec_4 *fn,
+                                  arg_rrxr_esz *a)
+{
+    return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->index);
+}
+
 /* Invoke an out-of-line helper on 2 Zregs and a predicate. */
 static void gen_gvec_ool_zzp(DisasContext *s, gen_helper_gvec_3 *fn,
                              int rd, int rn, int pg, int data)
@@ -3820,15 +3826,9 @@ TRANS_FEAT(DOT_zzzz, aa64_sve, gen_gvec_ool_zzzz,
  * SVE Multiply - Indexed
  */
 
-static bool do_zzxz_ool(DisasContext *s, arg_rrxr_esz *a,
-                        gen_helper_gvec_4 *fn)
-{
-    return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->index);
-}
-
 #define DO_RRXR(NAME, FUNC) \
     static bool NAME(DisasContext *s, arg_rrxr_esz *a)  \
-    { return do_zzxz_ool(s, a, FUNC); }
+    { return gen_gvec_ool_arg_zzxz(s, FUNC, a); }
 
 DO_RRXR(trans_SDOT_zzxw_s, gen_helper_gvec_sdot_idx_b)
 DO_RRXR(trans_SDOT_zzxw_d, gen_helper_gvec_sdot_idx_h)
@@ -3840,7 +3840,7 @@ static bool trans_SUDOT_zzxw_s(DisasContext *s, arg_rrxr_esz *a)
     if (!dc_isar_feature(aa64_sve_i8mm, s)) {
         return false;
     }
-    return do_zzxz_ool(s, a, gen_helper_gvec_sudot_idx_b);
+    return gen_gvec_ool_arg_zzxz(s, gen_helper_gvec_sudot_idx_b, a);
 }
 
 static bool trans_USDOT_zzxw_s(DisasContext *s, arg_rrxr_esz *a)
@@ -3848,7 +3848,7 @@ static bool trans_USDOT_zzxw_s(DisasContext *s, arg_rrxr_esz *a)
     if (!dc_isar_feature(aa64_sve_i8mm, s)) {
         return false;
     }
-    return do_zzxz_ool(s, a, gen_helper_gvec_usdot_idx_b);
+    return gen_gvec_ool_arg_zzxz(s, gen_helper_gvec_usdot_idx_b, a);
 }
 
 #undef DO_RRXR

From f3500a25fd12218fb74a09e9a6813108c10bf83c Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:27 -0700
Subject: [PATCH 599/935] target/arm: Use TRANS_FEAT for gen_gvec_ool_arg_zzxz

Convert SVE translation functions directly using
gen_gvec_ool_arg_zzxz to TRANS_FEAT.  Also include
BFDOT_zzxz, which was using gen_gvec_ool_zzzz.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-15-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 48 +++++++++++---------------------------
 1 file changed, 14 insertions(+), 34 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index b42df76c69..b097b44d9f 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3826,32 +3826,19 @@ TRANS_FEAT(DOT_zzzz, aa64_sve, gen_gvec_ool_zzzz,
  * SVE Multiply - Indexed
  */
 
-#define DO_RRXR(NAME, FUNC) \
-    static bool NAME(DisasContext *s, arg_rrxr_esz *a)  \
-    { return gen_gvec_ool_arg_zzxz(s, FUNC, a); }
+TRANS_FEAT(SDOT_zzxw_s, aa64_sve, gen_gvec_ool_arg_zzxz,
+           gen_helper_gvec_sdot_idx_b, a)
+TRANS_FEAT(SDOT_zzxw_d, aa64_sve, gen_gvec_ool_arg_zzxz,
+           gen_helper_gvec_sdot_idx_h, a)
+TRANS_FEAT(UDOT_zzxw_s, aa64_sve, gen_gvec_ool_arg_zzxz,
+           gen_helper_gvec_udot_idx_b, a)
+TRANS_FEAT(UDOT_zzxw_d, aa64_sve, gen_gvec_ool_arg_zzxz,
+           gen_helper_gvec_udot_idx_h, a)
 
-DO_RRXR(trans_SDOT_zzxw_s, gen_helper_gvec_sdot_idx_b)
-DO_RRXR(trans_SDOT_zzxw_d, gen_helper_gvec_sdot_idx_h)
-DO_RRXR(trans_UDOT_zzxw_s, gen_helper_gvec_udot_idx_b)
-DO_RRXR(trans_UDOT_zzxw_d, gen_helper_gvec_udot_idx_h)
-
-static bool trans_SUDOT_zzxw_s(DisasContext *s, arg_rrxr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve_i8mm, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zzxz(s, gen_helper_gvec_sudot_idx_b, a);
-}
-
-static bool trans_USDOT_zzxw_s(DisasContext *s, arg_rrxr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve_i8mm, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zzxz(s, gen_helper_gvec_usdot_idx_b, a);
-}
-
-#undef DO_RRXR
+TRANS_FEAT(SUDOT_zzxw_s, aa64_sve_i8mm, gen_gvec_ool_arg_zzxz,
+           gen_helper_gvec_sudot_idx_b, a)
+TRANS_FEAT(USDOT_zzxw_s, aa64_sve_i8mm, gen_gvec_ool_arg_zzxz,
+           gen_helper_gvec_usdot_idx_b, a)
 
 static bool do_sve2_zzz_data(DisasContext *s, int rd, int rn, int rm, int data,
                              gen_helper_gvec_3 *fn)
@@ -8311,15 +8298,8 @@ TRANS_FEAT(UMMLA, aa64_sve_i8mm, gen_gvec_ool_arg_zzzz,
 
 TRANS_FEAT(BFDOT_zzzz, aa64_sve_bf16, gen_gvec_ool_arg_zzzz,
            gen_helper_gvec_bfdot, a, 0)
-
-static bool trans_BFDOT_zzxz(DisasContext *s, arg_rrxr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve_bf16, s)) {
-        return false;
-    }
-    return gen_gvec_ool_zzzz(s, gen_helper_gvec_bfdot_idx,
-                             a->rd, a->rn, a->rm, a->ra, a->index);
-}
+TRANS_FEAT(BFDOT_zzxz, aa64_sve_bf16, gen_gvec_ool_arg_zzxz,
+           gen_helper_gvec_bfdot_idx, a)
 
 TRANS_FEAT(BFMMLA, aa64_sve_bf16, gen_gvec_ool_arg_zzzz,
            gen_helper_gvec_bfmmla, a, 0)

From af031f64287f95590bdc12a673e3571d8d4696ee Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:28 -0700
Subject: [PATCH 600/935] target/arm: Use TRANS_FEAT for do_sve2_zzz_data

Convert SVE translation functions using do_sve2_zzz_data
to use TRANS_FEAT and gen_gvec_ool_zzz.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-16-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 69 ++++++++++++++------------------------
 1 file changed, 25 insertions(+), 44 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index b097b44d9f..ac76705da6 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3840,61 +3840,42 @@ TRANS_FEAT(SUDOT_zzxw_s, aa64_sve_i8mm, gen_gvec_ool_arg_zzxz,
 TRANS_FEAT(USDOT_zzxw_s, aa64_sve_i8mm, gen_gvec_ool_arg_zzxz,
            gen_helper_gvec_usdot_idx_b, a)
 
-static bool do_sve2_zzz_data(DisasContext *s, int rd, int rn, int rm, int data,
-                             gen_helper_gvec_3 *fn)
-{
-    if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        tcg_gen_gvec_3_ool(vec_full_reg_offset(s, rd),
-                           vec_full_reg_offset(s, rn),
-                           vec_full_reg_offset(s, rm),
-                           vsz, vsz, data, fn);
-    }
-    return true;
-}
-
 #define DO_SVE2_RRX(NAME, FUNC) \
-    static bool NAME(DisasContext *s, arg_rrx_esz *a)  \
-    { return do_sve2_zzz_data(s, a->rd, a->rn, a->rm, a->index, FUNC); }
+    TRANS_FEAT(NAME, aa64_sve, gen_gvec_ool_zzz, FUNC,          \
+               a->rd, a->rn, a->rm, a->index)
 
-DO_SVE2_RRX(trans_MUL_zzx_h, gen_helper_gvec_mul_idx_h)
-DO_SVE2_RRX(trans_MUL_zzx_s, gen_helper_gvec_mul_idx_s)
-DO_SVE2_RRX(trans_MUL_zzx_d, gen_helper_gvec_mul_idx_d)
+DO_SVE2_RRX(MUL_zzx_h, gen_helper_gvec_mul_idx_h)
+DO_SVE2_RRX(MUL_zzx_s, gen_helper_gvec_mul_idx_s)
+DO_SVE2_RRX(MUL_zzx_d, gen_helper_gvec_mul_idx_d)
 
-DO_SVE2_RRX(trans_SQDMULH_zzx_h, gen_helper_sve2_sqdmulh_idx_h)
-DO_SVE2_RRX(trans_SQDMULH_zzx_s, gen_helper_sve2_sqdmulh_idx_s)
-DO_SVE2_RRX(trans_SQDMULH_zzx_d, gen_helper_sve2_sqdmulh_idx_d)
+DO_SVE2_RRX(SQDMULH_zzx_h, gen_helper_sve2_sqdmulh_idx_h)
+DO_SVE2_RRX(SQDMULH_zzx_s, gen_helper_sve2_sqdmulh_idx_s)
+DO_SVE2_RRX(SQDMULH_zzx_d, gen_helper_sve2_sqdmulh_idx_d)
 
-DO_SVE2_RRX(trans_SQRDMULH_zzx_h, gen_helper_sve2_sqrdmulh_idx_h)
-DO_SVE2_RRX(trans_SQRDMULH_zzx_s, gen_helper_sve2_sqrdmulh_idx_s)
-DO_SVE2_RRX(trans_SQRDMULH_zzx_d, gen_helper_sve2_sqrdmulh_idx_d)
+DO_SVE2_RRX(SQRDMULH_zzx_h, gen_helper_sve2_sqrdmulh_idx_h)
+DO_SVE2_RRX(SQRDMULH_zzx_s, gen_helper_sve2_sqrdmulh_idx_s)
+DO_SVE2_RRX(SQRDMULH_zzx_d, gen_helper_sve2_sqrdmulh_idx_d)
 
 #undef DO_SVE2_RRX
 
 #define DO_SVE2_RRX_TB(NAME, FUNC, TOP) \
-    static bool NAME(DisasContext *s, arg_rrx_esz *a)           \
-    {                                                           \
-        return do_sve2_zzz_data(s, a->rd, a->rn, a->rm,         \
-                                (a->index << 1) | TOP, FUNC);   \
-    }
+    TRANS_FEAT(NAME, aa64_sve, gen_gvec_ool_zzz, FUNC,          \
+               a->rd, a->rn, a->rm, (a->index << 1) | TOP)
 
-DO_SVE2_RRX_TB(trans_SQDMULLB_zzx_s, gen_helper_sve2_sqdmull_idx_s, false)
-DO_SVE2_RRX_TB(trans_SQDMULLB_zzx_d, gen_helper_sve2_sqdmull_idx_d, false)
-DO_SVE2_RRX_TB(trans_SQDMULLT_zzx_s, gen_helper_sve2_sqdmull_idx_s, true)
-DO_SVE2_RRX_TB(trans_SQDMULLT_zzx_d, gen_helper_sve2_sqdmull_idx_d, true)
+DO_SVE2_RRX_TB(SQDMULLB_zzx_s, gen_helper_sve2_sqdmull_idx_s, false)
+DO_SVE2_RRX_TB(SQDMULLB_zzx_d, gen_helper_sve2_sqdmull_idx_d, false)
+DO_SVE2_RRX_TB(SQDMULLT_zzx_s, gen_helper_sve2_sqdmull_idx_s, true)
+DO_SVE2_RRX_TB(SQDMULLT_zzx_d, gen_helper_sve2_sqdmull_idx_d, true)
 
-DO_SVE2_RRX_TB(trans_SMULLB_zzx_s, gen_helper_sve2_smull_idx_s, false)
-DO_SVE2_RRX_TB(trans_SMULLB_zzx_d, gen_helper_sve2_smull_idx_d, false)
-DO_SVE2_RRX_TB(trans_SMULLT_zzx_s, gen_helper_sve2_smull_idx_s, true)
-DO_SVE2_RRX_TB(trans_SMULLT_zzx_d, gen_helper_sve2_smull_idx_d, true)
+DO_SVE2_RRX_TB(SMULLB_zzx_s, gen_helper_sve2_smull_idx_s, false)
+DO_SVE2_RRX_TB(SMULLB_zzx_d, gen_helper_sve2_smull_idx_d, false)
+DO_SVE2_RRX_TB(SMULLT_zzx_s, gen_helper_sve2_smull_idx_s, true)
+DO_SVE2_RRX_TB(SMULLT_zzx_d, gen_helper_sve2_smull_idx_d, true)
 
-DO_SVE2_RRX_TB(trans_UMULLB_zzx_s, gen_helper_sve2_umull_idx_s, false)
-DO_SVE2_RRX_TB(trans_UMULLB_zzx_d, gen_helper_sve2_umull_idx_d, false)
-DO_SVE2_RRX_TB(trans_UMULLT_zzx_s, gen_helper_sve2_umull_idx_s, true)
-DO_SVE2_RRX_TB(trans_UMULLT_zzx_d, gen_helper_sve2_umull_idx_d, true)
+DO_SVE2_RRX_TB(UMULLB_zzx_s, gen_helper_sve2_umull_idx_s, false)
+DO_SVE2_RRX_TB(UMULLB_zzx_d, gen_helper_sve2_umull_idx_d, false)
+DO_SVE2_RRX_TB(UMULLT_zzx_s, gen_helper_sve2_umull_idx_s, true)
+DO_SVE2_RRX_TB(UMULLT_zzx_d, gen_helper_sve2_umull_idx_d, true)
 
 #undef DO_SVE2_RRX_TB
 

From 8681eb76224c959eaf9c9e290c9d598f712dd566 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:29 -0700
Subject: [PATCH 601/935] target/arm: Use TRANS_FEAT for do_sve2_zzzz_data

Convert SVE translation functions using do_sve2_zzzz_data
to use TRANS_FEAT and gen_gvec_ool_{zzzz,zzxz}.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-17-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 106 ++++++++++++++-----------------------
 1 file changed, 41 insertions(+), 65 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index ac76705da6..c345399ace 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3879,90 +3879,66 @@ DO_SVE2_RRX_TB(UMULLT_zzx_d, gen_helper_sve2_umull_idx_d, true)
 
 #undef DO_SVE2_RRX_TB
 
-static bool do_sve2_zzzz_data(DisasContext *s, int rd, int rn, int rm, int ra,
-                              int data, gen_helper_gvec_4 *fn)
-{
-    if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        tcg_gen_gvec_4_ool(vec_full_reg_offset(s, rd),
-                           vec_full_reg_offset(s, rn),
-                           vec_full_reg_offset(s, rm),
-                           vec_full_reg_offset(s, ra),
-                           vsz, vsz, data, fn);
-    }
-    return true;
-}
-
 #define DO_SVE2_RRXR(NAME, FUNC) \
-    static bool NAME(DisasContext *s, arg_rrxr_esz *a)  \
-    { return do_sve2_zzzz_data(s, a->rd, a->rn, a->rm, a->ra, a->index, FUNC); }
+    TRANS_FEAT(NAME, aa64_sve2, gen_gvec_ool_arg_zzxz, FUNC, a)
 
-DO_SVE2_RRXR(trans_MLA_zzxz_h, gen_helper_gvec_mla_idx_h)
-DO_SVE2_RRXR(trans_MLA_zzxz_s, gen_helper_gvec_mla_idx_s)
-DO_SVE2_RRXR(trans_MLA_zzxz_d, gen_helper_gvec_mla_idx_d)
+DO_SVE2_RRXR(MLA_zzxz_h, gen_helper_gvec_mla_idx_h)
+DO_SVE2_RRXR(MLA_zzxz_s, gen_helper_gvec_mla_idx_s)
+DO_SVE2_RRXR(MLA_zzxz_d, gen_helper_gvec_mla_idx_d)
 
-DO_SVE2_RRXR(trans_MLS_zzxz_h, gen_helper_gvec_mls_idx_h)
-DO_SVE2_RRXR(trans_MLS_zzxz_s, gen_helper_gvec_mls_idx_s)
-DO_SVE2_RRXR(trans_MLS_zzxz_d, gen_helper_gvec_mls_idx_d)
+DO_SVE2_RRXR(MLS_zzxz_h, gen_helper_gvec_mls_idx_h)
+DO_SVE2_RRXR(MLS_zzxz_s, gen_helper_gvec_mls_idx_s)
+DO_SVE2_RRXR(MLS_zzxz_d, gen_helper_gvec_mls_idx_d)
 
-DO_SVE2_RRXR(trans_SQRDMLAH_zzxz_h, gen_helper_sve2_sqrdmlah_idx_h)
-DO_SVE2_RRXR(trans_SQRDMLAH_zzxz_s, gen_helper_sve2_sqrdmlah_idx_s)
-DO_SVE2_RRXR(trans_SQRDMLAH_zzxz_d, gen_helper_sve2_sqrdmlah_idx_d)
+DO_SVE2_RRXR(SQRDMLAH_zzxz_h, gen_helper_sve2_sqrdmlah_idx_h)
+DO_SVE2_RRXR(SQRDMLAH_zzxz_s, gen_helper_sve2_sqrdmlah_idx_s)
+DO_SVE2_RRXR(SQRDMLAH_zzxz_d, gen_helper_sve2_sqrdmlah_idx_d)
 
-DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_h, gen_helper_sve2_sqrdmlsh_idx_h)
-DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_s, gen_helper_sve2_sqrdmlsh_idx_s)
-DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_d, gen_helper_sve2_sqrdmlsh_idx_d)
+DO_SVE2_RRXR(SQRDMLSH_zzxz_h, gen_helper_sve2_sqrdmlsh_idx_h)
+DO_SVE2_RRXR(SQRDMLSH_zzxz_s, gen_helper_sve2_sqrdmlsh_idx_s)
+DO_SVE2_RRXR(SQRDMLSH_zzxz_d, gen_helper_sve2_sqrdmlsh_idx_d)
 
 #undef DO_SVE2_RRXR
 
 #define DO_SVE2_RRXR_TB(NAME, FUNC, TOP) \
-    static bool NAME(DisasContext *s, arg_rrxr_esz *a)          \
-    {                                                           \
-        return do_sve2_zzzz_data(s, a->rd, a->rn, a->rm, a->rd, \
-                                 (a->index << 1) | TOP, FUNC);  \
-    }
+    TRANS_FEAT(NAME, aa64_sve2, gen_gvec_ool_zzzz, FUNC,        \
+               a->rd, a->rn, a->rm, a->ra, (a->index << 1) | TOP)
 
-DO_SVE2_RRXR_TB(trans_SQDMLALB_zzxw_s, gen_helper_sve2_sqdmlal_idx_s, false)
-DO_SVE2_RRXR_TB(trans_SQDMLALB_zzxw_d, gen_helper_sve2_sqdmlal_idx_d, false)
-DO_SVE2_RRXR_TB(trans_SQDMLALT_zzxw_s, gen_helper_sve2_sqdmlal_idx_s, true)
-DO_SVE2_RRXR_TB(trans_SQDMLALT_zzxw_d, gen_helper_sve2_sqdmlal_idx_d, true)
+DO_SVE2_RRXR_TB(SQDMLALB_zzxw_s, gen_helper_sve2_sqdmlal_idx_s, false)
+DO_SVE2_RRXR_TB(SQDMLALB_zzxw_d, gen_helper_sve2_sqdmlal_idx_d, false)
+DO_SVE2_RRXR_TB(SQDMLALT_zzxw_s, gen_helper_sve2_sqdmlal_idx_s, true)
+DO_SVE2_RRXR_TB(SQDMLALT_zzxw_d, gen_helper_sve2_sqdmlal_idx_d, true)
 
-DO_SVE2_RRXR_TB(trans_SQDMLSLB_zzxw_s, gen_helper_sve2_sqdmlsl_idx_s, false)
-DO_SVE2_RRXR_TB(trans_SQDMLSLB_zzxw_d, gen_helper_sve2_sqdmlsl_idx_d, false)
-DO_SVE2_RRXR_TB(trans_SQDMLSLT_zzxw_s, gen_helper_sve2_sqdmlsl_idx_s, true)
-DO_SVE2_RRXR_TB(trans_SQDMLSLT_zzxw_d, gen_helper_sve2_sqdmlsl_idx_d, true)
+DO_SVE2_RRXR_TB(SQDMLSLB_zzxw_s, gen_helper_sve2_sqdmlsl_idx_s, false)
+DO_SVE2_RRXR_TB(SQDMLSLB_zzxw_d, gen_helper_sve2_sqdmlsl_idx_d, false)
+DO_SVE2_RRXR_TB(SQDMLSLT_zzxw_s, gen_helper_sve2_sqdmlsl_idx_s, true)
+DO_SVE2_RRXR_TB(SQDMLSLT_zzxw_d, gen_helper_sve2_sqdmlsl_idx_d, true)
 
-DO_SVE2_RRXR_TB(trans_SMLALB_zzxw_s, gen_helper_sve2_smlal_idx_s, false)
-DO_SVE2_RRXR_TB(trans_SMLALB_zzxw_d, gen_helper_sve2_smlal_idx_d, false)
-DO_SVE2_RRXR_TB(trans_SMLALT_zzxw_s, gen_helper_sve2_smlal_idx_s, true)
-DO_SVE2_RRXR_TB(trans_SMLALT_zzxw_d, gen_helper_sve2_smlal_idx_d, true)
+DO_SVE2_RRXR_TB(SMLALB_zzxw_s, gen_helper_sve2_smlal_idx_s, false)
+DO_SVE2_RRXR_TB(SMLALB_zzxw_d, gen_helper_sve2_smlal_idx_d, false)
+DO_SVE2_RRXR_TB(SMLALT_zzxw_s, gen_helper_sve2_smlal_idx_s, true)
+DO_SVE2_RRXR_TB(SMLALT_zzxw_d, gen_helper_sve2_smlal_idx_d, true)
 
-DO_SVE2_RRXR_TB(trans_UMLALB_zzxw_s, gen_helper_sve2_umlal_idx_s, false)
-DO_SVE2_RRXR_TB(trans_UMLALB_zzxw_d, gen_helper_sve2_umlal_idx_d, false)
-DO_SVE2_RRXR_TB(trans_UMLALT_zzxw_s, gen_helper_sve2_umlal_idx_s, true)
-DO_SVE2_RRXR_TB(trans_UMLALT_zzxw_d, gen_helper_sve2_umlal_idx_d, true)
+DO_SVE2_RRXR_TB(UMLALB_zzxw_s, gen_helper_sve2_umlal_idx_s, false)
+DO_SVE2_RRXR_TB(UMLALB_zzxw_d, gen_helper_sve2_umlal_idx_d, false)
+DO_SVE2_RRXR_TB(UMLALT_zzxw_s, gen_helper_sve2_umlal_idx_s, true)
+DO_SVE2_RRXR_TB(UMLALT_zzxw_d, gen_helper_sve2_umlal_idx_d, true)
 
-DO_SVE2_RRXR_TB(trans_SMLSLB_zzxw_s, gen_helper_sve2_smlsl_idx_s, false)
-DO_SVE2_RRXR_TB(trans_SMLSLB_zzxw_d, gen_helper_sve2_smlsl_idx_d, false)
-DO_SVE2_RRXR_TB(trans_SMLSLT_zzxw_s, gen_helper_sve2_smlsl_idx_s, true)
-DO_SVE2_RRXR_TB(trans_SMLSLT_zzxw_d, gen_helper_sve2_smlsl_idx_d, true)
+DO_SVE2_RRXR_TB(SMLSLB_zzxw_s, gen_helper_sve2_smlsl_idx_s, false)
+DO_SVE2_RRXR_TB(SMLSLB_zzxw_d, gen_helper_sve2_smlsl_idx_d, false)
+DO_SVE2_RRXR_TB(SMLSLT_zzxw_s, gen_helper_sve2_smlsl_idx_s, true)
+DO_SVE2_RRXR_TB(SMLSLT_zzxw_d, gen_helper_sve2_smlsl_idx_d, true)
 
-DO_SVE2_RRXR_TB(trans_UMLSLB_zzxw_s, gen_helper_sve2_umlsl_idx_s, false)
-DO_SVE2_RRXR_TB(trans_UMLSLB_zzxw_d, gen_helper_sve2_umlsl_idx_d, false)
-DO_SVE2_RRXR_TB(trans_UMLSLT_zzxw_s, gen_helper_sve2_umlsl_idx_s, true)
-DO_SVE2_RRXR_TB(trans_UMLSLT_zzxw_d, gen_helper_sve2_umlsl_idx_d, true)
+DO_SVE2_RRXR_TB(UMLSLB_zzxw_s, gen_helper_sve2_umlsl_idx_s, false)
+DO_SVE2_RRXR_TB(UMLSLB_zzxw_d, gen_helper_sve2_umlsl_idx_d, false)
+DO_SVE2_RRXR_TB(UMLSLT_zzxw_s, gen_helper_sve2_umlsl_idx_s, true)
+DO_SVE2_RRXR_TB(UMLSLT_zzxw_d, gen_helper_sve2_umlsl_idx_d, true)
 
 #undef DO_SVE2_RRXR_TB
 
 #define DO_SVE2_RRXR_ROT(NAME, FUNC) \
-    static bool trans_##NAME(DisasContext *s, arg_##NAME *a)       \
-    {                                                              \
-        return do_sve2_zzzz_data(s, a->rd, a->rn, a->rm, a->ra,    \
-                                 (a->index << 2) | a->rot, FUNC);  \
-    }
+    TRANS_FEAT(NAME, aa64_sve2, gen_gvec_ool_zzzz, FUNC,           \
+               a->rd, a->rn, a->rm, a->ra, (a->index << 2) | a->rot)
 
 DO_SVE2_RRXR_ROT(CMLA_zzxz_h, gen_helper_sve2_cmla_idx_h)
 DO_SVE2_RRXR_ROT(CMLA_zzxz_s, gen_helper_sve2_cmla_idx_s)

From 615f19fefb24318e6c37860bf370a6dc39628b37 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:30 -0700
Subject: [PATCH 602/935] target/arm: Use TRANS_FEAT for do_sve2_zzw_data

Convert SVE translation functions using do_sve2_zzw_data
to use TRANS_FEAT and gen_gvec_ool_arg_zzz.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-18-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 297 ++++++++++++++++++-------------------
 1 file changed, 145 insertions(+), 152 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index c345399ace..2347b60d8e 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -6711,79 +6711,99 @@ DO_SVE2_ZPZZ(USQADD, usqadd)
  * SVE2 Widening Integer Arithmetic
  */
 
-static bool do_sve2_zzw_ool(DisasContext *s, arg_rrr_esz *a,
-                            gen_helper_gvec_3 *fn, int data)
-{
-    if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           vsz, vsz, data, fn);
-    }
-    return true;
-}
+static gen_helper_gvec_3 * const saddl_fns[4] = {
+    NULL,                    gen_helper_sve2_saddl_h,
+    gen_helper_sve2_saddl_s, gen_helper_sve2_saddl_d,
+};
+TRANS_FEAT(SADDLB, aa64_sve2, gen_gvec_ool_arg_zzz,
+           saddl_fns[a->esz], a, 0)
+TRANS_FEAT(SADDLT, aa64_sve2, gen_gvec_ool_arg_zzz,
+           saddl_fns[a->esz], a, 3)
+TRANS_FEAT(SADDLBT, aa64_sve2, gen_gvec_ool_arg_zzz,
+           saddl_fns[a->esz], a, 2)
 
-#define DO_SVE2_ZZZ_TB(NAME, name, SEL1, SEL2) \
-static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a)               \
-{                                                                       \
-    static gen_helper_gvec_3 * const fns[4] = {                         \
-        NULL,                       gen_helper_sve2_##name##_h,         \
-        gen_helper_sve2_##name##_s, gen_helper_sve2_##name##_d,         \
-    };                                                                  \
-    return do_sve2_zzw_ool(s, a, fns[a->esz], (SEL2 << 1) | SEL1);      \
-}
+static gen_helper_gvec_3 * const ssubl_fns[4] = {
+    NULL,                    gen_helper_sve2_ssubl_h,
+    gen_helper_sve2_ssubl_s, gen_helper_sve2_ssubl_d,
+};
+TRANS_FEAT(SSUBLB, aa64_sve2, gen_gvec_ool_arg_zzz,
+           ssubl_fns[a->esz], a, 0)
+TRANS_FEAT(SSUBLT, aa64_sve2, gen_gvec_ool_arg_zzz,
+           ssubl_fns[a->esz], a, 3)
+TRANS_FEAT(SSUBLBT, aa64_sve2, gen_gvec_ool_arg_zzz,
+           ssubl_fns[a->esz], a, 2)
+TRANS_FEAT(SSUBLTB, aa64_sve2, gen_gvec_ool_arg_zzz,
+           ssubl_fns[a->esz], a, 1)
 
-DO_SVE2_ZZZ_TB(SADDLB, saddl, false, false)
-DO_SVE2_ZZZ_TB(SSUBLB, ssubl, false, false)
-DO_SVE2_ZZZ_TB(SABDLB, sabdl, false, false)
+static gen_helper_gvec_3 * const sabdl_fns[4] = {
+    NULL,                    gen_helper_sve2_sabdl_h,
+    gen_helper_sve2_sabdl_s, gen_helper_sve2_sabdl_d,
+};
+TRANS_FEAT(SABDLB, aa64_sve2, gen_gvec_ool_arg_zzz,
+           sabdl_fns[a->esz], a, 0)
+TRANS_FEAT(SABDLT, aa64_sve2, gen_gvec_ool_arg_zzz,
+           sabdl_fns[a->esz], a, 3)
 
-DO_SVE2_ZZZ_TB(UADDLB, uaddl, false, false)
-DO_SVE2_ZZZ_TB(USUBLB, usubl, false, false)
-DO_SVE2_ZZZ_TB(UABDLB, uabdl, false, false)
+static gen_helper_gvec_3 * const uaddl_fns[4] = {
+    NULL,                    gen_helper_sve2_uaddl_h,
+    gen_helper_sve2_uaddl_s, gen_helper_sve2_uaddl_d,
+};
+TRANS_FEAT(UADDLB, aa64_sve2, gen_gvec_ool_arg_zzz,
+           uaddl_fns[a->esz], a, 0)
+TRANS_FEAT(UADDLT, aa64_sve2, gen_gvec_ool_arg_zzz,
+           uaddl_fns[a->esz], a, 3)
 
-DO_SVE2_ZZZ_TB(SADDLT, saddl, true, true)
-DO_SVE2_ZZZ_TB(SSUBLT, ssubl, true, true)
-DO_SVE2_ZZZ_TB(SABDLT, sabdl, true, true)
+static gen_helper_gvec_3 * const usubl_fns[4] = {
+    NULL,                    gen_helper_sve2_usubl_h,
+    gen_helper_sve2_usubl_s, gen_helper_sve2_usubl_d,
+};
+TRANS_FEAT(USUBLB, aa64_sve2, gen_gvec_ool_arg_zzz,
+           usubl_fns[a->esz], a, 0)
+TRANS_FEAT(USUBLT, aa64_sve2, gen_gvec_ool_arg_zzz,
+           usubl_fns[a->esz], a, 3)
 
-DO_SVE2_ZZZ_TB(UADDLT, uaddl, true, true)
-DO_SVE2_ZZZ_TB(USUBLT, usubl, true, true)
-DO_SVE2_ZZZ_TB(UABDLT, uabdl, true, true)
+static gen_helper_gvec_3 * const uabdl_fns[4] = {
+    NULL,                    gen_helper_sve2_uabdl_h,
+    gen_helper_sve2_uabdl_s, gen_helper_sve2_uabdl_d,
+};
+TRANS_FEAT(UABDLB, aa64_sve2, gen_gvec_ool_arg_zzz,
+           uabdl_fns[a->esz], a, 0)
+TRANS_FEAT(UABDLT, aa64_sve2, gen_gvec_ool_arg_zzz,
+           uabdl_fns[a->esz], a, 3)
 
-DO_SVE2_ZZZ_TB(SADDLBT, saddl, false, true)
-DO_SVE2_ZZZ_TB(SSUBLBT, ssubl, false, true)
-DO_SVE2_ZZZ_TB(SSUBLTB, ssubl, true, false)
+static gen_helper_gvec_3 * const sqdmull_fns[4] = {
+    NULL,                          gen_helper_sve2_sqdmull_zzz_h,
+    gen_helper_sve2_sqdmull_zzz_s, gen_helper_sve2_sqdmull_zzz_d,
+};
+TRANS_FEAT(SQDMULLB_zzz, aa64_sve2, gen_gvec_ool_arg_zzz,
+           sqdmull_fns[a->esz], a, 0)
+TRANS_FEAT(SQDMULLT_zzz, aa64_sve2, gen_gvec_ool_arg_zzz,
+           sqdmull_fns[a->esz], a, 3)
 
-DO_SVE2_ZZZ_TB(SQDMULLB_zzz, sqdmull_zzz, false, false)
-DO_SVE2_ZZZ_TB(SQDMULLT_zzz, sqdmull_zzz, true, true)
+static gen_helper_gvec_3 * const smull_fns[4] = {
+    NULL,                        gen_helper_sve2_smull_zzz_h,
+    gen_helper_sve2_smull_zzz_s, gen_helper_sve2_smull_zzz_d,
+};
+TRANS_FEAT(SMULLB_zzz, aa64_sve2, gen_gvec_ool_arg_zzz,
+           smull_fns[a->esz], a, 0)
+TRANS_FEAT(SMULLT_zzz, aa64_sve2, gen_gvec_ool_arg_zzz,
+           smull_fns[a->esz], a, 3)
 
-DO_SVE2_ZZZ_TB(SMULLB_zzz, smull_zzz, false, false)
-DO_SVE2_ZZZ_TB(SMULLT_zzz, smull_zzz, true, true)
+static gen_helper_gvec_3 * const umull_fns[4] = {
+    NULL,                        gen_helper_sve2_umull_zzz_h,
+    gen_helper_sve2_umull_zzz_s, gen_helper_sve2_umull_zzz_d,
+};
+TRANS_FEAT(UMULLB_zzz, aa64_sve2, gen_gvec_ool_arg_zzz,
+           umull_fns[a->esz], a, 0)
+TRANS_FEAT(UMULLT_zzz, aa64_sve2, gen_gvec_ool_arg_zzz,
+           umull_fns[a->esz], a, 3)
 
-DO_SVE2_ZZZ_TB(UMULLB_zzz, umull_zzz, false, false)
-DO_SVE2_ZZZ_TB(UMULLT_zzz, umull_zzz, true, true)
-
-static bool do_eor_tb(DisasContext *s, arg_rrr_esz *a, bool sel1)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve2_eoril_b, gen_helper_sve2_eoril_h,
-        gen_helper_sve2_eoril_s, gen_helper_sve2_eoril_d,
-    };
-    return do_sve2_zzw_ool(s, a, fns[a->esz], (!sel1 << 1) | sel1);
-}
-
-static bool trans_EORBT(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_eor_tb(s, a, false);
-}
-
-static bool trans_EORTB(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_eor_tb(s, a, true);
-}
+static gen_helper_gvec_3 * const eoril_fns[4] = {
+    gen_helper_sve2_eoril_b, gen_helper_sve2_eoril_h,
+    gen_helper_sve2_eoril_s, gen_helper_sve2_eoril_d,
+};
+TRANS_FEAT(EORBT, aa64_sve2, gen_gvec_ool_arg_zzz, eoril_fns[a->esz], a, 2)
+TRANS_FEAT(EORTB, aa64_sve2, gen_gvec_ool_arg_zzz, eoril_fns[a->esz], a, 1)
 
 static bool do_trans_pmull(DisasContext *s, arg_rrr_esz *a, bool sel)
 {
@@ -6794,38 +6814,39 @@ static bool do_trans_pmull(DisasContext *s, arg_rrr_esz *a, bool sel)
     if (a->esz == 0 && !dc_isar_feature(aa64_sve2_pmull128, s)) {
         return false;
     }
-    return do_sve2_zzw_ool(s, a, fns[a->esz], sel);
+    return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, sel);
 }
 
-static bool trans_PMULLB(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_trans_pmull(s, a, false);
-}
+TRANS_FEAT(PMULLB, aa64_sve2, do_trans_pmull, a, false)
+TRANS_FEAT(PMULLT, aa64_sve2, do_trans_pmull, a, true)
 
-static bool trans_PMULLT(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_trans_pmull(s, a, true);
-}
+static gen_helper_gvec_3 * const saddw_fns[4] = {
+    NULL,                    gen_helper_sve2_saddw_h,
+    gen_helper_sve2_saddw_s, gen_helper_sve2_saddw_d,
+};
+TRANS_FEAT(SADDWB, aa64_sve2, gen_gvec_ool_arg_zzz, saddw_fns[a->esz], a, 0)
+TRANS_FEAT(SADDWT, aa64_sve2, gen_gvec_ool_arg_zzz, saddw_fns[a->esz], a, 1)
 
-#define DO_SVE2_ZZZ_WTB(NAME, name, SEL2) \
-static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a)       \
-{                                                               \
-    static gen_helper_gvec_3 * const fns[4] = {                 \
-        NULL,                       gen_helper_sve2_##name##_h, \
-        gen_helper_sve2_##name##_s, gen_helper_sve2_##name##_d, \
-    };                                                          \
-    return do_sve2_zzw_ool(s, a, fns[a->esz], SEL2);            \
-}
+static gen_helper_gvec_3 * const ssubw_fns[4] = {
+    NULL,                    gen_helper_sve2_ssubw_h,
+    gen_helper_sve2_ssubw_s, gen_helper_sve2_ssubw_d,
+};
+TRANS_FEAT(SSUBWB, aa64_sve2, gen_gvec_ool_arg_zzz, ssubw_fns[a->esz], a, 0)
+TRANS_FEAT(SSUBWT, aa64_sve2, gen_gvec_ool_arg_zzz, ssubw_fns[a->esz], a, 1)
 
-DO_SVE2_ZZZ_WTB(SADDWB, saddw, false)
-DO_SVE2_ZZZ_WTB(SADDWT, saddw, true)
-DO_SVE2_ZZZ_WTB(SSUBWB, ssubw, false)
-DO_SVE2_ZZZ_WTB(SSUBWT, ssubw, true)
+static gen_helper_gvec_3 * const uaddw_fns[4] = {
+    NULL,                    gen_helper_sve2_uaddw_h,
+    gen_helper_sve2_uaddw_s, gen_helper_sve2_uaddw_d,
+};
+TRANS_FEAT(UADDWB, aa64_sve2, gen_gvec_ool_arg_zzz, uaddw_fns[a->esz], a, 0)
+TRANS_FEAT(UADDWT, aa64_sve2, gen_gvec_ool_arg_zzz, uaddw_fns[a->esz], a, 1)
 
-DO_SVE2_ZZZ_WTB(UADDWB, uaddw, false)
-DO_SVE2_ZZZ_WTB(UADDWT, uaddw, true)
-DO_SVE2_ZZZ_WTB(USUBWB, usubw, false)
-DO_SVE2_ZZZ_WTB(USUBWT, usubw, true)
+static gen_helper_gvec_3 * const usubw_fns[4] = {
+    NULL,                    gen_helper_sve2_usubw_h,
+    gen_helper_sve2_usubw_s, gen_helper_sve2_usubw_d,
+};
+TRANS_FEAT(USUBWB, aa64_sve2, gen_gvec_ool_arg_zzz, usubw_fns[a->esz], a, 0)
+TRANS_FEAT(USUBWT, aa64_sve2, gen_gvec_ool_arg_zzz, usubw_fns[a->esz], a, 1)
 
 static void gen_sshll_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t imm)
 {
@@ -6986,72 +7007,44 @@ static bool trans_USHLLT(DisasContext *s, arg_rri_esz *a)
     return do_sve2_shll_tb(s, a, true, true);
 }
 
-static bool trans_BEXT(DisasContext *s, arg_rrr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve2_bext_b, gen_helper_sve2_bext_h,
-        gen_helper_sve2_bext_s, gen_helper_sve2_bext_d,
-    };
-    if (!dc_isar_feature(aa64_sve2_bitperm, s)) {
-        return false;
-    }
-    return do_sve2_zzw_ool(s, a, fns[a->esz], 0);
-}
+static gen_helper_gvec_3 * const bext_fns[4] = {
+    gen_helper_sve2_bext_b, gen_helper_sve2_bext_h,
+    gen_helper_sve2_bext_s, gen_helper_sve2_bext_d,
+};
+TRANS_FEAT(BEXT, aa64_sve2_bitperm, gen_gvec_ool_arg_zzz,
+           bext_fns[a->esz], a, 0)
 
-static bool trans_BDEP(DisasContext *s, arg_rrr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve2_bdep_b, gen_helper_sve2_bdep_h,
-        gen_helper_sve2_bdep_s, gen_helper_sve2_bdep_d,
-    };
-    if (!dc_isar_feature(aa64_sve2_bitperm, s)) {
-        return false;
-    }
-    return do_sve2_zzw_ool(s, a, fns[a->esz], 0);
-}
+static gen_helper_gvec_3 * const bdep_fns[4] = {
+    gen_helper_sve2_bdep_b, gen_helper_sve2_bdep_h,
+    gen_helper_sve2_bdep_s, gen_helper_sve2_bdep_d,
+};
+TRANS_FEAT(BDEP, aa64_sve2_bitperm, gen_gvec_ool_arg_zzz,
+           bdep_fns[a->esz], a, 0)
 
-static bool trans_BGRP(DisasContext *s, arg_rrr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve2_bgrp_b, gen_helper_sve2_bgrp_h,
-        gen_helper_sve2_bgrp_s, gen_helper_sve2_bgrp_d,
-    };
-    if (!dc_isar_feature(aa64_sve2_bitperm, s)) {
-        return false;
-    }
-    return do_sve2_zzw_ool(s, a, fns[a->esz], 0);
-}
+static gen_helper_gvec_3 * const bgrp_fns[4] = {
+    gen_helper_sve2_bgrp_b, gen_helper_sve2_bgrp_h,
+    gen_helper_sve2_bgrp_s, gen_helper_sve2_bgrp_d,
+};
+TRANS_FEAT(BGRP, aa64_sve2_bitperm, gen_gvec_ool_arg_zzz,
+           bgrp_fns[a->esz], a, 0)
 
-static bool do_cadd(DisasContext *s, arg_rrr_esz *a, bool sq, bool rot)
-{
-    static gen_helper_gvec_3 * const fns[2][4] = {
-        { gen_helper_sve2_cadd_b, gen_helper_sve2_cadd_h,
-          gen_helper_sve2_cadd_s, gen_helper_sve2_cadd_d },
-        { gen_helper_sve2_sqcadd_b, gen_helper_sve2_sqcadd_h,
-          gen_helper_sve2_sqcadd_s, gen_helper_sve2_sqcadd_d },
-    };
-    return do_sve2_zzw_ool(s, a, fns[sq][a->esz], rot);
-}
+static gen_helper_gvec_3 * const cadd_fns[4] = {
+    gen_helper_sve2_cadd_b, gen_helper_sve2_cadd_h,
+    gen_helper_sve2_cadd_s, gen_helper_sve2_cadd_d,
+};
+TRANS_FEAT(CADD_rot90, aa64_sve2, gen_gvec_ool_arg_zzz,
+           cadd_fns[a->esz], a, 0)
+TRANS_FEAT(CADD_rot270, aa64_sve2, gen_gvec_ool_arg_zzz,
+           cadd_fns[a->esz], a, 1)
 
-static bool trans_CADD_rot90(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_cadd(s, a, false, false);
-}
-
-static bool trans_CADD_rot270(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_cadd(s, a, false, true);
-}
-
-static bool trans_SQCADD_rot90(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_cadd(s, a, true, false);
-}
-
-static bool trans_SQCADD_rot270(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_cadd(s, a, true, true);
-}
+static gen_helper_gvec_3 * const sqcadd_fns[4] = {
+    gen_helper_sve2_sqcadd_b, gen_helper_sve2_sqcadd_h,
+    gen_helper_sve2_sqcadd_s, gen_helper_sve2_sqcadd_d,
+};
+TRANS_FEAT(SQCADD_rot90, aa64_sve2, gen_gvec_ool_arg_zzz,
+           sqcadd_fns[a->esz], a, 0)
+TRANS_FEAT(SQCADD_rot270, aa64_sve2, gen_gvec_ool_arg_zzz,
+           sqcadd_fns[a->esz], a, 1)
 
 static gen_helper_gvec_4 * const sabal_fns[4] = {
     NULL,                    gen_helper_sve2_sabal_h,

From 8740d69416f556d68832d6f2b46c9972d574e94b Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:31 -0700
Subject: [PATCH 603/935] target/arm: Use TRANS_FEAT for USDOT_zzzz

This is the last direct user of tcg_gen_gvec_4_ool.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-19-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 17 ++---------------
 1 file changed, 2 insertions(+), 15 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 2347b60d8e..b4307f062c 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -8044,21 +8044,8 @@ static gen_helper_gvec_4 * const sqrdcmlah_fns[] = {
 TRANS_FEAT(SQRDCMLAH_zzzz, aa64_sve2, gen_gvec_ool_zzzz,
            sqrdcmlah_fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot)
 
-static bool trans_USDOT_zzzz(DisasContext *s, arg_USDOT_zzzz *a)
-{
-    if (a->esz != 2 || !dc_isar_feature(aa64_sve_i8mm, s)) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        tcg_gen_gvec_4_ool(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           vec_full_reg_offset(s, a->ra),
-                           vsz, vsz, 0, gen_helper_gvec_usdot_b);
-    }
-    return true;
-}
+TRANS_FEAT(USDOT_zzzz, aa64_sve_i8mm, gen_gvec_ool_arg_zzzz,
+           a->esz == 2 ? gen_helper_gvec_usdot_b : NULL, a, 0)
 
 TRANS_FEAT(AESMC, aa64_sve2_aes, gen_gvec_ool_zz,
            gen_helper_crypto_aesmc, a->rd, a->rd, a->decrypt)

From 8fb27a21b1a0b29ad60e155de838e199cdf4ee38 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:32 -0700
Subject: [PATCH 604/935] target/arm: Move null function and sve check into
 gen_gvec_ool_zzp

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-20-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 37 +++++++++++++++----------------------
 1 file changed, 15 insertions(+), 22 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index b4307f062c..e81c2de37f 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -206,14 +206,20 @@ static bool gen_gvec_ool_arg_zzxz(DisasContext *s, gen_helper_gvec_4 *fn,
 }
 
 /* Invoke an out-of-line helper on 2 Zregs and a predicate. */
-static void gen_gvec_ool_zzp(DisasContext *s, gen_helper_gvec_3 *fn,
+static bool gen_gvec_ool_zzp(DisasContext *s, gen_helper_gvec_3 *fn,
                              int rd, int rn, int pg, int data)
 {
-    unsigned vsz = vec_full_reg_size(s);
-    tcg_gen_gvec_3_ool(vec_full_reg_offset(s, rd),
-                       vec_full_reg_offset(s, rn),
-                       pred_full_reg_offset(s, pg),
-                       vsz, vsz, data, fn);
+    if (fn == NULL) {
+        return false;
+    }
+    if (sve_access_check(s)) {
+        unsigned vsz = vec_full_reg_size(s);
+        tcg_gen_gvec_3_ool(vec_full_reg_offset(s, rd),
+                           vec_full_reg_offset(s, rn),
+                           pred_full_reg_offset(s, pg),
+                           vsz, vsz, data, fn);
+    }
+    return true;
 }
 
 /* Invoke an out-of-line helper on 3 Zregs and a predicate. */
@@ -801,13 +807,7 @@ static bool trans_SEL_zpzz(DisasContext *s, arg_rprr_esz *a)
 
 static bool do_zpz_ool(DisasContext *s, arg_rpr_esz *a, gen_helper_gvec_3 *fn)
 {
-    if (fn == NULL) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, 0);
-    }
-    return true;
+    return gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, 0);
 }
 
 #define DO_ZPZ(NAME, name) \
@@ -986,20 +986,13 @@ static bool do_movz_zpz(DisasContext *s, int rd, int rn, int pg,
         gen_helper_sve_movz_b, gen_helper_sve_movz_h,
         gen_helper_sve_movz_s, gen_helper_sve_movz_d,
     };
-
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzp(s, fns[esz], rd, rn, pg, invert);
-    }
-    return true;
+    return gen_gvec_ool_zzp(s, fns[esz], rd, rn, pg, invert);
 }
 
 static bool do_zpzi_ool(DisasContext *s, arg_rpri_esz *a,
                         gen_helper_gvec_3 *fn)
 {
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, a->imm);
-    }
-    return true;
+    return gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, a->imm);
 }
 
 static bool trans_ASR_zpzi(DisasContext *s, arg_rpri_esz *a)

From b051809adf08dc75869f5adc229d981f2a7f6070 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:33 -0700
Subject: [PATCH 605/935] target/arm: Introduce gen_gvec_ool_arg_zpz

Use gen_gvec_ool_arg_zpz instead of gen_gvec_ool_zzp
when the arguments come from arg_rpr_esz.
Replaces do_zpz_ool.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-21-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 45 +++++++++++++++++++++-----------------
 1 file changed, 25 insertions(+), 20 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index e81c2de37f..be426db6a9 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -222,6 +222,13 @@ static bool gen_gvec_ool_zzp(DisasContext *s, gen_helper_gvec_3 *fn,
     return true;
 }
 
+static bool gen_gvec_ool_arg_zpz(DisasContext *s, gen_helper_gvec_3 *fn,
+                                 arg_rpr_esz *a, int data)
+{
+    return gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, data);
+}
+
+
 /* Invoke an out-of-line helper on 3 Zregs and a predicate. */
 static void gen_gvec_ool_zzzp(DisasContext *s, gen_helper_gvec_4 *fn,
                               int rd, int rn, int rm, int pg, int data)
@@ -805,11 +812,6 @@ static bool trans_SEL_zpzz(DisasContext *s, arg_rprr_esz *a)
  *** SVE Integer Arithmetic - Unary Predicated Group
  */
 
-static bool do_zpz_ool(DisasContext *s, arg_rpr_esz *a, gen_helper_gvec_3 *fn)
-{
-    return gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, 0);
-}
-
 #define DO_ZPZ(NAME, name) \
 static bool trans_##NAME(DisasContext *s, arg_rpr_esz *a)           \
 {                                                                   \
@@ -817,7 +819,7 @@ static bool trans_##NAME(DisasContext *s, arg_rpr_esz *a)           \
         gen_helper_sve_##name##_b, gen_helper_sve_##name##_h,       \
         gen_helper_sve_##name##_s, gen_helper_sve_##name##_d,       \
     };                                                              \
-    return do_zpz_ool(s, a, fns[a->esz]);                           \
+    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);              \
 }
 
 DO_ZPZ(CLS, cls)
@@ -836,7 +838,7 @@ static bool trans_FABS(DisasContext *s, arg_rpr_esz *a)
         gen_helper_sve_fabs_s,
         gen_helper_sve_fabs_d
     };
-    return do_zpz_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
 }
 
 static bool trans_FNEG(DisasContext *s, arg_rpr_esz *a)
@@ -847,7 +849,7 @@ static bool trans_FNEG(DisasContext *s, arg_rpr_esz *a)
         gen_helper_sve_fneg_s,
         gen_helper_sve_fneg_d
     };
-    return do_zpz_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
 }
 
 static bool trans_SXTB(DisasContext *s, arg_rpr_esz *a)
@@ -858,7 +860,7 @@ static bool trans_SXTB(DisasContext *s, arg_rpr_esz *a)
         gen_helper_sve_sxtb_s,
         gen_helper_sve_sxtb_d
     };
-    return do_zpz_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
 }
 
 static bool trans_UXTB(DisasContext *s, arg_rpr_esz *a)
@@ -869,7 +871,7 @@ static bool trans_UXTB(DisasContext *s, arg_rpr_esz *a)
         gen_helper_sve_uxtb_s,
         gen_helper_sve_uxtb_d
     };
-    return do_zpz_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
 }
 
 static bool trans_SXTH(DisasContext *s, arg_rpr_esz *a)
@@ -879,7 +881,7 @@ static bool trans_SXTH(DisasContext *s, arg_rpr_esz *a)
         gen_helper_sve_sxth_s,
         gen_helper_sve_sxth_d
     };
-    return do_zpz_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
 }
 
 static bool trans_UXTH(DisasContext *s, arg_rpr_esz *a)
@@ -889,17 +891,19 @@ static bool trans_UXTH(DisasContext *s, arg_rpr_esz *a)
         gen_helper_sve_uxth_s,
         gen_helper_sve_uxth_d
     };
-    return do_zpz_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
 }
 
 static bool trans_SXTW(DisasContext *s, arg_rpr_esz *a)
 {
-    return do_zpz_ool(s, a, a->esz == 3 ? gen_helper_sve_sxtw_d : NULL);
+    return gen_gvec_ool_arg_zpz(s, a->esz == 3 ? gen_helper_sve_sxtw_d
+                                : NULL, a, 0);
 }
 
 static bool trans_UXTW(DisasContext *s, arg_rpr_esz *a)
 {
-    return do_zpz_ool(s, a, a->esz == 3 ? gen_helper_sve_uxtw_d : NULL);
+    return gen_gvec_ool_arg_zpz(s, a->esz == 3 ? gen_helper_sve_uxtw_d
+                                : NULL, a, 0);
 }
 
 #undef DO_ZPZ
@@ -2659,7 +2663,7 @@ static bool trans_COMPACT(DisasContext *s, arg_rpr_esz *a)
     static gen_helper_gvec_3 * const fns[4] = {
         NULL, NULL, gen_helper_sve_compact_s, gen_helper_sve_compact_d
     };
-    return do_zpz_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
 }
 
 /* Call the helper that computes the ARM LastActiveElement pseudocode
@@ -3008,7 +3012,7 @@ static bool trans_REVB(DisasContext *s, arg_rpr_esz *a)
         gen_helper_sve_revb_s,
         gen_helper_sve_revb_d,
     };
-    return do_zpz_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
 }
 
 static bool trans_REVH(DisasContext *s, arg_rpr_esz *a)
@@ -3019,12 +3023,13 @@ static bool trans_REVH(DisasContext *s, arg_rpr_esz *a)
         gen_helper_sve_revh_s,
         gen_helper_sve_revh_d,
     };
-    return do_zpz_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
 }
 
 static bool trans_REVW(DisasContext *s, arg_rpr_esz *a)
 {
-    return do_zpz_ool(s, a, a->esz == 3 ? gen_helper_sve_revw_d : NULL);
+    return gen_gvec_ool_arg_zpz(s, a->esz == 3 ? gen_helper_sve_revw_d
+                                : NULL, a, 0);
 }
 
 static bool trans_RBIT(DisasContext *s, arg_rpr_esz *a)
@@ -3035,7 +3040,7 @@ static bool trans_RBIT(DisasContext *s, arg_rpr_esz *a)
         gen_helper_sve_rbit_s,
         gen_helper_sve_rbit_d,
     };
-    return do_zpz_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
 }
 
 static bool trans_SPLICE(DisasContext *s, arg_rprr_esz *a)
@@ -6624,7 +6629,7 @@ static bool do_sve2_zpz_ool(DisasContext *s, arg_rpr_esz *a,
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    return do_zpz_ool(s, a, fn);
+    return gen_gvec_ool_arg_zpz(s, fn, a, 0);
 }
 
 static bool trans_URECPE(DisasContext *s, arg_rpr_esz *a)

From 817bd5c98e2cfe7a3477248e01aa316450769ce0 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:34 -0700
Subject: [PATCH 606/935] target/arm: Use TRANS_FEAT for gen_gvec_ool_arg_zpz

Convert SVE translation functions directly using
gen_gvec_ool_arg_zpz to TRANS_FEAT.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-22-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 189 ++++++++++++-------------------------
 1 file changed, 60 insertions(+), 129 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index be426db6a9..d5a09a60af 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -812,101 +812,60 @@ static bool trans_SEL_zpzz(DisasContext *s, arg_rprr_esz *a)
  *** SVE Integer Arithmetic - Unary Predicated Group
  */
 
-#define DO_ZPZ(NAME, name) \
-static bool trans_##NAME(DisasContext *s, arg_rpr_esz *a)           \
-{                                                                   \
-    static gen_helper_gvec_3 * const fns[4] = {                     \
-        gen_helper_sve_##name##_b, gen_helper_sve_##name##_h,       \
-        gen_helper_sve_##name##_s, gen_helper_sve_##name##_d,       \
+#define DO_ZPZ(NAME, FEAT, name) \
+    static gen_helper_gvec_3 * const name##_fns[4] = {              \
+        gen_helper_##name##_b, gen_helper_##name##_h,               \
+        gen_helper_##name##_s, gen_helper_##name##_d,               \
     };                                                              \
-    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);              \
-}
+    TRANS_FEAT(NAME, FEAT, gen_gvec_ool_arg_zpz, name##_fns[a->esz], a, 0)
 
-DO_ZPZ(CLS, cls)
-DO_ZPZ(CLZ, clz)
-DO_ZPZ(CNT_zpz, cnt_zpz)
-DO_ZPZ(CNOT, cnot)
-DO_ZPZ(NOT_zpz, not_zpz)
-DO_ZPZ(ABS, abs)
-DO_ZPZ(NEG, neg)
+DO_ZPZ(CLS, aa64_sve, sve_cls)
+DO_ZPZ(CLZ, aa64_sve, sve_clz)
+DO_ZPZ(CNT_zpz, aa64_sve, sve_cnt_zpz)
+DO_ZPZ(CNOT, aa64_sve, sve_cnot)
+DO_ZPZ(NOT_zpz, aa64_sve, sve_not_zpz)
+DO_ZPZ(ABS, aa64_sve, sve_abs)
+DO_ZPZ(NEG, aa64_sve, sve_neg)
+DO_ZPZ(RBIT, aa64_sve, sve_rbit)
 
-static bool trans_FABS(DisasContext *s, arg_rpr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        NULL,
-        gen_helper_sve_fabs_h,
-        gen_helper_sve_fabs_s,
-        gen_helper_sve_fabs_d
-    };
-    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
-}
+static gen_helper_gvec_3 * const fabs_fns[4] = {
+    NULL,                  gen_helper_sve_fabs_h,
+    gen_helper_sve_fabs_s, gen_helper_sve_fabs_d,
+};
+TRANS_FEAT(FABS, aa64_sve, gen_gvec_ool_arg_zpz, fabs_fns[a->esz], a, 0)
 
-static bool trans_FNEG(DisasContext *s, arg_rpr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        NULL,
-        gen_helper_sve_fneg_h,
-        gen_helper_sve_fneg_s,
-        gen_helper_sve_fneg_d
-    };
-    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
-}
+static gen_helper_gvec_3 * const fneg_fns[4] = {
+    NULL,                  gen_helper_sve_fneg_h,
+    gen_helper_sve_fneg_s, gen_helper_sve_fneg_d,
+};
+TRANS_FEAT(FNEG, aa64_sve, gen_gvec_ool_arg_zpz, fneg_fns[a->esz], a, 0)
 
-static bool trans_SXTB(DisasContext *s, arg_rpr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        NULL,
-        gen_helper_sve_sxtb_h,
-        gen_helper_sve_sxtb_s,
-        gen_helper_sve_sxtb_d
-    };
-    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
-}
+static gen_helper_gvec_3 * const sxtb_fns[4] = {
+    NULL,                  gen_helper_sve_sxtb_h,
+    gen_helper_sve_sxtb_s, gen_helper_sve_sxtb_d,
+};
+TRANS_FEAT(SXTB, aa64_sve, gen_gvec_ool_arg_zpz, sxtb_fns[a->esz], a, 0)
 
-static bool trans_UXTB(DisasContext *s, arg_rpr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        NULL,
-        gen_helper_sve_uxtb_h,
-        gen_helper_sve_uxtb_s,
-        gen_helper_sve_uxtb_d
-    };
-    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
-}
+static gen_helper_gvec_3 * const uxtb_fns[4] = {
+    NULL,                  gen_helper_sve_uxtb_h,
+    gen_helper_sve_uxtb_s, gen_helper_sve_uxtb_d,
+};
+TRANS_FEAT(UXTB, aa64_sve, gen_gvec_ool_arg_zpz, uxtb_fns[a->esz], a, 0)
 
-static bool trans_SXTH(DisasContext *s, arg_rpr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        NULL, NULL,
-        gen_helper_sve_sxth_s,
-        gen_helper_sve_sxth_d
-    };
-    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
-}
+static gen_helper_gvec_3 * const sxth_fns[4] = {
+    NULL, NULL, gen_helper_sve_sxth_s, gen_helper_sve_sxth_d
+};
+TRANS_FEAT(SXTH, aa64_sve, gen_gvec_ool_arg_zpz, sxth_fns[a->esz], a, 0)
 
-static bool trans_UXTH(DisasContext *s, arg_rpr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        NULL, NULL,
-        gen_helper_sve_uxth_s,
-        gen_helper_sve_uxth_d
-    };
-    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
-}
+static gen_helper_gvec_3 * const uxth_fns[4] = {
+    NULL, NULL, gen_helper_sve_uxth_s, gen_helper_sve_uxth_d
+};
+TRANS_FEAT(UXTH, aa64_sve, gen_gvec_ool_arg_zpz, uxth_fns[a->esz], a, 0)
 
-static bool trans_SXTW(DisasContext *s, arg_rpr_esz *a)
-{
-    return gen_gvec_ool_arg_zpz(s, a->esz == 3 ? gen_helper_sve_sxtw_d
-                                : NULL, a, 0);
-}
-
-static bool trans_UXTW(DisasContext *s, arg_rpr_esz *a)
-{
-    return gen_gvec_ool_arg_zpz(s, a->esz == 3 ? gen_helper_sve_uxtw_d
-                                : NULL, a, 0);
-}
-
-#undef DO_ZPZ
+TRANS_FEAT(SXTW, aa64_sve, gen_gvec_ool_arg_zpz,
+           a->esz == 3 ? gen_helper_sve_sxtw_d : NULL, a, 0)
+TRANS_FEAT(UXTW, aa64_sve, gen_gvec_ool_arg_zpz,
+           a->esz == 3 ? gen_helper_sve_uxtw_d : NULL, a, 0)
 
 /*
  *** SVE Integer Reduction Group
@@ -2658,13 +2617,10 @@ TRANS_FEAT(TRN2_q, aa64_sve_f64mm, gen_gvec_ool_arg_zzz,
  *** SVE Permute Vector - Predicated Group
  */
 
-static bool trans_COMPACT(DisasContext *s, arg_rpr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        NULL, NULL, gen_helper_sve_compact_s, gen_helper_sve_compact_d
-    };
-    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
-}
+static gen_helper_gvec_3 * const compact_fns[4] = {
+    NULL, NULL, gen_helper_sve_compact_s, gen_helper_sve_compact_d
+};
+TRANS_FEAT(COMPACT, aa64_sve, gen_gvec_ool_arg_zpz, compact_fns[a->esz], a, 0)
 
 /* Call the helper that computes the ARM LastActiveElement pseudocode
  * function, scaled by the element size.  This includes the not found
@@ -3004,44 +2960,19 @@ static bool trans_CPY_m_v(DisasContext *s, arg_rpr_esz *a)
     return true;
 }
 
-static bool trans_REVB(DisasContext *s, arg_rpr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        NULL,
-        gen_helper_sve_revb_h,
-        gen_helper_sve_revb_s,
-        gen_helper_sve_revb_d,
-    };
-    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
-}
+static gen_helper_gvec_3 * const revb_fns[4] = {
+    NULL,                  gen_helper_sve_revb_h,
+    gen_helper_sve_revb_s, gen_helper_sve_revb_d,
+};
+TRANS_FEAT(REVB, aa64_sve, gen_gvec_ool_arg_zpz, revb_fns[a->esz], a, 0)
 
-static bool trans_REVH(DisasContext *s, arg_rpr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        NULL,
-        NULL,
-        gen_helper_sve_revh_s,
-        gen_helper_sve_revh_d,
-    };
-    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
-}
+static gen_helper_gvec_3 * const revh_fns[4] = {
+    NULL, NULL, gen_helper_sve_revh_s, gen_helper_sve_revh_d,
+};
+TRANS_FEAT(REVH, aa64_sve, gen_gvec_ool_arg_zpz, revh_fns[a->esz], a, 0)
 
-static bool trans_REVW(DisasContext *s, arg_rpr_esz *a)
-{
-    return gen_gvec_ool_arg_zpz(s, a->esz == 3 ? gen_helper_sve_revw_d
-                                : NULL, a, 0);
-}
-
-static bool trans_RBIT(DisasContext *s, arg_rpr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve_rbit_b,
-        gen_helper_sve_rbit_h,
-        gen_helper_sve_rbit_s,
-        gen_helper_sve_rbit_d,
-    };
-    return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0);
-}
+TRANS_FEAT(REVW, aa64_sve, gen_gvec_ool_arg_zpz,
+           a->esz == 3 ? gen_helper_sve_revw_d : NULL, a, 0)
 
 static bool trans_SPLICE(DisasContext *s, arg_rprr_esz *a)
 {

From b2c0096119836008b61c21e1714ec1d90d4a0ec2 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:35 -0700
Subject: [PATCH 607/935] target/arm: Use TRANS_FEAT for do_sve2_zpz_data

Convert SVE translation functions using do_sve2_zpz_data
to use TRANS_FEAT and gen_gvec_ool_arg_zpz.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-23-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 53 ++++++++++----------------------------
 1 file changed, 14 insertions(+), 39 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index d5a09a60af..fe11cfed6b 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -6554,48 +6554,23 @@ static bool trans_UADALP_zpzz(DisasContext *s, arg_rprr_esz *a)
  * SVE2 integer unary operations (predicated)
  */
 
-static bool do_sve2_zpz_ool(DisasContext *s, arg_rpr_esz *a,
-                            gen_helper_gvec_3 *fn)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zpz(s, fn, a, 0);
-}
+TRANS_FEAT(URECPE, aa64_sve2, gen_gvec_ool_arg_zpz,
+           a->esz == 2 ? gen_helper_sve2_urecpe_s : NULL, a, 0)
 
-static bool trans_URECPE(DisasContext *s, arg_rpr_esz *a)
-{
-    if (a->esz != 2) {
-        return false;
-    }
-    return do_sve2_zpz_ool(s, a, gen_helper_sve2_urecpe_s);
-}
+TRANS_FEAT(URSQRTE, aa64_sve2, gen_gvec_ool_arg_zpz,
+           a->esz == 2 ? gen_helper_sve2_ursqrte_s : NULL, a, 0)
 
-static bool trans_URSQRTE(DisasContext *s, arg_rpr_esz *a)
-{
-    if (a->esz != 2) {
-        return false;
-    }
-    return do_sve2_zpz_ool(s, a, gen_helper_sve2_ursqrte_s);
-}
+static gen_helper_gvec_3 * const sqabs_fns[4] = {
+    gen_helper_sve2_sqabs_b, gen_helper_sve2_sqabs_h,
+    gen_helper_sve2_sqabs_s, gen_helper_sve2_sqabs_d,
+};
+TRANS_FEAT(SQABS, aa64_sve2, gen_gvec_ool_arg_zpz, sqabs_fns[a->esz], a, 0)
 
-static bool trans_SQABS(DisasContext *s, arg_rpr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve2_sqabs_b, gen_helper_sve2_sqabs_h,
-        gen_helper_sve2_sqabs_s, gen_helper_sve2_sqabs_d,
-    };
-    return do_sve2_zpz_ool(s, a, fns[a->esz]);
-}
-
-static bool trans_SQNEG(DisasContext *s, arg_rpr_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve2_sqneg_b, gen_helper_sve2_sqneg_h,
-        gen_helper_sve2_sqneg_s, gen_helper_sve2_sqneg_d,
-    };
-    return do_sve2_zpz_ool(s, a, fns[a->esz]);
-}
+static gen_helper_gvec_3 * const sqneg_fns[4] = {
+    gen_helper_sve2_sqneg_b, gen_helper_sve2_sqneg_h,
+    gen_helper_sve2_sqneg_s, gen_helper_sve2_sqneg_d,
+};
+TRANS_FEAT(SQNEG, aa64_sve2, gen_gvec_ool_arg_zpz, sqneg_fns[a->esz], a, 0)
 
 #define DO_SVE2_ZPZZ(NAME, name) \
 static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a)                \

From afa2529c263502c1a0fbe0e9fa23a80c174392d1 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:36 -0700
Subject: [PATCH 608/935] target/arm: Rename do_zpzi_ool to
 gen_gvec_ool_arg_zpzi

Rename the function to match gen_gvec_ool_arg_zpz,
and move to be adjacent.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-24-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 29 ++++++++++++++---------------
 1 file changed, 14 insertions(+), 15 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index fe11cfed6b..86e87a2078 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -228,6 +228,11 @@ static bool gen_gvec_ool_arg_zpz(DisasContext *s, gen_helper_gvec_3 *fn,
     return gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, data);
 }
 
+static bool gen_gvec_ool_arg_zpzi(DisasContext *s, gen_helper_gvec_3 *fn,
+                                  arg_rpri_esz *a)
+{
+    return gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, a->imm);
+}
 
 /* Invoke an out-of-line helper on 3 Zregs and a predicate. */
 static void gen_gvec_ool_zzzp(DisasContext *s, gen_helper_gvec_4 *fn,
@@ -952,12 +957,6 @@ static bool do_movz_zpz(DisasContext *s, int rd, int rn, int pg,
     return gen_gvec_ool_zzp(s, fns[esz], rd, rn, pg, invert);
 }
 
-static bool do_zpzi_ool(DisasContext *s, arg_rpri_esz *a,
-                        gen_helper_gvec_3 *fn)
-{
-    return gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, a->imm);
-}
-
 static bool trans_ASR_zpzi(DisasContext *s, arg_rpri_esz *a)
 {
     static gen_helper_gvec_3 * const fns[4] = {
@@ -971,7 +970,7 @@ static bool trans_ASR_zpzi(DisasContext *s, arg_rpri_esz *a)
     /* Shift by element size is architecturally valid.  For
        arithmetic right-shift, it's the same as by one less. */
     a->imm = MIN(a->imm, (8 << a->esz) - 1);
-    return do_zpzi_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
 }
 
 static bool trans_LSR_zpzi(DisasContext *s, arg_rpri_esz *a)
@@ -988,7 +987,7 @@ static bool trans_LSR_zpzi(DisasContext *s, arg_rpri_esz *a)
     if (a->imm >= (8 << a->esz)) {
         return do_movz_zpz(s, a->rd, a->rd, a->pg, a->esz, true);
     } else {
-        return do_zpzi_ool(s, a, fns[a->esz]);
+        return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
     }
 }
 
@@ -1006,7 +1005,7 @@ static bool trans_LSL_zpzi(DisasContext *s, arg_rpri_esz *a)
     if (a->imm >= (8 << a->esz)) {
         return do_movz_zpz(s, a->rd, a->rd, a->pg, a->esz, true);
     } else {
-        return do_zpzi_ool(s, a, fns[a->esz]);
+        return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
     }
 }
 
@@ -1024,7 +1023,7 @@ static bool trans_ASRD(DisasContext *s, arg_rpri_esz *a)
     if (a->imm >= (8 << a->esz)) {
         return do_movz_zpz(s, a->rd, a->rd, a->pg, a->esz, true);
     } else {
-        return do_zpzi_ool(s, a, fns[a->esz]);
+        return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
     }
 }
 
@@ -1037,7 +1036,7 @@ static bool trans_SQSHL_zpzi(DisasContext *s, arg_rpri_esz *a)
     if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    return do_zpzi_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
 }
 
 static bool trans_UQSHL_zpzi(DisasContext *s, arg_rpri_esz *a)
@@ -1049,7 +1048,7 @@ static bool trans_UQSHL_zpzi(DisasContext *s, arg_rpri_esz *a)
     if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    return do_zpzi_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
 }
 
 static bool trans_SRSHR(DisasContext *s, arg_rpri_esz *a)
@@ -1061,7 +1060,7 @@ static bool trans_SRSHR(DisasContext *s, arg_rpri_esz *a)
     if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    return do_zpzi_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
 }
 
 static bool trans_URSHR(DisasContext *s, arg_rpri_esz *a)
@@ -1073,7 +1072,7 @@ static bool trans_URSHR(DisasContext *s, arg_rpri_esz *a)
     if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    return do_zpzi_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
 }
 
 static bool trans_SQSHLU(DisasContext *s, arg_rpri_esz *a)
@@ -1085,7 +1084,7 @@ static bool trans_SQSHLU(DisasContext *s, arg_rpri_esz *a)
     if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    return do_zpzi_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
 }
 
 /*

From 4df37e414d812a4beb757fbb3f9db107b5451f7f Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:37 -0700
Subject: [PATCH 609/935] target/arm: Use TRANS_FEAT for gen_gvec_ool_arg_zpzi

Convert some SVE translation functions using
gen_gvec_ool_arg_zpzi to TRANS_FEAT.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-25-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 85 ++++++++++++++------------------------
 1 file changed, 30 insertions(+), 55 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 86e87a2078..10614bf915 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -1027,65 +1027,40 @@ static bool trans_ASRD(DisasContext *s, arg_rpri_esz *a)
     }
 }
 
-static bool trans_SQSHL_zpzi(DisasContext *s, arg_rpri_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve2_sqshl_zpzi_b, gen_helper_sve2_sqshl_zpzi_h,
-        gen_helper_sve2_sqshl_zpzi_s, gen_helper_sve2_sqshl_zpzi_d,
-    };
-    if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
-}
+static gen_helper_gvec_3 * const sqshl_zpzi_fns[4] = {
+    gen_helper_sve2_sqshl_zpzi_b, gen_helper_sve2_sqshl_zpzi_h,
+    gen_helper_sve2_sqshl_zpzi_s, gen_helper_sve2_sqshl_zpzi_d,
+};
+TRANS_FEAT(SQSHL_zpzi, aa64_sve2, gen_gvec_ool_arg_zpzi,
+           a->esz < 0 ? NULL : sqshl_zpzi_fns[a->esz], a)
 
-static bool trans_UQSHL_zpzi(DisasContext *s, arg_rpri_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve2_uqshl_zpzi_b, gen_helper_sve2_uqshl_zpzi_h,
-        gen_helper_sve2_uqshl_zpzi_s, gen_helper_sve2_uqshl_zpzi_d,
-    };
-    if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
-}
+static gen_helper_gvec_3 * const uqshl_zpzi_fns[4] = {
+    gen_helper_sve2_uqshl_zpzi_b, gen_helper_sve2_uqshl_zpzi_h,
+    gen_helper_sve2_uqshl_zpzi_s, gen_helper_sve2_uqshl_zpzi_d,
+};
+TRANS_FEAT(UQSHL_zpzi, aa64_sve2, gen_gvec_ool_arg_zpzi,
+           a->esz < 0 ? NULL : uqshl_zpzi_fns[a->esz], a)
 
-static bool trans_SRSHR(DisasContext *s, arg_rpri_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve2_srshr_b, gen_helper_sve2_srshr_h,
-        gen_helper_sve2_srshr_s, gen_helper_sve2_srshr_d,
-    };
-    if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
-}
+static gen_helper_gvec_3 * const srshr_fns[4] = {
+    gen_helper_sve2_srshr_b, gen_helper_sve2_srshr_h,
+    gen_helper_sve2_srshr_s, gen_helper_sve2_srshr_d,
+};
+TRANS_FEAT(SRSHR, aa64_sve2, gen_gvec_ool_arg_zpzi,
+           a->esz < 0 ? NULL : srshr_fns[a->esz], a)
 
-static bool trans_URSHR(DisasContext *s, arg_rpri_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve2_urshr_b, gen_helper_sve2_urshr_h,
-        gen_helper_sve2_urshr_s, gen_helper_sve2_urshr_d,
-    };
-    if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
-}
+static gen_helper_gvec_3 * const urshr_fns[4] = {
+    gen_helper_sve2_urshr_b, gen_helper_sve2_urshr_h,
+    gen_helper_sve2_urshr_s, gen_helper_sve2_urshr_d,
+};
+TRANS_FEAT(URSHR, aa64_sve2, gen_gvec_ool_arg_zpzi,
+           a->esz < 0 ? NULL : urshr_fns[a->esz], a)
 
-static bool trans_SQSHLU(DisasContext *s, arg_rpri_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve2_sqshlu_b, gen_helper_sve2_sqshlu_h,
-        gen_helper_sve2_sqshlu_s, gen_helper_sve2_sqshlu_d,
-    };
-    if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
-}
+static gen_helper_gvec_3 * const sqshlu_fns[4] = {
+    gen_helper_sve2_sqshlu_b, gen_helper_sve2_sqshlu_h,
+    gen_helper_sve2_sqshlu_s, gen_helper_sve2_sqshlu_d,
+};
+TRANS_FEAT(SQSHLU, aa64_sve2, gen_gvec_ool_arg_zpzi,
+           a->esz < 0 ? NULL : sqshlu_fns[a->esz], a)
 
 /*
  *** SVE Bitwise Shift - Predicated Group

From 2a753d1e1dd65c093440e3ecd079da624ebd64b3 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:38 -0700
Subject: [PATCH 610/935] target/arm: Move null function and sve check into
 gen_gvec_ool_zzzp

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-26-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 42 ++++++++++++++++----------------------
 1 file changed, 18 insertions(+), 24 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 10614bf915..fea7164d72 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -235,15 +235,21 @@ static bool gen_gvec_ool_arg_zpzi(DisasContext *s, gen_helper_gvec_3 *fn,
 }
 
 /* Invoke an out-of-line helper on 3 Zregs and a predicate. */
-static void gen_gvec_ool_zzzp(DisasContext *s, gen_helper_gvec_4 *fn,
+static bool gen_gvec_ool_zzzp(DisasContext *s, gen_helper_gvec_4 *fn,
                               int rd, int rn, int rm, int pg, int data)
 {
-    unsigned vsz = vec_full_reg_size(s);
-    tcg_gen_gvec_4_ool(vec_full_reg_offset(s, rd),
-                       vec_full_reg_offset(s, rn),
-                       vec_full_reg_offset(s, rm),
-                       pred_full_reg_offset(s, pg),
-                       vsz, vsz, data, fn);
+    if (fn == NULL) {
+        return false;
+    }
+    if (sve_access_check(s)) {
+        unsigned vsz = vec_full_reg_size(s);
+        tcg_gen_gvec_4_ool(vec_full_reg_offset(s, rd),
+                           vec_full_reg_offset(s, rn),
+                           vec_full_reg_offset(s, rm),
+                           pred_full_reg_offset(s, pg),
+                           vsz, vsz, data, fn);
+    }
+    return true;
 }
 
 /* Invoke a vector expander on two Zregs.  */
@@ -733,13 +739,7 @@ static bool trans_UQSUB_zzz(DisasContext *s, arg_rrr_esz *a)
 
 static bool do_zpzz_ool(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4 *fn)
 {
-    if (fn == NULL) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzzp(s, fn, a->rd, a->rn, a->rm, a->pg, 0);
-    }
-    return true;
+    return gen_gvec_ool_zzzp(s, fn, a->rd, a->rn, a->rm, a->pg, 0);
 }
 
 /* Select active elememnts from Zn and inactive elements from Zm,
@@ -2950,11 +2950,8 @@ TRANS_FEAT(REVW, aa64_sve, gen_gvec_ool_arg_zpz,
 
 static bool trans_SPLICE(DisasContext *s, arg_rprr_esz *a)
 {
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzzp(s, gen_helper_sve_splice,
-                          a->rd, a->rn, a->rm, a->pg, a->esz);
-    }
-    return true;
+    return gen_gvec_ool_zzzp(s, gen_helper_sve_splice,
+                             a->rd, a->rn, a->rm, a->pg, a->esz);
 }
 
 static bool trans_SPLICE_sve2(DisasContext *s, arg_rpr_esz *a)
@@ -2962,11 +2959,8 @@ static bool trans_SPLICE_sve2(DisasContext *s, arg_rpr_esz *a)
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_ool_zzzp(s, gen_helper_sve_splice,
-                          a->rd, a->rn, (a->rn + 1) % 32, a->pg, a->esz);
-    }
-    return true;
+    return gen_gvec_ool_zzzp(s, gen_helper_sve_splice,
+                             a->rd, a->rn, (a->rn + 1) % 32, a->pg, a->esz);
 }
 
 /*

From 312016c96a914aa6440f7d9f0fca146426619eca Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:39 -0700
Subject: [PATCH 611/935] target/arm: Introduce gen_gvec_ool_arg_zpzz

Use gen_gvec_ool_arg_zpzz instead of gen_gvec_ool_zzzp
when the arguments come from arg_rprr_esz.
Replaces do_zpzz_ool.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-27-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index fea7164d72..d63099c5b7 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -252,6 +252,12 @@ static bool gen_gvec_ool_zzzp(DisasContext *s, gen_helper_gvec_4 *fn,
     return true;
 }
 
+static bool gen_gvec_ool_arg_zpzz(DisasContext *s, gen_helper_gvec_4 *fn,
+                                  arg_rprr_esz *a, int data)
+{
+    return gen_gvec_ool_zzzp(s, fn, a->rd, a->rn, a->rm, a->pg, data);
+}
+
 /* Invoke a vector expander on two Zregs.  */
 static void gen_gvec_fn_zz(DisasContext *s, GVecGen2Fn *gvec_fn,
                            int esz, int rd, int rn)
@@ -737,11 +743,6 @@ static bool trans_UQSUB_zzz(DisasContext *s, arg_rrr_esz *a)
  *** SVE Integer Arithmetic - Binary Predicated Group
  */
 
-static bool do_zpzz_ool(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4 *fn)
-{
-    return gen_gvec_ool_zzzp(s, fn, a->rd, a->rn, a->rm, a->pg, 0);
-}
-
 /* Select active elememnts from Zn and inactive elements from Zm,
  * storing the result in Zd.
  */
@@ -761,7 +762,7 @@ static bool trans_##NAME##_zpzz(DisasContext *s, arg_rprr_esz *a)         \
         gen_helper_sve_##name##_zpzz_b, gen_helper_sve_##name##_zpzz_h,   \
         gen_helper_sve_##name##_zpzz_s, gen_helper_sve_##name##_zpzz_d,   \
     };                                                                    \
-    return do_zpzz_ool(s, a, fns[a->esz]);                                \
+    return gen_gvec_ool_arg_zpzz(s, fns[a->esz], a, 0);                   \
 }
 
 DO_ZPZZ(AND, and)
@@ -792,7 +793,7 @@ static bool trans_SDIV_zpzz(DisasContext *s, arg_rprr_esz *a)
     static gen_helper_gvec_4 * const fns[4] = {
         NULL, NULL, gen_helper_sve_sdiv_zpzz_s, gen_helper_sve_sdiv_zpzz_d
     };
-    return do_zpzz_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpzz(s, fns[a->esz], a, 0);
 }
 
 static bool trans_UDIV_zpzz(DisasContext *s, arg_rprr_esz *a)
@@ -800,7 +801,7 @@ static bool trans_UDIV_zpzz(DisasContext *s, arg_rprr_esz *a)
     static gen_helper_gvec_4 * const fns[4] = {
         NULL, NULL, gen_helper_sve_udiv_zpzz_s, gen_helper_sve_udiv_zpzz_d
     };
-    return do_zpzz_ool(s, a, fns[a->esz]);
+    return gen_gvec_ool_arg_zpzz(s, fns[a->esz], a, 0);
 }
 
 static bool trans_SEL_zpzz(DisasContext *s, arg_rprr_esz *a)
@@ -1076,7 +1077,7 @@ static bool trans_##NAME##_zpzw(DisasContext *s, arg_rprr_esz *a)         \
     if (a->esz < 0 || a->esz >= 3) {                                      \
         return false;                                                     \
     }                                                                     \
-    return do_zpzz_ool(s, a, fns[a->esz]);                                \
+    return gen_gvec_ool_arg_zpzz(s, fns[a->esz], a, 0);                   \
 }
 
 DO_ZPZW(ASR, asr)
@@ -6489,7 +6490,7 @@ static bool do_sve2_zpzz_ool(DisasContext *s, arg_rprr_esz *a,
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    return do_zpzz_ool(s, a, fn);
+    return gen_gvec_ool_arg_zpzz(s, fn, a, 0);
 }
 
 static bool trans_SADALP_zpzz(DisasContext *s, arg_rprr_esz *a)

From 8e7acb2443031d6395c18de09406803e9d3f3ac4 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:40 -0700
Subject: [PATCH 612/935] target/arm: Use TRANS_FEAT for gen_gvec_ool_arg_zpzz

Convert SVE translation functions directly using
gen_gvec_ool_arg_zpzz to TRANS_FEAT.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-28-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 85 ++++++++++++++++----------------------
 1 file changed, 36 insertions(+), 49 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index d63099c5b7..f8277eeb7c 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -755,54 +755,46 @@ static void do_sel_z(DisasContext *s, int rd, int rn, int rm, int pg, int esz)
     gen_gvec_ool_zzzp(s, fns[esz], rd, rn, rm, pg, 0);
 }
 
-#define DO_ZPZZ(NAME, name) \
-static bool trans_##NAME##_zpzz(DisasContext *s, arg_rprr_esz *a)         \
-{                                                                         \
-    static gen_helper_gvec_4 * const fns[4] = {                           \
-        gen_helper_sve_##name##_zpzz_b, gen_helper_sve_##name##_zpzz_h,   \
-        gen_helper_sve_##name##_zpzz_s, gen_helper_sve_##name##_zpzz_d,   \
+#define DO_ZPZZ(NAME, FEAT, name) \
+    static gen_helper_gvec_4 * const name##_zpzz_fns[4] = {               \
+        gen_helper_##name##_zpzz_b, gen_helper_##name##_zpzz_h,           \
+        gen_helper_##name##_zpzz_s, gen_helper_##name##_zpzz_d,           \
     };                                                                    \
-    return gen_gvec_ool_arg_zpzz(s, fns[a->esz], a, 0);                   \
-}
+    TRANS_FEAT(NAME, FEAT, gen_gvec_ool_arg_zpzz,                         \
+               name##_zpzz_fns[a->esz], a, 0)
 
-DO_ZPZZ(AND, and)
-DO_ZPZZ(EOR, eor)
-DO_ZPZZ(ORR, orr)
-DO_ZPZZ(BIC, bic)
+DO_ZPZZ(AND_zpzz, aa64_sve, sve_and)
+DO_ZPZZ(EOR_zpzz, aa64_sve, sve_eor)
+DO_ZPZZ(ORR_zpzz, aa64_sve, sve_orr)
+DO_ZPZZ(BIC_zpzz, aa64_sve, sve_bic)
 
-DO_ZPZZ(ADD, add)
-DO_ZPZZ(SUB, sub)
+DO_ZPZZ(ADD_zpzz, aa64_sve, sve_add)
+DO_ZPZZ(SUB_zpzz, aa64_sve, sve_sub)
 
-DO_ZPZZ(SMAX, smax)
-DO_ZPZZ(UMAX, umax)
-DO_ZPZZ(SMIN, smin)
-DO_ZPZZ(UMIN, umin)
-DO_ZPZZ(SABD, sabd)
-DO_ZPZZ(UABD, uabd)
+DO_ZPZZ(SMAX_zpzz, aa64_sve, sve_smax)
+DO_ZPZZ(UMAX_zpzz, aa64_sve, sve_umax)
+DO_ZPZZ(SMIN_zpzz, aa64_sve, sve_smin)
+DO_ZPZZ(UMIN_zpzz, aa64_sve, sve_umin)
+DO_ZPZZ(SABD_zpzz, aa64_sve, sve_sabd)
+DO_ZPZZ(UABD_zpzz, aa64_sve, sve_uabd)
 
-DO_ZPZZ(MUL, mul)
-DO_ZPZZ(SMULH, smulh)
-DO_ZPZZ(UMULH, umulh)
+DO_ZPZZ(MUL_zpzz, aa64_sve, sve_mul)
+DO_ZPZZ(SMULH_zpzz, aa64_sve, sve_smulh)
+DO_ZPZZ(UMULH_zpzz, aa64_sve, sve_umulh)
 
-DO_ZPZZ(ASR, asr)
-DO_ZPZZ(LSR, lsr)
-DO_ZPZZ(LSL, lsl)
+DO_ZPZZ(ASR_zpzz, aa64_sve, sve_asr)
+DO_ZPZZ(LSR_zpzz, aa64_sve, sve_lsr)
+DO_ZPZZ(LSL_zpzz, aa64_sve, sve_lsl)
 
-static bool trans_SDIV_zpzz(DisasContext *s, arg_rprr_esz *a)
-{
-    static gen_helper_gvec_4 * const fns[4] = {
-        NULL, NULL, gen_helper_sve_sdiv_zpzz_s, gen_helper_sve_sdiv_zpzz_d
-    };
-    return gen_gvec_ool_arg_zpzz(s, fns[a->esz], a, 0);
-}
+static gen_helper_gvec_4 * const sdiv_fns[4] = {
+    NULL, NULL, gen_helper_sve_sdiv_zpzz_s, gen_helper_sve_sdiv_zpzz_d
+};
+TRANS_FEAT(SDIV_zpzz, aa64_sve, gen_gvec_ool_arg_zpzz, sdiv_fns[a->esz], a, 0)
 
-static bool trans_UDIV_zpzz(DisasContext *s, arg_rprr_esz *a)
-{
-    static gen_helper_gvec_4 * const fns[4] = {
-        NULL, NULL, gen_helper_sve_udiv_zpzz_s, gen_helper_sve_udiv_zpzz_d
-    };
-    return gen_gvec_ool_arg_zpzz(s, fns[a->esz], a, 0);
-}
+static gen_helper_gvec_4 * const udiv_fns[4] = {
+    NULL, NULL, gen_helper_sve_udiv_zpzz_s, gen_helper_sve_udiv_zpzz_d
+};
+TRANS_FEAT(UDIV_zpzz, aa64_sve, gen_gvec_ool_arg_zpzz, udiv_fns[a->esz], a, 0)
 
 static bool trans_SEL_zpzz(DisasContext *s, arg_rprr_esz *a)
 {
@@ -1068,17 +1060,12 @@ TRANS_FEAT(SQSHLU, aa64_sve2, gen_gvec_ool_arg_zpzi,
  */
 
 #define DO_ZPZW(NAME, name) \
-static bool trans_##NAME##_zpzw(DisasContext *s, arg_rprr_esz *a)         \
-{                                                                         \
-    static gen_helper_gvec_4 * const fns[3] = {                           \
+    static gen_helper_gvec_4 * const name##_zpzw_fns[4] = {               \
         gen_helper_sve_##name##_zpzw_b, gen_helper_sve_##name##_zpzw_h,   \
-        gen_helper_sve_##name##_zpzw_s,                                   \
+        gen_helper_sve_##name##_zpzw_s, NULL                              \
     };                                                                    \
-    if (a->esz < 0 || a->esz >= 3) {                                      \
-        return false;                                                     \
-    }                                                                     \
-    return gen_gvec_ool_arg_zpzz(s, fns[a->esz], a, 0);                   \
-}
+    TRANS_FEAT(NAME##_zpzw, aa64_sve, gen_gvec_ool_arg_zpzz,              \
+               a->esz < 0 ? NULL : name##_zpzw_fns[a->esz], a, 0)
 
 DO_ZPZW(ASR, asr)
 DO_ZPZW(LSR, lsr)

From 5880bdc03e4ee90b08c1e8242254920d1fac7512 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:41 -0700
Subject: [PATCH 613/935] target/arm: Use TRANS_FEAT for do_sve2_zpzz_ool

Convert SVE translation functions using do_sve2_zpzz_ool
to use TRANS_FEAT and gen_gvec_ool_arg_zpzz.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-29-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 118 +++++++++++++------------------------
 1 file changed, 40 insertions(+), 78 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index f8277eeb7c..0a69a1ef65 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -804,8 +804,6 @@ static bool trans_SEL_zpzz(DisasContext *s, arg_rprr_esz *a)
     return true;
 }
 
-#undef DO_ZPZZ
-
 /*
  *** SVE Integer Arithmetic - Unary Predicated Group
  */
@@ -6471,40 +6469,19 @@ TRANS_FEAT(SQRDMULH_zzz, aa64_sve2, gen_gvec_ool_arg_zzz,
  * SVE2 Integer - Predicated
  */
 
-static bool do_sve2_zpzz_ool(DisasContext *s, arg_rprr_esz *a,
-                             gen_helper_gvec_4 *fn)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zpzz(s, fn, a, 0);
-}
+static gen_helper_gvec_4 * const sadlp_fns[4] = {
+    NULL,                          gen_helper_sve2_sadalp_zpzz_h,
+    gen_helper_sve2_sadalp_zpzz_s, gen_helper_sve2_sadalp_zpzz_d,
+};
+TRANS_FEAT(SADALP_zpzz, aa64_sve2, gen_gvec_ool_arg_zpzz,
+           sadlp_fns[a->esz], a, 0)
 
-static bool trans_SADALP_zpzz(DisasContext *s, arg_rprr_esz *a)
-{
-    static gen_helper_gvec_4 * const fns[3] = {
-        gen_helper_sve2_sadalp_zpzz_h,
-        gen_helper_sve2_sadalp_zpzz_s,
-        gen_helper_sve2_sadalp_zpzz_d,
-    };
-    if (a->esz == 0) {
-        return false;
-    }
-    return do_sve2_zpzz_ool(s, a, fns[a->esz - 1]);
-}
-
-static bool trans_UADALP_zpzz(DisasContext *s, arg_rprr_esz *a)
-{
-    static gen_helper_gvec_4 * const fns[3] = {
-        gen_helper_sve2_uadalp_zpzz_h,
-        gen_helper_sve2_uadalp_zpzz_s,
-        gen_helper_sve2_uadalp_zpzz_d,
-    };
-    if (a->esz == 0) {
-        return false;
-    }
-    return do_sve2_zpzz_ool(s, a, fns[a->esz - 1]);
-}
+static gen_helper_gvec_4 * const uadlp_fns[4] = {
+    NULL,                          gen_helper_sve2_uadalp_zpzz_h,
+    gen_helper_sve2_uadalp_zpzz_s, gen_helper_sve2_uadalp_zpzz_d,
+};
+TRANS_FEAT(UADALP_zpzz, aa64_sve2, gen_gvec_ool_arg_zpzz,
+           uadlp_fns[a->esz], a, 0)
 
 /*
  * SVE2 integer unary operations (predicated)
@@ -6528,44 +6505,34 @@ static gen_helper_gvec_3 * const sqneg_fns[4] = {
 };
 TRANS_FEAT(SQNEG, aa64_sve2, gen_gvec_ool_arg_zpz, sqneg_fns[a->esz], a, 0)
 
-#define DO_SVE2_ZPZZ(NAME, name) \
-static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a)                \
-{                                                                         \
-    static gen_helper_gvec_4 * const fns[4] = {                           \
-        gen_helper_sve2_##name##_zpzz_b, gen_helper_sve2_##name##_zpzz_h, \
-        gen_helper_sve2_##name##_zpzz_s, gen_helper_sve2_##name##_zpzz_d, \
-    };                                                                    \
-    return do_sve2_zpzz_ool(s, a, fns[a->esz]);                           \
-}
+DO_ZPZZ(SQSHL, aa64_sve2, sve2_sqshl)
+DO_ZPZZ(SQRSHL, aa64_sve2, sve2_sqrshl)
+DO_ZPZZ(SRSHL, aa64_sve2, sve2_srshl)
 
-DO_SVE2_ZPZZ(SQSHL, sqshl)
-DO_SVE2_ZPZZ(SQRSHL, sqrshl)
-DO_SVE2_ZPZZ(SRSHL, srshl)
+DO_ZPZZ(UQSHL, aa64_sve2, sve2_uqshl)
+DO_ZPZZ(UQRSHL, aa64_sve2, sve2_uqrshl)
+DO_ZPZZ(URSHL, aa64_sve2, sve2_urshl)
 
-DO_SVE2_ZPZZ(UQSHL, uqshl)
-DO_SVE2_ZPZZ(UQRSHL, uqrshl)
-DO_SVE2_ZPZZ(URSHL, urshl)
+DO_ZPZZ(SHADD, aa64_sve2, sve2_shadd)
+DO_ZPZZ(SRHADD, aa64_sve2, sve2_srhadd)
+DO_ZPZZ(SHSUB, aa64_sve2, sve2_shsub)
 
-DO_SVE2_ZPZZ(SHADD, shadd)
-DO_SVE2_ZPZZ(SRHADD, srhadd)
-DO_SVE2_ZPZZ(SHSUB, shsub)
+DO_ZPZZ(UHADD, aa64_sve2, sve2_uhadd)
+DO_ZPZZ(URHADD, aa64_sve2, sve2_urhadd)
+DO_ZPZZ(UHSUB, aa64_sve2, sve2_uhsub)
 
-DO_SVE2_ZPZZ(UHADD, uhadd)
-DO_SVE2_ZPZZ(URHADD, urhadd)
-DO_SVE2_ZPZZ(UHSUB, uhsub)
+DO_ZPZZ(ADDP, aa64_sve2, sve2_addp)
+DO_ZPZZ(SMAXP, aa64_sve2, sve2_smaxp)
+DO_ZPZZ(UMAXP, aa64_sve2, sve2_umaxp)
+DO_ZPZZ(SMINP, aa64_sve2, sve2_sminp)
+DO_ZPZZ(UMINP, aa64_sve2, sve2_uminp)
 
-DO_SVE2_ZPZZ(ADDP, addp)
-DO_SVE2_ZPZZ(SMAXP, smaxp)
-DO_SVE2_ZPZZ(UMAXP, umaxp)
-DO_SVE2_ZPZZ(SMINP, sminp)
-DO_SVE2_ZPZZ(UMINP, uminp)
-
-DO_SVE2_ZPZZ(SQADD_zpzz, sqadd)
-DO_SVE2_ZPZZ(UQADD_zpzz, uqadd)
-DO_SVE2_ZPZZ(SQSUB_zpzz, sqsub)
-DO_SVE2_ZPZZ(UQSUB_zpzz, uqsub)
-DO_SVE2_ZPZZ(SUQADD, suqadd)
-DO_SVE2_ZPZZ(USQADD, usqadd)
+DO_ZPZZ(SQADD_zpzz, aa64_sve2, sve2_sqadd)
+DO_ZPZZ(UQADD_zpzz, aa64_sve2, sve2_uqadd)
+DO_ZPZZ(SQSUB_zpzz, aa64_sve2, sve2_sqsub)
+DO_ZPZZ(UQSUB_zpzz, aa64_sve2, sve2_uqsub)
+DO_ZPZZ(SUQADD, aa64_sve2, sve2_suqadd)
+DO_ZPZZ(USQADD, aa64_sve2, sve2_usqadd)
 
 /*
  * SVE2 Widening Integer Arithmetic
@@ -7735,16 +7702,11 @@ static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a)                  \
 DO_SVE2_PPZZ_MATCH(MATCH, match)
 DO_SVE2_PPZZ_MATCH(NMATCH, nmatch)
 
-static bool trans_HISTCNT(DisasContext *s, arg_rprr_esz *a)
-{
-    static gen_helper_gvec_4 * const fns[2] = {
-        gen_helper_sve2_histcnt_s, gen_helper_sve2_histcnt_d
-    };
-    if (a->esz < 2) {
-        return false;
-    }
-    return do_sve2_zpzz_ool(s, a, fns[a->esz - 2]);
-}
+static gen_helper_gvec_4 * const histcnt_fns[4] = {
+    NULL, NULL, gen_helper_sve2_histcnt_s, gen_helper_sve2_histcnt_d
+};
+TRANS_FEAT(HISTCNT, aa64_sve2, gen_gvec_ool_arg_zpzz,
+           histcnt_fns[a->esz], a, 0)
 
 TRANS_FEAT(HISTSEG, aa64_sve2, gen_gvec_ool_arg_zzz,
            a->esz == 0 ? gen_helper_sve2_histseg : NULL, a, 0)

From 5f730621ea285cc1f45c03baac8adb86632e0904 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:42 -0700
Subject: [PATCH 614/935] target/arm: Merge gen_gvec_fn_zz into do_mov_z

There is only one caller for gen_gvec_fn_zz; inline it.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-30-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 13 +++----------
 1 file changed, 3 insertions(+), 10 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 0a69a1ef65..5ab9de46a7 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -258,15 +258,6 @@ static bool gen_gvec_ool_arg_zpzz(DisasContext *s, gen_helper_gvec_4 *fn,
     return gen_gvec_ool_zzzp(s, fn, a->rd, a->rn, a->rm, a->pg, data);
 }
 
-/* Invoke a vector expander on two Zregs.  */
-static void gen_gvec_fn_zz(DisasContext *s, GVecGen2Fn *gvec_fn,
-                           int esz, int rd, int rn)
-{
-    unsigned vsz = vec_full_reg_size(s);
-    gvec_fn(esz, vec_full_reg_offset(s, rd),
-            vec_full_reg_offset(s, rn), vsz, vsz);
-}
-
 /* Invoke a vector expander on three Zregs.  */
 static void gen_gvec_fn_zzz(DisasContext *s, GVecGen3Fn *gvec_fn,
                             int esz, int rd, int rn, int rm)
@@ -292,7 +283,9 @@ static void gen_gvec_fn_zzzz(DisasContext *s, GVecGen4Fn *gvec_fn,
 static bool do_mov_z(DisasContext *s, int rd, int rn)
 {
     if (sve_access_check(s)) {
-        gen_gvec_fn_zz(s, tcg_gen_gvec_mov, MO_8, rd, rn);
+        unsigned vsz = vec_full_reg_size(s);
+        tcg_gen_gvec_mov(MO_8, vec_full_reg_offset(s, rd),
+                         vec_full_reg_offset(s, rn), vsz, vsz);
     }
     return true;
 }

From 50f6db5f23e2a837a98a97cc388de9afc46fae2f Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:43 -0700
Subject: [PATCH 615/935] target/arm: Move null function and sve check into
 gen_gvec_fn_zzz

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-31-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 36 +++++++++++++++---------------------
 1 file changed, 15 insertions(+), 21 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 5ab9de46a7..3af4626e58 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -259,13 +259,19 @@ static bool gen_gvec_ool_arg_zpzz(DisasContext *s, gen_helper_gvec_4 *fn,
 }
 
 /* Invoke a vector expander on three Zregs.  */
-static void gen_gvec_fn_zzz(DisasContext *s, GVecGen3Fn *gvec_fn,
+static bool gen_gvec_fn_zzz(DisasContext *s, GVecGen3Fn *gvec_fn,
                             int esz, int rd, int rn, int rm)
 {
-    unsigned vsz = vec_full_reg_size(s);
-    gvec_fn(esz, vec_full_reg_offset(s, rd),
-            vec_full_reg_offset(s, rn),
-            vec_full_reg_offset(s, rm), vsz, vsz);
+    if (gvec_fn == NULL) {
+        return false;
+    }
+    if (sve_access_check(s)) {
+        unsigned vsz = vec_full_reg_size(s);
+        gvec_fn(esz, vec_full_reg_offset(s, rd),
+                vec_full_reg_offset(s, rn),
+                vec_full_reg_offset(s, rm), vsz, vsz);
+    }
+    return true;
 }
 
 /* Invoke a vector expander on four Zregs.  */
@@ -366,10 +372,7 @@ const uint64_t pred_esz_masks[4] = {
 
 static bool do_zzz_fn(DisasContext *s, arg_rrr_esz *a, GVecGen3Fn *gvec_fn)
 {
-    if (sve_access_check(s)) {
-        gen_gvec_fn_zzz(s, gvec_fn, a->esz, a->rd, a->rn, a->rm);
-    }
-    return true;
+    return gen_gvec_fn_zzz(s, gvec_fn, a->esz, a->rd, a->rn, a->rm);
 }
 
 static bool trans_AND_zzz(DisasContext *s, arg_rrr_esz *a)
@@ -6421,10 +6424,7 @@ static bool trans_MUL_zzz(DisasContext *s, arg_rrr_esz *a)
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_fn_zzz(s, tcg_gen_gvec_mul, a->esz, a->rd, a->rn, a->rm);
-    }
-    return true;
+    return gen_gvec_fn_zzz(s, tcg_gen_gvec_mul, a->esz, a->rd, a->rn, a->rm);
 }
 
 static gen_helper_gvec_3 * const smulh_zzz_fns[4] = {
@@ -6945,10 +6945,7 @@ static bool do_sve2_fn_zzz(DisasContext *s, arg_rrr_esz *a, GVecGen3Fn *fn)
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_fn_zzz(s, fn, a->esz, a->rd, a->rn, a->rm);
-    }
-    return true;
+    return gen_gvec_fn_zzz(s, fn, a->esz, a->rd, a->rn, a->rm);
 }
 
 static bool trans_SABA(DisasContext *s, arg_rrr_esz *a)
@@ -7880,10 +7877,7 @@ static bool trans_RAX1(DisasContext *s, arg_rrr_esz *a)
     if (!dc_isar_feature(aa64_sve2_sha3, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_fn_zzz(s, gen_gvec_rax1, MO_64, a->rd, a->rn, a->rm);
-    }
-    return true;
+    return gen_gvec_fn_zzz(s, gen_gvec_rax1, MO_64, a->rd, a->rn, a->rm);
 }
 
 static bool trans_FCVTNT_sh(DisasContext *s, arg_rpr_esz *a)

From cd54bbe66288a75073fbb67d12d1300876390b02 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:44 -0700
Subject: [PATCH 616/935] target/arm: Rename do_zzz_fn to gen_gvec_fn_arg_zzz

Rename the function to match gen_gvec_fn_zzz,
and move to be adjacent.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-32-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 31 ++++++++++++++++---------------
 1 file changed, 16 insertions(+), 15 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 3af4626e58..1b3afcc24c 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -274,6 +274,12 @@ static bool gen_gvec_fn_zzz(DisasContext *s, GVecGen3Fn *gvec_fn,
     return true;
 }
 
+static bool gen_gvec_fn_arg_zzz(DisasContext *s, GVecGen3Fn *fn,
+                                arg_rrr_esz *a)
+{
+    return gen_gvec_fn_zzz(s, fn, a->esz, a->rd, a->rn, a->rm);
+}
+
 /* Invoke a vector expander on four Zregs.  */
 static void gen_gvec_fn_zzzz(DisasContext *s, GVecGen4Fn *gvec_fn,
                              int esz, int rd, int rn, int rm, int ra)
@@ -370,29 +376,24 @@ const uint64_t pred_esz_masks[4] = {
  *** SVE Logical - Unpredicated Group
  */
 
-static bool do_zzz_fn(DisasContext *s, arg_rrr_esz *a, GVecGen3Fn *gvec_fn)
-{
-    return gen_gvec_fn_zzz(s, gvec_fn, a->esz, a->rd, a->rn, a->rm);
-}
-
 static bool trans_AND_zzz(DisasContext *s, arg_rrr_esz *a)
 {
-    return do_zzz_fn(s, a, tcg_gen_gvec_and);
+    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_and, a);
 }
 
 static bool trans_ORR_zzz(DisasContext *s, arg_rrr_esz *a)
 {
-    return do_zzz_fn(s, a, tcg_gen_gvec_or);
+    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_or, a);
 }
 
 static bool trans_EOR_zzz(DisasContext *s, arg_rrr_esz *a)
 {
-    return do_zzz_fn(s, a, tcg_gen_gvec_xor);
+    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_xor, a);
 }
 
 static bool trans_BIC_zzz(DisasContext *s, arg_rrr_esz *a)
 {
-    return do_zzz_fn(s, a, tcg_gen_gvec_andc);
+    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_andc, a);
 }
 
 static void gen_xar8_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, int64_t sh)
@@ -707,32 +708,32 @@ static bool trans_NBSL(DisasContext *s, arg_rrrr_esz *a)
 
 static bool trans_ADD_zzz(DisasContext *s, arg_rrr_esz *a)
 {
-    return do_zzz_fn(s, a, tcg_gen_gvec_add);
+    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_add, a);
 }
 
 static bool trans_SUB_zzz(DisasContext *s, arg_rrr_esz *a)
 {
-    return do_zzz_fn(s, a, tcg_gen_gvec_sub);
+    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_sub, a);
 }
 
 static bool trans_SQADD_zzz(DisasContext *s, arg_rrr_esz *a)
 {
-    return do_zzz_fn(s, a, tcg_gen_gvec_ssadd);
+    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_ssadd, a);
 }
 
 static bool trans_SQSUB_zzz(DisasContext *s, arg_rrr_esz *a)
 {
-    return do_zzz_fn(s, a, tcg_gen_gvec_sssub);
+    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_sssub, a);
 }
 
 static bool trans_UQADD_zzz(DisasContext *s, arg_rrr_esz *a)
 {
-    return do_zzz_fn(s, a, tcg_gen_gvec_usadd);
+    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_usadd, a);
 }
 
 static bool trans_UQSUB_zzz(DisasContext *s, arg_rrr_esz *a)
 {
-    return do_zzz_fn(s, a, tcg_gen_gvec_ussub);
+    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_ussub, a);
 }
 
 /*

From f96aae7649cca25b427b80d6cc9e67a617af6795 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:45 -0700
Subject: [PATCH 617/935] target/arm: More use of gen_gvec_fn_arg_zzz

Two uses of gen_gvec_fn_zzz can pass on arg_rrr_esz instead.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-33-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 1b3afcc24c..2dbf296128 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -6425,7 +6425,7 @@ static bool trans_MUL_zzz(DisasContext *s, arg_rrr_esz *a)
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    return gen_gvec_fn_zzz(s, tcg_gen_gvec_mul, a->esz, a->rd, a->rn, a->rm);
+    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_mul, a);
 }
 
 static gen_helper_gvec_3 * const smulh_zzz_fns[4] = {
@@ -6946,7 +6946,7 @@ static bool do_sve2_fn_zzz(DisasContext *s, arg_rrr_esz *a, GVecGen3Fn *fn)
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    return gen_gvec_fn_zzz(s, fn, a->esz, a->rd, a->rn, a->rm);
+    return gen_gvec_fn_arg_zzz(s, fn, a);
 }
 
 static bool trans_SABA(DisasContext *s, arg_rrr_esz *a)

From b262215bf3eb90e06101c94c886691cc4f7776a0 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:46 -0700
Subject: [PATCH 618/935] target/arm: Use TRANS_FEAT for gen_gvec_fn_arg_zzz

Convert SVE translation functions directly using
gen_gvec_fn_arg_zzz to TRANS_FEAT.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-34-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 66 +++++++-------------------------------
 1 file changed, 11 insertions(+), 55 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 2dbf296128..ddb34cad8e 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -376,25 +376,10 @@ const uint64_t pred_esz_masks[4] = {
  *** SVE Logical - Unpredicated Group
  */
 
-static bool trans_AND_zzz(DisasContext *s, arg_rrr_esz *a)
-{
-    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_and, a);
-}
-
-static bool trans_ORR_zzz(DisasContext *s, arg_rrr_esz *a)
-{
-    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_or, a);
-}
-
-static bool trans_EOR_zzz(DisasContext *s, arg_rrr_esz *a)
-{
-    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_xor, a);
-}
-
-static bool trans_BIC_zzz(DisasContext *s, arg_rrr_esz *a)
-{
-    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_andc, a);
-}
+TRANS_FEAT(AND_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_and, a)
+TRANS_FEAT(ORR_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_or, a)
+TRANS_FEAT(EOR_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_xor, a)
+TRANS_FEAT(BIC_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_andc, a)
 
 static void gen_xar8_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, int64_t sh)
 {
@@ -706,35 +691,12 @@ static bool trans_NBSL(DisasContext *s, arg_rrrr_esz *a)
  *** SVE Integer Arithmetic - Unpredicated Group
  */
 
-static bool trans_ADD_zzz(DisasContext *s, arg_rrr_esz *a)
-{
-    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_add, a);
-}
-
-static bool trans_SUB_zzz(DisasContext *s, arg_rrr_esz *a)
-{
-    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_sub, a);
-}
-
-static bool trans_SQADD_zzz(DisasContext *s, arg_rrr_esz *a)
-{
-    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_ssadd, a);
-}
-
-static bool trans_SQSUB_zzz(DisasContext *s, arg_rrr_esz *a)
-{
-    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_sssub, a);
-}
-
-static bool trans_UQADD_zzz(DisasContext *s, arg_rrr_esz *a)
-{
-    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_usadd, a);
-}
-
-static bool trans_UQSUB_zzz(DisasContext *s, arg_rrr_esz *a)
-{
-    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_ussub, a);
-}
+TRANS_FEAT(ADD_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_add, a)
+TRANS_FEAT(SUB_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_sub, a)
+TRANS_FEAT(SQADD_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_ssadd, a)
+TRANS_FEAT(SQSUB_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_sssub, a)
+TRANS_FEAT(UQADD_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_usadd, a)
+TRANS_FEAT(UQSUB_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_ussub, a)
 
 /*
  *** SVE Integer Arithmetic - Binary Predicated Group
@@ -6420,13 +6382,7 @@ static bool trans_MOVPRFX_z(DisasContext *s, arg_rpr_esz *a)
  * SVE2 Integer Multiply - Unpredicated
  */
 
-static bool trans_MUL_zzz(DisasContext *s, arg_rrr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_fn_arg_zzz(s, tcg_gen_gvec_mul, a);
-}
+TRANS_FEAT(MUL_zzz, aa64_sve2, gen_gvec_fn_arg_zzz, tcg_gen_gvec_mul, a)
 
 static gen_helper_gvec_3 * const smulh_zzz_fns[4] = {
     gen_helper_gvec_smulh_b, gen_helper_gvec_smulh_h,

From 79828dcbf5a497e83d350a2f8f8e61429fc60c89 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:47 -0700
Subject: [PATCH 619/935] target/arm: Use TRANS_FEAT for do_sve2_fn_zzz

Convert SVE translation functions using do_sve2_fn_zzz
to use TRANS_FEAT and gen_gvec_fn_arg_zzz.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-35-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 19 ++-----------------
 1 file changed, 2 insertions(+), 17 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index ddb34cad8e..e92fef2304 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -6897,23 +6897,8 @@ static bool trans_SLI(DisasContext *s, arg_rri_esz *a)
     return do_sve2_fn2i(s, a, gen_gvec_sli);
 }
 
-static bool do_sve2_fn_zzz(DisasContext *s, arg_rrr_esz *a, GVecGen3Fn *fn)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_fn_arg_zzz(s, fn, a);
-}
-
-static bool trans_SABA(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_sve2_fn_zzz(s, a, gen_gvec_saba);
-}
-
-static bool trans_UABA(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_sve2_fn_zzz(s, a, gen_gvec_uaba);
-}
+TRANS_FEAT(SABA, aa64_sve2, gen_gvec_fn_arg_zzz, gen_gvec_saba, a)
+TRANS_FEAT(UABA, aa64_sve2, gen_gvec_fn_arg_zzz, gen_gvec_uaba, a)
 
 static bool do_sve2_narrow_extract(DisasContext *s, arg_rri_esz *a,
                                    const GVecGen2 ops[3])

From 2aa469ff5fde3f1c59b14ddf48d58018e2f03982 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:48 -0700
Subject: [PATCH 620/935] target/arm: Use TRANS_FEAT for RAX1

The decode for RAX1 sets esz to MO_8, because that's what
we use by default for "no esz present".  We changed that
to MO_64 during translation because it is more logical for
the operation.  However, the esz argument to gen_gvec_rax1
is unused and forces MO_64 within that function, so there
is no need to do it here as well.

Simplify to use gen_gvec_fn_arg_zzz and TRANS_FEAT.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-36-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index e92fef2304..36d739d7b2 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -7814,13 +7814,7 @@ TRANS_FEAT(SM4E, aa64_sve2_sm4, gen_gvec_ool_arg_zzz,
 TRANS_FEAT(SM4EKEY, aa64_sve2_sm4, gen_gvec_ool_arg_zzz,
            gen_helper_crypto_sm4ekey, a, 0)
 
-static bool trans_RAX1(DisasContext *s, arg_rrr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve2_sha3, s)) {
-        return false;
-    }
-    return gen_gvec_fn_zzz(s, gen_gvec_rax1, MO_64, a->rd, a->rn, a->rm);
-}
+TRANS_FEAT(RAX1, aa64_sve2_sha3, gen_gvec_fn_arg_zzz, gen_gvec_rax1, a)
 
 static bool trans_FCVTNT_sh(DisasContext *s, arg_rpr_esz *a)
 {

From 189876af9cf33fff1de5ad16d0537ed2417ead97 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:49 -0700
Subject: [PATCH 621/935] target/arm: Introduce gen_gvec_fn_arg_zzzz

Merge gen_gvec_fn_zzzz with the sve access check and the
dereference of arg_rrrr_esz.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-37-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 36d739d7b2..e0b083f861 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -281,14 +281,20 @@ static bool gen_gvec_fn_arg_zzz(DisasContext *s, GVecGen3Fn *fn,
 }
 
 /* Invoke a vector expander on four Zregs.  */
-static void gen_gvec_fn_zzzz(DisasContext *s, GVecGen4Fn *gvec_fn,
-                             int esz, int rd, int rn, int rm, int ra)
+static bool gen_gvec_fn_arg_zzzz(DisasContext *s, GVecGen4Fn *gvec_fn,
+                                 arg_rrrr_esz *a)
 {
-    unsigned vsz = vec_full_reg_size(s);
-    gvec_fn(esz, vec_full_reg_offset(s, rd),
-            vec_full_reg_offset(s, rn),
-            vec_full_reg_offset(s, rm),
-            vec_full_reg_offset(s, ra), vsz, vsz);
+    if (gvec_fn == NULL) {
+        return false;
+    }
+    if (sve_access_check(s)) {
+        unsigned vsz = vec_full_reg_size(s);
+        gvec_fn(a->esz, vec_full_reg_offset(s, a->rd),
+                vec_full_reg_offset(s, a->rn),
+                vec_full_reg_offset(s, a->rm),
+                vec_full_reg_offset(s, a->ra), vsz, vsz);
+    }
+    return true;
 }
 
 /* Invoke a vector move on two Zregs.  */
@@ -490,10 +496,7 @@ static bool do_sve2_zzzz_fn(DisasContext *s, arg_rrrr_esz *a, GVecGen4Fn *fn)
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        gen_gvec_fn_zzzz(s, fn, a->esz, a->rd, a->rn, a->rm, a->ra);
-    }
-    return true;
+    return gen_gvec_fn_arg_zzzz(s, fn, a);
 }
 
 static void gen_eor3_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k)

From b773a5c8080f67faeace1169f6a4770eb45646c3 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:50 -0700
Subject: [PATCH 622/935] target/arm: Use TRANS_FEAT for do_sve2_zzzz_fn

Convert SVE translation functions using do_sve2_zzzz_fn
to use TRANS_FEAT and gen_gvec_fn_arg_zzzz.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-38-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 38 ++++++--------------------------------
 1 file changed, 6 insertions(+), 32 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index e0b083f861..f89c78a23e 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -491,14 +491,6 @@ static bool trans_XAR(DisasContext *s, arg_rrri_esz *a)
     return true;
 }
 
-static bool do_sve2_zzzz_fn(DisasContext *s, arg_rrrr_esz *a, GVecGen4Fn *fn)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_fn_arg_zzzz(s, fn, a);
-}
-
 static void gen_eor3_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k)
 {
     tcg_gen_xor_i64(d, n, m);
@@ -525,10 +517,7 @@ static void gen_eor3(unsigned vece, uint32_t d, uint32_t n, uint32_t m,
     tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op);
 }
 
-static bool trans_EOR3(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_sve2_zzzz_fn(s, a, gen_eor3);
-}
+TRANS_FEAT(EOR3, aa64_sve2, gen_gvec_fn_arg_zzzz, gen_eor3, a)
 
 static void gen_bcax_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k)
 {
@@ -556,10 +545,7 @@ static void gen_bcax(unsigned vece, uint32_t d, uint32_t n, uint32_t m,
     tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op);
 }
 
-static bool trans_BCAX(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_sve2_zzzz_fn(s, a, gen_bcax);
-}
+TRANS_FEAT(BCAX, aa64_sve2, gen_gvec_fn_arg_zzzz, gen_bcax, a)
 
 static void gen_bsl(unsigned vece, uint32_t d, uint32_t n, uint32_t m,
                     uint32_t a, uint32_t oprsz, uint32_t maxsz)
@@ -568,10 +554,7 @@ static void gen_bsl(unsigned vece, uint32_t d, uint32_t n, uint32_t m,
     tcg_gen_gvec_bitsel(vece, d, a, n, m, oprsz, maxsz);
 }
 
-static bool trans_BSL(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_sve2_zzzz_fn(s, a, gen_bsl);
-}
+TRANS_FEAT(BSL, aa64_sve2, gen_gvec_fn_arg_zzzz, gen_bsl, a)
 
 static void gen_bsl1n_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k)
 {
@@ -606,10 +589,7 @@ static void gen_bsl1n(unsigned vece, uint32_t d, uint32_t n, uint32_t m,
     tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op);
 }
 
-static bool trans_BSL1N(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_sve2_zzzz_fn(s, a, gen_bsl1n);
-}
+TRANS_FEAT(BSL1N, aa64_sve2, gen_gvec_fn_arg_zzzz, gen_bsl1n, a)
 
 static void gen_bsl2n_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k)
 {
@@ -653,10 +633,7 @@ static void gen_bsl2n(unsigned vece, uint32_t d, uint32_t n, uint32_t m,
     tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op);
 }
 
-static bool trans_BSL2N(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_sve2_zzzz_fn(s, a, gen_bsl2n);
-}
+TRANS_FEAT(BSL2N, aa64_sve2, gen_gvec_fn_arg_zzzz, gen_bsl2n, a)
 
 static void gen_nbsl_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k)
 {
@@ -685,10 +662,7 @@ static void gen_nbsl(unsigned vece, uint32_t d, uint32_t n, uint32_t m,
     tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op);
 }
 
-static bool trans_NBSL(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_sve2_zzzz_fn(s, a, gen_nbsl);
-}
+TRANS_FEAT(NBSL, aa64_sve2, gen_gvec_fn_arg_zzzz, gen_nbsl, a)
 
 /*
  *** SVE Integer Arithmetic - Unpredicated Group

From faf915e2a162bb679b4054d931ce3b464459be79 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:51 -0700
Subject: [PATCH 623/935] target/arm: Introduce gen_gvec_fn_zzi

We have two places that perform this particular operation.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-39-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 30 +++++++++++++++++-------------
 1 file changed, 17 insertions(+), 13 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index f89c78a23e..7938c5393e 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -258,6 +258,21 @@ static bool gen_gvec_ool_arg_zpzz(DisasContext *s, gen_helper_gvec_4 *fn,
     return gen_gvec_ool_zzzp(s, fn, a->rd, a->rn, a->rm, a->pg, data);
 }
 
+/* Invoke a vector expander on two Zregs and an immediate.  */
+static bool gen_gvec_fn_zzi(DisasContext *s, GVecGen2iFn *gvec_fn,
+                            int esz, int rd, int rn, uint64_t imm)
+{
+    if (gvec_fn == NULL) {
+        return false;
+    }
+    if (sve_access_check(s)) {
+        unsigned vsz = vec_full_reg_size(s);
+        gvec_fn(esz, vec_full_reg_offset(s, rd),
+                vec_full_reg_offset(s, rn), imm, vsz, vsz);
+    }
+    return true;
+}
+
 /* Invoke a vector expander on three Zregs.  */
 static bool gen_gvec_fn_zzz(DisasContext *s, GVecGen3Fn *gvec_fn,
                             int esz, int rd, int rn, int rm)
@@ -2028,12 +2043,7 @@ static bool do_zz_dbm(DisasContext *s, arg_rr_dbm *a, GVecGen2iFn *gvec_fn)
                                 extract32(a->dbm, 6, 6))) {
         return false;
     }
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        gvec_fn(MO_64, vec_full_reg_offset(s, a->rd),
-                vec_full_reg_offset(s, a->rn), imm, vsz, vsz);
-    }
-    return true;
+    return gen_gvec_fn_zzi(s, gvec_fn, MO_64, a->rd, a->rn, imm);
 }
 
 static bool trans_AND_zzi(DisasContext *s, arg_rr_dbm *a)
@@ -6835,13 +6845,7 @@ static bool do_sve2_fn2i(DisasContext *s, arg_rri_esz *a, GVecGen2iFn *fn)
     if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        unsigned rd_ofs = vec_full_reg_offset(s, a->rd);
-        unsigned rn_ofs = vec_full_reg_offset(s, a->rn);
-        fn(a->esz, rd_ofs, rn_ofs, a->imm, vsz, vsz);
-    }
-    return true;
+    return gen_gvec_fn_zzi(s, fn, a->esz, a->rd, a->rn, a->imm);
 }
 
 static bool trans_SSRA(DisasContext *s, arg_rri_esz *a)

From 15a314dad5898eef560456d084c245881290e8e2 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:52 -0700
Subject: [PATCH 624/935] target/arm: Use TRANS_FEAT for do_zz_dbm

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-40-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 17 +++--------------
 1 file changed, 3 insertions(+), 14 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 7938c5393e..6fa721eca6 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2046,20 +2046,9 @@ static bool do_zz_dbm(DisasContext *s, arg_rr_dbm *a, GVecGen2iFn *gvec_fn)
     return gen_gvec_fn_zzi(s, gvec_fn, MO_64, a->rd, a->rn, imm);
 }
 
-static bool trans_AND_zzi(DisasContext *s, arg_rr_dbm *a)
-{
-    return do_zz_dbm(s, a, tcg_gen_gvec_andi);
-}
-
-static bool trans_ORR_zzi(DisasContext *s, arg_rr_dbm *a)
-{
-    return do_zz_dbm(s, a, tcg_gen_gvec_ori);
-}
-
-static bool trans_EOR_zzi(DisasContext *s, arg_rr_dbm *a)
-{
-    return do_zz_dbm(s, a, tcg_gen_gvec_xori);
-}
+TRANS_FEAT(AND_zzi, aa64_sve, do_zz_dbm, a, tcg_gen_gvec_andi)
+TRANS_FEAT(ORR_zzi, aa64_sve, do_zz_dbm, a, tcg_gen_gvec_ori)
+TRANS_FEAT(EOR_zzi, aa64_sve, do_zz_dbm, a, tcg_gen_gvec_xori)
 
 static bool trans_DUPM(DisasContext *s, arg_DUPM *a)
 {

From 68cc4ee3f98bbc13224582e13f87db04e4be016a Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:53 -0700
Subject: [PATCH 625/935] target/arm: Hoist sve access check through do_sel_z

The check is already done in gen_gvec_ool_zzzp,
which is called by do_sel_z; remove from callers.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-41-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 14 ++++----------
 1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 6fa721eca6..62bfc6fe7c 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -697,13 +697,13 @@ TRANS_FEAT(UQSUB_zzz, aa64_sve, gen_gvec_fn_arg_zzz, tcg_gen_gvec_ussub, a)
 /* Select active elememnts from Zn and inactive elements from Zm,
  * storing the result in Zd.
  */
-static void do_sel_z(DisasContext *s, int rd, int rn, int rm, int pg, int esz)
+static bool do_sel_z(DisasContext *s, int rd, int rn, int rm, int pg, int esz)
 {
     static gen_helper_gvec_4 * const fns[4] = {
         gen_helper_sve_sel_zpzz_b, gen_helper_sve_sel_zpzz_h,
         gen_helper_sve_sel_zpzz_s, gen_helper_sve_sel_zpzz_d
     };
-    gen_gvec_ool_zzzp(s, fns[esz], rd, rn, rm, pg, 0);
+    return gen_gvec_ool_zzzp(s, fns[esz], rd, rn, rm, pg, 0);
 }
 
 #define DO_ZPZZ(NAME, FEAT, name) \
@@ -749,10 +749,7 @@ TRANS_FEAT(UDIV_zpzz, aa64_sve, gen_gvec_ool_arg_zpzz, udiv_fns[a->esz], a, 0)
 
 static bool trans_SEL_zpzz(DisasContext *s, arg_rprr_esz *a)
 {
-    if (sve_access_check(s)) {
-        do_sel_z(s, a->rd, a->rn, a->rm, a->pg, a->esz);
-    }
-    return true;
+    return do_sel_z(s, a->rd, a->rn, a->rm, a->pg, a->esz);
 }
 
 /*
@@ -6343,10 +6340,7 @@ static bool trans_MOVPRFX(DisasContext *s, arg_MOVPRFX *a)
 
 static bool trans_MOVPRFX_m(DisasContext *s, arg_rpr_esz *a)
 {
-    if (sve_access_check(s)) {
-        do_sel_z(s, a->rd, a->rn, a->rd, a->pg, a->esz);
-    }
-    return true;
+    return do_sel_z(s, a->rd, a->rn, a->rd, a->pg, a->esz);
 }
 
 static bool trans_MOVPRFX_z(DisasContext *s, arg_rpr_esz *a)

From ada378f042edbad9430f0e7c0cfb0f4946ecb370 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:54 -0700
Subject: [PATCH 626/935] target/arm: Introduce gen_gvec_fn_arg_zzi

We have two places that perform this particular operation.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-42-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 62bfc6fe7c..7a3b5f137a 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -273,6 +273,16 @@ static bool gen_gvec_fn_zzi(DisasContext *s, GVecGen2iFn *gvec_fn,
     return true;
 }
 
+static bool gen_gvec_fn_arg_zzi(DisasContext *s, GVecGen2iFn *gvec_fn,
+                                arg_rri_esz *a)
+{
+    if (a->esz < 0) {
+        /* Invalid tsz encoding -- see tszimm_esz. */
+        return false;
+    }
+    return gen_gvec_fn_zzi(s, gvec_fn, a->esz, a->rd, a->rn, a->imm);
+}
+
 /* Invoke a vector expander on three Zregs.  */
 static bool gen_gvec_fn_zzz(DisasContext *s, GVecGen3Fn *gvec_fn,
                             int esz, int rd, int rn, int rm)
@@ -3503,12 +3513,7 @@ static bool trans_ADD_zzi(DisasContext *s, arg_rri_esz *a)
     if (a->esz == 0 && extract32(s->insn, 13, 1)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        tcg_gen_gvec_addi(a->esz, vec_full_reg_offset(s, a->rd),
-                          vec_full_reg_offset(s, a->rn), a->imm, vsz, vsz);
-    }
-    return true;
+    return gen_gvec_fn_arg_zzi(s, tcg_gen_gvec_addi, a);
 }
 
 static bool trans_SUB_zzi(DisasContext *s, arg_rri_esz *a)
@@ -6825,10 +6830,10 @@ TRANS_FEAT(ADCLT, aa64_sve2, do_adcl, a, true)
 
 static bool do_sve2_fn2i(DisasContext *s, arg_rri_esz *a, GVecGen2iFn *fn)
 {
-    if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) {
+    if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    return gen_gvec_fn_zzi(s, fn, a->esz, a->rd, a->rn, a->imm);
+    return gen_gvec_fn_arg_zzi(s, fn, a);
 }
 
 static bool trans_SSRA(DisasContext *s, arg_rri_esz *a)

From f2be26a5fe7b419ac7bdc38e82a1362000ad84c7 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:55 -0700
Subject: [PATCH 627/935] target/arm: Use TRANS_FEAT for do_sve2_fn2i

Convert SVE translation functions using do_sve2_fn2i
to use TRANS_FEAT and gen_gvec_fn_arg_zzi.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-43-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 43 ++++++--------------------------------
 1 file changed, 6 insertions(+), 37 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 7a3b5f137a..911d2e28bf 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -6828,43 +6828,12 @@ static bool do_adcl(DisasContext *s, arg_rrrr_esz *a, bool sel)
 TRANS_FEAT(ADCLB, aa64_sve2, do_adcl, a, false)
 TRANS_FEAT(ADCLT, aa64_sve2, do_adcl, a, true)
 
-static bool do_sve2_fn2i(DisasContext *s, arg_rri_esz *a, GVecGen2iFn *fn)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_fn_arg_zzi(s, fn, a);
-}
-
-static bool trans_SSRA(DisasContext *s, arg_rri_esz *a)
-{
-    return do_sve2_fn2i(s, a, gen_gvec_ssra);
-}
-
-static bool trans_USRA(DisasContext *s, arg_rri_esz *a)
-{
-    return do_sve2_fn2i(s, a, gen_gvec_usra);
-}
-
-static bool trans_SRSRA(DisasContext *s, arg_rri_esz *a)
-{
-    return do_sve2_fn2i(s, a, gen_gvec_srsra);
-}
-
-static bool trans_URSRA(DisasContext *s, arg_rri_esz *a)
-{
-    return do_sve2_fn2i(s, a, gen_gvec_ursra);
-}
-
-static bool trans_SRI(DisasContext *s, arg_rri_esz *a)
-{
-    return do_sve2_fn2i(s, a, gen_gvec_sri);
-}
-
-static bool trans_SLI(DisasContext *s, arg_rri_esz *a)
-{
-    return do_sve2_fn2i(s, a, gen_gvec_sli);
-}
+TRANS_FEAT(SSRA, aa64_sve2, gen_gvec_fn_arg_zzi, gen_gvec_ssra, a)
+TRANS_FEAT(USRA, aa64_sve2, gen_gvec_fn_arg_zzi, gen_gvec_usra, a)
+TRANS_FEAT(SRSRA, aa64_sve2, gen_gvec_fn_arg_zzi, gen_gvec_srsra, a)
+TRANS_FEAT(URSRA, aa64_sve2, gen_gvec_fn_arg_zzi, gen_gvec_ursra, a)
+TRANS_FEAT(SRI, aa64_sve2, gen_gvec_fn_arg_zzi, gen_gvec_sri, a)
+TRANS_FEAT(SLI, aa64_sve2, gen_gvec_fn_arg_zzi, gen_gvec_sli, a)
 
 TRANS_FEAT(SABA, aa64_sve2, gen_gvec_fn_arg_zzz, gen_gvec_saba, a)
 TRANS_FEAT(UABA, aa64_sve2, gen_gvec_fn_arg_zzz, gen_gvec_uaba, a)

From 9ac24f1f350eebb7f3b58bf3365fc9f39c802f43 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:56 -0700
Subject: [PATCH 628/935] target/arm: Use TRANS_FEAT for do_vpz_ool

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-44-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 20 +++++++-------------
 1 file changed, 7 insertions(+), 13 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 911d2e28bf..6103bd7f1d 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -858,14 +858,11 @@ static bool do_vpz_ool(DisasContext *s, arg_rpr_esz *a,
 }
 
 #define DO_VPZ(NAME, name) \
-static bool trans_##NAME(DisasContext *s, arg_rpr_esz *a)                \
-{                                                                        \
-    static gen_helper_gvec_reduc * const fns[4] = {                      \
+    static gen_helper_gvec_reduc * const name##_fns[4] = {               \
         gen_helper_sve_##name##_b, gen_helper_sve_##name##_h,            \
         gen_helper_sve_##name##_s, gen_helper_sve_##name##_d,            \
     };                                                                   \
-    return do_vpz_ool(s, a, fns[a->esz]);                                \
-}
+    TRANS_FEAT(NAME, aa64_sve, do_vpz_ool, a, name##_fns[a->esz])
 
 DO_VPZ(ORV, orv)
 DO_VPZ(ANDV, andv)
@@ -877,14 +874,11 @@ DO_VPZ(UMAXV, umaxv)
 DO_VPZ(SMINV, sminv)
 DO_VPZ(UMINV, uminv)
 
-static bool trans_SADDV(DisasContext *s, arg_rpr_esz *a)
-{
-    static gen_helper_gvec_reduc * const fns[4] = {
-        gen_helper_sve_saddv_b, gen_helper_sve_saddv_h,
-        gen_helper_sve_saddv_s, NULL
-    };
-    return do_vpz_ool(s, a, fns[a->esz]);
-}
+static gen_helper_gvec_reduc * const saddv_fns[4] = {
+    gen_helper_sve_saddv_b, gen_helper_sve_saddv_h,
+    gen_helper_sve_saddv_s, NULL
+};
+TRANS_FEAT(SADDV, aa64_sve, do_vpz_ool, a, saddv_fns[a->esz])
 
 #undef DO_VPZ
 

From 5e612f807303d216e51c9a84b3764f72965f546e Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:57 -0700
Subject: [PATCH 629/935] target/arm: Use TRANS_FEAT for do_shift_imm

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-45-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 17 +++--------------
 1 file changed, 3 insertions(+), 14 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 6103bd7f1d..f15e9a30b3 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -1054,20 +1054,9 @@ static bool do_shift_imm(DisasContext *s, arg_rri_esz *a, bool asr,
     return true;
 }
 
-static bool trans_ASR_zzi(DisasContext *s, arg_rri_esz *a)
-{
-    return do_shift_imm(s, a, true, tcg_gen_gvec_sari);
-}
-
-static bool trans_LSR_zzi(DisasContext *s, arg_rri_esz *a)
-{
-    return do_shift_imm(s, a, false, tcg_gen_gvec_shri);
-}
-
-static bool trans_LSL_zzi(DisasContext *s, arg_rri_esz *a)
-{
-    return do_shift_imm(s, a, false, tcg_gen_gvec_shli);
-}
+TRANS_FEAT(ASR_zzi, aa64_sve, do_shift_imm, a, true, tcg_gen_gvec_sari)
+TRANS_FEAT(LSR_zzi, aa64_sve, do_shift_imm, a, false, tcg_gen_gvec_shri)
+TRANS_FEAT(LSL_zzi, aa64_sve, do_shift_imm, a, false, tcg_gen_gvec_shli)
 
 #define DO_ZZW(NAME, name) \
     static gen_helper_gvec_3 * const name##_zzw_fns[4] = {                \

From 73c558a85d57782aedf5a184702742861fcd53c7 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:58 -0700
Subject: [PATCH 630/935] target/arm: Introduce do_shift_zpzi

Share code between the various shifts using arg_rpri_esz.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-46-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 68 +++++++++++++++++---------------------
 1 file changed, 30 insertions(+), 38 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index f15e9a30b3..c7c16863c0 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -900,20 +900,39 @@ static bool do_movz_zpz(DisasContext *s, int rd, int rn, int pg,
     return gen_gvec_ool_zzp(s, fns[esz], rd, rn, pg, invert);
 }
 
+static bool do_shift_zpzi(DisasContext *s, arg_rpri_esz *a, bool asr,
+                          gen_helper_gvec_3 * const fns[4])
+{
+    int max;
+
+    if (a->esz < 0) {
+        /* Invalid tsz encoding -- see tszimm_esz. */
+        return false;
+    }
+
+    /*
+     * Shift by element size is architecturally valid.
+     * For arithmetic right-shift, it's the same as by one less.
+     * For logical shifts and ASRD, it is a zeroing operation.
+     */
+    max = 8 << a->esz;
+    if (a->imm >= max) {
+        if (asr) {
+            a->imm = max - 1;
+        } else {
+            return do_movz_zpz(s, a->rd, a->rd, a->pg, a->esz, true);
+        }
+    }
+    return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
+}
+
 static bool trans_ASR_zpzi(DisasContext *s, arg_rpri_esz *a)
 {
     static gen_helper_gvec_3 * const fns[4] = {
         gen_helper_sve_asr_zpzi_b, gen_helper_sve_asr_zpzi_h,
         gen_helper_sve_asr_zpzi_s, gen_helper_sve_asr_zpzi_d,
     };
-    if (a->esz < 0) {
-        /* Invalid tsz encoding -- see tszimm_esz. */
-        return false;
-    }
-    /* Shift by element size is architecturally valid.  For
-       arithmetic right-shift, it's the same as by one less. */
-    a->imm = MIN(a->imm, (8 << a->esz) - 1);
-    return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
+    return do_shift_zpzi(s, a, true, fns);
 }
 
 static bool trans_LSR_zpzi(DisasContext *s, arg_rpri_esz *a)
@@ -922,16 +941,7 @@ static bool trans_LSR_zpzi(DisasContext *s, arg_rpri_esz *a)
         gen_helper_sve_lsr_zpzi_b, gen_helper_sve_lsr_zpzi_h,
         gen_helper_sve_lsr_zpzi_s, gen_helper_sve_lsr_zpzi_d,
     };
-    if (a->esz < 0) {
-        return false;
-    }
-    /* Shift by element size is architecturally valid.
-       For logical shifts, it is a zeroing operation.  */
-    if (a->imm >= (8 << a->esz)) {
-        return do_movz_zpz(s, a->rd, a->rd, a->pg, a->esz, true);
-    } else {
-        return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
-    }
+    return do_shift_zpzi(s, a, false, fns);
 }
 
 static bool trans_LSL_zpzi(DisasContext *s, arg_rpri_esz *a)
@@ -940,16 +950,7 @@ static bool trans_LSL_zpzi(DisasContext *s, arg_rpri_esz *a)
         gen_helper_sve_lsl_zpzi_b, gen_helper_sve_lsl_zpzi_h,
         gen_helper_sve_lsl_zpzi_s, gen_helper_sve_lsl_zpzi_d,
     };
-    if (a->esz < 0) {
-        return false;
-    }
-    /* Shift by element size is architecturally valid.
-       For logical shifts, it is a zeroing operation.  */
-    if (a->imm >= (8 << a->esz)) {
-        return do_movz_zpz(s, a->rd, a->rd, a->pg, a->esz, true);
-    } else {
-        return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
-    }
+    return do_shift_zpzi(s, a, false, fns);
 }
 
 static bool trans_ASRD(DisasContext *s, arg_rpri_esz *a)
@@ -958,16 +959,7 @@ static bool trans_ASRD(DisasContext *s, arg_rpri_esz *a)
         gen_helper_sve_asrd_b, gen_helper_sve_asrd_h,
         gen_helper_sve_asrd_s, gen_helper_sve_asrd_d,
     };
-    if (a->esz < 0) {
-        return false;
-    }
-    /* Shift by element size is architecturally valid.  For arithmetic
-       right shift for division, it is a zeroing operation.  */
-    if (a->imm >= (8 << a->esz)) {
-        return do_movz_zpz(s, a->rd, a->rd, a->pg, a->esz, true);
-    } else {
-        return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
-    }
+    return do_shift_zpzi(s, a, false, fns);
 }
 
 static gen_helper_gvec_3 * const sqshl_zpzi_fns[4] = {

From 5cccd1f182bc7e066908bc4f5a9a992360f11248 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:17:59 -0700
Subject: [PATCH 631/935] target/arm: Use TRANS_FEAT for do_shift_zpzi

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-47-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 52 +++++++++++++++-----------------------
 1 file changed, 20 insertions(+), 32 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index c7c16863c0..98f9cfa86c 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -926,41 +926,29 @@ static bool do_shift_zpzi(DisasContext *s, arg_rpri_esz *a, bool asr,
     return gen_gvec_ool_arg_zpzi(s, fns[a->esz], a);
 }
 
-static bool trans_ASR_zpzi(DisasContext *s, arg_rpri_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve_asr_zpzi_b, gen_helper_sve_asr_zpzi_h,
-        gen_helper_sve_asr_zpzi_s, gen_helper_sve_asr_zpzi_d,
-    };
-    return do_shift_zpzi(s, a, true, fns);
-}
+static gen_helper_gvec_3 * const asr_zpzi_fns[4] = {
+    gen_helper_sve_asr_zpzi_b, gen_helper_sve_asr_zpzi_h,
+    gen_helper_sve_asr_zpzi_s, gen_helper_sve_asr_zpzi_d,
+};
+TRANS_FEAT(ASR_zpzi, aa64_sve, do_shift_zpzi, a, true, asr_zpzi_fns)
 
-static bool trans_LSR_zpzi(DisasContext *s, arg_rpri_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve_lsr_zpzi_b, gen_helper_sve_lsr_zpzi_h,
-        gen_helper_sve_lsr_zpzi_s, gen_helper_sve_lsr_zpzi_d,
-    };
-    return do_shift_zpzi(s, a, false, fns);
-}
+static gen_helper_gvec_3 * const lsr_zpzi_fns[4] = {
+    gen_helper_sve_lsr_zpzi_b, gen_helper_sve_lsr_zpzi_h,
+    gen_helper_sve_lsr_zpzi_s, gen_helper_sve_lsr_zpzi_d,
+};
+TRANS_FEAT(LSR_zpzi, aa64_sve, do_shift_zpzi, a, false, lsr_zpzi_fns)
 
-static bool trans_LSL_zpzi(DisasContext *s, arg_rpri_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve_lsl_zpzi_b, gen_helper_sve_lsl_zpzi_h,
-        gen_helper_sve_lsl_zpzi_s, gen_helper_sve_lsl_zpzi_d,
-    };
-    return do_shift_zpzi(s, a, false, fns);
-}
+static gen_helper_gvec_3 * const lsl_zpzi_fns[4] = {
+    gen_helper_sve_lsl_zpzi_b, gen_helper_sve_lsl_zpzi_h,
+    gen_helper_sve_lsl_zpzi_s, gen_helper_sve_lsl_zpzi_d,
+};
+TRANS_FEAT(LSL_zpzi, aa64_sve, do_shift_zpzi, a, false, lsl_zpzi_fns)
 
-static bool trans_ASRD(DisasContext *s, arg_rpri_esz *a)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve_asrd_b, gen_helper_sve_asrd_h,
-        gen_helper_sve_asrd_s, gen_helper_sve_asrd_d,
-    };
-    return do_shift_zpzi(s, a, false, fns);
-}
+static gen_helper_gvec_3 * const asrd_fns[4] = {
+    gen_helper_sve_asrd_b, gen_helper_sve_asrd_h,
+    gen_helper_sve_asrd_s, gen_helper_sve_asrd_d,
+};
+TRANS_FEAT(ASRD, aa64_sve, do_shift_zpzi, a, false, asrd_fns)
 
 static gen_helper_gvec_3 * const sqshl_zpzi_fns[4] = {
     gen_helper_sve2_sqshl_zpzi_b, gen_helper_sve2_sqshl_zpzi_h,

From dc67e645fb27ef86b48a29de972628edaffe9946 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:00 -0700
Subject: [PATCH 632/935] target/arm: Use TRANS_FEAT for do_zpzzz_ool

Remove the DO_ZPZZZ macro, as it had just the two uses.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-48-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 23 ++++++++++-------------
 1 file changed, 10 insertions(+), 13 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 98f9cfa86c..52bbd1a4fa 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -1071,20 +1071,17 @@ static bool do_zpzzz_ool(DisasContext *s, arg_rprrr_esz *a,
     return true;
 }
 
-#define DO_ZPZZZ(NAME, name) \
-static bool trans_##NAME(DisasContext *s, arg_rprrr_esz *a)          \
-{                                                                    \
-    static gen_helper_gvec_5 * const fns[4] = {                      \
-        gen_helper_sve_##name##_b, gen_helper_sve_##name##_h,        \
-        gen_helper_sve_##name##_s, gen_helper_sve_##name##_d,        \
-    };                                                               \
-    return do_zpzzz_ool(s, a, fns[a->esz]);                          \
-}
+static gen_helper_gvec_5 * const mla_fns[4] = {
+    gen_helper_sve_mla_b, gen_helper_sve_mla_h,
+    gen_helper_sve_mla_s, gen_helper_sve_mla_d,
+};
+TRANS_FEAT(MLA, aa64_sve, do_zpzzz_ool, a, mla_fns[a->esz])
 
-DO_ZPZZZ(MLA, mla)
-DO_ZPZZZ(MLS, mls)
-
-#undef DO_ZPZZZ
+static gen_helper_gvec_5 * const mls_fns[4] = {
+    gen_helper_sve_mls_b, gen_helper_sve_mls_h,
+    gen_helper_sve_mls_s, gen_helper_sve_mls_d,
+};
+TRANS_FEAT(MLS, aa64_sve, do_zpzzz_ool, a, mls_fns[a->esz])
 
 /*
  *** SVE Index Generation Group

From 6687d05dc318aed9dbd1fc724bcfcbea1a48809a Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:01 -0700
Subject: [PATCH 633/935] target/arm: Move sve check into do_index

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-49-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 53 ++++++++++++++++++--------------------
 1 file changed, 25 insertions(+), 28 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 52bbd1a4fa..44c2342923 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -1087,12 +1087,20 @@ TRANS_FEAT(MLS, aa64_sve, do_zpzzz_ool, a, mls_fns[a->esz])
  *** SVE Index Generation Group
  */
 
-static void do_index(DisasContext *s, int esz, int rd,
+static bool do_index(DisasContext *s, int esz, int rd,
                      TCGv_i64 start, TCGv_i64 incr)
 {
-    unsigned vsz = vec_full_reg_size(s);
-    TCGv_i32 desc = tcg_constant_i32(simd_desc(vsz, vsz, 0));
-    TCGv_ptr t_zd = tcg_temp_new_ptr();
+    unsigned vsz;
+    TCGv_i32 desc;
+    TCGv_ptr t_zd;
+
+    if (!sve_access_check(s)) {
+        return true;
+    }
+
+    vsz = vec_full_reg_size(s);
+    desc = tcg_constant_i32(simd_desc(vsz, vsz, 0));
+    t_zd = tcg_temp_new_ptr();
 
     tcg_gen_addi_ptr(t_zd, cpu_env, vec_full_reg_offset(s, rd));
     if (esz == 3) {
@@ -1115,46 +1123,35 @@ static void do_index(DisasContext *s, int esz, int rd,
         tcg_temp_free_i32(i32);
     }
     tcg_temp_free_ptr(t_zd);
+    return true;
 }
 
 static bool trans_INDEX_ii(DisasContext *s, arg_INDEX_ii *a)
 {
-    if (sve_access_check(s)) {
-        TCGv_i64 start = tcg_constant_i64(a->imm1);
-        TCGv_i64 incr = tcg_constant_i64(a->imm2);
-        do_index(s, a->esz, a->rd, start, incr);
-    }
-    return true;
+    TCGv_i64 start = tcg_constant_i64(a->imm1);
+    TCGv_i64 incr = tcg_constant_i64(a->imm2);
+    return do_index(s, a->esz, a->rd, start, incr);
 }
 
 static bool trans_INDEX_ir(DisasContext *s, arg_INDEX_ir *a)
 {
-    if (sve_access_check(s)) {
-        TCGv_i64 start = tcg_constant_i64(a->imm);
-        TCGv_i64 incr = cpu_reg(s, a->rm);
-        do_index(s, a->esz, a->rd, start, incr);
-    }
-    return true;
+    TCGv_i64 start = tcg_constant_i64(a->imm);
+    TCGv_i64 incr = cpu_reg(s, a->rm);
+    return do_index(s, a->esz, a->rd, start, incr);
 }
 
 static bool trans_INDEX_ri(DisasContext *s, arg_INDEX_ri *a)
 {
-    if (sve_access_check(s)) {
-        TCGv_i64 start = cpu_reg(s, a->rn);
-        TCGv_i64 incr = tcg_constant_i64(a->imm);
-        do_index(s, a->esz, a->rd, start, incr);
-    }
-    return true;
+    TCGv_i64 start = cpu_reg(s, a->rn);
+    TCGv_i64 incr = tcg_constant_i64(a->imm);
+    return do_index(s, a->esz, a->rd, start, incr);
 }
 
 static bool trans_INDEX_rr(DisasContext *s, arg_INDEX_rr *a)
 {
-    if (sve_access_check(s)) {
-        TCGv_i64 start = cpu_reg(s, a->rn);
-        TCGv_i64 incr = cpu_reg(s, a->rm);
-        do_index(s, a->esz, a->rd, start, incr);
-    }
-    return true;
+    TCGv_i64 start = cpu_reg(s, a->rn);
+    TCGv_i64 incr = cpu_reg(s, a->rm);
+    return do_index(s, a->esz, a->rd, start, incr);
 }
 
 /*

From 9aa60c83ea424036469076cfb2989fe85969673e Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:02 -0700
Subject: [PATCH 634/935] target/arm: Use TRANS_FEAT for do_index

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-50-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 35 ++++++++---------------------------
 1 file changed, 8 insertions(+), 27 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 44c2342923..dac29749ce 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -1126,33 +1126,14 @@ static bool do_index(DisasContext *s, int esz, int rd,
     return true;
 }
 
-static bool trans_INDEX_ii(DisasContext *s, arg_INDEX_ii *a)
-{
-    TCGv_i64 start = tcg_constant_i64(a->imm1);
-    TCGv_i64 incr = tcg_constant_i64(a->imm2);
-    return do_index(s, a->esz, a->rd, start, incr);
-}
-
-static bool trans_INDEX_ir(DisasContext *s, arg_INDEX_ir *a)
-{
-    TCGv_i64 start = tcg_constant_i64(a->imm);
-    TCGv_i64 incr = cpu_reg(s, a->rm);
-    return do_index(s, a->esz, a->rd, start, incr);
-}
-
-static bool trans_INDEX_ri(DisasContext *s, arg_INDEX_ri *a)
-{
-    TCGv_i64 start = cpu_reg(s, a->rn);
-    TCGv_i64 incr = tcg_constant_i64(a->imm);
-    return do_index(s, a->esz, a->rd, start, incr);
-}
-
-static bool trans_INDEX_rr(DisasContext *s, arg_INDEX_rr *a)
-{
-    TCGv_i64 start = cpu_reg(s, a->rn);
-    TCGv_i64 incr = cpu_reg(s, a->rm);
-    return do_index(s, a->esz, a->rd, start, incr);
-}
+TRANS_FEAT(INDEX_ii, aa64_sve, do_index, a->esz, a->rd,
+           tcg_constant_i64(a->imm1), tcg_constant_i64(a->imm2))
+TRANS_FEAT(INDEX_ir, aa64_sve, do_index, a->esz, a->rd,
+           tcg_constant_i64(a->imm), cpu_reg(s, a->rm))
+TRANS_FEAT(INDEX_ri, aa64_sve, do_index, a->esz, a->rd,
+           cpu_reg(s, a->rn), tcg_constant_i64(a->imm))
+TRANS_FEAT(INDEX_rr, aa64_sve, do_index, a->esz, a->rd,
+           cpu_reg(s, a->rn), cpu_reg(s, a->rm))
 
 /*
  *** SVE Stack Allocation Group

From dcba3d6741f721c5e3b66aac641ebcbc4e104839 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:03 -0700
Subject: [PATCH 635/935] target/arm: Use TRANS_FEAT for do_adr

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-51-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 23 ++++-------------------
 1 file changed, 4 insertions(+), 19 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index dac29749ce..c8eb2c684b 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -1177,25 +1177,10 @@ static bool do_adr(DisasContext *s, arg_rrri *a, gen_helper_gvec_3 *fn)
     return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, a->imm);
 }
 
-static bool trans_ADR_p32(DisasContext *s, arg_rrri *a)
-{
-    return do_adr(s, a, gen_helper_sve_adr_p32);
-}
-
-static bool trans_ADR_p64(DisasContext *s, arg_rrri *a)
-{
-    return do_adr(s, a, gen_helper_sve_adr_p64);
-}
-
-static bool trans_ADR_s32(DisasContext *s, arg_rrri *a)
-{
-    return do_adr(s, a, gen_helper_sve_adr_s32);
-}
-
-static bool trans_ADR_u32(DisasContext *s, arg_rrri *a)
-{
-    return do_adr(s, a, gen_helper_sve_adr_u32);
-}
+TRANS_FEAT(ADR_p32, aa64_sve, do_adr, a, gen_helper_sve_adr_p32)
+TRANS_FEAT(ADR_p64, aa64_sve, do_adr, a, gen_helper_sve_adr_p64)
+TRANS_FEAT(ADR_s32, aa64_sve, do_adr, a, gen_helper_sve_adr_s32)
+TRANS_FEAT(ADR_u32, aa64_sve, do_adr, a, gen_helper_sve_adr_u32)
 
 /*
  *** SVE Integer Misc - Unpredicated Group

From b03a85010f7b3ba828dbb49c0ce39b18156d16f1 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:04 -0700
Subject: [PATCH 636/935] target/arm: Use TRANS_FEAT for do_predset

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-52-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 19 +++++--------------
 1 file changed, 5 insertions(+), 14 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index c8eb2c684b..ce6e000f6f 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -1616,22 +1616,13 @@ static bool do_predset(DisasContext *s, int esz, int rd, int pat, bool setflag)
     return true;
 }
 
-static bool trans_PTRUE(DisasContext *s, arg_PTRUE *a)
-{
-    return do_predset(s, a->esz, a->rd, a->pat, a->s);
-}
+TRANS_FEAT(PTRUE, aa64_sve, do_predset, a->esz, a->rd, a->pat, a->s)
 
-static bool trans_SETFFR(DisasContext *s, arg_SETFFR *a)
-{
-    /* Note pat == 31 is #all, to set all elements.  */
-    return do_predset(s, 0, FFR_PRED_NUM, 31, false);
-}
+/* Note pat == 31 is #all, to set all elements.  */
+TRANS_FEAT(SETFFR, aa64_sve, do_predset, 0, FFR_PRED_NUM, 31, false)
 
-static bool trans_PFALSE(DisasContext *s, arg_PFALSE *a)
-{
-    /* Note pat == 32 is #unimp, to set no elements.  */
-    return do_predset(s, 0, a->rd, 32, false);
-}
+/* Note pat == 32 is #unimp, to set no elements.  */
+TRANS_FEAT(PFALSE, aa64_sve, do_predset, 0, a->rd, 32, false)
 
 static bool trans_RDFFR_p(DisasContext *s, arg_RDFFR_p *a)
 {

From ff502658566ecc58a768a68fefc062366e74a65b Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:05 -0700
Subject: [PATCH 637/935] target/arm: Use TRANS_FEAT for RDFFR, WRFFR

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-53-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index ce6e000f6f..6fd9a42ef9 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -1636,15 +1636,8 @@ static bool trans_RDFFR_p(DisasContext *s, arg_RDFFR_p *a)
     return trans_AND_pppp(s, &alt_a);
 }
 
-static bool trans_RDFFR(DisasContext *s, arg_RDFFR *a)
-{
-    return do_mov_p(s, a->rd, FFR_PRED_NUM);
-}
-
-static bool trans_WRFFR(DisasContext *s, arg_WRFFR *a)
-{
-    return do_mov_p(s, FFR_PRED_NUM, a->rn);
-}
+TRANS_FEAT(RDFFR, aa64_sve, do_mov_p, a->rd, FFR_PRED_NUM)
+TRANS_FEAT(WRFFR, aa64_sve, do_mov_p, FFR_PRED_NUM, a->rn)
 
 static bool do_pfirst_pnext(DisasContext *s, arg_rr_esz *a,
                             void (*gen_fn)(TCGv_i32, TCGv_ptr,

From d95040e3df45537fc7ed76b1d9928fa18f1829ad Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:06 -0700
Subject: [PATCH 638/935] target/arm: Use TRANS_FEAT for do_pfirst_pnext

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-54-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 6fd9a42ef9..abb5433ee5 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -1668,15 +1668,8 @@ static bool do_pfirst_pnext(DisasContext *s, arg_rr_esz *a,
     return true;
 }
 
-static bool trans_PFIRST(DisasContext *s, arg_rr_esz *a)
-{
-    return do_pfirst_pnext(s, a, gen_helper_sve_pfirst);
-}
-
-static bool trans_PNEXT(DisasContext *s, arg_rr_esz *a)
-{
-    return do_pfirst_pnext(s, a, gen_helper_sve_pnext);
-}
+TRANS_FEAT(PFIRST, aa64_sve, do_pfirst_pnext, a, gen_helper_sve_pfirst)
+TRANS_FEAT(PNEXT, aa64_sve, do_pfirst_pnext, a, gen_helper_sve_pnext)
 
 /*
  *** SVE Element Count Group

From c799c115dbf55dad6ee909fd94b362ec760ffa11 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:07 -0700
Subject: [PATCH 639/935] target/arm: Use TRANS_FEAT for do_EXT

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-55-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 14 ++------------
 1 file changed, 2 insertions(+), 12 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index abb5433ee5..7139e6c0b0 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2081,18 +2081,8 @@ static bool do_EXT(DisasContext *s, int rd, int rn, int rm, int imm)
     return true;
 }
 
-static bool trans_EXT(DisasContext *s, arg_EXT *a)
-{
-    return do_EXT(s, a->rd, a->rn, a->rm, a->imm);
-}
-
-static bool trans_EXT_sve2(DisasContext *s, arg_rri *a)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return do_EXT(s, a->rd, a->rn, (a->rn + 1) % 32, a->imm);
-}
+TRANS_FEAT(EXT, aa64_sve, do_EXT, a->rd, a->rn, a->rm, a->imm)
+TRANS_FEAT(EXT_sve2, aa64_sve2, do_EXT, a->rd, a->rn, (a->rn + 1) % 32, a->imm)
 
 /*
  *** SVE Permute - Unpredicated Group

From bdb349f5dcea97512e183f335ff037270170e794 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:08 -0700
Subject: [PATCH 640/935] target/arm: Use TRANS_FEAT for do_perm_pred3

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-56-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 35 ++++++-----------------------------
 1 file changed, 6 insertions(+), 29 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 7139e6c0b0..413e89b19c 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2272,35 +2272,12 @@ static bool do_perm_pred2(DisasContext *s, arg_rr_esz *a, bool high_odd,
     return true;
 }
 
-static bool trans_ZIP1_p(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_perm_pred3(s, a, 0, gen_helper_sve_zip_p);
-}
-
-static bool trans_ZIP2_p(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_perm_pred3(s, a, 1, gen_helper_sve_zip_p);
-}
-
-static bool trans_UZP1_p(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_perm_pred3(s, a, 0, gen_helper_sve_uzp_p);
-}
-
-static bool trans_UZP2_p(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_perm_pred3(s, a, 1, gen_helper_sve_uzp_p);
-}
-
-static bool trans_TRN1_p(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_perm_pred3(s, a, 0, gen_helper_sve_trn_p);
-}
-
-static bool trans_TRN2_p(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_perm_pred3(s, a, 1, gen_helper_sve_trn_p);
-}
+TRANS_FEAT(ZIP1_p, aa64_sve, do_perm_pred3, a, 0, gen_helper_sve_zip_p)
+TRANS_FEAT(ZIP2_p, aa64_sve, do_perm_pred3, a, 1, gen_helper_sve_zip_p)
+TRANS_FEAT(UZP1_p, aa64_sve, do_perm_pred3, a, 0, gen_helper_sve_uzp_p)
+TRANS_FEAT(UZP2_p, aa64_sve, do_perm_pred3, a, 1, gen_helper_sve_uzp_p)
+TRANS_FEAT(TRN1_p, aa64_sve, do_perm_pred3, a, 0, gen_helper_sve_trn_p)
+TRANS_FEAT(TRN2_p, aa64_sve, do_perm_pred3, a, 1, gen_helper_sve_trn_p)
 
 static bool trans_REV_p(DisasContext *s, arg_rr_esz *a)
 {

From 1d0fce4bd0c6ceff32f14f8bab118351c4366a6c Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:09 -0700
Subject: [PATCH 641/935] target/arm: Use TRANS_FEAT for do_perm_pred2

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-57-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 17 +++--------------
 1 file changed, 3 insertions(+), 14 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 413e89b19c..1e6bcedb9d 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2279,20 +2279,9 @@ TRANS_FEAT(UZP2_p, aa64_sve, do_perm_pred3, a, 1, gen_helper_sve_uzp_p)
 TRANS_FEAT(TRN1_p, aa64_sve, do_perm_pred3, a, 0, gen_helper_sve_trn_p)
 TRANS_FEAT(TRN2_p, aa64_sve, do_perm_pred3, a, 1, gen_helper_sve_trn_p)
 
-static bool trans_REV_p(DisasContext *s, arg_rr_esz *a)
-{
-    return do_perm_pred2(s, a, 0, gen_helper_sve_rev_p);
-}
-
-static bool trans_PUNPKLO(DisasContext *s, arg_PUNPKLO *a)
-{
-    return do_perm_pred2(s, a, 0, gen_helper_sve_punpk_p);
-}
-
-static bool trans_PUNPKHI(DisasContext *s, arg_PUNPKHI *a)
-{
-    return do_perm_pred2(s, a, 1, gen_helper_sve_punpk_p);
-}
+TRANS_FEAT(REV_p, aa64_sve, do_perm_pred2, a, 0, gen_helper_sve_rev_p)
+TRANS_FEAT(PUNPKLO, aa64_sve, do_perm_pred2, a, 0, gen_helper_sve_punpk_p)
+TRANS_FEAT(PUNPKHI, aa64_sve, do_perm_pred2, a, 1, gen_helper_sve_punpk_p)
 
 /*
  *** SVE Permute - Interleaving Group

From 09eb6d7025d14bc01293a1bd6ecfcde985dec4d9 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:10 -0700
Subject: [PATCH 642/935] target/arm: Move sve zip high_ofs into simd_data

This is in line with how we treat uzp, and will
eliminate the special case code during translation.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-58-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/sve_helper.c    |  6 ++++--
 target/arm/translate-sve.c | 12 ++++++------
 2 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c
index e0f9aa9983..3bdcd4ce9d 100644
--- a/target/arm/sve_helper.c
+++ b/target/arm/sve_helper.c
@@ -3382,6 +3382,7 @@ void HELPER(sve_punpk_p)(void *vd, void *vn, uint32_t pred_desc)
 void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc)       \
 {                                                                    \
     intptr_t oprsz = simd_oprsz(desc);                               \
+    intptr_t odd_ofs = simd_data(desc);                              \
     intptr_t i, oprsz_2 = oprsz / 2;                                 \
     ARMVectorReg tmp_n, tmp_m;                                       \
     /* We produce output faster than we consume input.               \
@@ -3393,8 +3394,9 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc)       \
         vm = memcpy(&tmp_m, vm, oprsz_2);                            \
     }                                                                \
     for (i = 0; i < oprsz_2; i += sizeof(TYPE)) {                    \
-        *(TYPE *)(vd + H(2 * i + 0)) = *(TYPE *)(vn + H(i));         \
-        *(TYPE *)(vd + H(2 * i + sizeof(TYPE))) = *(TYPE *)(vm + H(i)); \
+        *(TYPE *)(vd + H(2 * i + 0)) = *(TYPE *)(vn + odd_ofs + H(i)); \
+        *(TYPE *)(vd + H(2 * i + sizeof(TYPE))) =                    \
+            *(TYPE *)(vm + odd_ofs + H(i));                          \
     }                                                                \
     if (sizeof(TYPE) == 16 && unlikely(oprsz & 16)) {                \
         memset(vd + oprsz - 16, 0, 16);                              \
diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 1e6bcedb9d..c2ced3e2bb 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2298,9 +2298,9 @@ static bool do_zip(DisasContext *s, arg_rrr_esz *a, bool high)
         unsigned vsz = vec_full_reg_size(s);
         unsigned high_ofs = high ? vsz / 2 : 0;
         tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn) + high_ofs,
-                           vec_full_reg_offset(s, a->rm) + high_ofs,
-                           vsz, vsz, 0, fns[a->esz]);
+                           vec_full_reg_offset(s, a->rn),
+                           vec_full_reg_offset(s, a->rm),
+                           vsz, vsz, high_ofs, fns[a->esz]);
     }
     return true;
 }
@@ -2324,9 +2324,9 @@ static bool do_zip_q(DisasContext *s, arg_rrr_esz *a, bool high)
         unsigned vsz = vec_full_reg_size(s);
         unsigned high_ofs = high ? QEMU_ALIGN_DOWN(vsz, 32) / 2 : 0;
         tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn) + high_ofs,
-                           vec_full_reg_offset(s, a->rm) + high_ofs,
-                           vsz, vsz, 0, gen_helper_sve2_zip_q);
+                           vec_full_reg_offset(s, a->rn),
+                           vec_full_reg_offset(s, a->rm),
+                           vsz, vsz, high_ofs, gen_helper_sve2_zip_q);
     }
     return true;
 }

From c9e0bd5f964efec6f1119c3aa622e712370789dd Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:11 -0700
Subject: [PATCH 643/935] target/arm: Use gen_gvec_ool_arg_zzz for do_zip,
 do_zip_q

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-59-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 25 +++++++------------------
 1 file changed, 7 insertions(+), 18 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index c2ced3e2bb..75c52d8ce1 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2293,16 +2293,10 @@ static bool do_zip(DisasContext *s, arg_rrr_esz *a, bool high)
         gen_helper_sve_zip_b, gen_helper_sve_zip_h,
         gen_helper_sve_zip_s, gen_helper_sve_zip_d,
     };
+    unsigned vsz = vec_full_reg_size(s);
+    unsigned high_ofs = high ? vsz / 2 : 0;
 
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        unsigned high_ofs = high ? vsz / 2 : 0;
-        tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           vsz, vsz, high_ofs, fns[a->esz]);
-    }
-    return true;
+    return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, high_ofs);
 }
 
 static bool trans_ZIP1_z(DisasContext *s, arg_rrr_esz *a)
@@ -2317,18 +2311,13 @@ static bool trans_ZIP2_z(DisasContext *s, arg_rrr_esz *a)
 
 static bool do_zip_q(DisasContext *s, arg_rrr_esz *a, bool high)
 {
+    unsigned vsz = vec_full_reg_size(s);
+    unsigned high_ofs = high ? QEMU_ALIGN_DOWN(vsz, 32) / 2 : 0;
+
     if (!dc_isar_feature(aa64_sve_f64mm, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        unsigned high_ofs = high ? QEMU_ALIGN_DOWN(vsz, 32) / 2 : 0;
-        tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           vsz, vsz, high_ofs, gen_helper_sve2_zip_q);
-    }
-    return true;
+    return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_zip_q, a, high_ofs);
 }
 
 static bool trans_ZIP1_q(DisasContext *s, arg_rrr_esz *a)

From a95b9618b0312b23e297b8f4631f530618c809f8 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:12 -0700
Subject: [PATCH 644/935] target/arm: Use TRANS_FEAT for do_zip, do_zip_q

Convert SVE translation functions using do_zip*
to use TRANS_FEAT and gen_gvec_ool_arg_zzz.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-60-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 54 +++++++++-----------------------------
 1 file changed, 13 insertions(+), 41 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 75c52d8ce1..7c9deb267f 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2287,48 +2287,20 @@ TRANS_FEAT(PUNPKHI, aa64_sve, do_perm_pred2, a, 1, gen_helper_sve_punpk_p)
  *** SVE Permute - Interleaving Group
  */
 
-static bool do_zip(DisasContext *s, arg_rrr_esz *a, bool high)
-{
-    static gen_helper_gvec_3 * const fns[4] = {
-        gen_helper_sve_zip_b, gen_helper_sve_zip_h,
-        gen_helper_sve_zip_s, gen_helper_sve_zip_d,
-    };
-    unsigned vsz = vec_full_reg_size(s);
-    unsigned high_ofs = high ? vsz / 2 : 0;
+static gen_helper_gvec_3 * const zip_fns[4] = {
+    gen_helper_sve_zip_b, gen_helper_sve_zip_h,
+    gen_helper_sve_zip_s, gen_helper_sve_zip_d,
+};
+TRANS_FEAT(ZIP1_z, aa64_sve, gen_gvec_ool_arg_zzz,
+           zip_fns[a->esz], a, 0)
+TRANS_FEAT(ZIP2_z, aa64_sve, gen_gvec_ool_arg_zzz,
+           zip_fns[a->esz], a, vec_full_reg_size(s) / 2)
 
-    return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, high_ofs);
-}
-
-static bool trans_ZIP1_z(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_zip(s, a, false);
-}
-
-static bool trans_ZIP2_z(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_zip(s, a, true);
-}
-
-static bool do_zip_q(DisasContext *s, arg_rrr_esz *a, bool high)
-{
-    unsigned vsz = vec_full_reg_size(s);
-    unsigned high_ofs = high ? QEMU_ALIGN_DOWN(vsz, 32) / 2 : 0;
-
-    if (!dc_isar_feature(aa64_sve_f64mm, s)) {
-        return false;
-    }
-    return gen_gvec_ool_arg_zzz(s, gen_helper_sve2_zip_q, a, high_ofs);
-}
-
-static bool trans_ZIP1_q(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_zip_q(s, a, false);
-}
-
-static bool trans_ZIP2_q(DisasContext *s, arg_rrr_esz *a)
-{
-    return do_zip_q(s, a, true);
-}
+TRANS_FEAT(ZIP1_q, aa64_sve_f64mm, gen_gvec_ool_arg_zzz,
+           gen_helper_sve2_zip_q, a, 0)
+TRANS_FEAT(ZIP2_q, aa64_sve_f64mm, gen_gvec_ool_arg_zzz,
+           gen_helper_sve2_zip_q, a,
+           QEMU_ALIGN_DOWN(vec_full_reg_size(s), 32) / 2)
 
 static gen_helper_gvec_3 * const uzp_fns[4] = {
     gen_helper_sve_uzp_b, gen_helper_sve_uzp_h,

From db7fa5d82156985f978da7d6985b4e8b5f451455 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:13 -0700
Subject: [PATCH 645/935] target/arm: Use TRANS_FEAT for do_clast_vector

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-61-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 7c9deb267f..5135866798 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2492,15 +2492,8 @@ static bool do_clast_vector(DisasContext *s, arg_rprr_esz *a, bool before)
     return true;
 }
 
-static bool trans_CLASTA_z(DisasContext *s, arg_rprr_esz *a)
-{
-    return do_clast_vector(s, a, false);
-}
-
-static bool trans_CLASTB_z(DisasContext *s, arg_rprr_esz *a)
-{
-    return do_clast_vector(s, a, true);
-}
+TRANS_FEAT(CLASTA_z, aa64_sve, do_clast_vector, a, false)
+TRANS_FEAT(CLASTB_z, aa64_sve, do_clast_vector, a, true)
 
 /* Compute CLAST for a scalar.  */
 static void do_clast_scalar(DisasContext *s, int esz, int pg, int rm,

From ac4fb247fb288d6a2bdeb6a4b6cbf68cf3f7b150 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:14 -0700
Subject: [PATCH 646/935] target/arm: Use TRANS_FEAT for do_clast_fp

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-62-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 5135866798..21c2bd099d 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2542,15 +2542,8 @@ static bool do_clast_fp(DisasContext *s, arg_rpr_esz *a, bool before)
     return true;
 }
 
-static bool trans_CLASTA_v(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_clast_fp(s, a, false);
-}
-
-static bool trans_CLASTB_v(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_clast_fp(s, a, true);
-}
+TRANS_FEAT(CLASTA_v, aa64_sve, do_clast_fp, a, false)
+TRANS_FEAT(CLASTB_v, aa64_sve, do_clast_fp, a, true)
 
 /* Compute CLAST for a Xreg.  */
 static bool do_clast_general(DisasContext *s, arg_rpr_esz *a, bool before)

From c673404a6371a03833bab46a28ed1f7f835eb086 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:15 -0700
Subject: [PATCH 647/935] target/arm: Use TRANS_FEAT for do_clast_general

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-63-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 21c2bd099d..f5453e99e1 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2575,15 +2575,8 @@ static bool do_clast_general(DisasContext *s, arg_rpr_esz *a, bool before)
     return true;
 }
 
-static bool trans_CLASTA_r(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_clast_general(s, a, false);
-}
-
-static bool trans_CLASTB_r(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_clast_general(s, a, true);
-}
+TRANS_FEAT(CLASTA_r, aa64_sve, do_clast_general, a, false)
+TRANS_FEAT(CLASTB_r, aa64_sve, do_clast_general, a, true)
 
 /* Compute LAST for a scalar.  */
 static TCGv_i64 do_last_scalar(DisasContext *s, int esz,

From 75de9fd4d240d5b74a4176b392f49ac3c8fe4675 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:16 -0700
Subject: [PATCH 648/935] target/arm: Use TRANS_FEAT for do_last_fp

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-64-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index f5453e99e1..841c1b5644 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2608,15 +2608,8 @@ static bool do_last_fp(DisasContext *s, arg_rpr_esz *a, bool before)
     return true;
 }
 
-static bool trans_LASTA_v(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_last_fp(s, a, false);
-}
-
-static bool trans_LASTB_v(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_last_fp(s, a, true);
-}
+TRANS_FEAT(LASTA_v, aa64_sve, do_last_fp, a, false)
+TRANS_FEAT(LASTB_v, aa64_sve, do_last_fp, a, true)
 
 /* Compute LAST for a Xreg.  */
 static bool do_last_general(DisasContext *s, arg_rpr_esz *a, bool before)

From 884c5a802e784c508840395586dc45f9e014b2ee Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:17 -0700
Subject: [PATCH 649/935] target/arm: Use TRANS_FEAT for do_last_general

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-65-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 841c1b5644..caa587506c 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2622,15 +2622,8 @@ static bool do_last_general(DisasContext *s, arg_rpr_esz *a, bool before)
     return true;
 }
 
-static bool trans_LASTA_r(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_last_general(s, a, false);
-}
-
-static bool trans_LASTB_r(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_last_general(s, a, true);
-}
+TRANS_FEAT(LASTA_r, aa64_sve, do_last_general, a, false)
+TRANS_FEAT(LASTB_r, aa64_sve, do_last_general, a, true)
 
 static bool trans_CPY_m_r(DisasContext *s, arg_rpr_esz *a)
 {

From 897ebd70742faa8d2f1e139813f37bc560950202 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:18 -0700
Subject: [PATCH 650/935] target/arm: Use TRANS_FEAT for SPLICE

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-66-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 17 ++++-------------
 1 file changed, 4 insertions(+), 13 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index caa587506c..8eb70fd56f 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2658,20 +2658,11 @@ TRANS_FEAT(REVH, aa64_sve, gen_gvec_ool_arg_zpz, revh_fns[a->esz], a, 0)
 TRANS_FEAT(REVW, aa64_sve, gen_gvec_ool_arg_zpz,
            a->esz == 3 ? gen_helper_sve_revw_d : NULL, a, 0)
 
-static bool trans_SPLICE(DisasContext *s, arg_rprr_esz *a)
-{
-    return gen_gvec_ool_zzzp(s, gen_helper_sve_splice,
-                             a->rd, a->rn, a->rm, a->pg, a->esz);
-}
+TRANS_FEAT(SPLICE, aa64_sve, gen_gvec_ool_arg_zpzz,
+           gen_helper_sve_splice, a, a->esz)
 
-static bool trans_SPLICE_sve2(DisasContext *s, arg_rpr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_ool_zzzp(s, gen_helper_sve_splice,
-                             a->rd, a->rn, (a->rn + 1) % 32, a->pg, a->esz);
-}
+TRANS_FEAT(SPLICE_sve2, aa64_sve2, gen_gvec_ool_zzzp, gen_helper_sve_splice,
+           a->rd, a->rn, (a->rn + 1) % 32, a->pg, a->esz)
 
 /*
  *** SVE Integer Compare - Vectors Group

From 671bdb2e1ffc8ed4e7e61264abcf6193494665c7 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:19 -0700
Subject: [PATCH 651/935] target/arm: Use TRANS_FEAT for do_ppzz_flags

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-67-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 28 ++++++++++++----------------
 1 file changed, 12 insertions(+), 16 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 8eb70fd56f..73b5b67c25 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2708,14 +2708,12 @@ static bool do_ppzz_flags(DisasContext *s, arg_rprr_esz *a,
 }
 
 #define DO_PPZZ(NAME, name) \
-static bool trans_##NAME##_ppzz(DisasContext *s, arg_rprr_esz *a)         \
-{                                                                         \
-    static gen_helper_gvec_flags_4 * const fns[4] = {                     \
-        gen_helper_sve_##name##_ppzz_b, gen_helper_sve_##name##_ppzz_h,   \
-        gen_helper_sve_##name##_ppzz_s, gen_helper_sve_##name##_ppzz_d,   \
-    };                                                                    \
-    return do_ppzz_flags(s, a, fns[a->esz]);                              \
-}
+    static gen_helper_gvec_flags_4 * const name##_ppzz_fns[4] = {       \
+        gen_helper_sve_##name##_ppzz_b, gen_helper_sve_##name##_ppzz_h, \
+        gen_helper_sve_##name##_ppzz_s, gen_helper_sve_##name##_ppzz_d, \
+    };                                                                  \
+    TRANS_FEAT(NAME##_ppzz, aa64_sve, do_ppzz_flags,                    \
+               a, name##_ppzz_fns[a->esz])
 
 DO_PPZZ(CMPEQ, cmpeq)
 DO_PPZZ(CMPNE, cmpne)
@@ -2727,14 +2725,12 @@ DO_PPZZ(CMPHS, cmphs)
 #undef DO_PPZZ
 
 #define DO_PPZW(NAME, name) \
-static bool trans_##NAME##_ppzw(DisasContext *s, arg_rprr_esz *a)         \
-{                                                                         \
-    static gen_helper_gvec_flags_4 * const fns[4] = {                     \
-        gen_helper_sve_##name##_ppzw_b, gen_helper_sve_##name##_ppzw_h,   \
-        gen_helper_sve_##name##_ppzw_s, NULL                              \
-    };                                                                    \
-    return do_ppzz_flags(s, a, fns[a->esz]);                              \
-}
+    static gen_helper_gvec_flags_4 * const name##_ppzw_fns[4] = {       \
+        gen_helper_sve_##name##_ppzw_b, gen_helper_sve_##name##_ppzw_h, \
+        gen_helper_sve_##name##_ppzw_s, NULL                            \
+    };                                                                  \
+    TRANS_FEAT(NAME##_ppzw, aa64_sve, do_ppzz_flags,                    \
+               a, name##_ppzw_fns[a->esz])
 
 DO_PPZW(CMPEQ, cmpeq)
 DO_PPZW(CMPNE, cmpne)

From ef75309be9b4619b29963df2a2b0b068c43bd5b2 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:20 -0700
Subject: [PATCH 652/935] target/arm: Use TRANS_FEAT for do_sve2_ppzz_flags

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-68-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 28 ++++++++--------------------
 1 file changed, 8 insertions(+), 20 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 73b5b67c25..22acd5ead0 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -7319,27 +7319,15 @@ DO_SVE2_ZZZ_NARROW(SUBHNT, subhnt)
 DO_SVE2_ZZZ_NARROW(RSUBHNB, rsubhnb)
 DO_SVE2_ZZZ_NARROW(RSUBHNT, rsubhnt)
 
-static bool do_sve2_ppzz_flags(DisasContext *s, arg_rprr_esz *a,
-                               gen_helper_gvec_flags_4 *fn)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return do_ppzz_flags(s, a, fn);
-}
+static gen_helper_gvec_flags_4 * const match_fns[4] = {
+    gen_helper_sve2_match_ppzz_b, gen_helper_sve2_match_ppzz_h, NULL, NULL
+};
+TRANS_FEAT(MATCH, aa64_sve2, do_ppzz_flags, a, match_fns[a->esz])
 
-#define DO_SVE2_PPZZ_MATCH(NAME, name)                                      \
-static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a)                  \
-{                                                                           \
-    static gen_helper_gvec_flags_4 * const fns[4] = {                       \
-        gen_helper_sve2_##name##_ppzz_b, gen_helper_sve2_##name##_ppzz_h,   \
-        NULL,                            NULL                               \
-    };                                                                      \
-    return do_sve2_ppzz_flags(s, a, fns[a->esz]);                           \
-}
-
-DO_SVE2_PPZZ_MATCH(MATCH, match)
-DO_SVE2_PPZZ_MATCH(NMATCH, nmatch)
+static gen_helper_gvec_flags_4 * const nmatch_fns[4] = {
+    gen_helper_sve2_nmatch_ppzz_b, gen_helper_sve2_nmatch_ppzz_h, NULL, NULL
+};
+TRANS_FEAT(NMATCH, aa64_sve2, do_ppzz_flags, a, nmatch_fns[a->esz])
 
 static gen_helper_gvec_4 * const histcnt_fns[4] = {
     NULL, NULL, gen_helper_sve2_histcnt_s, gen_helper_sve2_histcnt_d

From 9c545be60d0f04e65769117ac744bd372dc35ed4 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:21 -0700
Subject: [PATCH 653/935] target/arm: Use TRANS_FEAT for do_ppzi_flags

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-69-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 22acd5ead0..03b2eddd8b 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2786,14 +2786,12 @@ static bool do_ppzi_flags(DisasContext *s, arg_rpri_esz *a,
 }
 
 #define DO_PPZI(NAME, name) \
-static bool trans_##NAME##_ppzi(DisasContext *s, arg_rpri_esz *a)         \
-{                                                                         \
-    static gen_helper_gvec_flags_3 * const fns[4] = {                     \
+    static gen_helper_gvec_flags_3 * const name##_ppzi_fns[4] = {         \
         gen_helper_sve_##name##_ppzi_b, gen_helper_sve_##name##_ppzi_h,   \
         gen_helper_sve_##name##_ppzi_s, gen_helper_sve_##name##_ppzi_d,   \
     };                                                                    \
-    return do_ppzi_flags(s, a, fns[a->esz]);                              \
-}
+    TRANS_FEAT(NAME##_ppzi, aa64_sve, do_ppzi_flags, a,                   \
+               name##_ppzi_fns[a->esz])
 
 DO_PPZI(CMPEQ, cmpeq)
 DO_PPZI(CMPNE, cmpne)

From 2224d24d037b78d0c4ce7a34ee640c99b6c74dcf Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:22 -0700
Subject: [PATCH 654/935] target/arm: Use TRANS_FEAT for do_brk2, do_brk3

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-70-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 45 ++++++++++++--------------------------
 1 file changed, 14 insertions(+), 31 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 03b2eddd8b..d44b24e988 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2879,40 +2879,23 @@ static bool do_brk2(DisasContext *s, arg_rpr_s *a,
     return true;
 }
 
-static bool trans_BRKPA(DisasContext *s, arg_rprr_s *a)
-{
-    return do_brk3(s, a, gen_helper_sve_brkpa, gen_helper_sve_brkpas);
-}
+TRANS_FEAT(BRKPA, aa64_sve, do_brk3, a,
+           gen_helper_sve_brkpa, gen_helper_sve_brkpas)
+TRANS_FEAT(BRKPB, aa64_sve, do_brk3, a,
+           gen_helper_sve_brkpb, gen_helper_sve_brkpbs)
 
-static bool trans_BRKPB(DisasContext *s, arg_rprr_s *a)
-{
-    return do_brk3(s, a, gen_helper_sve_brkpb, gen_helper_sve_brkpbs);
-}
+TRANS_FEAT(BRKA_m, aa64_sve, do_brk2, a,
+           gen_helper_sve_brka_m, gen_helper_sve_brkas_m)
+TRANS_FEAT(BRKB_m, aa64_sve, do_brk2, a,
+           gen_helper_sve_brkb_m, gen_helper_sve_brkbs_m)
 
-static bool trans_BRKA_m(DisasContext *s, arg_rpr_s *a)
-{
-    return do_brk2(s, a, gen_helper_sve_brka_m, gen_helper_sve_brkas_m);
-}
+TRANS_FEAT(BRKA_z, aa64_sve, do_brk2, a,
+           gen_helper_sve_brka_z, gen_helper_sve_brkas_z)
+TRANS_FEAT(BRKB_z, aa64_sve, do_brk2, a,
+           gen_helper_sve_brkb_z, gen_helper_sve_brkbs_z)
 
-static bool trans_BRKB_m(DisasContext *s, arg_rpr_s *a)
-{
-    return do_brk2(s, a, gen_helper_sve_brkb_m, gen_helper_sve_brkbs_m);
-}
-
-static bool trans_BRKA_z(DisasContext *s, arg_rpr_s *a)
-{
-    return do_brk2(s, a, gen_helper_sve_brka_z, gen_helper_sve_brkas_z);
-}
-
-static bool trans_BRKB_z(DisasContext *s, arg_rpr_s *a)
-{
-    return do_brk2(s, a, gen_helper_sve_brkb_z, gen_helper_sve_brkbs_z);
-}
-
-static bool trans_BRKN(DisasContext *s, arg_rpr_s *a)
-{
-    return do_brk2(s, a, gen_helper_sve_brkn, gen_helper_sve_brkns);
-}
+TRANS_FEAT(BRKN, aa64_sve, do_brk2, a,
+           gen_helper_sve_brkn, gen_helper_sve_brkns)
 
 /*
  *** SVE Predicate Count Group

From fa4bd72cc19e7309038f656c91caf1f1d4a00cee Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:23 -0700
Subject: [PATCH 655/935] target/arm: Use TRANS_FEAT for MUL_zzi

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-71-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 10 +---------
 1 file changed, 1 insertion(+), 9 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index d44b24e988..c0781ecf60 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3315,15 +3315,7 @@ static bool trans_SUBR_zzi(DisasContext *s, arg_rri_esz *a)
     return true;
 }
 
-static bool trans_MUL_zzi(DisasContext *s, arg_rri_esz *a)
-{
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        tcg_gen_gvec_muli(a->esz, vec_full_reg_offset(s, a->rd),
-                          vec_full_reg_offset(s, a->rn), a->imm, vsz, vsz);
-    }
-    return true;
-}
+TRANS_FEAT(MUL_zzi, aa64_sve, gen_gvec_fn_arg_zzi, tcg_gen_gvec_muli, a)
 
 static bool do_zzi_sat(DisasContext *s, arg_rri_esz *a, bool u, bool d)
 {

From c437c59ba1842dc8488316412cb071d57d8231d8 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:24 -0700
Subject: [PATCH 656/935] target/arm: Reject dup_i w/ shifted byte early

Remove the unparsed extraction in trans_DUP_i,
which is intended to reject an 8-bit shift of
an 8-bit constant for 8-bit element.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-72-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/sve.decode      |  5 ++++-
 target/arm/translate-sve.c | 10 ++++++----
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/target/arm/sve.decode b/target/arm/sve.decode
index 0388cce3bd..c02da0a082 100644
--- a/target/arm/sve.decode
+++ b/target/arm/sve.decode
@@ -787,7 +787,10 @@ WHILE_ptr       00100101 esz:2 1 rm:5 001 100 rn:5 rw:1 rd:4
 FDUP            00100101 esz:2 111 00 1110 imm:8 rd:5
 
 # SVE broadcast integer immediate (unpredicated)
-DUP_i           00100101 esz:2 111 00 011 . ........ rd:5       imm=%sh8_i8s
+{
+  INVALID       00100101 00    111 00 011 1 -------- -----
+  DUP_i         00100101 esz:2 111 00 011 . ........ rd:5       imm=%sh8_i8s
+}
 
 # SVE integer add/subtract immediate (unpredicated)
 ADD_zzi         00100101 .. 100 000 11 . ........ .....         @rdn_sh_i8u
diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index c0781ecf60..14faef0564 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -403,6 +403,12 @@ const uint64_t pred_esz_masks[4] = {
     0x1111111111111111ull, 0x0101010101010101ull
 };
 
+static bool trans_INVALID(DisasContext *s, arg_INVALID *a)
+{
+    unallocated_encoding(s);
+    return true;
+}
+
 /*
  *** SVE Logical - Unpredicated Group
  */
@@ -3246,13 +3252,9 @@ static bool trans_FDUP(DisasContext *s, arg_FDUP *a)
 
 static bool trans_DUP_i(DisasContext *s, arg_DUP_i *a)
 {
-    if (a->esz == 0 && extract32(s->insn, 13, 1)) {
-        return false;
-    }
     if (sve_access_check(s)) {
         unsigned vsz = vec_full_reg_size(s);
         int dofs = vec_full_reg_offset(s, a->rd);
-
         tcg_gen_gvec_dup_imm(a->esz, dofs, vsz, vsz, a->imm);
     }
     return true;

From 3a40518079ff295b560b9ee193768e57a25007e2 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:25 -0700
Subject: [PATCH 657/935] target/arm: Reject add/sub w/ shifted byte early

Remove the unparsed extractions in trans_ADD_zzi, trans_SUBR_zzi,
and do_zzi_sat which are intended to reject an 8-bit shift of an
8-bit constant for 8-bit element.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-73-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/sve.decode      | 35 ++++++++++++++++++++++++++++-------
 target/arm/translate-sve.c |  9 ---------
 2 files changed, 28 insertions(+), 16 deletions(-)

diff --git a/target/arm/sve.decode b/target/arm/sve.decode
index c02da0a082..8cff63cf25 100644
--- a/target/arm/sve.decode
+++ b/target/arm/sve.decode
@@ -793,13 +793,34 @@ FDUP            00100101 esz:2 111 00 1110 imm:8 rd:5
 }
 
 # SVE integer add/subtract immediate (unpredicated)
-ADD_zzi         00100101 .. 100 000 11 . ........ .....         @rdn_sh_i8u
-SUB_zzi         00100101 .. 100 001 11 . ........ .....         @rdn_sh_i8u
-SUBR_zzi        00100101 .. 100 011 11 . ........ .....         @rdn_sh_i8u
-SQADD_zzi       00100101 .. 100 100 11 . ........ .....         @rdn_sh_i8u
-UQADD_zzi       00100101 .. 100 101 11 . ........ .....         @rdn_sh_i8u
-SQSUB_zzi       00100101 .. 100 110 11 . ........ .....         @rdn_sh_i8u
-UQSUB_zzi       00100101 .. 100 111 11 . ........ .....         @rdn_sh_i8u
+{
+  INVALID       00100101 00 100 000 11 1 -------- -----
+  ADD_zzi       00100101 .. 100 000 11 . ........ .....         @rdn_sh_i8u
+}
+{
+  INVALID       00100101 00 100 001 11 1 -------- -----
+  SUB_zzi       00100101 .. 100 001 11 . ........ .....         @rdn_sh_i8u
+}
+{
+  INVALID       00100101 00 100 011 11 1 -------- -----
+  SUBR_zzi      00100101 .. 100 011 11 . ........ .....         @rdn_sh_i8u
+}
+{
+  INVALID       00100101 00 100 100 11 1 -------- -----
+  SQADD_zzi     00100101 .. 100 100 11 . ........ .....         @rdn_sh_i8u
+}
+{
+  INVALID       00100101 00 100 101 11 1 -------- -----
+  UQADD_zzi     00100101 .. 100 101 11 . ........ .....         @rdn_sh_i8u
+}
+{
+  INVALID       00100101 00 100 110 11 1 -------- -----
+  SQSUB_zzi     00100101 .. 100 110 11 . ........ .....         @rdn_sh_i8u
+}
+{
+  INVALID       00100101 00 100 111 11 1 -------- -----
+  UQSUB_zzi     00100101 .. 100 111 11 . ........ .....         @rdn_sh_i8u
+}
 
 # SVE integer min/max immediate (unpredicated)
 SMAX_zzi        00100101 .. 101 000 110 ........ .....          @rdn_i8s
diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 14faef0564..bf988cab3e 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3262,9 +3262,6 @@ static bool trans_DUP_i(DisasContext *s, arg_DUP_i *a)
 
 static bool trans_ADD_zzi(DisasContext *s, arg_rri_esz *a)
 {
-    if (a->esz == 0 && extract32(s->insn, 13, 1)) {
-        return false;
-    }
     return gen_gvec_fn_arg_zzi(s, tcg_gen_gvec_addi, a);
 }
 
@@ -3305,9 +3302,6 @@ static bool trans_SUBR_zzi(DisasContext *s, arg_rri_esz *a)
           .scalar_first = true }
     };
 
-    if (a->esz == 0 && extract32(s->insn, 13, 1)) {
-        return false;
-    }
     if (sve_access_check(s)) {
         unsigned vsz = vec_full_reg_size(s);
         tcg_gen_gvec_2s(vec_full_reg_offset(s, a->rd),
@@ -3321,9 +3315,6 @@ TRANS_FEAT(MUL_zzi, aa64_sve, gen_gvec_fn_arg_zzi, tcg_gen_gvec_muli, a)
 
 static bool do_zzi_sat(DisasContext *s, arg_rri_esz *a, bool u, bool d)
 {
-    if (a->esz == 0 && extract32(s->insn, 13, 1)) {
-        return false;
-    }
     if (sve_access_check(s)) {
         do_sat_addsub_vec(s, a->esz, a->rd, a->rn,
                           tcg_constant_i64(a->imm), u, d);

From 7836c9414f51257d673b794abd562c770590e4fd Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:26 -0700
Subject: [PATCH 658/935] target/arm: Reject copy w/ shifted byte early

Remove the unparsed extractions in trans_CPY_{m,z}_i which are intended
to reject an 8-bit shift of an 8-bit constant for 8-bit element.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-74-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/sve.decode      | 10 ++++++++--
 target/arm/translate-sve.c |  6 ------
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/target/arm/sve.decode b/target/arm/sve.decode
index 8cff63cf25..7e79198f5b 100644
--- a/target/arm/sve.decode
+++ b/target/arm/sve.decode
@@ -528,8 +528,14 @@ DUPM            00000101 11 0000 dbm:13 rd:5
 FCPY            00000101 .. 01 .... 110 imm:8 .....             @rdn_pg4
 
 # SVE copy integer immediate (predicated)
-CPY_m_i         00000101 .. 01 .... 01 . ........ .....   @rdn_pg4 imm=%sh8_i8s
-CPY_z_i         00000101 .. 01 .... 00 . ........ .....   @rdn_pg4 imm=%sh8_i8s
+{
+  INVALID       00000101 00 01 ---- 01 1 -------- -----
+  CPY_m_i       00000101 .. 01 .... 01 . ........ .....   @rdn_pg4 imm=%sh8_i8s
+}
+{
+  INVALID       00000101 00 01 ---- 00 1 -------- -----
+  CPY_z_i       00000101 .. 01 .... 00 . ........ .....   @rdn_pg4 imm=%sh8_i8s
+}
 
 ### SVE Permute - Extract Group
 
diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index bf988cab3e..83980f5ee6 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -2024,9 +2024,6 @@ static bool trans_FCPY(DisasContext *s, arg_FCPY *a)
 
 static bool trans_CPY_m_i(DisasContext *s, arg_rpri_esz *a)
 {
-    if (a->esz == 0 && extract32(s->insn, 13, 1)) {
-        return false;
-    }
     if (sve_access_check(s)) {
         do_cpy_m(s, a->esz, a->rd, a->rn, a->pg, tcg_constant_i64(a->imm));
     }
@@ -2040,9 +2037,6 @@ static bool trans_CPY_z_i(DisasContext *s, arg_CPY_z_i *a)
         gen_helper_sve_cpy_z_s, gen_helper_sve_cpy_z_d,
     };
 
-    if (a->esz == 0 && extract32(s->insn, 13, 1)) {
-        return false;
-    }
     if (sve_access_check(s)) {
         unsigned vsz = vec_full_reg_size(s);
         tcg_gen_gvec_2i_ool(vec_full_reg_offset(s, a->rd),

From 48ca613df5c914b898f9bcb32b8711db5b2e8f0c Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:27 -0700
Subject: [PATCH 659/935] target/arm: Use TRANS_FEAT for ADD_zzi

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-75-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 83980f5ee6..6b2f235e4a 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3254,10 +3254,7 @@ static bool trans_DUP_i(DisasContext *s, arg_DUP_i *a)
     return true;
 }
 
-static bool trans_ADD_zzi(DisasContext *s, arg_rri_esz *a)
-{
-    return gen_gvec_fn_arg_zzi(s, tcg_gen_gvec_addi, a);
-}
+TRANS_FEAT(ADD_zzi, aa64_sve, gen_gvec_fn_arg_zzi, tcg_gen_gvec_addi, a)
 
 static bool trans_SUB_zzi(DisasContext *s, arg_rri_esz *a)
 {

From 17b54d1c7877577e1e7fd568be0b281bad83a7c5 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:28 -0700
Subject: [PATCH 660/935] target/arm: Use TRANS_FEAT for do_zzi_sat

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-76-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 23 ++++-------------------
 1 file changed, 4 insertions(+), 19 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 6b2f235e4a..e6434589f4 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3313,25 +3313,10 @@ static bool do_zzi_sat(DisasContext *s, arg_rri_esz *a, bool u, bool d)
     return true;
 }
 
-static bool trans_SQADD_zzi(DisasContext *s, arg_rri_esz *a)
-{
-    return do_zzi_sat(s, a, false, false);
-}
-
-static bool trans_UQADD_zzi(DisasContext *s, arg_rri_esz *a)
-{
-    return do_zzi_sat(s, a, true, false);
-}
-
-static bool trans_SQSUB_zzi(DisasContext *s, arg_rri_esz *a)
-{
-    return do_zzi_sat(s, a, false, true);
-}
-
-static bool trans_UQSUB_zzi(DisasContext *s, arg_rri_esz *a)
-{
-    return do_zzi_sat(s, a, true, true);
-}
+TRANS_FEAT(SQADD_zzi, aa64_sve, do_zzi_sat, a, false, false)
+TRANS_FEAT(UQADD_zzi, aa64_sve, do_zzi_sat, a, true, false)
+TRANS_FEAT(SQSUB_zzi, aa64_sve, do_zzi_sat, a, false, true)
+TRANS_FEAT(UQSUB_zzi, aa64_sve, do_zzi_sat, a, true, true)
 
 static bool do_zzi_ool(DisasContext *s, arg_rri_esz *a, gen_helper_gvec_2i *fn)
 {

From ef4a3958633e0ca9e0e656d5fbef002c23329aac Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:29 -0700
Subject: [PATCH 661/935] target/arm: Use TRANS_FEAT for do_zzi_ool

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-77-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index e6434589f4..b8bd1047b0 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3330,14 +3330,11 @@ static bool do_zzi_ool(DisasContext *s, arg_rri_esz *a, gen_helper_gvec_2i *fn)
 }
 
 #define DO_ZZI(NAME, name) \
-static bool trans_##NAME##_zzi(DisasContext *s, arg_rri_esz *a)         \
-{                                                                       \
-    static gen_helper_gvec_2i * const fns[4] = {                        \
+    static gen_helper_gvec_2i * const name##i_fns[4] = {                \
         gen_helper_sve_##name##i_b, gen_helper_sve_##name##i_h,         \
         gen_helper_sve_##name##i_s, gen_helper_sve_##name##i_d,         \
     };                                                                  \
-    return do_zzi_ool(s, a, fns[a->esz]);                               \
-}
+    TRANS_FEAT(NAME##_zzi, aa64_sve, do_zzi_ool, a, name##i_fns[a->esz])
 
 DO_ZZI(SMAX, smax)
 DO_ZZI(UMAX, umax)

From 41bf9b679953e54331cbf703dbd1ef6a7a779ea1 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:30 -0700
Subject: [PATCH 662/935] target/arm: Introduce gen_gvec_{ptr,fpst}_zzzz

Use these for the several varieties of floating-point
multiply-add instructions.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-78-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 140 ++++++++++++++-----------------------
 1 file changed, 53 insertions(+), 87 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index b8bd1047b0..a799ce3110 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -205,6 +205,35 @@ static bool gen_gvec_ool_arg_zzxz(DisasContext *s, gen_helper_gvec_4 *fn,
     return gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->index);
 }
 
+/* Invoke an out-of-line helper on 4 Zregs, plus a pointer. */
+static bool gen_gvec_ptr_zzzz(DisasContext *s, gen_helper_gvec_4_ptr *fn,
+                              int rd, int rn, int rm, int ra,
+                              int data, TCGv_ptr ptr)
+{
+    if (fn == NULL) {
+        return false;
+    }
+    if (sve_access_check(s)) {
+        unsigned vsz = vec_full_reg_size(s);
+        tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, rd),
+                           vec_full_reg_offset(s, rn),
+                           vec_full_reg_offset(s, rm),
+                           vec_full_reg_offset(s, ra),
+                           ptr, vsz, vsz, data, fn);
+    }
+    return true;
+}
+
+static bool gen_gvec_fpst_zzzz(DisasContext *s, gen_helper_gvec_4_ptr *fn,
+                               int rd, int rn, int rm, int ra,
+                               int data, ARMFPStatusFlavour flavour)
+{
+    TCGv_ptr status = fpstatus_ptr(flavour);
+    bool ret = gen_gvec_ptr_zzzz(s, fn, rd, rn, rm, ra, data, status);
+    tcg_temp_free_ptr(status);
+    return ret;
+}
+
 /* Invoke an out-of-line helper on 2 Zregs and a predicate. */
 static bool gen_gvec_ool_zzp(DisasContext *s, gen_helper_gvec_3 *fn,
                              int rd, int rn, int pg, int data)
@@ -3485,24 +3514,15 @@ DO_SVE2_RRXR_ROT(CDOT_zzxw_d, gen_helper_sve2_cdot_idx_d)
 
 static bool do_FMLA_zzxz(DisasContext *s, arg_rrxr_esz *a, bool sub)
 {
-    static gen_helper_gvec_4_ptr * const fns[3] = {
+    static gen_helper_gvec_4_ptr * const fns[4] = {
+        NULL,
         gen_helper_gvec_fmla_idx_h,
         gen_helper_gvec_fmla_idx_s,
         gen_helper_gvec_fmla_idx_d,
     };
-
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
-        tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           vec_full_reg_offset(s, a->ra),
-                           status, vsz, vsz, (a->index << 1) | sub,
-                           fns[a->esz - 1]);
-        tcg_temp_free_ptr(status);
-    }
-    return true;
+    return gen_gvec_fpst_zzzz(s, fns[a->esz], a->rd, a->rn, a->rm, a->ra,
+                              (a->index << 1) | sub,
+                              a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
 }
 
 static bool trans_FMLA_zzxz(DisasContext *s, arg_FMLA_zzxz *a)
@@ -4040,26 +4060,18 @@ static bool trans_FCMLA_zpzzz(DisasContext *s, arg_FCMLA_zpzzz *a)
 
 static bool trans_FCMLA_zzxz(DisasContext *s, arg_FCMLA_zzxz *a)
 {
-    static gen_helper_gvec_4_ptr * const fns[2] = {
+    static gen_helper_gvec_4_ptr * const fns[4] = {
+        NULL,
         gen_helper_gvec_fcmlah_idx,
         gen_helper_gvec_fcmlas_idx,
+        NULL,
     };
 
-    tcg_debug_assert(a->esz == 1 || a->esz == 2);
     tcg_debug_assert(a->rd == a->ra);
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
-        tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           vec_full_reg_offset(s, a->ra),
-                           status, vsz, vsz,
-                           a->index * 4 + a->rot,
-                           fns[a->esz - 1]);
-        tcg_temp_free_ptr(status);
-    }
-    return true;
+
+    return gen_gvec_fpst_zzzz(s, fns[a->esz], a->rd, a->rn, a->rm, a->ra,
+                              a->index * 4 + a->rot,
+                              a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
 }
 
 /*
@@ -7327,17 +7339,7 @@ static bool trans_FMMLA(DisasContext *s, arg_rrrr_esz *a)
         return false;
     }
 
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        TCGv_ptr status = fpstatus_ptr(FPST_FPCR);
-        tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           vec_full_reg_offset(s, a->ra),
-                           status, vsz, vsz, 0, fn);
-        tcg_temp_free_ptr(status);
-    }
-    return true;
+    return gen_gvec_fpst_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, 0, FPST_FPCR);
 }
 
 static gen_helper_gvec_4 * const sqdmlal_zzzw_fns[] = {
@@ -7535,16 +7537,9 @@ static bool do_FMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sub, bool sel)
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           vec_full_reg_offset(s, a->ra),
-                           cpu_env, vsz, vsz, (sel << 1) | sub,
-                           gen_helper_sve2_fmlal_zzzw_s);
-    }
-    return true;
+    return gen_gvec_ptr_zzzz(s, gen_helper_sve2_fmlal_zzzw_s,
+                             a->rd, a->rn, a->rm, a->ra,
+                             (sel << 1) | sub, cpu_env);
 }
 
 static bool trans_FMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a)
@@ -7572,17 +7567,9 @@ static bool do_FMLAL_zzxw(DisasContext *s, arg_rrxr_esz *a, bool sub, bool sel)
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           vec_full_reg_offset(s, a->ra),
-                           cpu_env, vsz, vsz,
-                           (a->index << 2) | (sel << 1) | sub,
-                           gen_helper_sve2_fmlal_zzxw_s);
-    }
-    return true;
+    return gen_gvec_ptr_zzzz(s, gen_helper_sve2_fmlal_zzxw_s,
+                             a->rd, a->rn, a->rm, a->ra,
+                             (a->index << 2) | (sel << 1) | sub, cpu_env);
 }
 
 static bool trans_FMLALB_zzxw(DisasContext *s, arg_rrxr_esz *a)
@@ -7625,19 +7612,8 @@ static bool do_BFMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel)
     if (!dc_isar_feature(aa64_sve_bf16, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        TCGv_ptr status = fpstatus_ptr(FPST_FPCR);
-        unsigned vsz = vec_full_reg_size(s);
-
-        tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           vec_full_reg_offset(s, a->ra),
-                           status, vsz, vsz, sel,
-                           gen_helper_gvec_bfmlal);
-        tcg_temp_free_ptr(status);
-    }
-    return true;
+    return gen_gvec_fpst_zzzz(s, gen_helper_gvec_bfmlal,
+                              a->rd, a->rn, a->rm, a->ra, sel, FPST_FPCR);
 }
 
 static bool trans_BFMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a)
@@ -7655,19 +7631,9 @@ static bool do_BFMLAL_zzxw(DisasContext *s, arg_rrxr_esz *a, bool sel)
     if (!dc_isar_feature(aa64_sve_bf16, s)) {
         return false;
     }
-    if (sve_access_check(s)) {
-        TCGv_ptr status = fpstatus_ptr(FPST_FPCR);
-        unsigned vsz = vec_full_reg_size(s);
-
-        tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           vec_full_reg_offset(s, a->ra),
-                           status, vsz, vsz, (a->index << 1) | sel,
-                           gen_helper_gvec_bfmlal_idx);
-        tcg_temp_free_ptr(status);
-    }
-    return true;
+    return gen_gvec_fpst_zzzz(s, gen_helper_gvec_bfmlal_idx,
+                              a->rd, a->rn, a->rm, a->ra,
+                              (a->index << 1) | sel, FPST_FPCR);
 }
 
 static bool trans_BFMLALB_zzxw(DisasContext *s, arg_rrxr_esz *a)

From 25aee7cc3bd0c9a1b273ec2ead99ddde42f5c04a Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:31 -0700
Subject: [PATCH 663/935] target/arm: Use TRANS_FEAT for FMMLA

Being able to specify the feature predicate in TRANS_FEAT
makes it easier to split trans_FMMLA by element size,
which also happens to simplify the decode.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-79-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/sve.decode      |  7 +++----
 target/arm/translate-sve.c | 27 ++++-----------------------
 2 files changed, 7 insertions(+), 27 deletions(-)

diff --git a/target/arm/sve.decode b/target/arm/sve.decode
index 7e79198f5b..a54feb2f61 100644
--- a/target/arm/sve.decode
+++ b/target/arm/sve.decode
@@ -1598,10 +1598,9 @@ SQRDCMLAH_zzzz  01000100 esz:2 0 rm:5 0011 rot:2 rn:5 rd:5  ra=%reg_movprfx
 USDOT_zzzz      01000100 .. 0 ..... 011 110 ..... .....  @rda_rn_rm
 
 ### SVE2 floating point matrix multiply accumulate
-{
-  BFMMLA        01100100 01 1 ..... 111 001 ..... .....  @rda_rn_rm_e0
-  FMMLA         01100100 .. 1 ..... 111 001 ..... .....  @rda_rn_rm
-}
+BFMMLA          01100100 01 1 ..... 111 001 ..... .....  @rda_rn_rm_e0
+FMMLA_s         01100100 10 1 ..... 111 001 ..... .....  @rda_rn_rm_e0
+FMMLA_d         01100100 11 1 ..... 111 001 ..... .....  @rda_rn_rm_e0
 
 ### SVE2 Memory Gather Load Group
 
diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index a799ce3110..364e419f3e 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -7318,29 +7318,10 @@ DO_SVE2_ZPZZ_FP(FMINP, fminp)
  * SVE Integer Multiply-Add (unpredicated)
  */
 
-static bool trans_FMMLA(DisasContext *s, arg_rrrr_esz *a)
-{
-    gen_helper_gvec_4_ptr *fn;
-
-    switch (a->esz) {
-    case MO_32:
-        if (!dc_isar_feature(aa64_sve_f32mm, s)) {
-            return false;
-        }
-        fn = gen_helper_fmmla_s;
-        break;
-    case MO_64:
-        if (!dc_isar_feature(aa64_sve_f64mm, s)) {
-            return false;
-        }
-        fn = gen_helper_fmmla_d;
-        break;
-    default:
-        return false;
-    }
-
-    return gen_gvec_fpst_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, 0, FPST_FPCR);
-}
+TRANS_FEAT(FMMLA_s, aa64_sve_f32mm, gen_gvec_fpst_zzzz, gen_helper_fmmla_s,
+           a->rd, a->rn, a->rm, a->ra, 0, FPST_FPCR)
+TRANS_FEAT(FMMLA_d, aa64_sve_f64mm, gen_gvec_fpst_zzzz, gen_helper_fmmla_d,
+           a->rd, a->rn, a->rm, a->ra, 0, FPST_FPCR)
 
 static gen_helper_gvec_4 * const sqdmlal_zzzw_fns[] = {
     NULL,                           gen_helper_sve2_sqdmlal_zzzw_h,

From 23e5fa5f90a6073f270ec89418085e4cb341a4ea Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:32 -0700
Subject: [PATCH 664/935] target/arm: Move sve check into gen_gvec_fn_ppp

Combined with the check already present in gen_mov_p,
we can simplify some special cases in trans_AND_pppp
and trans_BIC_pppp.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-80-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 30 ++++++++++++------------------
 1 file changed, 12 insertions(+), 18 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 364e419f3e..f33bc9d480 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -370,13 +370,16 @@ static void do_dupi_z(DisasContext *s, int rd, uint64_t word)
 }
 
 /* Invoke a vector expander on three Pregs.  */
-static void gen_gvec_fn_ppp(DisasContext *s, GVecGen3Fn *gvec_fn,
+static bool gen_gvec_fn_ppp(DisasContext *s, GVecGen3Fn *gvec_fn,
                             int rd, int rn, int rm)
 {
-    unsigned psz = pred_gvec_reg_size(s);
-    gvec_fn(MO_64, pred_full_reg_offset(s, rd),
-            pred_full_reg_offset(s, rn),
-            pred_full_reg_offset(s, rm), psz, psz);
+    if (sve_access_check(s)) {
+        unsigned psz = pred_gvec_reg_size(s);
+        gvec_fn(MO_64, pred_full_reg_offset(s, rd),
+                pred_full_reg_offset(s, rn),
+                pred_full_reg_offset(s, rm), psz, psz);
+    }
+    return true;
 }
 
 /* Invoke a vector move on two Pregs.  */
@@ -1317,19 +1320,13 @@ static bool trans_AND_pppp(DisasContext *s, arg_rprr_s *a)
     };
 
     if (!a->s) {
-        if (!sve_access_check(s)) {
-            return true;
-        }
         if (a->rn == a->rm) {
             if (a->pg == a->rn) {
-                do_mov_p(s, a->rd, a->rn);
-            } else {
-                gen_gvec_fn_ppp(s, tcg_gen_gvec_and, a->rd, a->rn, a->pg);
+                return do_mov_p(s, a->rd, a->rn);
             }
-            return true;
+            return gen_gvec_fn_ppp(s, tcg_gen_gvec_and, a->rd, a->rn, a->pg);
         } else if (a->pg == a->rn || a->pg == a->rm) {
-            gen_gvec_fn_ppp(s, tcg_gen_gvec_and, a->rd, a->rn, a->rm);
-            return true;
+            return gen_gvec_fn_ppp(s, tcg_gen_gvec_and, a->rd, a->rn, a->rm);
         }
     }
     return do_pppp_flags(s, a, &op);
@@ -1358,10 +1355,7 @@ static bool trans_BIC_pppp(DisasContext *s, arg_rprr_s *a)
     };
 
     if (!a->s && a->pg == a->rn) {
-        if (sve_access_check(s)) {
-            gen_gvec_fn_ppp(s, tcg_gen_gvec_andc, a->rd, a->rn, a->rm);
-        }
-        return true;
+        return gen_gvec_fn_ppp(s, tcg_gen_gvec_andc, a->rd, a->rn, a->rm);
     }
     return do_pppp_flags(s, a, &op);
 }

From 738b679cc961bd086f14d7d7e3f4be116a719810 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:33 -0700
Subject: [PATCH 665/935] target/arm: Implement NOT (prediates) alias

This alias is defined on EOR (prediates).  While the
same operation could be performed with NAND or NOR,
only bother with the official alias.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-81-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index f33bc9d480..b6b5980e2d 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -1381,6 +1381,11 @@ static bool trans_EOR_pppp(DisasContext *s, arg_rprr_s *a)
         .fno = gen_helper_sve_eor_pppp,
         .prefer_i64 = TCG_TARGET_REG_BITS == 64,
     };
+
+    /* Alias NOT (predicate) is EOR Pd.B, Pg/Z, Pn.B, Pg.B */
+    if (!a->s && a->pg == a->rm) {
+        return gen_gvec_fn_ppp(s, tcg_gen_gvec_andc, a->rd, a->pg, a->rn);
+    }
     return do_pppp_flags(s, a, &op);
 }
 

From 29693f5f97c716718f05e601ea2b954b4a1236de Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:34 -0700
Subject: [PATCH 666/935] target/arm: Use TRANS_FEAT for SEL_zpzz

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-82-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index b6b5980e2d..a040d694ea 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -795,10 +795,7 @@ static gen_helper_gvec_4 * const udiv_fns[4] = {
 };
 TRANS_FEAT(UDIV_zpzz, aa64_sve, gen_gvec_ool_arg_zpzz, udiv_fns[a->esz], a, 0)
 
-static bool trans_SEL_zpzz(DisasContext *s, arg_rprr_esz *a)
-{
-    return do_sel_z(s, a->rd, a->rn, a->rm, a->pg, a->esz);
-}
+TRANS_FEAT(SEL_zpzz, aa64_sve, do_sel_z, a->rd, a->rn, a->rm, a->pg, a->esz)
 
 /*
  *** SVE Integer Arithmetic - Unary Predicated Group

From 4b0b37e900700889f7cecd812bd05005fb2fbd57 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:35 -0700
Subject: [PATCH 667/935] target/arm: Use TRANS_FEAT for MOVPRFX

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-83-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 17 +++--------------
 1 file changed, 3 insertions(+), 14 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index a040d694ea..6e8d8d54bf 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -6054,20 +6054,9 @@ static bool trans_PRF_rr(DisasContext *s, arg_PRF_rr *a)
  * In the meantime, just emit the moves.
  */
 
-static bool trans_MOVPRFX(DisasContext *s, arg_MOVPRFX *a)
-{
-    return do_mov_z(s, a->rd, a->rn);
-}
-
-static bool trans_MOVPRFX_m(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_sel_z(s, a->rd, a->rn, a->rd, a->pg, a->esz);
-}
-
-static bool trans_MOVPRFX_z(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_movz_zpz(s, a->rd, a->rn, a->pg, a->esz, false);
-}
+TRANS_FEAT(MOVPRFX, aa64_sve, do_mov_z, a->rd, a->rn)
+TRANS_FEAT(MOVPRFX_m, aa64_sve, do_sel_z, a->rd, a->rn, a->rd, a->pg, a->esz)
+TRANS_FEAT(MOVPRFX_z, aa64_sve, do_movz_zpz, a->rd, a->rn, a->pg, a->esz, false)
 
 /*
  * SVE2 Integer Multiply - Unpredicated

From 3b879c28826e3d731e2bea427d0ca09aadd19c29 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:36 -0700
Subject: [PATCH 668/935] target/arm: Use TRANS_FEAT for FMLA

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-84-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 6e8d8d54bf..5aa3e477cf 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3521,15 +3521,8 @@ static bool do_FMLA_zzxz(DisasContext *s, arg_rrxr_esz *a, bool sub)
                               a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
 }
 
-static bool trans_FMLA_zzxz(DisasContext *s, arg_FMLA_zzxz *a)
-{
-    return do_FMLA_zzxz(s, a, false);
-}
-
-static bool trans_FMLS_zzxz(DisasContext *s, arg_FMLA_zzxz *a)
-{
-    return do_FMLA_zzxz(s, a, true);
-}
+TRANS_FEAT(FMLA_zzxz, aa64_sve, do_FMLA_zzxz, a, false)
+TRANS_FEAT(FMLS_zzxz, aa64_sve, do_FMLA_zzxz, a, true)
 
 /*
  *** SVE Floating Point Multiply Indexed Group

From 698ddb9d02aea308631a538fefb243547b884fe1 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:37 -0700
Subject: [PATCH 669/935] target/arm: Use TRANS_FEAT for BFMLA

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-85-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 28 ++++------------------------
 1 file changed, 4 insertions(+), 24 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 5aa3e477cf..f2939fbeb9 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -7568,39 +7568,19 @@ TRANS_FEAT(BFMMLA, aa64_sve_bf16, gen_gvec_ool_arg_zzzz,
 
 static bool do_BFMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel)
 {
-    if (!dc_isar_feature(aa64_sve_bf16, s)) {
-        return false;
-    }
     return gen_gvec_fpst_zzzz(s, gen_helper_gvec_bfmlal,
                               a->rd, a->rn, a->rm, a->ra, sel, FPST_FPCR);
 }
 
-static bool trans_BFMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_BFMLAL_zzzw(s, a, false);
-}
-
-static bool trans_BFMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_BFMLAL_zzzw(s, a, true);
-}
+TRANS_FEAT(BFMLALB_zzzw, aa64_sve_bf16, do_BFMLAL_zzzw, a, false)
+TRANS_FEAT(BFMLALT_zzzw, aa64_sve_bf16, do_BFMLAL_zzzw, a, true)
 
 static bool do_BFMLAL_zzxw(DisasContext *s, arg_rrxr_esz *a, bool sel)
 {
-    if (!dc_isar_feature(aa64_sve_bf16, s)) {
-        return false;
-    }
     return gen_gvec_fpst_zzzz(s, gen_helper_gvec_bfmlal_idx,
                               a->rd, a->rn, a->rm, a->ra,
                               (a->index << 1) | sel, FPST_FPCR);
 }
 
-static bool trans_BFMLALB_zzxw(DisasContext *s, arg_rrxr_esz *a)
-{
-    return do_BFMLAL_zzxw(s, a, false);
-}
-
-static bool trans_BFMLALT_zzxw(DisasContext *s, arg_rrxr_esz *a)
-{
-    return do_BFMLAL_zzxw(s, a, true);
-}
+TRANS_FEAT(BFMLALB_zzxw, aa64_sve_bf16, do_BFMLAL_zzxw, a, false)
+TRANS_FEAT(BFMLALT_zzxw, aa64_sve_bf16, do_BFMLAL_zzxw, a, true)

From 532724e439fb8911644d3592b21c0e7e0bffc5d1 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:38 -0700
Subject: [PATCH 670/935] target/arm: Rename do_zzz_fp to
 gen_gvec_ool_fpst_arg_zzz

Rename the function to match gen_gvec_ool_arg_zzz,
and move to be adjacent.  Split out gen_gvec_fpst_zzz
as a helper while we're at it.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-86-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 50 +++++++++++++++++++++++---------------
 1 file changed, 30 insertions(+), 20 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index f2939fbeb9..61bf5f5757 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -175,6 +175,35 @@ static bool gen_gvec_ool_arg_zzz(DisasContext *s, gen_helper_gvec_3 *fn,
     return gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, data);
 }
 
+/* Invoke an out-of-line helper on 3 Zregs, plus float_status. */
+static bool gen_gvec_fpst_zzz(DisasContext *s, gen_helper_gvec_3_ptr *fn,
+                              int rd, int rn, int rm,
+                              int data, ARMFPStatusFlavour flavour)
+{
+    if (fn == NULL) {
+        return false;
+    }
+    if (sve_access_check(s)) {
+        unsigned vsz = vec_full_reg_size(s);
+        TCGv_ptr status = fpstatus_ptr(flavour);
+
+        tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, rd),
+                           vec_full_reg_offset(s, rn),
+                           vec_full_reg_offset(s, rm),
+                           status, vsz, vsz, data, fn);
+
+        tcg_temp_free_ptr(status);
+    }
+    return true;
+}
+
+static bool gen_gvec_fpst_arg_zzz(DisasContext *s, gen_helper_gvec_3_ptr *fn,
+                                  arg_rrr_esz *a, int data)
+{
+    return gen_gvec_fpst_zzz(s, fn, a->rd, a->rn, a->rm, data,
+                             a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
+}
+
 /* Invoke an out-of-line helper on 4 Zregs. */
 static bool gen_gvec_ool_zzzz(DisasContext *s, gen_helper_gvec_4 *fn,
                               int rd, int rn, int rm, int ra, int data)
@@ -3769,25 +3798,6 @@ static bool trans_FADDA(DisasContext *s, arg_rprr_esz *a)
  *** SVE Floating Point Arithmetic - Unpredicated Group
  */
 
-static bool do_zzz_fp(DisasContext *s, arg_rrr_esz *a,
-                      gen_helper_gvec_3_ptr *fn)
-{
-    if (fn == NULL) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
-        tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           status, vsz, vsz, 0, fn);
-        tcg_temp_free_ptr(status);
-    }
-    return true;
-}
-
-
 #define DO_FP3(NAME, name) \
 static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a)           \
 {                                                                   \
@@ -3795,7 +3805,7 @@ static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a)           \
         NULL, gen_helper_gvec_##name##_h,                           \
         gen_helper_gvec_##name##_s, gen_helper_gvec_##name##_d      \
     };                                                              \
-    return do_zzz_fp(s, a, fns[a->esz]);                            \
+    return gen_gvec_fpst_arg_zzz(s, fns[a->esz], a, 0);             \
 }
 
 DO_FP3(FADD_zzz, fadd)

From bdd4ce0d0e4f1e767bb1fa0462d7a8542e1af04b Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:39 -0700
Subject: [PATCH 671/935] target/arm: Use TRANS_FEAT for DO_FP3

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-87-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 61bf5f5757..d596e7a027 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3799,14 +3799,11 @@ static bool trans_FADDA(DisasContext *s, arg_rprr_esz *a)
  */
 
 #define DO_FP3(NAME, name) \
-static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a)           \
-{                                                                   \
-    static gen_helper_gvec_3_ptr * const fns[4] = {                 \
+    static gen_helper_gvec_3_ptr * const name##_fns[4] = {          \
         NULL, gen_helper_gvec_##name##_h,                           \
         gen_helper_gvec_##name##_s, gen_helper_gvec_##name##_d      \
     };                                                              \
-    return gen_gvec_fpst_arg_zzz(s, fns[a->esz], a, 0);             \
-}
+    TRANS_FEAT(NAME, aa64_sve, gen_gvec_fpst_arg_zzz, name##_fns[a->esz], a, 0)
 
 DO_FP3(FADD_zzz, fadd)
 DO_FP3(FSUB_zzz, fsub)

From 9c99ef66770d03e077535cca64c7ac827de308fb Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:40 -0700
Subject: [PATCH 672/935] target/arm: Use TRANS_FEAT for FMUL_zzx

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-88-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 26 +++++++-------------------
 1 file changed, 7 insertions(+), 19 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index d596e7a027..29fcc8b014 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3557,25 +3557,13 @@ TRANS_FEAT(FMLS_zzxz, aa64_sve, do_FMLA_zzxz, a, true)
  *** SVE Floating Point Multiply Indexed Group
  */
 
-static bool trans_FMUL_zzx(DisasContext *s, arg_FMUL_zzx *a)
-{
-    static gen_helper_gvec_3_ptr * const fns[3] = {
-        gen_helper_gvec_fmul_idx_h,
-        gen_helper_gvec_fmul_idx_s,
-        gen_helper_gvec_fmul_idx_d,
-    };
-
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
-        tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           status, vsz, vsz, a->index, fns[a->esz - 1]);
-        tcg_temp_free_ptr(status);
-    }
-    return true;
-}
+static gen_helper_gvec_3_ptr * const fmul_idx_fns[4] = {
+    NULL,                       gen_helper_gvec_fmul_idx_h,
+    gen_helper_gvec_fmul_idx_s, gen_helper_gvec_fmul_idx_d,
+};
+TRANS_FEAT(FMUL_zzx, aa64_sve, gen_gvec_fpst_zzz,
+           fmul_idx_fns[a->esz], a->rd, a->rn, a->rm, a->index,
+           a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR)
 
 /*
  *** SVE Floating Point Fast Reduction Group

From cdd85923370bc232472c79bdd8f0336f547add1d Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:41 -0700
Subject: [PATCH 673/935] target/arm: Use TRANS_FEAT for FTMAD

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-89-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 29 +++++++----------------------
 1 file changed, 7 insertions(+), 22 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 29fcc8b014..11e4b4e1e4 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3715,28 +3715,13 @@ DO_PPZ(FCMNE_ppz0, fcmne0)
  *** SVE floating-point trig multiply-add coefficient
  */
 
-static bool trans_FTMAD(DisasContext *s, arg_FTMAD *a)
-{
-    static gen_helper_gvec_3_ptr * const fns[3] = {
-        gen_helper_sve_ftmad_h,
-        gen_helper_sve_ftmad_s,
-        gen_helper_sve_ftmad_d,
-    };
-
-    if (a->esz == 0) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
-        tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           status, vsz, vsz, a->imm, fns[a->esz - 1]);
-        tcg_temp_free_ptr(status);
-    }
-    return true;
-}
+static gen_helper_gvec_3_ptr * const ftmad_fns[4] = {
+    NULL,                   gen_helper_sve_ftmad_h,
+    gen_helper_sve_ftmad_s, gen_helper_sve_ftmad_d,
+};
+TRANS_FEAT(FTMAD, aa64_sve, gen_gvec_fpst_zzz,
+           ftmad_fns[a->esz], a->rd, a->rn, a->rm, a->imm,
+           a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR)
 
 /*
  *** SVE Floating Point Accumulating Reduction Group

From 5ce18efe306b9eb1f74074df3eab8dac96a8fe51 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:42 -0700
Subject: [PATCH 674/935] target/arm: Move null function and sve check into
 do_reduce

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-90-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 30 +++++++++++++++++-------------
 1 file changed, 17 insertions(+), 13 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 11e4b4e1e4..0d71072f83 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3572,15 +3572,24 @@ TRANS_FEAT(FMUL_zzx, aa64_sve, gen_gvec_fpst_zzz,
 typedef void gen_helper_fp_reduce(TCGv_i64, TCGv_ptr, TCGv_ptr,
                                   TCGv_ptr, TCGv_i32);
 
-static void do_reduce(DisasContext *s, arg_rpr_esz *a,
+static bool do_reduce(DisasContext *s, arg_rpr_esz *a,
                       gen_helper_fp_reduce *fn)
 {
-    unsigned vsz = vec_full_reg_size(s);
-    unsigned p2vsz = pow2ceil(vsz);
-    TCGv_i32 t_desc = tcg_constant_i32(simd_desc(vsz, vsz, p2vsz));
+    unsigned vsz, p2vsz;
+    TCGv_i32 t_desc;
     TCGv_ptr t_zn, t_pg, status;
     TCGv_i64 temp;
 
+    if (fn == NULL) {
+        return false;
+    }
+    if (!sve_access_check(s)) {
+        return true;
+    }
+
+    vsz = vec_full_reg_size(s);
+    p2vsz = pow2ceil(vsz);
+    t_desc = tcg_constant_i32(simd_desc(vsz, vsz, p2vsz));
     temp = tcg_temp_new_i64();
     t_zn = tcg_temp_new_ptr();
     t_pg = tcg_temp_new_ptr();
@@ -3596,23 +3605,18 @@ static void do_reduce(DisasContext *s, arg_rpr_esz *a,
 
     write_fp_dreg(s, a->rd, temp);
     tcg_temp_free_i64(temp);
+    return true;
 }
 
 #define DO_VPZ(NAME, name) \
 static bool trans_##NAME(DisasContext *s, arg_rpr_esz *a)                \
 {                                                                        \
-    static gen_helper_fp_reduce * const fns[3] = {                       \
-        gen_helper_sve_##name##_h,                                       \
+    static gen_helper_fp_reduce * const fns[4] = {                       \
+        NULL, gen_helper_sve_##name##_h,                                 \
         gen_helper_sve_##name##_s,                                       \
         gen_helper_sve_##name##_d,                                       \
     };                                                                   \
-    if (a->esz == 0) {                                                   \
-        return false;                                                    \
-    }                                                                    \
-    if (sve_access_check(s)) {                                           \
-        do_reduce(s, a, fns[a->esz - 1]);                                \
-    }                                                                    \
-    return true;                                                         \
+    return do_reduce(s, a, fns[a->esz]);                                 \
 }
 
 DO_VPZ(FADDV, faddv)

From 8003e7cf15fa00496fe2c4d0208d88aa6802d4f6 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:43 -0700
Subject: [PATCH 675/935] target/arm: Use TRANS_FEAT for do_reduce

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-91-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 0d71072f83..4a9ecd5e72 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3609,15 +3609,11 @@ static bool do_reduce(DisasContext *s, arg_rpr_esz *a,
 }
 
 #define DO_VPZ(NAME, name) \
-static bool trans_##NAME(DisasContext *s, arg_rpr_esz *a)                \
-{                                                                        \
-    static gen_helper_fp_reduce * const fns[4] = {                       \
-        NULL, gen_helper_sve_##name##_h,                                 \
-        gen_helper_sve_##name##_s,                                       \
-        gen_helper_sve_##name##_d,                                       \
+    static gen_helper_fp_reduce * const name##_fns[4] = {                \
+        NULL,                      gen_helper_sve_##name##_h,            \
+        gen_helper_sve_##name##_s, gen_helper_sve_##name##_d,            \
     };                                                                   \
-    return do_reduce(s, a, fns[a->esz]);                                 \
-}
+    TRANS_FEAT(NAME, aa64_sve, do_reduce, a, name##_fns[a->esz])
 
 DO_VPZ(FADDV, faddv)
 DO_VPZ(FMINNMV, fminnmv)
@@ -3625,6 +3621,8 @@ DO_VPZ(FMAXNMV, fmaxnmv)
 DO_VPZ(FMINV, fminv)
 DO_VPZ(FMAXV, fmaxv)
 
+#undef DO_VPZ
+
 /*
  *** SVE Floating Point Unary Operations - Unpredicated Group
  */

From de58c6b09fcea732edac2a90dc4eb49ab448bf8b Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:44 -0700
Subject: [PATCH 676/935] target/arm: Use TRANS_FEAT for FRECPE, FRSQRTE

Rename do_zz_fp to gen_gvec_fpst_arg_zz, and move up.
Split out gen_gvec_fpst_zz as a helper while we're at it.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-92-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 77 ++++++++++++++++++--------------------
 1 file changed, 36 insertions(+), 41 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 4a9ecd5e72..2f7651249a 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -152,6 +152,32 @@ static bool gen_gvec_ool_zz(DisasContext *s, gen_helper_gvec_2 *fn,
     return true;
 }
 
+static bool gen_gvec_fpst_zz(DisasContext *s, gen_helper_gvec_2_ptr *fn,
+                             int rd, int rn, int data,
+                             ARMFPStatusFlavour flavour)
+{
+    if (fn == NULL) {
+        return false;
+    }
+    if (sve_access_check(s)) {
+        unsigned vsz = vec_full_reg_size(s);
+        TCGv_ptr status = fpstatus_ptr(flavour);
+
+        tcg_gen_gvec_2_ptr(vec_full_reg_offset(s, rd),
+                           vec_full_reg_offset(s, rn),
+                           status, vsz, vsz, data, fn);
+        tcg_temp_free_ptr(status);
+    }
+    return true;
+}
+
+static bool gen_gvec_fpst_arg_zz(DisasContext *s, gen_helper_gvec_2_ptr *fn,
+                                 arg_rr_esz *a, int data)
+{
+    return gen_gvec_fpst_zz(s, fn, a->rd, a->rn, data,
+                            a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
+}
+
 /* Invoke an out-of-line helper on 3 Zregs. */
 static bool gen_gvec_ool_zzz(DisasContext *s, gen_helper_gvec_3 *fn,
                              int rd, int rn, int rm, int data)
@@ -3627,48 +3653,17 @@ DO_VPZ(FMAXV, fmaxv)
  *** SVE Floating Point Unary Operations - Unpredicated Group
  */
 
-static void do_zz_fp(DisasContext *s, arg_rr_esz *a, gen_helper_gvec_2_ptr *fn)
-{
-    unsigned vsz = vec_full_reg_size(s);
-    TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
+static gen_helper_gvec_2_ptr * const frecpe_fns[] = {
+    NULL,                     gen_helper_gvec_frecpe_h,
+    gen_helper_gvec_frecpe_s, gen_helper_gvec_frecpe_d,
+};
+TRANS_FEAT(FRECPE, aa64_sve, gen_gvec_fpst_arg_zz, frecpe_fns[a->esz], a, 0)
 
-    tcg_gen_gvec_2_ptr(vec_full_reg_offset(s, a->rd),
-                       vec_full_reg_offset(s, a->rn),
-                       status, vsz, vsz, 0, fn);
-    tcg_temp_free_ptr(status);
-}
-
-static bool trans_FRECPE(DisasContext *s, arg_rr_esz *a)
-{
-    static gen_helper_gvec_2_ptr * const fns[3] = {
-        gen_helper_gvec_frecpe_h,
-        gen_helper_gvec_frecpe_s,
-        gen_helper_gvec_frecpe_d,
-    };
-    if (a->esz == 0) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        do_zz_fp(s, a, fns[a->esz - 1]);
-    }
-    return true;
-}
-
-static bool trans_FRSQRTE(DisasContext *s, arg_rr_esz *a)
-{
-    static gen_helper_gvec_2_ptr * const fns[3] = {
-        gen_helper_gvec_frsqrte_h,
-        gen_helper_gvec_frsqrte_s,
-        gen_helper_gvec_frsqrte_d,
-    };
-    if (a->esz == 0) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        do_zz_fp(s, a, fns[a->esz - 1]);
-    }
-    return true;
-}
+static gen_helper_gvec_2_ptr * const frsqrte_fns[] = {
+    NULL,                      gen_helper_gvec_frsqrte_h,
+    gen_helper_gvec_frsqrte_s, gen_helper_gvec_frsqrte_d,
+};
+TRANS_FEAT(FRSQRTE, aa64_sve, gen_gvec_fpst_arg_zz, frsqrte_fns[a->esz], a, 0)
 
 /*
  *** SVE Floating Point Compare with Zero Group

From ed6bb6b4d2817e5edc04a6214912f55b5e54c2c8 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:45 -0700
Subject: [PATCH 677/935] target/arm: Expand frint_fns for MO_8

Simplify indexing of this array.  This will allow folding
of the illegal esz == 0 into the normal fn == NULL check.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-93-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 2f7651249a..99e5d89645 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -4167,7 +4167,8 @@ static bool trans_FCVTZU_dd(DisasContext *s, arg_rpr_esz *a)
     return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzu_dd);
 }
 
-static gen_helper_gvec_3_ptr * const frint_fns[3] = {
+static gen_helper_gvec_3_ptr * const frint_fns[] = {
+    NULL,
     gen_helper_sve_frint_h,
     gen_helper_sve_frint_s,
     gen_helper_sve_frint_d
@@ -4179,7 +4180,7 @@ static bool trans_FRINTI(DisasContext *s, arg_rpr_esz *a)
         return false;
     }
     return do_zpz_ptr(s, a->rd, a->rn, a->pg, a->esz == MO_16,
-                      frint_fns[a->esz - 1]);
+                      frint_fns[a->esz]);
 }
 
 static bool trans_FRINTX(DisasContext *s, arg_rpr_esz *a)
@@ -4222,7 +4223,7 @@ static bool trans_FRINTN(DisasContext *s, arg_rpr_esz *a)
     if (a->esz == 0) {
         return false;
     }
-    return do_frint_mode(s, a, float_round_nearest_even, frint_fns[a->esz - 1]);
+    return do_frint_mode(s, a, float_round_nearest_even, frint_fns[a->esz]);
 }
 
 static bool trans_FRINTP(DisasContext *s, arg_rpr_esz *a)
@@ -4230,7 +4231,7 @@ static bool trans_FRINTP(DisasContext *s, arg_rpr_esz *a)
     if (a->esz == 0) {
         return false;
     }
-    return do_frint_mode(s, a, float_round_up, frint_fns[a->esz - 1]);
+    return do_frint_mode(s, a, float_round_up, frint_fns[a->esz]);
 }
 
 static bool trans_FRINTM(DisasContext *s, arg_rpr_esz *a)
@@ -4238,7 +4239,7 @@ static bool trans_FRINTM(DisasContext *s, arg_rpr_esz *a)
     if (a->esz == 0) {
         return false;
     }
-    return do_frint_mode(s, a, float_round_down, frint_fns[a->esz - 1]);
+    return do_frint_mode(s, a, float_round_down, frint_fns[a->esz]);
 }
 
 static bool trans_FRINTZ(DisasContext *s, arg_rpr_esz *a)
@@ -4246,7 +4247,7 @@ static bool trans_FRINTZ(DisasContext *s, arg_rpr_esz *a)
     if (a->esz == 0) {
         return false;
     }
-    return do_frint_mode(s, a, float_round_to_zero, frint_fns[a->esz - 1]);
+    return do_frint_mode(s, a, float_round_to_zero, frint_fns[a->esz]);
 }
 
 static bool trans_FRINTA(DisasContext *s, arg_rpr_esz *a)
@@ -4254,7 +4255,7 @@ static bool trans_FRINTA(DisasContext *s, arg_rpr_esz *a)
     if (a->esz == 0) {
         return false;
     }
-    return do_frint_mode(s, a, float_round_ties_away, frint_fns[a->esz - 1]);
+    return do_frint_mode(s, a, float_round_ties_away, frint_fns[a->esz]);
 }
 
 static bool trans_FRECPX(DisasContext *s, arg_rpr_esz *a)

From 0360730c467397c32be9b1bc4dcb31b62ab7baac Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:46 -0700
Subject: [PATCH 678/935] target/arm: Rename do_zpz_ptr to
 gen_gvec_ool_fpst_arg_zpz

Rename the function to match other expansion function and
move to be adjacent.  Split out gen_gvec_fpst_zzp as a
helper while we're at it.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-94-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 392 ++++++++++++-------------------------
 1 file changed, 129 insertions(+), 263 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 99e5d89645..2a5fbec2d6 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -318,6 +318,33 @@ static bool gen_gvec_ool_arg_zpzi(DisasContext *s, gen_helper_gvec_3 *fn,
     return gen_gvec_ool_zzp(s, fn, a->rd, a->rn, a->pg, a->imm);
 }
 
+static bool gen_gvec_fpst_zzp(DisasContext *s, gen_helper_gvec_3_ptr *fn,
+                              int rd, int rn, int pg, int data,
+                              ARMFPStatusFlavour flavour)
+{
+    if (fn == NULL) {
+        return false;
+    }
+    if (sve_access_check(s)) {
+        unsigned vsz = vec_full_reg_size(s);
+        TCGv_ptr status = fpstatus_ptr(flavour);
+
+        tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, rd),
+                           vec_full_reg_offset(s, rn),
+                           pred_full_reg_offset(s, pg),
+                           status, vsz, vsz, data, fn);
+        tcg_temp_free_ptr(status);
+    }
+    return true;
+}
+
+static bool gen_gvec_fpst_arg_zpz(DisasContext *s, gen_helper_gvec_3_ptr *fn,
+                                  arg_rpr_esz *a, int data,
+                                  ARMFPStatusFlavour flavour)
+{
+    return gen_gvec_fpst_zzp(s, fn, a->rd, a->rn, a->pg, data, flavour);
+}
+
 /* Invoke an out-of-line helper on 3 Zregs and a predicate. */
 static bool gen_gvec_ool_zzzp(DisasContext *s, gen_helper_gvec_4 *fn,
                               int rd, int rn, int rm, int pg, int data)
@@ -4044,128 +4071,53 @@ static bool trans_FCMLA_zzxz(DisasContext *s, arg_FCMLA_zzxz *a)
  *** SVE Floating Point Unary Operations Predicated Group
  */
 
-static bool do_zpz_ptr(DisasContext *s, int rd, int rn, int pg,
-                       bool is_fp16, gen_helper_gvec_3_ptr *fn)
-{
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        TCGv_ptr status = fpstatus_ptr(is_fp16 ? FPST_FPCR_F16 : FPST_FPCR);
-        tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, rd),
-                           vec_full_reg_offset(s, rn),
-                           pred_full_reg_offset(s, pg),
-                           status, vsz, vsz, 0, fn);
-        tcg_temp_free_ptr(status);
-    }
-    return true;
-}
+TRANS_FEAT(FCVT_sh, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvt_sh, a, 0, FPST_FPCR)
+TRANS_FEAT(FCVT_hs, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvt_hs, a, 0, FPST_FPCR)
 
-static bool trans_FCVT_sh(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvt_sh);
-}
+TRANS_FEAT(BFCVT, aa64_sve_bf16, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_bfcvt, a, 0, FPST_FPCR)
 
-static bool trans_FCVT_hs(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvt_hs);
-}
+TRANS_FEAT(FCVT_dh, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvt_dh, a, 0, FPST_FPCR)
+TRANS_FEAT(FCVT_hd, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvt_hd, a, 0, FPST_FPCR)
+TRANS_FEAT(FCVT_ds, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvt_ds, a, 0, FPST_FPCR)
+TRANS_FEAT(FCVT_sd, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvt_sd, a, 0, FPST_FPCR)
 
-static bool trans_BFCVT(DisasContext *s, arg_rpr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve_bf16, s)) {
-        return false;
-    }
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_bfcvt);
-}
+TRANS_FEAT(FCVTZS_hh, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvtzs_hh, a, 0, FPST_FPCR_F16)
+TRANS_FEAT(FCVTZU_hh, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvtzu_hh, a, 0, FPST_FPCR_F16)
+TRANS_FEAT(FCVTZS_hs, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvtzs_hs, a, 0, FPST_FPCR_F16)
+TRANS_FEAT(FCVTZU_hs, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvtzu_hs, a, 0, FPST_FPCR_F16)
+TRANS_FEAT(FCVTZS_hd, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvtzs_hd, a, 0, FPST_FPCR_F16)
+TRANS_FEAT(FCVTZU_hd, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvtzu_hd, a, 0, FPST_FPCR_F16)
 
-static bool trans_FCVT_dh(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvt_dh);
-}
+TRANS_FEAT(FCVTZS_ss, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvtzs_ss, a, 0, FPST_FPCR)
+TRANS_FEAT(FCVTZU_ss, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvtzu_ss, a, 0, FPST_FPCR)
+TRANS_FEAT(FCVTZS_sd, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvtzs_sd, a, 0, FPST_FPCR)
+TRANS_FEAT(FCVTZU_sd, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvtzu_sd, a, 0, FPST_FPCR)
+TRANS_FEAT(FCVTZS_ds, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvtzs_ds, a, 0, FPST_FPCR)
+TRANS_FEAT(FCVTZU_ds, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvtzu_ds, a, 0, FPST_FPCR)
 
-static bool trans_FCVT_hd(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvt_hd);
-}
-
-static bool trans_FCVT_ds(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvt_ds);
-}
-
-static bool trans_FCVT_sd(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvt_sd);
-}
-
-static bool trans_FCVTZS_hh(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_fcvtzs_hh);
-}
-
-static bool trans_FCVTZU_hh(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_fcvtzu_hh);
-}
-
-static bool trans_FCVTZS_hs(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_fcvtzs_hs);
-}
-
-static bool trans_FCVTZU_hs(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_fcvtzu_hs);
-}
-
-static bool trans_FCVTZS_hd(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_fcvtzs_hd);
-}
-
-static bool trans_FCVTZU_hd(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_fcvtzu_hd);
-}
-
-static bool trans_FCVTZS_ss(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzs_ss);
-}
-
-static bool trans_FCVTZU_ss(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzu_ss);
-}
-
-static bool trans_FCVTZS_sd(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzs_sd);
-}
-
-static bool trans_FCVTZU_sd(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzu_sd);
-}
-
-static bool trans_FCVTZS_ds(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzs_ds);
-}
-
-static bool trans_FCVTZU_ds(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzu_ds);
-}
-
-static bool trans_FCVTZS_dd(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzs_dd);
-}
-
-static bool trans_FCVTZU_dd(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_fcvtzu_dd);
-}
+TRANS_FEAT(FCVTZS_dd, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvtzs_dd, a, 0, FPST_FPCR)
+TRANS_FEAT(FCVTZU_dd, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_fcvtzu_dd, a, 0, FPST_FPCR)
 
 static gen_helper_gvec_3_ptr * const frint_fns[] = {
     NULL,
@@ -4173,28 +4125,17 @@ static gen_helper_gvec_3_ptr * const frint_fns[] = {
     gen_helper_sve_frint_s,
     gen_helper_sve_frint_d
 };
+TRANS_FEAT(FRINTI, aa64_sve, gen_gvec_fpst_arg_zpz, frint_fns[a->esz],
+           a, 0, a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR)
 
-static bool trans_FRINTI(DisasContext *s, arg_rpr_esz *a)
-{
-    if (a->esz == 0) {
-        return false;
-    }
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, a->esz == MO_16,
-                      frint_fns[a->esz]);
-}
-
-static bool trans_FRINTX(DisasContext *s, arg_rpr_esz *a)
-{
-    static gen_helper_gvec_3_ptr * const fns[3] = {
-        gen_helper_sve_frintx_h,
-        gen_helper_sve_frintx_s,
-        gen_helper_sve_frintx_d
-    };
-    if (a->esz == 0) {
-        return false;
-    }
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, a->esz == MO_16, fns[a->esz - 1]);
-}
+static gen_helper_gvec_3_ptr * const frintx_fns[] = {
+    NULL,
+    gen_helper_sve_frintx_h,
+    gen_helper_sve_frintx_s,
+    gen_helper_sve_frintx_d
+};
+TRANS_FEAT(FRINTX, aa64_sve, gen_gvec_fpst_arg_zpz, frintx_fns[a->esz],
+           a, 0, a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
 
 static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a,
                           int mode, gen_helper_gvec_3_ptr *fn)
@@ -4258,101 +4199,53 @@ static bool trans_FRINTA(DisasContext *s, arg_rpr_esz *a)
     return do_frint_mode(s, a, float_round_ties_away, frint_fns[a->esz]);
 }
 
-static bool trans_FRECPX(DisasContext *s, arg_rpr_esz *a)
-{
-    static gen_helper_gvec_3_ptr * const fns[3] = {
-        gen_helper_sve_frecpx_h,
-        gen_helper_sve_frecpx_s,
-        gen_helper_sve_frecpx_d
-    };
-    if (a->esz == 0) {
-        return false;
-    }
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, a->esz == MO_16, fns[a->esz - 1]);
-}
+static gen_helper_gvec_3_ptr * const frecpx_fns[] = {
+    NULL,                    gen_helper_sve_frecpx_h,
+    gen_helper_sve_frecpx_s, gen_helper_sve_frecpx_d,
+};
+TRANS_FEAT(FRECPX, aa64_sve, gen_gvec_fpst_arg_zpz, frecpx_fns[a->esz],
+           a, 0, a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR)
 
-static bool trans_FSQRT(DisasContext *s, arg_rpr_esz *a)
-{
-    static gen_helper_gvec_3_ptr * const fns[3] = {
-        gen_helper_sve_fsqrt_h,
-        gen_helper_sve_fsqrt_s,
-        gen_helper_sve_fsqrt_d
-    };
-    if (a->esz == 0) {
-        return false;
-    }
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, a->esz == MO_16, fns[a->esz - 1]);
-}
+static gen_helper_gvec_3_ptr * const fsqrt_fns[] = {
+    NULL,                   gen_helper_sve_fsqrt_h,
+    gen_helper_sve_fsqrt_s, gen_helper_sve_fsqrt_d,
+};
+TRANS_FEAT(FSQRT, aa64_sve, gen_gvec_fpst_arg_zpz, fsqrt_fns[a->esz],
+           a, 0, a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR)
 
-static bool trans_SCVTF_hh(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_scvt_hh);
-}
+TRANS_FEAT(SCVTF_hh, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_scvt_hh, a, 0, FPST_FPCR_F16)
+TRANS_FEAT(SCVTF_sh, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_scvt_sh, a, 0, FPST_FPCR_F16)
+TRANS_FEAT(SCVTF_dh, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_scvt_dh, a, 0, FPST_FPCR_F16)
 
-static bool trans_SCVTF_sh(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_scvt_sh);
-}
+TRANS_FEAT(SCVTF_ss, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_scvt_ss, a, 0, FPST_FPCR)
+TRANS_FEAT(SCVTF_ds, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_scvt_ds, a, 0, FPST_FPCR)
 
-static bool trans_SCVTF_dh(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_scvt_dh);
-}
+TRANS_FEAT(SCVTF_sd, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_scvt_sd, a, 0, FPST_FPCR)
+TRANS_FEAT(SCVTF_dd, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_scvt_dd, a, 0, FPST_FPCR)
 
-static bool trans_SCVTF_ss(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_scvt_ss);
-}
+TRANS_FEAT(UCVTF_hh, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_ucvt_hh, a, 0, FPST_FPCR_F16)
+TRANS_FEAT(UCVTF_sh, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_ucvt_sh, a, 0, FPST_FPCR_F16)
+TRANS_FEAT(UCVTF_dh, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_ucvt_dh, a, 0, FPST_FPCR_F16)
 
-static bool trans_SCVTF_ds(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_scvt_ds);
-}
+TRANS_FEAT(UCVTF_ss, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_ucvt_ss, a, 0, FPST_FPCR)
+TRANS_FEAT(UCVTF_ds, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_ucvt_ds, a, 0, FPST_FPCR)
+TRANS_FEAT(UCVTF_sd, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_ucvt_sd, a, 0, FPST_FPCR)
 
-static bool trans_SCVTF_sd(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_scvt_sd);
-}
-
-static bool trans_SCVTF_dd(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_scvt_dd);
-}
-
-static bool trans_UCVTF_hh(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_ucvt_hh);
-}
-
-static bool trans_UCVTF_sh(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_ucvt_sh);
-}
-
-static bool trans_UCVTF_dh(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, true, gen_helper_sve_ucvt_dh);
-}
-
-static bool trans_UCVTF_ss(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_ucvt_ss);
-}
-
-static bool trans_UCVTF_ds(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_ucvt_ds);
-}
-
-static bool trans_UCVTF_sd(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_ucvt_sd);
-}
-
-static bool trans_UCVTF_dd(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_ucvt_dd);
-}
+TRANS_FEAT(UCVTF_dd, aa64_sve, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_ucvt_dd, a, 0, FPST_FPCR)
 
 /*
  *** SVE Memory - 32-bit Gather and Unsized Contiguous Group
@@ -7389,45 +7282,18 @@ TRANS_FEAT(SM4EKEY, aa64_sve2_sm4, gen_gvec_ool_arg_zzz,
 
 TRANS_FEAT(RAX1, aa64_sve2_sha3, gen_gvec_fn_arg_zzz, gen_gvec_rax1, a)
 
-static bool trans_FCVTNT_sh(DisasContext *s, arg_rpr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtnt_sh);
-}
+TRANS_FEAT(FCVTNT_sh, aa64_sve2, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve2_fcvtnt_sh, a, 0, FPST_FPCR)
+TRANS_FEAT(FCVTNT_ds, aa64_sve2, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve2_fcvtnt_ds, a, 0, FPST_FPCR)
 
-static bool trans_BFCVTNT(DisasContext *s, arg_rpr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve_bf16, s)) {
-        return false;
-    }
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve_bfcvtnt);
-}
+TRANS_FEAT(BFCVTNT, aa64_sve_bf16, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve_bfcvtnt, a, 0, FPST_FPCR)
 
-static bool trans_FCVTNT_ds(DisasContext *s, arg_rpr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtnt_ds);
-}
-
-static bool trans_FCVTLT_hs(DisasContext *s, arg_rpr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtlt_hs);
-}
-
-static bool trans_FCVTLT_sd(DisasContext *s, arg_rpr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtlt_sd);
-}
+TRANS_FEAT(FCVTLT_hs, aa64_sve2, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve2_fcvtlt_hs, a, 0, FPST_FPCR)
+TRANS_FEAT(FCVTLT_sd, aa64_sve2, gen_gvec_fpst_arg_zpz,
+           gen_helper_sve2_fcvtlt_sd, a, 0, FPST_FPCR)
 
 static bool trans_FCVTX_ds(DisasContext *s, arg_rpr_esz *a)
 {

From 13c0dd173952b22a8705017e234373f2ba3b0d25 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:47 -0700
Subject: [PATCH 679/935] target/arm: Move null function and sve check into
 do_frint_mode

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-95-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 52 +++++++++++++++++---------------------
 1 file changed, 23 insertions(+), 29 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 2a5fbec2d6..43cfd2818e 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -4140,62 +4140,56 @@ TRANS_FEAT(FRINTX, aa64_sve, gen_gvec_fpst_arg_zpz, frintx_fns[a->esz],
 static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a,
                           int mode, gen_helper_gvec_3_ptr *fn)
 {
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        TCGv_i32 tmode = tcg_const_i32(mode);
-        TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
+    unsigned vsz;
+    TCGv_i32 tmode;
+    TCGv_ptr status;
 
-        gen_helper_set_rmode(tmode, tmode, status);
-
-        tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           pred_full_reg_offset(s, a->pg),
-                           status, vsz, vsz, 0, fn);
-
-        gen_helper_set_rmode(tmode, tmode, status);
-        tcg_temp_free_i32(tmode);
-        tcg_temp_free_ptr(status);
+    if (fn == NULL) {
+        return false;
     }
+    if (!sve_access_check(s)) {
+        return true;
+    }
+
+    vsz = vec_full_reg_size(s);
+    tmode = tcg_const_i32(mode);
+    status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
+
+    gen_helper_set_rmode(tmode, tmode, status);
+
+    tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd),
+                       vec_full_reg_offset(s, a->rn),
+                       pred_full_reg_offset(s, a->pg),
+                       status, vsz, vsz, 0, fn);
+
+    gen_helper_set_rmode(tmode, tmode, status);
+    tcg_temp_free_i32(tmode);
+    tcg_temp_free_ptr(status);
     return true;
 }
 
 static bool trans_FRINTN(DisasContext *s, arg_rpr_esz *a)
 {
-    if (a->esz == 0) {
-        return false;
-    }
     return do_frint_mode(s, a, float_round_nearest_even, frint_fns[a->esz]);
 }
 
 static bool trans_FRINTP(DisasContext *s, arg_rpr_esz *a)
 {
-    if (a->esz == 0) {
-        return false;
-    }
     return do_frint_mode(s, a, float_round_up, frint_fns[a->esz]);
 }
 
 static bool trans_FRINTM(DisasContext *s, arg_rpr_esz *a)
 {
-    if (a->esz == 0) {
-        return false;
-    }
     return do_frint_mode(s, a, float_round_down, frint_fns[a->esz]);
 }
 
 static bool trans_FRINTZ(DisasContext *s, arg_rpr_esz *a)
 {
-    if (a->esz == 0) {
-        return false;
-    }
     return do_frint_mode(s, a, float_round_to_zero, frint_fns[a->esz]);
 }
 
 static bool trans_FRINTA(DisasContext *s, arg_rpr_esz *a)
 {
-    if (a->esz == 0) {
-        return false;
-    }
     return do_frint_mode(s, a, float_round_ties_away, frint_fns[a->esz]);
 }
 

From 27645836952081775138f8c9a118de63e327f7fb Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:48 -0700
Subject: [PATCH 680/935] target/arm: Use TRANS_FEAT for do_frint_mode

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-96-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 53 ++++++++++----------------------------
 1 file changed, 14 insertions(+), 39 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 43cfd2818e..552a551fef 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -4168,30 +4168,16 @@ static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a,
     return true;
 }
 
-static bool trans_FRINTN(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_frint_mode(s, a, float_round_nearest_even, frint_fns[a->esz]);
-}
-
-static bool trans_FRINTP(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_frint_mode(s, a, float_round_up, frint_fns[a->esz]);
-}
-
-static bool trans_FRINTM(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_frint_mode(s, a, float_round_down, frint_fns[a->esz]);
-}
-
-static bool trans_FRINTZ(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_frint_mode(s, a, float_round_to_zero, frint_fns[a->esz]);
-}
-
-static bool trans_FRINTA(DisasContext *s, arg_rpr_esz *a)
-{
-    return do_frint_mode(s, a, float_round_ties_away, frint_fns[a->esz]);
-}
+TRANS_FEAT(FRINTN, aa64_sve, do_frint_mode, a,
+           float_round_nearest_even, frint_fns[a->esz])
+TRANS_FEAT(FRINTP, aa64_sve, do_frint_mode, a,
+           float_round_up, frint_fns[a->esz])
+TRANS_FEAT(FRINTM, aa64_sve, do_frint_mode, a,
+           float_round_down, frint_fns[a->esz])
+TRANS_FEAT(FRINTZ, aa64_sve, do_frint_mode, a,
+           float_round_to_zero, frint_fns[a->esz])
+TRANS_FEAT(FRINTA, aa64_sve, do_frint_mode, a,
+           float_round_ties_away, frint_fns[a->esz])
 
 static gen_helper_gvec_3_ptr * const frecpx_fns[] = {
     NULL,                    gen_helper_sve_frecpx_h,
@@ -7289,21 +7275,10 @@ TRANS_FEAT(FCVTLT_hs, aa64_sve2, gen_gvec_fpst_arg_zpz,
 TRANS_FEAT(FCVTLT_sd, aa64_sve2, gen_gvec_fpst_arg_zpz,
            gen_helper_sve2_fcvtlt_sd, a, 0, FPST_FPCR)
 
-static bool trans_FCVTX_ds(DisasContext *s, arg_rpr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return do_frint_mode(s, a, float_round_to_odd, gen_helper_sve_fcvt_ds);
-}
-
-static bool trans_FCVTXNT_ds(DisasContext *s, arg_rpr_esz *a)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return do_frint_mode(s, a, float_round_to_odd, gen_helper_sve2_fcvtnt_ds);
-}
+TRANS_FEAT(FCVTX_ds, aa64_sve2, do_frint_mode, a,
+           float_round_to_odd, gen_helper_sve_fcvt_ds)
+TRANS_FEAT(FCVTXNT_ds, aa64_sve2, do_frint_mode, a,
+           float_round_to_odd, gen_helper_sve2_fcvtnt_ds)
 
 static bool trans_FLOGB(DisasContext *s, arg_rpr_esz *a)
 {

From 7b9dfcfec889ba10cda7f2426de253f73226f129 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:49 -0700
Subject: [PATCH 681/935] target/arm: Use TRANS_FEAT for FLOGB

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-97-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 29 ++++++-----------------------
 1 file changed, 6 insertions(+), 23 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 552a551fef..2f96f52293 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -7280,29 +7280,12 @@ TRANS_FEAT(FCVTX_ds, aa64_sve2, do_frint_mode, a,
 TRANS_FEAT(FCVTXNT_ds, aa64_sve2, do_frint_mode, a,
            float_round_to_odd, gen_helper_sve2_fcvtnt_ds)
 
-static bool trans_FLOGB(DisasContext *s, arg_rpr_esz *a)
-{
-    static gen_helper_gvec_3_ptr * const fns[] = {
-        NULL,               gen_helper_flogb_h,
-        gen_helper_flogb_s, gen_helper_flogb_d
-    };
-
-    if (!dc_isar_feature(aa64_sve2, s) || fns[a->esz] == NULL) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        TCGv_ptr status =
-            fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
-        unsigned vsz = vec_full_reg_size(s);
-
-        tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           pred_full_reg_offset(s, a->pg),
-                           status, vsz, vsz, 0, fns[a->esz]);
-        tcg_temp_free_ptr(status);
-    }
-    return true;
-}
+static gen_helper_gvec_3_ptr * const flogb_fns[] = {
+    NULL,               gen_helper_flogb_h,
+    gen_helper_flogb_s, gen_helper_flogb_d
+};
+TRANS_FEAT(FLOGB, aa64_sve2, gen_gvec_fpst_arg_zpz, flogb_fns[a->esz],
+           a, 0, a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR)
 
 static bool do_FMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sub, bool sel)
 {

From 63d6aef86c137765ce3d0b884cf2d54a14bbaa3b Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:50 -0700
Subject: [PATCH 682/935] target/arm: Use TRANS_FEAT for do_ppz_fp

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-98-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 41 ++++++++++++++++++--------------------
 1 file changed, 19 insertions(+), 22 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 2f96f52293..2ee48186ba 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3696,35 +3696,32 @@ TRANS_FEAT(FRSQRTE, aa64_sve, gen_gvec_fpst_arg_zz, frsqrte_fns[a->esz], a, 0)
  *** SVE Floating Point Compare with Zero Group
  */
 
-static void do_ppz_fp(DisasContext *s, arg_rpr_esz *a,
+static bool do_ppz_fp(DisasContext *s, arg_rpr_esz *a,
                       gen_helper_gvec_3_ptr *fn)
 {
-    unsigned vsz = vec_full_reg_size(s);
-    TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
+    if (fn == NULL) {
+        return false;
+    }
+    if (sve_access_check(s)) {
+        unsigned vsz = vec_full_reg_size(s);
+        TCGv_ptr status =
+            fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
 
-    tcg_gen_gvec_3_ptr(pred_full_reg_offset(s, a->rd),
-                       vec_full_reg_offset(s, a->rn),
-                       pred_full_reg_offset(s, a->pg),
-                       status, vsz, vsz, 0, fn);
-    tcg_temp_free_ptr(status);
+        tcg_gen_gvec_3_ptr(pred_full_reg_offset(s, a->rd),
+                           vec_full_reg_offset(s, a->rn),
+                           pred_full_reg_offset(s, a->pg),
+                           status, vsz, vsz, 0, fn);
+        tcg_temp_free_ptr(status);
+    }
+    return true;
 }
 
 #define DO_PPZ(NAME, name) \
-static bool trans_##NAME(DisasContext *s, arg_rpr_esz *a)         \
-{                                                                 \
-    static gen_helper_gvec_3_ptr * const fns[3] = {               \
-        gen_helper_sve_##name##_h,                                \
-        gen_helper_sve_##name##_s,                                \
-        gen_helper_sve_##name##_d,                                \
+    static gen_helper_gvec_3_ptr * const name##_fns[] = {         \
+        NULL,                      gen_helper_sve_##name##_h,     \
+        gen_helper_sve_##name##_s, gen_helper_sve_##name##_d,     \
     };                                                            \
-    if (a->esz == 0) {                                            \
-        return false;                                             \
-    }                                                             \
-    if (sve_access_check(s)) {                                    \
-        do_ppz_fp(s, a, fns[a->esz - 1]);                         \
-    }                                                             \
-    return true;                                                  \
-}
+    TRANS_FEAT(NAME, aa64_sve, do_ppz_fp, a, name##_fns[a->esz])
 
 DO_PPZ(FCMGE_ppz0, fcmge0)
 DO_PPZ(FCMGT_ppz0, fcmgt0)

From 7e2d07ff87f43e892335fad7fccae6497afa6ebc Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:51 -0700
Subject: [PATCH 683/935] target/arm: Rename do_zpzz_ptr to
 gen_gvec_fpst_arg_zpzz

Rename the function to match other expansion functions and
move to be adjacent.  Split out gen_gvec_fpst_zzzp as a
helper while we're at it.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-99-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 52 +++++++++++++++++++++++---------------
 1 file changed, 31 insertions(+), 21 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 2ee48186ba..f0f2db351e 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -369,6 +369,35 @@ static bool gen_gvec_ool_arg_zpzz(DisasContext *s, gen_helper_gvec_4 *fn,
     return gen_gvec_ool_zzzp(s, fn, a->rd, a->rn, a->rm, a->pg, data);
 }
 
+/* Invoke an out-of-line helper on 3 Zregs and a predicate. */
+static bool gen_gvec_fpst_zzzp(DisasContext *s, gen_helper_gvec_4_ptr *fn,
+                               int rd, int rn, int rm, int pg, int data,
+                               ARMFPStatusFlavour flavour)
+{
+    if (fn == NULL) {
+        return false;
+    }
+    if (sve_access_check(s)) {
+        unsigned vsz = vec_full_reg_size(s);
+        TCGv_ptr status = fpstatus_ptr(flavour);
+
+        tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, rd),
+                           vec_full_reg_offset(s, rn),
+                           vec_full_reg_offset(s, rm),
+                           pred_full_reg_offset(s, pg),
+                           status, vsz, vsz, data, fn);
+        tcg_temp_free_ptr(status);
+    }
+    return true;
+}
+
+static bool gen_gvec_fpst_arg_zpzz(DisasContext *s, gen_helper_gvec_4_ptr *fn,
+                                   arg_rprr_esz *a)
+{
+    return gen_gvec_fpst_zzzp(s, fn, a->rd, a->rn, a->rm, a->pg, 0,
+                              a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
+}
+
 /* Invoke a vector expander on two Zregs and an immediate.  */
 static bool gen_gvec_fn_zzi(DisasContext *s, GVecGen2iFn *gvec_fn,
                             int esz, int rd, int rn, uint64_t imm)
@@ -3812,25 +3841,6 @@ DO_FP3(FRSQRTS, rsqrts)
  *** SVE Floating Point Arithmetic - Predicated Group
  */
 
-static bool do_zpzz_fp(DisasContext *s, arg_rprr_esz *a,
-                       gen_helper_gvec_4_ptr *fn)
-{
-    if (fn == NULL) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
-        tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           pred_full_reg_offset(s, a->pg),
-                           status, vsz, vsz, 0, fn);
-        tcg_temp_free_ptr(status);
-    }
-    return true;
-}
-
 #define DO_FP3(NAME, name) \
 static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a)          \
 {                                                                   \
@@ -3838,7 +3848,7 @@ static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a)          \
         NULL, gen_helper_sve_##name##_h,                            \
         gen_helper_sve_##name##_s, gen_helper_sve_##name##_d        \
     };                                                              \
-    return do_zpzz_fp(s, a, fns[a->esz]);                           \
+    return gen_gvec_fpst_arg_zpzz(s, fns[a->esz], a);               \
 }
 
 DO_FP3(FADD_zpzz, fadd)
@@ -7121,7 +7131,7 @@ static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a,
     if (!dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
-    return do_zpzz_fp(s, a, fn);
+    return gen_gvec_fpst_arg_zpzz(s, fn, a);
 }
 
 #define DO_SVE2_ZPZZ_FP(NAME, name)                                         \

From 7de2617b7ae54536f291dc8df1b5a50863badae0 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:52 -0700
Subject: [PATCH 684/935] target/arm: Use TRANS_FEAT for gen_gvec_fpst_arg_zpzz

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-100-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 68 ++++++++++++--------------------------
 1 file changed, 22 insertions(+), 46 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index f0f2db351e..1108494919 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3841,29 +3841,24 @@ DO_FP3(FRSQRTS, rsqrts)
  *** SVE Floating Point Arithmetic - Predicated Group
  */
 
-#define DO_FP3(NAME, name) \
-static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a)          \
-{                                                                   \
-    static gen_helper_gvec_4_ptr * const fns[4] = {                 \
-        NULL, gen_helper_sve_##name##_h,                            \
-        gen_helper_sve_##name##_s, gen_helper_sve_##name##_d        \
-    };                                                              \
-    return gen_gvec_fpst_arg_zpzz(s, fns[a->esz], a);               \
-}
+#define DO_ZPZZ_FP(NAME, FEAT, name) \
+    static gen_helper_gvec_4_ptr * const name##_zpzz_fns[4] = { \
+        NULL,                  gen_helper_##name##_h,           \
+        gen_helper_##name##_s, gen_helper_##name##_d            \
+    };                                                          \
+    TRANS_FEAT(NAME, FEAT, gen_gvec_fpst_arg_zpzz, name##_zpzz_fns[a->esz], a)
 
-DO_FP3(FADD_zpzz, fadd)
-DO_FP3(FSUB_zpzz, fsub)
-DO_FP3(FMUL_zpzz, fmul)
-DO_FP3(FMIN_zpzz, fmin)
-DO_FP3(FMAX_zpzz, fmax)
-DO_FP3(FMINNM_zpzz, fminnum)
-DO_FP3(FMAXNM_zpzz, fmaxnum)
-DO_FP3(FABD, fabd)
-DO_FP3(FSCALE, fscalbn)
-DO_FP3(FDIV, fdiv)
-DO_FP3(FMULX, fmulx)
-
-#undef DO_FP3
+DO_ZPZZ_FP(FADD_zpzz, aa64_sve, sve_fadd)
+DO_ZPZZ_FP(FSUB_zpzz, aa64_sve, sve_fsub)
+DO_ZPZZ_FP(FMUL_zpzz, aa64_sve, sve_fmul)
+DO_ZPZZ_FP(FMIN_zpzz, aa64_sve, sve_fmin)
+DO_ZPZZ_FP(FMAX_zpzz, aa64_sve, sve_fmax)
+DO_ZPZZ_FP(FMINNM_zpzz, aa64_sve, sve_fminnum)
+DO_ZPZZ_FP(FMAXNM_zpzz, aa64_sve, sve_fmaxnum)
+DO_ZPZZ_FP(FABD, aa64_sve, sve_fabd)
+DO_ZPZZ_FP(FSCALE, aa64_sve, sve_fscalbn)
+DO_ZPZZ_FP(FDIV, aa64_sve, sve_fdiv)
+DO_ZPZZ_FP(FMULX, aa64_sve, sve_fmulx)
 
 typedef void gen_helper_sve_fp2scalar(TCGv_ptr, TCGv_ptr, TCGv_ptr,
                                       TCGv_i64, TCGv_ptr, TCGv_i32);
@@ -7125,30 +7120,11 @@ TRANS_FEAT(HISTCNT, aa64_sve2, gen_gvec_ool_arg_zpzz,
 TRANS_FEAT(HISTSEG, aa64_sve2, gen_gvec_ool_arg_zzz,
            a->esz == 0 ? gen_helper_sve2_histseg : NULL, a, 0)
 
-static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a,
-                            gen_helper_gvec_4_ptr *fn)
-{
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
-    return gen_gvec_fpst_arg_zpzz(s, fn, a);
-}
-
-#define DO_SVE2_ZPZZ_FP(NAME, name)                                         \
-static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a)                  \
-{                                                                           \
-    static gen_helper_gvec_4_ptr * const fns[4] = {                         \
-        NULL,                            gen_helper_sve2_##name##_zpzz_h,   \
-        gen_helper_sve2_##name##_zpzz_s, gen_helper_sve2_##name##_zpzz_d    \
-    };                                                                      \
-    return do_sve2_zpzz_fp(s, a, fns[a->esz]);                              \
-}
-
-DO_SVE2_ZPZZ_FP(FADDP, faddp)
-DO_SVE2_ZPZZ_FP(FMAXNMP, fmaxnmp)
-DO_SVE2_ZPZZ_FP(FMINNMP, fminnmp)
-DO_SVE2_ZPZZ_FP(FMAXP, fmaxp)
-DO_SVE2_ZPZZ_FP(FMINP, fminp)
+DO_ZPZZ_FP(FADDP, aa64_sve2, sve2_faddp_zpzz)
+DO_ZPZZ_FP(FMAXNMP, aa64_sve2, sve2_fmaxnmp_zpzz)
+DO_ZPZZ_FP(FMINNMP, aa64_sve2, sve2_fminnmp_zpzz)
+DO_ZPZZ_FP(FMAXP, aa64_sve2, sve2_fmaxp_zpzz)
+DO_ZPZZ_FP(FMINP, aa64_sve2, sve2_fminp_zpzz)
 
 /*
  * SVE Integer Multiply-Add (unpredicated)

From 6f5cd67008ec3fc3e270ac41bbd80f3b510613a5 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:53 -0700
Subject: [PATCH 685/935] target/arm: Use TRANS_FEAT for FCADD

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-101-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 30 +++++++-----------------------
 1 file changed, 7 insertions(+), 23 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 1108494919..e323b2d6d5 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3966,29 +3966,13 @@ DO_FPCMP(FACGT, facgt)
 
 #undef DO_FPCMP
 
-static bool trans_FCADD(DisasContext *s, arg_FCADD *a)
-{
-    static gen_helper_gvec_4_ptr * const fns[3] = {
-        gen_helper_sve_fcadd_h,
-        gen_helper_sve_fcadd_s,
-        gen_helper_sve_fcadd_d
-    };
-
-    if (a->esz == 0) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
-        tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           pred_full_reg_offset(s, a->pg),
-                           status, vsz, vsz, a->rot, fns[a->esz - 1]);
-        tcg_temp_free_ptr(status);
-    }
-    return true;
-}
+static gen_helper_gvec_4_ptr * const fcadd_fns[] = {
+    NULL,                   gen_helper_sve_fcadd_h,
+    gen_helper_sve_fcadd_s, gen_helper_sve_fcadd_d,
+};
+TRANS_FEAT(FCADD, aa64_sve, gen_gvec_fpst_zzzp, fcadd_fns[a->esz],
+           a->rd, a->rn, a->rm, a->pg, a->rot,
+           a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR)
 
 static bool do_fmla(DisasContext *s, arg_rprrr_esz *a,
                     gen_helper_gvec_5_ptr *fn)

From e14da11035e165798e611bcd6479ab0e54516de8 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:54 -0700
Subject: [PATCH 686/935] target/arm: Introduce gen_gvec_fpst_zzzzp

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-102-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 59 +++++++++++++++++++-------------------
 1 file changed, 29 insertions(+), 30 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index e323b2d6d5..8f50956d3b 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -289,6 +289,30 @@ static bool gen_gvec_fpst_zzzz(DisasContext *s, gen_helper_gvec_4_ptr *fn,
     return ret;
 }
 
+/* Invoke an out-of-line helper on 4 Zregs, 1 Preg, plus fpst. */
+static bool gen_gvec_fpst_zzzzp(DisasContext *s, gen_helper_gvec_5_ptr *fn,
+                                int rd, int rn, int rm, int ra, int pg,
+                                int data, ARMFPStatusFlavour flavour)
+{
+    if (fn == NULL) {
+        return false;
+    }
+    if (sve_access_check(s)) {
+        unsigned vsz = vec_full_reg_size(s);
+        TCGv_ptr status = fpstatus_ptr(flavour);
+
+        tcg_gen_gvec_5_ptr(vec_full_reg_offset(s, rd),
+                           vec_full_reg_offset(s, rn),
+                           vec_full_reg_offset(s, rm),
+                           vec_full_reg_offset(s, ra),
+                           pred_full_reg_offset(s, pg),
+                           status, vsz, vsz, data, fn);
+
+        tcg_temp_free_ptr(status);
+    }
+    return true;
+}
+
 /* Invoke an out-of-line helper on 2 Zregs and a predicate. */
 static bool gen_gvec_ool_zzp(DisasContext *s, gen_helper_gvec_3 *fn,
                              int rd, int rn, int pg, int data)
@@ -3977,21 +4001,8 @@ TRANS_FEAT(FCADD, aa64_sve, gen_gvec_fpst_zzzp, fcadd_fns[a->esz],
 static bool do_fmla(DisasContext *s, arg_rprrr_esz *a,
                     gen_helper_gvec_5_ptr *fn)
 {
-    if (a->esz == 0) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
-        tcg_gen_gvec_5_ptr(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           vec_full_reg_offset(s, a->ra),
-                           pred_full_reg_offset(s, a->pg),
-                           status, vsz, vsz, 0, fn);
-        tcg_temp_free_ptr(status);
-    }
-    return true;
+    return gen_gvec_fpst_zzzzp(s, fn, a->rd, a->rn, a->rm, a->ra, a->pg, 0,
+                               a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
 }
 
 #define DO_FMLA(NAME, name) \
@@ -4020,21 +4031,9 @@ static bool trans_FCMLA_zpzzz(DisasContext *s, arg_FCMLA_zpzzz *a)
         gen_helper_sve_fcmla_zpzzz_d,
     };
 
-    if (a->esz == 0) {
-        return false;
-    }
-    if (sve_access_check(s)) {
-        unsigned vsz = vec_full_reg_size(s);
-        TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
-        tcg_gen_gvec_5_ptr(vec_full_reg_offset(s, a->rd),
-                           vec_full_reg_offset(s, a->rn),
-                           vec_full_reg_offset(s, a->rm),
-                           vec_full_reg_offset(s, a->ra),
-                           pred_full_reg_offset(s, a->pg),
-                           status, vsz, vsz, a->rot, fns[a->esz]);
-        tcg_temp_free_ptr(status);
-    }
-    return true;
+    return gen_gvec_fpst_zzzzp(s, fns[a->esz], a->rd, a->rn, a->rm,
+                               a->ra, a->pg, a->rot,
+                               a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
 }
 
 static bool trans_FCMLA_zzxz(DisasContext *s, arg_FCMLA_zzxz *a)

From 498be5b87d61f377b38701b31e460eed7a104c0d Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:55 -0700
Subject: [PATCH 687/935] target/arm: Use TRANS_FEAT for gen_gvec_fpst_zzzzp

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-103-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 42 +++++++++++++-------------------------
 1 file changed, 14 insertions(+), 28 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 8f50956d3b..75854a7c6c 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3998,22 +3998,14 @@ TRANS_FEAT(FCADD, aa64_sve, gen_gvec_fpst_zzzp, fcadd_fns[a->esz],
            a->rd, a->rn, a->rm, a->pg, a->rot,
            a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR)
 
-static bool do_fmla(DisasContext *s, arg_rprrr_esz *a,
-                    gen_helper_gvec_5_ptr *fn)
-{
-    return gen_gvec_fpst_zzzzp(s, fn, a->rd, a->rn, a->rm, a->ra, a->pg, 0,
-                               a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
-}
-
 #define DO_FMLA(NAME, name) \
-static bool trans_##NAME(DisasContext *s, arg_rprrr_esz *a)          \
-{                                                                    \
-    static gen_helper_gvec_5_ptr * const fns[4] = {                  \
-        NULL, gen_helper_sve_##name##_h,                             \
-        gen_helper_sve_##name##_s, gen_helper_sve_##name##_d         \
-    };                                                               \
-    return do_fmla(s, a, fns[a->esz]);                               \
-}
+    static gen_helper_gvec_5_ptr * const name##_fns[4] = {              \
+        NULL, gen_helper_sve_##name##_h,                                \
+        gen_helper_sve_##name##_s, gen_helper_sve_##name##_d            \
+    };                                                                  \
+    TRANS_FEAT(NAME, aa64_sve, gen_gvec_fpst_zzzzp, name##_fns[a->esz], \
+               a->rd, a->rn, a->rm, a->ra, a->pg, 0,                    \
+               a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR)
 
 DO_FMLA(FMLA_zpzzz, fmla_zpzzz)
 DO_FMLA(FMLS_zpzzz, fmls_zpzzz)
@@ -4022,19 +4014,13 @@ DO_FMLA(FNMLS_zpzzz, fnmls_zpzzz)
 
 #undef DO_FMLA
 
-static bool trans_FCMLA_zpzzz(DisasContext *s, arg_FCMLA_zpzzz *a)
-{
-    static gen_helper_gvec_5_ptr * const fns[4] = {
-        NULL,
-        gen_helper_sve_fcmla_zpzzz_h,
-        gen_helper_sve_fcmla_zpzzz_s,
-        gen_helper_sve_fcmla_zpzzz_d,
-    };
-
-    return gen_gvec_fpst_zzzzp(s, fns[a->esz], a->rd, a->rn, a->rm,
-                               a->ra, a->pg, a->rot,
-                               a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
-}
+static gen_helper_gvec_5_ptr * const fcmla_fns[4] = {
+    NULL,                         gen_helper_sve_fcmla_zpzzz_h,
+    gen_helper_sve_fcmla_zpzzz_s, gen_helper_sve_fcmla_zpzzz_d,
+};
+TRANS_FEAT(FCMLA_zpzzz, aa64_sve, gen_gvec_fpst_zzzzp, fcmla_fns[a->esz],
+           a->rd, a->rn, a->rm, a->ra, a->pg, a->rot,
+           a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR)
 
 static bool trans_FCMLA_zzxz(DisasContext *s, arg_FCMLA_zzxz *a)
 {

From 413ee8e41a42571179482511313904f418331232 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:56 -0700
Subject: [PATCH 688/935] target/arm: Move null function and sve check into
 do_fp_imm

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-104-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 27 ++++++++++++++-------------
 1 file changed, 14 insertions(+), 13 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 75854a7c6c..b47d5d7f21 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3911,33 +3911,34 @@ static void do_fp_scalar(DisasContext *s, int zd, int zn, int pg, bool is_fp16,
     tcg_temp_free_ptr(t_zd);
 }
 
-static void do_fp_imm(DisasContext *s, arg_rpri_esz *a, uint64_t imm,
+static bool do_fp_imm(DisasContext *s, arg_rpri_esz *a, uint64_t imm,
                       gen_helper_sve_fp2scalar *fn)
 {
-    do_fp_scalar(s, a->rd, a->rn, a->pg, a->esz == MO_16,
-                 tcg_constant_i64(imm), fn);
+    if (fn == NULL) {
+        return false;
+    }
+    if (sve_access_check(s)) {
+        do_fp_scalar(s, a->rd, a->rn, a->pg, a->esz == MO_16,
+                     tcg_constant_i64(imm), fn);
+    }
+    return true;
 }
 
 #define DO_FP_IMM(NAME, name, const0, const1) \
 static bool trans_##NAME##_zpzi(DisasContext *s, arg_rpri_esz *a)         \
 {                                                                         \
-    static gen_helper_sve_fp2scalar * const fns[3] = {                    \
-        gen_helper_sve_##name##_h,                                        \
+    static gen_helper_sve_fp2scalar * const fns[4] = {                    \
+        NULL, gen_helper_sve_##name##_h,                                  \
         gen_helper_sve_##name##_s,                                        \
         gen_helper_sve_##name##_d                                         \
     };                                                                    \
-    static uint64_t const val[3][2] = {                                   \
+    static uint64_t const val[4][2] = {                                   \
+        { -1, -1 },                                                       \
         { float16_##const0, float16_##const1 },                           \
         { float32_##const0, float32_##const1 },                           \
         { float64_##const0, float64_##const1 },                           \
     };                                                                    \
-    if (a->esz == 0) {                                                    \
-        return false;                                                     \
-    }                                                                     \
-    if (sve_access_check(s)) {                                            \
-        do_fp_imm(s, a, val[a->esz - 1][a->imm], fns[a->esz - 1]);        \
-    }                                                                     \
-    return true;                                                          \
+    return do_fp_imm(s, a, val[a->esz][a->imm], fns[a->esz]);             \
 }
 
 DO_FP_IMM(FADD, fadds, half, one)

From 98c37459c346f84ec032747bc7cd692877a0bc9e Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:57 -0700
Subject: [PATCH 689/935] target/arm: Use TRANS_FEAT for DO_FP_IMM

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-105-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 30 ++++++++++++++----------------
 1 file changed, 14 insertions(+), 16 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index b47d5d7f21..e2ae387d62 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3924,22 +3924,20 @@ static bool do_fp_imm(DisasContext *s, arg_rpri_esz *a, uint64_t imm,
     return true;
 }
 
-#define DO_FP_IMM(NAME, name, const0, const1) \
-static bool trans_##NAME##_zpzi(DisasContext *s, arg_rpri_esz *a)         \
-{                                                                         \
-    static gen_helper_sve_fp2scalar * const fns[4] = {                    \
-        NULL, gen_helper_sve_##name##_h,                                  \
-        gen_helper_sve_##name##_s,                                        \
-        gen_helper_sve_##name##_d                                         \
-    };                                                                    \
-    static uint64_t const val[4][2] = {                                   \
-        { -1, -1 },                                                       \
-        { float16_##const0, float16_##const1 },                           \
-        { float32_##const0, float32_##const1 },                           \
-        { float64_##const0, float64_##const1 },                           \
-    };                                                                    \
-    return do_fp_imm(s, a, val[a->esz][a->imm], fns[a->esz]);             \
-}
+#define DO_FP_IMM(NAME, name, const0, const1)                           \
+    static gen_helper_sve_fp2scalar * const name##_fns[4] = {           \
+        NULL, gen_helper_sve_##name##_h,                                \
+        gen_helper_sve_##name##_s,                                      \
+        gen_helper_sve_##name##_d                                       \
+    };                                                                  \
+    static uint64_t const name##_const[4][2] = {                        \
+        { -1, -1 },                                                     \
+        { float16_##const0, float16_##const1 },                         \
+        { float32_##const0, float32_##const1 },                         \
+        { float64_##const0, float64_##const1 },                         \
+    };                                                                  \
+    TRANS_FEAT(NAME##_zpzi, aa64_sve, do_fp_imm, a,                     \
+               name##_const[a->esz][a->imm], name##_fns[a->esz])
 
 DO_FP_IMM(FADD, fadds, half, one)
 DO_FP_IMM(FSUB, fsubs, half, one)

From d961b3e40bd2c98de8bac554597e905daab13b91 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:58 -0700
Subject: [PATCH 690/935] target/arm: Use TRANS_FEAT for DO_FPCMP

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-106-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index e2ae387d62..886cf539a5 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -3970,14 +3970,11 @@ static bool do_fp_cmp(DisasContext *s, arg_rprr_esz *a,
 }
 
 #define DO_FPCMP(NAME, name) \
-static bool trans_##NAME##_ppzz(DisasContext *s, arg_rprr_esz *a)     \
-{                                                                     \
-    static gen_helper_gvec_4_ptr * const fns[4] = {                   \
+    static gen_helper_gvec_4_ptr * const name##_fns[4] = {            \
         NULL, gen_helper_sve_##name##_h,                              \
         gen_helper_sve_##name##_s, gen_helper_sve_##name##_d          \
     };                                                                \
-    return do_fp_cmp(s, a, fns[a->esz]);                              \
-}
+    TRANS_FEAT(NAME##_ppzz, aa64_sve, do_fp_cmp, a, name##_fns[a->esz])
 
 DO_FPCMP(FCMGE, fcmge)
 DO_FPCMP(FCMGT, fcmgt)

From df9024760efa9a32e24629eb80665c8c4ec5f145 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:18:59 -0700
Subject: [PATCH 691/935] target/arm: Remove assert in trans_FCMLA_zzxz

Since 636ddeb15c0, we do not require rd == ra.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-107-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 886cf539a5..436d09b928 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -4027,8 +4027,6 @@ static bool trans_FCMLA_zzxz(DisasContext *s, arg_FCMLA_zzxz *a)
         NULL,
     };
 
-    tcg_debug_assert(a->rd == a->ra);
-
     return gen_gvec_fpst_zzzz(s, fns[a->esz], a->rd, a->rn, a->rm, a->ra,
                               a->index * 4 + a->rot,
                               a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);

From e600d64980f2a2f1377ade1c055497e164ec3a61 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:19:00 -0700
Subject: [PATCH 692/935] target/arm: Use TRANS_FEAT for FCMLA_zzxz

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-108-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 19 ++++++-------------
 1 file changed, 6 insertions(+), 13 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 436d09b928..70e8d90ae8 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -4018,19 +4018,12 @@ TRANS_FEAT(FCMLA_zpzzz, aa64_sve, gen_gvec_fpst_zzzzp, fcmla_fns[a->esz],
            a->rd, a->rn, a->rm, a->ra, a->pg, a->rot,
            a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR)
 
-static bool trans_FCMLA_zzxz(DisasContext *s, arg_FCMLA_zzxz *a)
-{
-    static gen_helper_gvec_4_ptr * const fns[4] = {
-        NULL,
-        gen_helper_gvec_fcmlah_idx,
-        gen_helper_gvec_fcmlas_idx,
-        NULL,
-    };
-
-    return gen_gvec_fpst_zzzz(s, fns[a->esz], a->rd, a->rn, a->rm, a->ra,
-                              a->index * 4 + a->rot,
-                              a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR);
-}
+static gen_helper_gvec_4_ptr * const fcmla_idx_fns[4] = {
+    NULL, gen_helper_gvec_fcmlah_idx, gen_helper_gvec_fcmlas_idx, NULL
+};
+TRANS_FEAT(FCMLA_zzxz, aa64_sve, gen_gvec_fpst_zzzz, fcmla_idx_fns[a->esz],
+           a->rd, a->rn, a->rm, a->ra, a->index * 4 + a->rot,
+           a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR)
 
 /*
  *** SVE Floating Point Unary Operations Predicated Group

From 6100d084934e8152886625a071ca6b9579ebb260 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:19:01 -0700
Subject: [PATCH 693/935] target/arm: Use TRANS_FEAT for do_narrow_extract

Rename from do_sve2_narrow_extract and hoist the sve2
check into the TRANS_FEAT macro.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-109-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 223 +++++++++++++++++--------------------
 1 file changed, 102 insertions(+), 121 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 70e8d90ae8..8e7f8308c7 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -6345,11 +6345,10 @@ TRANS_FEAT(SLI, aa64_sve2, gen_gvec_fn_arg_zzi, gen_gvec_sli, a)
 TRANS_FEAT(SABA, aa64_sve2, gen_gvec_fn_arg_zzz, gen_gvec_saba, a)
 TRANS_FEAT(UABA, aa64_sve2, gen_gvec_fn_arg_zzz, gen_gvec_uaba, a)
 
-static bool do_sve2_narrow_extract(DisasContext *s, arg_rri_esz *a,
-                                   const GVecGen2 ops[3])
+static bool do_narrow_extract(DisasContext *s, arg_rri_esz *a,
+                              const GVecGen2 ops[3])
 {
-    if (a->esz < 0 || a->esz > MO_32 || a->imm != 0 ||
-        !dc_isar_feature(aa64_sve2, s)) {
+    if (a->esz < 0 || a->esz > MO_32 || a->imm != 0) {
         return false;
     }
     if (sve_access_check(s)) {
@@ -6382,24 +6381,21 @@ static void gen_sqxtnb_vec(unsigned vece, TCGv_vec d, TCGv_vec n)
     tcg_temp_free_vec(t);
 }
 
-static bool trans_SQXTNB(DisasContext *s, arg_rri_esz *a)
-{
-    static const GVecGen2 ops[3] = {
-        { .fniv = gen_sqxtnb_vec,
-          .opt_opc = sqxtn_list,
-          .fno = gen_helper_sve2_sqxtnb_h,
-          .vece = MO_16 },
-        { .fniv = gen_sqxtnb_vec,
-          .opt_opc = sqxtn_list,
-          .fno = gen_helper_sve2_sqxtnb_s,
-          .vece = MO_32 },
-        { .fniv = gen_sqxtnb_vec,
-          .opt_opc = sqxtn_list,
-          .fno = gen_helper_sve2_sqxtnb_d,
-          .vece = MO_64 },
-    };
-    return do_sve2_narrow_extract(s, a, ops);
-}
+static const GVecGen2 sqxtnb_ops[3] = {
+    { .fniv = gen_sqxtnb_vec,
+      .opt_opc = sqxtn_list,
+      .fno = gen_helper_sve2_sqxtnb_h,
+      .vece = MO_16 },
+    { .fniv = gen_sqxtnb_vec,
+      .opt_opc = sqxtn_list,
+      .fno = gen_helper_sve2_sqxtnb_s,
+      .vece = MO_32 },
+    { .fniv = gen_sqxtnb_vec,
+      .opt_opc = sqxtn_list,
+      .fno = gen_helper_sve2_sqxtnb_d,
+      .vece = MO_64 },
+};
+TRANS_FEAT(SQXTNB, aa64_sve2, do_narrow_extract, a, sqxtnb_ops)
 
 static void gen_sqxtnt_vec(unsigned vece, TCGv_vec d, TCGv_vec n)
 {
@@ -6419,27 +6415,24 @@ static void gen_sqxtnt_vec(unsigned vece, TCGv_vec d, TCGv_vec n)
     tcg_temp_free_vec(t);
 }
 
-static bool trans_SQXTNT(DisasContext *s, arg_rri_esz *a)
-{
-    static const GVecGen2 ops[3] = {
-        { .fniv = gen_sqxtnt_vec,
-          .opt_opc = sqxtn_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_sqxtnt_h,
-          .vece = MO_16 },
-        { .fniv = gen_sqxtnt_vec,
-          .opt_opc = sqxtn_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_sqxtnt_s,
-          .vece = MO_32 },
-        { .fniv = gen_sqxtnt_vec,
-          .opt_opc = sqxtn_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_sqxtnt_d,
-          .vece = MO_64 },
-    };
-    return do_sve2_narrow_extract(s, a, ops);
-}
+static const GVecGen2 sqxtnt_ops[3] = {
+    { .fniv = gen_sqxtnt_vec,
+      .opt_opc = sqxtn_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_sqxtnt_h,
+      .vece = MO_16 },
+    { .fniv = gen_sqxtnt_vec,
+      .opt_opc = sqxtn_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_sqxtnt_s,
+      .vece = MO_32 },
+    { .fniv = gen_sqxtnt_vec,
+      .opt_opc = sqxtn_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_sqxtnt_d,
+      .vece = MO_64 },
+};
+TRANS_FEAT(SQXTNT, aa64_sve2, do_narrow_extract, a, sqxtnt_ops)
 
 static const TCGOpcode uqxtn_list[] = {
     INDEX_op_shli_vec, INDEX_op_umin_vec, 0
@@ -6456,24 +6449,21 @@ static void gen_uqxtnb_vec(unsigned vece, TCGv_vec d, TCGv_vec n)
     tcg_temp_free_vec(t);
 }
 
-static bool trans_UQXTNB(DisasContext *s, arg_rri_esz *a)
-{
-    static const GVecGen2 ops[3] = {
-        { .fniv = gen_uqxtnb_vec,
-          .opt_opc = uqxtn_list,
-          .fno = gen_helper_sve2_uqxtnb_h,
-          .vece = MO_16 },
-        { .fniv = gen_uqxtnb_vec,
-          .opt_opc = uqxtn_list,
-          .fno = gen_helper_sve2_uqxtnb_s,
-          .vece = MO_32 },
-        { .fniv = gen_uqxtnb_vec,
-          .opt_opc = uqxtn_list,
-          .fno = gen_helper_sve2_uqxtnb_d,
-          .vece = MO_64 },
-    };
-    return do_sve2_narrow_extract(s, a, ops);
-}
+static const GVecGen2 uqxtnb_ops[3] = {
+    { .fniv = gen_uqxtnb_vec,
+      .opt_opc = uqxtn_list,
+      .fno = gen_helper_sve2_uqxtnb_h,
+      .vece = MO_16 },
+    { .fniv = gen_uqxtnb_vec,
+      .opt_opc = uqxtn_list,
+      .fno = gen_helper_sve2_uqxtnb_s,
+      .vece = MO_32 },
+    { .fniv = gen_uqxtnb_vec,
+      .opt_opc = uqxtn_list,
+      .fno = gen_helper_sve2_uqxtnb_d,
+      .vece = MO_64 },
+};
+TRANS_FEAT(UQXTNB, aa64_sve2, do_narrow_extract, a, uqxtnb_ops)
 
 static void gen_uqxtnt_vec(unsigned vece, TCGv_vec d, TCGv_vec n)
 {
@@ -6488,27 +6478,24 @@ static void gen_uqxtnt_vec(unsigned vece, TCGv_vec d, TCGv_vec n)
     tcg_temp_free_vec(t);
 }
 
-static bool trans_UQXTNT(DisasContext *s, arg_rri_esz *a)
-{
-    static const GVecGen2 ops[3] = {
-        { .fniv = gen_uqxtnt_vec,
-          .opt_opc = uqxtn_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_uqxtnt_h,
-          .vece = MO_16 },
-        { .fniv = gen_uqxtnt_vec,
-          .opt_opc = uqxtn_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_uqxtnt_s,
-          .vece = MO_32 },
-        { .fniv = gen_uqxtnt_vec,
-          .opt_opc = uqxtn_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_uqxtnt_d,
-          .vece = MO_64 },
-    };
-    return do_sve2_narrow_extract(s, a, ops);
-}
+static const GVecGen2 uqxtnt_ops[3] = {
+    { .fniv = gen_uqxtnt_vec,
+      .opt_opc = uqxtn_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_uqxtnt_h,
+      .vece = MO_16 },
+    { .fniv = gen_uqxtnt_vec,
+      .opt_opc = uqxtn_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_uqxtnt_s,
+      .vece = MO_32 },
+    { .fniv = gen_uqxtnt_vec,
+      .opt_opc = uqxtn_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_uqxtnt_d,
+      .vece = MO_64 },
+};
+TRANS_FEAT(UQXTNT, aa64_sve2, do_narrow_extract, a, uqxtnt_ops)
 
 static const TCGOpcode sqxtun_list[] = {
     INDEX_op_shli_vec, INDEX_op_umin_vec, INDEX_op_smax_vec, 0
@@ -6527,24 +6514,21 @@ static void gen_sqxtunb_vec(unsigned vece, TCGv_vec d, TCGv_vec n)
     tcg_temp_free_vec(t);
 }
 
-static bool trans_SQXTUNB(DisasContext *s, arg_rri_esz *a)
-{
-    static const GVecGen2 ops[3] = {
-        { .fniv = gen_sqxtunb_vec,
-          .opt_opc = sqxtun_list,
-          .fno = gen_helper_sve2_sqxtunb_h,
-          .vece = MO_16 },
-        { .fniv = gen_sqxtunb_vec,
-          .opt_opc = sqxtun_list,
-          .fno = gen_helper_sve2_sqxtunb_s,
-          .vece = MO_32 },
-        { .fniv = gen_sqxtunb_vec,
-          .opt_opc = sqxtun_list,
-          .fno = gen_helper_sve2_sqxtunb_d,
-          .vece = MO_64 },
-    };
-    return do_sve2_narrow_extract(s, a, ops);
-}
+static const GVecGen2 sqxtunb_ops[3] = {
+    { .fniv = gen_sqxtunb_vec,
+      .opt_opc = sqxtun_list,
+      .fno = gen_helper_sve2_sqxtunb_h,
+      .vece = MO_16 },
+    { .fniv = gen_sqxtunb_vec,
+      .opt_opc = sqxtun_list,
+      .fno = gen_helper_sve2_sqxtunb_s,
+      .vece = MO_32 },
+    { .fniv = gen_sqxtunb_vec,
+      .opt_opc = sqxtun_list,
+      .fno = gen_helper_sve2_sqxtunb_d,
+      .vece = MO_64 },
+};
+TRANS_FEAT(SQXTUNB, aa64_sve2, do_narrow_extract, a, sqxtunb_ops)
 
 static void gen_sqxtunt_vec(unsigned vece, TCGv_vec d, TCGv_vec n)
 {
@@ -6561,27 +6545,24 @@ static void gen_sqxtunt_vec(unsigned vece, TCGv_vec d, TCGv_vec n)
     tcg_temp_free_vec(t);
 }
 
-static bool trans_SQXTUNT(DisasContext *s, arg_rri_esz *a)
-{
-    static const GVecGen2 ops[3] = {
-        { .fniv = gen_sqxtunt_vec,
-          .opt_opc = sqxtun_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_sqxtunt_h,
-          .vece = MO_16 },
-        { .fniv = gen_sqxtunt_vec,
-          .opt_opc = sqxtun_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_sqxtunt_s,
-          .vece = MO_32 },
-        { .fniv = gen_sqxtunt_vec,
-          .opt_opc = sqxtun_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_sqxtunt_d,
-          .vece = MO_64 },
-    };
-    return do_sve2_narrow_extract(s, a, ops);
-}
+static const GVecGen2 sqxtunt_ops[3] = {
+    { .fniv = gen_sqxtunt_vec,
+      .opt_opc = sqxtun_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_sqxtunt_h,
+      .vece = MO_16 },
+    { .fniv = gen_sqxtunt_vec,
+      .opt_opc = sqxtun_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_sqxtunt_s,
+      .vece = MO_32 },
+    { .fniv = gen_sqxtunt_vec,
+      .opt_opc = sqxtun_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_sqxtunt_d,
+      .vece = MO_64 },
+};
+TRANS_FEAT(SQXTUNT, aa64_sve2, do_narrow_extract, a, sqxtunt_ops)
 
 static bool do_sve2_shr_narrow(DisasContext *s, arg_rri_esz *a,
                                const GVecGen2i ops[3])

From 5a528bb5d8e8e298317975e68510f329f89fc709 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:19:02 -0700
Subject: [PATCH 694/935] target/arm: Use TRANS_FEAT for do_shll_tb

Rename from do_sve2_shll_tb and hoist the sve2
check into the TRANS_FEAT macro.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-110-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 102 ++++++++++++++++---------------------
 1 file changed, 45 insertions(+), 57 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 8e7f8308c7..0fb118f6ef 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -6194,46 +6194,11 @@ static void gen_ushll_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t imm)
     }
 }
 
-static bool do_sve2_shll_tb(DisasContext *s, arg_rri_esz *a,
-                            bool sel, bool uns)
+static bool do_shll_tb(DisasContext *s, arg_rri_esz *a,
+                       const GVecGen2i ops[3], bool sel)
 {
-    static const TCGOpcode sshll_list[] = {
-        INDEX_op_shli_vec, INDEX_op_sari_vec, 0
-    };
-    static const TCGOpcode ushll_list[] = {
-        INDEX_op_shli_vec, INDEX_op_shri_vec, 0
-    };
-    static const GVecGen2i ops[2][3] = {
-        { { .fniv = gen_sshll_vec,
-            .opt_opc = sshll_list,
-            .fno = gen_helper_sve2_sshll_h,
-            .vece = MO_16 },
-          { .fniv = gen_sshll_vec,
-            .opt_opc = sshll_list,
-            .fno = gen_helper_sve2_sshll_s,
-            .vece = MO_32 },
-          { .fniv = gen_sshll_vec,
-            .opt_opc = sshll_list,
-            .fno = gen_helper_sve2_sshll_d,
-            .vece = MO_64 } },
-        { { .fni8 = gen_ushll16_i64,
-            .fniv = gen_ushll_vec,
-            .opt_opc = ushll_list,
-            .fno = gen_helper_sve2_ushll_h,
-            .vece = MO_16 },
-          { .fni8 = gen_ushll32_i64,
-            .fniv = gen_ushll_vec,
-            .opt_opc = ushll_list,
-            .fno = gen_helper_sve2_ushll_s,
-            .vece = MO_32 },
-          { .fni8 = gen_ushll64_i64,
-            .fniv = gen_ushll_vec,
-            .opt_opc = ushll_list,
-            .fno = gen_helper_sve2_ushll_d,
-            .vece = MO_64 } },
-    };
 
-    if (a->esz < 0 || a->esz > 2 || !dc_isar_feature(aa64_sve2, s)) {
+    if (a->esz < 0 || a->esz > 2) {
         return false;
     }
     if (sve_access_check(s)) {
@@ -6241,30 +6206,53 @@ static bool do_sve2_shll_tb(DisasContext *s, arg_rri_esz *a,
         tcg_gen_gvec_2i(vec_full_reg_offset(s, a->rd),
                         vec_full_reg_offset(s, a->rn),
                         vsz, vsz, (a->imm << 1) | sel,
-                        &ops[uns][a->esz]);
+                        &ops[a->esz]);
     }
     return true;
 }
 
-static bool trans_SSHLLB(DisasContext *s, arg_rri_esz *a)
-{
-    return do_sve2_shll_tb(s, a, false, false);
-}
+static const TCGOpcode sshll_list[] = {
+    INDEX_op_shli_vec, INDEX_op_sari_vec, 0
+};
+static const GVecGen2i sshll_ops[3] = {
+    { .fniv = gen_sshll_vec,
+      .opt_opc = sshll_list,
+      .fno = gen_helper_sve2_sshll_h,
+      .vece = MO_16 },
+    { .fniv = gen_sshll_vec,
+      .opt_opc = sshll_list,
+      .fno = gen_helper_sve2_sshll_s,
+      .vece = MO_32 },
+    { .fniv = gen_sshll_vec,
+      .opt_opc = sshll_list,
+      .fno = gen_helper_sve2_sshll_d,
+      .vece = MO_64 }
+};
+TRANS_FEAT(SSHLLB, aa64_sve2, do_shll_tb, a, sshll_ops, false)
+TRANS_FEAT(SSHLLT, aa64_sve2, do_shll_tb, a, sshll_ops, true)
 
-static bool trans_SSHLLT(DisasContext *s, arg_rri_esz *a)
-{
-    return do_sve2_shll_tb(s, a, true, false);
-}
-
-static bool trans_USHLLB(DisasContext *s, arg_rri_esz *a)
-{
-    return do_sve2_shll_tb(s, a, false, true);
-}
-
-static bool trans_USHLLT(DisasContext *s, arg_rri_esz *a)
-{
-    return do_sve2_shll_tb(s, a, true, true);
-}
+static const TCGOpcode ushll_list[] = {
+    INDEX_op_shli_vec, INDEX_op_shri_vec, 0
+};
+static const GVecGen2i ushll_ops[3] = {
+    { .fni8 = gen_ushll16_i64,
+      .fniv = gen_ushll_vec,
+      .opt_opc = ushll_list,
+      .fno = gen_helper_sve2_ushll_h,
+      .vece = MO_16 },
+    { .fni8 = gen_ushll32_i64,
+      .fniv = gen_ushll_vec,
+      .opt_opc = ushll_list,
+      .fno = gen_helper_sve2_ushll_s,
+      .vece = MO_32 },
+    { .fni8 = gen_ushll64_i64,
+      .fniv = gen_ushll_vec,
+      .opt_opc = ushll_list,
+      .fno = gen_helper_sve2_ushll_d,
+      .vece = MO_64 },
+};
+TRANS_FEAT(USHLLB, aa64_sve2, do_shll_tb, a, ushll_ops, false)
+TRANS_FEAT(USHLLT, aa64_sve2, do_shll_tb, a, ushll_ops, true)
 
 static gen_helper_gvec_3 * const bext_fns[4] = {
     gen_helper_sve2_bext_b, gen_helper_sve2_bext_h,

From f7f2f0faa2b5091e27da442b48c0d27d1274b2f6 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:19:03 -0700
Subject: [PATCH 695/935] target/arm: Use TRANS_FEAT for do_shr_narrow

Rename from do_sve2_shr_narrow and hoist the sve2
check into the TRANS_FEAT macro.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-111-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 470 +++++++++++++++++--------------------
 1 file changed, 211 insertions(+), 259 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 0fb118f6ef..44af7530b6 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -6552,10 +6552,10 @@ static const GVecGen2 sqxtunt_ops[3] = {
 };
 TRANS_FEAT(SQXTUNT, aa64_sve2, do_narrow_extract, a, sqxtunt_ops)
 
-static bool do_sve2_shr_narrow(DisasContext *s, arg_rri_esz *a,
-                               const GVecGen2i ops[3])
+static bool do_shr_narrow(DisasContext *s, arg_rri_esz *a,
+                          const GVecGen2i ops[3])
 {
-    if (a->esz < 0 || a->esz > MO_32 || !dc_isar_feature(aa64_sve2, s)) {
+    if (a->esz < 0 || a->esz > MO_32) {
         return false;
     }
     assert(a->imm > 0 && a->imm <= (8 << a->esz));
@@ -6604,28 +6604,25 @@ static void gen_shrnb_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t shr)
     tcg_temp_free_vec(t);
 }
 
-static bool trans_SHRNB(DisasContext *s, arg_rri_esz *a)
-{
-    static const TCGOpcode vec_list[] = { INDEX_op_shri_vec, 0 };
-    static const GVecGen2i ops[3] = {
-        { .fni8 = gen_shrnb16_i64,
-          .fniv = gen_shrnb_vec,
-          .opt_opc = vec_list,
-          .fno = gen_helper_sve2_shrnb_h,
-          .vece = MO_16 },
-        { .fni8 = gen_shrnb32_i64,
-          .fniv = gen_shrnb_vec,
-          .opt_opc = vec_list,
-          .fno = gen_helper_sve2_shrnb_s,
-          .vece = MO_32 },
-        { .fni8 = gen_shrnb64_i64,
-          .fniv = gen_shrnb_vec,
-          .opt_opc = vec_list,
-          .fno = gen_helper_sve2_shrnb_d,
-          .vece = MO_64 },
-    };
-    return do_sve2_shr_narrow(s, a, ops);
-}
+static const TCGOpcode shrnb_vec_list[] = { INDEX_op_shri_vec, 0 };
+static const GVecGen2i shrnb_ops[3] = {
+    { .fni8 = gen_shrnb16_i64,
+      .fniv = gen_shrnb_vec,
+      .opt_opc = shrnb_vec_list,
+      .fno = gen_helper_sve2_shrnb_h,
+      .vece = MO_16 },
+    { .fni8 = gen_shrnb32_i64,
+      .fniv = gen_shrnb_vec,
+      .opt_opc = shrnb_vec_list,
+      .fno = gen_helper_sve2_shrnb_s,
+      .vece = MO_32 },
+    { .fni8 = gen_shrnb64_i64,
+      .fniv = gen_shrnb_vec,
+      .opt_opc = shrnb_vec_list,
+      .fno = gen_helper_sve2_shrnb_d,
+      .vece = MO_64 },
+};
+TRANS_FEAT(SHRNB, aa64_sve2, do_shr_narrow, a, shrnb_ops)
 
 static void gen_shrnt_i64(unsigned vece, TCGv_i64 d, TCGv_i64 n, int shr)
 {
@@ -6666,51 +6663,42 @@ static void gen_shrnt_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t shr)
     tcg_temp_free_vec(t);
 }
 
-static bool trans_SHRNT(DisasContext *s, arg_rri_esz *a)
-{
-    static const TCGOpcode vec_list[] = { INDEX_op_shli_vec, 0 };
-    static const GVecGen2i ops[3] = {
-        { .fni8 = gen_shrnt16_i64,
-          .fniv = gen_shrnt_vec,
-          .opt_opc = vec_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_shrnt_h,
-          .vece = MO_16 },
-        { .fni8 = gen_shrnt32_i64,
-          .fniv = gen_shrnt_vec,
-          .opt_opc = vec_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_shrnt_s,
-          .vece = MO_32 },
-        { .fni8 = gen_shrnt64_i64,
-          .fniv = gen_shrnt_vec,
-          .opt_opc = vec_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_shrnt_d,
-          .vece = MO_64 },
-    };
-    return do_sve2_shr_narrow(s, a, ops);
-}
+static const TCGOpcode shrnt_vec_list[] = { INDEX_op_shli_vec, 0 };
+static const GVecGen2i shrnt_ops[3] = {
+    { .fni8 = gen_shrnt16_i64,
+      .fniv = gen_shrnt_vec,
+      .opt_opc = shrnt_vec_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_shrnt_h,
+      .vece = MO_16 },
+    { .fni8 = gen_shrnt32_i64,
+      .fniv = gen_shrnt_vec,
+      .opt_opc = shrnt_vec_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_shrnt_s,
+      .vece = MO_32 },
+    { .fni8 = gen_shrnt64_i64,
+      .fniv = gen_shrnt_vec,
+      .opt_opc = shrnt_vec_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_shrnt_d,
+      .vece = MO_64 },
+};
+TRANS_FEAT(SHRNT, aa64_sve2, do_shr_narrow, a, shrnt_ops)
 
-static bool trans_RSHRNB(DisasContext *s, arg_rri_esz *a)
-{
-    static const GVecGen2i ops[3] = {
-        { .fno = gen_helper_sve2_rshrnb_h },
-        { .fno = gen_helper_sve2_rshrnb_s },
-        { .fno = gen_helper_sve2_rshrnb_d },
-    };
-    return do_sve2_shr_narrow(s, a, ops);
-}
+static const GVecGen2i rshrnb_ops[3] = {
+    { .fno = gen_helper_sve2_rshrnb_h },
+    { .fno = gen_helper_sve2_rshrnb_s },
+    { .fno = gen_helper_sve2_rshrnb_d },
+};
+TRANS_FEAT(RSHRNB, aa64_sve2, do_shr_narrow, a, rshrnb_ops)
 
-static bool trans_RSHRNT(DisasContext *s, arg_rri_esz *a)
-{
-    static const GVecGen2i ops[3] = {
-        { .fno = gen_helper_sve2_rshrnt_h },
-        { .fno = gen_helper_sve2_rshrnt_s },
-        { .fno = gen_helper_sve2_rshrnt_d },
-    };
-    return do_sve2_shr_narrow(s, a, ops);
-}
+static const GVecGen2i rshrnt_ops[3] = {
+    { .fno = gen_helper_sve2_rshrnt_h },
+    { .fno = gen_helper_sve2_rshrnt_s },
+    { .fno = gen_helper_sve2_rshrnt_d },
+};
+TRANS_FEAT(RSHRNT, aa64_sve2, do_shr_narrow, a, rshrnt_ops)
 
 static void gen_sqshrunb_vec(unsigned vece, TCGv_vec d,
                              TCGv_vec n, int64_t shr)
@@ -6726,27 +6714,24 @@ static void gen_sqshrunb_vec(unsigned vece, TCGv_vec d,
     tcg_temp_free_vec(t);
 }
 
-static bool trans_SQSHRUNB(DisasContext *s, arg_rri_esz *a)
-{
-    static const TCGOpcode vec_list[] = {
-        INDEX_op_sari_vec, INDEX_op_smax_vec, INDEX_op_umin_vec, 0
-    };
-    static const GVecGen2i ops[3] = {
-        { .fniv = gen_sqshrunb_vec,
-          .opt_opc = vec_list,
-          .fno = gen_helper_sve2_sqshrunb_h,
-          .vece = MO_16 },
-        { .fniv = gen_sqshrunb_vec,
-          .opt_opc = vec_list,
-          .fno = gen_helper_sve2_sqshrunb_s,
-          .vece = MO_32 },
-        { .fniv = gen_sqshrunb_vec,
-          .opt_opc = vec_list,
-          .fno = gen_helper_sve2_sqshrunb_d,
-          .vece = MO_64 },
-    };
-    return do_sve2_shr_narrow(s, a, ops);
-}
+static const TCGOpcode sqshrunb_vec_list[] = {
+    INDEX_op_sari_vec, INDEX_op_smax_vec, INDEX_op_umin_vec, 0
+};
+static const GVecGen2i sqshrunb_ops[3] = {
+    { .fniv = gen_sqshrunb_vec,
+      .opt_opc = sqshrunb_vec_list,
+      .fno = gen_helper_sve2_sqshrunb_h,
+      .vece = MO_16 },
+    { .fniv = gen_sqshrunb_vec,
+      .opt_opc = sqshrunb_vec_list,
+      .fno = gen_helper_sve2_sqshrunb_s,
+      .vece = MO_32 },
+    { .fniv = gen_sqshrunb_vec,
+      .opt_opc = sqshrunb_vec_list,
+      .fno = gen_helper_sve2_sqshrunb_d,
+      .vece = MO_64 },
+};
+TRANS_FEAT(SQSHRUNB, aa64_sve2, do_shr_narrow, a, sqshrunb_ops)
 
 static void gen_sqshrunt_vec(unsigned vece, TCGv_vec d,
                              TCGv_vec n, int64_t shr)
@@ -6764,51 +6749,42 @@ static void gen_sqshrunt_vec(unsigned vece, TCGv_vec d,
     tcg_temp_free_vec(t);
 }
 
-static bool trans_SQSHRUNT(DisasContext *s, arg_rri_esz *a)
-{
-    static const TCGOpcode vec_list[] = {
-        INDEX_op_shli_vec, INDEX_op_sari_vec,
-        INDEX_op_smax_vec, INDEX_op_umin_vec, 0
-    };
-    static const GVecGen2i ops[3] = {
-        { .fniv = gen_sqshrunt_vec,
-          .opt_opc = vec_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_sqshrunt_h,
-          .vece = MO_16 },
-        { .fniv = gen_sqshrunt_vec,
-          .opt_opc = vec_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_sqshrunt_s,
-          .vece = MO_32 },
-        { .fniv = gen_sqshrunt_vec,
-          .opt_opc = vec_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_sqshrunt_d,
-          .vece = MO_64 },
-    };
-    return do_sve2_shr_narrow(s, a, ops);
-}
+static const TCGOpcode sqshrunt_vec_list[] = {
+    INDEX_op_shli_vec, INDEX_op_sari_vec,
+    INDEX_op_smax_vec, INDEX_op_umin_vec, 0
+};
+static const GVecGen2i sqshrunt_ops[3] = {
+    { .fniv = gen_sqshrunt_vec,
+      .opt_opc = sqshrunt_vec_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_sqshrunt_h,
+      .vece = MO_16 },
+    { .fniv = gen_sqshrunt_vec,
+      .opt_opc = sqshrunt_vec_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_sqshrunt_s,
+      .vece = MO_32 },
+    { .fniv = gen_sqshrunt_vec,
+      .opt_opc = sqshrunt_vec_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_sqshrunt_d,
+      .vece = MO_64 },
+};
+TRANS_FEAT(SQSHRUNT, aa64_sve2, do_shr_narrow, a, sqshrunt_ops)
 
-static bool trans_SQRSHRUNB(DisasContext *s, arg_rri_esz *a)
-{
-    static const GVecGen2i ops[3] = {
-        { .fno = gen_helper_sve2_sqrshrunb_h },
-        { .fno = gen_helper_sve2_sqrshrunb_s },
-        { .fno = gen_helper_sve2_sqrshrunb_d },
-    };
-    return do_sve2_shr_narrow(s, a, ops);
-}
+static const GVecGen2i sqrshrunb_ops[3] = {
+    { .fno = gen_helper_sve2_sqrshrunb_h },
+    { .fno = gen_helper_sve2_sqrshrunb_s },
+    { .fno = gen_helper_sve2_sqrshrunb_d },
+};
+TRANS_FEAT(SQRSHRUNB, aa64_sve2, do_shr_narrow, a, sqrshrunb_ops)
 
-static bool trans_SQRSHRUNT(DisasContext *s, arg_rri_esz *a)
-{
-    static const GVecGen2i ops[3] = {
-        { .fno = gen_helper_sve2_sqrshrunt_h },
-        { .fno = gen_helper_sve2_sqrshrunt_s },
-        { .fno = gen_helper_sve2_sqrshrunt_d },
-    };
-    return do_sve2_shr_narrow(s, a, ops);
-}
+static const GVecGen2i sqrshrunt_ops[3] = {
+    { .fno = gen_helper_sve2_sqrshrunt_h },
+    { .fno = gen_helper_sve2_sqrshrunt_s },
+    { .fno = gen_helper_sve2_sqrshrunt_d },
+};
+TRANS_FEAT(SQRSHRUNT, aa64_sve2, do_shr_narrow, a, sqrshrunt_ops)
 
 static void gen_sqshrnb_vec(unsigned vece, TCGv_vec d,
                             TCGv_vec n, int64_t shr)
@@ -6828,27 +6804,24 @@ static void gen_sqshrnb_vec(unsigned vece, TCGv_vec d,
     tcg_temp_free_vec(t);
 }
 
-static bool trans_SQSHRNB(DisasContext *s, arg_rri_esz *a)
-{
-    static const TCGOpcode vec_list[] = {
-        INDEX_op_sari_vec, INDEX_op_smax_vec, INDEX_op_smin_vec, 0
-    };
-    static const GVecGen2i ops[3] = {
-        { .fniv = gen_sqshrnb_vec,
-          .opt_opc = vec_list,
-          .fno = gen_helper_sve2_sqshrnb_h,
-          .vece = MO_16 },
-        { .fniv = gen_sqshrnb_vec,
-          .opt_opc = vec_list,
-          .fno = gen_helper_sve2_sqshrnb_s,
-          .vece = MO_32 },
-        { .fniv = gen_sqshrnb_vec,
-          .opt_opc = vec_list,
-          .fno = gen_helper_sve2_sqshrnb_d,
-          .vece = MO_64 },
-    };
-    return do_sve2_shr_narrow(s, a, ops);
-}
+static const TCGOpcode sqshrnb_vec_list[] = {
+    INDEX_op_sari_vec, INDEX_op_smax_vec, INDEX_op_smin_vec, 0
+};
+static const GVecGen2i sqshrnb_ops[3] = {
+    { .fniv = gen_sqshrnb_vec,
+      .opt_opc = sqshrnb_vec_list,
+      .fno = gen_helper_sve2_sqshrnb_h,
+      .vece = MO_16 },
+    { .fniv = gen_sqshrnb_vec,
+      .opt_opc = sqshrnb_vec_list,
+      .fno = gen_helper_sve2_sqshrnb_s,
+      .vece = MO_32 },
+    { .fniv = gen_sqshrnb_vec,
+      .opt_opc = sqshrnb_vec_list,
+      .fno = gen_helper_sve2_sqshrnb_d,
+      .vece = MO_64 },
+};
+TRANS_FEAT(SQSHRNB, aa64_sve2, do_shr_narrow, a, sqshrnb_ops)
 
 static void gen_sqshrnt_vec(unsigned vece, TCGv_vec d,
                              TCGv_vec n, int64_t shr)
@@ -6869,51 +6842,42 @@ static void gen_sqshrnt_vec(unsigned vece, TCGv_vec d,
     tcg_temp_free_vec(t);
 }
 
-static bool trans_SQSHRNT(DisasContext *s, arg_rri_esz *a)
-{
-    static const TCGOpcode vec_list[] = {
-        INDEX_op_shli_vec, INDEX_op_sari_vec,
-        INDEX_op_smax_vec, INDEX_op_smin_vec, 0
-    };
-    static const GVecGen2i ops[3] = {
-        { .fniv = gen_sqshrnt_vec,
-          .opt_opc = vec_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_sqshrnt_h,
-          .vece = MO_16 },
-        { .fniv = gen_sqshrnt_vec,
-          .opt_opc = vec_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_sqshrnt_s,
-          .vece = MO_32 },
-        { .fniv = gen_sqshrnt_vec,
-          .opt_opc = vec_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_sqshrnt_d,
-          .vece = MO_64 },
-    };
-    return do_sve2_shr_narrow(s, a, ops);
-}
+static const TCGOpcode sqshrnt_vec_list[] = {
+    INDEX_op_shli_vec, INDEX_op_sari_vec,
+    INDEX_op_smax_vec, INDEX_op_smin_vec, 0
+};
+static const GVecGen2i sqshrnt_ops[3] = {
+    { .fniv = gen_sqshrnt_vec,
+      .opt_opc = sqshrnt_vec_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_sqshrnt_h,
+      .vece = MO_16 },
+    { .fniv = gen_sqshrnt_vec,
+      .opt_opc = sqshrnt_vec_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_sqshrnt_s,
+      .vece = MO_32 },
+    { .fniv = gen_sqshrnt_vec,
+      .opt_opc = sqshrnt_vec_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_sqshrnt_d,
+      .vece = MO_64 },
+};
+TRANS_FEAT(SQSHRNT, aa64_sve2, do_shr_narrow, a, sqshrnt_ops)
 
-static bool trans_SQRSHRNB(DisasContext *s, arg_rri_esz *a)
-{
-    static const GVecGen2i ops[3] = {
-        { .fno = gen_helper_sve2_sqrshrnb_h },
-        { .fno = gen_helper_sve2_sqrshrnb_s },
-        { .fno = gen_helper_sve2_sqrshrnb_d },
-    };
-    return do_sve2_shr_narrow(s, a, ops);
-}
+static const GVecGen2i sqrshrnb_ops[3] = {
+    { .fno = gen_helper_sve2_sqrshrnb_h },
+    { .fno = gen_helper_sve2_sqrshrnb_s },
+    { .fno = gen_helper_sve2_sqrshrnb_d },
+};
+TRANS_FEAT(SQRSHRNB, aa64_sve2, do_shr_narrow, a, sqrshrnb_ops)
 
-static bool trans_SQRSHRNT(DisasContext *s, arg_rri_esz *a)
-{
-    static const GVecGen2i ops[3] = {
-        { .fno = gen_helper_sve2_sqrshrnt_h },
-        { .fno = gen_helper_sve2_sqrshrnt_s },
-        { .fno = gen_helper_sve2_sqrshrnt_d },
-    };
-    return do_sve2_shr_narrow(s, a, ops);
-}
+static const GVecGen2i sqrshrnt_ops[3] = {
+    { .fno = gen_helper_sve2_sqrshrnt_h },
+    { .fno = gen_helper_sve2_sqrshrnt_s },
+    { .fno = gen_helper_sve2_sqrshrnt_d },
+};
+TRANS_FEAT(SQRSHRNT, aa64_sve2, do_shr_narrow, a, sqrshrnt_ops)
 
 static void gen_uqshrnb_vec(unsigned vece, TCGv_vec d,
                             TCGv_vec n, int64_t shr)
@@ -6927,27 +6891,24 @@ static void gen_uqshrnb_vec(unsigned vece, TCGv_vec d,
     tcg_temp_free_vec(t);
 }
 
-static bool trans_UQSHRNB(DisasContext *s, arg_rri_esz *a)
-{
-    static const TCGOpcode vec_list[] = {
-        INDEX_op_shri_vec, INDEX_op_umin_vec, 0
-    };
-    static const GVecGen2i ops[3] = {
-        { .fniv = gen_uqshrnb_vec,
-          .opt_opc = vec_list,
-          .fno = gen_helper_sve2_uqshrnb_h,
-          .vece = MO_16 },
-        { .fniv = gen_uqshrnb_vec,
-          .opt_opc = vec_list,
-          .fno = gen_helper_sve2_uqshrnb_s,
-          .vece = MO_32 },
-        { .fniv = gen_uqshrnb_vec,
-          .opt_opc = vec_list,
-          .fno = gen_helper_sve2_uqshrnb_d,
-          .vece = MO_64 },
-    };
-    return do_sve2_shr_narrow(s, a, ops);
-}
+static const TCGOpcode uqshrnb_vec_list[] = {
+    INDEX_op_shri_vec, INDEX_op_umin_vec, 0
+};
+static const GVecGen2i uqshrnb_ops[3] = {
+    { .fniv = gen_uqshrnb_vec,
+      .opt_opc = uqshrnb_vec_list,
+      .fno = gen_helper_sve2_uqshrnb_h,
+      .vece = MO_16 },
+    { .fniv = gen_uqshrnb_vec,
+      .opt_opc = uqshrnb_vec_list,
+      .fno = gen_helper_sve2_uqshrnb_s,
+      .vece = MO_32 },
+    { .fniv = gen_uqshrnb_vec,
+      .opt_opc = uqshrnb_vec_list,
+      .fno = gen_helper_sve2_uqshrnb_d,
+      .vece = MO_64 },
+};
+TRANS_FEAT(UQSHRNB, aa64_sve2, do_shr_narrow, a, uqshrnb_ops)
 
 static void gen_uqshrnt_vec(unsigned vece, TCGv_vec d,
                             TCGv_vec n, int64_t shr)
@@ -6963,50 +6924,41 @@ static void gen_uqshrnt_vec(unsigned vece, TCGv_vec d,
     tcg_temp_free_vec(t);
 }
 
-static bool trans_UQSHRNT(DisasContext *s, arg_rri_esz *a)
-{
-    static const TCGOpcode vec_list[] = {
-        INDEX_op_shli_vec, INDEX_op_shri_vec, INDEX_op_umin_vec, 0
-    };
-    static const GVecGen2i ops[3] = {
-        { .fniv = gen_uqshrnt_vec,
-          .opt_opc = vec_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_uqshrnt_h,
-          .vece = MO_16 },
-        { .fniv = gen_uqshrnt_vec,
-          .opt_opc = vec_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_uqshrnt_s,
-          .vece = MO_32 },
-        { .fniv = gen_uqshrnt_vec,
-          .opt_opc = vec_list,
-          .load_dest = true,
-          .fno = gen_helper_sve2_uqshrnt_d,
-          .vece = MO_64 },
-    };
-    return do_sve2_shr_narrow(s, a, ops);
-}
+static const TCGOpcode uqshrnt_vec_list[] = {
+    INDEX_op_shli_vec, INDEX_op_shri_vec, INDEX_op_umin_vec, 0
+};
+static const GVecGen2i uqshrnt_ops[3] = {
+    { .fniv = gen_uqshrnt_vec,
+      .opt_opc = uqshrnt_vec_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_uqshrnt_h,
+      .vece = MO_16 },
+    { .fniv = gen_uqshrnt_vec,
+      .opt_opc = uqshrnt_vec_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_uqshrnt_s,
+      .vece = MO_32 },
+    { .fniv = gen_uqshrnt_vec,
+      .opt_opc = uqshrnt_vec_list,
+      .load_dest = true,
+      .fno = gen_helper_sve2_uqshrnt_d,
+      .vece = MO_64 },
+};
+TRANS_FEAT(UQSHRNT, aa64_sve2, do_shr_narrow, a, uqshrnt_ops)
 
-static bool trans_UQRSHRNB(DisasContext *s, arg_rri_esz *a)
-{
-    static const GVecGen2i ops[3] = {
-        { .fno = gen_helper_sve2_uqrshrnb_h },
-        { .fno = gen_helper_sve2_uqrshrnb_s },
-        { .fno = gen_helper_sve2_uqrshrnb_d },
-    };
-    return do_sve2_shr_narrow(s, a, ops);
-}
+static const GVecGen2i uqrshrnb_ops[3] = {
+    { .fno = gen_helper_sve2_uqrshrnb_h },
+    { .fno = gen_helper_sve2_uqrshrnb_s },
+    { .fno = gen_helper_sve2_uqrshrnb_d },
+};
+TRANS_FEAT(UQRSHRNB, aa64_sve2, do_shr_narrow, a, uqrshrnb_ops)
 
-static bool trans_UQRSHRNT(DisasContext *s, arg_rri_esz *a)
-{
-    static const GVecGen2i ops[3] = {
-        { .fno = gen_helper_sve2_uqrshrnt_h },
-        { .fno = gen_helper_sve2_uqrshrnt_s },
-        { .fno = gen_helper_sve2_uqrshrnt_d },
-    };
-    return do_sve2_shr_narrow(s, a, ops);
-}
+static const GVecGen2i uqrshrnt_ops[3] = {
+    { .fno = gen_helper_sve2_uqrshrnt_h },
+    { .fno = gen_helper_sve2_uqrshrnt_s },
+    { .fno = gen_helper_sve2_uqrshrnt_d },
+};
+TRANS_FEAT(UQRSHRNT, aa64_sve2, do_shr_narrow, a, uqrshrnt_ops)
 
 #define DO_SVE2_ZZZ_NARROW(NAME, name)                                    \
     static gen_helper_gvec_3 * const name##_fns[4] = {                    \

From 72c7f90621b46b6d7efb98fb64045638c7e3032f Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:19:04 -0700
Subject: [PATCH 696/935] target/arm: Use TRANS_FEAT for do_FMLAL_zzzw

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-112-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 26 ++++----------------------
 1 file changed, 4 insertions(+), 22 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 44af7530b6..57bff0d345 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -7149,33 +7149,15 @@ TRANS_FEAT(FLOGB, aa64_sve2, gen_gvec_fpst_arg_zpz, flogb_fns[a->esz],
 
 static bool do_FMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sub, bool sel)
 {
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
     return gen_gvec_ptr_zzzz(s, gen_helper_sve2_fmlal_zzzw_s,
                              a->rd, a->rn, a->rm, a->ra,
                              (sel << 1) | sub, cpu_env);
 }
 
-static bool trans_FMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_FMLAL_zzzw(s, a, false, false);
-}
-
-static bool trans_FMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_FMLAL_zzzw(s, a, false, true);
-}
-
-static bool trans_FMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_FMLAL_zzzw(s, a, true, false);
-}
-
-static bool trans_FMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a)
-{
-    return do_FMLAL_zzzw(s, a, true, true);
-}
+TRANS_FEAT(FMLALB_zzzw, aa64_sve2, do_FMLAL_zzzw, a, false, false)
+TRANS_FEAT(FMLALT_zzzw, aa64_sve2, do_FMLAL_zzzw, a, false, true)
+TRANS_FEAT(FMLSLB_zzzw, aa64_sve2, do_FMLAL_zzzw, a, true, false)
+TRANS_FEAT(FMLSLT_zzzw, aa64_sve2, do_FMLAL_zzzw, a, true, true)
 
 static bool do_FMLAL_zzxw(DisasContext *s, arg_rrxr_esz *a, bool sub, bool sel)
 {

From fc7c8829447b8b01ed8d33f01228562f8af36790 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:19:05 -0700
Subject: [PATCH 697/935] target/arm: Use TRANS_FEAT for do_FMLAL_zzxw

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-113-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 26 ++++----------------------
 1 file changed, 4 insertions(+), 22 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 57bff0d345..5fb66547ec 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -7161,33 +7161,15 @@ TRANS_FEAT(FMLSLT_zzzw, aa64_sve2, do_FMLAL_zzzw, a, true, true)
 
 static bool do_FMLAL_zzxw(DisasContext *s, arg_rrxr_esz *a, bool sub, bool sel)
 {
-    if (!dc_isar_feature(aa64_sve2, s)) {
-        return false;
-    }
     return gen_gvec_ptr_zzzz(s, gen_helper_sve2_fmlal_zzxw_s,
                              a->rd, a->rn, a->rm, a->ra,
                              (a->index << 2) | (sel << 1) | sub, cpu_env);
 }
 
-static bool trans_FMLALB_zzxw(DisasContext *s, arg_rrxr_esz *a)
-{
-    return do_FMLAL_zzxw(s, a, false, false);
-}
-
-static bool trans_FMLALT_zzxw(DisasContext *s, arg_rrxr_esz *a)
-{
-    return do_FMLAL_zzxw(s, a, false, true);
-}
-
-static bool trans_FMLSLB_zzxw(DisasContext *s, arg_rrxr_esz *a)
-{
-    return do_FMLAL_zzxw(s, a, true, false);
-}
-
-static bool trans_FMLSLT_zzxw(DisasContext *s, arg_rrxr_esz *a)
-{
-    return do_FMLAL_zzxw(s, a, true, true);
-}
+TRANS_FEAT(FMLALB_zzxw, aa64_sve2, do_FMLAL_zzxw, a, false, false)
+TRANS_FEAT(FMLALT_zzxw, aa64_sve2, do_FMLAL_zzxw, a, false, true)
+TRANS_FEAT(FMLSLB_zzxw, aa64_sve2, do_FMLAL_zzxw, a, true, false)
+TRANS_FEAT(FMLSLT_zzxw, aa64_sve2, do_FMLAL_zzxw, a, true, true)
 
 TRANS_FEAT(SMMLA, aa64_sve_i8mm, gen_gvec_ool_arg_zzzz,
            gen_helper_gvec_smmla_b, a, 0)

From 1402a6b88f8d7ec0591427d8e8221c6d18d16d9b Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:19:06 -0700
Subject: [PATCH 698/935] target/arm: Add sve feature check for remaining
 trans_* functions

For all remaining trans_* functions that do not already
have a check, add one now.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-114-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-sve.c | 177 ++++++++++++++++++++++++++++++++++---
 1 file changed, 163 insertions(+), 14 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 5fb66547ec..836511d719 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -1311,6 +1311,9 @@ TRANS_FEAT(INDEX_rr, aa64_sve, do_index, a->esz, a->rd,
 
 static bool trans_ADDVL(DisasContext *s, arg_ADDVL *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         TCGv_i64 rd = cpu_reg_sp(s, a->rd);
         TCGv_i64 rn = cpu_reg_sp(s, a->rn);
@@ -1321,6 +1324,9 @@ static bool trans_ADDVL(DisasContext *s, arg_ADDVL *a)
 
 static bool trans_ADDPL(DisasContext *s, arg_ADDPL *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         TCGv_i64 rd = cpu_reg_sp(s, a->rd);
         TCGv_i64 rn = cpu_reg_sp(s, a->rn);
@@ -1331,6 +1337,9 @@ static bool trans_ADDPL(DisasContext *s, arg_ADDPL *a)
 
 static bool trans_RDVL(DisasContext *s, arg_RDVL *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         TCGv_i64 reg = cpu_reg(s, a->rd);
         tcg_gen_movi_i64(reg, a->imm * vec_full_reg_size(s));
@@ -1451,6 +1460,9 @@ static bool trans_AND_pppp(DisasContext *s, arg_rprr_s *a)
         .prefer_i64 = TCG_TARGET_REG_BITS == 64,
     };
 
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (!a->s) {
         if (a->rn == a->rm) {
             if (a->pg == a->rn) {
@@ -1486,6 +1498,9 @@ static bool trans_BIC_pppp(DisasContext *s, arg_rprr_s *a)
         .prefer_i64 = TCG_TARGET_REG_BITS == 64,
     };
 
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (!a->s && a->pg == a->rn) {
         return gen_gvec_fn_ppp(s, tcg_gen_gvec_andc, a->rd, a->rn, a->rm);
     }
@@ -1514,6 +1529,9 @@ static bool trans_EOR_pppp(DisasContext *s, arg_rprr_s *a)
         .prefer_i64 = TCG_TARGET_REG_BITS == 64,
     };
 
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     /* Alias NOT (predicate) is EOR Pd.B, Pg/Z, Pn.B, Pg.B */
     if (!a->s && a->pg == a->rm) {
         return gen_gvec_fn_ppp(s, tcg_gen_gvec_andc, a->rd, a->pg, a->rn);
@@ -1523,7 +1541,7 @@ static bool trans_EOR_pppp(DisasContext *s, arg_rprr_s *a)
 
 static bool trans_SEL_pppp(DisasContext *s, arg_rprr_s *a)
 {
-    if (a->s) {
+    if (a->s || !dc_isar_feature(aa64_sve, s)) {
         return false;
     }
     if (sve_access_check(s)) {
@@ -1558,6 +1576,9 @@ static bool trans_ORR_pppp(DisasContext *s, arg_rprr_s *a)
         .prefer_i64 = TCG_TARGET_REG_BITS == 64,
     };
 
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (!a->s && a->pg == a->rn && a->rn == a->rm) {
         return do_mov_p(s, a->rd, a->rn);
     }
@@ -1585,6 +1606,10 @@ static bool trans_ORN_pppp(DisasContext *s, arg_rprr_s *a)
         .fno = gen_helper_sve_orn_pppp,
         .prefer_i64 = TCG_TARGET_REG_BITS == 64,
     };
+
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     return do_pppp_flags(s, a, &op);
 }
 
@@ -1609,6 +1634,10 @@ static bool trans_NOR_pppp(DisasContext *s, arg_rprr_s *a)
         .fno = gen_helper_sve_nor_pppp,
         .prefer_i64 = TCG_TARGET_REG_BITS == 64,
     };
+
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     return do_pppp_flags(s, a, &op);
 }
 
@@ -1633,6 +1662,10 @@ static bool trans_NAND_pppp(DisasContext *s, arg_rprr_s *a)
         .fno = gen_helper_sve_nand_pppp,
         .prefer_i64 = TCG_TARGET_REG_BITS == 64,
     };
+
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     return do_pppp_flags(s, a, &op);
 }
 
@@ -1642,6 +1675,9 @@ static bool trans_NAND_pppp(DisasContext *s, arg_rprr_s *a)
 
 static bool trans_PTEST(DisasContext *s, arg_PTEST *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         int nofs = pred_full_reg_offset(s, a->rn);
         int gofs = pred_full_reg_offset(s, a->pg);
@@ -1998,6 +2034,9 @@ static void do_sat_addsub_vec(DisasContext *s, int esz, int rd, int rn,
 
 static bool trans_CNT_r(DisasContext *s, arg_CNT_r *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         unsigned fullsz = vec_full_reg_size(s);
         unsigned numelem = decode_pred_count(fullsz, a->pat, a->esz);
@@ -2008,6 +2047,9 @@ static bool trans_CNT_r(DisasContext *s, arg_CNT_r *a)
 
 static bool trans_INCDEC_r(DisasContext *s, arg_incdec_cnt *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         unsigned fullsz = vec_full_reg_size(s);
         unsigned numelem = decode_pred_count(fullsz, a->pat, a->esz);
@@ -2021,6 +2063,9 @@ static bool trans_INCDEC_r(DisasContext *s, arg_incdec_cnt *a)
 
 static bool trans_SINCDEC_r_32(DisasContext *s, arg_incdec_cnt *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (!sve_access_check(s)) {
         return true;
     }
@@ -2045,6 +2090,9 @@ static bool trans_SINCDEC_r_32(DisasContext *s, arg_incdec_cnt *a)
 
 static bool trans_SINCDEC_r_64(DisasContext *s, arg_incdec_cnt *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (!sve_access_check(s)) {
         return true;
     }
@@ -2062,7 +2110,7 @@ static bool trans_SINCDEC_r_64(DisasContext *s, arg_incdec_cnt *a)
 
 static bool trans_INCDEC_v(DisasContext *s, arg_incdec2_cnt *a)
 {
-    if (a->esz == 0) {
+    if (a->esz == 0 || !dc_isar_feature(aa64_sve, s)) {
         return false;
     }
 
@@ -2085,7 +2133,7 @@ static bool trans_INCDEC_v(DisasContext *s, arg_incdec2_cnt *a)
 
 static bool trans_SINCDEC_v(DisasContext *s, arg_incdec2_cnt *a)
 {
-    if (a->esz == 0) {
+    if (a->esz == 0 || !dc_isar_feature(aa64_sve, s)) {
         return false;
     }
 
@@ -2126,6 +2174,10 @@ TRANS_FEAT(EOR_zzi, aa64_sve, do_zz_dbm, a, tcg_gen_gvec_xori)
 static bool trans_DUPM(DisasContext *s, arg_DUPM *a)
 {
     uint64_t imm;
+
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (!logic_imm_decode_wmask(&imm, extract32(a->dbm, 12, 1),
                                 extract32(a->dbm, 0, 6),
                                 extract32(a->dbm, 6, 6))) {
@@ -2171,7 +2223,7 @@ static void do_cpy_m(DisasContext *s, int esz, int rd, int rn, int pg,
 
 static bool trans_FCPY(DisasContext *s, arg_FCPY *a)
 {
-    if (a->esz == 0) {
+    if (a->esz == 0 || !dc_isar_feature(aa64_sve, s)) {
         return false;
     }
     if (sve_access_check(s)) {
@@ -2184,6 +2236,9 @@ static bool trans_FCPY(DisasContext *s, arg_FCPY *a)
 
 static bool trans_CPY_m_i(DisasContext *s, arg_rpri_esz *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         do_cpy_m(s, a->esz, a->rd, a->rn, a->pg, tcg_constant_i64(a->imm));
     }
@@ -2197,6 +2252,9 @@ static bool trans_CPY_z_i(DisasContext *s, arg_CPY_z_i *a)
         gen_helper_sve_cpy_z_s, gen_helper_sve_cpy_z_d,
     };
 
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         unsigned vsz = vec_full_reg_size(s);
         tcg_gen_gvec_2i_ool(vec_full_reg_offset(s, a->rd),
@@ -2250,6 +2308,9 @@ TRANS_FEAT(EXT_sve2, aa64_sve2, do_EXT, a->rd, a->rn, (a->rn + 1) % 32, a->imm)
 
 static bool trans_DUP_s(DisasContext *s, arg_DUP_s *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         unsigned vsz = vec_full_reg_size(s);
         tcg_gen_gvec_dup_i64(a->esz, vec_full_reg_offset(s, a->rd),
@@ -2260,6 +2321,9 @@ static bool trans_DUP_s(DisasContext *s, arg_DUP_s *a)
 
 static bool trans_DUP_x(DisasContext *s, arg_DUP_x *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if ((a->imm & 0x1f) == 0) {
         return false;
     }
@@ -2308,6 +2372,9 @@ static void do_insr_i64(DisasContext *s, arg_rrr_esz *a, TCGv_i64 val)
 
 static bool trans_INSR_f(DisasContext *s, arg_rrr_esz *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         TCGv_i64 t = tcg_temp_new_i64();
         tcg_gen_ld_i64(t, cpu_env, vec_reg_offset(s, a->rm, 0, MO_64));
@@ -2319,6 +2386,9 @@ static bool trans_INSR_f(DisasContext *s, arg_rrr_esz *a)
 
 static bool trans_INSR_r(DisasContext *s, arg_rrr_esz *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         do_insr_i64(s, a, cpu_reg(s, a->rm));
     }
@@ -2359,7 +2429,7 @@ static bool trans_UNPK(DisasContext *s, arg_UNPK *a)
         { gen_helper_sve_sunpk_d, gen_helper_sve_uunpk_d },
     };
 
-    if (a->esz == 0) {
+    if (a->esz == 0 || !dc_isar_feature(aa64_sve, s)) {
         return false;
     }
     if (sve_access_check(s)) {
@@ -2787,6 +2857,9 @@ TRANS_FEAT(LASTB_r, aa64_sve, do_last_general, a, true)
 
 static bool trans_CPY_m_r(DisasContext *s, arg_rpr_esz *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         do_cpy_m(s, a->esz, a->rd, a->rd, a->pg, cpu_reg_sp(s, a->rn));
     }
@@ -2795,6 +2868,9 @@ static bool trans_CPY_m_r(DisasContext *s, arg_rpr_esz *a)
 
 static bool trans_CPY_m_v(DisasContext *s, arg_rpr_esz *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         int ofs = vec_reg_offset(s, a->rn, 0, a->esz);
         TCGv_i64 t = load_esz(cpu_env, ofs, a->esz);
@@ -3102,6 +3178,9 @@ static void do_cntp(DisasContext *s, TCGv_i64 val, int esz, int pn, int pg)
 
 static bool trans_CNTP(DisasContext *s, arg_CNTP *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         do_cntp(s, cpu_reg(s, a->rd), a->esz, a->rn, a->pg);
     }
@@ -3110,6 +3189,9 @@ static bool trans_CNTP(DisasContext *s, arg_CNTP *a)
 
 static bool trans_INCDECP_r(DisasContext *s, arg_incdec_pred *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         TCGv_i64 reg = cpu_reg(s, a->rd);
         TCGv_i64 val = tcg_temp_new_i64();
@@ -3127,7 +3209,7 @@ static bool trans_INCDECP_r(DisasContext *s, arg_incdec_pred *a)
 
 static bool trans_INCDECP_z(DisasContext *s, arg_incdec2_pred *a)
 {
-    if (a->esz == 0) {
+    if (a->esz == 0 || !dc_isar_feature(aa64_sve, s)) {
         return false;
     }
     if (sve_access_check(s)) {
@@ -3144,6 +3226,9 @@ static bool trans_INCDECP_z(DisasContext *s, arg_incdec2_pred *a)
 
 static bool trans_SINCDECP_r_32(DisasContext *s, arg_incdec_pred *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         TCGv_i64 reg = cpu_reg(s, a->rd);
         TCGv_i64 val = tcg_temp_new_i64();
@@ -3156,6 +3241,9 @@ static bool trans_SINCDECP_r_32(DisasContext *s, arg_incdec_pred *a)
 
 static bool trans_SINCDECP_r_64(DisasContext *s, arg_incdec_pred *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         TCGv_i64 reg = cpu_reg(s, a->rd);
         TCGv_i64 val = tcg_temp_new_i64();
@@ -3168,7 +3256,7 @@ static bool trans_SINCDECP_r_64(DisasContext *s, arg_incdec_pred *a)
 
 static bool trans_SINCDECP_z(DisasContext *s, arg_incdec2_pred *a)
 {
-    if (a->esz == 0) {
+    if (a->esz == 0 || !dc_isar_feature(aa64_sve, s)) {
         return false;
     }
     if (sve_access_check(s)) {
@@ -3185,6 +3273,9 @@ static bool trans_SINCDECP_z(DisasContext *s, arg_incdec2_pred *a)
 
 static bool trans_CTERM(DisasContext *s, arg_CTERM *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (!sve_access_check(s)) {
         return true;
     }
@@ -3221,7 +3312,9 @@ static bool trans_WHILE(DisasContext *s, arg_WHILE *a)
     bool eq = a->eq == a->lt;
 
     /* The greater-than conditions are all SVE2. */
-    if (!a->lt && !dc_isar_feature(aa64_sve2, s)) {
+    if (a->lt
+        ? !dc_isar_feature(aa64_sve, s)
+        : !dc_isar_feature(aa64_sve2, s)) {
         return false;
     }
     if (!sve_access_check(s)) {
@@ -3389,7 +3482,7 @@ static bool trans_WHILE_ptr(DisasContext *s, arg_WHILE_ptr *a)
 
 static bool trans_FDUP(DisasContext *s, arg_FDUP *a)
 {
-    if (a->esz == 0) {
+    if (a->esz == 0 || !dc_isar_feature(aa64_sve, s)) {
         return false;
     }
     if (sve_access_check(s)) {
@@ -3406,6 +3499,9 @@ static bool trans_FDUP(DisasContext *s, arg_FDUP *a)
 
 static bool trans_DUP_i(DisasContext *s, arg_DUP_i *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         unsigned vsz = vec_full_reg_size(s);
         int dofs = vec_full_reg_offset(s, a->rd);
@@ -3453,6 +3549,9 @@ static bool trans_SUBR_zzi(DisasContext *s, arg_rri_esz *a)
           .scalar_first = true }
     };
 
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         unsigned vsz = vec_full_reg_size(s);
         tcg_gen_gvec_2s(vec_full_reg_offset(s, a->rd),
@@ -3815,7 +3914,7 @@ static bool trans_FADDA(DisasContext *s, arg_rprr_esz *a)
     TCGv_i64 t_val;
     TCGv_i32 t_desc;
 
-    if (a->esz == 0) {
+    if (a->esz == 0 || !dc_isar_feature(aa64_sve, s)) {
         return false;
     }
     if (!sve_access_check(s)) {
@@ -4367,6 +4466,9 @@ static void do_str(DisasContext *s, uint32_t vofs, int len, int rn, int imm)
 
 static bool trans_LDR_zri(DisasContext *s, arg_rri *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         int size = vec_full_reg_size(s);
         int off = vec_full_reg_offset(s, a->rd);
@@ -4377,6 +4479,9 @@ static bool trans_LDR_zri(DisasContext *s, arg_rri *a)
 
 static bool trans_LDR_pri(DisasContext *s, arg_rri *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         int size = pred_full_reg_size(s);
         int off = pred_full_reg_offset(s, a->rd);
@@ -4387,6 +4492,9 @@ static bool trans_LDR_pri(DisasContext *s, arg_rri *a)
 
 static bool trans_STR_zri(DisasContext *s, arg_rri *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         int size = vec_full_reg_size(s);
         int off = vec_full_reg_offset(s, a->rd);
@@ -4397,6 +4505,9 @@ static bool trans_STR_zri(DisasContext *s, arg_rri *a)
 
 static bool trans_STR_pri(DisasContext *s, arg_rri *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         int size = pred_full_reg_size(s);
         int off = pred_full_reg_offset(s, a->rd);
@@ -4597,7 +4708,7 @@ static void do_ld_zpa(DisasContext *s, int zt, int pg,
 
 static bool trans_LD_zprr(DisasContext *s, arg_rprr_load *a)
 {
-    if (a->rm == 31) {
+    if (a->rm == 31 || !dc_isar_feature(aa64_sve, s)) {
         return false;
     }
     if (sve_access_check(s)) {
@@ -4611,6 +4722,9 @@ static bool trans_LD_zprr(DisasContext *s, arg_rprr_load *a)
 
 static bool trans_LD_zpri(DisasContext *s, arg_rpri_load *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         int vsz = vec_full_reg_size(s);
         int elements = vsz >> dtype_esz[a->dtype];
@@ -4712,6 +4826,9 @@ static bool trans_LDFF1_zprr(DisasContext *s, arg_rprr_load *a)
             gen_helper_sve_ldff1dd_be_r_mte } },
     };
 
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         TCGv_i64 addr = new_tmp_a64(s);
         tcg_gen_shli_i64(addr, cpu_reg(s, a->rm), dtype_msz(a->dtype));
@@ -4810,6 +4927,9 @@ static bool trans_LDNF1_zpri(DisasContext *s, arg_rpri_load *a)
             gen_helper_sve_ldnf1dd_be_r_mte } },
     };
 
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         int vsz = vec_full_reg_size(s);
         int elements = vsz >> dtype_esz[a->dtype];
@@ -4867,7 +4987,7 @@ static void do_ldrq(DisasContext *s, int zt, int pg, TCGv_i64 addr, int dtype)
 
 static bool trans_LD1RQ_zprr(DisasContext *s, arg_rprr_load *a)
 {
-    if (a->rm == 31) {
+    if (a->rm == 31 || !dc_isar_feature(aa64_sve, s)) {
         return false;
     }
     if (sve_access_check(s)) {
@@ -4882,6 +5002,9 @@ static bool trans_LD1RQ_zprr(DisasContext *s, arg_rprr_load *a)
 
 static bool trans_LD1RQ_zpri(DisasContext *s, arg_rpri_load *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (sve_access_check(s)) {
         TCGv_i64 addr = new_tmp_a64(s);
         tcg_gen_addi_i64(addr, cpu_reg_sp(s, a->rn), a->imm * 16);
@@ -4993,6 +5116,9 @@ static bool trans_LD1R_zpri(DisasContext *s, arg_rpri_load *a)
     TCGLabel *over;
     TCGv_i64 temp, clean_addr;
 
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (!sve_access_check(s)) {
         return true;
     }
@@ -5161,6 +5287,9 @@ static void do_st_zpa(DisasContext *s, int zt, int pg, TCGv_i64 addr,
 
 static bool trans_ST_zprr(DisasContext *s, arg_rprr_store *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (a->rm == 31 || a->msz > a->esz) {
         return false;
     }
@@ -5175,6 +5304,9 @@ static bool trans_ST_zprr(DisasContext *s, arg_rprr_store *a)
 
 static bool trans_ST_zpri(DisasContext *s, arg_rpri_store *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (a->msz > a->esz) {
         return false;
     }
@@ -5558,6 +5690,9 @@ static bool trans_LD1_zprz(DisasContext *s, arg_LD1_zprz *a)
     bool be = s->be_data == MO_BE;
     bool mte = s->mte_active[0];
 
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (!sve_access_check(s)) {
         return true;
     }
@@ -5586,6 +5721,9 @@ static bool trans_LD1_zpiz(DisasContext *s, arg_LD1_zpiz *a)
     if (a->esz < a->msz || (a->esz == a->msz && !a->u)) {
         return false;
     }
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (!sve_access_check(s)) {
         return true;
     }
@@ -5740,6 +5878,9 @@ static bool trans_ST1_zprz(DisasContext *s, arg_ST1_zprz *a)
     if (a->esz < a->msz || (a->msz == 0 && a->scale)) {
         return false;
     }
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (!sve_access_check(s)) {
         return true;
     }
@@ -5767,6 +5908,9 @@ static bool trans_ST1_zpiz(DisasContext *s, arg_ST1_zpiz *a)
     if (a->esz < a->msz) {
         return false;
     }
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     if (!sve_access_check(s)) {
         return true;
     }
@@ -5827,6 +5971,9 @@ static bool trans_STNT1_zprz(DisasContext *s, arg_ST1_zprz *a)
 
 static bool trans_PRF(DisasContext *s, arg_PRF *a)
 {
+    if (!dc_isar_feature(aa64_sve, s)) {
+        return false;
+    }
     /* Prefetch is a nop within QEMU.  */
     (void)sve_access_check(s);
     return true;
@@ -5834,7 +5981,7 @@ static bool trans_PRF(DisasContext *s, arg_PRF *a)
 
 static bool trans_PRF_rr(DisasContext *s, arg_PRF_rr *a)
 {
-    if (a->rm == 31) {
+    if (a->rm == 31 || !dc_isar_feature(aa64_sve, s)) {
         return false;
     }
     /* Prefetch is a nop within QEMU.  */
@@ -6070,7 +6217,9 @@ static bool do_trans_pmull(DisasContext *s, arg_rrr_esz *a, bool sel)
         gen_helper_gvec_pmull_q, gen_helper_sve2_pmull_h,
         NULL,                    gen_helper_sve2_pmull_d,
     };
-    if (a->esz == 0 && !dc_isar_feature(aa64_sve2_pmull128, s)) {
+    if (a->esz == 0
+        ? !dc_isar_feature(aa64_sve2_pmull128, s)
+        : !dc_isar_feature(aa64_sve, s)) {
         return false;
     }
     return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, sel);

From b1071174d2a2ab371082b7d4b5f19e98edc61ac6 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Fri, 27 May 2022 11:19:07 -0700
Subject: [PATCH 699/935] target/arm: Remove aa64_sve check from before
 disas_sve

We now have individual checks on all insns within disas_sve.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220527181907.189259-115-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/translate-a64.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index f502545307..935e1929bb 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -14772,7 +14772,7 @@ static void aarch64_tr_translate_insn(DisasContextBase *dcbase, CPUState *cpu)
         unallocated_encoding(s);
         break;
     case 0x2:
-        if (!dc_isar_feature(aa64_sve, s) || !disas_sve(s, insn)) {
+        if (!disas_sve(s, insn)) {
             unallocated_encoding(s);
         }
         break;

From ee344768c1a19417e005bd96c2c6e256986380d5 Mon Sep 17 00:00:00 2001
From: "Maciej S. Szmigiero" <maciej.szmigiero@oracle.com>
Date: Sat, 20 Nov 2021 19:41:58 +0100
Subject: [PATCH 700/935] MAINTAINERS: Add myself as the maintainer for Hyper-V
 VMBus

This way there is some contact point for incoming patches,
and somebody to review and pick up them.

Signed-off-by: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
---
 MAINTAINERS | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index dff0200f70..00dc4a8ecb 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1777,6 +1777,12 @@ F: include/hw/block/fdc.h
 F: tests/qtest/fdc-test.c
 T: git https://gitlab.com/jsnow/qemu.git ide
 
+Hyper-V VMBus
+M: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
+S: Odd Fixes
+F: hw/hyperv/vmbus.c
+F: include/hw/hyperv/vmbus*.h
+
 OMAP
 M: Peter Maydell <peter.maydell@linaro.org>
 L: qemu-arm@nongnu.org

From 6ede46b910ac66fd10bc169fb0a6f681429a9c5c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= <philmd@redhat.com>
Date: Sat, 6 Nov 2021 14:41:55 +0100
Subject: [PATCH 701/935] hw/hyperv/vmbus: Remove unused vmbus_load/save_req()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

vmbus_save_req() and vmbus_load_req() are not used.
Remove them to avoid maintaining dead code.

This essentially reverts commit 4dd8a7064b8a6527f99a62be11
("vmbus: add infrastructure to save/load vmbus requests").

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20211106134155.582312-2-philmd@redhat.com>
[MSS: Remove also corresponding variables, which are now unused]
Signed-off-by: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
---
 hw/hyperv/vmbus.c         | 99 ---------------------------------------
 include/hw/hyperv/vmbus.h |  3 --
 2 files changed, 102 deletions(-)

diff --git a/hw/hyperv/vmbus.c b/hw/hyperv/vmbus.c
index 8aad29f1bb..30bc04e1c4 100644
--- a/hw/hyperv/vmbus.c
+++ b/hw/hyperv/vmbus.c
@@ -1273,105 +1273,6 @@ void vmbus_free_req(void *req)
     g_free(req);
 }
 
-static const VMStateDescription vmstate_sgent = {
-    .name = "vmbus/sgentry",
-    .version_id = 0,
-    .minimum_version_id = 0,
-    .fields = (VMStateField[]) {
-        VMSTATE_UINT64(base, ScatterGatherEntry),
-        VMSTATE_UINT64(len, ScatterGatherEntry),
-        VMSTATE_END_OF_LIST()
-    }
-};
-
-typedef struct VMBusChanReqSave {
-    uint16_t chan_idx;
-    uint16_t pkt_type;
-    uint32_t msglen;
-    void *msg;
-    uint64_t transaction_id;
-    bool need_comp;
-    uint32_t num;
-    ScatterGatherEntry *sgl;
-} VMBusChanReqSave;
-
-static const VMStateDescription vmstate_vmbus_chan_req = {
-    .name = "vmbus/vmbus_chan_req",
-    .version_id = 0,
-    .minimum_version_id = 0,
-    .fields = (VMStateField[]) {
-        VMSTATE_UINT16(chan_idx, VMBusChanReqSave),
-        VMSTATE_UINT16(pkt_type, VMBusChanReqSave),
-        VMSTATE_UINT32(msglen, VMBusChanReqSave),
-        VMSTATE_VBUFFER_ALLOC_UINT32(msg, VMBusChanReqSave, 0, NULL, msglen),
-        VMSTATE_UINT64(transaction_id, VMBusChanReqSave),
-        VMSTATE_BOOL(need_comp, VMBusChanReqSave),
-        VMSTATE_UINT32(num, VMBusChanReqSave),
-        VMSTATE_STRUCT_VARRAY_POINTER_UINT32(sgl, VMBusChanReqSave, num,
-                                             vmstate_sgent, ScatterGatherEntry),
-        VMSTATE_END_OF_LIST()
-    }
-};
-
-void vmbus_save_req(QEMUFile *f, VMBusChanReq *req)
-{
-    VMBusChanReqSave req_save;
-
-    req_save.chan_idx = req->chan->subchan_idx;
-    req_save.pkt_type = req->pkt_type;
-    req_save.msglen = req->msglen;
-    req_save.msg = req->msg;
-    req_save.transaction_id = req->transaction_id;
-    req_save.need_comp = req->need_comp;
-    req_save.num = req->sgl.nsg;
-    req_save.sgl = g_memdup(req->sgl.sg,
-                            req_save.num * sizeof(ScatterGatherEntry));
-
-    vmstate_save_state(f, &vmstate_vmbus_chan_req, &req_save, NULL);
-
-    g_free(req_save.sgl);
-}
-
-void *vmbus_load_req(QEMUFile *f, VMBusDevice *dev, uint32_t size)
-{
-    VMBusChanReqSave req_save;
-    VMBusChanReq *req = NULL;
-    VMBusChannel *chan = NULL;
-    uint32_t i;
-
-    vmstate_load_state(f, &vmstate_vmbus_chan_req, &req_save, 0);
-
-    if (req_save.chan_idx >= dev->num_channels) {
-        error_report("%s: %u(chan_idx) > %u(num_channels)", __func__,
-                     req_save.chan_idx, dev->num_channels);
-        goto out;
-    }
-    chan = &dev->channels[req_save.chan_idx];
-
-    if (vmbus_channel_reserve(chan, 0, req_save.msglen)) {
-        goto out;
-    }
-
-    req = vmbus_alloc_req(chan, size, req_save.pkt_type, req_save.msglen,
-                          req_save.transaction_id, req_save.need_comp);
-    if (req_save.msglen) {
-        memcpy(req->msg, req_save.msg, req_save.msglen);
-    }
-
-    for (i = 0; i < req_save.num; i++) {
-        qemu_sglist_add(&req->sgl, req_save.sgl[i].base, req_save.sgl[i].len);
-    }
-
-out:
-    if (req_save.msglen) {
-        g_free(req_save.msg);
-    }
-    if (req_save.num) {
-        g_free(req_save.sgl);
-    }
-    return req;
-}
-
 static void channel_event_cb(EventNotifier *e)
 {
     VMBusChannel *chan = container_of(e, VMBusChannel, notifier);
diff --git a/include/hw/hyperv/vmbus.h b/include/hw/hyperv/vmbus.h
index f98bea3888..8ea660dd8e 100644
--- a/include/hw/hyperv/vmbus.h
+++ b/include/hw/hyperv/vmbus.h
@@ -223,7 +223,4 @@ int vmbus_map_sgl(VMBusChanReq *req, DMADirection dir, struct iovec *iov,
 void vmbus_unmap_sgl(VMBusChanReq *req, DMADirection dir, struct iovec *iov,
                      unsigned iov_cnt, size_t accessed);
 
-void vmbus_save_req(QEMUFile *f, VMBusChanReq *req);
-void *vmbus_load_req(QEMUFile *f, VMBusDevice *dev, uint32_t size);
-
 #endif

From 658f2da1f33aaf86d7b345891f479462a7d1e2d3 Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Fri, 27 May 2022 16:35:31 +0100
Subject: [PATCH 702/935] .gitlab-ci.d/container-cross: Fix RISC-V container
 dependencies / stages
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The "riscv64-debian-cross-container" job does not depend on any other
container job from the first stage, so we can move it to the first
stage, too.

The "riscv64-debian-test-cross-container" job needs the debian11
container, so we should add a proper "needs:" statement here.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220524093141.91012-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20220527153603.887929-2-alex.bennee@linaro.org>
---
 .gitlab-ci.d/container-cross.yml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index e622ac2d21..ac15fce9b6 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -125,7 +125,7 @@ ppc64el-debian-cross-container:
 
 riscv64-debian-cross-container:
   extends: .container_job_template
-  stage: containers-layer2
+  stage: containers
   # as we are currently based on 'sid/unstable' we may break so...
   allow_failure: true
   variables:
@@ -135,6 +135,7 @@ riscv64-debian-cross-container:
 riscv64-debian-test-cross-container:
   extends: .container_job_template
   stage: containers-layer2
+  needs: ['amd64-debian11-container']
   variables:
     NAME: debian-riscv64-test-cross
 

From 6fd4e75610c08820faabf9c544f2f2af2b1b8334 Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Fri, 27 May 2022 16:35:32 +0100
Subject: [PATCH 703/935] .gitlab-ci.d/crossbuilds: Fix the dependency of the
 cross-i386-tci job
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The cross-i386-tci job uses the fedora-i386-cross image, so we should make sure
that the corresponding job that builds it (the i386-fedora-cross-container job)
has finished before we start the TCI job.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220524092600.89997-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220527153603.887929-3-alex.bennee@linaro.org>
---
 .gitlab-ci.d/crossbuilds.yml | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
index 17d6cb3e45..4a5fb6ea2a 100644
--- a/.gitlab-ci.d/crossbuilds.yml
+++ b/.gitlab-ci.d/crossbuilds.yml
@@ -62,6 +62,8 @@ cross-i386-user:
 cross-i386-tci:
   extends: .cross_accel_build_job
   timeout: 60m
+  needs:
+    job: i386-fedora-cross-container
   variables:
     IMAGE: fedora-i386-cross
     ACCEL: tcg-interpreter

From 861dca3707a73cbe022cefe52f7224b28350316b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= <marcandre.lureau@redhat.com>
Date: Fri, 27 May 2022 16:35:33 +0100
Subject: [PATCH 704/935] gitlab-ci: add meson JUnit test result into report
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This allows the gitlab UI to show the test results in different ways,
see doc:

  https://docs.gitlab.com/ee/ci/unit_test_reports.html#how-it-works

Previous we only reports avocado test results (.avocado_test_job_template),
with this change, the qemu/meson tests are also covered.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20220525173411.612224-1-marcandre.lureau@redhat.com>
[AJB: expand the commit description]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220527153603.887929-4-alex.bennee@linaro.org>
---
 .gitlab-ci.d/buildtest-template.yml | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index dc6d67aacf..b381345dbc 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -44,6 +44,8 @@
     expire_in: 7 days
     paths:
       - build/meson-logs/testlog.txt
+    reports:
+      junit: build/meson-logs/testlog.junit.xml
 
 .avocado_test_job_template:
   extends: .common_test_job_template

From 85b141ea6ae448b718d438d9cf0009d73116f8d1 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Fri, 27 May 2022 16:35:34 +0100
Subject: [PATCH 705/935] meson.build: fix summary display of test compilers
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The recent refactoring of configure.sh dropped a number of variables
we relied on for printing out information. Make it simpler.

Fixes: eebf199c09 (tests/tcg: invoke Makefile.target directly from QEMU's makefile)
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220527153603.887929-5-alex.bennee@linaro.org>
---
 meson.build | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/meson.build b/meson.build
index bf318d9cbb..bc6234c85e 100644
--- a/meson.build
+++ b/meson.build
@@ -3735,12 +3735,8 @@ foreach target: target_dirs
     config_cross_tcg = keyval.load(tcg_mak)
     target = config_cross_tcg['TARGET_NAME']
     compiler = ''
-    if 'DOCKER_CROSS_CC_GUEST' in config_cross_tcg
-      summary_info += {target + ' tests': config_cross_tcg['DOCKER_CROSS_CC_GUEST'] +
-                                          ' via ' + config_cross_tcg['DOCKER_IMAGE']}
-    elif 'CROSS_CC_GUEST' in config_cross_tcg
-      summary_info += {target + ' tests'
-                                : config_cross_tcg['CROSS_CC_GUEST'] }
+    if 'CC' in config_cross_tcg
+      summary_info += {target + ' tests': config_cross_tcg['CC']}
     endif
    endif
 endforeach

From 06885cf93556adc3fd67aad0c279a0deb400bb7d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Fri, 27 May 2022 16:35:35 +0100
Subject: [PATCH 706/935] tests/lcitool: fix up indentation to correct style
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

3 space indentation snuck into the initial commit. Clean it up before
we let it get established. I've also:

  - removed unused os import
  - added double lines between functions
  - added some comments and grouped and sorted the generation stanzas

My lint tool is also recommending using f-strings but that requires
python 3.6.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-6-alex.bennee@linaro.org>
---
 tests/lcitool/refresh | 128 ++++++++++++++++++++++++------------------
 1 file changed, 73 insertions(+), 55 deletions(-)

diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index fb49bbc441..dc1fc21ef9 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -13,14 +13,13 @@
 # the top-level directory.
 
 import sys
-import os
 import subprocess
 
 from pathlib import Path
 
 if len(sys.argv) != 1:
-   print("syntax: %s" % sys.argv[0], file=sys.stderr)
-   sys.exit(1)
+    print("syntax: %s" % sys.argv[0], file=sys.stderr)
+    sys.exit(1)
 
 self_dir = Path(__file__).parent
 src_dir = self_dir.parent.parent
@@ -30,76 +29,95 @@ lcitool_path = Path(self_dir, "libvirt-ci", "lcitool")
 
 lcitool_cmd = [lcitool_path, "--data-dir", self_dir]
 
+
 def atomic_write(filename, content):
-   tmp = filename.with_suffix(filename.suffix + ".tmp")
-   try:
-      with tmp.open("w") as fp:
-         print(content, file=fp, end="")
-         tmp.rename(filename)
-   except Exception as ex:
-      tmp.unlink()
-      raise
+    tmp = filename.with_suffix(filename.suffix + ".tmp")
+    try:
+        with tmp.open("w") as fp:
+            print(content, file=fp, end="")
+            tmp.rename(filename)
+    except Exception as ex:
+        tmp.unlink()
+        raise
+
 
 def generate(filename, cmd, trailer):
-   print("Generate %s" % filename)
-   lcitool=subprocess.run(cmd, capture_output=True)
+    print("Generate %s" % filename)
+    lcitool = subprocess.run(cmd, capture_output=True)
 
-   if lcitool.returncode != 0:
-      raise Exception("Failed to generate %s: %s" % (filename, lcitool.stderr))
+    if lcitool.returncode != 0:
+        raise Exception("Failed to generate %s: %s" % (filename, lcitool.stderr))
+
+    content = lcitool.stdout.decode("utf8")
+    if trailer is not None:
+        content += trailer
+    atomic_write(filename, content)
 
-   content = lcitool.stdout.decode("utf8")
-   if trailer is not None:
-      content += trailer
-   atomic_write(filename, content)
 
 def generate_dockerfile(host, target, cross=None, trailer=None):
-   filename = Path(src_dir, "tests", "docker", "dockerfiles", host + ".docker")
-   cmd = lcitool_cmd + ["dockerfile"]
-   if cross is not None:
-      cmd.extend(["--cross", cross])
-   cmd.extend([target, "qemu"])
-   generate(filename, cmd, trailer)
+    filename = Path(src_dir, "tests", "docker", "dockerfiles", host + ".docker")
+    cmd = lcitool_cmd + ["dockerfile"]
+    if cross is not None:
+        cmd.extend(["--cross", cross])
+    cmd.extend([target, "qemu"])
+    generate(filename, cmd, trailer)
+
 
 def generate_cirrus(target, trailer=None):
-   filename = Path(src_dir, ".gitlab-ci.d", "cirrus", target + ".vars")
-   cmd = lcitool_cmd + ["variables", target, "qemu"]
-   generate(filename, cmd, trailer)
+    filename = Path(src_dir, ".gitlab-ci.d", "cirrus", target + ".vars")
+    cmd = lcitool_cmd + ["variables", target, "qemu"]
+    generate(filename, cmd, trailer)
+
 
 ubuntu2004_tsanhack = [
-   "# Apply patch https://reviews.llvm.org/D75820\n",
-   "# This is required for TSan in clang-10 to compile with QEMU.\n",
-   "RUN sed -i 's/^const/static const/g' /usr/lib/llvm-10/lib/clang/10.0.0/include/sanitizer/tsan_interface.h\n"
+    "# Apply patch https://reviews.llvm.org/D75820\n",
+    "# This is required for TSan in clang-10 to compile with QEMU.\n",
+    "RUN sed -i 's/^const/static const/g' /usr/lib/llvm-10/lib/clang/10.0.0/include/sanitizer/tsan_interface.h\n"
 ]
 
-def debian_cross_build(prefix, targets):
-   conf = "ENV QEMU_CONFIGURE_OPTS --cross-prefix=%s\n" % (prefix)
-   targets = "ENV DEF_TARGET_LIST %s\n" % (targets)
-   return "".join([conf, targets])
 
+def debian_cross_build(prefix, targets):
+    conf = "ENV QEMU_CONFIGURE_OPTS --cross-prefix=%s\n" % (prefix)
+    targets = "ENV DEF_TARGET_LIST %s\n" % (targets)
+    return "".join([conf, targets])
+
+#
+# Update all the various build configurations.
+# Please keep each group sorted alphabetically for easy reading.
+#
 
 try:
-   generate_dockerfile("centos8", "centos-stream-8")
-   generate_dockerfile("fedora", "fedora-35")
-   generate_dockerfile("ubuntu2004", "ubuntu-2004",
-                       trailer="".join(ubuntu2004_tsanhack))
-   generate_dockerfile("opensuse-leap", "opensuse-leap-152")
-   generate_dockerfile("alpine", "alpine-edge")
+    #
+    # Standard native builds
+    #
+    generate_dockerfile("alpine", "alpine-edge")
+    generate_dockerfile("centos8", "centos-stream-8")
+    generate_dockerfile("fedora", "fedora-35")
+    generate_dockerfile("opensuse-leap", "opensuse-leap-152")
+    generate_dockerfile("ubuntu2004", "ubuntu-2004",
+                        trailer="".join(ubuntu2004_tsanhack))
 
-   generate_dockerfile("debian-arm64-cross", "debian-11",
-                       cross="aarch64",
-                       trailer=debian_cross_build("aarch64-linux-gnu-",
-                                                  "aarch64-softmmu,aarch64-linux-user"))
+    #
+    # Cross compiling builds
+    #
+    generate_dockerfile("debian-arm64-cross", "debian-11",
+                        cross="aarch64",
+                        trailer=debian_cross_build("aarch64-linux-gnu-",
+                                                   "aarch64-softmmu,aarch64-linux-user"))
 
-   generate_dockerfile("debian-s390x-cross", "debian-11",
-                       cross="s390x",
-                       trailer=debian_cross_build("s390x-linux-gnu-",
-                                                  "s390x-softmmu,s390x-linux-user"))
+    generate_dockerfile("debian-s390x-cross", "debian-11",
+                        cross="s390x",
+                        trailer=debian_cross_build("s390x-linux-gnu-",
+                                                   "s390x-softmmu,s390x-linux-user"))
 
-   generate_cirrus("freebsd-12")
-   generate_cirrus("freebsd-13")
-   generate_cirrus("macos-11")
+    #
+    # Cirrus packages lists for GitLab
+    #
+    generate_cirrus("freebsd-12")
+    generate_cirrus("freebsd-13")
+    generate_cirrus("macos-11")
 
-   sys.exit(0)
+    sys.exit(0)
 except Exception as ex:
-   print(str(ex), file=sys.stderr)
-   sys.exit(1)
+    print(str(ex), file=sys.stderr)
+    sys.exit(1)

From 18ad049d90feede8affee797822fca41d1472b71 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Fri, 27 May 2022 16:35:36 +0100
Subject: [PATCH 707/935] tests/docker: update debian-armhf-cross with lcitool
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Use lcitool to update debian-armhf-cross to a Debian 11 based system.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-7-alex.bennee@linaro.org>
---
 .gitlab-ci.d/container-cross.yml              |   3 +-
 tests/docker/Makefile.include                 |   1 -
 .../dockerfiles/debian-armhf-cross.docker     | 184 +++++++++++++++---
 tests/lcitool/refresh                         |   5 +
 4 files changed, 166 insertions(+), 27 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index ac15fce9b6..4d1830f3fc 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -34,8 +34,7 @@ armel-debian-cross-container:
 
 armhf-debian-cross-container:
   extends: .container_job_template
-  stage: containers-layer2
-  needs: ['amd64-debian10-container']
+  stage: containers
   variables:
     NAME: debian-armhf-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index ca2157db46..d6e0710554 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -90,7 +90,6 @@ endif
 
 docker-image-debian-alpha-cross: docker-image-debian10
 docker-image-debian-armel-cross: docker-image-debian10
-docker-image-debian-armhf-cross: docker-image-debian10
 docker-image-debian-hppa-cross: docker-image-debian10
 docker-image-debian-m68k-cross: docker-image-debian10
 docker-image-debian-mips-cross: docker-image-debian10
diff --git a/tests/docker/dockerfiles/debian-armhf-cross.docker b/tests/docker/dockerfiles/debian-armhf-cross.docker
index 25d7618833..a2ebce96f8 100644
--- a/tests/docker/dockerfiles/debian-armhf-cross.docker
+++ b/tests/docker/dockerfiles/debian-armhf-cross.docker
@@ -1,29 +1,165 @@
+# THIS FILE WAS AUTO-GENERATED
 #
-# Docker armhf cross-compiler target
+#  $ lcitool dockerfile --layers all --cross armv7l debian-11 qemu
 #
-# This docker target builds on the debian Stretch base image.
-#
-FROM qemu/debian10
+# https://gitlab.com/libvirt/libvirt-ci
 
-# Add the foreign architecture we want and install dependencies
-RUN dpkg --add-architecture armhf
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt install -y --no-install-recommends \
-        crossbuild-essential-armhf
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt build-dep -yy -a armhf --arch-only qemu
+FROM docker.io/library/debian:11-slim
 
-# Specify the cross prefix for this image (see tests/docker/common.rc)
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    apt-get update && \
+    apt-get install -y eatmydata && \
+    eatmydata apt-get dist-upgrade -y && \
+    eatmydata apt-get install --no-install-recommends -y \
+            bash \
+            bc \
+            bsdextrautils \
+            bzip2 \
+            ca-certificates \
+            ccache \
+            dbus \
+            debianutils \
+            diffutils \
+            exuberant-ctags \
+            findutils \
+            gcovr \
+            genisoimage \
+            gettext \
+            git \
+            hostname \
+            libpcre2-dev \
+            libspice-protocol-dev \
+            llvm \
+            locales \
+            make \
+            meson \
+            ncat \
+            ninja-build \
+            openssh-client \
+            perl-base \
+            pkgconf \
+            python3 \
+            python3-numpy \
+            python3-opencv \
+            python3-pillow \
+            python3-pip \
+            python3-sphinx \
+            python3-sphinx-rtd-theme \
+            python3-venv \
+            python3-yaml \
+            rpm2cpio \
+            sed \
+            sparse \
+            tar \
+            tesseract-ocr \
+            tesseract-ocr-eng \
+            texinfo && \
+    eatmydata apt-get autoremove -y && \
+    eatmydata apt-get autoclean -y && \
+    sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+    dpkg-reconfigure locales
+
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
+
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    dpkg --add-architecture armhf && \
+    eatmydata apt-get update && \
+    eatmydata apt-get dist-upgrade -y && \
+    eatmydata apt-get install --no-install-recommends -y dpkg-dev && \
+    eatmydata apt-get install --no-install-recommends -y \
+            g++-arm-linux-gnueabihf \
+            gcc-arm-linux-gnueabihf \
+            libaio-dev:armhf \
+            libasan5:armhf \
+            libasound2-dev:armhf \
+            libattr1-dev:armhf \
+            libbpf-dev:armhf \
+            libbrlapi-dev:armhf \
+            libbz2-dev:armhf \
+            libc6-dev:armhf \
+            libcacard-dev:armhf \
+            libcap-ng-dev:armhf \
+            libcapstone-dev:armhf \
+            libcurl4-gnutls-dev:armhf \
+            libdaxctl-dev:armhf \
+            libdrm-dev:armhf \
+            libepoxy-dev:armhf \
+            libfdt-dev:armhf \
+            libffi-dev:armhf \
+            libfuse3-dev:armhf \
+            libgbm-dev:armhf \
+            libgcrypt20-dev:armhf \
+            libglib2.0-dev:armhf \
+            libglusterfs-dev:armhf \
+            libgnutls28-dev:armhf \
+            libgtk-3-dev:armhf \
+            libibumad-dev:armhf \
+            libibverbs-dev:armhf \
+            libiscsi-dev:armhf \
+            libjemalloc-dev:armhf \
+            libjpeg62-turbo-dev:armhf \
+            liblttng-ust-dev:armhf \
+            liblzo2-dev:armhf \
+            libncursesw5-dev:armhf \
+            libnfs-dev:armhf \
+            libnuma-dev:armhf \
+            libpam0g-dev:armhf \
+            libpixman-1-dev:armhf \
+            libpng-dev:armhf \
+            libpulse-dev:armhf \
+            librbd-dev:armhf \
+            librdmacm-dev:armhf \
+            libsasl2-dev:armhf \
+            libsdl2-dev:armhf \
+            libsdl2-image-dev:armhf \
+            libseccomp-dev:armhf \
+            libselinux1-dev:armhf \
+            libslirp-dev:armhf \
+            libsnappy-dev:armhf \
+            libspice-server-dev:armhf \
+            libssh-gcrypt-dev:armhf \
+            libsystemd-dev:armhf \
+            libtasn1-6-dev:armhf \
+            libubsan1:armhf \
+            libudev-dev:armhf \
+            liburing-dev:armhf \
+            libusb-1.0-0-dev:armhf \
+            libusbredirhost-dev:armhf \
+            libvdeplug-dev:armhf \
+            libvirglrenderer-dev:armhf \
+            libvte-2.91-dev:armhf \
+            libxen-dev:armhf \
+            libzstd-dev:armhf \
+            nettle-dev:armhf \
+            systemtap-sdt-dev:armhf \
+            xfslibs-dev:armhf \
+            zlib1g-dev:armhf && \
+    eatmydata apt-get autoremove -y && \
+    eatmydata apt-get autoclean -y && \
+    mkdir -p /usr/local/share/meson/cross && \
+    echo "[binaries]\n\
+c = '/usr/bin/arm-linux-gnueabihf-gcc'\n\
+ar = '/usr/bin/arm-linux-gnueabihf-gcc-ar'\n\
+strip = '/usr/bin/arm-linux-gnueabihf-strip'\n\
+pkgconfig = '/usr/bin/arm-linux-gnueabihf-pkg-config'\n\
+\n\
+[host_machine]\n\
+system = 'linux'\n\
+cpu_family = 'arm'\n\
+cpu = 'armhf'\n\
+endian = 'little'" > /usr/local/share/meson/cross/arm-linux-gnueabihf && \
+    dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \
+    mkdir -p /usr/libexec/ccache-wrappers && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-c++ && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-cc && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-g++ && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-gcc
+
+ENV ABI "arm-linux-gnueabihf"
+ENV MESON_OPTS "--cross-file=arm-linux-gnueabihf"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=arm-linux-gnueabihf-
-ENV DEF_TARGET_LIST arm-softmmu,arm-linux-user,armeb-linux-user
-
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt install -y --no-install-recommends \
-        libbz2-dev:armhf \
-        liblzo2-dev:armhf \
-        librdmacm-dev:armhf \
-        libsnappy-dev:armhf \
-        libxen-dev:armhf
+ENV DEF_TARGET_LIST arm-softmmu,arm-linux-user
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index dc1fc21ef9..7e4a92f630 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -105,6 +105,11 @@ try:
                         trailer=debian_cross_build("aarch64-linux-gnu-",
                                                    "aarch64-softmmu,aarch64-linux-user"))
 
+    generate_dockerfile("debian-armhf-cross", "debian-11",
+                        cross="armv7l",
+                        trailer=debian_cross_build("arm-linux-gnueabihf-",
+                                                   "arm-softmmu,arm-linux-user"))
+
     generate_dockerfile("debian-s390x-cross", "debian-11",
                         cross="s390x",
                         trailer=debian_cross_build("s390x-linux-gnu-",

From 94806241878e2f8908a04f82fe4086ce56690d80 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Fri, 27 May 2022 16:35:37 +0100
Subject: [PATCH 708/935] tests/docker: update debian-armel-cross with lcitool
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Use lcitool to update debian-armel-cross to a Debian 11 based system.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-8-alex.bennee@linaro.org>
---
 .gitlab-ci.d/container-cross.yml              |   3 +-
 tests/docker/Makefile.include                 |   1 -
 .../dockerfiles/debian-armel-cross.docker     | 178 ++++++++++++++++--
 tests/lcitool/refresh                         |   5 +
 4 files changed, 164 insertions(+), 23 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 4d1830f3fc..caef7decf4 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -27,8 +27,7 @@ arm64-debian-cross-container:
 
 armel-debian-cross-container:
   extends: .container_job_template
-  stage: containers-layer2
-  needs: ['amd64-debian10-container']
+  stage: containers
   variables:
     NAME: debian-armel-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index d6e0710554..d9109bcc77 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -89,7 +89,6 @@ DOCKER_PARTIAL_IMAGES += fedora
 endif
 
 docker-image-debian-alpha-cross: docker-image-debian10
-docker-image-debian-armel-cross: docker-image-debian10
 docker-image-debian-hppa-cross: docker-image-debian10
 docker-image-debian-m68k-cross: docker-image-debian10
 docker-image-debian-mips-cross: docker-image-debian10
diff --git a/tests/docker/dockerfiles/debian-armel-cross.docker b/tests/docker/dockerfiles/debian-armel-cross.docker
index b7b1a3585f..a6153e5a83 100644
--- a/tests/docker/dockerfiles/debian-armel-cross.docker
+++ b/tests/docker/dockerfiles/debian-armel-cross.docker
@@ -1,26 +1,164 @@
+# THIS FILE WAS AUTO-GENERATED
 #
-# Docker armel cross-compiler target
+#  $ lcitool dockerfile --layers all --cross armv6l debian-11 qemu
 #
-# This docker target builds on the debian Stretch base image.
-#
-FROM qemu/debian10
-MAINTAINER Philippe Mathieu-Daudé <f4bug@amsat.org>
+# https://gitlab.com/libvirt/libvirt-ci
 
-# Add the foreign architecture we want and install dependencies
-RUN dpkg --add-architecture armel && \
-    apt update && \
-    apt install -yy crossbuild-essential-armel && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt build-dep -yy -a armel --arch-only qemu
+FROM docker.io/library/debian:11-slim
 
-# Specify the cross prefix for this image (see tests/docker/common.rc)
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    apt-get update && \
+    apt-get install -y eatmydata && \
+    eatmydata apt-get dist-upgrade -y && \
+    eatmydata apt-get install --no-install-recommends -y \
+            bash \
+            bc \
+            bsdextrautils \
+            bzip2 \
+            ca-certificates \
+            ccache \
+            dbus \
+            debianutils \
+            diffutils \
+            exuberant-ctags \
+            findutils \
+            gcovr \
+            genisoimage \
+            gettext \
+            git \
+            hostname \
+            libpcre2-dev \
+            libspice-protocol-dev \
+            llvm \
+            locales \
+            make \
+            meson \
+            ncat \
+            ninja-build \
+            openssh-client \
+            perl-base \
+            pkgconf \
+            python3 \
+            python3-numpy \
+            python3-opencv \
+            python3-pillow \
+            python3-pip \
+            python3-sphinx \
+            python3-sphinx-rtd-theme \
+            python3-venv \
+            python3-yaml \
+            rpm2cpio \
+            sed \
+            sparse \
+            tar \
+            tesseract-ocr \
+            tesseract-ocr-eng \
+            texinfo && \
+    eatmydata apt-get autoremove -y && \
+    eatmydata apt-get autoclean -y && \
+    sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+    dpkg-reconfigure locales
+
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
+
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    dpkg --add-architecture armel && \
+    eatmydata apt-get update && \
+    eatmydata apt-get dist-upgrade -y && \
+    eatmydata apt-get install --no-install-recommends -y dpkg-dev && \
+    eatmydata apt-get install --no-install-recommends -y \
+            g++-arm-linux-gnueabi \
+            gcc-arm-linux-gnueabi \
+            libaio-dev:armel \
+            libasan5:armel \
+            libasound2-dev:armel \
+            libattr1-dev:armel \
+            libbpf-dev:armel \
+            libbrlapi-dev:armel \
+            libbz2-dev:armel \
+            libc6-dev:armel \
+            libcacard-dev:armel \
+            libcap-ng-dev:armel \
+            libcapstone-dev:armel \
+            libcurl4-gnutls-dev:armel \
+            libdaxctl-dev:armel \
+            libdrm-dev:armel \
+            libepoxy-dev:armel \
+            libfdt-dev:armel \
+            libffi-dev:armel \
+            libfuse3-dev:armel \
+            libgbm-dev:armel \
+            libgcrypt20-dev:armel \
+            libglib2.0-dev:armel \
+            libglusterfs-dev:armel \
+            libgnutls28-dev:armel \
+            libgtk-3-dev:armel \
+            libibumad-dev:armel \
+            libibverbs-dev:armel \
+            libiscsi-dev:armel \
+            libjemalloc-dev:armel \
+            libjpeg62-turbo-dev:armel \
+            liblttng-ust-dev:armel \
+            liblzo2-dev:armel \
+            libncursesw5-dev:armel \
+            libnfs-dev:armel \
+            libnuma-dev:armel \
+            libpam0g-dev:armel \
+            libpixman-1-dev:armel \
+            libpng-dev:armel \
+            libpulse-dev:armel \
+            librbd-dev:armel \
+            librdmacm-dev:armel \
+            libsasl2-dev:armel \
+            libsdl2-dev:armel \
+            libsdl2-image-dev:armel \
+            libseccomp-dev:armel \
+            libselinux1-dev:armel \
+            libslirp-dev:armel \
+            libsnappy-dev:armel \
+            libspice-server-dev:armel \
+            libssh-gcrypt-dev:armel \
+            libsystemd-dev:armel \
+            libtasn1-6-dev:armel \
+            libubsan1:armel \
+            libudev-dev:armel \
+            liburing-dev:armel \
+            libusb-1.0-0-dev:armel \
+            libusbredirhost-dev:armel \
+            libvdeplug-dev:armel \
+            libvirglrenderer-dev:armel \
+            libvte-2.91-dev:armel \
+            libzstd-dev:armel \
+            nettle-dev:armel \
+            systemtap-sdt-dev:armel \
+            xfslibs-dev:armel \
+            zlib1g-dev:armel && \
+    eatmydata apt-get autoremove -y && \
+    eatmydata apt-get autoclean -y && \
+    mkdir -p /usr/local/share/meson/cross && \
+    echo "[binaries]\n\
+c = '/usr/bin/arm-linux-gnueabi-gcc'\n\
+ar = '/usr/bin/arm-linux-gnueabi-gcc-ar'\n\
+strip = '/usr/bin/arm-linux-gnueabi-strip'\n\
+pkgconfig = '/usr/bin/arm-linux-gnueabi-pkg-config'\n\
+\n\
+[host_machine]\n\
+system = 'linux'\n\
+cpu_family = 'arm'\n\
+cpu = 'arm'\n\
+endian = 'little'" > /usr/local/share/meson/cross/arm-linux-gnueabi && \
+    dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \
+    mkdir -p /usr/libexec/ccache-wrappers && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabi-c++ && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabi-cc && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabi-g++ && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabi-gcc
+
+ENV ABI "arm-linux-gnueabi"
+ENV MESON_OPTS "--cross-file=arm-linux-gnueabi"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=arm-linux-gnueabi-
 ENV DEF_TARGET_LIST arm-softmmu,arm-linux-user,armeb-linux-user
-
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt install -y --no-install-recommends \
-        libbz2-dev:armel \
-        liblzo2-dev:armel \
-        librdmacm-dev:armel \
-        libsnappy-dev:armel
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 7e4a92f630..4dc5527234 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -105,6 +105,11 @@ try:
                         trailer=debian_cross_build("aarch64-linux-gnu-",
                                                    "aarch64-softmmu,aarch64-linux-user"))
 
+    generate_dockerfile("debian-armel-cross", "debian-11",
+                        cross="armv6l",
+                        trailer=debian_cross_build("arm-linux-gnueabi-",
+                                                   "arm-softmmu,arm-linux-user,armeb-linux-user"))
+
     generate_dockerfile("debian-armhf-cross", "debian-11",
                         cross="armv7l",
                         trailer=debian_cross_build("arm-linux-gnueabihf-",

From 432ae739c6ce6f31f74da62be34325890ded278b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Fri, 27 May 2022 16:35:38 +0100
Subject: [PATCH 709/935] tests/docker: update debian-mipsel-cross with lcitool
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Use lcitool to update debian-mipsel-cross to a Debian 11 based system.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-9-alex.bennee@linaro.org>
---
 .gitlab-ci.d/container-cross.yml              |   3 +-
 tests/docker/Makefile.include                 |   1 -
 .../dockerfiles/debian-mipsel-cross.docker    | 179 +++++++++++++++---
 tests/lcitool/refresh                         |   5 +
 4 files changed, 161 insertions(+), 27 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index caef7decf4..1a533e6fc0 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -102,8 +102,7 @@ mips-debian-cross-container:
 
 mipsel-debian-cross-container:
   extends: .container_job_template
-  stage: containers-layer2
-  needs: ['amd64-debian10-container']
+  stage: containers
   variables:
     NAME: debian-mipsel-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index d9109bcc77..0ac5975419 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -94,7 +94,6 @@ docker-image-debian-m68k-cross: docker-image-debian10
 docker-image-debian-mips-cross: docker-image-debian10
 docker-image-debian-mips64-cross: docker-image-debian10
 docker-image-debian-mips64el-cross: docker-image-debian10
-docker-image-debian-mipsel-cross: docker-image-debian10
 docker-image-debian-ppc64el-cross: docker-image-debian10
 docker-image-debian-sh4-cross: docker-image-debian10
 docker-image-debian-sparc64-cross: docker-image-debian10
diff --git a/tests/docker/dockerfiles/debian-mipsel-cross.docker b/tests/docker/dockerfiles/debian-mipsel-cross.docker
index 0e5dd42d3c..b6d99ae324 100644
--- a/tests/docker/dockerfiles/debian-mipsel-cross.docker
+++ b/tests/docker/dockerfiles/debian-mipsel-cross.docker
@@ -1,31 +1,162 @@
+# THIS FILE WAS AUTO-GENERATED
 #
-# Docker mipsel cross-compiler target
+#  $ lcitool dockerfile --layers all --cross mipsel debian-11 qemu
 #
-# This docker target builds on the debian Stretch base image.
-#
-FROM qemu/debian10
+# https://gitlab.com/libvirt/libvirt-ci
 
-MAINTAINER Philippe Mathieu-Daudé <f4bug@amsat.org>
+FROM docker.io/library/debian:11-slim
 
-# Add the foreign architecture we want and install dependencies
-RUN dpkg --add-architecture mipsel
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt install -y --no-install-recommends \
-        gcc-mipsel-linux-gnu
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    apt-get update && \
+    apt-get install -y eatmydata && \
+    eatmydata apt-get dist-upgrade -y && \
+    eatmydata apt-get install --no-install-recommends -y \
+            bash \
+            bc \
+            bsdextrautils \
+            bzip2 \
+            ca-certificates \
+            ccache \
+            dbus \
+            debianutils \
+            diffutils \
+            exuberant-ctags \
+            findutils \
+            gcovr \
+            genisoimage \
+            gettext \
+            git \
+            hostname \
+            libpcre2-dev \
+            libspice-protocol-dev \
+            llvm \
+            locales \
+            make \
+            meson \
+            ncat \
+            ninja-build \
+            openssh-client \
+            perl-base \
+            pkgconf \
+            python3 \
+            python3-numpy \
+            python3-opencv \
+            python3-pillow \
+            python3-pip \
+            python3-sphinx \
+            python3-sphinx-rtd-theme \
+            python3-venv \
+            python3-yaml \
+            rpm2cpio \
+            sed \
+            sparse \
+            tar \
+            tesseract-ocr \
+            tesseract-ocr-eng \
+            texinfo && \
+    eatmydata apt-get autoremove -y && \
+    eatmydata apt-get autoclean -y && \
+    sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+    dpkg-reconfigure locales
 
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt build-dep -yy -a mipsel --arch-only qemu
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
 
-# Specify the cross prefix for this image (see tests/docker/common.rc)
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    dpkg --add-architecture mipsel && \
+    eatmydata apt-get update && \
+    eatmydata apt-get dist-upgrade -y && \
+    eatmydata apt-get install --no-install-recommends -y dpkg-dev && \
+    eatmydata apt-get install --no-install-recommends -y \
+            g++-mipsel-linux-gnu \
+            gcc-mipsel-linux-gnu \
+            libaio-dev:mipsel \
+            libasound2-dev:mipsel \
+            libattr1-dev:mipsel \
+            libbpf-dev:mipsel \
+            libbrlapi-dev:mipsel \
+            libbz2-dev:mipsel \
+            libc6-dev:mipsel \
+            libcacard-dev:mipsel \
+            libcap-ng-dev:mipsel \
+            libcapstone-dev:mipsel \
+            libcurl4-gnutls-dev:mipsel \
+            libdaxctl-dev:mipsel \
+            libdrm-dev:mipsel \
+            libepoxy-dev:mipsel \
+            libfdt-dev:mipsel \
+            libffi-dev:mipsel \
+            libfuse3-dev:mipsel \
+            libgbm-dev:mipsel \
+            libgcrypt20-dev:mipsel \
+            libglib2.0-dev:mipsel \
+            libglusterfs-dev:mipsel \
+            libgnutls28-dev:mipsel \
+            libgtk-3-dev:mipsel \
+            libibumad-dev:mipsel \
+            libibverbs-dev:mipsel \
+            libiscsi-dev:mipsel \
+            libjemalloc-dev:mipsel \
+            libjpeg62-turbo-dev:mipsel \
+            liblttng-ust-dev:mipsel \
+            liblzo2-dev:mipsel \
+            libncursesw5-dev:mipsel \
+            libnfs-dev:mipsel \
+            libnuma-dev:mipsel \
+            libpam0g-dev:mipsel \
+            libpixman-1-dev:mipsel \
+            libpng-dev:mipsel \
+            libpulse-dev:mipsel \
+            librbd-dev:mipsel \
+            librdmacm-dev:mipsel \
+            libsasl2-dev:mipsel \
+            libsdl2-dev:mipsel \
+            libsdl2-image-dev:mipsel \
+            libseccomp-dev:mipsel \
+            libselinux1-dev:mipsel \
+            libslirp-dev:mipsel \
+            libsnappy-dev:mipsel \
+            libspice-server-dev:mipsel \
+            libssh-gcrypt-dev:mipsel \
+            libsystemd-dev:mipsel \
+            libtasn1-6-dev:mipsel \
+            libudev-dev:mipsel \
+            liburing-dev:mipsel \
+            libusb-1.0-0-dev:mipsel \
+            libusbredirhost-dev:mipsel \
+            libvdeplug-dev:mipsel \
+            libvirglrenderer-dev:mipsel \
+            libvte-2.91-dev:mipsel \
+            libzstd-dev:mipsel \
+            nettle-dev:mipsel \
+            systemtap-sdt-dev:mipsel \
+            xfslibs-dev:mipsel \
+            zlib1g-dev:mipsel && \
+    eatmydata apt-get autoremove -y && \
+    eatmydata apt-get autoclean -y && \
+    mkdir -p /usr/local/share/meson/cross && \
+    echo "[binaries]\n\
+c = '/usr/bin/mipsel-linux-gnu-gcc'\n\
+ar = '/usr/bin/mipsel-linux-gnu-gcc-ar'\n\
+strip = '/usr/bin/mipsel-linux-gnu-strip'\n\
+pkgconfig = '/usr/bin/mipsel-linux-gnu-pkg-config'\n\
+\n\
+[host_machine]\n\
+system = 'linux'\n\
+cpu_family = 'mips'\n\
+cpu = 'mipsel'\n\
+endian = 'little'" > /usr/local/share/meson/cross/mipsel-linux-gnu && \
+    dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \
+    mkdir -p /usr/libexec/ccache-wrappers && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-c++ && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-cc && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-g++ && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-gcc
+
+ENV ABI "mipsel-linux-gnu"
+ENV MESON_OPTS "--cross-file=mipsel-linux-gnu"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=mipsel-linux-gnu-
-
-# Install extra libraries to increase code coverage
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt install -y --no-install-recommends \
-        libbz2-dev:mipsel \
-        liblzo2-dev:mipsel \
-        librdmacm-dev:mipsel \
-        libsnappy-dev:mipsel
+ENV DEF_TARGET_LIST mipsel-softmmu,mipsel-linux-user
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 4dc5527234..a4b7452189 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -115,6 +115,11 @@ try:
                         trailer=debian_cross_build("arm-linux-gnueabihf-",
                                                    "arm-softmmu,arm-linux-user"))
 
+    generate_dockerfile("debian-mipsel-cross", "debian-11",
+                        cross="mipsel",
+                        trailer=debian_cross_build("mipsel-linux-gnu-",
+                                                   "mipsel-softmmu,mipsel-linux-user"))
+
     generate_dockerfile("debian-s390x-cross", "debian-11",
                         cross="s390x",
                         trailer=debian_cross_build("s390x-linux-gnu-",

From 1e834d1714e72579d595d5395a3bb35b12a05d09 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Fri, 27 May 2022 16:35:39 +0100
Subject: [PATCH 710/935] tests/docker: update debian-mips64el-cross with
 lcitool
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Use lcitool to update debian-mips64el-cross to a Debian 11 based system.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-10-alex.bennee@linaro.org>
---
 .gitlab-ci.d/container-cross.yml              |   3 +-
 tests/docker/Makefile.include                 |   1 -
 .../dockerfiles/debian-mips64el-cross.docker  | 177 +++++++++++++++---
 tests/lcitool/refresh                         |   5 +
 4 files changed, 159 insertions(+), 27 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 1a533e6fc0..411dc06bf8 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -88,8 +88,7 @@ mips64-debian-cross-container:
 
 mips64el-debian-cross-container:
   extends: .container_job_template
-  stage: containers-layer2
-  needs: ['amd64-debian10-container']
+  stage: containers
   variables:
     NAME: debian-mips64el-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 0ac5975419..d9f37ae8fa 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -93,7 +93,6 @@ docker-image-debian-hppa-cross: docker-image-debian10
 docker-image-debian-m68k-cross: docker-image-debian10
 docker-image-debian-mips-cross: docker-image-debian10
 docker-image-debian-mips64-cross: docker-image-debian10
-docker-image-debian-mips64el-cross: docker-image-debian10
 docker-image-debian-ppc64el-cross: docker-image-debian10
 docker-image-debian-sh4-cross: docker-image-debian10
 docker-image-debian-sparc64-cross: docker-image-debian10
diff --git a/tests/docker/dockerfiles/debian-mips64el-cross.docker b/tests/docker/dockerfiles/debian-mips64el-cross.docker
index c990b683b7..b02dcb7fd9 100644
--- a/tests/docker/dockerfiles/debian-mips64el-cross.docker
+++ b/tests/docker/dockerfiles/debian-mips64el-cross.docker
@@ -1,33 +1,162 @@
+# THIS FILE WAS AUTO-GENERATED
 #
-# Docker mips64el cross-compiler target
-#
-# This docker target builds on the debian Stretch base image.
+#  $ lcitool dockerfile --layers all --cross mips64el debian-11 qemu
 #
+# https://gitlab.com/libvirt/libvirt-ci
 
-FROM qemu/debian10
+FROM docker.io/library/debian:11-slim
 
-MAINTAINER Philippe Mathieu-Daudé <f4bug@amsat.org>
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    apt-get update && \
+    apt-get install -y eatmydata && \
+    eatmydata apt-get dist-upgrade -y && \
+    eatmydata apt-get install --no-install-recommends -y \
+            bash \
+            bc \
+            bsdextrautils \
+            bzip2 \
+            ca-certificates \
+            ccache \
+            dbus \
+            debianutils \
+            diffutils \
+            exuberant-ctags \
+            findutils \
+            gcovr \
+            genisoimage \
+            gettext \
+            git \
+            hostname \
+            libpcre2-dev \
+            libspice-protocol-dev \
+            llvm \
+            locales \
+            make \
+            meson \
+            ncat \
+            ninja-build \
+            openssh-client \
+            perl-base \
+            pkgconf \
+            python3 \
+            python3-numpy \
+            python3-opencv \
+            python3-pillow \
+            python3-pip \
+            python3-sphinx \
+            python3-sphinx-rtd-theme \
+            python3-venv \
+            python3-yaml \
+            rpm2cpio \
+            sed \
+            sparse \
+            tar \
+            tesseract-ocr \
+            tesseract-ocr-eng \
+            texinfo && \
+    eatmydata apt-get autoremove -y && \
+    eatmydata apt-get autoclean -y && \
+    sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+    dpkg-reconfigure locales
 
-# Add the foreign architecture we want and install dependencies
-RUN dpkg --add-architecture mips64el && \
-    apt update && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt install -y --no-install-recommends \
-        gcc-mips64el-linux-gnuabi64
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
 
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt build-dep -yy -a mips64el --arch-only qemu
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    dpkg --add-architecture mips64el && \
+    eatmydata apt-get update && \
+    eatmydata apt-get dist-upgrade -y && \
+    eatmydata apt-get install --no-install-recommends -y dpkg-dev && \
+    eatmydata apt-get install --no-install-recommends -y \
+            g++-mips64el-linux-gnuabi64 \
+            gcc-mips64el-linux-gnuabi64 \
+            libaio-dev:mips64el \
+            libasound2-dev:mips64el \
+            libattr1-dev:mips64el \
+            libbpf-dev:mips64el \
+            libbrlapi-dev:mips64el \
+            libbz2-dev:mips64el \
+            libc6-dev:mips64el \
+            libcacard-dev:mips64el \
+            libcap-ng-dev:mips64el \
+            libcapstone-dev:mips64el \
+            libcurl4-gnutls-dev:mips64el \
+            libdaxctl-dev:mips64el \
+            libdrm-dev:mips64el \
+            libepoxy-dev:mips64el \
+            libfdt-dev:mips64el \
+            libffi-dev:mips64el \
+            libfuse3-dev:mips64el \
+            libgbm-dev:mips64el \
+            libgcrypt20-dev:mips64el \
+            libglib2.0-dev:mips64el \
+            libglusterfs-dev:mips64el \
+            libgnutls28-dev:mips64el \
+            libgtk-3-dev:mips64el \
+            libibumad-dev:mips64el \
+            libibverbs-dev:mips64el \
+            libiscsi-dev:mips64el \
+            libjemalloc-dev:mips64el \
+            libjpeg62-turbo-dev:mips64el \
+            liblttng-ust-dev:mips64el \
+            liblzo2-dev:mips64el \
+            libncursesw5-dev:mips64el \
+            libnfs-dev:mips64el \
+            libnuma-dev:mips64el \
+            libpam0g-dev:mips64el \
+            libpixman-1-dev:mips64el \
+            libpng-dev:mips64el \
+            libpulse-dev:mips64el \
+            librbd-dev:mips64el \
+            librdmacm-dev:mips64el \
+            libsasl2-dev:mips64el \
+            libsdl2-dev:mips64el \
+            libsdl2-image-dev:mips64el \
+            libseccomp-dev:mips64el \
+            libselinux1-dev:mips64el \
+            libslirp-dev:mips64el \
+            libsnappy-dev:mips64el \
+            libspice-server-dev:mips64el \
+            libssh-gcrypt-dev:mips64el \
+            libsystemd-dev:mips64el \
+            libtasn1-6-dev:mips64el \
+            libudev-dev:mips64el \
+            liburing-dev:mips64el \
+            libusb-1.0-0-dev:mips64el \
+            libusbredirhost-dev:mips64el \
+            libvdeplug-dev:mips64el \
+            libvirglrenderer-dev:mips64el \
+            libvte-2.91-dev:mips64el \
+            libzstd-dev:mips64el \
+            nettle-dev:mips64el \
+            systemtap-sdt-dev:mips64el \
+            xfslibs-dev:mips64el \
+            zlib1g-dev:mips64el && \
+    eatmydata apt-get autoremove -y && \
+    eatmydata apt-get autoclean -y && \
+    mkdir -p /usr/local/share/meson/cross && \
+    echo "[binaries]\n\
+c = '/usr/bin/mips64el-linux-gnuabi64-gcc'\n\
+ar = '/usr/bin/mips64el-linux-gnuabi64-gcc-ar'\n\
+strip = '/usr/bin/mips64el-linux-gnuabi64-strip'\n\
+pkgconfig = '/usr/bin/mips64el-linux-gnuabi64-pkg-config'\n\
+\n\
+[host_machine]\n\
+system = 'linux'\n\
+cpu_family = 'mips64'\n\
+cpu = 'mips64el'\n\
+endian = 'little'" > /usr/local/share/meson/cross/mips64el-linux-gnuabi64 && \
+    dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \
+    mkdir -p /usr/libexec/ccache-wrappers && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-c++ && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-cc && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-g++ && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-gcc
 
-# Specify the cross prefix for this image (see tests/docker/common.rc)
+ENV ABI "mips64el-linux-gnuabi64"
+ENV MESON_OPTS "--cross-file=mips64el-linux-gnuabi64"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=mips64el-linux-gnuabi64-
 ENV DEF_TARGET_LIST mips64el-softmmu,mips64el-linux-user
-
-# Install extra libraries to increase code coverage
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt install -y --no-install-recommends \
-        libbz2-dev:mips64el \
-        liblzo2-dev:mips64el \
-        librdmacm-dev:mips64el \
-        libsnappy-dev:mips64el
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index a4b7452189..341a07e677 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -115,6 +115,11 @@ try:
                         trailer=debian_cross_build("arm-linux-gnueabihf-",
                                                    "arm-softmmu,arm-linux-user"))
 
+    generate_dockerfile("debian-mips64el-cross", "debian-11",
+                        cross="mips64el",
+                        trailer=debian_cross_build("mips64el-linux-gnuabi64-",
+                                                  "mips64el-softmmu,mips64el-linux-user"))
+
     generate_dockerfile("debian-mipsel-cross", "debian-11",
                         cross="mipsel",
                         trailer=debian_cross_build("mipsel-linux-gnu-",

From 4ec740e12da8a969f1ae3f9147dde9ce3a79a506 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Fri, 27 May 2022 16:35:40 +0100
Subject: [PATCH 711/935] tests/docker: update debian-ppc64el-cross with
 lcitool
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Use lcitool to update debian-ppc64el-cross to a Debian 11 based system.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-11-alex.bennee@linaro.org>
---
 .gitlab-ci.d/container-cross.yml              |   3 +-
 tests/docker/Makefile.include                 |   1 -
 .../dockerfiles/debian-ppc64el-cross.docker   | 178 +++++++++++++++---
 tests/lcitool/refresh                         |   5 +
 4 files changed, 163 insertions(+), 24 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 411dc06bf8..147e667744 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -114,8 +114,7 @@ powerpc-test-cross-container:
 
 ppc64el-debian-cross-container:
   extends: .container_job_template
-  stage: containers-layer2
-  needs: ['amd64-debian10-container']
+  stage: containers
   variables:
     NAME: debian-ppc64el-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index d9f37ae8fa..e68f91b853 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -93,7 +93,6 @@ docker-image-debian-hppa-cross: docker-image-debian10
 docker-image-debian-m68k-cross: docker-image-debian10
 docker-image-debian-mips-cross: docker-image-debian10
 docker-image-debian-mips64-cross: docker-image-debian10
-docker-image-debian-ppc64el-cross: docker-image-debian10
 docker-image-debian-sh4-cross: docker-image-debian10
 docker-image-debian-sparc64-cross: docker-image-debian10
 
diff --git a/tests/docker/dockerfiles/debian-ppc64el-cross.docker b/tests/docker/dockerfiles/debian-ppc64el-cross.docker
index 5de12b01cd..bcf04bc90b 100644
--- a/tests/docker/dockerfiles/debian-ppc64el-cross.docker
+++ b/tests/docker/dockerfiles/debian-ppc64el-cross.docker
@@ -1,28 +1,164 @@
+# THIS FILE WAS AUTO-GENERATED
 #
-# Docker ppc64el cross-compiler target
+#  $ lcitool dockerfile --layers all --cross ppc64le debian-11 qemu
 #
-# This docker target builds on the debian Stretch base image.
-#
-FROM qemu/debian10
+# https://gitlab.com/libvirt/libvirt-ci
 
-# Add the foreign architecture we want and install dependencies
-RUN dpkg --add-architecture ppc64el && \
-    apt update && \
-    apt install -yy crossbuild-essential-ppc64el
+FROM docker.io/library/debian:11-slim
 
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt build-dep -yy -a ppc64el --arch-only qemu
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    apt-get update && \
+    apt-get install -y eatmydata && \
+    eatmydata apt-get dist-upgrade -y && \
+    eatmydata apt-get install --no-install-recommends -y \
+            bash \
+            bc \
+            bsdextrautils \
+            bzip2 \
+            ca-certificates \
+            ccache \
+            dbus \
+            debianutils \
+            diffutils \
+            exuberant-ctags \
+            findutils \
+            gcovr \
+            genisoimage \
+            gettext \
+            git \
+            hostname \
+            libpcre2-dev \
+            libspice-protocol-dev \
+            llvm \
+            locales \
+            make \
+            meson \
+            ncat \
+            ninja-build \
+            openssh-client \
+            perl-base \
+            pkgconf \
+            python3 \
+            python3-numpy \
+            python3-opencv \
+            python3-pillow \
+            python3-pip \
+            python3-sphinx \
+            python3-sphinx-rtd-theme \
+            python3-venv \
+            python3-yaml \
+            rpm2cpio \
+            sed \
+            sparse \
+            tar \
+            tesseract-ocr \
+            tesseract-ocr-eng \
+            texinfo && \
+    eatmydata apt-get autoremove -y && \
+    eatmydata apt-get autoclean -y && \
+    sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+    dpkg-reconfigure locales
 
-# Specify the cross prefix for this image (see tests/docker/common.rc)
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
+
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    dpkg --add-architecture ppc64el && \
+    eatmydata apt-get update && \
+    eatmydata apt-get dist-upgrade -y && \
+    eatmydata apt-get install --no-install-recommends -y dpkg-dev && \
+    eatmydata apt-get install --no-install-recommends -y \
+            g++-powerpc64le-linux-gnu \
+            gcc-powerpc64le-linux-gnu \
+            libaio-dev:ppc64el \
+            libasan5:ppc64el \
+            libasound2-dev:ppc64el \
+            libattr1-dev:ppc64el \
+            libbpf-dev:ppc64el \
+            libbrlapi-dev:ppc64el \
+            libbz2-dev:ppc64el \
+            libc6-dev:ppc64el \
+            libcacard-dev:ppc64el \
+            libcap-ng-dev:ppc64el \
+            libcapstone-dev:ppc64el \
+            libcurl4-gnutls-dev:ppc64el \
+            libdaxctl-dev:ppc64el \
+            libdrm-dev:ppc64el \
+            libepoxy-dev:ppc64el \
+            libfdt-dev:ppc64el \
+            libffi-dev:ppc64el \
+            libfuse3-dev:ppc64el \
+            libgbm-dev:ppc64el \
+            libgcrypt20-dev:ppc64el \
+            libglib2.0-dev:ppc64el \
+            libglusterfs-dev:ppc64el \
+            libgnutls28-dev:ppc64el \
+            libgtk-3-dev:ppc64el \
+            libibumad-dev:ppc64el \
+            libibverbs-dev:ppc64el \
+            libiscsi-dev:ppc64el \
+            libjemalloc-dev:ppc64el \
+            libjpeg62-turbo-dev:ppc64el \
+            liblttng-ust-dev:ppc64el \
+            liblzo2-dev:ppc64el \
+            libncursesw5-dev:ppc64el \
+            libnfs-dev:ppc64el \
+            libnuma-dev:ppc64el \
+            libpam0g-dev:ppc64el \
+            libpixman-1-dev:ppc64el \
+            libpng-dev:ppc64el \
+            libpulse-dev:ppc64el \
+            librbd-dev:ppc64el \
+            librdmacm-dev:ppc64el \
+            libsasl2-dev:ppc64el \
+            libsdl2-dev:ppc64el \
+            libsdl2-image-dev:ppc64el \
+            libseccomp-dev:ppc64el \
+            libselinux1-dev:ppc64el \
+            libslirp-dev:ppc64el \
+            libsnappy-dev:ppc64el \
+            libspice-server-dev:ppc64el \
+            libssh-gcrypt-dev:ppc64el \
+            libsystemd-dev:ppc64el \
+            libtasn1-6-dev:ppc64el \
+            libubsan1:ppc64el \
+            libudev-dev:ppc64el \
+            liburing-dev:ppc64el \
+            libusb-1.0-0-dev:ppc64el \
+            libusbredirhost-dev:ppc64el \
+            libvdeplug-dev:ppc64el \
+            libvirglrenderer-dev:ppc64el \
+            libvte-2.91-dev:ppc64el \
+            libzstd-dev:ppc64el \
+            nettle-dev:ppc64el \
+            systemtap-sdt-dev:ppc64el \
+            xfslibs-dev:ppc64el \
+            zlib1g-dev:ppc64el && \
+    eatmydata apt-get autoremove -y && \
+    eatmydata apt-get autoclean -y && \
+    mkdir -p /usr/local/share/meson/cross && \
+    echo "[binaries]\n\
+c = '/usr/bin/powerpc64le-linux-gnu-gcc'\n\
+ar = '/usr/bin/powerpc64le-linux-gnu-gcc-ar'\n\
+strip = '/usr/bin/powerpc64le-linux-gnu-strip'\n\
+pkgconfig = '/usr/bin/powerpc64le-linux-gnu-pkg-config'\n\
+\n\
+[host_machine]\n\
+system = 'linux'\n\
+cpu_family = 'ppc64'\n\
+cpu = 'powerpc64le'\n\
+endian = 'little'" > /usr/local/share/meson/cross/powerpc64le-linux-gnu && \
+    dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \
+    mkdir -p /usr/libexec/ccache-wrappers && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-c++ && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-cc && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-g++ && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-gcc
+
+ENV ABI "powerpc64le-linux-gnu"
+ENV MESON_OPTS "--cross-file=powerpc64le-linux-gnu"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=powerpc64le-linux-gnu-
 ENV DEF_TARGET_LIST ppc64-softmmu,ppc64-linux-user
-
-# Install extra libraries to increase code coverage
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt install -y --no-install-recommends \
-        libbz2-dev:ppc64el \
-        liblzo2-dev:ppc64el \
-        librdmacm-dev:ppc64el \
-        libsnappy-dev:ppc64el
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 341a07e677..b947bb170d 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -125,6 +125,11 @@ try:
                         trailer=debian_cross_build("mipsel-linux-gnu-",
                                                    "mipsel-softmmu,mipsel-linux-user"))
 
+    generate_dockerfile("debian-ppc64el-cross", "debian-11",
+                        cross="ppc64le",
+                        trailer=debian_cross_build("powerpc64le-linux-gnu-",
+                                                   "ppc64-softmmu,ppc64-linux-user"))
+
     generate_dockerfile("debian-s390x-cross", "debian-11",
                         cross="s390x",
                         trailer=debian_cross_build("s390x-linux-gnu-",

From 9e19fd7d4a569f5802ede460fe03fd2d4d0c295c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Fri, 27 May 2022 16:35:41 +0100
Subject: [PATCH 712/935] tests/docker: update debian-amd64 with lcitool
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The one minor wrinkle we need to account for is the netmap support
still requires building from source. We also include cscope and GNU
global as they are used in one of the builds.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Luigi Rizzo <rizzo@iet.unipi.it>
Cc: Giuseppe Lettieri <g.lettieri@iet.unipi.it>
Cc: Vincenzo Maffione <v.maffione@gmail.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-12-alex.bennee@linaro.org>
---
 .gitlab-ci.d/containers.yml                  |   3 +-
 tests/docker/dockerfiles/debian-amd64.docker | 194 ++++++++++++++-----
 tests/lcitool/refresh                        |  19 ++
 3 files changed, 164 insertions(+), 52 deletions(-)

diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
index e9df90bbdd..be34cbc7ba 100644
--- a/.gitlab-ci.d/containers.yml
+++ b/.gitlab-ci.d/containers.yml
@@ -14,8 +14,7 @@ amd64-debian11-container:
 
 amd64-debian-container:
   extends: .container_job_template
-  stage: containers-layer2
-  needs: ['amd64-debian10-container']
+  stage: containers
   variables:
     NAME: debian-amd64
 
diff --git a/tests/docker/dockerfiles/debian-amd64.docker b/tests/docker/dockerfiles/debian-amd64.docker
index ed546edcd6..503e282802 100644
--- a/tests/docker/dockerfiles/debian-amd64.docker
+++ b/tests/docker/dockerfiles/debian-amd64.docker
@@ -1,59 +1,153 @@
+# THIS FILE WAS AUTO-GENERATED
 #
-# Docker x86_64 target
+#  $ lcitool dockerfile --layers all debian-11 qemu
 #
-# This docker target builds on the Debian Buster base image. Further
-# libraries which are not widely available are installed by hand.
-#
-FROM qemu/debian10
-MAINTAINER Philippe Mathieu-Daudé <f4bug@amsat.org>
+# https://gitlab.com/libvirt/libvirt-ci
 
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt build-dep -yy qemu
+FROM docker.io/library/debian:11-slim
 
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt install -y --no-install-recommends \
-        cscope \
-        genisoimage \
-        exuberant-ctags \
-        global \
-        libbz2-dev \
-        liblzo2-dev \
-        libgcrypt20-dev \
-        libfdt-dev \
-        librdmacm-dev \
-        libsasl2-dev \
-        libsnappy-dev \
-        libvte-dev \
-        netcat-openbsd \
-        openssh-client \
-        python3-numpy \
-        python3-opencv \
-        python3-venv
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    apt-get update && \
+    apt-get install -y eatmydata && \
+    eatmydata apt-get dist-upgrade -y && \
+    eatmydata apt-get install --no-install-recommends -y \
+            bash \
+            bc \
+            bsdextrautils \
+            bzip2 \
+            ca-certificates \
+            ccache \
+            clang \
+            dbus \
+            debianutils \
+            diffutils \
+            exuberant-ctags \
+            findutils \
+            g++ \
+            gcc \
+            gcovr \
+            genisoimage \
+            gettext \
+            git \
+            hostname \
+            libaio-dev \
+            libasan5 \
+            libasound2-dev \
+            libattr1-dev \
+            libbpf-dev \
+            libbrlapi-dev \
+            libbz2-dev \
+            libc6-dev \
+            libcacard-dev \
+            libcap-ng-dev \
+            libcapstone-dev \
+            libcurl4-gnutls-dev \
+            libdaxctl-dev \
+            libdrm-dev \
+            libepoxy-dev \
+            libfdt-dev \
+            libffi-dev \
+            libfuse3-dev \
+            libgbm-dev \
+            libgcrypt20-dev \
+            libglib2.0-dev \
+            libglusterfs-dev \
+            libgnutls28-dev \
+            libgtk-3-dev \
+            libibumad-dev \
+            libibverbs-dev \
+            libiscsi-dev \
+            libjemalloc-dev \
+            libjpeg62-turbo-dev \
+            liblttng-ust-dev \
+            liblzo2-dev \
+            libncursesw5-dev \
+            libnfs-dev \
+            libnuma-dev \
+            libpam0g-dev \
+            libpcre2-dev \
+            libpixman-1-dev \
+            libpmem-dev \
+            libpng-dev \
+            libpulse-dev \
+            librbd-dev \
+            librdmacm-dev \
+            libsasl2-dev \
+            libsdl2-dev \
+            libsdl2-image-dev \
+            libseccomp-dev \
+            libselinux1-dev \
+            libslirp-dev \
+            libsnappy-dev \
+            libspice-protocol-dev \
+            libspice-server-dev \
+            libssh-gcrypt-dev \
+            libsystemd-dev \
+            libtasn1-6-dev \
+            libubsan1 \
+            libudev-dev \
+            liburing-dev \
+            libusb-1.0-0-dev \
+            libusbredirhost-dev \
+            libvdeplug-dev \
+            libvirglrenderer-dev \
+            libvte-2.91-dev \
+            libxen-dev \
+            libzstd-dev \
+            llvm \
+            locales \
+            make \
+            meson \
+            multipath-tools \
+            ncat \
+            nettle-dev \
+            ninja-build \
+            openssh-client \
+            perl-base \
+            pkgconf \
+            python3 \
+            python3-numpy \
+            python3-opencv \
+            python3-pillow \
+            python3-pip \
+            python3-sphinx \
+            python3-sphinx-rtd-theme \
+            python3-venv \
+            python3-yaml \
+            rpm2cpio \
+            sed \
+            sparse \
+            systemtap-sdt-dev \
+            tar \
+            tesseract-ocr \
+            tesseract-ocr-eng \
+            texinfo \
+            xfslibs-dev \
+            zlib1g-dev && \
+    eatmydata apt-get autoremove -y && \
+    eatmydata apt-get autoclean -y && \
+    sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+    dpkg-reconfigure locales && \
+    dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \
+    mkdir -p /usr/libexec/ccache-wrappers && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/c++ && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/clang && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/g++ && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc
 
-# virgl
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt install -y --no-install-recommends \
-        libegl1-mesa-dev \
-        libepoxy-dev \
-        libgbm-dev
-RUN git clone https://gitlab.freedesktop.org/virgl/virglrenderer.git /usr/src/virglrenderer && \
-    cd /usr/src/virglrenderer && git checkout virglrenderer-0.8.0
-RUN cd /usr/src/virglrenderer && ./autogen.sh && ./configure --disable-tests && make install
-
-# netmap
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt install -y --no-install-recommends \
-        linux-headers-amd64
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
+# netmap/cscope/global
+RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+  apt install -y --no-install-recommends \
+  cscope\
+  global\
+  linux-headers-amd64
 RUN git clone https://github.com/luigirizzo/netmap.git /usr/src/netmap
 RUN cd /usr/src/netmap && git checkout v11.3
 RUN cd /usr/src/netmap/LINUX && ./configure --no-drivers --no-apps --kernel-dir=$(ls -d /usr/src/linux-headers-*-amd64) && make install
 ENV QEMU_CONFIGURE_OPTS --enable-netmap
-
-RUN ldconfig
-
-# gcrypt
-ENV QEMU_CONFIGURE_OPTS $QEMU_CONFIGURE_OPTS --enable-gcrypt
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index b947bb170d..5e260f8cd6 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -76,6 +76,23 @@ ubuntu2004_tsanhack = [
 ]
 
 
+# Netmap still needs to be manually built as it is yet to be packaged
+# into a distro. We also add cscope and gtags which are used in the CI
+# test
+debian11_extras = [
+    "# netmap/cscope/global\n",
+    "RUN DEBIAN_FRONTEND=noninteractive eatmydata \\\n",
+    "  apt install -y --no-install-recommends \\\n",
+    "  cscope\\\n",
+    "  global\\\n",
+    "  linux-headers-amd64\n",
+    "RUN git clone https://github.com/luigirizzo/netmap.git /usr/src/netmap\n",
+    "RUN cd /usr/src/netmap && git checkout v11.3\n",
+    "RUN cd /usr/src/netmap/LINUX && ./configure --no-drivers --no-apps --kernel-dir=$(ls -d /usr/src/linux-headers-*-amd64) && make install\n",
+    "ENV QEMU_CONFIGURE_OPTS --enable-netmap\n"
+]
+
+
 def debian_cross_build(prefix, targets):
     conf = "ENV QEMU_CONFIGURE_OPTS --cross-prefix=%s\n" % (prefix)
     targets = "ENV DEF_TARGET_LIST %s\n" % (targets)
@@ -92,6 +109,8 @@ try:
     #
     generate_dockerfile("alpine", "alpine-edge")
     generate_dockerfile("centos8", "centos-stream-8")
+    generate_dockerfile("debian-amd64", "debian-11",
+                        trailer="".join(debian11_extras))
     generate_dockerfile("fedora", "fedora-35")
     generate_dockerfile("opensuse-leap", "opensuse-leap-152")
     generate_dockerfile("ubuntu2004", "ubuntu-2004",

From ea857a3b7976010a7fe7b59a89941964028f502a Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 27 May 2022 16:35:42 +0100
Subject: [PATCH 713/935] configure: do not define or use the CPP variable
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Just hardcode $(CC) -E, it should be enough.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517092616.1272238-2-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220527153603.887929-13-alex.bennee@linaro.org>
---
 configure                  | 3 ---
 pc-bios/optionrom/Makefile | 2 +-
 2 files changed, 1 insertion(+), 4 deletions(-)

diff --git a/configure b/configure
index f2baf2f526..c88ef94fec 100755
--- a/configure
+++ b/configure
@@ -376,7 +376,6 @@ fi
 ar="${AR-${cross_prefix}ar}"
 as="${AS-${cross_prefix}as}"
 ccas="${CCAS-$cc}"
-cpp="${CPP-$cc -E}"
 objcopy="${OBJCOPY-${cross_prefix}objcopy}"
 ld="${LD-${cross_prefix}ld}"
 ranlib="${RANLIB-${cross_prefix}ranlib}"
@@ -2014,7 +2013,6 @@ echo "CC=$cc" >> $config_host_mak
 echo "AR=$ar" >> $config_host_mak
 echo "AS=$as" >> $config_host_mak
 echo "CCAS=$ccas" >> $config_host_mak
-echo "CPP=$cpp" >> $config_host_mak
 echo "OBJCOPY=$objcopy" >> $config_host_mak
 echo "LD=$ld" >> $config_host_mak
 echo "QEMU_CFLAGS=$QEMU_CFLAGS" >> $config_host_mak
@@ -2257,7 +2255,6 @@ preserve_env() {
 preserve_env AR
 preserve_env AS
 preserve_env CC
-preserve_env CPP
 preserve_env CFLAGS
 preserve_env CXX
 preserve_env CXXFLAGS
diff --git a/pc-bios/optionrom/Makefile b/pc-bios/optionrom/Makefile
index 2494ad9c25..17ccc76241 100644
--- a/pc-bios/optionrom/Makefile
+++ b/pc-bios/optionrom/Makefile
@@ -50,7 +50,7 @@ override LDFLAGS = -m $(LD_I386_EMULATION) -T $(SRC_DIR)/flat.lds
 pvh.img: pvh.o pvh_main.o
 
 %.o: %.S
-	$(call quiet-command,$(CPP) $(CPPFLAGS) -c -o - $< | $(AS) $(ASFLAGS) -o $@,"AS","$@")
+	$(call quiet-command,$(CC) $(CPPFLAGS) -E -o - $< | $(AS) $(ASFLAGS) -o $@,"AS","$@")
 
 %.o: %.c
 	$(call quiet-command,$(CC) $(CPPFLAGS) $(CFLAGS) -c $< -o $@,"CC","$@")

From 231ca96316f17d0af8c5e72cc2c496c68ee23ab6 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 27 May 2022 16:35:43 +0100
Subject: [PATCH 714/935] build: clean up ninja invocation
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Fix an incorrect "@@:" and move "-d keepdepfile" to the NINJAFLAGS variable.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517092616.1272238-3-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220527153603.887929-14-alex.bennee@linaro.org>
---
 Makefile | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
index b842dbccdb..fad312040f 100644
--- a/Makefile
+++ b/Makefile
@@ -143,7 +143,7 @@ MAKE.q = $(findstring q,$(firstword $(filter-out --%,$(MAKEFLAGS))))
 MAKE.nq = $(if $(word 2, $(MAKE.n) $(MAKE.q)),nq)
 NINJAFLAGS = $(if $V,-v) $(if $(MAKE.n), -n) $(if $(MAKE.k), -k0) \
         $(filter-out -j, $(lastword -j1 $(filter -l% -j%, $(MAKEFLAGS)))) \
-
+        -d keepdepfile
 ninja-cmd-goals = $(or $(MAKECMDGOALS), all)
 ninja-cmd-goals += $(foreach t, $(.check.build-suites), $(.check-$t.deps))
 ninja-cmd-goals += $(foreach t, $(.bench.build-suites), $(.bench-$t.deps))
@@ -160,8 +160,8 @@ $(ninja-targets): run-ninja
 # --output-sync line.
 run-ninja: config-host.mak
 ifneq ($(filter $(ninja-targets), $(ninja-cmd-goals)),)
-	+$(quiet-@)$(if $(MAKE.nq),@:, $(NINJA) -d keepdepfile \
-	   $(NINJAFLAGS) $(sort $(filter $(ninja-targets), $(ninja-cmd-goals))) | cat)
+	+$(if $(MAKE.nq),@:,$(quiet-@)$(NINJA) $(NINJAFLAGS) \
+	   $(sort $(filter $(ninja-targets), $(ninja-cmd-goals))) | cat)
 endif
 endif
 

From 9b32ba53f36428abdecc79a0a047c2636af130d1 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 27 May 2022 16:35:44 +0100
Subject: [PATCH 715/935] build: add a more generic way to specify make->ninja
 dependencies
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Let any make target specify ninja goals that needs to be built for it
(though selecting the goals is _not_ recursive on depending targets)
instead of having a custom mechanism only for "make check" and "make
bench".

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-4-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220527153603.887929-15-alex.bennee@linaro.org>
---
 Makefile              | 3 +--
 scripts/mtest2make.py | 8 ++++----
 2 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/Makefile b/Makefile
index fad312040f..3c0d89057e 100644
--- a/Makefile
+++ b/Makefile
@@ -145,8 +145,7 @@ NINJAFLAGS = $(if $V,-v) $(if $(MAKE.n), -n) $(if $(MAKE.k), -k0) \
         $(filter-out -j, $(lastword -j1 $(filter -l% -j%, $(MAKEFLAGS)))) \
         -d keepdepfile
 ninja-cmd-goals = $(or $(MAKECMDGOALS), all)
-ninja-cmd-goals += $(foreach t, $(.check.build-suites), $(.check-$t.deps))
-ninja-cmd-goals += $(foreach t, $(.bench.build-suites), $(.bench-$t.deps))
+ninja-cmd-goals += $(foreach g, $(MAKECMDGOALS), $(.ninja-goals.$g))))
 
 makefile-targets := build.ninja ctags TAGS cscope dist clean uninstall
 # "ninja -t targets" also lists all prerequisites.  If build system
diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
index 304634b71e..0fe81efbbc 100644
--- a/scripts/mtest2make.py
+++ b/scripts/mtest2make.py
@@ -81,12 +81,12 @@ def emit_prolog(suites, prefix):
 
 def emit_suite_deps(name, suite, prefix):
     deps = ' '.join(suite.deps)
-    targets = f'{prefix}-{name} {prefix}-report-{name}.junit.xml {prefix} {prefix}-report.junit.xml'
+    targets = [f'{prefix}-{name}', f'{prefix}-report-{name}.junit.xml', f'{prefix}', f'{prefix}-report.junit.xml',
+               f'{prefix}-build']
     print()
     print(f'.{prefix}-{name}.deps = {deps}')
-    print(f'ifneq ($(filter {prefix}-build {targets}, $(MAKECMDGOALS)),)')
-    print(f'.{prefix}.build-suites += {name}')
-    print(f'endif')
+    for t in targets:
+        print(f'.ninja-goals.{t} += $(.{prefix}-{name}.deps)')
 
 def emit_suite(name, suite, prefix):
     emit_suite_deps(name, suite, prefix)

From 95808875be96e17b885a9e4f050897c67e8b9aaf Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 27 May 2022 16:35:45 +0100
Subject: [PATCH 716/935] build: do a full build before running TCG tests
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

TCG tests need both QEMU and firmware to be built, so do "ninja all" before
trying to run them.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517092616.1272238-5-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220527153603.887929-16-alex.bennee@linaro.org>
---
 tests/Makefile.include | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index ec84b2ebc0..72ce0561f4 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -57,7 +57,7 @@ $(TCG_TESTS_TARGETS:%=build-tcg-tests-%): build-tcg-tests-%: $(BUILD_DIR)/tests/
         "BUILD","$* guest-tests")
 
 .PHONY: $(TCG_TESTS_TARGETS:%=run-tcg-tests-%)
-$(TCG_TESTS_TARGETS:%=run-tcg-tests-%): run-tcg-tests-%: build-tcg-tests-% $(if $(CONFIG_PLUGIN),test-plugins)
+$(TCG_TESTS_TARGETS:%=run-tcg-tests-%): run-tcg-tests-%: build-tcg-tests-%
 	$(call quiet-command, \
            $(MAKE) -C tests/tcg/$* -f ../Makefile.target $(SUBDIR_MAKEFLAGS) \
                         TARGET="$*" SRC_PATH="$(SRC_PATH)" SPEED=$(SPEED) run, \
@@ -74,6 +74,7 @@ $(TCG_TESTS_TARGETS:%=clean-tcg-tests-%): clean-tcg-tests-%:
 build-tcg: $(BUILD_TCG_TARGET_RULES)
 
 .PHONY: check-tcg
+.ninja-goals.check-tcg = all $(if $(CONFIG_PLUGIN),test-plugins)
 check-tcg: $(RUN_TCG_TARGET_RULES)
 
 .PHONY: clean-tcg

From 45f1eecdd63f9e4fa93fef01dd826e7706ac6d7b Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 27 May 2022 16:35:46 +0100
Subject: [PATCH 717/935] configure, meson: move symlinking of ROMs to meson
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This is useful because pc-bios/meson.build already has a list of all ROM
files, and thus does not need to use wildcards.  The problems with
wildcards are mentioned above the definition of the LINKS variable,
but then the recommendation is disattended.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-6-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220527153603.887929-17-alex.bennee@linaro.org>
---
 configure           | 15 ---------------
 pc-bios/meson.build | 17 ++++++++++++-----
 2 files changed, 12 insertions(+), 20 deletions(-)

diff --git a/configure b/configure
index c88ef94fec..7b6adc29fe 100755
--- a/configure
+++ b/configure
@@ -2100,21 +2100,6 @@ LINKS="$LINKS tests/avocado tests/data"
 LINKS="$LINKS tests/qemu-iotests/check"
 LINKS="$LINKS python"
 LINKS="$LINKS contrib/plugins/Makefile "
-for bios_file in \
-    $source_path/pc-bios/*.bin \
-    $source_path/pc-bios/*.elf \
-    $source_path/pc-bios/*.lid \
-    $source_path/pc-bios/*.rom \
-    $source_path/pc-bios/*.dtb \
-    $source_path/pc-bios/*.img \
-    $source_path/pc-bios/openbios-* \
-    $source_path/pc-bios/u-boot.* \
-    $source_path/pc-bios/palcode-* \
-    $source_path/pc-bios/qemu_vga.ndrv
-
-do
-    LINKS="$LINKS pc-bios/$(basename $bios_file)"
-done
 for f in $LINKS ; do
     if [ -e "$source_path/$f" ]; then
         mkdir -p `dirname ./$f`
diff --git a/pc-bios/meson.build b/pc-bios/meson.build
index c86dedf7df..41ba1c0ec7 100644
--- a/pc-bios/meson.build
+++ b/pc-bios/meson.build
@@ -23,7 +23,7 @@ if unpack_edk2_blobs
   endforeach
 endif
 
-blobs = files(
+blobs = [
   'bios.bin',
   'bios-256k.bin',
   'bios-microvm.bin',
@@ -83,11 +83,18 @@ blobs = files(
   'npcm7xx_bootrom.bin',
   'vof.bin',
   'vof-nvram.bin',
-)
+]
 
-if get_option('install_blobs')
-  install_data(blobs, install_dir: qemu_datadir)
-endif
+ln_s = [find_program('ln', required: true), '-sf']
+foreach f : blobs
+  roms += custom_target(f,
+                build_by_default: have_system,
+                output: f,
+                input: files('meson.build'),            # dummy input
+                install: get_option('install_blobs'),
+                install_dir: qemu_datadir,
+                command: [ ln_s, meson.project_source_root() / 'pc-bios' / f, '@OUTPUT@' ])
+endforeach
 
 subdir('descriptors')
 subdir('keymaps')

From 867998cf5680329a150cb6d51a15d6990be49c77 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 27 May 2022 16:35:47 +0100
Subject: [PATCH 718/935] tests/tcg: correct target CPU for sparc32
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

We do not want v8plus for pure sparc32, as the difference with the V8 ABI
are only meaningful on 64-bit CPUs suh as ultrasparc; supersparc is the
best CPU to use for 32-bit.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517092616.1272238-7-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220527153603.887929-18-alex.bennee@linaro.org>
---
 tests/tcg/configure.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index 691d90abac..59f2403d1a 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -70,7 +70,7 @@ fi
 : ${cross_cc_riscv64="riscv64-linux-gnu-gcc"}
 : ${cross_cc_s390x="s390x-linux-gnu-gcc"}
 : ${cross_cc_sh4="sh4-linux-gnu-gcc"}
-: ${cross_cc_cflags_sparc="-m32 -mv8plus -mcpu=ultrasparc"}
+: ${cross_cc_cflags_sparc="-m32 -mcpu=supersparc"}
 : ${cross_cc_sparc64="sparc64-linux-gnu-gcc"}
 : ${cross_cc_cflags_sparc64="-m64 -mcpu=ultrasparc"}
 : ${cross_cc_x86_64="x86_64-linux-gnu-gcc"}

From cd362defbbd09cbbc08b3bb465141542887b8cef Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 27 May 2022 16:35:48 +0100
Subject: [PATCH 719/935] tests/tcg: merge configure.sh back into main
 configure script
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

tests/tcg/configure.sh has a complicated story.

In the beginning its code ran as part of the creation of config-target.mak
files, and that is where it placed the information on the target compiler.
However, probing for the buildability of TCG tests required multiple
inclusions of config-target.mak in the _main_ Makefile (not in
Makefile.target, which took care of building the QEMU executables in
the pre-Meson era), which polluted the namespace.

Thus, it was moved to a separate directory.  It created small config-*.mak
files in $(BUILD_DIR)/tests/tcg.  Those were also included multiple
times, but at least they were small and manageable; this was also an
important step in disentangling the TCG tests from Makefile.target.

Since then, Meson has allowed the configure script to go on a diet.
A few compilation tests survive (mostly for sanitizers) but these days
it mostly takes care of command line parsing, looking for tools, and
setting up the environment for Meson to do its stuff.

It's time to extend configure with the capability to build for more
than just one target: not just tests, but also firmware.  As a first
step, integrate all the logic to find cross compilers in the configure
script, and move tests/tcg/configure.sh back there (though as a
separate loop, not integrated in the one that generates target
configurations for Meson).

tests/tcg is actually very close to being buildable as a standalone
project, so I actually expect the compiler tests to move back to
tests/tcg, as a "configure" script of sorts which would run at Make
time after the docker images are built.  The GCC tree has a similar idea
of doing only bare-bones tree-wide configuration and leaving the rest
for Make time.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517092616.1272238-8-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220527153603.887929-19-alex.bennee@linaro.org>
---
 configure              | 398 +++++++++++++++++++++++++++++++++++++++--
 tests/Makefile.include |   1 -
 tests/tcg/configure.sh | 376 --------------------------------------
 3 files changed, 388 insertions(+), 387 deletions(-)
 delete mode 100755 tests/tcg/configure.sh

diff --git a/configure b/configure
index 7b6adc29fe..f91ac632e7 100755
--- a/configure
+++ b/configure
@@ -109,6 +109,20 @@ error_exit() {
 }
 
 do_compiler() {
+  # Run the compiler, capturing its output to the log. First argument
+  # is compiler binary to execute.
+  local compiler="$1"
+  shift
+  if test -n "$BASH_VERSION"; then eval '
+      echo >>config.log "
+funcs: ${FUNCNAME[*]}
+lines: ${BASH_LINENO[*]}"
+  '; fi
+  echo $compiler "$@" >> config.log
+  $compiler "$@" >> config.log 2>&1 || return $?
+}
+
+do_compiler_werror() {
     # Run the compiler, capturing its output to the log. First argument
     # is compiler binary to execute.
     compiler="$1"
@@ -142,15 +156,15 @@ lines: ${BASH_LINENO[*]}"
 }
 
 do_cc() {
-    do_compiler "$cc" $CPU_CFLAGS "$@"
+    do_compiler_werror "$cc" $CPU_CFLAGS "$@"
 }
 
 do_cxx() {
-    do_compiler "$cxx" $CPU_CFLAGS "$@"
+    do_compiler_werror "$cxx" $CPU_CFLAGS "$@"
 }
 
 do_objc() {
-    do_compiler "$objcc" $CPU_CFLAGS "$@"
+    do_compiler_werror "$objcc" $CPU_CFLAGS "$@"
 }
 
 # Append $2 to the variable named $1, with space separation
@@ -345,11 +359,9 @@ for opt do
   ;;
   --cross-cc-cflags-*) cc_arch=${opt#--cross-cc-cflags-}; cc_arch=${cc_arch%%=*}
                       eval "cross_cc_cflags_${cc_arch}=\$optarg"
-                      cross_cc_vars="$cross_cc_vars cross_cc_cflags_${cc_arch}"
   ;;
   --cross-cc-*) cc_arch=${opt#--cross-cc-}; cc_arch=${cc_arch%%=*}
                 eval "cross_cc_${cc_arch}=\$optarg"
-                cross_cc_vars="$cross_cc_vars cross_cc_${cc_arch}"
   ;;
   esac
 done
@@ -944,7 +956,6 @@ esac
 
 if eval test -z "\${cross_cc_$cpu}"; then
     eval "cross_cc_${cpu}=\$cc"
-    cross_cc_vars="$cross_cc_vars cross_cc_${cpu}"
 fi
 
 default_target_list=""
@@ -1800,6 +1811,248 @@ case "$slirp" in
     ;;
 esac
 
+##########################################
+# functions to probe cross compilers
+
+container="no"
+if test $use_containers = "yes"; then
+    if has "docker" || has "podman"; then
+        container=$($python $source_path/tests/docker/docker.py probe)
+    fi
+fi
+
+# cross compilers defaults, can be overridden with --cross-cc-ARCH
+: ${cross_cc_aarch64="aarch64-linux-gnu-gcc"}
+: ${cross_cc_aarch64_be="$cross_cc_aarch64"}
+: ${cross_cc_cflags_aarch64_be="-mbig-endian"}
+: ${cross_cc_alpha="alpha-linux-gnu-gcc"}
+: ${cross_cc_arm="arm-linux-gnueabihf-gcc"}
+: ${cross_cc_cflags_armeb="-mbig-endian"}
+: ${cross_cc_hexagon="hexagon-unknown-linux-musl-clang"}
+: ${cross_cc_cflags_hexagon="-mv67 -O2 -static"}
+: ${cross_cc_hppa="hppa-linux-gnu-gcc"}
+: ${cross_cc_i386="i686-linux-gnu-gcc"}
+: ${cross_cc_cflags_i386="-m32"}
+: ${cross_cc_m68k="m68k-linux-gnu-gcc"}
+: ${cross_cc_microblaze="microblaze-linux-musl-gcc"}
+: ${cross_cc_mips64el="mips64el-linux-gnuabi64-gcc"}
+: ${cross_cc_mips64="mips64-linux-gnuabi64-gcc"}
+: ${cross_cc_mipsel="mipsel-linux-gnu-gcc"}
+: ${cross_cc_mips="mips-linux-gnu-gcc"}
+: ${cross_cc_nios2="nios2-linux-gnu-gcc"}
+: ${cross_cc_ppc="powerpc-linux-gnu-gcc"}
+: ${cross_cc_cflags_ppc="-m32"}
+: ${cross_cc_ppc64="powerpc64-linux-gnu-gcc"}
+: ${cross_cc_cflags_ppc64="-m64 -mbig-endian"}
+: ${cross_cc_ppc64le="$cross_cc_ppc64"}
+: ${cross_cc_cflags_ppc64le="-m64 -mlittle-endian"}
+: ${cross_cc_riscv64="riscv64-linux-gnu-gcc"}
+: ${cross_cc_s390x="s390x-linux-gnu-gcc"}
+: ${cross_cc_sh4="sh4-linux-gnu-gcc"}
+: ${cross_cc_cflags_sparc="-m32 -mcpu=supersparc"}
+: ${cross_cc_sparc64="sparc64-linux-gnu-gcc"}
+: ${cross_cc_cflags_sparc64="-m64 -mcpu=ultrasparc"}
+: ${cross_cc_x86_64="x86_64-linux-gnu-gcc"}
+: ${cross_cc_cflags_x86_64="-m64"}
+
+# tricore is special as it doesn't have a compiler
+: ${cross_as_tricore="tricore-as"}
+: ${cross_ld_tricore="tricore-ld"}
+
+probe_target_compiler() {
+  # reset all output variables
+  container_image=
+  container_hosts=
+  container_cross_cc=
+  container_cross_as=
+  container_cross_ld=
+  target_cc=
+  target_as=
+  target_ld=
+
+  case $1 in
+    aarch64) container_hosts="x86_64 aarch64" ;;
+    alpha) container_hosts=x86_64 ;;
+    arm) container_hosts="x86_64 aarch64" ;;
+    cris) container_hosts=x86_64 ;;
+    hexagon) container_hosts=x86_64 ;;
+    hppa) container_hosts=x86_64 ;;
+    i386) container_hosts=x86_64 ;;
+    m68k) container_hosts=x86_64 ;;
+    microblaze) container_hosts=x86_64 ;;
+    mips64el) container_hosts=x86_64 ;;
+    mips64) container_hosts=x86_64 ;;
+    mipsel) container_hosts=x86_64 ;;
+    mips) container_hosts=x86_64 ;;
+    nios2) container_hosts=x86_64 ;;
+    ppc) container_hosts=x86_64 ;;
+    ppc64|ppc64le) container_hosts=x86_64 ;;
+    riscv64) container_hosts=x86_64 ;;
+    s390x) container_hosts=x86_64 ;;
+    sh4) container_hosts=x86_64 ;;
+    sparc64) container_hosts=x86_64 ;;
+    tricore) container_hosts=x86_64 ;;
+    x86_64) container_hosts="aarch64 ppc64el x86_64" ;;
+    xtensa*) container_hosts=x86_64 ;;
+  esac
+
+  for host in $container_hosts; do
+    test "$container" != no || continue
+    test "$host" = "$cpu" || continue
+    case $1 in
+      aarch64)
+        # We don't have any bigendian build tools so we only use this for AArch64
+        container_image=debian-arm64-cross
+        container_cross_cc=aarch64-linux-gnu-gcc-10
+        ;;
+      alpha)
+        container_image=debian-alpha-cross
+        container_cross_cc=alpha-linux-gnu-gcc
+        ;;
+      arm)
+        # We don't have any bigendian build tools so we only use this for ARM
+        container_image=debian-armhf-cross
+        container_cross_cc=arm-linux-gnueabihf-gcc
+        ;;
+      cris)
+        container_image=fedora-cris-cross
+        container_cross_cc=cris-linux-gnu-gcc
+        ;;
+      hexagon)
+        container_image=debian-hexagon-cross
+        container_cross_cc=hexagon-unknown-linux-musl-clang
+        ;;
+      hppa)
+        container_image=debian-hppa-cross
+        container_cross_cc=hppa-linux-gnu-gcc
+        ;;
+      i386)
+        container_image=fedora-i386-cross
+        container_cross_cc=gcc
+        ;;
+      m68k)
+        container_image=debian-m68k-cross
+        container_cross_cc=m68k-linux-gnu-gcc
+        ;;
+      microblaze)
+        container_image=debian-microblaze-cross
+        container_cross_cc=microblaze-linux-musl-gcc
+        ;;
+      mips64el)
+        container_image=debian-mips64el-cross
+        container_cross_cc=mips64el-linux-gnuabi64-gcc
+        ;;
+      mips64)
+        container_image=debian-mips64-cross
+        container_cross_cc=mips64-linux-gnuabi64-gcc
+        ;;
+      mipsel)
+        container_image=debian-mipsel-cross
+        container_cross_cc=mipsel-linux-gnu-gcc
+        ;;
+      mips)
+        container_image=debian-mips-cross
+        container_cross_cc=mips-linux-gnu-gcc
+        ;;
+      nios2)
+        container_image=debian-nios2-cross
+        container_cross_cc=nios2-linux-gnu-gcc
+        ;;
+      ppc)
+        container_image=debian-powerpc-test-cross
+        container_cross_cc=powerpc-linux-gnu-gcc-10
+        ;;
+      ppc64|ppc64le)
+        container_image=debian-powerpc-test-cross
+        container_cross_cc=powerpc${1#ppc}-linux-gnu-gcc-10
+        ;;
+      riscv64)
+        container_image=debian-riscv64-test-cross
+        container_cross_cc=riscv64-linux-gnu-gcc
+        ;;
+      s390x)
+        container_image=debian-s390x-cross
+        container_cross_cc=s390x-linux-gnu-gcc
+        ;;
+      sh4)
+        container_image=debian-sh4-cross
+        container_cross_cc=sh4-linux-gnu-gcc
+        ;;
+      sparc64)
+        container_image=debian-sparc64-cross
+        container_cross_cc=sparc64-linux-gnu-gcc
+        ;;
+      tricore)
+        container_image=debian-tricore-cross
+        container_cross_as=tricore-as
+        container_cross_ld=tricore-ld
+        ;;
+      x86_64)
+        container_image=debian-amd64-cross
+        container_cross_cc=x86_64-linux-gnu-gcc
+        ;;
+      xtensa*)
+        # FIXME: xtensa-linux-user?
+        container_hosts=x86_64
+        container_image=debian-xtensa-cross
+
+        # default to the dc232b cpu
+        container_cross_cc=/opt/2020.07/xtensa-dc232b-elf/bin/xtensa-dc232b-elf-gcc
+        ;;
+    esac
+  done
+
+  eval "target_cflags=\${cross_cc_cflags_$1}"
+  if eval test -n "\"\${cross_cc_$1}\""; then
+    if eval has "\"\${cross_cc_$1}\""; then
+      eval "target_cc=\"\${cross_cc_$1}\""
+      case $1 in
+        i386|x86_64)
+          if $target_cc --version | grep -qi "clang"; then
+            unset target_cc
+          fi
+          ;;
+      esac
+    fi
+  fi
+  if eval test -n "\"\${cross_as_$1}\""; then
+    if eval has "\"\${cross_as_$1}\""; then
+      eval "target_as=\"\${cross_as_$1}\""
+    fi
+  fi
+  if eval test -n "\"\${cross_ld_$1}\""; then
+    if eval has "\"\${cross_ld_$1}\""; then
+      eval "target_ld=\"\${cross_ld_$1}\""
+    fi
+  fi
+}
+
+write_target_makefile() {
+  if test -n "$target_cc"; then
+    echo "CC=$target_cc"
+  fi
+  if test -n "$target_as"; then
+    echo "AS=$target_as"
+  fi
+  if test -n "$target_ld"; then
+    echo "LD=$target_ld"
+  fi
+}
+
+write_container_target_makefile() {
+  if test -n "$container_cross_cc"; then
+    echo "CC=\$(DOCKER_SCRIPT) cc --cc $container_cross_cc -i qemu/$container_image -s $source_path --"
+  fi
+  if test -n "$container_cross_as"; then
+    echo "AS=\$(DOCKER_SCRIPT) cc --cc $container_cross_as -i qemu/$container_image -s $source_path --"
+  fi
+  if test -n "$container_cross_ld"; then
+    echo "LD=\$(DOCKER_SCRIPT) cc --cc $container_cross_ld -i qemu/$container_image -s $source_path --"
+  fi
+}
+
+
+
 ##########################################
 # End of CC checks
 # After here, no more $cc or $ld runs
@@ -2107,11 +2360,136 @@ for f in $LINKS ; do
     fi
 done
 
-(for i in $cross_cc_vars; do
-  export $i
+# tests/tcg configuration
+(makefile=tests/tcg/Makefile.prereqs
+echo "# Automatically generated by configure - do not modify" > $makefile
+
+config_host_mak=tests/tcg/config-host.mak
+echo "# Automatically generated by configure - do not modify" > $config_host_mak
+echo "SRC_PATH=$source_path" >> $config_host_mak
+echo "HOST_CC=$host_cc" >> $config_host_mak
+
+tcg_tests_targets=
+for target in $target_list; do
+  arch=${target%%-*}
+
+  probe_target_compiler ${arch}
+  config_target_mak=tests/tcg/config-$target.mak
+
+  echo "# Automatically generated by configure - do not modify" > $config_target_mak
+  echo "TARGET_NAME=$arch" >> $config_target_mak
+  case $target in
+    *-softmmu)
+      test -f $source_path/tests/tcg/$arch/Makefile.softmmu-target || continue
+      qemu="qemu-system-$arch"
+      ;;
+    *-linux-user|*-bsd-user)
+      qemu="qemu-$arch"
+      ;;
+  esac
+
+  got_cross_cc=no
+  unset build_static
+
+  if test -n "$target_cc"; then
+      write_c_skeleton
+      if ! do_compiler "$target_cc" $target_cflags \
+           -o $TMPE $TMPC -static ; then
+          # For host systems we might get away with building without -static
+          if do_compiler "$target_cc" $target_cflags \
+                         -o $TMPE $TMPC ; then
+              got_cross_cc=yes
+          fi
+      else
+          got_cross_cc=yes
+          build_static=y
+      fi
+  elif test -n "$target_as" && test -n "$target_ld"; then
+      # Special handling for assembler only tests
+      case $target in
+          tricore-softmmu) got_cross_cc=yes ;;
+      esac
+  fi
+
+  if test $got_cross_cc = yes; then
+      # Test for compiler features for optional tests. We only do this
+      # for cross compilers because ensuring the docker containers based
+      # compilers is a requirememt for adding a new test that needs a
+      # compiler feature.
+
+      echo "BUILD_STATIC=$build_static" >> $config_target_mak
+      write_target_makefile >> $config_target_mak
+      case $target in
+          aarch64-*)
+              if do_compiler "$target_cc" $target_cflags \
+                             -march=armv8.1-a+sve -o $TMPE $TMPC; then
+                  echo "CROSS_CC_HAS_SVE=y" >> $config_target_mak
+              fi
+              if do_compiler "$target_cc" $target_cflags \
+                             -march=armv8.1-a+sve2 -o $TMPE $TMPC; then
+                  echo "CROSS_CC_HAS_SVE2=y" >> $config_target_mak
+              fi
+              if do_compiler "$target_cc" $target_cflags \
+                             -march=armv8.3-a -o $TMPE $TMPC; then
+                  echo "CROSS_CC_HAS_ARMV8_3=y" >> $config_target_mak
+              fi
+              if do_compiler "$target_cc" $target_cflags \
+                             -mbranch-protection=standard -o $TMPE $TMPC; then
+                  echo "CROSS_CC_HAS_ARMV8_BTI=y" >> $config_target_mak
+              fi
+              if do_compiler "$target_cc" $target_cflags \
+                             -march=armv8.5-a+memtag -o $TMPE $TMPC; then
+                  echo "CROSS_CC_HAS_ARMV8_MTE=y" >> $config_target_mak
+              fi
+              ;;
+          ppc*)
+              if do_compiler "$target_cc" $target_cflags \
+                             -mpower8-vector -o $TMPE $TMPC; then
+                  echo "CROSS_CC_HAS_POWER8_VECTOR=y" >> $config_target_mak
+              fi
+              if do_compiler "$target_cc" $target_cflags \
+                             -mpower10 -o $TMPE $TMPC; then
+                  echo "CROSS_CC_HAS_POWER10=y" >> $config_target_mak
+              fi
+              ;;
+          i386-linux-user)
+              if do_compiler "$target_cc" $target_cflags \
+                             -Werror -fno-pie -o $TMPE $TMPC; then
+                  echo "CROSS_CC_HAS_I386_NOPIE=y" >> $config_target_mak
+              fi
+              ;;
+      esac
+  elif test -n "$container_image"; then
+      echo "build-tcg-tests-$target: docker-image-$container_image" >> $makefile
+      echo "BUILD_STATIC=y" >> $config_target_mak
+      write_container_target_makefile >> $config_target_mak
+      case $target in
+          aarch64-*)
+              echo "CROSS_CC_HAS_SVE=y" >> $config_target_mak
+              echo "CROSS_CC_HAS_SVE2=y" >> $config_target_mak
+              echo "CROSS_CC_HAS_ARMV8_3=y" >> $config_target_mak
+              echo "CROSS_CC_HAS_ARMV8_BTI=y" >> $config_target_mak
+              echo "CROSS_CC_HAS_ARMV8_MTE=y" >> $config_target_mak
+              ;;
+          ppc*)
+              echo "CROSS_CC_HAS_POWER8_VECTOR=y" >> $config_target_mak
+              echo "CROSS_CC_HAS_POWER10=y" >> $config_target_mak
+              ;;
+          i386-linux-user)
+              echo "CROSS_CC_HAS_I386_NOPIE=y" >> $config_target_mak
+              ;;
+      esac
+      got_cross_cc=yes
+  fi
+  if test $got_cross_cc = yes; then
+      mkdir -p tests/tcg/$target
+      echo "QEMU=$PWD/$qemu" >> $config_target_mak
+      echo "EXTRA_CFLAGS=$target_cflags" >> $config_target_mak
+      echo "run-tcg-tests-$target: $qemu\$(EXESUF)" >> $makefile
+      tcg_tests_targets="$tcg_tests_targets $target"
+  fi
 done
-export target_list source_path use_containers cpu host_cc
-$source_path/tests/tcg/configure.sh)
+echo "TCG_TESTS_TARGETS=$tcg_tests_targets" >> $makefile)
 
 config_mak=pc-bios/optionrom/config.mak
 echo "# Automatically generated by configure - do not modify" > $config_mak
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 72ce0561f4..6a1688e33e 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -37,7 +37,6 @@ export SRC_PATH
 SPEED = quick
 
 -include tests/tcg/Makefile.prereqs
-config-host.mak: $(SRC_PATH)/tests/tcg/configure.sh
 tests/tcg/Makefile.prereqs: config-host.mak
 
 # Per guest TCG tests
diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
deleted file mode 100755
index 59f2403d1a..0000000000
--- a/tests/tcg/configure.sh
+++ /dev/null
@@ -1,376 +0,0 @@
-#! /bin/sh
-
-if test -z "$source_path"; then
-  echo Do not invoke this script directly.  It is called
-  echo automatically by configure.
-  exit 1
-fi
-
-write_c_skeleton() {
-    cat > $TMPC <<EOF
-int main(void) { return 0; }
-EOF
-}
-
-has() {
-  command -v "$1" >/dev/null 2>&1
-}
-
-do_compiler() {
-  # Run the compiler, capturing its output to the log. First argument
-  # is compiler binary to execute.
-  local compiler="$1"
-  shift
-  if test -n "$BASH_VERSION"; then eval '
-      echo >>config.log "
-funcs: ${FUNCNAME[*]}
-lines: ${BASH_LINENO[*]}"
-  '; fi
-  echo $compiler "$@" >> config.log
-  $compiler "$@" >> config.log 2>&1 || return $?
-}
-
-
-TMPDIR1="config-temp"
-TMPC="${TMPDIR1}/qemu-conf.c"
-TMPE="${TMPDIR1}/qemu-conf.exe"
-
-container="no"
-if test $use_containers = "yes"; then
-    if has "docker" || has "podman"; then
-        container=$($python $source_path/tests/docker/docker.py probe)
-    fi
-fi
-
-# cross compilers defaults, can be overridden with --cross-cc-ARCH
-: ${cross_cc_aarch64="aarch64-linux-gnu-gcc"}
-: ${cross_cc_aarch64_be="$cross_cc_aarch64"}
-: ${cross_cc_cflags_aarch64_be="-mbig-endian"}
-: ${cross_cc_alpha="alpha-linux-gnu-gcc"}
-: ${cross_cc_arm="arm-linux-gnueabihf-gcc"}
-: ${cross_cc_cflags_armeb="-mbig-endian"}
-: ${cross_cc_hexagon="hexagon-unknown-linux-musl-clang"}
-: ${cross_cc_cflags_hexagon="-mv67 -O2 -static"}
-: ${cross_cc_hppa="hppa-linux-gnu-gcc"}
-: ${cross_cc_i386="i686-linux-gnu-gcc"}
-: ${cross_cc_cflags_i386="-m32"}
-: ${cross_cc_m68k="m68k-linux-gnu-gcc"}
-: ${cross_cc_microblaze="microblaze-linux-musl-gcc"}
-: ${cross_cc_mips64el="mips64el-linux-gnuabi64-gcc"}
-: ${cross_cc_mips64="mips64-linux-gnuabi64-gcc"}
-: ${cross_cc_mipsel="mipsel-linux-gnu-gcc"}
-: ${cross_cc_mips="mips-linux-gnu-gcc"}
-: ${cross_cc_nios2="nios2-linux-gnu-gcc"}
-: ${cross_cc_ppc="powerpc-linux-gnu-gcc"}
-: ${cross_cc_cflags_ppc="-m32"}
-: ${cross_cc_ppc64="powerpc64-linux-gnu-gcc"}
-: ${cross_cc_cflags_ppc64="-m64 -mbig-endian"}
-: ${cross_cc_ppc64le="$cross_cc_ppc64"}
-: ${cross_cc_cflags_ppc64le="-m64 -mlittle-endian"}
-: ${cross_cc_riscv64="riscv64-linux-gnu-gcc"}
-: ${cross_cc_s390x="s390x-linux-gnu-gcc"}
-: ${cross_cc_sh4="sh4-linux-gnu-gcc"}
-: ${cross_cc_cflags_sparc="-m32 -mcpu=supersparc"}
-: ${cross_cc_sparc64="sparc64-linux-gnu-gcc"}
-: ${cross_cc_cflags_sparc64="-m64 -mcpu=ultrasparc"}
-: ${cross_cc_x86_64="x86_64-linux-gnu-gcc"}
-: ${cross_cc_cflags_x86_64="-m64"}
-
-# tricore is special as it doesn't have a compiler
-: ${cross_as_tricore="tricore-as"}
-: ${cross_ld_tricore="tricore-ld"}
-
-makefile=tests/tcg/Makefile.prereqs
-echo "# Automatically generated by configure - do not modify" > $makefile
-
-config_host_mak=tests/tcg/config-host.mak
-echo "# Automatically generated by configure - do not modify" > $config_host_mak
-echo "SRC_PATH=$source_path" >> $config_host_mak
-echo "HOST_CC=$host_cc" >> $config_host_mak
-
-tcg_tests_targets=
-for target in $target_list; do
-  arch=${target%%-*}
-
-  # reset all container fields
-  container_image=
-  container_hosts=
-  container_cross_cc=
-  container_cross_as=
-  container_cross_ld=
-
-  # suppress clang
-  supress_clang=
-
-  case $target in
-    aarch64-*)
-      # We don't have any bigendian build tools so we only use this for AArch64
-      container_hosts="x86_64 aarch64"
-      container_image=debian-arm64-cross
-      container_cross_cc=aarch64-linux-gnu-gcc-10
-      ;;
-    alpha-*)
-      container_hosts=x86_64
-      container_image=debian-alpha-cross
-      container_cross_cc=alpha-linux-gnu-gcc
-      ;;
-    arm-*)
-      # We don't have any bigendian build tools so we only use this for ARM
-      container_hosts="x86_64 aarch64"
-      container_image=debian-armhf-cross
-      container_cross_cc=arm-linux-gnueabihf-gcc
-      ;;
-    cris-*)
-      container_hosts=x86_64
-      container_image=fedora-cris-cross
-      container_cross_cc=cris-linux-gnu-gcc
-      ;;
-    hexagon-*)
-      container_hosts=x86_64
-      container_image=debian-hexagon-cross
-      container_cross_cc=hexagon-unknown-linux-musl-clang
-      ;;
-    hppa-*)
-      container_hosts=x86_64
-      container_image=debian-hppa-cross
-      container_cross_cc=hppa-linux-gnu-gcc
-      ;;
-    i386-*)
-      container_hosts=x86_64
-      container_image=fedora-i386-cross
-      container_cross_cc=gcc
-      supress_clang=yes
-      ;;
-    m68k-*)
-      container_hosts=x86_64
-      container_image=debian-m68k-cross
-      container_cross_cc=m68k-linux-gnu-gcc
-      ;;
-    microblaze-*)
-      container_hosts=x86_64
-      container_image=debian-microblaze-cross
-      container_cross_cc=microblaze-linux-musl-gcc
-      ;;
-    mips64el-*)
-      container_hosts=x86_64
-      container_image=debian-mips64el-cross
-      container_cross_cc=mips64el-linux-gnuabi64-gcc
-      ;;
-    mips64-*)
-      container_hosts=x86_64
-      container_image=debian-mips64-cross
-      container_cross_cc=mips64-linux-gnuabi64-gcc
-      ;;
-    mipsel-*)
-      container_hosts=x86_64
-      container_image=debian-mipsel-cross
-      container_cross_cc=mipsel-linux-gnu-gcc
-      ;;
-    mips-*)
-      container_hosts=x86_64
-      container_image=debian-mips-cross
-      container_cross_cc=mips-linux-gnu-gcc
-      ;;
-    nios2-*)
-      container_hosts=x86_64
-      container_image=debian-nios2-cross
-      container_cross_cc=nios2-linux-gnu-gcc
-      ;;
-    ppc-*)
-      container_hosts=x86_64
-      container_image=debian-powerpc-test-cross
-      container_cross_cc=powerpc-linux-gnu-gcc-10
-      ;;
-    ppc64-*|ppc64le-*)
-      container_hosts=x86_64
-      container_image=debian-powerpc-test-cross
-      container_cross_cc=${target%%-*}-linux-gnu-gcc-10
-      container_cross_cc=powerpc${container_cross_cc#ppc}
-      ;;
-    riscv64-*)
-      container_hosts=x86_64
-      container_image=debian-riscv64-test-cross
-      container_cross_cc=riscv64-linux-gnu-gcc
-      ;;
-    s390x-*)
-      container_hosts=x86_64
-      container_image=debian-s390x-cross
-      container_cross_cc=s390x-linux-gnu-gcc
-      ;;
-    sh4-*)
-      container_hosts=x86_64
-      container_image=debian-sh4-cross
-      container_cross_cc=sh4-linux-gnu-gcc
-      ;;
-    sparc64-*)
-      container_hosts=x86_64
-      container_image=debian-sparc64-cross
-      container_cross_cc=sparc64-linux-gnu-gcc
-      ;;
-    tricore-softmmu)
-      container_hosts=x86_64
-      container_image=debian-tricore-cross
-      container_cross_as=tricore-as
-      container_cross_ld=tricore-ld
-      ;;
-    x86_64-*)
-      container_hosts="aarch64 ppc64el x86_64"
-      container_image=debian-amd64-cross
-      container_cross_cc=x86_64-linux-gnu-gcc
-      supress_clang=yes
-      ;;
-    xtensa*-softmmu)
-      container_hosts=x86_64
-      container_image=debian-xtensa-cross
-
-      # default to the dc232b cpu
-      container_cross_cc=/opt/2020.07/xtensa-dc232b-elf/bin/xtensa-dc232b-elf-gcc
-      ;;
-  esac
-
-  config_target_mak=tests/tcg/config-$target.mak
-
-  echo "# Automatically generated by configure - do not modify" > $config_target_mak
-  echo "TARGET_NAME=$arch" >> $config_target_mak
-  case $target in
-    *-softmmu)
-      test -f $source_path/tests/tcg/$arch/Makefile.softmmu-target || continue
-      qemu="qemu-system-$arch"
-      ;;
-    *-linux-user|*-bsd-user)
-      qemu="qemu-$arch"
-      ;;
-  esac
-
-  eval "target_compiler_cflags=\${cross_cc_cflags_$arch}"
-
-  got_cross_cc=no
-
-  if eval test "x\"\${cross_cc_$arch}\"" != xyes; then
-      eval "target_compiler=\"\${cross_cc_$arch}\""
-
-      if has $target_compiler; then
-          if test "$supress_clang" = yes &&
-                  $target_compiler --version | grep -qi "clang"; then
-              got_cross_cc=no
-          else
-              write_c_skeleton
-              if ! do_compiler "$target_compiler" $target_compiler_cflags \
-                   -o $TMPE $TMPC -static ; then
-                  # For host systems we might get away with building without -static
-                  if do_compiler "$target_compiler" $target_compiler_cflags \
-                                 -o $TMPE $TMPC ; then
-                      got_cross_cc=yes
-                      echo "CC=$target_compiler" >> $config_target_mak
-                  fi
-              else
-                  got_cross_cc=yes
-                  echo "BUILD_STATIC=y" >> $config_target_mak
-                  echo "CC=$target_compiler" >> $config_target_mak
-              fi
-          fi
-      fi
-
-      # Special handling for assembler only tests
-      eval "target_as=\"\${cross_as_$arch}\""
-      eval "target_ld=\"\${cross_ld_$arch}\""
-      if has $target_as && has $target_ld; then
-          case $target in
-              tricore-softmmu)
-                  echo "AS=$target_as" >> $config_target_mak
-                  echo "LD=$target_ld" >> $config_target_mak
-                  got_cross_cc=yes
-                  ;;
-          esac
-      fi
-  fi
-
-  if test $got_cross_cc = yes; then
-      # Test for compiler features for optional tests. We only do this
-      # for cross compilers because ensuring the docker containers based
-      # compilers is a requirememt for adding a new test that needs a
-      # compiler feature.
-
-      case $target in
-          aarch64-*)
-              if do_compiler "$target_compiler" $target_compiler_cflags \
-                             -march=armv8.1-a+sve -o $TMPE $TMPC; then
-                  echo "CROSS_CC_HAS_SVE=y" >> $config_target_mak
-              fi
-              if do_compiler "$target_compiler" $target_compiler_cflags \
-                             -march=armv8.1-a+sve2 -o $TMPE $TMPC; then
-                  echo "CROSS_CC_HAS_SVE2=y" >> $config_target_mak
-              fi
-              if do_compiler "$target_compiler" $target_compiler_cflags \
-                             -march=armv8.3-a -o $TMPE $TMPC; then
-                  echo "CROSS_CC_HAS_ARMV8_3=y" >> $config_target_mak
-              fi
-              if do_compiler "$target_compiler" $target_compiler_cflags \
-                             -mbranch-protection=standard -o $TMPE $TMPC; then
-                  echo "CROSS_CC_HAS_ARMV8_BTI=y" >> $config_target_mak
-              fi
-              if do_compiler "$target_compiler" $target_compiler_cflags \
-                             -march=armv8.5-a+memtag -o $TMPE $TMPC; then
-                  echo "CROSS_CC_HAS_ARMV8_MTE=y" >> $config_target_mak
-              fi
-              ;;
-          ppc*)
-              if do_compiler "$target_compiler" $target_compiler_cflags \
-                             -mpower8-vector -o $TMPE $TMPC; then
-                  echo "CROSS_CC_HAS_POWER8_VECTOR=y" >> $config_target_mak
-              fi
-              if do_compiler "$target_compiler" $target_compiler_cflags \
-                             -mpower10 -o $TMPE $TMPC; then
-                  echo "CROSS_CC_HAS_POWER10=y" >> $config_target_mak
-              fi
-              ;;
-          i386-linux-user)
-              if do_compiler "$target_compiler" $target_compiler_cflags \
-                             -Werror -fno-pie -o $TMPE $TMPC; then
-                  echo "CROSS_CC_HAS_I386_NOPIE=y" >> $config_target_mak
-              fi
-              ;;
-      esac
-  elif test $got_cross_cc = no && test "$container" != no && \
-          test -n "$container_image"; then
-      for host in $container_hosts; do
-          if test "$host" = "$cpu"; then
-              echo "build-tcg-tests-$target: docker-image-$container_image" >> $makefile
-              echo "BUILD_STATIC=y" >> $config_target_mak
-              echo "CC=\$(DOCKER_SCRIPT) cc --cc $container_cross_cc -i qemu/$container_image -s $source_path --" >> $config_target_mak
-              if test -n "$container_cross_as"; then
-                  echo "AS=\$(DOCKER_SCRIPT) cc --cc $container_cross_as -i qemu/$container_image -s $source_path --" >> $config_target_mak
-              fi
-              if test -n "$container_cross_ld"; then
-                  echo "LD=\$(DOCKER_SCRIPT) cc --cc $container_cross_ld -i qemu/$container_image -s $source_path --" >> $config_target_mak
-              fi
-              case $target in
-                  aarch64-*)
-                      echo "CROSS_CC_HAS_SVE=y" >> $config_target_mak
-                      echo "CROSS_CC_HAS_SVE2=y" >> $config_target_mak
-                      echo "CROSS_CC_HAS_ARMV8_3=y" >> $config_target_mak
-                      echo "CROSS_CC_HAS_ARMV8_BTI=y" >> $config_target_mak
-                      echo "CROSS_CC_HAS_ARMV8_MTE=y" >> $config_target_mak
-                      ;;
-                  ppc*)
-                      echo "CROSS_CC_HAS_POWER8_VECTOR=y" >> $config_target_mak
-                      echo "CROSS_CC_HAS_POWER10=y" >> $config_target_mak
-                      ;;
-                  i386-linux-user)
-                      echo "CROSS_CC_HAS_I386_NOPIE=y" >> $config_target_mak
-                      ;;
-              esac
-              got_cross_cc=yes
-              break
-          fi
-      done
-  fi
-  if test $got_cross_cc = yes; then
-      mkdir -p tests/tcg/$target
-      echo "QEMU=$PWD/$qemu" >> $config_target_mak
-      echo "EXTRA_CFLAGS=$target_compiler_cflags" >> $config_target_mak
-      echo "run-tcg-tests-$target: $qemu\$(EXESUF)" >> $makefile
-      tcg_tests_targets="$tcg_tests_targets $target"
-  fi
-done
-echo "TCG_TESTS_TARGETS=$tcg_tests_targets" >> $makefile

From 46af66edcb9c123069a941f495ba8b13f486851a Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 27 May 2022 16:35:49 +0100
Subject: [PATCH 720/935] configure: add missing cross compiler fallbacks
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The arm compiler can be used for armeb, and the sparc64 compiler
can be used for sparc.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517092616.1272238-9-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220527153603.887929-20-alex.bennee@linaro.org>
---
 configure | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/configure b/configure
index f91ac632e7..fbf6d39f96 100755
--- a/configure
+++ b/configure
@@ -1827,6 +1827,7 @@ fi
 : ${cross_cc_cflags_aarch64_be="-mbig-endian"}
 : ${cross_cc_alpha="alpha-linux-gnu-gcc"}
 : ${cross_cc_arm="arm-linux-gnueabihf-gcc"}
+: ${cross_cc_armeb="$cross_cc_arm"}
 : ${cross_cc_cflags_armeb="-mbig-endian"}
 : ${cross_cc_hexagon="hexagon-unknown-linux-musl-clang"}
 : ${cross_cc_cflags_hexagon="-mv67 -O2 -static"}
@@ -1849,9 +1850,10 @@ fi
 : ${cross_cc_riscv64="riscv64-linux-gnu-gcc"}
 : ${cross_cc_s390x="s390x-linux-gnu-gcc"}
 : ${cross_cc_sh4="sh4-linux-gnu-gcc"}
-: ${cross_cc_cflags_sparc="-m32 -mcpu=supersparc"}
 : ${cross_cc_sparc64="sparc64-linux-gnu-gcc"}
 : ${cross_cc_cflags_sparc64="-m64 -mcpu=ultrasparc"}
+: ${cross_cc_sparc="$cross_cc_sparc64"}
+: ${cross_cc_cflags_sparc="-m32 -mcpu=supersparc"}
 : ${cross_cc_x86_64="x86_64-linux-gnu-gcc"}
 : ${cross_cc_cflags_x86_64="-m64"}
 

From 2ad60f6f8c12ca0acd8834fdd70e088361b8791f Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 27 May 2022 16:35:50 +0100
Subject: [PATCH 721/935] configure: handle host compiler in
 probe_target_compiler
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

In preparation for handling more binaries than just cc, handle
the case of "probe_target_compiler $cpu" directly in the function,
setting the target_* variables based on the ones that are used to
build QEMU.  The clang check also needs to be moved after this
fallback.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517092616.1272238-10-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220527153603.887929-21-alex.bennee@linaro.org>
---
 configure | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)

diff --git a/configure b/configure
index fbf6d39f96..217c8b3cac 100755
--- a/configure
+++ b/configure
@@ -954,10 +954,6 @@ case $git_submodules_action in
     ;;
 esac
 
-if eval test -z "\${cross_cc_$cpu}"; then
-    eval "cross_cc_${cpu}=\$cc"
-fi
-
 default_target_list=""
 mak_wilds=""
 
@@ -2008,13 +2004,6 @@ probe_target_compiler() {
   if eval test -n "\"\${cross_cc_$1}\""; then
     if eval has "\"\${cross_cc_$1}\""; then
       eval "target_cc=\"\${cross_cc_$1}\""
-      case $1 in
-        i386|x86_64)
-          if $target_cc --version | grep -qi "clang"; then
-            unset target_cc
-          fi
-          ;;
-      esac
     fi
   fi
   if eval test -n "\"\${cross_as_$1}\""; then
@@ -2027,6 +2016,20 @@ probe_target_compiler() {
       eval "target_ld=\"\${cross_ld_$1}\""
     fi
   fi
+  if test "$1" = $cpu; then
+    : ${target_cc:=$cc}
+    : ${target_as:=$as}
+    : ${target_ld:=$ld}
+  fi
+  if test -n "$target_cc"; then
+    case $1 in
+      i386|x86_64)
+        if $target_cc --version | grep -qi "clang"; then
+          unset target_cc
+        fi
+        ;;
+    esac
+  fi
 }
 
 write_target_makefile() {

From 5adb43be79dd125f8bf7bce0fd4b0c4f03e0aa71 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 27 May 2022 16:35:51 +0100
Subject: [PATCH 722/935] configure: introduce --cross-prefix-*=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Also in preparation for handling more binaries from the cross binutils,
support an option --cross-prefix-ARCH.  All cross_cc_* defaults are
replaced with cross_prefix_*; the cross_cc_* fallbacks are extended
to the cross-compilation prefix, but the compiler fallbacks remain
as well.  This way, for example, --cross-cc-arm=arm-linux-gnueabihf-clang
also applies to armeb binaries.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-11-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220527153603.887929-22-alex.bennee@linaro.org>
---
 configure | 137 ++++++++++++++++++++++++++++++------------------------
 1 file changed, 77 insertions(+), 60 deletions(-)

diff --git a/configure b/configure
index 217c8b3cac..f55ae82a5d 100755
--- a/configure
+++ b/configure
@@ -363,6 +363,11 @@ for opt do
   --cross-cc-*) cc_arch=${opt#--cross-cc-}; cc_arch=${cc_arch%%=*}
                 eval "cross_cc_${cc_arch}=\$optarg"
   ;;
+  --cross-prefix-*[!a-zA-Z0-9_-]*=*) error_exit "Passed bad --cross-prefix-FOO option"
+  ;;
+  --cross-prefix-*) cc_arch=${opt#--cross-prefix-}; cc_arch=${cc_arch%%=*}
+                    eval "cross_prefix_${cc_arch}=\$optarg"
+  ;;
   esac
 done
 # OS specific
@@ -728,6 +733,8 @@ for opt do
   ;;
   --cross-cc-*)
   ;;
+  --cross-prefix-*)
+  ;;
   --enable-debug-info) meson_option_add -Ddebug=true
   ;;
   --disable-debug-info) meson_option_add -Ddebug=false
@@ -1016,6 +1023,7 @@ Advanced options (experts only):
   --extra-ldflags=LDFLAGS  append extra linker flags LDFLAGS
   --cross-cc-ARCH=CC       use compiler when building ARCH guest test cases
   --cross-cc-cflags-ARCH=  use compiler flags when building ARCH guest tests
+  --cross-prefix-ARCH=PREFIX cross compiler prefix when building ARCH guest test cases
   --make=MAKE              use specified make [$make]
   --python=PYTHON          use specified python [$python]
   --meson=MESON            use specified meson [$meson]
@@ -1818,44 +1826,54 @@ if test $use_containers = "yes"; then
 fi
 
 # cross compilers defaults, can be overridden with --cross-cc-ARCH
-: ${cross_cc_aarch64="aarch64-linux-gnu-gcc"}
+: ${cross_prefix_aarch64="aarch64-linux-gnu-"}
+: ${cross_prefix_aarch64_be="$cross_prefix_aarch64"}
+: ${cross_prefix_alpha="alpha-linux-gnu-"}
+: ${cross_prefix_arm="arm-linux-gnueabihf-"}
+: ${cross_prefix_armeb="$cross_prefix_arm"}
+: ${cross_prefix_hexagon="hexagon-unknown-linux-musl-"}
+: ${cross_prefix_hppa="hppa-linux-gnu-"}
+: ${cross_prefix_i386="i686-linux-gnu-"}
+: ${cross_prefix_m68k="m68k-linux-gnu-"}
+: ${cross_prefix_microblaze="microblaze-linux-musl-"}
+: ${cross_prefix_mips64el="mips64el-linux-gnuabi64-"}
+: ${cross_prefix_mips64="mips64-linux-gnuabi64-"}
+: ${cross_prefix_mipsel="mipsel-linux-gnu-"}
+: ${cross_prefix_mips="mips-linux-gnu-"}
+: ${cross_prefix_nios2="nios2-linux-gnu-"}
+: ${cross_prefix_ppc="powerpc-linux-gnu-"}
+: ${cross_prefix_ppc64="powerpc64-linux-gnu-"}
+: ${cross_prefix_ppc64le="$cross_prefix_ppc64"}
+: ${cross_prefix_riscv64="riscv64-linux-gnu-"}
+: ${cross_prefix_s390x="s390x-linux-gnu-"}
+: ${cross_prefix_sh4="sh4-linux-gnu-"}
+: ${cross_prefix_sparc64="sparc64-linux-gnu-"}
+: ${cross_prefix_sparc="$cross_prefix_sparc64"}
+: ${cross_prefix_x86_64="x86_64-linux-gnu-"}
+
 : ${cross_cc_aarch64_be="$cross_cc_aarch64"}
 : ${cross_cc_cflags_aarch64_be="-mbig-endian"}
-: ${cross_cc_alpha="alpha-linux-gnu-gcc"}
-: ${cross_cc_arm="arm-linux-gnueabihf-gcc"}
 : ${cross_cc_armeb="$cross_cc_arm"}
 : ${cross_cc_cflags_armeb="-mbig-endian"}
 : ${cross_cc_hexagon="hexagon-unknown-linux-musl-clang"}
 : ${cross_cc_cflags_hexagon="-mv67 -O2 -static"}
-: ${cross_cc_hppa="hppa-linux-gnu-gcc"}
-: ${cross_cc_i386="i686-linux-gnu-gcc"}
 : ${cross_cc_cflags_i386="-m32"}
-: ${cross_cc_m68k="m68k-linux-gnu-gcc"}
-: ${cross_cc_microblaze="microblaze-linux-musl-gcc"}
-: ${cross_cc_mips64el="mips64el-linux-gnuabi64-gcc"}
-: ${cross_cc_mips64="mips64-linux-gnuabi64-gcc"}
-: ${cross_cc_mipsel="mipsel-linux-gnu-gcc"}
-: ${cross_cc_mips="mips-linux-gnu-gcc"}
-: ${cross_cc_nios2="nios2-linux-gnu-gcc"}
-: ${cross_cc_ppc="powerpc-linux-gnu-gcc"}
 : ${cross_cc_cflags_ppc="-m32"}
-: ${cross_cc_ppc64="powerpc64-linux-gnu-gcc"}
 : ${cross_cc_cflags_ppc64="-m64 -mbig-endian"}
 : ${cross_cc_ppc64le="$cross_cc_ppc64"}
 : ${cross_cc_cflags_ppc64le="-m64 -mlittle-endian"}
-: ${cross_cc_riscv64="riscv64-linux-gnu-gcc"}
-: ${cross_cc_s390x="s390x-linux-gnu-gcc"}
-: ${cross_cc_sh4="sh4-linux-gnu-gcc"}
-: ${cross_cc_sparc64="sparc64-linux-gnu-gcc"}
 : ${cross_cc_cflags_sparc64="-m64 -mcpu=ultrasparc"}
 : ${cross_cc_sparc="$cross_cc_sparc64"}
 : ${cross_cc_cflags_sparc="-m32 -mcpu=supersparc"}
-: ${cross_cc_x86_64="x86_64-linux-gnu-gcc"}
 : ${cross_cc_cflags_x86_64="-m64"}
 
-# tricore is special as it doesn't have a compiler
-: ${cross_as_tricore="tricore-as"}
-: ${cross_ld_tricore="tricore-ld"}
+compute_target_variable() {
+  if eval test -n "\"\${cross_prefix_$1}\""; then
+    if eval has "\"\${cross_prefix_$1}\$3\""; then
+      eval "$2=\"\${cross_prefix_$1}\$3\""
+    fi
+  fi
+}
 
 probe_target_compiler() {
   # reset all output variables
@@ -1901,93 +1919,99 @@ probe_target_compiler() {
       aarch64)
         # We don't have any bigendian build tools so we only use this for AArch64
         container_image=debian-arm64-cross
-        container_cross_cc=aarch64-linux-gnu-gcc-10
+        container_cross_prefix=aarch64-linux-gnu-
+        container_cross_cc=${container_cross_prefix}gcc-10
         ;;
       alpha)
         container_image=debian-alpha-cross
-        container_cross_cc=alpha-linux-gnu-gcc
+        container_cross_prefix=alpha-linux-gnu-
         ;;
       arm)
         # We don't have any bigendian build tools so we only use this for ARM
         container_image=debian-armhf-cross
-        container_cross_cc=arm-linux-gnueabihf-gcc
+        container_cross_prefix=arm-linux-gnueabihf-
         ;;
       cris)
         container_image=fedora-cris-cross
-        container_cross_cc=cris-linux-gnu-gcc
+        container_cross_prefix=cris-linux-gnu-
         ;;
       hexagon)
         container_image=debian-hexagon-cross
-        container_cross_cc=hexagon-unknown-linux-musl-clang
+        container_cross_prefix=hexagon-unknown-linux-musl-
+        container_cross_cc=${container_cross_prefix}clang
         ;;
       hppa)
         container_image=debian-hppa-cross
-        container_cross_cc=hppa-linux-gnu-gcc
+        container_cross_prefix=hppa-linux-gnu-
         ;;
       i386)
         container_image=fedora-i386-cross
-        container_cross_cc=gcc
+        container_cross_prefix=
         ;;
       m68k)
         container_image=debian-m68k-cross
-        container_cross_cc=m68k-linux-gnu-gcc
+        container_cross_prefix=m68k-linux-gnu-
         ;;
       microblaze)
         container_image=debian-microblaze-cross
-        container_cross_cc=microblaze-linux-musl-gcc
+        container_cross_prefix=microblaze-linux-musl-
         ;;
       mips64el)
         container_image=debian-mips64el-cross
-        container_cross_cc=mips64el-linux-gnuabi64-gcc
+        container_cross_prefix=mips64el-linux-gnuabi64-
         ;;
       mips64)
         container_image=debian-mips64-cross
-        container_cross_cc=mips64-linux-gnuabi64-gcc
+        container_cross_prefix=mips64-linux-gnuabi64-
         ;;
       mipsel)
         container_image=debian-mipsel-cross
-        container_cross_cc=mipsel-linux-gnu-gcc
+        container_cross_prefix=mipsel-linux-gnu-
         ;;
       mips)
         container_image=debian-mips-cross
-        container_cross_cc=mips-linux-gnu-gcc
+        container_cross_prefix=mips-linux-gnu-
         ;;
       nios2)
         container_image=debian-nios2-cross
-        container_cross_cc=nios2-linux-gnu-gcc
+        container_cross_prefix=nios2-linux-gnu-
         ;;
       ppc)
         container_image=debian-powerpc-test-cross
-        container_cross_cc=powerpc-linux-gnu-gcc-10
+        container_cross_prefix=powerpc-linux-gnu-
+        container_cross_cc=${container_cross_prefix}gcc-10
         ;;
       ppc64|ppc64le)
         container_image=debian-powerpc-test-cross
-        container_cross_cc=powerpc${1#ppc}-linux-gnu-gcc-10
+        container_cross_prefix=powerpc${1#ppc}-linux-gnu-
+        container_cross_cc=${container_cross_prefix}gcc-10
         ;;
       riscv64)
         container_image=debian-riscv64-test-cross
-        container_cross_cc=riscv64-linux-gnu-gcc
+        container_cross_prefix=riscv64-linux-gnu-
         ;;
       s390x)
         container_image=debian-s390x-cross
-        container_cross_cc=s390x-linux-gnu-gcc
+        container_cross_prefix=s390x-linux-gnu-
         ;;
       sh4)
         container_image=debian-sh4-cross
-        container_cross_cc=sh4-linux-gnu-gcc
+        container_cross_prefix=sh4-linux-gnu-
         ;;
       sparc64)
         container_image=debian-sparc64-cross
-        container_cross_cc=sparc64-linux-gnu-gcc
+        container_cross_prefix=sparc64-linux-gnu-
         ;;
       tricore)
         container_image=debian-tricore-cross
+        container_cross_prefix=tricore-
         container_cross_as=tricore-as
         container_cross_ld=tricore-ld
+        break
         ;;
       x86_64)
         container_image=debian-amd64-cross
-        container_cross_cc=x86_64-linux-gnu-gcc
+        container_cross_prefix=x86_64-linux-gnu-
         ;;
       xtensa*)
         # FIXME: xtensa-linux-user?
@@ -1995,9 +2019,12 @@ probe_target_compiler() {
         container_image=debian-xtensa-cross
 
         # default to the dc232b cpu
-        container_cross_cc=/opt/2020.07/xtensa-dc232b-elf/bin/xtensa-dc232b-elf-gcc
+        container_cross_prefix=/opt/2020.07/xtensa-dc232b-elf/bin/xtensa-dc232b-elf-
         ;;
     esac
+    : ${container_cross_cc:=${container_cross_prefix}gcc}
+    : ${container_cross_as:=${container_cross_prefix}as}
+    : ${container_cross_ld:=${container_cross_prefix}ld}
   done
 
   eval "target_cflags=\${cross_cc_cflags_$1}"
@@ -2005,17 +2032,11 @@ probe_target_compiler() {
     if eval has "\"\${cross_cc_$1}\""; then
       eval "target_cc=\"\${cross_cc_$1}\""
     fi
+  else
+    compute_target_variable $1 target_cc gcc
   fi
-  if eval test -n "\"\${cross_as_$1}\""; then
-    if eval has "\"\${cross_as_$1}\""; then
-      eval "target_as=\"\${cross_as_$1}\""
-    fi
-  fi
-  if eval test -n "\"\${cross_ld_$1}\""; then
-    if eval has "\"\${cross_ld_$1}\""; then
-      eval "target_ld=\"\${cross_ld_$1}\""
-    fi
-  fi
+  compute_target_variable $1 target_as as
+  compute_target_variable $1 target_ld ld
   if test "$1" = $cpu; then
     : ${target_cc:=$cc}
     : ${target_as:=$as}
@@ -2048,12 +2069,8 @@ write_container_target_makefile() {
   if test -n "$container_cross_cc"; then
     echo "CC=\$(DOCKER_SCRIPT) cc --cc $container_cross_cc -i qemu/$container_image -s $source_path --"
   fi
-  if test -n "$container_cross_as"; then
-    echo "AS=\$(DOCKER_SCRIPT) cc --cc $container_cross_as -i qemu/$container_image -s $source_path --"
-  fi
-  if test -n "$container_cross_ld"; then
-    echo "LD=\$(DOCKER_SCRIPT) cc --cc $container_cross_ld -i qemu/$container_image -s $source_path --"
-  fi
+  echo "AS=\$(DOCKER_SCRIPT) cc --cc $container_cross_as -i qemu/$container_image -s $source_path --"
+  echo "LD=\$(DOCKER_SCRIPT) cc --cc $container_cross_ld -i qemu/$container_image -s $source_path --"
 }
 
 

From 87eb014c5e80039bf6bee4f05557f48827239cb4 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 27 May 2022 16:35:52 +0100
Subject: [PATCH 723/935] configure: include more binutils in tests/tcg
 makefile
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Firmware builds require paths to all the binutils; it is not enough to
use only cc, or even as/ld as in the case of tests/tcg/tricore.
Adjust the cross-compiler configurator to detect also ar, nm, objcopy,
ranlib and strip.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-12-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220527153603.887929-23-alex.bennee@linaro.org>
---
 configure | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)

diff --git a/configure b/configure
index f55ae82a5d..ced8283b98 100755
--- a/configure
+++ b/configure
@@ -1880,11 +1880,21 @@ probe_target_compiler() {
   container_image=
   container_hosts=
   container_cross_cc=
+  container_cross_ar=
   container_cross_as=
   container_cross_ld=
+  container_cross_nm=
+  container_cross_objcopy=
+  container_cross_ranlib=
+  container_cross_strip=
   target_cc=
+  target_ar=
   target_as=
   target_ld=
+  target_nm=
+  target_objcopy=
+  target_ranlib=
+  target_strip=
 
   case $1 in
     aarch64) container_hosts="x86_64 aarch64" ;;
@@ -2023,8 +2033,13 @@ probe_target_compiler() {
         ;;
     esac
     : ${container_cross_cc:=${container_cross_prefix}gcc}
+    : ${container_cross_ar:=${container_cross_prefix}ar}
     : ${container_cross_as:=${container_cross_prefix}as}
     : ${container_cross_ld:=${container_cross_prefix}ld}
+    : ${container_cross_nm:=${container_cross_prefix}nm}
+    : ${container_cross_objcopy:=${container_cross_prefix}objcopy}
+    : ${container_cross_ranlib:=${container_cross_prefix}ranlib}
+    : ${container_cross_strip:=${container_cross_prefix}strip}
   done
 
   eval "target_cflags=\${cross_cc_cflags_$1}"
@@ -2035,12 +2050,26 @@ probe_target_compiler() {
   else
     compute_target_variable $1 target_cc gcc
   fi
+  target_ccas=$target_cc
+  compute_target_variable $1 target_ar ar
   compute_target_variable $1 target_as as
   compute_target_variable $1 target_ld ld
+  compute_target_variable $1 target_nm nm
+  compute_target_variable $1 target_objcopy objcopy
+  compute_target_variable $1 target_ranlib ranlib
+  compute_target_variable $1 target_strip strip
   if test "$1" = $cpu; then
     : ${target_cc:=$cc}
+    : ${target_ccas:=$ccas}
     : ${target_as:=$as}
     : ${target_ld:=$ld}
+    : ${target_ar:=$ar}
+    : ${target_as:=$as}
+    : ${target_ld:=$ld}
+    : ${target_nm:=$nm}
+    : ${target_objcopy:=$objcopy}
+    : ${target_ranlib:=$ranlib}
+    : ${target_strip:=$strip}
   fi
   if test -n "$target_cc"; then
     case $1 in
@@ -2056,6 +2085,10 @@ probe_target_compiler() {
 write_target_makefile() {
   if test -n "$target_cc"; then
     echo "CC=$target_cc"
+    echo "CCAS=$target_ccas"
+  fi
+  if test -n "$target_ar"; then
+    echo "AR=$target_ar"
   fi
   if test -n "$target_as"; then
     echo "AS=$target_as"
@@ -2063,14 +2096,32 @@ write_target_makefile() {
   if test -n "$target_ld"; then
     echo "LD=$target_ld"
   fi
+  if test -n "$target_nm"; then
+    echo "NM=$target_nm"
+  fi
+  if test -n "$target_objcopy"; then
+    echo "OBJCOPY=$target_objcopy"
+  fi
+  if test -n "$target_ranlib"; then
+    echo "RANLIB=$target_ranlib"
+  fi
+  if test -n "$target_strip"; then
+    echo "STRIP=$target_strip"
+  fi
 }
 
 write_container_target_makefile() {
   if test -n "$container_cross_cc"; then
     echo "CC=\$(DOCKER_SCRIPT) cc --cc $container_cross_cc -i qemu/$container_image -s $source_path --"
+    echo "CCAS=\$(DOCKER_SCRIPT) cc --cc $container_cross_cc -i qemu/$container_image -s $source_path --"
   fi
+  echo "AR=\$(DOCKER_SCRIPT) cc --cc $container_cross_ar -i qemu/$container_image -s $source_path --"
   echo "AS=\$(DOCKER_SCRIPT) cc --cc $container_cross_as -i qemu/$container_image -s $source_path --"
   echo "LD=\$(DOCKER_SCRIPT) cc --cc $container_cross_ld -i qemu/$container_image -s $source_path --"
+  echo "NM=\$(DOCKER_SCRIPT) cc --cc $container_cross_nm -i qemu/$container_image -s $source_path --"
+  echo "OBJCOPY=\$(DOCKER_SCRIPT) cc --cc $container_cross_objcopy -i qemu/$container_image -s $source_path --"
+  echo "RANLIB=\$(DOCKER_SCRIPT) cc --cc $container_cross_ranlib -i qemu/$container_image -s $source_path --"
+  echo "STRIP=\$(DOCKER_SCRIPT) cc --cc $container_cross_strip -i qemu/$container_image -s $source_path --"
 }
 
 

From ad38884512a72206463a95a30451253946d4218b Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 27 May 2022 16:35:53 +0100
Subject: [PATCH 724/935] configure: move symlink configuration earlier
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Ensure that the pc-bios/optionrom and pc-bios/s390-ccw directory
exist at the time when we'll write out the compiler configuration
for them.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-13-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220527153603.887929-24-alex.bennee@linaro.org>
---
 configure | 49 ++++++++++++++++++++++++-------------------------
 1 file changed, 24 insertions(+), 25 deletions(-)

diff --git a/configure b/configure
index ced8283b98..aa17b0fa55 100755
--- a/configure
+++ b/configure
@@ -2190,6 +2190,30 @@ if test "$QEMU_GA_VERSION" = ""; then
     QEMU_GA_VERSION=$(cat $source_path/VERSION)
 fi
 
+# Set up build tree symlinks that point back into the source tree
+# (these can be both files and directories).
+# Caution: avoid adding files or directories here using wildcards. This
+# will result in problems later if a new file matching the wildcard is
+# added to the source tree -- nothing will cause configure to be rerun
+# so the build tree will be missing the link back to the new file, and
+# tests might fail. Prefer to keep the relevant files in their own
+# directory and symlink the directory instead.
+LINKS="Makefile"
+LINKS="$LINKS tests/tcg/Makefile.target"
+LINKS="$LINKS pc-bios/optionrom/Makefile"
+LINKS="$LINKS pc-bios/s390-ccw/Makefile"
+LINKS="$LINKS .gdbinit scripts" # scripts needed by relative path in .gdbinit
+LINKS="$LINKS tests/avocado tests/data"
+LINKS="$LINKS tests/qemu-iotests/check"
+LINKS="$LINKS python"
+LINKS="$LINKS contrib/plugins/Makefile "
+for f in $LINKS ; do
+    if [ -e "$source_path/$f" ]; then
+        mkdir -p `dirname ./$f`
+        symlink "$source_path/$f" "$f"
+    fi
+done
+
 # Mac OS X ships with a broken assembler
 roms=
 if { test "$cpu" = "i386" || test "$cpu" = "x86_64"; } && \
@@ -2408,31 +2432,6 @@ if test "$safe_stack" = "yes"; then
   echo "CONFIG_SAFESTACK=y" >> $config_host_mak
 fi
 
-# If we're using a separate build tree, set it up now.
-# LINKS are things to symlink back into the source tree
-# (these can be both files and directories).
-# Caution: do not add files or directories here using wildcards. This
-# will result in problems later if a new file matching the wildcard is
-# added to the source tree -- nothing will cause configure to be rerun
-# so the build tree will be missing the link back to the new file, and
-# tests might fail. Prefer to keep the relevant files in their own
-# directory and symlink the directory instead.
-LINKS="Makefile"
-LINKS="$LINKS tests/tcg/Makefile.target"
-LINKS="$LINKS pc-bios/optionrom/Makefile"
-LINKS="$LINKS pc-bios/s390-ccw/Makefile"
-LINKS="$LINKS .gdbinit scripts" # scripts needed by relative path in .gdbinit
-LINKS="$LINKS tests/avocado tests/data"
-LINKS="$LINKS tests/qemu-iotests/check"
-LINKS="$LINKS python"
-LINKS="$LINKS contrib/plugins/Makefile "
-for f in $LINKS ; do
-    if [ -e "$source_path/$f" ]; then
-        mkdir -p `dirname ./$f`
-        symlink "$source_path/$f" "$f"
-    fi
-done
-
 # tests/tcg configuration
 (makefile=tests/tcg/Makefile.prereqs
 echo "# Automatically generated by configure - do not modify" > $makefile

From 9ffed42614585026834fd63cdf586b78b1f72c66 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 27 May 2022 16:35:54 +0100
Subject: [PATCH 725/935] configure: enable cross-compilation of s390-ccw
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

While container-based cross compilers are not supported, this already makes
it possible to build s390-ccw on any machine that has s390x GCC and binutils
installed.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-14-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220527153603.887929-25-alex.bennee@linaro.org>
---
 configure                    | 18 +++++++++++++-----
 pc-bios/s390-ccw/Makefile    |  9 +++++----
 pc-bios/s390-ccw/netboot.mak |  2 +-
 3 files changed, 19 insertions(+), 10 deletions(-)

diff --git a/configure b/configure
index aa17b0fa55..f509bce304 100755
--- a/configure
+++ b/configure
@@ -2231,24 +2231,32 @@ if { test "$cpu" = "i386" || test "$cpu" = "x86_64"; } && \
     done
 fi
 
-# Only build s390-ccw bios if we're on s390x and the compiler has -march=z900
-# or -march=z10 (which is the lowest architecture level that Clang supports)
-if test "$cpu" = "s390x" ; then
+# Only build s390-ccw bios if the compiler has -march=z900 or -march=z10
+# (which is the lowest architecture level that Clang supports)
+probe_target_compiler s390x
+if test -n "$target_cc" && test "$softmmu" = yes; then
   write_c_skeleton
-  compile_prog "-march=z900" ""
+  do_compiler "$target_cc" $target_cc_cflags -march=z900 -o $TMPO -c $TMPC
   has_z900=$?
-  if [ $has_z900 = 0 ] || compile_object "-march=z10 -msoft-float -Werror"; then
+  if [ $has_z900 = 0 ] || do_compiler "$target_cc" $target_cc_cflags -march=z10 -msoft-float -Werror -o $TMPO -c $TMPC; then
     if [ $has_z900 != 0 ]; then
       echo "WARNING: Your compiler does not support the z900!"
       echo "         The s390-ccw bios will only work with guest CPUs >= z10."
     fi
     roms="$roms s390-ccw"
+    config_mak=pc-bios/s390-ccw/config-host.mak
+    echo "# Automatically generated by configure - do not modify" > $config_mak
+    echo "SRC_PATH=$source_path/pc-bios/s390-ccw" >> $config_mak
+    write_target_makefile >> $config_mak
     # SLOF is required for building the s390-ccw firmware on s390x,
     # since it is using the libnet code from SLOF for network booting.
     git_submodules="${git_submodules} roms/SLOF"
   fi
 fi
 
+#######################################
+# generate config-host.mak
+
 # Check that the C++ compiler exists and works with the C compiler.
 # All the QEMU_CXXFLAGS are based on QEMU_CFLAGS. Keep this at the end to don't miss any other that could be added.
 if has $cxx; then
diff --git a/pc-bios/s390-ccw/Makefile b/pc-bios/s390-ccw/Makefile
index 0eb68efc7b..6eb713bf37 100644
--- a/pc-bios/s390-ccw/Makefile
+++ b/pc-bios/s390-ccw/Makefile
@@ -2,8 +2,9 @@ all: build-all
 # Dummy command so that make thinks it has done something
 	@true
 
-include ../../config-host.mak
+include config-host.mak
 CFLAGS = -O2 -g
+MAKEFLAGS += -rR
 
 quiet-command = $(if $(V),$1,$(if $(2),@printf "  %-7s %s\n" $2 $3 && $1, @$1))
 cc-option = $(if $(shell $(CC) $1 $2 -S -o /dev/null -xc /dev/null \
@@ -11,7 +12,7 @@ cc-option = $(if $(shell $(CC) $1 $2 -S -o /dev/null -xc /dev/null \
 
 VPATH_SUFFIXES = %.c %.h %.S %.m %.mak %.sh %.rc Kconfig% %.json.in
 set-vpath = $(if $1,$(foreach PATTERN,$(VPATH_SUFFIXES),$(eval vpath $(PATTERN) $1)))
-$(call set-vpath, $(SRC_PATH)/pc-bios/s390-ccw)
+$(call set-vpath, $(SRC_PATH))
 
 # Flags for dependency generation
 QEMU_DGFLAGS = -MMD -MP -MT $@ -MF $(@D)/$(*F).d
@@ -49,8 +50,8 @@ s390-ccw.img: s390-ccw.elf
 
 $(OBJECTS): Makefile
 
-ifneq ($(wildcard $(SRC_PATH)/roms/SLOF/lib/libnet),)
-include $(SRC_PATH)/pc-bios/s390-ccw/netboot.mak
+ifneq ($(wildcard $(SRC_PATH)/../../roms/SLOF/lib/libnet),)
+include $(SRC_PATH)/netboot.mak
 else
 s390-netboot.img:
 	@echo "s390-netboot.img not built since roms/SLOF/ is not available."
diff --git a/pc-bios/s390-ccw/netboot.mak b/pc-bios/s390-ccw/netboot.mak
index 68b4d7edcb..1a06befa4b 100644
--- a/pc-bios/s390-ccw/netboot.mak
+++ b/pc-bios/s390-ccw/netboot.mak
@@ -1,5 +1,5 @@
 
-SLOF_DIR := $(SRC_PATH)/roms/SLOF
+SLOF_DIR := $(SRC_PATH)/../../roms/SLOF
 
 NETOBJS := start.o sclp.o cio.o virtio.o virtio-net.o jump2ipl.o netmain.o
 

From 33ab4787422b04a3f4c94bbded6b05021d08b27d Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 27 May 2022 16:35:55 +0100
Subject: [PATCH 726/935] configure: enable cross-compilation of optionrom
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

While container-based cross compilers are not supported, this already makes
it possible to build x86 optionroms on any machine that has an installation
of GCC and binutils for 32- or 64-bit x86.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-15-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220527153603.887929-26-alex.bennee@linaro.org>
---
 configure                  | 30 ++++++++++++++++++++++--------
 pc-bios/optionrom/Makefile |  2 --
 2 files changed, 22 insertions(+), 10 deletions(-)

diff --git a/configure b/configure
index f509bce304..1ae5c950f0 100755
--- a/configure
+++ b/configure
@@ -2082,6 +2082,13 @@ probe_target_compiler() {
   fi
 }
 
+probe_target_compilers() {
+  for i; do
+    probe_target_compiler $i
+    test -n "$target_cc" && return 0
+  done
+}
+
 write_target_makefile() {
   if test -n "$target_cc"; then
     echo "CC=$target_cc"
@@ -2190,6 +2197,10 @@ if test "$QEMU_GA_VERSION" = ""; then
     QEMU_GA_VERSION=$(cat $source_path/VERSION)
 fi
 
+
+#######################################
+# cross-compiled firmware targets
+
 # Set up build tree symlinks that point back into the source tree
 # (these can be both files and directories).
 # Caution: avoid adding files or directories here using wildcards. This
@@ -2216,19 +2227,27 @@ done
 
 # Mac OS X ships with a broken assembler
 roms=
-if { test "$cpu" = "i386" || test "$cpu" = "x86_64"; } && \
+probe_target_compilers i386 x86_64
+if test -n "$target_cc" &&
         test "$targetos" != "darwin" && test "$targetos" != "sunos" && \
         test "$targetos" != "haiku" && test "$softmmu" = yes ; then
     # Different host OS linkers have different ideas about the name of the ELF
     # emulation. Linux and OpenBSD/amd64 use 'elf_i386'; FreeBSD uses the _fbsd
     # variant; OpenBSD/i386 uses the _obsd variant; and Windows uses i386pe.
     for emu in elf_i386 elf_i386_fbsd elf_i386_obsd i386pe; do
-        if "$ld" -verbose 2>&1 | grep -q "^[[:space:]]*$emu[[:space:]]*$"; then
+        if "$target_ld" -verbose 2>&1 | grep -q "^[[:space:]]*$emu[[:space:]]*$"; then
             ld_i386_emulation="$emu"
-            roms="optionrom"
             break
         fi
     done
+    if test -n "$ld_i386_emulation"; then
+        roms="optionrom"
+        config_mak=pc-bios/optionrom/config.mak
+        echo "# Automatically generated by configure - do not modify" > $config_mak
+        echo "TOPSRC_DIR=$source_path" >> $config_mak
+        echo "LD_I386_EMULATION=$ld_i386_emulation" >> $config_mak
+        write_target_makefile >> $config_mak
+    fi
 fi
 
 # Only build s390-ccw bios if the compiler has -march=z900 or -march=z10
@@ -2381,7 +2400,6 @@ echo "GLIB_LIBS=$glib_libs" >> $config_host_mak
 echo "GLIB_BINDIR=$glib_bindir" >> $config_host_mak
 echo "GLIB_VERSION=$(pkg-config --modversion glib-2.0)" >> $config_host_mak
 echo "QEMU_LDFLAGS=$QEMU_LDFLAGS" >> $config_host_mak
-echo "LD_I386_EMULATION=$ld_i386_emulation" >> $config_host_mak
 echo "STRIP=$strip" >> $config_host_mak
 echo "EXESUF=$EXESUF" >> $config_host_mak
 
@@ -2571,10 +2589,6 @@ for target in $target_list; do
 done
 echo "TCG_TESTS_TARGETS=$tcg_tests_targets" >> $makefile)
 
-config_mak=pc-bios/optionrom/config.mak
-echo "# Automatically generated by configure - do not modify" > $config_mak
-echo "TOPSRC_DIR=$source_path" >> $config_mak
-
 if test "$skip_meson" = no; then
   cross="config-meson.cross.new"
   meson_quote() {
diff --git a/pc-bios/optionrom/Makefile b/pc-bios/optionrom/Makefile
index 17ccc76241..f639915b4f 100644
--- a/pc-bios/optionrom/Makefile
+++ b/pc-bios/optionrom/Makefile
@@ -6,7 +6,6 @@ all: multiboot.bin multiboot_dma.bin linuxboot.bin linuxboot_dma.bin kvmvapic.bi
 # Dummy command so that make thinks it has done something
 	@true
 
-include ../../config-host.mak
 CFLAGS = -O2 -g
 
 quiet-command = $(if $(V),$1,$(if $(2),@printf "  %-7s %s\n" $2 $3 && $1, @$1))
@@ -44,7 +43,6 @@ Wa = -Wa,
 override ASFLAGS += -32
 override CFLAGS += $(call cc-option, $(Wa)-32)
 
-LD_I386_EMULATION ?= elf_i386
 override LDFLAGS = -m $(LD_I386_EMULATION) -T $(SRC_DIR)/flat.lds
 
 pvh.img: pvh.o pvh_main.o

From d695918f7bd92604e561b6963e4ca9aef83f489b Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 27 May 2022 16:35:56 +0100
Subject: [PATCH 727/935] configure: enable cross compilation of vof
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

While container-based cross compilers are not supported, this already
makes it possible to build vof on any machine that has an installation
of GCC and binutils for 32- or 64-bit PowerPC.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-16-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220527153603.887929-27-alex.bennee@linaro.org>
---
 configure            | 10 ++++++++++
 pc-bios/vof/Makefile | 17 +++++++++--------
 2 files changed, 19 insertions(+), 8 deletions(-)

diff --git a/configure b/configure
index 1ae5c950f0..11e7f30553 100755
--- a/configure
+++ b/configure
@@ -2213,6 +2213,7 @@ LINKS="Makefile"
 LINKS="$LINKS tests/tcg/Makefile.target"
 LINKS="$LINKS pc-bios/optionrom/Makefile"
 LINKS="$LINKS pc-bios/s390-ccw/Makefile"
+LINKS="$LINKS pc-bios/vof/Makefile"
 LINKS="$LINKS .gdbinit scripts" # scripts needed by relative path in .gdbinit
 LINKS="$LINKS tests/avocado tests/data"
 LINKS="$LINKS tests/qemu-iotests/check"
@@ -2250,6 +2251,15 @@ if test -n "$target_cc" &&
     fi
 fi
 
+probe_target_compilers ppc ppc64
+if test -n "$target_cc" && test "$softmmu" = yes; then
+    roms="$roms vof"
+    config_mak=pc-bios/vof/config.mak
+    echo "# Automatically generated by configure - do not modify" > $config_mak
+    echo "SRC_DIR=$source_path/pc-bios/vof" >> $config_mak
+    write_target_makefile >> $config_mak
+fi
+
 # Only build s390-ccw bios if the compiler has -march=z900 or -march=z10
 # (which is the lowest architecture level that Clang supports)
 probe_target_compiler s390x
diff --git a/pc-bios/vof/Makefile b/pc-bios/vof/Makefile
index aa1678c4d8..391ac0d600 100644
--- a/pc-bios/vof/Makefile
+++ b/pc-bios/vof/Makefile
@@ -1,11 +1,10 @@
-all: build-all
+include config.mak
+VPATH=$(SRC_DIR)
+all: vof.bin
 
-build-all: vof.bin
-
-CROSS ?=
-CC = $(CROSS)gcc
-LD = $(CROSS)ld
-OBJCOPY = $(CROSS)objcopy
+CC ?= $(CROSS)gcc
+LD ?= $(CROSS)ld
+OBJCOPY ?= $(CROSS)objcopy
 
 %.o: %.S
 	$(CC) -m32 -mbig-endian -mcpu=power4 -c -o $@ $<
@@ -14,10 +13,12 @@ OBJCOPY = $(CROSS)objcopy
 	$(CC) -m32 -mbig-endian -mcpu=power4 -c -fno-stack-protector -o $@ $<
 
 vof.elf: entry.o main.o ci.o bootmem.o libc.o
-	$(LD) -nostdlib -e_start -Tvof.lds -EB -o $@ $^
+	$(LD) -nostdlib -e_start -T$(SRC_DIR)/vof.lds -EB -o $@ $^
 
 %.bin: %.elf
 	$(OBJCOPY) -O binary -j .text -j .data -j .toc -j .got2 $^ $@
 
 clean:
 	rm -f *.o vof.bin vof.elf *~
+
+.PHONY: all clean

From f21db42631ae1f3feaa3bf94edd9c06c24b4aeb6 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Fri, 27 May 2022 16:35:57 +0100
Subject: [PATCH 728/935] configure: remove unused variables from
 config-host.mak
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The only compiler variable that is still needed is $(CC), for
contrib/plugins/Makefile.  All firmware builds have their own
config-host.mak file.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-17-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220527153603.887929-28-alex.bennee@linaro.org>
---
 configure | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/configure b/configure
index 11e7f30553..b9ccff9067 100755
--- a/configure
+++ b/configure
@@ -2397,11 +2397,6 @@ echo "GENISOIMAGE=$genisoimage" >> $config_host_mak
 echo "MESON=$meson" >> $config_host_mak
 echo "NINJA=$ninja" >> $config_host_mak
 echo "CC=$cc" >> $config_host_mak
-echo "AR=$ar" >> $config_host_mak
-echo "AS=$as" >> $config_host_mak
-echo "CCAS=$ccas" >> $config_host_mak
-echo "OBJCOPY=$objcopy" >> $config_host_mak
-echo "LD=$ld" >> $config_host_mak
 echo "QEMU_CFLAGS=$QEMU_CFLAGS" >> $config_host_mak
 echo "QEMU_CXXFLAGS=$QEMU_CXXFLAGS" >> $config_host_mak
 echo "QEMU_OBJCFLAGS=$QEMU_OBJCFLAGS" >> $config_host_mak
@@ -2410,7 +2405,6 @@ echo "GLIB_LIBS=$glib_libs" >> $config_host_mak
 echo "GLIB_BINDIR=$glib_bindir" >> $config_host_mak
 echo "GLIB_VERSION=$(pkg-config --modversion glib-2.0)" >> $config_host_mak
 echo "QEMU_LDFLAGS=$QEMU_LDFLAGS" >> $config_host_mak
-echo "STRIP=$strip" >> $config_host_mak
 echo "EXESUF=$EXESUF" >> $config_host_mak
 
 # use included Linux headers

From 6a0e7ea7b87cfa684bbcebaebafaef14c29a8872 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Fri, 27 May 2022 16:35:58 +0100
Subject: [PATCH 729/935] gitlab: introduce a common base job template
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Currently job rules are spread across the various templates
and jobs, making it hard to understand exactly what runs in
what scenario. This leads to inconsistency in the rules and
increased maint burden.

The intent is that we introduce a common '.base_job_template'
which will have a general purpose 'rules:' block. No other
template or job should define 'rules:', but instead they must
rely on the inherited rules. To allow behaviour to be tweaked,
rules will be influenced by a number of variables with the
naming scheme 'QEMU_JOB_nnnn'.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220526110705.59952-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220527153603.887929-29-alex.bennee@linaro.org>
---
 .gitlab-ci.d/base.yml         | 28 +++++++++++++++++++++++++++
 .gitlab-ci.d/qemu-project.yml |  1 +
 docs/devel/ci-jobs.rst.inc    | 36 ++++++++++++++++++++++++++++++++++-
 3 files changed, 64 insertions(+), 1 deletion(-)
 create mode 100644 .gitlab-ci.d/base.yml

diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml
new file mode 100644
index 0000000000..10eb6ab8bc
--- /dev/null
+++ b/.gitlab-ci.d/base.yml
@@ -0,0 +1,28 @@
+
+# The order of rules defined here is critically important.
+# They are evaluated in order and first match wins.
+#
+# Thus we group them into a number of stages, ordered from
+# most restrictive to least restrictive
+#
+.base_job_template:
+  rules:
+    #############################################################
+    # Stage 1: exclude scenarios where we definitely don't
+    # want jobs to run
+    #############################################################
+
+
+    #############################################################
+    # Stage 2: fine tune execution of jobs in specific scenarios
+    # where the catch all logic is inapprorpaite
+    #############################################################
+
+
+    #############################################################
+    # Stage 3: catch all logic applying to any job not matching
+    # an earlier criteria
+    #############################################################
+
+    # Jobs can run if any jobs they depend on were successfull
+    - when: on_success
diff --git a/.gitlab-ci.d/qemu-project.yml b/.gitlab-ci.d/qemu-project.yml
index 871262fe0e..691d9bf5dc 100644
--- a/.gitlab-ci.d/qemu-project.yml
+++ b/.gitlab-ci.d/qemu-project.yml
@@ -2,6 +2,7 @@
 # https://gitlab.com/qemu-project/qemu/-/pipelines
 
 include:
+  - local: '/.gitlab-ci.d/base.yml'
   - local: '/.gitlab-ci.d/stages.yml'
   - local: '/.gitlab-ci.d/edk2.yml'
   - local: '/.gitlab-ci.d/opensbi.yml'
diff --git a/docs/devel/ci-jobs.rst.inc b/docs/devel/ci-jobs.rst.inc
index 92e25872aa..eb6a9e6122 100644
--- a/docs/devel/ci-jobs.rst.inc
+++ b/docs/devel/ci-jobs.rst.inc
@@ -28,7 +28,35 @@ For further information about how to set these variables, please refer to::
 
   https://docs.gitlab.com/ee/user/project/push_options.html#push-options-for-gitlab-cicd
 
-Here is a list of the most used variables:
+Variable naming and grouping
+----------------------------
+
+The variables used by QEMU's CI configuration are grouped together
+in a handful of namespaces
+
+ * QEMU_JOB_nnnn - variables to be defined in individual jobs
+   or templates, to influence the shared rules defined in the
+   .base_job_template.
+
+ * QEMU_CI_nnn - variables to be set by contributors in their
+   repository CI settings, or as git push variables, to influence
+   which jobs get run in a pipeline
+
+ * nnn - other misc variables not falling into the above
+   categories, or using different names for historical reasons
+   and not yet converted.
+
+Maintainer controlled job variables
+-----------------------------------
+
+The following variables may be set when defining a job in the
+CI configuration file.
+
+Contributor controlled runtime variables
+----------------------------------------
+
+The following variables may be set by contributors to control
+job execution
 
 QEMU_CI_AVOCADO_TESTING
 ~~~~~~~~~~~~~~~~~~~~~~~
@@ -38,6 +66,12 @@ these artifacts are not already cached, downloading them make the jobs
 reach the timeout limit). Set this variable to have the tests using the
 Avocado framework run automatically.
 
+Other misc variables
+--------------------
+
+These variables are primarily to control execution of jobs on
+private runners
+
 AARCH64_RUNNER_AVAILABLE
 ~~~~~~~~~~~~~~~~~~~~~~~~
 If you've got access to an aarch64 host that can be used as a gitlab-CI

From 00125414ba1b9d04dfec3af6fb75e39591b53892 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Fri, 27 May 2022 16:35:59 +0100
Subject: [PATCH 730/935] gitlab: convert Cirrus jobs to .base_job_template
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This folds the Cirrus job rules into the base job
template, introducing two new variables

  - QEMU_JOB_CIRRUS - identifies the job as making
    use of Cirrus CI via cirrus-run

  - QEMU_JOB_OPTIONAL - identifies the job as one
    that is not run by default, primarily due to
    resource constraints. It can be manually invoked
    by users if they wish to validate that scenario.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220526110705.59952-3-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220527153603.887929-30-alex.bennee@linaro.org>
---
 .gitlab-ci.d/base.yml      |  9 +++++++++
 .gitlab-ci.d/cirrus.yml    | 16 ++++++++--------
 docs/devel/ci-jobs.rst.inc | 14 ++++++++++++++
 3 files changed, 31 insertions(+), 8 deletions(-)

diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml
index 10eb6ab8bc..5734caf9fe 100644
--- a/.gitlab-ci.d/base.yml
+++ b/.gitlab-ci.d/base.yml
@@ -12,12 +12,21 @@
     # want jobs to run
     #############################################################
 
+    # Cirrus jobs can't run unless the creds / target repo are set
+    - if: '$QEMU_JOB_CIRRUS && ($CIRRUS_GITHUB_REPO == "" || $CIRRUS_API_TOKEN == "")'
+      when: never
+
 
     #############################################################
     # Stage 2: fine tune execution of jobs in specific scenarios
     # where the catch all logic is inapprorpaite
     #############################################################
 
+    # Optional jobs should not be run unless manually triggered
+    - if: '$QEMU_JOB_OPTIONAL'
+      when: manual
+      allow_failure: true
+
 
     #############################################################
     # Stage 3: catch all logic applying to any job not matching
diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
index b96b22e269..609c364308 100644
--- a/.gitlab-ci.d/cirrus.yml
+++ b/.gitlab-ci.d/cirrus.yml
@@ -11,6 +11,7 @@
 # special care, because we can't just override it at the GitLab CI job
 # definition level or we risk breaking it completely.
 .cirrus_build_job:
+  extends: .base_job_template
   stage: build
   image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master
   needs: []
@@ -40,11 +41,8 @@
       <.gitlab-ci.d/cirrus/build.yml >.gitlab-ci.d/cirrus/$NAME.yml
     - cat .gitlab-ci.d/cirrus/$NAME.yml
     - cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
-  rules:
-    # Allow on 'staging' branch and 'stable-X.Y-staging' branches only
-    - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH !~ /staging/'
-      when: never
-    - if: "$CIRRUS_GITHUB_REPO && $CIRRUS_API_TOKEN"
+  variables:
+    QEMU_JOB_CIRRUS: 1
 
 x64-freebsd-12-build:
   extends: .cirrus_build_job
@@ -90,11 +88,11 @@ x64-macos-11-base-build:
 
 # The following jobs run VM-based tests via KVM on a Linux-based Cirrus-CI job
 .cirrus_kvm_job:
+  extends: .base_job_template
   stage: build
   image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master
   needs: []
   timeout: 80m
-  allow_failure: true
   script:
     - sed -e "s|[@]CI_REPOSITORY_URL@|$CI_REPOSITORY_URL|g"
           -e "s|[@]CI_COMMIT_REF_NAME@|$CI_COMMIT_REF_NAME|g"
@@ -105,8 +103,10 @@ x64-macos-11-base-build:
       <.gitlab-ci.d/cirrus/kvm-build.yml >.gitlab-ci.d/cirrus/$NAME.yml
     - cat .gitlab-ci.d/cirrus/$NAME.yml
     - cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
-  rules:
-    - when: manual
+  variables:
+    QEMU_JOB_CIRRUS: 1
+    QEMU_JOB_OPTIONAL: 1
+
 
 x86-netbsd:
   extends: .cirrus_kvm_job
diff --git a/docs/devel/ci-jobs.rst.inc b/docs/devel/ci-jobs.rst.inc
index eb6a9e6122..a539f502da 100644
--- a/docs/devel/ci-jobs.rst.inc
+++ b/docs/devel/ci-jobs.rst.inc
@@ -52,6 +52,20 @@ Maintainer controlled job variables
 The following variables may be set when defining a job in the
 CI configuration file.
 
+QEMU_JOB_CIRRUS
+~~~~~~~~~~~~~~~
+
+The job makes use of Cirrus CI infrastructure, requiring the
+configuration setup for cirrus-run to be present in the repository
+
+QEMU_JOB_OPTIONAL
+~~~~~~~~~~~~~~~~~
+
+The job is expected to be successful in general, but is not run
+by default due to need to conserve limited CI resources. It is
+available to be started manually by the contributor in the CI
+pipelines UI.
+
 Contributor controlled runtime variables
 ----------------------------------------
 

From 16fee101d9011ea5dfd56a055d19df802e59eb29 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Fri, 27 May 2022 16:36:00 +0100
Subject: [PATCH 731/935] gitlab: convert static checks to .base_job_template
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This folds the static checks into using the base job
template rules, introducing one new variable

 - QEMU_JOB_ONLY_FORKS - a job that should never run
   on an upstream pipeline. The information it reports
   is only applicable to contributors in a pre-submission
   scenario, not time of merge.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220526110705.59952-4-berrange@redhat.com>
[AJB: fix typo]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220527153603.887929-31-alex.bennee@linaro.org>
---
 .gitlab-ci.d/base.yml          |  4 ++++
 .gitlab-ci.d/static_checks.yml | 19 +++++++------------
 docs/devel/ci-jobs.rst.inc     |  7 +++++++
 3 files changed, 18 insertions(+), 12 deletions(-)

diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml
index 5734caf9fe..e6953c77ae 100644
--- a/.gitlab-ci.d/base.yml
+++ b/.gitlab-ci.d/base.yml
@@ -16,6 +16,10 @@
     - if: '$QEMU_JOB_CIRRUS && ($CIRRUS_GITHUB_REPO == "" || $CIRRUS_API_TOKEN == "")'
       when: never
 
+    # Jobs only intended for forks should always be skipped on upstream
+    - if: '$QEMU_JOB_ONLY_FORKS == "1" && $CI_PROJECT_NAMESPACE == "qemu-project"'
+      when: never
+
 
     #############################################################
     # Stage 2: fine tune execution of jobs in specific scenarios
diff --git a/.gitlab-ci.d/static_checks.yml b/.gitlab-ci.d/static_checks.yml
index 94858e3272..289ad1359e 100644
--- a/.gitlab-ci.d/static_checks.yml
+++ b/.gitlab-ci.d/static_checks.yml
@@ -1,4 +1,5 @@
 check-patch:
+  extends: .base_job_template
   stage: build
   image: python:3.10-alpine
   needs: []
@@ -6,15 +7,13 @@ check-patch:
     - .gitlab-ci.d/check-patch.py
   variables:
     GIT_DEPTH: 1000
+    QEMU_JOB_ONLY_FORKS: 1
   before_script:
     - apk -U add git perl
-  rules:
-    - if: '$CI_PROJECT_NAMESPACE == "qemu-project"'
-      when: never
-    - when: on_success
-      allow_failure: true
+  allow_failure: true
 
 check-dco:
+  extends: .base_job_template
   stage: build
   image: python:3.10-alpine
   needs: []
@@ -23,12 +22,9 @@ check-dco:
     GIT_DEPTH: 1000
   before_script:
     - apk -U add git
-  rules:
-    - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
-      when: never
-    - when: on_success
 
 check-python-pipenv:
+  extends: .base_job_template
   stage: test
   image: $CI_REGISTRY_IMAGE/qemu/python:latest
   script:
@@ -39,6 +35,7 @@ check-python-pipenv:
     job: python-container
 
 check-python-tox:
+  extends: .base_job_template
   stage: test
   image: $CI_REGISTRY_IMAGE/qemu/python:latest
   script:
@@ -46,8 +43,6 @@ check-python-tox:
   variables:
     GIT_DEPTH: 1
     QEMU_TOX_EXTRA_ARGS: --skip-missing-interpreters=false
+    QEMU_JOB_OPTIONAL: 1
   needs:
     job: python-container
-  rules:
-    - when: manual
-  allow_failure: true
diff --git a/docs/devel/ci-jobs.rst.inc b/docs/devel/ci-jobs.rst.inc
index a539f502da..4c7e30ab08 100644
--- a/docs/devel/ci-jobs.rst.inc
+++ b/docs/devel/ci-jobs.rst.inc
@@ -66,6 +66,13 @@ by default due to need to conserve limited CI resources. It is
 available to be started manually by the contributor in the CI
 pipelines UI.
 
+QEMU_JOB_ONLY_FORKS
+~~~~~~~~~~~~~~~~~~~
+
+The job results are only of interest to contributors prior to
+submitting code. They are not required as part of the gating
+CI pipeline.
+
 Contributor controlled runtime variables
 ----------------------------------------
 

From e312d1fdbbb3a53ee30ab84203344588154129f9 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Fri, 27 May 2022 16:36:01 +0100
Subject: [PATCH 732/935] gitlab: convert build/container jobs to
 .base_job_template
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This converts the main build and container jobs to use the
base job rules, defining the following new variables

 - QEMU_JOB_SKIPPED - jobs that are known to be currently
   broken and should not be run. Can still be manually
   launched if desired.

 - QEMU_JOB_AVOCADO - jobs that run the Avocado integration
   test harness.

 - QEMU_JOB_PUBLISH - jobs that publish content after the
   branch is merged upstream

As build-tools-and-docs runs on master we declare the requirement of
building amd64-debian-container optional as it should already exits
once we merge.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220526110705.59952-5-berrange@redhat.com>
[AJB: fix upstream typo, mention optional container req]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220527153603.887929-32-alex.bennee@linaro.org>
---
 .gitlab-ci.d/base.yml                | 22 ++++++++++++++++++++++
 .gitlab-ci.d/buildtest-template.yml  | 16 ++++------------
 .gitlab-ci.d/buildtest.yml           | 28 +++++++++++++---------------
 .gitlab-ci.d/container-cross.yml     |  6 ++----
 .gitlab-ci.d/container-template.yml  |  1 +
 .gitlab-ci.d/crossbuild-template.yml |  3 +++
 .gitlab-ci.d/windows.yml             |  1 +
 docs/devel/ci-jobs.rst.inc           | 19 +++++++++++++++++++
 8 files changed, 65 insertions(+), 31 deletions(-)

diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml
index e6953c77ae..4f091d5aad 100644
--- a/.gitlab-ci.d/base.yml
+++ b/.gitlab-ci.d/base.yml
@@ -16,10 +16,22 @@
     - if: '$QEMU_JOB_CIRRUS && ($CIRRUS_GITHUB_REPO == "" || $CIRRUS_API_TOKEN == "")'
       when: never
 
+    # Publishing jobs should only run on the default branch in upstream
+    - if: '$QEMU_JOB_PUBLISH == "1" && $CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH'
+      when: never
+
+    # Non-publishing jobs should only run on staging branches in upstream
+    - if: '$QEMU_JOB_PUBLISH != "1" && $CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH !~ /staging/'
+      when: never
+
     # Jobs only intended for forks should always be skipped on upstream
     - if: '$QEMU_JOB_ONLY_FORKS == "1" && $CI_PROJECT_NAMESPACE == "qemu-project"'
       when: never
 
+    # Avocado jobs don't run in forks unless $QEMU_CI_AVOCADO_TESTING is set
+    - if: '$QEMU_JOB_AVOCADO && $QEMU_CI_AVOCADO_TESTING != "1" && $CI_PROJECT_NAMESPACE != "qemu-project"'
+      when: never
+
 
     #############################################################
     # Stage 2: fine tune execution of jobs in specific scenarios
@@ -31,6 +43,16 @@
       when: manual
       allow_failure: true
 
+    # Skipped jobs should not be run unless manually triggered
+    - if: '$QEMU_JOB_SKIPPED'
+      when: manual
+      allow_failure: true
+
+    # Avocado jobs can be manually start in forks if $QEMU_CI_AVOCADO_TESTING is unset
+    - if: '$QEMU_JOB_AVOCADO && $CI_PROJECT_NAMESPACE != "qemu-project"'
+      when: manual
+      allow_failure: true
+
 
     #############################################################
     # Stage 3: catch all logic applying to any job not matching
diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index b381345dbc..73ecfabb8d 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -1,4 +1,5 @@
 .native_build_job_template:
+  extends: .base_job_template
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
   before_script:
@@ -27,6 +28,7 @@
       fi
 
 .common_test_job_template:
+  extends: .base_job_template
   stage: test
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
   script:
@@ -77,15 +79,5 @@
   after_script:
     - cd build
     - du -chs ${CI_PROJECT_DIR}/avocado-cache
-  rules:
-    # Only run these jobs if running on the mainstream namespace,
-    # or if the user set the QEMU_CI_AVOCADO_TESTING variable (either
-    # in its namespace setting or via git-push option, see documentation
-    # in /.gitlab-ci.yml of this repository).
-    - if: '$CI_PROJECT_NAMESPACE == "qemu-project"'
-      when: on_success
-    - if: '$QEMU_CI_AVOCADO_TESTING'
-      when: on_success
-    # Otherwise, set to manual (the jobs are created but not run).
-    - when: manual
-      allow_failure: true
+  variables:
+    QEMU_JOB_AVOCADO: 1
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index e9620c3074..ecac3ec50c 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -360,12 +360,11 @@ build-cfi-aarch64:
     expire_in: 2 days
     paths:
       - build
-  rules:
+  variables:
     # FIXME: This job is often failing, likely due to out-of-memory problems in
     # the constrained containers of the shared runners. Thus this is marked as
-    # manual until the situation has been solved.
-    - when: manual
-      allow_failure: true
+    # skipped until the situation has been solved.
+    QEMU_JOB_SKIPPED: 1
 
 check-cfi-aarch64:
   extends: .native_test_job_template
@@ -402,12 +401,11 @@ build-cfi-ppc64-s390x:
     expire_in: 2 days
     paths:
       - build
-  rules:
+  variables:
     # FIXME: This job is often failing, likely due to out-of-memory problems in
     # the constrained containers of the shared runners. Thus this is marked as
-    # manual until the situation has been solved.
-    - when: manual
-      allow_failure: true
+    # skipped until the situation has been solved.
+    QEMU_JOB_SKIPPED: 1
 
 check-cfi-ppc64-s390x:
   extends: .native_test_job_template
@@ -579,6 +577,7 @@ build-without-default-features:
     MAKE_CHECK_ARGS: check-unit check-qtest SPEED=slow
 
 build-libvhost-user:
+  extends: .base_job_template
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/fedora:latest
   needs:
@@ -595,10 +594,13 @@ build-tools-and-docs-debian:
   extends: .native_build_job_template
   needs:
     job: amd64-debian-container
+    # when running on 'master' we use pre-existing container
+    optional: true
   variables:
     IMAGE: debian-amd64
     MAKE_CHECK_ARGS: check-unit check-softfloat ctags TAGS cscope
     CONFIGURE_ARGS: --disable-system --disable-user --enable-docs --enable-tools
+    QEMU_JOB_PUBLISH: 1
   artifacts:
     expire_in: 2 days
     paths:
@@ -618,6 +620,7 @@ build-tools-and-docs-debian:
 # that users can see the results of their commits, regardless
 # of what topic branch they're currently using
 pages:
+  extends: .base_job_template
   image: $CI_REGISTRY_IMAGE/qemu/debian-amd64:latest
   stage: test
   needs:
@@ -635,10 +638,5 @@ pages:
   artifacts:
     paths:
       - public
-  rules:
-    - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
-      when: on_success
-    - if: '$CI_PROJECT_NAMESPACE == "qemu-project"'
-      when: never
-    - if: '$CI_PROJECT_NAMESPACE != "qemu-project"'
-      when: on_success
+  variables:
+    QEMU_JOB_PUBLISH: 1
diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 147e667744..b7963498a3 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -40,15 +40,13 @@ armhf-debian-cross-container:
 # We never want to build hexagon in the CI system and by default we
 # always want to refer to the master registry where it lives.
 hexagon-cross-container:
+  extends: .base_job_template
   image: docker:stable
   stage: containers
-  rules:
-    - if: '$CI_PROJECT_NAMESPACE == "qemu-project"'
-      when: never
-    - when: always
   variables:
     NAME: debian-hexagon-cross
     GIT_DEPTH: 1
+    QEMU_JOB_ONLY_FORKS: 1
   services:
     - docker:dind
   before_script:
diff --git a/.gitlab-ci.d/container-template.yml b/.gitlab-ci.d/container-template.yml
index 1baecd9460..c434b9c8f3 100644
--- a/.gitlab-ci.d/container-template.yml
+++ b/.gitlab-ci.d/container-template.yml
@@ -1,4 +1,5 @@
 .container_job_template:
+  extends: .base_job_template
   image: docker:stable
   stage: containers
   services:
diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
index 29c3c2b826..28b2142ec2 100644
--- a/.gitlab-ci.d/crossbuild-template.yml
+++ b/.gitlab-ci.d/crossbuild-template.yml
@@ -1,4 +1,5 @@
 .cross_system_build_job:
+  extends: .base_job_template
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
   timeout: 80m
@@ -24,6 +25,7 @@
 # KVM), and set extra options (such disabling other accelerators) via the
 # $EXTRA_CONFIGURE_OPTS variable.
 .cross_accel_build_job:
+  extends: .base_job_template
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
   timeout: 30m
@@ -36,6 +38,7 @@
     - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
 
 .cross_user_build_job:
+  extends: .base_job_template
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
   script:
diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
index cf7724b8e5..1b2ede49e1 100644
--- a/.gitlab-ci.d/windows.yml
+++ b/.gitlab-ci.d/windows.yml
@@ -1,4 +1,5 @@
 .shared_msys2_builder:
+  extends: .base_job_template
   tags:
   - shared-windows
   - windows
diff --git a/docs/devel/ci-jobs.rst.inc b/docs/devel/ci-jobs.rst.inc
index 4c7e30ab08..0b4926e537 100644
--- a/docs/devel/ci-jobs.rst.inc
+++ b/docs/devel/ci-jobs.rst.inc
@@ -73,6 +73,25 @@ The job results are only of interest to contributors prior to
 submitting code. They are not required as part of the gating
 CI pipeline.
 
+QEMU_JOB_SKIPPED
+~~~~~~~~~~~~~~~~
+
+The job is not reliably successsful in general, so is not
+currently suitable to be run by default. Ideally this should
+be a temporary marker until the problems can be addressed, or
+the job permanently removed.
+
+QEMU_JOB_PUBLISH
+~~~~~~~~~~~~~~~~
+
+The job is for publishing content after a branch has been
+merged into the upstream default branch.
+
+QEMU_JOB_AVOCADO
+~~~~~~~~~~~~~~~~
+
+The job runs the Avocado integration test suite
+
 Contributor controlled runtime variables
 ----------------------------------------
 

From 28357dc525b4798cdef1101cbb459afcd7233280 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>
Date: Fri, 27 May 2022 16:36:02 +0100
Subject: [PATCH 733/935] gitlab: don't run CI jobs in forks by default
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

To preserve CI shared runner credits we don't want to run
pipelines on every push.

This sets up the config so that pipelines are never created
for contributors by default. To override this the QEMU_CI
variable can be set to a non-zero value. If set to 1, the
pipeline will be created but all jobs will remain manually
started. The contributor can selectively run jobs that they
care about. If set to 2, the pipeline will be created and
all jobs will immediately start.

This behavior can be controlled using push variables

  git push -o ci.variable=QEMU_CI=1

To make this more convenient define an alias

   git config --local alias.push-ci "push -o ci.variable=QEMU_CI=1"
   git config --local alias.push-ci-now "push -o ci.variable=QEMU_CI=2"

Which lets you run

  git push-ci

to create the pipeline, or

  git push-ci-now

to create and run the pipeline

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220526110705.59952-6-berrange@redhat.com>
[AJB: fix typo, replicate alias tips in ci.rst]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220527153603.887929-33-alex.bennee@linaro.org>
---
 .gitlab-ci.d/base.yml      |  9 +++++++++
 docs/devel/ci-jobs.rst.inc | 38 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 47 insertions(+)

diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml
index 4f091d5aad..f334f3ded7 100644
--- a/.gitlab-ci.d/base.yml
+++ b/.gitlab-ci.d/base.yml
@@ -28,6 +28,10 @@
     - if: '$QEMU_JOB_ONLY_FORKS == "1" && $CI_PROJECT_NAMESPACE == "qemu-project"'
       when: never
 
+    # Forks don't get pipelines unless QEMU_CI=1 or QEMU_CI=2 is set
+    - if: '$QEMU_CI != "1" && $QEMU_CI != "2" && $CI_PROJECT_NAMESPACE != "qemu-project"'
+      when: never
+
     # Avocado jobs don't run in forks unless $QEMU_CI_AVOCADO_TESTING is set
     - if: '$QEMU_JOB_AVOCADO && $QEMU_CI_AVOCADO_TESTING != "1" && $CI_PROJECT_NAMESPACE != "qemu-project"'
       when: never
@@ -59,5 +63,10 @@
     # an earlier criteria
     #############################################################
 
+    # Forks pipeline jobs don't start automatically unless
+    # QEMU_CI=2 is set
+    - if: '$QEMU_CI != "2" && $CI_PROJECT_NAMESPACE != "qemu-project"'
+      when: manual
+
     # Jobs can run if any jobs they depend on were successfull
     - when: on_success
diff --git a/docs/devel/ci-jobs.rst.inc b/docs/devel/ci-jobs.rst.inc
index 0b4926e537..9118a61a17 100644
--- a/docs/devel/ci-jobs.rst.inc
+++ b/docs/devel/ci-jobs.rst.inc
@@ -28,6 +28,32 @@ For further information about how to set these variables, please refer to::
 
   https://docs.gitlab.com/ee/user/project/push_options.html#push-options-for-gitlab-cicd
 
+Setting aliases in your git config
+----------------------------------
+
+You can use aliases to make it easier to push branches with different
+CI configurations. For example define an alias for triggering CI:
+
+.. code::
+
+   git config --local alias.push-ci "push -o ci.variable=QEMU_CI=1"
+   git config --local alias.push-ci-now "push -o ci.variable=QEMU_CI=2"
+
+Which lets you run:
+
+.. code::
+
+   git push-ci
+
+to create the pipeline, or:
+
+.. code::
+
+   git push-ci-now
+
+to create and run the pipeline
+
+
 Variable naming and grouping
 ----------------------------
 
@@ -98,6 +124,18 @@ Contributor controlled runtime variables
 The following variables may be set by contributors to control
 job execution
 
+QEMU_CI
+~~~~~~~
+
+By default, no pipelines will be created on contributor forks
+in order to preserve CI credits
+
+Set this variable to 1 to create the pipelines, but leave all
+the jobs to be manually started from the UI
+
+Set this variable to 2 to create the pipelines and run all
+the jobs immediately, as was historicaly behaviour
+
 QEMU_CI_AVOCADO_TESTING
 ~~~~~~~~~~~~~~~~~~~~~~~
 By default, tests using the Avocado framework are not run automatically in

From 7266ecce502c31387a3cbf83d7297bc9cf27b139 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Fri, 27 May 2022 16:36:03 +0100
Subject: [PATCH 734/935] docs/devel: clean-up the CI links in the docs
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

There where some broken links so fix those up with proper references
to the devel docs. I also did a little light copy-editing to reflect
the current state and broke up a paragraph to reduce the "wall of
text" effect.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-34-alex.bennee@linaro.org>
---
 docs/devel/ci-jobs.rst.inc        |  2 ++
 docs/devel/ci.rst                 | 11 +++++-----
 docs/devel/submitting-a-patch.rst | 36 ++++++++++++++++---------------
 docs/devel/testing.rst            |  2 ++
 4 files changed, 29 insertions(+), 22 deletions(-)

diff --git a/docs/devel/ci-jobs.rst.inc b/docs/devel/ci-jobs.rst.inc
index 9118a61a17..1f28fec0d0 100644
--- a/docs/devel/ci-jobs.rst.inc
+++ b/docs/devel/ci-jobs.rst.inc
@@ -1,3 +1,5 @@
+.. _ci_var:
+
 Custom CI/CD variables
 ======================
 
diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
index d106610096..ed88a2010b 100644
--- a/docs/devel/ci.rst
+++ b/docs/devel/ci.rst
@@ -1,12 +1,13 @@
+.. _ci:
+
 ==
 CI
 ==
 
-QEMU has configurations enabled for a number of different CI services.
-The most up to date information about them and their status can be
-found at::
-
-   https://wiki.qemu.org/Testing/CI
+Most of QEMU's CI is run on GitLab's infrastructure although a number
+of other CI services are used for specialised purposes. The most up to
+date information about them and their status can be found on the
+`project wiki testing page <https://wiki.qemu.org/Testing/CI>`_.
 
 .. include:: ci-definitions.rst.inc
 .. include:: ci-jobs.rst.inc
diff --git a/docs/devel/submitting-a-patch.rst b/docs/devel/submitting-a-patch.rst
index e51259eb9c..d3876ec1b7 100644
--- a/docs/devel/submitting-a-patch.rst
+++ b/docs/devel/submitting-a-patch.rst
@@ -204,23 +204,25 @@ log`` for these keywords for example usage.
 Test your patches
 ~~~~~~~~~~~~~~~~~
 
-Although QEMU has `continuous integration
-services <Testing#Continuous_Integration>`__ that attempt to test
-patches submitted to the list, it still saves everyone time if you have
-already tested that your patch compiles and works. Because QEMU is such
-a large project, it's okay to use configure arguments to limit what is
-built for faster turnaround during your development time; but it is
-still wise to also check that your patches work with a full build before
-submitting a series, especially if your changes might have an unintended
-effect on other areas of the code you don't normally experiment with.
-See `Testing <Testing>`__ for more details on what tests are available.
-Also, it is a wise idea to include a testsuite addition as part of your
-patches - either to ensure that future changes won't regress your new
-feature, or to add a test which exposes the bug that the rest of your
-series fixes. Keeping separate commits for the test and the fix allows
-reviewers to rebase the test to occur first to prove it catches the
-problem, then again to place it last in the series so that bisection
-doesn't land on a known-broken state.
+Although QEMU uses various :ref:`ci` services that attempt to test
+patches submitted to the list, it still saves everyone time if you
+have already tested that your patch compiles and works. Because QEMU
+is such a large project the default configuration won't create a
+testing pipeline on GitLab when a branch is pushed. See the :ref:`CI
+variable documentation<ci_var>` for details on how to control the
+running of tests; but it is still wise to also check that your patches
+work with a full build before submitting a series, especially if your
+changes might have an unintended effect on other areas of the code you
+don't normally experiment with. See :ref:`testing` for more details on
+what tests are available.
+
+Also, it is a wise idea to include a testsuite addition as part of
+your patches - either to ensure that future changes won't regress your
+new feature, or to add a test which exposes the bug that the rest of
+your series fixes. Keeping separate commits for the test and the fix
+allows reviewers to rebase the test to occur first to prove it catches
+the problem, then again to place it last in the series so that
+bisection doesn't land on a known-broken state.
 
 .. _submitting_your_patches:
 
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 5b60a31807..3f6ebd5073 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -1,3 +1,5 @@
+.. _testing:
+
 Testing in QEMU
 ===============
 

From 4d84bb6c8b42cc781a02e1ac6648875966abc877 Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Wed, 25 May 2022 08:59:04 -0400
Subject: [PATCH 735/935] hw/tpm/tpm_tis_common.c: Assert that locty is in
 range
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

In tpm_tis_mmio_read(), tpm_tis_mmio_write() and
tpm_tis_dump_state(), we calculate a locality index with
tpm_tis_locality_from_addr() and then use it as an index into the
s->loc[] array.  In all these cases, the array index can't overflow
because the MemoryRegion is sized to be TPM_TIS_NUM_LOCALITIES <<
TPM_TIS_LOCALITY_SHIFT bytes.  However, Coverity can't see that, and
it complains (CID 1487138, 1487180, 1487188, 1487198, 1487240).

Add an assertion to tpm_tis_locality_from_addr() that the calculated
locality index is valid, which will help Coverity and also catch any
potential future bug where the MemoryRegion isn't sized exactly.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-id: 20220525125904.483075-1-stefanb@linux.ibm.com
---
 hw/tpm/tpm_tis_common.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/hw/tpm/tpm_tis_common.c b/hw/tpm/tpm_tis_common.c
index e700d82181..503be2a541 100644
--- a/hw/tpm/tpm_tis_common.c
+++ b/hw/tpm/tpm_tis_common.c
@@ -50,7 +50,12 @@ static uint64_t tpm_tis_mmio_read(void *opaque, hwaddr addr,
 
 static uint8_t tpm_tis_locality_from_addr(hwaddr addr)
 {
-    return (uint8_t)((addr >> TPM_TIS_LOCALITY_SHIFT) & 0x7);
+    uint8_t locty;
+
+    locty = (uint8_t)((addr >> TPM_TIS_LOCALITY_SHIFT) & 0x7);
+    assert(TPM_TIS_IS_VALID_LOCTY(locty));
+
+    return locty;
 }
 
 

From 79e1d527e13d35b976c947c48a70c23ef3586e76 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 18:33:45 -0700
Subject: [PATCH 736/935] target/m68k: Raise the TRAPn exception with the
 correct pc

Rather than adjust the PC in all of the consumers, raise
the exception with the correct PC in the first place.

Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220602013401.303699-2-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/m68k/cpu_loop.c | 1 -
 target/m68k/op_helper.c    | 9 ---------
 target/m68k/translate.c    | 2 +-
 3 files changed, 1 insertion(+), 11 deletions(-)

diff --git a/linux-user/m68k/cpu_loop.c b/linux-user/m68k/cpu_loop.c
index d1bf8548b7..56417f7401 100644
--- a/linux-user/m68k/cpu_loop.c
+++ b/linux-user/m68k/cpu_loop.c
@@ -56,7 +56,6 @@ void cpu_loop(CPUM68KState *env)
             {
                 abi_long ret;
                 n = env->dregs[0];
-                env->pc += 2;
                 ret = do_syscall(env,
                                  n,
                                  env->dregs[1],
diff --git a/target/m68k/op_helper.c b/target/m68k/op_helper.c
index 8decc61240..d30f988ae0 100644
--- a/target/m68k/op_helper.c
+++ b/target/m68k/op_helper.c
@@ -217,11 +217,6 @@ static void cf_interrupt_all(CPUM68KState *env, int is_hw)
             cpu_loop_exit(cs);
             return;
         }
-        if (cs->exception_index >= EXCP_TRAP0
-            && cs->exception_index <= EXCP_TRAP15) {
-            /* Move the PC after the trap instruction.  */
-            retaddr += 2;
-        }
     }
 
     vector = cs->exception_index << 2;
@@ -304,10 +299,6 @@ static void m68k_interrupt_all(CPUM68KState *env, int is_hw)
             /* Return from an exception.  */
             m68k_rte(env);
             return;
-        case EXCP_TRAP0 ...  EXCP_TRAP15:
-            /* Move the PC after the trap instruction.  */
-            retaddr += 2;
-            break;
         }
     }
 
diff --git a/target/m68k/translate.c b/target/m68k/translate.c
index e4efd988d2..22e5379d3c 100644
--- a/target/m68k/translate.c
+++ b/target/m68k/translate.c
@@ -4860,7 +4860,7 @@ DISAS_INSN(wdebug)
 
 DISAS_INSN(trap)
 {
-    gen_exception(s, s->base.pc_next, EXCP_TRAP0 + (insn & 0xf));
+    gen_exception(s, s->pc, EXCP_TRAP0 + (insn & 0xf));
 }
 
 static void gen_load_fcr(DisasContext *s, TCGv res, int reg)

From 02ea42b36ddcd34739c9320ae2262b1d1e814a6d Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 18:33:46 -0700
Subject: [PATCH 737/935] target/m68k: Switch over exception type in
 m68k_interrupt_all
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Replace an if ladder with a switch for clarity.

Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220602013401.303699-3-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 target/m68k/op_helper.c | 49 +++++++++++++++++++++++++----------------
 1 file changed, 30 insertions(+), 19 deletions(-)

diff --git a/target/m68k/op_helper.c b/target/m68k/op_helper.c
index d30f988ae0..2b94a6ec84 100644
--- a/target/m68k/op_helper.c
+++ b/target/m68k/op_helper.c
@@ -333,7 +333,8 @@ static void m68k_interrupt_all(CPUM68KState *env, int is_hw)
         sp &= ~1;
     }
 
-    if (cs->exception_index == EXCP_ACCESS) {
+    switch (cs->exception_index) {
+    case EXCP_ACCESS:
         if (env->mmu.fault) {
             cpu_abort(cs, "DOUBLE MMU FAULT\n");
         }
@@ -391,29 +392,39 @@ static void m68k_interrupt_all(CPUM68KState *env, int is_hw)
                      "ssw:  %08x ea:   %08x sfc:  %d    dfc: %d\n",
                      env->mmu.ssw, env->mmu.ar, env->sfc, env->dfc);
         }
-    } else if (cs->exception_index == EXCP_ADDRESS) {
+        break;
+
+    case EXCP_ADDRESS:
         do_stack_frame(env, &sp, 2, oldsr, 0, retaddr);
-    } else if (cs->exception_index == EXCP_ILLEGAL ||
-               cs->exception_index == EXCP_DIV0 ||
-               cs->exception_index == EXCP_CHK ||
-               cs->exception_index == EXCP_TRAPCC ||
-               cs->exception_index == EXCP_TRACE) {
+        break;
+
+    case EXCP_ILLEGAL:
+    case EXCP_DIV0:
+    case EXCP_CHK:
+    case EXCP_TRAPCC:
+    case EXCP_TRACE:
         /* FIXME: addr is not only env->pc */
         do_stack_frame(env, &sp, 2, oldsr, env->pc, retaddr);
-    } else if (is_hw && oldsr & SR_M &&
-               cs->exception_index >= EXCP_SPURIOUS &&
-               cs->exception_index <= EXCP_INT_LEVEL_7) {
-        do_stack_frame(env, &sp, 0, oldsr, 0, retaddr);
-        oldsr = sr;
-        env->aregs[7] = sp;
-        cpu_m68k_set_sr(env, sr &= ~SR_M);
-        sp = env->aregs[7];
-        if (!m68k_feature(env, M68K_FEATURE_UNALIGNED_DATA)) {
-            sp &= ~1;
+        break;
+
+    case EXCP_SPURIOUS ... EXCP_INT_LEVEL_7:
+        if (is_hw && oldsr & SR_M) {
+            do_stack_frame(env, &sp, 0, oldsr, 0, retaddr);
+            oldsr = sr;
+            env->aregs[7] = sp;
+            cpu_m68k_set_sr(env, sr &= ~SR_M);
+            sp = env->aregs[7];
+            if (!m68k_feature(env, M68K_FEATURE_UNALIGNED_DATA)) {
+                sp &= ~1;
+            }
+            do_stack_frame(env, &sp, 1, oldsr, 0, retaddr);
+            break;
         }
-        do_stack_frame(env, &sp, 1, oldsr, 0, retaddr);
-    } else {
+        /* fall through */
+
+    default:
         do_stack_frame(env, &sp, 0, oldsr, 0, retaddr);
+        break;
     }
 
     env->aregs[7] = sp;

From eeb8f7b0f84f86b5fa1e17aed851d758e1c7ee0f Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 18:33:47 -0700
Subject: [PATCH 738/935] target/m68k: Fix coding style in m68k_interrupt_all
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add parenthesis around & vs &&.

Remove assignment to sr in function call argument -- note that
sr is unused after the call, so the assignment was never needed,
only the result of the & expression.

Suggested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220602013401.303699-4-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 target/m68k/op_helper.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/m68k/op_helper.c b/target/m68k/op_helper.c
index 2b94a6ec84..0f41c2dce3 100644
--- a/target/m68k/op_helper.c
+++ b/target/m68k/op_helper.c
@@ -408,11 +408,11 @@ static void m68k_interrupt_all(CPUM68KState *env, int is_hw)
         break;
 
     case EXCP_SPURIOUS ... EXCP_INT_LEVEL_7:
-        if (is_hw && oldsr & SR_M) {
+        if (is_hw && (oldsr & SR_M)) {
             do_stack_frame(env, &sp, 0, oldsr, 0, retaddr);
             oldsr = sr;
             env->aregs[7] = sp;
-            cpu_m68k_set_sr(env, sr &= ~SR_M);
+            cpu_m68k_set_sr(env, sr & ~SR_M);
             sp = env->aregs[7];
             if (!m68k_feature(env, M68K_FEATURE_UNALIGNED_DATA)) {
                 sp &= ~1;

From cf213dacf8d9bc37889aeaebc781f5f55d705f0d Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 18:33:48 -0700
Subject: [PATCH 739/935] linux-user/m68k: Handle EXCP_TRAP1 through
 EXCP_TRAP15

These are raised by guest instructions, and should not
fall through into the default abort case.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220602013401.303699-5-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/m68k/cpu_loop.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/linux-user/m68k/cpu_loop.c b/linux-user/m68k/cpu_loop.c
index 56417f7401..12e5d9cd53 100644
--- a/linux-user/m68k/cpu_loop.c
+++ b/linux-user/m68k/cpu_loop.c
@@ -75,7 +75,11 @@ void cpu_loop(CPUM68KState *env)
         case EXCP_INTERRUPT:
             /* just indicate that signals should be handled asap */
             break;
+        case EXCP_TRAP0 + 1 ... EXCP_TRAP0 + 14:
+            force_sig_fault(TARGET_SIGILL, TARGET_ILL_ILLTRP, env->pc);
+            break;
         case EXCP_DEBUG:
+        case EXCP_TRAP15:
             force_sig_fault(TARGET_SIGTRAP, TARGET_TRAP_BRKPT, env->pc);
             break;
         case EXCP_ATOMIC:

From 035c6e7b5d86edee9e96423a590ee3ee0546c921 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 18:33:49 -0700
Subject: [PATCH 740/935] target/m68k: Remove retaddr in m68k_interrupt_all
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The only value this variable holds is now env->pc.

Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220602013401.303699-6-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 target/m68k/op_helper.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/target/m68k/op_helper.c b/target/m68k/op_helper.c
index 0f41c2dce3..777869790b 100644
--- a/target/m68k/op_helper.c
+++ b/target/m68k/op_helper.c
@@ -287,12 +287,9 @@ static void m68k_interrupt_all(CPUM68KState *env, int is_hw)
 {
     CPUState *cs = env_cpu(env);
     uint32_t sp;
-    uint32_t retaddr;
     uint32_t vector;
     uint16_t sr, oldsr;
 
-    retaddr = env->pc;
-
     if (!is_hw) {
         switch (cs->exception_index) {
         case EXCP_RTE:
@@ -385,7 +382,7 @@ static void m68k_interrupt_all(CPUM68KState *env, int is_hw)
         sp -= 4;
         cpu_stl_mmuidx_ra(env, sp, env->mmu.ar, MMU_KERNEL_IDX, 0);
 
-        do_stack_frame(env, &sp, 7, oldsr, 0, retaddr);
+        do_stack_frame(env, &sp, 7, oldsr, 0, env->pc);
         env->mmu.fault = false;
         if (qemu_loglevel_mask(CPU_LOG_INT)) {
             qemu_log("            "
@@ -395,7 +392,7 @@ static void m68k_interrupt_all(CPUM68KState *env, int is_hw)
         break;
 
     case EXCP_ADDRESS:
-        do_stack_frame(env, &sp, 2, oldsr, 0, retaddr);
+        do_stack_frame(env, &sp, 2, oldsr, 0, env->pc);
         break;
 
     case EXCP_ILLEGAL:
@@ -404,12 +401,12 @@ static void m68k_interrupt_all(CPUM68KState *env, int is_hw)
     case EXCP_TRAPCC:
     case EXCP_TRACE:
         /* FIXME: addr is not only env->pc */
-        do_stack_frame(env, &sp, 2, oldsr, env->pc, retaddr);
+        do_stack_frame(env, &sp, 2, oldsr, env->pc, env->pc);
         break;
 
     case EXCP_SPURIOUS ... EXCP_INT_LEVEL_7:
         if (is_hw && (oldsr & SR_M)) {
-            do_stack_frame(env, &sp, 0, oldsr, 0, retaddr);
+            do_stack_frame(env, &sp, 0, oldsr, 0, env->pc);
             oldsr = sr;
             env->aregs[7] = sp;
             cpu_m68k_set_sr(env, sr & ~SR_M);
@@ -417,13 +414,13 @@ static void m68k_interrupt_all(CPUM68KState *env, int is_hw)
             if (!m68k_feature(env, M68K_FEATURE_UNALIGNED_DATA)) {
                 sp &= ~1;
             }
-            do_stack_frame(env, &sp, 1, oldsr, 0, retaddr);
+            do_stack_frame(env, &sp, 1, oldsr, 0, env->pc);
             break;
         }
         /* fall through */
 
     default:
-        do_stack_frame(env, &sp, 0, oldsr, 0, retaddr);
+        do_stack_frame(env, &sp, 0, oldsr, 0, env->pc);
         break;
     }
 

From ad5a5cf97d80501be95f5d255d2ce133e0623b50 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 18:33:50 -0700
Subject: [PATCH 741/935] target/m68k: Fix address argument for EXCP_CHK

According to the M68040 Users Manual, section 8.4.3,
Six word stack frame (format 2), CHK, CHK2 (and others)
are supposed to record the next insn in PC and the
address of the trapping instruction in ADDRESS.

Create a raise_exception_format2 function to centralize recording
of the trapping pc in mmu.ar, plus advancing to the next insn.

Update m68k_interrupt_all to pass mmu.ar to do_stack_frame.
Update cpu_loop to pass mmu.ar to siginfo.si_addr, as the
kernel does in trap_c().

Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220602013401.303699-7-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/m68k/cpu_loop.c |  2 +-
 target/m68k/cpu.h          |  6 +++++
 target/m68k/op_helper.c    | 54 ++++++++++++++++++++------------------
 3 files changed, 36 insertions(+), 26 deletions(-)

diff --git a/linux-user/m68k/cpu_loop.c b/linux-user/m68k/cpu_loop.c
index 12e5d9cd53..e24d17e180 100644
--- a/linux-user/m68k/cpu_loop.c
+++ b/linux-user/m68k/cpu_loop.c
@@ -47,7 +47,7 @@ void cpu_loop(CPUM68KState *env)
             force_sig_fault(TARGET_SIGILL, TARGET_ILL_ILLOPN, env->pc);
             break;
         case EXCP_CHK:
-            force_sig_fault(TARGET_SIGFPE, TARGET_FPE_INTOVF, env->pc);
+            force_sig_fault(TARGET_SIGFPE, TARGET_FPE_INTOVF, env->mmu.ar);
             break;
         case EXCP_DIV0:
             force_sig_fault(TARGET_SIGFPE, TARGET_FPE_INTDIV, env->pc);
diff --git a/target/m68k/cpu.h b/target/m68k/cpu.h
index 9b3bf7a448..558c3c67d6 100644
--- a/target/m68k/cpu.h
+++ b/target/m68k/cpu.h
@@ -122,6 +122,12 @@ typedef struct CPUArchState {
 
     /* MMU status.  */
     struct {
+        /*
+         * Holds the "address" value in between raising an exception
+         * and creation of the exception stack frame.
+         * Used for both Format 7 exceptions (Access, i.e. mmu)
+         * and Format 2 exceptions (chk, div0, trapcc, etc).
+         */
         uint32_t ar;
         uint32_t ssw;
         /* 68040 */
diff --git a/target/m68k/op_helper.c b/target/m68k/op_helper.c
index 777869790b..750d65576f 100644
--- a/target/m68k/op_helper.c
+++ b/target/m68k/op_helper.c
@@ -397,13 +397,16 @@ static void m68k_interrupt_all(CPUM68KState *env, int is_hw)
 
     case EXCP_ILLEGAL:
     case EXCP_DIV0:
-    case EXCP_CHK:
     case EXCP_TRAPCC:
     case EXCP_TRACE:
         /* FIXME: addr is not only env->pc */
         do_stack_frame(env, &sp, 2, oldsr, env->pc, env->pc);
         break;
 
+    case EXCP_CHK:
+        do_stack_frame(env, &sp, 2, oldsr, env->mmu.ar, env->pc);
+        break;
+
     case EXCP_SPURIOUS ... EXCP_INT_LEVEL_7:
         if (is_hw && (oldsr & SR_M)) {
             do_stack_frame(env, &sp, 0, oldsr, 0, env->pc);
@@ -548,6 +551,29 @@ void HELPER(raise_exception)(CPUM68KState *env, uint32_t tt)
     raise_exception(env, tt);
 }
 
+G_NORETURN static void
+raise_exception_format2(CPUM68KState *env, int tt, int ilen, uintptr_t raddr)
+{
+    CPUState *cs = env_cpu(env);
+
+    cs->exception_index = tt;
+
+    /* Recover PC and CC_OP for the beginning of the insn.  */
+    cpu_restore_state(cs, raddr, true);
+
+    /* Flags are current in env->cc_*, or are undefined. */
+    env->cc_op = CC_OP_FLAGS;
+
+    /*
+     * Remember original pc in mmu.ar, for the Format 2 stack frame.
+     * Adjust PC to end of the insn.
+     */
+    env->mmu.ar = env->pc;
+    env->pc += ilen;
+
+    cpu_loop_exit(cs);
+}
+
 void HELPER(divuw)(CPUM68KState *env, int destr, uint32_t den)
 {
     uint32_t num = env->dregs[destr];
@@ -1065,18 +1091,7 @@ void HELPER(chk)(CPUM68KState *env, int32_t val, int32_t ub)
     env->cc_c = 0 <= ub ? val < 0 || val > ub : val > ub && val < 0;
 
     if (val < 0 || val > ub) {
-        CPUState *cs = env_cpu(env);
-
-        /* Recover PC and CC_OP for the beginning of the insn.  */
-        cpu_restore_state(cs, GETPC(), true);
-
-        /* flags have been modified by gen_flush_flags() */
-        env->cc_op = CC_OP_FLAGS;
-        /* Adjust PC to end of the insn.  */
-        env->pc += 2;
-
-        cs->exception_index = EXCP_CHK;
-        cpu_loop_exit(cs);
+        raise_exception_format2(env, EXCP_CHK, 2, GETPC());
     }
 }
 
@@ -1097,17 +1112,6 @@ void HELPER(chk2)(CPUM68KState *env, int32_t val, int32_t lb, int32_t ub)
     env->cc_c = lb <= ub ? val < lb || val > ub : val > ub && val < lb;
 
     if (env->cc_c) {
-        CPUState *cs = env_cpu(env);
-
-        /* Recover PC and CC_OP for the beginning of the insn.  */
-        cpu_restore_state(cs, GETPC(), true);
-
-        /* flags have been modified by gen_flush_flags() */
-        env->cc_op = CC_OP_FLAGS;
-        /* Adjust PC to end of the insn.  */
-        env->pc += 4;
-
-        cs->exception_index = EXCP_CHK;
-        cpu_loop_exit(cs);
+        raise_exception_format2(env, EXCP_CHK, 4, GETPC());
     }
 }

From 710d747b2deaf5f5678aebb1fabbe00224e5cdde Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 18:33:51 -0700
Subject: [PATCH 742/935] target/m68k: Fix pc, c flag, and address argument for
 EXCP_DIV0

According to the M68040 Users Manual, section 8.4.3,
Six word stack frame (format 2), Zero Div (and others)
is supposed to record the next insn in PC and the
address of the trapping instruction in ADDRESS.

While the N, Z and V flags are documented to be undefine on DIV0,
the C flag is documented as always cleared.

Update helper_div* to take the instruction length as an argument
and use raise_exception_format2.  Hoist the reset of the C flag
above the division by zero check.

Update m68k_interrupt_all to pass mmu.ar to do_stack_frame.
Update cpu_loop to pass mmu.ar to siginfo.si_addr, as the
kernel does in trap_c().

Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220602013401.303699-8-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/m68k/cpu_loop.c |  2 +-
 target/m68k/helper.h       | 12 +++++-----
 target/m68k/op_helper.c    | 48 +++++++++++++++++++++++---------------
 target/m68k/translate.c    | 33 +++++++++++++-------------
 4 files changed, 52 insertions(+), 43 deletions(-)

diff --git a/linux-user/m68k/cpu_loop.c b/linux-user/m68k/cpu_loop.c
index e24d17e180..6598bce3c4 100644
--- a/linux-user/m68k/cpu_loop.c
+++ b/linux-user/m68k/cpu_loop.c
@@ -50,7 +50,7 @@ void cpu_loop(CPUM68KState *env)
             force_sig_fault(TARGET_SIGFPE, TARGET_FPE_INTOVF, env->mmu.ar);
             break;
         case EXCP_DIV0:
-            force_sig_fault(TARGET_SIGFPE, TARGET_FPE_INTDIV, env->pc);
+            force_sig_fault(TARGET_SIGFPE, TARGET_FPE_INTDIV, env->mmu.ar);
             break;
         case EXCP_TRAP0:
             {
diff --git a/target/m68k/helper.h b/target/m68k/helper.h
index 0a6b4146f6..f016c4c1c2 100644
--- a/target/m68k/helper.h
+++ b/target/m68k/helper.h
@@ -1,12 +1,12 @@
 DEF_HELPER_1(bitrev, i32, i32)
 DEF_HELPER_1(ff1, i32, i32)
 DEF_HELPER_FLAGS_2(sats, TCG_CALL_NO_RWG_SE, i32, i32, i32)
-DEF_HELPER_3(divuw, void, env, int, i32)
-DEF_HELPER_3(divsw, void, env, int, s32)
-DEF_HELPER_4(divul, void, env, int, int, i32)
-DEF_HELPER_4(divsl, void, env, int, int, s32)
-DEF_HELPER_4(divull, void, env, int, int, i32)
-DEF_HELPER_4(divsll, void, env, int, int, s32)
+DEF_HELPER_4(divuw, void, env, int, i32, int)
+DEF_HELPER_4(divsw, void, env, int, s32, int)
+DEF_HELPER_5(divul, void, env, int, int, i32, int)
+DEF_HELPER_5(divsl, void, env, int, int, s32, int)
+DEF_HELPER_5(divull, void, env, int, int, i32, int)
+DEF_HELPER_5(divsll, void, env, int, int, s32, int)
 DEF_HELPER_2(set_sr, void, env, i32)
 DEF_HELPER_3(cf_movec_to, void, env, i32, i32)
 DEF_HELPER_3(m68k_movec_to, void, env, i32, i32)
diff --git a/target/m68k/op_helper.c b/target/m68k/op_helper.c
index 750d65576f..729ee0e934 100644
--- a/target/m68k/op_helper.c
+++ b/target/m68k/op_helper.c
@@ -396,7 +396,6 @@ static void m68k_interrupt_all(CPUM68KState *env, int is_hw)
         break;
 
     case EXCP_ILLEGAL:
-    case EXCP_DIV0:
     case EXCP_TRAPCC:
     case EXCP_TRACE:
         /* FIXME: addr is not only env->pc */
@@ -404,6 +403,7 @@ static void m68k_interrupt_all(CPUM68KState *env, int is_hw)
         break;
 
     case EXCP_CHK:
+    case EXCP_DIV0:
         do_stack_frame(env, &sp, 2, oldsr, env->mmu.ar, env->pc);
         break;
 
@@ -574,18 +574,19 @@ raise_exception_format2(CPUM68KState *env, int tt, int ilen, uintptr_t raddr)
     cpu_loop_exit(cs);
 }
 
-void HELPER(divuw)(CPUM68KState *env, int destr, uint32_t den)
+void HELPER(divuw)(CPUM68KState *env, int destr, uint32_t den, int ilen)
 {
     uint32_t num = env->dregs[destr];
     uint32_t quot, rem;
 
+    env->cc_c = 0; /* always cleared, even if div0 */
+
     if (den == 0) {
-        raise_exception_ra(env, EXCP_DIV0, GETPC());
+        raise_exception_format2(env, EXCP_DIV0, ilen, GETPC());
     }
     quot = num / den;
     rem = num % den;
 
-    env->cc_c = 0; /* always cleared, even if overflow */
     if (quot > 0xffff) {
         env->cc_v = -1;
         /*
@@ -601,18 +602,19 @@ void HELPER(divuw)(CPUM68KState *env, int destr, uint32_t den)
     env->cc_v = 0;
 }
 
-void HELPER(divsw)(CPUM68KState *env, int destr, int32_t den)
+void HELPER(divsw)(CPUM68KState *env, int destr, int32_t den, int ilen)
 {
     int32_t num = env->dregs[destr];
     uint32_t quot, rem;
 
+    env->cc_c = 0; /* always cleared, even if overflow/div0 */
+
     if (den == 0) {
-        raise_exception_ra(env, EXCP_DIV0, GETPC());
+        raise_exception_format2(env, EXCP_DIV0, ilen, GETPC());
     }
     quot = num / den;
     rem = num % den;
 
-    env->cc_c = 0; /* always cleared, even if overflow */
     if (quot != (int16_t)quot) {
         env->cc_v = -1;
         /* nothing else is modified */
@@ -629,18 +631,20 @@ void HELPER(divsw)(CPUM68KState *env, int destr, int32_t den)
     env->cc_v = 0;
 }
 
-void HELPER(divul)(CPUM68KState *env, int numr, int regr, uint32_t den)
+void HELPER(divul)(CPUM68KState *env, int numr, int regr,
+                   uint32_t den, int ilen)
 {
     uint32_t num = env->dregs[numr];
     uint32_t quot, rem;
 
+    env->cc_c = 0; /* always cleared, even if div0 */
+
     if (den == 0) {
-        raise_exception_ra(env, EXCP_DIV0, GETPC());
+        raise_exception_format2(env, EXCP_DIV0, ilen, GETPC());
     }
     quot = num / den;
     rem = num % den;
 
-    env->cc_c = 0;
     env->cc_z = quot;
     env->cc_n = quot;
     env->cc_v = 0;
@@ -657,18 +661,20 @@ void HELPER(divul)(CPUM68KState *env, int numr, int regr, uint32_t den)
     }
 }
 
-void HELPER(divsl)(CPUM68KState *env, int numr, int regr, int32_t den)
+void HELPER(divsl)(CPUM68KState *env, int numr, int regr,
+                   int32_t den, int ilen)
 {
     int32_t num = env->dregs[numr];
     int32_t quot, rem;
 
+    env->cc_c = 0; /* always cleared, even if overflow/div0 */
+
     if (den == 0) {
-        raise_exception_ra(env, EXCP_DIV0, GETPC());
+        raise_exception_format2(env, EXCP_DIV0, ilen, GETPC());
     }
     quot = num / den;
     rem = num % den;
 
-    env->cc_c = 0;
     env->cc_z = quot;
     env->cc_n = quot;
     env->cc_v = 0;
@@ -685,19 +691,21 @@ void HELPER(divsl)(CPUM68KState *env, int numr, int regr, int32_t den)
     }
 }
 
-void HELPER(divull)(CPUM68KState *env, int numr, int regr, uint32_t den)
+void HELPER(divull)(CPUM68KState *env, int numr, int regr,
+                    uint32_t den, int ilen)
 {
     uint64_t num = deposit64(env->dregs[numr], 32, 32, env->dregs[regr]);
     uint64_t quot;
     uint32_t rem;
 
+    env->cc_c = 0; /* always cleared, even if overflow/div0 */
+
     if (den == 0) {
-        raise_exception_ra(env, EXCP_DIV0, GETPC());
+        raise_exception_format2(env, EXCP_DIV0, ilen, GETPC());
     }
     quot = num / den;
     rem = num % den;
 
-    env->cc_c = 0; /* always cleared, even if overflow */
     if (quot > 0xffffffffULL) {
         env->cc_v = -1;
         /*
@@ -720,19 +728,21 @@ void HELPER(divull)(CPUM68KState *env, int numr, int regr, uint32_t den)
     env->dregs[numr] = quot;
 }
 
-void HELPER(divsll)(CPUM68KState *env, int numr, int regr, int32_t den)
+void HELPER(divsll)(CPUM68KState *env, int numr, int regr,
+                    int32_t den, int ilen)
 {
     int64_t num = deposit64(env->dregs[numr], 32, 32, env->dregs[regr]);
     int64_t quot;
     int32_t rem;
 
+    env->cc_c = 0; /* always cleared, even if overflow/div0 */
+
     if (den == 0) {
-        raise_exception_ra(env, EXCP_DIV0, GETPC());
+        raise_exception_format2(env, EXCP_DIV0, ilen, GETPC());
     }
     quot = num / den;
     rem = num % den;
 
-    env->cc_c = 0; /* always cleared, even if overflow */
     if (quot != (int32_t)quot) {
         env->cc_v = -1;
         /*
diff --git a/target/m68k/translate.c b/target/m68k/translate.c
index 22e5379d3c..6075f49930 100644
--- a/target/m68k/translate.c
+++ b/target/m68k/translate.c
@@ -1601,6 +1601,7 @@ DISAS_INSN(divw)
     int sign;
     TCGv src;
     TCGv destr;
+    TCGv ilen;
 
     /* divX.w <EA>,Dn    32/16 -> 16r:16q */
 
@@ -1609,20 +1610,20 @@ DISAS_INSN(divw)
     /* dest.l / src.w */
 
     SRC_EA(env, src, OS_WORD, sign, NULL);
-    destr = tcg_const_i32(REG(insn, 9));
+    destr = tcg_constant_i32(REG(insn, 9));
+    ilen = tcg_constant_i32(s->pc - s->base.pc_next);
     if (sign) {
-        gen_helper_divsw(cpu_env, destr, src);
+        gen_helper_divsw(cpu_env, destr, src, ilen);
     } else {
-        gen_helper_divuw(cpu_env, destr, src);
+        gen_helper_divuw(cpu_env, destr, src, ilen);
     }
-    tcg_temp_free(destr);
 
     set_cc_op(s, CC_OP_FLAGS);
 }
 
 DISAS_INSN(divl)
 {
-    TCGv num, reg, den;
+    TCGv num, reg, den, ilen;
     int sign;
     uint16_t ext;
 
@@ -1639,15 +1640,14 @@ DISAS_INSN(divl)
         /* divX.l <EA>, Dr:Dq    64/32 -> 32r:32q */
 
         SRC_EA(env, den, OS_LONG, 0, NULL);
-        num = tcg_const_i32(REG(ext, 12));
-        reg = tcg_const_i32(REG(ext, 0));
+        num = tcg_constant_i32(REG(ext, 12));
+        reg = tcg_constant_i32(REG(ext, 0));
+        ilen = tcg_constant_i32(s->pc - s->base.pc_next);
         if (sign) {
-            gen_helper_divsll(cpu_env, num, reg, den);
+            gen_helper_divsll(cpu_env, num, reg, den, ilen);
         } else {
-            gen_helper_divull(cpu_env, num, reg, den);
+            gen_helper_divull(cpu_env, num, reg, den, ilen);
         }
-        tcg_temp_free(reg);
-        tcg_temp_free(num);
         set_cc_op(s, CC_OP_FLAGS);
         return;
     }
@@ -1656,15 +1656,14 @@ DISAS_INSN(divl)
     /* divXl.l <EA>, Dr:Dq    32/32 -> 32r:32q */
 
     SRC_EA(env, den, OS_LONG, 0, NULL);
-    num = tcg_const_i32(REG(ext, 12));
-    reg = tcg_const_i32(REG(ext, 0));
+    num = tcg_constant_i32(REG(ext, 12));
+    reg = tcg_constant_i32(REG(ext, 0));
+    ilen = tcg_constant_i32(s->pc - s->base.pc_next);
     if (sign) {
-        gen_helper_divsl(cpu_env, num, reg, den);
+        gen_helper_divsl(cpu_env, num, reg, den, ilen);
     } else {
-        gen_helper_divul(cpu_env, num, reg, den);
+        gen_helper_divul(cpu_env, num, reg, den, ilen);
     }
-    tcg_temp_free(reg);
-    tcg_temp_free(num);
 
     set_cc_op(s, CC_OP_FLAGS);
 }

From 8115fc9368e41f91a8bc27a78c2840beda989cb5 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 18:33:52 -0700
Subject: [PATCH 743/935] target/m68k: Fix address argument for EXCP_TRACE

According to the M68040 Users Manual, section 8.4.3,
Six word stack frame (format 2), Trace (and others) is
supposed to record the next insn in PC and the address
of the trapping instruction in ADDRESS.

Create gen_raise_exception_format2 to record the trapping
pc in env->mmu.ar.  Update m68k_interrupt_all to pass the
value to do_stack_frame.  Update cpu_loop to handle EXCP_TRACE.

Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220602013401.303699-9-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/m68k/cpu_loop.c |  3 +++
 target/m68k/op_helper.c    |  2 +-
 target/m68k/translate.c    | 49 +++++++++++++++++++++++++-------------
 3 files changed, 36 insertions(+), 18 deletions(-)

diff --git a/linux-user/m68k/cpu_loop.c b/linux-user/m68k/cpu_loop.c
index 6598bce3c4..fcf9220552 100644
--- a/linux-user/m68k/cpu_loop.c
+++ b/linux-user/m68k/cpu_loop.c
@@ -52,6 +52,9 @@ void cpu_loop(CPUM68KState *env)
         case EXCP_DIV0:
             force_sig_fault(TARGET_SIGFPE, TARGET_FPE_INTDIV, env->mmu.ar);
             break;
+        case EXCP_TRACE:
+            force_sig_fault(TARGET_SIGTRAP, TARGET_TRAP_TRACE, env->mmu.ar);
+            break;
         case EXCP_TRAP0:
             {
                 abi_long ret;
diff --git a/target/m68k/op_helper.c b/target/m68k/op_helper.c
index 729ee0e934..3cb71c9140 100644
--- a/target/m68k/op_helper.c
+++ b/target/m68k/op_helper.c
@@ -397,13 +397,13 @@ static void m68k_interrupt_all(CPUM68KState *env, int is_hw)
 
     case EXCP_ILLEGAL:
     case EXCP_TRAPCC:
-    case EXCP_TRACE:
         /* FIXME: addr is not only env->pc */
         do_stack_frame(env, &sp, 2, oldsr, env->pc, env->pc);
         break;
 
     case EXCP_CHK:
     case EXCP_DIV0:
+    case EXCP_TRACE:
         do_stack_frame(env, &sp, 2, oldsr, env->mmu.ar, env->pc);
         break;
 
diff --git a/target/m68k/translate.c b/target/m68k/translate.c
index 6075f49930..38b72d282a 100644
--- a/target/m68k/translate.c
+++ b/target/m68k/translate.c
@@ -114,6 +114,7 @@ typedef struct DisasContext {
     DisasContextBase base;
     CPUM68KState *env;
     target_ulong pc;
+    target_ulong pc_prev;
     CCOp cc_op; /* Current CC operation */
     int cc_op_synced;
     TCGv_i64 mactmp;
@@ -298,6 +299,21 @@ static void gen_raise_exception(int nr)
     tcg_temp_free_i32(tmp);
 }
 
+static void gen_raise_exception_format2(DisasContext *s, int nr,
+                                        target_ulong this_pc)
+{
+    /*
+     * Pass the address of the insn to the exception handler,
+     * for recording in the Format $2 (6-word) stack frame.
+     * Re-use mmu.ar for the purpose, since that's only valid
+     * after tlb_fill.
+     */
+    tcg_gen_st_i32(tcg_constant_i32(this_pc), cpu_env,
+                   offsetof(CPUM68KState, mmu.ar));
+    gen_raise_exception(nr);
+    s->base.is_jmp = DISAS_NORETURN;
+}
+
 static void gen_exception(DisasContext *s, uint32_t dest, int nr)
 {
     update_cc_op(s);
@@ -1494,12 +1510,13 @@ static void gen_exit_tb(DisasContext *s)
     } while (0)
 
 /* Generate a jump to an immediate address.  */
-static void gen_jmp_tb(DisasContext *s, int n, uint32_t dest)
+static void gen_jmp_tb(DisasContext *s, int n, target_ulong dest,
+                       target_ulong src)
 {
     if (unlikely(s->ss_active)) {
         update_cc_op(s);
         tcg_gen_movi_i32(QREG_PC, dest);
-        gen_raise_exception(EXCP_TRACE);
+        gen_raise_exception_format2(s, EXCP_TRACE, src);
     } else if (translator_use_goto_tb(&s->base, dest)) {
         tcg_gen_goto_tb(n);
         tcg_gen_movi_i32(QREG_PC, dest);
@@ -1548,9 +1565,9 @@ DISAS_INSN(dbcc)
     tcg_gen_addi_i32(tmp, tmp, -1);
     gen_partset_reg(OS_WORD, reg, tmp);
     tcg_gen_brcondi_i32(TCG_COND_EQ, tmp, -1, l1);
-    gen_jmp_tb(s, 1, base + offset);
+    gen_jmp_tb(s, 1, base + offset, s->base.pc_next);
     gen_set_label(l1);
-    gen_jmp_tb(s, 0, s->pc);
+    gen_jmp_tb(s, 0, s->pc, s->base.pc_next);
 }
 
 DISAS_INSN(undef_mac)
@@ -3096,13 +3113,13 @@ DISAS_INSN(branch)
         /* Bcc */
         TCGLabel *l1 = gen_new_label();
         gen_jmpcc(s, ((insn >> 8) & 0xf) ^ 1, l1);
-        gen_jmp_tb(s, 1, base + offset);
+        gen_jmp_tb(s, 1, base + offset, s->base.pc_next);
         gen_set_label(l1);
-        gen_jmp_tb(s, 0, s->pc);
+        gen_jmp_tb(s, 0, s->pc, s->base.pc_next);
     } else {
         /* Unconditional branch.  */
         update_cc_op(s);
-        gen_jmp_tb(s, 0, base + offset);
+        gen_jmp_tb(s, 0, base + offset, s->base.pc_next);
     }
 }
 
@@ -5485,9 +5502,9 @@ DISAS_INSN(fbcc)
     l1 = gen_new_label();
     update_cc_op(s);
     gen_fjmpcc(s, insn & 0x3f, l1);
-    gen_jmp_tb(s, 0, s->pc);
+    gen_jmp_tb(s, 0, s->pc, s->base.pc_next);
     gen_set_label(l1);
-    gen_jmp_tb(s, 1, base + offset);
+    gen_jmp_tb(s, 1, base + offset, s->base.pc_next);
 }
 
 DISAS_INSN(fscc)
@@ -6159,6 +6176,8 @@ static void m68k_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cpu)
 
     dc->env = env;
     dc->pc = dc->base.pc_first;
+    /* This value will always be filled in properly before m68k_tr_tb_stop. */
+    dc->pc_prev = 0xdeadbeef;
     dc->cc_op = CC_OP_DYNAMIC;
     dc->cc_op_synced = 1;
     dc->done_mac = 0;
@@ -6192,6 +6211,7 @@ static void m68k_tr_translate_insn(DisasContextBase *dcbase, CPUState *cpu)
     do_writebacks(dc);
     do_release(dc);
 
+    dc->pc_prev = dc->base.pc_next;
     dc->base.pc_next = dc->pc;
 
     if (dc->base.is_jmp == DISAS_NEXT) {
@@ -6226,17 +6246,12 @@ static void m68k_tr_tb_stop(DisasContextBase *dcbase, CPUState *cpu)
         break;
     case DISAS_TOO_MANY:
         update_cc_op(dc);
-        if (dc->ss_active) {
-            tcg_gen_movi_i32(QREG_PC, dc->pc);
-            gen_raise_exception(EXCP_TRACE);
-        } else {
-            gen_jmp_tb(dc, 0, dc->pc);
-        }
+        gen_jmp_tb(dc, 0, dc->pc, dc->pc_prev);
         break;
     case DISAS_JUMP:
         /* We updated CC_OP and PC in gen_jmp/gen_jmp_im.  */
         if (dc->ss_active) {
-            gen_raise_exception(EXCP_TRACE);
+            gen_raise_exception_format2(dc, EXCP_TRACE, dc->pc_prev);
         } else {
             tcg_gen_lookup_and_goto_ptr();
         }
@@ -6247,7 +6262,7 @@ static void m68k_tr_tb_stop(DisasContextBase *dcbase, CPUState *cpu)
          * other state that may require returning to the main loop.
          */
         if (dc->ss_active) {
-            gen_raise_exception(EXCP_TRACE);
+            gen_raise_exception_format2(dc, EXCP_TRACE, dc->pc_prev);
         } else {
             tcg_gen_exit_tb(NULL, 0);
         }

From a1aedd6cbdec67c1d47d961144285f4b95af5fc0 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 18:33:53 -0700
Subject: [PATCH 744/935] target/m68k: Fix stack frame for EXCP_ILLEGAL

According to the M68040 Users Manual, section 8.4.1, Four word
stack frame (format 0), includes Illegal Instruction.  Use the
correct frame format, which does not use the ADDR argument.

Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220602013401.303699-10-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 target/m68k/op_helper.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/target/m68k/op_helper.c b/target/m68k/op_helper.c
index 3cb71c9140..aa62158eb9 100644
--- a/target/m68k/op_helper.c
+++ b/target/m68k/op_helper.c
@@ -391,11 +391,14 @@ static void m68k_interrupt_all(CPUM68KState *env, int is_hw)
         }
         break;
 
+    case EXCP_ILLEGAL:
+        do_stack_frame(env, &sp, 0, oldsr, 0, env->pc);
+        break;
+
     case EXCP_ADDRESS:
         do_stack_frame(env, &sp, 2, oldsr, 0, env->pc);
         break;
 
-    case EXCP_ILLEGAL:
     case EXCP_TRAPCC:
         /* FIXME: addr is not only env->pc */
         do_stack_frame(env, &sp, 2, oldsr, env->pc, env->pc);

From aeeb90afcec3e18254bc6ac9c511f3b0a1a3796c Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 18:33:54 -0700
Subject: [PATCH 745/935] target/m68k: Implement TRAPcc

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/754
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220602013401.303699-11-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/m68k/cpu_loop.c |  1 +
 target/m68k/cpu.c          |  1 +
 target/m68k/cpu.h          |  2 ++
 target/m68k/op_helper.c    |  6 +----
 target/m68k/translate.c    | 49 ++++++++++++++++++++++++++++++++++++++
 5 files changed, 54 insertions(+), 5 deletions(-)

diff --git a/linux-user/m68k/cpu_loop.c b/linux-user/m68k/cpu_loop.c
index fcf9220552..3d3033155f 100644
--- a/linux-user/m68k/cpu_loop.c
+++ b/linux-user/m68k/cpu_loop.c
@@ -47,6 +47,7 @@ void cpu_loop(CPUM68KState *env)
             force_sig_fault(TARGET_SIGILL, TARGET_ILL_ILLOPN, env->pc);
             break;
         case EXCP_CHK:
+        case EXCP_TRAPCC:
             force_sig_fault(TARGET_SIGFPE, TARGET_FPE_INTOVF, env->mmu.ar);
             break;
         case EXCP_DIV0:
diff --git a/target/m68k/cpu.c b/target/m68k/cpu.c
index 5671067923..5bbefda575 100644
--- a/target/m68k/cpu.c
+++ b/target/m68k/cpu.c
@@ -158,6 +158,7 @@ static void m68020_cpu_initfn(Object *obj)
     m68k_set_feature(env, M68K_FEATURE_CHK2);
     m68k_set_feature(env, M68K_FEATURE_MSP);
     m68k_set_feature(env, M68K_FEATURE_UNALIGNED_DATA);
+    m68k_set_feature(env, M68K_FEATURE_TRAPCC);
 }
 
 /*
diff --git a/target/m68k/cpu.h b/target/m68k/cpu.h
index 558c3c67d6..4d8f48e8c7 100644
--- a/target/m68k/cpu.h
+++ b/target/m68k/cpu.h
@@ -534,6 +534,8 @@ enum m68k_features {
     M68K_FEATURE_MOVEC,
     /* Unaligned data accesses (680[2346]0) */
     M68K_FEATURE_UNALIGNED_DATA,
+    /* TRAPcc insn. (680[2346]0, and CPU32) */
+    M68K_FEATURE_TRAPCC,
 };
 
 static inline int m68k_feature(CPUM68KState *env, int feature)
diff --git a/target/m68k/op_helper.c b/target/m68k/op_helper.c
index aa62158eb9..61948d92bb 100644
--- a/target/m68k/op_helper.c
+++ b/target/m68k/op_helper.c
@@ -399,14 +399,10 @@ static void m68k_interrupt_all(CPUM68KState *env, int is_hw)
         do_stack_frame(env, &sp, 2, oldsr, 0, env->pc);
         break;
 
-    case EXCP_TRAPCC:
-        /* FIXME: addr is not only env->pc */
-        do_stack_frame(env, &sp, 2, oldsr, env->pc, env->pc);
-        break;
-
     case EXCP_CHK:
     case EXCP_DIV0:
     case EXCP_TRACE:
+    case EXCP_TRAPCC:
         do_stack_frame(env, &sp, 2, oldsr, env->mmu.ar, env->pc);
         break;
 
diff --git a/target/m68k/translate.c b/target/m68k/translate.c
index 38b72d282a..e9aa96d768 100644
--- a/target/m68k/translate.c
+++ b/target/m68k/translate.c
@@ -4879,6 +4879,53 @@ DISAS_INSN(trap)
     gen_exception(s, s->pc, EXCP_TRAP0 + (insn & 0xf));
 }
 
+static void do_trapcc(DisasContext *s, DisasCompare *c)
+{
+    if (c->tcond != TCG_COND_NEVER) {
+        TCGLabel *over = NULL;
+
+        update_cc_op(s);
+
+        if (c->tcond != TCG_COND_ALWAYS) {
+            /* Jump over if !c. */
+            over = gen_new_label();
+            tcg_gen_brcond_i32(tcg_invert_cond(c->tcond), c->v1, c->v2, over);
+        }
+
+        tcg_gen_movi_i32(QREG_PC, s->pc);
+        gen_raise_exception_format2(s, EXCP_TRAPCC, s->base.pc_next);
+
+        if (over != NULL) {
+            gen_set_label(over);
+            s->base.is_jmp = DISAS_NEXT;
+        }
+    }
+    free_cond(c);
+}
+
+DISAS_INSN(trapcc)
+{
+    DisasCompare c;
+
+    /* Consume and discard the immediate operand. */
+    switch (extract32(insn, 0, 3)) {
+    case 2: /* trapcc.w */
+        (void)read_im16(env, s);
+        break;
+    case 3: /* trapcc.l */
+        (void)read_im32(env, s);
+        break;
+    case 4: /* trapcc (no operand) */
+        break;
+    default:
+        /* trapcc registered with only valid opmodes */
+        g_assert_not_reached();
+    }
+
+    gen_cc_cond(&c, s, extract32(insn, 8, 4));
+    do_trapcc(s, &c);
+}
+
 static void gen_load_fcr(DisasContext *s, TCGv res, int reg)
 {
     switch (reg) {
@@ -6051,6 +6098,8 @@ void register_m68k_insns (CPUM68KState *env)
     INSN(scc,       50c0, f0f8, CF_ISA_A); /* Scc.B Dx   */
     INSN(scc,       50c0, f0c0, M68000);   /* Scc.B <EA> */
     INSN(dbcc,      50c8, f0f8, M68000);
+    INSN(trapcc,    50fa, f0fe, TRAPCC);   /* opmode 010, 011 */
+    INSN(trapcc,    50fc, f0ff, TRAPCC);   /* opmode 100 */
     INSN(tpf,       51f8, fff8, CF_ISA_A);
 
     /* Branch instructions.  */

From 815c6dea464c661032c6cc76f42160a6240c930e Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 18:33:55 -0700
Subject: [PATCH 746/935] target/m68k: Implement TPF in terms of TRAPcc
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

TPF stands for "trap false", and is a long-form nop for ColdFire.
Re-use the immediate consumption code from trapcc; the insn will
already expand to a nop because of the TCG_COND_NEVER test
within do_trapcc.

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220602013401.303699-12-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 target/m68k/translate.c | 19 ++-----------------
 1 file changed, 2 insertions(+), 17 deletions(-)

diff --git a/target/m68k/translate.c b/target/m68k/translate.c
index e9aa96d768..8b2157c31f 100644
--- a/target/m68k/translate.c
+++ b/target/m68k/translate.c
@@ -3075,22 +3075,6 @@ DISAS_INSN(addsubq)
     tcg_temp_free(dest);
 }
 
-DISAS_INSN(tpf)
-{
-    switch (insn & 7) {
-    case 2: /* One extension word.  */
-        s->pc += 2;
-        break;
-    case 3: /* Two extension words.  */
-        s->pc += 4;
-        break;
-    case 4: /* No extension words.  */
-        break;
-    default:
-        disas_undef(env, s, insn);
-    }
-}
-
 DISAS_INSN(branch)
 {
     int32_t offset;
@@ -6100,7 +6084,8 @@ void register_m68k_insns (CPUM68KState *env)
     INSN(dbcc,      50c8, f0f8, M68000);
     INSN(trapcc,    50fa, f0fe, TRAPCC);   /* opmode 010, 011 */
     INSN(trapcc,    50fc, f0ff, TRAPCC);   /* opmode 100 */
-    INSN(tpf,       51f8, fff8, CF_ISA_A);
+    INSN(trapcc,    51fa, fffe, CF_ISA_A); /* TPF (trapf) opmode 010, 011 */
+    INSN(trapcc,    51fc, ffff, CF_ISA_A); /* TPF (trapf) opmode 100 */
 
     /* Branch instructions.  */
     BASE(branch,    6000, f000);

From 43accc4862e0a88710411b205fdaf833dadf9951 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 18:33:56 -0700
Subject: [PATCH 747/935] target/m68k: Implement TRAPV

Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220602013401.303699-13-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 target/m68k/translate.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/target/m68k/translate.c b/target/m68k/translate.c
index 8b2157c31f..0dfddaa056 100644
--- a/target/m68k/translate.c
+++ b/target/m68k/translate.c
@@ -4910,6 +4910,14 @@ DISAS_INSN(trapcc)
     do_trapcc(s, &c);
 }
 
+DISAS_INSN(trapv)
+{
+    DisasCompare c;
+
+    gen_cc_cond(&c, s, 9); /* V set */
+    do_trapcc(s, &c);
+}
+
 static void gen_load_fcr(DisasContext *s, TCGv res, int reg)
 {
     switch (reg) {
@@ -6074,6 +6082,7 @@ void register_m68k_insns (CPUM68KState *env)
     BASE(nop,       4e71, ffff);
     INSN(rtd,       4e74, ffff, RTD);
     BASE(rts,       4e75, ffff);
+    INSN(trapv,     4e76, ffff, M68000);
     INSN(rtr,       4e77, ffff, M68000);
     BASE(jump,      4e80, ffc0);
     BASE(jump,      4ec0, ffc0);

From cc1cc264b14c75c4f3ddd8e33c9dd6f1e497bfdf Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 18:33:57 -0700
Subject: [PATCH 748/935] target/m68k: Implement FTRAPcc

Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220602013401.303699-14-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 target/m68k/translate.c | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/target/m68k/translate.c b/target/m68k/translate.c
index 0dfddaa056..8f3c298ad0 100644
--- a/target/m68k/translate.c
+++ b/target/m68k/translate.c
@@ -5566,6 +5566,34 @@ DISAS_INSN(fscc)
     tcg_temp_free(tmp);
 }
 
+DISAS_INSN(ftrapcc)
+{
+    DisasCompare c;
+    uint16_t ext;
+    int cond;
+
+    ext = read_im16(env, s);
+    cond = ext & 0x3f;
+
+    /* Consume and discard the immediate operand. */
+    switch (extract32(insn, 0, 3)) {
+    case 2: /* ftrapcc.w */
+        (void)read_im16(env, s);
+        break;
+    case 3: /* ftrapcc.l */
+        (void)read_im32(env, s);
+        break;
+    case 4: /* ftrapcc (no operand) */
+        break;
+    default:
+        /* ftrapcc registered with only valid opmodes */
+        g_assert_not_reached();
+    }
+
+    gen_fcc_cond(&c, s, cond);
+    do_trapcc(s, &c);
+}
+
 #if defined(CONFIG_SOFTMMU)
 DISAS_INSN(frestore)
 {
@@ -6192,6 +6220,8 @@ void register_m68k_insns (CPUM68KState *env)
     INSN(fbcc,      f280, ffc0, CF_FPU);
     INSN(fpu,       f200, ffc0, FPU);
     INSN(fscc,      f240, ffc0, FPU);
+    INSN(ftrapcc,   f27a, fffe, FPU);       /* opmode 010, 011 */
+    INSN(ftrapcc,   f27c, ffff, FPU);       /* opmode 100 */
     INSN(fbcc,      f280, ff80, FPU);
 #if defined(CONFIG_SOFTMMU)
     INSN(frestore,  f340, ffc0, CF_FPU);

From e105db022715d0a418dbda843fe19eefeb272380 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 18:33:58 -0700
Subject: [PATCH 749/935] tests/tcg/m68k: Add trap.c

Test various trap instructions: chk, div, trap, trapv, trapcc, ftrapcc,
and the signals and addresses that we expect from them.

Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220602013401.303699-15-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 tests/tcg/m68k/Makefile.target |   3 +
 tests/tcg/m68k/trap.c          | 129 +++++++++++++++++++++++++++++++++
 2 files changed, 132 insertions(+)
 create mode 100644 tests/tcg/m68k/trap.c

diff --git a/tests/tcg/m68k/Makefile.target b/tests/tcg/m68k/Makefile.target
index 62f109eef4..1163c7ef03 100644
--- a/tests/tcg/m68k/Makefile.target
+++ b/tests/tcg/m68k/Makefile.target
@@ -3,5 +3,8 @@
 # m68k specific tweaks - specifically masking out broken tests
 #
 
+VPATH += $(SRC_PATH)/tests/tcg/m68k
+TESTS += trap
+
 # On m68k Linux supports 4k and 8k pages (but 8k is currently broken)
 EXTRA_RUNS+=run-test-mmap-4096 # run-test-mmap-8192
diff --git a/tests/tcg/m68k/trap.c b/tests/tcg/m68k/trap.c
new file mode 100644
index 0000000000..96cac18d4d
--- /dev/null
+++ b/tests/tcg/m68k/trap.c
@@ -0,0 +1,129 @@
+/*
+ * Test m68k trap addresses.
+ */
+
+#define _GNU_SOURCE 1
+#include <signal.h>
+#include <assert.h>
+#include <limits.h>
+
+static int expect_sig;
+static int expect_si_code;
+static void *expect_si_addr;
+static greg_t expect_mc_pc;
+static volatile int got_signal;
+
+static void sig_handler(int sig, siginfo_t *si, void *puc)
+{
+    ucontext_t *uc = puc;
+    mcontext_t *mc = &uc->uc_mcontext;
+
+    assert(sig == expect_sig);
+    assert(si->si_code == expect_si_code);
+    assert(si->si_addr == expect_si_addr);
+    assert(mc->gregs[R_PC] == expect_mc_pc);
+
+    got_signal = 1;
+}
+
+#define FMT_INS     [ad] "a"(&expect_si_addr), [pc] "a"(&expect_mc_pc)
+#define FMT0_STR(S) \
+    "move.l #1f, (%[ad])\n\tmove.l #1f, (%[pc])\n" S "\n1:\n"
+#define FMT2_STR(S) \
+    "move.l #0f, (%[ad])\n\tmove.l #1f, (%[pc])\n" S "\n1:\n"
+
+#define CHECK_SIG   do { assert(got_signal); got_signal = 0; } while (0)
+
+int main(int argc, char **argv)
+{
+    struct sigaction act = {
+        .sa_sigaction = sig_handler,
+        .sa_flags = SA_SIGINFO
+    };
+    int t0, t1;
+
+    sigaction(SIGILL, &act, NULL);
+    sigaction(SIGTRAP, &act, NULL);
+    sigaction(SIGFPE, &act, NULL);
+
+    expect_sig = SIGFPE;
+    expect_si_code = FPE_INTOVF;
+    asm volatile(FMT2_STR("0:\tchk %0, %1") : : "d"(0), "d"(-1), FMT_INS);
+    CHECK_SIG;
+
+#if 0
+    /* FIXME: chk2 not correctly translated. */
+    int bounds[2] = { 0, 1 };
+    asm volatile(FMT2_STR("0:\tchk2.l %0, %1")
+                 : : "m"(bounds), "d"(2), FMT_INS);
+    CHECK_SIG;
+#endif
+
+    asm volatile(FMT2_STR("cmp.l %0, %1\n0:\ttrapv")
+                 : : "d"(INT_MIN), "d"(1), FMT_INS);
+    CHECK_SIG;
+
+    asm volatile(FMT2_STR("cmp.l %0, %0\n0:\ttrapeq")
+                 : : "d"(0), FMT_INS);
+    CHECK_SIG;
+
+    asm volatile(FMT2_STR("cmp.l %0, %0\n0:\ttrapeq.w #0x1234")
+                 : : "d"(0), FMT_INS);
+    CHECK_SIG;
+
+    asm volatile(FMT2_STR("cmp.l %0, %0\n0:\ttrapeq.l #0x12345678")
+                 : : "d"(0), FMT_INS);
+    CHECK_SIG;
+
+    asm volatile(FMT2_STR("fcmp.x %0, %0\n0:\tftrapeq")
+                 : : "f"(0.0L), FMT_INS);
+    CHECK_SIG;
+
+    expect_si_code = FPE_INTDIV;
+
+    asm volatile(FMT2_STR("0:\tdivs.w %1, %0")
+                 : "=d"(t0) : "d"(0), "0"(1), FMT_INS);
+    CHECK_SIG;
+
+    asm volatile(FMT2_STR("0:\tdivsl.l %2, %1:%0")
+                 : "=d"(t0), "=d"(t1) : "d"(0), "0"(1), FMT_INS);
+    CHECK_SIG;
+
+    expect_sig = SIGILL;
+    expect_si_code = ILL_ILLTRP;
+    asm volatile(FMT0_STR("trap #1") : : FMT_INS);
+    CHECK_SIG;
+    asm volatile(FMT0_STR("trap #2") : : FMT_INS);
+    CHECK_SIG;
+    asm volatile(FMT0_STR("trap #3") : : FMT_INS);
+    CHECK_SIG;
+    asm volatile(FMT0_STR("trap #4") : : FMT_INS);
+    CHECK_SIG;
+    asm volatile(FMT0_STR("trap #5") : : FMT_INS);
+    CHECK_SIG;
+    asm volatile(FMT0_STR("trap #6") : : FMT_INS);
+    CHECK_SIG;
+    asm volatile(FMT0_STR("trap #7") : : FMT_INS);
+    CHECK_SIG;
+    asm volatile(FMT0_STR("trap #8") : : FMT_INS);
+    CHECK_SIG;
+    asm volatile(FMT0_STR("trap #9") : : FMT_INS);
+    CHECK_SIG;
+    asm volatile(FMT0_STR("trap #10") : : FMT_INS);
+    CHECK_SIG;
+    asm volatile(FMT0_STR("trap #11") : : FMT_INS);
+    CHECK_SIG;
+    asm volatile(FMT0_STR("trap #12") : : FMT_INS);
+    CHECK_SIG;
+    asm volatile(FMT0_STR("trap #13") : : FMT_INS);
+    CHECK_SIG;
+    asm volatile(FMT0_STR("trap #14") : : FMT_INS);
+    CHECK_SIG;
+
+    expect_sig = SIGTRAP;
+    expect_si_code = TRAP_BRKPT;
+    asm volatile(FMT0_STR("trap #15") : : FMT_INS);
+    CHECK_SIG;
+
+    return 0;
+}

From 7c75571c07def2ded9998a06ad171380a19f6063 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 18:33:59 -0700
Subject: [PATCH 750/935] linux-user/strace: Use is_error in print_syscall_err
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Errors are not all negative numbers: use is_error.

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220602013401.303699-16-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/strace.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/linux-user/strace.c b/linux-user/strace.c
index 9fa681dea9..7d882526da 100644
--- a/linux-user/strace.c
+++ b/linux-user/strace.c
@@ -689,7 +689,7 @@ print_syscall_err(abi_long ret)
     const char *errstr;
 
     qemu_log(" = ");
-    if (ret < 0) {
+    if (is_error(ret)) {
         errstr = target_strerror(-ret);
         if (errstr) {
             qemu_log("-1 errno=%d (%s)", (int)-ret, errstr);

From dc3e83d5b17ee896be0478d0231a6ed7f966e0e0 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 18:34:00 -0700
Subject: [PATCH 751/935] linux-user/strace: Adjust get_thread_area for m68k

Unlike i386, m68k get_thread_area has no arguments.

Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220602013401.303699-17-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/strace.list | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/linux-user/strace.list b/linux-user/strace.list
index 278596acd1..72e17b1acf 100644
--- a/linux-user/strace.list
+++ b/linux-user/strace.list
@@ -384,8 +384,13 @@
 { TARGET_NR_getsockopt, "getsockopt" , NULL, NULL, NULL },
 #endif
 #ifdef TARGET_NR_get_thread_area
+#if defined(TARGET_I386) && defined(TARGET_ABI32)
 { TARGET_NR_get_thread_area, "get_thread_area", "%s(0x"TARGET_ABI_FMT_lx")",
   NULL, NULL },
+#elif defined(TARGET_M68K)
+{ TARGET_NR_get_thread_area, "get_thread_area" , "%s()",
+  NULL, print_syscall_ret_addr },
+#endif
 #endif
 #ifdef TARGET_NR_gettid
 { TARGET_NR_gettid, "gettid" , "%s()", NULL, NULL },

From 36a0ab595f4e24b987e67faa52d4b174f67144b6 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 18:34:01 -0700
Subject: [PATCH 752/935] target/m68k: Mark helper_raise_exception as noreturn
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Also mark raise_exception_ra and raise_exception, lest we
generate a warning about helper_raise_exception returning.

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220602013401.303699-18-richard.henderson@linaro.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 target/m68k/helper.h    | 2 +-
 target/m68k/op_helper.c | 5 +++--
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/target/m68k/helper.h b/target/m68k/helper.h
index f016c4c1c2..c9bed2b884 100644
--- a/target/m68k/helper.h
+++ b/target/m68k/helper.h
@@ -109,7 +109,7 @@ DEF_HELPER_3(set_mac_extu, void, env, i32, i32)
 DEF_HELPER_2(flush_flags, void, env, i32)
 DEF_HELPER_2(set_ccr, void, env, i32)
 DEF_HELPER_FLAGS_1(get_ccr, TCG_CALL_NO_WG_SE, i32, env)
-DEF_HELPER_2(raise_exception, void, env, i32)
+DEF_HELPER_2(raise_exception, noreturn, env, i32)
 
 DEF_HELPER_FLAGS_3(bfffo_reg, TCG_CALL_NO_RWG_SE, i32, i32, i32, i32)
 
diff --git a/target/m68k/op_helper.c b/target/m68k/op_helper.c
index 61948d92bb..d9937ca8dc 100644
--- a/target/m68k/op_helper.c
+++ b/target/m68k/op_helper.c
@@ -532,7 +532,8 @@ bool m68k_cpu_exec_interrupt(CPUState *cs, int interrupt_request)
 
 #endif /* !CONFIG_USER_ONLY */
 
-static void raise_exception_ra(CPUM68KState *env, int tt, uintptr_t raddr)
+G_NORETURN static void
+raise_exception_ra(CPUM68KState *env, int tt, uintptr_t raddr)
 {
     CPUState *cs = env_cpu(env);
 
@@ -540,7 +541,7 @@ static void raise_exception_ra(CPUM68KState *env, int tt, uintptr_t raddr)
     cpu_loop_exit_restore(cs, raddr);
 }
 
-static void raise_exception(CPUM68KState *env, int tt)
+G_NORETURN static void raise_exception(CPUM68KState *env, int tt)
 {
     raise_exception_ra(env, tt, 0);
 }

From dc24c99116e926b0c56be63acbc086d13e32032b Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Mon, 16 May 2022 16:10:51 -0700
Subject: [PATCH 753/935] tcg: Add tcg_gen_mov_ptr

Add an interface to perform moves between TCGv_ptr.

Reviewed-by: Matheus Ferst <matheus.ferst@eldorado.org.br>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/tcg/tcg-op.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/include/tcg/tcg-op.h b/include/tcg/tcg-op.h
index b09b8b4a05..209e168305 100644
--- a/include/tcg/tcg-op.h
+++ b/include/tcg/tcg-op.h
@@ -1288,6 +1288,11 @@ static inline void tcg_gen_addi_ptr(TCGv_ptr r, TCGv_ptr a, intptr_t b)
     glue(tcg_gen_addi_,PTR)((NAT)r, (NAT)a, b);
 }
 
+static inline void tcg_gen_mov_ptr(TCGv_ptr d, TCGv_ptr s)
+{
+    glue(tcg_gen_mov_,PTR)((NAT)d, (NAT)s);
+}
+
 static inline void tcg_gen_brcondi_ptr(TCGCond cond, TCGv_ptr a,
                                        intptr_t b, TCGLabel *label)
 {

From 3cc18d18cc3865d7b1ce2c8b35d52e52abbff397 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 1 Jun 2022 16:22:19 -0700
Subject: [PATCH 754/935] tcg/i386: Fix encoding of OPC_VPSRAQ for
 INDEX_op_sars_vec

We wanted the VPSRAQ variant with the scalar vector shift operand,
not the variant with an immediate operand.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1022
Fixes: 47b331b2a8da ("tcg/i386: Implement avx512 scalar shift")
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/i386/tcg-target.c.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc
index b5c6159853..d52206ba4d 100644
--- a/tcg/i386/tcg-target.c.inc
+++ b/tcg/i386/tcg-target.c.inc
@@ -375,7 +375,7 @@ static bool tcg_target_const_match(int64_t val, TCGType type, int ct)
 #define OPC_PSLLQ       (0xf3 | P_EXT | P_DATA16)
 #define OPC_PSRAW       (0xe1 | P_EXT | P_DATA16)
 #define OPC_PSRAD       (0xe2 | P_EXT | P_DATA16)
-#define OPC_VPSRAQ      (0x72 | P_EXT | P_DATA16 | P_VEXW | P_EVEX)
+#define OPC_VPSRAQ      (0xe2 | P_EXT | P_DATA16 | P_VEXW | P_EVEX)
 #define OPC_PSRLW       (0xd1 | P_EXT | P_DATA16)
 #define OPC_PSRLD       (0xd2 | P_EXT | P_DATA16)
 #define OPC_PSRLQ       (0xd3 | P_EXT | P_DATA16)

From 94bcc91b2e95e02ec57ed18d5a5e7cb75aa19a50 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Thu, 2 Jun 2022 01:04:30 +0000
Subject: [PATCH 755/935] tcg/aarch64: Fix illegal insn from out-of-range shli

The masking in tcg_out_shl was incorrect, producing an
illegal instruction, rather than merely unspecified results
for the out-of-range shift.

Tested-by: Joel Stanley <joel@jms.id.au>
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1051
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/aarch64/tcg-target.c.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc
index 61e284bb5c..d997f7922a 100644
--- a/tcg/aarch64/tcg-target.c.inc
+++ b/tcg/aarch64/tcg-target.c.inc
@@ -1261,7 +1261,7 @@ static inline void tcg_out_shl(TCGContext *s, TCGType ext,
 {
     int bits = ext ? 64 : 32;
     int max = bits - 1;
-    tcg_out_ubfm(s, ext, rd, rn, bits - (m & max), max - (m & max));
+    tcg_out_ubfm(s, ext, rd, rn, (bits - m) & max, (max - m) & max);
 }
 
 static inline void tcg_out_shr(TCGContext *s, TCGType ext,

From 000117066bbea59a9a37ebf9bf7347a029e2f02c Mon Sep 17 00:00:00 2001
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Date: Mon, 23 May 2022 12:51:23 +0100
Subject: [PATCH 756/935] s390: Typo fix FLOATING_POINT_SUPPPORT_ENH

One less P needed.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20220523115123.150340-1-dgilbert@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 target/s390x/cpu_features_def.h.inc | 2 +-
 target/s390x/gen-features.c         | 6 +++---
 target/s390x/tcg/translate.c        | 8 ++++----
 3 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/target/s390x/cpu_features_def.h.inc b/target/s390x/cpu_features_def.h.inc
index e86662bb3b..3603e5fb12 100644
--- a/target/s390x/cpu_features_def.h.inc
+++ b/target/s390x/cpu_features_def.h.inc
@@ -58,7 +58,7 @@ DEF_FEAT(ENHANCED_MONITOR, "emon", STFL, 36, "Enhanced-monitor facility")
 DEF_FEAT(FLOATING_POINT_EXT, "fpe", STFL, 37, "Floating-point extension facility")
 DEF_FEAT(ORDER_PRESERVING_COMPRESSION, "opc", STFL, 38, "Order Preserving Compression facility")
 DEF_FEAT(SET_PROGRAM_PARAMETERS, "sprogp", STFL, 40, "Set-program-parameters facility")
-DEF_FEAT(FLOATING_POINT_SUPPPORT_ENH, "fpseh", STFL, 41, "Floating-point-support-enhancement facilities")
+DEF_FEAT(FLOATING_POINT_SUPPORT_ENH, "fpseh", STFL, 41, "Floating-point-support-enhancement facilities")
 DEF_FEAT(DFP, "dfp", STFL, 42, "DFP (decimal-floating-point) facility")
 DEF_FEAT(DFP_FAST, "dfphp", STFL, 43, "DFP (decimal-floating-point) facility has high performance")
 DEF_FEAT(PFPO, "pfpo", STFL, 44, "PFPO instruction")
diff --git a/target/s390x/gen-features.c b/target/s390x/gen-features.c
index c03ec2c9a9..ad140184b9 100644
--- a/target/s390x/gen-features.c
+++ b/target/s390x/gen-features.c
@@ -374,7 +374,7 @@ static uint16_t base_GEN10_GA1[] = {
     S390_FEAT_COMPARE_AND_SWAP_AND_STORE_2,
     S390_FEAT_GENERAL_INSTRUCTIONS_EXT,
     S390_FEAT_EXECUTE_EXT,
-    S390_FEAT_FLOATING_POINT_SUPPPORT_ENH,
+    S390_FEAT_FLOATING_POINT_SUPPORT_ENH,
     S390_FEAT_DFP,
     S390_FEAT_DFP_FAST,
     S390_FEAT_PFPO,
@@ -476,7 +476,7 @@ static uint16_t full_GEN9_GA2[] = {
     S390_FEAT_MOVE_WITH_OPTIONAL_SPEC,
     S390_FEAT_EXTRACT_CPU_TIME,
     S390_FEAT_COMPARE_AND_SWAP_AND_STORE,
-    S390_FEAT_FLOATING_POINT_SUPPPORT_ENH,
+    S390_FEAT_FLOATING_POINT_SUPPORT_ENH,
     S390_FEAT_DFP,
 };
 
@@ -700,7 +700,7 @@ static uint16_t qemu_V3_1[] = {
     S390_FEAT_GENERAL_INSTRUCTIONS_EXT,
     S390_FEAT_EXECUTE_EXT,
     S390_FEAT_SET_PROGRAM_PARAMETERS,
-    S390_FEAT_FLOATING_POINT_SUPPPORT_ENH,
+    S390_FEAT_FLOATING_POINT_SUPPORT_ENH,
     S390_FEAT_STFLE_45,
     S390_FEAT_STFLE_49,
     S390_FEAT_LOCAL_TLB_CLEARING,
diff --git a/target/s390x/tcg/translate.c b/target/s390x/tcg/translate.c
index b40cb84bae..fd2433d625 100644
--- a/target/s390x/tcg/translate.c
+++ b/target/s390x/tcg/translate.c
@@ -6185,17 +6185,17 @@ enum DisasInsnEnum {
 #define FAC_Z           S390_FEAT_ZARCH
 #define FAC_CASS        S390_FEAT_COMPARE_AND_SWAP_AND_STORE
 #define FAC_DFP         S390_FEAT_DFP
-#define FAC_DFPR        S390_FEAT_FLOATING_POINT_SUPPPORT_ENH /* DFP-rounding */
+#define FAC_DFPR        S390_FEAT_FLOATING_POINT_SUPPORT_ENH /* DFP-rounding */
 #define FAC_DO          S390_FEAT_STFLE_45 /* distinct-operands */
 #define FAC_EE          S390_FEAT_EXECUTE_EXT
 #define FAC_EI          S390_FEAT_EXTENDED_IMMEDIATE
 #define FAC_FPE         S390_FEAT_FLOATING_POINT_EXT
-#define FAC_FPSSH       S390_FEAT_FLOATING_POINT_SUPPPORT_ENH /* FPS-sign-handling */
-#define FAC_FPRGR       S390_FEAT_FLOATING_POINT_SUPPPORT_ENH /* FPR-GR-transfer */
+#define FAC_FPSSH       S390_FEAT_FLOATING_POINT_SUPPORT_ENH /* FPS-sign-handling */
+#define FAC_FPRGR       S390_FEAT_FLOATING_POINT_SUPPORT_ENH /* FPR-GR-transfer */
 #define FAC_GIE         S390_FEAT_GENERAL_INSTRUCTIONS_EXT
 #define FAC_HFP_MA      S390_FEAT_HFP_MADDSUB
 #define FAC_HW          S390_FEAT_STFLE_45 /* high-word */
-#define FAC_IEEEE_SIM   S390_FEAT_FLOATING_POINT_SUPPPORT_ENH /* IEEE-exception-simulation */
+#define FAC_IEEEE_SIM   S390_FEAT_FLOATING_POINT_SUPPORT_ENH /* IEEE-exception-simulation */
 #define FAC_MIE         S390_FEAT_STFLE_49 /* misc-instruction-extensions */
 #define FAC_LAT         S390_FEAT_STFLE_49 /* load-and-trap */
 #define FAC_LOC         S390_FEAT_STFLE_45 /* load/store on condition 1 */

From ec8c293678ffe7b3c01678103608ef75cecb909c Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Fri, 6 May 2022 08:50:26 +0200
Subject: [PATCH 757/935] hw/s390x/s390-virtio-ccw: Improve the machine
 description string

The machine name already contains the words "ccw" and "virtio", so
using "VirtIO-ccw" in the description likely does not really help
the average user to get an idea what this machine type is about.
Thus let's switch to "Virtual s390x machine" now, since "virtual
machine" should be a familiar term, and "s390x" signals that this
is about 64-bit guests (unlike S390 which could mean that it is
31-bit only).
Also expand "v" to "version", since this makes it easier to use
this macro also with non-numeric machine names in downstream.

Message-Id: <20220506065026.513590-1-thuth@redhat.com>
Acked-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 hw/s390x/s390-virtio-ccw.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 047cca0487..cc3097bfee 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -768,7 +768,7 @@ bool css_migration_enabled(void)
     {                                                                         \
         MachineClass *mc = MACHINE_CLASS(oc);                                 \
         ccw_machine_##suffix##_class_options(mc);                             \
-        mc->desc = "VirtIO-ccw based S390 machine v" verstr;                  \
+        mc->desc = "Virtual s390x machine (version " verstr ")";              \
         if (latest) {                                                         \
             mc->alias = "s390-ccw-virtio";                                    \
             mc->is_default = true;                                            \

From 54354861d21b69ec0781f43e67b8d4f6edad7e3f Mon Sep 17 00:00:00 2001
From: Janis Schoetterl-Glausch <scgl@linux.ibm.com>
Date: Fri, 6 May 2022 17:39:56 +0200
Subject: [PATCH 758/935] target/s390x: kvm: Honor storage keys during
 emulation

Storage key controlled protection is currently not honored when
emulating instructions.
If available, enable key protection for the MEM_OP ioctl, thereby
enabling it for the s390_cpu_virt_mem_* functions, when using kvm.
As a result, the emulation of the following instructions honors storage
keys:

* CLP
  	The Synch I/O CLP command would need special handling in order
  	to support storage keys, but is currently not supported.
* CHSC
	Performing commands asynchronously would require special
	handling, but commands are currently always synchronous.
* STSI
* TSCH
	Must (and does) not change channel if terminated due to
	protection.
* MSCH
	Suppressed on protection, works because fetching instruction.
* SSCH
	Suppressed on protection, works because fetching instruction.
* STSCH
* STCRW
	Suppressed on protection, this works because no partial store is
	possible, because the operand cannot span multiple pages.
* PCISTB
* MPCIFC
* STPCIFC

Signed-off-by: Janis Schoetterl-Glausch <scgl@linux.ibm.com>
Message-Id: <20220506153956.2217601-3-scgl@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 target/s390x/kvm/kvm.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 53098bf541..7bd8db0e7b 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -151,12 +151,15 @@ const KVMCapabilityInfo kvm_arch_required_capabilities[] = {
 static int cap_sync_regs;
 static int cap_async_pf;
 static int cap_mem_op;
+static int cap_mem_op_extension;
 static int cap_s390_irq;
 static int cap_ri;
 static int cap_hpage_1m;
 static int cap_vcpu_resets;
 static int cap_protected;
 
+static bool mem_op_storage_key_support;
+
 static int active_cmma;
 
 static int kvm_s390_query_mem_limit(uint64_t *memory_limit)
@@ -354,6 +357,8 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     cap_sync_regs = kvm_check_extension(s, KVM_CAP_SYNC_REGS);
     cap_async_pf = kvm_check_extension(s, KVM_CAP_ASYNC_PF);
     cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP);
+    cap_mem_op_extension = kvm_check_extension(s, KVM_CAP_S390_MEM_OP_EXTENSION);
+    mem_op_storage_key_support = cap_mem_op_extension > 0;
     cap_s390_irq = kvm_check_extension(s, KVM_CAP_S390_INJECT_IRQ);
     cap_vcpu_resets = kvm_check_extension(s, KVM_CAP_S390_VCPU_RESETS);
     cap_protected = kvm_check_extension(s, KVM_CAP_S390_PROTECTED);
@@ -842,6 +847,7 @@ int kvm_s390_mem_op(S390CPU *cpu, vaddr addr, uint8_t ar, void *hostbuf,
                        : KVM_S390_MEMOP_LOGICAL_READ,
         .buf = (uint64_t)hostbuf,
         .ar = ar,
+        .key = (cpu->env.psw.mask & PSW_MASK_KEY) >> PSW_SHIFT_KEY,
     };
     int ret;
 
@@ -851,6 +857,9 @@ int kvm_s390_mem_op(S390CPU *cpu, vaddr addr, uint8_t ar, void *hostbuf,
     if (!hostbuf) {
         mem_op.flags |= KVM_S390_MEMOP_F_CHECK_ONLY;
     }
+    if (mem_op_storage_key_support) {
+        mem_op.flags |= KVM_S390_MEMOP_F_SKEY_PROTECTION;
+    }
 
     ret = kvm_vcpu_ioctl(CPU(cpu), KVM_S390_MEM_OP, &mem_op);
     if (ret < 0) {

From 117d794396cea7d9c661c95e716eb618406d31aa Mon Sep 17 00:00:00 2001
From: Eric Farman <farman@linux.ibm.com>
Date: Wed, 25 May 2022 16:58:14 +0200
Subject: [PATCH 759/935] MAINTAINERS: Update s390 vhost entries

Commit 7a523d96a0 ("virtio-ccw: move vhost_ccw_scsi to a separate file")
introduced a new file hw/s390x/vhost-scsi-ccw.c, which received a
couple comments [1][2] to update MAINTAINERS that were missed.

Fix that by making the vhost CCW entries a wildcard.

[1] https://lore.kernel.org/r/d8d2bbd5021076bdba444d31a6da74f507baede3.camel@linux.ibm.com/
[2] https://lore.kernel.org/r/87k0c4gb9f.fsf@redhat.com/

Signed-off-by: Eric Farman <farman@linux.ibm.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Message-Id: <20220525145814.2750501-1-farman@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 MAINTAINERS | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 00dc4a8ecb..7b7d3f9c02 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2018,8 +2018,7 @@ M: Halil Pasic <pasic@linux.ibm.com>
 M: Eric Farman <farman@linux.ibm.com>
 S: Supported
 F: hw/s390x/virtio-ccw*.[hc]
-F: hw/s390x/vhost-vsock-ccw.c
-F: hw/s390x/vhost-user-fs-ccw.c
+F: hw/s390x/vhost-*-ccw.c
 T: git https://gitlab.com/cohuck/qemu.git s390-next
 T: git https://github.com/borntraeger/qemu.git s390-next
 L: qemu-s390x@nongnu.org

From 69d0535db941d1095d6ee2dba019c4c8cf8e2fa1 Mon Sep 17 00:00:00 2001
From: Gautam Agrawal <gautamnagrawal@gmail.com>
Date: Wed, 1 Jun 2022 00:05:24 +0530
Subject: [PATCH 760/935] tests/tcg: Test overflow conditions

Add a test to check for overflow conditions in s390x.
This patch is based on the following patches :
* https://git.qemu.org/?p=qemu.git;a=commitdiff;h=5a2e67a691501
* https://git.qemu.org/?p=qemu.git;a=commitdiff;h=fc6e0d0f2db51

Signed-off-by: Gautam Agrawal <gautamnagrawal@gmail.com>
Message-Id: <20220531183524.40948-1-gautamnagrawal@gmail.com>
[thuth: Move overflow.c to tests/tcg/multiarch/ to make it generic]
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/tcg/multiarch/overflow.c | 58 ++++++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)
 create mode 100644 tests/tcg/multiarch/overflow.c

diff --git a/tests/tcg/multiarch/overflow.c b/tests/tcg/multiarch/overflow.c
new file mode 100644
index 0000000000..1c59c2cb70
--- /dev/null
+++ b/tests/tcg/multiarch/overflow.c
@@ -0,0 +1,58 @@
+#include <stdio.h>
+
+int overflow_add_32(int x, int y)
+{
+    int res;
+    return __builtin_add_overflow(x, y, &res);
+}
+
+int overflow_add_64(long long x, long long y)
+{
+    long long res;
+    return __builtin_add_overflow(x, y, &res);
+}
+
+int overflow_sub_32(int x, int y)
+{
+    int res;
+    return __builtin_sub_overflow(x, y, &res);
+}
+
+int overflow_sub_64(long long x, long long y)
+{
+    long long res;
+    return __builtin_sub_overflow(x, y, &res);
+}
+
+int a1_add = -2147483648;
+int b1_add = -2147483648;
+long long a2_add = -9223372036854775808ULL;
+long long b2_add = -9223372036854775808ULL;
+
+int a1_sub;
+int b1_sub = -2147483648;
+long long a2_sub = 0L;
+long long b2_sub = -9223372036854775808ULL;
+
+int main()
+{
+    int ret = 0;
+
+    if (!overflow_add_32(a1_add, b1_add)) {
+        fprintf(stderr, "data overflow while adding 32 bits\n");
+        ret = 1;
+    }
+    if (!overflow_add_64(a2_add, b2_add)) {
+        fprintf(stderr, "data overflow while adding 64 bits\n");
+        ret = 1;
+    }
+    if (!overflow_sub_32(a1_sub, b1_sub)) {
+        fprintf(stderr, "data overflow while subtracting 32 bits\n");
+        ret = 1;
+    }
+    if (!overflow_sub_64(a2_sub, b2_sub)) {
+        fprintf(stderr, "data overflow while subtracting 64 bits\n");
+        ret = 1;
+    }
+    return ret;
+}

From 2ac7d741170bec72bc0a1fd333927f8b98cdc4ca Mon Sep 17 00:00:00 2001
From: Hailiang Zhang <zhanghailiang@xfusion.com>
Date: Tue, 14 Dec 2021 15:54:24 +0800
Subject: [PATCH 761/935] MAINTAINERS: Change my email address

The zhang.zhanghailiang@huawei.com email address has been
stopped. Change it to my new email address.

Signed-off-by: Hailiang Zhang <zhanghailiang@xfusion.com>
Message-Id: <20211214075424.6920-1-zhanghailiang@xfusion.com>
Acked-by: Gonglei <arei.gonglei@huawei.com>
Acked-by: Zhang Chen <chen.zhang@intel.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 MAINTAINERS | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 7b7d3f9c02..ee9693dc3a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3102,7 +3102,7 @@ F: include/qemu/yank.h
 F: qapi/yank.json
 
 COLO Framework
-M: zhanghailiang <zhang.zhanghailiang@huawei.com>
+M: Hailiang Zhang <zhanghailiang@xfusion.com>
 S: Maintained
 F: migration/colo*
 F: include/migration/colo.h

From c5f1bb4272a6c2d10dd92fd2285ed5adb988088e Mon Sep 17 00:00:00 2001
From: Wenchao Wang <wenchao.wang@intel.com>
Date: Tue, 15 Mar 2022 20:51:22 +0800
Subject: [PATCH 762/935] MAINTAINERS: Update maintainers for Guest x86 HAXM
 CPUs

Clean up the maintainer list.

Reviewed-by: Hang Yuan <hang.yuan@intel.com>
Signed-off-by: Wenchao Wang <wenchao.wang@intel.com>
Message-Id: <DM6PR11MB4090A58ACCA4AD8C752AEEA587199@DM6PR11MB4090.namprd11.prod.outlook.com>
[thuth: Note: Colin Xu's address bounces]
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 MAINTAINERS | 1 -
 1 file changed, 1 deletion(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index ee9693dc3a..5fe8f7eca2 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -488,7 +488,6 @@ Guest CPU Cores (HAXM)
 ---------------------
 X86 HAXM CPUs
 M: Wenchao Wang <wenchao.wang@intel.com>
-M: Colin Xu <colin.xu@intel.com>
 L: haxm-team@intel.com
 W: https://github.com/intel/haxm/issues
 S: Maintained

From d412597ec5a8406b2af6aa5fb7740e77c1bd3f8c Mon Sep 17 00:00:00 2001
From: Miaoqian Lin <linmq006@gmail.com>
Date: Tue, 31 May 2022 12:09:21 +0400
Subject: [PATCH 763/935] qtest/npcm7xx_pwm-test: Fix memory leak in
 mft_qom_set

g_strdup_printf() allocated memory for path, we should free it with
g_free() when no longer needed.

Signed-off-by: Miaoqian Lin <linmq006@gmail.com>
Reviewed-by: Hao Wu <wuhaotsh@google.com>
Message-Id: <20220531080921.4704-1-linmq006@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qtest/npcm7xx_pwm-test.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tests/qtest/npcm7xx_pwm-test.c b/tests/qtest/npcm7xx_pwm-test.c
index c4a5fdcacd..e320a625c4 100644
--- a/tests/qtest/npcm7xx_pwm-test.c
+++ b/tests/qtest/npcm7xx_pwm-test.c
@@ -268,6 +268,9 @@ static void mft_qom_set(QTestState *qts, int index, const char *name,
             path, name, value);
     /* The qom set message returns successfully. */
     g_assert_true(qdict_haskey(response, "return"));
+
+    qobject_unref(response);
+    g_free(path);
 }
 
 static uint32_t get_pll(uint32_t con)

From 55c269829de131d611103a7e643d48fe3bd08299 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <alex.bennee@linaro.org>
Date: Tue, 24 May 2022 16:40:52 +0100
Subject: [PATCH 764/935] tests/qtest: use g_autofree for
 test_server_create_chr
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220524154056.2896913-12-alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qtest/vhost-user-test.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/tests/qtest/vhost-user-test.c b/tests/qtest/vhost-user-test.c
index a2cec87684..8bf390be20 100644
--- a/tests/qtest/vhost-user-test.c
+++ b/tests/qtest/vhost-user-test.c
@@ -524,14 +524,13 @@ static void chr_event(void *opaque, QEMUChrEvent event)
 
 static void test_server_create_chr(TestServer *server, const gchar *opt)
 {
-    gchar *chr_path;
+    g_autofree gchar *chr_path = g_strdup_printf("unix:%s%s",
+                                                 server->socket_path, opt);
     Chardev *chr;
 
-    chr_path = g_strdup_printf("unix:%s%s", server->socket_path, opt);
     chr = qemu_chr_new(server->chr_name, chr_path, server->context);
-    g_free(chr_path);
+    g_assert(chr);
 
-    g_assert_nonnull(chr);
     qemu_chr_fe_init(&server->chr, chr, &error_abort);
     qemu_chr_fe_set_handlers(&server->chr, chr_can_read, chr_read,
                              chr_event, NULL, server, server->context, true);

From a743d60bcaa6a3a33f4376f87c1fc7cc977bc811 Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Thu, 19 May 2022 17:56:23 +0200
Subject: [PATCH 765/935] ui: Remove deprecated parameters of the "-display
 sdl" option
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Dropping these deprecated parameters simplifies further refactoring
(e.g. QAPIfication is easier without underscores in the name).

Message-Id: <20220519155625.1414365-2-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 docs/about/deprecated.rst       | 16 -------------
 docs/about/removed-features.rst | 17 ++++++++++++++
 qemu-options.hx                 | 32 ++-----------------------
 softmmu/vl.c                    | 41 +--------------------------------
 4 files changed, 20 insertions(+), 86 deletions(-)

diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst
index a92ae0f162..562a133f18 100644
--- a/docs/about/deprecated.rst
+++ b/docs/about/deprecated.rst
@@ -81,22 +81,6 @@ the process listing. This is replaced by the new ``password-secret``
 option which lets the password be securely provided on the command
 line using a ``secret`` object instance.
 
-``-display sdl,window_close=...`` (since 6.1)
-'''''''''''''''''''''''''''''''''''''''''''''
-
-Use ``-display sdl,window-close=...`` instead (i.e. with a minus instead of
-an underscore between "window" and "close").
-
-``-alt-grab`` and ``-display sdl,alt_grab=on`` (since 6.2)
-''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
-
-Use ``-display sdl,grab-mod=lshift-lctrl-lalt`` instead.
-
-``-ctrl-grab`` and ``-display sdl,ctrl_grab=on`` (since 6.2)
-''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
-
-Use ``-display sdl,grab-mod=rctrl`` instead.
-
 ``-sdl`` (since 6.2)
 ''''''''''''''''''''
 
diff --git a/docs/about/removed-features.rst b/docs/about/removed-features.rst
index eb76974347..4c9e001c35 100644
--- a/docs/about/removed-features.rst
+++ b/docs/about/removed-features.rst
@@ -370,6 +370,23 @@ The ``opened=on`` option in the command line or QMP ``object-add`` either had
 no effect (if ``opened`` was the last option) or caused errors.  The property
 is therefore useless and should simply be removed.
 
+``-display sdl,window_close=...`` (removed in 7.1)
+''''''''''''''''''''''''''''''''''''''''''''''''''
+
+Use ``-display sdl,window-close=...`` instead (i.e. with a minus instead of
+an underscore between "window" and "close").
+
+``-alt-grab`` and ``-display sdl,alt_grab=on`` (removed in 7.1)
+'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
+
+Use ``-display sdl,grab-mod=lshift-lctrl-lalt`` instead.
+
+``-ctrl-grab`` and ``-display sdl,ctrl_grab=on`` (removed in 7.1)
+'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
+
+Use ``-display sdl,grab-mod=rctrl`` instead.
+
+
 QEMU Machine Protocol (QMP) commands
 ------------------------------------
 
diff --git a/qemu-options.hx b/qemu-options.hx
index a664baaa18..726e437a97 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -1938,8 +1938,8 @@ DEF("display", HAS_ARG, QEMU_OPTION_display,
     "-display spice-app[,gl=on|off]\n"
 #endif
 #if defined(CONFIG_SDL)
-    "-display sdl[,alt_grab=on|off][,ctrl_grab=on|off][,gl=on|core|es|off]\n"
-    "            [,grab-mod=<mod>][,show-cursor=on|off][,window-close=on|off]\n"
+    "-display sdl[,gl=on|core|es|off][,grab-mod=<mod>][,show-cursor=on|off]\n"
+    "            [,window-close=on|off]\n"
 #endif
 #if defined(CONFIG_GTK)
     "-display gtk[,full-screen=on|off][,gl=on|off][,grab-on-hover=on|off]\n"
@@ -2012,12 +2012,6 @@ SRST
         the mouse grabbing in conjunction with the "g" key. ``<mods>`` can be
         either ``lshift-lctrl-lalt`` or ``rctrl``.
 
-        ``alt_grab=on|off`` : Use Control+Alt+Shift-g to toggle mouse grabbing.
-        This parameter is deprecated - use ``grab-mod`` instead.
-
-        ``ctrl_grab=on|off`` : Use Right-Control-g to toggle mouse grabbing.
-        This parameter is deprecated - use ``grab-mod`` instead.
-
         ``gl=on|off|core|es`` : Use OpenGL for displaying
 
         ``show-cursor=on|off`` :  Force showing the mouse cursor
@@ -2103,28 +2097,6 @@ SRST
     is displayed in graphical mode.
 ERST
 
-DEF("alt-grab", 0, QEMU_OPTION_alt_grab,
-    "-alt-grab       use Ctrl-Alt-Shift to grab mouse (instead of Ctrl-Alt)\n",
-    QEMU_ARCH_ALL)
-SRST
-``-alt-grab``
-    Use Ctrl-Alt-Shift to grab mouse (instead of Ctrl-Alt). Note that
-    this also affects the special keys (for fullscreen, monitor-mode
-    switching, etc). This option is deprecated - please use
-    ``-display sdl,grab-mod=lshift-lctrl-lalt`` instead.
-ERST
-
-DEF("ctrl-grab", 0, QEMU_OPTION_ctrl_grab,
-    "-ctrl-grab      use Right-Ctrl to grab mouse (instead of Ctrl-Alt)\n",
-    QEMU_ARCH_ALL)
-SRST
-``-ctrl-grab``
-    Use Right-Ctrl to grab mouse (instead of Ctrl-Alt). Note that this
-    also affects the special keys (for fullscreen, monitor-mode
-    switching, etc). This option is deprecated - please use
-    ``-display sdl,grab-mod=rctrl`` instead.
-ERST
-
 DEF("sdl", 0, QEMU_OPTION_sdl,
     "-sdl            shorthand for -display sdl\n", QEMU_ARCH_ALL)
 SRST
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 84a31eba76..57ab9d5322 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -1079,32 +1079,7 @@ static void parse_display(const char *p)
                 } else {
                     goto invalid_sdl_args;
                 }
-            } else if (strstart(opts, ",alt_grab=", &nextopt)) {
-                opts = nextopt;
-                if (strstart(opts, "on", &nextopt)) {
-                    alt_grab = 1;
-                } else if (strstart(opts, "off", &nextopt)) {
-                    alt_grab = 0;
-                } else {
-                    goto invalid_sdl_args;
-                }
-                warn_report("alt_grab is deprecated, use grab-mod instead.");
-            } else if (strstart(opts, ",ctrl_grab=", &nextopt)) {
-                opts = nextopt;
-                if (strstart(opts, "on", &nextopt)) {
-                    ctrl_grab = 1;
-                } else if (strstart(opts, "off", &nextopt)) {
-                    ctrl_grab = 0;
-                } else {
-                    goto invalid_sdl_args;
-                }
-                warn_report("ctrl_grab is deprecated, use grab-mod instead.");
-            } else if (strstart(opts, ",window_close=", &nextopt) ||
-                       strstart(opts, ",window-close=", &nextopt)) {
-                if (strstart(opts, ",window_close=", NULL)) {
-                    warn_report("window_close with an underscore is deprecated,"
-                                " please use window-close instead.");
-                }
+            } else if (strstart(opts, ",window-close=", &nextopt)) {
                 opts = nextopt;
                 dpy.has_window_close = true;
                 if (strstart(opts, "on", &nextopt)) {
@@ -1962,10 +1937,6 @@ static void qemu_create_early_backends(void)
     const bool use_gtk = false;
 #endif
 
-    if ((alt_grab || ctrl_grab) && !use_sdl) {
-        error_report("-alt-grab and -ctrl-grab are only valid "
-                     "for SDL, ignoring option");
-    }
     if (dpy.has_window_close && !use_gtk && !use_sdl) {
         error_report("window-close is only valid for GTK and SDL, "
                      "ignoring option");
@@ -3273,16 +3244,6 @@ void qemu_init(int argc, char **argv, char **envp)
                 dpy.has_full_screen = true;
                 dpy.full_screen = true;
                 break;
-            case QEMU_OPTION_alt_grab:
-                alt_grab = 1;
-                warn_report("-alt-grab is deprecated, please use "
-                            "-display sdl,grab-mod=lshift-lctrl-lalt instead.");
-                break;
-            case QEMU_OPTION_ctrl_grab:
-                ctrl_grab = 1;
-                warn_report("-ctrl-grab is deprecated, please use "
-                            "-display sdl,grab-mod=rctrl instead.");
-                break;
             case QEMU_OPTION_sdl:
                 warn_report("-sdl is deprecated, use -display sdl instead.");
 #ifdef CONFIG_SDL

From 9eafdeeac3513eb515c0e602fc56aa73353cf20d Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Thu, 19 May 2022 17:56:24 +0200
Subject: [PATCH 766/935] ui: Switch "-display sdl" to use the QAPI parser

The "-display sdl" option still uses a hand-crafted parser for its
parameters since we didn't want to drag an interface we considered
somewhat flawed into the QAPI schema. Since the flaws are gone now,
it's time to QAPIfy.

This introduces the new "DisplaySDL" QAPI struct that is used to hold
the parameters that are unique to the SDL display. The only specific
parameter is currently "grab-mod" that is used to specify the required
modifier keys to escape from the mouse grabbing mode.

Message-Id: <20220519155625.1414365-3-thuth@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 include/sysemu/sysemu.h |  2 --
 qapi/ui.json            | 26 ++++++++++++++-
 softmmu/globals.c       |  2 --
 softmmu/vl.c            | 70 +----------------------------------------
 ui/sdl2.c               | 10 ++++++
 5 files changed, 36 insertions(+), 74 deletions(-)

diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index b4030acd74..812f66a31a 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -42,8 +42,6 @@ extern int graphic_depth;
 extern int display_opengl;
 extern const char *keyboard_layout;
 extern int win2k_install_hack;
-extern int alt_grab;
-extern int ctrl_grab;
 extern int graphic_rotate;
 extern int old_param;
 extern uint8_t *boot_splash_filedata;
diff --git a/qapi/ui.json b/qapi/ui.json
index 11a827d10f..413371d5e8 100644
--- a/qapi/ui.json
+++ b/qapi/ui.json
@@ -1295,6 +1295,29 @@
       '*swap-opt-cmd': 'bool'
   } }
 
+##
+# @HotKeyMod:
+#
+# Set of modifier keys that need to be held for shortcut key actions.
+#
+# Since: 7.1
+##
+{ 'enum'  : 'HotKeyMod',
+  'data'  : [ 'lctrl-lalt', 'lshift-lctrl-lalt', 'rctrl' ] }
+
+##
+# @DisplaySDL:
+#
+# SDL2 display options.
+#
+# @grab-mod:  Modifier keys that should be pressed together with the
+#             "G" key to release the mouse grab.
+#
+# Since: 7.1
+##
+{ 'struct'  : 'DisplaySDL',
+  'data'    : { '*grab-mod'   : 'HotKeyMod' } }
+
 ##
 # @DisplayType:
 #
@@ -1374,7 +1397,8 @@
       'curses': { 'type': 'DisplayCurses', 'if': 'CONFIG_CURSES' },
       'egl-headless': { 'type': 'DisplayEGLHeadless',
                         'if': { 'all': ['CONFIG_OPENGL', 'CONFIG_GBM'] } },
-      'dbus': { 'type': 'DisplayDBus', 'if': 'CONFIG_DBUS_DISPLAY' }
+      'dbus': { 'type': 'DisplayDBus', 'if': 'CONFIG_DBUS_DISPLAY' },
+      'sdl': { 'type': 'DisplaySDL', 'if': 'CONFIG_SDL' }
   }
 }
 
diff --git a/softmmu/globals.c b/softmmu/globals.c
index 916bc12e2b..527edbefdd 100644
--- a/softmmu/globals.c
+++ b/softmmu/globals.c
@@ -50,8 +50,6 @@ QEMUOptionRom option_rom[MAX_OPTION_ROMS];
 int nb_option_roms;
 int old_param;
 const char *qemu_name;
-int alt_grab;
-int ctrl_grab;
 unsigned int nb_prom_envs;
 const char *prom_envs[MAX_PROM_ENVS];
 uint8_t *boot_splash_filedata;
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 57ab9d5322..484e9d9921 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -1056,75 +1056,7 @@ static void parse_display(const char *p)
         exit(0);
     }
 
-    if (strstart(p, "sdl", &opts)) {
-        /*
-         * sdl DisplayType needs hand-crafted parser instead of
-         * parse_display_qapi() due to some options not in
-         * DisplayOptions, specifically:
-         *   - ctrl_grab + alt_grab
-         *     They can't be moved into the QAPI since they use underscores,
-         *     thus they will get replaced by "grab-mod" in the long term
-         */
-#if defined(CONFIG_SDL)
-        dpy.type = DISPLAY_TYPE_SDL;
-        while (*opts) {
-            const char *nextopt;
-
-            if (strstart(opts, ",grab-mod=", &nextopt)) {
-                opts = nextopt;
-                if (strstart(opts, "lshift-lctrl-lalt", &nextopt)) {
-                    alt_grab = 1;
-                } else if (strstart(opts, "rctrl", &nextopt)) {
-                    ctrl_grab = 1;
-                } else {
-                    goto invalid_sdl_args;
-                }
-            } else if (strstart(opts, ",window-close=", &nextopt)) {
-                opts = nextopt;
-                dpy.has_window_close = true;
-                if (strstart(opts, "on", &nextopt)) {
-                    dpy.window_close = true;
-                } else if (strstart(opts, "off", &nextopt)) {
-                    dpy.window_close = false;
-                } else {
-                    goto invalid_sdl_args;
-                }
-            } else if (strstart(opts, ",show-cursor=", &nextopt)) {
-                opts = nextopt;
-                dpy.has_show_cursor = true;
-                if (strstart(opts, "on", &nextopt)) {
-                    dpy.show_cursor = true;
-                } else if (strstart(opts, "off", &nextopt)) {
-                    dpy.show_cursor = false;
-                } else {
-                    goto invalid_sdl_args;
-                }
-            } else if (strstart(opts, ",gl=", &nextopt)) {
-                opts = nextopt;
-                dpy.has_gl = true;
-                if (strstart(opts, "on", &nextopt)) {
-                    dpy.gl = DISPLAYGL_MODE_ON;
-                } else if (strstart(opts, "core", &nextopt)) {
-                    dpy.gl = DISPLAYGL_MODE_CORE;
-                } else if (strstart(opts, "es", &nextopt)) {
-                    dpy.gl = DISPLAYGL_MODE_ES;
-                } else if (strstart(opts, "off", &nextopt)) {
-                    dpy.gl = DISPLAYGL_MODE_OFF;
-                } else {
-                    goto invalid_sdl_args;
-                }
-            } else {
-            invalid_sdl_args:
-                error_report("invalid SDL option string");
-                exit(1);
-            }
-            opts = nextopt;
-        }
-#else
-        error_report("SDL display supported is not available in this binary");
-        exit(1);
-#endif
-    } else if (strstart(p, "vnc", &opts)) {
+    if (strstart(p, "vnc", &opts)) {
         /*
          * vnc isn't a (local) DisplayType but a protocol for remote
          * display access.
diff --git a/ui/sdl2.c b/ui/sdl2.c
index d3741f9b75..8cb77416af 100644
--- a/ui/sdl2.c
+++ b/ui/sdl2.c
@@ -40,6 +40,8 @@ static struct sdl2_console *sdl2_console;
 
 static SDL_Surface *guest_sprite_surface;
 static int gui_grab; /* if true, all keyboard/mouse events are grabbed */
+static bool alt_grab;
+static bool ctrl_grab;
 
 static int gui_saved_grab;
 static int gui_fullscreen;
@@ -853,6 +855,14 @@ static void sdl2_display_init(DisplayState *ds, DisplayOptions *o)
 
     gui_fullscreen = o->has_full_screen && o->full_screen;
 
+    if (o->u.sdl.has_grab_mod) {
+        if (o->u.sdl.grab_mod == HOT_KEY_MOD_LSHIFT_LCTRL_LALT) {
+            alt_grab = true;
+        } else if (o->u.sdl.grab_mod == HOT_KEY_MOD_RCTRL) {
+            ctrl_grab = true;
+        }
+    }
+
     for (i = 0;; i++) {
         QemuConsole *con = qemu_console_lookup_by_index(i);
         if (!con) {

From 707d93d4abc6485c638e2aecd93fbd904f2f6b3e Mon Sep 17 00:00:00 2001
From: Thomas Huth <thuth@redhat.com>
Date: Thu, 19 May 2022 17:56:25 +0200
Subject: [PATCH 767/935] ui: Remove deprecated options "-sdl" and "-curses"
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

We have "-sdl" and "-curses", but no "-gtk" and no "-cocoa" ...
these old-style options are rather confusing than helpful nowadays.
Now that the deprecation period is over, let's remove them, so we
get a cleaner interface (where "-display" is the only way to select
the user interface).

Message-Id: <20220519155625.1414365-4-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 docs/about/deprecated.rst       | 10 ----------
 docs/about/removed-features.rst | 10 ++++++++++
 qemu-options.hx                 | 24 ++----------------------
 softmmu/vl.c                    | 19 -------------------
 4 files changed, 12 insertions(+), 51 deletions(-)

diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst
index 562a133f18..e19bcba242 100644
--- a/docs/about/deprecated.rst
+++ b/docs/about/deprecated.rst
@@ -81,16 +81,6 @@ the process listing. This is replaced by the new ``password-secret``
 option which lets the password be securely provided on the command
 line using a ``secret`` object instance.
 
-``-sdl`` (since 6.2)
-''''''''''''''''''''
-
-Use ``-display sdl`` instead.
-
-``-curses`` (since 6.2)
-'''''''''''''''''''''''
-
-Use ``-display curses`` instead.
-
 ``-watchdog`` (since 6.2)
 '''''''''''''''''''''''''
 
diff --git a/docs/about/removed-features.rst b/docs/about/removed-features.rst
index 4c9e001c35..c7b9dadd5d 100644
--- a/docs/about/removed-features.rst
+++ b/docs/about/removed-features.rst
@@ -386,6 +386,16 @@ Use ``-display sdl,grab-mod=lshift-lctrl-lalt`` instead.
 
 Use ``-display sdl,grab-mod=rctrl`` instead.
 
+``-sdl`` (removed in 7.1)
+'''''''''''''''''''''''''
+
+Use ``-display sdl`` instead.
+
+``-curses`` (removed in 7.1)
+''''''''''''''''''''''''''''
+
+Use ``-display curses`` instead.
+
 
 QEMU Machine Protocol (QMP) commands
 ------------------------------------
diff --git a/qemu-options.hx b/qemu-options.hx
index 726e437a97..60cf188da4 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -1981,9 +1981,8 @@ DEF("display", HAS_ARG, QEMU_OPTION_display,
     , QEMU_ARCH_ALL)
 SRST
 ``-display type``
-    Select type of display to use. This option is a replacement for the
-    old style -sdl/-curses/... options. Use ``-display help`` to list
-    the available display types. Valid values for type are
+    Select type of display to use. Use ``-display help`` to list the available
+    display types. Valid values for type are
 
     ``spice-app[,gl=on|off]``
         Start QEMU as a Spice server and launch the default Spice client
@@ -2085,25 +2084,6 @@ SRST
     Use C-a h for help on switching between the console and monitor.
 ERST
 
-DEF("curses", 0, QEMU_OPTION_curses,
-    "-curses         shorthand for -display curses\n",
-    QEMU_ARCH_ALL)
-SRST
-``-curses``
-    Normally, if QEMU is compiled with graphical window support, it
-    displays output such as guest graphics, guest console, and the QEMU
-    monitor in a window. With this option, QEMU can display the VGA
-    output when in text mode using a curses/ncurses interface. Nothing
-    is displayed in graphical mode.
-ERST
-
-DEF("sdl", 0, QEMU_OPTION_sdl,
-    "-sdl            shorthand for -display sdl\n", QEMU_ARCH_ALL)
-SRST
-``-sdl``
-    Enable SDL.
-ERST
-
 #ifdef CONFIG_SPICE
 DEF("spice", HAS_ARG, QEMU_OPTION_spice,
     "-spice [port=port][,tls-port=secured-port][,x509-dir=<dir>]\n"
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 484e9d9921..4c1e94b00e 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -2800,16 +2800,6 @@ void qemu_init(int argc, char **argv, char **envp)
                 nographic = true;
                 dpy.type = DISPLAY_TYPE_NONE;
                 break;
-            case QEMU_OPTION_curses:
-                warn_report("-curses is deprecated, "
-                            "use -display curses instead.");
-#ifdef CONFIG_CURSES
-                dpy.type = DISPLAY_TYPE_CURSES;
-#else
-                error_report("curses or iconv support is disabled");
-                exit(1);
-#endif
-                break;
             case QEMU_OPTION_portrait:
                 graphic_rotate = 90;
                 break;
@@ -3176,15 +3166,6 @@ void qemu_init(int argc, char **argv, char **envp)
                 dpy.has_full_screen = true;
                 dpy.full_screen = true;
                 break;
-            case QEMU_OPTION_sdl:
-                warn_report("-sdl is deprecated, use -display sdl instead.");
-#ifdef CONFIG_SDL
-                dpy.type = DISPLAY_TYPE_SDL;
-                break;
-#else
-                error_report("SDL support is disabled");
-                exit(1);
-#endif
             case QEMU_OPTION_pidfile:
                 pid_file = optarg;
                 break;

From 1e64facc015e16d8e4efa239feaeda9e4e9aeb04 Mon Sep 17 00:00:00 2001
From: Dmitry Tikhov <ddtikhov@gmail.com>
Date: Tue, 12 Apr 2022 11:59:09 +0300
Subject: [PATCH 768/935] hw/nvme: fix narrowing conversion

Since nlbas is of type int, it does not work with large namespace size
values, e.g., 9 TB size of file backing namespace and 8 byte metadata
with 4096 bytes lbasz gives negative nlbas value, which is later
promoted to negative int64_t type value and results in negative
ns->moff which breaks namespace

Signed-off-by: Dmitry Tikhov <ddtikhov@gmail.com>
Reviewed-by: Klaus Jensen <k.jensen@samsung.com>
Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
---
 hw/nvme/ns.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/hw/nvme/ns.c b/hw/nvme/ns.c
index 324f53ea0c..af6504fad2 100644
--- a/hw/nvme/ns.c
+++ b/hw/nvme/ns.c
@@ -29,7 +29,8 @@ void nvme_ns_init_format(NvmeNamespace *ns)
 {
     NvmeIdNs *id_ns = &ns->id_ns;
     BlockDriverInfo bdi;
-    int npdg, nlbas, ret;
+    int npdg, ret;
+    int64_t nlbas;
 
     ns->lbaf = id_ns->lbaf[NVME_ID_NS_FLBAS_INDEX(id_ns->flbas)];
     ns->lbasz = 1 << ns->lbaf.ds;
@@ -42,7 +43,7 @@ void nvme_ns_init_format(NvmeNamespace *ns)
     id_ns->ncap = id_ns->nsze;
     id_ns->nuse = id_ns->ncap;
 
-    ns->moff = (int64_t)nlbas << ns->lbaf.ds;
+    ns->moff = nlbas << ns->lbaf.ds;
 
     npdg = ns->blkconf.discard_granularity / ns->lbasz;
 

From 51c453266309166c2737623211c0afc12884cccd Mon Sep 17 00:00:00 2001
From: Dmitry Tikhov <d.tihov@yadro.com>
Date: Fri, 15 Apr 2022 23:48:32 +0300
Subject: [PATCH 769/935] hw/nvme: add missing return statement

Since there is no return after nvme_dsm_cb invocation, metadata
associated with non-zero block range is currently zeroed. Also this
behaviour leads to segfault since we schedule iocb->bh two times.
First when entering nvme_dsm_cb with iocb->idx == iocb->nr and
second because of missing return on call stack unwinding by calling
blk_aio_pwrite_zeroes and subsequent nvme_dsm_cb callback.

Fixes: d7d1474fd85d ("hw/nvme: reimplement dsm to allow cancellation")
Signed-off-by: Dmitry Tikhov <d.tihov@yadro.com>
Reviewed-by: Klaus Jensen <k.jensen@samsung.com>
Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
---
 hw/nvme/ctrl.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c
index 03760ddeae..74540a03d5 100644
--- a/hw/nvme/ctrl.c
+++ b/hw/nvme/ctrl.c
@@ -2372,6 +2372,7 @@ static void nvme_dsm_md_cb(void *opaque, int ret)
         }
 
         nvme_dsm_cb(iocb, 0);
+        return;
     }
 
     iocb->aiocb = blk_aio_pwrite_zeroes(ns->blkconf.blk, nvme_moff(ns, slba),

From 2e8f952ae7de23b4847937dbbf51f7a1ab10a2af Mon Sep 17 00:00:00 2001
From: Dmitry Tikhov <d.tihov@yadro.com>
Date: Thu, 21 Apr 2022 13:51:58 +0300
Subject: [PATCH 770/935] hw/nvme: fix copy cmd for pi enabled namespaces

Current implementation have problem in the read part of copy command.
Because there is no metadata mangling before nvme_dif_check invocation,
reftag error could be thrown for blocks of namespace that have not been
previously written to.

Signed-off-by: Dmitry Tikhov <d.tihov@yadro.com>
Reviewed-by: Klaus Jensen <k.jensen@samsung.com>
Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
---
 hw/nvme/ctrl.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c
index 74540a03d5..08574c4dcb 100644
--- a/hw/nvme/ctrl.c
+++ b/hw/nvme/ctrl.c
@@ -2787,6 +2787,10 @@ static void nvme_copy_in_completed_cb(void *opaque, int ret)
         size_t mlen = nvme_m2b(ns, nlb);
         uint8_t *mbounce = iocb->bounce + nvme_l2b(ns, nlb);
 
+        status = nvme_dif_mangle_mdata(ns, mbounce, mlen, slba);
+        if (status) {
+            goto invalid;
+        }
         status = nvme_dif_check(ns, iocb->bounce, len, mbounce, mlen, prinfor,
                                 slba, apptag, appmask, &reftag);
         if (status) {

From 9235a72a5df0fae1ede89f02717b597ef91cf6ad Mon Sep 17 00:00:00 2001
From: Klaus Jensen <k.jensen@samsung.com>
Date: Fri, 6 May 2022 00:21:47 +0200
Subject: [PATCH 771/935] hw/nvme: fix smart aen

Pass the right constant to nvme_smart_event(). The NVME_AER* values hold
the bit position in the SMART byte, not the shifted value that we expect
it to be in nvme_smart_event().

Fixes: c62720f137df ("hw/block/nvme: trigger async event during injecting smart warning")
Acked-by: zhenwei pi <pizhenwei@bytedance.com>
Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
---
 hw/nvme/ctrl.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c
index 08574c4dcb..a2f6069f7f 100644
--- a/hw/nvme/ctrl.c
+++ b/hw/nvme/ctrl.c
@@ -5325,7 +5325,7 @@ static uint16_t nvme_set_feature(NvmeCtrl *n, NvmeRequest *req)
 
         if ((n->temperature >= n->features.temp_thresh_hi) ||
             (n->temperature <= n->features.temp_thresh_low)) {
-            nvme_smart_event(n, NVME_AER_INFO_SMART_TEMP_THRESH);
+            nvme_smart_event(n, NVME_SMART_TEMPERATURE);
         }
 
         break;

From a859eb9f8f64e116671048a43a07d87bc6527a55 Mon Sep 17 00:00:00 2001
From: Klaus Jensen <k.jensen@samsung.com>
Date: Fri, 29 Apr 2022 10:33:32 +0200
Subject: [PATCH 772/935] hw/nvme: enforce common serial per subsystem

The Identify Controller Serial Number (SN) is the serial number for the
NVM subsystem and must be the same across all controller in the NVM
subsystem.

Enforce this.

Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Keith Busch <kbusch@kernel.org>
Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
---
 hw/nvme/nvme.h   | 1 +
 hw/nvme/subsys.c | 7 +++++++
 2 files changed, 8 insertions(+)

diff --git a/hw/nvme/nvme.h b/hw/nvme/nvme.h
index 6773819325..e41771604f 100644
--- a/hw/nvme/nvme.h
+++ b/hw/nvme/nvme.h
@@ -48,6 +48,7 @@ typedef struct NvmeSubsystem {
     DeviceState parent_obj;
     NvmeBus     bus;
     uint8_t     subnqn[256];
+    char        *serial;
 
     NvmeCtrl      *ctrls[NVME_MAX_CONTROLLERS];
     NvmeNamespace *namespaces[NVME_MAX_NAMESPACES + 1];
diff --git a/hw/nvme/subsys.c b/hw/nvme/subsys.c
index fb58d63950..691a90d209 100644
--- a/hw/nvme/subsys.c
+++ b/hw/nvme/subsys.c
@@ -27,6 +27,13 @@ int nvme_subsys_register_ctrl(NvmeCtrl *n, Error **errp)
         return -1;
     }
 
+    if (!subsys->serial) {
+        subsys->serial = g_strdup(n->params.serial);
+    } else if (strcmp(subsys->serial, n->params.serial)) {
+        error_setg(errp, "invalid controller serial");
+        return -1;
+    }
+
     subsys->ctrls[cntlid] = n;
 
     for (nsid = 1; nsid < ARRAY_SIZE(subsys->namespaces); nsid++) {

From 36d83272d5e45dff13e988ee0a59f11c58b442ba Mon Sep 17 00:00:00 2001
From: Klaus Jensen <k.jensen@samsung.com>
Date: Fri, 29 Apr 2022 10:33:33 +0200
Subject: [PATCH 773/935] hw/nvme: do not auto-generate eui64

We cannot provide auto-generated unique or persistent namespace
identifiers (EUI64, NGUID, UUID) easily. Since 6.1, namespaces have been
assigned a generated EUI64 of the form "52:54:00:<namespace counter>".
This is will be unique within a QEMU instance, but not globally.

Revert that this is assigned automatically and immediately deprecate the
compatibility parameter. Users can opt-in to this with the
`eui64-default=on` device parameter or set it explicitly with
`eui64=UINT64`.

Cc: libvir-list@redhat.com
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
---
 docs/about/deprecated.rst | 7 +++++++
 hw/core/machine.c         | 1 +
 hw/nvme/ns.c              | 2 +-
 3 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst
index e19bcba242..47a8628b56 100644
--- a/docs/about/deprecated.rst
+++ b/docs/about/deprecated.rst
@@ -296,6 +296,13 @@ contains native support for this feature and thus use of the option
 ROM approach is obsolete. The native SeaBIOS support can be activated
 by using ``-machine graphics=off``.
 
+``-device nvme-ns,eui64-default=on|off`` (since 7.1)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In QEMU versions 6.1, 6.2 and 7.0, the ``nvme-ns`` generates an EUI-64
+identifer that is not globally unique. If an EUI-64 identifer is required, the
+user must set it explicitly using the ``nvme-ns`` device parameter ``eui64``.
+
 
 Block device options
 ''''''''''''''''''''
diff --git a/hw/core/machine.c b/hw/core/machine.c
index bb0dc8f6a9..c53548d0b1 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -43,6 +43,7 @@
 
 GlobalProperty hw_compat_7_0[] = {
     { "arm-gicv3-common", "force-8-bit-prio", "on" },
+    { "nvme-ns", "eui64-default", "on"},
 };
 const size_t hw_compat_7_0_len = G_N_ELEMENTS(hw_compat_7_0);
 
diff --git a/hw/nvme/ns.c b/hw/nvme/ns.c
index af6504fad2..06a04131f1 100644
--- a/hw/nvme/ns.c
+++ b/hw/nvme/ns.c
@@ -641,7 +641,7 @@ static Property nvme_ns_props[] = {
     DEFINE_PROP_SIZE("zoned.zrwas", NvmeNamespace, params.zrwas, 0),
     DEFINE_PROP_SIZE("zoned.zrwafg", NvmeNamespace, params.zrwafg, -1),
     DEFINE_PROP_BOOL("eui64-default", NvmeNamespace, params.eui64_default,
-                     true),
+                     false),
     DEFINE_PROP_END_OF_LIST(),
 };
 

From bd9f371c6f6eeb8e907dfc770876ad8ef4ff85fc Mon Sep 17 00:00:00 2001
From: Klaus Jensen <k.jensen@samsung.com>
Date: Fri, 29 Apr 2022 10:33:34 +0200
Subject: [PATCH 774/935] hw/nvme: do not auto-generate uuid

Do not default to generate an UUID for namespaces if it is not
explicitly specified.

This is a technically a breaking change in behavior. However, since the
UUID changes on every VM launch, it is not spec compliant and is of
little use since the UUID cannot be used reliably anyway and the
behavior prior to this patch must be considered buggy.

Reviewed-by: Keith Busch <kbusch@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
---
 hw/nvme/ns.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/nvme/ns.c b/hw/nvme/ns.c
index 06a04131f1..1b9c9d1156 100644
--- a/hw/nvme/ns.c
+++ b/hw/nvme/ns.c
@@ -614,7 +614,7 @@ static Property nvme_ns_props[] = {
     DEFINE_PROP_BOOL("detached", NvmeNamespace, params.detached, false),
     DEFINE_PROP_BOOL("shared", NvmeNamespace, params.shared, true),
     DEFINE_PROP_UINT32("nsid", NvmeNamespace, params.nsid, 0),
-    DEFINE_PROP_UUID("uuid", NvmeNamespace, params.uuid),
+    DEFINE_PROP_UUID_NODEFAULT("uuid", NvmeNamespace, params.uuid),
     DEFINE_PROP_UINT64("eui64", NvmeNamespace, params.eui64, 0),
     DEFINE_PROP_UINT16("ms", NvmeNamespace, params.ms, 0),
     DEFINE_PROP_UINT8("mset", NvmeNamespace, params.mset, 0),

From 9f2e1acf83c332752f52c39dad390c94ec2ba9f5 Mon Sep 17 00:00:00 2001
From: Klaus Jensen <k.jensen@samsung.com>
Date: Fri, 29 Apr 2022 10:33:35 +0200
Subject: [PATCH 775/935] hw/nvme: do not report null uuid

Do not report the "null uuid" (all zeros) in the namespace
identification descriptors.

Reported-by: Luis Chamberlain <mcgrof@kernel.org>
Reported-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Keith Busch <kbusch@kernel.org>
Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
---
 hw/nvme/ctrl.c | 17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c
index a2f6069f7f..909e357a7e 100644
--- a/hw/nvme/ctrl.c
+++ b/hw/nvme/ctrl.c
@@ -4955,16 +4955,13 @@ static uint16_t nvme_identify_ns_descr_list(NvmeCtrl *n, NvmeRequest *req)
         return NVME_INVALID_FIELD | NVME_DNR;
     }
 
-    /*
-     * If the EUI-64 field is 0 and the NGUID field is 0, the namespace must
-     * provide a valid Namespace UUID in the Namespace Identification Descriptor
-     * data structure. QEMU does not yet support setting NGUID.
-     */
-    uuid.hdr.nidt = NVME_NIDT_UUID;
-    uuid.hdr.nidl = NVME_NIDL_UUID;
-    memcpy(uuid.v, ns->params.uuid.data, NVME_NIDL_UUID);
-    memcpy(pos, &uuid, sizeof(uuid));
-    pos += sizeof(uuid);
+    if (!qemu_uuid_is_null(&ns->params.uuid)) {
+        uuid.hdr.nidt = NVME_NIDT_UUID;
+        uuid.hdr.nidl = NVME_NIDL_UUID;
+        memcpy(uuid.v, ns->params.uuid.data, NVME_NIDL_UUID);
+        memcpy(pos, &uuid, sizeof(uuid));
+        pos += sizeof(uuid);
+    }
 
     if (ns->params.eui64) {
         eui64.hdr.nidt = NVME_NIDT_EUI64;

From fbba243bc700a4e479331e20544c7f6a41ae87b3 Mon Sep 17 00:00:00 2001
From: Klaus Jensen <k.jensen@samsung.com>
Date: Fri, 29 Apr 2022 10:33:36 +0200
Subject: [PATCH 776/935] hw/nvme: bump firmware revision

The Linux kernel quirks the QEMU NVMe controller pretty heavily because
of the namespace identifier mess. Since this is now fixed, bump the
firmware revision number to allow the quirk to be disabled for this
revision.

As of now, bump the firmware revision number to be equal to the QEMU
release version number.

Reviewed-by: Keith Busch <kbusch@kernel.org>
Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
---
 hw/nvme/ctrl.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c
index 909e357a7e..1e6e0fcad9 100644
--- a/hw/nvme/ctrl.c
+++ b/hw/nvme/ctrl.c
@@ -6713,7 +6713,7 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev)
     id->vid = cpu_to_le16(pci_get_word(pci_conf + PCI_VENDOR_ID));
     id->ssvid = cpu_to_le16(pci_get_word(pci_conf + PCI_SUBSYSTEM_VENDOR_ID));
     strpadcpy((char *)id->mn, sizeof(id->mn), "QEMU NVMe Ctrl", ' ');
-    strpadcpy((char *)id->fr, sizeof(id->fr), "1.0", ' ');
+    strpadcpy((char *)id->fr, sizeof(id->fr), QEMU_VERSION, ' ');
     strpadcpy((char *)id->sn, sizeof(id->sn), n->params.serial, ' ');
 
     id->cntlid = cpu_to_le16(n->cntlid);

From 8b1e59a6873662a01379cf052384e5dedefe7447 Mon Sep 17 00:00:00 2001
From: Klaus Jensen <k.jensen@samsung.com>
Date: Tue, 19 Apr 2022 13:24:23 +0200
Subject: [PATCH 777/935] hw/nvme: deprecate the use-intel-id compatibility
 parameter
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Since version 5.2 commit 6eb7a071292a ("hw/block/nvme: change controller
pci id"), the emulated NVMe controller has defaulted to a non-Intel PCI
identifier.

Deprecate the compatibility parameter so we can get rid of it once and
for all.

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
---
 docs/about/deprecated.rst | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst
index 47a8628b56..aa2e320207 100644
--- a/docs/about/deprecated.rst
+++ b/docs/about/deprecated.rst
@@ -303,6 +303,14 @@ In QEMU versions 6.1, 6.2 and 7.0, the ``nvme-ns`` generates an EUI-64
 identifer that is not globally unique. If an EUI-64 identifer is required, the
 user must set it explicitly using the ``nvme-ns`` device parameter ``eui64``.
 
+``-device nvme,use-intel-id=on|off`` (since 7.1)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``nvme`` device originally used a PCI Vendor/Device Identifier combination
+from Intel that was not properly allocated. Since version 5.2, the controller
+has used a properly allocated identifier. Deprecate the ``use-intel-id``
+machine compatibility parameter.
+
 
 Block device options
 ''''''''''''''''''''

From d7fe639cabf778903f6cab23ff58c905c71375ec Mon Sep 17 00:00:00 2001
From: Dmitry Tikhov <d.tihov@yadro.com>
Date: Wed, 20 Apr 2022 11:20:44 +0300
Subject: [PATCH 778/935] hw/nvme: add new command abort case
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

NVMe command set specification for end-to-end data protection formatted
namespace states:

    o If the Reference Tag Check bit of the PRCHK field is set to ‘1’ and
      the namespace is formatted for Type 3 protection, then the
      controller:
          ▪ should not compare the protection Information Reference Tag
            field to the computed reference tag; and
          ▪ may ignore the ILBRT and EILBRT fields. If a command is
            aborted as a result of the Reference Tag Check bit of the
            PRCHK field being set to ‘1’, then that command should be
            aborted with a status code of Invalid Protection Information,
            but may be aborted with a status code of Invalid Field in
            Command.

Currently qemu compares reftag in the nvme_dif_prchk function whenever
Reference Tag Check bit is set in the command. For type 3 namespaces
however, caller of nvme_dif_prchk - nvme_dif_check does not increment
reftag for each subsequent logical block. That way commands incorporating
more than one logical block for type 3 formatted namespaces with reftag
check bit set, always fail with End-to-end Reference Tag Check Error.
Comply with spec by handling case of set Reference Tag Check
bit in the type 3 formatted namespace.

Fixes: 146f720c5563 ("hw/block/nvme: end-to-end data protection")
Signed-off-by: Dmitry Tikhov <d.tihov@yadro.com>
Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
---
 hw/nvme/dif.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/hw/nvme/dif.c b/hw/nvme/dif.c
index 62d885f83e..63c44c86ab 100644
--- a/hw/nvme/dif.c
+++ b/hw/nvme/dif.c
@@ -26,6 +26,11 @@ uint16_t nvme_check_prinfo(NvmeNamespace *ns, uint8_t prinfo, uint64_t slba,
         return NVME_INVALID_PROT_INFO | NVME_DNR;
     }
 
+    if ((NVME_ID_NS_DPS_TYPE(ns->id_ns.dps) == NVME_ID_NS_DPS_TYPE_3) &&
+        (prinfo & NVME_PRINFO_PRCHK_REF)) {
+        return NVME_INVALID_PROT_INFO;
+    }
+
     return NVME_SUCCESS;
 }
 

From 9f9dcb96a46a60fcf95f7baebafa3ec5e2a1b5ce Mon Sep 17 00:00:00 2001
From: Stephen Michael Jothen <sjothen@gmail.com>
Date: Wed, 25 May 2022 17:33:36 +0200
Subject: [PATCH 779/935] target/i386/tcg: Fix masking of real-mode addresses
 with A20 bit

The correct A20 masking is done if paging is enabled (protected mode) but it
seems to have been forgotten in real mode. For example from the AMD64 APM Vol. 2
section 1.2.4:

> If the sum of the segment base and effective address carries over into bit 20,
> that bit can be optionally truncated to mimic the 20-bit address wrapping of the
> 8086 processor by using the A20M# input signal to mask the A20 address bit.

Most BIOSes will enable the A20 line on boot, but I found by disabling the A20 line
afterwards, the correct wrapping wasn't taking place.

`handle_mmu_fault' in target/i386/tcg/sysemu/excp_helper.c seems to be the culprit.
In real mode, it fills the TLB with the raw unmasked address. However, for the
protected mode, the `mmu_translate' function does the correct A20 masking.

The fix then should be to just apply the A20 mask in the first branch of the if
statement.

Signed-off-by: Stephen Michael Jothen <sjothen@gmail.com>
Message-Id: <Yo5MUMSz80jXtvt9@air-old.local>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/tcg/sysemu/excp_helper.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/target/i386/tcg/sysemu/excp_helper.c b/target/i386/tcg/sysemu/excp_helper.c
index e1b6d88683..48feba7e75 100644
--- a/target/i386/tcg/sysemu/excp_helper.c
+++ b/target/i386/tcg/sysemu/excp_helper.c
@@ -359,6 +359,7 @@ static int handle_mmu_fault(CPUState *cs, vaddr addr, int size,
     CPUX86State *env = &cpu->env;
     int error_code = PG_ERROR_OK;
     int pg_mode, prot, page_size;
+    int32_t a20_mask;
     hwaddr paddr;
     hwaddr vaddr;
 
@@ -368,7 +369,8 @@ static int handle_mmu_fault(CPUState *cs, vaddr addr, int size,
 #endif
 
     if (!(env->cr[0] & CR0_PG_MASK)) {
-        paddr = addr;
+        a20_mask = x86_get_a20_mask(env);
+        paddr = addr & a20_mask;
 #ifdef TARGET_X86_64
         if (!(env->hflags & HF_LMA_MASK)) {
             /* Without long mode we can only address 32bits in real mode */

From d8f3a60931a700c327aa2ee10d0beaeea2cb97da Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Mon, 30 May 2022 14:24:34 +0200
Subject: [PATCH 780/935] build: add a "make modules" target

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 meson.build | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meson.build b/meson.build
index bc6234c85e..9b3fd4e52b 100644
--- a/meson.build
+++ b/meson.build
@@ -3285,6 +3285,9 @@ foreach m : block_mods + softmmu_mods
                 install: true,
                 install_dir: qemu_moddir)
 endforeach
+if emulator_modules.length() > 0
+  alias_target('modules', emulator_modules)
+endif
 
 softmmu_ss.add(authz, blockdev, chardev, crypto, io, qmp)
 common_ss.add(qom, qemuutil)

From b0f3184e82817b700ea81bc76ebc8d999de35278 Mon Sep 17 00:00:00 2001
From: Yang Zhong <yang.zhong@intel.com>
Date: Mon, 30 May 2022 09:18:34 -0400
Subject: [PATCH 781/935] target/i386: Fix wrong count setting

The previous patch used wrong count setting with index value, which got wrong
value from CPUID(EAX=12,ECX=0):EAX. So the SGX1 instruction can't be exposed
to VM and the SGX decice can't work in VM.

Fixes: d19d6ffa0710 ("target/i386: introduce helper to access supported CPUID")

Signed-off-by: Yang Zhong <yang.zhong@intel.com>
Message-Id: <20220530131834.1222801-1-yang.zhong@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/cpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index bb6a5dd498..9fdfec9d8b 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5559,7 +5559,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
          * supports.  Features can be further restricted by userspace, but not
          * made more permissive.
          */
-        x86_cpu_get_supported_cpuid(0x12, index, eax, ebx, ecx, edx);
+        x86_cpu_get_supported_cpuid(0x12, count, eax, ebx, ecx, edx);
 
         if (count == 0) {
             *eax &= env->features[FEAT_SGX_12_0_EAX];

From 24ce7aa77dd1a3095d994bf53d60cce6e672cf4d Mon Sep 17 00:00:00 2001
From: "Jose R. Ziviani" <jziviani@suse.de>
Date: Sat, 28 May 2022 00:20:23 +0200
Subject: [PATCH 782/935] modules: introduces module_kconfig directive

module_kconfig is a new directive that should be used with module_obj
whenever that module depends on the Kconfig to be enabled.

When the module is enabled in Kconfig we are sure that its dependencies
will be enabled as well, thus the module will be loaded without any
problem.

The correct way to use module_kconfig is by passing the Kconfig option
to module_kconfig (or the *config-devices.mak without CONFIG_).

Signed-off-by: Jose R. Ziviani <jziviani@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Message-Id: <165369002370.5857.12150544416563557322.stgit@work>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/display/qxl.c                |  1 +
 hw/display/vhost-user-gpu-pci.c |  1 +
 hw/display/vhost-user-gpu.c     |  1 +
 hw/display/vhost-user-vga.c     |  1 +
 hw/display/virtio-gpu-base.c    |  1 +
 hw/display/virtio-gpu-gl.c      |  1 +
 hw/display/virtio-gpu-pci-gl.c  |  1 +
 hw/display/virtio-gpu-pci.c     |  1 +
 hw/display/virtio-gpu.c         |  1 +
 hw/display/virtio-vga-gl.c      |  1 +
 hw/display/virtio-vga.c         |  1 +
 hw/s390x/virtio-ccw-gpu.c       |  1 +
 hw/usb/ccid-card-emulated.c     |  1 +
 hw/usb/ccid-card-passthru.c     |  1 +
 hw/usb/host-libusb.c            |  1 +
 hw/usb/redirect.c               |  1 +
 include/qemu/module.h           | 10 ++++++++++
 scripts/modinfo-generate.py     |  2 ++
 18 files changed, 28 insertions(+)

diff --git a/hw/display/qxl.c b/hw/display/qxl.c
index 2db34714fb..5b10f697f1 100644
--- a/hw/display/qxl.c
+++ b/hw/display/qxl.c
@@ -2515,6 +2515,7 @@ static const TypeInfo qxl_primary_info = {
     .class_init    = qxl_primary_class_init,
 };
 module_obj("qxl-vga");
+module_kconfig(QXL);
 
 static void qxl_secondary_class_init(ObjectClass *klass, void *data)
 {
diff --git a/hw/display/vhost-user-gpu-pci.c b/hw/display/vhost-user-gpu-pci.c
index daefcf7101..d119bcae45 100644
--- a/hw/display/vhost-user-gpu-pci.c
+++ b/hw/display/vhost-user-gpu-pci.c
@@ -44,6 +44,7 @@ static const VirtioPCIDeviceTypeInfo vhost_user_gpu_pci_info = {
     .instance_init = vhost_user_gpu_pci_initfn,
 };
 module_obj(TYPE_VHOST_USER_GPU_PCI);
+module_kconfig(VHOST_USER_GPU);
 
 static void vhost_user_gpu_pci_register_types(void)
 {
diff --git a/hw/display/vhost-user-gpu.c b/hw/display/vhost-user-gpu.c
index 96e56c4467..3340ef9e5f 100644
--- a/hw/display/vhost-user-gpu.c
+++ b/hw/display/vhost-user-gpu.c
@@ -606,6 +606,7 @@ static const TypeInfo vhost_user_gpu_info = {
     .class_init = vhost_user_gpu_class_init,
 };
 module_obj(TYPE_VHOST_USER_GPU);
+module_kconfig(VHOST_USER_GPU);
 
 static void vhost_user_gpu_register_types(void)
 {
diff --git a/hw/display/vhost-user-vga.c b/hw/display/vhost-user-vga.c
index 072c9c65bc..0c146080fd 100644
--- a/hw/display/vhost-user-vga.c
+++ b/hw/display/vhost-user-vga.c
@@ -45,6 +45,7 @@ static const VirtioPCIDeviceTypeInfo vhost_user_vga_info = {
     .instance_init = vhost_user_vga_inst_initfn,
 };
 module_obj(TYPE_VHOST_USER_VGA);
+module_kconfig(VHOST_USER_VGA);
 
 static void vhost_user_vga_register_types(void)
 {
diff --git a/hw/display/virtio-gpu-base.c b/hw/display/virtio-gpu-base.c
index 8ba5da4312..790cec333c 100644
--- a/hw/display/virtio-gpu-base.c
+++ b/hw/display/virtio-gpu-base.c
@@ -260,6 +260,7 @@ static const TypeInfo virtio_gpu_base_info = {
     .abstract = true
 };
 module_obj(TYPE_VIRTIO_GPU_BASE);
+module_kconfig(VIRTIO_GPU);
 
 static void
 virtio_register_types(void)
diff --git a/hw/display/virtio-gpu-gl.c b/hw/display/virtio-gpu-gl.c
index 0bca887703..e06be60dfb 100644
--- a/hw/display/virtio-gpu-gl.c
+++ b/hw/display/virtio-gpu-gl.c
@@ -160,6 +160,7 @@ static const TypeInfo virtio_gpu_gl_info = {
     .class_init = virtio_gpu_gl_class_init,
 };
 module_obj(TYPE_VIRTIO_GPU_GL);
+module_kconfig(VIRTIO_GPU);
 
 static void virtio_register_types(void)
 {
diff --git a/hw/display/virtio-gpu-pci-gl.c b/hw/display/virtio-gpu-pci-gl.c
index 99b14a0718..a2819e1ca9 100644
--- a/hw/display/virtio-gpu-pci-gl.c
+++ b/hw/display/virtio-gpu-pci-gl.c
@@ -47,6 +47,7 @@ static const VirtioPCIDeviceTypeInfo virtio_gpu_gl_pci_info = {
     .instance_init = virtio_gpu_gl_initfn,
 };
 module_obj(TYPE_VIRTIO_GPU_GL_PCI);
+module_kconfig(VIRTIO_PCI);
 
 static void virtio_gpu_gl_pci_register_types(void)
 {
diff --git a/hw/display/virtio-gpu-pci.c b/hw/display/virtio-gpu-pci.c
index e36eee0c40..93f214ff58 100644
--- a/hw/display/virtio-gpu-pci.c
+++ b/hw/display/virtio-gpu-pci.c
@@ -65,6 +65,7 @@ static const TypeInfo virtio_gpu_pci_base_info = {
     .abstract = true
 };
 module_obj(TYPE_VIRTIO_GPU_PCI_BASE);
+module_kconfig(VIRTIO_PCI);
 
 #define TYPE_VIRTIO_GPU_PCI "virtio-gpu-pci"
 typedef struct VirtIOGPUPCI VirtIOGPUPCI;
diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
index 529b5246b2..cd4a56056f 100644
--- a/hw/display/virtio-gpu.c
+++ b/hw/display/virtio-gpu.c
@@ -1452,6 +1452,7 @@ static const TypeInfo virtio_gpu_info = {
     .class_init = virtio_gpu_class_init,
 };
 module_obj(TYPE_VIRTIO_GPU);
+module_kconfig(VIRTIO_GPU);
 
 static void virtio_register_types(void)
 {
diff --git a/hw/display/virtio-vga-gl.c b/hw/display/virtio-vga-gl.c
index f22549097c..984faa6b39 100644
--- a/hw/display/virtio-vga-gl.c
+++ b/hw/display/virtio-vga-gl.c
@@ -37,6 +37,7 @@ static VirtioPCIDeviceTypeInfo virtio_vga_gl_info = {
     .instance_init = virtio_vga_gl_inst_initfn,
 };
 module_obj(TYPE_VIRTIO_VGA_GL);
+module_kconfig(VIRTIO_VGA);
 
 static void virtio_vga_register_types(void)
 {
diff --git a/hw/display/virtio-vga.c b/hw/display/virtio-vga.c
index 7b55c8d0e7..c206b5da38 100644
--- a/hw/display/virtio-vga.c
+++ b/hw/display/virtio-vga.c
@@ -231,6 +231,7 @@ static const TypeInfo virtio_vga_base_info = {
     .abstract      = true,
 };
 module_obj(TYPE_VIRTIO_VGA_BASE);
+module_kconfig(VIRTIO_VGA);
 
 #define TYPE_VIRTIO_VGA "virtio-vga"
 
diff --git a/hw/s390x/virtio-ccw-gpu.c b/hw/s390x/virtio-ccw-gpu.c
index 8d995fcb33..0642c5281d 100644
--- a/hw/s390x/virtio-ccw-gpu.c
+++ b/hw/s390x/virtio-ccw-gpu.c
@@ -69,6 +69,7 @@ static const TypeInfo virtio_ccw_gpu = {
     .class_init    = virtio_ccw_gpu_class_init,
 };
 module_obj(TYPE_VIRTIO_GPU_CCW);
+module_kconfig(VIRTIO_CCW);
 
 static void virtio_ccw_gpu_register(void)
 {
diff --git a/hw/usb/ccid-card-emulated.c b/hw/usb/ccid-card-emulated.c
index 6c8c0355e0..1ddf7297f6 100644
--- a/hw/usb/ccid-card-emulated.c
+++ b/hw/usb/ccid-card-emulated.c
@@ -613,6 +613,7 @@ static const TypeInfo emulated_card_info = {
     .class_init    = emulated_class_initfn,
 };
 module_obj(TYPE_EMULATED_CCID);
+module_kconfig(USB);
 
 static void ccid_card_emulated_register_types(void)
 {
diff --git a/hw/usb/ccid-card-passthru.c b/hw/usb/ccid-card-passthru.c
index f530ab2565..07ee42f304 100644
--- a/hw/usb/ccid-card-passthru.c
+++ b/hw/usb/ccid-card-passthru.c
@@ -415,6 +415,7 @@ static const TypeInfo passthru_card_info = {
     .class_init    = passthru_class_initfn,
 };
 module_obj(TYPE_CCID_PASSTHRU);
+module_kconfig(USB);
 
 static void ccid_card_passthru_register_types(void)
 {
diff --git a/hw/usb/host-libusb.c b/hw/usb/host-libusb.c
index 2b35cb6cdd..28f8af8941 100644
--- a/hw/usb/host-libusb.c
+++ b/hw/usb/host-libusb.c
@@ -1809,6 +1809,7 @@ static const TypeInfo usb_host_dev_info = {
     .instance_init = usb_host_instance_init,
 };
 module_obj(TYPE_USB_HOST_DEVICE);
+module_kconfig(USB);
 
 static void usb_host_register_types(void)
 {
diff --git a/hw/usb/redirect.c b/hw/usb/redirect.c
index 3bc4dee7fe..fd7df599bc 100644
--- a/hw/usb/redirect.c
+++ b/hw/usb/redirect.c
@@ -2620,6 +2620,7 @@ static const TypeInfo usbredir_dev_info = {
     .instance_init = usbredir_instance_init,
 };
 module_obj(TYPE_USB_REDIR);
+module_kconfig(USB);
 
 static void usbredir_register_types(void)
 {
diff --git a/include/qemu/module.h b/include/qemu/module.h
index 5fcc323b2a..bd73607104 100644
--- a/include/qemu/module.h
+++ b/include/qemu/module.h
@@ -135,6 +135,16 @@ void module_allow_arch(const char *arch);
  */
 #define module_opts(name) modinfo(opts, name)
 
+/**
+ * module_kconfig
+ *
+ * @name: Kconfig requirement necessary to load the module
+ *
+ * This module requires a core module that should be implemented and
+ * enabled in Kconfig.
+ */
+#define module_kconfig(name) modinfo(kconfig, name)
+
 /*
  * module info database
  *
diff --git a/scripts/modinfo-generate.py b/scripts/modinfo-generate.py
index f559eed007..689f33c0f2 100755
--- a/scripts/modinfo-generate.py
+++ b/scripts/modinfo-generate.py
@@ -48,6 +48,8 @@ def generate(name, lines):
                 opts.append(data)
             elif kind == 'arch':
                 arch = data;
+            elif kind == 'kconfig':
+                pass # ignore
             else:
                 print("unknown:", kind)
                 exit(1)

From 05d6814c3eb16524e992bb7048d3385f8e99dd6a Mon Sep 17 00:00:00 2001
From: "Jose R. Ziviani" <jziviani@suse.de>
Date: Sat, 28 May 2022 00:20:35 +0200
Subject: [PATCH 783/935] modules: generates per-target modinfo

This patch changes the way modinfo is generated and built. Instead of
one modinfo.c it generates one modinfo-<target>-softmmu.c per target. It
aims a fine-tune control of modules by configuring Kconfig.

Signed-off-by: Jose R. Ziviani <jziviani@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Message-Id: <165369003038.5857.13084289285185196779.stgit@work>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 meson.build                 | 25 +++++++++++++------
 scripts/modinfo-generate.py | 49 ++++++++++++++++++++++++++-----------
 2 files changed, 52 insertions(+), 22 deletions(-)

diff --git a/meson.build b/meson.build
index 9b3fd4e52b..d738391810 100644
--- a/meson.build
+++ b/meson.build
@@ -3175,14 +3175,23 @@ foreach d, list : target_modules
 endforeach
 
 if enable_modules
-  modinfo_src = custom_target('modinfo.c',
-                              output: 'modinfo.c',
-                              input: modinfo_files,
-                              command: [modinfo_generate, '@INPUT@'],
-                              capture: true)
-  modinfo_lib = static_library('modinfo', modinfo_src)
-  modinfo_dep = declare_dependency(link_whole: modinfo_lib)
-  softmmu_ss.add(modinfo_dep)
+  foreach target : target_dirs
+    if target.endswith('-softmmu')
+      config_target = config_target_mak[target]
+      config_devices_mak = target + '-config-devices.mak'
+      modinfo_src = custom_target('modinfo-' + target + '.c',
+                                  output: 'modinfo-' + target + '.c',
+                                  input: modinfo_files,
+                                  command: [modinfo_generate, '--devices', config_devices_mak, '@INPUT@'],
+                                  capture: true)
+
+      modinfo_lib = static_library('modinfo-' + target + '.c', modinfo_src)
+      modinfo_dep = declare_dependency(link_with: modinfo_lib)
+
+      arch = config_target['TARGET_NAME'] == 'sparc64' ? 'sparc64' : config_target['TARGET_BASE_ARCH']
+      hw_arch[arch].add(modinfo_dep)
+    endif
+  endforeach
 endif
 
 nm = find_program('nm')
diff --git a/scripts/modinfo-generate.py b/scripts/modinfo-generate.py
index 689f33c0f2..b1538fcced 100755
--- a/scripts/modinfo-generate.py
+++ b/scripts/modinfo-generate.py
@@ -32,7 +32,7 @@ def parse_line(line):
             continue
     return (kind, data)
 
-def generate(name, lines):
+def generate(name, lines, enabled):
     arch = ""
     objs = []
     deps = []
@@ -49,7 +49,13 @@ def generate(name, lines):
             elif kind == 'arch':
                 arch = data;
             elif kind == 'kconfig':
-                pass # ignore
+                # don't add a module which dependency is not enabled
+                # in kconfig
+                if data.strip() not in enabled:
+                    print("    /* module {} isn't enabled in Kconfig. */"
+                          .format(data.strip()))
+                    print("/* },{ */")
+                    return None
             else:
                 print("unknown:", kind)
                 exit(1)
@@ -60,8 +66,8 @@ def generate(name, lines):
     print_array("objs", objs)
     print_array("deps", deps)
     print_array("opts", opts)
-    print("},{");
-    return deps
+    print("},{")
+    return {dep.strip('" ') for dep in deps}
 
 def print_pre():
     print("/* generated by scripts/modinfo-generate.py */")
@@ -74,23 +80,38 @@ def print_post():
     print("}};")
 
 def main(args):
-    deps = {}
+    if len(args) < 3 or args[0] != '--devices':
+        print('Expected: modinfo-generate.py --devices '
+              'config-device.mak [modinfo files]', file=sys.stderr)
+        exit(1)
+
+    # get all devices enabled in kconfig, from *-config-device.mak
+    enabled = set()
+    with open(args[1]) as file:
+        for line in file.readlines():
+            config = line.split('=')
+            if config[1].rstrip() == 'y':
+                enabled.add(config[0][7:]) # remove CONFIG_
+
+    deps = set()
+    modules = set()
     print_pre()
-    for modinfo in args:
+    for modinfo in args[2:]:
         with open(modinfo) as f:
             lines = f.readlines()
         print("    /* %s */" % modinfo)
-        (basename, ext) = os.path.splitext(modinfo)
-        deps[basename] = generate(basename, lines)
+        (basename, _) = os.path.splitext(modinfo)
+        moddeps = generate(basename, lines, enabled)
+        if moddeps is not None:
+            modules.add(basename)
+            deps.update(moddeps)
     print_post()
 
-    flattened_deps = {flat.strip('" ') for dep in deps.values() for flat in dep}
     error = False
-    for dep in flattened_deps:
-        if dep not in deps.keys():
-            print("Dependency {} cannot be satisfied".format(dep),
-                  file=sys.stderr)
-            error = True
+    for dep in deps.difference(modules):
+        print("Dependency {} cannot be satisfied".format(dep),
+              file=sys.stderr)
+        error = True
 
     if error:
         exit(1)

From c4b8ffcbb8531206e12cf3ad92fa90f7c80ed464 Mon Sep 17 00:00:00 2001
From: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Date: Fri, 27 May 2022 13:46:07 +0300
Subject: [PATCH 784/935] replay: fix event queue flush for qemu shutdown

This patch fixes event queue flush in the case of emulator
shutdown. replay_finish_events should be called when replay_mode
is not cleared.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <165364836758.688121.7959245442743676491.stgit@pasha-ThinkPad-X280>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 replay/replay.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/replay/replay.c b/replay/replay.c
index 6df2abc18c..2d3607998a 100644
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -387,9 +387,8 @@ void replay_finish(void)
     g_free(replay_snapshot);
     replay_snapshot = NULL;
 
-    replay_mode = REPLAY_MODE_NONE;
-
     replay_finish_events();
+    replay_mode = REPLAY_MODE_NONE;
 }
 
 void replay_add_blocker(Error *reason)

From 75bbe5e5ec2867f098a31bfd553a1fb084289cc2 Mon Sep 17 00:00:00 2001
From: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Date: Fri, 27 May 2022 13:46:13 +0300
Subject: [PATCH 785/935] replay: notify vCPU when BH is scheduled

vCPU execution should be suspended when new BH is scheduled.
This is needed to avoid guest timeouts caused by the long cycles
of the execution. In replay mode execution may hang when
vCPU sleeps and block event comes to the queue.
This patch adds notification which wakes up vCPU or interrupts
execution of guest code.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

--

v2: changed first_cpu to current_cpu (suggested by Richard Henderson)
v4: moved vCPU notification to aio_bh_enqueue (suggested by Paolo Bonzini)
Message-Id: <165364837317.688121.17680519919871405281.stgit@pasha-ThinkPad-X280>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 include/sysemu/cpu-timers.h | 1 +
 softmmu/icount.c            | 8 ++++++++
 stubs/icount.c              | 4 ++++
 util/async.c                | 8 ++++++++
 4 files changed, 21 insertions(+)

diff --git a/include/sysemu/cpu-timers.h b/include/sysemu/cpu-timers.h
index ed6ee5c46c..2e786fe7fb 100644
--- a/include/sysemu/cpu-timers.h
+++ b/include/sysemu/cpu-timers.h
@@ -59,6 +59,7 @@ int64_t icount_round(int64_t count);
 /* if the CPUs are idle, start accounting real time to virtual clock. */
 void icount_start_warp_timer(void);
 void icount_account_warp_timer(void);
+void icount_notify_exit(void);
 
 /*
  * CPU Ticks and Clock
diff --git a/softmmu/icount.c b/softmmu/icount.c
index 5ca271620d..1cafec5014 100644
--- a/softmmu/icount.c
+++ b/softmmu/icount.c
@@ -486,3 +486,11 @@ void icount_configure(QemuOpts *opts, Error **errp)
                    qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) +
                    NANOSECONDS_PER_SECOND / 10);
 }
+
+void icount_notify_exit(void)
+{
+    if (icount_enabled() && current_cpu) {
+        qemu_cpu_kick(current_cpu);
+        qemu_clock_notify(QEMU_CLOCK_VIRTUAL);
+    }
+}
diff --git a/stubs/icount.c b/stubs/icount.c
index f13c43568b..6df8c2bf7d 100644
--- a/stubs/icount.c
+++ b/stubs/icount.c
@@ -43,3 +43,7 @@ void icount_account_warp_timer(void)
 {
     abort();
 }
+
+void icount_notify_exit(void)
+{
+}
diff --git a/util/async.c b/util/async.c
index 554ba70cca..63434ddae4 100644
--- a/util/async.c
+++ b/util/async.c
@@ -33,6 +33,7 @@
 #include "block/raw-aio.h"
 #include "qemu/coroutine_int.h"
 #include "qemu/coroutine-tls.h"
+#include "sysemu/cpu-timers.h"
 #include "trace.h"
 
 /***********************************************************/
@@ -84,6 +85,13 @@ static void aio_bh_enqueue(QEMUBH *bh, unsigned new_flags)
     }
 
     aio_notify(ctx);
+    /*
+     * Workaround for record/replay.
+     * vCPU execution should be suspended when new BH is set.
+     * This is needed to avoid guest timeouts caused
+     * by the long cycles of the execution.
+     */
+    icount_notify_exit();
 }
 
 /* Only called from aio_bh_poll() and aio_ctx_finalize() */

From 60618e2d77691e44bb78e23b2b0cf07b5c405e56 Mon Sep 17 00:00:00 2001
From: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Date: Fri, 27 May 2022 13:46:18 +0300
Subject: [PATCH 786/935] replay: rewrite async event handling

This patch decouples checkpoints and async events.
It was a tricky part of replay implementation. Now it becomes
much simpler and easier to maintain.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <165364837856.688121.8785039478408995979.stgit@pasha-ThinkPad-X280>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 accel/tcg/tcg-accel-ops-icount.c |  5 +--
 docs/replay.txt                  | 11 ++---
 include/sysemu/replay.h          |  9 +++-
 replay/replay-events.c           | 20 +++------
 replay/replay-internal.h         |  6 +--
 replay/replay-snapshot.c         |  1 -
 replay/replay.c                  | 74 +++++++++++++-------------------
 softmmu/icount.c                 |  4 +-
 8 files changed, 54 insertions(+), 76 deletions(-)

diff --git a/accel/tcg/tcg-accel-ops-icount.c b/accel/tcg/tcg-accel-ops-icount.c
index 24520ea112..8f1dda4344 100644
--- a/accel/tcg/tcg-accel-ops-icount.c
+++ b/accel/tcg/tcg-accel-ops-icount.c
@@ -84,8 +84,7 @@ void icount_handle_deadline(void)
      * Don't interrupt cpu thread, when these events are waiting
      * (i.e., there is no checkpoint)
      */
-    if (deadline == 0
-        && (replay_mode != REPLAY_MODE_PLAY || replay_has_checkpoint())) {
+    if (deadline == 0) {
         icount_notify_aio_contexts();
     }
 }
@@ -109,7 +108,7 @@ void icount_prepare_for_run(CPUState *cpu)
 
     replay_mutex_lock();
 
-    if (cpu->icount_budget == 0 && replay_has_checkpoint()) {
+    if (cpu->icount_budget == 0) {
         icount_notify_aio_contexts();
     }
 }
diff --git a/docs/replay.txt b/docs/replay.txt
index 5b008ca491..6c9fdff09d 100644
--- a/docs/replay.txt
+++ b/docs/replay.txt
@@ -366,11 +366,9 @@ Here is the list of events that are written into the log:
    Argument: 4-byte number of executed instructions.
  - EVENT_INTERRUPT. Used to synchronize interrupt processing.
  - EVENT_EXCEPTION. Used to synchronize exception handling.
- - EVENT_ASYNC. This is a group of events. They are always processed
-   together with checkpoints. When such an event is generated, it is
-   stored in the queue and processed only when checkpoint occurs.
-   Every such event is followed by 1-byte checkpoint id and 1-byte
-   async event id from the following list:
+ - EVENT_ASYNC. This is a group of events. When such an event is generated,
+   it is stored in the queue and processed in icount_account_warp_timer().
+   Every such event has it's own id from the following list:
      - REPLAY_ASYNC_EVENT_BH. Bottom-half callback. This event synchronizes
        callbacks that affect virtual machine state, but normally called
        asynchronously.
@@ -405,6 +403,5 @@ Here is the list of events that are written into the log:
  - EVENT_CLOCK + clock_id. Group of events for host clock read operations.
    Argument: 8-byte clock value.
  - EVENT_CHECKPOINT + checkpoint_id. Checkpoint for synchronization of
-   CPU, internal threads, and asynchronous input events. May be followed
-   by one or more EVENT_ASYNC events.
+   CPU, internal threads, and asynchronous input events.
  - EVENT_END. Last event in the log.
diff --git a/include/sysemu/replay.h b/include/sysemu/replay.h
index 0f3b0f7eac..73dee9ccdf 100644
--- a/include/sysemu/replay.h
+++ b/include/sysemu/replay.h
@@ -160,9 +160,14 @@ void replay_shutdown_request(ShutdownCause cause);
     Returns 0 in PLAY mode if checkpoint was not found.
     Returns 1 in all other cases. */
 bool replay_checkpoint(ReplayCheckpoint checkpoint);
-/*! Used to determine that checkpoint is pending.
+/*! Used to determine that checkpoint or async event is pending.
     Does not proceed to the next event in the log. */
-bool replay_has_checkpoint(void);
+bool replay_has_event(void);
+/*
+ * Processes the async events added to the queue (while recording)
+ * or reads the events from the file (while replaying).
+ */
+void replay_async_events(void);
 
 /* Asynchronous events queue */
 
diff --git a/replay/replay-events.c b/replay/replay-events.c
index ac47c89834..db1decf9dd 100644
--- a/replay/replay-events.c
+++ b/replay/replay-events.c
@@ -170,12 +170,11 @@ void replay_block_event(QEMUBH *bh, uint64_t id)
     }
 }
 
-static void replay_save_event(Event *event, int checkpoint)
+static void replay_save_event(Event *event)
 {
     if (replay_mode != REPLAY_MODE_PLAY) {
         /* put the event into the file */
         replay_put_event(EVENT_ASYNC);
-        replay_put_byte(checkpoint);
         replay_put_byte(event->event_kind);
 
         /* save event-specific data */
@@ -206,34 +205,27 @@ static void replay_save_event(Event *event, int checkpoint)
 }
 
 /* Called with replay mutex locked */
-void replay_save_events(int checkpoint)
+void replay_save_events(void)
 {
     g_assert(replay_mutex_locked());
-    g_assert(checkpoint != CHECKPOINT_CLOCK_WARP_START);
-    g_assert(checkpoint != CHECKPOINT_CLOCK_VIRTUAL);
     while (!QTAILQ_EMPTY(&events_list)) {
         Event *event = QTAILQ_FIRST(&events_list);
-        replay_save_event(event, checkpoint);
+        replay_save_event(event);
         replay_run_event(event);
         QTAILQ_REMOVE(&events_list, event, events);
         g_free(event);
     }
 }
 
-static Event *replay_read_event(int checkpoint)
+static Event *replay_read_event(void)
 {
     Event *event;
     if (replay_state.read_event_kind == -1) {
-        replay_state.read_event_checkpoint = replay_get_byte();
         replay_state.read_event_kind = replay_get_byte();
         replay_state.read_event_id = -1;
         replay_check_error();
     }
 
-    if (checkpoint != replay_state.read_event_checkpoint) {
-        return NULL;
-    }
-
     /* Events that has not to be in the queue */
     switch (replay_state.read_event_kind) {
     case REPLAY_ASYNC_EVENT_BH:
@@ -294,11 +286,11 @@ static Event *replay_read_event(int checkpoint)
 }
 
 /* Called with replay mutex locked */
-void replay_read_events(int checkpoint)
+void replay_read_events(void)
 {
     g_assert(replay_mutex_locked());
     while (replay_state.data_kind == EVENT_ASYNC) {
-        Event *event = replay_read_event(checkpoint);
+        Event *event = replay_read_event();
         if (!event) {
             break;
         }
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 97649ed8d7..d6e631a394 100644
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -87,8 +87,6 @@ typedef struct ReplayState {
     int32_t read_event_kind;
     /*! Asynchronous event id read from the log */
     uint64_t read_event_id;
-    /*! Asynchronous event checkpoint id read from the log */
-    int32_t read_event_checkpoint;
 } ReplayState;
 extern ReplayState replay_state;
 
@@ -152,9 +150,9 @@ void replay_finish_events(void);
 /*! Returns true if there are any unsaved events in the queue */
 bool replay_has_events(void);
 /*! Saves events from queue into the file */
-void replay_save_events(int checkpoint);
+void replay_save_events(void);
 /*! Read events from the file into the input queue */
-void replay_read_events(int checkpoint);
+void replay_read_events(void);
 /*! Adds specified async event to the queue */
 void replay_add_event(ReplayAsyncEventKind event_kind, void *opaque,
                       void *opaque2, uint64_t id);
diff --git a/replay/replay-snapshot.c b/replay/replay-snapshot.c
index e8767a1937..7e935deb15 100644
--- a/replay/replay-snapshot.c
+++ b/replay/replay-snapshot.c
@@ -61,7 +61,6 @@ static const VMStateDescription vmstate_replay = {
         VMSTATE_UINT64(block_request_id, ReplayState),
         VMSTATE_INT32(read_event_kind, ReplayState),
         VMSTATE_UINT64(read_event_id, ReplayState),
-        VMSTATE_INT32(read_event_checkpoint, ReplayState),
         VMSTATE_END_OF_LIST()
     },
 };
diff --git a/replay/replay.c b/replay/replay.c
index 2d3607998a..ccd7edec76 100644
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -22,7 +22,7 @@
 
 /* Current version of the replay mechanism.
    Increase it when file format changes. */
-#define REPLAY_VERSION              0xe0200a
+#define REPLAY_VERSION              0xe0200b
 /* Size of replay log header */
 #define HEADER_SIZE                 (sizeof(uint32_t) + sizeof(uint64_t))
 
@@ -171,64 +171,49 @@ void replay_shutdown_request(ShutdownCause cause)
 
 bool replay_checkpoint(ReplayCheckpoint checkpoint)
 {
-    bool res = false;
-    static bool in_checkpoint;
     assert(EVENT_CHECKPOINT + checkpoint <= EVENT_CHECKPOINT_LAST);
 
-    if (!replay_file) {
-        return true;
-    }
-
-    if (in_checkpoint) {
-        /*
-           Recursion occurs when HW event modifies timers.
-           Prevent performing icount warp in this case and
-           wait for another invocation of the checkpoint.
-        */
-        g_assert(replay_mode == REPLAY_MODE_PLAY);
-        return false;
-    }
-    in_checkpoint = true;
-
     replay_save_instructions();
 
     if (replay_mode == REPLAY_MODE_PLAY) {
         g_assert(replay_mutex_locked());
         if (replay_next_event_is(EVENT_CHECKPOINT + checkpoint)) {
             replay_finish_event();
-        } else if (replay_state.data_kind != EVENT_ASYNC) {
-            res = false;
-            goto out;
+        } else {
+            return false;
         }
-        replay_read_events(checkpoint);
-        /* replay_read_events may leave some unread events.
-           Return false if not all of the events associated with
-           checkpoint were processed */
-        res = replay_state.data_kind != EVENT_ASYNC;
     } else if (replay_mode == REPLAY_MODE_RECORD) {
         g_assert(replay_mutex_locked());
         replay_put_event(EVENT_CHECKPOINT + checkpoint);
-        /* This checkpoint belongs to several threads.
-           Processing events from different threads is
-           non-deterministic */
-        if (checkpoint != CHECKPOINT_CLOCK_WARP_START
-            /* FIXME: this is temporary fix, other checkpoints
-                      may also be invoked from the different threads someday.
-                      Asynchronous event processing should be refactored
-                      to create additional replay event kind which is
-                      nailed to the one of the threads and which processes
-                      the event queue. */
-            && checkpoint != CHECKPOINT_CLOCK_VIRTUAL) {
-            replay_save_events(checkpoint);
-        }
-        res = true;
     }
-out:
-    in_checkpoint = false;
-    return res;
+    return true;
 }
 
-bool replay_has_checkpoint(void)
+void replay_async_events(void)
+{
+    static bool processing = false;
+    /*
+     * If we are already processing the events, recursion may occur
+     * in case of incorrect implementation when HW event modifies timers.
+     * Timer modification may invoke the icount warp, event processing,
+     * and cause the recursion.
+     */
+    g_assert(!processing);
+    processing = true;
+
+    replay_save_instructions();
+
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        g_assert(replay_mutex_locked());
+        replay_read_events();
+    } else if (replay_mode == REPLAY_MODE_RECORD) {
+        g_assert(replay_mutex_locked());
+        replay_save_events();
+    }
+    processing = false;
+}
+
+bool replay_has_event(void)
 {
     bool res = false;
     if (replay_mode == REPLAY_MODE_PLAY) {
@@ -236,6 +221,7 @@ bool replay_has_checkpoint(void)
         replay_account_executed_instructions();
         res = EVENT_CHECKPOINT <= replay_state.data_kind
               && replay_state.data_kind <= EVENT_CHECKPOINT_LAST;
+        res = res || replay_state.data_kind == EVENT_ASYNC;
     }
     return res;
 }
diff --git a/softmmu/icount.c b/softmmu/icount.c
index 1cafec5014..4504433e16 100644
--- a/softmmu/icount.c
+++ b/softmmu/icount.c
@@ -322,7 +322,7 @@ void icount_start_warp_timer(void)
              * to vCPU was processed in advance and vCPU went to sleep.
              * Therefore we have to wake it up for doing someting.
              */
-            if (replay_has_checkpoint()) {
+            if (replay_has_event()) {
                 qemu_clock_notify(QEMU_CLOCK_VIRTUAL);
             }
             return;
@@ -404,6 +404,8 @@ void icount_account_warp_timer(void)
         return;
     }
 
+    replay_async_events();
+
     /* warp clock deterministically in record/replay mode */
     if (!replay_checkpoint(CHECKPOINT_CLOCK_WARP_ACCOUNT)) {
         return;

From 3e21408b0596aa2babe1e26996a15e7ad5eee71a Mon Sep 17 00:00:00 2001
From: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Date: Fri, 27 May 2022 13:46:23 +0300
Subject: [PATCH 787/935] replay: simplify async event processing

This patch joins replay event id and async event id into single byte in the log.
It makes processing a bit faster and log a bit smaller.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

--

v2: minor enum fixes (suggested by Richard Henderson)
Message-Id: <165364838393.688121.8191379555130516329.stgit@pasha-ThinkPad-X280>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 replay/replay-events.c   | 36 ++++++++++++++----------------------
 replay/replay-internal.h | 31 ++++++++++++++-----------------
 replay/replay-snapshot.c |  1 -
 replay/replay.c          |  5 +++--
 4 files changed, 31 insertions(+), 42 deletions(-)

diff --git a/replay/replay-events.c b/replay/replay-events.c
index db1decf9dd..af0721cc1a 100644
--- a/replay/replay-events.c
+++ b/replay/replay-events.c
@@ -174,8 +174,8 @@ static void replay_save_event(Event *event)
 {
     if (replay_mode != REPLAY_MODE_PLAY) {
         /* put the event into the file */
-        replay_put_event(EVENT_ASYNC);
-        replay_put_byte(event->event_kind);
+        g_assert(event->event_kind < REPLAY_ASYNC_COUNT);
+        replay_put_event(EVENT_ASYNC + event->event_kind);
 
         /* save event-specific data */
         switch (event->event_kind) {
@@ -220,14 +220,10 @@ void replay_save_events(void)
 static Event *replay_read_event(void)
 {
     Event *event;
-    if (replay_state.read_event_kind == -1) {
-        replay_state.read_event_kind = replay_get_byte();
-        replay_state.read_event_id = -1;
-        replay_check_error();
-    }
+    ReplayAsyncEventKind event_kind = replay_state.data_kind - EVENT_ASYNC;
 
     /* Events that has not to be in the queue */
-    switch (replay_state.read_event_kind) {
+    switch (event_kind) {
     case REPLAY_ASYNC_EVENT_BH:
     case REPLAY_ASYNC_EVENT_BH_ONESHOT:
         if (replay_state.read_event_id == -1) {
@@ -236,17 +232,17 @@ static Event *replay_read_event(void)
         break;
     case REPLAY_ASYNC_EVENT_INPUT:
         event = g_new0(Event, 1);
-        event->event_kind = replay_state.read_event_kind;
+        event->event_kind = event_kind;
         event->opaque = replay_read_input_event();
         return event;
     case REPLAY_ASYNC_EVENT_INPUT_SYNC:
         event = g_new0(Event, 1);
-        event->event_kind = replay_state.read_event_kind;
+        event->event_kind = event_kind;
         event->opaque = 0;
         return event;
     case REPLAY_ASYNC_EVENT_CHAR_READ:
         event = g_new0(Event, 1);
-        event->event_kind = replay_state.read_event_kind;
+        event->event_kind = event_kind;
         event->opaque = replay_event_char_read_load();
         return event;
     case REPLAY_ASYNC_EVENT_BLOCK:
@@ -256,18 +252,17 @@ static Event *replay_read_event(void)
         break;
     case REPLAY_ASYNC_EVENT_NET:
         event = g_new0(Event, 1);
-        event->event_kind = replay_state.read_event_kind;
+        event->event_kind = event_kind;
         event->opaque = replay_event_net_load();
         return event;
     default:
-        error_report("Unknown ID %d of replay event",
-            replay_state.read_event_kind);
+        error_report("Unknown ID %d of replay event", event_kind);
         exit(1);
         break;
     }
 
     QTAILQ_FOREACH(event, &events_list, events) {
-        if (event->event_kind == replay_state.read_event_kind
+        if (event->event_kind == event_kind
             && (replay_state.read_event_id == -1
                 || replay_state.read_event_id == event->id)) {
             break;
@@ -276,12 +271,8 @@ static Event *replay_read_event(void)
 
     if (event) {
         QTAILQ_REMOVE(&events_list, event, events);
-    } else {
-        return NULL;
     }
 
-    /* Read event-specific data */
-
     return event;
 }
 
@@ -289,13 +280,14 @@ static Event *replay_read_event(void)
 void replay_read_events(void)
 {
     g_assert(replay_mutex_locked());
-    while (replay_state.data_kind == EVENT_ASYNC) {
+    while (replay_state.data_kind >= EVENT_ASYNC
+        && replay_state.data_kind <= EVENT_ASYNC_LAST) {
         Event *event = replay_read_event();
         if (!event) {
             break;
         }
         replay_finish_event();
-        replay_state.read_event_kind = -1;
+        replay_state.read_event_id = -1;
         replay_run_event(event);
 
         g_free(event);
@@ -304,7 +296,7 @@ void replay_read_events(void)
 
 void replay_init_events(void)
 {
-    replay_state.read_event_kind = -1;
+    replay_state.read_event_id = -1;
 }
 
 void replay_finish_events(void)
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index d6e631a394..89e377be90 100644
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -12,6 +12,19 @@
  *
  */
 
+/* Asynchronous events IDs */
+
+typedef enum ReplayAsyncEventKind {
+    REPLAY_ASYNC_EVENT_BH,
+    REPLAY_ASYNC_EVENT_BH_ONESHOT,
+    REPLAY_ASYNC_EVENT_INPUT,
+    REPLAY_ASYNC_EVENT_INPUT_SYNC,
+    REPLAY_ASYNC_EVENT_CHAR_READ,
+    REPLAY_ASYNC_EVENT_BLOCK,
+    REPLAY_ASYNC_EVENT_NET,
+    REPLAY_ASYNC_COUNT
+} ReplayAsyncEventKind;
+
 /* Any changes to order/number of events will need to bump REPLAY_VERSION */
 enum ReplayEvents {
     /* for instruction event */
@@ -22,6 +35,7 @@ enum ReplayEvents {
     EVENT_EXCEPTION,
     /* for async events */
     EVENT_ASYNC,
+    EVENT_ASYNC_LAST = EVENT_ASYNC + REPLAY_ASYNC_COUNT - 1,
     /* for shutdown requests, range allows recovery of ShutdownCause */
     EVENT_SHUTDOWN,
     EVENT_SHUTDOWN_LAST = EVENT_SHUTDOWN + SHUTDOWN_CAUSE__MAX,
@@ -49,21 +63,6 @@ enum ReplayEvents {
     EVENT_COUNT
 };
 
-/* Asynchronous events IDs */
-
-enum ReplayAsyncEventKind {
-    REPLAY_ASYNC_EVENT_BH,
-    REPLAY_ASYNC_EVENT_BH_ONESHOT,
-    REPLAY_ASYNC_EVENT_INPUT,
-    REPLAY_ASYNC_EVENT_INPUT_SYNC,
-    REPLAY_ASYNC_EVENT_CHAR_READ,
-    REPLAY_ASYNC_EVENT_BLOCK,
-    REPLAY_ASYNC_EVENT_NET,
-    REPLAY_ASYNC_COUNT
-};
-
-typedef enum ReplayAsyncEventKind ReplayAsyncEventKind;
-
 typedef struct ReplayState {
     /*! Cached clock values. */
     int64_t cached_clock[REPLAY_CLOCK_COUNT];
@@ -83,8 +82,6 @@ typedef struct ReplayState {
     uint64_t block_request_id;
     /*! Prior value of the host clock */
     uint64_t host_clock_last;
-    /*! Asynchronous event type read from the log */
-    int32_t read_event_kind;
     /*! Asynchronous event id read from the log */
     uint64_t read_event_id;
 } ReplayState;
diff --git a/replay/replay-snapshot.c b/replay/replay-snapshot.c
index 7e935deb15..10a7cf7992 100644
--- a/replay/replay-snapshot.c
+++ b/replay/replay-snapshot.c
@@ -59,7 +59,6 @@ static const VMStateDescription vmstate_replay = {
         VMSTATE_UINT32(has_unread_data, ReplayState),
         VMSTATE_UINT64(file_offset, ReplayState),
         VMSTATE_UINT64(block_request_id, ReplayState),
-        VMSTATE_INT32(read_event_kind, ReplayState),
         VMSTATE_UINT64(read_event_id, ReplayState),
         VMSTATE_END_OF_LIST()
     },
diff --git a/replay/replay.c b/replay/replay.c
index ccd7edec76..4c396bb376 100644
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -22,7 +22,7 @@
 
 /* Current version of the replay mechanism.
    Increase it when file format changes. */
-#define REPLAY_VERSION              0xe0200b
+#define REPLAY_VERSION              0xe0200c
 /* Size of replay log header */
 #define HEADER_SIZE                 (sizeof(uint32_t) + sizeof(uint64_t))
 
@@ -221,7 +221,8 @@ bool replay_has_event(void)
         replay_account_executed_instructions();
         res = EVENT_CHECKPOINT <= replay_state.data_kind
               && replay_state.data_kind <= EVENT_CHECKPOINT_LAST;
-        res = res || replay_state.data_kind == EVENT_ASYNC;
+        res = res || (EVENT_ASYNC <= replay_state.data_kind
+                     && replay_state.data_kind <= EVENT_ASYNC_LAST);
     }
     return res;
 }

From 04d0583a4f5d00061bf57d17947aa0d5c6a6cecf Mon Sep 17 00:00:00 2001
From: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Date: Fri, 27 May 2022 13:46:30 +0300
Subject: [PATCH 788/935] docs: convert docs/devel/replay page to rst

This patch converts prior .txt replay devel documentation to .rst.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <165364839013.688121.11935249420738873044.stgit@pasha-ThinkPad-X280>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/devel/index-tcg.rst              |  1 +
 docs/devel/{replay.txt => replay.rst} | 16 ++++++++++++----
 2 files changed, 13 insertions(+), 4 deletions(-)
 rename docs/devel/{replay.txt => replay.rst} (90%)

diff --git a/docs/devel/index-tcg.rst b/docs/devel/index-tcg.rst
index 0b0ad12c22..7b9760b26f 100644
--- a/docs/devel/index-tcg.rst
+++ b/docs/devel/index-tcg.rst
@@ -13,3 +13,4 @@ are only implementing things for HW accelerated hypervisors.
    multi-thread-tcg
    tcg-icount
    tcg-plugins
+   replay
diff --git a/docs/devel/replay.txt b/docs/devel/replay.rst
similarity index 90%
rename from docs/devel/replay.txt
rename to docs/devel/replay.rst
index e641c35add..dd8bf3b195 100644
--- a/docs/devel/replay.txt
+++ b/docs/devel/replay.rst
@@ -1,3 +1,11 @@
+..
+   Copyright (c) 2022, ISP RAS
+   Written by Pavel Dovgalyuk
+
+=======================
+Execution Record/Replay
+=======================
+
 Record/replay mechanism, that could be enabled through icount mode, expects
 the virtual devices to satisfy the following requirements.
 
@@ -5,7 +13,7 @@ The main idea behind this document is that everything that affects
 the guest state during execution in icount mode should be deterministic.
 
 Timers
-======
+------
 
 All virtual devices should use virtual clock for timers that change the guest
 state. Virtual clock is deterministic, therefore such timers are deterministic
@@ -19,7 +27,7 @@ the virtual devices (e.g., slirp routing device) that lie outside the
 replayed guest.
 
 Bottom halves
-=============
+-------------
 
 Bottom half callbacks, that affect the guest state, should be invoked through
 replay_bh_schedule_event or replay_bh_schedule_oneshot_event functions.
@@ -27,7 +35,7 @@ Their invocations are saved in record mode and synchronized with the existing
 log in replay mode.
 
 Saving/restoring the VM state
-=============================
+-----------------------------
 
 All fields in the device state structure (including virtual timers)
 should be restored by loadvm to the same values they had before savevm.
@@ -38,7 +46,7 @@ is not defined. It means that you should not call functions like
 the dependencies that may make restoring the VM state non-deterministic.
 
 Stopping the VM
-===============
+---------------
 
 Stopping the guest should not interfere with its state (with the exception
 of the network connections, that could be broken by the remote timeouts).

From 43185f7bd4cbe730e7d5a6abcefbf83a5c98f3db Mon Sep 17 00:00:00 2001
From: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Date: Fri, 27 May 2022 13:46:36 +0300
Subject: [PATCH 789/935] docs: move replay docs to docs/system/replay.rst

This patch adds replay description page, converting prior
text from docs/replay.txt.
The text was also updated and some sections were moved
to devel part of the docs.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <165364839601.688121.5131456980322853233.stgit@pasha-ThinkPad-X280>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/devel/replay.rst  | 264 +++++++++++++++++++++++++-
 docs/replay.txt        | 407 -----------------------------------------
 docs/system/index.rst  |   1 +
 docs/system/replay.rst | 237 ++++++++++++++++++++++++
 4 files changed, 496 insertions(+), 413 deletions(-)
 delete mode 100644 docs/replay.txt
 create mode 100644 docs/system/replay.rst

diff --git a/docs/devel/replay.rst b/docs/devel/replay.rst
index dd8bf3b195..0244be8b9c 100644
--- a/docs/devel/replay.rst
+++ b/docs/devel/replay.rst
@@ -1,20 +1,149 @@
 ..
    Copyright (c) 2022, ISP RAS
-   Written by Pavel Dovgalyuk
+   Written by Pavel Dovgalyuk and Alex Bennée
 
 =======================
 Execution Record/Replay
 =======================
 
-Record/replay mechanism, that could be enabled through icount mode, expects
-the virtual devices to satisfy the following requirements.
+Core concepts
+=============
 
-The main idea behind this document is that everything that affects
+Record/replay functions are used for the deterministic replay of qemu
+execution. Execution recording writes a non-deterministic events log, which
+can be later used for replaying the execution anywhere and for unlimited
+number of times. Execution replaying reads the log and replays all
+non-deterministic events including external input, hardware clocks,
+and interrupts.
+
+Several parts of QEMU include function calls to make event log recording
+and replaying.
+Devices' models that have non-deterministic input from external devices were
+changed to write every external event into the execution log immediately.
+E.g. network packets are written into the log when they arrive into the virtual
+network adapter.
+
+All non-deterministic events are coming from these devices. But to
+replay them we need to know at which moments they occur. We specify
+these moments by counting the number of instructions executed between
+every pair of consecutive events.
+
+Academic papers with description of deterministic replay implementation:
+
+* `Deterministic Replay of System's Execution with Multi-target QEMU Simulator for Dynamic Analysis and Reverse Debugging <https://www.computer.org/csdl/proceedings/csmr/2012/4666/00/4666a553-abs.html>`_
+* `Don't panic: reverse debugging of kernel drivers <https://dl.acm.org/citation.cfm?id=2786805.2803179>`_
+
+Modifications of qemu include:
+
+ * wrappers for clock and time functions to save their return values in the log
+ * saving different asynchronous events (e.g. system shutdown) into the log
+ * synchronization of the bottom halves execution
+ * synchronization of the threads from thread pool
+ * recording/replaying user input (mouse, keyboard, and microphone)
+ * adding internal checkpoints for cpu and io synchronization
+ * network filter for recording and replaying the packets
+ * block driver for making block layer deterministic
+ * serial port input record and replay
+ * recording of random numbers obtained from the external sources
+
+Instruction counting
+--------------------
+
+QEMU should work in icount mode to use record/replay feature. icount was
+designed to allow deterministic execution in absence of external inputs
+of the virtual machine. We also use icount to control the occurrence of the
+non-deterministic events. The number of instructions elapsed from the last event
+is written to the log while recording the execution. In replay mode we
+can predict when to inject that event using the instruction counter.
+
+Locking and thread synchronisation
+----------------------------------
+
+Previously the synchronisation of the main thread and the vCPU thread
+was ensured by the holding of the BQL. However the trend has been to
+reduce the time the BQL was held across the system including under TCG
+system emulation. As it is important that batches of events are kept
+in sequence (e.g. expiring timers and checkpoints in the main thread
+while instruction checkpoints are written by the vCPU thread) we need
+another lock to keep things in lock-step. This role is now handled by
+the replay_mutex_lock. It used to be held only for each event being
+written but now it is held for a whole execution period. This results
+in a deterministic ping-pong between the two main threads.
+
+As the BQL is now a finer grained lock than the replay_lock it is almost
+certainly a bug, and a source of deadlocks, to take the
+replay_mutex_lock while the BQL is held. This is enforced by an assert.
+While the unlocks are usually in the reverse order, this is not
+necessary; you can drop the replay_lock while holding the BQL, without
+doing a more complicated unlock_iothread/replay_unlock/lock_iothread
+sequence.
+
+Checkpoints
+-----------
+
+Replaying the execution of virtual machine is bound by sources of
+non-determinism. These are inputs from clock and peripheral devices,
+and QEMU thread scheduling. Thread scheduling affect on processing events
+from timers, asynchronous input-output, and bottom halves.
+
+Invocations of timers are coupled with clock reads and changing the state
+of the virtual machine. Reads produce non-deterministic data taken from
+host clock. And VM state changes should preserve their order. Their relative
+order in replay mode must replicate the order of callbacks in record mode.
+To preserve this order we use checkpoints. When a specific clock is processed
+in record mode we save to the log special "checkpoint" event.
+Checkpoints here do not refer to virtual machine snapshots. They are just
+record/replay events used for synchronization.
+
+QEMU in replay mode will try to invoke timers processing in random moment
+of time. That's why we do not process a group of timers until the checkpoint
+event will be read from the log. Such an event allows synchronizing CPU
+execution and timer events.
+
+Two other checkpoints govern the "warping" of the virtual clock.
+While the virtual machine is idle, the virtual clock increments at
+1 ns per *real time* nanosecond.  This is done by setting up a timer
+(called the warp timer) on the virtual real time clock, so that the
+timer fires at the next deadline of the virtual clock; the virtual clock
+is then incremented (which is called "warping" the virtual clock) as
+soon as the timer fires or the CPUs need to go out of the idle state.
+Two functions are used for this purpose; because these actions change
+virtual machine state and must be deterministic, each of them creates a
+checkpoint. ``icount_start_warp_timer`` checks if the CPUs are idle and if so
+starts accounting real time to virtual clock. ``icount_account_warp_timer``
+is called when the CPUs get an interrupt or when the warp timer fires,
+and it warps the virtual clock by the amount of real time that has passed
+since ``icount_start_warp_timer``.
+
+Virtual devices
+===============
+
+Record/replay mechanism, that could be enabled through icount mode, expects
+the virtual devices to satisfy the following requirement:
+everything that affects
 the guest state during execution in icount mode should be deterministic.
 
 Timers
 ------
 
+Timers are used to execute callbacks from different subsystems of QEMU
+at the specified moments of time. There are several kinds of timers:
+
+ * Real time clock. Based on host time and used only for callbacks that
+   do not change the virtual machine state. For this reason real time
+   clock and timers does not affect deterministic replay at all.
+ * Virtual clock. These timers run only during the emulation. In icount
+   mode virtual clock value is calculated using executed instructions counter.
+   That is why it is completely deterministic and does not have to be recorded.
+ * Host clock. This clock is used by device models that simulate real time
+   sources (e.g. real time clock chip). Host clock is the one of the sources
+   of non-determinism. Host clock read operations should be logged to
+   make the execution deterministic.
+ * Virtual real time clock. This clock is similar to real time clock but
+   it is used only for increasing virtual clock while virtual machine is
+   sleeping. Due to its nature it is also non-deterministic as the host clock
+   and has to be logged too.
+
 All virtual devices should use virtual clock for timers that change the guest
 state. Virtual clock is deterministic, therefore such timers are deterministic
 too.
@@ -26,14 +155,50 @@ but its speed depends on the guest execution. This clock is used by
 the virtual devices (e.g., slirp routing device) that lie outside the
 replayed guest.
 
+Block devices
+-------------
+
+Block devices record/replay module (``blkreplay``) intercepts calls of
+bdrv coroutine functions at the top of block drivers stack.
+
+All block completion operations are added to the queue in the coroutines.
+When the queue is flushed the information about processed requests
+is recorded to the log. In replay phase the queue is matched with
+events read from the log. Therefore block devices requests are processed
+deterministically.
+
 Bottom halves
 -------------
 
 Bottom half callbacks, that affect the guest state, should be invoked through
-replay_bh_schedule_event or replay_bh_schedule_oneshot_event functions.
+``replay_bh_schedule_event`` or ``replay_bh_schedule_oneshot_event`` functions.
 Their invocations are saved in record mode and synchronized with the existing
 log in replay mode.
 
+Disk I/O events are completely deterministic in our model, because
+in both record and replay modes we start virtual machine from the same
+disk state. But callbacks that virtual disk controller uses for reading and
+writing the disk may occur at different moments of time in record and replay
+modes.
+
+Reading and writing requests are created by CPU thread of QEMU. Later these
+requests proceed to block layer which creates "bottom halves". Bottom
+halves consist of callback and its parameters. They are processed when
+main loop locks the global mutex. These locks are not synchronized with
+replaying process because main loop also processes the events that do not
+affect the virtual machine state (like user interaction with monitor).
+
+That is why we had to implement saving and replaying bottom halves callbacks
+synchronously to the CPU execution. When the callback is about to execute
+it is added to the queue in the replay module. This queue is written to the
+log when its callbacks are executed. In replay mode callbacks are not processed
+until the corresponding event is read from the events log file.
+
+Sometimes the block layer uses asynchronous callbacks for its internal purposes
+(like reading or writing VM snapshots or disk image cluster tables). In this
+case bottom halves are not marked as "replayable" and do not saved
+into the log.
+
 Saving/restoring the VM state
 -----------------------------
 
@@ -42,7 +207,7 @@ should be restored by loadvm to the same values they had before savevm.
 
 Avoid accessing other devices' state, because the order of saving/restoring
 is not defined. It means that you should not call functions like
-'update_irq' in post_load callback. Save everything explicitly to avoid
+``update_irq`` in ``post_load`` callback. Save everything explicitly to avoid
 the dependencies that may make restoring the VM state non-deterministic.
 
 Stopping the VM
@@ -52,3 +217,90 @@ Stopping the guest should not interfere with its state (with the exception
 of the network connections, that could be broken by the remote timeouts).
 VM can be stopped at any moment of replay by the user. Restarting the VM
 after that stop should not break the replay by the unneeded guest state change.
+
+Replay log format
+=================
+
+Record/replay log consists of the header and the sequence of execution
+events. The header includes 4-byte replay version id and 8-byte reserved
+field. Version is updated every time replay log format changes to prevent
+using replay log created by another build of qemu.
+
+The sequence of the events describes virtual machine state changes.
+It includes all non-deterministic inputs of VM, synchronization marks and
+instruction counts used to correctly inject inputs at replay.
+
+Synchronization marks (checkpoints) are used for synchronizing qemu threads
+that perform operations with virtual hardware. These operations may change
+system's state (e.g., change some register or generate interrupt) and
+therefore should execute synchronously with CPU thread.
+
+Every event in the log includes 1-byte event id and optional arguments.
+When argument is an array, it is stored as 4-byte array length
+and corresponding number of bytes with data.
+Here is the list of events that are written into the log:
+
+ - EVENT_INSTRUCTION. Instructions executed since last event. Followed by:
+
+   - 4-byte number of executed instructions.
+
+ - EVENT_INTERRUPT. Used to synchronize interrupt processing.
+ - EVENT_EXCEPTION. Used to synchronize exception handling.
+ - EVENT_ASYNC. This is a group of events. When such an event is generated,
+   it is stored in the queue and processed in icount_account_warp_timer().
+   Every such event has it's own id from the following list:
+
+     - REPLAY_ASYNC_EVENT_BH. Bottom-half callback. This event synchronizes
+       callbacks that affect virtual machine state, but normally called
+       asynchronously. Followed by:
+
+        - 8-byte operation id.
+
+     - REPLAY_ASYNC_EVENT_INPUT. Input device event. Contains
+       parameters of keyboard and mouse input operations
+       (key press/release, mouse pointer movement). Followed by:
+
+        - 9-16 bytes depending of input event.
+
+     - REPLAY_ASYNC_EVENT_INPUT_SYNC. Internal input synchronization event.
+     - REPLAY_ASYNC_EVENT_CHAR_READ. Character (e.g., serial port) device input
+       initiated by the sender. Followed by:
+
+        - 1-byte character device id.
+        - Array with bytes were read.
+
+     - REPLAY_ASYNC_EVENT_BLOCK. Block device operation. Used to synchronize
+       operations with disk and flash drives with CPU. Followed by:
+
+        - 8-byte operation id.
+
+     - REPLAY_ASYNC_EVENT_NET. Incoming network packet. Followed by:
+
+        - 1-byte network adapter id.
+        - 4-byte packet flags.
+        - Array with packet bytes.
+
+ - EVENT_SHUTDOWN. Occurs when user sends shutdown event to qemu,
+   e.g., by closing the window.
+ - EVENT_CHAR_WRITE. Used to synchronize character output operations. Followed by:
+
+    - 4-byte output function return value.
+    - 4-byte offset in the output array.
+
+ - EVENT_CHAR_READ_ALL. Used to synchronize character input operations,
+   initiated by qemu. Followed by:
+
+    - Array with bytes that were read.
+
+ - EVENT_CHAR_READ_ALL_ERROR. Unsuccessful character input operation,
+   initiated by qemu. Followed by:
+
+    - 4-byte error code.
+
+ - EVENT_CLOCK + clock_id. Group of events for host clock read operations. Followed by:
+
+    - 8-byte clock value.
+
+ - EVENT_CHECKPOINT + checkpoint_id. Checkpoint for synchronization of
+   CPU, internal threads, and asynchronous input events.
+ - EVENT_END. Last event in the log.
diff --git a/docs/replay.txt b/docs/replay.txt
deleted file mode 100644
index 6c9fdff09d..0000000000
--- a/docs/replay.txt
+++ /dev/null
@@ -1,407 +0,0 @@
-Copyright (c) 2010-2015 Institute for System Programming
-                        of the Russian Academy of Sciences.
-
-This work is licensed under the terms of the GNU GPL, version 2 or later.
-See the COPYING file in the top-level directory.
-
-Record/replay
--------------
-
-Record/replay functions are used for the deterministic replay of qemu execution.
-Execution recording writes a non-deterministic events log, which can be later
-used for replaying the execution anywhere and for unlimited number of times.
-It also supports checkpointing for faster rewind to the specific replay moment.
-Execution replaying reads the log and replays all non-deterministic events
-including external input, hardware clocks, and interrupts.
-
-Deterministic replay has the following features:
- * Deterministically replays whole system execution and all contents of
-   the memory, state of the hardware devices, clocks, and screen of the VM.
- * Writes execution log into the file for later replaying for multiple times
-   on different machines.
- * Supports i386, x86_64, and Arm hardware platforms.
- * Performs deterministic replay of all operations with keyboard and mouse
-   input devices.
-
-Usage of the record/replay:
- * First, record the execution with the following command line:
-    qemu-system-i386 \
-     -icount shift=7,rr=record,rrfile=replay.bin \
-     -drive file=disk.qcow2,if=none,snapshot,id=img-direct \
-     -drive driver=blkreplay,if=none,image=img-direct,id=img-blkreplay \
-     -device ide-hd,drive=img-blkreplay \
-     -netdev user,id=net1 -device rtl8139,netdev=net1 \
-     -object filter-replay,id=replay,netdev=net1
- * After recording, you can replay it by using another command line:
-    qemu-system-i386 \
-     -icount shift=7,rr=replay,rrfile=replay.bin \
-     -drive file=disk.qcow2,if=none,snapshot,id=img-direct \
-     -drive driver=blkreplay,if=none,image=img-direct,id=img-blkreplay \
-     -device ide-hd,drive=img-blkreplay \
-     -netdev user,id=net1 -device rtl8139,netdev=net1 \
-     -object filter-replay,id=replay,netdev=net1
-   The only difference with recording is changing the rr option
-   from record to replay.
- * Block device images are not actually changed in the recording mode,
-   because all of the changes are written to the temporary overlay file.
-   This behavior is enabled by using blkreplay driver. It should be used
-   for every enabled block device, as described in 'Block devices' section.
- * '-net none' option should be specified when network is not used,
-   because QEMU adds network card by default. When network is needed,
-   it should be configured explicitly with replay filter, as described
-   in 'Network devices' section.
- * Interaction with audio devices and serial ports are recorded and replayed
-   automatically when such devices are enabled.
-
-Academic papers with description of deterministic replay implementation:
-http://www.computer.org/csdl/proceedings/csmr/2012/4666/00/4666a553-abs.html
-http://dl.acm.org/citation.cfm?id=2786805.2803179
-
-Modifications of qemu include:
- * wrappers for clock and time functions to save their return values in the log
- * saving different asynchronous events (e.g. system shutdown) into the log
- * synchronization of the bottom halves execution
- * synchronization of the threads from thread pool
- * recording/replaying user input (mouse, keyboard, and microphone)
- * adding internal checkpoints for cpu and io synchronization
- * network filter for recording and replaying the packets
- * block driver for making block layer deterministic
- * serial port input record and replay
- * recording of random numbers obtained from the external sources
-
-Locking and thread synchronisation
-----------------------------------
-
-Previously the synchronisation of the main thread and the vCPU thread
-was ensured by the holding of the BQL. However the trend has been to
-reduce the time the BQL was held across the system including under TCG
-system emulation. As it is important that batches of events are kept
-in sequence (e.g. expiring timers and checkpoints in the main thread
-while instruction checkpoints are written by the vCPU thread) we need
-another lock to keep things in lock-step. This role is now handled by
-the replay_mutex_lock. It used to be held only for each event being
-written but now it is held for a whole execution period. This results
-in a deterministic ping-pong between the two main threads.
-
-As the BQL is now a finer grained lock than the replay_lock it is almost
-certainly a bug, and a source of deadlocks, to take the
-replay_mutex_lock while the BQL is held. This is enforced by an assert.
-While the unlocks are usually in the reverse order, this is not
-necessary; you can drop the replay_lock while holding the BQL, without
-doing a more complicated unlock_iothread/replay_unlock/lock_iothread
-sequence.
-
-Non-deterministic events
-------------------------
-
-Our record/replay system is based on saving and replaying non-deterministic
-events (e.g. keyboard input) and simulating deterministic ones (e.g. reading
-from HDD or memory of the VM). Saving only non-deterministic events makes
-log file smaller and simulation faster.
-
-The following non-deterministic data from peripheral devices is saved into
-the log: mouse and keyboard input, network packets, audio controller input,
-serial port input, and hardware clocks (they are non-deterministic
-too, because their values are taken from the host machine). Inputs from
-simulated hardware, memory of VM, software interrupts, and execution of
-instructions are not saved into the log, because they are deterministic and
-can be replayed by simulating the behavior of virtual machine starting from
-initial state.
-
-We had to solve three tasks to implement deterministic replay: recording
-non-deterministic events, replaying non-deterministic events, and checking
-that there is no divergence between record and replay modes.
-
-We changed several parts of QEMU to make event log recording and replaying.
-Devices' models that have non-deterministic input from external devices were
-changed to write every external event into the execution log immediately.
-E.g. network packets are written into the log when they arrive into the virtual
-network adapter.
-
-All non-deterministic events are coming from these devices. But to
-replay them we need to know at which moments they occur. We specify
-these moments by counting the number of instructions executed between
-every pair of consecutive events.
-
-Instruction counting
---------------------
-
-QEMU should work in icount mode to use record/replay feature. icount was
-designed to allow deterministic execution in absence of external inputs
-of the virtual machine. We also use icount to control the occurrence of the
-non-deterministic events. The number of instructions elapsed from the last event
-is written to the log while recording the execution. In replay mode we
-can predict when to inject that event using the instruction counter.
-
-Timers
-------
-
-Timers are used to execute callbacks from different subsystems of QEMU
-at the specified moments of time. There are several kinds of timers:
- * Real time clock. Based on host time and used only for callbacks that
-   do not change the virtual machine state. For this reason real time
-   clock and timers does not affect deterministic replay at all.
- * Virtual clock. These timers run only during the emulation. In icount
-   mode virtual clock value is calculated using executed instructions counter.
-   That is why it is completely deterministic and does not have to be recorded.
- * Host clock. This clock is used by device models that simulate real time
-   sources (e.g. real time clock chip). Host clock is the one of the sources
-   of non-determinism. Host clock read operations should be logged to
-   make the execution deterministic.
- * Virtual real time clock. This clock is similar to real time clock but
-   it is used only for increasing virtual clock while virtual machine is
-   sleeping. Due to its nature it is also non-deterministic as the host clock
-   and has to be logged too.
-
-Checkpoints
------------
-
-Replaying of the execution of virtual machine is bound by sources of
-non-determinism. These are inputs from clock and peripheral devices,
-and QEMU thread scheduling. Thread scheduling affect on processing events
-from timers, asynchronous input-output, and bottom halves.
-
-Invocations of timers are coupled with clock reads and changing the state
-of the virtual machine. Reads produce non-deterministic data taken from
-host clock. And VM state changes should preserve their order. Their relative
-order in replay mode must replicate the order of callbacks in record mode.
-To preserve this order we use checkpoints. When a specific clock is processed
-in record mode we save to the log special "checkpoint" event.
-Checkpoints here do not refer to virtual machine snapshots. They are just
-record/replay events used for synchronization.
-
-QEMU in replay mode will try to invoke timers processing in random moment
-of time. That's why we do not process a group of timers until the checkpoint
-event will be read from the log. Such an event allows synchronizing CPU
-execution and timer events.
-
-Two other checkpoints govern the "warping" of the virtual clock.
-While the virtual machine is idle, the virtual clock increments at
-1 ns per *real time* nanosecond.  This is done by setting up a timer
-(called the warp timer) on the virtual real time clock, so that the
-timer fires at the next deadline of the virtual clock; the virtual clock
-is then incremented (which is called "warping" the virtual clock) as
-soon as the timer fires or the CPUs need to go out of the idle state.
-Two functions are used for this purpose; because these actions change
-virtual machine state and must be deterministic, each of them creates a
-checkpoint.  icount_start_warp_timer checks if the CPUs are idle and if so
-starts accounting real time to virtual clock.  icount_account_warp_timer
-is called when the CPUs get an interrupt or when the warp timer fires,
-and it warps the virtual clock by the amount of real time that has passed
-since icount_start_warp_timer.
-
-Bottom halves
--------------
-
-Disk I/O events are completely deterministic in our model, because
-in both record and replay modes we start virtual machine from the same
-disk state. But callbacks that virtual disk controller uses for reading and
-writing the disk may occur at different moments of time in record and replay
-modes.
-
-Reading and writing requests are created by CPU thread of QEMU. Later these
-requests proceed to block layer which creates "bottom halves". Bottom
-halves consist of callback and its parameters. They are processed when
-main loop locks the global mutex. These locks are not synchronized with
-replaying process because main loop also processes the events that do not
-affect the virtual machine state (like user interaction with monitor).
-
-That is why we had to implement saving and replaying bottom halves callbacks
-synchronously to the CPU execution. When the callback is about to execute
-it is added to the queue in the replay module. This queue is written to the
-log when its callbacks are executed. In replay mode callbacks are not processed
-until the corresponding event is read from the events log file.
-
-Sometimes the block layer uses asynchronous callbacks for its internal purposes
-(like reading or writing VM snapshots or disk image cluster tables). In this
-case bottom halves are not marked as "replayable" and do not saved
-into the log.
-
-Block devices
--------------
-
-Block devices record/replay module intercepts calls of
-bdrv coroutine functions at the top of block drivers stack.
-To record and replay block operations the drive must be configured
-as following:
- -drive file=disk.qcow2,if=none,snapshot,id=img-direct
- -drive driver=blkreplay,if=none,image=img-direct,id=img-blkreplay
- -device ide-hd,drive=img-blkreplay
-
-blkreplay driver should be inserted between disk image and virtual driver
-controller. Therefore all disk requests may be recorded and replayed.
-
-All block completion operations are added to the queue in the coroutines.
-Queue is flushed at checkpoints and information about processed requests
-is recorded to the log. In replay phase the queue is matched with
-events read from the log. Therefore block devices requests are processed
-deterministically.
-
-Snapshotting
-------------
-
-New VM snapshots may be created in replay mode. They can be used later
-to recover the desired VM state. All VM states created in replay mode
-are associated with the moment of time in the replay scenario.
-After recovering the VM state replay will start from that position.
-
-Default starting snapshot name may be specified with icount field
-rrsnapshot as follows:
- -icount shift=7,rr=record,rrfile=replay.bin,rrsnapshot=snapshot_name
-
-This snapshot is created at start of recording and restored at start
-of replaying. It also can be loaded while replaying to roll back
-the execution.
-
-'snapshot' flag of the disk image must be removed to save the snapshots
-in the overlay (or original image) instead of using the temporary overlay.
- -drive file=disk.ovl,if=none,id=img-direct
- -drive driver=blkreplay,if=none,image=img-direct,id=img-blkreplay
- -device ide-hd,drive=img-blkreplay
-
-Use QEMU monitor to create additional snapshots. 'savevm <name>' command
-created the snapshot and 'loadvm <name>' restores it. To prevent corruption
-of the original disk image, use overlay files linked to the original images.
-Therefore all new snapshots (including the starting one) will be saved in
-overlays and the original image remains unchanged.
-
-When you need to use snapshots with diskless virtual machine,
-it must be started with 'orphan' qcow2 image. This image will be used
-for storing VM snapshots. Here is the example of the command line for this:
-
-  qemu-system-i386 -icount shift=3,rr=replay,rrfile=record.bin,rrsnapshot=init \
-    -net none -drive file=empty.qcow2,if=none,id=rr
-
-empty.qcow2 drive does not connected to any virtual block device and used
-for VM snapshots only.
-
-Network devices
----------------
-
-Record and replay for network interactions is performed with the network filter.
-Each backend must have its own instance of the replay filter as follows:
- -netdev user,id=net1 -device rtl8139,netdev=net1
- -object filter-replay,id=replay,netdev=net1
-
-Replay network filter is used to record and replay network packets. While
-recording the virtual machine this filter puts all packets coming from
-the outer world into the log. In replay mode packets from the log are
-injected into the network device. All interactions with network backend
-in replay mode are disabled.
-
-Audio devices
--------------
-
-Audio data is recorded and replay automatically. The command line for recording
-and replaying must contain identical specifications of audio hardware, e.g.:
- -soundhw ac97
-
-Serial ports
-------------
-
-Serial ports input is recorded and replay automatically. The command lines
-for recording and replaying must contain identical number of ports in record
-and replay modes, but their backends may differ.
-E.g., '-serial stdio' in record mode, and '-serial null' in replay mode.
-
-Reverse debugging
------------------
-
-Reverse debugging allows "executing" the program in reverse direction.
-GDB remote protocol supports "reverse step" and "reverse continue"
-commands. The first one steps single instruction backwards in time,
-and the second one finds the last breakpoint in the past.
-
-Recorded executions may be used to enable reverse debugging. QEMU can't
-execute the code in backwards direction, but can load a snapshot and
-replay forward to find the desired position or breakpoint.
-
-The following GDB commands are supported:
- - reverse-stepi (or rsi) - step one instruction backwards
- - reverse-continue (or rc) - find last breakpoint in the past
-
-Reverse step loads the nearest snapshot and replays the execution until
-the required instruction is met.
-
-Reverse continue may include several passes of examining the execution
-between the snapshots. Each of the passes include the following steps:
- 1. loading the snapshot
- 2. replaying to examine the breakpoints
- 3. if breakpoint or watchpoint was met
-    - loading the snapshot again
-    - replaying to the required breakpoint
- 4. else
-    - proceeding to the p.1 with the earlier snapshot
-
-Therefore usage of the reverse debugging requires at least one snapshot
-created in advance. This can be done by omitting 'snapshot' option
-for the block drives and adding 'rrsnapshot' for both record and replay
-command lines.
-See the "Snapshotting" section to learn more about running record/replay
-and creating the snapshot in these modes.
-
-Replay log format
------------------
-
-Record/replay log consists of the header and the sequence of execution
-events. The header includes 4-byte replay version id and 8-byte reserved
-field. Version is updated every time replay log format changes to prevent
-using replay log created by another build of qemu.
-
-The sequence of the events describes virtual machine state changes.
-It includes all non-deterministic inputs of VM, synchronization marks and
-instruction counts used to correctly inject inputs at replay.
-
-Synchronization marks (checkpoints) are used for synchronizing qemu threads
-that perform operations with virtual hardware. These operations may change
-system's state (e.g., change some register or generate interrupt) and
-therefore should execute synchronously with CPU thread.
-
-Every event in the log includes 1-byte event id and optional arguments.
-When argument is an array, it is stored as 4-byte array length
-and corresponding number of bytes with data.
-Here is the list of events that are written into the log:
-
- - EVENT_INSTRUCTION. Instructions executed since last event.
-   Argument: 4-byte number of executed instructions.
- - EVENT_INTERRUPT. Used to synchronize interrupt processing.
- - EVENT_EXCEPTION. Used to synchronize exception handling.
- - EVENT_ASYNC. This is a group of events. When such an event is generated,
-   it is stored in the queue and processed in icount_account_warp_timer().
-   Every such event has it's own id from the following list:
-     - REPLAY_ASYNC_EVENT_BH. Bottom-half callback. This event synchronizes
-       callbacks that affect virtual machine state, but normally called
-       asynchronously.
-       Argument: 8-byte operation id.
-     - REPLAY_ASYNC_EVENT_INPUT. Input device event. Contains
-       parameters of keyboard and mouse input operations
-       (key press/release, mouse pointer movement).
-       Arguments: 9-16 bytes depending of input event.
-     - REPLAY_ASYNC_EVENT_INPUT_SYNC. Internal input synchronization event.
-     - REPLAY_ASYNC_EVENT_CHAR_READ. Character (e.g., serial port) device input
-       initiated by the sender.
-       Arguments: 1-byte character device id.
-                  Array with bytes were read.
-     - REPLAY_ASYNC_EVENT_BLOCK. Block device operation. Used to synchronize
-       operations with disk and flash drives with CPU.
-       Argument: 8-byte operation id.
-     - REPLAY_ASYNC_EVENT_NET. Incoming network packet.
-       Arguments: 1-byte network adapter id.
-                  4-byte packet flags.
-                  Array with packet bytes.
- - EVENT_SHUTDOWN. Occurs when user sends shutdown event to qemu,
-   e.g., by closing the window.
- - EVENT_CHAR_WRITE. Used to synchronize character output operations.
-   Arguments: 4-byte output function return value.
-              4-byte offset in the output array.
- - EVENT_CHAR_READ_ALL. Used to synchronize character input operations,
-   initiated by qemu.
-   Argument: Array with bytes that were read.
- - EVENT_CHAR_READ_ALL_ERROR. Unsuccessful character input operation,
-   initiated by qemu.
-   Argument: 4-byte error code.
- - EVENT_CLOCK + clock_id. Group of events for host clock read operations.
-   Argument: 8-byte clock value.
- - EVENT_CHECKPOINT + checkpoint_id. Checkpoint for synchronization of
-   CPU, internal threads, and asynchronous input events.
- - EVENT_END. Last event in the log.
diff --git a/docs/system/index.rst b/docs/system/index.rst
index 23e30e26e5..e3695649c5 100644
--- a/docs/system/index.rst
+++ b/docs/system/index.rst
@@ -27,6 +27,7 @@ or Hypervisor.Framework.
    secrets
    authz
    gdb
+   replay
    managed-startup
    bootindex
    cpu-hotplug
diff --git a/docs/system/replay.rst b/docs/system/replay.rst
new file mode 100644
index 0000000000..3105327423
--- /dev/null
+++ b/docs/system/replay.rst
@@ -0,0 +1,237 @@
+.. _replay:
+
+..
+    Copyright (c) 2010-2022 Institute for System Programming
+                        of the Russian Academy of Sciences.
+
+    This work is licensed under the terms of the GNU GPL, version 2 or later.
+    See the COPYING file in the top-level directory.
+
+Record/replay
+=============
+
+Record/replay functions are used for the deterministic replay of qemu execution.
+Execution recording writes a non-deterministic events log, which can be later
+used for replaying the execution anywhere and for unlimited number of times.
+It also supports checkpointing for faster rewind to the specific replay moment.
+Execution replaying reads the log and replays all non-deterministic events
+including external input, hardware clocks, and interrupts.
+
+Deterministic replay has the following features:
+
+ * Deterministically replays whole system execution and all contents of
+   the memory, state of the hardware devices, clocks, and screen of the VM.
+ * Writes execution log into the file for later replaying for multiple times
+   on different machines.
+ * Supports i386, x86_64, ARM, AArch64, Risc-V, MIPS, MIPS64, S390X, Alpha,
+   PowerPC, PowerPC64, M68000, Microblaze, OpenRISC, Nios II, SPARC,
+   and Xtensa hardware platforms.
+ * Performs deterministic replay of all operations with keyboard and mouse
+   input devices, serial ports, and network.
+
+Usage of the record/replay:
+
+ * First, record the execution with the following command line:
+
+    .. parsed-literal::
+        |qemu_system| \\
+        -icount shift=auto,rr=record,rrfile=replay.bin \\
+        -drive file=disk.qcow2,if=none,snapshot,id=img-direct \\
+        -drive driver=blkreplay,if=none,image=img-direct,id=img-blkreplay \\
+        -device ide-hd,drive=img-blkreplay \\
+        -netdev user,id=net1 -device rtl8139,netdev=net1 \\
+        -object filter-replay,id=replay,netdev=net1
+
+ * After recording, you can replay it by using another command line:
+
+    .. parsed-literal::
+        |qemu_system| \\
+        -icount shift=auto,rr=replay,rrfile=replay.bin \\
+        -drive file=disk.qcow2,if=none,snapshot,id=img-direct \\
+        -drive driver=blkreplay,if=none,image=img-direct,id=img-blkreplay \\
+        -device ide-hd,drive=img-blkreplay \\
+        -netdev user,id=net1 -device rtl8139,netdev=net1 \\
+        -object filter-replay,id=replay,netdev=net1
+
+   The only difference with recording is changing the rr option
+   from record to replay.
+ * Block device images are not actually changed in the recording mode,
+   because all of the changes are written to the temporary overlay file.
+   This behavior is enabled by using blkreplay driver. It should be used
+   for every enabled block device, as described in :ref:`block-label` section.
+ * ``-net none`` option should be specified when network is not used,
+   because QEMU adds network card by default. When network is needed,
+   it should be configured explicitly with replay filter, as described
+   in :ref:`network-label` section.
+ * Interaction with audio devices and serial ports are recorded and replayed
+   automatically when such devices are enabled.
+
+Core idea
+---------
+
+Record/replay system is based on saving and replaying non-deterministic
+events (e.g. keyboard input) and simulating deterministic ones (e.g. reading
+from HDD or memory of the VM). Saving only non-deterministic events makes
+log file smaller and simulation faster.
+
+The following non-deterministic data from peripheral devices is saved into
+the log: mouse and keyboard input, network packets, audio controller input,
+serial port input, and hardware clocks (they are non-deterministic
+too, because their values are taken from the host machine). Inputs from
+simulated hardware, memory of VM, software interrupts, and execution of
+instructions are not saved into the log, because they are deterministic and
+can be replayed by simulating the behavior of virtual machine starting from
+initial state.
+
+Instruction counting
+--------------------
+
+QEMU should work in icount mode to use record/replay feature. icount was
+designed to allow deterministic execution in absence of external inputs
+of the virtual machine. Record/replay feature is enabled through ``-icount``
+command-line option, making possible deterministic execution of the machine,
+interacting with user or network.
+
+.. _block-label:
+
+Block devices
+-------------
+
+Block devices record/replay module intercepts calls of
+bdrv coroutine functions at the top of block drivers stack.
+To record and replay block operations the drive must be configured
+as following:
+
+.. parsed-literal::
+    -drive file=disk.qcow2,if=none,snapshot,id=img-direct
+    -drive driver=blkreplay,if=none,image=img-direct,id=img-blkreplay
+    -device ide-hd,drive=img-blkreplay
+
+blkreplay driver should be inserted between disk image and virtual driver
+controller. Therefore all disk requests may be recorded and replayed.
+
+.. _snapshotting-label:
+
+Snapshotting
+------------
+
+New VM snapshots may be created in replay mode. They can be used later
+to recover the desired VM state. All VM states created in replay mode
+are associated with the moment of time in the replay scenario.
+After recovering the VM state replay will start from that position.
+
+Default starting snapshot name may be specified with icount field
+rrsnapshot as follows:
+
+.. parsed-literal::
+    -icount shift=auto,rr=record,rrfile=replay.bin,rrsnapshot=snapshot_name
+
+This snapshot is created at start of recording and restored at start
+of replaying. It also can be loaded while replaying to roll back
+the execution.
+
+``snapshot`` flag of the disk image must be removed to save the snapshots
+in the overlay (or original image) instead of using the temporary overlay.
+
+.. parsed-literal::
+    -drive file=disk.ovl,if=none,id=img-direct
+    -drive driver=blkreplay,if=none,image=img-direct,id=img-blkreplay
+    -device ide-hd,drive=img-blkreplay
+
+Use QEMU monitor to create additional snapshots. ``savevm <name>`` command
+created the snapshot and ``loadvm <name>`` restores it. To prevent corruption
+of the original disk image, use overlay files linked to the original images.
+Therefore all new snapshots (including the starting one) will be saved in
+overlays and the original image remains unchanged.
+
+When you need to use snapshots with diskless virtual machine,
+it must be started with "orphan" qcow2 image. This image will be used
+for storing VM snapshots. Here is the example of the command line for this:
+
+.. parsed-literal::
+    |qemu_system| \\
+      -icount shift=auto,rr=replay,rrfile=record.bin,rrsnapshot=init \\
+      -net none -drive file=empty.qcow2,if=none,id=rr
+
+``empty.qcow2`` drive does not connected to any virtual block device and used
+for VM snapshots only.
+
+.. _network-label:
+
+Network devices
+---------------
+
+Record and replay for network interactions is performed with the network filter.
+Each backend must have its own instance of the replay filter as follows:
+
+.. parsed-literal::
+    -netdev user,id=net1 -device rtl8139,netdev=net1
+    -object filter-replay,id=replay,netdev=net1
+
+Replay network filter is used to record and replay network packets. While
+recording the virtual machine this filter puts all packets coming from
+the outer world into the log. In replay mode packets from the log are
+injected into the network device. All interactions with network backend
+in replay mode are disabled.
+
+Audio devices
+-------------
+
+Audio data is recorded and replay automatically. The command line for recording
+and replaying must contain identical specifications of audio hardware, e.g.:
+
+.. parsed-literal::
+    -soundhw ac97
+
+Serial ports
+------------
+
+Serial ports input is recorded and replay automatically. The command lines
+for recording and replaying must contain identical number of ports in record
+and replay modes, but their backends may differ.
+E.g., ``-serial stdio`` in record mode, and ``-serial null`` in replay mode.
+
+Reverse debugging
+-----------------
+
+Reverse debugging allows "executing" the program in reverse direction.
+GDB remote protocol supports "reverse step" and "reverse continue"
+commands. The first one steps single instruction backwards in time,
+and the second one finds the last breakpoint in the past.
+
+Recorded executions may be used to enable reverse debugging. QEMU can't
+execute the code in backwards direction, but can load a snapshot and
+replay forward to find the desired position or breakpoint.
+
+The following GDB commands are supported:
+
+ - ``reverse-stepi`` (or ``rsi``) - step one instruction backwards
+ - ``reverse-continue`` (or ``rc``) - find last breakpoint in the past
+
+Reverse step loads the nearest snapshot and replays the execution until
+the required instruction is met.
+
+Reverse continue may include several passes of examining the execution
+between the snapshots. Each of the passes include the following steps:
+
+ #. loading the snapshot
+ #. replaying to examine the breakpoints
+ #. if breakpoint or watchpoint was met
+
+    * loading the snapshot again
+    * replaying to the required breakpoint
+
+ #. else
+
+    * proceeding to the p.1 with the earlier snapshot
+
+Therefore usage of the reverse debugging requires at least one snapshot
+created. This can be done by omitting ``snapshot`` option
+for the block drives and adding ``rrsnapshot`` for both record and replay
+command lines.
+See the :ref:`snapshotting-label` section to learn more about running record/replay
+and creating the snapshot in these modes.
+
+When ``rrsnapshot`` is not used, then snapshot named ``start_debugging``
+created in temporary overlay. This allows using reverse debugging, but with
+temporary snapshots (existing within the session).

From c9d27a0f9f2ee7e6a57167de7d0dcab406f8bfa7 Mon Sep 17 00:00:00 2001
From: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Date: Fri, 27 May 2022 13:46:42 +0300
Subject: [PATCH 790/935] tests/avocado: update replay_linux test

This patch updates replay_linux test to make it compatible with
new LinuxTest class.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Message-Id: <165364840253.688121.10404266209986316381.stgit@pasha-ThinkPad-X280>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/avocado/replay_linux.py | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/tests/avocado/replay_linux.py b/tests/avocado/replay_linux.py
index 15953f9e49..1099b5647f 100644
--- a/tests/avocado/replay_linux.py
+++ b/tests/avocado/replay_linux.py
@@ -32,9 +32,16 @@ class ReplayLinux(LinuxTest):
     bus = 'ide'
 
     def setUp(self):
-        super(ReplayLinux, self).setUp()
+        # LinuxTest does many replay-incompatible things, but includes
+        # useful methods. Do not setup LinuxTest here and just
+        # call some functions.
+        super(LinuxTest, self).setUp()
+        self._set_distro()
         self.boot_path = self.download_boot()
-        self.cloudinit_path = self.prepare_cloudinit()
+        self.phone_server = cloudinit.PhoneHomeServer(('0.0.0.0', 0),
+                                                      self.name)
+        ssh_pubkey, self.ssh_key = self.set_up_existing_ssh_keys()
+        self.cloudinit_path = self.prepare_cloudinit(ssh_pubkey)
 
     def vm_add_disk(self, vm, path, id, device):
         bus_string = ''
@@ -50,7 +57,9 @@ class ReplayLinux(LinuxTest):
         vm = self.get_vm()
         vm.add_args('-smp', '1')
         vm.add_args('-m', '1024')
-        vm.add_args('-object', 'filter-replay,id=replay,netdev=hub0port0')
+        vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22',
+                    '-device', 'virtio-net,netdev=vnet')
+        vm.add_args('-object', 'filter-replay,id=replay,netdev=vnet')
         if args:
             vm.add_args(*args)
         self.vm_add_disk(vm, self.boot_path, 0, self.hdd)
@@ -75,8 +84,8 @@ class ReplayLinux(LinuxTest):
                                     stop_check=(lambda : not vm.is_running()))
         console_drainer.start()
         if record:
-            cloudinit.wait_for_phone_home(('0.0.0.0', self.phone_home_port),
-                                          self.name)
+            while not self.phone_server.instance_phoned_back:
+                self.phone_server.handle_request()
             vm.shutdown()
             logger.info('finished the recording with log size %s bytes'
                 % os.path.getsize(replay_path))

From 3cd1dce33d2359fe410bb20f6f980f3f91da79bd Mon Sep 17 00:00:00 2001
From: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Date: Fri, 27 May 2022 13:46:48 +0300
Subject: [PATCH 791/935] tests/avocado: add replay Linux tests for virtio
 machine

This patch adds two tests for replaying Linux boot process
on x86_64 virtio platform.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Message-Id: <165364840811.688121.11931681195199516354.stgit@pasha-ThinkPad-X280>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/avocado/replay_linux.py | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/tests/avocado/replay_linux.py b/tests/avocado/replay_linux.py
index 1099b5647f..3bb1bc8816 100644
--- a/tests/avocado/replay_linux.py
+++ b/tests/avocado/replay_linux.py
@@ -123,3 +123,29 @@ class ReplayLinuxX8664(ReplayLinux):
         :avocado: tags=machine:q35
         """
         self.run_rr(shift=3)
+
+@skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
+class ReplayLinuxX8664Virtio(ReplayLinux):
+    """
+    :avocado: tags=arch:x86_64
+    :avocado: tags=virtio
+    :avocado: tags=accel:tcg
+    """
+
+    hdd = 'virtio-blk-pci'
+    cd = 'virtio-blk-pci'
+    bus = None
+
+    chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'
+
+    def test_pc_i440fx(self):
+        """
+        :avocado: tags=machine:pc
+        """
+        self.run_rr(shift=1)
+
+    def test_pc_q35(self):
+        """
+        :avocado: tags=machine:q35
+        """
+        self.run_rr(shift=3)

From 9b59af628c97eada68eebc1a9af0c72b75ecf1b8 Mon Sep 17 00:00:00 2001
From: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Date: Fri, 27 May 2022 13:46:53 +0300
Subject: [PATCH 792/935] tests/avocado: add replay Linux test for Aarch64
 machines

This patch adds two tests for replaying Linux boot process
on Aarch64 platform.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Message-Id: <165364841373.688121.8868079200312201658.stgit@pasha-ThinkPad-X280>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/avocado/replay_linux.py | 40 +++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/tests/avocado/replay_linux.py b/tests/avocado/replay_linux.py
index 3bb1bc8816..40e4f6908e 100644
--- a/tests/avocado/replay_linux.py
+++ b/tests/avocado/replay_linux.py
@@ -13,6 +13,7 @@ import logging
 import time
 
 from avocado import skipUnless
+from avocado_qemu import BUILD_DIR
 from avocado.utils import cloudinit
 from avocado.utils import network
 from avocado.utils import vmimage
@@ -149,3 +150,42 @@ class ReplayLinuxX8664Virtio(ReplayLinux):
         :avocado: tags=machine:q35
         """
         self.run_rr(shift=3)
+
+@skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
+class ReplayLinuxAarch64(ReplayLinux):
+    """
+    :avocado: tags=accel:tcg
+    :avocado: tags=arch:aarch64
+    :avocado: tags=machine:virt
+    :avocado: tags=cpu:max
+    """
+
+    chksum = '1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49'
+
+    hdd = 'virtio-blk-device'
+    cd = 'virtio-blk-device'
+    bus = None
+
+    def get_common_args(self):
+        return ('-bios',
+                os.path.join(BUILD_DIR, 'pc-bios', 'edk2-aarch64-code.fd'),
+                "-cpu", "max,lpa2=off",
+                '-device', 'virtio-rng-pci,rng=rng0',
+                '-object', 'rng-builtin,id=rng0')
+
+    def test_virt_gicv2(self):
+        """
+        :avocado: tags=machine:gic-version=2
+        """
+
+        self.run_rr(shift=3,
+                    args=(*self.get_common_args(),
+                          "-machine", "virt,gic-version=2"))
+
+    def test_virt_gicv3(self):
+        """
+        :avocado: tags=machine:gic-version=3
+        """
+
+        self.run_rr(shift=3,
+                    args=(*self.get_common_args(),

From d426ff938b3f95e0b04bb410d47ae50359340a16 Mon Sep 17 00:00:00 2001
From: Dario Faggioli <dfaggioli@suse.com>
Date: Fri, 27 May 2022 17:30:54 +0200
Subject: [PATCH 793/935] tests/Makefile.include: Fix 'make check-help' output
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Since commit 3d2f73ef75e ("build: use "meson test" as the test harness"),
check-report.tap is no more, and we have check-report.junit.xml.

Update the output of 'make check-help', which was still listing
'check-report.tap', accordingly.

Fixes: 3d2f73ef75e
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Message-Id: <165366545439.6869.11633009118019728798.stgit@work>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/Makefile.include | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 6a1688e33e..2a1c77440a 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -3,28 +3,28 @@
 .PHONY: check-help
 check-help:
 	@echo "Regression testing targets:"
-	@echo " $(MAKE) check                Run block, qapi-schema, unit, softfloat, qtest and decodetree tests"
-	@echo " $(MAKE) bench                Run speed tests"
+	@echo " $(MAKE) check                  Run block, qapi-schema, unit, softfloat, qtest and decodetree tests"
+	@echo " $(MAKE) bench                  Run speed tests"
 	@echo
 	@echo "Individual test suites:"
-	@echo " $(MAKE) check-qtest-TARGET   Run qtest tests for given target"
-	@echo " $(MAKE) check-qtest          Run qtest tests"
-	@echo " $(MAKE) check-unit           Run qobject tests"
-	@echo " $(MAKE) check-qapi-schema    Run QAPI schema tests"
-	@echo " $(MAKE) check-block          Run block tests"
+	@echo " $(MAKE) check-qtest-TARGET     Run qtest tests for given target"
+	@echo " $(MAKE) check-qtest            Run qtest tests"
+	@echo " $(MAKE) check-unit             Run qobject tests"
+	@echo " $(MAKE) check-qapi-schema      Run QAPI schema tests"
+	@echo " $(MAKE) check-block            Run block tests"
 ifneq ($(filter $(all-check-targets), check-softfloat),)
-	@echo " $(MAKE) check-tcg            Run TCG tests"
-	@echo " $(MAKE) check-softfloat      Run FPU emulation tests"
+	@echo " $(MAKE) check-tcg              Run TCG tests"
+	@echo " $(MAKE) check-softfloat        Run FPU emulation tests"
 endif
-	@echo " $(MAKE) check-avocado        Run avocado (integration) tests for currently configured targets"
+	@echo " $(MAKE) check-avocado          Run avocado (integration) tests for currently configured targets"
 	@echo
-	@echo " $(MAKE) check-report.tap     Generates an aggregated TAP test report"
-	@echo " $(MAKE) check-venv           Creates a Python venv for tests"
-	@echo " $(MAKE) check-clean          Clean the tests and related data"
+	@echo " $(MAKE) check-report.junit.xml Generates an aggregated XML test report"
+	@echo " $(MAKE) check-venv             Creates a Python venv for tests"
+	@echo " $(MAKE) check-clean            Clean the tests and related data"
 	@echo
 	@echo "The following are useful for CI builds"
-	@echo " $(MAKE) check-build          Build most test binaries"
-	@echo " $(MAKE) get-vm-images        Downloads all images used by avocado tests, according to configured targets (~350 MB each, 1.5 GB max)"
+	@echo " $(MAKE) check-build            Build most test binaries"
+	@echo " $(MAKE) get-vm-images          Downloads all images used by avocado tests, according to configured targets (~350 MB each, 1.5 GB max)"
 	@echo
 	@echo
 	@echo "The variable SPEED can be set to control the gtester speed setting."

From efb3934adf9ee7794db7e0ade9f576c634592891 Mon Sep 17 00:00:00 2001
From: Igor Mammedov <imammedo@redhat.com>
Date: Tue, 24 May 2022 11:10:19 -0400
Subject: [PATCH 794/935] x86: cpu: make sure number of addressable IDs for
 processor cores meets the spec

Accourding Intel's CPUID[EAX=04H] resulting bits 31 - 26 in EAX
should be:
"
 **** The nearest power-of-2 integer that is not smaller than (1 + EAX[31:26]) is the number of unique
    Core_IDs reserved for addressing different processor cores in a physical package. Core ID is a subset of
    bits of the initial APIC ID.
"

ensure that values stored in EAX[31-26] always meets this condition.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20220524151020.2541698-2-imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/cpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 9fdfec9d8b..94cc4a8700 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5287,7 +5287,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
             /* QEMU gives out its own APIC IDs, never pass down bits 31..26.  */
             *eax &= ~0xFC000000;
             if ((*eax & 31) && cs->nr_cores > 1) {
-                *eax |= (cs->nr_cores - 1) << 26;
+                *eax |= (pow2ceil(cs->nr_cores) - 1) << 26;
             }
         } else if (cpu->vendor_cpuid_only && IS_AMD_CPU(env)) {
             *eax = *ebx = *ecx = *edx = 0;

From d7caf13b5fcf742e5680c1d3448ba070fc811644 Mon Sep 17 00:00:00 2001
From: Igor Mammedov <imammedo@redhat.com>
Date: Tue, 24 May 2022 11:10:20 -0400
Subject: [PATCH 795/935] x86: cpu: fixup number of addressable IDs for logical
 processors sharing cache

When QEMU is started with '-cpu host,host-cache-info=on', it will
passthrough host's number of logical processors sharing cache and
number of processor cores in the physical package. QEMU already
fixes up the later to correctly reflect number of configured cores
for VM, however number of logical processors sharing cache is still
comes from host CPU, which confuses guest started with:

       -machine q35,accel=kvm \
       -cpu host,host-cache-info=on,l3-cache=off \
       -smp 20,sockets=2,dies=1,cores=10,threads=1  \
       -numa node,nodeid=0,memdev=ram-node0 \
       -numa node,nodeid=1,memdev=ram-node1 \
       -numa cpu,socket-id=0,node-id=0 \
       -numa cpu,socket-id=1,node-id=1

on 2 socket Xeon 4210R host with 10 cores per socket
with CPUID[04H]:
      ...
        --- cache 3 ---
      cache type                           = unified cache (3)
      cache level                          = 0x3 (3)
      self-initializing cache level        = true
      fully associative cache              = false
      maximum IDs for CPUs sharing cache   = 0x1f (31)
      maximum IDs for cores in pkg         = 0xf (15)
      ...
that doesn't match number of logical processors VM was
configured with and as result RHEL 9.0 guest complains:

   sched: CPU #10's llc-sibling CPU #0 is not on the same node! [node: 1 != 0]. Ignoring dependency.
   WARNING: CPU: 10 PID: 0 at arch/x86/kernel/smpboot.c:421 topology_sane.isra.0+0x67/0x80
   ...
   Call Trace:
     set_cpu_sibling_map+0x176/0x590
     start_secondary+0x5b/0x150
     secondary_startup_64_no_verify+0xc2/0xcb

Fix it by capping max number of logical processors to vcpus/socket
as it was configured, which fixes the issue.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2088311
Message-Id: <20220524151020.2541698-3-imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/cpu.c | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 94cc4a8700..6a57ef13af 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5284,10 +5284,22 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         /* cache info: needed for Core compatibility */
         if (cpu->cache_info_passthrough) {
             x86_cpu_get_cache_cpuid(index, count, eax, ebx, ecx, edx);
-            /* QEMU gives out its own APIC IDs, never pass down bits 31..26.  */
-            *eax &= ~0xFC000000;
-            if ((*eax & 31) && cs->nr_cores > 1) {
-                *eax |= (pow2ceil(cs->nr_cores) - 1) << 26;
+            /*
+             * QEMU has its own number of cores/logical cpus,
+             * set 24..14, 31..26 bit to configured values
+             */
+            if (*eax & 31) {
+                int host_vcpus_per_cache = 1 + ((*eax & 0x3FFC000) >> 14);
+                int vcpus_per_socket = env->nr_dies * cs->nr_cores *
+                                       cs->nr_threads;
+                if (cs->nr_cores > 1) {
+                    *eax &= ~0xFC000000;
+                    *eax |= (pow2ceil(cs->nr_cores) - 1) << 26;
+                }
+                if (host_vcpus_per_cache > vcpus_per_socket) {
+                    *eax &= ~0x3FFC000;
+                    *eax |= (pow2ceil(vcpus_per_socket) - 1) << 14;
+                }
             }
         } else if (cpu->vendor_cpuid_only && IS_AMD_CPU(env)) {
             *eax = *ebx = *ecx = *edx = 0;

From e7874a50ff3f5b20fb46f36958ad88334e4d55cd Mon Sep 17 00:00:00 2001
From: John Snow <jsnow@redhat.com>
Date: Wed, 25 May 2022 20:09:13 -0400
Subject: [PATCH 796/935] python: update for mypy 0.950

typeshed (included in mypy) recently updated to improve the typing for
WriteTransport objects. I was working around this, but now there's a
version where I shouldn't work around it.

Unfortunately this creates some minor ugliness if I want to support both
pre- and post-0.950 versions. For now, for my sanity, just disable the
unused-ignores warning.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220526000921.1581503-2-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 python/qemu/qmp/util.py | 4 +++-
 python/setup.cfg        | 1 +
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/python/qemu/qmp/util.py b/python/qemu/qmp/util.py
index eaa5fc7d5f..ca6225e9cd 100644
--- a/python/qemu/qmp/util.py
+++ b/python/qemu/qmp/util.py
@@ -40,7 +40,9 @@ async def flush(writer: asyncio.StreamWriter) -> None:
     drain. The flow control limits are restored after the call is
     completed.
     """
-    transport = cast(asyncio.WriteTransport, writer.transport)
+    transport = cast(  # type: ignore[redundant-cast]
+        asyncio.WriteTransport, writer.transport
+    )
 
     # https://github.com/python/typeshed/issues/5779
     low, high = transport.get_write_buffer_limits()  # type: ignore
diff --git a/python/setup.cfg b/python/setup.cfg
index e877ea5647..c2c61c7519 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -79,6 +79,7 @@ strict = True
 python_version = 3.6
 warn_unused_configs = True
 namespace_packages = True
+warn_unused_ignores = False
 
 [mypy-qemu.utils.qom_fuse]
 # fusepy has no type stubs:

From caa0f7cbaf4159186d614849ddfbdd89f26d597c Mon Sep 17 00:00:00 2001
From: John Snow <jsnow@redhat.com>
Date: Wed, 25 May 2022 20:09:14 -0400
Subject: [PATCH 797/935] tests: add "TESTS_PYTHON" variable to Makefile

This is a convenience feature: $(PYTHON) points to the Python executable
we were instructed to use by the configure script. We use that Python to
create a virtual environment with the "check-venv" target in
tests/Makefile.include.

$(TESTS_PYTHON) points to the Python executable belonging to the virtual
environment tied to the build. This Python executable is a symlink to
the binary used to create the venv, which will be the version provided
at configure time.

Using $(TESTS_PYTHON) therefore uses the $(PYTHON) executable, but with
paths modified to use packages installed to the venv.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220526000921.1581503-3-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/Makefile.include | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 2a1c77440a..50237f073a 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -89,6 +89,7 @@ TARGETS=$(patsubst libqemu-%.fa, %, $(filter libqemu-%.fa, $(ninja-targets)))
 TESTS_VENV_DIR=$(BUILD_DIR)/tests/venv
 TESTS_VENV_REQ=$(SRC_PATH)/tests/requirements.txt
 TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
+TESTS_PYTHON=$(TESTS_VENV_DIR)/bin/python
 ifndef AVOCADO_TESTS
 	AVOCADO_TESTS=tests/avocado
 endif
@@ -108,7 +109,7 @@ $(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
             $(PYTHON) -m venv $@, \
             VENV, $@)
 	$(call quiet-command, \
-            $(TESTS_VENV_DIR)/bin/python -m pip -q install -r $(TESTS_VENV_REQ), \
+            $(TESTS_PYTHON) -m pip -q install -r $(TESTS_VENV_REQ), \
             PIP, $(TESTS_VENV_REQ))
 	$(call quiet-command, touch $@)
 
@@ -126,7 +127,7 @@ FEDORA_31_DOWNLOAD=$(filter $(FEDORA_31_ARCHES),$(FEDORA_31_ARCHES_CANDIDATES))
 # download one specific Fedora 31 image
 get-vm-image-fedora-31-%: check-venv
 	$(call quiet-command, \
-             $(TESTS_VENV_DIR)/bin/python -m avocado vmimage get \
+             $(TESTS_PYTHON) -m avocado vmimage get \
              --distro=fedora --distro-version=31 --arch=$*, \
 	"AVOCADO", "Downloading avocado tests VM image for $*")
 
@@ -135,7 +136,7 @@ get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_31_DOW
 
 check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images
 	$(call quiet-command, \
-            $(TESTS_VENV_DIR)/bin/python -m avocado \
+            $(TESTS_PYTHON) -m avocado \
             --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
             $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \
 			--filter-by-tags-include-empty-key) \

From aa19dd33ce616c1a34bb569da8e642f0b6f51fa0 Mon Sep 17 00:00:00 2001
From: John Snow <jsnow@redhat.com>
Date: Wed, 25 May 2022 20:09:15 -0400
Subject: [PATCH 798/935] tests: use python3 as the python executable name

Use "python3" instead of "python" as per PEP0394:
https://peps.python.org/pep-0394/

This should always be defined (in a venv, at least!), matching the
preferred python shebang of "#!/usr/bin/env python3".

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220526000921.1581503-4-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/Makefile.include | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 50237f073a..fae2dd0f3c 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -89,7 +89,7 @@ TARGETS=$(patsubst libqemu-%.fa, %, $(filter libqemu-%.fa, $(ninja-targets)))
 TESTS_VENV_DIR=$(BUILD_DIR)/tests/venv
 TESTS_VENV_REQ=$(SRC_PATH)/tests/requirements.txt
 TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
-TESTS_PYTHON=$(TESTS_VENV_DIR)/bin/python
+TESTS_PYTHON=$(TESTS_VENV_DIR)/bin/python3
 ifndef AVOCADO_TESTS
 	AVOCADO_TESTS=tests/avocado
 endif

From a547c82701925400949856488a2c8c8750a3276e Mon Sep 17 00:00:00 2001
From: John Snow <jsnow@redhat.com>
Date: Wed, 25 May 2022 20:09:16 -0400
Subject: [PATCH 799/935] tests: silence pip upgrade warnings during venv
 creation

Turn off the nag warning coaxing us to upgrade pip. It's not really that
interesting to see in CI logs, and as long as nothing is broken --
nothing is broken.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220526000921.1581503-5-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/Makefile.include | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index fae2dd0f3c..ba9f5bd65c 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -109,8 +109,8 @@ $(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
             $(PYTHON) -m venv $@, \
             VENV, $@)
 	$(call quiet-command, \
-            $(TESTS_PYTHON) -m pip -q install -r $(TESTS_VENV_REQ), \
-            PIP, $(TESTS_VENV_REQ))
+            $(TESTS_PYTHON) -m pip -q --disable-pip-version-check install \
+            -r $(TESTS_VENV_REQ), PIP, $(TESTS_VENV_REQ))
 	$(call quiet-command, touch $@)
 
 $(TESTS_RESULTS_DIR):

From 5a6f1199cb5fca105ca99356ae05f8e48f5c8d97 Mon Sep 17 00:00:00 2001
From: John Snow <jsnow@redhat.com>
Date: Wed, 25 May 2022 20:09:17 -0400
Subject: [PATCH 800/935] tests: add quiet-venv-pip macro

Factor out the "test venv pip" macro; rewrite the "check-venv" rule to
be a little more compact. Replace the "PIP" pseudo-command output with
"VENVPIP" to make it 1% more clear that we are talking about using pip
to install something into a venv.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220526000921.1581503-6-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/Makefile.include | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index ba9f5bd65c..fa46c0c61b 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -104,13 +104,13 @@ else
 	AVOCADO_CMDLINE_TAGS=$(addprefix -t , $(AVOCADO_TAGS))
 endif
 
+quiet-venv-pip = $(quiet-@)$(call quiet-command-run, \
+    $(TESTS_PYTHON) -m pip -q --disable-pip-version-check $1, \
+    "VENVPIP","$1")
+
 $(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
-	$(call quiet-command, \
-            $(PYTHON) -m venv $@, \
-            VENV, $@)
-	$(call quiet-command, \
-            $(TESTS_PYTHON) -m pip -q --disable-pip-version-check install \
-            -r $(TESTS_VENV_REQ), PIP, $(TESTS_VENV_REQ))
+	$(call quiet-command, $(PYTHON) -m venv $@, VENV, $@)
+	$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
 	$(call quiet-command, touch $@)
 
 $(TESTS_RESULTS_DIR):

From 0e7647aa8a5d5b99a0ffb401ba75f0ae8c171c96 Mon Sep 17 00:00:00 2001
From: John Snow <jsnow@redhat.com>
Date: Wed, 25 May 2022 20:09:18 -0400
Subject: [PATCH 801/935] tests: install "qemu" namespace package into venv

This patch adds the "qemu" namespace package to the $build/tests/venv
directory. It does so in "editable" mode, which means that changes to
the source python directory will actively be reflected by the venv.

This patch also then removes any sys.path hacking from the avocado test
scripts directly. By doing this, the environment of where to find these
packages is managed entirely by the virtual environment and not by the
scripts themselves.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220526000921.1581503-7-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/Makefile.include                 |  1 +
 tests/avocado/avocado_qemu/__init__.py | 11 +++++------
 tests/avocado/virtio_check_params.py   |  1 -
 tests/avocado/virtio_version.py        |  1 -
 tests/requirements.txt                 |  1 +
 5 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index fa46c0c61b..3accb83b13 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -110,6 +110,7 @@ quiet-venv-pip = $(quiet-@)$(call quiet-command-run, \
 
 $(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
 	$(call quiet-command, $(PYTHON) -m venv $@, VENV, $@)
+	$(call quiet-venv-pip,install -e "$(SRC_PATH)/python/")
 	$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
 	$(call quiet-command, touch $@)
 
diff --git a/tests/avocado/avocado_qemu/__init__.py b/tests/avocado/avocado_qemu/__init__.py
index 39f15c1d51..b656a70c55 100644
--- a/tests/avocado/avocado_qemu/__init__.py
+++ b/tests/avocado/avocado_qemu/__init__.py
@@ -21,6 +21,11 @@ import avocado
 from avocado.utils import cloudinit, datadrainer, process, ssh, vmimage
 from avocado.utils.path import find_command
 
+from qemu.machine import QEMUMachine
+from qemu.utils import (get_info_usernet_hostfwd_port, kvm_available,
+                        tcg_available)
+
+
 #: The QEMU build root directory.  It may also be the source directory
 #: if building from the source dir, but it's safer to use BUILD_DIR for
 #: that purpose.  Be aware that if this code is moved outside of a source
@@ -35,12 +40,6 @@ if os.path.islink(os.path.dirname(os.path.dirname(__file__))):
 else:
     SOURCE_DIR = BUILD_DIR
 
-sys.path.append(os.path.join(SOURCE_DIR, 'python'))
-
-from qemu.machine import QEMUMachine
-from qemu.utils import (get_info_usernet_hostfwd_port, kvm_available,
-                        tcg_available)
-
 
 def has_cmd(name, args=None):
     """
diff --git a/tests/avocado/virtio_check_params.py b/tests/avocado/virtio_check_params.py
index e869690473..4093da8a67 100644
--- a/tests/avocado/virtio_check_params.py
+++ b/tests/avocado/virtio_check_params.py
@@ -22,7 +22,6 @@ import os
 import re
 import logging
 
-sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
 from qemu.machine import QEMUMachine
 from avocado_qemu import QemuSystemTest
 from avocado import skip
diff --git a/tests/avocado/virtio_version.py b/tests/avocado/virtio_version.py
index 208910bb84..c84e48813a 100644
--- a/tests/avocado/virtio_version.py
+++ b/tests/avocado/virtio_version.py
@@ -11,7 +11,6 @@ Check compatibility of virtio device types
 import sys
 import os
 
-sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
 from qemu.machine import QEMUMachine
 from avocado_qemu import QemuSystemTest
 
diff --git a/tests/requirements.txt b/tests/requirements.txt
index a21b59b443..0ba561b6bd 100644
--- a/tests/requirements.txt
+++ b/tests/requirements.txt
@@ -1,5 +1,6 @@
 # Add Python module requirements, one per line, to be installed
 # in the tests/venv Python virtual environment. For more info,
 # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
+# Note that qemu.git/python/ is always implicitly installed.
 avocado-framework==88.1
 pycdlib==1.11.0

From f4c66f17051f0dee0e6070c6b78a78e7425b5c96 Mon Sep 17 00:00:00 2001
From: John Snow <jsnow@redhat.com>
Date: Wed, 25 May 2022 20:09:19 -0400
Subject: [PATCH 802/935] tests: use tests/venv to run basevm.py-based scripts

This patch co-opts the virtual environment being used by avocado tests
to also run the basevm.py tests. This is being done in preparation for
for the qemu.qmp package being removed from qemu.git.

As part of the change, remove any sys.path() hacks and treat "qemu" as a
normal third-party import.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220526000921.1581503-8-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/vm/Makefile.include | 13 +++++++------
 tests/vm/basevm.py        |  6 +++---
 2 files changed, 10 insertions(+), 9 deletions(-)

diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index ae91f5043e..588bc999cc 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -84,10 +84,11 @@ vm-clean-all:
 
 $(IMAGES_DIR)/%.img:	$(SRC_PATH)/tests/vm/% \
 			$(SRC_PATH)/tests/vm/basevm.py \
-			$(SRC_PATH)/tests/vm/Makefile.include
+			$(SRC_PATH)/tests/vm/Makefile.include \
+			check-venv
 	@mkdir -p $(IMAGES_DIR)
 	$(call quiet-command, \
-		$(PYTHON) $< \
+		$(TESTS_PYTHON) $< \
 		$(if $(V)$(DEBUG), --debug) \
 		$(if $(GENISOIMAGE),--genisoimage $(GENISOIMAGE)) \
 		$(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \
@@ -101,9 +102,9 @@ $(IMAGES_DIR)/%.img:	$(SRC_PATH)/tests/vm/% \
 
 
 # Build in VM $(IMAGE)
-vm-build-%: $(IMAGES_DIR)/%.img
+vm-build-%: $(IMAGES_DIR)/%.img check-venv
 	$(call quiet-command, \
-		$(PYTHON) $(SRC_PATH)/tests/vm/$* \
+		$(TESTS_PYTHON) $(SRC_PATH)/tests/vm/$* \
 		$(if $(V)$(DEBUG), --debug) \
 		$(if $(DEBUG), --interactive) \
 		$(if $(J),--jobs $(J)) \
@@ -127,9 +128,9 @@ vm-boot-serial-%: $(IMAGES_DIR)/%.img
 		-device virtio-net-pci,netdev=vnet \
 	|| true
 
-vm-boot-ssh-%: $(IMAGES_DIR)/%.img
+vm-boot-ssh-%: $(IMAGES_DIR)/%.img check-venv
 	$(call quiet-command, \
-		$(PYTHON) $(SRC_PATH)/tests/vm/$* \
+		$(TESTS_PYTHON) $(SRC_PATH)/tests/vm/$* \
 		$(if $(J),--jobs $(J)) \
 		$(if $(V)$(DEBUG), --debug) \
 		$(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 254e11c932..d7d0413df3 100644
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -18,9 +18,6 @@ import socket
 import logging
 import time
 import datetime
-sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
-from qemu.machine import QEMUMachine
-from qemu.utils import get_info_usernet_hostfwd_port, kvm_available
 import subprocess
 import hashlib
 import argparse
@@ -31,6 +28,9 @@ import multiprocessing
 import traceback
 import shlex
 
+from qemu.machine import QEMUMachine
+from qemu.utils import get_info_usernet_hostfwd_port, kvm_available
+
 SSH_KEY_FILE = os.path.join(os.path.dirname(__file__),
                "..", "keys", "id_rsa")
 SSH_PUB_KEY_FILE = os.path.join(os.path.dirname(__file__),

From e184036a0e12cbfb8980362cd4de18268b4d0c38 Mon Sep 17 00:00:00 2001
From: John Snow <jsnow@redhat.com>
Date: Wed, 25 May 2022 20:09:20 -0400
Subject: [PATCH 803/935] tests: add python3-venv to debian10.docker

This is needed to be able to add a venv-building step to 'make check';
the clang-user job in particular needs this to be able to run
check-unit.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220526000921.1581503-9-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/docker/dockerfiles/debian10.docker | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/docker/dockerfiles/debian10.docker b/tests/docker/dockerfiles/debian10.docker
index b414af1b9f..03be923066 100644
--- a/tests/docker/dockerfiles/debian10.docker
+++ b/tests/docker/dockerfiles/debian10.docker
@@ -34,4 +34,5 @@ RUN apt update && \
         python3 \
         python3-sphinx \
         python3-sphinx-rtd-theme \
+        python3-venv \
         $(apt-get -s build-dep --arch-only qemu | egrep ^Inst | fgrep '[all]' | cut -d\  -f2)

From 1d8cf47e5b8b4faa95954824cb6610d81c50d7d2 Mon Sep 17 00:00:00 2001
From: John Snow <jsnow@redhat.com>
Date: Wed, 25 May 2022 20:09:21 -0400
Subject: [PATCH 804/935] tests: run 'device-crash-test' from tests/venv

Remove the sys.path hacking from device-crash-test, and add in a little
user-friendly message for anyone who was used to running this script
directly from the source tree.

Modify the GitLab job recipes to create the tests/venv first, then run
device-crash-test from that venv.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220526000921.1581503-10-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 .gitlab-ci.d/buildtest.yml |  8 +++++---
 scripts/device-crash-test  | 14 +++++++++++---
 2 files changed, 16 insertions(+), 6 deletions(-)

diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index ecac3ec50c..544385f5be 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -110,7 +110,8 @@ crash-test-debian:
     IMAGE: debian-amd64
   script:
     - cd build
-    - scripts/device-crash-test -q ./qemu-system-i386
+    - make check-venv
+    - tests/venv/bin/python3 scripts/device-crash-test -q ./qemu-system-i386
 
 build-system-fedora:
   extends: .native_build_job_template
@@ -155,8 +156,9 @@ crash-test-fedora:
     IMAGE: fedora
   script:
     - cd build
-    - scripts/device-crash-test -q ./qemu-system-ppc
-    - scripts/device-crash-test -q ./qemu-system-riscv32
+    - make check-venv
+    - tests/venv/bin/python3 scripts/device-crash-test -q ./qemu-system-ppc
+    - tests/venv/bin/python3 scripts/device-crash-test -q ./qemu-system-riscv32
 
 build-system-centos:
   extends: .native_build_job_template
diff --git a/scripts/device-crash-test b/scripts/device-crash-test
index a203b3fdea..73bcb98693 100755
--- a/scripts/device-crash-test
+++ b/scripts/device-crash-test
@@ -33,10 +33,18 @@ import re
 import random
 import argparse
 from itertools import chain
+from pathlib import Path
 
-sys.path.append(os.path.join(os.path.dirname(__file__), '..', 'python'))
-from qemu.machine import QEMUMachine
-from qemu.qmp import ConnectError
+try:
+    from qemu.machine import QEMUMachine
+    from qemu.qmp import ConnectError
+except ModuleNotFoundError as exc:
+    path = Path(__file__).resolve()
+    print(f"Module '{exc.name}' not found.")
+    print("  Try 'make check-venv' from your build directory,")
+    print("  and then one way to run this script is like so:")
+    print(f'  > $builddir/tests/venv/bin/python3 "{path}"')
+    sys.exit(1)
 
 logger = logging.getLogger('device-crash-test')
 dbg = logger.debug

From d4bb4432ab43febc0eb41b031c0c3aaf5113a0c6 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Mon, 6 Jun 2022 12:44:45 +0200
Subject: [PATCH 805/935] regenerate meson-buildoptions.sh

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 scripts/meson-buildoptions.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index 731e5ea1cf..00ea4d8cd1 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -156,13 +156,13 @@ meson_options_help() {
   printf "%s\n" '  vhost-kernel    vhost kernel backend support'
   printf "%s\n" '  vhost-net       vhost-net kernel acceleration support'
   printf "%s\n" '  vhost-user      vhost-user backend support'
-  printf "%s\n" '  vmnet           vmnet.framework network backend support'
   printf "%s\n" '  vhost-user-blk-server'
   printf "%s\n" '                  build vhost-user-blk server'
   printf "%s\n" '  vhost-vdpa      vhost-vdpa kernel backend support'
   printf "%s\n" '  virglrenderer   virgl rendering support'
   printf "%s\n" '  virtfs          virtio-9p support'
   printf "%s\n" '  virtiofsd       build virtiofs daemon (virtiofsd)'
+  printf "%s\n" '  vmnet           vmnet.framework network backend support'
   printf "%s\n" '  vnc             VNC server'
   printf "%s\n" '  vnc-jpeg        JPEG lossy compression for VNC server'
   printf "%s\n" '  vnc-sasl        SASL authentication for VNC server'
@@ -430,6 +430,8 @@ _meson_option_parse() {
     --disable-virtfs) printf "%s" -Dvirtfs=disabled ;;
     --enable-virtiofsd) printf "%s" -Dvirtiofsd=enabled ;;
     --disable-virtiofsd) printf "%s" -Dvirtiofsd=disabled ;;
+    --enable-vmnet) printf "%s" -Dvmnet=enabled ;;
+    --disable-vmnet) printf "%s" -Dvmnet=disabled ;;
     --enable-vnc) printf "%s" -Dvnc=enabled ;;
     --disable-vnc) printf "%s" -Dvnc=disabled ;;
     --enable-vnc-jpeg) printf "%s" -Dvnc_jpeg=enabled ;;

From 7632a38e7c21c993da74d17afa8dd66d9552da06 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Mon, 6 Jun 2022 12:44:57 +0200
Subject: [PATCH 806/935] configure: remove reference to removed option

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure | 1 -
 1 file changed, 1 deletion(-)

diff --git a/configure b/configure
index b9ccff9067..ac18ed4f3a 100755
--- a/configure
+++ b/configure
@@ -1035,7 +1035,6 @@ Advanced options (experts only):
   --with-git-submodules=ignore   do not update or check git submodules (default if no .git dir)
   --static                 enable static build [$static]
   --bindir=PATH            install binaries in PATH
-  --efi-aarch64=PATH       PATH of efi file to use for aarch64 VMs.
   --with-suffix=SUFFIX     suffix for QEMU data inside datadir/libdir/sysconfdir/docdir [$qemu_suffix]
   --without-default-features default all --enable-* options to "disabled"
   --without-default-devices  do not include any device that is not needed to

From ca5765c8521b8284b8277ed4b811ac0ab9010fc8 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Mon, 6 Jun 2022 16:03:38 +0200
Subject: [PATCH 807/935] meson: qga: do not use deprecated meson.build_root()

The function will return the build root of the parent project if called from a
subproject; that is irrelevant for QEMU's usage but rarely desirable, and
therefore the function was deprecated and replaced by two functions
project_build_root() and global_build_root().  Replace it with the former.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 qga/meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/qga/meson.build b/qga/meson.build
index 619ff095bc..65c1e93846 100644
--- a/qga/meson.build
+++ b/qga/meson.build
@@ -125,7 +125,7 @@ if targetos == 'windows'
                               wixl, '-o', '@OUTPUT0@', '@INPUT0@',
                               qemu_ga_msi_arch[cpu],
                               qemu_ga_msi_vss,
-                              '-D', 'BUILD_DIR=' + meson.build_root(),
+                              '-D', 'BUILD_DIR=' + meson.project_build_root(),
                               '-D', 'BIN_DIR=' + glib.get_variable('bindir'),
                               '-D', 'QEMU_GA_VERSION=' + config_host['QEMU_GA_VERSION'],
                               '-D', 'QEMU_GA_MANUFACTURER=' + config_host['QEMU_GA_MANUFACTURER'],

From 64baad62cd3d9f2e57f65a8f3be7b5639a77e0b4 Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:42:51 +0800
Subject: [PATCH 808/935] target/loongarch: Add README

This patch gives an introduction to the LoongArch target.

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-2-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 MAINTAINERS             |  6 ++++++
 target/loongarch/README | 10 ++++++++++
 2 files changed, 16 insertions(+)
 create mode 100644 target/loongarch/README

diff --git a/MAINTAINERS b/MAINTAINERS
index 5fe8f7eca2..319a5d805a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -212,6 +212,12 @@ S: Maintained
 F: target/hppa/
 F: disas/hppa.c
 
+LoongArch TCG CPUs
+M: Song Gao <gaosong@loongson.cn>
+M: Xiaojuan Yang <yangxiaojuan@loongson.cn>
+S: Maintained
+F: target/loongarch/
+
 M68K TCG CPUs
 M: Laurent Vivier <laurent@vivier.eu>
 S: Maintained
diff --git a/target/loongarch/README b/target/loongarch/README
new file mode 100644
index 0000000000..de141c1a58
--- /dev/null
+++ b/target/loongarch/README
@@ -0,0 +1,10 @@
+- Introduction
+
+  LoongArch is the general processor architecture of Loongson.
+
+  The following versions of the LoongArch core are supported
+    core: 3A5000
+    https://github.com/loongson/LoongArch-Documentation/releases/download/2021.08.17/LoongArch-Vol1-v1.00-EN.pdf
+
+  We can get the latest loongarch documents at https://github.com/loongson/LoongArch-Documentation/tags.
+

From 228021f05ebfd89a3abc225d47a0d0e6c139dade Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:42:52 +0800
Subject: [PATCH 809/935] target/loongarch: Add core definition
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This patch adds target state header, target definitions
and initialization routines.

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220606124333.2060567-3-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/cpu-param.h |  18 ++
 target/loongarch/cpu.c       | 324 +++++++++++++++++++++++++++++++++++
 target/loongarch/cpu.h       | 243 ++++++++++++++++++++++++++
 target/loongarch/internals.h |  21 +++
 4 files changed, 606 insertions(+)
 create mode 100644 target/loongarch/cpu-param.h
 create mode 100644 target/loongarch/cpu.c
 create mode 100644 target/loongarch/cpu.h
 create mode 100644 target/loongarch/internals.h

diff --git a/target/loongarch/cpu-param.h b/target/loongarch/cpu-param.h
new file mode 100644
index 0000000000..9a769b67e0
--- /dev/null
+++ b/target/loongarch/cpu-param.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * LoongArch CPU parameters for QEMU.
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+#ifndef LOONGARCH_CPU_PARAM_H
+#define LOONGARCH_CPU_PARAM_H
+
+#define TARGET_LONG_BITS 64
+#define TARGET_PHYS_ADDR_SPACE_BITS 48
+#define TARGET_VIRT_ADDR_SPACE_BITS 48
+
+#define TARGET_PAGE_BITS 14
+#define NB_MMU_MODES 4
+
+#endif
diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c
new file mode 100644
index 0000000000..1f8e37906a
--- /dev/null
+++ b/target/loongarch/cpu.c
@@ -0,0 +1,324 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * QEMU LoongArch CPU
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "qemu/qemu-print.h"
+#include "qapi/error.h"
+#include "qemu/module.h"
+#include "sysemu/qtest.h"
+#include "exec/exec-all.h"
+#include "qapi/qapi-commands-machine-target.h"
+#include "cpu.h"
+#include "internals.h"
+#include "fpu/softfloat-helpers.h"
+
+const char * const regnames[32] = {
+    "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7",
+    "r8", "r9", "r10", "r11", "r12", "r13", "r14", "r15",
+    "r16", "r17", "r18", "r19", "r20", "r21", "r22", "r23",
+    "r24", "r25", "r26", "r27", "r28", "r29", "r30", "r31",
+};
+
+const char * const fregnames[32] = {
+    "f0", "f1", "f2", "f3", "f4", "f5", "f6", "f7",
+    "f8", "f9", "f10", "f11", "f12", "f13", "f14", "f15",
+    "f16", "f17", "f18", "f19", "f20", "f21", "f22", "f23",
+    "f24", "f25", "f26", "f27", "f28", "f29", "f30", "f31",
+};
+
+static const char * const excp_names[] = {
+    [EXCCODE_INT] = "Interrupt",
+    [EXCCODE_PIL] = "Page invalid exception for load",
+    [EXCCODE_PIS] = "Page invalid exception for store",
+    [EXCCODE_PIF] = "Page invalid exception for fetch",
+    [EXCCODE_PME] = "Page modified exception",
+    [EXCCODE_PNR] = "Page Not Readable exception",
+    [EXCCODE_PNX] = "Page Not Executable exception",
+    [EXCCODE_PPI] = "Page Privilege error",
+    [EXCCODE_ADEF] = "Address error for instruction fetch",
+    [EXCCODE_ADEM] = "Address error for Memory access",
+    [EXCCODE_SYS] = "Syscall",
+    [EXCCODE_BRK] = "Break",
+    [EXCCODE_INE] = "Instruction Non-Existent",
+    [EXCCODE_IPE] = "Instruction privilege error",
+    [EXCCODE_FPE] = "Floating Point Exception",
+    [EXCCODE_DBP] = "Debug breakpoint",
+};
+
+const char *loongarch_exception_name(int32_t exception)
+{
+    assert(excp_names[exception]);
+    return excp_names[exception];
+}
+
+void G_NORETURN do_raise_exception(CPULoongArchState *env,
+                                   uint32_t exception,
+                                   uintptr_t pc)
+{
+    CPUState *cs = env_cpu(env);
+
+    qemu_log_mask(CPU_LOG_INT, "%s: %d (%s)\n",
+                  __func__,
+                  exception,
+                  loongarch_exception_name(exception));
+    cs->exception_index = exception;
+
+    cpu_loop_exit_restore(cs, pc);
+}
+
+static void loongarch_cpu_set_pc(CPUState *cs, vaddr value)
+{
+    LoongArchCPU *cpu = LOONGARCH_CPU(cs);
+    CPULoongArchState *env = &cpu->env;
+
+    env->pc = value;
+}
+
+#ifdef CONFIG_TCG
+static void loongarch_cpu_synchronize_from_tb(CPUState *cs,
+                                              const TranslationBlock *tb)
+{
+    LoongArchCPU *cpu = LOONGARCH_CPU(cs);
+    CPULoongArchState *env = &cpu->env;
+
+    env->pc = tb->pc;
+}
+#endif /* CONFIG_TCG */
+
+static void loongarch_la464_initfn(Object *obj)
+{
+    LoongArchCPU *cpu = LOONGARCH_CPU(obj);
+    CPULoongArchState *env = &cpu->env;
+    int i;
+
+    for (i = 0; i < 21; i++) {
+        env->cpucfg[i] = 0x0;
+    }
+
+    env->cpucfg[0] = 0x14c010;  /* PRID */
+
+    uint32_t data = 0;
+    data = FIELD_DP32(data, CPUCFG1, ARCH, 2);
+    data = FIELD_DP32(data, CPUCFG1, PGMMU, 1);
+    data = FIELD_DP32(data, CPUCFG1, IOCSR, 1);
+    data = FIELD_DP32(data, CPUCFG1, PALEN, 0x2f);
+    data = FIELD_DP32(data, CPUCFG1, VALEN, 0x2f);
+    data = FIELD_DP32(data, CPUCFG1, UAL, 1);
+    data = FIELD_DP32(data, CPUCFG1, RI, 1);
+    data = FIELD_DP32(data, CPUCFG1, EP, 1);
+    data = FIELD_DP32(data, CPUCFG1, RPLV, 1);
+    data = FIELD_DP32(data, CPUCFG1, HP, 1);
+    data = FIELD_DP32(data, CPUCFG1, IOCSR_BRD, 1);
+    env->cpucfg[1] = data;
+
+    data = 0;
+    data = FIELD_DP32(data, CPUCFG2, FP, 1);
+    data = FIELD_DP32(data, CPUCFG2, FP_SP, 1);
+    data = FIELD_DP32(data, CPUCFG2, FP_DP, 1);
+    data = FIELD_DP32(data, CPUCFG2, FP_VER, 1);
+    data = FIELD_DP32(data, CPUCFG2, LLFTP, 1);
+    data = FIELD_DP32(data, CPUCFG2, LLFTP_VER, 1);
+    data = FIELD_DP32(data, CPUCFG2, LAM, 1);
+    env->cpucfg[2] = data;
+
+    env->cpucfg[4] = 100 * 1000 * 1000; /* Crystal frequency */
+
+    data = 0;
+    data = FIELD_DP32(data, CPUCFG5, CC_MUL, 1);
+    data = FIELD_DP32(data, CPUCFG5, CC_DIV, 1);
+    env->cpucfg[5] = data;
+
+    data = 0;
+    data = FIELD_DP32(data, CPUCFG16, L1_IUPRE, 1);
+    data = FIELD_DP32(data, CPUCFG16, L1_DPRE, 1);
+    data = FIELD_DP32(data, CPUCFG16, L2_IUPRE, 1);
+    data = FIELD_DP32(data, CPUCFG16, L2_IUUNIFY, 1);
+    data = FIELD_DP32(data, CPUCFG16, L2_IUPRIV, 1);
+    data = FIELD_DP32(data, CPUCFG16, L3_IUPRE, 1);
+    data = FIELD_DP32(data, CPUCFG16, L3_IUUNIFY, 1);
+    data = FIELD_DP32(data, CPUCFG16, L3_IUINCL, 1);
+    env->cpucfg[16] = data;
+
+    data = 0;
+    data = FIELD_DP32(data, CPUCFG17, L1IU_WAYS, 3);
+    data = FIELD_DP32(data, CPUCFG17, L1IU_SETS, 8);
+    data = FIELD_DP32(data, CPUCFG17, L1IU_SIZE, 6);
+    env->cpucfg[17] = data;
+
+    data = 0;
+    data = FIELD_DP32(data, CPUCFG18, L1D_WAYS, 3);
+    data = FIELD_DP32(data, CPUCFG18, L1D_SETS, 8);
+    data = FIELD_DP32(data, CPUCFG18, L1D_SIZE, 6);
+    env->cpucfg[18] = data;
+
+    data = 0;
+    data = FIELD_DP32(data, CPUCFG19, L2IU_WAYS, 15);
+    data = FIELD_DP32(data, CPUCFG19, L2IU_SETS, 8);
+    data = FIELD_DP32(data, CPUCFG19, L2IU_SIZE, 6);
+    env->cpucfg[19] = data;
+
+    data = 0;
+    data = FIELD_DP32(data, CPUCFG20, L3IU_WAYS, 15);
+    data = FIELD_DP32(data, CPUCFG20, L3IU_SETS, 14);
+    data = FIELD_DP32(data, CPUCFG20, L3IU_SETS, 6);
+    env->cpucfg[20] = data;
+}
+
+static void loongarch_cpu_list_entry(gpointer data, gpointer user_data)
+{
+    const char *typename = object_class_get_name(OBJECT_CLASS(data));
+
+    qemu_printf("%s\n", typename);
+}
+
+void loongarch_cpu_list(void)
+{
+    GSList *list;
+    list = object_class_get_list_sorted(TYPE_LOONGARCH_CPU, false);
+    g_slist_foreach(list, loongarch_cpu_list_entry, NULL);
+    g_slist_free(list);
+}
+
+static void loongarch_cpu_reset(DeviceState *dev)
+{
+    CPUState *cs = CPU(dev);
+    LoongArchCPU *cpu = LOONGARCH_CPU(cs);
+    LoongArchCPUClass *lacc = LOONGARCH_CPU_GET_CLASS(cpu);
+    CPULoongArchState *env = &cpu->env;
+
+    lacc->parent_reset(dev);
+
+    env->fcsr0_mask = FCSR0_M1 | FCSR0_M2 | FCSR0_M3;
+    env->fcsr0 = 0x0;
+
+    cs->exception_index = -1;
+}
+
+static void loongarch_cpu_disas_set_info(CPUState *s, disassemble_info *info)
+{
+    info->print_insn = print_insn_loongarch;
+}
+
+static void loongarch_cpu_realizefn(DeviceState *dev, Error **errp)
+{
+    CPUState *cs = CPU(dev);
+    LoongArchCPUClass *lacc = LOONGARCH_CPU_GET_CLASS(dev);
+    Error *local_err = NULL;
+
+    cpu_exec_realizefn(cs, &local_err);
+    if (local_err != NULL) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
+    cpu_reset(cs);
+    qemu_init_vcpu(cs);
+
+    lacc->parent_realize(dev, errp);
+}
+
+static void loongarch_cpu_init(Object *obj)
+{
+    LoongArchCPU *cpu = LOONGARCH_CPU(obj);
+
+    cpu_set_cpustate_pointers(cpu);
+}
+
+static ObjectClass *loongarch_cpu_class_by_name(const char *cpu_model)
+{
+    ObjectClass *oc;
+    char *typename;
+
+    typename = g_strdup_printf(LOONGARCH_CPU_TYPE_NAME("%s"), cpu_model);
+    oc = object_class_by_name(typename);
+    g_free(typename);
+    return oc;
+}
+
+void loongarch_cpu_dump_state(CPUState *cs, FILE *f, int flags)
+{
+    LoongArchCPU *cpu = LOONGARCH_CPU(cs);
+    CPULoongArchState *env = &cpu->env;
+    int i;
+
+    qemu_fprintf(f, " PC=%016" PRIx64 " ", env->pc);
+    qemu_fprintf(f, " FCSR0 0x%08x  fp_status 0x%02x\n", env->fcsr0,
+                 get_float_exception_flags(&env->fp_status));
+
+    /* gpr */
+    for (i = 0; i < 32; i++) {
+        if ((i & 3) == 0) {
+            qemu_fprintf(f, " GPR%02d:", i);
+        }
+        qemu_fprintf(f, " %s %016" PRIx64, regnames[i], env->gpr[i]);
+        if ((i & 3) == 3) {
+            qemu_fprintf(f, "\n");
+        }
+    }
+
+    /* fpr */
+    if (flags & CPU_DUMP_FPU) {
+        for (i = 0; i < 32; i++) {
+            qemu_fprintf(f, " %s %016" PRIx64, fregnames[i], env->fpr[i]);
+            if ((i & 3) == 3) {
+                qemu_fprintf(f, "\n");
+            }
+        }
+    }
+}
+
+#ifdef CONFIG_TCG
+#include "hw/core/tcg-cpu-ops.h"
+
+static struct TCGCPUOps loongarch_tcg_ops = {
+    .initialize = loongarch_translate_init,
+    .synchronize_from_tb = loongarch_cpu_synchronize_from_tb,
+};
+#endif /* CONFIG_TCG */
+
+static void loongarch_cpu_class_init(ObjectClass *c, void *data)
+{
+    LoongArchCPUClass *lacc = LOONGARCH_CPU_CLASS(c);
+    CPUClass *cc = CPU_CLASS(c);
+    DeviceClass *dc = DEVICE_CLASS(c);
+
+    device_class_set_parent_realize(dc, loongarch_cpu_realizefn,
+                                    &lacc->parent_realize);
+    device_class_set_parent_reset(dc, loongarch_cpu_reset, &lacc->parent_reset);
+
+    cc->class_by_name = loongarch_cpu_class_by_name;
+    cc->dump_state = loongarch_cpu_dump_state;
+    cc->set_pc = loongarch_cpu_set_pc;
+    cc->disas_set_info = loongarch_cpu_disas_set_info;
+#ifdef CONFIG_TCG
+    cc->tcg_ops = &loongarch_tcg_ops;
+#endif
+}
+
+#define DEFINE_LOONGARCH_CPU_TYPE(model, initfn) \
+    { \
+        .parent = TYPE_LOONGARCH_CPU, \
+        .instance_init = initfn, \
+        .name = LOONGARCH_CPU_TYPE_NAME(model), \
+    }
+
+static const TypeInfo loongarch_cpu_type_infos[] = {
+    {
+        .name = TYPE_LOONGARCH_CPU,
+        .parent = TYPE_CPU,
+        .instance_size = sizeof(LoongArchCPU),
+        .instance_init = loongarch_cpu_init,
+
+        .abstract = true,
+        .class_size = sizeof(LoongArchCPUClass),
+        .class_init = loongarch_cpu_class_init,
+    },
+    DEFINE_LOONGARCH_CPU_TYPE("la464", loongarch_la464_initfn),
+};
+
+DEFINE_TYPES(loongarch_cpu_type_infos)
diff --git a/target/loongarch/cpu.h b/target/loongarch/cpu.h
new file mode 100644
index 0000000000..3e8ba46377
--- /dev/null
+++ b/target/loongarch/cpu.h
@@ -0,0 +1,243 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * QEMU LoongArch CPU
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+#ifndef LOONGARCH_CPU_H
+#define LOONGARCH_CPU_H
+
+#include "exec/cpu-defs.h"
+#include "fpu/softfloat-types.h"
+#include "hw/registerfields.h"
+
+#define TCG_GUEST_DEFAULT_MO (0)
+
+#define FCSR0_M1    0x1f         /* FCSR1 mask, Enables */
+#define FCSR0_M2    0x1f1f0000   /* FCSR2 mask, Cause and Flags */
+#define FCSR0_M3    0x300        /* FCSR3 mask, Round Mode */
+#define FCSR0_RM    8            /* Round Mode bit num on fcsr0 */
+
+FIELD(FCSR0, ENABLES, 0, 5)
+FIELD(FCSR0, RM, 8, 2)
+FIELD(FCSR0, FLAGS, 16, 5)
+FIELD(FCSR0, CAUSE, 24, 5)
+
+#define GET_FP_CAUSE(REG)      FIELD_EX32(REG, FCSR0, CAUSE)
+#define SET_FP_CAUSE(REG, V)   FIELD_DP32(REG, FCSR0, CAUSE, V)
+#define GET_FP_ENABLES(REG)    FIELD_EX32(REG, FCSR0, ENABLES)
+#define SET_FP_ENABLES(REG, V) FIELD_DP32(REG, FCSR0, ENABLES, V)
+#define GET_FP_FLAGS(REG)      FIELD_EX32(REG, FCSR0, FLAGS)
+#define SET_FP_FLAGS(REG, V)   FIELD_DP32(REG, FCSR0, FLAGS, V)
+#define UPDATE_FP_FLAGS(REG, V) \
+    do { \
+        (REG) |= FIELD_DP32(0, FCSR0, FLAGS, V); \
+    } while (0)
+
+#define FP_INEXACT        1
+#define FP_UNDERFLOW      2
+#define FP_OVERFLOW       4
+#define FP_DIV0           8
+#define FP_INVALID        16
+
+#define  EXCCODE_EXTERNAL_INT   64   /* plus external interrupt number */
+#define  EXCCODE_INT                 0
+#define  EXCCODE_PIL                 1
+#define  EXCCODE_PIS                 2
+#define  EXCCODE_PIF                 3
+#define  EXCCODE_PME                 4
+#define  EXCCODE_PNR                 5
+#define  EXCCODE_PNX                 6
+#define  EXCCODE_PPI                 7
+#define  EXCCODE_ADEF                8 /* Different exception subcode */
+#define  EXCCODE_ADEM                8
+#define  EXCCODE_ALE                 9
+#define  EXCCODE_BCE                 10
+#define  EXCCODE_SYS                 11
+#define  EXCCODE_BRK                 12
+#define  EXCCODE_INE                 13
+#define  EXCCODE_IPE                 14
+#define  EXCCODE_FPD                 15
+#define  EXCCODE_SXD                 16
+#define  EXCCODE_ASXD                17
+#define  EXCCODE_FPE                 18 /* Different exception subcode */
+#define  EXCCODE_VFPE                18
+#define  EXCCODE_WPEF                19 /* Different exception subcode */
+#define  EXCCODE_WPEM                19
+#define  EXCCODE_BTD                 20
+#define  EXCCODE_BTE                 21
+#define  EXCCODE_DBP                 26 /* Reserved subcode used for debug */
+
+/* cpucfg[0] bits */
+FIELD(CPUCFG0, PRID, 0, 32)
+
+/* cpucfg[1] bits */
+FIELD(CPUCFG1, ARCH, 0, 2)
+FIELD(CPUCFG1, PGMMU, 2, 1)
+FIELD(CPUCFG1, IOCSR, 3, 1)
+FIELD(CPUCFG1, PALEN, 4, 8)
+FIELD(CPUCFG1, VALEN, 12, 8)
+FIELD(CPUCFG1, UAL, 20, 1)
+FIELD(CPUCFG1, RI, 21, 1)
+FIELD(CPUCFG1, EP, 22, 1)
+FIELD(CPUCFG1, RPLV, 23, 1)
+FIELD(CPUCFG1, HP, 24, 1)
+FIELD(CPUCFG1, IOCSR_BRD, 25, 1)
+FIELD(CPUCFG1, MSG_INT, 26, 1)
+
+/* cpucfg[2] bits */
+FIELD(CPUCFG2, FP, 0, 1)
+FIELD(CPUCFG2, FP_SP, 1, 1)
+FIELD(CPUCFG2, FP_DP, 2, 1)
+FIELD(CPUCFG2, FP_VER, 3, 3)
+FIELD(CPUCFG2, LSX, 6, 1)
+FIELD(CPUCFG2, LASX, 7, 1)
+FIELD(CPUCFG2, COMPLEX, 8, 1)
+FIELD(CPUCFG2, CRYPTO, 9, 1)
+FIELD(CPUCFG2, LVZ, 10, 1)
+FIELD(CPUCFG2, LVZ_VER, 11, 3)
+FIELD(CPUCFG2, LLFTP, 14, 1)
+FIELD(CPUCFG2, LLFTP_VER, 15, 3)
+FIELD(CPUCFG2, LBT_X86, 18, 1)
+FIELD(CPUCFG2, LBT_ARM, 19, 1)
+FIELD(CPUCFG2, LBT_MIPS, 20, 1)
+FIELD(CPUCFG2, LSPW, 21, 1)
+FIELD(CPUCFG2, LAM, 22, 1)
+
+/* cpucfg[3] bits */
+FIELD(CPUCFG3, CCDMA, 0, 1)
+FIELD(CPUCFG3, SFB, 1, 1)
+FIELD(CPUCFG3, UCACC, 2, 1)
+FIELD(CPUCFG3, LLEXC, 3, 1)
+FIELD(CPUCFG3, SCDLY, 4, 1)
+FIELD(CPUCFG3, LLDBAR, 5, 1)
+FIELD(CPUCFG3, ITLBHMC, 6, 1)
+FIELD(CPUCFG3, ICHMC, 7, 1)
+FIELD(CPUCFG3, SPW_LVL, 8, 3)
+FIELD(CPUCFG3, SPW_HP_HF, 11, 1)
+FIELD(CPUCFG3, RVA, 12, 1)
+FIELD(CPUCFG3, RVAMAX, 13, 4)
+
+/* cpucfg[4] bits */
+FIELD(CPUCFG4, CC_FREQ, 0, 32)
+
+/* cpucfg[5] bits */
+FIELD(CPUCFG5, CC_MUL, 0, 16)
+FIELD(CPUCFG5, CC_DIV, 16, 16)
+
+/* cpucfg[6] bits */
+FIELD(CPUCFG6, PMP, 0, 1)
+FIELD(CPUCFG6, PMVER, 1, 3)
+FIELD(CPUCFG6, PMNUM, 4, 4)
+FIELD(CPUCFG6, PMBITS, 8, 6)
+FIELD(CPUCFG6, UPM, 14, 1)
+
+/* cpucfg[16] bits */
+FIELD(CPUCFG16, L1_IUPRE, 0, 1)
+FIELD(CPUCFG16, L1_IUUNIFY, 1, 1)
+FIELD(CPUCFG16, L1_DPRE, 2, 1)
+FIELD(CPUCFG16, L2_IUPRE, 3, 1)
+FIELD(CPUCFG16, L2_IUUNIFY, 4, 1)
+FIELD(CPUCFG16, L2_IUPRIV, 5, 1)
+FIELD(CPUCFG16, L2_IUINCL, 6, 1)
+FIELD(CPUCFG16, L2_DPRE, 7, 1)
+FIELD(CPUCFG16, L2_DPRIV, 8, 1)
+FIELD(CPUCFG16, L2_DINCL, 9, 1)
+FIELD(CPUCFG16, L3_IUPRE, 10, 1)
+FIELD(CPUCFG16, L3_IUUNIFY, 11, 1)
+FIELD(CPUCFG16, L3_IUPRIV, 12, 1)
+FIELD(CPUCFG16, L3_IUINCL, 13, 1)
+FIELD(CPUCFG16, L3_DPRE, 14, 1)
+FIELD(CPUCFG16, L3_DPRIV, 15, 1)
+FIELD(CPUCFG16, L3_DINCL, 16, 1)
+
+/* cpucfg[17] bits */
+FIELD(CPUCFG17, L1IU_WAYS, 0, 16)
+FIELD(CPUCFG17, L1IU_SETS, 16, 8)
+FIELD(CPUCFG17, L1IU_SIZE, 24, 7)
+
+/* cpucfg[18] bits */
+FIELD(CPUCFG18, L1D_WAYS, 0, 16)
+FIELD(CPUCFG18, L1D_SETS, 16, 8)
+FIELD(CPUCFG18, L1D_SIZE, 24, 7)
+
+/* cpucfg[19] bits */
+FIELD(CPUCFG19, L2IU_WAYS, 0, 16)
+FIELD(CPUCFG19, L2IU_SETS, 16, 8)
+FIELD(CPUCFG19, L2IU_SIZE, 24, 7)
+
+/* cpucfg[20] bits */
+FIELD(CPUCFG20, L3IU_WAYS, 0, 16)
+FIELD(CPUCFG20, L3IU_SETS, 16, 8)
+FIELD(CPUCFG20, L3IU_SIZE, 24, 7)
+
+extern const char * const regnames[32];
+extern const char * const fregnames[32];
+
+typedef struct CPUArchState {
+    uint64_t gpr[32];
+    uint64_t pc;
+
+    uint64_t fpr[32];
+    float_status fp_status;
+    bool cf[8];
+
+    uint32_t fcsr0;
+    uint32_t fcsr0_mask;
+
+    uint32_t cpucfg[21];
+
+    uint64_t lladdr; /* LL virtual address compared against SC */
+    uint64_t llval;
+
+    uint64_t badaddr;
+} CPULoongArchState;
+
+/**
+ * LoongArchCPU:
+ * @env: #CPULoongArchState
+ *
+ * A LoongArch CPU.
+ */
+struct ArchCPU {
+    /*< private >*/
+    CPUState parent_obj;
+    /*< public >*/
+
+    CPUNegativeOffsetState neg;
+    CPULoongArchState env;
+};
+
+#define TYPE_LOONGARCH_CPU "loongarch-cpu"
+
+OBJECT_DECLARE_CPU_TYPE(LoongArchCPU, LoongArchCPUClass,
+                        LOONGARCH_CPU)
+
+/**
+ * LoongArchCPUClass:
+ * @parent_realize: The parent class' realize handler.
+ * @parent_reset: The parent class' reset handler.
+ *
+ * A LoongArch CPU model.
+ */
+struct LoongArchCPUClass {
+    /*< private >*/
+    CPUClass parent_class;
+    /*< public >*/
+
+    DeviceRealize parent_realize;
+    DeviceReset parent_reset;
+};
+
+void loongarch_cpu_list(void);
+
+#define cpu_list loongarch_cpu_list
+
+#include "exec/cpu-all.h"
+
+#define LOONGARCH_CPU_TYPE_SUFFIX "-" TYPE_LOONGARCH_CPU
+#define LOONGARCH_CPU_TYPE_NAME(model) model LOONGARCH_CPU_TYPE_SUFFIX
+#define CPU_RESOLVING_TYPE TYPE_LOONGARCH_CPU
+
+#endif /* LOONGARCH_CPU_H */
diff --git a/target/loongarch/internals.h b/target/loongarch/internals.h
new file mode 100644
index 0000000000..9f02287a90
--- /dev/null
+++ b/target/loongarch/internals.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * QEMU LoongArch CPU -- internal functions and types
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+#ifndef LOONGARCH_INTERNALS_H
+#define LOONGARCH_INTERNALS_H
+
+void loongarch_translate_init(void);
+
+void loongarch_cpu_dump_state(CPUState *cpu, FILE *f, int flags);
+
+void G_NORETURN do_raise_exception(CPULoongArchState *env,
+                                   uint32_t exception,
+                                   uintptr_t pc);
+
+const char *loongarch_exception_name(int32_t exception);
+
+#endif

From f8da88d78f71da751f089913784fa884df31bd03 Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:42:53 +0800
Subject: [PATCH 810/935] target/loongarch: Add main translation routines

This patch adds main translation routines and
basic functions for translation.

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-4-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/helper.h    |   6 ++
 target/loongarch/op_helper.c |  21 +++++
 target/loongarch/translate.c | 161 +++++++++++++++++++++++++++++++++++
 target/loongarch/translate.h |  26 ++++++
 4 files changed, 214 insertions(+)
 create mode 100644 target/loongarch/helper.h
 create mode 100644 target/loongarch/op_helper.c
 create mode 100644 target/loongarch/translate.c
 create mode 100644 target/loongarch/translate.h

diff --git a/target/loongarch/helper.h b/target/loongarch/helper.h
new file mode 100644
index 0000000000..eb771c0628
--- /dev/null
+++ b/target/loongarch/helper.h
@@ -0,0 +1,6 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+DEF_HELPER_2(raise_exception, noreturn, env, i32)
diff --git a/target/loongarch/op_helper.c b/target/loongarch/op_helper.c
new file mode 100644
index 0000000000..903810951e
--- /dev/null
+++ b/target/loongarch/op_helper.c
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * LoongArch emulation helpers for QEMU.
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/main-loop.h"
+#include "cpu.h"
+#include "qemu/host-utils.h"
+#include "exec/helper-proto.h"
+#include "exec/exec-all.h"
+#include "exec/cpu_ldst.h"
+#include "internals.h"
+
+/* Exceptions helpers */
+void helper_raise_exception(CPULoongArchState *env, uint32_t exception)
+{
+    do_raise_exception(env, exception, GETPC());
+}
diff --git a/target/loongarch/translate.c b/target/loongarch/translate.c
new file mode 100644
index 0000000000..332a6d2d58
--- /dev/null
+++ b/target/loongarch/translate.c
@@ -0,0 +1,161 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * LoongArch emulation for QEMU - main translation routines.
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "tcg/tcg-op.h"
+#include "exec/translator.h"
+#include "exec/helper-proto.h"
+#include "exec/helper-gen.h"
+
+#include "exec/translator.h"
+#include "exec/log.h"
+#include "qemu/qemu-print.h"
+#include "translate.h"
+#include "internals.h"
+
+/* Global register indices */
+TCGv cpu_gpr[32], cpu_pc;
+static TCGv cpu_lladdr, cpu_llval;
+TCGv_i32 cpu_fcsr0;
+TCGv_i64 cpu_fpr[32];
+
+#define DISAS_STOP       DISAS_TARGET_0
+
+void generate_exception(DisasContext *ctx, int excp)
+{
+    tcg_gen_movi_tl(cpu_pc, ctx->base.pc_next);
+    gen_helper_raise_exception(cpu_env, tcg_constant_i32(excp));
+    ctx->base.is_jmp = DISAS_NORETURN;
+}
+
+static inline void gen_goto_tb(DisasContext *ctx, int n, target_ulong dest)
+{
+    if (translator_use_goto_tb(&ctx->base, dest)) {
+        tcg_gen_goto_tb(n);
+        tcg_gen_movi_tl(cpu_pc, dest);
+        tcg_gen_exit_tb(ctx->base.tb, n);
+    } else {
+        tcg_gen_movi_tl(cpu_pc, dest);
+        tcg_gen_lookup_and_goto_ptr();
+    }
+}
+
+static void loongarch_tr_init_disas_context(DisasContextBase *dcbase,
+                                            CPUState *cs)
+{
+    int64_t bound;
+    DisasContext *ctx = container_of(dcbase, DisasContext, base);
+
+    ctx->page_start = ctx->base.pc_first & TARGET_PAGE_MASK;
+    ctx->mem_idx = ctx->base.tb->flags;
+
+    /* Bound the number of insns to execute to those left on the page.  */
+    bound = -(ctx->base.pc_first | TARGET_PAGE_MASK) / 4;
+    ctx->base.max_insns = MIN(ctx->base.max_insns, bound);
+}
+
+static void loongarch_tr_tb_start(DisasContextBase *dcbase, CPUState *cs)
+{
+}
+
+static void loongarch_tr_insn_start(DisasContextBase *dcbase, CPUState *cs)
+{
+    DisasContext *ctx = container_of(dcbase, DisasContext, base);
+
+    tcg_gen_insn_start(ctx->base.pc_next);
+}
+
+static void loongarch_tr_translate_insn(DisasContextBase *dcbase, CPUState *cs)
+{
+    CPULoongArchState *env = cs->env_ptr;
+    DisasContext *ctx = container_of(dcbase, DisasContext, base);
+
+    ctx->opcode = cpu_ldl_code(env, ctx->base.pc_next);
+
+    if (!decode(ctx, ctx->opcode)) {
+        qemu_log_mask(LOG_UNIMP, "Error: unknown opcode. "
+                      TARGET_FMT_lx ": 0x%x\n",
+                      ctx->base.pc_next, ctx->opcode);
+        generate_exception(ctx, EXCCODE_INE);
+    }
+
+    ctx->base.pc_next += 4;
+}
+
+static void loongarch_tr_tb_stop(DisasContextBase *dcbase, CPUState *cs)
+{
+    DisasContext *ctx = container_of(dcbase, DisasContext, base);
+
+    switch (ctx->base.is_jmp) {
+    case DISAS_STOP:
+        tcg_gen_movi_tl(cpu_pc, ctx->base.pc_next);
+        tcg_gen_lookup_and_goto_ptr();
+        break;
+    case DISAS_TOO_MANY:
+        gen_goto_tb(ctx, 0, ctx->base.pc_next);
+        break;
+    case DISAS_NORETURN:
+        break;
+    default:
+        g_assert_not_reached();
+    }
+}
+
+static void loongarch_tr_disas_log(const DisasContextBase *dcbase,
+                                   CPUState *cpu, FILE *logfile)
+{
+    qemu_log("IN: %s\n", lookup_symbol(dcbase->pc_first));
+    target_disas(logfile, cpu, dcbase->pc_first, dcbase->tb->size);
+}
+
+static const TranslatorOps loongarch_tr_ops = {
+    .init_disas_context = loongarch_tr_init_disas_context,
+    .tb_start           = loongarch_tr_tb_start,
+    .insn_start         = loongarch_tr_insn_start,
+    .translate_insn     = loongarch_tr_translate_insn,
+    .tb_stop            = loongarch_tr_tb_stop,
+    .disas_log          = loongarch_tr_disas_log,
+};
+
+void gen_intermediate_code(CPUState *cs, TranslationBlock *tb, int max_insns)
+{
+    DisasContext ctx;
+
+    translator_loop(&loongarch_tr_ops, &ctx.base, cs, tb, max_insns);
+}
+
+void loongarch_translate_init(void)
+{
+    int i;
+
+    cpu_gpr[0] = NULL;
+    for (i = 1; i < 32; i++) {
+        cpu_gpr[i] = tcg_global_mem_new(cpu_env,
+                                        offsetof(CPULoongArchState, gpr[i]),
+                                        regnames[i]);
+    }
+
+    for (i = 0; i < 32; i++) {
+        int off = offsetof(CPULoongArchState, fpr[i]);
+        cpu_fpr[i] = tcg_global_mem_new_i64(cpu_env, off, fregnames[i]);
+    }
+
+    cpu_pc = tcg_global_mem_new(cpu_env, offsetof(CPULoongArchState, pc), "pc");
+    cpu_fcsr0 = tcg_global_mem_new_i32(cpu_env,
+                    offsetof(CPULoongArchState, fcsr0), "fcsr0");
+    cpu_lladdr = tcg_global_mem_new(cpu_env,
+                    offsetof(CPULoongArchState, lladdr), "lladdr");
+    cpu_llval = tcg_global_mem_new(cpu_env,
+                    offsetof(CPULoongArchState, llval), "llval");
+}
+
+void restore_state_to_opc(CPULoongArchState *env, TranslationBlock *tb,
+                          target_ulong *data)
+{
+    env->pc = data[0];
+}
diff --git a/target/loongarch/translate.h b/target/loongarch/translate.h
new file mode 100644
index 0000000000..6cc7f1a7cd
--- /dev/null
+++ b/target/loongarch/translate.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * LoongArch translation routines.
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+#ifndef TARGET_LOONGARCH_TRANSLATE_H
+#define TARGET_LOONGARCH_TRANSLATE_H
+
+#include "exec/translator.h"
+
+typedef struct DisasContext {
+    DisasContextBase base;
+    target_ulong page_start;
+    uint32_t opcode;
+    int mem_idx;
+} DisasContext;
+
+void generate_exception(DisasContext *ctx, int excp);
+
+extern TCGv cpu_gpr[32], cpu_pc;
+extern TCGv_i32 cpu_fscr0;
+extern TCGv_i64 cpu_fpr[32];
+
+#endif

From 143d6785ef725aff6d09bc112c21119c32d9d0ae Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:42:54 +0800
Subject: [PATCH 811/935] target/loongarch: Add fixed point arithmetic
 instruction translation

This includes:
- ADD.{W/D}, SUB.{W/D}
- ADDI.{W/D}, ADDU16ID
- ALSL.{W[U]/D}
- LU12I.W, LU32I.D LU52I.D
- SLT[U], SLT[U]I
- PCADDI, PCADDU12I, PCADDU18I, PCALAU12I
- AND, OR, NOR, XOR, ANDN, ORN
- MUL.{W/D}, MULH.{W[U]/D[U]}
- MULW.D.W[U]
- DIV.{W[U]/D[U]}, MOD.{W[U]/D[U]}
- ANDI, ORI, XORI

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-5-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/insn_trans/trans_arith.c.inc | 304 ++++++++++++++++++
 target/loongarch/insns.decode                 |  79 +++++
 target/loongarch/translate.c                  |  83 +++++
 target/loongarch/translate.h                  |  19 ++
 4 files changed, 485 insertions(+)
 create mode 100644 target/loongarch/insn_trans/trans_arith.c.inc
 create mode 100644 target/loongarch/insns.decode

diff --git a/target/loongarch/insn_trans/trans_arith.c.inc b/target/loongarch/insn_trans/trans_arith.c.inc
new file mode 100644
index 0000000000..8e45eadbc8
--- /dev/null
+++ b/target/loongarch/insn_trans/trans_arith.c.inc
@@ -0,0 +1,304 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+static bool gen_rrr(DisasContext *ctx, arg_rrr *a,
+                    DisasExtend src1_ext, DisasExtend src2_ext,
+                    DisasExtend dst_ext, void (*func)(TCGv, TCGv, TCGv))
+{
+    TCGv dest = gpr_dst(ctx, a->rd, dst_ext);
+    TCGv src1 = gpr_src(ctx, a->rj, src1_ext);
+    TCGv src2 = gpr_src(ctx, a->rk, src2_ext);
+
+    func(dest, src1, src2);
+    gen_set_gpr(a->rd, dest, dst_ext);
+
+    return true;
+}
+
+static bool gen_rri_v(DisasContext *ctx, arg_rr_i *a,
+                      DisasExtend src_ext, DisasExtend dst_ext,
+                      void (*func)(TCGv, TCGv, TCGv))
+{
+    TCGv dest = gpr_dst(ctx, a->rd, dst_ext);
+    TCGv src1 = gpr_src(ctx, a->rj, src_ext);
+    TCGv src2 = tcg_constant_tl(a->imm);
+
+    func(dest, src1, src2);
+    gen_set_gpr(a->rd, dest, dst_ext);
+
+    return true;
+}
+
+static bool gen_rri_c(DisasContext *ctx, arg_rr_i *a,
+                      DisasExtend src_ext, DisasExtend dst_ext,
+                      void (*func)(TCGv, TCGv, target_long))
+{
+    TCGv dest = gpr_dst(ctx, a->rd, dst_ext);
+    TCGv src1 = gpr_src(ctx, a->rj, src_ext);
+
+    func(dest, src1, a->imm);
+    gen_set_gpr(a->rd, dest, dst_ext);
+
+    return true;
+}
+
+static bool gen_rrr_sa(DisasContext *ctx, arg_rrr_sa *a,
+                       DisasExtend src_ext, DisasExtend dst_ext,
+                       void (*func)(TCGv, TCGv, TCGv, target_long))
+{
+    TCGv dest = gpr_dst(ctx, a->rd, dst_ext);
+    TCGv src1 = gpr_src(ctx, a->rj, src_ext);
+    TCGv src2 = gpr_src(ctx, a->rk, src_ext);
+
+    func(dest, src1, src2, a->sa);
+    gen_set_gpr(a->rd, dest, dst_ext);
+
+    return true;
+}
+
+static bool trans_lu12i_w(DisasContext *ctx, arg_lu12i_w *a)
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+
+    tcg_gen_movi_tl(dest, a->imm << 12);
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+
+    return true;
+}
+
+static bool gen_pc(DisasContext *ctx, arg_r_i *a,
+                   target_ulong (*func)(target_ulong, int))
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+    target_ulong addr = func(ctx->base.pc_next, a->imm);
+
+    tcg_gen_movi_tl(dest, addr);
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+
+    return true;
+}
+
+static void gen_slt(TCGv dest, TCGv src1, TCGv src2)
+{
+    tcg_gen_setcond_tl(TCG_COND_LT, dest, src1, src2);
+}
+
+static void gen_sltu(TCGv dest, TCGv src1, TCGv src2)
+{
+    tcg_gen_setcond_tl(TCG_COND_LTU, dest, src1, src2);
+}
+
+static void gen_mulh_w(TCGv dest, TCGv src1, TCGv src2)
+{
+    tcg_gen_mul_i64(dest, src1, src2);
+    tcg_gen_sari_i64(dest, dest, 32);
+}
+
+static void gen_mulh_d(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv discard = tcg_temp_new();
+    tcg_gen_muls2_tl(discard, dest, src1, src2);
+    tcg_temp_free(discard);
+}
+
+static void gen_mulh_du(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv discard = tcg_temp_new();
+    tcg_gen_mulu2_tl(discard, dest, src1, src2);
+    tcg_temp_free(discard);
+}
+
+static void prep_divisor_d(TCGv ret, TCGv src1, TCGv src2)
+{
+    TCGv t0 = tcg_temp_new();
+    TCGv t1 = tcg_temp_new();
+    TCGv zero = tcg_constant_tl(0);
+
+    /*
+     * If min / -1, set the divisor to 1.
+     * This avoids potential host overflow trap and produces min.
+     * If x / 0, set the divisor to 1.
+     * This avoids potential host overflow trap;
+     * the required result is undefined.
+     */
+    tcg_gen_setcondi_tl(TCG_COND_EQ, ret, src1, INT64_MIN);
+    tcg_gen_setcondi_tl(TCG_COND_EQ, t0, src2, -1);
+    tcg_gen_setcondi_tl(TCG_COND_EQ, t1, src2, 0);
+    tcg_gen_and_tl(ret, ret, t0);
+    tcg_gen_or_tl(ret, ret, t1);
+    tcg_gen_movcond_tl(TCG_COND_NE, ret, ret, zero, ret, src2);
+
+    tcg_temp_free(t0);
+    tcg_temp_free(t1);
+}
+
+static void prep_divisor_du(TCGv ret, TCGv src2)
+{
+    TCGv zero = tcg_constant_tl(0);
+    TCGv one = tcg_constant_tl(1);
+
+    /*
+     * If x / 0, set the divisor to 1.
+     * This avoids potential host overflow trap;
+     * the required result is undefined.
+     */
+    tcg_gen_movcond_tl(TCG_COND_EQ, ret, src2, zero, one, src2);
+}
+
+static void gen_div_d(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv t0 = tcg_temp_new();
+    prep_divisor_d(t0, src1, src2);
+    tcg_gen_div_tl(dest, src1, t0);
+    tcg_temp_free(t0);
+}
+
+static void gen_rem_d(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv t0 = tcg_temp_new();
+    prep_divisor_d(t0, src1, src2);
+    tcg_gen_rem_tl(dest, src1, t0);
+    tcg_temp_free(t0);
+}
+
+static void gen_div_du(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv t0 = tcg_temp_new();
+    prep_divisor_du(t0, src2);
+    tcg_gen_divu_tl(dest, src1, t0);
+    tcg_temp_free(t0);
+}
+
+static void gen_rem_du(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv t0 = tcg_temp_new();
+    prep_divisor_du(t0, src2);
+    tcg_gen_remu_tl(dest, src1, t0);
+    tcg_temp_free(t0);
+}
+
+static void gen_div_w(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv t0 = tcg_temp_new();
+    /* We need not check for integer overflow for div_w. */
+    prep_divisor_du(t0, src2);
+    tcg_gen_div_tl(dest, src1, t0);
+    tcg_temp_free(t0);
+}
+
+static void gen_rem_w(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv t0 = tcg_temp_new();
+    /* We need not check for integer overflow for rem_w. */
+    prep_divisor_du(t0, src2);
+    tcg_gen_rem_tl(dest, src1, t0);
+    tcg_temp_free(t0);
+}
+
+static void gen_alsl(TCGv dest, TCGv src1, TCGv src2, target_long sa)
+{
+    TCGv t0 = tcg_temp_new();
+    tcg_gen_shli_tl(t0, src1, sa);
+    tcg_gen_add_tl(dest, t0, src2);
+    tcg_temp_free(t0);
+}
+
+static bool trans_lu32i_d(DisasContext *ctx, arg_lu32i_d *a)
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+    TCGv src1 = gpr_src(ctx, a->rd, EXT_NONE);
+    TCGv src2 = tcg_constant_tl(a->imm);
+
+    tcg_gen_deposit_tl(dest, src1, src2, 32, 32);
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+
+    return true;
+}
+
+static bool trans_lu52i_d(DisasContext *ctx, arg_lu52i_d *a)
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = tcg_constant_tl(a->imm);
+
+    tcg_gen_deposit_tl(dest, src1, src2, 52, 12);
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+
+    return true;
+}
+
+static target_ulong gen_pcaddi(target_ulong pc, int imm)
+{
+    return pc + (imm << 2);
+}
+
+static target_ulong gen_pcalau12i(target_ulong pc, int imm)
+{
+    return (pc + (imm << 12)) & ~0xfff;
+}
+
+static target_ulong gen_pcaddu12i(target_ulong pc, int imm)
+{
+    return pc + (imm << 12);
+}
+
+static target_ulong gen_pcaddu18i(target_ulong pc, int imm)
+{
+    return pc + ((target_ulong)(imm) << 18);
+}
+
+static bool trans_addu16i_d(DisasContext *ctx, arg_addu16i_d *a)
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+
+    tcg_gen_addi_tl(dest, src1, a->imm << 16);
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+
+    return true;
+}
+
+TRANS(add_w, gen_rrr, EXT_NONE, EXT_NONE, EXT_SIGN, tcg_gen_add_tl)
+TRANS(add_d, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, tcg_gen_add_tl)
+TRANS(sub_w, gen_rrr, EXT_NONE, EXT_NONE, EXT_SIGN, tcg_gen_sub_tl)
+TRANS(sub_d, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, tcg_gen_sub_tl)
+TRANS(and, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, tcg_gen_and_tl)
+TRANS(or, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, tcg_gen_or_tl)
+TRANS(xor, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, tcg_gen_xor_tl)
+TRANS(nor, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, tcg_gen_nor_tl)
+TRANS(andn, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, tcg_gen_andc_tl)
+TRANS(orn, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, tcg_gen_orc_tl)
+TRANS(slt, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, gen_slt)
+TRANS(sltu, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, gen_sltu)
+TRANS(mul_w, gen_rrr, EXT_SIGN, EXT_SIGN, EXT_SIGN, tcg_gen_mul_tl)
+TRANS(mul_d, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, tcg_gen_mul_tl)
+TRANS(mulh_w, gen_rrr, EXT_SIGN, EXT_SIGN, EXT_NONE, gen_mulh_w)
+TRANS(mulh_wu, gen_rrr, EXT_ZERO, EXT_ZERO, EXT_NONE, gen_mulh_w)
+TRANS(mulh_d, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, gen_mulh_d)
+TRANS(mulh_du, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, gen_mulh_du)
+TRANS(mulw_d_w, gen_rrr, EXT_SIGN, EXT_SIGN, EXT_NONE, tcg_gen_mul_tl)
+TRANS(mulw_d_wu, gen_rrr, EXT_ZERO, EXT_ZERO, EXT_NONE, tcg_gen_mul_tl)
+TRANS(div_w, gen_rrr, EXT_SIGN, EXT_SIGN, EXT_SIGN, gen_div_w)
+TRANS(mod_w, gen_rrr, EXT_SIGN, EXT_SIGN, EXT_SIGN, gen_rem_w)
+TRANS(div_wu, gen_rrr, EXT_ZERO, EXT_ZERO, EXT_SIGN, gen_div_du)
+TRANS(mod_wu, gen_rrr, EXT_ZERO, EXT_ZERO, EXT_SIGN, gen_rem_du)
+TRANS(div_d, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, gen_div_d)
+TRANS(mod_d, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, gen_rem_d)
+TRANS(div_du, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, gen_div_du)
+TRANS(mod_du, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, gen_rem_du)
+TRANS(slti, gen_rri_v, EXT_NONE, EXT_NONE, gen_slt)
+TRANS(sltui, gen_rri_v, EXT_NONE, EXT_NONE, gen_sltu)
+TRANS(addi_w, gen_rri_c, EXT_NONE, EXT_SIGN, tcg_gen_addi_tl)
+TRANS(addi_d, gen_rri_c, EXT_NONE, EXT_NONE, tcg_gen_addi_tl)
+TRANS(alsl_w, gen_rrr_sa, EXT_NONE, EXT_SIGN, gen_alsl)
+TRANS(alsl_wu, gen_rrr_sa, EXT_NONE, EXT_ZERO, gen_alsl)
+TRANS(alsl_d, gen_rrr_sa, EXT_NONE, EXT_NONE, gen_alsl)
+TRANS(pcaddi, gen_pc, gen_pcaddi)
+TRANS(pcalau12i, gen_pc, gen_pcalau12i)
+TRANS(pcaddu12i, gen_pc, gen_pcaddu12i)
+TRANS(pcaddu18i, gen_pc, gen_pcaddu18i)
+TRANS(andi, gen_rri_c, EXT_NONE, EXT_NONE, tcg_gen_andi_tl)
+TRANS(ori, gen_rri_c, EXT_NONE, EXT_NONE, tcg_gen_ori_tl)
+TRANS(xori, gen_rri_c, EXT_NONE, EXT_NONE, tcg_gen_xori_tl)
diff --git a/target/loongarch/insns.decode b/target/loongarch/insns.decode
new file mode 100644
index 0000000000..8579c11984
--- /dev/null
+++ b/target/loongarch/insns.decode
@@ -0,0 +1,79 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# LoongArch instruction decode definitions.
+#
+# Copyright (c) 2021 Loongson Technology Corporation Limited
+#
+
+#
+# Fields
+#
+%sa2p1     15:2         !function=plus_1
+
+#
+# Argument sets
+#
+&r_i          rd imm
+&rrr          rd rj rk
+&rr_i         rd rj imm
+&rrr_sa       rd rj rk sa
+
+#
+# Formats
+#
+@rrr               .... ........ ..... rk:5 rj:5 rd:5    &rrr
+@r_i20                          .... ... imm:s20 rd:5    &r_i
+@rr_i12                 .... ...... imm:s12 rj:5 rd:5    &rr_i
+@rr_ui12                 .... ...... imm:12 rj:5 rd:5    &rr_i
+@rr_i16                     .... .. imm:s16 rj:5 rd:5    &rr_i
+@rrr_sa2p1        .... ........ ... .. rk:5 rj:5 rd:5    &rrr_sa  sa=%sa2p1
+
+#
+# Fixed point arithmetic operation instruction
+#
+add_w           0000 00000001 00000 ..... ..... .....    @rrr
+add_d           0000 00000001 00001 ..... ..... .....    @rrr
+sub_w           0000 00000001 00010 ..... ..... .....    @rrr
+sub_d           0000 00000001 00011 ..... ..... .....    @rrr
+slt             0000 00000001 00100 ..... ..... .....    @rrr
+sltu            0000 00000001 00101 ..... ..... .....    @rrr
+slti            0000 001000 ............ ..... .....     @rr_i12
+sltui           0000 001001 ............ ..... .....     @rr_i12
+nor             0000 00000001 01000 ..... ..... .....    @rrr
+and             0000 00000001 01001 ..... ..... .....    @rrr
+or              0000 00000001 01010 ..... ..... .....    @rrr
+xor             0000 00000001 01011 ..... ..... .....    @rrr
+orn             0000 00000001 01100 ..... ..... .....    @rrr
+andn            0000 00000001 01101 ..... ..... .....    @rrr
+mul_w           0000 00000001 11000 ..... ..... .....    @rrr
+mulh_w          0000 00000001 11001 ..... ..... .....    @rrr
+mulh_wu         0000 00000001 11010 ..... ..... .....    @rrr
+mul_d           0000 00000001 11011 ..... ..... .....    @rrr
+mulh_d          0000 00000001 11100 ..... ..... .....    @rrr
+mulh_du         0000 00000001 11101 ..... ..... .....    @rrr
+mulw_d_w        0000 00000001 11110 ..... ..... .....    @rrr
+mulw_d_wu       0000 00000001 11111 ..... ..... .....    @rrr
+div_w           0000 00000010 00000 ..... ..... .....    @rrr
+mod_w           0000 00000010 00001 ..... ..... .....    @rrr
+div_wu          0000 00000010 00010 ..... ..... .....    @rrr
+mod_wu          0000 00000010 00011 ..... ..... .....    @rrr
+div_d           0000 00000010 00100 ..... ..... .....    @rrr
+mod_d           0000 00000010 00101 ..... ..... .....    @rrr
+div_du          0000 00000010 00110 ..... ..... .....    @rrr
+mod_du          0000 00000010 00111 ..... ..... .....    @rrr
+alsl_w          0000 00000000 010 .. ..... ..... .....   @rrr_sa2p1
+alsl_wu         0000 00000000 011 .. ..... ..... .....   @rrr_sa2p1
+alsl_d          0000 00000010 110 .. ..... ..... .....   @rrr_sa2p1
+lu12i_w         0001 010 .................... .....      @r_i20
+lu32i_d         0001 011 .................... .....      @r_i20
+lu52i_d         0000 001100 ............ ..... .....     @rr_i12
+pcaddi          0001 100 .................... .....      @r_i20
+pcalau12i       0001 101 .................... .....      @r_i20
+pcaddu12i       0001 110 .................... .....      @r_i20
+pcaddu18i       0001 111 .................... .....      @r_i20
+addi_w          0000 001010 ............ ..... .....     @rr_i12
+addi_d          0000 001011 ............ ..... .....     @rr_i12
+addu16i_d       0001 00 ................ ..... .....     @rr_i16
+andi            0000 001101 ............ ..... .....     @rr_ui12
+ori             0000 001110 ............ ..... .....     @rr_ui12
+xori            0000 001111 ............ ..... .....     @rr_ui12
diff --git a/target/loongarch/translate.c b/target/loongarch/translate.c
index 332a6d2d58..65c7dfd3da 100644
--- a/target/loongarch/translate.c
+++ b/target/loongarch/translate.c
@@ -26,6 +26,11 @@ TCGv_i64 cpu_fpr[32];
 
 #define DISAS_STOP       DISAS_TARGET_0
 
+static inline int plus_1(DisasContext *ctx, int x)
+{
+    return x + 1;
+}
+
 void generate_exception(DisasContext *ctx, int excp)
 {
     tcg_gen_movi_tl(cpu_pc, ctx->base.pc_next);
@@ -57,6 +62,11 @@ static void loongarch_tr_init_disas_context(DisasContextBase *dcbase,
     /* Bound the number of insns to execute to those left on the page.  */
     bound = -(ctx->base.pc_first | TARGET_PAGE_MASK) / 4;
     ctx->base.max_insns = MIN(ctx->base.max_insns, bound);
+
+    ctx->ntemp = 0;
+    memset(ctx->temp, 0, sizeof(ctx->temp));
+
+    ctx->zero = tcg_constant_tl(0);
 }
 
 static void loongarch_tr_tb_start(DisasContextBase *dcbase, CPUState *cs)
@@ -70,6 +80,73 @@ static void loongarch_tr_insn_start(DisasContextBase *dcbase, CPUState *cs)
     tcg_gen_insn_start(ctx->base.pc_next);
 }
 
+/*
+ * Wrappers for getting reg values.
+ *
+ * The $zero register does not have cpu_gpr[0] allocated -- we supply the
+ * constant zero as a source, and an uninitialized sink as destination.
+ *
+ * Further, we may provide an extension for word operations.
+ */
+static TCGv temp_new(DisasContext *ctx)
+{
+    assert(ctx->ntemp < ARRAY_SIZE(ctx->temp));
+    return ctx->temp[ctx->ntemp++] = tcg_temp_new();
+}
+
+static TCGv gpr_src(DisasContext *ctx, int reg_num, DisasExtend src_ext)
+{
+    TCGv t;
+
+    if (reg_num == 0) {
+        return ctx->zero;
+    }
+
+    switch (src_ext) {
+    case EXT_NONE:
+        return cpu_gpr[reg_num];
+    case EXT_SIGN:
+        t = temp_new(ctx);
+        tcg_gen_ext32s_tl(t, cpu_gpr[reg_num]);
+        return t;
+    case EXT_ZERO:
+        t = temp_new(ctx);
+        tcg_gen_ext32u_tl(t, cpu_gpr[reg_num]);
+        return t;
+    }
+    g_assert_not_reached();
+}
+
+static TCGv gpr_dst(DisasContext *ctx, int reg_num, DisasExtend dst_ext)
+{
+    if (reg_num == 0 || dst_ext) {
+        return temp_new(ctx);
+    }
+    return cpu_gpr[reg_num];
+}
+
+static void gen_set_gpr(int reg_num, TCGv t, DisasExtend dst_ext)
+{
+    if (reg_num != 0) {
+        switch (dst_ext) {
+        case EXT_NONE:
+            tcg_gen_mov_tl(cpu_gpr[reg_num], t);
+            break;
+        case EXT_SIGN:
+            tcg_gen_ext32s_tl(cpu_gpr[reg_num], t);
+            break;
+        case EXT_ZERO:
+            tcg_gen_ext32u_tl(cpu_gpr[reg_num], t);
+            break;
+        default:
+            g_assert_not_reached();
+        }
+    }
+}
+
+#include "decode-insns.c.inc"
+#include "insn_trans/trans_arith.c.inc"
+
 static void loongarch_tr_translate_insn(DisasContextBase *dcbase, CPUState *cs)
 {
     CPULoongArchState *env = cs->env_ptr;
@@ -84,6 +161,12 @@ static void loongarch_tr_translate_insn(DisasContextBase *dcbase, CPUState *cs)
         generate_exception(ctx, EXCCODE_INE);
     }
 
+    for (int i = ctx->ntemp - 1; i >= 0; --i) {
+        tcg_temp_free(ctx->temp[i]);
+        ctx->temp[i] = NULL;
+    }
+    ctx->ntemp = 0;
+
     ctx->base.pc_next += 4;
 }
 
diff --git a/target/loongarch/translate.h b/target/loongarch/translate.h
index 6cc7f1a7cd..9cc12512d1 100644
--- a/target/loongarch/translate.h
+++ b/target/loongarch/translate.h
@@ -10,11 +10,30 @@
 
 #include "exec/translator.h"
 
+#define TRANS(NAME, FUNC, ...) \
+    static bool trans_##NAME(DisasContext *ctx, arg_##NAME * a) \
+    { return FUNC(ctx, a, __VA_ARGS__); }
+
+/*
+ * If an operation is being performed on less than TARGET_LONG_BITS,
+ * it may require the inputs to be sign- or zero-extended; which will
+ * depend on the exact operation being performed.
+ */
+typedef enum {
+    EXT_NONE,
+    EXT_SIGN,
+    EXT_ZERO,
+} DisasExtend;
+
 typedef struct DisasContext {
     DisasContextBase base;
     target_ulong page_start;
     uint32_t opcode;
     int mem_idx;
+    TCGv zero;
+    /* Space for 3 operands plus 1 extra for address computation. */
+    TCGv temp[4];
+    uint8_t ntemp;
 } DisasContext;
 
 void generate_exception(DisasContext *ctx, int excp);

From 63cfcd47d70de7b3133da894edd06d5bf0a5ad93 Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:42:55 +0800
Subject: [PATCH 812/935] target/loongarch: Add fixed point shift instruction
 translation

This includes:
- SLL.W, SRL.W, SRA.W, ROTR.W
- SLLI.W, SRLI.W, SRAI.W, ROTRI.W
- SLL.D, SRL.D, SRA.D, ROTR.D
- SLLI.D, SRLI.D, SRAI.D, ROTRI.D

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-6-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/insn_trans/trans_shift.c.inc | 106 ++++++++++++++++++
 target/loongarch/insns.decode                 |  22 ++++
 target/loongarch/translate.c                  |   1 +
 3 files changed, 129 insertions(+)
 create mode 100644 target/loongarch/insn_trans/trans_shift.c.inc

diff --git a/target/loongarch/insn_trans/trans_shift.c.inc b/target/loongarch/insn_trans/trans_shift.c.inc
new file mode 100644
index 0000000000..5260af2337
--- /dev/null
+++ b/target/loongarch/insn_trans/trans_shift.c.inc
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+static void gen_sll_w(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv t0 = tcg_temp_new();
+    tcg_gen_andi_tl(t0, src2, 0x1f);
+    tcg_gen_shl_tl(dest, src1, t0);
+    tcg_temp_free(t0);
+}
+
+static void gen_srl_w(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv t0 = tcg_temp_new();
+    tcg_gen_andi_tl(t0, src2, 0x1f);
+    tcg_gen_shr_tl(dest, src1, t0);
+    tcg_temp_free(t0);
+}
+
+static void gen_sra_w(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv t0 = tcg_temp_new();
+    tcg_gen_andi_tl(t0, src2, 0x1f);
+    tcg_gen_sar_tl(dest, src1, t0);
+    tcg_temp_free(t0);
+}
+
+static void gen_sll_d(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv t0 = tcg_temp_new();
+    tcg_gen_andi_tl(t0, src2, 0x3f);
+    tcg_gen_shl_tl(dest, src1, t0);
+    tcg_temp_free(t0);
+}
+
+static void gen_srl_d(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv t0 = tcg_temp_new();
+    tcg_gen_andi_tl(t0, src2, 0x3f);
+    tcg_gen_shr_tl(dest, src1, t0);
+    tcg_temp_free(t0);
+}
+
+static void gen_sra_d(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv t0 = tcg_temp_new();
+    tcg_gen_andi_tl(t0, src2, 0x3f);
+    tcg_gen_sar_tl(dest, src1, t0);
+    tcg_temp_free(t0);
+}
+
+static void gen_rotr_w(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv_i32 t1 = tcg_temp_new_i32();
+    TCGv_i32 t2 = tcg_temp_new_i32();
+    TCGv t0 = tcg_temp_new();
+
+    tcg_gen_andi_tl(t0, src2, 0x1f);
+
+    tcg_gen_trunc_tl_i32(t1, src1);
+    tcg_gen_trunc_tl_i32(t2, t0);
+
+    tcg_gen_rotr_i32(t1, t1, t2);
+    tcg_gen_ext_i32_tl(dest, t1);
+
+    tcg_temp_free_i32(t1);
+    tcg_temp_free_i32(t2);
+    tcg_temp_free(t0);
+}
+
+static void gen_rotr_d(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv t0 = tcg_temp_new();
+    tcg_gen_andi_tl(t0, src2, 0x3f);
+    tcg_gen_rotr_tl(dest, src1, t0);
+    tcg_temp_free(t0);
+}
+
+static bool trans_srai_w(DisasContext *ctx, arg_srai_w *a)
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_ZERO);
+
+    tcg_gen_sextract_tl(dest, src1, a->imm, 32 - a->imm);
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+
+    return true;
+}
+
+TRANS(sll_w, gen_rrr, EXT_ZERO, EXT_NONE, EXT_SIGN, gen_sll_w)
+TRANS(srl_w, gen_rrr, EXT_ZERO, EXT_NONE, EXT_SIGN, gen_srl_w)
+TRANS(sra_w, gen_rrr, EXT_SIGN, EXT_NONE, EXT_SIGN, gen_sra_w)
+TRANS(sll_d, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, gen_sll_d)
+TRANS(srl_d, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, gen_srl_d)
+TRANS(sra_d, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, gen_sra_d)
+TRANS(rotr_w, gen_rrr, EXT_ZERO, EXT_NONE, EXT_SIGN, gen_rotr_w)
+TRANS(rotr_d, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, gen_rotr_d)
+TRANS(slli_w, gen_rri_c, EXT_NONE, EXT_SIGN, tcg_gen_shli_tl)
+TRANS(slli_d, gen_rri_c, EXT_NONE, EXT_NONE, tcg_gen_shli_tl)
+TRANS(srli_w, gen_rri_c, EXT_ZERO, EXT_SIGN, tcg_gen_shri_tl)
+TRANS(srli_d, gen_rri_c, EXT_NONE, EXT_NONE, tcg_gen_shri_tl)
+TRANS(srai_d, gen_rri_c, EXT_NONE, EXT_NONE, tcg_gen_sari_tl)
+TRANS(rotri_w, gen_rri_v, EXT_NONE, EXT_NONE, gen_rotr_w)
+TRANS(rotri_d, gen_rri_c, EXT_NONE, EXT_NONE, tcg_gen_rotri_tl)
diff --git a/target/loongarch/insns.decode b/target/loongarch/insns.decode
index 8579c11984..673aee4be5 100644
--- a/target/loongarch/insns.decode
+++ b/target/loongarch/insns.decode
@@ -23,6 +23,8 @@
 #
 @rrr               .... ........ ..... rk:5 rj:5 rd:5    &rrr
 @r_i20                          .... ... imm:s20 rd:5    &r_i
+@rr_ui5           .... ........ ..... imm:5 rj:5 rd:5    &rr_i
+@rr_ui6            .... ........ .... imm:6 rj:5 rd:5    &rr_i
 @rr_i12                 .... ...... imm:s12 rj:5 rd:5    &rr_i
 @rr_ui12                 .... ...... imm:12 rj:5 rd:5    &rr_i
 @rr_i16                     .... .. imm:s16 rj:5 rd:5    &rr_i
@@ -77,3 +79,23 @@ addu16i_d       0001 00 ................ ..... .....     @rr_i16
 andi            0000 001101 ............ ..... .....     @rr_ui12
 ori             0000 001110 ............ ..... .....     @rr_ui12
 xori            0000 001111 ............ ..... .....     @rr_ui12
+
+#
+# Fixed point shift operation instruction
+#
+sll_w           0000 00000001 01110 ..... ..... .....    @rrr
+srl_w           0000 00000001 01111 ..... ..... .....    @rrr
+sra_w           0000 00000001 10000 ..... ..... .....    @rrr
+sll_d           0000 00000001 10001 ..... ..... .....    @rrr
+srl_d           0000 00000001 10010 ..... ..... .....    @rrr
+sra_d           0000 00000001 10011 ..... ..... .....    @rrr
+rotr_w          0000 00000001 10110 ..... ..... .....    @rrr
+rotr_d          0000 00000001 10111 ..... ..... .....    @rrr
+slli_w          0000 00000100 00001 ..... ..... .....    @rr_ui5
+slli_d          0000 00000100 0001 ...... ..... .....    @rr_ui6
+srli_w          0000 00000100 01001 ..... ..... .....    @rr_ui5
+srli_d          0000 00000100 0101 ...... ..... .....    @rr_ui6
+srai_w          0000 00000100 10001 ..... ..... .....    @rr_ui5
+srai_d          0000 00000100 1001 ...... ..... .....    @rr_ui6
+rotri_w         0000 00000100 11001 ..... ..... .....    @rr_ui5
+rotri_d         0000 00000100 1101 ...... ..... .....    @rr_ui6
diff --git a/target/loongarch/translate.c b/target/loongarch/translate.c
index 65c7dfd3da..d5640d5d79 100644
--- a/target/loongarch/translate.c
+++ b/target/loongarch/translate.c
@@ -146,6 +146,7 @@ static void gen_set_gpr(int reg_num, TCGv t, DisasExtend dst_ext)
 
 #include "decode-insns.c.inc"
 #include "insn_trans/trans_arith.c.inc"
+#include "insn_trans/trans_shift.c.inc"
 
 static void loongarch_tr_translate_insn(DisasContextBase *dcbase, CPUState *cs)
 {

From ad08cb3f9728caf83fff7958e308ef7f1ad03fae Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:42:56 +0800
Subject: [PATCH 813/935] target/loongarch: Add fixed point bit instruction
 translation

This includes:
- EXT.W.{B/H}
- CL{O/Z}.{W/D}, CT{O/Z}.{W/D}
- BYTEPICK.{W/D}
- REVB.{2H/4H/2W/D}
- REVH.{2W/D}
- BITREV.{4B/8B}, BITREV.{W/D}
- BSTRINS.{W/D}, BSTRPICK.{W/D}
- MASKEQZ, MASKNEZ

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-7-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/helper.h                   |   4 +
 target/loongarch/insn_trans/trans_bit.c.inc | 212 ++++++++++++++++++++
 target/loongarch/insns.decode               |  39 ++++
 target/loongarch/op_helper.c                |  21 ++
 target/loongarch/translate.c                |   1 +
 5 files changed, 277 insertions(+)
 create mode 100644 target/loongarch/insn_trans/trans_bit.c.inc

diff --git a/target/loongarch/helper.h b/target/loongarch/helper.h
index eb771c0628..04e0245d5e 100644
--- a/target/loongarch/helper.h
+++ b/target/loongarch/helper.h
@@ -4,3 +4,7 @@
  */
 
 DEF_HELPER_2(raise_exception, noreturn, env, i32)
+
+DEF_HELPER_FLAGS_1(bitrev_w, TCG_CALL_NO_RWG_SE, tl, tl)
+DEF_HELPER_FLAGS_1(bitrev_d, TCG_CALL_NO_RWG_SE, tl, tl)
+DEF_HELPER_FLAGS_1(bitswap, TCG_CALL_NO_RWG_SE, tl, tl)
diff --git a/target/loongarch/insn_trans/trans_bit.c.inc b/target/loongarch/insn_trans/trans_bit.c.inc
new file mode 100644
index 0000000000..9337714ec4
--- /dev/null
+++ b/target/loongarch/insn_trans/trans_bit.c.inc
@@ -0,0 +1,212 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+static bool gen_rr(DisasContext *ctx, arg_rr *a,
+                   DisasExtend src_ext, DisasExtend dst_ext,
+                   void (*func)(TCGv, TCGv))
+{
+    TCGv dest = gpr_dst(ctx, a->rd, dst_ext);
+    TCGv src1 = gpr_src(ctx, a->rj, src_ext);
+
+    func(dest, src1);
+    gen_set_gpr(a->rd, dest, dst_ext);
+
+    return true;
+}
+
+static void gen_bytepick_w(TCGv dest, TCGv src1, TCGv src2, target_long sa)
+{
+    tcg_gen_concat_tl_i64(dest, src1, src2);
+    tcg_gen_sextract_i64(dest, dest, (32 - sa * 8), 32);
+}
+
+static void gen_bytepick_d(TCGv dest, TCGv src1, TCGv src2, target_long sa)
+{
+    tcg_gen_extract2_i64(dest, src1, src2, (64 - sa * 8));
+}
+
+static void gen_bstrins(TCGv dest, TCGv src1,
+                        unsigned int ls, unsigned int len)
+{
+    tcg_gen_deposit_tl(dest, dest, src1, ls, len);
+}
+
+static bool gen_rr_ms_ls(DisasContext *ctx, arg_rr_ms_ls *a,
+                         DisasExtend src_ext, DisasExtend dst_ext,
+                         void (*func)(TCGv, TCGv, unsigned int, unsigned int))
+{
+    TCGv dest = gpr_dst(ctx, a->rd, dst_ext);
+    TCGv src1 = gpr_src(ctx, a->rj, src_ext);
+
+    if (a->ls > a->ms) {
+        return false;
+    }
+
+    func(dest, src1, a->ls, a->ms - a->ls + 1);
+    gen_set_gpr(a->rd, dest, dst_ext);
+
+    return true;
+}
+
+static void gen_clz_w(TCGv dest, TCGv src1)
+{
+    tcg_gen_clzi_tl(dest, src1, TARGET_LONG_BITS);
+    tcg_gen_subi_tl(dest, dest, TARGET_LONG_BITS - 32);
+}
+
+static void gen_clo_w(TCGv dest, TCGv src1)
+{
+    tcg_gen_not_tl(dest, src1);
+    tcg_gen_ext32u_tl(dest, dest);
+    gen_clz_w(dest, dest);
+}
+
+static void gen_ctz_w(TCGv dest, TCGv src1)
+{
+    tcg_gen_ori_tl(dest, src1, (target_ulong)MAKE_64BIT_MASK(32, 32));
+    tcg_gen_ctzi_tl(dest, dest, TARGET_LONG_BITS);
+}
+
+static void gen_cto_w(TCGv dest, TCGv src1)
+{
+    tcg_gen_not_tl(dest, src1);
+    gen_ctz_w(dest, dest);
+}
+
+static void gen_clz_d(TCGv dest, TCGv src1)
+{
+    tcg_gen_clzi_i64(dest, src1, TARGET_LONG_BITS);
+}
+
+static void gen_clo_d(TCGv dest, TCGv src1)
+{
+    tcg_gen_not_tl(dest, src1);
+    gen_clz_d(dest, dest);
+}
+
+static void gen_ctz_d(TCGv dest, TCGv src1)
+{
+    tcg_gen_ctzi_tl(dest, src1, TARGET_LONG_BITS);
+}
+
+static void gen_cto_d(TCGv dest, TCGv src1)
+{
+    tcg_gen_not_tl(dest, src1);
+    gen_ctz_d(dest, dest);
+}
+
+static void gen_revb_2w(TCGv dest, TCGv src1)
+{
+    tcg_gen_bswap64_i64(dest, src1);
+    tcg_gen_rotri_i64(dest, dest, 32);
+}
+
+static void gen_revb_2h(TCGv dest, TCGv src1)
+{
+    TCGv mask = tcg_constant_tl(0x00FF00FF);
+    TCGv t0 = tcg_temp_new();
+    TCGv t1 = tcg_temp_new();
+
+    tcg_gen_shri_tl(t0, src1, 8);
+    tcg_gen_and_tl(t0, t0, mask);
+    tcg_gen_and_tl(t1, src1, mask);
+    tcg_gen_shli_tl(t1, t1, 8);
+    tcg_gen_or_tl(dest, t0, t1);
+
+    tcg_temp_free(t0);
+    tcg_temp_free(t1);
+}
+
+static void gen_revb_4h(TCGv dest, TCGv src1)
+{
+    TCGv mask = tcg_constant_tl(0x00FF00FF00FF00FFULL);
+    TCGv t0 = tcg_temp_new();
+    TCGv t1 = tcg_temp_new();
+
+    tcg_gen_shri_tl(t0, src1, 8);
+    tcg_gen_and_tl(t0, t0, mask);
+    tcg_gen_and_tl(t1, src1, mask);
+    tcg_gen_shli_tl(t1, t1, 8);
+    tcg_gen_or_tl(dest, t0, t1);
+
+    tcg_temp_free(t0);
+    tcg_temp_free(t1);
+}
+
+static void gen_revh_2w(TCGv dest, TCGv src1)
+{
+    TCGv_i64 t0 = tcg_temp_new_i64();
+    TCGv_i64 t1 = tcg_temp_new_i64();
+    TCGv_i64 mask = tcg_constant_i64(0x0000ffff0000ffffull);
+
+    tcg_gen_shri_i64(t0, src1, 16);
+    tcg_gen_and_i64(t1, src1, mask);
+    tcg_gen_and_i64(t0, t0, mask);
+    tcg_gen_shli_i64(t1, t1, 16);
+    tcg_gen_or_i64(dest, t1, t0);
+
+    tcg_temp_free_i64(t0);
+    tcg_temp_free_i64(t1);
+}
+
+static void gen_revh_d(TCGv dest, TCGv src1)
+{
+    TCGv t0 = tcg_temp_new();
+    TCGv t1 = tcg_temp_new();
+    TCGv mask = tcg_constant_tl(0x0000FFFF0000FFFFULL);
+
+    tcg_gen_shri_tl(t1, src1, 16);
+    tcg_gen_and_tl(t1, t1, mask);
+    tcg_gen_and_tl(t0, src1, mask);
+    tcg_gen_shli_tl(t0, t0, 16);
+    tcg_gen_or_tl(t0, t0, t1);
+    tcg_gen_rotri_tl(dest, t0, 32);
+
+    tcg_temp_free(t0);
+    tcg_temp_free(t1);
+}
+
+static void gen_maskeqz(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv zero = tcg_constant_tl(0);
+
+    tcg_gen_movcond_tl(TCG_COND_EQ, dest, src2, zero, zero, src1);
+}
+
+static void gen_masknez(TCGv dest, TCGv src1, TCGv src2)
+{
+    TCGv zero = tcg_constant_tl(0);
+
+    tcg_gen_movcond_tl(TCG_COND_NE, dest, src2, zero, zero, src1);
+}
+
+TRANS(ext_w_h, gen_rr, EXT_NONE, EXT_NONE, tcg_gen_ext16s_tl)
+TRANS(ext_w_b, gen_rr, EXT_NONE, EXT_NONE, tcg_gen_ext8s_tl)
+TRANS(clo_w, gen_rr, EXT_NONE, EXT_NONE, gen_clo_w)
+TRANS(clz_w, gen_rr, EXT_ZERO, EXT_NONE, gen_clz_w)
+TRANS(cto_w, gen_rr, EXT_NONE, EXT_NONE, gen_cto_w)
+TRANS(ctz_w, gen_rr, EXT_NONE, EXT_NONE, gen_ctz_w)
+TRANS(clo_d, gen_rr, EXT_NONE, EXT_NONE, gen_clo_d)
+TRANS(clz_d, gen_rr, EXT_NONE, EXT_NONE, gen_clz_d)
+TRANS(cto_d, gen_rr, EXT_NONE, EXT_NONE, gen_cto_d)
+TRANS(ctz_d, gen_rr, EXT_NONE, EXT_NONE, gen_ctz_d)
+TRANS(revb_2h, gen_rr, EXT_NONE, EXT_SIGN, gen_revb_2h)
+TRANS(revb_4h, gen_rr, EXT_NONE, EXT_NONE, gen_revb_4h)
+TRANS(revb_2w, gen_rr, EXT_NONE, EXT_NONE, gen_revb_2w)
+TRANS(revb_d, gen_rr, EXT_NONE, EXT_NONE, tcg_gen_bswap64_i64)
+TRANS(revh_2w, gen_rr, EXT_NONE, EXT_NONE, gen_revh_2w)
+TRANS(revh_d, gen_rr, EXT_NONE, EXT_NONE, gen_revh_d)
+TRANS(bitrev_4b, gen_rr, EXT_ZERO, EXT_SIGN, gen_helper_bitswap)
+TRANS(bitrev_8b, gen_rr, EXT_NONE, EXT_NONE, gen_helper_bitswap)
+TRANS(bitrev_w, gen_rr, EXT_NONE, EXT_SIGN, gen_helper_bitrev_w)
+TRANS(bitrev_d, gen_rr, EXT_NONE, EXT_NONE, gen_helper_bitrev_d)
+TRANS(maskeqz, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, gen_maskeqz)
+TRANS(masknez, gen_rrr, EXT_NONE, EXT_NONE, EXT_NONE, gen_masknez)
+TRANS(bytepick_w, gen_rrr_sa, EXT_NONE, EXT_NONE, gen_bytepick_w)
+TRANS(bytepick_d, gen_rrr_sa, EXT_NONE, EXT_NONE, gen_bytepick_d)
+TRANS(bstrins_w, gen_rr_ms_ls, EXT_NONE, EXT_NONE, gen_bstrins)
+TRANS(bstrins_d, gen_rr_ms_ls, EXT_NONE, EXT_NONE, gen_bstrins)
+TRANS(bstrpick_w, gen_rr_ms_ls, EXT_NONE, EXT_SIGN, tcg_gen_extract_tl)
+TRANS(bstrpick_d, gen_rr_ms_ls, EXT_NONE, EXT_NONE, tcg_gen_extract_tl)
diff --git a/target/loongarch/insns.decode b/target/loongarch/insns.decode
index 673aee4be5..b0bed5531b 100644
--- a/target/loongarch/insns.decode
+++ b/target/loongarch/insns.decode
@@ -14,13 +14,16 @@
 # Argument sets
 #
 &r_i          rd imm
+&rr           rd rj
 &rrr          rd rj rk
 &rr_i         rd rj imm
 &rrr_sa       rd rj rk sa
+&rr_ms_ls     rd rj ms ls
 
 #
 # Formats
 #
+@rr               .... ........ ..... ..... rj:5 rd:5    &rr
 @rrr               .... ........ ..... rk:5 rj:5 rd:5    &rrr
 @r_i20                          .... ... imm:s20 rd:5    &r_i
 @rr_ui5           .... ........ ..... imm:5 rj:5 rd:5    &rr_i
@@ -29,6 +32,10 @@
 @rr_ui12                 .... ...... imm:12 rj:5 rd:5    &rr_i
 @rr_i16                     .... .. imm:s16 rj:5 rd:5    &rr_i
 @rrr_sa2p1        .... ........ ... .. rk:5 rj:5 rd:5    &rrr_sa  sa=%sa2p1
+@rrr_sa2        .... ........ ... sa:2 rk:5 rj:5 rd:5    &rrr_sa
+@rrr_sa3         .... ........ .. sa:3 rk:5 rj:5 rd:5    &rrr_sa
+@rr_2bw            .... ....... ms:5 . ls:5 rj:5 rd:5    &rr_ms_ls
+@rr_2bd               .... ...... ms:6 ls:6 rj:5 rd:5    &rr_ms_ls
 
 #
 # Fixed point arithmetic operation instruction
@@ -99,3 +106,35 @@ srai_w          0000 00000100 10001 ..... ..... .....    @rr_ui5
 srai_d          0000 00000100 1001 ...... ..... .....    @rr_ui6
 rotri_w         0000 00000100 11001 ..... ..... .....    @rr_ui5
 rotri_d         0000 00000100 1101 ...... ..... .....    @rr_ui6
+
+#
+# Fixed point bit operation instruction
+#
+ext_w_h         0000 00000000 00000 10110 ..... .....    @rr
+ext_w_b         0000 00000000 00000 10111 ..... .....    @rr
+clo_w           0000 00000000 00000 00100 ..... .....    @rr
+clz_w           0000 00000000 00000 00101 ..... .....    @rr
+cto_w           0000 00000000 00000 00110 ..... .....    @rr
+ctz_w           0000 00000000 00000 00111 ..... .....    @rr
+clo_d           0000 00000000 00000 01000 ..... .....    @rr
+clz_d           0000 00000000 00000 01001 ..... .....    @rr
+cto_d           0000 00000000 00000 01010 ..... .....    @rr
+ctz_d           0000 00000000 00000 01011 ..... .....    @rr
+revb_2h         0000 00000000 00000 01100 ..... .....    @rr
+revb_4h         0000 00000000 00000 01101 ..... .....    @rr
+revb_2w         0000 00000000 00000 01110 ..... .....    @rr
+revb_d          0000 00000000 00000 01111 ..... .....    @rr
+revh_2w         0000 00000000 00000 10000 ..... .....    @rr
+revh_d          0000 00000000 00000 10001 ..... .....    @rr
+bitrev_4b       0000 00000000 00000 10010 ..... .....    @rr
+bitrev_8b       0000 00000000 00000 10011 ..... .....    @rr
+bitrev_w        0000 00000000 00000 10100 ..... .....    @rr
+bitrev_d        0000 00000000 00000 10101 ..... .....    @rr
+bytepick_w      0000 00000000 100 .. ..... ..... .....   @rrr_sa2
+bytepick_d      0000 00000000 11 ... ..... ..... .....   @rrr_sa3
+maskeqz         0000 00000001 00110 ..... ..... .....    @rrr
+masknez         0000 00000001 00111 ..... ..... .....    @rrr
+bstrins_w       0000 0000011 ..... 0 ..... ..... .....   @rr_2bw
+bstrpick_w      0000 0000011 ..... 1 ..... ..... .....   @rr_2bw
+bstrins_d       0000 000010 ...... ...... ..... .....    @rr_2bd
+bstrpick_d      0000 000011 ...... ...... ..... .....    @rr_2bd
diff --git a/target/loongarch/op_helper.c b/target/loongarch/op_helper.c
index 903810951e..f4b22c70a0 100644
--- a/target/loongarch/op_helper.c
+++ b/target/loongarch/op_helper.c
@@ -19,3 +19,24 @@ void helper_raise_exception(CPULoongArchState *env, uint32_t exception)
 {
     do_raise_exception(env, exception, GETPC());
 }
+
+target_ulong helper_bitrev_w(target_ulong rj)
+{
+    return (int32_t)revbit32(rj);
+}
+
+target_ulong helper_bitrev_d(target_ulong rj)
+{
+    return revbit64(rj);
+}
+
+target_ulong helper_bitswap(target_ulong v)
+{
+    v = ((v >> 1) & (target_ulong)0x5555555555555555ULL) |
+        ((v & (target_ulong)0x5555555555555555ULL) << 1);
+    v = ((v >> 2) & (target_ulong)0x3333333333333333ULL) |
+        ((v & (target_ulong)0x3333333333333333ULL) << 2);
+    v = ((v >> 4) & (target_ulong)0x0F0F0F0F0F0F0F0FULL) |
+        ((v & (target_ulong)0x0F0F0F0F0F0F0F0FULL) << 4);
+    return v;
+}
diff --git a/target/loongarch/translate.c b/target/loongarch/translate.c
index d5640d5d79..88afd9b3a8 100644
--- a/target/loongarch/translate.c
+++ b/target/loongarch/translate.c
@@ -147,6 +147,7 @@ static void gen_set_gpr(int reg_num, TCGv t, DisasExtend dst_ext)
 #include "decode-insns.c.inc"
 #include "insn_trans/trans_arith.c.inc"
 #include "insn_trans/trans_shift.c.inc"
+#include "insn_trans/trans_bit.c.inc"
 
 static void loongarch_tr_translate_insn(DisasContextBase *dcbase, CPUState *cs)
 {

From bb79174d4e191dc83414072e1a7859095faed567 Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:42:57 +0800
Subject: [PATCH 814/935] target/loongarch: Add fixed point load/store
 instruction translation

This includes:
- LD.{B[U]/H[U]/W[U]/D}, ST.{B/H/W/D}
- LDX.{B[U]/H[U]/W[U]/D}, STX.{B/H/W/D}
- LDPTR.{W/D}, STPTR.{W/D}
- PRELD
- LD{GT/LE}.{B/H/W/D}, ST{GT/LE}.{B/H/W/D}
- DBAR, IBAR

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-8-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/helper.h                     |   3 +
 .../loongarch/insn_trans/trans_memory.c.inc   | 229 ++++++++++++++++++
 target/loongarch/insns.decode                 |  55 +++++
 target/loongarch/op_helper.c                  |  15 ++
 target/loongarch/translate.c                  |   6 +
 5 files changed, 308 insertions(+)
 create mode 100644 target/loongarch/insn_trans/trans_memory.c.inc

diff --git a/target/loongarch/helper.h b/target/loongarch/helper.h
index 04e0245d5e..100622bfc2 100644
--- a/target/loongarch/helper.h
+++ b/target/loongarch/helper.h
@@ -8,3 +8,6 @@ DEF_HELPER_2(raise_exception, noreturn, env, i32)
 DEF_HELPER_FLAGS_1(bitrev_w, TCG_CALL_NO_RWG_SE, tl, tl)
 DEF_HELPER_FLAGS_1(bitrev_d, TCG_CALL_NO_RWG_SE, tl, tl)
 DEF_HELPER_FLAGS_1(bitswap, TCG_CALL_NO_RWG_SE, tl, tl)
+
+DEF_HELPER_FLAGS_3(asrtle_d, TCG_CALL_NO_WG, void, env, tl, tl)
+DEF_HELPER_FLAGS_3(asrtgt_d, TCG_CALL_NO_WG, void, env, tl, tl)
diff --git a/target/loongarch/insn_trans/trans_memory.c.inc b/target/loongarch/insn_trans/trans_memory.c.inc
new file mode 100644
index 0000000000..10914acf52
--- /dev/null
+++ b/target/loongarch/insn_trans/trans_memory.c.inc
@@ -0,0 +1,229 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+static bool gen_load(DisasContext *ctx, arg_rr_i *a, MemOp mop)
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+    TCGv addr = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv temp = NULL;
+
+    if (a->imm) {
+        temp = tcg_temp_new();
+        tcg_gen_addi_tl(temp, addr, a->imm);
+        addr = temp;
+    }
+
+    tcg_gen_qemu_ld_tl(dest, addr, ctx->mem_idx, mop);
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+
+    if (temp) {
+        tcg_temp_free(temp);
+    }
+
+    return true;
+}
+
+static bool gen_store(DisasContext *ctx, arg_rr_i *a, MemOp mop)
+{
+    TCGv data = gpr_src(ctx, a->rd, EXT_NONE);
+    TCGv addr = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv temp = NULL;
+
+    if (a->imm) {
+        temp = tcg_temp_new();
+        tcg_gen_addi_tl(temp, addr, a->imm);
+        addr = temp;
+    }
+
+    tcg_gen_qemu_st_tl(data, addr, ctx->mem_idx, mop);
+
+    if (temp) {
+        tcg_temp_free(temp);
+    }
+
+    return true;
+}
+
+static bool gen_loadx(DisasContext *ctx, arg_rrr *a, MemOp mop)
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = gpr_src(ctx, a->rk, EXT_NONE);
+    TCGv addr = tcg_temp_new();
+
+    tcg_gen_add_tl(addr, src1, src2);
+    tcg_gen_qemu_ld_tl(dest, addr, ctx->mem_idx, mop);
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+    tcg_temp_free(addr);
+
+    return true;
+}
+
+static bool gen_storex(DisasContext *ctx, arg_rrr *a, MemOp mop)
+{
+    TCGv data = gpr_src(ctx, a->rd, EXT_NONE);
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = gpr_src(ctx, a->rk, EXT_NONE);
+    TCGv addr = tcg_temp_new();
+
+    tcg_gen_add_tl(addr, src1, src2);
+    tcg_gen_qemu_st_tl(data, addr, ctx->mem_idx, mop);
+    tcg_temp_free(addr);
+
+    return true;
+}
+
+static bool gen_load_gt(DisasContext *ctx, arg_rrr *a, MemOp mop)
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = gpr_src(ctx, a->rk, EXT_NONE);
+
+    gen_helper_asrtgt_d(cpu_env, src1, src2);
+    tcg_gen_qemu_ld_tl(dest, src1, ctx->mem_idx, mop);
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+
+    return true;
+}
+
+static bool gen_load_le(DisasContext *ctx, arg_rrr *a, MemOp mop)
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = gpr_src(ctx, a->rk, EXT_NONE);
+
+    gen_helper_asrtle_d(cpu_env, src1, src2);
+    tcg_gen_qemu_ld_tl(dest, src1, ctx->mem_idx, mop);
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+
+    return true;
+}
+
+static bool gen_store_gt(DisasContext *ctx, arg_rrr *a, MemOp mop)
+{
+    TCGv data = gpr_src(ctx, a->rd, EXT_NONE);
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = gpr_src(ctx, a->rk, EXT_NONE);
+
+    gen_helper_asrtgt_d(cpu_env, src1, src2);
+    tcg_gen_qemu_st_tl(data, src1, ctx->mem_idx, mop);
+
+    return true;
+}
+
+static bool gen_store_le(DisasContext *ctx, arg_rrr *a, MemOp mop)
+{
+    TCGv data = gpr_src(ctx, a->rd, EXT_NONE);
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = gpr_src(ctx, a->rk, EXT_NONE);
+
+    gen_helper_asrtle_d(cpu_env, src1, src2);
+    tcg_gen_qemu_st_tl(data, src1, ctx->mem_idx, mop);
+
+    return true;
+}
+
+static bool trans_preld(DisasContext *ctx, arg_preld *a)
+{
+    return true;
+}
+
+static bool trans_dbar(DisasContext *ctx, arg_dbar * a)
+{
+    tcg_gen_mb(TCG_BAR_SC | TCG_MO_ALL);
+    return true;
+}
+
+static bool trans_ibar(DisasContext *ctx, arg_ibar *a)
+{
+    ctx->base.is_jmp = DISAS_STOP;
+    return true;
+}
+
+static bool gen_ldptr(DisasContext *ctx, arg_rr_i *a, MemOp mop)
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+    TCGv addr = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv temp = NULL;
+
+    if (a->imm) {
+        temp = tcg_temp_new();
+        tcg_gen_addi_tl(temp, addr, a->imm);
+        addr = temp;
+    }
+
+    tcg_gen_qemu_ld_tl(dest, addr, ctx->mem_idx, mop);
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+
+    if (temp) {
+        tcg_temp_free(temp);
+    }
+
+    return true;
+}
+
+static bool gen_stptr(DisasContext *ctx, arg_rr_i *a, MemOp mop)
+{
+    TCGv data = gpr_src(ctx, a->rd, EXT_NONE);
+    TCGv addr = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv temp = NULL;
+
+    if (a->imm) {
+        temp = tcg_temp_new();
+        tcg_gen_addi_tl(temp, addr, a->im);
+        addr = temp;
+    }
+
+    tcg_gen_qemu_st_tl(data, addr, ctx->mem_idx, mop);
+
+    if (temp) {
+        tcg_temp_free(temp);
+    }
+
+    return true;
+}
+
+TRANS(ld_b, gen_load, MO_SB)
+TRANS(ld_h, gen_load, MO_TESW)
+TRANS(ld_w, gen_load, MO_TESL)
+TRANS(ld_d, gen_load, MO_TEUQ)
+TRANS(st_b, gen_store, MO_UB)
+TRANS(st_h, gen_store, MO_TEUW)
+TRANS(st_w, gen_store, MO_TEUL)
+TRANS(st_d, gen_store, MO_TEUQ)
+TRANS(ld_bu, gen_load, MO_UB)
+TRANS(ld_hu, gen_load, MO_TEUW)
+TRANS(ld_wu, gen_load, MO_TEUL)
+TRANS(ldx_b, gen_loadx, MO_SB)
+TRANS(ldx_h, gen_loadx, MO_TESW)
+TRANS(ldx_w, gen_loadx, MO_TESL)
+TRANS(ldx_d, gen_loadx, MO_TEUQ)
+TRANS(stx_b, gen_storex, MO_UB)
+TRANS(stx_h, gen_storex, MO_TEUW)
+TRANS(stx_w, gen_storex, MO_TEUL)
+TRANS(stx_d, gen_storex, MO_TEUQ)
+TRANS(ldx_bu, gen_loadx, MO_UB)
+TRANS(ldx_hu, gen_loadx, MO_TEUW)
+TRANS(ldx_wu, gen_loadx, MO_TEUL)
+TRANS(ldptr_w, gen_ldptr, MO_TESL)
+TRANS(stptr_w, gen_stptr, MO_TEUL)
+TRANS(ldptr_d, gen_ldptr, MO_TEUQ)
+TRANS(stptr_d, gen_stptr, MO_TEUQ)
+TRANS(ldgt_b, gen_load_gt, MO_SB)
+TRANS(ldgt_h, gen_load_gt, MO_TESW)
+TRANS(ldgt_w, gen_load_gt, MO_TESL)
+TRANS(ldgt_d, gen_load_gt, MO_TEUQ)
+TRANS(ldle_b, gen_load_le, MO_SB)
+TRANS(ldle_h, gen_load_le, MO_TESW)
+TRANS(ldle_w, gen_load_le, MO_TESL)
+TRANS(ldle_d, gen_load_le, MO_TEUQ)
+TRANS(stgt_b, gen_store_gt, MO_UB)
+TRANS(stgt_h, gen_store_gt, MO_TEUW)
+TRANS(stgt_w, gen_store_gt, MO_TEUL)
+TRANS(stgt_d, gen_store_gt, MO_TEUQ)
+TRANS(stle_b, gen_store_le, MO_UB)
+TRANS(stle_h, gen_store_le, MO_TEUW)
+TRANS(stle_w, gen_store_le, MO_TEUL)
+TRANS(stle_d, gen_store_le, MO_TEUQ)
diff --git a/target/loongarch/insns.decode b/target/loongarch/insns.decode
index b0bed5531b..1156e6965c 100644
--- a/target/loongarch/insns.decode
+++ b/target/loongarch/insns.decode
@@ -8,21 +8,25 @@
 #
 # Fields
 #
+%i14s2     10:s14       !function=shl_2
 %sa2p1     15:2         !function=plus_1
 
 #
 # Argument sets
 #
+&i            imm
 &r_i          rd imm
 &rr           rd rj
 &rrr          rd rj rk
 &rr_i         rd rj imm
+&hint_r_i     hint rj imm
 &rrr_sa       rd rj rk sa
 &rr_ms_ls     rd rj ms ls
 
 #
 # Formats
 #
+@i15                       .... ........ ..... imm:15    &i
 @rr               .... ........ ..... ..... rj:5 rd:5    &rr
 @rrr               .... ........ ..... rk:5 rj:5 rd:5    &rrr
 @r_i20                          .... ... imm:s20 rd:5    &r_i
@@ -30,7 +34,9 @@
 @rr_ui6            .... ........ .... imm:6 rj:5 rd:5    &rr_i
 @rr_i12                 .... ...... imm:s12 rj:5 rd:5    &rr_i
 @rr_ui12                 .... ...... imm:12 rj:5 rd:5    &rr_i
+@rr_i14s2         .... ....  .............. rj:5 rd:5    &rr_i imm=%i14s2
 @rr_i16                     .... .. imm:s16 rj:5 rd:5    &rr_i
+@hint_r_i12           .... ...... imm:s12 rj:5 hint:5    &hint_r_i
 @rrr_sa2p1        .... ........ ... .. rk:5 rj:5 rd:5    &rrr_sa  sa=%sa2p1
 @rrr_sa2        .... ........ ... sa:2 rk:5 rj:5 rd:5    &rrr_sa
 @rrr_sa3         .... ........ .. sa:3 rk:5 rj:5 rd:5    &rrr_sa
@@ -138,3 +144,52 @@ bstrins_w       0000 0000011 ..... 0 ..... ..... .....   @rr_2bw
 bstrpick_w      0000 0000011 ..... 1 ..... ..... .....   @rr_2bw
 bstrins_d       0000 000010 ...... ...... ..... .....    @rr_2bd
 bstrpick_d      0000 000011 ...... ...... ..... .....    @rr_2bd
+
+#
+# Fixed point load/store instruction
+#
+ld_b            0010 100000 ............ ..... .....     @rr_i12
+ld_h            0010 100001 ............ ..... .....     @rr_i12
+ld_w            0010 100010 ............ ..... .....     @rr_i12
+ld_d            0010 100011 ............ ..... .....     @rr_i12
+st_b            0010 100100 ............ ..... .....     @rr_i12
+st_h            0010 100101 ............ ..... .....     @rr_i12
+st_w            0010 100110 ............ ..... .....     @rr_i12
+st_d            0010 100111 ............ ..... .....     @rr_i12
+ld_bu           0010 101000 ............ ..... .....     @rr_i12
+ld_hu           0010 101001 ............ ..... .....     @rr_i12
+ld_wu           0010 101010 ............ ..... .....     @rr_i12
+ldx_b           0011 10000000 00000 ..... ..... .....    @rrr
+ldx_h           0011 10000000 01000 ..... ..... .....    @rrr
+ldx_w           0011 10000000 10000 ..... ..... .....    @rrr
+ldx_d           0011 10000000 11000 ..... ..... .....    @rrr
+stx_b           0011 10000001 00000 ..... ..... .....    @rrr
+stx_h           0011 10000001 01000 ..... ..... .....    @rrr
+stx_w           0011 10000001 10000 ..... ..... .....    @rrr
+stx_d           0011 10000001 11000 ..... ..... .....    @rrr
+ldx_bu          0011 10000010 00000 ..... ..... .....    @rrr
+ldx_hu          0011 10000010 01000 ..... ..... .....    @rrr
+ldx_wu          0011 10000010 10000 ..... ..... .....    @rrr
+preld           0010 101011 ............ ..... .....     @hint_r_i12
+dbar            0011 10000111 00100 ...............      @i15
+ibar            0011 10000111 00101 ...............      @i15
+ldptr_w         0010 0100 .............. ..... .....     @rr_i14s2
+stptr_w         0010 0101 .............. ..... .....     @rr_i14s2
+ldptr_d         0010 0110 .............. ..... .....     @rr_i14s2
+stptr_d         0010 0111 .............. ..... .....     @rr_i14s2
+ldgt_b          0011 10000111 10000 ..... ..... .....    @rrr
+ldgt_h          0011 10000111 10001 ..... ..... .....    @rrr
+ldgt_w          0011 10000111 10010 ..... ..... .....    @rrr
+ldgt_d          0011 10000111 10011 ..... ..... .....    @rrr
+ldle_b          0011 10000111 10100 ..... ..... .....    @rrr
+ldle_h          0011 10000111 10101 ..... ..... .....    @rrr
+ldle_w          0011 10000111 10110 ..... ..... .....    @rrr
+ldle_d          0011 10000111 10111 ..... ..... .....    @rrr
+stgt_b          0011 10000111 11000 ..... ..... .....    @rrr
+stgt_h          0011 10000111 11001 ..... ..... .....    @rrr
+stgt_w          0011 10000111 11010 ..... ..... .....    @rrr
+stgt_d          0011 10000111 11011 ..... ..... .....    @rrr
+stle_b          0011 10000111 11100 ..... ..... .....    @rrr
+stle_h          0011 10000111 11101 ..... ..... .....    @rrr
+stle_w          0011 10000111 11110 ..... ..... .....    @rrr
+stle_d          0011 10000111 11111 ..... ..... .....    @rrr
diff --git a/target/loongarch/op_helper.c b/target/loongarch/op_helper.c
index f4b22c70a0..bd2db783c9 100644
--- a/target/loongarch/op_helper.c
+++ b/target/loongarch/op_helper.c
@@ -40,3 +40,18 @@ target_ulong helper_bitswap(target_ulong v)
         ((v & (target_ulong)0x0F0F0F0F0F0F0F0FULL) << 4);
     return v;
 }
+
+/* loongarch assert op */
+void helper_asrtle_d(CPULoongArchState *env, target_ulong rj, target_ulong rk)
+{
+    if (rj > rk) {
+        do_raise_exception(env, EXCCODE_ADEM, GETPC());
+    }
+}
+
+void helper_asrtgt_d(CPULoongArchState *env, target_ulong rj, target_ulong rk)
+{
+    if (rj <= rk) {
+        do_raise_exception(env, EXCCODE_ADEM, GETPC());
+    }
+}
diff --git a/target/loongarch/translate.c b/target/loongarch/translate.c
index 88afd9b3a8..b8fed26699 100644
--- a/target/loongarch/translate.c
+++ b/target/loongarch/translate.c
@@ -31,6 +31,11 @@ static inline int plus_1(DisasContext *ctx, int x)
     return x + 1;
 }
 
+static inline int shl_2(DisasContext *ctx, int x)
+{
+    return x << 2;
+}
+
 void generate_exception(DisasContext *ctx, int excp)
 {
     tcg_gen_movi_tl(cpu_pc, ctx->base.pc_next);
@@ -148,6 +153,7 @@ static void gen_set_gpr(int reg_num, TCGv t, DisasExtend dst_ext)
 #include "insn_trans/trans_arith.c.inc"
 #include "insn_trans/trans_shift.c.inc"
 #include "insn_trans/trans_bit.c.inc"
+#include "insn_trans/trans_memory.c.inc"
 
 static void loongarch_tr_translate_insn(DisasContextBase *dcbase, CPUState *cs)
 {

From 94b02d57b09eeb2dcb07a2a196b91310420bd0bf Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:42:58 +0800
Subject: [PATCH 815/935] target/loongarch: Add fixed point atomic instruction
 translation

This includes:
- LL.{W/D}, SC.{W/D}
- AM{SWAP/ADD/AND/OR/XOR/MAX/MIN}[_DB].{W/D}
- AM{MAX/MIN}[_DB].{WU/DU}

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-9-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 .../loongarch/insn_trans/trans_atomic.c.inc   | 113 ++++++++++++++++++
 .../loongarch/insn_trans/trans_memory.c.inc   |   2 +-
 target/loongarch/insns.decode                 |  44 +++++++
 target/loongarch/translate.c                  |   1 +
 4 files changed, 159 insertions(+), 1 deletion(-)
 create mode 100644 target/loongarch/insn_trans/trans_atomic.c.inc

diff --git a/target/loongarch/insn_trans/trans_atomic.c.inc b/target/loongarch/insn_trans/trans_atomic.c.inc
new file mode 100644
index 0000000000..6763c1c301
--- /dev/null
+++ b/target/loongarch/insn_trans/trans_atomic.c.inc
@@ -0,0 +1,113 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+static bool gen_ll(DisasContext *ctx, arg_rr_i *a, MemOp mop)
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv t0 = tcg_temp_new();
+
+    tcg_gen_addi_tl(t0, src1, a->imm);
+    tcg_gen_qemu_ld_i64(dest, t0, ctx->mem_idx, mop);
+    tcg_gen_st_tl(t0, cpu_env, offsetof(CPULoongArchState, lladdr));
+    tcg_gen_st_tl(dest, cpu_env, offsetof(CPULoongArchState, llval));
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+    tcg_temp_free(t0);
+
+    return true;
+}
+
+static bool gen_sc(DisasContext *ctx, arg_rr_i *a, MemOp mop)
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = gpr_src(ctx, a->rd, EXT_NONE);
+    TCGv t0 = tcg_temp_new();
+    TCGv val = tcg_temp_new();
+
+    TCGLabel *l1 = gen_new_label();
+    TCGLabel *done = gen_new_label();
+
+    tcg_gen_addi_tl(t0, src1, a->imm);
+    tcg_gen_brcond_tl(TCG_COND_EQ, t0, cpu_lladdr, l1);
+    tcg_gen_movi_tl(dest, 0);
+    tcg_gen_br(done);
+
+    gen_set_label(l1);
+    tcg_gen_mov_tl(val, src2);
+    /* generate cmpxchg */
+    tcg_gen_atomic_cmpxchg_tl(t0, cpu_lladdr, cpu_llval,
+                              val, ctx->mem_idx, mop);
+    tcg_gen_setcond_tl(TCG_COND_EQ, dest, t0, cpu_llval);
+    gen_set_label(done);
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+    tcg_temp_free(t0);
+    tcg_temp_free(val);
+
+    return true;
+}
+
+static bool gen_am(DisasContext *ctx, arg_rrr *a,
+                   void (*func)(TCGv, TCGv, TCGv, TCGArg, MemOp),
+                   MemOp mop)
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+    TCGv addr = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv val = gpr_src(ctx, a->rk, EXT_NONE);
+
+    if (a->rd != 0 && (a->rj == a->rd || a->rk == a->rd)) {
+        qemu_log_mask(LOG_GUEST_ERROR,
+                      "Warning: source register overlaps destination register"
+                      "in atomic insn at pc=0x" TARGET_FMT_lx "\n",
+                      ctx->base.pc_next - 4);
+        return false;
+    }
+
+    func(dest, addr, val, ctx->mem_idx, mop);
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+
+    return true;
+}
+
+TRANS(ll_w, gen_ll, MO_TESL)
+TRANS(sc_w, gen_sc, MO_TESL)
+TRANS(ll_d, gen_ll, MO_TEUQ)
+TRANS(sc_d, gen_sc, MO_TEUQ)
+TRANS(amswap_w, gen_am, tcg_gen_atomic_xchg_tl, MO_TESL)
+TRANS(amswap_d, gen_am, tcg_gen_atomic_xchg_tl, MO_TEUQ)
+TRANS(amadd_w, gen_am, tcg_gen_atomic_fetch_add_tl, MO_TESL)
+TRANS(amadd_d, gen_am, tcg_gen_atomic_fetch_add_tl, MO_TEUQ)
+TRANS(amand_w, gen_am, tcg_gen_atomic_fetch_and_tl, MO_TESL)
+TRANS(amand_d, gen_am, tcg_gen_atomic_fetch_and_tl, MO_TEUQ)
+TRANS(amor_w, gen_am, tcg_gen_atomic_fetch_or_tl, MO_TESL)
+TRANS(amor_d, gen_am, tcg_gen_atomic_fetch_or_tl, MO_TEUQ)
+TRANS(amxor_w, gen_am, tcg_gen_atomic_fetch_xor_tl, MO_TESL)
+TRANS(amxor_d, gen_am, tcg_gen_atomic_fetch_xor_tl, MO_TEUQ)
+TRANS(ammax_w, gen_am, tcg_gen_atomic_fetch_smax_tl, MO_TESL)
+TRANS(ammax_d, gen_am, tcg_gen_atomic_fetch_smax_tl, MO_TEUQ)
+TRANS(ammin_w, gen_am, tcg_gen_atomic_fetch_smin_tl, MO_TESL)
+TRANS(ammin_d, gen_am, tcg_gen_atomic_fetch_smin_tl, MO_TEUQ)
+TRANS(ammax_wu, gen_am, tcg_gen_atomic_fetch_umax_tl, MO_TESL)
+TRANS(ammax_du, gen_am, tcg_gen_atomic_fetch_umax_tl, MO_TEUQ)
+TRANS(ammin_wu, gen_am, tcg_gen_atomic_fetch_umin_tl, MO_TESL)
+TRANS(ammin_du, gen_am, tcg_gen_atomic_fetch_umin_tl, MO_TEUQ)
+TRANS(amswap_db_w, gen_am, tcg_gen_atomic_xchg_tl, MO_TESL)
+TRANS(amswap_db_d, gen_am, tcg_gen_atomic_xchg_tl, MO_TEUQ)
+TRANS(amadd_db_w, gen_am, tcg_gen_atomic_fetch_add_tl, MO_TESL)
+TRANS(amadd_db_d, gen_am, tcg_gen_atomic_fetch_add_tl, MO_TEUQ)
+TRANS(amand_db_w, gen_am, tcg_gen_atomic_fetch_and_tl, MO_TESL)
+TRANS(amand_db_d, gen_am, tcg_gen_atomic_fetch_and_tl, MO_TEUQ)
+TRANS(amor_db_w, gen_am, tcg_gen_atomic_fetch_or_tl, MO_TESL)
+TRANS(amor_db_d, gen_am, tcg_gen_atomic_fetch_or_tl, MO_TEUQ)
+TRANS(amxor_db_w, gen_am, tcg_gen_atomic_fetch_xor_tl, MO_TESL)
+TRANS(amxor_db_d, gen_am, tcg_gen_atomic_fetch_xor_tl, MO_TEUQ)
+TRANS(ammax_db_w, gen_am, tcg_gen_atomic_fetch_smax_tl, MO_TESL)
+TRANS(ammax_db_d, gen_am, tcg_gen_atomic_fetch_smax_tl, MO_TEUQ)
+TRANS(ammin_db_w, gen_am, tcg_gen_atomic_fetch_smin_tl, MO_TESL)
+TRANS(ammin_db_d, gen_am, tcg_gen_atomic_fetch_smin_tl, MO_TEUQ)
+TRANS(ammax_db_wu, gen_am, tcg_gen_atomic_fetch_umax_tl, MO_TESL)
+TRANS(ammax_db_du, gen_am, tcg_gen_atomic_fetch_umax_tl, MO_TEUQ)
+TRANS(ammin_db_wu, gen_am, tcg_gen_atomic_fetch_umin_tl, MO_TESL)
+TRANS(ammin_db_du, gen_am, tcg_gen_atomic_fetch_umin_tl, MO_TEUQ)
diff --git a/target/loongarch/insn_trans/trans_memory.c.inc b/target/loongarch/insn_trans/trans_memory.c.inc
index 10914acf52..d5eb31147c 100644
--- a/target/loongarch/insn_trans/trans_memory.c.inc
+++ b/target/loongarch/insn_trans/trans_memory.c.inc
@@ -172,7 +172,7 @@ static bool gen_stptr(DisasContext *ctx, arg_rr_i *a, MemOp mop)
 
     if (a->imm) {
         temp = tcg_temp_new();
-        tcg_gen_addi_tl(temp, addr, a->im);
+        tcg_gen_addi_tl(temp, addr, a->imm);
         addr = temp;
     }
 
diff --git a/target/loongarch/insns.decode b/target/loongarch/insns.decode
index 1156e6965c..8d247aa68c 100644
--- a/target/loongarch/insns.decode
+++ b/target/loongarch/insns.decode
@@ -193,3 +193,47 @@ stle_b          0011 10000111 11100 ..... ..... .....    @rrr
 stle_h          0011 10000111 11101 ..... ..... .....    @rrr
 stle_w          0011 10000111 11110 ..... ..... .....    @rrr
 stle_d          0011 10000111 11111 ..... ..... .....    @rrr
+
+#
+# Fixed point atomic instruction
+#
+ll_w            0010 0000 .............. ..... .....     @rr_i14s2
+sc_w            0010 0001 .............. ..... .....     @rr_i14s2
+ll_d            0010 0010 .............. ..... .....     @rr_i14s2
+sc_d            0010 0011 .............. ..... .....     @rr_i14s2
+amswap_w        0011 10000110 00000 ..... ..... .....    @rrr
+amswap_d        0011 10000110 00001 ..... ..... .....    @rrr
+amadd_w         0011 10000110 00010 ..... ..... .....    @rrr
+amadd_d         0011 10000110 00011 ..... ..... .....    @rrr
+amand_w         0011 10000110 00100 ..... ..... .....    @rrr
+amand_d         0011 10000110 00101 ..... ..... .....    @rrr
+amor_w          0011 10000110 00110 ..... ..... .....    @rrr
+amor_d          0011 10000110 00111 ..... ..... .....    @rrr
+amxor_w         0011 10000110 01000 ..... ..... .....    @rrr
+amxor_d         0011 10000110 01001 ..... ..... .....    @rrr
+ammax_w         0011 10000110 01010 ..... ..... .....    @rrr
+ammax_d         0011 10000110 01011 ..... ..... .....    @rrr
+ammin_w         0011 10000110 01100 ..... ..... .....    @rrr
+ammin_d         0011 10000110 01101 ..... ..... .....    @rrr
+ammax_wu        0011 10000110 01110 ..... ..... .....    @rrr
+ammax_du        0011 10000110 01111 ..... ..... .....    @rrr
+ammin_wu        0011 10000110 10000 ..... ..... .....    @rrr
+ammin_du        0011 10000110 10001 ..... ..... .....    @rrr
+amswap_db_w     0011 10000110 10010 ..... ..... .....    @rrr
+amswap_db_d     0011 10000110 10011 ..... ..... .....    @rrr
+amadd_db_w      0011 10000110 10100 ..... ..... .....    @rrr
+amadd_db_d      0011 10000110 10101 ..... ..... .....    @rrr
+amand_db_w      0011 10000110 10110 ..... ..... .....    @rrr
+amand_db_d      0011 10000110 10111 ..... ..... .....    @rrr
+amor_db_w       0011 10000110 11000 ..... ..... .....    @rrr
+amor_db_d       0011 10000110 11001 ..... ..... .....    @rrr
+amxor_db_w      0011 10000110 11010 ..... ..... .....    @rrr
+amxor_db_d      0011 10000110 11011 ..... ..... .....    @rrr
+ammax_db_w      0011 10000110 11100 ..... ..... .....    @rrr
+ammax_db_d      0011 10000110 11101 ..... ..... .....    @rrr
+ammin_db_w      0011 10000110 11110 ..... ..... .....    @rrr
+ammin_db_d      0011 10000110 11111 ..... ..... .....    @rrr
+ammax_db_wu     0011 10000111 00000 ..... ..... .....    @rrr
+ammax_db_du     0011 10000111 00001 ..... ..... .....    @rrr
+ammin_db_wu     0011 10000111 00010 ..... ..... .....    @rrr
+ammin_db_du     0011 10000111 00011 ..... ..... .....    @rrr
diff --git a/target/loongarch/translate.c b/target/loongarch/translate.c
index b8fed26699..01791bf1a2 100644
--- a/target/loongarch/translate.c
+++ b/target/loongarch/translate.c
@@ -154,6 +154,7 @@ static void gen_set_gpr(int reg_num, TCGv t, DisasExtend dst_ext)
 #include "insn_trans/trans_shift.c.inc"
 #include "insn_trans/trans_bit.c.inc"
 #include "insn_trans/trans_memory.c.inc"
+#include "insn_trans/trans_atomic.c.inc"
 
 static void loongarch_tr_translate_insn(DisasContextBase *dcbase, CPUState *cs)
 {

From 8708a04a6145b4c9289fed28358fb1273dcd6aea Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:42:59 +0800
Subject: [PATCH 816/935] target/loongarch: Add fixed point extra instruction
 translation

This includes:
- CRC[C].W.{B/H/W/D}.W
- SYSCALL
- BREAK
- ASRT{LE/GT}.D
- RDTIME{L/H}.W, RDTIME.D
- CPUCFG

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-10-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/helper.h                     |  4 ++
 target/loongarch/insn_trans/trans_extra.c.inc | 68 +++++++++++++++++++
 target/loongarch/insns.decode                 | 19 ++++++
 target/loongarch/op_helper.c                  | 26 +++++++
 target/loongarch/translate.c                  |  1 +
 5 files changed, 118 insertions(+)
 create mode 100644 target/loongarch/insn_trans/trans_extra.c.inc

diff --git a/target/loongarch/helper.h b/target/loongarch/helper.h
index 100622bfc2..638c2efc51 100644
--- a/target/loongarch/helper.h
+++ b/target/loongarch/helper.h
@@ -11,3 +11,7 @@ DEF_HELPER_FLAGS_1(bitswap, TCG_CALL_NO_RWG_SE, tl, tl)
 
 DEF_HELPER_FLAGS_3(asrtle_d, TCG_CALL_NO_WG, void, env, tl, tl)
 DEF_HELPER_FLAGS_3(asrtgt_d, TCG_CALL_NO_WG, void, env, tl, tl)
+
+DEF_HELPER_FLAGS_3(crc32, TCG_CALL_NO_RWG_SE, tl, tl, tl, tl)
+DEF_HELPER_FLAGS_3(crc32c, TCG_CALL_NO_RWG_SE, tl, tl, tl, tl)
+DEF_HELPER_FLAGS_2(cpucfg, TCG_CALL_NO_RWG_SE, tl, env, tl)
diff --git a/target/loongarch/insn_trans/trans_extra.c.inc b/target/loongarch/insn_trans/trans_extra.c.inc
new file mode 100644
index 0000000000..549f75a867
--- /dev/null
+++ b/target/loongarch/insn_trans/trans_extra.c.inc
@@ -0,0 +1,68 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+static bool trans_break(DisasContext *ctx, arg_break *a)
+{
+    generate_exception(ctx, EXCCODE_BRK);
+    return true;
+}
+
+static bool trans_syscall(DisasContext *ctx, arg_syscall *a)
+{
+    generate_exception(ctx, EXCCODE_SYS);
+    return true;
+}
+
+static bool trans_asrtle_d(DisasContext *ctx, arg_asrtle_d * a)
+{
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = gpr_src(ctx, a->rk, EXT_NONE);
+
+    gen_helper_asrtle_d(cpu_env, src1, src2);
+    return true;
+}
+
+static bool trans_asrtgt_d(DisasContext *ctx, arg_asrtgt_d * a)
+{
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = gpr_src(ctx, a->rk, EXT_NONE);
+
+    gen_helper_asrtgt_d(cpu_env, src1, src2);
+    return true;
+}
+
+static bool trans_cpucfg(DisasContext *ctx, arg_cpucfg *a)
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+
+    gen_helper_cpucfg(dest, cpu_env, src1);
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+
+    return true;
+}
+
+static bool gen_crc(DisasContext *ctx, arg_rrr *a,
+                    void (*func)(TCGv, TCGv, TCGv, TCGv),
+                    TCGv tsz)
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_SIGN);
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = gpr_src(ctx, a->rk, EXT_NONE);
+
+    func(dest, src2, src1, tsz);
+    gen_set_gpr(a->rd, dest, EXT_SIGN);
+
+    return true;
+}
+
+TRANS(crc_w_b_w, gen_crc, gen_helper_crc32, tcg_constant_tl(1))
+TRANS(crc_w_h_w, gen_crc, gen_helper_crc32, tcg_constant_tl(2))
+TRANS(crc_w_w_w, gen_crc, gen_helper_crc32, tcg_constant_tl(4))
+TRANS(crc_w_d_w, gen_crc, gen_helper_crc32, tcg_constant_tl(8))
+TRANS(crcc_w_b_w, gen_crc, gen_helper_crc32c, tcg_constant_tl(1))
+TRANS(crcc_w_h_w, gen_crc, gen_helper_crc32c, tcg_constant_tl(2))
+TRANS(crcc_w_w_w, gen_crc, gen_helper_crc32c, tcg_constant_tl(4))
+TRANS(crcc_w_d_w, gen_crc, gen_helper_crc32c, tcg_constant_tl(8))
diff --git a/target/loongarch/insns.decode b/target/loongarch/insns.decode
index 8d247aa68c..98774dbddb 100644
--- a/target/loongarch/insns.decode
+++ b/target/loongarch/insns.decode
@@ -17,6 +17,7 @@
 &i            imm
 &r_i          rd imm
 &rr           rd rj
+&rr_jk        rj rk
 &rrr          rd rj rk
 &rr_i         rd rj imm
 &hint_r_i     hint rj imm
@@ -28,6 +29,7 @@
 #
 @i15                       .... ........ ..... imm:15    &i
 @rr               .... ........ ..... ..... rj:5 rd:5    &rr
+@rr_jk            .... ........ ..... rk:5 rj:5 .....    &rr_jk
 @rrr               .... ........ ..... rk:5 rj:5 rd:5    &rrr
 @r_i20                          .... ... imm:s20 rd:5    &r_i
 @rr_ui5           .... ........ ..... imm:5 rj:5 rd:5    &rr_i
@@ -237,3 +239,20 @@ ammax_db_wu     0011 10000111 00000 ..... ..... .....    @rrr
 ammax_db_du     0011 10000111 00001 ..... ..... .....    @rrr
 ammin_db_wu     0011 10000111 00010 ..... ..... .....    @rrr
 ammin_db_du     0011 10000111 00011 ..... ..... .....    @rrr
+
+#
+# Fixed point extra instruction
+#
+crc_w_b_w       0000 00000010 01000 ..... ..... .....    @rrr
+crc_w_h_w       0000 00000010 01001 ..... ..... .....    @rrr
+crc_w_w_w       0000 00000010 01010 ..... ..... .....    @rrr
+crc_w_d_w       0000 00000010 01011 ..... ..... .....    @rrr
+crcc_w_b_w      0000 00000010 01100 ..... ..... .....    @rrr
+crcc_w_h_w      0000 00000010 01101 ..... ..... .....    @rrr
+crcc_w_w_w      0000 00000010 01110 ..... ..... .....    @rrr
+crcc_w_d_w      0000 00000010 01111 ..... ..... .....    @rrr
+break           0000 00000010 10100 ...............      @i15
+syscall         0000 00000010 10110 ...............      @i15
+asrtle_d        0000 00000000 00010 ..... ..... 00000    @rr_jk
+asrtgt_d        0000 00000000 00011 ..... ..... 00000    @rr_jk
+cpucfg          0000 00000000 00000 11011 ..... .....    @rr
diff --git a/target/loongarch/op_helper.c b/target/loongarch/op_helper.c
index bd2db783c9..18e565ce7f 100644
--- a/target/loongarch/op_helper.c
+++ b/target/loongarch/op_helper.c
@@ -13,6 +13,8 @@
 #include "exec/exec-all.h"
 #include "exec/cpu_ldst.h"
 #include "internals.h"
+#include "qemu/crc32c.h"
+#include <zlib.h>
 
 /* Exceptions helpers */
 void helper_raise_exception(CPULoongArchState *env, uint32_t exception)
@@ -55,3 +57,27 @@ void helper_asrtgt_d(CPULoongArchState *env, target_ulong rj, target_ulong rk)
         do_raise_exception(env, EXCCODE_ADEM, GETPC());
     }
 }
+
+target_ulong helper_crc32(target_ulong val, target_ulong m, uint64_t sz)
+{
+    uint8_t buf[8];
+    target_ulong mask = ((sz * 8) == 64) ? -1ULL : ((1ULL << (sz * 8)) - 1);
+
+    m &= mask;
+    stq_le_p(buf, m);
+    return (int32_t) (crc32(val ^ 0xffffffff, buf, sz) ^ 0xffffffff);
+}
+
+target_ulong helper_crc32c(target_ulong val, target_ulong m, uint64_t sz)
+{
+    uint8_t buf[8];
+    target_ulong mask = ((sz * 8) == 64) ? -1ULL : ((1ULL << (sz * 8)) - 1);
+    m &= mask;
+    stq_le_p(buf, m);
+    return (int32_t) (crc32c(val, buf, sz) ^ 0xffffffff);
+}
+
+target_ulong helper_cpucfg(CPULoongArchState *env, target_ulong rj)
+{
+    return rj > 21 ? 0 : env->cpucfg[rj];
+}
diff --git a/target/loongarch/translate.c b/target/loongarch/translate.c
index 01791bf1a2..b946dc4d5f 100644
--- a/target/loongarch/translate.c
+++ b/target/loongarch/translate.c
@@ -155,6 +155,7 @@ static void gen_set_gpr(int reg_num, TCGv t, DisasExtend dst_ext)
 #include "insn_trans/trans_bit.c.inc"
 #include "insn_trans/trans_memory.c.inc"
 #include "insn_trans/trans_atomic.c.inc"
+#include "insn_trans/trans_extra.c.inc"
 
 static void loongarch_tr_translate_insn(DisasContextBase *dcbase, CPUState *cs)
 {

From d578ca6cbba6056f17901c869a6325d571bb124d Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:00 +0800
Subject: [PATCH 817/935] target/loongarch: Add floating point arithmetic
 instruction translation

This includes:
- F{ADD/SUB/MUL/DIV}.{S/D}
- F{MADD/MSUB/NMADD/NMSUB}.{S/D}
- F{MAX/MIN}.{S/D}
- F{MAXA/MINA}.{S/D}
- F{ABS/NEG}.{S/D}
- F{SQRT/RECIP/RSQRT}.{S/D}
- F{SCALEB/LOGB/COPYSIGN}.{S/D}
- FCLASS.{S/D}

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-11-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/cpu.c                        |   1 +
 target/loongarch/fpu_helper.c                 | 403 ++++++++++++++++++
 target/loongarch/helper.h                     |  37 ++
 .../loongarch/insn_trans/trans_farith.c.inc   | 105 +++++
 target/loongarch/insns.decode                 |  52 +++
 target/loongarch/internals.h                  |   2 +
 target/loongarch/translate.c                  |  11 +
 7 files changed, 611 insertions(+)
 create mode 100644 target/loongarch/fpu_helper.c
 create mode 100644 target/loongarch/insn_trans/trans_farith.c.inc

diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c
index 1f8e37906a..95900be257 100644
--- a/target/loongarch/cpu.c
+++ b/target/loongarch/cpu.c
@@ -196,6 +196,7 @@ static void loongarch_cpu_reset(DeviceState *dev)
     env->fcsr0_mask = FCSR0_M1 | FCSR0_M2 | FCSR0_M3;
     env->fcsr0 = 0x0;
 
+    restore_fp_status(env);
     cs->exception_index = -1;
 }
 
diff --git a/target/loongarch/fpu_helper.c b/target/loongarch/fpu_helper.c
new file mode 100644
index 0000000000..d7442cc188
--- /dev/null
+++ b/target/loongarch/fpu_helper.c
@@ -0,0 +1,403 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * LoongArch float point emulation helpers for QEMU
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "exec/helper-proto.h"
+#include "exec/exec-all.h"
+#include "exec/cpu_ldst.h"
+#include "fpu/softfloat.h"
+#include "internals.h"
+
+#define FLOAT_TO_INT32_OVERFLOW 0x7fffffff
+#define FLOAT_TO_INT64_OVERFLOW 0x7fffffffffffffffULL
+
+static inline uint64_t nanbox_s(float32 fp)
+{
+    return fp | MAKE_64BIT_MASK(32, 32);
+}
+
+/* Convert loongarch rounding mode in fcsr0 to IEEE library */
+static const FloatRoundMode ieee_rm[4] = {
+    float_round_nearest_even,
+    float_round_to_zero,
+    float_round_up,
+    float_round_down
+};
+
+void restore_fp_status(CPULoongArchState *env)
+{
+    set_float_rounding_mode(ieee_rm[(env->fcsr0 >> FCSR0_RM) & 0x3],
+                            &env->fp_status);
+    set_flush_to_zero(0, &env->fp_status);
+}
+
+static int ieee_ex_to_loongarch(int xcpt)
+{
+    int ret = 0;
+    if (xcpt & float_flag_invalid) {
+        ret |= FP_INVALID;
+    }
+    if (xcpt & float_flag_overflow) {
+        ret |= FP_OVERFLOW;
+    }
+    if (xcpt & float_flag_underflow) {
+        ret |= FP_UNDERFLOW;
+    }
+    if (xcpt & float_flag_divbyzero) {
+        ret |= FP_DIV0;
+    }
+    if (xcpt & float_flag_inexact) {
+        ret |= FP_INEXACT;
+    }
+    return ret;
+}
+
+static void update_fcsr0_mask(CPULoongArchState *env, uintptr_t pc, int mask)
+{
+    int flags = get_float_exception_flags(&env->fp_status);
+
+    set_float_exception_flags(0, &env->fp_status);
+
+    flags &= ~mask;
+
+    if (!flags) {
+        SET_FP_CAUSE(env->fcsr0, flags);
+        return;
+    } else {
+        flags = ieee_ex_to_loongarch(flags);
+        SET_FP_CAUSE(env->fcsr0, flags);
+    }
+
+    if (GET_FP_ENABLES(env->fcsr0) & flags) {
+        do_raise_exception(env, EXCCODE_FPE, pc);
+    } else {
+        UPDATE_FP_FLAGS(env->fcsr0, flags);
+    }
+}
+
+static void update_fcsr0(CPULoongArchState *env, uintptr_t pc)
+{
+    update_fcsr0_mask(env, pc, 0);
+}
+
+uint64_t helper_fadd_s(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+
+    fd = nanbox_s(float32_add((uint32_t)fj, (uint32_t)fk, &env->fp_status));
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fadd_d(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+
+    fd = float64_add(fj, fk, &env->fp_status);
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fsub_s(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+
+    fd = nanbox_s(float32_sub((uint32_t)fj, (uint32_t)fk, &env->fp_status));
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fsub_d(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+
+    fd = float64_sub(fj, fk, &env->fp_status);
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fmul_s(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+
+    fd = nanbox_s(float32_mul((uint32_t)fj, (uint32_t)fk, &env->fp_status));
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fmul_d(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+
+    fd = float64_mul(fj, fk, &env->fp_status);
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fdiv_s(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+
+    fd = nanbox_s(float32_div((uint32_t)fj, (uint32_t)fk, &env->fp_status));
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fdiv_d(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+
+    fd = float64_div(fj, fk, &env->fp_status);
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fmax_s(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+
+    fd = nanbox_s(float32_maxnum((uint32_t)fj, (uint32_t)fk, &env->fp_status));
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fmax_d(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+
+    fd = float64_maxnum(fj, fk, &env->fp_status);
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fmin_s(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+
+    fd = nanbox_s(float32_minnum((uint32_t)fj, (uint32_t)fk, &env->fp_status));
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fmin_d(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+
+    fd = float64_minnum(fj, fk, &env->fp_status);
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fmaxa_s(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+
+    fd = nanbox_s(float32_maxnummag((uint32_t)fj,
+                                    (uint32_t)fk, &env->fp_status));
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fmaxa_d(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+
+    fd = float64_maxnummag(fj, fk, &env->fp_status);
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fmina_s(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+
+    fd = nanbox_s(float32_minnummag((uint32_t)fj,
+                                    (uint32_t)fk, &env->fp_status));
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fmina_d(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+
+    fd = float64_minnummag(fj, fk, &env->fp_status);
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fscaleb_s(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+    int32_t n = (int32_t)fk;
+
+    fd = nanbox_s(float32_scalbn((uint32_t)fj,
+                                 n >  0x200 ?  0x200 :
+                                 n < -0x200 ? -0x200 : n,
+                                 &env->fp_status));
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fscaleb_d(CPULoongArchState *env, uint64_t fj, uint64_t fk)
+{
+    uint64_t fd;
+    int64_t n = (int64_t)fk;
+
+    fd = float64_scalbn(fj,
+                        n >  0x1000 ?  0x1000 :
+                        n < -0x1000 ? -0x1000 : n,
+                        &env->fp_status);
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fsqrt_s(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+
+    fd = nanbox_s(float32_sqrt((uint32_t)fj, &env->fp_status));
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fsqrt_d(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+
+    fd = float64_sqrt(fj, &env->fp_status);
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_frecip_s(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+
+    fd = nanbox_s(float32_div(float32_one, (uint32_t)fj, &env->fp_status));
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_frecip_d(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+
+    fd = float64_div(float64_one, fj, &env->fp_status);
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_frsqrt_s(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+    uint32_t fp;
+
+    fp = float32_sqrt((uint32_t)fj, &env->fp_status);
+    fd = nanbox_s(float32_div(float32_one, fp, &env->fp_status));
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_frsqrt_d(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fp, fd;
+
+    fp = float64_sqrt(fj, &env->fp_status);
+    fd = float64_div(float64_one, fp, &env->fp_status);
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_flogb_s(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+    uint32_t fp;
+    float_status *status = &env->fp_status;
+    FloatRoundMode old_mode = get_float_rounding_mode(status);
+
+    set_float_rounding_mode(float_round_down, status);
+    fp = float32_log2((uint32_t)fj, status);
+    fd = nanbox_s(float32_round_to_int(fp, status));
+    set_float_rounding_mode(old_mode, status);
+    update_fcsr0_mask(env, GETPC(), float_flag_inexact);
+    return fd;
+}
+
+uint64_t helper_flogb_d(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+    float_status *status = &env->fp_status;
+    FloatRoundMode old_mode = get_float_rounding_mode(status);
+
+    set_float_rounding_mode(float_round_down, status);
+    fd = float64_log2(fj, status);
+    fd = float64_round_to_int(fd, status);
+    set_float_rounding_mode(old_mode, status);
+    update_fcsr0_mask(env, GETPC(), float_flag_inexact);
+    return fd;
+}
+
+uint64_t helper_fclass_s(CPULoongArchState *env, uint64_t fj)
+{
+    float32 f = fj;
+    bool sign = float32_is_neg(f);
+
+    if (float32_is_infinity(f)) {
+        return sign ? 1 << 2 : 1 << 6;
+    } else if (float32_is_zero(f)) {
+        return sign ? 1 << 5 : 1 << 9;
+    } else if (float32_is_zero_or_denormal(f)) {
+        return sign ? 1 << 4 : 1 << 8;
+    } else if (float32_is_any_nan(f)) {
+        float_status s = { }; /* for snan_bit_is_one */
+        return float32_is_quiet_nan(f, &s) ? 1 << 1 : 1 << 0;
+    } else {
+        return sign ? 1 << 3 : 1 << 7;
+    }
+}
+
+uint64_t helper_fclass_d(CPULoongArchState *env, uint64_t fj)
+{
+    float64 f = fj;
+    bool sign = float64_is_neg(f);
+
+    if (float64_is_infinity(f)) {
+        return sign ? 1 << 2 : 1 << 6;
+    } else if (float64_is_zero(f)) {
+        return sign ? 1 << 5 : 1 << 9;
+    } else if (float64_is_zero_or_denormal(f)) {
+        return sign ? 1 << 4 : 1 << 8;
+    } else if (float64_is_any_nan(f)) {
+        float_status s = { }; /* for snan_bit_is_one */
+        return float64_is_quiet_nan(f, &s) ? 1 << 1 : 1 << 0;
+    } else {
+        return sign ? 1 << 3 : 1 << 7;
+    }
+}
+
+uint64_t helper_fmuladd_s(CPULoongArchState *env, uint64_t fj,
+                          uint64_t fk, uint64_t fa, uint32_t flag)
+{
+    uint64_t fd;
+
+    fd = nanbox_s(float32_muladd((uint32_t)fj, (uint32_t)fk,
+                                 (uint32_t)fa, flag, &env->fp_status));
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fmuladd_d(CPULoongArchState *env, uint64_t fj,
+                          uint64_t fk, uint64_t fa, uint32_t flag)
+{
+    uint64_t fd;
+
+    fd = float64_muladd(fj, fk, fa, flag, &env->fp_status);
+    update_fcsr0(env, GETPC());
+    return fd;
+}
diff --git a/target/loongarch/helper.h b/target/loongarch/helper.h
index 638c2efc51..840bad9b2f 100644
--- a/target/loongarch/helper.h
+++ b/target/loongarch/helper.h
@@ -15,3 +15,40 @@ DEF_HELPER_FLAGS_3(asrtgt_d, TCG_CALL_NO_WG, void, env, tl, tl)
 DEF_HELPER_FLAGS_3(crc32, TCG_CALL_NO_RWG_SE, tl, tl, tl, tl)
 DEF_HELPER_FLAGS_3(crc32c, TCG_CALL_NO_RWG_SE, tl, tl, tl, tl)
 DEF_HELPER_FLAGS_2(cpucfg, TCG_CALL_NO_RWG_SE, tl, env, tl)
+
+/* Floating-point helper */
+DEF_HELPER_FLAGS_3(fadd_s, TCG_CALL_NO_WG, i64, env, i64, i64)
+DEF_HELPER_FLAGS_3(fadd_d, TCG_CALL_NO_WG, i64, env, i64, i64)
+DEF_HELPER_FLAGS_3(fsub_s, TCG_CALL_NO_WG, i64, env, i64, i64)
+DEF_HELPER_FLAGS_3(fsub_d, TCG_CALL_NO_WG, i64, env, i64, i64)
+DEF_HELPER_FLAGS_3(fmul_s, TCG_CALL_NO_WG, i64, env, i64, i64)
+DEF_HELPER_FLAGS_3(fmul_d, TCG_CALL_NO_WG, i64, env, i64, i64)
+DEF_HELPER_FLAGS_3(fdiv_s, TCG_CALL_NO_WG, i64, env, i64, i64)
+DEF_HELPER_FLAGS_3(fdiv_d, TCG_CALL_NO_WG, i64, env, i64, i64)
+DEF_HELPER_FLAGS_3(fmax_s, TCG_CALL_NO_WG, i64, env, i64, i64)
+DEF_HELPER_FLAGS_3(fmax_d, TCG_CALL_NO_WG, i64, env, i64, i64)
+DEF_HELPER_FLAGS_3(fmin_s, TCG_CALL_NO_WG, i64, env, i64, i64)
+DEF_HELPER_FLAGS_3(fmin_d, TCG_CALL_NO_WG, i64, env, i64, i64)
+DEF_HELPER_FLAGS_3(fmaxa_s, TCG_CALL_NO_WG, i64, env, i64, i64)
+DEF_HELPER_FLAGS_3(fmaxa_d, TCG_CALL_NO_WG, i64, env, i64, i64)
+DEF_HELPER_FLAGS_3(fmina_s, TCG_CALL_NO_WG, i64, env, i64, i64)
+DEF_HELPER_FLAGS_3(fmina_d, TCG_CALL_NO_WG, i64, env, i64, i64)
+
+DEF_HELPER_FLAGS_5(fmuladd_s, TCG_CALL_NO_WG, i64, env, i64, i64, i64, i32)
+DEF_HELPER_FLAGS_5(fmuladd_d, TCG_CALL_NO_WG, i64, env, i64, i64, i64, i32)
+
+DEF_HELPER_FLAGS_3(fscaleb_s, TCG_CALL_NO_WG, i64, env, i64, i64)
+DEF_HELPER_FLAGS_3(fscaleb_d, TCG_CALL_NO_WG, i64, env, i64, i64)
+
+DEF_HELPER_FLAGS_2(flogb_s, TCG_CALL_NO_WG, i64, env, i64)
+DEF_HELPER_FLAGS_2(flogb_d, TCG_CALL_NO_WG, i64, env, i64)
+
+DEF_HELPER_FLAGS_2(fsqrt_s, TCG_CALL_NO_WG, i64, env, i64)
+DEF_HELPER_FLAGS_2(fsqrt_d, TCG_CALL_NO_WG, i64, env, i64)
+DEF_HELPER_FLAGS_2(frsqrt_s, TCG_CALL_NO_WG, i64, env, i64)
+DEF_HELPER_FLAGS_2(frsqrt_d, TCG_CALL_NO_WG, i64, env, i64)
+DEF_HELPER_FLAGS_2(frecip_s, TCG_CALL_NO_WG, i64, env, i64)
+DEF_HELPER_FLAGS_2(frecip_d, TCG_CALL_NO_WG, i64, env, i64)
+
+DEF_HELPER_FLAGS_2(fclass_s, TCG_CALL_NO_RWG_SE, i64, env, i64)
+DEF_HELPER_FLAGS_2(fclass_d, TCG_CALL_NO_RWG_SE, i64, env, i64)
diff --git a/target/loongarch/insn_trans/trans_farith.c.inc b/target/loongarch/insn_trans/trans_farith.c.inc
new file mode 100644
index 0000000000..65ad2ffab8
--- /dev/null
+++ b/target/loongarch/insn_trans/trans_farith.c.inc
@@ -0,0 +1,105 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+static bool gen_fff(DisasContext *ctx, arg_fff *a,
+                    void (*func)(TCGv, TCGv_env, TCGv, TCGv))
+{
+    func(cpu_fpr[a->fd], cpu_env, cpu_fpr[a->fj], cpu_fpr[a->fk]);
+    return true;
+}
+
+static bool gen_ff(DisasContext *ctx, arg_ff *a,
+                   void (*func)(TCGv, TCGv_env, TCGv))
+{
+    func(cpu_fpr[a->fd], cpu_env, cpu_fpr[a->fj]);
+    return true;
+}
+
+static bool gen_muladd(DisasContext *ctx, arg_ffff *a,
+                       void (*func)(TCGv, TCGv_env, TCGv, TCGv, TCGv, TCGv_i32),
+                       int flag)
+{
+    TCGv_i32 tflag = tcg_constant_i32(flag);
+    func(cpu_fpr[a->fd], cpu_env, cpu_fpr[a->fj],
+         cpu_fpr[a->fk], cpu_fpr[a->fa], tflag);
+    return true;
+}
+
+static bool trans_fcopysign_s(DisasContext *ctx, arg_fcopysign_s *a)
+{
+    tcg_gen_deposit_i64(cpu_fpr[a->fd], cpu_fpr[a->fk], cpu_fpr[a->fj], 0, 31);
+    return true;
+}
+
+static bool trans_fcopysign_d(DisasContext *ctx, arg_fcopysign_d *a)
+{
+    tcg_gen_deposit_i64(cpu_fpr[a->fd], cpu_fpr[a->fk], cpu_fpr[a->fj], 0, 63);
+    return true;
+}
+
+static bool trans_fabs_s(DisasContext *ctx, arg_fabs_s *a)
+{
+    tcg_gen_andi_i64(cpu_fpr[a->fd], cpu_fpr[a->fj], MAKE_64BIT_MASK(0, 31));
+    gen_nanbox_s(cpu_fpr[a->fd], cpu_fpr[a->fd]);
+    return true;
+}
+
+static bool trans_fabs_d(DisasContext *ctx, arg_fabs_d *a)
+{
+    tcg_gen_andi_i64(cpu_fpr[a->fd], cpu_fpr[a->fj], MAKE_64BIT_MASK(0, 63));
+    return true;
+}
+
+static bool trans_fneg_s(DisasContext *ctx, arg_fneg_s *a)
+{
+    tcg_gen_xori_i64(cpu_fpr[a->fd], cpu_fpr[a->fj], 0x80000000);
+    gen_nanbox_s(cpu_fpr[a->fd], cpu_fpr[a->fd]);
+    return true;
+}
+
+static bool trans_fneg_d(DisasContext *ctx, arg_fneg_d *a)
+{
+    tcg_gen_xori_i64(cpu_fpr[a->fd], cpu_fpr[a->fj], 0x8000000000000000LL);
+    return true;
+}
+
+TRANS(fadd_s, gen_fff, gen_helper_fadd_s)
+TRANS(fadd_d, gen_fff, gen_helper_fadd_d)
+TRANS(fsub_s, gen_fff, gen_helper_fsub_s)
+TRANS(fsub_d, gen_fff, gen_helper_fsub_d)
+TRANS(fmul_s, gen_fff, gen_helper_fmul_s)
+TRANS(fmul_d, gen_fff, gen_helper_fmul_d)
+TRANS(fdiv_s, gen_fff, gen_helper_fdiv_s)
+TRANS(fdiv_d, gen_fff, gen_helper_fdiv_d)
+TRANS(fmax_s, gen_fff, gen_helper_fmax_s)
+TRANS(fmax_d, gen_fff, gen_helper_fmax_d)
+TRANS(fmin_s, gen_fff, gen_helper_fmin_s)
+TRANS(fmin_d, gen_fff, gen_helper_fmin_d)
+TRANS(fmaxa_s, gen_fff, gen_helper_fmaxa_s)
+TRANS(fmaxa_d, gen_fff, gen_helper_fmaxa_d)
+TRANS(fmina_s, gen_fff, gen_helper_fmina_s)
+TRANS(fmina_d, gen_fff, gen_helper_fmina_d)
+TRANS(fscaleb_s, gen_fff, gen_helper_fscaleb_s)
+TRANS(fscaleb_d, gen_fff, gen_helper_fscaleb_d)
+TRANS(fsqrt_s, gen_ff, gen_helper_fsqrt_s)
+TRANS(fsqrt_d, gen_ff, gen_helper_fsqrt_d)
+TRANS(frecip_s, gen_ff, gen_helper_frecip_s)
+TRANS(frecip_d, gen_ff, gen_helper_frecip_d)
+TRANS(frsqrt_s, gen_ff, gen_helper_frsqrt_s)
+TRANS(frsqrt_d, gen_ff, gen_helper_frsqrt_d)
+TRANS(flogb_s, gen_ff, gen_helper_flogb_s)
+TRANS(flogb_d, gen_ff, gen_helper_flogb_d)
+TRANS(fclass_s, gen_ff, gen_helper_fclass_s)
+TRANS(fclass_d, gen_ff, gen_helper_fclass_d)
+TRANS(fmadd_s, gen_muladd, gen_helper_fmuladd_s, 0)
+TRANS(fmadd_d, gen_muladd, gen_helper_fmuladd_d, 0)
+TRANS(fmsub_s, gen_muladd, gen_helper_fmuladd_s, float_muladd_negate_c)
+TRANS(fmsub_d, gen_muladd, gen_helper_fmuladd_d, float_muladd_negate_c)
+TRANS(fnmadd_s, gen_muladd, gen_helper_fmuladd_s,
+      float_muladd_negate_product | float_muladd_negate_c)
+TRANS(fnmadd_d, gen_muladd, gen_helper_fmuladd_d,
+      float_muladd_negate_product | float_muladd_negate_c)
+TRANS(fnmsub_s, gen_muladd, gen_helper_fmuladd_s, float_muladd_negate_product)
+TRANS(fnmsub_d, gen_muladd, gen_helper_fmuladd_d, float_muladd_negate_product)
diff --git a/target/loongarch/insns.decode b/target/loongarch/insns.decode
index 98774dbddb..6455e09ebe 100644
--- a/target/loongarch/insns.decode
+++ b/target/loongarch/insns.decode
@@ -23,6 +23,9 @@
 &hint_r_i     hint rj imm
 &rrr_sa       rd rj rk sa
 &rr_ms_ls     rd rj ms ls
+&ff           fd fj
+&fff          fd fj fk
+&ffff         fd fj fk fa
 
 #
 # Formats
@@ -44,6 +47,9 @@
 @rrr_sa3         .... ........ .. sa:3 rk:5 rj:5 rd:5    &rrr_sa
 @rr_2bw            .... ....... ms:5 . ls:5 rj:5 rd:5    &rr_ms_ls
 @rr_2bd               .... ...... ms:6 ls:6 rj:5 rd:5    &rr_ms_ls
+@ff               .... ........ ..... ..... fj:5 fd:5    &ff
+@fff               .... ........ ..... fk:5 fj:5 fd:5    &fff
+@ffff               .... ........ fa:5 fk:5 fj:5 fd:5    &ffff
 
 #
 # Fixed point arithmetic operation instruction
@@ -256,3 +262,49 @@ syscall         0000 00000010 10110 ...............      @i15
 asrtle_d        0000 00000000 00010 ..... ..... 00000    @rr_jk
 asrtgt_d        0000 00000000 00011 ..... ..... 00000    @rr_jk
 cpucfg          0000 00000000 00000 11011 ..... .....    @rr
+
+#
+# Floating point arithmetic operation instruction
+#
+fadd_s          0000 00010000 00001 ..... ..... .....    @fff
+fadd_d          0000 00010000 00010 ..... ..... .....    @fff
+fsub_s          0000 00010000 00101 ..... ..... .....    @fff
+fsub_d          0000 00010000 00110 ..... ..... .....    @fff
+fmul_s          0000 00010000 01001 ..... ..... .....    @fff
+fmul_d          0000 00010000 01010 ..... ..... .....    @fff
+fdiv_s          0000 00010000 01101 ..... ..... .....    @fff
+fdiv_d          0000 00010000 01110 ..... ..... .....    @fff
+fmadd_s         0000 10000001 ..... ..... ..... .....    @ffff
+fmadd_d         0000 10000010 ..... ..... ..... .....    @ffff
+fmsub_s         0000 10000101 ..... ..... ..... .....    @ffff
+fmsub_d         0000 10000110 ..... ..... ..... .....    @ffff
+fnmadd_s        0000 10001001 ..... ..... ..... .....    @ffff
+fnmadd_d        0000 10001010 ..... ..... ..... .....    @ffff
+fnmsub_s        0000 10001101 ..... ..... ..... .....    @ffff
+fnmsub_d        0000 10001110 ..... ..... ..... .....    @ffff
+fmax_s          0000 00010000 10001 ..... ..... .....    @fff
+fmax_d          0000 00010000 10010 ..... ..... .....    @fff
+fmin_s          0000 00010000 10101 ..... ..... .....    @fff
+fmin_d          0000 00010000 10110 ..... ..... .....    @fff
+fmaxa_s         0000 00010000 11001 ..... ..... .....    @fff
+fmaxa_d         0000 00010000 11010 ..... ..... .....    @fff
+fmina_s         0000 00010000 11101 ..... ..... .....    @fff
+fmina_d         0000 00010000 11110 ..... ..... .....    @fff
+fabs_s          0000 00010001 01000 00001 ..... .....    @ff
+fabs_d          0000 00010001 01000 00010 ..... .....    @ff
+fneg_s          0000 00010001 01000 00101 ..... .....    @ff
+fneg_d          0000 00010001 01000 00110 ..... .....    @ff
+fsqrt_s         0000 00010001 01000 10001 ..... .....    @ff
+fsqrt_d         0000 00010001 01000 10010 ..... .....    @ff
+frecip_s        0000 00010001 01000 10101 ..... .....    @ff
+frecip_d        0000 00010001 01000 10110 ..... .....    @ff
+frsqrt_s        0000 00010001 01000 11001 ..... .....    @ff
+frsqrt_d        0000 00010001 01000 11010 ..... .....    @ff
+fscaleb_s       0000 00010001 00001 ..... ..... .....    @fff
+fscaleb_d       0000 00010001 00010 ..... ..... .....    @fff
+flogb_s         0000 00010001 01000 01001 ..... .....    @ff
+flogb_d         0000 00010001 01000 01010 ..... .....    @ff
+fcopysign_s     0000 00010001 00101 ..... ..... .....    @fff
+fcopysign_d     0000 00010001 00110 ..... ..... .....    @fff
+fclass_s        0000 00010001 01000 01101 ..... .....    @ff
+fclass_d        0000 00010001 01000 01110 ..... .....    @ff
diff --git a/target/loongarch/internals.h b/target/loongarch/internals.h
index 9f02287a90..43b865ec21 100644
--- a/target/loongarch/internals.h
+++ b/target/loongarch/internals.h
@@ -18,4 +18,6 @@ void G_NORETURN do_raise_exception(CPULoongArchState *env,
 
 const char *loongarch_exception_name(int32_t exception);
 
+void restore_fp_status(CPULoongArchState *env);
+
 #endif
diff --git a/target/loongarch/translate.c b/target/loongarch/translate.c
index b946dc4d5f..050d5f2c3b 100644
--- a/target/loongarch/translate.c
+++ b/target/loongarch/translate.c
@@ -15,6 +15,7 @@
 #include "exec/translator.h"
 #include "exec/log.h"
 #include "qemu/qemu-print.h"
+#include "fpu/softfloat.h"
 #include "translate.h"
 #include "internals.h"
 
@@ -36,6 +37,15 @@ static inline int shl_2(DisasContext *ctx, int x)
     return x << 2;
 }
 
+/*
+ * LoongArch the upper 32 bits are undefined ("can be any value").
+ * QEMU chooses to nanbox, because it is most likely to show guest bugs early.
+ */
+static void gen_nanbox_s(TCGv_i64 out, TCGv_i64 in)
+{
+    tcg_gen_ori_i64(out, in, MAKE_64BIT_MASK(32, 32));
+}
+
 void generate_exception(DisasContext *ctx, int excp)
 {
     tcg_gen_movi_tl(cpu_pc, ctx->base.pc_next);
@@ -156,6 +166,7 @@ static void gen_set_gpr(int reg_num, TCGv t, DisasExtend dst_ext)
 #include "insn_trans/trans_memory.c.inc"
 #include "insn_trans/trans_atomic.c.inc"
 #include "insn_trans/trans_extra.c.inc"
+#include "insn_trans/trans_farith.c.inc"
 
 static void loongarch_tr_translate_insn(DisasContextBase *dcbase, CPUState *cs)
 {

From 9b7410763aedcd99b7f2302c564c8778179ef347 Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:01 +0800
Subject: [PATCH 818/935] target/loongarch: Add floating point comparison
 instruction translation

This includes:
- FCMP.cond.{S/D}

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-12-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/fpu_helper.c                | 60 ++++++++++++++++++++
 target/loongarch/helper.h                    |  9 +++
 target/loongarch/insn_trans/trans_fcmp.c.inc | 56 ++++++++++++++++++
 target/loongarch/insns.decode                |  8 +++
 target/loongarch/internals.h                 |  5 ++
 target/loongarch/translate.c                 |  1 +
 6 files changed, 139 insertions(+)
 create mode 100644 target/loongarch/insn_trans/trans_fcmp.c.inc

diff --git a/target/loongarch/fpu_helper.c b/target/loongarch/fpu_helper.c
index d7442cc188..1e514cce74 100644
--- a/target/loongarch/fpu_helper.c
+++ b/target/loongarch/fpu_helper.c
@@ -401,3 +401,63 @@ uint64_t helper_fmuladd_d(CPULoongArchState *env, uint64_t fj,
     update_fcsr0(env, GETPC());
     return fd;
 }
+
+static uint64_t fcmp_common(CPULoongArchState *env, FloatRelation cmp,
+                            uint32_t flags)
+{
+    bool ret;
+
+    switch (cmp) {
+    case float_relation_less:
+        ret = (flags & FCMP_LT);
+        break;
+    case float_relation_equal:
+        ret = (flags & FCMP_EQ);
+        break;
+    case float_relation_greater:
+        ret = (flags & FCMP_GT);
+        break;
+    case float_relation_unordered:
+        ret = (flags & FCMP_UN);
+        break;
+    default:
+        g_assert_not_reached();
+    }
+    update_fcsr0(env, GETPC());
+
+    return ret;
+}
+
+/* fcmp_cXXX_s */
+uint64_t helper_fcmp_c_s(CPULoongArchState *env, uint64_t fj,
+                         uint64_t fk, uint32_t flags)
+{
+    FloatRelation cmp = float32_compare_quiet((uint32_t)fj,
+                                              (uint32_t)fk, &env->fp_status);
+    return fcmp_common(env, cmp, flags);
+}
+
+/* fcmp_sXXX_s */
+uint64_t helper_fcmp_s_s(CPULoongArchState *env, uint64_t fj,
+                         uint64_t fk, uint32_t flags)
+{
+    FloatRelation cmp = float32_compare((uint32_t)fj,
+                                        (uint32_t)fk, &env->fp_status);
+    return fcmp_common(env, cmp, flags);
+}
+
+/* fcmp_cXXX_d */
+uint64_t helper_fcmp_c_d(CPULoongArchState *env, uint64_t fj,
+                         uint64_t fk, uint32_t flags)
+{
+    FloatRelation cmp = float64_compare_quiet(fj, fk, &env->fp_status);
+    return fcmp_common(env, cmp, flags);
+}
+
+/* fcmp_sXXX_d */
+uint64_t helper_fcmp_s_d(CPULoongArchState *env, uint64_t fj,
+                         uint64_t fk, uint32_t flags)
+{
+    FloatRelation cmp = float64_compare(fj, fk, &env->fp_status);
+    return fcmp_common(env, cmp, flags);
+}
diff --git a/target/loongarch/helper.h b/target/loongarch/helper.h
index 840bad9b2f..25a891bf8b 100644
--- a/target/loongarch/helper.h
+++ b/target/loongarch/helper.h
@@ -52,3 +52,12 @@ DEF_HELPER_FLAGS_2(frecip_d, TCG_CALL_NO_WG, i64, env, i64)
 
 DEF_HELPER_FLAGS_2(fclass_s, TCG_CALL_NO_RWG_SE, i64, env, i64)
 DEF_HELPER_FLAGS_2(fclass_d, TCG_CALL_NO_RWG_SE, i64, env, i64)
+
+/* fcmp.cXXX.s */
+DEF_HELPER_4(fcmp_c_s, i64, env, i64, i64, i32)
+/* fcmp.sXXX.s */
+DEF_HELPER_4(fcmp_s_s, i64, env, i64, i64, i32)
+/* fcmp.cXXX.d */
+DEF_HELPER_4(fcmp_c_d, i64, env, i64, i64, i32)
+/* fcmp.sXXX.d */
+DEF_HELPER_4(fcmp_s_d, i64, env, i64, i64, i32)
diff --git a/target/loongarch/insn_trans/trans_fcmp.c.inc b/target/loongarch/insn_trans/trans_fcmp.c.inc
new file mode 100644
index 0000000000..93a6a2230f
--- /dev/null
+++ b/target/loongarch/insn_trans/trans_fcmp.c.inc
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+/* bit0(signaling/quiet) bit1(lt) bit2(eq) bit3(un) bit4(neq) */
+static uint32_t get_fcmp_flags(int cond)
+{
+    uint32_t flags = 0;
+
+    if (cond & 0x1) {
+        flags |= FCMP_LT;
+    }
+    if (cond & 0x2) {
+        flags |= FCMP_EQ;
+    }
+    if (cond & 0x4) {
+        flags |= FCMP_UN;
+    }
+    if (cond & 0x8) {
+        flags |= FCMP_GT | FCMP_LT;
+    }
+    return flags;
+}
+
+static bool trans_fcmp_cond_s(DisasContext *ctx, arg_fcmp_cond_s *a)
+{
+    TCGv var = tcg_temp_new();
+    uint32_t flags;
+    void (*fn)(TCGv, TCGv_env, TCGv, TCGv, TCGv_i32);
+
+    fn = (a->fcond & 1 ? gen_helper_fcmp_s_s : gen_helper_fcmp_c_s);
+    flags = get_fcmp_flags(a->fcond >> 1);
+
+    fn(var, cpu_env, cpu_fpr[a->fj], cpu_fpr[a->fk], tcg_constant_i32(flags));
+
+    tcg_gen_st8_tl(var, cpu_env, offsetof(CPULoongArchState, cf[a->cd]));
+    tcg_temp_free(var);
+    return true;
+}
+
+static bool trans_fcmp_cond_d(DisasContext *ctx, arg_fcmp_cond_d *a)
+{
+    TCGv var = tcg_temp_new();
+    uint32_t flags;
+    void (*fn)(TCGv, TCGv_env, TCGv, TCGv, TCGv_i32);
+    fn = (a->fcond & 1 ? gen_helper_fcmp_s_d : gen_helper_fcmp_c_d);
+    flags = get_fcmp_flags(a->fcond >> 1);
+
+    fn(var, cpu_env, cpu_fpr[a->fj], cpu_fpr[a->fk], tcg_constant_i32(flags));
+
+    tcg_gen_st8_tl(var, cpu_env, offsetof(CPULoongArchState, cf[a->cd]));
+
+    tcg_temp_free(var);
+    return true;
+}
diff --git a/target/loongarch/insns.decode b/target/loongarch/insns.decode
index 6455e09ebe..d7ed80b0b9 100644
--- a/target/loongarch/insns.decode
+++ b/target/loongarch/insns.decode
@@ -26,6 +26,7 @@
 &ff           fd fj
 &fff          fd fj fk
 &ffff         fd fj fk fa
+&cff_fcond    cd fj fk fcond
 
 #
 # Formats
@@ -50,6 +51,7 @@
 @ff               .... ........ ..... ..... fj:5 fd:5    &ff
 @fff               .... ........ ..... fk:5 fj:5 fd:5    &fff
 @ffff               .... ........ fa:5 fk:5 fj:5 fd:5    &ffff
+@cff_fcond    .... ........ fcond:5 fk:5 fj:5 .. cd:3    &cff_fcond
 
 #
 # Fixed point arithmetic operation instruction
@@ -308,3 +310,9 @@ fcopysign_s     0000 00010001 00101 ..... ..... .....    @fff
 fcopysign_d     0000 00010001 00110 ..... ..... .....    @fff
 fclass_s        0000 00010001 01000 01101 ..... .....    @ff
 fclass_d        0000 00010001 01000 01110 ..... .....    @ff
+
+#
+# Floating point compare instruction
+#
+fcmp_cond_s     0000 11000001 ..... ..... ..... 00 ...   @cff_fcond
+fcmp_cond_d     0000 11000010 ..... ..... ..... 00 ...   @cff_fcond
diff --git a/target/loongarch/internals.h b/target/loongarch/internals.h
index 43b865ec21..1a3b39e0be 100644
--- a/target/loongarch/internals.h
+++ b/target/loongarch/internals.h
@@ -8,6 +8,11 @@
 #ifndef LOONGARCH_INTERNALS_H
 #define LOONGARCH_INTERNALS_H
 
+#define FCMP_LT   0b0001  /* fp0 < fp1 */
+#define FCMP_EQ   0b0010  /* fp0 = fp1 */
+#define FCMP_UN   0b0100  /* unordered */
+#define FCMP_GT   0b1000  /* fp0 > fp1 */
+
 void loongarch_translate_init(void);
 
 void loongarch_cpu_dump_state(CPUState *cpu, FILE *f, int flags);
diff --git a/target/loongarch/translate.c b/target/loongarch/translate.c
index 050d5f2c3b..a338db50b7 100644
--- a/target/loongarch/translate.c
+++ b/target/loongarch/translate.c
@@ -167,6 +167,7 @@ static void gen_set_gpr(int reg_num, TCGv t, DisasExtend dst_ext)
 #include "insn_trans/trans_atomic.c.inc"
 #include "insn_trans/trans_extra.c.inc"
 #include "insn_trans/trans_farith.c.inc"
+#include "insn_trans/trans_fcmp.c.inc"
 
 static void loongarch_tr_translate_insn(DisasContextBase *dcbase, CPUState *cs)
 {

From 7c1f88703d216cc6b2e30689fbc295423ca13d40 Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:02 +0800
Subject: [PATCH 819/935] target/loongarch: Add floating point conversion
 instruction translation

This includes:
- FCVT.S.D, FCVT.D.S
- FFINT.{S/D}.{W/L}, FTINT.{W/L}.{S/D}
- FTINT{RM/RP/RZ/RNE}.{W/L}.{S/D}
- FRINT.{S/D}

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-13-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/fpu_helper.c                | 393 +++++++++++++++++++
 target/loongarch/helper.h                    |  29 ++
 target/loongarch/insn_trans/trans_fcnv.c.inc |  33 ++
 target/loongarch/insns.decode                |  32 ++
 target/loongarch/translate.c                 |   1 +
 5 files changed, 488 insertions(+)
 create mode 100644 target/loongarch/insn_trans/trans_fcnv.c.inc

diff --git a/target/loongarch/fpu_helper.c b/target/loongarch/fpu_helper.c
index 1e514cce74..81466678eb 100644
--- a/target/loongarch/fpu_helper.c
+++ b/target/loongarch/fpu_helper.c
@@ -461,3 +461,396 @@ uint64_t helper_fcmp_s_d(CPULoongArchState *env, uint64_t fj,
     FloatRelation cmp = float64_compare(fj, fk, &env->fp_status);
     return fcmp_common(env, cmp, flags);
 }
+
+/* floating point conversion */
+uint64_t helper_fcvt_s_d(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+
+    fd = nanbox_s(float64_to_float32(fj, &env->fp_status));
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_fcvt_d_s(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+
+    fd = float32_to_float64((uint32_t)fj, &env->fp_status);
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ffint_s_w(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+
+    fd = nanbox_s(int32_to_float32((int32_t)fj, &env->fp_status));
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ffint_s_l(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+
+    fd = nanbox_s(int64_to_float32(fj, &env->fp_status));
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ffint_d_w(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+
+    fd = int32_to_float64((int32_t)fj, &env->fp_status);
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ffint_d_l(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+
+    fd = int64_to_float64(fj, &env->fp_status);
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_frint_s(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+
+    fd = (uint64_t)(float32_round_to_int((uint32_t)fj, &env->fp_status));
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_frint_d(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+
+    fd = float64_round_to_int(fj, &env->fp_status);
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftintrm_l_d(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+    FloatRoundMode old_mode = get_float_rounding_mode(&env->fp_status);
+
+    set_float_rounding_mode(float_round_down, &env->fp_status);
+    fd = float64_to_int64(fj, &env->fp_status);
+    set_float_rounding_mode(old_mode, &env->fp_status);
+
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT64_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftintrm_l_s(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+    FloatRoundMode old_mode = get_float_rounding_mode(&env->fp_status);
+
+    set_float_rounding_mode(float_round_down, &env->fp_status);
+    fd = float32_to_int64((uint32_t)fj, &env->fp_status);
+    set_float_rounding_mode(old_mode, &env->fp_status);
+
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT64_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftintrm_w_d(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+    FloatRoundMode old_mode = get_float_rounding_mode(&env->fp_status);
+
+    set_float_rounding_mode(float_round_down, &env->fp_status);
+    fd = (uint64_t)float64_to_int32(fj, &env->fp_status);
+    set_float_rounding_mode(old_mode, &env->fp_status);
+
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT32_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftintrm_w_s(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+    FloatRoundMode old_mode = get_float_rounding_mode(&env->fp_status);
+
+    set_float_rounding_mode(float_round_down, &env->fp_status);
+    fd = (uint64_t)float32_to_int32((uint32_t)fj, &env->fp_status);
+    set_float_rounding_mode(old_mode, &env->fp_status);
+
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT32_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftintrp_l_d(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+    FloatRoundMode old_mode = get_float_rounding_mode(&env->fp_status);
+
+    set_float_rounding_mode(float_round_up, &env->fp_status);
+    fd = float64_to_int64(fj, &env->fp_status);
+    set_float_rounding_mode(old_mode, &env->fp_status);
+
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT64_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftintrp_l_s(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+    FloatRoundMode old_mode = get_float_rounding_mode(&env->fp_status);
+
+    set_float_rounding_mode(float_round_up, &env->fp_status);
+    fd = float32_to_int64((uint32_t)fj, &env->fp_status);
+    set_float_rounding_mode(old_mode, &env->fp_status);
+
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT64_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftintrp_w_d(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+    FloatRoundMode old_mode = get_float_rounding_mode(&env->fp_status);
+
+    set_float_rounding_mode(float_round_up, &env->fp_status);
+    fd = (uint64_t)float64_to_int32(fj, &env->fp_status);
+    set_float_rounding_mode(old_mode, &env->fp_status);
+
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT32_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftintrp_w_s(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+    FloatRoundMode old_mode = get_float_rounding_mode(&env->fp_status);
+
+    set_float_rounding_mode(float_round_up, &env->fp_status);
+    fd = (uint64_t)float32_to_int32((uint32_t)fj, &env->fp_status);
+    set_float_rounding_mode(old_mode, &env->fp_status);
+
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT32_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftintrz_l_d(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+    FloatRoundMode old_mode = get_float_rounding_mode(&env->fp_status);
+
+    fd = float64_to_int64_round_to_zero(fj, &env->fp_status);
+    set_float_rounding_mode(old_mode, &env->fp_status);
+
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT64_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftintrz_l_s(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+    FloatRoundMode old_mode = get_float_rounding_mode(&env->fp_status);
+
+    fd = float32_to_int64_round_to_zero((uint32_t)fj, &env->fp_status);
+    set_float_rounding_mode(old_mode, &env->fp_status);
+
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT64_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftintrz_w_d(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+    FloatRoundMode old_mode = get_float_rounding_mode(&env->fp_status);
+
+    fd = (uint64_t)float64_to_int32_round_to_zero(fj, &env->fp_status);
+    set_float_rounding_mode(old_mode, &env->fp_status);
+
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT32_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftintrz_w_s(CPULoongArchState *env, uint64_t fj)
+{
+    uint32_t fd;
+    FloatRoundMode old_mode = get_float_rounding_mode(&env->fp_status);
+
+    fd = float32_to_int32_round_to_zero((uint32_t)fj, &env->fp_status);
+    set_float_rounding_mode(old_mode, &env->fp_status);
+
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT32_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return (uint64_t)fd;
+}
+
+uint64_t helper_ftintrne_l_d(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+    FloatRoundMode old_mode = get_float_rounding_mode(&env->fp_status);
+
+    set_float_rounding_mode(float_round_nearest_even, &env->fp_status);
+    fd = float64_to_int64(fj, &env->fp_status);
+    set_float_rounding_mode(old_mode, &env->fp_status);
+
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT64_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftintrne_l_s(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+    FloatRoundMode old_mode = get_float_rounding_mode(&env->fp_status);
+
+    set_float_rounding_mode(float_round_nearest_even, &env->fp_status);
+    fd = float32_to_int64((uint32_t)fj, &env->fp_status);
+    set_float_rounding_mode(old_mode, &env->fp_status);
+
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT64_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftintrne_w_d(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+    FloatRoundMode old_mode = get_float_rounding_mode(&env->fp_status);
+
+    set_float_rounding_mode(float_round_nearest_even, &env->fp_status);
+    fd = (uint64_t)float64_to_int32(fj, &env->fp_status);
+    set_float_rounding_mode(old_mode, &env->fp_status);
+
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT32_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftintrne_w_s(CPULoongArchState *env, uint64_t fj)
+{
+    uint32_t fd;
+    FloatRoundMode old_mode = get_float_rounding_mode(&env->fp_status);
+
+    set_float_rounding_mode(float_round_nearest_even, &env->fp_status);
+    fd = float32_to_int32((uint32_t)fj, &env->fp_status);
+    set_float_rounding_mode(old_mode, &env->fp_status);
+
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT32_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return (uint64_t)fd;
+}
+
+uint64_t helper_ftint_l_d(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+
+    fd = float64_to_int64(fj, &env->fp_status);
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT64_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftint_l_s(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+
+    fd = float32_to_int64((uint32_t)fj, &env->fp_status);
+    if (get_float_exception_flags(&env->fp_status) &
+        (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT64_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftint_w_s(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+
+    fd = (uint64_t)float32_to_int32((uint32_t)fj, &env->fp_status);
+    if (get_float_exception_flags(&env->fp_status)
+        & (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT32_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
+
+uint64_t helper_ftint_w_d(CPULoongArchState *env, uint64_t fj)
+{
+    uint64_t fd;
+
+    fd = (uint64_t)float64_to_int32(fj, &env->fp_status);
+    if (get_float_exception_flags(&env->fp_status)
+        & (float_flag_invalid | float_flag_overflow)) {
+        fd = FLOAT_TO_INT32_OVERFLOW;
+    }
+    update_fcsr0(env, GETPC());
+    return fd;
+}
diff --git a/target/loongarch/helper.h b/target/loongarch/helper.h
index 25a891bf8b..1e8749433a 100644
--- a/target/loongarch/helper.h
+++ b/target/loongarch/helper.h
@@ -61,3 +61,32 @@ DEF_HELPER_4(fcmp_s_s, i64, env, i64, i64, i32)
 DEF_HELPER_4(fcmp_c_d, i64, env, i64, i64, i32)
 /* fcmp.sXXX.d */
 DEF_HELPER_4(fcmp_s_d, i64, env, i64, i64, i32)
+
+DEF_HELPER_2(fcvt_d_s, i64, env, i64)
+DEF_HELPER_2(fcvt_s_d, i64, env, i64)
+DEF_HELPER_2(ffint_d_w, i64, env, i64)
+DEF_HELPER_2(ffint_d_l, i64, env, i64)
+DEF_HELPER_2(ffint_s_w, i64, env, i64)
+DEF_HELPER_2(ffint_s_l, i64, env, i64)
+DEF_HELPER_2(ftintrm_l_s, i64, env, i64)
+DEF_HELPER_2(ftintrm_l_d, i64, env, i64)
+DEF_HELPER_2(ftintrm_w_s, i64, env, i64)
+DEF_HELPER_2(ftintrm_w_d, i64, env, i64)
+DEF_HELPER_2(ftintrp_l_s, i64, env, i64)
+DEF_HELPER_2(ftintrp_l_d, i64, env, i64)
+DEF_HELPER_2(ftintrp_w_s, i64, env, i64)
+DEF_HELPER_2(ftintrp_w_d, i64, env, i64)
+DEF_HELPER_2(ftintrz_l_s, i64, env, i64)
+DEF_HELPER_2(ftintrz_l_d, i64, env, i64)
+DEF_HELPER_2(ftintrz_w_s, i64, env, i64)
+DEF_HELPER_2(ftintrz_w_d, i64, env, i64)
+DEF_HELPER_2(ftintrne_l_s, i64, env, i64)
+DEF_HELPER_2(ftintrne_l_d, i64, env, i64)
+DEF_HELPER_2(ftintrne_w_s, i64, env, i64)
+DEF_HELPER_2(ftintrne_w_d, i64, env, i64)
+DEF_HELPER_2(ftint_l_s, i64, env, i64)
+DEF_HELPER_2(ftint_l_d, i64, env, i64)
+DEF_HELPER_2(ftint_w_s, i64, env, i64)
+DEF_HELPER_2(ftint_w_d, i64, env, i64)
+DEF_HELPER_2(frint_s, i64, env, i64)
+DEF_HELPER_2(frint_d, i64, env, i64)
diff --git a/target/loongarch/insn_trans/trans_fcnv.c.inc b/target/loongarch/insn_trans/trans_fcnv.c.inc
new file mode 100644
index 0000000000..c1c6918ad1
--- /dev/null
+++ b/target/loongarch/insn_trans/trans_fcnv.c.inc
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+TRANS(fcvt_s_d, gen_ff, gen_helper_fcvt_s_d)
+TRANS(fcvt_d_s, gen_ff, gen_helper_fcvt_d_s)
+TRANS(ftintrm_w_s, gen_ff, gen_helper_ftintrm_w_s)
+TRANS(ftintrm_w_d, gen_ff, gen_helper_ftintrm_w_d)
+TRANS(ftintrm_l_s, gen_ff, gen_helper_ftintrm_l_s)
+TRANS(ftintrm_l_d, gen_ff, gen_helper_ftintrm_l_d)
+TRANS(ftintrp_w_s, gen_ff, gen_helper_ftintrp_w_s)
+TRANS(ftintrp_w_d, gen_ff, gen_helper_ftintrp_w_d)
+TRANS(ftintrp_l_s, gen_ff, gen_helper_ftintrp_l_s)
+TRANS(ftintrp_l_d, gen_ff, gen_helper_ftintrp_l_d)
+TRANS(ftintrz_w_s, gen_ff, gen_helper_ftintrz_w_s)
+TRANS(ftintrz_w_d, gen_ff, gen_helper_ftintrz_w_d)
+TRANS(ftintrz_l_s, gen_ff, gen_helper_ftintrz_l_s)
+TRANS(ftintrz_l_d, gen_ff, gen_helper_ftintrz_l_d)
+TRANS(ftintrne_w_s, gen_ff, gen_helper_ftintrne_w_s)
+TRANS(ftintrne_w_d, gen_ff, gen_helper_ftintrne_w_d)
+TRANS(ftintrne_l_s, gen_ff, gen_helper_ftintrne_l_s)
+TRANS(ftintrne_l_d, gen_ff, gen_helper_ftintrne_l_d)
+TRANS(ftint_w_s, gen_ff, gen_helper_ftint_w_s)
+TRANS(ftint_w_d, gen_ff, gen_helper_ftint_w_d)
+TRANS(ftint_l_s, gen_ff, gen_helper_ftint_l_s)
+TRANS(ftint_l_d, gen_ff, gen_helper_ftint_l_d)
+TRANS(ffint_s_w, gen_ff, gen_helper_ffint_s_w)
+TRANS(ffint_s_l, gen_ff, gen_helper_ffint_s_l)
+TRANS(ffint_d_w, gen_ff, gen_helper_ffint_d_w)
+TRANS(ffint_d_l, gen_ff, gen_helper_ffint_d_l)
+TRANS(frint_s, gen_ff, gen_helper_frint_s)
+TRANS(frint_d, gen_ff, gen_helper_frint_d)
diff --git a/target/loongarch/insns.decode b/target/loongarch/insns.decode
index d7ed80b0b9..b9f135d36f 100644
--- a/target/loongarch/insns.decode
+++ b/target/loongarch/insns.decode
@@ -316,3 +316,35 @@ fclass_d        0000 00010001 01000 01110 ..... .....    @ff
 #
 fcmp_cond_s     0000 11000001 ..... ..... ..... 00 ...   @cff_fcond
 fcmp_cond_d     0000 11000010 ..... ..... ..... 00 ...   @cff_fcond
+
+#
+# Floating point conversion instruction
+#
+fcvt_s_d        0000 00010001 10010 00110 ..... .....    @ff
+fcvt_d_s        0000 00010001 10010 01001 ..... .....    @ff
+ftintrm_w_s     0000 00010001 10100 00001 ..... .....    @ff
+ftintrm_w_d     0000 00010001 10100 00010 ..... .....    @ff
+ftintrm_l_s     0000 00010001 10100 01001 ..... .....    @ff
+ftintrm_l_d     0000 00010001 10100 01010 ..... .....    @ff
+ftintrp_w_s     0000 00010001 10100 10001 ..... .....    @ff
+ftintrp_w_d     0000 00010001 10100 10010 ..... .....    @ff
+ftintrp_l_s     0000 00010001 10100 11001 ..... .....    @ff
+ftintrp_l_d     0000 00010001 10100 11010 ..... .....    @ff
+ftintrz_w_s     0000 00010001 10101 00001 ..... .....    @ff
+ftintrz_w_d     0000 00010001 10101 00010 ..... .....    @ff
+ftintrz_l_s     0000 00010001 10101 01001 ..... .....    @ff
+ftintrz_l_d     0000 00010001 10101 01010 ..... .....    @ff
+ftintrne_w_s    0000 00010001 10101 10001 ..... .....    @ff
+ftintrne_w_d    0000 00010001 10101 10010 ..... .....    @ff
+ftintrne_l_s    0000 00010001 10101 11001 ..... .....    @ff
+ftintrne_l_d    0000 00010001 10101 11010 ..... .....    @ff
+ftint_w_s       0000 00010001 10110 00001 ..... .....    @ff
+ftint_w_d       0000 00010001 10110 00010 ..... .....    @ff
+ftint_l_s       0000 00010001 10110 01001 ..... .....    @ff
+ftint_l_d       0000 00010001 10110 01010 ..... .....    @ff
+ffint_s_w       0000 00010001 11010 00100 ..... .....    @ff
+ffint_s_l       0000 00010001 11010 00110 ..... .....    @ff
+ffint_d_w       0000 00010001 11010 01000 ..... .....    @ff
+ffint_d_l       0000 00010001 11010 01010 ..... .....    @ff
+frint_s         0000 00010001 11100 10001 ..... .....    @ff
+frint_d         0000 00010001 11100 10010 ..... .....    @ff
diff --git a/target/loongarch/translate.c b/target/loongarch/translate.c
index a338db50b7..26ef1e366c 100644
--- a/target/loongarch/translate.c
+++ b/target/loongarch/translate.c
@@ -168,6 +168,7 @@ static void gen_set_gpr(int reg_num, TCGv t, DisasExtend dst_ext)
 #include "insn_trans/trans_extra.c.inc"
 #include "insn_trans/trans_farith.c.inc"
 #include "insn_trans/trans_fcmp.c.inc"
+#include "insn_trans/trans_fcnv.c.inc"
 
 static void loongarch_tr_translate_insn(DisasContextBase *dcbase, CPUState *cs)
 {

From b7dabd5624326b116d6147c659de22037f357cf8 Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:03 +0800
Subject: [PATCH 820/935] target/loongarch: Add floating point move instruction
 translation

This includes:
- FMOV.{S/D}
- FSEL
- MOVGR2FR.{W/D}, MOVGR2FRH.W
- MOVFR2GR.{S/D}, MOVFRH2GR.S
- MOVGR2FCSR, MOVFCSR2GR
- MOVFR2CF, MOVCF2FR
- MOVGR2CF, MOVCF2GR

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-14-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/fpu_helper.c                |   6 +
 target/loongarch/helper.h                    |   2 +
 target/loongarch/insn_trans/trans_fmov.c.inc | 157 +++++++++++++++++++
 target/loongarch/insns.decode                |  37 +++++
 target/loongarch/translate.c                 |   1 +
 5 files changed, 203 insertions(+)
 create mode 100644 target/loongarch/insn_trans/trans_fmov.c.inc

diff --git a/target/loongarch/fpu_helper.c b/target/loongarch/fpu_helper.c
index 81466678eb..3d0cb8dd0d 100644
--- a/target/loongarch/fpu_helper.c
+++ b/target/loongarch/fpu_helper.c
@@ -854,3 +854,9 @@ uint64_t helper_ftint_w_d(CPULoongArchState *env, uint64_t fj)
     update_fcsr0(env, GETPC());
     return fd;
 }
+
+void helper_set_rounding_mode(CPULoongArchState *env, uint32_t fcsr0)
+{
+    set_float_rounding_mode(ieee_rm[(fcsr0 >> FCSR0_RM) & 0x3],
+                            &env->fp_status);
+}
diff --git a/target/loongarch/helper.h b/target/loongarch/helper.h
index 1e8749433a..da1a2bced7 100644
--- a/target/loongarch/helper.h
+++ b/target/loongarch/helper.h
@@ -90,3 +90,5 @@ DEF_HELPER_2(ftint_w_s, i64, env, i64)
 DEF_HELPER_2(ftint_w_d, i64, env, i64)
 DEF_HELPER_2(frint_s, i64, env, i64)
 DEF_HELPER_2(frint_d, i64, env, i64)
+
+DEF_HELPER_FLAGS_2(set_rounding_mode, TCG_CALL_NO_RWG, void, env, i32)
diff --git a/target/loongarch/insn_trans/trans_fmov.c.inc b/target/loongarch/insn_trans/trans_fmov.c.inc
new file mode 100644
index 0000000000..24753d4568
--- /dev/null
+++ b/target/loongarch/insn_trans/trans_fmov.c.inc
@@ -0,0 +1,157 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+static const uint32_t fcsr_mask[4] = {
+    UINT32_MAX, FCSR0_M1, FCSR0_M2, FCSR0_M3
+};
+
+static bool trans_fsel(DisasContext *ctx, arg_fsel *a)
+{
+    TCGv zero = tcg_constant_tl(0);
+    TCGv cond = tcg_temp_new();
+
+    tcg_gen_ld8u_tl(cond, cpu_env, offsetof(CPULoongArchState, cf[a->ca]));
+    tcg_gen_movcond_tl(TCG_COND_EQ, cpu_fpr[a->fd], cond, zero,
+                       cpu_fpr[a->fj], cpu_fpr[a->fk]);
+    tcg_temp_free(cond);
+
+    return true;
+}
+
+static bool gen_f2f(DisasContext *ctx, arg_ff *a,
+                    void (*func)(TCGv, TCGv), bool nanbox)
+{
+    TCGv dest = cpu_fpr[a->fd];
+    TCGv src = cpu_fpr[a->fj];
+
+    func(dest, src);
+    if (nanbox) {
+        gen_nanbox_s(cpu_fpr[a->fd], cpu_fpr[a->fd]);
+    }
+
+    return true;
+}
+
+static bool gen_r2f(DisasContext *ctx, arg_fr *a,
+                    void (*func)(TCGv, TCGv))
+{
+    TCGv src = gpr_src(ctx, a->rj, EXT_NONE);
+
+    func(cpu_fpr[a->fd], src);
+    return true;
+}
+
+static bool gen_f2r(DisasContext *ctx, arg_rf *a,
+                    void (*func)(TCGv, TCGv))
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+
+    func(dest, cpu_fpr[a->fj]);
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+
+    return true;
+}
+
+static bool trans_movgr2fcsr(DisasContext *ctx, arg_movgr2fcsr *a)
+{
+    uint32_t mask = fcsr_mask[a->fcsrd];
+    TCGv Rj = gpr_src(ctx, a->rj, EXT_NONE);
+
+    if (mask == UINT32_MAX) {
+        tcg_gen_extrl_i64_i32(cpu_fcsr0, Rj);
+    } else {
+        TCGv_i32 temp = tcg_temp_new_i32();
+
+        tcg_gen_extrl_i64_i32(temp, Rj);
+        tcg_gen_andi_i32(temp, temp, mask);
+        tcg_gen_andi_i32(cpu_fcsr0, cpu_fcsr0, ~mask);
+        tcg_gen_or_i32(cpu_fcsr0, cpu_fcsr0, temp);
+        tcg_temp_free_i32(temp);
+
+        /*
+         * Install the new rounding mode to fpu_status, if changed.
+         * Note that FCSR3 is exactly the rounding mode field.
+         */
+        if (mask != FCSR0_M3) {
+            return true;
+        }
+    }
+    gen_helper_set_rounding_mode(cpu_env, cpu_fcsr0);
+
+    return true;
+}
+
+static bool trans_movfcsr2gr(DisasContext *ctx, arg_movfcsr2gr *a)
+{
+    TCGv_i32 temp = tcg_temp_new_i32();
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+
+    tcg_gen_andi_i32(temp, cpu_fcsr0, fcsr_mask[a->fcsrs]);
+    tcg_gen_ext_i32_i64(dest, temp);
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+    tcg_temp_free_i32(temp);
+
+    return true;
+}
+
+static void gen_movgr2fr_w(TCGv dest, TCGv src)
+{
+    tcg_gen_deposit_i64(dest, dest, src, 0, 32);
+}
+
+static void gen_movgr2frh_w(TCGv dest, TCGv src)
+{
+    tcg_gen_deposit_i64(dest, dest, src, 32, 32);
+}
+
+static void gen_movfrh2gr_s(TCGv dest, TCGv src)
+{
+    tcg_gen_sextract_tl(dest, src, 32, 32);
+}
+
+static bool trans_movfr2cf(DisasContext *ctx, arg_movfr2cf *a)
+{
+    TCGv t0 = tcg_temp_new();
+
+    tcg_gen_andi_tl(t0, cpu_fpr[a->fj], 0x1);
+    tcg_gen_st8_tl(t0, cpu_env, offsetof(CPULoongArchState, cf[a->cd & 0x7]));
+    tcg_temp_free(t0);
+
+    return true;
+}
+
+static bool trans_movcf2fr(DisasContext *ctx, arg_movcf2fr *a)
+{
+    tcg_gen_ld8u_tl(cpu_fpr[a->fd], cpu_env,
+                    offsetof(CPULoongArchState, cf[a->cj & 0x7]));
+    return true;
+}
+
+static bool trans_movgr2cf(DisasContext *ctx, arg_movgr2cf *a)
+{
+    TCGv t0 = tcg_temp_new();
+
+    tcg_gen_andi_tl(t0, gpr_src(ctx, a->rj, EXT_NONE), 0x1);
+    tcg_gen_st8_tl(t0, cpu_env, offsetof(CPULoongArchState, cf[a->cd & 0x7]));
+    tcg_temp_free(t0);
+
+    return true;
+}
+
+static bool trans_movcf2gr(DisasContext *ctx, arg_movcf2gr *a)
+{
+    tcg_gen_ld8u_tl(gpr_dst(ctx, a->rd, EXT_NONE), cpu_env,
+                    offsetof(CPULoongArchState, cf[a->cj & 0x7]));
+    return true;
+}
+
+TRANS(fmov_s, gen_f2f, tcg_gen_mov_tl, true)
+TRANS(fmov_d, gen_f2f, tcg_gen_mov_tl, false)
+TRANS(movgr2fr_w, gen_r2f, gen_movgr2fr_w)
+TRANS(movgr2fr_d, gen_r2f, tcg_gen_mov_tl)
+TRANS(movgr2frh_w, gen_r2f, gen_movgr2frh_w)
+TRANS(movfr2gr_s, gen_f2r, tcg_gen_ext32s_tl)
+TRANS(movfr2gr_d, gen_f2r, tcg_gen_mov_tl)
+TRANS(movfrh2gr_s, gen_f2r, gen_movfrh2gr_s)
diff --git a/target/loongarch/insns.decode b/target/loongarch/insns.decode
index b9f135d36f..c62a4f6dcd 100644
--- a/target/loongarch/insns.decode
+++ b/target/loongarch/insns.decode
@@ -27,6 +27,15 @@
 &fff          fd fj fk
 &ffff         fd fj fk fa
 &cff_fcond    cd fj fk fcond
+&fffc         fd fj fk ca
+&fr           fd rj
+&rf           rd fj
+&fcsrd_r      fcsrd rj
+&r_fcsrs      rd fcsrs
+&cf           cd fj
+&fc           fd cj
+&cr           cd rj
+&rc           rd cj
 
 #
 # Formats
@@ -52,6 +61,15 @@
 @fff               .... ........ ..... fk:5 fj:5 fd:5    &fff
 @ffff               .... ........ fa:5 fk:5 fj:5 fd:5    &ffff
 @cff_fcond    .... ........ fcond:5 fk:5 fj:5 .. cd:3    &cff_fcond
+@fffc            .... ........ .. ca:3 fk:5 fj:5 fd:5    &fffc
+@fr               .... ........ ..... ..... rj:5 fd:5    &fr
+@rf               .... ........ ..... ..... fj:5 rd:5    &rf
+@fcsrd_r       .... ........ ..... ..... rj:5 fcsrd:5    &fcsrd_r
+@r_fcsrs       .... ........ ..... ..... fcsrs:5 rd:5    &r_fcsrs
+@cf            .... ........ ..... ..... fj:5 .. cd:3    &cf
+@fc            .... ........ ..... ..... .. cj:3 fd:5    &fc
+@cr            .... ........ ..... ..... rj:5 .. cd:3    &cr
+@rc            .... ........ ..... ..... .. cj:3 rd:5    &rc
 
 #
 # Fixed point arithmetic operation instruction
@@ -348,3 +366,22 @@ ffint_d_w       0000 00010001 11010 01000 ..... .....    @ff
 ffint_d_l       0000 00010001 11010 01010 ..... .....    @ff
 frint_s         0000 00010001 11100 10001 ..... .....    @ff
 frint_d         0000 00010001 11100 10010 ..... .....    @ff
+
+#
+# Floating point move instruction
+#
+fmov_s          0000 00010001 01001 00101 ..... .....    @ff
+fmov_d          0000 00010001 01001 00110 ..... .....    @ff
+fsel            0000 11010000 00 ... ..... ..... .....   @fffc
+movgr2fr_w      0000 00010001 01001 01001 ..... .....    @fr
+movgr2fr_d      0000 00010001 01001 01010 ..... .....    @fr
+movgr2frh_w     0000 00010001 01001 01011 ..... .....    @fr
+movfr2gr_s      0000 00010001 01001 01101 ..... .....    @rf
+movfr2gr_d      0000 00010001 01001 01110 ..... .....    @rf
+movfrh2gr_s     0000 00010001 01001 01111 ..... .....    @rf
+movgr2fcsr      0000 00010001 01001 10000 ..... .....    @fcsrd_r
+movfcsr2gr      0000 00010001 01001 10010 ..... .....    @r_fcsrs
+movfr2cf        0000 00010001 01001 10100 ..... 00 ...   @cf
+movcf2fr        0000 00010001 01001 10101 00 ... .....   @fc
+movgr2cf        0000 00010001 01001 10110 ..... 00 ...   @cr
+movcf2gr        0000 00010001 01001 10111 00 ... .....   @rc
diff --git a/target/loongarch/translate.c b/target/loongarch/translate.c
index 26ef1e366c..26d60b50fd 100644
--- a/target/loongarch/translate.c
+++ b/target/loongarch/translate.c
@@ -169,6 +169,7 @@ static void gen_set_gpr(int reg_num, TCGv t, DisasExtend dst_ext)
 #include "insn_trans/trans_farith.c.inc"
 #include "insn_trans/trans_fcmp.c.inc"
 #include "insn_trans/trans_fcnv.c.inc"
+#include "insn_trans/trans_fmov.c.inc"
 
 static void loongarch_tr_translate_insn(DisasContextBase *dcbase, CPUState *cs)
 {

From e616bdfd0159965bd65f12be83ebf28dc8c44bae Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:04 +0800
Subject: [PATCH 821/935] target/loongarch: Add floating point load/store
 instruction translation

This includes:
- FLD.{S/D}, FST.{S/D}
- FLDX.{S/D}, FSTX.{S/D}
- FLD{GT/LE}.{S/D}, FST{GT/LE}.{S/D}

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-15-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 .../loongarch/insn_trans/trans_fmemory.c.inc  | 153 ++++++++++++++++++
 target/loongarch/insns.decode                 |  24 +++
 target/loongarch/translate.c                  |   1 +
 3 files changed, 178 insertions(+)
 create mode 100644 target/loongarch/insn_trans/trans_fmemory.c.inc

diff --git a/target/loongarch/insn_trans/trans_fmemory.c.inc b/target/loongarch/insn_trans/trans_fmemory.c.inc
new file mode 100644
index 0000000000..74ee98f63a
--- /dev/null
+++ b/target/loongarch/insn_trans/trans_fmemory.c.inc
@@ -0,0 +1,153 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+static void maybe_nanbox_load(TCGv freg, MemOp mop)
+{
+    if ((mop & MO_SIZE) == MO_32) {
+        gen_nanbox_s(freg, freg);
+    }
+}
+
+static bool gen_fload_i(DisasContext *ctx, arg_fr_i *a, MemOp mop)
+{
+    TCGv addr = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv temp = NULL;
+
+    if (a->imm) {
+        temp = tcg_temp_new();
+        tcg_gen_addi_tl(temp, addr, a->imm);
+        addr = temp;
+    }
+
+    tcg_gen_qemu_ld_tl(cpu_fpr[a->fd], addr, ctx->mem_idx, mop);
+    maybe_nanbox_load(cpu_fpr[a->fd], mop);
+
+    if (temp) {
+        tcg_temp_free(temp);
+    }
+
+    return true;
+}
+
+static bool gen_fstore_i(DisasContext *ctx, arg_fr_i *a, MemOp mop)
+{
+    TCGv addr = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv temp = NULL;
+
+    if (a->imm) {
+        temp = tcg_temp_new();
+        tcg_gen_addi_tl(temp, addr, a->imm);
+        addr = temp;
+    }
+
+    tcg_gen_qemu_st_tl(cpu_fpr[a->fd], addr, ctx->mem_idx, mop);
+
+    if (temp) {
+        tcg_temp_free(temp);
+    }
+    return true;
+}
+
+static bool gen_floadx(DisasContext *ctx, arg_frr *a, MemOp mop)
+{
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = gpr_src(ctx, a->rk, EXT_NONE);
+    TCGv addr = tcg_temp_new();
+
+    tcg_gen_add_tl(addr, src1, src2);
+    tcg_gen_qemu_ld_tl(cpu_fpr[a->fd], addr, ctx->mem_idx, mop);
+    maybe_nanbox_load(cpu_fpr[a->fd], mop);
+    tcg_temp_free(addr);
+
+    return true;
+}
+
+static bool gen_fstorex(DisasContext *ctx, arg_frr *a, MemOp mop)
+{
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = gpr_src(ctx, a->rk, EXT_NONE);
+    TCGv addr = tcg_temp_new();
+
+    tcg_gen_add_tl(addr, src1, src2);
+    tcg_gen_qemu_st_tl(cpu_fpr[a->fd], addr, ctx->mem_idx, mop);
+    tcg_temp_free(addr);
+
+    return true;
+}
+
+static bool gen_fload_gt(DisasContext *ctx, arg_frr *a, MemOp mop)
+{
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = gpr_src(ctx, a->rk, EXT_NONE);
+    TCGv addr = tcg_temp_new();
+
+    gen_helper_asrtgt_d(cpu_env, src1, src2);
+    tcg_gen_add_tl(addr, src1, src2);
+    tcg_gen_qemu_ld_tl(cpu_fpr[a->fd], addr, ctx->mem_idx, mop);
+    maybe_nanbox_load(cpu_fpr[a->fd], mop);
+    tcg_temp_free(addr);
+
+    return true;
+}
+
+static bool gen_fstore_gt(DisasContext *ctx, arg_frr *a, MemOp mop)
+{
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = gpr_src(ctx, a->rk, EXT_NONE);
+    TCGv addr = tcg_temp_new();
+
+    gen_helper_asrtgt_d(cpu_env, src1, src2);
+    tcg_gen_add_tl(addr, src1, src2);
+    tcg_gen_qemu_st_tl(cpu_fpr[a->fd], addr, ctx->mem_idx, mop);
+    tcg_temp_free(addr);
+
+    return true;
+}
+
+static bool gen_fload_le(DisasContext *ctx, arg_frr *a, MemOp mop)
+{
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = gpr_src(ctx, a->rk, EXT_NONE);
+    TCGv addr = tcg_temp_new();
+
+    gen_helper_asrtle_d(cpu_env, src1, src2);
+    tcg_gen_add_tl(addr, src1, src2);
+    tcg_gen_qemu_ld_tl(cpu_fpr[a->fd], addr, ctx->mem_idx, mop);
+    maybe_nanbox_load(cpu_fpr[a->fd], mop);
+    tcg_temp_free(addr);
+
+    return true;
+}
+
+static bool gen_fstore_le(DisasContext *ctx, arg_frr *a, MemOp mop)
+{
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = gpr_src(ctx, a->rk, EXT_NONE);
+    TCGv addr = tcg_temp_new();
+
+    gen_helper_asrtle_d(cpu_env, src1, src2);
+    tcg_gen_add_tl(addr, src1, src2);
+    tcg_gen_qemu_st_tl(cpu_fpr[a->fd], addr, ctx->mem_idx, mop);
+    tcg_temp_free(addr);
+
+    return true;
+}
+
+TRANS(fld_s, gen_fload_i, MO_TEUL)
+TRANS(fst_s, gen_fstore_i, MO_TEUL)
+TRANS(fld_d, gen_fload_i, MO_TEUQ)
+TRANS(fst_d, gen_fstore_i, MO_TEUQ)
+TRANS(fldx_s, gen_floadx, MO_TEUL)
+TRANS(fldx_d, gen_floadx, MO_TEUQ)
+TRANS(fstx_s, gen_fstorex, MO_TEUL)
+TRANS(fstx_d, gen_fstorex, MO_TEUQ)
+TRANS(fldgt_s, gen_fload_gt, MO_TEUL)
+TRANS(fldgt_d, gen_fload_gt, MO_TEUQ)
+TRANS(fldle_s, gen_fload_le, MO_TEUL)
+TRANS(fldle_d, gen_fload_le, MO_TEUQ)
+TRANS(fstgt_s, gen_fstore_gt, MO_TEUL)
+TRANS(fstgt_d, gen_fstore_gt, MO_TEUQ)
+TRANS(fstle_s, gen_fstore_le, MO_TEUL)
+TRANS(fstle_d, gen_fstore_le, MO_TEUQ)
diff --git a/target/loongarch/insns.decode b/target/loongarch/insns.decode
index c62a4f6dcd..8f286e7233 100644
--- a/target/loongarch/insns.decode
+++ b/target/loongarch/insns.decode
@@ -36,6 +36,8 @@
 &fc           fd cj
 &cr           cd rj
 &rc           rd cj
+&frr          fd rj rk
+&fr_i         fd rj imm
 
 #
 # Formats
@@ -70,6 +72,8 @@
 @fc            .... ........ ..... ..... .. cj:3 fd:5    &fc
 @cr            .... ........ ..... ..... rj:5 .. cd:3    &cr
 @rc            .... ........ ..... ..... .. cj:3 rd:5    &rc
+@frr               .... ........ ..... rk:5 rj:5 fd:5    &frr
+@fr_i12                 .... ...... imm:s12 rj:5 fd:5    &fr_i
 
 #
 # Fixed point arithmetic operation instruction
@@ -385,3 +389,23 @@ movfr2cf        0000 00010001 01001 10100 ..... 00 ...   @cf
 movcf2fr        0000 00010001 01001 10101 00 ... .....   @fc
 movgr2cf        0000 00010001 01001 10110 ..... 00 ...   @cr
 movcf2gr        0000 00010001 01001 10111 00 ... .....   @rc
+
+#
+# Floating point load/store instruction
+#
+fld_s           0010 101100 ............ ..... .....     @fr_i12
+fst_s           0010 101101 ............ ..... .....     @fr_i12
+fld_d           0010 101110 ............ ..... .....     @fr_i12
+fst_d           0010 101111 ............ ..... .....     @fr_i12
+fldx_s          0011 10000011 00000 ..... ..... .....    @frr
+fldx_d          0011 10000011 01000 ..... ..... .....    @frr
+fstx_s          0011 10000011 10000 ..... ..... .....    @frr
+fstx_d          0011 10000011 11000 ..... ..... .....    @frr
+fldgt_s         0011 10000111 01000 ..... ..... .....    @frr
+fldgt_d         0011 10000111 01001 ..... ..... .....    @frr
+fldle_s         0011 10000111 01010 ..... ..... .....    @frr
+fldle_d         0011 10000111 01011 ..... ..... .....    @frr
+fstgt_s         0011 10000111 01100 ..... ..... .....    @frr
+fstgt_d         0011 10000111 01101 ..... ..... .....    @frr
+fstle_s         0011 10000111 01110 ..... ..... .....    @frr
+fstle_d         0011 10000111 01111 ..... ..... .....    @frr
diff --git a/target/loongarch/translate.c b/target/loongarch/translate.c
index 26d60b50fd..daa77ade33 100644
--- a/target/loongarch/translate.c
+++ b/target/loongarch/translate.c
@@ -170,6 +170,7 @@ static void gen_set_gpr(int reg_num, TCGv t, DisasExtend dst_ext)
 #include "insn_trans/trans_fcmp.c.inc"
 #include "insn_trans/trans_fcnv.c.inc"
 #include "insn_trans/trans_fmov.c.inc"
+#include "insn_trans/trans_fmemory.c.inc"
 
 static void loongarch_tr_translate_insn(DisasContextBase *dcbase, CPUState *cs)
 {

From ee86bd58b85ce9afe727b6664ba597acc371bdf2 Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:05 +0800
Subject: [PATCH 822/935] target/loongarch: Add branch instruction translation

This includes:
- BEQ, BNE, BLT[U], BGE[U]
- BEQZ, BNEZ
- B
- BL
- JIRL
- BCEQZ, BCNEZ

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-16-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 .../loongarch/insn_trans/trans_branch.c.inc   | 83 +++++++++++++++++++
 target/loongarch/insns.decode                 | 28 +++++++
 target/loongarch/translate.c                  |  1 +
 3 files changed, 112 insertions(+)
 create mode 100644 target/loongarch/insn_trans/trans_branch.c.inc

diff --git a/target/loongarch/insn_trans/trans_branch.c.inc b/target/loongarch/insn_trans/trans_branch.c.inc
new file mode 100644
index 0000000000..65dbdff41e
--- /dev/null
+++ b/target/loongarch/insn_trans/trans_branch.c.inc
@@ -0,0 +1,83 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+static bool trans_b(DisasContext *ctx, arg_b *a)
+{
+    gen_goto_tb(ctx, 0, ctx->base.pc_next + a->offs);
+    ctx->base.is_jmp = DISAS_NORETURN;
+    return true;
+}
+
+static bool trans_bl(DisasContext *ctx, arg_bl *a)
+{
+    tcg_gen_movi_tl(cpu_gpr[1], ctx->base.pc_next + 4);
+    gen_goto_tb(ctx, 0, ctx->base.pc_next + a->offs);
+    ctx->base.is_jmp = DISAS_NORETURN;
+    return true;
+}
+
+static bool trans_jirl(DisasContext *ctx, arg_jirl *a)
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+
+    tcg_gen_addi_tl(cpu_pc, src1, a->offs);
+    tcg_gen_movi_tl(dest, ctx->base.pc_next + 4);
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+    tcg_gen_lookup_and_goto_ptr();
+    ctx->base.is_jmp = DISAS_NORETURN;
+    return true;
+}
+
+static void gen_bc(DisasContext *ctx, TCGv src1, TCGv src2,
+                   target_long offs, TCGCond cond)
+{
+    TCGLabel *l = gen_new_label();
+    tcg_gen_brcond_tl(cond, src1, src2, l);
+    gen_goto_tb(ctx, 1, ctx->base.pc_next + 4);
+    gen_set_label(l);
+    gen_goto_tb(ctx, 0, ctx->base.pc_next + offs);
+    ctx->base.is_jmp = DISAS_NORETURN;
+}
+
+static bool gen_rr_bc(DisasContext *ctx, arg_rr_offs *a, TCGCond cond)
+{
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = gpr_src(ctx, a->rd, EXT_NONE);
+
+    gen_bc(ctx, src1, src2, a->offs, cond);
+    return true;
+}
+
+static bool gen_rz_bc(DisasContext *ctx, arg_r_offs *a, TCGCond cond)
+{
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv src2 = tcg_constant_tl(0);
+
+    gen_bc(ctx, src1, src2, a->offs, cond);
+    return true;
+}
+
+static bool gen_cz_bc(DisasContext *ctx, arg_c_offs *a, TCGCond cond)
+{
+    TCGv src1 = tcg_temp_new();
+    TCGv src2 = tcg_constant_tl(0);
+
+    tcg_gen_ld8u_tl(src1, cpu_env,
+                    offsetof(CPULoongArchState, cf[a->cj]));
+    gen_bc(ctx, src1, src2, a->offs, cond);
+    return true;
+}
+
+TRANS(beq, gen_rr_bc, TCG_COND_EQ)
+TRANS(bne, gen_rr_bc, TCG_COND_NE)
+TRANS(blt, gen_rr_bc, TCG_COND_LT)
+TRANS(bge, gen_rr_bc, TCG_COND_GE)
+TRANS(bltu, gen_rr_bc, TCG_COND_LTU)
+TRANS(bgeu, gen_rr_bc, TCG_COND_GEU)
+TRANS(beqz, gen_rz_bc, TCG_COND_EQ)
+TRANS(bnez, gen_rz_bc, TCG_COND_NE)
+TRANS(bceqz, gen_cz_bc, TCG_COND_EQ)
+TRANS(bcnez, gen_cz_bc, TCG_COND_NE)
diff --git a/target/loongarch/insns.decode b/target/loongarch/insns.decode
index 8f286e7233..9b293dfdf9 100644
--- a/target/loongarch/insns.decode
+++ b/target/loongarch/insns.decode
@@ -10,6 +10,9 @@
 #
 %i14s2     10:s14       !function=shl_2
 %sa2p1     15:2         !function=plus_1
+%offs21    0:s5 10:16   !function=shl_2
+%offs16    10:s16       !function=shl_2
+%offs26    0:s10 10:16  !function=shl_2
 
 #
 # Argument sets
@@ -38,6 +41,10 @@
 &rc           rd cj
 &frr          fd rj rk
 &fr_i         fd rj imm
+&r_offs       rj offs
+&c_offs       cj offs
+&offs         offs
+&rr_offs      rj rd offs
 
 #
 # Formats
@@ -74,6 +81,10 @@
 @rc            .... ........ ..... ..... .. cj:3 rd:5    &rc
 @frr               .... ........ ..... rk:5 rj:5 fd:5    &frr
 @fr_i12                 .... ...... imm:s12 rj:5 fd:5    &fr_i
+@r_offs21         .... .. ................ rj:5 .....    &r_offs      offs=%offs21
+@c_offs21      .... .. ................ .. cj:3 .....    &c_offs      offs=%offs21
+@offs26            .... .. ..........................    &offs        offs=%offs26
+@rr_offs16         .... .. ................ rj:5 rd:5    &rr_offs     offs=%offs16
 
 #
 # Fixed point arithmetic operation instruction
@@ -409,3 +420,20 @@ fstgt_s         0011 10000111 01100 ..... ..... .....    @frr
 fstgt_d         0011 10000111 01101 ..... ..... .....    @frr
 fstle_s         0011 10000111 01110 ..... ..... .....    @frr
 fstle_d         0011 10000111 01111 ..... ..... .....    @frr
+
+#
+# Branch instructions
+#
+beqz            0100 00 ................ ..... .....     @r_offs21
+bnez            0100 01 ................ ..... .....     @r_offs21
+bceqz           0100 10 ................ 00 ... .....    @c_offs21
+bcnez           0100 10 ................ 01 ... .....    @c_offs21
+jirl            0100 11 ................ ..... .....     @rr_offs16
+b               0101 00 ..........................       @offs26
+bl              0101 01 ..........................       @offs26
+beq             0101 10 ................ ..... .....     @rr_offs16
+bne             0101 11 ................ ..... .....     @rr_offs16
+blt             0110 00 ................ ..... .....     @rr_offs16
+bge             0110 01 ................ ..... .....     @rr_offs16
+bltu            0110 10 ................ ..... .....     @rr_offs16
+bgeu            0110 11 ................ ..... .....     @rr_offs16
diff --git a/target/loongarch/translate.c b/target/loongarch/translate.c
index daa77ade33..0ad6b7b2ee 100644
--- a/target/loongarch/translate.c
+++ b/target/loongarch/translate.c
@@ -171,6 +171,7 @@ static void gen_set_gpr(int reg_num, TCGv t, DisasExtend dst_ext)
 #include "insn_trans/trans_fcnv.c.inc"
 #include "insn_trans/trans_fmov.c.inc"
 #include "insn_trans/trans_fmemory.c.inc"
+#include "insn_trans/trans_branch.c.inc"
 
 static void loongarch_tr_translate_insn(DisasContextBase *dcbase, CPUState *cs)
 {

From aae1746c726d6cc54e555bcca21e1eb5b56c21bf Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:06 +0800
Subject: [PATCH 823/935] target/loongarch: Add disassembler

This patch adds support for disassembling via option '-d in_asm'.

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-17-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/disas/dis-asm.h  |   2 +
 meson.build              |   1 +
 target/loongarch/disas.c | 610 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 613 insertions(+)
 create mode 100644 target/loongarch/disas.c

diff --git a/include/disas/dis-asm.h b/include/disas/dis-asm.h
index fc7cb7af5a..64247ecb11 100644
--- a/include/disas/dis-asm.h
+++ b/include/disas/dis-asm.h
@@ -253,6 +253,7 @@ enum bfd_architecture
 #define bfd_mach_rx            0x75
 #define bfd_mach_rx_v2         0x76
 #define bfd_mach_rx_v3         0x77
+  bfd_arch_loongarch,
   bfd_arch_last
   };
 #define bfd_mach_s390_31 31
@@ -458,6 +459,7 @@ int print_insn_riscv64          (bfd_vma, disassemble_info*);
 int print_insn_riscv128         (bfd_vma, disassemble_info*);
 int print_insn_rx(bfd_vma, disassemble_info *);
 int print_insn_hexagon(bfd_vma, disassemble_info *);
+int print_insn_loongarch(bfd_vma, disassemble_info *);
 
 #ifdef CONFIG_CAPSTONE
 bool cap_disas_target(disassemble_info *info, uint64_t pc, size_t size);
diff --git a/meson.build b/meson.build
index d738391810..21cd949082 100644
--- a/meson.build
+++ b/meson.build
@@ -2349,6 +2349,7 @@ disassemblers = {
   'sh4' : ['CONFIG_SH4_DIS'],
   'sparc' : ['CONFIG_SPARC_DIS'],
   'xtensa' : ['CONFIG_XTENSA_DIS'],
+  'loongarch' : ['CONFIG_LOONGARCH_DIS'],
 }
 if link_language == 'cpp'
   disassemblers += {
diff --git a/target/loongarch/disas.c b/target/loongarch/disas.c
new file mode 100644
index 0000000000..9454ebb8e9
--- /dev/null
+++ b/target/loongarch/disas.c
@@ -0,0 +1,610 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * QEMU LoongArch Disassembler
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited.
+ */
+
+#include "qemu/osdep.h"
+#include "disas/dis-asm.h"
+#include "qemu/bitops.h"
+
+typedef struct {
+    disassemble_info *info;
+    uint64_t pc;
+    uint32_t insn;
+} DisasContext;
+
+static inline int plus_1(DisasContext *ctx, int x)
+{
+    return x + 1;
+}
+
+static inline int shl_2(DisasContext *ctx, int x)
+{
+    return x << 2;
+}
+
+#define output(C, INSN, FMT, ...)                                   \
+{                                                                   \
+    (C)->info->fprintf_func((C)->info->stream, "%08x   %-9s\t" FMT, \
+                            (C)->insn, INSN, ##__VA_ARGS__);        \
+}
+
+#include "decode-insns.c.inc"
+
+int print_insn_loongarch(bfd_vma memaddr, struct disassemble_info *info)
+{
+    bfd_byte buffer[4];
+    uint32_t insn;
+    int status;
+
+    status = (*info->read_memory_func)(memaddr, buffer, 4, info);
+    if (status != 0) {
+        (*info->memory_error_func)(status, memaddr, info);
+        return -1;
+    }
+    insn = bfd_getl32(buffer);
+    DisasContext ctx = {
+        .info = info,
+        .pc = memaddr,
+        .insn = insn
+    };
+
+    if (!decode(&ctx, insn)) {
+        output(&ctx, "illegal", "");
+    }
+    return 4;
+}
+
+static void output_r_i(DisasContext *ctx, arg_r_i *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "r%d, %d", a->rd, a->imm);
+}
+
+static void output_rrr(DisasContext *ctx, arg_rrr *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "r%d, r%d, r%d", a->rd, a->rj, a->rk);
+}
+
+static void output_rr_i(DisasContext *ctx, arg_rr_i *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "r%d, r%d, %d", a->rd, a->rj, a->imm);
+}
+
+static void output_rrr_sa(DisasContext *ctx, arg_rrr_sa *a,
+                          const char *mnemonic)
+{
+    output(ctx, mnemonic, "r%d, r%d, r%d, %d", a->rd, a->rj, a->rk, a->sa);
+}
+
+static void output_rr(DisasContext *ctx, arg_rr *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "r%d, r%d", a->rd, a->rj);
+}
+
+static void output_rr_ms_ls(DisasContext *ctx, arg_rr_ms_ls *a,
+                          const char *mnemonic)
+{
+    output(ctx, mnemonic, "r%d, r%d, %d, %d", a->rd, a->rj, a->ms, a->ls);
+}
+
+static void output_hint_r_i(DisasContext *ctx, arg_hint_r_i *a,
+                            const char *mnemonic)
+{
+    output(ctx, mnemonic, "%d, r%d, %d", a->hint, a->rj, a->imm);
+}
+
+static void output_i(DisasContext *ctx, arg_i *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "%d", a->imm);
+}
+
+static void output_rr_jk(DisasContext *ctx, arg_rr_jk *a,
+                         const char *mnemonic)
+{
+    output(ctx, mnemonic, "r%d, r%d", a->rj, a->rk);
+}
+
+static void output_ff(DisasContext *ctx, arg_ff *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "f%d, f%d", a->fd, a->fj);
+}
+
+static void output_fff(DisasContext *ctx, arg_fff *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "f%d, f%d, f%d", a->fd, a->fj, a->fk);
+}
+
+static void output_ffff(DisasContext *ctx, arg_ffff *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "f%d, f%d, f%d, f%d", a->fd, a->fj, a->fk, a->fa);
+}
+
+static void output_fffc(DisasContext *ctx, arg_fffc *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "f%d, f%d, f%d, %d", a->fd, a->fj, a->fk, a->ca);
+}
+
+static void output_fr(DisasContext *ctx, arg_fr *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "f%d, r%d", a->fd, a->rj);
+}
+
+static void output_rf(DisasContext *ctx, arg_rf *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "r%d, f%d", a->rd, a->fj);
+}
+
+static void output_fcsrd_r(DisasContext *ctx, arg_fcsrd_r *a,
+                           const char *mnemonic)
+{
+    output(ctx, mnemonic, "fcsr%d, r%d", a->fcsrd, a->rj);
+}
+
+static void output_r_fcsrs(DisasContext *ctx, arg_r_fcsrs *a,
+                           const char *mnemonic)
+{
+    output(ctx, mnemonic, "r%d, fcsr%d", a->rd, a->fcsrs);
+}
+
+static void output_cf(DisasContext *ctx, arg_cf *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "fcc%d, f%d", a->cd, a->fj);
+}
+
+static void output_fc(DisasContext *ctx, arg_fc *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "f%d, fcc%d", a->fd, a->cj);
+}
+
+static void output_cr(DisasContext *ctx, arg_cr *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "fcc%d, r%d", a->cd, a->rj);
+}
+
+static void output_rc(DisasContext *ctx, arg_rc *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "r%d, fcc%d", a->rd, a->cj);
+}
+
+static void output_frr(DisasContext *ctx, arg_frr *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "f%d, r%d, r%d", a->fd, a->rj, a->rk);
+}
+
+static void output_fr_i(DisasContext *ctx, arg_fr_i *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "f%d, r%d, %d", a->fd, a->rj, a->imm);
+}
+
+static void output_r_offs(DisasContext *ctx, arg_r_offs *a,
+                          const char *mnemonic)
+{
+    output(ctx, mnemonic, "r%d, %d # 0x%" PRIx64, a->rj, a->offs,
+           ctx->pc + a->offs);
+}
+
+static void output_c_offs(DisasContext *ctx, arg_c_offs *a,
+                          const char *mnemonic)
+{
+    output(ctx, mnemonic, "fcc%d, %d # 0x%" PRIx64, a->cj, a->offs,
+           ctx->pc + a->offs);
+}
+
+static void output_offs(DisasContext *ctx, arg_offs *a,
+                        const char *mnemonic)
+{
+    output(ctx, mnemonic, "%d # 0x%" PRIx64, a->offs, ctx->pc + a->offs);
+}
+
+static void output_rr_offs(DisasContext *ctx, arg_rr_offs *a,
+                           const char *mnemonic)
+{
+    output(ctx, mnemonic, "r%d, r%d, %d # 0x%" PRIx64, a->rj,
+           a->rd, a->offs, ctx->pc + a->offs);
+}
+
+#define INSN(insn, type)                                    \
+static bool trans_##insn(DisasContext *ctx, arg_##type * a) \
+{                                                           \
+    output_##type(ctx, a, #insn);                           \
+    return true;                                            \
+}
+
+INSN(clo_w,        rr)
+INSN(clz_w,        rr)
+INSN(cto_w,        rr)
+INSN(ctz_w,        rr)
+INSN(clo_d,        rr)
+INSN(clz_d,        rr)
+INSN(cto_d,        rr)
+INSN(ctz_d,        rr)
+INSN(revb_2h,      rr)
+INSN(revb_4h,      rr)
+INSN(revb_2w,      rr)
+INSN(revb_d,       rr)
+INSN(revh_2w,      rr)
+INSN(revh_d,       rr)
+INSN(bitrev_4b,    rr)
+INSN(bitrev_8b,    rr)
+INSN(bitrev_w,     rr)
+INSN(bitrev_d,     rr)
+INSN(ext_w_h,      rr)
+INSN(ext_w_b,      rr)
+INSN(cpucfg,       rr)
+INSN(asrtle_d,     rr_jk)
+INSN(asrtgt_d,     rr_jk)
+INSN(alsl_w,       rrr_sa)
+INSN(alsl_wu,      rrr_sa)
+INSN(bytepick_w,   rrr_sa)
+INSN(bytepick_d,   rrr_sa)
+INSN(add_w,        rrr)
+INSN(add_d,        rrr)
+INSN(sub_w,        rrr)
+INSN(sub_d,        rrr)
+INSN(slt,          rrr)
+INSN(sltu,         rrr)
+INSN(maskeqz,      rrr)
+INSN(masknez,      rrr)
+INSN(nor,          rrr)
+INSN(and,          rrr)
+INSN(or,           rrr)
+INSN(xor,          rrr)
+INSN(orn,          rrr)
+INSN(andn,         rrr)
+INSN(sll_w,        rrr)
+INSN(srl_w,        rrr)
+INSN(sra_w,        rrr)
+INSN(sll_d,        rrr)
+INSN(srl_d,        rrr)
+INSN(sra_d,        rrr)
+INSN(rotr_w,       rrr)
+INSN(rotr_d,       rrr)
+INSN(mul_w,        rrr)
+INSN(mulh_w,       rrr)
+INSN(mulh_wu,      rrr)
+INSN(mul_d,        rrr)
+INSN(mulh_d,       rrr)
+INSN(mulh_du,      rrr)
+INSN(mulw_d_w,     rrr)
+INSN(mulw_d_wu,    rrr)
+INSN(div_w,        rrr)
+INSN(mod_w,        rrr)
+INSN(div_wu,       rrr)
+INSN(mod_wu,       rrr)
+INSN(div_d,        rrr)
+INSN(mod_d,        rrr)
+INSN(div_du,       rrr)
+INSN(mod_du,       rrr)
+INSN(crc_w_b_w,    rrr)
+INSN(crc_w_h_w,    rrr)
+INSN(crc_w_w_w,    rrr)
+INSN(crc_w_d_w,    rrr)
+INSN(crcc_w_b_w,   rrr)
+INSN(crcc_w_h_w,   rrr)
+INSN(crcc_w_w_w,   rrr)
+INSN(crcc_w_d_w,   rrr)
+INSN(break,        i)
+INSN(syscall,      i)
+INSN(alsl_d,       rrr_sa)
+INSN(slli_w,       rr_i)
+INSN(slli_d,       rr_i)
+INSN(srli_w,       rr_i)
+INSN(srli_d,       rr_i)
+INSN(srai_w,       rr_i)
+INSN(srai_d,       rr_i)
+INSN(rotri_w,      rr_i)
+INSN(rotri_d,      rr_i)
+INSN(bstrins_w,    rr_ms_ls)
+INSN(bstrpick_w,   rr_ms_ls)
+INSN(bstrins_d,    rr_ms_ls)
+INSN(bstrpick_d,   rr_ms_ls)
+INSN(fadd_s,       fff)
+INSN(fadd_d,       fff)
+INSN(fsub_s,       fff)
+INSN(fsub_d,       fff)
+INSN(fmul_s,       fff)
+INSN(fmul_d,       fff)
+INSN(fdiv_s,       fff)
+INSN(fdiv_d,       fff)
+INSN(fmax_s,       fff)
+INSN(fmax_d,       fff)
+INSN(fmin_s,       fff)
+INSN(fmin_d,       fff)
+INSN(fmaxa_s,      fff)
+INSN(fmaxa_d,      fff)
+INSN(fmina_s,      fff)
+INSN(fmina_d,      fff)
+INSN(fscaleb_s,    fff)
+INSN(fscaleb_d,    fff)
+INSN(fcopysign_s,  fff)
+INSN(fcopysign_d,  fff)
+INSN(fabs_s,       ff)
+INSN(fabs_d,       ff)
+INSN(fneg_s,       ff)
+INSN(fneg_d,       ff)
+INSN(flogb_s,      ff)
+INSN(flogb_d,      ff)
+INSN(fclass_s,     ff)
+INSN(fclass_d,     ff)
+INSN(fsqrt_s,      ff)
+INSN(fsqrt_d,      ff)
+INSN(frecip_s,     ff)
+INSN(frecip_d,     ff)
+INSN(frsqrt_s,     ff)
+INSN(frsqrt_d,     ff)
+INSN(fmov_s,       ff)
+INSN(fmov_d,       ff)
+INSN(movgr2fr_w,   fr)
+INSN(movgr2fr_d,   fr)
+INSN(movgr2frh_w,  fr)
+INSN(movfr2gr_s,   rf)
+INSN(movfr2gr_d,   rf)
+INSN(movfrh2gr_s,  rf)
+INSN(movgr2fcsr,   fcsrd_r)
+INSN(movfcsr2gr,   r_fcsrs)
+INSN(movfr2cf,     cf)
+INSN(movcf2fr,     fc)
+INSN(movgr2cf,     cr)
+INSN(movcf2gr,     rc)
+INSN(fcvt_s_d,     ff)
+INSN(fcvt_d_s,     ff)
+INSN(ftintrm_w_s,  ff)
+INSN(ftintrm_w_d,  ff)
+INSN(ftintrm_l_s,  ff)
+INSN(ftintrm_l_d,  ff)
+INSN(ftintrp_w_s,  ff)
+INSN(ftintrp_w_d,  ff)
+INSN(ftintrp_l_s,  ff)
+INSN(ftintrp_l_d,  ff)
+INSN(ftintrz_w_s,  ff)
+INSN(ftintrz_w_d,  ff)
+INSN(ftintrz_l_s,  ff)
+INSN(ftintrz_l_d,  ff)
+INSN(ftintrne_w_s, ff)
+INSN(ftintrne_w_d, ff)
+INSN(ftintrne_l_s, ff)
+INSN(ftintrne_l_d, ff)
+INSN(ftint_w_s,    ff)
+INSN(ftint_w_d,    ff)
+INSN(ftint_l_s,    ff)
+INSN(ftint_l_d,    ff)
+INSN(ffint_s_w,    ff)
+INSN(ffint_s_l,    ff)
+INSN(ffint_d_w,    ff)
+INSN(ffint_d_l,    ff)
+INSN(frint_s,      ff)
+INSN(frint_d,      ff)
+INSN(slti,         rr_i)
+INSN(sltui,        rr_i)
+INSN(addi_w,       rr_i)
+INSN(addi_d,       rr_i)
+INSN(lu52i_d,      rr_i)
+INSN(andi,         rr_i)
+INSN(ori,          rr_i)
+INSN(xori,         rr_i)
+INSN(fmadd_s,      ffff)
+INSN(fmadd_d,      ffff)
+INSN(fmsub_s,      ffff)
+INSN(fmsub_d,      ffff)
+INSN(fnmadd_s,     ffff)
+INSN(fnmadd_d,     ffff)
+INSN(fnmsub_s,     ffff)
+INSN(fnmsub_d,     ffff)
+INSN(fsel,         fffc)
+INSN(addu16i_d,    rr_i)
+INSN(lu12i_w,      r_i)
+INSN(lu32i_d,      r_i)
+INSN(pcaddi,       r_i)
+INSN(pcalau12i,    r_i)
+INSN(pcaddu12i,    r_i)
+INSN(pcaddu18i,    r_i)
+INSN(ll_w,         rr_i)
+INSN(sc_w,         rr_i)
+INSN(ll_d,         rr_i)
+INSN(sc_d,         rr_i)
+INSN(ldptr_w,      rr_i)
+INSN(stptr_w,      rr_i)
+INSN(ldptr_d,      rr_i)
+INSN(stptr_d,      rr_i)
+INSN(ld_b,         rr_i)
+INSN(ld_h,         rr_i)
+INSN(ld_w,         rr_i)
+INSN(ld_d,         rr_i)
+INSN(st_b,         rr_i)
+INSN(st_h,         rr_i)
+INSN(st_w,         rr_i)
+INSN(st_d,         rr_i)
+INSN(ld_bu,        rr_i)
+INSN(ld_hu,        rr_i)
+INSN(ld_wu,        rr_i)
+INSN(preld,        hint_r_i)
+INSN(fld_s,        fr_i)
+INSN(fst_s,        fr_i)
+INSN(fld_d,        fr_i)
+INSN(fst_d,        fr_i)
+INSN(ldx_b,        rrr)
+INSN(ldx_h,        rrr)
+INSN(ldx_w,        rrr)
+INSN(ldx_d,        rrr)
+INSN(stx_b,        rrr)
+INSN(stx_h,        rrr)
+INSN(stx_w,        rrr)
+INSN(stx_d,        rrr)
+INSN(ldx_bu,       rrr)
+INSN(ldx_hu,       rrr)
+INSN(ldx_wu,       rrr)
+INSN(fldx_s,       frr)
+INSN(fldx_d,       frr)
+INSN(fstx_s,       frr)
+INSN(fstx_d,       frr)
+INSN(amswap_w,     rrr)
+INSN(amswap_d,     rrr)
+INSN(amadd_w,      rrr)
+INSN(amadd_d,      rrr)
+INSN(amand_w,      rrr)
+INSN(amand_d,      rrr)
+INSN(amor_w,       rrr)
+INSN(amor_d,       rrr)
+INSN(amxor_w,      rrr)
+INSN(amxor_d,      rrr)
+INSN(ammax_w,      rrr)
+INSN(ammax_d,      rrr)
+INSN(ammin_w,      rrr)
+INSN(ammin_d,      rrr)
+INSN(ammax_wu,     rrr)
+INSN(ammax_du,     rrr)
+INSN(ammin_wu,     rrr)
+INSN(ammin_du,     rrr)
+INSN(amswap_db_w,  rrr)
+INSN(amswap_db_d,  rrr)
+INSN(amadd_db_w,   rrr)
+INSN(amadd_db_d,   rrr)
+INSN(amand_db_w,   rrr)
+INSN(amand_db_d,   rrr)
+INSN(amor_db_w,    rrr)
+INSN(amor_db_d,    rrr)
+INSN(amxor_db_w,   rrr)
+INSN(amxor_db_d,   rrr)
+INSN(ammax_db_w,   rrr)
+INSN(ammax_db_d,   rrr)
+INSN(ammin_db_w,   rrr)
+INSN(ammin_db_d,   rrr)
+INSN(ammax_db_wu,  rrr)
+INSN(ammax_db_du,  rrr)
+INSN(ammin_db_wu,  rrr)
+INSN(ammin_db_du,  rrr)
+INSN(dbar,         i)
+INSN(ibar,         i)
+INSN(fldgt_s,      frr)
+INSN(fldgt_d,      frr)
+INSN(fldle_s,      frr)
+INSN(fldle_d,      frr)
+INSN(fstgt_s,      frr)
+INSN(fstgt_d,      frr)
+INSN(fstle_s,      frr)
+INSN(fstle_d,      frr)
+INSN(ldgt_b,       rrr)
+INSN(ldgt_h,       rrr)
+INSN(ldgt_w,       rrr)
+INSN(ldgt_d,       rrr)
+INSN(ldle_b,       rrr)
+INSN(ldle_h,       rrr)
+INSN(ldle_w,       rrr)
+INSN(ldle_d,       rrr)
+INSN(stgt_b,       rrr)
+INSN(stgt_h,       rrr)
+INSN(stgt_w,       rrr)
+INSN(stgt_d,       rrr)
+INSN(stle_b,       rrr)
+INSN(stle_h,       rrr)
+INSN(stle_w,       rrr)
+INSN(stle_d,       rrr)
+INSN(beqz,         r_offs)
+INSN(bnez,         r_offs)
+INSN(bceqz,        c_offs)
+INSN(bcnez,        c_offs)
+INSN(jirl,         rr_offs)
+INSN(b,            offs)
+INSN(bl,           offs)
+INSN(beq,          rr_offs)
+INSN(bne,          rr_offs)
+INSN(blt,          rr_offs)
+INSN(bge,          rr_offs)
+INSN(bltu,         rr_offs)
+INSN(bgeu,         rr_offs)
+
+#define output_fcmp(C, PREFIX, SUFFIX)                                         \
+{                                                                              \
+    (C)->info->fprintf_func((C)->info->stream, "%08x   %s%s\tfcc%d, f%d, f%d", \
+                            (C)->insn, PREFIX, SUFFIX, a->cd,                  \
+                            a->fj, a->fk);                                     \
+}
+
+static bool output_cff_fcond(DisasContext *ctx, arg_cff_fcond * a,
+                               const char *suffix)
+{
+    bool ret = true;
+    switch (a->fcond) {
+    case 0x0:
+        output_fcmp(ctx, "fcmp_caf_", suffix);
+        break;
+    case 0x1:
+        output_fcmp(ctx, "fcmp_saf_", suffix);
+        break;
+    case 0x2:
+        output_fcmp(ctx, "fcmp_clt_", suffix);
+        break;
+    case 0x3:
+        output_fcmp(ctx, "fcmp_slt_", suffix);
+        break;
+    case 0x4:
+        output_fcmp(ctx, "fcmp_ceq_", suffix);
+        break;
+    case 0x5:
+        output_fcmp(ctx, "fcmp_seq_", suffix);
+        break;
+    case 0x6:
+        output_fcmp(ctx, "fcmp_cle_", suffix);
+        break;
+    case 0x7:
+        output_fcmp(ctx, "fcmp_sle_", suffix);
+        break;
+    case 0x8:
+        output_fcmp(ctx, "fcmp_cun_", suffix);
+        break;
+    case 0x9:
+        output_fcmp(ctx, "fcmp_sun_", suffix);
+        break;
+    case 0xA:
+        output_fcmp(ctx, "fcmp_cult_", suffix);
+        break;
+    case 0xB:
+        output_fcmp(ctx, "fcmp_sult_", suffix);
+        break;
+    case 0xC:
+        output_fcmp(ctx, "fcmp_cueq_", suffix);
+        break;
+    case 0xD:
+        output_fcmp(ctx, "fcmp_sueq_", suffix);
+        break;
+    case 0xE:
+        output_fcmp(ctx, "fcmp_cule_", suffix);
+        break;
+    case 0xF:
+        output_fcmp(ctx, "fcmp_sule_", suffix);
+        break;
+    case 0x10:
+        output_fcmp(ctx, "fcmp_cne_", suffix);
+        break;
+    case 0x11:
+        output_fcmp(ctx, "fcmp_sne_", suffix);
+        break;
+    case 0x14:
+        output_fcmp(ctx, "fcmp_cor_", suffix);
+        break;
+    case 0x15:
+        output_fcmp(ctx, "fcmp_sor_", suffix);
+        break;
+    case 0x18:
+        output_fcmp(ctx, "fcmp_cune_", suffix);
+        break;
+    case 0x19:
+        output_fcmp(ctx, "fcmp_sune_", suffix);
+        break;
+    default:
+        ret = false;
+    }
+    return ret;
+}
+
+#define FCMP_INSN(suffix)                               \
+static bool trans_fcmp_cond_##suffix(DisasContext *ctx, \
+                                     arg_cff_fcond * a) \
+{                                                       \
+    return output_cff_fcond(ctx, a, #suffix);           \
+}
+
+FCMP_INSN(s)
+FCMP_INSN(d)

From 14f2b0b74188582c6f016f8659b24baa340d765a Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:07 +0800
Subject: [PATCH 824/935] target/loongarch: Add target build suport
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220606124333.2060567-18-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/meson.build | 19 +++++++++++++++++++
 target/meson.build           |  1 +
 2 files changed, 20 insertions(+)
 create mode 100644 target/loongarch/meson.build

diff --git a/target/loongarch/meson.build b/target/loongarch/meson.build
new file mode 100644
index 0000000000..bcb076e55f
--- /dev/null
+++ b/target/loongarch/meson.build
@@ -0,0 +1,19 @@
+gen = decodetree.process('insns.decode')
+
+loongarch_ss = ss.source_set()
+loongarch_ss.add(files(
+  'cpu.c',
+  'disas.c',
+))
+loongarch_tcg_ss = ss.source_set()
+loongarch_tcg_ss.add(gen)
+loongarch_tcg_ss.add(files(
+  'fpu_helper.c',
+  'op_helper.c',
+  'translate.c',
+))
+loongarch_tcg_ss.add(zlib)
+
+loongarch_ss.add_all(when: 'CONFIG_TCG', if_true: [loongarch_tcg_ss])
+
+target_arch += {'loongarch': loongarch_ss}
diff --git a/target/meson.build b/target/meson.build
index 2f6940255e..a53a60486f 100644
--- a/target/meson.build
+++ b/target/meson.build
@@ -5,6 +5,7 @@ subdir('cris')
 subdir('hexagon')
 subdir('hppa')
 subdir('i386')
+subdir('loongarch')
 subdir('m68k')
 subdir('microblaze')
 subdir('mips')

From d88b51dc26c0e29d012f52d0215ebef29329ca87 Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:08 +0800
Subject: [PATCH 825/935] target/loongarch: Add system emulation introduction

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-19-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 MAINTAINERS                         |  8 +++++
 docs/system/loongarch/loongson3.rst | 41 ++++++++++++++++++++++
 target/loongarch/README             | 54 +++++++++++++++++++++++++++++
 3 files changed, 103 insertions(+)
 create mode 100644 docs/system/loongarch/loongson3.rst

diff --git a/MAINTAINERS b/MAINTAINERS
index 319a5d805a..e3f875071a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1122,6 +1122,14 @@ F: include/hw/net/lasi_82596.h
 F: include/hw/pci-host/dino.h
 F: pc-bios/hppa-firmware.img
 
+LoongArch Machines
+------------------
+Virt
+M: Xiaojuan Yang <yangxiaojuan@loongson.cn>
+M: Song Gao <gaosong@loongson.cn>
+S: Maintained
+F: docs/system/loongarch/loongson3.rst
+
 M68K Machines
 -------------
 an5206
diff --git a/docs/system/loongarch/loongson3.rst b/docs/system/loongarch/loongson3.rst
new file mode 100644
index 0000000000..fa3acd01c0
--- /dev/null
+++ b/docs/system/loongarch/loongson3.rst
@@ -0,0 +1,41 @@
+:orphan:
+
+==========================================
+loongson3 virt generic platform (``virt``)
+==========================================
+
+The ``virt`` machine use gpex host bridge, and there are some
+emulated devices on virt board, such as loongson7a RTC device,
+IOAPIC device, ACPI device and so on.
+
+Supported devices
+-----------------
+
+The ``virt`` machine supports:
+- Gpex host bridge
+- Ls7a RTC device
+- Ls7a IOAPIC device
+- Ls7a ACPI device
+- Fw_cfg device
+- PCI/PCIe devices
+- Memory device
+- CPU device. Type: Loongson-3A5000.
+
+CPU and machine Type
+--------------------
+
+The ``qemu-system-loongarch64`` provides emulation for virt
+machine. You can specify the machine type ``virt`` and
+cpu type ``Loongson-3A5000``.
+
+Boot options
+------------
+
+Now the ``virt`` machine can run test program in ELF format and the
+method of compiling is in target/loongarch/README.
+
+.. code-block:: bash
+
+  $ qemu-system-loongarch64 -machine virt -m 4G -cpu Loongson-3A5000 \
+      -smp 1 -kernel hello -monitor none -display none \
+      -chardev file,path=hello.out,id=output -serial chardev:output
diff --git a/target/loongarch/README b/target/loongarch/README
index de141c1a58..4dcd0f1682 100644
--- a/target/loongarch/README
+++ b/target/loongarch/README
@@ -8,3 +8,57 @@
 
   We can get the latest loongarch documents at https://github.com/loongson/LoongArch-Documentation/tags.
 
+
+- System emulation
+
+  Mainly emulate a virt 3A5000 board and ls7a bridge that is not exactly the same as the host.
+  3A5000 support multiple interrupt cascading while here we just emulate the extioi interrupt
+  cascading. LS7A1000 host bridge support multiple devices, such as sata, gmac, uart, rtc
+  and so on. But we just realize the rtc. Others use the qemu common devices. It does not affect
+  the general use. We also introduced the emulation of devices at docs/system/loongarch/loongson3.rst.
+
+  This version only supports running binary files in ELF format, and does not depend on BIOS and kernel file.
+  You can compile the test program with 'make & make check-tcg' and run the test case with the following command:
+
+  1. Install LoongArch cross-tools on X86 machines.
+
+    Download cross-tools.
+
+      wget https://github.com/loongson/build-tools/releases/latest/download/loongarch64-clfs-20211202-cross-tools.tar.xz
+
+      tar -vxf loongarch64-clfs-20211202-cross-tools.tar.xz -C /opt
+
+    Config cross-tools env.
+
+      . setenv.sh
+
+      setenv.sh:
+
+          #!/bin/sh
+          set -x
+          CC_PREFIX=/opt/cross-tools
+
+          export PATH=$CC_PREFIX/bin:$PATH
+          export LD_LIBRARY_PATH=$CC_PREFIX/lib:$LD_LIBRARY_PATH
+          export LD_LIBRARY_PATH=$CC_PREFIX/loongarch64-unknown-linux-gnu/lib/:$LD_LIBRARY_PATH
+          set +x
+
+  2. Test tests/tcg/multiarch.
+
+    ./configure --disable-rdma --disable-pvrdma --prefix=/usr  \
+            --target-list="loongarch64-softmmu"  \
+            --disable-libiscsi --disable-libnfs --disable-libpmem \
+            --disable-glusterfs --enable-libusb --enable-usb-redir \
+            --disable-opengl --disable-xen --enable-spice --disable-werror \
+            --enable-debug --disable-capstone --disable-kvm --enable-profiler
+
+    cd  build/
+
+    make && make check-tcg
+
+    or
+
+    ./build/qemu-system-loongarch64 -machine virt -m 4G -cpu Loongson-3A5000 -smp 1 -kernel build/tests/tcg/loongarch64-softmmu/hello -monitor none -display none -chardev file,path=hello.out,id=output -serial chardev:output
+
+- Note.
+  We can get the latest LoongArch documents or LoongArch tools at https://github.com/loongson/

From 398cecb9c3e756afcbf5c5f2bfa9796be0324e2c Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:09 +0800
Subject: [PATCH 826/935] target/loongarch: Add CSRs definition

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-20-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/cpu-csr.h | 208 +++++++++++++++++++++++++++++++++++++
 target/loongarch/cpu.c     |  41 ++++++++
 target/loongarch/cpu.h     |  64 ++++++++++++
 3 files changed, 313 insertions(+)
 create mode 100644 target/loongarch/cpu-csr.h

diff --git a/target/loongarch/cpu-csr.h b/target/loongarch/cpu-csr.h
new file mode 100644
index 0000000000..4c8ce7fed5
--- /dev/null
+++ b/target/loongarch/cpu-csr.h
@@ -0,0 +1,208 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * QEMU LoongArch CSRs
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+#ifndef LOONGARCH_CPU_CSR_H
+#define LOONGARCH_CPU_CSR_H
+
+#include "hw/registerfields.h"
+
+/* Base on kernal definitions: arch/loongarch/include/asm/loongarch.h */
+
+/* Basic CSRs */
+#define LOONGARCH_CSR_CRMD           0x0 /* Current mode info */
+
+#define LOONGARCH_CSR_PRMD           0x1 /* Prev-exception mode info */
+FIELD(CSR_PRMD, PPLV, 0, 2)
+FIELD(CSR_PRMD, PIE, 2, 1)
+FIELD(CSR_PRMD, PWE, 3, 1)
+
+#define LOONGARCH_CSR_EUEN           0x2 /* Extended unit enable */
+FIELD(CSR_EUEN, FPE, 0, 1)
+FIELD(CSR_EUEN, SXE, 1, 1)
+FIELD(CSR_EUEN, ASXE, 2, 1)
+FIELD(CSR_EUEN, BTE, 3, 1)
+
+#define LOONGARCH_CSR_MISC           0x3 /* Misc config */
+FIELD(CSR_MISC, VA32, 0, 4)
+FIELD(CSR_MISC, DRDTL, 4, 4)
+FIELD(CSR_MISC, RPCNTL, 8, 4)
+FIELD(CSR_MISC, ALCL, 12, 4)
+FIELD(CSR_MISC, DWPL, 16, 3)
+
+#define LOONGARCH_CSR_ECFG           0x4 /* Exception config */
+FIELD(CSR_ECFG, LIE, 0, 13)
+FIELD(CSR_ECFG, VS, 16, 3)
+
+#define LOONGARCH_CSR_ESTAT          0x5 /* Exception status */
+FIELD(CSR_ESTAT, IS, 0, 13)
+FIELD(CSR_ESTAT, ECODE, 16, 6)
+FIELD(CSR_ESTAT, ESUBCODE, 22, 9)
+
+#define LOONGARCH_CSR_ERA            0x6 /* Exception return address */
+
+#define LOONGARCH_CSR_BADV           0x7 /* Bad virtual address */
+
+#define LOONGARCH_CSR_BADI           0x8 /* Bad instruction */
+
+#define LOONGARCH_CSR_EENTRY         0xc /* Exception entry address */
+
+/* TLB related CSRs */
+#define LOONGARCH_CSR_TLBIDX         0x10 /* TLB Index, EHINV, PageSize, NP */
+FIELD(CSR_TLBIDX, INDEX, 0, 12)
+FIELD(CSR_TLBIDX, PS, 24, 6)
+FIELD(CSR_TLBIDX, NE, 31, 1)
+
+#define LOONGARCH_CSR_TLBEHI         0x11 /* TLB EntryHi */
+FIELD(CSR_TLBEHI, VPPN, 13, 35)
+
+#define LOONGARCH_CSR_TLBELO0        0x12 /* TLB EntryLo0 */
+#define LOONGARCH_CSR_TLBELO1        0x13 /* TLB EntryLo1 */
+FIELD(TLBENTRY, V, 0, 1)
+FIELD(TLBENTRY, D, 1, 1)
+FIELD(TLBENTRY, PLV, 2, 2)
+FIELD(TLBENTRY, MAT, 4, 2)
+FIELD(TLBENTRY, G, 6, 1)
+FIELD(TLBENTRY, PPN, 12, 36)
+FIELD(TLBENTRY, NR, 61, 1)
+FIELD(TLBENTRY, NX, 62, 1)
+FIELD(TLBENTRY, RPLV, 63, 1)
+
+#define LOONGARCH_CSR_ASID           0x18 /* Address space identifier */
+FIELD(CSR_ASID, ASID, 0, 10)
+FIELD(CSR_ASID, ASIDBITS, 16, 8)
+
+/* Page table base address when badv[47] = 0 */
+#define LOONGARCH_CSR_PGDL           0x19
+/* Page table base address when badv[47] = 1 */
+#define LOONGARCH_CSR_PGDH           0x1a
+
+#define LOONGARCH_CSR_PGD            0x1b /* Page table base address */
+
+/* Page walk controller's low addr */
+#define LOONGARCH_CSR_PWCL           0x1c
+FIELD(CSR_PWCL, PTBASE, 0, 5)
+FIELD(CSR_PWCL, PTWIDTH, 5, 5)
+FIELD(CSR_PWCL, DIR1_BASE, 10, 5)
+FIELD(CSR_PWCL, DIR1_WIDTH, 15, 5)
+FIELD(CSR_PWCL, DIR2_BASE, 20, 5)
+FIELD(CSR_PWCL, DIR2_WIDTH, 25, 5)
+FIELD(CSR_PWCL, PTEWIDTH, 30, 2)
+
+/* Page walk controller's high addr */
+#define LOONGARCH_CSR_PWCH           0x1d
+FIELD(CSR_PWCH, DIR3_BASE, 0, 6)
+FIELD(CSR_PWCH, DIR3_WIDTH, 6, 6)
+FIELD(CSR_PWCH, DIR4_BASE, 12, 6)
+FIELD(CSR_PWCH, DIR4_WIDTH, 18, 6)
+
+#define LOONGARCH_CSR_STLBPS         0x1e /* Stlb page size */
+FIELD(CSR_STLBPS, PS, 0, 5)
+
+#define LOONGARCH_CSR_RVACFG         0x1f /* Reduced virtual address config */
+FIELD(CSR_RVACFG, RBITS, 0, 4)
+
+/* Config CSRs */
+#define LOONGARCH_CSR_CPUID          0x20 /* CPU core id */
+
+#define LOONGARCH_CSR_PRCFG1         0x21 /* Config1 */
+FIELD(CSR_PRCFG1, SAVE_NUM, 0, 4)
+FIELD(CSR_PRCFG1, TIMER_BITS, 4, 8)
+FIELD(CSR_PRCFG1, VSMAX, 12, 3)
+
+#define LOONGARCH_CSR_PRCFG2         0x22 /* Config2 */
+
+#define LOONGARCH_CSR_PRCFG3         0x23 /* Config3 */
+FIELD(CSR_PRCFG3, TLB_TYPE, 0, 4)
+FIELD(CSR_PRCFG3, MTLB_ENTRY, 4, 8)
+FIELD(CSR_PRCFG3, STLB_WAYS, 12, 8)
+FIELD(CSR_PRCFG3, STLB_SETS, 20, 8)
+
+/*
+ * Save registers count can read from PRCFG1.SAVE_NUM
+ * The Min count is 1. Max count is 15.
+ */
+#define LOONGARCH_CSR_SAVE(N)        (0x30 + N)
+
+/* Timer CSRs */
+#define LOONGARCH_CSR_TID            0x40 /* Timer ID */
+
+#define LOONGARCH_CSR_TCFG           0x41 /* Timer config */
+FIELD(CSR_TCFG, EN, 0, 1)
+FIELD(CSR_TCFG, PERIODIC, 1, 1)
+FIELD(CSR_TCFG, INIT_VAL, 2, 46)
+
+#define LOONGARCH_CSR_TVAL           0x42 /* Timer ticks remain */
+
+#define LOONGARCH_CSR_CNTC           0x43 /* Timer offset */
+
+#define LOONGARCH_CSR_TICLR          0x44 /* Timer interrupt clear */
+
+/* LLBCTL CSRs */
+#define LOONGARCH_CSR_LLBCTL         0x60 /* LLBit control */
+FIELD(CSR_LLBCTL, ROLLB, 0, 1)
+FIELD(CSR_LLBCTL, WCLLB, 1, 1)
+FIELD(CSR_LLBCTL, KLO, 2, 1)
+
+/* Implement dependent */
+#define LOONGARCH_CSR_IMPCTL1        0x80 /* LoongArch config1 */
+
+#define LOONGARCH_CSR_IMPCTL2        0x81 /* LoongArch config2*/
+
+/* TLB Refill CSRs */
+#define LOONGARCH_CSR_TLBRENTRY      0x88 /* TLB refill exception address */
+#define LOONGARCH_CSR_TLBRBADV       0x89 /* TLB refill badvaddr */
+#define LOONGARCH_CSR_TLBRERA        0x8a /* TLB refill ERA */
+#define LOONGARCH_CSR_TLBRSAVE       0x8b /* KScratch for TLB refill */
+FIELD(CSR_TLBRERA, ISTLBR, 0, 1)
+FIELD(CSR_TLBRERA, PC, 2, 62)
+#define LOONGARCH_CSR_TLBRELO0       0x8c /* TLB refill entrylo0 */
+#define LOONGARCH_CSR_TLBRELO1       0x8d /* TLB refill entrylo1 */
+#define LOONGARCH_CSR_TLBREHI        0x8e /* TLB refill entryhi */
+FIELD(CSR_TLBREHI, PS, 0, 6)
+FIELD(CSR_TLBREHI, VPPN, 13, 35)
+#define LOONGARCH_CSR_TLBRPRMD       0x8f /* TLB refill mode info */
+FIELD(CSR_TLBRPRMD, PPLV, 0, 2)
+FIELD(CSR_TLBRPRMD, PIE, 2, 1)
+FIELD(CSR_TLBRPRMD, PWE, 4, 1)
+
+/* Machine Error CSRs */
+#define LOONGARCH_CSR_MERRCTL        0x90 /* ERRCTL */
+FIELD(CSR_MERRCTL, ISMERR, 0, 1)
+#define LOONGARCH_CSR_MERRINFO1      0x91
+#define LOONGARCH_CSR_MERRINFO2      0x92
+#define LOONGARCH_CSR_MERRENTRY      0x93 /* MError exception base */
+#define LOONGARCH_CSR_MERRERA        0x94 /* MError exception PC */
+#define LOONGARCH_CSR_MERRSAVE       0x95 /* KScratch for error exception */
+
+#define LOONGARCH_CSR_CTAG           0x98 /* TagLo + TagHi */
+
+/* Direct map windows CSRs*/
+#define LOONGARCH_CSR_DMW(N)         (0x180 + N)
+FIELD(CSR_DMW, PLV0, 0, 1)
+FIELD(CSR_DMW, PLV1, 1, 1)
+FIELD(CSR_DMW, PLV2, 2, 1)
+FIELD(CSR_DMW, PLV3, 3, 1)
+FIELD(CSR_DMW, MAT, 4, 2)
+FIELD(CSR_DMW, VSEG, 60, 4)
+
+#define dmw_va2pa(va) \
+    (va & MAKE_64BIT_MASK(0, TARGET_VIRT_ADDR_SPACE_BITS))
+
+/* Debug CSRs */
+#define LOONGARCH_CSR_DBG            0x500 /* debug config */
+FIELD(CSR_DBG, DST, 0, 1)
+FIELD(CSR_DBG, DREV, 1, 7)
+FIELD(CSR_DBG, DEI, 8, 1)
+FIELD(CSR_DBG, DCL, 9, 1)
+FIELD(CSR_DBG, DFW, 10, 1)
+FIELD(CSR_DBG, DMW, 11, 1)
+FIELD(CSR_DBG, ECODE, 16, 6)
+
+#define LOONGARCH_CSR_DERA           0x501 /* Debug era */
+#define LOONGARCH_CSR_DSAVE          0x502 /* Debug save */
+
+#endif /* LOONGARCH_CPU_CSR_H */
diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c
index 95900be257..33d9a9450d 100644
--- a/target/loongarch/cpu.c
+++ b/target/loongarch/cpu.c
@@ -16,6 +16,7 @@
 #include "cpu.h"
 #include "internals.h"
 #include "fpu/softfloat-helpers.h"
+#include "cpu-csr.h"
 
 const char * const regnames[32] = {
     "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7",
@@ -167,6 +168,8 @@ static void loongarch_la464_initfn(Object *obj)
     data = FIELD_DP32(data, CPUCFG20, L3IU_SETS, 14);
     data = FIELD_DP32(data, CPUCFG20, L3IU_SETS, 6);
     env->cpucfg[20] = data;
+
+    env->CSR_ASID = FIELD_DP64(0, CSR_ASID, ASIDBITS, 0xa);
 }
 
 static void loongarch_cpu_list_entry(gpointer data, gpointer user_data)
@@ -196,6 +199,44 @@ static void loongarch_cpu_reset(DeviceState *dev)
     env->fcsr0_mask = FCSR0_M1 | FCSR0_M2 | FCSR0_M3;
     env->fcsr0 = 0x0;
 
+    int n;
+    /* Set csr registers value after reset */
+    env->CSR_CRMD = FIELD_DP64(env->CSR_CRMD, CSR_CRMD, PLV, 0);
+    env->CSR_CRMD = FIELD_DP64(env->CSR_CRMD, CSR_CRMD, IE, 0);
+    env->CSR_CRMD = FIELD_DP64(env->CSR_CRMD, CSR_CRMD, DA, 1);
+    env->CSR_CRMD = FIELD_DP64(env->CSR_CRMD, CSR_CRMD, PG, 0);
+    env->CSR_CRMD = FIELD_DP64(env->CSR_CRMD, CSR_CRMD, DATF, 1);
+    env->CSR_CRMD = FIELD_DP64(env->CSR_CRMD, CSR_CRMD, DATM, 1);
+
+    env->CSR_EUEN = FIELD_DP64(env->CSR_EUEN, CSR_EUEN, FPE, 0);
+    env->CSR_EUEN = FIELD_DP64(env->CSR_EUEN, CSR_EUEN, SXE, 0);
+    env->CSR_EUEN = FIELD_DP64(env->CSR_EUEN, CSR_EUEN, ASXE, 0);
+    env->CSR_EUEN = FIELD_DP64(env->CSR_EUEN, CSR_EUEN, BTE, 0);
+
+    env->CSR_MISC = 0;
+
+    env->CSR_ECFG = FIELD_DP64(env->CSR_ECFG, CSR_ECFG, VS, 0);
+    env->CSR_ECFG = FIELD_DP64(env->CSR_ECFG, CSR_ECFG, LIE, 0);
+
+    env->CSR_ESTAT = env->CSR_ESTAT & (~MAKE_64BIT_MASK(0, 2));
+    env->CSR_RVACFG = FIELD_DP64(env->CSR_RVACFG, CSR_RVACFG, RBITS, 0);
+    env->CSR_TCFG = FIELD_DP64(env->CSR_TCFG, CSR_TCFG, EN, 0);
+    env->CSR_LLBCTL = FIELD_DP64(env->CSR_LLBCTL, CSR_LLBCTL, KLO, 0);
+    env->CSR_TLBRERA = FIELD_DP64(env->CSR_TLBRERA, CSR_TLBRERA, ISTLBR, 0);
+    env->CSR_MERRCTL = FIELD_DP64(env->CSR_MERRCTL, CSR_MERRCTL, ISMERR, 0);
+
+    env->CSR_PRCFG3 = FIELD_DP64(env->CSR_PRCFG3, CSR_PRCFG3, TLB_TYPE, 2);
+    env->CSR_PRCFG3 = FIELD_DP64(env->CSR_PRCFG3, CSR_PRCFG3, MTLB_ENTRY, 63);
+    env->CSR_PRCFG3 = FIELD_DP64(env->CSR_PRCFG3, CSR_PRCFG3, STLB_WAYS, 7);
+    env->CSR_PRCFG3 = FIELD_DP64(env->CSR_PRCFG3, CSR_PRCFG3, STLB_SETS, 8);
+
+    for (n = 0; n < 4; n++) {
+        env->CSR_DMW[n] = FIELD_DP64(env->CSR_DMW[n], CSR_DMW, PLV0, 0);
+        env->CSR_DMW[n] = FIELD_DP64(env->CSR_DMW[n], CSR_DMW, PLV1, 0);
+        env->CSR_DMW[n] = FIELD_DP64(env->CSR_DMW[n], CSR_DMW, PLV2, 0);
+        env->CSR_DMW[n] = FIELD_DP64(env->CSR_DMW[n], CSR_DMW, PLV3, 0);
+    }
+
     restore_fp_status(env);
     cs->exception_index = -1;
 }
diff --git a/target/loongarch/cpu.h b/target/loongarch/cpu.h
index 3e8ba46377..3ec978f47c 100644
--- a/target/loongarch/cpu.h
+++ b/target/loongarch/cpu.h
@@ -172,6 +172,15 @@ FIELD(CPUCFG20, L3IU_WAYS, 0, 16)
 FIELD(CPUCFG20, L3IU_SETS, 16, 8)
 FIELD(CPUCFG20, L3IU_SIZE, 24, 7)
 
+/*CSR_CRMD */
+FIELD(CSR_CRMD, PLV, 0, 2)
+FIELD(CSR_CRMD, IE, 2, 1)
+FIELD(CSR_CRMD, DA, 3, 1)
+FIELD(CSR_CRMD, PG, 4, 1)
+FIELD(CSR_CRMD, DATF, 5, 2)
+FIELD(CSR_CRMD, DATM, 7, 2)
+FIELD(CSR_CRMD, WE, 9, 1)
+
 extern const char * const regnames[32];
 extern const char * const fregnames[32];
 
@@ -192,6 +201,61 @@ typedef struct CPUArchState {
     uint64_t llval;
 
     uint64_t badaddr;
+
+    /* LoongArch CSRs */
+    uint64_t CSR_CRMD;
+    uint64_t CSR_PRMD;
+    uint64_t CSR_EUEN;
+    uint64_t CSR_MISC;
+    uint64_t CSR_ECFG;
+    uint64_t CSR_ESTAT;
+    uint64_t CSR_ERA;
+    uint64_t CSR_BADV;
+    uint64_t CSR_BADI;
+    uint64_t CSR_EENTRY;
+    uint64_t CSR_TLBIDX;
+    uint64_t CSR_TLBEHI;
+    uint64_t CSR_TLBELO0;
+    uint64_t CSR_TLBELO1;
+    uint64_t CSR_ASID;
+    uint64_t CSR_PGDL;
+    uint64_t CSR_PGDH;
+    uint64_t CSR_PGD;
+    uint64_t CSR_PWCL;
+    uint64_t CSR_PWCH;
+    uint64_t CSR_STLBPS;
+    uint64_t CSR_RVACFG;
+    uint64_t CSR_PRCFG1;
+    uint64_t CSR_PRCFG2;
+    uint64_t CSR_PRCFG3;
+    uint64_t CSR_SAVE[16];
+    uint64_t CSR_TID;
+    uint64_t CSR_TCFG;
+    uint64_t CSR_TVAL;
+    uint64_t CSR_CNTC;
+    uint64_t CSR_TICLR;
+    uint64_t CSR_LLBCTL;
+    uint64_t CSR_IMPCTL1;
+    uint64_t CSR_IMPCTL2;
+    uint64_t CSR_TLBRENTRY;
+    uint64_t CSR_TLBRBADV;
+    uint64_t CSR_TLBRERA;
+    uint64_t CSR_TLBRSAVE;
+    uint64_t CSR_TLBRELO0;
+    uint64_t CSR_TLBRELO1;
+    uint64_t CSR_TLBREHI;
+    uint64_t CSR_TLBRPRMD;
+    uint64_t CSR_MERRCTL;
+    uint64_t CSR_MERRINFO1;
+    uint64_t CSR_MERRINFO2;
+    uint64_t CSR_MERRENTRY;
+    uint64_t CSR_MERRERA;
+    uint64_t CSR_MERRSAVE;
+    uint64_t CSR_CTAG;
+    uint64_t CSR_DMW[4];
+    uint64_t CSR_DBG;
+    uint64_t CSR_DERA;
+    uint64_t CSR_DSAVE;
 } CPULoongArchState;
 
 /**

From 67ebd42a4884b5d6f29391beada050823108bd4f Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:10 +0800
Subject: [PATCH 827/935] target/loongarch: Add basic vmstate description of
 CPU.

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-21-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/cpu.c       |  1 +
 target/loongarch/internals.h |  2 +
 target/loongarch/machine.c   | 85 ++++++++++++++++++++++++++++++++++++
 target/loongarch/meson.build |  6 +++
 4 files changed, 94 insertions(+)
 create mode 100644 target/loongarch/machine.c

diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c
index 33d9a9450d..267d96c9a8 100644
--- a/target/loongarch/cpu.c
+++ b/target/loongarch/cpu.c
@@ -336,6 +336,7 @@ static void loongarch_cpu_class_init(ObjectClass *c, void *data)
     cc->class_by_name = loongarch_cpu_class_by_name;
     cc->dump_state = loongarch_cpu_dump_state;
     cc->set_pc = loongarch_cpu_set_pc;
+    dc->vmsd = &vmstate_loongarch_cpu;
     cc->disas_set_info = loongarch_cpu_disas_set_info;
 #ifdef CONFIG_TCG
     cc->tcg_ops = &loongarch_tcg_ops;
diff --git a/target/loongarch/internals.h b/target/loongarch/internals.h
index 1a3b39e0be..39960dee27 100644
--- a/target/loongarch/internals.h
+++ b/target/loongarch/internals.h
@@ -25,4 +25,6 @@ const char *loongarch_exception_name(int32_t exception);
 
 void restore_fp_status(CPULoongArchState *env);
 
+extern const VMStateDescription vmstate_loongarch_cpu;
+
 #endif
diff --git a/target/loongarch/machine.c b/target/loongarch/machine.c
new file mode 100644
index 0000000000..49a06fdf28
--- /dev/null
+++ b/target/loongarch/machine.c
@@ -0,0 +1,85 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * QEMU LoongArch Machine State
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "migration/cpu.h"
+
+/* LoongArch CPU state */
+
+const VMStateDescription vmstate_loongarch_cpu = {
+    .name = "cpu",
+    .version_id = 0,
+    .minimum_version_id = 0,
+    .fields = (VMStateField[]) {
+
+        VMSTATE_UINTTL_ARRAY(env.gpr, LoongArchCPU, 32),
+        VMSTATE_UINTTL(env.pc, LoongArchCPU),
+        VMSTATE_UINT64_ARRAY(env.fpr, LoongArchCPU, 32),
+        VMSTATE_UINT32(env.fcsr0, LoongArchCPU),
+        VMSTATE_BOOL_ARRAY(env.cf, LoongArchCPU, 8),
+
+        /* Remaining CSRs */
+        VMSTATE_UINT64(env.CSR_CRMD, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_PRMD, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_EUEN, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_MISC, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_ECFG, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_ESTAT, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_ERA, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_BADV, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_BADI, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_EENTRY, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_TLBIDX, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_TLBEHI, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_TLBELO0, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_TLBELO1, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_ASID, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_PGDL, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_PGDH, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_PGD, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_PWCL, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_PWCH, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_STLBPS, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_RVACFG, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_PRCFG1, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_PRCFG2, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_PRCFG3, LoongArchCPU),
+        VMSTATE_UINT64_ARRAY(env.CSR_SAVE, LoongArchCPU, 16),
+        VMSTATE_UINT64(env.CSR_TID, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_TCFG, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_TVAL, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_CNTC, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_TICLR, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_LLBCTL, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_IMPCTL1, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_IMPCTL2, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_TLBRENTRY, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_TLBRBADV, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_TLBRERA, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_TLBRSAVE, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_TLBRELO0, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_TLBRELO1, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_TLBREHI, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_TLBRPRMD, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_MERRCTL, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_MERRINFO1, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_MERRINFO2, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_MERRENTRY, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_MERRERA, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_MERRSAVE, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_CTAG, LoongArchCPU),
+        VMSTATE_UINT64_ARRAY(env.CSR_DMW, LoongArchCPU, 4),
+
+        /* Debug CSRs */
+        VMSTATE_UINT64(env.CSR_DBG, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_DERA, LoongArchCPU),
+        VMSTATE_UINT64(env.CSR_DSAVE, LoongArchCPU),
+
+        VMSTATE_END_OF_LIST()
+    },
+};
diff --git a/target/loongarch/meson.build b/target/loongarch/meson.build
index bcb076e55f..103f36ee15 100644
--- a/target/loongarch/meson.build
+++ b/target/loongarch/meson.build
@@ -14,6 +14,12 @@ loongarch_tcg_ss.add(files(
 ))
 loongarch_tcg_ss.add(zlib)
 
+loongarch_softmmu_ss = ss.source_set()
+loongarch_softmmu_ss.add(files(
+  'machine.c',
+))
+
 loongarch_ss.add_all(when: 'CONFIG_TCG', if_true: [loongarch_tcg_ss])
 
 target_arch += {'loongarch': loongarch_ss}
+target_softmmu_arch += {'loongarch': loongarch_softmmu_ss}

From 425876f5d8110d5c004843d104a1cb1b90b35542 Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:11 +0800
Subject: [PATCH 828/935] target/loongarch: Implement
 qmp_query_cpu_definitions()

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-22-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 qapi/machine-target.json |  6 ++++--
 target/loongarch/cpu.c   | 26 ++++++++++++++++++++++++++
 2 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 06b0d2ca61..2e267fa458 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -323,7 +323,8 @@
                    'TARGET_ARM',
                    'TARGET_I386',
                    'TARGET_S390X',
-                   'TARGET_MIPS' ] } }
+                   'TARGET_MIPS',
+                   'TARGET_LOONGARCH64' ] } }
 
 ##
 # @query-cpu-definitions:
@@ -339,4 +340,5 @@
                    'TARGET_ARM',
                    'TARGET_I386',
                    'TARGET_S390X',
-                   'TARGET_MIPS' ] } }
+                   'TARGET_MIPS',
+                   'TARGET_LOONGARCH64' ] } }
diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c
index 267d96c9a8..aa0e1f9167 100644
--- a/target/loongarch/cpu.c
+++ b/target/loongarch/cpu.c
@@ -365,3 +365,29 @@ static const TypeInfo loongarch_cpu_type_infos[] = {
 };
 
 DEFINE_TYPES(loongarch_cpu_type_infos)
+
+static void loongarch_cpu_add_definition(gpointer data, gpointer user_data)
+{
+    ObjectClass *oc = data;
+    CpuDefinitionInfoList **cpu_list = user_data;
+    CpuDefinitionInfo *info = g_new0(CpuDefinitionInfo, 1);
+    const char *typename = object_class_get_name(oc);
+
+    info->name = g_strndup(typename,
+                           strlen(typename) - strlen("-" TYPE_LOONGARCH_CPU));
+    info->q_typename = g_strdup(typename);
+
+    QAPI_LIST_PREPEND(*cpu_list, info);
+}
+
+CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
+{
+    CpuDefinitionInfoList *cpu_list = NULL;
+    GSList *list;
+
+    list = object_class_get_list(TYPE_LOONGARCH_CPU, false);
+    g_slist_foreach(list, loongarch_cpu_add_definition, &cpu_list);
+    g_slist_free(list);
+
+    return cpu_list;
+}

From 7e1c521e2a108a4ffceecd8bec846625c956c8f8 Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:12 +0800
Subject: [PATCH 829/935] target/loongarch: Add MMU support for LoongArch CPU.

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-23-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/cpu-param.h  |   2 +-
 target/loongarch/cpu.c        |  24 +++
 target/loongarch/cpu.h        |  51 ++++++
 target/loongarch/internals.h  |   9 +
 target/loongarch/machine.c    |  17 ++
 target/loongarch/meson.build  |   1 +
 target/loongarch/tlb_helper.c | 315 ++++++++++++++++++++++++++++++++++
 7 files changed, 418 insertions(+), 1 deletion(-)
 create mode 100644 target/loongarch/tlb_helper.c

diff --git a/target/loongarch/cpu-param.h b/target/loongarch/cpu-param.h
index 9a769b67e0..414d8fff46 100644
--- a/target/loongarch/cpu-param.h
+++ b/target/loongarch/cpu-param.h
@@ -13,6 +13,6 @@
 #define TARGET_VIRT_ADDR_SPACE_BITS 48
 
 #define TARGET_PAGE_BITS 14
-#define NB_MMU_MODES 4
+#define NB_MMU_MODES 5
 
 #endif
diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c
index aa0e1f9167..9a5cabbf4c 100644
--- a/target/loongarch/cpu.c
+++ b/target/loongarch/cpu.c
@@ -303,6 +303,21 @@ void loongarch_cpu_dump_state(CPUState *cs, FILE *f, int flags)
         }
     }
 
+    qemu_fprintf(f, "CRMD=%016" PRIx64 "\n", env->CSR_CRMD);
+    qemu_fprintf(f, "PRMD=%016" PRIx64 "\n", env->CSR_PRMD);
+    qemu_fprintf(f, "EUEN=%016" PRIx64 "\n", env->CSR_EUEN);
+    qemu_fprintf(f, "ESTAT=%016" PRIx64 "\n", env->CSR_ESTAT);
+    qemu_fprintf(f, "ERA=%016" PRIx64 "\n", env->CSR_ERA);
+    qemu_fprintf(f, "BADV=%016" PRIx64 "\n", env->CSR_BADV);
+    qemu_fprintf(f, "BADI=%016" PRIx64 "\n", env->CSR_BADI);
+    qemu_fprintf(f, "EENTRY=%016" PRIx64 "\n", env->CSR_EENTRY);
+    qemu_fprintf(f, "PRCFG1=%016" PRIx64 ", PRCFG2=%016" PRIx64 ","
+                 " PRCFG3=%016" PRIx64 "\n",
+                 env->CSR_PRCFG1, env->CSR_PRCFG3, env->CSR_PRCFG3);
+    qemu_fprintf(f, "TLBRENTRY=%016" PRIx64 "\n", env->CSR_TLBRENTRY);
+    qemu_fprintf(f, "TLBRBADV=%016" PRIx64 "\n", env->CSR_TLBRBADV);
+    qemu_fprintf(f, "TLBRERA=%016" PRIx64 "\n", env->CSR_TLBRERA);
+
     /* fpr */
     if (flags & CPU_DUMP_FPU) {
         for (i = 0; i < 32; i++) {
@@ -320,9 +335,17 @@ void loongarch_cpu_dump_state(CPUState *cs, FILE *f, int flags)
 static struct TCGCPUOps loongarch_tcg_ops = {
     .initialize = loongarch_translate_init,
     .synchronize_from_tb = loongarch_cpu_synchronize_from_tb,
+
+    .tlb_fill = loongarch_cpu_tlb_fill,
 };
 #endif /* CONFIG_TCG */
 
+#include "hw/core/sysemu-cpu-ops.h"
+
+static const struct SysemuCPUOps loongarch_sysemu_ops = {
+    .get_phys_page_debug = loongarch_cpu_get_phys_page_debug,
+};
+
 static void loongarch_cpu_class_init(ObjectClass *c, void *data)
 {
     LoongArchCPUClass *lacc = LOONGARCH_CPU_CLASS(c);
@@ -337,6 +360,7 @@ static void loongarch_cpu_class_init(ObjectClass *c, void *data)
     cc->dump_state = loongarch_cpu_dump_state;
     cc->set_pc = loongarch_cpu_set_pc;
     dc->vmsd = &vmstate_loongarch_cpu;
+    cc->sysemu_ops = &loongarch_sysemu_ops;
     cc->disas_set_info = loongarch_cpu_disas_set_info;
 #ifdef CONFIG_TCG
     cc->tcg_ops = &loongarch_tcg_ops;
diff --git a/target/loongarch/cpu.h b/target/loongarch/cpu.h
index 3ec978f47c..e0415d8929 100644
--- a/target/loongarch/cpu.h
+++ b/target/loongarch/cpu.h
@@ -184,6 +184,26 @@ FIELD(CSR_CRMD, WE, 9, 1)
 extern const char * const regnames[32];
 extern const char * const fregnames[32];
 
+#define LOONGARCH_STLB         2048 /* 2048 STLB */
+#define LOONGARCH_MTLB         64   /* 64 MTLB */
+#define LOONGARCH_TLB_MAX      (LOONGARCH_STLB + LOONGARCH_MTLB)
+
+/*
+ * define the ASID PS E VPPN field of TLB
+ */
+FIELD(TLB_MISC, E, 0, 1)
+FIELD(TLB_MISC, ASID, 1, 10)
+FIELD(TLB_MISC, VPPN, 13, 35)
+FIELD(TLB_MISC, PS, 48, 6)
+
+struct LoongArchTLB {
+    uint64_t tlb_misc;
+    /* Fields corresponding to CSR_TLBELO0/1 */
+    uint64_t tlb_entry0;
+    uint64_t tlb_entry1;
+};
+typedef struct LoongArchTLB LoongArchTLB;
+
 typedef struct CPUArchState {
     uint64_t gpr[32];
     uint64_t pc;
@@ -256,6 +276,8 @@ typedef struct CPUArchState {
     uint64_t CSR_DBG;
     uint64_t CSR_DERA;
     uint64_t CSR_DSAVE;
+
+    LoongArchTLB  tlb[LOONGARCH_TLB_MAX];
 } CPULoongArchState;
 
 /**
@@ -294,6 +316,35 @@ struct LoongArchCPUClass {
     DeviceReset parent_reset;
 };
 
+/*
+ * LoongArch CPUs has 4 privilege levels.
+ * 0 for kernel mode, 3 for user mode.
+ * Define an extra index for DA(direct addressing) mode.
+ */
+#define MMU_KERNEL_IDX   0
+#define MMU_USER_IDX     3
+#define MMU_DA_IDX       4
+
+static inline int cpu_mmu_index(CPULoongArchState *env, bool ifetch)
+{
+    uint8_t pg = FIELD_EX64(env->CSR_CRMD, CSR_CRMD, PG);
+
+    if (!pg) {
+        return MMU_DA_IDX;
+    }
+    return FIELD_EX64(env->CSR_CRMD, CSR_CRMD, PLV);
+}
+
+static inline void cpu_get_tb_cpu_state(CPULoongArchState *env,
+                                        target_ulong *pc,
+                                        target_ulong *cs_base,
+                                        uint32_t *flags)
+{
+    *pc = env->pc;
+    *cs_base = 0;
+    *flags = cpu_mmu_index(env, false);
+}
+
 void loongarch_cpu_list(void);
 
 #define cpu_list loongarch_cpu_list
diff --git a/target/loongarch/internals.h b/target/loongarch/internals.h
index 39960dee27..2fed768653 100644
--- a/target/loongarch/internals.h
+++ b/target/loongarch/internals.h
@@ -13,6 +13,9 @@
 #define FCMP_UN   0b0100  /* unordered */
 #define FCMP_GT   0b1000  /* fp0 > fp1 */
 
+#define TARGET_PHYS_MASK MAKE_64BIT_MASK(0, TARGET_PHYS_ADDR_SPACE_BITS)
+#define TARGET_VIRT_MASK MAKE_64BIT_MASK(0, TARGET_VIRT_ADDR_SPACE_BITS)
+
 void loongarch_translate_init(void);
 
 void loongarch_cpu_dump_state(CPUState *cpu, FILE *f, int flags);
@@ -27,4 +30,10 @@ void restore_fp_status(CPULoongArchState *env);
 
 extern const VMStateDescription vmstate_loongarch_cpu;
 
+bool loongarch_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
+                            MMUAccessType access_type, int mmu_idx,
+                            bool probe, uintptr_t retaddr);
+
+hwaddr loongarch_cpu_get_phys_page_debug(CPUState *cpu, vaddr addr);
+
 #endif
diff --git a/target/loongarch/machine.c b/target/loongarch/machine.c
index 49a06fdf28..b1e523ea72 100644
--- a/target/loongarch/machine.c
+++ b/target/loongarch/machine.c
@@ -8,6 +8,20 @@
 #include "qemu/osdep.h"
 #include "cpu.h"
 #include "migration/cpu.h"
+#include "internals.h"
+
+/* TLB state */
+const VMStateDescription vmstate_tlb = {
+    .name = "cpu/tlb",
+    .version_id = 0,
+    .minimum_version_id = 0,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT64(tlb_misc, LoongArchTLB),
+        VMSTATE_UINT64(tlb_entry0, LoongArchTLB),
+        VMSTATE_UINT64(tlb_entry1, LoongArchTLB),
+        VMSTATE_END_OF_LIST()
+    }
+};
 
 /* LoongArch CPU state */
 
@@ -79,6 +93,9 @@ const VMStateDescription vmstate_loongarch_cpu = {
         VMSTATE_UINT64(env.CSR_DBG, LoongArchCPU),
         VMSTATE_UINT64(env.CSR_DERA, LoongArchCPU),
         VMSTATE_UINT64(env.CSR_DSAVE, LoongArchCPU),
+        /* TLB */
+        VMSTATE_STRUCT_ARRAY(env.tlb, LoongArchCPU, LOONGARCH_TLB_MAX,
+                             0, vmstate_tlb, LoongArchTLB),
 
         VMSTATE_END_OF_LIST()
     },
diff --git a/target/loongarch/meson.build b/target/loongarch/meson.build
index 103f36ee15..435cc75999 100644
--- a/target/loongarch/meson.build
+++ b/target/loongarch/meson.build
@@ -17,6 +17,7 @@ loongarch_tcg_ss.add(zlib)
 loongarch_softmmu_ss = ss.source_set()
 loongarch_softmmu_ss.add(files(
   'machine.c',
+  'tlb_helper.c',
 ))
 
 loongarch_ss.add_all(when: 'CONFIG_TCG', if_true: [loongarch_tcg_ss])
diff --git a/target/loongarch/tlb_helper.c b/target/loongarch/tlb_helper.c
new file mode 100644
index 0000000000..fad8bc7746
--- /dev/null
+++ b/target/loongarch/tlb_helper.c
@@ -0,0 +1,315 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * QEMU LoongArch TLB helpers
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ *
+ */
+
+#include "qemu/osdep.h"
+
+#include "cpu.h"
+#include "internals.h"
+#include "exec/exec-all.h"
+#include "exec/cpu_ldst.h"
+#include "exec/log.h"
+#include "cpu-csr.h"
+
+enum {
+    TLBRET_MATCH = 0,
+    TLBRET_BADADDR = 1,
+    TLBRET_NOMATCH = 2,
+    TLBRET_INVALID = 3,
+    TLBRET_DIRTY = 4,
+    TLBRET_RI = 5,
+    TLBRET_XI = 6,
+    TLBRET_PE = 7,
+};
+
+static int loongarch_map_tlb_entry(CPULoongArchState *env, hwaddr *physical,
+                                   int *prot, target_ulong address,
+                                   int access_type, int index, int mmu_idx)
+{
+    LoongArchTLB *tlb = &env->tlb[index];
+    uint64_t plv = mmu_idx;
+    uint64_t tlb_entry, tlb_ppn;
+    uint8_t tlb_ps, n, tlb_v, tlb_d, tlb_plv, tlb_nx, tlb_nr, tlb_rplv;
+
+    if (index >= LOONGARCH_STLB) {
+        tlb_ps = FIELD_EX64(tlb->tlb_misc, TLB_MISC, PS);
+    } else {
+        tlb_ps = FIELD_EX64(env->CSR_STLBPS, CSR_STLBPS, PS);
+    }
+    n = (address >> tlb_ps) & 0x1;/* Odd or even */
+
+    tlb_entry = n ? tlb->tlb_entry1 : tlb->tlb_entry0;
+    tlb_v = FIELD_EX64(tlb_entry, TLBENTRY, V);
+    tlb_d = FIELD_EX64(tlb_entry, TLBENTRY, D);
+    tlb_plv = FIELD_EX64(tlb_entry, TLBENTRY, PLV);
+    tlb_ppn = FIELD_EX64(tlb_entry, TLBENTRY, PPN);
+    tlb_nx = FIELD_EX64(tlb_entry, TLBENTRY, NX);
+    tlb_nr = FIELD_EX64(tlb_entry, TLBENTRY, NR);
+    tlb_rplv = FIELD_EX64(tlb_entry, TLBENTRY, RPLV);
+
+    /* Check access rights */
+    if (!tlb_v) {
+        return TLBRET_INVALID;
+    }
+
+    if (access_type == MMU_INST_FETCH && tlb_nx) {
+        return TLBRET_XI;
+    }
+
+    if (access_type == MMU_DATA_LOAD && tlb_nr) {
+        return TLBRET_RI;
+    }
+
+    if (((tlb_rplv == 0) && (plv > tlb_plv)) ||
+        ((tlb_rplv == 1) && (plv != tlb_plv))) {
+        return TLBRET_PE;
+    }
+
+    if ((access_type == MMU_DATA_STORE) && !tlb_d) {
+        return TLBRET_DIRTY;
+    }
+
+    /*
+     * tlb_entry contains ppn[47:12] while 16KiB ppn is [47:15]
+     * need adjust.
+     */
+    *physical = (tlb_ppn << R_TLBENTRY_PPN_SHIFT) |
+                (address & MAKE_64BIT_MASK(0, tlb_ps));
+    *prot = PAGE_READ;
+    if (tlb_d) {
+        *prot |= PAGE_WRITE;
+    }
+    if (!tlb_nx) {
+        *prot |= PAGE_EXEC;
+    }
+    return TLBRET_MATCH;
+}
+
+/*
+ * One tlb entry holds an adjacent odd/even pair, the vpn is the
+ * content of the virtual page number divided by 2. So the
+ * compare vpn is bit[47:15] for 16KiB page. while the vppn
+ * field in tlb entry contains bit[47:13], so need adjust.
+ * virt_vpn = vaddr[47:13]
+ */
+static bool loongarch_tlb_search(CPULoongArchState *env, target_ulong vaddr,
+                                 int *index)
+{
+    LoongArchTLB *tlb;
+    uint16_t csr_asid, tlb_asid, stlb_idx;
+    uint8_t tlb_e, tlb_ps, tlb_g, stlb_ps;
+    int i, compare_shift;
+    uint64_t vpn, tlb_vppn;
+
+    csr_asid = FIELD_EX64(env->CSR_ASID, CSR_ASID, ASID);
+    stlb_ps = FIELD_EX64(env->CSR_STLBPS, CSR_STLBPS, PS);
+    vpn = (vaddr & TARGET_VIRT_MASK) >> (stlb_ps + 1);
+    stlb_idx = vpn & 0xff; /* VA[25:15] <==> TLBIDX.index for 16KiB Page */
+    compare_shift = stlb_ps + 1 - R_TLB_MISC_VPPN_SHIFT;
+
+    /* Search STLB */
+    for (i = 0; i < 8; ++i) {
+        tlb = &env->tlb[i * 256 + stlb_idx];
+        tlb_e = FIELD_EX64(tlb->tlb_misc, TLB_MISC, E);
+        if (tlb_e) {
+            tlb_vppn = FIELD_EX64(tlb->tlb_misc, TLB_MISC, VPPN);
+            tlb_asid = FIELD_EX64(tlb->tlb_misc, TLB_MISC, ASID);
+            tlb_g = FIELD_EX64(tlb->tlb_entry0, TLBENTRY, G);
+
+            if ((tlb_g == 1 || tlb_asid == csr_asid) &&
+                (vpn == (tlb_vppn >> compare_shift))) {
+                *index = i * 256 + stlb_idx;
+                return true;
+            }
+        }
+    }
+
+    /* Search MTLB */
+    for (i = LOONGARCH_STLB; i < LOONGARCH_TLB_MAX; ++i) {
+        tlb = &env->tlb[i];
+        tlb_e = FIELD_EX64(tlb->tlb_misc, TLB_MISC, E);
+        if (tlb_e) {
+            tlb_vppn = FIELD_EX64(tlb->tlb_misc, TLB_MISC, VPPN);
+            tlb_ps = FIELD_EX64(tlb->tlb_misc, TLB_MISC, PS);
+            tlb_asid = FIELD_EX64(tlb->tlb_misc, TLB_MISC, ASID);
+            tlb_g = FIELD_EX64(tlb->tlb_entry0, TLBENTRY, G);
+            compare_shift = tlb_ps + 1 - R_TLB_MISC_VPPN_SHIFT;
+            vpn = (vaddr & TARGET_VIRT_MASK) >> (tlb_ps + 1);
+            if ((tlb_g == 1 || tlb_asid == csr_asid) &&
+                (vpn == (tlb_vppn >> compare_shift))) {
+                *index = i;
+                return true;
+            }
+        }
+    }
+    return false;
+}
+
+static int loongarch_map_address(CPULoongArchState *env, hwaddr *physical,
+                                 int *prot, target_ulong address,
+                                 MMUAccessType access_type, int mmu_idx)
+{
+    int index, match;
+
+    match = loongarch_tlb_search(env, address, &index);
+    if (match) {
+        return loongarch_map_tlb_entry(env, physical, prot,
+                                       address, access_type, index, mmu_idx);
+    }
+
+    return TLBRET_NOMATCH;
+}
+
+static int get_physical_address(CPULoongArchState *env, hwaddr *physical,
+                                int *prot, target_ulong address,
+                                MMUAccessType access_type, int mmu_idx)
+{
+    int user_mode = mmu_idx == MMU_USER_IDX;
+    int kernel_mode = mmu_idx == MMU_KERNEL_IDX;
+    uint32_t plv, base_c, base_v;
+    int64_t addr_high;
+    uint8_t da = FIELD_EX64(env->CSR_CRMD, CSR_CRMD, DA);
+    uint8_t pg = FIELD_EX64(env->CSR_CRMD, CSR_CRMD, PG);
+
+    /* Check PG and DA */
+    if (da & !pg) {
+        *physical = address & TARGET_PHYS_MASK;
+        *prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
+        return TLBRET_MATCH;
+    }
+
+    plv = kernel_mode | (user_mode << R_CSR_DMW_PLV3_SHIFT);
+    base_v = address >> TARGET_VIRT_ADDR_SPACE_BITS;
+    /* Check direct map window */
+    for (int i = 0; i < 4; i++) {
+        base_c = env->CSR_DMW[i] >> TARGET_VIRT_ADDR_SPACE_BITS;
+        if ((plv & env->CSR_DMW[i]) && (base_c == base_v)) {
+            *physical = dmw_va2pa(address);
+            *prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
+            return TLBRET_MATCH;
+        }
+    }
+
+    /* Check valid extension */
+    addr_high = sextract64(address, TARGET_VIRT_ADDR_SPACE_BITS, 16);
+    if (!(addr_high == 0 || addr_high == -1)) {
+        return TLBRET_BADADDR;
+    }
+
+    /* Mapped address */
+    return loongarch_map_address(env, physical, prot, address,
+                                 access_type, mmu_idx);
+}
+
+hwaddr loongarch_cpu_get_phys_page_debug(CPUState *cs, vaddr addr)
+{
+    LoongArchCPU *cpu = LOONGARCH_CPU(cs);
+    CPULoongArchState *env = &cpu->env;
+    hwaddr phys_addr;
+    int prot;
+
+    if (get_physical_address(env, &phys_addr, &prot, addr, MMU_DATA_LOAD,
+                             cpu_mmu_index(env, false)) != 0) {
+        return -1;
+    }
+    return phys_addr;
+}
+
+static void raise_mmu_exception(CPULoongArchState *env, target_ulong address,
+                                MMUAccessType access_type, int tlb_error)
+{
+    CPUState *cs = env_cpu(env);
+
+    switch (tlb_error) {
+    default:
+    case TLBRET_BADADDR:
+        cs->exception_index = EXCCODE_ADEM;
+        break;
+    case TLBRET_NOMATCH:
+        /* No TLB match for a mapped address */
+        if (access_type == MMU_DATA_LOAD) {
+            cs->exception_index = EXCCODE_PIL;
+        } else if (access_type == MMU_DATA_STORE) {
+            cs->exception_index = EXCCODE_PIS;
+        } else if (access_type == MMU_INST_FETCH) {
+            cs->exception_index = EXCCODE_PIF;
+        }
+        env->CSR_TLBRERA = FIELD_DP64(env->CSR_TLBRERA, CSR_TLBRERA, ISTLBR, 1);
+        break;
+    case TLBRET_INVALID:
+        /* TLB match with no valid bit */
+        if (access_type == MMU_DATA_LOAD) {
+            cs->exception_index = EXCCODE_PIL;
+        } else if (access_type == MMU_DATA_STORE) {
+            cs->exception_index = EXCCODE_PIS;
+        } else if (access_type == MMU_INST_FETCH) {
+            cs->exception_index = EXCCODE_PIF;
+        }
+        break;
+    case TLBRET_DIRTY:
+        /* TLB match but 'D' bit is cleared */
+        cs->exception_index = EXCCODE_PME;
+        break;
+    case TLBRET_XI:
+        /* Execute-Inhibit Exception */
+        cs->exception_index = EXCCODE_PNX;
+        break;
+    case TLBRET_RI:
+        /* Read-Inhibit Exception */
+        cs->exception_index = EXCCODE_PNR;
+        break;
+    case TLBRET_PE:
+        /* Privileged Exception */
+        cs->exception_index = EXCCODE_PPI;
+        break;
+    }
+
+    if (tlb_error == TLBRET_NOMATCH) {
+        env->CSR_TLBRBADV = address;
+        env->CSR_TLBREHI = FIELD_DP64(env->CSR_TLBREHI, CSR_TLBREHI, VPPN,
+                                      extract64(address, 13, 35));
+    } else {
+        if (!FIELD_EX64(env->CSR_DBG, CSR_DBG, DST)) {
+            env->CSR_BADV = address;
+        }
+        env->CSR_TLBEHI = address & (TARGET_PAGE_MASK << 1);
+   }
+}
+
+bool loongarch_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
+                            MMUAccessType access_type, int mmu_idx,
+                            bool probe, uintptr_t retaddr)
+{
+    LoongArchCPU *cpu = LOONGARCH_CPU(cs);
+    CPULoongArchState *env = &cpu->env;
+    hwaddr physical;
+    int prot;
+    int ret = TLBRET_BADADDR;
+
+    /* Data access */
+    ret = get_physical_address(env, &physical, &prot, address,
+                               access_type, mmu_idx);
+
+    if (ret == TLBRET_MATCH) {
+        tlb_set_page(cs, address & TARGET_PAGE_MASK,
+                     physical & TARGET_PAGE_MASK, prot,
+                     mmu_idx, TARGET_PAGE_SIZE);
+        qemu_log_mask(CPU_LOG_MMU,
+                      "%s address=%" VADDR_PRIx " physical " TARGET_FMT_plx
+                      " prot %d\n", __func__, address, physical, prot);
+        return true;
+    } else {
+        qemu_log_mask(CPU_LOG_MMU,
+                      "%s address=%" VADDR_PRIx " ret %d\n", __func__, address,
+                      ret);
+    }
+    if (probe) {
+        return false;
+    }
+    raise_mmu_exception(env, address, access_type, ret);
+    cpu_loop_exit_restore(cs, retaddr);
+}

From f757a2cd6948344b8303e375f6f9d72887abf1f4 Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:13 +0800
Subject: [PATCH 830/935] target/loongarch: Add LoongArch interrupt and
 exception handle

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-24-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/cpu.c       | 230 +++++++++++++++++++++++++++++++++++
 target/loongarch/cpu.h       |   2 +
 target/loongarch/internals.h |   2 +
 3 files changed, 234 insertions(+)

diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c
index 9a5cabbf4c..01a0b4878a 100644
--- a/target/loongarch/cpu.c
+++ b/target/loongarch/cpu.c
@@ -80,6 +80,215 @@ static void loongarch_cpu_set_pc(CPUState *cs, vaddr value)
     env->pc = value;
 }
 
+void loongarch_cpu_set_irq(void *opaque, int irq, int level)
+{
+    LoongArchCPU *cpu = opaque;
+    CPULoongArchState *env = &cpu->env;
+    CPUState *cs = CPU(cpu);
+
+    if (irq < 0 || irq >= N_IRQS) {
+        return;
+    }
+
+    env->CSR_ESTAT = deposit64(env->CSR_ESTAT, irq, 1, level != 0);
+
+    if (FIELD_EX64(env->CSR_ESTAT, CSR_ESTAT, IS)) {
+        cpu_interrupt(cs, CPU_INTERRUPT_HARD);
+    } else {
+        cpu_reset_interrupt(cs, CPU_INTERRUPT_HARD);
+    }
+}
+
+static inline bool cpu_loongarch_hw_interrupts_enabled(CPULoongArchState *env)
+{
+    bool ret = 0;
+
+    ret = (FIELD_EX64(env->CSR_CRMD, CSR_CRMD, IE) &&
+          !(FIELD_EX64(env->CSR_DBG, CSR_DBG, DST)));
+
+    return ret;
+}
+
+/* Check if there is pending and not masked out interrupt */
+static inline bool cpu_loongarch_hw_interrupts_pending(CPULoongArchState *env)
+{
+    uint32_t pending;
+    uint32_t status;
+    bool r;
+
+    pending = FIELD_EX64(env->CSR_ESTAT, CSR_ESTAT, IS);
+    status  = FIELD_EX64(env->CSR_ECFG, CSR_ECFG, LIE);
+
+    r = (pending & status) != 0;
+    return r;
+}
+
+static void loongarch_cpu_do_interrupt(CPUState *cs)
+{
+    LoongArchCPU *cpu = LOONGARCH_CPU(cs);
+    CPULoongArchState *env = &cpu->env;
+    bool update_badinstr = 1;
+    int cause = -1;
+    const char *name;
+    bool tlbfill = FIELD_EX64(env->CSR_TLBRERA, CSR_TLBRERA, ISTLBR);
+    uint32_t vec_size = FIELD_EX64(env->CSR_ECFG, CSR_ECFG, VS);
+
+    if (cs->exception_index != EXCCODE_INT) {
+        if (cs->exception_index < 0 ||
+            cs->exception_index > ARRAY_SIZE(excp_names)) {
+            name = "unknown";
+        } else {
+            name = excp_names[cs->exception_index];
+        }
+
+        qemu_log_mask(CPU_LOG_INT,
+                     "%s enter: pc " TARGET_FMT_lx " ERA " TARGET_FMT_lx
+                     " TLBRERA " TARGET_FMT_lx " %s exception\n", __func__,
+                     env->pc, env->CSR_ERA, env->CSR_TLBRERA, name);
+    }
+
+    switch (cs->exception_index) {
+    case EXCCODE_DBP:
+        env->CSR_DBG = FIELD_DP64(env->CSR_DBG, CSR_DBG, DCL, 1);
+        env->CSR_DBG = FIELD_DP64(env->CSR_DBG, CSR_DBG, ECODE, 0xC);
+        goto set_DERA;
+    set_DERA:
+        env->CSR_DERA = env->pc;
+        env->CSR_DBG = FIELD_DP64(env->CSR_DBG, CSR_DBG, DST, 1);
+        env->pc = env->CSR_EENTRY + 0x480;
+        break;
+    case EXCCODE_INT:
+        if (FIELD_EX64(env->CSR_DBG, CSR_DBG, DST)) {
+            env->CSR_DBG = FIELD_DP64(env->CSR_DBG, CSR_DBG, DEI, 1);
+            goto set_DERA;
+        }
+        QEMU_FALLTHROUGH;
+    case EXCCODE_PIF:
+        cause = cs->exception_index;
+        update_badinstr = 0;
+        break;
+    case EXCCODE_ADEM:
+    case EXCCODE_SYS:
+    case EXCCODE_BRK:
+    case EXCCODE_PIL:
+    case EXCCODE_PIS:
+    case EXCCODE_PME:
+    case EXCCODE_PNR:
+    case EXCCODE_PNX:
+    case EXCCODE_PPI:
+    case EXCCODE_INE:
+    case EXCCODE_IPE:
+    case EXCCODE_FPE:
+        cause = cs->exception_index;
+        break;
+    default:
+        qemu_log("Error: exception(%d) '%s' has not been supported\n",
+                 cs->exception_index, excp_names[cs->exception_index]);
+        abort();
+    }
+
+    if (update_badinstr) {
+        env->CSR_BADI = cpu_ldl_code(env, env->pc);
+    }
+
+    /* Save PLV and IE */
+    if (tlbfill) {
+        env->CSR_TLBRPRMD = FIELD_DP64(env->CSR_TLBRPRMD, CSR_TLBRPRMD, PPLV,
+                                       FIELD_EX64(env->CSR_CRMD,
+                                       CSR_CRMD, PLV));
+        env->CSR_TLBRPRMD = FIELD_DP64(env->CSR_TLBRPRMD, CSR_TLBRPRMD, PIE,
+                                       FIELD_EX64(env->CSR_CRMD, CSR_CRMD, IE));
+        /* set the DA mode */
+        env->CSR_CRMD = FIELD_DP64(env->CSR_CRMD, CSR_CRMD, DA, 1);
+        env->CSR_CRMD = FIELD_DP64(env->CSR_CRMD, CSR_CRMD, PG, 0);
+        env->CSR_TLBRERA = FIELD_DP64(env->CSR_TLBRERA, CSR_TLBRERA,
+                                      PC, (env->pc >> 2));
+    } else {
+        env->CSR_ESTAT = FIELD_DP64(env->CSR_ESTAT, CSR_ESTAT, ECODE, cause);
+        env->CSR_PRMD = FIELD_DP64(env->CSR_PRMD, CSR_PRMD, PPLV,
+                                   FIELD_EX64(env->CSR_CRMD, CSR_CRMD, PLV));
+        env->CSR_PRMD = FIELD_DP64(env->CSR_PRMD, CSR_PRMD, PIE,
+                                   FIELD_EX64(env->CSR_CRMD, CSR_CRMD, IE));
+        env->CSR_ERA = env->pc;
+    }
+
+    env->CSR_CRMD = FIELD_DP64(env->CSR_CRMD, CSR_CRMD, PLV, 0);
+    env->CSR_CRMD = FIELD_DP64(env->CSR_CRMD, CSR_CRMD, IE, 0);
+
+    if  (cs->exception_index == EXCCODE_INT) {
+        /* Interrupt */
+        uint32_t vector = 0;
+        uint32_t pending = FIELD_EX64(env->CSR_ESTAT, CSR_ESTAT, IS);
+        pending &= FIELD_EX64(env->CSR_ECFG, CSR_ECFG, LIE);
+
+        /* Find the highest-priority interrupt. */
+        vector = 31 - clz32(pending);
+        env->pc = env->CSR_EENTRY + (EXCCODE_EXTERNAL_INT + vector) * vec_size;
+        qemu_log_mask(CPU_LOG_INT,
+                      "%s: PC " TARGET_FMT_lx " ERA " TARGET_FMT_lx
+                      " cause %d\n" "    A " TARGET_FMT_lx " D "
+                      TARGET_FMT_lx " vector = %d ExC " TARGET_FMT_lx "ExS"
+                      TARGET_FMT_lx "\n",
+                      __func__, env->pc, env->CSR_ERA,
+                      cause, env->CSR_BADV, env->CSR_DERA, vector,
+                      env->CSR_ECFG, env->CSR_ESTAT);
+    } else {
+        if (tlbfill) {
+            env->pc = env->CSR_TLBRENTRY;
+        } else {
+            env->pc = env->CSR_EENTRY;
+            env->pc += cause * vec_size;
+        }
+        qemu_log_mask(CPU_LOG_INT,
+                      "%s: PC " TARGET_FMT_lx " ERA " TARGET_FMT_lx
+                      " cause %d%s\n, ESTAT " TARGET_FMT_lx
+                      " EXCFG " TARGET_FMT_lx " BADVA " TARGET_FMT_lx
+                      "BADI " TARGET_FMT_lx " SYS_NUM " TARGET_FMT_lu
+                      " cpu %d asid " TARGET_FMT_lx "\n", __func__, env->pc,
+                      tlbfill ? env->CSR_TLBRERA : env->CSR_ERA,
+                      cause, tlbfill ? "(refill)" : "", env->CSR_ESTAT,
+                      env->CSR_ECFG,
+                      tlbfill ? env->CSR_TLBRBADV : env->CSR_BADV,
+                      env->CSR_BADI, env->gpr[11], cs->cpu_index,
+                      env->CSR_ASID);
+    }
+    cs->exception_index = -1;
+}
+
+static void loongarch_cpu_do_transaction_failed(CPUState *cs, hwaddr physaddr,
+                                                vaddr addr, unsigned size,
+                                                MMUAccessType access_type,
+                                                int mmu_idx, MemTxAttrs attrs,
+                                                MemTxResult response,
+                                                uintptr_t retaddr)
+{
+    LoongArchCPU *cpu = LOONGARCH_CPU(cs);
+    CPULoongArchState *env = &cpu->env;
+
+    if (access_type == MMU_INST_FETCH) {
+        do_raise_exception(env, EXCCODE_ADEF, retaddr);
+    } else {
+        do_raise_exception(env, EXCCODE_ADEM, retaddr);
+    }
+}
+
+static bool loongarch_cpu_exec_interrupt(CPUState *cs, int interrupt_request)
+{
+    if (interrupt_request & CPU_INTERRUPT_HARD) {
+        LoongArchCPU *cpu = LOONGARCH_CPU(cs);
+        CPULoongArchState *env = &cpu->env;
+
+        if (cpu_loongarch_hw_interrupts_enabled(env) &&
+            cpu_loongarch_hw_interrupts_pending(env)) {
+            /* Raise it */
+            cs->exception_index = EXCCODE_INT;
+            loongarch_cpu_do_interrupt(cs);
+            return true;
+        }
+    }
+    return false;
+}
+
 #ifdef CONFIG_TCG
 static void loongarch_cpu_synchronize_from_tb(CPUState *cs,
                                               const TranslationBlock *tb)
@@ -91,6 +300,20 @@ static void loongarch_cpu_synchronize_from_tb(CPUState *cs,
 }
 #endif /* CONFIG_TCG */
 
+static bool loongarch_cpu_has_work(CPUState *cs)
+{
+    LoongArchCPU *cpu = LOONGARCH_CPU(cs);
+    CPULoongArchState *env = &cpu->env;
+    bool has_work = false;
+
+    if ((cs->interrupt_request & CPU_INTERRUPT_HARD) &&
+        cpu_loongarch_hw_interrupts_pending(env)) {
+        has_work = true;
+    }
+
+    return has_work;
+}
+
 static void loongarch_la464_initfn(Object *obj)
 {
     LoongArchCPU *cpu = LOONGARCH_CPU(obj);
@@ -237,6 +460,8 @@ static void loongarch_cpu_reset(DeviceState *dev)
         env->CSR_DMW[n] = FIELD_DP64(env->CSR_DMW[n], CSR_DMW, PLV3, 0);
     }
 
+    env->pc = 0x1c000000;
+
     restore_fp_status(env);
     cs->exception_index = -1;
 }
@@ -269,6 +494,7 @@ static void loongarch_cpu_init(Object *obj)
     LoongArchCPU *cpu = LOONGARCH_CPU(obj);
 
     cpu_set_cpustate_pointers(cpu);
+    qdev_init_gpio_in(DEVICE(cpu), loongarch_cpu_set_irq, N_IRQS);
 }
 
 static ObjectClass *loongarch_cpu_class_by_name(const char *cpu_model)
@@ -337,6 +563,9 @@ static struct TCGCPUOps loongarch_tcg_ops = {
     .synchronize_from_tb = loongarch_cpu_synchronize_from_tb,
 
     .tlb_fill = loongarch_cpu_tlb_fill,
+    .cpu_exec_interrupt = loongarch_cpu_exec_interrupt,
+    .do_interrupt = loongarch_cpu_do_interrupt,
+    .do_transaction_failed = loongarch_cpu_do_transaction_failed,
 };
 #endif /* CONFIG_TCG */
 
@@ -357,6 +586,7 @@ static void loongarch_cpu_class_init(ObjectClass *c, void *data)
     device_class_set_parent_reset(dc, loongarch_cpu_reset, &lacc->parent_reset);
 
     cc->class_by_name = loongarch_cpu_class_by_name;
+    cc->has_work = loongarch_cpu_has_work;
     cc->dump_state = loongarch_cpu_dump_state;
     cc->set_pc = loongarch_cpu_set_pc;
     dc->vmsd = &vmstate_loongarch_cpu;
diff --git a/target/loongarch/cpu.h b/target/loongarch/cpu.h
index e0415d8929..b983ce241c 100644
--- a/target/loongarch/cpu.h
+++ b/target/loongarch/cpu.h
@@ -184,6 +184,8 @@ FIELD(CSR_CRMD, WE, 9, 1)
 extern const char * const regnames[32];
 extern const char * const fregnames[32];
 
+#define N_IRQS      13
+
 #define LOONGARCH_STLB         2048 /* 2048 STLB */
 #define LOONGARCH_MTLB         64   /* 64 MTLB */
 #define LOONGARCH_TLB_MAX      (LOONGARCH_STLB + LOONGARCH_MTLB)
diff --git a/target/loongarch/internals.h b/target/loongarch/internals.h
index 2fed768653..69183e8bb5 100644
--- a/target/loongarch/internals.h
+++ b/target/loongarch/internals.h
@@ -30,6 +30,8 @@ void restore_fp_status(CPULoongArchState *env);
 
 extern const VMStateDescription vmstate_loongarch_cpu;
 
+void loongarch_cpu_set_irq(void *opaque, int irq, int level);
+
 bool loongarch_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
                             MMUAccessType access_type, int mmu_idx,
                             bool probe, uintptr_t retaddr);

From dd615fa48da89b2308a907cc4e4956771c75d68f Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:14 +0800
Subject: [PATCH 831/935] target/loongarch: Add constant timer support

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-25-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/constant_timer.c | 64 +++++++++++++++++++++++++++++++
 target/loongarch/cpu.c            |  2 +
 target/loongarch/cpu.h            |  4 ++
 target/loongarch/internals.h      |  6 +++
 target/loongarch/meson.build      |  1 +
 5 files changed, 77 insertions(+)
 create mode 100644 target/loongarch/constant_timer.c

diff --git a/target/loongarch/constant_timer.c b/target/loongarch/constant_timer.c
new file mode 100644
index 0000000000..1851f53fd6
--- /dev/null
+++ b/target/loongarch/constant_timer.c
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * QEMU LoongArch constant timer support
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/timer.h"
+#include "cpu.h"
+#include "internals.h"
+#include "cpu-csr.h"
+
+#define TIMER_PERIOD                10 /* 10 ns period for 100 MHz frequency */
+#define CONSTANT_TIMER_TICK_MASK    0xfffffffffffcUL
+#define CONSTANT_TIMER_ENABLE       0x1UL
+
+uint64_t cpu_loongarch_get_constant_timer_counter(LoongArchCPU *cpu)
+{
+    return qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) / TIMER_PERIOD;
+}
+
+uint64_t cpu_loongarch_get_constant_timer_ticks(LoongArchCPU *cpu)
+{
+    uint64_t now, expire;
+
+    now = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
+    expire = timer_expire_time_ns(&cpu->timer);
+
+    return (expire - now) / TIMER_PERIOD;
+}
+
+void cpu_loongarch_store_constant_timer_config(LoongArchCPU *cpu,
+                                               uint64_t value)
+{
+    CPULoongArchState *env = &cpu->env;
+    uint64_t now, next;
+
+    env->CSR_TCFG = value;
+    if (value & CONSTANT_TIMER_ENABLE) {
+        now = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
+        next = now + (value & CONSTANT_TIMER_TICK_MASK) * TIMER_PERIOD;
+        timer_mod(&cpu->timer, next);
+    } else {
+        timer_del(&cpu->timer);
+    }
+}
+
+void loongarch_constant_timer_cb(void *opaque)
+{
+    LoongArchCPU *cpu  = opaque;
+    CPULoongArchState *env = &cpu->env;
+    uint64_t now, next;
+
+    if (FIELD_EX64(env->CSR_TCFG, CSR_TCFG, PERIODIC)) {
+        now = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
+        next = now + (env->CSR_TCFG & CONSTANT_TIMER_TICK_MASK) * TIMER_PERIOD;
+        timer_mod(&cpu->timer, next);
+    } else {
+        env->CSR_TCFG = FIELD_DP64(env->CSR_TCFG, CSR_TCFG, EN, 0);
+    }
+
+    loongarch_cpu_set_irq(opaque, IRQ_TIMER, 1);
+}
diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c
index 01a0b4878a..5ec0ba1cfa 100644
--- a/target/loongarch/cpu.c
+++ b/target/loongarch/cpu.c
@@ -495,6 +495,8 @@ static void loongarch_cpu_init(Object *obj)
 
     cpu_set_cpustate_pointers(cpu);
     qdev_init_gpio_in(DEVICE(cpu), loongarch_cpu_set_irq, N_IRQS);
+    timer_init_ns(&cpu->timer, QEMU_CLOCK_VIRTUAL,
+                  &loongarch_constant_timer_cb, cpu);
 }
 
 static ObjectClass *loongarch_cpu_class_by_name(const char *cpu_model)
diff --git a/target/loongarch/cpu.h b/target/loongarch/cpu.h
index b983ce241c..2081902f2e 100644
--- a/target/loongarch/cpu.h
+++ b/target/loongarch/cpu.h
@@ -11,6 +11,7 @@
 #include "exec/cpu-defs.h"
 #include "fpu/softfloat-types.h"
 #include "hw/registerfields.h"
+#include "qemu/timer.h"
 
 #define TCG_GUEST_DEFAULT_MO (0)
 
@@ -185,6 +186,8 @@ extern const char * const regnames[32];
 extern const char * const fregnames[32];
 
 #define N_IRQS      13
+#define IRQ_TIMER   11
+#define IRQ_IPI     12
 
 #define LOONGARCH_STLB         2048 /* 2048 STLB */
 #define LOONGARCH_MTLB         64   /* 64 MTLB */
@@ -295,6 +298,7 @@ struct ArchCPU {
 
     CPUNegativeOffsetState neg;
     CPULoongArchState env;
+    QEMUTimer timer;
 };
 
 #define TYPE_LOONGARCH_CPU "loongarch-cpu"
diff --git a/target/loongarch/internals.h b/target/loongarch/internals.h
index 69183e8bb5..4b1bcd7c0f 100644
--- a/target/loongarch/internals.h
+++ b/target/loongarch/internals.h
@@ -32,6 +32,12 @@ extern const VMStateDescription vmstate_loongarch_cpu;
 
 void loongarch_cpu_set_irq(void *opaque, int irq, int level);
 
+void loongarch_constant_timer_cb(void *opaque);
+uint64_t cpu_loongarch_get_constant_timer_counter(LoongArchCPU *cpu);
+uint64_t cpu_loongarch_get_constant_timer_ticks(LoongArchCPU *cpu);
+void cpu_loongarch_store_constant_timer_config(LoongArchCPU *cpu,
+                                               uint64_t value);
+
 bool loongarch_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
                             MMUAccessType access_type, int mmu_idx,
                             bool probe, uintptr_t retaddr);
diff --git a/target/loongarch/meson.build b/target/loongarch/meson.build
index 435cc75999..04e15ba1e3 100644
--- a/target/loongarch/meson.build
+++ b/target/loongarch/meson.build
@@ -18,6 +18,7 @@ loongarch_softmmu_ss = ss.source_set()
 loongarch_softmmu_ss.add(files(
   'machine.c',
   'tlb_helper.c',
+  'constant_timer.c',
 ))
 
 loongarch_ss.add_all(when: 'CONFIG_TCG', if_true: [loongarch_tcg_ss])

From 5b1dedfe848b61521aa5b46b81a4cc676e9e7c1b Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:15 +0800
Subject: [PATCH 832/935] target/loongarch: Add LoongArch CSR instruction

This includes:
- CSRRD
- CSRWR
- CSRXCHG

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-26-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/csr_helper.c                 |  87 ++++++
 target/loongarch/disas.c                      | 101 +++++++
 target/loongarch/helper.h                     |   8 +
 .../insn_trans/trans_privileged.c.inc         | 264 ++++++++++++++++++
 target/loongarch/insns.decode                 |  13 +
 target/loongarch/meson.build                  |   1 +
 target/loongarch/translate.c                  |  11 +-
 7 files changed, 484 insertions(+), 1 deletion(-)
 create mode 100644 target/loongarch/csr_helper.c
 create mode 100644 target/loongarch/insn_trans/trans_privileged.c.inc

diff --git a/target/loongarch/csr_helper.c b/target/loongarch/csr_helper.c
new file mode 100644
index 0000000000..24a9389364
--- /dev/null
+++ b/target/loongarch/csr_helper.c
@@ -0,0 +1,87 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * LoongArch emulation helpers for CSRs
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/main-loop.h"
+#include "cpu.h"
+#include "internals.h"
+#include "qemu/host-utils.h"
+#include "exec/helper-proto.h"
+#include "exec/exec-all.h"
+#include "exec/cpu_ldst.h"
+#include "hw/irq.h"
+#include "cpu-csr.h"
+#include "tcg/tcg-ldst.h"
+
+target_ulong helper_csrrd_pgd(CPULoongArchState *env)
+{
+    int64_t v;
+
+    if (env->CSR_TLBRERA & 0x1) {
+        v = env->CSR_TLBRBADV;
+    } else {
+        v = env->CSR_BADV;
+    }
+
+    if ((v >> 63) & 0x1) {
+        v = env->CSR_PGDH;
+    } else {
+        v = env->CSR_PGDL;
+    }
+
+    return v;
+}
+
+target_ulong helper_csrrd_tval(CPULoongArchState *env)
+{
+    LoongArchCPU *cpu = env_archcpu(env);
+
+    return cpu_loongarch_get_constant_timer_ticks(cpu);
+}
+
+target_ulong helper_csrwr_estat(CPULoongArchState *env, target_ulong val)
+{
+    int64_t old_v = env->CSR_ESTAT;
+
+    /* Only IS[1:0] can be written */
+    env->CSR_ESTAT = deposit64(env->CSR_ESTAT, 0, 2, val);
+
+    return old_v;
+}
+
+target_ulong helper_csrwr_asid(CPULoongArchState *env, target_ulong val)
+{
+    int64_t old_v = env->CSR_ASID;
+
+    /* Only ASID filed of CSR_ASID can be written */
+    env->CSR_ASID = deposit64(env->CSR_ASID, 0, 10, val);
+    if (old_v != env->CSR_ASID) {
+        tlb_flush(env_cpu(env));
+    }
+    return old_v;
+}
+
+target_ulong helper_csrwr_tcfg(CPULoongArchState *env, target_ulong val)
+{
+    LoongArchCPU *cpu = env_archcpu(env);
+    int64_t old_v = env->CSR_TCFG;
+
+    cpu_loongarch_store_constant_timer_config(cpu, val);
+
+    return old_v;
+}
+
+target_ulong helper_csrwr_ticlr(CPULoongArchState *env, target_ulong val)
+{
+    LoongArchCPU *cpu = env_archcpu(env);
+    int64_t old_v = 0;
+
+    if (val & 0x1) {
+        loongarch_cpu_set_irq(cpu, IRQ_TIMER, 0);
+    }
+    return old_v;
+}
diff --git a/target/loongarch/disas.c b/target/loongarch/disas.c
index 9454ebb8e9..11a704ff7c 100644
--- a/target/loongarch/disas.c
+++ b/target/loongarch/disas.c
@@ -8,6 +8,7 @@
 #include "qemu/osdep.h"
 #include "disas/dis-asm.h"
 #include "qemu/bitops.h"
+#include "cpu-csr.h"
 
 typedef struct {
     disassemble_info *info;
@@ -25,6 +26,90 @@ static inline int shl_2(DisasContext *ctx, int x)
     return x << 2;
 }
 
+#define CSR_NAME(REG) \
+    [LOONGARCH_CSR_##REG] = (#REG)
+
+static const char * const csr_names[] = {
+    CSR_NAME(CRMD),
+    CSR_NAME(PRMD),
+    CSR_NAME(EUEN),
+    CSR_NAME(MISC),
+    CSR_NAME(ECFG),
+    CSR_NAME(ESTAT),
+    CSR_NAME(ERA),
+    CSR_NAME(BADV),
+    CSR_NAME(BADI),
+    CSR_NAME(EENTRY),
+    CSR_NAME(TLBIDX),
+    CSR_NAME(TLBEHI),
+    CSR_NAME(TLBELO0),
+    CSR_NAME(TLBELO1),
+    CSR_NAME(ASID),
+    CSR_NAME(PGDL),
+    CSR_NAME(PGDH),
+    CSR_NAME(PGD),
+    CSR_NAME(PWCL),
+    CSR_NAME(PWCH),
+    CSR_NAME(STLBPS),
+    CSR_NAME(RVACFG),
+    CSR_NAME(CPUID),
+    CSR_NAME(PRCFG1),
+    CSR_NAME(PRCFG2),
+    CSR_NAME(PRCFG3),
+    CSR_NAME(SAVE(0)),
+    CSR_NAME(SAVE(1)),
+    CSR_NAME(SAVE(2)),
+    CSR_NAME(SAVE(3)),
+    CSR_NAME(SAVE(4)),
+    CSR_NAME(SAVE(5)),
+    CSR_NAME(SAVE(6)),
+    CSR_NAME(SAVE(7)),
+    CSR_NAME(SAVE(8)),
+    CSR_NAME(SAVE(9)),
+    CSR_NAME(SAVE(10)),
+    CSR_NAME(SAVE(11)),
+    CSR_NAME(SAVE(12)),
+    CSR_NAME(SAVE(13)),
+    CSR_NAME(SAVE(14)),
+    CSR_NAME(SAVE(15)),
+    CSR_NAME(TID),
+    CSR_NAME(TCFG),
+    CSR_NAME(TVAL),
+    CSR_NAME(CNTC),
+    CSR_NAME(TICLR),
+    CSR_NAME(LLBCTL),
+    CSR_NAME(IMPCTL1),
+    CSR_NAME(IMPCTL2),
+    CSR_NAME(TLBRENTRY),
+    CSR_NAME(TLBRBADV),
+    CSR_NAME(TLBRERA),
+    CSR_NAME(TLBRSAVE),
+    CSR_NAME(TLBRELO0),
+    CSR_NAME(TLBRELO1),
+    CSR_NAME(TLBREHI),
+    CSR_NAME(TLBRPRMD),
+    CSR_NAME(MERRCTL),
+    CSR_NAME(MERRINFO1),
+    CSR_NAME(MERRINFO2),
+    CSR_NAME(MERRENTRY),
+    CSR_NAME(MERRERA),
+    CSR_NAME(MERRSAVE),
+    CSR_NAME(CTAG),
+    CSR_NAME(DMW(0)),
+    CSR_NAME(DMW(1)),
+    CSR_NAME(DMW(2)),
+    CSR_NAME(DMW(3)),
+    CSR_NAME(DBG),
+    CSR_NAME(DERA),
+    CSR_NAME(DSAVE),
+};
+
+static const char *get_csr_name(unsigned num)
+{
+    return ((num < ARRAY_SIZE(csr_names)) && (csr_names[num] != NULL)) ?
+           csr_names[num] : "Undefined CSR";
+}
+
 #define output(C, INSN, FMT, ...)                                   \
 {                                                                   \
     (C)->info->fprintf_func((C)->info->stream, "%08x   %-9s\t" FMT, \
@@ -205,6 +290,19 @@ static void output_rr_offs(DisasContext *ctx, arg_rr_offs *a,
            a->rd, a->offs, ctx->pc + a->offs);
 }
 
+static void output_r_csr(DisasContext *ctx, arg_r_csr *a,
+                         const char *mnemonic)
+{
+    output(ctx, mnemonic, "r%d, %d # %s", a->rd, a->csr, get_csr_name(a->csr));
+}
+
+static void output_rr_csr(DisasContext *ctx, arg_rr_csr *a,
+                          const char *mnemonic)
+{
+    output(ctx, mnemonic, "r%d, r%d, %d # %s",
+           a->rd, a->rj, a->csr, get_csr_name(a->csr));
+}
+
 #define INSN(insn, type)                                    \
 static bool trans_##insn(DisasContext *ctx, arg_##type * a) \
 {                                                           \
@@ -514,6 +612,9 @@ INSN(blt,          rr_offs)
 INSN(bge,          rr_offs)
 INSN(bltu,         rr_offs)
 INSN(bgeu,         rr_offs)
+INSN(csrrd,        r_csr)
+INSN(csrwr,        r_csr)
+INSN(csrxchg,      rr_csr)
 
 #define output_fcmp(C, PREFIX, SUFFIX)                                         \
 {                                                                              \
diff --git a/target/loongarch/helper.h b/target/loongarch/helper.h
index da1a2bced7..5a6754eb65 100644
--- a/target/loongarch/helper.h
+++ b/target/loongarch/helper.h
@@ -92,3 +92,11 @@ DEF_HELPER_2(frint_s, i64, env, i64)
 DEF_HELPER_2(frint_d, i64, env, i64)
 
 DEF_HELPER_FLAGS_2(set_rounding_mode, TCG_CALL_NO_RWG, void, env, i32)
+
+/* CSRs helper */
+DEF_HELPER_1(csrrd_pgd, i64, env)
+DEF_HELPER_1(csrrd_tval, i64, env)
+DEF_HELPER_2(csrwr_estat, i64, env, tl)
+DEF_HELPER_2(csrwr_asid, i64, env, tl)
+DEF_HELPER_2(csrwr_tcfg, i64, env, tl)
+DEF_HELPER_2(csrwr_ticlr, i64, env, tl)
diff --git a/target/loongarch/insn_trans/trans_privileged.c.inc b/target/loongarch/insn_trans/trans_privileged.c.inc
new file mode 100644
index 0000000000..f984e3f68e
--- /dev/null
+++ b/target/loongarch/insn_trans/trans_privileged.c.inc
@@ -0,0 +1,264 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ *
+ * LoongArch translation routines for the privileged instructions.
+ */
+
+#include "cpu-csr.h"
+
+typedef void (*GenCSRRead)(TCGv dest, TCGv_ptr env);
+typedef void (*GenCSRWrite)(TCGv dest, TCGv_ptr env, TCGv src);
+
+typedef struct {
+    int offset;
+    int flags;
+    GenCSRRead readfn;
+    GenCSRWrite writefn;
+} CSRInfo;
+
+enum {
+    CSRFL_READONLY = (1 << 0),
+    CSRFL_EXITTB   = (1 << 1),
+    CSRFL_IO       = (1 << 2),
+};
+
+#define CSR_OFF_FUNCS(NAME, FL, RD, WR)                    \
+    [LOONGARCH_CSR_##NAME] = {                             \
+        .offset = offsetof(CPULoongArchState, CSR_##NAME), \
+        .flags = FL, .readfn = RD, .writefn = WR           \
+    }
+
+#define CSR_OFF_ARRAY(NAME, N)                                \
+    [LOONGARCH_CSR_##NAME(N)] = {                             \
+        .offset = offsetof(CPULoongArchState, CSR_##NAME[N]), \
+        .flags = 0, .readfn = NULL, .writefn = NULL           \
+    }
+
+#define CSR_OFF_FLAGS(NAME, FL) \
+    CSR_OFF_FUNCS(NAME, FL, NULL, NULL)
+
+#define CSR_OFF(NAME) \
+    CSR_OFF_FLAGS(NAME, 0)
+
+static const CSRInfo csr_info[] = {
+    CSR_OFF_FLAGS(CRMD, CSRFL_EXITTB),
+    CSR_OFF(PRMD),
+    CSR_OFF_FLAGS(EUEN, CSRFL_EXITTB),
+    CSR_OFF_FLAGS(MISC, CSRFL_READONLY),
+    CSR_OFF(ECFG),
+    CSR_OFF_FUNCS(ESTAT, CSRFL_EXITTB, NULL, gen_helper_csrwr_estat),
+    CSR_OFF(ERA),
+    CSR_OFF(BADV),
+    CSR_OFF_FLAGS(BADI, CSRFL_READONLY),
+    CSR_OFF(EENTRY),
+    CSR_OFF(TLBIDX),
+    CSR_OFF(TLBEHI),
+    CSR_OFF(TLBELO0),
+    CSR_OFF(TLBELO1),
+    CSR_OFF_FUNCS(ASID, CSRFL_EXITTB, NULL, gen_helper_csrwr_asid),
+    CSR_OFF(PGDL),
+    CSR_OFF(PGDH),
+    CSR_OFF_FUNCS(PGD, CSRFL_READONLY, gen_helper_csrrd_pgd, NULL),
+    CSR_OFF(PWCL),
+    CSR_OFF(PWCH),
+    CSR_OFF(STLBPS),
+    CSR_OFF(RVACFG),
+    [LOONGARCH_CSR_CPUID] = {
+        .offset = (int)offsetof(CPUState, cpu_index)
+                  - (int)offsetof(LoongArchCPU, env),
+        .flags = CSRFL_READONLY,
+        .readfn = NULL,
+        .writefn = NULL
+    },
+    CSR_OFF_FLAGS(PRCFG1, CSRFL_READONLY),
+    CSR_OFF_FLAGS(PRCFG2, CSRFL_READONLY),
+    CSR_OFF_FLAGS(PRCFG3, CSRFL_READONLY),
+    CSR_OFF_ARRAY(SAVE, 0),
+    CSR_OFF_ARRAY(SAVE, 1),
+    CSR_OFF_ARRAY(SAVE, 2),
+    CSR_OFF_ARRAY(SAVE, 3),
+    CSR_OFF_ARRAY(SAVE, 4),
+    CSR_OFF_ARRAY(SAVE, 5),
+    CSR_OFF_ARRAY(SAVE, 6),
+    CSR_OFF_ARRAY(SAVE, 7),
+    CSR_OFF_ARRAY(SAVE, 8),
+    CSR_OFF_ARRAY(SAVE, 9),
+    CSR_OFF_ARRAY(SAVE, 10),
+    CSR_OFF_ARRAY(SAVE, 11),
+    CSR_OFF_ARRAY(SAVE, 12),
+    CSR_OFF_ARRAY(SAVE, 13),
+    CSR_OFF_ARRAY(SAVE, 14),
+    CSR_OFF_ARRAY(SAVE, 15),
+    CSR_OFF(TID),
+    CSR_OFF_FUNCS(TCFG, CSRFL_IO, NULL, gen_helper_csrwr_tcfg),
+    CSR_OFF_FUNCS(TVAL, CSRFL_READONLY | CSRFL_IO, gen_helper_csrrd_tval, NULL),
+    CSR_OFF(CNTC),
+    CSR_OFF_FUNCS(TICLR, CSRFL_IO, NULL, gen_helper_csrwr_ticlr),
+    CSR_OFF(LLBCTL),
+    CSR_OFF(IMPCTL1),
+    CSR_OFF(IMPCTL2),
+    CSR_OFF(TLBRENTRY),
+    CSR_OFF(TLBRBADV),
+    CSR_OFF(TLBRERA),
+    CSR_OFF(TLBRSAVE),
+    CSR_OFF(TLBRELO0),
+    CSR_OFF(TLBRELO1),
+    CSR_OFF(TLBREHI),
+    CSR_OFF(TLBRPRMD),
+    CSR_OFF(MERRCTL),
+    CSR_OFF(MERRINFO1),
+    CSR_OFF(MERRINFO2),
+    CSR_OFF(MERRENTRY),
+    CSR_OFF(MERRERA),
+    CSR_OFF(MERRSAVE),
+    CSR_OFF(CTAG),
+    CSR_OFF_ARRAY(DMW, 0),
+    CSR_OFF_ARRAY(DMW, 1),
+    CSR_OFF_ARRAY(DMW, 2),
+    CSR_OFF_ARRAY(DMW, 3),
+    CSR_OFF(DBG),
+    CSR_OFF(DERA),
+    CSR_OFF(DSAVE),
+};
+
+static bool check_plv(DisasContext *ctx)
+{
+    if (ctx->base.tb->flags == MMU_USER_IDX) {
+        generate_exception(ctx, EXCCODE_IPE);
+        return true;
+    }
+    return false;
+}
+
+static const CSRInfo *get_csr(unsigned csr_num)
+{
+    const CSRInfo *csr;
+
+    if (csr_num >= ARRAY_SIZE(csr_info)) {
+        return NULL;
+    }
+    csr = &csr_info[csr_num];
+    if (csr->offset == 0) {
+        return NULL;
+    }
+    return csr;
+}
+
+static bool check_csr_flags(DisasContext *ctx, const CSRInfo *csr, bool write)
+{
+    if ((csr->flags & CSRFL_READONLY) && write) {
+        return false;
+    }
+    if ((csr->flags & CSRFL_IO) &&
+        (tb_cflags(ctx->base.tb) & CF_USE_ICOUNT)) {
+        gen_io_start();
+        ctx->base.is_jmp = DISAS_EXIT_UPDATE;
+    } else if ((csr->flags & CSRFL_EXITTB) && write) {
+        ctx->base.is_jmp = DISAS_EXIT_UPDATE;
+    }
+    return true;
+}
+
+static bool trans_csrrd(DisasContext *ctx, arg_csrrd *a)
+{
+    TCGv dest;
+    const CSRInfo *csr;
+
+    if (check_plv(ctx)) {
+        return false;
+    }
+    csr = get_csr(a->csr);
+    if (csr == NULL) {
+        /* CSR is undefined: read as 0. */
+        dest = tcg_constant_tl(0);
+    } else {
+        check_csr_flags(ctx, csr, false);
+        dest = gpr_dst(ctx, a->rd, EXT_NONE);
+        if (csr->readfn) {
+            csr->readfn(dest, cpu_env);
+        } else {
+            tcg_gen_ld_tl(dest, cpu_env, csr->offset);
+        }
+    }
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+    return true;
+}
+
+static bool trans_csrwr(DisasContext *ctx, arg_csrwr *a)
+{
+    TCGv dest, src1;
+    const CSRInfo *csr;
+
+    if (check_plv(ctx)) {
+        return false;
+    }
+    csr = get_csr(a->csr);
+    if (csr == NULL) {
+        /* CSR is undefined: write ignored, read old_value as 0. */
+        gen_set_gpr(a->rd, tcg_constant_tl(0), EXT_NONE);
+        return true;
+    }
+    if (!check_csr_flags(ctx, csr, true)) {
+        /* CSR is readonly: trap. */
+        return false;
+    }
+    src1 = gpr_src(ctx, a->rd, EXT_NONE);
+    if (csr->writefn) {
+        dest = gpr_dst(ctx, a->rd, EXT_NONE);
+        csr->writefn(dest, cpu_env, src1);
+    } else {
+        dest = temp_new(ctx);
+        tcg_gen_ld_tl(dest, cpu_env, csr->offset);
+        tcg_gen_st_tl(src1, cpu_env, csr->offset);
+    }
+    gen_set_gpr(a->rd, dest, EXT_NONE);
+    return true;
+}
+
+static bool trans_csrxchg(DisasContext *ctx, arg_csrxchg *a)
+{
+    TCGv src1, mask, oldv, newv, temp;
+    const CSRInfo *csr;
+
+    if (check_plv(ctx)) {
+        return false;
+    }
+    csr = get_csr(a->csr);
+    if (csr == NULL) {
+        /* CSR is undefined: write ignored, read old_value as 0. */
+        gen_set_gpr(a->rd, tcg_constant_tl(0), EXT_NONE);
+        return true;
+    }
+
+    if (!check_csr_flags(ctx, csr, true)) {
+        /* CSR is readonly: trap. */
+        return false;
+    }
+
+    /* So far only readonly csrs have readfn. */
+    assert(csr->readfn == NULL);
+
+    src1 = gpr_src(ctx, a->rd, EXT_NONE);
+    mask = gpr_src(ctx, a->rj, EXT_NONE);
+    oldv = tcg_temp_new();
+    newv = tcg_temp_new();
+    temp = tcg_temp_new();
+
+    tcg_gen_ld_tl(oldv, cpu_env, csr->offset);
+    tcg_gen_and_tl(newv, src1, mask);
+    tcg_gen_andc_tl(temp, oldv, mask);
+    tcg_gen_or_tl(newv, newv, temp);
+
+    if (csr->writefn) {
+        csr->writefn(oldv, cpu_env, newv);
+    } else {
+        tcg_gen_st_tl(newv, cpu_env, csr->offset);
+    }
+    gen_set_gpr(a->rd, oldv, EXT_NONE);
+
+    tcg_temp_free(temp);
+    tcg_temp_free(newv);
+    tcg_temp_free(oldv);
+    return true;
+}
diff --git a/target/loongarch/insns.decode b/target/loongarch/insns.decode
index 9b293dfdf9..43005ca283 100644
--- a/target/loongarch/insns.decode
+++ b/target/loongarch/insns.decode
@@ -45,6 +45,8 @@
 &c_offs       cj offs
 &offs         offs
 &rr_offs      rj rd offs
+&r_csr        rd csr
+&rr_csr       rd rj csr
 
 #
 # Formats
@@ -85,6 +87,8 @@
 @c_offs21      .... .. ................ .. cj:3 .....    &c_offs      offs=%offs21
 @offs26            .... .. ..........................    &offs        offs=%offs26
 @rr_offs16         .... .. ................ rj:5 rd:5    &rr_offs     offs=%offs16
+@r_csr                    .... .... csr:14 ..... rd:5    &r_csr
+@rr_csr                    .... .... csr:14 rj:5 rd:5    &rr_csr
 
 #
 # Fixed point arithmetic operation instruction
@@ -437,3 +441,12 @@ blt             0110 00 ................ ..... .....     @rr_offs16
 bge             0110 01 ................ ..... .....     @rr_offs16
 bltu            0110 10 ................ ..... .....     @rr_offs16
 bgeu            0110 11 ................ ..... .....     @rr_offs16
+
+#
+# Core instructions
+#
+{
+  csrrd             0000 0100 .............. 00000 .....     @r_csr
+  csrwr             0000 0100 .............. 00001 .....     @r_csr
+  csrxchg           0000 0100 .............. ..... .....     @rr_csr
+}
diff --git a/target/loongarch/meson.build b/target/loongarch/meson.build
index 04e15ba1e3..d11829a6cc 100644
--- a/target/loongarch/meson.build
+++ b/target/loongarch/meson.build
@@ -19,6 +19,7 @@ loongarch_softmmu_ss.add(files(
   'machine.c',
   'tlb_helper.c',
   'constant_timer.c',
+  'csr_helper.c',
 ))
 
 loongarch_ss.add_all(when: 'CONFIG_TCG', if_true: [loongarch_tcg_ss])
diff --git a/target/loongarch/translate.c b/target/loongarch/translate.c
index 0ad6b7b2ee..0f098924a3 100644
--- a/target/loongarch/translate.c
+++ b/target/loongarch/translate.c
@@ -25,7 +25,9 @@ static TCGv cpu_lladdr, cpu_llval;
 TCGv_i32 cpu_fcsr0;
 TCGv_i64 cpu_fpr[32];
 
-#define DISAS_STOP       DISAS_TARGET_0
+#define DISAS_STOP        DISAS_TARGET_0
+#define DISAS_EXIT        DISAS_TARGET_1
+#define DISAS_EXIT_UPDATE DISAS_TARGET_2
 
 static inline int plus_1(DisasContext *ctx, int x)
 {
@@ -172,6 +174,7 @@ static void gen_set_gpr(int reg_num, TCGv t, DisasExtend dst_ext)
 #include "insn_trans/trans_fmov.c.inc"
 #include "insn_trans/trans_fmemory.c.inc"
 #include "insn_trans/trans_branch.c.inc"
+#include "insn_trans/trans_privileged.c.inc"
 
 static void loongarch_tr_translate_insn(DisasContextBase *dcbase, CPUState *cs)
 {
@@ -210,6 +213,12 @@ static void loongarch_tr_tb_stop(DisasContextBase *dcbase, CPUState *cs)
         break;
     case DISAS_NORETURN:
         break;
+    case DISAS_EXIT_UPDATE:
+        tcg_gen_movi_tl(cpu_pc, ctx->base.pc_next);
+        QEMU_FALLTHROUGH;
+    case DISAS_EXIT:
+        tcg_gen_exit_tb(NULL, 0);
+        break;
     default:
         g_assert_not_reached();
     }

From f84a2aacf5d1679b1d1cceabb6006e02864232f3 Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:16 +0800
Subject: [PATCH 833/935] target/loongarch: Add LoongArch IOCSR instruction

This includes:
- IOCSR{RD/WR}.{B/H/W/D}

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-27-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/cpu.c                        | 44 ++++++++++++
 target/loongarch/cpu.h                        | 25 +++++++
 target/loongarch/disas.c                      |  8 +++
 target/loongarch/helper.h                     |  8 +++
 .../insn_trans/trans_privileged.c.inc         | 35 ++++++++++
 target/loongarch/insns.decode                 |  9 +++
 target/loongarch/iocsr_helper.c               | 67 +++++++++++++++++++
 target/loongarch/meson.build                  |  1 +
 8 files changed, 197 insertions(+)
 create mode 100644 target/loongarch/iocsr_helper.c

diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c
index 5ec0ba1cfa..b846bd1a8b 100644
--- a/target/loongarch/cpu.c
+++ b/target/loongarch/cpu.c
@@ -17,6 +17,8 @@
 #include "internals.h"
 #include "fpu/softfloat-helpers.h"
 #include "cpu-csr.h"
+#include "sysemu/reset.h"
+#include "hw/loader.h"
 
 const char * const regnames[32] = {
     "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7",
@@ -489,14 +491,56 @@ static void loongarch_cpu_realizefn(DeviceState *dev, Error **errp)
     lacc->parent_realize(dev, errp);
 }
 
+static void loongarch_qemu_write(void *opaque, hwaddr addr,
+                                 uint64_t val, unsigned size)
+{
+}
+
+static uint64_t loongarch_qemu_read(void *opaque, hwaddr addr, unsigned size)
+{
+    switch (addr) {
+    case FEATURE_REG:
+        return 1ULL << IOCSRF_MSI | 1ULL << IOCSRF_EXTIOI |
+               1ULL << IOCSRF_CSRIPI;
+    case VENDOR_REG:
+        return 0x6e6f73676e6f6f4cULL; /* "Loongson" */
+    case CPUNAME_REG:
+        return 0x303030354133ULL;     /* "3A5000" */
+    case MISC_FUNC_REG:
+        return 1ULL << IOCSRM_EXTIOI_EN;
+    }
+    return 0ULL;
+}
+
+static const MemoryRegionOps loongarch_qemu_ops = {
+    .read = loongarch_qemu_read,
+    .write = loongarch_qemu_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 4,
+        .max_access_size = 8,
+    },
+    .impl = {
+        .min_access_size = 8,
+        .max_access_size = 8,
+    },
+};
+
 static void loongarch_cpu_init(Object *obj)
 {
     LoongArchCPU *cpu = LOONGARCH_CPU(obj);
+    CPULoongArchState *env = &cpu->env;
 
     cpu_set_cpustate_pointers(cpu);
     qdev_init_gpio_in(DEVICE(cpu), loongarch_cpu_set_irq, N_IRQS);
     timer_init_ns(&cpu->timer, QEMU_CLOCK_VIRTUAL,
                   &loongarch_constant_timer_cb, cpu);
+    memory_region_init_io(&env->system_iocsr, OBJECT(cpu), NULL,
+                      env, "iocsr", UINT64_MAX);
+    address_space_init(&env->address_space_iocsr, &env->system_iocsr, "IOCSR");
+    memory_region_init_io(&env->iocsr_mem, OBJECT(cpu), &loongarch_qemu_ops,
+                          NULL, "iocsr_misc", 0x428);
+    memory_region_add_subregion(&env->system_iocsr, 0, &env->iocsr_mem);
 }
 
 static ObjectClass *loongarch_cpu_class_by_name(const char *cpu_model)
diff --git a/target/loongarch/cpu.h b/target/loongarch/cpu.h
index 2081902f2e..03cc96ee9b 100644
--- a/target/loongarch/cpu.h
+++ b/target/loongarch/cpu.h
@@ -12,6 +12,27 @@
 #include "fpu/softfloat-types.h"
 #include "hw/registerfields.h"
 #include "qemu/timer.h"
+#include "exec/memory.h"
+#include "hw/sysbus.h"
+
+#define IOCSRF_TEMP             0
+#define IOCSRF_NODECNT          1
+#define IOCSRF_MSI              2
+#define IOCSRF_EXTIOI           3
+#define IOCSRF_CSRIPI           4
+#define IOCSRF_FREQCSR          5
+#define IOCSRF_FREQSCALE        6
+#define IOCSRF_DVFSV1           7
+#define IOCSRF_GMOD             9
+#define IOCSRF_VM               11
+
+#define FEATURE_REG             0x8
+#define VENDOR_REG              0x10
+#define CPUNAME_REG             0x20
+#define MISC_FUNC_REG           0x420
+#define IOCSRM_EXTIOI_EN        48
+
+#define IOCSR_MEM_SIZE          0x428
 
 #define TCG_GUEST_DEFAULT_MO (0)
 
@@ -283,6 +304,10 @@ typedef struct CPUArchState {
     uint64_t CSR_DSAVE;
 
     LoongArchTLB  tlb[LOONGARCH_TLB_MAX];
+
+    AddressSpace address_space_iocsr;
+    MemoryRegion system_iocsr;
+    MemoryRegion iocsr_mem;
 } CPULoongArchState;
 
 /**
diff --git a/target/loongarch/disas.c b/target/loongarch/disas.c
index 11a704ff7c..ea26aea728 100644
--- a/target/loongarch/disas.c
+++ b/target/loongarch/disas.c
@@ -615,6 +615,14 @@ INSN(bgeu,         rr_offs)
 INSN(csrrd,        r_csr)
 INSN(csrwr,        r_csr)
 INSN(csrxchg,      rr_csr)
+INSN(iocsrrd_b,    rr)
+INSN(iocsrrd_h,    rr)
+INSN(iocsrrd_w,    rr)
+INSN(iocsrrd_d,    rr)
+INSN(iocsrwr_b,    rr)
+INSN(iocsrwr_h,    rr)
+INSN(iocsrwr_w,    rr)
+INSN(iocsrwr_d,    rr)
 
 #define output_fcmp(C, PREFIX, SUFFIX)                                         \
 {                                                                              \
diff --git a/target/loongarch/helper.h b/target/loongarch/helper.h
index 5a6754eb65..4664a02dcf 100644
--- a/target/loongarch/helper.h
+++ b/target/loongarch/helper.h
@@ -100,3 +100,11 @@ DEF_HELPER_2(csrwr_estat, i64, env, tl)
 DEF_HELPER_2(csrwr_asid, i64, env, tl)
 DEF_HELPER_2(csrwr_tcfg, i64, env, tl)
 DEF_HELPER_2(csrwr_ticlr, i64, env, tl)
+DEF_HELPER_2(iocsrrd_b, i64, env, tl)
+DEF_HELPER_2(iocsrrd_h, i64, env, tl)
+DEF_HELPER_2(iocsrrd_w, i64, env, tl)
+DEF_HELPER_2(iocsrrd_d, i64, env, tl)
+DEF_HELPER_3(iocsrwr_b, void, env, tl, tl)
+DEF_HELPER_3(iocsrwr_h, void, env, tl, tl)
+DEF_HELPER_3(iocsrwr_w, void, env, tl, tl)
+DEF_HELPER_3(iocsrwr_d, void, env, tl, tl)
diff --git a/target/loongarch/insn_trans/trans_privileged.c.inc b/target/loongarch/insn_trans/trans_privileged.c.inc
index f984e3f68e..3e50cd3ac6 100644
--- a/target/loongarch/insn_trans/trans_privileged.c.inc
+++ b/target/loongarch/insn_trans/trans_privileged.c.inc
@@ -262,3 +262,38 @@ static bool trans_csrxchg(DisasContext *ctx, arg_csrxchg *a)
     tcg_temp_free(oldv);
     return true;
 }
+
+static bool gen_iocsrrd(DisasContext *ctx, arg_rr *a,
+                        void (*func)(TCGv, TCGv_ptr, TCGv))
+{
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+
+    if (check_plv(ctx)) {
+        return false;
+    }
+    func(dest, cpu_env, src1);
+    return true;
+}
+
+static bool gen_iocsrwr(DisasContext *ctx, arg_rr *a,
+                        void (*func)(TCGv_ptr, TCGv, TCGv))
+{
+    TCGv val = gpr_src(ctx, a->rd, EXT_NONE);
+    TCGv addr = gpr_src(ctx, a->rj, EXT_NONE);
+
+    if (check_plv(ctx)) {
+        return false;
+    }
+    func(cpu_env, addr, val);
+    return true;
+}
+
+TRANS(iocsrrd_b, gen_iocsrrd, gen_helper_iocsrrd_b)
+TRANS(iocsrrd_h, gen_iocsrrd, gen_helper_iocsrrd_h)
+TRANS(iocsrrd_w, gen_iocsrrd, gen_helper_iocsrrd_w)
+TRANS(iocsrrd_d, gen_iocsrrd, gen_helper_iocsrrd_d)
+TRANS(iocsrwr_b, gen_iocsrwr, gen_helper_iocsrwr_b)
+TRANS(iocsrwr_h, gen_iocsrwr, gen_helper_iocsrwr_h)
+TRANS(iocsrwr_w, gen_iocsrwr, gen_helper_iocsrwr_w)
+TRANS(iocsrwr_d, gen_iocsrwr, gen_helper_iocsrwr_d)
diff --git a/target/loongarch/insns.decode b/target/loongarch/insns.decode
index 43005ca283..2b436d3cd6 100644
--- a/target/loongarch/insns.decode
+++ b/target/loongarch/insns.decode
@@ -450,3 +450,12 @@ bgeu            0110 11 ................ ..... .....     @rr_offs16
   csrwr             0000 0100 .............. 00001 .....     @r_csr
   csrxchg           0000 0100 .............. ..... .....     @rr_csr
 }
+
+iocsrrd_b        0000 01100100 10000 00000 ..... .....    @rr
+iocsrrd_h        0000 01100100 10000 00001 ..... .....    @rr
+iocsrrd_w        0000 01100100 10000 00010 ..... .....    @rr
+iocsrrd_d        0000 01100100 10000 00011 ..... .....    @rr
+iocsrwr_b        0000 01100100 10000 00100 ..... .....    @rr
+iocsrwr_h        0000 01100100 10000 00101 ..... .....    @rr
+iocsrwr_w        0000 01100100 10000 00110 ..... .....    @rr
+iocsrwr_d        0000 01100100 10000 00111 ..... .....    @rr
diff --git a/target/loongarch/iocsr_helper.c b/target/loongarch/iocsr_helper.c
new file mode 100644
index 0000000000..0e9c537dc7
--- /dev/null
+++ b/target/loongarch/iocsr_helper.c
@@ -0,0 +1,67 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ *
+ * Helpers for IOCSR reads/writes
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/main-loop.h"
+#include "cpu.h"
+#include "qemu/host-utils.h"
+#include "exec/helper-proto.h"
+#include "exec/exec-all.h"
+#include "exec/cpu_ldst.h"
+#include "tcg/tcg-ldst.h"
+
+uint64_t helper_iocsrrd_b(CPULoongArchState *env, target_ulong r_addr)
+{
+    return address_space_ldub(&env->address_space_iocsr, r_addr,
+                              MEMTXATTRS_UNSPECIFIED, NULL);
+}
+
+uint64_t helper_iocsrrd_h(CPULoongArchState *env, target_ulong r_addr)
+{
+    return address_space_lduw(&env->address_space_iocsr, r_addr,
+                              MEMTXATTRS_UNSPECIFIED, NULL);
+}
+
+uint64_t helper_iocsrrd_w(CPULoongArchState *env, target_ulong r_addr)
+{
+    return address_space_ldl(&env->address_space_iocsr, r_addr,
+                             MEMTXATTRS_UNSPECIFIED, NULL);
+}
+
+uint64_t helper_iocsrrd_d(CPULoongArchState *env, target_ulong r_addr)
+{
+    return address_space_ldq(&env->address_space_iocsr, r_addr,
+                             MEMTXATTRS_UNSPECIFIED, NULL);
+}
+
+void helper_iocsrwr_b(CPULoongArchState *env, target_ulong w_addr,
+                      target_ulong val)
+{
+    address_space_stb(&env->address_space_iocsr, w_addr,
+                      val, MEMTXATTRS_UNSPECIFIED, NULL);
+}
+
+void helper_iocsrwr_h(CPULoongArchState *env, target_ulong w_addr,
+                      target_ulong val)
+{
+    address_space_stw(&env->address_space_iocsr, w_addr,
+                      val, MEMTXATTRS_UNSPECIFIED, NULL);
+}
+
+void helper_iocsrwr_w(CPULoongArchState *env, target_ulong w_addr,
+                      target_ulong val)
+{
+    address_space_stl(&env->address_space_iocsr, w_addr,
+                      val, MEMTXATTRS_UNSPECIFIED, NULL);
+}
+
+void helper_iocsrwr_d(CPULoongArchState *env, target_ulong w_addr,
+                      target_ulong val)
+{
+    address_space_stq(&env->address_space_iocsr, w_addr,
+                      val, MEMTXATTRS_UNSPECIFIED, NULL);
+}
diff --git a/target/loongarch/meson.build b/target/loongarch/meson.build
index d11829a6cc..74e5f3b2a7 100644
--- a/target/loongarch/meson.build
+++ b/target/loongarch/meson.build
@@ -20,6 +20,7 @@ loongarch_softmmu_ss.add(files(
   'tlb_helper.c',
   'constant_timer.c',
   'csr_helper.c',
+  'iocsr_helper.c',
 ))
 
 loongarch_ss.add_all(when: 'CONFIG_TCG', if_true: [loongarch_tcg_ss])

From fcbbeb8ecd8b755e9244e12f2f0d7579350ab23e Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:17 +0800
Subject: [PATCH 834/935] target/loongarch: Add TLB instruction support

This includes:
- TLBSRCH
- TLBRD
- TLBWR
- TLBFILL
- TLBCLR
- TLBFLUSH
- INVTLB

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-28-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/disas.c                      |  18 +
 target/loongarch/helper.h                     |  13 +
 .../insn_trans/trans_privileged.c.inc         | 102 +++++
 target/loongarch/insns.decode                 |  11 +
 target/loongarch/tlb_helper.c                 | 355 ++++++++++++++++++
 5 files changed, 499 insertions(+)

diff --git a/target/loongarch/disas.c b/target/loongarch/disas.c
index ea26aea728..6a56607302 100644
--- a/target/loongarch/disas.c
+++ b/target/loongarch/disas.c
@@ -303,6 +303,17 @@ static void output_rr_csr(DisasContext *ctx, arg_rr_csr *a,
            a->rd, a->rj, a->csr, get_csr_name(a->csr));
 }
 
+static void output_empty(DisasContext *ctx, arg_empty *a,
+                         const char *mnemonic)
+{
+    output(ctx, mnemonic, "");
+}
+
+static void output_i_rr(DisasContext *ctx, arg_i_rr *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "%d, r%d, r%d", a->imm, a->rj, a->rk);
+}
+
 #define INSN(insn, type)                                    \
 static bool trans_##insn(DisasContext *ctx, arg_##type * a) \
 {                                                           \
@@ -623,6 +634,13 @@ INSN(iocsrwr_b,    rr)
 INSN(iocsrwr_h,    rr)
 INSN(iocsrwr_w,    rr)
 INSN(iocsrwr_d,    rr)
+INSN(tlbsrch,      empty)
+INSN(tlbrd,        empty)
+INSN(tlbwr,        empty)
+INSN(tlbfill,      empty)
+INSN(tlbclr,       empty)
+INSN(tlbflush,     empty)
+INSN(invtlb,       i_rr)
 
 #define output_fcmp(C, PREFIX, SUFFIX)                                         \
 {                                                                              \
diff --git a/target/loongarch/helper.h b/target/loongarch/helper.h
index 4664a02dcf..b092ca75fe 100644
--- a/target/loongarch/helper.h
+++ b/target/loongarch/helper.h
@@ -108,3 +108,16 @@ DEF_HELPER_3(iocsrwr_b, void, env, tl, tl)
 DEF_HELPER_3(iocsrwr_h, void, env, tl, tl)
 DEF_HELPER_3(iocsrwr_w, void, env, tl, tl)
 DEF_HELPER_3(iocsrwr_d, void, env, tl, tl)
+
+/* TLB helper */
+DEF_HELPER_1(tlbwr, void, env)
+DEF_HELPER_1(tlbfill, void, env)
+DEF_HELPER_1(tlbsrch, void, env)
+DEF_HELPER_1(tlbrd, void, env)
+DEF_HELPER_1(tlbclr, void, env)
+DEF_HELPER_1(tlbflush, void, env)
+DEF_HELPER_1(invtlb_all, void, env)
+DEF_HELPER_2(invtlb_all_g, void, env, i32)
+DEF_HELPER_2(invtlb_all_asid, void, env, tl)
+DEF_HELPER_3(invtlb_page_asid, void, env, tl, tl)
+DEF_HELPER_3(invtlb_page_asid_or_g, void, env, tl, tl)
diff --git a/target/loongarch/insn_trans/trans_privileged.c.inc b/target/loongarch/insn_trans/trans_privileged.c.inc
index 3e50cd3ac6..d47918785b 100644
--- a/target/loongarch/insn_trans/trans_privileged.c.inc
+++ b/target/loongarch/insn_trans/trans_privileged.c.inc
@@ -297,3 +297,105 @@ TRANS(iocsrwr_b, gen_iocsrwr, gen_helper_iocsrwr_b)
 TRANS(iocsrwr_h, gen_iocsrwr, gen_helper_iocsrwr_h)
 TRANS(iocsrwr_w, gen_iocsrwr, gen_helper_iocsrwr_w)
 TRANS(iocsrwr_d, gen_iocsrwr, gen_helper_iocsrwr_d)
+
+static void check_mmu_idx(DisasContext *ctx)
+{
+    if (ctx->mem_idx != MMU_DA_IDX) {
+        tcg_gen_movi_tl(cpu_pc, ctx->base.pc_next + 4);
+        ctx->base.is_jmp = DISAS_EXIT;
+    }
+}
+
+static bool trans_tlbsrch(DisasContext *ctx, arg_tlbsrch *a)
+{
+    if (check_plv(ctx)) {
+        return false;
+    }
+    gen_helper_tlbsrch(cpu_env);
+    return true;
+}
+
+static bool trans_tlbrd(DisasContext *ctx, arg_tlbrd *a)
+{
+    if (check_plv(ctx)) {
+        return false;
+    }
+    gen_helper_tlbrd(cpu_env);
+    return true;
+}
+
+static bool trans_tlbwr(DisasContext *ctx, arg_tlbwr *a)
+{
+    if (check_plv(ctx)) {
+        return false;
+    }
+    gen_helper_tlbwr(cpu_env);
+    check_mmu_idx(ctx);
+    return true;
+}
+
+static bool trans_tlbfill(DisasContext *ctx, arg_tlbfill *a)
+{
+    if (check_plv(ctx)) {
+        return false;
+    }
+    gen_helper_tlbfill(cpu_env);
+    check_mmu_idx(ctx);
+    return true;
+}
+
+static bool trans_tlbclr(DisasContext *ctx, arg_tlbclr *a)
+{
+    if (check_plv(ctx)) {
+        return false;
+    }
+    gen_helper_tlbclr(cpu_env);
+    check_mmu_idx(ctx);
+    return true;
+}
+
+static bool trans_tlbflush(DisasContext *ctx, arg_tlbflush *a)
+{
+    if (check_plv(ctx)) {
+        return false;
+    }
+    gen_helper_tlbflush(cpu_env);
+    check_mmu_idx(ctx);
+    return true;
+}
+
+static bool trans_invtlb(DisasContext *ctx, arg_invtlb *a)
+{
+    TCGv rj = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv rk = gpr_src(ctx, a->rk, EXT_NONE);
+
+    if (check_plv(ctx)) {
+        return false;
+    }
+
+    switch (a->imm) {
+    case 0:
+    case 1:
+        gen_helper_invtlb_all(cpu_env);
+        break;
+    case 2:
+        gen_helper_invtlb_all_g(cpu_env, tcg_constant_i32(1));
+        break;
+    case 3:
+        gen_helper_invtlb_all_g(cpu_env, tcg_constant_i32(0));
+        break;
+    case 4:
+        gen_helper_invtlb_all_asid(cpu_env, rj);
+        break;
+    case 5:
+        gen_helper_invtlb_page_asid(cpu_env, rj, rk);
+        break;
+    case 6:
+        gen_helper_invtlb_page_asid_or_g(cpu_env, rj, rk);
+        break;
+    default:
+        return false;
+    }
+    ctx->base.is_jmp = DISAS_STOP;
+    return true;
+}
diff --git a/target/loongarch/insns.decode b/target/loongarch/insns.decode
index 2b436d3cd6..f8ed11d83e 100644
--- a/target/loongarch/insns.decode
+++ b/target/loongarch/insns.decode
@@ -47,6 +47,8 @@
 &rr_offs      rj rd offs
 &r_csr        rd csr
 &rr_csr       rd rj csr
+&empty
+&i_rr         imm rj rk
 
 #
 # Formats
@@ -89,6 +91,8 @@
 @rr_offs16         .... .. ................ rj:5 rd:5    &rr_offs     offs=%offs16
 @r_csr                    .... .... csr:14 ..... rd:5    &r_csr
 @rr_csr                    .... .... csr:14 rj:5 rd:5    &rr_csr
+@empty          .... ........ ..... ..... ..... .....    &empty
+@i_rr             ...... ...... ..... rk:5 rj:5 imm:5    &i_rr
 
 #
 # Fixed point arithmetic operation instruction
@@ -459,3 +463,10 @@ iocsrwr_b        0000 01100100 10000 00100 ..... .....    @rr
 iocsrwr_h        0000 01100100 10000 00101 ..... .....    @rr
 iocsrwr_w        0000 01100100 10000 00110 ..... .....    @rr
 iocsrwr_d        0000 01100100 10000 00111 ..... .....    @rr
+tlbsrch          0000 01100100 10000 01010 00000 00000    @empty
+tlbrd            0000 01100100 10000 01011 00000 00000    @empty
+tlbwr            0000 01100100 10000 01100 00000 00000    @empty
+tlbfill          0000 01100100 10000 01101 00000 00000    @empty
+tlbclr           0000 01100100 10000 01000 00000 00000    @empty
+tlbflush         0000 01100100 10000 01001 00000 00000    @empty
+invtlb           0000 01100100 10011 ..... ..... .....    @i_rr
diff --git a/target/loongarch/tlb_helper.c b/target/loongarch/tlb_helper.c
index fad8bc7746..4f84ef3e4b 100644
--- a/target/loongarch/tlb_helper.c
+++ b/target/loongarch/tlb_helper.c
@@ -7,9 +7,11 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/guest-random.h"
 
 #include "cpu.h"
 #include "internals.h"
+#include "exec/helper-proto.h"
 #include "exec/exec-all.h"
 #include "exec/cpu_ldst.h"
 #include "exec/log.h"
@@ -280,6 +282,359 @@ static void raise_mmu_exception(CPULoongArchState *env, target_ulong address,
    }
 }
 
+static void invalidate_tlb_entry(CPULoongArchState *env, int index)
+{
+    target_ulong addr, mask, pagesize;
+    uint8_t tlb_ps;
+    LoongArchTLB *tlb = &env->tlb[index];
+
+    int mmu_idx = cpu_mmu_index(env, false);
+    uint8_t tlb_v0 = FIELD_EX64(tlb->tlb_entry0, TLBENTRY, V);
+    uint8_t tlb_v1 = FIELD_EX64(tlb->tlb_entry1, TLBENTRY, V);
+    uint64_t tlb_vppn = FIELD_EX64(tlb->tlb_misc, TLB_MISC, VPPN);
+
+    if (index >= LOONGARCH_STLB) {
+        tlb_ps = FIELD_EX64(tlb->tlb_misc, TLB_MISC, PS);
+    } else {
+        tlb_ps = FIELD_EX64(env->CSR_STLBPS, CSR_STLBPS, PS);
+    }
+    pagesize = 1 << tlb_ps;
+    mask = MAKE_64BIT_MASK(0, tlb_ps + 1);
+
+    if (tlb_v0) {
+        addr = (tlb_vppn << R_TLB_MISC_VPPN_SHIFT) & ~mask;    /* even */
+        tlb_flush_range_by_mmuidx(env_cpu(env), addr, pagesize,
+                                  mmu_idx, TARGET_LONG_BITS);
+    }
+
+    if (tlb_v1) {
+        addr = (tlb_vppn << R_TLB_MISC_VPPN_SHIFT) & pagesize;    /* odd */
+        tlb_flush_range_by_mmuidx(env_cpu(env), addr, pagesize,
+                                  mmu_idx, TARGET_LONG_BITS);
+    }
+}
+
+static void invalidate_tlb(CPULoongArchState *env, int index)
+{
+    LoongArchTLB *tlb;
+    uint16_t csr_asid, tlb_asid, tlb_g;
+
+    csr_asid = FIELD_EX64(env->CSR_ASID, CSR_ASID, ASID);
+    tlb = &env->tlb[index];
+    tlb_asid = FIELD_EX64(tlb->tlb_misc, TLB_MISC, ASID);
+    tlb_g = FIELD_EX64(tlb->tlb_entry0, TLBENTRY, G);
+    if (tlb_g == 0 && tlb_asid != csr_asid) {
+        return;
+    }
+    invalidate_tlb_entry(env, index);
+}
+
+static void fill_tlb_entry(CPULoongArchState *env, int index)
+{
+    LoongArchTLB *tlb = &env->tlb[index];
+    uint64_t lo0, lo1, csr_vppn;
+    uint16_t csr_asid;
+    uint8_t csr_ps;
+
+    if (FIELD_EX64(env->CSR_TLBRERA, CSR_TLBRERA, ISTLBR)) {
+        csr_ps = FIELD_EX64(env->CSR_TLBREHI, CSR_TLBREHI, PS);
+        csr_vppn = FIELD_EX64(env->CSR_TLBREHI, CSR_TLBREHI, VPPN);
+        lo0 = env->CSR_TLBRELO0;
+        lo1 = env->CSR_TLBRELO1;
+    } else {
+        csr_ps = FIELD_EX64(env->CSR_TLBIDX, CSR_TLBIDX, PS);
+        csr_vppn = FIELD_EX64(env->CSR_TLBEHI, CSR_TLBEHI, VPPN);
+        lo0 = env->CSR_TLBELO0;
+        lo1 = env->CSR_TLBELO1;
+    }
+
+    if (csr_ps == 0) {
+        qemu_log_mask(CPU_LOG_MMU, "page size is 0\n");
+    }
+
+    /* Only MTLB has the ps fields */
+    if (index >= LOONGARCH_STLB) {
+        tlb->tlb_misc = FIELD_DP64(tlb->tlb_misc, TLB_MISC, PS, csr_ps);
+    }
+
+    tlb->tlb_misc = FIELD_DP64(tlb->tlb_misc, TLB_MISC, VPPN, csr_vppn);
+    tlb->tlb_misc = FIELD_DP64(tlb->tlb_misc, TLB_MISC, E, 1);
+    csr_asid = FIELD_EX64(env->CSR_ASID, CSR_ASID, ASID);
+    tlb->tlb_misc = FIELD_DP64(tlb->tlb_misc, TLB_MISC, ASID, csr_asid);
+
+    tlb->tlb_entry0 = lo0;
+    tlb->tlb_entry1 = lo1;
+}
+
+/* Return an random value between low and high */
+static uint32_t get_random_tlb(uint32_t low, uint32_t high)
+{
+    uint32_t val;
+
+    qemu_guest_getrandom_nofail(&val, sizeof(val));
+    return val % (high - low + 1) + low;
+}
+
+void helper_tlbsrch(CPULoongArchState *env)
+{
+    int index, match;
+
+    if (FIELD_EX64(env->CSR_TLBRERA, CSR_TLBRERA, ISTLBR)) {
+        match = loongarch_tlb_search(env, env->CSR_TLBREHI, &index);
+    } else {
+        match = loongarch_tlb_search(env, env->CSR_TLBEHI, &index);
+    }
+
+    if (match) {
+        env->CSR_TLBIDX = FIELD_DP64(env->CSR_TLBIDX, CSR_TLBIDX, INDEX, index);
+        env->CSR_TLBIDX = FIELD_DP64(env->CSR_TLBIDX, CSR_TLBIDX, NE, 0);
+        return;
+    }
+
+    env->CSR_TLBIDX = FIELD_DP64(env->CSR_TLBIDX, CSR_TLBIDX, NE, 1);
+}
+
+void helper_tlbrd(CPULoongArchState *env)
+{
+    LoongArchTLB *tlb;
+    int index;
+    uint8_t tlb_ps, tlb_e;
+
+    index = FIELD_EX64(env->CSR_TLBIDX, CSR_TLBIDX, INDEX);
+    tlb = &env->tlb[index];
+
+    if (index >= LOONGARCH_STLB) {
+        tlb_ps = FIELD_EX64(tlb->tlb_misc, TLB_MISC, PS);
+    } else {
+        tlb_ps = FIELD_EX64(env->CSR_STLBPS, CSR_STLBPS, PS);
+    }
+    tlb_e = FIELD_EX64(tlb->tlb_misc, TLB_MISC, E);
+
+    if (!tlb_e) {
+        /* Invalid TLB entry */
+        env->CSR_TLBIDX = FIELD_DP64(env->CSR_TLBIDX, CSR_TLBIDX, NE, 1);
+        env->CSR_ASID  = FIELD_DP64(env->CSR_ASID, CSR_ASID, ASID, 0);
+        env->CSR_TLBEHI = 0;
+        env->CSR_TLBELO0 = 0;
+        env->CSR_TLBELO1 = 0;
+        env->CSR_TLBIDX = FIELD_DP64(env->CSR_TLBIDX, CSR_TLBIDX, PS, 0);
+    } else {
+        /* Valid TLB entry */
+        env->CSR_TLBIDX = FIELD_DP64(env->CSR_TLBIDX, CSR_TLBIDX, NE, 0);
+        env->CSR_TLBIDX = FIELD_DP64(env->CSR_TLBIDX, CSR_TLBIDX,
+                                     PS, (tlb_ps & 0x3f));
+        env->CSR_TLBEHI = FIELD_EX64(tlb->tlb_misc, TLB_MISC, VPPN) <<
+                                     R_TLB_MISC_VPPN_SHIFT;
+        env->CSR_TLBELO0 = tlb->tlb_entry0;
+        env->CSR_TLBELO1 = tlb->tlb_entry1;
+    }
+}
+
+void helper_tlbwr(CPULoongArchState *env)
+{
+    int index = FIELD_EX64(env->CSR_TLBIDX, CSR_TLBIDX, INDEX);
+
+    invalidate_tlb(env, index);
+
+    if (FIELD_EX64(env->CSR_TLBIDX, CSR_TLBIDX, NE)) {
+        env->tlb[index].tlb_misc = FIELD_DP64(env->tlb[index].tlb_misc,
+                                              TLB_MISC, E, 0);
+        return;
+    }
+
+    fill_tlb_entry(env, index);
+}
+
+void helper_tlbfill(CPULoongArchState *env)
+{
+    uint64_t address, entryhi;
+    int index, set, stlb_idx;
+    uint16_t pagesize, stlb_ps;
+
+    if (FIELD_EX64(env->CSR_TLBRERA, CSR_TLBRERA, ISTLBR)) {
+        entryhi = env->CSR_TLBREHI;
+        pagesize = FIELD_EX64(env->CSR_TLBREHI, CSR_TLBREHI, PS);
+    } else {
+        entryhi = env->CSR_TLBEHI;
+        pagesize = FIELD_EX64(env->CSR_TLBIDX, CSR_TLBIDX, PS);
+    }
+
+    stlb_ps = FIELD_EX64(env->CSR_STLBPS, CSR_STLBPS, PS);
+
+    if (pagesize == stlb_ps) {
+        /* Only write into STLB bits [47:13] */
+        address = entryhi & ~MAKE_64BIT_MASK(0, R_CSR_TLBEHI_VPPN_SHIFT);
+
+        /* Choose one set ramdomly */
+        set = get_random_tlb(0, 7);
+
+        /* Index in one set */
+        stlb_idx = (address >> (stlb_ps + 1)) & 0xff; /* [0,255] */
+
+        index = set * 256 + stlb_idx;
+    } else {
+        /* Only write into MTLB */
+        index = get_random_tlb(LOONGARCH_STLB, LOONGARCH_TLB_MAX - 1);
+    }
+
+    invalidate_tlb(env, index);
+    fill_tlb_entry(env, index);
+}
+
+void helper_tlbclr(CPULoongArchState *env)
+{
+    LoongArchTLB *tlb;
+    int i, index;
+    uint16_t csr_asid, tlb_asid, tlb_g;
+
+    csr_asid = FIELD_EX64(env->CSR_ASID, CSR_ASID, ASID);
+    index = FIELD_EX64(env->CSR_TLBIDX, CSR_TLBIDX, INDEX);
+
+    if (index < LOONGARCH_STLB) {
+        /* STLB. One line per operation */
+        for (i = 0; i < 8; i++) {
+            tlb = &env->tlb[i * 256 + (index % 256)];
+            tlb_asid = FIELD_EX64(tlb->tlb_misc, TLB_MISC, ASID);
+            tlb_g = FIELD_EX64(tlb->tlb_entry0, TLBENTRY, G);
+            if (!tlb_g && tlb_asid == csr_asid) {
+                tlb->tlb_misc = FIELD_DP64(tlb->tlb_misc, TLB_MISC, E, 0);
+            }
+        }
+    } else if (index < LOONGARCH_TLB_MAX) {
+        /* All MTLB entries */
+        for (i = LOONGARCH_STLB; i < LOONGARCH_TLB_MAX; i++) {
+            tlb = &env->tlb[i];
+            tlb_asid = FIELD_EX64(tlb->tlb_misc, TLB_MISC, ASID);
+            tlb_g = FIELD_EX64(tlb->tlb_entry0, TLBENTRY, G);
+            if (!tlb_g && tlb_asid == csr_asid) {
+                tlb->tlb_misc = FIELD_DP64(tlb->tlb_misc, TLB_MISC, E, 0);
+            }
+        }
+    }
+
+    tlb_flush(env_cpu(env));
+}
+
+void helper_tlbflush(CPULoongArchState *env)
+{
+    int i, index;
+
+    index = FIELD_EX64(env->CSR_TLBIDX, CSR_TLBIDX, INDEX);
+
+    if (index < LOONGARCH_STLB) {
+        /* STLB. One line per operation */
+        for (i = 0; i < 8; i++) {
+            int s_idx = i * 256 + (index % 256);
+            env->tlb[s_idx].tlb_misc = FIELD_DP64(env->tlb[s_idx].tlb_misc,
+                                                  TLB_MISC, E, 0);
+        }
+    } else if (index < LOONGARCH_TLB_MAX) {
+        /* All MTLB entries */
+        for (i = LOONGARCH_STLB; i < LOONGARCH_TLB_MAX; i++) {
+            env->tlb[i].tlb_misc = FIELD_DP64(env->tlb[i].tlb_misc,
+                                              TLB_MISC, E, 0);
+        }
+    }
+
+    tlb_flush(env_cpu(env));
+}
+
+void helper_invtlb_all(CPULoongArchState *env)
+{
+    for (int i = 0; i < LOONGARCH_TLB_MAX; i++) {
+        env->tlb[i].tlb_misc = FIELD_DP64(env->tlb[i].tlb_misc,
+                                          TLB_MISC, E, 0);
+    }
+    tlb_flush(env_cpu(env));
+}
+
+void helper_invtlb_all_g(CPULoongArchState *env, uint32_t g)
+{
+    for (int i = 0; i < LOONGARCH_TLB_MAX; i++) {
+        LoongArchTLB *tlb = &env->tlb[i];
+        uint8_t tlb_g = FIELD_EX64(tlb->tlb_entry0, TLBENTRY, G);
+
+        if (tlb_g == g) {
+            tlb->tlb_misc = FIELD_DP64(tlb->tlb_misc, TLB_MISC, E, 0);
+        }
+    }
+    tlb_flush(env_cpu(env));
+}
+
+void helper_invtlb_all_asid(CPULoongArchState *env, target_ulong info)
+{
+    uint16_t asid = info & R_CSR_ASID_ASID_MASK;
+
+    for (int i = 0; i < LOONGARCH_TLB_MAX; i++) {
+        LoongArchTLB *tlb = &env->tlb[i];
+        uint8_t tlb_g = FIELD_EX64(tlb->tlb_entry0, TLBENTRY, G);
+        uint16_t tlb_asid = FIELD_EX64(tlb->tlb_misc, TLB_MISC, ASID);
+
+        if (!tlb_g && (tlb_asid == asid)) {
+            tlb->tlb_misc = FIELD_DP64(tlb->tlb_misc, TLB_MISC, E, 0);
+        }
+    }
+    tlb_flush(env_cpu(env));
+}
+
+void helper_invtlb_page_asid(CPULoongArchState *env, target_ulong info,
+                             target_ulong addr)
+{
+    uint16_t asid = info & 0x3ff;
+
+    for (int i = 0; i < LOONGARCH_TLB_MAX; i++) {
+        LoongArchTLB *tlb = &env->tlb[i];
+        uint8_t tlb_g = FIELD_EX64(tlb->tlb_entry0, TLBENTRY, G);
+        uint16_t tlb_asid = FIELD_EX64(tlb->tlb_misc, TLB_MISC, ASID);
+        uint64_t vpn, tlb_vppn;
+        uint8_t tlb_ps, compare_shift;
+
+        if (i >= LOONGARCH_STLB) {
+            tlb_ps = FIELD_EX64(tlb->tlb_misc, TLB_MISC, PS);
+        } else {
+            tlb_ps = FIELD_EX64(env->CSR_STLBPS, CSR_STLBPS, PS);
+        }
+        tlb_vppn = FIELD_EX64(tlb->tlb_misc, TLB_MISC, VPPN);
+        vpn = (addr & TARGET_VIRT_MASK) >> (tlb_ps + 1);
+        compare_shift = tlb_ps + 1 - R_TLB_MISC_VPPN_SHIFT;
+
+        if (!tlb_g && (tlb_asid == asid) &&
+           (vpn == (tlb_vppn >> compare_shift))) {
+            tlb->tlb_misc = FIELD_DP64(tlb->tlb_misc, TLB_MISC, E, 0);
+        }
+    }
+    tlb_flush(env_cpu(env));
+}
+
+void helper_invtlb_page_asid_or_g(CPULoongArchState *env,
+                                  target_ulong info, target_ulong addr)
+{
+    uint16_t asid = info & 0x3ff;
+
+    for (int i = 0; i < LOONGARCH_TLB_MAX; i++) {
+        LoongArchTLB *tlb = &env->tlb[i];
+        uint8_t tlb_g = FIELD_EX64(tlb->tlb_entry0, TLBENTRY, G);
+        uint16_t tlb_asid = FIELD_EX64(tlb->tlb_misc, TLB_MISC, ASID);
+        uint64_t vpn, tlb_vppn;
+        uint8_t tlb_ps, compare_shift;
+
+        if (i >= LOONGARCH_STLB) {
+            tlb_ps = FIELD_EX64(tlb->tlb_misc, TLB_MISC, PS);
+        } else {
+            tlb_ps = FIELD_EX64(env->CSR_STLBPS, CSR_STLBPS, PS);
+        }
+        tlb_vppn = FIELD_EX64(tlb->tlb_misc, TLB_MISC, VPPN);
+        vpn = (addr & TARGET_VIRT_MASK) >> (tlb_ps + 1);
+        compare_shift = tlb_ps + 1 - R_TLB_MISC_VPPN_SHIFT;
+
+        if ((tlb_g || (tlb_asid == asid)) &&
+            (vpn == (tlb_vppn >> compare_shift))) {
+            tlb->tlb_misc = FIELD_DP64(tlb->tlb_misc, TLB_MISC, E, 0);
+        }
+    }
+    tlb_flush(env_cpu(env));
+}
+
 bool loongarch_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
                             MMUAccessType access_type, int mmu_idx,
                             bool probe, uintptr_t retaddr)

From d2cba6f7cea9d55aa0567fa8efdeaf2028e1de5e Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:18 +0800
Subject: [PATCH 835/935] target/loongarch: Add other core instructions support

This includes:
-CACOP
-LDDIR
-LDPTE
-ERTN
-DBCL
-IDLE

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-29-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/disas.c                      | 17 ++++
 target/loongarch/helper.h                     |  5 +
 .../insn_trans/trans_privileged.c.inc         | 65 +++++++++++++
 target/loongarch/insns.decode                 | 11 +++
 target/loongarch/internals.h                  |  5 +
 target/loongarch/op_helper.c                  | 37 ++++++++
 target/loongarch/tlb_helper.c                 | 93 +++++++++++++++++++
 7 files changed, 233 insertions(+)

diff --git a/target/loongarch/disas.c b/target/loongarch/disas.c
index 6a56607302..9d790b172c 100644
--- a/target/loongarch/disas.c
+++ b/target/loongarch/disas.c
@@ -314,6 +314,17 @@ static void output_i_rr(DisasContext *ctx, arg_i_rr *a, const char *mnemonic)
     output(ctx, mnemonic, "%d, r%d, r%d", a->imm, a->rj, a->rk);
 }
 
+static void output_cop_r_i(DisasContext *ctx, arg_cop_r_i *a,
+                           const char *mnemonic)
+{
+    output(ctx, mnemonic, "%d, r%d, %d", a->cop, a->rj, a->imm);
+}
+
+static void output_j_i(DisasContext *ctx, arg_j_i *a, const char *mnemonic)
+{
+    output(ctx, mnemonic, "r%d, %d", a->rj, a->imm);
+}
+
 #define INSN(insn, type)                                    \
 static bool trans_##insn(DisasContext *ctx, arg_##type * a) \
 {                                                           \
@@ -641,6 +652,12 @@ INSN(tlbfill,      empty)
 INSN(tlbclr,       empty)
 INSN(tlbflush,     empty)
 INSN(invtlb,       i_rr)
+INSN(cacop,        cop_r_i)
+INSN(lddir,        rr_i)
+INSN(ldpte,        j_i)
+INSN(ertn,         empty)
+INSN(idle,         i)
+INSN(dbcl,         i)
 
 #define output_fcmp(C, PREFIX, SUFFIX)                                         \
 {                                                                              \
diff --git a/target/loongarch/helper.h b/target/loongarch/helper.h
index b092ca75fe..626fc32e1e 100644
--- a/target/loongarch/helper.h
+++ b/target/loongarch/helper.h
@@ -121,3 +121,8 @@ DEF_HELPER_2(invtlb_all_g, void, env, i32)
 DEF_HELPER_2(invtlb_all_asid, void, env, tl)
 DEF_HELPER_3(invtlb_page_asid, void, env, tl, tl)
 DEF_HELPER_3(invtlb_page_asid_or_g, void, env, tl, tl)
+
+DEF_HELPER_4(lddir, tl, env, tl, tl, i32)
+DEF_HELPER_4(ldpte, void, env, tl, tl, i32)
+DEF_HELPER_1(ertn, void, env)
+DEF_HELPER_1(idle, void, env)
diff --git a/target/loongarch/insn_trans/trans_privileged.c.inc b/target/loongarch/insn_trans/trans_privileged.c.inc
index d47918785b..53596c4f77 100644
--- a/target/loongarch/insn_trans/trans_privileged.c.inc
+++ b/target/loongarch/insn_trans/trans_privileged.c.inc
@@ -399,3 +399,68 @@ static bool trans_invtlb(DisasContext *ctx, arg_invtlb *a)
     ctx->base.is_jmp = DISAS_STOP;
     return true;
 }
+
+static bool trans_cacop(DisasContext *ctx, arg_cacop *a)
+{
+    /* Treat the cacop as a nop */
+    if (check_plv(ctx)) {
+        return false;
+    }
+    return true;
+}
+
+static bool trans_ldpte(DisasContext *ctx, arg_ldpte *a)
+{
+    TCGv_i32 mem_idx = tcg_constant_i32(ctx->mem_idx);
+    TCGv src1 = gpr_src(ctx, a->rj, EXT_NONE);
+
+    if (check_plv(ctx)) {
+        return false;
+    }
+    gen_helper_ldpte(cpu_env, src1, tcg_constant_tl(a->imm), mem_idx);
+    return true;
+}
+
+static bool trans_lddir(DisasContext *ctx, arg_lddir *a)
+{
+    TCGv_i32 mem_idx = tcg_constant_i32(ctx->mem_idx);
+    TCGv src = gpr_src(ctx, a->rj, EXT_NONE);
+    TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
+
+    if (check_plv(ctx)) {
+        return false;
+    }
+    gen_helper_lddir(dest, cpu_env, src, tcg_constant_tl(a->imm), mem_idx);
+    return true;
+}
+
+static bool trans_ertn(DisasContext *ctx, arg_ertn *a)
+{
+    if (check_plv(ctx)) {
+        return false;
+    }
+    gen_helper_ertn(cpu_env);
+    ctx->base.is_jmp = DISAS_EXIT;
+    return true;
+}
+
+static bool trans_dbcl(DisasContext *ctx, arg_dbcl *a)
+{
+    if (check_plv(ctx)) {
+        return false;
+    }
+    generate_exception(ctx, EXCCODE_DBP);
+    return true;
+}
+
+static bool trans_idle(DisasContext *ctx, arg_idle *a)
+{
+    if (check_plv(ctx)) {
+        return false;
+    }
+
+    tcg_gen_movi_tl(cpu_pc, ctx->base.pc_next + 4);
+    gen_helper_idle(cpu_env);
+    ctx->base.is_jmp = DISAS_NORETURN;
+    return true;
+}
diff --git a/target/loongarch/insns.decode b/target/loongarch/insns.decode
index f8ed11d83e..ebd3d505fb 100644
--- a/target/loongarch/insns.decode
+++ b/target/loongarch/insns.decode
@@ -49,6 +49,8 @@
 &rr_csr       rd rj csr
 &empty
 &i_rr         imm rj rk
+&cop_r_i      cop rj imm
+&j_i          rj imm
 
 #
 # Formats
@@ -60,6 +62,7 @@
 @r_i20                          .... ... imm:s20 rd:5    &r_i
 @rr_ui5           .... ........ ..... imm:5 rj:5 rd:5    &rr_i
 @rr_ui6            .... ........ .... imm:6 rj:5 rd:5    &rr_i
+@rr_ui8              .. ........ .... imm:8 rj:5 rd:5    &rr_i
 @rr_i12                 .... ...... imm:s12 rj:5 rd:5    &rr_i
 @rr_ui12                 .... ...... imm:12 rj:5 rd:5    &rr_i
 @rr_i14s2         .... ....  .............. rj:5 rd:5    &rr_i imm=%i14s2
@@ -93,6 +96,8 @@
 @rr_csr                    .... .... csr:14 rj:5 rd:5    &rr_csr
 @empty          .... ........ ..... ..... ..... .....    &empty
 @i_rr             ...... ...... ..... rk:5 rj:5 imm:5    &i_rr
+@cop_r_i              .... ......  imm:s12 rj:5 cop:5    &cop_r_i
+@j_i               .... ........ .. imm:8 rj:5 .....    &j_i
 
 #
 # Fixed point arithmetic operation instruction
@@ -470,3 +475,9 @@ tlbfill          0000 01100100 10000 01101 00000 00000    @empty
 tlbclr           0000 01100100 10000 01000 00000 00000    @empty
 tlbflush         0000 01100100 10000 01001 00000 00000    @empty
 invtlb           0000 01100100 10011 ..... ..... .....    @i_rr
+cacop            0000 011000 ............ ..... .....     @cop_r_i
+lddir            0000 01100100 00 ........ ..... .....    @rr_ui8
+ldpte            0000 01100100 01 ........ ..... 00000    @j_i
+ertn             0000 01100100 10000 01110 00000 00000    @empty
+idle             0000 01100100 10001 ...............      @i15
+dbcl             0000 00000010 10101 ...............      @i15
diff --git a/target/loongarch/internals.h b/target/loongarch/internals.h
index 4b1bcd7c0f..a410c41c37 100644
--- a/target/loongarch/internals.h
+++ b/target/loongarch/internals.h
@@ -16,6 +16,11 @@
 #define TARGET_PHYS_MASK MAKE_64BIT_MASK(0, TARGET_PHYS_ADDR_SPACE_BITS)
 #define TARGET_VIRT_MASK MAKE_64BIT_MASK(0, TARGET_VIRT_ADDR_SPACE_BITS)
 
+/* Global bit used for lddir/ldpte */
+#define LOONGARCH_PAGE_HUGE_SHIFT   6
+/* Global bit for huge page */
+#define LOONGARCH_HGLOBAL_SHIFT     12
+
 void loongarch_translate_init(void);
 
 void loongarch_cpu_dump_state(CPUState *cpu, FILE *f, int flags);
diff --git a/target/loongarch/op_helper.c b/target/loongarch/op_helper.c
index 18e565ce7f..a9ba72d5b2 100644
--- a/target/loongarch/op_helper.c
+++ b/target/loongarch/op_helper.c
@@ -6,6 +6,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/log.h"
 #include "qemu/main-loop.h"
 #include "cpu.h"
 #include "qemu/host-utils.h"
@@ -15,6 +16,7 @@
 #include "internals.h"
 #include "qemu/crc32c.h"
 #include <zlib.h>
+#include "cpu-csr.h"
 
 /* Exceptions helpers */
 void helper_raise_exception(CPULoongArchState *env, uint32_t exception)
@@ -81,3 +83,38 @@ target_ulong helper_cpucfg(CPULoongArchState *env, target_ulong rj)
 {
     return rj > 21 ? 0 : env->cpucfg[rj];
 }
+
+void helper_ertn(CPULoongArchState *env)
+{
+    uint64_t csr_pplv, csr_pie;
+    if (FIELD_EX64(env->CSR_TLBRERA, CSR_TLBRERA, ISTLBR)) {
+        csr_pplv = FIELD_EX64(env->CSR_TLBRPRMD, CSR_TLBRPRMD, PPLV);
+        csr_pie = FIELD_EX64(env->CSR_TLBRPRMD, CSR_TLBRPRMD, PIE);
+
+        env->CSR_TLBRERA = FIELD_DP64(env->CSR_TLBRERA, CSR_TLBRERA, ISTLBR, 0);
+        env->CSR_CRMD = FIELD_DP64(env->CSR_CRMD, CSR_CRMD, DA, 0);
+        env->CSR_CRMD = FIELD_DP64(env->CSR_CRMD, CSR_CRMD, PG, 1);
+        env->pc = env->CSR_TLBRERA;
+        qemu_log_mask(CPU_LOG_INT, "%s: TLBRERA " TARGET_FMT_lx "\n",
+                      __func__, env->CSR_TLBRERA);
+    } else {
+        csr_pplv = FIELD_EX64(env->CSR_PRMD, CSR_PRMD, PPLV);
+        csr_pie = FIELD_EX64(env->CSR_PRMD, CSR_PRMD, PIE);
+
+        env->pc = env->CSR_ERA;
+        qemu_log_mask(CPU_LOG_INT, "%s: ERA " TARGET_FMT_lx "\n",
+                      __func__, env->CSR_ERA);
+    }
+    env->CSR_CRMD = FIELD_DP64(env->CSR_CRMD, CSR_CRMD, PLV, csr_pplv);
+    env->CSR_CRMD = FIELD_DP64(env->CSR_CRMD, CSR_CRMD, IE, csr_pie);
+
+    env->lladdr = 1;
+}
+
+void helper_idle(CPULoongArchState *env)
+{
+    CPUState *cs = env_cpu(env);
+
+    cs->halted = 1;
+    do_raise_exception(env, EXCP_HLT, 0);
+}
diff --git a/target/loongarch/tlb_helper.c b/target/loongarch/tlb_helper.c
index 4f84ef3e4b..bab19c7e05 100644
--- a/target/loongarch/tlb_helper.c
+++ b/target/loongarch/tlb_helper.c
@@ -668,3 +668,96 @@ bool loongarch_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
     raise_mmu_exception(env, address, access_type, ret);
     cpu_loop_exit_restore(cs, retaddr);
 }
+
+target_ulong helper_lddir(CPULoongArchState *env, target_ulong base,
+                          target_ulong level, uint32_t mem_idx)
+{
+    CPUState *cs = env_cpu(env);
+    target_ulong badvaddr, index, phys, ret;
+    int shift;
+    uint64_t dir_base, dir_width;
+    bool huge = (base >> LOONGARCH_PAGE_HUGE_SHIFT) & 0x1;
+
+    badvaddr = env->CSR_TLBRBADV;
+    base = base & TARGET_PHYS_MASK;
+
+    /* 0:64bit, 1:128bit, 2:192bit, 3:256bit */
+    shift = FIELD_EX64(env->CSR_PWCL, CSR_PWCL, PTEWIDTH);
+    shift = (shift + 1) * 3;
+
+    if (huge) {
+        return base;
+    }
+    switch (level) {
+    case 1:
+        dir_base = FIELD_EX64(env->CSR_PWCL, CSR_PWCL, DIR1_BASE);
+        dir_width = FIELD_EX64(env->CSR_PWCL, CSR_PWCL, DIR1_WIDTH);
+        break;
+    case 2:
+        dir_base = FIELD_EX64(env->CSR_PWCL, CSR_PWCL, DIR2_BASE);
+        dir_width = FIELD_EX64(env->CSR_PWCL, CSR_PWCL, DIR2_WIDTH);
+        break;
+    case 3:
+        dir_base = FIELD_EX64(env->CSR_PWCH, CSR_PWCH, DIR3_BASE);
+        dir_width = FIELD_EX64(env->CSR_PWCH, CSR_PWCH, DIR3_WIDTH);
+        break;
+    case 4:
+        dir_base = FIELD_EX64(env->CSR_PWCH, CSR_PWCH, DIR4_BASE);
+        dir_width = FIELD_EX64(env->CSR_PWCH, CSR_PWCH, DIR4_WIDTH);
+        break;
+    default:
+        do_raise_exception(env, EXCCODE_INE, GETPC());
+        return 0;
+    }
+    index = (badvaddr >> dir_base) & ((1 << dir_width) - 1);
+    phys = base | index << shift;
+    ret = ldq_phys(cs->as, phys) & TARGET_PHYS_MASK;
+    return ret;
+}
+
+void helper_ldpte(CPULoongArchState *env, target_ulong base, target_ulong odd,
+                  uint32_t mem_idx)
+{
+    CPUState *cs = env_cpu(env);
+    target_ulong phys, tmp0, ptindex, ptoffset0, ptoffset1, ps, badv;
+    int shift;
+    bool huge = (base >> LOONGARCH_PAGE_HUGE_SHIFT) & 0x1;
+    uint64_t ptbase = FIELD_EX64(env->CSR_PWCL, CSR_PWCL, PTBASE);
+    uint64_t ptwidth = FIELD_EX64(env->CSR_PWCL, CSR_PWCL, PTWIDTH);
+
+    base = base & TARGET_PHYS_MASK;
+
+    if (huge) {
+        /* Huge Page. base is paddr */
+        tmp0 = base ^ (1 << LOONGARCH_PAGE_HUGE_SHIFT);
+        /* Move Global bit */
+        tmp0 = ((tmp0 & (1 << LOONGARCH_HGLOBAL_SHIFT))  >>
+                LOONGARCH_HGLOBAL_SHIFT) << R_TLBENTRY_G_SHIFT |
+                (tmp0 & (~(1 << R_TLBENTRY_G_SHIFT)));
+        ps = ptbase + ptwidth - 1;
+        if (odd) {
+            tmp0 += (1 << ps);
+        }
+    } else {
+        /* 0:64bit, 1:128bit, 2:192bit, 3:256bit */
+        shift = FIELD_EX64(env->CSR_PWCL, CSR_PWCL, PTEWIDTH);
+        shift = (shift + 1) * 3;
+        badv = env->CSR_TLBRBADV;
+
+        ptindex = (badv >> ptbase) & ((1 << ptwidth) - 1);
+        ptindex = ptindex & ~0x1;   /* clear bit 0 */
+        ptoffset0 = ptindex << shift;
+        ptoffset1 = (ptindex + 1) << shift;
+
+        phys = base | (odd ? ptoffset1 : ptoffset0);
+        tmp0 = ldq_phys(cs->as, phys) & TARGET_PHYS_MASK;
+        ps = ptbase;
+    }
+
+    if (odd) {
+        env->CSR_TLBRELO1 = tmp0;
+    } else {
+        env->CSR_TLBRELO0 = tmp0;
+    }
+    env->CSR_TLBREHI = FIELD_DP64(env->CSR_TLBREHI, CSR_TLBREHI, PS, ps);
+}

From f9bf50745f29c85e531857898ed7927b6db7c763 Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:19 +0800
Subject: [PATCH 836/935] target/loongarch: Add timer related instructions
 support.

This includes:
-RDTIME{L/H}.W
-RDTIME.D

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-30-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/loongarch/disas.c                      |  3 ++
 target/loongarch/helper.h                     |  2 ++
 target/loongarch/insn_trans/trans_extra.c.inc | 33 +++++++++++++++++++
 target/loongarch/insns.decode                 |  3 ++
 target/loongarch/op_helper.c                  | 13 ++++++++
 target/loongarch/translate.c                  |  2 ++
 6 files changed, 56 insertions(+)

diff --git a/target/loongarch/disas.c b/target/loongarch/disas.c
index 9d790b172c..858dfcc53a 100644
--- a/target/loongarch/disas.c
+++ b/target/loongarch/disas.c
@@ -352,6 +352,9 @@ INSN(bitrev_w,     rr)
 INSN(bitrev_d,     rr)
 INSN(ext_w_h,      rr)
 INSN(ext_w_b,      rr)
+INSN(rdtimel_w,    rr)
+INSN(rdtimeh_w,    rr)
+INSN(rdtime_d,     rr)
 INSN(cpucfg,       rr)
 INSN(asrtle_d,     rr_jk)
 INSN(asrtgt_d,     rr_jk)
diff --git a/target/loongarch/helper.h b/target/loongarch/helper.h
index 626fc32e1e..85c11a60d4 100644
--- a/target/loongarch/helper.h
+++ b/target/loongarch/helper.h
@@ -93,6 +93,8 @@ DEF_HELPER_2(frint_d, i64, env, i64)
 
 DEF_HELPER_FLAGS_2(set_rounding_mode, TCG_CALL_NO_RWG, void, env, i32)
 
+DEF_HELPER_1(rdtime_d, i64, env)
+
 /* CSRs helper */
 DEF_HELPER_1(csrrd_pgd, i64, env)
 DEF_HELPER_1(csrrd_tval, i64, env)
diff --git a/target/loongarch/insn_trans/trans_extra.c.inc b/target/loongarch/insn_trans/trans_extra.c.inc
index 549f75a867..ad713cd61e 100644
--- a/target/loongarch/insn_trans/trans_extra.c.inc
+++ b/target/loongarch/insn_trans/trans_extra.c.inc
@@ -33,6 +33,39 @@ static bool trans_asrtgt_d(DisasContext *ctx, arg_asrtgt_d * a)
     return true;
 }
 
+static bool gen_rdtime(DisasContext *ctx, arg_rr *a,
+                       bool word, bool high)
+{
+    TCGv dst1 = gpr_dst(ctx, a->rd, EXT_NONE);
+    TCGv dst2 = gpr_dst(ctx, a->rj, EXT_NONE);
+
+    if (tb_cflags(ctx->base.tb) & CF_USE_ICOUNT) {
+        gen_io_start();
+    }
+    gen_helper_rdtime_d(dst1, cpu_env);
+    if (word) {
+        tcg_gen_sextract_tl(dst1, dst1, high ? 32 : 0, 32);
+    }
+    tcg_gen_ld_i64(dst2, cpu_env, offsetof(CPULoongArchState, CSR_TID));
+
+    return true;
+}
+
+static bool trans_rdtimel_w(DisasContext *ctx, arg_rdtimel_w *a)
+{
+    return gen_rdtime(ctx, a, 1, 0);
+}
+
+static bool trans_rdtimeh_w(DisasContext *ctx, arg_rdtimeh_w *a)
+{
+    return gen_rdtime(ctx, a, 1, 1);
+}
+
+static bool trans_rdtime_d(DisasContext *ctx, arg_rdtime_d *a)
+{
+    return gen_rdtime(ctx, a, 0, 0);
+}
+
 static bool trans_cpucfg(DisasContext *ctx, arg_cpucfg *a)
 {
     TCGv dest = gpr_dst(ctx, a->rd, EXT_NONE);
diff --git a/target/loongarch/insns.decode b/target/loongarch/insns.decode
index ebd3d505fb..3fdc6e148c 100644
--- a/target/loongarch/insns.decode
+++ b/target/loongarch/insns.decode
@@ -309,6 +309,9 @@ break           0000 00000010 10100 ...............      @i15
 syscall         0000 00000010 10110 ...............      @i15
 asrtle_d        0000 00000000 00010 ..... ..... 00000    @rr_jk
 asrtgt_d        0000 00000000 00011 ..... ..... 00000    @rr_jk
+rdtimel_w       0000 00000000 00000 11000 ..... .....    @rr
+rdtimeh_w       0000 00000000 00000 11001 ..... .....    @rr
+rdtime_d        0000 00000000 00000 11010 ..... .....    @rr
 cpucfg          0000 00000000 00000 11011 ..... .....    @rr
 
 #
diff --git a/target/loongarch/op_helper.c b/target/loongarch/op_helper.c
index a9ba72d5b2..d87049851f 100644
--- a/target/loongarch/op_helper.c
+++ b/target/loongarch/op_helper.c
@@ -84,6 +84,19 @@ target_ulong helper_cpucfg(CPULoongArchState *env, target_ulong rj)
     return rj > 21 ? 0 : env->cpucfg[rj];
 }
 
+uint64_t helper_rdtime_d(CPULoongArchState *env)
+{
+    uint64_t plv;
+    LoongArchCPU *cpu = env_archcpu(env);
+
+    plv = FIELD_EX64(env->CSR_CRMD, CSR_CRMD, PLV);
+    if (extract64(env->CSR_MISC, R_CSR_MISC_DRDTL_SHIFT + plv, 1)) {
+        do_raise_exception(env, EXCCODE_IPE, GETPC());
+    }
+
+    return cpu_loongarch_get_constant_timer_counter(cpu);
+}
+
 void helper_ertn(CPULoongArchState *env)
 {
     uint64_t csr_pplv, csr_pie;
diff --git a/target/loongarch/translate.c b/target/loongarch/translate.c
index 0f098924a3..c9afd11420 100644
--- a/target/loongarch/translate.c
+++ b/target/loongarch/translate.c
@@ -25,6 +25,8 @@ static TCGv cpu_lladdr, cpu_llval;
 TCGv_i32 cpu_fcsr0;
 TCGv_i64 cpu_fpr[32];
 
+#include "exec/gen-icount.h"
+
 #define DISAS_STOP        DISAS_TARGET_0
 #define DISAS_EXIT        DISAS_TARGET_1
 #define DISAS_EXIT_UPDATE DISAS_TARGET_2

From a8a506c3907093a064dd2d475564e677fb1bf148 Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:20 +0800
Subject: [PATCH 837/935] hw/loongarch: Add support loongson3 virt machine
 type.

Emulate a 3A5000 board use the new loongarch instruction.
3A5000 belongs to the Loongson3 series processors.
The board consists of a 3A5000 cpu model and the virt
bridge. The host 3A5000 board is really complicated and
contains many functions.Now for the tcg softmmu mode
only part functions are emulated.

More detailed info you can see
https://github.com/loongson/LoongArch-Documentation

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-31-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 MAINTAINERS                                   |  4 +
 .../devices/loongarch64-softmmu/default.mak   |  3 +
 configs/targets/loongarch64-softmmu.mak       |  3 +
 hw/Kconfig                                    |  1 +
 hw/loongarch/Kconfig                          |  4 +
 hw/loongarch/loongson3.c                      | 94 +++++++++++++++++++
 hw/loongarch/meson.build                      |  4 +
 hw/meson.build                                |  1 +
 include/exec/poison.h                         |  2 +
 include/hw/loongarch/virt.h                   | 31 ++++++
 include/sysemu/arch_init.h                    |  1 +
 qapi/machine.json                             |  2 +-
 target/Kconfig                                |  1 +
 target/loongarch/Kconfig                      |  2 +
 target/loongarch/cpu.c                        |  2 +
 15 files changed, 154 insertions(+), 1 deletion(-)
 create mode 100644 configs/devices/loongarch64-softmmu/default.mak
 create mode 100644 configs/targets/loongarch64-softmmu.mak
 create mode 100644 hw/loongarch/Kconfig
 create mode 100644 hw/loongarch/loongson3.c
 create mode 100644 hw/loongarch/meson.build
 create mode 100644 include/hw/loongarch/virt.h
 create mode 100644 target/loongarch/Kconfig

diff --git a/MAINTAINERS b/MAINTAINERS
index e3f875071a..0c54c03cef 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1129,6 +1129,10 @@ M: Xiaojuan Yang <yangxiaojuan@loongson.cn>
 M: Song Gao <gaosong@loongson.cn>
 S: Maintained
 F: docs/system/loongarch/loongson3.rst
+F: configs/targets/loongarch64-softmmu.mak
+F: configs/devices/loongarch64-softmmu/default.mak
+F: hw/loongarch/
+F: include/hw/loongarch/virt.h
 
 M68K Machines
 -------------
diff --git a/configs/devices/loongarch64-softmmu/default.mak b/configs/devices/loongarch64-softmmu/default.mak
new file mode 100644
index 0000000000..928bc117ef
--- /dev/null
+++ b/configs/devices/loongarch64-softmmu/default.mak
@@ -0,0 +1,3 @@
+# Default configuration for loongarch64-softmmu
+
+CONFIG_LOONGARCH_VIRT=y
diff --git a/configs/targets/loongarch64-softmmu.mak b/configs/targets/loongarch64-softmmu.mak
new file mode 100644
index 0000000000..666154022f
--- /dev/null
+++ b/configs/targets/loongarch64-softmmu.mak
@@ -0,0 +1,3 @@
+TARGET_ARCH=loongarch64
+TARGET_BASE_ARCH=loongarch
+TARGET_SUPPORTS_MTTCG=y
diff --git a/hw/Kconfig b/hw/Kconfig
index 50e0952889..38233bbb0f 100644
--- a/hw/Kconfig
+++ b/hw/Kconfig
@@ -50,6 +50,7 @@ source avr/Kconfig
 source cris/Kconfig
 source hppa/Kconfig
 source i386/Kconfig
+source loongarch/Kconfig
 source m68k/Kconfig
 source microblaze/Kconfig
 source mips/Kconfig
diff --git a/hw/loongarch/Kconfig b/hw/loongarch/Kconfig
new file mode 100644
index 0000000000..13e8501897
--- /dev/null
+++ b/hw/loongarch/Kconfig
@@ -0,0 +1,4 @@
+config LOONGARCH_VIRT
+    bool
+    select PCI
+    select PCI_EXPRESS_GENERIC_BRIDGE
diff --git a/hw/loongarch/loongson3.c b/hw/loongarch/loongson3.c
new file mode 100644
index 0000000000..7df32f777e
--- /dev/null
+++ b/hw/loongarch/loongson3.c
@@ -0,0 +1,94 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * QEMU loongson 3a5000 develop board emulation
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+#include "qemu/osdep.h"
+#include "qemu/units.h"
+#include "qemu/datadir.h"
+#include "qapi/error.h"
+#include "hw/boards.h"
+#include "sysemu/sysemu.h"
+#include "sysemu/qtest.h"
+#include "sysemu/runstate.h"
+#include "sysemu/reset.h"
+#include "sysemu/rtc.h"
+#include "hw/loongarch/virt.h"
+#include "exec/address-spaces.h"
+#include "target/loongarch/cpu.h"
+
+static void loongarch_init(MachineState *machine)
+{
+    const char *cpu_model = machine->cpu_type;
+    ram_addr_t offset = 0;
+    ram_addr_t ram_size = machine->ram_size;
+    uint64_t highram_size = 0;
+    MemoryRegion *address_space_mem = get_system_memory();
+    LoongArchMachineState *lams = LOONGARCH_MACHINE(machine);
+    int i;
+
+    if (!cpu_model) {
+        cpu_model = LOONGARCH_CPU_TYPE_NAME("la464");
+    }
+
+    if (!strstr(cpu_model, "la464")) {
+        error_report("LoongArch/TCG needs cpu type la464");
+        exit(1);
+    }
+
+    if (ram_size < 1 * GiB) {
+        error_report("ram_size must be greater than 1G.");
+        exit(1);
+    }
+
+    /* Init CPUs */
+    for (i = 0; i < machine->smp.cpus; i++) {
+        cpu_create(machine->cpu_type);
+    }
+
+    /* Add memory region */
+    memory_region_init_alias(&lams->lowmem, NULL, "loongarch.lowram",
+                             machine->ram, 0, 256 * MiB);
+    memory_region_add_subregion(address_space_mem, offset, &lams->lowmem);
+    offset += 256 * MiB;
+
+    highram_size = ram_size - 256 * MiB;
+    memory_region_init_alias(&lams->highmem, NULL, "loongarch.highmem",
+                             machine->ram, offset, highram_size);
+    memory_region_add_subregion(address_space_mem, 0x90000000, &lams->highmem);
+
+    /* Add isa io region */
+    memory_region_init_alias(&lams->isa_io, NULL, "isa-io",
+                             get_system_io(), 0, LOONGARCH_ISA_IO_SIZE);
+    memory_region_add_subregion(address_space_mem, LOONGARCH_ISA_IO_BASE,
+                                &lams->isa_io);
+}
+
+static void loongarch_class_init(ObjectClass *oc, void *data)
+{
+    MachineClass *mc = MACHINE_CLASS(oc);
+
+    mc->desc = "Loongson-3A5000 LS7A1000 machine";
+    mc->init = loongarch_init;
+    mc->default_ram_size = 1 * GiB;
+    mc->default_cpu_type = LOONGARCH_CPU_TYPE_NAME("la464");
+    mc->default_ram_id = "loongarch.ram";
+    mc->max_cpus = LOONGARCH_MAX_VCPUS;
+    mc->is_default = 1;
+    mc->default_kernel_irqchip_split = false;
+    mc->block_default_type = IF_VIRTIO;
+    mc->default_boot_order = "c";
+    mc->no_cdrom = 1;
+}
+
+static const TypeInfo loongarch_machine_types[] = {
+    {
+        .name           = TYPE_LOONGARCH_MACHINE,
+        .parent         = TYPE_MACHINE,
+        .instance_size  = sizeof(LoongArchMachineState),
+        .class_init     = loongarch_class_init,
+    }
+};
+
+DEFINE_TYPES(loongarch_machine_types)
diff --git a/hw/loongarch/meson.build b/hw/loongarch/meson.build
new file mode 100644
index 0000000000..cecb1a5d65
--- /dev/null
+++ b/hw/loongarch/meson.build
@@ -0,0 +1,4 @@
+loongarch_ss = ss.source_set()
+loongarch_ss.add(when: 'CONFIG_LOONGARCH_VIRT', if_true: files('loongson3.c'))
+
+hw_arch += {'loongarch': loongarch_ss}
diff --git a/hw/meson.build b/hw/meson.build
index 9992c5101e..c7ac7d3d75 100644
--- a/hw/meson.build
+++ b/hw/meson.build
@@ -50,6 +50,7 @@ subdir('avr')
 subdir('cris')
 subdir('hppa')
 subdir('i386')
+subdir('loongarch')
 subdir('m68k')
 subdir('microblaze')
 subdir('mips')
diff --git a/include/exec/poison.h b/include/exec/poison.h
index 9f1ca3409c..bbb82cf9ec 100644
--- a/include/exec/poison.h
+++ b/include/exec/poison.h
@@ -14,6 +14,7 @@
 #pragma GCC poison TARGET_CRIS
 #pragma GCC poison TARGET_HEXAGON
 #pragma GCC poison TARGET_HPPA
+#pragma GCC poison TARGET_LOONGARCH64
 #pragma GCC poison TARGET_M68K
 #pragma GCC poison TARGET_MICROBLAZE
 #pragma GCC poison TARGET_MIPS
@@ -71,6 +72,7 @@
 #pragma GCC poison CONFIG_HPPA_DIS
 #pragma GCC poison CONFIG_I386_DIS
 #pragma GCC poison CONFIG_HEXAGON_DIS
+#pragma GCC poison CONFIG_LOONGARCH_DIS
 #pragma GCC poison CONFIG_M68K_DIS
 #pragma GCC poison CONFIG_MICROBLAZE_DIS
 #pragma GCC poison CONFIG_MIPS_DIS
diff --git a/include/hw/loongarch/virt.h b/include/hw/loongarch/virt.h
new file mode 100644
index 0000000000..4a4bb3f51f
--- /dev/null
+++ b/include/hw/loongarch/virt.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Definitions for loongarch board emulation.
+ *
+ * Copyright (C) 2021 Loongson Technology Corporation Limited
+ */
+
+#ifndef HW_LOONGARCH_H
+#define HW_LOONGARCH_H
+
+#include "target/loongarch/cpu.h"
+#include "hw/boards.h"
+#include "qemu/queue.h"
+
+#define LOONGARCH_MAX_VCPUS     4
+
+#define LOONGARCH_ISA_IO_BASE   0x18000000UL
+#define LOONGARCH_ISA_IO_SIZE   0x0004000
+
+struct LoongArchMachineState {
+    /*< private >*/
+    MachineState parent_obj;
+
+    MemoryRegion lowmem;
+    MemoryRegion highmem;
+    MemoryRegion isa_io;
+};
+
+#define TYPE_LOONGARCH_MACHINE  MACHINE_TYPE_NAME("virt")
+OBJECT_DECLARE_SIMPLE_TYPE(LoongArchMachineState, LOONGARCH_MACHINE)
+#endif
diff --git a/include/sysemu/arch_init.h b/include/sysemu/arch_init.h
index 79c2591425..8850cb1a14 100644
--- a/include/sysemu/arch_init.h
+++ b/include/sysemu/arch_init.h
@@ -24,6 +24,7 @@ enum {
     QEMU_ARCH_RX = (1 << 20),
     QEMU_ARCH_AVR = (1 << 21),
     QEMU_ARCH_HEXAGON = (1 << 22),
+    QEMU_ARCH_LOONGARCH = (1 << 23),
 };
 
 extern const uint32_t arch_type;
diff --git a/qapi/machine.json b/qapi/machine.json
index 883ce3f9ea..f750a16396 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -30,7 +30,7 @@
 ##
 { 'enum' : 'SysEmuTarget',
   'data' : [ 'aarch64', 'alpha', 'arm', 'avr', 'cris', 'hppa', 'i386',
-             'm68k', 'microblaze', 'microblazeel', 'mips', 'mips64',
+             'loongarch64', 'm68k', 'microblaze', 'microblazeel', 'mips', 'mips64',
              'mips64el', 'mipsel', 'nios2', 'or1k', 'ppc',
              'ppc64', 'riscv32', 'riscv64', 'rx', 's390x', 'sh4',
              'sh4eb', 'sparc', 'sparc64', 'tricore',
diff --git a/target/Kconfig b/target/Kconfig
index ae7f24fc66..83da0bd293 100644
--- a/target/Kconfig
+++ b/target/Kconfig
@@ -4,6 +4,7 @@ source avr/Kconfig
 source cris/Kconfig
 source hppa/Kconfig
 source i386/Kconfig
+source loongarch/Kconfig
 source m68k/Kconfig
 source microblaze/Kconfig
 source mips/Kconfig
diff --git a/target/loongarch/Kconfig b/target/loongarch/Kconfig
new file mode 100644
index 0000000000..46b26b1a85
--- /dev/null
+++ b/target/loongarch/Kconfig
@@ -0,0 +1,2 @@
+config LOONGARCH64
+    bool
diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c
index b846bd1a8b..3b7d6289d2 100644
--- a/target/loongarch/cpu.c
+++ b/target/loongarch/cpu.c
@@ -82,6 +82,8 @@ static void loongarch_cpu_set_pc(CPUState *cs, vaddr value)
     env->pc = value;
 }
 
+#include "hw/loongarch/virt.h"
+
 void loongarch_cpu_set_irq(void *opaque, int irq, int level)
 {
     LoongArchCPU *cpu = opaque;

From f6783e34380955e9ec0656c7b9fb8936b9733a6a Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:21 +0800
Subject: [PATCH 838/935] hw/loongarch: Add LoongArch ipi interrupt
 support(IPI)

This patch realize the IPI interrupt controller.

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-32-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 MAINTAINERS                     |   2 +
 hw/intc/Kconfig                 |   3 +
 hw/intc/loongarch_ipi.c         | 242 ++++++++++++++++++++++++++++++++
 hw/intc/meson.build             |   1 +
 hw/intc/trace-events            |   4 +
 hw/loongarch/Kconfig            |   1 +
 include/hw/intc/loongarch_ipi.h |  52 +++++++
 include/hw/loongarch/virt.h     |   2 +
 8 files changed, 307 insertions(+)
 create mode 100644 hw/intc/loongarch_ipi.c
 create mode 100644 include/hw/intc/loongarch_ipi.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 0c54c03cef..271eb89e2a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1133,6 +1133,8 @@ F: configs/targets/loongarch64-softmmu.mak
 F: configs/devices/loongarch64-softmmu/default.mak
 F: hw/loongarch/
 F: include/hw/loongarch/virt.h
+F: include/hw/intc/loongarch_*.h
+F: hw/intc/loongarch_*.c
 
 M68K Machines
 -------------
diff --git a/hw/intc/Kconfig b/hw/intc/Kconfig
index eded1b557e..1122c33cec 100644
--- a/hw/intc/Kconfig
+++ b/hw/intc/Kconfig
@@ -87,3 +87,6 @@ config M68K_IRQC
 
 config NIOS2_VIC
     bool
+
+config LOONGARCH_IPI
+    bool
diff --git a/hw/intc/loongarch_ipi.c b/hw/intc/loongarch_ipi.c
new file mode 100644
index 0000000000..66bee93675
--- /dev/null
+++ b/hw/intc/loongarch_ipi.c
@@ -0,0 +1,242 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * LoongArch ipi interrupt support
+ *
+ * Copyright (C) 2021 Loongson Technology Corporation Limited
+ */
+
+#include "qemu/osdep.h"
+#include "hw/sysbus.h"
+#include "hw/intc/loongarch_ipi.h"
+#include "hw/irq.h"
+#include "qapi/error.h"
+#include "qemu/log.h"
+#include "exec/address-spaces.h"
+#include "hw/loongarch/virt.h"
+#include "migration/vmstate.h"
+#include "target/loongarch/internals.h"
+#include "trace.h"
+
+static uint64_t loongarch_ipi_readl(void *opaque, hwaddr addr, unsigned size)
+{
+    IPICore *s = opaque;
+    uint64_t ret = 0;
+    int index = 0;
+
+    addr &= 0xff;
+    switch (addr) {
+    case CORE_STATUS_OFF:
+        ret = s->status;
+        break;
+    case CORE_EN_OFF:
+        ret = s->en;
+        break;
+    case CORE_SET_OFF:
+        ret = 0;
+        break;
+    case CORE_CLEAR_OFF:
+        ret = 0;
+        break;
+    case CORE_BUF_20 ... CORE_BUF_38 + 4:
+        index = (addr - CORE_BUF_20) >> 2;
+        ret = s->buf[index];
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "invalid read: %x", (uint32_t)addr);
+        break;
+    }
+
+    trace_loongarch_ipi_read(size, (uint64_t)addr, ret);
+    return ret;
+}
+
+static int get_ipi_data(target_ulong val)
+{
+    int i, mask, data;
+
+    data = val >> 32;
+    mask = (val >> 27) & 0xf;
+
+    for (i = 0; i < 4; i++) {
+        if ((mask >> i) & 1) {
+            data &= ~(0xff << (i * 8));
+        }
+    }
+    return data;
+}
+
+static void ipi_send(uint64_t val)
+{
+    int cpuid, data;
+    CPULoongArchState *env;
+
+    cpuid = (val >> 16) & 0x3ff;
+    /* IPI status vector */
+    data = 1 << (val & 0x1f);
+    qemu_mutex_lock_iothread();
+    CPUState *cs = qemu_get_cpu(cpuid);
+    LoongArchCPU *cpu = LOONGARCH_CPU(cs);
+    env = &cpu->env;
+    loongarch_cpu_set_irq(cpu, IRQ_IPI, 1);
+    qemu_mutex_unlock_iothread();
+    address_space_stl(&env->address_space_iocsr, 0x1008,
+                      data, MEMTXATTRS_UNSPECIFIED, NULL);
+
+}
+
+static void mail_send(uint64_t val)
+{
+    int cpuid, data;
+    hwaddr addr;
+    CPULoongArchState *env;
+
+    cpuid = (val >> 16) & 0x3ff;
+    addr = 0x1020 + (val & 0x1c);
+    CPUState *cs = qemu_get_cpu(cpuid);
+    LoongArchCPU *cpu = LOONGARCH_CPU(cs);
+    env = &cpu->env;
+    data = get_ipi_data(val);
+    address_space_stl(&env->address_space_iocsr, addr,
+                      data, MEMTXATTRS_UNSPECIFIED, NULL);
+}
+
+static void any_send(uint64_t val)
+{
+    int cpuid, data;
+    hwaddr addr;
+    CPULoongArchState *env;
+
+    cpuid = (val >> 16) & 0x3ff;
+    addr = val & 0xffff;
+    CPUState *cs = qemu_get_cpu(cpuid);
+    LoongArchCPU *cpu = LOONGARCH_CPU(cs);
+    env = &cpu->env;
+    data = get_ipi_data(val);
+    address_space_stl(&env->address_space_iocsr, addr,
+                      data, MEMTXATTRS_UNSPECIFIED, NULL);
+}
+
+static void loongarch_ipi_writel(void *opaque, hwaddr addr, uint64_t val,
+                                 unsigned size)
+{
+    IPICore *s = opaque;
+    int index = 0;
+
+    addr &= 0xff;
+    trace_loongarch_ipi_write(size, (uint64_t)addr, val);
+    switch (addr) {
+    case CORE_STATUS_OFF:
+        qemu_log_mask(LOG_GUEST_ERROR, "can not be written");
+        break;
+    case CORE_EN_OFF:
+        s->en = val;
+        break;
+    case CORE_SET_OFF:
+        s->status |= val;
+        if (s->status != 0 && (s->status & s->en) != 0) {
+            qemu_irq_raise(s->irq);
+        }
+        break;
+    case CORE_CLEAR_OFF:
+        s->status &= ~val;
+        if (s->status == 0 && s->en != 0) {
+            qemu_irq_lower(s->irq);
+        }
+        break;
+    case CORE_BUF_20 ... CORE_BUF_38 + 4:
+        index = (addr - CORE_BUF_20) >> 2;
+        s->buf[index] = val;
+        break;
+    case IOCSR_IPI_SEND:
+        ipi_send(val);
+        break;
+    case IOCSR_MAIL_SEND:
+        mail_send(val);
+        break;
+    case IOCSR_ANY_SEND:
+        any_send(val);
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "invalid write: %x", (uint32_t)addr);
+        break;
+    }
+}
+
+static const MemoryRegionOps loongarch_ipi_ops = {
+    .read = loongarch_ipi_readl,
+    .write = loongarch_ipi_writel,
+    .impl.min_access_size = 4,
+    .impl.max_access_size = 4,
+    .valid.min_access_size = 4,
+    .valid.max_access_size = 8,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
+static void loongarch_ipi_init(Object *obj)
+{
+    int cpu;
+    LoongArchMachineState *lams;
+    LoongArchIPI *s = LOONGARCH_IPI(obj);
+    SysBusDevice *sbd = SYS_BUS_DEVICE(obj);
+    Object *machine = qdev_get_machine();
+    ObjectClass *mc = object_get_class(machine);
+    /* 'lams' should be initialized */
+    if (!strcmp(MACHINE_CLASS(mc)->name, "none")) {
+        return;
+    }
+    lams = LOONGARCH_MACHINE(machine);
+    for (cpu = 0; cpu < MAX_IPI_CORE_NUM; cpu++) {
+        memory_region_init_io(&s->ipi_iocsr_mem[cpu], obj, &loongarch_ipi_ops,
+                            &lams->ipi_core[cpu], "loongarch_ipi_iocsr", 0x100);
+        sysbus_init_mmio(sbd, &s->ipi_iocsr_mem[cpu]);
+        qdev_init_gpio_out(DEVICE(obj), &lams->ipi_core[cpu].irq, 1);
+    }
+}
+
+static const VMStateDescription vmstate_ipi_core = {
+    .name = "ipi-single",
+    .version_id = 0,
+    .minimum_version_id = 0,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT32(status, IPICore),
+        VMSTATE_UINT32(en, IPICore),
+        VMSTATE_UINT32(set, IPICore),
+        VMSTATE_UINT32(clear, IPICore),
+        VMSTATE_UINT32_ARRAY(buf, IPICore, MAX_IPI_MBX_NUM * 2),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static const VMStateDescription vmstate_loongarch_ipi = {
+    .name = TYPE_LOONGARCH_IPI,
+    .version_id = 0,
+    .minimum_version_id = 0,
+    .fields = (VMStateField[]) {
+        VMSTATE_STRUCT_ARRAY(ipi_core, LoongArchMachineState,
+                             MAX_IPI_CORE_NUM, 0,
+                             vmstate_ipi_core, IPICore),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static void loongarch_ipi_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+
+    dc->vmsd = &vmstate_loongarch_ipi;
+}
+
+static const TypeInfo loongarch_ipi_info = {
+    .name          = TYPE_LOONGARCH_IPI,
+    .parent        = TYPE_SYS_BUS_DEVICE,
+    .instance_size = sizeof(LoongArchIPI),
+    .instance_init = loongarch_ipi_init,
+    .class_init    = loongarch_ipi_class_init,
+};
+
+static void loongarch_ipi_register_types(void)
+{
+    type_register_static(&loongarch_ipi_info);
+}
+
+type_init(loongarch_ipi_register_types)
diff --git a/hw/intc/meson.build b/hw/intc/meson.build
index 8b35139f82..57af7fb33f 100644
--- a/hw/intc/meson.build
+++ b/hw/intc/meson.build
@@ -63,3 +63,4 @@ specific_ss.add(when: ['CONFIG_KVM', 'CONFIG_XIVE'],
 specific_ss.add(when: 'CONFIG_GOLDFISH_PIC', if_true: files('goldfish_pic.c'))
 specific_ss.add(when: 'CONFIG_M68K_IRQC', if_true: files('m68k_irqc.c'))
 specific_ss.add(when: 'CONFIG_NIOS2_VIC', if_true: files('nios2_vic.c'))
+specific_ss.add(when: 'CONFIG_LOONGARCH_IPI', if_true: files('loongarch_ipi.c'))
diff --git a/hw/intc/trace-events b/hw/intc/trace-events
index 5271590304..36774662af 100644
--- a/hw/intc/trace-events
+++ b/hw/intc/trace-events
@@ -287,3 +287,7 @@ sh_intc_register(const char *s, int id, unsigned short v, int c, int m) "%s %u -
 sh_intc_read(unsigned size, uint64_t offset, unsigned long val) "size %u 0x%" PRIx64 " -> 0x%lx"
 sh_intc_write(unsigned size, uint64_t offset, unsigned long val) "size %u 0x%" PRIx64 " <- 0x%lx"
 sh_intc_set(int id, int enable) "setting interrupt group %d to %d"
+
+# loongarch_ipi.c
+loongarch_ipi_read(unsigned size, uint64_t addr, uint64_t val) "size: %u addr: 0x%"PRIx64 "val: 0x%"PRIx64
+loongarch_ipi_write(unsigned size, uint64_t addr, uint64_t val) "size: %u addr: 0x%"PRIx64 "val: 0x%"PRIx64
diff --git a/hw/loongarch/Kconfig b/hw/loongarch/Kconfig
index 13e8501897..f0dad3329a 100644
--- a/hw/loongarch/Kconfig
+++ b/hw/loongarch/Kconfig
@@ -2,3 +2,4 @@ config LOONGARCH_VIRT
     bool
     select PCI
     select PCI_EXPRESS_GENERIC_BRIDGE
+    select LOONGARCH_IPI
diff --git a/include/hw/intc/loongarch_ipi.h b/include/hw/intc/loongarch_ipi.h
new file mode 100644
index 0000000000..996ed7ea93
--- /dev/null
+++ b/include/hw/intc/loongarch_ipi.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * LoongArch ipi interrupt header files
+ *
+ * Copyright (C) 2021 Loongson Technology Corporation Limited
+ */
+
+#ifndef HW_LOONGARCH_IPI_H
+#define HW_LOONGARCH_IPI_H
+
+#include "hw/sysbus.h"
+
+/* Mainy used by iocsr read and write */
+#define SMP_IPI_MAILBOX      0x1000ULL
+#define CORE_STATUS_OFF       0x0
+#define CORE_EN_OFF           0x4
+#define CORE_SET_OFF          0x8
+#define CORE_CLEAR_OFF        0xc
+#define CORE_BUF_20           0x20
+#define CORE_BUF_28           0x28
+#define CORE_BUF_30           0x30
+#define CORE_BUF_38           0x38
+#define IOCSR_IPI_SEND        0x40
+#define IOCSR_MAIL_SEND       0x48
+#define IOCSR_ANY_SEND        0x158
+
+/* IPI system memory address */
+#define IPI_SYSTEM_MEM        0x1fe01000
+
+#define MAX_IPI_CORE_NUM      4
+#define MAX_IPI_MBX_NUM       4
+
+#define TYPE_LOONGARCH_IPI "loongarch_ipi"
+OBJECT_DECLARE_SIMPLE_TYPE(LoongArchIPI, LOONGARCH_IPI)
+
+typedef struct IPICore {
+    uint32_t status;
+    uint32_t en;
+    uint32_t set;
+    uint32_t clear;
+    /* 64bit buf divide into 2 32bit buf */
+    uint32_t buf[MAX_IPI_MBX_NUM * 2];
+    qemu_irq irq;
+} IPICore;
+
+struct LoongArchIPI {
+    SysBusDevice parent_obj;
+    MemoryRegion ipi_iocsr_mem[MAX_IPI_CORE_NUM];
+    MemoryRegion ipi_system_mem[MAX_IPI_CORE_NUM];
+};
+
+#endif
diff --git a/include/hw/loongarch/virt.h b/include/hw/loongarch/virt.h
index 4a4bb3f51f..09a816191c 100644
--- a/include/hw/loongarch/virt.h
+++ b/include/hw/loongarch/virt.h
@@ -11,6 +11,7 @@
 #include "target/loongarch/cpu.h"
 #include "hw/boards.h"
 #include "qemu/queue.h"
+#include "hw/intc/loongarch_ipi.h"
 
 #define LOONGARCH_MAX_VCPUS     4
 
@@ -21,6 +22,7 @@ struct LoongArchMachineState {
     /*< private >*/
     MachineState parent_obj;
 
+    IPICore ipi_core[MAX_IPI_CORE_NUM];
     MemoryRegion lowmem;
     MemoryRegion highmem;
     MemoryRegion isa_io;

From 0f4fcf1845fe188901d4ff4cc807bd78690dddd0 Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:22 +0800
Subject: [PATCH 839/935] hw/intc: Add LoongArch ls7a interrupt controller
 support(PCH-PIC)

This patch realize the PCH-PIC interrupt controller.

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-33-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 MAINTAINERS                         |   1 +
 hw/intc/Kconfig                     |   4 +
 hw/intc/loongarch_pch_pic.c         | 431 ++++++++++++++++++++++++++++
 hw/intc/meson.build                 |   1 +
 hw/intc/trace-events                |   9 +
 hw/loongarch/Kconfig                |   1 +
 include/hw/intc/loongarch_pch_pic.h |  69 +++++
 include/hw/pci-host/ls7a.h          |  30 ++
 8 files changed, 546 insertions(+)
 create mode 100644 hw/intc/loongarch_pch_pic.c
 create mode 100644 include/hw/intc/loongarch_pch_pic.h
 create mode 100644 include/hw/pci-host/ls7a.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 271eb89e2a..3ce5674da5 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1135,6 +1135,7 @@ F: hw/loongarch/
 F: include/hw/loongarch/virt.h
 F: include/hw/intc/loongarch_*.h
 F: hw/intc/loongarch_*.c
+F: include/hw/pci-host/ls7a.h
 
 M68K Machines
 -------------
diff --git a/hw/intc/Kconfig b/hw/intc/Kconfig
index 1122c33cec..362980ca8c 100644
--- a/hw/intc/Kconfig
+++ b/hw/intc/Kconfig
@@ -90,3 +90,7 @@ config NIOS2_VIC
 
 config LOONGARCH_IPI
     bool
+
+config LOONGARCH_PCH_PIC
+    bool
+    select UNIMP
diff --git a/hw/intc/loongarch_pch_pic.c b/hw/intc/loongarch_pch_pic.c
new file mode 100644
index 0000000000..3c9814a3b4
--- /dev/null
+++ b/hw/intc/loongarch_pch_pic.c
@@ -0,0 +1,431 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * QEMU Loongson 7A1000 I/O interrupt controller.
+ *
+ * Copyright (C) 2021 Loongson Technology Corporation Limited
+ */
+
+#include "qemu/osdep.h"
+#include "hw/sysbus.h"
+#include "hw/loongarch/virt.h"
+#include "hw/irq.h"
+#include "hw/intc/loongarch_pch_pic.h"
+#include "migration/vmstate.h"
+#include "trace.h"
+
+static void pch_pic_update_irq(LoongArchPCHPIC *s, uint64_t mask, int level)
+{
+    unsigned long val;
+    int irq;
+
+    if (level) {
+        val = mask & s->intirr & ~s->int_mask;
+        if (val) {
+            irq = find_first_bit(&val, 64);
+            s->intisr |= 0x1ULL << irq;
+            qemu_set_irq(s->parent_irq[s->htmsi_vector[irq]], 1);
+        }
+    } else {
+        val = mask & s->intisr;
+        if (val) {
+            irq = find_first_bit(&val, 64);
+            s->intisr &= ~(0x1ULL << irq);
+            qemu_set_irq(s->parent_irq[s->htmsi_vector[irq]], 0);
+        }
+    }
+}
+
+static void pch_pic_irq_handler(void *opaque, int irq, int level)
+{
+    LoongArchPCHPIC *s = LOONGARCH_PCH_PIC(opaque);
+    uint64_t mask = 1ULL << irq;
+
+    assert(irq < PCH_PIC_IRQ_NUM);
+    trace_loongarch_pch_pic_irq_handler(irq, level);
+
+    if (s->intedge & mask) {
+        /* Edge triggered */
+        if (level) {
+            if ((s->last_intirr & mask) == 0) {
+                s->intirr |= mask;
+            }
+            s->last_intirr |= mask;
+        } else {
+            s->last_intirr &= ~mask;
+        }
+    } else {
+        /* Level triggered */
+        if (level) {
+            s->intirr |= mask;
+            s->last_intirr |= mask;
+        } else {
+            s->intirr &= ~mask;
+            s->last_intirr &= ~mask;
+        }
+    }
+    pch_pic_update_irq(s, mask, level);
+}
+
+static uint64_t loongarch_pch_pic_low_readw(void *opaque, hwaddr addr,
+                                            unsigned size)
+{
+    LoongArchPCHPIC *s = LOONGARCH_PCH_PIC(opaque);
+    uint64_t val = 0;
+    uint32_t offset = addr & 0xfff;
+
+    switch (offset) {
+    case PCH_PIC_INT_ID_LO:
+        val = PCH_PIC_INT_ID_VAL;
+        break;
+    case PCH_PIC_INT_ID_HI:
+        val = PCH_PIC_INT_ID_NUM;
+        break;
+    case PCH_PIC_INT_MASK_LO:
+        val = (uint32_t)s->int_mask;
+        break;
+    case PCH_PIC_INT_MASK_HI:
+        val = s->int_mask >> 32;
+        break;
+    case PCH_PIC_INT_EDGE_LO:
+        val = (uint32_t)s->intedge;
+        break;
+    case PCH_PIC_INT_EDGE_HI:
+        val = s->intedge >> 32;
+        break;
+    case PCH_PIC_HTMSI_EN_LO:
+        val = (uint32_t)s->htmsi_en;
+        break;
+    case PCH_PIC_HTMSI_EN_HI:
+        val = s->htmsi_en >> 32;
+        break;
+    case PCH_PIC_AUTO_CTRL0_LO:
+    case PCH_PIC_AUTO_CTRL0_HI:
+    case PCH_PIC_AUTO_CTRL1_LO:
+    case PCH_PIC_AUTO_CTRL1_HI:
+        break;
+    default:
+        break;
+    }
+
+    trace_loongarch_pch_pic_low_readw(size, addr, val);
+    return val;
+}
+
+static uint64_t get_writew_val(uint64_t value, uint32_t target, bool hi)
+{
+    uint64_t mask = 0xffffffff00000000;
+    uint64_t data = target;
+
+    return hi ? (value & ~mask) | (data << 32) : (value & mask) | data;
+}
+
+static void loongarch_pch_pic_low_writew(void *opaque, hwaddr addr,
+                                         uint64_t value, unsigned size)
+{
+    LoongArchPCHPIC *s = LOONGARCH_PCH_PIC(opaque);
+    uint32_t offset, old_valid, data = (uint32_t)value;
+    uint64_t old, int_mask;
+    offset = addr & 0xfff;
+
+    trace_loongarch_pch_pic_low_writew(size, addr, data);
+
+    switch (offset) {
+    case PCH_PIC_INT_MASK_LO:
+        old = s->int_mask;
+        s->int_mask = get_writew_val(old, data, 0);
+        old_valid = (uint32_t)old;
+        if (old_valid & ~data) {
+            pch_pic_update_irq(s, (old_valid & ~data), 1);
+        }
+        if (~old_valid & data) {
+            pch_pic_update_irq(s, (~old_valid & data), 0);
+        }
+        break;
+    case PCH_PIC_INT_MASK_HI:
+        old = s->int_mask;
+        s->int_mask = get_writew_val(old, data, 1);
+        old_valid = (uint32_t)(old >> 32);
+        int_mask = old_valid & ~data;
+        if (int_mask) {
+            pch_pic_update_irq(s, int_mask << 32, 1);
+        }
+        int_mask = ~old_valid & data;
+        if (int_mask) {
+            pch_pic_update_irq(s, int_mask << 32, 0);
+        }
+        break;
+    case PCH_PIC_INT_EDGE_LO:
+        s->intedge = get_writew_val(s->intedge, data, 0);
+        break;
+    case PCH_PIC_INT_EDGE_HI:
+        s->intedge = get_writew_val(s->intedge, data, 1);
+        break;
+    case PCH_PIC_INT_CLEAR_LO:
+        if (s->intedge & data) {
+            s->intirr &= (~data);
+            pch_pic_update_irq(s, data, 0);
+            s->intisr &= (~data);
+        }
+        break;
+    case PCH_PIC_INT_CLEAR_HI:
+        value <<= 32;
+        if (s->intedge & value) {
+            s->intirr &= (~value);
+            pch_pic_update_irq(s, value, 0);
+            s->intisr &= (~value);
+        }
+        break;
+    case PCH_PIC_HTMSI_EN_LO:
+        s->htmsi_en = get_writew_val(s->htmsi_en, data, 0);
+        break;
+    case PCH_PIC_HTMSI_EN_HI:
+        s->htmsi_en = get_writew_val(s->htmsi_en, data, 1);
+        break;
+    case PCH_PIC_AUTO_CTRL0_LO:
+    case PCH_PIC_AUTO_CTRL0_HI:
+    case PCH_PIC_AUTO_CTRL1_LO:
+    case PCH_PIC_AUTO_CTRL1_HI:
+        break;
+    default:
+        break;
+    }
+}
+
+static uint64_t loongarch_pch_pic_high_readw(void *opaque, hwaddr addr,
+                                        unsigned size)
+{
+    LoongArchPCHPIC *s = LOONGARCH_PCH_PIC(opaque);
+    uint64_t val = 0;
+    uint32_t offset = addr & 0xfff;
+
+    switch (offset) {
+    case STATUS_LO_START:
+        val = (uint32_t)(s->intisr & (~s->int_mask));
+        break;
+    case STATUS_HI_START:
+        val = (s->intisr & (~s->int_mask)) >> 32;
+        break;
+    case POL_LO_START:
+        val = (uint32_t)s->int_polarity;
+        break;
+    case POL_HI_START:
+        val = s->int_polarity >> 32;
+        break;
+    default:
+        break;
+    }
+
+    trace_loongarch_pch_pic_high_readw(size, addr, val);
+    return val;
+}
+
+static void loongarch_pch_pic_high_writew(void *opaque, hwaddr addr,
+                                     uint64_t value, unsigned size)
+{
+    LoongArchPCHPIC *s = LOONGARCH_PCH_PIC(opaque);
+    uint32_t offset, data = (uint32_t)value;
+    offset = addr & 0xfff;
+
+    trace_loongarch_pch_pic_high_writew(size, addr, data);
+
+    switch (offset) {
+    case STATUS_LO_START:
+        s->intisr = get_writew_val(s->intisr, data, 0);
+        break;
+    case STATUS_HI_START:
+        s->intisr = get_writew_val(s->intisr, data, 1);
+        break;
+    case POL_LO_START:
+        s->int_polarity = get_writew_val(s->int_polarity, data, 0);
+        break;
+    case POL_HI_START:
+        s->int_polarity = get_writew_val(s->int_polarity, data, 1);
+        break;
+    default:
+        break;
+    }
+}
+
+static uint64_t loongarch_pch_pic_readb(void *opaque, hwaddr addr,
+                                        unsigned size)
+{
+    LoongArchPCHPIC *s = LOONGARCH_PCH_PIC(opaque);
+    uint64_t val = 0;
+    uint32_t offset = (addr & 0xfff) + PCH_PIC_ROUTE_ENTRY_OFFSET;
+    int64_t offset_tmp;
+
+    switch (offset) {
+    case PCH_PIC_HTMSI_VEC_OFFSET ... PCH_PIC_HTMSI_VEC_END:
+        offset_tmp = offset - PCH_PIC_HTMSI_VEC_OFFSET;
+        if (offset_tmp >= 0 && offset_tmp < 64) {
+            val = s->htmsi_vector[offset_tmp];
+        }
+        break;
+    case PCH_PIC_ROUTE_ENTRY_OFFSET ... PCH_PIC_ROUTE_ENTRY_END:
+        offset_tmp = offset - PCH_PIC_ROUTE_ENTRY_OFFSET;
+        if (offset_tmp >= 0 && offset_tmp < 64) {
+            val = s->route_entry[offset_tmp];
+        }
+        break;
+    default:
+        break;
+    }
+
+    trace_loongarch_pch_pic_readb(size, addr, val);
+    return val;
+}
+
+static void loongarch_pch_pic_writeb(void *opaque, hwaddr addr,
+                                     uint64_t data, unsigned size)
+{
+    LoongArchPCHPIC *s = LOONGARCH_PCH_PIC(opaque);
+    int32_t offset_tmp;
+    uint32_t offset = (addr & 0xfff) + PCH_PIC_ROUTE_ENTRY_OFFSET;
+
+    trace_loongarch_pch_pic_writeb(size, addr, data);
+
+    switch (offset) {
+    case PCH_PIC_HTMSI_VEC_OFFSET ... PCH_PIC_HTMSI_VEC_END:
+        offset_tmp = offset - PCH_PIC_HTMSI_VEC_OFFSET;
+        if (offset_tmp >= 0 && offset_tmp < 64) {
+            s->htmsi_vector[offset_tmp] = (uint8_t)(data & 0xff);
+        }
+        break;
+    case PCH_PIC_ROUTE_ENTRY_OFFSET ... PCH_PIC_ROUTE_ENTRY_END:
+        offset_tmp = offset - PCH_PIC_ROUTE_ENTRY_OFFSET;
+        if (offset_tmp >= 0 && offset_tmp < 64) {
+            s->route_entry[offset_tmp] = (uint8_t)(data & 0xff);
+        }
+        break;
+    default:
+        break;
+    }
+}
+
+static const MemoryRegionOps loongarch_pch_pic_reg32_low_ops = {
+    .read = loongarch_pch_pic_low_readw,
+    .write = loongarch_pch_pic_low_writew,
+    .valid = {
+        .min_access_size = 4,
+        .max_access_size = 8,
+    },
+    .impl = {
+        .min_access_size = 4,
+        .max_access_size = 4,
+    },
+    .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
+static const MemoryRegionOps loongarch_pch_pic_reg32_high_ops = {
+    .read = loongarch_pch_pic_high_readw,
+    .write = loongarch_pch_pic_high_writew,
+    .valid = {
+        .min_access_size = 4,
+        .max_access_size = 8,
+    },
+    .impl = {
+        .min_access_size = 4,
+        .max_access_size = 4,
+    },
+    .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
+static const MemoryRegionOps loongarch_pch_pic_reg8_ops = {
+    .read = loongarch_pch_pic_readb,
+    .write = loongarch_pch_pic_writeb,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 1,
+    },
+    .impl = {
+        .min_access_size = 1,
+        .max_access_size = 1,
+    },
+    .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
+static void loongarch_pch_pic_reset(DeviceState *d)
+{
+    LoongArchPCHPIC *s = LOONGARCH_PCH_PIC(d);
+    int i;
+
+    s->int_mask = -1;
+    s->htmsi_en = 0x0;
+    s->intedge  = 0x0;
+    s->intclr   = 0x0;
+    s->auto_crtl0 = 0x0;
+    s->auto_crtl1 = 0x0;
+    for (i = 0; i < 64; i++) {
+        s->route_entry[i] = 0x1;
+        s->htmsi_vector[i] = 0x0;
+    }
+    s->intirr = 0x0;
+    s->intisr = 0x0;
+    s->last_intirr = 0x0;
+    s->int_polarity = 0x0;
+}
+
+static void loongarch_pch_pic_init(Object *obj)
+{
+    LoongArchPCHPIC *s = LOONGARCH_PCH_PIC(obj);
+    SysBusDevice *sbd = SYS_BUS_DEVICE(obj);
+
+    memory_region_init_io(&s->iomem32_low, obj,
+                          &loongarch_pch_pic_reg32_low_ops,
+                          s, PCH_PIC_NAME(.reg32_part1), 0x100);
+    memory_region_init_io(&s->iomem8, obj, &loongarch_pch_pic_reg8_ops,
+                          s, PCH_PIC_NAME(.reg8), 0x2a0);
+    memory_region_init_io(&s->iomem32_high, obj,
+                          &loongarch_pch_pic_reg32_high_ops,
+                          s, PCH_PIC_NAME(.reg32_part2), 0xc60);
+    sysbus_init_mmio(sbd, &s->iomem32_low);
+    sysbus_init_mmio(sbd, &s->iomem8);
+    sysbus_init_mmio(sbd, &s->iomem32_high);
+
+    qdev_init_gpio_out(DEVICE(obj), s->parent_irq, PCH_PIC_IRQ_NUM);
+    qdev_init_gpio_in(DEVICE(obj), pch_pic_irq_handler, PCH_PIC_IRQ_NUM);
+}
+
+static const VMStateDescription vmstate_loongarch_pch_pic = {
+    .name = TYPE_LOONGARCH_PCH_PIC,
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT64(int_mask, LoongArchPCHPIC),
+        VMSTATE_UINT64(htmsi_en, LoongArchPCHPIC),
+        VMSTATE_UINT64(intedge, LoongArchPCHPIC),
+        VMSTATE_UINT64(intclr, LoongArchPCHPIC),
+        VMSTATE_UINT64(auto_crtl0, LoongArchPCHPIC),
+        VMSTATE_UINT64(auto_crtl1, LoongArchPCHPIC),
+        VMSTATE_UINT8_ARRAY(route_entry, LoongArchPCHPIC, 64),
+        VMSTATE_UINT8_ARRAY(htmsi_vector, LoongArchPCHPIC, 64),
+        VMSTATE_UINT64(last_intirr, LoongArchPCHPIC),
+        VMSTATE_UINT64(intirr, LoongArchPCHPIC),
+        VMSTATE_UINT64(intisr, LoongArchPCHPIC),
+        VMSTATE_UINT64(int_polarity, LoongArchPCHPIC),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static void loongarch_pch_pic_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+
+    dc->reset = loongarch_pch_pic_reset;
+    dc->vmsd = &vmstate_loongarch_pch_pic;
+}
+
+static const TypeInfo loongarch_pch_pic_info = {
+    .name          = TYPE_LOONGARCH_PCH_PIC,
+    .parent        = TYPE_SYS_BUS_DEVICE,
+    .instance_size = sizeof(LoongArchPCHPIC),
+    .instance_init = loongarch_pch_pic_init,
+    .class_init    = loongarch_pch_pic_class_init,
+};
+
+static void loongarch_pch_pic_register_types(void)
+{
+    type_register_static(&loongarch_pch_pic_info);
+}
+
+type_init(loongarch_pch_pic_register_types)
diff --git a/hw/intc/meson.build b/hw/intc/meson.build
index 57af7fb33f..03f13f1c49 100644
--- a/hw/intc/meson.build
+++ b/hw/intc/meson.build
@@ -64,3 +64,4 @@ specific_ss.add(when: 'CONFIG_GOLDFISH_PIC', if_true: files('goldfish_pic.c'))
 specific_ss.add(when: 'CONFIG_M68K_IRQC', if_true: files('m68k_irqc.c'))
 specific_ss.add(when: 'CONFIG_NIOS2_VIC', if_true: files('nios2_vic.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_IPI', if_true: files('loongarch_ipi.c'))
+specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_PIC', if_true: files('loongarch_pch_pic.c'))
diff --git a/hw/intc/trace-events b/hw/intc/trace-events
index 36774662af..4cdbc01a07 100644
--- a/hw/intc/trace-events
+++ b/hw/intc/trace-events
@@ -291,3 +291,12 @@ sh_intc_set(int id, int enable) "setting interrupt group %d to %d"
 # loongarch_ipi.c
 loongarch_ipi_read(unsigned size, uint64_t addr, uint64_t val) "size: %u addr: 0x%"PRIx64 "val: 0x%"PRIx64
 loongarch_ipi_write(unsigned size, uint64_t addr, uint64_t val) "size: %u addr: 0x%"PRIx64 "val: 0x%"PRIx64
+
+# loongarch_pch_pic.c
+loongarch_pch_pic_irq_handler(int irq, int level) "irq %d level %d"
+loongarch_pch_pic_low_readw(unsigned size, uint64_t addr, uint64_t val) "size: %u addr: 0x%"PRIx64 "val: 0x%" PRIx64
+loongarch_pch_pic_low_writew(unsigned size, uint64_t addr, uint64_t val) "size: %u addr: 0x%"PRIx64 "val: 0x%" PRIx64
+loongarch_pch_pic_high_readw(unsigned size, uint64_t addr, uint64_t val) "size: %u addr: 0x%"PRIx64 "val: 0x%" PRIx64
+loongarch_pch_pic_high_writew(unsigned size, uint64_t addr, uint64_t val) "size: %u addr: 0x%"PRIx64 "val: 0x%" PRIx64
+loongarch_pch_pic_readb(unsigned size, uint64_t addr, uint64_t val) "size: %u addr: 0x%"PRIx64 "val: 0x%" PRIx64
+loongarch_pch_pic_writeb(unsigned size, uint64_t addr, uint64_t val) "size: %u addr: 0x%"PRIx64 "val: 0x%" PRIx64
diff --git a/hw/loongarch/Kconfig b/hw/loongarch/Kconfig
index f0dad3329a..2df45f7e8f 100644
--- a/hw/loongarch/Kconfig
+++ b/hw/loongarch/Kconfig
@@ -3,3 +3,4 @@ config LOONGARCH_VIRT
     select PCI
     select PCI_EXPRESS_GENERIC_BRIDGE
     select LOONGARCH_IPI
+    select LOONGARCH_PCH_PIC
diff --git a/include/hw/intc/loongarch_pch_pic.h b/include/hw/intc/loongarch_pch_pic.h
new file mode 100644
index 0000000000..2d4aa9ed6f
--- /dev/null
+++ b/include/hw/intc/loongarch_pch_pic.h
@@ -0,0 +1,69 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * LoongArch 7A1000 I/O interrupt controller definitions
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+#define TYPE_LOONGARCH_PCH_PIC "loongarch_pch_pic"
+#define PCH_PIC_NAME(name) TYPE_LOONGARCH_PCH_PIC#name
+OBJECT_DECLARE_SIMPLE_TYPE(LoongArchPCHPIC, LOONGARCH_PCH_PIC)
+
+#define PCH_PIC_IRQ_START               0
+#define PCH_PIC_IRQ_END                 63
+#define PCH_PIC_IRQ_NUM                 64
+#define PCH_PIC_INT_ID_VAL              0x7000000UL
+#define PCH_PIC_INT_ID_NUM              0x3f0001UL
+
+#define PCH_PIC_INT_ID_LO               0x00
+#define PCH_PIC_INT_ID_HI               0x04
+#define PCH_PIC_INT_MASK_LO             0x20
+#define PCH_PIC_INT_MASK_HI             0x24
+#define PCH_PIC_HTMSI_EN_LO             0x40
+#define PCH_PIC_HTMSI_EN_HI             0x44
+#define PCH_PIC_INT_EDGE_LO             0x60
+#define PCH_PIC_INT_EDGE_HI             0x64
+#define PCH_PIC_INT_CLEAR_LO            0x80
+#define PCH_PIC_INT_CLEAR_HI            0x84
+#define PCH_PIC_AUTO_CTRL0_LO           0xc0
+#define PCH_PIC_AUTO_CTRL0_HI           0xc4
+#define PCH_PIC_AUTO_CTRL1_LO           0xe0
+#define PCH_PIC_AUTO_CTRL1_HI           0xe4
+#define PCH_PIC_ROUTE_ENTRY_OFFSET      0x100
+#define PCH_PIC_ROUTE_ENTRY_END         0x13f
+#define PCH_PIC_HTMSI_VEC_OFFSET        0x200
+#define PCH_PIC_HTMSI_VEC_END           0x23f
+#define PCH_PIC_INT_STATUS_LO           0x3a0
+#define PCH_PIC_INT_STATUS_HI           0x3a4
+#define PCH_PIC_INT_POL_LO              0x3e0
+#define PCH_PIC_INT_POL_HI              0x3e4
+
+#define STATUS_LO_START                 0
+#define STATUS_HI_START                 0x4
+#define POL_LO_START                    0x40
+#define POL_HI_START                    0x44
+struct LoongArchPCHPIC {
+    SysBusDevice parent_obj;
+    qemu_irq parent_irq[64];
+    uint64_t int_mask; /*0x020 interrupt mask register*/
+    uint64_t htmsi_en; /*0x040 1=msi*/
+    uint64_t intedge; /*0x060 edge=1 level  =0*/
+    uint64_t intclr; /*0x080 for clean edge int,set 1 clean,set 0 is noused*/
+    uint64_t auto_crtl0; /*0x0c0*/
+    uint64_t auto_crtl1; /*0x0e0*/
+    uint64_t last_intirr;    /* edge detection */
+    uint64_t intirr; /* 0x380 interrupt request register */
+    uint64_t intisr; /* 0x3a0 interrupt service register */
+    /*
+     * 0x3e0 interrupt level polarity selection
+     * register 0 for high level trigger
+     */
+    uint64_t int_polarity;
+
+    uint8_t route_entry[64]; /*0x100 - 0x138*/
+    uint8_t htmsi_vector[64]; /*0x200 - 0x238*/
+
+    MemoryRegion iomem32_low;
+    MemoryRegion iomem32_high;
+    MemoryRegion iomem8;
+};
diff --git a/include/hw/pci-host/ls7a.h b/include/hw/pci-host/ls7a.h
new file mode 100644
index 0000000000..bf80e99ce1
--- /dev/null
+++ b/include/hw/pci-host/ls7a.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * QEMU LoongArch CPU
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+#ifndef HW_LS7A_H
+#define HW_LS7A_H
+
+#include "hw/pci/pci.h"
+#include "hw/pci/pcie_host.h"
+#include "hw/pci-host/pam.h"
+#include "qemu/units.h"
+#include "qemu/range.h"
+#include "qom/object.h"
+
+#define LS7A_PCH_REG_BASE       0x10000000UL
+#define LS7A_IOAPIC_REG_BASE    (LS7A_PCH_REG_BASE)
+#define LS7A_PCH_MSI_ADDR_LOW   0x2FF00000UL
+
+/*
+ * According to the kernel pch irq start from 64 offset
+ * 0 ~ 16 irqs used for non-pci device while 16 ~ 64 irqs
+ * used for pci device.
+ */
+#define PCH_PIC_IRQ_OFFSET      64
+#define LS7A_DEVICE_IRQS        16
+#define LS7A_PCI_IRQS           48
+#endif

From 249ad85a4b4ba6e949bba3c5b9932c389e07249c Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:23 +0800
Subject: [PATCH 840/935] hw/intc: Add LoongArch ls7a msi interrupt controller
 support(PCH-MSI)

This patch realize PCH-MSI interrupt controller.

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-34-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 hw/intc/Kconfig                     |  5 ++
 hw/intc/loongarch_pch_msi.c         | 73 +++++++++++++++++++++++++++++
 hw/intc/meson.build                 |  1 +
 hw/intc/trace-events                |  3 ++
 hw/loongarch/Kconfig                |  1 +
 include/hw/intc/loongarch_pch_msi.h | 20 ++++++++
 include/hw/pci-host/ls7a.h          |  3 ++
 7 files changed, 106 insertions(+)
 create mode 100644 hw/intc/loongarch_pch_msi.c
 create mode 100644 include/hw/intc/loongarch_pch_msi.h

diff --git a/hw/intc/Kconfig b/hw/intc/Kconfig
index 362980ca8c..58f550b865 100644
--- a/hw/intc/Kconfig
+++ b/hw/intc/Kconfig
@@ -94,3 +94,8 @@ config LOONGARCH_IPI
 config LOONGARCH_PCH_PIC
     bool
     select UNIMP
+
+config LOONGARCH_PCH_MSI
+    select MSI_NONBROKEN
+    bool
+    select UNIMP
diff --git a/hw/intc/loongarch_pch_msi.c b/hw/intc/loongarch_pch_msi.c
new file mode 100644
index 0000000000..74bcdbdb48
--- /dev/null
+++ b/hw/intc/loongarch_pch_msi.c
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * QEMU Loongson 7A1000 msi interrupt controller.
+ *
+ * Copyright (C) 2021 Loongson Technology Corporation Limited
+ */
+
+#include "qemu/osdep.h"
+#include "hw/sysbus.h"
+#include "hw/irq.h"
+#include "hw/intc/loongarch_pch_msi.h"
+#include "hw/intc/loongarch_pch_pic.h"
+#include "hw/pci/msi.h"
+#include "hw/misc/unimp.h"
+#include "migration/vmstate.h"
+#include "trace.h"
+
+static uint64_t loongarch_msi_mem_read(void *opaque, hwaddr addr, unsigned size)
+{
+    return 0;
+}
+
+static void loongarch_msi_mem_write(void *opaque, hwaddr addr,
+                                    uint64_t val, unsigned size)
+{
+    LoongArchPCHMSI *s = LOONGARCH_PCH_MSI(opaque);
+    int irq_num = val & 0xff;
+
+    trace_loongarch_msi_set_irq(irq_num);
+    assert(irq_num < PCH_MSI_IRQ_NUM);
+    qemu_set_irq(s->pch_msi_irq[irq_num], 1);
+}
+
+static const MemoryRegionOps loongarch_pch_msi_ops = {
+    .read  = loongarch_msi_mem_read,
+    .write = loongarch_msi_mem_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
+static void pch_msi_irq_handler(void *opaque, int irq, int level)
+{
+    LoongArchPCHMSI *s = LOONGARCH_PCH_MSI(opaque);
+
+    qemu_set_irq(s->pch_msi_irq[irq], level);
+}
+
+static void loongarch_pch_msi_init(Object *obj)
+{
+    LoongArchPCHMSI *s = LOONGARCH_PCH_MSI(obj);
+    SysBusDevice *sbd = SYS_BUS_DEVICE(obj);
+
+    memory_region_init_io(&s->msi_mmio, obj, &loongarch_pch_msi_ops,
+                          s, TYPE_LOONGARCH_PCH_MSI, 0x8);
+    sysbus_init_mmio(sbd, &s->msi_mmio);
+    msi_nonbroken = true;
+
+    qdev_init_gpio_out(DEVICE(obj), s->pch_msi_irq, PCH_MSI_IRQ_NUM);
+    qdev_init_gpio_in(DEVICE(obj), pch_msi_irq_handler, PCH_MSI_IRQ_NUM);
+}
+
+static const TypeInfo loongarch_pch_msi_info = {
+    .name          = TYPE_LOONGARCH_PCH_MSI,
+    .parent        = TYPE_SYS_BUS_DEVICE,
+    .instance_size = sizeof(LoongArchPCHMSI),
+    .instance_init = loongarch_pch_msi_init,
+};
+
+static void loongarch_pch_msi_register_types(void)
+{
+    type_register_static(&loongarch_pch_msi_info);
+}
+
+type_init(loongarch_pch_msi_register_types)
diff --git a/hw/intc/meson.build b/hw/intc/meson.build
index 03f13f1c49..1d407c046d 100644
--- a/hw/intc/meson.build
+++ b/hw/intc/meson.build
@@ -65,3 +65,4 @@ specific_ss.add(when: 'CONFIG_M68K_IRQC', if_true: files('m68k_irqc.c'))
 specific_ss.add(when: 'CONFIG_NIOS2_VIC', if_true: files('nios2_vic.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_IPI', if_true: files('loongarch_ipi.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_PIC', if_true: files('loongarch_pch_pic.c'))
+specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_MSI', if_true: files('loongarch_pch_msi.c'))
diff --git a/hw/intc/trace-events b/hw/intc/trace-events
index 4cdbc01a07..63c9851923 100644
--- a/hw/intc/trace-events
+++ b/hw/intc/trace-events
@@ -300,3 +300,6 @@ loongarch_pch_pic_high_readw(unsigned size, uint64_t addr, uint64_t val) "size:
 loongarch_pch_pic_high_writew(unsigned size, uint64_t addr, uint64_t val) "size: %u addr: 0x%"PRIx64 "val: 0x%" PRIx64
 loongarch_pch_pic_readb(unsigned size, uint64_t addr, uint64_t val) "size: %u addr: 0x%"PRIx64 "val: 0x%" PRIx64
 loongarch_pch_pic_writeb(unsigned size, uint64_t addr, uint64_t val) "size: %u addr: 0x%"PRIx64 "val: 0x%" PRIx64
+
+# loongarch_pch_msi.c
+loongarch_msi_set_irq(int irq_num) "set msi irq %d"
diff --git a/hw/loongarch/Kconfig b/hw/loongarch/Kconfig
index 2df45f7e8f..d814fc6103 100644
--- a/hw/loongarch/Kconfig
+++ b/hw/loongarch/Kconfig
@@ -4,3 +4,4 @@ config LOONGARCH_VIRT
     select PCI_EXPRESS_GENERIC_BRIDGE
     select LOONGARCH_IPI
     select LOONGARCH_PCH_PIC
+    select LOONGARCH_PCH_MSI
diff --git a/include/hw/intc/loongarch_pch_msi.h b/include/hw/intc/loongarch_pch_msi.h
new file mode 100644
index 0000000000..f668bfca7a
--- /dev/null
+++ b/include/hw/intc/loongarch_pch_msi.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * LoongArch 7A1000 I/O interrupt controller definitions
+ *
+ * Copyright (C) 2021 Loongson Technology Corporation Limited
+ */
+
+#define TYPE_LOONGARCH_PCH_MSI "loongarch_pch_msi"
+OBJECT_DECLARE_SIMPLE_TYPE(LoongArchPCHMSI, LOONGARCH_PCH_MSI)
+
+/* Msi irq start start from 64 to 255 */
+#define PCH_MSI_IRQ_START   64
+#define PCH_MSI_IRQ_END     255
+#define PCH_MSI_IRQ_NUM     192
+
+struct LoongArchPCHMSI {
+    SysBusDevice parent_obj;
+    qemu_irq pch_msi_irq[PCH_MSI_IRQ_NUM];
+    MemoryRegion msi_mmio;
+};
diff --git a/include/hw/pci-host/ls7a.h b/include/hw/pci-host/ls7a.h
index bf80e99ce1..089d3e5438 100644
--- a/include/hw/pci-host/ls7a.h
+++ b/include/hw/pci-host/ls7a.h
@@ -15,6 +15,9 @@
 #include "qemu/range.h"
 #include "qom/object.h"
 
+#define LS7A_PCI_MEM_BASE        0x40000000UL
+#define LS7A_PCI_MEM_SIZE        0x40000000UL
+
 #define LS7A_PCH_REG_BASE       0x10000000UL
 #define LS7A_IOAPIC_REG_BASE    (LS7A_PCH_REG_BASE)
 #define LS7A_PCH_MSI_ADDR_LOW   0x2FF00000UL

From cbff2db1e92f8759db0f0716a41a3e11b18f2eee Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:24 +0800
Subject: [PATCH 841/935] hw/intc: Add LoongArch extioi interrupt
 controller(EIOINTC)

This patch realize the EIOINTC interrupt controller.

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-35-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 hw/intc/Kconfig                    |   3 +
 hw/intc/loongarch_extioi.c         | 312 +++++++++++++++++++++++++++++
 hw/intc/meson.build                |   1 +
 hw/intc/trace-events               |   6 +
 hw/loongarch/Kconfig               |   1 +
 include/hw/intc/loongarch_extioi.h |  62 ++++++
 6 files changed, 385 insertions(+)
 create mode 100644 hw/intc/loongarch_extioi.c
 create mode 100644 include/hw/intc/loongarch_extioi.h

diff --git a/hw/intc/Kconfig b/hw/intc/Kconfig
index 58f550b865..ecd2883ceb 100644
--- a/hw/intc/Kconfig
+++ b/hw/intc/Kconfig
@@ -99,3 +99,6 @@ config LOONGARCH_PCH_MSI
     select MSI_NONBROKEN
     bool
     select UNIMP
+
+config LOONGARCH_EXTIOI
+    bool
diff --git a/hw/intc/loongarch_extioi.c b/hw/intc/loongarch_extioi.c
new file mode 100644
index 0000000000..22803969bc
--- /dev/null
+++ b/hw/intc/loongarch_extioi.c
@@ -0,0 +1,312 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Loongson 3A5000 ext interrupt controller emulation
+ *
+ * Copyright (C) 2021 Loongson Technology Corporation Limited
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/module.h"
+#include "qemu/log.h"
+#include "hw/irq.h"
+#include "hw/sysbus.h"
+#include "hw/loongarch/virt.h"
+#include "hw/qdev-properties.h"
+#include "exec/address-spaces.h"
+#include "hw/intc/loongarch_extioi.h"
+#include "migration/vmstate.h"
+#include "trace.h"
+
+
+static void extioi_update_irq(LoongArchExtIOI *s, int irq, int level)
+{
+    int ipnum, cpu, found, irq_index, irq_mask;
+
+    ipnum = s->sw_ipmap[irq / 32];
+    cpu = s->sw_coremap[irq];
+    irq_index = irq / 32;
+    irq_mask = 1 << (irq & 0x1f);
+
+    if (level) {
+        /* if not enable return false */
+        if (((s->enable[irq_index]) & irq_mask) == 0) {
+            return;
+        }
+        s->coreisr[cpu][irq_index] |= irq_mask;
+        found = find_first_bit(s->sw_isr[cpu][ipnum], EXTIOI_IRQS);
+        set_bit(irq, s->sw_isr[cpu][ipnum]);
+        if (found < EXTIOI_IRQS) {
+            /* other irq is handling, need not update parent irq level */
+            return;
+        }
+    } else {
+        s->coreisr[cpu][irq_index] &= ~irq_mask;
+        clear_bit(irq, s->sw_isr[cpu][ipnum]);
+        found = find_first_bit(s->sw_isr[cpu][ipnum], EXTIOI_IRQS);
+        if (found < EXTIOI_IRQS) {
+            /* other irq is handling, need not update parent irq level */
+            return;
+        }
+    }
+    qemu_set_irq(s->parent_irq[cpu][ipnum], level);
+}
+
+static void extioi_setirq(void *opaque, int irq, int level)
+{
+    LoongArchExtIOI *s = LOONGARCH_EXTIOI(opaque);
+    trace_loongarch_extioi_setirq(irq, level);
+    if (level) {
+        /*
+         * s->isr should be used in vmstate structure,
+         * but it not support 'unsigned long',
+         * so we have to switch it.
+         */
+        set_bit(irq, (unsigned long *)s->isr);
+    } else {
+        clear_bit(irq, (unsigned long *)s->isr);
+    }
+    extioi_update_irq(s, irq, level);
+}
+
+static uint64_t extioi_readw(void *opaque, hwaddr addr, unsigned size)
+{
+    LoongArchExtIOI *s = LOONGARCH_EXTIOI(opaque);
+    unsigned long offset = addr & 0xffff;
+    uint32_t index, cpu, ret = 0;
+
+    switch (offset) {
+    case EXTIOI_NODETYPE_START ... EXTIOI_NODETYPE_END - 1:
+        index = (offset - EXTIOI_NODETYPE_START) >> 2;
+        ret = s->nodetype[index];
+        break;
+    case EXTIOI_IPMAP_START ... EXTIOI_IPMAP_END - 1:
+        index = (offset - EXTIOI_IPMAP_START) >> 2;
+        ret = s->ipmap[index];
+        break;
+    case EXTIOI_ENABLE_START ... EXTIOI_ENABLE_END - 1:
+        index = (offset - EXTIOI_ENABLE_START) >> 2;
+        ret = s->enable[index];
+        break;
+    case EXTIOI_BOUNCE_START ... EXTIOI_BOUNCE_END - 1:
+        index = (offset - EXTIOI_BOUNCE_START) >> 2;
+        ret = s->bounce[index];
+        break;
+    case EXTIOI_COREISR_START ... EXTIOI_COREISR_END - 1:
+        index = ((offset - EXTIOI_COREISR_START) & 0x1f) >> 2;
+        cpu = ((offset - EXTIOI_COREISR_START) >> 8) & 0x3;
+        ret = s->coreisr[cpu][index];
+        break;
+    case EXTIOI_COREMAP_START ... EXTIOI_COREMAP_END - 1:
+        index = (offset - EXTIOI_COREMAP_START) >> 2;
+        ret = s->coremap[index];
+        break;
+    default:
+        break;
+    }
+
+    trace_loongarch_extioi_readw(addr, ret);
+    return ret;
+}
+
+static inline void extioi_enable_irq(LoongArchExtIOI *s, int index,\
+                                     uint32_t mask, int level)
+{
+    uint32_t val;
+    int irq;
+
+    val = mask & s->isr[index];
+    irq = ctz32(val);
+    while (irq != 32) {
+        /*
+         * enable bit change from 0 to 1,
+         * need to update irq by pending bits
+         */
+        extioi_update_irq(s, irq + index * 32, level);
+        val &= ~(1 << irq);
+        irq = ctz32(val);
+    }
+}
+
+static void extioi_writew(void *opaque, hwaddr addr,
+                          uint64_t val, unsigned size)
+{
+    LoongArchExtIOI *s = LOONGARCH_EXTIOI(opaque);
+    int i, cpu, index, old_data, irq;
+    uint32_t offset;
+
+    trace_loongarch_extioi_writew(addr, val);
+    offset = addr & 0xffff;
+
+    switch (offset) {
+    case EXTIOI_NODETYPE_START ... EXTIOI_NODETYPE_END - 1:
+        index = (offset - EXTIOI_NODETYPE_START) >> 2;
+        s->nodetype[index] = val;
+        break;
+    case EXTIOI_IPMAP_START ... EXTIOI_IPMAP_END - 1:
+        /*
+         * ipmap cannot be set at runtime, can be set only at the beginning
+         * of intr driver, need not update upper irq level
+         */
+        index = (offset - EXTIOI_IPMAP_START) >> 2;
+        s->ipmap[index] = val;
+        /*
+         * loongarch only support little endian,
+         * so we paresd the value with little endian.
+         */
+        val = cpu_to_le64(val);
+        for (i = 0; i < 4; i++) {
+            uint8_t ipnum;
+            ipnum = val & 0xff;
+            ipnum = ctz32(ipnum);
+            ipnum = (ipnum >= 4) ? 0 : ipnum;
+            s->sw_ipmap[index * 4 + i] = ipnum;
+            val = val >> 8;
+        }
+
+        break;
+    case EXTIOI_ENABLE_START ... EXTIOI_ENABLE_END - 1:
+        index = (offset - EXTIOI_ENABLE_START) >> 2;
+        old_data = s->enable[index];
+        s->enable[index] = val;
+
+        /* unmask irq */
+        val = s->enable[index] & ~old_data;
+        extioi_enable_irq(s, index, val, 1);
+
+        /* mask irq */
+        val = ~s->enable[index] & old_data;
+        extioi_enable_irq(s, index, val, 0);
+        break;
+    case EXTIOI_BOUNCE_START ... EXTIOI_BOUNCE_END - 1:
+        /* do not emulate hw bounced irq routing */
+        index = (offset - EXTIOI_BOUNCE_START) >> 2;
+        s->bounce[index] = val;
+        break;
+    case EXTIOI_COREISR_START ... EXTIOI_COREISR_END - 1:
+        index = ((offset - EXTIOI_COREISR_START) & 0x1f) >> 2;
+        cpu = ((offset - EXTIOI_COREISR_START) >> 8) & 0x3;
+        old_data = s->coreisr[cpu][index];
+        s->coreisr[cpu][index] = old_data & ~val;
+        /* write 1 to clear interrrupt */
+        old_data &= val;
+        irq = ctz32(old_data);
+        while (irq != 32) {
+            extioi_update_irq(s, irq + index * 32, 0);
+            old_data &= ~(1 << irq);
+            irq = ctz32(old_data);
+        }
+        break;
+    case EXTIOI_COREMAP_START ... EXTIOI_COREMAP_END - 1:
+        irq = offset - EXTIOI_COREMAP_START;
+        index = irq / 4;
+        s->coremap[index] = val;
+        /*
+         * loongarch only support little endian,
+         * so we paresd the value with little endian.
+         */
+        val = cpu_to_le64(val);
+
+        for (i = 0; i < 4; i++) {
+            cpu = val & 0xff;
+            cpu = ctz32(cpu);
+            cpu = (cpu >= 4) ? 0 : cpu;
+            val = val >> 8;
+
+            if (s->sw_coremap[irq + i] == cpu) {
+                continue;
+            }
+
+            if (test_bit(irq, (unsigned long *)s->isr)) {
+                /*
+                 * lower irq at old cpu and raise irq at new cpu
+                 */
+                extioi_update_irq(s, irq + i, 0);
+                s->sw_coremap[irq + i] = cpu;
+                extioi_update_irq(s, irq + i, 1);
+            } else {
+                s->sw_coremap[irq + i] = cpu;
+            }
+        }
+        break;
+    default:
+        break;
+    }
+}
+
+static const MemoryRegionOps extioi_ops = {
+    .read = extioi_readw,
+    .write = extioi_writew,
+    .impl.min_access_size = 4,
+    .impl.max_access_size = 4,
+    .valid.min_access_size = 4,
+    .valid.max_access_size = 8,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
+static const VMStateDescription vmstate_loongarch_extioi = {
+    .name = TYPE_LOONGARCH_EXTIOI,
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT32_ARRAY(bounce, LoongArchExtIOI, EXTIOI_IRQS_GROUP_COUNT),
+        VMSTATE_UINT32_2DARRAY(coreisr, LoongArchExtIOI, LOONGARCH_MAX_VCPUS,
+                               EXTIOI_IRQS_GROUP_COUNT),
+        VMSTATE_UINT32_ARRAY(nodetype, LoongArchExtIOI,
+                             EXTIOI_IRQS_NODETYPE_COUNT / 2),
+        VMSTATE_UINT32_ARRAY(enable, LoongArchExtIOI, EXTIOI_IRQS / 32),
+        VMSTATE_UINT32_ARRAY(isr, LoongArchExtIOI, EXTIOI_IRQS / 32),
+        VMSTATE_UINT32_ARRAY(ipmap, LoongArchExtIOI, EXTIOI_IRQS_IPMAP_SIZE / 4),
+        VMSTATE_UINT32_ARRAY(coremap, LoongArchExtIOI, EXTIOI_IRQS / 4),
+        VMSTATE_UINT8_ARRAY(sw_ipmap, LoongArchExtIOI, EXTIOI_IRQS_IPMAP_SIZE),
+        VMSTATE_UINT8_ARRAY(sw_coremap, LoongArchExtIOI, EXTIOI_IRQS),
+
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static void loongarch_extioi_instance_init(Object *obj)
+{
+    SysBusDevice *dev = SYS_BUS_DEVICE(obj);
+    LoongArchExtIOI *s = LOONGARCH_EXTIOI(obj);
+    int i, cpu, pin;
+
+    for (i = 0; i < EXTIOI_IRQS; i++) {
+        sysbus_init_irq(SYS_BUS_DEVICE(dev), &s->irq[i]);
+    }
+
+    qdev_init_gpio_in(DEVICE(obj), extioi_setirq, EXTIOI_IRQS);
+
+    for (cpu = 0; cpu < LOONGARCH_MAX_VCPUS; cpu++) {
+        memory_region_init_io(&s->extioi_iocsr_mem[cpu], OBJECT(s), &extioi_ops,
+                              s, "extioi_iocsr", 0x900);
+        sysbus_init_mmio(SYS_BUS_DEVICE(dev), &s->extioi_iocsr_mem[cpu]);
+        for (pin = 0; pin < LS3A_INTC_IP; pin++) {
+            qdev_init_gpio_out(DEVICE(obj), &s->parent_irq[cpu][pin], 1);
+        }
+    }
+    memory_region_init_io(&s->extioi_system_mem, OBJECT(s), &extioi_ops,
+                          s, "extioi_system_mem", 0x900);
+    sysbus_init_mmio(SYS_BUS_DEVICE(dev), &s->extioi_system_mem);
+}
+
+static void loongarch_extioi_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+
+    dc->vmsd = &vmstate_loongarch_extioi;
+}
+
+static const TypeInfo loongarch_extioi_info = {
+    .name          = TYPE_LOONGARCH_EXTIOI,
+    .parent        = TYPE_SYS_BUS_DEVICE,
+    .instance_init = loongarch_extioi_instance_init,
+    .instance_size = sizeof(struct LoongArchExtIOI),
+    .class_init    = loongarch_extioi_class_init,
+};
+
+static void loongarch_extioi_register_types(void)
+{
+    type_register_static(&loongarch_extioi_info);
+}
+
+type_init(loongarch_extioi_register_types)
diff --git a/hw/intc/meson.build b/hw/intc/meson.build
index 1d407c046d..bcbf22ff51 100644
--- a/hw/intc/meson.build
+++ b/hw/intc/meson.build
@@ -66,3 +66,4 @@ specific_ss.add(when: 'CONFIG_NIOS2_VIC', if_true: files('nios2_vic.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_IPI', if_true: files('loongarch_ipi.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_PIC', if_true: files('loongarch_pch_pic.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_MSI', if_true: files('loongarch_pch_msi.c'))
+specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI', if_true: files('loongarch_extioi.c'))
diff --git a/hw/intc/trace-events b/hw/intc/trace-events
index 63c9851923..0a90c1cdec 100644
--- a/hw/intc/trace-events
+++ b/hw/intc/trace-events
@@ -303,3 +303,9 @@ loongarch_pch_pic_writeb(unsigned size, uint64_t addr, uint64_t val) "size: %u a
 
 # loongarch_pch_msi.c
 loongarch_msi_set_irq(int irq_num) "set msi irq %d"
+
+# loongarch_extioi.c
+loongarch_extioi_setirq(int irq, int level) "set extirq irq %d level %d"
+loongarch_extioi_readw(uint64_t addr, uint32_t val) "addr: 0x%"PRIx64 "val: 0x%x"
+loongarch_extioi_writew(uint64_t addr, uint64_t val) "addr: 0x%"PRIx64 "val: 0x%" PRIx64
+
diff --git a/hw/loongarch/Kconfig b/hw/loongarch/Kconfig
index d814fc6103..f779087416 100644
--- a/hw/loongarch/Kconfig
+++ b/hw/loongarch/Kconfig
@@ -5,3 +5,4 @@ config LOONGARCH_VIRT
     select LOONGARCH_IPI
     select LOONGARCH_PCH_PIC
     select LOONGARCH_PCH_MSI
+    select LOONGARCH_EXTIOI
diff --git a/include/hw/intc/loongarch_extioi.h b/include/hw/intc/loongarch_extioi.h
new file mode 100644
index 0000000000..15b8c999f6
--- /dev/null
+++ b/include/hw/intc/loongarch_extioi.h
@@ -0,0 +1,62 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * LoongArch 3A5000 ext interrupt controller definitions
+ *
+ * Copyright (C) 2021 Loongson Technology Corporation Limited
+ */
+
+#include "hw/sysbus.h"
+#include "hw/loongarch/virt.h"
+
+#ifndef LOONGARCH_EXTIOI_H
+#define LOONGARCH_EXTIOI_H
+
+#define LS3A_INTC_IP               8
+#define EXTIOI_IRQS                (256)
+#define EXTIOI_IRQS_BITMAP_SIZE    (256 / 8)
+/* map to ipnum per 32 irqs */
+#define EXTIOI_IRQS_IPMAP_SIZE     (256 / 32)
+#define EXTIOI_IRQS_COREMAP_SIZE   256
+#define EXTIOI_IRQS_NODETYPE_COUNT  16
+#define EXTIOI_IRQS_GROUP_COUNT    8
+
+#define APIC_OFFSET                  0x400
+#define APIC_BASE                    (0x1000ULL + APIC_OFFSET)
+
+#define EXTIOI_NODETYPE_START        (0x4a0 - APIC_OFFSET)
+#define EXTIOI_NODETYPE_END          (0x4c0 - APIC_OFFSET)
+#define EXTIOI_IPMAP_START           (0x4c0 - APIC_OFFSET)
+#define EXTIOI_IPMAP_END             (0x4c8 - APIC_OFFSET)
+#define EXTIOI_ENABLE_START          (0x600 - APIC_OFFSET)
+#define EXTIOI_ENABLE_END            (0x620 - APIC_OFFSET)
+#define EXTIOI_BOUNCE_START          (0x680 - APIC_OFFSET)
+#define EXTIOI_BOUNCE_END            (0x6a0 - APIC_OFFSET)
+#define EXTIOI_ISR_START             (0x700 - APIC_OFFSET)
+#define EXTIOI_ISR_END               (0x720 - APIC_OFFSET)
+#define EXTIOI_COREISR_START         (0x800 - APIC_OFFSET)
+#define EXTIOI_COREISR_END           (0xB20 - APIC_OFFSET)
+#define EXTIOI_COREMAP_START         (0xC00 - APIC_OFFSET)
+#define EXTIOI_COREMAP_END           (0xD00 - APIC_OFFSET)
+
+#define TYPE_LOONGARCH_EXTIOI "loongarch.extioi"
+OBJECT_DECLARE_SIMPLE_TYPE(LoongArchExtIOI, LOONGARCH_EXTIOI)
+struct LoongArchExtIOI {
+    SysBusDevice parent_obj;
+    /* hardware state */
+    uint32_t nodetype[EXTIOI_IRQS_NODETYPE_COUNT / 2];
+    uint32_t bounce[EXTIOI_IRQS_GROUP_COUNT];
+    uint32_t isr[EXTIOI_IRQS / 32];
+    uint32_t coreisr[LOONGARCH_MAX_VCPUS][EXTIOI_IRQS_GROUP_COUNT];
+    uint32_t enable[EXTIOI_IRQS / 32];
+    uint32_t ipmap[EXTIOI_IRQS_IPMAP_SIZE / 4];
+    uint32_t coremap[EXTIOI_IRQS / 4];
+    uint32_t sw_pending[EXTIOI_IRQS / 32];
+    DECLARE_BITMAP(sw_isr[LOONGARCH_MAX_VCPUS][LS3A_INTC_IP], EXTIOI_IRQS);
+    uint8_t  sw_ipmap[EXTIOI_IRQS_IPMAP_SIZE];
+    uint8_t  sw_coremap[EXTIOI_IRQS];
+    qemu_irq parent_irq[LOONGARCH_MAX_VCPUS][LS3A_INTC_IP];
+    qemu_irq irq[EXTIOI_IRQS];
+    MemoryRegion extioi_iocsr_mem[LOONGARCH_MAX_VCPUS];
+    MemoryRegion extioi_system_mem;
+};
+#endif /* LOONGARCH_EXTIOI_H */

From 69d9c74fa9143b58073772cd5a8a7dd2b1d8ffcf Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:25 +0800
Subject: [PATCH 842/935] hw/loongarch: Add irq hierarchy for the system

This patch add the irq hierarchy for the virt board.

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-36-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 hw/loongarch/loongson3.c | 104 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 104 insertions(+)

diff --git a/hw/loongarch/loongson3.c b/hw/loongarch/loongson3.c
index 7df32f777e..7a5c61e2df 100644
--- a/hw/loongarch/loongson3.c
+++ b/hw/loongarch/loongson3.c
@@ -16,8 +16,110 @@
 #include "sysemu/rtc.h"
 #include "hw/loongarch/virt.h"
 #include "exec/address-spaces.h"
+#include "hw/intc/loongarch_ipi.h"
+#include "hw/intc/loongarch_extioi.h"
+#include "hw/intc/loongarch_pch_pic.h"
+#include "hw/intc/loongarch_pch_msi.h"
+#include "hw/pci-host/ls7a.h"
+
 #include "target/loongarch/cpu.h"
 
+static void loongarch_irq_init(LoongArchMachineState *lams)
+{
+    MachineState *ms = MACHINE(lams);
+    DeviceState *pch_pic, *pch_msi, *cpudev;
+    DeviceState *ipi, *extioi;
+    SysBusDevice *d;
+    LoongArchCPU *lacpu;
+    CPULoongArchState *env;
+    CPUState *cpu_state;
+    int cpu, pin, i;
+
+    ipi = qdev_new(TYPE_LOONGARCH_IPI);
+    sysbus_realize_and_unref(SYS_BUS_DEVICE(ipi), &error_fatal);
+
+    extioi = qdev_new(TYPE_LOONGARCH_EXTIOI);
+    sysbus_realize_and_unref(SYS_BUS_DEVICE(extioi), &error_fatal);
+
+    /*
+     * The connection of interrupts:
+     *   +-----+    +---------+     +-------+
+     *   | IPI |--> | CPUINTC | <-- | Timer |
+     *   +-----+    +---------+     +-------+
+     *                  ^
+     *                  |
+     *            +---------+
+     *            | EIOINTC |
+     *            +---------+
+     *             ^       ^
+     *             |       |
+     *      +---------+ +---------+
+     *      | PCH-PIC | | PCH-MSI |
+     *      +---------+ +---------+
+     *        ^      ^          ^
+     *        |      |          |
+     * +--------+ +---------+ +---------+
+     * | UARTs  | | Devices | | Devices |
+     * +--------+ +---------+ +---------+
+     */
+    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
+        cpu_state = qemu_get_cpu(cpu);
+        cpudev = DEVICE(cpu_state);
+        lacpu = LOONGARCH_CPU(cpu_state);
+        env = &(lacpu->env);
+
+        /* connect ipi irq to cpu irq */
+        qdev_connect_gpio_out(ipi, cpu, qdev_get_gpio_in(cpudev, IRQ_IPI));
+        /* IPI iocsr memory region */
+        memory_region_add_subregion(&env->system_iocsr, SMP_IPI_MAILBOX,
+                                    sysbus_mmio_get_region(SYS_BUS_DEVICE(ipi),
+                                    cpu));
+        /* extioi iocsr memory region */
+        memory_region_add_subregion(&env->system_iocsr, APIC_BASE,
+                                sysbus_mmio_get_region(SYS_BUS_DEVICE(extioi),
+                                cpu));
+    }
+
+    /*
+     * connect ext irq to the cpu irq
+     * cpu_pin[9:2] <= intc_pin[7:0]
+     */
+    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
+        cpudev = DEVICE(qemu_get_cpu(cpu));
+        for (pin = 0; pin < LS3A_INTC_IP; pin++) {
+            qdev_connect_gpio_out(extioi, (cpu * 8 + pin),
+                                  qdev_get_gpio_in(cpudev, pin + 2));
+        }
+    }
+
+    pch_pic = qdev_new(TYPE_LOONGARCH_PCH_PIC);
+    d = SYS_BUS_DEVICE(pch_pic);
+    sysbus_realize_and_unref(d, &error_fatal);
+    memory_region_add_subregion(get_system_memory(), LS7A_IOAPIC_REG_BASE,
+                            sysbus_mmio_get_region(d, 0));
+    memory_region_add_subregion(get_system_memory(),
+                            LS7A_IOAPIC_REG_BASE + PCH_PIC_ROUTE_ENTRY_OFFSET,
+                            sysbus_mmio_get_region(d, 1));
+    memory_region_add_subregion(get_system_memory(),
+                            LS7A_IOAPIC_REG_BASE + PCH_PIC_INT_STATUS_LO,
+                            sysbus_mmio_get_region(d, 2));
+
+    /* Connect 64 pch_pic irqs to extioi */
+    for (int i = 0; i < PCH_PIC_IRQ_NUM; i++) {
+        qdev_connect_gpio_out(DEVICE(d), i, qdev_get_gpio_in(extioi, i));
+    }
+
+    pch_msi = qdev_new(TYPE_LOONGARCH_PCH_MSI);
+    d = SYS_BUS_DEVICE(pch_msi);
+    sysbus_realize_and_unref(d, &error_fatal);
+    sysbus_mmio_map(d, 0, LS7A_PCH_MSI_ADDR_LOW);
+    for (i = 0; i < PCH_MSI_IRQ_NUM; i++) {
+        /* Connect 192 pch_msi irqs to extioi */
+        qdev_connect_gpio_out(DEVICE(d), i,
+                              qdev_get_gpio_in(extioi, i + PCH_MSI_IRQ_START));
+    }
+}
+
 static void loongarch_init(MachineState *machine)
 {
     const char *cpu_model = machine->cpu_type;
@@ -63,6 +165,8 @@ static void loongarch_init(MachineState *machine)
                              get_system_io(), 0, LOONGARCH_ISA_IO_SIZE);
     memory_region_add_subregion(address_space_mem, LOONGARCH_ISA_IO_BASE,
                                 &lams->isa_io);
+    /* Initialize the IO interrupt subsystem */
+    loongarch_irq_init(lams);
 }
 
 static void loongarch_class_init(ObjectClass *oc, void *data)

From 256309e1884898a9cedb44abab0f15c4843e06f6 Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:26 +0800
Subject: [PATCH 843/935] Enable common virtio pci support for LoongArch

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-37-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 softmmu/qdev-monitor.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/softmmu/qdev-monitor.c b/softmmu/qdev-monitor.c
index 12fe60c467..bb5897fc76 100644
--- a/softmmu/qdev-monitor.c
+++ b/softmmu/qdev-monitor.c
@@ -60,7 +60,8 @@ typedef struct QDevAlias
                               QEMU_ARCH_HPPA | QEMU_ARCH_I386 | \
                               QEMU_ARCH_MIPS | QEMU_ARCH_PPC |  \
                               QEMU_ARCH_RISCV | QEMU_ARCH_SH4 | \
-                              QEMU_ARCH_SPARC | QEMU_ARCH_XTENSA)
+                              QEMU_ARCH_SPARC | QEMU_ARCH_XTENSA | \
+                              QEMU_ARCH_LOONGARCH)
 #define QEMU_ARCH_VIRTIO_CCW (QEMU_ARCH_S390X)
 #define QEMU_ARCH_VIRTIO_MMIO (QEMU_ARCH_M68K)
 

From dc93b8df8a166d2df0c007affd49bcfd93e77391 Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:27 +0800
Subject: [PATCH 844/935] hw/loongarch: Add some devices support for 3A5000.

1.Add uart,virtio-net,vga and usb for 3A5000.
2.Add irq set and map for the pci host. Non pci device
use irq 0-16, pci device use 16-64.
3.Add some unimplented device to emulate guest unused
memory space.

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-38-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 hw/loongarch/Kconfig       |  7 ++++
 hw/loongarch/loongson3.c   | 77 ++++++++++++++++++++++++++++++++++++++
 include/hw/pci-host/ls7a.h |  7 ++++
 3 files changed, 91 insertions(+)

diff --git a/hw/loongarch/Kconfig b/hw/loongarch/Kconfig
index f779087416..8552ff4bee 100644
--- a/hw/loongarch/Kconfig
+++ b/hw/loongarch/Kconfig
@@ -2,6 +2,13 @@ config LOONGARCH_VIRT
     bool
     select PCI
     select PCI_EXPRESS_GENERIC_BRIDGE
+    imply VGA_PCI
+    imply VIRTIO_VGA
+    imply PCI_DEVICES
+    select ISA_BUS
+    select SERIAL
+    select SERIAL_ISA
+    select VIRTIO_PCI
     select LOONGARCH_IPI
     select LOONGARCH_PCH_PIC
     select LOONGARCH_PCH_MSI
diff --git a/hw/loongarch/loongson3.c b/hw/loongarch/loongson3.c
index 7a5c61e2df..7bc17113dc 100644
--- a/hw/loongarch/loongson3.c
+++ b/hw/loongarch/loongson3.c
@@ -9,6 +9,7 @@
 #include "qemu/datadir.h"
 #include "qapi/error.h"
 #include "hw/boards.h"
+#include "hw/char/serial.h"
 #include "sysemu/sysemu.h"
 #include "sysemu/qtest.h"
 #include "sysemu/runstate.h"
@@ -16,14 +17,88 @@
 #include "sysemu/rtc.h"
 #include "hw/loongarch/virt.h"
 #include "exec/address-spaces.h"
+#include "hw/irq.h"
+#include "net/net.h"
 #include "hw/intc/loongarch_ipi.h"
 #include "hw/intc/loongarch_extioi.h"
 #include "hw/intc/loongarch_pch_pic.h"
 #include "hw/intc/loongarch_pch_msi.h"
 #include "hw/pci-host/ls7a.h"
+#include "hw/pci-host/gpex.h"
+#include "hw/misc/unimp.h"
 
 #include "target/loongarch/cpu.h"
 
+static void loongarch_devices_init(DeviceState *pch_pic)
+{
+    DeviceState *gpex_dev;
+    SysBusDevice *d;
+    PCIBus *pci_bus;
+    MemoryRegion *ecam_alias, *ecam_reg, *pio_alias, *pio_reg;
+    MemoryRegion *mmio_alias, *mmio_reg;
+    int i;
+
+    gpex_dev = qdev_new(TYPE_GPEX_HOST);
+    d = SYS_BUS_DEVICE(gpex_dev);
+    sysbus_realize_and_unref(d, &error_fatal);
+    pci_bus = PCI_HOST_BRIDGE(gpex_dev)->bus;
+
+    /* Map only part size_ecam bytes of ECAM space */
+    ecam_alias = g_new0(MemoryRegion, 1);
+    ecam_reg = sysbus_mmio_get_region(d, 0);
+    memory_region_init_alias(ecam_alias, OBJECT(gpex_dev), "pcie-ecam",
+                             ecam_reg, 0, LS_PCIECFG_SIZE);
+    memory_region_add_subregion(get_system_memory(), LS_PCIECFG_BASE,
+                                ecam_alias);
+
+    /* Map PCI mem space */
+    mmio_alias = g_new0(MemoryRegion, 1);
+    mmio_reg = sysbus_mmio_get_region(d, 1);
+    memory_region_init_alias(mmio_alias, OBJECT(gpex_dev), "pcie-mmio",
+                             mmio_reg, LS7A_PCI_MEM_BASE, LS7A_PCI_MEM_SIZE);
+    memory_region_add_subregion(get_system_memory(), LS7A_PCI_MEM_BASE,
+                                mmio_alias);
+
+    /* Map PCI IO port space. */
+    pio_alias = g_new0(MemoryRegion, 1);
+    pio_reg = sysbus_mmio_get_region(d, 2);
+    memory_region_init_alias(pio_alias, OBJECT(gpex_dev), "pcie-io", pio_reg,
+                             LS7A_PCI_IO_OFFSET, LS7A_PCI_IO_SIZE);
+    memory_region_add_subregion(get_system_memory(), LS7A_PCI_IO_BASE,
+                                pio_alias);
+
+    for (i = 0; i < GPEX_NUM_IRQS; i++) {
+        sysbus_connect_irq(d, i,
+                           qdev_get_gpio_in(pch_pic, 16 + i));
+        gpex_set_irq_num(GPEX_HOST(gpex_dev), i, 16 + i);
+    }
+
+    serial_mm_init(get_system_memory(), LS7A_UART_BASE, 0,
+                   qdev_get_gpio_in(pch_pic,
+                                    LS7A_UART_IRQ - PCH_PIC_IRQ_OFFSET),
+                   115200, serial_hd(0), DEVICE_LITTLE_ENDIAN);
+
+    /* Network init */
+    for (i = 0; i < nb_nics; i++) {
+        NICInfo *nd = &nd_table[i];
+
+        if (!nd->model) {
+            nd->model = g_strdup("virtio");
+        }
+
+        pci_nic_init_nofail(nd, pci_bus, nd->model, NULL);
+    }
+
+    /* VGA setup */
+    pci_vga_init(pci_bus);
+
+    /*
+     * There are some invalid guest memory access.
+     * Create some unimplemented devices to emulate this.
+     */
+    create_unimplemented_device("pci-dma-cfg", 0x1001041c, 0x4);
+}
+
 static void loongarch_irq_init(LoongArchMachineState *lams)
 {
     MachineState *ms = MACHINE(lams);
@@ -118,6 +193,8 @@ static void loongarch_irq_init(LoongArchMachineState *lams)
         qdev_connect_gpio_out(DEVICE(d), i,
                               qdev_get_gpio_in(extioi, i + PCH_MSI_IRQ_START));
     }
+
+    loongarch_devices_init(pch_pic);
 }
 
 static void loongarch_init(MachineState *machine)
diff --git a/include/hw/pci-host/ls7a.h b/include/hw/pci-host/ls7a.h
index 089d3e5438..5c653527f0 100644
--- a/include/hw/pci-host/ls7a.h
+++ b/include/hw/pci-host/ls7a.h
@@ -17,6 +17,11 @@
 
 #define LS7A_PCI_MEM_BASE        0x40000000UL
 #define LS7A_PCI_MEM_SIZE        0x40000000UL
+#define LS7A_PCI_IO_OFFSET      0x4000
+#define LS_PCIECFG_BASE         0x20000000
+#define LS_PCIECFG_SIZE         0x08000000
+#define LS7A_PCI_IO_BASE        0x18004000UL
+#define LS7A_PCI_IO_SIZE        0xC000
 
 #define LS7A_PCH_REG_BASE       0x10000000UL
 #define LS7A_IOAPIC_REG_BASE    (LS7A_PCH_REG_BASE)
@@ -30,4 +35,6 @@
 #define PCH_PIC_IRQ_OFFSET      64
 #define LS7A_DEVICE_IRQS        16
 #define LS7A_PCI_IRQS           48
+#define LS7A_UART_IRQ           (PCH_PIC_IRQ_OFFSET + 2)
+#define LS7A_UART_BASE          0x1fe001e0
 #endif

From c117f68a46f4b3fa363635fd610873707d4899fa Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:28 +0800
Subject: [PATCH 845/935] hw/loongarch: Add LoongArch ls7a rtc device support

This patch add ls7a rtc device support.

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-39-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 MAINTAINERS                |   1 +
 hw/loongarch/Kconfig       |   1 +
 hw/loongarch/loongson3.c   |   3 +
 hw/rtc/Kconfig             |   3 +
 hw/rtc/ls7a_rtc.c          | 528 +++++++++++++++++++++++++++++++++++++
 hw/rtc/meson.build         |   1 +
 include/hw/pci-host/ls7a.h |   4 +
 7 files changed, 541 insertions(+)
 create mode 100644 hw/rtc/ls7a_rtc.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 3ce5674da5..e8938db694 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1136,6 +1136,7 @@ F: include/hw/loongarch/virt.h
 F: include/hw/intc/loongarch_*.h
 F: hw/intc/loongarch_*.c
 F: include/hw/pci-host/ls7a.h
+F: hw/rtc/ls7a_rtc.c
 
 M68K Machines
 -------------
diff --git a/hw/loongarch/Kconfig b/hw/loongarch/Kconfig
index 8552ff4bee..35b6680772 100644
--- a/hw/loongarch/Kconfig
+++ b/hw/loongarch/Kconfig
@@ -13,3 +13,4 @@ config LOONGARCH_VIRT
     select LOONGARCH_PCH_PIC
     select LOONGARCH_PCH_MSI
     select LOONGARCH_EXTIOI
+    select LS7A_RTC
diff --git a/hw/loongarch/loongson3.c b/hw/loongarch/loongson3.c
index 7bc17113dc..95984c9086 100644
--- a/hw/loongarch/loongson3.c
+++ b/hw/loongarch/loongson3.c
@@ -97,6 +97,9 @@ static void loongarch_devices_init(DeviceState *pch_pic)
      * Create some unimplemented devices to emulate this.
      */
     create_unimplemented_device("pci-dma-cfg", 0x1001041c, 0x4);
+    sysbus_create_simple("ls7a_rtc", LS7A_RTC_REG_BASE,
+                         qdev_get_gpio_in(pch_pic,
+                         LS7A_RTC_IRQ - PCH_PIC_IRQ_OFFSET));
 }
 
 static void loongarch_irq_init(LoongArchMachineState *lams)
diff --git a/hw/rtc/Kconfig b/hw/rtc/Kconfig
index 730c272bc5..d0d8dda084 100644
--- a/hw/rtc/Kconfig
+++ b/hw/rtc/Kconfig
@@ -27,3 +27,6 @@ config SUN4V_RTC
 
 config GOLDFISH_RTC
     bool
+
+config LS7A_RTC
+    bool
diff --git a/hw/rtc/ls7a_rtc.c b/hw/rtc/ls7a_rtc.c
new file mode 100644
index 0000000000..fe6710310f
--- /dev/null
+++ b/hw/rtc/ls7a_rtc.c
@@ -0,0 +1,528 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Loongarch LS7A Real Time Clock emulation
+ *
+ * Copyright (C) 2021 Loongson Technology Corporation Limited
+ */
+
+#include "qemu/osdep.h"
+#include "hw/sysbus.h"
+#include "hw/irq.h"
+#include "include/hw/register.h"
+#include "qemu/timer.h"
+#include "sysemu/sysemu.h"
+#include "qemu/cutils.h"
+#include "qemu/log.h"
+#include "migration/vmstate.h"
+#include "hw/misc/unimp.h"
+#include "sysemu/rtc.h"
+#include "hw/registerfields.h"
+
+#define SYS_TOYTRIM        0x20
+#define SYS_TOYWRITE0      0x24
+#define SYS_TOYWRITE1      0x28
+#define SYS_TOYREAD0       0x2C
+#define SYS_TOYREAD1       0x30
+#define SYS_TOYMATCH0      0x34
+#define SYS_TOYMATCH1      0x38
+#define SYS_TOYMATCH2      0x3C
+#define SYS_RTCCTRL        0x40
+#define SYS_RTCTRIM        0x60
+#define SYS_RTCWRTIE0      0x64
+#define SYS_RTCREAD0       0x68
+#define SYS_RTCMATCH0      0x6C
+#define SYS_RTCMATCH1      0x70
+#define SYS_RTCMATCH2      0x74
+
+#define LS7A_RTC_FREQ     32768
+#define TIMER_NUMS        3
+/*
+ * Shift bits and filed mask
+ */
+
+FIELD(TOY, MON, 26, 6)
+FIELD(TOY, DAY, 21, 5)
+FIELD(TOY, HOUR, 16, 5)
+FIELD(TOY, MIN, 10, 6)
+FIELD(TOY, SEC, 4, 6)
+FIELD(TOY, MSEC, 0, 4)
+
+FIELD(TOY_MATCH, YEAR, 26, 6)
+FIELD(TOY_MATCH, MON, 22, 4)
+FIELD(TOY_MATCH, DAY, 17, 5)
+FIELD(TOY_MATCH, HOUR, 12, 5)
+FIELD(TOY_MATCH, MIN, 6, 6)
+FIELD(TOY_MATCH, SEC, 0, 6)
+
+FIELD(RTC_CTRL, RTCEN, 13, 1)
+FIELD(RTC_CTRL, TOYEN, 11, 1)
+FIELD(RTC_CTRL, EO, 8, 1)
+
+#define TYPE_LS7A_RTC "ls7a_rtc"
+OBJECT_DECLARE_SIMPLE_TYPE(LS7ARtcState, LS7A_RTC)
+
+struct LS7ARtcState {
+    SysBusDevice parent_obj;
+
+    MemoryRegion iomem;
+    /*
+     * Needed to preserve the tick_count across migration, even if the
+     * absolute value of the rtc_clock is different on the source and
+     * destination.
+     */
+    int64_t offset_toy;
+    int64_t offset_rtc;
+    uint64_t save_toy_mon;
+    uint64_t save_toy_year;
+    uint64_t save_rtc;
+    int64_t data;
+    int tidx;
+    uint32_t toymatch[3];
+    uint32_t toytrim;
+    uint32_t cntrctl;
+    uint32_t rtctrim;
+    uint32_t rtccount;
+    uint32_t rtcmatch[3];
+    QEMUTimer *toy_timer[TIMER_NUMS];
+    QEMUTimer *rtc_timer[TIMER_NUMS];
+    qemu_irq irq;
+};
+
+/* switch nanoseconds time to rtc ticks */
+static inline uint64_t ls7a_rtc_ticks(void)
+{
+    return qemu_clock_get_ns(rtc_clock) * LS7A_RTC_FREQ / NANOSECONDS_PER_SECOND;
+}
+
+/* switch rtc ticks to nanoseconds */
+static inline uint64_t ticks_to_ns(uint64_t ticks)
+{
+    return ticks * NANOSECONDS_PER_SECOND / LS7A_RTC_FREQ;
+}
+
+static inline bool toy_enabled(LS7ARtcState *s)
+{
+    return FIELD_EX32(s->cntrctl, RTC_CTRL, TOYEN) &&
+           FIELD_EX32(s->cntrctl, RTC_CTRL, EO);
+}
+
+static inline bool rtc_enabled(LS7ARtcState *s)
+{
+    return FIELD_EX32(s->cntrctl, RTC_CTRL, RTCEN) &&
+           FIELD_EX32(s->cntrctl, RTC_CTRL, EO);
+}
+
+/* parse toy value to struct tm */
+static inline void toy_val_to_time_mon(uint64_t toy_val, struct tm *tm)
+{
+    tm->tm_sec = FIELD_EX32(toy_val, TOY, SEC);
+    tm->tm_min = FIELD_EX32(toy_val, TOY, MIN);
+    tm->tm_hour = FIELD_EX32(toy_val, TOY, HOUR);
+    tm->tm_mday = FIELD_EX32(toy_val, TOY, DAY);
+    tm->tm_mon = FIELD_EX32(toy_val, TOY, MON) - 1;
+}
+
+static inline void toy_val_to_time_year(uint64_t toy_year, struct tm *tm)
+{
+    tm->tm_year = toy_year;
+}
+
+/* parse struct tm to toy value */
+static inline uint64_t toy_time_to_val_mon(struct tm tm)
+{
+    uint64_t val = 0;
+
+    val = FIELD_DP32(val, TOY, MON, tm.tm_mon + 1);
+    val = FIELD_DP32(val, TOY, DAY, tm.tm_mday);
+    val = FIELD_DP32(val, TOY, HOUR, tm.tm_hour);
+    val = FIELD_DP32(val, TOY, MIN, tm.tm_min);
+    val = FIELD_DP32(val, TOY, SEC, tm.tm_sec);
+    return val;
+}
+
+static inline uint64_t toy_time_to_val_year(struct tm tm)
+{
+    uint64_t year;
+
+    year = tm.tm_year;
+    return year;
+}
+
+static inline void toymatch_val_to_time(uint64_t val, struct tm *tm)
+{
+    tm->tm_sec = FIELD_EX32(val, TOY_MATCH, SEC);
+    tm->tm_min = FIELD_EX32(val, TOY_MATCH, MIN);
+    tm->tm_hour = FIELD_EX32(val, TOY_MATCH, HOUR);
+    tm->tm_mday = FIELD_EX32(val, TOY_MATCH, DAY);
+    tm->tm_mon = FIELD_EX32(val, TOY_MATCH, MON) - 1;
+    tm->tm_year += (FIELD_EX32(val, TOY_MATCH, YEAR) - (tm->tm_year & 0x3f));
+}
+
+static void toymatch_write(LS7ARtcState *s, struct tm *tm, uint64_t val, int num)
+{
+    int64_t now, expire_time;
+
+    /* it do not support write when toy disabled */
+    if (toy_enabled(s)) {
+        s->toymatch[num] = val;
+        /* caculate expire time */
+        now = qemu_clock_get_ms(rtc_clock);
+        toymatch_val_to_time(val, tm);
+        expire_time = now + (qemu_timedate_diff(tm) - s->offset_toy) * 1000;
+        timer_mod(s->toy_timer[num], expire_time);
+    }
+}
+
+static void rtcmatch_write(LS7ARtcState *s, uint64_t val, int num)
+{
+    uint64_t expire_ns;
+
+    /* it do not support write when toy disabled */
+    if (rtc_enabled(s)) {
+        s->rtcmatch[num] = val;
+        /* caculate expire time */
+        expire_ns = ticks_to_ns(val) - ticks_to_ns(s->offset_rtc);
+        timer_mod_ns(s->rtc_timer[num], expire_ns);
+    }
+}
+
+static void ls7a_toy_stop(LS7ARtcState *s)
+{
+    int i;
+    struct tm tm;
+    /*
+     * save time when disabled toy,
+     * because toy time not add counters.
+     */
+    qemu_get_timedate(&tm, s->offset_toy);
+    s->save_toy_mon = toy_time_to_val_mon(tm);
+    s->save_toy_year = toy_time_to_val_year(tm);
+
+    /* delete timers, and when re-enabled, recaculate expire time */
+    for (i = 0; i < TIMER_NUMS; i++) {
+        timer_del(s->toy_timer[i]);
+    }
+}
+
+static void ls7a_rtc_stop(LS7ARtcState *s)
+{
+    int i;
+    uint64_t time;
+
+    /* save rtc time */
+    time = ls7a_rtc_ticks() + s->offset_rtc;
+    s->save_rtc = time;
+
+    /* delete timers, and when re-enabled, recaculate expire time */
+    for (i = 0; i < TIMER_NUMS; i++) {
+        timer_del(s->rtc_timer[i]);
+    }
+}
+
+static void ls7a_toy_start(LS7ARtcState *s)
+{
+    int i;
+    uint64_t expire_time, now;
+    struct tm tm;
+    /*
+     * need to recaculate toy offset
+     * and expire time when enable it.
+     */
+    toy_val_to_time_mon(s->save_toy_mon, &tm);
+    toy_val_to_time_year(s->save_toy_year, &tm);
+
+    s->offset_toy = qemu_timedate_diff(&tm);
+    now = qemu_clock_get_ms(rtc_clock);
+
+    /* recaculate expire time and enable timer */
+    for (i = 0; i < TIMER_NUMS; i++) {
+        toymatch_val_to_time(s->toymatch[i], &tm);
+        expire_time = now + (qemu_timedate_diff(&tm) - s->offset_toy) * 1000;
+        timer_mod(s->toy_timer[i], expire_time);
+    }
+}
+
+static void ls7a_rtc_start(LS7ARtcState *s)
+{
+    int i;
+    uint64_t expire_time, now;
+
+    /*
+     * need to recaculate rtc offset
+     * and expire time when enable it.
+     */
+    now = ls7a_rtc_ticks();
+    s->offset_rtc = s->save_rtc - now;
+
+    /* recaculate expire time and enable timer */
+    for (i = 0; i < TIMER_NUMS; i++) {
+        expire_time = ticks_to_ns(s->rtcmatch[i]) - ticks_to_ns(s->offset_rtc);
+        timer_mod_ns(s->rtc_timer[i], expire_time);
+    }
+}
+
+static uint64_t ls7a_rtc_read(void *opaque, hwaddr addr, unsigned size)
+{
+    LS7ARtcState *s = LS7A_RTC(opaque);
+    struct tm tm;
+    int val = 0;
+
+    switch (addr) {
+    case SYS_TOYREAD0:
+        /* if toy disabled, read save toy time */
+        if (toy_enabled(s)) {
+            qemu_get_timedate(&tm, s->offset_toy);
+            val = toy_time_to_val_mon(tm);
+        } else {
+            /* read save mon val */
+            val = s->save_toy_mon;
+        }
+        break;
+    case SYS_TOYREAD1:
+        /* if toy disabled, read save toy time */
+        if (toy_enabled(s)) {
+            qemu_get_timedate(&tm, s->offset_toy);
+            val = tm.tm_year;
+        } else {
+            /* read save year val */
+            val = s->save_toy_year;
+        }
+        break;
+    case SYS_TOYMATCH0:
+        val = s->toymatch[0];
+        break;
+    case SYS_TOYMATCH1:
+        val = s->toymatch[1];
+        break;
+    case SYS_TOYMATCH2:
+        val = s->toymatch[2];
+        break;
+    case SYS_RTCCTRL:
+        val = s->cntrctl;
+        break;
+    case SYS_RTCREAD0:
+        /* if rtc disabled, read save rtc time */
+        if (rtc_enabled(s)) {
+            val = ls7a_rtc_ticks() + s->offset_rtc;
+        } else {
+            val = s->save_rtc;
+        }
+        break;
+    case SYS_RTCMATCH0:
+        val = s->rtcmatch[0];
+        break;
+    case SYS_RTCMATCH1:
+        val = s->rtcmatch[1];
+        break;
+    case SYS_RTCMATCH2:
+        val = s->rtcmatch[2];
+        break;
+    default:
+        val = 0;
+        break;
+    }
+    return val;
+}
+
+static void ls7a_rtc_write(void *opaque, hwaddr addr,
+                           uint64_t val, unsigned size)
+{
+    int old_toyen, old_rtcen, new_toyen, new_rtcen;
+    LS7ARtcState *s = LS7A_RTC(opaque);
+    struct tm tm;
+
+    switch (addr) {
+    case SYS_TOYWRITE0:
+        /* it do not support write when toy disabled */
+        if (toy_enabled(s)) {
+            qemu_get_timedate(&tm, s->offset_toy);
+            tm.tm_sec = FIELD_EX32(val, TOY, SEC);
+            tm.tm_min = FIELD_EX32(val, TOY, MIN);
+            tm.tm_hour = FIELD_EX32(val, TOY, HOUR);
+            tm.tm_mday = FIELD_EX32(val, TOY, DAY);
+            tm.tm_mon = FIELD_EX32(val, TOY, MON) - 1;
+            s->offset_toy = qemu_timedate_diff(&tm);
+        }
+    break;
+    case SYS_TOYWRITE1:
+        if (toy_enabled(s)) {
+            qemu_get_timedate(&tm, s->offset_toy);
+            tm.tm_year = val;
+            s->offset_toy = qemu_timedate_diff(&tm);
+        }
+        break;
+    case SYS_TOYMATCH0:
+        toymatch_write(s, &tm, val, 0);
+        break;
+    case SYS_TOYMATCH1:
+        toymatch_write(s, &tm, val, 1);
+        break;
+    case SYS_TOYMATCH2:
+        toymatch_write(s, &tm, val, 2);
+        break;
+    case SYS_RTCCTRL:
+        /* get old ctrl */
+        old_toyen = toy_enabled(s);
+        old_rtcen = rtc_enabled(s);
+
+        s->cntrctl = val;
+        /* get new ctrl */
+        new_toyen = toy_enabled(s);
+        new_rtcen = rtc_enabled(s);
+
+        /*
+         * we do not consider if EO changed, as it always set at most time.
+         * toy or rtc enabled should start timer. otherwise, stop timer
+         */
+        if (old_toyen != new_toyen) {
+            if (new_toyen) {
+                ls7a_toy_start(s);
+            } else {
+                ls7a_toy_stop(s);
+            }
+        }
+        if (old_rtcen != new_rtcen) {
+            if (new_rtcen) {
+                ls7a_rtc_start(s);
+            } else {
+                ls7a_rtc_stop(s);
+            }
+        }
+        break;
+    case SYS_RTCWRTIE0:
+        if (rtc_enabled(s)) {
+            s->offset_rtc = val - ls7a_rtc_ticks();
+        }
+        break;
+    case SYS_RTCMATCH0:
+        rtcmatch_write(s, val, 0);
+        break;
+    case SYS_RTCMATCH1:
+        rtcmatch_write(s, val, 1);
+        break;
+    case SYS_RTCMATCH2:
+        rtcmatch_write(s, val, 2);
+        break;
+    default:
+        break;
+    }
+}
+
+static const MemoryRegionOps ls7a_rtc_ops = {
+    .read = ls7a_rtc_read,
+    .write = ls7a_rtc_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 4,
+        .max_access_size = 4,
+    },
+};
+
+static void toy_timer_cb(void *opaque)
+{
+    LS7ARtcState *s = opaque;
+
+    if (toy_enabled(s)) {
+        qemu_irq_pulse(s->irq);
+    }
+}
+
+static void rtc_timer_cb(void *opaque)
+{
+    LS7ARtcState *s = opaque;
+
+    if (rtc_enabled(s)) {
+        qemu_irq_pulse(s->irq);
+    }
+}
+
+static void ls7a_rtc_realize(DeviceState *dev, Error **errp)
+{
+    int i;
+    SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
+    LS7ARtcState *d = LS7A_RTC(sbd);
+    memory_region_init_io(&d->iomem, NULL, &ls7a_rtc_ops,
+                         (void *)d, "ls7a_rtc", 0x100);
+
+    sysbus_init_irq(sbd, &d->irq);
+
+    sysbus_init_mmio(sbd, &d->iomem);
+    for (i = 0; i < TIMER_NUMS; i++) {
+        d->toymatch[i] = 0;
+        d->rtcmatch[i] = 0;
+        d->toy_timer[i] = timer_new_ms(rtc_clock, toy_timer_cb, d);
+        d->rtc_timer[i] = timer_new_ms(rtc_clock, rtc_timer_cb, d);
+    }
+    d->offset_toy = 0;
+    d->offset_rtc = 0;
+    d->save_toy_mon = 0;
+    d->save_toy_year = 0;
+    d->save_rtc = 0;
+
+    create_unimplemented_device("mmio fallback 1", 0x10013ffc, 0x4);
+}
+
+static int ls7a_rtc_pre_save(void *opaque)
+{
+    LS7ARtcState *s = LS7A_RTC(opaque);
+
+    ls7a_toy_stop(s);
+    ls7a_rtc_stop(s);
+
+    return 0;
+}
+
+static int ls7a_rtc_post_load(void *opaque, int version_id)
+{
+    LS7ARtcState *s = LS7A_RTC(opaque);
+    if (toy_enabled(s)) {
+        ls7a_toy_start(s);
+    }
+
+    if (rtc_enabled(s)) {
+        ls7a_rtc_start(s);
+    }
+
+    return 0;
+}
+
+static const VMStateDescription vmstate_ls7a_rtc = {
+    .name = "ls7a_rtc",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .pre_save = ls7a_rtc_pre_save,
+    .post_load = ls7a_rtc_post_load,
+    .fields = (VMStateField[]) {
+        VMSTATE_INT64(offset_toy, LS7ARtcState),
+        VMSTATE_INT64(offset_rtc, LS7ARtcState),
+        VMSTATE_UINT64(save_toy_mon, LS7ARtcState),
+        VMSTATE_UINT64(save_toy_year, LS7ARtcState),
+        VMSTATE_UINT64(save_rtc, LS7ARtcState),
+        VMSTATE_UINT32_ARRAY(toymatch, LS7ARtcState, TIMER_NUMS),
+        VMSTATE_UINT32_ARRAY(rtcmatch, LS7ARtcState, TIMER_NUMS),
+        VMSTATE_UINT32(cntrctl, LS7ARtcState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static void ls7a_rtc_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+    dc->vmsd = &vmstate_ls7a_rtc;
+    dc->realize = ls7a_rtc_realize;
+    dc->desc = "ls7a rtc";
+}
+
+static const TypeInfo ls7a_rtc_info = {
+    .name          = TYPE_LS7A_RTC,
+    .parent        = TYPE_SYS_BUS_DEVICE,
+    .instance_size = sizeof(LS7ARtcState),
+    .class_init    = ls7a_rtc_class_init,
+};
+
+static void ls7a_rtc_register_types(void)
+{
+    type_register_static(&ls7a_rtc_info);
+}
+
+type_init(ls7a_rtc_register_types)
diff --git a/hw/rtc/meson.build b/hw/rtc/meson.build
index 7cecdee5dd..dc33973384 100644
--- a/hw/rtc/meson.build
+++ b/hw/rtc/meson.build
@@ -11,6 +11,7 @@ softmmu_ss.add(when: 'CONFIG_EXYNOS4', if_true: files('exynos4210_rtc.c'))
 softmmu_ss.add(when: 'CONFIG_SUN4V_RTC', if_true: files('sun4v-rtc.c'))
 softmmu_ss.add(when: 'CONFIG_ASPEED_SOC', if_true: files('aspeed_rtc.c'))
 softmmu_ss.add(when: 'CONFIG_GOLDFISH_RTC', if_true: files('goldfish_rtc.c'))
+softmmu_ss.add(when: 'CONFIG_LS7A_RTC', if_true: files('ls7a_rtc.c'))
 softmmu_ss.add(when: 'CONFIG_ALLWINNER_H3', if_true: files('allwinner-rtc.c'))
 
 specific_ss.add(when: 'CONFIG_MC146818RTC', if_true: files('mc146818rtc.c'))
diff --git a/include/hw/pci-host/ls7a.h b/include/hw/pci-host/ls7a.h
index 5c653527f0..08c5f78be2 100644
--- a/include/hw/pci-host/ls7a.h
+++ b/include/hw/pci-host/ls7a.h
@@ -37,4 +37,8 @@
 #define LS7A_PCI_IRQS           48
 #define LS7A_UART_IRQ           (PCH_PIC_IRQ_OFFSET + 2)
 #define LS7A_UART_BASE          0x1fe001e0
+#define LS7A_RTC_IRQ            (PCH_PIC_IRQ_OFFSET + 3)
+#define LS7A_MISC_REG_BASE      (LS7A_PCH_REG_BASE + 0x00080000)
+#define LS7A_RTC_REG_BASE       (LS7A_MISC_REG_BASE + 0x00050100)
+#define LS7A_RTC_LEN            0x100
 #endif

From 6a6f26f481204bb95d855aa0427389ef10073e62 Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:29 +0800
Subject: [PATCH 846/935] hw/loongarch: Add LoongArch load elf function.

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-40-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 hw/loongarch/loongson3.c | 61 ++++++++++++++++++++++++++++++++++++++--
 target/loongarch/cpu.h   |  2 ++
 2 files changed, 61 insertions(+), 2 deletions(-)

diff --git a/hw/loongarch/loongson3.c b/hw/loongarch/loongson3.c
index 95984c9086..3c8fcb828c 100644
--- a/hw/loongarch/loongson3.c
+++ b/hw/loongarch/loongson3.c
@@ -19,6 +19,8 @@
 #include "exec/address-spaces.h"
 #include "hw/irq.h"
 #include "net/net.h"
+#include "hw/loader.h"
+#include "elf.h"
 #include "hw/intc/loongarch_ipi.h"
 #include "hw/intc/loongarch_extioi.h"
 #include "hw/intc/loongarch_pch_pic.h"
@@ -29,6 +31,36 @@
 
 #include "target/loongarch/cpu.h"
 
+static struct _loaderparams {
+    uint64_t ram_size;
+    const char *kernel_filename;
+} loaderparams;
+
+static uint64_t cpu_loongarch_virt_to_phys(void *opaque, uint64_t addr)
+{
+    return addr & 0x1fffffffll;
+}
+
+static int64_t load_kernel_info(void)
+{
+    uint64_t kernel_entry, kernel_low, kernel_high;
+    ssize_t kernel_size;
+
+    kernel_size = load_elf(loaderparams.kernel_filename, NULL,
+                           cpu_loongarch_virt_to_phys, NULL,
+                           &kernel_entry, &kernel_low,
+                           &kernel_high, NULL, 0,
+                           EM_LOONGARCH, 1, 0);
+
+    if (kernel_size < 0) {
+        error_report("could not load kernel '%s': %s",
+                     loaderparams.kernel_filename,
+                     load_elf_strerror(kernel_size));
+        exit(1);
+    }
+    return kernel_entry;
+}
+
 static void loongarch_devices_init(DeviceState *pch_pic)
 {
     DeviceState *gpex_dev;
@@ -200,15 +232,29 @@ static void loongarch_irq_init(LoongArchMachineState *lams)
     loongarch_devices_init(pch_pic);
 }
 
+static void reset_load_elf(void *opaque)
+{
+    LoongArchCPU *cpu = opaque;
+    CPULoongArchState *env = &cpu->env;
+
+    cpu_reset(CPU(cpu));
+    if (env->load_elf) {
+        cpu_set_pc(CPU(cpu), env->elf_address);
+    }
+}
+
 static void loongarch_init(MachineState *machine)
 {
     const char *cpu_model = machine->cpu_type;
+    const char *kernel_filename = machine->kernel_filename;
     ram_addr_t offset = 0;
     ram_addr_t ram_size = machine->ram_size;
     uint64_t highram_size = 0;
     MemoryRegion *address_space_mem = get_system_memory();
     LoongArchMachineState *lams = LOONGARCH_MACHINE(machine);
+    LoongArchCPU *lacpu;
     int i;
+    int64_t kernel_addr = 0;
 
     if (!cpu_model) {
         cpu_model = LOONGARCH_CPU_TYPE_NAME("la464");
@@ -234,17 +280,28 @@ static void loongarch_init(MachineState *machine)
                              machine->ram, 0, 256 * MiB);
     memory_region_add_subregion(address_space_mem, offset, &lams->lowmem);
     offset += 256 * MiB;
-
     highram_size = ram_size - 256 * MiB;
     memory_region_init_alias(&lams->highmem, NULL, "loongarch.highmem",
                              machine->ram, offset, highram_size);
     memory_region_add_subregion(address_space_mem, 0x90000000, &lams->highmem);
-
     /* Add isa io region */
     memory_region_init_alias(&lams->isa_io, NULL, "isa-io",
                              get_system_io(), 0, LOONGARCH_ISA_IO_SIZE);
     memory_region_add_subregion(address_space_mem, LOONGARCH_ISA_IO_BASE,
                                 &lams->isa_io);
+    if (kernel_filename) {
+        loaderparams.ram_size = ram_size;
+        loaderparams.kernel_filename = kernel_filename;
+        kernel_addr = load_kernel_info();
+        if (!machine->firmware) {
+            for (i = 0; i < machine->smp.cpus; i++) {
+                lacpu = LOONGARCH_CPU(qemu_get_cpu(i));
+                lacpu->env.load_elf = true;
+                lacpu->env.elf_address = kernel_addr;
+                qemu_register_reset(reset_load_elf, lacpu);
+            }
+        }
+    }
     /* Initialize the IO interrupt subsystem */
     loongarch_irq_init(lams);
 }
diff --git a/target/loongarch/cpu.h b/target/loongarch/cpu.h
index 03cc96ee9b..71a5036c3c 100644
--- a/target/loongarch/cpu.h
+++ b/target/loongarch/cpu.h
@@ -308,6 +308,8 @@ typedef struct CPUArchState {
     AddressSpace address_space_iocsr;
     MemoryRegion system_iocsr;
     MemoryRegion iocsr_mem;
+    bool load_elf;
+    uint64_t elf_address;
 } CPULoongArchState;
 
 /**

From 9e6602d65704dc8d2a3fbcd76bfad1bb22fc3d72 Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:30 +0800
Subject: [PATCH 847/935] hw/loongarch: Add LoongArch virt power manager
 support.

This is a placeholder for missing ACPI, and will eventually be replaced.

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-41-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 hw/loongarch/loongson3.c | 49 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 48 insertions(+), 1 deletion(-)

diff --git a/hw/loongarch/loongson3.c b/hw/loongarch/loongson3.c
index 3c8fcb828c..bd20ebbb78 100644
--- a/hw/loongarch/loongson3.c
+++ b/hw/loongarch/loongson3.c
@@ -31,6 +31,48 @@
 
 #include "target/loongarch/cpu.h"
 
+#define PM_BASE 0x10080000
+#define PM_SIZE 0x100
+#define PM_CTRL 0x10
+
+/*
+ * This is a placeholder for missing ACPI,
+ * and will eventually be replaced.
+ */
+static uint64_t loongarch_virt_pm_read(void *opaque, hwaddr addr, unsigned size)
+{
+    return 0;
+}
+
+static void loongarch_virt_pm_write(void *opaque, hwaddr addr,
+                               uint64_t val, unsigned size)
+{
+    if (addr != PM_CTRL) {
+        return;
+    }
+
+    switch (val) {
+    case 0x00:
+        qemu_system_reset_request(SHUTDOWN_CAUSE_GUEST_RESET);
+        return;
+    case 0xff:
+        qemu_system_shutdown_request(SHUTDOWN_CAUSE_GUEST_SHUTDOWN);
+        return;
+    default:
+        return;
+    }
+}
+
+static const MemoryRegionOps loongarch_virt_pm_ops = {
+    .read  = loongarch_virt_pm_read,
+    .write = loongarch_virt_pm_write,
+    .endianness = DEVICE_NATIVE_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 1
+    }
+};
+
 static struct _loaderparams {
     uint64_t ram_size;
     const char *kernel_filename;
@@ -67,7 +109,7 @@ static void loongarch_devices_init(DeviceState *pch_pic)
     SysBusDevice *d;
     PCIBus *pci_bus;
     MemoryRegion *ecam_alias, *ecam_reg, *pio_alias, *pio_reg;
-    MemoryRegion *mmio_alias, *mmio_reg;
+    MemoryRegion *mmio_alias, *mmio_reg, *pm_mem;
     int i;
 
     gpex_dev = qdev_new(TYPE_GPEX_HOST);
@@ -132,6 +174,11 @@ static void loongarch_devices_init(DeviceState *pch_pic)
     sysbus_create_simple("ls7a_rtc", LS7A_RTC_REG_BASE,
                          qdev_get_gpio_in(pch_pic,
                          LS7A_RTC_IRQ - PCH_PIC_IRQ_OFFSET));
+
+    pm_mem = g_new(MemoryRegion, 1);
+    memory_region_init_io(pm_mem, NULL, &loongarch_virt_pm_ops,
+                          NULL, "loongarch_virt_pm", PM_SIZE);
+    memory_region_add_subregion(get_system_memory(), PM_BASE, pm_mem);
 }
 
 static void loongarch_irq_init(LoongArchMachineState *lams)

From ca61e75071c647cf93b3161a228c6a54178cd58c Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:31 +0800
Subject: [PATCH 848/935] target/loongarch: Add gdb support.

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-42-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 MAINTAINERS                             |  1 +
 configs/targets/loongarch64-softmmu.mak |  1 +
 gdb-xml/loongarch-base64.xml            | 44 ++++++++++++++
 gdb-xml/loongarch-fpu64.xml             | 57 +++++++++++++++++
 target/loongarch/cpu.c                  |  9 +++
 target/loongarch/gdbstub.c              | 81 +++++++++++++++++++++++++
 target/loongarch/internals.h            |  4 ++
 target/loongarch/meson.build            |  1 +
 8 files changed, 198 insertions(+)
 create mode 100644 gdb-xml/loongarch-base64.xml
 create mode 100644 gdb-xml/loongarch-fpu64.xml
 create mode 100644 target/loongarch/gdbstub.c

diff --git a/MAINTAINERS b/MAINTAINERS
index e8938db694..954fb95218 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1137,6 +1137,7 @@ F: include/hw/intc/loongarch_*.h
 F: hw/intc/loongarch_*.c
 F: include/hw/pci-host/ls7a.h
 F: hw/rtc/ls7a_rtc.c
+F: gdb-xml/loongarch*.xml
 
 M68K Machines
 -------------
diff --git a/configs/targets/loongarch64-softmmu.mak b/configs/targets/loongarch64-softmmu.mak
index 666154022f..7bc06c850c 100644
--- a/configs/targets/loongarch64-softmmu.mak
+++ b/configs/targets/loongarch64-softmmu.mak
@@ -1,3 +1,4 @@
 TARGET_ARCH=loongarch64
 TARGET_BASE_ARCH=loongarch
 TARGET_SUPPORTS_MTTCG=y
+TARGET_XML_FILES= gdb-xml/loongarch-base64.xml gdb-xml/loongarch-fpu64.xml
diff --git a/gdb-xml/loongarch-base64.xml b/gdb-xml/loongarch-base64.xml
new file mode 100644
index 0000000000..4962bdbd28
--- /dev/null
+++ b/gdb-xml/loongarch-base64.xml
@@ -0,0 +1,44 @@
+<?xml version="1.0"?>
+<!-- Copyright (C) 2021 Free Software Foundation, Inc.
+
+     Copying and distribution of this file, with or without modification,
+     are permitted in any medium without royalty provided the copyright
+     notice and this notice are preserved.  -->
+
+<!DOCTYPE feature SYSTEM "gdb-target.dtd">
+<feature name="org.gnu.gdb.loongarch.base">
+  <reg name="r0" bitsize="64" type="uint64" group="general"/>
+  <reg name="r1" bitsize="64" type="uint64" group="general"/>
+  <reg name="r2" bitsize="64" type="uint64" group="general"/>
+  <reg name="r3" bitsize="64" type="uint64" group="general"/>
+  <reg name="r4" bitsize="64" type="uint64" group="general"/>
+  <reg name="r5" bitsize="64" type="uint64" group="general"/>
+  <reg name="r6" bitsize="64" type="uint64" group="general"/>
+  <reg name="r7" bitsize="64" type="uint64" group="general"/>
+  <reg name="r8" bitsize="64" type="uint64" group="general"/>
+  <reg name="r9" bitsize="64" type="uint64" group="general"/>
+  <reg name="r10" bitsize="64" type="uint64" group="general"/>
+  <reg name="r11" bitsize="64" type="uint64" group="general"/>
+  <reg name="r12" bitsize="64" type="uint64" group="general"/>
+  <reg name="r13" bitsize="64" type="uint64" group="general"/>
+  <reg name="r14" bitsize="64" type="uint64" group="general"/>
+  <reg name="r15" bitsize="64" type="uint64" group="general"/>
+  <reg name="r16" bitsize="64" type="uint64" group="general"/>
+  <reg name="r17" bitsize="64" type="uint64" group="general"/>
+  <reg name="r18" bitsize="64" type="uint64" group="general"/>
+  <reg name="r19" bitsize="64" type="uint64" group="general"/>
+  <reg name="r20" bitsize="64" type="uint64" group="general"/>
+  <reg name="r21" bitsize="64" type="uint64" group="general"/>
+  <reg name="r22" bitsize="64" type="uint64" group="general"/>
+  <reg name="r23" bitsize="64" type="uint64" group="general"/>
+  <reg name="r24" bitsize="64" type="uint64" group="general"/>
+  <reg name="r25" bitsize="64" type="uint64" group="general"/>
+  <reg name="r26" bitsize="64" type="uint64" group="general"/>
+  <reg name="r27" bitsize="64" type="uint64" group="general"/>
+  <reg name="r28" bitsize="64" type="uint64" group="general"/>
+  <reg name="r29" bitsize="64" type="uint64" group="general"/>
+  <reg name="r30" bitsize="64" type="uint64" group="general"/>
+  <reg name="r31" bitsize="64" type="uint64" group="general"/>
+  <reg name="pc" bitsize="64" type="code_ptr" group="general"/>
+  <reg name="badvaddr" bitsize="64" type="code_ptr" group="general"/>
+</feature>
diff --git a/gdb-xml/loongarch-fpu64.xml b/gdb-xml/loongarch-fpu64.xml
new file mode 100644
index 0000000000..e52cf89fbc
--- /dev/null
+++ b/gdb-xml/loongarch-fpu64.xml
@@ -0,0 +1,57 @@
+<?xml version="1.0"?>
+<!-- Copyright (C) 2021 Free Software Foundation, Inc.
+
+     Copying and distribution of this file, with or without modification,
+     are permitted in any medium without royalty provided the copyright
+     notice and this notice are preserved.  -->
+
+<!DOCTYPE feature SYSTEM "gdb-target.dtd">
+<feature name="org.gnu.gdb.loongarch.fpu">
+
+  <union id="fpu64type">
+    <field name="f" type="ieee_single"/>
+    <field name="d" type="ieee_double"/>
+  </union>
+
+  <reg name="f0" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f1" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f2" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f3" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f4" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f5" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f6" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f7" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f8" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f9" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f10" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f11" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f12" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f13" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f14" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f15" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f16" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f17" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f18" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f19" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f20" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f21" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f22" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f23" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f24" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f25" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f26" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f27" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f28" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f29" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f30" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="f31" bitsize="64" type="fpu64type" group="float"/>
+  <reg name="fcc0" bitsize="8" type="uint8" group="float"/>
+  <reg name="fcc1" bitsize="8" type="uint8" group="float"/>
+  <reg name="fcc2" bitsize="8" type="uint8" group="float"/>
+  <reg name="fcc3" bitsize="8" type="uint8" group="float"/>
+  <reg name="fcc4" bitsize="8" type="uint8" group="float"/>
+  <reg name="fcc5" bitsize="8" type="uint8" group="float"/>
+  <reg name="fcc6" bitsize="8" type="uint8" group="float"/>
+  <reg name="fcc7" bitsize="8" type="uint8" group="float"/>
+  <reg name="fcsr" bitsize="32" type="uint32" group="float"/>
+</feature>
diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c
index 3b7d6289d2..4c8f96bc3a 100644
--- a/target/loongarch/cpu.c
+++ b/target/loongarch/cpu.c
@@ -487,6 +487,8 @@ static void loongarch_cpu_realizefn(DeviceState *dev, Error **errp)
         return;
     }
 
+    loongarch_cpu_register_gdb_regs_for_features(cs);
+
     cpu_reset(cs);
     qemu_init_vcpu(cs);
 
@@ -640,6 +642,13 @@ static void loongarch_cpu_class_init(ObjectClass *c, void *data)
     dc->vmsd = &vmstate_loongarch_cpu;
     cc->sysemu_ops = &loongarch_sysemu_ops;
     cc->disas_set_info = loongarch_cpu_disas_set_info;
+    cc->gdb_read_register = loongarch_cpu_gdb_read_register;
+    cc->gdb_write_register = loongarch_cpu_gdb_write_register;
+    cc->disas_set_info = loongarch_cpu_disas_set_info;
+    cc->gdb_num_core_regs = 34;
+    cc->gdb_core_xml_file = "loongarch-base64.xml";
+    cc->gdb_stop_before_watchpoint = true;
+
 #ifdef CONFIG_TCG
     cc->tcg_ops = &loongarch_tcg_ops;
 #endif
diff --git a/target/loongarch/gdbstub.c b/target/loongarch/gdbstub.c
new file mode 100644
index 0000000000..0c48834201
--- /dev/null
+++ b/target/loongarch/gdbstub.c
@@ -0,0 +1,81 @@
+/*
+ * LOONGARCH gdb server stub
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ *
+ * SPDX-License-Identifier: LGPL-2.1+
+ */
+
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "internals.h"
+#include "exec/gdbstub.h"
+
+int loongarch_cpu_gdb_read_register(CPUState *cs, GByteArray *mem_buf, int n)
+{
+    LoongArchCPU *cpu = LOONGARCH_CPU(cs);
+    CPULoongArchState *env = &cpu->env;
+
+    if (0 <= n && n < 32) {
+        return gdb_get_regl(mem_buf, env->gpr[n]);
+    } else if (n == 32) {
+        return gdb_get_regl(mem_buf, env->pc);
+    } else if (n == 33) {
+        return gdb_get_regl(mem_buf, env->badaddr);
+    }
+    return 0;
+}
+
+int loongarch_cpu_gdb_write_register(CPUState *cs, uint8_t *mem_buf, int n)
+{
+    LoongArchCPU *cpu = LOONGARCH_CPU(cs);
+    CPULoongArchState *env = &cpu->env;
+    target_ulong tmp = ldtul_p(mem_buf);
+    int length = 0;
+
+    if (0 <= n && n < 32) {
+        env->gpr[n] = tmp;
+        length = sizeof(target_ulong);
+    } else if (n == 32) {
+        env->pc = tmp;
+        length = sizeof(target_ulong);
+    }
+    return length;
+}
+
+static int loongarch_gdb_get_fpu(CPULoongArchState *env,
+                                 GByteArray *mem_buf, int n)
+{
+    if (0 <= n && n < 32) {
+        return gdb_get_reg64(mem_buf, env->fpr[n]);
+    } else if (32 <= n && n < 40) {
+        return gdb_get_reg8(mem_buf, env->cf[n - 32]);
+    } else if (n == 40) {
+        return gdb_get_reg32(mem_buf, env->fcsr0);
+    }
+    return 0;
+}
+
+static int loongarch_gdb_set_fpu(CPULoongArchState *env,
+                                 uint8_t *mem_buf, int n)
+{
+    int length = 0;
+
+    if (0 <= n && n < 32) {
+        env->fpr[n] = ldq_p(mem_buf);
+        length = 8;
+    } else if (32 <= n && n < 40) {
+        env->cf[n - 32] = ldub_p(mem_buf);
+        length = 1;
+    } else if (n == 40) {
+        env->fcsr0 = ldl_p(mem_buf);
+        length = 4;
+    }
+    return length;
+}
+
+void loongarch_cpu_register_gdb_regs_for_features(CPUState *cs)
+{
+    gdb_register_coprocessor(cs, loongarch_gdb_get_fpu, loongarch_gdb_set_fpu,
+                             41, "loongarch-fpu64.xml", 0);
+}
diff --git a/target/loongarch/internals.h b/target/loongarch/internals.h
index a410c41c37..9d50fbdd81 100644
--- a/target/loongarch/internals.h
+++ b/target/loongarch/internals.h
@@ -49,4 +49,8 @@ bool loongarch_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
 
 hwaddr loongarch_cpu_get_phys_page_debug(CPUState *cpu, vaddr addr);
 
+int loongarch_cpu_gdb_read_register(CPUState *cs, GByteArray *mem_buf, int n);
+int loongarch_cpu_gdb_write_register(CPUState *cs, uint8_t *mem_buf, int n);
+void loongarch_cpu_register_gdb_regs_for_features(CPUState *cs);
+
 #endif
diff --git a/target/loongarch/meson.build b/target/loongarch/meson.build
index 74e5f3b2a7..6376f9e84b 100644
--- a/target/loongarch/meson.build
+++ b/target/loongarch/meson.build
@@ -11,6 +11,7 @@ loongarch_tcg_ss.add(files(
   'fpu_helper.c',
   'op_helper.c',
   'translate.c',
+  'gdbstub.c',
 ))
 loongarch_tcg_ss.add(zlib)
 

From c4293333982e9303699dab25140b2d63dfc9a179 Mon Sep 17 00:00:00 2001
From: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:32 +0800
Subject: [PATCH 849/935] tests/tcg/loongarch64: Add hello/memory test in
 loongarch64 system

- We write a very minimal softmmu harness.
- This is a very simple smoke test with no need to run a full Linux/kernel.
- The Makefile.softmmu-target record the rule to run.

Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606124333.2060567-43-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 MAINTAINERS                                   |  1 +
 tests/tcg/loongarch64/Makefile.softmmu-target | 33 +++++++
 tests/tcg/loongarch64/system/boot.S           | 56 ++++++++++++
 tests/tcg/loongarch64/system/kernel.ld        | 30 +++++++
 tests/tcg/loongarch64/system/regdef.h         | 86 +++++++++++++++++++
 5 files changed, 206 insertions(+)
 create mode 100644 tests/tcg/loongarch64/Makefile.softmmu-target
 create mode 100644 tests/tcg/loongarch64/system/boot.S
 create mode 100644 tests/tcg/loongarch64/system/kernel.ld
 create mode 100644 tests/tcg/loongarch64/system/regdef.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 954fb95218..5580a36b68 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -217,6 +217,7 @@ M: Song Gao <gaosong@loongson.cn>
 M: Xiaojuan Yang <yangxiaojuan@loongson.cn>
 S: Maintained
 F: target/loongarch/
+F: tests/tcg/loongarch64/
 
 M68K TCG CPUs
 M: Laurent Vivier <laurent@vivier.eu>
diff --git a/tests/tcg/loongarch64/Makefile.softmmu-target b/tests/tcg/loongarch64/Makefile.softmmu-target
new file mode 100644
index 0000000000..908f3a8c0f
--- /dev/null
+++ b/tests/tcg/loongarch64/Makefile.softmmu-target
@@ -0,0 +1,33 @@
+#
+# Loongarch64 system tests
+#
+
+LOONGARCH64_SYSTEM_SRC=$(SRC_PATH)/tests/tcg/loongarch64/system
+VPATH+=$(LOONGARCH64_SYSTEM_SRC)
+
+# These objects provide the basic boot code and helper functions for all tests
+CRT_OBJS=boot.o
+
+LOONGARCH64_TEST_SRCS=$(wildcard $(LOONGARCH64_SYSTEM_SRC)/*.c)
+LOONGARCH64_TESTS = $(patsubst $(LOONGARCH64_SYSTEM_SRC)/%.c, %, $(LOONGARCH64_TEST_SRCS))
+
+CRT_PATH=$(LOONGARCH64_SYSTEM_SRC)
+LINK_SCRIPT=$(LOONGARCH64_SYSTEM_SRC)/kernel.ld
+LDFLAGS=-Wl,-T$(LINK_SCRIPT)
+TESTS+=$(LOONGARCH64_TESTS) $(MULTIARCH_TESTS)
+CFLAGS+=-nostdlib -g -O1 -march=loongarch64 -mabi=lp64d $(MINILIB_INC)
+LDFLAGS+=-static -nostdlib $(CRT_OBJS) $(MINILIB_OBJS) -lgcc
+
+# building head blobs
+.PRECIOUS: $(CRT_OBJS)
+
+%.o: $(CRT_PATH)/%.S
+	$(CC) $(CFLAGS) $(EXTRA_CFLAGS) -x assembler-with-cpp -c $< -o $@
+
+# Build and link the tests
+%: %.c $(LINK_SCRIPT) $(CRT_OBJS) $(MINILIB_OBJS)
+	$(CC) $(CFLAGS) $(EXTRA_CFLAGS) $< -o $@ $(LDFLAGS)
+
+memory: CFLAGS+=-DCHECK_UNALIGNED=0
+# Running
+QEMU_OPTS+=-serial chardev:output -kernel
diff --git a/tests/tcg/loongarch64/system/boot.S b/tests/tcg/loongarch64/system/boot.S
new file mode 100644
index 0000000000..67eb1c04ce
--- /dev/null
+++ b/tests/tcg/loongarch64/system/boot.S
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Minimal LoongArch system boot code.
+ *
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+
+#include "regdef.h"
+
+	.global _start
+	.align 16
+_start:
+	la.local t0, stack_end
+	move sp, t0
+	bl main
+
+	.type _start 2
+	.size _start, .-_start
+
+	.global _exit
+	.align 16
+_exit:
+2:      /* QEMU ACPI poweroff */
+	li.w  t0, 0xff
+	li.w  t1, 0x10080010
+	st.w  t0, t1, 0
+	idle  0
+	bl    2b
+
+	.type _exit 2
+	.size _exit, .-_exit
+
+	.global __sys_outc
+__sys_outc:
+	li.d t1, 1000000
+loop:
+	lu12i.w	t2, 0x1fe00
+	ori	t0, t2, 0x1e5
+	ld.bu	t0, t0, 0
+	andi	t0, t0, 0x20
+	ext.w.b	t0, t0
+	bnez	t0, in
+	addi.w	t1, t1, -1
+	bnez	t1, loop
+in:
+	ext.w.b	a0, a0
+	lu12i.w	t0, 0x1fe00
+	ori	t0, t0, 0x1e0
+	st.b	a0, t0, 0
+	jirl	$r0, ra, 0
+
+	.data
+	.align 4
+stack:
+	.space	65536
+stack_end:
diff --git a/tests/tcg/loongarch64/system/kernel.ld b/tests/tcg/loongarch64/system/kernel.ld
new file mode 100644
index 0000000000..f1a7c0168c
--- /dev/null
+++ b/tests/tcg/loongarch64/system/kernel.ld
@@ -0,0 +1,30 @@
+ENTRY(_start)
+
+SECTIONS
+{
+    /* Linux kernel legacy start address.  */
+    . = 0x9000000000200000;
+    _text = .;
+    .text : {
+        *(.text)
+    }
+    .rodata : {
+        *(.rodata)
+    }
+    _etext = .;
+
+    . = ALIGN(8192);
+    _data = .;
+    .got : {
+        *(.got)
+    }
+    .data : {
+	*(.sdata)
+        *(.data)
+    }
+    _edata = .;
+    .bss : {
+        *(.bss)
+    }
+    _end = .;
+}
diff --git a/tests/tcg/loongarch64/system/regdef.h b/tests/tcg/loongarch64/system/regdef.h
new file mode 100644
index 0000000000..faa09b2377
--- /dev/null
+++ b/tests/tcg/loongarch64/system/regdef.h
@@ -0,0 +1,86 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_REGDEF_H
+#define _ASM_REGDEF_H
+
+#define zero    $r0     /* wired zero */
+#define ra      $r1     /* return address */
+#define tp      $r2
+#define sp      $r3     /* stack pointer */
+#define v0      $r4     /* return value - caller saved */
+#define v1      $r5
+#define a0      $r4     /* argument registers */
+#define a1      $r5
+#define a2      $r6
+#define a3      $r7
+#define a4      $r8
+#define a5      $r9
+#define a6      $r10
+#define a7      $r11
+#define t0      $r12    /* caller saved */
+#define t1      $r13
+#define t2      $r14
+#define t3      $r15
+#define t4      $r16
+#define t5      $r17
+#define t6      $r18
+#define t7      $r19
+#define t8      $r20
+                        /* $r21: Temporarily reserved */
+#define fp      $r22    /* frame pointer */
+#define s0      $r23    /* callee saved */
+#define s1      $r24
+#define s2      $r25
+#define s3      $r26
+#define s4      $r27
+#define s5      $r28
+#define s6      $r29
+#define s7      $r30
+#define s8      $r31
+
+#define gr0     $r0
+#define gr1     $r1
+#define gr2     $r2
+#define gr3     $r3
+#define gr4     $r4
+#define gr5     $r5
+#define gr6     $r6
+#define gr7     $r7
+#define gr8     $r8
+#define gr9     $r9
+#define gr10    $r10
+#define gr11    $r11
+#define gr12    $r12
+#define gr13    $r13
+#define gr14    $r14
+#define gr15    $r15
+#define gr16    $r16
+#define gr17    $r17
+#define gr18    $r18
+#define gr19    $r19
+#define gr20    $r20
+#define gr21    $r21
+#define gr22    $r22
+#define gr23    $r23
+#define gr24    $r24
+#define gr25    $r25
+#define gr26    $r26
+#define gr27    $r27
+#define gr28    $r28
+#define gr29    $r29
+#define gr30    $r30
+#define gr31    $r31
+
+#define STT_NOTYPE  0
+#define STT_OBJECT  1
+#define STT_FUNC    2
+#define STT_SECTION 3
+#define STT_FILE    4
+#define STT_COMMON  5
+#define STT_TLS     6
+
+#define ASM_NL           ;
+
+#endif /* _ASM_REGDEF_H */

From 34bb43b074906a7cd642ccf03e2b7bea63b53d95 Mon Sep 17 00:00:00 2001
From: Song Gao <gaosong@loongson.cn>
Date: Mon, 6 Jun 2022 20:43:33 +0800
Subject: [PATCH 850/935] target/loongarch: 'make check-tcg' support
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220606124333.2060567-44-yangxiaojuan@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 configure | 1 +
 1 file changed, 1 insertion(+)

diff --git a/configure b/configure
index ac18ed4f3a..e69537c756 100755
--- a/configure
+++ b/configure
@@ -1831,6 +1831,7 @@ fi
 : ${cross_prefix_arm="arm-linux-gnueabihf-"}
 : ${cross_prefix_armeb="$cross_prefix_arm"}
 : ${cross_prefix_hexagon="hexagon-unknown-linux-musl-"}
+: ${cross_prefix_loongarch64="loongarch64-unknown-linux-gnu-"}
 : ${cross_prefix_hppa="hppa-linux-gnu-"}
 : ${cross_prefix_i386="i686-linux-gnu-"}
 : ${cross_prefix_m68k="m68k-linux-gnu-"}

From e37a0ef4605e5d2041785ff3fc89ca6021faf7a0 Mon Sep 17 00:00:00 2001
From: Anthony PERARD <anthony.perard@citrix.com>
Date: Mon, 11 Apr 2022 15:47:49 +0100
Subject: [PATCH 851/935] tpm_crb: mark command buffer as dirty on request
 completion

At the moment, there doesn't seems to be any way to know that QEMU
made modification to the command buffer. This is potentially an issue
on Xen while migrating a guest, as modification to the buffer after
the migration as started could be ignored and not transfered to the
destination.

Mark the memory region of the command buffer as dirty once a request
is completed.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Message-id: 20220411144749.47185-1-anthony.perard@citrix.com
---
 hw/tpm/tpm_crb.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/tpm/tpm_crb.c b/hw/tpm/tpm_crb.c
index aa9c00aad3..67db594c48 100644
--- a/hw/tpm/tpm_crb.c
+++ b/hw/tpm/tpm_crb.c
@@ -197,6 +197,7 @@ static void tpm_crb_request_completed(TPMIf *ti, int ret)
         ARRAY_FIELD_DP32(s->regs, CRB_CTRL_STS,
                          tpmSts, 1); /* fatal error */
     }
+    memory_region_set_dirty(&s->cmdmem, 0, CRB_CTRL_CMD_SIZE);
 }
 
 static enum TPMVersion tpm_crb_get_version(TPMIf *ti)

From ec6600be0dc16982181c7ad80d94c143c0807dd2 Mon Sep 17 00:00:00 2001
From: Eric Auger <eric.auger@redhat.com>
Date: Tue, 24 May 2022 05:14:05 -0400
Subject: [PATCH 852/935] vfio/common: remove spurious warning on
 vfio_listener_region_del

851d6d1a0f ("vfio/common: remove spurious tpm-crb-cmd misalignment
warning") removed the warning on vfio_listener_region_add() path.

However the same warning also hits on region_del path. Let's remove
it and reword the dynamic trace as this can be called on both
map and unmap path.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Link: https://lore.kernel.org/r/20220524091405.416256-1-eric.auger@redhat.com
Fixes: 851d6d1a0ff2 ("vfio/common: remove spurious tpm-crb-cmd misalignment warning")
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/common.c     | 10 +++++++++-
 hw/vfio/trace-events |  2 +-
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index 29982c7af8..ace9562a9b 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -1145,7 +1145,15 @@ static void vfio_listener_region_del(MemoryListener *listener,
     if (unlikely((section->offset_within_address_space &
                   ~qemu_real_host_page_mask()) !=
                  (section->offset_within_region & ~qemu_real_host_page_mask()))) {
-        error_report("%s received unaligned region", __func__);
+        if (!vfio_known_safe_misalignment(section)) {
+            error_report("%s received unaligned region %s iova=0x%"PRIx64
+                         " offset_within_region=0x%"PRIx64
+                         " qemu_real_host_page_size=0x%"PRIxPTR,
+                         __func__, memory_region_name(section->mr),
+                         section->offset_within_address_space,
+                         section->offset_within_region,
+                         qemu_real_host_page_size());
+        }
         return;
     }
 
diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events
index 582882db91..73dffe9e00 100644
--- a/hw/vfio/trace-events
+++ b/hw/vfio/trace-events
@@ -100,7 +100,7 @@ vfio_listener_region_add_skip(uint64_t start, uint64_t end) "SKIPPING region_add
 vfio_spapr_group_attach(int groupfd, int tablefd) "Attached groupfd %d to liobn fd %d"
 vfio_listener_region_add_iommu(uint64_t start, uint64_t end) "region_add [iommu] 0x%"PRIx64" - 0x%"PRIx64
 vfio_listener_region_add_ram(uint64_t iova_start, uint64_t iova_end, void *vaddr) "region_add [ram] 0x%"PRIx64" - 0x%"PRIx64" [%p]"
-vfio_known_safe_misalignment(const char *name, uint64_t iova, uint64_t offset_within_region, uintptr_t page_size) "Region \"%s\" iova=0x%"PRIx64" offset_within_region=0x%"PRIx64" qemu_real_host_page_size=0x%"PRIxPTR ": cannot be mapped for DMA"
+vfio_known_safe_misalignment(const char *name, uint64_t iova, uint64_t offset_within_region, uintptr_t page_size) "Region \"%s\" iova=0x%"PRIx64" offset_within_region=0x%"PRIx64" qemu_real_host_page_size=0x%"PRIxPTR
 vfio_listener_region_add_no_dma_map(const char *name, uint64_t iova, uint64_t size, uint64_t page_size) "Region \"%s\" 0x%"PRIx64" size=0x%"PRIx64" is not aligned to 0x%"PRIx64" and cannot be mapped for DMA"
 vfio_listener_region_del_skip(uint64_t start, uint64_t end) "SKIPPING region_del 0x%"PRIx64" - 0x%"PRIx64
 vfio_listener_region_del(uint64_t start, uint64_t end) "region_del 0x%"PRIx64" - 0x%"PRIx64

From d507bc3b0541581fc9960f157968e80c02247255 Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Wed, 8 Jun 2022 19:38:46 +0100
Subject: [PATCH 853/935] target/arm: Declare support for FEAT_RASv1p1

The architectural feature RASv1p1 introduces the following new
features:
 * new registers ERXPFGCDN_EL1, ERXPFGCTL_EL1 and ERXPFGF_EL1
 * new bits in the fine-grained trap registers that control traps
   for these new registers
 * new trap bits HCR_EL2.FIEN and SCR_EL3.FIEN that control traps
   for ERXPFGCDN_EL1, ERXPFGCTL_EL1, ERXPFGP_EL1
 * a larger number of the ERXMISC<n>_EL1 registers
 * the format of ERR<n>STATUS registers changes

The architecture permits that if ERRIDR_EL1.NUM is 0 (as it is for
QEMU) then all these new registers may UNDEF, and the HCR_EL2.FIEN
and SCR_EL3.FIEN bits may be RES0.  We don't have any ERR<n>STATUS
registers (again, because ERRIDR_EL1.NUM is 0).  QEMU does not yet
implement the fine-grained-trap extension.  So there is nothing we
need to implement to be compliant with the feature spec.  Make the
'max' CPU report the feature in its ID registers, and document it.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220531114258.855804-1-peter.maydell@linaro.org
---
 docs/system/arm/emulation.rst | 1 +
 target/arm/cpu64.c            | 1 +
 2 files changed, 2 insertions(+)

diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst
index 49cc3e8340..81467f02ce 100644
--- a/docs/system/arm/emulation.rst
+++ b/docs/system/arm/emulation.rst
@@ -52,6 +52,7 @@ the following architecture extensions:
 - FEAT_PMUv3p1 (PMU Extensions v3.1)
 - FEAT_PMUv3p4 (PMU Extensions v3.4)
 - FEAT_RAS (Reliability, availability, and serviceability)
+- FEAT_RASv1p1 (RAS Extension v1.1)
 - FEAT_RDM (Advanced SIMD rounding double multiply accumulate instructions)
 - FEAT_RNG (Random number generator)
 - FEAT_S2FWB (Stage 2 forced Write-Back)
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 3ff9219ca3..bd1c62a342 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -916,6 +916,7 @@ static void aarch64_max_initfn(Object *obj)
      * we do for EL2 with the virtualization=on property.
      */
     t = FIELD_DP64(t, ID_AA64PFR1, MTE, 3);       /* FEAT_MTE3 */
+    t = FIELD_DP64(t, ID_AA64PFR1, RAS_FRAC, 1);  /* FEAT_RASv1p1 */
     t = FIELD_DP64(t, ID_AA64PFR1, CSV2_FRAC, 0); /* FEAT_CSV2_2 */
     cpu->isar.id_aa64pfr1 = t;
 

From 7ac610206a0049460c392c9559b51246af0afd6d Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Wed, 8 Jun 2022 19:38:46 +0100
Subject: [PATCH 854/935] target/arm: Implement FEAT_DoubleFault

The FEAT_DoubleFault extension adds the following:

 * All external aborts on instruction fetches and translation table
   walks for instruction fetches must be synchronous.  For QEMU this
   is already true.

 * SCR_EL3 has a new bit NMEA which disables the masking of SError
   interrupts by PSTATE.A when the SError interrupt is taken to EL3.
   For QEMU we only need to make the bit writable, because we have no
   sources of SError interrupts.

 * SCR_EL3 has a new bit EASE which causes synchronous external
   aborts taken to EL3 to be taken at the same entry point as SError.
   (Note that this does not mean that they are SErrors for purposes
   of PSTATE.A masking or that the syndrome register reports them as
   SErrors: it just means that the vector offset is different.)

 * The existing SCTLR_EL3.IESB has an effective value of 1 when
   SCR_EL3.NMEA is 1.  For QEMU this is a no-op because we don't need
   different behaviour based on IESB (we don't need to do anything to
   ensure that error exceptions are synchronized).

So for QEMU the things we need to change are:
 * Make SCR_EL3.{NMEA,EASE} writable
 * When taking a synchronous external abort at EL3, adjust the
   vector entry point if SCR_EL3.EASE is set
 * Advertise the feature in the ID registers

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220531151431.949322-1-peter.maydell@linaro.org
---
 docs/system/arm/emulation.rst |  1 +
 target/arm/cpu.h              |  5 +++++
 target/arm/cpu64.c            |  4 ++--
 target/arm/helper.c           | 36 +++++++++++++++++++++++++++++++++++
 4 files changed, 44 insertions(+), 2 deletions(-)

diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst
index 81467f02ce..83b4410065 100644
--- a/docs/system/arm/emulation.rst
+++ b/docs/system/arm/emulation.rst
@@ -23,6 +23,7 @@ the following architecture extensions:
 - FEAT_Debugv8p2 (Debug changes for v8.2)
 - FEAT_Debugv8p4 (Debug changes for v8.4)
 - FEAT_DotProd (Advanced SIMD dot product instructions)
+- FEAT_DoubleFault (Double Fault Extension)
 - FEAT_FCMA (Floating-point complex number instructions)
 - FEAT_FHM (Floating-point half-precision multiplication instructions)
 - FEAT_FP16 (Half-precision floating-point data processing)
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index c1865ad5da..0ee1705a4f 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -3952,6 +3952,11 @@ static inline bool isar_feature_aa64_ras(const ARMISARegisters *id)
     return FIELD_EX64(id->id_aa64pfr0, ID_AA64PFR0, RAS) != 0;
 }
 
+static inline bool isar_feature_aa64_doublefault(const ARMISARegisters *id)
+{
+    return FIELD_EX64(id->id_aa64pfr0, ID_AA64PFR0, RAS) >= 2;
+}
+
 static inline bool isar_feature_aa64_sve(const ARMISARegisters *id)
 {
     return FIELD_EX64(id->id_aa64pfr0, ID_AA64PFR0, SVE) != 0;
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index bd1c62a342..cce68dd82a 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -899,7 +899,7 @@ static void aarch64_max_initfn(Object *obj)
     t = cpu->isar.id_aa64pfr0;
     t = FIELD_DP64(t, ID_AA64PFR0, FP, 1);        /* FEAT_FP16 */
     t = FIELD_DP64(t, ID_AA64PFR0, ADVSIMD, 1);   /* FEAT_FP16 */
-    t = FIELD_DP64(t, ID_AA64PFR0, RAS, 1);       /* FEAT_RAS */
+    t = FIELD_DP64(t, ID_AA64PFR0, RAS, 2);       /* FEAT_RASv1p1 + FEAT_DoubleFault */
     t = FIELD_DP64(t, ID_AA64PFR0, SVE, 1);
     t = FIELD_DP64(t, ID_AA64PFR0, SEL2, 1);      /* FEAT_SEL2 */
     t = FIELD_DP64(t, ID_AA64PFR0, DIT, 1);       /* FEAT_DIT */
@@ -916,7 +916,7 @@ static void aarch64_max_initfn(Object *obj)
      * we do for EL2 with the virtualization=on property.
      */
     t = FIELD_DP64(t, ID_AA64PFR1, MTE, 3);       /* FEAT_MTE3 */
-    t = FIELD_DP64(t, ID_AA64PFR1, RAS_FRAC, 1);  /* FEAT_RASv1p1 */
+    t = FIELD_DP64(t, ID_AA64PFR1, RAS_FRAC, 0);  /* FEAT_RASv1p1 + FEAT_DoubleFault */
     t = FIELD_DP64(t, ID_AA64PFR1, CSV2_FRAC, 0); /* FEAT_CSV2_2 */
     cpu->isar.id_aa64pfr1 = t;
 
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 40da63913c..7f2c14bea9 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -1776,6 +1776,9 @@ static void scr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value)
         if (cpu_isar_feature(aa64_scxtnum, cpu)) {
             valid_mask |= SCR_ENSCXT;
         }
+        if (cpu_isar_feature(aa64_doublefault, cpu)) {
+            valid_mask |= SCR_EASE | SCR_NMEA;
+        }
     } else {
         valid_mask &= ~(SCR_RW | SCR_ST);
         if (cpu_isar_feature(aa32_ras, cpu)) {
@@ -10113,6 +10116,31 @@ static uint32_t cpsr_read_for_spsr_elx(CPUARMState *env)
     return ret;
 }
 
+static bool syndrome_is_sync_extabt(uint32_t syndrome)
+{
+    /* Return true if this syndrome value is a synchronous external abort */
+    switch (syn_get_ec(syndrome)) {
+    case EC_INSNABORT:
+    case EC_INSNABORT_SAME_EL:
+    case EC_DATAABORT:
+    case EC_DATAABORT_SAME_EL:
+        /* Look at fault status code for all the synchronous ext abort cases */
+        switch (syndrome & 0x3f) {
+        case 0x10:
+        case 0x13:
+        case 0x14:
+        case 0x15:
+        case 0x16:
+        case 0x17:
+            return true;
+        default:
+            return false;
+        }
+    default:
+        return false;
+    }
+}
+
 /* Handle exception entry to a target EL which is using AArch64 */
 static void arm_cpu_do_interrupt_aarch64(CPUState *cs)
 {
@@ -10168,6 +10196,14 @@ static void arm_cpu_do_interrupt_aarch64(CPUState *cs)
     switch (cs->exception_index) {
     case EXCP_PREFETCH_ABORT:
     case EXCP_DATA_ABORT:
+        /*
+         * FEAT_DoubleFault allows synchronous external aborts taken to EL3
+         * to be taken to the SError vector entrypoint.
+         */
+        if (new_el == 3 && (env->cp15.scr_el3 & SCR_EASE) &&
+            syndrome_is_sync_extabt(env->exception.syndrome)) {
+            addr += 0x180;
+        }
         env->cp15.far_el[new_el] = env->exception.vaddress;
         qemu_log_mask(CPU_LOG_INT, "...with FAR 0x%" PRIx64 "\n",
                       env->cp15.far_el[new_el]);

From 9323e79f10e5f5d8fffc3b307776173ca11faeae Mon Sep 17 00:00:00 2001
From: Peter Maydell <peter.maydell@linaro.org>
Date: Wed, 8 Jun 2022 19:38:47 +0100
Subject: [PATCH 855/935] Fix 'writeable' typos
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

We have about 30 instances of the typo/variant spelling 'writeable',
and over 500 of the more common 'writable'.  Standardize on the
latter.

Change produced with:

  sed -i -e 's/\([Ww][Rr][Ii][Tt]\)[Ee]\([Aa][Bb][Ll][Ee]\)/\1\2/g' $(git grep -il writeable)

and then hand-undoing the instance in linux-headers/linux/kvm.h.

Most of these changes are in comments or documentation; the
exceptions are:
 * a local variable in accel/hvf/hvf-accel-ops.c
 * a local variable in accel/kvm/kvm-all.c
 * the PMCR_WRITABLE_MASK macro in target/arm/internals.h
 * the EPT_VIOLATION_GPA_WRITABLE macro in target/i386/hvf/vmcs.h
   (which is never used anywhere)
 * the AR_TYPE_WRITABLE_MASK macro in target/i386/hvf/vmx.h
   (which is never used anywhere)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Stefan Weil <sw@weilnetz.de>
Message-id: 20220505095015.2714666-1-peter.maydell@linaro.org
---
 accel/hvf/hvf-accel-ops.c      | 4 ++--
 accel/kvm/kvm-all.c            | 4 ++--
 accel/tcg/user-exec.c          | 6 +++---
 docs/interop/vhost-user.rst    | 2 +-
 docs/specs/vmgenid.txt         | 4 ++--
 hw/acpi/ghes.c                 | 2 +-
 hw/intc/arm_gicv3_cpuif.c      | 2 +-
 hw/intc/arm_gicv3_dist.c       | 2 +-
 hw/intc/arm_gicv3_redist.c     | 4 ++--
 hw/intc/riscv_aclint.c         | 2 +-
 hw/intc/riscv_aplic.c          | 2 +-
 hw/pci/shpc.c                  | 2 +-
 hw/scsi/mfi.h                  | 2 +-
 hw/sparc64/sun4u_iommu.c       | 2 +-
 hw/timer/sse-timer.c           | 2 +-
 python/qemu/machine/machine.py | 2 +-
 target/arm/gdbstub.c           | 2 +-
 target/arm/helper.c            | 4 ++--
 target/arm/hvf/hvf.c           | 4 ++--
 target/arm/internals.h         | 4 ++--
 target/i386/cpu-sysemu.c       | 2 +-
 target/i386/hvf/vmcs.h         | 2 +-
 target/i386/hvf/vmx.h          | 2 +-
 target/s390x/ioinst.c          | 2 +-
 tests/tcg/x86_64/system/boot.S | 2 +-
 25 files changed, 34 insertions(+), 34 deletions(-)

diff --git a/accel/hvf/hvf-accel-ops.c b/accel/hvf/hvf-accel-ops.c
index a70e2eb375..24913ca9c4 100644
--- a/accel/hvf/hvf-accel-ops.c
+++ b/accel/hvf/hvf-accel-ops.c
@@ -120,12 +120,12 @@ static void hvf_set_phys_mem(MemoryRegionSection *section, bool add)
 {
     hvf_slot *mem;
     MemoryRegion *area = section->mr;
-    bool writeable = !area->readonly && !area->rom_device;
+    bool writable = !area->readonly && !area->rom_device;
     hv_memory_flags_t flags;
     uint64_t page_size = qemu_real_host_page_size();
 
     if (!memory_region_is_ram(area)) {
-        if (writeable) {
+        if (writable) {
             return;
         } else if (!memory_region_is_romd(area)) {
             /*
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 32e177bd26..a4c4863f53 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -1346,13 +1346,13 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
     KVMSlot *mem;
     int err;
     MemoryRegion *mr = section->mr;
-    bool writeable = !mr->readonly && !mr->rom_device;
+    bool writable = !mr->readonly && !mr->rom_device;
     hwaddr start_addr, size, slot_size, mr_offset;
     ram_addr_t ram_start_offset;
     void *ram;
 
     if (!memory_region_is_ram(mr)) {
-        if (writeable || !kvm_readonly_mem_allowed) {
+        if (writable || !kvm_readonly_mem_allowed) {
             return;
         } else if (!mr->romd_mode) {
             /* If the memory device is not in romd_mode, then we actually want
diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c
index ac57324d4f..20ada5472b 100644
--- a/accel/tcg/user-exec.c
+++ b/accel/tcg/user-exec.c
@@ -101,10 +101,10 @@ MMUAccessType adjust_signal_pc(uintptr_t *pc, bool is_write)
  * Return true if the write fault has been handled, and should be re-tried.
  *
  * Note that it is important that we don't call page_unprotect() unless
- * this is really a "write to nonwriteable page" fault, because
+ * this is really a "write to nonwritable page" fault, because
  * page_unprotect() assumes that if it is called for an access to
- * a page that's writeable this means we had two threads racing and
- * another thread got there first and already made the page writeable;
+ * a page that's writable this means we had two threads racing and
+ * another thread got there first and already made the page writable;
  * so we will retry the access. If we were to call page_unprotect()
  * for some other kind of fault that should really be passed to the
  * guest, we'd end up in an infinite loop of retrying the faulting access.
diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
index a99ba4433c..d7cf904f7f 100644
--- a/docs/interop/vhost-user.rst
+++ b/docs/interop/vhost-user.rst
@@ -222,7 +222,7 @@ Virtio device config space
 :size: a 32-bit configuration space access size in bytes
 
 :flags: a 32-bit value:
-  - 0: Vhost front-end messages used for writeable fields
+  - 0: Vhost front-end messages used for writable fields
   - 1: Vhost front-end messages used for live migration
 
 :payload: Size bytes array holding the contents of the virtio
diff --git a/docs/specs/vmgenid.txt b/docs/specs/vmgenid.txt
index aa9f518676..80ff69f31c 100644
--- a/docs/specs/vmgenid.txt
+++ b/docs/specs/vmgenid.txt
@@ -153,7 +153,7 @@ change the contents of the memory at runtime, specifically when starting a
 backed-up or snapshotted image.  In order to do this, QEMU must know the
 address that has been allocated.
 
-The mechanism chosen for this memory sharing is writeable fw_cfg blobs.
+The mechanism chosen for this memory sharing is writable fw_cfg blobs.
 These are data object that are visible to both QEMU and guests, and are
 addressable as sequential files.
 
@@ -164,7 +164,7 @@ Two fw_cfg blobs are used in this case:
 /etc/vmgenid_guid - contains the actual VM Generation ID GUID
                   - read-only to the guest
 /etc/vmgenid_addr - contains the address of the downloaded vmgenid blob
-                  - writeable by the guest
+                  - writable by the guest
 
 
 QEMU sends the following commands to the guest at startup:
diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c
index 45d9a809cc..e9511d9b8f 100644
--- a/hw/acpi/ghes.c
+++ b/hw/acpi/ghes.c
@@ -249,7 +249,7 @@ void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker)
     for (i = 0; i < ACPI_GHES_ERROR_SOURCE_COUNT; i++) {
         /*
          * Initialize the value of read_ack_register to 1, so GHES can be
-         * writeable after (re)boot.
+         * writable after (re)boot.
          * ACPI 6.2: 18.3.2.8 Generic Hardware Error Source version 2
          * (GHESv2 - Type 10)
          */
diff --git a/hw/intc/arm_gicv3_cpuif.c b/hw/intc/arm_gicv3_cpuif.c
index 8867e2e496..8ca630e5ad 100644
--- a/hw/intc/arm_gicv3_cpuif.c
+++ b/hw/intc/arm_gicv3_cpuif.c
@@ -2047,7 +2047,7 @@ static void icc_ctlr_el3_write(CPUARMState *env, const ARMCPRegInfo *ri,
         cs->icc_ctlr_el1[GICV3_S] |= ICC_CTLR_EL1_CBPR;
     }
 
-    /* The only bit stored in icc_ctlr_el3 which is writeable is EOIMODE_EL3: */
+    /* The only bit stored in icc_ctlr_el3 which is writable is EOIMODE_EL3: */
     mask = ICC_CTLR_EL3_EOIMODE_EL3;
 
     cs->icc_ctlr_el3 &= ~mask;
diff --git a/hw/intc/arm_gicv3_dist.c b/hw/intc/arm_gicv3_dist.c
index b9ed955e36..eea0368118 100644
--- a/hw/intc/arm_gicv3_dist.c
+++ b/hw/intc/arm_gicv3_dist.c
@@ -611,7 +611,7 @@ static bool gicd_writel(GICv3State *s, hwaddr offset,
         if (value & mask & GICD_CTLR_DS) {
             /* We just set DS, so the ARE_NS and EnG1S bits are now RES0.
              * Note that this is a one-way transition because if DS is set
-             * then it's not writeable, so it can only go back to 0 with a
+             * then it's not writable, so it can only go back to 0 with a
              * hardware reset.
              */
             s->gicd_ctlr &= ~(GICD_CTLR_EN_GRP1S | GICD_CTLR_ARE_NS);
diff --git a/hw/intc/arm_gicv3_redist.c b/hw/intc/arm_gicv3_redist.c
index c3d4cdd66b..f1ecb2502b 100644
--- a/hw/intc/arm_gicv3_redist.c
+++ b/hw/intc/arm_gicv3_redist.c
@@ -257,7 +257,7 @@ static void gicr_write_vpendbaser(GICv3CPUState *cs, uint64_t newval)
 
     /*
      * The DIRTY bit is read-only and for us is always zero;
-     * other fields are writeable.
+     * other fields are writable.
      */
     newval &= R_GICR_VPENDBASER_INNERCACHE_MASK |
         R_GICR_VPENDBASER_SHAREABILITY_MASK |
@@ -491,7 +491,7 @@ static MemTxResult gicr_writel(GICv3CPUState *cs, hwaddr offset,
         /* RAZ/WI for our implementation */
         return MEMTX_OK;
     case GICR_WAKER:
-        /* Only the ProcessorSleep bit is writeable. When the guest sets
+        /* Only the ProcessorSleep bit is writable. When the guest sets
          * it it requests that we transition the channel between the
          * redistributor and the cpu interface to quiescent, and that
          * we set the ChildrenAsleep bit once the inteface has reached the
diff --git a/hw/intc/riscv_aclint.c b/hw/intc/riscv_aclint.c
index e6bceceefd..e7942c4e5a 100644
--- a/hw/intc/riscv_aclint.c
+++ b/hw/intc/riscv_aclint.c
@@ -463,7 +463,7 @@ static void riscv_aclint_swi_realize(DeviceState *dev, Error **errp)
     /* Claim software interrupt bits */
     for (i = 0; i < swi->num_harts; i++) {
         RISCVCPU *cpu = RISCV_CPU(qemu_get_cpu(swi->hartid_base + i));
-        /* We don't claim mip.SSIP because it is writeable by software */
+        /* We don't claim mip.SSIP because it is writable by software */
         if (riscv_cpu_claim_interrupts(cpu, swi->sswi ? 0 : MIP_MSIP) < 0) {
             error_report("MSIP already claimed");
             exit(1);
diff --git a/hw/intc/riscv_aplic.c b/hw/intc/riscv_aplic.c
index e7809fb6b2..cfd007e629 100644
--- a/hw/intc/riscv_aplic.c
+++ b/hw/intc/riscv_aplic.c
@@ -646,7 +646,7 @@ static void riscv_aplic_write(void *opaque, hwaddr addr, uint64_t value,
     }
 
     if (addr == APLIC_DOMAINCFG) {
-        /* Only IE bit writeable at the moment */
+        /* Only IE bit writable at the moment */
         value &= APLIC_DOMAINCFG_IE;
         aplic->domaincfg = value;
     } else if ((APLIC_SOURCECFG_BASE <= addr) &&
diff --git a/hw/pci/shpc.c b/hw/pci/shpc.c
index 28e62174c4..f822f18b98 100644
--- a/hw/pci/shpc.c
+++ b/hw/pci/shpc.c
@@ -456,7 +456,7 @@ static int shpc_cap_add_config(PCIDevice *d, Error **errp)
     pci_set_byte(config + SHPC_CAP_CxP, 0);
     pci_set_long(config + SHPC_CAP_DWORD_DATA, 0);
     d->shpc->cap = config_offset;
-    /* Make dword select and data writeable. */
+    /* Make dword select and data writable. */
     pci_set_byte(d->wmask + config_offset + SHPC_CAP_DWORD_SELECT, 0xff);
     pci_set_long(d->wmask + config_offset + SHPC_CAP_DWORD_DATA, 0xffffffff);
     return 0;
diff --git a/hw/scsi/mfi.h b/hw/scsi/mfi.h
index e67a5c0b47..0b4ee53dfc 100644
--- a/hw/scsi/mfi.h
+++ b/hw/scsi/mfi.h
@@ -633,7 +633,7 @@ struct mfi_ctrl_props {
                               * metadata and user data
                               * 1=5%, 2=10%, 3=15% and so on
                               */
-    uint8_t viewSpace;       /* snapshot writeable VIEWs
+    uint8_t viewSpace;       /* snapshot writable VIEWs
                               * capacity as a % of source LD
                               * capacity. 0=READ only
                               * 1=5%, 2=10%, 3=15% and so on
diff --git a/hw/sparc64/sun4u_iommu.c b/hw/sparc64/sun4u_iommu.c
index 9178277f82..1c1dca712e 100644
--- a/hw/sparc64/sun4u_iommu.c
+++ b/hw/sparc64/sun4u_iommu.c
@@ -165,7 +165,7 @@ static IOMMUTLBEntry sun4u_translate_iommu(IOMMUMemoryRegion *iommu,
     }
 
     if (tte & IOMMU_TTE_DATA_W) {
-        /* Writeable */
+        /* Writable */
         ret.perm = IOMMU_RW;
     } else {
         ret.perm = IOMMU_RO;
diff --git a/hw/timer/sse-timer.c b/hw/timer/sse-timer.c
index f959cb9d60..e92e83747d 100644
--- a/hw/timer/sse-timer.c
+++ b/hw/timer/sse-timer.c
@@ -324,7 +324,7 @@ static void sse_timer_write(void *opaque, hwaddr offset, uint64_t value,
     {
         uint32_t old_ctl = s->cntp_aival_ctl;
 
-        /* EN bit is writeable; CLR bit is write-0-to-clear, write-1-ignored */
+        /* EN bit is writable; CLR bit is write-0-to-clear, write-1-ignored */
         s->cntp_aival_ctl &= ~R_CNTP_AIVAL_CTL_EN_MASK;
         s->cntp_aival_ctl |= value & R_CNTP_AIVAL_CTL_EN_MASK;
         if (!(value & R_CNTP_AIVAL_CTL_CLR_MASK)) {
diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py
index 07ac5a710b..37191f433b 100644
--- a/python/qemu/machine/machine.py
+++ b/python/qemu/machine/machine.py
@@ -495,7 +495,7 @@ class QEMUMachine:
         """
         # If we keep the console socket open, we may deadlock waiting
         # for QEMU to exit, while QEMU is waiting for the socket to
-        # become writeable.
+        # become writable.
         if self._console_socket is not None:
             self._console_socket.close()
             self._console_socket = None
diff --git a/target/arm/gdbstub.c b/target/arm/gdbstub.c
index f5b35cd55f..2f806512d0 100644
--- a/target/arm/gdbstub.c
+++ b/target/arm/gdbstub.c
@@ -118,7 +118,7 @@ int arm_cpu_gdb_write_register(CPUState *cs, uint8_t *mem_buf, int n)
             /*
              * Don't allow writing to XPSR.Exception as it can cause
              * a transition into or out of handler mode (it's not
-             * writeable via the MSR insn so this is a reasonable
+             * writable via the MSR insn so this is a reasonable
              * restriction). Other fields are safe to update.
              */
             xpsr_write(env, tmp, ~XPSR_EXCP);
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 7f2c14bea9..5727ead5e4 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -1411,8 +1411,8 @@ static void pmcr_write(CPUARMState *env, const ARMCPRegInfo *ri,
         }
     }
 
-    env->cp15.c9_pmcr &= ~PMCR_WRITEABLE_MASK;
-    env->cp15.c9_pmcr |= (value & PMCR_WRITEABLE_MASK);
+    env->cp15.c9_pmcr &= ~PMCR_WRITABLE_MASK;
+    env->cp15.c9_pmcr |= (value & PMCR_WRITABLE_MASK);
 
     pmu_op_finish(env);
 }
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 1fdc5eef92..060aa0ccf4 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -978,8 +978,8 @@ static int hvf_sysreg_write(CPUState *cpu, uint32_t reg, uint64_t val)
             }
         }
 
-        env->cp15.c9_pmcr &= ~PMCR_WRITEABLE_MASK;
-        env->cp15.c9_pmcr |= (val & PMCR_WRITEABLE_MASK);
+        env->cp15.c9_pmcr &= ~PMCR_WRITABLE_MASK;
+        env->cp15.c9_pmcr |= (val & PMCR_WRITABLE_MASK);
 
         pmu_op_finish(env);
         break;
diff --git a/target/arm/internals.h b/target/arm/internals.h
index b654bee468..1e4887b2dd 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -1280,10 +1280,10 @@ enum MVEECIState {
 #define PMCRP   0x2
 #define PMCRE   0x1
 /*
- * Mask of PMCR bits writeable by guest (not including WO bits like C, P,
+ * Mask of PMCR bits writable by guest (not including WO bits like C, P,
  * which can be written as 1 to trigger behaviour but which stay RAZ).
  */
-#define PMCR_WRITEABLE_MASK (PMCRLC | PMCRDP | PMCRX | PMCRD | PMCRE)
+#define PMCR_WRITABLE_MASK (PMCRLC | PMCRDP | PMCRX | PMCRD | PMCRE)
 
 #define PMXEVTYPER_P          0x80000000
 #define PMXEVTYPER_U          0x40000000
diff --git a/target/i386/cpu-sysemu.c b/target/i386/cpu-sysemu.c
index e254d8ba10..a6f47b7d11 100644
--- a/target/i386/cpu-sysemu.c
+++ b/target/i386/cpu-sysemu.c
@@ -103,7 +103,7 @@ static void x86_cpu_to_dict(X86CPU *cpu, QDict *props)
 
 /* Convert CPU model data from X86CPU object to a property dictionary
  * that can recreate exactly the same CPU model, including every
- * writeable QOM property.
+ * writable QOM property.
  */
 static void x86_cpu_to_dict_full(X86CPU *cpu, QDict *props)
 {
diff --git a/target/i386/hvf/vmcs.h b/target/i386/hvf/vmcs.h
index 42de7ebc3a..b4692f63f6 100644
--- a/target/i386/hvf/vmcs.h
+++ b/target/i386/hvf/vmcs.h
@@ -330,7 +330,7 @@
 #define EPT_VIOLATION_DATA_WRITE (1UL << 1)
 #define EPT_VIOLATION_INST_FETCH (1UL << 2)
 #define EPT_VIOLATION_GPA_READABLE (1UL << 3)
-#define EPT_VIOLATION_GPA_WRITEABLE (1UL << 4)
+#define EPT_VIOLATION_GPA_WRITABLE (1UL << 4)
 #define EPT_VIOLATION_GPA_EXECUTABLE (1UL << 5)
 #define EPT_VIOLATION_GLA_VALID (1UL << 7)
 #define EPT_VIOLATION_XLAT_VALID (1UL << 8)
diff --git a/target/i386/hvf/vmx.h b/target/i386/hvf/vmx.h
index 573ddc33c0..fcd9a95e5b 100644
--- a/target/i386/hvf/vmx.h
+++ b/target/i386/hvf/vmx.h
@@ -80,7 +80,7 @@ static inline uint64_t cap2ctrl(uint64_t cap, uint64_t ctrl)
 
 #define AR_TYPE_ACCESSES_MASK 1
 #define AR_TYPE_READABLE_MASK (1 << 1)
-#define AR_TYPE_WRITEABLE_MASK (1 << 2)
+#define AR_TYPE_WRITABLE_MASK (1 << 2)
 #define AR_TYPE_CODE_MASK (1 << 3)
 #define AR_TYPE_MASK 0x0f
 #define AR_TYPE_BUSY_64_TSS 11
diff --git a/target/s390x/ioinst.c b/target/s390x/ioinst.c
index bdae5090bc..b12f18d346 100644
--- a/target/s390x/ioinst.c
+++ b/target/s390x/ioinst.c
@@ -284,7 +284,7 @@ void ioinst_handle_stsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb,
         g_assert(!s390_is_pv());
         /*
          * As operand exceptions have a lower priority than access exceptions,
-         * we check whether the memory area is writeable (injecting the
+         * we check whether the memory area is writable (injecting the
          * access execption if it is not) first.
          */
         if (!s390_cpu_virt_mem_check_write(cpu, addr, ar, sizeof(schib))) {
diff --git a/tests/tcg/x86_64/system/boot.S b/tests/tcg/x86_64/system/boot.S
index f8a2fcc839..ed0f638406 100644
--- a/tests/tcg/x86_64/system/boot.S
+++ b/tests/tcg/x86_64/system/boot.S
@@ -56,7 +56,7 @@
 	*
 	* - `ebx`: contains the physical memory address where the loader has placed
 	*          the boot start info structure.
-	* - `cr0`: bit 0 (PE) must be set. All the other writeable bits are cleared.
+	* - `cr0`: bit 0 (PE) must be set. All the other writable bits are cleared.
 	* - `cr4`: all bits are cleared.
 	* - `cs `: must be a 32-bit read/execute code segment with a base of ‘0’
 	*          and a limit of ‘0xFFFFFFFF’. The selector value is unspecified.

From d2008b3355061ab0cca823739cd98045e769a809 Mon Sep 17 00:00:00 2001
From: Frederic Konrad <fkonrad@amd.com>
Date: Wed, 8 Jun 2022 19:38:47 +0100
Subject: [PATCH 856/935] xlnx_dp: fix the wrong register size

The core and the vblend registers size are wrong, they should respectively be
0x3B0 and 0x1E0 according to:
  https://www.xilinx.com/htmldocs/registers/ug1087/ug1087-zynq-ultrascale-registers.html.

Let's fix that and use macros when creating the mmio region.

Fixes: 58ac482a66d ("introduce xlnx-dp")
Signed-off-by: Frederic Konrad <fkonrad@amd.com>
Reviewed-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20220601172353.3220232-2-fkonrad@xilinx.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/display/xlnx_dp.c         | 17 ++++++++++-------
 include/hw/display/xlnx_dp.h |  9 +++++++--
 2 files changed, 17 insertions(+), 9 deletions(-)

diff --git a/hw/display/xlnx_dp.c b/hw/display/xlnx_dp.c
index 9bb781e312..0378570459 100644
--- a/hw/display/xlnx_dp.c
+++ b/hw/display/xlnx_dp.c
@@ -1219,19 +1219,22 @@ static void xlnx_dp_init(Object *obj)
     SysBusDevice *sbd = SYS_BUS_DEVICE(obj);
     XlnxDPState *s = XLNX_DP(obj);
 
-    memory_region_init(&s->container, obj, TYPE_XLNX_DP, 0xC050);
+    memory_region_init(&s->container, obj, TYPE_XLNX_DP, DP_CONTAINER_SIZE);
 
     memory_region_init_io(&s->core_iomem, obj, &dp_ops, s, TYPE_XLNX_DP
-                          ".core", 0x3AF);
-    memory_region_add_subregion(&s->container, 0x0000, &s->core_iomem);
+                          ".core", sizeof(s->core_registers));
+    memory_region_add_subregion(&s->container, DP_CORE_REG_OFFSET,
+                                &s->core_iomem);
 
     memory_region_init_io(&s->vblend_iomem, obj, &vblend_ops, s, TYPE_XLNX_DP
-                          ".v_blend", 0x1DF);
-    memory_region_add_subregion(&s->container, 0xA000, &s->vblend_iomem);
+                          ".v_blend", sizeof(s->vblend_registers));
+    memory_region_add_subregion(&s->container, DP_VBLEND_REG_OFFSET,
+                                &s->vblend_iomem);
 
     memory_region_init_io(&s->avbufm_iomem, obj, &avbufm_ops, s, TYPE_XLNX_DP
-                          ".av_buffer_manager", 0x238);
-    memory_region_add_subregion(&s->container, 0xB000, &s->avbufm_iomem);
+                          ".av_buffer_manager", sizeof(s->avbufm_registers));
+    memory_region_add_subregion(&s->container, DP_AVBUF_REG_OFFSET,
+                                &s->avbufm_iomem);
 
     memory_region_init_io(&s->audio_iomem, obj, &audio_ops, s, TYPE_XLNX_DP
                           ".audio", sizeof(s->audio_registers));
diff --git a/include/hw/display/xlnx_dp.h b/include/hw/display/xlnx_dp.h
index 8ab4733bb8..1ef5a89ee7 100644
--- a/include/hw/display/xlnx_dp.h
+++ b/include/hw/display/xlnx_dp.h
@@ -39,10 +39,15 @@
 #define AUD_CHBUF_MAX_DEPTH                 (32 * KiB)
 #define MAX_QEMU_BUFFER_SIZE                (4 * KiB)
 
-#define DP_CORE_REG_ARRAY_SIZE              (0x3AF >> 2)
+#define DP_CORE_REG_OFFSET                  (0x0000)
+#define DP_CORE_REG_ARRAY_SIZE              (0x3B0 >> 2)
+#define DP_AVBUF_REG_OFFSET                 (0xB000)
 #define DP_AVBUF_REG_ARRAY_SIZE             (0x238 >> 2)
-#define DP_VBLEND_REG_ARRAY_SIZE            (0x1DF >> 2)
+#define DP_VBLEND_REG_OFFSET                (0xA000)
+#define DP_VBLEND_REG_ARRAY_SIZE            (0x1E0 >> 2)
+#define DP_AUDIO_REG_OFFSET                 (0xC000)
 #define DP_AUDIO_REG_ARRAY_SIZE             (0x50 >> 2)
+#define DP_CONTAINER_SIZE                   (0xC050)
 
 struct PixmanPlane {
     pixman_format_code_t format;

From 759ae1b47e7e0203eb6881304b4bbf0d935fd550 Mon Sep 17 00:00:00 2001
From: Sai Pavan Boddu <sai.pavan.boddu@xilinx.com>
Date: Wed, 8 Jun 2022 19:38:47 +0100
Subject: [PATCH 857/935] xlnx_dp: Introduce a vblank signal

Add a periodic timer which raises vblank at a frequency of 30Hz.

Note that this is a migration compatibility break for the
xlnx-zcu102 board type.

Signed-off-by: Sai Pavan Boddu <saipava@xilinx.com>
Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Signed-off-by: Frederic Konrad <fkonrad@amd.com>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20220601172353.3220232-3-fkonrad@xilinx.com
Changes by fkonrad:
  - Switched to transaction-based ptimer API.
  - Added the DP_INT_VBLNK_START macro.
Signed-off-by: Frederic Konrad <fkonrad@amd.com>
[PMM: bump vmstate version, add commit message note about
 compat break]
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/display/xlnx_dp.c         | 30 ++++++++++++++++++++++++++----
 include/hw/display/xlnx_dp.h |  3 +++
 2 files changed, 29 insertions(+), 4 deletions(-)

diff --git a/hw/display/xlnx_dp.c b/hw/display/xlnx_dp.c
index 0378570459..ed856b596d 100644
--- a/hw/display/xlnx_dp.c
+++ b/hw/display/xlnx_dp.c
@@ -114,6 +114,7 @@
 #define DP_TX_N_AUD                         (0x032C >> 2)
 #define DP_TX_AUDIO_EXT_DATA(n)             ((0x0330 + 4 * n) >> 2)
 #define DP_INT_STATUS                       (0x03A0 >> 2)
+#define DP_INT_VBLNK_START                  (1 << 13)
 #define DP_INT_MASK                         (0x03A4 >> 2)
 #define DP_INT_EN                           (0x03A8 >> 2)
 #define DP_INT_DS                           (0x03AC >> 2)
@@ -260,7 +261,7 @@ typedef enum DPVideoFmt DPVideoFmt;
 
 static const VMStateDescription vmstate_dp = {
     .name = TYPE_XLNX_DP,
-    .version_id = 1,
+    .version_id = 2,
     .fields = (VMStateField[]){
         VMSTATE_UINT32_ARRAY(core_registers, XlnxDPState,
                              DP_CORE_REG_ARRAY_SIZE),
@@ -270,10 +271,15 @@ static const VMStateDescription vmstate_dp = {
                              DP_VBLEND_REG_ARRAY_SIZE),
         VMSTATE_UINT32_ARRAY(audio_registers, XlnxDPState,
                              DP_AUDIO_REG_ARRAY_SIZE),
+        VMSTATE_PTIMER(vblank, XlnxDPState),
         VMSTATE_END_OF_LIST()
     }
 };
 
+#define DP_VBLANK_PTIMER_POLICY (PTIMER_POLICY_WRAP_AFTER_ONE_PERIOD | \
+                                 PTIMER_POLICY_CONTINUOUS_TRIGGER |    \
+                                 PTIMER_POLICY_NO_IMMEDIATE_TRIGGER)
+
 static void xlnx_dp_update_irq(XlnxDPState *s);
 
 static uint64_t xlnx_dp_audio_read(void *opaque, hwaddr offset, unsigned size)
@@ -773,6 +779,13 @@ static void xlnx_dp_write(void *opaque, hwaddr offset, uint64_t value,
         break;
     case DP_TRANSMITTER_ENABLE:
         s->core_registers[offset] = value & 0x01;
+        ptimer_transaction_begin(s->vblank);
+        if (value & 0x1) {
+            ptimer_run(s->vblank, 0);
+        } else {
+            ptimer_stop(s->vblank);
+        }
+        ptimer_transaction_commit(s->vblank);
         break;
     case DP_FORCE_SCRAMBLER_RESET:
         /*
@@ -1177,9 +1190,6 @@ static void xlnx_dp_update_display(void *opaque)
         return;
     }
 
-    s->core_registers[DP_INT_STATUS] |= (1 << 13);
-    xlnx_dp_update_irq(s);
-
     xlnx_dpdma_trigger_vsync_irq(s->dpdma);
 
     /*
@@ -1275,6 +1285,14 @@ static void xlnx_dp_finalize(Object *obj)
     fifo8_destroy(&s->rx_fifo);
 }
 
+static void vblank_hit(void *opaque)
+{
+    XlnxDPState *s = XLNX_DP(opaque);
+
+    s->core_registers[DP_INT_STATUS] |= DP_INT_VBLNK_START;
+    xlnx_dp_update_irq(s);
+}
+
 static void xlnx_dp_realize(DeviceState *dev, Error **errp)
 {
     XlnxDPState *s = XLNX_DP(dev);
@@ -1309,6 +1327,10 @@ static void xlnx_dp_realize(DeviceState *dev, Error **errp)
                                            &as);
     AUD_set_volume_out(s->amixer_output_stream, 0, 255, 255);
     xlnx_dp_audio_activate(s);
+    s->vblank = ptimer_init(vblank_hit, s, DP_VBLANK_PTIMER_POLICY);
+    ptimer_transaction_begin(s->vblank);
+    ptimer_set_freq(s->vblank, 30);
+    ptimer_transaction_commit(s->vblank);
 }
 
 static void xlnx_dp_reset(DeviceState *dev)
diff --git a/include/hw/display/xlnx_dp.h b/include/hw/display/xlnx_dp.h
index 1ef5a89ee7..e86a87f235 100644
--- a/include/hw/display/xlnx_dp.h
+++ b/include/hw/display/xlnx_dp.h
@@ -35,6 +35,7 @@
 #include "hw/dma/xlnx_dpdma.h"
 #include "audio/audio.h"
 #include "qom/object.h"
+#include "hw/ptimer.h"
 
 #define AUD_CHBUF_MAX_DEPTH                 (32 * KiB)
 #define MAX_QEMU_BUFFER_SIZE                (4 * KiB)
@@ -107,6 +108,8 @@ struct XlnxDPState {
      */
     DPCDState *dpcd;
     I2CDDCState *edid;
+
+    ptimer_state *vblank;
 };
 
 #define TYPE_XLNX_DP "xlnx.v-dp"

From 39f40d02f6af8e4718100b6a3854e963c962cfd7 Mon Sep 17 00:00:00 2001
From: Sai Pavan Boddu <sai.pavan.boddu@xilinx.com>
Date: Wed, 8 Jun 2022 19:38:47 +0100
Subject: [PATCH 858/935] xlnx_dp: Fix the interrupt disable logic

Fix interrupt disable logic. Mask value 1 indicates that interrupts are
disabled.

Signed-off-by: Sai Pavan Boddu <saipava@xilinx.com>
Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Signed-off-by: Frederic Konrad <fkonrad@amd.com>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20220601172353.3220232-4-fkonrad@xilinx.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/display/xlnx_dp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/display/xlnx_dp.c b/hw/display/xlnx_dp.c
index ed856b596d..a071c81883 100644
--- a/hw/display/xlnx_dp.c
+++ b/hw/display/xlnx_dp.c
@@ -889,7 +889,7 @@ static void xlnx_dp_write(void *opaque, hwaddr offset, uint64_t value,
         xlnx_dp_update_irq(s);
         break;
     case DP_INT_DS:
-        s->core_registers[DP_INT_MASK] |= ~value;
+        s->core_registers[DP_INT_MASK] |= value;
         xlnx_dp_update_irq(s);
         break;
     default:

From b3f5cc3fda5775b34aa52e111d50c9d10911b352 Mon Sep 17 00:00:00 2001
From: Frederic Konrad <fkonrad@amd.com>
Date: Wed, 8 Jun 2022 19:38:48 +0100
Subject: [PATCH 859/935] xlnx-zynqmp: fix the irq mapping for the display port
 and its dma

When the display port has been initially implemented the device
driver wasn't using interrupts.  Now that the display port driver
waits for vblank interrupt it has been noticed that the irq mapping
is wrong.  So use the value from the linux device tree and the
ultrascale+ reference manual.

Signed-off-by: Frederic Konrad <fkonrad@amd.com>
Reviewed-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20220601172353.3220232-5-fkonrad@xilinx.com
[PMM: refold lines in commit message]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 hw/arm/xlnx-zynqmp.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/arm/xlnx-zynqmp.c b/hw/arm/xlnx-zynqmp.c
index 375309e68e..383e177a00 100644
--- a/hw/arm/xlnx-zynqmp.c
+++ b/hw/arm/xlnx-zynqmp.c
@@ -60,10 +60,10 @@
 #define SERDES_SIZE         0x20000
 
 #define DP_ADDR             0xfd4a0000
-#define DP_IRQ              113
+#define DP_IRQ              0x77
 
 #define DPDMA_ADDR          0xfd4c0000
-#define DPDMA_IRQ           116
+#define DPDMA_IRQ           0x7a
 
 #define APU_ADDR            0xfd5c0000
 #define APU_IRQ             153

From d8cca960a9a5dbd216d2331cef3cc72f2c7338e0 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:48 +0100
Subject: [PATCH 860/935] target/arm: Move stage_1_mmu_idx decl to internals.h

Move the decl from ptw.h to internals.h.  Provide an inline
version for user-only, just as we do for arm_stage1_mmu_idx.
Move an endif down to make the definition in helper.c be
system only.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-2-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c    | 5 ++---
 target/arm/internals.h | 5 +++++
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 5727ead5e4..829b660db9 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10517,12 +10517,10 @@ static inline uint64_t regime_ttbr(CPUARMState *env, ARMMMUIdx mmu_idx,
     }
 }
 
-#endif /* !CONFIG_USER_ONLY */
-
 /* Convert a possible stage1+2 MMU index into the appropriate
  * stage 1 MMU index
  */
-static inline ARMMMUIdx stage_1_mmu_idx(ARMMMUIdx mmu_idx)
+ARMMMUIdx stage_1_mmu_idx(ARMMMUIdx mmu_idx)
 {
     switch (mmu_idx) {
     case ARMMMUIdx_SE10_0:
@@ -10541,6 +10539,7 @@ static inline ARMMMUIdx stage_1_mmu_idx(ARMMMUIdx mmu_idx)
         return mmu_idx;
     }
 }
+#endif /* !CONFIG_USER_ONLY */
 
 /* Return true if the translation regime is using LPAE format page tables */
 static inline bool regime_using_lpae_format(CPUARMState *env,
diff --git a/target/arm/internals.h b/target/arm/internals.h
index 1e4887b2dd..049edce946 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -979,11 +979,16 @@ ARMMMUIdx arm_mmu_idx(CPUARMState *env);
  * Return the ARMMMUIdx for the stage1 traversal for the current regime.
  */
 #ifdef CONFIG_USER_ONLY
+static inline ARMMMUIdx stage_1_mmu_idx(ARMMMUIdx mmu_idx)
+{
+    return ARMMMUIdx_Stage1_E0;
+}
 static inline ARMMMUIdx arm_stage1_mmu_idx(CPUARMState *env)
 {
     return ARMMMUIdx_Stage1_E0;
 }
 #else
+ARMMMUIdx stage_1_mmu_idx(ARMMMUIdx mmu_idx);
 ARMMMUIdx arm_stage1_mmu_idx(CPUARMState *env);
 #endif
 

From 8ae0886002aaf917601701df933617b2449e94b5 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:48 +0100
Subject: [PATCH 861/935] target/arm: Move get_phys_addr to ptw.c

Begin moving all of the page table walking functions
out of helper.c, starting with get_phys_addr().

Create a temporary header file, "ptw.h", in which to
share declarations between the two C files while we
are moving functions.

Move a few declarations to "internals.h", which will
remain used by multiple C files.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-3-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c    | 344 +++++------------------------------------
 target/arm/internals.h |  18 ++-
 target/arm/meson.build |   1 +
 target/arm/ptw.c       | 267 ++++++++++++++++++++++++++++++++
 target/arm/ptw.h       |  51 ++++++
 5 files changed, 372 insertions(+), 309 deletions(-)
 create mode 100644 target/arm/ptw.c
 create mode 100644 target/arm/ptw.h

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 829b660db9..3ffd122178 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -37,22 +37,11 @@
 #include "semihosting/common-semi.h"
 #endif
 #include "cpregs.h"
+#include "ptw.h"
 
 #define ARM_CPU_FREQ 1000000000 /* FIXME: 1 GHz, should be configurable */
 
-#ifndef CONFIG_USER_ONLY
-
-static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
-                               MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                               bool s1_is_el0,
-                               hwaddr *phys_ptr, MemTxAttrs *txattrs, int *prot,
-                               target_ulong *page_size_ptr,
-                               ARMMMUFaultInfo *fi, ARMCacheAttrs *cacheattrs)
-    __attribute__((nonnull));
-#endif
-
 static void switch_mode(CPUARMState *env, int mode);
-static int aa64_va_parameter_tbi(uint64_t tcr, ARMMMUIdx mmu_idx);
 
 static uint64_t raw_read(CPUARMState *env, const ARMCPRegInfo *ri)
 {
@@ -10440,17 +10429,10 @@ uint64_t arm_sctlr(CPUARMState *env, int el)
     return env->cp15.sctlr_el[el];
 }
 
-/* Return the SCTLR value which controls this address translation regime */
-static inline uint64_t regime_sctlr(CPUARMState *env, ARMMMUIdx mmu_idx)
-{
-    return env->cp15.sctlr_el[regime_el(env, mmu_idx)];
-}
-
 #ifndef CONFIG_USER_ONLY
 
 /* Return true if the specified stage of address translation is disabled */
-static inline bool regime_translation_disabled(CPUARMState *env,
-                                               ARMMMUIdx mmu_idx)
+bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx)
 {
     uint64_t hcr_el2;
 
@@ -10542,8 +10524,7 @@ ARMMMUIdx stage_1_mmu_idx(ARMMMUIdx mmu_idx)
 #endif /* !CONFIG_USER_ONLY */
 
 /* Return true if the translation regime is using LPAE format page tables */
-static inline bool regime_using_lpae_format(CPUARMState *env,
-                                            ARMMMUIdx mmu_idx)
+bool regime_using_lpae_format(CPUARMState *env, ARMMMUIdx mmu_idx)
 {
     int el = regime_el(env, mmu_idx);
     if (el == 2 || arm_el_is_aa64(env, el)) {
@@ -10567,7 +10548,7 @@ bool arm_s1_regime_using_lpae_format(CPUARMState *env, ARMMMUIdx mmu_idx)
 }
 
 #ifndef CONFIG_USER_ONLY
-static inline bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx)
+bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx)
 {
     switch (mmu_idx) {
     case ARMMMUIdx_SE10_0:
@@ -10959,11 +10940,11 @@ static uint64_t arm_ldq_ptw(CPUState *cs, hwaddr addr, bool is_secure,
     return 0;
 }
 
-static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
-                             MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                             hwaddr *phys_ptr, int *prot,
-                             target_ulong *page_size,
-                             ARMMMUFaultInfo *fi)
+bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
+                      MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                      hwaddr *phys_ptr, int *prot,
+                      target_ulong *page_size,
+                      ARMMMUFaultInfo *fi)
 {
     CPUState *cs = env_cpu(env);
     int level = 1;
@@ -11081,10 +11062,10 @@ do_fault:
     return true;
 }
 
-static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
-                             MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                             hwaddr *phys_ptr, MemTxAttrs *attrs, int *prot,
-                             target_ulong *page_size, ARMMMUFaultInfo *fi)
+bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
+                      MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                      hwaddr *phys_ptr, MemTxAttrs *attrs, int *prot,
+                      target_ulong *page_size, ARMMMUFaultInfo *fi)
 {
     CPUState *cs = env_cpu(env);
     ARMCPU *cpu = env_archcpu(env);
@@ -11360,7 +11341,7 @@ unsigned int arm_pamax(ARMCPU *cpu)
     return pamax_map[parange];
 }
 
-static int aa64_va_parameter_tbi(uint64_t tcr, ARMMMUIdx mmu_idx)
+int aa64_va_parameter_tbi(uint64_t tcr, ARMMMUIdx mmu_idx)
 {
     if (regime_has_2_ranges(mmu_idx)) {
         return extract64(tcr, 37, 2);
@@ -11372,7 +11353,7 @@ static int aa64_va_parameter_tbi(uint64_t tcr, ARMMMUIdx mmu_idx)
     }
 }
 
-static int aa64_va_parameter_tbid(uint64_t tcr, ARMMMUIdx mmu_idx)
+int aa64_va_parameter_tbid(uint64_t tcr, ARMMMUIdx mmu_idx)
 {
     if (regime_has_2_ranges(mmu_idx)) {
         return extract64(tcr, 51, 2);
@@ -11602,12 +11583,12 @@ static ARMVAParameters aa32_va_parameters(CPUARMState *env, uint32_t va,
  * @fi: set to fault info if the translation fails
  * @cacheattrs: (if non-NULL) set to the cacheability/shareability attributes
  */
-static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
-                               MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                               bool s1_is_el0,
-                               hwaddr *phys_ptr, MemTxAttrs *txattrs, int *prot,
-                               target_ulong *page_size_ptr,
-                               ARMMMUFaultInfo *fi, ARMCacheAttrs *cacheattrs)
+bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
+                        MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                        bool s1_is_el0,
+                        hwaddr *phys_ptr, MemTxAttrs *txattrs, int *prot,
+                        target_ulong *page_size_ptr,
+                        ARMMMUFaultInfo *fi, ARMCacheAttrs *cacheattrs)
 {
     ARMCPU *cpu = env_archcpu(env);
     CPUState *cs = CPU(cpu);
@@ -12055,11 +12036,11 @@ static inline bool m_is_system_region(CPUARMState *env, uint32_t address)
     return arm_feature(env, ARM_FEATURE_M) && extract32(address, 29, 3) == 0x7;
 }
 
-static bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
-                                 MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                                 hwaddr *phys_ptr, int *prot,
-                                 target_ulong *page_size,
-                                 ARMMMUFaultInfo *fi)
+bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
+                          MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                          hwaddr *phys_ptr, int *prot,
+                          target_ulong *page_size,
+                          ARMMMUFaultInfo *fi)
 {
     ARMCPU *cpu = env_archcpu(env);
     int n;
@@ -12501,11 +12482,11 @@ bool pmsav8_mpu_lookup(CPUARMState *env, uint32_t address,
 }
 
 
-static bool get_phys_addr_pmsav8(CPUARMState *env, uint32_t address,
-                                 MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                                 hwaddr *phys_ptr, MemTxAttrs *txattrs,
-                                 int *prot, target_ulong *page_size,
-                                 ARMMMUFaultInfo *fi)
+bool get_phys_addr_pmsav8(CPUARMState *env, uint32_t address,
+                          MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                          hwaddr *phys_ptr, MemTxAttrs *txattrs,
+                          int *prot, target_ulong *page_size,
+                          ARMMMUFaultInfo *fi)
 {
     uint32_t secure = regime_is_secure(env, mmu_idx);
     V8M_SAttributes sattrs = {};
@@ -12575,10 +12556,10 @@ static bool get_phys_addr_pmsav8(CPUARMState *env, uint32_t address,
     return ret;
 }
 
-static bool get_phys_addr_pmsav5(CPUARMState *env, uint32_t address,
-                                 MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                                 hwaddr *phys_ptr, int *prot,
-                                 ARMMMUFaultInfo *fi)
+bool get_phys_addr_pmsav5(CPUARMState *env, uint32_t address,
+                          MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                          hwaddr *phys_ptr, int *prot,
+                          ARMMMUFaultInfo *fi)
 {
     int n;
     uint32_t mask;
@@ -12795,8 +12776,8 @@ static uint8_t combined_attrs_fwb(CPUARMState *env,
  * @s1:      Attributes from stage 1 walk
  * @s2:      Attributes from stage 2 walk
  */
-static ARMCacheAttrs combine_cacheattrs(CPUARMState *env,
-                                        ARMCacheAttrs s1, ARMCacheAttrs s2)
+ARMCacheAttrs combine_cacheattrs(CPUARMState *env,
+                                 ARMCacheAttrs s1, ARMCacheAttrs s2)
 {
     ARMCacheAttrs ret;
     bool tagged = false;
@@ -12848,256 +12829,6 @@ static ARMCacheAttrs combine_cacheattrs(CPUARMState *env,
     return ret;
 }
 
-
-/* get_phys_addr - get the physical address for this virtual address
- *
- * Find the physical address corresponding to the given virtual address,
- * by doing a translation table walk on MMU based systems or using the
- * MPU state on MPU based systems.
- *
- * Returns false if the translation was successful. Otherwise, phys_ptr, attrs,
- * prot and page_size may not be filled in, and the populated fsr value provides
- * information on why the translation aborted, in the format of a
- * DFSR/IFSR fault register, with the following caveats:
- *  * we honour the short vs long DFSR format differences.
- *  * the WnR bit is never set (the caller must do this).
- *  * for PSMAv5 based systems we don't bother to return a full FSR format
- *    value.
- *
- * @env: CPUARMState
- * @address: virtual address to get physical address for
- * @access_type: 0 for read, 1 for write, 2 for execute
- * @mmu_idx: MMU index indicating required translation regime
- * @phys_ptr: set to the physical address corresponding to the virtual address
- * @attrs: set to the memory transaction attributes to use
- * @prot: set to the permissions for the page containing phys_ptr
- * @page_size: set to the size of the page containing phys_ptr
- * @fi: set to fault info if the translation fails
- * @cacheattrs: (if non-NULL) set to the cacheability/shareability attributes
- */
-bool get_phys_addr(CPUARMState *env, target_ulong address,
-                   MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                   hwaddr *phys_ptr, MemTxAttrs *attrs, int *prot,
-                   target_ulong *page_size,
-                   ARMMMUFaultInfo *fi, ARMCacheAttrs *cacheattrs)
-{
-    ARMMMUIdx s1_mmu_idx = stage_1_mmu_idx(mmu_idx);
-
-    if (mmu_idx != s1_mmu_idx) {
-        /* Call ourselves recursively to do the stage 1 and then stage 2
-         * translations if mmu_idx is a two-stage regime.
-         */
-        if (arm_feature(env, ARM_FEATURE_EL2)) {
-            hwaddr ipa;
-            int s2_prot;
-            int ret;
-            bool ipa_secure;
-            ARMCacheAttrs cacheattrs2 = {};
-            ARMMMUIdx s2_mmu_idx;
-            bool is_el0;
-
-            ret = get_phys_addr(env, address, access_type, s1_mmu_idx, &ipa,
-                                attrs, prot, page_size, fi, cacheattrs);
-
-            /* If S1 fails or S2 is disabled, return early.  */
-            if (ret || regime_translation_disabled(env, ARMMMUIdx_Stage2)) {
-                *phys_ptr = ipa;
-                return ret;
-            }
-
-            ipa_secure = attrs->secure;
-            if (arm_is_secure_below_el3(env)) {
-                if (ipa_secure) {
-                    attrs->secure = !(env->cp15.vstcr_el2.raw_tcr & VSTCR_SW);
-                } else {
-                    attrs->secure = !(env->cp15.vtcr_el2.raw_tcr & VTCR_NSW);
-                }
-            } else {
-                assert(!ipa_secure);
-            }
-
-            s2_mmu_idx = attrs->secure ? ARMMMUIdx_Stage2_S : ARMMMUIdx_Stage2;
-            is_el0 = mmu_idx == ARMMMUIdx_E10_0 || mmu_idx == ARMMMUIdx_SE10_0;
-
-            /* S1 is done. Now do S2 translation.  */
-            ret = get_phys_addr_lpae(env, ipa, access_type, s2_mmu_idx, is_el0,
-                                     phys_ptr, attrs, &s2_prot,
-                                     page_size, fi, &cacheattrs2);
-            fi->s2addr = ipa;
-            /* Combine the S1 and S2 perms.  */
-            *prot &= s2_prot;
-
-            /* If S2 fails, return early.  */
-            if (ret) {
-                return ret;
-            }
-
-            /* Combine the S1 and S2 cache attributes. */
-            if (arm_hcr_el2_eff(env) & HCR_DC) {
-                /*
-                 * HCR.DC forces the first stage attributes to
-                 *  Normal Non-Shareable,
-                 *  Inner Write-Back Read-Allocate Write-Allocate,
-                 *  Outer Write-Back Read-Allocate Write-Allocate.
-                 * Do not overwrite Tagged within attrs.
-                 */
-                if (cacheattrs->attrs != 0xf0) {
-                    cacheattrs->attrs = 0xff;
-                }
-                cacheattrs->shareability = 0;
-            }
-            *cacheattrs = combine_cacheattrs(env, *cacheattrs, cacheattrs2);
-
-            /* Check if IPA translates to secure or non-secure PA space. */
-            if (arm_is_secure_below_el3(env)) {
-                if (ipa_secure) {
-                    attrs->secure =
-                        !(env->cp15.vstcr_el2.raw_tcr & (VSTCR_SA | VSTCR_SW));
-                } else {
-                    attrs->secure =
-                        !((env->cp15.vtcr_el2.raw_tcr & (VTCR_NSA | VTCR_NSW))
-                        || (env->cp15.vstcr_el2.raw_tcr & (VSTCR_SA | VSTCR_SW)));
-                }
-            }
-            return 0;
-        } else {
-            /*
-             * For non-EL2 CPUs a stage1+stage2 translation is just stage 1.
-             */
-            mmu_idx = stage_1_mmu_idx(mmu_idx);
-        }
-    }
-
-    /* The page table entries may downgrade secure to non-secure, but
-     * cannot upgrade an non-secure translation regime's attributes
-     * to secure.
-     */
-    attrs->secure = regime_is_secure(env, mmu_idx);
-    attrs->user = regime_is_user(env, mmu_idx);
-
-    /* Fast Context Switch Extension. This doesn't exist at all in v8.
-     * In v7 and earlier it affects all stage 1 translations.
-     */
-    if (address < 0x02000000 && mmu_idx != ARMMMUIdx_Stage2
-        && !arm_feature(env, ARM_FEATURE_V8)) {
-        if (regime_el(env, mmu_idx) == 3) {
-            address += env->cp15.fcseidr_s;
-        } else {
-            address += env->cp15.fcseidr_ns;
-        }
-    }
-
-    if (arm_feature(env, ARM_FEATURE_PMSA)) {
-        bool ret;
-        *page_size = TARGET_PAGE_SIZE;
-
-        if (arm_feature(env, ARM_FEATURE_V8)) {
-            /* PMSAv8 */
-            ret = get_phys_addr_pmsav8(env, address, access_type, mmu_idx,
-                                       phys_ptr, attrs, prot, page_size, fi);
-        } else if (arm_feature(env, ARM_FEATURE_V7)) {
-            /* PMSAv7 */
-            ret = get_phys_addr_pmsav7(env, address, access_type, mmu_idx,
-                                       phys_ptr, prot, page_size, fi);
-        } else {
-            /* Pre-v7 MPU */
-            ret = get_phys_addr_pmsav5(env, address, access_type, mmu_idx,
-                                       phys_ptr, prot, fi);
-        }
-        qemu_log_mask(CPU_LOG_MMU, "PMSA MPU lookup for %s at 0x%08" PRIx32
-                      " mmu_idx %u -> %s (prot %c%c%c)\n",
-                      access_type == MMU_DATA_LOAD ? "reading" :
-                      (access_type == MMU_DATA_STORE ? "writing" : "execute"),
-                      (uint32_t)address, mmu_idx,
-                      ret ? "Miss" : "Hit",
-                      *prot & PAGE_READ ? 'r' : '-',
-                      *prot & PAGE_WRITE ? 'w' : '-',
-                      *prot & PAGE_EXEC ? 'x' : '-');
-
-        return ret;
-    }
-
-    /* Definitely a real MMU, not an MPU */
-
-    if (regime_translation_disabled(env, mmu_idx)) {
-        uint64_t hcr;
-        uint8_t memattr;
-
-        /*
-         * MMU disabled.  S1 addresses within aa64 translation regimes are
-         * still checked for bounds -- see AArch64.TranslateAddressS1Off.
-         */
-        if (mmu_idx != ARMMMUIdx_Stage2 && mmu_idx != ARMMMUIdx_Stage2_S) {
-            int r_el = regime_el(env, mmu_idx);
-            if (arm_el_is_aa64(env, r_el)) {
-                int pamax = arm_pamax(env_archcpu(env));
-                uint64_t tcr = env->cp15.tcr_el[r_el].raw_tcr;
-                int addrtop, tbi;
-
-                tbi = aa64_va_parameter_tbi(tcr, mmu_idx);
-                if (access_type == MMU_INST_FETCH) {
-                    tbi &= ~aa64_va_parameter_tbid(tcr, mmu_idx);
-                }
-                tbi = (tbi >> extract64(address, 55, 1)) & 1;
-                addrtop = (tbi ? 55 : 63);
-
-                if (extract64(address, pamax, addrtop - pamax + 1) != 0) {
-                    fi->type = ARMFault_AddressSize;
-                    fi->level = 0;
-                    fi->stage2 = false;
-                    return 1;
-                }
-
-                /*
-                 * When TBI is disabled, we've just validated that all of the
-                 * bits above PAMax are zero, so logically we only need to
-                 * clear the top byte for TBI.  But it's clearer to follow
-                 * the pseudocode set of addrdesc.paddress.
-                 */
-                address = extract64(address, 0, 52);
-            }
-        }
-        *phys_ptr = address;
-        *prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
-        *page_size = TARGET_PAGE_SIZE;
-
-        /* Fill in cacheattr a-la AArch64.TranslateAddressS1Off. */
-        hcr = arm_hcr_el2_eff(env);
-        cacheattrs->shareability = 0;
-        cacheattrs->is_s2_format = false;
-        if (hcr & HCR_DC) {
-            if (hcr & HCR_DCT) {
-                memattr = 0xf0;  /* Tagged, Normal, WB, RWA */
-            } else {
-                memattr = 0xff;  /* Normal, WB, RWA */
-            }
-        } else if (access_type == MMU_INST_FETCH) {
-            if (regime_sctlr(env, mmu_idx) & SCTLR_I) {
-                memattr = 0xee;  /* Normal, WT, RA, NT */
-            } else {
-                memattr = 0x44;  /* Normal, NC, No */
-            }
-            cacheattrs->shareability = 2; /* outer sharable */
-        } else {
-            memattr = 0x00;      /* Device, nGnRnE */
-        }
-        cacheattrs->attrs = memattr;
-        return 0;
-    }
-
-    if (regime_using_lpae_format(env, mmu_idx)) {
-        return get_phys_addr_lpae(env, address, access_type, mmu_idx, false,
-                                  phys_ptr, attrs, prot, page_size,
-                                  fi, cacheattrs);
-    } else if (regime_sctlr(env, mmu_idx) & SCTLR_XP) {
-        return get_phys_addr_v6(env, address, access_type, mmu_idx,
-                                phys_ptr, attrs, prot, page_size, fi);
-    } else {
-        return get_phys_addr_v5(env, address, access_type, mmu_idx,
-                                    phys_ptr, prot, page_size, fi);
-    }
-}
-
 hwaddr arm_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr,
                                          MemTxAttrs *attrs)
 {
@@ -13121,7 +12852,6 @@ hwaddr arm_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr,
     }
     return phys_addr;
 }
-
 #endif
 
 /* Note that signed overflow is undefined in C.  The following routines are
diff --git a/target/arm/internals.h b/target/arm/internals.h
index 049edce946..1d83146d56 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -613,8 +613,13 @@ ARMMMUIdx arm_v7m_mmu_idx_for_secstate_and_priv(CPUARMState *env,
 /* Return the MMU index for a v7M CPU in the specified security state */
 ARMMMUIdx arm_v7m_mmu_idx_for_secstate(CPUARMState *env, bool secstate);
 
-/* Return true if the stage 1 translation regime is using LPAE format page
- * tables */
+/* Return true if the translation regime is using LPAE format page tables */
+bool regime_using_lpae_format(CPUARMState *env, ARMMMUIdx mmu_idx);
+
+/*
+ * Return true if the stage 1 translation regime is using LPAE
+ * format page tables
+ */
 bool arm_s1_regime_using_lpae_format(CPUARMState *env, ARMMMUIdx mmu_idx);
 
 /* Raise a data fault alignment exception for the specified virtual address */
@@ -777,6 +782,12 @@ static inline uint32_t regime_el(CPUARMState *env, ARMMMUIdx mmu_idx)
     }
 }
 
+/* Return the SCTLR value which controls this address translation regime */
+static inline uint64_t regime_sctlr(CPUARMState *env, ARMMMUIdx mmu_idx)
+{
+    return env->cp15.sctlr_el[regime_el(env, mmu_idx)];
+}
+
 /* Return the TCR controlling this translation regime */
 static inline TCR *regime_tcr(CPUARMState *env, ARMMMUIdx mmu_idx)
 {
@@ -1095,6 +1106,9 @@ typedef struct ARMVAParameters {
 ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va,
                                    ARMMMUIdx mmu_idx, bool data);
 
+int aa64_va_parameter_tbi(uint64_t tcr, ARMMMUIdx mmu_idx);
+int aa64_va_parameter_tbid(uint64_t tcr, ARMMMUIdx mmu_idx);
+
 static inline int exception_target_el(CPUARMState *env)
 {
     int target_el = MAX(1, arm_current_el(env));
diff --git a/target/arm/meson.build b/target/arm/meson.build
index 50f152214a..ac571fc45d 100644
--- a/target/arm/meson.build
+++ b/target/arm/meson.build
@@ -58,6 +58,7 @@ arm_softmmu_ss.add(files(
   'machine.c',
   'monitor.c',
   'psci.c',
+  'ptw.c',
 ))
 
 subdir('hvf')
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
new file mode 100644
index 0000000000..318000f6d9
--- /dev/null
+++ b/target/arm/ptw.c
@@ -0,0 +1,267 @@
+/*
+ * ARM page table walking.
+ *
+ * This code is licensed under the GNU GPL v2 or later.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "cpu.h"
+#include "internals.h"
+#include "ptw.h"
+
+
+/**
+ * get_phys_addr - get the physical address for this virtual address
+ *
+ * Find the physical address corresponding to the given virtual address,
+ * by doing a translation table walk on MMU based systems or using the
+ * MPU state on MPU based systems.
+ *
+ * Returns false if the translation was successful. Otherwise, phys_ptr, attrs,
+ * prot and page_size may not be filled in, and the populated fsr value provides
+ * information on why the translation aborted, in the format of a
+ * DFSR/IFSR fault register, with the following caveats:
+ *  * we honour the short vs long DFSR format differences.
+ *  * the WnR bit is never set (the caller must do this).
+ *  * for PSMAv5 based systems we don't bother to return a full FSR format
+ *    value.
+ *
+ * @env: CPUARMState
+ * @address: virtual address to get physical address for
+ * @access_type: 0 for read, 1 for write, 2 for execute
+ * @mmu_idx: MMU index indicating required translation regime
+ * @phys_ptr: set to the physical address corresponding to the virtual address
+ * @attrs: set to the memory transaction attributes to use
+ * @prot: set to the permissions for the page containing phys_ptr
+ * @page_size: set to the size of the page containing phys_ptr
+ * @fi: set to fault info if the translation fails
+ * @cacheattrs: (if non-NULL) set to the cacheability/shareability attributes
+ */
+bool get_phys_addr(CPUARMState *env, target_ulong address,
+                   MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                   hwaddr *phys_ptr, MemTxAttrs *attrs, int *prot,
+                   target_ulong *page_size,
+                   ARMMMUFaultInfo *fi, ARMCacheAttrs *cacheattrs)
+{
+    ARMMMUIdx s1_mmu_idx = stage_1_mmu_idx(mmu_idx);
+
+    if (mmu_idx != s1_mmu_idx) {
+        /*
+         * Call ourselves recursively to do the stage 1 and then stage 2
+         * translations if mmu_idx is a two-stage regime.
+         */
+        if (arm_feature(env, ARM_FEATURE_EL2)) {
+            hwaddr ipa;
+            int s2_prot;
+            int ret;
+            bool ipa_secure;
+            ARMCacheAttrs cacheattrs2 = {};
+            ARMMMUIdx s2_mmu_idx;
+            bool is_el0;
+
+            ret = get_phys_addr(env, address, access_type, s1_mmu_idx, &ipa,
+                                attrs, prot, page_size, fi, cacheattrs);
+
+            /* If S1 fails or S2 is disabled, return early.  */
+            if (ret || regime_translation_disabled(env, ARMMMUIdx_Stage2)) {
+                *phys_ptr = ipa;
+                return ret;
+            }
+
+            ipa_secure = attrs->secure;
+            if (arm_is_secure_below_el3(env)) {
+                if (ipa_secure) {
+                    attrs->secure = !(env->cp15.vstcr_el2.raw_tcr & VSTCR_SW);
+                } else {
+                    attrs->secure = !(env->cp15.vtcr_el2.raw_tcr & VTCR_NSW);
+                }
+            } else {
+                assert(!ipa_secure);
+            }
+
+            s2_mmu_idx = attrs->secure ? ARMMMUIdx_Stage2_S : ARMMMUIdx_Stage2;
+            is_el0 = mmu_idx == ARMMMUIdx_E10_0 || mmu_idx == ARMMMUIdx_SE10_0;
+
+            /* S1 is done. Now do S2 translation.  */
+            ret = get_phys_addr_lpae(env, ipa, access_type, s2_mmu_idx, is_el0,
+                                     phys_ptr, attrs, &s2_prot,
+                                     page_size, fi, &cacheattrs2);
+            fi->s2addr = ipa;
+            /* Combine the S1 and S2 perms.  */
+            *prot &= s2_prot;
+
+            /* If S2 fails, return early.  */
+            if (ret) {
+                return ret;
+            }
+
+            /* Combine the S1 and S2 cache attributes. */
+            if (arm_hcr_el2_eff(env) & HCR_DC) {
+                /*
+                 * HCR.DC forces the first stage attributes to
+                 *  Normal Non-Shareable,
+                 *  Inner Write-Back Read-Allocate Write-Allocate,
+                 *  Outer Write-Back Read-Allocate Write-Allocate.
+                 * Do not overwrite Tagged within attrs.
+                 */
+                if (cacheattrs->attrs != 0xf0) {
+                    cacheattrs->attrs = 0xff;
+                }
+                cacheattrs->shareability = 0;
+            }
+            *cacheattrs = combine_cacheattrs(env, *cacheattrs, cacheattrs2);
+
+            /* Check if IPA translates to secure or non-secure PA space. */
+            if (arm_is_secure_below_el3(env)) {
+                if (ipa_secure) {
+                    attrs->secure =
+                        !(env->cp15.vstcr_el2.raw_tcr & (VSTCR_SA | VSTCR_SW));
+                } else {
+                    attrs->secure =
+                        !((env->cp15.vtcr_el2.raw_tcr & (VTCR_NSA | VTCR_NSW))
+                        || (env->cp15.vstcr_el2.raw_tcr & (VSTCR_SA | VSTCR_SW)));
+                }
+            }
+            return 0;
+        } else {
+            /*
+             * For non-EL2 CPUs a stage1+stage2 translation is just stage 1.
+             */
+            mmu_idx = stage_1_mmu_idx(mmu_idx);
+        }
+    }
+
+    /*
+     * The page table entries may downgrade secure to non-secure, but
+     * cannot upgrade an non-secure translation regime's attributes
+     * to secure.
+     */
+    attrs->secure = regime_is_secure(env, mmu_idx);
+    attrs->user = regime_is_user(env, mmu_idx);
+
+    /*
+     * Fast Context Switch Extension. This doesn't exist at all in v8.
+     * In v7 and earlier it affects all stage 1 translations.
+     */
+    if (address < 0x02000000 && mmu_idx != ARMMMUIdx_Stage2
+        && !arm_feature(env, ARM_FEATURE_V8)) {
+        if (regime_el(env, mmu_idx) == 3) {
+            address += env->cp15.fcseidr_s;
+        } else {
+            address += env->cp15.fcseidr_ns;
+        }
+    }
+
+    if (arm_feature(env, ARM_FEATURE_PMSA)) {
+        bool ret;
+        *page_size = TARGET_PAGE_SIZE;
+
+        if (arm_feature(env, ARM_FEATURE_V8)) {
+            /* PMSAv8 */
+            ret = get_phys_addr_pmsav8(env, address, access_type, mmu_idx,
+                                       phys_ptr, attrs, prot, page_size, fi);
+        } else if (arm_feature(env, ARM_FEATURE_V7)) {
+            /* PMSAv7 */
+            ret = get_phys_addr_pmsav7(env, address, access_type, mmu_idx,
+                                       phys_ptr, prot, page_size, fi);
+        } else {
+            /* Pre-v7 MPU */
+            ret = get_phys_addr_pmsav5(env, address, access_type, mmu_idx,
+                                       phys_ptr, prot, fi);
+        }
+        qemu_log_mask(CPU_LOG_MMU, "PMSA MPU lookup for %s at 0x%08" PRIx32
+                      " mmu_idx %u -> %s (prot %c%c%c)\n",
+                      access_type == MMU_DATA_LOAD ? "reading" :
+                      (access_type == MMU_DATA_STORE ? "writing" : "execute"),
+                      (uint32_t)address, mmu_idx,
+                      ret ? "Miss" : "Hit",
+                      *prot & PAGE_READ ? 'r' : '-',
+                      *prot & PAGE_WRITE ? 'w' : '-',
+                      *prot & PAGE_EXEC ? 'x' : '-');
+
+        return ret;
+    }
+
+    /* Definitely a real MMU, not an MPU */
+
+    if (regime_translation_disabled(env, mmu_idx)) {
+        uint64_t hcr;
+        uint8_t memattr;
+
+        /*
+         * MMU disabled.  S1 addresses within aa64 translation regimes are
+         * still checked for bounds -- see AArch64.TranslateAddressS1Off.
+         */
+        if (mmu_idx != ARMMMUIdx_Stage2 && mmu_idx != ARMMMUIdx_Stage2_S) {
+            int r_el = regime_el(env, mmu_idx);
+            if (arm_el_is_aa64(env, r_el)) {
+                int pamax = arm_pamax(env_archcpu(env));
+                uint64_t tcr = env->cp15.tcr_el[r_el].raw_tcr;
+                int addrtop, tbi;
+
+                tbi = aa64_va_parameter_tbi(tcr, mmu_idx);
+                if (access_type == MMU_INST_FETCH) {
+                    tbi &= ~aa64_va_parameter_tbid(tcr, mmu_idx);
+                }
+                tbi = (tbi >> extract64(address, 55, 1)) & 1;
+                addrtop = (tbi ? 55 : 63);
+
+                if (extract64(address, pamax, addrtop - pamax + 1) != 0) {
+                    fi->type = ARMFault_AddressSize;
+                    fi->level = 0;
+                    fi->stage2 = false;
+                    return 1;
+                }
+
+                /*
+                 * When TBI is disabled, we've just validated that all of the
+                 * bits above PAMax are zero, so logically we only need to
+                 * clear the top byte for TBI.  But it's clearer to follow
+                 * the pseudocode set of addrdesc.paddress.
+                 */
+                address = extract64(address, 0, 52);
+            }
+        }
+        *phys_ptr = address;
+        *prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
+        *page_size = TARGET_PAGE_SIZE;
+
+        /* Fill in cacheattr a-la AArch64.TranslateAddressS1Off. */
+        hcr = arm_hcr_el2_eff(env);
+        cacheattrs->shareability = 0;
+        cacheattrs->is_s2_format = false;
+        if (hcr & HCR_DC) {
+            if (hcr & HCR_DCT) {
+                memattr = 0xf0;  /* Tagged, Normal, WB, RWA */
+            } else {
+                memattr = 0xff;  /* Normal, WB, RWA */
+            }
+        } else if (access_type == MMU_INST_FETCH) {
+            if (regime_sctlr(env, mmu_idx) & SCTLR_I) {
+                memattr = 0xee;  /* Normal, WT, RA, NT */
+            } else {
+                memattr = 0x44;  /* Normal, NC, No */
+            }
+            cacheattrs->shareability = 2; /* outer sharable */
+        } else {
+            memattr = 0x00;      /* Device, nGnRnE */
+        }
+        cacheattrs->attrs = memattr;
+        return 0;
+    }
+
+    if (regime_using_lpae_format(env, mmu_idx)) {
+        return get_phys_addr_lpae(env, address, access_type, mmu_idx, false,
+                                  phys_ptr, attrs, prot, page_size,
+                                  fi, cacheattrs);
+    } else if (regime_sctlr(env, mmu_idx) & SCTLR_XP) {
+        return get_phys_addr_v6(env, address, access_type, mmu_idx,
+                                phys_ptr, attrs, prot, page_size, fi);
+    } else {
+        return get_phys_addr_v5(env, address, access_type, mmu_idx,
+                                    phys_ptr, prot, page_size, fi);
+    }
+}
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
new file mode 100644
index 0000000000..e2023ae750
--- /dev/null
+++ b/target/arm/ptw.h
@@ -0,0 +1,51 @@
+/*
+ * ARM page table walking.
+ *
+ * This code is licensed under the GNU GPL v2 or later.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef TARGET_ARM_PTW_H
+#define TARGET_ARM_PTW_H
+
+#ifndef CONFIG_USER_ONLY
+
+bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx);
+bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx);
+ARMCacheAttrs combine_cacheattrs(CPUARMState *env,
+                                 ARMCacheAttrs s1, ARMCacheAttrs s2);
+
+bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
+                      MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                      hwaddr *phys_ptr, int *prot,
+                      target_ulong *page_size,
+                      ARMMMUFaultInfo *fi);
+bool get_phys_addr_pmsav5(CPUARMState *env, uint32_t address,
+                          MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                          hwaddr *phys_ptr, int *prot,
+                          ARMMMUFaultInfo *fi);
+bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
+                      MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                      hwaddr *phys_ptr, MemTxAttrs *attrs, int *prot,
+                      target_ulong *page_size, ARMMMUFaultInfo *fi);
+bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
+                          MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                          hwaddr *phys_ptr, int *prot,
+                          target_ulong *page_size,
+                          ARMMMUFaultInfo *fi);
+bool get_phys_addr_pmsav8(CPUARMState *env, uint32_t address,
+                          MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                          hwaddr *phys_ptr, MemTxAttrs *txattrs,
+                          int *prot, target_ulong *page_size,
+                          ARMMMUFaultInfo *fi);
+bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
+                        MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                        bool s1_is_el0,
+                        hwaddr *phys_ptr, MemTxAttrs *txattrs, int *prot,
+                        target_ulong *page_size_ptr,
+                        ARMMMUFaultInfo *fi, ARMCacheAttrs *cacheattrs)
+    __attribute__((nonnull));
+
+#endif /* !CONFIG_USER_ONLY */
+#endif /* TARGET_ARM_PTW_H */

From f2d2f5ceb87e828546f7b544f1289193f3b920ce Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:48 +0100
Subject: [PATCH 862/935] target/arm: Move get_phys_addr_v5 to ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-4-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 137 +++-----------------------------------------
 target/arm/ptw.c    | 123 +++++++++++++++++++++++++++++++++++++++
 target/arm/ptw.h    |  15 +++--
 3 files changed, 140 insertions(+), 135 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 3ffd122178..321716914b 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10578,8 +10578,7 @@ bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx)
  * @ap:          The 3-bit access permissions (AP[2:0])
  * @domain_prot: The 2-bit domain access permissions
  */
-static inline int ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx,
-                                int ap, int domain_prot)
+int ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap, int domain_prot)
 {
     bool is_user = regime_is_user(env, mmu_idx);
 
@@ -10782,8 +10781,8 @@ static int get_S1prot(CPUARMState *env, ARMMMUIdx mmu_idx, bool is_aa64,
     return prot_rw | PAGE_EXEC;
 }
 
-static bool get_level1_table_address(CPUARMState *env, ARMMMUIdx mmu_idx,
-                                     uint32_t *table, uint32_t address)
+bool get_level1_table_address(CPUARMState *env, ARMMMUIdx mmu_idx,
+                              uint32_t *table, uint32_t address)
 {
     /* Note that we can only get here for an AArch32 PL0/PL1 lookup */
     TCR *tcr = regime_tcr(env, mmu_idx);
@@ -10882,8 +10881,8 @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
 }
 
 /* All loads done in the course of a page table walk go through here. */
-static uint32_t arm_ldl_ptw(CPUState *cs, hwaddr addr, bool is_secure,
-                            ARMMMUIdx mmu_idx, ARMMMUFaultInfo *fi)
+uint32_t arm_ldl_ptw(CPUState *cs, hwaddr addr, bool is_secure,
+                     ARMMMUIdx mmu_idx, ARMMMUFaultInfo *fi)
 {
     ARMCPU *cpu = ARM_CPU(cs);
     CPUARMState *env = &cpu->env;
@@ -10911,8 +10910,8 @@ static uint32_t arm_ldl_ptw(CPUState *cs, hwaddr addr, bool is_secure,
     return 0;
 }
 
-static uint64_t arm_ldq_ptw(CPUState *cs, hwaddr addr, bool is_secure,
-                            ARMMMUIdx mmu_idx, ARMMMUFaultInfo *fi)
+uint64_t arm_ldq_ptw(CPUState *cs, hwaddr addr, bool is_secure,
+                     ARMMMUIdx mmu_idx, ARMMMUFaultInfo *fi)
 {
     ARMCPU *cpu = ARM_CPU(cs);
     CPUARMState *env = &cpu->env;
@@ -10940,128 +10939,6 @@ static uint64_t arm_ldq_ptw(CPUState *cs, hwaddr addr, bool is_secure,
     return 0;
 }
 
-bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
-                      MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                      hwaddr *phys_ptr, int *prot,
-                      target_ulong *page_size,
-                      ARMMMUFaultInfo *fi)
-{
-    CPUState *cs = env_cpu(env);
-    int level = 1;
-    uint32_t table;
-    uint32_t desc;
-    int type;
-    int ap;
-    int domain = 0;
-    int domain_prot;
-    hwaddr phys_addr;
-    uint32_t dacr;
-
-    /* Pagetable walk.  */
-    /* Lookup l1 descriptor.  */
-    if (!get_level1_table_address(env, mmu_idx, &table, address)) {
-        /* Section translation fault if page walk is disabled by PD0 or PD1 */
-        fi->type = ARMFault_Translation;
-        goto do_fault;
-    }
-    desc = arm_ldl_ptw(cs, table, regime_is_secure(env, mmu_idx),
-                       mmu_idx, fi);
-    if (fi->type != ARMFault_None) {
-        goto do_fault;
-    }
-    type = (desc & 3);
-    domain = (desc >> 5) & 0x0f;
-    if (regime_el(env, mmu_idx) == 1) {
-        dacr = env->cp15.dacr_ns;
-    } else {
-        dacr = env->cp15.dacr_s;
-    }
-    domain_prot = (dacr >> (domain * 2)) & 3;
-    if (type == 0) {
-        /* Section translation fault.  */
-        fi->type = ARMFault_Translation;
-        goto do_fault;
-    }
-    if (type != 2) {
-        level = 2;
-    }
-    if (domain_prot == 0 || domain_prot == 2) {
-        fi->type = ARMFault_Domain;
-        goto do_fault;
-    }
-    if (type == 2) {
-        /* 1Mb section.  */
-        phys_addr = (desc & 0xfff00000) | (address & 0x000fffff);
-        ap = (desc >> 10) & 3;
-        *page_size = 1024 * 1024;
-    } else {
-        /* Lookup l2 entry.  */
-        if (type == 1) {
-            /* Coarse pagetable.  */
-            table = (desc & 0xfffffc00) | ((address >> 10) & 0x3fc);
-        } else {
-            /* Fine pagetable.  */
-            table = (desc & 0xfffff000) | ((address >> 8) & 0xffc);
-        }
-        desc = arm_ldl_ptw(cs, table, regime_is_secure(env, mmu_idx),
-                           mmu_idx, fi);
-        if (fi->type != ARMFault_None) {
-            goto do_fault;
-        }
-        switch (desc & 3) {
-        case 0: /* Page translation fault.  */
-            fi->type = ARMFault_Translation;
-            goto do_fault;
-        case 1: /* 64k page.  */
-            phys_addr = (desc & 0xffff0000) | (address & 0xffff);
-            ap = (desc >> (4 + ((address >> 13) & 6))) & 3;
-            *page_size = 0x10000;
-            break;
-        case 2: /* 4k page.  */
-            phys_addr = (desc & 0xfffff000) | (address & 0xfff);
-            ap = (desc >> (4 + ((address >> 9) & 6))) & 3;
-            *page_size = 0x1000;
-            break;
-        case 3: /* 1k page, or ARMv6/XScale "extended small (4k) page" */
-            if (type == 1) {
-                /* ARMv6/XScale extended small page format */
-                if (arm_feature(env, ARM_FEATURE_XSCALE)
-                    || arm_feature(env, ARM_FEATURE_V6)) {
-                    phys_addr = (desc & 0xfffff000) | (address & 0xfff);
-                    *page_size = 0x1000;
-                } else {
-                    /* UNPREDICTABLE in ARMv5; we choose to take a
-                     * page translation fault.
-                     */
-                    fi->type = ARMFault_Translation;
-                    goto do_fault;
-                }
-            } else {
-                phys_addr = (desc & 0xfffffc00) | (address & 0x3ff);
-                *page_size = 0x400;
-            }
-            ap = (desc >> 4) & 3;
-            break;
-        default:
-            /* Never happens, but compiler isn't smart enough to tell.  */
-            g_assert_not_reached();
-        }
-    }
-    *prot = ap_to_rw_prot(env, mmu_idx, ap, domain_prot);
-    *prot |= *prot ? PAGE_EXEC : 0;
-    if (!(*prot & (1 << access_type))) {
-        /* Access permission fault.  */
-        fi->type = ARMFault_Permission;
-        goto do_fault;
-    }
-    *phys_ptr = phys_addr;
-    return false;
-do_fault:
-    fi->domain = domain;
-    fi->level = level;
-    return true;
-}
-
 bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
                       MMUAccessType access_type, ARMMMUIdx mmu_idx,
                       hwaddr *phys_ptr, MemTxAttrs *attrs, int *prot,
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 318000f6d9..09c4472628 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -13,6 +13,129 @@
 #include "ptw.h"
 
 
+static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
+                             MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                             hwaddr *phys_ptr, int *prot,
+                             target_ulong *page_size,
+                             ARMMMUFaultInfo *fi)
+{
+    CPUState *cs = env_cpu(env);
+    int level = 1;
+    uint32_t table;
+    uint32_t desc;
+    int type;
+    int ap;
+    int domain = 0;
+    int domain_prot;
+    hwaddr phys_addr;
+    uint32_t dacr;
+
+    /* Pagetable walk.  */
+    /* Lookup l1 descriptor.  */
+    if (!get_level1_table_address(env, mmu_idx, &table, address)) {
+        /* Section translation fault if page walk is disabled by PD0 or PD1 */
+        fi->type = ARMFault_Translation;
+        goto do_fault;
+    }
+    desc = arm_ldl_ptw(cs, table, regime_is_secure(env, mmu_idx),
+                       mmu_idx, fi);
+    if (fi->type != ARMFault_None) {
+        goto do_fault;
+    }
+    type = (desc & 3);
+    domain = (desc >> 5) & 0x0f;
+    if (regime_el(env, mmu_idx) == 1) {
+        dacr = env->cp15.dacr_ns;
+    } else {
+        dacr = env->cp15.dacr_s;
+    }
+    domain_prot = (dacr >> (domain * 2)) & 3;
+    if (type == 0) {
+        /* Section translation fault.  */
+        fi->type = ARMFault_Translation;
+        goto do_fault;
+    }
+    if (type != 2) {
+        level = 2;
+    }
+    if (domain_prot == 0 || domain_prot == 2) {
+        fi->type = ARMFault_Domain;
+        goto do_fault;
+    }
+    if (type == 2) {
+        /* 1Mb section.  */
+        phys_addr = (desc & 0xfff00000) | (address & 0x000fffff);
+        ap = (desc >> 10) & 3;
+        *page_size = 1024 * 1024;
+    } else {
+        /* Lookup l2 entry.  */
+        if (type == 1) {
+            /* Coarse pagetable.  */
+            table = (desc & 0xfffffc00) | ((address >> 10) & 0x3fc);
+        } else {
+            /* Fine pagetable.  */
+            table = (desc & 0xfffff000) | ((address >> 8) & 0xffc);
+        }
+        desc = arm_ldl_ptw(cs, table, regime_is_secure(env, mmu_idx),
+                           mmu_idx, fi);
+        if (fi->type != ARMFault_None) {
+            goto do_fault;
+        }
+        switch (desc & 3) {
+        case 0: /* Page translation fault.  */
+            fi->type = ARMFault_Translation;
+            goto do_fault;
+        case 1: /* 64k page.  */
+            phys_addr = (desc & 0xffff0000) | (address & 0xffff);
+            ap = (desc >> (4 + ((address >> 13) & 6))) & 3;
+            *page_size = 0x10000;
+            break;
+        case 2: /* 4k page.  */
+            phys_addr = (desc & 0xfffff000) | (address & 0xfff);
+            ap = (desc >> (4 + ((address >> 9) & 6))) & 3;
+            *page_size = 0x1000;
+            break;
+        case 3: /* 1k page, or ARMv6/XScale "extended small (4k) page" */
+            if (type == 1) {
+                /* ARMv6/XScale extended small page format */
+                if (arm_feature(env, ARM_FEATURE_XSCALE)
+                    || arm_feature(env, ARM_FEATURE_V6)) {
+                    phys_addr = (desc & 0xfffff000) | (address & 0xfff);
+                    *page_size = 0x1000;
+                } else {
+                    /*
+                     * UNPREDICTABLE in ARMv5; we choose to take a
+                     * page translation fault.
+                     */
+                    fi->type = ARMFault_Translation;
+                    goto do_fault;
+                }
+            } else {
+                phys_addr = (desc & 0xfffffc00) | (address & 0x3ff);
+                *page_size = 0x400;
+            }
+            ap = (desc >> 4) & 3;
+            break;
+        default:
+            /* Never happens, but compiler isn't smart enough to tell.  */
+            g_assert_not_reached();
+        }
+    }
+    *prot = ap_to_rw_prot(env, mmu_idx, ap, domain_prot);
+    *prot |= *prot ? PAGE_EXEC : 0;
+    if (!(*prot & (1 << access_type))) {
+        /* Access permission fault.  */
+        fi->type = ARMFault_Permission;
+        goto do_fault;
+    }
+    *phys_ptr = phys_addr;
+    return false;
+do_fault:
+    fi->domain = domain;
+    fi->level = level;
+    return true;
+}
+
 /**
  * get_phys_addr - get the physical address for this virtual address
  *
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index e2023ae750..2dbd97b8cb 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -11,16 +11,21 @@
 
 #ifndef CONFIG_USER_ONLY
 
+uint32_t arm_ldl_ptw(CPUState *cs, hwaddr addr, bool is_secure,
+                     ARMMMUIdx mmu_idx, ARMMMUFaultInfo *fi);
+uint64_t arm_ldq_ptw(CPUState *cs, hwaddr addr, bool is_secure,
+                     ARMMMUIdx mmu_idx, ARMMMUFaultInfo *fi);
+
 bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx);
 bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx);
 ARMCacheAttrs combine_cacheattrs(CPUARMState *env,
                                  ARMCacheAttrs s1, ARMCacheAttrs s2);
 
-bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
-                      MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                      hwaddr *phys_ptr, int *prot,
-                      target_ulong *page_size,
-                      ARMMMUFaultInfo *fi);
+bool get_level1_table_address(CPUARMState *env, ARMMMUIdx mmu_idx,
+                              uint32_t *table, uint32_t address);
+int ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx,
+                  int ap, int domain_prot);
+
 bool get_phys_addr_pmsav5(CPUARMState *env, uint32_t address,
                           MMUAccessType access_type, ARMMMUIdx mmu_idx,
                           hwaddr *phys_ptr, int *prot,

From 53c038efb7a634dd5b6ede8eb034791c0dc83f43 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:48 +0100
Subject: [PATCH 863/935] target/arm: Move get_phys_addr_v6 to ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-5-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 161 +-------------------------------------------
 target/arm/ptw.c    | 153 +++++++++++++++++++++++++++++++++++++++++
 target/arm/ptw.h    |  11 +--
 3 files changed, 161 insertions(+), 164 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 321716914b..4a58822025 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10631,7 +10631,7 @@ int ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap, int domain_prot)
  * @ap:      The 2-bit simple AP (AP[2:1])
  * @is_user: TRUE if accessing from PL0
  */
-static inline int simple_ap_to_rw_prot_is_user(int ap, bool is_user)
+int simple_ap_to_rw_prot_is_user(int ap, bool is_user)
 {
     switch (ap) {
     case 0:
@@ -10647,12 +10647,6 @@ static inline int simple_ap_to_rw_prot_is_user(int ap, bool is_user)
     }
 }
 
-static inline int
-simple_ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap)
-{
-    return simple_ap_to_rw_prot_is_user(ap, regime_is_user(env, mmu_idx));
-}
-
 /* Translate S2 section/page access permissions to protection flags
  *
  * @env:     CPUARMState
@@ -10939,159 +10933,6 @@ uint64_t arm_ldq_ptw(CPUState *cs, hwaddr addr, bool is_secure,
     return 0;
 }
 
-bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
-                      MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                      hwaddr *phys_ptr, MemTxAttrs *attrs, int *prot,
-                      target_ulong *page_size, ARMMMUFaultInfo *fi)
-{
-    CPUState *cs = env_cpu(env);
-    ARMCPU *cpu = env_archcpu(env);
-    int level = 1;
-    uint32_t table;
-    uint32_t desc;
-    uint32_t xn;
-    uint32_t pxn = 0;
-    int type;
-    int ap;
-    int domain = 0;
-    int domain_prot;
-    hwaddr phys_addr;
-    uint32_t dacr;
-    bool ns;
-
-    /* Pagetable walk.  */
-    /* Lookup l1 descriptor.  */
-    if (!get_level1_table_address(env, mmu_idx, &table, address)) {
-        /* Section translation fault if page walk is disabled by PD0 or PD1 */
-        fi->type = ARMFault_Translation;
-        goto do_fault;
-    }
-    desc = arm_ldl_ptw(cs, table, regime_is_secure(env, mmu_idx),
-                       mmu_idx, fi);
-    if (fi->type != ARMFault_None) {
-        goto do_fault;
-    }
-    type = (desc & 3);
-    if (type == 0 || (type == 3 && !cpu_isar_feature(aa32_pxn, cpu))) {
-        /* Section translation fault, or attempt to use the encoding
-         * which is Reserved on implementations without PXN.
-         */
-        fi->type = ARMFault_Translation;
-        goto do_fault;
-    }
-    if ((type == 1) || !(desc & (1 << 18))) {
-        /* Page or Section.  */
-        domain = (desc >> 5) & 0x0f;
-    }
-    if (regime_el(env, mmu_idx) == 1) {
-        dacr = env->cp15.dacr_ns;
-    } else {
-        dacr = env->cp15.dacr_s;
-    }
-    if (type == 1) {
-        level = 2;
-    }
-    domain_prot = (dacr >> (domain * 2)) & 3;
-    if (domain_prot == 0 || domain_prot == 2) {
-        /* Section or Page domain fault */
-        fi->type = ARMFault_Domain;
-        goto do_fault;
-    }
-    if (type != 1) {
-        if (desc & (1 << 18)) {
-            /* Supersection.  */
-            phys_addr = (desc & 0xff000000) | (address & 0x00ffffff);
-            phys_addr |= (uint64_t)extract32(desc, 20, 4) << 32;
-            phys_addr |= (uint64_t)extract32(desc, 5, 4) << 36;
-            *page_size = 0x1000000;
-        } else {
-            /* Section.  */
-            phys_addr = (desc & 0xfff00000) | (address & 0x000fffff);
-            *page_size = 0x100000;
-        }
-        ap = ((desc >> 10) & 3) | ((desc >> 13) & 4);
-        xn = desc & (1 << 4);
-        pxn = desc & 1;
-        ns = extract32(desc, 19, 1);
-    } else {
-        if (cpu_isar_feature(aa32_pxn, cpu)) {
-            pxn = (desc >> 2) & 1;
-        }
-        ns = extract32(desc, 3, 1);
-        /* Lookup l2 entry.  */
-        table = (desc & 0xfffffc00) | ((address >> 10) & 0x3fc);
-        desc = arm_ldl_ptw(cs, table, regime_is_secure(env, mmu_idx),
-                           mmu_idx, fi);
-        if (fi->type != ARMFault_None) {
-            goto do_fault;
-        }
-        ap = ((desc >> 4) & 3) | ((desc >> 7) & 4);
-        switch (desc & 3) {
-        case 0: /* Page translation fault.  */
-            fi->type = ARMFault_Translation;
-            goto do_fault;
-        case 1: /* 64k page.  */
-            phys_addr = (desc & 0xffff0000) | (address & 0xffff);
-            xn = desc & (1 << 15);
-            *page_size = 0x10000;
-            break;
-        case 2: case 3: /* 4k page.  */
-            phys_addr = (desc & 0xfffff000) | (address & 0xfff);
-            xn = desc & 1;
-            *page_size = 0x1000;
-            break;
-        default:
-            /* Never happens, but compiler isn't smart enough to tell.  */
-            g_assert_not_reached();
-        }
-    }
-    if (domain_prot == 3) {
-        *prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
-    } else {
-        if (pxn && !regime_is_user(env, mmu_idx)) {
-            xn = 1;
-        }
-        if (xn && access_type == MMU_INST_FETCH) {
-            fi->type = ARMFault_Permission;
-            goto do_fault;
-        }
-
-        if (arm_feature(env, ARM_FEATURE_V6K) &&
-                (regime_sctlr(env, mmu_idx) & SCTLR_AFE)) {
-            /* The simplified model uses AP[0] as an access control bit.  */
-            if ((ap & 1) == 0) {
-                /* Access flag fault.  */
-                fi->type = ARMFault_AccessFlag;
-                goto do_fault;
-            }
-            *prot = simple_ap_to_rw_prot(env, mmu_idx, ap >> 1);
-        } else {
-            *prot = ap_to_rw_prot(env, mmu_idx, ap, domain_prot);
-        }
-        if (*prot && !xn) {
-            *prot |= PAGE_EXEC;
-        }
-        if (!(*prot & (1 << access_type))) {
-            /* Access permission fault.  */
-            fi->type = ARMFault_Permission;
-            goto do_fault;
-        }
-    }
-    if (ns) {
-        /* The NS bit will (as required by the architecture) have no effect if
-         * the CPU doesn't support TZ or this is a non-secure translation
-         * regime, because the attribute will already be non-secure.
-         */
-        attrs->secure = false;
-    }
-    *phys_ptr = phys_addr;
-    return false;
-do_fault:
-    fi->domain = domain;
-    fi->level = level;
-    return true;
-}
-
 /*
  * check_s2_mmu_setup
  * @cpu:        ARMCPU
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 09c4472628..6a1f4b549d 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -136,6 +136,159 @@ do_fault:
     return true;
 }
 
+static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
+                             MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                             hwaddr *phys_ptr, MemTxAttrs *attrs, int *prot,
+                             target_ulong *page_size, ARMMMUFaultInfo *fi)
+{
+    CPUState *cs = env_cpu(env);
+    ARMCPU *cpu = env_archcpu(env);
+    int level = 1;
+    uint32_t table;
+    uint32_t desc;
+    uint32_t xn;
+    uint32_t pxn = 0;
+    int type;
+    int ap;
+    int domain = 0;
+    int domain_prot;
+    hwaddr phys_addr;
+    uint32_t dacr;
+    bool ns;
+
+    /* Pagetable walk.  */
+    /* Lookup l1 descriptor.  */
+    if (!get_level1_table_address(env, mmu_idx, &table, address)) {
+        /* Section translation fault if page walk is disabled by PD0 or PD1 */
+        fi->type = ARMFault_Translation;
+        goto do_fault;
+    }
+    desc = arm_ldl_ptw(cs, table, regime_is_secure(env, mmu_idx),
+                       mmu_idx, fi);
+    if (fi->type != ARMFault_None) {
+        goto do_fault;
+    }
+    type = (desc & 3);
+    if (type == 0 || (type == 3 && !cpu_isar_feature(aa32_pxn, cpu))) {
+        /* Section translation fault, or attempt to use the encoding
+         * which is Reserved on implementations without PXN.
+         */
+        fi->type = ARMFault_Translation;
+        goto do_fault;
+    }
+    if ((type == 1) || !(desc & (1 << 18))) {
+        /* Page or Section.  */
+        domain = (desc >> 5) & 0x0f;
+    }
+    if (regime_el(env, mmu_idx) == 1) {
+        dacr = env->cp15.dacr_ns;
+    } else {
+        dacr = env->cp15.dacr_s;
+    }
+    if (type == 1) {
+        level = 2;
+    }
+    domain_prot = (dacr >> (domain * 2)) & 3;
+    if (domain_prot == 0 || domain_prot == 2) {
+        /* Section or Page domain fault */
+        fi->type = ARMFault_Domain;
+        goto do_fault;
+    }
+    if (type != 1) {
+        if (desc & (1 << 18)) {
+            /* Supersection.  */
+            phys_addr = (desc & 0xff000000) | (address & 0x00ffffff);
+            phys_addr |= (uint64_t)extract32(desc, 20, 4) << 32;
+            phys_addr |= (uint64_t)extract32(desc, 5, 4) << 36;
+            *page_size = 0x1000000;
+        } else {
+            /* Section.  */
+            phys_addr = (desc & 0xfff00000) | (address & 0x000fffff);
+            *page_size = 0x100000;
+        }
+        ap = ((desc >> 10) & 3) | ((desc >> 13) & 4);
+        xn = desc & (1 << 4);
+        pxn = desc & 1;
+        ns = extract32(desc, 19, 1);
+    } else {
+        if (cpu_isar_feature(aa32_pxn, cpu)) {
+            pxn = (desc >> 2) & 1;
+        }
+        ns = extract32(desc, 3, 1);
+        /* Lookup l2 entry.  */
+        table = (desc & 0xfffffc00) | ((address >> 10) & 0x3fc);
+        desc = arm_ldl_ptw(cs, table, regime_is_secure(env, mmu_idx),
+                           mmu_idx, fi);
+        if (fi->type != ARMFault_None) {
+            goto do_fault;
+        }
+        ap = ((desc >> 4) & 3) | ((desc >> 7) & 4);
+        switch (desc & 3) {
+        case 0: /* Page translation fault.  */
+            fi->type = ARMFault_Translation;
+            goto do_fault;
+        case 1: /* 64k page.  */
+            phys_addr = (desc & 0xffff0000) | (address & 0xffff);
+            xn = desc & (1 << 15);
+            *page_size = 0x10000;
+            break;
+        case 2: case 3: /* 4k page.  */
+            phys_addr = (desc & 0xfffff000) | (address & 0xfff);
+            xn = desc & 1;
+            *page_size = 0x1000;
+            break;
+        default:
+            /* Never happens, but compiler isn't smart enough to tell.  */
+            g_assert_not_reached();
+        }
+    }
+    if (domain_prot == 3) {
+        *prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
+    } else {
+        if (pxn && !regime_is_user(env, mmu_idx)) {
+            xn = 1;
+        }
+        if (xn && access_type == MMU_INST_FETCH) {
+            fi->type = ARMFault_Permission;
+            goto do_fault;
+        }
+
+        if (arm_feature(env, ARM_FEATURE_V6K) &&
+                (regime_sctlr(env, mmu_idx) & SCTLR_AFE)) {
+            /* The simplified model uses AP[0] as an access control bit.  */
+            if ((ap & 1) == 0) {
+                /* Access flag fault.  */
+                fi->type = ARMFault_AccessFlag;
+                goto do_fault;
+            }
+            *prot = simple_ap_to_rw_prot(env, mmu_idx, ap >> 1);
+        } else {
+            *prot = ap_to_rw_prot(env, mmu_idx, ap, domain_prot);
+        }
+        if (*prot && !xn) {
+            *prot |= PAGE_EXEC;
+        }
+        if (!(*prot & (1 << access_type))) {
+            /* Access permission fault.  */
+            fi->type = ARMFault_Permission;
+            goto do_fault;
+        }
+    }
+    if (ns) {
+        /* The NS bit will (as required by the architecture) have no effect if
+         * the CPU doesn't support TZ or this is a non-secure translation
+         * regime, because the attribute will already be non-secure.
+         */
+        attrs->secure = false;
+    }
+    *phys_ptr = phys_addr;
+    return false;
+do_fault:
+    fi->domain = domain;
+    fi->level = level;
+    return true;
+}
+
 /**
  * get_phys_addr - get the physical address for this virtual address
  *
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index 2dbd97b8cb..349b842d3c 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -25,15 +25,18 @@ bool get_level1_table_address(CPUARMState *env, ARMMMUIdx mmu_idx,
                               uint32_t *table, uint32_t address);
 int ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx,
                   int ap, int domain_prot);
+int simple_ap_to_rw_prot_is_user(int ap, bool is_user);
+
+static inline int
+simple_ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap)
+{
+    return simple_ap_to_rw_prot_is_user(ap, regime_is_user(env, mmu_idx));
+}
 
 bool get_phys_addr_pmsav5(CPUARMState *env, uint32_t address,
                           MMUAccessType access_type, ARMMMUIdx mmu_idx,
                           hwaddr *phys_ptr, int *prot,
                           ARMMMUFaultInfo *fi);
-bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
-                      MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                      hwaddr *phys_ptr, MemTxAttrs *attrs, int *prot,
-                      target_ulong *page_size, ARMMMUFaultInfo *fi);
 bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
                           MMUAccessType access_type, ARMMMUIdx mmu_idx,
                           hwaddr *phys_ptr, int *prot,

From 9a12fb366d442751603dc91ab5c5f9b7c828c783 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:49 +0100
Subject: [PATCH 864/935] target/arm: Move get_phys_addr_pmsav5 to ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-6-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 85 ---------------------------------------------
 target/arm/ptw.c    | 85 +++++++++++++++++++++++++++++++++++++++++++++
 target/arm/ptw.h    |  4 ---
 3 files changed, 85 insertions(+), 89 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 4a58822025..5d01019010 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -12274,91 +12274,6 @@ bool get_phys_addr_pmsav8(CPUARMState *env, uint32_t address,
     return ret;
 }
 
-bool get_phys_addr_pmsav5(CPUARMState *env, uint32_t address,
-                          MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                          hwaddr *phys_ptr, int *prot,
-                          ARMMMUFaultInfo *fi)
-{
-    int n;
-    uint32_t mask;
-    uint32_t base;
-    bool is_user = regime_is_user(env, mmu_idx);
-
-    if (regime_translation_disabled(env, mmu_idx)) {
-        /* MPU disabled.  */
-        *phys_ptr = address;
-        *prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
-        return false;
-    }
-
-    *phys_ptr = address;
-    for (n = 7; n >= 0; n--) {
-        base = env->cp15.c6_region[n];
-        if ((base & 1) == 0) {
-            continue;
-        }
-        mask = 1 << ((base >> 1) & 0x1f);
-        /* Keep this shift separate from the above to avoid an
-           (undefined) << 32.  */
-        mask = (mask << 1) - 1;
-        if (((base ^ address) & ~mask) == 0) {
-            break;
-        }
-    }
-    if (n < 0) {
-        fi->type = ARMFault_Background;
-        return true;
-    }
-
-    if (access_type == MMU_INST_FETCH) {
-        mask = env->cp15.pmsav5_insn_ap;
-    } else {
-        mask = env->cp15.pmsav5_data_ap;
-    }
-    mask = (mask >> (n * 4)) & 0xf;
-    switch (mask) {
-    case 0:
-        fi->type = ARMFault_Permission;
-        fi->level = 1;
-        return true;
-    case 1:
-        if (is_user) {
-            fi->type = ARMFault_Permission;
-            fi->level = 1;
-            return true;
-        }
-        *prot = PAGE_READ | PAGE_WRITE;
-        break;
-    case 2:
-        *prot = PAGE_READ;
-        if (!is_user) {
-            *prot |= PAGE_WRITE;
-        }
-        break;
-    case 3:
-        *prot = PAGE_READ | PAGE_WRITE;
-        break;
-    case 5:
-        if (is_user) {
-            fi->type = ARMFault_Permission;
-            fi->level = 1;
-            return true;
-        }
-        *prot = PAGE_READ;
-        break;
-    case 6:
-        *prot = PAGE_READ;
-        break;
-    default:
-        /* Bad permission.  */
-        fi->type = ARMFault_Permission;
-        fi->level = 1;
-        return true;
-    }
-    *prot |= PAGE_EXEC;
-    return false;
-}
-
 /* Combine either inner or outer cacheability attributes for normal
  * memory, according to table D4-42 and pseudocode procedure
  * CombineS1S2AttrHints() of ARM DDI 0487B.b (the ARMv8 ARM).
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 6a1f4b549d..5c32648a16 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -289,6 +289,91 @@ do_fault:
     return true;
 }
 
+static bool get_phys_addr_pmsav5(CPUARMState *env, uint32_t address,
+                                 MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                                 hwaddr *phys_ptr, int *prot,
+                                 ARMMMUFaultInfo *fi)
+{
+    int n;
+    uint32_t mask;
+    uint32_t base;
+    bool is_user = regime_is_user(env, mmu_idx);
+
+    if (regime_translation_disabled(env, mmu_idx)) {
+        /* MPU disabled.  */
+        *phys_ptr = address;
+        *prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
+        return false;
+    }
+
+    *phys_ptr = address;
+    for (n = 7; n >= 0; n--) {
+        base = env->cp15.c6_region[n];
+        if ((base & 1) == 0) {
+            continue;
+        }
+        mask = 1 << ((base >> 1) & 0x1f);
+        /* Keep this shift separate from the above to avoid an
+           (undefined) << 32.  */
+        mask = (mask << 1) - 1;
+        if (((base ^ address) & ~mask) == 0) {
+            break;
+        }
+    }
+    if (n < 0) {
+        fi->type = ARMFault_Background;
+        return true;
+    }
+
+    if (access_type == MMU_INST_FETCH) {
+        mask = env->cp15.pmsav5_insn_ap;
+    } else {
+        mask = env->cp15.pmsav5_data_ap;
+    }
+    mask = (mask >> (n * 4)) & 0xf;
+    switch (mask) {
+    case 0:
+        fi->type = ARMFault_Permission;
+        fi->level = 1;
+        return true;
+    case 1:
+        if (is_user) {
+            fi->type = ARMFault_Permission;
+            fi->level = 1;
+            return true;
+        }
+        *prot = PAGE_READ | PAGE_WRITE;
+        break;
+    case 2:
+        *prot = PAGE_READ;
+        if (!is_user) {
+            *prot |= PAGE_WRITE;
+        }
+        break;
+    case 3:
+        *prot = PAGE_READ | PAGE_WRITE;
+        break;
+    case 5:
+        if (is_user) {
+            fi->type = ARMFault_Permission;
+            fi->level = 1;
+            return true;
+        }
+        *prot = PAGE_READ;
+        break;
+    case 6:
+        *prot = PAGE_READ;
+        break;
+    default:
+        /* Bad permission.  */
+        fi->type = ARMFault_Permission;
+        fi->level = 1;
+        return true;
+    }
+    *prot |= PAGE_EXEC;
+    return false;
+}
+
 /**
  * get_phys_addr - get the physical address for this virtual address
  *
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index 349b842d3c..324a9dde14 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -33,10 +33,6 @@ simple_ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap)
     return simple_ap_to_rw_prot_is_user(ap, regime_is_user(env, mmu_idx));
 }
 
-bool get_phys_addr_pmsav5(CPUARMState *env, uint32_t address,
-                          MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                          hwaddr *phys_ptr, int *prot,
-                          ARMMMUFaultInfo *fi);
 bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
                           MMUAccessType access_type, ARMMMUIdx mmu_idx,
                           hwaddr *phys_ptr, int *prot,

From 7d2e08c96077bcf7ed31b6b7c7b7066801b4b89d Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:49 +0100
Subject: [PATCH 865/935] target/arm: Move get_phys_addr_pmsav7_default to
 ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-7-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 41 -----------------------------------------
 target/arm/ptw.c    | 41 +++++++++++++++++++++++++++++++++++++++++
 target/arm/ptw.h    |  3 +++
 3 files changed, 44 insertions(+), 41 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 5d01019010..d4f7c05625 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -11678,47 +11678,6 @@ do_fault:
     return true;
 }
 
-static inline void get_phys_addr_pmsav7_default(CPUARMState *env,
-                                                ARMMMUIdx mmu_idx,
-                                                int32_t address, int *prot)
-{
-    if (!arm_feature(env, ARM_FEATURE_M)) {
-        *prot = PAGE_READ | PAGE_WRITE;
-        switch (address) {
-        case 0xF0000000 ... 0xFFFFFFFF:
-            if (regime_sctlr(env, mmu_idx) & SCTLR_V) {
-                /* hivecs execing is ok */
-                *prot |= PAGE_EXEC;
-            }
-            break;
-        case 0x00000000 ... 0x7FFFFFFF:
-            *prot |= PAGE_EXEC;
-            break;
-        }
-    } else {
-        /* Default system address map for M profile cores.
-         * The architecture specifies which regions are execute-never;
-         * at the MPU level no other checks are defined.
-         */
-        switch (address) {
-        case 0x00000000 ... 0x1fffffff: /* ROM */
-        case 0x20000000 ... 0x3fffffff: /* SRAM */
-        case 0x60000000 ... 0x7fffffff: /* RAM */
-        case 0x80000000 ... 0x9fffffff: /* RAM */
-            *prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
-            break;
-        case 0x40000000 ... 0x5fffffff: /* Peripheral */
-        case 0xa0000000 ... 0xbfffffff: /* Device */
-        case 0xc0000000 ... 0xdfffffff: /* Device */
-        case 0xe0000000 ... 0xffffffff: /* System */
-            *prot = PAGE_READ | PAGE_WRITE;
-            break;
-        default:
-            g_assert_not_reached();
-        }
-    }
-}
-
 static bool pmsav7_use_background_region(ARMCPU *cpu,
                                          ARMMMUIdx mmu_idx, bool is_user)
 {
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 5c32648a16..74650c6c52 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -374,6 +374,47 @@ static bool get_phys_addr_pmsav5(CPUARMState *env, uint32_t address,
     return false;
 }
 
+void get_phys_addr_pmsav7_default(CPUARMState *env,
+                                  ARMMMUIdx mmu_idx,
+                                  int32_t address, int *prot)
+{
+    if (!arm_feature(env, ARM_FEATURE_M)) {
+        *prot = PAGE_READ | PAGE_WRITE;
+        switch (address) {
+        case 0xF0000000 ... 0xFFFFFFFF:
+            if (regime_sctlr(env, mmu_idx) & SCTLR_V) {
+                /* hivecs execing is ok */
+                *prot |= PAGE_EXEC;
+            }
+            break;
+        case 0x00000000 ... 0x7FFFFFFF:
+            *prot |= PAGE_EXEC;
+            break;
+        }
+    } else {
+        /* Default system address map for M profile cores.
+         * The architecture specifies which regions are execute-never;
+         * at the MPU level no other checks are defined.
+         */
+        switch (address) {
+        case 0x00000000 ... 0x1fffffff: /* ROM */
+        case 0x20000000 ... 0x3fffffff: /* SRAM */
+        case 0x60000000 ... 0x7fffffff: /* RAM */
+        case 0x80000000 ... 0x9fffffff: /* RAM */
+            *prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
+            break;
+        case 0x40000000 ... 0x5fffffff: /* Peripheral */
+        case 0xa0000000 ... 0xbfffffff: /* Device */
+        case 0xc0000000 ... 0xdfffffff: /* Device */
+        case 0xe0000000 ... 0xffffffff: /* System */
+            *prot = PAGE_READ | PAGE_WRITE;
+            break;
+        default:
+            g_assert_not_reached();
+        }
+    }
+}
+
 /**
  * get_phys_addr - get the physical address for this virtual address
  *
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index 324a9dde14..d6e3fee152 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -33,6 +33,9 @@ simple_ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap)
     return simple_ap_to_rw_prot_is_user(ap, regime_is_user(env, mmu_idx));
 }
 
+void get_phys_addr_pmsav7_default(CPUARMState *env,
+                                  ARMMMUIdx mmu_idx,
+                                  int32_t address, int *prot);
 bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
                           MMUAccessType access_type, ARMMMUIdx mmu_idx,
                           hwaddr *phys_ptr, int *prot,

From 1f2e87e5aba133ecf4c85000a61d5d760176dd6c Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:49 +0100
Subject: [PATCH 866/935] target/arm: Move get_phys_addr_pmsav7 to ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-8-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 194 +-------------------------------------------
 target/arm/ptw.c    | 190 +++++++++++++++++++++++++++++++++++++++++++
 target/arm/ptw.h    |  10 +--
 3 files changed, 198 insertions(+), 196 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index d4f7c05625..2ebaf69407 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -11678,8 +11678,7 @@ do_fault:
     return true;
 }
 
-static bool pmsav7_use_background_region(ARMCPU *cpu,
-                                         ARMMMUIdx mmu_idx, bool is_user)
+bool pmsav7_use_background_region(ARMCPU *cpu, ARMMMUIdx mmu_idx, bool is_user)
 {
     /* Return true if we should use the default memory map as a
      * "background" region if there are no hits against any MPU regions.
@@ -11698,14 +11697,14 @@ static bool pmsav7_use_background_region(ARMCPU *cpu,
     }
 }
 
-static inline bool m_is_ppb_region(CPUARMState *env, uint32_t address)
+bool m_is_ppb_region(CPUARMState *env, uint32_t address)
 {
     /* True if address is in the M profile PPB region 0xe0000000 - 0xe00fffff */
     return arm_feature(env, ARM_FEATURE_M) &&
         extract32(address, 20, 12) == 0xe00;
 }
 
-static inline bool m_is_system_region(CPUARMState *env, uint32_t address)
+bool m_is_system_region(CPUARMState *env, uint32_t address)
 {
     /* True if address is in the M profile system region
      * 0xe0000000 - 0xffffffff
@@ -11713,193 +11712,6 @@ static inline bool m_is_system_region(CPUARMState *env, uint32_t address)
     return arm_feature(env, ARM_FEATURE_M) && extract32(address, 29, 3) == 0x7;
 }
 
-bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
-                          MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                          hwaddr *phys_ptr, int *prot,
-                          target_ulong *page_size,
-                          ARMMMUFaultInfo *fi)
-{
-    ARMCPU *cpu = env_archcpu(env);
-    int n;
-    bool is_user = regime_is_user(env, mmu_idx);
-
-    *phys_ptr = address;
-    *page_size = TARGET_PAGE_SIZE;
-    *prot = 0;
-
-    if (regime_translation_disabled(env, mmu_idx) ||
-        m_is_ppb_region(env, address)) {
-        /* MPU disabled or M profile PPB access: use default memory map.
-         * The other case which uses the default memory map in the
-         * v7M ARM ARM pseudocode is exception vector reads from the vector
-         * table. In QEMU those accesses are done in arm_v7m_load_vector(),
-         * which always does a direct read using address_space_ldl(), rather
-         * than going via this function, so we don't need to check that here.
-         */
-        get_phys_addr_pmsav7_default(env, mmu_idx, address, prot);
-    } else { /* MPU enabled */
-        for (n = (int)cpu->pmsav7_dregion - 1; n >= 0; n--) {
-            /* region search */
-            uint32_t base = env->pmsav7.drbar[n];
-            uint32_t rsize = extract32(env->pmsav7.drsr[n], 1, 5);
-            uint32_t rmask;
-            bool srdis = false;
-
-            if (!(env->pmsav7.drsr[n] & 0x1)) {
-                continue;
-            }
-
-            if (!rsize) {
-                qemu_log_mask(LOG_GUEST_ERROR,
-                              "DRSR[%d]: Rsize field cannot be 0\n", n);
-                continue;
-            }
-            rsize++;
-            rmask = (1ull << rsize) - 1;
-
-            if (base & rmask) {
-                qemu_log_mask(LOG_GUEST_ERROR,
-                              "DRBAR[%d]: 0x%" PRIx32 " misaligned "
-                              "to DRSR region size, mask = 0x%" PRIx32 "\n",
-                              n, base, rmask);
-                continue;
-            }
-
-            if (address < base || address > base + rmask) {
-                /*
-                 * Address not in this region. We must check whether the
-                 * region covers addresses in the same page as our address.
-                 * In that case we must not report a size that covers the
-                 * whole page for a subsequent hit against a different MPU
-                 * region or the background region, because it would result in
-                 * incorrect TLB hits for subsequent accesses to addresses that
-                 * are in this MPU region.
-                 */
-                if (ranges_overlap(base, rmask,
-                                   address & TARGET_PAGE_MASK,
-                                   TARGET_PAGE_SIZE)) {
-                    *page_size = 1;
-                }
-                continue;
-            }
-
-            /* Region matched */
-
-            if (rsize >= 8) { /* no subregions for regions < 256 bytes */
-                int i, snd;
-                uint32_t srdis_mask;
-
-                rsize -= 3; /* sub region size (power of 2) */
-                snd = ((address - base) >> rsize) & 0x7;
-                srdis = extract32(env->pmsav7.drsr[n], snd + 8, 1);
-
-                srdis_mask = srdis ? 0x3 : 0x0;
-                for (i = 2; i <= 8 && rsize < TARGET_PAGE_BITS; i *= 2) {
-                    /* This will check in groups of 2, 4 and then 8, whether
-                     * the subregion bits are consistent. rsize is incremented
-                     * back up to give the region size, considering consistent
-                     * adjacent subregions as one region. Stop testing if rsize
-                     * is already big enough for an entire QEMU page.
-                     */
-                    int snd_rounded = snd & ~(i - 1);
-                    uint32_t srdis_multi = extract32(env->pmsav7.drsr[n],
-                                                     snd_rounded + 8, i);
-                    if (srdis_mask ^ srdis_multi) {
-                        break;
-                    }
-                    srdis_mask = (srdis_mask << i) | srdis_mask;
-                    rsize++;
-                }
-            }
-            if (srdis) {
-                continue;
-            }
-            if (rsize < TARGET_PAGE_BITS) {
-                *page_size = 1 << rsize;
-            }
-            break;
-        }
-
-        if (n == -1) { /* no hits */
-            if (!pmsav7_use_background_region(cpu, mmu_idx, is_user)) {
-                /* background fault */
-                fi->type = ARMFault_Background;
-                return true;
-            }
-            get_phys_addr_pmsav7_default(env, mmu_idx, address, prot);
-        } else { /* a MPU hit! */
-            uint32_t ap = extract32(env->pmsav7.dracr[n], 8, 3);
-            uint32_t xn = extract32(env->pmsav7.dracr[n], 12, 1);
-
-            if (m_is_system_region(env, address)) {
-                /* System space is always execute never */
-                xn = 1;
-            }
-
-            if (is_user) { /* User mode AP bit decoding */
-                switch (ap) {
-                case 0:
-                case 1:
-                case 5:
-                    break; /* no access */
-                case 3:
-                    *prot |= PAGE_WRITE;
-                    /* fall through */
-                case 2:
-                case 6:
-                    *prot |= PAGE_READ | PAGE_EXEC;
-                    break;
-                case 7:
-                    /* for v7M, same as 6; for R profile a reserved value */
-                    if (arm_feature(env, ARM_FEATURE_M)) {
-                        *prot |= PAGE_READ | PAGE_EXEC;
-                        break;
-                    }
-                    /* fall through */
-                default:
-                    qemu_log_mask(LOG_GUEST_ERROR,
-                                  "DRACR[%d]: Bad value for AP bits: 0x%"
-                                  PRIx32 "\n", n, ap);
-                }
-            } else { /* Priv. mode AP bits decoding */
-                switch (ap) {
-                case 0:
-                    break; /* no access */
-                case 1:
-                case 2:
-                case 3:
-                    *prot |= PAGE_WRITE;
-                    /* fall through */
-                case 5:
-                case 6:
-                    *prot |= PAGE_READ | PAGE_EXEC;
-                    break;
-                case 7:
-                    /* for v7M, same as 6; for R profile a reserved value */
-                    if (arm_feature(env, ARM_FEATURE_M)) {
-                        *prot |= PAGE_READ | PAGE_EXEC;
-                        break;
-                    }
-                    /* fall through */
-                default:
-                    qemu_log_mask(LOG_GUEST_ERROR,
-                                  "DRACR[%d]: Bad value for AP bits: 0x%"
-                                  PRIx32 "\n", n, ap);
-                }
-            }
-
-            /* execute never */
-            if (xn) {
-                *prot &= ~PAGE_EXEC;
-            }
-        }
-    }
-
-    fi->type = ARMFault_Permission;
-    fi->level = 1;
-    return !(*prot & (1 << access_type));
-}
-
 static bool v8m_is_sau_exempt(CPUARMState *env,
                               uint32_t address, MMUAccessType access_type)
 {
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 74650c6c52..27715dbfa8 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -8,6 +8,7 @@
 
 #include "qemu/osdep.h"
 #include "qemu/log.h"
+#include "qemu/range.h"
 #include "cpu.h"
 #include "internals.h"
 #include "ptw.h"
@@ -415,6 +416,195 @@ void get_phys_addr_pmsav7_default(CPUARMState *env,
     }
 }
 
+static bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
+                                 MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                                 hwaddr *phys_ptr, int *prot,
+                                 target_ulong *page_size,
+                                 ARMMMUFaultInfo *fi)
+{
+    ARMCPU *cpu = env_archcpu(env);
+    int n;
+    bool is_user = regime_is_user(env, mmu_idx);
+
+    *phys_ptr = address;
+    *page_size = TARGET_PAGE_SIZE;
+    *prot = 0;
+
+    if (regime_translation_disabled(env, mmu_idx) ||
+        m_is_ppb_region(env, address)) {
+        /*
+         * MPU disabled or M profile PPB access: use default memory map.
+         * The other case which uses the default memory map in the
+         * v7M ARM ARM pseudocode is exception vector reads from the vector
+         * table. In QEMU those accesses are done in arm_v7m_load_vector(),
+         * which always does a direct read using address_space_ldl(), rather
+         * than going via this function, so we don't need to check that here.
+         */
+        get_phys_addr_pmsav7_default(env, mmu_idx, address, prot);
+    } else { /* MPU enabled */
+        for (n = (int)cpu->pmsav7_dregion - 1; n >= 0; n--) {
+            /* region search */
+            uint32_t base = env->pmsav7.drbar[n];
+            uint32_t rsize = extract32(env->pmsav7.drsr[n], 1, 5);
+            uint32_t rmask;
+            bool srdis = false;
+
+            if (!(env->pmsav7.drsr[n] & 0x1)) {
+                continue;
+            }
+
+            if (!rsize) {
+                qemu_log_mask(LOG_GUEST_ERROR,
+                              "DRSR[%d]: Rsize field cannot be 0\n", n);
+                continue;
+            }
+            rsize++;
+            rmask = (1ull << rsize) - 1;
+
+            if (base & rmask) {
+                qemu_log_mask(LOG_GUEST_ERROR,
+                              "DRBAR[%d]: 0x%" PRIx32 " misaligned "
+                              "to DRSR region size, mask = 0x%" PRIx32 "\n",
+                              n, base, rmask);
+                continue;
+            }
+
+            if (address < base || address > base + rmask) {
+                /*
+                 * Address not in this region. We must check whether the
+                 * region covers addresses in the same page as our address.
+                 * In that case we must not report a size that covers the
+                 * whole page for a subsequent hit against a different MPU
+                 * region or the background region, because it would result in
+                 * incorrect TLB hits for subsequent accesses to addresses that
+                 * are in this MPU region.
+                 */
+                if (ranges_overlap(base, rmask,
+                                   address & TARGET_PAGE_MASK,
+                                   TARGET_PAGE_SIZE)) {
+                    *page_size = 1;
+                }
+                continue;
+            }
+
+            /* Region matched */
+
+            if (rsize >= 8) { /* no subregions for regions < 256 bytes */
+                int i, snd;
+                uint32_t srdis_mask;
+
+                rsize -= 3; /* sub region size (power of 2) */
+                snd = ((address - base) >> rsize) & 0x7;
+                srdis = extract32(env->pmsav7.drsr[n], snd + 8, 1);
+
+                srdis_mask = srdis ? 0x3 : 0x0;
+                for (i = 2; i <= 8 && rsize < TARGET_PAGE_BITS; i *= 2) {
+                    /*
+                     * This will check in groups of 2, 4 and then 8, whether
+                     * the subregion bits are consistent. rsize is incremented
+                     * back up to give the region size, considering consistent
+                     * adjacent subregions as one region. Stop testing if rsize
+                     * is already big enough for an entire QEMU page.
+                     */
+                    int snd_rounded = snd & ~(i - 1);
+                    uint32_t srdis_multi = extract32(env->pmsav7.drsr[n],
+                                                     snd_rounded + 8, i);
+                    if (srdis_mask ^ srdis_multi) {
+                        break;
+                    }
+                    srdis_mask = (srdis_mask << i) | srdis_mask;
+                    rsize++;
+                }
+            }
+            if (srdis) {
+                continue;
+            }
+            if (rsize < TARGET_PAGE_BITS) {
+                *page_size = 1 << rsize;
+            }
+            break;
+        }
+
+        if (n == -1) { /* no hits */
+            if (!pmsav7_use_background_region(cpu, mmu_idx, is_user)) {
+                /* background fault */
+                fi->type = ARMFault_Background;
+                return true;
+            }
+            get_phys_addr_pmsav7_default(env, mmu_idx, address, prot);
+        } else { /* a MPU hit! */
+            uint32_t ap = extract32(env->pmsav7.dracr[n], 8, 3);
+            uint32_t xn = extract32(env->pmsav7.dracr[n], 12, 1);
+
+            if (m_is_system_region(env, address)) {
+                /* System space is always execute never */
+                xn = 1;
+            }
+
+            if (is_user) { /* User mode AP bit decoding */
+                switch (ap) {
+                case 0:
+                case 1:
+                case 5:
+                    break; /* no access */
+                case 3:
+                    *prot |= PAGE_WRITE;
+                    /* fall through */
+                case 2:
+                case 6:
+                    *prot |= PAGE_READ | PAGE_EXEC;
+                    break;
+                case 7:
+                    /* for v7M, same as 6; for R profile a reserved value */
+                    if (arm_feature(env, ARM_FEATURE_M)) {
+                        *prot |= PAGE_READ | PAGE_EXEC;
+                        break;
+                    }
+                    /* fall through */
+                default:
+                    qemu_log_mask(LOG_GUEST_ERROR,
+                                  "DRACR[%d]: Bad value for AP bits: 0x%"
+                                  PRIx32 "\n", n, ap);
+                }
+            } else { /* Priv. mode AP bits decoding */
+                switch (ap) {
+                case 0:
+                    break; /* no access */
+                case 1:
+                case 2:
+                case 3:
+                    *prot |= PAGE_WRITE;
+                    /* fall through */
+                case 5:
+                case 6:
+                    *prot |= PAGE_READ | PAGE_EXEC;
+                    break;
+                case 7:
+                    /* for v7M, same as 6; for R profile a reserved value */
+                    if (arm_feature(env, ARM_FEATURE_M)) {
+                        *prot |= PAGE_READ | PAGE_EXEC;
+                        break;
+                    }
+                    /* fall through */
+                default:
+                    qemu_log_mask(LOG_GUEST_ERROR,
+                                  "DRACR[%d]: Bad value for AP bits: 0x%"
+                                  PRIx32 "\n", n, ap);
+                }
+            }
+
+            /* execute never */
+            if (xn) {
+                *prot &= ~PAGE_EXEC;
+            }
+        }
+    }
+
+    fi->type = ARMFault_Permission;
+    fi->level = 1;
+    return !(*prot & (1 << access_type));
+}
+
 /**
  * get_phys_addr - get the physical address for this virtual address
  *
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index d6e3fee152..d24b7c263a 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -33,14 +33,14 @@ simple_ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap)
     return simple_ap_to_rw_prot_is_user(ap, regime_is_user(env, mmu_idx));
 }
 
+bool m_is_ppb_region(CPUARMState *env, uint32_t address);
+bool m_is_system_region(CPUARMState *env, uint32_t address);
+
 void get_phys_addr_pmsav7_default(CPUARMState *env,
                                   ARMMMUIdx mmu_idx,
                                   int32_t address, int *prot);
-bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
-                          MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                          hwaddr *phys_ptr, int *prot,
-                          target_ulong *page_size,
-                          ARMMMUFaultInfo *fi);
+bool pmsav7_use_background_region(ARMCPU *cpu, ARMMMUIdx mmu_idx, bool is_user);
+
 bool get_phys_addr_pmsav8(CPUARMState *env, uint32_t address,
                           MMUAccessType access_type, ARMMMUIdx mmu_idx,
                           hwaddr *phys_ptr, MemTxAttrs *txattrs,

From 730d5c31d8872e94d08877ad62f44cc49bc01c90 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:49 +0100
Subject: [PATCH 867/935] target/arm: Move get_phys_addr_pmsav8 to ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-9-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 75 -------------------------------------------
 target/arm/ptw.c    | 77 +++++++++++++++++++++++++++++++++++++++++++++
 target/arm/ptw.h    |  5 ---
 3 files changed, 77 insertions(+), 80 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 2ebaf69407..44997fd179 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -11970,81 +11970,6 @@ bool pmsav8_mpu_lookup(CPUARMState *env, uint32_t address,
     return !(*prot & (1 << access_type));
 }
 
-
-bool get_phys_addr_pmsav8(CPUARMState *env, uint32_t address,
-                          MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                          hwaddr *phys_ptr, MemTxAttrs *txattrs,
-                          int *prot, target_ulong *page_size,
-                          ARMMMUFaultInfo *fi)
-{
-    uint32_t secure = regime_is_secure(env, mmu_idx);
-    V8M_SAttributes sattrs = {};
-    bool ret;
-    bool mpu_is_subpage;
-
-    if (arm_feature(env, ARM_FEATURE_M_SECURITY)) {
-        v8m_security_lookup(env, address, access_type, mmu_idx, &sattrs);
-        if (access_type == MMU_INST_FETCH) {
-            /* Instruction fetches always use the MMU bank and the
-             * transaction attribute determined by the fetch address,
-             * regardless of CPU state. This is painful for QEMU
-             * to handle, because it would mean we need to encode
-             * into the mmu_idx not just the (user, negpri) information
-             * for the current security state but also that for the
-             * other security state, which would balloon the number
-             * of mmu_idx values needed alarmingly.
-             * Fortunately we can avoid this because it's not actually
-             * possible to arbitrarily execute code from memory with
-             * the wrong security attribute: it will always generate
-             * an exception of some kind or another, apart from the
-             * special case of an NS CPU executing an SG instruction
-             * in S&NSC memory. So we always just fail the translation
-             * here and sort things out in the exception handler
-             * (including possibly emulating an SG instruction).
-             */
-            if (sattrs.ns != !secure) {
-                if (sattrs.nsc) {
-                    fi->type = ARMFault_QEMU_NSCExec;
-                } else {
-                    fi->type = ARMFault_QEMU_SFault;
-                }
-                *page_size = sattrs.subpage ? 1 : TARGET_PAGE_SIZE;
-                *phys_ptr = address;
-                *prot = 0;
-                return true;
-            }
-        } else {
-            /* For data accesses we always use the MMU bank indicated
-             * by the current CPU state, but the security attributes
-             * might downgrade a secure access to nonsecure.
-             */
-            if (sattrs.ns) {
-                txattrs->secure = false;
-            } else if (!secure) {
-                /* NS access to S memory must fault.
-                 * Architecturally we should first check whether the
-                 * MPU information for this address indicates that we
-                 * are doing an unaligned access to Device memory, which
-                 * should generate a UsageFault instead. QEMU does not
-                 * currently check for that kind of unaligned access though.
-                 * If we added it we would need to do so as a special case
-                 * for M_FAKE_FSR_SFAULT in arm_v7m_cpu_do_interrupt().
-                 */
-                fi->type = ARMFault_QEMU_SFault;
-                *page_size = sattrs.subpage ? 1 : TARGET_PAGE_SIZE;
-                *phys_ptr = address;
-                *prot = 0;
-                return true;
-            }
-        }
-    }
-
-    ret = pmsav8_mpu_lookup(env, address, access_type, mmu_idx, phys_ptr,
-                            txattrs, prot, &mpu_is_subpage, fi, NULL);
-    *page_size = sattrs.subpage || mpu_is_subpage ? 1 : TARGET_PAGE_SIZE;
-    return ret;
-}
-
 /* Combine either inner or outer cacheability attributes for normal
  * memory, according to table D4-42 and pseudocode procedure
  * CombineS1S2AttrHints() of ARM DDI 0487B.b (the ARMv8 ARM).
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 27715dbfa8..28caa7a7ae 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -605,6 +605,83 @@ static bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
     return !(*prot & (1 << access_type));
 }
 
+static bool get_phys_addr_pmsav8(CPUARMState *env, uint32_t address,
+                                 MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                                 hwaddr *phys_ptr, MemTxAttrs *txattrs,
+                                 int *prot, target_ulong *page_size,
+                                 ARMMMUFaultInfo *fi)
+{
+    uint32_t secure = regime_is_secure(env, mmu_idx);
+    V8M_SAttributes sattrs = {};
+    bool ret;
+    bool mpu_is_subpage;
+
+    if (arm_feature(env, ARM_FEATURE_M_SECURITY)) {
+        v8m_security_lookup(env, address, access_type, mmu_idx, &sattrs);
+        if (access_type == MMU_INST_FETCH) {
+            /*
+             * Instruction fetches always use the MMU bank and the
+             * transaction attribute determined by the fetch address,
+             * regardless of CPU state. This is painful for QEMU
+             * to handle, because it would mean we need to encode
+             * into the mmu_idx not just the (user, negpri) information
+             * for the current security state but also that for the
+             * other security state, which would balloon the number
+             * of mmu_idx values needed alarmingly.
+             * Fortunately we can avoid this because it's not actually
+             * possible to arbitrarily execute code from memory with
+             * the wrong security attribute: it will always generate
+             * an exception of some kind or another, apart from the
+             * special case of an NS CPU executing an SG instruction
+             * in S&NSC memory. So we always just fail the translation
+             * here and sort things out in the exception handler
+             * (including possibly emulating an SG instruction).
+             */
+            if (sattrs.ns != !secure) {
+                if (sattrs.nsc) {
+                    fi->type = ARMFault_QEMU_NSCExec;
+                } else {
+                    fi->type = ARMFault_QEMU_SFault;
+                }
+                *page_size = sattrs.subpage ? 1 : TARGET_PAGE_SIZE;
+                *phys_ptr = address;
+                *prot = 0;
+                return true;
+            }
+        } else {
+            /*
+             * For data accesses we always use the MMU bank indicated
+             * by the current CPU state, but the security attributes
+             * might downgrade a secure access to nonsecure.
+             */
+            if (sattrs.ns) {
+                txattrs->secure = false;
+            } else if (!secure) {
+                /*
+                 * NS access to S memory must fault.
+                 * Architecturally we should first check whether the
+                 * MPU information for this address indicates that we
+                 * are doing an unaligned access to Device memory, which
+                 * should generate a UsageFault instead. QEMU does not
+                 * currently check for that kind of unaligned access though.
+                 * If we added it we would need to do so as a special case
+                 * for M_FAKE_FSR_SFAULT in arm_v7m_cpu_do_interrupt().
+                 */
+                fi->type = ARMFault_QEMU_SFault;
+                *page_size = sattrs.subpage ? 1 : TARGET_PAGE_SIZE;
+                *phys_ptr = address;
+                *prot = 0;
+                return true;
+            }
+        }
+    }
+
+    ret = pmsav8_mpu_lookup(env, address, access_type, mmu_idx, phys_ptr,
+                            txattrs, prot, &mpu_is_subpage, fi, NULL);
+    *page_size = sattrs.subpage || mpu_is_subpage ? 1 : TARGET_PAGE_SIZE;
+    return ret;
+}
+
 /**
  * get_phys_addr - get the physical address for this virtual address
  *
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index d24b7c263a..d569507951 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -41,11 +41,6 @@ void get_phys_addr_pmsav7_default(CPUARMState *env,
                                   int32_t address, int *prot);
 bool pmsav7_use_background_region(ARMCPU *cpu, ARMMMUIdx mmu_idx, bool is_user);
 
-bool get_phys_addr_pmsav8(CPUARMState *env, uint32_t address,
-                          MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                          hwaddr *phys_ptr, MemTxAttrs *txattrs,
-                          int *prot, target_ulong *page_size,
-                          ARMMMUFaultInfo *fi);
 bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
                         MMUAccessType access_type, ARMMMUIdx mmu_idx,
                         bool s1_is_el0,

From fedbaa0503d2c56d4ef58ca9a08121e2af7f5e54 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:50 +0100
Subject: [PATCH 868/935] target/arm: Move pmsav8_mpu_lookup to ptw.c

This is the final user of get_phys_addr_pmsav7_default
within helper.c, so make it static within ptw.c.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-10-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 136 -----------------------------------------
 target/arm/ptw.c    | 146 +++++++++++++++++++++++++++++++++++++++++++-
 target/arm/ptw.h    |   3 -
 3 files changed, 143 insertions(+), 142 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 44997fd179..cb23413d8e 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -11834,142 +11834,6 @@ void v8m_security_lookup(CPUARMState *env, uint32_t address,
     }
 }
 
-bool pmsav8_mpu_lookup(CPUARMState *env, uint32_t address,
-                              MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                              hwaddr *phys_ptr, MemTxAttrs *txattrs,
-                              int *prot, bool *is_subpage,
-                              ARMMMUFaultInfo *fi, uint32_t *mregion)
-{
-    /* Perform a PMSAv8 MPU lookup (without also doing the SAU check
-     * that a full phys-to-virt translation does).
-     * mregion is (if not NULL) set to the region number which matched,
-     * or -1 if no region number is returned (MPU off, address did not
-     * hit a region, address hit in multiple regions).
-     * We set is_subpage to true if the region hit doesn't cover the
-     * entire TARGET_PAGE the address is within.
-     */
-    ARMCPU *cpu = env_archcpu(env);
-    bool is_user = regime_is_user(env, mmu_idx);
-    uint32_t secure = regime_is_secure(env, mmu_idx);
-    int n;
-    int matchregion = -1;
-    bool hit = false;
-    uint32_t addr_page_base = address & TARGET_PAGE_MASK;
-    uint32_t addr_page_limit = addr_page_base + (TARGET_PAGE_SIZE - 1);
-
-    *is_subpage = false;
-    *phys_ptr = address;
-    *prot = 0;
-    if (mregion) {
-        *mregion = -1;
-    }
-
-    /* Unlike the ARM ARM pseudocode, we don't need to check whether this
-     * was an exception vector read from the vector table (which is always
-     * done using the default system address map), because those accesses
-     * are done in arm_v7m_load_vector(), which always does a direct
-     * read using address_space_ldl(), rather than going via this function.
-     */
-    if (regime_translation_disabled(env, mmu_idx)) { /* MPU disabled */
-        hit = true;
-    } else if (m_is_ppb_region(env, address)) {
-        hit = true;
-    } else {
-        if (pmsav7_use_background_region(cpu, mmu_idx, is_user)) {
-            hit = true;
-        }
-
-        for (n = (int)cpu->pmsav7_dregion - 1; n >= 0; n--) {
-            /* region search */
-            /* Note that the base address is bits [31:5] from the register
-             * with bits [4:0] all zeroes, but the limit address is bits
-             * [31:5] from the register with bits [4:0] all ones.
-             */
-            uint32_t base = env->pmsav8.rbar[secure][n] & ~0x1f;
-            uint32_t limit = env->pmsav8.rlar[secure][n] | 0x1f;
-
-            if (!(env->pmsav8.rlar[secure][n] & 0x1)) {
-                /* Region disabled */
-                continue;
-            }
-
-            if (address < base || address > limit) {
-                /*
-                 * Address not in this region. We must check whether the
-                 * region covers addresses in the same page as our address.
-                 * In that case we must not report a size that covers the
-                 * whole page for a subsequent hit against a different MPU
-                 * region or the background region, because it would result in
-                 * incorrect TLB hits for subsequent accesses to addresses that
-                 * are in this MPU region.
-                 */
-                if (limit >= base &&
-                    ranges_overlap(base, limit - base + 1,
-                                   addr_page_base,
-                                   TARGET_PAGE_SIZE)) {
-                    *is_subpage = true;
-                }
-                continue;
-            }
-
-            if (base > addr_page_base || limit < addr_page_limit) {
-                *is_subpage = true;
-            }
-
-            if (matchregion != -1) {
-                /* Multiple regions match -- always a failure (unlike
-                 * PMSAv7 where highest-numbered-region wins)
-                 */
-                fi->type = ARMFault_Permission;
-                fi->level = 1;
-                return true;
-            }
-
-            matchregion = n;
-            hit = true;
-        }
-    }
-
-    if (!hit) {
-        /* background fault */
-        fi->type = ARMFault_Background;
-        return true;
-    }
-
-    if (matchregion == -1) {
-        /* hit using the background region */
-        get_phys_addr_pmsav7_default(env, mmu_idx, address, prot);
-    } else {
-        uint32_t ap = extract32(env->pmsav8.rbar[secure][matchregion], 1, 2);
-        uint32_t xn = extract32(env->pmsav8.rbar[secure][matchregion], 0, 1);
-        bool pxn = false;
-
-        if (arm_feature(env, ARM_FEATURE_V8_1M)) {
-            pxn = extract32(env->pmsav8.rlar[secure][matchregion], 4, 1);
-        }
-
-        if (m_is_system_region(env, address)) {
-            /* System space is always execute never */
-            xn = 1;
-        }
-
-        *prot = simple_ap_to_rw_prot(env, mmu_idx, ap);
-        if (*prot && !xn && !(pxn && !is_user)) {
-            *prot |= PAGE_EXEC;
-        }
-        /* We don't need to look the attribute up in the MAIR0/MAIR1
-         * registers because that only tells us about cacheability.
-         */
-        if (mregion) {
-            *mregion = matchregion;
-        }
-    }
-
-    fi->type = ARMFault_Permission;
-    fi->level = 1;
-    return !(*prot & (1 << access_type));
-}
-
 /* Combine either inner or outer cacheability attributes for normal
  * memory, according to table D4-42 and pseudocode procedure
  * CombineS1S2AttrHints() of ARM DDI 0487B.b (the ARMv8 ARM).
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 28caa7a7ae..989e783cce 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -375,9 +375,8 @@ static bool get_phys_addr_pmsav5(CPUARMState *env, uint32_t address,
     return false;
 }
 
-void get_phys_addr_pmsav7_default(CPUARMState *env,
-                                  ARMMMUIdx mmu_idx,
-                                  int32_t address, int *prot)
+static void get_phys_addr_pmsav7_default(CPUARMState *env, ARMMMUIdx mmu_idx,
+                                         int32_t address, int *prot)
 {
     if (!arm_feature(env, ARM_FEATURE_M)) {
         *prot = PAGE_READ | PAGE_WRITE;
@@ -605,6 +604,147 @@ static bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
     return !(*prot & (1 << access_type));
 }
 
+bool pmsav8_mpu_lookup(CPUARMState *env, uint32_t address,
+                       MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                       hwaddr *phys_ptr, MemTxAttrs *txattrs,
+                       int *prot, bool *is_subpage,
+                       ARMMMUFaultInfo *fi, uint32_t *mregion)
+{
+    /*
+     * Perform a PMSAv8 MPU lookup (without also doing the SAU check
+     * that a full phys-to-virt translation does).
+     * mregion is (if not NULL) set to the region number which matched,
+     * or -1 if no region number is returned (MPU off, address did not
+     * hit a region, address hit in multiple regions).
+     * We set is_subpage to true if the region hit doesn't cover the
+     * entire TARGET_PAGE the address is within.
+     */
+    ARMCPU *cpu = env_archcpu(env);
+    bool is_user = regime_is_user(env, mmu_idx);
+    uint32_t secure = regime_is_secure(env, mmu_idx);
+    int n;
+    int matchregion = -1;
+    bool hit = false;
+    uint32_t addr_page_base = address & TARGET_PAGE_MASK;
+    uint32_t addr_page_limit = addr_page_base + (TARGET_PAGE_SIZE - 1);
+
+    *is_subpage = false;
+    *phys_ptr = address;
+    *prot = 0;
+    if (mregion) {
+        *mregion = -1;
+    }
+
+    /*
+     * Unlike the ARM ARM pseudocode, we don't need to check whether this
+     * was an exception vector read from the vector table (which is always
+     * done using the default system address map), because those accesses
+     * are done in arm_v7m_load_vector(), which always does a direct
+     * read using address_space_ldl(), rather than going via this function.
+     */
+    if (regime_translation_disabled(env, mmu_idx)) { /* MPU disabled */
+        hit = true;
+    } else if (m_is_ppb_region(env, address)) {
+        hit = true;
+    } else {
+        if (pmsav7_use_background_region(cpu, mmu_idx, is_user)) {
+            hit = true;
+        }
+
+        for (n = (int)cpu->pmsav7_dregion - 1; n >= 0; n--) {
+            /* region search */
+            /*
+             * Note that the base address is bits [31:5] from the register
+             * with bits [4:0] all zeroes, but the limit address is bits
+             * [31:5] from the register with bits [4:0] all ones.
+             */
+            uint32_t base = env->pmsav8.rbar[secure][n] & ~0x1f;
+            uint32_t limit = env->pmsav8.rlar[secure][n] | 0x1f;
+
+            if (!(env->pmsav8.rlar[secure][n] & 0x1)) {
+                /* Region disabled */
+                continue;
+            }
+
+            if (address < base || address > limit) {
+                /*
+                 * Address not in this region. We must check whether the
+                 * region covers addresses in the same page as our address.
+                 * In that case we must not report a size that covers the
+                 * whole page for a subsequent hit against a different MPU
+                 * region or the background region, because it would result in
+                 * incorrect TLB hits for subsequent accesses to addresses that
+                 * are in this MPU region.
+                 */
+                if (limit >= base &&
+                    ranges_overlap(base, limit - base + 1,
+                                   addr_page_base,
+                                   TARGET_PAGE_SIZE)) {
+                    *is_subpage = true;
+                }
+                continue;
+            }
+
+            if (base > addr_page_base || limit < addr_page_limit) {
+                *is_subpage = true;
+            }
+
+            if (matchregion != -1) {
+                /*
+                 * Multiple regions match -- always a failure (unlike
+                 * PMSAv7 where highest-numbered-region wins)
+                 */
+                fi->type = ARMFault_Permission;
+                fi->level = 1;
+                return true;
+            }
+
+            matchregion = n;
+            hit = true;
+        }
+    }
+
+    if (!hit) {
+        /* background fault */
+        fi->type = ARMFault_Background;
+        return true;
+    }
+
+    if (matchregion == -1) {
+        /* hit using the background region */
+        get_phys_addr_pmsav7_default(env, mmu_idx, address, prot);
+    } else {
+        uint32_t ap = extract32(env->pmsav8.rbar[secure][matchregion], 1, 2);
+        uint32_t xn = extract32(env->pmsav8.rbar[secure][matchregion], 0, 1);
+        bool pxn = false;
+
+        if (arm_feature(env, ARM_FEATURE_V8_1M)) {
+            pxn = extract32(env->pmsav8.rlar[secure][matchregion], 4, 1);
+        }
+
+        if (m_is_system_region(env, address)) {
+            /* System space is always execute never */
+            xn = 1;
+        }
+
+        *prot = simple_ap_to_rw_prot(env, mmu_idx, ap);
+        if (*prot && !xn && !(pxn && !is_user)) {
+            *prot |= PAGE_EXEC;
+        }
+        /*
+         * We don't need to look the attribute up in the MAIR0/MAIR1
+         * registers because that only tells us about cacheability.
+         */
+        if (mregion) {
+            *mregion = matchregion;
+        }
+    }
+
+    fi->type = ARMFault_Permission;
+    fi->level = 1;
+    return !(*prot & (1 << access_type));
+}
+
 static bool get_phys_addr_pmsav8(CPUARMState *env, uint32_t address,
                                  MMUAccessType access_type, ARMMMUIdx mmu_idx,
                                  hwaddr *phys_ptr, MemTxAttrs *txattrs,
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index d569507951..8d2e239714 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -36,9 +36,6 @@ simple_ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap)
 bool m_is_ppb_region(CPUARMState *env, uint32_t address);
 bool m_is_system_region(CPUARMState *env, uint32_t address);
 
-void get_phys_addr_pmsav7_default(CPUARMState *env,
-                                  ARMMMUIdx mmu_idx,
-                                  int32_t address, int *prot);
 bool pmsav7_use_background_region(ARMCPU *cpu, ARMMMUIdx mmu_idx, bool is_user);
 
 bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,

From c8e436c9f72bca95e37a155d6ead42d55f74eed2 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:50 +0100
Subject: [PATCH 869/935] target/arm: Move pmsav7_use_background_region to
 ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-11-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 19 -------------------
 target/arm/ptw.c    | 21 +++++++++++++++++++++
 target/arm/ptw.h    |  2 --
 3 files changed, 21 insertions(+), 21 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index cb23413d8e..62e48f0925 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -11678,25 +11678,6 @@ do_fault:
     return true;
 }
 
-bool pmsav7_use_background_region(ARMCPU *cpu, ARMMMUIdx mmu_idx, bool is_user)
-{
-    /* Return true if we should use the default memory map as a
-     * "background" region if there are no hits against any MPU regions.
-     */
-    CPUARMState *env = &cpu->env;
-
-    if (is_user) {
-        return false;
-    }
-
-    if (arm_feature(env, ARM_FEATURE_M)) {
-        return env->v7m.mpu_ctrl[regime_is_secure(env, mmu_idx)]
-            & R_V7M_MPU_CTRL_PRIVDEFENA_MASK;
-    } else {
-        return regime_sctlr(env, mmu_idx) & SCTLR_BR;
-    }
-}
-
 bool m_is_ppb_region(CPUARMState *env, uint32_t address)
 {
     /* True if address is in the M profile PPB region 0xe0000000 - 0xe00fffff */
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 989e783cce..b82638b5a0 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -415,6 +415,27 @@ static void get_phys_addr_pmsav7_default(CPUARMState *env, ARMMMUIdx mmu_idx,
     }
 }
 
+static bool pmsav7_use_background_region(ARMCPU *cpu, ARMMMUIdx mmu_idx,
+                                         bool is_user)
+{
+    /*
+     * Return true if we should use the default memory map as a
+     * "background" region if there are no hits against any MPU regions.
+     */
+    CPUARMState *env = &cpu->env;
+
+    if (is_user) {
+        return false;
+    }
+
+    if (arm_feature(env, ARM_FEATURE_M)) {
+        return env->v7m.mpu_ctrl[regime_is_secure(env, mmu_idx)]
+            & R_V7M_MPU_CTRL_PRIVDEFENA_MASK;
+    } else {
+        return regime_sctlr(env, mmu_idx) & SCTLR_BR;
+    }
+}
+
 static bool get_phys_addr_pmsav7(CPUARMState *env, uint32_t address,
                                  MMUAccessType access_type, ARMMMUIdx mmu_idx,
                                  hwaddr *phys_ptr, int *prot,
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index 8d2e239714..d2d2711908 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -36,8 +36,6 @@ simple_ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap)
 bool m_is_ppb_region(CPUARMState *env, uint32_t address);
 bool m_is_system_region(CPUARMState *env, uint32_t address);
 
-bool pmsav7_use_background_region(ARMCPU *cpu, ARMMMUIdx mmu_idx, bool is_user);
-
 bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
                         MMUAccessType access_type, ARMMMUIdx mmu_idx,
                         bool s1_is_el0,

From 2c1f429df38822ce6955224d784f1567f8af219f Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:50 +0100
Subject: [PATCH 870/935] target/arm: Move v8m_security_lookup to ptw.c

This function has one private helper, v8m_is_sau_exempt,
so move that at the same time.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-12-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 123 ------------------------------------------
 target/arm/ptw.c    | 126 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 126 insertions(+), 123 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 62e48f0925..d6a749ad0e 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -9,7 +9,6 @@
 #include "qemu/osdep.h"
 #include "qemu/units.h"
 #include "qemu/log.h"
-#include "target/arm/idau.h"
 #include "trace.h"
 #include "cpu.h"
 #include "internals.h"
@@ -11693,128 +11692,6 @@ bool m_is_system_region(CPUARMState *env, uint32_t address)
     return arm_feature(env, ARM_FEATURE_M) && extract32(address, 29, 3) == 0x7;
 }
 
-static bool v8m_is_sau_exempt(CPUARMState *env,
-                              uint32_t address, MMUAccessType access_type)
-{
-    /* The architecture specifies that certain address ranges are
-     * exempt from v8M SAU/IDAU checks.
-     */
-    return
-        (access_type == MMU_INST_FETCH && m_is_system_region(env, address)) ||
-        (address >= 0xe0000000 && address <= 0xe0002fff) ||
-        (address >= 0xe000e000 && address <= 0xe000efff) ||
-        (address >= 0xe002e000 && address <= 0xe002efff) ||
-        (address >= 0xe0040000 && address <= 0xe0041fff) ||
-        (address >= 0xe00ff000 && address <= 0xe00fffff);
-}
-
-void v8m_security_lookup(CPUARMState *env, uint32_t address,
-                                MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                                V8M_SAttributes *sattrs)
-{
-    /* Look up the security attributes for this address. Compare the
-     * pseudocode SecurityCheck() function.
-     * We assume the caller has zero-initialized *sattrs.
-     */
-    ARMCPU *cpu = env_archcpu(env);
-    int r;
-    bool idau_exempt = false, idau_ns = true, idau_nsc = true;
-    int idau_region = IREGION_NOTVALID;
-    uint32_t addr_page_base = address & TARGET_PAGE_MASK;
-    uint32_t addr_page_limit = addr_page_base + (TARGET_PAGE_SIZE - 1);
-
-    if (cpu->idau) {
-        IDAUInterfaceClass *iic = IDAU_INTERFACE_GET_CLASS(cpu->idau);
-        IDAUInterface *ii = IDAU_INTERFACE(cpu->idau);
-
-        iic->check(ii, address, &idau_region, &idau_exempt, &idau_ns,
-                   &idau_nsc);
-    }
-
-    if (access_type == MMU_INST_FETCH && extract32(address, 28, 4) == 0xf) {
-        /* 0xf0000000..0xffffffff is always S for insn fetches */
-        return;
-    }
-
-    if (idau_exempt || v8m_is_sau_exempt(env, address, access_type)) {
-        sattrs->ns = !regime_is_secure(env, mmu_idx);
-        return;
-    }
-
-    if (idau_region != IREGION_NOTVALID) {
-        sattrs->irvalid = true;
-        sattrs->iregion = idau_region;
-    }
-
-    switch (env->sau.ctrl & 3) {
-    case 0: /* SAU.ENABLE == 0, SAU.ALLNS == 0 */
-        break;
-    case 2: /* SAU.ENABLE == 0, SAU.ALLNS == 1 */
-        sattrs->ns = true;
-        break;
-    default: /* SAU.ENABLE == 1 */
-        for (r = 0; r < cpu->sau_sregion; r++) {
-            if (env->sau.rlar[r] & 1) {
-                uint32_t base = env->sau.rbar[r] & ~0x1f;
-                uint32_t limit = env->sau.rlar[r] | 0x1f;
-
-                if (base <= address && limit >= address) {
-                    if (base > addr_page_base || limit < addr_page_limit) {
-                        sattrs->subpage = true;
-                    }
-                    if (sattrs->srvalid) {
-                        /* If we hit in more than one region then we must report
-                         * as Secure, not NS-Callable, with no valid region
-                         * number info.
-                         */
-                        sattrs->ns = false;
-                        sattrs->nsc = false;
-                        sattrs->sregion = 0;
-                        sattrs->srvalid = false;
-                        break;
-                    } else {
-                        if (env->sau.rlar[r] & 2) {
-                            sattrs->nsc = true;
-                        } else {
-                            sattrs->ns = true;
-                        }
-                        sattrs->srvalid = true;
-                        sattrs->sregion = r;
-                    }
-                } else {
-                    /*
-                     * Address not in this region. We must check whether the
-                     * region covers addresses in the same page as our address.
-                     * In that case we must not report a size that covers the
-                     * whole page for a subsequent hit against a different MPU
-                     * region or the background region, because it would result
-                     * in incorrect TLB hits for subsequent accesses to
-                     * addresses that are in this MPU region.
-                     */
-                    if (limit >= base &&
-                        ranges_overlap(base, limit - base + 1,
-                                       addr_page_base,
-                                       TARGET_PAGE_SIZE)) {
-                        sattrs->subpage = true;
-                    }
-                }
-            }
-        }
-        break;
-    }
-
-    /*
-     * The IDAU will override the SAU lookup results if it specifies
-     * higher security than the SAU does.
-     */
-    if (!idau_ns) {
-        if (sattrs->ns || (!idau_nsc && sattrs->nsc)) {
-            sattrs->ns = false;
-            sattrs->nsc = idau_nsc;
-        }
-    }
-}
-
 /* Combine either inner or outer cacheability attributes for normal
  * memory, according to table D4-42 and pseudocode procedure
  * CombineS1S2AttrHints() of ARM DDI 0487B.b (the ARMv8 ARM).
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index b82638b5a0..c15fba43c3 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -11,6 +11,7 @@
 #include "qemu/range.h"
 #include "cpu.h"
 #include "internals.h"
+#include "idau.h"
 #include "ptw.h"
 
 
@@ -766,6 +767,131 @@ bool pmsav8_mpu_lookup(CPUARMState *env, uint32_t address,
     return !(*prot & (1 << access_type));
 }
 
+static bool v8m_is_sau_exempt(CPUARMState *env,
+                              uint32_t address, MMUAccessType access_type)
+{
+    /*
+     * The architecture specifies that certain address ranges are
+     * exempt from v8M SAU/IDAU checks.
+     */
+    return
+        (access_type == MMU_INST_FETCH && m_is_system_region(env, address)) ||
+        (address >= 0xe0000000 && address <= 0xe0002fff) ||
+        (address >= 0xe000e000 && address <= 0xe000efff) ||
+        (address >= 0xe002e000 && address <= 0xe002efff) ||
+        (address >= 0xe0040000 && address <= 0xe0041fff) ||
+        (address >= 0xe00ff000 && address <= 0xe00fffff);
+}
+
+void v8m_security_lookup(CPUARMState *env, uint32_t address,
+                                MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                                V8M_SAttributes *sattrs)
+{
+    /*
+     * Look up the security attributes for this address. Compare the
+     * pseudocode SecurityCheck() function.
+     * We assume the caller has zero-initialized *sattrs.
+     */
+    ARMCPU *cpu = env_archcpu(env);
+    int r;
+    bool idau_exempt = false, idau_ns = true, idau_nsc = true;
+    int idau_region = IREGION_NOTVALID;
+    uint32_t addr_page_base = address & TARGET_PAGE_MASK;
+    uint32_t addr_page_limit = addr_page_base + (TARGET_PAGE_SIZE - 1);
+
+    if (cpu->idau) {
+        IDAUInterfaceClass *iic = IDAU_INTERFACE_GET_CLASS(cpu->idau);
+        IDAUInterface *ii = IDAU_INTERFACE(cpu->idau);
+
+        iic->check(ii, address, &idau_region, &idau_exempt, &idau_ns,
+                   &idau_nsc);
+    }
+
+    if (access_type == MMU_INST_FETCH && extract32(address, 28, 4) == 0xf) {
+        /* 0xf0000000..0xffffffff is always S for insn fetches */
+        return;
+    }
+
+    if (idau_exempt || v8m_is_sau_exempt(env, address, access_type)) {
+        sattrs->ns = !regime_is_secure(env, mmu_idx);
+        return;
+    }
+
+    if (idau_region != IREGION_NOTVALID) {
+        sattrs->irvalid = true;
+        sattrs->iregion = idau_region;
+    }
+
+    switch (env->sau.ctrl & 3) {
+    case 0: /* SAU.ENABLE == 0, SAU.ALLNS == 0 */
+        break;
+    case 2: /* SAU.ENABLE == 0, SAU.ALLNS == 1 */
+        sattrs->ns = true;
+        break;
+    default: /* SAU.ENABLE == 1 */
+        for (r = 0; r < cpu->sau_sregion; r++) {
+            if (env->sau.rlar[r] & 1) {
+                uint32_t base = env->sau.rbar[r] & ~0x1f;
+                uint32_t limit = env->sau.rlar[r] | 0x1f;
+
+                if (base <= address && limit >= address) {
+                    if (base > addr_page_base || limit < addr_page_limit) {
+                        sattrs->subpage = true;
+                    }
+                    if (sattrs->srvalid) {
+                        /*
+                         * If we hit in more than one region then we must report
+                         * as Secure, not NS-Callable, with no valid region
+                         * number info.
+                         */
+                        sattrs->ns = false;
+                        sattrs->nsc = false;
+                        sattrs->sregion = 0;
+                        sattrs->srvalid = false;
+                        break;
+                    } else {
+                        if (env->sau.rlar[r] & 2) {
+                            sattrs->nsc = true;
+                        } else {
+                            sattrs->ns = true;
+                        }
+                        sattrs->srvalid = true;
+                        sattrs->sregion = r;
+                    }
+                } else {
+                    /*
+                     * Address not in this region. We must check whether the
+                     * region covers addresses in the same page as our address.
+                     * In that case we must not report a size that covers the
+                     * whole page for a subsequent hit against a different MPU
+                     * region or the background region, because it would result
+                     * in incorrect TLB hits for subsequent accesses to
+                     * addresses that are in this MPU region.
+                     */
+                    if (limit >= base &&
+                        ranges_overlap(base, limit - base + 1,
+                                       addr_page_base,
+                                       TARGET_PAGE_SIZE)) {
+                        sattrs->subpage = true;
+                    }
+                }
+            }
+        }
+        break;
+    }
+
+    /*
+     * The IDAU will override the SAU lookup results if it specifies
+     * higher security than the SAU does.
+     */
+    if (!idau_ns) {
+        if (sattrs->ns || (!idau_nsc && sattrs->nsc)) {
+            sattrs->ns = false;
+            sattrs->nsc = idau_nsc;
+        }
+    }
+}
+
 static bool get_phys_addr_pmsav8(CPUARMState *env, uint32_t address,
                                  MMUAccessType access_type, ARMMMUIdx mmu_idx,
                                  hwaddr *phys_ptr, MemTxAttrs *txattrs,

From 47ff5ba9d08c2c769794a59b88f1c19ada004730 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:50 +0100
Subject: [PATCH 871/935] target/arm: Move m_is_{ppb,system}_region to ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-13-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 15 ---------------
 target/arm/ptw.c    | 16 ++++++++++++++++
 target/arm/ptw.h    |  3 ---
 3 files changed, 16 insertions(+), 18 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index d6a749ad0e..d2ef12346b 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -11677,21 +11677,6 @@ do_fault:
     return true;
 }
 
-bool m_is_ppb_region(CPUARMState *env, uint32_t address)
-{
-    /* True if address is in the M profile PPB region 0xe0000000 - 0xe00fffff */
-    return arm_feature(env, ARM_FEATURE_M) &&
-        extract32(address, 20, 12) == 0xe00;
-}
-
-bool m_is_system_region(CPUARMState *env, uint32_t address)
-{
-    /* True if address is in the M profile system region
-     * 0xe0000000 - 0xffffffff
-     */
-    return arm_feature(env, ARM_FEATURE_M) && extract32(address, 29, 3) == 0x7;
-}
-
 /* Combine either inner or outer cacheability attributes for normal
  * memory, according to table D4-42 and pseudocode procedure
  * CombineS1S2AttrHints() of ARM DDI 0487B.b (the ARMv8 ARM).
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index c15fba43c3..32ba2e5e8b 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -416,6 +416,22 @@ static void get_phys_addr_pmsav7_default(CPUARMState *env, ARMMMUIdx mmu_idx,
     }
 }
 
+static bool m_is_ppb_region(CPUARMState *env, uint32_t address)
+{
+    /* True if address is in the M profile PPB region 0xe0000000 - 0xe00fffff */
+    return arm_feature(env, ARM_FEATURE_M) &&
+        extract32(address, 20, 12) == 0xe00;
+}
+
+static bool m_is_system_region(CPUARMState *env, uint32_t address)
+{
+    /*
+     * True if address is in the M profile system region
+     * 0xe0000000 - 0xffffffff
+     */
+    return arm_feature(env, ARM_FEATURE_M) && extract32(address, 29, 3) == 0x7;
+}
+
 static bool pmsav7_use_background_region(ARMCPU *cpu, ARMMMUIdx mmu_idx,
                                          bool is_user)
 {
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index d2d2711908..6c47a57599 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -33,9 +33,6 @@ simple_ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap)
     return simple_ap_to_rw_prot_is_user(ap, regime_is_user(env, mmu_idx));
 }
 
-bool m_is_ppb_region(CPUARMState *env, uint32_t address);
-bool m_is_system_region(CPUARMState *env, uint32_t address);
-
 bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
                         MMUAccessType access_type, ARMMMUIdx mmu_idx,
                         bool s1_is_el0,

From 4c74ab157b056710b043a02c8101c449c179ae11 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:51 +0100
Subject: [PATCH 872/935] target/arm: Move get_level1_table_address to ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-14-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 26 +-------------------------
 target/arm/ptw.c    | 23 +++++++++++++++++++++++
 target/arm/ptw.h    |  4 ++--
 3 files changed, 26 insertions(+), 27 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index d2ef12346b..a144cb2641 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10482,8 +10482,7 @@ static inline bool regime_translation_big_endian(CPUARMState *env,
 }
 
 /* Return the TTBR associated with this translation regime */
-static inline uint64_t regime_ttbr(CPUARMState *env, ARMMMUIdx mmu_idx,
-                                   int ttbrn)
+uint64_t regime_ttbr(CPUARMState *env, ARMMMUIdx mmu_idx, int ttbrn)
 {
     if (mmu_idx == ARMMMUIdx_Stage2) {
         return env->cp15.vttbr_el2;
@@ -10774,29 +10773,6 @@ static int get_S1prot(CPUARMState *env, ARMMMUIdx mmu_idx, bool is_aa64,
     return prot_rw | PAGE_EXEC;
 }
 
-bool get_level1_table_address(CPUARMState *env, ARMMMUIdx mmu_idx,
-                              uint32_t *table, uint32_t address)
-{
-    /* Note that we can only get here for an AArch32 PL0/PL1 lookup */
-    TCR *tcr = regime_tcr(env, mmu_idx);
-
-    if (address & tcr->mask) {
-        if (tcr->raw_tcr & TTBCR_PD1) {
-            /* Translation table walk disabled for TTBR1 */
-            return false;
-        }
-        *table = regime_ttbr(env, mmu_idx, 1) & 0xffffc000;
-    } else {
-        if (tcr->raw_tcr & TTBCR_PD0) {
-            /* Translation table walk disabled for TTBR0 */
-            return false;
-        }
-        *table = regime_ttbr(env, mmu_idx, 0) & tcr->base_mask;
-    }
-    *table |= (address >> 18) & 0x3ffc;
-    return true;
-}
-
 static bool ptw_attrs_are_device(CPUARMState *env, ARMCacheAttrs cacheattrs)
 {
     /*
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 32ba2e5e8b..5737a3976b 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -15,6 +15,29 @@
 #include "ptw.h"
 
 
+static bool get_level1_table_address(CPUARMState *env, ARMMMUIdx mmu_idx,
+                                     uint32_t *table, uint32_t address)
+{
+    /* Note that we can only get here for an AArch32 PL0/PL1 lookup */
+    TCR *tcr = regime_tcr(env, mmu_idx);
+
+    if (address & tcr->mask) {
+        if (tcr->raw_tcr & TTBCR_PD1) {
+            /* Translation table walk disabled for TTBR1 */
+            return false;
+        }
+        *table = regime_ttbr(env, mmu_idx, 1) & 0xffffc000;
+    } else {
+        if (tcr->raw_tcr & TTBCR_PD0) {
+            /* Translation table walk disabled for TTBR0 */
+            return false;
+        }
+        *table = regime_ttbr(env, mmu_idx, 0) & tcr->base_mask;
+    }
+    *table |= (address >> 18) & 0x3ffc;
+    return true;
+}
+
 static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
                              MMUAccessType access_type, ARMMMUIdx mmu_idx,
                              hwaddr *phys_ptr, int *prot,
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index 6c47a57599..dd6fb93f33 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -18,11 +18,11 @@ uint64_t arm_ldq_ptw(CPUState *cs, hwaddr addr, bool is_secure,
 
 bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx);
 bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx);
+uint64_t regime_ttbr(CPUARMState *env, ARMMMUIdx mmu_idx, int ttbrn);
+
 ARMCacheAttrs combine_cacheattrs(CPUARMState *env,
                                  ARMCacheAttrs s1, ARMCacheAttrs s2);
 
-bool get_level1_table_address(CPUARMState *env, ARMMMUIdx mmu_idx,
-                              uint32_t *table, uint32_t address);
 int ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx,
                   int ap, int domain_prot);
 int simple_ap_to_rw_prot_is_user(int ap, bool is_user);

From 966f4bb7d88554ed74e1006c516ddeef37e523d6 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:51 +0100
Subject: [PATCH 873/935] target/arm: Move combine_cacheattrs and subroutines
 to ptw.c

There are a handful of helpers for combine_cacheattrs
that we can move at the same time as the main entry point.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-15-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 218 -------------------------------------------
 target/arm/ptw.c    | 221 ++++++++++++++++++++++++++++++++++++++++++++
 target/arm/ptw.h    |   3 -
 3 files changed, 221 insertions(+), 221 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index a144cb2641..dab485e64a 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10977,36 +10977,6 @@ static bool check_s2_mmu_setup(ARMCPU *cpu, bool is_aa64, int level,
     }
     return true;
 }
-
-/* Translate from the 4-bit stage 2 representation of
- * memory attributes (without cache-allocation hints) to
- * the 8-bit representation of the stage 1 MAIR registers
- * (which includes allocation hints).
- *
- * ref: shared/translation/attrs/S2AttrDecode()
- *      .../S2ConvertAttrsHints()
- */
-static uint8_t convert_stage2_attrs(CPUARMState *env, uint8_t s2attrs)
-{
-    uint8_t hiattr = extract32(s2attrs, 2, 2);
-    uint8_t loattr = extract32(s2attrs, 0, 2);
-    uint8_t hihint = 0, lohint = 0;
-
-    if (hiattr != 0) { /* normal memory */
-        if (arm_hcr_el2_eff(env) & HCR_CD) { /* cache disabled */
-            hiattr = loattr = 1; /* non-cacheable */
-        } else {
-            if (hiattr != 1) { /* Write-through or write-back */
-                hihint = 3; /* RW allocate */
-            }
-            if (loattr != 1) { /* Write-through or write-back */
-                lohint = 3; /* RW allocate */
-            }
-        }
-    }
-
-    return (hiattr << 6) | (hihint << 4) | (loattr << 2) | lohint;
-}
 #endif /* !CONFIG_USER_ONLY */
 
 /* This mapping is common between ID_AA64MMFR0.PARANGE and TCR_ELx.{I}PS. */
@@ -11653,194 +11623,6 @@ do_fault:
     return true;
 }
 
-/* Combine either inner or outer cacheability attributes for normal
- * memory, according to table D4-42 and pseudocode procedure
- * CombineS1S2AttrHints() of ARM DDI 0487B.b (the ARMv8 ARM).
- *
- * NB: only stage 1 includes allocation hints (RW bits), leading to
- * some asymmetry.
- */
-static uint8_t combine_cacheattr_nibble(uint8_t s1, uint8_t s2)
-{
-    if (s1 == 4 || s2 == 4) {
-        /* non-cacheable has precedence */
-        return 4;
-    } else if (extract32(s1, 2, 2) == 0 || extract32(s1, 2, 2) == 2) {
-        /* stage 1 write-through takes precedence */
-        return s1;
-    } else if (extract32(s2, 2, 2) == 2) {
-        /* stage 2 write-through takes precedence, but the allocation hint
-         * is still taken from stage 1
-         */
-        return (2 << 2) | extract32(s1, 0, 2);
-    } else { /* write-back */
-        return s1;
-    }
-}
-
-/*
- * Combine the memory type and cacheability attributes of
- * s1 and s2 for the HCR_EL2.FWB == 0 case, returning the
- * combined attributes in MAIR_EL1 format.
- */
-static uint8_t combined_attrs_nofwb(CPUARMState *env,
-                                    ARMCacheAttrs s1, ARMCacheAttrs s2)
-{
-    uint8_t s1lo, s2lo, s1hi, s2hi, s2_mair_attrs, ret_attrs;
-
-    s2_mair_attrs = convert_stage2_attrs(env, s2.attrs);
-
-    s1lo = extract32(s1.attrs, 0, 4);
-    s2lo = extract32(s2_mair_attrs, 0, 4);
-    s1hi = extract32(s1.attrs, 4, 4);
-    s2hi = extract32(s2_mair_attrs, 4, 4);
-
-    /* Combine memory type and cacheability attributes */
-    if (s1hi == 0 || s2hi == 0) {
-        /* Device has precedence over normal */
-        if (s1lo == 0 || s2lo == 0) {
-            /* nGnRnE has precedence over anything */
-            ret_attrs = 0;
-        } else if (s1lo == 4 || s2lo == 4) {
-            /* non-Reordering has precedence over Reordering */
-            ret_attrs = 4;  /* nGnRE */
-        } else if (s1lo == 8 || s2lo == 8) {
-            /* non-Gathering has precedence over Gathering */
-            ret_attrs = 8;  /* nGRE */
-        } else {
-            ret_attrs = 0xc; /* GRE */
-        }
-    } else { /* Normal memory */
-        /* Outer/inner cacheability combine independently */
-        ret_attrs = combine_cacheattr_nibble(s1hi, s2hi) << 4
-                  | combine_cacheattr_nibble(s1lo, s2lo);
-    }
-    return ret_attrs;
-}
-
-static uint8_t force_cacheattr_nibble_wb(uint8_t attr)
-{
-    /*
-     * Given the 4 bits specifying the outer or inner cacheability
-     * in MAIR format, return a value specifying Normal Write-Back,
-     * with the allocation and transient hints taken from the input
-     * if the input specified some kind of cacheable attribute.
-     */
-    if (attr == 0 || attr == 4) {
-        /*
-         * 0 == an UNPREDICTABLE encoding
-         * 4 == Non-cacheable
-         * Either way, force Write-Back RW allocate non-transient
-         */
-        return 0xf;
-    }
-    /* Change WriteThrough to WriteBack, keep allocation and transient hints */
-    return attr | 4;
-}
-
-/*
- * Combine the memory type and cacheability attributes of
- * s1 and s2 for the HCR_EL2.FWB == 1 case, returning the
- * combined attributes in MAIR_EL1 format.
- */
-static uint8_t combined_attrs_fwb(CPUARMState *env,
-                                  ARMCacheAttrs s1, ARMCacheAttrs s2)
-{
-    switch (s2.attrs) {
-    case 7:
-        /* Use stage 1 attributes */
-        return s1.attrs;
-    case 6:
-        /*
-         * Force Normal Write-Back. Note that if S1 is Normal cacheable
-         * then we take the allocation hints from it; otherwise it is
-         * RW allocate, non-transient.
-         */
-        if ((s1.attrs & 0xf0) == 0) {
-            /* S1 is Device */
-            return 0xff;
-        }
-        /* Need to check the Inner and Outer nibbles separately */
-        return force_cacheattr_nibble_wb(s1.attrs & 0xf) |
-            force_cacheattr_nibble_wb(s1.attrs >> 4) << 4;
-    case 5:
-        /* If S1 attrs are Device, use them; otherwise Normal Non-cacheable */
-        if ((s1.attrs & 0xf0) == 0) {
-            return s1.attrs;
-        }
-        return 0x44;
-    case 0 ... 3:
-        /* Force Device, of subtype specified by S2 */
-        return s2.attrs << 2;
-    default:
-        /*
-         * RESERVED values (including RES0 descriptor bit [5] being nonzero);
-         * arbitrarily force Device.
-         */
-        return 0;
-    }
-}
-
-/* Combine S1 and S2 cacheability/shareability attributes, per D4.5.4
- * and CombineS1S2Desc()
- *
- * @env:     CPUARMState
- * @s1:      Attributes from stage 1 walk
- * @s2:      Attributes from stage 2 walk
- */
-ARMCacheAttrs combine_cacheattrs(CPUARMState *env,
-                                 ARMCacheAttrs s1, ARMCacheAttrs s2)
-{
-    ARMCacheAttrs ret;
-    bool tagged = false;
-
-    assert(s2.is_s2_format && !s1.is_s2_format);
-    ret.is_s2_format = false;
-
-    if (s1.attrs == 0xf0) {
-        tagged = true;
-        s1.attrs = 0xff;
-    }
-
-    /* Combine shareability attributes (table D4-43) */
-    if (s1.shareability == 2 || s2.shareability == 2) {
-        /* if either are outer-shareable, the result is outer-shareable */
-        ret.shareability = 2;
-    } else if (s1.shareability == 3 || s2.shareability == 3) {
-        /* if either are inner-shareable, the result is inner-shareable */
-        ret.shareability = 3;
-    } else {
-        /* both non-shareable */
-        ret.shareability = 0;
-    }
-
-    /* Combine memory type and cacheability attributes */
-    if (arm_hcr_el2_eff(env) & HCR_FWB) {
-        ret.attrs = combined_attrs_fwb(env, s1, s2);
-    } else {
-        ret.attrs = combined_attrs_nofwb(env, s1, s2);
-    }
-
-    /*
-     * Any location for which the resultant memory type is any
-     * type of Device memory is always treated as Outer Shareable.
-     * Any location for which the resultant memory type is Normal
-     * Inner Non-cacheable, Outer Non-cacheable is always treated
-     * as Outer Shareable.
-     * TODO: FEAT_XS adds another value (0x40) also meaning iNCoNC
-     */
-    if ((ret.attrs & 0xf0) == 0 || ret.attrs == 0x44) {
-        ret.shareability = 2;
-    }
-
-    /* TODO: CombineS1S2Desc does not consider transient, only WB, RWA. */
-    if (tagged && ret.attrs == 0xff) {
-        ret.attrs = 0xf0;
-    }
-
-    return ret;
-}
-
 hwaddr arm_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr,
                                          MemTxAttrs *attrs)
 {
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 5737a3976b..f2ca2bb8fe 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -1008,6 +1008,227 @@ static bool get_phys_addr_pmsav8(CPUARMState *env, uint32_t address,
     return ret;
 }
 
+/*
+ * Translate from the 4-bit stage 2 representation of
+ * memory attributes (without cache-allocation hints) to
+ * the 8-bit representation of the stage 1 MAIR registers
+ * (which includes allocation hints).
+ *
+ * ref: shared/translation/attrs/S2AttrDecode()
+ *      .../S2ConvertAttrsHints()
+ */
+static uint8_t convert_stage2_attrs(CPUARMState *env, uint8_t s2attrs)
+{
+    uint8_t hiattr = extract32(s2attrs, 2, 2);
+    uint8_t loattr = extract32(s2attrs, 0, 2);
+    uint8_t hihint = 0, lohint = 0;
+
+    if (hiattr != 0) { /* normal memory */
+        if (arm_hcr_el2_eff(env) & HCR_CD) { /* cache disabled */
+            hiattr = loattr = 1; /* non-cacheable */
+        } else {
+            if (hiattr != 1) { /* Write-through or write-back */
+                hihint = 3; /* RW allocate */
+            }
+            if (loattr != 1) { /* Write-through or write-back */
+                lohint = 3; /* RW allocate */
+            }
+        }
+    }
+
+    return (hiattr << 6) | (hihint << 4) | (loattr << 2) | lohint;
+}
+
+/*
+ * Combine either inner or outer cacheability attributes for normal
+ * memory, according to table D4-42 and pseudocode procedure
+ * CombineS1S2AttrHints() of ARM DDI 0487B.b (the ARMv8 ARM).
+ *
+ * NB: only stage 1 includes allocation hints (RW bits), leading to
+ * some asymmetry.
+ */
+static uint8_t combine_cacheattr_nibble(uint8_t s1, uint8_t s2)
+{
+    if (s1 == 4 || s2 == 4) {
+        /* non-cacheable has precedence */
+        return 4;
+    } else if (extract32(s1, 2, 2) == 0 || extract32(s1, 2, 2) == 2) {
+        /* stage 1 write-through takes precedence */
+        return s1;
+    } else if (extract32(s2, 2, 2) == 2) {
+        /* stage 2 write-through takes precedence, but the allocation hint
+         * is still taken from stage 1
+         */
+        return (2 << 2) | extract32(s1, 0, 2);
+    } else { /* write-back */
+        return s1;
+    }
+}
+
+/*
+ * Combine the memory type and cacheability attributes of
+ * s1 and s2 for the HCR_EL2.FWB == 0 case, returning the
+ * combined attributes in MAIR_EL1 format.
+ */
+static uint8_t combined_attrs_nofwb(CPUARMState *env,
+                                    ARMCacheAttrs s1, ARMCacheAttrs s2)
+{
+    uint8_t s1lo, s2lo, s1hi, s2hi, s2_mair_attrs, ret_attrs;
+
+    s2_mair_attrs = convert_stage2_attrs(env, s2.attrs);
+
+    s1lo = extract32(s1.attrs, 0, 4);
+    s2lo = extract32(s2_mair_attrs, 0, 4);
+    s1hi = extract32(s1.attrs, 4, 4);
+    s2hi = extract32(s2_mair_attrs, 4, 4);
+
+    /* Combine memory type and cacheability attributes */
+    if (s1hi == 0 || s2hi == 0) {
+        /* Device has precedence over normal */
+        if (s1lo == 0 || s2lo == 0) {
+            /* nGnRnE has precedence over anything */
+            ret_attrs = 0;
+        } else if (s1lo == 4 || s2lo == 4) {
+            /* non-Reordering has precedence over Reordering */
+            ret_attrs = 4;  /* nGnRE */
+        } else if (s1lo == 8 || s2lo == 8) {
+            /* non-Gathering has precedence over Gathering */
+            ret_attrs = 8;  /* nGRE */
+        } else {
+            ret_attrs = 0xc; /* GRE */
+        }
+    } else { /* Normal memory */
+        /* Outer/inner cacheability combine independently */
+        ret_attrs = combine_cacheattr_nibble(s1hi, s2hi) << 4
+                  | combine_cacheattr_nibble(s1lo, s2lo);
+    }
+    return ret_attrs;
+}
+
+static uint8_t force_cacheattr_nibble_wb(uint8_t attr)
+{
+    /*
+     * Given the 4 bits specifying the outer or inner cacheability
+     * in MAIR format, return a value specifying Normal Write-Back,
+     * with the allocation and transient hints taken from the input
+     * if the input specified some kind of cacheable attribute.
+     */
+    if (attr == 0 || attr == 4) {
+        /*
+         * 0 == an UNPREDICTABLE encoding
+         * 4 == Non-cacheable
+         * Either way, force Write-Back RW allocate non-transient
+         */
+        return 0xf;
+    }
+    /* Change WriteThrough to WriteBack, keep allocation and transient hints */
+    return attr | 4;
+}
+
+/*
+ * Combine the memory type and cacheability attributes of
+ * s1 and s2 for the HCR_EL2.FWB == 1 case, returning the
+ * combined attributes in MAIR_EL1 format.
+ */
+static uint8_t combined_attrs_fwb(CPUARMState *env,
+                                  ARMCacheAttrs s1, ARMCacheAttrs s2)
+{
+    switch (s2.attrs) {
+    case 7:
+        /* Use stage 1 attributes */
+        return s1.attrs;
+    case 6:
+        /*
+         * Force Normal Write-Back. Note that if S1 is Normal cacheable
+         * then we take the allocation hints from it; otherwise it is
+         * RW allocate, non-transient.
+         */
+        if ((s1.attrs & 0xf0) == 0) {
+            /* S1 is Device */
+            return 0xff;
+        }
+        /* Need to check the Inner and Outer nibbles separately */
+        return force_cacheattr_nibble_wb(s1.attrs & 0xf) |
+            force_cacheattr_nibble_wb(s1.attrs >> 4) << 4;
+    case 5:
+        /* If S1 attrs are Device, use them; otherwise Normal Non-cacheable */
+        if ((s1.attrs & 0xf0) == 0) {
+            return s1.attrs;
+        }
+        return 0x44;
+    case 0 ... 3:
+        /* Force Device, of subtype specified by S2 */
+        return s2.attrs << 2;
+    default:
+        /*
+         * RESERVED values (including RES0 descriptor bit [5] being nonzero);
+         * arbitrarily force Device.
+         */
+        return 0;
+    }
+}
+
+/*
+ * Combine S1 and S2 cacheability/shareability attributes, per D4.5.4
+ * and CombineS1S2Desc()
+ *
+ * @env:     CPUARMState
+ * @s1:      Attributes from stage 1 walk
+ * @s2:      Attributes from stage 2 walk
+ */
+static ARMCacheAttrs combine_cacheattrs(CPUARMState *env,
+                                        ARMCacheAttrs s1, ARMCacheAttrs s2)
+{
+    ARMCacheAttrs ret;
+    bool tagged = false;
+
+    assert(s2.is_s2_format && !s1.is_s2_format);
+    ret.is_s2_format = false;
+
+    if (s1.attrs == 0xf0) {
+        tagged = true;
+        s1.attrs = 0xff;
+    }
+
+    /* Combine shareability attributes (table D4-43) */
+    if (s1.shareability == 2 || s2.shareability == 2) {
+        /* if either are outer-shareable, the result is outer-shareable */
+        ret.shareability = 2;
+    } else if (s1.shareability == 3 || s2.shareability == 3) {
+        /* if either are inner-shareable, the result is inner-shareable */
+        ret.shareability = 3;
+    } else {
+        /* both non-shareable */
+        ret.shareability = 0;
+    }
+
+    /* Combine memory type and cacheability attributes */
+    if (arm_hcr_el2_eff(env) & HCR_FWB) {
+        ret.attrs = combined_attrs_fwb(env, s1, s2);
+    } else {
+        ret.attrs = combined_attrs_nofwb(env, s1, s2);
+    }
+
+    /*
+     * Any location for which the resultant memory type is any
+     * type of Device memory is always treated as Outer Shareable.
+     * Any location for which the resultant memory type is Normal
+     * Inner Non-cacheable, Outer Non-cacheable is always treated
+     * as Outer Shareable.
+     * TODO: FEAT_XS adds another value (0x40) also meaning iNCoNC
+     */
+    if ((ret.attrs & 0xf0) == 0 || ret.attrs == 0x44) {
+        ret.shareability = 2;
+    }
+
+    /* TODO: CombineS1S2Desc does not consider transient, only WB, RWA. */
+    if (tagged && ret.attrs == 0xff) {
+        ret.attrs = 0xf0;
+    }
+
+    return ret;
+}
+
 /**
  * get_phys_addr - get the physical address for this virtual address
  *
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index dd6fb93f33..b2dfe489bb 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -20,9 +20,6 @@ bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx);
 bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx);
 uint64_t regime_ttbr(CPUARMState *env, ARMMMUIdx mmu_idx, int ttbrn);
 
-ARMCacheAttrs combine_cacheattrs(CPUARMState *env,
-                                 ARMCacheAttrs s1, ARMCacheAttrs s2);
-
 int ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx,
                   int ap, int domain_prot);
 int simple_ap_to_rw_prot_is_user(int ap, bool is_user);

From 3283222acd7579cdbc338360fa6e611027f3c8dc Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:51 +0100
Subject: [PATCH 874/935] target/arm: Move get_phys_addr_lpae to ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-16-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 416 +-------------------------------------------
 target/arm/ptw.c    | 411 +++++++++++++++++++++++++++++++++++++++++++
 target/arm/ptw.h    |  10 ++
 3 files changed, 429 insertions(+), 408 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index dab485e64a..7de815fe98 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10652,7 +10652,7 @@ int simple_ap_to_rw_prot_is_user(int ap, bool is_user)
  * @xn:      XN (execute-never) bits
  * @s1_is_el0: true if this is S2 of an S1+2 walk for EL0
  */
-static int get_S2prot(CPUARMState *env, int s2ap, int xn, bool s1_is_el0)
+int get_S2prot(CPUARMState *env, int s2ap, int xn, bool s1_is_el0)
 {
     int prot = 0;
 
@@ -10703,8 +10703,8 @@ static int get_S2prot(CPUARMState *env, int s2ap, int xn, bool s1_is_el0)
  * @xn:      XN (execute-never) bit
  * @pxn:     PXN (privileged execute-never) bit
  */
-static int get_S1prot(CPUARMState *env, ARMMMUIdx mmu_idx, bool is_aa64,
-                      int ap, int ns, int xn, int pxn)
+int get_S1prot(CPUARMState *env, ARMMMUIdx mmu_idx, bool is_aa64,
+               int ap, int ns, int xn, int pxn)
 {
     bool is_user = regime_is_user(env, mmu_idx);
     int prot_rw, user_rw;
@@ -10919,8 +10919,8 @@ uint64_t arm_ldq_ptw(CPUState *cs, hwaddr addr, bool is_secure,
  * Returns true if the suggested S2 translation parameters are OK and
  * false otherwise.
  */
-static bool check_s2_mmu_setup(ARMCPU *cpu, bool is_aa64, int level,
-                               int inputsize, int stride, int outputsize)
+bool check_s2_mmu_setup(ARMCPU *cpu, bool is_aa64, int level,
+                        int inputsize, int stride, int outputsize)
 {
     const int grainsize = stride + 3;
     int startsizecheck;
@@ -10980,7 +10980,7 @@ static bool check_s2_mmu_setup(ARMCPU *cpu, bool is_aa64, int level,
 #endif /* !CONFIG_USER_ONLY */
 
 /* This mapping is common between ID_AA64MMFR0.PARANGE and TCR_ELx.{I}PS. */
-static const uint8_t pamax_map[] = {
+const uint8_t pamax_map[] = {
     [0] = 32,
     [1] = 36,
     [2] = 40,
@@ -11159,8 +11159,8 @@ ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va,
 }
 
 #ifndef CONFIG_USER_ONLY
-static ARMVAParameters aa32_va_parameters(CPUARMState *env, uint32_t va,
-                                          ARMMMUIdx mmu_idx)
+ARMVAParameters aa32_va_parameters(CPUARMState *env, uint32_t va,
+                                   ARMMMUIdx mmu_idx)
 {
     uint64_t tcr = regime_tcr(env, mmu_idx)->raw_tcr;
     uint32_t el = regime_el(env, mmu_idx);
@@ -11223,406 +11223,6 @@ static ARMVAParameters aa32_va_parameters(CPUARMState *env, uint32_t va,
     };
 }
 
-/**
- * get_phys_addr_lpae: perform one stage of page table walk, LPAE format
- *
- * Returns false if the translation was successful. Otherwise, phys_ptr, attrs,
- * prot and page_size may not be filled in, and the populated fsr value provides
- * information on why the translation aborted, in the format of a long-format
- * DFSR/IFSR fault register, with the following caveats:
- *  * the WnR bit is never set (the caller must do this).
- *
- * @env: CPUARMState
- * @address: virtual address to get physical address for
- * @access_type: MMU_DATA_LOAD, MMU_DATA_STORE or MMU_INST_FETCH
- * @mmu_idx: MMU index indicating required translation regime
- * @s1_is_el0: if @mmu_idx is ARMMMUIdx_Stage2 (so this is a stage 2 page table
- *             walk), must be true if this is stage 2 of a stage 1+2 walk for an
- *             EL0 access). If @mmu_idx is anything else, @s1_is_el0 is ignored.
- * @phys_ptr: set to the physical address corresponding to the virtual address
- * @attrs: set to the memory transaction attributes to use
- * @prot: set to the permissions for the page containing phys_ptr
- * @page_size_ptr: set to the size of the page containing phys_ptr
- * @fi: set to fault info if the translation fails
- * @cacheattrs: (if non-NULL) set to the cacheability/shareability attributes
- */
-bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
-                        MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                        bool s1_is_el0,
-                        hwaddr *phys_ptr, MemTxAttrs *txattrs, int *prot,
-                        target_ulong *page_size_ptr,
-                        ARMMMUFaultInfo *fi, ARMCacheAttrs *cacheattrs)
-{
-    ARMCPU *cpu = env_archcpu(env);
-    CPUState *cs = CPU(cpu);
-    /* Read an LPAE long-descriptor translation table. */
-    ARMFaultType fault_type = ARMFault_Translation;
-    uint32_t level;
-    ARMVAParameters param;
-    uint64_t ttbr;
-    hwaddr descaddr, indexmask, indexmask_grainsize;
-    uint32_t tableattrs;
-    target_ulong page_size;
-    uint32_t attrs;
-    int32_t stride;
-    int addrsize, inputsize, outputsize;
-    TCR *tcr = regime_tcr(env, mmu_idx);
-    int ap, ns, xn, pxn;
-    uint32_t el = regime_el(env, mmu_idx);
-    uint64_t descaddrmask;
-    bool aarch64 = arm_el_is_aa64(env, el);
-    bool guarded = false;
-
-    /* TODO: This code does not support shareability levels. */
-    if (aarch64) {
-        int ps;
-
-        param = aa64_va_parameters(env, address, mmu_idx,
-                                   access_type != MMU_INST_FETCH);
-        level = 0;
-
-        /*
-         * If TxSZ is programmed to a value larger than the maximum,
-         * or smaller than the effective minimum, it is IMPLEMENTATION
-         * DEFINED whether we behave as if the field were programmed
-         * within bounds, or if a level 0 Translation fault is generated.
-         *
-         * With FEAT_LVA, fault on less than minimum becomes required,
-         * so our choice is to always raise the fault.
-         */
-        if (param.tsz_oob) {
-            fault_type = ARMFault_Translation;
-            goto do_fault;
-        }
-
-        addrsize = 64 - 8 * param.tbi;
-        inputsize = 64 - param.tsz;
-
-        /*
-         * Bound PS by PARANGE to find the effective output address size.
-         * ID_AA64MMFR0 is a read-only register so values outside of the
-         * supported mappings can be considered an implementation error.
-         */
-        ps = FIELD_EX64(cpu->isar.id_aa64mmfr0, ID_AA64MMFR0, PARANGE);
-        ps = MIN(ps, param.ps);
-        assert(ps < ARRAY_SIZE(pamax_map));
-        outputsize = pamax_map[ps];
-    } else {
-        param = aa32_va_parameters(env, address, mmu_idx);
-        level = 1;
-        addrsize = (mmu_idx == ARMMMUIdx_Stage2 ? 40 : 32);
-        inputsize = addrsize - param.tsz;
-        outputsize = 40;
-    }
-
-    /*
-     * We determined the region when collecting the parameters, but we
-     * have not yet validated that the address is valid for the region.
-     * Extract the top bits and verify that they all match select.
-     *
-     * For aa32, if inputsize == addrsize, then we have selected the
-     * region by exclusion in aa32_va_parameters and there is no more
-     * validation to do here.
-     */
-    if (inputsize < addrsize) {
-        target_ulong top_bits = sextract64(address, inputsize,
-                                           addrsize - inputsize);
-        if (-top_bits != param.select) {
-            /* The gap between the two regions is a Translation fault */
-            fault_type = ARMFault_Translation;
-            goto do_fault;
-        }
-    }
-
-    if (param.using64k) {
-        stride = 13;
-    } else if (param.using16k) {
-        stride = 11;
-    } else {
-        stride = 9;
-    }
-
-    /* Note that QEMU ignores shareability and cacheability attributes,
-     * so we don't need to do anything with the SH, ORGN, IRGN fields
-     * in the TTBCR.  Similarly, TTBCR:A1 selects whether we get the
-     * ASID from TTBR0 or TTBR1, but QEMU's TLB doesn't currently
-     * implement any ASID-like capability so we can ignore it (instead
-     * we will always flush the TLB any time the ASID is changed).
-     */
-    ttbr = regime_ttbr(env, mmu_idx, param.select);
-
-    /* Here we should have set up all the parameters for the translation:
-     * inputsize, ttbr, epd, stride, tbi
-     */
-
-    if (param.epd) {
-        /* Translation table walk disabled => Translation fault on TLB miss
-         * Note: This is always 0 on 64-bit EL2 and EL3.
-         */
-        goto do_fault;
-    }
-
-    if (mmu_idx != ARMMMUIdx_Stage2 && mmu_idx != ARMMMUIdx_Stage2_S) {
-        /* The starting level depends on the virtual address size (which can
-         * be up to 48 bits) and the translation granule size. It indicates
-         * the number of strides (stride bits at a time) needed to
-         * consume the bits of the input address. In the pseudocode this is:
-         *  level = 4 - RoundUp((inputsize - grainsize) / stride)
-         * where their 'inputsize' is our 'inputsize', 'grainsize' is
-         * our 'stride + 3' and 'stride' is our 'stride'.
-         * Applying the usual "rounded up m/n is (m+n-1)/n" and simplifying:
-         * = 4 - (inputsize - stride - 3 + stride - 1) / stride
-         * = 4 - (inputsize - 4) / stride;
-         */
-        level = 4 - (inputsize - 4) / stride;
-    } else {
-        /* For stage 2 translations the starting level is specified by the
-         * VTCR_EL2.SL0 field (whose interpretation depends on the page size)
-         */
-        uint32_t sl0 = extract32(tcr->raw_tcr, 6, 2);
-        uint32_t sl2 = extract64(tcr->raw_tcr, 33, 1);
-        uint32_t startlevel;
-        bool ok;
-
-        /* SL2 is RES0 unless DS=1 & 4kb granule. */
-        if (param.ds && stride == 9 && sl2) {
-            if (sl0 != 0) {
-                level = 0;
-                fault_type = ARMFault_Translation;
-                goto do_fault;
-            }
-            startlevel = -1;
-        } else if (!aarch64 || stride == 9) {
-            /* AArch32 or 4KB pages */
-            startlevel = 2 - sl0;
-
-            if (cpu_isar_feature(aa64_st, cpu)) {
-                startlevel &= 3;
-            }
-        } else {
-            /* 16KB or 64KB pages */
-            startlevel = 3 - sl0;
-        }
-
-        /* Check that the starting level is valid. */
-        ok = check_s2_mmu_setup(cpu, aarch64, startlevel,
-                                inputsize, stride, outputsize);
-        if (!ok) {
-            fault_type = ARMFault_Translation;
-            goto do_fault;
-        }
-        level = startlevel;
-    }
-
-    indexmask_grainsize = MAKE_64BIT_MASK(0, stride + 3);
-    indexmask = MAKE_64BIT_MASK(0, inputsize - (stride * (4 - level)));
-
-    /* Now we can extract the actual base address from the TTBR */
-    descaddr = extract64(ttbr, 0, 48);
-
-    /*
-     * For FEAT_LPA and PS=6, bits [51:48] of descaddr are in [5:2] of TTBR.
-     *
-     * Otherwise, if the base address is out of range, raise AddressSizeFault.
-     * In the pseudocode, this is !IsZero(baseregister<47:outputsize>),
-     * but we've just cleared the bits above 47, so simplify the test.
-     */
-    if (outputsize > 48) {
-        descaddr |= extract64(ttbr, 2, 4) << 48;
-    } else if (descaddr >> outputsize) {
-        level = 0;
-        fault_type = ARMFault_AddressSize;
-        goto do_fault;
-    }
-
-    /*
-     * We rely on this masking to clear the RES0 bits at the bottom of the TTBR
-     * and also to mask out CnP (bit 0) which could validly be non-zero.
-     */
-    descaddr &= ~indexmask;
-
-    /*
-     * For AArch32, the address field in the descriptor goes up to bit 39
-     * for both v7 and v8.  However, for v8 the SBZ bits [47:40] must be 0
-     * or an AddressSize fault is raised.  So for v8 we extract those SBZ
-     * bits as part of the address, which will be checked via outputsize.
-     * For AArch64, the address field goes up to bit 47, or 49 with FEAT_LPA2;
-     * the highest bits of a 52-bit output are placed elsewhere.
-     */
-    if (param.ds) {
-        descaddrmask = MAKE_64BIT_MASK(0, 50);
-    } else if (arm_feature(env, ARM_FEATURE_V8)) {
-        descaddrmask = MAKE_64BIT_MASK(0, 48);
-    } else {
-        descaddrmask = MAKE_64BIT_MASK(0, 40);
-    }
-    descaddrmask &= ~indexmask_grainsize;
-
-    /* Secure accesses start with the page table in secure memory and
-     * can be downgraded to non-secure at any step. Non-secure accesses
-     * remain non-secure. We implement this by just ORing in the NSTable/NS
-     * bits at each step.
-     */
-    tableattrs = regime_is_secure(env, mmu_idx) ? 0 : (1 << 4);
-    for (;;) {
-        uint64_t descriptor;
-        bool nstable;
-
-        descaddr |= (address >> (stride * (4 - level))) & indexmask;
-        descaddr &= ~7ULL;
-        nstable = extract32(tableattrs, 4, 1);
-        descriptor = arm_ldq_ptw(cs, descaddr, !nstable, mmu_idx, fi);
-        if (fi->type != ARMFault_None) {
-            goto do_fault;
-        }
-
-        if (!(descriptor & 1) ||
-            (!(descriptor & 2) && (level == 3))) {
-            /* Invalid, or the Reserved level 3 encoding */
-            goto do_fault;
-        }
-
-        descaddr = descriptor & descaddrmask;
-
-        /*
-         * For FEAT_LPA and PS=6, bits [51:48] of descaddr are in [15:12]
-         * of descriptor.  For FEAT_LPA2 and effective DS, bits [51:50] of
-         * descaddr are in [9:8].  Otherwise, if descaddr is out of range,
-         * raise AddressSizeFault.
-         */
-        if (outputsize > 48) {
-            if (param.ds) {
-                descaddr |= extract64(descriptor, 8, 2) << 50;
-            } else {
-                descaddr |= extract64(descriptor, 12, 4) << 48;
-            }
-        } else if (descaddr >> outputsize) {
-            fault_type = ARMFault_AddressSize;
-            goto do_fault;
-        }
-
-        if ((descriptor & 2) && (level < 3)) {
-            /* Table entry. The top five bits are attributes which may
-             * propagate down through lower levels of the table (and
-             * which are all arranged so that 0 means "no effect", so
-             * we can gather them up by ORing in the bits at each level).
-             */
-            tableattrs |= extract64(descriptor, 59, 5);
-            level++;
-            indexmask = indexmask_grainsize;
-            continue;
-        }
-        /*
-         * Block entry at level 1 or 2, or page entry at level 3.
-         * These are basically the same thing, although the number
-         * of bits we pull in from the vaddr varies. Note that although
-         * descaddrmask masks enough of the low bits of the descriptor
-         * to give a correct page or table address, the address field
-         * in a block descriptor is smaller; so we need to explicitly
-         * clear the lower bits here before ORing in the low vaddr bits.
-         */
-        page_size = (1ULL << ((stride * (4 - level)) + 3));
-        descaddr &= ~(page_size - 1);
-        descaddr |= (address & (page_size - 1));
-        /* Extract attributes from the descriptor */
-        attrs = extract64(descriptor, 2, 10)
-            | (extract64(descriptor, 52, 12) << 10);
-
-        if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
-            /* Stage 2 table descriptors do not include any attribute fields */
-            break;
-        }
-        /* Merge in attributes from table descriptors */
-        attrs |= nstable << 3; /* NS */
-        guarded = extract64(descriptor, 50, 1);  /* GP */
-        if (param.hpd) {
-            /* HPD disables all the table attributes except NSTable.  */
-            break;
-        }
-        attrs |= extract32(tableattrs, 0, 2) << 11;     /* XN, PXN */
-        /* The sense of AP[1] vs APTable[0] is reversed, as APTable[0] == 1
-         * means "force PL1 access only", which means forcing AP[1] to 0.
-         */
-        attrs &= ~(extract32(tableattrs, 2, 1) << 4);   /* !APT[0] => AP[1] */
-        attrs |= extract32(tableattrs, 3, 1) << 5;      /* APT[1] => AP[2] */
-        break;
-    }
-    /* Here descaddr is the final physical address, and attributes
-     * are all in attrs.
-     */
-    fault_type = ARMFault_AccessFlag;
-    if ((attrs & (1 << 8)) == 0) {
-        /* Access flag */
-        goto do_fault;
-    }
-
-    ap = extract32(attrs, 4, 2);
-
-    if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
-        ns = mmu_idx == ARMMMUIdx_Stage2;
-        xn = extract32(attrs, 11, 2);
-        *prot = get_S2prot(env, ap, xn, s1_is_el0);
-    } else {
-        ns = extract32(attrs, 3, 1);
-        xn = extract32(attrs, 12, 1);
-        pxn = extract32(attrs, 11, 1);
-        *prot = get_S1prot(env, mmu_idx, aarch64, ap, ns, xn, pxn);
-    }
-
-    fault_type = ARMFault_Permission;
-    if (!(*prot & (1 << access_type))) {
-        goto do_fault;
-    }
-
-    if (ns) {
-        /* The NS bit will (as required by the architecture) have no effect if
-         * the CPU doesn't support TZ or this is a non-secure translation
-         * regime, because the attribute will already be non-secure.
-         */
-        txattrs->secure = false;
-    }
-    /* When in aarch64 mode, and BTI is enabled, remember GP in the IOTLB.  */
-    if (aarch64 && guarded && cpu_isar_feature(aa64_bti, cpu)) {
-        arm_tlb_bti_gp(txattrs) = true;
-    }
-
-    if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
-        cacheattrs->is_s2_format = true;
-        cacheattrs->attrs = extract32(attrs, 0, 4);
-    } else {
-        /* Index into MAIR registers for cache attributes */
-        uint8_t attrindx = extract32(attrs, 0, 3);
-        uint64_t mair = env->cp15.mair_el[regime_el(env, mmu_idx)];
-        assert(attrindx <= 7);
-        cacheattrs->is_s2_format = false;
-        cacheattrs->attrs = extract64(mair, attrindx * 8, 8);
-    }
-
-    /*
-     * For FEAT_LPA2 and effective DS, the SH field in the attributes
-     * was re-purposed for output address bits.  The SH attribute in
-     * that case comes from TCR_ELx, which we extracted earlier.
-     */
-    if (param.ds) {
-        cacheattrs->shareability = param.sh;
-    } else {
-        cacheattrs->shareability = extract32(attrs, 6, 2);
-    }
-
-    *phys_ptr = descaddr;
-    *page_size_ptr = page_size;
-    return false;
-
-do_fault:
-    fi->type = fault_type;
-    fi->level = level;
-    /* Tag the error as S2 for failed S1 PTW at S2 or ordinary S2.  */
-    fi->stage2 = fi->s1ptw || (mmu_idx == ARMMMUIdx_Stage2 ||
-                               mmu_idx == ARMMMUIdx_Stage2_S);
-    fi->s1ns = mmu_idx == ARMMMUIdx_Stage2;
-    return true;
-}
-
 hwaddr arm_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr,
                                          MemTxAttrs *attrs)
 {
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index f2ca2bb8fe..cbccf91b13 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -314,6 +314,417 @@ do_fault:
     return true;
 }
 
+/**
+ * get_phys_addr_lpae: perform one stage of page table walk, LPAE format
+ *
+ * Returns false if the translation was successful. Otherwise, phys_ptr,
+ * attrs, prot and page_size may not be filled in, and the populated fsr
+ * value provides information on why the translation aborted, in the format
+ * of a long-format DFSR/IFSR fault register, with the following caveat:
+ * the WnR bit is never set (the caller must do this).
+ *
+ * @env: CPUARMState
+ * @address: virtual address to get physical address for
+ * @access_type: MMU_DATA_LOAD, MMU_DATA_STORE or MMU_INST_FETCH
+ * @mmu_idx: MMU index indicating required translation regime
+ * @s1_is_el0: if @mmu_idx is ARMMMUIdx_Stage2 (so this is a stage 2 page
+ *             table walk), must be true if this is stage 2 of a stage 1+2
+ *             walk for an EL0 access. If @mmu_idx is anything else,
+ *             @s1_is_el0 is ignored.
+ * @phys_ptr: set to the physical address corresponding to the virtual address
+ * @attrs: set to the memory transaction attributes to use
+ * @prot: set to the permissions for the page containing phys_ptr
+ * @page_size_ptr: set to the size of the page containing phys_ptr
+ * @fi: set to fault info if the translation fails
+ * @cacheattrs: (if non-NULL) set to the cacheability/shareability attributes
+ */
+bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
+                        MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                        bool s1_is_el0,
+                        hwaddr *phys_ptr, MemTxAttrs *txattrs, int *prot,
+                        target_ulong *page_size_ptr,
+                        ARMMMUFaultInfo *fi, ARMCacheAttrs *cacheattrs)
+{
+    ARMCPU *cpu = env_archcpu(env);
+    CPUState *cs = CPU(cpu);
+    /* Read an LPAE long-descriptor translation table. */
+    ARMFaultType fault_type = ARMFault_Translation;
+    uint32_t level;
+    ARMVAParameters param;
+    uint64_t ttbr;
+    hwaddr descaddr, indexmask, indexmask_grainsize;
+    uint32_t tableattrs;
+    target_ulong page_size;
+    uint32_t attrs;
+    int32_t stride;
+    int addrsize, inputsize, outputsize;
+    TCR *tcr = regime_tcr(env, mmu_idx);
+    int ap, ns, xn, pxn;
+    uint32_t el = regime_el(env, mmu_idx);
+    uint64_t descaddrmask;
+    bool aarch64 = arm_el_is_aa64(env, el);
+    bool guarded = false;
+
+    /* TODO: This code does not support shareability levels. */
+    if (aarch64) {
+        int ps;
+
+        param = aa64_va_parameters(env, address, mmu_idx,
+                                   access_type != MMU_INST_FETCH);
+        level = 0;
+
+        /*
+         * If TxSZ is programmed to a value larger than the maximum,
+         * or smaller than the effective minimum, it is IMPLEMENTATION
+         * DEFINED whether we behave as if the field were programmed
+         * within bounds, or if a level 0 Translation fault is generated.
+         *
+         * With FEAT_LVA, fault on less than minimum becomes required,
+         * so our choice is to always raise the fault.
+         */
+        if (param.tsz_oob) {
+            fault_type = ARMFault_Translation;
+            goto do_fault;
+        }
+
+        addrsize = 64 - 8 * param.tbi;
+        inputsize = 64 - param.tsz;
+
+        /*
+         * Bound PS by PARANGE to find the effective output address size.
+         * ID_AA64MMFR0 is a read-only register so values outside of the
+         * supported mappings can be considered an implementation error.
+         */
+        ps = FIELD_EX64(cpu->isar.id_aa64mmfr0, ID_AA64MMFR0, PARANGE);
+        ps = MIN(ps, param.ps);
+        assert(ps < ARRAY_SIZE(pamax_map));
+        outputsize = pamax_map[ps];
+    } else {
+        param = aa32_va_parameters(env, address, mmu_idx);
+        level = 1;
+        addrsize = (mmu_idx == ARMMMUIdx_Stage2 ? 40 : 32);
+        inputsize = addrsize - param.tsz;
+        outputsize = 40;
+    }
+
+    /*
+     * We determined the region when collecting the parameters, but we
+     * have not yet validated that the address is valid for the region.
+     * Extract the top bits and verify that they all match select.
+     *
+     * For aa32, if inputsize == addrsize, then we have selected the
+     * region by exclusion in aa32_va_parameters and there is no more
+     * validation to do here.
+     */
+    if (inputsize < addrsize) {
+        target_ulong top_bits = sextract64(address, inputsize,
+                                           addrsize - inputsize);
+        if (-top_bits != param.select) {
+            /* The gap between the two regions is a Translation fault */
+            fault_type = ARMFault_Translation;
+            goto do_fault;
+        }
+    }
+
+    if (param.using64k) {
+        stride = 13;
+    } else if (param.using16k) {
+        stride = 11;
+    } else {
+        stride = 9;
+    }
+
+    /*
+     * Note that QEMU ignores shareability and cacheability attributes,
+     * so we don't need to do anything with the SH, ORGN, IRGN fields
+     * in the TTBCR.  Similarly, TTBCR:A1 selects whether we get the
+     * ASID from TTBR0 or TTBR1, but QEMU's TLB doesn't currently
+     * implement any ASID-like capability so we can ignore it (instead
+     * we will always flush the TLB any time the ASID is changed).
+     */
+    ttbr = regime_ttbr(env, mmu_idx, param.select);
+
+    /*
+     * Here we should have set up all the parameters for the translation:
+     * inputsize, ttbr, epd, stride, tbi
+     */
+
+    if (param.epd) {
+        /*
+         * Translation table walk disabled => Translation fault on TLB miss
+         * Note: This is always 0 on 64-bit EL2 and EL3.
+         */
+        goto do_fault;
+    }
+
+    if (mmu_idx != ARMMMUIdx_Stage2 && mmu_idx != ARMMMUIdx_Stage2_S) {
+        /*
+         * The starting level depends on the virtual address size (which can
+         * be up to 48 bits) and the translation granule size. It indicates
+         * the number of strides (stride bits at a time) needed to
+         * consume the bits of the input address. In the pseudocode this is:
+         *  level = 4 - RoundUp((inputsize - grainsize) / stride)
+         * where their 'inputsize' is our 'inputsize', 'grainsize' is
+         * our 'stride + 3' and 'stride' is our 'stride'.
+         * Applying the usual "rounded up m/n is (m+n-1)/n" and simplifying:
+         * = 4 - (inputsize - stride - 3 + stride - 1) / stride
+         * = 4 - (inputsize - 4) / stride;
+         */
+        level = 4 - (inputsize - 4) / stride;
+    } else {
+        /*
+         * For stage 2 translations the starting level is specified by the
+         * VTCR_EL2.SL0 field (whose interpretation depends on the page size)
+         */
+        uint32_t sl0 = extract32(tcr->raw_tcr, 6, 2);
+        uint32_t sl2 = extract64(tcr->raw_tcr, 33, 1);
+        uint32_t startlevel;
+        bool ok;
+
+        /* SL2 is RES0 unless DS=1 & 4kb granule. */
+        if (param.ds && stride == 9 && sl2) {
+            if (sl0 != 0) {
+                level = 0;
+                fault_type = ARMFault_Translation;
+                goto do_fault;
+            }
+            startlevel = -1;
+        } else if (!aarch64 || stride == 9) {
+            /* AArch32 or 4KB pages */
+            startlevel = 2 - sl0;
+
+            if (cpu_isar_feature(aa64_st, cpu)) {
+                startlevel &= 3;
+            }
+        } else {
+            /* 16KB or 64KB pages */
+            startlevel = 3 - sl0;
+        }
+
+        /* Check that the starting level is valid. */
+        ok = check_s2_mmu_setup(cpu, aarch64, startlevel,
+                                inputsize, stride, outputsize);
+        if (!ok) {
+            fault_type = ARMFault_Translation;
+            goto do_fault;
+        }
+        level = startlevel;
+    }
+
+    indexmask_grainsize = MAKE_64BIT_MASK(0, stride + 3);
+    indexmask = MAKE_64BIT_MASK(0, inputsize - (stride * (4 - level)));
+
+    /* Now we can extract the actual base address from the TTBR */
+    descaddr = extract64(ttbr, 0, 48);
+
+    /*
+     * For FEAT_LPA and PS=6, bits [51:48] of descaddr are in [5:2] of TTBR.
+     *
+     * Otherwise, if the base address is out of range, raise AddressSizeFault.
+     * In the pseudocode, this is !IsZero(baseregister<47:outputsize>),
+     * but we've just cleared the bits above 47, so simplify the test.
+     */
+    if (outputsize > 48) {
+        descaddr |= extract64(ttbr, 2, 4) << 48;
+    } else if (descaddr >> outputsize) {
+        level = 0;
+        fault_type = ARMFault_AddressSize;
+        goto do_fault;
+    }
+
+    /*
+     * We rely on this masking to clear the RES0 bits at the bottom of the TTBR
+     * and also to mask out CnP (bit 0) which could validly be non-zero.
+     */
+    descaddr &= ~indexmask;
+
+    /*
+     * For AArch32, the address field in the descriptor goes up to bit 39
+     * for both v7 and v8.  However, for v8 the SBZ bits [47:40] must be 0
+     * or an AddressSize fault is raised.  So for v8 we extract those SBZ
+     * bits as part of the address, which will be checked via outputsize.
+     * For AArch64, the address field goes up to bit 47, or 49 with FEAT_LPA2;
+     * the highest bits of a 52-bit output are placed elsewhere.
+     */
+    if (param.ds) {
+        descaddrmask = MAKE_64BIT_MASK(0, 50);
+    } else if (arm_feature(env, ARM_FEATURE_V8)) {
+        descaddrmask = MAKE_64BIT_MASK(0, 48);
+    } else {
+        descaddrmask = MAKE_64BIT_MASK(0, 40);
+    }
+    descaddrmask &= ~indexmask_grainsize;
+
+    /*
+     * Secure accesses start with the page table in secure memory and
+     * can be downgraded to non-secure at any step. Non-secure accesses
+     * remain non-secure. We implement this by just ORing in the NSTable/NS
+     * bits at each step.
+     */
+    tableattrs = regime_is_secure(env, mmu_idx) ? 0 : (1 << 4);
+    for (;;) {
+        uint64_t descriptor;
+        bool nstable;
+
+        descaddr |= (address >> (stride * (4 - level))) & indexmask;
+        descaddr &= ~7ULL;
+        nstable = extract32(tableattrs, 4, 1);
+        descriptor = arm_ldq_ptw(cs, descaddr, !nstable, mmu_idx, fi);
+        if (fi->type != ARMFault_None) {
+            goto do_fault;
+        }
+
+        if (!(descriptor & 1) ||
+            (!(descriptor & 2) && (level == 3))) {
+            /* Invalid, or the Reserved level 3 encoding */
+            goto do_fault;
+        }
+
+        descaddr = descriptor & descaddrmask;
+
+        /*
+         * For FEAT_LPA and PS=6, bits [51:48] of descaddr are in [15:12]
+         * of descriptor.  For FEAT_LPA2 and effective DS, bits [51:50] of
+         * descaddr are in [9:8].  Otherwise, if descaddr is out of range,
+         * raise AddressSizeFault.
+         */
+        if (outputsize > 48) {
+            if (param.ds) {
+                descaddr |= extract64(descriptor, 8, 2) << 50;
+            } else {
+                descaddr |= extract64(descriptor, 12, 4) << 48;
+            }
+        } else if (descaddr >> outputsize) {
+            fault_type = ARMFault_AddressSize;
+            goto do_fault;
+        }
+
+        if ((descriptor & 2) && (level < 3)) {
+            /*
+             * Table entry. The top five bits are attributes which may
+             * propagate down through lower levels of the table (and
+             * which are all arranged so that 0 means "no effect", so
+             * we can gather them up by ORing in the bits at each level).
+             */
+            tableattrs |= extract64(descriptor, 59, 5);
+            level++;
+            indexmask = indexmask_grainsize;
+            continue;
+        }
+        /*
+         * Block entry at level 1 or 2, or page entry at level 3.
+         * These are basically the same thing, although the number
+         * of bits we pull in from the vaddr varies. Note that although
+         * descaddrmask masks enough of the low bits of the descriptor
+         * to give a correct page or table address, the address field
+         * in a block descriptor is smaller; so we need to explicitly
+         * clear the lower bits here before ORing in the low vaddr bits.
+         */
+        page_size = (1ULL << ((stride * (4 - level)) + 3));
+        descaddr &= ~(page_size - 1);
+        descaddr |= (address & (page_size - 1));
+        /* Extract attributes from the descriptor */
+        attrs = extract64(descriptor, 2, 10)
+            | (extract64(descriptor, 52, 12) << 10);
+
+        if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
+            /* Stage 2 table descriptors do not include any attribute fields */
+            break;
+        }
+        /* Merge in attributes from table descriptors */
+        attrs |= nstable << 3; /* NS */
+        guarded = extract64(descriptor, 50, 1);  /* GP */
+        if (param.hpd) {
+            /* HPD disables all the table attributes except NSTable.  */
+            break;
+        }
+        attrs |= extract32(tableattrs, 0, 2) << 11;     /* XN, PXN */
+        /*
+         * The sense of AP[1] vs APTable[0] is reversed, as APTable[0] == 1
+         * means "force PL1 access only", which means forcing AP[1] to 0.
+         */
+        attrs &= ~(extract32(tableattrs, 2, 1) << 4);   /* !APT[0] => AP[1] */
+        attrs |= extract32(tableattrs, 3, 1) << 5;      /* APT[1] => AP[2] */
+        break;
+    }
+    /*
+     * Here descaddr is the final physical address, and attributes
+     * are all in attrs.
+     */
+    fault_type = ARMFault_AccessFlag;
+    if ((attrs & (1 << 8)) == 0) {
+        /* Access flag */
+        goto do_fault;
+    }
+
+    ap = extract32(attrs, 4, 2);
+
+    if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
+        ns = mmu_idx == ARMMMUIdx_Stage2;
+        xn = extract32(attrs, 11, 2);
+        *prot = get_S2prot(env, ap, xn, s1_is_el0);
+    } else {
+        ns = extract32(attrs, 3, 1);
+        xn = extract32(attrs, 12, 1);
+        pxn = extract32(attrs, 11, 1);
+        *prot = get_S1prot(env, mmu_idx, aarch64, ap, ns, xn, pxn);
+    }
+
+    fault_type = ARMFault_Permission;
+    if (!(*prot & (1 << access_type))) {
+        goto do_fault;
+    }
+
+    if (ns) {
+        /*
+         * The NS bit will (as required by the architecture) have no effect if
+         * the CPU doesn't support TZ or this is a non-secure translation
+         * regime, because the attribute will already be non-secure.
+         */
+        txattrs->secure = false;
+    }
+    /* When in aarch64 mode, and BTI is enabled, remember GP in the IOTLB.  */
+    if (aarch64 && guarded && cpu_isar_feature(aa64_bti, cpu)) {
+        arm_tlb_bti_gp(txattrs) = true;
+    }
+
+    if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
+        cacheattrs->is_s2_format = true;
+        cacheattrs->attrs = extract32(attrs, 0, 4);
+    } else {
+        /* Index into MAIR registers for cache attributes */
+        uint8_t attrindx = extract32(attrs, 0, 3);
+        uint64_t mair = env->cp15.mair_el[regime_el(env, mmu_idx)];
+        assert(attrindx <= 7);
+        cacheattrs->is_s2_format = false;
+        cacheattrs->attrs = extract64(mair, attrindx * 8, 8);
+    }
+
+    /*
+     * For FEAT_LPA2 and effective DS, the SH field in the attributes
+     * was re-purposed for output address bits.  The SH attribute in
+     * that case comes from TCR_ELx, which we extracted earlier.
+     */
+    if (param.ds) {
+        cacheattrs->shareability = param.sh;
+    } else {
+        cacheattrs->shareability = extract32(attrs, 6, 2);
+    }
+
+    *phys_ptr = descaddr;
+    *page_size_ptr = page_size;
+    return false;
+
+do_fault:
+    fi->type = fault_type;
+    fi->level = level;
+    /* Tag the error as S2 for failed S1 PTW at S2 or ordinary S2.  */
+    fi->stage2 = fi->s1ptw || (mmu_idx == ARMMMUIdx_Stage2 ||
+                               mmu_idx == ARMMMUIdx_Stage2_S);
+    fi->s1ns = mmu_idx == ARMMMUIdx_Stage2;
+    return true;
+}
+
 static bool get_phys_addr_pmsav5(CPUARMState *env, uint32_t address,
                                  MMUAccessType access_type, ARMMMUIdx mmu_idx,
                                  hwaddr *phys_ptr, int *prot,
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index b2dfe489bb..31744df664 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -11,6 +11,8 @@
 
 #ifndef CONFIG_USER_ONLY
 
+extern const uint8_t pamax_map[7];
+
 uint32_t arm_ldl_ptw(CPUState *cs, hwaddr addr, bool is_secure,
                      ARMMMUIdx mmu_idx, ARMMMUFaultInfo *fi);
 uint64_t arm_ldq_ptw(CPUState *cs, hwaddr addr, bool is_secure,
@@ -30,6 +32,14 @@ simple_ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap)
     return simple_ap_to_rw_prot_is_user(ap, regime_is_user(env, mmu_idx));
 }
 
+ARMVAParameters aa32_va_parameters(CPUARMState *env, uint32_t va,
+                                   ARMMMUIdx mmu_idx);
+bool check_s2_mmu_setup(ARMCPU *cpu, bool is_aa64, int level,
+                        int inputsize, int stride, int outputsize);
+int get_S2prot(CPUARMState *env, int s2ap, int xn, bool s1_is_el0);
+int get_S1prot(CPUARMState *env, ARMMMUIdx mmu_idx, bool is_aa64,
+               int ap, int ns, int xn, int pxn);
+
 bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
                         MMUAccessType access_type, ARMMMUIdx mmu_idx,
                         bool s1_is_el0,

From 11552bb0d95484c99233ec2f09c25261885d08ed Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:51 +0100
Subject: [PATCH 875/935] target/arm: Move arm_{ldl,ldq}_ptw to ptw.c

Move the ptw load functions, plus 3 common subroutines:
S1_ptw_translate, ptw_attrs_are_device, and regime_translation_big_endian.
This also allows get_phys_addr_lpae to become static again.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-17-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 141 --------------------------------------
 target/arm/ptw.c    | 160 ++++++++++++++++++++++++++++++++++++++++++--
 target/arm/ptw.h    |  13 ----
 3 files changed, 154 insertions(+), 160 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 7de815fe98..398bcd62ab 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10475,12 +10475,6 @@ bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx)
     return (regime_sctlr(env, mmu_idx) & SCTLR_M) == 0;
 }
 
-static inline bool regime_translation_big_endian(CPUARMState *env,
-                                                 ARMMMUIdx mmu_idx)
-{
-    return (regime_sctlr(env, mmu_idx) & SCTLR_EE) != 0;
-}
-
 /* Return the TTBR associated with this translation regime */
 uint64_t regime_ttbr(CPUARMState *env, ARMMMUIdx mmu_idx, int ttbrn)
 {
@@ -10773,141 +10767,6 @@ int get_S1prot(CPUARMState *env, ARMMMUIdx mmu_idx, bool is_aa64,
     return prot_rw | PAGE_EXEC;
 }
 
-static bool ptw_attrs_are_device(CPUARMState *env, ARMCacheAttrs cacheattrs)
-{
-    /*
-     * For an S1 page table walk, the stage 1 attributes are always
-     * some form of "this is Normal memory". The combined S1+S2
-     * attributes are therefore only Device if stage 2 specifies Device.
-     * With HCR_EL2.FWB == 0 this is when descriptor bits [5:4] are 0b00,
-     * ie when cacheattrs.attrs bits [3:2] are 0b00.
-     * With HCR_EL2.FWB == 1 this is when descriptor bit [4] is 0, ie
-     * when cacheattrs.attrs bit [2] is 0.
-     */
-    assert(cacheattrs.is_s2_format);
-    if (arm_hcr_el2_eff(env) & HCR_FWB) {
-        return (cacheattrs.attrs & 0x4) == 0;
-    } else {
-        return (cacheattrs.attrs & 0xc) == 0;
-    }
-}
-
-/* Translate a S1 pagetable walk through S2 if needed.  */
-static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
-                               hwaddr addr, bool *is_secure,
-                               ARMMMUFaultInfo *fi)
-{
-    if (arm_mmu_idx_is_stage1_of_2(mmu_idx) &&
-        !regime_translation_disabled(env, ARMMMUIdx_Stage2)) {
-        target_ulong s2size;
-        hwaddr s2pa;
-        int s2prot;
-        int ret;
-        ARMMMUIdx s2_mmu_idx = *is_secure ? ARMMMUIdx_Stage2_S
-                                          : ARMMMUIdx_Stage2;
-        ARMCacheAttrs cacheattrs = {};
-        MemTxAttrs txattrs = {};
-
-        ret = get_phys_addr_lpae(env, addr, MMU_DATA_LOAD, s2_mmu_idx, false,
-                                 &s2pa, &txattrs, &s2prot, &s2size, fi,
-                                 &cacheattrs);
-        if (ret) {
-            assert(fi->type != ARMFault_None);
-            fi->s2addr = addr;
-            fi->stage2 = true;
-            fi->s1ptw = true;
-            fi->s1ns = !*is_secure;
-            return ~0;
-        }
-        if ((arm_hcr_el2_eff(env) & HCR_PTW) &&
-            ptw_attrs_are_device(env, cacheattrs)) {
-            /*
-             * PTW set and S1 walk touched S2 Device memory:
-             * generate Permission fault.
-             */
-            fi->type = ARMFault_Permission;
-            fi->s2addr = addr;
-            fi->stage2 = true;
-            fi->s1ptw = true;
-            fi->s1ns = !*is_secure;
-            return ~0;
-        }
-
-        if (arm_is_secure_below_el3(env)) {
-            /* Check if page table walk is to secure or non-secure PA space. */
-            if (*is_secure) {
-                *is_secure = !(env->cp15.vstcr_el2.raw_tcr & VSTCR_SW);
-            } else {
-                *is_secure = !(env->cp15.vtcr_el2.raw_tcr & VTCR_NSW);
-            }
-        } else {
-            assert(!*is_secure);
-        }
-
-        addr = s2pa;
-    }
-    return addr;
-}
-
-/* All loads done in the course of a page table walk go through here. */
-uint32_t arm_ldl_ptw(CPUState *cs, hwaddr addr, bool is_secure,
-                     ARMMMUIdx mmu_idx, ARMMMUFaultInfo *fi)
-{
-    ARMCPU *cpu = ARM_CPU(cs);
-    CPUARMState *env = &cpu->env;
-    MemTxAttrs attrs = {};
-    MemTxResult result = MEMTX_OK;
-    AddressSpace *as;
-    uint32_t data;
-
-    addr = S1_ptw_translate(env, mmu_idx, addr, &is_secure, fi);
-    attrs.secure = is_secure;
-    as = arm_addressspace(cs, attrs);
-    if (fi->s1ptw) {
-        return 0;
-    }
-    if (regime_translation_big_endian(env, mmu_idx)) {
-        data = address_space_ldl_be(as, addr, attrs, &result);
-    } else {
-        data = address_space_ldl_le(as, addr, attrs, &result);
-    }
-    if (result == MEMTX_OK) {
-        return data;
-    }
-    fi->type = ARMFault_SyncExternalOnWalk;
-    fi->ea = arm_extabort_type(result);
-    return 0;
-}
-
-uint64_t arm_ldq_ptw(CPUState *cs, hwaddr addr, bool is_secure,
-                     ARMMMUIdx mmu_idx, ARMMMUFaultInfo *fi)
-{
-    ARMCPU *cpu = ARM_CPU(cs);
-    CPUARMState *env = &cpu->env;
-    MemTxAttrs attrs = {};
-    MemTxResult result = MEMTX_OK;
-    AddressSpace *as;
-    uint64_t data;
-
-    addr = S1_ptw_translate(env, mmu_idx, addr, &is_secure, fi);
-    attrs.secure = is_secure;
-    as = arm_addressspace(cs, attrs);
-    if (fi->s1ptw) {
-        return 0;
-    }
-    if (regime_translation_big_endian(env, mmu_idx)) {
-        data = address_space_ldq_be(as, addr, attrs, &result);
-    } else {
-        data = address_space_ldq_le(as, addr, attrs, &result);
-    }
-    if (result == MEMTX_OK) {
-        return data;
-    }
-    fi->type = ARMFault_SyncExternalOnWalk;
-    fi->ea = arm_extabort_type(result);
-    return 0;
-}
-
 /*
  * check_s2_mmu_setup
  * @cpu:        ARMCPU
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index cbccf91b13..e4b860d2ae 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -15,6 +15,154 @@
 #include "ptw.h"
 
 
+static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
+                               MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                               bool s1_is_el0, hwaddr *phys_ptr,
+                               MemTxAttrs *txattrs, int *prot,
+                               target_ulong *page_size_ptr,
+                               ARMMMUFaultInfo *fi, ARMCacheAttrs *cacheattrs)
+    __attribute__((nonnull));
+
+static bool regime_translation_big_endian(CPUARMState *env, ARMMMUIdx mmu_idx)
+{
+    return (regime_sctlr(env, mmu_idx) & SCTLR_EE) != 0;
+}
+
+static bool ptw_attrs_are_device(CPUARMState *env, ARMCacheAttrs cacheattrs)
+{
+    /*
+     * For an S1 page table walk, the stage 1 attributes are always
+     * some form of "this is Normal memory". The combined S1+S2
+     * attributes are therefore only Device if stage 2 specifies Device.
+     * With HCR_EL2.FWB == 0 this is when descriptor bits [5:4] are 0b00,
+     * ie when cacheattrs.attrs bits [3:2] are 0b00.
+     * With HCR_EL2.FWB == 1 this is when descriptor bit [4] is 0, ie
+     * when cacheattrs.attrs bit [2] is 0.
+     */
+    assert(cacheattrs.is_s2_format);
+    if (arm_hcr_el2_eff(env) & HCR_FWB) {
+        return (cacheattrs.attrs & 0x4) == 0;
+    } else {
+        return (cacheattrs.attrs & 0xc) == 0;
+    }
+}
+
+/* Translate a S1 pagetable walk through S2 if needed.  */
+static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
+                               hwaddr addr, bool *is_secure,
+                               ARMMMUFaultInfo *fi)
+{
+    if (arm_mmu_idx_is_stage1_of_2(mmu_idx) &&
+        !regime_translation_disabled(env, ARMMMUIdx_Stage2)) {
+        target_ulong s2size;
+        hwaddr s2pa;
+        int s2prot;
+        int ret;
+        ARMMMUIdx s2_mmu_idx = *is_secure ? ARMMMUIdx_Stage2_S
+                                          : ARMMMUIdx_Stage2;
+        ARMCacheAttrs cacheattrs = {};
+        MemTxAttrs txattrs = {};
+
+        ret = get_phys_addr_lpae(env, addr, MMU_DATA_LOAD, s2_mmu_idx, false,
+                                 &s2pa, &txattrs, &s2prot, &s2size, fi,
+                                 &cacheattrs);
+        if (ret) {
+            assert(fi->type != ARMFault_None);
+            fi->s2addr = addr;
+            fi->stage2 = true;
+            fi->s1ptw = true;
+            fi->s1ns = !*is_secure;
+            return ~0;
+        }
+        if ((arm_hcr_el2_eff(env) & HCR_PTW) &&
+            ptw_attrs_are_device(env, cacheattrs)) {
+            /*
+             * PTW set and S1 walk touched S2 Device memory:
+             * generate Permission fault.
+             */
+            fi->type = ARMFault_Permission;
+            fi->s2addr = addr;
+            fi->stage2 = true;
+            fi->s1ptw = true;
+            fi->s1ns = !*is_secure;
+            return ~0;
+        }
+
+        if (arm_is_secure_below_el3(env)) {
+            /* Check if page table walk is to secure or non-secure PA space. */
+            if (*is_secure) {
+                *is_secure = !(env->cp15.vstcr_el2.raw_tcr & VSTCR_SW);
+            } else {
+                *is_secure = !(env->cp15.vtcr_el2.raw_tcr & VTCR_NSW);
+            }
+        } else {
+            assert(!*is_secure);
+        }
+
+        addr = s2pa;
+    }
+    return addr;
+}
+
+/* All loads done in the course of a page table walk go through here. */
+static uint32_t arm_ldl_ptw(CPUState *cs, hwaddr addr, bool is_secure,
+                            ARMMMUIdx mmu_idx, ARMMMUFaultInfo *fi)
+{
+    ARMCPU *cpu = ARM_CPU(cs);
+    CPUARMState *env = &cpu->env;
+    MemTxAttrs attrs = {};
+    MemTxResult result = MEMTX_OK;
+    AddressSpace *as;
+    uint32_t data;
+
+    addr = S1_ptw_translate(env, mmu_idx, addr, &is_secure, fi);
+    attrs.secure = is_secure;
+    as = arm_addressspace(cs, attrs);
+    if (fi->s1ptw) {
+        return 0;
+    }
+    if (regime_translation_big_endian(env, mmu_idx)) {
+        data = address_space_ldl_be(as, addr, attrs, &result);
+    } else {
+        data = address_space_ldl_le(as, addr, attrs, &result);
+    }
+    if (result == MEMTX_OK) {
+        return data;
+    }
+    fi->type = ARMFault_SyncExternalOnWalk;
+    fi->ea = arm_extabort_type(result);
+    return 0;
+}
+
+static uint64_t arm_ldq_ptw(CPUState *cs, hwaddr addr, bool is_secure,
+                            ARMMMUIdx mmu_idx, ARMMMUFaultInfo *fi)
+{
+    ARMCPU *cpu = ARM_CPU(cs);
+    CPUARMState *env = &cpu->env;
+    MemTxAttrs attrs = {};
+    MemTxResult result = MEMTX_OK;
+    AddressSpace *as;
+    uint64_t data;
+
+    addr = S1_ptw_translate(env, mmu_idx, addr, &is_secure, fi);
+    attrs.secure = is_secure;
+    as = arm_addressspace(cs, attrs);
+    if (fi->s1ptw) {
+        return 0;
+    }
+    if (regime_translation_big_endian(env, mmu_idx)) {
+        data = address_space_ldq_be(as, addr, attrs, &result);
+    } else {
+        data = address_space_ldq_le(as, addr, attrs, &result);
+    }
+    if (result == MEMTX_OK) {
+        return data;
+    }
+    fi->type = ARMFault_SyncExternalOnWalk;
+    fi->ea = arm_extabort_type(result);
+    return 0;
+}
+
 static bool get_level1_table_address(CPUARMState *env, ARMMMUIdx mmu_idx,
                                      uint32_t *table, uint32_t address)
 {
@@ -338,12 +486,12 @@ do_fault:
  * @fi: set to fault info if the translation fails
  * @cacheattrs: (if non-NULL) set to the cacheability/shareability attributes
  */
-bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
-                        MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                        bool s1_is_el0,
-                        hwaddr *phys_ptr, MemTxAttrs *txattrs, int *prot,
-                        target_ulong *page_size_ptr,
-                        ARMMMUFaultInfo *fi, ARMCacheAttrs *cacheattrs)
+static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
+                               MMUAccessType access_type, ARMMMUIdx mmu_idx,
+                               bool s1_is_el0, hwaddr *phys_ptr,
+                               MemTxAttrs *txattrs, int *prot,
+                               target_ulong *page_size_ptr,
+                               ARMMMUFaultInfo *fi, ARMCacheAttrs *cacheattrs)
 {
     ARMCPU *cpu = env_archcpu(env);
     CPUState *cs = CPU(cpu);
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index 31744df664..28b8cb9fb8 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -13,11 +13,6 @@
 
 extern const uint8_t pamax_map[7];
 
-uint32_t arm_ldl_ptw(CPUState *cs, hwaddr addr, bool is_secure,
-                     ARMMMUIdx mmu_idx, ARMMMUFaultInfo *fi);
-uint64_t arm_ldq_ptw(CPUState *cs, hwaddr addr, bool is_secure,
-                     ARMMMUIdx mmu_idx, ARMMMUFaultInfo *fi);
-
 bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx);
 bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx);
 uint64_t regime_ttbr(CPUARMState *env, ARMMMUIdx mmu_idx, int ttbrn);
@@ -40,13 +35,5 @@ int get_S2prot(CPUARMState *env, int s2ap, int xn, bool s1_is_el0);
 int get_S1prot(CPUARMState *env, ARMMMUIdx mmu_idx, bool is_aa64,
                int ap, int ns, int xn, int pxn);
 
-bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
-                        MMUAccessType access_type, ARMMMUIdx mmu_idx,
-                        bool s1_is_el0,
-                        hwaddr *phys_ptr, MemTxAttrs *txattrs, int *prot,
-                        target_ulong *page_size_ptr,
-                        ARMMMUFaultInfo *fi, ARMCacheAttrs *cacheattrs)
-    __attribute__((nonnull));
-
 #endif /* !CONFIG_USER_ONLY */
 #endif /* TARGET_ARM_PTW_H */

From cd6bc4d51730e9cf47489029d078d18c3bcb3ae2 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:52 +0100
Subject: [PATCH 876/935] target/arm: Move {arm_s1_, }regime_using_lpae_format
 to tlb_helper.c

These functions are used for both page table walking and for
deciding what format in which to deliver exception results.
Since ptw.c is only present for system mode, put the functions
into tlb_helper.c.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-18-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c     | 24 ------------------------
 target/arm/tlb_helper.c | 26 ++++++++++++++++++++++++++
 2 files changed, 26 insertions(+), 24 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 398bcd62ab..d2b196ff3e 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10515,30 +10515,6 @@ ARMMMUIdx stage_1_mmu_idx(ARMMMUIdx mmu_idx)
 }
 #endif /* !CONFIG_USER_ONLY */
 
-/* Return true if the translation regime is using LPAE format page tables */
-bool regime_using_lpae_format(CPUARMState *env, ARMMMUIdx mmu_idx)
-{
-    int el = regime_el(env, mmu_idx);
-    if (el == 2 || arm_el_is_aa64(env, el)) {
-        return true;
-    }
-    if (arm_feature(env, ARM_FEATURE_LPAE)
-        && (regime_tcr(env, mmu_idx)->raw_tcr & TTBCR_EAE)) {
-        return true;
-    }
-    return false;
-}
-
-/* Returns true if the stage 1 translation regime is using LPAE format page
- * tables. Used when raising alignment exceptions, whose FSR changes depending
- * on whether the long or short descriptor format is in use. */
-bool arm_s1_regime_using_lpae_format(CPUARMState *env, ARMMMUIdx mmu_idx)
-{
-    mmu_idx = stage_1_mmu_idx(mmu_idx);
-
-    return regime_using_lpae_format(env, mmu_idx);
-}
-
 #ifndef CONFIG_USER_ONLY
 bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx)
 {
diff --git a/target/arm/tlb_helper.c b/target/arm/tlb_helper.c
index 6421e16202..7d8a86b3c4 100644
--- a/target/arm/tlb_helper.c
+++ b/target/arm/tlb_helper.c
@@ -11,6 +11,32 @@
 #include "exec/exec-all.h"
 #include "exec/helper-proto.h"
 
+
+/* Return true if the translation regime is using LPAE format page tables */
+bool regime_using_lpae_format(CPUARMState *env, ARMMMUIdx mmu_idx)
+{
+    int el = regime_el(env, mmu_idx);
+    if (el == 2 || arm_el_is_aa64(env, el)) {
+        return true;
+    }
+    if (arm_feature(env, ARM_FEATURE_LPAE)
+        && (regime_tcr(env, mmu_idx)->raw_tcr & TTBCR_EAE)) {
+        return true;
+    }
+    return false;
+}
+
+/*
+ * Returns true if the stage 1 translation regime is using LPAE format page
+ * tables. Used when raising alignment exceptions, whose FSR changes depending
+ * on whether the long or short descriptor format is in use.
+ */
+bool arm_s1_regime_using_lpae_format(CPUARMState *env, ARMMMUIdx mmu_idx)
+{
+    mmu_idx = stage_1_mmu_idx(mmu_idx);
+    return regime_using_lpae_format(env, mmu_idx);
+}
+
 static inline uint32_t merge_syn_data_abort(uint32_t template_syn,
                                             unsigned int target_el,
                                             bool same_el, bool ea,

From 1c73d84807712aad4c7b2e4e753d6870ab81145d Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:52 +0100
Subject: [PATCH 877/935] target/arm: Move arm_pamax, pamax_map into ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-19-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 25 -------------------------
 target/arm/ptw.c    | 25 +++++++++++++++++++++++++
 target/arm/ptw.h    |  2 --
 3 files changed, 25 insertions(+), 27 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index d2b196ff3e..563e34ecde 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10814,31 +10814,6 @@ bool check_s2_mmu_setup(ARMCPU *cpu, bool is_aa64, int level,
 }
 #endif /* !CONFIG_USER_ONLY */
 
-/* This mapping is common between ID_AA64MMFR0.PARANGE and TCR_ELx.{I}PS. */
-const uint8_t pamax_map[] = {
-    [0] = 32,
-    [1] = 36,
-    [2] = 40,
-    [3] = 42,
-    [4] = 44,
-    [5] = 48,
-    [6] = 52,
-};
-
-/* The cpu-specific constant value of PAMax; also used by hw/arm/virt. */
-unsigned int arm_pamax(ARMCPU *cpu)
-{
-    unsigned int parange =
-        FIELD_EX64(cpu->isar.id_aa64mmfr0, ID_AA64MMFR0, PARANGE);
-
-    /*
-     * id_aa64mmfr0 is a read-only register so values outside of the
-     * supported mappings can be considered an implementation error.
-     */
-    assert(parange < ARRAY_SIZE(pamax_map));
-    return pamax_map[parange];
-}
-
 int aa64_va_parameter_tbi(uint64_t tcr, ARMMMUIdx mmu_idx)
 {
     if (regime_has_2_ranges(mmu_idx)) {
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index e4b860d2ae..d754273fa1 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -23,6 +23,31 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
                                ARMMMUFaultInfo *fi, ARMCacheAttrs *cacheattrs)
     __attribute__((nonnull));
 
+/* This mapping is common between ID_AA64MMFR0.PARANGE and TCR_ELx.{I}PS. */
+static const uint8_t pamax_map[] = {
+    [0] = 32,
+    [1] = 36,
+    [2] = 40,
+    [3] = 42,
+    [4] = 44,
+    [5] = 48,
+    [6] = 52,
+};
+
+/* The cpu-specific constant value of PAMax; also used by hw/arm/virt. */
+unsigned int arm_pamax(ARMCPU *cpu)
+{
+    unsigned int parange =
+        FIELD_EX64(cpu->isar.id_aa64mmfr0, ID_AA64MMFR0, PARANGE);
+
+    /*
+     * id_aa64mmfr0 is a read-only register so values outside of the
+     * supported mappings can be considered an implementation error.
+     */
+    assert(parange < ARRAY_SIZE(pamax_map));
+    return pamax_map[parange];
+}
+
 static bool regime_translation_big_endian(CPUARMState *env, ARMMMUIdx mmu_idx)
 {
     return (regime_sctlr(env, mmu_idx) & SCTLR_EE) != 0;
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index 28b8cb9fb8..fba650d01c 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -11,8 +11,6 @@
 
 #ifndef CONFIG_USER_ONLY
 
-extern const uint8_t pamax_map[7];
-
 bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx);
 bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx);
 uint64_t regime_ttbr(CPUARMState *env, ARMMMUIdx mmu_idx, int ttbrn);

From f8526edc2f32b66a585618afc1dabd960a4c9b95 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:52 +0100
Subject: [PATCH 878/935] target/arm: Move get_S1prot, get_S2prot to ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-20-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 128 --------------------------------------------
 target/arm/ptw.c    | 128 ++++++++++++++++++++++++++++++++++++++++++++
 target/arm/ptw.h    |   3 --
 3 files changed, 128 insertions(+), 131 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 563e34ecde..148eb28ba8 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10615,134 +10615,6 @@ int simple_ap_to_rw_prot_is_user(int ap, bool is_user)
     }
 }
 
-/* Translate S2 section/page access permissions to protection flags
- *
- * @env:     CPUARMState
- * @s2ap:    The 2-bit stage2 access permissions (S2AP)
- * @xn:      XN (execute-never) bits
- * @s1_is_el0: true if this is S2 of an S1+2 walk for EL0
- */
-int get_S2prot(CPUARMState *env, int s2ap, int xn, bool s1_is_el0)
-{
-    int prot = 0;
-
-    if (s2ap & 1) {
-        prot |= PAGE_READ;
-    }
-    if (s2ap & 2) {
-        prot |= PAGE_WRITE;
-    }
-
-    if (cpu_isar_feature(any_tts2uxn, env_archcpu(env))) {
-        switch (xn) {
-        case 0:
-            prot |= PAGE_EXEC;
-            break;
-        case 1:
-            if (s1_is_el0) {
-                prot |= PAGE_EXEC;
-            }
-            break;
-        case 2:
-            break;
-        case 3:
-            if (!s1_is_el0) {
-                prot |= PAGE_EXEC;
-            }
-            break;
-        default:
-            g_assert_not_reached();
-        }
-    } else {
-        if (!extract32(xn, 1, 1)) {
-            if (arm_el_is_aa64(env, 2) || prot & PAGE_READ) {
-                prot |= PAGE_EXEC;
-            }
-        }
-    }
-    return prot;
-}
-
-/* Translate section/page access permissions to protection flags
- *
- * @env:     CPUARMState
- * @mmu_idx: MMU index indicating required translation regime
- * @is_aa64: TRUE if AArch64
- * @ap:      The 2-bit simple AP (AP[2:1])
- * @ns:      NS (non-secure) bit
- * @xn:      XN (execute-never) bit
- * @pxn:     PXN (privileged execute-never) bit
- */
-int get_S1prot(CPUARMState *env, ARMMMUIdx mmu_idx, bool is_aa64,
-               int ap, int ns, int xn, int pxn)
-{
-    bool is_user = regime_is_user(env, mmu_idx);
-    int prot_rw, user_rw;
-    bool have_wxn;
-    int wxn = 0;
-
-    assert(mmu_idx != ARMMMUIdx_Stage2);
-    assert(mmu_idx != ARMMMUIdx_Stage2_S);
-
-    user_rw = simple_ap_to_rw_prot_is_user(ap, true);
-    if (is_user) {
-        prot_rw = user_rw;
-    } else {
-        if (user_rw && regime_is_pan(env, mmu_idx)) {
-            /* PAN forbids data accesses but doesn't affect insn fetch */
-            prot_rw = 0;
-        } else {
-            prot_rw = simple_ap_to_rw_prot_is_user(ap, false);
-        }
-    }
-
-    if (ns && arm_is_secure(env) && (env->cp15.scr_el3 & SCR_SIF)) {
-        return prot_rw;
-    }
-
-    /* TODO have_wxn should be replaced with
-     *   ARM_FEATURE_V8 || (ARM_FEATURE_V7 && ARM_FEATURE_EL2)
-     * when ARM_FEATURE_EL2 starts getting set. For now we assume all LPAE
-     * compatible processors have EL2, which is required for [U]WXN.
-     */
-    have_wxn = arm_feature(env, ARM_FEATURE_LPAE);
-
-    if (have_wxn) {
-        wxn = regime_sctlr(env, mmu_idx) & SCTLR_WXN;
-    }
-
-    if (is_aa64) {
-        if (regime_has_2_ranges(mmu_idx) && !is_user) {
-            xn = pxn || (user_rw & PAGE_WRITE);
-        }
-    } else if (arm_feature(env, ARM_FEATURE_V7)) {
-        switch (regime_el(env, mmu_idx)) {
-        case 1:
-        case 3:
-            if (is_user) {
-                xn = xn || !(user_rw & PAGE_READ);
-            } else {
-                int uwxn = 0;
-                if (have_wxn) {
-                    uwxn = regime_sctlr(env, mmu_idx) & SCTLR_UWXN;
-                }
-                xn = xn || !(prot_rw & PAGE_READ) || pxn ||
-                     (uwxn && (user_rw & PAGE_WRITE));
-            }
-            break;
-        case 2:
-            break;
-        }
-    } else {
-        xn = wxn = 0;
-    }
-
-    if (xn || (wxn && (prot_rw & PAGE_WRITE))) {
-        return prot_rw;
-    }
-    return prot_rw | PAGE_EXEC;
-}
-
 /*
  * check_s2_mmu_setup
  * @cpu:        ARMCPU
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index d754273fa1..af9ad42028 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -487,6 +487,134 @@ do_fault:
     return true;
 }
 
+/*
+ * Translate S2 section/page access permissions to protection flags
+ * @env:     CPUARMState
+ * @s2ap:    The 2-bit stage2 access permissions (S2AP)
+ * @xn:      XN (execute-never) bits
+ * @s1_is_el0: true if this is S2 of an S1+2 walk for EL0
+ */
+static int get_S2prot(CPUARMState *env, int s2ap, int xn, bool s1_is_el0)
+{
+    int prot = 0;
+
+    if (s2ap & 1) {
+        prot |= PAGE_READ;
+    }
+    if (s2ap & 2) {
+        prot |= PAGE_WRITE;
+    }
+
+    if (cpu_isar_feature(any_tts2uxn, env_archcpu(env))) {
+        switch (xn) {
+        case 0:
+            prot |= PAGE_EXEC;
+            break;
+        case 1:
+            if (s1_is_el0) {
+                prot |= PAGE_EXEC;
+            }
+            break;
+        case 2:
+            break;
+        case 3:
+            if (!s1_is_el0) {
+                prot |= PAGE_EXEC;
+            }
+            break;
+        default:
+            g_assert_not_reached();
+        }
+    } else {
+        if (!extract32(xn, 1, 1)) {
+            if (arm_el_is_aa64(env, 2) || prot & PAGE_READ) {
+                prot |= PAGE_EXEC;
+            }
+        }
+    }
+    return prot;
+}
+
+/*
+ * Translate section/page access permissions to protection flags
+ * @env:     CPUARMState
+ * @mmu_idx: MMU index indicating required translation regime
+ * @is_aa64: TRUE if AArch64
+ * @ap:      The 2-bit simple AP (AP[2:1])
+ * @ns:      NS (non-secure) bit
+ * @xn:      XN (execute-never) bit
+ * @pxn:     PXN (privileged execute-never) bit
+ */
+static int get_S1prot(CPUARMState *env, ARMMMUIdx mmu_idx, bool is_aa64,
+                      int ap, int ns, int xn, int pxn)
+{
+    bool is_user = regime_is_user(env, mmu_idx);
+    int prot_rw, user_rw;
+    bool have_wxn;
+    int wxn = 0;
+
+    assert(mmu_idx != ARMMMUIdx_Stage2);
+    assert(mmu_idx != ARMMMUIdx_Stage2_S);
+
+    user_rw = simple_ap_to_rw_prot_is_user(ap, true);
+    if (is_user) {
+        prot_rw = user_rw;
+    } else {
+        if (user_rw && regime_is_pan(env, mmu_idx)) {
+            /* PAN forbids data accesses but doesn't affect insn fetch */
+            prot_rw = 0;
+        } else {
+            prot_rw = simple_ap_to_rw_prot_is_user(ap, false);
+        }
+    }
+
+    if (ns && arm_is_secure(env) && (env->cp15.scr_el3 & SCR_SIF)) {
+        return prot_rw;
+    }
+
+    /* TODO have_wxn should be replaced with
+     *   ARM_FEATURE_V8 || (ARM_FEATURE_V7 && ARM_FEATURE_EL2)
+     * when ARM_FEATURE_EL2 starts getting set. For now we assume all LPAE
+     * compatible processors have EL2, which is required for [U]WXN.
+     */
+    have_wxn = arm_feature(env, ARM_FEATURE_LPAE);
+
+    if (have_wxn) {
+        wxn = regime_sctlr(env, mmu_idx) & SCTLR_WXN;
+    }
+
+    if (is_aa64) {
+        if (regime_has_2_ranges(mmu_idx) && !is_user) {
+            xn = pxn || (user_rw & PAGE_WRITE);
+        }
+    } else if (arm_feature(env, ARM_FEATURE_V7)) {
+        switch (regime_el(env, mmu_idx)) {
+        case 1:
+        case 3:
+            if (is_user) {
+                xn = xn || !(user_rw & PAGE_READ);
+            } else {
+                int uwxn = 0;
+                if (have_wxn) {
+                    uwxn = regime_sctlr(env, mmu_idx) & SCTLR_UWXN;
+                }
+                xn = xn || !(prot_rw & PAGE_READ) || pxn ||
+                     (uwxn && (user_rw & PAGE_WRITE));
+            }
+            break;
+        case 2:
+            break;
+        }
+    } else {
+        xn = wxn = 0;
+    }
+
+    if (xn || (wxn && (prot_rw & PAGE_WRITE))) {
+        return prot_rw;
+    }
+    return prot_rw | PAGE_EXEC;
+}
+
 /**
  * get_phys_addr_lpae: perform one stage of page table walk, LPAE format
  *
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index fba650d01c..93147e0b06 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -29,9 +29,6 @@ ARMVAParameters aa32_va_parameters(CPUARMState *env, uint32_t va,
                                    ARMMMUIdx mmu_idx);
 bool check_s2_mmu_setup(ARMCPU *cpu, bool is_aa64, int level,
                         int inputsize, int stride, int outputsize);
-int get_S2prot(CPUARMState *env, int s2ap, int xn, bool s1_is_el0);
-int get_S1prot(CPUARMState *env, ARMMMUIdx mmu_idx, bool is_aa64,
-               int ap, int ns, int xn, int pxn);
 
 #endif /* !CONFIG_USER_ONLY */
 #endif /* TARGET_ARM_PTW_H */

From c5168785d208527c45f455619abe2539182a9fff Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:52 +0100
Subject: [PATCH 879/935] target/arm: Move check_s2_mmu_setup to ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-21-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 70 ---------------------------------------------
 target/arm/ptw.c    | 70 +++++++++++++++++++++++++++++++++++++++++++++
 target/arm/ptw.h    |  2 --
 3 files changed, 70 insertions(+), 72 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 148eb28ba8..2526f4c6c4 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10614,76 +10614,6 @@ int simple_ap_to_rw_prot_is_user(int ap, bool is_user)
         g_assert_not_reached();
     }
 }
-
-/*
- * check_s2_mmu_setup
- * @cpu:        ARMCPU
- * @is_aa64:    True if the translation regime is in AArch64 state
- * @startlevel: Suggested starting level
- * @inputsize:  Bitsize of IPAs
- * @stride:     Page-table stride (See the ARM ARM)
- *
- * Returns true if the suggested S2 translation parameters are OK and
- * false otherwise.
- */
-bool check_s2_mmu_setup(ARMCPU *cpu, bool is_aa64, int level,
-                        int inputsize, int stride, int outputsize)
-{
-    const int grainsize = stride + 3;
-    int startsizecheck;
-
-    /*
-     * Negative levels are usually not allowed...
-     * Except for FEAT_LPA2, 4k page table, 52-bit address space, which
-     * begins with level -1.  Note that previous feature tests will have
-     * eliminated this combination if it is not enabled.
-     */
-    if (level < (inputsize == 52 && stride == 9 ? -1 : 0)) {
-        return false;
-    }
-
-    startsizecheck = inputsize - ((3 - level) * stride + grainsize);
-    if (startsizecheck < 1 || startsizecheck > stride + 4) {
-        return false;
-    }
-
-    if (is_aa64) {
-        switch (stride) {
-        case 13: /* 64KB Pages.  */
-            if (level == 0 || (level == 1 && outputsize <= 42)) {
-                return false;
-            }
-            break;
-        case 11: /* 16KB Pages.  */
-            if (level == 0 || (level == 1 && outputsize <= 40)) {
-                return false;
-            }
-            break;
-        case 9: /* 4KB Pages.  */
-            if (level == 0 && outputsize <= 42) {
-                return false;
-            }
-            break;
-        default:
-            g_assert_not_reached();
-        }
-
-        /* Inputsize checks.  */
-        if (inputsize > outputsize &&
-            (arm_el_is_aa64(&cpu->env, 1) || inputsize > 40)) {
-            /* This is CONSTRAINED UNPREDICTABLE and we choose to fault.  */
-            return false;
-        }
-    } else {
-        /* AArch32 only supports 4KB pages. Assert on that.  */
-        assert(stride == 9);
-
-        if (level == 0) {
-            return false;
-        }
-    }
-    return true;
-}
 #endif /* !CONFIG_USER_ONLY */
 
 int aa64_va_parameter_tbi(uint64_t tcr, ARMMMUIdx mmu_idx)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index af9ad42028..525272e99a 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -615,6 +615,76 @@ static int get_S1prot(CPUARMState *env, ARMMMUIdx mmu_idx, bool is_aa64,
     return prot_rw | PAGE_EXEC;
 }
 
+/*
+ * check_s2_mmu_setup
+ * @cpu:        ARMCPU
+ * @is_aa64:    True if the translation regime is in AArch64 state
+ * @startlevel: Suggested starting level
+ * @inputsize:  Bitsize of IPAs
+ * @stride:     Page-table stride (See the ARM ARM)
+ *
+ * Returns true if the suggested S2 translation parameters are OK and
+ * false otherwise.
+ */
+static bool check_s2_mmu_setup(ARMCPU *cpu, bool is_aa64, int level,
+                               int inputsize, int stride, int outputsize)
+{
+    const int grainsize = stride + 3;
+    int startsizecheck;
+
+    /*
+     * Negative levels are usually not allowed...
+     * Except for FEAT_LPA2, 4k page table, 52-bit address space, which
+     * begins with level -1.  Note that previous feature tests will have
+     * eliminated this combination if it is not enabled.
+     */
+    if (level < (inputsize == 52 && stride == 9 ? -1 : 0)) {
+        return false;
+    }
+
+    startsizecheck = inputsize - ((3 - level) * stride + grainsize);
+    if (startsizecheck < 1 || startsizecheck > stride + 4) {
+        return false;
+    }
+
+    if (is_aa64) {
+        switch (stride) {
+        case 13: /* 64KB Pages.  */
+            if (level == 0 || (level == 1 && outputsize <= 42)) {
+                return false;
+            }
+            break;
+        case 11: /* 16KB Pages.  */
+            if (level == 0 || (level == 1 && outputsize <= 40)) {
+                return false;
+            }
+            break;
+        case 9: /* 4KB Pages.  */
+            if (level == 0 && outputsize <= 42) {
+                return false;
+            }
+            break;
+        default:
+            g_assert_not_reached();
+        }
+
+        /* Inputsize checks.  */
+        if (inputsize > outputsize &&
+            (arm_el_is_aa64(&cpu->env, 1) || inputsize > 40)) {
+            /* This is CONSTRAINED UNPREDICTABLE and we choose to fault.  */
+            return false;
+        }
+    } else {
+        /* AArch32 only supports 4KB pages. Assert on that.  */
+        assert(stride == 9);
+
+        if (level == 0) {
+            return false;
+        }
+    }
+    return true;
+}
+
 /**
  * get_phys_addr_lpae: perform one stage of page table walk, LPAE format
  *
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index 93147e0b06..a71161b01b 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -27,8 +27,6 @@ simple_ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap)
 
 ARMVAParameters aa32_va_parameters(CPUARMState *env, uint32_t va,
                                    ARMMMUIdx mmu_idx);
-bool check_s2_mmu_setup(ARMCPU *cpu, bool is_aa64, int level,
-                        int inputsize, int stride, int outputsize);
 
 #endif /* !CONFIG_USER_ONLY */
 #endif /* TARGET_ARM_PTW_H */

From 2f0ec92e9488e1ac6c7293fe37a5f1c0e1737b36 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:53 +0100
Subject: [PATCH 880/935] target/arm: Move aa32_va_parameters to ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-22-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 64 ---------------------------------------------
 target/arm/ptw.c    | 64 +++++++++++++++++++++++++++++++++++++++++++++
 target/arm/ptw.h    |  3 ---
 3 files changed, 64 insertions(+), 67 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 2526f4c6c4..f61f1da61e 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10771,70 +10771,6 @@ ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va,
 }
 
 #ifndef CONFIG_USER_ONLY
-ARMVAParameters aa32_va_parameters(CPUARMState *env, uint32_t va,
-                                   ARMMMUIdx mmu_idx)
-{
-    uint64_t tcr = regime_tcr(env, mmu_idx)->raw_tcr;
-    uint32_t el = regime_el(env, mmu_idx);
-    int select, tsz;
-    bool epd, hpd;
-
-    assert(mmu_idx != ARMMMUIdx_Stage2_S);
-
-    if (mmu_idx == ARMMMUIdx_Stage2) {
-        /* VTCR */
-        bool sext = extract32(tcr, 4, 1);
-        bool sign = extract32(tcr, 3, 1);
-
-        /*
-         * If the sign-extend bit is not the same as t0sz[3], the result
-         * is unpredictable. Flag this as a guest error.
-         */
-        if (sign != sext) {
-            qemu_log_mask(LOG_GUEST_ERROR,
-                          "AArch32: VTCR.S / VTCR.T0SZ[3] mismatch\n");
-        }
-        tsz = sextract32(tcr, 0, 4) + 8;
-        select = 0;
-        hpd = false;
-        epd = false;
-    } else if (el == 2) {
-        /* HTCR */
-        tsz = extract32(tcr, 0, 3);
-        select = 0;
-        hpd = extract64(tcr, 24, 1);
-        epd = false;
-    } else {
-        int t0sz = extract32(tcr, 0, 3);
-        int t1sz = extract32(tcr, 16, 3);
-
-        if (t1sz == 0) {
-            select = va > (0xffffffffu >> t0sz);
-        } else {
-            /* Note that we will detect errors later.  */
-            select = va >= ~(0xffffffffu >> t1sz);
-        }
-        if (!select) {
-            tsz = t0sz;
-            epd = extract32(tcr, 7, 1);
-            hpd = extract64(tcr, 41, 1);
-        } else {
-            tsz = t1sz;
-            epd = extract32(tcr, 23, 1);
-            hpd = extract64(tcr, 42, 1);
-        }
-        /* For aarch32, hpd0 is not enabled without t2e as well.  */
-        hpd &= extract32(tcr, 6, 1);
-    }
-
-    return (ARMVAParameters) {
-        .tsz = tsz,
-        .select = select,
-        .epd = epd,
-        .hpd = hpd,
-    };
-}
-
 hwaddr arm_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr,
                                          MemTxAttrs *attrs)
 {
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 525272e99a..427813ea56 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -615,6 +615,70 @@ static int get_S1prot(CPUARMState *env, ARMMMUIdx mmu_idx, bool is_aa64,
     return prot_rw | PAGE_EXEC;
 }
 
+static ARMVAParameters aa32_va_parameters(CPUARMState *env, uint32_t va,
+                                          ARMMMUIdx mmu_idx)
+{
+    uint64_t tcr = regime_tcr(env, mmu_idx)->raw_tcr;
+    uint32_t el = regime_el(env, mmu_idx);
+    int select, tsz;
+    bool epd, hpd;
+
+    assert(mmu_idx != ARMMMUIdx_Stage2_S);
+
+    if (mmu_idx == ARMMMUIdx_Stage2) {
+        /* VTCR */
+        bool sext = extract32(tcr, 4, 1);
+        bool sign = extract32(tcr, 3, 1);
+
+        /*
+         * If the sign-extend bit is not the same as t0sz[3], the result
+         * is unpredictable. Flag this as a guest error.
+         */
+        if (sign != sext) {
+            qemu_log_mask(LOG_GUEST_ERROR,
+                          "AArch32: VTCR.S / VTCR.T0SZ[3] mismatch\n");
+        }
+        tsz = sextract32(tcr, 0, 4) + 8;
+        select = 0;
+        hpd = false;
+        epd = false;
+    } else if (el == 2) {
+        /* HTCR */
+        tsz = extract32(tcr, 0, 3);
+        select = 0;
+        hpd = extract64(tcr, 24, 1);
+        epd = false;
+    } else {
+        int t0sz = extract32(tcr, 0, 3);
+        int t1sz = extract32(tcr, 16, 3);
+
+        if (t1sz == 0) {
+            select = va > (0xffffffffu >> t0sz);
+        } else {
+            /* Note that we will detect errors later.  */
+            select = va >= ~(0xffffffffu >> t1sz);
+        }
+        if (!select) {
+            tsz = t0sz;
+            epd = extract32(tcr, 7, 1);
+            hpd = extract64(tcr, 41, 1);
+        } else {
+            tsz = t1sz;
+            epd = extract32(tcr, 23, 1);
+            hpd = extract64(tcr, 42, 1);
+        }
+        /* For aarch32, hpd0 is not enabled without t2e as well.  */
+        hpd &= extract32(tcr, 6, 1);
+    }
+
+    return (ARMVAParameters) {
+        .tsz = tsz,
+        .select = select,
+        .epd = epd,
+        .hpd = hpd,
+    };
+}
+
 /*
  * check_s2_mmu_setup
  * @cpu:        ARMCPU
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index a71161b01b..9314fb4d23 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -25,8 +25,5 @@ simple_ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap)
     return simple_ap_to_rw_prot_is_user(ap, regime_is_user(env, mmu_idx));
 }
 
-ARMVAParameters aa32_va_parameters(CPUARMState *env, uint32_t va,
-                                   ARMMMUIdx mmu_idx);
-
 #endif /* !CONFIG_USER_ONLY */
 #endif /* TARGET_ARM_PTW_H */

From 4845d3be1239a956a4cedd5c2ca7863837487927 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:53 +0100
Subject: [PATCH 881/935] target/arm: Move ap_to_tw_prot etc to ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-23-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 77 ------------------------------------------
 target/arm/ptw.c    | 81 +++++++++++++++++++++++++++++++++++++++++++++
 target/arm/ptw.h    | 10 ------
 3 files changed, 81 insertions(+), 87 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index f61f1da61e..e894afcb49 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10537,83 +10537,6 @@ bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx)
         g_assert_not_reached();
     }
 }
-
-/* Translate section/page access permissions to page
- * R/W protection flags
- *
- * @env:         CPUARMState
- * @mmu_idx:     MMU index indicating required translation regime
- * @ap:          The 3-bit access permissions (AP[2:0])
- * @domain_prot: The 2-bit domain access permissions
- */
-int ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap, int domain_prot)
-{
-    bool is_user = regime_is_user(env, mmu_idx);
-
-    if (domain_prot == 3) {
-        return PAGE_READ | PAGE_WRITE;
-    }
-
-    switch (ap) {
-    case 0:
-        if (arm_feature(env, ARM_FEATURE_V7)) {
-            return 0;
-        }
-        switch (regime_sctlr(env, mmu_idx) & (SCTLR_S | SCTLR_R)) {
-        case SCTLR_S:
-            return is_user ? 0 : PAGE_READ;
-        case SCTLR_R:
-            return PAGE_READ;
-        default:
-            return 0;
-        }
-    case 1:
-        return is_user ? 0 : PAGE_READ | PAGE_WRITE;
-    case 2:
-        if (is_user) {
-            return PAGE_READ;
-        } else {
-            return PAGE_READ | PAGE_WRITE;
-        }
-    case 3:
-        return PAGE_READ | PAGE_WRITE;
-    case 4: /* Reserved.  */
-        return 0;
-    case 5:
-        return is_user ? 0 : PAGE_READ;
-    case 6:
-        return PAGE_READ;
-    case 7:
-        if (!arm_feature(env, ARM_FEATURE_V6K)) {
-            return 0;
-        }
-        return PAGE_READ;
-    default:
-        g_assert_not_reached();
-    }
-}
-
-/* Translate section/page access permissions to page
- * R/W protection flags.
- *
- * @ap:      The 2-bit simple AP (AP[2:1])
- * @is_user: TRUE if accessing from PL0
- */
-int simple_ap_to_rw_prot_is_user(int ap, bool is_user)
-{
-    switch (ap) {
-    case 0:
-        return is_user ? 0 : PAGE_READ | PAGE_WRITE;
-    case 1:
-        return PAGE_READ | PAGE_WRITE;
-    case 2:
-        return is_user ? 0 : PAGE_READ;
-    case 3:
-        return PAGE_READ;
-    default:
-        g_assert_not_reached();
-    }
-}
 #endif /* !CONFIG_USER_ONLY */
 
 int aa64_va_parameter_tbi(uint64_t tcr, ARMMMUIdx mmu_idx)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 427813ea56..9ab77c3998 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -211,6 +211,87 @@ static bool get_level1_table_address(CPUARMState *env, ARMMMUIdx mmu_idx,
     return true;
 }
 
+/*
+ * Translate section/page access permissions to page R/W protection flags
+ * @env:         CPUARMState
+ * @mmu_idx:     MMU index indicating required translation regime
+ * @ap:          The 3-bit access permissions (AP[2:0])
+ * @domain_prot: The 2-bit domain access permissions
+ */
+static int ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx,
+                         int ap, int domain_prot)
+{
+    bool is_user = regime_is_user(env, mmu_idx);
+
+    if (domain_prot == 3) {
+        return PAGE_READ | PAGE_WRITE;
+    }
+
+    switch (ap) {
+    case 0:
+        if (arm_feature(env, ARM_FEATURE_V7)) {
+            return 0;
+        }
+        switch (regime_sctlr(env, mmu_idx) & (SCTLR_S | SCTLR_R)) {
+        case SCTLR_S:
+            return is_user ? 0 : PAGE_READ;
+        case SCTLR_R:
+            return PAGE_READ;
+        default:
+            return 0;
+        }
+    case 1:
+        return is_user ? 0 : PAGE_READ | PAGE_WRITE;
+    case 2:
+        if (is_user) {
+            return PAGE_READ;
+        } else {
+            return PAGE_READ | PAGE_WRITE;
+        }
+    case 3:
+        return PAGE_READ | PAGE_WRITE;
+    case 4: /* Reserved.  */
+        return 0;
+    case 5:
+        return is_user ? 0 : PAGE_READ;
+    case 6:
+        return PAGE_READ;
+    case 7:
+        if (!arm_feature(env, ARM_FEATURE_V6K)) {
+            return 0;
+        }
+        return PAGE_READ;
+    default:
+        g_assert_not_reached();
+    }
+}
+
+/*
+ * Translate section/page access permissions to page R/W protection flags.
+ * @ap:      The 2-bit simple AP (AP[2:1])
+ * @is_user: TRUE if accessing from PL0
+ */
+static int simple_ap_to_rw_prot_is_user(int ap, bool is_user)
+{
+    switch (ap) {
+    case 0:
+        return is_user ? 0 : PAGE_READ | PAGE_WRITE;
+    case 1:
+        return PAGE_READ | PAGE_WRITE;
+    case 2:
+        return is_user ? 0 : PAGE_READ;
+    case 3:
+        return PAGE_READ;
+    default:
+        g_assert_not_reached();
+    }
+}
+
+static int simple_ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap)
+{
+    return simple_ap_to_rw_prot_is_user(ap, regime_is_user(env, mmu_idx));
+}
+
 static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
                              MMUAccessType access_type, ARMMMUIdx mmu_idx,
                              hwaddr *phys_ptr, int *prot,
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index 9314fb4d23..85ad576794 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -15,15 +15,5 @@ bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx);
 bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx);
 uint64_t regime_ttbr(CPUARMState *env, ARMMMUIdx mmu_idx, int ttbrn);
 
-int ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx,
-                  int ap, int domain_prot);
-int simple_ap_to_rw_prot_is_user(int ap, bool is_user);
-
-static inline int
-simple_ap_to_rw_prot(CPUARMState *env, ARMMMUIdx mmu_idx, int ap)
-{
-    return simple_ap_to_rw_prot_is_user(ap, regime_is_user(env, mmu_idx));
-}
-
 #endif /* !CONFIG_USER_ONLY */
 #endif /* TARGET_ARM_PTW_H */

From 0c23d56fc149ada0823648481c09b5d2047570f4 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:53 +0100
Subject: [PATCH 882/935] target/arm: Move regime_is_user to ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-24-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 24 ------------------------
 target/arm/ptw.c    | 22 ++++++++++++++++++++++
 target/arm/ptw.h    |  1 -
 3 files changed, 22 insertions(+), 25 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index e894afcb49..8deb0fa94c 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10515,30 +10515,6 @@ ARMMMUIdx stage_1_mmu_idx(ARMMMUIdx mmu_idx)
 }
 #endif /* !CONFIG_USER_ONLY */
 
-#ifndef CONFIG_USER_ONLY
-bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx)
-{
-    switch (mmu_idx) {
-    case ARMMMUIdx_SE10_0:
-    case ARMMMUIdx_E20_0:
-    case ARMMMUIdx_SE20_0:
-    case ARMMMUIdx_Stage1_E0:
-    case ARMMMUIdx_Stage1_SE0:
-    case ARMMMUIdx_MUser:
-    case ARMMMUIdx_MSUser:
-    case ARMMMUIdx_MUserNegPri:
-    case ARMMMUIdx_MSUserNegPri:
-        return true;
-    default:
-        return false;
-    case ARMMMUIdx_E10_0:
-    case ARMMMUIdx_E10_1:
-    case ARMMMUIdx_E10_1_PAN:
-        g_assert_not_reached();
-    }
-}
-#endif /* !CONFIG_USER_ONLY */
-
 int aa64_va_parameter_tbi(uint64_t tcr, ARMMMUIdx mmu_idx)
 {
     if (regime_has_2_ranges(mmu_idx)) {
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 9ab77c3998..8db4b5edf1 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -53,6 +53,28 @@ static bool regime_translation_big_endian(CPUARMState *env, ARMMMUIdx mmu_idx)
     return (regime_sctlr(env, mmu_idx) & SCTLR_EE) != 0;
 }
 
+static bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx)
+{
+    switch (mmu_idx) {
+    case ARMMMUIdx_SE10_0:
+    case ARMMMUIdx_E20_0:
+    case ARMMMUIdx_SE20_0:
+    case ARMMMUIdx_Stage1_E0:
+    case ARMMMUIdx_Stage1_SE0:
+    case ARMMMUIdx_MUser:
+    case ARMMMUIdx_MSUser:
+    case ARMMMUIdx_MUserNegPri:
+    case ARMMMUIdx_MSUserNegPri:
+        return true;
+    default:
+        return false;
+    case ARMMMUIdx_E10_0:
+    case ARMMMUIdx_E10_1:
+    case ARMMMUIdx_E10_1_PAN:
+        g_assert_not_reached();
+    }
+}
+
 static bool ptw_attrs_are_device(CPUARMState *env, ARMCacheAttrs cacheattrs)
 {
     /*
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index 85ad576794..3d3061a435 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -11,7 +11,6 @@
 
 #ifndef CONFIG_USER_ONLY
 
-bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx);
 bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx);
 uint64_t regime_ttbr(CPUARMState *env, ARMMMUIdx mmu_idx, int ttbrn);
 

From 3b318aaeef120a71544771cd84884ccad5fdff0a Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:53 +0100
Subject: [PATCH 883/935] target/arm: Move regime_ttbr to ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-25-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 16 ----------------
 target/arm/ptw.c    | 16 ++++++++++++++++
 target/arm/ptw.h    |  1 -
 3 files changed, 16 insertions(+), 17 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 8deb0fa94c..fdda87e87e 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10475,22 +10475,6 @@ bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx)
     return (regime_sctlr(env, mmu_idx) & SCTLR_M) == 0;
 }
 
-/* Return the TTBR associated with this translation regime */
-uint64_t regime_ttbr(CPUARMState *env, ARMMMUIdx mmu_idx, int ttbrn)
-{
-    if (mmu_idx == ARMMMUIdx_Stage2) {
-        return env->cp15.vttbr_el2;
-    }
-    if (mmu_idx == ARMMMUIdx_Stage2_S) {
-        return env->cp15.vsttbr_el2;
-    }
-    if (ttbrn == 0) {
-        return env->cp15.ttbr0_el[regime_el(env, mmu_idx)];
-    } else {
-        return env->cp15.ttbr1_el[regime_el(env, mmu_idx)];
-    }
-}
-
 /* Convert a possible stage1+2 MMU index into the appropriate
  * stage 1 MMU index
  */
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 8db4b5edf1..dc559e6bdf 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -75,6 +75,22 @@ static bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx)
     }
 }
 
+/* Return the TTBR associated with this translation regime */
+static uint64_t regime_ttbr(CPUARMState *env, ARMMMUIdx mmu_idx, int ttbrn)
+{
+    if (mmu_idx == ARMMMUIdx_Stage2) {
+        return env->cp15.vttbr_el2;
+    }
+    if (mmu_idx == ARMMMUIdx_Stage2_S) {
+        return env->cp15.vsttbr_el2;
+    }
+    if (ttbrn == 0) {
+        return env->cp15.ttbr0_el[regime_el(env, mmu_idx)];
+    } else {
+        return env->cp15.ttbr1_el[regime_el(env, mmu_idx)];
+    }
+}
+
 static bool ptw_attrs_are_device(CPUARMState *env, ARMCacheAttrs cacheattrs)
 {
     /*
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
index 3d3061a435..ed152ddaf4 100644
--- a/target/arm/ptw.h
+++ b/target/arm/ptw.h
@@ -12,7 +12,6 @@
 #ifndef CONFIG_USER_ONLY
 
 bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx);
-uint64_t regime_ttbr(CPUARMState *env, ARMMMUIdx mmu_idx, int ttbrn);
 
 #endif /* !CONFIG_USER_ONLY */
 #endif /* TARGET_ARM_PTW_H */

From 8db1a3a0bba1dbe71a3d363b23416a3c30653ddd Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:53 +0100
Subject: [PATCH 884/935] target/arm: Move regime_translation_disabled to ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-26-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 47 ---------------------------------------------
 target/arm/ptw.c    | 47 ++++++++++++++++++++++++++++++++++++++++++++-
 target/arm/ptw.h    | 17 ----------------
 3 files changed, 46 insertions(+), 65 deletions(-)
 delete mode 100644 target/arm/ptw.h

diff --git a/target/arm/helper.c b/target/arm/helper.c
index fdda87e87e..69b1c060c1 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -36,7 +36,6 @@
 #include "semihosting/common-semi.h"
 #endif
 #include "cpregs.h"
-#include "ptw.h"
 
 #define ARM_CPU_FREQ 1000000000 /* FIXME: 1 GHz, should be configurable */
 
@@ -10429,52 +10428,6 @@ uint64_t arm_sctlr(CPUARMState *env, int el)
 }
 
 #ifndef CONFIG_USER_ONLY
-
-/* Return true if the specified stage of address translation is disabled */
-bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx)
-{
-    uint64_t hcr_el2;
-
-    if (arm_feature(env, ARM_FEATURE_M)) {
-        switch (env->v7m.mpu_ctrl[regime_is_secure(env, mmu_idx)] &
-                (R_V7M_MPU_CTRL_ENABLE_MASK | R_V7M_MPU_CTRL_HFNMIENA_MASK)) {
-        case R_V7M_MPU_CTRL_ENABLE_MASK:
-            /* Enabled, but not for HardFault and NMI */
-            return mmu_idx & ARM_MMU_IDX_M_NEGPRI;
-        case R_V7M_MPU_CTRL_ENABLE_MASK | R_V7M_MPU_CTRL_HFNMIENA_MASK:
-            /* Enabled for all cases */
-            return false;
-        case 0:
-        default:
-            /* HFNMIENA set and ENABLE clear is UNPREDICTABLE, but
-             * we warned about that in armv7m_nvic.c when the guest set it.
-             */
-            return true;
-        }
-    }
-
-    hcr_el2 = arm_hcr_el2_eff(env);
-
-    if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
-        /* HCR.DC means HCR.VM behaves as 1 */
-        return (hcr_el2 & (HCR_DC | HCR_VM)) == 0;
-    }
-
-    if (hcr_el2 & HCR_TGE) {
-        /* TGE means that NS EL0/1 act as if SCTLR_EL1.M is zero */
-        if (!regime_is_secure(env, mmu_idx) && regime_el(env, mmu_idx) == 1) {
-            return true;
-        }
-    }
-
-    if ((hcr_el2 & HCR_DC) && arm_mmu_idx_is_stage1_of_2(mmu_idx)) {
-        /* HCR.DC means SCTLR_EL1.M behaves as 0 */
-        return true;
-    }
-
-    return (regime_sctlr(env, mmu_idx) & SCTLR_M) == 0;
-}
-
 /* Convert a possible stage1+2 MMU index into the appropriate
  * stage 1 MMU index
  */
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index dc559e6bdf..ec60afd9bf 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -12,7 +12,6 @@
 #include "cpu.h"
 #include "internals.h"
 #include "idau.h"
-#include "ptw.h"
 
 
 static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
@@ -91,6 +90,52 @@ static uint64_t regime_ttbr(CPUARMState *env, ARMMMUIdx mmu_idx, int ttbrn)
     }
 }
 
+/* Return true if the specified stage of address translation is disabled */
+static bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx)
+{
+    uint64_t hcr_el2;
+
+    if (arm_feature(env, ARM_FEATURE_M)) {
+        switch (env->v7m.mpu_ctrl[regime_is_secure(env, mmu_idx)] &
+                (R_V7M_MPU_CTRL_ENABLE_MASK | R_V7M_MPU_CTRL_HFNMIENA_MASK)) {
+        case R_V7M_MPU_CTRL_ENABLE_MASK:
+            /* Enabled, but not for HardFault and NMI */
+            return mmu_idx & ARM_MMU_IDX_M_NEGPRI;
+        case R_V7M_MPU_CTRL_ENABLE_MASK | R_V7M_MPU_CTRL_HFNMIENA_MASK:
+            /* Enabled for all cases */
+            return false;
+        case 0:
+        default:
+            /*
+             * HFNMIENA set and ENABLE clear is UNPREDICTABLE, but
+             * we warned about that in armv7m_nvic.c when the guest set it.
+             */
+            return true;
+        }
+    }
+
+    hcr_el2 = arm_hcr_el2_eff(env);
+
+    if (mmu_idx == ARMMMUIdx_Stage2 || mmu_idx == ARMMMUIdx_Stage2_S) {
+        /* HCR.DC means HCR.VM behaves as 1 */
+        return (hcr_el2 & (HCR_DC | HCR_VM)) == 0;
+    }
+
+    if (hcr_el2 & HCR_TGE) {
+        /* TGE means that NS EL0/1 act as if SCTLR_EL1.M is zero */
+        if (!regime_is_secure(env, mmu_idx) && regime_el(env, mmu_idx) == 1) {
+            return true;
+        }
+    }
+
+    if ((hcr_el2 & HCR_DC) && arm_mmu_idx_is_stage1_of_2(mmu_idx)) {
+        /* HCR.DC means SCTLR_EL1.M behaves as 0 */
+        return true;
+    }
+
+    return (regime_sctlr(env, mmu_idx) & SCTLR_M) == 0;
+}
+
 static bool ptw_attrs_are_device(CPUARMState *env, ARMCacheAttrs cacheattrs)
 {
     /*
diff --git a/target/arm/ptw.h b/target/arm/ptw.h
deleted file mode 100644
index ed152ddaf4..0000000000
--- a/target/arm/ptw.h
+++ /dev/null
@@ -1,17 +0,0 @@
-/*
- * ARM page table walking.
- *
- * This code is licensed under the GNU GPL v2 or later.
- *
- * SPDX-License-Identifier: GPL-2.0-or-later
- */
-
-#ifndef TARGET_ARM_PTW_H
-#define TARGET_ARM_PTW_H
-
-#ifndef CONFIG_USER_ONLY
-
-bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx);
-
-#endif /* !CONFIG_USER_ONLY */
-#endif /* TARGET_ARM_PTW_H */

From 23971205cf38e63964b7e110df7d5a293277a48e Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:54 +0100
Subject: [PATCH 885/935] target/arm: Move arm_cpu_get_phys_page_attrs_debug to
 ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-27-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 26 --------------------------
 target/arm/ptw.c    | 24 ++++++++++++++++++++++++
 2 files changed, 24 insertions(+), 26 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 69b1c060c1..fe1e426f88 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10606,32 +10606,6 @@ ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va,
     };
 }
 
-#ifndef CONFIG_USER_ONLY
-hwaddr arm_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr,
-                                         MemTxAttrs *attrs)
-{
-    ARMCPU *cpu = ARM_CPU(cs);
-    CPUARMState *env = &cpu->env;
-    hwaddr phys_addr;
-    target_ulong page_size;
-    int prot;
-    bool ret;
-    ARMMMUFaultInfo fi = {};
-    ARMMMUIdx mmu_idx = arm_mmu_idx(env);
-    ARMCacheAttrs cacheattrs = {};
-
-    *attrs = (MemTxAttrs) {};
-
-    ret = get_phys_addr(env, addr, MMU_DATA_LOAD, mmu_idx, &phys_addr,
-                        attrs, &prot, &page_size, &fi, &cacheattrs);
-
-    if (ret) {
-        return -1;
-    }
-    return phys_addr;
-}
-#endif
-
 /* Note that signed overflow is undefined in C.  The following routines are
    careful to use unsigned types where modulo arithmetic is required.
    Failure to do so _will_ break on newer gcc.  */
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index ec60afd9bf..e9f6870d0a 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -2491,3 +2491,27 @@ bool get_phys_addr(CPUARMState *env, target_ulong address,
                                     phys_ptr, prot, page_size, fi);
     }
 }
+
+hwaddr arm_cpu_get_phys_page_attrs_debug(CPUState *cs, vaddr addr,
+                                         MemTxAttrs *attrs)
+{
+    ARMCPU *cpu = ARM_CPU(cs);
+    CPUARMState *env = &cpu->env;
+    hwaddr phys_addr;
+    target_ulong page_size;
+    int prot;
+    bool ret;
+    ARMMMUFaultInfo fi = {};
+    ARMMMUIdx mmu_idx = arm_mmu_idx(env);
+    ARMCacheAttrs cacheattrs = {};
+
+    *attrs = (MemTxAttrs) {};
+
+    ret = get_phys_addr(env, addr, MMU_DATA_LOAD, mmu_idx, &phys_addr,
+                        attrs, &prot, &page_size, &fi, &cacheattrs);
+
+    if (ret) {
+        return -1;
+    }
+    return phys_addr;
+}

From 1d2612553640a79e049708b9d87a9cd3ddb29af6 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:54 +0100
Subject: [PATCH 886/935] target/arm: Move stage_1_mmu_idx, arm_stage1_mmu_idx
 to ptw.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-28-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 32 --------------------------------
 target/arm/ptw.c    | 28 ++++++++++++++++++++++++++++
 2 files changed, 28 insertions(+), 32 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index fe1e426f88..37cf9fa6ab 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10427,31 +10427,6 @@ uint64_t arm_sctlr(CPUARMState *env, int el)
     return env->cp15.sctlr_el[el];
 }
 
-#ifndef CONFIG_USER_ONLY
-/* Convert a possible stage1+2 MMU index into the appropriate
- * stage 1 MMU index
- */
-ARMMMUIdx stage_1_mmu_idx(ARMMMUIdx mmu_idx)
-{
-    switch (mmu_idx) {
-    case ARMMMUIdx_SE10_0:
-        return ARMMMUIdx_Stage1_SE0;
-    case ARMMMUIdx_SE10_1:
-        return ARMMMUIdx_Stage1_SE1;
-    case ARMMMUIdx_SE10_1_PAN:
-        return ARMMMUIdx_Stage1_SE1_PAN;
-    case ARMMMUIdx_E10_0:
-        return ARMMMUIdx_Stage1_E0;
-    case ARMMMUIdx_E10_1:
-        return ARMMMUIdx_Stage1_E1;
-    case ARMMMUIdx_E10_1_PAN:
-        return ARMMMUIdx_Stage1_E1_PAN;
-    default:
-        return mmu_idx;
-    }
-}
-#endif /* !CONFIG_USER_ONLY */
-
 int aa64_va_parameter_tbi(uint64_t tcr, ARMMMUIdx mmu_idx)
 {
     if (regime_has_2_ranges(mmu_idx)) {
@@ -11081,13 +11056,6 @@ ARMMMUIdx arm_mmu_idx(CPUARMState *env)
     return arm_mmu_idx_el(env, arm_current_el(env));
 }
 
-#ifndef CONFIG_USER_ONLY
-ARMMMUIdx arm_stage1_mmu_idx(CPUARMState *env)
-{
-    return stage_1_mmu_idx(arm_mmu_idx(env));
-}
-#endif
-
 static CPUARMTBFlags rebuild_hflags_common(CPUARMState *env, int fp_el,
                                            ARMMMUIdx mmu_idx,
                                            CPUARMTBFlags flags)
diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index e9f6870d0a..49e9a1d108 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -47,6 +47,34 @@ unsigned int arm_pamax(ARMCPU *cpu)
     return pamax_map[parange];
 }
 
+/*
+ * Convert a possible stage1+2 MMU index into the appropriate stage 1 MMU index
+ */
+ARMMMUIdx stage_1_mmu_idx(ARMMMUIdx mmu_idx)
+{
+    switch (mmu_idx) {
+    case ARMMMUIdx_SE10_0:
+        return ARMMMUIdx_Stage1_SE0;
+    case ARMMMUIdx_SE10_1:
+        return ARMMMUIdx_Stage1_SE1;
+    case ARMMMUIdx_SE10_1_PAN:
+        return ARMMMUIdx_Stage1_SE1_PAN;
+    case ARMMMUIdx_E10_0:
+        return ARMMMUIdx_Stage1_E0;
+    case ARMMMUIdx_E10_1:
+        return ARMMMUIdx_Stage1_E1;
+    case ARMMMUIdx_E10_1_PAN:
+        return ARMMMUIdx_Stage1_E1_PAN;
+    default:
+        return mmu_idx;
+    }
+}
+
+ARMMMUIdx arm_stage1_mmu_idx(CPUARMState *env)
+{
+    return stage_1_mmu_idx(arm_mmu_idx(env));
+}
+
 static bool regime_translation_big_endian(CPUARMState *env, ARMMMUIdx mmu_idx)
 {
     return (regime_sctlr(env, mmu_idx) & SCTLR_EE) != 0;

From 5e79887ba67e22bfd890b72f94c482176a200fbc Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:54 +0100
Subject: [PATCH 887/935] target/arm: Pass CPUARMState to arm_ld[lq]_ptw

The use of ARM_CPU to recover env from cs calls
object_class_dynamic_cast, which shows up on the profile.
This is pointless, because all callers already have env, and
the reverse operation, env_cpu, is only pointer arithmetic.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220604040607.269301-29-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/ptw.c | 23 +++++++++--------------
 1 file changed, 9 insertions(+), 14 deletions(-)

diff --git a/target/arm/ptw.c b/target/arm/ptw.c
index 49e9a1d108..4d97a24808 100644
--- a/target/arm/ptw.c
+++ b/target/arm/ptw.c
@@ -241,11 +241,10 @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx,
 }
 
 /* All loads done in the course of a page table walk go through here. */
-static uint32_t arm_ldl_ptw(CPUState *cs, hwaddr addr, bool is_secure,
+static uint32_t arm_ldl_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
                             ARMMMUIdx mmu_idx, ARMMMUFaultInfo *fi)
 {
-    ARMCPU *cpu = ARM_CPU(cs);
-    CPUARMState *env = &cpu->env;
+    CPUState *cs = env_cpu(env);
     MemTxAttrs attrs = {};
     MemTxResult result = MEMTX_OK;
     AddressSpace *as;
@@ -270,11 +269,10 @@ static uint32_t arm_ldl_ptw(CPUState *cs, hwaddr addr, bool is_secure,
     return 0;
 }
 
-static uint64_t arm_ldq_ptw(CPUState *cs, hwaddr addr, bool is_secure,
+static uint64_t arm_ldq_ptw(CPUARMState *env, hwaddr addr, bool is_secure,
                             ARMMMUIdx mmu_idx, ARMMMUFaultInfo *fi)
 {
-    ARMCPU *cpu = ARM_CPU(cs);
-    CPUARMState *env = &cpu->env;
+    CPUState *cs = env_cpu(env);
     MemTxAttrs attrs = {};
     MemTxResult result = MEMTX_OK;
     AddressSpace *as;
@@ -409,7 +407,6 @@ static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
                              target_ulong *page_size,
                              ARMMMUFaultInfo *fi)
 {
-    CPUState *cs = env_cpu(env);
     int level = 1;
     uint32_t table;
     uint32_t desc;
@@ -427,7 +424,7 @@ static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
         fi->type = ARMFault_Translation;
         goto do_fault;
     }
-    desc = arm_ldl_ptw(cs, table, regime_is_secure(env, mmu_idx),
+    desc = arm_ldl_ptw(env, table, regime_is_secure(env, mmu_idx),
                        mmu_idx, fi);
     if (fi->type != ARMFault_None) {
         goto do_fault;
@@ -466,7 +463,7 @@ static bool get_phys_addr_v5(CPUARMState *env, uint32_t address,
             /* Fine pagetable.  */
             table = (desc & 0xfffff000) | ((address >> 8) & 0xffc);
         }
-        desc = arm_ldl_ptw(cs, table, regime_is_secure(env, mmu_idx),
+        desc = arm_ldl_ptw(env, table, regime_is_secure(env, mmu_idx),
                            mmu_idx, fi);
         if (fi->type != ARMFault_None) {
             goto do_fault;
@@ -531,7 +528,6 @@ static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
                              hwaddr *phys_ptr, MemTxAttrs *attrs, int *prot,
                              target_ulong *page_size, ARMMMUFaultInfo *fi)
 {
-    CPUState *cs = env_cpu(env);
     ARMCPU *cpu = env_archcpu(env);
     int level = 1;
     uint32_t table;
@@ -553,7 +549,7 @@ static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
         fi->type = ARMFault_Translation;
         goto do_fault;
     }
-    desc = arm_ldl_ptw(cs, table, regime_is_secure(env, mmu_idx),
+    desc = arm_ldl_ptw(env, table, regime_is_secure(env, mmu_idx),
                        mmu_idx, fi);
     if (fi->type != ARMFault_None) {
         goto do_fault;
@@ -607,7 +603,7 @@ static bool get_phys_addr_v6(CPUARMState *env, uint32_t address,
         ns = extract32(desc, 3, 1);
         /* Lookup l2 entry.  */
         table = (desc & 0xfffffc00) | ((address >> 10) & 0x3fc);
-        desc = arm_ldl_ptw(cs, table, regime_is_secure(env, mmu_idx),
+        desc = arm_ldl_ptw(env, table, regime_is_secure(env, mmu_idx),
                            mmu_idx, fi);
         if (fi->type != ARMFault_None) {
             goto do_fault;
@@ -973,7 +969,6 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
                                ARMMMUFaultInfo *fi, ARMCacheAttrs *cacheattrs)
 {
     ARMCPU *cpu = env_archcpu(env);
-    CPUState *cs = CPU(cpu);
     /* Read an LPAE long-descriptor translation table. */
     ARMFaultType fault_type = ARMFault_Translation;
     uint32_t level;
@@ -1196,7 +1191,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address,
         descaddr |= (address >> (stride * (4 - level))) & indexmask;
         descaddr &= ~7ULL;
         nstable = extract32(tableattrs, 4, 1);
-        descriptor = arm_ldq_ptw(cs, descaddr, !nstable, mmu_idx, fi);
+        descriptor = arm_ldq_ptw(env, descaddr, !nstable, mmu_idx, fi);
         if (fi->type != ARMFault_None) {
             goto do_fault;
         }

From f45ce4c35f5e0873bbbc3119eff8539610233b7e Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:54 +0100
Subject: [PATCH 888/935] target/arm: Rename TBFLAG_A64 ZCR_LEN to VL

With SME, the vector length does not only come from ZCR_ELx.
Comment that this is either NVL or SVL, like the pseudocode.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-2-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu.h           | 3 ++-
 target/arm/helper.c        | 2 +-
 target/arm/translate-a64.c | 2 +-
 target/arm/translate-a64.h | 2 +-
 target/arm/translate-sve.c | 2 +-
 target/arm/translate.h     | 2 +-
 6 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 0ee1705a4f..e791ffdd6b 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -3241,7 +3241,8 @@ FIELD(TBFLAG_M32, MVE_NO_PRED, 5, 1)            /* Not cached. */
  */
 FIELD(TBFLAG_A64, TBII, 0, 2)
 FIELD(TBFLAG_A64, SVEEXC_EL, 2, 2)
-FIELD(TBFLAG_A64, ZCR_LEN, 4, 4)
+/* The current vector length, either NVL or SVL. */
+FIELD(TBFLAG_A64, VL, 4, 4)
 FIELD(TBFLAG_A64, PAUTH_ACTIVE, 8, 1)
 FIELD(TBFLAG_A64, BT, 9, 1)
 FIELD(TBFLAG_A64, BTYPE, 10, 2)         /* Not cached. */
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 37cf9fa6ab..c228deca75 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -11181,7 +11181,7 @@ static CPUARMTBFlags rebuild_hflags_a64(CPUARMState *env, int el, int fp_el,
             zcr_len = sve_zcr_len_for_el(env, el);
         }
         DP_TBFLAG_A64(flags, SVEEXC_EL, sve_el);
-        DP_TBFLAG_A64(flags, ZCR_LEN, zcr_len);
+        DP_TBFLAG_A64(flags, VL, zcr_len);
     }
 
     sctlr = regime_sctlr(env, stage1);
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 935e1929bb..d438fb89e7 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -14608,7 +14608,7 @@ static void aarch64_tr_init_disas_context(DisasContextBase *dcbase,
     dc->align_mem = EX_TBFLAG_ANY(tb_flags, ALIGN_MEM);
     dc->pstate_il = EX_TBFLAG_ANY(tb_flags, PSTATE__IL);
     dc->sve_excp_el = EX_TBFLAG_A64(tb_flags, SVEEXC_EL);
-    dc->sve_len = (EX_TBFLAG_A64(tb_flags, ZCR_LEN) + 1) * 16;
+    dc->vl = (EX_TBFLAG_A64(tb_flags, VL) + 1) * 16;
     dc->pauth_active = EX_TBFLAG_A64(tb_flags, PAUTH_ACTIVE);
     dc->bt = EX_TBFLAG_A64(tb_flags, BT);
     dc->btype = EX_TBFLAG_A64(tb_flags, BTYPE);
diff --git a/target/arm/translate-a64.h b/target/arm/translate-a64.h
index f2e8ee0ee1..dbc917ee65 100644
--- a/target/arm/translate-a64.h
+++ b/target/arm/translate-a64.h
@@ -104,7 +104,7 @@ static inline TCGv_ptr vec_full_reg_ptr(DisasContext *s, int regno)
 /* Return the byte size of the "whole" vector register, VL / 8.  */
 static inline int vec_full_reg_size(DisasContext *s)
 {
-    return s->sve_len;
+    return s->vl;
 }
 
 bool disas_sve(DisasContext *, uint32_t);
diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 836511d719..67761bf2cc 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -111,7 +111,7 @@ static inline int pred_full_reg_offset(DisasContext *s, int regno)
 /* Return the byte size of the whole predicate register, VL / 64.  */
 static inline int pred_full_reg_size(DisasContext *s)
 {
-    return s->sve_len >> 3;
+    return s->vl >> 3;
 }
 
 /* Round up the size of a register to a size allowed by
diff --git a/target/arm/translate.h b/target/arm/translate.h
index 9f0bb270c5..f473a21ed4 100644
--- a/target/arm/translate.h
+++ b/target/arm/translate.h
@@ -42,7 +42,7 @@ typedef struct DisasContext {
     bool ns;        /* Use non-secure CPREG bank on access */
     int fp_excp_el; /* FP exception EL or 0 if enabled */
     int sve_excp_el; /* SVE exception EL or 0 if enabled */
-    int sve_len;     /* SVE vector length in bytes */
+    int vl;          /* current vector length in bytes */
     /* Flag indicating that exceptions from secure mode are routed to EL3. */
     bool secure_routed_to_el3;
     bool vfp_enabled; /* FP enabled via FPSCR.EN */

From 8b599e5c0274cd5e4a33be6a10d6cd04631a74d1 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:55 +0100
Subject: [PATCH 889/935] linux-user/aarch64: Introduce sve_vq

Add an interface function to extract the digested vector length
rather than the raw zcr_el[1] value.  This fixes an incorrect
return from do_prctl_set_vl where we didn't take into account
the set of vector lengths supported by the cpu.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-3-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 linux-user/aarch64/signal.c       |  4 ++--
 linux-user/aarch64/target_prctl.h | 20 +++++++++++++-------
 target/arm/cpu.h                  | 11 +++++++++++
 3 files changed, 26 insertions(+), 9 deletions(-)

diff --git a/linux-user/aarch64/signal.c b/linux-user/aarch64/signal.c
index 7de4c96eb9..7da0e36c6d 100644
--- a/linux-user/aarch64/signal.c
+++ b/linux-user/aarch64/signal.c
@@ -315,7 +315,7 @@ static int target_restore_sigframe(CPUARMState *env,
 
         case TARGET_SVE_MAGIC:
             if (cpu_isar_feature(aa64_sve, env_archcpu(env))) {
-                vq = (env->vfp.zcr_el[1] & 0xf) + 1;
+                vq = sve_vq(env);
                 sve_size = QEMU_ALIGN_UP(TARGET_SVE_SIG_CONTEXT_SIZE(vq), 16);
                 if (!sve && size == sve_size) {
                     sve = (struct target_sve_context *)ctx;
@@ -434,7 +434,7 @@ static void target_setup_frame(int usig, struct target_sigaction *ka,
 
     /* SVE state needs saving only if it exists.  */
     if (cpu_isar_feature(aa64_sve, env_archcpu(env))) {
-        vq = (env->vfp.zcr_el[1] & 0xf) + 1;
+        vq = sve_vq(env);
         sve_size = QEMU_ALIGN_UP(TARGET_SVE_SIG_CONTEXT_SIZE(vq), 16);
         sve_ofs = alloc_sigframe_space(sve_size, &layout);
     }
diff --git a/linux-user/aarch64/target_prctl.h b/linux-user/aarch64/target_prctl.h
index 3f5a5d3933..1d440ffbea 100644
--- a/linux-user/aarch64/target_prctl.h
+++ b/linux-user/aarch64/target_prctl.h
@@ -10,7 +10,7 @@ static abi_long do_prctl_get_vl(CPUArchState *env)
 {
     ARMCPU *cpu = env_archcpu(env);
     if (cpu_isar_feature(aa64_sve, cpu)) {
-        return ((cpu->env.vfp.zcr_el[1] & 0xf) + 1) * 16;
+        return sve_vq(env) * 16;
     }
     return -TARGET_EINVAL;
 }
@@ -25,18 +25,24 @@ static abi_long do_prctl_set_vl(CPUArchState *env, abi_long arg2)
      */
     if (cpu_isar_feature(aa64_sve, env_archcpu(env))
         && arg2 >= 0 && arg2 <= 512 * 16 && !(arg2 & 15)) {
-        ARMCPU *cpu = env_archcpu(env);
         uint32_t vq, old_vq;
 
-        old_vq = (env->vfp.zcr_el[1] & 0xf) + 1;
-        vq = MAX(arg2 / 16, 1);
-        vq = MIN(vq, cpu->sve_max_vq);
+        old_vq = sve_vq(env);
 
+        /*
+         * Bound the value of arg2, so that we know that it fits into
+         * the 4-bit field in ZCR_EL1.  Rely on the hflags rebuild to
+         * sort out the length supported by the cpu.
+         */
+        vq = MAX(arg2 / 16, 1);
+        vq = MIN(vq, ARM_MAX_VQ);
+        env->vfp.zcr_el[1] = vq - 1;
+        arm_rebuild_hflags(env);
+
+        vq = sve_vq(env);
         if (vq < old_vq) {
             aarch64_sve_narrow_vq(env, vq);
         }
-        env->vfp.zcr_el[1] = vq - 1;
-        arm_rebuild_hflags(env);
         return vq * 16;
     }
     return -TARGET_EINVAL;
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index e791ffdd6b..f5af88b686 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -3286,6 +3286,17 @@ static inline int cpu_mmu_index(CPUARMState *env, bool ifetch)
     return EX_TBFLAG_ANY(env->hflags, MMUIDX);
 }
 
+/**
+ * sve_vq
+ * @env: the cpu context
+ *
+ * Return the VL cached within env->hflags, in units of quadwords.
+ */
+static inline int sve_vq(CPUARMState *env)
+{
+    return EX_TBFLAG_A64(env->hflags, VL) + 1;
+}
+
 static inline bool bswap_code(bool sctlr_b)
 {
 #ifdef CONFIG_USER_ONLY

From 61a8c23a3b56607ee6e745b4cf7975170df88801 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:55 +0100
Subject: [PATCH 890/935] target/arm: Remove route_to_el2 check from
 sve_exception_el

We handle this routing in raise_exception.  Promoting the value early
means that we can't directly compare FPEXC_EL and SVEEXC_EL.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-4-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index c228deca75..1bd77af7e5 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -6149,8 +6149,7 @@ int sve_exception_el(CPUARMState *env, int el)
             /* fall through */
         case 0:
         case 2:
-            /* route_to_el2 */
-            return hcr_el2 & HCR_TGE ? 2 : 1;
+            return 1;
         }
 
         /* Check CPACR.FPEN.  */

From 397d922c6248509c6d490f82088f00cbc716287c Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:55 +0100
Subject: [PATCH 891/935] target/arm: Remove fp checks from sve_exception_el

Instead of checking these bits in fp_exception_el and
also in sve_exception_el, document that we must compare
the results.  The only place where we have not already
checked that FP EL is zero is in rebuild_hflags_a64.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-5-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 58 +++++++++++++++------------------------------
 1 file changed, 19 insertions(+), 39 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 1bd77af7e5..4f4044c688 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -6129,11 +6129,15 @@ static const ARMCPRegInfo minimal_ras_reginfo[] = {
       .access = PL2_RW, .fieldoffset = offsetof(CPUARMState, cp15.vsesr_el2) },
 };
 
-/* Return the exception level to which exceptions should be taken
- * via SVEAccessTrap.  If an exception should be routed through
- * AArch64.AdvSIMDFPAccessTrap, return 0; fp_exception_el should
- * take care of raising that exception.
- * C.f. the ARM pseudocode function CheckSVEEnabled.
+/*
+ * Return the exception level to which exceptions should be taken
+ * via SVEAccessTrap.  This excludes the check for whether the exception
+ * should be routed through AArch64.AdvSIMDFPAccessTrap.  That can easily
+ * be found by testing 0 < fp_exception_el < sve_exception_el.
+ *
+ * C.f. the ARM pseudocode function CheckSVEEnabled.  Note that the
+ * pseudocode does *not* separate out the FP trap checks, but has them
+ * all in one function.
  */
 int sve_exception_el(CPUARMState *env, int el)
 {
@@ -6151,18 +6155,6 @@ int sve_exception_el(CPUARMState *env, int el)
         case 2:
             return 1;
         }
-
-        /* Check CPACR.FPEN.  */
-        switch (FIELD_EX64(env->cp15.cpacr_el1, CPACR_EL1, FPEN)) {
-        case 1:
-            if (el != 0) {
-                break;
-            }
-            /* fall through */
-        case 0:
-        case 2:
-            return 0;
-        }
     }
 
     /*
@@ -6180,24 +6172,10 @@ int sve_exception_el(CPUARMState *env, int el)
             case 2:
                 return 2;
             }
-
-            switch (FIELD_EX32(env->cp15.cptr_el[2], CPTR_EL2, FPEN)) {
-            case 1:
-                if (el == 2 || !(hcr_el2 & HCR_TGE)) {
-                    break;
-                }
-                /* fall through */
-            case 0:
-            case 2:
-                return 0;
-            }
         } else if (arm_is_el2_enabled(env)) {
             if (FIELD_EX64(env->cp15.cptr_el[2], CPTR_EL2, TZ)) {
                 return 2;
             }
-            if (FIELD_EX64(env->cp15.cptr_el[2], CPTR_EL2, TFP)) {
-                return 0;
-            }
         }
     }
 
@@ -11168,19 +11146,21 @@ static CPUARMTBFlags rebuild_hflags_a64(CPUARMState *env, int el, int fp_el,
 
     if (cpu_isar_feature(aa64_sve, env_archcpu(env))) {
         int sve_el = sve_exception_el(env, el);
-        uint32_t zcr_len;
 
         /*
-         * If SVE is disabled, but FP is enabled,
-         * then the effective len is 0.
+         * If either FP or SVE are disabled, translator does not need len.
+         * If SVE EL > FP EL, FP exception has precedence, and translator
+         * does not need SVE EL.  Save potential re-translations by forcing
+         * the unneeded data to zero.
          */
-        if (sve_el != 0 && fp_el == 0) {
-            zcr_len = 0;
-        } else {
-            zcr_len = sve_zcr_len_for_el(env, el);
+        if (fp_el != 0) {
+            if (sve_el > fp_el) {
+                sve_el = 0;
+            }
+        } else if (sve_el == 0) {
+            DP_TBFLAG_A64(flags, VL, sve_zcr_len_for_el(env, el));
         }
         DP_TBFLAG_A64(flags, SVEEXC_EL, sve_el);
-        DP_TBFLAG_A64(flags, VL, zcr_len);
     }
 
     sctlr = regime_sctlr(env, stage1);

From 19668718ad188e1b6a162bb52357a67fd407c96c Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:55 +0100
Subject: [PATCH 892/935] target/arm: Add el_is_in_host

This (newish) ARM pseudocode function is easier to work with
than open-coded tests for HCR_E2H etc.  Use of the function
will be staged into the code base in parts.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-6-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c    | 28 ++++++++++++++++++++++++++++
 target/arm/internals.h |  2 ++
 2 files changed, 30 insertions(+)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 4f4044c688..322508170e 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -5282,6 +5282,34 @@ uint64_t arm_hcr_el2_eff(CPUARMState *env)
     return ret;
 }
 
+/*
+ * Corresponds to ARM pseudocode function ELIsInHost().
+ */
+bool el_is_in_host(CPUARMState *env, int el)
+{
+    uint64_t mask;
+
+    /*
+     * Since we only care about E2H and TGE, we can skip arm_hcr_el2_eff().
+     * Perform the simplest bit tests first, and validate EL2 afterward.
+     */
+    if (el & 1) {
+        return false; /* EL1 or EL3 */
+    }
+
+    /*
+     * Note that hcr_write() checks isar_feature_aa64_vh(),
+     * aka HaveVirtHostExt(), in allowing HCR_E2H to be set.
+     */
+    mask = el ? HCR_E2H : HCR_E2H | HCR_TGE;
+    if ((env->cp15.hcr_el2 & mask) != mask) {
+        return false;
+    }
+
+    /* TGE and/or E2H set: double check those bits are currently legal. */
+    return arm_is_el2_enabled(env) && arm_el_is_aa64(env, 2);
+}
+
 static void hcrx_write(CPUARMState *env, const ARMCPRegInfo *ri,
                        uint64_t value)
 {
diff --git a/target/arm/internals.h b/target/arm/internals.h
index 1d83146d56..ceaddcbfd6 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -1347,6 +1347,8 @@ static inline void define_cortex_a72_a57_a53_cp_reginfo(ARMCPU *cpu) { }
 void define_cortex_a72_a57_a53_cp_reginfo(ARMCPU *cpu);
 #endif
 
+bool el_is_in_host(CPUARMState *env, int el);
+
 void aa32_max_features(ARMCPU *cpu);
 
 #endif

From c6225bebc2498e9dd85bdd4de1840925650c7a9f Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:56 +0100
Subject: [PATCH 893/935] target/arm: Use el_is_in_host for sve_zcr_len_for_el

The ARM pseudocode function NVL uses this predicate now,
and I think it's a bit clearer.  Simplify the pseudocode
condition by noting that IsInHost is always false for EL1.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-7-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 322508170e..6b17c0f507 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -6238,8 +6238,7 @@ uint32_t sve_zcr_len_for_el(CPUARMState *env, int el)
     ARMCPU *cpu = env_archcpu(env);
     uint32_t zcr_len = cpu->sve_max_vq - 1;
 
-    if (el <= 1 &&
-        (arm_hcr_el2_eff(env) & (HCR_E2H | HCR_TGE)) != (HCR_E2H | HCR_TGE)) {
+    if (el <= 1 && !el_is_in_host(env, el)) {
         zcr_len = MIN(zcr_len, 0xf & (uint32_t)env->vfp.zcr_el[1]);
     }
     if (el <= 2 && arm_feature(env, ARM_FEATURE_EL2)) {

From aa4451b60e6a2b1bc4e9a8f6bbf90b266e2a4390 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:56 +0100
Subject: [PATCH 894/935] target/arm: Use el_is_in_host for sve_exception_el

The ARM pseudocode function CheckNormalSVEEnabled uses this
predicate now, and I think it's a bit clearer.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-8-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 6b17c0f507..40b60b1eea 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -6170,9 +6170,7 @@ static const ARMCPRegInfo minimal_ras_reginfo[] = {
 int sve_exception_el(CPUARMState *env, int el)
 {
 #ifndef CONFIG_USER_ONLY
-    uint64_t hcr_el2 = arm_hcr_el2_eff(env);
-
-    if (el <= 1 && (hcr_el2 & (HCR_E2H | HCR_TGE)) != (HCR_E2H | HCR_TGE)) {
+    if (el <= 1 && !el_is_in_host(env, el)) {
         switch (FIELD_EX64(env->cp15.cpacr_el1, CPACR_EL1, ZEN)) {
         case 1:
             if (el != 0) {
@@ -6189,6 +6187,7 @@ int sve_exception_el(CPUARMState *env, int el)
      * CPTR_EL2 changes format with HCR_EL2.E2H (regardless of TGE).
      */
     if (el <= 2) {
+        uint64_t hcr_el2 = arm_hcr_el2_eff(env);
         if (hcr_el2 & HCR_E2H) {
             switch (FIELD_EX64(env->cp15.cptr_el[2], CPTR_EL2, ZEN)) {
             case 1:

From 7d38cb92aad848b9c935588815d0b3a5e157aabb Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:56 +0100
Subject: [PATCH 895/935] target/arm: Hoist arm_is_el2_enabled check in
 sve_exception_el

This check is buried within arm_hcr_el2_eff(), but since we
have to have the explicit check for CPTR_EL2.TZ, we might as
well just check it once at the beginning of the block.

Once this is done, we can test HCR_EL2.{E2H,TGE} directly,
rather than going through arm_hcr_el2_eff().

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-9-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c | 13 +++++--------
 1 file changed, 5 insertions(+), 8 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 40b60b1eea..61e8026d0e 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -6183,15 +6183,12 @@ int sve_exception_el(CPUARMState *env, int el)
         }
     }
 
-    /*
-     * CPTR_EL2 changes format with HCR_EL2.E2H (regardless of TGE).
-     */
-    if (el <= 2) {
-        uint64_t hcr_el2 = arm_hcr_el2_eff(env);
-        if (hcr_el2 & HCR_E2H) {
+    if (el <= 2 && arm_is_el2_enabled(env)) {
+        /* CPTR_EL2 changes format with HCR_EL2.E2H (regardless of TGE). */
+        if (env->cp15.hcr_el2 & HCR_E2H) {
             switch (FIELD_EX64(env->cp15.cptr_el[2], CPTR_EL2, ZEN)) {
             case 1:
-                if (el != 0 || !(hcr_el2 & HCR_TGE)) {
+                if (el != 0 || !(env->cp15.hcr_el2 & HCR_TGE)) {
                     break;
                 }
                 /* fall through */
@@ -6199,7 +6196,7 @@ int sve_exception_el(CPUARMState *env, int el)
             case 2:
                 return 2;
             }
-        } else if (arm_is_el2_enabled(env)) {
+        } else {
             if (FIELD_EX64(env->cp15.cptr_el[2], CPTR_EL2, TZ)) {
                 return 2;
             }

From 87252bdecdafd49f8f4da6244a40415c27053bc3 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:56 +0100
Subject: [PATCH 896/935] target/arm: Do not use aarch64_sve_zcr_get_valid_len
 in reset

We don't need to constrain the value set in zcr_el[1],
because it will be done by sve_zcr_len_for_el.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-10-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index d2bd74c2ed..0621944167 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -208,8 +208,7 @@ static void arm_cpu_reset(DeviceState *dev)
                                          CPACR_EL1, ZEN, 3);
         /* with reasonable vector length */
         if (cpu_isar_feature(aa64_sve, cpu)) {
-            env->vfp.zcr_el[1] =
-                aarch64_sve_zcr_get_valid_len(cpu, cpu->sve_default_vq - 1);
+            env->vfp.zcr_el[1] = cpu->sve_default_vq - 1;
         }
         /*
          * Enable 48-bit address space (TODO: take reserved_va into account).

From 9b5f422559a528f4117aa9c3098fddf3d7c535ba Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:57 +0100
Subject: [PATCH 897/935] target/arm: Merge aarch64_sve_zcr_get_valid_len into
 caller

This function is used only once, and will need modification
for Streaming SVE mode.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-11-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/helper.c    | 30 +++++++++++-------------------
 target/arm/internals.h | 11 -----------
 2 files changed, 11 insertions(+), 30 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 61e8026d0e..de159c644c 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -6212,39 +6212,31 @@ int sve_exception_el(CPUARMState *env, int el)
     return 0;
 }
 
-uint32_t aarch64_sve_zcr_get_valid_len(ARMCPU *cpu, uint32_t start_len)
-{
-    uint32_t end_len;
-
-    start_len = MIN(start_len, ARM_MAX_VQ - 1);
-    end_len = start_len;
-
-    if (!test_bit(start_len, cpu->sve_vq_map)) {
-        end_len = find_last_bit(cpu->sve_vq_map, start_len);
-        assert(end_len < start_len);
-    }
-    return end_len;
-}
-
 /*
  * Given that SVE is enabled, return the vector length for EL.
  */
 uint32_t sve_zcr_len_for_el(CPUARMState *env, int el)
 {
     ARMCPU *cpu = env_archcpu(env);
-    uint32_t zcr_len = cpu->sve_max_vq - 1;
+    uint32_t len = cpu->sve_max_vq - 1;
+    uint32_t end_len;
 
     if (el <= 1 && !el_is_in_host(env, el)) {
-        zcr_len = MIN(zcr_len, 0xf & (uint32_t)env->vfp.zcr_el[1]);
+        len = MIN(len, 0xf & (uint32_t)env->vfp.zcr_el[1]);
     }
     if (el <= 2 && arm_feature(env, ARM_FEATURE_EL2)) {
-        zcr_len = MIN(zcr_len, 0xf & (uint32_t)env->vfp.zcr_el[2]);
+        len = MIN(len, 0xf & (uint32_t)env->vfp.zcr_el[2]);
     }
     if (arm_feature(env, ARM_FEATURE_EL3)) {
-        zcr_len = MIN(zcr_len, 0xf & (uint32_t)env->vfp.zcr_el[3]);
+        len = MIN(len, 0xf & (uint32_t)env->vfp.zcr_el[3]);
     }
 
-    return aarch64_sve_zcr_get_valid_len(cpu, zcr_len);
+    end_len = len;
+    if (!test_bit(len, cpu->sve_vq_map)) {
+        end_len = find_last_bit(cpu->sve_vq_map, len);
+        assert(end_len < len);
+    }
+    return end_len;
 }
 
 static void zcr_write(CPUARMState *env, const ARMCPRegInfo *ri,
diff --git a/target/arm/internals.h b/target/arm/internals.h
index ceaddcbfd6..79eb463753 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -189,17 +189,6 @@ void arm_translate_init(void);
 void arm_cpu_synchronize_from_tb(CPUState *cs, const TranslationBlock *tb);
 #endif /* CONFIG_TCG */
 
-/**
- * aarch64_sve_zcr_get_valid_len:
- * @cpu: cpu context
- * @start_len: maximum len to consider
- *
- * Return the maximum supported sve vector length <= @start_len.
- * Note that both @start_len and the return value are in units
- * of ZCR_ELx.LEN, so the vector bit length is (x + 1) * 128.
- */
-uint32_t aarch64_sve_zcr_get_valid_len(ARMCPU *cpu, uint32_t start_len);
-
 enum arm_fprounding {
     FPROUNDING_TIEEVEN,
     FPROUNDING_POSINF,

From 886902ece71b5e795fea3e052a32f047d2f8fe33 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:57 +0100
Subject: [PATCH 898/935] target/arm: Use uint32_t instead of bitmap for sve
 vq's

The bitmap need only hold 15 bits; bitmap is over-complicated.
We can simplify operations quite a bit with plain logical ops.

The introduction of SVE_VQ_POW2_MAP eliminates the need for
looping in order to search for powers of two.  Simply perform
the logical ops and use count leading or trailing zeros as
required to find the result.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-12-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu.h       |   6 +--
 target/arm/cpu64.c     | 117 ++++++++++++++++++++---------------------
 target/arm/helper.c    |   9 +---
 target/arm/internals.h |   5 ++
 target/arm/kvm64.c     |  36 +++----------
 target/arm/kvm_arm.h   |   7 ++-
 6 files changed, 75 insertions(+), 105 deletions(-)

diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index f5af88b686..73f24a5760 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -1041,9 +1041,9 @@ struct ArchCPU {
      * Bits set in sve_vq_supported represent valid vector lengths for
      * the CPU type.
      */
-    DECLARE_BITMAP(sve_vq_map, ARM_MAX_VQ);
-    DECLARE_BITMAP(sve_vq_init, ARM_MAX_VQ);
-    DECLARE_BITMAP(sve_vq_supported, ARM_MAX_VQ);
+    uint32_t sve_vq_map;
+    uint32_t sve_vq_init;
+    uint32_t sve_vq_supported;
 
     /* Generic timer counter frequency, in Hz */
     uint64_t gt_cntfrq_hz;
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index cce68dd82a..15665c962b 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -355,8 +355,11 @@ void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp)
      * any of the above.  Finally, if SVE is not disabled, then at least one
      * vector length must be enabled.
      */
-    DECLARE_BITMAP(tmp, ARM_MAX_VQ);
-    uint32_t vq, max_vq = 0;
+    uint32_t vq_map = cpu->sve_vq_map;
+    uint32_t vq_init = cpu->sve_vq_init;
+    uint32_t vq_supported;
+    uint32_t vq_mask = 0;
+    uint32_t tmp, vq, max_vq = 0;
 
     /*
      * CPU models specify a set of supported vector lengths which are
@@ -364,10 +367,16 @@ void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp)
      * in the supported bitmap results in an error.  When KVM is enabled we
      * fetch the supported bitmap from the host.
      */
-    if (kvm_enabled() && kvm_arm_sve_supported()) {
-        kvm_arm_sve_get_vls(CPU(cpu), cpu->sve_vq_supported);
-    } else if (kvm_enabled()) {
-        assert(!cpu_isar_feature(aa64_sve, cpu));
+    if (kvm_enabled()) {
+        if (kvm_arm_sve_supported()) {
+            cpu->sve_vq_supported = kvm_arm_sve_get_vls(CPU(cpu));
+            vq_supported = cpu->sve_vq_supported;
+        } else {
+            assert(!cpu_isar_feature(aa64_sve, cpu));
+            vq_supported = 0;
+        }
+    } else {
+        vq_supported = cpu->sve_vq_supported;
     }
 
     /*
@@ -375,8 +384,9 @@ void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp)
      * From the properties, sve_vq_map<N> implies sve_vq_init<N>.
      * Check first for any sve<N> enabled.
      */
-    if (!bitmap_empty(cpu->sve_vq_map, ARM_MAX_VQ)) {
-        max_vq = find_last_bit(cpu->sve_vq_map, ARM_MAX_VQ) + 1;
+    if (vq_map != 0) {
+        max_vq = 32 - clz32(vq_map);
+        vq_mask = MAKE_64BIT_MASK(0, max_vq);
 
         if (cpu->sve_max_vq && max_vq > cpu->sve_max_vq) {
             error_setg(errp, "cannot enable sve%d", max_vq * 128);
@@ -392,15 +402,10 @@ void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp)
              * For KVM we have to automatically enable all supported unitialized
              * lengths, even when the smaller lengths are not all powers-of-two.
              */
-            bitmap_andnot(tmp, cpu->sve_vq_supported, cpu->sve_vq_init, max_vq);
-            bitmap_or(cpu->sve_vq_map, cpu->sve_vq_map, tmp, max_vq);
+            vq_map |= vq_supported & ~vq_init & vq_mask;
         } else {
             /* Propagate enabled bits down through required powers-of-two. */
-            for (vq = pow2floor(max_vq); vq >= 1; vq >>= 1) {
-                if (!test_bit(vq - 1, cpu->sve_vq_init)) {
-                    set_bit(vq - 1, cpu->sve_vq_map);
-                }
-            }
+            vq_map |= SVE_VQ_POW2_MAP & ~vq_init & vq_mask;
         }
     } else if (cpu->sve_max_vq == 0) {
         /*
@@ -413,25 +418,18 @@ void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp)
 
         if (kvm_enabled()) {
             /* Disabling a supported length disables all larger lengths. */
-            for (vq = 1; vq <= ARM_MAX_VQ; ++vq) {
-                if (test_bit(vq - 1, cpu->sve_vq_init) &&
-                    test_bit(vq - 1, cpu->sve_vq_supported)) {
-                    break;
-                }
-            }
+            tmp = vq_init & vq_supported;
         } else {
             /* Disabling a power-of-two disables all larger lengths. */
-            for (vq = 1; vq <= ARM_MAX_VQ; vq <<= 1) {
-                if (test_bit(vq - 1, cpu->sve_vq_init)) {
-                    break;
-                }
-            }
+            tmp = vq_init & SVE_VQ_POW2_MAP;
         }
+        vq = ctz32(tmp) + 1;
 
         max_vq = vq <= ARM_MAX_VQ ? vq - 1 : ARM_MAX_VQ;
-        bitmap_andnot(cpu->sve_vq_map, cpu->sve_vq_supported,
-                      cpu->sve_vq_init, max_vq);
-        if (max_vq == 0 || bitmap_empty(cpu->sve_vq_map, max_vq)) {
+        vq_mask = MAKE_64BIT_MASK(0, max_vq);
+        vq_map = vq_supported & ~vq_init & vq_mask;
+
+        if (max_vq == 0 || vq_map == 0) {
             error_setg(errp, "cannot disable sve%d", vq * 128);
             error_append_hint(errp, "Disabling sve%d results in all "
                               "vector lengths being disabled.\n",
@@ -441,7 +439,8 @@ void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp)
             return;
         }
 
-        max_vq = find_last_bit(cpu->sve_vq_map, max_vq) + 1;
+        max_vq = 32 - clz32(vq_map);
+        vq_mask = MAKE_64BIT_MASK(0, max_vq);
     }
 
     /*
@@ -451,9 +450,9 @@ void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp)
      */
     if (cpu->sve_max_vq != 0) {
         max_vq = cpu->sve_max_vq;
+        vq_mask = MAKE_64BIT_MASK(0, max_vq);
 
-        if (!test_bit(max_vq - 1, cpu->sve_vq_map) &&
-            test_bit(max_vq - 1, cpu->sve_vq_init)) {
+        if (vq_init & ~vq_map & (1 << (max_vq - 1))) {
             error_setg(errp, "cannot disable sve%d", max_vq * 128);
             error_append_hint(errp, "The maximum vector length must be "
                               "enabled, sve-max-vq=%d (%d bits)\n",
@@ -462,8 +461,7 @@ void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp)
         }
 
         /* Set all bits not explicitly set within sve-max-vq. */
-        bitmap_complement(tmp, cpu->sve_vq_init, max_vq);
-        bitmap_or(cpu->sve_vq_map, cpu->sve_vq_map, tmp, max_vq);
+        vq_map |= ~vq_init & vq_mask;
     }
 
     /*
@@ -472,13 +470,14 @@ void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp)
      * are clear, just in case anybody looks.
      */
     assert(max_vq != 0);
-    bitmap_clear(cpu->sve_vq_map, max_vq, ARM_MAX_VQ - max_vq);
+    assert(vq_mask != 0);
+    vq_map &= vq_mask;
 
     /* Ensure the set of lengths matches what is supported. */
-    bitmap_xor(tmp, cpu->sve_vq_map, cpu->sve_vq_supported, max_vq);
-    if (!bitmap_empty(tmp, max_vq)) {
-        vq = find_last_bit(tmp, max_vq) + 1;
-        if (test_bit(vq - 1, cpu->sve_vq_map)) {
+    tmp = vq_map ^ (vq_supported & vq_mask);
+    if (tmp) {
+        vq = 32 - clz32(tmp);
+        if (vq_map & (1 << (vq - 1))) {
             if (cpu->sve_max_vq) {
                 error_setg(errp, "cannot set sve-max-vq=%d", cpu->sve_max_vq);
                 error_append_hint(errp, "This CPU does not support "
@@ -502,15 +501,15 @@ void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp)
                 return;
             } else {
                 /* Ensure all required powers-of-two are enabled. */
-                for (vq = pow2floor(max_vq); vq >= 1; vq >>= 1) {
-                    if (!test_bit(vq - 1, cpu->sve_vq_map)) {
-                        error_setg(errp, "cannot disable sve%d", vq * 128);
-                        error_append_hint(errp, "sve%d is required as it "
-                                          "is a power-of-two length smaller "
-                                          "than the maximum, sve%d\n",
-                                          vq * 128, max_vq * 128);
-                        return;
-                    }
+                tmp = SVE_VQ_POW2_MAP & vq_mask & ~vq_map;
+                if (tmp) {
+                    vq = 32 - clz32(tmp);
+                    error_setg(errp, "cannot disable sve%d", vq * 128);
+                    error_append_hint(errp, "sve%d is required as it "
+                                      "is a power-of-two length smaller "
+                                      "than the maximum, sve%d\n",
+                                      vq * 128, max_vq * 128);
+                    return;
                 }
             }
         }
@@ -530,6 +529,7 @@ void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp)
 
     /* From now on sve_max_vq is the actual maximum supported length. */
     cpu->sve_max_vq = max_vq;
+    cpu->sve_vq_map = vq_map;
 }
 
 static void cpu_max_get_sve_max_vq(Object *obj, Visitor *v, const char *name,
@@ -590,7 +590,7 @@ static void cpu_arm_get_sve_vq(Object *obj, Visitor *v, const char *name,
     if (!cpu_isar_feature(aa64_sve, cpu)) {
         value = false;
     } else {
-        value = test_bit(vq - 1, cpu->sve_vq_map);
+        value = extract32(cpu->sve_vq_map, vq - 1, 1);
     }
     visit_type_bool(v, name, &value, errp);
 }
@@ -612,12 +612,8 @@ static void cpu_arm_set_sve_vq(Object *obj, Visitor *v, const char *name,
         return;
     }
 
-    if (value) {
-        set_bit(vq - 1, cpu->sve_vq_map);
-    } else {
-        clear_bit(vq - 1, cpu->sve_vq_map);
-    }
-    set_bit(vq - 1, cpu->sve_vq_init);
+    cpu->sve_vq_map = deposit32(cpu->sve_vq_map, vq - 1, 1, value);
+    cpu->sve_vq_init |= 1 << (vq - 1);
 }
 
 static bool cpu_arm_get_sve(Object *obj, Error **errp)
@@ -979,7 +975,7 @@ static void aarch64_max_initfn(Object *obj)
     cpu->dcz_blocksize = 7; /*  512 bytes */
 #endif
 
-    bitmap_fill(cpu->sve_vq_supported, ARM_MAX_VQ);
+    cpu->sve_vq_supported = MAKE_64BIT_MASK(0, ARM_MAX_VQ);
 
     aarch64_add_pauth_properties(obj);
     aarch64_add_sve_properties(obj);
@@ -1026,12 +1022,11 @@ static void aarch64_a64fx_initfn(Object *obj)
     cpu->gic_vprebits = 5;
     cpu->gic_pribits = 5;
 
-    /* Suppport of A64FX's vector length are 128,256 and 512bit only */
+    /* The A64FX supports only 128, 256 and 512 bit vector lengths */
     aarch64_add_sve_properties(obj);
-    bitmap_zero(cpu->sve_vq_supported, ARM_MAX_VQ);
-    set_bit(0, cpu->sve_vq_supported); /* 128bit */
-    set_bit(1, cpu->sve_vq_supported); /* 256bit */
-    set_bit(3, cpu->sve_vq_supported); /* 512bit */
+    cpu->sve_vq_supported = (1 << 0)  /* 128bit */
+                          | (1 << 1)  /* 256bit */
+                          | (1 << 3); /* 512bit */
 
     cpu->isar.reset_pmcr_el0 = 0x46014040;
 
diff --git a/target/arm/helper.c b/target/arm/helper.c
index de159c644c..90aac6bc12 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -6219,7 +6219,6 @@ uint32_t sve_zcr_len_for_el(CPUARMState *env, int el)
 {
     ARMCPU *cpu = env_archcpu(env);
     uint32_t len = cpu->sve_max_vq - 1;
-    uint32_t end_len;
 
     if (el <= 1 && !el_is_in_host(env, el)) {
         len = MIN(len, 0xf & (uint32_t)env->vfp.zcr_el[1]);
@@ -6231,12 +6230,8 @@ uint32_t sve_zcr_len_for_el(CPUARMState *env, int el)
         len = MIN(len, 0xf & (uint32_t)env->vfp.zcr_el[3]);
     }
 
-    end_len = len;
-    if (!test_bit(len, cpu->sve_vq_map)) {
-        end_len = find_last_bit(cpu->sve_vq_map, len);
-        assert(end_len < len);
-    }
-    return end_len;
+    len = 31 - clz32(cpu->sve_vq_map & MAKE_64BIT_MASK(0, len + 1));
+    return len;
 }
 
 static void zcr_write(CPUARMState *env, const ARMCPRegInfo *ri,
diff --git a/target/arm/internals.h b/target/arm/internals.h
index 79eb463753..a1bae4588a 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -1340,4 +1340,9 @@ bool el_is_in_host(CPUARMState *env, int el);
 
 void aa32_max_features(ARMCPU *cpu);
 
+/* Powers of 2 for sve_vq_map et al. */
+#define SVE_VQ_POW2_MAP                                 \
+    ((1 << (1 - 1)) | (1 << (2 - 1)) |                  \
+     (1 << (4 - 1)) | (1 << (8 - 1)) | (1 << (16 - 1)))
+
 #endif
diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c
index 363032da90..b3f635fc95 100644
--- a/target/arm/kvm64.c
+++ b/target/arm/kvm64.c
@@ -760,15 +760,13 @@ bool kvm_arm_steal_time_supported(void)
 
 QEMU_BUILD_BUG_ON(KVM_ARM64_SVE_VQ_MIN != 1);
 
-void kvm_arm_sve_get_vls(CPUState *cs, unsigned long *map)
+uint32_t kvm_arm_sve_get_vls(CPUState *cs)
 {
     /* Only call this function if kvm_arm_sve_supported() returns true. */
     static uint64_t vls[KVM_ARM64_SVE_VLS_WORDS];
     static bool probed;
     uint32_t vq = 0;
-    int i, j;
-
-    bitmap_zero(map, ARM_MAX_VQ);
+    int i;
 
     /*
      * KVM ensures all host CPUs support the same set of vector lengths.
@@ -809,46 +807,24 @@ void kvm_arm_sve_get_vls(CPUState *cs, unsigned long *map)
         if (vq > ARM_MAX_VQ) {
             warn_report("KVM supports vector lengths larger than "
                         "QEMU can enable");
+            vls[0] &= MAKE_64BIT_MASK(0, ARM_MAX_VQ);
         }
     }
 
-    for (i = 0; i < KVM_ARM64_SVE_VLS_WORDS; ++i) {
-        if (!vls[i]) {
-            continue;
-        }
-        for (j = 1; j <= 64; ++j) {
-            vq = j + i * 64;
-            if (vq > ARM_MAX_VQ) {
-                return;
-            }
-            if (vls[i] & (1UL << (j - 1))) {
-                set_bit(vq - 1, map);
-            }
-        }
-    }
+    return vls[0];
 }
 
 static int kvm_arm_sve_set_vls(CPUState *cs)
 {
-    uint64_t vls[KVM_ARM64_SVE_VLS_WORDS] = {0};
+    ARMCPU *cpu = ARM_CPU(cs);
+    uint64_t vls[KVM_ARM64_SVE_VLS_WORDS] = { cpu->sve_vq_map };
     struct kvm_one_reg reg = {
         .id = KVM_REG_ARM64_SVE_VLS,
         .addr = (uint64_t)&vls[0],
     };
-    ARMCPU *cpu = ARM_CPU(cs);
-    uint32_t vq;
-    int i, j;
 
     assert(cpu->sve_max_vq <= KVM_ARM64_SVE_VQ_MAX);
 
-    for (vq = 1; vq <= cpu->sve_max_vq; ++vq) {
-        if (test_bit(vq - 1, cpu->sve_vq_map)) {
-            i = (vq - 1) / 64;
-            j = (vq - 1) % 64;
-            vls[i] |= 1UL << j;
-        }
-    }
-
     return kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &reg);
 }
 
diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h
index b7f78b5215..99017b635c 100644
--- a/target/arm/kvm_arm.h
+++ b/target/arm/kvm_arm.h
@@ -239,13 +239,12 @@ bool kvm_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf);
 /**
  * kvm_arm_sve_get_vls:
  * @cs: CPUState
- * @map: bitmap to fill in
  *
  * Get all the SVE vector lengths supported by the KVM host, setting
  * the bits corresponding to their length in quadwords minus one
- * (vq - 1) in @map up to ARM_MAX_VQ.
+ * (vq - 1) up to ARM_MAX_VQ.  Return the resulting map.
  */
-void kvm_arm_sve_get_vls(CPUState *cs, unsigned long *map);
+uint32_t kvm_arm_sve_get_vls(CPUState *cs);
 
 /**
  * kvm_arm_set_cpu_features_from_host:
@@ -439,7 +438,7 @@ static inline void kvm_arm_steal_time_finalize(ARMCPU *cpu, Error **errp)
     g_assert_not_reached();
 }
 
-static inline void kvm_arm_sve_get_vls(CPUState *cs, unsigned long *map)
+static inline uint32_t kvm_arm_sve_get_vls(CPUState *cs)
 {
     g_assert_not_reached();
 }

From 5ef3cc563699d9b96e6b1acb15c7c02cf75d8266 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:57 +0100
Subject: [PATCH 899/935] target/arm: Rename sve_zcr_len_for_el to
 sve_vqm1_for_el

This will be used for both Normal and Streaming SVE, and the value
does not necessarily come from ZCR_ELx.  While we're at it, emphasize
the units in which the value is returned.

Patch produced by
    git grep -l sve_zcr_len_for_el | \
    xargs -n1 sed -i 's/sve_zcr_len_for_el/sve_vqm1_for_el/g'

and then adding a function comment.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-13-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/arch_dump.c |  2 +-
 target/arm/cpu.c       |  2 +-
 target/arm/cpu.h       | 11 ++++++++++-
 target/arm/gdbstub64.c |  2 +-
 target/arm/helper.c    | 12 ++++++------
 5 files changed, 19 insertions(+), 10 deletions(-)

diff --git a/target/arm/arch_dump.c b/target/arm/arch_dump.c
index 0184845310..b1f040e69f 100644
--- a/target/arm/arch_dump.c
+++ b/target/arm/arch_dump.c
@@ -166,7 +166,7 @@ static off_t sve_fpcr_offset(uint32_t vq)
 
 static uint32_t sve_current_vq(CPUARMState *env)
 {
-    return sve_zcr_len_for_el(env, arm_current_el(env)) + 1;
+    return sve_vqm1_for_el(env, arm_current_el(env)) + 1;
 }
 
 static size_t sve_size_vq(uint32_t vq)
diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index 0621944167..1b5d535788 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -925,7 +925,7 @@ static void aarch64_cpu_dump_state(CPUState *cs, FILE *f, int flags)
                  vfp_get_fpcr(env), vfp_get_fpsr(env));
 
     if (cpu_isar_feature(aa64_sve, cpu) && sve_exception_el(env, el) == 0) {
-        int j, zcr_len = sve_zcr_len_for_el(env, el);
+        int j, zcr_len = sve_vqm1_for_el(env, el);
 
         for (i = 0; i <= FFR_PRED_NUM; i++) {
             bool eol;
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 73f24a5760..e45b5cb7fe 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -1132,7 +1132,16 @@ void aarch64_sync_64_to_32(CPUARMState *env);
 
 int fp_exception_el(CPUARMState *env, int cur_el);
 int sve_exception_el(CPUARMState *env, int cur_el);
-uint32_t sve_zcr_len_for_el(CPUARMState *env, int el);
+
+/**
+ * sve_vqm1_for_el:
+ * @env: CPUARMState
+ * @el: exception level
+ *
+ * Compute the current SVE vector length for @el, in units of
+ * Quadwords Minus 1 -- the same scale used for ZCR_ELx.LEN.
+ */
+uint32_t sve_vqm1_for_el(CPUARMState *env, int el);
 
 static inline bool is_a64(CPUARMState *env)
 {
diff --git a/target/arm/gdbstub64.c b/target/arm/gdbstub64.c
index 596878666d..07a6746944 100644
--- a/target/arm/gdbstub64.c
+++ b/target/arm/gdbstub64.c
@@ -152,7 +152,7 @@ int arm_gdb_get_svereg(CPUARMState *env, GByteArray *buf, int reg)
          * We report in Vector Granules (VG) which is 64bit in a Z reg
          * while the ZCR works in Vector Quads (VQ) which is 128bit chunks.
          */
-        int vq = sve_zcr_len_for_el(env, arm_current_el(env)) + 1;
+        int vq = sve_vqm1_for_el(env, arm_current_el(env)) + 1;
         return gdb_get_reg64(buf, vq * 2);
     }
     default:
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 90aac6bc12..400f7cd1db 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -6215,7 +6215,7 @@ int sve_exception_el(CPUARMState *env, int el)
 /*
  * Given that SVE is enabled, return the vector length for EL.
  */
-uint32_t sve_zcr_len_for_el(CPUARMState *env, int el)
+uint32_t sve_vqm1_for_el(CPUARMState *env, int el)
 {
     ARMCPU *cpu = env_archcpu(env);
     uint32_t len = cpu->sve_max_vq - 1;
@@ -6238,7 +6238,7 @@ static void zcr_write(CPUARMState *env, const ARMCPRegInfo *ri,
                       uint64_t value)
 {
     int cur_el = arm_current_el(env);
-    int old_len = sve_zcr_len_for_el(env, cur_el);
+    int old_len = sve_vqm1_for_el(env, cur_el);
     int new_len;
 
     /* Bits other than [3:0] are RAZ/WI.  */
@@ -6249,7 +6249,7 @@ static void zcr_write(CPUARMState *env, const ARMCPRegInfo *ri,
      * Because we arrived here, we know both FP and SVE are enabled;
      * otherwise we would have trapped access to the ZCR_ELn register.
      */
-    new_len = sve_zcr_len_for_el(env, cur_el);
+    new_len = sve_vqm1_for_el(env, cur_el);
     if (new_len < old_len) {
         aarch64_sve_narrow_vq(env, new_len + 1);
     }
@@ -11168,7 +11168,7 @@ static CPUARMTBFlags rebuild_hflags_a64(CPUARMState *env, int el, int fp_el,
                 sve_el = 0;
             }
         } else if (sve_el == 0) {
-            DP_TBFLAG_A64(flags, VL, sve_zcr_len_for_el(env, el));
+            DP_TBFLAG_A64(flags, VL, sve_vqm1_for_el(env, el));
         }
         DP_TBFLAG_A64(flags, SVEEXC_EL, sve_el);
     }
@@ -11534,10 +11534,10 @@ void aarch64_sve_change_el(CPUARMState *env, int old_el,
      */
     old_a64 = old_el ? arm_el_is_aa64(env, old_el) : el0_a64;
     old_len = (old_a64 && !sve_exception_el(env, old_el)
-               ? sve_zcr_len_for_el(env, old_el) : 0);
+               ? sve_vqm1_for_el(env, old_el) : 0);
     new_a64 = new_el ? arm_el_is_aa64(env, new_el) : el0_a64;
     new_len = (new_a64 && !sve_exception_el(env, new_el)
-               ? sve_zcr_len_for_el(env, new_el) : 0);
+               ? sve_vqm1_for_el(env, new_el) : 0);
 
     /* When changing vector length, clear inaccessible state.  */
     if (new_len < old_len) {

From d1ce862602ac23de7b3e8541507fc78a8c4a5766 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:57 +0100
Subject: [PATCH 900/935] target/arm: Split out load/store primitives to
 sve_ldst_internal.h

Begin creation of sve_ldst_internal.h by moving the primitives
that access host and tlb memory.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-14-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/sve_helper.c        | 107 +--------------------------
 target/arm/sve_ldst_internal.h | 127 +++++++++++++++++++++++++++++++++
 2 files changed, 128 insertions(+), 106 deletions(-)
 create mode 100644 target/arm/sve_ldst_internal.h

diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c
index 3bdcd4ce9d..0c6dde00aa 100644
--- a/target/arm/sve_helper.c
+++ b/target/arm/sve_helper.c
@@ -21,12 +21,12 @@
 #include "cpu.h"
 #include "internals.h"
 #include "exec/exec-all.h"
-#include "exec/cpu_ldst.h"
 #include "exec/helper-proto.h"
 #include "tcg/tcg-gvec-desc.h"
 #include "fpu/softfloat.h"
 #include "tcg/tcg.h"
 #include "vec_internal.h"
+#include "sve_ldst_internal.h"
 
 
 /* Return a value for NZCV as per the ARM PredTest pseudofunction.
@@ -5301,111 +5301,6 @@ void HELPER(sve_fcmla_zpzzz_d)(void *vd, void *vn, void *vm, void *va,
  * Load contiguous data, protected by a governing predicate.
  */
 
-/*
- * Load one element into @vd + @reg_off from @host.
- * The controlling predicate is known to be true.
- */
-typedef void sve_ldst1_host_fn(void *vd, intptr_t reg_off, void *host);
-
-/*
- * Load one element into @vd + @reg_off from (@env, @vaddr, @ra).
- * The controlling predicate is known to be true.
- */
-typedef void sve_ldst1_tlb_fn(CPUARMState *env, void *vd, intptr_t reg_off,
-                              target_ulong vaddr, uintptr_t retaddr);
-
-/*
- * Generate the above primitives.
- */
-
-#define DO_LD_HOST(NAME, H, TYPEE, TYPEM, HOST) \
-static void sve_##NAME##_host(void *vd, intptr_t reg_off, void *host)  \
-{                                                                      \
-    TYPEM val = HOST(host);                                            \
-    *(TYPEE *)(vd + H(reg_off)) = val;                                 \
-}
-
-#define DO_ST_HOST(NAME, H, TYPEE, TYPEM, HOST) \
-static void sve_##NAME##_host(void *vd, intptr_t reg_off, void *host)  \
-{ HOST(host, (TYPEM)*(TYPEE *)(vd + H(reg_off))); }
-
-#define DO_LD_TLB(NAME, H, TYPEE, TYPEM, TLB) \
-static void sve_##NAME##_tlb(CPUARMState *env, void *vd, intptr_t reg_off,  \
-                             target_ulong addr, uintptr_t ra)               \
-{                                                                           \
-    *(TYPEE *)(vd + H(reg_off)) =                                           \
-        (TYPEM)TLB(env, useronly_clean_ptr(addr), ra);                      \
-}
-
-#define DO_ST_TLB(NAME, H, TYPEE, TYPEM, TLB) \
-static void sve_##NAME##_tlb(CPUARMState *env, void *vd, intptr_t reg_off,  \
-                             target_ulong addr, uintptr_t ra)               \
-{                                                                           \
-    TLB(env, useronly_clean_ptr(addr),                                      \
-        (TYPEM)*(TYPEE *)(vd + H(reg_off)), ra);                            \
-}
-
-#define DO_LD_PRIM_1(NAME, H, TE, TM)                   \
-    DO_LD_HOST(NAME, H, TE, TM, ldub_p)                 \
-    DO_LD_TLB(NAME, H, TE, TM, cpu_ldub_data_ra)
-
-DO_LD_PRIM_1(ld1bb,  H1,   uint8_t,  uint8_t)
-DO_LD_PRIM_1(ld1bhu, H1_2, uint16_t, uint8_t)
-DO_LD_PRIM_1(ld1bhs, H1_2, uint16_t,  int8_t)
-DO_LD_PRIM_1(ld1bsu, H1_4, uint32_t, uint8_t)
-DO_LD_PRIM_1(ld1bss, H1_4, uint32_t,  int8_t)
-DO_LD_PRIM_1(ld1bdu, H1_8, uint64_t, uint8_t)
-DO_LD_PRIM_1(ld1bds, H1_8, uint64_t,  int8_t)
-
-#define DO_ST_PRIM_1(NAME, H, TE, TM)                   \
-    DO_ST_HOST(st1##NAME, H, TE, TM, stb_p)             \
-    DO_ST_TLB(st1##NAME, H, TE, TM, cpu_stb_data_ra)
-
-DO_ST_PRIM_1(bb,   H1,  uint8_t, uint8_t)
-DO_ST_PRIM_1(bh, H1_2, uint16_t, uint8_t)
-DO_ST_PRIM_1(bs, H1_4, uint32_t, uint8_t)
-DO_ST_PRIM_1(bd, H1_8, uint64_t, uint8_t)
-
-#define DO_LD_PRIM_2(NAME, H, TE, TM, LD) \
-    DO_LD_HOST(ld1##NAME##_be, H, TE, TM, LD##_be_p)    \
-    DO_LD_HOST(ld1##NAME##_le, H, TE, TM, LD##_le_p)    \
-    DO_LD_TLB(ld1##NAME##_be, H, TE, TM, cpu_##LD##_be_data_ra) \
-    DO_LD_TLB(ld1##NAME##_le, H, TE, TM, cpu_##LD##_le_data_ra)
-
-#define DO_ST_PRIM_2(NAME, H, TE, TM, ST) \
-    DO_ST_HOST(st1##NAME##_be, H, TE, TM, ST##_be_p)    \
-    DO_ST_HOST(st1##NAME##_le, H, TE, TM, ST##_le_p)    \
-    DO_ST_TLB(st1##NAME##_be, H, TE, TM, cpu_##ST##_be_data_ra) \
-    DO_ST_TLB(st1##NAME##_le, H, TE, TM, cpu_##ST##_le_data_ra)
-
-DO_LD_PRIM_2(hh,  H1_2, uint16_t, uint16_t, lduw)
-DO_LD_PRIM_2(hsu, H1_4, uint32_t, uint16_t, lduw)
-DO_LD_PRIM_2(hss, H1_4, uint32_t,  int16_t, lduw)
-DO_LD_PRIM_2(hdu, H1_8, uint64_t, uint16_t, lduw)
-DO_LD_PRIM_2(hds, H1_8, uint64_t,  int16_t, lduw)
-
-DO_ST_PRIM_2(hh, H1_2, uint16_t, uint16_t, stw)
-DO_ST_PRIM_2(hs, H1_4, uint32_t, uint16_t, stw)
-DO_ST_PRIM_2(hd, H1_8, uint64_t, uint16_t, stw)
-
-DO_LD_PRIM_2(ss,  H1_4, uint32_t, uint32_t, ldl)
-DO_LD_PRIM_2(sdu, H1_8, uint64_t, uint32_t, ldl)
-DO_LD_PRIM_2(sds, H1_8, uint64_t,  int32_t, ldl)
-
-DO_ST_PRIM_2(ss, H1_4, uint32_t, uint32_t, stl)
-DO_ST_PRIM_2(sd, H1_8, uint64_t, uint32_t, stl)
-
-DO_LD_PRIM_2(dd, H1_8, uint64_t, uint64_t, ldq)
-DO_ST_PRIM_2(dd, H1_8, uint64_t, uint64_t, stq)
-
-#undef DO_LD_TLB
-#undef DO_ST_TLB
-#undef DO_LD_HOST
-#undef DO_LD_PRIM_1
-#undef DO_ST_PRIM_1
-#undef DO_LD_PRIM_2
-#undef DO_ST_PRIM_2
-
 /*
  * Skip through a sequence of inactive elements in the guarding predicate @vg,
  * beginning at @reg_off bounded by @reg_max.  Return the offset of the active
diff --git a/target/arm/sve_ldst_internal.h b/target/arm/sve_ldst_internal.h
new file mode 100644
index 0000000000..ef9117e84c
--- /dev/null
+++ b/target/arm/sve_ldst_internal.h
@@ -0,0 +1,127 @@
+/*
+ * ARM SVE Load/Store Helpers
+ *
+ * Copyright (c) 2018-2022 Linaro
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef TARGET_ARM_SVE_LDST_INTERNAL_H
+#define TARGET_ARM_SVE_LDST_INTERNAL_H
+
+#include "exec/cpu_ldst.h"
+
+/*
+ * Load one element into @vd + @reg_off from @host.
+ * The controlling predicate is known to be true.
+ */
+typedef void sve_ldst1_host_fn(void *vd, intptr_t reg_off, void *host);
+
+/*
+ * Load one element into @vd + @reg_off from (@env, @vaddr, @ra).
+ * The controlling predicate is known to be true.
+ */
+typedef void sve_ldst1_tlb_fn(CPUARMState *env, void *vd, intptr_t reg_off,
+                              target_ulong vaddr, uintptr_t retaddr);
+
+/*
+ * Generate the above primitives.
+ */
+
+#define DO_LD_HOST(NAME, H, TYPEE, TYPEM, HOST)                              \
+static inline void sve_##NAME##_host(void *vd, intptr_t reg_off, void *host) \
+{ TYPEM val = HOST(host); *(TYPEE *)(vd + H(reg_off)) = val; }
+
+#define DO_ST_HOST(NAME, H, TYPEE, TYPEM, HOST)                              \
+static inline void sve_##NAME##_host(void *vd, intptr_t reg_off, void *host) \
+{ TYPEM val = *(TYPEE *)(vd + H(reg_off)); HOST(host, val); }
+
+#define DO_LD_TLB(NAME, H, TYPEE, TYPEM, TLB)                              \
+static inline void sve_##NAME##_tlb(CPUARMState *env, void *vd,            \
+                        intptr_t reg_off, target_ulong addr, uintptr_t ra) \
+{                                                                          \
+    TYPEM val = TLB(env, useronly_clean_ptr(addr), ra);                    \
+    *(TYPEE *)(vd + H(reg_off)) = val;                                     \
+}
+
+#define DO_ST_TLB(NAME, H, TYPEE, TYPEM, TLB)                              \
+static inline void sve_##NAME##_tlb(CPUARMState *env, void *vd,            \
+                        intptr_t reg_off, target_ulong addr, uintptr_t ra) \
+{                                                                          \
+    TYPEM val = *(TYPEE *)(vd + H(reg_off));                               \
+    TLB(env, useronly_clean_ptr(addr), val, ra);                           \
+}
+
+#define DO_LD_PRIM_1(NAME, H, TE, TM)                   \
+    DO_LD_HOST(NAME, H, TE, TM, ldub_p)                 \
+    DO_LD_TLB(NAME, H, TE, TM, cpu_ldub_data_ra)
+
+DO_LD_PRIM_1(ld1bb,  H1,   uint8_t,  uint8_t)
+DO_LD_PRIM_1(ld1bhu, H1_2, uint16_t, uint8_t)
+DO_LD_PRIM_1(ld1bhs, H1_2, uint16_t,  int8_t)
+DO_LD_PRIM_1(ld1bsu, H1_4, uint32_t, uint8_t)
+DO_LD_PRIM_1(ld1bss, H1_4, uint32_t,  int8_t)
+DO_LD_PRIM_1(ld1bdu, H1_8, uint64_t, uint8_t)
+DO_LD_PRIM_1(ld1bds, H1_8, uint64_t,  int8_t)
+
+#define DO_ST_PRIM_1(NAME, H, TE, TM)                   \
+    DO_ST_HOST(st1##NAME, H, TE, TM, stb_p)             \
+    DO_ST_TLB(st1##NAME, H, TE, TM, cpu_stb_data_ra)
+
+DO_ST_PRIM_1(bb,   H1,  uint8_t, uint8_t)
+DO_ST_PRIM_1(bh, H1_2, uint16_t, uint8_t)
+DO_ST_PRIM_1(bs, H1_4, uint32_t, uint8_t)
+DO_ST_PRIM_1(bd, H1_8, uint64_t, uint8_t)
+
+#define DO_LD_PRIM_2(NAME, H, TE, TM, LD) \
+    DO_LD_HOST(ld1##NAME##_be, H, TE, TM, LD##_be_p)    \
+    DO_LD_HOST(ld1##NAME##_le, H, TE, TM, LD##_le_p)    \
+    DO_LD_TLB(ld1##NAME##_be, H, TE, TM, cpu_##LD##_be_data_ra) \
+    DO_LD_TLB(ld1##NAME##_le, H, TE, TM, cpu_##LD##_le_data_ra)
+
+#define DO_ST_PRIM_2(NAME, H, TE, TM, ST) \
+    DO_ST_HOST(st1##NAME##_be, H, TE, TM, ST##_be_p)    \
+    DO_ST_HOST(st1##NAME##_le, H, TE, TM, ST##_le_p)    \
+    DO_ST_TLB(st1##NAME##_be, H, TE, TM, cpu_##ST##_be_data_ra) \
+    DO_ST_TLB(st1##NAME##_le, H, TE, TM, cpu_##ST##_le_data_ra)
+
+DO_LD_PRIM_2(hh,  H1_2, uint16_t, uint16_t, lduw)
+DO_LD_PRIM_2(hsu, H1_4, uint32_t, uint16_t, lduw)
+DO_LD_PRIM_2(hss, H1_4, uint32_t,  int16_t, lduw)
+DO_LD_PRIM_2(hdu, H1_8, uint64_t, uint16_t, lduw)
+DO_LD_PRIM_2(hds, H1_8, uint64_t,  int16_t, lduw)
+
+DO_ST_PRIM_2(hh, H1_2, uint16_t, uint16_t, stw)
+DO_ST_PRIM_2(hs, H1_4, uint32_t, uint16_t, stw)
+DO_ST_PRIM_2(hd, H1_8, uint64_t, uint16_t, stw)
+
+DO_LD_PRIM_2(ss,  H1_4, uint32_t, uint32_t, ldl)
+DO_LD_PRIM_2(sdu, H1_8, uint64_t, uint32_t, ldl)
+DO_LD_PRIM_2(sds, H1_8, uint64_t,  int32_t, ldl)
+
+DO_ST_PRIM_2(ss, H1_4, uint32_t, uint32_t, stl)
+DO_ST_PRIM_2(sd, H1_8, uint64_t, uint32_t, stl)
+
+DO_LD_PRIM_2(dd, H1_8, uint64_t, uint64_t, ldq)
+DO_ST_PRIM_2(dd, H1_8, uint64_t, uint64_t, stq)
+
+#undef DO_LD_TLB
+#undef DO_ST_TLB
+#undef DO_LD_HOST
+#undef DO_LD_PRIM_1
+#undef DO_ST_PRIM_1
+#undef DO_LD_PRIM_2
+#undef DO_ST_PRIM_2
+
+#endif /* TARGET_ARM_SVE_LDST_INTERNAL_H */

From 0b68112b399d77c765a0a5d9bb4f4b4e57541ee7 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:58 +0100
Subject: [PATCH 901/935] target/arm: Export sve contiguous ldst support
 functions

Export all of the support functions for performing bulk
fault analysis on a set of elements at contiguous addresses
controlled by a predicate.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-15-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/sve_helper.c        | 87 ++++++-------------------------
 target/arm/sve_ldst_internal.h | 94 ++++++++++++++++++++++++++++++++++
 2 files changed, 111 insertions(+), 70 deletions(-)

diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c
index 0c6dde00aa..8cd371e3e3 100644
--- a/target/arm/sve_helper.c
+++ b/target/arm/sve_helper.c
@@ -5341,16 +5341,9 @@ static intptr_t find_next_active(uint64_t *vg, intptr_t reg_off,
  * exit via page fault exception.
  */
 
-typedef struct {
-    void *host;
-    int flags;
-    MemTxAttrs attrs;
-} SVEHostPage;
-
-static bool sve_probe_page(SVEHostPage *info, bool nofault,
-                           CPUARMState *env, target_ulong addr,
-                           int mem_off, MMUAccessType access_type,
-                           int mmu_idx, uintptr_t retaddr)
+bool sve_probe_page(SVEHostPage *info, bool nofault, CPUARMState *env,
+                    target_ulong addr, int mem_off, MMUAccessType access_type,
+                    int mmu_idx, uintptr_t retaddr)
 {
     int flags;
 
@@ -5406,59 +5399,13 @@ static bool sve_probe_page(SVEHostPage *info, bool nofault,
     return true;
 }
 
-
-/*
- * Analyse contiguous data, protected by a governing predicate.
- */
-
-typedef enum {
-    FAULT_NO,
-    FAULT_FIRST,
-    FAULT_ALL,
-} SVEContFault;
-
-typedef struct {
-    /*
-     * First and last element wholly contained within the two pages.
-     * mem_off_first[0] and reg_off_first[0] are always set >= 0.
-     * reg_off_last[0] may be < 0 if the first element crosses pages.
-     * All of mem_off_first[1], reg_off_first[1] and reg_off_last[1]
-     * are set >= 0 only if there are complete elements on a second page.
-     *
-     * The reg_off_* offsets are relative to the internal vector register.
-     * The mem_off_first offset is relative to the memory address; the
-     * two offsets are different when a load operation extends, a store
-     * operation truncates, or for multi-register operations.
-     */
-    int16_t mem_off_first[2];
-    int16_t reg_off_first[2];
-    int16_t reg_off_last[2];
-
-    /*
-     * One element that is misaligned and spans both pages,
-     * or -1 if there is no such active element.
-     */
-    int16_t mem_off_split;
-    int16_t reg_off_split;
-
-    /*
-     * The byte offset at which the entire operation crosses a page boundary.
-     * Set >= 0 if and only if the entire operation spans two pages.
-     */
-    int16_t page_split;
-
-    /* TLB data for the two pages. */
-    SVEHostPage page[2];
-} SVEContLdSt;
-
 /*
  * Find first active element on each page, and a loose bound for the
  * final element on each page.  Identify any single element that spans
  * the page boundary.  Return true if there are any active elements.
  */
-static bool sve_cont_ldst_elements(SVEContLdSt *info, target_ulong addr,
-                                   uint64_t *vg, intptr_t reg_max,
-                                   int esz, int msize)
+bool sve_cont_ldst_elements(SVEContLdSt *info, target_ulong addr, uint64_t *vg,
+                            intptr_t reg_max, int esz, int msize)
 {
     const int esize = 1 << esz;
     const uint64_t pg_mask = pred_esz_masks[esz];
@@ -5548,9 +5495,9 @@ static bool sve_cont_ldst_elements(SVEContLdSt *info, target_ulong addr,
  * Control the generation of page faults with @fault.  Return false if
  * there is no work to do, which can only happen with @fault == FAULT_NO.
  */
-static bool sve_cont_ldst_pages(SVEContLdSt *info, SVEContFault fault,
-                                CPUARMState *env, target_ulong addr,
-                                MMUAccessType access_type, uintptr_t retaddr)
+bool sve_cont_ldst_pages(SVEContLdSt *info, SVEContFault fault,
+                         CPUARMState *env, target_ulong addr,
+                         MMUAccessType access_type, uintptr_t retaddr)
 {
     int mmu_idx = cpu_mmu_index(env, false);
     int mem_off = info->mem_off_first[0];
@@ -5606,12 +5553,12 @@ static bool sve_cont_ldst_pages(SVEContLdSt *info, SVEContFault fault,
     return have_work;
 }
 
-static void sve_cont_ldst_watchpoints(SVEContLdSt *info, CPUARMState *env,
-                                      uint64_t *vg, target_ulong addr,
-                                      int esize, int msize, int wp_access,
-                                      uintptr_t retaddr)
-{
 #ifndef CONFIG_USER_ONLY
+void sve_cont_ldst_watchpoints(SVEContLdSt *info, CPUARMState *env,
+                               uint64_t *vg, target_ulong addr,
+                               int esize, int msize, int wp_access,
+                               uintptr_t retaddr)
+{
     intptr_t mem_off, reg_off, reg_last;
     int flags0 = info->page[0].flags;
     int flags1 = info->page[1].flags;
@@ -5667,12 +5614,12 @@ static void sve_cont_ldst_watchpoints(SVEContLdSt *info, CPUARMState *env,
             } while (reg_off & 63);
         } while (reg_off <= reg_last);
     }
-#endif
 }
+#endif
 
-static void sve_cont_ldst_mte_check(SVEContLdSt *info, CPUARMState *env,
-                                    uint64_t *vg, target_ulong addr, int esize,
-                                    int msize, uint32_t mtedesc, uintptr_t ra)
+void sve_cont_ldst_mte_check(SVEContLdSt *info, CPUARMState *env,
+                             uint64_t *vg, target_ulong addr, int esize,
+                             int msize, uint32_t mtedesc, uintptr_t ra)
 {
     intptr_t mem_off, reg_off, reg_last;
 
diff --git a/target/arm/sve_ldst_internal.h b/target/arm/sve_ldst_internal.h
index ef9117e84c..b5c473fc48 100644
--- a/target/arm/sve_ldst_internal.h
+++ b/target/arm/sve_ldst_internal.h
@@ -124,4 +124,98 @@ DO_ST_PRIM_2(dd, H1_8, uint64_t, uint64_t, stq)
 #undef DO_LD_PRIM_2
 #undef DO_ST_PRIM_2
 
+/*
+ * Resolve the guest virtual address to info->host and info->flags.
+ * If @nofault, return false if the page is invalid, otherwise
+ * exit via page fault exception.
+ */
+
+typedef struct {
+    void *host;
+    int flags;
+    MemTxAttrs attrs;
+} SVEHostPage;
+
+bool sve_probe_page(SVEHostPage *info, bool nofault, CPUARMState *env,
+                    target_ulong addr, int mem_off, MMUAccessType access_type,
+                    int mmu_idx, uintptr_t retaddr);
+
+/*
+ * Analyse contiguous data, protected by a governing predicate.
+ */
+
+typedef enum {
+    FAULT_NO,
+    FAULT_FIRST,
+    FAULT_ALL,
+} SVEContFault;
+
+typedef struct {
+    /*
+     * First and last element wholly contained within the two pages.
+     * mem_off_first[0] and reg_off_first[0] are always set >= 0.
+     * reg_off_last[0] may be < 0 if the first element crosses pages.
+     * All of mem_off_first[1], reg_off_first[1] and reg_off_last[1]
+     * are set >= 0 only if there are complete elements on a second page.
+     *
+     * The reg_off_* offsets are relative to the internal vector register.
+     * The mem_off_first offset is relative to the memory address; the
+     * two offsets are different when a load operation extends, a store
+     * operation truncates, or for multi-register operations.
+     */
+    int16_t mem_off_first[2];
+    int16_t reg_off_first[2];
+    int16_t reg_off_last[2];
+
+    /*
+     * One element that is misaligned and spans both pages,
+     * or -1 if there is no such active element.
+     */
+    int16_t mem_off_split;
+    int16_t reg_off_split;
+
+    /*
+     * The byte offset at which the entire operation crosses a page boundary.
+     * Set >= 0 if and only if the entire operation spans two pages.
+     */
+    int16_t page_split;
+
+    /* TLB data for the two pages. */
+    SVEHostPage page[2];
+} SVEContLdSt;
+
+/*
+ * Find first active element on each page, and a loose bound for the
+ * final element on each page.  Identify any single element that spans
+ * the page boundary.  Return true if there are any active elements.
+ */
+bool sve_cont_ldst_elements(SVEContLdSt *info, target_ulong addr, uint64_t *vg,
+                            intptr_t reg_max, int esz, int msize);
+
+/*
+ * Resolve the guest virtual addresses to info->page[].
+ * Control the generation of page faults with @fault.  Return false if
+ * there is no work to do, which can only happen with @fault == FAULT_NO.
+ */
+bool sve_cont_ldst_pages(SVEContLdSt *info, SVEContFault fault,
+                         CPUARMState *env, target_ulong addr,
+                         MMUAccessType access_type, uintptr_t retaddr);
+
+#ifdef CONFIG_USER_ONLY
+static inline void
+sve_cont_ldst_watchpoints(SVEContLdSt *info, CPUARMState *env, uint64_t *vg,
+                          target_ulong addr, int esize, int msize,
+                          int wp_access, uintptr_t retaddr)
+{ }
+#else
+void sve_cont_ldst_watchpoints(SVEContLdSt *info, CPUARMState *env,
+                               uint64_t *vg, target_ulong addr,
+                               int esize, int msize, int wp_access,
+                               uintptr_t retaddr);
+#endif
+
+void sve_cont_ldst_mte_check(SVEContLdSt *info, CPUARMState *env, uint64_t *vg,
+                             target_ulong addr, int esize, int msize,
+                             uint32_t mtedesc, uintptr_t ra);
+
 #endif /* TARGET_ARM_SVE_LDST_INTERNAL_H */

From 820e0bb9ce1941af82d14ec4a981792cb3697956 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:58 +0100
Subject: [PATCH 902/935] target/arm: Move expand_pred_b to vec_internal.h

Put the inline function near the array declaration.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-16-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/sve_helper.c   | 9 ---------
 target/arm/vec_internal.h | 8 +++++++-
 2 files changed, 7 insertions(+), 10 deletions(-)

diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c
index 8cd371e3e3..e865c12527 100644
--- a/target/arm/sve_helper.c
+++ b/target/arm/sve_helper.c
@@ -103,15 +103,6 @@ uint32_t HELPER(sve_predtest)(void *vd, void *vg, uint32_t words)
     return flags;
 }
 
-/*
- * Expand active predicate bits to bytes, for byte elements.
- * (The data table itself is in vec_helper.c as MVE also needs it.)
- */
-static inline uint64_t expand_pred_b(uint8_t byte)
-{
-    return expand_pred_b_data[byte];
-}
-
 /* Similarly for half-word elements.
  *  for (i = 0; i < 256; ++i) {
  *      unsigned long m = 0;
diff --git a/target/arm/vec_internal.h b/target/arm/vec_internal.h
index 1d63402042..d1a1ea4a66 100644
--- a/target/arm/vec_internal.h
+++ b/target/arm/vec_internal.h
@@ -50,8 +50,14 @@
 #define H8(x)   (x)
 #define H1_8(x) (x)
 
-/* Data for expanding active predicate bits to bytes, for byte elements. */
+/*
+ * Expand active predicate bits to bytes, for byte elements.
+ */
 extern const uint64_t expand_pred_b_data[256];
+static inline uint64_t expand_pred_b(uint8_t byte)
+{
+    return expand_pred_b_data[byte];
+}
 
 static inline void clear_tail(void *vd, uintptr_t opr_sz, uintptr_t max_sz)
 {

From 05dd14bdfa89ffeb1a82b08776687d44b538185b Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:58 +0100
Subject: [PATCH 903/935] target/arm: Use expand_pred_b in mve_helper.c

Use the function instead of the array directly.

Because the function performs its own masking, via the uint8_t
parameter, we need to do nothing extra within the users: the bits
above the first 2 (_uh) or 4 (_uw) will be discarded by assignment
to the local bmask variables, and of course _uq uses the entire
uint64_t result.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-17-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/mve_helper.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/target/arm/mve_helper.c b/target/arm/mve_helper.c
index 846962bf4c..403b345ea3 100644
--- a/target/arm/mve_helper.c
+++ b/target/arm/mve_helper.c
@@ -726,7 +726,7 @@ static void mergemask_sb(int8_t *d, int8_t r, uint16_t mask)
 
 static void mergemask_uh(uint16_t *d, uint16_t r, uint16_t mask)
 {
-    uint16_t bmask = expand_pred_b_data[mask & 3];
+    uint16_t bmask = expand_pred_b(mask);
     *d = (*d & ~bmask) | (r & bmask);
 }
 
@@ -737,7 +737,7 @@ static void mergemask_sh(int16_t *d, int16_t r, uint16_t mask)
 
 static void mergemask_uw(uint32_t *d, uint32_t r, uint16_t mask)
 {
-    uint32_t bmask = expand_pred_b_data[mask & 0xf];
+    uint32_t bmask = expand_pred_b(mask);
     *d = (*d & ~bmask) | (r & bmask);
 }
 
@@ -748,7 +748,7 @@ static void mergemask_sw(int32_t *d, int32_t r, uint16_t mask)
 
 static void mergemask_uq(uint64_t *d, uint64_t r, uint16_t mask)
 {
-    uint64_t bmask = expand_pred_b_data[mask & 0xff];
+    uint64_t bmask = expand_pred_b(mask);
     *d = (*d & ~bmask) | (r & bmask);
 }
 

From a613cf2d4ae61fbcdf084103ea10de193bfbe169 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:58 +0100
Subject: [PATCH 904/935] target/arm: Move expand_pred_h to vec_internal.h

Move the data to vec_helper.c and the inline to vec_internal.h.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-18-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/sve_helper.c   | 29 -----------------------------
 target/arm/vec_helper.c   | 26 ++++++++++++++++++++++++++
 target/arm/vec_internal.h |  7 +++++++
 3 files changed, 33 insertions(+), 29 deletions(-)

diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c
index e865c12527..1654c0bbf9 100644
--- a/target/arm/sve_helper.c
+++ b/target/arm/sve_helper.c
@@ -103,35 +103,6 @@ uint32_t HELPER(sve_predtest)(void *vd, void *vg, uint32_t words)
     return flags;
 }
 
-/* Similarly for half-word elements.
- *  for (i = 0; i < 256; ++i) {
- *      unsigned long m = 0;
- *      if (i & 0xaa) {
- *          continue;
- *      }
- *      for (j = 0; j < 8; j += 2) {
- *          if ((i >> j) & 1) {
- *              m |= 0xfffful << (j << 3);
- *          }
- *      }
- *      printf("[0x%x] = 0x%016lx,\n", i, m);
- *  }
- */
-static inline uint64_t expand_pred_h(uint8_t byte)
-{
-    static const uint64_t word[] = {
-        [0x01] = 0x000000000000ffff, [0x04] = 0x00000000ffff0000,
-        [0x05] = 0x00000000ffffffff, [0x10] = 0x0000ffff00000000,
-        [0x11] = 0x0000ffff0000ffff, [0x14] = 0x0000ffffffff0000,
-        [0x15] = 0x0000ffffffffffff, [0x40] = 0xffff000000000000,
-        [0x41] = 0xffff00000000ffff, [0x44] = 0xffff0000ffff0000,
-        [0x45] = 0xffff0000ffffffff, [0x50] = 0xffffffff00000000,
-        [0x51] = 0xffffffff0000ffff, [0x54] = 0xffffffffffff0000,
-        [0x55] = 0xffffffffffffffff,
-    };
-    return word[byte & 0x55];
-}
-
 /* Similarly for single word elements.  */
 static inline uint64_t expand_pred_s(uint8_t byte)
 {
diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c
index 17fb158362..26c373e522 100644
--- a/target/arm/vec_helper.c
+++ b/target/arm/vec_helper.c
@@ -127,6 +127,32 @@ const uint64_t expand_pred_b_data[256] = {
     0xffffffffffffffff,
 };
 
+/*
+ * Similarly for half-word elements.
+ *  for (i = 0; i < 256; ++i) {
+ *      unsigned long m = 0;
+ *      if (i & 0xaa) {
+ *          continue;
+ *      }
+ *      for (j = 0; j < 8; j += 2) {
+ *          if ((i >> j) & 1) {
+ *              m |= 0xfffful << (j << 3);
+ *          }
+ *      }
+ *      printf("[0x%x] = 0x%016lx,\n", i, m);
+ *  }
+ */
+const uint64_t expand_pred_h_data[0x55 + 1] = {
+    [0x01] = 0x000000000000ffff, [0x04] = 0x00000000ffff0000,
+    [0x05] = 0x00000000ffffffff, [0x10] = 0x0000ffff00000000,
+    [0x11] = 0x0000ffff0000ffff, [0x14] = 0x0000ffffffff0000,
+    [0x15] = 0x0000ffffffffffff, [0x40] = 0xffff000000000000,
+    [0x41] = 0xffff00000000ffff, [0x44] = 0xffff0000ffff0000,
+    [0x45] = 0xffff0000ffffffff, [0x50] = 0xffffffff00000000,
+    [0x51] = 0xffffffff0000ffff, [0x54] = 0xffffffffffff0000,
+    [0x55] = 0xffffffffffffffff,
+};
+
 /* Signed saturating rounding doubling multiply-accumulate high half, 8-bit */
 int8_t do_sqrdmlah_b(int8_t src1, int8_t src2, int8_t src3,
                      bool neg, bool round)
diff --git a/target/arm/vec_internal.h b/target/arm/vec_internal.h
index d1a1ea4a66..1d527fadac 100644
--- a/target/arm/vec_internal.h
+++ b/target/arm/vec_internal.h
@@ -59,6 +59,13 @@ static inline uint64_t expand_pred_b(uint8_t byte)
     return expand_pred_b_data[byte];
 }
 
+/* Similarly for half-word elements. */
+extern const uint64_t expand_pred_h_data[0x55 + 1];
+static inline uint64_t expand_pred_h(uint8_t byte)
+{
+    return expand_pred_h_data[byte & 0x55];
+}
+
 static inline void clear_tail(void *vd, uintptr_t opr_sz, uintptr_t max_sz)
 {
     uint64_t *d = vd + opr_sz;

From 72db2aa353a98bd3486c9e5ef7e31ae7cf934849 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:58 +0100
Subject: [PATCH 905/935] target/arm: Export bfdotadd from vec_helper.c

We will need this over in sme_helper.c.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-19-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/vec_helper.c   |  2 +-
 target/arm/vec_internal.h | 13 +++++++++++++
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c
index 26c373e522..9a9c034e36 100644
--- a/target/arm/vec_helper.c
+++ b/target/arm/vec_helper.c
@@ -2557,7 +2557,7 @@ DO_MMLA_B(gvec_usmmla_b, do_usmmla_b)
  * BFloat16 Dot Product
  */
 
-static float32 bfdotadd(float32 sum, uint32_t e1, uint32_t e2)
+float32 bfdotadd(float32 sum, uint32_t e1, uint32_t e2)
 {
     /* FPCR is ignored for BFDOT and BFMMLA. */
     float_status bf_status = {
diff --git a/target/arm/vec_internal.h b/target/arm/vec_internal.h
index 1d527fadac..1f4ed80ff7 100644
--- a/target/arm/vec_internal.h
+++ b/target/arm/vec_internal.h
@@ -230,4 +230,17 @@ uint64_t pmull_h(uint64_t op1, uint64_t op2);
  */
 uint64_t pmull_w(uint64_t op1, uint64_t op2);
 
+/**
+ * bfdotadd:
+ * @sum: addend
+ * @e1, @e2: multiplicand vectors
+ *
+ * BFloat16 2-way dot product of @e1 & @e2, accumulating with @sum.
+ * The @e1 and @e2 operands correspond to the 32-bit source vector
+ * slots and contain two Bfloat16 values each.
+ *
+ * Corresponds to the ARM pseudocode function BFDotAdd.
+ */
+float32 bfdotadd(float32 sum, uint32_t e1, uint32_t e2);
+
 #endif /* TARGET_ARM_VEC_INTERNAL_H */

From f305bf9436896b4cd9ef622034e166b024780874 Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:59 +0100
Subject: [PATCH 906/935] target/arm: Add isar_feature_aa64_sme

This will be used for implementing FEAT_SME.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-20-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index e45b5cb7fe..2e6153c540 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -4048,6 +4048,11 @@ static inline bool isar_feature_aa64_mte(const ARMISARegisters *id)
     return FIELD_EX64(id->id_aa64pfr1, ID_AA64PFR1, MTE) >= 2;
 }
 
+static inline bool isar_feature_aa64_sme(const ARMISARegisters *id)
+{
+    return FIELD_EX64(id->id_aa64pfr1, ID_AA64PFR1, SME) != 0;
+}
+
 static inline bool isar_feature_aa64_pmu_8_1(const ARMISARegisters *id)
 {
     return FIELD_EX64(id->id_aa64dfr0, ID_AA64DFR0, PMUVER) >= 4 &&

From 414c54d515dba16bfaef643a8acec200c05f229a Mon Sep 17 00:00:00 2001
From: Richard Henderson <richard.henderson@linaro.org>
Date: Wed, 8 Jun 2022 19:38:59 +0100
Subject: [PATCH 907/935] target/arm: Add ID_AA64SMFR0_EL1

This register is allocated from the existing block of id registers,
so it is already RES0 for cpus that do not implement SME.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20220607203306.657998-21-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 target/arm/cpu.h    | 25 +++++++++++++++++++++++++
 target/arm/helper.c |  4 ++--
 target/arm/kvm64.c  | 11 +++++++----
 3 files changed, 34 insertions(+), 6 deletions(-)

diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 2e6153c540..78dbcb5592 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -966,6 +966,7 @@ struct ArchCPU {
         uint64_t id_aa64dfr0;
         uint64_t id_aa64dfr1;
         uint64_t id_aa64zfr0;
+        uint64_t id_aa64smfr0;
         uint64_t reset_pmcr_el0;
     } isar;
     uint64_t midr;
@@ -2190,6 +2191,15 @@ FIELD(ID_AA64ZFR0, I8MM, 44, 4)
 FIELD(ID_AA64ZFR0, F32MM, 52, 4)
 FIELD(ID_AA64ZFR0, F64MM, 56, 4)
 
+FIELD(ID_AA64SMFR0, F32F32, 32, 1)
+FIELD(ID_AA64SMFR0, B16F32, 34, 1)
+FIELD(ID_AA64SMFR0, F16F32, 35, 1)
+FIELD(ID_AA64SMFR0, I8I32, 36, 4)
+FIELD(ID_AA64SMFR0, F64F64, 48, 1)
+FIELD(ID_AA64SMFR0, I16I64, 52, 4)
+FIELD(ID_AA64SMFR0, SMEVER, 56, 4)
+FIELD(ID_AA64SMFR0, FA64, 63, 1)
+
 FIELD(ID_DFR0, COPDBG, 0, 4)
 FIELD(ID_DFR0, COPSDBG, 4, 4)
 FIELD(ID_DFR0, MMAPDBG, 8, 4)
@@ -4195,6 +4205,21 @@ static inline bool isar_feature_aa64_sve_f64mm(const ARMISARegisters *id)
     return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, F64MM) != 0;
 }
 
+static inline bool isar_feature_aa64_sme_f64f64(const ARMISARegisters *id)
+{
+    return FIELD_EX64(id->id_aa64smfr0, ID_AA64SMFR0, F64F64);
+}
+
+static inline bool isar_feature_aa64_sme_i16i64(const ARMISARegisters *id)
+{
+    return FIELD_EX64(id->id_aa64smfr0, ID_AA64SMFR0, I16I64) == 0xf;
+}
+
+static inline bool isar_feature_aa64_sme_fa64(const ARMISARegisters *id)
+{
+    return FIELD_EX64(id->id_aa64smfr0, ID_AA64SMFR0, FA64);
+}
+
 /*
  * Feature tests for "does this exist in either 32-bit or 64-bit?"
  */
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 400f7cd1db..ac9942d750 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -7722,11 +7722,11 @@ void register_cp_regs_for_features(ARMCPU *cpu)
               .access = PL1_R, .type = ARM_CP_CONST,
               .accessfn = access_aa64_tid3,
               .resetvalue = cpu->isar.id_aa64zfr0 },
-            { .name = "ID_AA64PFR5_EL1_RESERVED", .state = ARM_CP_STATE_AA64,
+            { .name = "ID_AA64SMFR0_EL1", .state = ARM_CP_STATE_AA64,
               .opc0 = 3, .opc1 = 0, .crn = 0, .crm = 4, .opc2 = 5,
               .access = PL1_R, .type = ARM_CP_CONST,
               .accessfn = access_aa64_tid3,
-              .resetvalue = 0 },
+              .resetvalue = cpu->isar.id_aa64smfr0 },
             { .name = "ID_AA64PFR6_EL1_RESERVED", .state = ARM_CP_STATE_AA64,
               .opc0 = 3, .opc1 = 0, .crn = 0, .crm = 4, .opc2 = 6,
               .access = PL1_R, .type = ARM_CP_CONST,
diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c
index b3f635fc95..ff8f65da22 100644
--- a/target/arm/kvm64.c
+++ b/target/arm/kvm64.c
@@ -574,6 +574,8 @@ bool kvm_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
     } else {
         err |= read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64pfr1,
                               ARM64_SYS_REG(3, 0, 0, 4, 1));
+        err |= read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64smfr0,
+                              ARM64_SYS_REG(3, 0, 0, 4, 5));
         err |= read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64dfr0,
                               ARM64_SYS_REG(3, 0, 0, 5, 0));
         err |= read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64dfr1,
@@ -682,10 +684,11 @@ bool kvm_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
         ahcf->isar.id_aa64pfr0 = t;
 
         /*
-         * Before v5.1, KVM did not support SVE and did not expose
-         * ID_AA64ZFR0_EL1 even as RAZ.  After v5.1, KVM still does
-         * not expose the register to "user" requests like this
-         * unless the host supports SVE.
+         * There is a range of kernels between kernel commit 73433762fcae
+         * and f81cb2c3ad41 which have a bug where the kernel doesn't expose
+         * SYS_ID_AA64ZFR0_EL1 via the ONE_REG API unless the VM has enabled
+         * SVE support, so we only read it here, rather than together with all
+         * the other ID registers earlier.
          */
         err |= read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64zfr0,
                               ARM64_SYS_REG(3, 0, 0, 4, 4));

From 7851b21a8192750adecbcf6e8780a20de5891ad6 Mon Sep 17 00:00:00 2001
From: Bernhard Beschow <shentey@gmail.com>
Date: Fri, 13 May 2022 20:09:55 +0200
Subject: [PATCH 908/935] hw/ide/piix: Remove redundant "piix3-ide-xen" device
 class

Commit 0f8445820f11a69154309863960328dda3dc1ad4 'xen: piix reuse pci
generic class init function' already resolved redundant code which in
turn rendered piix3-ide-xen redundant.

Signed-off-by: Bernhard Beschow <shentey@gmail.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Message-Id: <20220513180957.90514-2-shentey@gmail.com>
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 hw/i386/pc_piix.c | 3 +--
 hw/ide/piix.c     | 7 -------
 2 files changed, 1 insertion(+), 9 deletions(-)

diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index 578e537b35..0e45521e74 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -246,8 +246,7 @@ static void pc_init1(MachineState *machine,
     if (pcmc->pci_enabled) {
         PCIDevice *dev;
 
-        dev = pci_create_simple(pci_bus, piix3_devfn + 1,
-                                xen_enabled() ? "piix3-ide-xen" : "piix3-ide");
+        dev = pci_create_simple(pci_bus, piix3_devfn + 1, "piix3-ide");
         pci_ide_create_devs(dev);
         idebus[0] = qdev_get_child_bus(&dev->qdev, "ide.0");
         idebus[1] = qdev_get_child_bus(&dev->qdev, "ide.1");
diff --git a/hw/ide/piix.c b/hw/ide/piix.c
index ce89fd0aa3..2345fe9e1d 100644
--- a/hw/ide/piix.c
+++ b/hw/ide/piix.c
@@ -241,12 +241,6 @@ static const TypeInfo piix3_ide_info = {
     .class_init    = piix3_ide_class_init,
 };
 
-static const TypeInfo piix3_ide_xen_info = {
-    .name          = "piix3-ide-xen",
-    .parent        = TYPE_PCI_IDE,
-    .class_init    = piix3_ide_class_init,
-};
-
 /* NOTE: for the PIIX4, the IRQs and IOports are hardcoded */
 static void piix4_ide_class_init(ObjectClass *klass, void *data)
 {
@@ -272,7 +266,6 @@ static const TypeInfo piix4_ide_info = {
 static void piix_ide_register_types(void)
 {
     type_register_static(&piix3_ide_info);
-    type_register_static(&piix3_ide_xen_info);
     type_register_static(&piix4_ide_info);
 }
 

From 36902417460e3f9f1c63a79d802622fd774f1c30 Mon Sep 17 00:00:00 2001
From: Bernhard Beschow <shentey@gmail.com>
Date: Fri, 13 May 2022 20:09:56 +0200
Subject: [PATCH 909/935] hw/ide/piix: Add some documentation to
 pci_piix3_xen_ide_unplug()

The comment is based on commit message
ae4d2eb273b167dad748ea4249720319240b1ac2 'xen-platform: add missing disk
unplug option'. Since it seems to describe design decisions and
limitations that still apply it seems worth having.

Signed-off-by: Bernhard Beschow <shentey@gmail.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Message-Id: <20220513180957.90514-3-shentey@gmail.com>
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 hw/ide/piix.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/hw/ide/piix.c b/hw/ide/piix.c
index 2345fe9e1d..bc1b37512a 100644
--- a/hw/ide/piix.c
+++ b/hw/ide/piix.c
@@ -173,6 +173,17 @@ static void pci_piix_ide_realize(PCIDevice *dev, Error **errp)
     }
 }
 
+/*
+ * The Xen HVM unplug protocol [1] specifies a mechanism to allow guests to
+ * request unplug of 'aux' disks (which is stated to mean all IDE disks,
+ * except the primary master).
+ *
+ * NOTE: The semantics of what happens if unplug of all disks and 'aux' disks
+ *       is simultaneously requested is not clear. The implementation assumes
+ *       that an 'all' request overrides an 'aux' request.
+ *
+ * [1] https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=docs/misc/hvm-emulated-unplug.pandoc
+ */
 int pci_piix3_xen_ide_unplug(DeviceState *dev, bool aux)
 {
     PCIIDEState *pci_ide;

From 6a8a8b62bdc8e3d7c5fc0f82ef4583707183b12f Mon Sep 17 00:00:00 2001
From: Bernhard Beschow <shentey@gmail.com>
Date: Fri, 13 May 2022 20:09:57 +0200
Subject: [PATCH 910/935] include/hw/ide: Unexport pci_piix3_xen_ide_unplug()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This function was declared in a generic and public header, implemented
in a device-specific source file but only used in xen_platform. Given its
'aux' parameter, this function is more xen-specific than piix-specific.
Also, the hardcoded magic constants seem to be generic and related to
PCIIDEState and IDEBus rather than piix.

Therefore, move this function to xen_platform, unexport it, and drop the
"piix3" in the function name as well.

Signed-off-by: Bernhard Beschow <shentey@gmail.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220513180957.90514-4-shentey@gmail.com>
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 hw/i386/xen/xen_platform.c | 48 +++++++++++++++++++++++++++++++++++++-
 hw/ide/piix.c              | 46 ------------------------------------
 include/hw/ide.h           |  3 ---
 3 files changed, 47 insertions(+), 50 deletions(-)

diff --git a/hw/i386/xen/xen_platform.c b/hw/i386/xen/xen_platform.c
index 72028449ba..a64265cca0 100644
--- a/hw/i386/xen/xen_platform.c
+++ b/hw/i386/xen/xen_platform.c
@@ -26,6 +26,7 @@
 #include "qemu/osdep.h"
 #include "qapi/error.h"
 #include "hw/ide.h"
+#include "hw/ide/pci.h"
 #include "hw/pci/pci.h"
 #include "hw/xen/xen_common.h"
 #include "migration/vmstate.h"
@@ -134,6 +135,51 @@ static void pci_unplug_nics(PCIBus *bus)
     pci_for_each_device(bus, 0, unplug_nic, NULL);
 }
 
+/*
+ * The Xen HVM unplug protocol [1] specifies a mechanism to allow guests to
+ * request unplug of 'aux' disks (which is stated to mean all IDE disks,
+ * except the primary master).
+ *
+ * NOTE: The semantics of what happens if unplug of all disks and 'aux' disks
+ *       is simultaneously requested is not clear. The implementation assumes
+ *       that an 'all' request overrides an 'aux' request.
+ *
+ * [1] https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=docs/misc/hvm-emulated-unplug.pandoc
+ */
+static void pci_xen_ide_unplug(DeviceState *dev, bool aux)
+{
+    PCIIDEState *pci_ide;
+    int i;
+    IDEDevice *idedev;
+    IDEBus *idebus;
+    BlockBackend *blk;
+
+    pci_ide = PCI_IDE(dev);
+
+    for (i = aux ? 1 : 0; i < 4; i++) {
+        idebus = &pci_ide->bus[i / 2];
+        blk = idebus->ifs[i % 2].blk;
+
+        if (blk && idebus->ifs[i % 2].drive_kind != IDE_CD) {
+            if (!(i % 2)) {
+                idedev = idebus->master;
+            } else {
+                idedev = idebus->slave;
+            }
+
+            blk_drain(blk);
+            blk_flush(blk);
+
+            blk_detach_dev(blk, DEVICE(idedev));
+            idebus->ifs[i % 2].blk = NULL;
+            idedev->conf.blk = NULL;
+            monitor_remove_blk(blk);
+            blk_unref(blk);
+        }
+    }
+    qdev_reset_all(dev);
+}
+
 static void unplug_disks(PCIBus *b, PCIDevice *d, void *opaque)
 {
     uint32_t flags = *(uint32_t *)opaque;
@@ -147,7 +193,7 @@ static void unplug_disks(PCIBus *b, PCIDevice *d, void *opaque)
 
     switch (pci_get_word(d->config + PCI_CLASS_DEVICE)) {
     case PCI_CLASS_STORAGE_IDE:
-        pci_piix3_xen_ide_unplug(DEVICE(d), aux);
+        pci_xen_ide_unplug(DEVICE(d), aux);
         break;
 
     case PCI_CLASS_STORAGE_SCSI:
diff --git a/hw/ide/piix.c b/hw/ide/piix.c
index bc1b37512a..9a9b28078e 100644
--- a/hw/ide/piix.c
+++ b/hw/ide/piix.c
@@ -173,52 +173,6 @@ static void pci_piix_ide_realize(PCIDevice *dev, Error **errp)
     }
 }
 
-/*
- * The Xen HVM unplug protocol [1] specifies a mechanism to allow guests to
- * request unplug of 'aux' disks (which is stated to mean all IDE disks,
- * except the primary master).
- *
- * NOTE: The semantics of what happens if unplug of all disks and 'aux' disks
- *       is simultaneously requested is not clear. The implementation assumes
- *       that an 'all' request overrides an 'aux' request.
- *
- * [1] https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=docs/misc/hvm-emulated-unplug.pandoc
- */
-int pci_piix3_xen_ide_unplug(DeviceState *dev, bool aux)
-{
-    PCIIDEState *pci_ide;
-    int i;
-    IDEDevice *idedev;
-    IDEBus *idebus;
-    BlockBackend *blk;
-
-    pci_ide = PCI_IDE(dev);
-
-    for (i = aux ? 1 : 0; i < 4; i++) {
-        idebus = &pci_ide->bus[i / 2];
-        blk = idebus->ifs[i % 2].blk;
-
-        if (blk && idebus->ifs[i % 2].drive_kind != IDE_CD) {
-            if (!(i % 2)) {
-                idedev = idebus->master;
-            } else {
-                idedev = idebus->slave;
-            }
-
-            blk_drain(blk);
-            blk_flush(blk);
-
-            blk_detach_dev(blk, DEVICE(idedev));
-            idebus->ifs[i % 2].blk = NULL;
-            idedev->conf.blk = NULL;
-            monitor_remove_blk(blk);
-            blk_unref(blk);
-        }
-    }
-    qdev_reset_all(dev);
-    return 0;
-}
-
 static void pci_piix_ide_exitfn(PCIDevice *dev)
 {
     PCIIDEState *d = PCI_IDE(dev);
diff --git a/include/hw/ide.h b/include/hw/ide.h
index c5ce5da4f4..60f1f4f714 100644
--- a/include/hw/ide.h
+++ b/include/hw/ide.h
@@ -8,9 +8,6 @@
 ISADevice *isa_ide_init(ISABus *bus, int iobase, int iobase2, int isairq,
                         DriveInfo *hd0, DriveInfo *hd1);
 
-/* ide-pci.c */
-int pci_piix3_xen_ide_unplug(DeviceState *dev, bool aux);
-
 /* ide-mmio.c */
 void mmio_ide_init_drives(DeviceState *dev, DriveInfo *hd0, DriveInfo *hd1);
 

From efe1592c43fe9b4053bf2987581a05736062a3cd Mon Sep 17 00:00:00 2001
From: Alistair Francis <alistair.francis@wdc.com>
Date: Mon, 9 May 2022 11:13:39 +0200
Subject: [PATCH 911/935] MAINTAINERS: Cover hw/core/uboot_image.h within
 Generic Loader section
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220509091339.26016-1-alistair.francis@wdc.com>
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 5580a36b68..b3af081c51 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2198,6 +2198,7 @@ Generic Loader
 M: Alistair Francis <alistair@alistair23.me>
 S: Maintained
 F: hw/core/generic-loader.c
+F: hw/core/uboot_image.h
 F: include/hw/core/generic-loader.h
 F: docs/system/generic-loader.rst
 

From de799beba7f927b2a1ed38128309316511311605 Mon Sep 17 00:00:00 2001
From: Weiwei Li <liweiwei@iscas.ac.cn>
Date: Tue, 31 May 2022 11:07:32 +0800
Subject: [PATCH 912/935] target/riscv: add support for zmmul extension v0.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add support for the zmmul extension v0.1. This extension includes all
multiplication operations from the M extension but not the divide ops.

Signed-off-by: Weiwei Li <liweiwei@iscas.ac.cn>
Signed-off-by: Junqiang Wang <wangjunqiang@iscas.ac.cn>
Reviewed-by: Víctor Colombo <victor.colombo@eldorado.org.br>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20220531030732.3850-1-liweiwei@iscas.ac.cn>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.c                      |  7 +++++++
 target/riscv/cpu.h                      |  1 +
 target/riscv/insn_trans/trans_rvm.c.inc | 18 ++++++++++++------
 3 files changed, 20 insertions(+), 6 deletions(-)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index a91253d4bd..bcbba3fbd5 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -600,6 +600,11 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)
             cpu->cfg.ext_ifencei = true;
         }
 
+        if (cpu->cfg.ext_m && cpu->cfg.ext_zmmul) {
+            warn_report("Zmmul will override M");
+            cpu->cfg.ext_m = false;
+        }
+
         if (cpu->cfg.ext_i && cpu->cfg.ext_e) {
             error_setg(errp,
                        "I and E extensions are incompatible");
@@ -905,6 +910,7 @@ static Property riscv_cpu_properties[] = {
 
     /* These are experimental so mark with 'x-' */
     DEFINE_PROP_BOOL("x-j", RISCVCPU, cfg.ext_j, false),
+    DEFINE_PROP_BOOL("x-zmmul", RISCVCPU, cfg.ext_zmmul, false),
     /* ePMP 0.9.3 */
     DEFINE_PROP_BOOL("x-epmp", RISCVCPU, cfg.epmp, false),
     DEFINE_PROP_BOOL("x-aia", RISCVCPU, cfg.aia, false),
@@ -1031,6 +1037,7 @@ static void riscv_isa_string_ext(RISCVCPU *cpu, char **isa_str, int max_str_len)
     struct isa_ext_data isa_edata_arr[] = {
         ISA_EDATA_ENTRY(zicsr, ext_icsr),
         ISA_EDATA_ENTRY(zifencei, ext_ifencei),
+        ISA_EDATA_ENTRY(zmmul, ext_zmmul),
         ISA_EDATA_ENTRY(zfh, ext_zfh),
         ISA_EDATA_ENTRY(zfhmin, ext_zfhmin),
         ISA_EDATA_ENTRY(zfinx, ext_zfinx),
diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
index f08c3e8813..890d33cebb 100644
--- a/target/riscv/cpu.h
+++ b/target/riscv/cpu.h
@@ -411,6 +411,7 @@ struct RISCVCPUConfig {
     bool ext_zhinxmin;
     bool ext_zve32f;
     bool ext_zve64f;
+    bool ext_zmmul;
 
     uint32_t mvendorid;
     uint64_t marchid;
diff --git a/target/riscv/insn_trans/trans_rvm.c.inc b/target/riscv/insn_trans/trans_rvm.c.inc
index 16b029edf0..ec7f705aab 100644
--- a/target/riscv/insn_trans/trans_rvm.c.inc
+++ b/target/riscv/insn_trans/trans_rvm.c.inc
@@ -18,6 +18,12 @@
  * this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
+#define REQUIRE_M_OR_ZMMUL(ctx) do {                      \
+    if (!ctx->cfg_ptr->ext_zmmul && !has_ext(ctx, RVM)) { \
+        return false;                                     \
+    }                                                     \
+} while (0)
+
 static void gen_mulhu_i128(TCGv r2, TCGv r3, TCGv al, TCGv ah, TCGv bl, TCGv bh)
 {
     TCGv tmpl = tcg_temp_new();
@@ -65,7 +71,7 @@ static void gen_mul_i128(TCGv rl, TCGv rh,
 
 static bool trans_mul(DisasContext *ctx, arg_mul *a)
 {
-    REQUIRE_EXT(ctx, RVM);
+    REQUIRE_M_OR_ZMMUL(ctx);
     return gen_arith(ctx, a, EXT_NONE, tcg_gen_mul_tl, gen_mul_i128);
 }
 
@@ -109,7 +115,7 @@ static void gen_mulh_w(TCGv ret, TCGv s1, TCGv s2)
 
 static bool trans_mulh(DisasContext *ctx, arg_mulh *a)
 {
-    REQUIRE_EXT(ctx, RVM);
+    REQUIRE_M_OR_ZMMUL(ctx);
     return gen_arith_per_ol(ctx, a, EXT_SIGN, gen_mulh, gen_mulh_w,
                             gen_mulh_i128);
 }
@@ -161,7 +167,7 @@ static void gen_mulhsu_w(TCGv ret, TCGv arg1, TCGv arg2)
 
 static bool trans_mulhsu(DisasContext *ctx, arg_mulhsu *a)
 {
-    REQUIRE_EXT(ctx, RVM);
+    REQUIRE_M_OR_ZMMUL(ctx);
     return gen_arith_per_ol(ctx, a, EXT_NONE, gen_mulhsu, gen_mulhsu_w,
                             gen_mulhsu_i128);
 }
@@ -176,7 +182,7 @@ static void gen_mulhu(TCGv ret, TCGv s1, TCGv s2)
 
 static bool trans_mulhu(DisasContext *ctx, arg_mulhu *a)
 {
-    REQUIRE_EXT(ctx, RVM);
+    REQUIRE_M_OR_ZMMUL(ctx);
     /* gen_mulh_w works for either sign as input. */
     return gen_arith_per_ol(ctx, a, EXT_ZERO, gen_mulhu, gen_mulh_w,
                             gen_mulhu_i128);
@@ -349,7 +355,7 @@ static bool trans_remu(DisasContext *ctx, arg_remu *a)
 static bool trans_mulw(DisasContext *ctx, arg_mulw *a)
 {
     REQUIRE_64_OR_128BIT(ctx);
-    REQUIRE_EXT(ctx, RVM);
+    REQUIRE_M_OR_ZMMUL(ctx);
     ctx->ol = MXL_RV32;
     return gen_arith(ctx, a, EXT_NONE, tcg_gen_mul_tl, NULL);
 }
@@ -389,7 +395,7 @@ static bool trans_remuw(DisasContext *ctx, arg_remuw *a)
 static bool trans_muld(DisasContext *ctx, arg_muld *a)
 {
     REQUIRE_128BIT(ctx);
-    REQUIRE_EXT(ctx, RVM);
+    REQUIRE_M_OR_ZMMUL(ctx);
     ctx->ol = MXL_RV64;
     return gen_arith(ctx, a, EXT_SIGN, tcg_gen_mul_tl, NULL);
 }

From f9a461b2d3b8ef4f36b7891eb4040693ee071719 Mon Sep 17 00:00:00 2001
From: Atish Patra <atishp@rivosinc.com>
Date: Thu, 26 May 2022 13:35:00 -0700
Subject: [PATCH 913/935] hw/riscv: virt: Generate fw_cfg DT node correctly

fw_cfg DT node is generated after the create_fdt without any check
if the DT is being loaded from the commandline. This results in
FDT_ERR_EXISTS error if dtb is loaded from the commandline.

Generate fw_cfg node only if the DT is not loaded from the commandline.

Signed-off-by: Atish Patra <atishp@rivosinc.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20220526203500.847165-1-atishp@rivosinc.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 hw/riscv/virt.c | 28 ++++++++++++++++++----------
 1 file changed, 18 insertions(+), 10 deletions(-)

diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 293e9c95b7..bc424dd2f5 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -975,6 +975,23 @@ static void create_fdt_flash(RISCVVirtState *s, const MemMapEntry *memmap)
     g_free(name);
 }
 
+static void create_fdt_fw_cfg(RISCVVirtState *s, const MemMapEntry *memmap)
+{
+    char *nodename;
+    MachineState *mc = MACHINE(s);
+    hwaddr base = memmap[VIRT_FW_CFG].base;
+    hwaddr size = memmap[VIRT_FW_CFG].size;
+
+    nodename = g_strdup_printf("/fw-cfg@%" PRIx64, base);
+    qemu_fdt_add_subnode(mc->fdt, nodename);
+    qemu_fdt_setprop_string(mc->fdt, nodename,
+                            "compatible", "qemu,fw-cfg-mmio");
+    qemu_fdt_setprop_sized_cells(mc->fdt, nodename, "reg",
+                                 2, base, 2, size);
+    qemu_fdt_setprop(mc->fdt, nodename, "dma-coherent", NULL, 0);
+    g_free(nodename);
+}
+
 static void create_fdt(RISCVVirtState *s, const MemMapEntry *memmap,
                        uint64_t mem_size, const char *cmdline, bool is_32_bit)
 {
@@ -1023,6 +1040,7 @@ static void create_fdt(RISCVVirtState *s, const MemMapEntry *memmap,
     create_fdt_rtc(s, memmap, irq_mmio_phandle);
 
     create_fdt_flash(s, memmap);
+    create_fdt_fw_cfg(s, memmap);
 
 update_bootargs:
     if (cmdline && *cmdline) {
@@ -1082,22 +1100,12 @@ static inline DeviceState *gpex_pcie_init(MemoryRegion *sys_mem,
 static FWCfgState *create_fw_cfg(const MachineState *mc)
 {
     hwaddr base = virt_memmap[VIRT_FW_CFG].base;
-    hwaddr size = virt_memmap[VIRT_FW_CFG].size;
     FWCfgState *fw_cfg;
-    char *nodename;
 
     fw_cfg = fw_cfg_init_mem_wide(base + 8, base, 8, base + 16,
                                   &address_space_memory);
     fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)mc->smp.cpus);
 
-    nodename = g_strdup_printf("/fw-cfg@%" PRIx64, base);
-    qemu_fdt_add_subnode(mc->fdt, nodename);
-    qemu_fdt_setprop_string(mc->fdt, nodename,
-                            "compatible", "qemu,fw-cfg-mmio");
-    qemu_fdt_setprop_sized_cells(mc->fdt, nodename, "reg",
-                                 2, base, 2, size);
-    qemu_fdt_setprop(mc->fdt, nodename, "dma-coherent", NULL, 0);
-    g_free(nodename);
     return fw_cfg;
 }
 

From 40244040a7ac00d40db4dea02234d13502c30112 Mon Sep 17 00:00:00 2001
From: Alistair Francis <alistair.francis@wdc.com>
Date: Wed, 1 Jun 2022 11:36:31 +1000
Subject: [PATCH 914/935] hw/intc: sifive_plic: Avoid overflowing the
 addr_config buffer
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Since commit ad40be27 "target/riscv: Support start kernel directly by
KVM" we have been overflowing the addr_config on "M,MS..."
configurations, as reported https://gitlab.com/qemu-project/qemu/-/issues/1050.

This commit changes the loop in sifive_plic_create() from iterating over
the number of harts to just iterating over the addr_config. The
addr_config is based on the hart_config, and will contain interrup details
for all harts. This way we can't iterate past the end of addr_config.

Fixes: ad40be27084536 ("target/riscv: Support start kernel directly by KVM")
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1050
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Mingwang Li <limingwang@huawei.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220601013631.196854-1-alistair.francis@opensource.wdc.com>
---
 hw/intc/sifive_plic.c | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/hw/intc/sifive_plic.c b/hw/intc/sifive_plic.c
index eebbcf33d4..56d60e9ac9 100644
--- a/hw/intc/sifive_plic.c
+++ b/hw/intc/sifive_plic.c
@@ -431,7 +431,7 @@ DeviceState *sifive_plic_create(hwaddr addr, char *hart_config,
     uint32_t context_stride, uint32_t aperture_size)
 {
     DeviceState *dev = qdev_new(TYPE_SIFIVE_PLIC);
-    int i, j = 0;
+    int i;
     SiFivePLICState *plic;
 
     assert(enable_stride == (enable_stride & -enable_stride));
@@ -451,18 +451,17 @@ DeviceState *sifive_plic_create(hwaddr addr, char *hart_config,
     sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
 
     plic = SIFIVE_PLIC(dev);
-    for (i = 0; i < num_harts; i++) {
-        CPUState *cpu = qemu_get_cpu(hartid_base + i);
 
-        if (plic->addr_config[j].mode == PLICMode_M) {
-            j++;
-            qdev_connect_gpio_out(dev, num_harts + i,
+    for (i = 0; i < plic->num_addrs; i++) {
+        int cpu_num = plic->addr_config[i].hartid;
+        CPUState *cpu = qemu_get_cpu(hartid_base + cpu_num);
+
+        if (plic->addr_config[i].mode == PLICMode_M) {
+            qdev_connect_gpio_out(dev, num_harts + cpu_num,
                                   qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT));
         }
-
-        if (plic->addr_config[j].mode == PLICMode_S) {
-            j++;
-            qdev_connect_gpio_out(dev, i,
+        if (plic->addr_config[i].mode == PLICMode_S) {
+            qdev_connect_gpio_out(dev, cpu_num,
                                   qdev_get_gpio_in(DEVICE(cpu), IRQ_S_EXT));
         }
     }

From af9751316e53cdf7e98131afe6928a5f4445fe16 Mon Sep 17 00:00:00 2001
From: Jamie Iles <jamie@nuviainc.com>
Date: Thu, 11 Nov 2021 14:11:40 +0000
Subject: [PATCH 915/935] hw/core/loader: return image sizes as ssize_t
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Various loader functions return an int which limits images to 2GB which
is fine for things like a BIOS/kernel image, but if we want to be able
to load memory images or large ramdisks then any file over 2GB would
silently fail to load.

Cc: Luc Michel <lmichel@kalray.eu>
Signed-off-by: Jamie Iles <jamie@nuviainc.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Luc Michel <lmichel@kalray.eu>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20211111141141.3295094-2-jamie@nuviainc.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 hw/arm/armv7m.c          |  2 +-
 hw/arm/boot.c            |  8 ++--
 hw/core/generic-loader.c |  2 +-
 hw/core/loader.c         | 81 +++++++++++++++++++++-------------------
 hw/i386/x86.c            |  2 +-
 hw/riscv/boot.c          |  5 ++-
 include/hw/loader.h      | 55 +++++++++++++--------------
 7 files changed, 80 insertions(+), 75 deletions(-)

diff --git a/hw/arm/armv7m.c b/hw/arm/armv7m.c
index 32349ec94b..990861ee5e 100644
--- a/hw/arm/armv7m.c
+++ b/hw/arm/armv7m.c
@@ -570,7 +570,7 @@ static void armv7m_reset(void *opaque)
 
 void armv7m_load_kernel(ARMCPU *cpu, const char *kernel_filename, int mem_size)
 {
-    int image_size;
+    ssize_t image_size;
     uint64_t entry;
     int big_endian;
     AddressSpace *as;
diff --git a/hw/arm/boot.c b/hw/arm/boot.c
index a8de33fd64..ada2717f76 100644
--- a/hw/arm/boot.c
+++ b/hw/arm/boot.c
@@ -881,7 +881,7 @@ static int do_arm_linux_init(Object *obj, void *opaque)
     return 0;
 }
 
-static int64_t arm_load_elf(struct arm_boot_info *info, uint64_t *pentry,
+static ssize_t arm_load_elf(struct arm_boot_info *info, uint64_t *pentry,
                             uint64_t *lowaddr, uint64_t *highaddr,
                             int elf_machine, AddressSpace *as)
 {
@@ -892,7 +892,7 @@ static int64_t arm_load_elf(struct arm_boot_info *info, uint64_t *pentry,
     } elf_header;
     int data_swab = 0;
     bool big_endian;
-    int64_t ret = -1;
+    ssize_t ret = -1;
     Error *err = NULL;
 
 
@@ -1014,7 +1014,7 @@ static void arm_setup_direct_kernel_boot(ARMCPU *cpu,
     /* Set up for a direct boot of a kernel image file. */
     CPUState *cs;
     AddressSpace *as = arm_boot_address_space(cpu, info);
-    int kernel_size;
+    ssize_t kernel_size;
     int initrd_size;
     int is_linux = 0;
     uint64_t elf_entry;
@@ -1093,7 +1093,7 @@ static void arm_setup_direct_kernel_boot(ARMCPU *cpu,
 
     if (kernel_size > info->ram_size) {
         error_report("kernel '%s' is too large to fit in RAM "
-                     "(kernel size %d, RAM size %" PRId64 ")",
+                     "(kernel size %zd, RAM size %" PRId64 ")",
                      info->kernel_filename, kernel_size, info->ram_size);
         exit(1);
     }
diff --git a/hw/core/generic-loader.c b/hw/core/generic-loader.c
index c666545aa0..4f4d77908d 100644
--- a/hw/core/generic-loader.c
+++ b/hw/core/generic-loader.c
@@ -67,7 +67,7 @@ static void generic_loader_realize(DeviceState *dev, Error **errp)
     GenericLoaderState *s = GENERIC_LOADER(dev);
     hwaddr entry;
     int big_endian;
-    int size = 0;
+    ssize_t size = 0;
 
     s->set_pc = false;
 
diff --git a/hw/core/loader.c b/hw/core/loader.c
index edde657ac3..0548830733 100644
--- a/hw/core/loader.c
+++ b/hw/core/loader.c
@@ -114,17 +114,17 @@ ssize_t read_targphys(const char *name,
     return did;
 }
 
-int load_image_targphys(const char *filename,
-                        hwaddr addr, uint64_t max_sz)
+ssize_t load_image_targphys(const char *filename,
+                            hwaddr addr, uint64_t max_sz)
 {
     return load_image_targphys_as(filename, addr, max_sz, NULL);
 }
 
 /* return the size or -1 if error */
-int load_image_targphys_as(const char *filename,
-                           hwaddr addr, uint64_t max_sz, AddressSpace *as)
+ssize_t load_image_targphys_as(const char *filename,
+                               hwaddr addr, uint64_t max_sz, AddressSpace *as)
 {
-    int size;
+    ssize_t size;
 
     size = get_image_size(filename);
     if (size < 0 || size > max_sz) {
@@ -138,9 +138,9 @@ int load_image_targphys_as(const char *filename,
     return size;
 }
 
-int load_image_mr(const char *filename, MemoryRegion *mr)
+ssize_t load_image_mr(const char *filename, MemoryRegion *mr)
 {
-    int size;
+    ssize_t size;
 
     if (!memory_access_is_direct(mr, false)) {
         /* Can only load an image into RAM or ROM */
@@ -222,8 +222,8 @@ static void bswap_ahdr(struct exec *e)
      : (_N_SEGMENT_ROUND (_N_TXTENDADDR(x, target_page_size), target_page_size)))
 
 
-int load_aout(const char *filename, hwaddr addr, int max_sz,
-              int bswap_needed, hwaddr target_page_size)
+ssize_t load_aout(const char *filename, hwaddr addr, int max_sz,
+                  int bswap_needed, hwaddr target_page_size)
 {
     int fd;
     ssize_t size, ret;
@@ -617,13 +617,14 @@ toosmall:
 }
 
 /* Load a U-Boot image.  */
-static int load_uboot_image(const char *filename, hwaddr *ep, hwaddr *loadaddr,
-                            int *is_linux, uint8_t image_type,
-                            uint64_t (*translate_fn)(void *, uint64_t),
-                            void *translate_opaque, AddressSpace *as)
+static ssize_t load_uboot_image(const char *filename, hwaddr *ep,
+                                hwaddr *loadaddr, int *is_linux,
+                                uint8_t image_type,
+                                uint64_t (*translate_fn)(void *, uint64_t),
+                                void *translate_opaque, AddressSpace *as)
 {
     int fd;
-    int size;
+    ssize_t size;
     hwaddr address;
     uboot_image_header_t h;
     uboot_image_header_t *hdr = &h;
@@ -760,40 +761,40 @@ out:
     return ret;
 }
 
-int load_uimage(const char *filename, hwaddr *ep, hwaddr *loadaddr,
-                int *is_linux,
-                uint64_t (*translate_fn)(void *, uint64_t),
-                void *translate_opaque)
+ssize_t load_uimage(const char *filename, hwaddr *ep, hwaddr *loadaddr,
+                    int *is_linux,
+                    uint64_t (*translate_fn)(void *, uint64_t),
+                    void *translate_opaque)
 {
     return load_uboot_image(filename, ep, loadaddr, is_linux, IH_TYPE_KERNEL,
                             translate_fn, translate_opaque, NULL);
 }
 
-int load_uimage_as(const char *filename, hwaddr *ep, hwaddr *loadaddr,
-                   int *is_linux,
-                   uint64_t (*translate_fn)(void *, uint64_t),
-                   void *translate_opaque, AddressSpace *as)
+ssize_t load_uimage_as(const char *filename, hwaddr *ep, hwaddr *loadaddr,
+                       int *is_linux,
+                       uint64_t (*translate_fn)(void *, uint64_t),
+                       void *translate_opaque, AddressSpace *as)
 {
     return load_uboot_image(filename, ep, loadaddr, is_linux, IH_TYPE_KERNEL,
                             translate_fn, translate_opaque, as);
 }
 
 /* Load a ramdisk.  */
-int load_ramdisk(const char *filename, hwaddr addr, uint64_t max_sz)
+ssize_t load_ramdisk(const char *filename, hwaddr addr, uint64_t max_sz)
 {
     return load_ramdisk_as(filename, addr, max_sz, NULL);
 }
 
-int load_ramdisk_as(const char *filename, hwaddr addr, uint64_t max_sz,
-                    AddressSpace *as)
+ssize_t load_ramdisk_as(const char *filename, hwaddr addr, uint64_t max_sz,
+                        AddressSpace *as)
 {
     return load_uboot_image(filename, NULL, &addr, NULL, IH_TYPE_RAMDISK,
                             NULL, NULL, as);
 }
 
 /* Load a gzip-compressed kernel to a dynamically allocated buffer. */
-int load_image_gzipped_buffer(const char *filename, uint64_t max_sz,
-                              uint8_t **buffer)
+ssize_t load_image_gzipped_buffer(const char *filename, uint64_t max_sz,
+                                  uint8_t **buffer)
 {
     uint8_t *compressed_data = NULL;
     uint8_t *data = NULL;
@@ -838,9 +839,9 @@ int load_image_gzipped_buffer(const char *filename, uint64_t max_sz,
 }
 
 /* Load a gzip-compressed kernel. */
-int load_image_gzipped(const char *filename, hwaddr addr, uint64_t max_sz)
+ssize_t load_image_gzipped(const char *filename, hwaddr addr, uint64_t max_sz)
 {
-    int bytes;
+    ssize_t bytes;
     uint8_t *data;
 
     bytes = load_image_gzipped_buffer(filename, max_sz, &data);
@@ -970,14 +971,15 @@ static void *rom_set_mr(Rom *rom, Object *owner, const char *name, bool ro)
     return data;
 }
 
-int rom_add_file(const char *file, const char *fw_dir,
-                 hwaddr addr, int32_t bootindex,
-                 bool option_rom, MemoryRegion *mr,
-                 AddressSpace *as)
+ssize_t rom_add_file(const char *file, const char *fw_dir,
+                     hwaddr addr, int32_t bootindex,
+                     bool option_rom, MemoryRegion *mr,
+                     AddressSpace *as)
 {
     MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine());
     Rom *rom;
-    int rc, fd = -1;
+    ssize_t rc;
+    int fd = -1;
     char devpath[100];
 
     if (as && mr) {
@@ -1019,7 +1021,7 @@ int rom_add_file(const char *file, const char *fw_dir,
     lseek(fd, 0, SEEK_SET);
     rc = read(fd, rom->data, rom->datasize);
     if (rc != rom->datasize) {
-        fprintf(stderr, "rom: file %-20s: read error: rc=%d (expected %zd)\n",
+        fprintf(stderr, "rom: file %-20s: read error: rc=%zd (expected %zd)\n",
                 rom->name, rc, rom->datasize);
         goto err;
     }
@@ -1138,12 +1140,12 @@ int rom_add_elf_program(const char *name, GMappedFile *mapped_file, void *data,
     return 0;
 }
 
-int rom_add_vga(const char *file)
+ssize_t rom_add_vga(const char *file)
 {
     return rom_add_file(file, "vgaroms", 0, -1, true, NULL, NULL);
 }
 
-int rom_add_option(const char *file, int32_t bootindex)
+ssize_t rom_add_option(const char *file, int32_t bootindex)
 {
     return rom_add_file(file, "genroms", 0, bootindex, true, NULL, NULL);
 }
@@ -1846,11 +1848,12 @@ out:
 }
 
 /* return size or -1 if error */
-int load_targphys_hex_as(const char *filename, hwaddr *entry, AddressSpace *as)
+ssize_t load_targphys_hex_as(const char *filename, hwaddr *entry,
+                             AddressSpace *as)
 {
     gsize hex_blob_size;
     gchar *hex_blob;
-    int total_size = 0;
+    ssize_t total_size = 0;
 
     if (!g_file_get_contents(filename, &hex_blob, &hex_blob_size, NULL)) {
         return -1;
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 78b05ab7a2..6003b4b2df 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -1115,7 +1115,7 @@ void x86_bios_rom_init(MachineState *ms, const char *default_firmware,
     char *filename;
     MemoryRegion *bios, *isa_bios;
     int bios_size, isa_bios_size;
-    int ret;
+    ssize_t ret;
 
     /* BIOS load */
     bios_name = ms->firmware ?: default_firmware;
diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c
index 57a41df8e9..2d80f40b31 100644
--- a/hw/riscv/boot.c
+++ b/hw/riscv/boot.c
@@ -129,7 +129,8 @@ target_ulong riscv_load_firmware(const char *firmware_filename,
                                  hwaddr firmware_load_addr,
                                  symbol_fn_t sym_cb)
 {
-    uint64_t firmware_entry, firmware_size, firmware_end;
+    uint64_t firmware_entry, firmware_end;
+    ssize_t firmware_size;
 
     if (load_elf_ram_sym(firmware_filename, NULL, NULL, NULL,
                          &firmware_entry, NULL, &firmware_end, NULL,
@@ -185,7 +186,7 @@ target_ulong riscv_load_kernel(const char *kernel_filename,
 hwaddr riscv_load_initrd(const char *filename, uint64_t mem_size,
                          uint64_t kernel_entry, hwaddr *start)
 {
-    int size;
+    ssize_t size;
 
     /*
      * We want to put the initrd far enough into RAM that when the
diff --git a/include/hw/loader.h b/include/hw/loader.h
index 5572108ba5..70248e0da7 100644
--- a/include/hw/loader.h
+++ b/include/hw/loader.h
@@ -40,8 +40,8 @@ ssize_t load_image_size(const char *filename, void *addr, size_t size);
  *
  * Returns the size of the loaded image on success, -1 otherwise.
  */
-int load_image_targphys_as(const char *filename,
-                           hwaddr addr, uint64_t max_sz, AddressSpace *as);
+ssize_t load_image_targphys_as(const char *filename,
+                               hwaddr addr, uint64_t max_sz, AddressSpace *as);
 
 /**load_targphys_hex_as:
  * @filename: Path to the .hex file
@@ -53,14 +53,15 @@ int load_image_targphys_as(const char *filename,
  *
  * Returns the size of the loaded .hex file on success, -1 otherwise.
  */
-int load_targphys_hex_as(const char *filename, hwaddr *entry, AddressSpace *as);
+ssize_t load_targphys_hex_as(const char *filename, hwaddr *entry,
+                             AddressSpace *as);
 
 /** load_image_targphys:
  * Same as load_image_targphys_as(), but doesn't allow the caller to specify
  * an AddressSpace.
  */
-int load_image_targphys(const char *filename, hwaddr,
-                        uint64_t max_sz);
+ssize_t load_image_targphys(const char *filename, hwaddr,
+                            uint64_t max_sz);
 
 /**
  * load_image_mr: load an image into a memory region
@@ -73,7 +74,7 @@ int load_image_targphys(const char *filename, hwaddr,
  * If the file is larger than the memory region's size the call will fail.
  * Returns -1 on failure, or the size of the file.
  */
-int load_image_mr(const char *filename, MemoryRegion *mr);
+ssize_t load_image_mr(const char *filename, MemoryRegion *mr);
 
 /* This is the limit on the maximum uncompressed image size that
  * load_image_gzipped_buffer() and load_image_gzipped() will read. It prevents
@@ -81,9 +82,9 @@ int load_image_mr(const char *filename, MemoryRegion *mr);
  */
 #define LOAD_IMAGE_MAX_GUNZIP_BYTES (256 << 20)
 
-int load_image_gzipped_buffer(const char *filename, uint64_t max_sz,
-                              uint8_t **buffer);
-int load_image_gzipped(const char *filename, hwaddr addr, uint64_t max_sz);
+ssize_t load_image_gzipped_buffer(const char *filename, uint64_t max_sz,
+                                  uint8_t **buffer);
+ssize_t load_image_gzipped(const char *filename, hwaddr addr, uint64_t max_sz);
 
 #define ELF_LOAD_FAILED       -1
 #define ELF_LOAD_NOT_ELF      -2
@@ -183,8 +184,8 @@ ssize_t load_elf(const char *filename,
  */
 void load_elf_hdr(const char *filename, void *hdr, bool *is64, Error **errp);
 
-int load_aout(const char *filename, hwaddr addr, int max_sz,
-              int bswap_needed, hwaddr target_page_size);
+ssize_t load_aout(const char *filename, hwaddr addr, int max_sz,
+                  int bswap_needed, hwaddr target_page_size);
 
 #define LOAD_UIMAGE_LOADADDR_INVALID (-1)
 
@@ -205,19 +206,19 @@ int load_aout(const char *filename, hwaddr addr, int max_sz,
  *
  * Returns the size of the loaded image on success, -1 otherwise.
  */
-int load_uimage_as(const char *filename, hwaddr *ep,
-                   hwaddr *loadaddr, int *is_linux,
-                   uint64_t (*translate_fn)(void *, uint64_t),
-                   void *translate_opaque, AddressSpace *as);
+ssize_t load_uimage_as(const char *filename, hwaddr *ep,
+                       hwaddr *loadaddr, int *is_linux,
+                       uint64_t (*translate_fn)(void *, uint64_t),
+                       void *translate_opaque, AddressSpace *as);
 
 /** load_uimage:
  * Same as load_uimage_as(), but doesn't allow the caller to specify an
  * AddressSpace.
  */
-int load_uimage(const char *filename, hwaddr *ep,
-                hwaddr *loadaddr, int *is_linux,
-                uint64_t (*translate_fn)(void *, uint64_t),
-                void *translate_opaque);
+ssize_t load_uimage(const char *filename, hwaddr *ep,
+                    hwaddr *loadaddr, int *is_linux,
+                    uint64_t (*translate_fn)(void *, uint64_t),
+                    void *translate_opaque);
 
 /**
  * load_ramdisk_as:
@@ -232,15 +233,15 @@ int load_uimage(const char *filename, hwaddr *ep,
  *
  * Returns the size of the loaded image on success, -1 otherwise.
  */
-int load_ramdisk_as(const char *filename, hwaddr addr, uint64_t max_sz,
-                    AddressSpace *as);
+ssize_t load_ramdisk_as(const char *filename, hwaddr addr, uint64_t max_sz,
+                        AddressSpace *as);
 
 /**
  * load_ramdisk:
  * Same as load_ramdisk_as(), but doesn't allow the caller to specify
  * an AddressSpace.
  */
-int load_ramdisk(const char *filename, hwaddr addr, uint64_t max_sz);
+ssize_t load_ramdisk(const char *filename, hwaddr addr, uint64_t max_sz);
 
 ssize_t gunzip(void *dst, size_t dstlen, uint8_t *src, size_t srclen);
 
@@ -253,9 +254,9 @@ void pstrcpy_targphys(const char *name,
 extern bool option_rom_has_mr;
 extern bool rom_file_has_mr;
 
-int rom_add_file(const char *file, const char *fw_dir,
-                 hwaddr addr, int32_t bootindex,
-                 bool option_rom, MemoryRegion *mr, AddressSpace *as);
+ssize_t rom_add_file(const char *file, const char *fw_dir,
+                     hwaddr addr, int32_t bootindex,
+                     bool option_rom, MemoryRegion *mr, AddressSpace *as);
 MemoryRegion *rom_add_blob(const char *name, const void *blob, size_t len,
                            size_t max_len, hwaddr addr,
                            const char *fw_file_name,
@@ -336,8 +337,8 @@ void hmp_info_roms(Monitor *mon, const QDict *qdict);
 #define rom_add_blob_fixed_as(_f, _b, _l, _a, _as)      \
     rom_add_blob(_f, _b, _l, _l, _a, NULL, NULL, NULL, _as, true)
 
-int rom_add_vga(const char *file);
-int rom_add_option(const char *file, int32_t bootindex);
+ssize_t rom_add_vga(const char *file);
+ssize_t rom_add_option(const char *file, int32_t bootindex);
 
 /* This is the usual maximum in uboot, so if a uImage overflows this, it would
  * overflow on real hardware too. */

From 8f42415fc1d1bb462f2001bf5e2ad3b78f14b2e3 Mon Sep 17 00:00:00 2001
From: Andrew Bresticker <abrestic@rivosinc.com>
Date: Tue, 31 May 2022 17:05:44 -0400
Subject: [PATCH 916/935] target/riscv: Wake on VS-level external interrupts

Whether or not VSEIP is pending isn't reflected in env->mip and must
instead be determined from hstatus.vgein and hgeip. As a result a
CPU in WFI won't wake on a VSEIP, which violates the WFI behavior as
specified in the privileged ISA. Just use riscv_cpu_all_pending()
instead, which already accounts for VSEIP.

Signed-off-by: Andrew Bresticker <abrestic@rivosinc.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20220531210544.181322-1-abrestic@rivosinc.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.c        | 2 +-
 target/riscv/cpu.h        | 1 +
 target/riscv/cpu_helper.c | 2 +-
 3 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index bcbba3fbd5..0497af45cc 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -391,7 +391,7 @@ static bool riscv_cpu_has_work(CPUState *cs)
      * Definition of the WFI instruction requires it to ignore the privilege
      * mode and delegation registers, but respect individual enables
      */
-    return (env->mip & env->mie) != 0;
+    return riscv_cpu_all_pending(env) != 0;
 #else
     return true;
 #endif
diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
index 890d33cebb..194a58d760 100644
--- a/target/riscv/cpu.h
+++ b/target/riscv/cpu.h
@@ -489,6 +489,7 @@ int riscv_cpu_gdb_read_register(CPUState *cpu, GByteArray *buf, int reg);
 int riscv_cpu_gdb_write_register(CPUState *cpu, uint8_t *buf, int reg);
 int riscv_cpu_hviprio_index2irq(int index, int *out_irq, int *out_rdzero);
 uint8_t riscv_cpu_default_priority(int irq);
+uint64_t riscv_cpu_all_pending(CPURISCVState *env);
 int riscv_cpu_mirq_pending(CPURISCVState *env);
 int riscv_cpu_sirq_pending(CPURISCVState *env);
 int riscv_cpu_vsirq_pending(CPURISCVState *env);
diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index d99fac9d2d..16c6045459 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -340,7 +340,7 @@ static int riscv_cpu_pending_to_irq(CPURISCVState *env,
     return best_irq;
 }
 
-static uint64_t riscv_cpu_all_pending(CPURISCVState *env)
+uint64_t riscv_cpu_all_pending(CPURISCVState *env)
 {
     uint32_t gein = get_field(env->hstatus, HSTATUS_VGEIN);
     uint64_t vsgein = (env->hgeip & (1ULL << gein)) ? MIP_VSEIP : 0;

From d1d8541217ce8a23e9e751cd868c7d618817134a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Fr=C3=A9d=C3=A9ric=20P=C3=A9trot?=
 <frederic.petrot@univ-grenoble-alpes.fr>
Date: Thu, 2 Jun 2022 17:52:46 +0200
Subject: [PATCH 917/935] target/riscv/debug.c: keep experimental rv128 support
 working
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add an MXL_RV128 case in two switches so that no error is triggered when
using the -cpu x-rv128 option.

Signed-off-by: Frédéric Pétrot <frederic.petrot@univ-grenoble-alpes.fr>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Message-Id: <20220602155246.38837-1-frederic.petrot@univ-grenoble-alpes.fr>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/debug.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/target/riscv/debug.c b/target/riscv/debug.c
index 2f2a51c732..fc6e13222f 100644
--- a/target/riscv/debug.c
+++ b/target/riscv/debug.c
@@ -77,6 +77,7 @@ static inline target_ulong trigger_type(CPURISCVState *env,
         tdata1 = RV32_TYPE(type);
         break;
     case MXL_RV64:
+    case MXL_RV128:
         tdata1 = RV64_TYPE(type);
         break;
     default:
@@ -123,6 +124,7 @@ static target_ulong tdata1_validate(CPURISCVState *env, target_ulong val,
         tdata1 = RV32_TYPE(t);
         break;
     case MXL_RV64:
+    case MXL_RV128:
         type = extract64(val, 60, 4);
         dmode = extract64(val, 59, 1);
         tdata1 = RV64_TYPE(t);

From 8a085fb2ad812bf06145779bbb6a18a3e7439771 Mon Sep 17 00:00:00 2001
From: eopXD <yueh.ting.chen@gmail.com>
Date: Mon, 6 Jun 2022 06:16:16 +0000
Subject: [PATCH 918/935] target/riscv: rvv: Prune redundant ESZ, DSZ parameter
 passed

No functional change intended in this commit.

Signed-off-by: eop Chen <eop.chen@sifive.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <165449614532.19704.7000832880482980398-1@git.sr.ht>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/vector_helper.c | 1132 +++++++++++++++++-----------------
 1 file changed, 565 insertions(+), 567 deletions(-)

diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c
index 576b14e5a3..85dd611cd9 100644
--- a/target/riscv/vector_helper.c
+++ b/target/riscv/vector_helper.c
@@ -710,7 +710,6 @@ RVVCALL(OPIVV2, vsub_vv_d, OP_SSS_D, H8, H8, H8, DO_SUB)
 
 static void do_vext_vv(void *vd, void *v0, void *vs1, void *vs2,
                        CPURISCVState *env, uint32_t desc,
-                       uint32_t esz, uint32_t dsz,
                        opivv2_fn *fn)
 {
     uint32_t vm = vext_vm(desc);
@@ -727,23 +726,23 @@ static void do_vext_vv(void *vd, void *v0, void *vs1, void *vs2,
 }
 
 /* generate the helpers for OPIVV */
-#define GEN_VEXT_VV(NAME, ESZ, DSZ)                       \
+#define GEN_VEXT_VV(NAME)                                 \
 void HELPER(NAME)(void *vd, void *v0, void *vs1,          \
                   void *vs2, CPURISCVState *env,          \
                   uint32_t desc)                          \
 {                                                         \
-    do_vext_vv(vd, v0, vs1, vs2, env, desc, ESZ, DSZ,     \
+    do_vext_vv(vd, v0, vs1, vs2, env, desc,               \
                do_##NAME);                                \
 }
 
-GEN_VEXT_VV(vadd_vv_b, 1, 1)
-GEN_VEXT_VV(vadd_vv_h, 2, 2)
-GEN_VEXT_VV(vadd_vv_w, 4, 4)
-GEN_VEXT_VV(vadd_vv_d, 8, 8)
-GEN_VEXT_VV(vsub_vv_b, 1, 1)
-GEN_VEXT_VV(vsub_vv_h, 2, 2)
-GEN_VEXT_VV(vsub_vv_w, 4, 4)
-GEN_VEXT_VV(vsub_vv_d, 8, 8)
+GEN_VEXT_VV(vadd_vv_b)
+GEN_VEXT_VV(vadd_vv_h)
+GEN_VEXT_VV(vadd_vv_w)
+GEN_VEXT_VV(vadd_vv_d)
+GEN_VEXT_VV(vsub_vv_b)
+GEN_VEXT_VV(vsub_vv_h)
+GEN_VEXT_VV(vsub_vv_w)
+GEN_VEXT_VV(vsub_vv_d)
 
 typedef void opivx2_fn(void *vd, target_long s1, void *vs2, int i);
 
@@ -773,7 +772,6 @@ RVVCALL(OPIVX2, vrsub_vx_d, OP_SSS_D, H8, H8, DO_RSUB)
 
 static void do_vext_vx(void *vd, void *v0, target_long s1, void *vs2,
                        CPURISCVState *env, uint32_t desc,
-                       uint32_t esz, uint32_t dsz,
                        opivx2_fn fn)
 {
     uint32_t vm = vext_vm(desc);
@@ -790,27 +788,27 @@ static void do_vext_vx(void *vd, void *v0, target_long s1, void *vs2,
 }
 
 /* generate the helpers for OPIVX */
-#define GEN_VEXT_VX(NAME, ESZ, DSZ)                       \
+#define GEN_VEXT_VX(NAME)                                 \
 void HELPER(NAME)(void *vd, void *v0, target_ulong s1,    \
                   void *vs2, CPURISCVState *env,          \
                   uint32_t desc)                          \
 {                                                         \
-    do_vext_vx(vd, v0, s1, vs2, env, desc, ESZ, DSZ,      \
+    do_vext_vx(vd, v0, s1, vs2, env, desc,                \
                do_##NAME);                                \
 }
 
-GEN_VEXT_VX(vadd_vx_b, 1, 1)
-GEN_VEXT_VX(vadd_vx_h, 2, 2)
-GEN_VEXT_VX(vadd_vx_w, 4, 4)
-GEN_VEXT_VX(vadd_vx_d, 8, 8)
-GEN_VEXT_VX(vsub_vx_b, 1, 1)
-GEN_VEXT_VX(vsub_vx_h, 2, 2)
-GEN_VEXT_VX(vsub_vx_w, 4, 4)
-GEN_VEXT_VX(vsub_vx_d, 8, 8)
-GEN_VEXT_VX(vrsub_vx_b, 1, 1)
-GEN_VEXT_VX(vrsub_vx_h, 2, 2)
-GEN_VEXT_VX(vrsub_vx_w, 4, 4)
-GEN_VEXT_VX(vrsub_vx_d, 8, 8)
+GEN_VEXT_VX(vadd_vx_b)
+GEN_VEXT_VX(vadd_vx_h)
+GEN_VEXT_VX(vadd_vx_w)
+GEN_VEXT_VX(vadd_vx_d)
+GEN_VEXT_VX(vsub_vx_b)
+GEN_VEXT_VX(vsub_vx_h)
+GEN_VEXT_VX(vsub_vx_w)
+GEN_VEXT_VX(vsub_vx_d)
+GEN_VEXT_VX(vrsub_vx_b)
+GEN_VEXT_VX(vrsub_vx_h)
+GEN_VEXT_VX(vrsub_vx_w)
+GEN_VEXT_VX(vrsub_vx_d)
 
 void HELPER(vec_rsubs8)(void *d, void *a, uint64_t b, uint32_t desc)
 {
@@ -889,30 +887,30 @@ RVVCALL(OPIVV2, vwadd_wv_w, WOP_WSSS_W, H8, H4, H4, DO_ADD)
 RVVCALL(OPIVV2, vwsub_wv_b, WOP_WSSS_B, H2, H1, H1, DO_SUB)
 RVVCALL(OPIVV2, vwsub_wv_h, WOP_WSSS_H, H4, H2, H2, DO_SUB)
 RVVCALL(OPIVV2, vwsub_wv_w, WOP_WSSS_W, H8, H4, H4, DO_SUB)
-GEN_VEXT_VV(vwaddu_vv_b, 1, 2)
-GEN_VEXT_VV(vwaddu_vv_h, 2, 4)
-GEN_VEXT_VV(vwaddu_vv_w, 4, 8)
-GEN_VEXT_VV(vwsubu_vv_b, 1, 2)
-GEN_VEXT_VV(vwsubu_vv_h, 2, 4)
-GEN_VEXT_VV(vwsubu_vv_w, 4, 8)
-GEN_VEXT_VV(vwadd_vv_b, 1, 2)
-GEN_VEXT_VV(vwadd_vv_h, 2, 4)
-GEN_VEXT_VV(vwadd_vv_w, 4, 8)
-GEN_VEXT_VV(vwsub_vv_b, 1, 2)
-GEN_VEXT_VV(vwsub_vv_h, 2, 4)
-GEN_VEXT_VV(vwsub_vv_w, 4, 8)
-GEN_VEXT_VV(vwaddu_wv_b, 1, 2)
-GEN_VEXT_VV(vwaddu_wv_h, 2, 4)
-GEN_VEXT_VV(vwaddu_wv_w, 4, 8)
-GEN_VEXT_VV(vwsubu_wv_b, 1, 2)
-GEN_VEXT_VV(vwsubu_wv_h, 2, 4)
-GEN_VEXT_VV(vwsubu_wv_w, 4, 8)
-GEN_VEXT_VV(vwadd_wv_b, 1, 2)
-GEN_VEXT_VV(vwadd_wv_h, 2, 4)
-GEN_VEXT_VV(vwadd_wv_w, 4, 8)
-GEN_VEXT_VV(vwsub_wv_b, 1, 2)
-GEN_VEXT_VV(vwsub_wv_h, 2, 4)
-GEN_VEXT_VV(vwsub_wv_w, 4, 8)
+GEN_VEXT_VV(vwaddu_vv_b)
+GEN_VEXT_VV(vwaddu_vv_h)
+GEN_VEXT_VV(vwaddu_vv_w)
+GEN_VEXT_VV(vwsubu_vv_b)
+GEN_VEXT_VV(vwsubu_vv_h)
+GEN_VEXT_VV(vwsubu_vv_w)
+GEN_VEXT_VV(vwadd_vv_b)
+GEN_VEXT_VV(vwadd_vv_h)
+GEN_VEXT_VV(vwadd_vv_w)
+GEN_VEXT_VV(vwsub_vv_b)
+GEN_VEXT_VV(vwsub_vv_h)
+GEN_VEXT_VV(vwsub_vv_w)
+GEN_VEXT_VV(vwaddu_wv_b)
+GEN_VEXT_VV(vwaddu_wv_h)
+GEN_VEXT_VV(vwaddu_wv_w)
+GEN_VEXT_VV(vwsubu_wv_b)
+GEN_VEXT_VV(vwsubu_wv_h)
+GEN_VEXT_VV(vwsubu_wv_w)
+GEN_VEXT_VV(vwadd_wv_b)
+GEN_VEXT_VV(vwadd_wv_h)
+GEN_VEXT_VV(vwadd_wv_w)
+GEN_VEXT_VV(vwsub_wv_b)
+GEN_VEXT_VV(vwsub_wv_h)
+GEN_VEXT_VV(vwsub_wv_w)
 
 RVVCALL(OPIVX2, vwaddu_vx_b, WOP_UUU_B, H2, H1, DO_ADD)
 RVVCALL(OPIVX2, vwaddu_vx_h, WOP_UUU_H, H4, H2, DO_ADD)
@@ -938,30 +936,30 @@ RVVCALL(OPIVX2, vwadd_wx_w, WOP_WSSS_W, H8, H4, DO_ADD)
 RVVCALL(OPIVX2, vwsub_wx_b, WOP_WSSS_B, H2, H1, DO_SUB)
 RVVCALL(OPIVX2, vwsub_wx_h, WOP_WSSS_H, H4, H2, DO_SUB)
 RVVCALL(OPIVX2, vwsub_wx_w, WOP_WSSS_W, H8, H4, DO_SUB)
-GEN_VEXT_VX(vwaddu_vx_b, 1, 2)
-GEN_VEXT_VX(vwaddu_vx_h, 2, 4)
-GEN_VEXT_VX(vwaddu_vx_w, 4, 8)
-GEN_VEXT_VX(vwsubu_vx_b, 1, 2)
-GEN_VEXT_VX(vwsubu_vx_h, 2, 4)
-GEN_VEXT_VX(vwsubu_vx_w, 4, 8)
-GEN_VEXT_VX(vwadd_vx_b, 1, 2)
-GEN_VEXT_VX(vwadd_vx_h, 2, 4)
-GEN_VEXT_VX(vwadd_vx_w, 4, 8)
-GEN_VEXT_VX(vwsub_vx_b, 1, 2)
-GEN_VEXT_VX(vwsub_vx_h, 2, 4)
-GEN_VEXT_VX(vwsub_vx_w, 4, 8)
-GEN_VEXT_VX(vwaddu_wx_b, 1, 2)
-GEN_VEXT_VX(vwaddu_wx_h, 2, 4)
-GEN_VEXT_VX(vwaddu_wx_w, 4, 8)
-GEN_VEXT_VX(vwsubu_wx_b, 1, 2)
-GEN_VEXT_VX(vwsubu_wx_h, 2, 4)
-GEN_VEXT_VX(vwsubu_wx_w, 4, 8)
-GEN_VEXT_VX(vwadd_wx_b, 1, 2)
-GEN_VEXT_VX(vwadd_wx_h, 2, 4)
-GEN_VEXT_VX(vwadd_wx_w, 4, 8)
-GEN_VEXT_VX(vwsub_wx_b, 1, 2)
-GEN_VEXT_VX(vwsub_wx_h, 2, 4)
-GEN_VEXT_VX(vwsub_wx_w, 4, 8)
+GEN_VEXT_VX(vwaddu_vx_b)
+GEN_VEXT_VX(vwaddu_vx_h)
+GEN_VEXT_VX(vwaddu_vx_w)
+GEN_VEXT_VX(vwsubu_vx_b)
+GEN_VEXT_VX(vwsubu_vx_h)
+GEN_VEXT_VX(vwsubu_vx_w)
+GEN_VEXT_VX(vwadd_vx_b)
+GEN_VEXT_VX(vwadd_vx_h)
+GEN_VEXT_VX(vwadd_vx_w)
+GEN_VEXT_VX(vwsub_vx_b)
+GEN_VEXT_VX(vwsub_vx_h)
+GEN_VEXT_VX(vwsub_vx_w)
+GEN_VEXT_VX(vwaddu_wx_b)
+GEN_VEXT_VX(vwaddu_wx_h)
+GEN_VEXT_VX(vwaddu_wx_w)
+GEN_VEXT_VX(vwsubu_wx_b)
+GEN_VEXT_VX(vwsubu_wx_h)
+GEN_VEXT_VX(vwsubu_wx_w)
+GEN_VEXT_VX(vwadd_wx_b)
+GEN_VEXT_VX(vwadd_wx_h)
+GEN_VEXT_VX(vwadd_wx_w)
+GEN_VEXT_VX(vwsub_wx_b)
+GEN_VEXT_VX(vwsub_wx_h)
+GEN_VEXT_VX(vwsub_wx_w)
 
 /* Vector Integer Add-with-Carry / Subtract-with-Borrow Instructions */
 #define DO_VADC(N, M, C) (N + M + C)
@@ -1091,18 +1089,18 @@ RVVCALL(OPIVV2, vxor_vv_b, OP_SSS_B, H1, H1, H1, DO_XOR)
 RVVCALL(OPIVV2, vxor_vv_h, OP_SSS_H, H2, H2, H2, DO_XOR)
 RVVCALL(OPIVV2, vxor_vv_w, OP_SSS_W, H4, H4, H4, DO_XOR)
 RVVCALL(OPIVV2, vxor_vv_d, OP_SSS_D, H8, H8, H8, DO_XOR)
-GEN_VEXT_VV(vand_vv_b, 1, 1)
-GEN_VEXT_VV(vand_vv_h, 2, 2)
-GEN_VEXT_VV(vand_vv_w, 4, 4)
-GEN_VEXT_VV(vand_vv_d, 8, 8)
-GEN_VEXT_VV(vor_vv_b, 1, 1)
-GEN_VEXT_VV(vor_vv_h, 2, 2)
-GEN_VEXT_VV(vor_vv_w, 4, 4)
-GEN_VEXT_VV(vor_vv_d, 8, 8)
-GEN_VEXT_VV(vxor_vv_b, 1, 1)
-GEN_VEXT_VV(vxor_vv_h, 2, 2)
-GEN_VEXT_VV(vxor_vv_w, 4, 4)
-GEN_VEXT_VV(vxor_vv_d, 8, 8)
+GEN_VEXT_VV(vand_vv_b)
+GEN_VEXT_VV(vand_vv_h)
+GEN_VEXT_VV(vand_vv_w)
+GEN_VEXT_VV(vand_vv_d)
+GEN_VEXT_VV(vor_vv_b)
+GEN_VEXT_VV(vor_vv_h)
+GEN_VEXT_VV(vor_vv_w)
+GEN_VEXT_VV(vor_vv_d)
+GEN_VEXT_VV(vxor_vv_b)
+GEN_VEXT_VV(vxor_vv_h)
+GEN_VEXT_VV(vxor_vv_w)
+GEN_VEXT_VV(vxor_vv_d)
 
 RVVCALL(OPIVX2, vand_vx_b, OP_SSS_B, H1, H1, DO_AND)
 RVVCALL(OPIVX2, vand_vx_h, OP_SSS_H, H2, H2, DO_AND)
@@ -1116,18 +1114,18 @@ RVVCALL(OPIVX2, vxor_vx_b, OP_SSS_B, H1, H1, DO_XOR)
 RVVCALL(OPIVX2, vxor_vx_h, OP_SSS_H, H2, H2, DO_XOR)
 RVVCALL(OPIVX2, vxor_vx_w, OP_SSS_W, H4, H4, DO_XOR)
 RVVCALL(OPIVX2, vxor_vx_d, OP_SSS_D, H8, H8, DO_XOR)
-GEN_VEXT_VX(vand_vx_b, 1, 1)
-GEN_VEXT_VX(vand_vx_h, 2, 2)
-GEN_VEXT_VX(vand_vx_w, 4, 4)
-GEN_VEXT_VX(vand_vx_d, 8, 8)
-GEN_VEXT_VX(vor_vx_b, 1, 1)
-GEN_VEXT_VX(vor_vx_h, 2, 2)
-GEN_VEXT_VX(vor_vx_w, 4, 4)
-GEN_VEXT_VX(vor_vx_d, 8, 8)
-GEN_VEXT_VX(vxor_vx_b, 1, 1)
-GEN_VEXT_VX(vxor_vx_h, 2, 2)
-GEN_VEXT_VX(vxor_vx_w, 4, 4)
-GEN_VEXT_VX(vxor_vx_d, 8, 8)
+GEN_VEXT_VX(vand_vx_b)
+GEN_VEXT_VX(vand_vx_h)
+GEN_VEXT_VX(vand_vx_w)
+GEN_VEXT_VX(vand_vx_d)
+GEN_VEXT_VX(vor_vx_b)
+GEN_VEXT_VX(vor_vx_h)
+GEN_VEXT_VX(vor_vx_w)
+GEN_VEXT_VX(vor_vx_d)
+GEN_VEXT_VX(vxor_vx_b)
+GEN_VEXT_VX(vxor_vx_h)
+GEN_VEXT_VX(vxor_vx_w)
+GEN_VEXT_VX(vxor_vx_d)
 
 /* Vector Single-Width Bit Shift Instructions */
 #define DO_SLL(N, M)  (N << (M))
@@ -1348,22 +1346,22 @@ RVVCALL(OPIVV2, vmax_vv_b, OP_SSS_B, H1, H1, H1, DO_MAX)
 RVVCALL(OPIVV2, vmax_vv_h, OP_SSS_H, H2, H2, H2, DO_MAX)
 RVVCALL(OPIVV2, vmax_vv_w, OP_SSS_W, H4, H4, H4, DO_MAX)
 RVVCALL(OPIVV2, vmax_vv_d, OP_SSS_D, H8, H8, H8, DO_MAX)
-GEN_VEXT_VV(vminu_vv_b, 1, 1)
-GEN_VEXT_VV(vminu_vv_h, 2, 2)
-GEN_VEXT_VV(vminu_vv_w, 4, 4)
-GEN_VEXT_VV(vminu_vv_d, 8, 8)
-GEN_VEXT_VV(vmin_vv_b, 1, 1)
-GEN_VEXT_VV(vmin_vv_h, 2, 2)
-GEN_VEXT_VV(vmin_vv_w, 4, 4)
-GEN_VEXT_VV(vmin_vv_d, 8, 8)
-GEN_VEXT_VV(vmaxu_vv_b, 1, 1)
-GEN_VEXT_VV(vmaxu_vv_h, 2, 2)
-GEN_VEXT_VV(vmaxu_vv_w, 4, 4)
-GEN_VEXT_VV(vmaxu_vv_d, 8, 8)
-GEN_VEXT_VV(vmax_vv_b, 1, 1)
-GEN_VEXT_VV(vmax_vv_h, 2, 2)
-GEN_VEXT_VV(vmax_vv_w, 4, 4)
-GEN_VEXT_VV(vmax_vv_d, 8, 8)
+GEN_VEXT_VV(vminu_vv_b)
+GEN_VEXT_VV(vminu_vv_h)
+GEN_VEXT_VV(vminu_vv_w)
+GEN_VEXT_VV(vminu_vv_d)
+GEN_VEXT_VV(vmin_vv_b)
+GEN_VEXT_VV(vmin_vv_h)
+GEN_VEXT_VV(vmin_vv_w)
+GEN_VEXT_VV(vmin_vv_d)
+GEN_VEXT_VV(vmaxu_vv_b)
+GEN_VEXT_VV(vmaxu_vv_h)
+GEN_VEXT_VV(vmaxu_vv_w)
+GEN_VEXT_VV(vmaxu_vv_d)
+GEN_VEXT_VV(vmax_vv_b)
+GEN_VEXT_VV(vmax_vv_h)
+GEN_VEXT_VV(vmax_vv_w)
+GEN_VEXT_VV(vmax_vv_d)
 
 RVVCALL(OPIVX2, vminu_vx_b, OP_UUU_B, H1, H1, DO_MIN)
 RVVCALL(OPIVX2, vminu_vx_h, OP_UUU_H, H2, H2, DO_MIN)
@@ -1381,22 +1379,22 @@ RVVCALL(OPIVX2, vmax_vx_b, OP_SSS_B, H1, H1, DO_MAX)
 RVVCALL(OPIVX2, vmax_vx_h, OP_SSS_H, H2, H2, DO_MAX)
 RVVCALL(OPIVX2, vmax_vx_w, OP_SSS_W, H4, H4, DO_MAX)
 RVVCALL(OPIVX2, vmax_vx_d, OP_SSS_D, H8, H8, DO_MAX)
-GEN_VEXT_VX(vminu_vx_b, 1, 1)
-GEN_VEXT_VX(vminu_vx_h, 2, 2)
-GEN_VEXT_VX(vminu_vx_w, 4, 4)
-GEN_VEXT_VX(vminu_vx_d, 8, 8)
-GEN_VEXT_VX(vmin_vx_b, 1, 1)
-GEN_VEXT_VX(vmin_vx_h, 2, 2)
-GEN_VEXT_VX(vmin_vx_w, 4, 4)
-GEN_VEXT_VX(vmin_vx_d, 8, 8)
-GEN_VEXT_VX(vmaxu_vx_b, 1, 1)
-GEN_VEXT_VX(vmaxu_vx_h, 2, 2)
-GEN_VEXT_VX(vmaxu_vx_w, 4, 4)
-GEN_VEXT_VX(vmaxu_vx_d, 8, 8)
-GEN_VEXT_VX(vmax_vx_b, 1, 1)
-GEN_VEXT_VX(vmax_vx_h, 2, 2)
-GEN_VEXT_VX(vmax_vx_w, 4, 4)
-GEN_VEXT_VX(vmax_vx_d, 8, 8)
+GEN_VEXT_VX(vminu_vx_b)
+GEN_VEXT_VX(vminu_vx_h)
+GEN_VEXT_VX(vminu_vx_w)
+GEN_VEXT_VX(vminu_vx_d)
+GEN_VEXT_VX(vmin_vx_b)
+GEN_VEXT_VX(vmin_vx_h)
+GEN_VEXT_VX(vmin_vx_w)
+GEN_VEXT_VX(vmin_vx_d)
+GEN_VEXT_VX(vmaxu_vx_b)
+GEN_VEXT_VX(vmaxu_vx_h)
+GEN_VEXT_VX(vmaxu_vx_w)
+GEN_VEXT_VX(vmaxu_vx_d)
+GEN_VEXT_VX(vmax_vx_b)
+GEN_VEXT_VX(vmax_vx_h)
+GEN_VEXT_VX(vmax_vx_w)
+GEN_VEXT_VX(vmax_vx_d)
 
 /* Vector Single-Width Integer Multiply Instructions */
 #define DO_MUL(N, M) (N * M)
@@ -1404,10 +1402,10 @@ RVVCALL(OPIVV2, vmul_vv_b, OP_SSS_B, H1, H1, H1, DO_MUL)
 RVVCALL(OPIVV2, vmul_vv_h, OP_SSS_H, H2, H2, H2, DO_MUL)
 RVVCALL(OPIVV2, vmul_vv_w, OP_SSS_W, H4, H4, H4, DO_MUL)
 RVVCALL(OPIVV2, vmul_vv_d, OP_SSS_D, H8, H8, H8, DO_MUL)
-GEN_VEXT_VV(vmul_vv_b, 1, 1)
-GEN_VEXT_VV(vmul_vv_h, 2, 2)
-GEN_VEXT_VV(vmul_vv_w, 4, 4)
-GEN_VEXT_VV(vmul_vv_d, 8, 8)
+GEN_VEXT_VV(vmul_vv_b)
+GEN_VEXT_VV(vmul_vv_h)
+GEN_VEXT_VV(vmul_vv_w)
+GEN_VEXT_VV(vmul_vv_d)
 
 static int8_t do_mulh_b(int8_t s2, int8_t s1)
 {
@@ -1511,18 +1509,18 @@ RVVCALL(OPIVV2, vmulhsu_vv_b, OP_SUS_B, H1, H1, H1, do_mulhsu_b)
 RVVCALL(OPIVV2, vmulhsu_vv_h, OP_SUS_H, H2, H2, H2, do_mulhsu_h)
 RVVCALL(OPIVV2, vmulhsu_vv_w, OP_SUS_W, H4, H4, H4, do_mulhsu_w)
 RVVCALL(OPIVV2, vmulhsu_vv_d, OP_SUS_D, H8, H8, H8, do_mulhsu_d)
-GEN_VEXT_VV(vmulh_vv_b, 1, 1)
-GEN_VEXT_VV(vmulh_vv_h, 2, 2)
-GEN_VEXT_VV(vmulh_vv_w, 4, 4)
-GEN_VEXT_VV(vmulh_vv_d, 8, 8)
-GEN_VEXT_VV(vmulhu_vv_b, 1, 1)
-GEN_VEXT_VV(vmulhu_vv_h, 2, 2)
-GEN_VEXT_VV(vmulhu_vv_w, 4, 4)
-GEN_VEXT_VV(vmulhu_vv_d, 8, 8)
-GEN_VEXT_VV(vmulhsu_vv_b, 1, 1)
-GEN_VEXT_VV(vmulhsu_vv_h, 2, 2)
-GEN_VEXT_VV(vmulhsu_vv_w, 4, 4)
-GEN_VEXT_VV(vmulhsu_vv_d, 8, 8)
+GEN_VEXT_VV(vmulh_vv_b)
+GEN_VEXT_VV(vmulh_vv_h)
+GEN_VEXT_VV(vmulh_vv_w)
+GEN_VEXT_VV(vmulh_vv_d)
+GEN_VEXT_VV(vmulhu_vv_b)
+GEN_VEXT_VV(vmulhu_vv_h)
+GEN_VEXT_VV(vmulhu_vv_w)
+GEN_VEXT_VV(vmulhu_vv_d)
+GEN_VEXT_VV(vmulhsu_vv_b)
+GEN_VEXT_VV(vmulhsu_vv_h)
+GEN_VEXT_VV(vmulhsu_vv_w)
+GEN_VEXT_VV(vmulhsu_vv_d)
 
 RVVCALL(OPIVX2, vmul_vx_b, OP_SSS_B, H1, H1, DO_MUL)
 RVVCALL(OPIVX2, vmul_vx_h, OP_SSS_H, H2, H2, DO_MUL)
@@ -1540,22 +1538,22 @@ RVVCALL(OPIVX2, vmulhsu_vx_b, OP_SUS_B, H1, H1, do_mulhsu_b)
 RVVCALL(OPIVX2, vmulhsu_vx_h, OP_SUS_H, H2, H2, do_mulhsu_h)
 RVVCALL(OPIVX2, vmulhsu_vx_w, OP_SUS_W, H4, H4, do_mulhsu_w)
 RVVCALL(OPIVX2, vmulhsu_vx_d, OP_SUS_D, H8, H8, do_mulhsu_d)
-GEN_VEXT_VX(vmul_vx_b, 1, 1)
-GEN_VEXT_VX(vmul_vx_h, 2, 2)
-GEN_VEXT_VX(vmul_vx_w, 4, 4)
-GEN_VEXT_VX(vmul_vx_d, 8, 8)
-GEN_VEXT_VX(vmulh_vx_b, 1, 1)
-GEN_VEXT_VX(vmulh_vx_h, 2, 2)
-GEN_VEXT_VX(vmulh_vx_w, 4, 4)
-GEN_VEXT_VX(vmulh_vx_d, 8, 8)
-GEN_VEXT_VX(vmulhu_vx_b, 1, 1)
-GEN_VEXT_VX(vmulhu_vx_h, 2, 2)
-GEN_VEXT_VX(vmulhu_vx_w, 4, 4)
-GEN_VEXT_VX(vmulhu_vx_d, 8, 8)
-GEN_VEXT_VX(vmulhsu_vx_b, 1, 1)
-GEN_VEXT_VX(vmulhsu_vx_h, 2, 2)
-GEN_VEXT_VX(vmulhsu_vx_w, 4, 4)
-GEN_VEXT_VX(vmulhsu_vx_d, 8, 8)
+GEN_VEXT_VX(vmul_vx_b)
+GEN_VEXT_VX(vmul_vx_h)
+GEN_VEXT_VX(vmul_vx_w)
+GEN_VEXT_VX(vmul_vx_d)
+GEN_VEXT_VX(vmulh_vx_b)
+GEN_VEXT_VX(vmulh_vx_h)
+GEN_VEXT_VX(vmulh_vx_w)
+GEN_VEXT_VX(vmulh_vx_d)
+GEN_VEXT_VX(vmulhu_vx_b)
+GEN_VEXT_VX(vmulhu_vx_h)
+GEN_VEXT_VX(vmulhu_vx_w)
+GEN_VEXT_VX(vmulhu_vx_d)
+GEN_VEXT_VX(vmulhsu_vx_b)
+GEN_VEXT_VX(vmulhsu_vx_h)
+GEN_VEXT_VX(vmulhsu_vx_w)
+GEN_VEXT_VX(vmulhsu_vx_d)
 
 /* Vector Integer Divide Instructions */
 #define DO_DIVU(N, M) (unlikely(M == 0) ? (__typeof(N))(-1) : N / M)
@@ -1581,22 +1579,22 @@ RVVCALL(OPIVV2, vrem_vv_b, OP_SSS_B, H1, H1, H1, DO_REM)
 RVVCALL(OPIVV2, vrem_vv_h, OP_SSS_H, H2, H2, H2, DO_REM)
 RVVCALL(OPIVV2, vrem_vv_w, OP_SSS_W, H4, H4, H4, DO_REM)
 RVVCALL(OPIVV2, vrem_vv_d, OP_SSS_D, H8, H8, H8, DO_REM)
-GEN_VEXT_VV(vdivu_vv_b, 1, 1)
-GEN_VEXT_VV(vdivu_vv_h, 2, 2)
-GEN_VEXT_VV(vdivu_vv_w, 4, 4)
-GEN_VEXT_VV(vdivu_vv_d, 8, 8)
-GEN_VEXT_VV(vdiv_vv_b, 1, 1)
-GEN_VEXT_VV(vdiv_vv_h, 2, 2)
-GEN_VEXT_VV(vdiv_vv_w, 4, 4)
-GEN_VEXT_VV(vdiv_vv_d, 8, 8)
-GEN_VEXT_VV(vremu_vv_b, 1, 1)
-GEN_VEXT_VV(vremu_vv_h, 2, 2)
-GEN_VEXT_VV(vremu_vv_w, 4, 4)
-GEN_VEXT_VV(vremu_vv_d, 8, 8)
-GEN_VEXT_VV(vrem_vv_b, 1, 1)
-GEN_VEXT_VV(vrem_vv_h, 2, 2)
-GEN_VEXT_VV(vrem_vv_w, 4, 4)
-GEN_VEXT_VV(vrem_vv_d, 8, 8)
+GEN_VEXT_VV(vdivu_vv_b)
+GEN_VEXT_VV(vdivu_vv_h)
+GEN_VEXT_VV(vdivu_vv_w)
+GEN_VEXT_VV(vdivu_vv_d)
+GEN_VEXT_VV(vdiv_vv_b)
+GEN_VEXT_VV(vdiv_vv_h)
+GEN_VEXT_VV(vdiv_vv_w)
+GEN_VEXT_VV(vdiv_vv_d)
+GEN_VEXT_VV(vremu_vv_b)
+GEN_VEXT_VV(vremu_vv_h)
+GEN_VEXT_VV(vremu_vv_w)
+GEN_VEXT_VV(vremu_vv_d)
+GEN_VEXT_VV(vrem_vv_b)
+GEN_VEXT_VV(vrem_vv_h)
+GEN_VEXT_VV(vrem_vv_w)
+GEN_VEXT_VV(vrem_vv_d)
 
 RVVCALL(OPIVX2, vdivu_vx_b, OP_UUU_B, H1, H1, DO_DIVU)
 RVVCALL(OPIVX2, vdivu_vx_h, OP_UUU_H, H2, H2, DO_DIVU)
@@ -1614,22 +1612,22 @@ RVVCALL(OPIVX2, vrem_vx_b, OP_SSS_B, H1, H1, DO_REM)
 RVVCALL(OPIVX2, vrem_vx_h, OP_SSS_H, H2, H2, DO_REM)
 RVVCALL(OPIVX2, vrem_vx_w, OP_SSS_W, H4, H4, DO_REM)
 RVVCALL(OPIVX2, vrem_vx_d, OP_SSS_D, H8, H8, DO_REM)
-GEN_VEXT_VX(vdivu_vx_b, 1, 1)
-GEN_VEXT_VX(vdivu_vx_h, 2, 2)
-GEN_VEXT_VX(vdivu_vx_w, 4, 4)
-GEN_VEXT_VX(vdivu_vx_d, 8, 8)
-GEN_VEXT_VX(vdiv_vx_b, 1, 1)
-GEN_VEXT_VX(vdiv_vx_h, 2, 2)
-GEN_VEXT_VX(vdiv_vx_w, 4, 4)
-GEN_VEXT_VX(vdiv_vx_d, 8, 8)
-GEN_VEXT_VX(vremu_vx_b, 1, 1)
-GEN_VEXT_VX(vremu_vx_h, 2, 2)
-GEN_VEXT_VX(vremu_vx_w, 4, 4)
-GEN_VEXT_VX(vremu_vx_d, 8, 8)
-GEN_VEXT_VX(vrem_vx_b, 1, 1)
-GEN_VEXT_VX(vrem_vx_h, 2, 2)
-GEN_VEXT_VX(vrem_vx_w, 4, 4)
-GEN_VEXT_VX(vrem_vx_d, 8, 8)
+GEN_VEXT_VX(vdivu_vx_b)
+GEN_VEXT_VX(vdivu_vx_h)
+GEN_VEXT_VX(vdivu_vx_w)
+GEN_VEXT_VX(vdivu_vx_d)
+GEN_VEXT_VX(vdiv_vx_b)
+GEN_VEXT_VX(vdiv_vx_h)
+GEN_VEXT_VX(vdiv_vx_w)
+GEN_VEXT_VX(vdiv_vx_d)
+GEN_VEXT_VX(vremu_vx_b)
+GEN_VEXT_VX(vremu_vx_h)
+GEN_VEXT_VX(vremu_vx_w)
+GEN_VEXT_VX(vremu_vx_d)
+GEN_VEXT_VX(vrem_vx_b)
+GEN_VEXT_VX(vrem_vx_h)
+GEN_VEXT_VX(vrem_vx_w)
+GEN_VEXT_VX(vrem_vx_d)
 
 /* Vector Widening Integer Multiply Instructions */
 RVVCALL(OPIVV2, vwmul_vv_b, WOP_SSS_B, H2, H1, H1, DO_MUL)
@@ -1641,15 +1639,15 @@ RVVCALL(OPIVV2, vwmulu_vv_w, WOP_UUU_W, H8, H4, H4, DO_MUL)
 RVVCALL(OPIVV2, vwmulsu_vv_b, WOP_SUS_B, H2, H1, H1, DO_MUL)
 RVVCALL(OPIVV2, vwmulsu_vv_h, WOP_SUS_H, H4, H2, H2, DO_MUL)
 RVVCALL(OPIVV2, vwmulsu_vv_w, WOP_SUS_W, H8, H4, H4, DO_MUL)
-GEN_VEXT_VV(vwmul_vv_b, 1, 2)
-GEN_VEXT_VV(vwmul_vv_h, 2, 4)
-GEN_VEXT_VV(vwmul_vv_w, 4, 8)
-GEN_VEXT_VV(vwmulu_vv_b, 1, 2)
-GEN_VEXT_VV(vwmulu_vv_h, 2, 4)
-GEN_VEXT_VV(vwmulu_vv_w, 4, 8)
-GEN_VEXT_VV(vwmulsu_vv_b, 1, 2)
-GEN_VEXT_VV(vwmulsu_vv_h, 2, 4)
-GEN_VEXT_VV(vwmulsu_vv_w, 4, 8)
+GEN_VEXT_VV(vwmul_vv_b)
+GEN_VEXT_VV(vwmul_vv_h)
+GEN_VEXT_VV(vwmul_vv_w)
+GEN_VEXT_VV(vwmulu_vv_b)
+GEN_VEXT_VV(vwmulu_vv_h)
+GEN_VEXT_VV(vwmulu_vv_w)
+GEN_VEXT_VV(vwmulsu_vv_b)
+GEN_VEXT_VV(vwmulsu_vv_h)
+GEN_VEXT_VV(vwmulsu_vv_w)
 
 RVVCALL(OPIVX2, vwmul_vx_b, WOP_SSS_B, H2, H1, DO_MUL)
 RVVCALL(OPIVX2, vwmul_vx_h, WOP_SSS_H, H4, H2, DO_MUL)
@@ -1660,15 +1658,15 @@ RVVCALL(OPIVX2, vwmulu_vx_w, WOP_UUU_W, H8, H4, DO_MUL)
 RVVCALL(OPIVX2, vwmulsu_vx_b, WOP_SUS_B, H2, H1, DO_MUL)
 RVVCALL(OPIVX2, vwmulsu_vx_h, WOP_SUS_H, H4, H2, DO_MUL)
 RVVCALL(OPIVX2, vwmulsu_vx_w, WOP_SUS_W, H8, H4, DO_MUL)
-GEN_VEXT_VX(vwmul_vx_b, 1, 2)
-GEN_VEXT_VX(vwmul_vx_h, 2, 4)
-GEN_VEXT_VX(vwmul_vx_w, 4, 8)
-GEN_VEXT_VX(vwmulu_vx_b, 1, 2)
-GEN_VEXT_VX(vwmulu_vx_h, 2, 4)
-GEN_VEXT_VX(vwmulu_vx_w, 4, 8)
-GEN_VEXT_VX(vwmulsu_vx_b, 1, 2)
-GEN_VEXT_VX(vwmulsu_vx_h, 2, 4)
-GEN_VEXT_VX(vwmulsu_vx_w, 4, 8)
+GEN_VEXT_VX(vwmul_vx_b)
+GEN_VEXT_VX(vwmul_vx_h)
+GEN_VEXT_VX(vwmul_vx_w)
+GEN_VEXT_VX(vwmulu_vx_b)
+GEN_VEXT_VX(vwmulu_vx_h)
+GEN_VEXT_VX(vwmulu_vx_w)
+GEN_VEXT_VX(vwmulsu_vx_b)
+GEN_VEXT_VX(vwmulsu_vx_h)
+GEN_VEXT_VX(vwmulsu_vx_w)
 
 /* Vector Single-Width Integer Multiply-Add Instructions */
 #define OPIVV3(NAME, TD, T1, T2, TX1, TX2, HD, HS1, HS2, OP)   \
@@ -1700,22 +1698,22 @@ RVVCALL(OPIVV3, vnmsub_vv_b, OP_SSS_B, H1, H1, H1, DO_NMSUB)
 RVVCALL(OPIVV3, vnmsub_vv_h, OP_SSS_H, H2, H2, H2, DO_NMSUB)
 RVVCALL(OPIVV3, vnmsub_vv_w, OP_SSS_W, H4, H4, H4, DO_NMSUB)
 RVVCALL(OPIVV3, vnmsub_vv_d, OP_SSS_D, H8, H8, H8, DO_NMSUB)
-GEN_VEXT_VV(vmacc_vv_b, 1, 1)
-GEN_VEXT_VV(vmacc_vv_h, 2, 2)
-GEN_VEXT_VV(vmacc_vv_w, 4, 4)
-GEN_VEXT_VV(vmacc_vv_d, 8, 8)
-GEN_VEXT_VV(vnmsac_vv_b, 1, 1)
-GEN_VEXT_VV(vnmsac_vv_h, 2, 2)
-GEN_VEXT_VV(vnmsac_vv_w, 4, 4)
-GEN_VEXT_VV(vnmsac_vv_d, 8, 8)
-GEN_VEXT_VV(vmadd_vv_b, 1, 1)
-GEN_VEXT_VV(vmadd_vv_h, 2, 2)
-GEN_VEXT_VV(vmadd_vv_w, 4, 4)
-GEN_VEXT_VV(vmadd_vv_d, 8, 8)
-GEN_VEXT_VV(vnmsub_vv_b, 1, 1)
-GEN_VEXT_VV(vnmsub_vv_h, 2, 2)
-GEN_VEXT_VV(vnmsub_vv_w, 4, 4)
-GEN_VEXT_VV(vnmsub_vv_d, 8, 8)
+GEN_VEXT_VV(vmacc_vv_b)
+GEN_VEXT_VV(vmacc_vv_h)
+GEN_VEXT_VV(vmacc_vv_w)
+GEN_VEXT_VV(vmacc_vv_d)
+GEN_VEXT_VV(vnmsac_vv_b)
+GEN_VEXT_VV(vnmsac_vv_h)
+GEN_VEXT_VV(vnmsac_vv_w)
+GEN_VEXT_VV(vnmsac_vv_d)
+GEN_VEXT_VV(vmadd_vv_b)
+GEN_VEXT_VV(vmadd_vv_h)
+GEN_VEXT_VV(vmadd_vv_w)
+GEN_VEXT_VV(vmadd_vv_d)
+GEN_VEXT_VV(vnmsub_vv_b)
+GEN_VEXT_VV(vnmsub_vv_h)
+GEN_VEXT_VV(vnmsub_vv_w)
+GEN_VEXT_VV(vnmsub_vv_d)
 
 #define OPIVX3(NAME, TD, T1, T2, TX1, TX2, HD, HS2, OP)             \
 static void do_##NAME(void *vd, target_long s1, void *vs2, int i)   \
@@ -1741,22 +1739,22 @@ RVVCALL(OPIVX3, vnmsub_vx_b, OP_SSS_B, H1, H1, DO_NMSUB)
 RVVCALL(OPIVX3, vnmsub_vx_h, OP_SSS_H, H2, H2, DO_NMSUB)
 RVVCALL(OPIVX3, vnmsub_vx_w, OP_SSS_W, H4, H4, DO_NMSUB)
 RVVCALL(OPIVX3, vnmsub_vx_d, OP_SSS_D, H8, H8, DO_NMSUB)
-GEN_VEXT_VX(vmacc_vx_b, 1, 1)
-GEN_VEXT_VX(vmacc_vx_h, 2, 2)
-GEN_VEXT_VX(vmacc_vx_w, 4, 4)
-GEN_VEXT_VX(vmacc_vx_d, 8, 8)
-GEN_VEXT_VX(vnmsac_vx_b, 1, 1)
-GEN_VEXT_VX(vnmsac_vx_h, 2, 2)
-GEN_VEXT_VX(vnmsac_vx_w, 4, 4)
-GEN_VEXT_VX(vnmsac_vx_d, 8, 8)
-GEN_VEXT_VX(vmadd_vx_b, 1, 1)
-GEN_VEXT_VX(vmadd_vx_h, 2, 2)
-GEN_VEXT_VX(vmadd_vx_w, 4, 4)
-GEN_VEXT_VX(vmadd_vx_d, 8, 8)
-GEN_VEXT_VX(vnmsub_vx_b, 1, 1)
-GEN_VEXT_VX(vnmsub_vx_h, 2, 2)
-GEN_VEXT_VX(vnmsub_vx_w, 4, 4)
-GEN_VEXT_VX(vnmsub_vx_d, 8, 8)
+GEN_VEXT_VX(vmacc_vx_b)
+GEN_VEXT_VX(vmacc_vx_h)
+GEN_VEXT_VX(vmacc_vx_w)
+GEN_VEXT_VX(vmacc_vx_d)
+GEN_VEXT_VX(vnmsac_vx_b)
+GEN_VEXT_VX(vnmsac_vx_h)
+GEN_VEXT_VX(vnmsac_vx_w)
+GEN_VEXT_VX(vnmsac_vx_d)
+GEN_VEXT_VX(vmadd_vx_b)
+GEN_VEXT_VX(vmadd_vx_h)
+GEN_VEXT_VX(vmadd_vx_w)
+GEN_VEXT_VX(vmadd_vx_d)
+GEN_VEXT_VX(vnmsub_vx_b)
+GEN_VEXT_VX(vnmsub_vx_h)
+GEN_VEXT_VX(vnmsub_vx_w)
+GEN_VEXT_VX(vnmsub_vx_d)
 
 /* Vector Widening Integer Multiply-Add Instructions */
 RVVCALL(OPIVV3, vwmaccu_vv_b, WOP_UUU_B, H2, H1, H1, DO_MACC)
@@ -1768,15 +1766,15 @@ RVVCALL(OPIVV3, vwmacc_vv_w, WOP_SSS_W, H8, H4, H4, DO_MACC)
 RVVCALL(OPIVV3, vwmaccsu_vv_b, WOP_SSU_B, H2, H1, H1, DO_MACC)
 RVVCALL(OPIVV3, vwmaccsu_vv_h, WOP_SSU_H, H4, H2, H2, DO_MACC)
 RVVCALL(OPIVV3, vwmaccsu_vv_w, WOP_SSU_W, H8, H4, H4, DO_MACC)
-GEN_VEXT_VV(vwmaccu_vv_b, 1, 2)
-GEN_VEXT_VV(vwmaccu_vv_h, 2, 4)
-GEN_VEXT_VV(vwmaccu_vv_w, 4, 8)
-GEN_VEXT_VV(vwmacc_vv_b, 1, 2)
-GEN_VEXT_VV(vwmacc_vv_h, 2, 4)
-GEN_VEXT_VV(vwmacc_vv_w, 4, 8)
-GEN_VEXT_VV(vwmaccsu_vv_b, 1, 2)
-GEN_VEXT_VV(vwmaccsu_vv_h, 2, 4)
-GEN_VEXT_VV(vwmaccsu_vv_w, 4, 8)
+GEN_VEXT_VV(vwmaccu_vv_b)
+GEN_VEXT_VV(vwmaccu_vv_h)
+GEN_VEXT_VV(vwmaccu_vv_w)
+GEN_VEXT_VV(vwmacc_vv_b)
+GEN_VEXT_VV(vwmacc_vv_h)
+GEN_VEXT_VV(vwmacc_vv_w)
+GEN_VEXT_VV(vwmaccsu_vv_b)
+GEN_VEXT_VV(vwmaccsu_vv_h)
+GEN_VEXT_VV(vwmaccsu_vv_w)
 
 RVVCALL(OPIVX3, vwmaccu_vx_b, WOP_UUU_B, H2, H1, DO_MACC)
 RVVCALL(OPIVX3, vwmaccu_vx_h, WOP_UUU_H, H4, H2, DO_MACC)
@@ -1790,18 +1788,18 @@ RVVCALL(OPIVX3, vwmaccsu_vx_w, WOP_SSU_W, H8, H4, DO_MACC)
 RVVCALL(OPIVX3, vwmaccus_vx_b, WOP_SUS_B, H2, H1, DO_MACC)
 RVVCALL(OPIVX3, vwmaccus_vx_h, WOP_SUS_H, H4, H2, DO_MACC)
 RVVCALL(OPIVX3, vwmaccus_vx_w, WOP_SUS_W, H8, H4, DO_MACC)
-GEN_VEXT_VX(vwmaccu_vx_b, 1, 2)
-GEN_VEXT_VX(vwmaccu_vx_h, 2, 4)
-GEN_VEXT_VX(vwmaccu_vx_w, 4, 8)
-GEN_VEXT_VX(vwmacc_vx_b, 1, 2)
-GEN_VEXT_VX(vwmacc_vx_h, 2, 4)
-GEN_VEXT_VX(vwmacc_vx_w, 4, 8)
-GEN_VEXT_VX(vwmaccsu_vx_b, 1, 2)
-GEN_VEXT_VX(vwmaccsu_vx_h, 2, 4)
-GEN_VEXT_VX(vwmaccsu_vx_w, 4, 8)
-GEN_VEXT_VX(vwmaccus_vx_b, 1, 2)
-GEN_VEXT_VX(vwmaccus_vx_h, 2, 4)
-GEN_VEXT_VX(vwmaccus_vx_w, 4, 8)
+GEN_VEXT_VX(vwmaccu_vx_b)
+GEN_VEXT_VX(vwmaccu_vx_h)
+GEN_VEXT_VX(vwmaccu_vx_w)
+GEN_VEXT_VX(vwmacc_vx_b)
+GEN_VEXT_VX(vwmacc_vx_h)
+GEN_VEXT_VX(vwmacc_vx_w)
+GEN_VEXT_VX(vwmaccsu_vx_b)
+GEN_VEXT_VX(vwmaccsu_vx_h)
+GEN_VEXT_VX(vwmaccsu_vx_w)
+GEN_VEXT_VX(vwmaccus_vx_b)
+GEN_VEXT_VX(vwmaccus_vx_h)
+GEN_VEXT_VX(vwmaccus_vx_w)
 
 /* Vector Integer Merge and Move Instructions */
 #define GEN_VEXT_VMV_VV(NAME, ETYPE, H)                              \
@@ -1922,7 +1920,7 @@ vext_vv_rm_1(void *vd, void *v0, void *vs1, void *vs2,
 static inline void
 vext_vv_rm_2(void *vd, void *v0, void *vs1, void *vs2,
              CPURISCVState *env,
-             uint32_t desc, uint32_t esz, uint32_t dsz,
+             uint32_t desc,
              opivv2_rm_fn *fn)
 {
     uint32_t vm = vext_vm(desc);
@@ -1949,11 +1947,11 @@ vext_vv_rm_2(void *vd, void *v0, void *vs1, void *vs2,
 }
 
 /* generate helpers for fixed point instructions with OPIVV format */
-#define GEN_VEXT_VV_RM(NAME, ESZ, DSZ)                          \
+#define GEN_VEXT_VV_RM(NAME)                                    \
 void HELPER(NAME)(void *vd, void *v0, void *vs1, void *vs2,     \
                   CPURISCVState *env, uint32_t desc)            \
 {                                                               \
-    vext_vv_rm_2(vd, v0, vs1, vs2, env, desc, ESZ, DSZ,         \
+    vext_vv_rm_2(vd, v0, vs1, vs2, env, desc,                   \
                  do_##NAME);                                    \
 }
 
@@ -2004,10 +2002,10 @@ RVVCALL(OPIVV2_RM, vsaddu_vv_b, OP_UUU_B, H1, H1, H1, saddu8)
 RVVCALL(OPIVV2_RM, vsaddu_vv_h, OP_UUU_H, H2, H2, H2, saddu16)
 RVVCALL(OPIVV2_RM, vsaddu_vv_w, OP_UUU_W, H4, H4, H4, saddu32)
 RVVCALL(OPIVV2_RM, vsaddu_vv_d, OP_UUU_D, H8, H8, H8, saddu64)
-GEN_VEXT_VV_RM(vsaddu_vv_b, 1, 1)
-GEN_VEXT_VV_RM(vsaddu_vv_h, 2, 2)
-GEN_VEXT_VV_RM(vsaddu_vv_w, 4, 4)
-GEN_VEXT_VV_RM(vsaddu_vv_d, 8, 8)
+GEN_VEXT_VV_RM(vsaddu_vv_b)
+GEN_VEXT_VV_RM(vsaddu_vv_h)
+GEN_VEXT_VV_RM(vsaddu_vv_w)
+GEN_VEXT_VV_RM(vsaddu_vv_d)
 
 typedef void opivx2_rm_fn(void *vd, target_long s1, void *vs2, int i,
                           CPURISCVState *env, int vxrm);
@@ -2039,7 +2037,7 @@ vext_vx_rm_1(void *vd, void *v0, target_long s1, void *vs2,
 static inline void
 vext_vx_rm_2(void *vd, void *v0, target_long s1, void *vs2,
              CPURISCVState *env,
-             uint32_t desc, uint32_t esz, uint32_t dsz,
+             uint32_t desc,
              opivx2_rm_fn *fn)
 {
     uint32_t vm = vext_vm(desc);
@@ -2066,11 +2064,11 @@ vext_vx_rm_2(void *vd, void *v0, target_long s1, void *vs2,
 }
 
 /* generate helpers for fixed point instructions with OPIVX format */
-#define GEN_VEXT_VX_RM(NAME, ESZ, DSZ)                    \
+#define GEN_VEXT_VX_RM(NAME)                              \
 void HELPER(NAME)(void *vd, void *v0, target_ulong s1,    \
         void *vs2, CPURISCVState *env, uint32_t desc)     \
 {                                                         \
-    vext_vx_rm_2(vd, v0, s1, vs2, env, desc, ESZ, DSZ,    \
+    vext_vx_rm_2(vd, v0, s1, vs2, env, desc,              \
                  do_##NAME);                              \
 }
 
@@ -2078,10 +2076,10 @@ RVVCALL(OPIVX2_RM, vsaddu_vx_b, OP_UUU_B, H1, H1, saddu8)
 RVVCALL(OPIVX2_RM, vsaddu_vx_h, OP_UUU_H, H2, H2, saddu16)
 RVVCALL(OPIVX2_RM, vsaddu_vx_w, OP_UUU_W, H4, H4, saddu32)
 RVVCALL(OPIVX2_RM, vsaddu_vx_d, OP_UUU_D, H8, H8, saddu64)
-GEN_VEXT_VX_RM(vsaddu_vx_b, 1, 1)
-GEN_VEXT_VX_RM(vsaddu_vx_h, 2, 2)
-GEN_VEXT_VX_RM(vsaddu_vx_w, 4, 4)
-GEN_VEXT_VX_RM(vsaddu_vx_d, 8, 8)
+GEN_VEXT_VX_RM(vsaddu_vx_b)
+GEN_VEXT_VX_RM(vsaddu_vx_h)
+GEN_VEXT_VX_RM(vsaddu_vx_w)
+GEN_VEXT_VX_RM(vsaddu_vx_d)
 
 static inline int8_t sadd8(CPURISCVState *env, int vxrm, int8_t a, int8_t b)
 {
@@ -2127,19 +2125,19 @@ RVVCALL(OPIVV2_RM, vsadd_vv_b, OP_SSS_B, H1, H1, H1, sadd8)
 RVVCALL(OPIVV2_RM, vsadd_vv_h, OP_SSS_H, H2, H2, H2, sadd16)
 RVVCALL(OPIVV2_RM, vsadd_vv_w, OP_SSS_W, H4, H4, H4, sadd32)
 RVVCALL(OPIVV2_RM, vsadd_vv_d, OP_SSS_D, H8, H8, H8, sadd64)
-GEN_VEXT_VV_RM(vsadd_vv_b, 1, 1)
-GEN_VEXT_VV_RM(vsadd_vv_h, 2, 2)
-GEN_VEXT_VV_RM(vsadd_vv_w, 4, 4)
-GEN_VEXT_VV_RM(vsadd_vv_d, 8, 8)
+GEN_VEXT_VV_RM(vsadd_vv_b)
+GEN_VEXT_VV_RM(vsadd_vv_h)
+GEN_VEXT_VV_RM(vsadd_vv_w)
+GEN_VEXT_VV_RM(vsadd_vv_d)
 
 RVVCALL(OPIVX2_RM, vsadd_vx_b, OP_SSS_B, H1, H1, sadd8)
 RVVCALL(OPIVX2_RM, vsadd_vx_h, OP_SSS_H, H2, H2, sadd16)
 RVVCALL(OPIVX2_RM, vsadd_vx_w, OP_SSS_W, H4, H4, sadd32)
 RVVCALL(OPIVX2_RM, vsadd_vx_d, OP_SSS_D, H8, H8, sadd64)
-GEN_VEXT_VX_RM(vsadd_vx_b, 1, 1)
-GEN_VEXT_VX_RM(vsadd_vx_h, 2, 2)
-GEN_VEXT_VX_RM(vsadd_vx_w, 4, 4)
-GEN_VEXT_VX_RM(vsadd_vx_d, 8, 8)
+GEN_VEXT_VX_RM(vsadd_vx_b)
+GEN_VEXT_VX_RM(vsadd_vx_h)
+GEN_VEXT_VX_RM(vsadd_vx_w)
+GEN_VEXT_VX_RM(vsadd_vx_d)
 
 static inline uint8_t ssubu8(CPURISCVState *env, int vxrm, uint8_t a, uint8_t b)
 {
@@ -2188,19 +2186,19 @@ RVVCALL(OPIVV2_RM, vssubu_vv_b, OP_UUU_B, H1, H1, H1, ssubu8)
 RVVCALL(OPIVV2_RM, vssubu_vv_h, OP_UUU_H, H2, H2, H2, ssubu16)
 RVVCALL(OPIVV2_RM, vssubu_vv_w, OP_UUU_W, H4, H4, H4, ssubu32)
 RVVCALL(OPIVV2_RM, vssubu_vv_d, OP_UUU_D, H8, H8, H8, ssubu64)
-GEN_VEXT_VV_RM(vssubu_vv_b, 1, 1)
-GEN_VEXT_VV_RM(vssubu_vv_h, 2, 2)
-GEN_VEXT_VV_RM(vssubu_vv_w, 4, 4)
-GEN_VEXT_VV_RM(vssubu_vv_d, 8, 8)
+GEN_VEXT_VV_RM(vssubu_vv_b)
+GEN_VEXT_VV_RM(vssubu_vv_h)
+GEN_VEXT_VV_RM(vssubu_vv_w)
+GEN_VEXT_VV_RM(vssubu_vv_d)
 
 RVVCALL(OPIVX2_RM, vssubu_vx_b, OP_UUU_B, H1, H1, ssubu8)
 RVVCALL(OPIVX2_RM, vssubu_vx_h, OP_UUU_H, H2, H2, ssubu16)
 RVVCALL(OPIVX2_RM, vssubu_vx_w, OP_UUU_W, H4, H4, ssubu32)
 RVVCALL(OPIVX2_RM, vssubu_vx_d, OP_UUU_D, H8, H8, ssubu64)
-GEN_VEXT_VX_RM(vssubu_vx_b, 1, 1)
-GEN_VEXT_VX_RM(vssubu_vx_h, 2, 2)
-GEN_VEXT_VX_RM(vssubu_vx_w, 4, 4)
-GEN_VEXT_VX_RM(vssubu_vx_d, 8, 8)
+GEN_VEXT_VX_RM(vssubu_vx_b)
+GEN_VEXT_VX_RM(vssubu_vx_h)
+GEN_VEXT_VX_RM(vssubu_vx_w)
+GEN_VEXT_VX_RM(vssubu_vx_d)
 
 static inline int8_t ssub8(CPURISCVState *env, int vxrm, int8_t a, int8_t b)
 {
@@ -2246,19 +2244,19 @@ RVVCALL(OPIVV2_RM, vssub_vv_b, OP_SSS_B, H1, H1, H1, ssub8)
 RVVCALL(OPIVV2_RM, vssub_vv_h, OP_SSS_H, H2, H2, H2, ssub16)
 RVVCALL(OPIVV2_RM, vssub_vv_w, OP_SSS_W, H4, H4, H4, ssub32)
 RVVCALL(OPIVV2_RM, vssub_vv_d, OP_SSS_D, H8, H8, H8, ssub64)
-GEN_VEXT_VV_RM(vssub_vv_b, 1, 1)
-GEN_VEXT_VV_RM(vssub_vv_h, 2, 2)
-GEN_VEXT_VV_RM(vssub_vv_w, 4, 4)
-GEN_VEXT_VV_RM(vssub_vv_d, 8, 8)
+GEN_VEXT_VV_RM(vssub_vv_b)
+GEN_VEXT_VV_RM(vssub_vv_h)
+GEN_VEXT_VV_RM(vssub_vv_w)
+GEN_VEXT_VV_RM(vssub_vv_d)
 
 RVVCALL(OPIVX2_RM, vssub_vx_b, OP_SSS_B, H1, H1, ssub8)
 RVVCALL(OPIVX2_RM, vssub_vx_h, OP_SSS_H, H2, H2, ssub16)
 RVVCALL(OPIVX2_RM, vssub_vx_w, OP_SSS_W, H4, H4, ssub32)
 RVVCALL(OPIVX2_RM, vssub_vx_d, OP_SSS_D, H8, H8, ssub64)
-GEN_VEXT_VX_RM(vssub_vx_b, 1, 1)
-GEN_VEXT_VX_RM(vssub_vx_h, 2, 2)
-GEN_VEXT_VX_RM(vssub_vx_w, 4, 4)
-GEN_VEXT_VX_RM(vssub_vx_d, 8, 8)
+GEN_VEXT_VX_RM(vssub_vx_b)
+GEN_VEXT_VX_RM(vssub_vx_h)
+GEN_VEXT_VX_RM(vssub_vx_w)
+GEN_VEXT_VX_RM(vssub_vx_d)
 
 /* Vector Single-Width Averaging Add and Subtract */
 static inline uint8_t get_round(int vxrm, uint64_t v, uint8_t shift)
@@ -2310,19 +2308,19 @@ RVVCALL(OPIVV2_RM, vaadd_vv_b, OP_SSS_B, H1, H1, H1, aadd32)
 RVVCALL(OPIVV2_RM, vaadd_vv_h, OP_SSS_H, H2, H2, H2, aadd32)
 RVVCALL(OPIVV2_RM, vaadd_vv_w, OP_SSS_W, H4, H4, H4, aadd32)
 RVVCALL(OPIVV2_RM, vaadd_vv_d, OP_SSS_D, H8, H8, H8, aadd64)
-GEN_VEXT_VV_RM(vaadd_vv_b, 1, 1)
-GEN_VEXT_VV_RM(vaadd_vv_h, 2, 2)
-GEN_VEXT_VV_RM(vaadd_vv_w, 4, 4)
-GEN_VEXT_VV_RM(vaadd_vv_d, 8, 8)
+GEN_VEXT_VV_RM(vaadd_vv_b)
+GEN_VEXT_VV_RM(vaadd_vv_h)
+GEN_VEXT_VV_RM(vaadd_vv_w)
+GEN_VEXT_VV_RM(vaadd_vv_d)
 
 RVVCALL(OPIVX2_RM, vaadd_vx_b, OP_SSS_B, H1, H1, aadd32)
 RVVCALL(OPIVX2_RM, vaadd_vx_h, OP_SSS_H, H2, H2, aadd32)
 RVVCALL(OPIVX2_RM, vaadd_vx_w, OP_SSS_W, H4, H4, aadd32)
 RVVCALL(OPIVX2_RM, vaadd_vx_d, OP_SSS_D, H8, H8, aadd64)
-GEN_VEXT_VX_RM(vaadd_vx_b, 1, 1)
-GEN_VEXT_VX_RM(vaadd_vx_h, 2, 2)
-GEN_VEXT_VX_RM(vaadd_vx_w, 4, 4)
-GEN_VEXT_VX_RM(vaadd_vx_d, 8, 8)
+GEN_VEXT_VX_RM(vaadd_vx_b)
+GEN_VEXT_VX_RM(vaadd_vx_h)
+GEN_VEXT_VX_RM(vaadd_vx_w)
+GEN_VEXT_VX_RM(vaadd_vx_d)
 
 static inline uint32_t aaddu32(CPURISCVState *env, int vxrm,
                                uint32_t a, uint32_t b)
@@ -2347,19 +2345,19 @@ RVVCALL(OPIVV2_RM, vaaddu_vv_b, OP_UUU_B, H1, H1, H1, aaddu32)
 RVVCALL(OPIVV2_RM, vaaddu_vv_h, OP_UUU_H, H2, H2, H2, aaddu32)
 RVVCALL(OPIVV2_RM, vaaddu_vv_w, OP_UUU_W, H4, H4, H4, aaddu32)
 RVVCALL(OPIVV2_RM, vaaddu_vv_d, OP_UUU_D, H8, H8, H8, aaddu64)
-GEN_VEXT_VV_RM(vaaddu_vv_b, 1, 1)
-GEN_VEXT_VV_RM(vaaddu_vv_h, 2, 2)
-GEN_VEXT_VV_RM(vaaddu_vv_w, 4, 4)
-GEN_VEXT_VV_RM(vaaddu_vv_d, 8, 8)
+GEN_VEXT_VV_RM(vaaddu_vv_b)
+GEN_VEXT_VV_RM(vaaddu_vv_h)
+GEN_VEXT_VV_RM(vaaddu_vv_w)
+GEN_VEXT_VV_RM(vaaddu_vv_d)
 
 RVVCALL(OPIVX2_RM, vaaddu_vx_b, OP_UUU_B, H1, H1, aaddu32)
 RVVCALL(OPIVX2_RM, vaaddu_vx_h, OP_UUU_H, H2, H2, aaddu32)
 RVVCALL(OPIVX2_RM, vaaddu_vx_w, OP_UUU_W, H4, H4, aaddu32)
 RVVCALL(OPIVX2_RM, vaaddu_vx_d, OP_UUU_D, H8, H8, aaddu64)
-GEN_VEXT_VX_RM(vaaddu_vx_b, 1, 1)
-GEN_VEXT_VX_RM(vaaddu_vx_h, 2, 2)
-GEN_VEXT_VX_RM(vaaddu_vx_w, 4, 4)
-GEN_VEXT_VX_RM(vaaddu_vx_d, 8, 8)
+GEN_VEXT_VX_RM(vaaddu_vx_b)
+GEN_VEXT_VX_RM(vaaddu_vx_h)
+GEN_VEXT_VX_RM(vaaddu_vx_w)
+GEN_VEXT_VX_RM(vaaddu_vx_d)
 
 static inline int32_t asub32(CPURISCVState *env, int vxrm, int32_t a, int32_t b)
 {
@@ -2383,19 +2381,19 @@ RVVCALL(OPIVV2_RM, vasub_vv_b, OP_SSS_B, H1, H1, H1, asub32)
 RVVCALL(OPIVV2_RM, vasub_vv_h, OP_SSS_H, H2, H2, H2, asub32)
 RVVCALL(OPIVV2_RM, vasub_vv_w, OP_SSS_W, H4, H4, H4, asub32)
 RVVCALL(OPIVV2_RM, vasub_vv_d, OP_SSS_D, H8, H8, H8, asub64)
-GEN_VEXT_VV_RM(vasub_vv_b, 1, 1)
-GEN_VEXT_VV_RM(vasub_vv_h, 2, 2)
-GEN_VEXT_VV_RM(vasub_vv_w, 4, 4)
-GEN_VEXT_VV_RM(vasub_vv_d, 8, 8)
+GEN_VEXT_VV_RM(vasub_vv_b)
+GEN_VEXT_VV_RM(vasub_vv_h)
+GEN_VEXT_VV_RM(vasub_vv_w)
+GEN_VEXT_VV_RM(vasub_vv_d)
 
 RVVCALL(OPIVX2_RM, vasub_vx_b, OP_SSS_B, H1, H1, asub32)
 RVVCALL(OPIVX2_RM, vasub_vx_h, OP_SSS_H, H2, H2, asub32)
 RVVCALL(OPIVX2_RM, vasub_vx_w, OP_SSS_W, H4, H4, asub32)
 RVVCALL(OPIVX2_RM, vasub_vx_d, OP_SSS_D, H8, H8, asub64)
-GEN_VEXT_VX_RM(vasub_vx_b, 1, 1)
-GEN_VEXT_VX_RM(vasub_vx_h, 2, 2)
-GEN_VEXT_VX_RM(vasub_vx_w, 4, 4)
-GEN_VEXT_VX_RM(vasub_vx_d, 8, 8)
+GEN_VEXT_VX_RM(vasub_vx_b)
+GEN_VEXT_VX_RM(vasub_vx_h)
+GEN_VEXT_VX_RM(vasub_vx_w)
+GEN_VEXT_VX_RM(vasub_vx_d)
 
 static inline uint32_t asubu32(CPURISCVState *env, int vxrm,
                                uint32_t a, uint32_t b)
@@ -2420,19 +2418,19 @@ RVVCALL(OPIVV2_RM, vasubu_vv_b, OP_UUU_B, H1, H1, H1, asubu32)
 RVVCALL(OPIVV2_RM, vasubu_vv_h, OP_UUU_H, H2, H2, H2, asubu32)
 RVVCALL(OPIVV2_RM, vasubu_vv_w, OP_UUU_W, H4, H4, H4, asubu32)
 RVVCALL(OPIVV2_RM, vasubu_vv_d, OP_UUU_D, H8, H8, H8, asubu64)
-GEN_VEXT_VV_RM(vasubu_vv_b, 1, 1)
-GEN_VEXT_VV_RM(vasubu_vv_h, 2, 2)
-GEN_VEXT_VV_RM(vasubu_vv_w, 4, 4)
-GEN_VEXT_VV_RM(vasubu_vv_d, 8, 8)
+GEN_VEXT_VV_RM(vasubu_vv_b)
+GEN_VEXT_VV_RM(vasubu_vv_h)
+GEN_VEXT_VV_RM(vasubu_vv_w)
+GEN_VEXT_VV_RM(vasubu_vv_d)
 
 RVVCALL(OPIVX2_RM, vasubu_vx_b, OP_UUU_B, H1, H1, asubu32)
 RVVCALL(OPIVX2_RM, vasubu_vx_h, OP_UUU_H, H2, H2, asubu32)
 RVVCALL(OPIVX2_RM, vasubu_vx_w, OP_UUU_W, H4, H4, asubu32)
 RVVCALL(OPIVX2_RM, vasubu_vx_d, OP_UUU_D, H8, H8, asubu64)
-GEN_VEXT_VX_RM(vasubu_vx_b, 1, 1)
-GEN_VEXT_VX_RM(vasubu_vx_h, 2, 2)
-GEN_VEXT_VX_RM(vasubu_vx_w, 4, 4)
-GEN_VEXT_VX_RM(vasubu_vx_d, 8, 8)
+GEN_VEXT_VX_RM(vasubu_vx_b)
+GEN_VEXT_VX_RM(vasubu_vx_h)
+GEN_VEXT_VX_RM(vasubu_vx_w)
+GEN_VEXT_VX_RM(vasubu_vx_d)
 
 /* Vector Single-Width Fractional Multiply with Rounding and Saturation */
 static inline int8_t vsmul8(CPURISCVState *env, int vxrm, int8_t a, int8_t b)
@@ -2527,19 +2525,19 @@ RVVCALL(OPIVV2_RM, vsmul_vv_b, OP_SSS_B, H1, H1, H1, vsmul8)
 RVVCALL(OPIVV2_RM, vsmul_vv_h, OP_SSS_H, H2, H2, H2, vsmul16)
 RVVCALL(OPIVV2_RM, vsmul_vv_w, OP_SSS_W, H4, H4, H4, vsmul32)
 RVVCALL(OPIVV2_RM, vsmul_vv_d, OP_SSS_D, H8, H8, H8, vsmul64)
-GEN_VEXT_VV_RM(vsmul_vv_b, 1, 1)
-GEN_VEXT_VV_RM(vsmul_vv_h, 2, 2)
-GEN_VEXT_VV_RM(vsmul_vv_w, 4, 4)
-GEN_VEXT_VV_RM(vsmul_vv_d, 8, 8)
+GEN_VEXT_VV_RM(vsmul_vv_b)
+GEN_VEXT_VV_RM(vsmul_vv_h)
+GEN_VEXT_VV_RM(vsmul_vv_w)
+GEN_VEXT_VV_RM(vsmul_vv_d)
 
 RVVCALL(OPIVX2_RM, vsmul_vx_b, OP_SSS_B, H1, H1, vsmul8)
 RVVCALL(OPIVX2_RM, vsmul_vx_h, OP_SSS_H, H2, H2, vsmul16)
 RVVCALL(OPIVX2_RM, vsmul_vx_w, OP_SSS_W, H4, H4, vsmul32)
 RVVCALL(OPIVX2_RM, vsmul_vx_d, OP_SSS_D, H8, H8, vsmul64)
-GEN_VEXT_VX_RM(vsmul_vx_b, 1, 1)
-GEN_VEXT_VX_RM(vsmul_vx_h, 2, 2)
-GEN_VEXT_VX_RM(vsmul_vx_w, 4, 4)
-GEN_VEXT_VX_RM(vsmul_vx_d, 8, 8)
+GEN_VEXT_VX_RM(vsmul_vx_b)
+GEN_VEXT_VX_RM(vsmul_vx_h)
+GEN_VEXT_VX_RM(vsmul_vx_w)
+GEN_VEXT_VX_RM(vsmul_vx_d)
 
 /* Vector Single-Width Scaling Shift Instructions */
 static inline uint8_t
@@ -2586,19 +2584,19 @@ RVVCALL(OPIVV2_RM, vssrl_vv_b, OP_UUU_B, H1, H1, H1, vssrl8)
 RVVCALL(OPIVV2_RM, vssrl_vv_h, OP_UUU_H, H2, H2, H2, vssrl16)
 RVVCALL(OPIVV2_RM, vssrl_vv_w, OP_UUU_W, H4, H4, H4, vssrl32)
 RVVCALL(OPIVV2_RM, vssrl_vv_d, OP_UUU_D, H8, H8, H8, vssrl64)
-GEN_VEXT_VV_RM(vssrl_vv_b, 1, 1)
-GEN_VEXT_VV_RM(vssrl_vv_h, 2, 2)
-GEN_VEXT_VV_RM(vssrl_vv_w, 4, 4)
-GEN_VEXT_VV_RM(vssrl_vv_d, 8, 8)
+GEN_VEXT_VV_RM(vssrl_vv_b)
+GEN_VEXT_VV_RM(vssrl_vv_h)
+GEN_VEXT_VV_RM(vssrl_vv_w)
+GEN_VEXT_VV_RM(vssrl_vv_d)
 
 RVVCALL(OPIVX2_RM, vssrl_vx_b, OP_UUU_B, H1, H1, vssrl8)
 RVVCALL(OPIVX2_RM, vssrl_vx_h, OP_UUU_H, H2, H2, vssrl16)
 RVVCALL(OPIVX2_RM, vssrl_vx_w, OP_UUU_W, H4, H4, vssrl32)
 RVVCALL(OPIVX2_RM, vssrl_vx_d, OP_UUU_D, H8, H8, vssrl64)
-GEN_VEXT_VX_RM(vssrl_vx_b, 1, 1)
-GEN_VEXT_VX_RM(vssrl_vx_h, 2, 2)
-GEN_VEXT_VX_RM(vssrl_vx_w, 4, 4)
-GEN_VEXT_VX_RM(vssrl_vx_d, 8, 8)
+GEN_VEXT_VX_RM(vssrl_vx_b)
+GEN_VEXT_VX_RM(vssrl_vx_h)
+GEN_VEXT_VX_RM(vssrl_vx_w)
+GEN_VEXT_VX_RM(vssrl_vx_d)
 
 static inline int8_t
 vssra8(CPURISCVState *env, int vxrm, int8_t a, int8_t b)
@@ -2645,19 +2643,19 @@ RVVCALL(OPIVV2_RM, vssra_vv_b, OP_SSS_B, H1, H1, H1, vssra8)
 RVVCALL(OPIVV2_RM, vssra_vv_h, OP_SSS_H, H2, H2, H2, vssra16)
 RVVCALL(OPIVV2_RM, vssra_vv_w, OP_SSS_W, H4, H4, H4, vssra32)
 RVVCALL(OPIVV2_RM, vssra_vv_d, OP_SSS_D, H8, H8, H8, vssra64)
-GEN_VEXT_VV_RM(vssra_vv_b, 1, 1)
-GEN_VEXT_VV_RM(vssra_vv_h, 2, 2)
-GEN_VEXT_VV_RM(vssra_vv_w, 4, 4)
-GEN_VEXT_VV_RM(vssra_vv_d, 8, 8)
+GEN_VEXT_VV_RM(vssra_vv_b)
+GEN_VEXT_VV_RM(vssra_vv_h)
+GEN_VEXT_VV_RM(vssra_vv_w)
+GEN_VEXT_VV_RM(vssra_vv_d)
 
 RVVCALL(OPIVX2_RM, vssra_vx_b, OP_SSS_B, H1, H1, vssra8)
 RVVCALL(OPIVX2_RM, vssra_vx_h, OP_SSS_H, H2, H2, vssra16)
 RVVCALL(OPIVX2_RM, vssra_vx_w, OP_SSS_W, H4, H4, vssra32)
 RVVCALL(OPIVX2_RM, vssra_vx_d, OP_SSS_D, H8, H8, vssra64)
-GEN_VEXT_VX_RM(vssra_vx_b, 1, 1)
-GEN_VEXT_VX_RM(vssra_vx_h, 2, 2)
-GEN_VEXT_VX_RM(vssra_vx_w, 4, 4)
-GEN_VEXT_VX_RM(vssra_vx_d, 8, 8)
+GEN_VEXT_VX_RM(vssra_vx_b)
+GEN_VEXT_VX_RM(vssra_vx_h)
+GEN_VEXT_VX_RM(vssra_vx_w)
+GEN_VEXT_VX_RM(vssra_vx_d)
 
 /* Vector Narrowing Fixed-Point Clip Instructions */
 static inline int8_t
@@ -2720,16 +2718,16 @@ vnclip32(CPURISCVState *env, int vxrm, int64_t a, int32_t b)
 RVVCALL(OPIVV2_RM, vnclip_wv_b, NOP_SSS_B, H1, H2, H1, vnclip8)
 RVVCALL(OPIVV2_RM, vnclip_wv_h, NOP_SSS_H, H2, H4, H2, vnclip16)
 RVVCALL(OPIVV2_RM, vnclip_wv_w, NOP_SSS_W, H4, H8, H4, vnclip32)
-GEN_VEXT_VV_RM(vnclip_wv_b, 1, 1)
-GEN_VEXT_VV_RM(vnclip_wv_h, 2, 2)
-GEN_VEXT_VV_RM(vnclip_wv_w, 4, 4)
+GEN_VEXT_VV_RM(vnclip_wv_b)
+GEN_VEXT_VV_RM(vnclip_wv_h)
+GEN_VEXT_VV_RM(vnclip_wv_w)
 
 RVVCALL(OPIVX2_RM, vnclip_wx_b, NOP_SSS_B, H1, H2, vnclip8)
 RVVCALL(OPIVX2_RM, vnclip_wx_h, NOP_SSS_H, H2, H4, vnclip16)
 RVVCALL(OPIVX2_RM, vnclip_wx_w, NOP_SSS_W, H4, H8, vnclip32)
-GEN_VEXT_VX_RM(vnclip_wx_b, 1, 1)
-GEN_VEXT_VX_RM(vnclip_wx_h, 2, 2)
-GEN_VEXT_VX_RM(vnclip_wx_w, 4, 4)
+GEN_VEXT_VX_RM(vnclip_wx_b)
+GEN_VEXT_VX_RM(vnclip_wx_h)
+GEN_VEXT_VX_RM(vnclip_wx_w)
 
 static inline uint8_t
 vnclipu8(CPURISCVState *env, int vxrm, uint16_t a, uint8_t b)
@@ -2782,16 +2780,16 @@ vnclipu32(CPURISCVState *env, int vxrm, uint64_t a, uint32_t b)
 RVVCALL(OPIVV2_RM, vnclipu_wv_b, NOP_UUU_B, H1, H2, H1, vnclipu8)
 RVVCALL(OPIVV2_RM, vnclipu_wv_h, NOP_UUU_H, H2, H4, H2, vnclipu16)
 RVVCALL(OPIVV2_RM, vnclipu_wv_w, NOP_UUU_W, H4, H8, H4, vnclipu32)
-GEN_VEXT_VV_RM(vnclipu_wv_b, 1, 1)
-GEN_VEXT_VV_RM(vnclipu_wv_h, 2, 2)
-GEN_VEXT_VV_RM(vnclipu_wv_w, 4, 4)
+GEN_VEXT_VV_RM(vnclipu_wv_b)
+GEN_VEXT_VV_RM(vnclipu_wv_h)
+GEN_VEXT_VV_RM(vnclipu_wv_w)
 
 RVVCALL(OPIVX2_RM, vnclipu_wx_b, NOP_UUU_B, H1, H2, vnclipu8)
 RVVCALL(OPIVX2_RM, vnclipu_wx_h, NOP_UUU_H, H2, H4, vnclipu16)
 RVVCALL(OPIVX2_RM, vnclipu_wx_w, NOP_UUU_W, H4, H8, vnclipu32)
-GEN_VEXT_VX_RM(vnclipu_wx_b, 1, 1)
-GEN_VEXT_VX_RM(vnclipu_wx_h, 2, 2)
-GEN_VEXT_VX_RM(vnclipu_wx_w, 4, 4)
+GEN_VEXT_VX_RM(vnclipu_wx_b)
+GEN_VEXT_VX_RM(vnclipu_wx_h)
+GEN_VEXT_VX_RM(vnclipu_wx_w)
 
 /*
  *** Vector Float Point Arithmetic Instructions
@@ -2806,7 +2804,7 @@ static void do_##NAME(void *vd, void *vs1, void *vs2, int i,   \
     *((TD *)vd + HD(i)) = OP(s2, s1, &env->fp_status);         \
 }
 
-#define GEN_VEXT_VV_ENV(NAME, ESZ, DSZ)                   \
+#define GEN_VEXT_VV_ENV(NAME)                             \
 void HELPER(NAME)(void *vd, void *v0, void *vs1,          \
                   void *vs2, CPURISCVState *env,          \
                   uint32_t desc)                          \
@@ -2827,9 +2825,9 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1,          \
 RVVCALL(OPFVV2, vfadd_vv_h, OP_UUU_H, H2, H2, H2, float16_add)
 RVVCALL(OPFVV2, vfadd_vv_w, OP_UUU_W, H4, H4, H4, float32_add)
 RVVCALL(OPFVV2, vfadd_vv_d, OP_UUU_D, H8, H8, H8, float64_add)
-GEN_VEXT_VV_ENV(vfadd_vv_h, 2, 2)
-GEN_VEXT_VV_ENV(vfadd_vv_w, 4, 4)
-GEN_VEXT_VV_ENV(vfadd_vv_d, 8, 8)
+GEN_VEXT_VV_ENV(vfadd_vv_h)
+GEN_VEXT_VV_ENV(vfadd_vv_w)
+GEN_VEXT_VV_ENV(vfadd_vv_d)
 
 #define OPFVF2(NAME, TD, T1, T2, TX1, TX2, HD, HS2, OP)        \
 static void do_##NAME(void *vd, uint64_t s1, void *vs2, int i, \
@@ -2839,7 +2837,7 @@ static void do_##NAME(void *vd, uint64_t s1, void *vs2, int i, \
     *((TD *)vd + HD(i)) = OP(s2, (TX1)(T1)s1, &env->fp_status);\
 }
 
-#define GEN_VEXT_VF(NAME, ESZ, DSZ)                       \
+#define GEN_VEXT_VF(NAME)                                 \
 void HELPER(NAME)(void *vd, void *v0, uint64_t s1,        \
                   void *vs2, CPURISCVState *env,          \
                   uint32_t desc)                          \
@@ -2860,22 +2858,22 @@ void HELPER(NAME)(void *vd, void *v0, uint64_t s1,        \
 RVVCALL(OPFVF2, vfadd_vf_h, OP_UUU_H, H2, H2, float16_add)
 RVVCALL(OPFVF2, vfadd_vf_w, OP_UUU_W, H4, H4, float32_add)
 RVVCALL(OPFVF2, vfadd_vf_d, OP_UUU_D, H8, H8, float64_add)
-GEN_VEXT_VF(vfadd_vf_h, 2, 2)
-GEN_VEXT_VF(vfadd_vf_w, 4, 4)
-GEN_VEXT_VF(vfadd_vf_d, 8, 8)
+GEN_VEXT_VF(vfadd_vf_h)
+GEN_VEXT_VF(vfadd_vf_w)
+GEN_VEXT_VF(vfadd_vf_d)
 
 RVVCALL(OPFVV2, vfsub_vv_h, OP_UUU_H, H2, H2, H2, float16_sub)
 RVVCALL(OPFVV2, vfsub_vv_w, OP_UUU_W, H4, H4, H4, float32_sub)
 RVVCALL(OPFVV2, vfsub_vv_d, OP_UUU_D, H8, H8, H8, float64_sub)
-GEN_VEXT_VV_ENV(vfsub_vv_h, 2, 2)
-GEN_VEXT_VV_ENV(vfsub_vv_w, 4, 4)
-GEN_VEXT_VV_ENV(vfsub_vv_d, 8, 8)
+GEN_VEXT_VV_ENV(vfsub_vv_h)
+GEN_VEXT_VV_ENV(vfsub_vv_w)
+GEN_VEXT_VV_ENV(vfsub_vv_d)
 RVVCALL(OPFVF2, vfsub_vf_h, OP_UUU_H, H2, H2, float16_sub)
 RVVCALL(OPFVF2, vfsub_vf_w, OP_UUU_W, H4, H4, float32_sub)
 RVVCALL(OPFVF2, vfsub_vf_d, OP_UUU_D, H8, H8, float64_sub)
-GEN_VEXT_VF(vfsub_vf_h, 2, 2)
-GEN_VEXT_VF(vfsub_vf_w, 4, 4)
-GEN_VEXT_VF(vfsub_vf_d, 8, 8)
+GEN_VEXT_VF(vfsub_vf_h)
+GEN_VEXT_VF(vfsub_vf_w)
+GEN_VEXT_VF(vfsub_vf_d)
 
 static uint16_t float16_rsub(uint16_t a, uint16_t b, float_status *s)
 {
@@ -2895,9 +2893,9 @@ static uint64_t float64_rsub(uint64_t a, uint64_t b, float_status *s)
 RVVCALL(OPFVF2, vfrsub_vf_h, OP_UUU_H, H2, H2, float16_rsub)
 RVVCALL(OPFVF2, vfrsub_vf_w, OP_UUU_W, H4, H4, float32_rsub)
 RVVCALL(OPFVF2, vfrsub_vf_d, OP_UUU_D, H8, H8, float64_rsub)
-GEN_VEXT_VF(vfrsub_vf_h, 2, 2)
-GEN_VEXT_VF(vfrsub_vf_w, 4, 4)
-GEN_VEXT_VF(vfrsub_vf_d, 8, 8)
+GEN_VEXT_VF(vfrsub_vf_h)
+GEN_VEXT_VF(vfrsub_vf_w)
+GEN_VEXT_VF(vfrsub_vf_d)
 
 /* Vector Widening Floating-Point Add/Subtract Instructions */
 static uint32_t vfwadd16(uint16_t a, uint16_t b, float_status *s)
@@ -2915,12 +2913,12 @@ static uint64_t vfwadd32(uint32_t a, uint32_t b, float_status *s)
 
 RVVCALL(OPFVV2, vfwadd_vv_h, WOP_UUU_H, H4, H2, H2, vfwadd16)
 RVVCALL(OPFVV2, vfwadd_vv_w, WOP_UUU_W, H8, H4, H4, vfwadd32)
-GEN_VEXT_VV_ENV(vfwadd_vv_h, 2, 4)
-GEN_VEXT_VV_ENV(vfwadd_vv_w, 4, 8)
+GEN_VEXT_VV_ENV(vfwadd_vv_h)
+GEN_VEXT_VV_ENV(vfwadd_vv_w)
 RVVCALL(OPFVF2, vfwadd_vf_h, WOP_UUU_H, H4, H2, vfwadd16)
 RVVCALL(OPFVF2, vfwadd_vf_w, WOP_UUU_W, H8, H4, vfwadd32)
-GEN_VEXT_VF(vfwadd_vf_h, 2, 4)
-GEN_VEXT_VF(vfwadd_vf_w, 4, 8)
+GEN_VEXT_VF(vfwadd_vf_h)
+GEN_VEXT_VF(vfwadd_vf_w)
 
 static uint32_t vfwsub16(uint16_t a, uint16_t b, float_status *s)
 {
@@ -2937,12 +2935,12 @@ static uint64_t vfwsub32(uint32_t a, uint32_t b, float_status *s)
 
 RVVCALL(OPFVV2, vfwsub_vv_h, WOP_UUU_H, H4, H2, H2, vfwsub16)
 RVVCALL(OPFVV2, vfwsub_vv_w, WOP_UUU_W, H8, H4, H4, vfwsub32)
-GEN_VEXT_VV_ENV(vfwsub_vv_h, 2, 4)
-GEN_VEXT_VV_ENV(vfwsub_vv_w, 4, 8)
+GEN_VEXT_VV_ENV(vfwsub_vv_h)
+GEN_VEXT_VV_ENV(vfwsub_vv_w)
 RVVCALL(OPFVF2, vfwsub_vf_h, WOP_UUU_H, H4, H2, vfwsub16)
 RVVCALL(OPFVF2, vfwsub_vf_w, WOP_UUU_W, H8, H4, vfwsub32)
-GEN_VEXT_VF(vfwsub_vf_h, 2, 4)
-GEN_VEXT_VF(vfwsub_vf_w, 4, 8)
+GEN_VEXT_VF(vfwsub_vf_h)
+GEN_VEXT_VF(vfwsub_vf_w)
 
 static uint32_t vfwaddw16(uint32_t a, uint16_t b, float_status *s)
 {
@@ -2956,12 +2954,12 @@ static uint64_t vfwaddw32(uint64_t a, uint32_t b, float_status *s)
 
 RVVCALL(OPFVV2, vfwadd_wv_h, WOP_WUUU_H, H4, H2, H2, vfwaddw16)
 RVVCALL(OPFVV2, vfwadd_wv_w, WOP_WUUU_W, H8, H4, H4, vfwaddw32)
-GEN_VEXT_VV_ENV(vfwadd_wv_h, 2, 4)
-GEN_VEXT_VV_ENV(vfwadd_wv_w, 4, 8)
+GEN_VEXT_VV_ENV(vfwadd_wv_h)
+GEN_VEXT_VV_ENV(vfwadd_wv_w)
 RVVCALL(OPFVF2, vfwadd_wf_h, WOP_WUUU_H, H4, H2, vfwaddw16)
 RVVCALL(OPFVF2, vfwadd_wf_w, WOP_WUUU_W, H8, H4, vfwaddw32)
-GEN_VEXT_VF(vfwadd_wf_h, 2, 4)
-GEN_VEXT_VF(vfwadd_wf_w, 4, 8)
+GEN_VEXT_VF(vfwadd_wf_h)
+GEN_VEXT_VF(vfwadd_wf_w)
 
 static uint32_t vfwsubw16(uint32_t a, uint16_t b, float_status *s)
 {
@@ -2975,39 +2973,39 @@ static uint64_t vfwsubw32(uint64_t a, uint32_t b, float_status *s)
 
 RVVCALL(OPFVV2, vfwsub_wv_h, WOP_WUUU_H, H4, H2, H2, vfwsubw16)
 RVVCALL(OPFVV2, vfwsub_wv_w, WOP_WUUU_W, H8, H4, H4, vfwsubw32)
-GEN_VEXT_VV_ENV(vfwsub_wv_h, 2, 4)
-GEN_VEXT_VV_ENV(vfwsub_wv_w, 4, 8)
+GEN_VEXT_VV_ENV(vfwsub_wv_h)
+GEN_VEXT_VV_ENV(vfwsub_wv_w)
 RVVCALL(OPFVF2, vfwsub_wf_h, WOP_WUUU_H, H4, H2, vfwsubw16)
 RVVCALL(OPFVF2, vfwsub_wf_w, WOP_WUUU_W, H8, H4, vfwsubw32)
-GEN_VEXT_VF(vfwsub_wf_h, 2, 4)
-GEN_VEXT_VF(vfwsub_wf_w, 4, 8)
+GEN_VEXT_VF(vfwsub_wf_h)
+GEN_VEXT_VF(vfwsub_wf_w)
 
 /* Vector Single-Width Floating-Point Multiply/Divide Instructions */
 RVVCALL(OPFVV2, vfmul_vv_h, OP_UUU_H, H2, H2, H2, float16_mul)
 RVVCALL(OPFVV2, vfmul_vv_w, OP_UUU_W, H4, H4, H4, float32_mul)
 RVVCALL(OPFVV2, vfmul_vv_d, OP_UUU_D, H8, H8, H8, float64_mul)
-GEN_VEXT_VV_ENV(vfmul_vv_h, 2, 2)
-GEN_VEXT_VV_ENV(vfmul_vv_w, 4, 4)
-GEN_VEXT_VV_ENV(vfmul_vv_d, 8, 8)
+GEN_VEXT_VV_ENV(vfmul_vv_h)
+GEN_VEXT_VV_ENV(vfmul_vv_w)
+GEN_VEXT_VV_ENV(vfmul_vv_d)
 RVVCALL(OPFVF2, vfmul_vf_h, OP_UUU_H, H2, H2, float16_mul)
 RVVCALL(OPFVF2, vfmul_vf_w, OP_UUU_W, H4, H4, float32_mul)
 RVVCALL(OPFVF2, vfmul_vf_d, OP_UUU_D, H8, H8, float64_mul)
-GEN_VEXT_VF(vfmul_vf_h, 2, 2)
-GEN_VEXT_VF(vfmul_vf_w, 4, 4)
-GEN_VEXT_VF(vfmul_vf_d, 8, 8)
+GEN_VEXT_VF(vfmul_vf_h)
+GEN_VEXT_VF(vfmul_vf_w)
+GEN_VEXT_VF(vfmul_vf_d)
 
 RVVCALL(OPFVV2, vfdiv_vv_h, OP_UUU_H, H2, H2, H2, float16_div)
 RVVCALL(OPFVV2, vfdiv_vv_w, OP_UUU_W, H4, H4, H4, float32_div)
 RVVCALL(OPFVV2, vfdiv_vv_d, OP_UUU_D, H8, H8, H8, float64_div)
-GEN_VEXT_VV_ENV(vfdiv_vv_h, 2, 2)
-GEN_VEXT_VV_ENV(vfdiv_vv_w, 4, 4)
-GEN_VEXT_VV_ENV(vfdiv_vv_d, 8, 8)
+GEN_VEXT_VV_ENV(vfdiv_vv_h)
+GEN_VEXT_VV_ENV(vfdiv_vv_w)
+GEN_VEXT_VV_ENV(vfdiv_vv_d)
 RVVCALL(OPFVF2, vfdiv_vf_h, OP_UUU_H, H2, H2, float16_div)
 RVVCALL(OPFVF2, vfdiv_vf_w, OP_UUU_W, H4, H4, float32_div)
 RVVCALL(OPFVF2, vfdiv_vf_d, OP_UUU_D, H8, H8, float64_div)
-GEN_VEXT_VF(vfdiv_vf_h, 2, 2)
-GEN_VEXT_VF(vfdiv_vf_w, 4, 4)
-GEN_VEXT_VF(vfdiv_vf_d, 8, 8)
+GEN_VEXT_VF(vfdiv_vf_h)
+GEN_VEXT_VF(vfdiv_vf_w)
+GEN_VEXT_VF(vfdiv_vf_d)
 
 static uint16_t float16_rdiv(uint16_t a, uint16_t b, float_status *s)
 {
@@ -3027,9 +3025,9 @@ static uint64_t float64_rdiv(uint64_t a, uint64_t b, float_status *s)
 RVVCALL(OPFVF2, vfrdiv_vf_h, OP_UUU_H, H2, H2, float16_rdiv)
 RVVCALL(OPFVF2, vfrdiv_vf_w, OP_UUU_W, H4, H4, float32_rdiv)
 RVVCALL(OPFVF2, vfrdiv_vf_d, OP_UUU_D, H8, H8, float64_rdiv)
-GEN_VEXT_VF(vfrdiv_vf_h, 2, 2)
-GEN_VEXT_VF(vfrdiv_vf_w, 4, 4)
-GEN_VEXT_VF(vfrdiv_vf_d, 8, 8)
+GEN_VEXT_VF(vfrdiv_vf_h)
+GEN_VEXT_VF(vfrdiv_vf_w)
+GEN_VEXT_VF(vfrdiv_vf_d)
 
 /* Vector Widening Floating-Point Multiply */
 static uint32_t vfwmul16(uint16_t a, uint16_t b, float_status *s)
@@ -3046,12 +3044,12 @@ static uint64_t vfwmul32(uint32_t a, uint32_t b, float_status *s)
 }
 RVVCALL(OPFVV2, vfwmul_vv_h, WOP_UUU_H, H4, H2, H2, vfwmul16)
 RVVCALL(OPFVV2, vfwmul_vv_w, WOP_UUU_W, H8, H4, H4, vfwmul32)
-GEN_VEXT_VV_ENV(vfwmul_vv_h, 2, 4)
-GEN_VEXT_VV_ENV(vfwmul_vv_w, 4, 8)
+GEN_VEXT_VV_ENV(vfwmul_vv_h)
+GEN_VEXT_VV_ENV(vfwmul_vv_w)
 RVVCALL(OPFVF2, vfwmul_vf_h, WOP_UUU_H, H4, H2, vfwmul16)
 RVVCALL(OPFVF2, vfwmul_vf_w, WOP_UUU_W, H8, H4, vfwmul32)
-GEN_VEXT_VF(vfwmul_vf_h, 2, 4)
-GEN_VEXT_VF(vfwmul_vf_w, 4, 8)
+GEN_VEXT_VF(vfwmul_vf_h)
+GEN_VEXT_VF(vfwmul_vf_w)
 
 /* Vector Single-Width Floating-Point Fused Multiply-Add Instructions */
 #define OPFVV3(NAME, TD, T1, T2, TX1, TX2, HD, HS1, HS2, OP)       \
@@ -3082,9 +3080,9 @@ static uint64_t fmacc64(uint64_t a, uint64_t b, uint64_t d, float_status *s)
 RVVCALL(OPFVV3, vfmacc_vv_h, OP_UUU_H, H2, H2, H2, fmacc16)
 RVVCALL(OPFVV3, vfmacc_vv_w, OP_UUU_W, H4, H4, H4, fmacc32)
 RVVCALL(OPFVV3, vfmacc_vv_d, OP_UUU_D, H8, H8, H8, fmacc64)
-GEN_VEXT_VV_ENV(vfmacc_vv_h, 2, 2)
-GEN_VEXT_VV_ENV(vfmacc_vv_w, 4, 4)
-GEN_VEXT_VV_ENV(vfmacc_vv_d, 8, 8)
+GEN_VEXT_VV_ENV(vfmacc_vv_h)
+GEN_VEXT_VV_ENV(vfmacc_vv_w)
+GEN_VEXT_VV_ENV(vfmacc_vv_d)
 
 #define OPFVF3(NAME, TD, T1, T2, TX1, TX2, HD, HS2, OP)           \
 static void do_##NAME(void *vd, uint64_t s1, void *vs2, int i,    \
@@ -3098,9 +3096,9 @@ static void do_##NAME(void *vd, uint64_t s1, void *vs2, int i,    \
 RVVCALL(OPFVF3, vfmacc_vf_h, OP_UUU_H, H2, H2, fmacc16)
 RVVCALL(OPFVF3, vfmacc_vf_w, OP_UUU_W, H4, H4, fmacc32)
 RVVCALL(OPFVF3, vfmacc_vf_d, OP_UUU_D, H8, H8, fmacc64)
-GEN_VEXT_VF(vfmacc_vf_h, 2, 2)
-GEN_VEXT_VF(vfmacc_vf_w, 4, 4)
-GEN_VEXT_VF(vfmacc_vf_d, 8, 8)
+GEN_VEXT_VF(vfmacc_vf_h)
+GEN_VEXT_VF(vfmacc_vf_w)
+GEN_VEXT_VF(vfmacc_vf_d)
 
 static uint16_t fnmacc16(uint16_t a, uint16_t b, uint16_t d, float_status *s)
 {
@@ -3123,15 +3121,15 @@ static uint64_t fnmacc64(uint64_t a, uint64_t b, uint64_t d, float_status *s)
 RVVCALL(OPFVV3, vfnmacc_vv_h, OP_UUU_H, H2, H2, H2, fnmacc16)
 RVVCALL(OPFVV3, vfnmacc_vv_w, OP_UUU_W, H4, H4, H4, fnmacc32)
 RVVCALL(OPFVV3, vfnmacc_vv_d, OP_UUU_D, H8, H8, H8, fnmacc64)
-GEN_VEXT_VV_ENV(vfnmacc_vv_h, 2, 2)
-GEN_VEXT_VV_ENV(vfnmacc_vv_w, 4, 4)
-GEN_VEXT_VV_ENV(vfnmacc_vv_d, 8, 8)
+GEN_VEXT_VV_ENV(vfnmacc_vv_h)
+GEN_VEXT_VV_ENV(vfnmacc_vv_w)
+GEN_VEXT_VV_ENV(vfnmacc_vv_d)
 RVVCALL(OPFVF3, vfnmacc_vf_h, OP_UUU_H, H2, H2, fnmacc16)
 RVVCALL(OPFVF3, vfnmacc_vf_w, OP_UUU_W, H4, H4, fnmacc32)
 RVVCALL(OPFVF3, vfnmacc_vf_d, OP_UUU_D, H8, H8, fnmacc64)
-GEN_VEXT_VF(vfnmacc_vf_h, 2, 2)
-GEN_VEXT_VF(vfnmacc_vf_w, 4, 4)
-GEN_VEXT_VF(vfnmacc_vf_d, 8, 8)
+GEN_VEXT_VF(vfnmacc_vf_h)
+GEN_VEXT_VF(vfnmacc_vf_w)
+GEN_VEXT_VF(vfnmacc_vf_d)
 
 static uint16_t fmsac16(uint16_t a, uint16_t b, uint16_t d, float_status *s)
 {
@@ -3151,15 +3149,15 @@ static uint64_t fmsac64(uint64_t a, uint64_t b, uint64_t d, float_status *s)
 RVVCALL(OPFVV3, vfmsac_vv_h, OP_UUU_H, H2, H2, H2, fmsac16)
 RVVCALL(OPFVV3, vfmsac_vv_w, OP_UUU_W, H4, H4, H4, fmsac32)
 RVVCALL(OPFVV3, vfmsac_vv_d, OP_UUU_D, H8, H8, H8, fmsac64)
-GEN_VEXT_VV_ENV(vfmsac_vv_h, 2, 2)
-GEN_VEXT_VV_ENV(vfmsac_vv_w, 4, 4)
-GEN_VEXT_VV_ENV(vfmsac_vv_d, 8, 8)
+GEN_VEXT_VV_ENV(vfmsac_vv_h)
+GEN_VEXT_VV_ENV(vfmsac_vv_w)
+GEN_VEXT_VV_ENV(vfmsac_vv_d)
 RVVCALL(OPFVF3, vfmsac_vf_h, OP_UUU_H, H2, H2, fmsac16)
 RVVCALL(OPFVF3, vfmsac_vf_w, OP_UUU_W, H4, H4, fmsac32)
 RVVCALL(OPFVF3, vfmsac_vf_d, OP_UUU_D, H8, H8, fmsac64)
-GEN_VEXT_VF(vfmsac_vf_h, 2, 2)
-GEN_VEXT_VF(vfmsac_vf_w, 4, 4)
-GEN_VEXT_VF(vfmsac_vf_d, 8, 8)
+GEN_VEXT_VF(vfmsac_vf_h)
+GEN_VEXT_VF(vfmsac_vf_w)
+GEN_VEXT_VF(vfmsac_vf_d)
 
 static uint16_t fnmsac16(uint16_t a, uint16_t b, uint16_t d, float_status *s)
 {
@@ -3179,15 +3177,15 @@ static uint64_t fnmsac64(uint64_t a, uint64_t b, uint64_t d, float_status *s)
 RVVCALL(OPFVV3, vfnmsac_vv_h, OP_UUU_H, H2, H2, H2, fnmsac16)
 RVVCALL(OPFVV3, vfnmsac_vv_w, OP_UUU_W, H4, H4, H4, fnmsac32)
 RVVCALL(OPFVV3, vfnmsac_vv_d, OP_UUU_D, H8, H8, H8, fnmsac64)
-GEN_VEXT_VV_ENV(vfnmsac_vv_h, 2, 2)
-GEN_VEXT_VV_ENV(vfnmsac_vv_w, 4, 4)
-GEN_VEXT_VV_ENV(vfnmsac_vv_d, 8, 8)
+GEN_VEXT_VV_ENV(vfnmsac_vv_h)
+GEN_VEXT_VV_ENV(vfnmsac_vv_w)
+GEN_VEXT_VV_ENV(vfnmsac_vv_d)
 RVVCALL(OPFVF3, vfnmsac_vf_h, OP_UUU_H, H2, H2, fnmsac16)
 RVVCALL(OPFVF3, vfnmsac_vf_w, OP_UUU_W, H4, H4, fnmsac32)
 RVVCALL(OPFVF3, vfnmsac_vf_d, OP_UUU_D, H8, H8, fnmsac64)
-GEN_VEXT_VF(vfnmsac_vf_h, 2, 2)
-GEN_VEXT_VF(vfnmsac_vf_w, 4, 4)
-GEN_VEXT_VF(vfnmsac_vf_d, 8, 8)
+GEN_VEXT_VF(vfnmsac_vf_h)
+GEN_VEXT_VF(vfnmsac_vf_w)
+GEN_VEXT_VF(vfnmsac_vf_d)
 
 static uint16_t fmadd16(uint16_t a, uint16_t b, uint16_t d, float_status *s)
 {
@@ -3207,15 +3205,15 @@ static uint64_t fmadd64(uint64_t a, uint64_t b, uint64_t d, float_status *s)
 RVVCALL(OPFVV3, vfmadd_vv_h, OP_UUU_H, H2, H2, H2, fmadd16)
 RVVCALL(OPFVV3, vfmadd_vv_w, OP_UUU_W, H4, H4, H4, fmadd32)
 RVVCALL(OPFVV3, vfmadd_vv_d, OP_UUU_D, H8, H8, H8, fmadd64)
-GEN_VEXT_VV_ENV(vfmadd_vv_h, 2, 2)
-GEN_VEXT_VV_ENV(vfmadd_vv_w, 4, 4)
-GEN_VEXT_VV_ENV(vfmadd_vv_d, 8, 8)
+GEN_VEXT_VV_ENV(vfmadd_vv_h)
+GEN_VEXT_VV_ENV(vfmadd_vv_w)
+GEN_VEXT_VV_ENV(vfmadd_vv_d)
 RVVCALL(OPFVF3, vfmadd_vf_h, OP_UUU_H, H2, H2, fmadd16)
 RVVCALL(OPFVF3, vfmadd_vf_w, OP_UUU_W, H4, H4, fmadd32)
 RVVCALL(OPFVF3, vfmadd_vf_d, OP_UUU_D, H8, H8, fmadd64)
-GEN_VEXT_VF(vfmadd_vf_h, 2, 2)
-GEN_VEXT_VF(vfmadd_vf_w, 4, 4)
-GEN_VEXT_VF(vfmadd_vf_d, 8, 8)
+GEN_VEXT_VF(vfmadd_vf_h)
+GEN_VEXT_VF(vfmadd_vf_w)
+GEN_VEXT_VF(vfmadd_vf_d)
 
 static uint16_t fnmadd16(uint16_t a, uint16_t b, uint16_t d, float_status *s)
 {
@@ -3238,15 +3236,15 @@ static uint64_t fnmadd64(uint64_t a, uint64_t b, uint64_t d, float_status *s)
 RVVCALL(OPFVV3, vfnmadd_vv_h, OP_UUU_H, H2, H2, H2, fnmadd16)
 RVVCALL(OPFVV3, vfnmadd_vv_w, OP_UUU_W, H4, H4, H4, fnmadd32)
 RVVCALL(OPFVV3, vfnmadd_vv_d, OP_UUU_D, H8, H8, H8, fnmadd64)
-GEN_VEXT_VV_ENV(vfnmadd_vv_h, 2, 2)
-GEN_VEXT_VV_ENV(vfnmadd_vv_w, 4, 4)
-GEN_VEXT_VV_ENV(vfnmadd_vv_d, 8, 8)
+GEN_VEXT_VV_ENV(vfnmadd_vv_h)
+GEN_VEXT_VV_ENV(vfnmadd_vv_w)
+GEN_VEXT_VV_ENV(vfnmadd_vv_d)
 RVVCALL(OPFVF3, vfnmadd_vf_h, OP_UUU_H, H2, H2, fnmadd16)
 RVVCALL(OPFVF3, vfnmadd_vf_w, OP_UUU_W, H4, H4, fnmadd32)
 RVVCALL(OPFVF3, vfnmadd_vf_d, OP_UUU_D, H8, H8, fnmadd64)
-GEN_VEXT_VF(vfnmadd_vf_h, 2, 2)
-GEN_VEXT_VF(vfnmadd_vf_w, 4, 4)
-GEN_VEXT_VF(vfnmadd_vf_d, 8, 8)
+GEN_VEXT_VF(vfnmadd_vf_h)
+GEN_VEXT_VF(vfnmadd_vf_w)
+GEN_VEXT_VF(vfnmadd_vf_d)
 
 static uint16_t fmsub16(uint16_t a, uint16_t b, uint16_t d, float_status *s)
 {
@@ -3266,15 +3264,15 @@ static uint64_t fmsub64(uint64_t a, uint64_t b, uint64_t d, float_status *s)
 RVVCALL(OPFVV3, vfmsub_vv_h, OP_UUU_H, H2, H2, H2, fmsub16)
 RVVCALL(OPFVV3, vfmsub_vv_w, OP_UUU_W, H4, H4, H4, fmsub32)
 RVVCALL(OPFVV3, vfmsub_vv_d, OP_UUU_D, H8, H8, H8, fmsub64)
-GEN_VEXT_VV_ENV(vfmsub_vv_h, 2, 2)
-GEN_VEXT_VV_ENV(vfmsub_vv_w, 4, 4)
-GEN_VEXT_VV_ENV(vfmsub_vv_d, 8, 8)
+GEN_VEXT_VV_ENV(vfmsub_vv_h)
+GEN_VEXT_VV_ENV(vfmsub_vv_w)
+GEN_VEXT_VV_ENV(vfmsub_vv_d)
 RVVCALL(OPFVF3, vfmsub_vf_h, OP_UUU_H, H2, H2, fmsub16)
 RVVCALL(OPFVF3, vfmsub_vf_w, OP_UUU_W, H4, H4, fmsub32)
 RVVCALL(OPFVF3, vfmsub_vf_d, OP_UUU_D, H8, H8, fmsub64)
-GEN_VEXT_VF(vfmsub_vf_h, 2, 2)
-GEN_VEXT_VF(vfmsub_vf_w, 4, 4)
-GEN_VEXT_VF(vfmsub_vf_d, 8, 8)
+GEN_VEXT_VF(vfmsub_vf_h)
+GEN_VEXT_VF(vfmsub_vf_w)
+GEN_VEXT_VF(vfmsub_vf_d)
 
 static uint16_t fnmsub16(uint16_t a, uint16_t b, uint16_t d, float_status *s)
 {
@@ -3294,15 +3292,15 @@ static uint64_t fnmsub64(uint64_t a, uint64_t b, uint64_t d, float_status *s)
 RVVCALL(OPFVV3, vfnmsub_vv_h, OP_UUU_H, H2, H2, H2, fnmsub16)
 RVVCALL(OPFVV3, vfnmsub_vv_w, OP_UUU_W, H4, H4, H4, fnmsub32)
 RVVCALL(OPFVV3, vfnmsub_vv_d, OP_UUU_D, H8, H8, H8, fnmsub64)
-GEN_VEXT_VV_ENV(vfnmsub_vv_h, 2, 2)
-GEN_VEXT_VV_ENV(vfnmsub_vv_w, 4, 4)
-GEN_VEXT_VV_ENV(vfnmsub_vv_d, 8, 8)
+GEN_VEXT_VV_ENV(vfnmsub_vv_h)
+GEN_VEXT_VV_ENV(vfnmsub_vv_w)
+GEN_VEXT_VV_ENV(vfnmsub_vv_d)
 RVVCALL(OPFVF3, vfnmsub_vf_h, OP_UUU_H, H2, H2, fnmsub16)
 RVVCALL(OPFVF3, vfnmsub_vf_w, OP_UUU_W, H4, H4, fnmsub32)
 RVVCALL(OPFVF3, vfnmsub_vf_d, OP_UUU_D, H8, H8, fnmsub64)
-GEN_VEXT_VF(vfnmsub_vf_h, 2, 2)
-GEN_VEXT_VF(vfnmsub_vf_w, 4, 4)
-GEN_VEXT_VF(vfnmsub_vf_d, 8, 8)
+GEN_VEXT_VF(vfnmsub_vf_h)
+GEN_VEXT_VF(vfnmsub_vf_w)
+GEN_VEXT_VF(vfnmsub_vf_d)
 
 /* Vector Widening Floating-Point Fused Multiply-Add Instructions */
 static uint32_t fwmacc16(uint16_t a, uint16_t b, uint32_t d, float_status *s)
@@ -3319,12 +3317,12 @@ static uint64_t fwmacc32(uint32_t a, uint32_t b, uint64_t d, float_status *s)
 
 RVVCALL(OPFVV3, vfwmacc_vv_h, WOP_UUU_H, H4, H2, H2, fwmacc16)
 RVVCALL(OPFVV3, vfwmacc_vv_w, WOP_UUU_W, H8, H4, H4, fwmacc32)
-GEN_VEXT_VV_ENV(vfwmacc_vv_h, 2, 4)
-GEN_VEXT_VV_ENV(vfwmacc_vv_w, 4, 8)
+GEN_VEXT_VV_ENV(vfwmacc_vv_h)
+GEN_VEXT_VV_ENV(vfwmacc_vv_w)
 RVVCALL(OPFVF3, vfwmacc_vf_h, WOP_UUU_H, H4, H2, fwmacc16)
 RVVCALL(OPFVF3, vfwmacc_vf_w, WOP_UUU_W, H8, H4, fwmacc32)
-GEN_VEXT_VF(vfwmacc_vf_h, 2, 4)
-GEN_VEXT_VF(vfwmacc_vf_w, 4, 8)
+GEN_VEXT_VF(vfwmacc_vf_h)
+GEN_VEXT_VF(vfwmacc_vf_w)
 
 static uint32_t fwnmacc16(uint16_t a, uint16_t b, uint32_t d, float_status *s)
 {
@@ -3342,12 +3340,12 @@ static uint64_t fwnmacc32(uint32_t a, uint32_t b, uint64_t d, float_status *s)
 
 RVVCALL(OPFVV3, vfwnmacc_vv_h, WOP_UUU_H, H4, H2, H2, fwnmacc16)
 RVVCALL(OPFVV3, vfwnmacc_vv_w, WOP_UUU_W, H8, H4, H4, fwnmacc32)
-GEN_VEXT_VV_ENV(vfwnmacc_vv_h, 2, 4)
-GEN_VEXT_VV_ENV(vfwnmacc_vv_w, 4, 8)
+GEN_VEXT_VV_ENV(vfwnmacc_vv_h)
+GEN_VEXT_VV_ENV(vfwnmacc_vv_w)
 RVVCALL(OPFVF3, vfwnmacc_vf_h, WOP_UUU_H, H4, H2, fwnmacc16)
 RVVCALL(OPFVF3, vfwnmacc_vf_w, WOP_UUU_W, H8, H4, fwnmacc32)
-GEN_VEXT_VF(vfwnmacc_vf_h, 2, 4)
-GEN_VEXT_VF(vfwnmacc_vf_w, 4, 8)
+GEN_VEXT_VF(vfwnmacc_vf_h)
+GEN_VEXT_VF(vfwnmacc_vf_w)
 
 static uint32_t fwmsac16(uint16_t a, uint16_t b, uint32_t d, float_status *s)
 {
@@ -3365,12 +3363,12 @@ static uint64_t fwmsac32(uint32_t a, uint32_t b, uint64_t d, float_status *s)
 
 RVVCALL(OPFVV3, vfwmsac_vv_h, WOP_UUU_H, H4, H2, H2, fwmsac16)
 RVVCALL(OPFVV3, vfwmsac_vv_w, WOP_UUU_W, H8, H4, H4, fwmsac32)
-GEN_VEXT_VV_ENV(vfwmsac_vv_h, 2, 4)
-GEN_VEXT_VV_ENV(vfwmsac_vv_w, 4, 8)
+GEN_VEXT_VV_ENV(vfwmsac_vv_h)
+GEN_VEXT_VV_ENV(vfwmsac_vv_w)
 RVVCALL(OPFVF3, vfwmsac_vf_h, WOP_UUU_H, H4, H2, fwmsac16)
 RVVCALL(OPFVF3, vfwmsac_vf_w, WOP_UUU_W, H8, H4, fwmsac32)
-GEN_VEXT_VF(vfwmsac_vf_h, 2, 4)
-GEN_VEXT_VF(vfwmsac_vf_w, 4, 8)
+GEN_VEXT_VF(vfwmsac_vf_h)
+GEN_VEXT_VF(vfwmsac_vf_w)
 
 static uint32_t fwnmsac16(uint16_t a, uint16_t b, uint32_t d, float_status *s)
 {
@@ -3388,12 +3386,12 @@ static uint64_t fwnmsac32(uint32_t a, uint32_t b, uint64_t d, float_status *s)
 
 RVVCALL(OPFVV3, vfwnmsac_vv_h, WOP_UUU_H, H4, H2, H2, fwnmsac16)
 RVVCALL(OPFVV3, vfwnmsac_vv_w, WOP_UUU_W, H8, H4, H4, fwnmsac32)
-GEN_VEXT_VV_ENV(vfwnmsac_vv_h, 2, 4)
-GEN_VEXT_VV_ENV(vfwnmsac_vv_w, 4, 8)
+GEN_VEXT_VV_ENV(vfwnmsac_vv_h)
+GEN_VEXT_VV_ENV(vfwnmsac_vv_w)
 RVVCALL(OPFVF3, vfwnmsac_vf_h, WOP_UUU_H, H4, H2, fwnmsac16)
 RVVCALL(OPFVF3, vfwnmsac_vf_w, WOP_UUU_W, H8, H4, fwnmsac32)
-GEN_VEXT_VF(vfwnmsac_vf_h, 2, 4)
-GEN_VEXT_VF(vfwnmsac_vf_w, 4, 8)
+GEN_VEXT_VF(vfwnmsac_vf_h)
+GEN_VEXT_VF(vfwnmsac_vf_w)
 
 /* Vector Floating-Point Square-Root Instruction */
 /* (TD, T2, TX2) */
@@ -3409,7 +3407,7 @@ static void do_##NAME(void *vd, void *vs2, int i,      \
     *((TD *)vd + HD(i)) = OP(s2, &env->fp_status);     \
 }
 
-#define GEN_VEXT_V_ENV(NAME, ESZ, DSZ)                 \
+#define GEN_VEXT_V_ENV(NAME)                           \
 void HELPER(NAME)(void *vd, void *v0, void *vs2,       \
         CPURISCVState *env, uint32_t desc)             \
 {                                                      \
@@ -3432,9 +3430,9 @@ void HELPER(NAME)(void *vd, void *v0, void *vs2,       \
 RVVCALL(OPFVV1, vfsqrt_v_h, OP_UU_H, H2, H2, float16_sqrt)
 RVVCALL(OPFVV1, vfsqrt_v_w, OP_UU_W, H4, H4, float32_sqrt)
 RVVCALL(OPFVV1, vfsqrt_v_d, OP_UU_D, H8, H8, float64_sqrt)
-GEN_VEXT_V_ENV(vfsqrt_v_h, 2, 2)
-GEN_VEXT_V_ENV(vfsqrt_v_w, 4, 4)
-GEN_VEXT_V_ENV(vfsqrt_v_d, 8, 8)
+GEN_VEXT_V_ENV(vfsqrt_v_h)
+GEN_VEXT_V_ENV(vfsqrt_v_w)
+GEN_VEXT_V_ENV(vfsqrt_v_d)
 
 /*
  * Vector Floating-Point Reciprocal Square-Root Estimate Instruction
@@ -3614,9 +3612,9 @@ static float64 frsqrt7_d(float64 f, float_status *s)
 RVVCALL(OPFVV1, vfrsqrt7_v_h, OP_UU_H, H2, H2, frsqrt7_h)
 RVVCALL(OPFVV1, vfrsqrt7_v_w, OP_UU_W, H4, H4, frsqrt7_s)
 RVVCALL(OPFVV1, vfrsqrt7_v_d, OP_UU_D, H8, H8, frsqrt7_d)
-GEN_VEXT_V_ENV(vfrsqrt7_v_h, 2, 2)
-GEN_VEXT_V_ENV(vfrsqrt7_v_w, 4, 4)
-GEN_VEXT_V_ENV(vfrsqrt7_v_d, 8, 8)
+GEN_VEXT_V_ENV(vfrsqrt7_v_h)
+GEN_VEXT_V_ENV(vfrsqrt7_v_w)
+GEN_VEXT_V_ENV(vfrsqrt7_v_d)
 
 /*
  * Vector Floating-Point Reciprocal Estimate Instruction
@@ -3805,36 +3803,36 @@ static float64 frec7_d(float64 f, float_status *s)
 RVVCALL(OPFVV1, vfrec7_v_h, OP_UU_H, H2, H2, frec7_h)
 RVVCALL(OPFVV1, vfrec7_v_w, OP_UU_W, H4, H4, frec7_s)
 RVVCALL(OPFVV1, vfrec7_v_d, OP_UU_D, H8, H8, frec7_d)
-GEN_VEXT_V_ENV(vfrec7_v_h, 2, 2)
-GEN_VEXT_V_ENV(vfrec7_v_w, 4, 4)
-GEN_VEXT_V_ENV(vfrec7_v_d, 8, 8)
+GEN_VEXT_V_ENV(vfrec7_v_h)
+GEN_VEXT_V_ENV(vfrec7_v_w)
+GEN_VEXT_V_ENV(vfrec7_v_d)
 
 /* Vector Floating-Point MIN/MAX Instructions */
 RVVCALL(OPFVV2, vfmin_vv_h, OP_UUU_H, H2, H2, H2, float16_minimum_number)
 RVVCALL(OPFVV2, vfmin_vv_w, OP_UUU_W, H4, H4, H4, float32_minimum_number)
 RVVCALL(OPFVV2, vfmin_vv_d, OP_UUU_D, H8, H8, H8, float64_minimum_number)
-GEN_VEXT_VV_ENV(vfmin_vv_h, 2, 2)
-GEN_VEXT_VV_ENV(vfmin_vv_w, 4, 4)
-GEN_VEXT_VV_ENV(vfmin_vv_d, 8, 8)
+GEN_VEXT_VV_ENV(vfmin_vv_h)
+GEN_VEXT_VV_ENV(vfmin_vv_w)
+GEN_VEXT_VV_ENV(vfmin_vv_d)
 RVVCALL(OPFVF2, vfmin_vf_h, OP_UUU_H, H2, H2, float16_minimum_number)
 RVVCALL(OPFVF2, vfmin_vf_w, OP_UUU_W, H4, H4, float32_minimum_number)
 RVVCALL(OPFVF2, vfmin_vf_d, OP_UUU_D, H8, H8, float64_minimum_number)
-GEN_VEXT_VF(vfmin_vf_h, 2, 2)
-GEN_VEXT_VF(vfmin_vf_w, 4, 4)
-GEN_VEXT_VF(vfmin_vf_d, 8, 8)
+GEN_VEXT_VF(vfmin_vf_h)
+GEN_VEXT_VF(vfmin_vf_w)
+GEN_VEXT_VF(vfmin_vf_d)
 
 RVVCALL(OPFVV2, vfmax_vv_h, OP_UUU_H, H2, H2, H2, float16_maximum_number)
 RVVCALL(OPFVV2, vfmax_vv_w, OP_UUU_W, H4, H4, H4, float32_maximum_number)
 RVVCALL(OPFVV2, vfmax_vv_d, OP_UUU_D, H8, H8, H8, float64_maximum_number)
-GEN_VEXT_VV_ENV(vfmax_vv_h, 2, 2)
-GEN_VEXT_VV_ENV(vfmax_vv_w, 4, 4)
-GEN_VEXT_VV_ENV(vfmax_vv_d, 8, 8)
+GEN_VEXT_VV_ENV(vfmax_vv_h)
+GEN_VEXT_VV_ENV(vfmax_vv_w)
+GEN_VEXT_VV_ENV(vfmax_vv_d)
 RVVCALL(OPFVF2, vfmax_vf_h, OP_UUU_H, H2, H2, float16_maximum_number)
 RVVCALL(OPFVF2, vfmax_vf_w, OP_UUU_W, H4, H4, float32_maximum_number)
 RVVCALL(OPFVF2, vfmax_vf_d, OP_UUU_D, H8, H8, float64_maximum_number)
-GEN_VEXT_VF(vfmax_vf_h, 2, 2)
-GEN_VEXT_VF(vfmax_vf_w, 4, 4)
-GEN_VEXT_VF(vfmax_vf_d, 8, 8)
+GEN_VEXT_VF(vfmax_vf_h)
+GEN_VEXT_VF(vfmax_vf_w)
+GEN_VEXT_VF(vfmax_vf_d)
 
 /* Vector Floating-Point Sign-Injection Instructions */
 static uint16_t fsgnj16(uint16_t a, uint16_t b, float_status *s)
@@ -3855,15 +3853,15 @@ static uint64_t fsgnj64(uint64_t a, uint64_t b, float_status *s)
 RVVCALL(OPFVV2, vfsgnj_vv_h, OP_UUU_H, H2, H2, H2, fsgnj16)
 RVVCALL(OPFVV2, vfsgnj_vv_w, OP_UUU_W, H4, H4, H4, fsgnj32)
 RVVCALL(OPFVV2, vfsgnj_vv_d, OP_UUU_D, H8, H8, H8, fsgnj64)
-GEN_VEXT_VV_ENV(vfsgnj_vv_h, 2, 2)
-GEN_VEXT_VV_ENV(vfsgnj_vv_w, 4, 4)
-GEN_VEXT_VV_ENV(vfsgnj_vv_d, 8, 8)
+GEN_VEXT_VV_ENV(vfsgnj_vv_h)
+GEN_VEXT_VV_ENV(vfsgnj_vv_w)
+GEN_VEXT_VV_ENV(vfsgnj_vv_d)
 RVVCALL(OPFVF2, vfsgnj_vf_h, OP_UUU_H, H2, H2, fsgnj16)
 RVVCALL(OPFVF2, vfsgnj_vf_w, OP_UUU_W, H4, H4, fsgnj32)
 RVVCALL(OPFVF2, vfsgnj_vf_d, OP_UUU_D, H8, H8, fsgnj64)
-GEN_VEXT_VF(vfsgnj_vf_h, 2, 2)
-GEN_VEXT_VF(vfsgnj_vf_w, 4, 4)
-GEN_VEXT_VF(vfsgnj_vf_d, 8, 8)
+GEN_VEXT_VF(vfsgnj_vf_h)
+GEN_VEXT_VF(vfsgnj_vf_w)
+GEN_VEXT_VF(vfsgnj_vf_d)
 
 static uint16_t fsgnjn16(uint16_t a, uint16_t b, float_status *s)
 {
@@ -3883,15 +3881,15 @@ static uint64_t fsgnjn64(uint64_t a, uint64_t b, float_status *s)
 RVVCALL(OPFVV2, vfsgnjn_vv_h, OP_UUU_H, H2, H2, H2, fsgnjn16)
 RVVCALL(OPFVV2, vfsgnjn_vv_w, OP_UUU_W, H4, H4, H4, fsgnjn32)
 RVVCALL(OPFVV2, vfsgnjn_vv_d, OP_UUU_D, H8, H8, H8, fsgnjn64)
-GEN_VEXT_VV_ENV(vfsgnjn_vv_h, 2, 2)
-GEN_VEXT_VV_ENV(vfsgnjn_vv_w, 4, 4)
-GEN_VEXT_VV_ENV(vfsgnjn_vv_d, 8, 8)
+GEN_VEXT_VV_ENV(vfsgnjn_vv_h)
+GEN_VEXT_VV_ENV(vfsgnjn_vv_w)
+GEN_VEXT_VV_ENV(vfsgnjn_vv_d)
 RVVCALL(OPFVF2, vfsgnjn_vf_h, OP_UUU_H, H2, H2, fsgnjn16)
 RVVCALL(OPFVF2, vfsgnjn_vf_w, OP_UUU_W, H4, H4, fsgnjn32)
 RVVCALL(OPFVF2, vfsgnjn_vf_d, OP_UUU_D, H8, H8, fsgnjn64)
-GEN_VEXT_VF(vfsgnjn_vf_h, 2, 2)
-GEN_VEXT_VF(vfsgnjn_vf_w, 4, 4)
-GEN_VEXT_VF(vfsgnjn_vf_d, 8, 8)
+GEN_VEXT_VF(vfsgnjn_vf_h)
+GEN_VEXT_VF(vfsgnjn_vf_w)
+GEN_VEXT_VF(vfsgnjn_vf_d)
 
 static uint16_t fsgnjx16(uint16_t a, uint16_t b, float_status *s)
 {
@@ -3911,15 +3909,15 @@ static uint64_t fsgnjx64(uint64_t a, uint64_t b, float_status *s)
 RVVCALL(OPFVV2, vfsgnjx_vv_h, OP_UUU_H, H2, H2, H2, fsgnjx16)
 RVVCALL(OPFVV2, vfsgnjx_vv_w, OP_UUU_W, H4, H4, H4, fsgnjx32)
 RVVCALL(OPFVV2, vfsgnjx_vv_d, OP_UUU_D, H8, H8, H8, fsgnjx64)
-GEN_VEXT_VV_ENV(vfsgnjx_vv_h, 2, 2)
-GEN_VEXT_VV_ENV(vfsgnjx_vv_w, 4, 4)
-GEN_VEXT_VV_ENV(vfsgnjx_vv_d, 8, 8)
+GEN_VEXT_VV_ENV(vfsgnjx_vv_h)
+GEN_VEXT_VV_ENV(vfsgnjx_vv_w)
+GEN_VEXT_VV_ENV(vfsgnjx_vv_d)
 RVVCALL(OPFVF2, vfsgnjx_vf_h, OP_UUU_H, H2, H2, fsgnjx16)
 RVVCALL(OPFVF2, vfsgnjx_vf_w, OP_UUU_W, H4, H4, fsgnjx32)
 RVVCALL(OPFVF2, vfsgnjx_vf_d, OP_UUU_D, H8, H8, fsgnjx64)
-GEN_VEXT_VF(vfsgnjx_vf_h, 2, 2)
-GEN_VEXT_VF(vfsgnjx_vf_w, 4, 4)
-GEN_VEXT_VF(vfsgnjx_vf_d, 8, 8)
+GEN_VEXT_VF(vfsgnjx_vf_h)
+GEN_VEXT_VF(vfsgnjx_vf_w)
+GEN_VEXT_VF(vfsgnjx_vf_d)
 
 /* Vector Floating-Point Compare Instructions */
 #define GEN_VEXT_CMP_VV_ENV(NAME, ETYPE, H, DO_OP)            \
@@ -4063,7 +4061,7 @@ static void do_##NAME(void *vd, void *vs2, int i)      \
     *((TD *)vd + HD(i)) = OP(s2);                      \
 }
 
-#define GEN_VEXT_V(NAME, ESZ, DSZ)                     \
+#define GEN_VEXT_V(NAME)                               \
 void HELPER(NAME)(void *vd, void *v0, void *vs2,       \
                   CPURISCVState *env, uint32_t desc)   \
 {                                                      \
@@ -4140,9 +4138,9 @@ target_ulong fclass_d(uint64_t frs1)
 RVVCALL(OPIVV1, vfclass_v_h, OP_UU_H, H2, H2, fclass_h)
 RVVCALL(OPIVV1, vfclass_v_w, OP_UU_W, H4, H4, fclass_s)
 RVVCALL(OPIVV1, vfclass_v_d, OP_UU_D, H8, H8, fclass_d)
-GEN_VEXT_V(vfclass_v_h, 2, 2)
-GEN_VEXT_V(vfclass_v_w, 4, 4)
-GEN_VEXT_V(vfclass_v_d, 8, 8)
+GEN_VEXT_V(vfclass_v_h)
+GEN_VEXT_V(vfclass_v_w)
+GEN_VEXT_V(vfclass_v_d)
 
 /* Vector Floating-Point Merge Instruction */
 #define GEN_VFMERGE_VF(NAME, ETYPE, H)                        \
@@ -4170,33 +4168,33 @@ GEN_VFMERGE_VF(vfmerge_vfm_d, int64_t, H8)
 RVVCALL(OPFVV1, vfcvt_xu_f_v_h, OP_UU_H, H2, H2, float16_to_uint16)
 RVVCALL(OPFVV1, vfcvt_xu_f_v_w, OP_UU_W, H4, H4, float32_to_uint32)
 RVVCALL(OPFVV1, vfcvt_xu_f_v_d, OP_UU_D, H8, H8, float64_to_uint64)
-GEN_VEXT_V_ENV(vfcvt_xu_f_v_h, 2, 2)
-GEN_VEXT_V_ENV(vfcvt_xu_f_v_w, 4, 4)
-GEN_VEXT_V_ENV(vfcvt_xu_f_v_d, 8, 8)
+GEN_VEXT_V_ENV(vfcvt_xu_f_v_h)
+GEN_VEXT_V_ENV(vfcvt_xu_f_v_w)
+GEN_VEXT_V_ENV(vfcvt_xu_f_v_d)
 
 /* vfcvt.x.f.v vd, vs2, vm # Convert float to signed integer. */
 RVVCALL(OPFVV1, vfcvt_x_f_v_h, OP_UU_H, H2, H2, float16_to_int16)
 RVVCALL(OPFVV1, vfcvt_x_f_v_w, OP_UU_W, H4, H4, float32_to_int32)
 RVVCALL(OPFVV1, vfcvt_x_f_v_d, OP_UU_D, H8, H8, float64_to_int64)
-GEN_VEXT_V_ENV(vfcvt_x_f_v_h, 2, 2)
-GEN_VEXT_V_ENV(vfcvt_x_f_v_w, 4, 4)
-GEN_VEXT_V_ENV(vfcvt_x_f_v_d, 8, 8)
+GEN_VEXT_V_ENV(vfcvt_x_f_v_h)
+GEN_VEXT_V_ENV(vfcvt_x_f_v_w)
+GEN_VEXT_V_ENV(vfcvt_x_f_v_d)
 
 /* vfcvt.f.xu.v vd, vs2, vm # Convert unsigned integer to float. */
 RVVCALL(OPFVV1, vfcvt_f_xu_v_h, OP_UU_H, H2, H2, uint16_to_float16)
 RVVCALL(OPFVV1, vfcvt_f_xu_v_w, OP_UU_W, H4, H4, uint32_to_float32)
 RVVCALL(OPFVV1, vfcvt_f_xu_v_d, OP_UU_D, H8, H8, uint64_to_float64)
-GEN_VEXT_V_ENV(vfcvt_f_xu_v_h, 2, 2)
-GEN_VEXT_V_ENV(vfcvt_f_xu_v_w, 4, 4)
-GEN_VEXT_V_ENV(vfcvt_f_xu_v_d, 8, 8)
+GEN_VEXT_V_ENV(vfcvt_f_xu_v_h)
+GEN_VEXT_V_ENV(vfcvt_f_xu_v_w)
+GEN_VEXT_V_ENV(vfcvt_f_xu_v_d)
 
 /* vfcvt.f.x.v vd, vs2, vm # Convert integer to float. */
 RVVCALL(OPFVV1, vfcvt_f_x_v_h, OP_UU_H, H2, H2, int16_to_float16)
 RVVCALL(OPFVV1, vfcvt_f_x_v_w, OP_UU_W, H4, H4, int32_to_float32)
 RVVCALL(OPFVV1, vfcvt_f_x_v_d, OP_UU_D, H8, H8, int64_to_float64)
-GEN_VEXT_V_ENV(vfcvt_f_x_v_h, 2, 2)
-GEN_VEXT_V_ENV(vfcvt_f_x_v_w, 4, 4)
-GEN_VEXT_V_ENV(vfcvt_f_x_v_d, 8, 8)
+GEN_VEXT_V_ENV(vfcvt_f_x_v_h)
+GEN_VEXT_V_ENV(vfcvt_f_x_v_w)
+GEN_VEXT_V_ENV(vfcvt_f_x_v_d)
 
 /* Widening Floating-Point/Integer Type-Convert Instructions */
 /* (TD, T2, TX2) */
@@ -4206,30 +4204,30 @@ GEN_VEXT_V_ENV(vfcvt_f_x_v_d, 8, 8)
 /* vfwcvt.xu.f.v vd, vs2, vm # Convert float to double-width unsigned integer.*/
 RVVCALL(OPFVV1, vfwcvt_xu_f_v_h, WOP_UU_H, H4, H2, float16_to_uint32)
 RVVCALL(OPFVV1, vfwcvt_xu_f_v_w, WOP_UU_W, H8, H4, float32_to_uint64)
-GEN_VEXT_V_ENV(vfwcvt_xu_f_v_h, 2, 4)
-GEN_VEXT_V_ENV(vfwcvt_xu_f_v_w, 4, 8)
+GEN_VEXT_V_ENV(vfwcvt_xu_f_v_h)
+GEN_VEXT_V_ENV(vfwcvt_xu_f_v_w)
 
 /* vfwcvt.x.f.v vd, vs2, vm # Convert float to double-width signed integer. */
 RVVCALL(OPFVV1, vfwcvt_x_f_v_h, WOP_UU_H, H4, H2, float16_to_int32)
 RVVCALL(OPFVV1, vfwcvt_x_f_v_w, WOP_UU_W, H8, H4, float32_to_int64)
-GEN_VEXT_V_ENV(vfwcvt_x_f_v_h, 2, 4)
-GEN_VEXT_V_ENV(vfwcvt_x_f_v_w, 4, 8)
+GEN_VEXT_V_ENV(vfwcvt_x_f_v_h)
+GEN_VEXT_V_ENV(vfwcvt_x_f_v_w)
 
 /* vfwcvt.f.xu.v vd, vs2, vm # Convert unsigned integer to double-width float */
 RVVCALL(OPFVV1, vfwcvt_f_xu_v_b, WOP_UU_B, H2, H1, uint8_to_float16)
 RVVCALL(OPFVV1, vfwcvt_f_xu_v_h, WOP_UU_H, H4, H2, uint16_to_float32)
 RVVCALL(OPFVV1, vfwcvt_f_xu_v_w, WOP_UU_W, H8, H4, uint32_to_float64)
-GEN_VEXT_V_ENV(vfwcvt_f_xu_v_b, 1, 2)
-GEN_VEXT_V_ENV(vfwcvt_f_xu_v_h, 2, 4)
-GEN_VEXT_V_ENV(vfwcvt_f_xu_v_w, 4, 8)
+GEN_VEXT_V_ENV(vfwcvt_f_xu_v_b)
+GEN_VEXT_V_ENV(vfwcvt_f_xu_v_h)
+GEN_VEXT_V_ENV(vfwcvt_f_xu_v_w)
 
 /* vfwcvt.f.x.v vd, vs2, vm # Convert integer to double-width float. */
 RVVCALL(OPFVV1, vfwcvt_f_x_v_b, WOP_UU_B, H2, H1, int8_to_float16)
 RVVCALL(OPFVV1, vfwcvt_f_x_v_h, WOP_UU_H, H4, H2, int16_to_float32)
 RVVCALL(OPFVV1, vfwcvt_f_x_v_w, WOP_UU_W, H8, H4, int32_to_float64)
-GEN_VEXT_V_ENV(vfwcvt_f_x_v_b, 1, 2)
-GEN_VEXT_V_ENV(vfwcvt_f_x_v_h, 2, 4)
-GEN_VEXT_V_ENV(vfwcvt_f_x_v_w, 4, 8)
+GEN_VEXT_V_ENV(vfwcvt_f_x_v_b)
+GEN_VEXT_V_ENV(vfwcvt_f_x_v_h)
+GEN_VEXT_V_ENV(vfwcvt_f_x_v_w)
 
 /*
  * vfwcvt.f.f.v vd, vs2, vm
@@ -4242,8 +4240,8 @@ static uint32_t vfwcvtffv16(uint16_t a, float_status *s)
 
 RVVCALL(OPFVV1, vfwcvt_f_f_v_h, WOP_UU_H, H4, H2, vfwcvtffv16)
 RVVCALL(OPFVV1, vfwcvt_f_f_v_w, WOP_UU_W, H8, H4, float32_to_float64)
-GEN_VEXT_V_ENV(vfwcvt_f_f_v_h, 2, 4)
-GEN_VEXT_V_ENV(vfwcvt_f_f_v_w, 4, 8)
+GEN_VEXT_V_ENV(vfwcvt_f_f_v_h)
+GEN_VEXT_V_ENV(vfwcvt_f_f_v_w)
 
 /* Narrowing Floating-Point/Integer Type-Convert Instructions */
 /* (TD, T2, TX2) */
@@ -4254,29 +4252,29 @@ GEN_VEXT_V_ENV(vfwcvt_f_f_v_w, 4, 8)
 RVVCALL(OPFVV1, vfncvt_xu_f_w_b, NOP_UU_B, H1, H2, float16_to_uint8)
 RVVCALL(OPFVV1, vfncvt_xu_f_w_h, NOP_UU_H, H2, H4, float32_to_uint16)
 RVVCALL(OPFVV1, vfncvt_xu_f_w_w, NOP_UU_W, H4, H8, float64_to_uint32)
-GEN_VEXT_V_ENV(vfncvt_xu_f_w_b, 1, 1)
-GEN_VEXT_V_ENV(vfncvt_xu_f_w_h, 2, 2)
-GEN_VEXT_V_ENV(vfncvt_xu_f_w_w, 4, 4)
+GEN_VEXT_V_ENV(vfncvt_xu_f_w_b)
+GEN_VEXT_V_ENV(vfncvt_xu_f_w_h)
+GEN_VEXT_V_ENV(vfncvt_xu_f_w_w)
 
 /* vfncvt.x.f.v vd, vs2, vm # Convert double-width float to signed integer. */
 RVVCALL(OPFVV1, vfncvt_x_f_w_b, NOP_UU_B, H1, H2, float16_to_int8)
 RVVCALL(OPFVV1, vfncvt_x_f_w_h, NOP_UU_H, H2, H4, float32_to_int16)
 RVVCALL(OPFVV1, vfncvt_x_f_w_w, NOP_UU_W, H4, H8, float64_to_int32)
-GEN_VEXT_V_ENV(vfncvt_x_f_w_b, 1, 1)
-GEN_VEXT_V_ENV(vfncvt_x_f_w_h, 2, 2)
-GEN_VEXT_V_ENV(vfncvt_x_f_w_w, 4, 4)
+GEN_VEXT_V_ENV(vfncvt_x_f_w_b)
+GEN_VEXT_V_ENV(vfncvt_x_f_w_h)
+GEN_VEXT_V_ENV(vfncvt_x_f_w_w)
 
 /* vfncvt.f.xu.v vd, vs2, vm # Convert double-width unsigned integer to float */
 RVVCALL(OPFVV1, vfncvt_f_xu_w_h, NOP_UU_H, H2, H4, uint32_to_float16)
 RVVCALL(OPFVV1, vfncvt_f_xu_w_w, NOP_UU_W, H4, H8, uint64_to_float32)
-GEN_VEXT_V_ENV(vfncvt_f_xu_w_h, 2, 2)
-GEN_VEXT_V_ENV(vfncvt_f_xu_w_w, 4, 4)
+GEN_VEXT_V_ENV(vfncvt_f_xu_w_h)
+GEN_VEXT_V_ENV(vfncvt_f_xu_w_w)
 
 /* vfncvt.f.x.v vd, vs2, vm # Convert double-width integer to float. */
 RVVCALL(OPFVV1, vfncvt_f_x_w_h, NOP_UU_H, H2, H4, int32_to_float16)
 RVVCALL(OPFVV1, vfncvt_f_x_w_w, NOP_UU_W, H4, H8, int64_to_float32)
-GEN_VEXT_V_ENV(vfncvt_f_x_w_h, 2, 2)
-GEN_VEXT_V_ENV(vfncvt_f_x_w_w, 4, 4)
+GEN_VEXT_V_ENV(vfncvt_f_x_w_h)
+GEN_VEXT_V_ENV(vfncvt_f_x_w_w)
 
 /* vfncvt.f.f.v vd, vs2, vm # Convert double float to single-width float. */
 static uint16_t vfncvtffv16(uint32_t a, float_status *s)
@@ -4286,8 +4284,8 @@ static uint16_t vfncvtffv16(uint32_t a, float_status *s)
 
 RVVCALL(OPFVV1, vfncvt_f_f_w_h, NOP_UU_H, H2, H4, vfncvtffv16)
 RVVCALL(OPFVV1, vfncvt_f_f_w_w, NOP_UU_W, H4, H8, float64_to_float32)
-GEN_VEXT_V_ENV(vfncvt_f_f_w_h, 2, 2)
-GEN_VEXT_V_ENV(vfncvt_f_f_w_w, 4, 4)
+GEN_VEXT_V_ENV(vfncvt_f_f_w_h)
+GEN_VEXT_V_ENV(vfncvt_f_f_w_w)
 
 /*
  *** Vector Reduction Operations

From 25eae0486db6e24f86550e3c42af2b17cc43ddbf Mon Sep 17 00:00:00 2001
From: eopXD <yueh.ting.chen@gmail.com>
Date: Mon, 6 Jun 2022 06:16:16 +0000
Subject: [PATCH 919/935] target/riscv: rvv: Prune redundant access_type
 parameter passed

No functional change intended in this commit.

Signed-off-by: eop Chen <eop.chen@sifive.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <165449614532.19704.7000832880482980398-2@git.sr.ht>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/vector_helper.c | 35 ++++++++++++++++-------------------
 1 file changed, 16 insertions(+), 19 deletions(-)

diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c
index 85dd611cd9..60840325c4 100644
--- a/target/riscv/vector_helper.c
+++ b/target/riscv/vector_helper.c
@@ -231,7 +231,7 @@ vext_ldst_stride(void *vd, void *v0, target_ulong base,
                  target_ulong stride, CPURISCVState *env,
                  uint32_t desc, uint32_t vm,
                  vext_ldst_elem_fn *ldst_elem,
-                 uint32_t esz, uintptr_t ra, MMUAccessType access_type)
+                 uint32_t esz, uintptr_t ra)
 {
     uint32_t i, k;
     uint32_t nf = vext_nf(desc);
@@ -259,7 +259,7 @@ void HELPER(NAME)(void *vd, void * v0, target_ulong base,               \
 {                                                                       \
     uint32_t vm = vext_vm(desc);                                        \
     vext_ldst_stride(vd, v0, base, stride, env, desc, vm, LOAD_FN,      \
-                     ctzl(sizeof(ETYPE)), GETPC(), MMU_DATA_LOAD);      \
+                     ctzl(sizeof(ETYPE)), GETPC());                     \
 }
 
 GEN_VEXT_LD_STRIDE(vlse8_v,  int8_t,  lde_b)
@@ -274,7 +274,7 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong base,                \
 {                                                                       \
     uint32_t vm = vext_vm(desc);                                        \
     vext_ldst_stride(vd, v0, base, stride, env, desc, vm, STORE_FN,     \
-                     ctzl(sizeof(ETYPE)), GETPC(), MMU_DATA_STORE);     \
+                     ctzl(sizeof(ETYPE)), GETPC());                     \
 }
 
 GEN_VEXT_ST_STRIDE(vsse8_v,  int8_t,  ste_b)
@@ -290,7 +290,7 @@ GEN_VEXT_ST_STRIDE(vsse64_v, int64_t, ste_d)
 static void
 vext_ldst_us(void *vd, target_ulong base, CPURISCVState *env, uint32_t desc,
              vext_ldst_elem_fn *ldst_elem, uint32_t esz, uint32_t evl,
-             uintptr_t ra, MMUAccessType access_type)
+             uintptr_t ra)
 {
     uint32_t i, k;
     uint32_t nf = vext_nf(desc);
@@ -319,14 +319,14 @@ void HELPER(NAME##_mask)(void *vd, void *v0, target_ulong base,         \
 {                                                                       \
     uint32_t stride = vext_nf(desc) << ctzl(sizeof(ETYPE));             \
     vext_ldst_stride(vd, v0, base, stride, env, desc, false, LOAD_FN,   \
-                     ctzl(sizeof(ETYPE)), GETPC(), MMU_DATA_LOAD);      \
+                     ctzl(sizeof(ETYPE)), GETPC());                     \
 }                                                                       \
                                                                         \
 void HELPER(NAME)(void *vd, void *v0, target_ulong base,                \
                   CPURISCVState *env, uint32_t desc)                    \
 {                                                                       \
     vext_ldst_us(vd, base, env, desc, LOAD_FN,                          \
-                 ctzl(sizeof(ETYPE)), env->vl, GETPC(), MMU_DATA_LOAD); \
+                 ctzl(sizeof(ETYPE)), env->vl, GETPC());                \
 }
 
 GEN_VEXT_LD_US(vle8_v,  int8_t,  lde_b)
@@ -340,14 +340,14 @@ void HELPER(NAME##_mask)(void *vd, void *v0, target_ulong base,          \
 {                                                                        \
     uint32_t stride = vext_nf(desc) << ctzl(sizeof(ETYPE));              \
     vext_ldst_stride(vd, v0, base, stride, env, desc, false, STORE_FN,   \
-                     ctzl(sizeof(ETYPE)), GETPC(), MMU_DATA_STORE);      \
+                     ctzl(sizeof(ETYPE)), GETPC());                      \
 }                                                                        \
                                                                          \
 void HELPER(NAME)(void *vd, void *v0, target_ulong base,                 \
                   CPURISCVState *env, uint32_t desc)                     \
 {                                                                        \
     vext_ldst_us(vd, base, env, desc, STORE_FN,                          \
-                 ctzl(sizeof(ETYPE)), env->vl, GETPC(), MMU_DATA_STORE); \
+                 ctzl(sizeof(ETYPE)), env->vl, GETPC());                 \
 }
 
 GEN_VEXT_ST_US(vse8_v,  int8_t,  ste_b)
@@ -364,7 +364,7 @@ void HELPER(vlm_v)(void *vd, void *v0, target_ulong base,
     /* evl = ceil(vl/8) */
     uint8_t evl = (env->vl + 7) >> 3;
     vext_ldst_us(vd, base, env, desc, lde_b,
-                 0, evl, GETPC(), MMU_DATA_LOAD);
+                 0, evl, GETPC());
 }
 
 void HELPER(vsm_v)(void *vd, void *v0, target_ulong base,
@@ -373,7 +373,7 @@ void HELPER(vsm_v)(void *vd, void *v0, target_ulong base,
     /* evl = ceil(vl/8) */
     uint8_t evl = (env->vl + 7) >> 3;
     vext_ldst_us(vd, base, env, desc, ste_b,
-                 0, evl, GETPC(), MMU_DATA_STORE);
+                 0, evl, GETPC());
 }
 
 /*
@@ -399,7 +399,7 @@ vext_ldst_index(void *vd, void *v0, target_ulong base,
                 void *vs2, CPURISCVState *env, uint32_t desc,
                 vext_get_index_addr get_index_addr,
                 vext_ldst_elem_fn *ldst_elem,
-                uint32_t esz, uintptr_t ra, MMUAccessType access_type)
+                uint32_t esz, uintptr_t ra)
 {
     uint32_t i, k;
     uint32_t nf = vext_nf(desc);
@@ -427,7 +427,7 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong base,                   \
                   void *vs2, CPURISCVState *env, uint32_t desc)            \
 {                                                                          \
     vext_ldst_index(vd, v0, base, vs2, env, desc, INDEX_FN,                \
-                    LOAD_FN, ctzl(sizeof(ETYPE)), GETPC(), MMU_DATA_LOAD); \
+                    LOAD_FN, ctzl(sizeof(ETYPE)), GETPC());                \
 }
 
 GEN_VEXT_LD_INDEX(vlxei8_8_v,   int8_t,  idx_b, lde_b)
@@ -453,7 +453,7 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong base,         \
 {                                                                \
     vext_ldst_index(vd, v0, base, vs2, env, desc, INDEX_FN,      \
                     STORE_FN, ctzl(sizeof(ETYPE)),               \
-                    GETPC(), MMU_DATA_STORE);                    \
+                    GETPC());                                    \
 }
 
 GEN_VEXT_ST_INDEX(vsxei8_8_v,   int8_t,  idx_b, ste_b)
@@ -576,8 +576,7 @@ GEN_VEXT_LDFF(vle64ff_v, int64_t, lde_d)
  */
 static void
 vext_ldst_whole(void *vd, target_ulong base, CPURISCVState *env, uint32_t desc,
-                vext_ldst_elem_fn *ldst_elem, uint32_t esz, uintptr_t ra,
-                MMUAccessType access_type)
+                vext_ldst_elem_fn *ldst_elem, uint32_t esz, uintptr_t ra)
 {
     uint32_t i, k, off, pos;
     uint32_t nf = vext_nf(desc);
@@ -612,8 +611,7 @@ void HELPER(NAME)(void *vd, target_ulong base,       \
                   CPURISCVState *env, uint32_t desc) \
 {                                                    \
     vext_ldst_whole(vd, base, env, desc, LOAD_FN,    \
-                    ctzl(sizeof(ETYPE)), GETPC(),    \
-                    MMU_DATA_LOAD);                  \
+                    ctzl(sizeof(ETYPE)), GETPC());   \
 }
 
 GEN_VEXT_LD_WHOLE(vl1re8_v,  int8_t,  lde_b)
@@ -638,8 +636,7 @@ void HELPER(NAME)(void *vd, target_ulong base,       \
                   CPURISCVState *env, uint32_t desc) \
 {                                                    \
     vext_ldst_whole(vd, base, env, desc, STORE_FN,   \
-                    ctzl(sizeof(ETYPE)), GETPC(),    \
-                    MMU_DATA_STORE);                 \
+                    ctzl(sizeof(ETYPE)), GETPC());   \
 }
 
 GEN_VEXT_ST_WHOLE(vs1r_v, int8_t, ste_b)

From c7b8a4213b0dba710e31d633e17e661151c3d23a Mon Sep 17 00:00:00 2001
From: eopXD <yueh.ting.chen@gmail.com>
Date: Mon, 6 Jun 2022 06:16:16 +0000
Subject: [PATCH 920/935] target/riscv: rvv: Rename ambiguous esz

No functional change intended in this commit.

Signed-off-by: eop Chen <eop.chen@sifive.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <165449614532.19704.7000832880482980398-3@git.sr.ht>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/vector_helper.c | 76 ++++++++++++++++++------------------
 1 file changed, 38 insertions(+), 38 deletions(-)

diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c
index 60840325c4..3b79b9cbc2 100644
--- a/target/riscv/vector_helper.c
+++ b/target/riscv/vector_helper.c
@@ -125,9 +125,9 @@ static inline int32_t vext_lmul(uint32_t desc)
 /*
  * Get the maximum number of elements can be operated.
  *
- * esz: log2 of element size in bytes.
+ * log2_esz: log2 of element size in bytes.
  */
-static inline uint32_t vext_max_elems(uint32_t desc, uint32_t esz)
+static inline uint32_t vext_max_elems(uint32_t desc, uint32_t log2_esz)
 {
     /*
      * As simd_desc support at most 2048 bytes, the max vlen is 1024 bits.
@@ -136,7 +136,7 @@ static inline uint32_t vext_max_elems(uint32_t desc, uint32_t esz)
     uint32_t vlenb = simd_maxsz(desc);
 
     /* Return VLMAX */
-    int scale = vext_lmul(desc) - esz;
+    int scale = vext_lmul(desc) - log2_esz;
     return scale < 0 ? vlenb >> -scale : vlenb << scale;
 }
 
@@ -231,11 +231,11 @@ vext_ldst_stride(void *vd, void *v0, target_ulong base,
                  target_ulong stride, CPURISCVState *env,
                  uint32_t desc, uint32_t vm,
                  vext_ldst_elem_fn *ldst_elem,
-                 uint32_t esz, uintptr_t ra)
+                 uint32_t log2_esz, uintptr_t ra)
 {
     uint32_t i, k;
     uint32_t nf = vext_nf(desc);
-    uint32_t max_elems = vext_max_elems(desc, esz);
+    uint32_t max_elems = vext_max_elems(desc, log2_esz);
 
     for (i = env->vstart; i < env->vl; i++, env->vstart++) {
         if (!vm && !vext_elem_mask(v0, i)) {
@@ -244,7 +244,7 @@ vext_ldst_stride(void *vd, void *v0, target_ulong base,
 
         k = 0;
         while (k < nf) {
-            target_ulong addr = base + stride * i + (k << esz);
+            target_ulong addr = base + stride * i + (k << log2_esz);
             ldst_elem(env, adjust_addr(env, addr), i + k * max_elems, vd, ra);
             k++;
         }
@@ -289,18 +289,18 @@ GEN_VEXT_ST_STRIDE(vsse64_v, int64_t, ste_d)
 /* unmasked unit-stride load and store operation*/
 static void
 vext_ldst_us(void *vd, target_ulong base, CPURISCVState *env, uint32_t desc,
-             vext_ldst_elem_fn *ldst_elem, uint32_t esz, uint32_t evl,
+             vext_ldst_elem_fn *ldst_elem, uint32_t log2_esz, uint32_t evl,
              uintptr_t ra)
 {
     uint32_t i, k;
     uint32_t nf = vext_nf(desc);
-    uint32_t max_elems = vext_max_elems(desc, esz);
+    uint32_t max_elems = vext_max_elems(desc, log2_esz);
 
     /* load bytes from guest memory */
     for (i = env->vstart; i < evl; i++, env->vstart++) {
         k = 0;
         while (k < nf) {
-            target_ulong addr = base + ((i * nf + k) << esz);
+            target_ulong addr = base + ((i * nf + k) << log2_esz);
             ldst_elem(env, adjust_addr(env, addr), i + k * max_elems, vd, ra);
             k++;
         }
@@ -399,12 +399,12 @@ vext_ldst_index(void *vd, void *v0, target_ulong base,
                 void *vs2, CPURISCVState *env, uint32_t desc,
                 vext_get_index_addr get_index_addr,
                 vext_ldst_elem_fn *ldst_elem,
-                uint32_t esz, uintptr_t ra)
+                uint32_t log2_esz, uintptr_t ra)
 {
     uint32_t i, k;
     uint32_t nf = vext_nf(desc);
     uint32_t vm = vext_vm(desc);
-    uint32_t max_elems = vext_max_elems(desc, esz);
+    uint32_t max_elems = vext_max_elems(desc, log2_esz);
 
     /* load bytes from guest memory */
     for (i = env->vstart; i < env->vl; i++, env->vstart++) {
@@ -414,7 +414,7 @@ vext_ldst_index(void *vd, void *v0, target_ulong base,
 
         k = 0;
         while (k < nf) {
-            abi_ptr addr = get_index_addr(base, i, vs2) + (k << esz);
+            abi_ptr addr = get_index_addr(base, i, vs2) + (k << log2_esz);
             ldst_elem(env, adjust_addr(env, addr), i + k * max_elems, vd, ra);
             k++;
         }
@@ -480,13 +480,13 @@ static inline void
 vext_ldff(void *vd, void *v0, target_ulong base,
           CPURISCVState *env, uint32_t desc,
           vext_ldst_elem_fn *ldst_elem,
-          uint32_t esz, uintptr_t ra)
+          uint32_t log2_esz, uintptr_t ra)
 {
     void *host;
     uint32_t i, k, vl = 0;
     uint32_t nf = vext_nf(desc);
     uint32_t vm = vext_vm(desc);
-    uint32_t max_elems = vext_max_elems(desc, esz);
+    uint32_t max_elems = vext_max_elems(desc, log2_esz);
     target_ulong addr, offset, remain;
 
     /* probe every access*/
@@ -494,12 +494,12 @@ vext_ldff(void *vd, void *v0, target_ulong base,
         if (!vm && !vext_elem_mask(v0, i)) {
             continue;
         }
-        addr = adjust_addr(env, base + i * (nf << esz));
+        addr = adjust_addr(env, base + i * (nf << log2_esz));
         if (i == 0) {
-            probe_pages(env, addr, nf << esz, ra, MMU_DATA_LOAD);
+            probe_pages(env, addr, nf << log2_esz, ra, MMU_DATA_LOAD);
         } else {
             /* if it triggers an exception, no need to check watchpoint */
-            remain = nf << esz;
+            remain = nf << log2_esz;
             while (remain > 0) {
                 offset = -(addr | TARGET_PAGE_MASK);
                 host = tlb_vaddr_to_host(env, addr, MMU_DATA_LOAD,
@@ -536,7 +536,7 @@ ProbeSuccess:
             continue;
         }
         while (k < nf) {
-            target_ulong addr = base + ((i * nf + k) << esz);
+            target_ulong addr = base + ((i * nf + k) << log2_esz);
             ldst_elem(env, adjust_addr(env, addr), i + k * max_elems, vd, ra);
             k++;
         }
@@ -576,12 +576,12 @@ GEN_VEXT_LDFF(vle64ff_v, int64_t, lde_d)
  */
 static void
 vext_ldst_whole(void *vd, target_ulong base, CPURISCVState *env, uint32_t desc,
-                vext_ldst_elem_fn *ldst_elem, uint32_t esz, uintptr_t ra)
+                vext_ldst_elem_fn *ldst_elem, uint32_t log2_esz, uintptr_t ra)
 {
     uint32_t i, k, off, pos;
     uint32_t nf = vext_nf(desc);
     uint32_t vlenb = env_archcpu(env)->cfg.vlen >> 3;
-    uint32_t max_elems = vlenb >> esz;
+    uint32_t max_elems = vlenb >> log2_esz;
 
     k = env->vstart / max_elems;
     off = env->vstart % max_elems;
@@ -589,7 +589,7 @@ vext_ldst_whole(void *vd, target_ulong base, CPURISCVState *env, uint32_t desc,
     if (off) {
         /* load/store rest of elements of current segment pointed by vstart */
         for (pos = off; pos < max_elems; pos++, env->vstart++) {
-            target_ulong addr = base + ((pos + k * max_elems) << esz);
+            target_ulong addr = base + ((pos + k * max_elems) << log2_esz);
             ldst_elem(env, adjust_addr(env, addr), pos + k * max_elems, vd, ra);
         }
         k++;
@@ -598,7 +598,7 @@ vext_ldst_whole(void *vd, target_ulong base, CPURISCVState *env, uint32_t desc,
     /* load/store elements for rest of segments */
     for (; k < nf; k++) {
         for (i = 0; i < max_elems; i++, env->vstart++) {
-            target_ulong addr = base + ((i + k * max_elems) << esz);
+            target_ulong addr = base + ((i + k * max_elems) << log2_esz);
             ldst_elem(env, adjust_addr(env, addr), i + k * max_elems, vd, ra);
         }
     }
@@ -4688,11 +4688,11 @@ GEN_VEXT_VSLIDEDOWN_VX(vslidedown_vx_h, uint16_t, H2)
 GEN_VEXT_VSLIDEDOWN_VX(vslidedown_vx_w, uint32_t, H4)
 GEN_VEXT_VSLIDEDOWN_VX(vslidedown_vx_d, uint64_t, H8)
 
-#define GEN_VEXT_VSLIE1UP(ESZ, H)                                           \
-static void vslide1up_##ESZ(void *vd, void *v0, target_ulong s1, void *vs2, \
-                     CPURISCVState *env, uint32_t desc)                     \
+#define GEN_VEXT_VSLIE1UP(BITWIDTH, H)                                      \
+static void vslide1up_##BITWIDTH(void *vd, void *v0, target_ulong s1,       \
+                     void *vs2, CPURISCVState *env, uint32_t desc)          \
 {                                                                           \
-    typedef uint##ESZ##_t ETYPE;                                            \
+    typedef uint##BITWIDTH##_t ETYPE;                                       \
     uint32_t vm = vext_vm(desc);                                            \
     uint32_t vl = env->vl;                                                  \
     uint32_t i;                                                             \
@@ -4715,11 +4715,11 @@ GEN_VEXT_VSLIE1UP(16, H2)
 GEN_VEXT_VSLIE1UP(32, H4)
 GEN_VEXT_VSLIE1UP(64, H8)
 
-#define GEN_VEXT_VSLIDE1UP_VX(NAME, ESZ)                          \
+#define GEN_VEXT_VSLIDE1UP_VX(NAME, BITWIDTH)                     \
 void HELPER(NAME)(void *vd, void *v0, target_ulong s1, void *vs2, \
                   CPURISCVState *env, uint32_t desc)              \
 {                                                                 \
-    vslide1up_##ESZ(vd, v0, s1, vs2, env, desc);                  \
+    vslide1up_##BITWIDTH(vd, v0, s1, vs2, env, desc);             \
 }
 
 /* vslide1up.vx vd, vs2, rs1, vm # vd[0]=x[rs1], vd[i+1] = vs2[i] */
@@ -4728,11 +4728,11 @@ GEN_VEXT_VSLIDE1UP_VX(vslide1up_vx_h, 16)
 GEN_VEXT_VSLIDE1UP_VX(vslide1up_vx_w, 32)
 GEN_VEXT_VSLIDE1UP_VX(vslide1up_vx_d, 64)
 
-#define GEN_VEXT_VSLIDE1DOWN(ESZ, H)                                          \
-static void vslide1down_##ESZ(void *vd, void *v0, target_ulong s1, void *vs2, \
-                       CPURISCVState *env, uint32_t desc)                     \
+#define GEN_VEXT_VSLIDE1DOWN(BITWIDTH, H)                                     \
+static void vslide1down_##BITWIDTH(void *vd, void *v0, target_ulong s1,       \
+                       void *vs2, CPURISCVState *env, uint32_t desc)          \
 {                                                                             \
-    typedef uint##ESZ##_t ETYPE;                                              \
+    typedef uint##BITWIDTH##_t ETYPE;                                         \
     uint32_t vm = vext_vm(desc);                                              \
     uint32_t vl = env->vl;                                                    \
     uint32_t i;                                                               \
@@ -4755,11 +4755,11 @@ GEN_VEXT_VSLIDE1DOWN(16, H2)
 GEN_VEXT_VSLIDE1DOWN(32, H4)
 GEN_VEXT_VSLIDE1DOWN(64, H8)
 
-#define GEN_VEXT_VSLIDE1DOWN_VX(NAME, ESZ)                        \
+#define GEN_VEXT_VSLIDE1DOWN_VX(NAME, BITWIDTH)                   \
 void HELPER(NAME)(void *vd, void *v0, target_ulong s1, void *vs2, \
                   CPURISCVState *env, uint32_t desc)              \
 {                                                                 \
-    vslide1down_##ESZ(vd, v0, s1, vs2, env, desc);                \
+    vslide1down_##BITWIDTH(vd, v0, s1, vs2, env, desc);           \
 }
 
 /* vslide1down.vx vd, vs2, rs1, vm # vd[i] = vs2[i+1], vd[vl-1]=x[rs1] */
@@ -4769,11 +4769,11 @@ GEN_VEXT_VSLIDE1DOWN_VX(vslide1down_vx_w, 32)
 GEN_VEXT_VSLIDE1DOWN_VX(vslide1down_vx_d, 64)
 
 /* Vector Floating-Point Slide Instructions */
-#define GEN_VEXT_VFSLIDE1UP_VF(NAME, ESZ)                     \
+#define GEN_VEXT_VFSLIDE1UP_VF(NAME, BITWIDTH)                \
 void HELPER(NAME)(void *vd, void *v0, uint64_t s1, void *vs2, \
                   CPURISCVState *env, uint32_t desc)          \
 {                                                             \
-    vslide1up_##ESZ(vd, v0, s1, vs2, env, desc);              \
+    vslide1up_##BITWIDTH(vd, v0, s1, vs2, env, desc);         \
 }
 
 /* vfslide1up.vf vd, vs2, rs1, vm # vd[0]=f[rs1], vd[i+1] = vs2[i] */
@@ -4781,11 +4781,11 @@ GEN_VEXT_VFSLIDE1UP_VF(vfslide1up_vf_h, 16)
 GEN_VEXT_VFSLIDE1UP_VF(vfslide1up_vf_w, 32)
 GEN_VEXT_VFSLIDE1UP_VF(vfslide1up_vf_d, 64)
 
-#define GEN_VEXT_VFSLIDE1DOWN_VF(NAME, ESZ)                   \
+#define GEN_VEXT_VFSLIDE1DOWN_VF(NAME, BITWIDTH)              \
 void HELPER(NAME)(void *vd, void *v0, uint64_t s1, void *vs2, \
                   CPURISCVState *env, uint32_t desc)          \
 {                                                             \
-    vslide1down_##ESZ(vd, v0, s1, vs2, env, desc);            \
+    vslide1down_##BITWIDTH(vd, v0, s1, vs2, env, desc);       \
 }
 
 /* vfslide1down.vf vd, vs2, rs1, vm # vd[i] = vs2[i+1], vd[vl-1]=f[rs1] */

From 41d3d7f76aa7060c0cbc1c8b3a9767a5997b231a Mon Sep 17 00:00:00 2001
From: eopXD <yueh.ting.chen@gmail.com>
Date: Mon, 6 Jun 2022 06:16:16 +0000
Subject: [PATCH 921/935] target/riscv: rvv: Early exit when vstart >= vl
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

According to v-spec (section 5.4):
When vstart ≥ vl, there are no body elements, and no elements are
updated in any destination vector register group, including that
no tail elements are updated with agnostic values.

vmsbf.m, vmsif.m, vmsof.m, viota.m, vcompress instructions themselves
require vstart to be zero. So they don't need the early exit.

Signed-off-by: eop Chen <eop.chen@sifive.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <165449614532.19704.7000832880482980398-4@git.sr.ht>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/insn_trans/trans_rvv.c.inc | 27 +++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/target/riscv/insn_trans/trans_rvv.c.inc b/target/riscv/insn_trans/trans_rvv.c.inc
index 391c61fe93..6750f5d04a 100644
--- a/target/riscv/insn_trans/trans_rvv.c.inc
+++ b/target/riscv/insn_trans/trans_rvv.c.inc
@@ -652,6 +652,7 @@ static bool ldst_us_trans(uint32_t vd, uint32_t rs1, uint32_t data,
 
     TCGLabel *over = gen_new_label();
     tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
+    tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
     dest = tcg_temp_new_ptr();
     mask = tcg_temp_new_ptr();
@@ -818,6 +819,7 @@ static bool ldst_stride_trans(uint32_t vd, uint32_t rs1, uint32_t rs2,
 
     TCGLabel *over = gen_new_label();
     tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
+    tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
     dest = tcg_temp_new_ptr();
     mask = tcg_temp_new_ptr();
@@ -925,6 +927,7 @@ static bool ldst_index_trans(uint32_t vd, uint32_t rs1, uint32_t vs2,
 
     TCGLabel *over = gen_new_label();
     tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
+    tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
     dest = tcg_temp_new_ptr();
     mask = tcg_temp_new_ptr();
@@ -1067,6 +1070,7 @@ static bool ldff_trans(uint32_t vd, uint32_t rs1, uint32_t data,
 
     TCGLabel *over = gen_new_label();
     tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
+    tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
     dest = tcg_temp_new_ptr();
     mask = tcg_temp_new_ptr();
@@ -1225,6 +1229,7 @@ do_opivv_gvec(DisasContext *s, arg_rmrr *a, GVecGen3Fn *gvec_fn,
     }
 
     tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
+    tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
     if (a->vm && s->vl_eq_vlmax) {
         gvec_fn(s->sew, vreg_ofs(s, a->rd),
@@ -1272,6 +1277,7 @@ static bool opivx_trans(uint32_t vd, uint32_t rs1, uint32_t vs2, uint32_t vm,
 
     TCGLabel *over = gen_new_label();
     tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
+    tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
     dest = tcg_temp_new_ptr();
     mask = tcg_temp_new_ptr();
@@ -1436,6 +1442,7 @@ static bool opivi_trans(uint32_t vd, uint32_t imm, uint32_t vs2, uint32_t vm,
 
     TCGLabel *over = gen_new_label();
     tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
+    tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
     dest = tcg_temp_new_ptr();
     mask = tcg_temp_new_ptr();
@@ -1522,6 +1529,7 @@ static bool do_opivv_widen(DisasContext *s, arg_rmrr *a,
         uint32_t data = 0;
         TCGLabel *over = gen_new_label();
         tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
+        tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
         data = FIELD_DP32(data, VDATA, VM, a->vm);
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);
@@ -1602,6 +1610,7 @@ static bool do_opiwv_widen(DisasContext *s, arg_rmrr *a,
         uint32_t data = 0;
         TCGLabel *over = gen_new_label();
         tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
+        tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
         data = FIELD_DP32(data, VDATA, VM, a->vm);
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);
@@ -1679,6 +1688,7 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a)             \
         };                                                         \
         TCGLabel *over = gen_new_label();                          \
         tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);          \
+        tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \
                                                                    \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                 \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);             \
@@ -1860,6 +1870,7 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a)             \
         };                                                         \
         TCGLabel *over = gen_new_label();                          \
         tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);          \
+        tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \
                                                                    \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                 \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);             \
@@ -2070,6 +2081,7 @@ static bool trans_vmv_v_v(DisasContext *s, arg_vmv_v_v *a)
             };
             TCGLabel *over = gen_new_label();
             tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
+            tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
             tcg_gen_gvec_2_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, a->rs1),
                                cpu_env, s->cfg_ptr->vlen / 8,
@@ -2093,6 +2105,7 @@ static bool trans_vmv_v_x(DisasContext *s, arg_vmv_v_x *a)
         TCGv s1;
         TCGLabel *over = gen_new_label();
         tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
+        tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
         s1 = get_gpr(s, a->rs1, EXT_SIGN);
 
@@ -2148,6 +2161,7 @@ static bool trans_vmv_v_i(DisasContext *s, arg_vmv_v_i *a)
             };
             TCGLabel *over = gen_new_label();
             tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
+            tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
             s1 = tcg_constant_i64(simm);
             dest = tcg_temp_new_ptr();
@@ -2300,6 +2314,7 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a)             \
         TCGLabel *over = gen_new_label();                          \
         gen_set_rm(s, RISCV_FRM_DYN);                              \
         tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);          \
+        tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \
                                                                    \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                 \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);             \
@@ -2330,6 +2345,7 @@ static bool opfvf_trans(uint32_t vd, uint32_t rs1, uint32_t vs2,
 
     TCGLabel *over = gen_new_label();
     tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
+    tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
     dest = tcg_temp_new_ptr();
     mask = tcg_temp_new_ptr();
@@ -2418,6 +2434,7 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a)           \
         TCGLabel *over = gen_new_label();                        \
         gen_set_rm(s, RISCV_FRM_DYN);                            \
         tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);        \
+        tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);\
                                                                  \
         data = FIELD_DP32(data, VDATA, VM, a->vm);               \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);           \
@@ -2492,6 +2509,7 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a)             \
         TCGLabel *over = gen_new_label();                          \
         gen_set_rm(s, RISCV_FRM_DYN);                              \
         tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);          \
+        tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \
                                                                    \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                 \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);             \
@@ -2613,6 +2631,7 @@ static bool do_opfv(DisasContext *s, arg_rmr *a,
         TCGLabel *over = gen_new_label();
         gen_set_rm(s, rm);
         tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
+        tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
         data = FIELD_DP32(data, VDATA, VM, a->vm);
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);
@@ -2726,6 +2745,7 @@ static bool trans_vfmv_v_f(DisasContext *s, arg_vfmv_v_f *a)
             };
             TCGLabel *over = gen_new_label();
             tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
+            tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
             t1 = tcg_temp_new_i64();
             /* NaN-box f[rs1] */
@@ -2814,6 +2834,7 @@ static bool trans_##NAME(DisasContext *s, arg_rmr *a)              \
         TCGLabel *over = gen_new_label();                          \
         gen_set_rm(s, FRM);                                        \
         tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);          \
+        tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \
                                                                    \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                 \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);             \
@@ -2865,6 +2886,7 @@ static bool trans_##NAME(DisasContext *s, arg_rmr *a)              \
         TCGLabel *over = gen_new_label();                          \
         gen_set_rm(s, RISCV_FRM_DYN);                              \
         tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);          \
+        tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \
                                                                    \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                 \
         tcg_gen_gvec_3_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),     \
@@ -2930,6 +2952,7 @@ static bool trans_##NAME(DisasContext *s, arg_rmr *a)              \
         TCGLabel *over = gen_new_label();                          \
         gen_set_rm(s, FRM);                                        \
         tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);          \
+        tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \
                                                                    \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                 \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);             \
@@ -2983,6 +3006,7 @@ static bool trans_##NAME(DisasContext *s, arg_rmr *a)              \
         TCGLabel *over = gen_new_label();                          \
         gen_set_rm(s, FRM);                                        \
         tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);          \
+        tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \
                                                                    \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                 \
         tcg_gen_gvec_3_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),     \
@@ -3070,6 +3094,7 @@ static bool trans_##NAME(DisasContext *s, arg_r *a)                \
         gen_helper_gvec_4_ptr *fn = gen_helper_##NAME;             \
         TCGLabel *over = gen_new_label();                          \
         tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);          \
+        tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \
                                                                    \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);             \
         tcg_gen_gvec_4_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),     \
@@ -3238,6 +3263,7 @@ static bool trans_vid_v(DisasContext *s, arg_vid_v *a)
         uint32_t data = 0;
         TCGLabel *over = gen_new_label();
         tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
+        tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
         data = FIELD_DP32(data, VDATA, VM, a->vm);
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);
@@ -3748,6 +3774,7 @@ static bool int_ext_op(DisasContext *s, arg_rmr *a, uint8_t seq)
     gen_helper_gvec_3_ptr *fn;
     TCGLabel *over = gen_new_label();
     tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
+    tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
     static gen_helper_gvec_3_ptr * const fns[6][4] = {
         {

From f1eed927fb3a1212af8e324cf242cf6f4bd6fd04 Mon Sep 17 00:00:00 2001
From: eopXD <eop.chen@sifive.com>
Date: Mon, 6 Jun 2022 06:16:16 +0000
Subject: [PATCH 922/935] target/riscv: rvv: Add tail agnostic for vv
 instructions

According to v-spec, tail agnostic behavior can be either kept as
undisturbed or set elements' bits to all 1s. To distinguish the
difference of tail policies, QEMU should be able to simulate the tail
agnostic behavior as "set tail elements' bits to all 1s".

There are multiple possibility for agnostic elements according to
v-spec. The main intent of this patch-set tries to add option that
can distinguish between tail policies. Setting agnostic elements to
all 1s allows QEMU to express this.

This is the first commit regarding the optional tail agnostic
behavior. Follow-up commits will add this optional behavior
for all rvv instructions.

Signed-off-by: eop Chen <eop.chen@sifive.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <165449614532.19704.7000832880482980398-5@git.sr.ht>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.h                      |   2 +
 target/riscv/cpu_helper.c               |   2 +
 target/riscv/insn_trans/trans_rvv.c.inc |   3 +-
 target/riscv/internals.h                |   5 +-
 target/riscv/translate.c                |   2 +
 target/riscv/vector_helper.c            | 296 +++++++++++++-----------
 6 files changed, 178 insertions(+), 132 deletions(-)

diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
index 194a58d760..7d6397acdf 100644
--- a/target/riscv/cpu.h
+++ b/target/riscv/cpu.h
@@ -412,6 +412,7 @@ struct RISCVCPUConfig {
     bool ext_zve32f;
     bool ext_zve64f;
     bool ext_zmmul;
+    bool rvv_ta_all_1s;
 
     uint32_t mvendorid;
     uint64_t marchid;
@@ -567,6 +568,7 @@ FIELD(TB_FLAGS, XL, 20, 2)
 /* If PointerMasking should be applied */
 FIELD(TB_FLAGS, PM_MASK_ENABLED, 22, 1)
 FIELD(TB_FLAGS, PM_BASE_ENABLED, 23, 1)
+FIELD(TB_FLAGS, VTA, 24, 1)
 
 #ifdef TARGET_RISCV32
 #define riscv_cpu_mxl(env)  ((void)(env), MXL_RV32)
diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index 16c6045459..4a6700c890 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -65,6 +65,8 @@ void cpu_get_tb_cpu_state(CPURISCVState *env, target_ulong *pc,
         flags = FIELD_DP32(flags, TB_FLAGS, LMUL,
                     FIELD_EX64(env->vtype, VTYPE, VLMUL));
         flags = FIELD_DP32(flags, TB_FLAGS, VL_EQ_VLMAX, vl_eq_vlmax);
+        flags = FIELD_DP32(flags, TB_FLAGS, VTA,
+                    FIELD_EX64(env->vtype, VTYPE, VTA));
     } else {
         flags = FIELD_DP32(flags, TB_FLAGS, VILL, 1);
     }
diff --git a/target/riscv/insn_trans/trans_rvv.c.inc b/target/riscv/insn_trans/trans_rvv.c.inc
index 6750f5d04a..bf9886a93d 100644
--- a/target/riscv/insn_trans/trans_rvv.c.inc
+++ b/target/riscv/insn_trans/trans_rvv.c.inc
@@ -1231,7 +1231,7 @@ do_opivv_gvec(DisasContext *s, arg_rmrr *a, GVecGen3Fn *gvec_fn,
     tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
     tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
 
-    if (a->vm && s->vl_eq_vlmax) {
+    if (a->vm && s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) {
         gvec_fn(s->sew, vreg_ofs(s, a->rd),
                 vreg_ofs(s, a->rs2), vreg_ofs(s, a->rs1),
                 MAXSZ(s), MAXSZ(s));
@@ -1240,6 +1240,7 @@ do_opivv_gvec(DisasContext *s, arg_rmrr *a, GVecGen3Fn *gvec_fn,
 
         data = FIELD_DP32(data, VDATA, VM, a->vm);
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);
         tcg_gen_gvec_4_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),
                            vreg_ofs(s, a->rs1), vreg_ofs(s, a->rs2),
                            cpu_env, s->cfg_ptr->vlen / 8,
diff --git a/target/riscv/internals.h b/target/riscv/internals.h
index dbb322bfa7..512c6c30cf 100644
--- a/target/riscv/internals.h
+++ b/target/riscv/internals.h
@@ -24,8 +24,9 @@
 /* share data between vector helpers and decode code */
 FIELD(VDATA, VM, 0, 1)
 FIELD(VDATA, LMUL, 1, 3)
-FIELD(VDATA, NF, 4, 4)
-FIELD(VDATA, WD, 4, 1)
+FIELD(VDATA, VTA, 4, 1)
+FIELD(VDATA, NF, 5, 4)
+FIELD(VDATA, WD, 5, 1)
 
 /* float point classify helpers */
 target_ulong fclass_h(uint64_t frs1);
diff --git a/target/riscv/translate.c b/target/riscv/translate.c
index 55a4713af2..59f0ee9a50 100644
--- a/target/riscv/translate.c
+++ b/target/riscv/translate.c
@@ -94,6 +94,7 @@ typedef struct DisasContext {
      */
     int8_t lmul;
     uint8_t sew;
+    uint8_t vta;
     target_ulong vstart;
     bool vl_eq_vlmax;
     uint8_t ntemp;
@@ -1099,6 +1100,7 @@ static void riscv_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cs)
     ctx->vill = FIELD_EX32(tb_flags, TB_FLAGS, VILL);
     ctx->sew = FIELD_EX32(tb_flags, TB_FLAGS, SEW);
     ctx->lmul = sextract32(FIELD_EX32(tb_flags, TB_FLAGS, LMUL), 0, 3);
+    ctx->vta = FIELD_EX32(tb_flags, TB_FLAGS, VTA) && cpu->cfg.rvv_ta_all_1s;
     ctx->vstart = env->vstart;
     ctx->vl_eq_vlmax = FIELD_EX32(tb_flags, TB_FLAGS, VL_EQ_VLMAX);
     ctx->misa_mxl_max = env->misa_mxl_max;
diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c
index 3b79b9cbc2..e2a2979bad 100644
--- a/target/riscv/vector_helper.c
+++ b/target/riscv/vector_helper.c
@@ -122,6 +122,11 @@ static inline int32_t vext_lmul(uint32_t desc)
     return sextract32(FIELD_EX32(simd_data(desc), VDATA, LMUL), 0, 3);
 }
 
+static inline uint32_t vext_vta(uint32_t desc)
+{
+    return FIELD_EX32(simd_data(desc), VDATA, VTA);
+}
+
 /*
  * Get the maximum number of elements can be operated.
  *
@@ -140,6 +145,21 @@ static inline uint32_t vext_max_elems(uint32_t desc, uint32_t log2_esz)
     return scale < 0 ? vlenb >> -scale : vlenb << scale;
 }
 
+/*
+ * Get number of total elements, including prestart, body and tail elements.
+ * Note that when LMUL < 1, the tail includes the elements past VLMAX that
+ * are held in the same vector register.
+ */
+static inline uint32_t vext_get_total_elems(CPURISCVState *env, uint32_t desc,
+                                            uint32_t esz)
+{
+    uint32_t vlenb = simd_maxsz(desc);
+    uint32_t sew = 1 << FIELD_EX64(env->vtype, VTYPE, VSEW);
+    int8_t emul = ctzl(esz) - ctzl(sew) + vext_lmul(desc) < 0 ? 0 :
+                  ctzl(esz) - ctzl(sew) + vext_lmul(desc);
+    return (vlenb << emul) / esz;
+}
+
 static inline target_ulong adjust_addr(CPURISCVState *env, target_ulong addr)
 {
     return (addr & env->cur_pmmask) | env->cur_pmbase;
@@ -172,6 +192,20 @@ static void probe_pages(CPURISCVState *env, target_ulong addr,
     }
 }
 
+/* set agnostic elements to 1s */
+static void vext_set_elems_1s(void *base, uint32_t is_agnostic, uint32_t cnt,
+                              uint32_t tot)
+{
+    if (is_agnostic == 0) {
+        /* policy undisturbed */
+        return;
+    }
+    if (tot - cnt == 0) {
+        return ;
+    }
+    memset(base + cnt, -1, tot - cnt);
+}
+
 static inline void vext_set_elem_mask(void *v0, int index,
                                       uint8_t value)
 {
@@ -707,10 +741,12 @@ RVVCALL(OPIVV2, vsub_vv_d, OP_SSS_D, H8, H8, H8, DO_SUB)
 
 static void do_vext_vv(void *vd, void *v0, void *vs1, void *vs2,
                        CPURISCVState *env, uint32_t desc,
-                       opivv2_fn *fn)
+                       opivv2_fn *fn, uint32_t esz)
 {
     uint32_t vm = vext_vm(desc);
     uint32_t vl = env->vl;
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);
+    uint32_t vta = vext_vta(desc);
     uint32_t i;
 
     for (i = env->vstart; i < vl; i++) {
@@ -720,26 +756,28 @@ static void do_vext_vv(void *vd, void *v0, void *vs1, void *vs2,
         fn(vd, vs1, vs2, i);
     }
     env->vstart = 0;
+    /* set tail elements to 1s */
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);
 }
 
 /* generate the helpers for OPIVV */
-#define GEN_VEXT_VV(NAME)                                 \
+#define GEN_VEXT_VV(NAME, ESZ)                            \
 void HELPER(NAME)(void *vd, void *v0, void *vs1,          \
                   void *vs2, CPURISCVState *env,          \
                   uint32_t desc)                          \
 {                                                         \
     do_vext_vv(vd, v0, vs1, vs2, env, desc,               \
-               do_##NAME);                                \
+               do_##NAME, ESZ);                           \
 }
 
-GEN_VEXT_VV(vadd_vv_b)
-GEN_VEXT_VV(vadd_vv_h)
-GEN_VEXT_VV(vadd_vv_w)
-GEN_VEXT_VV(vadd_vv_d)
-GEN_VEXT_VV(vsub_vv_b)
-GEN_VEXT_VV(vsub_vv_h)
-GEN_VEXT_VV(vsub_vv_w)
-GEN_VEXT_VV(vsub_vv_d)
+GEN_VEXT_VV(vadd_vv_b, 1)
+GEN_VEXT_VV(vadd_vv_h, 2)
+GEN_VEXT_VV(vadd_vv_w, 4)
+GEN_VEXT_VV(vadd_vv_d, 8)
+GEN_VEXT_VV(vsub_vv_b, 1)
+GEN_VEXT_VV(vsub_vv_h, 2)
+GEN_VEXT_VV(vsub_vv_w, 4)
+GEN_VEXT_VV(vsub_vv_d, 8)
 
 typedef void opivx2_fn(void *vd, target_long s1, void *vs2, int i);
 
@@ -884,30 +922,30 @@ RVVCALL(OPIVV2, vwadd_wv_w, WOP_WSSS_W, H8, H4, H4, DO_ADD)
 RVVCALL(OPIVV2, vwsub_wv_b, WOP_WSSS_B, H2, H1, H1, DO_SUB)
 RVVCALL(OPIVV2, vwsub_wv_h, WOP_WSSS_H, H4, H2, H2, DO_SUB)
 RVVCALL(OPIVV2, vwsub_wv_w, WOP_WSSS_W, H8, H4, H4, DO_SUB)
-GEN_VEXT_VV(vwaddu_vv_b)
-GEN_VEXT_VV(vwaddu_vv_h)
-GEN_VEXT_VV(vwaddu_vv_w)
-GEN_VEXT_VV(vwsubu_vv_b)
-GEN_VEXT_VV(vwsubu_vv_h)
-GEN_VEXT_VV(vwsubu_vv_w)
-GEN_VEXT_VV(vwadd_vv_b)
-GEN_VEXT_VV(vwadd_vv_h)
-GEN_VEXT_VV(vwadd_vv_w)
-GEN_VEXT_VV(vwsub_vv_b)
-GEN_VEXT_VV(vwsub_vv_h)
-GEN_VEXT_VV(vwsub_vv_w)
-GEN_VEXT_VV(vwaddu_wv_b)
-GEN_VEXT_VV(vwaddu_wv_h)
-GEN_VEXT_VV(vwaddu_wv_w)
-GEN_VEXT_VV(vwsubu_wv_b)
-GEN_VEXT_VV(vwsubu_wv_h)
-GEN_VEXT_VV(vwsubu_wv_w)
-GEN_VEXT_VV(vwadd_wv_b)
-GEN_VEXT_VV(vwadd_wv_h)
-GEN_VEXT_VV(vwadd_wv_w)
-GEN_VEXT_VV(vwsub_wv_b)
-GEN_VEXT_VV(vwsub_wv_h)
-GEN_VEXT_VV(vwsub_wv_w)
+GEN_VEXT_VV(vwaddu_vv_b, 2)
+GEN_VEXT_VV(vwaddu_vv_h, 4)
+GEN_VEXT_VV(vwaddu_vv_w, 8)
+GEN_VEXT_VV(vwsubu_vv_b, 2)
+GEN_VEXT_VV(vwsubu_vv_h, 4)
+GEN_VEXT_VV(vwsubu_vv_w, 8)
+GEN_VEXT_VV(vwadd_vv_b, 2)
+GEN_VEXT_VV(vwadd_vv_h, 4)
+GEN_VEXT_VV(vwadd_vv_w, 8)
+GEN_VEXT_VV(vwsub_vv_b, 2)
+GEN_VEXT_VV(vwsub_vv_h, 4)
+GEN_VEXT_VV(vwsub_vv_w, 8)
+GEN_VEXT_VV(vwaddu_wv_b, 2)
+GEN_VEXT_VV(vwaddu_wv_h, 4)
+GEN_VEXT_VV(vwaddu_wv_w, 8)
+GEN_VEXT_VV(vwsubu_wv_b, 2)
+GEN_VEXT_VV(vwsubu_wv_h, 4)
+GEN_VEXT_VV(vwsubu_wv_w, 8)
+GEN_VEXT_VV(vwadd_wv_b, 2)
+GEN_VEXT_VV(vwadd_wv_h, 4)
+GEN_VEXT_VV(vwadd_wv_w, 8)
+GEN_VEXT_VV(vwsub_wv_b, 2)
+GEN_VEXT_VV(vwsub_wv_h, 4)
+GEN_VEXT_VV(vwsub_wv_w, 8)
 
 RVVCALL(OPIVX2, vwaddu_vx_b, WOP_UUU_B, H2, H1, DO_ADD)
 RVVCALL(OPIVX2, vwaddu_vx_h, WOP_UUU_H, H4, H2, DO_ADD)
@@ -1086,18 +1124,18 @@ RVVCALL(OPIVV2, vxor_vv_b, OP_SSS_B, H1, H1, H1, DO_XOR)
 RVVCALL(OPIVV2, vxor_vv_h, OP_SSS_H, H2, H2, H2, DO_XOR)
 RVVCALL(OPIVV2, vxor_vv_w, OP_SSS_W, H4, H4, H4, DO_XOR)
 RVVCALL(OPIVV2, vxor_vv_d, OP_SSS_D, H8, H8, H8, DO_XOR)
-GEN_VEXT_VV(vand_vv_b)
-GEN_VEXT_VV(vand_vv_h)
-GEN_VEXT_VV(vand_vv_w)
-GEN_VEXT_VV(vand_vv_d)
-GEN_VEXT_VV(vor_vv_b)
-GEN_VEXT_VV(vor_vv_h)
-GEN_VEXT_VV(vor_vv_w)
-GEN_VEXT_VV(vor_vv_d)
-GEN_VEXT_VV(vxor_vv_b)
-GEN_VEXT_VV(vxor_vv_h)
-GEN_VEXT_VV(vxor_vv_w)
-GEN_VEXT_VV(vxor_vv_d)
+GEN_VEXT_VV(vand_vv_b, 1)
+GEN_VEXT_VV(vand_vv_h, 2)
+GEN_VEXT_VV(vand_vv_w, 4)
+GEN_VEXT_VV(vand_vv_d, 8)
+GEN_VEXT_VV(vor_vv_b, 1)
+GEN_VEXT_VV(vor_vv_h, 2)
+GEN_VEXT_VV(vor_vv_w, 4)
+GEN_VEXT_VV(vor_vv_d, 8)
+GEN_VEXT_VV(vxor_vv_b, 1)
+GEN_VEXT_VV(vxor_vv_h, 2)
+GEN_VEXT_VV(vxor_vv_w, 4)
+GEN_VEXT_VV(vxor_vv_d, 8)
 
 RVVCALL(OPIVX2, vand_vx_b, OP_SSS_B, H1, H1, DO_AND)
 RVVCALL(OPIVX2, vand_vx_h, OP_SSS_H, H2, H2, DO_AND)
@@ -1343,22 +1381,22 @@ RVVCALL(OPIVV2, vmax_vv_b, OP_SSS_B, H1, H1, H1, DO_MAX)
 RVVCALL(OPIVV2, vmax_vv_h, OP_SSS_H, H2, H2, H2, DO_MAX)
 RVVCALL(OPIVV2, vmax_vv_w, OP_SSS_W, H4, H4, H4, DO_MAX)
 RVVCALL(OPIVV2, vmax_vv_d, OP_SSS_D, H8, H8, H8, DO_MAX)
-GEN_VEXT_VV(vminu_vv_b)
-GEN_VEXT_VV(vminu_vv_h)
-GEN_VEXT_VV(vminu_vv_w)
-GEN_VEXT_VV(vminu_vv_d)
-GEN_VEXT_VV(vmin_vv_b)
-GEN_VEXT_VV(vmin_vv_h)
-GEN_VEXT_VV(vmin_vv_w)
-GEN_VEXT_VV(vmin_vv_d)
-GEN_VEXT_VV(vmaxu_vv_b)
-GEN_VEXT_VV(vmaxu_vv_h)
-GEN_VEXT_VV(vmaxu_vv_w)
-GEN_VEXT_VV(vmaxu_vv_d)
-GEN_VEXT_VV(vmax_vv_b)
-GEN_VEXT_VV(vmax_vv_h)
-GEN_VEXT_VV(vmax_vv_w)
-GEN_VEXT_VV(vmax_vv_d)
+GEN_VEXT_VV(vminu_vv_b, 1)
+GEN_VEXT_VV(vminu_vv_h, 2)
+GEN_VEXT_VV(vminu_vv_w, 4)
+GEN_VEXT_VV(vminu_vv_d, 8)
+GEN_VEXT_VV(vmin_vv_b, 1)
+GEN_VEXT_VV(vmin_vv_h, 2)
+GEN_VEXT_VV(vmin_vv_w, 4)
+GEN_VEXT_VV(vmin_vv_d, 8)
+GEN_VEXT_VV(vmaxu_vv_b, 1)
+GEN_VEXT_VV(vmaxu_vv_h, 2)
+GEN_VEXT_VV(vmaxu_vv_w, 4)
+GEN_VEXT_VV(vmaxu_vv_d, 8)
+GEN_VEXT_VV(vmax_vv_b, 1)
+GEN_VEXT_VV(vmax_vv_h, 2)
+GEN_VEXT_VV(vmax_vv_w, 4)
+GEN_VEXT_VV(vmax_vv_d, 8)
 
 RVVCALL(OPIVX2, vminu_vx_b, OP_UUU_B, H1, H1, DO_MIN)
 RVVCALL(OPIVX2, vminu_vx_h, OP_UUU_H, H2, H2, DO_MIN)
@@ -1399,10 +1437,10 @@ RVVCALL(OPIVV2, vmul_vv_b, OP_SSS_B, H1, H1, H1, DO_MUL)
 RVVCALL(OPIVV2, vmul_vv_h, OP_SSS_H, H2, H2, H2, DO_MUL)
 RVVCALL(OPIVV2, vmul_vv_w, OP_SSS_W, H4, H4, H4, DO_MUL)
 RVVCALL(OPIVV2, vmul_vv_d, OP_SSS_D, H8, H8, H8, DO_MUL)
-GEN_VEXT_VV(vmul_vv_b)
-GEN_VEXT_VV(vmul_vv_h)
-GEN_VEXT_VV(vmul_vv_w)
-GEN_VEXT_VV(vmul_vv_d)
+GEN_VEXT_VV(vmul_vv_b, 1)
+GEN_VEXT_VV(vmul_vv_h, 2)
+GEN_VEXT_VV(vmul_vv_w, 4)
+GEN_VEXT_VV(vmul_vv_d, 8)
 
 static int8_t do_mulh_b(int8_t s2, int8_t s1)
 {
@@ -1506,18 +1544,18 @@ RVVCALL(OPIVV2, vmulhsu_vv_b, OP_SUS_B, H1, H1, H1, do_mulhsu_b)
 RVVCALL(OPIVV2, vmulhsu_vv_h, OP_SUS_H, H2, H2, H2, do_mulhsu_h)
 RVVCALL(OPIVV2, vmulhsu_vv_w, OP_SUS_W, H4, H4, H4, do_mulhsu_w)
 RVVCALL(OPIVV2, vmulhsu_vv_d, OP_SUS_D, H8, H8, H8, do_mulhsu_d)
-GEN_VEXT_VV(vmulh_vv_b)
-GEN_VEXT_VV(vmulh_vv_h)
-GEN_VEXT_VV(vmulh_vv_w)
-GEN_VEXT_VV(vmulh_vv_d)
-GEN_VEXT_VV(vmulhu_vv_b)
-GEN_VEXT_VV(vmulhu_vv_h)
-GEN_VEXT_VV(vmulhu_vv_w)
-GEN_VEXT_VV(vmulhu_vv_d)
-GEN_VEXT_VV(vmulhsu_vv_b)
-GEN_VEXT_VV(vmulhsu_vv_h)
-GEN_VEXT_VV(vmulhsu_vv_w)
-GEN_VEXT_VV(vmulhsu_vv_d)
+GEN_VEXT_VV(vmulh_vv_b, 1)
+GEN_VEXT_VV(vmulh_vv_h, 2)
+GEN_VEXT_VV(vmulh_vv_w, 4)
+GEN_VEXT_VV(vmulh_vv_d, 8)
+GEN_VEXT_VV(vmulhu_vv_b, 1)
+GEN_VEXT_VV(vmulhu_vv_h, 2)
+GEN_VEXT_VV(vmulhu_vv_w, 4)
+GEN_VEXT_VV(vmulhu_vv_d, 8)
+GEN_VEXT_VV(vmulhsu_vv_b, 1)
+GEN_VEXT_VV(vmulhsu_vv_h, 2)
+GEN_VEXT_VV(vmulhsu_vv_w, 4)
+GEN_VEXT_VV(vmulhsu_vv_d, 8)
 
 RVVCALL(OPIVX2, vmul_vx_b, OP_SSS_B, H1, H1, DO_MUL)
 RVVCALL(OPIVX2, vmul_vx_h, OP_SSS_H, H2, H2, DO_MUL)
@@ -1576,22 +1614,22 @@ RVVCALL(OPIVV2, vrem_vv_b, OP_SSS_B, H1, H1, H1, DO_REM)
 RVVCALL(OPIVV2, vrem_vv_h, OP_SSS_H, H2, H2, H2, DO_REM)
 RVVCALL(OPIVV2, vrem_vv_w, OP_SSS_W, H4, H4, H4, DO_REM)
 RVVCALL(OPIVV2, vrem_vv_d, OP_SSS_D, H8, H8, H8, DO_REM)
-GEN_VEXT_VV(vdivu_vv_b)
-GEN_VEXT_VV(vdivu_vv_h)
-GEN_VEXT_VV(vdivu_vv_w)
-GEN_VEXT_VV(vdivu_vv_d)
-GEN_VEXT_VV(vdiv_vv_b)
-GEN_VEXT_VV(vdiv_vv_h)
-GEN_VEXT_VV(vdiv_vv_w)
-GEN_VEXT_VV(vdiv_vv_d)
-GEN_VEXT_VV(vremu_vv_b)
-GEN_VEXT_VV(vremu_vv_h)
-GEN_VEXT_VV(vremu_vv_w)
-GEN_VEXT_VV(vremu_vv_d)
-GEN_VEXT_VV(vrem_vv_b)
-GEN_VEXT_VV(vrem_vv_h)
-GEN_VEXT_VV(vrem_vv_w)
-GEN_VEXT_VV(vrem_vv_d)
+GEN_VEXT_VV(vdivu_vv_b, 1)
+GEN_VEXT_VV(vdivu_vv_h, 2)
+GEN_VEXT_VV(vdivu_vv_w, 4)
+GEN_VEXT_VV(vdivu_vv_d, 8)
+GEN_VEXT_VV(vdiv_vv_b, 1)
+GEN_VEXT_VV(vdiv_vv_h, 2)
+GEN_VEXT_VV(vdiv_vv_w, 4)
+GEN_VEXT_VV(vdiv_vv_d, 8)
+GEN_VEXT_VV(vremu_vv_b, 1)
+GEN_VEXT_VV(vremu_vv_h, 2)
+GEN_VEXT_VV(vremu_vv_w, 4)
+GEN_VEXT_VV(vremu_vv_d, 8)
+GEN_VEXT_VV(vrem_vv_b, 1)
+GEN_VEXT_VV(vrem_vv_h, 2)
+GEN_VEXT_VV(vrem_vv_w, 4)
+GEN_VEXT_VV(vrem_vv_d, 8)
 
 RVVCALL(OPIVX2, vdivu_vx_b, OP_UUU_B, H1, H1, DO_DIVU)
 RVVCALL(OPIVX2, vdivu_vx_h, OP_UUU_H, H2, H2, DO_DIVU)
@@ -1636,15 +1674,15 @@ RVVCALL(OPIVV2, vwmulu_vv_w, WOP_UUU_W, H8, H4, H4, DO_MUL)
 RVVCALL(OPIVV2, vwmulsu_vv_b, WOP_SUS_B, H2, H1, H1, DO_MUL)
 RVVCALL(OPIVV2, vwmulsu_vv_h, WOP_SUS_H, H4, H2, H2, DO_MUL)
 RVVCALL(OPIVV2, vwmulsu_vv_w, WOP_SUS_W, H8, H4, H4, DO_MUL)
-GEN_VEXT_VV(vwmul_vv_b)
-GEN_VEXT_VV(vwmul_vv_h)
-GEN_VEXT_VV(vwmul_vv_w)
-GEN_VEXT_VV(vwmulu_vv_b)
-GEN_VEXT_VV(vwmulu_vv_h)
-GEN_VEXT_VV(vwmulu_vv_w)
-GEN_VEXT_VV(vwmulsu_vv_b)
-GEN_VEXT_VV(vwmulsu_vv_h)
-GEN_VEXT_VV(vwmulsu_vv_w)
+GEN_VEXT_VV(vwmul_vv_b, 2)
+GEN_VEXT_VV(vwmul_vv_h, 4)
+GEN_VEXT_VV(vwmul_vv_w, 8)
+GEN_VEXT_VV(vwmulu_vv_b, 2)
+GEN_VEXT_VV(vwmulu_vv_h, 4)
+GEN_VEXT_VV(vwmulu_vv_w, 8)
+GEN_VEXT_VV(vwmulsu_vv_b, 2)
+GEN_VEXT_VV(vwmulsu_vv_h, 4)
+GEN_VEXT_VV(vwmulsu_vv_w, 8)
 
 RVVCALL(OPIVX2, vwmul_vx_b, WOP_SSS_B, H2, H1, DO_MUL)
 RVVCALL(OPIVX2, vwmul_vx_h, WOP_SSS_H, H4, H2, DO_MUL)
@@ -1695,22 +1733,22 @@ RVVCALL(OPIVV3, vnmsub_vv_b, OP_SSS_B, H1, H1, H1, DO_NMSUB)
 RVVCALL(OPIVV3, vnmsub_vv_h, OP_SSS_H, H2, H2, H2, DO_NMSUB)
 RVVCALL(OPIVV3, vnmsub_vv_w, OP_SSS_W, H4, H4, H4, DO_NMSUB)
 RVVCALL(OPIVV3, vnmsub_vv_d, OP_SSS_D, H8, H8, H8, DO_NMSUB)
-GEN_VEXT_VV(vmacc_vv_b)
-GEN_VEXT_VV(vmacc_vv_h)
-GEN_VEXT_VV(vmacc_vv_w)
-GEN_VEXT_VV(vmacc_vv_d)
-GEN_VEXT_VV(vnmsac_vv_b)
-GEN_VEXT_VV(vnmsac_vv_h)
-GEN_VEXT_VV(vnmsac_vv_w)
-GEN_VEXT_VV(vnmsac_vv_d)
-GEN_VEXT_VV(vmadd_vv_b)
-GEN_VEXT_VV(vmadd_vv_h)
-GEN_VEXT_VV(vmadd_vv_w)
-GEN_VEXT_VV(vmadd_vv_d)
-GEN_VEXT_VV(vnmsub_vv_b)
-GEN_VEXT_VV(vnmsub_vv_h)
-GEN_VEXT_VV(vnmsub_vv_w)
-GEN_VEXT_VV(vnmsub_vv_d)
+GEN_VEXT_VV(vmacc_vv_b, 1)
+GEN_VEXT_VV(vmacc_vv_h, 2)
+GEN_VEXT_VV(vmacc_vv_w, 4)
+GEN_VEXT_VV(vmacc_vv_d, 8)
+GEN_VEXT_VV(vnmsac_vv_b, 1)
+GEN_VEXT_VV(vnmsac_vv_h, 2)
+GEN_VEXT_VV(vnmsac_vv_w, 4)
+GEN_VEXT_VV(vnmsac_vv_d, 8)
+GEN_VEXT_VV(vmadd_vv_b, 1)
+GEN_VEXT_VV(vmadd_vv_h, 2)
+GEN_VEXT_VV(vmadd_vv_w, 4)
+GEN_VEXT_VV(vmadd_vv_d, 8)
+GEN_VEXT_VV(vnmsub_vv_b, 1)
+GEN_VEXT_VV(vnmsub_vv_h, 2)
+GEN_VEXT_VV(vnmsub_vv_w, 4)
+GEN_VEXT_VV(vnmsub_vv_d, 8)
 
 #define OPIVX3(NAME, TD, T1, T2, TX1, TX2, HD, HS2, OP)             \
 static void do_##NAME(void *vd, target_long s1, void *vs2, int i)   \
@@ -1763,15 +1801,15 @@ RVVCALL(OPIVV3, vwmacc_vv_w, WOP_SSS_W, H8, H4, H4, DO_MACC)
 RVVCALL(OPIVV3, vwmaccsu_vv_b, WOP_SSU_B, H2, H1, H1, DO_MACC)
 RVVCALL(OPIVV3, vwmaccsu_vv_h, WOP_SSU_H, H4, H2, H2, DO_MACC)
 RVVCALL(OPIVV3, vwmaccsu_vv_w, WOP_SSU_W, H8, H4, H4, DO_MACC)
-GEN_VEXT_VV(vwmaccu_vv_b)
-GEN_VEXT_VV(vwmaccu_vv_h)
-GEN_VEXT_VV(vwmaccu_vv_w)
-GEN_VEXT_VV(vwmacc_vv_b)
-GEN_VEXT_VV(vwmacc_vv_h)
-GEN_VEXT_VV(vwmacc_vv_w)
-GEN_VEXT_VV(vwmaccsu_vv_b)
-GEN_VEXT_VV(vwmaccsu_vv_h)
-GEN_VEXT_VV(vwmaccsu_vv_w)
+GEN_VEXT_VV(vwmaccu_vv_b, 2)
+GEN_VEXT_VV(vwmaccu_vv_h, 4)
+GEN_VEXT_VV(vwmaccu_vv_w, 8)
+GEN_VEXT_VV(vwmacc_vv_b, 2)
+GEN_VEXT_VV(vwmacc_vv_h, 4)
+GEN_VEXT_VV(vwmacc_vv_w, 8)
+GEN_VEXT_VV(vwmaccsu_vv_b, 2)
+GEN_VEXT_VV(vwmaccsu_vv_h, 4)
+GEN_VEXT_VV(vwmaccsu_vv_w, 8)
 
 RVVCALL(OPIVX3, vwmaccu_vx_b, WOP_UUU_B, H2, H1, DO_MACC)
 RVVCALL(OPIVX3, vwmaccu_vx_h, WOP_UUU_H, H4, H2, DO_MACC)

From 752614cab8e61bb6ba96cee1ec127eba6c35398e Mon Sep 17 00:00:00 2001
From: eopXD <yueh.ting.chen@gmail.com>
Date: Mon, 6 Jun 2022 06:16:16 +0000
Subject: [PATCH 923/935] target/riscv: rvv: Add tail agnostic for vector load
 / store instructions

Destination register of unit-stride mask load and store instructions are
always written with a tail-agnostic policy.

A vector segment load / store instruction may contain fractional lmul
with nf * lmul > 1. The rest of the elements in the last register should
be treated as tail elements.

Signed-off-by: eop Chen <eop.chen@sifive.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <165449614532.19704.7000832880482980398-6@git.sr.ht>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/insn_trans/trans_rvv.c.inc |  6 +++
 target/riscv/translate.c                |  2 +
 target/riscv/vector_helper.c            | 60 +++++++++++++++++++++++++
 3 files changed, 68 insertions(+)

diff --git a/target/riscv/insn_trans/trans_rvv.c.inc b/target/riscv/insn_trans/trans_rvv.c.inc
index bf9886a93d..cd73fd6119 100644
--- a/target/riscv/insn_trans/trans_rvv.c.inc
+++ b/target/riscv/insn_trans/trans_rvv.c.inc
@@ -711,6 +711,7 @@ static bool ld_us_op(DisasContext *s, arg_r2nfvm *a, uint8_t eew)
     data = FIELD_DP32(data, VDATA, VM, a->vm);
     data = FIELD_DP32(data, VDATA, LMUL, emul);
     data = FIELD_DP32(data, VDATA, NF, a->nf);
+    data = FIELD_DP32(data, VDATA, VTA, s->vta);
     return ldst_us_trans(a->rd, a->rs1, data, fn, s, false);
 }
 
@@ -774,6 +775,8 @@ static bool ld_us_mask_op(DisasContext *s, arg_vlm_v *a, uint8_t eew)
     /* EMUL = 1, NFIELDS = 1 */
     data = FIELD_DP32(data, VDATA, LMUL, 0);
     data = FIELD_DP32(data, VDATA, NF, 1);
+    /* Mask destination register are always tail-agnostic */
+    data = FIELD_DP32(data, VDATA, VTA, s->cfg_vta_all_1s);
     return ldst_us_trans(a->rd, a->rs1, data, fn, s, false);
 }
 
@@ -862,6 +865,7 @@ static bool ld_stride_op(DisasContext *s, arg_rnfvm *a, uint8_t eew)
     data = FIELD_DP32(data, VDATA, VM, a->vm);
     data = FIELD_DP32(data, VDATA, LMUL, emul);
     data = FIELD_DP32(data, VDATA, NF, a->nf);
+    data = FIELD_DP32(data, VDATA, VTA, s->vta);
     return ldst_stride_trans(a->rd, a->rs1, a->rs2, data, fn, s, false);
 }
 
@@ -991,6 +995,7 @@ static bool ld_index_op(DisasContext *s, arg_rnfvm *a, uint8_t eew)
     data = FIELD_DP32(data, VDATA, VM, a->vm);
     data = FIELD_DP32(data, VDATA, LMUL, emul);
     data = FIELD_DP32(data, VDATA, NF, a->nf);
+    data = FIELD_DP32(data, VDATA, VTA, s->vta);
     return ldst_index_trans(a->rd, a->rs1, a->rs2, data, fn, s, false);
 }
 
@@ -1108,6 +1113,7 @@ static bool ldff_op(DisasContext *s, arg_r2nfvm *a, uint8_t eew)
     data = FIELD_DP32(data, VDATA, VM, a->vm);
     data = FIELD_DP32(data, VDATA, LMUL, emul);
     data = FIELD_DP32(data, VDATA, NF, a->nf);
+    data = FIELD_DP32(data, VDATA, VTA, s->vta);
     return ldff_trans(a->rd, a->rs1, data, fn, s);
 }
 
diff --git a/target/riscv/translate.c b/target/riscv/translate.c
index 59f0ee9a50..b151c20674 100644
--- a/target/riscv/translate.c
+++ b/target/riscv/translate.c
@@ -95,6 +95,7 @@ typedef struct DisasContext {
     int8_t lmul;
     uint8_t sew;
     uint8_t vta;
+    bool cfg_vta_all_1s;
     target_ulong vstart;
     bool vl_eq_vlmax;
     uint8_t ntemp;
@@ -1101,6 +1102,7 @@ static void riscv_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cs)
     ctx->sew = FIELD_EX32(tb_flags, TB_FLAGS, SEW);
     ctx->lmul = sextract32(FIELD_EX32(tb_flags, TB_FLAGS, LMUL), 0, 3);
     ctx->vta = FIELD_EX32(tb_flags, TB_FLAGS, VTA) && cpu->cfg.rvv_ta_all_1s;
+    ctx->cfg_vta_all_1s = cpu->cfg.rvv_ta_all_1s;
     ctx->vstart = env->vstart;
     ctx->vl_eq_vlmax = FIELD_EX32(tb_flags, TB_FLAGS, VL_EQ_VLMAX);
     ctx->misa_mxl_max = env->misa_mxl_max;
diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c
index e2a2979bad..ee28e1b92d 100644
--- a/target/riscv/vector_helper.c
+++ b/target/riscv/vector_helper.c
@@ -270,6 +270,9 @@ vext_ldst_stride(void *vd, void *v0, target_ulong base,
     uint32_t i, k;
     uint32_t nf = vext_nf(desc);
     uint32_t max_elems = vext_max_elems(desc, log2_esz);
+    uint32_t esz = 1 << log2_esz;
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);
+    uint32_t vta = vext_vta(desc);
 
     for (i = env->vstart; i < env->vl; i++, env->vstart++) {
         if (!vm && !vext_elem_mask(v0, i)) {
@@ -284,6 +287,18 @@ vext_ldst_stride(void *vd, void *v0, target_ulong base,
         }
     }
     env->vstart = 0;
+    /* set tail elements to 1s */
+    for (k = 0; k < nf; ++k) {
+        vext_set_elems_1s(vd, vta, (k * max_elems + env->vl) * esz,
+                          (k * max_elems + max_elems) * esz);
+    }
+    if (nf * max_elems % total_elems != 0) {
+        uint32_t vlenb = env_archcpu(env)->cfg.vlen >> 3;
+        uint32_t registers_used =
+            ((nf * max_elems) * esz + (vlenb - 1)) / vlenb;
+        vext_set_elems_1s(vd, vta, (nf * max_elems) * esz,
+                          registers_used * vlenb);
+    }
 }
 
 #define GEN_VEXT_LD_STRIDE(NAME, ETYPE, LOAD_FN)                        \
@@ -329,6 +344,9 @@ vext_ldst_us(void *vd, target_ulong base, CPURISCVState *env, uint32_t desc,
     uint32_t i, k;
     uint32_t nf = vext_nf(desc);
     uint32_t max_elems = vext_max_elems(desc, log2_esz);
+    uint32_t esz = 1 << log2_esz;
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);
+    uint32_t vta = vext_vta(desc);
 
     /* load bytes from guest memory */
     for (i = env->vstart; i < evl; i++, env->vstart++) {
@@ -340,6 +358,18 @@ vext_ldst_us(void *vd, target_ulong base, CPURISCVState *env, uint32_t desc,
         }
     }
     env->vstart = 0;
+    /* set tail elements to 1s */
+    for (k = 0; k < nf; ++k) {
+        vext_set_elems_1s(vd, vta, (k * max_elems + evl) * esz,
+                          (k * max_elems + max_elems) * esz);
+    }
+    if (nf * max_elems % total_elems != 0) {
+        uint32_t vlenb = env_archcpu(env)->cfg.vlen >> 3;
+        uint32_t registers_used =
+            ((nf * max_elems) * esz + (vlenb - 1)) / vlenb;
+        vext_set_elems_1s(vd, vta, (nf * max_elems) * esz,
+                          registers_used * vlenb);
+    }
 }
 
 /*
@@ -439,6 +469,9 @@ vext_ldst_index(void *vd, void *v0, target_ulong base,
     uint32_t nf = vext_nf(desc);
     uint32_t vm = vext_vm(desc);
     uint32_t max_elems = vext_max_elems(desc, log2_esz);
+    uint32_t esz = 1 << log2_esz;
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);
+    uint32_t vta = vext_vta(desc);
 
     /* load bytes from guest memory */
     for (i = env->vstart; i < env->vl; i++, env->vstart++) {
@@ -454,6 +487,18 @@ vext_ldst_index(void *vd, void *v0, target_ulong base,
         }
     }
     env->vstart = 0;
+    /* set tail elements to 1s */
+    for (k = 0; k < nf; ++k) {
+        vext_set_elems_1s(vd, vta, (k * max_elems + env->vl) * esz,
+                          (k * max_elems + max_elems) * esz);
+    }
+    if (nf * max_elems % total_elems != 0) {
+        uint32_t vlenb = env_archcpu(env)->cfg.vlen >> 3;
+        uint32_t registers_used =
+            ((nf * max_elems) * esz + (vlenb - 1)) / vlenb;
+        vext_set_elems_1s(vd, vta, (nf * max_elems) * esz,
+                          registers_used * vlenb);
+    }
 }
 
 #define GEN_VEXT_LD_INDEX(NAME, ETYPE, INDEX_FN, LOAD_FN)                  \
@@ -521,6 +566,9 @@ vext_ldff(void *vd, void *v0, target_ulong base,
     uint32_t nf = vext_nf(desc);
     uint32_t vm = vext_vm(desc);
     uint32_t max_elems = vext_max_elems(desc, log2_esz);
+    uint32_t esz = 1 << log2_esz;
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);
+    uint32_t vta = vext_vta(desc);
     target_ulong addr, offset, remain;
 
     /* probe every access*/
@@ -576,6 +624,18 @@ ProbeSuccess:
         }
     }
     env->vstart = 0;
+    /* set tail elements to 1s */
+    for (k = 0; k < nf; ++k) {
+        vext_set_elems_1s(vd, vta, (k * max_elems + env->vl) * esz,
+                          (k * max_elems + max_elems) * esz);
+    }
+    if (nf * max_elems % total_elems != 0) {
+        uint32_t vlenb = env_archcpu(env)->cfg.vlen >> 3;
+        uint32_t registers_used =
+            ((nf * max_elems) * esz + (vlenb - 1)) / vlenb;
+        vext_set_elems_1s(vd, vta, (nf * max_elems) * esz,
+                          registers_used * vlenb);
+    }
 }
 
 #define GEN_VEXT_LDFF(NAME, ETYPE, LOAD_FN)               \

From 5c19fc156e5cea2516085c487eb72cdb331c54b6 Mon Sep 17 00:00:00 2001
From: eopXD <yueh.ting.chen@gmail.com>
Date: Mon, 6 Jun 2022 06:16:38 +0000
Subject: [PATCH 924/935] target/riscv: rvv: Add tail agnostic for vx, vvm, vxm
 instructions

`vmadc` and `vmsbc` produces a mask value, they always operate with
a tail agnostic policy.

Signed-off-by: eop Chen <eop.chen@sifive.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <165449614532.19704.7000832880482980398-7@git.sr.ht>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/insn_trans/trans_rvv.c.inc |  13 +-
 target/riscv/internals.h                |   5 +-
 target/riscv/vector_helper.c            | 310 +++++++++++++-----------
 3 files changed, 188 insertions(+), 140 deletions(-)

diff --git a/target/riscv/insn_trans/trans_rvv.c.inc b/target/riscv/insn_trans/trans_rvv.c.inc
index cd73fd6119..22edf6228d 100644
--- a/target/riscv/insn_trans/trans_rvv.c.inc
+++ b/target/riscv/insn_trans/trans_rvv.c.inc
@@ -1293,6 +1293,8 @@ static bool opivx_trans(uint32_t vd, uint32_t rs1, uint32_t vs2, uint32_t vm,
 
     data = FIELD_DP32(data, VDATA, VM, vm);
     data = FIELD_DP32(data, VDATA, LMUL, s->lmul);
+    data = FIELD_DP32(data, VDATA, VTA, s->vta);
+    data = FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s);
     desc = tcg_constant_i32(simd_desc(s->cfg_ptr->vlen / 8,
                                       s->cfg_ptr->vlen / 8, data));
 
@@ -1328,7 +1330,7 @@ do_opivx_gvec(DisasContext *s, arg_rmrr *a, GVecGen2sFn *gvec_fn,
         return false;
     }
 
-    if (a->vm && s->vl_eq_vlmax) {
+    if (a->vm && s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) {
         TCGv_i64 src1 = tcg_temp_new_i64();
 
         tcg_gen_ext_tl_i64(src1, get_gpr(s, a->rs1, EXT_SIGN));
@@ -1458,6 +1460,8 @@ static bool opivi_trans(uint32_t vd, uint32_t imm, uint32_t vs2, uint32_t vm,
 
     data = FIELD_DP32(data, VDATA, VM, vm);
     data = FIELD_DP32(data, VDATA, LMUL, s->lmul);
+    data = FIELD_DP32(data, VDATA, VTA, s->vta);
+    data = FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s);
     desc = tcg_constant_i32(simd_desc(s->cfg_ptr->vlen / 8,
                                       s->cfg_ptr->vlen / 8, data));
 
@@ -1486,7 +1490,7 @@ do_opivi_gvec(DisasContext *s, arg_rmrr *a, GVecGen2iFn *gvec_fn,
         return false;
     }
 
-    if (a->vm && s->vl_eq_vlmax) {
+    if (a->vm && s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) {
         gvec_fn(s->sew, vreg_ofs(s, a->rd), vreg_ofs(s, a->rs2),
                 extract_imm(s, a->rs1, imm_mode), MAXSZ(s), MAXSZ(s));
         mark_vs_dirty(s);
@@ -1540,6 +1544,7 @@ static bool do_opivv_widen(DisasContext *s, arg_rmrr *a,
 
         data = FIELD_DP32(data, VDATA, VM, a->vm);
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);
         tcg_gen_gvec_4_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),
                            vreg_ofs(s, a->rs1),
                            vreg_ofs(s, a->rs2),
@@ -1621,6 +1626,7 @@ static bool do_opiwv_widen(DisasContext *s, arg_rmrr *a,
 
         data = FIELD_DP32(data, VDATA, VM, a->vm);
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);
         tcg_gen_gvec_4_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),
                            vreg_ofs(s, a->rs1),
                            vreg_ofs(s, a->rs2),
@@ -1699,6 +1705,9 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a)             \
                                                                    \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                 \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);             \
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);               \
+        data =                                                     \
+            FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s);\
         tcg_gen_gvec_4_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),     \
                            vreg_ofs(s, a->rs1),                    \
                            vreg_ofs(s, a->rs2), cpu_env,           \
diff --git a/target/riscv/internals.h b/target/riscv/internals.h
index 512c6c30cf..193ce57a6d 100644
--- a/target/riscv/internals.h
+++ b/target/riscv/internals.h
@@ -25,8 +25,9 @@
 FIELD(VDATA, VM, 0, 1)
 FIELD(VDATA, LMUL, 1, 3)
 FIELD(VDATA, VTA, 4, 1)
-FIELD(VDATA, NF, 5, 4)
-FIELD(VDATA, WD, 5, 1)
+FIELD(VDATA, VTA_ALL_1S, 5, 1)
+FIELD(VDATA, NF, 6, 4)
+FIELD(VDATA, WD, 6, 1)
 
 /* float point classify helpers */
 target_ulong fclass_h(uint64_t frs1);
diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c
index ee28e1b92d..7cdb5d12af 100644
--- a/target/riscv/vector_helper.c
+++ b/target/riscv/vector_helper.c
@@ -127,6 +127,11 @@ static inline uint32_t vext_vta(uint32_t desc)
     return FIELD_EX32(simd_data(desc), VDATA, VTA);
 }
 
+static inline uint32_t vext_vta_all_1s(uint32_t desc)
+{
+    return FIELD_EX32(simd_data(desc), VDATA, VTA_ALL_1S);
+}
+
 /*
  * Get the maximum number of elements can be operated.
  *
@@ -867,10 +872,12 @@ RVVCALL(OPIVX2, vrsub_vx_d, OP_SSS_D, H8, H8, DO_RSUB)
 
 static void do_vext_vx(void *vd, void *v0, target_long s1, void *vs2,
                        CPURISCVState *env, uint32_t desc,
-                       opivx2_fn fn)
+                       opivx2_fn fn, uint32_t esz)
 {
     uint32_t vm = vext_vm(desc);
     uint32_t vl = env->vl;
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);
+    uint32_t vta = vext_vta(desc);
     uint32_t i;
 
     for (i = env->vstart; i < vl; i++) {
@@ -880,30 +887,32 @@ static void do_vext_vx(void *vd, void *v0, target_long s1, void *vs2,
         fn(vd, s1, vs2, i);
     }
     env->vstart = 0;
+    /* set tail elements to 1s */
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);
 }
 
 /* generate the helpers for OPIVX */
-#define GEN_VEXT_VX(NAME)                                 \
+#define GEN_VEXT_VX(NAME, ESZ)                            \
 void HELPER(NAME)(void *vd, void *v0, target_ulong s1,    \
                   void *vs2, CPURISCVState *env,          \
                   uint32_t desc)                          \
 {                                                         \
     do_vext_vx(vd, v0, s1, vs2, env, desc,                \
-               do_##NAME);                                \
+               do_##NAME, ESZ);                           \
 }
 
-GEN_VEXT_VX(vadd_vx_b)
-GEN_VEXT_VX(vadd_vx_h)
-GEN_VEXT_VX(vadd_vx_w)
-GEN_VEXT_VX(vadd_vx_d)
-GEN_VEXT_VX(vsub_vx_b)
-GEN_VEXT_VX(vsub_vx_h)
-GEN_VEXT_VX(vsub_vx_w)
-GEN_VEXT_VX(vsub_vx_d)
-GEN_VEXT_VX(vrsub_vx_b)
-GEN_VEXT_VX(vrsub_vx_h)
-GEN_VEXT_VX(vrsub_vx_w)
-GEN_VEXT_VX(vrsub_vx_d)
+GEN_VEXT_VX(vadd_vx_b, 1)
+GEN_VEXT_VX(vadd_vx_h, 2)
+GEN_VEXT_VX(vadd_vx_w, 4)
+GEN_VEXT_VX(vadd_vx_d, 8)
+GEN_VEXT_VX(vsub_vx_b, 1)
+GEN_VEXT_VX(vsub_vx_h, 2)
+GEN_VEXT_VX(vsub_vx_w, 4)
+GEN_VEXT_VX(vsub_vx_d, 8)
+GEN_VEXT_VX(vrsub_vx_b, 1)
+GEN_VEXT_VX(vrsub_vx_h, 2)
+GEN_VEXT_VX(vrsub_vx_w, 4)
+GEN_VEXT_VX(vrsub_vx_d, 8)
 
 void HELPER(vec_rsubs8)(void *d, void *a, uint64_t b, uint32_t desc)
 {
@@ -1031,30 +1040,30 @@ RVVCALL(OPIVX2, vwadd_wx_w, WOP_WSSS_W, H8, H4, DO_ADD)
 RVVCALL(OPIVX2, vwsub_wx_b, WOP_WSSS_B, H2, H1, DO_SUB)
 RVVCALL(OPIVX2, vwsub_wx_h, WOP_WSSS_H, H4, H2, DO_SUB)
 RVVCALL(OPIVX2, vwsub_wx_w, WOP_WSSS_W, H8, H4, DO_SUB)
-GEN_VEXT_VX(vwaddu_vx_b)
-GEN_VEXT_VX(vwaddu_vx_h)
-GEN_VEXT_VX(vwaddu_vx_w)
-GEN_VEXT_VX(vwsubu_vx_b)
-GEN_VEXT_VX(vwsubu_vx_h)
-GEN_VEXT_VX(vwsubu_vx_w)
-GEN_VEXT_VX(vwadd_vx_b)
-GEN_VEXT_VX(vwadd_vx_h)
-GEN_VEXT_VX(vwadd_vx_w)
-GEN_VEXT_VX(vwsub_vx_b)
-GEN_VEXT_VX(vwsub_vx_h)
-GEN_VEXT_VX(vwsub_vx_w)
-GEN_VEXT_VX(vwaddu_wx_b)
-GEN_VEXT_VX(vwaddu_wx_h)
-GEN_VEXT_VX(vwaddu_wx_w)
-GEN_VEXT_VX(vwsubu_wx_b)
-GEN_VEXT_VX(vwsubu_wx_h)
-GEN_VEXT_VX(vwsubu_wx_w)
-GEN_VEXT_VX(vwadd_wx_b)
-GEN_VEXT_VX(vwadd_wx_h)
-GEN_VEXT_VX(vwadd_wx_w)
-GEN_VEXT_VX(vwsub_wx_b)
-GEN_VEXT_VX(vwsub_wx_h)
-GEN_VEXT_VX(vwsub_wx_w)
+GEN_VEXT_VX(vwaddu_vx_b, 2)
+GEN_VEXT_VX(vwaddu_vx_h, 4)
+GEN_VEXT_VX(vwaddu_vx_w, 8)
+GEN_VEXT_VX(vwsubu_vx_b, 2)
+GEN_VEXT_VX(vwsubu_vx_h, 4)
+GEN_VEXT_VX(vwsubu_vx_w, 8)
+GEN_VEXT_VX(vwadd_vx_b, 2)
+GEN_VEXT_VX(vwadd_vx_h, 4)
+GEN_VEXT_VX(vwadd_vx_w, 8)
+GEN_VEXT_VX(vwsub_vx_b, 2)
+GEN_VEXT_VX(vwsub_vx_h, 4)
+GEN_VEXT_VX(vwsub_vx_w, 8)
+GEN_VEXT_VX(vwaddu_wx_b, 2)
+GEN_VEXT_VX(vwaddu_wx_h, 4)
+GEN_VEXT_VX(vwaddu_wx_w, 8)
+GEN_VEXT_VX(vwsubu_wx_b, 2)
+GEN_VEXT_VX(vwsubu_wx_h, 4)
+GEN_VEXT_VX(vwsubu_wx_w, 8)
+GEN_VEXT_VX(vwadd_wx_b, 2)
+GEN_VEXT_VX(vwadd_wx_h, 4)
+GEN_VEXT_VX(vwadd_wx_w, 8)
+GEN_VEXT_VX(vwsub_wx_b, 2)
+GEN_VEXT_VX(vwsub_wx_h, 4)
+GEN_VEXT_VX(vwsub_wx_w, 8)
 
 /* Vector Integer Add-with-Carry / Subtract-with-Borrow Instructions */
 #define DO_VADC(N, M, C) (N + M + C)
@@ -1065,6 +1074,10 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1, void *vs2,   \
                   CPURISCVState *env, uint32_t desc)          \
 {                                                             \
     uint32_t vl = env->vl;                                    \
+    uint32_t esz = sizeof(ETYPE);                             \
+    uint32_t total_elems =                                    \
+        vext_get_total_elems(env, desc, esz);                 \
+    uint32_t vta = vext_vta(desc);                            \
     uint32_t i;                                               \
                                                               \
     for (i = env->vstart; i < vl; i++) {                      \
@@ -1075,6 +1088,8 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1, void *vs2,   \
         *((ETYPE *)vd + H(i)) = DO_OP(s2, s1, carry);         \
     }                                                         \
     env->vstart = 0;                                          \
+    /* set tail elements to 1s */                             \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);  \
 }
 
 GEN_VEXT_VADC_VVM(vadc_vvm_b, uint8_t,  H1, DO_VADC)
@@ -1092,6 +1107,9 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong s1, void *vs2,        \
                   CPURISCVState *env, uint32_t desc)                     \
 {                                                                        \
     uint32_t vl = env->vl;                                               \
+    uint32_t esz = sizeof(ETYPE);                                        \
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);         \
+    uint32_t vta = vext_vta(desc);                                       \
     uint32_t i;                                                          \
                                                                          \
     for (i = env->vstart; i < vl; i++) {                                 \
@@ -1101,6 +1119,8 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong s1, void *vs2,        \
         *((ETYPE *)vd + H(i)) = DO_OP(s2, (ETYPE)(target_long)s1, carry);\
     }                                                                    \
     env->vstart = 0;                                          \
+    /* set tail elements to 1s */                                        \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);             \
 }
 
 GEN_VEXT_VADC_VXM(vadc_vxm_b, uint8_t,  H1, DO_VADC)
@@ -1123,6 +1143,8 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1, void *vs2,   \
 {                                                             \
     uint32_t vl = env->vl;                                    \
     uint32_t vm = vext_vm(desc);                              \
+    uint32_t total_elems = env_archcpu(env)->cfg.vlen;        \
+    uint32_t vta_all_1s = vext_vta_all_1s(desc);              \
     uint32_t i;                                               \
                                                               \
     for (i = env->vstart; i < vl; i++) {                      \
@@ -1132,6 +1154,13 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1, void *vs2,   \
         vext_set_elem_mask(vd, i, DO_OP(s2, s1, carry));      \
     }                                                         \
     env->vstart = 0;                                          \
+    /* mask destination register are always tail-agnostic */  \
+    /* set tail elements to 1s */                             \
+    if (vta_all_1s) {                                         \
+        for (; i < total_elems; i++) {                        \
+            vext_set_elem_mask(vd, i, 1);                     \
+        }                                                     \
+    }                                                         \
 }
 
 GEN_VEXT_VMADC_VVM(vmadc_vvm_b, uint8_t,  H1, DO_MADC)
@@ -1150,6 +1179,8 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong s1,          \
 {                                                               \
     uint32_t vl = env->vl;                                      \
     uint32_t vm = vext_vm(desc);                                \
+    uint32_t total_elems = env_archcpu(env)->cfg.vlen;          \
+    uint32_t vta_all_1s = vext_vta_all_1s(desc);                \
     uint32_t i;                                                 \
                                                                 \
     for (i = env->vstart; i < vl; i++) {                        \
@@ -1159,6 +1190,13 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong s1,          \
                 DO_OP(s2, (ETYPE)(target_long)s1, carry));      \
     }                                                           \
     env->vstart = 0;                                            \
+    /* mask destination register are always tail-agnostic */    \
+    /* set tail elements to 1s */                               \
+    if (vta_all_1s) {                                           \
+        for (; i < total_elems; i++) {                          \
+            vext_set_elem_mask(vd, i, 1);                       \
+        }                                                       \
+    }                                                           \
 }
 
 GEN_VEXT_VMADC_VXM(vmadc_vxm_b, uint8_t,  H1, DO_MADC)
@@ -1209,18 +1247,18 @@ RVVCALL(OPIVX2, vxor_vx_b, OP_SSS_B, H1, H1, DO_XOR)
 RVVCALL(OPIVX2, vxor_vx_h, OP_SSS_H, H2, H2, DO_XOR)
 RVVCALL(OPIVX2, vxor_vx_w, OP_SSS_W, H4, H4, DO_XOR)
 RVVCALL(OPIVX2, vxor_vx_d, OP_SSS_D, H8, H8, DO_XOR)
-GEN_VEXT_VX(vand_vx_b)
-GEN_VEXT_VX(vand_vx_h)
-GEN_VEXT_VX(vand_vx_w)
-GEN_VEXT_VX(vand_vx_d)
-GEN_VEXT_VX(vor_vx_b)
-GEN_VEXT_VX(vor_vx_h)
-GEN_VEXT_VX(vor_vx_w)
-GEN_VEXT_VX(vor_vx_d)
-GEN_VEXT_VX(vxor_vx_b)
-GEN_VEXT_VX(vxor_vx_h)
-GEN_VEXT_VX(vxor_vx_w)
-GEN_VEXT_VX(vxor_vx_d)
+GEN_VEXT_VX(vand_vx_b, 1)
+GEN_VEXT_VX(vand_vx_h, 2)
+GEN_VEXT_VX(vand_vx_w, 4)
+GEN_VEXT_VX(vand_vx_d, 8)
+GEN_VEXT_VX(vor_vx_b, 1)
+GEN_VEXT_VX(vor_vx_h, 2)
+GEN_VEXT_VX(vor_vx_w, 4)
+GEN_VEXT_VX(vor_vx_d, 8)
+GEN_VEXT_VX(vxor_vx_b, 1)
+GEN_VEXT_VX(vxor_vx_h, 2)
+GEN_VEXT_VX(vxor_vx_w, 4)
+GEN_VEXT_VX(vxor_vx_d, 8)
 
 /* Vector Single-Width Bit Shift Instructions */
 #define DO_SLL(N, M)  (N << (M))
@@ -1474,22 +1512,22 @@ RVVCALL(OPIVX2, vmax_vx_b, OP_SSS_B, H1, H1, DO_MAX)
 RVVCALL(OPIVX2, vmax_vx_h, OP_SSS_H, H2, H2, DO_MAX)
 RVVCALL(OPIVX2, vmax_vx_w, OP_SSS_W, H4, H4, DO_MAX)
 RVVCALL(OPIVX2, vmax_vx_d, OP_SSS_D, H8, H8, DO_MAX)
-GEN_VEXT_VX(vminu_vx_b)
-GEN_VEXT_VX(vminu_vx_h)
-GEN_VEXT_VX(vminu_vx_w)
-GEN_VEXT_VX(vminu_vx_d)
-GEN_VEXT_VX(vmin_vx_b)
-GEN_VEXT_VX(vmin_vx_h)
-GEN_VEXT_VX(vmin_vx_w)
-GEN_VEXT_VX(vmin_vx_d)
-GEN_VEXT_VX(vmaxu_vx_b)
-GEN_VEXT_VX(vmaxu_vx_h)
-GEN_VEXT_VX(vmaxu_vx_w)
-GEN_VEXT_VX(vmaxu_vx_d)
-GEN_VEXT_VX(vmax_vx_b)
-GEN_VEXT_VX(vmax_vx_h)
-GEN_VEXT_VX(vmax_vx_w)
-GEN_VEXT_VX(vmax_vx_d)
+GEN_VEXT_VX(vminu_vx_b, 1)
+GEN_VEXT_VX(vminu_vx_h, 2)
+GEN_VEXT_VX(vminu_vx_w, 4)
+GEN_VEXT_VX(vminu_vx_d, 8)
+GEN_VEXT_VX(vmin_vx_b, 1)
+GEN_VEXT_VX(vmin_vx_h, 2)
+GEN_VEXT_VX(vmin_vx_w, 4)
+GEN_VEXT_VX(vmin_vx_d, 8)
+GEN_VEXT_VX(vmaxu_vx_b, 1)
+GEN_VEXT_VX(vmaxu_vx_h, 2)
+GEN_VEXT_VX(vmaxu_vx_w, 4)
+GEN_VEXT_VX(vmaxu_vx_d, 8)
+GEN_VEXT_VX(vmax_vx_b, 1)
+GEN_VEXT_VX(vmax_vx_h, 2)
+GEN_VEXT_VX(vmax_vx_w, 4)
+GEN_VEXT_VX(vmax_vx_d, 8)
 
 /* Vector Single-Width Integer Multiply Instructions */
 #define DO_MUL(N, M) (N * M)
@@ -1633,22 +1671,22 @@ RVVCALL(OPIVX2, vmulhsu_vx_b, OP_SUS_B, H1, H1, do_mulhsu_b)
 RVVCALL(OPIVX2, vmulhsu_vx_h, OP_SUS_H, H2, H2, do_mulhsu_h)
 RVVCALL(OPIVX2, vmulhsu_vx_w, OP_SUS_W, H4, H4, do_mulhsu_w)
 RVVCALL(OPIVX2, vmulhsu_vx_d, OP_SUS_D, H8, H8, do_mulhsu_d)
-GEN_VEXT_VX(vmul_vx_b)
-GEN_VEXT_VX(vmul_vx_h)
-GEN_VEXT_VX(vmul_vx_w)
-GEN_VEXT_VX(vmul_vx_d)
-GEN_VEXT_VX(vmulh_vx_b)
-GEN_VEXT_VX(vmulh_vx_h)
-GEN_VEXT_VX(vmulh_vx_w)
-GEN_VEXT_VX(vmulh_vx_d)
-GEN_VEXT_VX(vmulhu_vx_b)
-GEN_VEXT_VX(vmulhu_vx_h)
-GEN_VEXT_VX(vmulhu_vx_w)
-GEN_VEXT_VX(vmulhu_vx_d)
-GEN_VEXT_VX(vmulhsu_vx_b)
-GEN_VEXT_VX(vmulhsu_vx_h)
-GEN_VEXT_VX(vmulhsu_vx_w)
-GEN_VEXT_VX(vmulhsu_vx_d)
+GEN_VEXT_VX(vmul_vx_b, 1)
+GEN_VEXT_VX(vmul_vx_h, 2)
+GEN_VEXT_VX(vmul_vx_w, 4)
+GEN_VEXT_VX(vmul_vx_d, 8)
+GEN_VEXT_VX(vmulh_vx_b, 1)
+GEN_VEXT_VX(vmulh_vx_h, 2)
+GEN_VEXT_VX(vmulh_vx_w, 4)
+GEN_VEXT_VX(vmulh_vx_d, 8)
+GEN_VEXT_VX(vmulhu_vx_b, 1)
+GEN_VEXT_VX(vmulhu_vx_h, 2)
+GEN_VEXT_VX(vmulhu_vx_w, 4)
+GEN_VEXT_VX(vmulhu_vx_d, 8)
+GEN_VEXT_VX(vmulhsu_vx_b, 1)
+GEN_VEXT_VX(vmulhsu_vx_h, 2)
+GEN_VEXT_VX(vmulhsu_vx_w, 4)
+GEN_VEXT_VX(vmulhsu_vx_d, 8)
 
 /* Vector Integer Divide Instructions */
 #define DO_DIVU(N, M) (unlikely(M == 0) ? (__typeof(N))(-1) : N / M)
@@ -1707,22 +1745,22 @@ RVVCALL(OPIVX2, vrem_vx_b, OP_SSS_B, H1, H1, DO_REM)
 RVVCALL(OPIVX2, vrem_vx_h, OP_SSS_H, H2, H2, DO_REM)
 RVVCALL(OPIVX2, vrem_vx_w, OP_SSS_W, H4, H4, DO_REM)
 RVVCALL(OPIVX2, vrem_vx_d, OP_SSS_D, H8, H8, DO_REM)
-GEN_VEXT_VX(vdivu_vx_b)
-GEN_VEXT_VX(vdivu_vx_h)
-GEN_VEXT_VX(vdivu_vx_w)
-GEN_VEXT_VX(vdivu_vx_d)
-GEN_VEXT_VX(vdiv_vx_b)
-GEN_VEXT_VX(vdiv_vx_h)
-GEN_VEXT_VX(vdiv_vx_w)
-GEN_VEXT_VX(vdiv_vx_d)
-GEN_VEXT_VX(vremu_vx_b)
-GEN_VEXT_VX(vremu_vx_h)
-GEN_VEXT_VX(vremu_vx_w)
-GEN_VEXT_VX(vremu_vx_d)
-GEN_VEXT_VX(vrem_vx_b)
-GEN_VEXT_VX(vrem_vx_h)
-GEN_VEXT_VX(vrem_vx_w)
-GEN_VEXT_VX(vrem_vx_d)
+GEN_VEXT_VX(vdivu_vx_b, 1)
+GEN_VEXT_VX(vdivu_vx_h, 2)
+GEN_VEXT_VX(vdivu_vx_w, 4)
+GEN_VEXT_VX(vdivu_vx_d, 8)
+GEN_VEXT_VX(vdiv_vx_b, 1)
+GEN_VEXT_VX(vdiv_vx_h, 2)
+GEN_VEXT_VX(vdiv_vx_w, 4)
+GEN_VEXT_VX(vdiv_vx_d, 8)
+GEN_VEXT_VX(vremu_vx_b, 1)
+GEN_VEXT_VX(vremu_vx_h, 2)
+GEN_VEXT_VX(vremu_vx_w, 4)
+GEN_VEXT_VX(vremu_vx_d, 8)
+GEN_VEXT_VX(vrem_vx_b, 1)
+GEN_VEXT_VX(vrem_vx_h, 2)
+GEN_VEXT_VX(vrem_vx_w, 4)
+GEN_VEXT_VX(vrem_vx_d, 8)
 
 /* Vector Widening Integer Multiply Instructions */
 RVVCALL(OPIVV2, vwmul_vv_b, WOP_SSS_B, H2, H1, H1, DO_MUL)
@@ -1753,15 +1791,15 @@ RVVCALL(OPIVX2, vwmulu_vx_w, WOP_UUU_W, H8, H4, DO_MUL)
 RVVCALL(OPIVX2, vwmulsu_vx_b, WOP_SUS_B, H2, H1, DO_MUL)
 RVVCALL(OPIVX2, vwmulsu_vx_h, WOP_SUS_H, H4, H2, DO_MUL)
 RVVCALL(OPIVX2, vwmulsu_vx_w, WOP_SUS_W, H8, H4, DO_MUL)
-GEN_VEXT_VX(vwmul_vx_b)
-GEN_VEXT_VX(vwmul_vx_h)
-GEN_VEXT_VX(vwmul_vx_w)
-GEN_VEXT_VX(vwmulu_vx_b)
-GEN_VEXT_VX(vwmulu_vx_h)
-GEN_VEXT_VX(vwmulu_vx_w)
-GEN_VEXT_VX(vwmulsu_vx_b)
-GEN_VEXT_VX(vwmulsu_vx_h)
-GEN_VEXT_VX(vwmulsu_vx_w)
+GEN_VEXT_VX(vwmul_vx_b, 2)
+GEN_VEXT_VX(vwmul_vx_h, 4)
+GEN_VEXT_VX(vwmul_vx_w, 8)
+GEN_VEXT_VX(vwmulu_vx_b, 2)
+GEN_VEXT_VX(vwmulu_vx_h, 4)
+GEN_VEXT_VX(vwmulu_vx_w, 8)
+GEN_VEXT_VX(vwmulsu_vx_b, 2)
+GEN_VEXT_VX(vwmulsu_vx_h, 4)
+GEN_VEXT_VX(vwmulsu_vx_w, 8)
 
 /* Vector Single-Width Integer Multiply-Add Instructions */
 #define OPIVV3(NAME, TD, T1, T2, TX1, TX2, HD, HS1, HS2, OP)   \
@@ -1834,22 +1872,22 @@ RVVCALL(OPIVX3, vnmsub_vx_b, OP_SSS_B, H1, H1, DO_NMSUB)
 RVVCALL(OPIVX3, vnmsub_vx_h, OP_SSS_H, H2, H2, DO_NMSUB)
 RVVCALL(OPIVX3, vnmsub_vx_w, OP_SSS_W, H4, H4, DO_NMSUB)
 RVVCALL(OPIVX3, vnmsub_vx_d, OP_SSS_D, H8, H8, DO_NMSUB)
-GEN_VEXT_VX(vmacc_vx_b)
-GEN_VEXT_VX(vmacc_vx_h)
-GEN_VEXT_VX(vmacc_vx_w)
-GEN_VEXT_VX(vmacc_vx_d)
-GEN_VEXT_VX(vnmsac_vx_b)
-GEN_VEXT_VX(vnmsac_vx_h)
-GEN_VEXT_VX(vnmsac_vx_w)
-GEN_VEXT_VX(vnmsac_vx_d)
-GEN_VEXT_VX(vmadd_vx_b)
-GEN_VEXT_VX(vmadd_vx_h)
-GEN_VEXT_VX(vmadd_vx_w)
-GEN_VEXT_VX(vmadd_vx_d)
-GEN_VEXT_VX(vnmsub_vx_b)
-GEN_VEXT_VX(vnmsub_vx_h)
-GEN_VEXT_VX(vnmsub_vx_w)
-GEN_VEXT_VX(vnmsub_vx_d)
+GEN_VEXT_VX(vmacc_vx_b, 1)
+GEN_VEXT_VX(vmacc_vx_h, 2)
+GEN_VEXT_VX(vmacc_vx_w, 4)
+GEN_VEXT_VX(vmacc_vx_d, 8)
+GEN_VEXT_VX(vnmsac_vx_b, 1)
+GEN_VEXT_VX(vnmsac_vx_h, 2)
+GEN_VEXT_VX(vnmsac_vx_w, 4)
+GEN_VEXT_VX(vnmsac_vx_d, 8)
+GEN_VEXT_VX(vmadd_vx_b, 1)
+GEN_VEXT_VX(vmadd_vx_h, 2)
+GEN_VEXT_VX(vmadd_vx_w, 4)
+GEN_VEXT_VX(vmadd_vx_d, 8)
+GEN_VEXT_VX(vnmsub_vx_b, 1)
+GEN_VEXT_VX(vnmsub_vx_h, 2)
+GEN_VEXT_VX(vnmsub_vx_w, 4)
+GEN_VEXT_VX(vnmsub_vx_d, 8)
 
 /* Vector Widening Integer Multiply-Add Instructions */
 RVVCALL(OPIVV3, vwmaccu_vv_b, WOP_UUU_B, H2, H1, H1, DO_MACC)
@@ -1883,18 +1921,18 @@ RVVCALL(OPIVX3, vwmaccsu_vx_w, WOP_SSU_W, H8, H4, DO_MACC)
 RVVCALL(OPIVX3, vwmaccus_vx_b, WOP_SUS_B, H2, H1, DO_MACC)
 RVVCALL(OPIVX3, vwmaccus_vx_h, WOP_SUS_H, H4, H2, DO_MACC)
 RVVCALL(OPIVX3, vwmaccus_vx_w, WOP_SUS_W, H8, H4, DO_MACC)
-GEN_VEXT_VX(vwmaccu_vx_b)
-GEN_VEXT_VX(vwmaccu_vx_h)
-GEN_VEXT_VX(vwmaccu_vx_w)
-GEN_VEXT_VX(vwmacc_vx_b)
-GEN_VEXT_VX(vwmacc_vx_h)
-GEN_VEXT_VX(vwmacc_vx_w)
-GEN_VEXT_VX(vwmaccsu_vx_b)
-GEN_VEXT_VX(vwmaccsu_vx_h)
-GEN_VEXT_VX(vwmaccsu_vx_w)
-GEN_VEXT_VX(vwmaccus_vx_b)
-GEN_VEXT_VX(vwmaccus_vx_h)
-GEN_VEXT_VX(vwmaccus_vx_w)
+GEN_VEXT_VX(vwmaccu_vx_b, 2)
+GEN_VEXT_VX(vwmaccu_vx_h, 4)
+GEN_VEXT_VX(vwmaccu_vx_w, 8)
+GEN_VEXT_VX(vwmacc_vx_b, 2)
+GEN_VEXT_VX(vwmacc_vx_h, 4)
+GEN_VEXT_VX(vwmacc_vx_w, 8)
+GEN_VEXT_VX(vwmaccsu_vx_b, 2)
+GEN_VEXT_VX(vwmaccsu_vx_h, 4)
+GEN_VEXT_VX(vwmaccsu_vx_w, 8)
+GEN_VEXT_VX(vwmaccus_vx_b, 2)
+GEN_VEXT_VX(vwmaccus_vx_h, 4)
+GEN_VEXT_VX(vwmaccus_vx_w, 8)
 
 /* Vector Integer Merge and Move Instructions */
 #define GEN_VEXT_VMV_VV(NAME, ETYPE, H)                              \

From 7b1bff41c1524e5dc1c2815f68b0454656539993 Mon Sep 17 00:00:00 2001
From: eopXD <yueh.ting.chen@gmail.com>
Date: Mon, 6 Jun 2022 06:16:33 +0000
Subject: [PATCH 925/935] target/riscv: rvv: Add tail agnostic for vector
 integer shift instructions

Signed-off-by: eop Chen <eop.chen@sifive.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <165449614532.19704.7000832880482980398-8@git.sr.ht>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/insn_trans/trans_rvv.c.inc |  3 ++-
 target/riscv/vector_helper.c            | 11 +++++++++++
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/target/riscv/insn_trans/trans_rvv.c.inc b/target/riscv/insn_trans/trans_rvv.c.inc
index 22edf6228d..dbe687fb73 100644
--- a/target/riscv/insn_trans/trans_rvv.c.inc
+++ b/target/riscv/insn_trans/trans_rvv.c.inc
@@ -1831,7 +1831,7 @@ do_opivx_gvec_shift(DisasContext *s, arg_rmrr *a, GVecGen2sFn32 *gvec_fn,
         return false;
     }
 
-    if (a->vm && s->vl_eq_vlmax) {
+    if (a->vm && s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) {
         TCGv_i32 src1 = tcg_temp_new_i32();
 
         tcg_gen_trunc_tl_i32(src1, get_gpr(s, a->rs1, EXT_NONE));
@@ -1890,6 +1890,7 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a)             \
                                                                    \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                 \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);             \
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);               \
         tcg_gen_gvec_4_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),     \
                            vreg_ofs(s, a->rs1),                    \
                            vreg_ofs(s, a->rs2), cpu_env,           \
diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c
index 7cdb5d12af..056c2a1c7e 100644
--- a/target/riscv/vector_helper.c
+++ b/target/riscv/vector_helper.c
@@ -1271,6 +1271,9 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1,                          \
 {                                                                         \
     uint32_t vm = vext_vm(desc);                                          \
     uint32_t vl = env->vl;                                                \
+    uint32_t esz = sizeof(TS1);                                           \
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);          \
+    uint32_t vta = vext_vta(desc);                                        \
     uint32_t i;                                                           \
                                                                           \
     for (i = env->vstart; i < vl; i++) {                                  \
@@ -1282,6 +1285,8 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1,                          \
         *((TS1 *)vd + HS1(i)) = OP(s2, s1 & MASK);                        \
     }                                                                     \
     env->vstart = 0;                                                      \
+    /* set tail elements to 1s */                                         \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);              \
 }
 
 GEN_VEXT_SHIFT_VV(vsll_vv_b, uint8_t,  uint8_t, H1, H1, DO_SLL, 0x7)
@@ -1306,6 +1311,10 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong s1,      \
 {                                                           \
     uint32_t vm = vext_vm(desc);                            \
     uint32_t vl = env->vl;                                  \
+    uint32_t esz = sizeof(TD);                              \
+    uint32_t total_elems =                                  \
+        vext_get_total_elems(env, desc, esz);               \
+    uint32_t vta = vext_vta(desc);                          \
     uint32_t i;                                             \
                                                             \
     for (i = env->vstart; i < vl; i++) {                    \
@@ -1316,6 +1325,8 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong s1,      \
         *((TD *)vd + HD(i)) = OP(s2, s1 & MASK);            \
     }                                                       \
     env->vstart = 0;                                        \
+    /* set tail elements to 1s */                           \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);\
 }
 
 GEN_VEXT_SHIFT_VX(vsll_vx_b, uint8_t, int8_t, H1, H1, DO_SLL, 0x7)

From 38581e5c9a99d2ff9244709cb3577b06a1310006 Mon Sep 17 00:00:00 2001
From: eopXD <yueh.ting.chen@gmail.com>
Date: Mon, 6 Jun 2022 06:16:51 +0000
Subject: [PATCH 926/935] target/riscv: rvv: Add tail agnostic for vector
 integer comparison instructions

Compares write mask registers, and so always operate under a tail-
agnostic policy.

Signed-off-by: eop Chen <eop.chen@sifive.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <165449614532.19704.7000832880482980398-9@git.sr.ht>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/vector_helper.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c
index 056c2a1c7e..a64506bf02 100644
--- a/target/riscv/vector_helper.c
+++ b/target/riscv/vector_helper.c
@@ -1371,6 +1371,8 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1, void *vs2,   \
 {                                                             \
     uint32_t vm = vext_vm(desc);                              \
     uint32_t vl = env->vl;                                    \
+    uint32_t total_elems = env_archcpu(env)->cfg.vlen;        \
+    uint32_t vta_all_1s = vext_vta_all_1s(desc);              \
     uint32_t i;                                               \
                                                               \
     for (i = env->vstart; i < vl; i++) {                      \
@@ -1382,6 +1384,13 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1, void *vs2,   \
         vext_set_elem_mask(vd, i, DO_OP(s2, s1));             \
     }                                                         \
     env->vstart = 0;                                          \
+    /* mask destination register are always tail-agnostic */  \
+    /* set tail elements to 1s */                             \
+    if (vta_all_1s) {                                         \
+        for (; i < total_elems; i++) {                        \
+            vext_set_elem_mask(vd, i, 1);                     \
+        }                                                     \
+    }                                                         \
 }
 
 GEN_VEXT_CMP_VV(vmseq_vv_b, uint8_t,  H1, DO_MSEQ)
@@ -1420,6 +1429,8 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong s1, void *vs2,   \
 {                                                                   \
     uint32_t vm = vext_vm(desc);                                    \
     uint32_t vl = env->vl;                                          \
+    uint32_t total_elems = env_archcpu(env)->cfg.vlen;              \
+    uint32_t vta_all_1s = vext_vta_all_1s(desc);                    \
     uint32_t i;                                                     \
                                                                     \
     for (i = env->vstart; i < vl; i++) {                            \
@@ -1431,6 +1442,13 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong s1, void *vs2,   \
                 DO_OP(s2, (ETYPE)(target_long)s1));                 \
     }                                                               \
     env->vstart = 0;                                                \
+    /* mask destination register are always tail-agnostic */        \
+    /* set tail elements to 1s */                                   \
+    if (vta_all_1s) {                                               \
+        for (; i < total_elems; i++) {                              \
+            vext_set_elem_mask(vd, i, 1);                           \
+        }                                                           \
+    }                                                               \
 }
 
 GEN_VEXT_CMP_VX(vmseq_vx_b, uint8_t,  H1, DO_MSEQ)

From 89a32de2d57591006d4a5d76e664b0b01c6998dc Mon Sep 17 00:00:00 2001
From: eopXD <yueh.ting.chen@gmail.com>
Date: Mon, 6 Jun 2022 06:16:51 +0000
Subject: [PATCH 927/935] target/riscv: rvv: Add tail agnostic for vector
 integer merge and move instructions

Signed-off-by: eop Chen <eop.chen@sifive.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <165449614532.19704.7000832880482980398-10@git.sr.ht>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/insn_trans/trans_rvv.c.inc | 12 ++++++++----
 target/riscv/vector_helper.c            | 20 ++++++++++++++++++++
 2 files changed, 28 insertions(+), 4 deletions(-)

diff --git a/target/riscv/insn_trans/trans_rvv.c.inc b/target/riscv/insn_trans/trans_rvv.c.inc
index dbe687fb73..e75a2fd196 100644
--- a/target/riscv/insn_trans/trans_rvv.c.inc
+++ b/target/riscv/insn_trans/trans_rvv.c.inc
@@ -2086,12 +2086,13 @@ static bool trans_vmv_v_v(DisasContext *s, arg_vmv_v_v *a)
         vext_check_isa_ill(s) &&
         /* vmv.v.v has rs2 = 0 and vm = 1 */
         vext_check_sss(s, a->rd, a->rs1, 0, 1)) {
-        if (s->vl_eq_vlmax) {
+        if (s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) {
             tcg_gen_gvec_mov(s->sew, vreg_ofs(s, a->rd),
                              vreg_ofs(s, a->rs1),
                              MAXSZ(s), MAXSZ(s));
         } else {
             uint32_t data = FIELD_DP32(0, VDATA, LMUL, s->lmul);
+            data = FIELD_DP32(data, VDATA, VTA, s->vta);
             static gen_helper_gvec_2_ptr * const fns[4] = {
                 gen_helper_vmv_v_v_b, gen_helper_vmv_v_v_h,
                 gen_helper_vmv_v_v_w, gen_helper_vmv_v_v_d,
@@ -2126,7 +2127,7 @@ static bool trans_vmv_v_x(DisasContext *s, arg_vmv_v_x *a)
 
         s1 = get_gpr(s, a->rs1, EXT_SIGN);
 
-        if (s->vl_eq_vlmax) {
+        if (s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) {
             tcg_gen_gvec_dup_tl(s->sew, vreg_ofs(s, a->rd),
                                 MAXSZ(s), MAXSZ(s), s1);
         } else {
@@ -2134,6 +2135,7 @@ static bool trans_vmv_v_x(DisasContext *s, arg_vmv_v_x *a)
             TCGv_i64 s1_i64 = tcg_temp_new_i64();
             TCGv_ptr dest = tcg_temp_new_ptr();
             uint32_t data = FIELD_DP32(0, VDATA, LMUL, s->lmul);
+            data = FIELD_DP32(data, VDATA, VTA, s->vta);
             static gen_helper_vmv_vx * const fns[4] = {
                 gen_helper_vmv_v_x_b, gen_helper_vmv_v_x_h,
                 gen_helper_vmv_v_x_w, gen_helper_vmv_v_x_d,
@@ -2163,7 +2165,7 @@ static bool trans_vmv_v_i(DisasContext *s, arg_vmv_v_i *a)
         /* vmv.v.i has rs2 = 0 and vm = 1 */
         vext_check_ss(s, a->rd, 0, 1)) {
         int64_t simm = sextract64(a->rs1, 0, 5);
-        if (s->vl_eq_vlmax) {
+        if (s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) {
             tcg_gen_gvec_dup_imm(s->sew, vreg_ofs(s, a->rd),
                                  MAXSZ(s), MAXSZ(s), simm);
             mark_vs_dirty(s);
@@ -2172,6 +2174,7 @@ static bool trans_vmv_v_i(DisasContext *s, arg_vmv_v_i *a)
             TCGv_i64 s1;
             TCGv_ptr dest;
             uint32_t data = FIELD_DP32(0, VDATA, LMUL, s->lmul);
+            data = FIELD_DP32(data, VDATA, VTA, s->vta);
             static gen_helper_vmv_vx * const fns[4] = {
                 gen_helper_vmv_v_x_b, gen_helper_vmv_v_x_h,
                 gen_helper_vmv_v_x_w, gen_helper_vmv_v_x_d,
@@ -2743,7 +2746,7 @@ static bool trans_vfmv_v_f(DisasContext *s, arg_vfmv_v_f *a)
 
         TCGv_i64 t1;
 
-        if (s->vl_eq_vlmax) {
+        if (s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) {
             t1 = tcg_temp_new_i64();
             /* NaN-box f[rs1] */
             do_nanbox(s, t1, cpu_fpr[a->rs1]);
@@ -2755,6 +2758,7 @@ static bool trans_vfmv_v_f(DisasContext *s, arg_vfmv_v_f *a)
             TCGv_ptr dest;
             TCGv_i32 desc;
             uint32_t data = FIELD_DP32(0, VDATA, LMUL, s->lmul);
+            data = FIELD_DP32(data, VDATA, VTA, s->vta);
             static gen_helper_vmv_vx * const fns[3] = {
                 gen_helper_vmv_v_x_h,
                 gen_helper_vmv_v_x_w,
diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c
index a64506bf02..128238786e 100644
--- a/target/riscv/vector_helper.c
+++ b/target/riscv/vector_helper.c
@@ -1969,6 +1969,9 @@ void HELPER(NAME)(void *vd, void *vs1, CPURISCVState *env,           \
                   uint32_t desc)                                     \
 {                                                                    \
     uint32_t vl = env->vl;                                           \
+    uint32_t esz = sizeof(ETYPE);                                    \
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);     \
+    uint32_t vta = vext_vta(desc);                                   \
     uint32_t i;                                                      \
                                                                      \
     for (i = env->vstart; i < vl; i++) {                             \
@@ -1976,6 +1979,8 @@ void HELPER(NAME)(void *vd, void *vs1, CPURISCVState *env,           \
         *((ETYPE *)vd + H(i)) = s1;                                  \
     }                                                                \
     env->vstart = 0;                                                 \
+    /* set tail elements to 1s */                                    \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);         \
 }
 
 GEN_VEXT_VMV_VV(vmv_v_v_b, int8_t,  H1)
@@ -1988,12 +1993,17 @@ void HELPER(NAME)(void *vd, uint64_t s1, CPURISCVState *env,         \
                   uint32_t desc)                                     \
 {                                                                    \
     uint32_t vl = env->vl;                                           \
+    uint32_t esz = sizeof(ETYPE);                                    \
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);     \
+    uint32_t vta = vext_vta(desc);                                   \
     uint32_t i;                                                      \
                                                                      \
     for (i = env->vstart; i < vl; i++) {                             \
         *((ETYPE *)vd + H(i)) = (ETYPE)s1;                           \
     }                                                                \
     env->vstart = 0;                                                 \
+    /* set tail elements to 1s */                                    \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);         \
 }
 
 GEN_VEXT_VMV_VX(vmv_v_x_b, int8_t,  H1)
@@ -2006,6 +2016,9 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1, void *vs2,          \
                   CPURISCVState *env, uint32_t desc)                 \
 {                                                                    \
     uint32_t vl = env->vl;                                           \
+    uint32_t esz = sizeof(ETYPE);                                    \
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);     \
+    uint32_t vta = vext_vta(desc);                                   \
     uint32_t i;                                                      \
                                                                      \
     for (i = env->vstart; i < vl; i++) {                             \
@@ -2013,6 +2026,8 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1, void *vs2,          \
         *((ETYPE *)vd + H(i)) = *(vt + H(i));                        \
     }                                                                \
     env->vstart = 0;                                                 \
+    /* set tail elements to 1s */                                    \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);         \
 }
 
 GEN_VEXT_VMERGE_VV(vmerge_vvm_b, int8_t,  H1)
@@ -2025,6 +2040,9 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong s1,               \
                   void *vs2, CPURISCVState *env, uint32_t desc)      \
 {                                                                    \
     uint32_t vl = env->vl;                                           \
+    uint32_t esz = sizeof(ETYPE);                                    \
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);     \
+    uint32_t vta = vext_vta(desc);                                   \
     uint32_t i;                                                      \
                                                                      \
     for (i = env->vstart; i < vl; i++) {                             \
@@ -2034,6 +2052,8 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong s1,               \
         *((ETYPE *)vd + H(i)) = d;                                   \
     }                                                                \
     env->vstart = 0;                                                 \
+    /* set tail elements to 1s */                                    \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);         \
 }
 
 GEN_VEXT_VMERGE_VX(vmerge_vxm_b, int8_t,  H1)

From 09106eed3041d5eb57dd768332146abe6d86e0e4 Mon Sep 17 00:00:00 2001
From: eopXD <yueh.ting.chen@gmail.com>
Date: Mon, 6 Jun 2022 06:16:38 +0000
Subject: [PATCH 928/935] target/riscv: rvv: Add tail agnostic for vector
 fix-point arithmetic instructions

Signed-off-by: eop Chen <eop.chen@sifive.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <165449614532.19704.7000832880482980398-11@git.sr.ht>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/vector_helper.c | 220 ++++++++++++++++++-----------------
 1 file changed, 114 insertions(+), 106 deletions(-)

diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c
index 128238786e..1fa93cf1f0 100644
--- a/target/riscv/vector_helper.c
+++ b/target/riscv/vector_helper.c
@@ -2103,10 +2103,12 @@ static inline void
 vext_vv_rm_2(void *vd, void *v0, void *vs1, void *vs2,
              CPURISCVState *env,
              uint32_t desc,
-             opivv2_rm_fn *fn)
+             opivv2_rm_fn *fn, uint32_t esz)
 {
     uint32_t vm = vext_vm(desc);
     uint32_t vl = env->vl;
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);
+    uint32_t vta = vext_vta(desc);
 
     switch (env->vxrm) {
     case 0: /* rnu */
@@ -2126,15 +2128,17 @@ vext_vv_rm_2(void *vd, void *v0, void *vs1, void *vs2,
                      env, vl, vm, 3, fn);
         break;
     }
+    /* set tail elements to 1s */
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);
 }
 
 /* generate helpers for fixed point instructions with OPIVV format */
-#define GEN_VEXT_VV_RM(NAME)                                    \
+#define GEN_VEXT_VV_RM(NAME, ESZ)                               \
 void HELPER(NAME)(void *vd, void *v0, void *vs1, void *vs2,     \
                   CPURISCVState *env, uint32_t desc)            \
 {                                                               \
     vext_vv_rm_2(vd, v0, vs1, vs2, env, desc,                   \
-                 do_##NAME);                                    \
+                 do_##NAME, ESZ);                               \
 }
 
 static inline uint8_t saddu8(CPURISCVState *env, int vxrm, uint8_t a, uint8_t b)
@@ -2184,10 +2188,10 @@ RVVCALL(OPIVV2_RM, vsaddu_vv_b, OP_UUU_B, H1, H1, H1, saddu8)
 RVVCALL(OPIVV2_RM, vsaddu_vv_h, OP_UUU_H, H2, H2, H2, saddu16)
 RVVCALL(OPIVV2_RM, vsaddu_vv_w, OP_UUU_W, H4, H4, H4, saddu32)
 RVVCALL(OPIVV2_RM, vsaddu_vv_d, OP_UUU_D, H8, H8, H8, saddu64)
-GEN_VEXT_VV_RM(vsaddu_vv_b)
-GEN_VEXT_VV_RM(vsaddu_vv_h)
-GEN_VEXT_VV_RM(vsaddu_vv_w)
-GEN_VEXT_VV_RM(vsaddu_vv_d)
+GEN_VEXT_VV_RM(vsaddu_vv_b, 1)
+GEN_VEXT_VV_RM(vsaddu_vv_h, 2)
+GEN_VEXT_VV_RM(vsaddu_vv_w, 4)
+GEN_VEXT_VV_RM(vsaddu_vv_d, 8)
 
 typedef void opivx2_rm_fn(void *vd, target_long s1, void *vs2, int i,
                           CPURISCVState *env, int vxrm);
@@ -2220,10 +2224,12 @@ static inline void
 vext_vx_rm_2(void *vd, void *v0, target_long s1, void *vs2,
              CPURISCVState *env,
              uint32_t desc,
-             opivx2_rm_fn *fn)
+             opivx2_rm_fn *fn, uint32_t esz)
 {
     uint32_t vm = vext_vm(desc);
     uint32_t vl = env->vl;
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);
+    uint32_t vta = vext_vta(desc);
 
     switch (env->vxrm) {
     case 0: /* rnu */
@@ -2243,25 +2249,27 @@ vext_vx_rm_2(void *vd, void *v0, target_long s1, void *vs2,
                      env, vl, vm, 3, fn);
         break;
     }
+    /* set tail elements to 1s */
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);
 }
 
 /* generate helpers for fixed point instructions with OPIVX format */
-#define GEN_VEXT_VX_RM(NAME)                              \
+#define GEN_VEXT_VX_RM(NAME, ESZ)                         \
 void HELPER(NAME)(void *vd, void *v0, target_ulong s1,    \
         void *vs2, CPURISCVState *env, uint32_t desc)     \
 {                                                         \
     vext_vx_rm_2(vd, v0, s1, vs2, env, desc,              \
-                 do_##NAME);                              \
+                 do_##NAME, ESZ);                         \
 }
 
 RVVCALL(OPIVX2_RM, vsaddu_vx_b, OP_UUU_B, H1, H1, saddu8)
 RVVCALL(OPIVX2_RM, vsaddu_vx_h, OP_UUU_H, H2, H2, saddu16)
 RVVCALL(OPIVX2_RM, vsaddu_vx_w, OP_UUU_W, H4, H4, saddu32)
 RVVCALL(OPIVX2_RM, vsaddu_vx_d, OP_UUU_D, H8, H8, saddu64)
-GEN_VEXT_VX_RM(vsaddu_vx_b)
-GEN_VEXT_VX_RM(vsaddu_vx_h)
-GEN_VEXT_VX_RM(vsaddu_vx_w)
-GEN_VEXT_VX_RM(vsaddu_vx_d)
+GEN_VEXT_VX_RM(vsaddu_vx_b, 1)
+GEN_VEXT_VX_RM(vsaddu_vx_h, 2)
+GEN_VEXT_VX_RM(vsaddu_vx_w, 4)
+GEN_VEXT_VX_RM(vsaddu_vx_d, 8)
 
 static inline int8_t sadd8(CPURISCVState *env, int vxrm, int8_t a, int8_t b)
 {
@@ -2307,19 +2315,19 @@ RVVCALL(OPIVV2_RM, vsadd_vv_b, OP_SSS_B, H1, H1, H1, sadd8)
 RVVCALL(OPIVV2_RM, vsadd_vv_h, OP_SSS_H, H2, H2, H2, sadd16)
 RVVCALL(OPIVV2_RM, vsadd_vv_w, OP_SSS_W, H4, H4, H4, sadd32)
 RVVCALL(OPIVV2_RM, vsadd_vv_d, OP_SSS_D, H8, H8, H8, sadd64)
-GEN_VEXT_VV_RM(vsadd_vv_b)
-GEN_VEXT_VV_RM(vsadd_vv_h)
-GEN_VEXT_VV_RM(vsadd_vv_w)
-GEN_VEXT_VV_RM(vsadd_vv_d)
+GEN_VEXT_VV_RM(vsadd_vv_b, 1)
+GEN_VEXT_VV_RM(vsadd_vv_h, 2)
+GEN_VEXT_VV_RM(vsadd_vv_w, 4)
+GEN_VEXT_VV_RM(vsadd_vv_d, 8)
 
 RVVCALL(OPIVX2_RM, vsadd_vx_b, OP_SSS_B, H1, H1, sadd8)
 RVVCALL(OPIVX2_RM, vsadd_vx_h, OP_SSS_H, H2, H2, sadd16)
 RVVCALL(OPIVX2_RM, vsadd_vx_w, OP_SSS_W, H4, H4, sadd32)
 RVVCALL(OPIVX2_RM, vsadd_vx_d, OP_SSS_D, H8, H8, sadd64)
-GEN_VEXT_VX_RM(vsadd_vx_b)
-GEN_VEXT_VX_RM(vsadd_vx_h)
-GEN_VEXT_VX_RM(vsadd_vx_w)
-GEN_VEXT_VX_RM(vsadd_vx_d)
+GEN_VEXT_VX_RM(vsadd_vx_b, 1)
+GEN_VEXT_VX_RM(vsadd_vx_h, 2)
+GEN_VEXT_VX_RM(vsadd_vx_w, 4)
+GEN_VEXT_VX_RM(vsadd_vx_d, 8)
 
 static inline uint8_t ssubu8(CPURISCVState *env, int vxrm, uint8_t a, uint8_t b)
 {
@@ -2368,19 +2376,19 @@ RVVCALL(OPIVV2_RM, vssubu_vv_b, OP_UUU_B, H1, H1, H1, ssubu8)
 RVVCALL(OPIVV2_RM, vssubu_vv_h, OP_UUU_H, H2, H2, H2, ssubu16)
 RVVCALL(OPIVV2_RM, vssubu_vv_w, OP_UUU_W, H4, H4, H4, ssubu32)
 RVVCALL(OPIVV2_RM, vssubu_vv_d, OP_UUU_D, H8, H8, H8, ssubu64)
-GEN_VEXT_VV_RM(vssubu_vv_b)
-GEN_VEXT_VV_RM(vssubu_vv_h)
-GEN_VEXT_VV_RM(vssubu_vv_w)
-GEN_VEXT_VV_RM(vssubu_vv_d)
+GEN_VEXT_VV_RM(vssubu_vv_b, 1)
+GEN_VEXT_VV_RM(vssubu_vv_h, 2)
+GEN_VEXT_VV_RM(vssubu_vv_w, 4)
+GEN_VEXT_VV_RM(vssubu_vv_d, 8)
 
 RVVCALL(OPIVX2_RM, vssubu_vx_b, OP_UUU_B, H1, H1, ssubu8)
 RVVCALL(OPIVX2_RM, vssubu_vx_h, OP_UUU_H, H2, H2, ssubu16)
 RVVCALL(OPIVX2_RM, vssubu_vx_w, OP_UUU_W, H4, H4, ssubu32)
 RVVCALL(OPIVX2_RM, vssubu_vx_d, OP_UUU_D, H8, H8, ssubu64)
-GEN_VEXT_VX_RM(vssubu_vx_b)
-GEN_VEXT_VX_RM(vssubu_vx_h)
-GEN_VEXT_VX_RM(vssubu_vx_w)
-GEN_VEXT_VX_RM(vssubu_vx_d)
+GEN_VEXT_VX_RM(vssubu_vx_b, 1)
+GEN_VEXT_VX_RM(vssubu_vx_h, 2)
+GEN_VEXT_VX_RM(vssubu_vx_w, 4)
+GEN_VEXT_VX_RM(vssubu_vx_d, 8)
 
 static inline int8_t ssub8(CPURISCVState *env, int vxrm, int8_t a, int8_t b)
 {
@@ -2426,19 +2434,19 @@ RVVCALL(OPIVV2_RM, vssub_vv_b, OP_SSS_B, H1, H1, H1, ssub8)
 RVVCALL(OPIVV2_RM, vssub_vv_h, OP_SSS_H, H2, H2, H2, ssub16)
 RVVCALL(OPIVV2_RM, vssub_vv_w, OP_SSS_W, H4, H4, H4, ssub32)
 RVVCALL(OPIVV2_RM, vssub_vv_d, OP_SSS_D, H8, H8, H8, ssub64)
-GEN_VEXT_VV_RM(vssub_vv_b)
-GEN_VEXT_VV_RM(vssub_vv_h)
-GEN_VEXT_VV_RM(vssub_vv_w)
-GEN_VEXT_VV_RM(vssub_vv_d)
+GEN_VEXT_VV_RM(vssub_vv_b, 1)
+GEN_VEXT_VV_RM(vssub_vv_h, 2)
+GEN_VEXT_VV_RM(vssub_vv_w, 4)
+GEN_VEXT_VV_RM(vssub_vv_d, 8)
 
 RVVCALL(OPIVX2_RM, vssub_vx_b, OP_SSS_B, H1, H1, ssub8)
 RVVCALL(OPIVX2_RM, vssub_vx_h, OP_SSS_H, H2, H2, ssub16)
 RVVCALL(OPIVX2_RM, vssub_vx_w, OP_SSS_W, H4, H4, ssub32)
 RVVCALL(OPIVX2_RM, vssub_vx_d, OP_SSS_D, H8, H8, ssub64)
-GEN_VEXT_VX_RM(vssub_vx_b)
-GEN_VEXT_VX_RM(vssub_vx_h)
-GEN_VEXT_VX_RM(vssub_vx_w)
-GEN_VEXT_VX_RM(vssub_vx_d)
+GEN_VEXT_VX_RM(vssub_vx_b, 1)
+GEN_VEXT_VX_RM(vssub_vx_h, 2)
+GEN_VEXT_VX_RM(vssub_vx_w, 4)
+GEN_VEXT_VX_RM(vssub_vx_d, 8)
 
 /* Vector Single-Width Averaging Add and Subtract */
 static inline uint8_t get_round(int vxrm, uint64_t v, uint8_t shift)
@@ -2490,19 +2498,19 @@ RVVCALL(OPIVV2_RM, vaadd_vv_b, OP_SSS_B, H1, H1, H1, aadd32)
 RVVCALL(OPIVV2_RM, vaadd_vv_h, OP_SSS_H, H2, H2, H2, aadd32)
 RVVCALL(OPIVV2_RM, vaadd_vv_w, OP_SSS_W, H4, H4, H4, aadd32)
 RVVCALL(OPIVV2_RM, vaadd_vv_d, OP_SSS_D, H8, H8, H8, aadd64)
-GEN_VEXT_VV_RM(vaadd_vv_b)
-GEN_VEXT_VV_RM(vaadd_vv_h)
-GEN_VEXT_VV_RM(vaadd_vv_w)
-GEN_VEXT_VV_RM(vaadd_vv_d)
+GEN_VEXT_VV_RM(vaadd_vv_b, 1)
+GEN_VEXT_VV_RM(vaadd_vv_h, 2)
+GEN_VEXT_VV_RM(vaadd_vv_w, 4)
+GEN_VEXT_VV_RM(vaadd_vv_d, 8)
 
 RVVCALL(OPIVX2_RM, vaadd_vx_b, OP_SSS_B, H1, H1, aadd32)
 RVVCALL(OPIVX2_RM, vaadd_vx_h, OP_SSS_H, H2, H2, aadd32)
 RVVCALL(OPIVX2_RM, vaadd_vx_w, OP_SSS_W, H4, H4, aadd32)
 RVVCALL(OPIVX2_RM, vaadd_vx_d, OP_SSS_D, H8, H8, aadd64)
-GEN_VEXT_VX_RM(vaadd_vx_b)
-GEN_VEXT_VX_RM(vaadd_vx_h)
-GEN_VEXT_VX_RM(vaadd_vx_w)
-GEN_VEXT_VX_RM(vaadd_vx_d)
+GEN_VEXT_VX_RM(vaadd_vx_b, 1)
+GEN_VEXT_VX_RM(vaadd_vx_h, 2)
+GEN_VEXT_VX_RM(vaadd_vx_w, 4)
+GEN_VEXT_VX_RM(vaadd_vx_d, 8)
 
 static inline uint32_t aaddu32(CPURISCVState *env, int vxrm,
                                uint32_t a, uint32_t b)
@@ -2527,19 +2535,19 @@ RVVCALL(OPIVV2_RM, vaaddu_vv_b, OP_UUU_B, H1, H1, H1, aaddu32)
 RVVCALL(OPIVV2_RM, vaaddu_vv_h, OP_UUU_H, H2, H2, H2, aaddu32)
 RVVCALL(OPIVV2_RM, vaaddu_vv_w, OP_UUU_W, H4, H4, H4, aaddu32)
 RVVCALL(OPIVV2_RM, vaaddu_vv_d, OP_UUU_D, H8, H8, H8, aaddu64)
-GEN_VEXT_VV_RM(vaaddu_vv_b)
-GEN_VEXT_VV_RM(vaaddu_vv_h)
-GEN_VEXT_VV_RM(vaaddu_vv_w)
-GEN_VEXT_VV_RM(vaaddu_vv_d)
+GEN_VEXT_VV_RM(vaaddu_vv_b, 1)
+GEN_VEXT_VV_RM(vaaddu_vv_h, 2)
+GEN_VEXT_VV_RM(vaaddu_vv_w, 4)
+GEN_VEXT_VV_RM(vaaddu_vv_d, 8)
 
 RVVCALL(OPIVX2_RM, vaaddu_vx_b, OP_UUU_B, H1, H1, aaddu32)
 RVVCALL(OPIVX2_RM, vaaddu_vx_h, OP_UUU_H, H2, H2, aaddu32)
 RVVCALL(OPIVX2_RM, vaaddu_vx_w, OP_UUU_W, H4, H4, aaddu32)
 RVVCALL(OPIVX2_RM, vaaddu_vx_d, OP_UUU_D, H8, H8, aaddu64)
-GEN_VEXT_VX_RM(vaaddu_vx_b)
-GEN_VEXT_VX_RM(vaaddu_vx_h)
-GEN_VEXT_VX_RM(vaaddu_vx_w)
-GEN_VEXT_VX_RM(vaaddu_vx_d)
+GEN_VEXT_VX_RM(vaaddu_vx_b, 1)
+GEN_VEXT_VX_RM(vaaddu_vx_h, 2)
+GEN_VEXT_VX_RM(vaaddu_vx_w, 4)
+GEN_VEXT_VX_RM(vaaddu_vx_d, 8)
 
 static inline int32_t asub32(CPURISCVState *env, int vxrm, int32_t a, int32_t b)
 {
@@ -2563,19 +2571,19 @@ RVVCALL(OPIVV2_RM, vasub_vv_b, OP_SSS_B, H1, H1, H1, asub32)
 RVVCALL(OPIVV2_RM, vasub_vv_h, OP_SSS_H, H2, H2, H2, asub32)
 RVVCALL(OPIVV2_RM, vasub_vv_w, OP_SSS_W, H4, H4, H4, asub32)
 RVVCALL(OPIVV2_RM, vasub_vv_d, OP_SSS_D, H8, H8, H8, asub64)
-GEN_VEXT_VV_RM(vasub_vv_b)
-GEN_VEXT_VV_RM(vasub_vv_h)
-GEN_VEXT_VV_RM(vasub_vv_w)
-GEN_VEXT_VV_RM(vasub_vv_d)
+GEN_VEXT_VV_RM(vasub_vv_b, 1)
+GEN_VEXT_VV_RM(vasub_vv_h, 2)
+GEN_VEXT_VV_RM(vasub_vv_w, 4)
+GEN_VEXT_VV_RM(vasub_vv_d, 8)
 
 RVVCALL(OPIVX2_RM, vasub_vx_b, OP_SSS_B, H1, H1, asub32)
 RVVCALL(OPIVX2_RM, vasub_vx_h, OP_SSS_H, H2, H2, asub32)
 RVVCALL(OPIVX2_RM, vasub_vx_w, OP_SSS_W, H4, H4, asub32)
 RVVCALL(OPIVX2_RM, vasub_vx_d, OP_SSS_D, H8, H8, asub64)
-GEN_VEXT_VX_RM(vasub_vx_b)
-GEN_VEXT_VX_RM(vasub_vx_h)
-GEN_VEXT_VX_RM(vasub_vx_w)
-GEN_VEXT_VX_RM(vasub_vx_d)
+GEN_VEXT_VX_RM(vasub_vx_b, 1)
+GEN_VEXT_VX_RM(vasub_vx_h, 2)
+GEN_VEXT_VX_RM(vasub_vx_w, 4)
+GEN_VEXT_VX_RM(vasub_vx_d, 8)
 
 static inline uint32_t asubu32(CPURISCVState *env, int vxrm,
                                uint32_t a, uint32_t b)
@@ -2600,19 +2608,19 @@ RVVCALL(OPIVV2_RM, vasubu_vv_b, OP_UUU_B, H1, H1, H1, asubu32)
 RVVCALL(OPIVV2_RM, vasubu_vv_h, OP_UUU_H, H2, H2, H2, asubu32)
 RVVCALL(OPIVV2_RM, vasubu_vv_w, OP_UUU_W, H4, H4, H4, asubu32)
 RVVCALL(OPIVV2_RM, vasubu_vv_d, OP_UUU_D, H8, H8, H8, asubu64)
-GEN_VEXT_VV_RM(vasubu_vv_b)
-GEN_VEXT_VV_RM(vasubu_vv_h)
-GEN_VEXT_VV_RM(vasubu_vv_w)
-GEN_VEXT_VV_RM(vasubu_vv_d)
+GEN_VEXT_VV_RM(vasubu_vv_b, 1)
+GEN_VEXT_VV_RM(vasubu_vv_h, 2)
+GEN_VEXT_VV_RM(vasubu_vv_w, 4)
+GEN_VEXT_VV_RM(vasubu_vv_d, 8)
 
 RVVCALL(OPIVX2_RM, vasubu_vx_b, OP_UUU_B, H1, H1, asubu32)
 RVVCALL(OPIVX2_RM, vasubu_vx_h, OP_UUU_H, H2, H2, asubu32)
 RVVCALL(OPIVX2_RM, vasubu_vx_w, OP_UUU_W, H4, H4, asubu32)
 RVVCALL(OPIVX2_RM, vasubu_vx_d, OP_UUU_D, H8, H8, asubu64)
-GEN_VEXT_VX_RM(vasubu_vx_b)
-GEN_VEXT_VX_RM(vasubu_vx_h)
-GEN_VEXT_VX_RM(vasubu_vx_w)
-GEN_VEXT_VX_RM(vasubu_vx_d)
+GEN_VEXT_VX_RM(vasubu_vx_b, 1)
+GEN_VEXT_VX_RM(vasubu_vx_h, 2)
+GEN_VEXT_VX_RM(vasubu_vx_w, 4)
+GEN_VEXT_VX_RM(vasubu_vx_d, 8)
 
 /* Vector Single-Width Fractional Multiply with Rounding and Saturation */
 static inline int8_t vsmul8(CPURISCVState *env, int vxrm, int8_t a, int8_t b)
@@ -2707,19 +2715,19 @@ RVVCALL(OPIVV2_RM, vsmul_vv_b, OP_SSS_B, H1, H1, H1, vsmul8)
 RVVCALL(OPIVV2_RM, vsmul_vv_h, OP_SSS_H, H2, H2, H2, vsmul16)
 RVVCALL(OPIVV2_RM, vsmul_vv_w, OP_SSS_W, H4, H4, H4, vsmul32)
 RVVCALL(OPIVV2_RM, vsmul_vv_d, OP_SSS_D, H8, H8, H8, vsmul64)
-GEN_VEXT_VV_RM(vsmul_vv_b)
-GEN_VEXT_VV_RM(vsmul_vv_h)
-GEN_VEXT_VV_RM(vsmul_vv_w)
-GEN_VEXT_VV_RM(vsmul_vv_d)
+GEN_VEXT_VV_RM(vsmul_vv_b, 1)
+GEN_VEXT_VV_RM(vsmul_vv_h, 2)
+GEN_VEXT_VV_RM(vsmul_vv_w, 4)
+GEN_VEXT_VV_RM(vsmul_vv_d, 8)
 
 RVVCALL(OPIVX2_RM, vsmul_vx_b, OP_SSS_B, H1, H1, vsmul8)
 RVVCALL(OPIVX2_RM, vsmul_vx_h, OP_SSS_H, H2, H2, vsmul16)
 RVVCALL(OPIVX2_RM, vsmul_vx_w, OP_SSS_W, H4, H4, vsmul32)
 RVVCALL(OPIVX2_RM, vsmul_vx_d, OP_SSS_D, H8, H8, vsmul64)
-GEN_VEXT_VX_RM(vsmul_vx_b)
-GEN_VEXT_VX_RM(vsmul_vx_h)
-GEN_VEXT_VX_RM(vsmul_vx_w)
-GEN_VEXT_VX_RM(vsmul_vx_d)
+GEN_VEXT_VX_RM(vsmul_vx_b, 1)
+GEN_VEXT_VX_RM(vsmul_vx_h, 2)
+GEN_VEXT_VX_RM(vsmul_vx_w, 4)
+GEN_VEXT_VX_RM(vsmul_vx_d, 8)
 
 /* Vector Single-Width Scaling Shift Instructions */
 static inline uint8_t
@@ -2766,19 +2774,19 @@ RVVCALL(OPIVV2_RM, vssrl_vv_b, OP_UUU_B, H1, H1, H1, vssrl8)
 RVVCALL(OPIVV2_RM, vssrl_vv_h, OP_UUU_H, H2, H2, H2, vssrl16)
 RVVCALL(OPIVV2_RM, vssrl_vv_w, OP_UUU_W, H4, H4, H4, vssrl32)
 RVVCALL(OPIVV2_RM, vssrl_vv_d, OP_UUU_D, H8, H8, H8, vssrl64)
-GEN_VEXT_VV_RM(vssrl_vv_b)
-GEN_VEXT_VV_RM(vssrl_vv_h)
-GEN_VEXT_VV_RM(vssrl_vv_w)
-GEN_VEXT_VV_RM(vssrl_vv_d)
+GEN_VEXT_VV_RM(vssrl_vv_b, 1)
+GEN_VEXT_VV_RM(vssrl_vv_h, 2)
+GEN_VEXT_VV_RM(vssrl_vv_w, 4)
+GEN_VEXT_VV_RM(vssrl_vv_d, 8)
 
 RVVCALL(OPIVX2_RM, vssrl_vx_b, OP_UUU_B, H1, H1, vssrl8)
 RVVCALL(OPIVX2_RM, vssrl_vx_h, OP_UUU_H, H2, H2, vssrl16)
 RVVCALL(OPIVX2_RM, vssrl_vx_w, OP_UUU_W, H4, H4, vssrl32)
 RVVCALL(OPIVX2_RM, vssrl_vx_d, OP_UUU_D, H8, H8, vssrl64)
-GEN_VEXT_VX_RM(vssrl_vx_b)
-GEN_VEXT_VX_RM(vssrl_vx_h)
-GEN_VEXT_VX_RM(vssrl_vx_w)
-GEN_VEXT_VX_RM(vssrl_vx_d)
+GEN_VEXT_VX_RM(vssrl_vx_b, 1)
+GEN_VEXT_VX_RM(vssrl_vx_h, 2)
+GEN_VEXT_VX_RM(vssrl_vx_w, 4)
+GEN_VEXT_VX_RM(vssrl_vx_d, 8)
 
 static inline int8_t
 vssra8(CPURISCVState *env, int vxrm, int8_t a, int8_t b)
@@ -2825,19 +2833,19 @@ RVVCALL(OPIVV2_RM, vssra_vv_b, OP_SSS_B, H1, H1, H1, vssra8)
 RVVCALL(OPIVV2_RM, vssra_vv_h, OP_SSS_H, H2, H2, H2, vssra16)
 RVVCALL(OPIVV2_RM, vssra_vv_w, OP_SSS_W, H4, H4, H4, vssra32)
 RVVCALL(OPIVV2_RM, vssra_vv_d, OP_SSS_D, H8, H8, H8, vssra64)
-GEN_VEXT_VV_RM(vssra_vv_b)
-GEN_VEXT_VV_RM(vssra_vv_h)
-GEN_VEXT_VV_RM(vssra_vv_w)
-GEN_VEXT_VV_RM(vssra_vv_d)
+GEN_VEXT_VV_RM(vssra_vv_b, 1)
+GEN_VEXT_VV_RM(vssra_vv_h, 2)
+GEN_VEXT_VV_RM(vssra_vv_w, 4)
+GEN_VEXT_VV_RM(vssra_vv_d, 8)
 
 RVVCALL(OPIVX2_RM, vssra_vx_b, OP_SSS_B, H1, H1, vssra8)
 RVVCALL(OPIVX2_RM, vssra_vx_h, OP_SSS_H, H2, H2, vssra16)
 RVVCALL(OPIVX2_RM, vssra_vx_w, OP_SSS_W, H4, H4, vssra32)
 RVVCALL(OPIVX2_RM, vssra_vx_d, OP_SSS_D, H8, H8, vssra64)
-GEN_VEXT_VX_RM(vssra_vx_b)
-GEN_VEXT_VX_RM(vssra_vx_h)
-GEN_VEXT_VX_RM(vssra_vx_w)
-GEN_VEXT_VX_RM(vssra_vx_d)
+GEN_VEXT_VX_RM(vssra_vx_b, 1)
+GEN_VEXT_VX_RM(vssra_vx_h, 2)
+GEN_VEXT_VX_RM(vssra_vx_w, 4)
+GEN_VEXT_VX_RM(vssra_vx_d, 8)
 
 /* Vector Narrowing Fixed-Point Clip Instructions */
 static inline int8_t
@@ -2900,16 +2908,16 @@ vnclip32(CPURISCVState *env, int vxrm, int64_t a, int32_t b)
 RVVCALL(OPIVV2_RM, vnclip_wv_b, NOP_SSS_B, H1, H2, H1, vnclip8)
 RVVCALL(OPIVV2_RM, vnclip_wv_h, NOP_SSS_H, H2, H4, H2, vnclip16)
 RVVCALL(OPIVV2_RM, vnclip_wv_w, NOP_SSS_W, H4, H8, H4, vnclip32)
-GEN_VEXT_VV_RM(vnclip_wv_b)
-GEN_VEXT_VV_RM(vnclip_wv_h)
-GEN_VEXT_VV_RM(vnclip_wv_w)
+GEN_VEXT_VV_RM(vnclip_wv_b, 1)
+GEN_VEXT_VV_RM(vnclip_wv_h, 2)
+GEN_VEXT_VV_RM(vnclip_wv_w, 4)
 
 RVVCALL(OPIVX2_RM, vnclip_wx_b, NOP_SSS_B, H1, H2, vnclip8)
 RVVCALL(OPIVX2_RM, vnclip_wx_h, NOP_SSS_H, H2, H4, vnclip16)
 RVVCALL(OPIVX2_RM, vnclip_wx_w, NOP_SSS_W, H4, H8, vnclip32)
-GEN_VEXT_VX_RM(vnclip_wx_b)
-GEN_VEXT_VX_RM(vnclip_wx_h)
-GEN_VEXT_VX_RM(vnclip_wx_w)
+GEN_VEXT_VX_RM(vnclip_wx_b, 1)
+GEN_VEXT_VX_RM(vnclip_wx_h, 2)
+GEN_VEXT_VX_RM(vnclip_wx_w, 4)
 
 static inline uint8_t
 vnclipu8(CPURISCVState *env, int vxrm, uint16_t a, uint8_t b)
@@ -2962,16 +2970,16 @@ vnclipu32(CPURISCVState *env, int vxrm, uint64_t a, uint32_t b)
 RVVCALL(OPIVV2_RM, vnclipu_wv_b, NOP_UUU_B, H1, H2, H1, vnclipu8)
 RVVCALL(OPIVV2_RM, vnclipu_wv_h, NOP_UUU_H, H2, H4, H2, vnclipu16)
 RVVCALL(OPIVV2_RM, vnclipu_wv_w, NOP_UUU_W, H4, H8, H4, vnclipu32)
-GEN_VEXT_VV_RM(vnclipu_wv_b)
-GEN_VEXT_VV_RM(vnclipu_wv_h)
-GEN_VEXT_VV_RM(vnclipu_wv_w)
+GEN_VEXT_VV_RM(vnclipu_wv_b, 1)
+GEN_VEXT_VV_RM(vnclipu_wv_h, 2)
+GEN_VEXT_VV_RM(vnclipu_wv_w, 4)
 
 RVVCALL(OPIVX2_RM, vnclipu_wx_b, NOP_UUU_B, H1, H2, vnclipu8)
 RVVCALL(OPIVX2_RM, vnclipu_wx_h, NOP_UUU_H, H2, H4, vnclipu16)
 RVVCALL(OPIVX2_RM, vnclipu_wx_w, NOP_UUU_W, H4, H8, vnclipu32)
-GEN_VEXT_VX_RM(vnclipu_wx_b)
-GEN_VEXT_VX_RM(vnclipu_wx_h)
-GEN_VEXT_VX_RM(vnclipu_wx_w)
+GEN_VEXT_VX_RM(vnclipu_wx_b, 1)
+GEN_VEXT_VX_RM(vnclipu_wx_h, 2)
+GEN_VEXT_VX_RM(vnclipu_wx_w, 4)
 
 /*
  *** Vector Float Point Arithmetic Instructions

From 5eacf7d8a0fb19ea4d87eb678462fdb9a29b9190 Mon Sep 17 00:00:00 2001
From: eopXD <yueh.ting.chen@gmail.com>
Date: Mon, 6 Jun 2022 06:16:56 +0000
Subject: [PATCH 929/935] target/riscv: rvv: Add tail agnostic for vector
 floating-point instructions

Compares write mask registers, and so always operate under a tail-
agnostic policy.

Signed-off-by: eop Chen <eop.chen@sifive.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <165449614532.19704.7000832880482980398-12@git.sr.ht>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/insn_trans/trans_rvv.c.inc |  17 +
 target/riscv/vector_helper.c            | 440 +++++++++++++-----------
 2 files changed, 261 insertions(+), 196 deletions(-)

diff --git a/target/riscv/insn_trans/trans_rvv.c.inc b/target/riscv/insn_trans/trans_rvv.c.inc
index e75a2fd196..1add4cb655 100644
--- a/target/riscv/insn_trans/trans_rvv.c.inc
+++ b/target/riscv/insn_trans/trans_rvv.c.inc
@@ -2338,6 +2338,9 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a)             \
                                                                    \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                 \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);             \
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);               \
+        data =                                                     \
+            FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s);\
         tcg_gen_gvec_4_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),     \
                            vreg_ofs(s, a->rs1),                    \
                            vreg_ofs(s, a->rs2), cpu_env,           \
@@ -2420,6 +2423,9 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a)            \
         gen_set_rm(s, RISCV_FRM_DYN);                             \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);            \
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);              \
+        data = FIELD_DP32(data, VDATA, VTA_ALL_1S,                \
+                          s->cfg_vta_all_1s);                     \
         return opfvf_trans(a->rd, a->rs1, a->rs2, data,           \
                            fns[s->sew - 1], s);                   \
     }                                                             \
@@ -2458,6 +2464,7 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a)           \
                                                                  \
         data = FIELD_DP32(data, VDATA, VM, a->vm);               \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);           \
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);             \
         tcg_gen_gvec_4_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),   \
                            vreg_ofs(s, a->rs1),                  \
                            vreg_ofs(s, a->rs2), cpu_env,         \
@@ -2497,6 +2504,7 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a)           \
         gen_set_rm(s, RISCV_FRM_DYN);                            \
         data = FIELD_DP32(data, VDATA, VM, a->vm);               \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);           \
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);             \
         return opfvf_trans(a->rd, a->rs1, a->rs2, data,          \
                            fns[s->sew - 1], s);                  \
     }                                                            \
@@ -2533,6 +2541,7 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a)             \
                                                                    \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                 \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);             \
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);               \
         tcg_gen_gvec_4_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),     \
                            vreg_ofs(s, a->rs1),                    \
                            vreg_ofs(s, a->rs2), cpu_env,           \
@@ -2572,6 +2581,7 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a)           \
         gen_set_rm(s, RISCV_FRM_DYN);                            \
         data = FIELD_DP32(data, VDATA, VM, a->vm);               \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);           \
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);             \
         return opfvf_trans(a->rd, a->rs1, a->rs2, data,          \
                            fns[s->sew - 1], s);                  \
     }                                                            \
@@ -2655,6 +2665,7 @@ static bool do_opfv(DisasContext *s, arg_rmr *a,
 
         data = FIELD_DP32(data, VDATA, VM, a->vm);
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);
         tcg_gen_gvec_3_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),
                            vreg_ofs(s, a->rs2), cpu_env,
                            s->cfg_ptr->vlen / 8,
@@ -2859,6 +2870,7 @@ static bool trans_##NAME(DisasContext *s, arg_rmr *a)              \
                                                                    \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                 \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);             \
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);               \
         tcg_gen_gvec_3_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),     \
                            vreg_ofs(s, a->rs2), cpu_env,           \
                            s->cfg_ptr->vlen / 8,                   \
@@ -2910,6 +2922,8 @@ static bool trans_##NAME(DisasContext *s, arg_rmr *a)              \
         tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \
                                                                    \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                 \
+        data = FIELD_DP32(data, VDATA, LMUL, s->lmul);             \
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);               \
         tcg_gen_gvec_3_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),     \
                            vreg_ofs(s, a->rs2), cpu_env,           \
                            s->cfg_ptr->vlen / 8,                   \
@@ -2977,6 +2991,7 @@ static bool trans_##NAME(DisasContext *s, arg_rmr *a)              \
                                                                    \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                 \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);             \
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);               \
         tcg_gen_gvec_3_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),     \
                            vreg_ofs(s, a->rs2), cpu_env,           \
                            s->cfg_ptr->vlen / 8,                   \
@@ -3030,6 +3045,8 @@ static bool trans_##NAME(DisasContext *s, arg_rmr *a)              \
         tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \
                                                                    \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                 \
+        data = FIELD_DP32(data, VDATA, LMUL, s->lmul);             \
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);               \
         tcg_gen_gvec_3_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),     \
                            vreg_ofs(s, a->rs2), cpu_env,           \
                            s->cfg_ptr->vlen / 8,                   \
diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c
index 1fa93cf1f0..17390d8d06 100644
--- a/target/riscv/vector_helper.c
+++ b/target/riscv/vector_helper.c
@@ -2994,13 +2994,16 @@ static void do_##NAME(void *vd, void *vs1, void *vs2, int i,   \
     *((TD *)vd + HD(i)) = OP(s2, s1, &env->fp_status);         \
 }
 
-#define GEN_VEXT_VV_ENV(NAME)                             \
+#define GEN_VEXT_VV_ENV(NAME, ESZ)                        \
 void HELPER(NAME)(void *vd, void *v0, void *vs1,          \
                   void *vs2, CPURISCVState *env,          \
                   uint32_t desc)                          \
 {                                                         \
     uint32_t vm = vext_vm(desc);                          \
     uint32_t vl = env->vl;                                \
+    uint32_t total_elems =                                \
+        vext_get_total_elems(env, desc, ESZ);             \
+    uint32_t vta = vext_vta(desc);                        \
     uint32_t i;                                           \
                                                           \
     for (i = env->vstart; i < vl; i++) {                  \
@@ -3010,14 +3013,17 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1,          \
         do_##NAME(vd, vs1, vs2, i, env);                  \
     }                                                     \
     env->vstart = 0;                                      \
+    /* set tail elements to 1s */                         \
+    vext_set_elems_1s(vd, vta, vl * ESZ,                  \
+                      total_elems * ESZ);                 \
 }
 
 RVVCALL(OPFVV2, vfadd_vv_h, OP_UUU_H, H2, H2, H2, float16_add)
 RVVCALL(OPFVV2, vfadd_vv_w, OP_UUU_W, H4, H4, H4, float32_add)
 RVVCALL(OPFVV2, vfadd_vv_d, OP_UUU_D, H8, H8, H8, float64_add)
-GEN_VEXT_VV_ENV(vfadd_vv_h)
-GEN_VEXT_VV_ENV(vfadd_vv_w)
-GEN_VEXT_VV_ENV(vfadd_vv_d)
+GEN_VEXT_VV_ENV(vfadd_vv_h, 2)
+GEN_VEXT_VV_ENV(vfadd_vv_w, 4)
+GEN_VEXT_VV_ENV(vfadd_vv_d, 8)
 
 #define OPFVF2(NAME, TD, T1, T2, TX1, TX2, HD, HS2, OP)        \
 static void do_##NAME(void *vd, uint64_t s1, void *vs2, int i, \
@@ -3027,13 +3033,16 @@ static void do_##NAME(void *vd, uint64_t s1, void *vs2, int i, \
     *((TD *)vd + HD(i)) = OP(s2, (TX1)(T1)s1, &env->fp_status);\
 }
 
-#define GEN_VEXT_VF(NAME)                                 \
+#define GEN_VEXT_VF(NAME, ESZ)                            \
 void HELPER(NAME)(void *vd, void *v0, uint64_t s1,        \
                   void *vs2, CPURISCVState *env,          \
                   uint32_t desc)                          \
 {                                                         \
     uint32_t vm = vext_vm(desc);                          \
     uint32_t vl = env->vl;                                \
+    uint32_t total_elems =                                \
+        vext_get_total_elems(env, desc, ESZ);              \
+    uint32_t vta = vext_vta(desc);                        \
     uint32_t i;                                           \
                                                           \
     for (i = env->vstart; i < vl; i++) {                  \
@@ -3043,27 +3052,30 @@ void HELPER(NAME)(void *vd, void *v0, uint64_t s1,        \
         do_##NAME(vd, s1, vs2, i, env);                   \
     }                                                     \
     env->vstart = 0;                                      \
+    /* set tail elements to 1s */                         \
+    vext_set_elems_1s(vd, vta, vl * ESZ,                  \
+                      total_elems * ESZ);                 \
 }
 
 RVVCALL(OPFVF2, vfadd_vf_h, OP_UUU_H, H2, H2, float16_add)
 RVVCALL(OPFVF2, vfadd_vf_w, OP_UUU_W, H4, H4, float32_add)
 RVVCALL(OPFVF2, vfadd_vf_d, OP_UUU_D, H8, H8, float64_add)
-GEN_VEXT_VF(vfadd_vf_h)
-GEN_VEXT_VF(vfadd_vf_w)
-GEN_VEXT_VF(vfadd_vf_d)
+GEN_VEXT_VF(vfadd_vf_h, 2)
+GEN_VEXT_VF(vfadd_vf_w, 4)
+GEN_VEXT_VF(vfadd_vf_d, 8)
 
 RVVCALL(OPFVV2, vfsub_vv_h, OP_UUU_H, H2, H2, H2, float16_sub)
 RVVCALL(OPFVV2, vfsub_vv_w, OP_UUU_W, H4, H4, H4, float32_sub)
 RVVCALL(OPFVV2, vfsub_vv_d, OP_UUU_D, H8, H8, H8, float64_sub)
-GEN_VEXT_VV_ENV(vfsub_vv_h)
-GEN_VEXT_VV_ENV(vfsub_vv_w)
-GEN_VEXT_VV_ENV(vfsub_vv_d)
+GEN_VEXT_VV_ENV(vfsub_vv_h, 2)
+GEN_VEXT_VV_ENV(vfsub_vv_w, 4)
+GEN_VEXT_VV_ENV(vfsub_vv_d, 8)
 RVVCALL(OPFVF2, vfsub_vf_h, OP_UUU_H, H2, H2, float16_sub)
 RVVCALL(OPFVF2, vfsub_vf_w, OP_UUU_W, H4, H4, float32_sub)
 RVVCALL(OPFVF2, vfsub_vf_d, OP_UUU_D, H8, H8, float64_sub)
-GEN_VEXT_VF(vfsub_vf_h)
-GEN_VEXT_VF(vfsub_vf_w)
-GEN_VEXT_VF(vfsub_vf_d)
+GEN_VEXT_VF(vfsub_vf_h, 2)
+GEN_VEXT_VF(vfsub_vf_w, 4)
+GEN_VEXT_VF(vfsub_vf_d, 8)
 
 static uint16_t float16_rsub(uint16_t a, uint16_t b, float_status *s)
 {
@@ -3083,9 +3095,9 @@ static uint64_t float64_rsub(uint64_t a, uint64_t b, float_status *s)
 RVVCALL(OPFVF2, vfrsub_vf_h, OP_UUU_H, H2, H2, float16_rsub)
 RVVCALL(OPFVF2, vfrsub_vf_w, OP_UUU_W, H4, H4, float32_rsub)
 RVVCALL(OPFVF2, vfrsub_vf_d, OP_UUU_D, H8, H8, float64_rsub)
-GEN_VEXT_VF(vfrsub_vf_h)
-GEN_VEXT_VF(vfrsub_vf_w)
-GEN_VEXT_VF(vfrsub_vf_d)
+GEN_VEXT_VF(vfrsub_vf_h, 2)
+GEN_VEXT_VF(vfrsub_vf_w, 4)
+GEN_VEXT_VF(vfrsub_vf_d, 8)
 
 /* Vector Widening Floating-Point Add/Subtract Instructions */
 static uint32_t vfwadd16(uint16_t a, uint16_t b, float_status *s)
@@ -3103,12 +3115,12 @@ static uint64_t vfwadd32(uint32_t a, uint32_t b, float_status *s)
 
 RVVCALL(OPFVV2, vfwadd_vv_h, WOP_UUU_H, H4, H2, H2, vfwadd16)
 RVVCALL(OPFVV2, vfwadd_vv_w, WOP_UUU_W, H8, H4, H4, vfwadd32)
-GEN_VEXT_VV_ENV(vfwadd_vv_h)
-GEN_VEXT_VV_ENV(vfwadd_vv_w)
+GEN_VEXT_VV_ENV(vfwadd_vv_h, 4)
+GEN_VEXT_VV_ENV(vfwadd_vv_w, 8)
 RVVCALL(OPFVF2, vfwadd_vf_h, WOP_UUU_H, H4, H2, vfwadd16)
 RVVCALL(OPFVF2, vfwadd_vf_w, WOP_UUU_W, H8, H4, vfwadd32)
-GEN_VEXT_VF(vfwadd_vf_h)
-GEN_VEXT_VF(vfwadd_vf_w)
+GEN_VEXT_VF(vfwadd_vf_h, 4)
+GEN_VEXT_VF(vfwadd_vf_w, 8)
 
 static uint32_t vfwsub16(uint16_t a, uint16_t b, float_status *s)
 {
@@ -3125,12 +3137,12 @@ static uint64_t vfwsub32(uint32_t a, uint32_t b, float_status *s)
 
 RVVCALL(OPFVV2, vfwsub_vv_h, WOP_UUU_H, H4, H2, H2, vfwsub16)
 RVVCALL(OPFVV2, vfwsub_vv_w, WOP_UUU_W, H8, H4, H4, vfwsub32)
-GEN_VEXT_VV_ENV(vfwsub_vv_h)
-GEN_VEXT_VV_ENV(vfwsub_vv_w)
+GEN_VEXT_VV_ENV(vfwsub_vv_h, 4)
+GEN_VEXT_VV_ENV(vfwsub_vv_w, 8)
 RVVCALL(OPFVF2, vfwsub_vf_h, WOP_UUU_H, H4, H2, vfwsub16)
 RVVCALL(OPFVF2, vfwsub_vf_w, WOP_UUU_W, H8, H4, vfwsub32)
-GEN_VEXT_VF(vfwsub_vf_h)
-GEN_VEXT_VF(vfwsub_vf_w)
+GEN_VEXT_VF(vfwsub_vf_h, 4)
+GEN_VEXT_VF(vfwsub_vf_w, 8)
 
 static uint32_t vfwaddw16(uint32_t a, uint16_t b, float_status *s)
 {
@@ -3144,12 +3156,12 @@ static uint64_t vfwaddw32(uint64_t a, uint32_t b, float_status *s)
 
 RVVCALL(OPFVV2, vfwadd_wv_h, WOP_WUUU_H, H4, H2, H2, vfwaddw16)
 RVVCALL(OPFVV2, vfwadd_wv_w, WOP_WUUU_W, H8, H4, H4, vfwaddw32)
-GEN_VEXT_VV_ENV(vfwadd_wv_h)
-GEN_VEXT_VV_ENV(vfwadd_wv_w)
+GEN_VEXT_VV_ENV(vfwadd_wv_h, 4)
+GEN_VEXT_VV_ENV(vfwadd_wv_w, 8)
 RVVCALL(OPFVF2, vfwadd_wf_h, WOP_WUUU_H, H4, H2, vfwaddw16)
 RVVCALL(OPFVF2, vfwadd_wf_w, WOP_WUUU_W, H8, H4, vfwaddw32)
-GEN_VEXT_VF(vfwadd_wf_h)
-GEN_VEXT_VF(vfwadd_wf_w)
+GEN_VEXT_VF(vfwadd_wf_h, 4)
+GEN_VEXT_VF(vfwadd_wf_w, 8)
 
 static uint32_t vfwsubw16(uint32_t a, uint16_t b, float_status *s)
 {
@@ -3163,39 +3175,39 @@ static uint64_t vfwsubw32(uint64_t a, uint32_t b, float_status *s)
 
 RVVCALL(OPFVV2, vfwsub_wv_h, WOP_WUUU_H, H4, H2, H2, vfwsubw16)
 RVVCALL(OPFVV2, vfwsub_wv_w, WOP_WUUU_W, H8, H4, H4, vfwsubw32)
-GEN_VEXT_VV_ENV(vfwsub_wv_h)
-GEN_VEXT_VV_ENV(vfwsub_wv_w)
+GEN_VEXT_VV_ENV(vfwsub_wv_h, 4)
+GEN_VEXT_VV_ENV(vfwsub_wv_w, 8)
 RVVCALL(OPFVF2, vfwsub_wf_h, WOP_WUUU_H, H4, H2, vfwsubw16)
 RVVCALL(OPFVF2, vfwsub_wf_w, WOP_WUUU_W, H8, H4, vfwsubw32)
-GEN_VEXT_VF(vfwsub_wf_h)
-GEN_VEXT_VF(vfwsub_wf_w)
+GEN_VEXT_VF(vfwsub_wf_h, 4)
+GEN_VEXT_VF(vfwsub_wf_w, 8)
 
 /* Vector Single-Width Floating-Point Multiply/Divide Instructions */
 RVVCALL(OPFVV2, vfmul_vv_h, OP_UUU_H, H2, H2, H2, float16_mul)
 RVVCALL(OPFVV2, vfmul_vv_w, OP_UUU_W, H4, H4, H4, float32_mul)
 RVVCALL(OPFVV2, vfmul_vv_d, OP_UUU_D, H8, H8, H8, float64_mul)
-GEN_VEXT_VV_ENV(vfmul_vv_h)
-GEN_VEXT_VV_ENV(vfmul_vv_w)
-GEN_VEXT_VV_ENV(vfmul_vv_d)
+GEN_VEXT_VV_ENV(vfmul_vv_h, 2)
+GEN_VEXT_VV_ENV(vfmul_vv_w, 4)
+GEN_VEXT_VV_ENV(vfmul_vv_d, 8)
 RVVCALL(OPFVF2, vfmul_vf_h, OP_UUU_H, H2, H2, float16_mul)
 RVVCALL(OPFVF2, vfmul_vf_w, OP_UUU_W, H4, H4, float32_mul)
 RVVCALL(OPFVF2, vfmul_vf_d, OP_UUU_D, H8, H8, float64_mul)
-GEN_VEXT_VF(vfmul_vf_h)
-GEN_VEXT_VF(vfmul_vf_w)
-GEN_VEXT_VF(vfmul_vf_d)
+GEN_VEXT_VF(vfmul_vf_h, 2)
+GEN_VEXT_VF(vfmul_vf_w, 4)
+GEN_VEXT_VF(vfmul_vf_d, 8)
 
 RVVCALL(OPFVV2, vfdiv_vv_h, OP_UUU_H, H2, H2, H2, float16_div)
 RVVCALL(OPFVV2, vfdiv_vv_w, OP_UUU_W, H4, H4, H4, float32_div)
 RVVCALL(OPFVV2, vfdiv_vv_d, OP_UUU_D, H8, H8, H8, float64_div)
-GEN_VEXT_VV_ENV(vfdiv_vv_h)
-GEN_VEXT_VV_ENV(vfdiv_vv_w)
-GEN_VEXT_VV_ENV(vfdiv_vv_d)
+GEN_VEXT_VV_ENV(vfdiv_vv_h, 2)
+GEN_VEXT_VV_ENV(vfdiv_vv_w, 4)
+GEN_VEXT_VV_ENV(vfdiv_vv_d, 8)
 RVVCALL(OPFVF2, vfdiv_vf_h, OP_UUU_H, H2, H2, float16_div)
 RVVCALL(OPFVF2, vfdiv_vf_w, OP_UUU_W, H4, H4, float32_div)
 RVVCALL(OPFVF2, vfdiv_vf_d, OP_UUU_D, H8, H8, float64_div)
-GEN_VEXT_VF(vfdiv_vf_h)
-GEN_VEXT_VF(vfdiv_vf_w)
-GEN_VEXT_VF(vfdiv_vf_d)
+GEN_VEXT_VF(vfdiv_vf_h, 2)
+GEN_VEXT_VF(vfdiv_vf_w, 4)
+GEN_VEXT_VF(vfdiv_vf_d, 8)
 
 static uint16_t float16_rdiv(uint16_t a, uint16_t b, float_status *s)
 {
@@ -3215,9 +3227,9 @@ static uint64_t float64_rdiv(uint64_t a, uint64_t b, float_status *s)
 RVVCALL(OPFVF2, vfrdiv_vf_h, OP_UUU_H, H2, H2, float16_rdiv)
 RVVCALL(OPFVF2, vfrdiv_vf_w, OP_UUU_W, H4, H4, float32_rdiv)
 RVVCALL(OPFVF2, vfrdiv_vf_d, OP_UUU_D, H8, H8, float64_rdiv)
-GEN_VEXT_VF(vfrdiv_vf_h)
-GEN_VEXT_VF(vfrdiv_vf_w)
-GEN_VEXT_VF(vfrdiv_vf_d)
+GEN_VEXT_VF(vfrdiv_vf_h, 2)
+GEN_VEXT_VF(vfrdiv_vf_w, 4)
+GEN_VEXT_VF(vfrdiv_vf_d, 8)
 
 /* Vector Widening Floating-Point Multiply */
 static uint32_t vfwmul16(uint16_t a, uint16_t b, float_status *s)
@@ -3234,12 +3246,12 @@ static uint64_t vfwmul32(uint32_t a, uint32_t b, float_status *s)
 }
 RVVCALL(OPFVV2, vfwmul_vv_h, WOP_UUU_H, H4, H2, H2, vfwmul16)
 RVVCALL(OPFVV2, vfwmul_vv_w, WOP_UUU_W, H8, H4, H4, vfwmul32)
-GEN_VEXT_VV_ENV(vfwmul_vv_h)
-GEN_VEXT_VV_ENV(vfwmul_vv_w)
+GEN_VEXT_VV_ENV(vfwmul_vv_h, 4)
+GEN_VEXT_VV_ENV(vfwmul_vv_w, 8)
 RVVCALL(OPFVF2, vfwmul_vf_h, WOP_UUU_H, H4, H2, vfwmul16)
 RVVCALL(OPFVF2, vfwmul_vf_w, WOP_UUU_W, H8, H4, vfwmul32)
-GEN_VEXT_VF(vfwmul_vf_h)
-GEN_VEXT_VF(vfwmul_vf_w)
+GEN_VEXT_VF(vfwmul_vf_h, 4)
+GEN_VEXT_VF(vfwmul_vf_w, 8)
 
 /* Vector Single-Width Floating-Point Fused Multiply-Add Instructions */
 #define OPFVV3(NAME, TD, T1, T2, TX1, TX2, HD, HS1, HS2, OP)       \
@@ -3270,9 +3282,9 @@ static uint64_t fmacc64(uint64_t a, uint64_t b, uint64_t d, float_status *s)
 RVVCALL(OPFVV3, vfmacc_vv_h, OP_UUU_H, H2, H2, H2, fmacc16)
 RVVCALL(OPFVV3, vfmacc_vv_w, OP_UUU_W, H4, H4, H4, fmacc32)
 RVVCALL(OPFVV3, vfmacc_vv_d, OP_UUU_D, H8, H8, H8, fmacc64)
-GEN_VEXT_VV_ENV(vfmacc_vv_h)
-GEN_VEXT_VV_ENV(vfmacc_vv_w)
-GEN_VEXT_VV_ENV(vfmacc_vv_d)
+GEN_VEXT_VV_ENV(vfmacc_vv_h, 2)
+GEN_VEXT_VV_ENV(vfmacc_vv_w, 4)
+GEN_VEXT_VV_ENV(vfmacc_vv_d, 8)
 
 #define OPFVF3(NAME, TD, T1, T2, TX1, TX2, HD, HS2, OP)           \
 static void do_##NAME(void *vd, uint64_t s1, void *vs2, int i,    \
@@ -3286,9 +3298,9 @@ static void do_##NAME(void *vd, uint64_t s1, void *vs2, int i,    \
 RVVCALL(OPFVF3, vfmacc_vf_h, OP_UUU_H, H2, H2, fmacc16)
 RVVCALL(OPFVF3, vfmacc_vf_w, OP_UUU_W, H4, H4, fmacc32)
 RVVCALL(OPFVF3, vfmacc_vf_d, OP_UUU_D, H8, H8, fmacc64)
-GEN_VEXT_VF(vfmacc_vf_h)
-GEN_VEXT_VF(vfmacc_vf_w)
-GEN_VEXT_VF(vfmacc_vf_d)
+GEN_VEXT_VF(vfmacc_vf_h, 2)
+GEN_VEXT_VF(vfmacc_vf_w, 4)
+GEN_VEXT_VF(vfmacc_vf_d, 8)
 
 static uint16_t fnmacc16(uint16_t a, uint16_t b, uint16_t d, float_status *s)
 {
@@ -3311,15 +3323,15 @@ static uint64_t fnmacc64(uint64_t a, uint64_t b, uint64_t d, float_status *s)
 RVVCALL(OPFVV3, vfnmacc_vv_h, OP_UUU_H, H2, H2, H2, fnmacc16)
 RVVCALL(OPFVV3, vfnmacc_vv_w, OP_UUU_W, H4, H4, H4, fnmacc32)
 RVVCALL(OPFVV3, vfnmacc_vv_d, OP_UUU_D, H8, H8, H8, fnmacc64)
-GEN_VEXT_VV_ENV(vfnmacc_vv_h)
-GEN_VEXT_VV_ENV(vfnmacc_vv_w)
-GEN_VEXT_VV_ENV(vfnmacc_vv_d)
+GEN_VEXT_VV_ENV(vfnmacc_vv_h, 2)
+GEN_VEXT_VV_ENV(vfnmacc_vv_w, 4)
+GEN_VEXT_VV_ENV(vfnmacc_vv_d, 8)
 RVVCALL(OPFVF3, vfnmacc_vf_h, OP_UUU_H, H2, H2, fnmacc16)
 RVVCALL(OPFVF3, vfnmacc_vf_w, OP_UUU_W, H4, H4, fnmacc32)
 RVVCALL(OPFVF3, vfnmacc_vf_d, OP_UUU_D, H8, H8, fnmacc64)
-GEN_VEXT_VF(vfnmacc_vf_h)
-GEN_VEXT_VF(vfnmacc_vf_w)
-GEN_VEXT_VF(vfnmacc_vf_d)
+GEN_VEXT_VF(vfnmacc_vf_h, 2)
+GEN_VEXT_VF(vfnmacc_vf_w, 4)
+GEN_VEXT_VF(vfnmacc_vf_d, 8)
 
 static uint16_t fmsac16(uint16_t a, uint16_t b, uint16_t d, float_status *s)
 {
@@ -3339,15 +3351,15 @@ static uint64_t fmsac64(uint64_t a, uint64_t b, uint64_t d, float_status *s)
 RVVCALL(OPFVV3, vfmsac_vv_h, OP_UUU_H, H2, H2, H2, fmsac16)
 RVVCALL(OPFVV3, vfmsac_vv_w, OP_UUU_W, H4, H4, H4, fmsac32)
 RVVCALL(OPFVV3, vfmsac_vv_d, OP_UUU_D, H8, H8, H8, fmsac64)
-GEN_VEXT_VV_ENV(vfmsac_vv_h)
-GEN_VEXT_VV_ENV(vfmsac_vv_w)
-GEN_VEXT_VV_ENV(vfmsac_vv_d)
+GEN_VEXT_VV_ENV(vfmsac_vv_h, 2)
+GEN_VEXT_VV_ENV(vfmsac_vv_w, 4)
+GEN_VEXT_VV_ENV(vfmsac_vv_d, 8)
 RVVCALL(OPFVF3, vfmsac_vf_h, OP_UUU_H, H2, H2, fmsac16)
 RVVCALL(OPFVF3, vfmsac_vf_w, OP_UUU_W, H4, H4, fmsac32)
 RVVCALL(OPFVF3, vfmsac_vf_d, OP_UUU_D, H8, H8, fmsac64)
-GEN_VEXT_VF(vfmsac_vf_h)
-GEN_VEXT_VF(vfmsac_vf_w)
-GEN_VEXT_VF(vfmsac_vf_d)
+GEN_VEXT_VF(vfmsac_vf_h, 2)
+GEN_VEXT_VF(vfmsac_vf_w, 4)
+GEN_VEXT_VF(vfmsac_vf_d, 8)
 
 static uint16_t fnmsac16(uint16_t a, uint16_t b, uint16_t d, float_status *s)
 {
@@ -3367,15 +3379,15 @@ static uint64_t fnmsac64(uint64_t a, uint64_t b, uint64_t d, float_status *s)
 RVVCALL(OPFVV3, vfnmsac_vv_h, OP_UUU_H, H2, H2, H2, fnmsac16)
 RVVCALL(OPFVV3, vfnmsac_vv_w, OP_UUU_W, H4, H4, H4, fnmsac32)
 RVVCALL(OPFVV3, vfnmsac_vv_d, OP_UUU_D, H8, H8, H8, fnmsac64)
-GEN_VEXT_VV_ENV(vfnmsac_vv_h)
-GEN_VEXT_VV_ENV(vfnmsac_vv_w)
-GEN_VEXT_VV_ENV(vfnmsac_vv_d)
+GEN_VEXT_VV_ENV(vfnmsac_vv_h, 2)
+GEN_VEXT_VV_ENV(vfnmsac_vv_w, 4)
+GEN_VEXT_VV_ENV(vfnmsac_vv_d, 8)
 RVVCALL(OPFVF3, vfnmsac_vf_h, OP_UUU_H, H2, H2, fnmsac16)
 RVVCALL(OPFVF3, vfnmsac_vf_w, OP_UUU_W, H4, H4, fnmsac32)
 RVVCALL(OPFVF3, vfnmsac_vf_d, OP_UUU_D, H8, H8, fnmsac64)
-GEN_VEXT_VF(vfnmsac_vf_h)
-GEN_VEXT_VF(vfnmsac_vf_w)
-GEN_VEXT_VF(vfnmsac_vf_d)
+GEN_VEXT_VF(vfnmsac_vf_h, 2)
+GEN_VEXT_VF(vfnmsac_vf_w, 4)
+GEN_VEXT_VF(vfnmsac_vf_d, 8)
 
 static uint16_t fmadd16(uint16_t a, uint16_t b, uint16_t d, float_status *s)
 {
@@ -3395,15 +3407,15 @@ static uint64_t fmadd64(uint64_t a, uint64_t b, uint64_t d, float_status *s)
 RVVCALL(OPFVV3, vfmadd_vv_h, OP_UUU_H, H2, H2, H2, fmadd16)
 RVVCALL(OPFVV3, vfmadd_vv_w, OP_UUU_W, H4, H4, H4, fmadd32)
 RVVCALL(OPFVV3, vfmadd_vv_d, OP_UUU_D, H8, H8, H8, fmadd64)
-GEN_VEXT_VV_ENV(vfmadd_vv_h)
-GEN_VEXT_VV_ENV(vfmadd_vv_w)
-GEN_VEXT_VV_ENV(vfmadd_vv_d)
+GEN_VEXT_VV_ENV(vfmadd_vv_h, 2)
+GEN_VEXT_VV_ENV(vfmadd_vv_w, 4)
+GEN_VEXT_VV_ENV(vfmadd_vv_d, 8)
 RVVCALL(OPFVF3, vfmadd_vf_h, OP_UUU_H, H2, H2, fmadd16)
 RVVCALL(OPFVF3, vfmadd_vf_w, OP_UUU_W, H4, H4, fmadd32)
 RVVCALL(OPFVF3, vfmadd_vf_d, OP_UUU_D, H8, H8, fmadd64)
-GEN_VEXT_VF(vfmadd_vf_h)
-GEN_VEXT_VF(vfmadd_vf_w)
-GEN_VEXT_VF(vfmadd_vf_d)
+GEN_VEXT_VF(vfmadd_vf_h, 2)
+GEN_VEXT_VF(vfmadd_vf_w, 4)
+GEN_VEXT_VF(vfmadd_vf_d, 8)
 
 static uint16_t fnmadd16(uint16_t a, uint16_t b, uint16_t d, float_status *s)
 {
@@ -3426,15 +3438,15 @@ static uint64_t fnmadd64(uint64_t a, uint64_t b, uint64_t d, float_status *s)
 RVVCALL(OPFVV3, vfnmadd_vv_h, OP_UUU_H, H2, H2, H2, fnmadd16)
 RVVCALL(OPFVV3, vfnmadd_vv_w, OP_UUU_W, H4, H4, H4, fnmadd32)
 RVVCALL(OPFVV3, vfnmadd_vv_d, OP_UUU_D, H8, H8, H8, fnmadd64)
-GEN_VEXT_VV_ENV(vfnmadd_vv_h)
-GEN_VEXT_VV_ENV(vfnmadd_vv_w)
-GEN_VEXT_VV_ENV(vfnmadd_vv_d)
+GEN_VEXT_VV_ENV(vfnmadd_vv_h, 2)
+GEN_VEXT_VV_ENV(vfnmadd_vv_w, 4)
+GEN_VEXT_VV_ENV(vfnmadd_vv_d, 8)
 RVVCALL(OPFVF3, vfnmadd_vf_h, OP_UUU_H, H2, H2, fnmadd16)
 RVVCALL(OPFVF3, vfnmadd_vf_w, OP_UUU_W, H4, H4, fnmadd32)
 RVVCALL(OPFVF3, vfnmadd_vf_d, OP_UUU_D, H8, H8, fnmadd64)
-GEN_VEXT_VF(vfnmadd_vf_h)
-GEN_VEXT_VF(vfnmadd_vf_w)
-GEN_VEXT_VF(vfnmadd_vf_d)
+GEN_VEXT_VF(vfnmadd_vf_h, 2)
+GEN_VEXT_VF(vfnmadd_vf_w, 4)
+GEN_VEXT_VF(vfnmadd_vf_d, 8)
 
 static uint16_t fmsub16(uint16_t a, uint16_t b, uint16_t d, float_status *s)
 {
@@ -3454,15 +3466,15 @@ static uint64_t fmsub64(uint64_t a, uint64_t b, uint64_t d, float_status *s)
 RVVCALL(OPFVV3, vfmsub_vv_h, OP_UUU_H, H2, H2, H2, fmsub16)
 RVVCALL(OPFVV3, vfmsub_vv_w, OP_UUU_W, H4, H4, H4, fmsub32)
 RVVCALL(OPFVV3, vfmsub_vv_d, OP_UUU_D, H8, H8, H8, fmsub64)
-GEN_VEXT_VV_ENV(vfmsub_vv_h)
-GEN_VEXT_VV_ENV(vfmsub_vv_w)
-GEN_VEXT_VV_ENV(vfmsub_vv_d)
+GEN_VEXT_VV_ENV(vfmsub_vv_h, 2)
+GEN_VEXT_VV_ENV(vfmsub_vv_w, 4)
+GEN_VEXT_VV_ENV(vfmsub_vv_d, 8)
 RVVCALL(OPFVF3, vfmsub_vf_h, OP_UUU_H, H2, H2, fmsub16)
 RVVCALL(OPFVF3, vfmsub_vf_w, OP_UUU_W, H4, H4, fmsub32)
 RVVCALL(OPFVF3, vfmsub_vf_d, OP_UUU_D, H8, H8, fmsub64)
-GEN_VEXT_VF(vfmsub_vf_h)
-GEN_VEXT_VF(vfmsub_vf_w)
-GEN_VEXT_VF(vfmsub_vf_d)
+GEN_VEXT_VF(vfmsub_vf_h, 2)
+GEN_VEXT_VF(vfmsub_vf_w, 4)
+GEN_VEXT_VF(vfmsub_vf_d, 8)
 
 static uint16_t fnmsub16(uint16_t a, uint16_t b, uint16_t d, float_status *s)
 {
@@ -3482,15 +3494,15 @@ static uint64_t fnmsub64(uint64_t a, uint64_t b, uint64_t d, float_status *s)
 RVVCALL(OPFVV3, vfnmsub_vv_h, OP_UUU_H, H2, H2, H2, fnmsub16)
 RVVCALL(OPFVV3, vfnmsub_vv_w, OP_UUU_W, H4, H4, H4, fnmsub32)
 RVVCALL(OPFVV3, vfnmsub_vv_d, OP_UUU_D, H8, H8, H8, fnmsub64)
-GEN_VEXT_VV_ENV(vfnmsub_vv_h)
-GEN_VEXT_VV_ENV(vfnmsub_vv_w)
-GEN_VEXT_VV_ENV(vfnmsub_vv_d)
+GEN_VEXT_VV_ENV(vfnmsub_vv_h, 2)
+GEN_VEXT_VV_ENV(vfnmsub_vv_w, 4)
+GEN_VEXT_VV_ENV(vfnmsub_vv_d, 8)
 RVVCALL(OPFVF3, vfnmsub_vf_h, OP_UUU_H, H2, H2, fnmsub16)
 RVVCALL(OPFVF3, vfnmsub_vf_w, OP_UUU_W, H4, H4, fnmsub32)
 RVVCALL(OPFVF3, vfnmsub_vf_d, OP_UUU_D, H8, H8, fnmsub64)
-GEN_VEXT_VF(vfnmsub_vf_h)
-GEN_VEXT_VF(vfnmsub_vf_w)
-GEN_VEXT_VF(vfnmsub_vf_d)
+GEN_VEXT_VF(vfnmsub_vf_h, 2)
+GEN_VEXT_VF(vfnmsub_vf_w, 4)
+GEN_VEXT_VF(vfnmsub_vf_d, 8)
 
 /* Vector Widening Floating-Point Fused Multiply-Add Instructions */
 static uint32_t fwmacc16(uint16_t a, uint16_t b, uint32_t d, float_status *s)
@@ -3507,12 +3519,12 @@ static uint64_t fwmacc32(uint32_t a, uint32_t b, uint64_t d, float_status *s)
 
 RVVCALL(OPFVV3, vfwmacc_vv_h, WOP_UUU_H, H4, H2, H2, fwmacc16)
 RVVCALL(OPFVV3, vfwmacc_vv_w, WOP_UUU_W, H8, H4, H4, fwmacc32)
-GEN_VEXT_VV_ENV(vfwmacc_vv_h)
-GEN_VEXT_VV_ENV(vfwmacc_vv_w)
+GEN_VEXT_VV_ENV(vfwmacc_vv_h, 4)
+GEN_VEXT_VV_ENV(vfwmacc_vv_w, 8)
 RVVCALL(OPFVF3, vfwmacc_vf_h, WOP_UUU_H, H4, H2, fwmacc16)
 RVVCALL(OPFVF3, vfwmacc_vf_w, WOP_UUU_W, H8, H4, fwmacc32)
-GEN_VEXT_VF(vfwmacc_vf_h)
-GEN_VEXT_VF(vfwmacc_vf_w)
+GEN_VEXT_VF(vfwmacc_vf_h, 4)
+GEN_VEXT_VF(vfwmacc_vf_w, 8)
 
 static uint32_t fwnmacc16(uint16_t a, uint16_t b, uint32_t d, float_status *s)
 {
@@ -3530,12 +3542,12 @@ static uint64_t fwnmacc32(uint32_t a, uint32_t b, uint64_t d, float_status *s)
 
 RVVCALL(OPFVV3, vfwnmacc_vv_h, WOP_UUU_H, H4, H2, H2, fwnmacc16)
 RVVCALL(OPFVV3, vfwnmacc_vv_w, WOP_UUU_W, H8, H4, H4, fwnmacc32)
-GEN_VEXT_VV_ENV(vfwnmacc_vv_h)
-GEN_VEXT_VV_ENV(vfwnmacc_vv_w)
+GEN_VEXT_VV_ENV(vfwnmacc_vv_h, 4)
+GEN_VEXT_VV_ENV(vfwnmacc_vv_w, 8)
 RVVCALL(OPFVF3, vfwnmacc_vf_h, WOP_UUU_H, H4, H2, fwnmacc16)
 RVVCALL(OPFVF3, vfwnmacc_vf_w, WOP_UUU_W, H8, H4, fwnmacc32)
-GEN_VEXT_VF(vfwnmacc_vf_h)
-GEN_VEXT_VF(vfwnmacc_vf_w)
+GEN_VEXT_VF(vfwnmacc_vf_h, 4)
+GEN_VEXT_VF(vfwnmacc_vf_w, 8)
 
 static uint32_t fwmsac16(uint16_t a, uint16_t b, uint32_t d, float_status *s)
 {
@@ -3553,12 +3565,12 @@ static uint64_t fwmsac32(uint32_t a, uint32_t b, uint64_t d, float_status *s)
 
 RVVCALL(OPFVV3, vfwmsac_vv_h, WOP_UUU_H, H4, H2, H2, fwmsac16)
 RVVCALL(OPFVV3, vfwmsac_vv_w, WOP_UUU_W, H8, H4, H4, fwmsac32)
-GEN_VEXT_VV_ENV(vfwmsac_vv_h)
-GEN_VEXT_VV_ENV(vfwmsac_vv_w)
+GEN_VEXT_VV_ENV(vfwmsac_vv_h, 4)
+GEN_VEXT_VV_ENV(vfwmsac_vv_w, 8)
 RVVCALL(OPFVF3, vfwmsac_vf_h, WOP_UUU_H, H4, H2, fwmsac16)
 RVVCALL(OPFVF3, vfwmsac_vf_w, WOP_UUU_W, H8, H4, fwmsac32)
-GEN_VEXT_VF(vfwmsac_vf_h)
-GEN_VEXT_VF(vfwmsac_vf_w)
+GEN_VEXT_VF(vfwmsac_vf_h, 4)
+GEN_VEXT_VF(vfwmsac_vf_w, 8)
 
 static uint32_t fwnmsac16(uint16_t a, uint16_t b, uint32_t d, float_status *s)
 {
@@ -3576,12 +3588,12 @@ static uint64_t fwnmsac32(uint32_t a, uint32_t b, uint64_t d, float_status *s)
 
 RVVCALL(OPFVV3, vfwnmsac_vv_h, WOP_UUU_H, H4, H2, H2, fwnmsac16)
 RVVCALL(OPFVV3, vfwnmsac_vv_w, WOP_UUU_W, H8, H4, H4, fwnmsac32)
-GEN_VEXT_VV_ENV(vfwnmsac_vv_h)
-GEN_VEXT_VV_ENV(vfwnmsac_vv_w)
+GEN_VEXT_VV_ENV(vfwnmsac_vv_h, 4)
+GEN_VEXT_VV_ENV(vfwnmsac_vv_w, 8)
 RVVCALL(OPFVF3, vfwnmsac_vf_h, WOP_UUU_H, H4, H2, fwnmsac16)
 RVVCALL(OPFVF3, vfwnmsac_vf_w, WOP_UUU_W, H8, H4, fwnmsac32)
-GEN_VEXT_VF(vfwnmsac_vf_h)
-GEN_VEXT_VF(vfwnmsac_vf_w)
+GEN_VEXT_VF(vfwnmsac_vf_h, 4)
+GEN_VEXT_VF(vfwnmsac_vf_w, 8)
 
 /* Vector Floating-Point Square-Root Instruction */
 /* (TD, T2, TX2) */
@@ -3597,12 +3609,15 @@ static void do_##NAME(void *vd, void *vs2, int i,      \
     *((TD *)vd + HD(i)) = OP(s2, &env->fp_status);     \
 }
 
-#define GEN_VEXT_V_ENV(NAME)                           \
+#define GEN_VEXT_V_ENV(NAME, ESZ)                      \
 void HELPER(NAME)(void *vd, void *v0, void *vs2,       \
         CPURISCVState *env, uint32_t desc)             \
 {                                                      \
     uint32_t vm = vext_vm(desc);                       \
     uint32_t vl = env->vl;                             \
+    uint32_t total_elems =                             \
+        vext_get_total_elems(env, desc, ESZ);          \
+    uint32_t vta = vext_vta(desc);                     \
     uint32_t i;                                        \
                                                        \
     if (vl == 0) {                                     \
@@ -3615,14 +3630,16 @@ void HELPER(NAME)(void *vd, void *v0, void *vs2,       \
         do_##NAME(vd, vs2, i, env);                    \
     }                                                  \
     env->vstart = 0;                                   \
+    vext_set_elems_1s(vd, vta, vl * ESZ,               \
+                      total_elems * ESZ);              \
 }
 
 RVVCALL(OPFVV1, vfsqrt_v_h, OP_UU_H, H2, H2, float16_sqrt)
 RVVCALL(OPFVV1, vfsqrt_v_w, OP_UU_W, H4, H4, float32_sqrt)
 RVVCALL(OPFVV1, vfsqrt_v_d, OP_UU_D, H8, H8, float64_sqrt)
-GEN_VEXT_V_ENV(vfsqrt_v_h)
-GEN_VEXT_V_ENV(vfsqrt_v_w)
-GEN_VEXT_V_ENV(vfsqrt_v_d)
+GEN_VEXT_V_ENV(vfsqrt_v_h, 2)
+GEN_VEXT_V_ENV(vfsqrt_v_w, 4)
+GEN_VEXT_V_ENV(vfsqrt_v_d, 8)
 
 /*
  * Vector Floating-Point Reciprocal Square-Root Estimate Instruction
@@ -3802,9 +3819,9 @@ static float64 frsqrt7_d(float64 f, float_status *s)
 RVVCALL(OPFVV1, vfrsqrt7_v_h, OP_UU_H, H2, H2, frsqrt7_h)
 RVVCALL(OPFVV1, vfrsqrt7_v_w, OP_UU_W, H4, H4, frsqrt7_s)
 RVVCALL(OPFVV1, vfrsqrt7_v_d, OP_UU_D, H8, H8, frsqrt7_d)
-GEN_VEXT_V_ENV(vfrsqrt7_v_h)
-GEN_VEXT_V_ENV(vfrsqrt7_v_w)
-GEN_VEXT_V_ENV(vfrsqrt7_v_d)
+GEN_VEXT_V_ENV(vfrsqrt7_v_h, 2)
+GEN_VEXT_V_ENV(vfrsqrt7_v_w, 4)
+GEN_VEXT_V_ENV(vfrsqrt7_v_d, 8)
 
 /*
  * Vector Floating-Point Reciprocal Estimate Instruction
@@ -3993,36 +4010,36 @@ static float64 frec7_d(float64 f, float_status *s)
 RVVCALL(OPFVV1, vfrec7_v_h, OP_UU_H, H2, H2, frec7_h)
 RVVCALL(OPFVV1, vfrec7_v_w, OP_UU_W, H4, H4, frec7_s)
 RVVCALL(OPFVV1, vfrec7_v_d, OP_UU_D, H8, H8, frec7_d)
-GEN_VEXT_V_ENV(vfrec7_v_h)
-GEN_VEXT_V_ENV(vfrec7_v_w)
-GEN_VEXT_V_ENV(vfrec7_v_d)
+GEN_VEXT_V_ENV(vfrec7_v_h, 2)
+GEN_VEXT_V_ENV(vfrec7_v_w, 4)
+GEN_VEXT_V_ENV(vfrec7_v_d, 8)
 
 /* Vector Floating-Point MIN/MAX Instructions */
 RVVCALL(OPFVV2, vfmin_vv_h, OP_UUU_H, H2, H2, H2, float16_minimum_number)
 RVVCALL(OPFVV2, vfmin_vv_w, OP_UUU_W, H4, H4, H4, float32_minimum_number)
 RVVCALL(OPFVV2, vfmin_vv_d, OP_UUU_D, H8, H8, H8, float64_minimum_number)
-GEN_VEXT_VV_ENV(vfmin_vv_h)
-GEN_VEXT_VV_ENV(vfmin_vv_w)
-GEN_VEXT_VV_ENV(vfmin_vv_d)
+GEN_VEXT_VV_ENV(vfmin_vv_h, 2)
+GEN_VEXT_VV_ENV(vfmin_vv_w, 4)
+GEN_VEXT_VV_ENV(vfmin_vv_d, 8)
 RVVCALL(OPFVF2, vfmin_vf_h, OP_UUU_H, H2, H2, float16_minimum_number)
 RVVCALL(OPFVF2, vfmin_vf_w, OP_UUU_W, H4, H4, float32_minimum_number)
 RVVCALL(OPFVF2, vfmin_vf_d, OP_UUU_D, H8, H8, float64_minimum_number)
-GEN_VEXT_VF(vfmin_vf_h)
-GEN_VEXT_VF(vfmin_vf_w)
-GEN_VEXT_VF(vfmin_vf_d)
+GEN_VEXT_VF(vfmin_vf_h, 2)
+GEN_VEXT_VF(vfmin_vf_w, 4)
+GEN_VEXT_VF(vfmin_vf_d, 8)
 
 RVVCALL(OPFVV2, vfmax_vv_h, OP_UUU_H, H2, H2, H2, float16_maximum_number)
 RVVCALL(OPFVV2, vfmax_vv_w, OP_UUU_W, H4, H4, H4, float32_maximum_number)
 RVVCALL(OPFVV2, vfmax_vv_d, OP_UUU_D, H8, H8, H8, float64_maximum_number)
-GEN_VEXT_VV_ENV(vfmax_vv_h)
-GEN_VEXT_VV_ENV(vfmax_vv_w)
-GEN_VEXT_VV_ENV(vfmax_vv_d)
+GEN_VEXT_VV_ENV(vfmax_vv_h, 2)
+GEN_VEXT_VV_ENV(vfmax_vv_w, 4)
+GEN_VEXT_VV_ENV(vfmax_vv_d, 8)
 RVVCALL(OPFVF2, vfmax_vf_h, OP_UUU_H, H2, H2, float16_maximum_number)
 RVVCALL(OPFVF2, vfmax_vf_w, OP_UUU_W, H4, H4, float32_maximum_number)
 RVVCALL(OPFVF2, vfmax_vf_d, OP_UUU_D, H8, H8, float64_maximum_number)
-GEN_VEXT_VF(vfmax_vf_h)
-GEN_VEXT_VF(vfmax_vf_w)
-GEN_VEXT_VF(vfmax_vf_d)
+GEN_VEXT_VF(vfmax_vf_h, 2)
+GEN_VEXT_VF(vfmax_vf_w, 4)
+GEN_VEXT_VF(vfmax_vf_d, 8)
 
 /* Vector Floating-Point Sign-Injection Instructions */
 static uint16_t fsgnj16(uint16_t a, uint16_t b, float_status *s)
@@ -4043,15 +4060,15 @@ static uint64_t fsgnj64(uint64_t a, uint64_t b, float_status *s)
 RVVCALL(OPFVV2, vfsgnj_vv_h, OP_UUU_H, H2, H2, H2, fsgnj16)
 RVVCALL(OPFVV2, vfsgnj_vv_w, OP_UUU_W, H4, H4, H4, fsgnj32)
 RVVCALL(OPFVV2, vfsgnj_vv_d, OP_UUU_D, H8, H8, H8, fsgnj64)
-GEN_VEXT_VV_ENV(vfsgnj_vv_h)
-GEN_VEXT_VV_ENV(vfsgnj_vv_w)
-GEN_VEXT_VV_ENV(vfsgnj_vv_d)
+GEN_VEXT_VV_ENV(vfsgnj_vv_h, 2)
+GEN_VEXT_VV_ENV(vfsgnj_vv_w, 4)
+GEN_VEXT_VV_ENV(vfsgnj_vv_d, 8)
 RVVCALL(OPFVF2, vfsgnj_vf_h, OP_UUU_H, H2, H2, fsgnj16)
 RVVCALL(OPFVF2, vfsgnj_vf_w, OP_UUU_W, H4, H4, fsgnj32)
 RVVCALL(OPFVF2, vfsgnj_vf_d, OP_UUU_D, H8, H8, fsgnj64)
-GEN_VEXT_VF(vfsgnj_vf_h)
-GEN_VEXT_VF(vfsgnj_vf_w)
-GEN_VEXT_VF(vfsgnj_vf_d)
+GEN_VEXT_VF(vfsgnj_vf_h, 2)
+GEN_VEXT_VF(vfsgnj_vf_w, 4)
+GEN_VEXT_VF(vfsgnj_vf_d, 8)
 
 static uint16_t fsgnjn16(uint16_t a, uint16_t b, float_status *s)
 {
@@ -4071,15 +4088,15 @@ static uint64_t fsgnjn64(uint64_t a, uint64_t b, float_status *s)
 RVVCALL(OPFVV2, vfsgnjn_vv_h, OP_UUU_H, H2, H2, H2, fsgnjn16)
 RVVCALL(OPFVV2, vfsgnjn_vv_w, OP_UUU_W, H4, H4, H4, fsgnjn32)
 RVVCALL(OPFVV2, vfsgnjn_vv_d, OP_UUU_D, H8, H8, H8, fsgnjn64)
-GEN_VEXT_VV_ENV(vfsgnjn_vv_h)
-GEN_VEXT_VV_ENV(vfsgnjn_vv_w)
-GEN_VEXT_VV_ENV(vfsgnjn_vv_d)
+GEN_VEXT_VV_ENV(vfsgnjn_vv_h, 2)
+GEN_VEXT_VV_ENV(vfsgnjn_vv_w, 4)
+GEN_VEXT_VV_ENV(vfsgnjn_vv_d, 8)
 RVVCALL(OPFVF2, vfsgnjn_vf_h, OP_UUU_H, H2, H2, fsgnjn16)
 RVVCALL(OPFVF2, vfsgnjn_vf_w, OP_UUU_W, H4, H4, fsgnjn32)
 RVVCALL(OPFVF2, vfsgnjn_vf_d, OP_UUU_D, H8, H8, fsgnjn64)
-GEN_VEXT_VF(vfsgnjn_vf_h)
-GEN_VEXT_VF(vfsgnjn_vf_w)
-GEN_VEXT_VF(vfsgnjn_vf_d)
+GEN_VEXT_VF(vfsgnjn_vf_h, 2)
+GEN_VEXT_VF(vfsgnjn_vf_w, 4)
+GEN_VEXT_VF(vfsgnjn_vf_d, 8)
 
 static uint16_t fsgnjx16(uint16_t a, uint16_t b, float_status *s)
 {
@@ -4099,15 +4116,15 @@ static uint64_t fsgnjx64(uint64_t a, uint64_t b, float_status *s)
 RVVCALL(OPFVV2, vfsgnjx_vv_h, OP_UUU_H, H2, H2, H2, fsgnjx16)
 RVVCALL(OPFVV2, vfsgnjx_vv_w, OP_UUU_W, H4, H4, H4, fsgnjx32)
 RVVCALL(OPFVV2, vfsgnjx_vv_d, OP_UUU_D, H8, H8, H8, fsgnjx64)
-GEN_VEXT_VV_ENV(vfsgnjx_vv_h)
-GEN_VEXT_VV_ENV(vfsgnjx_vv_w)
-GEN_VEXT_VV_ENV(vfsgnjx_vv_d)
+GEN_VEXT_VV_ENV(vfsgnjx_vv_h, 2)
+GEN_VEXT_VV_ENV(vfsgnjx_vv_w, 4)
+GEN_VEXT_VV_ENV(vfsgnjx_vv_d, 8)
 RVVCALL(OPFVF2, vfsgnjx_vf_h, OP_UUU_H, H2, H2, fsgnjx16)
 RVVCALL(OPFVF2, vfsgnjx_vf_w, OP_UUU_W, H4, H4, fsgnjx32)
 RVVCALL(OPFVF2, vfsgnjx_vf_d, OP_UUU_D, H8, H8, fsgnjx64)
-GEN_VEXT_VF(vfsgnjx_vf_h)
-GEN_VEXT_VF(vfsgnjx_vf_w)
-GEN_VEXT_VF(vfsgnjx_vf_d)
+GEN_VEXT_VF(vfsgnjx_vf_h, 2)
+GEN_VEXT_VF(vfsgnjx_vf_w, 4)
+GEN_VEXT_VF(vfsgnjx_vf_d, 8)
 
 /* Vector Floating-Point Compare Instructions */
 #define GEN_VEXT_CMP_VV_ENV(NAME, ETYPE, H, DO_OP)            \
@@ -4116,6 +4133,8 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1, void *vs2,   \
 {                                                             \
     uint32_t vm = vext_vm(desc);                              \
     uint32_t vl = env->vl;                                    \
+    uint32_t total_elems = env_archcpu(env)->cfg.vlen;        \
+    uint32_t vta_all_1s = vext_vta_all_1s(desc);              \
     uint32_t i;                                               \
                                                               \
     for (i = env->vstart; i < vl; i++) {                      \
@@ -4128,6 +4147,13 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1, void *vs2,   \
                            DO_OP(s2, s1, &env->fp_status));   \
     }                                                         \
     env->vstart = 0;                                          \
+    /* mask destination register are always tail-agnostic */  \
+    /* set tail elements to 1s */                             \
+    if (vta_all_1s) {                                         \
+        for (; i < total_elems; i++) {                        \
+            vext_set_elem_mask(vd, i, 1);                     \
+        }                                                     \
+    }                                                         \
 }
 
 GEN_VEXT_CMP_VV_ENV(vmfeq_vv_h, uint16_t, H2, float16_eq_quiet)
@@ -4140,6 +4166,8 @@ void HELPER(NAME)(void *vd, void *v0, uint64_t s1, void *vs2,       \
 {                                                                   \
     uint32_t vm = vext_vm(desc);                                    \
     uint32_t vl = env->vl;                                          \
+    uint32_t total_elems = env_archcpu(env)->cfg.vlen;              \
+    uint32_t vta_all_1s = vext_vta_all_1s(desc);                    \
     uint32_t i;                                                     \
                                                                     \
     for (i = env->vstart; i < vl; i++) {                            \
@@ -4151,6 +4179,13 @@ void HELPER(NAME)(void *vd, void *v0, uint64_t s1, void *vs2,       \
                            DO_OP(s2, (ETYPE)s1, &env->fp_status));  \
     }                                                               \
     env->vstart = 0;                                                \
+    /* mask destination register are always tail-agnostic */        \
+    /* set tail elements to 1s */                                   \
+    if (vta_all_1s) {                                               \
+        for (; i < total_elems; i++) {                              \
+            vext_set_elem_mask(vd, i, 1);                           \
+        }                                                           \
+    }                                                               \
 }
 
 GEN_VEXT_CMP_VF(vmfeq_vf_h, uint16_t, H2, float16_eq_quiet)
@@ -4251,12 +4286,15 @@ static void do_##NAME(void *vd, void *vs2, int i)      \
     *((TD *)vd + HD(i)) = OP(s2);                      \
 }
 
-#define GEN_VEXT_V(NAME)                               \
+#define GEN_VEXT_V(NAME, ESZ)                          \
 void HELPER(NAME)(void *vd, void *v0, void *vs2,       \
                   CPURISCVState *env, uint32_t desc)   \
 {                                                      \
     uint32_t vm = vext_vm(desc);                       \
     uint32_t vl = env->vl;                             \
+    uint32_t total_elems =                             \
+        vext_get_total_elems(env, desc, ESZ);          \
+    uint32_t vta = vext_vta(desc);                     \
     uint32_t i;                                        \
                                                        \
     for (i = env->vstart; i < vl; i++) {               \
@@ -4266,6 +4304,9 @@ void HELPER(NAME)(void *vd, void *v0, void *vs2,       \
         do_##NAME(vd, vs2, i);                         \
     }                                                  \
     env->vstart = 0;                                   \
+    /* set tail elements to 1s */                      \
+    vext_set_elems_1s(vd, vta, vl * ESZ,               \
+                      total_elems * ESZ);              \
 }
 
 target_ulong fclass_h(uint64_t frs1)
@@ -4328,17 +4369,22 @@ target_ulong fclass_d(uint64_t frs1)
 RVVCALL(OPIVV1, vfclass_v_h, OP_UU_H, H2, H2, fclass_h)
 RVVCALL(OPIVV1, vfclass_v_w, OP_UU_W, H4, H4, fclass_s)
 RVVCALL(OPIVV1, vfclass_v_d, OP_UU_D, H8, H8, fclass_d)
-GEN_VEXT_V(vfclass_v_h)
-GEN_VEXT_V(vfclass_v_w)
-GEN_VEXT_V(vfclass_v_d)
+GEN_VEXT_V(vfclass_v_h, 2)
+GEN_VEXT_V(vfclass_v_w, 4)
+GEN_VEXT_V(vfclass_v_d, 8)
 
 /* Vector Floating-Point Merge Instruction */
+
 #define GEN_VFMERGE_VF(NAME, ETYPE, H)                        \
 void HELPER(NAME)(void *vd, void *v0, uint64_t s1, void *vs2, \
                   CPURISCVState *env, uint32_t desc)          \
 {                                                             \
     uint32_t vm = vext_vm(desc);                              \
     uint32_t vl = env->vl;                                    \
+    uint32_t esz = sizeof(ETYPE);                             \
+    uint32_t total_elems =                                    \
+        vext_get_total_elems(env, desc, esz);                 \
+    uint32_t vta = vext_vta(desc);                            \
     uint32_t i;                                               \
                                                               \
     for (i = env->vstart; i < vl; i++) {                      \
@@ -4347,6 +4393,8 @@ void HELPER(NAME)(void *vd, void *v0, uint64_t s1, void *vs2, \
           = (!vm && !vext_elem_mask(v0, i) ? s2 : s1);        \
     }                                                         \
     env->vstart = 0;                                          \
+    /* set tail elements to 1s */                             \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);  \
 }
 
 GEN_VFMERGE_VF(vfmerge_vfm_h, int16_t, H2)
@@ -4358,33 +4406,33 @@ GEN_VFMERGE_VF(vfmerge_vfm_d, int64_t, H8)
 RVVCALL(OPFVV1, vfcvt_xu_f_v_h, OP_UU_H, H2, H2, float16_to_uint16)
 RVVCALL(OPFVV1, vfcvt_xu_f_v_w, OP_UU_W, H4, H4, float32_to_uint32)
 RVVCALL(OPFVV1, vfcvt_xu_f_v_d, OP_UU_D, H8, H8, float64_to_uint64)
-GEN_VEXT_V_ENV(vfcvt_xu_f_v_h)
-GEN_VEXT_V_ENV(vfcvt_xu_f_v_w)
-GEN_VEXT_V_ENV(vfcvt_xu_f_v_d)
+GEN_VEXT_V_ENV(vfcvt_xu_f_v_h, 2)
+GEN_VEXT_V_ENV(vfcvt_xu_f_v_w, 4)
+GEN_VEXT_V_ENV(vfcvt_xu_f_v_d, 8)
 
 /* vfcvt.x.f.v vd, vs2, vm # Convert float to signed integer. */
 RVVCALL(OPFVV1, vfcvt_x_f_v_h, OP_UU_H, H2, H2, float16_to_int16)
 RVVCALL(OPFVV1, vfcvt_x_f_v_w, OP_UU_W, H4, H4, float32_to_int32)
 RVVCALL(OPFVV1, vfcvt_x_f_v_d, OP_UU_D, H8, H8, float64_to_int64)
-GEN_VEXT_V_ENV(vfcvt_x_f_v_h)
-GEN_VEXT_V_ENV(vfcvt_x_f_v_w)
-GEN_VEXT_V_ENV(vfcvt_x_f_v_d)
+GEN_VEXT_V_ENV(vfcvt_x_f_v_h, 2)
+GEN_VEXT_V_ENV(vfcvt_x_f_v_w, 4)
+GEN_VEXT_V_ENV(vfcvt_x_f_v_d, 8)
 
 /* vfcvt.f.xu.v vd, vs2, vm # Convert unsigned integer to float. */
 RVVCALL(OPFVV1, vfcvt_f_xu_v_h, OP_UU_H, H2, H2, uint16_to_float16)
 RVVCALL(OPFVV1, vfcvt_f_xu_v_w, OP_UU_W, H4, H4, uint32_to_float32)
 RVVCALL(OPFVV1, vfcvt_f_xu_v_d, OP_UU_D, H8, H8, uint64_to_float64)
-GEN_VEXT_V_ENV(vfcvt_f_xu_v_h)
-GEN_VEXT_V_ENV(vfcvt_f_xu_v_w)
-GEN_VEXT_V_ENV(vfcvt_f_xu_v_d)
+GEN_VEXT_V_ENV(vfcvt_f_xu_v_h, 2)
+GEN_VEXT_V_ENV(vfcvt_f_xu_v_w, 4)
+GEN_VEXT_V_ENV(vfcvt_f_xu_v_d, 8)
 
 /* vfcvt.f.x.v vd, vs2, vm # Convert integer to float. */
 RVVCALL(OPFVV1, vfcvt_f_x_v_h, OP_UU_H, H2, H2, int16_to_float16)
 RVVCALL(OPFVV1, vfcvt_f_x_v_w, OP_UU_W, H4, H4, int32_to_float32)
 RVVCALL(OPFVV1, vfcvt_f_x_v_d, OP_UU_D, H8, H8, int64_to_float64)
-GEN_VEXT_V_ENV(vfcvt_f_x_v_h)
-GEN_VEXT_V_ENV(vfcvt_f_x_v_w)
-GEN_VEXT_V_ENV(vfcvt_f_x_v_d)
+GEN_VEXT_V_ENV(vfcvt_f_x_v_h, 2)
+GEN_VEXT_V_ENV(vfcvt_f_x_v_w, 4)
+GEN_VEXT_V_ENV(vfcvt_f_x_v_d, 8)
 
 /* Widening Floating-Point/Integer Type-Convert Instructions */
 /* (TD, T2, TX2) */
@@ -4394,30 +4442,30 @@ GEN_VEXT_V_ENV(vfcvt_f_x_v_d)
 /* vfwcvt.xu.f.v vd, vs2, vm # Convert float to double-width unsigned integer.*/
 RVVCALL(OPFVV1, vfwcvt_xu_f_v_h, WOP_UU_H, H4, H2, float16_to_uint32)
 RVVCALL(OPFVV1, vfwcvt_xu_f_v_w, WOP_UU_W, H8, H4, float32_to_uint64)
-GEN_VEXT_V_ENV(vfwcvt_xu_f_v_h)
-GEN_VEXT_V_ENV(vfwcvt_xu_f_v_w)
+GEN_VEXT_V_ENV(vfwcvt_xu_f_v_h, 4)
+GEN_VEXT_V_ENV(vfwcvt_xu_f_v_w, 8)
 
 /* vfwcvt.x.f.v vd, vs2, vm # Convert float to double-width signed integer. */
 RVVCALL(OPFVV1, vfwcvt_x_f_v_h, WOP_UU_H, H4, H2, float16_to_int32)
 RVVCALL(OPFVV1, vfwcvt_x_f_v_w, WOP_UU_W, H8, H4, float32_to_int64)
-GEN_VEXT_V_ENV(vfwcvt_x_f_v_h)
-GEN_VEXT_V_ENV(vfwcvt_x_f_v_w)
+GEN_VEXT_V_ENV(vfwcvt_x_f_v_h, 4)
+GEN_VEXT_V_ENV(vfwcvt_x_f_v_w, 8)
 
 /* vfwcvt.f.xu.v vd, vs2, vm # Convert unsigned integer to double-width float */
 RVVCALL(OPFVV1, vfwcvt_f_xu_v_b, WOP_UU_B, H2, H1, uint8_to_float16)
 RVVCALL(OPFVV1, vfwcvt_f_xu_v_h, WOP_UU_H, H4, H2, uint16_to_float32)
 RVVCALL(OPFVV1, vfwcvt_f_xu_v_w, WOP_UU_W, H8, H4, uint32_to_float64)
-GEN_VEXT_V_ENV(vfwcvt_f_xu_v_b)
-GEN_VEXT_V_ENV(vfwcvt_f_xu_v_h)
-GEN_VEXT_V_ENV(vfwcvt_f_xu_v_w)
+GEN_VEXT_V_ENV(vfwcvt_f_xu_v_b, 2)
+GEN_VEXT_V_ENV(vfwcvt_f_xu_v_h, 4)
+GEN_VEXT_V_ENV(vfwcvt_f_xu_v_w, 8)
 
 /* vfwcvt.f.x.v vd, vs2, vm # Convert integer to double-width float. */
 RVVCALL(OPFVV1, vfwcvt_f_x_v_b, WOP_UU_B, H2, H1, int8_to_float16)
 RVVCALL(OPFVV1, vfwcvt_f_x_v_h, WOP_UU_H, H4, H2, int16_to_float32)
 RVVCALL(OPFVV1, vfwcvt_f_x_v_w, WOP_UU_W, H8, H4, int32_to_float64)
-GEN_VEXT_V_ENV(vfwcvt_f_x_v_b)
-GEN_VEXT_V_ENV(vfwcvt_f_x_v_h)
-GEN_VEXT_V_ENV(vfwcvt_f_x_v_w)
+GEN_VEXT_V_ENV(vfwcvt_f_x_v_b, 2)
+GEN_VEXT_V_ENV(vfwcvt_f_x_v_h, 4)
+GEN_VEXT_V_ENV(vfwcvt_f_x_v_w, 8)
 
 /*
  * vfwcvt.f.f.v vd, vs2, vm
@@ -4430,8 +4478,8 @@ static uint32_t vfwcvtffv16(uint16_t a, float_status *s)
 
 RVVCALL(OPFVV1, vfwcvt_f_f_v_h, WOP_UU_H, H4, H2, vfwcvtffv16)
 RVVCALL(OPFVV1, vfwcvt_f_f_v_w, WOP_UU_W, H8, H4, float32_to_float64)
-GEN_VEXT_V_ENV(vfwcvt_f_f_v_h)
-GEN_VEXT_V_ENV(vfwcvt_f_f_v_w)
+GEN_VEXT_V_ENV(vfwcvt_f_f_v_h, 4)
+GEN_VEXT_V_ENV(vfwcvt_f_f_v_w, 8)
 
 /* Narrowing Floating-Point/Integer Type-Convert Instructions */
 /* (TD, T2, TX2) */
@@ -4442,29 +4490,29 @@ GEN_VEXT_V_ENV(vfwcvt_f_f_v_w)
 RVVCALL(OPFVV1, vfncvt_xu_f_w_b, NOP_UU_B, H1, H2, float16_to_uint8)
 RVVCALL(OPFVV1, vfncvt_xu_f_w_h, NOP_UU_H, H2, H4, float32_to_uint16)
 RVVCALL(OPFVV1, vfncvt_xu_f_w_w, NOP_UU_W, H4, H8, float64_to_uint32)
-GEN_VEXT_V_ENV(vfncvt_xu_f_w_b)
-GEN_VEXT_V_ENV(vfncvt_xu_f_w_h)
-GEN_VEXT_V_ENV(vfncvt_xu_f_w_w)
+GEN_VEXT_V_ENV(vfncvt_xu_f_w_b, 1)
+GEN_VEXT_V_ENV(vfncvt_xu_f_w_h, 2)
+GEN_VEXT_V_ENV(vfncvt_xu_f_w_w, 4)
 
 /* vfncvt.x.f.v vd, vs2, vm # Convert double-width float to signed integer. */
 RVVCALL(OPFVV1, vfncvt_x_f_w_b, NOP_UU_B, H1, H2, float16_to_int8)
 RVVCALL(OPFVV1, vfncvt_x_f_w_h, NOP_UU_H, H2, H4, float32_to_int16)
 RVVCALL(OPFVV1, vfncvt_x_f_w_w, NOP_UU_W, H4, H8, float64_to_int32)
-GEN_VEXT_V_ENV(vfncvt_x_f_w_b)
-GEN_VEXT_V_ENV(vfncvt_x_f_w_h)
-GEN_VEXT_V_ENV(vfncvt_x_f_w_w)
+GEN_VEXT_V_ENV(vfncvt_x_f_w_b, 1)
+GEN_VEXT_V_ENV(vfncvt_x_f_w_h, 2)
+GEN_VEXT_V_ENV(vfncvt_x_f_w_w, 4)
 
 /* vfncvt.f.xu.v vd, vs2, vm # Convert double-width unsigned integer to float */
 RVVCALL(OPFVV1, vfncvt_f_xu_w_h, NOP_UU_H, H2, H4, uint32_to_float16)
 RVVCALL(OPFVV1, vfncvt_f_xu_w_w, NOP_UU_W, H4, H8, uint64_to_float32)
-GEN_VEXT_V_ENV(vfncvt_f_xu_w_h)
-GEN_VEXT_V_ENV(vfncvt_f_xu_w_w)
+GEN_VEXT_V_ENV(vfncvt_f_xu_w_h, 2)
+GEN_VEXT_V_ENV(vfncvt_f_xu_w_w, 4)
 
 /* vfncvt.f.x.v vd, vs2, vm # Convert double-width integer to float. */
 RVVCALL(OPFVV1, vfncvt_f_x_w_h, NOP_UU_H, H2, H4, int32_to_float16)
 RVVCALL(OPFVV1, vfncvt_f_x_w_w, NOP_UU_W, H4, H8, int64_to_float32)
-GEN_VEXT_V_ENV(vfncvt_f_x_w_h)
-GEN_VEXT_V_ENV(vfncvt_f_x_w_w)
+GEN_VEXT_V_ENV(vfncvt_f_x_w_h, 2)
+GEN_VEXT_V_ENV(vfncvt_f_x_w_w, 4)
 
 /* vfncvt.f.f.v vd, vs2, vm # Convert double float to single-width float. */
 static uint16_t vfncvtffv16(uint32_t a, float_status *s)
@@ -4474,8 +4522,8 @@ static uint16_t vfncvtffv16(uint32_t a, float_status *s)
 
 RVVCALL(OPFVV1, vfncvt_f_f_w_h, NOP_UU_H, H2, H4, vfncvtffv16)
 RVVCALL(OPFVV1, vfncvt_f_f_w_w, NOP_UU_W, H4, H8, float64_to_float32)
-GEN_VEXT_V_ENV(vfncvt_f_f_w_h)
-GEN_VEXT_V_ENV(vfncvt_f_f_w_w)
+GEN_VEXT_V_ENV(vfncvt_f_f_w_h, 2)
+GEN_VEXT_V_ENV(vfncvt_f_f_w_w, 4)
 
 /*
  *** Vector Reduction Operations

From df4f52a7582c5cd8d2d2f73684996ebc18ac72e3 Mon Sep 17 00:00:00 2001
From: eopXD <yueh.ting.chen@gmail.com>
Date: Mon, 6 Jun 2022 06:16:16 +0000
Subject: [PATCH 930/935] target/riscv: rvv: Add tail agnostic for vector
 reduction instructions

Signed-off-by: eop Chen <eop.chen@sifive.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <165449614532.19704.7000832880482980398-13@git.sr.ht>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/vector_helper.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c
index 17390d8d06..174a548ac2 100644
--- a/target/riscv/vector_helper.c
+++ b/target/riscv/vector_helper.c
@@ -4535,6 +4535,9 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1,          \
 {                                                         \
     uint32_t vm = vext_vm(desc);                          \
     uint32_t vl = env->vl;                                \
+    uint32_t esz = sizeof(TD);                            \
+    uint32_t vlenb = simd_maxsz(desc);                    \
+    uint32_t vta = vext_vta(desc);                        \
     uint32_t i;                                           \
     TD s1 =  *((TD *)vs1 + HD(0));                        \
                                                           \
@@ -4547,6 +4550,8 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1,          \
     }                                                     \
     *((TD *)vd + HD(0)) = s1;                             \
     env->vstart = 0;                                      \
+    /* set tail elements to 1s */                         \
+    vext_set_elems_1s(vd, vta, esz, vlenb);               \
 }
 
 /* vd[0] = sum(vs1[0], vs2[*]) */
@@ -4616,6 +4621,9 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1,           \
 {                                                          \
     uint32_t vm = vext_vm(desc);                           \
     uint32_t vl = env->vl;                                 \
+    uint32_t esz = sizeof(TD);                             \
+    uint32_t vlenb = simd_maxsz(desc);                     \
+    uint32_t vta = vext_vta(desc);                         \
     uint32_t i;                                            \
     TD s1 =  *((TD *)vs1 + HD(0));                         \
                                                            \
@@ -4628,6 +4636,8 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1,           \
     }                                                      \
     *((TD *)vd + HD(0)) = s1;                              \
     env->vstart = 0;                                       \
+    /* set tail elements to 1s */                          \
+    vext_set_elems_1s(vd, vta, esz, vlenb);                \
 }
 
 /* Unordered sum */
@@ -4652,6 +4662,9 @@ void HELPER(vfwredsum_vs_h)(void *vd, void *v0, void *vs1,
 {
     uint32_t vm = vext_vm(desc);
     uint32_t vl = env->vl;
+    uint32_t esz = sizeof(uint32_t);
+    uint32_t vlenb = simd_maxsz(desc);
+    uint32_t vta = vext_vta(desc);
     uint32_t i;
     uint32_t s1 =  *((uint32_t *)vs1 + H4(0));
 
@@ -4665,6 +4678,8 @@ void HELPER(vfwredsum_vs_h)(void *vd, void *v0, void *vs1,
     }
     *((uint32_t *)vd + H4(0)) = s1;
     env->vstart = 0;
+    /* set tail elements to 1s */
+    vext_set_elems_1s(vd, vta, esz, vlenb);
 }
 
 void HELPER(vfwredsum_vs_w)(void *vd, void *v0, void *vs1,
@@ -4672,6 +4687,9 @@ void HELPER(vfwredsum_vs_w)(void *vd, void *v0, void *vs1,
 {
     uint32_t vm = vext_vm(desc);
     uint32_t vl = env->vl;
+    uint32_t esz = sizeof(uint64_t);
+    uint32_t vlenb = simd_maxsz(desc);
+    uint32_t vta = vext_vta(desc);
     uint32_t i;
     uint64_t s1 =  *((uint64_t *)vs1);
 
@@ -4685,6 +4703,8 @@ void HELPER(vfwredsum_vs_w)(void *vd, void *v0, void *vs1,
     }
     *((uint64_t *)vd) = s1;
     env->vstart = 0;
+    /* set tail elements to 1s */
+    vext_set_elems_1s(vd, vta, esz, vlenb);
 }
 
 /*

From acc6ffd482f6b0435bea7434eb5b13441c6d4a33 Mon Sep 17 00:00:00 2001
From: eopXD <yueh.ting.chen@gmail.com>
Date: Mon, 6 Jun 2022 06:16:35 +0000
Subject: [PATCH 931/935] target/riscv: rvv: Add tail agnostic for vector mask
 instructions

The tail elements in the destination mask register are updated under
a tail-agnostic policy.

Signed-off-by: eop Chen <eop.chen@sifive.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <165449614532.19704.7000832880482980398-14@git.sr.ht>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/insn_trans/trans_rvv.c.inc |  6 +++++
 target/riscv/vector_helper.c            | 30 +++++++++++++++++++++++++
 2 files changed, 36 insertions(+)

diff --git a/target/riscv/insn_trans/trans_rvv.c.inc b/target/riscv/insn_trans/trans_rvv.c.inc
index 1add4cb655..a94e634a6b 100644
--- a/target/riscv/insn_trans/trans_rvv.c.inc
+++ b/target/riscv/insn_trans/trans_rvv.c.inc
@@ -3135,6 +3135,8 @@ static bool trans_##NAME(DisasContext *s, arg_r *a)                \
         tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \
                                                                    \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);             \
+        data =                                                     \
+            FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s);\
         tcg_gen_gvec_4_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),     \
                            vreg_ofs(s, a->rs1),                    \
                            vreg_ofs(s, a->rs2), cpu_env,           \
@@ -3239,6 +3241,8 @@ static bool trans_##NAME(DisasContext *s, arg_rmr *a)              \
                                                                    \
         data = FIELD_DP32(data, VDATA, VM, a->vm);                 \
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);             \
+        data =                                                     \
+            FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s);\
         tcg_gen_gvec_3_ptr(vreg_ofs(s, a->rd),                     \
                            vreg_ofs(s, 0), vreg_ofs(s, a->rs2),    \
                            cpu_env, s->cfg_ptr->vlen / 8,          \
@@ -3276,6 +3280,7 @@ static bool trans_viota_m(DisasContext *s, arg_viota_m *a)
 
         data = FIELD_DP32(data, VDATA, VM, a->vm);
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);
         static gen_helper_gvec_3_ptr * const fns[4] = {
             gen_helper_viota_m_b, gen_helper_viota_m_h,
             gen_helper_viota_m_w, gen_helper_viota_m_d,
@@ -3305,6 +3310,7 @@ static bool trans_vid_v(DisasContext *s, arg_vid_v *a)
 
         data = FIELD_DP32(data, VDATA, VM, a->vm);
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);
         static gen_helper_gvec_2_ptr * const fns[4] = {
             gen_helper_vid_v_b, gen_helper_vid_v_h,
             gen_helper_vid_v_w, gen_helper_vid_v_d,
diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c
index 174a548ac2..75b59cf917 100644
--- a/target/riscv/vector_helper.c
+++ b/target/riscv/vector_helper.c
@@ -4717,6 +4717,8 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1,          \
                   uint32_t desc)                          \
 {                                                         \
     uint32_t vl = env->vl;                                \
+    uint32_t total_elems = env_archcpu(env)->cfg.vlen;    \
+    uint32_t vta_all_1s = vext_vta_all_1s(desc);          \
     uint32_t i;                                           \
     int a, b;                                             \
                                                           \
@@ -4726,6 +4728,15 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1,          \
         vext_set_elem_mask(vd, i, OP(b, a));              \
     }                                                     \
     env->vstart = 0;                                      \
+    /* mask destination register are always tail-         \
+     * agnostic                                           \
+     */                                                   \
+    /* set tail elements to 1s */                         \
+    if (vta_all_1s) {                                     \
+        for (; i < total_elems; i++) {                    \
+            vext_set_elem_mask(vd, i, 1);                 \
+        }                                                 \
+    }                                                     \
 }
 
 #define DO_NAND(N, M)  (!(N & M))
@@ -4793,6 +4804,8 @@ static void vmsetm(void *vd, void *v0, void *vs2, CPURISCVState *env,
 {
     uint32_t vm = vext_vm(desc);
     uint32_t vl = env->vl;
+    uint32_t total_elems = env_archcpu(env)->cfg.vlen;
+    uint32_t vta_all_1s = vext_vta_all_1s(desc);
     int i;
     bool first_mask_bit = false;
 
@@ -4821,6 +4834,13 @@ static void vmsetm(void *vd, void *v0, void *vs2, CPURISCVState *env,
         }
     }
     env->vstart = 0;
+    /* mask destination register are always tail-agnostic */
+    /* set tail elements to 1s */
+    if (vta_all_1s) {
+        for (; i < total_elems; i++) {
+            vext_set_elem_mask(vd, i, 1);
+        }
+    }
 }
 
 void HELPER(vmsbf_m)(void *vd, void *v0, void *vs2, CPURISCVState *env,
@@ -4848,6 +4868,9 @@ void HELPER(NAME)(void *vd, void *v0, void *vs2, CPURISCVState *env,      \
 {                                                                         \
     uint32_t vm = vext_vm(desc);                                          \
     uint32_t vl = env->vl;                                                \
+    uint32_t esz = sizeof(ETYPE);                                         \
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);          \
+    uint32_t vta = vext_vta(desc);                                        \
     uint32_t sum = 0;                                                     \
     int i;                                                                \
                                                                           \
@@ -4861,6 +4884,8 @@ void HELPER(NAME)(void *vd, void *v0, void *vs2, CPURISCVState *env,      \
         }                                                                 \
     }                                                                     \
     env->vstart = 0;                                                      \
+    /* set tail elements to 1s */                                         \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);              \
 }
 
 GEN_VEXT_VIOTA_M(viota_m_b, uint8_t,  H1)
@@ -4874,6 +4899,9 @@ void HELPER(NAME)(void *vd, void *v0, CPURISCVState *env, uint32_t desc)  \
 {                                                                         \
     uint32_t vm = vext_vm(desc);                                          \
     uint32_t vl = env->vl;                                                \
+    uint32_t esz = sizeof(ETYPE);                                         \
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);          \
+    uint32_t vta = vext_vta(desc);                                        \
     int i;                                                                \
                                                                           \
     for (i = env->vstart; i < vl; i++) {                                  \
@@ -4883,6 +4911,8 @@ void HELPER(NAME)(void *vd, void *v0, CPURISCVState *env, uint32_t desc)  \
         *((ETYPE *)vd + H(i)) = i;                                        \
     }                                                                     \
     env->vstart = 0;                                                      \
+    /* set tail elements to 1s */                                         \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);              \
 }
 
 GEN_VEXT_VID_V(vid_v_b, uint8_t,  H1)

From 803963f7cb7220be0c80d9acd87a0ebea167f35e Mon Sep 17 00:00:00 2001
From: eopXD <yueh.ting.chen@gmail.com>
Date: Mon, 6 Jun 2022 06:16:56 +0000
Subject: [PATCH 932/935] target/riscv: rvv: Add tail agnostic for vector
 permutation instructions

Signed-off-by: eop Chen <eop.chen@sifive.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <165449614532.19704.7000832880482980398-15@git.sr.ht>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/insn_trans/trans_rvv.c.inc |  7 +++--
 target/riscv/vector_helper.c            | 40 +++++++++++++++++++++++++
 2 files changed, 45 insertions(+), 2 deletions(-)

diff --git a/target/riscv/insn_trans/trans_rvv.c.inc b/target/riscv/insn_trans/trans_rvv.c.inc
index a94e634a6b..4f84d4878a 100644
--- a/target/riscv/insn_trans/trans_rvv.c.inc
+++ b/target/riscv/insn_trans/trans_rvv.c.inc
@@ -3669,7 +3669,7 @@ static bool trans_vrgather_vx(DisasContext *s, arg_rmrr *a)
         return false;
     }
 
-    if (a->vm && s->vl_eq_vlmax) {
+    if (a->vm && s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) {
         int scale = s->lmul - (s->sew + 3);
         int vlmax = s->cfg_ptr->vlen >> -scale;
         TCGv_i64 dest = tcg_temp_new_i64();
@@ -3701,7 +3701,7 @@ static bool trans_vrgather_vi(DisasContext *s, arg_rmrr *a)
         return false;
     }
 
-    if (a->vm && s->vl_eq_vlmax) {
+    if (a->vm && s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) {
         int scale = s->lmul - (s->sew + 3);
         int vlmax = s->cfg_ptr->vlen >> -scale;
         if (a->rs1 >= vlmax) {
@@ -3753,6 +3753,7 @@ static bool trans_vcompress_vm(DisasContext *s, arg_r *a)
         tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over);
 
         data = FIELD_DP32(data, VDATA, LMUL, s->lmul);
+        data = FIELD_DP32(data, VDATA, VTA, s->vta);
         tcg_gen_gvec_4_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),
                            vreg_ofs(s, a->rs1), vreg_ofs(s, a->rs2),
                            cpu_env, s->cfg_ptr->vlen / 8,
@@ -3853,6 +3854,8 @@ static bool int_ext_op(DisasContext *s, arg_rmr *a, uint8_t seq)
     }
 
     data = FIELD_DP32(data, VDATA, VM, a->vm);
+    data = FIELD_DP32(data, VDATA, LMUL, s->lmul);
+    data = FIELD_DP32(data, VDATA, VTA, s->vta);
 
     tcg_gen_gvec_3_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0),
                        vreg_ofs(s, a->rs2), cpu_env,
diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c
index 75b59cf917..a96fc49c71 100644
--- a/target/riscv/vector_helper.c
+++ b/target/riscv/vector_helper.c
@@ -4931,6 +4931,9 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong s1, void *vs2,         \
 {                                                                         \
     uint32_t vm = vext_vm(desc);                                          \
     uint32_t vl = env->vl;                                                \
+    uint32_t esz = sizeof(ETYPE);                                         \
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);          \
+    uint32_t vta = vext_vta(desc);                                        \
     target_ulong offset = s1, i_min, i;                                   \
                                                                           \
     i_min = MAX(env->vstart, offset);                                     \
@@ -4940,6 +4943,8 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong s1, void *vs2,         \
         }                                                                 \
         *((ETYPE *)vd + H(i)) = *((ETYPE *)vs2 + H(i - offset));          \
     }                                                                     \
+    /* set tail elements to 1s */                                         \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);              \
 }
 
 /* vslideup.vx vd, vs2, rs1, vm # vd[i+rs1] = vs2[i] */
@@ -4955,6 +4960,9 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong s1, void *vs2,         \
     uint32_t vlmax = vext_max_elems(desc, ctzl(sizeof(ETYPE)));           \
     uint32_t vm = vext_vm(desc);                                          \
     uint32_t vl = env->vl;                                                \
+    uint32_t esz = sizeof(ETYPE);                                         \
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);          \
+    uint32_t vta = vext_vta(desc);                                        \
     target_ulong i_max, i;                                                \
                                                                           \
     i_max = MAX(MIN(s1 < vlmax ? vlmax - s1 : 0, vl), env->vstart);       \
@@ -4971,6 +4979,8 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong s1, void *vs2,         \
     }                                                                     \
                                                                           \
     env->vstart = 0;                                                      \
+    /* set tail elements to 1s */                                         \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);              \
 }
 
 /* vslidedown.vx vd, vs2, rs1, vm # vd[i] = vs2[i+rs1] */
@@ -4986,6 +4996,9 @@ static void vslide1up_##BITWIDTH(void *vd, void *v0, target_ulong s1,       \
     typedef uint##BITWIDTH##_t ETYPE;                                       \
     uint32_t vm = vext_vm(desc);                                            \
     uint32_t vl = env->vl;                                                  \
+    uint32_t esz = sizeof(ETYPE);                                           \
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);            \
+    uint32_t vta = vext_vta(desc);                                          \
     uint32_t i;                                                             \
                                                                             \
     for (i = env->vstart; i < vl; i++) {                                    \
@@ -4999,6 +5012,8 @@ static void vslide1up_##BITWIDTH(void *vd, void *v0, target_ulong s1,       \
         }                                                                   \
     }                                                                       \
     env->vstart = 0;                                                        \
+    /* set tail elements to 1s */                                           \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);                \
 }
 
 GEN_VEXT_VSLIE1UP(8,  H1)
@@ -5026,6 +5041,9 @@ static void vslide1down_##BITWIDTH(void *vd, void *v0, target_ulong s1,       \
     typedef uint##BITWIDTH##_t ETYPE;                                         \
     uint32_t vm = vext_vm(desc);                                              \
     uint32_t vl = env->vl;                                                    \
+    uint32_t esz = sizeof(ETYPE);                                             \
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);              \
+    uint32_t vta = vext_vta(desc);                                            \
     uint32_t i;                                                               \
                                                                               \
     for (i = env->vstart; i < vl; i++) {                                      \
@@ -5039,6 +5057,8 @@ static void vslide1down_##BITWIDTH(void *vd, void *v0, target_ulong s1,       \
         }                                                                     \
     }                                                                         \
     env->vstart = 0;                                                          \
+    /* set tail elements to 1s */                                             \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);                  \
 }
 
 GEN_VEXT_VSLIDE1DOWN(8,  H1)
@@ -5092,6 +5112,9 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1, void *vs2,               \
     uint32_t vlmax = vext_max_elems(desc, ctzl(sizeof(TS2)));             \
     uint32_t vm = vext_vm(desc);                                          \
     uint32_t vl = env->vl;                                                \
+    uint32_t esz = sizeof(TS2);                                           \
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);          \
+    uint32_t vta = vext_vta(desc);                                        \
     uint64_t index;                                                       \
     uint32_t i;                                                           \
                                                                           \
@@ -5107,6 +5130,8 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1, void *vs2,               \
         }                                                                 \
     }                                                                     \
     env->vstart = 0;                                                      \
+    /* set tail elements to 1s */                                         \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);              \
 }
 
 /* vd[i] = (vs1[i] >= VLMAX) ? 0 : vs2[vs1[i]]; */
@@ -5127,6 +5152,9 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong s1, void *vs2,         \
     uint32_t vlmax = vext_max_elems(desc, ctzl(sizeof(ETYPE)));           \
     uint32_t vm = vext_vm(desc);                                          \
     uint32_t vl = env->vl;                                                \
+    uint32_t esz = sizeof(ETYPE);                                         \
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);          \
+    uint32_t vta = vext_vta(desc);                                        \
     uint64_t index = s1;                                                  \
     uint32_t i;                                                           \
                                                                           \
@@ -5141,6 +5169,8 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong s1, void *vs2,         \
         }                                                                 \
     }                                                                     \
     env->vstart = 0;                                                      \
+    /* set tail elements to 1s */                                         \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);              \
 }
 
 /* vd[i] = (x[rs1] >= VLMAX) ? 0 : vs2[rs1] */
@@ -5155,6 +5185,9 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1, void *vs2,               \
                   CPURISCVState *env, uint32_t desc)                      \
 {                                                                         \
     uint32_t vl = env->vl;                                                \
+    uint32_t esz = sizeof(ETYPE);                                         \
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz);          \
+    uint32_t vta = vext_vta(desc);                                        \
     uint32_t num = 0, i;                                                  \
                                                                           \
     for (i = env->vstart; i < vl; i++) {                                  \
@@ -5165,6 +5198,8 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1, void *vs2,               \
         num++;                                                            \
     }                                                                     \
     env->vstart = 0;                                                      \
+    /* set tail elements to 1s */                                         \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);              \
 }
 
 /* Compress into vd elements of vs2 where vs1 is enabled */
@@ -5196,6 +5231,9 @@ void HELPER(NAME)(void *vd, void *v0, void *vs2,                 \
 {                                                                \
     uint32_t vl = env->vl;                                       \
     uint32_t vm = vext_vm(desc);                                 \
+    uint32_t esz = sizeof(ETYPE);                                \
+    uint32_t total_elems = vext_get_total_elems(env, desc, esz); \
+    uint32_t vta = vext_vta(desc);                               \
     uint32_t i;                                                  \
                                                                  \
     for (i = env->vstart; i < vl; i++) {                         \
@@ -5205,6 +5243,8 @@ void HELPER(NAME)(void *vd, void *v0, void *vs2,                 \
         *((ETYPE *)vd + HD(i)) = *((DTYPE *)vs2 + HS1(i));       \
     }                                                            \
     env->vstart = 0;                                             \
+    /* set tail elements to 1s */                                \
+    vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz);     \
 }
 
 GEN_VEXT_INT_EXT(vzext_vf2_h, uint16_t, uint8_t,  H2, H1)

From b8312675d62b878d6647065f01c2c1337a74d4ee Mon Sep 17 00:00:00 2001
From: eopXD <eop.chen@sifive.com>
Date: Mon, 6 Jun 2022 06:16:56 +0000
Subject: [PATCH 933/935] target/riscv: rvv: Add option 'rvv_ta_all_1s' to
 enable optional tail agnostic behavior

According to v-spec, tail agnostic behavior can be either kept as
undisturbed or set elements' bits to all 1s. To distinguish the
difference of tail policies, QEMU should be able to simulate the tail
agnostic behavior as "set tail elements' bits to all 1s".

There are multiple possibility for agnostic elements according to
v-spec. The main intent of this patch-set tries to add option that
can distinguish between tail policies. Setting agnostic elements to
all 1s allows QEMU to express this.

This commit adds option 'rvv_ta_all_1s' is added to enable the
behavior, it is default as disabled.

Signed-off-by: eop Chen <eop.chen@sifive.com>
Reviewed-by: Frank Chang <frank.chang@sifive.com>
Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <165449614532.19704.7000832880482980398-16@git.sr.ht>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index 0497af45cc..e5aa1e9c1b 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -918,6 +918,8 @@ static Property riscv_cpu_properties[] = {
     DEFINE_PROP_UINT64("resetvec", RISCVCPU, cfg.resetvec, DEFAULT_RSTVEC),
 
     DEFINE_PROP_BOOL("short-isa-string", RISCVCPU, cfg.short_isa_string, false),
+
+    DEFINE_PROP_BOOL("rvv_ta_all_1s", RISCVCPU, cfg.rvv_ta_all_1s, false),
     DEFINE_PROP_END_OF_LIST(),
 };
 

From 26b2bc58599c02b35e55afbd1bd050faa3d187c2 Mon Sep 17 00:00:00 2001
From: Alistair Francis <alistair.francis@wdc.com>
Date: Wed, 8 Jun 2022 16:14:37 +1000
Subject: [PATCH 934/935] target/riscv: Don't expose the CPU properties on
 names CPUs

There are currently two types of RISC-V CPUs:
 - Generic CPUs (base or any) that allow complete custimisation
 - "Named" CPUs that match existing hardware

Users can use the base CPUs to custimise the extensions that they want, for
example -cpu rv64,v=true.

We originally exposed these as part of the named CPUs as well, but that was
by accident.

Exposing the CPU properties to named CPUs means that we accidently
enable extensions that don't exist on the CPUs by default. For example
the SiFive E CPU currently support the zba extension, which is a bug.

This patch instead only exposes the CPU extensions to the generic CPUs.

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Message-Id: <20220608061437.314434-1-alistair.francis@opensource.wdc.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.c | 57 +++++++++++++++++++++++++++++++++++++---------
 1 file changed, 46 insertions(+), 11 deletions(-)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index e5aa1e9c1b..05e6521351 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -118,6 +118,8 @@ static const char * const riscv_intr_names[] = {
     "reserved"
 };
 
+static void register_cpu_props(DeviceState *dev);
+
 const char *riscv_cpu_get_trap_name(target_ulong cause, bool async)
 {
     if (async) {
@@ -161,6 +163,7 @@ static void riscv_any_cpu_init(Object *obj)
     set_misa(env, MXL_RV64, RVI | RVM | RVA | RVF | RVD | RVC | RVU);
 #endif
     set_priv_version(env, PRIV_VERSION_1_12_0);
+    register_cpu_props(DEVICE(obj));
 }
 
 #if defined(TARGET_RISCV64)
@@ -169,6 +172,7 @@ static void rv64_base_cpu_init(Object *obj)
     CPURISCVState *env = &RISCV_CPU(obj)->env;
     /* We set this in the realise function */
     set_misa(env, MXL_RV64, 0);
+    register_cpu_props(DEVICE(obj));
 }
 
 static void rv64_sifive_u_cpu_init(Object *obj)
@@ -181,9 +185,11 @@ static void rv64_sifive_u_cpu_init(Object *obj)
 static void rv64_sifive_e_cpu_init(Object *obj)
 {
     CPURISCVState *env = &RISCV_CPU(obj)->env;
+    RISCVCPU *cpu = RISCV_CPU(obj);
+
     set_misa(env, MXL_RV64, RVI | RVM | RVA | RVC | RVU);
     set_priv_version(env, PRIV_VERSION_1_10_0);
-    qdev_prop_set_bit(DEVICE(obj), "mmu", false);
+    cpu->cfg.mmu = false;
 }
 
 static void rv128_base_cpu_init(Object *obj)
@@ -197,6 +203,7 @@ static void rv128_base_cpu_init(Object *obj)
     CPURISCVState *env = &RISCV_CPU(obj)->env;
     /* We set this in the realise function */
     set_misa(env, MXL_RV128, 0);
+    register_cpu_props(DEVICE(obj));
 }
 #else
 static void rv32_base_cpu_init(Object *obj)
@@ -204,6 +211,7 @@ static void rv32_base_cpu_init(Object *obj)
     CPURISCVState *env = &RISCV_CPU(obj)->env;
     /* We set this in the realise function */
     set_misa(env, MXL_RV32, 0);
+    register_cpu_props(DEVICE(obj));
 }
 
 static void rv32_sifive_u_cpu_init(Object *obj)
@@ -216,27 +224,33 @@ static void rv32_sifive_u_cpu_init(Object *obj)
 static void rv32_sifive_e_cpu_init(Object *obj)
 {
     CPURISCVState *env = &RISCV_CPU(obj)->env;
+    RISCVCPU *cpu = RISCV_CPU(obj);
+
     set_misa(env, MXL_RV32, RVI | RVM | RVA | RVC | RVU);
     set_priv_version(env, PRIV_VERSION_1_10_0);
-    qdev_prop_set_bit(DEVICE(obj), "mmu", false);
+    cpu->cfg.mmu = false;
 }
 
 static void rv32_ibex_cpu_init(Object *obj)
 {
     CPURISCVState *env = &RISCV_CPU(obj)->env;
+    RISCVCPU *cpu = RISCV_CPU(obj);
+
     set_misa(env, MXL_RV32, RVI | RVM | RVC | RVU);
     set_priv_version(env, PRIV_VERSION_1_10_0);
-    qdev_prop_set_bit(DEVICE(obj), "mmu", false);
-    qdev_prop_set_bit(DEVICE(obj), "x-epmp", true);
+    cpu->cfg.mmu = false;
+    cpu->cfg.epmp = true;
 }
 
 static void rv32_imafcu_nommu_cpu_init(Object *obj)
 {
     CPURISCVState *env = &RISCV_CPU(obj)->env;
+    RISCVCPU *cpu = RISCV_CPU(obj);
+
     set_misa(env, MXL_RV32, RVI | RVM | RVA | RVF | RVC | RVU);
     set_priv_version(env, PRIV_VERSION_1_10_0);
     set_resetvec(env, DEFAULT_RSTVEC);
-    qdev_prop_set_bit(DEVICE(obj), "mmu", false);
+    cpu->cfg.mmu = false;
 }
 #endif
 
@@ -249,6 +263,7 @@ static void riscv_host_cpu_init(Object *obj)
 #elif defined(TARGET_RISCV64)
     set_misa(env, MXL_RV64, 0);
 #endif
+    register_cpu_props(DEVICE(obj));
 }
 #endif
 
@@ -836,6 +851,12 @@ static void riscv_cpu_init(Object *obj)
 {
     RISCVCPU *cpu = RISCV_CPU(obj);
 
+    cpu->cfg.ext_counters = true;
+    cpu->cfg.ext_ifencei = true;
+    cpu->cfg.ext_icsr = true;
+    cpu->cfg.mmu = true;
+    cpu->cfg.pmp = true;
+
     cpu_set_cpustate_pointers(cpu);
 
 #ifndef CONFIG_USER_ONLY
@@ -844,7 +865,7 @@ static void riscv_cpu_init(Object *obj)
 #endif /* CONFIG_USER_ONLY */
 }
 
-static Property riscv_cpu_properties[] = {
+static Property riscv_cpu_extensions[] = {
     /* Defaults for standard extensions */
     DEFINE_PROP_BOOL("i", RISCVCPU, cfg.ext_i, true),
     DEFINE_PROP_BOOL("e", RISCVCPU, cfg.ext_e, false),
@@ -867,17 +888,12 @@ static Property riscv_cpu_properties[] = {
     DEFINE_PROP_BOOL("Zve64f", RISCVCPU, cfg.ext_zve64f, false),
     DEFINE_PROP_BOOL("mmu", RISCVCPU, cfg.mmu, true),
     DEFINE_PROP_BOOL("pmp", RISCVCPU, cfg.pmp, true),
-    DEFINE_PROP_BOOL("debug", RISCVCPU, cfg.debug, true),
 
     DEFINE_PROP_STRING("priv_spec", RISCVCPU, cfg.priv_spec),
     DEFINE_PROP_STRING("vext_spec", RISCVCPU, cfg.vext_spec),
     DEFINE_PROP_UINT16("vlen", RISCVCPU, cfg.vlen, 128),
     DEFINE_PROP_UINT16("elen", RISCVCPU, cfg.elen, 64),
 
-    DEFINE_PROP_UINT32("mvendorid", RISCVCPU, cfg.mvendorid, 0),
-    DEFINE_PROP_UINT64("marchid", RISCVCPU, cfg.marchid, RISCV_CPU_MARCHID),
-    DEFINE_PROP_UINT64("mimpid", RISCVCPU, cfg.mimpid, RISCV_CPU_MIMPID),
-
     DEFINE_PROP_BOOL("svinval", RISCVCPU, cfg.ext_svinval, false),
     DEFINE_PROP_BOOL("svnapot", RISCVCPU, cfg.ext_svnapot, false),
     DEFINE_PROP_BOOL("svpbmt", RISCVCPU, cfg.ext_svpbmt, false),
@@ -915,6 +931,25 @@ static Property riscv_cpu_properties[] = {
     DEFINE_PROP_BOOL("x-epmp", RISCVCPU, cfg.epmp, false),
     DEFINE_PROP_BOOL("x-aia", RISCVCPU, cfg.aia, false),
 
+    DEFINE_PROP_END_OF_LIST(),
+};
+
+static void register_cpu_props(DeviceState *dev)
+{
+    Property *prop;
+
+    for (prop = riscv_cpu_extensions; prop && prop->name; prop++) {
+        qdev_property_add_static(dev, prop);
+    }
+}
+
+static Property riscv_cpu_properties[] = {
+    DEFINE_PROP_BOOL("debug", RISCVCPU, cfg.debug, true),
+
+    DEFINE_PROP_UINT32("mvendorid", RISCVCPU, cfg.mvendorid, 0),
+    DEFINE_PROP_UINT64("marchid", RISCVCPU, cfg.marchid, RISCV_CPU_MARCHID),
+    DEFINE_PROP_UINT64("mimpid", RISCVCPU, cfg.mimpid, RISCV_CPU_MIMPID),
+
     DEFINE_PROP_UINT64("resetvec", RISCVCPU, cfg.resetvec, DEFAULT_RSTVEC),
 
     DEFINE_PROP_BOOL("short-isa-string", RISCVCPU, cfg.short_isa_string, false),

From 07314158f6aa4d2589520c194a7531b9364a8d54 Mon Sep 17 00:00:00 2001
From: Alistair Francis <alistair.francis@wdc.com>
Date: Thu, 9 Jun 2022 09:47:01 +1000
Subject: [PATCH 935/935] target/riscv: trans_rvv: Avoid assert for RV32 and
 e64

When running a 32-bit guest, with a e64 vmv.v.x and vl_eq_vlmax set to
true the `tcg_debug_assert(vece <= MO_32)` will be triggered inside
tcg_gen_gvec_dup_i32().

This patch checks that condition and instead uses tcg_gen_gvec_dup_i64()
is required.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1028
Suggested-by: Robert Bu <robert.bu@gmail.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220608234701.369536-1-alistair.francis@opensource.wdc.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/insn_trans/trans_rvv.c.inc | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/target/riscv/insn_trans/trans_rvv.c.inc b/target/riscv/insn_trans/trans_rvv.c.inc
index 4f84d4878a..6c091824b6 100644
--- a/target/riscv/insn_trans/trans_rvv.c.inc
+++ b/target/riscv/insn_trans/trans_rvv.c.inc
@@ -2128,8 +2128,16 @@ static bool trans_vmv_v_x(DisasContext *s, arg_vmv_v_x *a)
         s1 = get_gpr(s, a->rs1, EXT_SIGN);
 
         if (s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) {
-            tcg_gen_gvec_dup_tl(s->sew, vreg_ofs(s, a->rd),
-                                MAXSZ(s), MAXSZ(s), s1);
+            if (get_xl(s) == MXL_RV32 && s->sew == MO_64) {
+                TCGv_i64 s1_i64 = tcg_temp_new_i64();
+                tcg_gen_ext_tl_i64(s1_i64, s1);
+                tcg_gen_gvec_dup_i64(s->sew, vreg_ofs(s, a->rd),
+                                     MAXSZ(s), MAXSZ(s), s1_i64);
+                tcg_temp_free_i64(s1_i64);
+            } else {
+                tcg_gen_gvec_dup_tl(s->sew, vreg_ofs(s, a->rd),
+                                    MAXSZ(s), MAXSZ(s), s1);
+            }
         } else {
             TCGv_i32 desc;
             TCGv_i64 s1_i64 = tcg_temp_new_i64();